Chapter 6: Inner product spaces
Problems for Chapter 6
These problems combine the sections of Chapter 6 with each other and with earlier chapters: linear systems, dimension, affine subspaces, determinants, and traces. The first three are review prompts: state a main result, prove it without looking, and then answer a question that the text did not ask. The rest are ordered roughly by difficulty. Problems of kind “competition” are harder than anything the course expects, and two problems that go beyond the course are marked. Section 6.7 compares the techniques these problems call for.
Review: state and prove
- (a)State the Cauchy–Schwarz inequality, including its equality case.
- (b)Prove it, by any of the three methods of Section 6.2.1, without looking.
(c)Let be nonzero vectors in an inner product space. Show that
and that equality holds if and only if or is a negative real multiple of .
Uses. Theorem 6.2.1 (Cauchy–Schwarz inequality), Proposition 6.1.29 (Expanding a norm), Lemma 6.2.3 (The easy half of (b)), Theorem 6.2.11 (Triangle inequality)
Techniques. Orthogonal decomposition (Technique 6.2.4), A nonnegative quadratic has nonpositive discriminant (Technique 6.2.7)
(a) For all vectors in an inner product space,
(b) If , both sides are and . Let , , and . Then
Dropping and multiplying by gives the inequality. Equality holds exactly when , that is, ; the converse is Lemma 6.2.3.
(c) Let , , and . By Proposition 6.1.29 with the real scalars and ,
and
Since
By Cauchy–Schwarz,
Equality holds if and only if
The inequality says that sending nonzero vectors to the unit sphere along rays moves them apart by at most times their distance. Its proof used only
Problem 6.2. Let be a finite-dimensional subspace of an inner product space .
- (a)State the orthogonal decomposition theorem and the best-approximation theorem for .
- (b)Prove both, using only that has an orthonormal basis (Theorem 6.4.20) and the formula
. - (c)On , let
. Find all points of closest to for this norm. Which step of your proof in (b) has no analogue for ?
Uses. Theorem 6.4.24 (Orthogonal decomposition), Theorem 6.4.33 (Best approximation), Proposition 6.3.9 (Properties of ), Theorem 6.3.3 (Pythagorean theorem), Proposition 6.1.30 (Parallelogram law), Aside (Norms that do not come from inner products) (Norms that do not come from inner products)
Techniques. Projections shrink norms (Technique 6.4.5), Recognize a projection by its two conditions (Technique 6.4.7)
(a) Orthogonal decomposition (Theorem 6.4.24): every can be written in exactly one way as with and , namely with . Best approximation (Theorem 6.4.33):
(b) Existence. Let
A vector orthogonal to each is orthogonal to their span (Proposition 6.3.9(c)), so , and is a decomposition.
Uniqueness. If with and , then
Best approximation. Let . Then
with equality if and only if , that is, .
(c) The points of are , and
The step with no analogue is the Pythagorean identity, which made the closest point unique: it adds the positive number for every . The norm does not come from any inner product, since it violates the parallelogram law (Proposition 6.1.30):
- (a)State the theorem on the kernels and images of and (Section 6.6.4).
- (b)Let have an adjoint. Prove that , and, if is finite-dimensional, that .
(c)(Fredholm alternative.) Let and . Show that exactly one of the following holds: has a solution ; or there is with and . Use this to decide whether the system
has a solution, and likewise with right-hand side .
Uses. Theorem 6.6.21 (Kernels and images of adjoints), Definition 6.6.8 (Adjoint), Proposition 6.1.37 (Inner products distinguish vectors), Proposition 6.6.18 (Properties of the adjoint), Theorem 6.4.26 (The double complement), Example 6.6.10 (The adjoint of a matrix), Corollary 6.6.22 ( and have the same rank), Theorem 4.2.11 (Rank–nullity)
Techniques. Move an operator across the inner product (Technique 6.6.20), Show a vector is zero via its norm (Technique 6.1.36)
(a) Let have an adjoint. Then and . If is finite-dimensional, also and , so that and (Theorem 6.6.21).
(b) Let . A vector of is if and only if it is orthogonal to every vector of (Proposition 6.1.37). So
using Definition 6.6.8 in the middle. This proves .
By Proposition 6.6.18, has the adjoint . Applying what we just proved to gives . If is finite-dimensional, so is its subspace , and Theorem 6.4.26 gives
(c) With dot products, the map from to has the adjoint (Example 6.6.10). Applying (b) to the adjoint, with the finite-dimensional space in the role of , gives . So has a solution if and only if for every . The second alternative says precisely that this fails for some , so exactly one of the two holds.
For the example, is real, so . Subtracting the second equation from the first gives , so satisfies , as one checks column by column. With , , so the system has no solution. With , . The first two rows of are independent and the third is their difference, so , (Corollary 6.6.22), and by Theorem 4.2.11. So , and the system is solvable; indeed works.
The vector is a certificate of inconsistency that can be checked without solving anything. Row reduction finds it too (as the combination of rows that produces ), but (b) explains why it must exist: the right-hand side has to be orthogonal to every relation among the rows.
Synthesis problems
Problem 6.4. Let be such that () are linearly independent, and let
- (a)Show that exactly one point is equidistant from . (It is the circumcenter of the points.)
- (b)Show that is equidistant from if and only if for every . Deduce that is the point of closest to every such , and that for the circumcenter is the only point of equidistant from the points.
- (c)Find the circumcenter and the common distance for , , and .
Uses. Proposition 6.1.29 (Expanding a norm), Definition 6.3.28 (Gram matrix), Proposition 6.3.30 (Gram matrices detect independence), Theorem 6.4.6 (The projection is characterized by orthogonality), Theorem 6.4.33 (Best approximation)
Techniques. Gram matrices (Technique 6.3.31), Recognize a projection by its two conditions (Technique 6.4.7)
(a) Every point of is with
So is equidistant from all the points if and only if
Write . Then
(b) The computation in (a) did not use : for every , is equidistant from the points if and only if satisfies these conditions. Since satisfies it, subtracting shows that is equidistant if and only if for all .
For such , the vector lies in , and
with equality only for . If , then , so and .
(c) Here and , so
So
By (b), the points of equidistant from the three vertices form the line through perpendicular to the plane of the triangle; the circumcenter is where that line meets the plane.
Problem 6.5. Let be distinct, let , and let
- (a)Show that this is an inner product on , and that the Lagrange polynomials of the nodes (Definition 3.4.26) form an orthonormal basis, in which the coordinates of are
. - (b)Deduce that the matrix with entry () is invertible. (Compare Problem 5.9.)
- (c)Let , let satisfy for all (Theorem 3.4.24), and let . Show that is the least-squares polynomial of degree at most for the data , in the sense of Proposition 6.5.21.
- (d)For the nodes and the values , find and the least-squares line by (c).
Uses. Definition 6.1.2 (Inner product), Lemma 3.4.23 (Too many roots), Proposition 6.3.18 (Coordinates in an orthonormal basis), Corollary 6.3.14 (Counting orthogonal vectors), Proposition 6.3.30 (Gram matrices detect independence), Theorem 6.4.33 (Best approximation), Proposition 6.5.21 (Polynomial fits)
Techniques. Take an inner product to isolate a coefficient (Technique 6.3.12), A sum of nonnegative terms (Technique 6.1.9)
(a) The formula is linear in and symmetric, since evaluation at a point is linear. Also
Since ,
So is orthonormal, and since it is an orthonormal basis (Corollary 6.3.14). By Proposition 6.3.18 the coordinates of are
(b) The entry is
(c) For every ,
the sum of squared residuals of . By Theorem 6.4.33, this is smallest exactly for , so that projection is the least-squares polynomial of Proposition 6.5.21, and in particular it is unique.
(d)
This agrees with Proposition 6.5.17: the mean of the nodes is , the mean of the values is , and the slope is
Problem 6.6. Find the error in the following argument, and show that the claim is false.
Claim. Let be a finite-dimensional real inner product space and with for every . Then is self-adjoint.
Proof. Let . For every ,
the last step because is real. So for every , and by Proposition 6.1.41.
- (a)Then show that the same argument is correct for complex inner product spaces.
- (b)Over , show that
for every , where .
Uses. Proposition 6.1.41 (Over , the values determine ), Example 6.1.39 (A rotation), Proposition 6.6.18 (Properties of the adjoint), Proposition 6.6.30 (Self-adjoint means real values)
Techniques. Finding counterexamples and minimal examples (Technique 1.4.18), Plug in clever test vectors (Technique 6.1.40)
The error. The last step cites Proposition 6.1.41, which is about complex inner product spaces. Over , satisfies (Proposition 6.6.18), and every such operator has
The claim is false. Let on with the dot product. Then
but
(a) Over every step is correct. The conjugation step is the definition of the adjoint followed by conjugate symmetry, and the hypothesis is used exactly where it should be: a nonnegative number equals its conjugate. Then for every , and Proposition 6.1.41 gives . (This is one half of Proposition 6.6.30.)
(b) Over ,
The numbers see only the self-adjoint part of a real operator, and adding any with to does not change them. This is why Section 8.3 requires positive operators to be self-adjoint by definition.
Problem 6.7. Prove or disprove each statement.
- (a)If has orthonormal columns, then .
- (b)For every , is a nonnegative real number.
- (c)If satisfies and , then is the matrix of an orthogonal projection.
- (d)If are subspaces of a finite-dimensional inner product space , then
. - (e)For : for every if and only if .
Uses. Theorem 6.4.31 (The matrix of an orthogonal projection), Theorem 5.4.10 (Multiplicativity), Exercise 6.3.57, Lemma 6.5.4 (The orthogonal complement of a column space), Theorem 6.4.6 (The projection is characterized by orthogonality), Proposition 6.4.29 (The orthogonal projection as a linear map), Theorem 4.2.11 (Rank–nullity), Exercise 6.4.59
Techniques. Finding counterexamples and minimal examples (Technique 1.4.18), Move an operator across the inner product (Technique 6.6.20)
(a) False. has one orthonormal column, but . In general is the matrix of (Theorem 6.4.31), which is only when , that is, when .
(b) True. By Theorem 5.4.10 and Exercise 6.3.57(a),
(c) True. Let . Using the dot product,
(d) True. Let be the restriction of to ; it maps into and is linear. Its kernel is (Proposition 6.4.29(a)), and its image is a subspace of . By Theorem 4.2.11,
In words: if is bigger than , then contains independent directions perpendicular to all of .
(e) “Only if” is true. The columns of are , with . By Hadamard’s inequality (Exercise 6.4.59(c)),
Problem 6.8. Let be a real inner product space and a function (not assumed linear) with and
- (a)Show that and
for all . - (b)Show that is linear. Give a proof that works for every , and, if is finite-dimensional, a second proof with an orthonormal basis.
- (c)Deduce that every distance-preserving has the form with .
- (d)Now let be a complex inner product space and satisfy the same two conditions. Show that
and for real , but that need not be linear over .
Uses. Proposition 6.1.29 (Expanding a norm), Proposition 6.1.27 (Basic properties of the norm), Corollary 6.3.14 (Counting orthogonal vectors), Proposition 6.3.18 (Coordinates in an orthonormal basis), Proposition 2.3.17 (Every linear map is some ), Theorem 6.3.29 (Orthonormal columns)
Techniques. Polarization (Technique 6.1.33), Show a vector is zero via its norm (Technique 6.1.36)
(a) Taking gives
and the squared lengths agree, so
(b) First proof. Let and
By (a), each term equals the same term with removed, and the result is the expansion of
So is additive and homogeneous, that is, linear. This proof works verbatim for between any two real inner product spaces.
Second proof, for . Let be an orthonormal basis. By (a),
and each is linear. This proof needs the values to span the target, which is why it asks for with finite-dimensional; the first proof needs nothing of the kind.
(c) Let
(d) Over , Proposition 6.1.29 contains instead of , so (a) becomes
For surjective distance-preserving maps between real normed spaces, the conclusion of (b) still holds (the Mazur–Ulam theorem), but the proof is much harder: without an inner product there is no polarization.
Problem 6.9. Let be inner product spaces over , and let and have the same Gram matrix:
- (a)Show that
for all . Deduce that if and only if . - (b)Show that there is exactly one linear map
with for all , and that is bijective with for all . - (c)Let . Show that if and only if for some unitary .
Uses. Proposition 6.3.30 (Gram matrices detect independence), Proposition 6.1.27 (Basic properties of the norm), Corollary 2.2.6 (Agreeing on a spanning list), Theorem 6.3.29 (Orthonormal columns), Theorem 6.4.20 (Orthonormal bases exist), Corollary 6.4.25 (Dimension of the orthogonal complement), Corollary 6.3.14 (Counting orthogonal vectors), Definition 6.3.33 (Unitary and orthogonal matrices), Exercise 6.3.37
Techniques. Gram matrices (Technique 6.3.31), Show a vector is zero via its norm (Technique 6.1.36)
(a) Let be the common Gram matrix and . By Proposition 6.3.30(a),
(b) Uniqueness. A linear map on is determined by its values on the spanning list (Corollary 2.2.6).
Existence. Every vector of is for some scalars, and we set
preserves inner products. For and ,
is bijective. If , then , so is injective. Its image contains every , hence their span, so is surjective.
(c) If with , then
Conversely, suppose . By Theorem 6.3.29, is the Gram matrix of the columns of , and that of the columns of . By (b) there is a bijective linear with that preserves inner products. In particular
Choose an orthonormal basis of (Theorem 6.4.20). Then is orthonormal in , hence an orthonormal basis of it (Corollary 6.3.14). By Corollary 6.4.25, and both have dimension ; choose orthonormal bases and of them. Vectors of are orthogonal to vectors of , so is an orthonormal list of length , hence an orthonormal basis of , and likewise for
Over , (c) says that two lists of vectors in with the same lengths and the same pairwise angles differ by an orthogonal matrix. The unitary is not unique when : it can be changed freely on .
Problem 6.10. Let be a finite-dimensional inner product space, and let be subspaces with . Give and the restricted inner products, and define and by and .
- (a)Show that
for all and . Conclude that . - (b)Show that the following are equivalent:
- (i);
- (ii);
- (iii) is bijective;
- (iv) is bijective.
- (c)Show by an example that (i) and (ii) need not be equivalent when .
Uses. Proposition 6.4.29 (The orthogonal projection as a linear map), Proposition 6.6.9 (Adjoints are unique and linear), Definition 6.6.8 (Adjoint), Corollary 6.6.22 ( and have the same rank), Corollary 4.2.18 (Injective if and only if surjective)
Techniques. Move an operator across the inner product (Technique 6.6.20), Dimension counting (Technique 4.2.24)
(a) By Proposition 6.4.29(c) and (b),
(b) Let . By Proposition 6.4.29(a), , so ; likewise . So (i) says that is injective, and (ii) that is injective. Since maps between spaces of the same dimension , it is injective if and only if it is bijective (Corollary 4.2.18), and the same holds for . So (i) (iii) and (ii) (iv). Finally, by (a) and Corollary 6.6.22,
(c) In , let and . Then , so , but
Condition (i) says that no nonzero vector of is perpendicular to all of . Part (b) says that for subspaces of equal dimension this relation is symmetric, which is not obvious from the definition; the proof gets the symmetry from . For two lines in , both conditions say that the lines are not perpendicular. In orthonormal bases of and , the matrix of has entries , and its singular values (Section 9.1) are the cosines of the principal angles between and .
Problem 6.11. Give the Frobenius inner product (Proposition 6.1.16). Let be the span of all commutators with , and let
- (a)Show that and that .
- (b)Show that
for all . Deduce that every commutes with every matrix. - (c)Deduce that : every matrix of trace is a sum of commutators.
- (d)Give a second proof of , valid over every field , by writing each matrix in the spanning list of found in Example 2.6.28 as a commutator of matrix units.
Uses. Proposition 2.4.28 (Cyclicity of the trace), Proposition 6.3.9 (Properties of ), Proposition 6.3.27 (Properties of the conjugate transpose), Proposition 6.1.37 (Inner products distinguish vectors), Exercise 2.4.43, Theorem 6.4.26 (The double complement), Lemma 2.4.24 (Matrix units), Example 2.6.28 (The trace)
Techniques. Cyclicity of trace (Technique 2.4.29), Show a vector is zero via its norm (Technique 6.1.36)
(a) By Proposition 2.4.28, , so every commutator lies in the subspace , and so does their span . For every ,
(b) By Proposition 2.4.28,
Let . By Proposition 6.3.27, , so the right side is , as claimed.
Now let and fix . Then for every , and gives (Proposition 6.1.37), that is, . As runs over so does , since . So commutes with every matrix.
(c) By (b) and Exercise 2.4.43 (a matrix that commutes with every upper triangular matrix is a scalar multiple of ; for every matrix is), . Conversely , since
(d) Write , so that
and for ,
By Example 2.6.28, these matrices span , so , and holds over every field by Proposition 2.4.28.
The second proof is shorter and works over every field. The first illustrates a pattern: to show that a subspace is all of a given subspace, compute , and test a vector of against everything. It needs an inner product, so it works only over and . (In fact every matrix of trace over or is a single commutator, a theorem of Shoda that is much harder.)
Problem 6.12. Let .
- (a)Show that is the Gram matrix of the columns of the matrix that has on top of .
- (b)Show that , with equality if and only if .
Solution (by Gram–Schmidt).
Uses. Theorem 2.4.34 (Block multiplication), Theorem 6.3.29 (Orthonormal columns), Exercise 6.4.59, Theorem 6.4.15 (Gram–Schmidt is correct)
Techniques. Gram matrices (Technique 6.3.31), A sum of nonnegative terms (Technique 6.1.9)
(a) , so by Theorem 2.4.34
(b) Write , where is column of . Coordinate of is for , so the columns are independent. Run Gram–Schmidt on them, with residuals . By Exercise 6.4.59(b),
The residual is minus its projection onto , so
If equality holds, then every , so has no coordinate other than the th, and . By Theorem 6.4.15(a),
Solution (by the Cauchy–Binet formula).
Uses. Exercise 5.5.52, Exercise 6.3.57, Corollary 5.4.15 (Rows), Proposition 5.3.15 (Triangular matrices)
Techniques. Gram matrices (Technique 6.3.31), A sum of nonnegative terms (Technique 6.1.9)
For an -element set , let be the matrix formed by the rows of with indices in . The columns of with indices in form , so by Exercise 5.5.52 and Exercise 6.3.57(a),
The term is , and the others are nonnegative, so . For equality, every other minor must vanish. Take
The first approach is the geometric one: each column sticks out of the span of the previous ones by at least , because of its identity part. The second expands the determinant into squares, one of which is the coming from . Either way, is invertible for every , which Exercise 5.6.12(c) proved for real through the kernel; in terms of the singular values of Section 9.1,
Uses. Corollary 6.4.25 (Dimension of the orthogonal complement), Proposition 6.1.29 (Expanding a norm), Exercise 6.4.62
Techniques. Orthogonal decomposition (Technique 6.2.4), Extremal choice and minimal counterexamples (Technique 1.5.9)
We induct on .
. Then for any nonzero , and with , so
. We may assume . Let , and for let
the part of orthogonal to (Technique 6.2.4). For , both less than ,
since . If , then with (it is , and nonzero because ). If this happened for two indices , then
Example. In , the vectors have pairwise inner products or .
With strict inequalities , the bound drops to (Exercise 6.4.62). The difference is visible in the proof: with strict inequalities, the projected vectors still have strictly negative inner products, and none of them can vanish once .
Problem 6.14. (The kernel trick.) Let have rows (the feature vectors of data points), let , and let , the matrix with entries .
- (a)Show that the least-squares solution of minimal norm of (Proposition 6.5.27) lies in .
(b)Show that for , is a least-squares solution of if and only if is a least-squares solution of . Deduce that for every least-squares solution of , and that for any ,
- (c)Data points have values , and each point has the feature vector
. Using only the kernel , compute , , and the fitted function .
Uses. Proposition 6.5.27 (The least-squares solution of minimal norm), Theorem 6.6.21 (Kernels and images of adjoints), Example 6.6.10 (The adjoint of a matrix), Theorem 6.5.5 (The normal equations), Proposition 6.5.9 (
Techniques. Move an operator across the inner product (Technique 6.6.20)
(a) By Proposition 6.5.27, is the only least-squares solution in . With dot products, is the adjoint of (Example 6.6.10), so by Theorem 6.6.21(c).
(b) By Theorem 6.5.5, is a least-squares solution of if and only if , that is, . Since , is a least-squares solution of if and only if , that is, . By Proposition 6.5.9(a) applied to ,
If is a least-squares solution of , then is a least-squares solution of that lies in , so it is by the uniqueness in Proposition 6.5.27. Since , we get
(c)
is invertible, so the least-squares solution of is
which takes the values at . Explicitly,
The point is that never appeared in the computation. The fit needs the matrix and the values only, so the feature space can be huge, or even infinite-dimensional, as long as its inner products are cheap to compute. Section 16.1 develops this.
Problem 6.15. For a continuous , let
Find the largest value of over all such , and the functions attaining it.
Uses. Proposition 6.1.13 (Integral inner products), Lemma 6.1.14 (Nonnegative functions with zero integral), Proposition 6.1.29 (Expanding a norm)
Techniques. Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21), A sum of nonnegative terms (Technique 6.1.9)
Let
With ,
so
with equality if and only if , that is, . The largest value is .
Pointwise, the identity is
This is Problem B5 of the 2006 Putnam Competition.
Problem 6.16. Let be positive real numbers with . Show that
and find when equality holds.
Uses. Corollary 6.2.26 (Engel’s form), Theorem 6.2.1 (Cauchy–Schwarz inequality)
Techniques. Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21), A sum of nonnegative terms (Technique 6.1.9)
Let , , , positive with . Then
and similarly for the other terms. By Corollary 6.2.26, with numerators and denominators , , ,
That is Cauchy–Schwarz for the vectors and , chosen so that their inner product is (Technique 6.2.21).
It remains to show . Let with , , , so that . Multiplying out,
which is , with equality only if (Technique 6.1.9). So
If equality holds, then , which forces , hence and . Conversely, at every term is .
This is Problem 2 of the 1995 International Mathematical Olympiad. The substitution is the real idea; after it, the problem is the same Engel-form estimate as Nesbitt’s inequality (Exercise 6.2.47).
Problem 6.17. Let satisfy and for every . For , let .
- (a)Show that
for all . (Decide which proof of Theorem 6.2.1 survives without definiteness.) - (b)Show that if and only if .
- (c)On the quotient space (Definition 4.6.20), show that
is well defined and is an inner product, and that is the Gram matrix of . - (d)Deduce that a matrix with is the Gram matrix of a list in some inner product space if and only if for all , if and only if for some , where one can take .
Uses. Lemma 6.2.6 (Nonnegative quadratics), Proposition 6.3.30 (Gram matrices detect independence), Theorem 6.3.22 (Parseval’s identity), Theorem 6.3.29 (Orthonormal columns), Proposition 6.5.9 (
Techniques. A nonnegative quadratic has nonpositive discriminant (Technique 6.2.7), Pass to orthonormal coordinates (Technique 6.7.2)
The function is linear in , and
(a) The second proof of Theorem 6.2.1 uses definiteness only to know that , so we follow it and handle that case separately. Choose with and
If , Lemma 6.2.6 gives
(b) If , then
(c) Let and , that is, and . Then
using
(d) If is the Gram matrix of , then
Matrices with these properties are called positive semidefinite. Section 8.3 produces the factorization differently, with a square root of , and Exercise 6.7.12 is the case
Problem 6.18. Let be a finite set of invertible matrices in that contains and is closed under multiplication and inverses, and let be the number of its elements.
- (a)Show that
is an inner product on , and that for every . - (b)Let
. Show that for every , that , and that is self-adjoint for . - (c)Deduce that . Conclude that if , then .
- (d)Show that there is an invertible such that is orthogonal for every .
Uses. Proposition 6.1.10 (The dot product is an inner product), Definition 6.6.8 (Adjoint), Proposition 6.6.9 (Adjoints are unique and linear), Proposition 6.6.18 (Properties of the adjoint), Corollary 6.6.29 (Orthogonal projections are the self-adjoint idempotents), Proposition 6.4.29 (The orthogonal projection as a linear map), Theorem 6.4.20 (Orthonormal bases exist), Corollary 6.4.25 (Dimension of the orthogonal complement), Proposition 2.4.28 (Cyclicity of the trace), Theorem 6.3.22 (Parseval’s identity), Theorem 6.3.29 (Orthonormal columns)
Techniques. Move an operator across the inner product (Technique 6.6.20), Cyclicity of trace (Technique 2.4.29)
Fix . The map sends into (closure) and is injective ( implies ), so it is a bijection of the finite set . The same holds for and for , which is its own inverse. So a sum over can be reindexed by any of these maps.
(a) Each term is linear in and symmetric, and so is the sum. Since ,
(b) Reindexing by ,
For and all , by (a) applied with ,
so the adjoint of for is (Definition 6.6.8, Proposition 6.6.9). Adjoints of sums and real multiples are the sums and multiples of the adjoints (Proposition 6.6.18), so the adjoint of is , which is after reindexing by .
(c) By (b) and Corollary 6.6.29, is the orthogonal projection onto for the inner product . Let . By Theorem 6.4.20 and Corollary 6.4.25, there is a basis of , orthonormal for , with a basis of and a basis of (Proposition 6.4.29(a)). So for and for . Let be the invertible matrix with columns . Then with
If , then
(d) Take with columns forming any basis that is orthonormal for , as in (c). For , is the coordinate vector of in this basis, since . By Theorem 6.3.22,
With and , the columns of are orthonormal, so is orthogonal (Theorem 6.3.29).
By (c), is the orthogonal projection onto the vectors fixed by all of : if for all then , and conversely by (b). The idempotence alone already gives , by the decomposition of Section 4.4, so the inner product is not needed for the conclusion of (c). It explains what is, and it gives (d): every finite group of real matrices is conjugate to a group of orthogonal matrices. Averaging over the group in this way is the basic tool of the representation theory of finite groups.
This is Problem B6 of the 1985 Putnam Competition, in a slightly restricted form: the original allows any finite set of real matrices that forms a group under multiplication, whose identity element need not be .