6.7 Proof patterns · Matrix Theory

An inner product adds one number $⟨ 𝐯, 𝐰 ⟩$ to every pair of vectors, and the chapter turned that number into lengths, angles, projections, least-squares solutions, and adjoints. Most of its proofs use a small number of moves to get from a hypothesis about inner products to a conclusion about vectors or maps: expand a squared norm and use positivity, test a vector against everything, move an operator to the other slot, choose an orthonormal basis, or split a vector into orthogonal pieces. This section sorts the chapter’s arguments by the question they answer, solves a few problems in several ways to compare the moves, and lists the errors that the chapter’s exercises warn against. The exercises at the end practice one move at a time, and the problem set (Problems for Chapter 6) mixes them with earlier chapters.

6.7.1Proving an inequality

Almost every inequality in the chapter is a squared norm that was dropped because it is nonnegative. The work is in choosing the vector whose norm to drop.

Expand a squared norm. Write $0 \leq {‖ 𝐱 - 𝑡 𝐲 ‖}^{2}$ , or the norm of some other combination, and expand with Proposition 6.1.29. All three proofs of Theorem 6.2.1 start this way and differ only in the choice of $𝑡$ (Section 6.2.1). The same move gives the reverse triangle inequality (Corollary 6.2.14) and identities such as Exercise 6.1.57.
Drop an orthogonal piece (Technique 6.2.4, Technique 6.4.5). If $𝐱 = 𝐩 + 𝐪$ with $𝐩 ⟂ 𝐪$ , then $‖ 𝐩 ‖ \leq ‖ 𝐱 ‖$ . This is Bessel’s inequality (Theorem 6.3.24), the best-approximation property (Theorem 6.4.33), and, with $𝐩$ the part of $𝐱$ along $𝐲$ , Cauchy–Schwarz again.
Cauchy–Schwarz for a well-chosen inner product (Technique 6.2.21). For inequalities between sums or integrals, the choice of space, weights, and splitting of each term is the whole proof (Proposition 6.2.22, Corollary 6.2.26, Exercise 6.2.36).
A discriminant (Technique 6.2.7). When the target has the shape $𝑏^{2} \leq 𝑎 𝑐$ and there is no picture, look for a quadratic in a real parameter that is nonnegative for an obvious reason.
Normalize. Both sides of a homogeneous inequality scale the same way, so it suffices to check it on unit vectors (third proof of Theorem 6.2.1). Over $ℂ$ , rotating the phase (Technique 6.2.8) makes an inner product real before a real-variable argument is run.

The equality case comes from the same computation: equality holds exactly when the dropped quantity is $0$ . That is why it pays to keep the dropped term written down, as in (6.2.2), instead of replacing it by “ $\geq 0$ ” at once. The triangle inequality (Theorem 6.2.11(b)), Bessel’s inequality, and best approximation all get their equality cases this way.

Example 6.7.1 (The Cauchy–Schwarz gap, three ways). Let $𝐱, 𝐲$ be vectors in an inner product space $𝑉$ , and let

𝐷 = {‖ 𝐱 ‖}^{2} {‖ 𝐲 ‖}^{2} - {| ⟨ 𝐱, 𝐲 ⟩ |}^{2},

the amount by which Cauchy–Schwarz fails to be an equality. Here are three formulas for $𝐷$ .

By orthogonal decomposition. If $𝐲 \neq 𝟎$ , (6.2.2) says $𝐷 =$ ${‖ 𝐲 ‖}^{2} {‖ 𝐱 - 𝑐 𝐲 ‖}^{2}$ with $𝑐 = ⟨ 𝐱, 𝐲 ⟩ / {‖ 𝐲 ‖}^{2}$ . Since $𝑐 𝐲$ is the projection of $𝐱$ onto the line $span (𝐲)$ , this reads $𝐷 =$ ${‖ 𝐲 ‖}^{2} {dist (𝐱, span (𝐲))}^{2}$ (Definition 6.4.34).

In orthonormal coordinates. If $𝐱, 𝐲$ are linearly dependent, $𝐷 = 0$ by Lemma 6.2.3. Otherwise $𝑊 = span (𝐱, 𝐲)$ has an orthonormal basis $𝒪︀ = 𝐮_{1}, 𝐮_{2}$ (Theorem 6.4.20). Write $𝐱 = 𝑎_{1} 𝐮_{1} + 𝑎_{2} 𝐮_{2}$ and $𝐲 = 𝑏_{1} 𝐮_{1} + 𝑏_{2} 𝐮_{2}$ . By Theorem 6.3.22, $⟨ 𝐱, 𝐲 ⟩ =$ $𝑎_{1} 𝑏_{1} +$ $𝑎_{2} 𝑏_{2}$ , ${‖ 𝐱 ‖}^{2} =$ ${| 𝑎_{1} |}^{2} +$ ${| 𝑎_{2} |}^{2}$ , and ${‖ 𝐲 ‖}^{2} =$ ${| 𝑏_{1} |}^{2} +$ ${| 𝑏_{2} |}^{2}$ . Multiplying out,

\begin{aligned} {‖ 𝐱 ‖}^{2} {‖ 𝐲 ‖}^{2} & = {| 𝑎_{1} 𝑏_{1} |}^{2} + {| 𝑎_{1} 𝑏_{2} |}^{2} + {| 𝑎_{2} 𝑏_{1} |}^{2} + {| 𝑎_{2} 𝑏_{2} |}^{2}, \\ {| ⟨ 𝐱, 𝐲 ⟩ |}^{2} & = {| 𝑎_{1} 𝑏_{1} |}^{2} + {| 𝑎_{2} 𝑏_{2} |}^{2} + 2 Re (𝑎_{1} 𝑏_{2} 𝑎_{2} 𝑏_{1}), \end{aligned}

and the difference is ${| 𝑎_{1} 𝑏_{2} |}^{2} -$ $2 Re (𝑎_{1} 𝑏_{2} 𝑎_{2} 𝑏_{1}) +$ ${| 𝑎_{2} 𝑏_{1} |}^{2}$ . So

𝐷 = {| 𝑎_{1} 𝑏_{2} - 𝑎_{2} 𝑏_{1} |}^{2} .

As a determinant. By definition (Definition 6.3.28), the Gram matrix of $𝐱, 𝐲$ is

𝐺 = (\begin{matrix} ⟨ 𝐱, 𝐱 ⟩ & ⟨ 𝐲, 𝐱 ⟩ \\ ⟨ 𝐱, 𝐲 ⟩ & ⟨ 𝐲, 𝐲 ⟩ \end{matrix}),

and $det 𝐺 =$ ${‖ 𝐱 ‖}^{2} {‖ 𝐲 ‖}^{2} -$ $⟨ 𝐲, 𝐱 ⟩ ⟨ 𝐱, 𝐲 ⟩ =$ $𝐷$ . In the coordinates above, let $𝐵 = (\begin{matrix} 𝑎_{1} & 𝑏_{1} \\ 𝑎_{2} & 𝑏_{2} \end{matrix})$ , with columns ${[𝐱]}_{𝒪︀}$ and ${[𝐲]}_{𝒪︀}$ . By Theorem 6.3.22, the $(𝑖, 𝑗)$ entry of $𝐵^{*} 𝐵$ is ${[𝐯_{𝑖}]}_{𝒪︀}^{*} {[𝐯_{𝑗}]}_{𝒪︀} =$ $⟨ 𝐯_{𝑗}, 𝐯_{𝑖} ⟩$ , where $𝐯_{1} = 𝐱$ and $𝐯_{2} = 𝐲$ ; so $𝐺 = 𝐵^{*} 𝐵$ . Then $det 𝐺 =$ $det (𝐵^{*}) det 𝐵 =$ ${| det 𝐵 |}^{2}$ (Theorem 5.4.10, Exercise 6.3.57), which is the second formula again.

Each formula proves Cauchy–Schwarz with its equality case: $𝐷 \geq 0$ , and $𝐷 = 0$ exactly when $𝐱 \in span (𝐲)$ , or $det 𝐵 = 0$ , or $𝐺$ is singular, all of which say that $𝐱, 𝐲$ are dependent (Proposition 6.3.30). They lead in different directions. The first measures the gap by a distance, and Section 6.4 replaces the line by any subspace. The second writes $𝐷$ as one squared modulus, a certificate of nonnegativity in the sense of Technique 6.1.9; over $ℝ^{𝑛}$ with the dot product it is Lagrange’s identity (Exercise 6.2.43), and the orthonormal basis is what reduces every inner product space to $𝑛 = 2$ . The third gives the equality case without any basis (Proposition 6.3.30), but its sign comes from $det 𝐺 = {| det 𝐵 |}^{2}$ , that is, from the coordinates. It extends to $𝑘$ vectors: the Gram determinant of any list is nonnegative and vanishes exactly for dependent lists, by orthonormal coordinates again (Exercise 6.7.11) or by Gram–Schmidt residuals (Exercise 6.4.59). In $ℝ^{2}$ , $| det 𝐵 |$ is the area of the parallelogram spanned by $𝐱$ and $𝐲$ (Remark 6.2.5).

The last two proofs could not have been given in Section 6.2, since they use orthonormal bases, but they are not circular: Gram–Schmidt and Parseval’s identity never use Cauchy–Schwarz.

The middle formula is an instance of a move that the chapter used several times without naming it.

Technique 6.7.2 (Pass to orthonormal coordinates). A statement about finitely many vectors $𝐯_{1}, \dots, 𝐯_{𝑘}$ of an inner product space $𝑉$ involves only $𝑊 =$ $span (𝐯_{1}, \dots, 𝐯_{𝑘})$ , which is finite-dimensional even when $𝑉$ is not. Choose an orthonormal basis $𝒪︀$ of $𝑊$ (Theorem 6.4.20). By Theorem 6.3.22, the coordinate map $𝐯 \mapsto {[𝐯]}_{𝒪︀}$ preserves inner products, so $𝑊$ behaves exactly like $𝔽^{𝑟}$ with the dot product, where $𝑟 = dim 𝑊 \leq 𝑘$ . The list becomes the columns of a matrix $𝐵 \in 𝔽^{𝑟 \times 𝑘}$ with $𝐵^{*} 𝐵 = 𝐺$ , the Gram matrix of the list, and the statement can be checked with entries, sums of squares, and determinants.

Reach for it when a statement about a fixed finite list is easy to verify in $𝔽^{𝑟}$ but awkward without coordinates, or when you need a matrix whose Gram matrix is prescribed. The canonical example is the second formula of Example 6.7.1. It also shows that Gram determinants are nonnegative (Exercise 6.7.11) and produces the matrix $𝐵$ in Problem 6.17. It does not apply as stated to a map on an infinite-dimensional $𝑉$ , since the map need not preserve $𝑊$ .

Used in 8 places and 1 practice exercise

Exercise 6.7.11, Problem 6.17, Theorem 8.1.3, Proposition 8.1.7, Theorem 8.2.18, Exercise 8.2.42, Theorem 8.3.5, Lemma 8.3.8; 1 practice exercise.

Practice: Exercise 6.7.6 and Exercise 6.7.12.

6.7.2Showing that a vector or a map is zero

An inner product gives several ways to prove $𝐯 = 𝟎$ , or $𝑆 = 𝑇$ , which a bare vector space does not have.

Its norm (Technique 6.1.36). Show $⟨ 𝐯, 𝐯 ⟩ = 0$ . If $𝐯$ is a matrix, show $tr (𝐯^{*} 𝐯) = 0$ (Technique 6.1.18). If the norm is a sum of squares, every term vanishes (Technique 6.1.9).
Test against everything. If $⟨ 𝐯, 𝐱 ⟩ = 0$ for every $𝐱$ in $𝑉$ , or in a spanning list, then $𝐯 = 𝟎$ : take $𝐱 = 𝐯$ (Proposition 6.1.37, Exercise 6.1.50). For a finite-dimensional subspace $𝑊$ this is $𝑊^{⟂} = {𝟎} \Rightarrow 𝑊 = 𝑉$ (Corollary 6.4.27), and for maps it is Corollary 6.1.38.
Test on the diagonal. Over $ℂ$ , the numbers $⟨ 𝑇 𝐯, 𝐯 ⟩$ already determine $𝑇$ (Proposition 6.1.41). Over $ℝ$ they determine only self-adjoint operators, and the rotation by $𝜋 / 2$ shows why (Example 6.1.39).
Move the operator across (Technique 6.6.20), to turn the hypothesis into one of the forms above. The standard instance is $⟨ 𝑇^{*} 𝑇 𝐯, 𝐯 ⟩ =$ ${‖ 𝑇 𝐯 ‖}^{2}$ , which gives $ker 𝑇^{*} 𝑇 = ker 𝑇$ (Proposition 6.6.23).
Orthogonal pieces. A vector in $𝑊 \cap 𝑊^{⟂}$ is orthogonal to itself, hence $𝟎$ . This is how uniqueness of orthogonal decompositions is proved (Theorem 6.4.24), and how $ker 𝑇^{*} = {(im 𝑇)}^{⟂}$ is used (Theorem 6.6.21).

Example 6.7.3 (A normal matrix with square zero, three certificates). Let $𝐴 \in ℂ^{𝑛 \times 𝑛}$ satisfy $𝐴^{*} 𝐴 = 𝐴 𝐴^{*}$ and $𝐴^{2} = 0$ . (Matrices with $𝐴^{*} 𝐴 = 𝐴 𝐴^{*}$ are called normal; Section 8.1 studies them.) We show that $𝐴 = 0$ in three ways.

Norms. With the dot product, $𝐴^{*}$ is the adjoint of $𝐯 \mapsto 𝐴 𝐯$ (Example 6.6.10). For every $𝐯 \in ℂ^{𝑛}$ ,

{‖ 𝐴 𝐯 ‖}^{2} = ⟨ 𝐴^{*} 𝐴 𝐯, 𝐯 ⟩ = ⟨ 𝐴 𝐴^{*} 𝐯, 𝐯 ⟩ = {‖ 𝐴^{*} 𝐯 ‖}^{2} .

Applied to $𝐴 𝐯$ this gives $‖ 𝐴^{*} 𝐴 𝐯 ‖ =$ $‖ 𝐴^{2} 𝐯 ‖ =$ $0$ , so $𝐴^{*} 𝐴 𝐯 = 𝟎$ , and then ${‖ 𝐴 𝐯 ‖}^{2} =$ $⟨ 𝐴^{*} 𝐴 𝐯, 𝐯 ⟩ =$ $0$ . So $𝐴 𝐯 = 𝟎$ for every $𝐯$ .

The trace of $𝐵^{*} 𝐵$ . Let $𝐵 = 𝐴^{*} 𝐴$ . Then $𝐵^{*} = 𝐵$ , and $𝐵^{2} =$ $𝐴^{*} (𝐴 𝐴^{*}) 𝐴 =$ $𝐴^{*} (𝐴^{*} 𝐴) 𝐴 =$ ${(𝐴^{*})}^{2} 𝐴^{2} =$ $0$ . So $tr (𝐵^{*} 𝐵) =$ $tr (𝐵^{2}) =$ $0$ , which forces $𝐵 = 0$ (Technique 6.1.18), and $𝐴^{*} 𝐴 = 0$ forces $𝐴 = 0$ (Corollary 6.1.19).

Subspaces. $𝐴^{2} = 0$ says $im 𝐴 \subseteq ker 𝐴$ . By Proposition 6.5.9(a), applied to $𝐴$ and to $𝐴^{*}$ , $ker 𝐴 =$ $ker (𝐴^{*} 𝐴) =$ $ker (𝐴 𝐴^{*}) =$ $ker 𝐴^{*}$ , and $ker 𝐴^{*} = {(im 𝐴)}^{⟂}$ by Lemma 6.5.4. So $im 𝐴 \subseteq {(im 𝐴)}^{⟂}$ : every $𝐰 \in im 𝐴$ is orthogonal to itself, and $im 𝐴 = {𝟎}$ .

The norm argument is the shortest; it uses nothing but moving $𝐴$ across the inner product twice. The trace argument never mentions a vector, and it is the one to try when all hypotheses are matrix identities: one nonnegative number controls every entry. The subspace argument shows what normality does: it makes $ker 𝐴 = ker 𝐴^{*}$ , so the image of $𝐴$ , which is orthogonal to $ker 𝐴^{*}$ , is also orthogonal to $ker 𝐴$ , and an $𝐴$ with $𝐴^{2} = 0$ would need its image inside that kernel.

Each argument breaks at the same hypothesis for $𝑁 = (\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix})$ , which has $𝑁^{2} = 0$ but is not normal: $‖ 𝑁 𝐞_{2} ‖ = 1$ while $‖ 𝑁^{*} 𝐞_{2} ‖ = 0$ , and $ker 𝑁 = span (𝐞_{1})$ while $ker 𝑁^{*} = span (𝐞_{2})$ . The norm argument proves more than was asked. It shows that $ker 𝐴^{2} = ker 𝐴$ for every normal $𝐴$ , so by induction $ker 𝐴^{𝑘} = ker 𝐴$ for all $𝑘 \geq 1$ , and a normal matrix with $𝐴^{𝑘} = 0$ is $0$ .

Practice: Exercise 6.7.7, Exercise 6.7.9, and Exercise 6.7.8.

6.7.3Projections, distances, and least squares

The projection ${Pr}_{𝑊} (𝐯)$ , the closest point of $𝑊$ to $𝐯$ , and the fitted values of a least-squares problem are the same vector (Theorem 6.4.33, Proposition 6.5.2). There are several ways to find it, or to use it without finding it.

Orthonormal basis first. Once $𝑊$ has an orthonormal or orthogonal basis, the projection is a sum of one-term projections (Definition 6.4.11, Corollary 6.4.10), and in $𝔽^{𝑛}$ its matrix is $𝑄 𝑄^{*}$ (Theorem 6.4.31). This pays when an orthogonal spanning list is visible, or can be made by symmetry (Exercise 6.4.54).
Normal equations (Theorem 6.5.5, Proposition 6.5.23). For any spanning list with Gram matrix $𝐺$ , solve $𝐺 𝐜 = 𝐝$ . Nothing has to be orthogonalized.
The complement. ${Pr}_{𝑊} = id - {Pr}_{𝑊^{⟂}}$ (Proposition 6.4.29(d)), which is the better route when $𝑊^{⟂}$ is the smaller space, as for hyperplanes (Example 6.4.35).
Recognize instead of compute (Technique 6.4.7). To show that $𝐩 = {Pr}_{𝑊} (𝐯)$ , check $𝐩 \in 𝑊$ and $𝐯 - 𝐩 ⟂ 𝑊$ . This is also how projections onto related subspaces are compared.
Perturb a minimizer (Technique 6.5.6). When a vector is defined by minimizing a squared norm, perturbing it in each direction produces the orthogonality condition, which is the second proof of Theorem 6.5.5.

Example 6.7.4 (One least-squares error, four ways). Fit a line $𝑦 = 𝑐_{0} + 𝑐_{1} 𝑥$ to the points $(1, 1),$ $(2, 3),$ $(3, 2),$ $(4, 5)$ . With $𝟏 = (1, 1, 1, 1)$ , $𝐱 = (1, 2, 3, 4)$ , $𝐲 = (1, 3, 2, 5)$ , and $𝑊 = span (𝟏, 𝐱) \subseteq ℝ^{4}$ , the squared least-squares error is ${dist (𝐲, 𝑊)}^{2}$ . Note ${‖ 𝐲 ‖}^{2} = 39$ .

Normal equations. With $𝐴 = (𝟏 𝐱)$ ,

𝐴^{𝖳} 𝐴 = (\begin{matrix} 4 & 10 \\ 10 & 30 \end{matrix}), 𝐴^{𝖳} 𝐲 = (\begin{matrix} 11 \\ 33 \end{matrix}),

so $𝐜 = (0, 11 / 10)$ and the line is $𝑦 = 11 𝑥 / 10$ . The residual is $𝐲 -$ $𝐴 𝐜 =$ $(- 1 / 10, 4 / 5, - 13 / 10, 3 / 5)$ , and its squared norm is $(1 + 64 + 169 + 36) / 100 =$ $27 / 10$ .

An orthogonal basis of $𝑊$ . One step of Gram–Schmidt replaces $𝐱$ by $𝐱 - (10 / 4) 𝟏$ , and doubling gives $𝐰 = (- 3, - 1, 1, 3) ⟂ 𝟏$ . By Corollary 6.4.10 and Pythagoras (Proposition 6.4.4),

{dist (𝐲, 𝑊)}^{2} = {‖ 𝐲 ‖}^{2} - \frac{{⟨ 𝐲, 𝟏 ⟩}^{2}}{{‖ 𝟏 ‖}^{2}} - \frac{{⟨ 𝐲, 𝐰 ⟩}^{2}}{{‖ 𝐰 ‖}^{2}} = 39 - \frac{121}{4} - \frac{121}{20} = \frac{27}{10} .

The complement. $𝑊^{⟂}$ has dimension $2$ (Corollary 6.4.25). Among vectors of the form $(𝑎, 𝑏, 𝑏, 𝑎)$ , the conditions $⟂ 𝟏$ and $⟂ 𝐱$ leave $𝐠_{1} = (1, - 1, - 1, 1)$ ; among vectors of the form $(𝑎, 𝑏, - 𝑏, - 𝑎)$ , they leave $𝐠_{2} = (1, - 3, 3, - 1)$ . Vectors of these two forms are always orthogonal, so $𝐠_{1}, 𝐠_{2}$ is an orthogonal basis of $𝑊^{⟂}$ . The residual is ${Pr}_{𝑊^{⟂}} (𝐲) =$ $(1 / 4) 𝐠_{1} -$ $(7 / 20) 𝐠_{2}$ (Proposition 6.4.29(d)), and

{dist (𝐲, 𝑊)}^{2} = \frac{1^{2}}{4} + \frac{7^{2}}{20} = \frac{27}{10} .

Gram determinants. By Exercise 6.4.59(b), ${dist (𝐲, 𝑊)}^{2}$ is the ratio of the Gram determinants of $𝟏, 𝐱, 𝐲$ and of $𝟏, 𝐱$ :

det (\begin{matrix} 4 & 10 & 11 \\ 10 & 30 & 33 \\ 11 & 33 & 39 \end{matrix}) / det (\begin{matrix} 4 & 10 \\ 10 & 30 \end{matrix}) = \frac{54}{20} = \frac{27}{10} .

With two unknowns, the normal equations are the quickest route, and they also produce the line. The orthogonal basis costs one Gram–Schmidt step, after which each coefficient is a single inner product, and fitting a parabola instead would only add a term: the coefficients already found do not change. The complement computes the residual without the fitted values, and it is the efficient route when $𝑊^{⟂}$ is smaller than $𝑊$ . Gram determinants need no solving, but they give only the distance, not the closest point.

Practice: Exercise 6.7.5 and Exercise 6.7.13.

6.7.4Choosing an orthonormal basis

In an orthonormal basis, coordinates are inner products (Proposition 6.3.18), inner products are dot products of coordinates (Theorem 6.3.22), the adjoint is the conjugate transpose (Theorem 6.6.15(b)), self-adjoint operators are Hermitian matrices (Corollary 6.6.28), and an orthogonal projection is $diag (1, \dots, 1, 0, \dots, 0)$ when the first basis vectors span its image (Corollary 6.4.25). So when a statement is basis-free but easier in coordinates, choose an orthonormal basis adapted to the problem: one whose first vectors span a given subspace (Theorem 6.4.20(b)), or one of the span of the vectors at hand (Technique 6.7.2). None of these formulas survives in a basis that is not orthonormal. There the Gram matrix $𝐺$ of the basis enters, as in $⟨ 𝐯, 𝐰 ⟩ =$ ${[𝐰]}^{*} 𝐺 [𝐯]$ and $[𝑇^{*}] = 𝐺^{- 1} {[𝑇]}^{*} 𝐺$ (Exercise 6.6.46, Example 6.6.16).

Coordinates are not always the shorter route. Statements about kernels, images, and complements usually have one-line proofs without a basis, as $ker 𝑇^{*} = {(im 𝑇)}^{⟂}$ does (Theorem 6.6.21), and a coordinate proof of them hides the reason. And in an infinite-dimensional space only finite-dimensional subspaces are guaranteed an orthonormal basis, so an argument that picks an orthonormal basis of $𝑉$ has assumed $dim 𝑉 < \infty$ . A useful check after a coordinate proof is to ask whether the quantity computed depends on the basis; if the statement was basis-free, it cannot, and showing that directly sometimes gives a better proof (Exercise 6.7.10).

Practice: Exercise 6.7.10 and Exercise 6.7.11.

6.7.5Lengths determine inner products

By polarization (Proposition 6.1.32, Technique 6.1.33), $⟨ 𝐱, 𝐲 ⟩$ is a combination of the four lengths $‖ 𝐱 + 𝑖^{𝑘} 𝐲 ‖$ , or of two over $ℝ$ . So a hypothesis about lengths alone is a hypothesis about inner products: a linear map that preserves lengths preserves inner products (Exercise 6.1.51, Exercise 6.6.40), and two inner products with the same norm are equal (Corollary 6.1.34). For operators, the corresponding statement is about the numbers $⟨ 𝑇 𝐯, 𝐯 ⟩$ , and here real and complex scalars part ways: over $ℂ$ these numbers determine $𝑇$ (Proposition 6.1.41), and they are all real exactly when $𝑇$ is self-adjoint (Proposition 6.6.30); over $ℝ$ they see only the self-adjoint part $(𝑇 + 𝑇^{*}) / 2$ (Exercise 6.1.52). The general move behind all of this is to apply a hypothesis to $𝐱 + 𝑐 𝐲$ for a few scalars $𝑐$ and solve for the cross term (Technique 6.1.40).

A useful habit is to write every proof over $𝔽$ , with $Re$ wherever a cross term appears, and to specialize to $ℝ$ by erasing the bars. The differences between the two cases are concentrated in a few places: the converse of Pythagoras (Theorem 6.3.3(b), Non-example 6.3.4), the polarization identities, rotating the phase, and tests on the diagonal. Practice: Exercise 6.7.15 and Exercise 6.7.8.

6.7.6Reading the chapter’s proofs

Uniqueness first, then existence. The projection was characterized by two conditions before anything was computed (Theorem 6.4.6), and uniqueness then gave independence of the basis for free (Corollary 6.4.8). Adjoints went the same way: uniqueness (Proposition 6.6.9) needs only that inner products distinguish vectors, while existence (Theorem 6.6.15) needs a finite-dimensional space and either the Riesz representation theorem or a matrix computation. Recognizing an object by a property that determines it (Technique 6.4.7, Technique 6.6.11) is the inner-product version of checking that a candidate is an inverse (Technique 2.5.16); compare Section 5.6.5.

Where finite dimension enters. Every existence statement of the chapter needs an orthonormal basis of something, and that is the only place where finite dimension is used. For ${Pr}_{𝑊}$ and $𝑉 = 𝑊 \oplus 𝑊^{⟂}$ , only $𝑊$ has to be finite-dimensional (Theorem 6.4.24). The identity $ker 𝑇^{*} = {(im 𝑇)}^{⟂}$ needs no dimension hypothesis, while $im 𝑇^{*} = {(ker 𝑇)}^{⟂}$ does (Theorem 6.6.21). When a hypothesis of this kind is dropped, the conclusion can fail (Exercise 6.4.63, Non-example 6.6.7).

Orthogonality turns sums into sums of squares. Pythagoras (Corollary 6.3.6), Parseval (Theorem 6.3.22), Bessel (Theorem 6.3.24), and the defect formula (6.2.2) all say that the squared length of a sum of orthogonal pieces is the sum of the squared lengths. That is what makes coefficients in an orthonormal basis independent of each other (Technique 6.3.12), and why projections, unlike arbitrary idempotents, shrink vectors (Exercise 6.4.57).

The scalars are $ℝ$ or $ℂ$ for a reason. Positivity and definiteness use the order on $ℝ$ . Over $𝔽_{𝑝}$ or with a symmetric bilinear form over $ℂ$ , there are nonzero vectors orthogonal to themselves (Exercise 6.1.58, Proposition 6.1.6), and then projections, Gram–Schmidt, and the normal equations can all fail (Exercise 6.5.44). Section 12.4 studies what survives.

6.7.7Common errors

Most of these appear in the chapter as find-the-error exercises.

A conjugate in the wrong place. The inner product is conjugate-linear in the second slot: $⟨ 𝐯, 𝑎 𝐰 ⟩ =$ $𝑎 ⟨ 𝐯, 𝐰 ⟩$ (Proposition 6.1.4). Coordinates need the vector in the first slot (Remark 6.3.19), and Gram–Schmidt with the slots exchanged produces vectors that are not orthogonal (Exercise 6.4.21).
The transpose instead of the conjugate transpose. Over $ℂ$ , $𝐴^{𝖳} 𝐴$ can be singular when the columns of $𝐴$ are independent, and $\sum_{𝑘} 𝑧_{𝑘} 𝑤_{𝑘}$ is not an inner product (Remark 6.5.10, Exercise 6.2.30).
Real facts used over $ℂ$ , and complex facts used over $ℝ$ . The converse of Pythagoras and the two-term polarization identity hold only over $ℝ$ , and a discriminant argument with a complex parameter proves nothing (Exercise 6.2.35). In the other direction, $⟨ 𝑇 𝐯, 𝐯 ⟩ = 0$ for all $𝐯$ forces $𝑇 = 0$ only over $ℂ$ (Example 6.1.39, Exercise 6.1.59, Problem 6.6).
Projecting onto a spanning list that is not orthogonal, one vector at a time (Exercise 6.4.42), and its relative ${Pr}_{𝑈 + 𝑊} = {Pr}_{𝑈} + {Pr}_{𝑊}$ (Exercise 6.4.50).
A one-sided inverse used as a two-sided one. $𝐴 {(𝐴^{*} 𝐴)}^{- 1} 𝐴^{*}$ is not $𝐼_{𝑚}$ (Exercise 6.5.36), $𝑇^{*} 𝑇 = id$ does not give $𝑇 𝑇^{*} = id$ (Exercise 6.6.41), and orthogonal columns do not give orthogonal rows (Exercise 6.3.55).
The conjugate transpose in a basis that is not orthonormal (Example 6.6.16, Exercise 6.6.46).
Assuming the orthogonality to be proved. Pythagoras needs orthogonal pieces, and using it for $𝐯 = 𝑃 𝐯 + (𝐯 - 𝑃 𝐯)$ assumes that $𝑃$ is an orthogonal projection (Exercise 6.7.13). Swapping $𝑇 𝑇^{*}$ for $𝑇^{*} 𝑇$ assumes normality (Exercise 6.7.14).
A hypothesis that is never used. In both of the previous errors, the argument never touches one of the hypotheses. That is worth checking in every proof, and it is a warning sign whenever it happens.
Dropping finite dimension (Exercise 6.4.63, Non-example 6.6.7), or definiteness (Exercise 6.1.60).

Exercises

Exercise 6.7.5.routine For each problem, choose a method, say why it fits, and compute.

(a)The distance from $𝐯 = (3, 1, 4, 1)$ to $𝑊 =$ $span ((1, 1, 1, 1), (1, - 1, 1, - 1))$ in $ℝ^{4}$ .
(b)The distance from $𝐯 = (1, 2, 3, 4)$ to the hyperplane $𝑥_{1} - 𝑥_{2} + 𝑥_{3} - 𝑥_{4} = 0$ in $ℝ^{4}$ .
(c)The least-squares solution of $𝑥_{1} + 𝑥_{2} = 1$ , $𝑥_{1} + 2 𝑥_{2} = 2$ , $𝑥_{1} + 3 𝑥_{2} = 2$ .

Solution.

Uses. Corollary 6.4.10 (Projecting with an orthogonal list), Example 6.4.35 (Distance to a hyperplane), Theorem 6.5.5 (The normal equations), Theorem 6.4.33 (Best approximation)

Techniques. Recognize a projection by its two conditions (Technique 6.4.7)

Hint 1. In (a), compute the inner product of the two spanning vectors. In (b), the complement of the hyperplane is a line. In (c), there are only two unknowns.

Hint 2. (a) Use the projection formula for an orthogonal list. (b)

dist (𝐯, 𝐻) =

| ⟨ 𝐯, 𝐧 ⟩ | / ‖ 𝐧 ‖

. (c) Solve

𝐴^{𝖳} 𝐴 𝐱 = 𝐴^{𝖳} 𝐛

, a

2 \times 2

system.

Sketch. (a) The spanning vectors are orthogonal, so

{Pr}_{𝑊} (𝐯) =

(9 / 4) 𝐰_{1} +

(5 / 4) 𝐰_{2} =

(7 / 2, 1, 7 / 2, 1)

and the distance is

‖ (- 1 / 2, 0, 1 / 2, 0) ‖ =

1 / \sqrt{2}

. (b) With

𝐧 = (1, - 1, 1, - 1)

, the distance is

| - 2 | / 2 = 1

. (c)

𝐴^{𝖳} 𝐴 = (\begin{matrix} 3 & 6 \\ 6 & 14 \end{matrix})

and

𝐴^{𝖳} 𝐛 = (5, 11)

give

\hat{𝐱} = (2 / 3, 1 / 2)

(a) The spanning vectors $𝐰_{1} = (1, 1, 1, 1)$ and $𝐰_{2} = (1, - 1, 1, - 1)$ are orthogonal, so no orthogonalization is needed and Corollary 6.4.10 applies directly. Here $⟨ 𝐯, 𝐰_{1} ⟩ = 9$ , $⟨ 𝐯, 𝐰_{2} ⟩ = 5$ , and ${‖ 𝐰_{1} ‖}^{2} = {‖ 𝐰_{2} ‖}^{2} = 4$ , so

{Pr}_{𝑊} (𝐯) = \frac{9}{4} 𝐰_{1} + \frac{5}{4} 𝐰_{2} = (7 / 2, 1, 7 / 2, 1) .

Then $𝐯 -$ ${Pr}_{𝑊} (𝐯) =$ $(- 1 / 2, 0, 1 / 2, 0)$ , and the distance is $1 / \sqrt{2}$ by Theorem 6.4.33. As a check, ${‖ 𝐯 ‖}^{2} -$ ${‖ {Pr}_{𝑊} (𝐯) ‖}^{2} =$ $27 -$ $53 / 2 =$ $1 / 2$ .

(b) The hyperplane is $𝐻 = {𝐧}^{⟂}$ with $𝐧 = (1, - 1, 1, - 1)$ , and its complement is the line through $𝐧$ , so it is quickest to project onto that line. By Example 6.4.35, $dist (𝐯, 𝐻) =$ $| ⟨ 𝐯, 𝐧 ⟩ | / ‖ 𝐧 ‖ =$ $| 1 -$ $2 +$ $3 -$ $4 | / 2 =$ $1$ .

(c) The columns $(1, 1, 1)$ and $(1, 2, 3)$ of the coefficient matrix $𝐴$ are not orthogonal, and there are only two unknowns, so we solve the normal equations (Theorem 6.5.5):

𝐴^{𝖳} 𝐴 = (\begin{matrix} 3 & 6 \\ 6 & 14 \end{matrix}), 𝐴^{𝖳} 𝐛 = (\begin{matrix} 5 \\ 11 \end{matrix}), \hat{𝐱} = \frac{1}{6} (\begin{matrix} 14 & - 6 \\ - 6 & 3 \end{matrix}) (\begin{matrix} 5 \\ 11 \end{matrix}) = (\begin{matrix} 2 / 3 \\ 1 / 2 \end{matrix}) .

The residual $𝐛 -$ $𝐴 \hat{𝐱} =$ $(- 1 / 6, 1 / 3, - 1 / 6)$ is orthogonal to both columns, as it must be.

Exercise 6.7.6.standard Let $𝐱, 𝐲$ be vectors in an inner product space.

(a)Show that $‖ 𝐱 + 𝑡 𝐲 ‖ \geq ‖ 𝐱 ‖$ for every $𝑡 \in 𝔽$ if and only if $⟨ 𝐱, 𝐲 ⟩ = 0$ .
(b)Over $ℂ$ , show that it is not enough to require the inequality in (a) for real $𝑡$ only.

Solution.

Uses. Proposition 6.1.29 (Expanding a norm)

Techniques. Perturb a minimizer (Technique 6.5.6), Rotate the phase (Technique 6.2.8)

Hint 1. Expand

{‖ 𝐱 + 𝑡 𝐲 ‖}^{2} - {‖ 𝐱 ‖}^{2}

as a function of

𝑡

Hint 2. For the converse, try

𝑡 = - 𝑠 ⟨ 𝐱, 𝐲 ⟩

with a small real

𝑠 > 0

. For (b), look at

𝑉 = ℂ

with

𝐱 = 1

and

𝐲 = 𝑖

Sketch.

{‖ 𝐱 + 𝑡 𝐲 ‖}^{2} -

{‖ 𝐱 ‖}^{2} =

2 Re (𝑡 ⟨ 𝐱, 𝐲 ⟩) +

{| 𝑡 |}^{2} {‖ 𝐲 ‖}^{2}

. If

⟨ 𝐱, 𝐲 ⟩ = 0

this is

\geq 0

. If

𝑐 = ⟨ 𝐱, 𝐲 ⟩ \neq 0

, then

𝑡 = - 𝑠 𝑐

gives

𝑠 {| 𝑐 |}^{2} (𝑠 {‖ 𝐲 ‖}^{2} - 2)

, which is negative for small

𝑠 > 0

. (b) In

ℂ

{| 1 + 𝑡 𝑖 |}^{2} = 1 + 𝑡^{2} \geq 1

for real

𝑡

, but

⟨ 1, 𝑖 ⟩ = - 𝑖

(a) By Proposition 6.1.29, for every $𝑡 \in 𝔽$ ,

{‖ 𝐱 + 𝑡 𝐲 ‖}^{2} - {‖ 𝐱 ‖}^{2} = 2 Re (𝑡 ⟨ 𝐱, 𝐲 ⟩) + {| 𝑡 |}^{2} {‖ 𝐲 ‖}^{2} .

If $⟨ 𝐱, 𝐲 ⟩ = 0$ , the right side is ${| 𝑡 |}^{2} {‖ 𝐲 ‖}^{2} \geq 0$ , so the inequality holds for every $𝑡$ .

Conversely, suppose $𝑐 = ⟨ 𝐱, 𝐲 ⟩ \neq 0$ , and take $𝑡 = - 𝑠 𝑐$ with $𝑠 > 0$ real. Then $𝑡 𝑐 = - 𝑠 {| 𝑐 |}^{2}$ , and the right side becomes

- 2 𝑠 {| 𝑐 |}^{2} + 𝑠^{2} {| 𝑐 |}^{2} {‖ 𝐲 ‖}^{2} = 𝑠 {| 𝑐 |}^{2} (𝑠 {‖ 𝐲 ‖}^{2} - 2) .

For $0 <$ $𝑠 <$ $2 / ({‖ 𝐲 ‖}^{2} + 1)$ this is negative, so $‖ 𝐱 + 𝑡 𝐲 ‖ < ‖ 𝐱 ‖$ for that $𝑡$ . This proves the contrapositive. The choice of $𝑡$ is Technique 6.5.6 in the direction $- 𝑐 𝐲$ : the factor $𝑐$ rotates the phase so that the linear term is real and negative.

(b) Take $𝑉 = ℂ$ with $⟨ 𝑧, 𝑤 ⟩ = 𝑧 𝑤$ , $𝐱 = 1$ , and $𝐲 = 𝑖$ . For real $𝑡$ , ${‖ 1 + 𝑡 𝑖 ‖}^{2} = 1 + 𝑡^{2} \geq 1$ , but $⟨ 1, 𝑖 ⟩ = 𝑖 = - 𝑖 \neq 0$ . Real $𝑡$ only see $Re ⟨ 𝐱, 𝐲 ⟩$ , and the imaginary part needs $𝑡 = 𝑖 𝑠$ .

The condition in (a) mentions only the norm, so it makes sense in any normed space, where it is used as a definition of orthogonality (Birkhoff–James orthogonality). In an inner product space it is ordinary orthogonality. Exercise 6.7.13 puts it to work.

Exercise 6.7.7.standard Let

𝑉

be a finite-dimensional inner product space and

𝑇 \in L (𝑉)

, and suppose that

⟨ 𝑇 𝐮, 𝐯 ⟩ = 0

whenever

𝐮 ⟂ 𝐯

. Show that

𝑇 = 𝑐 {id}_{𝑉}

for some

𝑐 \in 𝔽

Solution.

Uses. Theorem 6.4.20 (Orthonormal bases exist), Proposition 6.3.18 (Coordinates in an orthonormal basis), Theorem 2.2.1 (Linear maps are determined by a basis)

Techniques. Plug in clever test vectors (Technique 6.1.40), Take an inner product to isolate a coefficient (Technique 6.3.12)

Hint 1. Choose an orthonormal basis

𝐞_{1}, \dots, 𝐞_{𝑛}

. What does the hypothesis say about the coordinates of

𝑇 𝐞_{𝑖}

Hint 2. Each

𝑇 𝐞_{𝑖}

is a multiple

𝑐_{𝑖} 𝐞_{𝑖}

. To compare

𝑐_{𝑖}

with

𝑐_{𝑗}

, find two orthogonal vectors built from

𝐞_{𝑖}

and

𝐞_{𝑗}

other than

𝐞_{𝑖}, 𝐞_{𝑗}

themselves.

Sketch. In an orthonormal basis,

⟨ 𝑇 𝐞_{𝑖}, 𝐞_{𝑗} ⟩ = 0

for

𝑗 \neq 𝑖

, so

𝑇 𝐞_{𝑖} = 𝑐_{𝑖} 𝐞_{𝑖}

. Since

𝐞_{𝑖} +

𝐞_{𝑗} ⟂ 𝐞_{𝑖} -

𝐞_{𝑗}

0 =

⟨ 𝑇 (𝐞_{𝑖} + 𝐞_{𝑗}), 𝐞_{𝑖} - 𝐞_{𝑗} ⟩ =

𝑐_{𝑖} -

𝑐_{𝑗}

. So all

𝑐_{𝑖}

equal some

𝑐

, and

𝑇 = 𝑐 {id}_{𝑉}

on a basis.

If $𝑉 = {𝟎}$ , there is nothing to prove. Otherwise choose an orthonormal basis $𝐞_{1}, \dots, 𝐞_{𝑛}$ of $𝑉$ (Theorem 6.4.20). For $𝑗 \neq 𝑖$ , $𝐞_{𝑖} ⟂ 𝐞_{𝑗}$ , so $⟨ 𝑇 𝐞_{𝑖}, 𝐞_{𝑗} ⟩ = 0$ . By Proposition 6.3.18, $𝑇 𝐞_{𝑖} =$ $\sum_{𝑗} ⟨ 𝑇 𝐞_{𝑖}, 𝐞_{𝑗} ⟩ 𝐞_{𝑗} =$ $𝑐_{𝑖} 𝐞_{𝑖}$ , where $𝑐_{𝑖} =$ $⟨ 𝑇 𝐞_{𝑖}, 𝐞_{𝑖} ⟩$ .

Now let $𝑖 \neq 𝑗$ . The vectors $𝐞_{𝑖} + 𝐞_{𝑗}$ and $𝐞_{𝑖} - 𝐞_{𝑗}$ are orthogonal, since $⟨ 𝐞_{𝑖} + 𝐞_{𝑗}, 𝐞_{𝑖} - 𝐞_{𝑗} ⟩ =$ $1 -$ $0 +$ $0 -$ $1 =$ $0$ . By the hypothesis,

0 = ⟨ 𝑇 (𝐞_{𝑖} + 𝐞_{𝑗}), 𝐞_{𝑖} - 𝐞_{𝑗} ⟩ = ⟨ 𝑐_{𝑖} 𝐞_{𝑖} + 𝑐_{𝑗} 𝐞_{𝑗}, 𝐞_{𝑖} - 𝐞_{𝑗} ⟩ = 𝑐_{𝑖} - 𝑐_{𝑗} .

So $𝑐_{1} = \dots = 𝑐_{𝑛} = 𝑐$ , and $𝑇$ agrees with $𝑐 {id}_{𝑉}$ on a basis. By Theorem 2.2.1, $𝑇 = 𝑐 {id}_{𝑉}$ .

A basis-free version: for fixed $𝐮$ , the hypothesis says $𝑇 𝐮 ⟂ {span (𝐮)}^{⟂}$ , so $𝑇 𝐮 \in$ ${({span (𝐮)}^{⟂})}^{⟂} =$ $span (𝐮)$ by Theorem 6.4.26. A linear map that sends every vector to a multiple of itself is a multiple of the identity, by comparing $𝑇 (𝐮 + 𝐯)$ with $𝑇 𝐮 + 𝑇 𝐯$ for independent $𝐮, 𝐯$ .

Exercise 6.7.8.standard Let

𝑉

be a finite-dimensional inner product space and

𝑇 \in L (𝑉)

. Show that

𝑇^{*} 𝑇 = 𝑇 𝑇^{*}

if and only if

‖ 𝑇 𝐯 ‖ = ‖ 𝑇^{*} 𝐯 ‖

for every

𝐯 \in 𝑉

. (Such

𝑇

are called normal; see Section 8.1.) Over

ℝ

, where does your proof use that

𝑇^{*} 𝑇 - 𝑇 𝑇^{*}

is self-adjoint?

Solution.

Uses. Definition 6.6.8 (Adjoint), Proposition 6.6.18 (Properties of the adjoint), Proposition 6.1.41 (Over $ℂ$ , the values $⟨ 𝑇 𝐯, 𝐯 ⟩$ determine $𝑇$ ), Corollary 6.1.38 (Linear maps are determined by inner products)

Techniques. Move an operator across the inner product (Technique 6.6.20), Polarization (Technique 6.1.33), Plug in clever test vectors (Technique 6.1.40)

Hint 1. Write

{‖ 𝑇 𝐯 ‖}^{2}

and

{‖ 𝑇^{*} 𝐯 ‖}^{2}

⟨ 𝑆 𝐯, 𝐯 ⟩

for suitable operators

𝑆

Hint 2. For the converse,

𝑆 = 𝑇^{*} 𝑇 - 𝑇 𝑇^{*}

is self-adjoint with

⟨ 𝑆 𝐯, 𝐯 ⟩ = 0

for all

𝐯

. Over

ℂ

cite Proposition 6.1.41. Over

ℝ

, expand

⟨ 𝑆 (𝐱 + 𝐲), 𝐱 + 𝐲 ⟩

Sketch.

{‖ 𝑇 𝐯 ‖}^{2} -

{‖ 𝑇^{*} 𝐯 ‖}^{2} =

⟨ (𝑇^{*} 𝑇 - 𝑇 𝑇^{*}) 𝐯, 𝐯 ⟩

. So normality gives equal norms. Conversely

𝑆 = 𝑇^{*} 𝑇 - 𝑇 𝑇^{*}

is self-adjoint with zero diagonal values; expanding

⟨ 𝑆 (𝐱 + 𝐲), 𝐱 + 𝐲 ⟩ =

0

gives

2 Re ⟨ 𝑆 𝐱, 𝐲 ⟩ = 0

, and replacing

𝐲

𝑖 𝐲

over

ℂ

kills the imaginary part. So

𝑆 = 0

For every $𝐯$ , moving the operators across the inner product (Definition 6.6.8, with ${(𝑇^{*})}^{*} = 𝑇$ by Proposition 6.6.18),

{‖ 𝑇 𝐯 ‖}^{2} - {‖ 𝑇^{*} 𝐯 ‖}^{2} = ⟨ 𝑇^{*} 𝑇 𝐯, 𝐯 ⟩ - ⟨ 𝑇 𝑇^{*} 𝐯, 𝐯 ⟩ = ⟨ 𝑆 𝐯, 𝐯 ⟩,

where $𝑆 = 𝑇^{*} 𝑇 - 𝑇 𝑇^{*}$ . If $𝑇$ is normal, then $𝑆 = 0$ and the norms agree.

Conversely, suppose $⟨ 𝑆 𝐯, 𝐯 ⟩ = 0$ for every $𝐯$ . By Proposition 6.6.18, $𝑆^{*} =$ $𝑇^{*} 𝑇 -$ $𝑇 𝑇^{*} =$ $𝑆$ . For $𝐱, 𝐲 \in 𝑉$ ,

0 = ⟨ 𝑆 (𝐱 + 𝐲), 𝐱 + 𝐲 ⟩ = ⟨ 𝑆 𝐱, 𝐲 ⟩ + ⟨ 𝑆 𝐲, 𝐱 ⟩ = ⟨ 𝑆 𝐱, 𝐲 ⟩ + ⟨ 𝑆 𝐱, 𝐲 ⟩,

using $⟨ 𝑆 𝐲, 𝐱 ⟩ =$ $⟨ 𝐲, 𝑆 𝐱 ⟩ =$ $⟨ 𝑆 𝐱, 𝐲 ⟩$ . So $Re ⟨ 𝑆 𝐱, 𝐲 ⟩ = 0$ . Over $ℝ$ this says $⟨ 𝑆 𝐱, 𝐲 ⟩ = 0$ . Over $ℂ$ , apply it to $𝑖 𝐲$ in place of $𝐲$ : $Re ⟨ 𝑆 𝐱, 𝑖 𝐲 ⟩ =$ $Re (- 𝑖 ⟨ 𝑆 𝐱, 𝐲 ⟩) =$ $Im ⟨ 𝑆 𝐱, 𝐲 ⟩ =$ $0$ . Either way $⟨ 𝑆 𝐱, 𝐲 ⟩ = 0$ for all $𝐱, 𝐲$ , so $𝑆 = 0$ by Corollary 6.1.38. (Over $ℂ$ one can instead cite Proposition 6.1.41, which does not need $𝑆$ to be self-adjoint.)

Over $ℝ$ , self-adjointness was used to replace $⟨ 𝑆 𝐲, 𝐱 ⟩$ by $⟨ 𝑆 𝐱, 𝐲 ⟩$ . Without it the argument fails, as the rotation of Example 6.1.39 shows: it has zero diagonal values and is not $0$ .

Exercise 6.7.9.standard Let $𝑉$ be a finite-dimensional inner product space, and let $𝑇 \in L (𝑉)$ be self-adjoint with $𝑇^{3} = 𝑇^{2}$ .

(a)Show that $𝑇^{2} = 𝑇$ , so that $𝑇$ is an orthogonal projection.
(b)Show by an example that (a) fails without the hypothesis $𝑇^{*} = 𝑇$ .

Solution.

Uses. Proposition 6.6.18 (Properties of the adjoint), Proposition 6.1.27 (Basic properties of the norm), Corollary 6.6.29 (Orthogonal projections are the self-adjoint idempotents)

Techniques. Move an operator across the inner product (Technique 6.6.20), Show a vector is zero via its norm (Technique 6.1.36)

Hint 1. Let

𝑆 = 𝑇^{2} - 𝑇

. Is

𝑆

self-adjoint? Compute

𝑆^{2}

using

𝑇^{3} = 𝑇^{2}

Hint 2.

𝑆^{2} = 0

and

𝑆^{*} = 𝑆

. Now

{‖ 𝑆 𝐯 ‖}^{2} =

⟨ 𝑆^{*} 𝑆 𝐯, 𝐯 ⟩

. For (b), try a nilpotent matrix.

Sketch.

𝑆 = 𝑇^{2} - 𝑇

is self-adjoint, and

𝑇^{4} = 𝑇 𝑇^{3} = 𝑇^{3} = 𝑇^{2}

, so

𝑆^{2} =

𝑇^{4} -

2 𝑇^{3} +

𝑇^{2} =

0

. Then

{‖ 𝑆 𝐯 ‖}^{2} =

⟨ 𝑆^{2} 𝐯, 𝐯 ⟩ =

0

, so

𝑆 = 0

, and

𝑇^{2} = 𝑇 = 𝑇^{*}

is an orthogonal projection. (b)

𝑁 = (\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix})

has

𝑁^{3} = 𝑁^{2} = 0 \neq 𝑁

(a) Let $𝑆 = 𝑇^{2} - 𝑇$ . By Proposition 6.6.18, $𝑆^{*} = {(𝑇^{*})}^{2} - 𝑇^{*} = 𝑆$ . Since $𝑇^{4} =$ $𝑇 𝑇^{3} =$ $𝑇 𝑇^{2} =$ $𝑇^{3} =$ $𝑇^{2}$ ,

𝑆^{2} = 𝑇^{4} - 2 𝑇^{3} + 𝑇^{2} = 𝑇^{2} - 2 𝑇^{2} + 𝑇^{2} = 0 .

For every $𝐯$ , ${‖ 𝑆 𝐯 ‖}^{2} =$ $⟨ 𝑆 𝐯, 𝑆 𝐯 ⟩ =$ $⟨ 𝑆^{*} 𝑆 𝐯, 𝐯 ⟩ =$ $⟨ 𝑆^{2} 𝐯, 𝐯 ⟩ =$ $0$ , so $𝑆 𝐯 = 𝟎$ (Proposition 6.1.27). Hence $𝑇^{2} = 𝑇$ . Together with $𝑇^{*} = 𝑇$ , this makes $𝑇$ an orthogonal projection by Corollary 6.6.29.

(b) Let $𝑁 \in ℝ^{2 \times 2}$ have $𝑁 𝐞_{1} = 𝟎$ and $𝑁 𝐞_{2} = 𝐞_{1}$ , that is, $𝑁 = (\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix})$ . Then $𝑁^{2} = 0$ , so $𝑁^{3} = 𝑁^{2}$ , but $𝑁^{2} = 0 \neq 𝑁$ .

The proof used self-adjointness only to make $𝑆^{*} 𝑆 = 𝑆^{2}$ , so that the algebraic fact $𝑆^{2} = 0$ became the positive fact ${‖ 𝑆 𝐯 ‖}^{2} = 0$ . For a general $𝑇$ , $𝑆^{2} = 0$ says nothing about $𝑆$ , as $𝑁$ shows.

Exercise 6.7.10.standard Let $𝑉, 𝑊$ be finite-dimensional inner product spaces and $𝑇 \in L (𝑉, 𝑊)$ . Let $𝐞_{1}, \dots, 𝐞_{𝑛}$ be an orthonormal basis of $𝑉$ and $𝐟_{1}, \dots, 𝐟_{𝑚}$ one of $𝑊$ .

(a)Show that
$\sum_{𝑗 = 1}^{𝑛} {‖ 𝑇 𝐞_{𝑗} ‖}^{2} = \sum_{𝑖 = 1}^{𝑚} \sum_{𝑗 = 1}^{𝑛} {| ⟨ 𝑇 𝐞_{𝑗}, 𝐟_{𝑖} ⟩ |}^{2} = \sum_{𝑖 = 1}^{𝑚} {‖ 𝑇^{*} 𝐟_{𝑖} ‖}^{2} .$
(b)Deduce that $\sum_{𝑗} {‖ 𝑇 𝐞_{𝑗} ‖}^{2}$ does not depend on the choice of the orthonormal basis of $𝑉$ , and that it equals $tr (𝐴^{*} 𝐴)$ for the matrix $𝐴$ of $𝑇$ with respect to any orthonormal bases of $𝑉$ and $𝑊$ .

Solution.

Uses. Theorem 6.3.22 (Parseval’s identity), Definition 6.6.8 (Adjoint), Proposition 6.3.18 (Coordinates in an orthonormal basis), Proposition 6.1.16 (Frobenius inner product)

Techniques. Move an operator across the inner product (Technique 6.6.20), The trace of $𝐴^{*} 𝐴$ (Technique 6.1.18)

Hint 1. Apply Theorem 6.3.22(b) to

𝑇 𝐞_{𝑗}

in the basis

𝐟_{1}, \dots, 𝐟_{𝑚}

, and to

𝑇^{*} 𝐟_{𝑖}

in the basis

𝐞_{1}, \dots, 𝐞_{𝑛}

Hint 2. The middle sum in (a) is symmetric in the two bases. Which of the outer sums does not mention the basis of

𝑉

? For

tr (𝐴^{*} 𝐴)

, recall Proposition 6.1.16.

Sketch. By Parseval in

𝑊

{‖ 𝑇 𝐞_{𝑗} ‖}^{2} =

\sum_{𝑖} {| ⟨ 𝑇 𝐞_{𝑗}, 𝐟_{𝑖} ⟩ |}^{2}

; by Parseval in

𝑉

{‖ 𝑇^{*} 𝐟_{𝑖} ‖}^{2} =

\sum_{𝑗} {| ⟨ 𝑇^{*} 𝐟_{𝑖}, 𝐞_{𝑗} ⟩ |}^{2} =

\sum_{𝑗} {| ⟨ 𝑇 𝐞_{𝑗}, 𝐟_{𝑖} ⟩ |}^{2}

. The right side of (a) does not involve the

𝐞_{𝑗}

, so the left side does not depend on them. The entries of

𝐴

are

⟨ 𝑇 𝐞_{𝑗}, 𝐟_{𝑖} ⟩

, so the middle sum is

tr (𝐴^{*} 𝐴)

(a) By Theorem 6.3.22(b) in $𝑊$ , ${‖ 𝑇 𝐞_{𝑗} ‖}^{2} =$ $\sum_{𝑖 = 1}^{𝑚} {| ⟨ 𝑇 𝐞_{𝑗}, 𝐟_{𝑖} ⟩ |}^{2}$ ; summing over $𝑗$ gives the first equality. By Theorem 6.3.22(b) in $𝑉$ and the definition of the adjoint,

{‖ 𝑇^{*} 𝐟_{𝑖} ‖}^{2} = \sum_{𝑗 = 1}^{𝑛} {| ⟨ 𝑇^{*} 𝐟_{𝑖}, 𝐞_{𝑗} ⟩ |}^{2} = \sum_{𝑗 = 1}^{𝑛} {| ⟨ 𝐟_{𝑖}, 𝑇 𝐞_{𝑗} ⟩ |}^{2} = \sum_{𝑗 = 1}^{𝑛} {| ⟨ 𝑇 𝐞_{𝑗}, 𝐟_{𝑖} ⟩ |}^{2},

using $| 𝑧 | = | 𝑧 |$ . Summing over $𝑖$ gives the second equality.

(b) Fix the basis $𝐟_{1}, \dots, 𝐟_{𝑚}$ of $𝑊$ . By (a), $\sum_{𝑗} {‖ 𝑇 𝐞_{𝑗} ‖}^{2} =$ $\sum_{𝑖} {‖ 𝑇^{*} 𝐟_{𝑖} ‖}^{2}$ for every orthonormal basis $𝐞_{1}, \dots, 𝐞_{𝑛}$ of $𝑉$ , and the right side does not involve that basis. So the left side is the same for all of them. By Proposition 6.3.18, the $(𝑖, 𝑗)$ entry of $𝐴$ is the $𝑖$ th coordinate of $𝑇 𝐞_{𝑗}$ , namely $⟨ 𝑇 𝐞_{𝑗}, 𝐟_{𝑖} ⟩$ , so the middle sum of (a) is $\sum_{𝑖, 𝑗} {| 𝑎_{𝑖 𝑗} |}^{2} =$ $tr (𝐴^{*} 𝐴)$ (Proposition 6.1.16). Since the left side of (a) does not depend on the bases either, neither does $tr (𝐴^{*} 𝐴)$ .

The number $\sum_{𝑗} {‖ 𝑇 𝐞_{𝑗} ‖}^{2}$ is the squared Hilbert–Schmidt norm of $𝑇$ ; for $𝑉 = 𝔽^{𝑛}$ and $𝑊 = 𝔽^{𝑚}$ with dot products it is the squared Frobenius norm of the matrix. The proof never chose a convenient basis: it compared two arbitrary ones through a double sum that is symmetric in them.

Exercise 6.7.11.standard Let

𝐯_{1}, \dots, 𝐯_{𝑘}

be vectors in an inner product space

𝑉

, with Gram matrix

𝐺

(Definition 6.3.28). Show that

det 𝐺

is a nonnegative real number, and that

det 𝐺 = 0

if and only if the list is linearly dependent. Use an orthonormal basis of

span (𝐯_{1}, \dots, 𝐯_{𝑘})

, not Gram–Schmidt residuals as in Exercise 6.4.59.

Solution.

Uses. Theorem 6.4.20 (Orthonormal bases exist), Theorem 6.3.22 (Parseval’s identity), Proposition 6.3.30 (Gram matrices detect independence), Theorem 5.4.10 (Multiplicativity), Exercise 6.3.57, Theorem 5.4.7 (Invertibility and the determinant)

Techniques. Pass to orthonormal coordinates (Technique 6.7.2), Gram matrices (Technique 6.3.31)

Hint 1. If the list is dependent, Proposition 6.3.30 already says that

𝐺

is singular.

Hint 2. If it is independent,

𝑊 =

span (𝐯_{1}, \dots, 𝐯_{𝑘})

has dimension

𝑘

. Let

𝐵 \in 𝔽^{𝑘 \times 𝑘}

have columns

{[𝐯_{𝑗}]}_{𝒪︀}

for an orthonormal basis

𝒪︀

𝑊

, and show

𝐺 = 𝐵^{*} 𝐵

Sketch. If the list is dependent,

𝐺

is singular (Proposition 6.3.30) and

det 𝐺 = 0

. If it is independent, choose an orthonormal basis

𝒪︀

of its span, which has

𝑘

vectors, and let

𝐵

have columns

{[𝐯_{𝑗}]}_{𝒪︀}

. Parseval gives

𝐺 = 𝐵^{*} 𝐵

, and

𝐵

is invertible, so

det 𝐺 =

det 𝐵 det 𝐵 =

{| det 𝐵 |}^{2} >

0

If the list is dependent, then $𝐺$ is not invertible by Proposition 6.3.30, so $det 𝐺 = 0$ (Theorem 5.4.7).

Suppose the list is independent. Then it is a basis of $𝑊 =$ $span (𝐯_{1}, \dots, 𝐯_{𝑘})$ , so $dim 𝑊 = 𝑘$ , and $𝑊$ has an orthonormal basis $𝒪︀$ with $𝑘$ vectors (Theorem 6.4.20). Let $𝐵 \in 𝔽^{𝑘 \times 𝑘}$ be the matrix with columns ${[𝐯_{1}]}_{𝒪︀},$ $\dots,$ ${[𝐯_{𝑘}]}_{𝒪︀}$ . By Theorem 6.3.22, the $(𝑖, 𝑗)$ entry of $𝐵^{*} 𝐵$ is

{[𝐯_{𝑖}]}_{𝒪︀}^{*} {[𝐯_{𝑗}]}_{𝒪︀} = ⟨ 𝐯_{𝑗}, 𝐯_{𝑖} ⟩ = 𝑔_{𝑖 𝑗},

so $𝐺 = 𝐵^{*} 𝐵$ . The columns of $𝐵$ are the coordinate vectors of a basis, so they are independent and $𝐵$ is invertible, with $det 𝐵 \neq 0$ . By Theorem 5.4.10 and Exercise 6.3.57(a),

det 𝐺 = det (𝐵^{*}) det 𝐵 = det 𝐵 det 𝐵 = {| det 𝐵 |}^{2} > 0 .

So $det 𝐺 \geq 0$ always, with equality exactly for dependent lists. This is Technique 6.7.2: the list becomes the columns of a square matrix, and the Gram determinant becomes a squared modulus. The Gram–Schmidt route of Exercise 6.4.59 gives the more precise formula $det 𝐺 =$ ${‖ 𝐟_{1} ‖}^{2} \dots {‖ 𝐟_{𝑘} ‖}^{2}$ , and with it the geometric meaning, base times height, repeated.

Exercise 6.7.12.hard Let $𝑛 \geq 2$ and $𝑥 \in ℝ$ . Show that there are unit vectors $𝐯_{1}, \dots, 𝐯_{𝑛}$ in some real inner product space with $⟨ 𝐯_{𝑖}, 𝐯_{𝑗} ⟩ = 𝑥$ for all $𝑖 \neq 𝑗$ if and only if

- \frac{1}{𝑛 - 1} \leq 𝑥 \leq 1 .

Describe the configurations at the two endpoints.

Solution.

Uses. Theorem 6.2.1 (Cauchy–Schwarz inequality), Proposition 6.1.29 (Expanding a norm), Proposition 6.1.27 (Basic properties of the norm)

Techniques. A sum of nonnegative terms (Technique 6.1.9), Orthogonal decomposition (Technique 6.2.4)

Hint 1. For the lower bound, expand

{‖ 𝐯_{1} + \dots + 𝐯_{𝑛} ‖}^{2} \geq 0

Hint 2. For the construction, work in

ℝ^{𝑛}

with

𝐮 = (1 / \sqrt{𝑛}) (1, \dots, 1)

and

𝐟_{𝑖} =

𝐞_{𝑖} -

(1 / \sqrt{𝑛}) 𝐮

, which are orthogonal to

𝐮

. Try

𝐯_{𝑖} = 𝛼 𝐟_{𝑖} + 𝛽 𝐮

Sketch. Necessity:

𝑥 \leq 1

by Cauchy–Schwarz, and

0 \leq

{‖ \sum 𝐯_{𝑖} ‖}^{2} =

𝑛 +

𝑛 (𝑛 - 1) 𝑥

. Sufficiency: with

𝐟_{𝑖} =

𝐞_{𝑖} -

(1 / 𝑛) (1, \dots, 1)

⟨ 𝐟_{𝑖}, 𝐟_{𝑗} ⟩ =

𝛿_{𝑖 𝑗} -

1 / 𝑛

and

𝐟_{𝑖} ⟂ 𝐮

. Then

𝐯_{𝑖} = 𝛼 𝐟_{𝑖} + 𝛽 𝐮

has

⟨ 𝐯_{𝑖}, 𝐯_{𝑗} ⟩ =

𝛼^{2} (𝛿_{𝑖 𝑗} - 1 / 𝑛) +

𝛽^{2}

, and

𝛼^{2} = 1 - 𝑥

𝛽^{2} =

(1 + (𝑛 - 1) 𝑥) / 𝑛

work exactly when

- 1 / (𝑛 - 1) \leq 𝑥 \leq 1

Necessity. By Theorem 6.2.1, $𝑥 =$ $⟨ 𝐯_{1}, 𝐯_{2} ⟩ \leq$ $‖ 𝐯_{1} ‖ ‖ 𝐯_{2} ‖ =$ $1$ . By Proposition 6.1.29 (with all $𝑛 (𝑛 - 1)$ cross terms equal to $𝑥$ ),

0 \leq {‖ 𝐯_{1} + \dots + 𝐯_{𝑛} ‖}^{2} = \sum_{𝑖, 𝑗} ⟨ 𝐯_{𝑗}, 𝐯_{𝑖} ⟩ = 𝑛 + 𝑛 (𝑛 - 1) 𝑥,

so $𝑥 \geq - 1 / (𝑛 - 1)$ .

Sufficiency. Let $- 1 / (𝑛 - 1) \leq 𝑥 \leq 1$ , and work in $ℝ^{𝑛}$ with the dot product. Let $𝟏 = (1, \dots, 1)$ , $𝐮 = 𝟏 / \sqrt{𝑛}$ , and $𝐟_{𝑖} =$ $𝐞_{𝑖} -$ $(1 / 𝑛) 𝟏$ , the part of $𝐞_{𝑖}$ orthogonal to $𝐮$ (Technique 6.2.4). Then $⟨ 𝐟_{𝑖}, 𝐮 ⟩ =$ $(1 - 1) / \sqrt{𝑛} =$ $0$ and

⟨ 𝐟_{𝑖}, 𝐟_{𝑗} ⟩ = 𝛿_{𝑖 𝑗} - \frac{1}{𝑛} - \frac{1}{𝑛} + \frac{𝑛}{𝑛^{2}} = 𝛿_{𝑖 𝑗} - \frac{1}{𝑛} .

For real $𝛼, 𝛽$ , let $𝐯_{𝑖} = 𝛼 𝐟_{𝑖} + 𝛽 𝐮$ . Since $𝐟_{𝑖} ⟂ 𝐮$ and $‖ 𝐮 ‖ = 1$ ,

⟨ 𝐯_{𝑖}, 𝐯_{𝑗} ⟩ = 𝛼^{2} (𝛿_{𝑖 𝑗} - \frac{1}{𝑛}) + 𝛽^{2} .

We need $𝛼^{2} (1 - 1 / 𝑛) + 𝛽^{2} = 1$ and $- 𝛼^{2} / 𝑛 + 𝛽^{2} = 𝑥$ . Subtracting, $𝛼^{2} = 1 - 𝑥$ , and then $𝛽^{2} =$ $𝑥 +$ $(1 - 𝑥) / 𝑛 =$ $(1 + (𝑛 - 1) 𝑥) / 𝑛$ . Both right sides are nonnegative exactly when $- 1 / (𝑛 - 1) \leq 𝑥 \leq 1$ , so we can take $𝛼 = \sqrt{1 - 𝑥}$ and $𝛽 =$ $\sqrt{(1 + (𝑛 - 1) 𝑥) / 𝑛}$ .

Endpoints. At $𝑥 = 1$ , equality in Cauchy–Schwarz for unit vectors with $⟨ 𝐯_{𝑖}, 𝐯_{𝑗} ⟩ = 1$ forces $𝐯_{𝑖} = 𝐯_{𝑗}$ : all the vectors coincide. At $𝑥 = - 1 / (𝑛 - 1)$ , the expansion above gives $‖ 𝐯_{1} + \dots + 𝐯_{𝑛} ‖ = 0$ , so the vectors sum to $𝟎$ ; in the construction $𝛽 = 0$ , and the vectors $𝐯_{𝑖} =$ $\sqrt{𝑛 / (𝑛 - 1)} 𝐟_{𝑖}$ point from the center of the regular simplex with vertices $𝐞_{1}, \dots, 𝐞_{𝑛}$ to its vertices. For $𝑛 = 3$ these are three unit vectors in a plane at angles $2 𝜋 / 3$ . Every list attaining the lower endpoint has the same Gram matrix as this one, so by Problem 6.9 it is this configuration up to an isometry of spans.

The Gram matrix of the list is $(1 - 𝑥) 𝐼_{𝑛} + 𝑥 𝐽$ , where $𝐽$ has every entry $1$ . The exercise therefore says for which $𝑥$ this matrix is a Gram matrix; Problem 6.17 answers that question for every Hermitian matrix.

Exercise 6.7.13.standard Find the error in the following argument, and then prove the claim correctly. (The claim is true; it is part of Exercise 6.4.57.)

Claim. Let $𝑉$ be a finite-dimensional inner product space, and let $𝑃 \in L (𝑉)$ satisfy $𝑃^{2} = 𝑃$ and $‖ 𝑃 𝐯 ‖ \leq ‖ 𝐯 ‖$ for every $𝐯 \in 𝑉$ . Then $𝑃 = {Pr}_{im 𝑃}$ .

Proof. Let $𝐯 \in 𝑉$ . Since $𝑃 (𝐯 - 𝑃 𝐯) =$ $𝑃 𝐯 -$ $𝑃^{2} 𝐯 =$ $𝟎$ , we have $𝐯 = 𝑃 𝐯 + (𝐯 - 𝑃 𝐯)$ with $𝑃 𝐯 \in im 𝑃$ and $𝐯 - 𝑃 𝐯 \in ker 𝑃$ . By the Pythagorean theorem, ${‖ 𝐯 ‖}^{2} =$ ${‖ 𝑃 𝐯 ‖}^{2} +$ ${‖ 𝐯 - 𝑃 𝐯 ‖}^{2}$ , so the two pieces are orthogonal. Hence $𝐯 - 𝑃 𝐯 ⟂ im 𝑃$ , and $𝑃 𝐯 = {Pr}_{im 𝑃} (𝐯)$ by Theorem 6.4.6.

Solution.

Uses. Theorem 6.3.3 (Pythagorean theorem), Theorem 6.4.6 (The projection is characterized by orthogonality), Exercise 6.7.6

Techniques. Recognize a projection by its two conditions (Technique 6.4.7), Perturb a minimizer (Technique 6.5.6)

Hint 1. Which hypothesis does the argument never use? What does the Pythagorean theorem require of the two pieces?

Hint 2. For a correct proof, take

𝐮 \in im 𝑃

and

𝐰 \in ker 𝑃

, and compute

𝑃 (𝐮 + 𝑡 𝐰)

. Then use Exercise 6.7.6.

Sketch. The Pythagorean theorem needs

𝑃 𝐯 ⟂ 𝐯 - 𝑃 𝐯

, which is what has to be proved; the argument never uses

‖ 𝑃 𝐯 ‖ \leq ‖ 𝐯 ‖

, and the oblique projection

(\begin{matrix} 1 & 1 \\ 0 & 0 \end{matrix})

satisfies everything else. Correct proof: for

𝐮 \in im 𝑃

𝐰 \in ker 𝑃

𝑃 (𝐮 + 𝑡 𝐰) = 𝐮

, so

‖ 𝐮 ‖ \leq ‖ 𝐮 + 𝑡 𝐰 ‖

for all

𝑡

, and

𝐮 ⟂ 𝐰

by Exercise 6.7.6. Then

𝐯 - 𝑃 𝐯 ⟂ im 𝑃

and Theorem 6.4.6 finishes.

The error. The Pythagorean theorem (Theorem 6.3.3) applies to orthogonal vectors; its converse, which the argument uses in the form “the norms add up, so the pieces are orthogonal”, needs the equation ${‖ 𝐯 ‖}^{2} =$ ${‖ 𝑃 𝐯 ‖}^{2} +$ ${‖ 𝐯 - 𝑃 𝐯 ‖}^{2}$ , which nobody has shown. The argument assumes the conclusion. A sign of this is that the hypothesis $‖ 𝑃 𝐯 ‖ \leq ‖ 𝐯 ‖$ is never used. Every step except the Pythagorean one holds for the oblique projection $𝑃 = (\begin{matrix} 1 & 1 \\ 0 & 0 \end{matrix})$ on $ℝ^{2}$ , which has $𝑃^{2} = 𝑃$ , $ker 𝑃 =$ $span ((1, - 1))$ , and $im 𝑃 = span (𝐞_{1})$ , and for which the pieces of $𝐯 = (0, 1)$ are $(1, 0)$ and $(- 1, 1)$ , not orthogonal. (Consistently, this $𝑃$ violates the norm hypothesis: $𝑃 (1, 1) = (2, 0)$ is longer than $(1, 1)$ .) Over $ℂ$ there is a second gap: even if the norms did add up, the converse of Pythagoras fails there (Theorem 6.3.3(b) is stated for $𝔽 = ℝ$ only), so orthogonality would still not follow.

A correct proof. Let $𝐮 \in im 𝑃$ and $𝐰 \in ker 𝑃$ . Since $𝐮 = 𝑃 𝐱$ for some $𝐱$ , $𝑃 𝐮 = 𝑃^{2} 𝐱 = 𝐮$ , so for every $𝑡 \in 𝔽$ , $𝑃 (𝐮 + 𝑡 𝐰) =$ $𝐮 +$ $𝑡 𝑃 𝐰 =$ $𝐮$ . The norm hypothesis applied to $𝐮 + 𝑡 𝐰$ gives

‖ 𝐮 ‖ = ‖ 𝑃 (𝐮 + 𝑡 𝐰) ‖ \leq ‖ 𝐮 + 𝑡 𝐰 ‖ for all 𝑡 \in 𝔽,

so $⟨ 𝐮, 𝐰 ⟩ = 0$ by Exercise 6.7.6. Thus every vector of $ker 𝑃$ is orthogonal to $im 𝑃$ . Now for $𝐯 \in 𝑉$ , $𝑃 𝐯 \in im 𝑃$ and $𝐯 -$ $𝑃 𝐯 \in$ $ker 𝑃 \subseteq$ ${(im 𝑃)}^{⟂}$ , so $𝑃 𝐯 = {Pr}_{im 𝑃} (𝐯)$ by Theorem 6.4.6.

Exercise 6.7.14.standard Find the error in the following argument, give an operator for which its first displayed equation fails, and prove the claim correctly.

Claim. Let $𝑉$ be a finite-dimensional inner product space and $𝑇 \in L (𝑉)$ with $‖ 𝑇 𝐯 ‖ \leq ‖ 𝐯 ‖$ for all $𝐯$ . Then $‖ 𝑇^{*} 𝐯 ‖ \leq ‖ 𝐯 ‖$ for all $𝐯$ .

Proof. For every $𝐯 \in 𝑉$ ,

{‖ 𝑇^{*} 𝐯 ‖}^{2} = ⟨ 𝑇 𝑇^{*} 𝐯, 𝐯 ⟩ = ⟨ 𝑇^{*} 𝑇 𝐯, 𝐯 ⟩ = {‖ 𝑇 𝐯 ‖}^{2},

and ${‖ 𝑇 𝐯 ‖}^{2} \leq {‖ 𝐯 ‖}^{2}$ .

Solution.

Uses. Definition 6.6.8 (Adjoint), Theorem 6.2.1 (Cauchy–Schwarz inequality), Exercise 6.7.8

Techniques. Move an operator across the inner product (Technique 6.6.20)

Hint 1. Which step replaces one operator by another? When is that legitimate?

Hint 2. For a correct proof, start from

{‖ 𝑇^{*} 𝐯 ‖}^{2} =

⟨ 𝑇 𝑇^{*} 𝐯, 𝐯 ⟩

and apply Cauchy–Schwarz, then the hypothesis to the vector

𝑇^{*} 𝐯

Sketch. The step

𝑇 𝑇^{*} = 𝑇^{*} 𝑇

assumes that

𝑇

is normal; the argument would prove

‖ 𝑇^{*} 𝐯 ‖ = ‖ 𝑇 𝐯 ‖

for every

𝑇

, which fails for

𝑇 = (\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix})

𝐯 = 𝐞_{2}

. Correct proof:

{‖ 𝑇^{*} 𝐯 ‖}^{2} =

⟨ 𝑇 (𝑇^{*} 𝐯), 𝐯 ⟩ \leq

‖ 𝑇 𝑇^{*} 𝐯 ‖ ‖ 𝐯 ‖ \leq

‖ 𝑇^{*} 𝐯 ‖ ‖ 𝐯 ‖

, and divide by

‖ 𝑇^{*} 𝐯 ‖

when it is nonzero.

The error. The middle equality replaces $𝑇 𝑇^{*}$ by $𝑇^{*} 𝑇$ . These are equal only when $𝑇$ is normal, and the argument, which never uses the hypothesis on $𝑇$ , would prove $‖ 𝑇^{*} 𝐯 ‖ = ‖ 𝑇 𝐯 ‖$ for every operator. By Exercise 6.7.8 that holds only for normal $𝑇$ . For $𝑇 = (\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix})$ on $ℝ^{2}$ with the dot product, $𝑇^{*} = (\begin{matrix} 0 & 0 \\ 1 & 0 \end{matrix})$ , and at $𝐯 = 𝐞_{2}$ : ${‖ 𝑇^{*} 𝐞_{2} ‖}^{2} = 0$ while ${‖ 𝑇 𝐞_{2} ‖}^{2} = {‖ 𝐞_{1} ‖}^{2} = 1$ .

A correct proof. Let $𝐯 \in 𝑉$ . Moving $𝑇$ across the inner product, then using Cauchy–Schwarz (Theorem 6.2.1) and the hypothesis applied to $𝑇^{*} 𝐯$ ,

{‖ 𝑇^{*} 𝐯 ‖}^{2} = ⟨ 𝑇 𝑇^{*} 𝐯, 𝐯 ⟩ \leq ‖ 𝑇 (𝑇^{*} 𝐯) ‖ ‖ 𝐯 ‖ \leq ‖ 𝑇^{*} 𝐯 ‖ ‖ 𝐯 ‖ .

If $𝑇^{*} 𝐯 = 𝟎$ the claim is clear; otherwise divide by $‖ 𝑇^{*} 𝐯 ‖ > 0$ . So $‖ 𝑇^{*} 𝐯 ‖ \leq ‖ 𝐯 ‖$ .

In terms of the operator norm of Section 9.2, the claim says $‖ 𝑇^{*} ‖ \leq ‖ 𝑇 ‖$ , and applying it to $𝑇^{*}$ gives equality.

Exercise 6.7.15.routine For each statement, decide whether it holds in every real inner product space, and whether it holds in every complex one. Here $𝐱, 𝐲, 𝐯 \in 𝑉$ and $𝑇 \in L (𝑉)$ , with $𝑉$ finite-dimensional.

(a)If $⟨ 𝑇 𝐯, 𝐯 ⟩ = 0$ for every $𝐯$ , then $𝑇 = 0$ .
(b)If ${‖ 𝐱 + 𝐲 ‖}^{2} =$ ${‖ 𝐱 ‖}^{2} +$ ${‖ 𝐲 ‖}^{2}$ , then $𝐱 ⟂ 𝐲$ .
(c) $⟨ 𝐱, 𝐲 ⟩ =$ $\frac{1}{4} ({‖ 𝐱 + 𝐲 ‖}^{2} - {‖ 𝐱 - 𝐲 ‖}^{2})$ .
(d)If $⟨ 𝑇 𝐯, 𝐯 ⟩ \in ℝ$ for every $𝐯$ , then $𝑇$ is self-adjoint.

Solution.

Uses. Proposition 6.1.41 (Over $ℂ$ , the values $⟨ 𝑇 𝐯, 𝐯 ⟩$ determine $𝑇$ ), Example 6.1.39 (A rotation), Theorem 6.3.3 (Pythagorean theorem), Proposition 6.1.32 (Polarization identities), Proposition 6.6.30 (Self-adjoint means real values)

Techniques. Finding counterexamples and minimal examples (Technique 1.4.18), Plug in clever test vectors (Technique 6.1.40)

Hint 1. Each of the four holds over exactly one of

ℝ

and

ℂ

. Test the complex failures on

𝑉 = ℂ

with

𝐱 = 1

and

𝐲 = 𝑖

Hint 2. For the real failures, use the rotation

𝐽 = (\begin{matrix} 0 & - 1 \\ 1 & 0 \end{matrix})

Sketch. (a) True over

ℂ

(Proposition 6.1.41), false over

ℝ

(rotation by

𝜋 / 2

). (b) True over

ℝ

, false over

ℂ

{| 1 + 𝑖 |}^{2} = 2 = 1 + 1

but

⟨ 1, 𝑖 ⟩ = - 𝑖

. (c) True over

ℝ

; over

ℂ

the right side is

Re ⟨ 𝐱, 𝐲 ⟩

, so it fails for

1, 𝑖

. (d) True over

ℂ

(Proposition 6.6.30); over

ℝ

the hypothesis holds for every

𝑇

, and the rotation is not self-adjoint.

(a) Over $ℂ$ this is Proposition 6.1.41. Over $ℝ$ it is false: the rotation $𝑇 𝐯 = 𝐽 𝐯$ with $𝐽 = (\begin{matrix} 0 & - 1 \\ 1 & 0 \end{matrix})$ has $⟨ 𝑇 𝐯, 𝐯 ⟩ = 0$ for every $𝐯 \in ℝ^{2}$ (Example 6.1.39), and $𝑇 \neq 0$ .

(b) Over $ℝ$ this is Theorem 6.3.3(b). Over $ℂ$ it is false: in $𝑉 = ℂ$ with $⟨ 𝑧, 𝑤 ⟩ = 𝑧 𝑤$ , ${| 1 + 𝑖 |}^{2} =$ $2 =$ ${| 1 |}^{2} +$ ${| 𝑖 |}^{2}$ , but $⟨ 1, 𝑖 ⟩ = - 𝑖 \neq 0$ .

(c) Over $ℝ$ this is Proposition 6.1.32(a). Over $ℂ$ , Proposition 6.1.29 shows that the right side equals $Re ⟨ 𝐱, 𝐲 ⟩$ , so the formula fails whenever $⟨ 𝐱, 𝐲 ⟩$ is not real: for $𝐱 = 1$ and $𝐲 = 𝑖$ in $ℂ$ the right side is $(2 - 2) / 4 = 0$ while $⟨ 1, 𝑖 ⟩ = - 𝑖$ .

(d) Over $ℂ$ this is Proposition 6.6.30. Over $ℝ$ , every $⟨ 𝑇 𝐯, 𝐯 ⟩$ is real, so the hypothesis holds for every operator, and the rotation $𝑇$ of (a) is not self-adjoint: $⟨ 𝑇 𝐞_{1}, 𝐞_{2} ⟩ = 1$ while $⟨ 𝐞_{1}, 𝑇 𝐞_{2} ⟩ = - 1$ .

Statements (a) and (d) hold over $ℂ$ because a complex inner product can separate real and imaginary parts with the test vector $𝐱 + 𝑖 𝐲$ ; statements (b) and (c) hold over $ℝ$ because there the cross term $2 Re ⟨ 𝐱, 𝐲 ⟩$ is all of $2 ⟨ 𝐱, 𝐲 ⟩$ .