8.2 The spectral theorems · Matrix Theory

An operator is easiest to understand when it is diagonal in some basis. On an inner product space we want more: a basis that is diagonal for $𝑇$ and orthonormal, so that $𝑇$ stretches $𝑉$ by scalar factors along mutually perpendicular axes, and coordinates are computed by inner products instead of linear systems. This section answers the question of which operators have such a basis. Over $ℂ$ the answer is the normal operators (Definition 8.1.8), and over $ℝ$ it is the self-adjoint ones. Both answers come from the same short computation, which shows that these conditions are necessary; the work is in showing that they suffice.

Throughout, $𝑉$ is a finite-dimensional inner product space over $𝔽$ , with $dim 𝑉 \geq 1$ , and $𝑇 \in L (𝑉)$ . As in Section 6.6, inner products are linear in the first slot. For an orthonormal basis $𝒪︀$ we write ${[𝑇]}_{𝒪︀}$ for ${[𝑇]}_{𝒪︀ 𝒪︀}$ , and we use freely that ${[𝑇^{*}]}_{𝒪︀} =$ ${({[𝑇]}_{𝒪︀})}^{*}$ (Theorem 6.6.15) and ${[𝑆 𝑇]}_{𝒪︀} =$ ${[𝑆]}_{𝒪︀} {[𝑇]}_{𝒪︀}$ (Theorem 2.4.2).

8.2.1Orthonormal eigenbases

An orthonormal eigenbasis for $𝑇$ is an orthonormal basis of $𝑉$ consisting of eigenvectors of $𝑇$ . By Theorem 7.4.2, an orthonormal basis $𝒪︀$ is an eigenbasis exactly when ${[𝑇]}_{𝒪︀}$ is diagonal. For matrices, the change-of-basis matrix is then unitary.

Definition 8.2.1 (Unitarily and orthogonally diagonalizable). A matrix $𝐴 \in 𝔽^{𝑛 \times 𝑛}$ is unitarily diagonalizable if

𝐴 = 𝑈 𝐷 𝑈^{*}

for some unitary $𝑈 \in 𝔽^{𝑛 \times 𝑛}$ and some diagonal $𝐷 \in 𝔽^{𝑛 \times 𝑛}$ . When $𝔽 = ℝ$ , $𝑈$ is then an orthogonal matrix $𝑄$ and $𝐴 = 𝑄 𝐷 𝑄^{𝖳}$ ; we say $𝐴$ is orthogonally diagonalizable.

Since $𝑈^{*} = 𝑈^{- 1}$ for unitary $𝑈$ (Definition 6.3.33), $𝐴 = 𝑈 𝐷 𝑈^{*}$ is a diagonalization $𝐴 = 𝑆 𝐷 𝑆^{- 1}$ with $𝑆 = 𝑈$ . By the last part of Theorem 7.4.2, it says that the columns of $𝑈$ are eigenvectors of $𝐴$ with eigenvalues the diagonal entries of $𝐷$ ; and by Theorem 6.3.29, $𝑈$ is unitary exactly when its columns are orthonormal. So $𝐴$ is unitarily diagonalizable if and only if $𝔽^{𝑛}$ , with the dot product, has an orthonormal eigenbasis for $𝐴$ .

We start with the easy direction of both spectral theorems, because its proof shows where the conditions “normal” and “self-adjoint” come from.

Proposition 8.2.2 (Necessary conditions). Suppose $𝑉$ has an orthonormal eigenbasis for $𝑇$ .

(a) $𝑇$ is normal: $𝑇^{*} 𝑇 = 𝑇 𝑇^{*}$ .
(b)If every eigenvalue of $𝑇$ is real (in particular, if $𝔽 = ℝ$ ), then $𝑇$ is self-adjoint.

Proof.routine

Uses. Theorem 7.4.2 (Diagonalizable means an eigenbasis), Theorem 6.6.15 (Existence of adjoints), Theorem 2.4.2 (The matrix of a composite), Theorem 2.3.20 (Linear maps correspond to matrices), Definition 8.1.8 (Normal operator, normal matrix), Definition 6.6.26 (Self-adjoint operator)

Techniques. Choose a convenient basis (Technique 4.5.33)

Hint 1. Write down

{[𝑇]}_{𝒪︀}

for the orthonormal eigenbasis

𝒪︀

. What is

{[𝑇^{*}]}_{𝒪︀}

Hint 2. Diagonal matrices commute with each other, and

𝐷^{*} = 𝐷

for a real diagonal

𝐷

Sketch. In the eigenbasis

𝒪︀

{[𝑇]}_{𝒪︀} = 𝐷

is diagonal, so

{[𝑇^{*}]}_{𝒪︀} = 𝐷^{*}

is diagonal too. Diagonal matrices commute, so

{[𝑇^{*} 𝑇]}_{𝒪︀} =

𝐷^{*} 𝐷 =

𝐷 𝐷^{*} =

{[𝑇 𝑇^{*}]}_{𝒪︀}

. If the eigenvalues are real,

𝐷^{*} = 𝐷

, so

𝑇^{*} = 𝑇

Let $𝒪︀$ be an orthonormal eigenbasis, with $𝑇 𝐮_{𝑗} = 𝜆_{𝑗} 𝐮_{𝑗}$ . By Theorem 7.4.2, ${[𝑇]}_{𝒪︀} =$ $𝐷 =$ $diag (𝜆_{1}, \dots, 𝜆_{𝑛})$ , and by Theorem 6.6.15, ${[𝑇^{*}]}_{𝒪︀} =$ $𝐷^{*} =$ $diag (𝜆_{1}, \dots, 𝜆_{𝑛})$ .

(a) Products of diagonal matrices are computed entry by entry, so $𝐷^{*} 𝐷 =$ $diag ({| 𝜆_{1} |}^{2}, \dots, {| 𝜆_{𝑛} |}^{2}) =$ $𝐷 𝐷^{*}$ . By Theorem 2.4.2, ${[𝑇^{*} 𝑇]}_{𝒪︀} =$ $𝐷^{*} 𝐷 =$ $𝐷 𝐷^{*} =$ ${[𝑇 𝑇^{*}]}_{𝒪︀}$ , and maps with the same matrix are equal (Theorem 2.3.20). So $𝑇^{*} 𝑇 = 𝑇 𝑇^{*}$ .

(b) If every $𝜆_{𝑗}$ is real, then $𝐷^{*} = 𝐷$ , so ${[𝑇^{*}]}_{𝒪︀} = {[𝑇]}_{𝒪︀}$ and $𝑇^{*} = 𝑇$ by Theorem 2.3.20. □

So normality is what remains of “diagonal in an orthonormal basis” once the basis is forgotten: a diagonal matrix commutes with its conjugate transpose. The spectral theorems say that nothing else is needed. Before proving them, here are the examples that show which hypothesis goes with which field.

Example 8.2.3 (Which matrices have orthonormal eigenbases).

(a) $𝐴 = (\begin{matrix} 1 & 1 \\ 0 & 2 \end{matrix}) \in ℝ^{2 \times 2}$ has the distinct eigenvalues $1$ and $2$ , so it is diagonalizable (Corollary 7.4.5), with eigenvectors $(1, 0)$ and $(1, 1)$ . These are not orthogonal, and each eigenspace is a line, so no choice of eigenvectors is orthogonal. Correspondingly $𝐴$ is not normal: ${(𝐴^{*} 𝐴)}_{11} = 1$ while ${(𝐴 𝐴^{*})}_{11} = 2$ . Diagonalizable and unitarily diagonalizable are different conditions.
(b)The rotation $𝐽 = (\begin{matrix} 0 & - 1 \\ 1 & 0 \end{matrix})$ satisfies $𝐽^{𝖳} 𝐽 = 𝐼_{2} = 𝐽 𝐽^{𝖳}$ , so $𝑇_{𝐽}$ is normal on $ℝ^{2}$ and on $ℂ^{2}$ . Over $ℝ$ it has no eigenvector at all (Example 7.2.18). Over $ℂ$ ,
$𝐽 (\begin{matrix} 1 \\ - 𝑖 \end{matrix}) = (\begin{matrix} 𝑖 \\ 1 \end{matrix}) = 𝑖 (\begin{matrix} 1 \\ - 𝑖 \end{matrix}), 𝐽 (\begin{matrix} 1 \\ 𝑖 \end{matrix}) = (\begin{matrix} - 𝑖 \\ 1 \end{matrix}) = - 𝑖 (\begin{matrix} 1 \\ 𝑖 \end{matrix}),$
and $\frac{1}{\sqrt{2}} (1, - 𝑖)$ , $\frac{1}{\sqrt{2}} (1, 𝑖)$ is an orthonormal eigenbasis of $ℂ^{2}$ , since $(1, - 𝑖) \cdot$ $(1, 𝑖) =$ $1 +$ $(- 𝑖) 𝑖 =$ $1 -$ $1 =$ $0$ . So over $ℂ$ normality is the right condition, while over $ℝ$ we will need the stronger condition $𝑇^{*} = 𝑇$ .

Intuition (Two ways to build the basis). Suppose we want an orthonormal eigenbasis for a normal $𝑇$ on a complex space. There are two natural plans, and both work.

The first plan starts from something we already have. Schur’s theorem (Theorem 7.5.18) gives an orthonormal basis in which $𝑇$ is upper triangular, so it remains to show that normality kills everything above the diagonal. For an upper-triangular $𝑅$ , the first column has only one nonzero entry, $𝑟_{11}$ , while the first row is $𝑟_{11}, 𝑟_{12}, \dots, 𝑟_{1 𝑛}$ . The diagonal entries of $𝑅^{*} 𝑅$ and $𝑅 𝑅^{*}$ are the squared norms of the columns and of the rows, so $𝑅^{*} 𝑅 = 𝑅 𝑅^{*}$ forces the first row to have no more mass than the first column. Then the same comparison works for the second row, and so on down the diagonal.

The second plan builds the basis one vector at a time (Technique 7.5.12). Take a unit eigenvector $𝐮$ and try to recurse on $𝑊 = {span (𝐮)}^{⟂}$ . In the second proof of Schur’s theorem this required compressing $𝑇$ to $𝑊$ , because $𝑊$ need not be $𝑇$ -invariant, and the compression produced the entries above the diagonal. For a normal $𝑇$ , $𝐮$ is also an eigenvector of $𝑇^{*}$ , and this is exactly what makes $𝑊$ invariant (Exercise 6.6.38). The compression is then a restriction, and nothing appears above the diagonal.

8.2.2The complex spectral theorem

We start with the matrix fact behind the first plan.

Lemma 8.2.4 (Triangular and normal means diagonal). Let

𝑅 \in 𝔽^{𝑛 \times 𝑛}

be upper triangular with

𝑅^{*} 𝑅 = 𝑅 𝑅^{*}

. Then

𝑅

is diagonal.

Proof.standard

Uses. Definition 2.4.1 (Matrix product), Definition 6.3.26 (Conjugate transpose)

Techniques. A sum of nonnegative terms (Technique 6.1.9), Strong induction and well-ordering (Technique 5.2.12)

Hint 1. Compute the diagonal entries

{(𝑅^{*} 𝑅)}_{𝑘 𝑘}

and

{(𝑅 𝑅^{*})}_{𝑘 𝑘}

as sums of

{| 𝑟_{𝑖 𝑗} |}^{2}

. Which entries of

𝑅

appear in each?

Hint 2.

{(𝑅^{*} 𝑅)}_{𝑘 𝑘}

is the squared norm of column

𝑘

and

{(𝑅 𝑅^{*})}_{𝑘 𝑘}

that of row

𝑘

. Start with

𝑘 = 1

, where the column has a single possibly nonzero entry.

Hint 3. Prove by strong induction on

𝑘

that row

𝑘

has no nonzero entries to the right of the diagonal. When you reach row

𝑘

, the entries of column

𝑘

above the diagonal lie in earlier rows.

Sketch.

{(𝑅^{*} 𝑅)}_{𝑘 𝑘} =

\sum_{𝑖} {| 𝑟_{𝑖 𝑘} |}^{2}

is the squared norm of column

𝑘

, and

{(𝑅 𝑅^{*})}_{𝑘 𝑘} =

\sum_{𝑗} {| 𝑟_{𝑘 𝑗} |}^{2}

that of row

𝑘

. By strong induction on

𝑘

: if rows

1, \dots, 𝑘 - 1

vanish off the diagonal, then column

𝑘

has only the entry

𝑟_{𝑘 𝑘}

, so equating the two norms gives

\sum_{𝑗 > 𝑘} {| 𝑟_{𝑘 𝑗} |}^{2} = 0

, and row

𝑘

vanishes off the diagonal too.

For each $𝑘$ , the definitions of the product and of $𝑅^{*}$ give

{(𝑅^{*} 𝑅)}_{𝑘 𝑘} = \sum_{𝑖 = 1}^{𝑛} 𝑟_{𝑖 𝑘} 𝑟_{𝑖 𝑘} = \sum_{𝑖 = 1}^{𝑛} {| 𝑟_{𝑖 𝑘} |}^{2}, {(𝑅 𝑅^{*})}_{𝑘 𝑘} = \sum_{𝑗 = 1}^{𝑛} 𝑟_{𝑘 𝑗} 𝑟_{𝑘 𝑗} = \sum_{𝑗 = 1}^{𝑛} {| 𝑟_{𝑘 𝑗} |}^{2},

the squared norms of column $𝑘$ and of row $𝑘$ . We prove by strong induction on $𝑘$ that $𝑟_{𝑘 𝑗} = 0$ for all $𝑗 > 𝑘$ . Let $1 \leq 𝑘 \leq 𝑛$ and suppose this holds for every row $𝑖 < 𝑘$ . Then $𝑟_{𝑖 𝑘} = 0$ for $𝑖 < 𝑘$ by the hypothesis (as $𝑘 > 𝑖$ ), and $𝑟_{𝑖 𝑘} = 0$ for $𝑖 > 𝑘$ because $𝑅$ is upper triangular. So column $𝑘$ has the single entry $𝑟_{𝑘 𝑘}$ , and ${(𝑅^{*} 𝑅)}_{𝑘 𝑘} =$ ${(𝑅 𝑅^{*})}_{𝑘 𝑘}$ reads

{| 𝑟_{𝑘 𝑘} |}^{2} = {| 𝑟_{𝑘 𝑘} |}^{2} + \sum_{𝑗 > 𝑘} {| 𝑟_{𝑘 𝑗} |}^{2} .

Hence $\sum_{𝑗 > 𝑘} {| 𝑟_{𝑘 𝑗} |}^{2} = 0$ . A sum of nonnegative reals vanishes only if every term does, so $𝑟_{𝑘 𝑗} = 0$ for all $𝑗 > 𝑘$ . (For $𝑘 = 1$ the hypothesis is empty and the same argument applies.) So every entry above the diagonal is $0$ , and $𝑅$ is diagonal. □

Theorem 8.2.5 (Complex spectral theorem). Let $𝑉$ be a finite-dimensional complex inner product space with $dim 𝑉 \geq 1$ , and let $𝑇 \in L (𝑉)$ . The following are equivalent:

(I) $𝑇$ is normal;
(II) $𝑉$ has an orthonormal basis consisting of eigenvectors of $𝑇$ ;
(III) ${[𝑇]}_{𝒪︀}$ is diagonal for some orthonormal basis $𝒪︀$ of $𝑉$ .

In matrix form: $𝐴 \in ℂ^{𝑛 \times 𝑛}$ satisfies $𝐴^{*} 𝐴 = 𝐴 𝐴^{*}$ if and only if $𝐴$ is unitarily diagonalizable.

(II) and (III) say the same thing by Theorem 7.4.2, and (II) $\Rightarrow$ (I) is Proposition 8.2.2(a). The content is (I) $\Rightarrow$ (III). We give three proofs. The first two carry out the two plans of Intuition (Two ways to build the basis); the third avoids induction altogether.

Proof (Schur, then normality).routine

Uses. Theorem 7.5.18 (Schur’s theorem), Lemma 8.2.4 (Triangular and normal means diagonal), Theorem 6.6.15 (Existence of adjoints), Theorem 2.4.2 (The matrix of a composite), Theorem 7.4.2 (Diagonalizable means an eigenbasis), Proposition 8.2.2 (Necessary conditions), Theorem 4.5.17 (Change of basis), Definition 6.3.33 (Unitary and orthogonal matrices)

Techniques. Proving TFAE (Technique 2.7.1), Prove it for triangular matrices first (Technique 7.5.23)

Hint 1. Start from a Schur basis: an orthonormal

𝒪︀

with

𝑅 = {[𝑇]}_{𝒪︀}

upper triangular. What does normality of

𝑇

say about

𝑅

Hint 2. Since

𝒪︀

is orthonormal,

{[𝑇^{*}]}_{𝒪︀} = 𝑅^{*}

. Apply Lemma 8.2.4.

Sketch. (II)

\Leftrightarrow

(III) is Theorem 7.4.2, and (II)

\Rightarrow

(I) is Proposition 8.2.2. For (I)

\Rightarrow

(III), Schur gives an orthonormal

𝒪︀

with

𝑅 = {[𝑇]}_{𝒪︀}

upper triangular, and

𝑅^{*} 𝑅 =

{[𝑇^{*} 𝑇]}_{𝒪︀} =

{[𝑇 𝑇^{*}]}_{𝒪︀} =

𝑅 𝑅^{*}

. By Lemma 8.2.4,

𝑅

is diagonal. For matrices, apply this to

𝑇_{𝐴}

ℂ^{𝑛}

and use that the columns of an orthonormal basis form a unitary matrix.

(II) and (III) are equivalent by Theorem 7.4.2, and (II) $\Rightarrow$ (I) is Proposition 8.2.2(a). Suppose $𝑇$ is normal. By Theorem 7.5.18 there is an orthonormal basis $𝒪︀$ with $𝑅 = {[𝑇]}_{𝒪︀}$ upper triangular. By Theorem 6.6.15 and Theorem 2.4.2,

𝑅^{*} 𝑅 = {[𝑇^{*}]}_{𝒪︀} {[𝑇]}_{𝒪︀} = {[𝑇^{*} 𝑇]}_{𝒪︀} = {[𝑇 𝑇^{*}]}_{𝒪︀} = 𝑅 𝑅^{*} .

By Lemma 8.2.4, $𝑅$ is diagonal, which is (III).

For the matrix form, if $𝐴 = 𝑈 𝐷 𝑈^{*}$ with $𝑈$ unitary and $𝐷$ diagonal, then $𝐴^{*} 𝐴 =$ $𝑈 𝐷^{*} 𝐷 𝑈^{*} =$ $𝑈 𝐷 𝐷^{*} 𝑈^{*} =$ $𝐴 𝐴^{*}$ . Conversely, if $𝐴^{*} 𝐴 = 𝐴 𝐴^{*}$ , then $𝑇_{𝐴}$ is normal on $ℂ^{𝑛}$ with the dot product (Example 6.6.10), so by (II) there is an orthonormal eigenbasis $𝐮_{1}, \dots, 𝐮_{𝑛}$ of $ℂ^{𝑛}$ . The matrix $𝑈$ with these columns is unitary (Definition 6.3.33), and by Theorem 4.5.17, $𝑈^{*} 𝐴 𝑈 =$ $𝑈^{- 1} 𝐴 𝑈 =$ ${[𝑇_{𝐴}]}_{𝒪︀} =$ $𝐷$ is diagonal. Hence $𝐴 = 𝑈 𝐷 𝑈^{*}$ . □

Intuition (What the second proof needs from normality). The induction runs on $𝑊 = {span (𝐮)}^{⟂}$ , and two facts about normal operators make it work. Both have one-line proofs, given inside the proof below; Section 8.1 treats them properly.

$𝑇$ and $𝑇^{*}$ stretch every vector equally: $‖ 𝑇 𝐯 ‖ = ‖ 𝑇^{*} 𝐯 ‖$ for every $𝐯$ . Applied to the normal operator $𝑇 - 𝜆 id$ , this says that $𝑇 𝐮 = 𝜆 𝐮$ implies $𝑇^{*} 𝐮 = 𝜆 𝐮$ .
Restrictions stay normal: if $𝑊$ is invariant under both $𝑇$ and $𝑇^{*}$ , then the adjoint of $𝑇 ↾_{𝑊}$ is $𝑇^{*} ↾_{𝑊}$ .

Proof (induction with invariant complements).standard

Uses. Theorem 7.3.27 (Existence of eigenvalues over $ℂ$ ), Definition 8.1.8 (Normal operator, normal matrix), Definition 6.6.8 (Adjoint), Proposition 6.6.18 (Properties of the adjoint), Proposition 6.6.9 (Adjoints are unique and linear), Exercise 6.6.38, Theorem 6.4.24 (Orthogonal decomposition), Corollary 6.4.25 (Dimension of the orthogonal complement), Theorem 4.4.13 (Bases and dimension of a sum), Definition 6.3.15 (Orthonormal list, orthonormal basis)

Techniques. Induction on dimension via an invariant subspace (Technique 7.5.12), Move an operator across the inner product (Technique 6.6.20), Show a vector is zero via its norm (Technique 6.1.36)

Hint 1. Induct on

𝑛 = dim 𝑉

. Take a unit eigenvector

𝐮

with

𝑇 𝐮 = 𝜆 𝐮

, and let

𝑊 = {span (𝐮)}^{⟂}

. You want

𝑊

to be

𝑇

-invariant. By Exercise 6.6.38, what would suffice?

Hint 2. It suffices that

span (𝐮)

𝑇^{*}

-invariant, that is, that

𝐮

is an eigenvector of

𝑇^{*}

. Show

‖ 𝑇 𝐯 ‖ = ‖ 𝑇^{*} 𝐯 ‖

for normal

𝑇

, and apply it to

𝑇 - 𝜆 id

Hint 3. To apply the induction hypothesis to

𝑆 = 𝑇 ↾_{𝑊}

, check that

𝑊

is also

𝑇^{*}

-invariant and that

𝑆^{*} = 𝑇^{*} ↾_{𝑊}

. Then

𝑆

is normal.

Sketch. For normal

𝑇

{‖ 𝑇 𝐯 ‖}^{2} =

⟨ 𝑇^{*} 𝑇 𝐯, 𝐯 ⟩ =

⟨ 𝑇 𝑇^{*} 𝐯, 𝐯 ⟩ =

{‖ 𝑇^{*} 𝐯 ‖}^{2}

; since

𝑇 - 𝜆 id

is normal,

𝑇 𝐮 = 𝜆 𝐮

gives

𝑇^{*} 𝐮 = 𝜆 𝐮

. Take a unit eigenvector

𝐮

(it exists over

ℂ

). Then

span (𝐮)

is invariant under

𝑇

and

𝑇^{*}

, so

𝑊 = {span (𝐮)}^{⟂}

is too (Exercise 6.6.38). On

𝑊

𝑇 ↾_{𝑊}

has adjoint

𝑇^{*} ↾_{𝑊}

, so it is normal, and by induction

𝑊

has an orthonormal eigenbasis. Add

𝐮

We prove (I) $\Rightarrow$ (II) by induction on $𝑛 = dim 𝑉$ ; the rest is as in the first proof. First, two facts about a normal operator $𝑁$ on an inner product space.

For every $𝐯$ , $‖ 𝑁 𝐯 ‖ = ‖ 𝑁^{*} 𝐯 ‖$ , because
${‖ 𝑁 𝐯 ‖}^{2} = ⟨ 𝑁 𝐯, 𝑁 𝐯 ⟩ = ⟨ 𝑁^{*} 𝑁 𝐯, 𝐯 ⟩ = ⟨ 𝑁 𝑁^{*} 𝐯, 𝐯 ⟩ = ⟨ 𝑁^{*} 𝐯, 𝑁^{*} 𝐯 ⟩ = {‖ 𝑁^{*} 𝐯 ‖}^{2} .$
For every $𝜆 \in ℂ$ , $𝑁 - 𝜆 id$ is normal. Indeed ${(𝑁 - 𝜆 id)}^{*} =$ $𝑁^{*} -$ $𝜆 id$ by Proposition 6.6.18, and expanding,
$(𝑁 - 𝜆 id) (𝑁^{*} - 𝜆 id) = 𝑁 𝑁^{*} - 𝜆 𝑁 - 𝜆 𝑁^{*} + {| 𝜆 |}^{2} id = (𝑁^{*} - 𝜆 id) (𝑁 - 𝜆 id),$
using $𝑁 𝑁^{*} = 𝑁^{*} 𝑁$ . So if $𝑁 𝐮 = 𝜆 𝐮$ , the first fact applied to $𝑁 - 𝜆 id$ gives $‖ (𝑁^{*} - 𝜆 id) 𝐮 ‖ =$ $‖ (𝑁 - 𝜆 id) 𝐮 ‖ =$ $0$ , that is, $𝑁^{*} 𝐮 = 𝜆 𝐮$ .

Now let $𝑇$ be normal. If $𝑛 = 1$ , any unit vector $𝐮$ spans $𝑉$ , so $𝑇 𝐮 \in span (𝐮)$ , and $𝐮$ alone is an orthonormal eigenbasis. Let $𝑛 \geq 2$ , and assume (I) $\Rightarrow$ (II) for complex inner product spaces of dimension $𝑛 - 1$ . By Theorem 7.3.27, $𝑇$ has an eigenvector; dividing by its norm, we get $𝐮$ with $‖ 𝐮 ‖ = 1$ and $𝑇 𝐮 = 𝜆 𝐮$ . By the facts above, $𝑇^{*} 𝐮 = 𝜆 𝐮$ . Let $𝑈 = span (𝐮)$ and $𝑊 = 𝑈^{⟂}$ . Then $𝑈$ is invariant under $𝑇$ and under $𝑇^{*}$ . Applying Exercise 6.6.38 to $𝑇$ shows that $𝑇^{*}$ maps $𝑊$ into $𝑊$ , and applying it to $𝑇^{*}$ (whose adjoint is $𝑇$ , by Proposition 6.6.18(c)) shows that $𝑇$ maps $𝑊$ into $𝑊$ .

Let $𝑆 = 𝑇 ↾_{𝑊} \in L (𝑊)$ , where $𝑊$ carries the inner product of $𝑉$ . For $𝐰, 𝐰^{'} \in 𝑊$ we have $𝑇^{*} ↾_{𝑊} 𝐰^{'} =$ $𝑇^{*} 𝐰^{'} \in$ $𝑊$ and

⟨ 𝑆 𝐰, 𝐰^{'} ⟩ = ⟨ 𝑇 𝐰, 𝐰^{'} ⟩ = ⟨ 𝐰, 𝑇^{*} 𝐰^{'} ⟩,

so $𝑇^{*} ↾_{𝑊}$ is the adjoint of $𝑆$ (Definition 6.6.8, Proposition 6.6.9). Hence $𝑆^{*} 𝑆 =$ $(𝑇^{*} 𝑇) ↾_{𝑊} =$ $(𝑇 𝑇^{*}) ↾_{𝑊} =$ $𝑆 𝑆^{*}$ , and $𝑆$ is normal. By Theorem 6.4.24 and Corollary 6.4.25, $𝑉 = 𝑈 \oplus 𝑊$ and $dim 𝑊 = 𝑛 - 1$ . By the induction hypothesis, $𝑊$ has an orthonormal basis $𝐮_{2}, \dots, 𝐮_{𝑛}$ of eigenvectors of $𝑆$ , which are eigenvectors of $𝑇$ . Each $𝐮_{𝑗}$ lies in $𝑊 = 𝑈^{⟂}$ , so $𝐮, 𝐮_{2}, \dots, 𝐮_{𝑛}$ is orthonormal (Definition 6.3.15), and it is a basis of $𝑉 = 𝑈 \oplus 𝑊$ by Theorem 4.4.13. It consists of eigenvectors of $𝑇$ . □

The third proof looks at all eigenvectors at once. Let $𝐸$ be the sum of all the eigenspaces of $𝑇$ . If $𝐸 \neq 𝑉$ , we look for an eigenvector of $𝑇$ in $𝐸^{⟂}$ , which is impossible since every eigenvector lies in $𝐸$ . Finding one needs $𝐸^{⟂}$ to be $𝑇$ -invariant, and once again that comes from $𝑇^{*}$ preserving eigenspaces.

Proof (the sum of the eigenspaces is everything).standard

Uses. Definition 7.2.13 (Eigenspace), Theorem 7.3.27 (Existence of eigenvalues over $ℂ$ ), Exercise 6.6.38, Corollary 6.4.27 (Testing whether a subspace is everything), Theorem 6.4.20 (Orthonormal bases exist), Definition 6.3.15 (Orthonormal list, orthonormal basis), Proposition 6.3.13 (Orthogonal lists are independent)

Techniques. Extremal choice and minimal counterexamples (Technique 1.5.9), Move an operator across the inner product (Technique 6.6.20)

Hint 1. Let

𝐸 = 𝐸_{𝜆_{1}} + \dots + 𝐸_{𝜆_{𝑘}}

be the sum of the eigenspaces. Use, as in the second proof, that

𝑇 𝐮 = 𝜆 𝐮

implies

𝑇^{*} 𝐮 = 𝜆 𝐮

. Which subspaces does

𝑇^{*}

preserve?

Hint 2.

𝑇^{*}

maps each

𝐸_{𝜆}

into itself, so

𝐸

𝑇^{*}

-invariant, and

𝐸^{⟂}

𝑇

-invariant. What would an eigenvector of

𝑇 ↾_{𝐸^{⟂}}

be?

Hint 3. For orthogonality, compute

⟨ 𝑇 𝐮, 𝐯 ⟩

in two ways for

𝐮 \in 𝐸_{𝜆}

and

𝐯 \in 𝐸_{𝜇}

Sketch. Every eigenvector of

𝑇

is one of

𝑇^{*}

(second proof), so

𝑇^{*}

preserves each eigenspace and hence

𝐸 = \sum_{𝑗} 𝐸_{𝜆_{𝑗}}

; by Exercise 6.6.38,

𝑇

preserves

𝐸^{⟂}

. If

𝐸^{⟂} \neq {𝟎}

, then

𝑇 ↾_{𝐸^{⟂}}

has an eigenvector, which lies in

𝐸 \cap 𝐸^{⟂} = {𝟎}

, a contradiction. So

𝐸 = 𝑉

. Eigenspaces for

𝜆 \neq 𝜇

are orthogonal since

𝜆 ⟨ 𝐮, 𝐯 ⟩ =

⟨ 𝑇 𝐮, 𝐯 ⟩ =

⟨ 𝐮, 𝑇^{*} 𝐯 ⟩ =

𝜇 ⟨ 𝐮, 𝐯 ⟩

. Concatenate orthonormal bases of the eigenspaces.

Let $𝑇$ be normal, let $𝜆_{1}, \dots, 𝜆_{𝑘}$ be its distinct eigenvalues (there is at least one, by Theorem 7.3.27), and let $𝐸 = 𝐸_{𝜆_{1}} + \dots + 𝐸_{𝜆_{𝑘}}$ . As shown in the second proof, $𝑇 𝐮 = 𝜆 𝐮$ implies $𝑇^{*} 𝐮 = 𝜆 𝐮$ . So $𝑇^{*}$ maps each $𝐸_{𝜆_{𝑗}}$ into itself, and therefore maps $𝐸$ into $𝐸$ . By Exercise 6.6.38 applied to $𝑇^{*}$ , the operator $𝑇 = {(𝑇^{*})}^{*}$ maps $𝐸^{⟂}$ into $𝐸^{⟂}$ .

Suppose $𝐸^{⟂} \neq {𝟎}$ . Then $𝑇 ↾_{𝐸^{⟂}}$ is an operator on a nonzero finite-dimensional complex space, so it has an eigenvector $𝐰$ (Theorem 7.3.27). This $𝐰$ is an eigenvector of $𝑇$ , so $𝐰 \in 𝐸$ . Then $𝐰 \in 𝐸 \cap 𝐸^{⟂}$ , so $⟨ 𝐰, 𝐰 ⟩ = 0$ and $𝐰 = 𝟎$ , a contradiction. Hence $𝐸^{⟂} = {𝟎}$ , and $𝐸 = 𝑉$ by Corollary 6.4.27.

Next, eigenspaces for different eigenvalues are orthogonal. Let $𝐮 \in 𝐸_{𝜆}$ and $𝐯 \in 𝐸_{𝜇}$ with $𝜆 \neq 𝜇$ . Then $𝑇^{*} 𝐯 = 𝜇 𝐯$ , and

𝜆 ⟨ 𝐮, 𝐯 ⟩ = ⟨ 𝑇 𝐮, 𝐯 ⟩ = ⟨ 𝐮, 𝑇^{*} 𝐯 ⟩ = ⟨ 𝐮, 𝜇 𝐯 ⟩ = 𝜇 ⟨ 𝐮, 𝐯 ⟩ .

Since $𝜆 \neq 𝜇$ , $⟨ 𝐮, 𝐯 ⟩ = 0$ .

Finally, choose an orthonormal basis ${𝒪︀}_{𝑗}$ of each $𝐸_{𝜆_{𝑗}}$ (Theorem 6.4.20), and let $𝒪︀$ be the concatenation ${𝒪︀}_{1}, \dots, {𝒪︀}_{𝑘}$ . Vectors within one ${𝒪︀}_{𝑗}$ are orthonormal, and vectors from different ${𝒪︀}_{𝑗}$ are orthogonal by the previous paragraph, so $𝒪︀$ is orthonormal (Definition 6.3.15), hence linearly independent (Proposition 6.3.13). It spans $𝐸_{𝜆_{1}} + \dots + 𝐸_{𝜆_{𝑘}} = 𝑉$ . So $𝒪︀$ is an orthonormal eigenbasis. □

Remark 8.2.6 (Comparing the three proofs). The first proof is the shortest, because Schur’s theorem has already done the work of finding a basis, and it isolates the role of normality in a single matrix lemma. The second is self-contained apart from the existence of one eigenvalue, and it is the template for the real theorem below, where Schur’s theorem is not available in the same form. The third proves more than it states: it shows directly that

𝑉

is the orthogonal direct sum of the eigenspaces, which is the spectral decomposition of Section 8.2.4. It also shows a pattern worth remembering: to prove that some subspace

𝐸

is everything, show that

𝐸^{⟂}

is invariant and that any eigenvector in it would have to lie in

𝐸

The lecture states the spectral theorem for self-adjoint operators on complex spaces. It is a special case, and Schur’s theorem proves it without the lemma.

Corollary 8.2.7 (Hermitian matrices). Let

𝑉

be a finite-dimensional complex inner product space with

dim 𝑉 \geq 1

, and

𝑇 \in L (𝑉)

. Then

𝑇

is self-adjoint if and only if

𝑉

has an orthonormal eigenbasis for

𝑇

and every eigenvalue of

𝑇

is real. In matrix form:

𝐴 \in ℂ^{𝑛 \times 𝑛}

is Hermitian if and only if

𝐴 = 𝑈 𝐷 𝑈^{*}

with

𝑈

unitary and

𝐷

diagonal with real entries.

Proof (from the complex spectral theorem).routine

Uses. Theorem 8.2.5 (Complex spectral theorem), Proposition 8.1.1 (Eigenvalues and eigenvectors of self-adjoint operators), Proposition 8.2.2 (Necessary conditions)

Hint 1. A self-adjoint operator is normal.

Hint 2. Which result says the eigenvalues of a self-adjoint operator are real?

Sketch. If

𝑇^{*} = 𝑇

then

𝑇^{*} 𝑇 = 𝑇^{2} = 𝑇 𝑇^{*}

, so Theorem 8.2.5 gives an orthonormal eigenbasis, and the eigenvalues are real by Proposition 8.1.1. The converse is Proposition 8.2.2(b).

𝑇

is self-adjoint, then

𝑇^{*} 𝑇 = 𝑇 𝑇 = 𝑇 𝑇^{*}

, so

𝑇

is normal and has an orthonormal eigenbasis by Theorem 8.2.5; its eigenvalues are real by Proposition 8.1.1. The converse is Proposition 8.2.2(b). The matrix form follows as in the first proof of Theorem 8.2.5:

𝐴

is Hermitian if and only if

𝑇_{𝐴}

is self-adjoint (Example 6.6.10), and

𝑈^{*} 𝐴 𝑈 = 𝐷

is real diagonal exactly when the columns of

𝑈

form an orthonormal eigenbasis with real eigenvalues. □

Proof (directly from Schur, as in lecture).routine

Uses. Theorem 7.5.18 (Schur’s theorem), Corollary 6.6.28 (Self-adjointness in an orthonormal basis), Proposition 8.2.2 (Necessary conditions)

Hint 1. Take a Schur basis

𝒪︀

. If

𝑇

is self-adjoint, what is the relation between

{[𝑇]}_{𝒪︀}

and its conjugate transpose?

Hint 2. A matrix that equals its conjugate transpose and is upper triangular is also lower triangular.

Sketch. For a Schur basis,

𝑅 = {[𝑇]}_{𝒪︀}

is upper triangular and

𝑅^{*} = 𝑅

(Corollary 6.6.28). So

𝑅

is also lower triangular, hence diagonal, and

𝑟_{𝑗 𝑗} = 𝑟_{𝑗 𝑗}

is real.

Let

𝑇

be self-adjoint, and let

𝒪︀

be an orthonormal basis with

𝑅 = {[𝑇]}_{𝒪︀}

upper triangular (Theorem 7.5.18). By Corollary 6.6.28,

𝑅^{*} = 𝑅

, that is,

𝑟_{𝑖 𝑗} = 𝑟_{𝑗 𝑖}

for all

𝑖, 𝑗

. For

𝑖 > 𝑗

we have

𝑟_{𝑖 𝑗} = 0

, so

𝑟_{𝑗 𝑖} = 𝑟_{𝑖 𝑗} = 0

: the entries above the diagonal vanish too, and

𝑅

is diagonal. Its diagonal entries satisfy

𝑟_{𝑗 𝑗} = 𝑟_{𝑗 𝑗}

, so they are real, and they are the eigenvalues of the vectors of

𝒪︀

. The converse is Proposition 8.2.2(b). □

Example 8.2.8 (Two unitary diagonalizations).

(a)The real matrix $𝑁 = (\begin{matrix} 1 & - 1 \\ 1 & 1 \end{matrix}) = 𝐼_{2} + 𝐽$ is normal, since $𝐽$ commutes with $𝐽^{𝖳} = - 𝐽$ . It is not symmetric, so it is not orthogonally diagonalizable over $ℝ$ (and indeed $𝑝_{𝑁} = {(𝑥 - 1)}^{2} + 1$ has no real root). Over $ℂ$ its eigenvectors are those of $𝐽$ from Example 8.2.3, with eigenvalues $1 + 𝑖$ and $1 - 𝑖$ :
$𝑁 = 𝑈 (\begin{matrix} 1 + 𝑖 & 0 \\ 0 & 1 - 𝑖 \end{matrix}) 𝑈^{*}, 𝑈 = \frac{1}{\sqrt{2}} (\begin{matrix} 1 & 1 \\ - 𝑖 & 𝑖 \end{matrix}) .$
(b)The Hermitian matrix $𝐻 = (\begin{matrix} 2 & 1 - 𝑖 \\ 1 + 𝑖 & 3 \end{matrix})$ of Example 6.6.27 has $𝑝_{𝐻} =$ $(𝑥 - 2) (𝑥 - 3) -$ $(1 - 𝑖) (1 + 𝑖) =$ $𝑥^{2} -$ $5 𝑥 +$ $4 =$ $(𝑥 - 1) (𝑥 - 4)$ . Row reducing $𝐻 - 𝐼_{2}$ and $𝐻 - 4 𝐼_{2}$ gives the eigenvectors $(- 1 + 𝑖, 1)$ for $1$ and $(1, 1 + 𝑖)$ for $4$ . Each has squared norm $3$ , and they are orthogonal, as Corollary 8.2.7 predicts: $(- 1 + 𝑖) \cdot$ $1 +$ $1 \cdot$ $1 + 𝑖 =$ $- 1 +$ $𝑖 +$ $1 -$ $𝑖 =$ $0$ . So
$𝐻 = 𝑈 (\begin{matrix} 1 & 0 \\ 0 & 4 \end{matrix}) 𝑈^{*}, 𝑈 = \frac{1}{\sqrt{3}} (\begin{matrix} - 1 + 𝑖 & 1 \\ 1 & 1 + 𝑖 \end{matrix}) .$

Exercise 8.2.9.routine For each matrix, decide whether it is unitarily diagonalizable (over $ℂ$ ), and for the real ones also whether it is orthogonally diagonalizable (over $ℝ$ ).

(a) $(\begin{matrix} 1 & 2 \\ 0 & 3 \end{matrix})$
(b) $(\begin{matrix} 0 & - 2 \\ 2 & 0 \end{matrix})$
(c) $(\begin{matrix} 1 & 𝑖 \\ 𝑖 & 1 \end{matrix})$
(d) $(\begin{matrix} 1 & 2 \\ 2 & 1 \end{matrix})$
(e) $(\begin{matrix} 1 & 1 & 0 \\ 0 & 1 & 1 \\ 1 & 0 & 1 \end{matrix})$

Solution.

Uses. Theorem 8.2.5 (Complex spectral theorem), Proposition 8.2.2 (Necessary conditions)

Hint 1. By Theorem 8.2.5, check

𝐴^{*} 𝐴 = 𝐴 𝐴^{*}

for the first question. For the second, an orthogonally diagonalizable real matrix must be symmetric (Proposition 8.2.2).

Hint 2. In (e), the matrix is

𝐼_{3} + 𝑃

for a permutation matrix

𝑃

with

𝑃^{𝖳} = 𝑃^{- 1}

Sketch. (a) Not normal: neither. (b) Normal, not symmetric: unitarily only. (c)

𝐴^{*} 𝐴 = 2 𝐼_{2} = 𝐴 𝐴^{*}

: unitarily. (d) Both:

(1, 1)

and

(1, - 1)

are orthogonal eigenvectors. (e)

𝐼 + 𝑃

with

𝑃^{𝖳} 𝑃 = 𝑃 𝑃^{𝖳} = 𝐼

is normal but not symmetric: unitarily only.

By Theorem 8.2.5 a complex matrix is unitarily diagonalizable if and only if it is normal. A real orthogonally diagonalizable matrix is symmetric by Proposition 8.2.2(b), since its eigenvalues are real. (The converse is Theorem 8.2.12 below.)

(a) With $𝐴 = (\begin{matrix} 1 & 2 \\ 0 & 3 \end{matrix})$ , ${(𝐴^{*} 𝐴)}_{11} = 1$ but ${(𝐴 𝐴^{*})}_{11} = 1 + 4 = 5$ . Not normal, so neither.

(b) $𝐴 = 2 𝐽$ with $𝐽^{𝖳} 𝐽 = 𝐽 𝐽^{𝖳} = 𝐼_{2}$ , so $𝐴^{𝖳} 𝐴 =$ $4 𝐼_{2} =$ $𝐴 𝐴^{𝖳}$ and $𝐴$ is normal: unitarily diagonalizable. It is not symmetric, so not orthogonally diagonalizable.

(c) $𝐴^{*} = (\begin{matrix} 1 & - 𝑖 \\ - 𝑖 & 1 \end{matrix})$ , and $𝐴^{*} 𝐴 =$ $(\begin{matrix} 1 + 1 & 𝑖 - 𝑖 \\ - 𝑖 + 𝑖 & 1 + 1 \end{matrix}) =$ $2 𝐼_{2}$ ; similarly $𝐴 𝐴^{*} = 2 𝐼_{2}$ . Normal, so unitarily diagonalizable (although $𝐴^{*} \neq 𝐴$ ). It is not real.

(d) $𝐴 (1, 1) = 3 (1, 1)$ and $𝐴 (1, - 1) = - (1, - 1)$ , and these eigenvectors are orthogonal. So $𝐴 =$ $𝑄 diag (3, - 1) 𝑄^{𝖳}$ with $𝑄 = \frac{1}{\sqrt{2}} (\begin{matrix} 1 & 1 \\ 1 & - 1 \end{matrix})$ : orthogonally, and in particular unitarily, diagonalizable.

(e) $𝐴 = 𝐼_{3} + 𝑃$ where $𝑃$ is the permutation matrix with $𝑃 𝐞_{1} = 𝐞_{3}$ , $𝑃 𝐞_{2} = 𝐞_{1}$ , $𝑃 𝐞_{3} = 𝐞_{2}$ . Its columns are orthonormal, so $𝑃^{𝖳} 𝑃 = 𝐼_{3}$ , hence $𝑃^{𝖳} = 𝑃^{- 1}$ and $𝑃 𝑃^{𝖳} = 𝐼_{3}$ as well. Then $𝐴^{𝖳} 𝐴 =$ $𝐼 +$ $𝑃 +$ $𝑃^{𝖳} +$ $𝐼 =$ $𝐴 𝐴^{𝖳}$ , so $𝐴$ is normal: unitarily diagonalizable. It is not symmetric ( $𝑎_{12} = 1 \neq 0 = 𝑎_{21}$ ), so not orthogonally diagonalizable.

8.2.3The real spectral theorem

Over $ℝ$ , Example 8.2.3(b) shows that normality is not enough, and Proposition 8.2.2(b) shows that self-adjointness is necessary. To prove it is sufficient, we want to run the second proof of Theorem 8.2.5 again. Self-adjointness takes care of the invariance of $𝑊$ for free (Exercise 6.6.38), so the only step that used $ℂ$ is the existence of an eigenvector. That is the content of the next lemma.

Lemma 8.2.10 (Self-adjoint operators have eigenvalues). Let

𝑉

be a finite-dimensional real inner product space with

dim 𝑉 \geq 1

, and let

𝑇 \in L (𝑉)

be self-adjoint. Then

𝑇

has an eigenvalue. In fact every complex root of

𝑝_{𝑇}

is real, so

𝑝_{𝑇}

splits over

ℝ

The first proof borrows the eigenvalue from $ℂ$ . It turns $𝑇$ into a real symmetric matrix, regards that matrix as a complex Hermitian matrix, and uses that Hermitian matrices have only real eigenvalues.

Proof (via a Hermitian matrix).standard

Uses. Theorem 6.4.20 (Orthonormal bases exist), Corollary 6.6.28 (Self-adjointness in an orthonormal basis), Example 6.6.10 (The adjoint of a matrix), Corollary 7.1.35 (Factorization over $ℂ$ ), Theorem 7.3.6 (Eigenvalues are the roots of the characteristic polynomial), Proposition 8.1.1 (Eigenvalues and eigenvectors of self-adjoint operators), Definition 7.3.12 (Characteristic polynomial of an operator), Corollary 7.3.13 (Eigenvalues of an operator)

Hint 1. Choose an orthonormal basis and let

𝐴 = {[𝑇]}_{𝒪︀} \in ℝ^{𝑛 \times 𝑛}

. What do you know about

𝐴

Hint 2.

𝐴

is symmetric, so as a complex matrix it is Hermitian. Where are the complex roots of

𝑝_{𝐴}

Sketch.

𝐴 = {[𝑇]}_{𝒪︀}

is real symmetric, hence Hermitian as a complex matrix. Over

ℂ

𝑝_{𝐴}

factors into linear factors, and each root is an eigenvalue of the self-adjoint

𝑇_{𝐴}

ℂ^{𝑛}

, hence real. So

𝑝_{𝑇} = 𝑝_{𝐴}

splits over

ℝ

, and any of its roots is an eigenvalue of

𝑇

Let

𝒪︀

be an orthonormal basis of

𝑉

(Theorem 6.4.20) and

𝐴 = {[𝑇]}_{𝒪︀} \in ℝ^{𝑛 \times 𝑛}

𝑛 = dim 𝑉

. By Corollary 6.6.28,

𝐴^{𝖳} = 𝐴

. Regard

𝐴

as a matrix in

ℂ^{𝑛 \times 𝑛}

. Its entries are real, so

𝐴^{*} = 𝐴^{𝖳} = 𝐴^{𝖳} = 𝐴

, and

𝑇_{𝐴}

is a self-adjoint operator on

ℂ^{𝑛}

with the dot product (Example 6.6.10). By Corollary 7.1.35,

𝑝_{𝐴} =

(𝑥 - 𝜆_{1}) \dots (𝑥 - 𝜆_{𝑛})

for some

𝜆_{𝑖} \in ℂ

. Each

𝜆_{𝑖}

is an eigenvalue of

𝐴 \in ℂ^{𝑛 \times 𝑛}

(Theorem 7.3.6), that is, of

𝑇_{𝐴}

, so

𝜆_{𝑖} \in ℝ

by Proposition 8.1.1. Hence

𝑝_{𝐴}

splits over

ℝ

. (The determinant

det (𝑥 𝐼_{𝑛} - 𝐴)

is the same polynomial whether

𝐴

is viewed as real or complex.) Since

𝑝_{𝑇} = 𝑝_{𝐴}

(Definition 7.3.12), the real number

𝜆_{1}

is a root of

𝑝_{𝑇}

, hence an eigenvalue of

𝑇

by Corollary 7.3.13. □

The second proof stays inside $𝑉$ . The idea comes from factoring polynomials over $ℝ$ : some nonzero polynomial $𝑝$ satisfies $𝑝 (𝑇) 𝐯 = 𝟎$ , and $𝑝$ factors into linear factors and irreducible quadratics (Theorem 7.1.39). The rotation by $𝜋 / 2$ shows that a quadratic factor such as $𝑥^{2} + 1$ can be what kills $𝐯$ . Self-adjointness rules this out: for self-adjoint $𝑇$ , an irreducible quadratic $𝑞$ gives an invertible $𝑞 (𝑇)$ , because $⟨ 𝑞 (𝑇) 𝐯, 𝐯 ⟩$ is bounded below by a quadratic in $‖ 𝑇 𝐯 ‖$ with negative discriminant.

Proof (without complex numbers).hard

Uses. Proposition 7.2.35 (A basis of a Krylov subspace), Theorem 7.1.39 (Factorization over $ℝ$ ), Proposition 7.2.27 (Evaluating polynomials at an operator), Theorem 6.2.1 (Cauchy–Schwarz inequality), Definition 6.6.26 (Self-adjoint operator), Theorem 7.2.9 (Eigenvalues and invertibility)

Techniques. A nonnegative quadratic has nonpositive discriminant (Technique 6.2.7), Move an operator across the inner product (Technique 6.6.20)

Hint 1. Take

𝐯 \neq 𝟎

. Some monic

𝑝

of degree

𝑑 \geq 1

has

𝑝 (𝑇) 𝐯 = 𝟎

(Proposition 7.2.35). Factor

𝑝

over

ℝ

Hint 2. Show that

𝑞 (𝑇) =

𝑇^{2} +

𝑏 𝑇 +

𝑐 id

is injective when

𝑏^{2} < 4 𝑐

: for

𝐯 \neq 𝟎

, show

⟨ 𝑞 (𝑇) 𝐯, 𝐯 ⟩ > 0

. Use

⟨ 𝑇^{2} 𝐯, 𝐯 ⟩ =

{‖ 𝑇 𝐯 ‖}^{2}

and Cauchy–Schwarz on

⟨ 𝑇 𝐯, 𝐯 ⟩

Hint 3. Complete the square:

{‖ 𝑇 𝐯 ‖}^{2} -

| 𝑏 | ‖ 𝑇 𝐯 ‖ ‖ 𝐯 ‖ +

𝑐 {‖ 𝐯 ‖}^{2} =

{(‖ 𝑇 𝐯 ‖ - \frac{| 𝑏 |}{2} ‖ 𝐯 ‖)}^{2} +

(𝑐 - \frac{𝑏^{2}}{4}) {‖ 𝐯 ‖}^{2}

. Then

𝑝 (𝑇)

is a product of injective quadratic factors and factors

𝑇 - 𝜆_{𝑖} id

, and it is not injective.

Sketch. For

𝑏^{2} < 4 𝑐

and

𝐯 \neq 𝟎

, self-adjointness and Cauchy–Schwarz give

⟨ (𝑇^{2} + 𝑏 𝑇 + 𝑐 id) 𝐯, 𝐯 ⟩ \geq

{(‖ 𝑇 𝐯 ‖ - \frac{| 𝑏 |}{2} ‖ 𝐯 ‖)}^{2} +

(𝑐 - \frac{𝑏^{2}}{4}) {‖ 𝐯 ‖}^{2} >

0

, so

𝑇^{2} + 𝑏 𝑇 + 𝑐 id

is injective. Pick a monic

𝑝

of degree

\geq 1

with

𝑝 (𝑇) 𝐯 = 𝟎

for some

𝐯 \neq 𝟎

and factor it over

ℝ

. The product

𝑝 (𝑇)

of the factors is not injective, so some factor is not; it cannot be a quadratic one, so it is some

𝑇 - 𝜆 id

, and

𝜆

is an eigenvalue.

We prove that $𝑇$ has an eigenvalue. First let $𝑏, 𝑐 \in ℝ$ with $𝑏^{2} < 4 𝑐$ ; we show that $𝑞 (𝑇) =$ $𝑇^{2} +$ $𝑏 𝑇 +$ $𝑐 id$ is injective. Let $𝐯 \neq 𝟎$ . Since $𝑇$ is self-adjoint, $⟨ 𝑇^{2} 𝐯, 𝐯 ⟩ =$ $⟨ 𝑇 𝐯, 𝑇 𝐯 ⟩ =$ ${‖ 𝑇 𝐯 ‖}^{2}$ , and by Cauchy–Schwarz, $𝑏 ⟨ 𝑇 𝐯, 𝐯 ⟩ \geq$ $- | 𝑏 | ‖ 𝑇 𝐯 ‖ ‖ 𝐯 ‖$ . Hence

\begin{aligned} ⟨ 𝑞 (𝑇) 𝐯, 𝐯 ⟩ & = {‖ 𝑇 𝐯 ‖}^{2} + 𝑏 ⟨ 𝑇 𝐯, 𝐯 ⟩ + 𝑐 {‖ 𝐯 ‖}^{2} \\ \geq {‖ 𝑇 𝐯 ‖}^{2} - | 𝑏 | ‖ 𝑇 𝐯 ‖ ‖ 𝐯 ‖ + 𝑐 {‖ 𝐯 ‖}^{2} \\ = {(‖ 𝑇 𝐯 ‖ - \frac{| 𝑏 |}{2} ‖ 𝐯 ‖)}^{2} + (𝑐 - \frac{𝑏^{2}}{4}) {‖ 𝐯 ‖}^{2} > 0, \end{aligned}

because $𝑐 - 𝑏^{2} / 4 > 0$ and $‖ 𝐯 ‖ > 0$ . So $𝑞 (𝑇) 𝐯 \neq 𝟎$ .

Now fix $𝐯 \neq 𝟎$ . By Proposition 7.2.35 there is a monic $𝑝 \in ℝ [𝑥]$ of degree $𝑑 \geq 1$ with $𝑝 (𝑇) 𝐯 = 𝟎$ . By Theorem 7.1.39,

𝑝 = (𝑥 - 𝜆_{1}) \dots (𝑥 - 𝜆_{𝑘}) 𝑞_{1} \dots 𝑞_{𝑙}, 𝑞_{𝑗} = 𝑥^{2} + 𝑏_{𝑗} 𝑥 + 𝑐_{𝑗}, 𝑏_{𝑗}^{2} < 4 𝑐_{𝑗},

and by Proposition 7.2.27,

𝑝 (𝑇) = (𝑇 - 𝜆_{1} id) \dots (𝑇 - 𝜆_{𝑘} id) 𝑞_{1} (𝑇) \dots 𝑞_{𝑙} (𝑇) .

A composition of injective maps is injective, and $𝑝 (𝑇)$ is not injective since it kills $𝐯$ . So at least one factor is not injective. The factors $𝑞_{𝑗} (𝑇)$ are injective by the first paragraph, so some $𝑇 - 𝜆_{𝑖} id$ is not injective (in particular $𝑘 \geq 1$ ), and $𝜆_{𝑖}$ is an eigenvalue of $𝑇$ by Theorem 7.2.9.

The second statement of the lemma (all complex roots of $𝑝_{𝑇}$ are real) follows from the first proof of Theorem 8.2.12 below, which uses only the first statement: in an orthonormal eigenbasis, ${[𝑇]}_{𝒪︀} =$ $diag (𝜇_{1}, \dots, 𝜇_{𝑛})$ with real $𝜇_{𝑗}$ , so $𝑝_{𝑇} =$ $(𝑥 - 𝜇_{1}) \dots (𝑥 - 𝜇_{𝑛})$ . □

Remark 8.2.11 (A third proof, by maximizing). There is a third proof, which finds the eigenvector as the solution of an optimization problem. On the unit sphere

𝑆 = {𝐯 \in 𝑉 | ‖ 𝐯 ‖ = 1}

, the function

𝐯 \mapsto ⟨ 𝑇 𝐯, 𝐯 ⟩

is continuous, and

𝑆

is closed and bounded, so the function attains a maximum at some

𝐮 \in 𝑆

. Perturbing a maximizer (Technique 6.5.6) along any direction

𝐰 ⟂ 𝐮

shows that

⟨ 𝑇 𝐮, 𝐰 ⟩ = 0

(the derivative of the perturbed value is

2 ⟨ 𝑇 𝐮, 𝐰 ⟩

because

⟨ 𝑇 𝐰, 𝐮 ⟩ =

⟨ 𝐰, 𝑇 𝐮 ⟩

, which is where self-adjointness enters), that is,

𝑇 𝐮 \in span (𝐮)

, so

𝐮

is an eigenvector, and its eigenvalue is the maximum value. The existence of the maximum is a fact from analysis, which is why this book postpones the argument to Section 12.3. That section also explains what the other eigenvalues maximize (the Courant–Fischer theorem).

Theorem 8.2.12 (Real spectral theorem). Let $𝑉$ be a finite-dimensional real inner product space with $dim 𝑉 \geq 1$ , and let $𝑇 \in L (𝑉)$ . The following are equivalent:

(I) $𝑇$ is self-adjoint;
(II) $𝑉$ has an orthonormal basis consisting of eigenvectors of $𝑇$ ;
(III) ${[𝑇]}_{𝒪︀}$ is diagonal for some orthonormal basis $𝒪︀$ of $𝑉$ .

In matrix form: $𝐴 \in ℝ^{𝑛 \times 𝑛}$ is symmetric if and only if $𝐴 = 𝑄 𝐷 𝑄^{𝖳}$ for some orthogonal $𝑄 \in ℝ^{𝑛 \times 𝑛}$ and some diagonal $𝐷 \in ℝ^{𝑛 \times 𝑛}$ , that is, if and only if $𝐴$ is orthogonally diagonalizable.

Again (II) $\Leftrightarrow$ (III) is Theorem 7.4.2 and (II) $\Rightarrow$ (I) is Proposition 8.2.2(b), so we prove (I) $\Rightarrow$ (II).

Proof (induction with invariant complements).standard

Uses. Lemma 8.2.10 (Self-adjoint operators have eigenvalues), Exercise 6.6.38, Definition 6.6.26 (Self-adjoint operator), Theorem 6.4.24 (Orthogonal decomposition), Corollary 6.4.25 (Dimension of the orthogonal complement), Theorem 4.4.13 (Bases and dimension of a sum), Definition 6.3.15 (Orthonormal list, orthonormal basis), Proposition 8.2.2 (Necessary conditions), Theorem 4.5.17 (Change of basis), Definition 6.3.33 (Unitary and orthogonal matrices)

Techniques. Induction on dimension via an invariant subspace (Technique 7.5.12), Proving TFAE (Technique 2.7.1)

Hint 1. Follow the second proof of Theorem 8.2.5. Which step used

ℂ

, and what replaces it?

Hint 2. Lemma 8.2.10 provides a unit eigenvector

𝐮

. Why is

𝑊 = {span (𝐮)}^{⟂}

invariant this time, and why is

𝑇 ↾_{𝑊}

self-adjoint?

Sketch. Induct on

dim 𝑉

. By Lemma 8.2.10,

𝑇

has a unit eigenvector

𝐮

. Since

𝑇^{*} = 𝑇

maps

span (𝐮)

into itself,

𝑇

maps

𝑊 = {span (𝐮)}^{⟂}

into itself (Exercise 6.6.38), and

𝑇 ↾_{𝑊}

is self-adjoint because

⟨ 𝑇 𝐰, 𝐰^{'} ⟩ =

⟨ 𝐰, 𝑇 𝐰^{'} ⟩

for

𝐰, 𝐰^{'} \in 𝑊

. By induction

𝑊

has an orthonormal eigenbasis; add

𝐮

. For matrices, put the eigenbasis of

𝑇_{𝐴}

into the columns of

𝑄

We prove (I) $\Rightarrow$ (II) by induction on $𝑛 = dim 𝑉$ . If $𝑛 = 1$ , any unit vector is an orthonormal eigenbasis, as in the complex case. Let $𝑛 \geq 2$ , assume the claim in dimension $𝑛 - 1$ , and let $𝑇$ be self-adjoint. By Lemma 8.2.10, $𝑇$ has an eigenvector, and dividing by its norm gives $𝐮$ with $‖ 𝐮 ‖ = 1$ and $𝑇 𝐮 = 𝜆 𝐮$ . Let $𝑈 = span (𝐮)$ and $𝑊 = 𝑈^{⟂}$ . Since $𝑇^{*} = 𝑇$ maps $𝑈$ into $𝑈$ , Exercise 6.6.38 (applied to $𝑇^{*}$ , whose adjoint is $𝑇$ ) shows that $𝑇$ maps $𝑊$ into $𝑊$ . The operator $𝑆 = 𝑇 ↾_{𝑊} \in L (𝑊)$ is self-adjoint for the inner product of $𝑉$ restricted to $𝑊$ , since $⟨ 𝑆 𝐰, 𝐰^{'} ⟩ =$ $⟨ 𝑇 𝐰, 𝐰^{'} ⟩ =$ $⟨ 𝐰, 𝑇 𝐰^{'} ⟩ =$ $⟨ 𝐰, 𝑆 𝐰^{'} ⟩$ for $𝐰, 𝐰^{'} \in 𝑊$ (Definition 6.6.26). By Theorem 6.4.24 and Corollary 6.4.25, $𝑉 = 𝑈 \oplus 𝑊$ with $dim 𝑊 = 𝑛 - 1 \geq 1$ . By the induction hypothesis, $𝑊$ has an orthonormal basis $𝐮_{2}, \dots, 𝐮_{𝑛}$ of eigenvectors of $𝑆$ , hence of $𝑇$ . As in the complex case, $𝐮, 𝐮_{2}, \dots, 𝐮_{𝑛}$ is an orthonormal basis of $𝑉$ (Definition 6.3.15, Theorem 4.4.13) consisting of eigenvectors of $𝑇$ .

For the matrix form, ${(𝑄 𝐷 𝑄^{𝖳})}^{𝖳} =$ $𝑄 𝐷^{𝖳} 𝑄^{𝖳} =$ $𝑄 𝐷 𝑄^{𝖳}$ , so orthogonally diagonalizable matrices are symmetric. Conversely, if $𝐴$ is symmetric, $𝑇_{𝐴}$ is self-adjoint on $ℝ^{𝑛}$ with the dot product, so $ℝ^{𝑛}$ has an orthonormal eigenbasis $𝐪_{1}, \dots, 𝐪_{𝑛}$ . The matrix $𝑄$ with these columns is orthogonal (Definition 6.3.33), and $𝑄^{𝖳} 𝐴 𝑄 =$ $𝑄^{- 1} 𝐴 𝑄 =$ $𝐷$ is diagonal by Theorem 4.5.17. Hence $𝐴 = 𝑄 𝐷 𝑄^{𝖳}$ . □

The second proof follows the first proof of the complex theorem instead, with a real Schur basis.

Proof (triangularize, then use symmetry).standard

Uses. Lemma 8.2.10 (Self-adjoint operators have eigenvalues), Theorem 7.5.17 (Triangularizable means the characteristic polynomial splits), Proposition 7.5.2 (Upper-triangular matrices and invariant subspaces), Theorem 6.4.15 (Gram–Schmidt is correct), Corollary 6.6.28 (Self-adjointness in an orthonormal basis)

Techniques. Build a flag of invariant subspaces (Technique 7.5.4), Prove it for triangular matrices first (Technique 7.5.23)

Hint 1. By Lemma 8.2.10,

𝑝_{𝑇}

splits over

ℝ

. What does Theorem 7.5.17 give?

Hint 2. Gram–Schmidt preserves the flag of a triangularizing basis, exactly as in the first proof of Theorem 7.5.18. Then compare

{[𝑇]}_{𝒪︀}

with its transpose.

Sketch.

𝑝_{𝑇}

splits over

ℝ

(Lemma 8.2.10), so some basis

ℬ︀

triangularizes

𝑇

; Gram–Schmidt turns it into an orthonormal

𝒪︀

with the same invariant flag, so

𝑅 = {[𝑇]}_{𝒪︀}

is upper triangular. It is also symmetric (Corollary 6.6.28), hence diagonal.

By Lemma 8.2.10,

𝑝_{𝑇}

splits over

ℝ

, so by Theorem 7.5.17 there is a basis

ℬ︀ = 𝐛_{1}, \dots, 𝐛_{𝑛}

𝑉

with

{[𝑇]}_{ℬ︀}

upper triangular; by Proposition 7.5.2, every

𝑉_{𝑗} =

span (𝐛_{1}, \dots, 𝐛_{𝑗})

𝑇

-invariant. Gram–Schmidt applied to

ℬ︀

gives an orthonormal basis

𝒪︀ = 𝐮_{1}, \dots, 𝐮_{𝑛}

with

span (𝐮_{1}, \dots, 𝐮_{𝑗}) =

𝑉_{𝑗}

for every

𝑗

(Theorem 6.4.15(d)), so

𝑅 = {[𝑇]}_{𝒪︀}

is upper triangular by Proposition 7.5.2. (This is the first proof of Theorem 7.5.18, which used nothing about

ℂ

beyond the triangularizing basis.) By Corollary 6.6.28,

𝑅^{𝖳} = 𝑅

. For

𝑖 < 𝑗

𝑟_{𝑖 𝑗} = 𝑟_{𝑗 𝑖} = 0

, so

𝑅

is diagonal, and

𝒪︀

is an orthonormal eigenbasis. □

Non-example 8.2.13 (Symmetric matrices over other fields). The real spectral theorem is about real symmetric matrices, and the inner product on

ℝ^{𝑛}

is what makes it work. Over other fields the transpose still exists, but symmetric matrices need not be diagonalizable.

(a) $𝐴 = (\begin{matrix} 1 & 𝑖 \\ 𝑖 & - 1 \end{matrix}) \in ℂ^{2 \times 2}$ satisfies $𝐴^{𝖳} = 𝐴$ , and $𝐴^{2} =$ $(\begin{matrix} 1 + 𝑖^{2} & 𝑖 - 𝑖 \\ 𝑖 - 𝑖 & 𝑖^{2} + 1 \end{matrix}) =$ $0$ . So $𝐴$ is nilpotent and nonzero, and is not diagonalizable (its only eigenvalue is $0$ , and $𝐴 \neq 0$ ). Over $ℂ$ , the right condition is $𝐴^{*} = 𝐴$ , not $𝐴^{𝖳} = 𝐴$ .
(b) $𝐴 = (\begin{matrix} 1 & 1 \\ 1 & 1 \end{matrix}) \in 𝔽_{2}^{2 \times 2}$ is symmetric with $𝐴^{2} = (\begin{matrix} 2 & 2 \\ 2 & 2 \end{matrix}) = 0$ in $𝔽_{2}$ . Again a nonzero nilpotent matrix, so not diagonalizable.

In both cases the proofs above fail at the same point: the “inner product”

𝐱^{𝖳} 𝐲

has nonzero vectors with

𝐱^{𝖳} 𝐱 = 0

, such as

(1, 𝑖)

and

(1, 1)

, and every argument above that divided by

‖ 𝐮 ‖

or used

‖ 𝐰 ‖ = 0 \Rightarrow 𝐰 = 𝟎

breaks. Section 12.4 studies what survives.

Computing an orthogonal diagonalization

The proofs suggest the computation. Eigenspaces for different eigenvalues are automatically orthogonal (third proof of Theorem 8.2.5, which applies verbatim to self-adjoint operators on real spaces), so orthogonality only has to be arranged inside each eigenspace.

Algorithm 8.2.14 (Orthogonal diagonalization)

Input: A symmetric $𝐴 \in ℝ^{𝑛 \times 𝑛}$ .
Output: An orthogonal $𝑄$ and a diagonal $𝐷$ with $𝐴 = 𝑄 𝐷 𝑄^{𝖳}$ .

Compute

𝑝_{𝐴}

and its roots

𝜆_{1}, \dots, 𝜆_{𝑘}

(all real, by Lemma 8.2.10).

For each

𝜆_{𝑗}

, find a basis of

𝐸_{𝜆_{𝑗}} =

ker (𝐴 - 𝜆_{𝑗} 𝐼_{𝑛})

by row reduction.

Apply Gram–Schmidt to each of these bases separately, to get an orthonormal basis of each

𝐸_{𝜆_{𝑗}}

Let

𝑄

have these orthonormal bases as its columns, and let

𝐷

be diagonal with the eigenvalue of each column in the matching position.

The columns found in step 3 are orthonormal within each eigenspace, and orthogonal across eigenspaces by the computation in the third proof of Theorem 8.2.5. There are $𝑛$ of them, since $𝐴$ is diagonalizable and so the eigenspace dimensions add up to $𝑛$ (Theorem 7.4.16). Only step 3 is new compared with Algorithm 7.4.18. For a simple eigenvalue it just normalizes the eigenvector; Gram–Schmidt does real work only for repeated eigenvalues.

Example 8.2.15 (A repeated eigenvalue). Let $𝐴 =$ $(\begin{matrix} 2 & 1 & 1 \\ 1 & 2 & 1 \\ 1 & 1 & 2 \end{matrix}) =$ $𝐼_{3} +$ $𝐽_{3}$ , where $𝐽_{3}$ is the all-ones matrix. Since $𝐽_{3} 𝐱 =$ $(𝑥_{1} + 𝑥_{2} + 𝑥_{3}) (1, 1, 1)$ , the eigenvalues of $𝐽_{3}$ are $3$ (on $span ((1, 1, 1))$ ) and $0$ (on the plane $𝑥_{1} + 𝑥_{2} + 𝑥_{3} = 0$ ), so $𝐴$ has eigenvalues $4$ and $1$ , with

𝐸_{4} = span ((1, 1, 1)), 𝐸_{1} = {𝐱 | 𝑥_{1} + 𝑥_{2} + 𝑥_{3} = 0} = span ((1, - 1, 0), (1, 0, - 1)) .

The basis of $𝐸_{1}$ from row reduction is not orthogonal. Gram–Schmidt replaces $(1, 0, - 1)$ by $(1, 0, - 1) -$ $\frac{1}{2} (1, - 1, 0) =$ $\frac{1}{2} (1, 1, - 2)$ . Normalizing,

𝑄 = (\begin{matrix} \frac{1}{\sqrt{3}} & \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{6}} \\ \frac{1}{\sqrt{3}} & - \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{6}} \\ \frac{1}{\sqrt{3}} & 0 & - \frac{2}{\sqrt{6}} \end{matrix}), 𝐷 = (\begin{matrix} 4 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix}), 𝐴 = 𝑄 𝐷 𝑄^{𝖳} .

Any orthonormal basis of the plane $𝐸_{1}$ would do in the last two columns; step 3 only picks one. The first column is orthogonal to the plane $𝐸_{1}$ automatically.

Exercise 8.2.16.routine Find an orthogonal

𝑄

and a diagonal

𝐷

with

(\begin{matrix} 7 & - 2 \\ - 2 & 4 \end{matrix}) = 𝑄 𝐷 𝑄^{𝖳}

Solution.

Uses. Algorithm 8.2.14 (Orthogonal diagonalization)

Techniques. Diagonalize, solve, conjugate back (Technique 7.4.22)

Hint 1. The trace is

11

and the determinant is

24

Hint 2. The eigenvalues are

3

and

8

. Find one eigenvector for each and normalize; they are automatically orthogonal.

Sketch.

𝑝 =

𝑥^{2} -

11 𝑥 +

24 =

(𝑥 - 3) (𝑥 - 8)

𝐸_{8} = span ((2, - 1))

and

𝐸_{3} = span ((1, 2))

𝑄 = \frac{1}{\sqrt{5}} (\begin{matrix} 2 & 1 \\ - 1 & 2 \end{matrix})

𝐷 = diag (8, 3)

$𝑝_{𝐴} =$ $(𝑥 - 7) (𝑥 - 4) -$ $4 =$ $𝑥^{2} -$ $11 𝑥 +$ $24 =$ $(𝑥 - 3) (𝑥 - 8)$ . From $𝐴 -$ $8 𝐼_{2} =$ $(\begin{matrix} - 1 & - 2 \\ - 2 & - 4 \end{matrix})$ we get $𝐸_{8} = span ((2, - 1))$ , and from $𝐴 - 3 𝐼_{2} = (\begin{matrix} 4 & - 2 \\ - 2 & 1 \end{matrix})$ we get $𝐸_{3} = span ((1, 2))$ . These are orthogonal, as they must be. Normalizing,

𝑄 = \frac{1}{\sqrt{5}} (\begin{matrix} 2 & 1 \\ - 1 & 2 \end{matrix}), 𝐷 = (\begin{matrix} 8 & 0 \\ 0 & 3 \end{matrix}),

and one checks $𝐴 𝑄 =$ $\frac{1}{\sqrt{5}} (\begin{matrix} 16 & 3 \\ - 8 & 6 \end{matrix}) =$ $𝑄 𝐷$ .

Symmetric matrices and ellipses

Geometrically, the real spectral theorem says that a real symmetric matrix stretches $ℝ^{𝑛}$ by the factors $𝜆_{1}, \dots, 𝜆_{𝑛}$ along $𝑛$ perpendicular axes, with a flip when $𝜆_{𝑗} < 0$ . In the plane, this is visible on the unit circle.

Example 8.2.17 (The image of the unit circle). Let $𝐴 \in ℝ^{2 \times 2}$ be symmetric and invertible, and write $𝐴 = 𝑄 𝐷 𝑄^{𝖳}$ with $𝑄 = (𝐪_{1} | 𝐪_{2})$ orthogonal and $𝐷 = diag (𝜆_{1}, 𝜆_{2})$ . A unit vector is $𝐱 = 𝑦_{1} 𝐪_{1} + 𝑦_{2} 𝐪_{2}$ with $𝑦_{1}^{2} + 𝑦_{2}^{2} = 1$ (Proposition 6.3.18, Theorem 6.3.22), and $𝐴 𝐱 =$ $𝜆_{1} 𝑦_{1} 𝐪_{1} +$ $𝜆_{2} 𝑦_{2} 𝐪_{2}$ . So the image of the unit circle is

{𝑧_{1} 𝐪_{1} + 𝑧_{2} 𝐪_{2} | \frac{𝑧_{1}^{2}}{𝜆_{1}^{2}} + \frac{𝑧_{2}^{2}}{𝜆_{2}^{2}} = 1},

an ellipse whose axes point along the eigenvectors, with semi-axes $| 𝜆_{1} |$ and $| 𝜆_{2} |$ .

For instance, $𝐴 = (\begin{matrix} 1 & 2 \\ 2 & - 2 \end{matrix})$ has $𝑝_{𝐴} =$ $𝑥^{2} +$ $𝑥 -$ $6 =$ $(𝑥 - 2) (𝑥 + 3)$ , with $𝐸_{2} = span ((2, 1))$ and $𝐸_{- 3} = span ((- 1, 2))$ . So $𝐴 = 𝑄 𝐷 𝑄^{𝖳}$ with

𝑄 = \frac{1}{\sqrt{5}} (\begin{matrix} 2 & - 1 \\ 1 & 2 \end{matrix}), 𝐷 = (\begin{matrix} 2 & 0 \\ 0 & - 3 \end{matrix}),

and the image of the unit circle is an ellipse with semi-axis $2$ along $(2, 1)$ and $3$ along $(- 1, 2)$ . A non-symmetric matrix also maps the circle to an ellipse, but its axes are generally not along its eigenvectors; that is the subject of Section 9.1.

The unit circle and its image under the symmetric matrix

𝐴 = (\begin{matrix} 1 & 2 \\ 2 & - 2 \end{matrix})

. The eigenvectors

𝐪_{1}, 𝐪_{2}

are perpendicular, and

𝐴

scales them by

2

and

- 3

; the images lie along the axes of the ellipse. In the interactive version the two off-diagonal entries move together, so

𝐴

stays symmetric. Move the point

𝐱

around the circle:

𝐴 𝐱

is farthest from

𝟎

when

𝐱 = \pm 𝐪

for the eigenvalue of largest absolute value. Make an eigenvalue

0

to flatten the ellipse to a segment, and make

𝐴

a multiple of

𝐼

, where every direction is an eigenvector.

8.2.4The spectral decomposition

An orthonormal eigenbasis is not unique: in Example 8.2.15 any orthonormal basis of the plane $𝐸_{1}$ would do. What is unique is the splitting of $𝑉$ into eigenspaces, and the orthogonal projections onto them. Writing $𝑇$ in terms of these projections gives a basis-free form of the spectral theorems.

For the rest of the section, we say that $𝑇$ satisfies the spectral hypothesis if $𝑇$ is normal and $𝔽 = ℂ$ , or $𝑇$ is self-adjoint and $𝔽 = ℝ$ . By Theorem 8.2.5 and Theorem 8.2.12, these are exactly the cases in which $𝑉$ has an orthonormal eigenbasis. (Self-adjoint operators on complex spaces are included, being normal.)

Theorem 8.2.18 (Spectral decomposition). Let $𝑇$ satisfy the spectral hypothesis, let $𝜆_{1}, \dots, 𝜆_{𝑘}$ be the distinct eigenvalues of $𝑇$ , and let $𝑃_{𝑗} = {Pr}_{𝐸_{𝜆_{𝑗}}}$ be the orthogonal projection onto $𝐸_{𝜆_{𝑗}}$ .

(a)The eigenspaces are pairwise orthogonal, and $𝑉 = 𝐸_{𝜆_{1}} \oplus \dots \oplus 𝐸_{𝜆_{𝑘}}$ .
(b) $𝑃_{𝑖} 𝑃_{𝑗} = 0$ for $𝑖 \neq 𝑗$ , and $𝑃_{1} + \dots + 𝑃_{𝑘} = {id}_{𝑉}$ .
(c) $𝑇 =$ $𝜆_{1} 𝑃_{1} +$ $\dots +$ $𝜆_{𝑘} 𝑃_{𝑘}$ .

In matrix form: if $𝐴 = 𝑈 𝐷 𝑈^{*}$ with $𝑈$ unitary (orthogonal, if $𝔽 = ℝ$ ) with columns $𝐮_{1}, \dots, 𝐮_{𝑛}$ and $𝐷 =$ $diag (𝜇_{1}, \dots, 𝜇_{𝑛})$ , then

𝐴 = 𝜇_{1} 𝐮_{1} 𝐮_{1}^{*} + \dots + 𝜇_{𝑛} 𝐮_{𝑛} 𝐮_{𝑛}^{*},

and grouping the terms with equal $𝜇_{𝑖}$ gives $𝐴 = \sum_{𝑗} 𝜆_{𝑗} 𝑃_{𝑗}$ , where $𝑃_{𝑗}$ is the matrix of ${Pr}_{𝐸_{𝜆_{𝑗}}}$ .

Proof.standard

Uses. Theorem 8.2.5 (Complex spectral theorem), Theorem 8.2.12 (Real spectral theorem), Proposition 6.3.18 (Coordinates in an orthonormal basis), Definition 6.4.11 (Orthogonal projection), Proposition 6.4.29 (The orthogonal projection as a linear map), Theorem 4.4.13 (Bases and dimension of a sum), Theorem 6.4.31 (The matrix of an orthogonal projection), Proposition 2.4.8 (Four views of the product)

Techniques. Pass to orthonormal coordinates (Technique 6.7.2)

Hint 1. Take an orthonormal eigenbasis

𝐮_{1}, \dots, 𝐮_{𝑛}

and group its vectors by eigenvalue. Show that the group belonging to

𝜆_{𝑗}

spans

𝐸_{𝜆_{𝑗}}

Hint 2. If

𝐯 = \sum 𝑐_{𝑖} 𝐮_{𝑖}

is in

𝐸_{𝜆_{𝑗}}

, compare the coordinates of

𝑇 𝐯

and

𝜆_{𝑗} 𝐯

Hint 3. With an orthonormal basis of

𝐸_{𝜆_{𝑗}}

in hand,

𝑃_{𝑗} 𝐯

is the part of the expansion

𝐯 = \sum ⟨ 𝐯, 𝐮_{𝑖} ⟩ 𝐮_{𝑖}

coming from that group.

Sketch. Let

𝐮_{1}, \dots, 𝐮_{𝑛}

be an orthonormal eigenbasis with

𝑇 𝐮_{𝑖} = 𝜇_{𝑖} 𝐮_{𝑖}

, and let

{𝒪︀}_{𝑗}

consist of the

𝐮_{𝑖}

with

𝜇_{𝑖} = 𝜆_{𝑗}

. Comparing coordinates shows

𝐸_{𝜆_{𝑗}} =

span ({𝒪︀}_{𝑗})

. So the eigenspaces are spanned by disjoint pieces of one orthonormal basis, which gives (a), and

𝑃_{𝑗} 𝐯 =

\sum_{𝐮_{𝑖} \in {𝒪︀}_{𝑗}} ⟨ 𝐯, 𝐮_{𝑖} ⟩ 𝐮_{𝑖}

. Summing over

𝑗

gives

𝐯

, and applying

𝑇

to the expansion of

𝐯

gives

\sum_{𝑗} 𝜆_{𝑗} 𝑃_{𝑗} 𝐯

By Theorem 8.2.5 or Theorem 8.2.12, $𝑉$ has an orthonormal basis $𝐮_{1}, \dots, 𝐮_{𝑛}$ with $𝑇 𝐮_{𝑖} = 𝜇_{𝑖} 𝐮_{𝑖}$ , and every $𝜇_{𝑖}$ is one of $𝜆_{1}, \dots, 𝜆_{𝑘}$ . For each $𝑗$ , let ${𝒪︀}_{𝑗}$ be the list of those $𝐮_{𝑖}$ with $𝜇_{𝑖} = 𝜆_{𝑗}$ .

We claim $𝐸_{𝜆_{𝑗}} =$ $span ({𝒪︀}_{𝑗})$ . Each vector of ${𝒪︀}_{𝑗}$ lies in $𝐸_{𝜆_{𝑗}}$ , which gives $\supseteq$ . Conversely, let $𝐯 \in 𝐸_{𝜆_{𝑗}}$ and write $𝐯 = \sum_{𝑖} 𝑐_{𝑖} 𝐮_{𝑖}$ . Then

\sum_{𝑖} 𝑐_{𝑖} 𝜇_{𝑖} 𝐮_{𝑖} = 𝑇 𝐯 = 𝜆_{𝑗} 𝐯 = \sum_{𝑖} 𝑐_{𝑖} 𝜆_{𝑗} 𝐮_{𝑖},

so $𝑐_{𝑖} (𝜇_{𝑖} - 𝜆_{𝑗}) = 0$ for every $𝑖$ , by uniqueness of coordinates. Hence $𝑐_{𝑖} = 0$ unless $𝜇_{𝑖} = 𝜆_{𝑗}$ , and $𝐯 \in span ({𝒪︀}_{𝑗})$ .

(a) For $𝑖 \neq 𝑗$ , ${𝒪︀}_{𝑖}$ and ${𝒪︀}_{𝑗}$ have no vector in common, so every vector of ${𝒪︀}_{𝑖}$ is orthogonal to every vector of ${𝒪︀}_{𝑗}$ , and hence every vector of $span ({𝒪︀}_{𝑖}) =$ $𝐸_{𝜆_{𝑖}}$ is orthogonal to every vector of $𝐸_{𝜆_{𝑗}}$ . The lists ${𝒪︀}_{1}, \dots, {𝒪︀}_{𝑘}$ together are a reordering of a basis of $𝑉$ , so $𝑉 = 𝐸_{𝜆_{1}} \oplus \dots \oplus 𝐸_{𝜆_{𝑘}}$ by Theorem 4.4.13.

(b) Since ${𝒪︀}_{𝑗}$ is an orthonormal basis of $𝐸_{𝜆_{𝑗}}$ , Definition 6.4.11 gives

𝑃_{𝑗} 𝐯 = \sum_{𝐮_{𝑖} \in {𝒪︀}_{𝑗}} ⟨ 𝐯, 𝐮_{𝑖} ⟩ 𝐮_{𝑖} .

(8.2.1)

Summing over $𝑗$ gives $\sum_{𝑗} 𝑃_{𝑗} 𝐯 =$ $\sum_{𝑖 = 1}^{𝑛} ⟨ 𝐯, 𝐮_{𝑖} ⟩ 𝐮_{𝑖} =$ $𝐯$ by Proposition 6.3.18, so $\sum_{𝑗} 𝑃_{𝑗} = {id}_{𝑉}$ . For $𝑖 \neq 𝑗$ , $𝑃_{𝑗} 𝐯 \in$ $𝐸_{𝜆_{𝑗}} \subseteq$ $𝐸_{𝜆_{𝑖}}^{⟂} =$ $ker 𝑃_{𝑖}$ by (a) and Proposition 6.4.29(a), so $𝑃_{𝑖} 𝑃_{𝑗} = 0$ .

(c) By Proposition 6.3.18 and linearity,

𝑇 𝐯 = \sum_{𝑖 = 1}^{𝑛} ⟨ 𝐯, 𝐮_{𝑖} ⟩ 𝑇 𝐮_{𝑖} = \sum_{𝑖 = 1}^{𝑛} 𝜇_{𝑖} ⟨ 𝐯, 𝐮_{𝑖} ⟩ 𝐮_{𝑖} = \sum_{𝑗 = 1}^{𝑘} 𝜆_{𝑗} \sum_{𝐮_{𝑖} \in {𝒪︀}_{𝑗}} ⟨ 𝐯, 𝐮_{𝑖} ⟩ 𝐮_{𝑖} = \sum_{𝑗 = 1}^{𝑘} 𝜆_{𝑗} 𝑃_{𝑗} 𝐯,

by (8.2.1).

For the matrix form, $𝐴 =$ $𝑈 𝐷 𝑈^{*} =$ $(𝑈 𝐷) 𝑈^{*}$ , and the column–row expansion of a product (Proposition 2.4.8) gives $\sum_{𝑖} (𝜇_{𝑖} 𝐮_{𝑖}) 𝐮_{𝑖}^{*}$ . By Theorem 6.4.31, $\sum_{𝐮_{𝑖} \in {𝒪︀}_{𝑗}} 𝐮_{𝑖} 𝐮_{𝑖}^{*}$ is the matrix of ${Pr}_{𝐸_{𝜆_{𝑗}}}$ , where $𝐸_{𝜆_{𝑗}}$ is now the eigenspace of $𝐴$ in $𝔽^{𝑛}$ . □

So a normal operator is a linear combination of orthogonal projections onto mutually perpendicular subspaces that fill $𝑉$ . The converse also holds, and the decomposition is unique, which is what makes it useful for defining things.

Proposition 8.2.19 (Uniqueness of the spectral decomposition). Let $𝜇_{1}, \dots, 𝜇_{𝑚} \in 𝔽$ be distinct, and let $𝑄_{1}, \dots, 𝑄_{𝑚} \in L (𝑉)$ be nonzero orthogonal projections with $𝑄_{𝑖} 𝑄_{𝑗} = 0$ for $𝑖 \neq 𝑗$ and $𝑄_{1} + \dots + 𝑄_{𝑚} = {id}_{𝑉}$ . Let $𝑇 =$ $𝜇_{1} 𝑄_{1} +$ $\dots +$ $𝜇_{𝑚} 𝑄_{𝑚}$ . Then:

(a) $𝑇$ is normal, and self-adjoint if every $𝜇_{𝑗}$ is real;
(b)the eigenvalues of $𝑇$ are exactly $𝜇_{1}, \dots, 𝜇_{𝑚}$ , and $𝑄_{𝑗} = {Pr}_{𝐸_{𝜇_{𝑗}}}$ for every $𝑗$ .

Proof.standard

Uses. Corollary 6.6.29 (Orthogonal projections are the self-adjoint idempotents), Proposition 6.6.18 (Properties of the adjoint), Definition 7.2.13 (Eigenspace), Proposition 6.4.29 (The orthogonal projection as a linear map)

Hint 1. Each

𝑄_{𝑗}

is self-adjoint (Corollary 6.6.29). Compute

𝑇^{*}

Hint 2. For (b), first show that

𝑇

acts as

𝜇_{𝑗}

im 𝑄_{𝑗}

. Then take an eigenvector

𝐯

with eigenvalue

𝜆

, apply

𝑄_{𝑖}

𝑇 𝐯 = 𝜆 𝐯

, and use

𝑄_{𝑖} 𝑄_{𝑗} = 0

Sketch. (a)

𝑇^{*} = \sum 𝜇_{𝑗} 𝑄_{𝑗}

, and

𝑇 𝑇^{*} =

\sum_{𝑗} {| 𝜇_{𝑗} |}^{2} 𝑄_{𝑗} =

𝑇^{*} 𝑇

because

𝑄_{𝑖} 𝑄_{𝑗} = 0

for

𝑖 \neq 𝑗

and

𝑄_{𝑗}^{2} = 𝑄_{𝑗}

. (b) If

𝐯 \in im 𝑄_{𝑗}

then

𝑄_{𝑖} 𝐯 =

𝑄_{𝑖} 𝑄_{𝑗} 𝐯 =

0

for

𝑖 \neq 𝑗

, so

𝑇 𝐯 = 𝜇_{𝑗} 𝐯

im 𝑄_{𝑗} \subseteq 𝐸_{𝜇_{𝑗}}

. If

𝑇 𝐯 = 𝜆 𝐯

, applying

𝑄_{𝑖}

gives

𝜇_{𝑖} 𝑄_{𝑖} 𝐯 =

𝜆 𝑄_{𝑖} 𝐯

, so

𝑄_{𝑖} 𝐯 = 𝟎

unless

𝜇_{𝑖} = 𝜆

; as

𝐯 = \sum 𝑄_{𝑖} 𝐯 \neq 𝟎

𝜆

is some

𝜇_{𝑗}

and

𝐯 = 𝑄_{𝑗} 𝐯 \in im 𝑄_{𝑗}

Each $𝑄_{𝑗}$ is an orthogonal projection, so $𝑄_{𝑗}^{2} = 𝑄_{𝑗} = 𝑄_{𝑗}^{*}$ (Corollary 6.6.29).

(a) By Proposition 6.6.18, $𝑇^{*} = \sum_{𝑗} 𝜇_{𝑗} 𝑄_{𝑗}$ . Expanding the products and using $𝑄_{𝑖} 𝑄_{𝑗} = 0$ for $𝑖 \neq 𝑗$ and $𝑄_{𝑗}^{2} = 𝑄_{𝑗}$ ,

𝑇 𝑇^{*} = \sum_{𝑖, 𝑗} 𝜇_{𝑖} 𝜇_{𝑗} 𝑄_{𝑖} 𝑄_{𝑗} = \sum_{𝑗} {| 𝜇_{𝑗} |}^{2} 𝑄_{𝑗} = \sum_{𝑖, 𝑗} 𝜇_{𝑗} 𝜇_{𝑖} 𝑄_{𝑗} 𝑄_{𝑖} = 𝑇^{*} 𝑇 .

If every $𝜇_{𝑗}$ is real, then $𝑇^{*} = 𝑇$ .

(b) Let $𝐯 \in im 𝑄_{𝑗}$ , say $𝐯 = 𝑄_{𝑗} 𝐱$ . Then $𝑄_{𝑗} 𝐯 = 𝑄_{𝑗}^{2} 𝐱 = 𝐯$ and $𝑄_{𝑖} 𝐯 =$ $𝑄_{𝑖} 𝑄_{𝑗} 𝐱 =$ $𝟎$ for $𝑖 \neq 𝑗$ , so $𝑇 𝐯 = 𝜇_{𝑗} 𝐯$ . Hence $im 𝑄_{𝑗} \subseteq 𝐸_{𝜇_{𝑗}}$ , and since $𝑄_{𝑗} \neq 0$ , $𝜇_{𝑗}$ is an eigenvalue.

Conversely, let $𝑇 𝐯 = 𝜆 𝐯$ with $𝐯 \neq 𝟎$ . For each $𝑖$ , $𝑄_{𝑖} 𝑇 =$ $\sum_{𝑗} 𝜇_{𝑗} 𝑄_{𝑖} 𝑄_{𝑗} =$ $𝜇_{𝑖} 𝑄_{𝑖}$ , so applying $𝑄_{𝑖}$ to $𝑇 𝐯 = 𝜆 𝐯$ gives $𝜇_{𝑖} 𝑄_{𝑖} 𝐯 =$ $𝜆 𝑄_{𝑖} 𝐯$ , that is, $(𝜇_{𝑖} - 𝜆) 𝑄_{𝑖} 𝐯 =$ $𝟎$ . Since $𝐯 = \sum_{𝑖} 𝑄_{𝑖} 𝐯 \neq 𝟎$ , some $𝑄_{𝑗} 𝐯 \neq 𝟎$ , and then $𝜆 = 𝜇_{𝑗}$ . Moreover $𝑄_{𝑖} 𝐯 = 𝟎$ for all $𝑖 \neq 𝑗$ (as $𝜇_{𝑖} \neq 𝜇_{𝑗} = 𝜆$ ), so $𝐯 = 𝑄_{𝑗} 𝐯 \in im 𝑄_{𝑗}$ . This shows that the eigenvalues are exactly the $𝜇_{𝑗}$ and that $𝐸_{𝜇_{𝑗}} = im 𝑄_{𝑗}$ . Finally, an orthogonal projection is the orthogonal projection onto its image (Corollary 6.6.29), so $𝑄_{𝑗} =$ ${Pr}_{im 𝑄_{𝑗}} =$ ${Pr}_{𝐸_{𝜇_{𝑗}}}$ . □

The projections $𝑃_{𝑗}$ can be computed without finding any eigenvectors. On $𝐸_{𝜆_{𝑖}}$ , a polynomial $𝑝 (𝑇)$ acts as the scalar $𝑝 (𝜆_{𝑖})$ (Proposition 7.2.28), so a polynomial that is $1$ at $𝜆_{𝑗}$ and $0$ at the other eigenvalues acts as $𝑃_{𝑗}$ . These are the Lagrange polynomials.

Proposition 8.2.20 (Spectral projections are polynomials in $𝑇$ ). Let $𝑇$ satisfy the spectral hypothesis, with distinct eigenvalues $𝜆_{1}, \dots, 𝜆_{𝑘}$ and $𝑃_{𝑗} = {Pr}_{𝐸_{𝜆_{𝑗}}}$ . Then

𝑃_{𝑗} = 𝐿_{𝑗} (𝑇), where 𝐿_{𝑗} = \prod_{𝑖 \neq 𝑗} \frac{𝑥 - 𝜆_{𝑖}}{𝜆_{𝑗} - 𝜆_{𝑖}} .

Consequently:

(a)every operator $𝑆 \in L (𝑉)$ with $𝑆 𝑇 = 𝑇 𝑆$ commutes with every $𝑃_{𝑗}$ and maps every eigenspace $𝐸_{𝜆_{𝑗}}$ into itself;
(b)if $𝔽 = ℂ$ , then $𝑇^{*} = 𝑝 (𝑇)$ for the polynomial $𝑝 = \sum_{𝑗} 𝜆_{𝑗} 𝐿_{𝑗}$ .

Proof.standard

Uses. Theorem 8.2.18 (Spectral decomposition), Proposition 7.1.29 (Lagrange polynomials in closed form), Proposition 7.2.28 (Polynomials act on eigenvectors by their values), Proposition 7.2.27 (Evaluating polynomials at an operator), Corollary 6.6.29 (Orthogonal projections are the self-adjoint idempotents), Proposition 6.6.18 (Properties of the adjoint)

Techniques. Test an identity on eigenvectors (Technique 7.4.25)

Hint 1. Both

𝑃_{𝑗}

and

𝐿_{𝑗} (𝑇)

are linear, and

𝑉

is the sum of the eigenspaces. Compare them on a vector of

𝐸_{𝜆_{𝑖}}

Hint 2. For (a),

𝑆

commutes with

𝑇

, hence with every polynomial in

𝑇

Hint 3. For (b), take adjoints in

𝑇 = \sum_{𝑗} 𝜆_{𝑗} 𝑃_{𝑗}

Sketch. On

𝐮 \in 𝐸_{𝜆_{𝑖}}

𝐿_{𝑗} (𝑇) 𝐮 =

𝐿_{𝑗} (𝜆_{𝑖}) 𝐮

, which is

𝐮

𝑖 = 𝑗

and

𝟎

otherwise, and

𝑃_{𝑗}

does the same. Since

𝑉 = \oplus_{𝑖} 𝐸_{𝜆_{𝑖}}

𝑃_{𝑗} = 𝐿_{𝑗} (𝑇)

. Then (a):

𝑆

commutes with

𝑇^{𝑚}

for all

𝑚

, hence with

𝐿_{𝑗} (𝑇) = 𝑃_{𝑗}

, and

𝑆 𝑃_{𝑗} 𝐯 =

𝑃_{𝑗} 𝑆 𝐯 \in

im 𝑃_{𝑗}

. (b)

𝑇^{*} =

\sum 𝜆_{𝑗} 𝑃_{𝑗}^{*} =

\sum 𝜆_{𝑗} 𝐿_{𝑗} (𝑇)

The formula for $𝐿_{𝑗}$ is the Lagrange polynomial for the points $𝜆_{1}, \dots, 𝜆_{𝑘}$ (Proposition 7.1.29), so $𝐿_{𝑗} (𝜆_{𝑗}) = 1$ and $𝐿_{𝑗} (𝜆_{𝑖}) = 0$ for $𝑖 \neq 𝑗$ . (If $𝑘 = 1$ , $𝐿_{1} = 1$ and the claim is $𝑃_{1} = {id}_{𝑉}$ .) Let $𝐮 \in 𝐸_{𝜆_{𝑖}}$ . By Proposition 7.2.28, $𝐿_{𝑗} (𝑇) 𝐮 =$ $𝐿_{𝑗} (𝜆_{𝑖}) 𝐮$ . On the other hand, $𝑃_{𝑗} 𝐮 = 𝐮$ if $𝑖 = 𝑗$ , and $𝑃_{𝑗} 𝐮 = 𝟎$ if $𝑖 \neq 𝑗$ , since then $𝐮 \in$ $𝐸_{𝜆_{𝑗}}^{⟂} =$ $ker 𝑃_{𝑗}$ by Theorem 8.2.18(a). So $𝑃_{𝑗}$ and $𝐿_{𝑗} (𝑇)$ agree on every eigenspace. Every $𝐯 \in 𝑉$ is a sum of vectors from the eigenspaces (Theorem 8.2.18(a)), so $𝑃_{𝑗} = 𝐿_{𝑗} (𝑇)$ by linearity.

(a) If $𝑆 𝑇 = 𝑇 𝑆$ , then $𝑆 𝑇^{𝑚} = 𝑇^{𝑚} 𝑆$ for every $𝑚 \geq 0$ by induction on $𝑚$ , and so $𝑆 𝑞 (𝑇) = 𝑞 (𝑇) 𝑆$ for every polynomial $𝑞$ . In particular $𝑆 𝑃_{𝑗} = 𝑃_{𝑗} 𝑆$ , and for $𝐯 \in 𝐸_{𝜆_{𝑗}} = im 𝑃_{𝑗}$ we get $𝑆 𝐯 =$ $𝑆 𝑃_{𝑗} 𝐯 =$ $𝑃_{𝑗} 𝑆 𝐯 \in$ $im 𝑃_{𝑗} =$ $𝐸_{𝜆_{𝑗}}$ .

(b) By Theorem 8.2.18(c), Proposition 6.6.18, and $𝑃_{𝑗}^{*} = 𝑃_{𝑗}$ (Corollary 6.6.29), $𝑇^{*} =$ $\sum_{𝑗} 𝜆_{𝑗} 𝑃_{𝑗} =$ $\sum_{𝑗} 𝜆_{𝑗} 𝐿_{𝑗} (𝑇) =$ $𝑝 (𝑇)$ by Proposition 7.2.27. □

Part (b) gives another description of normality: on a finite-dimensional complex space, $𝑇$ is normal if and only if $𝑇^{*}$ is a polynomial in $𝑇$ . (The converse holds because polynomials in $𝑇$ commute with $𝑇$ .)

Example 8.2.21 (A spectral decomposition). For $𝐴 = 𝐼_{3} + 𝐽_{3}$ of Example 8.2.15, the eigenvalues are $4$ and $1$ , so

𝑃_{4} = 𝐿_{4} (𝐴) = \frac{𝐴 - 𝐼_{3}}{4 - 1} = \frac{1}{3} 𝐽_{3}, 𝑃_{1} = 𝐿_{1} (𝐴) = \frac{𝐴 - 4 𝐼_{3}}{1 - 4} = 𝐼_{3} - \frac{1}{3} 𝐽_{3} .

Indeed $\frac{1}{3} 𝐽_{3} = 𝐪_{1} 𝐪_{1}^{𝖳}$ for $𝐪_{1} = \frac{1}{\sqrt{3}} (1, 1, 1)$ , as Theorem 6.4.31 predicts, and $𝑃_{1}$ is the projection onto the plane $𝑥_{1} + 𝑥_{2} + 𝑥_{3} = 0$ . The decomposition reads

𝐴 = 4 \cdot \frac{1}{3} 𝐽_{3} + 1 \cdot (𝐼_{3} - \frac{1}{3} 𝐽_{3}) .

We found $𝑃_{1}$ without choosing a basis of the plane, which is exactly the choice that was not unique.

Exercise 8.2.22.routine Let

𝐻 = (\begin{matrix} 3 & 𝑖 \\ - 𝑖 & 3 \end{matrix})

. Find the spectral decomposition

𝐻 = 𝜆_{1} 𝑃_{1} + 𝜆_{2} 𝑃_{2}

, once with Lagrange polynomials and once from unit eigenvectors, and check that

𝑃_{1} 𝑃_{2} = 0

Solution.

Uses. Proposition 8.2.20 (Spectral projections are polynomials in $𝑇$ ), Theorem 8.2.18 (Spectral decomposition), Theorem 6.4.31 (The matrix of an orthogonal projection)

Hint 1.

𝑝_{𝐻} = {(𝑥 - 3)}^{2} - 1

Hint 2. With eigenvalues

2

and

4

𝑃_{4} = (𝐻 - 2 𝐼_{2}) / 2

and

𝑃_{2} =

(𝐻 - 4 𝐼_{2}) / (- 2)

Sketch. Eigenvalues

2, 4

𝑃_{4} =

\frac{1}{2} (\begin{matrix} 1 & 𝑖 \\ - 𝑖 & 1 \end{matrix}) =

𝐮 𝐮^{*}

with

𝐮 = \frac{1}{\sqrt{2}} (𝑖, 1)

, and

𝑃_{2} = \frac{1}{2} (\begin{matrix} 1 & - 𝑖 \\ 𝑖 & 1 \end{matrix})

𝑃_{4} 𝑃_{2} =

\frac{1}{4} (\begin{matrix} 1 + 𝑖^{2} & - 𝑖 + 𝑖 \\ - 𝑖 + 𝑖 & 𝑖^{2} + 1 \end{matrix}) =

0

$𝑝_{𝐻} =$ ${(𝑥 - 3)}^{2} -$ $𝑖 (- 𝑖) =$ ${(𝑥 - 3)}^{2} -$ $1 =$ $(𝑥 - 2) (𝑥 - 4)$ . By Proposition 8.2.20,

𝑃_{4} = \frac{𝐻 - 2 𝐼_{2}}{4 - 2} = \frac{1}{2} (\begin{matrix} 1 & 𝑖 \\ - 𝑖 & 1 \end{matrix}), 𝑃_{2} = \frac{𝐻 - 4 𝐼_{2}}{2 - 4} = \frac{1}{2} (\begin{matrix} 1 & - 𝑖 \\ 𝑖 & 1 \end{matrix}) .

From eigenvectors: $𝐻 (𝑖, 1) =$ $(3 𝑖 + 𝑖, 1 + 3) =$ $4 (𝑖, 1)$ and $𝐻 (- 𝑖, 1) =$ $(- 3 𝑖 + 𝑖, - 1 + 3) =$ $2 (- 𝑖, 1)$ . With $𝐮_{1} = \frac{1}{\sqrt{2}} (𝑖, 1)$ and $𝐮_{2} = \frac{1}{\sqrt{2}} (- 𝑖, 1)$ , Theorem 6.4.31 gives

𝐮_{1} 𝐮_{1}^{*} = \frac{1}{2} (\begin{matrix} 𝑖 \\ 1 \end{matrix}) (\begin{matrix} - 𝑖 & 1 \end{matrix}) = \frac{1}{2} (\begin{matrix} 1 & 𝑖 \\ - 𝑖 & 1 \end{matrix}) = 𝑃_{4}, 𝐮_{2} 𝐮_{2}^{*} = \frac{1}{2} (\begin{matrix} - 𝑖 \\ 1 \end{matrix}) (\begin{matrix} 𝑖 & 1 \end{matrix}) = \frac{1}{2} (\begin{matrix} 1 & - 𝑖 \\ 𝑖 & 1 \end{matrix}) = 𝑃_{2} .

So $𝐻 = 4 𝑃_{4} + 2 𝑃_{2}$ . Finally

𝑃_{4} 𝑃_{2} = \frac{1}{4} (\begin{matrix} 1 \cdot 1 + 𝑖 \cdot 𝑖 & 1 \cdot (- 𝑖) + 𝑖 \cdot 1 \\ - 𝑖 \cdot 1 + 1 \cdot 𝑖 & (- 𝑖) (- 𝑖) + 1 \cdot 1 \end{matrix}) = \frac{1}{4} (\begin{matrix} 0 & 0 \\ 0 & 0 \end{matrix}) .

8.2.5Functions of an operator

On an eigenspace $𝐸_{𝜆}$ , the operator $𝑇$ is just multiplication by $𝜆$ , so $𝑇^{2}$ is multiplication by $𝜆^{2}$ , $𝑇^{- 1}$ by $𝜆^{- 1}$ , and so on. The spectral decomposition lets us do this with any function of $𝜆$ , not only with powers.

Definition 8.2.23 (Functions of an operator). Let $𝑇$ satisfy the spectral hypothesis, with distinct eigenvalues $𝜆_{1}, \dots, 𝜆_{𝑘}$ and $𝑃_{𝑗} = {Pr}_{𝐸_{𝜆_{𝑗}}}$ . For a function $𝑓 : {𝜆_{1}, \dots, 𝜆_{𝑘}} \to$ $𝔽$ , define

𝑓 (𝑇) = 𝑓 (𝜆_{1}) 𝑃_{1} + \dots + 𝑓 (𝜆_{𝑘}) 𝑃_{𝑘} .

For a matrix $𝐴 = 𝑈 𝐷 𝑈^{*}$ as in Theorem 8.2.18, with $𝐷 =$ $diag (𝜇_{1}, \dots, 𝜇_{𝑛})$ , this is

𝑓 (𝐴) = 𝑈 diag (𝑓 (𝜇_{1}), \dots, 𝑓 (𝜇_{𝑛})) 𝑈^{*} .

The two formulas agree by the matrix form of Theorem 8.2.18, grouping equal $𝜇_{𝑖}$ . In particular, $𝑓 (𝐴)$ does not depend on the choice of $𝑈$ , because the projections $𝑃_{𝑗}$ do not. Note also that $𝑓$ only needs to be defined on the eigenvalues. When $𝔽 = ℝ$ , $𝑓$ is real-valued and $𝑓 (𝑇)$ is an operator on the real space $𝑉$ .

Proposition 8.2.24 (Properties of $𝑓 (𝑇)$ ). Let $𝑇$ satisfy the spectral hypothesis, with distinct eigenvalues $𝜆_{1}, \dots, 𝜆_{𝑘}$ , and let $𝑓, 𝑔$ be functions from ${𝜆_{1}, \dots, 𝜆_{𝑘}}$ to $𝔽$ , and $𝑐 \in 𝔽$ .

(a) $𝑓 (𝑇) 𝐯 =$ $𝑓 (𝜆_{𝑗}) 𝐯$ for every $𝐯 \in 𝐸_{𝜆_{𝑗}}$ .
(b)If $𝑝 \in 𝔽 [𝑥]$ and $𝑓 (𝜆_{𝑗}) = 𝑝 (𝜆_{𝑗})$ for every $𝑗$ , then $𝑓 (𝑇) = 𝑝 (𝑇)$ . In particular, $𝑓 (𝑇) = 𝑝 (𝑇)$ for the interpolating polynomial $𝑝 =$ $\sum_{𝑗} 𝑓 (𝜆_{𝑗}) 𝐿_{𝑗}$ of Proposition 8.2.20, and $𝑓 (𝑇)$ commutes with every operator that commutes with $𝑇$ .
(c) $(𝑓 + 𝑔) (𝑇) =$ $𝑓 (𝑇) +$ $𝑔 (𝑇)$ , $(𝑐 𝑓) (𝑇) =$ $𝑐 𝑓 (𝑇)$ , and $(𝑓 𝑔) (𝑇) =$ $𝑓 (𝑇) 𝑔 (𝑇)$ .
(d) ${𝑓 (𝑇)}^{*} = 𝑓 (𝑇)$ , where $𝑓 (𝜆) = 𝑓 (𝜆)$ . So $𝑓 (𝑇)$ is normal, and it is self-adjoint if and only if every $𝑓 (𝜆_{𝑗})$ is real.
(e)The eigenvalues of $𝑓 (𝑇)$ are the values $𝑓 (𝜆_{1}),$ $\dots,$ $𝑓 (𝜆_{𝑘})$ .

Proof.standard

Uses. Definition 8.2.23 (Functions of an operator), Theorem 8.2.18 (Spectral decomposition), Proposition 7.2.28 (Polynomials act on eigenvectors by their values), Proposition 8.2.20 (Spectral projections are polynomials in $𝑇$ ), Proposition 6.6.18 (Properties of the adjoint), Corollary 6.6.29 (Orthogonal projections are the self-adjoint idempotents), Proposition 8.2.19 (Uniqueness of the spectral decomposition)

Techniques. Test an identity on eigenvectors (Technique 7.4.25)

Hint 1. Prove (a) first, from

𝑃_{𝑖} 𝐯 = 𝟎

for

𝑖 \neq 𝑗

and

𝑃_{𝑗} 𝐯 = 𝐯

Hint 2. Then every identity between operators built from

𝑇

can be tested on each eigenspace, where everything is a scalar.

𝑉

is the sum of the eigenspaces.

Hint 3. For (d), take adjoints term by term. For (e), use Proposition 8.2.19, after grouping equal values of

𝑓

Sketch. (a) is immediate from

𝑃_{𝑖} 𝑃_{𝑗} = 0

and

𝑃_{𝑗} 𝐯 = 𝐯

𝐸_{𝜆_{𝑗}}

. Since

𝑉 = \oplus 𝐸_{𝜆_{𝑗}}

, two operators agree as soon as they agree on each

𝐸_{𝜆_{𝑗}}

; on

𝐸_{𝜆_{𝑗}}

𝑝 (𝑇)

𝑓 (𝑇) + 𝑔 (𝑇)

, and

𝑓 (𝑇) 𝑔 (𝑇)

act as

𝑝 (𝜆_{𝑗})

𝑓 (𝜆_{𝑗}) + 𝑔 (𝜆_{𝑗})

, and

𝑓 (𝜆_{𝑗}) 𝑔 (𝜆_{𝑗})

, which gives (b) and (c). (d)

{𝑓 (𝑇)}^{*} =

\sum 𝑓 (𝜆_{𝑗}) 𝑃_{𝑗}

, and

𝑓 (𝑇)

commutes with

𝑓 (𝑇)

by (c). (e) Group the

𝑃_{𝑗}

by the value of

𝑓

and apply Proposition 8.2.19.

By Theorem 8.2.18(a), $𝑉 = 𝐸_{𝜆_{1}} \oplus \dots \oplus 𝐸_{𝜆_{𝑘}}$ , so two linear maps on $𝑉$ are equal as soon as they agree on each $𝐸_{𝜆_{𝑗}}$ .

(a) Let $𝐯 \in 𝐸_{𝜆_{𝑗}}$ . Then $𝑃_{𝑗} 𝐯 = 𝐯$ , and $𝑃_{𝑖} 𝐯 = 𝟎$ for $𝑖 \neq 𝑗$ since $𝐯 \in 𝐸_{𝜆_{𝑖}}^{⟂}$ . So $𝑓 (𝑇) 𝐯 =$ $𝑓 (𝜆_{𝑗}) 𝐯$ .

(b) On $𝐸_{𝜆_{𝑗}}$ , $𝑝 (𝑇)$ acts as $𝑝 (𝜆_{𝑗})$ (Proposition 7.2.28) and $𝑓 (𝑇)$ as $𝑓 (𝜆_{𝑗})$ by (a). These agree, so $𝑓 (𝑇) = 𝑝 (𝑇)$ . The interpolating polynomial satisfies $𝑝 (𝜆_{𝑖}) =$ $\sum_{𝑗} 𝑓 (𝜆_{𝑗}) 𝐿_{𝑗} (𝜆_{𝑖}) =$ $𝑓 (𝜆_{𝑖})$ . If $𝑆 𝑇 = 𝑇 𝑆$ , then $𝑆$ commutes with $𝑝 (𝑇) = 𝑓 (𝑇)$ , as in the proof of Proposition 8.2.20(a).

(c) On $𝐸_{𝜆_{𝑗}}$ , $𝑓 (𝑇) + 𝑔 (𝑇)$ and $𝑐 𝑓 (𝑇)$ act as $𝑓 (𝜆_{𝑗}) + 𝑔 (𝜆_{𝑗})$ and $𝑐 𝑓 (𝜆_{𝑗})$ . For $𝐯 \in 𝐸_{𝜆_{𝑗}}$ , $𝑔 (𝑇) 𝐯 =$ $𝑔 (𝜆_{𝑗}) 𝐯 \in$ $𝐸_{𝜆_{𝑗}}$ , so $𝑓 (𝑇) 𝑔 (𝑇) 𝐯 =$ $𝑔 (𝜆_{𝑗}) 𝑓 (𝑇) 𝐯 =$ $𝑓 (𝜆_{𝑗}) 𝑔 (𝜆_{𝑗}) 𝐯 =$ $(𝑓 𝑔) (𝑇) 𝐯$ .

(d) By Proposition 6.6.18 and $𝑃_{𝑗}^{*} = 𝑃_{𝑗}$ (Corollary 6.6.29), ${𝑓 (𝑇)}^{*} =$ $\sum_{𝑗} 𝑓 (𝜆_{𝑗}) 𝑃_{𝑗} =$ $𝑓 (𝑇)$ . By (c), $𝑓 (𝑇) {𝑓 (𝑇)}^{*} =$ $(𝑓 𝑓) (𝑇) =$ $(𝑓 𝑓) (𝑇) =$ ${𝑓 (𝑇)}^{*} 𝑓 (𝑇)$ . If every $𝑓 (𝜆_{𝑗})$ is real, $𝑓 = 𝑓$ and ${𝑓 (𝑇)}^{*} = 𝑓 (𝑇)$ . Conversely, if ${𝑓 (𝑇)}^{*} = 𝑓 (𝑇)$ , then applying both sides to a nonzero $𝐯 \in 𝐸_{𝜆_{𝑗}}$ and using (a) gives $𝑓 (𝜆_{𝑗}) = 𝑓 (𝜆_{𝑗})$ .

(e) Let $𝜈_{1}, \dots, 𝜈_{𝑚}$ be the distinct values among $𝑓 (𝜆_{1}),$ $\dots,$ $𝑓 (𝜆_{𝑘})$ , and let $𝑄_{𝑖}$ be the sum of the $𝑃_{𝑗}$ with $𝑓 (𝜆_{𝑗}) = 𝜈_{𝑖}$ . Then $𝑓 (𝑇) =$ $\sum_{𝑖} 𝜈_{𝑖} 𝑄_{𝑖}$ . Each $𝑄_{𝑖}$ is nonzero, and by Theorem 8.2.18(b), $𝑄_{𝑖}^{2} = 𝑄_{𝑖} = 𝑄_{𝑖}^{*}$ , $𝑄_{𝑖} 𝑄_{𝑖^{'}} = 0$ for $𝑖 \neq 𝑖^{'}$ , and $\sum_{𝑖} 𝑄_{𝑖} = {id}_{𝑉}$ . So by Corollary 6.6.29 the $𝑄_{𝑖}$ are orthogonal projections, and Proposition 8.2.19 says that the eigenvalues of $𝑓 (𝑇)$ are $𝜈_{1}, \dots, 𝜈_{𝑚}$ . □

Technique 8.2.25 (Apply a function to the eigenvalues). To build an operator that does to $𝑇$ what a scalar function does to a number (a square root, an inverse, an exponential, the sign, a cutoff that keeps only large eigenvalues), keep the eigenspaces of $𝑇$ and apply the function to the eigenvalues: $𝑓 (𝑇) =$ $\sum_{𝑗} 𝑓 (𝜆_{𝑗}) {Pr}_{𝐸_{𝜆_{𝑗}}}$ , or $𝑓 (𝐴) =$ $𝑈 𝑓 (𝐷) 𝑈^{*}$ . Identities between such operators reduce to identities between numbers, one eigenvalue at a time (Proposition 8.2.24), which is Technique 7.4.25 in its strongest form.

Reach for it when the hypothesis says normal or self-adjoint and the problem asks for an operator with prescribed algebraic behavior: “find $𝑆$ with $𝑆^{2} = 𝑇$ ”, “show that $𝑇$ has a self-adjoint inverse square root”, “solve $𝐱^{'} = 𝐴 𝐱$ ”. For a diagonalizable but not normal operator the same recipe works with $𝑆 𝐷 𝑆^{- 1}$ in place of $𝑈 𝐷 𝑈^{*}$ (Technique 7.4.22), but the result is not normal in general. The canonical example is the square root (Example 8.2.26); Section 8.3 shows that this square root is the only self-adjoint square root with nonnegative eigenvalues.

Used in 10 places and 7 practice exercises

Exercise 8.2.33, Exercise 8.2.34, Exercise 8.2.38, Exercise 8.2.39, Exercise 8.2.40, Exercise 8.2.47, Notation 8.3.10, Exercise 8.3.12, Exercise 8.3.37, Exercise 8.4.38; 7 practice exercises.

Example 8.2.26 (A square root). Let $𝑇$ be self-adjoint with every eigenvalue $𝜆_{𝑗} \geq 0$ , and let $𝑓 (𝜆) = \sqrt{𝜆}$ . By Proposition 8.2.24, $\sqrt{𝑇} = \sum_{𝑗} \sqrt{𝜆_{𝑗}} 𝑃_{𝑗}$ is self-adjoint, its eigenvalues $\sqrt{𝜆_{𝑗}}$ are nonnegative, and ${(\sqrt{𝑇})}^{2} =$ $(𝑓^{2}) (𝑇) =$ $𝑇$ , since $𝑓^{2} (𝜆) = 𝜆$ on the eigenvalues.

For $𝐴 = (\begin{matrix} 5 & 4 \\ 4 & 5 \end{matrix})$ , the eigenvalues are $9$ and $1$ , with $𝐸_{9} = span ((1, 1))$ and $𝐸_{1} = span ((1, - 1))$ , so

𝑃_{9} = \frac{1}{2} (\begin{matrix} 1 & 1 \\ 1 & 1 \end{matrix}), 𝑃_{1} = \frac{1}{2} (\begin{matrix} 1 & - 1 \\ - 1 & 1 \end{matrix}), \sqrt{𝐴} = 3 𝑃_{9} + 1 𝑃_{1} = (\begin{matrix} 2 & 1 \\ 1 & 2 \end{matrix}) .

Check: ${(\begin{matrix} 2 & 1 \\ 1 & 2 \end{matrix})}^{2} = (\begin{matrix} 5 & 4 \\ 4 & 5 \end{matrix})$ . The choice of $3 = \sqrt{9}$ and $1 = \sqrt{1}$ was ours; $3 𝑃_{9} - 𝑃_{1} = (\begin{matrix} 1 & 2 \\ 2 & 1 \end{matrix})$ is another symmetric square root of $𝐴$ , with eigenvalues $3$ and $- 1$ .

Similarly, if no eigenvalue is $0$ , then $𝑓 (𝜆) = 1 / 𝜆$ gives $𝑓 (𝑇) =$ $\sum_{𝑗} 𝜆_{𝑗}^{- 1} 𝑃_{𝑗}$ with $𝑇 𝑓 (𝑇) =$ $(𝜆 \cdot 1 / 𝜆) (𝑇) =$ ${id}_{𝑉}$ , so $𝑇^{- 1} =$ $\sum_{𝑗} 𝜆_{𝑗}^{- 1} 𝑃_{𝑗}$ .

Beyond 21-242 The matrix exponential is not covered in 21-242.

Example 8.2.27 (The exponential). For a real symmetric $𝐴$ with $𝐴 = \sum_{𝑗} 𝜆_{𝑗} 𝑃_{𝑗}$ , define $𝑒^{𝑡 𝐴} =$ $\sum_{𝑗} 𝑒^{𝑡 𝜆_{𝑗}} 𝑃_{𝑗}$ for $𝑡 \in ℝ$ . By Proposition 8.2.24, $𝑒^{𝑠 𝐴} 𝑒^{𝑡 𝐴} =$ $𝑒^{(𝑠 + 𝑡) 𝐴}$ , and $𝑒^{0 𝐴} = id$ . For $𝐱_{0} \in ℝ^{𝑛}$ , the function

𝐱 (𝑡) = 𝑒^{𝑡 𝐴} 𝐱_{0} = \sum_{𝑗} 𝑒^{𝑡 𝜆_{𝑗}} 𝑃_{𝑗} 𝐱_{0}

satisfies $𝐱^{'} (𝑡) =$ $\sum_{𝑗} 𝜆_{𝑗} 𝑒^{𝑡 𝜆_{𝑗}} 𝑃_{𝑗} 𝐱_{0} =$ $𝐴 𝐱 (𝑡)$ , using $𝐴 𝑃_{𝑗} = 𝜆_{𝑗} 𝑃_{𝑗}$ , and $𝐱 (0) = 𝐱_{0}$ . This is the solution of Proposition 7.4.34 written with orthogonal projections: the component of $𝐱_{0}$ in each eigenspace grows or decays at its own rate $𝑒^{𝑡 𝜆_{𝑗}}$ , independently of the others. One can show that $𝑒^{𝑡 𝐴} =$ $\sum_{𝑚 \geq 0} 𝑡^{𝑚} 𝐴^{𝑚} / 𝑚!$ , the series that defines $𝑒^{𝑡 𝐴}$ for every square matrix, but we do not need this.

8.2.6Quadratic forms and principal axes

The historical origin of the spectral theorem is the problem of finding the axes of an ellipse or ellipsoid given by an equation such as $5 𝑥^{2} -$ $4 𝑥 𝑦 +$ $8 𝑦^{2} =$ $36$ . A homogeneous quadratic polynomial in $𝑥_{1}, \dots, 𝑥_{𝑛}$ can always be written as $𝐱^{𝖳} 𝐴 𝐱$ with $𝐴$ symmetric: put the coefficient of $𝑥_{𝑖}^{2}$ at $𝑎_{𝑖 𝑖}$ , and split the coefficient of $𝑥_{𝑖} 𝑥_{𝑗}$ ( $𝑖 \neq 𝑗$ ) evenly between $𝑎_{𝑖 𝑗}$ and $𝑎_{𝑗 𝑖}$ . Section 12.2 develops quadratic forms systematically; here is the one fact the spectral theorem gives immediately.

Corollary 8.2.28 (Principal axes). Let $𝐴 \in ℝ^{𝑛 \times 𝑛}$ be symmetric, and write $𝐴 = 𝑄 𝐷 𝑄^{𝖳}$ with $𝑄$ orthogonal with columns $𝐪_{1}, \dots, 𝐪_{𝑛}$ and $𝐷 =$ $diag (𝜆_{1}, \dots, 𝜆_{𝑛})$ (Theorem 8.2.12). For $𝐱 \in ℝ^{𝑛}$ , let $𝐲 = 𝑄^{𝖳} 𝐱$ , so that $𝑦_{𝑖} = ⟨ 𝐱, 𝐪_{𝑖} ⟩$ . Then

𝐱^{𝖳} 𝐴 𝐱 = 𝜆_{1} 𝑦_{1}^{2} + \dots + 𝜆_{𝑛} 𝑦_{𝑛}^{2} and ‖ 𝐲 ‖ = ‖ 𝐱 ‖ .

Proof.routine

Uses. Theorem 8.2.12 (Real spectral theorem), Proposition 2.4.23 (Transpose of a product), Definition 6.3.33 (Unitary and orthogonal matrices), Theorem 6.3.22 (Parseval’s identity)

Techniques. Choose a convenient basis (Technique 4.5.33)

Hint. Substitute

𝐴 = 𝑄 𝐷 𝑄^{𝖳}

and regroup.

Sketch.

𝐱^{𝖳} 𝑄 𝐷 𝑄^{𝖳} 𝐱 =

{(𝑄^{𝖳} 𝐱)}^{𝖳} 𝐷 (𝑄^{𝖳} 𝐱) =

𝐲^{𝖳} 𝐷 𝐲 =

\sum 𝜆_{𝑖} 𝑦_{𝑖}^{2}

. And

{‖ 𝐲 ‖}^{2} =

𝐱^{𝖳} 𝑄 𝑄^{𝖳} 𝐱 =

{‖ 𝐱 ‖}^{2}

By Proposition 2.4.23, $𝐱^{𝖳} 𝑄 =$ ${(𝑄^{𝖳} 𝐱)}^{𝖳} =$ $𝐲^{𝖳}$ , so

𝐱^{𝖳} 𝐴 𝐱 = (𝐱^{𝖳} 𝑄) 𝐷 (𝑄^{𝖳} 𝐱) = 𝐲^{𝖳} 𝐷 𝐲 = \sum_{𝑖 = 1}^{𝑛} 𝜆_{𝑖} 𝑦_{𝑖}^{2} .

The $𝑖$ -th entry of $𝑄^{𝖳} 𝐱$ is $𝐪_{𝑖}^{𝖳} 𝐱 =$ $⟨ 𝐱, 𝐪_{𝑖} ⟩$ . Since $𝑄$ is orthogonal, $𝑄 𝑄^{𝖳} = 𝐼_{𝑛}$ , so ${‖ 𝐲 ‖}^{2} =$ $𝐱^{𝖳} 𝑄 𝑄^{𝖳} 𝐱 =$ ${‖ 𝐱 ‖}^{2}$ (alternatively, this is Theorem 6.3.22 in the basis $𝐪_{1}, \dots, 𝐪_{𝑛}$ ). □

In the coordinates $𝑦_{𝑖}$ along the eigenvectors, the cross terms disappear. The lines $span (𝐪_{𝑖})$ are the principal axes of the quadratic form.

Example 8.2.29 (An ellipse). The equation $5 𝑥^{2} -$ $4 𝑥 𝑦 +$ $8 𝑦^{2} =$ $36$ is $𝐱^{𝖳} 𝐴 𝐱 = 36$ with $𝐴 = (\begin{matrix} 5 & - 2 \\ - 2 & 8 \end{matrix})$ ; the cross term $- 4 𝑥 𝑦$ is split as $- 2 𝑥 𝑦 - 2 𝑦 𝑥$ . Here $𝑝_{𝐴} =$ $𝑥^{2} -$ $13 𝑥 +$ $36 =$ $(𝑥 - 4) (𝑥 - 9)$ , with $𝐸_{4} = span ((2, 1))$ and $𝐸_{9} = span ((- 1, 2))$ , so $𝐪_{1} = \frac{1}{\sqrt{5}} (2, 1)$ and $𝐪_{2} = \frac{1}{\sqrt{5}} (- 1, 2)$ . In the coordinates $𝑦_{1} = (2 𝑥 + 𝑦) / \sqrt{5}$ , $𝑦_{2} = (- 𝑥 + 2 𝑦) / \sqrt{5}$ , Corollary 8.2.28 turns the equation into

4 𝑦_{1}^{2} + 9 𝑦_{2}^{2} = 36, that is, \frac{𝑦_{1}^{2}}{9} + \frac{𝑦_{2}^{2}}{4} = 1 .

This is an ellipse with semi-axis $3$ along $𝐪_{1}$ and semi-axis $2$ along $𝐪_{2}$ . Its vertices are $\pm 3 𝐪_{1} = \pm \frac{3}{\sqrt{5}} (2, 1)$ and $\pm 2 𝐪_{2} = \pm \frac{2}{\sqrt{5}} (- 1, 2)$ ; for instance $𝐱 = \frac{3}{\sqrt{5}} (2, 1)$ gives $\frac{1}{5} (5 \cdot 36 - 4 \cdot 18 + 8 \cdot 9) =$ $36$ .

Figure 8.2.1. The ellipse

5 𝑥^{2} -

4 𝑥 𝑦 +

8 𝑦^{2} =

36

and its principal axes (dashed), along the eigenvectors

𝐪_{1} = \frac{1}{\sqrt{5}} (2, 1)

and

𝐪_{2} = \frac{1}{\sqrt{5}} (- 1, 2)

𝐴 = (\begin{matrix} 5 & - 2 \\ - 2 & 8 \end{matrix})

. The semi-axes are

\sqrt{36 / 4} = 3

and

\sqrt{36 / 9} = 2

Remark 8.2.30 (Classifying conics). The same computation classifies every curve $𝑎 𝑥^{2} +$ $2 𝑏 𝑥 𝑦 +$ $𝑐 𝑦^{2} =$ $1$ . Let $𝜆_{1} \geq 𝜆_{2}$ be the eigenvalues of $𝐴 = (\begin{matrix} 𝑎 & 𝑏 \\ 𝑏 & 𝑐 \end{matrix})$ , so the curve is $𝜆_{1} 𝑦_{1}^{2} + 𝜆_{2} 𝑦_{2}^{2} = 1$ in the rotated coordinates of Corollary 8.2.28.

If $𝜆_{1} \geq 𝜆_{2} > 0$ , it is an ellipse with semi-axes $1 / \sqrt{𝜆_{1}}$ and $1 / \sqrt{𝜆_{2}}$ along $𝐪_{1}$ and $𝐪_{2}$ (a circle if $𝜆_{1} = 𝜆_{2}$ ).
If $𝜆_{1} > 0 > 𝜆_{2}$ , it is a hyperbola whose vertices $\pm 𝐪_{1} / \sqrt{𝜆_{1}}$ lie on the axis of the positive eigenvalue.
If $𝜆_{1} > 0 = 𝜆_{2}$ , it is the pair of parallel lines $𝑦_{1} = \pm 1 / \sqrt{𝜆_{1}}$ .
If $𝜆_{1} \leq 0$ , the left side is never positive and the curve is empty.

Since $𝜆_{1} 𝜆_{2} =$ $det 𝐴 =$ $𝑎 𝑐 -$ $𝑏^{2}$ and $𝜆_{1} +$ $𝜆_{2} =$ $tr 𝐴 =$ $𝑎 +$ $𝑐$ , the type can be read off without computing eigenvalues: an ellipse when $𝑎 𝑐 - 𝑏^{2} > 0$ and $𝑎 > 0$ , a hyperbola when $𝑎 𝑐 - 𝑏^{2} < 0$ . This is the discriminant test from analytic geometry, now with a proof.

Beyond 21-242 Infinite-dimensional spaces are not covered in 21-242.

Remark 8.2.31 (Infinite dimensions). The spectral theorems need

𝑉

to be finite-dimensional. On

𝐶 ([0, 1], ℂ)

with

⟨ 𝑓, 𝑔 ⟩ = \int_{0}^{1} 𝑓 𝑔

, the multiplication operator

(𝑀 𝑓) (𝑡) =

𝑡 𝑓 (𝑡)

is self-adjoint (Example 6.6.14), but it has no eigenvalues at all: if

𝑡 𝑓 (𝑡) = 𝜆 𝑓 (𝑡)

for all

𝑡

, then

𝑓 (𝑡) = 0

for every

𝑡 \neq 𝜆

, and by continuity

𝑓 = 0

. So no eigenvector exists, let alone an orthonormal eigenbasis. The infinite-dimensional theory replaces the sum

\sum 𝜆_{𝑗} 𝑃_{𝑗}

by an integral over a “continuous spectrum”; Section 15.3 gives a first look.

Exercises

Exercise 8.2.32.standard Find an orthogonal $𝑄$ and a diagonal $𝐷$ with

(\begin{matrix} 3 & - 2 & 4 \\ - 2 & 6 & 2 \\ 4 & 2 & 3 \end{matrix}) = 𝑄 𝐷 𝑄^{𝖳} .

Solution.

Uses. Algorithm 8.2.14 (Orthogonal diagonalization), Theorem 6.4.15 (Gram–Schmidt is correct), Theorem 7.5.22 (Trace and determinant from eigenvalues)

Techniques. Diagonalize, solve, conjugate back (Technique 7.4.22)

Hint 1. One eigenvalue is

- 2

, with eigenvector

(2, 1, - 2)

. Use the trace to find the others.

Hint 2. The other eigenvalue is

7

, with a two-dimensional eigenspace: the plane orthogonal to

(2, 1, - 2)

. Find a basis of it and apply Gram–Schmidt.

Sketch.

𝐴 (2, 1, - 2) =

- 2 (2, 1, - 2)

. Since

tr 𝐴 = 12

and

𝑝_{𝐴}

splits with real roots, and

𝐴 - 7 𝐼_{3}

has rank

1

, the eigenvalues are

- 2, 7, 7

𝐸_{7} = {(2, 1, - 2)}^{⟂}

has basis

(1, 0, 1), (1, - 2, 0)

; Gram–Schmidt gives

\frac{1}{\sqrt{2}} (1, 0, 1)

and

\frac{1}{3 \sqrt{2}} (1, - 4, - 1)

Let $𝐴$ be the matrix. We have

𝐴 - 7 𝐼_{3} = (\begin{matrix} - 4 & - 2 & 4 \\ - 2 & - 1 & 2 \\ 4 & 2 & - 4 \end{matrix}),

whose rows are all multiples of $(2, 1, - 2)$ . So $rank (𝐴 - 7 𝐼_{3}) = 1$ and $𝐸_{7} =$ $ker (𝐴 - 7 𝐼_{3}) =$ ${𝐱 | 2 𝑥_{1} + 𝑥_{2} - 2 𝑥_{3} = 0}$ is a plane. Since $tr 𝐴 = 12$ and $𝐴$ is diagonalizable with eigenvalue $7$ of multiplicity $2$ (Theorem 8.2.12), the third eigenvalue is $12 - 14 = - 2$ (Theorem 7.5.22). Indeed $𝐴 (2, 1, - 2) =$ $(6 - 2 - 8, - 4 + 6 - 4, 8 + 2 - 6) =$ $(- 4, - 2, 4) =$ $- 2 (2, 1, - 2)$ .

For $𝐸_{7}$ , take the basis $(1, 0, 1), (1, - 2, 0)$ (both satisfy $2 𝑥_{1} + 𝑥_{2} - 2 𝑥_{3} = 0$ ). Gram–Schmidt: $𝐮_{2} = \frac{1}{\sqrt{2}} (1, 0, 1)$ , and

(1, - 2, 0) - \frac{1 \cdot 1 + 0 + 0}{2} (1, 0, 1) = (\frac{1}{2}, - 2, - \frac{1}{2}) = \frac{1}{2} (1, - 4, - 1),

which normalizes to $𝐮_{3} = \frac{1}{3 \sqrt{2}} (1, - 4, - 1)$ . With $𝐮_{1} = \frac{1}{3} (2, 1, - 2)$ ,

𝑄 = (\begin{matrix} \frac{2}{3} & \frac{1}{\sqrt{2}} & \frac{1}{3 \sqrt{2}} \\ \frac{1}{3} & 0 & - \frac{4}{3 \sqrt{2}} \\ - \frac{2}{3} & \frac{1}{\sqrt{2}} & - \frac{1}{3 \sqrt{2}} \end{matrix}), 𝐷 = (\begin{matrix} - 2 & 0 & 0 \\ 0 & 7 & 0 \\ 0 & 0 & 7 \end{matrix}) .

The columns are orthonormal: $𝐮_{1} ⟂ 𝐸_{7}$ automatically, and $𝐮_{2} ⟂ 𝐮_{3}$ by construction.

Exercise 8.2.33.standard Let $𝑉$ be a finite-dimensional complex inner product space and $𝑇 \in L (𝑉)$ normal. Show:

(a) $𝑇$ is self-adjoint if and only if every eigenvalue of $𝑇$ is real;
(b) $𝑇^{*} = - 𝑇$ if and only if every eigenvalue of $𝑇$ is purely imaginary (in $𝑖 ℝ$ );
(c) $𝑇$ is an orthogonal projection if and only if every eigenvalue of $𝑇$ is $0$ or $1$ ;
(d) $𝑇^{*} 𝑇 = {id}_{𝑉}$ if and only if every eigenvalue of $𝑇$ has absolute value $1$ .

Then show by an example that (a) fails for operators that are not normal.

Solution.

Uses. Theorem 8.2.18 (Spectral decomposition), Proposition 8.2.19 (Uniqueness of the spectral decomposition), Proposition 8.2.24 (Properties of $𝑓 (𝑇)$ ), Corollary 6.6.29 (Orthogonal projections are the self-adjoint idempotents)

Techniques. Apply a function to the eigenvalues (Technique 8.2.25), Test an identity on eigenvectors (Technique 7.4.25)

Hint 1. Write

𝑇 = \sum_{𝑗} 𝜆_{𝑗} 𝑃_{𝑗}

. Then

𝑇^{*} = \sum_{𝑗} 𝜆_{𝑗} 𝑃_{𝑗}

𝑇^{2} = \sum_{𝑗} 𝜆_{𝑗}^{2} 𝑃_{𝑗}

, and

𝑇^{*} 𝑇 =

\sum_{𝑗} {| 𝜆_{𝑗} |}^{2} 𝑃_{𝑗}

Hint 2. Two operators

\sum 𝑎_{𝑗} 𝑃_{𝑗}

and

\sum 𝑏_{𝑗} 𝑃_{𝑗}

are equal if and only if

𝑎_{𝑗} = 𝑏_{𝑗}

for all

𝑗

: apply both to a vector of

𝐸_{𝜆_{𝑗}}

Sketch. With

𝑇 = \sum 𝜆_{𝑗} 𝑃_{𝑗}

𝑇^{*} = 𝑇

iff

𝜆_{𝑗} = 𝜆_{𝑗}

;

𝑇^{*} = - 𝑇

iff

𝜆_{𝑗} = - 𝜆_{𝑗}

;

𝑇^{2} = 𝑇 = 𝑇^{*}

iff

𝜆_{𝑗}^{2} = 𝜆_{𝑗} = 𝜆_{𝑗}

, i.e.

𝜆_{𝑗} \in {0, 1}

;

𝑇^{*} 𝑇 = id

iff

{| 𝜆_{𝑗} |}^{2} = 1

. Counterexample to (a):

(\begin{matrix} 1 & 1 \\ 0 & 1 \end{matrix})

Write $𝑇 =$ $\sum_{𝑗 = 1}^{𝑘} 𝜆_{𝑗} 𝑃_{𝑗}$ as in Theorem 8.2.18. By Proposition 8.2.24, $𝑇^{*} = \sum_{𝑗} 𝜆_{𝑗} 𝑃_{𝑗}$ , $𝑇^{2} = \sum_{𝑗} 𝜆_{𝑗}^{2} 𝑃_{𝑗}$ , and $𝑇^{*} 𝑇 =$ $\sum_{𝑗} {| 𝜆_{𝑗} |}^{2} 𝑃_{𝑗}$ , while ${id}_{𝑉} = \sum_{𝑗} 𝑃_{𝑗}$ . Two operators $\sum_{𝑗} 𝑎_{𝑗} 𝑃_{𝑗}$ and $\sum_{𝑗} 𝑏_{𝑗} 𝑃_{𝑗}$ are equal if and only if $𝑎_{𝑗} = 𝑏_{𝑗}$ for every $𝑗$ : apply both to a nonzero $𝐯 \in 𝐸_{𝜆_{𝑗}}$ , on which $\sum_{𝑖} 𝑎_{𝑖} 𝑃_{𝑖}$ acts as $𝑎_{𝑗}$ (Proposition 8.2.24(a)).

(a) $𝑇^{*} = 𝑇$ if and only if $𝜆_{𝑗} = 𝜆_{𝑗}$ for all $𝑗$ , that is, every $𝜆_{𝑗}$ is real.

(b) $𝑇^{*} = - 𝑇$ if and only if $𝜆_{𝑗} = - 𝜆_{𝑗}$ for all $𝑗$ , that is, $Re 𝜆_{𝑗} = 0$ .

(c) By Corollary 6.6.29, $𝑇$ is an orthogonal projection if and only if $𝑇^{2} = 𝑇 = 𝑇^{*}$ , that is, $𝜆_{𝑗}^{2} = 𝜆_{𝑗} = 𝜆_{𝑗}$ for all $𝑗$ . The first equation says $𝜆_{𝑗} \in {0, 1}$ , and then the second holds automatically.

(d) $𝑇^{*} 𝑇 = {id}_{𝑉}$ if and only if ${| 𝜆_{𝑗} |}^{2} = 1$ for all $𝑗$ .

For the example, $𝐴 = (\begin{matrix} 1 & 1 \\ 0 & 1 \end{matrix})$ has only the real eigenvalue $1$ , but $𝐴^{*} = (\begin{matrix} 1 & 0 \\ 1 & 1 \end{matrix}) \neq 𝐴$ .

Exercise 8.2.34.standard Prove or disprove each statement. Throughout, $𝑉$ is a finite-dimensional inner product space.

(a)If $𝑉$ is complex, $𝑇 \in L (𝑉)$ is normal, and $𝑇^{𝑚} = 0$ for some $𝑚 \geq 1$ , then $𝑇 = 0$ .
(b)If $𝑉$ is real, $𝑇 \in L (𝑉)$ is self-adjoint, and $𝑇^{3} + 𝑇 = 2 {id}_{𝑉}$ , then $𝑇 = {id}_{𝑉}$ .
(c)The statement in (b) remains true without the hypothesis that $𝑇$ is self-adjoint.
(d)If $𝐴 \in ℝ^{𝑛 \times 𝑛}$ is symmetric, then every basis of $ℝ^{𝑛}$ consisting of eigenvectors of $𝐴$ is orthogonal.
(e)If $𝐴 \in ℝ^{𝑛 \times 𝑛}$ is diagonalizable over $ℝ$ and its eigenspaces are pairwise orthogonal, then $𝐴$ is symmetric.

Solution.

Uses. Theorem 8.2.5 (Complex spectral theorem), Theorem 8.2.12 (Real spectral theorem), Proposition 8.2.24 (Properties of $𝑓 (𝑇)$ ), Theorem 7.3.24 (Every monic polynomial is a characteristic polynomial), Proposition 8.2.2 (Necessary conditions)

Techniques. Apply a function to the eigenvalues (Technique 8.2.25), Finding counterexamples and minimal examples (Technique 1.4.18)

Hint 1. For (a) and (b), apply the relation to an eigenvector: what equation does each eigenvalue satisfy?

Hint 2. For (c), look for a real

2 \times 2

matrix whose characteristic polynomial is the other factor of

𝑥^{3} +

𝑥 -

2 =

(𝑥 - 1) (𝑥^{2} + 𝑥 + 2)

Hint 3. For (d), think of repeated eigenvalues. For (e), choose an orthonormal basis inside each eigenspace.

Sketch. (a) True:

𝜆^{𝑚} = 0

forces every eigenvalue to be

0

, and

𝑇 = \sum 𝜆_{𝑗} 𝑃_{𝑗} = 0

. (b) True: real eigenvalues of

𝑇

satisfy

𝜆^{3} +

𝜆 -

2 =

(𝜆 - 1) (𝜆^{2} + 𝜆 + 2) =

0

, so

𝜆 = 1

and

𝑇 = 𝑃_{1} = id

. (c) False: the companion matrix

𝐶

𝑥^{2} + 𝑥 + 2

satisfies

𝐶^{2} = - 𝐶 - 2 𝐼

, hence

𝐶^{3} + 𝐶 = 2 𝐼

. (d) False:

𝐼_{2}

with the basis

(1, 0), (1, 1)

. (e) True: orthonormal bases of the eigenspaces combine into an orthonormal eigenbasis.

(a) True. By Theorem 8.2.5, $𝑇 = \sum_{𝑗} 𝜆_{𝑗} 𝑃_{𝑗}$ . If $𝐯 \in 𝐸_{𝜆_{𝑗}}$ is nonzero, then $𝟎 =$ $𝑇^{𝑚} 𝐯 =$ $𝜆_{𝑗}^{𝑚} 𝐯$ , so $𝜆_{𝑗} = 0$ . Hence the only eigenvalue is $0$ and $𝑇 = 0 \cdot 𝑃_{1} = 0$ .

(b) True. By Theorem 8.2.12, $𝑉$ has an orthonormal eigenbasis $𝐮_{1}, \dots, 𝐮_{𝑛}$ , with $𝑇 𝐮_{𝑖} = 𝜇_{𝑖} 𝐮_{𝑖}$ and $𝜇_{𝑖} \in ℝ$ . Applying $𝑇^{3} + 𝑇 = 2 {id}_{𝑉}$ to $𝐮_{𝑖}$ gives $𝜇_{𝑖}^{3} + 𝜇_{𝑖} - 2 = 0$ . Since $𝑥^{3} +$ $𝑥 -$ $2 =$ $(𝑥 - 1) (𝑥^{2} + 𝑥 + 2)$ and $𝑥^{2} + 𝑥 + 2$ has discriminant $1 - 8 < 0$ , the only real root is $1$ . So $𝑇 𝐮_{𝑖} = 𝐮_{𝑖}$ for all $𝑖$ , and $𝑇 = {id}_{𝑉}$ .

(c) False. Let $𝐶 = (\begin{matrix} 0 & - 2 \\ 1 & - 1 \end{matrix})$ , the companion matrix of $𝑥^{2} + 𝑥 + 2$ (Theorem 7.3.24); directly, $𝐶^{2} =$ $(\begin{matrix} - 2 & 2 \\ - 1 & - 1 \end{matrix}) =$ $- 𝐶 -$ $2 𝐼_{2}$ . Then $𝐶^{3} =$ $- 𝐶^{2} -$ $2 𝐶 =$ $𝐶 +$ $2 𝐼_{2} -$ $2 𝐶 =$ $2 𝐼_{2} -$ $𝐶$ , so $𝐶^{3} + 𝐶 = 2 𝐼_{2}$ , but $𝐶 \neq 𝐼_{2}$ . (Here $𝐶$ is not symmetric, and its eigenvalues are not real.)

(d) False. $𝐴 = 𝐼_{2}$ is symmetric, and every nonzero vector is an eigenvector, so $(1, 0), (1, 1)$ is an eigenbasis. It is not orthogonal. (What is true: eigenvectors for different eigenvalues are orthogonal.)

(e) True. Choose an orthonormal basis of each eigenspace (Theorem 6.4.20). Since $𝐴$ is diagonalizable, the eigenspaces together contain a basis, so their dimensions add up to $𝑛$ (Theorem 7.4.16), and the concatenation is an orthonormal list of $𝑛$ eigenvectors, because vectors from different eigenspaces are orthogonal. It is an orthonormal eigenbasis, so $𝐴$ is symmetric by Proposition 8.2.2(b) (or Theorem 8.2.12).

Exercise 8.2.35.standard Find the error in the following “proof” that every normal operator $𝑇$ on a finite-dimensional real inner product space $𝑉$ with $dim 𝑉 \geq 1$ has an orthonormal eigenbasis. Then give a counterexample to the claim, and name a hypothesis on $𝑇$ that repairs the proof.

“Proof.” Induct on $𝑛 = dim 𝑉$ ; the case $𝑛 = 1$ is clear. Let $𝑛 \geq 2$ , and let $𝐮$ be a unit eigenvector of $𝑇$ , with $𝑇 𝐮 = 𝜆 𝐮$ . Since $𝑇$ is normal, $‖ (𝑇^{*} - 𝜆 id) 𝐮 ‖ =$ $‖ (𝑇 - 𝜆 id) 𝐮 ‖ =$ $0$ , so $𝐮$ is an eigenvector of $𝑇^{*}$ as well. Hence $𝑊 = {span (𝐮)}^{⟂}$ is invariant under $𝑇$ and $𝑇^{*}$ , and $𝑇 ↾_{𝑊}$ is normal. By the induction hypothesis, $𝑊$ has an orthonormal eigenbasis for $𝑇 ↾_{𝑊}$ , and adding $𝐮$ gives one for $𝑇$ .

Solution.

Uses. Example 8.2.3 (Which matrices have orthonormal eigenbases), Lemma 8.2.10 (Self-adjoint operators have eigenvalues), Theorem 8.2.12 (Real spectral theorem)

Techniques. Induction on dimension via an invariant subspace (Technique 7.5.12)

Hint 1. Every step after the first one is the second proof of Theorem 8.2.5, which works over

ℝ

as well. Look at the first step.

Hint 2. Does every operator on

ℝ^{2}

have an eigenvector?

Sketch. The proof assumes that

𝑇

has an eigenvector. Over

ℝ

it need not: the rotation

𝐽

is normal (

𝐽^{𝖳} 𝐽 = 𝐼 = 𝐽 𝐽^{𝖳}

) and has no real eigenvalue. If

𝑇

is self-adjoint, Lemma 8.2.10 supplies the eigenvector, and the proof becomes that of Theorem 8.2.12.

The error is in the second sentence: “let $𝐮$ be a unit eigenvector of $𝑇$ ”. Over $ℝ$ an operator need not have any eigenvector. The rest of the argument is correct over $ℝ$ : the computation $‖ (𝑇 - 𝜆 id) 𝐮 ‖ =$ $‖ (𝑇^{*} - 𝜆 id) 𝐮 ‖$ for a real $𝜆$ , the invariance of $𝑊$ , and the normality of $𝑇 ↾_{𝑊}$ go through as in the second proof of Theorem 8.2.5.

Counterexample: $𝐽 = (\begin{matrix} 0 & - 1 \\ 1 & 0 \end{matrix})$ satisfies $𝐽^{𝖳} 𝐽 = 𝐼_{2} = 𝐽 𝐽^{𝖳}$ , so $𝑇_{𝐽}$ is normal on $ℝ^{2}$ , but $𝑝_{𝐽} = 𝑥^{2} + 1$ has no real root, so $𝑇_{𝐽}$ has no eigenvector, let alone an orthonormal eigenbasis (Example 8.2.3).

Repair: assume $𝑇$ is self-adjoint. Then Lemma 8.2.10 provides the eigenvector at every stage (restrictions of self-adjoint operators to invariant subspaces are self-adjoint), and the argument becomes the first proof of Theorem 8.2.12.

Exercise 8.2.36.standard Let

𝑉

be a finite-dimensional inner product space over

𝔽

and let

𝑆, 𝑇 \in L (𝑉)

be self-adjoint. Show that

𝑆 𝑇 = 𝑇 𝑆

if and only if

𝑉

has an orthonormal basis consisting of vectors that are eigenvectors of both

𝑆

and

𝑇

Solution.

Uses. Theorem 8.2.18 (Spectral decomposition), Proposition 8.2.20 (Spectral projections are polynomials in $𝑇$ ), Theorem 8.2.12 (Real spectral theorem), Corollary 8.2.7 (Hermitian matrices), Definition 6.6.26 (Self-adjoint operator)

Techniques. Look for invariant subspaces among kernels and images of commuting operators (Technique 7.2.31), Glue bases of the summands (Technique 4.4.14)

Hint 1. The easy direction: in a common orthonormal eigenbasis both matrices are diagonal.

Hint 2. For the other,

𝑆

maps each eigenspace

𝐸_{𝜆_{𝑗}}

𝑇

into itself (Proposition 8.2.20(a)). Is

𝑆 ↾_{𝐸_{𝜆_{𝑗}}}

self-adjoint?

Hint 3. Apply the spectral theorem to

𝑆 ↾_{𝐸_{𝜆_{𝑗}}}

on each

𝐸_{𝜆_{𝑗}}

, and glue the bases.

Sketch. If both are diagonal in an orthonormal basis

𝒪︀

, their matrices commute, so

𝑆 𝑇 = 𝑇 𝑆

. Conversely,

𝑉 = \oplus 𝐸_{𝜆_{𝑗}}

for the eigenvalues of

𝑇

, and

𝑆

maps each

𝐸_{𝜆_{𝑗}}

into itself;

𝑆 ↾_{𝐸_{𝜆_{𝑗}}}

is self-adjoint, so

𝐸_{𝜆_{𝑗}}

has an orthonormal eigenbasis for it. These vectors are eigenvectors of

𝑇

(with eigenvalue

𝜆_{𝑗}

) and of

𝑆

; the eigenspaces are orthogonal, so the concatenation is orthonormal.

If $𝒪︀$ is an orthonormal basis of common eigenvectors, then ${[𝑆]}_{𝒪︀}$ and ${[𝑇]}_{𝒪︀}$ are diagonal, so they commute, and ${[𝑆 𝑇]}_{𝒪︀} =$ ${[𝑆]}_{𝒪︀} {[𝑇]}_{𝒪︀} =$ ${[𝑇]}_{𝒪︀} {[𝑆]}_{𝒪︀} =$ ${[𝑇 𝑆]}_{𝒪︀}$ . Hence $𝑆 𝑇 = 𝑇 𝑆$ .

Conversely, suppose $𝑆 𝑇 = 𝑇 𝑆$ . A self-adjoint operator satisfies the spectral hypothesis (over $ℂ$ it is normal), so by Theorem 8.2.18, $𝑉 = 𝐸_{𝜆_{1}} \oplus \dots \oplus 𝐸_{𝜆_{𝑘}}$ with pairwise orthogonal eigenspaces of $𝑇$ . By Proposition 8.2.20(a), $𝑆$ maps each $𝐸_{𝜆_{𝑗}}$ into itself. The restriction $𝑆_{𝑗} =$ $𝑆 ↾_{𝐸_{𝜆_{𝑗}}} \in$ $L (𝐸_{𝜆_{𝑗}})$ is self-adjoint, because $⟨ 𝑆_{𝑗} 𝐯, 𝐰 ⟩ =$ $⟨ 𝑆 𝐯, 𝐰 ⟩ =$ $⟨ 𝐯, 𝑆 𝐰 ⟩ =$ $⟨ 𝐯, 𝑆_{𝑗} 𝐰 ⟩$ for $𝐯, 𝐰 \in 𝐸_{𝜆_{𝑗}}$ . By Theorem 8.2.12 or Corollary 8.2.7, $𝐸_{𝜆_{𝑗}}$ has an orthonormal basis ${𝒪︀}_{𝑗}$ of eigenvectors of $𝑆_{𝑗}$ , which are eigenvectors of $𝑆$ ; they lie in $𝐸_{𝜆_{𝑗}}$ , so they are eigenvectors of $𝑇$ too. The concatenation ${𝒪︀}_{1}, \dots, {𝒪︀}_{𝑘}$ is orthonormal, since the eigenspaces are pairwise orthogonal, and it has $\sum_{𝑗} dim 𝐸_{𝜆_{𝑗}} =$ $dim 𝑉$ vectors. So it is an orthonormal basis of common eigenvectors.

Exercise 8.2.37.standard Let $𝑉$ be a finite-dimensional complex inner product space, $𝑇 \in L (𝑉)$ normal, and $𝑆 \in L (𝑉)$ with $𝑆 𝑇 = 𝑇 𝑆$ .

(a)Show that $𝑆 𝑇^{*} = 𝑇^{*} 𝑆$ .
(b)Show by an example that (a) can fail if $𝑇$ is not normal.
(c)Suppose that $𝑆$ is normal as well. Show that $𝑆$ , $𝑆^{*}$ , $𝑇$ , $𝑇^{*}$ pairwise commute, and deduce that $𝑆 𝑇$ and $𝑆 + 𝑇$ are normal.

Solution.

Uses. Proposition 8.2.20 (Spectral projections are polynomials in $𝑇$ ), Proposition 7.2.27 (Evaluating polynomials at an operator), Proposition 6.6.18 (Properties of the adjoint), Definition 8.1.8 (Normal operator, normal matrix)

Hint 1. By Proposition 8.2.20(b),

𝑇^{*}

is a polynomial in

𝑇

Hint 2. For (b), try

𝑇 = 𝑆 = (\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix})

Hint 3. For (c), apply (a) twice, and take adjoints of

𝑆 𝑇 = 𝑇 𝑆

Sketch. (a)

𝑇^{*} = 𝑝 (𝑇)

, and

𝑆

commutes with every polynomial in

𝑇

. (b)

𝑇 = 𝑆 = (\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix})

𝑆 𝑇^{*} =

(\begin{matrix} 1 & 0 \\ 0 & 0 \end{matrix}) \neq

(\begin{matrix} 0 & 0 \\ 0 & 1 \end{matrix}) =

𝑇^{*} 𝑆

. (c)

𝑆 𝑇^{*} = 𝑇^{*} 𝑆

and (by symmetry)

𝑇 𝑆^{*} = 𝑆^{*} 𝑇

; taking adjoints of

𝑆 𝑇 = 𝑇 𝑆

gives

𝑆^{*} 𝑇^{*} = 𝑇^{*} 𝑆^{*}

; normality gives the rest. Then

(𝑆 𝑇) {(𝑆 𝑇)}^{*} =

𝑆 𝑇 𝑇^{*} 𝑆^{*}

and

{(𝑆 𝑇)}^{*} (𝑆 𝑇) =

𝑇^{*} 𝑆^{*} 𝑆 𝑇

are products of the same commuting operators.

(a) By Proposition 8.2.20(b), $𝑇^{*} = 𝑝 (𝑇)$ for some $𝑝 \in ℂ [𝑥]$ . Since $𝑆$ commutes with $𝑇$ , it commutes with $𝑇^{𝑚}$ for every $𝑚$ (induction on $𝑚$ ), hence with $𝑝 (𝑇) = 𝑇^{*}$ .

(b) Let $𝑇 = 𝑆 = (\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix})$ , which commute. Then $𝑇^{*} = (\begin{matrix} 0 & 0 \\ 1 & 0 \end{matrix})$ , and

𝑆 𝑇^{*} = (\begin{matrix} 1 & 0 \\ 0 & 0 \end{matrix}) \neq (\begin{matrix} 0 & 0 \\ 0 & 1 \end{matrix}) = 𝑇^{*} 𝑆 .

(This $𝑇$ is not normal, since $𝑇 𝑇^{*} \neq 𝑇^{*} 𝑇$ by the same computation.)

(c) By (a), $𝑆$ commutes with $𝑇^{*}$ ; by (a) with the roles of $𝑆$ and $𝑇$ exchanged (now $𝑆$ is normal and $𝑇$ commutes with it), $𝑇$ commutes with $𝑆^{*}$ . Taking adjoints of $𝑆 𝑇 = 𝑇 𝑆$ gives $𝑇^{*} 𝑆^{*} = 𝑆^{*} 𝑇^{*}$ (Proposition 6.6.18(d)). Finally $𝑆 𝑆^{*} = 𝑆^{*} 𝑆$ and $𝑇 𝑇^{*} = 𝑇^{*} 𝑇$ by normality. So the four operators pairwise commute. Then

(𝑆 𝑇) {(𝑆 𝑇)}^{*} = 𝑆 𝑇 𝑇^{*} 𝑆^{*} and {(𝑆 𝑇)}^{*} (𝑆 𝑇) = 𝑇^{*} 𝑆^{*} 𝑆 𝑇

are products of the same four pairwise commuting operators, so they are equal, and $𝑆 𝑇$ is normal. Similarly $(𝑆 + 𝑇) {(𝑆 + 𝑇)}^{*} =$ $𝑆 𝑆^{*} +$ $𝑆 𝑇^{*} +$ $𝑇 𝑆^{*} +$ $𝑇 𝑇^{*}$ and ${(𝑆 + 𝑇)}^{*} (𝑆 + 𝑇) =$ $𝑆^{*} 𝑆 +$ $𝑆^{*} 𝑇 +$ $𝑇^{*} 𝑆 +$ $𝑇^{*} 𝑇$ agree term by term.

Exercise 8.2.38.standard

(a)Find the symmetric $𝐴 \in ℝ^{2 \times 2}$ with eigenvalues $1$ and $3$ such that $(1, 2)$ is an eigenvector for $3$ , and show that it is unique.
(b)Find a matrix $𝐵 \in ℝ^{2 \times 2}$ , not symmetric, with eigenvalues $1$ and $3$ such that $(1, 2)$ is an eigenvector for $3$ .

Solution.

Uses. Theorem 8.2.18 (Spectral decomposition), Theorem 6.4.31 (The matrix of an orthogonal projection), Proposition 7.4.23 (Powers and polynomials of a diagonalizable matrix)

Techniques. Apply a function to the eigenvalues (Technique 8.2.25)

Hint 1. For a symmetric

𝐴

, the eigenspace for

1

is forced to be

{(1, 2)}^{⟂}

. So the spectral decomposition is determined.

Hint 2. For (b), drop the orthogonality: choose any other eigenvector for

1

, such as

(1, 0)

, and compute

𝑆 𝐷 𝑆^{- 1}

Sketch. (a)

𝐸_{3} = span ((1, 2))

and

𝐸_{1} = 𝐸_{3}^{⟂}

are forced, so

𝐴 = 3 𝑃 + (𝐼 - 𝑃)

with

𝑃 = \frac{1}{5} (\begin{matrix} 1 & 2 \\ 2 & 4 \end{matrix})

𝐴 = \frac{1}{5} (\begin{matrix} 7 & 4 \\ 4 & 13 \end{matrix})

. (b)

𝑆 = (\begin{matrix} 1 & 1 \\ 2 & 0 \end{matrix})

𝐷 = diag (3, 1)

𝐵 =

𝑆 𝐷 𝑆^{- 1} =

(\begin{matrix} 1 & 1 \\ 0 & 3 \end{matrix})

(a) Let $𝐴$ be such a matrix. By Theorem 8.2.18, $ℝ^{2} = 𝐸_{3} \oplus 𝐸_{1}$ with $𝐸_{3} ⟂ 𝐸_{1}$ . Each eigenspace is at least a line, so both are lines; $𝐸_{3} = span ((1, 2))$ and $𝐸_{1} =$ $𝐸_{3}^{⟂} =$ $span ((2, - 1))$ . So $𝐴 = 3 𝑃_{3} + 1 𝑃_{1}$ with $𝑃_{3} = {Pr}_{𝐸_{3}}$ and $𝑃_{1} = 𝐼_{2} - 𝑃_{3}$ . With $𝐮 = \frac{1}{\sqrt{5}} (1, 2)$ , Theorem 6.4.31 gives $𝑃_{3} =$ $𝐮 𝐮^{𝖳} =$ $\frac{1}{5} (\begin{matrix} 1 & 2 \\ 2 & 4 \end{matrix})$ , so

𝐴 = 𝐼_{2} + 2 𝑃_{3} = (\begin{matrix} 1 & 0 \\ 0 & 1 \end{matrix}) + \frac{2}{5} (\begin{matrix} 1 & 2 \\ 2 & 4 \end{matrix}) = \frac{1}{5} (\begin{matrix} 7 & 4 \\ 4 & 13 \end{matrix}) .

It is symmetric, and $𝐴 (1, 2) =$ $\frac{1}{5} (15, 30) =$ $3 (1, 2)$ , $𝐴 (2, - 1) =$ $\frac{1}{5} (10, - 5) =$ $(2, - 1)$ . The argument shows it is the only one.

(b) Take $(1, 0)$ as eigenvector for $1$ , and let $𝑆 = (\begin{matrix} 1 & 1 \\ 2 & 0 \end{matrix})$ (columns: the eigenvectors for $3$ and $1$ ) and $𝐷 = diag (3, 1)$ . Then $𝑆^{- 1} = (\begin{matrix} 0 & \frac{1}{2} \\ 1 & - \frac{1}{2} \end{matrix})$ and

𝐵 = 𝑆 𝐷 𝑆^{- 1} = (\begin{matrix} 3 & 1 \\ 6 & 0 \end{matrix}) (\begin{matrix} 0 & \frac{1}{2} \\ 1 & - \frac{1}{2} \end{matrix}) = (\begin{matrix} 1 & 1 \\ 0 & 3 \end{matrix}) .

Then $𝐵 (1, 2) = (3, 6)$ and $𝐵 (1, 0) = (1, 0)$ , and $𝐵$ is not symmetric. Uniqueness in (a) came from the orthogonality of eigenspaces, which only symmetric matrices guarantee.

Exercise 8.2.39.standard Let $𝐴 = (\begin{matrix} 10 & 6 \\ 6 & 10 \end{matrix})$ .

(a)Find the symmetric $𝐵$ with nonnegative eigenvalues such that $𝐵^{2} = 𝐴$ .
(b)Find all $𝐵 \in ℝ^{2 \times 2}$ with $𝐵^{2} = 𝐴$ .
(c)Explain why $𝐼_{2}$ , in contrast, has infinitely many square roots in $ℝ^{2 \times 2}$ .

Solution.

Uses. Proposition 8.2.24 (Properties of $𝑓 (𝑇)$ ), Proposition 7.2.30 (Kernels and images of commuting operators are invariant), Proposition 7.2.7 (Invariant lines are spanned by eigenvectors), Theorem 8.2.18 (Spectral decomposition)

Techniques. Apply a function to the eigenvalues (Technique 8.2.25)

Hint 1.

𝐴

has eigenvalues

16

and

4

, with eigenvectors

(1, 1)

and

(1, - 1)

Hint 2. If

𝐵^{2} = 𝐴

, then

𝐵

commutes with

𝐴

. What does

𝐵

do to the eigenspaces of

𝐴

Hint 3. Each eigenspace of

𝐴

is a line mapped into itself by

𝐵

, so it consists of eigenvectors of

𝐵

. For (c), every reflection squares to

𝐼_{2}

Sketch. (a)

𝑃 = \frac{1}{2} (\begin{matrix} 1 & 1 \\ 1 & 1 \end{matrix})

𝑄 = \frac{1}{2} (\begin{matrix} 1 & - 1 \\ - 1 & 1 \end{matrix})

𝐴 = 16 𝑃 + 4 𝑄

\sqrt{𝐴} = 4 𝑃 + 2 𝑄 = (\begin{matrix} 3 & 1 \\ 1 & 3 \end{matrix})

. (b)

𝐵

commutes with

𝐴

, so preserves the lines

𝐸_{16}

and

𝐸_{4}

, so

𝐵 = 𝛽_{1} 𝑃 + 𝛽_{2} 𝑄

with

𝛽_{1}^{2} = 16

and

𝛽_{2}^{2} = 4

: four matrices

\pm (\begin{matrix} 3 & 1 \\ 1 & 3 \end{matrix})

\pm (\begin{matrix} 1 & 3 \\ 3 & 1 \end{matrix})

. (c) For

𝐼_{2}

the eigenspace is all of

ℝ^{2}

, which imposes no constraint; any reflection is a square root.

(a) $𝑝_{𝐴} =$ ${(𝑥 - 10)}^{2} -$ $36 =$ $(𝑥 - 16) (𝑥 - 4)$ , with $𝐸_{16} = span ((1, 1))$ and $𝐸_{4} = span ((1, - 1))$ . With $𝑃 =$ ${Pr}_{𝐸_{16}} =$ $\frac{1}{2} (\begin{matrix} 1 & 1 \\ 1 & 1 \end{matrix})$ and $𝑄 =$ ${Pr}_{𝐸_{4}} =$ $\frac{1}{2} (\begin{matrix} 1 & - 1 \\ - 1 & 1 \end{matrix})$ , we have $𝐴 = 16 𝑃 + 4 𝑄$ , and

𝐵 = \sqrt{𝐴} = 4 𝑃 + 2 𝑄 = (\begin{matrix} 3 & 1 \\ 1 & 3 \end{matrix}) .

Check: ${(\begin{matrix} 3 & 1 \\ 1 & 3 \end{matrix})}^{2} = (\begin{matrix} 10 & 6 \\ 6 & 10 \end{matrix})$ .

(b) Suppose $𝐵^{2} = 𝐴$ . Then $𝐵 𝐴 = 𝐵^{3} = 𝐴 𝐵$ , so $𝐵$ maps $𝐸_{16} =$ $ker (𝐴 - 16 𝐼_{2})$ and $𝐸_{4} = ker (𝐴 - 4 𝐼_{2})$ into themselves (Proposition 7.2.30). These are lines, so by Proposition 7.2.7, $𝐵 (1, 1) = 𝛽_{1} (1, 1)$ and $𝐵 (1, - 1) =$ $𝛽_{2} (1, - 1)$ for some $𝛽_{1}, 𝛽_{2} \in ℝ$ . Then $𝐴 (1, 1) =$ $𝐵^{2} (1, 1) =$ $𝛽_{1}^{2} (1, 1)$ , so $𝛽_{1}^{2} = 16$ , and similarly $𝛽_{2}^{2} = 4$ . Since $(1, 1), (1, - 1)$ is a basis, $𝐵 = 𝛽_{1} 𝑃 + 𝛽_{2} 𝑄$ , with $𝛽_{1} = \pm 4$ and $𝛽_{2} = \pm 2$ . Conversely each of these four matrices squares to $16 𝑃 + 4 𝑄 = 𝐴$ by Proposition 8.2.24. They are

(\begin{matrix} 3 & 1 \\ 1 & 3 \end{matrix}), (\begin{matrix} - 3 & - 1 \\ - 1 & - 3 \end{matrix}), (\begin{matrix} 1 & 3 \\ 3 & 1 \end{matrix}), (\begin{matrix} - 1 & - 3 \\ - 3 & - 1 \end{matrix}) .

(c) For $𝐼_{2}$ there is a single eigenspace, $ℝ^{2}$ itself, and the argument of (b) gives no information. Every reflection $𝑅_{𝜃} =$ $(\begin{matrix} cos 𝜃 & sin 𝜃 \\ sin 𝜃 & - cos 𝜃 \end{matrix})$ satisfies $𝑅_{𝜃}^{2} = 𝐼_{2}$ (it is ${Pr}_{ℓ} - {Pr}_{ℓ^{⟂}}$ for a line $ℓ$ ), and these are infinitely many distinct square roots. In (b), the distinct eigenvalues of $𝐴$ forced every square root to have the same eigenvectors as $𝐴$ .

Exercise 8.2.40.standardBeyond 21-242 Using Example 8.2.27 (and its analogue

𝑒^{𝑡 𝑁} =

\sum_{𝑗} 𝑒^{𝑡 𝜆_{𝑗}} 𝑃_{𝑗}

for normal

𝑁 \in ℂ^{𝑛 \times 𝑛}

), compute

𝑒^{𝑡 𝐴}

for

𝐴 = (\begin{matrix} 0 & 1 \\ 1 & 0 \end{matrix})

and

𝑒^{𝑡 𝐽}

for

𝐽 = (\begin{matrix} 0 & - 1 \\ 1 & 0 \end{matrix})

, where

𝑡 \in ℝ

. Interpret

𝑒^{𝑡 𝐽}

geometrically.

Solution.

Uses. Definition 8.2.23 (Functions of an operator), Example 8.2.27 (The exponential), Example 8.2.3 (Which matrices have orthonormal eigenbases), Theorem 6.4.31 (The matrix of an orthogonal projection)

Techniques. Apply a function to the eigenvalues (Technique 8.2.25)

Hint 1.

𝐴

has eigenvalues

\pm 1

with eigenvectors

(1, \pm 1)

. Write down

𝑃_{1}

and

𝑃_{- 1}

Hint 2.

𝐽

has eigenvalues

\pm 𝑖

with eigenvectors

(1, \mp 𝑖)

. Use

𝑒^{𝑖 𝑡} +

𝑒^{- 𝑖 𝑡} =

2 cos 𝑡

and

𝑒^{𝑖 𝑡} -

𝑒^{- 𝑖 𝑡} =

2 𝑖 sin 𝑡

Sketch.

𝑒^{𝑡 𝐴} =

𝑒^{𝑡} 𝑃_{1} +

𝑒^{- 𝑡} 𝑃_{- 1} =

(\begin{matrix} cosh 𝑡 & sinh 𝑡 \\ sinh 𝑡 & cosh 𝑡 \end{matrix})

𝑒^{𝑡 𝐽} =

𝑒^{𝑖 𝑡} 𝑃_{𝑖} +

𝑒^{- 𝑖 𝑡} 𝑃_{- 𝑖}

with

𝑃_{\pm 𝑖} =

\frac{1}{2} (\begin{matrix} 1 & \pm 𝑖 \\ \mp 𝑖 & 1 \end{matrix})

, giving

(\begin{matrix} cos 𝑡 & - sin 𝑡 \\ sin 𝑡 & cos 𝑡 \end{matrix})

: rotation by

𝑡

For $𝐴$ : $𝐸_{1} = span ((1, 1))$ and $𝐸_{- 1} = span ((1, - 1))$ , so $𝑃_{1} = \frac{1}{2} (\begin{matrix} 1 & 1 \\ 1 & 1 \end{matrix})$ and $𝑃_{- 1} = \frac{1}{2} (\begin{matrix} 1 & - 1 \\ - 1 & 1 \end{matrix})$ , and

𝑒^{𝑡 𝐴} = 𝑒^{𝑡} 𝑃_{1} + 𝑒^{- 𝑡} 𝑃_{- 1} = \frac{1}{2} (\begin{matrix} 𝑒^{𝑡} + 𝑒^{- 𝑡} & 𝑒^{𝑡} - 𝑒^{- 𝑡} \\ 𝑒^{𝑡} - 𝑒^{- 𝑡} & 𝑒^{𝑡} + 𝑒^{- 𝑡} \end{matrix}) = (\begin{matrix} cosh 𝑡 & sinh 𝑡 \\ sinh 𝑡 & cosh 𝑡 \end{matrix}) .

For $𝐽$ : by Example 8.2.3, $𝐸_{𝑖} =$ $span ((1, - 𝑖))$ and $𝐸_{- 𝑖} =$ $span ((1, 𝑖))$ . With $𝐮 = \frac{1}{\sqrt{2}} (1, - 𝑖)$ , $𝑃_{𝑖} =$ $𝐮 𝐮^{*} =$ $\frac{1}{2} (\begin{matrix} 1 \\ - 𝑖 \end{matrix}) (\begin{matrix} 1 & 𝑖 \end{matrix}) =$ $\frac{1}{2} (\begin{matrix} 1 & 𝑖 \\ - 𝑖 & 1 \end{matrix})$ , and similarly $𝑃_{- 𝑖} = \frac{1}{2} (\begin{matrix} 1 & - 𝑖 \\ 𝑖 & 1 \end{matrix})$ . So

𝑒^{𝑡 𝐽} = 𝑒^{𝑖 𝑡} 𝑃_{𝑖} + 𝑒^{- 𝑖 𝑡} 𝑃_{- 𝑖} = \frac{1}{2} (\begin{matrix} 𝑒^{𝑖 𝑡} + 𝑒^{- 𝑖 𝑡} & 𝑖 (𝑒^{𝑖 𝑡} - 𝑒^{- 𝑖 𝑡}) \\ - 𝑖 (𝑒^{𝑖 𝑡} - 𝑒^{- 𝑖 𝑡}) & 𝑒^{𝑖 𝑡} + 𝑒^{- 𝑖 𝑡} \end{matrix}) = (\begin{matrix} cos 𝑡 & - sin 𝑡 \\ sin 𝑡 & cos 𝑡 \end{matrix}),

using $𝑒^{𝑖 𝑡} +$ $𝑒^{- 𝑖 𝑡} =$ $2 cos 𝑡$ and $𝑖 (𝑒^{𝑖 𝑡} - 𝑒^{- 𝑖 𝑡}) =$ $𝑖 \cdot$ $2 𝑖 sin 𝑡 =$ $- 2 sin 𝑡$ . So $𝑒^{𝑡 𝐽}$ is the rotation by the angle $𝑡$ : the solutions of $𝐱^{'} = 𝐽 𝐱$ , that is $𝑥_{1^{'}} = - 𝑥_{2}$ and $𝑥_{2^{'}} = 𝑥_{1}$ , run around circles at unit angular speed. The complex projections were only a tool; the answer is real.

Exercise 8.2.41.standard Identify each curve in $ℝ^{2}$ (ellipse, hyperbola, pair of lines, or empty), and find its principal axes.

(a) $𝑥^{2} + 4 𝑥 𝑦 + 𝑦^{2} = 1$
(b) $3 𝑥^{2} + 2 𝑥 𝑦 + 3 𝑦^{2} = 8$
(c) $2 𝑥^{2} + 4 𝑥 𝑦 + 2 𝑦^{2} = 1$

Solution.

Uses. Corollary 8.2.28 (Principal axes), Remark 8.2.30 (Classifying conics)

Hint 1. Write each left side as

𝐱^{𝖳} 𝐴 𝐱

, splitting the cross term evenly.

Hint 2. All three matrices have eigenvectors

(1, 1)

and

(1, - 1)

Sketch. (a)

𝐴 = (\begin{matrix} 1 & 2 \\ 2 & 1 \end{matrix})

, eigenvalues

3, - 1

3 𝑦_{1}^{2} - 𝑦_{2}^{2} = 1

, a hyperbola with vertices on the line

𝑦 = 𝑥

. (b)

𝐴 = (\begin{matrix} 3 & 1 \\ 1 & 3 \end{matrix})

, eigenvalues

4, 2

𝑦_{1}^{2} / 2 + 𝑦_{2}^{2} / 4 = 1

, an ellipse with semi-axes

\sqrt{2}

along

(1, 1)

and

2

along

(1, - 1)

. (c)

𝐴 = (\begin{matrix} 2 & 2 \\ 2 & 2 \end{matrix})

, eigenvalues

4, 0

4 𝑦_{1}^{2} = 1

, the lines

𝑥 + 𝑦 = \pm 1 / \sqrt{2}

In each case let $𝐪_{1} = \frac{1}{\sqrt{2}} (1, 1)$ and $𝐪_{2} = \frac{1}{\sqrt{2}} (1, - 1)$ , and $𝑦_{1} = (𝑥 + 𝑦) / \sqrt{2}$ , $𝑦_{2} = (𝑥 - 𝑦) / \sqrt{2}$ . Every symmetric matrix $(\begin{matrix} 𝑎 & 𝑏 \\ 𝑏 & 𝑎 \end{matrix})$ has $𝐴 𝐪_{1} = (𝑎 + 𝑏) 𝐪_{1}$ and $𝐴 𝐪_{2} = (𝑎 - 𝑏) 𝐪_{2}$ , so Corollary 8.2.28 applies with these coordinates. The principal axes are the lines $𝑦 = 𝑥$ and $𝑦 = - 𝑥$ in all three cases.

(a) $𝐴 = (\begin{matrix} 1 & 2 \\ 2 & 1 \end{matrix})$ , eigenvalues $3$ and $- 1$ . The curve is $3 𝑦_{1}^{2} - 𝑦_{2}^{2} = 1$ : a hyperbola, with vertices at $𝑦_{1} = \pm 1 / \sqrt{3}$ , $𝑦_{2} = 0$ , that is, at $\pm \frac{1}{\sqrt{6}} (1, 1)$ on the line $𝑦 = 𝑥$ .

(b) $𝐴 = (\begin{matrix} 3 & 1 \\ 1 & 3 \end{matrix})$ , eigenvalues $4$ and $2$ . The curve is $4 𝑦_{1}^{2} + 2 𝑦_{2}^{2} = 8$ , that is, $𝑦_{1}^{2} / 2 + 𝑦_{2}^{2} / 4 = 1$ : an ellipse with semi-axis $\sqrt{2}$ along $𝑦 = 𝑥$ and semi-axis $2$ along $𝑦 = - 𝑥$ .

(c) $𝐴 = (\begin{matrix} 2 & 2 \\ 2 & 2 \end{matrix})$ , eigenvalues $4$ and $0$ . The curve is $4 𝑦_{1}^{2} = 1$ , that is, $𝑦_{1} = \pm 1 / 2$ : the two parallel lines $𝑥 + 𝑦 = \pm 1 / \sqrt{2}$ . (Directly: $2 {(𝑥 + 𝑦)}^{2} = 1$ .)

Exercise 8.2.42.standard Let $𝑇 \in L (𝑉)$ be self-adjoint, and let $𝜆_{min}$ and $𝜆_{max}$ be its smallest and largest eigenvalues.

(a)Show that $𝜆_{min} {‖ 𝐯 ‖}^{2} \leq$ $⟨ 𝑇 𝐯, 𝐯 ⟩ \leq$ $𝜆_{max} {‖ 𝐯 ‖}^{2}$ for every $𝐯 \in 𝑉$ , with equality on the right if and only if $𝐯 \in 𝐸_{𝜆_{max}}$ .
(b)Deduce that for a real symmetric $𝐴 \in ℝ^{𝑛 \times 𝑛}$ , $𝜆_{min} \leq$ $𝑎_{𝑖 𝑖} \leq$ $𝜆_{max}$ for every $𝑖$ .
(c)Show that $(\begin{matrix} 1 & 2 & 0 \\ 2 & 1 & 2 \\ 0 & 2 & 1 \end{matrix})$ has an eigenvalue greater than $3$ .

Solution.

Uses. Theorem 8.2.12 (Real spectral theorem), Corollary 8.2.7 (Hermitian matrices), Proposition 6.3.18 (Coordinates in an orthonormal basis), Theorem 6.3.22 (Parseval’s identity)

Techniques. Pass to orthonormal coordinates (Technique 6.7.2)

Hint 1. Expand

𝐯 = \sum 𝑐_{𝑖} 𝐮_{𝑖}

in an orthonormal eigenbasis. Then

⟨ 𝑇 𝐯, 𝐯 ⟩ =

\sum 𝜇_{𝑖} {| 𝑐_{𝑖} |}^{2}

and

{‖ 𝐯 ‖}^{2} = \sum {| 𝑐_{𝑖} |}^{2}

Hint 2. For (b), take

𝐯 = 𝐞_{𝑖}

. For (c), find a vector

𝐯

with

⟨ 𝐴 𝐯, 𝐯 ⟩ > 3 {‖ 𝐯 ‖}^{2}

Sketch. (a)

⟨ 𝑇 𝐯, 𝐯 ⟩ =

\sum_{𝑖} 𝜇_{𝑖} {| 𝑐_{𝑖} |}^{2}

is a weighted average of the eigenvalues times

\sum {| 𝑐_{𝑖} |}^{2} = {‖ 𝐯 ‖}^{2}

; equality on the right iff

𝑐_{𝑖} = 0

whenever

𝜇_{𝑖} < 𝜆_{max}

. (b)

𝐞_{𝑖}^{𝖳} 𝐴 𝐞_{𝑖} =

𝑎_{𝑖 𝑖}

. (c)

𝐯 = (1, 1, 1)

𝐯^{𝖳} 𝐴 𝐯 =

3 +

2 \cdot

(2 + 2) =

11 >

9

(a) Let $𝐮_{1}, \dots, 𝐮_{𝑛}$ be an orthonormal eigenbasis with $𝑇 𝐮_{𝑖} = 𝜇_{𝑖} 𝐮_{𝑖}$ (Theorem 8.2.12 or Corollary 8.2.7), and write $𝐯 = \sum_{𝑖} 𝑐_{𝑖} 𝐮_{𝑖}$ . Then $𝑇 𝐯 =$ $\sum_{𝑖} 𝜇_{𝑖} 𝑐_{𝑖} 𝐮_{𝑖}$ , and by orthonormality

⟨ 𝑇 𝐯, 𝐯 ⟩ = \sum_{𝑖} 𝜇_{𝑖} 𝑐_{𝑖} 𝑐_{𝑖} = \sum_{𝑖} 𝜇_{𝑖} {| 𝑐_{𝑖} |}^{2}, {‖ 𝐯 ‖}^{2} = \sum_{𝑖} {| 𝑐_{𝑖} |}^{2}

(Theorem 6.3.22). Since $𝜆_{min} \leq 𝜇_{𝑖} \leq 𝜆_{max}$ and ${| 𝑐_{𝑖} |}^{2} \geq 0$ ,

𝜆_{min} \sum_{𝑖} {| 𝑐_{𝑖} |}^{2} \leq \sum_{𝑖} 𝜇_{𝑖} {| 𝑐_{𝑖} |}^{2} \leq 𝜆_{max} \sum_{𝑖} {| 𝑐_{𝑖} |}^{2} .

Equality on the right means $\sum_{𝑖} (𝜆_{max} - 𝜇_{𝑖}) {| 𝑐_{𝑖} |}^{2} =$ $0$ , a sum of nonnegative terms, so $𝑐_{𝑖} = 0$ whenever $𝜇_{𝑖} \neq 𝜆_{max}$ ; that is, $𝐯 \in$ $span {𝐮_{𝑖} | 𝜇_{𝑖} = 𝜆_{max}} =$ $𝐸_{𝜆_{max}}$ (as in the proof of Theorem 8.2.18). Conversely, $𝐯 \in 𝐸_{𝜆_{max}}$ gives $⟨ 𝑇 𝐯, 𝐯 ⟩ =$ $𝜆_{max} {‖ 𝐯 ‖}^{2}$ .

(b) Apply (a) to $𝑇_{𝐴}$ on $ℝ^{𝑛}$ with the dot product and $𝐯 = 𝐞_{𝑖}$ : $⟨ 𝐴 𝐞_{𝑖}, 𝐞_{𝑖} ⟩ =$ $𝑎_{𝑖 𝑖}$ and $‖ 𝐞_{𝑖} ‖ = 1$ .

(c) With $𝐯 = (1, 1, 1)$ , $𝐴 𝐯 = (3, 5, 3)$ , so $⟨ 𝐴 𝐯, 𝐯 ⟩ = 11$ while ${‖ 𝐯 ‖}^{2} = 3$ . By (a), $𝜆_{max} \geq 11 / 3 > 3$ . (Part (b) alone gives only $𝜆_{max} \geq 1$ . In fact $𝜆_{max} = 1 + 2 \sqrt{2} \approx 3.83$ .)

Exercise 8.2.43.hard Let $𝐴 \in ℝ^{𝑛 \times 𝑛}$ be symmetric and nonzero. Show that

rank 𝐴 \geq \frac{{(tr 𝐴)}^{2}}{tr (𝐴^{2})},

and determine when equality holds.

Solution.

Uses. Theorem 8.2.12 (Real spectral theorem), Proposition 4.5.27 (Similarity invariants), Proposition 6.2.22 (The square of a sum), Proposition 7.4.23 (Powers and polynomials of a diagonalizable matrix)

Techniques. Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1. Diagonalize: all three quantities can be read off from the eigenvalues

𝜇_{1}, \dots, 𝜇_{𝑛}

Hint 2.

rank 𝐴

is the number of nonzero

𝜇_{𝑖}

tr 𝐴 = \sum 𝜇_{𝑖}

, and

tr (𝐴^{2}) = \sum 𝜇_{𝑖}^{2} > 0

. Apply Cauchy–Schwarz to the nonzero

𝜇_{𝑖}

Sketch.

𝐴 = 𝑄 𝐷 𝑄^{𝖳}

; rank and trace are similarity invariants, and

𝐴^{2} = 𝑄 𝐷^{2} 𝑄^{𝖳}

. With

𝑟

nonzero eigenvalues,

{(\sum 𝜇_{𝑖})}^{2} \leq 𝑟 \sum 𝜇_{𝑖}^{2}

by Cauchy–Schwarz (Proposition 6.2.22), with equality iff the nonzero

𝜇_{𝑖}

are equal. Since

𝐴 \neq 0

tr (𝐴^{2}) > 0

By Theorem 8.2.12, $𝐴 = 𝑄 𝐷 𝑄^{𝖳}$ with $𝑄$ orthogonal and $𝐷 =$ $diag (𝜇_{1}, \dots, 𝜇_{𝑛})$ , and $𝐴^{2} = 𝑄 𝐷^{2} 𝑄^{𝖳}$ (Proposition 7.4.23). Rank and trace are invariant under similarity (Proposition 4.5.27), so with $𝑟$ the number of nonzero $𝜇_{𝑖}$ ,

rank 𝐴 = rank 𝐷 = 𝑟, tr 𝐴 = \sum_{𝑖} 𝜇_{𝑖}, tr (𝐴^{2}) = \sum_{𝑖} 𝜇_{𝑖}^{2} .

Since $𝐴 \neq 0$ , $𝐷 \neq 0$ , so $𝑟 \geq 1$ and $tr (𝐴^{2}) > 0$ . The sum $\sum_{𝑖} 𝜇_{𝑖}$ only involves the $𝑟$ nonzero eigenvalues, and by Proposition 6.2.22 (Cauchy–Schwarz for the vectors $(1, \dots, 1)$ and ${(𝜇_{𝑖})}_{𝜇_{𝑖} \neq 0}$ in $ℝ^{𝑟}$ ),

{(\sum_{𝑖} 𝜇_{𝑖})}^{2} \leq 𝑟 \sum_{𝜇_{𝑖} \neq 0} 𝜇_{𝑖}^{2} = 𝑟 tr (𝐴^{2}) .

Dividing by $tr (𝐴^{2}) > 0$ gives the inequality. Equality holds in Cauchy–Schwarz if and only if ${(𝜇_{𝑖})}_{𝜇_{𝑖} \neq 0}$ is a multiple of $(1, \dots, 1)$ , that is, if and only if all nonzero eigenvalues of $𝐴$ are equal. Equivalently, $𝐴 = 𝑐 𝑃$ for a scalar $𝑐 \neq 0$ and an orthogonal projection $𝑃$ (by Theorem 8.2.18).

Exercise 8.2.44.hard Let

𝑉

be a finite-dimensional complex inner product space and

𝑇 \in L (𝑉)

. Show that

𝑇

is normal if and only if every eigenvector of

𝑇

is also an eigenvector of

𝑇^{*}

Solution.

Uses. Theorem 8.2.5 (Complex spectral theorem), Exercise 6.6.38, Theorem 7.3.27 (Existence of eigenvalues over $ℂ$ ), Corollary 6.4.27 (Testing whether a subspace is everything)

Techniques. Extremal choice and minimal counterexamples (Technique 1.5.9), Move an operator across the inner product (Technique 6.6.20)

Hint 1. One direction was shown inside the second proof of Theorem 8.2.5. For the other, imitate the third proof.

Hint 2. If every vector of

𝐸_{𝜆}

is an eigenvector of

𝑇^{*}

, then

𝑇^{*} ↾_{𝐸_{𝜆}}

is an operator all of whose nonzero vectors are eigenvectors. What kind of operator is that?

Hint 3. Show that

𝑇^{*} 𝐮 = 𝜆 𝐮

for

𝐮 \in 𝐸_{𝜆}

by comparing

⟨ 𝑇 𝐮, 𝐮 ⟩

and

⟨ 𝐮, 𝑇^{*} 𝐮 ⟩

. Then the third proof of Theorem 8.2.5 runs unchanged.

Sketch. (

\Rightarrow

) shown in the second proof of Theorem 8.2.5. (

⟸

) For a nonzero

𝐮 \in 𝐸_{𝜆}

𝑇^{*} 𝐮 = 𝜈 𝐮

, and

𝜆 {‖ 𝐮 ‖}^{2} =

⟨ 𝑇 𝐮, 𝐮 ⟩ =

⟨ 𝐮, 𝑇^{*} 𝐮 ⟩ =

𝜈 {‖ 𝐮 ‖}^{2}

, so

𝜈 = 𝜆

. Now the third proof applies:

𝐸 = \sum 𝐸_{𝜆}

𝑇^{*}

-invariant,

𝐸^{⟂}

𝑇

-invariant and contains no eigenvector, so

𝐸 = 𝑉

; eigenspaces are orthogonal. So

𝑇

has an orthonormal eigenbasis and is normal.

( $\Rightarrow$ ) If $𝑇$ is normal and $𝑇 𝐮 = 𝜆 𝐮$ , then $𝑇^{*} 𝐮 = 𝜆 𝐮$ , as shown in the second proof of Theorem 8.2.5.

( $⟸$ ) Let $𝐮 \neq 𝟎$ with $𝑇 𝐮 = 𝜆 𝐮$ . By hypothesis $𝑇^{*} 𝐮 = 𝜈 𝐮$ for some $𝜈 \in ℂ$ , and

𝜆 {‖ 𝐮 ‖}^{2} = ⟨ 𝑇 𝐮, 𝐮 ⟩ = ⟨ 𝐮, 𝑇^{*} 𝐮 ⟩ = ⟨ 𝐮, 𝜈 𝐮 ⟩ = 𝜈 {‖ 𝐮 ‖}^{2},

so $𝜈 = 𝜆$ . Thus $𝑇^{*} 𝐮 = 𝜆 𝐮$ for every $𝐮 \in 𝐸_{𝜆}$ (including $𝟎$ ). This is the only property of normal operators used in the third proof of Theorem 8.2.5, so that proof applies verbatim: $𝐸 = \sum_{𝜆} 𝐸_{𝜆}$ is $𝑇^{*}$ -invariant, so $𝐸^{⟂}$ is $𝑇$ -invariant (Exercise 6.6.38); an eigenvector of $𝑇 ↾_{𝐸^{⟂}}$ would lie in $𝐸 \cap 𝐸^{⟂} = {𝟎}$ , so $𝐸^{⟂} = {𝟎}$ and $𝐸 = 𝑉$ (Theorem 7.3.27, Corollary 6.4.27); and $𝜆 ⟨ 𝐮, 𝐯 ⟩ =$ $⟨ 𝐮, 𝑇^{*} 𝐯 ⟩ =$ $𝜇 ⟨ 𝐮, 𝐯 ⟩$ for $𝐮 \in 𝐸_{𝜆}$ , $𝐯 \in 𝐸_{𝜇}$ shows that the eigenspaces are orthogonal. Concatenating orthonormal bases of the eigenspaces gives an orthonormal eigenbasis, so $𝑇$ is normal by Theorem 8.2.5.

Exercise 8.2.45.standard

(a)Show that $𝐴 \in ℝ^{2 \times 2}$ satisfies $𝐴^{𝖳} 𝐴 = 𝐴 𝐴^{𝖳}$ if and only if $𝐴$ is symmetric or $𝐴 = (\begin{matrix} 𝑎 & - 𝑏 \\ 𝑏 & 𝑎 \end{matrix})$ for some $𝑎, 𝑏 \in ℝ$ .
(b)Deduce that a real normal $2 \times 2$ matrix that is not symmetric has no real eigenvalue, and interpret $(\begin{matrix} 𝑎 & - 𝑏 \\ 𝑏 & 𝑎 \end{matrix})$ geometrically.

Solution.

Uses. Definition 8.1.8 (Normal operator, normal matrix), Theorem 7.3.6 (Eigenvalues are the roots of the characteristic polynomial)

Techniques. Compute entry by entry (Technique 2.4.15)

Hint 1. Write

𝐴 = (\begin{matrix} 𝑎 & 𝑏 \\ 𝑐 & 𝑑 \end{matrix})

and compare the entries of

𝐴^{𝖳} 𝐴

and

𝐴 𝐴^{𝖳}

Hint 2. The diagonal entries give

𝑏^{2} = 𝑐^{2}

. Split into the cases

𝑐 = 𝑏

and

𝑐 = - 𝑏 \neq 0

Sketch.

𝐴^{𝖳} 𝐴 =

(\begin{matrix} 𝑎^{2} + 𝑐^{2} & 𝑎 𝑏 + 𝑐 𝑑 \\ 𝑎 𝑏 + 𝑐 𝑑 & 𝑏^{2} + 𝑑^{2} \end{matrix})

and

𝐴 𝐴^{𝖳} =

(\begin{matrix} 𝑎^{2} + 𝑏^{2} & 𝑎 𝑐 + 𝑏 𝑑 \\ 𝑎 𝑐 + 𝑏 𝑑 & 𝑐^{2} + 𝑑^{2} \end{matrix})

. Equality means

𝑏^{2} = 𝑐^{2}

and

𝑎 𝑏 +

𝑐 𝑑 =

𝑎 𝑐 +

𝑏 𝑑

. If

𝑐 = 𝑏

𝐴

is symmetric. If

𝑐 = - 𝑏 \neq 0

, the second equation gives

2 𝑏 (𝑎 - 𝑑) = 0

, so

𝑑 = 𝑎

. (b)

𝑝 = {(𝑥 - 𝑎)}^{2} + 𝑏^{2}

has no real root for

𝑏 \neq 0

; the matrix is

\sqrt{𝑎^{2} + 𝑏^{2}}

times a rotation.

(a) Let $𝐴 = (\begin{matrix} 𝑎 & 𝑏 \\ 𝑐 & 𝑑 \end{matrix})$ . Then

𝐴^{𝖳} 𝐴 = (\begin{matrix} 𝑎^{2} + 𝑐^{2} & 𝑎 𝑏 + 𝑐 𝑑 \\ 𝑎 𝑏 + 𝑐 𝑑 & 𝑏^{2} + 𝑑^{2} \end{matrix}), 𝐴 𝐴^{𝖳} = (\begin{matrix} 𝑎^{2} + 𝑏^{2} & 𝑎 𝑐 + 𝑏 𝑑 \\ 𝑎 𝑐 + 𝑏 𝑑 & 𝑐^{2} + 𝑑^{2} \end{matrix}) .

So $𝐴$ is normal if and only if $𝑏^{2} = 𝑐^{2}$ and $𝑎 𝑏 +$ $𝑐 𝑑 =$ $𝑎 𝑐 +$ $𝑏 𝑑$ . If $𝑐 = 𝑏$ , the second equation holds and $𝐴$ is symmetric. Otherwise $𝑐 = - 𝑏$ with $𝑏 \neq 0$ , and the second equation becomes $𝑎 𝑏 -$ $𝑏 𝑑 =$ $- 𝑎 𝑏 +$ $𝑏 𝑑$ , that is, $2 𝑏 (𝑎 - 𝑑) = 0$ , so $𝑑 = 𝑎$ and $𝐴 = (\begin{matrix} 𝑎 & 𝑏 \\ - 𝑏 & 𝑎 \end{matrix})$ , which is of the stated form with $𝑏$ replaced by $- 𝑏$ . Conversely, symmetric matrices are normal, and for $𝐴 = (\begin{matrix} 𝑎 & - 𝑏 \\ 𝑏 & 𝑎 \end{matrix})$ both products equal $(𝑎^{2} + 𝑏^{2}) 𝐼_{2}$ .

(b) If $𝐴$ is normal and not symmetric, then $𝐴 = (\begin{matrix} 𝑎 & - 𝑏 \\ 𝑏 & 𝑎 \end{matrix})$ with $𝑏 \neq 0$ , and $𝑝_{𝐴} =$ ${(𝑥 - 𝑎)}^{2} +$ $𝑏^{2} >$ $0$ for every real $𝑥$ . So $𝐴$ has no real eigenvalue (Theorem 7.3.6). Writing $𝑟 = \sqrt{𝑎^{2} + 𝑏^{2}} > 0$ and $(𝑎, 𝑏) =$ $𝑟 (cos 𝜃, sin 𝜃)$ , we get $𝐴 =$ $𝑟 (\begin{matrix} cos 𝜃 & - sin 𝜃 \\ sin 𝜃 & cos 𝜃 \end{matrix})$ : a rotation by $𝜃$ followed by scaling by $𝑟$ . It is multiplication by $𝑎 + 𝑏 𝑖$ on $ℂ = ℝ^{2}$ .

Exercise 8.2.46.standard Adapt the third proof of Theorem 8.2.5 (the sum of the eigenspaces is everything) to give another proof of (I)

\Rightarrow

(II) in Theorem 8.2.12. Point out exactly where self-adjointness is used.

Solution.

Uses. Lemma 8.2.10 (Self-adjoint operators have eigenvalues), Exercise 6.6.38, Corollary 6.4.27 (Testing whether a subspace is everything), Theorem 6.4.20 (Orthonormal bases exist), Definition 6.6.26 (Self-adjoint operator)

Techniques. Extremal choice and minimal counterexamples (Technique 1.5.9)

Hint 1. Let

𝐸

be the sum of the eigenspaces of

𝑇

. Why is

𝐸^{⟂}

invariant? (This is easier than in the complex case.)

Hint 2. To find an eigenvector in

𝐸^{⟂}

, you need

𝑇 ↾_{𝐸^{⟂}}

to be an operator that has one. Which lemma provides it?

Sketch.

𝐸 = \sum 𝐸_{𝜆}

𝑇

-invariant, and

𝑇 = 𝑇^{*}

, so

𝐸^{⟂}

𝑇

-invariant (Exercise 6.6.38). The restriction

𝑇 ↾_{𝐸^{⟂}}

is self-adjoint, so if

𝐸^{⟂} \neq {𝟎}

it has an eigenvector (Lemma 8.2.10), which would lie in

𝐸 \cap 𝐸^{⟂}

. So

𝐸 = 𝑉

. Orthogonality:

𝜆 ⟨ 𝐮, 𝐯 ⟩ =

⟨ 𝑇 𝐮, 𝐯 ⟩ =

⟨ 𝐮, 𝑇 𝐯 ⟩ =

𝜇 ⟨ 𝐮, 𝐯 ⟩

Let $𝑇$ be self-adjoint, let $𝜆_{1}, \dots, 𝜆_{𝑘}$ be its distinct eigenvalues ( $𝑘 \geq 1$ by Lemma 8.2.10), and let $𝐸 = 𝐸_{𝜆_{1}} + \dots + 𝐸_{𝜆_{𝑘}}$ . Each $𝐸_{𝜆_{𝑗}}$ is $𝑇$ -invariant, so $𝐸$ is $𝑇$ -invariant. Since $𝑇^{*} = 𝑇$ , Exercise 6.6.38 shows that $𝐸^{⟂}$ is $𝑇^{*}$ -invariant, that is, $𝑇$ -invariant. (This is the first use of self-adjointness; in the complex case we needed the fact that eigenvectors of $𝑇$ are eigenvectors of $𝑇^{*}$ .)

Suppose $𝐸^{⟂} \neq {𝟎}$ . The restriction $𝑆 = 𝑇 ↾_{𝐸^{⟂}}$ is self-adjoint, since $⟨ 𝑆 𝐯, 𝐰 ⟩ =$ $⟨ 𝑇 𝐯, 𝐰 ⟩ =$ $⟨ 𝐯, 𝑇 𝐰 ⟩ =$ $⟨ 𝐯, 𝑆 𝐰 ⟩$ for $𝐯, 𝐰 \in 𝐸^{⟂}$ . By Lemma 8.2.10 (the second use of self-adjointness, replacing the existence of eigenvalues over $ℂ$ ), $𝑆$ has an eigenvector $𝐰 \in 𝐸^{⟂}$ . It is an eigenvector of $𝑇$ , so $𝐰 \in 𝐸 \cap 𝐸^{⟂} = {𝟎}$ , a contradiction. So $𝐸^{⟂} = {𝟎}$ and $𝐸 = 𝑉$ by Corollary 6.4.27.

For $𝐮 \in 𝐸_{𝜆}$ and $𝐯 \in 𝐸_{𝜇}$ with $𝜆 \neq 𝜇$ , $𝜆 ⟨ 𝐮, 𝐯 ⟩ =$ $⟨ 𝑇 𝐮, 𝐯 ⟩ =$ $⟨ 𝐮, 𝑇 𝐯 ⟩ =$ $𝜇 ⟨ 𝐮, 𝐯 ⟩$ (the third use; the eigenvalues are real), so $⟨ 𝐮, 𝐯 ⟩ = 0$ . Concatenating orthonormal bases of the eigenspaces (Theorem 6.4.20) gives an orthonormal list spanning $𝐸 = 𝑉$ , which is an orthonormal eigenbasis.

Exercise 8.2.47.hard Let $𝐻 \in ℂ^{𝑛 \times 𝑛}$ be Hermitian.

(a)Show that $𝐼_{𝑛} + 𝑖 𝐻$ is invertible.
(b)Show that $𝐶 =$ $(𝐼_{𝑛} - 𝑖 𝐻) {(𝐼_{𝑛} + 𝑖 𝐻)}^{- 1}$ is unitary, and that $- 1$ is not an eigenvalue of $𝐶$ .
(c)Conversely, show that every unitary $𝐶$ without the eigenvalue $- 1$ arises in this way from a unique Hermitian $𝐻$ .

Solution.

Uses. Corollary 8.2.7 (Hermitian matrices), Theorem 8.2.5 (Complex spectral theorem), Proposition 8.2.24 (Properties of $𝑓 (𝑇)$ ), Exercise 8.2.33

Techniques. Apply a function to the eigenvalues (Technique 8.2.25)

Hint 1. Write

𝐻 = 𝑈 𝐷 𝑈^{*}

with

𝐷

real diagonal. What are the eigenvalues of

𝐼 + 𝑖 𝐻

Hint 2.

𝐶 = 𝑓 (𝐻)

for

𝑓 (𝜆) =

(1 - 𝑖 𝜆) / (1 + 𝑖 𝜆)

. What is

| 𝑓 (𝜆) |

for real

𝜆

Hint 3. For (c), solve

𝑧 =

(1 - 𝑖 𝜆) / (1 + 𝑖 𝜆)

for

𝜆

, and apply that function to

𝐶

, which is normal.

Sketch. (a) The eigenvalues of

𝐼 + 𝑖 𝐻

are

1 + 𝑖 𝜆_{𝑗} \neq 0

. (b)

𝐶 = 𝑈 𝑓 (𝐷) 𝑈^{*}

with

𝑓 (𝜆) =

(1 - 𝑖 𝜆) / (1 + 𝑖 𝜆)

; for real

𝜆

| 𝑓 (𝜆) | = 1

and

𝑓 (𝜆) \neq - 1

; so

𝐶^{*} 𝐶 = 𝐼

. (c)

𝐶

is normal with eigenvalues

𝑧

on the unit circle,

𝑧 \neq - 1

;

𝑔 (𝑧) =

𝑖 (𝑧 - 1) / (𝑧 + 1)

is real there and inverts

𝑓

, so

𝐻 = 𝑔 (𝐶)

is Hermitian with

𝑓 (𝐻) = 𝐶

. Uniqueness:

𝐻 = 𝑔 (𝑓 (𝐻))

By Corollary 8.2.7, $𝐻 = 𝑈 𝐷 𝑈^{*}$ with $𝑈$ unitary and $𝐷 =$ $diag (𝜆_{1}, \dots, 𝜆_{𝑛})$ real.

(a) $𝐼_{𝑛} +$ $𝑖 𝐻 =$ $𝑈 (𝐼_{𝑛} + 𝑖 𝐷) 𝑈^{*}$ , and $𝐼_{𝑛} + 𝑖 𝐷$ is diagonal with entries $1 + 𝑖 𝜆_{𝑗}$ , which are nonzero because $𝜆_{𝑗}$ is real. So $𝐼_{𝑛} + 𝑖 𝐻$ is a product of invertible matrices.

(b) Let $𝑓 (𝜆) =$ $(1 - 𝑖 𝜆) / (1 + 𝑖 𝜆)$ for $𝜆 \in ℝ$ . With the matrix form of Definition 8.2.23, $(𝐼_{𝑛} - 𝑖 𝐻) {(𝐼_{𝑛} + 𝑖 𝐻)}^{- 1} =$ $𝑈 (𝐼_{𝑛} - 𝑖 𝐷) {(𝐼_{𝑛} + 𝑖 𝐷)}^{- 1} 𝑈^{*} =$ $𝑈 𝑓 (𝐷) 𝑈^{*} =$ $𝑓 (𝐻)$ . For real $𝜆$ , $1 - 𝑖 𝜆 = 1 + 𝑖 𝜆$ , so $| 𝑓 (𝜆) | = 1$ ; and $𝑓 (𝜆) = - 1$ would mean $1 - 𝑖 𝜆 = - 1 - 𝑖 𝜆$ , which is impossible. So $𝐶 = 𝑈 𝑓 (𝐷) 𝑈^{*}$ with $𝑓 (𝐷)$ diagonal, unimodular entries, and $𝐶^{*} 𝐶 =$ $𝑈 𝑓 (𝐷) 𝑓 (𝐷) 𝑈^{*} =$ $𝐼_{𝑛}$ : $𝐶$ is unitary, and its eigenvalues $𝑓 (𝜆_{𝑗})$ are not $- 1$ .

(c) Let $𝐶$ be unitary with no eigenvalue $- 1$ . Then $𝐶^{*} 𝐶 = 𝐼_{𝑛} = 𝐶 𝐶^{*}$ , so $𝐶$ is normal, and by Theorem 8.2.5, $𝐶 = 𝑉 𝑍 𝑉^{*}$ with $𝑉$ unitary and $𝑍 =$ $diag (𝑧_{1}, \dots, 𝑧_{𝑛})$ ; $| 𝑧_{𝑗} | = 1$ by Exercise 8.2.33(d), and $𝑧_{𝑗} \neq - 1$ . Let $𝑔 (𝑧) =$ $𝑖 (𝑧 - 1) / (𝑧 + 1)$ . For $| 𝑧 | = 1$ , $𝑧 = 1 / 𝑧$ , so

𝑔 (𝑧) = - 𝑖 \frac{1 / 𝑧 - 1}{1 / 𝑧 + 1} = - 𝑖 \frac{1 - 𝑧}{1 + 𝑧} = 𝑔 (𝑧),

and $𝑔 (𝑧)$ is real. Solving $𝑧 =$ $(1 - 𝑖 𝜆) / (1 + 𝑖 𝜆)$ for $𝜆$ gives $𝑧 + 𝑖 𝜆 𝑧 = 1 - 𝑖 𝜆$ , so $𝜆 =$ $𝑖 (𝑧 - 1) / (𝑧 + 1) =$ $𝑔 (𝑧)$ : thus $𝑓 (𝑔 (𝑧)) = 𝑧$ and $𝑔 (𝑓 (𝜆)) = 𝜆$ . Let $𝐻 =$ $𝑔 (𝐶) =$ $𝑉 𝑔 (𝑍) 𝑉^{*}$ . It is Hermitian since $𝑔 (𝑍)$ is real diagonal, and $𝑓 (𝐻) =$ $𝑉 𝑓 (𝑔 (𝑍)) 𝑉^{*} =$ $𝑉 𝑍 𝑉^{*} =$ $𝐶$ . For uniqueness, if $𝑓 (𝐻^{'}) = 𝐶$ with $𝐻^{'}$ Hermitian, $𝐻^{'} = 𝑈^{'} 𝐷^{'} 𝑈^{' *}$ , then $𝐶 = 𝑈^{'} 𝑓 (𝐷^{'}) 𝑈^{' *}$ , and $𝑔 (𝐶) =$ $𝑈^{'} 𝑔 (𝑓 (𝐷^{'})) 𝑈^{' *} =$ $𝐻^{'}$ (the matrix $𝑓 (𝐷^{'})$ is a diagonalization of $𝐶$ , and $𝑔 (𝐶)$ does not depend on the choice). So $𝐻^{'} = 𝑔 (𝐶) = 𝐻$ .

Exercise 8.2.48.challengeBeyond 21-242 Let

𝐴 \in ℝ^{𝑛 \times 𝑛}

be normal (

𝐴^{𝖳} 𝐴 = 𝐴 𝐴^{𝖳}

). Show that there is an orthogonal

𝑄

such that

𝑄^{𝖳} 𝐴 𝑄

is block diagonal, with each block either

1 \times 1

or of the form

(\begin{matrix} 𝑎 & - 𝑏 \\ 𝑏 & 𝑎 \end{matrix})

with

𝑏 > 0

Solution.

Uses. Theorem 8.2.5 (Complex spectral theorem), Exercise 8.2.45, Proposition 7.3.34 (Eigenvalues of a complexification), Exercise 6.6.38, Theorem 6.4.20 (Orthonormal bases exist), Theorem 4.5.17 (Change of basis)

Techniques. Complexify, then take real and imaginary parts (Technique 7.3.35), Induction on dimension via an invariant subspace (Technique 7.5.12)

Hint 1. Induct on

𝑛

. If

𝐴

has a real eigenvalue, split off a line as in the second proof of Theorem 8.2.5; it works over

ℝ

for a real eigenvector.

Hint 2. Otherwise take a complex eigenvector

𝐳 = 𝐱 + 𝑖 𝐲

𝐴

for

𝜆 = 𝑎 + 𝑏 𝑖

𝑏 \neq 0

. Since

𝐴

is normal as a complex matrix,

𝐴^{𝖳} 𝐳 = 𝐴^{*} 𝐳 = 𝜆 𝐳

. Take real and imaginary parts.

Hint 3.

𝑈 = span (𝐱, 𝐲)

is a plane invariant under

𝐴

and

𝐴^{𝖳}

, so

𝑈^{⟂}

is too. The restriction of

𝑇_{𝐴}

𝑈

, in an orthonormal basis of

𝑈

, is a real normal

2 \times 2

matrix without real eigenvalues. Use Exercise 8.2.45.

Sketch. Induction on

𝑛

, splitting off an invariant subspace

𝑈

of dimension

1

2

that is invariant under both

𝐴

and

𝐴^{𝖳}

; then

𝑈^{⟂}

is invariant under both, and both restrictions are normal. A real eigenvector gives

𝑈

of dimension

1

. Otherwise a complex eigenvector

𝐱 + 𝑖 𝐲

for

𝑎 + 𝑏 𝑖

(

𝑏 \neq 0

) is also an eigenvector of

𝐴^{*} = 𝐴^{𝖳}

for

𝑎 - 𝑏 𝑖

, and comparing real and imaginary parts shows

𝑈 = span (𝐱, 𝐲)

is invariant under

𝐴

and

𝐴^{𝖳}

. In an orthonormal basis of

𝑈

, the restriction is normal with no real eigenvalue, hence of the form

(\begin{matrix} 𝑎^{'} & - 𝑏^{'} \\ 𝑏^{'} & 𝑎^{'} \end{matrix})

(Exercise 8.2.45); replacing the second basis vector by its negative makes

𝑏^{'} > 0

We prove the statement for normal operators $𝑇$ on finite-dimensional real inner product spaces $𝑉$ (with “orthonormal basis” in place of $𝑄$ ), by strong induction on $dim 𝑉$ ; for $𝐴$ apply it to $𝑇_{𝐴}$ on $ℝ^{𝑛}$ and let $𝑄$ have the basis as columns (Theorem 4.5.17). Dimension $0$ is trivial. Suppose $dim 𝑉 = 𝑛 \geq 1$ . It suffices to find a subspace $𝑈 \neq {𝟎}$ of dimension $1$ or $2$ that is invariant under $𝑇$ and $𝑇^{*}$ , and an orthonormal basis of $𝑈$ in which $𝑇 ↾_{𝑈}$ has a block of the required form. Indeed, then $𝑊 = 𝑈^{⟂}$ is invariant under $𝑇$ and $𝑇^{*}$ (Exercise 6.6.38 applied to $𝑇$ and to $𝑇^{*}$ ), $𝑇 ↾_{𝑊}$ is normal with adjoint $𝑇^{*} ↾_{𝑊}$ (as in the second proof of Theorem 8.2.5), and the induction hypothesis applies to it; the two orthonormal bases together give the required basis of $𝑉 = 𝑈 \oplus 𝑊$ .

Case 1: $𝑇$ has a real eigenvalue $𝜆$ . Let $𝐮$ be a unit eigenvector. The computation $‖ (𝑇 - 𝜆 id) 𝐯 ‖ =$ $‖ (𝑇^{*} - 𝜆 id) 𝐯 ‖$ from the second proof of Theorem 8.2.5 works over $ℝ$ , so $𝑇^{*} 𝐮 = 𝜆 𝐮$ , and $𝑈 = span (𝐮)$ is invariant under $𝑇$ and $𝑇^{*}$ , with $1 \times 1$ block $(𝜆)$ .

Case 2: $𝑇$ has no real eigenvalue. Choose an orthonormal basis $𝒪︀$ of $𝑉$ and let $𝑀 = {[𝑇]}_{𝒪︀} \in ℝ^{𝑛 \times 𝑛}$ ; then $𝑀^{𝖳} = {[𝑇^{*}]}_{𝒪︀}$ and $𝑀$ is normal. As a complex matrix, $𝑀^{*} = 𝑀^{𝖳}$ , so $𝑀$ is normal in $ℂ^{𝑛 \times 𝑛}$ , and it has an eigenvalue $𝜆 = 𝑎 + 𝑏 𝑖$ with eigenvector $𝐳 = 𝐱 + 𝑖 𝐲$ , $𝐱, 𝐲 \in ℝ^{𝑛}$ ; $𝑏 \neq 0$ by assumption. By the second proof of Theorem 8.2.5, $𝑀^{𝖳} 𝐳 = 𝑀^{*} 𝐳 = 𝜆 𝐳$ . Comparing real and imaginary parts of $𝑀 𝐳 = 𝜆 𝐳$ and $𝑀^{𝖳} 𝐳 = 𝜆 𝐳$ (as in Proposition 7.3.34),

𝑀 𝐱 = 𝑎 𝐱 - 𝑏 𝐲, 𝑀 𝐲 = 𝑏 𝐱 + 𝑎 𝐲, 𝑀^{𝖳} 𝐱 = 𝑎 𝐱 + 𝑏 𝐲, 𝑀^{𝖳} 𝐲 = - 𝑏 𝐱 + 𝑎 𝐲 .

So $𝑈_{0} = span (𝐱, 𝐲)$ is invariant under $𝑀$ and $𝑀^{𝖳}$ , and it has dimension $2$ by Proposition 7.3.34(c). Let $𝑈 \subseteq 𝑉$ be the corresponding subspace (the vectors whose $𝒪︀$ -coordinates lie in $𝑈_{0}$ ); it is invariant under $𝑇$ and $𝑇^{*}$ . Let $𝐟_{1}, 𝐟_{2}$ be an orthonormal basis of $𝑈$ and $𝑁 =$ ${[𝑇 ↾_{𝑈}]}_{𝐟_{1}, 𝐟_{2}} \in$ $ℝ^{2 \times 2}$ . Since $𝑇 ↾_{𝑈}$ is normal with adjoint $𝑇^{*} ↾_{𝑈}$ , $𝑁$ is normal (Theorem 6.6.15). It has no real eigenvalue, since $𝑇$ has none, so by Exercise 8.2.45, $𝑁 =$ $(\begin{matrix} 𝑎^{'} & - 𝑏^{'} \\ 𝑏^{'} & 𝑎^{'} \end{matrix})$ with $𝑏^{'} \neq 0$ . If $𝑏^{'} < 0$ , replace $𝐟_{2}$ by $- 𝐟_{2}$ , which changes the signs of the off-diagonal entries. This gives the required block.

More practice

The practice bank has 20 exercises on this section’s topics.