8.3 Positive operators · Matrix Theory

Among complex numbers, the nonnegative reals stand out: they are the numbers of the form $𝑤 𝑤$ , and also the real numbers that are squares of real numbers. In the analogy between operators and complex numbers (Section 6.6.5), the adjoint plays the role of complex conjugation and self-adjoint operators play the role of real numbers. This section finds the operators that play the role of nonnegative numbers. We have already met several: orthogonal projections, the matrices $𝐴^{*} 𝐴$ of the normal equations (Theorem 6.5.5), and Gram matrices (Definition 6.3.28). Positive operators have nonnegative eigenvalues and a unique positive square root, and the positive definite ones are exactly the matrices of inner products. In Section 9.1 the positive operator $𝑇^{*} 𝑇$ produces the singular value decomposition.

Throughout, $𝑉$ is a finite-dimensional inner product space over $𝔽$ with $dim 𝑉 \geq 1$ , and $𝑇 \in L (𝑉)$ ; only the definitions make sense more generally. Matrices in $𝔽^{𝑛 \times 𝑛}$ act on $𝔽^{𝑛}$ with the dot product, so that $⟨ 𝐴 𝐱, 𝐱 ⟩ = 𝐱^{*} 𝐴 𝐱$ . For an orthonormal basis $𝒪︀$ we write ${[𝑇]}_{𝒪︀}$ for ${[𝑇]}_{𝒪︀ 𝒪︀}$ , as in Section 8.2.

8.3.1Positive operators

Definition 8.3.1 (Positive operator). Let $𝑉$ be an inner product space. An operator $𝑇 \in L (𝑉)$ is positive if $𝑇$ is self-adjoint and

⟨ 𝑇 𝐯, 𝐯 ⟩ \geq 0 for every 𝐯 \in 𝑉 .

A matrix $𝐴 \in 𝔽^{𝑛 \times 𝑛}$ is positive if $𝑇_{𝐴}$ is positive on $𝔽^{𝑛}$ with the dot product, that is, if $𝐴^{*} = 𝐴$ and $𝐱^{*} 𝐴 𝐱 \geq 0$ for every $𝐱 \in 𝔽^{𝑛}$ .

The name is traditional but a little misleading: “positive” here means “nonnegative”, in the sense in which $0 \geq 0$ . For matrices the same notion is often called positive semidefinite. The strict version, $⟨ 𝑇 𝐯, 𝐯 ⟩ > 0$ for $𝐯 \neq 𝟎$ , is positive definiteness (Definition 8.3.14).

Intuition (What the condition says). In a real space,

⟨ 𝑇 𝐯, 𝐯 ⟩ \geq 0

says that

𝑇 𝐯

never makes an obtuse angle with

𝐯

𝑇

may stretch, shrink, and turn a vector, but never by more than a right angle. For a self-adjoint

𝑇

the spectral theorem says more:

𝑇

stretches

𝑉

along perpendicular axes, and the stretch factors are its eigenvalues. On the axis of a unit eigenvector

𝐮

with

𝑇 𝐮 = 𝜆 𝐮

we have

⟨ 𝑇 𝐮, 𝐮 ⟩ = 𝜆

, so positivity says at least that no axis is flipped. Theorem 8.3.5 shows that this is all it says.

Example 8.3.2 (Positive operators).

(a)Every orthogonal projection $𝑃 = {Pr}_{𝑈}$ is positive. It is self-adjoint and $𝑃^{2} = 𝑃$ (Corollary 6.6.29), so
$⟨ 𝑃 𝐯, 𝐯 ⟩ = ⟨ 𝑃^{2} 𝐯, 𝐯 ⟩ = ⟨ 𝑃 𝐯, 𝑃 𝐯 ⟩ = {‖ 𝑃 𝐯 ‖}^{2} \geq 0 .$
(b)Let $𝑊$ be an inner product space and let $𝑆 \in L (𝑉, 𝑊)$ have an adjoint. Then $𝑆^{*} 𝑆 \in L (𝑉)$ is positive: ${(𝑆^{*} 𝑆)}^{*} =$ $𝑆^{*} {(𝑆^{*})}^{*} =$ $𝑆^{*} 𝑆$ by Proposition 6.6.18, and $⟨ 𝑆^{*} 𝑆 𝐯, 𝐯 ⟩ =$ $⟨ 𝑆 𝐯, 𝑆 𝐯 ⟩ =$ ${‖ 𝑆 𝐯 ‖}^{2} \geq$ $0$ . In particular $𝐴^{*} 𝐴$ is positive for every $𝐴 \in 𝔽^{𝑚 \times 𝑛}$ ; it is the coefficient matrix of the normal equations (Theorem 6.5.5).
(c)The Gram matrix $𝐺$ of a list $𝐯_{1}, \dots, 𝐯_{𝑘}$ in $𝑉$ (Definition 6.3.28) is positive. Its entries satisfy $𝑔_{𝑗 𝑖} =$ $⟨ 𝐯_{𝑖}, 𝐯_{𝑗} ⟩ =$ $⟨ 𝐯_{𝑗}, 𝐯_{𝑖} ⟩ =$ $𝑔_{𝑖 𝑗}$ , so $𝐺^{*} = 𝐺$ , and $𝐜^{*} 𝐺 𝐜 =$ ${‖ 𝑐_{1} 𝐯_{1} + \dots + 𝑐_{𝑘} 𝐯_{𝑘} ‖}^{2} \geq$ $0$ by Proposition 6.3.30. For matrices, (b) is the special case of the columns of $𝐴$ , whose Gram matrix is $𝐴^{*} 𝐴$ .
(d) $𝐴 = (\begin{matrix} 2 & - 1 \\ - 1 & 2 \end{matrix})$ is positive. It is Hermitian, and for $𝐱 \in ℂ^{2}$ ,
$𝐱^{*} 𝐴 𝐱 = 2 {| 𝑥_{1} |}^{2} - 2 Re (𝑥_{1} 𝑥_{2}) + 2 {| 𝑥_{2} |}^{2} = {| 𝑥_{1} - 𝑥_{2} |}^{2} + {| 𝑥_{1} |}^{2} + {| 𝑥_{2} |}^{2} \geq 0 .$

Non-example 8.3.3 (Matrices that are not positive).

(a) $𝐴 = (\begin{matrix} 1 & 2 \\ 2 & 1 \end{matrix})$ is symmetric with positive entries, but for $𝐱 = (1, - 1)$ we get $𝐱^{𝖳} 𝐴 𝐱 =$ $1 -$ $2 -$ $2 +$ $1 =$ $- 2 <$ $0$ . Together with Example 8.3.2(d), this shows that the signs of the entries say little about positivity.
(b)On $ℝ^{2}$ , let $𝐵 = (\begin{matrix} 1 & - 1 \\ 1 & 1 \end{matrix})$ . Then $𝐱^{𝖳} 𝐵 𝐱 =$ $𝑥_{1}^{2} -$ $𝑥_{1} 𝑥_{2} +$ $𝑥_{2} 𝑥_{1} +$ $𝑥_{2}^{2} =$ ${‖ 𝐱 ‖}^{2} \geq$ $0$ for every $𝐱 \in ℝ^{2}$ , but $𝐵$ is not symmetric, so it is not positive. Geometrically, $𝐵$ is $\sqrt{2}$ times the rotation by $𝜋 / 4$ : it turns every vector by an angle of $𝜋 / 4$ , less than a right angle, and its characteristic polynomial ${(𝑥 - 1)}^{2} + 1$ has no real roots.

Remark 8.3.4 (Why self-adjointness is part of the definition). Over $ℂ$ the self-adjointness in Definition 8.3.1 comes for free. If $⟨ 𝑇 𝐯, 𝐯 ⟩ \geq 0$ for every $𝐯$ , then in particular every value $⟨ 𝑇 𝐯, 𝐯 ⟩$ is real, and $𝑇$ is self-adjoint by Proposition 6.6.30. So on a complex space, $𝑇$ is positive if and only if $⟨ 𝑇 𝐯, 𝐯 ⟩ \geq 0$ for every $𝐯 \in 𝑉$ .

Over $ℝ$ it does not come for free, and Non-example 8.3.3(b) shows why we must ask for it. On a real space $⟨ 𝑇^{*} 𝐯, 𝐯 ⟩ =$ $⟨ 𝐯, 𝑇 𝐯 ⟩ =$ $⟨ 𝑇 𝐯, 𝐯 ⟩$ , so

⟨ 𝑇 𝐯, 𝐯 ⟩ = ⟨ \frac{1}{2} (𝑇 + 𝑇^{*}) 𝐯, 𝐯 ⟩ for every 𝐯 \in 𝑉 .

The values $⟨ 𝑇 𝐯, 𝐯 ⟩$ only see the self-adjoint part $(𝑇 + 𝑇^{*}) / 2$ of $𝑇$ (Exercise 6.1.52), and the condition $⟨ 𝑇 𝐯, 𝐯 ⟩ \geq 0$ says nothing about the rest. For $𝐵$ , the self-adjoint part is $𝐼_{2}$ and the rest is the rotation $(\begin{matrix} 0 & - 1 \\ 1 & 0 \end{matrix})$ . Without the self-adjointness requirement, $𝐵$ would count as positive although it has no real eigenvalues, it is not of the form $𝑆^{*} 𝑆$ (such operators are self-adjoint), and it has no self-adjoint square root (the square of a self-adjoint operator is self-adjoint). With the requirement, the real and complex notions agree: a real matrix is positive in $ℝ^{𝑛 \times 𝑛}$ if and only if it is positive in $ℂ^{𝑛 \times 𝑛}$ (Exercise 8.3.30), and the characterizations below hold over both fields with the same proofs.

Intuition (Positive operators as nonnegative numbers). A complex number

𝑧

is a nonnegative real number exactly when it can be written in any one of the forms

𝑧 = 𝑤 𝑤

with

𝑤 \in ℂ

𝑧 = 𝑟^{2}

with

𝑟 \in ℝ

, or

𝑧 = 𝑟^{2}

with

𝑟 \geq 0

. Each form has an operator analogue:

𝑇 = 𝑆^{*} 𝑆

𝑇 = 𝑅^{2}

with

𝑅

self-adjoint, and

𝑇 = 𝑅^{2}

with

𝑅

positive. One more description makes sense only for operators. By the spectral theorem, a self-adjoint operator is described by real numbers, its eigenvalues, attached to perpendicular axes, and we can ask that these numbers be nonnegative. The next theorem says that all of these descriptions agree with the definition.

8.3.2Characterizations

Theorem 8.3.5 (Characterizations of positive operators). Let $𝑉$ be a finite-dimensional inner product space with $dim 𝑉 \geq 1$ , and let $𝑇 \in L (𝑉)$ . The following are equivalent:

(I) $𝑇$ is positive;
(II) $𝑇$ is self-adjoint and every eigenvalue of $𝑇$ is nonnegative;
(III) ${[𝑇]}_{𝒪︀}$ is diagonal with nonnegative diagonal entries for some orthonormal basis $𝒪︀$ of $𝑉$ ;
(IV) $𝑇 = 𝑅^{2}$ for some positive $𝑅 \in L (𝑉)$ ;
(V) $𝑇 = 𝑅^{2}$ for some self-adjoint $𝑅 \in L (𝑉)$ ;
(VI) $𝑇 = 𝑆^{*} 𝑆$ for some $𝑆 \in L (𝑉, 𝑊)$ , where $𝑊$ is a finite-dimensional inner product space;
(VII)for some orthonormal basis $𝒪︀$ of $𝑉$ , ${[𝑇]}_{𝒪︀}$ is the Gram matrix of a list of vectors in some inner product space.

In matrix form: $𝐴 \in 𝔽^{𝑛 \times 𝑛}$ is positive if and only if $𝐴 = 𝑈 𝐷 𝑈^{*}$ with $𝑈$ unitary and $𝐷$ diagonal with nonnegative entries, if and only if $𝐴 = 𝐵^{*} 𝐵$ for some $𝐵 \in 𝔽^{𝑚 \times 𝑛}$ , if and only if $𝐴$ is a Gram matrix.

Over $ℂ$ , condition (I) may be replaced by “ $⟨ 𝑇 𝐯, 𝐯 ⟩ \geq 0$ for every $𝐯$ ” (Remark 8.3.4).

Intuition (How the cycle is arranged). Seven conditions need seven implications if we arrange them in a cycle (Technique 2.7.1). Most of the cycle runs downhill, each condition handing the next one a witness: (II)

\Rightarrow

(III) is the spectral theorem, (III)

\Rightarrow

(IV) takes square roots of the diagonal entries, and (IV)

\Rightarrow

(V)

\Rightarrow

(VI) forget information. The cycle closes with one computation. Once

𝑇 = 𝑆^{*} 𝑆

, we have

⟨ 𝑇 𝐯, 𝐯 ⟩ = {‖ 𝑆 𝐯 ‖}^{2}

, and the Gram condition (VII) is the same computation written in coordinates. The only deep step is the spectral theorem.

Proof.standard

Uses. Definition 8.3.1 (Positive operator), Theorem 8.2.12 (Real spectral theorem), Corollary 8.2.7 (Hermitian matrices), Theorem 7.4.2 (Diagonalizable means an eigenbasis), Theorem 2.2.1 (Linear maps are determined by a basis), Corollary 6.6.28 (Self-adjointness in an orthonormal basis), Proposition 6.3.18 (Coordinates in an orthonormal basis), Definition 6.3.28 (Gram matrix), Proposition 6.3.30 (Gram matrices detect independence), Theorem 6.6.15 (Existence of adjoints), Definition 8.2.1 (Unitarily and orthogonally diagonalizable)

Techniques. Proving TFAE (Technique 2.7.1), Gram matrices (Technique 6.3.31), Pass to orthonormal coordinates (Technique 6.7.2)

Hint 1. Use the cycle (I)

\Rightarrow

(II)

\Rightarrow

\dots

\Rightarrow

(VII)

\Rightarrow

(I). For (I)

\Rightarrow

(II), put an eigenvector into

⟨ 𝑇 𝐯, 𝐯 ⟩

Hint 2. (II)

\Rightarrow

(III) is a spectral theorem. For (III)

\Rightarrow

(IV), define

𝑅

on the orthonormal eigenbasis by taking square roots of the eigenvalues.

Hint 3. For (VI)

\Rightarrow

(VII), compute the entries

⟨ 𝑇 𝐞_{𝑗}, 𝐞_{𝑖} ⟩

{[𝑇]}_{𝒪︀}

when

𝑇 = 𝑆^{*} 𝑆

. For (VII)

\Rightarrow

(I), expand

⟨ 𝑇 𝐯, 𝐯 ⟩

in the coordinates

𝐜 = {[𝐯]}_{𝒪︀}

and use Proposition 6.3.30.

Sketch. (I)

\Rightarrow

(II):

𝜆 {‖ 𝐯 ‖}^{2} =

⟨ 𝑇 𝐯, 𝐯 ⟩ \geq

0

for an eigenvector. (II)

\Rightarrow

(III): spectral theorem. (III)

\Rightarrow

(IV):

𝑅 𝐞_{𝑖} = \sqrt{𝑑_{𝑖}} 𝐞_{𝑖}

on the eigenbasis. (IV)

\Rightarrow

(V)

\Rightarrow

(VI):

𝑅

is self-adjoint, so

𝑇 = 𝑅^{*} 𝑅

. (VI)

\Rightarrow

(VII):

⟨ 𝑇 𝐞_{𝑗}, 𝐞_{𝑖} ⟩ =

⟨ 𝑆 𝐞_{𝑗}, 𝑆 𝐞_{𝑖} ⟩

, so

{[𝑇]}_{𝒪︀}

is the Gram matrix of

𝑆 𝐞_{1}, \dots, 𝑆 𝐞_{𝑛}

. (VII)

\Rightarrow

(I): a Gram matrix is Hermitian, and

⟨ 𝑇 𝐯, 𝐯 ⟩ =

𝐜^{*} 𝐺 𝐜 =

{‖ \sum_{𝑗} 𝑐_{𝑗} 𝐰_{𝑗} ‖}^{2}

We prove (I) $\Rightarrow$ (II) $\Rightarrow$ (III) $\Rightarrow$ (IV) $\Rightarrow$ (V) $\Rightarrow$ (VI) $\Rightarrow$ (VII) $\Rightarrow$ (I).

(I) $\Rightarrow$ (II). $𝑇$ is self-adjoint by definition. Let $𝜆$ be an eigenvalue of $𝑇$ , with eigenvector $𝐯 \neq 𝟎$ . Then

𝜆 {‖ 𝐯 ‖}^{2} = ⟨ 𝜆 𝐯, 𝐯 ⟩ = ⟨ 𝑇 𝐯, 𝐯 ⟩ \geq 0,

and since ${‖ 𝐯 ‖}^{2} > 0$ , $𝜆 =$ $⟨ 𝑇 𝐯, 𝐯 ⟩ / {‖ 𝐯 ‖}^{2}$ is a nonnegative real number.

(II) $\Rightarrow$ (III). By Theorem 8.2.12 if $𝔽 = ℝ$ , or Corollary 8.2.7 if $𝔽 = ℂ$ , $𝑉$ has an orthonormal basis $𝒪︀ = 𝐞_{1}, \dots, 𝐞_{𝑛}$ of eigenvectors of $𝑇$ , say $𝑇 𝐞_{𝑖} = 𝑑_{𝑖} 𝐞_{𝑖}$ . Then ${[𝑇]}_{𝒪︀} =$ $diag (𝑑_{1}, \dots, 𝑑_{𝑛})$ (Theorem 7.4.2), and each $𝑑_{𝑖}$ is an eigenvalue of $𝑇$ , so $𝑑_{𝑖} \geq 0$ .

(III) $\Rightarrow$ (IV). Let $𝒪︀ = 𝐞_{1}, \dots, 𝐞_{𝑛}$ be an orthonormal basis with ${[𝑇]}_{𝒪︀} =$ $diag (𝑑_{1}, \dots, 𝑑_{𝑛})$ and every $𝑑_{𝑖} \geq 0$ , so that $𝑇 𝐞_{𝑖} = 𝑑_{𝑖} 𝐞_{𝑖}$ . By Theorem 2.2.1 there is $𝑅 \in L (𝑉)$ with $𝑅 𝐞_{𝑖} = \sqrt{𝑑_{𝑖}} 𝐞_{𝑖}$ for every $𝑖$ . Its matrix ${[𝑅]}_{𝒪︀} =$ $diag (\sqrt{𝑑_{1}}, \dots, \sqrt{𝑑_{𝑛}})$ is real and diagonal, hence Hermitian, so $𝑅$ is self-adjoint (Corollary 6.6.28). For $𝐯 =$ $𝑐_{1} 𝐞_{1} +$ $\dots +$ $𝑐_{𝑛} 𝐞_{𝑛}$ , orthonormality gives

⟨ 𝑅 𝐯, 𝐯 ⟩ = ⟨ \sum_{𝑖} \sqrt{𝑑_{𝑖}} 𝑐_{𝑖} 𝐞_{𝑖}, \sum_{𝑗} 𝑐_{𝑗} 𝐞_{𝑗} ⟩ = \sum_{𝑖} \sqrt{𝑑_{𝑖}} {| 𝑐_{𝑖} |}^{2} \geq 0,

so $𝑅$ is positive. Finally $𝑅^{2} 𝐞_{𝑖} =$ $\sqrt{𝑑_{𝑖}} \sqrt{𝑑_{𝑖}} 𝐞_{𝑖} =$ $𝑑_{𝑖} 𝐞_{𝑖} =$ $𝑇 𝐞_{𝑖}$ for every $𝑖$ , so $𝑅^{2} = 𝑇$ by the uniqueness part of Theorem 2.2.1.

(IV) $\Rightarrow$ (V). A positive operator is self-adjoint.

(V) $\Rightarrow$ (VI). Take $𝑊 = 𝑉$ and $𝑆 = 𝑅$ . Then $𝑆^{*} 𝑆 = 𝑅 𝑅 = 𝑇$ .

(VI) $\Rightarrow$ (VII). Let $𝒪︀ = 𝐞_{1}, \dots, 𝐞_{𝑛}$ be any orthonormal basis of $𝑉$ . By Proposition 6.3.18, the $𝑖$ -th coordinate of $𝑇 𝐞_{𝑗}$ is $⟨ 𝑇 𝐞_{𝑗}, 𝐞_{𝑖} ⟩$ , so the $(𝑖, 𝑗)$ entry of ${[𝑇]}_{𝒪︀}$ is

⟨ 𝑇 𝐞_{𝑗}, 𝐞_{𝑖} ⟩ = ⟨ 𝑆^{*} 𝑆 𝐞_{𝑗}, 𝐞_{𝑖} ⟩ = ⟨ 𝑆 𝐞_{𝑗}, 𝑆 𝐞_{𝑖} ⟩ .

This is the $(𝑖, 𝑗)$ entry of the Gram matrix of the list $𝑆 𝐞_{1}, \dots, 𝑆 𝐞_{𝑛}$ in $𝑊$ (Definition 6.3.28).

(VII) $\Rightarrow$ (I). Let $𝒪︀ = 𝐞_{1}, \dots, 𝐞_{𝑛}$ be an orthonormal basis such that $𝐺 = {[𝑇]}_{𝒪︀}$ is the Gram matrix of a list $𝐰_{1}, \dots, 𝐰_{𝑛}$ in an inner product space $𝑊$ . As in Example 8.3.2(c), $𝐺^{*} = 𝐺$ , so ${[𝑇^{*}]}_{𝒪︀} =$ $𝐺^{*} =$ ${[𝑇]}_{𝒪︀}$ by Theorem 6.6.15, and $𝑇^{*} = 𝑇$ . Let $𝐯 =$ $𝑐_{1} 𝐞_{1} +$ $\dots +$ $𝑐_{𝑛} 𝐞_{𝑛}$ and $𝐜 = (𝑐_{1}, \dots, 𝑐_{𝑛})$ . Since $𝑔_{𝑖 𝑗} =$ $⟨ 𝑇 𝐞_{𝑗}, 𝐞_{𝑖} ⟩$ ,

⟨ 𝑇 𝐯, 𝐯 ⟩ = \sum_{𝑖, 𝑗} 𝑐_{𝑗} 𝑐_{𝑖} ⟨ 𝑇 𝐞_{𝑗}, 𝐞_{𝑖} ⟩ = \sum_{𝑖, 𝑗} 𝑐_{𝑖} 𝑔_{𝑖 𝑗} 𝑐_{𝑗} = 𝐜^{*} 𝐺 𝐜 = {‖ 𝑐_{1} 𝐰_{1} + \dots + 𝑐_{𝑛} 𝐰_{𝑛} ‖}^{2} \geq 0

by Proposition 6.3.30. So $𝑇$ is positive.

For the matrix form, apply the theorem to $𝑇_{𝐴}$ on $𝔽^{𝑛}$ ; its matrix in the standard basis $ℰ︀$ is $𝐴$ . Condition (III) for $𝑇_{𝐴}$ says that $𝔽^{𝑛}$ has an orthonormal eigenbasis for $𝐴$ with nonnegative eigenvalues, which means $𝐴 = 𝑈 𝐷 𝑈^{*}$ with $𝑈$ unitary (its columns are the eigenbasis) and $𝐷$ diagonal with nonnegative entries, as explained after Definition 8.2.1. If $𝐴 = 𝐵^{*} 𝐵$ with $𝐵 \in 𝔽^{𝑚 \times 𝑛}$ , then $𝑇_{𝐴} = {(𝑇_{𝐵})}^{*} 𝑇_{𝐵}$ , because the adjoint of $𝑇_{𝐵}$ is $𝑇_{𝐵^{*}}$ (Theorem 6.6.15); this is (VI). Conversely, (V) gives $𝐴 = 𝐵^{*} 𝐵$ with $𝐵 = {[𝑅]}_{ℰ︀}$ . Finally, if $𝐴$ is a Gram matrix, then (VII) holds with $𝒪︀ = ℰ︀$ , so $𝐴$ is positive; and if $𝐴$ is positive, the proof of (VI) $\Rightarrow$ (VII), which worked for every orthonormal basis, shows that $𝐴 = {[𝑇_{𝐴}]}_{ℰ︀}$ is a Gram matrix. □

The proof of (VI) $\Rightarrow$ (VII) did not depend on the orthonormal basis. So for a positive $𝑇$ the matrix in every orthonormal basis is a Gram matrix, and in particular its diagonal entries $⟨ 𝑇 𝐞_{𝑖}, 𝐞_{𝑖} ⟩$ are nonnegative. For bases that are not orthonormal this fails (Exercise 8.3.29). Note also that conditions (V) and (VI) look weaker than (IV). A self-adjoint square root may have negative eigenvalues, and $𝑆$ need not be self-adjoint or even map $𝑉$ to itself, yet each of them forces $𝑇$ to be positive.

Example 8.3.6 (Three certificates of positivity). Let

𝐴 = (\begin{matrix} 2 & - 1 & 0 \\ - 1 & 2 & - 1 \\ 0 & - 1 & 2 \end{matrix}) .

Each of three conditions of Theorem 8.3.5 can be checked directly.

Eigenvalues (II). $𝐴$ is symmetric, and expanding $det (𝑥 𝐼_{3} - 𝐴)$ along the first row gives $𝑝_{𝐴} =$ ${(𝑥 - 2)}^{3} -$ $2 (𝑥 - 2) =$ $(𝑥 - 2) (𝑥^{2} - 4 𝑥 + 2)$ . The eigenvalues $2$ and $2 \pm \sqrt{2}$ are all positive.

The form (I). For $𝐱 \in ℝ^{3}$ ,

𝐱^{𝖳} 𝐴 𝐱 = 2 𝑥_{1}^{2} + 2 𝑥_{2}^{2} + 2 𝑥_{3}^{2} - 2 𝑥_{1} 𝑥_{2} - 2 𝑥_{2} 𝑥_{3} = 𝑥_{1}^{2} + {(𝑥_{1} - 𝑥_{2})}^{2} + {(𝑥_{2} - 𝑥_{3})}^{2} + 𝑥_{3}^{2} \geq 0 .

A factorization (VI). $𝐴 = 𝐵^{𝖳} 𝐵$ for the $4 \times 3$ difference matrix

𝐵 = (\begin{matrix} 1 & 0 & 0 \\ - 1 & 1 & 0 \\ 0 & - 1 & 1 \\ 0 & 0 & - 1 \end{matrix}), 𝐵 𝐱 = (𝑥_{1}, 𝑥_{2} - 𝑥_{1}, 𝑥_{3} - 𝑥_{2}, - 𝑥_{3}),

so $𝐴$ is the Gram matrix of the columns of $𝐵$ .

The last two certificates are the same computation, since $𝐱^{𝖳} 𝐵^{𝖳} 𝐵 𝐱 =$ ${‖ 𝐵 𝐱 ‖}^{2}$ is exactly the sum of squares. The factorization is the most useful in practice: a sum of squares or a product $𝐵^{*} 𝐵$ certifies positivity without computing any eigenvalue. (The matrix $𝐴$ is the second-difference matrix, a close relative of the graph Laplacians of Section 16.5.)

A symmetric

2 \times 2

matrix

𝐴

maps the unit circle to an ellipse with axes along the orthonormal eigenvectors

𝐪_{1}, 𝐪_{2}

𝐴

is positive exactly when neither

𝜆_{1} 𝐪_{1}

nor

𝜆_{2} 𝐪_{2}

is reversed, which is the same as saying that

𝐴 𝐱

never points more than a right angle away from

𝐱

. Drag

𝐴 𝐞_{1}

and

𝐴 𝐞_{2}

, or type entries, until an eigenvalue turns negative, then move

𝐱

to that eigenvector.

Exercise 8.3.7.routine Decide which of the following matrices are positive, and justify each answer by a condition of Theorem 8.3.5 or by a vector $𝐱$ with $𝐱^{*} 𝐴 𝐱 < 0$ .

(a) $(\begin{matrix} 3 & 1 \\ 1 & 3 \end{matrix}) \in ℝ^{2 \times 2}$ .
(b) $(\begin{matrix} 1 & 3 \\ 3 & 1 \end{matrix}) \in ℝ^{2 \times 2}$ .
(c) $(\begin{matrix} 1 & 𝑖 \\ - 𝑖 & 1 \end{matrix}) \in ℂ^{2 \times 2}$ .
(d) $(\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix}) \in ℝ^{2 \times 2}$ .
(e) $(\begin{matrix} 2 & 1 \\ - 1 & 2 \end{matrix}) \in ℝ^{2 \times 2}$ .

Solution.

Uses. Theorem 8.3.5 (Characterizations of positive operators), Definition 8.3.1 (Positive operator)

Hint 1. First check self-adjointness. For the symmetric ones, compute the eigenvalues from the trace and the determinant.

Hint 2. For (c), try to write

𝐱^{*} 𝐴 𝐱

{| \dots |}^{2}

, or

𝐴

𝐵^{*} 𝐵

with

𝐵

1 \times 2

matrix.

Sketch. (a) Positive: eigenvalues

4, 2

. (b) Not positive: eigenvalues

4, - 2

, and

𝐱 = (1, - 1)

gives

- 4

. (c) Positive:

𝐴 = 𝐵^{*} 𝐵

for

𝐵 = (\begin{matrix} 1 & 𝑖 \end{matrix})

, so

𝐱^{*} 𝐴 𝐱 =

{| 𝑥_{1} + 𝑖 𝑥_{2} |}^{2}

. (d) and (e) are not symmetric, so not positive, although

𝐱^{𝖳} 𝐴 𝐱 = 2 {‖ 𝐱 ‖}^{2} \geq 0

in (e).

(a) The matrix is symmetric with trace $6$ and determinant $8$ , so its eigenvalues are $4$ and $2$ (the roots of $𝑥^{2} - 6 𝑥 + 8$ ). Both are nonnegative, so it is positive by Theorem 8.3.5(II).

(b) The matrix is symmetric, but its eigenvalues are the roots of $𝑥^{2} - 2 𝑥 - 8$ , namely $4$ and $- 2$ . So it is not positive; concretely, $𝐱 = (1, - 1)$ gives $𝐱^{𝖳} 𝐴 𝐱 =$ $1 -$ $3 -$ $3 +$ $1 =$ $- 4 <$ $0$ .

(c) Let $𝐵 = (\begin{matrix} 1 & 𝑖 \end{matrix}) \in ℂ^{1 \times 2}$ . Then

𝐵^{*} 𝐵 = (\begin{matrix} 1 \\ - 𝑖 \end{matrix}) (\begin{matrix} 1 & 𝑖 \end{matrix}) = (\begin{matrix} 1 & 𝑖 \\ - 𝑖 & - 𝑖 \cdot 𝑖 \end{matrix}) = (\begin{matrix} 1 & 𝑖 \\ - 𝑖 & 1 \end{matrix}),

so the matrix is positive by Theorem 8.3.5(VI). Explicitly, $𝐱^{*} 𝐴 𝐱 =$ ${| 𝐵 𝐱 |}^{2} =$ ${| 𝑥_{1} + 𝑖 𝑥_{2} |}^{2} \geq$ $0$ . Its eigenvalues are $0$ and $2$ .

(d) The matrix is not symmetric, so it is not positive. (Also $𝐱^{𝖳} 𝐴 𝐱 = 𝑥_{1} 𝑥_{2}$ , which is $- 1$ at $𝐱 = (1, - 1)$ .)

(e) Here $𝐱^{𝖳} 𝐴 𝐱 =$ $2 𝑥_{1}^{2} +$ $𝑥_{1} 𝑥_{2} -$ $𝑥_{2} 𝑥_{1} +$ $2 𝑥_{2}^{2} =$ $2 {‖ 𝐱 ‖}^{2} \geq$ $0$ for every $𝐱 \in ℝ^{2}$ , but the matrix is not symmetric, so it is not positive (Remark 8.3.4).

The last characterization we need concerns vectors on which the form $⟨ 𝑇 𝐯, 𝐯 ⟩$ vanishes. For a positive number $𝑡$ , $𝑡 𝑥^{2} = 0$ forces $𝑥 = 0$ ; for a positive operator, the analogue is that $⟨ 𝑇 𝐯, 𝐯 ⟩ = 0$ forces $𝑇 𝐯 = 𝟎$ .

Lemma 8.3.8 (Where the form vanishes). Let $𝑇 \in L (𝑉)$ be positive and $𝐯 \in 𝑉$ . Then $⟨ 𝑇 𝐯, 𝐯 ⟩ = 0$ if and only if $𝑇 𝐯 = 𝟎$ . Consequently

ker 𝑇 = {𝐯 \in 𝑉 | ⟨ 𝑇 𝐯, 𝐯 ⟩ = 0} .

Proof (in an eigenbasis).routine

Uses. Theorem 8.3.5 (Characterizations of positive operators), Proposition 6.3.18 (Coordinates in an orthonormal basis)

Techniques. A sum of nonnegative terms (Technique 6.1.9), Pass to orthonormal coordinates (Technique 6.7.2)

Hint 1. Only one direction needs proof. Take an orthonormal basis as in Theorem 8.3.5(III) and expand

⟨ 𝑇 𝐯, 𝐯 ⟩

in it.

Hint 2. You get

\sum_{𝑖} 𝑑_{𝑖} {| 𝑐_{𝑖} |}^{2}

with every

𝑑_{𝑖} \geq 0

. When is such a sum

0

Sketch. In an orthonormal eigenbasis with

𝑇 𝐞_{𝑖} = 𝑑_{𝑖} 𝐞_{𝑖}

𝑑_{𝑖} \geq 0

, and

𝐯 = \sum 𝑐_{𝑖} 𝐞_{𝑖}

⟨ 𝑇 𝐯, 𝐯 ⟩ =

\sum_{𝑖} 𝑑_{𝑖} {| 𝑐_{𝑖} |}^{2}

. If this is

0

, every term is

0

, so

𝑑_{𝑖} 𝑐_{𝑖} = 0

for every

𝑖

, and

𝑇 𝐯 =

\sum_{𝑖} 𝑑_{𝑖} 𝑐_{𝑖} 𝐞_{𝑖} =

𝟎

If $𝑇 𝐯 = 𝟎$ , then $⟨ 𝑇 𝐯, 𝐯 ⟩ = 0$ . Conversely, by Theorem 8.3.5(III) there is an orthonormal basis $𝐞_{1}, \dots, 𝐞_{𝑛}$ with $𝑇 𝐞_{𝑖} = 𝑑_{𝑖} 𝐞_{𝑖}$ and every $𝑑_{𝑖} \geq 0$ . Write $𝐯 = \sum_{𝑖} 𝑐_{𝑖} 𝐞_{𝑖}$ . Then

⟨ 𝑇 𝐯, 𝐯 ⟩ = ⟨ \sum_{𝑖} 𝑑_{𝑖} 𝑐_{𝑖} 𝐞_{𝑖}, \sum_{𝑗} 𝑐_{𝑗} 𝐞_{𝑗} ⟩ = \sum_{𝑖} 𝑑_{𝑖} {| 𝑐_{𝑖} |}^{2} .

If this is $0$ , then, being a sum of nonnegative terms, each term $𝑑_{𝑖} {| 𝑐_{𝑖} |}^{2}$ is $0$ . So for each $𝑖$ , either $𝑑_{𝑖} = 0$ or $𝑐_{𝑖} = 0$ , and in both cases $𝑑_{𝑖} 𝑐_{𝑖} = 0$ . Hence $𝑇 𝐯 =$ $\sum_{𝑖} 𝑑_{𝑖} 𝑐_{𝑖} 𝐞_{𝑖} =$ $𝟎$ . □

Proof (with a square root).routine

Uses. Theorem 8.3.5 (Characterizations of positive operators), Definition 6.1.26 (Norm)

Techniques. Show a vector is zero via its norm (Technique 6.1.36), Move an operator across the inner product (Technique 6.6.20)

Hint. Write

𝑇 = 𝑅^{2}

with

𝑅

positive (Theorem 8.3.5(IV)) and move one factor

𝑅

across the inner product.

Sketch. With

𝑇 = 𝑅^{2}

and

𝑅

self-adjoint,

⟨ 𝑇 𝐯, 𝐯 ⟩ =

⟨ 𝑅 𝐯, 𝑅 𝐯 ⟩ =

{‖ 𝑅 𝐯 ‖}^{2}

. If this is

0

, then

𝑅 𝐯 = 𝟎

and

𝑇 𝐯 = 𝑅 (𝑅 𝐯) = 𝟎

By Theorem 8.3.5(IV), $𝑇 = 𝑅^{2}$ for some positive, hence self-adjoint, $𝑅$ . Then

⟨ 𝑇 𝐯, 𝐯 ⟩ = ⟨ 𝑅 (𝑅 𝐯), 𝐯 ⟩ = ⟨ 𝑅 𝐯, 𝑅 𝐯 ⟩ = {‖ 𝑅 𝐯 ‖}^{2} .

If this is $0$ , then $𝑅 𝐯 = 𝟎$ , and $𝑇 𝐯 =$ $𝑅 (𝑅 𝐯) =$ $𝑅 𝟎 =$ $𝟎$ . □

The second proof is the general pattern: once the form $⟨ 𝑇 𝐯, 𝐯 ⟩$ is a squared norm ${‖ 𝑅 𝐯 ‖}^{2}$ , its vanishing kills $𝑅 𝐯$ and therefore $𝑇 𝐯$ . Without positivity the lemma fails. On $ℝ^{2}$ the rotation $𝐽 = (\begin{matrix} 0 & - 1 \\ 1 & 0 \end{matrix})$ has $⟨ 𝐽 𝐯, 𝐯 ⟩ = 0$ for every $𝐯$ (Example 6.1.39), but $ker 𝐽 = {𝟎}$ ; and the symmetric matrix $(\begin{matrix} 1 & 0 \\ 0 & - 1 \end{matrix})$ gives $0$ at $𝐯 = (1, 1)$ , which is not in its kernel.

8.3.3The positive square root

Every nonnegative real number has exactly one nonnegative square root, although a positive number has two real square roots. Operators have many more square roots. Every reflection squares to the identity (Exercise 8.2.39), so when $dim 𝑉 \geq 2$ the identity has infinitely many self-adjoint square roots. Asking for a positive square root restores uniqueness.

Theorem 8.3.9 (The positive square root). Let $𝑇 \in L (𝑉)$ be positive, with spectral decomposition $𝑇 =$ $𝜆_{1} 𝑃_{1} +$ $\dots +$ $𝜆_{𝑘} 𝑃_{𝑘}$ (Theorem 8.2.18). Then

𝑅 = \sqrt{𝜆_{1}} 𝑃_{1} + \dots + \sqrt{𝜆_{𝑘}} 𝑃_{𝑘}

is the only positive operator $𝑅 \in L (𝑉)$ with $𝑅^{2} = 𝑇$ . Moreover $𝑅 = 𝑝 (𝑇)$ for some polynomial $𝑝 \in ℝ [𝑥]$ , so $𝑅$ commutes with every operator that commutes with $𝑇$ .

Notation 8.3.10. For a positive

𝑇

, the operator of Theorem 8.3.9 is the positive square root of

𝑇

, written

\sqrt{𝑇}

. In the notation of Definition 8.2.23,

\sqrt{𝑇} = 𝑓 (𝑇)

for

𝑓 (𝜆) = \sqrt{𝜆}

. For a positive matrix

𝐴

\sqrt{𝐴}

is the positive matrix whose square is

𝐴

; it is the matrix of

\sqrt{𝑇_{𝐴}}

A positive operator is self-adjoint, so it satisfies the spectral hypothesis of Section 8.2.4 (over $ℂ$ because self-adjoint operators are normal), and its eigenvalues $𝜆_{𝑗}$ are nonnegative by Theorem 8.3.5. So the formula for $𝑅$ makes sense.

Proof, part 1 of 2: existence.routine

Uses. Definition 8.2.23 (Functions of an operator), Proposition 8.2.24 (Properties of $𝑓 (𝑇)$ ), Theorem 8.2.18 (Spectral decomposition), Theorem 8.3.5 (Characterizations of positive operators)

Techniques. Apply a function to the eigenvalues (Technique 8.2.25)

Hint 1. This is Example 8.2.26. Apply Proposition 8.2.24 to

𝑓 (𝜆) = \sqrt{𝜆}

on the eigenvalues.

Hint 2. Which parts of Proposition 8.2.24 give self-adjointness, the eigenvalues of

𝑓 (𝑇)

, the identity

{𝑓 (𝑇)}^{2} = 𝑇

, and the polynomial?

Sketch.

𝑅 = 𝑓 (𝑇)

for

𝑓 = \sqrt{\cdot}

{𝜆_{1}, \dots, 𝜆_{𝑘}}

. It is self-adjoint because

𝑓

is real, its eigenvalues

\sqrt{𝜆_{𝑗}}

are nonnegative, so it is positive, and

𝑅^{2} = (𝑓^{2}) (𝑇) = 𝑇

because

𝑓^{2} (𝜆_{𝑗}) = 𝜆_{𝑗}

. It equals

𝑝 (𝑇)

for the real interpolating polynomial of Proposition 8.2.24(b).

Let $𝑓 (𝜆) = \sqrt{𝜆}$ on ${𝜆_{1}, \dots, 𝜆_{𝑘}}$ , so that $𝑅 = 𝑓 (𝑇)$ (Definition 8.2.23). By Proposition 8.2.24(d), $𝑅$ is self-adjoint, since every value $𝑓 (𝜆_{𝑗})$ is real, and by Proposition 8.2.24(e), its eigenvalues $\sqrt{𝜆_{𝑗}}$ are nonnegative. So $𝑅$ is positive by Theorem 8.3.5. By Proposition 8.2.24(c),

𝑅^{2} = (𝑓^{2}) (𝑇) = 𝜆_{1} 𝑃_{1} + \dots + 𝜆_{𝑘} 𝑃_{𝑘} = 𝑇,

because $𝑓^{2} (𝜆_{𝑗}) = 𝜆_{𝑗}$ and $𝑇 = \sum_{𝑗} 𝜆_{𝑗} 𝑃_{𝑗}$ (Theorem 8.2.18(c)). Finally, by Proposition 8.2.24(b), $𝑅 = 𝑝 (𝑇)$ for $𝑝 = \sum_{𝑗} \sqrt{𝜆_{𝑗}} 𝐿_{𝑗}$ , where the Lagrange polynomials $𝐿_{𝑗}$ have real coefficients because the $𝜆_{𝑗}$ are real; and $𝑅$ commutes with every operator that commutes with $𝑇$ .

Intuition (Three ways to see uniqueness). Let

𝑅

be any positive operator with

𝑅^{2} = 𝑇

. We must show that

𝑅

acts on each eigenspace

𝐸_{𝜆}

𝑇

\sqrt{𝜆}

. The three proofs below look at different objects. The first looks at the eigenspaces of

𝑇

𝑅

commutes with

𝑇

, so it preserves them, and on each one it is a positive operator whose square is a scalar. The second looks at the eigenspaces of

𝑅

: squaring the spectral decomposition of

𝑅

gives a spectral decomposition of

𝑇

, and such decompositions are unique. The third compares two square roots

𝑅

and

𝑅^{'}

directly and shows that their difference has no nonzero eigenvalue. In the first two proofs the nonnegativity of the eigenvalues of

𝑅

is used exactly once, to rule out

- \sqrt{𝜆}

; in the third, positivity enters through the signs of

⟨ 𝑅 𝐯, 𝐯 ⟩

and

⟨ 𝑅^{'} 𝐯, 𝐯 ⟩

Proof (via the eigenspaces of $𝑇$ ), part 2 of 2: uniqueness.standard

Uses. Proposition 8.2.20 (Spectral projections are polynomials in $𝑇$ ), Theorem 8.3.5 (Characterizations of positive operators), Theorem 8.2.18 (Spectral decomposition), Definition 6.6.26 (Self-adjoint operator)

Techniques. Restrict to a subspace (Technique 4.2.5), Test an identity on eigenvectors (Technique 7.4.25)

Hint 1.

𝑅

commutes with

𝑅^{2} = 𝑇

. What does Proposition 8.2.20(a) say about

𝑅

and the eigenspaces of

𝑇

Hint 2. Let

𝑅_{𝜆}

be the restriction of

𝑅

𝐸_{𝜆}

. Check that

𝑅_{𝜆}

is a positive operator on

𝐸_{𝜆}

and that

𝑅_{𝜆}^{2} = 𝜆 {id}_{𝐸_{𝜆}}

Hint 3. Apply Theorem 8.3.5 to

𝑅_{𝜆}

: its eigenvalues

𝜇

satisfy

𝜇^{2} = 𝜆

and

𝜇 \geq 0

Sketch.

𝑅 𝑇 = 𝑅^{3} = 𝑇 𝑅

, so

𝑅

maps each

𝐸_{𝜆}

into itself. The restriction

𝑅_{𝜆}

is self-adjoint with

⟨ 𝑅_{𝜆} 𝐯, 𝐯 ⟩ \geq 0

, so

𝐸_{𝜆}

has an orthonormal basis of eigenvectors of

𝑅_{𝜆}

with eigenvalues

𝜇 \geq 0

, and

𝜇^{2} 𝐮 =

𝑅^{2} 𝐮 =

𝑇 𝐮 =

𝜆 𝐮

gives

𝜇 = \sqrt{𝜆}

. So

𝑅 = \sqrt{𝜆} id

on every

𝐸_{𝜆}

, and

𝑉 = \oplus 𝐸_{𝜆}

gives

𝑅 = \sum \sqrt{𝜆_{𝑗}} 𝑃_{𝑗}

Let $𝑅 \in L (𝑉)$ be positive with $𝑅^{2} = 𝑇$ . Then $𝑅 𝑇 = 𝑅^{3} = 𝑇 𝑅$ , so by Proposition 8.2.20(a), $𝑅$ maps each eigenspace $𝐸_{𝜆_{𝑗}}$ of $𝑇$ into itself. Fix $𝑗$ , write $𝜆 = 𝜆_{𝑗}$ , and let $𝑅_{𝜆} \in L (𝐸_{𝜆})$ be the restriction of $𝑅$ to $𝐸_{𝜆}$ , which is an inner product space with the inner product of $𝑉$ . For $𝐯, 𝐰 \in 𝐸_{𝜆}$ ,

⟨ 𝑅_{𝜆} 𝐯, 𝐰 ⟩ = ⟨ 𝑅 𝐯, 𝐰 ⟩ = ⟨ 𝐯, 𝑅 𝐰 ⟩ = ⟨ 𝐯, 𝑅_{𝜆} 𝐰 ⟩ and ⟨ 𝑅_{𝜆} 𝐯, 𝐯 ⟩ = ⟨ 𝑅 𝐯, 𝐯 ⟩ \geq 0,

so $𝑅_{𝜆}$ is positive. By Theorem 8.3.5(III), applied on $𝐸_{𝜆}$ , there is an orthonormal basis $𝐮_{1}, \dots, 𝐮_{𝑚}$ of $𝐸_{𝜆}$ with $𝑅 𝐮_{𝑖} = 𝜇_{𝑖} 𝐮_{𝑖}$ and $𝜇_{𝑖} \geq 0$ . Then

𝜆 𝐮_{𝑖} = 𝑇 𝐮_{𝑖} = 𝑅^{2} 𝐮_{𝑖} = 𝜇_{𝑖}^{2} 𝐮_{𝑖},

so $𝜇_{𝑖}^{2} = 𝜆$ , and since $𝜇_{𝑖} \geq 0$ , $𝜇_{𝑖} = \sqrt{𝜆}$ . Thus $𝑅$ and $\sqrt{𝜆} id$ agree on a basis of $𝐸_{𝜆}$ , so $𝑅 𝐯 = \sqrt{𝜆_{𝑗}} 𝐯$ for every $𝐯 \in 𝐸_{𝜆_{𝑗}}$ , for every $𝑗$ . Finally, every $𝐯 \in 𝑉$ is $𝐯 = \sum_{𝑗} 𝑃_{𝑗} 𝐯$ with $𝑃_{𝑗} 𝐯 \in 𝐸_{𝜆_{𝑗}}$ (Theorem 8.2.18), so

𝑅 𝐯 = \sum_{𝑗} 𝑅 𝑃_{𝑗} 𝐯 = \sum_{𝑗} \sqrt{𝜆_{𝑗}} 𝑃_{𝑗} 𝐯,

that is, $𝑅 = \sum_{𝑗} \sqrt{𝜆_{𝑗}} 𝑃_{𝑗}$ . □

The step $𝜇_{𝑖}^{2} = 𝜆 \Rightarrow 𝜇_{𝑖} = \sqrt{𝜆}$ is where positivity of $𝑅$ entered. A self-adjoint $𝑅$ with $𝑅^{2} = 𝑇$ also preserves each $𝐸_{𝜆}$ , but on $𝐸_{𝜆}$ it only has to satisfy $𝑅_{𝜆}^{2} = 𝜆 id$ , and when $𝜆 > 0$ and $dim 𝐸_{𝜆} \geq 2$ there are infinitely many such $𝑅_{𝜆}$ (Exercise 8.3.38).

Proof (via the spectral decomposition of $𝑅$ ), part 2 of 2: uniqueness.standard

Uses. Theorem 8.2.18 (Spectral decomposition), Proposition 8.2.19 (Uniqueness of the spectral decomposition), Theorem 8.3.5 (Characterizations of positive operators)

Techniques. Uniqueness via two candidates (Technique 1.1.8)

Hint 1. Write

𝑅 =

𝜇_{1} 𝑄_{1} +

\dots +

𝜇_{𝑚} 𝑄_{𝑚}

, its own spectral decomposition, and square it.

Hint 2.

𝑅^{2} = \sum_{𝑖} 𝜇_{𝑖}^{2} 𝑄_{𝑖}

. Why are the numbers

𝜇_{𝑖}^{2}

distinct? Now use Proposition 8.2.19.

Sketch.

𝑅 = \sum_{𝑖} 𝜇_{𝑖} 𝑄_{𝑖}

with distinct

𝜇_{𝑖} \geq 0

, so

𝑇 =

𝑅^{2} =

\sum_{𝑖} 𝜇_{𝑖}^{2} 𝑄_{𝑖}

, and the

𝜇_{𝑖}^{2}

are distinct because squaring is injective on

[0, \infty)

. By Proposition 8.2.19, the

𝜇_{𝑖}^{2}

are the eigenvalues of

𝑇

and

𝑄_{𝑖} = {Pr}_{𝐸_{𝜇_{𝑖}^{2}}}

. So

𝑅 = \sum \sqrt{𝜆_{𝑗}} 𝑃_{𝑗}

Let $𝑅 \in L (𝑉)$ be positive with $𝑅^{2} = 𝑇$ . By Theorem 8.2.18, $𝑅 =$ $𝜇_{1} 𝑄_{1} +$ $\dots +$ $𝜇_{𝑚} 𝑄_{𝑚}$ , where $𝜇_{1}, \dots, 𝜇_{𝑚}$ are the distinct eigenvalues of $𝑅$ , all nonnegative by Theorem 8.3.5, and $𝑄_{𝑖}$ is the orthogonal projection onto the eigenspace of $𝑅$ for $𝜇_{𝑖}$ . The $𝑄_{𝑖}$ are nonzero, $𝑄_{𝑖} 𝑄_{𝑙} = 0$ for $𝑖 \neq 𝑙$ , $𝑄_{𝑖}^{2} = 𝑄_{𝑖}$ , and $\sum_{𝑖} 𝑄_{𝑖} = {id}_{𝑉}$ . Expanding,

𝑇 = 𝑅^{2} = \sum_{𝑖, 𝑙} 𝜇_{𝑖} 𝜇_{𝑙} 𝑄_{𝑖} 𝑄_{𝑙} = 𝜇_{1}^{2} 𝑄_{1} + \dots + 𝜇_{𝑚}^{2} 𝑄_{𝑚} .

The numbers $𝜇_{𝑖}^{2}$ are distinct, because $𝜇_{𝑖} \geq 0$ and $𝑡 \mapsto 𝑡^{2}$ is injective on $[0, \infty)$ . So Proposition 8.2.19 applies to this expression of $𝑇$ : the eigenvalues of $𝑇$ are exactly $𝜇_{1}^{2}, \dots, 𝜇_{𝑚}^{2}$ , and $𝑄_{𝑖}$ is the orthogonal projection onto $𝐸_{𝜇_{𝑖}^{2}}$ , the eigenspace of $𝑇$ . Relabelling so that $𝜆_{𝑖} = 𝜇_{𝑖}^{2}$ (and $𝑘 = 𝑚$ ), we get $𝜇_{𝑖} = \sqrt{𝜆_{𝑖}}$ and $𝑄_{𝑖} = 𝑃_{𝑖}$ , so $𝑅 = \sum_{𝑖} \sqrt{𝜆_{𝑖}} 𝑃_{𝑖}$ . □

Proof (by comparing two square roots), part 2 of 2: uniqueness.standard

Uses. Lemma 8.3.8 (Where the form vanishes), Proposition 8.1.1 (Eigenvalues and eigenvectors of self-adjoint operators), Theorem 8.2.12 (Real spectral theorem), Corollary 8.2.7 (Hermitian matrices), Proposition 6.6.18 (Properties of the adjoint)

Techniques. A sum of nonnegative terms (Technique 6.1.9), Extremal choice and minimal counterexamples (Technique 1.5.9)

Hint 1. Let

𝑅

and

𝑅^{'}

be positive square roots of

𝑇

and

𝐷 = 𝑅 - 𝑅^{'}

𝐷

is self-adjoint, so it suffices to show that every eigenvalue of

𝐷

0

Hint 2. Check the identity

𝑅^{2} -

𝑅^{' 2} =

𝑅 𝐷 +

𝐷 𝑅^{'}

. Apply it to an eigenvector

𝐯

𝐷

𝐷 𝐯 = 𝜇 𝐯

, and take the inner product with

𝐯

Hint 3. You get

0 =

𝜇 (⟨ 𝑅 𝐯, 𝐯 ⟩ + ⟨ 𝑅^{'} 𝐯, 𝐯 ⟩)

. If

𝜇 \neq 0

, use Lemma 8.3.8.

Sketch. Let

𝐷 = 𝑅 - 𝑅^{'}

, self-adjoint, with

𝐷 𝐯 = 𝜇 𝐯

𝐯 \neq 𝟎

𝜇 \in ℝ

. Since

0 =

𝑅^{2} -

𝑅^{' 2} =

𝑅 𝐷 +

𝐷 𝑅^{'}

, we get

0 =

⟨ (𝑅 𝐷 + 𝐷 𝑅^{'}) 𝐯, 𝐯 ⟩ =

𝜇 (⟨ 𝑅 𝐯, 𝐯 ⟩ + ⟨ 𝑅^{'} 𝐯, 𝐯 ⟩)

. If

𝜇 \neq 0

, both nonnegative terms vanish, so

𝑅 𝐯 = 𝑅^{'} 𝐯 = 𝟎

by Lemma 8.3.8, and

𝜇 𝐯 = 𝐷 𝐯 = 𝟎

, a contradiction. So all eigenvalues of

𝐷

are

0

, and a self-adjoint operator with only the eigenvalue

0

0

Let $𝑅, 𝑅^{'} \in L (𝑉)$ be positive with $𝑅^{2} = 𝑅^{' 2} = 𝑇$ , and let $𝐷 = 𝑅 - 𝑅^{'}$ . Then $𝐷$ is self-adjoint (Proposition 6.6.18), and

𝑅 𝐷 + 𝐷 𝑅^{'} = 𝑅^{2} - 𝑅 𝑅^{'} + 𝑅 𝑅^{'} - 𝑅^{' 2} = 𝑅^{2} - 𝑅^{' 2} = 0 .

Let $𝜇$ be an eigenvalue of $𝐷$ with eigenvector $𝐯 \neq 𝟎$ ; $𝜇$ is real by Proposition 8.1.1. Using $𝐷^{*} = 𝐷$ ,

0 = ⟨ (𝑅 𝐷 + 𝐷 𝑅^{'}) 𝐯, 𝐯 ⟩ = ⟨ 𝑅 (𝜇 𝐯), 𝐯 ⟩ + ⟨ 𝑅^{'} 𝐯, 𝐷 𝐯 ⟩ = 𝜇 ⟨ 𝑅 𝐯, 𝐯 ⟩ + 𝜇 ⟨ 𝑅^{'} 𝐯, 𝐯 ⟩,

and since $𝜇$ is real, $0 =$ $𝜇 (⟨ 𝑅 𝐯, 𝐯 ⟩ + ⟨ 𝑅^{'} 𝐯, 𝐯 ⟩)$ . Suppose $𝜇 \neq 0$ . Then the two nonnegative numbers $⟨ 𝑅 𝐯, 𝐯 ⟩$ and $⟨ 𝑅^{'} 𝐯, 𝐯 ⟩$ add up to $0$ , so both are $0$ , and by Lemma 8.3.8, $𝑅 𝐯 = 𝑅^{'} 𝐯 = 𝟎$ . Then $𝜇 𝐯 = 𝐷 𝐯 = 𝟎$ with $𝐯 \neq 𝟎$ , so $𝜇 = 0$ , a contradiction. Hence every eigenvalue of $𝐷$ is $0$ . By Theorem 8.2.12 or Corollary 8.2.7, $𝐷$ has an orthonormal eigenbasis, in which its matrix is diagonal with the eigenvalues on the diagonal, that is, zero. So $𝐷 = 0$ and $𝑅 = 𝑅^{'}$ . Since $\sqrt{𝑇}$ is a positive square root of $𝑇$ , every positive square root of $𝑇$ equals $\sqrt{𝑇}$ . □

The third proof never mentions the eigenspaces of $𝑇$ and uses only Lemma 8.3.8 and the identity $𝑅^{2} -$ $𝑅^{' 2} =$ $𝑅 𝐷 +$ $𝐷 𝑅^{'}$ . That makes it the one that adapts to inequalities: the same computation shows that $𝐴 \mapsto \sqrt{𝐴}$ preserves the order between positive operators (Exercise 8.3.40).

Example 8.3.11 (A square root as a polynomial). For $𝐴 =$ $𝐼_{3} +$ $𝐽_{3} =$ $(\begin{matrix} 2 & 1 & 1 \\ 1 & 2 & 1 \\ 1 & 1 & 2 \end{matrix})$ of Example 8.2.21, the eigenvalues are $4$ and $1$ , with $𝑃_{4} = (𝐴 - 𝐼_{3}) / 3$ and $𝑃_{1} =$ $(𝐴 - 4 𝐼_{3}) / (- 3)$ . So

\sqrt{𝐴} = 2 𝑃_{4} + 1 𝑃_{1} = \frac{2 (𝐴 - 𝐼_{3}) - (𝐴 - 4 𝐼_{3})}{3} = \frac{𝐴 + 2 𝐼_{3}}{3} = (\begin{matrix} 4 / 3 & 1 / 3 & 1 / 3 \\ 1 / 3 & 4 / 3 & 1 / 3 \\ 1 / 3 & 1 / 3 & 4 / 3 \end{matrix}) .

The polynomial is $𝑝 (𝑥) = (𝑥 + 2) / 3$ , the line through $(1, 1)$ and $(4, 2)$ : it agrees with $\sqrt{𝑥}$ at the two eigenvalues, which is all that matters. Check: ${(𝐼_{3} + 𝐽_{3} / 3)}^{2} =$ $𝐼_{3} +$ $2 𝐽_{3} / 3 +$ $𝐽_{3}^{2} / 9 =$ $𝐼_{3} +$ $𝐽_{3}$ , since $𝐽_{3}^{2} = 3 𝐽_{3}$ .

Exercise 8.3.12.routine Let $𝐴 = (\begin{matrix} 5 & - 4 \\ - 4 & 5 \end{matrix})$ and $𝐵 = (\begin{matrix} 1 & - 2 \\ - 2 & 1 \end{matrix})$ .

(a)Compute $\sqrt{𝐴}$ , and check that $𝐵^{2} = 𝐴$ .
(b)Compute $\sqrt{𝐴^{2}}$ and $\sqrt{𝐵^{2}}$ . Which of $\sqrt{𝐴^{2}} = 𝐴$ and $\sqrt{𝐵^{2}} = 𝐵$ holds, and why?

Solution.

Uses. Theorem 8.3.9 (The positive square root), Proposition 8.2.20 (Spectral projections are polynomials in $𝑇$ )

Techniques. Apply a function to the eigenvalues (Technique 8.2.25)

Hint 1. The eigenvalues of

𝐴

are

9

and

1

, with eigenvectors

(1, - 1)

and

(1, 1)

Hint 2. For (b),

𝐴

is a positive square root of

𝐴^{2}

, while

𝐵

has eigenvalues

3

and

- 1

Sketch. (a)

𝑃_{9} = \frac{1}{2} (\begin{matrix} 1 & - 1 \\ - 1 & 1 \end{matrix})

𝑃_{1} = \frac{1}{2} (\begin{matrix} 1 & 1 \\ 1 & 1 \end{matrix})

\sqrt{𝐴} =

3 𝑃_{9} +

𝑃_{1} =

(\begin{matrix} 2 & - 1 \\ - 1 & 2 \end{matrix})

;

𝐵^{2} = (\begin{matrix} 5 & - 4 \\ - 4 & 5 \end{matrix})

. (b)

𝐴

is positive and squares to

𝐴^{2}

, so

\sqrt{𝐴^{2}} = 𝐴

by uniqueness.

𝐵^{2} = 𝐴

, so

\sqrt{𝐵^{2}} =

\sqrt{𝐴} =

(\begin{matrix} 2 & - 1 \\ - 1 & 2 \end{matrix}) \neq

𝐵

;

𝐵

is not positive.

(a) $𝑝_{𝐴} =$ ${(𝑥 - 5)}^{2} -$ $16 =$ $(𝑥 - 9) (𝑥 - 1)$ . By Proposition 8.2.20,

𝑃_{9} = \frac{𝐴 - 𝐼_{2}}{8} = \frac{1}{2} (\begin{matrix} 1 & - 1 \\ - 1 & 1 \end{matrix}), 𝑃_{1} = \frac{𝐴 - 9 𝐼_{2}}{- 8} = \frac{1}{2} (\begin{matrix} 1 & 1 \\ 1 & 1 \end{matrix}),

so by Theorem 8.3.9,

\sqrt{𝐴} = 3 𝑃_{9} + 1 𝑃_{1} = \frac{1}{2} (\begin{matrix} 3 + 1 & - 3 + 1 \\ - 3 + 1 & 3 + 1 \end{matrix}) = (\begin{matrix} 2 & - 1 \\ - 1 & 2 \end{matrix}) .

Check: ${(\begin{matrix} 2 & - 1 \\ - 1 & 2 \end{matrix})}^{2} =$ $(\begin{matrix} 4 + 1 & - 2 - 2 \\ - 2 - 2 & 1 + 4 \end{matrix}) =$ $𝐴$ . Also $𝐵^{2} =$ $(\begin{matrix} 1 + 4 & - 2 - 2 \\ - 2 - 2 & 4 + 1 \end{matrix}) =$ $𝐴$ .

(b) $𝐴$ is positive (its eigenvalues are $9$ and $1$ ) and $𝐴^{2} = 𝐴^{2}$ , so $𝐴$ is a positive square root of $𝐴^{2}$ , and by uniqueness $\sqrt{𝐴^{2}} = 𝐴$ . On the other hand $𝐵^{2} = 𝐴$ , so $\sqrt{𝐵^{2}} =$ $\sqrt{𝐴} =$ $(\begin{matrix} 2 & - 1 \\ - 1 & 2 \end{matrix}) \neq$ $𝐵$ . The difference is that $𝐵$ is not positive: $𝐵 = 3 𝑃_{9} - 𝑃_{1}$ has the eigenvalue $- 1$ . In general $\sqrt{𝑆^{2}} = 𝑆$ for positive $𝑆$ , and for a self-adjoint $𝑆$ the operator $\sqrt{𝑆^{2}}$ replaces each eigenvalue of $𝑆$ by its absolute value (Exercise 8.3.37).

Remark 8.3.13 (The absolute value of an operator). Let $𝑊$ be a finite-dimensional inner product space and $𝑆 \in L (𝑉, 𝑊)$ . Then $𝑆^{*} 𝑆$ is positive (Example 8.3.2(b)), so it has a positive square root

| 𝑆 | = \sqrt{𝑆^{*} 𝑆} \in L (𝑉) .

It satisfies ${‖ | 𝑆 | 𝐯 ‖}^{2} =$ $⟨ {| 𝑆 |}^{2} 𝐯, 𝐯 ⟩ =$ $⟨ 𝑆^{*} 𝑆 𝐯, 𝐯 ⟩ =$ ${‖ 𝑆 𝐯 ‖}^{2}$ , so $| 𝑆 |$ moves every vector exactly as far as $𝑆$ does. For a complex number, $| 𝑧 | = \sqrt{𝑧 𝑧}$ is the size of $𝑧$ with its direction forgotten, and $| 𝑆 |$ plays the same role. In Section 9.1 the eigenvalues of $| 𝑆 |$ become the singular values of $𝑆$ .

8.3.4Positive definite operators

Definition 8.3.14 (Positive definite). An operator $𝑇 \in L (𝑉)$ is positive definite if $𝑇$ is self-adjoint and

⟨ 𝑇 𝐯, 𝐯 ⟩ > 0 for every nonzero 𝐯 \in 𝑉 .

A matrix $𝐴 \in 𝔽^{𝑛 \times 𝑛}$ is positive definite if $𝐴^{*} = 𝐴$ and $𝐱^{*} 𝐴 𝐱 > 0$ for every nonzero $𝐱 \in 𝔽^{𝑛}$ .

The lecture notes define a positive definite matrix as a matrix whose eigenvalues are all positive. For Hermitian matrices this is the same notion, by (II) of the next proposition. Every positive definite operator is positive, and by Lemma 8.3.8 a positive operator is positive definite exactly when its kernel is ${𝟎}$ . So positive definite operators are to positive operators what positive numbers are to nonnegative ones.

Proposition 8.3.15 (Characterizations of positive definite operators). Let $𝑇 \in L (𝑉)$ . The following are equivalent:

(I) $𝑇$ is positive definite;
(II) $𝑇$ is self-adjoint and every eigenvalue of $𝑇$ is positive;
(III) $𝑇$ is positive and invertible;
(IV) $𝑇 = 𝑆^{*} 𝑆$ for some injective $𝑆 \in L (𝑉, 𝑊)$ , where $𝑊$ is a finite-dimensional inner product space;
(V)for some orthonormal basis $𝒪︀$ of $𝑉$ , ${[𝑇]}_{𝒪︀}$ is the Gram matrix of a linearly independent list.

In matrix form: $𝐴 \in 𝔽^{𝑛 \times 𝑛}$ is positive definite if and only if $𝐴 = 𝐵^{*} 𝐵$ for some $𝐵 \in 𝔽^{𝑚 \times 𝑛}$ with $rank 𝐵 = 𝑛$ , if and only if $𝐴$ is the Gram matrix of a linearly independent list, if and only if $𝐴$ is positive and $det 𝐴 \neq 0$ .

Proof.standard

Uses. Theorem 8.3.5 (Characterizations of positive operators), Theorem 8.3.9 (The positive square root), Theorem 7.2.9 (Eigenvalues and invertibility), Proposition 6.3.30 (Gram matrices detect independence), Theorem 3.5.12 (First Fundamental Theorem, completed), Theorem 5.4.7 (Invertibility and the determinant), Corollary 4.2.18 (Injective if and only if surjective)

Techniques. Proving TFAE (Technique 2.7.1), Gram matrices (Technique 6.3.31)

Hint 1. Follow the proof of Theorem 8.3.5 with strict inequalities: (I)

\Rightarrow

(II)

\Rightarrow

(III)

\Rightarrow

(IV)

\Rightarrow

(V)

\Rightarrow

(I).

Hint 2. For (III)

\Rightarrow

(IV), take

𝑆 = \sqrt{𝑇}

; why is it injective? For (IV)

\Rightarrow

(V), injective maps send independent lists to independent lists.

Hint 3. For (V)

\Rightarrow

(I), the computation

⟨ 𝑇 𝐯, 𝐯 ⟩ =

{‖ \sum_{𝑗} 𝑐_{𝑗} 𝐰_{𝑗} ‖}^{2}

from Theorem 8.3.5 is still available. When is

\sum_{𝑗} 𝑐_{𝑗} 𝐰_{𝑗} = 𝟎

Sketch. (I)

\Rightarrow

(II):

𝜆 {‖ 𝐯 ‖}^{2} =

⟨ 𝑇 𝐯, 𝐯 ⟩ >

0

. (II)

\Rightarrow

(III): positive by Theorem 8.3.5, and

0

is not an eigenvalue. (III)

\Rightarrow

(IV):

𝑆 = \sqrt{𝑇}

;

𝑆 𝐯 = 𝟎

gives

𝑇 𝐯 = 𝟎

, so

𝐯 = 𝟎

. (IV)

\Rightarrow

(V):

{[𝑇]}_{𝒪︀}

is the Gram matrix of

𝑆 𝐞_{1}, \dots, 𝑆 𝐞_{𝑛}

, which is independent as

𝑆

is injective. (V)

\Rightarrow

(I):

𝑇

is positive, and

⟨ 𝑇 𝐯, 𝐯 ⟩ =

{‖ \sum 𝑐_{𝑗} 𝐰_{𝑗} ‖}^{2} >

0

for

𝐜 \neq 𝟎

by independence.

We prove (I) $\Rightarrow$ (II) $\Rightarrow$ (III) $\Rightarrow$ (IV) $\Rightarrow$ (V) $\Rightarrow$ (I).

(I) $\Rightarrow$ (II). If $𝑇 𝐯 = 𝜆 𝐯$ with $𝐯 \neq 𝟎$ , then $𝜆 {‖ 𝐯 ‖}^{2} =$ $⟨ 𝑇 𝐯, 𝐯 ⟩ >$ $0$ , so $𝜆 > 0$ .

(II) $\Rightarrow$ (III). $𝑇$ is positive by Theorem 8.3.5. Since $0$ is not an eigenvalue of $𝑇$ , $𝑇$ is invertible by Theorem 7.2.9.

(III) $\Rightarrow$ (IV). Let $𝑊 = 𝑉$ and $𝑆 = \sqrt{𝑇}$ (Theorem 8.3.9). Then $𝑆^{*} 𝑆 = 𝑆^{2} = 𝑇$ . If $𝑆 𝐯 = 𝟎$ , then $𝑇 𝐯 = 𝑆 (𝑆 𝐯) = 𝟎$ , so $𝐯 = 𝟎$ since $𝑇$ is invertible. Hence $𝑆$ is injective.

(IV) $\Rightarrow$ (V). Let $𝒪︀ = 𝐞_{1}, \dots, 𝐞_{𝑛}$ be an orthonormal basis. By the proof of Theorem 8.3.5 ((VI) $\Rightarrow$ (VII)), ${[𝑇]}_{𝒪︀}$ is the Gram matrix of $𝑆 𝐞_{1}, \dots, 𝑆 𝐞_{𝑛}$ . This list is independent: if $\sum_{𝑖} 𝑐_{𝑖} 𝑆 𝐞_{𝑖} = 𝟎$ , then $𝑆 (\sum_{𝑖} 𝑐_{𝑖} 𝐞_{𝑖}) =$ $𝟎$ , so $\sum_{𝑖} 𝑐_{𝑖} 𝐞_{𝑖} = 𝟎$ because $𝑆$ is injective, and every $𝑐_{𝑖} = 0$ .

(V) $\Rightarrow$ (I). $𝑇$ is positive by Theorem 8.3.5 ((VII) $\Rightarrow$ (I)), in particular self-adjoint. Let ${[𝑇]}_{𝒪︀}$ be the Gram matrix of the independent list $𝐰_{1}, \dots, 𝐰_{𝑛}$ , and let $𝐯 \neq 𝟎$ have coordinates $𝐜 = {[𝐯]}_{𝒪︀} \neq 𝟎$ . As in the proof of Theorem 8.3.5, $⟨ 𝑇 𝐯, 𝐯 ⟩ =$ ${‖ 𝑐_{1} 𝐰_{1} + \dots + 𝑐_{𝑛} 𝐰_{𝑛} ‖}^{2}$ , and the vector inside the norm is nonzero because the list is independent. So $⟨ 𝑇 𝐯, 𝐯 ⟩ > 0$ .

For the matrix form, apply this to $𝑇_{𝐴}$ as in Theorem 8.3.5: $𝑇_{𝐵}$ is injective if and only if $rank 𝐵 = 𝑛$ (Theorem 3.5.12), and the Gram matrix statement is (V) with the standard basis. Finally, if $𝐴$ is positive then $𝐴$ is invertible exactly when $det 𝐴 \neq 0$ (Theorem 5.4.7), which is (III). □

Corollary 8.3.16 (Inner products on $𝔽^{𝑛}$ ). The inner products on $𝔽^{𝑛}$ are exactly the functions

{⟨ 𝐱, 𝐲 ⟩}_{𝐴} = 𝐲^{*} 𝐴 𝐱

with $𝐴 \in 𝔽^{𝑛 \times 𝑛}$ positive definite. For such an inner product, $𝐴$ is the Gram matrix of the standard basis: $𝑎_{𝑖 𝑗} =$ ${⟨ 𝐞_{𝑗}, 𝐞_{𝑖} ⟩}_{𝐴}$ .

Proof.routine

Uses. Proposition 6.1.22 (Inner products on $𝔽^{𝑛}$ ), Definition 8.3.14 (Positive definite)

Hint. Compare Definition 8.3.14 with the condition in Proposition 6.1.22(b).

Sketch. Proposition 6.1.22(b) says that

𝐲^{*} 𝑀 𝐱

is an inner product if and only if

𝑀^{*} = 𝑀

and

𝐱^{*} 𝑀 𝐱 > 0

for

𝐱 \neq 𝟎

, that is,

𝑀

is positive definite. Part (a) says every inner product has this form with

𝑚_{𝑖 𝑗} =

⟨ 𝐞_{𝑗}, 𝐞_{𝑖} ⟩

By Proposition 6.1.22(b), the formula

{⟨ 𝐱, 𝐲 ⟩}_{𝑀} = 𝐲^{*} 𝑀 𝐱

defines an inner product if and only if

𝑀^{*} = 𝑀

and

𝐱^{*} 𝑀 𝐱 > 0

for every nonzero

𝐱

, that is, if and only if

𝑀

is positive definite. By Proposition 6.1.22(a), every inner product on

𝔽^{𝑛}

is of this form, with

𝑚_{𝑖 𝑗} =

⟨ 𝐞_{𝑗}, 𝐞_{𝑖} ⟩

. □

So positive definite matrices and inner products on $𝔽^{𝑛}$ are the same objects, and Example 6.1.23 was a positive definite matrix in disguise. Questions about one inner product compared with another become questions about a positive definite matrix compared with $𝐼_{𝑛}$ .

Example 8.3.17 (Positive, and positive definite). The matrix

𝐴

of Example 8.3.6 is positive definite: its eigenvalues are positive, and equivalently the difference matrix

𝐵

has independent columns (it has rank

3

). The matrix

𝐶 = (\begin{matrix} 1 & 𝑖 \\ - 𝑖 & 1 \end{matrix})

of Exercise 8.3.7 is positive but not positive definite:

𝐶 = 𝐵^{*} 𝐵

only for matrices

𝐵

with dependent columns, such as

𝐵 = (\begin{matrix} 1 & 𝑖 \end{matrix})

, whose columns

1, 𝑖

are dependent in

ℂ^{1}

. Accordingly

det 𝐶 = 0

and

𝐶 (- 𝑖, 1) = 𝟎

, where

𝐱 = (- 𝑖, 1)

has

𝐱^{*} 𝐶 𝐱 =

{| - 𝑖 + 𝑖 |}^{2} =

0

If $𝑇$ is positive definite, so are $\sqrt{𝑇}$ and $𝑇^{- 1}$ : their eigenvalues $\sqrt{𝜆_{𝑗}}$ and $1 / 𝜆_{𝑗}$ are positive (Proposition 8.2.24(e), with $𝑇^{- 1} =$ $\sum_{𝑗} 𝜆_{𝑗}^{- 1} 𝑃_{𝑗}$ as in Section 8.2.5). Applying Proposition 8.2.24(c) to $𝑓 (𝜆) = 1 / \sqrt{𝜆}$ shows that $𝑓 (𝑇) \sqrt{𝑇} = {id}_{𝑉}$ and ${𝑓 (𝑇)}^{2} = 𝑇^{- 1}$ ; since $𝑓 (𝑇)$ is positive, uniqueness of square roots gives ${(\sqrt{𝑇})}^{- 1} =$ $𝑓 (𝑇) =$ $\sqrt{𝑇^{- 1}}$ . We write $𝑇^{- 1 / 2}$ for this operator.

Exercise 8.3.18.routine For which real numbers

𝑡

𝐴_{𝑡} = (\begin{matrix} 1 & 𝑡 \\ 𝑡 & 4 \end{matrix})

positive? For which is it positive definite?

Solution.

Uses. Theorem 8.3.5 (Characterizations of positive operators), Proposition 8.3.15 (Characterizations of positive definite operators)

Hint 1. The eigenvalues of

𝐴_{𝑡}

are real. Their sum is

5

and their product is

4 - 𝑡^{2}

Hint 2. Alternatively, complete the square in

𝑥^{2} + 2 𝑡 𝑥 𝑦 + 4 𝑦^{2}

Sketch. Both eigenvalues are

\geq 0

if and only if their sum

5

and product

4 - 𝑡^{2}

are

\geq 0

: positive for

| 𝑡 | \leq 2

. Positive definite iff both

> 0

, iff

| 𝑡 | < 2

. Or:

𝑥^{2} +

2 𝑡 𝑥 𝑦 +

4 𝑦^{2} =

{(𝑥 + 𝑡 𝑦)}^{2} +

(4 - 𝑡^{2}) 𝑦^{2}

$𝐴_{𝑡}$ is symmetric, so its eigenvalues $𝜆_{1}, 𝜆_{2}$ are real, with $𝜆_{1} + 𝜆_{2} = tr 𝐴_{𝑡} = 5$ and $𝜆_{1} 𝜆_{2} =$ $det 𝐴_{𝑡} =$ $4 -$ $𝑡^{2}$ . Two real numbers with a positive sum are both nonnegative exactly when their product is nonnegative, and both positive exactly when their product is positive. So by Theorem 8.3.5 and Proposition 8.3.15, $𝐴_{𝑡}$ is positive if and only if $| 𝑡 | \leq 2$ , and positive definite if and only if $| 𝑡 | < 2$ .

The same answer comes from completing the square:

(𝑥, 𝑦) 𝐴_{𝑡} (\begin{matrix} 𝑥 \\ 𝑦 \end{matrix}) = 𝑥^{2} + 2 𝑡 𝑥 𝑦 + 4 𝑦^{2} = {(𝑥 + 𝑡 𝑦)}^{2} + (4 - 𝑡^{2}) 𝑦^{2} .

If $| 𝑡 | \leq 2$ this is $\geq 0$ , and if $| 𝑡 | < 2$ it is $0$ only when $𝑦 = 0$ and $𝑥 + 𝑡 𝑦 = 0$ , that is, at $(0, 0)$ . If $| 𝑡 | > 2$ , then $(𝑥, 𝑦) = (- 𝑡, 1)$ gives $4 - 𝑡^{2} < 0$ . At $𝑡 = \pm 2$ , the vector $(- 𝑡, 1) \neq 𝟎$ gives $0$ .

Quadratic forms

For a real symmetric matrix $𝐴$ , the function $𝑞 (𝐱) = 𝐱^{𝖳} 𝐴 𝐱$ on $ℝ^{𝑛}$ is the quadratic form of $𝐴$ (Section 8.2.6). By definition, $𝐴$ is positive definite exactly when $𝑞 (𝐱) > 0$ for every nonzero $𝐱$ , and this is how positive definite matrices usually enter: through quadratic forms. By Corollary 8.2.28, in the coordinates $𝐲 = 𝑄^{𝖳} 𝐱$ along an orthonormal eigenbasis $𝐪_{1}, \dots, 𝐪_{𝑛}$ ,

𝑞 (𝐱) = 𝜆_{1} 𝑦_{1}^{2} + \dots + 𝜆_{𝑛} 𝑦_{𝑛}^{2} .

If $𝐴$ is positive definite, the level set $𝑞 (𝐱) = 1$ is therefore an ellipsoid with semi-axes $1 / \sqrt{𝜆_{𝑖}}$ along the $𝐪_{𝑖}$ (in $ℝ^{2}$ , the ellipses of Remark 8.2.30). If $𝐴 \neq 0$ is positive but some $𝜆_{𝑖} = 0$ , the level set is unbounded in the direction of $𝐪_{𝑖}$ , and if some $𝜆_{𝑖} < 0$ , then $𝑞$ takes negative values. For example, the form $𝑞 (𝐱) =$ $2 𝑥_{1}^{2} +$ $2 𝑥_{2}^{2} +$ $2 𝑥_{3}^{2} -$ $2 𝑥_{1} 𝑥_{2} -$ $2 𝑥_{2} 𝑥_{3}$ of Example 8.3.6 is positive definite, and $𝑞 (𝐱) = 1$ is an ellipsoid with semi-axes $1 / \sqrt{2 + \sqrt{2}}$ , $1 / \sqrt{2}$ , and $1 / \sqrt{2 - \sqrt{2}}$ .

This is the linear algebra behind the second-derivative test. At a critical point $𝐱_{0}$ of a smooth $𝑓 : ℝ^{𝑛} \to ℝ$ , $𝑓 (𝐱_{0} + 𝐡) =$ $𝑓 (𝐱_{0}) +$ $\frac{1}{2} 𝐡^{𝖳} 𝐻 𝐡 +$ $(\dots)$ , where $𝐻$ is the symmetric matrix of second partial derivatives and the omitted terms are small compared with ${‖ 𝐡 ‖}^{2}$ ; a positive definite $𝐻$ makes $𝐱_{0}$ a strict local minimum. Section 12.2 studies quadratic forms in general, and Section 12.3 gives further tests for positive definiteness, among them Sylvester’s criterion on the leading minors, together with the min–max description of eigenvalues.

8.3.5Products of positive operators

The product of two nonnegative numbers is nonnegative. The product of two positive operators need not even be self-adjoint: for $𝐴 = (\begin{matrix} 2 & 1 \\ 1 & 1 \end{matrix})$ and the projection $𝐵 = (\begin{matrix} 1 & 0 \\ 0 & 0 \end{matrix})$ , the product $𝐴 𝐵 = (\begin{matrix} 2 & 0 \\ 1 & 0 \end{matrix})$ is not symmetric. The obstruction is that $𝐴$ and $𝐵$ need not commute. What survives is a statement about eigenvalues, and the proof introduces a move that we will use repeatedly.

Technique 8.3.19 (Sandwich between square roots). Two positive operators $𝐴$ and $𝐵$ do not commute in general, so $𝐴 𝐵$ need not be positive. Split $𝐴 = \sqrt{𝐴} \sqrt{𝐴}$ and move one factor to the other side: the operator $\sqrt{𝐴} 𝐵 \sqrt{𝐴}$ is always positive, since $⟨ \sqrt{𝐴} 𝐵 \sqrt{𝐴} 𝐯, 𝐯 ⟩ =$ $⟨ 𝐵 \sqrt{𝐴} 𝐯, \sqrt{𝐴} 𝐯 ⟩ \geq$ $0$ , and it is close to $𝐴 𝐵$ . It has the same characteristic polynomial (by Proposition 7.3.22, applied to $\sqrt{𝐴}$ and $\sqrt{𝐴} 𝐵$ ), and if $𝐴$ is invertible it is similar to $𝐴 𝐵$ . In the same way, when $𝐵$ is positive definite, the congruence $𝑋 \mapsto 𝐵^{- 1 / 2} 𝑋 𝐵^{- 1 / 2}$ turns the pair $(𝐴, 𝐵)$ into $(𝐶, id)$ with $𝐶 = 𝐵^{- 1 / 2} 𝐴 𝐵^{- 1 / 2}$ positive, and a question about two operators becomes a question about the eigenvalues of one.

Reach for it when a statement involves two positive operators (a product, a comparison $𝐴 ⪯ 𝐵$ , $det (𝐴 + 𝐵)$ , $tr (𝐴 𝐵)$ ) and the case of a single operator is easy by Technique 8.2.25. The canonical example is Proposition 8.3.20.

Used in 4 places and 1 practice exercise

Proposition 8.3.20, Exercise 8.3.34, Exercise 8.3.39, Exercise 8.3.42; 1 practice exercise.

Proposition 8.3.20 (Products of positive operators). Let $𝐴, 𝐵 \in L (𝑉)$ be positive.

(a) $𝐴 𝐵$ is positive if and only if $𝐴 𝐵 = 𝐵 𝐴$ .
(b)Every root of $𝑝_{𝐴 𝐵}$ in $ℂ$ is real and nonnegative. In particular every eigenvalue of $𝐴 𝐵$ is nonnegative.
(c)If $𝐴$ is invertible, then $𝐴 𝐵$ is diagonalizable.

Proof.standard

Uses. Theorem 8.3.9 (The positive square root), Proposition 6.6.18 (Properties of the adjoint), Proposition 7.3.22 ( $𝐴 𝐵$ and $𝐵 𝐴$ ), Definition 7.3.12 (Characteristic polynomial of an operator), Theorem 2.4.2 (The matrix of a composite), Theorem 8.3.5 (Characterizations of positive operators), Theorem 7.2.9 (Eigenvalues and invertibility), Theorem 7.4.2 (Diagonalizable means an eigenbasis), Corollary 7.3.13 (Eigenvalues of an operator)

Techniques. Sandwich between square roots (Technique 8.3.19), Compare $𝐴 𝐵$ and $𝐵 𝐴$ (Technique 7.3.21)

Hint 1. For (a), a positive operator is self-adjoint, and

{(𝐴 𝐵)}^{*} = 𝐵 𝐴

. For the converse,

𝐵

commutes with

\sqrt{𝐴}

(Theorem 8.3.9).

Hint 2. For (b), write

𝐴 𝐵 = 𝑋 𝑌

with

𝑋 = \sqrt{𝐴}

and

𝑌 = \sqrt{𝐴} 𝐵

. What is

𝑌 𝑋

? Use Proposition 7.3.22 on the matrices of

𝑋

and

𝑌

Hint 3. For (c),

𝐴 𝐵 =

\sqrt{𝐴} (\sqrt{𝐴} 𝐵 \sqrt{𝐴}) {(\sqrt{𝐴})}^{- 1}

. Push an eigenbasis of the positive operator

\sqrt{𝐴} 𝐵 \sqrt{𝐴}

forward by

\sqrt{𝐴}

Sketch. (a) If

𝐴 𝐵

is positive it is self-adjoint, so

𝐴 𝐵 = {(𝐴 𝐵)}^{*} = 𝐵 𝐴

. If

𝐴 𝐵 = 𝐵 𝐴

, then

𝐵

commutes with

\sqrt{𝐴}

, so

𝐴 𝐵 = \sqrt{𝐴} 𝐵 \sqrt{𝐴}

, which is positive. (b)

𝐴 𝐵

and

\sqrt{𝐴} 𝐵 \sqrt{𝐴}

are

𝑋 𝑌

and

𝑌 𝑋

for

𝑋 = \sqrt{𝐴}

𝑌 = \sqrt{𝐴} 𝐵

, so they have the same characteristic polynomial, which for the positive operator

\sqrt{𝐴} 𝐵 \sqrt{𝐴}

\prod (𝑥 - 𝑑_{𝑖})

with

𝑑_{𝑖} \geq 0

. (c) If

\sqrt{𝐴} 𝐵 \sqrt{𝐴} 𝐮_{𝑖} =

𝑑_{𝑖} 𝐮_{𝑖}

for an eigenbasis

𝐮_{𝑖}

, then

𝐴 𝐵 (\sqrt{𝐴} 𝐮_{𝑖}) =

𝑑_{𝑖} \sqrt{𝐴} 𝐮_{𝑖}

, and

\sqrt{𝐴} 𝐮_{1}, \dots, \sqrt{𝐴} 𝐮_{𝑛}

is a basis because

\sqrt{𝐴}

is invertible.

First, $𝐶 = \sqrt{𝐴} 𝐵 \sqrt{𝐴}$ is positive: it is self-adjoint by Proposition 6.6.18, and $⟨ 𝐶 𝐯, 𝐯 ⟩ =$ $⟨ 𝐵 \sqrt{𝐴} 𝐯, \sqrt{𝐴} 𝐯 ⟩ \geq$ $0$ .

(a) If $𝐴 𝐵$ is positive, it is self-adjoint, so $𝐴 𝐵 =$ ${(𝐴 𝐵)}^{*} =$ $𝐵^{*} 𝐴^{*} =$ $𝐵 𝐴$ . Conversely, if $𝐴 𝐵 = 𝐵 𝐴$ , then $𝐵$ commutes with $\sqrt{𝐴}$ by Theorem 8.3.9, and $𝐴 𝐵 =$ $\sqrt{𝐴} \sqrt{𝐴} 𝐵 =$ $\sqrt{𝐴} 𝐵 \sqrt{𝐴} =$ $𝐶$ , which is positive.

(b) Fix an orthonormal basis $𝒪︀$ and let $𝑋 = {[\sqrt{𝐴}]}_{𝒪︀}$ and $𝑌 = {[\sqrt{𝐴} 𝐵]}_{𝒪︀}$ . By Theorem 2.4.2, $𝑋 𝑌 =$ ${[\sqrt{𝐴} \sqrt{𝐴} 𝐵]}_{𝒪︀} =$ ${[𝐴 𝐵]}_{𝒪︀}$ and $𝑌 𝑋 =$ ${[\sqrt{𝐴} 𝐵 \sqrt{𝐴}]}_{𝒪︀} =$ ${[𝐶]}_{𝒪︀}$ . So by Definition 7.3.12 and Proposition 7.3.22, $𝑝_{𝐴 𝐵} =$ $𝑝_{𝑋 𝑌} =$ $𝑝_{𝑌 𝑋} =$ $𝑝_{𝐶}$ . Since $𝐶$ is positive, Theorem 8.3.5(III) gives an orthonormal basis in which $𝐶$ is $diag (𝑑_{1}, \dots, 𝑑_{𝑛})$ with every $𝑑_{𝑖} \geq 0$ , so $𝑝_{𝐶} =$ $(𝑥 - 𝑑_{1}) \dots (𝑥 - 𝑑_{𝑛})$ . Hence every root of $𝑝_{𝐴 𝐵}$ is one of the $𝑑_{𝑖}$ , and every eigenvalue of $𝐴 𝐵$ , being a root of $𝑝_{𝐴 𝐵}$ (Corollary 7.3.13), is nonnegative.

(c) If $𝐴$ is invertible, then $0$ is not an eigenvalue of $𝐴$ , so the eigenvalues $\sqrt{𝜆_{𝑗}}$ of $\sqrt{𝐴}$ are positive and $\sqrt{𝐴}$ is invertible (Theorem 7.2.9). Let $𝐮_{1}, \dots, 𝐮_{𝑛}$ be an eigenbasis of the positive operator $𝐶$ , with $𝐶 𝐮_{𝑖} = 𝑑_{𝑖} 𝐮_{𝑖}$ (Theorem 8.3.5(III)). Then

𝐴 𝐵 (\sqrt{𝐴} 𝐮_{𝑖}) = \sqrt{𝐴} (\sqrt{𝐴} 𝐵 \sqrt{𝐴}) 𝐮_{𝑖} = \sqrt{𝐴} 𝐶 𝐮_{𝑖} = 𝑑_{𝑖} \sqrt{𝐴} 𝐮_{𝑖},

and $\sqrt{𝐴} 𝐮_{1}, \dots, \sqrt{𝐴} 𝐮_{𝑛}$ is a basis of $𝑉$ , being the image of a basis under an invertible map. So $𝑉$ has a basis of eigenvectors of $𝐴 𝐵$ , and $𝐴 𝐵$ is diagonalizable (Theorem 7.4.2). □

For the example above, $𝐴 𝐵 = (\begin{matrix} 2 & 0 \\ 1 & 0 \end{matrix})$ has eigenvalues $2$ and $0$ , and it is diagonalizable, as (c) predicts since $det 𝐴 = 1$ . Its eigenvectors $(2, 1)$ and $(0, 1)$ are not orthogonal, so $𝐴 𝐵$ is diagonalizable but not unitarily diagonalizable. When $𝐴$ is invertible, $𝐴 𝐵$ is similar to a positive operator, $𝐴 𝐵 = \sqrt{𝐴} 𝐶 {(\sqrt{𝐴})}^{- 1}$ , but similarity does not preserve orthogonality.

8.3.6The Löwner order Beyond 21-242

The Löwner order is not covered in 21-242.

For real numbers, $𝑥 \leq 𝑦$ means that $𝑦 - 𝑥 \geq 0$ . Positivity gives the same kind of comparison between self-adjoint operators.

Definition 8.3.21 (Löwner order). For self-adjoint

𝑆, 𝑇 \in L (𝑉)

, write

𝑆 ⪯ 𝑇

𝑇 - 𝑆

is positive, that is, if

⟨ 𝑆 𝐯, 𝐯 ⟩ \leq

⟨ 𝑇 𝐯, 𝐯 ⟩

for every

𝐯 \in 𝑉

. In particular

0 ⪯ 𝑇

means that

𝑇

is positive.

The two descriptions agree because $𝑇 - 𝑆$ is automatically self-adjoint. For matrices, $𝐴 ⪯ 𝐵$ is defined in the same way for Hermitian $𝐴$ and $𝐵$ .

Proposition 8.3.22 (Properties of the Löwner order). Let $𝑆, 𝑇, 𝑈 \in L (𝑉)$ be self-adjoint.

(a) $⪯$ is a partial order: $𝑇 ⪯ 𝑇$ ; if $𝑆 ⪯ 𝑇$ and $𝑇 ⪯ 𝑆$ , then $𝑆 = 𝑇$ ; and if $𝑆 ⪯ 𝑇$ and $𝑇 ⪯ 𝑈$ , then $𝑆 ⪯ 𝑈$ .
(b)If $𝑆 ⪯ 𝑇$ , then $𝑆 + 𝑈 ⪯ 𝑇 + 𝑈$ , and $𝑐 𝑆 ⪯ 𝑐 𝑇$ for every real $𝑐 \geq 0$ .
(c)If $𝑆 ⪯ 𝑇$ , $𝑉^{'}$ is a finite-dimensional inner product space, and $𝑋 \in L (𝑉^{'}, 𝑉)$ , then $𝑋^{*} 𝑆 𝑋 ⪯ 𝑋^{*} 𝑇 𝑋$ .
(d)For $𝑐 \in ℝ$ , $𝑇 ⪯ 𝑐 {id}_{𝑉}$ if and only if every eigenvalue of $𝑇$ is at most $𝑐$ , and $𝑐 {id}_{𝑉} ⪯ 𝑇$ if and only if every eigenvalue of $𝑇$ is at least $𝑐$ .

Proof.routine

Uses. Definition 8.3.21 (Löwner order), Lemma 8.3.8 (Where the form vanishes), Proposition 6.6.18 (Properties of the adjoint), Theorem 8.3.5 (Characterizations of positive operators)

Techniques. Move an operator across the inner product (Technique 6.6.20)

Hint 1. Each part is a statement about one positive operator, such as

𝑇 - 𝑆

. For antisymmetry,

𝑃 = 𝑇 - 𝑆

and

- 𝑃

are both positive; use Lemma 8.3.8.

Hint 2. For (c), move

𝑋

across the inner product. For (d), compare the eigenvalues of

𝑐 {id}_{𝑉} - 𝑇

with those of

𝑇

Sketch. (a)

𝑇 - 𝑇 = 0

. If

𝑃 = 𝑇 - 𝑆

and

- 𝑃

are positive, then

⟨ 𝑃 𝐯, 𝐯 ⟩ = 0

for all

𝐯

, so

𝑃 = 0

by Lemma 8.3.8.

𝑈 -

𝑆 =

(𝑈 - 𝑇) +

(𝑇 - 𝑆)

is a sum of positive operators. (b) The differences are

𝑇 - 𝑆

and

𝑐 (𝑇 - 𝑆)

. (c)

⟨ 𝑋^{*} (𝑇 - 𝑆) 𝑋 𝐮, 𝐮 ⟩ =

⟨ (𝑇 - 𝑆) 𝑋 𝐮, 𝑋 𝐮 ⟩ \geq

0

. (d)

𝑐 {id}_{𝑉} - 𝑇

is self-adjoint with eigenvalues

𝑐 - 𝜆

; apply Theorem 8.3.5(II).

(a) $𝑇 - 𝑇 = 0$ is positive. If $𝑃 = 𝑇 - 𝑆$ and $- 𝑃 = 𝑆 - 𝑇$ are both positive, then $⟨ 𝑃 𝐯, 𝐯 ⟩ \geq 0$ and $- ⟨ 𝑃 𝐯, 𝐯 ⟩ \geq 0$ , so $⟨ 𝑃 𝐯, 𝐯 ⟩ = 0$ for every $𝐯$ , and $𝑃 𝐯 = 𝟎$ for every $𝐯$ by Lemma 8.3.8. So $𝑆 = 𝑇$ . If $𝑆 ⪯ 𝑇 ⪯ 𝑈$ , then $𝑈 -$ $𝑆 =$ $(𝑈 - 𝑇) +$ $(𝑇 - 𝑆)$ is self-adjoint and $⟨ (𝑈 - 𝑆) 𝐯, 𝐯 ⟩ =$ $⟨ (𝑈 - 𝑇) 𝐯, 𝐯 ⟩ +$ $⟨ (𝑇 - 𝑆) 𝐯, 𝐯 ⟩ \geq$ $0$ .

(b) $(𝑇 + 𝑈) -$ $(𝑆 + 𝑈) =$ $𝑇 -$ $𝑆$ is positive, and $𝑐 𝑇 -$ $𝑐 𝑆 =$ $𝑐 (𝑇 - 𝑆)$ is self-adjoint ( $𝑐$ is real) with $⟨ 𝑐 (𝑇 - 𝑆) 𝐯, 𝐯 ⟩ =$ $𝑐 ⟨ (𝑇 - 𝑆) 𝐯, 𝐯 ⟩ \geq$ $0$ .

(c) $𝑋^{*} 𝑇 𝑋 -$ $𝑋^{*} 𝑆 𝑋 =$ $𝑋^{*} (𝑇 - 𝑆) 𝑋$ is self-adjoint by Proposition 6.6.18, and for $𝐮 \in 𝑉^{'}$ , $⟨ 𝑋^{*} (𝑇 - 𝑆) 𝑋 𝐮, 𝐮 ⟩ =$ $⟨ (𝑇 - 𝑆) 𝑋 𝐮, 𝑋 𝐮 ⟩ \geq$ $0$ .

(d) $𝑇 𝐯 = 𝜆 𝐯$ if and only if $(𝑐 {id}_{𝑉} - 𝑇) 𝐯 =$ $(𝑐 - 𝜆) 𝐯$ , so the eigenvalues of the self-adjoint operator $𝑐 {id}_{𝑉} - 𝑇$ are the numbers $𝑐 - 𝜆$ with $𝜆$ an eigenvalue of $𝑇$ . By Theorem 8.3.5(II), $𝑐 {id}_{𝑉} - 𝑇$ is positive if and only if $𝑐 - 𝜆 \geq 0$ for all of them. The second statement is the same argument for $𝑇 - 𝑐 {id}_{𝑉}$ . □

For example, every orthogonal projection satisfies $0 ⪯ {Pr}_{𝑈} ⪯ {id}_{𝑉}$ , by (d), because its eigenvalues are $0$ and $1$ . Unlike the order on $ℝ$ , the Löwner order is not total: neither of $diag (1, 0)$ and $diag (0, 1)$ is $⪯$ the other, since their difference has eigenvalues $1$ and $- 1$ . Some familiar inequalities survive and some do not. Inverses reverse the order and square roots preserve it, while squares need not preserve it (Exercise 8.3.39, Exercise 8.3.40). The inverse inequality is proved with Technique 8.3.19, which compares an operator with ${id}_{𝑉}$ , where (d) applies; the square-root inequality uses the eigenvector trick of the third uniqueness proof in Theorem 8.3.9.

8.3.7Cholesky factorization Beyond 21-242

Cholesky factorization is standard in numerical linear algebra (LADR 7D), but it is not part of the course.

By Theorem 8.3.5 a positive matrix can be written as $𝐵^{*} 𝐵$ in many ways, for example with $𝐵 = \sqrt{𝐴}$ . One of these factorizations is triangular, and it can be computed by a short algorithm that needs no eigenvalues. It is the standard way to solve $𝐴 𝐱 = 𝐛$ when $𝐴$ is positive definite, and it is also the cheapest test of positive definiteness.

Theorem 8.3.23 (Cholesky factorization). Let $𝐴 \in 𝔽^{𝑛 \times 𝑛}$ be positive definite. Then there is exactly one upper triangular $𝑅 \in 𝔽^{𝑛 \times 𝑛}$ with positive real diagonal entries such that

𝐴 = 𝑅^{*} 𝑅 .

Proof (via QR), part 1 of 2: existence.routine

Uses. Proposition 8.3.15 (Characterizations of positive definite operators), Theorem 6.4.38 (QR factorization), Theorem 6.3.29 (Orthonormal columns)

Hint 1. Write

𝐴 = 𝐵^{*} 𝐵

with

𝐵

invertible, and factor

𝐵

Hint 2. Apply QR factorization (Theorem 6.4.38) to

𝐵

. What is

𝑄^{*} 𝑄

Sketch.

𝐴 = 𝐵^{*} 𝐵

with

𝐵

invertible (for instance

𝐵 = \sqrt{𝐴}

). QR gives

𝐵 = 𝑄 𝑅

with orthonormal columns in

𝑄

, so

𝐴 =

𝑅^{*} 𝑄^{*} 𝑄 𝑅 =

𝑅^{*} 𝑅

By Proposition 8.3.15, $𝐴 = 𝐵^{*} 𝐵$ for some $𝐵 \in 𝔽^{𝑛 \times 𝑛}$ with $rank 𝐵 = 𝑛$ (for instance $𝐵 = \sqrt{𝐴}$ ). The columns of $𝐵$ are independent, so Theorem 6.4.38 gives $𝐵 = 𝑄 𝑅$ with $𝑄 \in 𝔽^{𝑛 \times 𝑛}$ having orthonormal columns and $𝑅$ upper triangular with positive diagonal entries. Since $𝑄^{*} 𝑄 = 𝐼_{𝑛}$ (Theorem 6.3.29),

𝐴 = 𝐵^{*} 𝐵 = 𝑅^{*} 𝑄^{*} 𝑄 𝑅 = 𝑅^{*} 𝑅 .

The first proof hides the work inside QR. The second builds $𝑅$ one row at a time, and it is the proof behind the algorithm below.

Proof (by induction on $𝑛$ ), part 1 of 2: existence.standard

Uses. Definition 2.4.33 (Block matrix), Definition 8.3.14 (Positive definite)

Techniques. Induction on matrix size (Technique 5.5.17), Block decomposition (Technique 2.4.35), Plug in clever test vectors (Technique 6.1.40)

Hint 1. Write

𝐴 = (\begin{matrix} 𝑎 & 𝐰^{*} \\ 𝐰 & 𝐴^{'} \end{matrix})

with

𝑎 \in ℝ

. Why is

𝑎 > 0

? Guess the first row of

𝑅

from the

(1, 1)

and

(1, 2)

blocks of

𝑅^{*} 𝑅

Hint 2. The first row must be

(\sqrt{𝑎}, 𝐰^{*} / \sqrt{𝑎})

. The rest of

𝑅

must then be a Cholesky factor of

𝐴^{″} = 𝐴^{'} - 𝐰 𝐰^{*} / 𝑎

Hint 3. To see that

𝐴^{″}

is positive definite, evaluate

𝐱^{*} 𝐴 𝐱

𝐱 = (𝑡, 𝐲)

with

𝑡 = - 𝐰^{*} 𝐲 / 𝑎

Sketch. Write

𝐴 = (\begin{matrix} 𝑎 & 𝐰^{*} \\ 𝐰 & 𝐴^{'} \end{matrix})

with

𝑎 = 𝐞_{1}^{*} 𝐴 𝐞_{1} > 0

, and let

𝐴^{″} = 𝐴^{'} - 𝐰 𝐰^{*} / 𝑎

. For

𝐲 \neq 𝟎

and

𝑡 = - 𝐰^{*} 𝐲 / 𝑎

𝐲^{*} 𝐴^{″} 𝐲 =

{(𝑡, 𝐲)}^{*} 𝐴 (𝑡, 𝐲) >

0

, so

𝐴^{″}

is positive definite and

𝐴^{″} = 𝑅^{″ *} 𝑅^{″}

by induction. Then

𝑅 =

(\begin{matrix} \sqrt{𝑎} & 𝐰^{*} / \sqrt{𝑎} \\ 𝟎 & 𝑅^{″} \end{matrix})

works, by block multiplication.

We induct on $𝑛$ . If $𝑛 = 1$ , then $𝐴 = (𝑎)$ with $𝑎 = 𝐞_{1}^{*} 𝐴 𝐞_{1} > 0$ , and $𝑅 = (\sqrt{𝑎})$ . Let $𝑛 \geq 2$ , and write in block form (Definition 2.4.33)

𝐴 = (\begin{matrix} 𝑎 & 𝐰^{*} \\ 𝐰 & 𝐴^{'} \end{matrix}), 𝑎 \in ℝ, 𝐰 \in 𝔽^{𝑛 - 1}, 𝐴^{'} \in 𝔽^{(𝑛 - 1) \times (𝑛 - 1)},

where $𝑎$ is real and $𝐴^{' *} = 𝐴^{'}$ because $𝐴^{*} = 𝐴$ , and $𝑎 = 𝐞_{1}^{*} 𝐴 𝐞_{1} > 0$ . Let $𝐴^{″} = 𝐴^{'} - 𝐰 𝐰^{*} / 𝑎$ , which is Hermitian. For $𝑡 \in 𝔽$ and $𝐲 \in 𝔽^{𝑛 - 1}$ , block multiplication gives

{(\begin{matrix} 𝑡 \\ 𝐲 \end{matrix})}^{*} 𝐴 (\begin{matrix} 𝑡 \\ 𝐲 \end{matrix}) = 𝑎 {| 𝑡 |}^{2} + 𝑡 𝐰^{*} 𝐲 + 𝑡 𝐲^{*} 𝐰 + 𝐲^{*} 𝐴^{'} 𝐲 .

Take $𝐲 \neq 𝟎$ and $𝑡 = - 𝐰^{*} 𝐲 / 𝑎$ . Since $𝐲^{*} 𝐰 = 𝐰^{*} 𝐲$ , the first three terms are ${| 𝐰^{*} 𝐲 |}^{2} / 𝑎 -$ $2 {| 𝐰^{*} 𝐲 |}^{2} / 𝑎 =$ $- {| 𝐰^{*} 𝐲 |}^{2} / 𝑎 =$ $- 𝐲^{*} 𝐰 𝐰^{*} 𝐲 / 𝑎$ , so the whole expression equals $𝐲^{*} 𝐴^{″} 𝐲$ . The vector $(𝑡, 𝐲)$ is nonzero, so $𝐲^{*} 𝐴^{″} 𝐲 > 0$ , and $𝐴^{″}$ is positive definite. By the induction hypothesis, $𝐴^{″} = 𝑅^{″ *} 𝑅^{″}$ with $𝑅^{″}$ upper triangular with positive diagonal entries. Let

𝑅 = (\begin{matrix} \sqrt{𝑎} & 𝐰^{*} / \sqrt{𝑎} \\ 𝟎 & 𝑅^{″} \end{matrix}) .

It is upper triangular with positive diagonal entries, and

𝑅^{*} 𝑅 = (\begin{matrix} \sqrt{𝑎} & 𝟎^{*} \\ 𝐰 / \sqrt{𝑎} & 𝑅^{″ *} \end{matrix}) (\begin{matrix} \sqrt{𝑎} & 𝐰^{*} / \sqrt{𝑎} \\ 𝟎 & 𝑅^{″} \end{matrix}) = (\begin{matrix} 𝑎 & 𝐰^{*} \\ 𝐰 & 𝐰 𝐰^{*} / 𝑎 + 𝑅^{″ *} 𝑅^{″} \end{matrix}) = (\begin{matrix} 𝑎 & 𝐰^{*} \\ 𝐰 & 𝐴^{'} \end{matrix}) = 𝐴 .

Proof, part 2 of 2: uniqueness.standard

Uses. Lemma 3.3.32 (Products and inverses of triangular matrices), Definition 3.3.31 (Triangular matrices), Proposition 5.3.15 (Triangular matrices), Theorem 5.4.7 (Invertibility and the determinant)

Techniques. Uniqueness via two candidates (Technique 1.1.8)

Hint 1. Suppose

𝑅^{*} 𝑅 = 𝑆^{*} 𝑆

, with

𝑅, 𝑆

as in the theorem. Rearrange to

𝑆 𝑅^{- 1} = {(𝑆^{*})}^{- 1} 𝑅^{*}

Hint 2. The left side is upper triangular and the right side lower triangular, so both are diagonal. Compare their diagonal entries.

Sketch.

𝑅

and

𝑆

are invertible, and

𝑆 𝑅^{- 1} =

{(𝑆^{*})}^{- 1} 𝑅^{*} =

{(𝑅 𝑆^{- 1})}^{*}

is both upper and lower triangular, hence diagonal, with diagonal entries

𝑠_{𝑖 𝑖} / 𝑟_{𝑖 𝑖}

and

𝑟_{𝑖 𝑖} / 𝑠_{𝑖 𝑖}

. These agree, so

𝑠_{𝑖 𝑖}^{2} = 𝑟_{𝑖 𝑖}^{2}

and

𝑠_{𝑖 𝑖} = 𝑟_{𝑖 𝑖}

. Thus

𝑆 𝑅^{- 1} = 𝐼_{𝑛}

Suppose $𝐴 = 𝑅^{*} 𝑅 = 𝑆^{*} 𝑆$ with $𝑅$ and $𝑆$ upper triangular with positive diagonal entries. Both are invertible, since their determinants are the products of the diagonal entries (Proposition 5.3.15, Theorem 5.4.7). An invertible upper triangular matrix $𝑅$ has an upper triangular inverse whose diagonal entries are $1 / 𝑟_{𝑖 𝑖}$ : write $𝑅 = 𝐷 𝑈$ with $𝐷 =$ $diag (𝑟_{11}, \dots, 𝑟_{𝑛 𝑛})$ and $𝑈 = 𝐷^{- 1} 𝑅$ unit upper triangular, so that $𝑅^{- 1} = 𝑈^{- 1} 𝐷^{- 1}$ , and apply Lemma 3.3.32. Multiplying $𝑆^{*} 𝑆 = 𝑅^{*} 𝑅$ on the left by ${(𝑆^{*})}^{- 1}$ and on the right by $𝑅^{- 1}$ gives

𝑆 𝑅^{- 1} = {(𝑆^{*})}^{- 1} 𝑅^{*} = {(𝑅 𝑆^{- 1})}^{*} .

By Lemma 3.3.32, $𝑆 𝑅^{- 1}$ and $𝑅 𝑆^{- 1}$ are upper triangular, with diagonal entries $𝑠_{𝑖 𝑖} / 𝑟_{𝑖 𝑖}$ and $𝑟_{𝑖 𝑖} / 𝑠_{𝑖 𝑖}$ . So the matrix $𝑀 = 𝑆 𝑅^{- 1}$ is upper triangular, and also lower triangular, being the conjugate transpose of an upper triangular matrix. Hence $𝑀$ is diagonal, and comparing diagonal entries, $𝑠_{𝑖 𝑖} / 𝑟_{𝑖 𝑖} =$ $𝑟_{𝑖 𝑖} / 𝑠_{𝑖 𝑖} =$ $𝑟_{𝑖 𝑖} / 𝑠_{𝑖 𝑖}$ . So $𝑠_{𝑖 𝑖}^{2} = 𝑟_{𝑖 𝑖}^{2}$ , and since both are positive, $𝑠_{𝑖 𝑖} = 𝑟_{𝑖 𝑖}$ . Therefore $𝑀 = 𝐼_{𝑛}$ and $𝑆 = 𝑅$ . □

The equations behind the algorithm come from reading $𝐴 = 𝑅^{*} 𝑅$ entry by entry. Since $𝑟_{𝑖 𝑘} = 0$ for $𝑖 > 𝑘$ , for $𝑘 \leq 𝑗$ the $(𝑘, 𝑗)$ entry is

𝑎_{𝑘 𝑗} = \sum_{𝑖 \leq 𝑘} 𝑟_{𝑖 𝑘} 𝑟_{𝑖 𝑗} = \sum_{𝑖 < 𝑘} 𝑟_{𝑖 𝑘} 𝑟_{𝑖 𝑗} + 𝑟_{𝑘 𝑘} 𝑟_{𝑘 𝑗} .

The last term is the only one that involves row $𝑘$ of $𝑅$ . So if rows $1, \dots, 𝑘 - 1$ are known, the equation with $𝑗 = 𝑘$ determines $𝑟_{𝑘 𝑘}$ , and then the equations with $𝑗 > 𝑘$ determine the rest of row $𝑘$ .

Algorithm 8.3.24 (Cholesky factorization)

Input: A Hermitian matrix $𝐴 \in 𝔽^{𝑛 \times 𝑛}$ .
Output: The Cholesky factor $𝑅$ of $𝐴$ , or the conclusion that $𝐴$ is not positive definite.

Set

𝑘 = 1

Compute

𝑑 =

𝑎_{𝑘 𝑘} -

\sum_{𝑖 < 𝑘} {| 𝑟_{𝑖 𝑘} |}^{2}

. If

𝑑 \leq 0

, stop:

𝐴

is not positive definite. Otherwise set

𝑟_{𝑘 𝑘} = \sqrt{𝑑}

For

𝑗 = 𝑘 + 1, \dots, 𝑛

, set

𝑟_{𝑘 𝑗} =

(𝑎_{𝑘 𝑗} - \sum_{𝑖 < 𝑘} 𝑟_{𝑖 𝑘} 𝑟_{𝑖 𝑗}) / 𝑟_{𝑘 𝑘}

𝑘 < 𝑛

, increase

𝑘

1

and return to step 2. Otherwise output

𝑅 = (𝑟_{𝑖 𝑗})

, with

𝑟_{𝑖 𝑗} = 0

for

𝑖 > 𝑗

Proposition 8.3.25 (Correctness of the Cholesky algorithm). Let

𝐴 \in 𝔽^{𝑛 \times 𝑛}

be Hermitian. If

𝐴

is positive definite, Algorithm 8.3.24 outputs the matrix

𝑅

of Theorem 8.3.23. If

𝐴

is not positive definite, Algorithm 8.3.24 stops in step 2.

Proof.standard

Uses. Theorem 8.3.23 (Cholesky factorization), Proposition 8.3.15 (Characterizations of positive definite operators), Proposition 5.3.15 (Triangular matrices)

Techniques. Algorithms as proofs: invariants and termination (Technique 3.3.5), Compute entry by entry (Technique 2.4.15)

Hint 1. If

𝐴

is positive definite, the Cholesky factor exists. Show that its entries satisfy exactly the equations that the algorithm solves, in the same order.

Hint 2. If the algorithm runs to the end, it has produced an upper triangular

𝑅

with positive diagonal and

𝑎_{𝑘 𝑗} =

{(𝑅^{*} 𝑅)}_{𝑘 𝑗}

for

𝑘 \leq 𝑗

. Why does this give

𝐴 = 𝑅^{*} 𝑅

, and why is

𝐴

then positive definite?

Sketch. The entries of the Cholesky factor satisfy

𝑟_{𝑘 𝑘}^{2} =

𝑎_{𝑘 𝑘} -

\sum_{𝑖 < 𝑘} {| 𝑟_{𝑖 𝑘} |}^{2}

and

𝑟_{𝑘 𝑘} 𝑟_{𝑘 𝑗} =

𝑎_{𝑘 𝑗} -

\sum_{𝑖 < 𝑘} 𝑟_{𝑖 𝑘} 𝑟_{𝑖 𝑗}

, so by induction on the order of computation the algorithm reproduces them, with

𝑑 = 𝑟_{𝑘 𝑘}^{2} > 0

. Conversely, if the algorithm finishes,

𝐴

and

𝑅^{*} 𝑅

agree on and above the diagonal, hence everywhere, and

𝐴 = 𝑅^{*} 𝑅

with

𝑅

invertible is positive definite. So for

𝐴

not positive definite it cannot finish.

Suppose $𝐴$ is positive definite, and let $𝑅$ be its Cholesky factor (Theorem 8.3.23). By the entry formula above, for each $𝑘$ ,

𝑟_{𝑘 𝑘}^{2} = 𝑎_{𝑘 𝑘} - \sum_{𝑖 < 𝑘} {| 𝑟_{𝑖 𝑘} |}^{2}, 𝑟_{𝑘 𝑘} 𝑟_{𝑘 𝑗} = 𝑎_{𝑘 𝑗} - \sum_{𝑖 < 𝑘} 𝑟_{𝑖 𝑘} 𝑟_{𝑖 𝑗} (𝑗 > 𝑘) .

We show by induction on $𝑘$ that the algorithm computes row $𝑘$ of $𝑅$ . If it has computed rows $1, \dots, 𝑘 - 1$ correctly, then the number $𝑑$ of step 2 is $𝑟_{𝑘 𝑘}^{2} > 0$ , so the algorithm does not stop and sets $𝑟_{𝑘 𝑘} = \sqrt{𝑑}$ , the correct (positive) value; and step 3 solves the second equation for $𝑟_{𝑘 𝑗}$ . So the output is $𝑅$ .

Conversely, suppose the algorithm runs to the end and outputs $𝑅$ . Then $𝑅$ is upper triangular with positive diagonal entries, and the equations it solved say that $𝑎_{𝑘 𝑗} =$ ${(𝑅^{*} 𝑅)}_{𝑘 𝑗}$ for all $𝑘 \leq 𝑗$ . Both $𝐴$ and $𝑅^{*} 𝑅$ are Hermitian, so they also agree below the diagonal, and $𝐴 = 𝑅^{*} 𝑅$ . Since $det 𝑅$ is the product of its diagonal entries (Proposition 5.3.15), $𝑅$ has rank $𝑛$ , and $𝐴$ is positive definite by Proposition 8.3.15. So if $𝐴$ is not positive definite, the algorithm cannot run to the end, and it stops in step 2. □

Example 8.3.26 (A Cholesky factorization). Let

𝐴 = (\begin{matrix} 4 & 2 & - 2 \\ 2 & 10 & 2 \\ - 2 & 2 & 6 \end{matrix}) .

For $𝑘 = 1$ : $𝑟_{11} = \sqrt{4} = 2$ , $𝑟_{12} = 2 / 2 = 1$ , $𝑟_{13} = - 2 / 2 = - 1$ . For $𝑘 = 2$ : $𝑑 = 10 - 1^{2} = 9$ , so $𝑟_{22} = 3$ , and $𝑟_{23} =$ $(2 - 1 \cdot (- 1)) / 3 =$ $1$ . For $𝑘 = 3$ : $𝑑 = 6 - {(- 1)}^{2} - 1^{2} = 4$ , so $𝑟_{33} = 2$ . Hence

𝐴 = 𝑅^{𝖳} 𝑅, 𝑅 = (\begin{matrix} 2 & 1 & - 1 \\ 0 & 3 & 1 \\ 0 & 0 & 2 \end{matrix}),

and $𝐴$ is positive definite. If the corner entry $6$ is replaced by $2$ , the last step gives $𝑑 = 2 - 1 - 1 = 0$ , and the algorithm stops: the new matrix $𝐴^{'}$ is not positive definite. Indeed $𝐴^{'} (2, - 1, 3) = 𝟎$ , so ${(2, - 1, 3)}^{𝖳} 𝐴^{'} (2, - 1, 3) =$ $0$ .

Remark 8.3.27 (What Cholesky adds). For a real positive definite $𝐴 = 𝑅^{𝖳} 𝑅$ , the matrices $𝐿 =$ $𝑅^{𝖳} diag (1 / 𝑟_{11}, \dots, 1 / 𝑟_{𝑛 𝑛})$ , which is unit lower triangular, and $𝐷 =$ $diag (𝑟_{11}^{2}, \dots, 𝑟_{𝑛 𝑛}^{2})$ satisfy $𝐿 𝐷 𝐿^{𝖳} =$ $𝑅^{𝖳} 𝑅 =$ $𝐴$ . This is the factorization of Theorem 3.3.37, and $𝑅 = \sqrt{𝐷} 𝐿^{𝖳}$ : Cholesky is $𝐿 𝐷 𝐿^{𝖳}$ with the pivots split evenly between the two factors. Because of the symmetry it needs about $𝑛^{3} / 6$ multiplications, half as many as LU.

The induction proof never used eigenvalues. It shows directly that a positive definite matrix has the form $𝐵^{*} 𝐵$ , which is the hard implication of Theorem 8.3.5, by elimination alone. It also shows that a Hermitian matrix is positive definite if and only if the pivots $𝑟_{𝑘 𝑘}^{2}$ produced along the way are all positive, a criterion that Section 12.3 rephrases in terms of determinants (Sylvester’s criterion).

Exercises

Exercise 8.3.28.routine Let $𝑆, 𝑇 \in L (𝑉)$ be positive, let $𝑐 \geq 0$ be real, and let $𝑋 \in L (𝑈, 𝑉)$ , where $𝑈$ is a finite-dimensional inner product space. Show that:

(a) $𝑆 + 𝑇$ and $𝑐 𝑇$ are positive;
(b) $𝑋^{*} 𝑇 𝑋 \in L (𝑈)$ is positive;
(c) $𝑇^{𝑚}$ is positive for every integer $𝑚 \geq 0$ ;
(d)if $𝑇$ is invertible, then $𝑇^{- 1}$ is positive.

Solution.

Uses. Definition 8.3.1 (Positive operator), Proposition 6.6.18 (Properties of the adjoint)

Techniques. Move an operator across the inner product (Technique 6.6.20)

Hint 1. Check the two conditions of Definition 8.3.1 directly. For (b), move

𝑋

across the inner product.

Hint 2. For (c), write

𝑇^{2 𝑘} =

{(𝑇^{𝑘})}^{*} {id}_{𝑉} 𝑇^{𝑘}

and

𝑇^{2 𝑘 + 1} =

{(𝑇^{𝑘})}^{*} 𝑇 𝑇^{𝑘}

, and use (b).

Hint 3. For (d), substitute

𝐯 = 𝑇 𝐰

⟨ 𝑇^{- 1} 𝐯, 𝐯 ⟩

Sketch. (a) Sums and real multiples of self-adjoint operators are self-adjoint, and the forms add and scale. (b)

⟨ 𝑋^{*} 𝑇 𝑋 𝐮, 𝐮 ⟩ =

⟨ 𝑇 𝑋 𝐮, 𝑋 𝐮 ⟩ \geq

0

. (c)

𝑇^{𝑘}

is self-adjoint, and

𝑇^{2 𝑘} =

{(𝑇^{𝑘})}^{*} {id}_{𝑉} 𝑇^{𝑘}

𝑇^{2 𝑘 + 1} =

{(𝑇^{𝑘})}^{*} 𝑇 𝑇^{𝑘}

are positive by (b). (d)

{(𝑇^{- 1})}^{*} =

{(𝑇^{*})}^{- 1} =

𝑇^{- 1}

and

⟨ 𝑇^{- 1} 𝐯, 𝐯 ⟩ =

⟨ 𝐰, 𝑇 𝐰 ⟩ \geq

0

for

𝐰 = 𝑇^{- 1} 𝐯

(a) By Proposition 6.6.18, ${(𝑆 + 𝑇)}^{*} = 𝑆 + 𝑇$ and ${(𝑐 𝑇)}^{*} = 𝑐 𝑇 = 𝑐 𝑇$ , as $𝑐$ is real. For every $𝐯$ , $⟨ (𝑆 + 𝑇) 𝐯, 𝐯 ⟩ =$ $⟨ 𝑆 𝐯, 𝐯 ⟩ +$ $⟨ 𝑇 𝐯, 𝐯 ⟩ \geq$ $0$ and $⟨ 𝑐 𝑇 𝐯, 𝐯 ⟩ =$ $𝑐 ⟨ 𝑇 𝐯, 𝐯 ⟩ \geq$ $0$ .

(b) ${(𝑋^{*} 𝑇 𝑋)}^{*} =$ $𝑋^{*} 𝑇^{*} 𝑋^{* *} =$ $𝑋^{*} 𝑇 𝑋$ by Proposition 6.6.18, and for $𝐮 \in 𝑈$ , $⟨ 𝑋^{*} 𝑇 𝑋 𝐮, 𝐮 ⟩ =$ $⟨ 𝑇 𝑋 𝐮, 𝑋 𝐮 ⟩ \geq$ $0$ .

(c) For $𝑚 = 0$ , $𝑇^{0} = {id}_{𝑉}$ is positive. By Proposition 6.6.18(d) and induction, ${(𝑇^{𝑘})}^{*} =$ ${(𝑇^{*})}^{𝑘} =$ $𝑇^{𝑘}$ . If $𝑚 = 2 𝑘$ , then $𝑇^{𝑚} =$ ${(𝑇^{𝑘})}^{*} {id}_{𝑉} 𝑇^{𝑘}$ , which is positive by (b) applied to the positive operator ${id}_{𝑉}$ and $𝑋 = 𝑇^{𝑘}$ . If $𝑚 = 2 𝑘 + 1$ , then $𝑇^{𝑚} =$ ${(𝑇^{𝑘})}^{*} 𝑇 𝑇^{𝑘}$ , positive by (b) with $𝑋 = 𝑇^{𝑘}$ . (Alternatively, $𝑇^{𝑚} =$ $\sum_{𝑗} 𝜆_{𝑗}^{𝑚} 𝑃_{𝑗}$ has the nonnegative eigenvalues $𝜆_{𝑗}^{𝑚}$ , by Proposition 8.2.24.)

(d) By Proposition 6.6.18(f), ${(𝑇^{- 1})}^{*} =$ ${(𝑇^{*})}^{- 1} =$ $𝑇^{- 1}$ . For $𝐯 \in 𝑉$ , let $𝐰 = 𝑇^{- 1} 𝐯$ . Then

⟨ 𝑇^{- 1} 𝐯, 𝐯 ⟩ = ⟨ 𝐰, 𝑇 𝐰 ⟩ = ⟨ 𝑇 𝐰, 𝐰 ⟩ = ⟨ 𝑇 𝐰, 𝐰 ⟩ \geq 0,

since $⟨ 𝑇 𝐰, 𝐰 ⟩$ is a nonnegative real number.

Exercise 8.3.29.standard Prove or disprove each statement. Throughout, $𝑉$ is a finite-dimensional inner product space.

(a)If $𝑇 \in L (𝑉)$ is positive, then for every basis $ℬ︀$ of $𝑉$ , the diagonal entries of ${[𝑇]}_{ℬ︀ ℬ︀}$ are nonnegative.
(b)If $𝑇 \in L (𝑉)$ is self-adjoint and $𝑇^{3}$ is positive, then $𝑇$ is positive.
(c)If $𝑇 \in L (𝑉)$ is self-adjoint and $𝑇^{2}$ is positive, then $𝑇$ is positive.
(d)If $𝑆, 𝑇 \in L (𝑉)$ are positive, then $𝑆 𝑇 + 𝑇 𝑆$ is positive.
(e)If $𝐴 \in ℂ^{𝑛 \times 𝑛}$ and $𝐱^{*} 𝐴 𝐱 \geq 0$ for every $𝐱 \in ℝ^{𝑛}$ , then $𝐴$ is positive.

Solution.

Uses. Theorem 8.3.5 (Characterizations of positive operators), Definition 8.3.1 (Positive operator), Proposition 8.1.1 (Eigenvalues and eigenvectors of self-adjoint operators)

Techniques. Finding counterexamples and minimal examples (Technique 1.4.18)

Hint 1. For (a), the diagonal entries are nonnegative in orthonormal bases. Try

𝑇 = diag (0, 1)

ℝ^{2}

and a basis that is far from orthogonal.

Hint 2. For (b) and (c), compare the eigenvalues of

𝑇

with those of

𝑇^{3}

and

𝑇^{2}

Hint 3. For (d), try a projection and a matrix of ones in

ℝ^{2 \times 2}

. For (e), real test vectors cannot see an imaginary part: try

𝐴 = (\begin{matrix} 0 & 𝑖 \\ - 𝑖 & 0 \end{matrix})

Sketch. (a) False:

𝑇 = diag (0, 1)

and

ℬ︀ = (1, 1), (1, 2)

give

{[𝑇]}_{ℬ︀ ℬ︀} = (\begin{matrix} - 1 & - 2 \\ 1 & 2 \end{matrix})

. (b) True: an eigenvalue

𝜆

𝑇

gives the eigenvalue

𝜆^{3} \geq 0

𝑇^{3}

, so

𝜆 \geq 0

. (c) False:

𝑇 = - {id}_{𝑉}

. (d) False:

𝑆 = diag (1, 0)

𝑇 = (\begin{matrix} 1 & 1 \\ 1 & 1 \end{matrix})

give

(\begin{matrix} 2 & 1 \\ 1 & 0 \end{matrix})

, with determinant

- 1

. (e) False:

(\begin{matrix} 0 & 𝑖 \\ - 𝑖 & 0 \end{matrix})

vanishes on real vectors but has the eigenvalue

- 1

(a) False. Let $𝑇 = 𝑇_{𝐴}$ on $ℝ^{2}$ with $𝐴 = diag (0, 1)$ , which is positive, and let $ℬ︀ = 𝐛_{1}, 𝐛_{2}$ with $𝐛_{1} = (1, 1)$ , $𝐛_{2} = (1, 2)$ . Then $𝑇 𝐛_{1} =$ $(0, 1) =$ $- 𝐛_{1} +$ $𝐛_{2}$ and $𝑇 𝐛_{2} =$ $(0, 2) =$ $- 2 𝐛_{1} +$ $2 𝐛_{2}$ , so

{[𝑇]}_{ℬ︀ ℬ︀} = (\begin{matrix} - 1 & - 2 \\ 1 & 2 \end{matrix}),

whose $(1, 1)$ entry is negative. For an orthonormal basis the statement is true, since the diagonal entries are $⟨ 𝑇 𝐞_{𝑖}, 𝐞_{𝑖} ⟩ \geq 0$ .

(b) True. Let $𝜆$ be an eigenvalue of $𝑇$ , with $𝑇 𝐯 = 𝜆 𝐯$ , $𝐯 \neq 𝟎$ ; $𝜆$ is real by Proposition 8.1.1. Then $𝑇^{3} 𝐯 = 𝜆^{3} 𝐯$ , so $𝜆^{3}$ is an eigenvalue of the positive operator $𝑇^{3}$ , and $𝜆^{3} \geq 0$ by Theorem 8.3.5. For a real number this forces $𝜆 \geq 0$ . So $𝑇$ is self-adjoint with nonnegative eigenvalues, hence positive by Theorem 8.3.5(II).

(c) False. If $dim 𝑉 \geq 1$ , then $𝑇 = - {id}_{𝑉}$ is self-adjoint and $𝑇^{2} = {id}_{𝑉}$ is positive, but $⟨ 𝑇 𝐯, 𝐯 ⟩ =$ $- {‖ 𝐯 ‖}^{2} <$ $0$ for $𝐯 \neq 𝟎$ . Here the argument of (b) breaks down because $𝜆^{2} \geq 0$ does not force $𝜆 \geq 0$ .

(d) False. Let $𝑆 = (\begin{matrix} 1 & 0 \\ 0 & 0 \end{matrix})$ and $𝑇 = (\begin{matrix} 1 & 1 \\ 1 & 1 \end{matrix})$ , both positive (a projection, and $𝑇 = 𝐮 𝐮^{𝖳}$ with $𝐮 = (1, 1)$ ). Then

𝑆 𝑇 + 𝑇 𝑆 = (\begin{matrix} 1 & 1 \\ 0 & 0 \end{matrix}) + (\begin{matrix} 1 & 0 \\ 1 & 0 \end{matrix}) = (\begin{matrix} 2 & 1 \\ 1 & 0 \end{matrix}),

and $𝐱 = (1, - 2)$ gives $𝐱^{𝖳} (𝑆 𝑇 + 𝑇 𝑆) 𝐱 =$ $2 -$ $2 -$ $2 +$ $0 =$ $- 2 <$ $0$ .

(e) False. Let $𝐴 = (\begin{matrix} 0 & 𝑖 \\ - 𝑖 & 0 \end{matrix})$ . Then $𝐴^{*} = 𝐴$ , and for real $𝐱$ , $𝐱^{*} 𝐴 𝐱 =$ $𝑖 𝑥_{1} 𝑥_{2} -$ $𝑖 𝑥_{2} 𝑥_{1} =$ $0$ . But $𝑝_{𝐴} = 𝑥^{2} - 1$ , so $𝐴$ has the eigenvalue $- 1$ ; for instance $𝐱 = (1, 𝑖)$ gives $𝐴 𝐱 = (- 1, - 𝑖)$ and $𝐱^{*} 𝐴 𝐱 =$ $- 1 +$ $(- 𝑖) (- 𝑖) =$ $- 2 <$ $0$ . Positivity of a complex matrix must be tested on complex vectors.

Exercise 8.3.30.standard Let $𝐴 \in ℝ^{𝑛 \times 𝑛}$ . Show that the following are equivalent:

(i) $𝐴$ is positive as an element of $ℝ^{𝑛 \times 𝑛}$ : $𝐴^{𝖳} = 𝐴$ and $𝐱^{𝖳} 𝐴 𝐱 \geq 0$ for every $𝐱 \in ℝ^{𝑛}$ ;
(ii) $𝐱^{*} 𝐴 𝐱 \geq 0$ for every $𝐱 \in ℂ^{𝑛}$ .

Then show that (ii) does not follow from “ $𝐱^{𝖳} 𝐴 𝐱 \geq 0$ for every $𝐱 \in ℝ^{𝑛}$ ” alone. This is why Definition 8.3.1 requires self-adjointness over $ℝ$ (Remark 8.3.4).

Solution.

Uses. Proposition 6.6.30 (Self-adjoint means real values), Remark 8.3.4 (Why self-adjointness is part of the definition), Corollary 6.6.28 (Self-adjointness in an orthonormal basis)

Hint 1. For (ii)

\Rightarrow

(i), over

ℂ

nonnegative values force self-adjointness (Proposition 6.6.30). A real Hermitian matrix is symmetric.

Hint 2. For (i)

\Rightarrow

(ii), write

𝐱 = 𝐮 + 𝑖 𝐯

with

𝐮, 𝐯 \in ℝ^{𝑛}

and expand

𝐱^{*} 𝐴 𝐱

. Where does symmetry make the imaginary part vanish?

Hint 3. For the last part, use

𝐵 = (\begin{matrix} 1 & - 1 \\ 1 & 1 \end{matrix})

of Non-example 8.3.3 and

𝐱 = (1, 𝑖)

Sketch. (ii)

\Rightarrow

(i): the values

𝐱^{*} 𝐴 𝐱

are real, so

𝐴

is Hermitian, hence symmetric as it is real; restrict to real

𝐱

. (i)

\Rightarrow

(ii): for

𝐱 = 𝐮 + 𝑖 𝐯

𝐱^{*} 𝐴 𝐱 =

𝐮^{𝖳} 𝐴 𝐮 +

𝐯^{𝖳} 𝐴 𝐯 +

𝑖 (𝐮^{𝖳} 𝐴 𝐯 - 𝐯^{𝖳} 𝐴 𝐮)

, and the imaginary part is

0

by symmetry. For

𝐵 = (\begin{matrix} 1 & - 1 \\ 1 & 1 \end{matrix})

𝐱^{𝖳} 𝐵 𝐱 = {‖ 𝐱 ‖}^{2}

ℝ^{2}

but

{(1, 𝑖)}^{*} 𝐵 (1, 𝑖) =

2 -

2 𝑖

(ii) $\Rightarrow$ (i). All values $𝐱^{*} 𝐴 𝐱 =$ $⟨ 𝑇_{𝐴} 𝐱, 𝐱 ⟩$ on $ℂ^{𝑛}$ are real, so $𝑇_{𝐴}$ is self-adjoint on $ℂ^{𝑛}$ by Proposition 6.6.30, and $𝐴^{*} = 𝐴$ (Corollary 6.6.28). Since $𝐴$ is real, $𝐴^{*} = 𝐴^{𝖳}$ , so $𝐴^{𝖳} = 𝐴$ . Taking $𝐱 \in ℝ^{𝑛}$ in (ii) gives $𝐱^{𝖳} 𝐴 𝐱 \geq 0$ .

(i) $\Rightarrow$ (ii). Let $𝐱 = 𝐮 + 𝑖 𝐯$ with $𝐮, 𝐯 \in ℝ^{𝑛}$ . Then $𝐱^{*} = 𝐮^{𝖳} - 𝑖 𝐯^{𝖳}$ , and since $𝐴$ is real,

𝐱^{*} 𝐴 𝐱 = 𝐮^{𝖳} 𝐴 𝐮 + 𝐯^{𝖳} 𝐴 𝐯 + 𝑖 (𝐮^{𝖳} 𝐴 𝐯 - 𝐯^{𝖳} 𝐴 𝐮) .

The number $𝐯^{𝖳} 𝐴 𝐮$ is a $1 \times 1$ matrix, so it equals its transpose $𝐮^{𝖳} 𝐴^{𝖳} 𝐯 =$ $𝐮^{𝖳} 𝐴 𝐯$ . So the imaginary part vanishes, and $𝐱^{*} 𝐴 𝐱 =$ $𝐮^{𝖳} 𝐴 𝐮 +$ $𝐯^{𝖳} 𝐴 𝐯 \geq$ $0$ .

For the last part, let $𝐵 = (\begin{matrix} 1 & - 1 \\ 1 & 1 \end{matrix})$ . Then $𝐱^{𝖳} 𝐵 𝐱 = {‖ 𝐱 ‖}^{2} \geq 0$ for $𝐱 \in ℝ^{2}$ (Non-example 8.3.3(b)). For $𝐱 = (1, 𝑖)$ , $𝐵 𝐱 = (1 - 𝑖, 1 + 𝑖)$ and $𝐱^{*} 𝐵 𝐱 =$ $(1 - 𝑖) -$ $𝑖 (1 + 𝑖) =$ $2 -$ $2 𝑖$ , which is not even real. The computation above shows exactly what was lost: without symmetry, the imaginary part $𝐮^{𝖳} 𝐴 𝐯 - 𝐯^{𝖳} 𝐴 𝐮$ need not vanish.

Exercise 8.3.31.standard Let $𝐴 =$ $(\begin{matrix} 𝑎 & 𝑏 \\ 𝑏 & 𝑐 \end{matrix}) \in$ $𝔽^{2 \times 2}$ with $𝑎, 𝑐 \in ℝ$ .

(a)Show that $𝐴$ is positive if and only if $𝑎 \geq 0$ , $𝑐 \geq 0$ , and $𝑎 𝑐 - {| 𝑏 |}^{2} \geq 0$ .
(b)Show that $𝐴$ is positive definite if and only if $𝑎 > 0$ and $𝑎 𝑐 - {| 𝑏 |}^{2} > 0$ .
(c)Show by an example that $𝑎 \geq 0$ and $det 𝐴 \geq 0$ do not imply that $𝐴$ is positive.

Solution (via eigenvalues).

Uses. Theorem 8.3.5 (Characterizations of positive operators), Proposition 8.3.15 (Characterizations of positive definite operators), Theorem 7.5.22 (Trace and determinant from eigenvalues)

Hint 1.

𝐴

is Hermitian, so its eigenvalues

𝜆_{1}, 𝜆_{2}

are real, with sum

𝑎 + 𝑐

and product

𝑎 𝑐 - {| 𝑏 |}^{2}

Hint 2. Two real numbers are both

\geq 0

if and only if their sum and their product are

\geq 0

. Then translate “

𝑎 + 𝑐 \geq 0

and

𝑎 𝑐 \geq {| 𝑏 |}^{2}

” into the stated conditions.

Sketch. (a) Positive iff

𝜆_{1} + 𝜆_{2} = 𝑎 + 𝑐 \geq 0

and

𝜆_{1} 𝜆_{2} =

𝑎 𝑐 -

{| 𝑏 |}^{2} \geq

0

. If

𝑎 𝑐 \geq {| 𝑏 |}^{2} \geq 0

, then

𝑎

and

𝑐

do not have opposite strict signs, so with

𝑎 + 𝑐 \geq 0

both are

\geq 0

. (b) Both eigenvalues

> 0

iff

𝑎 𝑐 - {| 𝑏 |}^{2} > 0

and

𝑎 + 𝑐 > 0

; given the first,

𝑎 > 0

iff

𝑐 > 0

iff

𝑎 + 𝑐 > 0

. (c)

diag (0, - 1)

$𝐴$ is Hermitian, so it has real eigenvalues $𝜆_{1}, 𝜆_{2}$ (counted with multiplicity), and by Theorem 7.5.22, $𝜆_{1} + 𝜆_{2} = 𝑎 + 𝑐$ and $𝜆_{1} 𝜆_{2} = 𝑎 𝑐 - {| 𝑏 |}^{2}$ .

(a) By Theorem 8.3.5, $𝐴$ is positive if and only if $𝜆_{1}, 𝜆_{2} \geq 0$ . For real numbers this holds if and only if $𝜆_{1} + 𝜆_{2} \geq 0$ and $𝜆_{1} 𝜆_{2} \geq 0$ : if the product is $\geq 0$ , the two numbers do not have opposite strict signs, and then a nonnegative sum rules out both being negative. So $𝐴$ is positive if and only if $𝑎 + 𝑐 \geq 0$ and $𝑎 𝑐 - {| 𝑏 |}^{2} \geq 0$ . These two conditions are equivalent to the three in (a). If $𝑎, 𝑐 \geq 0$ , then $𝑎 + 𝑐 \geq 0$ . Conversely, if $𝑎 𝑐 \geq {| 𝑏 |}^{2} \geq 0$ and, say, $𝑎 < 0$ , then $𝑐 \leq 0$ and $𝑎 + 𝑐 < 0$ ; so $𝑎 + 𝑐 \geq 0$ forces $𝑎 \geq 0$ , and likewise $𝑐 \geq 0$ .

(b) By Proposition 8.3.15, $𝐴$ is positive definite if and only if $𝜆_{1}, 𝜆_{2} > 0$ , if and only if $𝜆_{1} 𝜆_{2} > 0$ and $𝜆_{1} + 𝜆_{2} > 0$ , that is, $𝑎 𝑐 - {| 𝑏 |}^{2} > 0$ and $𝑎 + 𝑐 > 0$ . If $𝑎 𝑐 > {| 𝑏 |}^{2} \geq 0$ , then $𝑎$ and $𝑐$ are nonzero with the same sign, so $𝑎 + 𝑐 > 0$ if and only if $𝑎 > 0$ .

(c) $𝐴 = diag (0, - 1)$ has $𝑎 = 0 \geq 0$ and $det 𝐴 = 0 \geq 0$ , but $𝐞_{2}^{𝖳} 𝐴 𝐞_{2} =$ $- 1 <$ $0$ . The condition $𝑐 \geq 0$ in (a) cannot be dropped. In (b), by contrast, $𝑎 > 0$ and $det 𝐴 > 0$ suffice, which is the $2 \times 2$ case of Sylvester’s criterion (Section 12.3).

Solution (by completing the square).

Uses. Definition 8.3.1 (Positive operator), Definition 8.3.14 (Positive definite)

Hint 1. If

𝑎 > 0

, write

𝐱^{*} 𝐴 𝐱

𝑎 {| 𝑥_{1} + (𝑏 / 𝑎) 𝑥_{2} |}^{2} +

(det 𝐴 / 𝑎) {| 𝑥_{2} |}^{2}

Hint 2. If

𝑎 = 0

, positivity forces

𝑏 = 0

: choose

𝑥_{2} = 1

and

𝑥_{1} = - 𝑡 𝑏

with

𝑡 > 0

large.

Sketch. For

𝑎 > 0

𝐱^{*} 𝐴 𝐱 =

𝑎 {| 𝑥_{1} + (𝑏 / 𝑎) 𝑥_{2} |}^{2} +

((𝑎 𝑐 - {| 𝑏 |}^{2}) / 𝑎) {| 𝑥_{2} |}^{2}

, which is

\geq 0

for all

𝐱

iff

𝑎 𝑐 - {| 𝑏 |}^{2} \geq 0

, and

> 0

for

𝐱 \neq 𝟎

iff

𝑎 𝑐 - {| 𝑏 |}^{2} > 0

(test

𝐱 = (- 𝑏 / 𝑎, 1)

). For

𝑎 = 0

𝐱^{*} 𝐴 𝐱 =

2 Re (𝑥_{1} 𝑏 𝑥_{2}) +

𝑐 {| 𝑥_{2} |}^{2}

\geq 0

for all

𝐱

iff

𝑏 = 0

and

𝑐 \geq 0

, and never positive definite. For

𝑎 < 0

𝐞_{1}

gives

𝑎 < 0

Expanding, $𝐱^{*} 𝐴 𝐱 =$ $𝑎 {| 𝑥_{1} |}^{2} +$ $2 Re (𝑥_{1} 𝑏 𝑥_{2}) +$ $𝑐 {| 𝑥_{2} |}^{2}$ . We split into cases by the sign of $𝑎$ .

If $𝑎 < 0$ , then $𝐞_{1}^{*} 𝐴 𝐞_{1} = 𝑎 < 0$ , so $𝐴$ is not positive, and the conditions in (a) and (b) fail.

If $𝑎 > 0$ , then expanding $𝑎 {| 𝑥_{1} + (𝑏 / 𝑎) 𝑥_{2} |}^{2} =$ $𝑎 {| 𝑥_{1} |}^{2} +$ $2 Re (𝑥_{1} 𝑏 𝑥_{2}) +$ $({| 𝑏 |}^{2} / 𝑎) {| 𝑥_{2} |}^{2}$ gives

𝐱^{*} 𝐴 𝐱 = 𝑎 {| 𝑥_{1} + \frac{𝑏}{𝑎} 𝑥_{2} |}^{2} + \frac{𝑎 𝑐 - {| 𝑏 |}^{2}}{𝑎} {| 𝑥_{2} |}^{2} .

If $𝑎 𝑐 - {| 𝑏 |}^{2} \geq 0$ , this is $\geq 0$ ; and if $𝑎 𝑐 - {| 𝑏 |}^{2} > 0$ , it is $0$ only when $𝑥_{2} = 0$ and $𝑥_{1} + (𝑏 / 𝑎) 𝑥_{2} = 0$ , that is, $𝐱 = 𝟎$ . Conversely, $𝐱 = (- 𝑏 / 𝑎, 1) \neq 𝟎$ gives the value $(𝑎 𝑐 - {| 𝑏 |}^{2}) / 𝑎$ , which must be $\geq 0$ if $𝐴$ is positive and $> 0$ if $𝐴$ is positive definite. Also $𝑐 \geq 0$ follows from $𝑎 𝑐 \geq {| 𝑏 |}^{2} \geq 0$ .

If $𝑎 = 0$ , then $𝐴$ is not positive definite ( $𝐞_{1}^{*} 𝐴 𝐞_{1} = 0$ ), and the condition in (b) fails. If $𝑏 \neq 0$ , take $𝑥_{2} = 1$ and $𝑥_{1} = - 𝑡 𝑏$ with $𝑡 > 0$ : the value is $- 2 𝑡 {| 𝑏 |}^{2} + 𝑐$ , which is negative for large $𝑡$ ; and indeed $𝑎 𝑐 -$ ${| 𝑏 |}^{2} =$ $- {| 𝑏 |}^{2} <$ $0$ . If $𝑏 = 0$ , then $𝐱^{*} 𝐴 𝐱 = 𝑐 {| 𝑥_{2} |}^{2}$ , which is $\geq 0$ for all $𝐱$ if and only if $𝑐 \geq 0$ . In every case $𝐴$ is positive exactly when $𝑎 \geq 0$ , $𝑐 \geq 0$ , and $𝑎 𝑐 - {| 𝑏 |}^{2} \geq 0$ , which proves (a) and (b). For (c), see the first solution.

Exercise 8.3.32.standard Let $𝑇 \in L (𝑉)$ be positive, let $𝒪︀ = 𝐞_{1}, \dots, 𝐞_{𝑛}$ be an orthonormal basis of $𝑉$ , and let $𝐴 = {[𝑇]}_{𝒪︀}$ .

(a)Show that ${| 𝑎_{𝑖 𝑗} |}^{2} \leq$ $𝑎_{𝑖 𝑖} 𝑎_{𝑗 𝑗}$ for all $𝑖, 𝑗$ .
(b)Deduce that if $𝑎_{𝑖 𝑖} = 0$ , then row $𝑖$ and column $𝑖$ of $𝐴$ are zero.
(c)Deduce that the largest of the numbers $| 𝑎_{𝑖 𝑗} |$ occurs on the diagonal.

Solution (via a Gram matrix).

Uses. Theorem 8.3.5 (Characterizations of positive operators), Theorem 6.2.1 (Cauchy–Schwarz inequality), Theorem 8.3.9 (The positive square root)

Techniques. Gram matrices (Technique 6.3.31), Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1. By the proof of Theorem 8.3.5,

𝐴

is the Gram matrix of

\sqrt{𝑇} 𝐞_{1}, \dots, \sqrt{𝑇} 𝐞_{𝑛}

Hint 2. Apply Cauchy–Schwarz to

𝑎_{𝑖 𝑗} =

⟨ \sqrt{𝑇} 𝐞_{𝑗}, \sqrt{𝑇} 𝐞_{𝑖} ⟩

Sketch.

𝑎_{𝑖 𝑗} =

⟨ 𝐰_{𝑗}, 𝐰_{𝑖} ⟩

with

𝐰_{𝑖} = \sqrt{𝑇} 𝐞_{𝑖}

, so

𝑎_{𝑖 𝑖} = {‖ 𝐰_{𝑖} ‖}^{2}

and Cauchy–Schwarz gives (a). (b)

{| 𝑎_{𝑖 𝑗} |}^{2} \leq 0

, and

𝑎_{𝑗 𝑖} = 𝑎_{𝑖 𝑗}

. (c)

| 𝑎_{𝑖 𝑗} | \leq

\sqrt{𝑎_{𝑖 𝑖} 𝑎_{𝑗 𝑗}} \leq

max (𝑎_{𝑖 𝑖}, 𝑎_{𝑗 𝑗})

(a) Let $𝐰_{𝑖} = \sqrt{𝑇} 𝐞_{𝑖}$ . Since $\sqrt{𝑇}$ is self-adjoint, $𝑎_{𝑖 𝑗} =$ $⟨ 𝑇 𝐞_{𝑗}, 𝐞_{𝑖} ⟩ =$ $⟨ \sqrt{𝑇} 𝐞_{𝑗}, \sqrt{𝑇} 𝐞_{𝑖} ⟩ =$ $⟨ 𝐰_{𝑗}, 𝐰_{𝑖} ⟩$ , and in particular $𝑎_{𝑖 𝑖} = {‖ 𝐰_{𝑖} ‖}^{2}$ . By Cauchy–Schwarz (Theorem 6.2.1),

{| 𝑎_{𝑖 𝑗} |}^{2} = {| ⟨ 𝐰_{𝑗}, 𝐰_{𝑖} ⟩ |}^{2} \leq {‖ 𝐰_{𝑗} ‖}^{2} {‖ 𝐰_{𝑖} ‖}^{2} = 𝑎_{𝑗 𝑗} 𝑎_{𝑖 𝑖} .

(b) If $𝑎_{𝑖 𝑖} = 0$ , then ${| 𝑎_{𝑖 𝑗} |}^{2} \leq 0$ , so $𝑎_{𝑖 𝑗} = 0$ for every $𝑗$ : row $𝑖$ is zero. Since $𝐴$ is Hermitian, $𝑎_{𝑗 𝑖} = 𝑎_{𝑖 𝑗} = 0$ , so column $𝑖$ is zero too.

(c) The diagonal entries are nonnegative reals. For all $𝑖, 𝑗$ , by (a),

| 𝑎_{𝑖 𝑗} | \leq \sqrt{𝑎_{𝑖 𝑖} 𝑎_{𝑗 𝑗}} \leq max (𝑎_{𝑖 𝑖}, 𝑎_{𝑗 𝑗}) \leq max_{𝑘} 𝑎_{𝑘 𝑘},

and ${max}_{𝑘} 𝑎_{𝑘 𝑘}$ is itself one of the numbers $| 𝑎_{𝑖 𝑗} |$ .

Solution (via $2 \times 2$ submatrices).

Uses. Theorem 8.3.5 (Characterizations of positive operators), Exercise 8.3.31, Proposition 6.3.18 (Coordinates in an orthonormal basis)

Techniques. Gram matrices (Technique 6.3.31), Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1. Restrict the form

⟨ 𝑇 𝐯, 𝐯 ⟩

to vectors

𝐯 = 𝑥 𝐞_{𝑖} + 𝑦 𝐞_{𝑗}

with

𝑖 \neq 𝑗

Hint 2. The matrix

(\begin{matrix} 𝑎_{𝑖 𝑖} & 𝑎_{𝑖 𝑗} \\ 𝑎_{𝑗 𝑖} & 𝑎_{𝑗 𝑗} \end{matrix})

is then positive, and Exercise 8.3.31 applies.

Sketch. For

𝑖 \neq 𝑗

and

𝐯 = 𝑥 𝐞_{𝑖} + 𝑦 𝐞_{𝑗}

⟨ 𝑇 𝐯, 𝐯 ⟩ =

{(𝑥, 𝑦)}^{*} 𝐴_{𝑖 𝑗} (𝑥, 𝑦)

with

𝐴_{𝑖 𝑗} =

(\begin{matrix} 𝑎_{𝑖 𝑖} & 𝑎_{𝑖 𝑗} \\ 𝑎_{𝑗 𝑖} & 𝑎_{𝑗 𝑗} \end{matrix})

, which is therefore positive (and Hermitian). By Exercise 8.3.31,

𝑎_{𝑖 𝑖} 𝑎_{𝑗 𝑗} -

{| 𝑎_{𝑖 𝑗} |}^{2} \geq

0

. For

𝑖 = 𝑗

the inequality is

𝑎_{𝑖 𝑖}^{2} \leq 𝑎_{𝑖 𝑖}^{2}

(a) For $𝑖 = 𝑗$ there is nothing to prove. Let $𝑖 \neq 𝑗$ , and for $(𝑥, 𝑦) \in 𝔽^{2}$ let $𝐯 = 𝑥 𝐞_{𝑖} + 𝑦 𝐞_{𝑗}$ . Since $𝑎_{𝑘 𝑙} =$ $⟨ 𝑇 𝐞_{𝑙}, 𝐞_{𝑘} ⟩$ ,

⟨ 𝑇 𝐯, 𝐯 ⟩ = 𝑥 𝑎_{𝑖 𝑖} 𝑥 + 𝑥 𝑎_{𝑖 𝑗} 𝑦 + 𝑦 𝑎_{𝑗 𝑖} 𝑥 + 𝑦 𝑎_{𝑗 𝑗} 𝑦 = {(\begin{matrix} 𝑥 \\ 𝑦 \end{matrix})}^{*} (\begin{matrix} 𝑎_{𝑖 𝑖} & 𝑎_{𝑖 𝑗} \\ 𝑎_{𝑗 𝑖} & 𝑎_{𝑗 𝑗} \end{matrix}) (\begin{matrix} 𝑥 \\ 𝑦 \end{matrix}) .

The $2 \times 2$ matrix is Hermitian (as $𝐴$ is) and the left side is $\geq 0$ , so it is positive, and Exercise 8.3.31(a) gives $𝑎_{𝑖 𝑖} 𝑎_{𝑗 𝑗} -$ ${| 𝑎_{𝑖 𝑗} |}^{2} \geq$ $0$ . Parts (b) and (c) follow as in the first solution.

Exercise 8.3.33.standard Let $𝑇 \in L (𝑉)$ be positive.

(a)Show that ${| ⟨ 𝑇 𝐯, 𝐰 ⟩ |}^{2} \leq$ $⟨ 𝑇 𝐯, 𝐯 ⟩ ⟨ 𝑇 𝐰, 𝐰 ⟩$ for all $𝐯, 𝐰 \in 𝑉$ .
(b)Use (a) to give a third proof of Lemma 8.3.8.
(c)Show that $ker \sqrt{𝑇} = ker 𝑇$ and $im \sqrt{𝑇} = im 𝑇$ .

Solution.

Uses. Theorem 8.3.9 (The positive square root), Theorem 6.2.1 (Cauchy–Schwarz inequality), Theorem 4.2.11 (Rank–nullity), Lemma 8.3.8 (Where the form vanishes)

Techniques. Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1. Write

⟨ 𝑇 𝐯, 𝐰 ⟩ =

⟨ \sqrt{𝑇} 𝐯, \sqrt{𝑇} 𝐰 ⟩

and apply Cauchy–Schwarz.

Hint 2. For (b), take

𝐰 = 𝑇 𝐯

in (a).

Hint 3. For (c),

⟨ 𝑇 𝐯, 𝐯 ⟩ = {‖ \sqrt{𝑇} 𝐯 ‖}^{2}

. For the images, one inclusion is clear and rank–nullity does the rest.

Sketch. (a)

⟨ 𝑇 𝐯, 𝐰 ⟩ =

⟨ \sqrt{𝑇} 𝐯, \sqrt{𝑇} 𝐰 ⟩

, and

{‖ \sqrt{𝑇} 𝐯 ‖}^{2} = ⟨ 𝑇 𝐯, 𝐯 ⟩

. (b) With

𝐰 = 𝑇 𝐯

{‖ 𝑇 𝐯 ‖}^{4} \leq

⟨ 𝑇 𝐯, 𝐯 ⟩ ⟨ 𝑇^{2} 𝐯, 𝑇 𝐯 ⟩

, which is

0

⟨ 𝑇 𝐯, 𝐯 ⟩ = 0

. (c)

\sqrt{𝑇} 𝐯 = 𝟎 \Rightarrow 𝑇 𝐯 = 𝟎

, and

𝑇 𝐯 = 𝟎 \Rightarrow {‖ \sqrt{𝑇} 𝐯 ‖}^{2} = 0

im 𝑇 =

im (\sqrt{𝑇} \sqrt{𝑇}) \subseteq

im \sqrt{𝑇}

, with equal dimensions.

Let $𝑅 = \sqrt{𝑇}$ , which is self-adjoint with $𝑅^{2} = 𝑇$ (Theorem 8.3.9). For all $𝐯, 𝐰$ , $⟨ 𝑇 𝐯, 𝐰 ⟩ =$ $⟨ 𝑅 𝐯, 𝑅 𝐰 ⟩$ , and in particular $⟨ 𝑇 𝐯, 𝐯 ⟩ = {‖ 𝑅 𝐯 ‖}^{2}$ .

(a) By Cauchy–Schwarz (Theorem 6.2.1),

{| ⟨ 𝑇 𝐯, 𝐰 ⟩ |}^{2} = {| ⟨ 𝑅 𝐯, 𝑅 𝐰 ⟩ |}^{2} \leq {‖ 𝑅 𝐯 ‖}^{2} {‖ 𝑅 𝐰 ‖}^{2} = ⟨ 𝑇 𝐯, 𝐯 ⟩ ⟨ 𝑇 𝐰, 𝐰 ⟩ .

(b) Suppose $⟨ 𝑇 𝐯, 𝐯 ⟩ = 0$ , and take $𝐰 = 𝑇 𝐯$ in (a):

{‖ 𝑇 𝐯 ‖}^{4} = {| ⟨ 𝑇 𝐯, 𝑇 𝐯 ⟩ |}^{2} \leq ⟨ 𝑇 𝐯, 𝐯 ⟩ ⟨ 𝑇^{2} 𝐯, 𝑇 𝐯 ⟩ = 0 .

So $𝑇 𝐯 = 𝟎$ . (This proof uses $\sqrt{𝑇}$ through (a); one can also prove (a) without it, by applying the argument of Theorem 6.2.1 to the form $(𝐯, 𝐰) \mapsto ⟨ 𝑇 𝐯, 𝐰 ⟩$ .)

(c) If $𝑅 𝐯 = 𝟎$ , then $𝑇 𝐯 = 𝑅 (𝑅 𝐯) = 𝟎$ . If $𝑇 𝐯 = 𝟎$ , then ${‖ 𝑅 𝐯 ‖}^{2} =$ $⟨ 𝑇 𝐯, 𝐯 ⟩ =$ $0$ , so $𝑅 𝐯 = 𝟎$ . Hence $ker 𝑅 = ker 𝑇$ . Next, $im 𝑇 =$ $im (𝑅 𝑅) \subseteq$ $im 𝑅$ , and by rank–nullity (Theorem 4.2.11) $dim im 𝑇 =$ $dim 𝑉 -$ $dim ker 𝑇 =$ $dim 𝑉 -$ $dim ker 𝑅 =$ $dim im 𝑅$ . A subspace of $im 𝑅$ with the same dimension is all of $im 𝑅$ (Proposition 4.1.21), so $im 𝑇 = im 𝑅$ .

Exercise 8.3.34.standard Let $𝑆, 𝑇 \in L (𝑉)$ be positive.

(a)Show that $tr 𝑇 \geq 0$ and $det 𝑇 \geq 0$ , and that $tr 𝑇 = 0$ only if $𝑇 = 0$ .
(b)Show that $tr (𝑆 𝑇) \geq 0$ .
(c)Show that $tr (𝑆 𝑇) = 0$ if and only if $𝑆 𝑇 = 0$ .

Solution.

Uses. Theorem 8.3.5 (Characterizations of positive operators), Proposition 7.5.25 (Cyclicity of the trace for linear maps), Theorem 8.3.9 (The positive square root), Exercise 8.3.28, Proposition 6.6.18 (Properties of the adjoint)

Techniques. Sandwich between square roots (Technique 8.3.19), Cyclicity of trace (Technique 2.4.29)

Hint 1. For (a), compute

tr

and

det

in an orthonormal eigenbasis.

Hint 2. For (b),

tr (𝑆 𝑇) =

tr (\sqrt{𝑆} (\sqrt{𝑆} 𝑇)) =

tr (\sqrt{𝑆} 𝑇 \sqrt{𝑆})

, and

\sqrt{𝑆} 𝑇 \sqrt{𝑆}

is positive.

Hint 3. For (c), if

\sqrt{𝑆} 𝑇 \sqrt{𝑆} = 0

, then

{(\sqrt{𝑇} \sqrt{𝑆})}^{*} (\sqrt{𝑇} \sqrt{𝑆}) = 0

. Deduce

𝑇 𝑆 = 0

and take adjoints.

Sketch. (a)

{[𝑇]}_{𝒪︀} =

diag (𝑑_{𝑖})

with

𝑑_{𝑖} \geq 0

, so

tr 𝑇 = \sum 𝑑_{𝑖} \geq 0

det 𝑇 = \prod 𝑑_{𝑖} \geq 0

, and

tr 𝑇 = 0

forces all

𝑑_{𝑖} = 0

. (b) By cyclicity

tr (𝑆 𝑇) = tr (𝐶)

with

𝐶 = \sqrt{𝑆} 𝑇 \sqrt{𝑆}

positive. (c) If

tr 𝐶 = 0

, then

𝐶 = 0

by (a). Since

𝐶 = 𝑀^{*} 𝑀

with

𝑀 = \sqrt{𝑇} \sqrt{𝑆}

{‖ 𝑀 𝐯 ‖}^{2} =

⟨ 𝐶 𝐯, 𝐯 ⟩ =

0

, so

𝑀 = 0

𝑇 𝑆 = \sqrt{𝑇} 𝑀 \sqrt{𝑆} = 0

, and

𝑆 𝑇 = {(𝑇 𝑆)}^{*} = 0

(a) By Theorem 8.3.5(III), ${[𝑇]}_{𝒪︀} =$ $diag (𝑑_{1}, \dots, 𝑑_{𝑛})$ with every $𝑑_{𝑖} \geq 0$ for some orthonormal basis $𝒪︀$ . So $tr 𝑇 = \sum_{𝑖} 𝑑_{𝑖} \geq 0$ and $det 𝑇 = \prod_{𝑖} 𝑑_{𝑖} \geq 0$ . If $tr 𝑇 = 0$ , then every $𝑑_{𝑖} = 0$ , so ${[𝑇]}_{𝒪︀} = 0$ and $𝑇 = 0$ .

(b) By Proposition 7.5.25, $tr (\sqrt{𝑆} (\sqrt{𝑆} 𝑇)) =$ $tr ((\sqrt{𝑆} 𝑇) \sqrt{𝑆})$ , so $tr (𝑆 𝑇) = tr (𝐶)$ with $𝐶 = \sqrt{𝑆} 𝑇 \sqrt{𝑆}$ . Since $\sqrt{𝑆}$ is self-adjoint, $𝐶 = {(\sqrt{𝑆})}^{*} 𝑇 \sqrt{𝑆}$ is positive by Exercise 8.3.28(b). By (a), $tr (𝑆 𝑇) = tr 𝐶 \geq 0$ .

(c) If $𝑆 𝑇 = 0$ , then $tr (𝑆 𝑇) = 0$ . Conversely, suppose $tr (𝑆 𝑇) = 0$ . Then $tr 𝐶 = 0$ , so $𝐶 = 0$ by (a). Let $𝑀 = \sqrt{𝑇} \sqrt{𝑆}$ . Since $\sqrt{𝑆}$ and $\sqrt{𝑇}$ are self-adjoint, $𝑀^{*} 𝑀 =$ $\sqrt{𝑆} \sqrt{𝑇} \sqrt{𝑇} \sqrt{𝑆} =$ $𝐶 =$ $0$ , so ${‖ 𝑀 𝐯 ‖}^{2} =$ $⟨ 𝑀^{*} 𝑀 𝐯, 𝐯 ⟩ =$ $0$ for every $𝐯$ , and $𝑀 = 0$ . Then $𝑇 𝑆 =$ $\sqrt{𝑇} (\sqrt{𝑇} \sqrt{𝑆}) \sqrt{𝑆} =$ $\sqrt{𝑇} 𝑀 \sqrt{𝑆} =$ $0$ , and $𝑆 𝑇 =$ $𝑆^{*} 𝑇^{*} =$ ${(𝑇 𝑆)}^{*} =$ $0$ by Proposition 6.6.18.

Exercise 8.3.35.standard

(a)Show that the $𝑛 \times 𝑛$ Hilbert matrix $𝐻_{𝑛}$ , with entries $ℎ_{𝑖 𝑗} = 1 / (𝑖 + 𝑗 - 1)$ , is positive definite. Deduce that $det 𝐻_{𝑛} > 0$ .
(b)Show that the $𝑛 \times 𝑛$ matrix $𝐾_{𝑛}$ with entries $𝑘_{𝑖 𝑗} = 1 / (𝑖 + 𝑗)$ is positive definite.

Solution.

Uses. Proposition 8.3.15 (Characterizations of positive definite operators), Proposition 6.1.13 (Integral inner products), Definition 6.3.28 (Gram matrix), Proposition 7.1.22 (Counting roots), Theorem 7.5.22 (Trace and determinant from eigenvalues)

Techniques. Gram matrices (Technique 6.3.31)

Hint 1. Look for functions

𝑓_{1}, \dots, 𝑓_{𝑛}

[0, 1]

with

\int_{0}^{1} 𝑓_{𝑖} (𝑡) 𝑓_{𝑗} (𝑡) d 𝑡 =

1 / (𝑖 + 𝑗 - 1)

Hint 2.

𝑓_{𝑖} (𝑡) = 𝑡^{𝑖 - 1}

works for

𝐻_{𝑛}

. For

𝐾_{𝑛}

you need

\int_{0}^{1} 𝑓_{𝑖} 𝑓_{𝑗} =

1 / (𝑖 + 𝑗)

; try

𝑓_{𝑖} (𝑡) = 𝑡^{𝑖 - 1 / 2}

Hint 3. A Gram matrix of a linearly independent list is positive definite (Proposition 8.3.15). Independence comes down to a polynomial with infinitely many roots.

Sketch. (a)

𝐻_{𝑛}

is the Gram matrix of

1, 𝑡, \dots, 𝑡^{𝑛 - 1}

𝐶 [0, 1]

with

⟨ 𝑓, 𝑔 ⟩ = \int_{0}^{1} 𝑓 𝑔

, an independent list, so it is positive definite and

det 𝐻_{𝑛}

is a product of positive eigenvalues. (b)

𝐾_{𝑛}

is the Gram matrix of

𝑡^{1 / 2},

𝑡^{3 / 2},

\dots,

𝑡^{𝑛 - 1 / 2}

; if

\sum 𝑐_{𝑖} 𝑡^{𝑖 - 1 / 2} = 0

[0, 1]

, multiplying by

𝑡^{1 / 2}

gives a polynomial vanishing on

[0, 1]

Give $𝐶 [0, 1]$ the inner product $⟨ 𝑓, 𝑔 ⟩ =$ $\int_{0}^{1} 𝑓 (𝑡) 𝑔 (𝑡) d 𝑡$ (Proposition 6.1.13).

(a) Let $𝑓_{𝑖} (𝑡) = 𝑡^{𝑖 - 1}$ for $𝑖 = 1, \dots, 𝑛$ . Then

⟨ 𝑓_{𝑗}, 𝑓_{𝑖} ⟩ = \int_{0}^{1} 𝑡^{𝑖 + 𝑗 - 2} d 𝑡 = \frac{1}{𝑖 + 𝑗 - 1} = ℎ_{𝑖 𝑗},

so $𝐻_{𝑛}$ is the Gram matrix of $𝑓_{1}, \dots, 𝑓_{𝑛}$ (Definition 6.3.28). This list is linearly independent: if $\sum_{𝑖} 𝑐_{𝑖} 𝑡^{𝑖 - 1} = 0$ for every $𝑡 \in [0, 1]$ , the polynomial $\sum_{𝑖} 𝑐_{𝑖} 𝑥^{𝑖 - 1}$ has infinitely many roots, so it is zero (Proposition 7.1.22) and every $𝑐_{𝑖} = 0$ . By Proposition 8.3.15, $𝐻_{𝑛}$ is positive definite. Its eigenvalues are positive and $det 𝐻_{𝑛}$ is their product (Theorem 7.5.22), so $det 𝐻_{𝑛} > 0$ .

(b) Let $𝑔_{𝑖} (𝑡) = 𝑡^{𝑖 - 1 / 2}$ for $𝑖 = 1, \dots, 𝑛$ , continuous on $[0, 1]$ since $𝑖 - 1 / 2 > 0$ . Then $⟨ 𝑔_{𝑗}, 𝑔_{𝑖} ⟩ =$ $\int_{0}^{1} 𝑡^{𝑖 + 𝑗 - 1} d 𝑡 =$ $1 / (𝑖 + 𝑗) =$ $𝑘_{𝑖 𝑗}$ . If $\sum_{𝑖} 𝑐_{𝑖} 𝑡^{𝑖 - 1 / 2} = 0$ for every $𝑡 \in [0, 1]$ , then multiplying by $𝑡^{1 / 2}$ gives $\sum_{𝑖} 𝑐_{𝑖} 𝑡^{𝑖} = 0$ on $[0, 1]$ , so as in (a) every $𝑐_{𝑖} = 0$ . Hence $𝐾_{𝑛}$ is the Gram matrix of an independent list and is positive definite.

No eigenvalue had to be computed. The eigenvalues of $𝐻_{𝑛}$ are in fact tiny (the smallest is below $10^{- 5}$ for $𝑛 = 5$ ), so a numerical test of positivity would be delicate, while the Gram description settles the question exactly.

Exercise 8.3.36.standard Let $𝐴 \in 𝔽^{2 \times 2}$ be positive and nonzero, and let $𝛿 = \sqrt{det 𝐴}$ .

(a)Show that
$\sqrt{𝐴} = \frac{𝐴 + 𝛿 𝐼_{2}}{\sqrt{tr 𝐴 + 2 𝛿}} .$
(b)Use (a) to compute $\sqrt{𝐴}$ for $𝐴 = (\begin{matrix} 2 & 1 \\ 1 & 2 \end{matrix})$ and for $𝐴 = (\begin{matrix} 5 & 4 𝑖 \\ - 4 𝑖 & 5 \end{matrix})$ .

Solution.

Uses. Theorem 8.3.9 (The positive square root), Theorem 8.3.5 (Characterizations of positive operators), Theorem 7.5.22 (Trace and determinant from eigenvalues), Exercise 8.3.31

Hint 1. Every

𝑅 \in 𝔽^{2 \times 2}

satisfies

𝑅^{2} -

(tr 𝑅) 𝑅 +

(det 𝑅) 𝐼_{2} =

0

. Check it by multiplying out, and apply it to

𝑅 = \sqrt{𝐴}

Hint 2. If

𝑅

has eigenvalues

𝜇_{1}, 𝜇_{2} \geq 0

, then

𝐴 = 𝑅^{2}

has eigenvalues

𝜇_{1}^{2}, 𝜇_{2}^{2}

. Express

det 𝑅

and

{(tr 𝑅)}^{2}

through

det 𝐴

and

tr 𝐴

Sketch. With

𝑅 = \sqrt{𝐴}

, eigenvalues

𝜇_{1}, 𝜇_{2} \geq 0

det 𝑅 = 𝜇_{1} 𝜇_{2} = 𝛿

and

{(tr 𝑅)}^{2} =

𝜇_{1}^{2} +

𝜇_{2}^{2} +

2 𝜇_{1} 𝜇_{2} =

tr 𝐴 +

2 𝛿

, so

tr 𝑅 =

𝑠 =

\sqrt{tr 𝐴 + 2 𝛿} >

0

(as

𝑅 \neq 0

). Then

𝐴 - 𝑠 𝑅 + 𝛿 𝐼_{2} = 0

gives

𝑅 = (𝐴 + 𝛿 𝐼_{2}) / 𝑠

. (b)

𝑠 = 1 + \sqrt{3}

and

\sqrt{𝐴} =

\frac{1}{2} (\begin{matrix} 1 + \sqrt{3} & \sqrt{3} - 1 \\ \sqrt{3} - 1 & 1 + \sqrt{3} \end{matrix})

; then

𝛿 = 3

𝑠 = 4

\sqrt{𝐴} = (\begin{matrix} 2 & 𝑖 \\ - 𝑖 & 2 \end{matrix})

(a) For $𝑅 = (\begin{matrix} 𝑝 & 𝑞 \\ 𝑟 & 𝑠^{'} \end{matrix})$ , multiplying out gives

𝑅^{2} - (𝑝 + 𝑠^{'}) 𝑅 = (\begin{matrix} 𝑝^{2} + 𝑞 𝑟 - 𝑝^{2} - 𝑝 𝑠^{'} & 𝑝 𝑞 + 𝑞 𝑠^{'} - 𝑝 𝑞 - 𝑞 𝑠^{'} \\ 𝑟 𝑝 + 𝑠^{'} 𝑟 - 𝑝 𝑟 - 𝑠^{'} 𝑟 & 𝑟 𝑞 + 𝑠^{' 2} - 𝑝 𝑠^{'} - 𝑠^{' 2} \end{matrix}) = - (𝑝 𝑠^{'} - 𝑞 𝑟) 𝐼_{2},

that is, $𝑅^{2} -$ $(tr 𝑅) 𝑅 +$ $(det 𝑅) 𝐼_{2} =$ $0$ . Let $𝑅 = \sqrt{𝐴}$ . By Theorem 8.3.5(III), ${[𝑅]}_{𝒪︀} =$ $diag (𝜇_{1}, 𝜇_{2})$ for an orthonormal basis $𝒪︀$ with $𝜇_{1}, 𝜇_{2} \geq 0$ , and then ${[𝐴]}_{𝒪︀} =$ $diag (𝜇_{1}^{2}, 𝜇_{2}^{2})$ . By Theorem 7.5.22,

det 𝑅 = 𝜇_{1} 𝜇_{2} = \sqrt{𝜇_{1}^{2} 𝜇_{2}^{2}} = \sqrt{det 𝐴} = 𝛿, {(tr 𝑅)}^{2} = {(𝜇_{1} + 𝜇_{2})}^{2} = 𝜇_{1}^{2} + 𝜇_{2}^{2} + 2 𝜇_{1} 𝜇_{2} = tr 𝐴 + 2 𝛿 .

Since $tr 𝑅 = 𝜇_{1} + 𝜇_{2} \geq 0$ , $tr 𝑅 = 𝑠$ with $𝑠 = \sqrt{tr 𝐴 + 2 𝛿}$ . If $𝑠 = 0$ , then $𝜇_{1} = 𝜇_{2} = 0$ , so $𝑅 = 0$ and $𝐴 = 0$ , which is excluded. So $𝑠 > 0$ , and the identity $𝑅^{2} - 𝑠 𝑅 + 𝛿 𝐼_{2} = 0$ with $𝑅^{2} = 𝐴$ gives $𝑅 = (𝐴 + 𝛿 𝐼_{2}) / 𝑠$ .

(b) For $𝐴 = (\begin{matrix} 2 & 1 \\ 1 & 2 \end{matrix})$ : $det 𝐴 = 3$ , $tr 𝐴 = 4$ , and $𝐴$ is positive (Exercise 8.3.31). So $𝛿 = \sqrt{3}$ and $𝑠 = \sqrt{4 + 2 \sqrt{3}} = 1 + \sqrt{3}$ , since ${(1 + \sqrt{3})}^{2} = 4 + 2 \sqrt{3}$ . Using $(2 + \sqrt{3}) / (1 + \sqrt{3}) =$ $(1 + \sqrt{3}) / 2$ and $1 / (1 + \sqrt{3}) = (\sqrt{3} - 1) / 2$ ,

\sqrt{𝐴} = \frac{1}{1 + \sqrt{3}} (\begin{matrix} 2 + \sqrt{3} & 1 \\ 1 & 2 + \sqrt{3} \end{matrix}) = \frac{1}{2} (\begin{matrix} 1 + \sqrt{3} & \sqrt{3} - 1 \\ \sqrt{3} - 1 & 1 + \sqrt{3} \end{matrix}) .

Check: the square of the right side is $\frac{1}{4} (\begin{matrix} {(1 + \sqrt{3})}^{2} + {(\sqrt{3} - 1)}^{2} & 2 (1 + \sqrt{3}) (\sqrt{3} - 1) \\ \dots & \dots \end{matrix}) =$ $\frac{1}{4} (\begin{matrix} 8 & 4 \\ 4 & 8 \end{matrix}) =$ $𝐴$ .

For $𝐴 = (\begin{matrix} 5 & 4 𝑖 \\ - 4 𝑖 & 5 \end{matrix})$ : $𝐴$ is Hermitian with $𝑎 = 5 > 0$ and $det 𝐴 =$ $25 -$ $(4 𝑖) (- 4 𝑖) =$ $25 -$ $16 =$ $9 >$ $0$ , so it is positive (even positive definite) by Exercise 8.3.31. Here $𝛿 = 3$ , $𝑠 = \sqrt{10 + 6} = 4$ , and

\sqrt{𝐴} = \frac{1}{4} (\begin{matrix} 8 & 4 𝑖 \\ - 4 𝑖 & 8 \end{matrix}) = (\begin{matrix} 2 & 𝑖 \\ - 𝑖 & 2 \end{matrix}) .

Check: ${(\begin{matrix} 2 & 𝑖 \\ - 𝑖 & 2 \end{matrix})}^{2} =$ $(\begin{matrix} 4 + 𝑖 (- 𝑖) & 2 𝑖 + 2 𝑖 \\ - 2 𝑖 - 2 𝑖 & (- 𝑖) 𝑖 + 4 \end{matrix}) =$ $(\begin{matrix} 5 & 4 𝑖 \\ - 4 𝑖 & 5 \end{matrix})$ .

Exercise 8.3.37.standard Let $𝑇 \in L (𝑉)$ be self-adjoint, with spectral decomposition $𝑇 = \sum_{𝑗} 𝜆_{𝑗} 𝑃_{𝑗}$ . Define $| 𝑇 | = \sqrt{𝑇^{2}}$ , which makes sense because $𝑇^{2} = 𝑇^{*} 𝑇$ is positive (Remark 8.3.13).

(a)Show that $| 𝑇 | =$ $\sum_{𝑗} | 𝜆_{𝑗} | 𝑃_{𝑗}$ .
(b)Show that $| 𝑇 | = 𝑇$ if and only if $𝑇$ is positive.
(c)Let $𝑇_{+} = (| 𝑇 | + 𝑇) / 2$ and $𝑇_{-} = (| 𝑇 | - 𝑇) / 2$ . Show that $𝑇_{+}$ and $𝑇_{-}$ are positive, $𝑇 = 𝑇_{+} - 𝑇_{-}$ , and $𝑇_{+} 𝑇_{-} = 0$ .

Solution.

Uses. Proposition 8.2.24 (Properties of $𝑓 (𝑇)$ ), Theorem 8.3.9 (The positive square root), Theorem 8.3.5 (Characterizations of positive operators)

Techniques. Apply a function to the eigenvalues (Technique 8.2.25)

Hint 1. Let

𝑓 (𝜆) = | 𝜆 |

. Show that

𝑓 (𝑇)

is positive and squares to

𝑇^{2}

, then use uniqueness of the positive square root.

Hint 2. For (b), if

𝑇

is positive, then

𝑇

itself is a positive square root of

𝑇^{2}

Hint 3. For (c),

𝑇_{+} =

\sum_{𝑗} max (𝜆_{𝑗}, 0) 𝑃_{𝑗}

and

𝑇_{-} =

\sum_{𝑗} max (- 𝜆_{𝑗}, 0) 𝑃_{𝑗}

Sketch. (a)

𝑓 (𝑇) =

\sum | 𝜆_{𝑗} | 𝑃_{𝑗}

is self-adjoint with eigenvalues

| 𝜆_{𝑗} | \geq 0

, so positive, and

{𝑓 (𝑇)}^{2} =

(𝑓^{2}) (𝑇) =

\sum 𝜆_{𝑗}^{2} 𝑃_{𝑗} =

𝑇^{2}

; by uniqueness it is

\sqrt{𝑇^{2}}

. (b)

| 𝑇 |

is positive; conversely a positive

𝑇

is a positive square root of

𝑇^{2}

. (c)

𝑇_{\pm} = 𝑔_{\pm} (𝑇)

with

𝑔_{+} (𝜆) = max (𝜆, 0)

𝑔_{-} (𝜆) =

max (- 𝜆, 0)

, which are

\geq 0

with

𝑔_{+} - 𝑔_{-} = 𝜆

and

𝑔_{+} 𝑔_{-} = 0

(a) Let $𝑓 (𝜆) = | 𝜆 |$ on the eigenvalues, which are real. By Proposition 8.2.24, $𝑓 (𝑇) =$ $\sum_{𝑗} | 𝜆_{𝑗} | 𝑃_{𝑗}$ is self-adjoint (its values are real) with nonnegative eigenvalues $| 𝜆_{𝑗} |$ , so it is positive by Theorem 8.3.5, and ${𝑓 (𝑇)}^{2} =$ $(𝑓^{2}) (𝑇) =$ $\sum_{𝑗} 𝜆_{𝑗}^{2} 𝑃_{𝑗} =$ $𝑇^{2}$ , because $𝑓^{2} (𝜆) = 𝜆^{2}$ and $𝑇^{2} = \sum_{𝑗} 𝜆_{𝑗}^{2} 𝑃_{𝑗}$ (Proposition 8.2.24(c) with $𝑔 (𝜆) = 𝜆$ ). By uniqueness in Theorem 8.3.9, $𝑓 (𝑇) = \sqrt{𝑇^{2}} = | 𝑇 |$ .

(b) $| 𝑇 |$ is positive, so if $| 𝑇 | = 𝑇$ , then $𝑇$ is positive. Conversely, if $𝑇$ is positive, then $𝑇$ is a positive operator whose square is $𝑇^{2}$ , so $𝑇 = \sqrt{𝑇^{2}} = | 𝑇 |$ by uniqueness.

(c) By (a), $𝑇_{+} =$ $\sum_{𝑗} 𝑔_{+} (𝜆_{𝑗}) 𝑃_{𝑗}$ and $𝑇_{-} =$ $\sum_{𝑗} 𝑔_{-} (𝜆_{𝑗}) 𝑃_{𝑗}$ , where $𝑔_{+} (𝜆) =$ $(| 𝜆 | + 𝜆) / 2 =$ $max (𝜆, 0)$ and $𝑔_{-} (𝜆) =$ $(| 𝜆 | - 𝜆) / 2 =$ $max (- 𝜆, 0)$ . Both functions are real and nonnegative, so $𝑇_{+}$ and $𝑇_{-}$ are positive (Proposition 8.2.24(d), (e), and Theorem 8.3.5). Clearly $𝑇_{+} - 𝑇_{-} = 𝑇$ . Finally $𝑔_{+} (𝜆) 𝑔_{-} (𝜆) = 0$ for every real $𝜆$ , since one of the two factors is $0$ , so $𝑇_{+} 𝑇_{-} =$ $(𝑔_{+} 𝑔_{-}) (𝑇) =$ $0$ by Proposition 8.2.24(c). This is the operator version of writing a real number as its positive part minus its negative part.

Exercise 8.3.38.standard Consider the following argument.

“Claim. A positive operator $𝑇$ has exactly one self-adjoint square root. Proof. Let $𝑅$ be self-adjoint with $𝑅^{2} = 𝑇$ . Then $𝑅$ commutes with $𝑇$ , so by Proposition 8.2.20(a) it maps each eigenspace $𝐸_{𝜆}$ of $𝑇$ into itself. On $𝐸_{𝜆}$ we have $𝑅^{2} = 𝑇 = 𝜆 id$ , so $𝑅 = \sqrt{𝜆} id$ on $𝐸_{𝜆}$ . Hence $𝑅 = \sum_{𝑗} \sqrt{𝜆_{𝑗}} 𝑃_{𝑗} = \sqrt{𝑇}$ .”

(a)Find the error, and show that the claim is false.
(b)Suppose $dim 𝑉 = 𝑛$ and $𝑇$ is positive with $𝑛$ distinct eigenvalues, $𝑘$ of which are nonzero. How many self-adjoint square roots does $𝑇$ have?

Solution.

Uses. Theorem 8.3.9 (The positive square root), Proposition 8.2.20 (Spectral projections are polynomials in $𝑇$ ), Proposition 8.2.24 (Properties of $𝑓 (𝑇)$ ), Proposition 8.1.1 (Eigenvalues and eigenvectors of self-adjoint operators)

Techniques. Finding counterexamples and minimal examples (Technique 1.4.18)

Hint 1. Is

𝑅^{2} = 𝜆 id

on a space of dimension

2

enough to conclude

𝑅 = \sqrt{𝜆} id

? Think of

𝑇 = {id}_{𝑉}

Hint 2. For (b), every eigenspace is a line, and on a line an operator is a scalar.

Sketch. (a) “

𝑅^{2} = 𝜆 id

𝐸_{𝜆}

, so

𝑅 = \sqrt{𝜆} id

” is wrong:

𝑅

may act as

- \sqrt{𝜆}

, or on a plane as any reflection scaled by

\sqrt{𝜆}

. Counterexamples:

- \sqrt{𝑇}

, and

diag (1, - 1)

for

𝑇 = 𝐼_{2}

. (b)

𝑅

preserves each eigenline

𝐸_{𝜆_{𝑗}}

, so acts on it as a real

𝜇_{𝑗}

with

𝜇_{𝑗}^{2} = 𝜆_{𝑗}

; there are two choices if

𝜆_{𝑗} > 0

and one if

𝜆_{𝑗} = 0

, and every choice works. So

2^{𝑘}

(a) Everything up to “ $𝑅^{2} = 𝜆 id$ on $𝐸_{𝜆}$ ” is correct. The error is the conclusion “ $𝑅 = \sqrt{𝜆} id$ on $𝐸_{𝜆}$ ”. A square root of $𝜆 id$ need not be $\sqrt{𝜆} id$ : the self-adjoint operator $𝑅_{𝜆} = 𝑅 ↾_{𝐸_{𝜆}}$ is diagonal in an orthonormal basis of $𝐸_{𝜆}$ with entries $𝜇$ satisfying $𝜇^{2} = 𝜆$ , so each entry may be $\sqrt{𝜆}$ or $- \sqrt{𝜆}$ . (In the first proof of Theorem 8.3.9 the extra hypothesis $𝜇 \geq 0$ settles this.) The claim is false: if $𝑇 \neq 0$ , then $- \sqrt{𝑇}$ is a second self-adjoint square root, and even for $𝑇 = 𝐼_{2}$ on $ℝ^{2}$ , every $diag (\pm 1, \pm 1)$ and every reflection is a self-adjoint square root.

(b) Let $𝜆_{1}, \dots, 𝜆_{𝑛}$ be the distinct eigenvalues, with eigenspaces $𝐸_{𝜆_{𝑗}} = span (𝐮_{𝑗})$ , which are lines because there are $𝑛$ of them in an $𝑛$ -dimensional space. Let $𝑅$ be self-adjoint with $𝑅^{2} = 𝑇$ . As in the argument, $𝑅$ maps $𝐸_{𝜆_{𝑗}}$ into itself, so $𝑅 𝐮_{𝑗} = 𝜇_{𝑗} 𝐮_{𝑗}$ for a scalar $𝜇_{𝑗}$ , which is real by Proposition 8.1.1, and $𝜇_{𝑗}^{2} 𝐮_{𝑗} =$ $𝑇 𝐮_{𝑗} =$ $𝜆_{𝑗} 𝐮_{𝑗}$ gives $𝜇_{𝑗}^{2} = 𝜆_{𝑗}$ . So $𝜇_{𝑗} \in {\sqrt{𝜆_{𝑗}}, - \sqrt{𝜆_{𝑗}}}$ , which is two choices if $𝜆_{𝑗} > 0$ and one if $𝜆_{𝑗} = 0$ , and $𝑅$ is determined by $𝜇_{1}, \dots, 𝜇_{𝑛}$ because $𝐮_{1}, \dots, 𝐮_{𝑛}$ is a basis. Conversely, for any such choice, $𝑅 = \sum_{𝑗} 𝜇_{𝑗} 𝑃_{𝑗}$ is self-adjoint (its coefficients are real) and $𝑅^{2} =$ $\sum_{𝑗} 𝜇_{𝑗}^{2} 𝑃_{𝑗} =$ $𝑇$ by Proposition 8.2.24. Different choices give different operators, as they differ on some $𝐮_{𝑗}$ . Hence $𝑇$ has exactly $2^{𝑘}$ self-adjoint square roots, and exactly one of them, $\sqrt{𝑇}$ , is positive.

Exercise 8.3.39.hardBeyond 21-242 Let $𝐴, 𝐵 \in 𝔽^{𝑛 \times 𝑛}$ be positive definite with $𝐴 ⪯ 𝐵$ (Definition 8.3.21).

(a)Show that $𝐵^{- 1} ⪯ 𝐴^{- 1}$ .
(b)Show that $det 𝐴 \leq det 𝐵$ .
(c)Show that for positive matrices, $𝐴 ⪯ 𝐵$ does not imply $𝐴^{2} ⪯ 𝐵^{2}$ : take $𝐴 = (\begin{matrix} 1 & 0 \\ 0 & 0 \end{matrix})$ and $𝐵 = (\begin{matrix} 2 & 1 \\ 1 & 1 \end{matrix})$ .

Solution.

Uses. Proposition 8.3.22 (Properties of the Löwner order), Theorem 8.3.9 (The positive square root), Proposition 8.3.15 (Characterizations of positive definite operators), Theorem 5.4.10 (Multiplicativity), Theorem 7.5.22 (Trace and determinant from eigenvalues)

Techniques. Sandwich between square roots (Technique 8.3.19)

Hint 1. Normalize

𝐵

to the identity: let

𝐶 = 𝐵^{- 1 / 2} 𝐴 𝐵^{- 1 / 2}

. By Proposition 8.3.22(c),

𝐶 ⪯ 𝐼_{𝑛}

Hint 2. By Proposition 8.3.22(d), the eigenvalues of

𝐶

lie in

(0, 1]

. So the eigenvalues of

𝐶^{- 1}

are at least

1

, and

𝐶^{- 1} =

𝐵^{1 / 2} 𝐴^{- 1} 𝐵^{1 / 2}

Hint 3. For (b),

det 𝐶 = det 𝐴 / det 𝐵

. For (c), compute the eigenvalues, or the determinant, of

𝐵 - 𝐴

and of

𝐵^{2} - 𝐴^{2}

Sketch.

𝐶 =

𝐵^{- 1 / 2} 𝐴 𝐵^{- 1 / 2} ⪯ 𝐵^{- 1 / 2} 𝐵 𝐵^{- 1 / 2} =

𝐼_{𝑛}

, and

𝐶

is positive definite, so its eigenvalues lie in

(0, 1]

. (a)

𝐶^{- 1} =

𝐵^{1 / 2} 𝐴^{- 1} 𝐵^{1 / 2}

has eigenvalues

\geq 1

, so

𝐼_{𝑛} ⪯ 𝐶^{- 1}

; congruence by

𝐵^{- 1 / 2}

gives

𝐵^{- 1} ⪯ 𝐴^{- 1}

. (b)

det 𝐴 / det 𝐵 =

det 𝐶 \leq

1

. (c)

𝐵 - 𝐴 = (\begin{matrix} 1 & 1 \\ 1 & 1 \end{matrix})

is positive, but

𝐵^{2} - 𝐴^{2} = (\begin{matrix} 4 & 3 \\ 3 & 2 \end{matrix})

has determinant

- 1

Let $𝐵^{1 / 2} = \sqrt{𝐵}$ and $𝐵^{- 1 / 2} = {(\sqrt{𝐵})}^{- 1}$ , both positive definite and Hermitian. Let $𝐶 = 𝐵^{- 1 / 2} 𝐴 𝐵^{- 1 / 2}$ . By Proposition 8.3.22(c) with $𝑋 = 𝐵^{- 1 / 2}$ ,

𝐶 ⪯ 𝐵^{- 1 / 2} 𝐵 𝐵^{- 1 / 2} = 𝐼_{𝑛} .

Also $𝐶 = 𝑋^{*} 𝐴 𝑋$ with $𝑋$ invertible, so $𝐶$ is positive definite: $𝐱^{*} 𝐶 𝐱 =$ ${(𝑋 𝐱)}^{*} 𝐴 (𝑋 𝐱) >$ $0$ for $𝐱 \neq 𝟎$ . By Proposition 8.3.22(d) and Proposition 8.3.15, every eigenvalue of $𝐶$ lies in $(0, 1]$ .

(a) $𝐶$ is invertible with $𝐶^{- 1} =$ $𝐵^{1 / 2} 𝐴^{- 1} 𝐵^{1 / 2}$ , and its eigenvalues are the reciprocals of those of $𝐶$ (if $𝐶 𝐯 = 𝜇 𝐯$ , then $𝐶^{- 1} 𝐯 = 𝜇^{- 1} 𝐯$ , and $𝐶^{- 1}$ is Hermitian with an orthonormal eigenbasis made of eigenvectors of $𝐶$ ). So they are all $\geq 1$ , and $𝐼_{𝑛} ⪯ 𝐶^{- 1}$ by Proposition 8.3.22(d). Applying Proposition 8.3.22(c) with $𝑋 = 𝐵^{- 1 / 2}$ ,

𝐵^{- 1} = 𝐵^{- 1 / 2} 𝐼_{𝑛} 𝐵^{- 1 / 2} ⪯ 𝐵^{- 1 / 2} 𝐵^{1 / 2} 𝐴^{- 1} 𝐵^{1 / 2} 𝐵^{- 1 / 2} = 𝐴^{- 1} .

(b) $det 𝐶$ is the product of the eigenvalues of $𝐶$ (Theorem 7.5.22), which lie in $(0, 1]$ , so $det 𝐶 \leq 1$ . By Theorem 5.4.10, $det 𝐶 =$ ${det (𝐵^{- 1 / 2})}^{2} det 𝐴$ , and ${det (𝐵^{- 1 / 2})}^{2} det 𝐵 =$ $det (𝐵^{- 1 / 2} 𝐵 𝐵^{- 1 / 2}) =$ $det 𝐼_{𝑛} =$ $1$ . So $det 𝐶 =$ $det 𝐴 / det 𝐵 \leq$ $1$ , and $det 𝐴 \leq det 𝐵$ because $det 𝐵 > 0$ .

(c) $𝐴$ is a projection and $𝐵$ has trace $3$ and determinant $1$ , so both are positive. $𝐵 - 𝐴 = (\begin{matrix} 1 & 1 \\ 1 & 1 \end{matrix})$ is positive (eigenvalues $2$ and $0$ ), so $𝐴 ⪯ 𝐵$ . But $𝐴^{2} = 𝐴$ and $𝐵^{2} = (\begin{matrix} 5 & 3 \\ 3 & 2 \end{matrix})$ , so $𝐵^{2} - 𝐴^{2} = (\begin{matrix} 4 & 3 \\ 3 & 2 \end{matrix})$ , whose determinant is $8 - 9 = - 1 < 0$ . Its eigenvalues have a negative product, so one of them is negative, and $𝐴^{2} ⪯ 𝐵^{2}$ fails. The tempting argument “ $𝐵^{2} -$ $𝐴^{2} =$ $(𝐵 - 𝐴) (𝐵 + 𝐴)$ is a product of positive operators” is wrong twice: the factorization needs $𝐴 𝐵 = 𝐵 𝐴$ , and a product of positive operators need not be positive (Proposition 8.3.20).

Exercise 8.3.40.hardBeyond 21-242 Let

𝐴, 𝐵 \in L (𝑉)

with

0 ⪯ 𝐴 ⪯ 𝐵

. Show that

\sqrt{𝐴} ⪯ \sqrt{𝐵}

. Then show that the converse fails:

\sqrt{𝐴} ⪯ \sqrt{𝐵}

does not imply

𝐴 ⪯ 𝐵

Solution.

Uses. Lemma 8.3.8 (Where the form vanishes), Theorem 8.3.5 (Characterizations of positive operators), Proposition 8.1.1 (Eigenvalues and eigenvectors of self-adjoint operators), Exercise 8.3.39

Techniques. A sum of nonnegative terms (Technique 6.1.9), Test an identity on eigenvectors (Technique 7.4.25)

Hint 1. Imitate the third proof of uniqueness in Theorem 8.3.9. Let

𝑋 = \sqrt{𝐵} - \sqrt{𝐴}

, which is self-adjoint; it suffices to show that every eigenvalue of

𝑋

\geq 0

Hint 2. Check that

𝐵 - 𝐴 = \sqrt{𝐵} 𝑋 + 𝑋 \sqrt{𝐴}

, apply both sides to an eigenvector

𝐯

𝑋

, and take the inner product with

𝐯

Hint 3. You get

⟨ (𝐵 - 𝐴) 𝐯, 𝐯 ⟩ =

𝜇 (⟨ \sqrt{𝐵} 𝐯, 𝐯 ⟩ + ⟨ \sqrt{𝐴} 𝐯, 𝐯 ⟩)

. The left side is

\geq 0

. What if the bracket is

0

Sketch. With

𝑋 = \sqrt{𝐵} - \sqrt{𝐴}

and

𝑋 𝐯 = 𝜇 𝐯

𝐯 \neq 𝟎

𝐵 - 𝐴 = \sqrt{𝐵} 𝑋 + 𝑋 \sqrt{𝐴}

gives

0 \leq

⟨ (𝐵 - 𝐴) 𝐯, 𝐯 ⟩ =

𝜇 (⟨ \sqrt{𝐵} 𝐯, 𝐯 ⟩ + ⟨ \sqrt{𝐴} 𝐯, 𝐯 ⟩)

. If the bracket is positive,

𝜇 \geq 0

. If it is

0

, then

\sqrt{𝐵} 𝐯 = \sqrt{𝐴} 𝐯 = 𝟎

by Lemma 8.3.8, so

𝜇 𝐯 = 𝑋 𝐯 = 𝟎

and

𝜇 = 0

. So

𝑋

is positive. Converse: square roots

diag (1, 0) ⪯ (\begin{matrix} 2 & 1 \\ 1 & 1 \end{matrix})

of Exercise 8.3.39(c).

Let $𝑋 = \sqrt{𝐵} - \sqrt{𝐴}$ , which is self-adjoint. Then

\sqrt{𝐵} 𝑋 + 𝑋 \sqrt{𝐴} = 𝐵 - \sqrt{𝐵} \sqrt{𝐴} + \sqrt{𝐵} \sqrt{𝐴} - 𝐴 = 𝐵 - 𝐴 .

Let $𝜇$ be an eigenvalue of $𝑋$ , with $𝑋 𝐯 = 𝜇 𝐯$ and $𝐯 \neq 𝟎$ ; $𝜇$ is real (Proposition 8.1.1). Using $𝑋^{*} = 𝑋$ ,

⟨ (𝐵 - 𝐴) 𝐯, 𝐯 ⟩ = ⟨ \sqrt{𝐵} (𝜇 𝐯), 𝐯 ⟩ + ⟨ \sqrt{𝐴} 𝐯, 𝑋 𝐯 ⟩ = 𝜇 (⟨ \sqrt{𝐵} 𝐯, 𝐯 ⟩ + ⟨ \sqrt{𝐴} 𝐯, 𝐯 ⟩) .

The left side is $\geq 0$ because $𝐴 ⪯ 𝐵$ , and the bracket $𝛽$ is a sum of two nonnegative numbers. If $𝛽 > 0$ , then $𝜇 =$ $⟨ (𝐵 - 𝐴) 𝐯, 𝐯 ⟩ / 𝛽 \geq$ $0$ . If $𝛽 = 0$ , then both terms are $0$ , so $\sqrt{𝐵} 𝐯 = \sqrt{𝐴} 𝐯 = 𝟎$ by Lemma 8.3.8, hence $𝜇 𝐯 = 𝑋 𝐯 = 𝟎$ and $𝜇 = 0$ . In either case $𝜇 \geq 0$ , so $𝑋$ is positive by Theorem 8.3.5(II), that is, $\sqrt{𝐴} ⪯ \sqrt{𝐵}$ .

For the converse, let $𝐴_{0} = (\begin{matrix} 1 & 0 \\ 0 & 0 \end{matrix})$ and $𝐵_{0} = (\begin{matrix} 2 & 1 \\ 1 & 1 \end{matrix})$ , which are positive with $𝐴_{0} ⪯ 𝐵_{0}$ (Exercise 8.3.39(c)). Put $𝐴 = 𝐴_{0}^{2}$ and $𝐵 = 𝐵_{0}^{2}$ . Then $\sqrt{𝐴} = 𝐴_{0}$ and $\sqrt{𝐵} = 𝐵_{0}$ by uniqueness of positive square roots, so $\sqrt{𝐴} ⪯ \sqrt{𝐵}$ ; but $𝐴 ⪯ 𝐵$ fails, by Exercise 8.3.39(c).

Exercise 8.3.41.standardBeyond 21-242

(a)Find the Cholesky factorization of $𝐴 = (\begin{matrix} 1 & 2 & 1 \\ 2 & 13 & 8 \\ 1 & 8 & 14 \end{matrix})$ with Algorithm 8.3.24.
(b)Use Algorithm 8.3.24 to decide whether $𝐵 = (\begin{matrix} 1 & 2 & 3 \\ 2 & 5 & 4 \\ 3 & 4 & 6 \end{matrix})$ is positive definite.
(c)From the computation in (b), find a vector $𝐱$ with $𝐱^{𝖳} 𝐵 𝐱 < 0$ .

Solution.

Uses. Algorithm 8.3.24 (Cholesky factorization), Proposition 8.3.25 (Correctness of the Cholesky algorithm), Theorem 8.3.23 (Cholesky factorization)

Hint 1. Work row by row:

𝑟_{𝑘 𝑘} =

\sqrt{𝑎_{𝑘 𝑘} - \sum_{𝑖 < 𝑘} 𝑟_{𝑖 𝑘}^{2}}

, then

𝑟_{𝑘 𝑗} =

(𝑎_{𝑘 𝑗} - \sum_{𝑖 < 𝑘} 𝑟_{𝑖 𝑘} 𝑟_{𝑖 𝑗}) / 𝑟_{𝑘 𝑘}

Hint 2. In (b) the algorithm stops at

𝑘 = 3

. For (c), look for

𝐱 = (𝑥_{1}, 𝑥_{2}, 1)

with the first two entries of

𝐵 𝐱

equal to

0

, as in the induction proof of Theorem 8.3.23.

Sketch. (a)

𝑅 = (\begin{matrix} 1 & 2 & 1 \\ 0 & 3 & 2 \\ 0 & 0 & 3 \end{matrix})

. (b)

𝑟_{1 *} = (1, 2, 3)

𝑟_{22} = 1

𝑟_{23} = - 2

, then

𝑑 = 6 - 9 - 4 = - 7 < 0

: not positive definite. (c) Solving

𝑥_{1} + 2 𝑥_{2} = - 3

2 𝑥_{1} + 5 𝑥_{2} = - 4

gives

𝐱 = (- 7, 2, 1)

𝐵 𝐱 = (0, 0, - 7)

, and

𝐱^{𝖳} 𝐵 𝐱 = - 7

(a) $𝑘 = 1$ : $𝑟_{11} = 1$ , $𝑟_{12} = 2$ , $𝑟_{13} = 1$ . $𝑘 = 2$ : $𝑑 = 13 - 2^{2} = 9$ , so $𝑟_{22} = 3$ , and $𝑟_{23} = (8 - 2 \cdot 1) / 3 = 2$ . $𝑘 = 3$ : $𝑑 = 14 - 1^{2} - 2^{2} = 9$ , so $𝑟_{33} = 3$ . Hence

𝐴 = 𝑅^{𝖳} 𝑅, 𝑅 = (\begin{matrix} 1 & 2 & 1 \\ 0 & 3 & 2 \\ 0 & 0 & 3 \end{matrix}) .

Check the last column of $𝑅^{𝖳} 𝑅$ : $(1 \cdot 1, 2 \cdot 1 + 3 \cdot 2, 1 + 4 + 9) =$ $(1, 8, 14)$ .

(b) $𝑘 = 1$ : $𝑟_{11} = 1$ , $𝑟_{12} = 2$ , $𝑟_{13} = 3$ . $𝑘 = 2$ : $𝑑 = 5 - 4 = 1$ , $𝑟_{22} = 1$ , $𝑟_{23} = (4 - 2 \cdot 3) / 1 = - 2$ . $𝑘 = 3$ : $𝑑 =$ $6 -$ $3^{2} -$ ${(- 2)}^{2} =$ $- 7 \leq$ $0$ , and the algorithm stops. By Proposition 8.3.25, $𝐵$ is not positive definite.

(c) Following the induction proof of Theorem 8.3.23, take $𝐱 = (𝑥_{1}, 𝑥_{2}, 1)$ with $𝑥_{1}, 𝑥_{2}$ chosen so that the first two entries of $𝐵 𝐱$ vanish:

𝑥_{1} + 2 𝑥_{2} + 3 = 0, 2 𝑥_{1} + 5 𝑥_{2} + 4 = 0 .

This gives $𝑥_{2} = 2$ and $𝑥_{1} = - 7$ . Then $𝐵 𝐱 =$ $(- 7 + 4 + 3, - 14 + 10 + 4, - 21 + 8 + 6) =$ $(0, 0, - 7)$ , and $𝐱^{𝖳} 𝐵 𝐱 =$ $1 \cdot$ $(- 7) =$ $- 7 <$ $0$ . The value $- 7$ is exactly the number $𝑑$ at which the algorithm stopped: it is the Schur complement of the leading $2 \times 2$ block.

Exercise 8.3.42.hardBeyond 21-242

(a)(Hadamard’s inequality.) Show that every positive $𝐴 \in 𝔽^{𝑛 \times 𝑛}$ satisfies $det 𝐴 \leq$ $𝑎_{11} 𝑎_{22} \dots 𝑎_{𝑛 𝑛}$ .
(b)Deduce that every $𝐵 \in 𝔽^{𝑛 \times 𝑛}$ with columns $𝐛_{1}, \dots, 𝐛_{𝑛}$ satisfies $| det 𝐵 | \leq$ $‖ 𝐛_{1} ‖ ‖ 𝐛_{2} ‖ \dots ‖ 𝐛_{𝑛} ‖$ . If $𝐵$ is invertible, show that equality holds if and only if the columns of $𝐵$ are orthogonal.

Solution (by normalizing the diagonal).

Uses. Exercise 8.3.32, Exercise 8.3.28, Theorem 7.5.22 (Trace and determinant from eigenvalues), Theorem 5.4.10 (Multiplicativity)

Techniques. Sandwich between square roots (Technique 8.3.19)

Hint 1. If some

𝑎_{𝑖 𝑖} = 0

, then

det 𝐴 = 0

(Exercise 8.3.32). Otherwise let

𝐷 =

diag (𝑎_{11}, \dots, 𝑎_{𝑛 𝑛})

and

𝐶 = 𝐷^{- 1 / 2} 𝐴 𝐷^{- 1 / 2}

. What are the diagonal entries of

𝐶

Hint 2.

𝐶

is positive with trace

𝑛

. Apply the AM–GM inequality to its eigenvalues.

Hint 3. For (b), apply (a) to

𝐴 = 𝐵^{*} 𝐵

Sketch. If some

𝑎_{𝑖 𝑖} = 0

, a row of

𝐴

vanishes and

det 𝐴 = 0

. Otherwise

𝐶 = 𝐷^{- 1 / 2} 𝐴 𝐷^{- 1 / 2}

is positive with

𝑐_{𝑖 𝑖} = 1

, so its eigenvalues

𝜇_{𝑖} \geq 0

have sum

𝑛

and

det 𝐶 =

\prod 𝜇_{𝑖} \leq

{(\sum 𝜇_{𝑖} / 𝑛)}^{𝑛} =

1

by AM–GM. Then

det 𝐴 =

det 𝐷 det 𝐶 \leq

\prod 𝑎_{𝑖 𝑖}

. (b)

𝐴 = 𝐵^{*} 𝐵

has

𝑎_{𝑖 𝑖} = {‖ 𝐛_{𝑖} ‖}^{2}

and

det 𝐴 = {| det 𝐵 |}^{2}

(a) All $𝑎_{𝑖 𝑖} \geq 0$ . If some $𝑎_{𝑖 𝑖} = 0$ , then row $𝑖$ of $𝐴$ is zero by Exercise 8.3.32(b), so $det 𝐴 = 0 \leq \prod_{𝑖} 𝑎_{𝑖 𝑖}$ . Otherwise let $𝐷 =$ $diag (𝑎_{11}, \dots, 𝑎_{𝑛 𝑛})$ and $𝐷^{- 1 / 2} =$ $diag (𝑎_{11}^{- 1 / 2}, \dots, 𝑎_{𝑛 𝑛}^{- 1 / 2})$ , and $𝐶 = 𝐷^{- 1 / 2} 𝐴 𝐷^{- 1 / 2}$ . Then $𝐶$ is positive by Exercise 8.3.28(b), and $𝑐_{𝑖 𝑗} =$ $𝑎_{𝑖 𝑗} / \sqrt{𝑎_{𝑖 𝑖} 𝑎_{𝑗 𝑗}}$ , so every $𝑐_{𝑖 𝑖} = 1$ and $tr 𝐶 = 𝑛$ . Let $𝜇_{1}, \dots, 𝜇_{𝑛} \geq 0$ be the eigenvalues of $𝐶$ counted with multiplicity. By Theorem 7.5.22 and the AM–GM inequality,

det 𝐶 = 𝜇_{1} \dots 𝜇_{𝑛} \leq {(\frac{𝜇_{1} + \dots + 𝜇_{𝑛}}{𝑛})}^{𝑛} = {(tr \frac{𝐶}{𝑛})}^{𝑛} = 1 .

By Theorem 5.4.10, $det 𝐴 =$ $det (𝐷^{1 / 2}) det 𝐶 det (𝐷^{1 / 2}) =$ $(\prod_{𝑖} 𝑎_{𝑖 𝑖}) det 𝐶 \leq$ $\prod_{𝑖} 𝑎_{𝑖 𝑖}$ .

(b) $𝐴 = 𝐵^{*} 𝐵$ is positive, with $𝑎_{𝑖 𝑖} =$ $𝐛_{𝑖}^{*} 𝐛_{𝑖} =$ ${‖ 𝐛_{𝑖} ‖}^{2}$ and $det 𝐴 =$ $det (𝐵^{*}) det 𝐵 =$ $det 𝐵 det 𝐵 =$ ${| det 𝐵 |}^{2}$ (Theorem 5.4.14). So (a) gives ${| det 𝐵 |}^{2} \leq$ $\prod_{𝑖} {‖ 𝐛_{𝑖} ‖}^{2}$ . For the equality case, see the second solution.

Solution (via Cholesky).

Uses. Theorem 8.3.23 (Cholesky factorization), Proposition 5.3.15 (Triangular matrices), Theorem 5.4.10 (Multiplicativity), Theorem 5.4.14 (The transpose)

Techniques. Sandwich between square roots (Technique 8.3.19)

Hint 1. If

𝐴

is not invertible,

det 𝐴 = 0

. Otherwise write

𝐴 = 𝑅^{*} 𝑅

(Theorem 8.3.23) and compute

det 𝐴

and

𝑎_{𝑖 𝑖}

in terms of the entries of

𝑅

Hint 2.

𝑎_{𝑖 𝑖}

is the squared length of column

𝑖

𝑅

, and

det 𝐴 = \prod_{𝑖} 𝑟_{𝑖 𝑖}^{2}

Sketch. For singular

𝐴

det 𝐴 = 0 \leq \prod 𝑎_{𝑖 𝑖}

. Otherwise

𝐴 = 𝑅^{*} 𝑅

det 𝐴 = \prod 𝑟_{𝑖 𝑖}^{2}

, and

𝑎_{𝑖 𝑖} =

\sum_{𝑘 \leq 𝑖} {| 𝑟_{𝑘 𝑖} |}^{2} \geq

𝑟_{𝑖 𝑖}^{2}

. Equality for all

𝑖

iff

𝑅

is diagonal iff

𝐴

is diagonal. In (b),

𝐴 = 𝐵^{*} 𝐵

is diagonal iff the columns of

𝐵

are orthogonal.

(a) If $𝐴$ is not invertible, $det 𝐴 = 0 \leq \prod_{𝑖} 𝑎_{𝑖 𝑖}$ . Otherwise $𝐴$ is positive definite (Proposition 8.3.15), and $𝐴 = 𝑅^{*} 𝑅$ with $𝑅$ upper triangular with positive diagonal entries (Theorem 8.3.23). By Proposition 5.3.15 and Theorem 5.4.10, $det 𝐴 =$ $det 𝑅 det 𝑅 =$ $\prod_{𝑖} 𝑟_{𝑖 𝑖}^{2}$ . The $(𝑖, 𝑖)$ entry of $𝑅^{*} 𝑅$ is the squared length of column $𝑖$ of $𝑅$ , whose nonzero entries are $𝑟_{1 𝑖}, \dots, 𝑟_{𝑖 𝑖}$ :

𝑎_{𝑖 𝑖} = \sum_{𝑘 \leq 𝑖} {| 𝑟_{𝑘 𝑖} |}^{2} \geq 𝑟_{𝑖 𝑖}^{2} .

(8.3.1)

Multiplying these $𝑛$ inequalities between nonnegative numbers gives $\prod_{𝑖} 𝑎_{𝑖 𝑖} \geq$ $\prod_{𝑖} 𝑟_{𝑖 𝑖}^{2} =$ $det 𝐴$ .

(b) As in the first solution, $𝐴 = 𝐵^{*} 𝐵$ is positive with $𝑎_{𝑖 𝑖} = {‖ 𝐛_{𝑖} ‖}^{2}$ and $det 𝐴 = {| det 𝐵 |}^{2}$ , which gives the inequality. Let $𝐵$ be invertible. Then $𝐴$ is positive definite and every $𝑟_{𝑖 𝑖} > 0$ , so equality $\prod_{𝑖} 𝑎_{𝑖 𝑖} =$ $\prod_{𝑖} 𝑟_{𝑖 𝑖}^{2}$ holds if and only if equality holds in every instance of (8.3.1), that is, if and only if $𝑟_{𝑘 𝑖} = 0$ for all $𝑘 < 𝑖$ . This means that $𝑅$ is diagonal, which makes $𝐴 = 𝑅^{*} 𝑅$ diagonal; conversely, if $𝐴$ is diagonal, then $det 𝐴 = \prod_{𝑖} 𝑎_{𝑖 𝑖}$ . So equality holds if and only if $𝐴 = 𝐵^{*} 𝐵$ is diagonal, that is, $𝐛_{𝑖}^{*} 𝐛_{𝑗} = 0$ for $𝑖 \neq 𝑗$ : the columns are orthogonal. Geometrically, the volume $| det 𝐵 |$ of the parallelepiped spanned by the columns is at most the product of the edge lengths, with equality for a box.

Exercise 8.3.43.challengeBeyond 21-242 Let $𝐴 \in 𝔽^{𝑛 \times 𝑛}$ be positive definite. For $𝐶 \in 𝔽^{𝑛 \times 𝑛}$ consider the equation

𝐴 𝑋 + 𝑋 𝐴 = 𝐶 (𝑋 \in 𝔽^{𝑛 \times 𝑛}) .

(a)Show that the equation has exactly one solution $𝑋$ .
(b)Show that if $𝐶$ is Hermitian, then so is $𝑋$ .
(c)Show that if $𝐶$ is positive, then so is $𝑋$ .
(d)Deduce that for real numbers $𝑎_{1}, \dots, 𝑎_{𝑛} > 0$ , the matrix with entries $1 / (𝑎_{𝑖} + 𝑎_{𝑗})$ is positive.

Solution.

Uses. Theorem 8.2.12 (Real spectral theorem), Corollary 8.2.7 (Hermitian matrices), Proposition 8.3.15 (Characterizations of positive definite operators), Theorem 8.3.5 (Characterizations of positive operators), Proposition 8.1.1 (Eigenvalues and eigenvectors of self-adjoint operators)

Techniques. Choose a convenient basis (Technique 4.5.33), Test an identity on eigenvectors (Technique 7.4.25)

Hint 1. Diagonalize

𝐴 = 𝑈 𝐷 𝑈^{*}

and substitute

𝑋 = 𝑈 𝑌 𝑈^{*}

. The equation becomes

𝐷 𝑌 + 𝑌 𝐷 = 𝑈^{*} 𝐶 𝑈

; read it entry by entry.

Hint 2. For (b), take the conjugate transpose of the equation and use (a).

Hint 3. For (c), let

𝑋 𝐯 = 𝜇 𝐯

and compute

⟨ 𝐶 𝐯, 𝐯 ⟩ =

𝐯^{*} (𝐴 𝑋 + 𝑋 𝐴) 𝐯

in terms of

𝜇

and

𝐯^{*} 𝐴 𝐯

Hint 4. For (d), take

𝐴 =

diag (𝑎_{1}, \dots, 𝑎_{𝑛})

and

𝐶

the matrix of all ones.

Sketch. (a) With

𝐴 = 𝑈 𝐷 𝑈^{*}

𝐷 = diag (𝑑_{𝑖})

𝑑_{𝑖} > 0

, the equation is

(𝑑_{𝑖} + 𝑑_{𝑗}) 𝑦_{𝑖 𝑗} =

{(𝑈^{*} 𝐶 𝑈)}_{𝑖 𝑗}

for

𝑌 = 𝑈^{*} 𝑋 𝑈

, which has exactly one solution since

𝑑_{𝑖} + 𝑑_{𝑗} > 0

. (b)

𝑋^{*}

solves the same equation. (c) For

𝑋 𝐯 = 𝜇 𝐯

𝐯^{*} 𝐶 𝐯 =

2 𝜇 𝐯^{*} 𝐴 𝐯

, and

𝐯^{*} 𝐴 𝐯 > 0

, so

𝜇 \geq 0

. (d) With

𝐴 = diag (𝑎_{𝑖})

and

𝐶 = 𝟏 𝟏^{𝖳}

positive, the solution is

𝑥_{𝑖 𝑗} =

1 / (𝑎_{𝑖} + 𝑎_{𝑗})

(a) By Theorem 8.2.12 or Corollary 8.2.7, $𝐴 = 𝑈 𝐷 𝑈^{*}$ with $𝑈$ unitary and $𝐷 =$ $diag (𝑑_{1}, \dots, 𝑑_{𝑛})$ , and $𝑑_{𝑖} > 0$ by Proposition 8.3.15. Since $𝑋 \mapsto 𝑈^{*} 𝑋 𝑈$ is a bijection of $𝔽^{𝑛 \times 𝑛}$ (with inverse $𝑌 \mapsto 𝑈 𝑌 𝑈^{*}$ ), we may write $𝑋 = 𝑈 𝑌 𝑈^{*}$ . Then

𝐴 𝑋 + 𝑋 𝐴 = 𝑈 (𝐷 𝑌 + 𝑌 𝐷) 𝑈^{*},

so $𝐴 𝑋 + 𝑋 𝐴 = 𝐶$ if and only if $𝐷 𝑌 + 𝑌 𝐷 = 𝐶^{'}$ , where $𝐶^{'} = 𝑈^{*} 𝐶 𝑈$ . The $(𝑖, 𝑗)$ entry of $𝐷 𝑌 + 𝑌 𝐷$ is $(𝑑_{𝑖} + 𝑑_{𝑗}) 𝑦_{𝑖 𝑗}$ , and $𝑑_{𝑖} + 𝑑_{𝑗} > 0$ . So the equation holds if and only if $𝑦_{𝑖 𝑗} =$ $𝑐_{𝑖 𝑗}^{'} / (𝑑_{𝑖} + 𝑑_{𝑗})$ for all $𝑖, 𝑗$ , and there is exactly one solution.

(b) Taking conjugate transposes of $𝐴 𝑋 + 𝑋 𝐴 = 𝐶$ and using $𝐴^{*} = 𝐴$ gives $𝑋^{*} 𝐴 +$ $𝐴 𝑋^{*} =$ $𝐶^{*} =$ $𝐶$ . So $𝑋^{*}$ is also a solution, and by (a), $𝑋^{*} = 𝑋$ .

(c) By (b), $𝑋$ is Hermitian, so its eigenvalues are real (Proposition 8.1.1). Let $𝑋 𝐯 = 𝜇 𝐯$ with $𝐯 \neq 𝟎$ . Then $𝐯^{*} 𝑋 =$ ${(𝑋 𝐯)}^{*} =$ $𝜇 𝐯^{*} =$ $𝜇 𝐯^{*}$ , and

𝐯^{*} 𝐶 𝐯 = 𝐯^{*} 𝐴 (𝑋 𝐯) + (𝐯^{*} 𝑋) 𝐴 𝐯 = 𝜇 𝐯^{*} 𝐴 𝐯 + 𝜇 𝐯^{*} 𝐴 𝐯 = 2 𝜇 𝐯^{*} 𝐴 𝐯 .

The left side is $\geq 0$ because $𝐶$ is positive, and $𝐯^{*} 𝐴 𝐯 > 0$ because $𝐴$ is positive definite. So $𝜇 \geq 0$ . Every eigenvalue of the Hermitian matrix $𝑋$ is nonnegative, and $𝑋$ is positive by Theorem 8.3.5(II).

(d) Let $𝐴 =$ $diag (𝑎_{1}, \dots, 𝑎_{𝑛})$ , which is positive definite, and let $𝐶 = 𝟏 𝟏^{𝖳}$ be the matrix of all ones, where $𝟏 = (1, \dots, 1)$ ; $𝐶$ is positive, being $𝐵^{*} 𝐵$ for the $1 \times 𝑛$ matrix $𝐵 = 𝟏^{𝖳}$ . The $(𝑖, 𝑗)$ entry of $𝐴 𝑋 + 𝑋 𝐴$ is $(𝑎_{𝑖} + 𝑎_{𝑗}) 𝑥_{𝑖 𝑗}$ , so the solution of $𝐴 𝑋 + 𝑋 𝐴 = 𝐶$ is $𝑥_{𝑖 𝑗} =$ $1 / (𝑎_{𝑖} + 𝑎_{𝑗})$ . By (c) this matrix is positive.

The heart of (c) is the trick from the third proof of Theorem 8.3.9: test the equation on an eigenvector of the unknown, not of the data.

More practice

The practice bank has 16 exercises on this section’s topics.