6.6 Adjoints · Matrix Theory

The conjugate transpose moves a matrix from one side of the dot product to the other: $(𝐴 𝐱) \cdot$ $𝐲 =$ $𝐱 \cdot$ $(𝐴^{*} 𝐲)$ (Proposition 6.3.27(d)). That identity is what made the normal equations and the projection matrices of the last two sections work. This section asks for the same thing for a linear map $𝑇 : 𝑉 \to 𝑊$ between inner product spaces: a map $𝑇^{*} : 𝑊 \to 𝑉$ with $⟨ 𝑇 𝐯, 𝐰 ⟩ =$ $⟨ 𝐯, 𝑇^{*} 𝐰 ⟩$ for all $𝐯$ and $𝐰$ . When $𝑉$ and $𝑊$ are finite-dimensional there is exactly one such map, the adjoint of $𝑇$ . It is defined without coordinates, and in orthonormal coordinates it is the conjugate transpose, so it is what the transpose of a matrix really is. Its kernel and image are the orthogonal complements of the image and kernel of $𝑇$ , which explains the orthogonality among the four fundamental subspaces of a matrix. The operators that equal their own adjoint, the self-adjoint operators, are the subject of Chapter 8. To construct $𝑇^{*}$ we first answer a simpler question: which linear functions $𝑉 \to 𝔽$ are inner products with a fixed vector?

Throughout, $𝑉$ and $𝑊$ are inner product spaces over $𝔽$ , where $𝔽$ is $ℝ$ or $ℂ$ (Notation 6.1.1), and we write $𝑇 𝐯$ for $𝑇 (𝐯)$ .

6.6.1Linear functionals and the Riesz representation theorem

Definition 6.6.1 (Linear functional). A linear functional on a vector space

𝑉

over a field

𝐹

is a linear map

𝜑 : 𝑉 \to 𝐹

The linear functionals on $𝑉$ form the vector space $L (𝑉, 𝐹)$ , the dual space of $𝑉$ , which Section 13.1 studies for its own sake. Here we need only a few examples.

Example 6.6.2 (Linear functionals).

(a)For scalars $𝑎_{1}, \dots, 𝑎_{𝑛} \in 𝐹$ , the map $𝜑 (𝐱) =$ $𝑎_{1} 𝑥_{1} +$ $\dots +$ $𝑎_{𝑛} 𝑥_{𝑛}$ is a linear functional on $𝐹^{𝑛}$ . Every linear functional on $𝐹^{𝑛}$ has this form, with $𝑎_{𝑗} = 𝜑 (𝐞_{𝑗})$ , since $𝜑 (𝐱) =$ $𝜑 (\sum_{𝑗} 𝑥_{𝑗} 𝐞_{𝑗}) =$ $\sum_{𝑗} 𝑥_{𝑗} 𝜑 (𝐞_{𝑗})$ (Corollary 2.2.17).
(b)The trace $tr : 𝐹^{𝑛 \times 𝑛} \to 𝐹$ (Proposition 2.4.27) and evaluation $𝑝 \mapsto 𝑝 (𝑐)$ on $𝐹 [𝑥]$ (Example 2.1.9) are linear functionals, and so is $𝑓 \mapsto \int_{0}^{1} 𝑓 (𝑡) d 𝑡$ on $𝐶 [0, 1]$ .
(c)Let $𝑉$ be an inner product space and $𝐮 \in 𝑉$ . Then $𝐯 \mapsto ⟨ 𝐯, 𝐮 ⟩$ is a linear functional on $𝑉$ , by linearity in the first slot. With $𝐮$ in the first slot instead, $𝐯 \mapsto ⟨ 𝐮, 𝐯 ⟩$ is not linear when $𝔽 = ℂ$ , since $⟨ 𝐮, 𝑖 𝐯 ⟩ =$ $- 𝑖 ⟨ 𝐮, 𝐯 ⟩$ (Proposition 6.1.4).

The Riesz representation theorem says that in a finite-dimensional inner product space, (c) is the only example: every linear functional is an inner product with a fixed vector.

Intuition (Level sets and the normal vector). Start with $𝑉 = ℝ^{2}$ and a nonzero functional $𝜑 (𝐱) =$ $𝑎_{1} 𝑥_{1} +$ $𝑎_{2} 𝑥_{2}$ . Here $𝜑 (𝐱) = 𝐱 \cdot 𝐮$ with $𝐮 = (𝑎_{1}, 𝑎_{2})$ , so a representing vector exists, and the picture shows why it had to. The level sets ${𝐱 | 𝜑 (𝐱) = 𝑐}$ are parallel lines, the translates of the line $ker 𝜑$ (Corollary 2.6.31). A vector $𝐮$ with $𝜑 (𝐱) = ⟨ 𝐱, 𝐮 ⟩$ for all $𝐱$ must be orthogonal to every vector of $ker 𝜑$ , so it lies on the line ${(ker 𝜑)}^{⟂}$ , normal to the level sets. That fixes $𝐮$ up to a scalar, and one value of $𝜑$ fixes the scalar: $𝜑 (𝐮) =$ $⟨ 𝐮, 𝐮 ⟩ =$ ${‖ 𝐮 ‖}^{2}$ . Geometrically, $⟨ 𝐱, 𝐮 ⟩$ is $‖ 𝐮 ‖$ times the signed distance from $𝐱$ to the line $ker 𝜑$ (Example 6.4.35), so $𝐮$ points in the direction in which $𝜑$ grows fastest, and $‖ 𝐮 ‖$ is the rate of growth (Figure 6.6.1). In the language of calculus, $𝐮$ is the gradient of $𝜑$ .

The same reasoning works in every finite-dimensional inner product space: the representing vector is normal to the kernel. Over $ℂ$ there is one twist. In $ℂ^{𝑛}$ , the functional $𝜑 (𝐱) =$ $\sum_{𝑗} 𝑎_{𝑗} 𝑥_{𝑗}$ equals $𝐱 \cdot 𝐮 = \sum_{𝑗} 𝑥_{𝑗} 𝑢_{𝑗}$ only for $𝑢_{𝑗} = 𝑎_{𝑗}$ . The coefficients of $𝜑$ enter $𝐮$ conjugated, because $𝐮$ sits in the conjugate-linear slot.

Figure 6.6.1. The level sets

𝜑 = 𝑐

𝜑 (𝐱) = 𝑥_{1} + 𝑥_{2}

ℝ^{2}

are parallel lines. The vector

𝐮 = (1, 1)

with

𝜑 (𝐱) = ⟨ 𝐱, 𝐮 ⟩

is perpendicular to all of them, and its tip lies on the level set

𝜑 = {‖ 𝐮 ‖}^{2} = 2

The lecture does not prove the next theorem; it constructs adjoints of matrices directly, as in the second proof of Theorem 6.6.15 below. We include it because it explains what an adjoint is, and because its two proofs illustrate two ways of producing a vector with prescribed inner products.

Theorem 6.6.3 (Riesz representation theorem).Beyond 21-242 Let $𝑉$ be a finite-dimensional inner product space, and let $𝜑$ be a linear functional on $𝑉$ . There is exactly one vector $𝐮 \in 𝑉$ such that

𝜑 (𝐯) = ⟨ 𝐯, 𝐮 ⟩ for all 𝐯 \in 𝑉 .

If $𝐞_{1}, \dots, 𝐞_{𝑛}$ is an orthonormal basis of $𝑉$ , then $𝐮 =$ $𝜑 (𝐞_{1}) 𝐞_{1} +$ $\dots +$ $𝜑 (𝐞_{𝑛}) 𝐞_{𝑛}$ .

Proof (with an orthonormal basis).standard

Uses. Theorem 6.4.20 (Orthonormal bases exist), Proposition 6.1.4 (Sesquilinearity), Definition 6.3.15 (Orthonormal list, orthonormal basis), Corollary 2.2.6 (Agreeing on a spanning list), Proposition 6.1.37 (Inner products distinguish vectors)

Techniques. Uniqueness via two candidates (Technique 1.1.8), Take an inner product to isolate a coefficient (Technique 6.3.12), Check it on a basis (Technique 2.2.7)

Hint 1. For uniqueness, suppose

𝐮

and

𝐮^{'}

both work, and compare

⟨ 𝐯, 𝐮 ⟩

with

⟨ 𝐯, 𝐮^{'} ⟩

. For existence, first find the only possible candidate.

Hint 2. If

𝐮

works, its coefficient on

𝐞_{𝑗}

⟨ 𝐮, 𝐞_{𝑗} ⟩ =

⟨ 𝐞_{𝑗}, 𝐮 ⟩ =

𝜑 (𝐞_{𝑗})

Hint 3. To check the candidate, note that

𝜑

and

𝐯 \mapsto ⟨ 𝐯, 𝐮 ⟩

are both linear in

𝐯

, so it suffices to compare them on the basis.

Sketch. Uniqueness: if

⟨ 𝐯, 𝐮 ⟩ = ⟨ 𝐯, 𝐮^{'} ⟩

for all

𝐯

, then

𝐮 = 𝐮^{'}

by Proposition 6.1.37. Existence: for

𝐮 =

\sum_{𝑗} 𝜑 (𝐞_{𝑗}) 𝐞_{𝑗}

, pulling the coefficients out of the second slot conjugates them back, so

⟨ 𝐞_{𝑘}, 𝐮 ⟩ =

\sum_{𝑗} 𝜑 (𝐞_{𝑗}) ⟨ 𝐞_{𝑘}, 𝐞_{𝑗} ⟩ =

𝜑 (𝐞_{𝑘})

. Two linear functionals that agree on a basis are equal.

Uniqueness. Suppose $⟨ 𝐯, 𝐮 ⟩ = ⟨ 𝐯, 𝐮^{'} ⟩$ for all $𝐯 \in 𝑉$ . Taking conjugates, $⟨ 𝐮, 𝐯 ⟩ = ⟨ 𝐮^{'}, 𝐯 ⟩$ for all $𝐯$ , so $𝐮 = 𝐮^{'}$ by Proposition 6.1.37.

Existence. By Theorem 6.4.20, $𝑉$ has an orthonormal basis $𝐞_{1}, \dots, 𝐞_{𝑛}$ . Let $𝐮 =$ $\sum_{𝑗 = 1}^{𝑛} 𝜑 (𝐞_{𝑗}) 𝐞_{𝑗}$ , and let $𝜓 (𝐯) = ⟨ 𝐯, 𝐮 ⟩$ , a linear functional by linearity in the first slot. For each $𝑘$ , Proposition 6.1.4 and orthonormality give

𝜓 (𝐞_{𝑘}) = ⟨ 𝐞_{𝑘}, \sum_{𝑗 = 1}^{𝑛} 𝜑 (𝐞_{𝑗}) 𝐞_{𝑗} ⟩ = \sum_{𝑗 = 1}^{𝑛} 𝜑 (𝐞_{𝑗}) ⟨ 𝐞_{𝑘}, 𝐞_{𝑗} ⟩ = 𝜑 (𝐞_{𝑘}),

since the conjugate of $𝜑 (𝐞_{𝑗})$ is $𝜑 (𝐞_{𝑗})$ and $⟨ 𝐞_{𝑘}, 𝐞_{𝑗} ⟩ =$ $𝛿_{𝑘 𝑗}$ . So the linear maps $𝜑$ and $𝜓$ agree on a basis of $𝑉$ , and $𝜑 = 𝜓$ by Corollary 2.2.6. That is, $𝜑 (𝐯) = ⟨ 𝐯, 𝐮 ⟩$ for every $𝐯$ , as desired. □

Proof (via the orthogonal complement of the kernel).standard

Uses. Proposition 2.6.8 (Kernels and images are subspaces), Proposition 4.1.21 (Dimensions of subspaces), Corollary 6.4.27 (Testing whether a subspace is everything), Proposition 6.3.9 (Properties of $𝐴^{⟂}$ ), Proposition 6.1.4 (Sesquilinearity)

Techniques. Orthogonal decomposition (Technique 6.2.4)

Hint 1. If

𝐮

works, it is orthogonal to every vector of

ker 𝜑

. Where does that put it?

Hint 2. For

𝜑 \neq 0

, pick

𝐳 \neq 𝟎

{(ker 𝜑)}^{⟂}

(why does one exist?) and look for

𝐮

as a multiple of

𝐳

. Testing at

𝐯 = 𝐳

determines the multiple.

Hint 3. To check the candidate at a general

𝐯

, subtract the multiple of

𝐳

that lies on the same level set of

𝜑

𝐯 -

(𝜑 (𝐯) / 𝜑 (𝐳)) 𝐳 \in

ker 𝜑

Sketch. Uniqueness is as in the first proof. If

𝜑 \neq 0

, then

ker 𝜑 \neq 𝑉

, so some

𝐳 \neq 𝟎

is orthogonal to

ker 𝜑

(Corollary 6.4.27), and

𝜑 (𝐳) \neq 0

. Every

𝐯

(𝜑 (𝐯) / 𝜑 (𝐳)) 𝐳

plus a vector of

ker 𝜑

, which is orthogonal to

𝐳

. So

⟨ 𝐯, 𝐳 ⟩ =

(𝜑 (𝐯) / 𝜑 (𝐳)) {‖ 𝐳 ‖}^{2}

, and

𝐮 =

(𝜑 (𝐳) / {‖ 𝐳 ‖}^{2}) 𝐳

works.

Uniqueness is proved as in the first proof. For existence, if $𝜑 = 0$ , take $𝐮 = 𝟎$ . Suppose $𝜑 \neq 0$ , and let $𝐾 = ker 𝜑$ . It is a subspace of $𝑉$ (Proposition 2.6.8), finite-dimensional (Proposition 4.1.21), and $𝐾 \neq 𝑉$ . By Corollary 6.4.27, $𝐾^{⟂} \neq {𝟎}$ , so we may choose $𝐳 \in 𝐾^{⟂}$ with $𝐳 \neq 𝟎$ . Since $𝐾 \cap 𝐾^{⟂} = {𝟎}$ (Proposition 6.3.9(d)), $𝐳 \notin 𝐾$ , that is, $𝜑 (𝐳) \neq 0$ . Let

𝐮 = \frac{𝜑 (𝐳)}{{‖ 𝐳 ‖}^{2}} 𝐳 .

Let $𝐯 \in 𝑉$ and $𝑐 = 𝜑 (𝐯) / 𝜑 (𝐳)$ . Then $𝜑 (𝐯 - 𝑐 𝐳) =$ $𝜑 (𝐯) -$ $𝑐 𝜑 (𝐳) =$ $0$ , so $𝐯 - 𝑐 𝐳 \in 𝐾$ . As $𝐳 \in 𝐾^{⟂}$ , conjugate symmetry gives $⟨ 𝐯 - 𝑐 𝐳, 𝐳 ⟩ = 0$ , and therefore

⟨ 𝐯, 𝐳 ⟩ = 𝑐 ⟨ 𝐳, 𝐳 ⟩ = \frac{𝜑 (𝐯)}{𝜑 (𝐳)} {‖ 𝐳 ‖}^{2} .

Pulling the real number ${‖ 𝐳 ‖}^{2}$ and the scalar $𝜑 (𝐳)$ out of the second slot (Proposition 6.1.4),

⟨ 𝐯, 𝐮 ⟩ = \frac{𝜑 (𝐳)}{{‖ 𝐳 ‖}^{2}} ⟨ 𝐯, 𝐳 ⟩ = 𝜑 (𝐯),

as desired. □

The first proof computes $𝐮$ , and uniqueness shows that the formula, although written with an orthonormal basis, does not depend on which one. The second proof chooses no basis. It says where $𝐮$ lives, on the line ${(ker 𝜑)}^{⟂}$ normal to the level sets, and it uses finite-dimensionality only to know that $ker 𝜑$ has a nonzero orthogonal complement. In infinite-dimensional spaces, where no finite orthonormal basis exists, that is the step that can sometimes be rescued (Section 15.3).

Remark 6.6.4 (Where the conjugate comes from). The representing vector must sit in the second slot, since

𝐯 \mapsto ⟨ 𝐮, 𝐯 ⟩

is not linear over

ℂ

(Example 6.6.2(c)). Moving the coefficients

𝜑 (𝐞_{𝑗})

into the second slot conjugates them, which is why

𝐮

has coefficients

𝜑 (𝐞_{𝑗})

. For the same reason the assignment

𝐮 \mapsto ⟨ \cdot, 𝐮 ⟩

, from

𝑉

to the linear functionals on

𝑉

, is additive but conjugates scalars:

⟨ \cdot, 𝑐 𝐮 ⟩ = 𝑐 ⟨ \cdot, 𝐮 ⟩

. So Theorem 6.6.3 matches the vectors of

𝑉

one to one with the linear functionals on

𝑉

, by a map that is linear when

𝔽 = ℝ

and conjugate-linear when

𝔽 = ℂ

. Section 13.2 returns to this identification.

Example 6.6.5 (Representing evaluation at $0$ ). Give ${ℝ [𝑥]}_{\leq 1}$ the inner product $⟨ 𝑝, 𝑞 ⟩ =$ $\int_{0}^{1} 𝑝 (𝑡) 𝑞 (𝑡) d 𝑡$ , and let $𝜑 (𝑝) = 𝑝 (0)$ . Gram–Schmidt on $1, 𝑥$ gives the orthonormal basis $𝑒_{1} = 1$ , $𝑒_{2} = \sqrt{3} (2 𝑥 - 1)$ (Example 6.4.19). Here $𝜑 (𝑒_{1}) = 1$ and $𝜑 (𝑒_{2}) = - \sqrt{3}$ , so Theorem 6.6.3 gives the representing polynomial

𝑞 = 1 - \sqrt{3} \cdot \sqrt{3} (2 𝑥 - 1) = 4 - 6 𝑥 .

Check on the basis $1, 𝑥$ : $\int_{0}^{1} (4 - 6 𝑡) d 𝑡 =$ $4 -$ $3 =$ $1 =$ $𝜑 (1)$ , and $\int_{0}^{1} 𝑡 (4 - 6 𝑡) d 𝑡 =$ $2 -$ $2 =$ $0 =$ $𝜑 (𝑥)$ . So $\int_{0}^{1} 𝑝 (𝑡) (4 - 6 𝑡) d 𝑡 =$ $𝑝 (0)$ for every $𝑝$ of degree at most $1$ .

The answer depends on the space, not only on the formula for $𝜑$ . On ${ℝ [𝑥]}_{\leq 2}$ with the same inner product, evaluation at $0$ is represented by $9 - 36 𝑥 + 30 𝑥^{2}$ , as one checks on $1, 𝑥, 𝑥^{2}$ . The representing polynomials also grow: ${‖ 4 - 6 𝑥 ‖}^{2} =$ $⟨ 𝑞, 𝑞 ⟩ =$ $𝜑 (𝑞) =$ $𝑞 (0) =$ $4$ , and ${‖ 9 - 36 𝑥 + 30 𝑥^{2} ‖}^{2} = 9$ . (On ${ℝ [𝑥]}_{\leq 𝑛}$ the representing polynomial has norm $𝑛 + 1$ ; Exercise 6.6.49 shows that its norm is at least $\sqrt{2 𝑛 + 1}$ .) A polynomial that integrates against every $𝑝$ to give $𝑝 (0)$ has to concentrate its weight near $𝑡 = 0$ , and higher degrees allow more concentration.

Exercise 6.6.6.routine

(a)Find $𝐮 \in ℂ^{3}$ with $2 𝑧_{1} -$ $𝑖 𝑧_{2} +$ $(1 + 𝑖) 𝑧_{3} =$ $𝐳 \cdot$ $𝐮$ for all $𝐳 \in ℂ^{3}$ .
(b)With the inner product of Example 6.6.5 on ${ℝ [𝑥]}_{\leq 1}$ , find the polynomial $𝑞$ with $⟨ 𝑝, 𝑞 ⟩ = 𝑝 (1)$ for all $𝑝$ .

Solution.

Uses. Theorem 6.6.3 (Riesz representation theorem), Definition 6.1.8 (Dot product), Example 6.6.5 (Representing evaluation at $0$ )

Hint 1. In (a),

𝐳 \cdot 𝐮 = \sum_{𝑗} 𝑧_{𝑗} 𝑢_{𝑗}

, so each

𝑢_{𝑗}

is the conjugate of a coefficient.

Hint 2. In (b), use the orthonormal basis

1, \sqrt{3} (2 𝑥 - 1)

, or write

𝑞 = 𝑐 + 𝑑 𝑥

and impose the two conditions

⟨ 1, 𝑞 ⟩ = 1

and

⟨ 𝑥, 𝑞 ⟩ = 1

Sketch. (a)

𝐮 = (2, 𝑖, 1 - 𝑖)

, the conjugated coefficients. (b)

𝑞 =

1 +

\sqrt{3} \cdot

\sqrt{3} (2 𝑥 - 1) =

6 𝑥 -

2

; check:

\int_{0}^{1} (6 𝑡 - 2) d 𝑡 = 1

and

\int_{0}^{1} 𝑡 (6 𝑡 - 2) d 𝑡 = 1

(a) For $𝐮 = (2, 𝑖, 1 - 𝑖)$ ,

𝐳 \cdot 𝐮 = 𝑧_{1} 2 + 𝑧_{2} 𝑖 + 𝑧_{3} 1 - 𝑖 = 2 𝑧_{1} - 𝑖 𝑧_{2} + (1 + 𝑖) 𝑧_{3},

as required. By Theorem 6.6.3 (or by comparing coefficients at $𝐳 = 𝐞_{1}, 𝐞_{2}, 𝐞_{3}$ ) it is the only such vector.

(b) With $𝑒_{1} = 1$ and $𝑒_{2} = \sqrt{3} (2 𝑥 - 1)$ , the functional $𝜑 (𝑝) = 𝑝 (1)$ has $𝜑 (𝑒_{1}) = 1$ and $𝜑 (𝑒_{2}) = \sqrt{3}$ , so Theorem 6.6.3 gives

𝑞 = 1 + \sqrt{3} \cdot \sqrt{3} (2 𝑥 - 1) = 6 𝑥 - 2 .

Check: $\int_{0}^{1} (6 𝑡 - 2) d 𝑡 =$ $3 -$ $2 =$ $1 =$ $𝜑 (1)$ and $\int_{0}^{1} 𝑡 (6 𝑡 - 2) d 𝑡 =$ $2 -$ $1 =$ $1 =$ $𝜑 (𝑥)$ , so $⟨ 𝑝, 𝑞 ⟩ = 𝑝 (1)$ for all $𝑝 \in {ℝ [𝑥]}_{\leq 1}$ by linearity. As expected from the symmetry $𝑡 \mapsto 1 - 𝑡$ of $[0, 1]$ , $𝑞 (𝑥)$ is $4 - 6 (1 - 𝑥)$ , the polynomial of Example 6.6.5 with $𝑥$ replaced by $1 - 𝑥$ .

Finite-dimensionality cannot be dropped from Theorem 6.6.3.

Non-example 6.6.7 (Riesz fails in infinite dimensions).Beyond 21-242 Let $𝑃$ be the space of real polynomial functions on $[0, 1]$ , a subspace of $𝐶 [0, 1]$ , with $⟨ 𝑓, 𝑔 ⟩ =$ $\int_{0}^{1} 𝑓 (𝑡) 𝑔 (𝑡) d 𝑡$ (Proposition 6.1.13). We show that the linear functional $𝜑 (𝑝) = 𝑝 (0)$ is not of the form $⟨ \cdot, 𝑞 ⟩$ for any $𝑞 \in 𝑃$ . Suppose that $\int_{0}^{1} 𝑝 (𝑡) 𝑞 (𝑡) d 𝑡 =$ $𝑝 (0)$ for every $𝑝 \in 𝑃$ . Apply this to $𝑝 (𝑡) = 𝑡 𝑞 (𝑡)$ , which vanishes at $0$ :

\int_{0}^{1} 𝑡 {𝑞 (𝑡)}^{2} d 𝑡 = 0 .

The integrand is continuous and nonnegative on $[0, 1]$ , so it is identically $0$ (Lemma 6.1.14). Hence $𝑞 (𝑡) = 0$ for $0 < 𝑡 \leq 1$ , and $𝑞 (0) = 0$ by continuity. But then $𝑝 (0) = ⟨ 𝑝, 𝑞 ⟩ = 0$ for every $𝑝$ , which fails for $𝑝 = 1$ .

Restricted to the polynomials of degree at most $𝑛$ , the same functional is represented by a polynomial of norm $𝑛 + 1$ (Example 6.6.5). As $𝑛$ grows, these representatives pile up near $𝑡 = 0$ without converging to any polynomial. Section 15.3 explains which functionals do have representatives in infinite dimensions.

6.6.2The adjoint

Intuition (Moving a map across the inner product). For a matrix $𝐴$ , read the identity $(𝐴 𝐱) \cdot$ $𝐲 =$ $𝐱 \cdot$ $(𝐴^{*} 𝐲)$ as follows. Fix $𝐲$ . The left side is a measurement of the output $𝐴 𝐱$ : for a unit vector $𝐲$ , it is the component of $𝐴 𝐱$ along $𝐲$ . The right side computes the same number from the input $𝐱$ alone, as its component along the vector $𝐴^{*} 𝐲$ . So $𝐴^{*}$ turns a measurement on the output side into a measurement on the input side.

For a linear map $𝑇 : 𝑉 \to 𝑊$ we ask for the same conversion. Fix $𝐰 \in 𝑊$ . The function $𝐯 \mapsto ⟨ 𝑇 𝐯, 𝐰 ⟩$ is a linear functional on $𝑉$ , and when $𝑉$ is finite-dimensional, Theorem 6.6.3 says that it is $𝐯 \mapsto ⟨ 𝐯, 𝐮 ⟩$ for exactly one $𝐮 \in 𝑉$ . Calling this vector $𝑇^{*} 𝐰$ defines a function $𝑇^{*} : 𝑊 \to 𝑉$ . The definition records the property that $𝑇^{*}$ is meant to have, without presupposing that such a function exists.

Definition 6.6.8 (Adjoint). Let $𝑉$ and $𝑊$ be inner product spaces and $𝑇 \in L (𝑉, 𝑊)$ . A function $𝑆 : 𝑊 \to 𝑉$ is an adjoint of $𝑇$ if

⟨ 𝑇 𝐯, 𝐰 ⟩ = ⟨ 𝐯, 𝑆 𝐰 ⟩ for all 𝐯 \in 𝑉 and 𝐰 \in 𝑊 .

By Proposition 6.6.9 below, $𝑇$ has at most one adjoint. When it exists, it is denoted $𝑇^{*}$ (“ $𝑇$ star”).

The inner product on the left is that of $𝑊$ , and the one on the right is that of $𝑉$ ; we use the same notation for both. Taking conjugates of both sides gives the equivalent form

⟨ 𝐰, 𝑇 𝐯 ⟩ = ⟨ 𝑇^{*} 𝐰, 𝐯 ⟩,

which is how the lecture writes the definition. So $𝑇$ can be moved out of either slot into the other, at the price of a star. We did not ask $𝑆$ to be linear; linearity comes for free.

Proposition 6.6.9 (Adjoints are unique and linear). Let $𝑉$ and $𝑊$ be inner product spaces and $𝑇 \in L (𝑉, 𝑊)$ .

(a) $𝑇$ has at most one adjoint.
(b)Every adjoint of $𝑇$ is linear. So $𝑇^{*} \in L (𝑊, 𝑉)$ whenever $𝑇^{*}$ exists.

Proof.routine

Uses. Definition 6.6.8 (Adjoint), Proposition 6.1.37 (Inner products distinguish vectors), Proposition 6.1.4 (Sesquilinearity)

Techniques. Uniqueness via two candidates (Technique 1.1.8)

Hint 1. For (a), let

𝑆

and

𝑆^{'}

be adjoints and fix

𝐰

. What do you know about

⟨ 𝐯, 𝑆 𝐰 ⟩

and

⟨ 𝐯, 𝑆^{'} 𝐰 ⟩

𝐯

varies?

Hint 2. Proposition 6.1.37 says that a vector is determined by its inner products with all vectors. Here the unknown vectors sit in the second slot, so conjugate first.

Hint 3. For (b), compute

⟨ 𝐯, 𝑆 (𝑎 𝐰 + 𝑏 𝐰^{'}) ⟩

with the defining identity and Proposition 6.1.4, and compare it with

⟨ 𝐯, 𝑎 𝑆 𝐰 + 𝑏 𝑆 𝐰^{'} ⟩

Sketch. (a) For each

𝐰

⟨ 𝐯, 𝑆 𝐰 ⟩ =

⟨ 𝑇 𝐯, 𝐰 ⟩ =

⟨ 𝐯, 𝑆^{'} 𝐰 ⟩

for every

𝐯

, so

𝑆 𝐰 = 𝑆^{'} 𝐰

by Proposition 6.1.37. (b) Both

⟨ 𝐯, 𝑆 (𝑎 𝐰 + 𝑏 𝐰^{'}) ⟩

and

⟨ 𝐯, 𝑎 𝑆 𝐰 + 𝑏 𝑆 𝐰^{'} ⟩

equal

𝑎 ⟨ 𝑇 𝐯, 𝐰 ⟩ +

𝑏 ⟨ 𝑇 𝐯, 𝐰^{'} ⟩

for every

𝐯

; apply Proposition 6.1.37 again.

(a) Let $𝑆, 𝑆^{'} : 𝑊 \to 𝑉$ be adjoints of $𝑇$ , and fix $𝐰 \in 𝑊$ . For every $𝐯 \in 𝑉$ ,

⟨ 𝐯, 𝑆 𝐰 ⟩ = ⟨ 𝑇 𝐯, 𝐰 ⟩ = ⟨ 𝐯, 𝑆^{'} 𝐰 ⟩ .

Taking conjugates, $⟨ 𝑆 𝐰, 𝐯 ⟩ =$ $⟨ 𝑆^{'} 𝐰, 𝐯 ⟩$ for every $𝐯 \in 𝑉$ , so $𝑆 𝐰 = 𝑆^{'} 𝐰$ by Proposition 6.1.37. As $𝐰$ was arbitrary, $𝑆 = 𝑆^{'}$ .

(b) Let $𝑆$ be an adjoint of $𝑇$ , and let $𝐰, 𝐰^{'} \in 𝑊$ and $𝑎, 𝑏 \in 𝔽$ . For every $𝐯 \in 𝑉$ , the defining identity and Proposition 6.1.4 give

\begin{aligned} ⟨ 𝐯, 𝑆 (𝑎 𝐰 + 𝑏 𝐰^{'}) ⟩ & = ⟨ 𝑇 𝐯, 𝑎 𝐰 + 𝑏 𝐰^{'} ⟩ \\ = 𝑎 ⟨ 𝑇 𝐯, 𝐰 ⟩ + 𝑏 ⟨ 𝑇 𝐯, 𝐰^{'} ⟩ \\ = 𝑎 ⟨ 𝐯, 𝑆 𝐰 ⟩ + 𝑏 ⟨ 𝐯, 𝑆 𝐰^{'} ⟩ \\ = ⟨ 𝐯, 𝑎 𝑆 𝐰 + 𝑏 𝑆 𝐰^{'} ⟩ . \end{aligned}

As in (a), taking conjugates and applying Proposition 6.1.37 gives $𝑆 (𝑎 𝐰 + 𝑏 𝐰^{'}) =$ $𝑎 𝑆 𝐰 +$ $𝑏 𝑆 𝐰^{'}$ . □

Neither part used a dimension: uniqueness and linearity hold in every inner product space, and this is the order in which the lecture proceeds. In practice it means that adjoints are found by guessing a formula and checking the defining identity; uniqueness does the rest.

Example 6.6.10 (The adjoint of a matrix). Let $𝐴 \in 𝔽^{𝑚 \times 𝑛}$ , and give $𝔽^{𝑛}$ and $𝔽^{𝑚}$ the dot product. By Proposition 6.3.27(d), $(𝐴 𝐱) \cdot$ $𝐲 =$ $𝐱 \cdot$ $(𝐴^{*} 𝐲)$ for all $𝐱$ and $𝐲$ ; that is, $𝑇_{𝐴^{*}}$ is an adjoint of $𝑇_{𝐴}$ . By Proposition 6.6.9,

{(𝑇_{𝐴})}^{*} = 𝑇_{𝐴^{*}} .

Over $ℝ$ this is $𝑇_{𝐴^{𝖳}}$ . Over $ℂ$ the transpose would be wrong. For $𝐴 = (𝑖) \in ℂ^{1 \times 1}$ , $𝑇_{𝐴}$ is multiplication by $𝑖$ on $ℂ$ , and $(𝑖 𝑥) \cdot 𝑦 = 𝑖 𝑥 𝑦$ , while $𝑥 \cdot$ $(𝑖 𝑦) =$ $𝑥 𝑖 𝑦 =$ $- 𝑖 𝑥 𝑦$ . The adjoint of multiplication by $𝑖$ is multiplication by $𝑖 = - 𝑖$ , in line with $𝐴^{*} = (𝑖)$ .

So the star on a matrix and the star on a map agree, as long as $𝔽^{𝑛}$ and $𝔽^{𝑚}$ carry the dot product. The adjoint depends on the inner products as well as on the map (Exercise 6.6.31).

Technique 6.6.11 (Find the adjoint by isolating the input). To compute $𝑇^{*} 𝐰$ , start from $⟨ 𝑇 𝐯, 𝐰 ⟩$ , substitute the formula for $𝑇$ , and rearrange until the expression reads $⟨ 𝐯, 𝐮 ⟩$ , with $𝐯$ alone in the first slot and $𝐮$ built from $𝐰$ only. Then $𝑇^{*} 𝐰 = 𝐮$ by Proposition 6.6.9. Scalars that move into the second slot pick up a bar. The same move proves identities: to show that a given map $𝑆$ is $𝑇^{*}$ , check $⟨ 𝑇 𝐯, 𝐰 ⟩ =$ $⟨ 𝐯, 𝑆 𝐰 ⟩$ for all $𝐯$ and $𝐰$ , without ever writing a formula for $𝑇^{*}$ . This is the analogue for adjoints of Technique 2.5.16 and Technique 6.4.7.

Reach for it when computing the adjoint of an explicit map, and when proving identities such as ${(𝑅 𝑇)}^{*} = 𝑇^{*} 𝑅^{*}$ . The canonical example is Example 6.6.13.

Used in 5 places and 1 practice exercise

Exercise 6.6.17, Proposition 6.6.18, Exercise 6.6.31, Exercise 6.6.33, Proposition 8.4.10; 1 practice exercise.

Example 6.6.12 (An adjoint from $ℂ^{3}$ to $ℂ^{2}$ ). Let $𝑇 : ℂ^{3} \to ℂ^{2}$ , $𝑇 (𝑧_{1}, 𝑧_{2}, 𝑧_{3}) =$ $(𝑧_{1} + 𝑖 𝑧_{3}, 2 𝑧_{2} - 𝑧_{3})$ , with dot products. For $𝐳 \in ℂ^{3}$ and $𝐰 \in ℂ^{2}$ ,

\begin{aligned} ⟨ 𝑇 𝐳, 𝐰 ⟩ & = (𝑧_{1} + 𝑖 𝑧_{3}) 𝑤_{1} + (2 𝑧_{2} - 𝑧_{3}) 𝑤_{2} \\ = 𝑧_{1} 𝑤_{1} + 𝑧_{2} \cdot 2 𝑤_{2} + 𝑧_{3} (𝑖 𝑤_{1} - 𝑤_{2}) \\ = 𝑧_{1} 𝑤_{1} + 𝑧_{2} 2 𝑤_{2} + 𝑧_{3} - 𝑖 𝑤_{1} - 𝑤_{2}, \end{aligned}

which is $⟨ 𝐳, (𝑤_{1}, 2 𝑤_{2}, - 𝑖 𝑤_{1} - 𝑤_{2}) ⟩$ . So $𝑇^{*} (𝑤_{1}, 𝑤_{2}) =$ $(𝑤_{1}, 2 𝑤_{2}, - 𝑖 𝑤_{1} - 𝑤_{2})$ . As Example 6.6.10 predicts, this is multiplication by the conjugate transpose of the matrix of $𝑇$ :

{(\begin{matrix} 1 & 0 & 𝑖 \\ 0 & 2 & - 1 \end{matrix})}^{*} = (\begin{matrix} 1 & 0 \\ 0 & 2 \\ - 𝑖 & - 1 \end{matrix}) .

Example 6.6.13 (Rank-one maps). Fix $𝐮 \in 𝑉$ and $𝐱 \in 𝑊$ , and let $𝑇 𝐯 = ⟨ 𝐯, 𝐮 ⟩ 𝐱$ . For $𝐯 \in 𝑉$ and $𝐰 \in 𝑊$ ,

⟨ 𝑇 𝐯, 𝐰 ⟩ = ⟨ 𝐯, 𝐮 ⟩ ⟨ 𝐱, 𝐰 ⟩ = ⟨ 𝐯, ⟨ 𝐱, 𝐰 ⟩ 𝐮 ⟩ = ⟨ 𝐯, ⟨ 𝐰, 𝐱 ⟩ 𝐮 ⟩,

using Proposition 6.1.4 to move the scalar $⟨ 𝐱, 𝐰 ⟩$ into the second slot. So

𝑇^{*} 𝐰 = ⟨ 𝐰, 𝐱 ⟩ 𝐮 .

The map $𝑇$ reads off the component of its input along $𝐮$ and writes the result along $𝐱$ ; its adjoint reads along $𝐱$ and writes along $𝐮$ . For $𝑉 = 𝔽^{𝑛}$ and $𝑊 = 𝔽^{𝑚}$ with dot products, $𝑇 𝐯 =$ $(𝐮^{*} 𝐯) 𝐱 =$ $𝐱 𝐮^{*} 𝐯$ is multiplication by the rank-one matrix $𝐱 𝐮^{*}$ , and $𝑇^{*}$ is multiplication by $𝐮 𝐱^{*} = {(𝐱 𝐮^{*})}^{*}$ . No dimension assumption was needed.

Example 6.6.14 (Multiplication operators). Let $𝑉 = 𝐶 ([0, 1], ℂ)$ with $⟨ 𝑓, 𝑔 ⟩ =$ $\int_{0}^{1} 𝑓 (𝑡) 𝑔 (𝑡) d 𝑡$ (Proposition 6.1.13). For a continuous function $ℎ : [0, 1] \to ℂ$ , let $𝑀_{ℎ} 𝑓 = ℎ 𝑓$ , the pointwise product. Then

⟨ ℎ 𝑓, 𝑔 ⟩ = \int_{0}^{1} ℎ (𝑡) 𝑓 (𝑡) 𝑔 (𝑡) d 𝑡 = \int_{0}^{1} 𝑓 (𝑡) ℎ (𝑡) 𝑔 (𝑡) d 𝑡 = ⟨ 𝑓, ℎ 𝑔 ⟩,

so $𝑀_{ℎ}$ has an adjoint, $𝑀_{ℎ}^{*} = 𝑀_{ℎ}$ , although $𝑉$ is infinite-dimensional. This is the continuous version of $diag {(𝑑_{1}, \dots, 𝑑_{𝑛})}^{*} =$ $diag (𝑑_{1}, \dots, 𝑑_{𝑛})$ : $𝑀_{ℎ}$ scales the value at each point $𝑡$ by $ℎ (𝑡)$ , as a diagonal matrix scales each coordinate.

Existence, and the matrix of the adjoint

In finite dimensions adjoints always exist, and orthonormal bases compute them.

Theorem 6.6.15 (Existence of adjoints). Let $𝑉$ and $𝑊$ be finite-dimensional inner product spaces and $𝑇 \in L (𝑉, 𝑊)$ .

(a) $𝑇$ has exactly one adjoint $𝑇^{*}$ , and $𝑇^{*} \in L (𝑊, 𝑉)$ .
(b)If $ℬ︀ = 𝐞_{1}, \dots, 𝐞_{𝑛}$ and $𝒞︀ = 𝐟_{1}, \dots, 𝐟_{𝑚}$ are orthonormal bases of $𝑉$ and $𝑊$ , then
${[𝑇^{*}]}_{ℬ︀ 𝒞︀} = {({[𝑇]}_{𝒞︀ ℬ︀})}^{*} .$

Proof (via the Riesz representation theorem).standardBeyond 21-242

Uses. Theorem 6.6.3 (Riesz representation theorem), Proposition 6.6.9 (Adjoints are unique and linear), Proposition 6.3.18 (Coordinates in an orthonormal basis), Definition 2.3.1 (Matrix of a linear map), Definition 6.3.26 (Conjugate transpose)

Techniques. Take an inner product to isolate a coefficient (Technique 6.3.12)

Hint 1. For (a), fix

𝐰

and look at

𝐯 \mapsto ⟨ 𝑇 𝐯, 𝐰 ⟩

. What kind of function of

𝐯

is it?

Hint 2. Apply Theorem 6.6.3 to that functional and call the resulting vector

𝑇^{*} 𝐰

. Linearity and uniqueness are Proposition 6.6.9.

Hint 3. For (b), the

(𝑖, 𝑗)

entry of

{[𝑇]}_{𝒞︀ ℬ︀}

is the

𝑖

th coordinate of

𝑇 𝐞_{𝑗}

in the orthonormal basis

𝒞︀

, which is an inner product (Proposition 6.3.18).

Sketch. For fixed

𝐰

𝐯 \mapsto ⟨ 𝑇 𝐯, 𝐰 ⟩

is a linear functional on

𝑉

; Theorem 6.6.3 represents it as

⟨ \cdot, 𝐮 ⟩

, and

𝑇^{*} 𝐰 = 𝐮

satisfies the defining identity. In orthonormal bases,

{[𝑇]}_{𝒞︀ ℬ︀}

has entries

⟨ 𝑇 𝐞_{𝑗}, 𝐟_{𝑖} ⟩

, and

{[𝑇^{*}]}_{ℬ︀ 𝒞︀}

has entries

⟨ 𝑇^{*} 𝐟_{𝑖}, 𝐞_{𝑗} ⟩ =

⟨ 𝑇 𝐞_{𝑗}, 𝐟_{𝑖} ⟩

(a) Fix $𝐰 \in 𝑊$ . The function $𝜑_{𝐰} : 𝑉 \to 𝔽$ , $𝜑_{𝐰} (𝐯) =$ $⟨ 𝑇 𝐯, 𝐰 ⟩$ , is a linear functional, since $𝑇$ is linear and the inner product is linear in its first slot. By Theorem 6.6.3 there is exactly one vector of $𝑉$ , which we call $𝑇^{*} 𝐰$ , with

⟨ 𝑇 𝐯, 𝐰 ⟩ = ⟨ 𝐯, 𝑇^{*} 𝐰 ⟩ for all 𝐯 \in 𝑉 .

Doing this for every $𝐰$ defines a function $𝑇^{*} : 𝑊 \to 𝑉$ that satisfies Definition 6.6.8. By Proposition 6.6.9 it is the only adjoint of $𝑇$ , and it is linear.

(b) Let $𝐴 = {[𝑇]}_{𝒞︀ ℬ︀}$ . Its $𝑗$ th column is ${[𝑇 𝐞_{𝑗}]}_{𝒞︀}$ , so by Proposition 6.3.18(b) its $(𝑖, 𝑗)$ entry is $𝑎_{𝑖 𝑗} =$ $⟨ 𝑇 𝐞_{𝑗}, 𝐟_{𝑖} ⟩$ . In the same way, the $(𝑗, 𝑖)$ entry of ${[𝑇^{*}]}_{ℬ︀ 𝒞︀}$ is the $𝑗$ th coordinate of $𝑇^{*} 𝐟_{𝑖}$ in $ℬ︀$ , namely

⟨ 𝑇^{*} 𝐟_{𝑖}, 𝐞_{𝑗} ⟩ = ⟨ 𝐞_{𝑗}, 𝑇^{*} 𝐟_{𝑖} ⟩ = ⟨ 𝑇 𝐞_{𝑗}, 𝐟_{𝑖} ⟩ = 𝑎_{𝑖 𝑗} .

So ${[𝑇^{*}]}_{ℬ︀ 𝒞︀} = 𝐴^{*}$ (Definition 6.3.26). □

Proof (via the conjugate transpose).routine

Uses. Theorem 6.4.20 (Orthonormal bases exist), Theorem 2.3.20 (Linear maps correspond to matrices), Theorem 2.3.9 (The matrix of a map computes the map), Theorem 6.3.22 (Parseval’s identity), Proposition 6.3.27 (Properties of the conjugate transpose), Proposition 6.6.9 (Adjoints are unique and linear)

Techniques. Translate between maps and matrices (Technique 2.3.27)

Hint 1. Fix orthonormal bases. In coordinates, inner products become dot products (Theorem 6.3.22), and

𝑇

becomes multiplication by

𝐴 = {[𝑇]}_{𝒞︀ ℬ︀}

. For matrices you already know an adjoint.

Hint 2. Let

𝑆 \in L (𝑊, 𝑉)

be the map with

{[𝑆]}_{ℬ︀ 𝒞︀} = 𝐴^{*}

, and compute

⟨ 𝑇 𝐯, 𝐰 ⟩

in coordinates.

Sketch. Let

𝑆

be the linear map with

{[𝑆]}_{ℬ︀ 𝒞︀} = 𝐴^{*}

. With

𝐱 = {[𝐯]}_{ℬ︀}

and

𝐲 = {[𝐰]}_{𝒞︀}

, Parseval’s identity gives

⟨ 𝑇 𝐯, 𝐰 ⟩ =

(𝐴 𝐱) \cdot

𝐲 =

𝐱 \cdot

(𝐴^{*} 𝐲) =

⟨ 𝐯, 𝑆 𝐰 ⟩

. So

𝑆

is an adjoint, it is the only one by Proposition 6.6.9, and (b) holds by construction.

Orthonormal bases of $𝑉$ and $𝑊$ exist by Theorem 6.4.20. We prove (a) and (b) together, for arbitrary orthonormal bases $ℬ︀$ and $𝒞︀$ . Let $𝐴 =$ ${[𝑇]}_{𝒞︀ ℬ︀} \in$ $𝔽^{𝑚 \times 𝑛}$ . By Theorem 2.3.20 there is a linear map $𝑆 : 𝑊 \to 𝑉$ with ${[𝑆]}_{ℬ︀ 𝒞︀} = 𝐴^{*}$ . Let $𝐯 \in 𝑉$ and $𝐰 \in 𝑊$ , and write $𝐱 = {[𝐯]}_{ℬ︀}$ and $𝐲 = {[𝐰]}_{𝒞︀}$ . By Theorem 2.3.9, ${[𝑇 𝐯]}_{𝒞︀} = 𝐴 𝐱$ and ${[𝑆 𝐰]}_{ℬ︀} = 𝐴^{*} 𝐲$ . Parseval’s identity (Theorem 6.3.22(a)) computes the inner products of $𝑉$ and of $𝑊$ as dot products of coordinate vectors, so

⟨ 𝑇 𝐯, 𝐰 ⟩ = (𝐴 𝐱) \cdot 𝐲 = 𝐱 \cdot (𝐴^{*} 𝐲) = ⟨ 𝐯, 𝑆 𝐰 ⟩,

where the middle step is Proposition 6.3.27(d). So $𝑆$ is an adjoint of $𝑇$ . By Proposition 6.6.9 it is the only one, so $𝑇^{*} = 𝑆 \in L (𝑊, 𝑉)$ and ${[𝑇^{*}]}_{ℬ︀ 𝒞︀} = 𝐴^{*}$ . □

The second proof is the lecture’s route: orthonormal coordinates turn $𝑉$ and $𝑊$ into $𝔽^{𝑛}$ and $𝔽^{𝑚}$ with their dot products, and there the adjoint is the conjugate transpose (Example 6.6.10). The first proof never chooses a basis of $𝑊$ , so it shows that (a) holds whenever $𝑉$ alone is finite-dimensional. Without any finite-dimensionality, adjoints can fail to exist: on the space $𝑃$ of Non-example 6.6.7, the operator $𝑇 𝑝 = 𝑝 (0) \cdot 1$ has none, since an adjoint would give $⟨ 𝑝, 𝑇^{*} 1 ⟩ =$ $⟨ 𝑇 𝑝, 1 ⟩ =$ $𝑝 (0)$ for every $𝑝$ , so that $𝑇^{*} 1$ would represent evaluation at $0$ .

Intuition (Reading and writing). Here is a formula for $𝑇^{*}$ that needs only an orthonormal basis $𝐞_{1}, \dots, 𝐞_{𝑛}$ of $𝑉$ . Every $𝐯 \in 𝑉$ is $\sum_{𝑗} ⟨ 𝐯, 𝐞_{𝑗} ⟩ 𝐞_{𝑗}$ (Proposition 6.3.18), so

𝑇 𝐯 = \sum_{𝑗 = 1}^{𝑛} ⟨ 𝐯, 𝐞_{𝑗} ⟩ 𝑇 𝐞_{𝑗} .

That is, $𝑇$ is a sum of $𝑛$ rank-one maps, the $𝑗$ th of which reads the $𝐞_{𝑗}$ -coordinate of its input and writes it along $𝑇 𝐞_{𝑗}$ . By Example 6.6.13, the adjoint of the $𝑗$ th map reads along $𝑇 𝐞_{𝑗}$ and writes along $𝐞_{𝑗}$ , and the adjoint of a sum is the sum of the adjoints (Proposition 6.6.18(a) below). So

𝑇^{*} 𝐰 = \sum_{𝑗 = 1}^{𝑛} ⟨ 𝐰, 𝑇 𝐞_{𝑗} ⟩ 𝐞_{𝑗} .

One can also read this off directly: the $𝐞_{𝑗}$ -coordinate of $𝑇^{*} 𝐰$ is $⟨ 𝑇^{*} 𝐰, 𝐞_{𝑗} ⟩ =$ $⟨ 𝐞_{𝑗}, 𝑇^{*} 𝐰 ⟩ =$ $⟨ 𝑇 𝐞_{𝑗}, 𝐰 ⟩ =$ $⟨ 𝐰, 𝑇 𝐞_{𝑗} ⟩$ . For $𝑇 = 𝑇_{𝐴}$ on $𝔽^{𝑛}$ with the standard basis, the vectors $𝑇 𝐞_{𝑗}$ are the columns $𝐚_{𝑗}$ of $𝐴$ , and the formula says that the $𝑗$ th entry of $𝐴^{*} 𝐰$ is $𝐰 \cdot 𝐚_{𝑗}$ : the rows of $𝐴^{*}$ are the conjugated columns of $𝐴$ .

The hypothesis that the bases are orthonormal in Theorem 6.6.15(b) cannot be dropped. In other bases the matrix of $𝑇^{*}$ is usually not the conjugate transpose of the matrix of $𝑇$ .

Example 6.6.16 (The adjoint of differentiation). Give $𝑉 = {ℝ [𝑥]}_{\leq 1}$ the inner product $⟨ 𝑝, 𝑞 ⟩ =$ $\int_{0}^{1} 𝑝 (𝑡) 𝑞 (𝑡) d 𝑡$ , and let $𝐷 (𝑎 + 𝑏 𝑥) = 𝑏$ be differentiation (Example 2.1.9). In the basis $ℬ︀ = 1, 𝑥$ ,

{[𝐷]}_{ℬ︀} = (\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix}), and {({[𝐷]}_{ℬ︀})}^{𝖳} = (\begin{matrix} 0 & 0 \\ 1 & 0 \end{matrix})

is the matrix of the map $𝑎 + 𝑏 𝑥 \mapsto 𝑎 𝑥$ . That map is not $𝐷^{*}$ : for $𝑝 = 𝑥$ and $𝑞 = 1$ , $⟨ 𝐷 𝑝, 𝑞 ⟩ = ⟨ 1, 1 ⟩ = 1$ , while $⟨ 𝑝, 1 \cdot 𝑥 ⟩ =$ $⟨ 𝑥, 𝑥 ⟩ =$ $1 / 3$ . The basis $ℬ︀$ is not orthonormal, since $⟨ 1, 𝑥 ⟩ = 1 / 2$ . To find $𝐷^{*}$ , use the orthonormal basis $𝒪︀ = 𝑒_{1}, 𝑒_{2}$ with $𝑒_{1} = 1$ and $𝑒_{2} = \sqrt{3} (2 𝑥 - 1)$ of Example 6.6.5. Since $𝐷 𝑒_{1} = 0$ and $𝐷 𝑒_{2} = 2 \sqrt{3} = 2 \sqrt{3} 𝑒_{1}$ ,

{[𝐷]}_{𝒪︀} = (\begin{matrix} 0 & 2 \sqrt{3} \\ 0 & 0 \end{matrix}), {[𝐷^{*}]}_{𝒪︀} = (\begin{matrix} 0 & 0 \\ 2 \sqrt{3} & 0 \end{matrix})

by Theorem 6.6.15(b). So $𝐷^{*} 𝑒_{1} =$ $2 \sqrt{3} 𝑒_{2} =$ $6 (2 𝑥 - 1)$ and $𝐷^{*} 𝑒_{2} = 0$ . Writing $𝑎 +$ $𝑏 𝑥 =$ $(𝑎 + 𝑏 / 2) 𝑒_{1} +$ $(𝑏 / (2 \sqrt{3})) 𝑒_{2}$ gives

𝐷^{*} (𝑎 + 𝑏 𝑥) = (2 𝑎 + 𝑏) (6 𝑥 - 3), {[𝐷^{*}]}_{ℬ︀} = (\begin{matrix} - 6 & - 3 \\ 12 & 6 \end{matrix}) .

As a check, for $𝑝 = 𝑎 + 𝑏 𝑥$ and $𝑞 = 𝑐 + 𝑑 𝑥$ , $⟨ 𝐷 𝑝, 𝑞 ⟩ =$ $\int_{0}^{1} 𝑏 (𝑐 + 𝑑 𝑡) d 𝑡 =$ $𝑏 (𝑐 + 𝑑 / 2)$ , and since $\int_{0}^{1} (6 𝑡 - 3) d 𝑡 = 0$ and $\int_{0}^{1} 𝑡 (6 𝑡 - 3) d 𝑡 =$ $1 / 2$ , $⟨ 𝑝, 𝐷^{*} 𝑞 ⟩ =$ $(2 𝑐 + 𝑑) 𝑏 / 2$ , the same number.

So ${[𝐷^{*}]}_{ℬ︀}$ has nothing to do with ${({[𝐷]}_{ℬ︀})}^{𝖳}$ . Nor is $𝐷^{*}$ a differentiation-like map, such as $- 𝐷$ : integrating by parts, $⟨ 𝐷 𝑝, 𝑞 ⟩ =$ $𝑝 (1) 𝑞 (1) -$ $𝑝 (0) 𝑞 (0) -$ $⟨ 𝑝, 𝐷 𝑞 ⟩$ , and the boundary terms are what $𝐷^{*}$ has to absorb (compare Exercise 6.6.39, where there are no boundary terms). Exercise 6.6.46 gives the general rule for a basis that is not orthonormal: ${[𝑇^{*}]}_{ℬ︀} =$ $𝐺^{- 1} {({[𝑇]}_{ℬ︀})}^{*} 𝐺$ , where $𝐺$ is the Gram matrix of $ℬ︀$ .

Intuition (What the adjoint is). In coordinates. In orthonormal bases, $𝑇^{*}$ is the conjugate transpose (Theorem 6.6.15(b)). The transpose of a matrix is defined by moving entries around, so it depends on the basis, and in other bases it describes some unrelated map (Example 6.6.16). The adjoint is defined by the inner product alone, and the conjugate transpose is what it looks like in orthonormal coordinates. This is the sense in which the adjoint is the right generalization of the transpose. What makes the transpose useful, in the normal equations, in $𝐴^{*} 𝐴$ , in projection matrices, is the identity $(𝐴 𝐱) \cdot$ $𝐲 =$ $𝐱 \cdot$ $(𝐴^{*} 𝐲)$ , and that identity is the definition of $𝑇^{*}$ . The index shuffle is a consequence that holds only in orthonormal bases. Over $ℂ$ the conjugate is forced, as Example 6.6.10 shows.

As a measurement. For a unit vector $𝐰$ , $⟨ 𝑇 𝐯, 𝐰 ⟩$ is the component of the output $𝑇 𝐯$ along $𝐰$ , and the definition says that it is also the component of the input $𝐯$ along $𝑇^{*} 𝐰$ . By Cauchy–Schwarz, among unit vectors $𝐯$ the real part of $⟨ 𝑇 𝐯, 𝐰 ⟩$ is largest when $𝐯$ points along $𝑇^{*} 𝐰$ (Exercise 6.6.42). So $𝑇^{*} 𝐰$ points toward the inputs that $𝑇$ pushes furthest in the direction $𝐰$ , and its length says how far.

Reading and writing. By Example 6.6.13, $𝑇^{*}$ exchanges the vectors along which $𝑇$ reads and writes. Every $𝑇$ is a sum of such rank-one maps, and $𝑇^{*}$ exchanges the roles in each.

Without an inner product. A matrix $𝐴 \in 𝐹^{𝑚 \times 𝑛}$ over any field still has a transpose, and Section 13.1 gives it a meaning: it is the matrix of the dual map, which acts on linear functionals rather than on vectors. In an inner product space, Theorem 6.6.3 identifies functionals with vectors, and the dual map becomes the adjoint (Section 13.2).

Aside (Gradients flow backward through the adjoint).Beyond 21-242 Over $ℝ$ , the derivative of a differentiable function $𝑓 : ℝ^{𝑛} \to ℝ$ at $𝐯$ is a linear functional, $𝐡 \mapsto \nabla 𝑓 (𝐯) \cdot 𝐡$ , and its Riesz vector is the gradient: $𝑓 (𝐯 + 𝐡) \approx$ $𝑓 (𝐯) +$ $\nabla 𝑓 (𝐯) \cdot$ $𝐡$ for small $𝐡$ . Now let $𝑓 (𝐯) = 𝑔 (𝐴 𝐯)$ for a matrix $𝐴$ and a differentiable $𝑔 : ℝ^{𝑚} \to ℝ$ . Then

𝑓 (𝐯 + 𝐡) = 𝑔 (𝐴 𝐯 + 𝐴 𝐡) \approx 𝑔 (𝐴 𝐯) + \nabla 𝑔 (𝐴 𝐯) \cdot (𝐴 𝐡) = 𝑓 (𝐯) + (𝐴^{𝖳} \nabla 𝑔 (𝐴 𝐯)) \cdot 𝐡,

so $\nabla 𝑓 (𝐯) =$ $𝐴^{𝖳} \nabla 𝑔 (𝐴 𝐯)$ . The map $𝐴$ carries inputs forward, and its adjoint carries gradients backward. A neural network composes many linear maps with simple nonlinear ones, and training it requires the gradient of a loss with respect to the inputs of every layer. These are computed from the last layer to the first, multiplying by transposed weight matrices along the way; this is backpropagation (Section 16.6).

Exercise 6.6.17.routine Give $𝔽^{𝑛 \times 𝑛}$ the Frobenius inner product $⟨ 𝑋, 𝑌 ⟩ = tr (𝑌^{*} 𝑋)$ (Proposition 6.1.16), and fix $𝐴 \in 𝔽^{𝑛 \times 𝑛}$ .

(a)Find the adjoint of $𝑅_{𝐴} (𝑋) = 𝑋 𝐴$ .
(b)Find the adjoint of $𝑋 \mapsto 𝑋^{𝖳}$ .
(c)For $𝔽 = ℂ$ , explain why $𝑋 \mapsto 𝑋^{*}$ has no adjoint in the sense of Definition 6.6.8.

Solution.

Uses. Proposition 6.1.16 (Frobenius inner product), Proposition 2.4.28 (Cyclicity of the trace), Proposition 6.3.27 (Properties of the conjugate transpose), Definition 6.6.8 (Adjoint)

Techniques. Find the adjoint by isolating the input (Technique 6.6.11)

Hint 1. For (a), use

tr (𝐵 𝐶) = tr (𝐶 𝐵)

to move

𝐴

next to

𝑌^{*}

, and recognize

𝐴 𝑌^{*}

{(𝑌 𝐴^{*})}^{*}

Hint 2. For (b), use the entrywise formula

⟨ 𝑋, 𝑌 ⟩ =

\sum_{𝑖, 𝑗} 𝑥_{𝑖 𝑗} 𝑦_{𝑖 𝑗}

and rename the indices.

Sketch. (a)

tr (𝑌^{*} 𝑋 𝐴) =

tr (𝐴 𝑌^{*} 𝑋) =

tr ({(𝑌 𝐴^{*})}^{*} 𝑋)

, so

𝑅_{𝐴}^{*} = 𝑅_{𝐴^{*}}

. (b)

⟨ 𝑋^{𝖳}, 𝑌 ⟩ =

\sum_{𝑖, 𝑗} 𝑥_{𝑗 𝑖} 𝑦_{𝑖 𝑗} =

⟨ 𝑋, 𝑌^{𝖳} ⟩

, so the transpose map is its own adjoint. (c)

{(𝑖 𝑋)}^{*} = - 𝑖 𝑋^{*}

, so the map is not linear over

ℂ

, and Definition 6.6.8 applies only to linear maps.

(a) By Proposition 2.4.28 and Proposition 6.3.27,

⟨ 𝑋 𝐴, 𝑌 ⟩ = tr (𝑌^{*} 𝑋 𝐴) = tr (𝐴 𝑌^{*} 𝑋) = tr ({(𝑌 𝐴^{*})}^{*} 𝑋) = ⟨ 𝑋, 𝑌 𝐴^{*} ⟩,

since ${(𝑌 𝐴^{*})}^{*} =$ ${(𝐴^{*})}^{*} 𝑌^{*} =$ $𝐴 𝑌^{*}$ . So $𝑅_{𝐴}^{*} (𝑌) = 𝑌 𝐴^{*}$ , that is, $𝑅_{𝐴}^{*} = 𝑅_{𝐴^{*}}$ .

(b) By Proposition 6.1.16, $⟨ 𝑋, 𝑌 ⟩ =$ $\sum_{𝑖, 𝑗} 𝑥_{𝑖 𝑗} 𝑦_{𝑖 𝑗}$ . The $(𝑖, 𝑗)$ entry of $𝑋^{𝖳}$ is $𝑥_{𝑗 𝑖}$ , so

⟨ 𝑋^{𝖳}, 𝑌 ⟩ = \sum_{𝑖, 𝑗} 𝑥_{𝑗 𝑖} 𝑦_{𝑖 𝑗} = \sum_{𝑖, 𝑗} 𝑥_{𝑖 𝑗} 𝑦_{𝑗 𝑖} = ⟨ 𝑋, 𝑌^{𝖳} ⟩,

after renaming $𝑖$ and $𝑗$ . So the transpose map is its own adjoint.

(c) ${(𝑖 𝑋)}^{*} =$ $𝑖 𝑋^{*} =$ $- 𝑖 𝑋^{*}$ (Proposition 6.3.27(a)), so $𝑋 \mapsto 𝑋^{*}$ is not linear over $ℂ$ (unless $𝑛 = 0$ ). Adjoints are defined for linear maps only. (Over $ℝ$ the map is the transpose, covered by (b).)

6.6.3The algebra of adjoints

The rules for computing with adjoints are those of Proposition 6.3.27 for matrices. We state them with as few hypotheses as possible: each part asks only that the adjoints on its right-hand side exist, which is automatic in finite dimensions (Theorem 6.6.15).

Proposition 6.6.18 (Properties of the adjoint). Let $𝑈$ , $𝑉$ , $𝑊$ be inner product spaces, let $𝑆, 𝑇 \in L (𝑉, 𝑊)$ and $𝑅 \in L (𝑊, 𝑈)$ , and let $𝑎 \in 𝔽$ . Suppose that $𝑆$ , $𝑇$ , and $𝑅$ have adjoints. Then:

(a) $𝑆 + 𝑇$ has an adjoint, and ${(𝑆 + 𝑇)}^{*} = 𝑆^{*} + 𝑇^{*}$ ;
(b) $𝑎 𝑇$ has an adjoint, and ${(𝑎 𝑇)}^{*} = 𝑎 𝑇^{*}$ ;
(c) $𝑇^{*}$ has an adjoint, and ${(𝑇^{*})}^{*} = 𝑇$ ;
(d) $𝑅 𝑇$ has an adjoint, and ${(𝑅 𝑇)}^{*} = 𝑇^{*} 𝑅^{*}$ ;
(e) ${id}_{𝑉}$ has an adjoint, and ${id}_{𝑉}^{*} = {id}_{𝑉}$ ;
(f)if $𝑇$ is invertible and $𝑇^{- 1}$ has an adjoint, then $𝑇^{*}$ is invertible and ${(𝑇^{*})}^{- 1} = {(𝑇^{- 1})}^{*}$ .

Proof (from the defining identity).routine

Uses. Definition 6.6.8 (Adjoint), Proposition 6.6.9 (Adjoints are unique and linear), Proposition 6.1.4 (Sesquilinearity), Definition 2.5.12 (Invertible)

Techniques. Find the adjoint by isolating the input (Technique 6.6.11), Guess the inverse, then check both products (Technique 2.5.16)

Hint 1. Each part names a candidate for the adjoint. Check the defining identity

⟨ 𝑋 𝐯, 𝐰 ⟩ =

⟨ 𝐯, 𝑌 𝐰 ⟩

for the candidate

𝑌

and let uniqueness do the rest.

Hint 2. For (b), pulling

𝑎

out of the second slot produces

𝑎 = 𝑎

. For (c), the defining identity of an adjoint of

𝑇^{*} : 𝑊 \to 𝑉

reads

⟨ 𝑇^{*} 𝐰, 𝐯 ⟩ =

⟨ 𝐰, 𝑆 𝐯 ⟩

; conjugate the identity for

𝑇

Hint 3. For (f), apply (d) and (e) to

𝑇^{- 1} 𝑇 = {id}_{𝑉}

and

𝑇 𝑇^{- 1} = {id}_{𝑊}

Sketch. (a)

⟨ (𝑆 + 𝑇) 𝐯, 𝐰 ⟩ =

⟨ 𝐯, 𝑆^{*} 𝐰 ⟩ +

⟨ 𝐯, 𝑇^{*} 𝐰 ⟩

. (b)

⟨ 𝑎 𝑇 𝐯, 𝐰 ⟩ =

𝑎 ⟨ 𝐯, 𝑇^{*} 𝐰 ⟩ =

⟨ 𝐯, 𝑎 𝑇^{*} 𝐰 ⟩

. (c)

⟨ 𝑇^{*} 𝐰, 𝐯 ⟩ =

⟨ 𝐯, 𝑇^{*} 𝐰 ⟩ =

⟨ 𝑇 𝐯, 𝐰 ⟩ =

⟨ 𝐰, 𝑇 𝐯 ⟩

. (d)

⟨ 𝑅 𝑇 𝐯, 𝐮 ⟩ =

⟨ 𝑇 𝐯, 𝑅^{*} 𝐮 ⟩ =

⟨ 𝐯, 𝑇^{*} 𝑅^{*} 𝐮 ⟩

. (e) is immediate. (f) By (d) and (e),

𝑇^{*} {(𝑇^{- 1})}^{*} =

{(𝑇^{- 1} 𝑇)}^{*} =

{id}_{𝑉}

and

{(𝑇^{- 1})}^{*} 𝑇^{*} =

{(𝑇 𝑇^{- 1})}^{*} =

{id}_{𝑊}

In each part we check the defining identity for the proposed adjoint; by Proposition 6.6.9 it is then the adjoint. Let $𝐯 \in 𝑉$ , $𝐰 \in 𝑊$ , and $𝐮 \in 𝑈$ .

(a) $⟨ (𝑆 + 𝑇) 𝐯, 𝐰 ⟩ =$ $⟨ 𝑆 𝐯, 𝐰 ⟩ +$ $⟨ 𝑇 𝐯, 𝐰 ⟩ =$ $⟨ 𝐯, 𝑆^{*} 𝐰 ⟩ +$ $⟨ 𝐯, 𝑇^{*} 𝐰 ⟩ =$ $⟨ 𝐯, (𝑆^{*} + 𝑇^{*}) 𝐰 ⟩$ .

(b) By Proposition 6.1.4, $⟨ 𝐯, 𝑎 𝑇^{*} 𝐰 ⟩ =$ $𝑎 ⟨ 𝐯, 𝑇^{*} 𝐰 ⟩$ , so $⟨ (𝑎 𝑇) 𝐯, 𝐰 ⟩ =$ $𝑎 ⟨ 𝑇 𝐯, 𝐰 ⟩ =$ $𝑎 ⟨ 𝐯, 𝑇^{*} 𝐰 ⟩ =$ $⟨ 𝐯, 𝑎 𝑇^{*} 𝐰 ⟩$ .

(c) An adjoint of $𝑇^{*} : 𝑊 \to 𝑉$ is a function $𝑆 : 𝑉 \to 𝑊$ with $⟨ 𝑇^{*} 𝐰, 𝐯 ⟩ =$ $⟨ 𝐰, 𝑆 𝐯 ⟩$ for all $𝐰, 𝐯$ . By conjugate symmetry and the identity for $𝑇$ ,

⟨ 𝑇^{*} 𝐰, 𝐯 ⟩ = ⟨ 𝐯, 𝑇^{*} 𝐰 ⟩ = ⟨ 𝑇 𝐯, 𝐰 ⟩ = ⟨ 𝐰, 𝑇 𝐯 ⟩,

so $𝑆 = 𝑇$ works.

(d) $⟨ 𝑅 𝑇 𝐯, 𝐮 ⟩ =$ $⟨ 𝑇 𝐯, 𝑅^{*} 𝐮 ⟩ =$ $⟨ 𝐯, 𝑇^{*} 𝑅^{*} 𝐮 ⟩$ , using the identity for $𝑅$ and then for $𝑇$ .

(e) $⟨ {id}_{𝑉} 𝐯, 𝐯^{'} ⟩ =$ $⟨ 𝐯, {id}_{𝑉} 𝐯^{'} ⟩$ for all $𝐯, 𝐯^{'} \in 𝑉$ .

(f) By (d) and (e), applied to $𝑇^{- 1} 𝑇 = {id}_{𝑉}$ and to $𝑇 𝑇^{- 1} = {id}_{𝑊}$ ,

𝑇^{*} {(𝑇^{- 1})}^{*} = {(𝑇^{- 1} 𝑇)}^{*} = {id}_{𝑉}, {(𝑇^{- 1})}^{*} 𝑇^{*} = {(𝑇 𝑇^{- 1})}^{*} = {id}_{𝑊} .

So ${(𝑇^{- 1})}^{*}$ is a two-sided inverse of $𝑇^{*}$ (Definition 2.5.12). □

Proof (via matrices, in finite dimensions).routine

Uses. Theorem 6.6.15 (Existence of adjoints), Theorem 2.3.20 (Linear maps correspond to matrices), Proposition 2.4.12 (Matrices of sums and scalar multiples), Theorem 2.4.2 (The matrix of a composite), Proposition 2.5.20 (Inverses and matrices), Proposition 6.3.27 (Properties of the conjugate transpose), Theorem 6.4.20 (Orthonormal bases exist)

Techniques. Translate between maps and matrices (Technique 2.3.27)

Hint 1. Assume

𝑈

𝑉

𝑊

are finite-dimensional and fix orthonormal bases. By Theorem 6.6.15(b), every adjoint has the conjugate transpose of the original matrix.

Hint 2. Two maps are equal if their matrices are (Theorem 2.3.20). Translate each identity into one about matrices and quote Proposition 6.3.27.

Sketch. In orthonormal bases,

[𝑋^{*}] = {[𝑋]}^{*}

for every map

𝑋

. So, for instance,

[{(𝑅 𝑇)}^{*}] =

{[𝑅 𝑇]}^{*} =

{([𝑅] [𝑇])}^{*} =

{[𝑇]}^{*} {[𝑅]}^{*} =

[𝑇^{*}] [𝑅^{*}] =

[𝑇^{*} 𝑅^{*}]

, and the maps agree because their matrices do. The other parts translate in the same way into (a)–(c) of Proposition 6.3.27.

Suppose $𝑈$ , $𝑉$ , $𝑊$ are finite-dimensional, so that all the adjoints in question exist (Theorem 6.6.15), and fix orthonormal bases of the three spaces (Theorem 6.4.20). We write $[𝑋]$ for the matrix of a map with respect to these bases. By Theorem 6.6.15(b), $[𝑋^{*}] = {[𝑋]}^{*}$ for every linear map $𝑋$ between two of the spaces, and by Theorem 2.3.20, two maps with the same matrix are equal. Now, using Proposition 2.4.12, Theorem 2.4.2, and Proposition 6.3.27,

\begin{aligned} [{(𝑆 + 𝑇)}^{*}] & = {([𝑆] + [𝑇])}^{*} = {[𝑆]}^{*} + {[𝑇]}^{*} = [𝑆^{*} + 𝑇^{*}], \\ [{(𝑎 𝑇)}^{*}] & = {(𝑎 [𝑇])}^{*} = 𝑎 {[𝑇]}^{*} = [𝑎 𝑇^{*}], \\ [{(𝑇^{*})}^{*}] & = {({[𝑇]}^{*})}^{*} = [𝑇], \\ [{(𝑅 𝑇)}^{*}] & = {([𝑅] [𝑇])}^{*} = {[𝑇]}^{*} {[𝑅]}^{*} = [𝑇^{*} 𝑅^{*}], \end{aligned}

and $[{id}_{𝑉}^{*}] =$ $𝐼^{*} =$ $𝐼 =$ $[{id}_{𝑉}]$ . For (f), $[𝑇^{- 1}] = {[𝑇]}^{- 1}$ (Proposition 2.5.20), and ${({[𝑇]}^{- 1})}^{*} =$ ${({[𝑇]}^{*})}^{- 1}$ by Proposition 6.3.27(c), so $[{(𝑇^{- 1})}^{*}] =$ ${({[𝑇]}^{*})}^{- 1} =$ ${[𝑇^{*}]}^{- 1}$ , which is the matrix of ${(𝑇^{*})}^{- 1}$ . □

The two proofs run in opposite directions. The second derives the rules for maps from those for matrices, and needs finite dimensions and orthonormal bases. The first needs neither, and applied to $𝑇_{𝐴}$ , $𝑇_{𝐵}$ on $𝔽^{𝑛}$ it reproves Proposition 6.3.27 without a single index: for instance ${(𝐴 𝐵)}^{*} = 𝐵^{*} 𝐴^{*}$ because ${(𝑇_{𝐴} 𝑇_{𝐵})}^{*} =$ $𝑇_{𝐵}^{*} 𝑇_{𝐴}^{*}$ . Part (c) is the lecture’s observation that $𝑇$ is an adjoint of $𝑇^{*}$ .

By (a)–(c), the map $𝑇 \mapsto 𝑇^{*}$ from $L (𝑉, 𝑊)$ to $L (𝑊, 𝑉)$ is additive, conjugates scalars, and is undone by the same operation in the other direction. In finite dimensions it is therefore a bijection; it is linear when $𝔽 = ℝ$ and conjugate-linear when $𝔽 = ℂ$ , just like $𝑧 \mapsto 𝑧$ .

Exercise 6.6.19.routine Let $𝑉$ be finite-dimensional, $𝑇 \in L (𝑉)$ , and $𝑐_{0}, \dots, 𝑐_{𝑘} \in 𝔽$ . Show that

{(𝑐_{0} id + 𝑐_{1} 𝑇 + \dots + 𝑐_{𝑘} 𝑇^{𝑘})}^{*} = 𝑐_{0} id + 𝑐_{1} 𝑇^{*} + \dots + 𝑐_{𝑘} {(𝑇^{*})}^{𝑘} .

Solution.

Uses. Proposition 6.6.18 (Properties of the adjoint)

Techniques. Induction on the length of a list (Technique 1.6.9)

Hint 1. First show

{(𝑇^{𝑗})}^{*} = {(𝑇^{*})}^{𝑗}

by induction on

𝑗

, using (d) and (e) of Proposition 6.6.18.

Hint 2. Then expand the left side with (a) and (b).

Sketch.

{(𝑇^{0})}^{*} = {id}^{*} = id

, and

{(𝑇^{𝑗 + 1})}^{*} =

{(𝑇 𝑇^{𝑗})}^{*} =

{(𝑇^{𝑗})}^{*} 𝑇^{*} =

{(𝑇^{*})}^{𝑗} 𝑇^{*} =

{(𝑇^{*})}^{𝑗 + 1}

. Then (a) and (b) give

{(\sum_{𝑗} 𝑐_{𝑗} 𝑇^{𝑗})}^{*} =

\sum_{𝑗} 𝑐_{𝑗} {(𝑇^{𝑗})}^{*} =

\sum_{𝑗} 𝑐_{𝑗} {(𝑇^{*})}^{𝑗}

All adjoints exist since $𝑉$ is finite-dimensional (Theorem 6.6.15). We show ${(𝑇^{𝑗})}^{*} = {(𝑇^{*})}^{𝑗}$ by induction on $𝑗 \geq 0$ . For $𝑗 = 0$ , $𝑇^{0} = id$ and ${id}^{*} = id$ by Proposition 6.6.18(e). If ${(𝑇^{𝑗})}^{*} = {(𝑇^{*})}^{𝑗}$ , then by Proposition 6.6.18(d),

{(𝑇^{𝑗 + 1})}^{*} = {(𝑇 𝑇^{𝑗})}^{*} = {(𝑇^{𝑗})}^{*} 𝑇^{*} = {(𝑇^{*})}^{𝑗} 𝑇^{*} = {(𝑇^{*})}^{𝑗 + 1} .

By (a) and (b) of Proposition 6.6.18, applied term by term,

{(\sum_{𝑗 = 0}^{𝑘} 𝑐_{𝑗} 𝑇^{𝑗})}^{*} = \sum_{𝑗 = 0}^{𝑘} 𝑐_{𝑗} {(𝑇^{𝑗})}^{*} = \sum_{𝑗 = 0}^{𝑘} 𝑐_{𝑗} {(𝑇^{*})}^{𝑗},

as desired.

6.6.4Kernels and images

For a matrix $𝑀$ we already know two orthogonality relations. One is ${(im 𝑀)}^{⟂} = ker 𝑀^{*}$ (Lemma 6.5.4). The other is $ker 𝑀 = {(im 𝑀^{*})}^{⟂}$ : solving $𝑀 𝐱 = 𝟎$ means finding the vectors orthogonal to the conjugated rows of $𝑀$ , which are the columns of $𝑀^{*}$ (Example 6.3.10). Both were proved by looking at entries. For linear maps between inner product spaces they follow from a single move, which the rest of the book uses constantly.

Technique 6.6.20 (Move an operator across the inner product). When a map acts on one argument of an inner product, move it to the other argument as its adjoint: $⟨ 𝑇 𝐯, 𝐰 ⟩ =$ $⟨ 𝐯, 𝑇^{*} 𝐰 ⟩$ and $⟨ 𝐰, 𝑇 𝐯 ⟩ =$ $⟨ 𝑇^{*} 𝐰, 𝐯 ⟩$ . Do this so that the map lands on the vector you know more about, or so that two vectors you want to compare end up in the same inner product. Typical outcomes:

“ $𝐰$ is orthogonal to $𝑇 𝐯$ for every $𝐯$ ” becomes “ $𝑇^{*} 𝐰$ is orthogonal to every vector”, so $𝑇^{*} 𝐰 = 𝟎$ ;
$⟨ 𝑇^{*} 𝑇 𝐯, 𝐯 ⟩$ becomes $⟨ 𝑇 𝐯, 𝑇 𝐯 ⟩ =$ ${‖ 𝑇 𝐯 ‖}^{2}$ , which is where Technique 6.1.36 takes over;
a hypothesis on $𝑇$ becomes a statement about $𝑇^{*}$ , or, for a self-adjoint $𝑇$ , about $𝑇$ itself acting on the other vector.

Reach for it whenever an inner product contains $𝑇 𝐯$ and the hypotheses or the goal concern $𝑇^{*}$ , or concern the vector in the other slot. The canonical example is part (a) of Theorem 6.6.21. In Chapter 8 the same move shows that the eigenvalues of a self-adjoint operator are real and that its eigenvectors for different eigenvalues are orthogonal.

Used in 31 places and 2 practice exercises

Theorem 6.6.21, Proposition 6.6.23, Proposition 6.6.30, Exercise 6.6.38, Exercise 6.6.40, Exercise 6.6.42, Exercise 6.6.44, Exercise 6.6.51, Exercise 6.7.8, Exercise 6.7.9, Exercise 6.7.10, Exercise 6.7.14, Problem 6.3, Problem 6.7, Problem 6.10, Problem 6.14, Problem 6.18, Proposition 8.1.1, Exercise 8.1.6, Proposition 8.1.13, Lemma 8.1.17, Exercise 8.1.28, Exercise 8.1.29, Theorem 8.2.5, Lemma 8.2.10, Exercise 8.2.44, Lemma 8.3.8, Proposition 8.3.22, Exercise 8.3.28, Theorem 8.4.4, Lemma 8.4.14; 2 practice exercises.

Theorem 6.6.21 (Kernels and images of adjoints). Let $𝑉$ and $𝑊$ be inner product spaces, and let $𝑇 \in L (𝑉, 𝑊)$ have an adjoint. Then

(a) $ker 𝑇^{*} = {(im 𝑇)}^{⟂}$ ;
(b) $ker 𝑇 = {(im 𝑇^{*})}^{⟂}$ .
If moreover $𝑉$ is finite-dimensional, then also
(c) $im 𝑇^{*} = {(ker 𝑇)}^{⟂}$ ;
(d) $im 𝑇 = {(ker 𝑇^{*})}^{⟂}$ ;
and there are orthogonal decompositions $𝑉 = ker 𝑇 \oplus im 𝑇^{*}$ and $𝑊 = im 𝑇 \oplus ker 𝑇^{*}$ .

Intuition. Part (a) is a chain of rewritings. A vector

𝐰

is in

{(im 𝑇)}^{⟂}

when

⟨ 𝑇 𝐯, 𝐰 ⟩ = 0

for every

𝐯

; moving

𝑇

across, that says

⟨ 𝐯, 𝑇^{*} 𝐰 ⟩ = 0

for every

𝐯

, and the only vector orthogonal to everything is

𝟎

. Part (b) is (a) for

𝑇^{*}

, whose adjoint is

𝑇

. Parts (c) and (d) take orthogonal complements of (b) and (a), which is allowed once the subspaces are finite-dimensional (Theorem 6.4.26).

Proof.standard

Uses. Definition 6.6.8 (Adjoint), Definition 6.3.8 (Vectors orthogonal to a set), Proposition 6.1.4 (Sesquilinearity), Definition 6.1.2 (Inner product), Proposition 6.6.18 (Properties of the adjoint), Theorem 6.4.26 (The double complement), Proposition 4.1.21 (Dimensions of subspaces), Theorem 4.2.11 (Rank–nullity), Theorem 6.4.24 (Orthogonal decomposition)

Techniques. Move an operator across the inner product (Technique 6.6.20), Show a vector is zero via its norm (Technique 6.1.36)

Hint 1. For (a), write out what

𝐰 \in {(im 𝑇)}^{⟂}

means, move

𝑇

to the other slot, and decide when a vector is orthogonal to every vector of

𝑉

Hint 2. For (b), apply (a) to

𝑇^{*}

; what is the adjoint of

𝑇^{*}

Hint 3. For (c) and (d), take orthogonal complements in (b) and (a). Why are

im 𝑇^{*}

and

im 𝑇

finite-dimensional when

𝑉

is?

Sketch. (a)

𝐰 \in ker 𝑇^{*}

iff

⟨ 𝐯, 𝑇^{*} 𝐰 ⟩ = 0

for all

𝐯

(take

𝐯 = 𝑇^{*} 𝐰

for one direction) iff

⟨ 𝑇 𝐯, 𝐰 ⟩ = 0

for all

𝐯

iff

𝐰 ⟂ im 𝑇

. (b) is (a) for

𝑇^{*}

, since

{(𝑇^{*})}^{*} = 𝑇

. If

𝑉

is finite-dimensional, so are

im 𝑇^{*} \subseteq 𝑉

and

im 𝑇

, and Theorem 6.4.26 turns (b) and (a) into (c) and (d). The decompositions are Theorem 6.4.24 for

ker 𝑇

and

im 𝑇

(a) Let $𝐰 \in 𝑊$ . We claim that

\begin{aligned} 𝑇^{*} 𝐰 = 𝟎 & ⟺ ⟨ 𝐯, 𝑇^{*} 𝐰 ⟩ = 0 for all 𝐯 \in 𝑉 \\ ⟺ ⟨ 𝑇 𝐯, 𝐰 ⟩ = 0 for all 𝐯 \in 𝑉 \\ ⟺ 𝐰 \in {(im 𝑇)}^{⟂} . \end{aligned}

In the first line, if $𝑇^{*} 𝐰 = 𝟎$ , then every $⟨ 𝐯, 𝑇^{*} 𝐰 ⟩$ is $0$ by Proposition 6.1.4(b); conversely, taking $𝐯 = 𝑇^{*} 𝐰$ gives ${‖ 𝑇^{*} 𝐰 ‖}^{2} = 0$ , so $𝑇^{*} 𝐰 = 𝟎$ by definiteness. The second equivalence is the defining identity of $𝑇^{*}$ . For the third, the elements of $im 𝑇$ are the vectors $𝑇 𝐯$ , and $⟨ 𝐰, 𝑇 𝐯 ⟩ =$ $⟨ 𝑇 𝐯, 𝐰 ⟩$ vanishes exactly when $⟨ 𝑇 𝐯, 𝐰 ⟩$ does (Definition 6.3.8). So $ker 𝑇^{*} = {(im 𝑇)}^{⟂}$ .

(b) By Proposition 6.6.18(c), $𝑇^{*}$ has an adjoint, namely $𝑇$ . Applying (a) to $𝑇^{*} \in L (𝑊, 𝑉)$ gives $ker 𝑇 =$ $ker {(𝑇^{*})}^{*} =$ ${(im 𝑇^{*})}^{⟂}$ .

(c) Suppose $𝑉$ is finite-dimensional. Then its subspace $im 𝑇^{*}$ is finite-dimensional (Proposition 4.1.21), so by (b) and Theorem 6.4.26,

{(ker 𝑇)}^{⟂} = {({(im 𝑇^{*})}^{⟂})}^{⟂} = im 𝑇^{*} .

(d) By Theorem 4.2.11, $dim im 𝑇 \leq dim 𝑉$ , so $im 𝑇$ is a finite-dimensional subspace of $𝑊$ . By (a) and Theorem 6.4.26, ${(ker 𝑇^{*})}^{⟂} =$ ${({(im 𝑇)}^{⟂})}^{⟂} =$ $im 𝑇$ .

Finally, $ker 𝑇$ and $im 𝑇$ are finite-dimensional, so Theorem 6.4.24 gives $𝑉 =$ $ker 𝑇 \oplus$ ${(ker 𝑇)}^{⟂} =$ $ker 𝑇 \oplus$ $im 𝑇^{*}$ by (c), and $𝑊 =$ $im 𝑇 \oplus$ ${(im 𝑇)}^{⟂} =$ $im 𝑇 \oplus$ $ker 𝑇^{*}$ by (a). □

For $𝑇 = 𝑇_{𝑀}$ , part (a) is Lemma 6.5.4 and part (b) is the computation of Example 6.3.10. The lecture proves the weaker statement that $im 𝑇^{*}$ is orthogonal to $ker 𝑇$ , which is half of (b) and (c): if $𝐯 = 𝑇^{*} 𝐰$ and $𝑇 𝐳 = 𝟎$ , then $⟨ 𝐯, 𝐳 ⟩ =$ $⟨ 𝑇^{*} 𝐰, 𝐳 ⟩ =$ $⟨ 𝐰, 𝑇 𝐳 ⟩ =$ $0$ .

In infinite dimensions, (a) and (b) survive but (c) and (d) can fail, because a subspace can have a zero orthogonal complement without being everything (Exercise 6.4.63).

Corollary 6.6.22 ( $𝑇$ and $𝑇^{*}$ have the same rank). Let $𝑉$ and $𝑊$ be finite-dimensional and $𝑇 \in L (𝑉, 𝑊)$ .

(a)The restriction $𝑇 ↾_{im 𝑇^{*}}$ is injective with image $im 𝑇$ . So $𝑇$ maps $im 𝑇^{*}$ bijectively onto $im 𝑇$ , and likewise $𝑇^{*}$ maps $im 𝑇$ bijectively onto $im 𝑇^{*}$ .
(b) $rank 𝑇^{*} = rank 𝑇$ .
(c)For every $𝐴 \in 𝔽^{𝑚 \times 𝑛}$ , $rank 𝐴^{*} = rank 𝐴$ .

Proof (by restriction).standard

Uses. Theorem 6.6.21 (Kernels and images of adjoints), Definition 4.2.1 (Restriction), Proposition 6.3.9 (Properties of $𝐴^{⟂}$ ), Proposition 4.2.6 (Injective maps preserve dimension), Proposition 6.6.18 (Properties of the adjoint), Example 6.6.10 (The adjoint of a matrix), Theorem 4.2.7 (The image of $𝑇_{𝐴}$ )

Techniques. Restrict to a subspace (Technique 4.2.5)

Hint 1. The kernel of

𝑇 ↾_{im 𝑇^{*}}

ker 𝑇 \cap im 𝑇^{*}

. Use Theorem 6.6.21(c).

Hint 2. For the image, decompose

𝐯 = 𝐳 + 𝐫

with

𝐳 \in ker 𝑇

and

𝐫 \in im 𝑇^{*}

, and compute

𝑇 𝐯

Hint 3. For (c), apply (b) to

𝑇_{𝐴}

, using

{(𝑇_{𝐴})}^{*} = 𝑇_{𝐴^{*}}

Sketch.

ker (𝑇 ↾_{im 𝑇^{*}}) =

ker 𝑇 \cap

im 𝑇^{*} =

ker 𝑇 \cap

{(ker 𝑇)}^{⟂} =

{𝟎}

. Writing

𝐯 = 𝐳 + 𝐫

with

𝐳 \in ker 𝑇

and

𝐫 \in im 𝑇^{*}

gives

𝑇 𝐯 = 𝑇 𝐫

, so the restriction already hits all of

im 𝑇

. An injective map preserves dimension, so

rank 𝑇 =

dim im 𝑇^{*} =

rank 𝑇^{*}

. The statement for

𝑇^{*}

follows by symmetry, and (c) is (b) for

𝑇_{𝐴}

(a) Let $𝑅 =$ $𝑇 ↾_{im 𝑇^{*}} : im 𝑇^{*} \to$ $𝑊$ . Its kernel is $ker 𝑇 \cap im 𝑇^{*}$ , which by Theorem 6.6.21(c) is $ker 𝑇 \cap$ ${(ker 𝑇)}^{⟂} =$ ${𝟎}$ (Proposition 6.3.9(d)). So $𝑅$ is injective. Clearly $im 𝑅 \subseteq im 𝑇$ . Conversely, let $𝐯 \in 𝑉$ . By Theorem 6.6.21, $𝐯 = 𝐳 + 𝐫$ with $𝐳 \in ker 𝑇$ and $𝐫 \in im 𝑇^{*}$ , so $𝑇 𝐯 =$ $𝑇 𝐳 +$ $𝑇 𝐫 =$ $𝑅 𝐫 \in$ $im 𝑅$ . Hence $im 𝑅 = im 𝑇$ , and $𝑅$ is a bijection from $im 𝑇^{*}$ onto $im 𝑇$ . Since $𝑇^{*}$ has adjoint $𝑇$ (Proposition 6.6.18(c)), the same argument applied to $𝑇^{*}$ shows that $𝑇^{*}$ maps $im 𝑇$ bijectively onto $im 𝑇^{*}$ .

(b) By (a) and Proposition 4.2.6(a), applied to the injective map $𝑅$ , $dim im 𝑇 =$ $dim im 𝑅 =$ $dim im 𝑇^{*}$ .

(c) With dot products, ${(𝑇_{𝐴})}^{*} = 𝑇_{𝐴^{*}}$ (Example 6.6.10), and $rank 𝐴 = dim im 𝑇_{𝐴}$ (Theorem 4.2.7). So (c) is (b) for $𝑇 = 𝑇_{𝐴}$ . □

Proof (by counting dimensions).routine

Uses. Theorem 6.6.21 (Kernels and images of adjoints), Corollary 6.4.25 (Dimension of the orthogonal complement), Theorem 4.2.11 (Rank–nullity), Example 6.6.10 (The adjoint of a matrix)

Techniques. Dimension counting (Technique 4.2.24)

Hint 1. Compute

dim im 𝑇^{*}

from Theorem 6.6.21(c) and Corollary 6.4.25.

Hint 2. Compare with rank–nullity for

𝑇

Sketch.

rank 𝑇^{*} =

dim {(ker 𝑇)}^{⟂} =

dim 𝑉 -

dim ker 𝑇 =

rank 𝑇

. This proves (b), and (c) as before; it does not give the bijection in (a).

By Theorem 6.6.21(c), Corollary 6.4.25, and Theorem 4.2.11,

rank 𝑇^{*} = dim {(ker 𝑇)}^{⟂} = dim 𝑉 - dim ker 𝑇 = rank 𝑇 .

This is (b), and (c) follows as in the first proof. This argument does not give (a). □

The counting proof is shorter; the first proof shows where the equality comes from. A linear map $𝑇$ throws away $ker 𝑇$ and is injective on the rest of $𝑉$ , and Theorem 6.6.21 identifies “the rest” as $im 𝑇^{*}$ . The two bijections in (a) are not inverse to each other in general: for $𝑇 = 2 {id}_{𝑉}$ , both are multiplication by $2$ .

The four fundamental subspaces

For a matrix $𝐴 \in 𝔽^{𝑚 \times 𝑛}$ , apply Theorem 6.6.21 to $𝑇_{𝐴} : 𝔽^{𝑛} \to 𝔽^{𝑚}$ , whose adjoint is $𝑇_{𝐴^{*}}$ (Example 6.6.10). It gives the orthogonal decompositions

𝔽^{𝑛} = ker 𝐴 \oplus im 𝐴^{*}, 𝔽^{𝑚} = im 𝐴 \oplus ker 𝐴^{*} .

Over $ℝ$ , $𝐴^{*} = 𝐴^{𝖳}$ , and these are the four fundamental subspaces of Definition 4.3.12: the null space $ker 𝐴$ and the row space $im 𝐴^{𝖳}$ are orthogonal complements in $ℝ^{𝑛}$ , and the column space $im 𝐴$ and the left null space $ker 𝐴^{𝖳}$ are orthogonal complements in $ℝ^{𝑚}$ . So $ℝ^{𝑛} = im 𝐴^{𝖳} \oplus ker 𝐴$ . Every $𝐱 \in ℝ^{𝑛}$ splits as $𝐱 = 𝐱_{𝑟} + 𝐱_{𝑛}$ with $𝐱_{𝑟}$ in the row space and $𝐱_{𝑛} \in ker 𝐴$ , and $𝐴 𝐱 = 𝐴 𝐱_{𝑟}$ ; by Corollary 6.6.22, $𝐴$ maps the row space bijectively onto the column space. In particular $rank 𝐴^{𝖳} = rank 𝐴$ , which is Theorem 4.3.8 for real matrices. Figure 6.6.2 shows the picture.

Figure 6.6.2. The four subspaces of

𝑇 \in L (𝑉, 𝑊)

, with

𝑛 = dim 𝑉

𝑚 = dim 𝑊

, and

𝑟 = rank 𝑇

. In

𝑉

, the subspaces

im 𝑇^{*}

and

ker 𝑇

are orthogonal complements, and in

𝑊

so are

im 𝑇

and

ker 𝑇^{*}

. The map

𝑇

(blue) sends

ker 𝑇

𝟎

and

im 𝑇^{*}

bijectively onto

im 𝑇

; its adjoint

𝑇^{*}

(ochre) sends

ker 𝑇^{*}

𝟎

and

im 𝑇

bijectively onto

im 𝑇^{*}

. For

𝑇 = 𝑇_{𝐴}

with

𝐴

real, these are the row space, null space, column space, and left null space of

𝐴

Over $ℂ$ the orthogonal complement of $ker 𝐴$ is $im 𝐴^{*}$ , the span of the conjugated rows, and the conjugation matters. For $𝐴 = (\begin{matrix} 1 & 𝑖 \end{matrix})$ , $ker 𝐴 =$ $span ((- 𝑖, 1))$ , and the conjugated row $(1, - 𝑖)$ is orthogonal to $(- 𝑖, 1)$ : $(1, - 𝑖) \cdot$ $(- 𝑖, 1) =$ $- 𝑖 +$ $(- 𝑖) =$ $0$ . The row $(1, 𝑖)$ itself is not: $(1, 𝑖) \cdot$ $(- 𝑖, 1) =$ $𝑖 +$ $𝑖 =$ $2 𝑖$ . (The rank of $𝐴^{𝖳}$ still equals the rank of $𝐴$ , since conjugating every entry does not change which columns are independent.)

The lecture’s other result about adjoints generalizes Proposition 6.5.9 from matrices to maps, with the same two proofs.

Proposition 6.6.23 ( $𝑇^{*} 𝑇$ and $𝑇$ ). Let $𝑉$ and $𝑊$ be inner product spaces, and let $𝑇 \in L (𝑉, 𝑊)$ have an adjoint.

(a) $ker (𝑇^{*} 𝑇) = ker 𝑇$ .
(b)If $𝑉$ is finite-dimensional, then $rank (𝑇^{*} 𝑇) =$ $rank 𝑇$ and $im (𝑇^{*} 𝑇) = im 𝑇^{*}$ .

Proof (by the norm).routine

Uses. Definition 6.6.8 (Adjoint), Definition 6.1.2 (Inner product), Theorem 4.2.11 (Rank–nullity), Corollary 6.6.22 ( $𝑇$ and $𝑇^{*}$ have the same rank), Proposition 4.1.21 (Dimensions of subspaces)

Techniques. Move an operator across the inner product (Technique 6.6.20), Show a vector is zero via its norm (Technique 6.1.36), Double containment (Technique 1.3.14)

Hint 1. One containment in (a) is immediate. For the other, suppose

𝑇^{*} 𝑇 𝐯 = 𝟎

and take the inner product with

𝐯

Hint 2. Move

𝑇^{*}

across:

⟨ 𝑇^{*} 𝑇 𝐯, 𝐯 ⟩ =

⟨ 𝑇 𝐯, 𝑇 𝐯 ⟩

Hint 3. For (b), use rank–nullity for

𝑇^{*} 𝑇

and

𝑇

, which have the same domain, and then compare

im (𝑇^{*} 𝑇) \subseteq im 𝑇^{*}

by dimension.

Sketch. If

𝑇^{*} 𝑇 𝐯 = 𝟎

, then

{‖ 𝑇 𝐯 ‖}^{2} =

⟨ 𝑇 𝐯, 𝑇 𝐯 ⟩ =

⟨ 𝑇^{*} 𝑇 𝐯, 𝐯 ⟩ =

0

, so

𝑇 𝐯 = 𝟎

; the other containment is clear. Equal kernels and rank–nullity give

rank (𝑇^{*} 𝑇) =

rank 𝑇

, which equals

rank 𝑇^{*}

by Corollary 6.6.22; since

im (𝑇^{*} 𝑇) \subseteq im 𝑇^{*}

, the images agree.

(a) If $𝑇 𝐯 = 𝟎$ , then $𝑇^{*} 𝑇 𝐯 = 𝑇^{*} 𝟎 = 𝟎$ . Conversely, suppose $𝑇^{*} 𝑇 𝐯 = 𝟎$ . By the defining identity of $𝑇^{*}$ with $𝐰 = 𝑇 𝐯$ ,

{‖ 𝑇 𝐯 ‖}^{2} = ⟨ 𝑇 𝐯, 𝑇 𝐯 ⟩ = ⟨ 𝐯, 𝑇^{*} 𝑇 𝐯 ⟩ = ⟨ 𝐯, 𝟎 ⟩ = 0,

so $𝑇 𝐯 = 𝟎$ by definiteness. Hence $ker (𝑇^{*} 𝑇) = ker 𝑇$ .

(b) Both $𝑇^{*} 𝑇$ and $𝑇$ have domain $𝑉$ , so by Theorem 4.2.11 and (a),

rank (𝑇^{*} 𝑇) = dim 𝑉 - dim ker (𝑇^{*} 𝑇) = dim 𝑉 - dim ker 𝑇 = rank 𝑇 .

By Corollary 6.6.22 this is $rank 𝑇^{*} = dim im 𝑇^{*}$ , and $im (𝑇^{*} 𝑇) \subseteq im 𝑇^{*}$ . A subspace of $im 𝑇^{*}$ of the same dimension is all of $im 𝑇^{*}$ (Proposition 4.1.21(b)), so $im (𝑇^{*} 𝑇) = im 𝑇^{*}$ . □

Proof (by orthogonality).routine

Uses. Theorem 6.6.21 (Kernels and images of adjoints), Proposition 6.3.9 (Properties of $𝐴^{⟂}$ )

Techniques. Move an operator across the inner product (Technique 6.6.20)

Hint. Suppose

𝑇^{*} (𝑇 𝐯) = 𝟎

. Which two subspaces of

𝑊

contain

𝑇 𝐯

Sketch. If

𝑇^{*} (𝑇 𝐯) = 𝟎

, then

𝑇 𝐯 \in

ker 𝑇^{*} =

{(im 𝑇)}^{⟂}

by Theorem 6.6.21(a), and also

𝑇 𝐯 \in im 𝑇

, so

𝑇 𝐯 \in

im 𝑇 \cap

{(im 𝑇)}^{⟂} =

{𝟎}

. Part (b) follows from (a) as in the first proof.

As before,

ker 𝑇 \subseteq ker (𝑇^{*} 𝑇)

. Conversely, if

𝑇^{*} (𝑇 𝐯) = 𝟎

, then

𝑇 𝐯 \in

ker 𝑇^{*} =

{(im 𝑇)}^{⟂}

by Theorem 6.6.21(a), and

𝑇 𝐯 \in im 𝑇

. So

𝑇 𝐯 \in

im 𝑇 \cap

{(im 𝑇)}^{⟂} =

{𝟎}

(Proposition 6.3.9(d)). This proves (a), and (b) follows from (a) exactly as in the first proof. □

The first proof is the lecture’s. Both proofs move $𝑇^{*}$ somewhere useful: the first next to $𝑇 𝐯$ , to produce a norm, and the second into the description of $ker 𝑇^{*}$ as an orthogonal complement.

The next example shows the theorem at work on a map that is not self-adjoint.

Example 6.6.24 (An oblique projection and its adjoint). Let $𝑃 : ℝ^{2} \to ℝ^{2}$ , $𝑃 (𝑥, 𝑦) =$ $(𝑥 - 𝑦, 0)$ , the projection onto the $𝑥$ -axis along the line $𝑦 = 𝑥$ from Remark 6.4.12. Its matrix is $(\begin{matrix} 1 & - 1 \\ 0 & 0 \end{matrix})$ , so $𝑃^{*}$ has matrix $(\begin{matrix} 1 & 0 \\ - 1 & 0 \end{matrix})$ , that is, $𝑃^{*} (𝑥, 𝑦) =$ $(𝑥, - 𝑥)$ . Then

\begin{aligned} im 𝑃 & = span (𝐞_{1}), & ker 𝑃 & = span ((1, 1)), \\ im 𝑃^{*} & = span ((1, - 1)) = {(ker 𝑃)}^{⟂}, & ker 𝑃^{*} & = span (𝐞_{2}) = {(im 𝑃)}^{⟂}, \end{aligned}

as Theorem 6.6.21 predicts. Moreover ${(𝑃^{*})}^{2} = {(𝑃^{2})}^{*} = 𝑃^{*}$ , so $𝑃^{*}$ is again a projection: it moves each point onto the line $𝑦 = - 𝑥$ along the vertical direction (Figure 6.6.3). In general, the adjoint of the projection onto $𝑈$ along $𝑋$ is the projection onto $𝑋^{⟂}$ along $𝑈^{⟂}$ (Exercise 6.6.47). The two coincide exactly when $𝑋 = 𝑈^{⟂}$ , which is when the projection is orthogonal.

Figure 6.6.3. (a) The projection

𝑃 (𝑥, 𝑦) =

(𝑥 - 𝑦, 0)

moves each point onto

im 𝑃

(the

𝑥

-axis) along

ker 𝑃

(the line

𝑦 = 𝑥

). (b) Its adjoint

𝑃^{*} (𝑥, 𝑦) =

(𝑥, - 𝑥)

moves each point onto

im 𝑃^{*} = {(ker 𝑃)}^{⟂}

along

ker 𝑃^{*} = {(im 𝑃)}^{⟂}

. The dotted lines are

im 𝑃

and

ker 𝑃

from (a).

Exercise 6.6.25.routine Let

𝐴 =

(\begin{matrix} 1 & 2 & 1 \\ 0 & 1 & 1 \\ 1 & 3 & 2 \end{matrix}) \in

ℝ^{3 \times 3}

. Find bases of

ker 𝐴

im 𝐴^{𝖳}

im 𝐴

, and

ker 𝐴^{𝖳}

, and check that

ℝ^{3} = ker 𝐴 \oplus im 𝐴^{𝖳}

and

ℝ^{3} = im 𝐴 \oplus ker 𝐴^{𝖳}

are orthogonal decompositions.

Solution.

Uses. Theorem 6.6.21 (Kernels and images of adjoints), Corollary 6.6.22 ( $𝑇$ and $𝑇^{*}$ have the same rank), Proposition 2.6.18 (The column space is spanned by the columns), Proposition 6.3.9 (Properties of $𝐴^{⟂}$ )

Hint 1. The third row is the sum of the first two, so

rank 𝐴 = 2

, and each kernel is a line.

Hint 2. Solve

𝐴 𝐱 = 𝟎

and

𝐴^{𝖳} 𝐲 = 𝟎

, then check orthogonality against the rows and the columns of

𝐴

Sketch.

ker 𝐴 =

span ((1, - 1, 1))

and

im 𝐴^{𝖳} =

span ((1, 2, 1), (0, 1, 1))

; the kernel vector is orthogonal to both rows.

im 𝐴 =

span ((1, 0, 1), (2, 1, 3))

and

ker 𝐴^{𝖳} =

span ((1, 1, - 1))

, orthogonal to every column. Dimensions

1 + 2 = 3

in both cases.

Row 3 is row 1 plus row 2, and rows 1 and 2 are not proportional, so $rank 𝐴 = 2$ and $im 𝐴^{𝖳} =$ $span ((1, 2, 1), (0, 1, 1))$ has dimension $2$ . Solving $𝑥_{1} + 2 𝑥_{2} + 𝑥_{3} = 0$ and $𝑥_{2} + 𝑥_{3} = 0$ gives $𝑥_{2} = - 𝑥_{3}$ and $𝑥_{1} = 𝑥_{3}$ , so $ker 𝐴 =$ $span ((1, - 1, 1))$ . Indeed $(1, - 1, 1) \cdot$ $(1, 2, 1) =$ $0$ and $(1, - 1, 1) \cdot$ $(0, 1, 1) =$ $0$ , so $ker 𝐴 ⟂ im 𝐴^{𝖳}$ (Proposition 6.3.9(c)), and the dimensions add up to $3$ , as Theorem 6.6.21 predicts.

The columns are $(1, 0, 1)$ , $(2, 1, 3)$ , $(1, 1, 2)$ , and the third is the second minus the first, so $im 𝐴 =$ $span ((1, 0, 1), (2, 1, 3))$ (Proposition 2.6.18). The equations of $𝐴^{𝖳} 𝐲 = 𝟎$ are $𝑦_{1} + 𝑦_{3} = 0$ , $2 𝑦_{1} + 𝑦_{2} + 3 𝑦_{3} = 0$ , $𝑦_{1} + 𝑦_{2} + 2 𝑦_{3} = 0$ ; the first gives $𝑦_{3} = - 𝑦_{1}$ , and then the second gives $𝑦_{2} = 𝑦_{1}$ , which also satisfies the third. So $ker 𝐴^{𝖳} =$ $span ((1, 1, - 1))$ , which is orthogonal to $(1, 0, 1)$ and $(2, 1, 3)$ . Again the dimensions are $2$ and $1$ .

6.6.5Self-adjoint operators

For an operator $𝑇 \in L (𝑉)$ , the maps $𝑇$ and $𝑇^{*}$ act on the same space, so they can be compared. The operators that equal their adjoint are the subject of Chapter 8. Here we define them, collect examples, and characterize them over $ℂ$ .

Definition 6.6.26 (Self-adjoint operator). Let $𝑉$ be an inner product space. An operator $𝑇 \in L (𝑉)$ is self-adjoint if

⟨ 𝑇 𝐯, 𝐰 ⟩ = ⟨ 𝐯, 𝑇 𝐰 ⟩ for all 𝐯, 𝐰 \in 𝑉,

that is, if $𝑇$ has an adjoint and $𝑇^{*} = 𝑇$ .

By Example 6.6.10, $𝑇_{𝐴}$ is self-adjoint for the dot product on $𝔽^{𝑛}$ exactly when $𝐴^{*} = 𝐴$ , that is, when $𝐴$ is Hermitian (Definition 6.3.26), and over $ℝ$ when $𝐴$ is symmetric. One says “self-adjoint” for operators and “Hermitian” for matrices; in orthonormal coordinates they are the same condition (Corollary 6.6.28).

Example 6.6.27 (Self-adjoint operators).

(a)The matrices $(\begin{matrix} 2 & 1 - 𝑖 \\ 1 + 𝑖 & 3 \end{matrix})$ and $(\begin{matrix} 1 & 2 \\ 2 & - 1 \end{matrix})$ are Hermitian, so they give self-adjoint operators on $ℂ^{2}$ and $ℝ^{2}$ .
(b)Let $𝑊$ be a finite-dimensional subspace of $𝑉$ . By Proposition 6.4.29(c), $⟨ {Pr}_{𝑊} 𝐯, 𝐱 ⟩ =$ $⟨ 𝐯, {Pr}_{𝑊} 𝐱 ⟩$ for all $𝐯, 𝐱$ , so ${Pr}_{𝑊}$ is self-adjoint.
(c)For a continuous $ℎ : [0, 1] \to ℝ$ , the multiplication operator $𝑀_{ℎ}$ on $𝐶 ([0, 1], ℂ)$ is self-adjoint, since $𝑀_{ℎ}^{*} = 𝑀_{ℎ} = 𝑀_{ℎ}$ (Example 6.6.14).
(d)If $𝑇 \in L (𝑉, 𝑊)$ has an adjoint, then $𝑇^{*} 𝑇 \in L (𝑉)$ and $𝑇 𝑇^{*} \in L (𝑊)$ are self-adjoint: ${(𝑇^{*} 𝑇)}^{*} =$ $𝑇^{*} {(𝑇^{*})}^{*} =$ $𝑇^{*} 𝑇$ by (c) and (d) of Proposition 6.6.18, and similarly for $𝑇 𝑇^{*}$ . In particular $𝐴^{*} 𝐴$ is Hermitian for every matrix $𝐴$ , as we knew from its description as a Gram matrix (Theorem 6.3.29). Similarly $𝑇 + 𝑇^{*}$ is self-adjoint for every $𝑇 \in L (𝑉)$ with an adjoint.

Some operators that are not self-adjoint: the rotation $𝐽 = (\begin{matrix} 0 & - 1 \\ 1 & 0 \end{matrix})$ of Example 6.1.39, with $𝐽^{𝖳} = - 𝐽$ ; multiplication by $𝑖$ on $ℂ$ , whose adjoint is multiplication by $- 𝑖$ (Example 6.6.10); the oblique projection of Example 6.6.24; and $𝑇_{𝐴}$ for the complex symmetric matrix $𝐴 = (\begin{matrix} 1 & 𝑖 \\ 𝑖 & - 1 \end{matrix})$ , for which $𝐴^{𝖳} = 𝐴$ but $𝐴^{*} \neq 𝐴$ (Exercise 6.6.34).

Corollary 6.6.28 (Self-adjointness in an orthonormal basis). Let

𝑉

be finite-dimensional, let

𝒪︀

be an orthonormal basis of

𝑉

, and let

𝑇 \in L (𝑉)

. Then

𝑇

is self-adjoint if and only if

{[𝑇]}_{𝒪︀}

is Hermitian.

Proof.routine

Uses. Theorem 6.6.15 (Existence of adjoints), Theorem 2.3.20 (Linear maps correspond to matrices), Definition 6.6.26 (Self-adjoint operator)

Hint. Compare the matrices of

𝑇

and

𝑇^{*}

in the basis

𝒪︀

Sketch.

{[𝑇^{*}]}_{𝒪︀} =

{({[𝑇]}_{𝒪︀})}^{*}

by Theorem 6.6.15(b), and maps with equal matrices are equal. So

𝑇^{*} = 𝑇

iff

{({[𝑇]}_{𝒪︀})}^{*} =

{[𝑇]}_{𝒪︀}

By Theorem 6.6.15,

𝑇^{*}

exists and

{[𝑇^{*}]}_{𝒪︀} =

{({[𝑇]}_{𝒪︀})}^{*}

. By Theorem 2.3.20,

𝑇^{*} = 𝑇

if and only if

{[𝑇^{*}]}_{𝒪︀} = {[𝑇]}_{𝒪︀}

, that is, if and only if

{({[𝑇]}_{𝒪︀})}^{*} =

{[𝑇]}_{𝒪︀}

. □

Orthonormality is essential here. The projection $Pr$ of $ℝ^{2}$ onto the $𝑥$ -axis is self-adjoint by (b) above, but in the basis $𝐛_{1} = (1, 0)$ , $𝐛_{2} = (1, 1)$ we have $Pr 𝐛_{1} = 𝐛_{1}$ and $Pr 𝐛_{2} = (1, 0) = 𝐛_{1}$ , so its matrix is $(\begin{matrix} 1 & 1 \\ 0 & 0 \end{matrix})$ , which is not symmetric.

The next result completes Proposition 6.4.30, which characterized orthogonal projections without naming the adjoint.

Corollary 6.6.29 (Orthogonal projections are the self-adjoint idempotents). Let $𝑉$ be finite-dimensional and $𝑃 \in L (𝑉)$ . Then $𝑃$ is an orthogonal projection, that is, $𝑃 = {Pr}_{𝑊}$ for some subspace $𝑊$ of $𝑉$ , if and only if

𝑃^{2} = 𝑃 = 𝑃^{*} .

In that case $𝑊 = im 𝑃$ .

Proof (from the earlier characterization).routine

Uses. Proposition 6.4.30 (Recognizing orthogonal projections), Definition 6.6.26 (Self-adjoint operator)

Hint. Read the second condition of Proposition 6.4.30 in the language of Definition 6.6.26.

Sketch. The condition

⟨ 𝑃 𝐯, 𝐱 ⟩ =

⟨ 𝐯, 𝑃 𝐱 ⟩

for all

𝐯, 𝐱

in Proposition 6.4.30 says exactly that

𝑃

is self-adjoint.

The second condition in Proposition 6.4.30,

⟨ 𝑃 𝐯, 𝐱 ⟩ =

⟨ 𝐯, 𝑃 𝐱 ⟩

for all

𝐯, 𝐱 \in 𝑉

, says exactly that

𝑃

is self-adjoint (Definition 6.6.26). So the corollary is Proposition 6.4.30 restated. □

Proof (via kernels and images).routine

Uses. Proposition 6.4.29 (The orthogonal projection as a linear map), Theorem 6.6.21 (Kernels and images of adjoints), Theorem 6.4.6 (The projection is characterized by orthogonality), Theorem 6.4.20 (Orthonormal bases exist), Proposition 4.1.21 (Dimensions of subspaces)

Techniques. Recognize a projection by its two conditions (Technique 6.4.7)

Hint 1. One direction is (b) and (c) of Proposition 6.4.29. For the other, let

𝑊 = im 𝑃

and split

𝐯 = 𝑃 𝐯 + (𝐯 - 𝑃 𝐯)

. Where does each piece lie?

Hint 2.

𝑃 (𝐯 - 𝑃 𝐯) = 𝟎

, and for a self-adjoint

𝑃

, Theorem 6.6.21(a) says

ker 𝑃 = {(im 𝑃)}^{⟂}

Sketch. Orthogonal projections satisfy

𝑃^{2} = 𝑃

and are self-adjoint (Proposition 6.4.29). Conversely,

𝑃 𝐯 \in im 𝑃

and

𝑃 (𝐯 - 𝑃 𝐯) =

𝑃 𝐯 -

𝑃^{2} 𝐯 =

𝟎

, so

𝐯 -

𝑃 𝐯 \in

ker 𝑃 =

ker 𝑃^{*} =

{(im 𝑃)}^{⟂}

. These are the two conditions that characterize

{Pr}_{im 𝑃} (𝐯)

If $𝑃 = {Pr}_{𝑊}$ , then $𝑃^{2} = 𝑃$ by Proposition 6.4.29(b), $𝑃$ is self-adjoint by (c) of the same proposition, and $im 𝑃 = 𝑊$ by part (a).

Conversely, suppose $𝑃^{2} = 𝑃 = 𝑃^{*}$ , and let $𝑊 = im 𝑃$ , a finite-dimensional subspace (Proposition 4.1.21), so that ${Pr}_{𝑊}$ is defined (Theorem 6.4.20). Let $𝐯 \in 𝑉$ . Then $𝑃 𝐯 \in 𝑊$ , and $𝑃 (𝐯 - 𝑃 𝐯) =$ $𝑃 𝐯 -$ $𝑃^{2} 𝐯 =$ $𝟎$ , so by Theorem 6.6.21(a),

𝐯 - 𝑃 𝐯 \in ker 𝑃 = ker 𝑃^{*} = {(im 𝑃)}^{⟂} = 𝑊^{⟂} .

By Theorem 6.4.6, $𝑃 𝐯 = {Pr}_{𝑊} (𝐯)$ . □

The second proof separates the roles of the two conditions. Any $𝑃$ with $𝑃^{2} = 𝑃$ projects onto $im 𝑃$ along $ker 𝑃$ , since $𝐯 = 𝑃 𝐯 + (𝐯 - 𝑃 𝐯)$ with $𝐯 - 𝑃 𝐯 \in ker 𝑃$ ; this is how the oblique projections of Remark 6.4.12 and Example 6.6.24 work. Self-adjointness is what makes $ker 𝑃 = {(im 𝑃)}^{⟂}$ , so that the direction of projection is perpendicular to the target.

Over $ℂ$ , self-adjointness can be read off the “diagonal” values $⟨ 𝑇 𝐯, 𝐯 ⟩$ alone. This is the operator version of the fact that a complex number $𝑧$ is real exactly when $𝑧 = 𝑧$ .

Proposition 6.6.30 (Self-adjoint means real values). Let

𝑉

be a finite-dimensional complex inner product space and

𝑇 \in L (𝑉)

. Then

𝑇

is self-adjoint if and only if

⟨ 𝑇 𝐯, 𝐯 ⟩ \in ℝ

for every

𝐯 \in 𝑉

Proof.standard

Uses. Theorem 6.6.15 (Existence of adjoints), Definition 6.6.8 (Adjoint), Proposition 6.1.41 (Over $ℂ$ , the values $⟨ 𝑇 𝐯, 𝐯 ⟩$ determine $𝑇$ ), Notation 6.1.1 (Real and complex scalars), Proposition 2.1.23 ( $L (𝑉, 𝑊)$ is a vector space)

Techniques. Move an operator across the inner product (Technique 6.6.20)

Hint 1. A complex number

𝑧

is real iff

𝑧 - 𝑧 = 0

. Write

⟨ 𝑇 𝐯, 𝐯 ⟩

as an inner product with

𝑇^{*}

in it.

Hint 2.

⟨ 𝑇 𝐯, 𝐯 ⟩ =

⟨ 𝐯, 𝑇 𝐯 ⟩ =

⟨ 𝑇^{*} 𝐯, 𝐯 ⟩

. So

⟨ 𝑇 𝐯, 𝐯 ⟩ -

⟨ 𝑇 𝐯, 𝐯 ⟩ =

⟨ (𝑇 - 𝑇^{*}) 𝐯, 𝐯 ⟩

Hint 3. Over

ℂ

, an operator

𝑆

with

⟨ 𝑆 𝐯, 𝐯 ⟩ = 0

for all

𝐯

0

(Proposition 6.1.41).

Sketch. For every

𝐯

⟨ 𝑇 𝐯, 𝐯 ⟩ -

⟨ 𝑇 𝐯, 𝐯 ⟩ =

⟨ 𝑇 𝐯, 𝐯 ⟩ -

⟨ 𝑇^{*} 𝐯, 𝐯 ⟩ =

⟨ (𝑇 - 𝑇^{*}) 𝐯, 𝐯 ⟩

. So all values

⟨ 𝑇 𝐯, 𝐯 ⟩

are real iff

⟨ (𝑇 - 𝑇^{*}) 𝐯, 𝐯 ⟩ = 0

for all

𝐯

, iff

𝑇 - 𝑇^{*} = 0

by Proposition 6.1.41.

By Theorem 6.6.15, $𝑇$ has an adjoint. For every $𝐯 \in 𝑉$ , conjugate symmetry and the conjugated form of the defining identity give

⟨ 𝑇 𝐯, 𝐯 ⟩ = ⟨ 𝐯, 𝑇 𝐯 ⟩ = ⟨ 𝑇^{*} 𝐯, 𝐯 ⟩,

and therefore

⟨ 𝑇 𝐯, 𝐯 ⟩ - ⟨ 𝑇 𝐯, 𝐯 ⟩ = ⟨ (𝑇 - 𝑇^{*}) 𝐯, 𝐯 ⟩ .

A complex number is real exactly when it equals its conjugate (Notation 6.1.1). So $⟨ 𝑇 𝐯, 𝐯 ⟩ \in ℝ$ for every $𝐯$ if and only if $⟨ (𝑇 - 𝑇^{*}) 𝐯, 𝐯 ⟩ = 0$ for every $𝐯$ . By Proposition 6.1.41, applied to $𝑇 - 𝑇^{*} \in L (𝑉)$ (Proposition 2.1.23), this holds if and only if $𝑇 - 𝑇^{*} = 0$ , that is, $𝑇$ is self-adjoint. □

Finite-dimensionality was used only to know that $𝑇^{*}$ exists. Exercise 6.1.53 proves the same equivalence on every complex inner product space, without adjoints, by polarization. Over $ℝ$ the proposition fails, since every $⟨ 𝑇 𝐯, 𝐯 ⟩$ is real; the rotation $𝐽$ is not self-adjoint. Over $ℝ$ , the values $⟨ 𝑇 𝐯, 𝐯 ⟩$ see only the self-adjoint part $(𝑇 + 𝑇^{*}) / 2$ of $𝑇$ (Exercise 6.1.52).

Intuition (Self-adjoint operators as real numbers). The analogy between $𝑇 \mapsto 𝑇^{*}$ and $𝑧 \mapsto 𝑧$ runs through Chapter 8. Both are additive, conjugate scalars, and undo themselves, and adjoints also reverse products, which for numbers is invisible. Self-adjoint operators play the role of real numbers: their values $⟨ 𝑇 𝐯, 𝐯 ⟩$ are real (Proposition 6.6.30), and every operator on a complex space splits uniquely as $𝐴 + 𝑖 𝐵$ with $𝐴$ and $𝐵$ self-adjoint, like $𝑧 = 𝑥 + 𝑖 𝑦$ (Exercise 6.6.37). Operators of the form $𝑇^{*} 𝑇$ play the role of ${| 𝑧 |}^{2} = 𝑧 𝑧$ : they are self-adjoint with $⟨ 𝑇^{*} 𝑇 𝐯, 𝐯 ⟩ =$ ${‖ 𝑇 𝐯 ‖}^{2} \geq$ $0$ (Section 8.3). Operators with $𝑇^{*} 𝑇 = id$ play the role of the unit circle; they are the maps that preserve lengths (Exercise 6.6.40, Section 8.4). The analogy is imperfect because operators need not commute, and it works best for the normal operators, those with $𝑇^{*} 𝑇 = 𝑇 𝑇^{*}$ (Section 8.1).

The main theorem about self-adjoint operators, proved in Section 8.2, is that on a finite-dimensional space they are exactly the operators that are diagonal, with real diagonal entries, in some orthonormal basis. Geometrically, a self-adjoint operator stretches $𝑉$ by real factors along mutually perpendicular axes. An orthogonal projection is the case in which every factor is $0$ or $1$ . The oblique projection of Example 6.6.24 also fixes one line and kills another, but the two lines are not perpendicular, and that is exactly the failure of self-adjointness.

Exercises

Exercise 6.6.31.routine Let $𝑇 : ℂ^{2} \to ℂ^{2}$ , $𝑇 (𝑧_{1}, 𝑧_{2}) =$ $(𝑧_{1} + 𝑖 𝑧_{2}, (1 - 𝑖) 𝑧_{1})$ .

(a)Find $𝑇^{*}$ when $ℂ^{2}$ has the dot product.
(b)Find $𝑇^{*}$ when $ℂ^{2}$ has the weighted inner product $⟨ 𝐳, 𝐰 ⟩ =$ $𝑧_{1} 𝑤_{1} +$ $2 𝑧_{2} 𝑤_{2}$ (Example 6.1.21).
(c)In (b), is $𝑇^{*}$ multiplication by the conjugate transpose of the matrix of $𝑇$ ? Why does this not contradict Theorem 6.6.15(b)?

Solution.

Uses. Example 6.6.10 (The adjoint of a matrix), Proposition 6.6.9 (Adjoints are unique and linear), Proposition 6.1.4 (Sesquilinearity), Theorem 6.6.15 (Existence of adjoints)

Techniques. Find the adjoint by isolating the input (Technique 6.6.11)

Hint 1. For (a), use Example 6.6.10 with

𝐴 = (\begin{matrix} 1 & 𝑖 \\ 1 - 𝑖 & 0 \end{matrix})

Hint 2. For (b), expand

⟨ 𝑇 𝐳, 𝐰 ⟩

and collect the terms with

𝑧_{1}

and with

𝑧_{2}

. The coefficient of

𝑧_{2}

must be written as

2

times a conjugate, to match the weight

2

Sketch. (a)

𝐴^{*} = (\begin{matrix} 1 & 1 + 𝑖 \\ - 𝑖 & 0 \end{matrix})

, so

𝑇^{*} (𝐰) =

(𝑤_{1} + (1 + 𝑖) 𝑤_{2}, - 𝑖 𝑤_{1})

. (b)

⟨ 𝑇 𝐳, 𝐰 ⟩ =

𝑧_{1} 𝑤_{1} + 2 (1 + 𝑖) 𝑤_{2} +

2 𝑧_{2} - 𝑖 𝑤_{1} / 2

, so

𝑇^{*} (𝐰) =

(𝑤_{1} + (2 + 2 𝑖) 𝑤_{2}, - 𝑖 𝑤_{1} / 2)

. (c) No; the standard basis is not orthonormal for the weighted inner product.

(a) The matrix of $𝑇$ is $𝐴 = (\begin{matrix} 1 & 𝑖 \\ 1 - 𝑖 & 0 \end{matrix})$ , so by Example 6.6.10, $𝑇^{*}$ is multiplication by $𝐴^{*} = (\begin{matrix} 1 & 1 + 𝑖 \\ - 𝑖 & 0 \end{matrix})$ :

𝑇^{*} (𝑤_{1}, 𝑤_{2}) = (𝑤_{1} + (1 + 𝑖) 𝑤_{2}, - 𝑖 𝑤_{1}) .

(b) For $𝐳, 𝐰 \in ℂ^{2}$ ,

\begin{aligned} ⟨ 𝑇 𝐳, 𝐰 ⟩ & = (𝑧_{1} + 𝑖 𝑧_{2}) 𝑤_{1} + 2 (1 - 𝑖) 𝑧_{1} 𝑤_{2} \\ = 𝑧_{1} (𝑤_{1} + 2 (1 - 𝑖) 𝑤_{2}) + 2 𝑧_{2} \cdot \frac{𝑖 𝑤_{1}}{2} \\ = 𝑧_{1} 𝑤_{1} + 2 (1 + 𝑖) 𝑤_{2} + 2 𝑧_{2} - 𝑖 𝑤_{1} / 2, \end{aligned}

which is $⟨ 𝐳, 𝐮 ⟩$ for $𝐮 =$ $(𝑤_{1} + (2 + 2 𝑖) 𝑤_{2}, - 𝑖 𝑤_{1} / 2)$ . By Proposition 6.6.9,

𝑇^{*} (𝑤_{1}, 𝑤_{2}) = (𝑤_{1} + (2 + 2 𝑖) 𝑤_{2}, - (𝑖 / 2) 𝑤_{1}) .

(c) The matrix of this $𝑇^{*}$ in the standard basis is $(\begin{matrix} 1 & 2 + 2 𝑖 \\ - 𝑖 / 2 & 0 \end{matrix}) \neq$ $𝐴^{*}$ . There is no contradiction: Theorem 6.6.15(b) needs orthonormal bases, and $𝐞_{1}, 𝐞_{2}$ is not orthonormal for the weighted inner product, since $⟨ 𝐞_{2}, 𝐞_{2} ⟩ = 2$ . The adjoint depends on the inner products, not only on the map.

Exercise 6.6.32.routine Give ${ℝ [𝑥]}_{\leq 2}$ the inner product $⟨ 𝑝, 𝑞 ⟩ =$ $\int_{- 1}^{1} 𝑝 (𝑡) 𝑞 (𝑡) d 𝑡$ . Find the polynomials $𝑞_{1}, 𝑞_{2} \in {ℝ [𝑥]}_{\leq 2}$ such that, for every $𝑝 \in {ℝ [𝑥]}_{\leq 2}$ ,

(a) $⟨ 𝑝, 𝑞_{1} ⟩ = 𝑝^{'} (0)$ ;
(b) $⟨ 𝑝, 𝑞_{2} ⟩ =$ $\int_{0}^{1} 𝑝 (𝑡) d 𝑡$ .

Solution.

Uses. Theorem 6.6.3 (Riesz representation theorem), Example 6.4.19 (Legendre polynomials), Corollary 6.4.10 (Projecting with an orthogonal list)

Hint 1. Use the orthogonal basis

1, 𝑥, 𝑥^{2} - 1 / 3

of Example 6.4.19, with squared norms

2

2 / 3

8 / 45

. For an orthogonal basis

𝑓_{𝑗}

, the formula of Theorem 6.6.3 becomes

𝑞 =

\sum_{𝑗} 𝜑 (𝑓_{𝑗}) / {‖ 𝑓_{𝑗} ‖}^{2} 𝑓_{𝑗}

over

ℝ

Hint 2. (a)

𝜑 (1) = 0

𝜑 (𝑥) = 1

𝜑 (𝑥^{2} - 1 / 3) = 0

. (b)

𝜑 (1) = 1

𝜑 (𝑥) = 1 / 2

𝜑 (𝑥^{2} - 1 / 3) = 0

Sketch. (a)

𝑞_{1} =

(1 / (2 / 3)) 𝑥 =

3 𝑥 / 2

. (b)

𝑞_{2} =

1 / 2 +

(1 / 2) / (2 / 3) 𝑥 =

1 / 2 +

3 𝑥 / 4

. Check both on

1, 𝑥, 𝑥^{2}

Let $𝑓_{1} = 1$ , $𝑓_{2} = 𝑥$ , $𝑓_{3} = 𝑥^{2} - 1 / 3$ , an orthogonal basis with ${‖ 𝑓_{1} ‖}^{2} = 2$ , ${‖ 𝑓_{2} ‖}^{2} = 2 / 3$ , ${‖ 𝑓_{3} ‖}^{2} = 8 / 45$ (Example 6.4.19). Normalizing, $𝑒_{𝑗} = 𝑓_{𝑗} / ‖ 𝑓_{𝑗} ‖$ , the formula of Theorem 6.6.3 gives, over $ℝ$ ,

𝑞 = \sum_{𝑗} 𝜑 (𝑒_{𝑗}) 𝑒_{𝑗} = \sum_{𝑗} \frac{𝜑 (𝑓_{𝑗})}{{‖ 𝑓_{𝑗} ‖}^{2}} 𝑓_{𝑗} .

(a) For $𝜑 (𝑝) = 𝑝^{'} (0)$ : $𝜑 (1) = 0$ , $𝜑 (𝑥) = 1$ , and $𝜑 (𝑥^{2} - 1 / 3) = 0$ . So $𝑞_{1} = (3 / 2) 𝑥$ . Check: for $𝑝 = 𝑎 + 𝑏 𝑥 + 𝑐 𝑥^{2}$ , $⟨ 𝑝, 𝑞_{1} ⟩ =$ $(3 / 2) (𝑎 \int_{- 1}^{1} 𝑡 d 𝑡 + 𝑏 \int_{- 1}^{1} 𝑡^{2} d 𝑡 + 𝑐 \int_{- 1}^{1} 𝑡^{3} d 𝑡) =$ $(3 / 2) (2 𝑏 / 3) =$ $𝑏 =$ $𝑝^{'} (0)$ .

(b) For $𝜑 (𝑝) = \int_{0}^{1} 𝑝$ : $𝜑 (1) = 1$ , $𝜑 (𝑥) = 1 / 2$ , and $𝜑 (𝑥^{2} - 1 / 3) =$ $1 / 3 -$ $1 / 3 =$ $0$ . So

𝑞_{2} = \frac{1}{2} + \frac{1 / 2}{2 / 3} 𝑥 = \frac{1}{2} + \frac{3}{4} 𝑥 .

Check: $⟨ 1, 𝑞_{2} ⟩ = 1 = \int_{0}^{1} 1$ , $⟨ 𝑥, 𝑞_{2} ⟩ =$ $(3 / 4) (2 / 3) =$ $1 / 2 =$ $\int_{0}^{1} 𝑡 d 𝑡$ , and $⟨ 𝑥^{2}, 𝑞_{2} ⟩ =$ $(1 / 2) (2 / 3) =$ $1 / 3 =$ $\int_{0}^{1} 𝑡^{2} d 𝑡$ .

Exercise 6.6.33.routine On $𝔽^{𝑛}$ with the dot product ( $𝑛 \geq 2$ ), let

𝑅 (𝑥_{1}, \dots, 𝑥_{𝑛}) = (0, 𝑥_{1}, \dots, 𝑥_{𝑛 - 1}), 𝐶 (𝑥_{1}, \dots, 𝑥_{𝑛}) = (𝑥_{𝑛}, 𝑥_{1}, \dots, 𝑥_{𝑛 - 1}) .

(a)Find $𝑅^{*}$ , and compute $𝑅^{*} 𝑅$ and $𝑅 𝑅^{*}$ . Identify each as an orthogonal projection.
(b)Find $𝐶^{*}$ , and show that $𝐶^{*} = 𝐶^{- 1}$ .

Solution.

Uses. Definition 6.1.8 (Dot product), Proposition 6.6.9 (Adjoints are unique and linear), Corollary 6.6.29 (Orthogonal projections are the self-adjoint idempotents), Definition 6.4.11 (Orthogonal projection)

Techniques. Find the adjoint by isolating the input (Technique 6.6.11)

Hint 1. Write

⟨ 𝑅 𝐱, 𝐲 ⟩ =

\sum_{𝑗 = 1}^{𝑛 - 1} 𝑥_{𝑗} 𝑦_{𝑗 + 1}

and read off the vector in the second slot.

Hint 2.

𝑅^{*}

shifts the other way. For (b),

𝐶

sends

𝐞_{𝑗}

𝐞_{𝑗 + 1}

and

𝐞_{𝑛}

𝐞_{1}

Sketch. (a)

𝑅^{*} (𝐲) =

(𝑦_{2}, \dots, 𝑦_{𝑛}, 0)

. Then

𝑅^{*} 𝑅 𝐱 =

(𝑥_{1}, \dots, 𝑥_{𝑛 - 1}, 0)

projects onto

span (𝐞_{1}, \dots, 𝐞_{𝑛 - 1})

and

𝑅 𝑅^{*} 𝐲 =

(0, 𝑦_{2}, \dots, 𝑦_{𝑛})

projects onto

span (𝐞_{2}, \dots, 𝐞_{𝑛})

. (b)

𝐶^{*} (𝐲) =

(𝑦_{2}, \dots, 𝑦_{𝑛}, 𝑦_{1})

, which undoes

𝐶

(a) For $𝐱, 𝐲 \in 𝔽^{𝑛}$ ,

⟨ 𝑅 𝐱, 𝐲 ⟩ = \sum_{𝑗 = 1}^{𝑛 - 1} 𝑥_{𝑗} 𝑦_{𝑗 + 1} = ⟨ 𝐱, (𝑦_{2}, \dots, 𝑦_{𝑛}, 0) ⟩,

so $𝑅^{*} (𝑦_{1}, \dots, 𝑦_{𝑛}) =$ $(𝑦_{2}, \dots, 𝑦_{𝑛}, 0)$ , the shift to the left. Then

𝑅^{*} 𝑅 𝐱 = 𝑅^{*} (0, 𝑥_{1}, \dots, 𝑥_{𝑛 - 1}) = (𝑥_{1}, \dots, 𝑥_{𝑛 - 1}, 0), 𝑅 𝑅^{*} 𝐲 = 𝑅 (𝑦_{2}, \dots, 𝑦_{𝑛}, 0) = (0, 𝑦_{2}, \dots, 𝑦_{𝑛}) .

The first keeps the coordinates along $𝐞_{1}, \dots, 𝐞_{𝑛 - 1}$ and kills the last, so it is ${Pr}_{𝑊}$ for $𝑊 =$ $span (𝐞_{1}, \dots, 𝐞_{𝑛 - 1})$ (Definition 6.4.11, with the orthonormal basis $𝐞_{1}, \dots, 𝐞_{𝑛 - 1}$ of $𝑊$ ). Likewise $𝑅 𝑅^{*} =$ ${Pr}_{span (𝐞_{2}, \dots, 𝐞_{𝑛})}$ . Note $𝑅^{*} 𝑅 \neq 𝑅 𝑅^{*}$ : the shift is not normal (Section 8.1).

(b) For $𝐱, 𝐲 \in 𝔽^{𝑛}$ ,

⟨ 𝐶 𝐱, 𝐲 ⟩ = 𝑥_{𝑛} 𝑦_{1} + \sum_{𝑗 = 1}^{𝑛 - 1} 𝑥_{𝑗} 𝑦_{𝑗 + 1} = ⟨ 𝐱, (𝑦_{2}, \dots, 𝑦_{𝑛}, 𝑦_{1}) ⟩,

so $𝐶^{*} (𝑦_{1}, \dots, 𝑦_{𝑛}) =$ $(𝑦_{2}, \dots, 𝑦_{𝑛}, 𝑦_{1})$ . Then $𝐶 (𝐶^{*} 𝐲) =$ $𝐶 (𝑦_{2}, \dots, 𝑦_{𝑛}, 𝑦_{1}) =$ $(𝑦_{1}, 𝑦_{2}, \dots, 𝑦_{𝑛})$ and $𝐶^{*} (𝐶 𝐱) =$ $𝐶^{*} (𝑥_{𝑛}, 𝑥_{1}, \dots, 𝑥_{𝑛 - 1}) =$ $(𝑥_{1}, \dots, 𝑥_{𝑛})$ . So $𝐶^{*}$ is a two-sided inverse of $𝐶$ .

Exercise 6.6.34.routine Let $𝐴 = (\begin{matrix} 1 & 𝑖 \\ 𝑖 & - 1 \end{matrix}) \in ℂ^{2 \times 2}$ , and give $ℂ^{2}$ the dot product.

(a)Find $ker 𝐴$ , $im 𝐴$ , $ker 𝐴^{*}$ , and $im 𝐴^{*}$ , and check the four statements of Theorem 6.6.21.
(b)Show that $ker 𝐴$ is not orthogonal to $im 𝐴^{𝖳}$ , although $𝐴^{𝖳} = 𝐴$ .
(c)Is $𝑇_{𝐴}$ self-adjoint?

Solution.

Uses. Theorem 6.6.21 (Kernels and images of adjoints), Definition 6.3.26 (Conjugate transpose), Definition 6.1.8 (Dot product), Definition 6.6.26 (Self-adjoint operator)

Hint 1. The second row is

𝑖

times the first, so

𝐴

has rank

1

and every subspace in question is a line.

Hint 2.

𝐴^{*} = (\begin{matrix} 1 & - 𝑖 \\ - 𝑖 & - 1 \end{matrix})

. Remember to conjugate the second vector in each dot product.

Sketch. (a)

ker 𝐴 =

span ((- 𝑖, 1))

im 𝐴 = span ((1, 𝑖))

ker 𝐴^{*} =

span ((𝑖, 1))

im 𝐴^{*} =

span ((1, - 𝑖))

. Then

(𝑖, 1) \cdot (1, 𝑖) = 0

and

(- 𝑖, 1) \cdot (1, - 𝑖) = 0

, and each pair consists of two lines in

ℂ^{2}

, so they are orthogonal complements. (b)

(- 𝑖, 1) \cdot

(1, 𝑖) =

- 2 𝑖

. (c) No,

𝐴^{*} \neq 𝐴

(a) The second row of $𝐴$ is $𝑖$ times the first, and the second column is $𝑖$ times the first, so $rank 𝐴 = 1$ . The kernel is given by $𝑥_{1} + 𝑖 𝑥_{2} = 0$ , so $ker 𝐴 =$ $span ((- 𝑖, 1))$ , and $im 𝐴 = span ((1, 𝑖))$ . Next, $𝐴^{*} = (\begin{matrix} 1 & - 𝑖 \\ - 𝑖 & - 1 \end{matrix})$ , again of rank $1$ : $ker 𝐴^{*}$ is given by $𝑥_{1} - 𝑖 𝑥_{2} = 0$ , so $ker 𝐴^{*} =$ $span ((𝑖, 1))$ , and $im 𝐴^{*} =$ $span ((1, - 𝑖))$ . Now

(𝑖, 1) \cdot (1, 𝑖) = 𝑖 + 𝑖 = 0, (- 𝑖, 1) \cdot (1, - 𝑖) = - 𝑖 + - 𝑖 = 0 .

So $ker 𝐴^{*} ⟂ im 𝐴$ and $ker 𝐴 ⟂ im 𝐴^{*}$ . In each case both are lines in $ℂ^{2}$ , and the perp of a line in $ℂ^{2}$ is a line (Corollary 6.4.25), so $ker 𝐴^{*} = {(im 𝐴)}^{⟂}$ and $ker 𝐴 = {(im 𝐴^{*})}^{⟂}$ , which are (a) and (b) of Theorem 6.6.21. Taking perps once more gives (c) and (d).

(b) $im 𝐴^{𝖳} =$ $im 𝐴 =$ $span ((1, 𝑖))$ , and $(- 𝑖, 1) \cdot$ $(1, 𝑖) =$ $- 𝑖 +$ $𝑖 =$ $- 2 𝑖 \neq$ $0$ .

(c) No: $𝐴^{*} =$ $(\begin{matrix} 1 & - 𝑖 \\ - 𝑖 & - 1 \end{matrix}) \neq$ $𝐴$ . A complex symmetric matrix need not be Hermitian, and (b) shows what goes wrong: the orthogonal complement of the kernel is spanned by the conjugated rows, not by the rows.

Exercise 6.6.35.standard Let

𝑉

be finite-dimensional and

𝑇 \in L (𝑉)

with

𝑇^{*} 𝑇 = 𝑇

. Show that

𝑇

is an orthogonal projection.

Solution.

Uses. Proposition 6.6.18 (Properties of the adjoint), Corollary 6.6.29 (Orthogonal projections are the self-adjoint idempotents)

Hint 1. Take the adjoint of both sides of

𝑇^{*} 𝑇 = 𝑇

Hint 2. You should find that

𝑇

is self-adjoint. Then

𝑇^{2} = 𝑇^{*} 𝑇

Sketch.

𝑇^{*} =

{(𝑇^{*} 𝑇)}^{*} =

𝑇^{*} 𝑇^{* *} =

𝑇^{*} 𝑇 =

𝑇

, so

𝑇

is self-adjoint, and

𝑇^{2} = 𝑇^{*} 𝑇 = 𝑇

. By Corollary 6.6.29,

𝑇 = {Pr}_{im 𝑇}

By (c) and (d) of Proposition 6.6.18,

𝑇^{*} = {(𝑇^{*} 𝑇)}^{*} = 𝑇^{*} {(𝑇^{*})}^{*} = 𝑇^{*} 𝑇 = 𝑇 .

So $𝑇$ is self-adjoint, and then $𝑇^{2} = 𝑇^{*} 𝑇 = 𝑇$ . By Corollary 6.6.29, $𝑇$ is the orthogonal projection onto $im 𝑇$ .

Exercise 6.6.36.standard Let $𝑉$ be a finite-dimensional inner product space, and let $𝑆, 𝑇 \in L (𝑉)$ be self-adjoint. Prove or disprove each statement.

(a) $𝑆 + 𝑇$ is self-adjoint.
(b) $𝑆 𝑇$ is self-adjoint.
(c) $𝑆 𝑇 𝑆$ is self-adjoint.
(d)If $𝔽 = ℂ$ , then $𝑖 𝑇$ is self-adjoint.
(e)If $𝑇$ is invertible, then $𝑇^{- 1}$ is self-adjoint.

Solution.

Uses. Proposition 6.6.18 (Properties of the adjoint), Corollary 6.6.28 (Self-adjointness in an orthonormal basis)

Techniques. Finding counterexamples and minimal examples (Technique 1.4.18)

Hint 1. Compute each adjoint with Proposition 6.6.18 and compare with the original operator.

Hint 2. For (b),

{(𝑆 𝑇)}^{*} = 𝑇 𝑆

. Look for two symmetric

2 \times 2

matrices that do not commute.

Sketch. (a) True. (b) False in general:

{(𝑆 𝑇)}^{*} = 𝑇 𝑆

, and

𝑆 = (\begin{matrix} 1 & 0 \\ 0 & 0 \end{matrix})

𝑇 = (\begin{matrix} 0 & 1 \\ 1 & 0 \end{matrix})

give

𝑆 𝑇 = (\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix})

;

𝑆 𝑇

is self-adjoint exactly when

𝑆 𝑇 = 𝑇 𝑆

. (c) True:

{(𝑆 𝑇 𝑆)}^{*} = 𝑆 𝑇 𝑆

. (d) False unless

𝑇 = 0

{(𝑖 𝑇)}^{*} = - 𝑖 𝑇

. (e) True:

{(𝑇^{- 1})}^{*} =

{(𝑇^{*})}^{- 1} =

𝑇^{- 1}

(a) True: ${(𝑆 + 𝑇)}^{*} =$ $𝑆^{*} +$ $𝑇^{*} =$ $𝑆 +$ $𝑇$ by Proposition 6.6.18(a).

(b) False in general. By Proposition 6.6.18(d), ${(𝑆 𝑇)}^{*} =$ $𝑇^{*} 𝑆^{*} =$ $𝑇 𝑆$ , so $𝑆 𝑇$ is self-adjoint exactly when $𝑆 𝑇 = 𝑇 𝑆$ . On $ℝ^{2}$ with the dot product, $𝑆 = (\begin{matrix} 1 & 0 \\ 0 & 0 \end{matrix})$ and $𝑇 = (\begin{matrix} 0 & 1 \\ 1 & 0 \end{matrix})$ are symmetric, but $𝑆 𝑇 = (\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix})$ is not, so $𝑇_{𝑆} 𝑇_{𝑇}$ is not self-adjoint (Corollary 6.6.28).

(c) True: ${(𝑆 𝑇 𝑆)}^{*} =$ $𝑆^{*} 𝑇^{*} 𝑆^{*} =$ $𝑆 𝑇 𝑆$ , applying Proposition 6.6.18(d) twice.

(d) False unless $𝑇 = 0$ . By Proposition 6.6.18(b), ${(𝑖 𝑇)}^{*} =$ $𝑖 𝑇^{*} =$ $- 𝑖 𝑇$ , which equals $𝑖 𝑇$ only if $2 𝑖 𝑇 = 0$ , that is, $𝑇 = 0$ . So the self-adjoint operators on a complex space are closed under addition and real scalar multiples, but they do not form a complex subspace of $L (𝑉)$ .

(e) True: by Proposition 6.6.18(f), ${(𝑇^{- 1})}^{*} =$ ${(𝑇^{*})}^{- 1} =$ $𝑇^{- 1}$ .

Exercise 6.6.37.standard Let $𝑉$ be a finite-dimensional complex inner product space.

(a)Show that every $𝑇 \in L (𝑉)$ can be written in exactly one way as $𝑇 = 𝐴 + 𝑖 𝐵$ with $𝐴, 𝐵 \in L (𝑉)$ self-adjoint.
(b)Find $𝐴$ and $𝐵$ for $𝑇 = 𝑇_{𝑀}$ on $ℂ^{2}$ with the dot product, where $𝑀 = (\begin{matrix} 1 & 2 \\ 0 & 𝑖 \end{matrix})$ .

Solution.

Uses. Proposition 6.6.18 (Properties of the adjoint), Definition 6.6.26 (Self-adjoint operator), Example 6.6.10 (The adjoint of a matrix)

Techniques. Uniqueness via two candidates (Technique 1.1.8)

Hint 1. Imitate

𝑥 = (𝑧 + 𝑧) / 2

and

𝑦 = (𝑧 - 𝑧) / (2 𝑖)

Hint 2. For uniqueness, if

𝐴 + 𝑖 𝐵 = 𝐴^{'} + 𝑖 𝐵^{'}

, then

𝐶 = 𝐴 - 𝐴^{'}

is self-adjoint and also

𝑖

times a self-adjoint operator. Compute

𝐶^{*}

in two ways.

Sketch. (a)

𝐴 = (𝑇 + 𝑇^{*}) / 2

and

𝐵 = (𝑇 - 𝑇^{*}) / (2 𝑖)

are self-adjoint with

𝐴 + 𝑖 𝐵 = 𝑇

. If also

𝑇 = 𝐴^{'} + 𝑖 𝐵^{'}

, then

𝐶 =

𝐴 -

𝐴^{'} =

𝑖 (𝐵^{'} - 𝐵)

satisfies

𝐶^{*} = 𝐶

and

𝐶^{*} =

- 𝑖 (𝐵^{'} - 𝐵) =

- 𝐶

, so

𝐶 = 0

. (b)

𝐴 = (\begin{matrix} 1 & 1 \\ 1 & 0 \end{matrix})

𝐵 = (\begin{matrix} 0 & - 𝑖 \\ 𝑖 & 1 \end{matrix})

(a) Let $𝐴 = (𝑇 + 𝑇^{*}) / 2$ and $𝐵 = (𝑇 - 𝑇^{*}) / (2 𝑖)$ . By Proposition 6.6.18, $𝐴^{*} = (𝑇^{*} + 𝑇) / 2 = 𝐴$ , and since $1 / (2 𝑖) = - 1 / (2 𝑖)$ ,

𝐵^{*} = - \frac{1}{2 𝑖} (𝑇^{*} - 𝑇) = \frac{1}{2 𝑖} (𝑇 - 𝑇^{*}) = 𝐵 .

Also $𝐴 +$ $𝑖 𝐵 =$ $(𝑇 + 𝑇^{*}) / 2 +$ $(𝑇 - 𝑇^{*}) / 2 =$ $𝑇$ . For uniqueness, suppose $𝑇 = 𝐴^{'} + 𝑖 𝐵^{'}$ with $𝐴^{'}, 𝐵^{'}$ self-adjoint, and let $𝐶 =$ $𝐴 -$ $𝐴^{'} =$ $𝑖 (𝐵^{'} - 𝐵)$ . Then $𝐶$ is self-adjoint, as a difference of self-adjoint operators, and also $𝐶^{*} =$ $𝑖 {(𝐵^{'} - 𝐵)}^{*} =$ $- 𝑖 (𝐵^{'} - 𝐵) =$ $- 𝐶$ . So $𝐶 = - 𝐶$ , hence $𝐶 = 0$ , $𝐴 = 𝐴^{'}$ , and $𝐵^{'} - 𝐵 = - 𝑖 𝐶 = 0$ .

(b) By Example 6.6.10 we may compute with matrices. Here $𝑀^{*} = (\begin{matrix} 1 & 0 \\ 2 & - 𝑖 \end{matrix})$ , so

\begin{aligned} 𝐴 & = \frac{1}{2} (𝑀 + 𝑀^{*}) = \frac{1}{2} (\begin{matrix} 2 & 2 \\ 2 & 0 \end{matrix}) = (\begin{matrix} 1 & 1 \\ 1 & 0 \end{matrix}), \\ 𝐵 & = \frac{1}{2 𝑖} (𝑀 - 𝑀^{*}) = \frac{1}{2 𝑖} (\begin{matrix} 0 & 2 \\ - 2 & 2 𝑖 \end{matrix}) = (\begin{matrix} 0 & - 𝑖 \\ 𝑖 & 1 \end{matrix}) . \end{aligned}

Both are Hermitian, and $𝐴 +$ $𝑖 𝐵 =$ $(\begin{matrix} 1 & 1 + 1 \\ 1 - 1 & 𝑖 \end{matrix}) =$ $𝑀$ .

Exercise 6.6.38.standard Let

𝑉

be finite-dimensional,

𝑇 \in L (𝑉)

, and

𝑈

a subspace of

𝑉

. Show that

𝑇

maps

𝑈

into

𝑈

if and only if

𝑇^{*}

maps

𝑈^{⟂}

into

𝑈^{⟂}

. Deduce that if

𝑇

is self-adjoint and maps

𝑈

into

𝑈

, then

𝑇

also maps

𝑈^{⟂}

into

𝑈^{⟂}

Solution.

Uses. Definition 6.6.8 (Adjoint), Definition 6.4.22 (Orthogonal complement), Proposition 6.6.18 (Properties of the adjoint), Theorem 6.4.26 (The double complement)

Techniques. Move an operator across the inner product (Technique 6.6.20)

Hint 1. Take

𝐰 \in 𝑈^{⟂}

and

𝐮 \in 𝑈

, and move

𝑇^{*}

across in

⟨ 𝑇^{*} 𝐰, 𝐮 ⟩

Hint 2. For the converse, apply the first direction to

𝑇^{*}

and the subspace

𝑈^{⟂}

Sketch. If

𝑇 (𝑈) \subseteq 𝑈

, then for

𝐰 \in 𝑈^{⟂}

and

𝐮 \in 𝑈

⟨ 𝑇^{*} 𝐰, 𝐮 ⟩ =

⟨ 𝐰, 𝑇 𝐮 ⟩ =

0

, so

𝑇^{*} 𝐰 \in 𝑈^{⟂}

. Conversely apply this to

𝑇^{*}

and

𝑈^{⟂}

, using

{(𝑇^{*})}^{*} = 𝑇

and

{(𝑈^{⟂})}^{⟂} = 𝑈

Suppose $𝑇$ maps $𝑈$ into $𝑈$ . Let $𝐰 \in 𝑈^{⟂}$ and $𝐮 \in 𝑈$ . Then $𝑇 𝐮 \in 𝑈$ , so

⟨ 𝑇^{*} 𝐰, 𝐮 ⟩ = ⟨ 𝐰, 𝑇 𝐮 ⟩ = 0 .

As $𝐮 \in 𝑈$ was arbitrary, $𝑇^{*} 𝐰 \in 𝑈^{⟂}$ . Conversely, suppose $𝑇^{*}$ maps $𝑈^{⟂}$ into $𝑈^{⟂}$ . Applying what we just proved to the operator $𝑇^{*}$ and the subspace $𝑈^{⟂}$ , the operator ${(𝑇^{*})}^{*} = 𝑇$ (Proposition 6.6.18(c)) maps ${(𝑈^{⟂})}^{⟂}$ into itself, and ${(𝑈^{⟂})}^{⟂} = 𝑈$ by Theorem 6.4.26. If $𝑇$ is self-adjoint and maps $𝑈$ into $𝑈$ , then $𝑇^{*} = 𝑇$ maps $𝑈^{⟂}$ into $𝑈^{⟂}$ . This is the step that lets the spectral theorem (Section 8.2) split $𝑉$ into perpendicular pieces one at a time.

Exercise 6.6.39.standard Let $𝑉$ be the real vector space spanned by $1,$ $cos 𝑡,$ $sin 𝑡,$ $cos 2 𝑡,$ $sin 2 𝑡$ (as functions on $[0, 2 𝜋]$ ), with $⟨ 𝑓, 𝑔 ⟩ =$ $\int_{0}^{2 𝜋} 𝑓 (𝑡) 𝑔 (𝑡) d 𝑡$ , an inner product by the proof of Proposition 6.1.13. Let $𝐷 𝑓 = 𝑓^{'}$ .

(a)Show that $𝐷$ maps $𝑉$ into $𝑉$ and that $𝐷^{*} = - 𝐷$ .
(b)Deduce that $⟨ 𝐷 𝑓, 𝑓 ⟩ = 0$ for every $𝑓 \in 𝑉$ , and that $𝐷$ is not self-adjoint.
(c)Find the matrix of $𝐷$ in the basis $1,$ $cos 𝑡,$ $sin 𝑡,$ $cos 2 𝑡,$ $sin 2 𝑡$ . Using that this basis is orthogonal with $‖ cos 𝑘 𝑡 ‖ =$ $‖ sin 𝑘 𝑡 ‖$ , explain how the matrix also shows $𝐷^{*} = - 𝐷$ .

Solution.

Uses. Proposition 6.6.9 (Adjoints are unique and linear), Theorem 6.6.15 (Existence of adjoints), Proposition 6.6.18 (Properties of the adjoint)

Hint 1. Integrate by parts:

\int_{0}^{2 𝜋} 𝑓^{'} 𝑔 =

{[𝑓 𝑔]}_{0}^{2 𝜋} -

\int_{0}^{2 𝜋} 𝑓 𝑔^{'}

. What is

{[𝑓 𝑔]}_{0}^{2 𝜋}

for these functions?

Hint 2. For (c), normalizing the basis divides

cos 𝑘 𝑡

and

sin 𝑘 𝑡

by the same number, so the

2 \times 2

blocks of the matrix do not change. Then use Theorem 6.6.15(b).

Sketch. (a)

𝐷

sends each spanning function into

𝑉

. Every

𝑓 \in 𝑉

satisfies

𝑓 (2 𝜋) = 𝑓 (0)

, so integration by parts gives

⟨ 𝐷 𝑓, 𝑔 ⟩ =

- ⟨ 𝑓, 𝐷 𝑔 ⟩

, that is,

𝐷^{*} = - 𝐷

. (b)

⟨ 𝐷 𝑓, 𝑓 ⟩ =

- ⟨ 𝑓, 𝐷 𝑓 ⟩ =

- ⟨ 𝐷 𝑓, 𝑓 ⟩

. (c) The matrix is block diagonal with blocks

(0)

(\begin{matrix} 0 & 1 \\ - 1 & 0 \end{matrix})

(\begin{matrix} 0 & 2 \\ - 2 & 0 \end{matrix})

; it stays the same in the orthonormal basis obtained by normalizing, and it is antisymmetric.

(a) $𝐷 1 = 0$ , $𝐷 cos 𝑘 𝑡 =$ $- 𝑘 sin 𝑘 𝑡$ , and $𝐷 sin 𝑘 𝑡 =$ $𝑘 cos 𝑘 𝑡$ , so $𝐷$ maps the spanning functions, and hence all of $𝑉$ , into $𝑉$ . Every $𝑓 \in 𝑉$ is $2 𝜋$ -periodic, so $𝑓 (2 𝜋) = 𝑓 (0)$ . For $𝑓, 𝑔 \in 𝑉$ , integration by parts gives

⟨ 𝐷 𝑓, 𝑔 ⟩ = \int_{0}^{2 𝜋} 𝑓^{'} (𝑡) 𝑔 (𝑡) d 𝑡 = 𝑓 (2 𝜋) 𝑔 (2 𝜋) - 𝑓 (0) 𝑔 (0) - \int_{0}^{2 𝜋} 𝑓 (𝑡) 𝑔^{'} (𝑡) d 𝑡 = ⟨ 𝑓, - 𝐷 𝑔 ⟩ .

By Proposition 6.6.9, $𝐷^{*} = - 𝐷$ .

(b) For $𝑓 \in 𝑉$ , $⟨ 𝐷 𝑓, 𝑓 ⟩ =$ $⟨ 𝑓, - 𝐷 𝑓 ⟩ =$ $- ⟨ 𝑓, 𝐷 𝑓 ⟩ =$ $- ⟨ 𝐷 𝑓, 𝑓 ⟩$ , by symmetry of a real inner product. So $⟨ 𝐷 𝑓, 𝑓 ⟩ = 0$ . Since $𝐷 sin 𝑡 = cos 𝑡 \neq 0$ , $𝐷 \neq 0$ , so $𝐷^{*} = - 𝐷 \neq 𝐷$ , and $𝐷$ is not self-adjoint.

(c) From the formulas in (a), the matrix of $𝐷$ in the basis $1,$ $cos 𝑡,$ $sin 𝑡,$ $cos 2 𝑡,$ $sin 2 𝑡$ is

(\begin{matrix} 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & - 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 2 \\ 0 & 0 & 0 & - 2 & 0 \end{matrix}) .

The basis is orthogonal, with $‖ 1 ‖ = \sqrt{2 𝜋}$ and $‖ cos 𝑘 𝑡 ‖ =$ $‖ sin 𝑘 𝑡 ‖ =$ $\sqrt{𝜋}$ . Dividing each basis function by its norm rescales $cos 𝑘 𝑡$ and $sin 𝑘 𝑡$ by the same factor, so in the resulting orthonormal basis the matrix is unchanged. It is antisymmetric, so by Theorem 6.6.15(b) the matrix of $𝐷^{*}$ is its transpose, which is its negative: $𝐷^{*} = - 𝐷$ again.

Exercise 6.6.40.standard Let

𝑉

and

𝑊

be finite-dimensional and

𝑇 \in L (𝑉, 𝑊)

. Show that the following are equivalent:

(i) $‖ 𝑇 𝐯 ‖ = ‖ 𝐯 ‖$ for all $𝐯 \in 𝑉$ ;
(ii) $⟨ 𝑇 𝐯, 𝑇 𝐰 ⟩ =$ $⟨ 𝐯, 𝐰 ⟩$ for all $𝐯, 𝐰 \in 𝑉$ ;
(iii) $𝑇^{*} 𝑇 = {id}_{𝑉}$ .

What does (iii) say about

𝐴

when

𝑇 = 𝑇_{𝐴}

with

𝐴 \in 𝔽^{𝑚 \times 𝑛}

and dot products?

Solution.

Uses. Exercise 6.1.51, Corollary 6.1.38 (Linear maps are determined by inner products), Definition 6.6.8 (Adjoint), Theorem 6.3.29 (Orthonormal columns), Example 6.6.10 (The adjoint of a matrix)

Techniques. Proving TFAE (Technique 2.7.1), Move an operator across the inner product (Technique 6.6.20)

Hint 1. (i)

\Rightarrow

(ii) is polarization, Exercise 6.1.51.

Hint 2. For (ii)

\Rightarrow

(iii), move

𝑇

out of the second slot:

⟨ 𝑇 𝐯, 𝑇 𝐰 ⟩ =

⟨ 𝑇^{*} 𝑇 𝐯, 𝐰 ⟩

. Then use Corollary 6.1.38.

Sketch. (i)

\Rightarrow

(ii) by polarization. (ii)

\Rightarrow

(iii):

⟨ 𝑇^{*} 𝑇 𝐯, 𝐰 ⟩ =

⟨ 𝑇 𝐯, 𝑇 𝐰 ⟩ =

⟨ 𝐯, 𝐰 ⟩

for all

𝐯, 𝐰

, so

𝑇^{*} 𝑇 = {id}_{𝑉}

. (iii)

\Rightarrow

(i):

{‖ 𝑇 𝐯 ‖}^{2} =

⟨ 𝑇^{*} 𝑇 𝐯, 𝐯 ⟩ =

{‖ 𝐯 ‖}^{2}

. For

𝑇_{𝐴}

, (iii) is

𝐴^{*} 𝐴 = 𝐼_{𝑛}

: the columns of

𝐴

are orthonormal.

We prove (i) $\Rightarrow$ (ii) $\Rightarrow$ (iii) $\Rightarrow$ (i). The first implication is Exercise 6.1.51. For the second, the conjugated form of the defining identity, $⟨ 𝐱, 𝑇 𝐰 ⟩ =$ $⟨ 𝑇^{*} 𝐱, 𝐰 ⟩$ , with $𝐱 = 𝑇 𝐯$ gives

⟨ 𝑇^{*} 𝑇 𝐯, 𝐰 ⟩ = ⟨ 𝑇 𝐯, 𝑇 𝐰 ⟩ = ⟨ 𝐯, 𝐰 ⟩ = ⟨ {id}_{𝑉} 𝐯, 𝐰 ⟩

for all $𝐯, 𝐰 \in 𝑉$ , so $𝑇^{*} 𝑇 = {id}_{𝑉}$ by Corollary 6.1.38. For the third, ${‖ 𝑇 𝐯 ‖}^{2} =$ $⟨ 𝑇 𝐯, 𝑇 𝐯 ⟩ =$ $⟨ 𝑇^{*} 𝑇 𝐯, 𝐯 ⟩ =$ $⟨ 𝐯, 𝐯 ⟩ =$ ${‖ 𝐯 ‖}^{2}$ .

For $𝑇 = 𝑇_{𝐴}$ , $𝑇^{*} 𝑇 = 𝑇_{𝐴^{*} 𝐴}$ (Example 6.6.10), so (iii) says $𝐴^{*} 𝐴 = 𝐼_{𝑛}$ , that is, the columns of $𝐴$ are orthonormal (Theorem 6.3.29). Such maps are studied in Section 8.4.

Exercise 6.6.41.standard Let $𝑉$ and $𝑊$ be finite-dimensional and $𝑇 \in L (𝑉, 𝑊)$ with $𝑇^{*} 𝑇 = {id}_{𝑉}$ . A student argues: “Since $𝑇^{*} 𝑇 = {id}_{𝑉}$ , $𝑇^{*}$ is the inverse of $𝑇$ , so also $𝑇 𝑇^{*} = {id}_{𝑊}$ .”

(a)Find the error, and give a counterexample to the conclusion.
(b)Show that $𝑇 𝑇^{*} = {Pr}_{im 𝑇}$ .
(c)Show that the conclusion is correct when $dim 𝑉 = dim 𝑊$ .

Solution.

Uses. Definition 2.5.12 (Invertible), Corollary 6.6.29 (Orthogonal projections are the self-adjoint idempotents), Proposition 6.6.18 (Properties of the adjoint), Proposition 2.5.4 (Left inverses give uniqueness), Theorem 4.2.11 (Rank–nullity)

Hint 1.

𝑇^{*} 𝑇 = {id}_{𝑉}

says that

𝑇^{*}

is a left inverse. Try

𝑇 : ℝ \to ℝ^{2}

𝑇 𝑥 = (𝑥, 0)

Hint 2. For (b), check that

𝑃 = 𝑇 𝑇^{*}

satisfies

𝑃^{2} = 𝑃 = 𝑃^{*}

, and compare

im 𝑃

with

im 𝑇

Hint 3. For (c),

𝑇

is injective. What does rank–nullity say when

dim 𝑉 = dim 𝑊

Sketch. (a) A left inverse need not be a right inverse; for

𝑇 𝑥 = (𝑥, 0)

𝑇^{*} (𝑦_{1}, 𝑦_{2}) = 𝑦_{1}

𝑇^{*} 𝑇 = {id}_{ℝ}

, but

𝑇 𝑇^{*} (𝑦_{1}, 𝑦_{2}) =

(𝑦_{1}, 0)

. (b)

{(𝑇 𝑇^{*})}^{2} =

𝑇 (𝑇^{*} 𝑇) 𝑇^{*} =

𝑇 𝑇^{*}

and

{(𝑇 𝑇^{*})}^{*} = 𝑇 𝑇^{*}

, so

𝑇 𝑇^{*}

is the orthogonal projection onto its image, which is

im 𝑇

because

𝑇 = (𝑇 𝑇^{*}) 𝑇

. (c)

𝑇

is injective, so if

dim 𝑉 = dim 𝑊

it is surjective, and

𝑇 𝑇^{*} = {Pr}_{𝑊} = {id}_{𝑊}

(a) The equation $𝑇^{*} 𝑇 = {id}_{𝑉}$ only says that $𝑇^{*}$ is a left inverse of $𝑇$ ; an inverse must also be a right inverse (Definition 2.5.12). Let $𝑇 : ℝ \to ℝ^{2}$ , $𝑇 𝑥 = (𝑥, 0)$ , with dot products. Then $⟨ 𝑇 𝑥, 𝐲 ⟩ =$ $𝑥 𝑦_{1} =$ $⟨ 𝑥, 𝑦_{1} ⟩$ , so $𝑇^{*} (𝑦_{1}, 𝑦_{2}) = 𝑦_{1}$ and $𝑇^{*} 𝑇 𝑥 = 𝑥$ . But $𝑇 𝑇^{*} (𝑦_{1}, 𝑦_{2}) =$ $(𝑦_{1}, 0)$ , which is not ${id}_{ℝ^{2}}$ .

(b) Let $𝑃 = 𝑇 𝑇^{*}$ . By Proposition 6.6.18,

𝑃^{2} = 𝑇 (𝑇^{*} 𝑇) 𝑇^{*} = 𝑇 𝑇^{*} = 𝑃, 𝑃^{*} = {(𝑇^{*})}^{*} 𝑇^{*} = 𝑇 𝑇^{*} = 𝑃 .

By Corollary 6.6.29, $𝑃 = {Pr}_{im 𝑃}$ . Clearly $im 𝑃 \subseteq im 𝑇$ , and $𝑇 = 𝑇 (𝑇^{*} 𝑇) = 𝑃 𝑇$ shows $im 𝑇 \subseteq im 𝑃$ . So $𝑇 𝑇^{*} = {Pr}_{im 𝑇}$ .

(c) $𝑇$ has a left inverse, so it is injective (Proposition 2.5.4). By Theorem 4.2.11, $rank 𝑇 =$ $dim 𝑉 =$ $dim 𝑊$ , so $im 𝑇 = 𝑊$ , and by (b), $𝑇 𝑇^{*} = {Pr}_{𝑊} = {id}_{𝑊}$ .

Exercise 6.6.42.standard Let $𝑉$ and $𝑊$ be inner product spaces, let $𝑇 \in L (𝑉, 𝑊)$ have an adjoint, and let $𝐰 \in 𝑊$ .

(a)Show that $Re ⟨ 𝑇 𝐯, 𝐰 ⟩ \leq$ $‖ 𝑇^{*} 𝐰 ‖$ for every unit vector $𝐯 \in 𝑉$ , with equality for $𝐯 = 𝑇^{*} 𝐰 / ‖ 𝑇^{*} 𝐰 ‖$ when $𝑇^{*} 𝐰 \neq 𝟎$ . So $𝑇^{*} 𝐰$ points along the input that $𝑇$ pushes furthest in the direction of $𝐰$ .
(b)Let $𝑐 \geq 0$ . Show that $‖ 𝑇 𝐯 ‖ \leq 𝑐 ‖ 𝐯 ‖$ for all $𝐯 \in 𝑉$ if and only if $‖ 𝑇^{*} 𝐰 ‖ \leq 𝑐 ‖ 𝐰 ‖$ for all $𝐰 \in 𝑊$ .

Solution.

Uses. Definition 6.6.8 (Adjoint), Theorem 6.2.1 (Cauchy–Schwarz inequality), Notation 6.1.1 (Real and complex scalars), Proposition 6.6.18 (Properties of the adjoint)

Techniques. Move an operator across the inner product (Technique 6.6.20)

Hint 1. For (a), move

𝑇

across and apply Cauchy–Schwarz to

⟨ 𝐯, 𝑇^{*} 𝐰 ⟩

Hint 2. For (b), write

{‖ 𝑇^{*} 𝐰 ‖}^{2} =

⟨ 𝑇 𝑇^{*} 𝐰, 𝐰 ⟩

and bound it by Cauchy–Schwarz and the hypothesis on

𝑇

Sketch. (a)

Re ⟨ 𝑇 𝐯, 𝐰 ⟩ =

Re ⟨ 𝐯, 𝑇^{*} 𝐰 ⟩ \leq

| ⟨ 𝐯, 𝑇^{*} 𝐰 ⟩ | \leq

‖ 𝑇^{*} 𝐰 ‖

, with equality at

𝐯 = 𝑇^{*} 𝐰 / ‖ 𝑇^{*} 𝐰 ‖

. (b)

{‖ 𝑇^{*} 𝐰 ‖}^{2} =

⟨ 𝑇 𝑇^{*} 𝐰, 𝐰 ⟩ \leq

‖ 𝑇 𝑇^{*} 𝐰 ‖ ‖ 𝐰 ‖ \leq

𝑐 ‖ 𝑇^{*} 𝐰 ‖ ‖ 𝐰 ‖

; divide by

‖ 𝑇^{*} 𝐰 ‖

. The converse is the same argument for

𝑇^{*}

, whose adjoint is

𝑇

(a) For a unit vector $𝐯$ , by $| Re 𝑧 | \leq | 𝑧 |$ (Notation 6.1.1) and Cauchy–Schwarz (Theorem 6.2.1),

Re ⟨ 𝑇 𝐯, 𝐰 ⟩ = Re ⟨ 𝐯, 𝑇^{*} 𝐰 ⟩ \leq | ⟨ 𝐯, 𝑇^{*} 𝐰 ⟩ | \leq ‖ 𝐯 ‖ ‖ 𝑇^{*} 𝐰 ‖ = ‖ 𝑇^{*} 𝐰 ‖ .

If $𝑇^{*} 𝐰 \neq 𝟎$ and $𝐯 = 𝑇^{*} 𝐰 / ‖ 𝑇^{*} 𝐰 ‖$ , then $⟨ 𝐯, 𝑇^{*} 𝐰 ⟩ =$ ${‖ 𝑇^{*} 𝐰 ‖}^{2} / ‖ 𝑇^{*} 𝐰 ‖ =$ $‖ 𝑇^{*} 𝐰 ‖$ , a real number, so equality holds.

(b) Suppose $‖ 𝑇 𝐯 ‖ \leq 𝑐 ‖ 𝐯 ‖$ for all $𝐯$ , and let $𝐰 \in 𝑊$ . By the defining identity with $𝐯 = 𝑇^{*} 𝐰$ , Cauchy–Schwarz, and the hypothesis,

{‖ 𝑇^{*} 𝐰 ‖}^{2} = ⟨ 𝑇 𝑇^{*} 𝐰, 𝐰 ⟩ \leq ‖ 𝑇 𝑇^{*} 𝐰 ‖ ‖ 𝐰 ‖ \leq 𝑐 ‖ 𝑇^{*} 𝐰 ‖ ‖ 𝐰 ‖ .

If $𝑇^{*} 𝐰 \neq 𝟎$ , dividing by $‖ 𝑇^{*} 𝐰 ‖$ gives $‖ 𝑇^{*} 𝐰 ‖ \leq 𝑐 ‖ 𝐰 ‖$ ; if $𝑇^{*} 𝐰 = 𝟎$ this is clear. Conversely, $𝑇^{*}$ has adjoint $𝑇$ (Proposition 6.6.18(c)), so the same argument with the roles of $𝑇$ and $𝑇^{*}$ exchanged proves the other implication.

Exercise 6.6.43.standard Let $𝑉$ be finite-dimensional and $𝑇 \in L (𝑉)$ .

(a)Show that every vector of $im 𝑇$ is orthogonal to every vector of $im 𝑇^{*}$ if and only if $𝑇^{2} = 0$ .
(b)Deduce that if $𝑇^{2} = 0$ , then $rank 𝑇 \leq (dim 𝑉) / 2$ .
(c)Find a nonzero operator $𝑇$ on $ℝ^{3}$ with $𝑇^{2} = 0$ , and check (a) for it.

Solution.

Uses. Theorem 6.6.21 (Kernels and images of adjoints), Definition 6.3.8 (Vectors orthogonal to a set), Theorem 4.2.11 (Rank–nullity), Example 6.6.10 (The adjoint of a matrix)

Techniques. Dimension counting (Technique 4.2.24)

Hint 1. Every vector of

im 𝑇

is orthogonal to

im 𝑇^{*}

exactly when

im 𝑇 \subseteq {(im 𝑇^{*})}^{⟂}

. Which subspace is

{(im 𝑇^{*})}^{⟂}

Hint 2. For (b),

im 𝑇 \subseteq ker 𝑇

; apply rank–nullity.

Sketch. (a) The condition says

im 𝑇 \subseteq

{(im 𝑇^{*})}^{⟂} =

ker 𝑇

(Theorem 6.6.21(b)), that is,

𝑇 (𝑇 𝐯) = 𝟎

for all

𝐯

. (b)

rank 𝑇 \leq

dim ker 𝑇 =

dim 𝑉 -

rank 𝑇

. (c)

𝑇_{𝑁}

with

𝑁

sending

𝐞_{3} \mapsto 𝐞_{1}

and killing

𝐞_{1}, 𝐞_{2}

im 𝑇_{𝑁} = span (𝐞_{1})

and

im 𝑇_{𝑁}^{*} = span (𝐞_{3})

(a) Every vector of $im 𝑇$ is orthogonal to every vector of $im 𝑇^{*}$ if and only if $im 𝑇 \subseteq {(im 𝑇^{*})}^{⟂}$ , and ${(im 𝑇^{*})}^{⟂} = ker 𝑇$ by Theorem 6.6.21(b). Now $im 𝑇 \subseteq ker 𝑇$ means $𝑇 (𝑇 𝐯) = 𝟎$ for all $𝐯$ , that is, $𝑇^{2} = 0$ .

(b) If $𝑇^{2} = 0$ , then $im 𝑇 \subseteq ker 𝑇$ , so by Theorem 4.2.11, $rank 𝑇 \leq$ $dim ker 𝑇 =$ $dim 𝑉 -$ $rank 𝑇$ , and $2 rank 𝑇 \leq dim 𝑉$ .

(c) Let $𝑁 = (\begin{matrix} 0 & 0 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{matrix})$ , so that $𝑁 𝐞_{3} = 𝐞_{1}$ and $𝑁 𝐞_{1} = 𝑁 𝐞_{2} = 𝟎$ . Then $𝑁^{2} 𝐞_{3} = 𝑁 𝐞_{1} = 𝟎$ , so $𝑁^{2} = 0$ , while $𝑁 \neq 0$ . Here $im 𝑇_{𝑁} = span (𝐞_{1})$ , and $𝑇_{𝑁}^{*} = 𝑇_{𝑁^{𝖳}}$ (Example 6.6.10) has $𝑁^{𝖳} 𝐞_{1} = 𝐞_{3}$ , so $im 𝑇_{𝑁}^{*} = span (𝐞_{3})$ , which is orthogonal to $𝐞_{1}$ . Also $rank 𝑁 = 1 \leq 3 / 2$ .

Exercise 6.6.44.standard Let $𝑉$ be finite-dimensional and $𝑇 \in L (𝑉)$ .

(a)Show that $rank (𝑇 𝑇^{*}) =$ $rank (𝑇^{*} 𝑇) =$ $rank 𝑇$ .
(b)Suppose $𝑇^{*} 𝑇 = 𝑇 𝑇^{*}$ . Show that $‖ 𝑇 𝐯 ‖ = ‖ 𝑇^{*} 𝐯 ‖$ for all $𝐯 \in 𝑉$ , and deduce that $ker 𝑇 = ker 𝑇^{*}$ and $im 𝑇 = im 𝑇^{*}$ .
(c)Find an operator $𝑇$ on $ℝ^{2}$ with $ker 𝑇 \neq ker 𝑇^{*}$ .

Solution.

Uses. Proposition 6.6.23 ( $𝑇^{*} 𝑇$ and $𝑇$ ), Corollary 6.6.22 ( $𝑇$ and $𝑇^{*}$ have the same rank), Proposition 6.6.18 (Properties of the adjoint), Theorem 6.6.21 (Kernels and images of adjoints), Definition 6.6.8 (Adjoint)

Techniques. Move an operator across the inner product (Technique 6.6.20)

Hint 1. For (a), apply Proposition 6.6.23 to

𝑇

and to

𝑇^{*}

Hint 2. For (b),

{‖ 𝑇 𝐯 ‖}^{2} =

⟨ 𝑇^{*} 𝑇 𝐯, 𝐯 ⟩

and

{‖ 𝑇^{*} 𝐯 ‖}^{2} =

⟨ 𝑇 𝑇^{*} 𝐯, 𝐯 ⟩

. For the images, use Theorem 6.6.21(d) and (c).

Sketch. (a)

rank (𝑇^{*} 𝑇) =

rank 𝑇

, and applied to

𝑇^{*}

(whose adjoint is

𝑇

rank (𝑇 𝑇^{*}) =

rank 𝑇^{*} =

rank 𝑇

. (b)

{‖ 𝑇 𝐯 ‖}^{2} =

⟨ 𝑇^{*} 𝑇 𝐯, 𝐯 ⟩ =

⟨ 𝑇 𝑇^{*} 𝐯, 𝐯 ⟩ =

{‖ 𝑇^{*} 𝐯 ‖}^{2}

, so the kernels agree, and

im 𝑇 =

{(ker 𝑇^{*})}^{⟂} =

{(ker 𝑇)}^{⟂} =

im 𝑇^{*}

. (c)

(\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix})

(a) By Proposition 6.6.23(b), $rank (𝑇^{*} 𝑇) =$ $rank 𝑇$ . Applying the same result to $𝑇^{*}$ , whose adjoint is $𝑇$ (Proposition 6.6.18(c)), gives $rank (𝑇 𝑇^{*}) =$ $rank 𝑇^{*}$ , which is $rank 𝑇$ by Corollary 6.6.22.

(b) By the defining identity of $𝑇^{*}$ , and of ${(𝑇^{*})}^{*} = 𝑇$ ,

{‖ 𝑇 𝐯 ‖}^{2} = ⟨ 𝑇 𝐯, 𝑇 𝐯 ⟩ = ⟨ 𝑇^{*} 𝑇 𝐯, 𝐯 ⟩ = ⟨ 𝑇 𝑇^{*} 𝐯, 𝐯 ⟩ = ⟨ 𝑇^{*} 𝐯, 𝑇^{*} 𝐯 ⟩ = {‖ 𝑇^{*} 𝐯 ‖}^{2} .

So $𝑇 𝐯 = 𝟎$ if and only if $𝑇^{*} 𝐯 = 𝟎$ : $ker 𝑇 = ker 𝑇^{*}$ . By (d) and (c) of Theorem 6.6.21, $im 𝑇 =$ ${(ker 𝑇^{*})}^{⟂} =$ ${(ker 𝑇)}^{⟂} =$ $im 𝑇^{*}$ .

(c) For $𝑁 = (\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix})$ , $ker 𝑁 = span (𝐞_{1})$ , while $ker 𝑁^{𝖳} = span (𝐞_{2})$ . Operators with $𝑇^{*} 𝑇 = 𝑇 𝑇^{*}$ are called normal; they are the subject of Section 8.1.

Exercise 6.6.45.standard On $𝑉 = {ℝ [𝑥]}_{\leq 2}$ , let $(𝑆 𝑝) (𝑥) =$ $𝑝 (1 - 𝑥)$ .

(a)Show that $𝑆$ is self-adjoint for $⟨ 𝑝, 𝑞 ⟩ =$ $\int_{0}^{1} 𝑝 (𝑡) 𝑞 (𝑡) d 𝑡$ , but not for $⟨ 𝑝, 𝑞 ⟩ =$ $\int_{0}^{2} 𝑝 (𝑡) 𝑞 (𝑡) d 𝑡$ .
(b)For the first inner product, show that $(id + 𝑆) / 2$ is the orthogonal projection onto the subspace $𝐸$ of polynomials with $𝑝 (1 - 𝑥) = 𝑝 (𝑥)$ , and find a basis of $𝐸$ .

Solution.

Uses. Definition 6.6.26 (Self-adjoint operator), Corollary 6.6.29 (Orthogonal projections are the self-adjoint idempotents), Proposition 6.6.18 (Properties of the adjoint)

Hint 1. Substitute

𝑠 = 1 - 𝑡

\int_{0}^{1} 𝑝 (1 - 𝑡) 𝑞 (𝑡) d 𝑡

. For the second inner product, test

𝑝 = 𝑥

and

𝑞 = 1

Hint 2.

𝑆^{2} = id

. Check

𝑃^{2} = 𝑃 = 𝑃^{*}

for

𝑃 = (id + 𝑆) / 2

, and identify

im 𝑃

Sketch. (a) The substitution

𝑠 = 1 - 𝑡

maps

[0, 1]

to itself, giving

⟨ 𝑆 𝑝, 𝑞 ⟩ =

⟨ 𝑝, 𝑆 𝑞 ⟩

. On

[0, 2]

𝑝 = 𝑥

𝑞 = 1

give

⟨ 𝑆 𝑝, 𝑞 ⟩ = 0

but

⟨ 𝑝, 𝑆 𝑞 ⟩ = 2

. (b)

𝑆^{2} = id

, so

𝑃^{2} = 𝑃

, and

𝑃^{*} = 𝑃

im 𝑃 =

{𝑝 | 𝑆 𝑝 = 𝑝} =

𝐸

, with basis

1, 𝑥^{2} - 𝑥

(a) For the first inner product, substituting $𝑠 = 1 - 𝑡$ ,

⟨ 𝑆 𝑝, 𝑞 ⟩ = \int_{0}^{1} 𝑝 (1 - 𝑡) 𝑞 (𝑡) d 𝑡 = \int_{0}^{1} 𝑝 (𝑠) 𝑞 (1 - 𝑠) d 𝑠 = ⟨ 𝑝, 𝑆 𝑞 ⟩ .

For the second, take $𝑝 = 𝑥$ and $𝑞 = 1$ : $⟨ 𝑆 𝑝, 𝑞 ⟩ =$ $\int_{0}^{2} (1 - 𝑡) d 𝑡 =$ $2 -$ $2 =$ $0$ , while $⟨ 𝑝, 𝑆 𝑞 ⟩ =$ $\int_{0}^{2} 𝑡 d 𝑡 =$ $2$ . So $𝑆$ is not self-adjoint for $\int_{0}^{2}$ ; the substitution maps $[0, 2]$ to $[- 1, 1]$ , not to itself.

(b) $𝑆^{2} 𝑝 (𝑥) =$ $𝑝 (1 - (1 - 𝑥)) =$ $𝑝 (𝑥)$ , so $𝑆^{2} = id$ . Let $𝑃 = (id + 𝑆) / 2$ . Then $𝑃^{2} =$ $(id + 2 𝑆 + 𝑆^{2}) / 4 =$ $(id + 𝑆) / 2 =$ $𝑃$ , and $𝑃^{*} = (id + 𝑆^{*}) / 2 = 𝑃$ by (a) and Proposition 6.6.18. By Corollary 6.6.29, $𝑃 = {Pr}_{im 𝑃}$ . If $𝑆 𝑝 = 𝑝$ , then $𝑃 𝑝 = 𝑝$ , so $𝐸 \subseteq im 𝑃$ ; conversely $𝑆 𝑃 =$ $(𝑆 + 𝑆^{2}) / 2 =$ $𝑃$ , so $𝑆$ fixes every vector of $im 𝑃$ . Hence $im 𝑃 = 𝐸$ . For $𝑝 = 𝑎 + 𝑏 𝑥 + 𝑐 𝑥^{2}$ ,

𝑝 (1 - 𝑥) = (𝑎 + 𝑏 + 𝑐) - (𝑏 + 2 𝑐) 𝑥 + 𝑐 𝑥^{2},

which equals $𝑝 (𝑥)$ if and only if $𝑏 = - 𝑐$ . So $𝐸 =$ ${𝑎 + 𝑐 (𝑥^{2} - 𝑥)}$ , with basis $1, 𝑥^{2} - 𝑥$ , and $dim 𝐸 = 2$ .

Exercise 6.6.46.hard Let $𝑉$ be finite-dimensional with a basis $ℬ︀ = 𝐛_{1}, \dots, 𝐛_{𝑛}$ , not necessarily orthonormal, and let $𝐺$ be its Gram matrix, $𝑔_{𝑖 𝑗} =$ $⟨ 𝐛_{𝑗}, 𝐛_{𝑖} ⟩$ (Definition 6.3.28).

(a)Show that $⟨ 𝐯, 𝐰 ⟩ =$ ${[𝐰]}_{ℬ︀}^{*} 𝐺 {[𝐯]}_{ℬ︀}$ for all $𝐯, 𝐰 \in 𝑉$ .
(b)Show that ${[𝑇^{*}]}_{ℬ︀} =$ $𝐺^{- 1} {({[𝑇]}_{ℬ︀})}^{*} 𝐺$ for every $𝑇 \in L (𝑉)$ .
(c)Recompute ${[𝐷^{*}]}_{ℬ︀}$ of Example 6.6.16 with (b).
(d)A student claims that ${[𝑇^{*}]}_{ℬ︀} = {({[𝑇]}_{ℬ︀})}^{*}$ for every basis, “since $⟨ 𝐯, 𝐰 ⟩ =$ ${[𝐯]}_{ℬ︀} \cdot$ ${[𝐰]}_{ℬ︀}$ , so the second proof of Theorem 6.6.15 applies.” Which step fails, and for which bases is the claim true?

Solution.

Uses. Proposition 6.1.4 (Sesquilinearity), Definition 6.3.28 (Gram matrix), Proposition 6.3.30 (Gram matrices detect independence), Theorem 2.3.9 (The matrix of a map computes the map), Proposition 6.3.27 (Properties of the conjugate transpose), Theorem 6.3.22 (Parseval’s identity), Example 6.6.16 (The adjoint of differentiation)

Techniques. Translate between maps and matrices (Technique 2.3.27), Gram matrices (Technique 6.3.31)

Hint 1. For (a), expand

⟨ \sum_{𝑗} 𝑥_{𝑗} 𝐛_{𝑗}, \sum_{𝑖} 𝑦_{𝑖} 𝐛_{𝑖} ⟩

by sesquilinearity, as in Proposition 6.1.22(a).

Hint 2. For (b), with

𝐴 = {[𝑇]}_{ℬ︀}

and

𝐶 = {[𝑇^{*}]}_{ℬ︀}

, (a) turns the defining identity into

𝐲^{*} 𝐺 𝐴 𝐱 =

𝐲^{*} 𝐶^{*} 𝐺 𝐱

for all

𝐱, 𝐲 \in 𝔽^{𝑛}

. Conclude

𝐺 𝐴 = 𝐶^{*} 𝐺

; is

𝐺

invertible?

Sketch. (a)

⟨ 𝐯, 𝐰 ⟩ =

\sum_{𝑖, 𝑗} 𝑦_{𝑖} 𝑥_{𝑗} ⟨ 𝐛_{𝑗}, 𝐛_{𝑖} ⟩ =

𝐲^{*} 𝐺 𝐱

. (b) The defining identity reads

𝐲^{*} 𝐺 𝐴 𝐱 =

𝐲^{*} 𝐶^{*} 𝐺 𝐱

for all

𝐱, 𝐲

; taking standard basis vectors gives

𝐺 𝐴 = 𝐶^{*} 𝐺

𝐺

is invertible (Gram matrix of an independent list) and Hermitian, so

𝐶 = 𝐺^{- 1} 𝐴^{*} 𝐺

. (c)

𝐺^{- 1} (\begin{matrix} 0 & 0 \\ 1 & 0 \end{matrix}) 𝐺 =

(\begin{matrix} - 6 & - 3 \\ 12 & 6 \end{matrix})

. (d) The formula

⟨ 𝐯, 𝐰 ⟩ =

{[𝐯]}_{ℬ︀} \cdot

{[𝐰]}_{ℬ︀}

is Parseval’s identity, valid only for orthonormal

ℬ︀

; the claim holds whenever

𝐺

is a scalar multiple of

𝐼_{𝑛}

(a) Write $𝐱 = {[𝐯]}_{ℬ︀}$ and $𝐲 = {[𝐰]}_{ℬ︀}$ . By linearity in the first slot and Proposition 6.1.4,

⟨ 𝐯, 𝐰 ⟩ = ⟨ \sum_{𝑗} 𝑥_{𝑗} 𝐛_{𝑗}, \sum_{𝑖} 𝑦_{𝑖} 𝐛_{𝑖} ⟩ = \sum_{𝑖, 𝑗} 𝑦_{𝑖} ⟨ 𝐛_{𝑗}, 𝐛_{𝑖} ⟩ 𝑥_{𝑗} = \sum_{𝑖} 𝑦_{𝑖} (\sum_{𝑗} 𝑔_{𝑖 𝑗} 𝑥_{𝑗}) = 𝐲^{*} 𝐺 𝐱 .

(b) Let $𝐴 = {[𝑇]}_{ℬ︀}$ and $𝐶 = {[𝑇^{*}]}_{ℬ︀}$ . For $𝐯, 𝐰 \in 𝑉$ with coordinates $𝐱, 𝐲$ , Theorem 2.3.9 gives ${[𝑇 𝐯]}_{ℬ︀} = 𝐴 𝐱$ and ${[𝑇^{*} 𝐰]}_{ℬ︀} = 𝐶 𝐲$ , so by (a) and Proposition 6.3.27,

⟨ 𝑇 𝐯, 𝐰 ⟩ = 𝐲^{*} 𝐺 𝐴 𝐱, ⟨ 𝐯, 𝑇^{*} 𝐰 ⟩ = {(𝐶 𝐲)}^{*} 𝐺 𝐱 = 𝐲^{*} 𝐶^{*} 𝐺 𝐱 .

These agree for all $𝐱, 𝐲 \in 𝔽^{𝑛}$ , since coordinates range over all of $𝔽^{𝑛}$ . Taking $𝐱 = 𝐞_{𝑗}$ and $𝐲 = 𝐞_{𝑖}$ shows that $𝐺 𝐴$ and $𝐶^{*} 𝐺$ have the same $(𝑖, 𝑗)$ entry, so $𝐺 𝐴 = 𝐶^{*} 𝐺$ . The Gram matrix of the independent list $ℬ︀$ is invertible (Proposition 6.3.30), and $𝐺^{*} = 𝐺$ , so $𝐶^{*} = 𝐺 𝐴 𝐺^{- 1}$ and

𝐶 = {(𝐺 𝐴 𝐺^{- 1})}^{*} = {(𝐺^{- 1})}^{*} 𝐴^{*} 𝐺^{*} = 𝐺^{- 1} 𝐴^{*} 𝐺,

using Proposition 6.3.27 and ${(𝐺^{- 1})}^{*} =$ ${(𝐺^{*})}^{- 1} =$ $𝐺^{- 1}$ .

(c) For $ℬ︀ = 1, 𝑥$ with $⟨ 𝑝, 𝑞 ⟩ = \int_{0}^{1} 𝑝 𝑞$ , $𝐺 = (\begin{matrix} 1 & 1 / 2 \\ 1 / 2 & 1 / 3 \end{matrix})$ , whose determinant is $1 / 12$ , so $𝐺^{- 1} = (\begin{matrix} 4 & - 6 \\ - 6 & 12 \end{matrix})$ . With ${[𝐷]}_{ℬ︀}^{*} = (\begin{matrix} 0 & 0 \\ 1 & 0 \end{matrix})$ ,

𝐺^{- 1} (\begin{matrix} 0 & 0 \\ 1 & 0 \end{matrix}) 𝐺 = (\begin{matrix} - 6 & 0 \\ 12 & 0 \end{matrix}) (\begin{matrix} 1 & 1 / 2 \\ 1 / 2 & 1 / 3 \end{matrix}) = (\begin{matrix} - 6 & - 3 \\ 12 & 6 \end{matrix}),

in agreement with Example 6.6.16.

(d) The identity $⟨ 𝐯, 𝐰 ⟩ =$ ${[𝐯]}_{ℬ︀} \cdot$ ${[𝐰]}_{ℬ︀}$ is Parseval’s identity (Theorem 6.3.22), which needs $ℬ︀$ orthonormal. By (a) it holds exactly when $𝐺 = 𝐼_{𝑛}$ . If $𝐺 = 𝑐 𝐼_{𝑛}$ for some $𝑐 > 0$ , that is, the basis vectors are orthogonal and of equal length, then (b) gives ${[𝑇^{*}]}_{ℬ︀} = 𝐴^{*}$ as well. For other bases the claim can fail, as in (c).

Exercise 6.6.47.hard Let $𝑉$ be finite-dimensional, and suppose $𝑉 = 𝑈 \oplus 𝑋$ for subspaces $𝑈$ and $𝑋$ (Definition 4.4.4). Let $𝑃 \in L (𝑉)$ be the projection onto $𝑈$ along $𝑋$ : $𝑃 (𝐮 + 𝐱) = 𝐮$ for $𝐮 \in 𝑈$ and $𝐱 \in 𝑋$ .

(a)Show that $𝑃^{2} = 𝑃$ , $im 𝑃 = 𝑈$ , and $ker 𝑃 = 𝑋$ .
(b)Show that $𝑉 = 𝑋^{⟂} \oplus 𝑈^{⟂}$ and that $𝑃^{*}$ is the projection onto $𝑋^{⟂}$ along $𝑈^{⟂}$ .
(c)Deduce that $𝑃$ is an orthogonal projection if and only if $𝑋 = 𝑈^{⟂}$ .

Solution.

Uses. Definition 4.4.4 (Direct sum), Proposition 6.6.18 (Properties of the adjoint), Theorem 6.6.21 (Kernels and images of adjoints), Theorem 6.4.26 (The double complement), Corollary 6.6.29 (Orthogonal projections are the self-adjoint idempotents)

Hint 1. Any

𝑄 \in L (𝑉)

with

𝑄^{2} = 𝑄

satisfies

𝑉 = im 𝑄 \oplus ker 𝑄

and is the projection onto

im 𝑄

along

ker 𝑄

: split

𝐯 = 𝑄 𝐯 + (𝐯 - 𝑄 𝐯)

Hint 2. Apply this to

𝑄 = 𝑃^{*}

, using

{(𝑃^{*})}^{2} = {(𝑃^{2})}^{*}

and Theorem 6.6.21 for

im 𝑃^{*}

and

ker 𝑃^{*}

Sketch. (a) is immediate from uniqueness of the decomposition. (b) An idempotent

𝑄

is the projection onto

im 𝑄

along

ker 𝑄

, and

𝑉 = im 𝑄 \oplus ker 𝑄

. Now

{(𝑃^{*})}^{2} = {(𝑃^{2})}^{*} = 𝑃^{*}

im 𝑃^{*} =

{(ker 𝑃)}^{⟂} =

𝑋^{⟂}

, and

ker 𝑃^{*} =

{(im 𝑃)}^{⟂} =

𝑈^{⟂}

. (c)

𝑃

is orthogonal iff

𝑃 = 𝑃^{*}

(Corollary 6.6.29), iff they have the same image and kernel, iff

𝑈 = 𝑋^{⟂}

, iff

𝑋 = 𝑈^{⟂}

(a) For $𝐮 \in 𝑈$ , the decomposition of $𝐮$ is $𝐮 + 𝟎$ , so $𝑃 𝐮 = 𝐮$ . Hence $𝑃 (𝑃 𝐯) = 𝑃 𝐯$ for every $𝐯$ , since $𝑃 𝐯 \in 𝑈$ , and $im 𝑃 = 𝑈$ . Finally $𝑃 (𝐮 + 𝐱) = 𝟎$ exactly when $𝐮 = 𝟎$ , so $ker 𝑃 = 𝑋$ .

(b) First a general fact. Let $𝑄 \in L (𝑉)$ with $𝑄^{2} = 𝑄$ . Every $𝐯$ is $𝑄 𝐯 + (𝐯 - 𝑄 𝐯)$ , with $𝑄 𝐯 \in im 𝑄$ and $𝑄 (𝐯 - 𝑄 𝐯) =$ $𝑄 𝐯 -$ $𝑄^{2} 𝐯 =$ $𝟎$ . If $𝐲 \in im 𝑄 \cap ker 𝑄$ , say $𝐲 = 𝑄 𝐳$ , then $𝟎 =$ $𝑄 𝐲 =$ $𝑄^{2} 𝐳 =$ $𝑄 𝐳 =$ $𝐲$ . So $𝑉 = im 𝑄 \oplus ker 𝑄$ , and $𝑄$ sends $𝐯 = 𝑄 𝐯 + (𝐯 - 𝑄 𝐯)$ to its component $𝑄 𝐯$ in $im 𝑄$ : $𝑄$ is the projection onto $im 𝑄$ along $ker 𝑄$ .

Apply this to $𝑄 = 𝑃^{*}$ . By Proposition 6.6.18(d), ${(𝑃^{*})}^{2} = {(𝑃^{2})}^{*} = 𝑃^{*}$ . By Theorem 6.6.21 and (a), $im 𝑃^{*} =$ ${(ker 𝑃)}^{⟂} =$ $𝑋^{⟂}$ and $ker 𝑃^{*} =$ ${(im 𝑃)}^{⟂} =$ $𝑈^{⟂}$ . So $𝑉 = 𝑋^{⟂} \oplus 𝑈^{⟂}$ , and $𝑃^{*}$ is the projection onto $𝑋^{⟂}$ along $𝑈^{⟂}$ .

(c) Since $𝑃^{2} = 𝑃$ , Corollary 6.6.29 says that $𝑃$ is an orthogonal projection if and only if $𝑃 = 𝑃^{*}$ . A projection along a complement is determined by its image and kernel (by the general fact in (b)), so $𝑃 = 𝑃^{*}$ if and only if $𝑈 = 𝑋^{⟂}$ and $𝑋 = 𝑈^{⟂}$ . These two conditions are equivalent to each other by Theorem 6.4.26: if $𝑋 = 𝑈^{⟂}$ , then $𝑋^{⟂} = {(𝑈^{⟂})}^{⟂} = 𝑈$ , and conversely. So $𝑃$ is an orthogonal projection if and only if $𝑋 = 𝑈^{⟂}$ .

Exercise 6.6.48.hard

(a)Let $𝐴 = (\begin{matrix} 1 & 1 \\ 0 & 2 \end{matrix})$ . Find an inner product on $ℝ^{2}$ for which $𝑇_{𝐴}$ is self-adjoint. (Look for $⟨ 𝐱, 𝐲 ⟩ = 𝐲^{𝖳} 𝑀 𝐱$ as in Proposition 6.1.22.)
(b)Show that the rotation $𝑇_{𝐽}$ , $𝐽 = (\begin{matrix} 0 & - 1 \\ 1 & 0 \end{matrix})$ , is not self-adjoint for any inner product on $ℝ^{2}$ .

Solution.

Uses. Proposition 6.1.22 (Inner products on $𝔽^{𝑛}$ ), Definition 6.6.26 (Self-adjoint operator)

Techniques. Undetermined coefficients (Technique 3.1.9)

Hint 1. For

⟨ 𝐱, 𝐲 ⟩ = 𝐲^{𝖳} 𝑀 𝐱

, show that

𝑇_{𝐴}

is self-adjoint if and only if

𝑀 𝐴

is symmetric.

Hint 2. Write

𝑀 = (\begin{matrix} 𝑎 & 𝑏 \\ 𝑏 & 𝑐 \end{matrix})

. For (a), solve for the symmetry condition and then make

𝑀

positive definite. For (b), the symmetry condition forces

𝑐 = - 𝑎

; why is that impossible?

Sketch.

⟨ 𝐴 𝐱, 𝐲 ⟩ =

𝐲^{𝖳} 𝑀 𝐴 𝐱

and

⟨ 𝐱, 𝐴 𝐲 ⟩ =

𝐲^{𝖳} 𝐴^{𝖳} 𝑀 𝐱

, so self-adjointness means

𝑀 𝐴 =

𝐴^{𝖳} 𝑀 =

{(𝑀 𝐴)}^{𝖳}

. (a)

𝑀 = (\begin{matrix} 𝑎 & 𝑏 \\ 𝑏 & 𝑐 \end{matrix})

gives

𝑀 𝐴 =

(\begin{matrix} 𝑎 & 𝑎 + 2 𝑏 \\ 𝑏 & 𝑏 + 2 𝑐 \end{matrix})

, symmetric iff

𝑏 = - 𝑎

; take

𝑀 = (\begin{matrix} 1 & - 1 \\ - 1 & 2 \end{matrix})

. (b)

𝑀 𝐽 = (\begin{matrix} 𝑏 & - 𝑎 \\ 𝑐 & - 𝑏 \end{matrix})

is symmetric iff

𝑐 = - 𝑎

, but

𝑎 = 𝐞_{1}^{𝖳} 𝑀 𝐞_{1} > 0

and

𝑐 = 𝐞_{2}^{𝖳} 𝑀 𝐞_{2} > 0

By Proposition 6.1.22, the inner products on $ℝ^{2}$ are exactly $⟨ 𝐱, 𝐲 ⟩ = 𝐲^{𝖳} 𝑀 𝐱$ with $𝑀 = (\begin{matrix} 𝑎 & 𝑏 \\ 𝑏 & 𝑐 \end{matrix})$ symmetric and $𝐱^{𝖳} 𝑀 𝐱 > 0$ for $𝐱 \neq 𝟎$ . For such an inner product and any $𝐴 \in ℝ^{2 \times 2}$ ,

⟨ 𝐴 𝐱, 𝐲 ⟩ = 𝐲^{𝖳} (𝑀 𝐴) 𝐱, ⟨ 𝐱, 𝐴 𝐲 ⟩ = {(𝐴 𝐲)}^{𝖳} 𝑀 𝐱 = 𝐲^{𝖳} (𝐴^{𝖳} 𝑀) 𝐱 .

These agree for all $𝐱, 𝐲$ if and only if $𝑀 𝐴 = 𝐴^{𝖳} 𝑀$ (test on standard basis vectors), that is, $𝑀 𝐴 = {(𝑀 𝐴)}^{𝖳}$ , since ${(𝑀 𝐴)}^{𝖳} =$ $𝐴^{𝖳} 𝑀^{𝖳} =$ $𝐴^{𝖳} 𝑀$ .

(a) $𝑀 𝐴 =$ $(\begin{matrix} 𝑎 & 𝑎 + 2 𝑏 \\ 𝑏 & 𝑏 + 2 𝑐 \end{matrix})$ is symmetric if and only if $𝑎 + 2 𝑏 = 𝑏$ , that is, $𝑏 = - 𝑎$ . Take $𝑎 = 1$ , $𝑏 = - 1$ , $𝑐 = 2$ . Then $𝑀 = (\begin{matrix} 1 & - 1 \\ - 1 & 2 \end{matrix})$ is symmetric and $𝐱^{𝖳} 𝑀 𝐱 =$ $𝑥_{1}^{2} -$ $2 𝑥_{1} 𝑥_{2} +$ $2 𝑥_{2}^{2} =$ ${(𝑥_{1} - 𝑥_{2})}^{2} +$ $𝑥_{2}^{2} >$ $0$ for $𝐱 \neq 𝟎$ . So $𝑇_{𝐴}$ is self-adjoint for

⟨ 𝐱, 𝐲 ⟩ = 𝑥_{1} 𝑦_{1} - 𝑥_{1} 𝑦_{2} - 𝑥_{2} 𝑦_{1} + 2 𝑥_{2} 𝑦_{2} .

For instance, $𝐴 𝐞_{1} = 𝐞_{1}$ and $𝐴 (1, 1) = 2 (1, 1)$ , and these two vectors are orthogonal for this inner product: $⟨ 𝐞_{1}, (1, 1) ⟩ = 1 - 1 = 0$ . That is what Section 8.2 leads one to expect.

(b) $𝑀 𝐽 = (\begin{matrix} 𝑏 & - 𝑎 \\ 𝑐 & - 𝑏 \end{matrix})$ is symmetric if and only if $𝑐 = - 𝑎$ . But $𝑎 = 𝐞_{1}^{𝖳} 𝑀 𝐞_{1} > 0$ and $𝑐 = 𝐞_{2}^{𝖳} 𝑀 𝐞_{2} > 0$ , so $𝑐 \neq - 𝑎$ . Hence $𝑇_{𝐽}$ is self-adjoint for no inner product on $ℝ^{2}$ . (By Section 8.2, a self-adjoint operator on $ℝ^{2}$ maps some line into itself, and a rotation by $𝜋 / 2$ maps no line into itself.)

Exercise 6.6.49.hardBeyond 21-242 Let $𝑃_{𝑛} = {ℝ [𝑥]}_{\leq 𝑛}$ with $⟨ 𝑝, 𝑞 ⟩ =$ $\int_{0}^{1} 𝑝 (𝑡) 𝑞 (𝑡) d 𝑡$ , and let $𝑞_{𝑛} \in 𝑃_{𝑛}$ represent evaluation at $0$ : $⟨ 𝑝, 𝑞_{𝑛} ⟩ = 𝑝 (0)$ for all $𝑝 \in 𝑃_{𝑛}$ (Theorem 6.6.3).

(a)Show that $‖ 𝑞_{𝑛} ‖ \geq$ $| 𝑝 (0) | / ‖ 𝑝 ‖$ for every nonzero $𝑝 \in 𝑃_{𝑛}$ .
(b)Using $𝑝 (𝑡) = {(1 - 𝑡)}^{𝑛}$ , deduce that $‖ 𝑞_{𝑛} ‖ \geq \sqrt{2 𝑛 + 1}$ .
(c)Deduce again (Non-example 6.6.7) that no polynomial $𝑞$ satisfies $\int_{0}^{1} 𝑝 (𝑡) 𝑞 (𝑡) d 𝑡 =$ $𝑝 (0)$ for all polynomials $𝑝$ .

Solution.

Uses. Theorem 6.6.3 (Riesz representation theorem), Theorem 6.2.1 (Cauchy–Schwarz inequality)

Techniques. Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1. For (a), apply Cauchy–Schwarz to

𝑝 (0) = ⟨ 𝑝, 𝑞_{𝑛} ⟩

Hint 2. For (c), if

𝑞

worked for all polynomials, compare

𝑞

with

𝑞_{𝑛}

for

𝑛 \geq deg 𝑞

Sketch. (a)

| 𝑝 (0) | =

| ⟨ 𝑝, 𝑞_{𝑛} ⟩ | \leq

‖ 𝑝 ‖ ‖ 𝑞_{𝑛} ‖

. (b)

𝑝 (0) = 1

and

{‖ 𝑝 ‖}^{2} =

\int_{0}^{1} {(1 - 𝑡)}^{2 𝑛} d 𝑡 =

1 / (2 𝑛 + 1)

. (c) Such a

𝑞

would lie in

𝑃_{𝑛}

for

𝑛 \geq deg 𝑞

and represent evaluation there, so

𝑞 = 𝑞_{𝑛}

by uniqueness, and then

‖ 𝑞 ‖ \geq \sqrt{2 𝑛 + 1}

for all large

𝑛

(a) By Cauchy–Schwarz, $| 𝑝 (0) | =$ $| ⟨ 𝑝, 𝑞_{𝑛} ⟩ | \leq$ $‖ 𝑝 ‖ ‖ 𝑞_{𝑛} ‖$ . Divide by $‖ 𝑝 ‖ > 0$ .

(b) For $𝑝 (𝑡) = {(1 - 𝑡)}^{𝑛}$ , $𝑝 (0) = 1$ and ${‖ 𝑝 ‖}^{2} =$ $\int_{0}^{1} {(1 - 𝑡)}^{2 𝑛} d 𝑡 =$ $1 / (2 𝑛 + 1)$ . By (a), $‖ 𝑞_{𝑛} ‖ \geq$ $1 / ‖ 𝑝 ‖ =$ $\sqrt{2 𝑛 + 1}$ . For $𝑛 = 1$ this gives $‖ 𝑞_{1} ‖ \geq \sqrt{3}$ ; in fact $‖ 𝑞_{1} ‖ = 2$ (Example 6.6.5).

(c) Suppose $𝑞$ is a polynomial with $⟨ 𝑝, 𝑞 ⟩ = 𝑝 (0)$ for all polynomials $𝑝$ , and let $𝑛 \geq deg 𝑞$ . Then $𝑞 \in 𝑃_{𝑛}$ and $⟨ 𝑝, 𝑞 ⟩ = 𝑝 (0)$ for all $𝑝 \in 𝑃_{𝑛}$ , so $𝑞 = 𝑞_{𝑛}$ by the uniqueness in Theorem 6.6.3. By (b), $‖ 𝑞 ‖ = ‖ 𝑞_{𝑛} ‖ \geq \sqrt{2 𝑛 + 1}$ for every $𝑛 \geq deg 𝑞$ , which is impossible for a fixed $𝑞$ .

Exercise 6.6.50.hard Let

𝐴 \in ℂ^{𝑛 \times 𝑛}

satisfy

𝐴^{*} 𝐴 = 𝐴^{2}

. Show that

𝐴^{*} = 𝐴

Solution.

Uses. Proposition 6.1.16 (Frobenius inner product), Corollary 6.1.19 ( $𝐴^{*} 𝐴 = 0$ forces $𝐴 = 0$ ), Proposition 2.4.28 (Cyclicity of the trace), Proposition 6.3.27 (Properties of the conjugate transpose)

Techniques. The trace of $𝐴^{*} 𝐴$ (Technique 6.1.18)

Hint 1. It suffices to show that

𝐵 = 𝐴 - 𝐴^{*}

satisfies

tr (𝐵^{*} 𝐵) = 0

Hint 2. Expand

tr ((𝐴^{*} - 𝐴) (𝐴 - 𝐴^{*}))

into four traces, and express each through

tr (𝐴^{*} 𝐴)

using

tr (𝑋 𝑌) = tr (𝑌 𝑋)

tr (𝑋^{*}) = tr 𝑋

, and the hypothesis.

Sketch. With

𝐵 = 𝐴 - 𝐴^{*}

tr (𝐵^{*} 𝐵) =

tr (𝐴^{*} 𝐴) -

tr ({(𝐴^{*})}^{2}) -

tr (𝐴^{2}) +

tr (𝐴 𝐴^{*})

. Here

tr (𝐴^{2}) = tr (𝐴^{*} 𝐴)

, which is real, so

tr ({(𝐴^{*})}^{2}) =

tr (𝐴^{2}) =

tr (𝐴^{*} 𝐴)

, and

tr (𝐴 𝐴^{*}) =

tr (𝐴^{*} 𝐴)

. So

tr (𝐵^{*} 𝐵) = 0

, and

𝐵 = 0

Let $𝐵 = 𝐴 - 𝐴^{*}$ , so that $𝐵^{*} = 𝐴^{*} - 𝐴$ (Proposition 6.3.27(a)). Then

tr (𝐵^{*} 𝐵) = tr (𝐴^{*} 𝐴) - tr ({(𝐴^{*})}^{2}) - tr (𝐴^{2}) + tr (𝐴 𝐴^{*}) .

We express each term through $𝑡 =$ $tr (𝐴^{*} 𝐴) =$ $\sum_{𝑖, 𝑗} {| 𝑎_{𝑖 𝑗} |}^{2}$ , a real number (Proposition 6.1.16). By hypothesis $tr (𝐴^{2}) = 𝑡$ . The diagonal entries of $𝑋^{*}$ are the conjugates of those of $𝑋$ , so $tr (𝑋^{*}) = tr 𝑋$ , and with Proposition 6.3.27(b), $tr ({(𝐴^{*})}^{2}) =$ $tr ({(𝐴^{2})}^{*}) =$ $tr (𝐴^{2}) =$ $𝑡 =$ $𝑡$ . By Proposition 2.4.28, $tr (𝐴 𝐴^{*}) =$ $tr (𝐴^{*} 𝐴) =$ $𝑡$ . Hence $tr (𝐵^{*} 𝐵) =$ $𝑡 -$ $𝑡 -$ $𝑡 +$ $𝑡 =$ $0$ . By Proposition 6.1.16, $tr (𝐵^{*} 𝐵) =$ $\sum_{𝑖, 𝑗} {| 𝑏_{𝑖 𝑗} |}^{2}$ , so every entry of $𝐵$ is $0$ (as in Corollary 6.1.19). That is, $𝐴^{*} = 𝐴$ .

Exercise 6.6.51.challengeBeyond 21-242 Let $𝑉$ be a finite-dimensional inner product space and $𝑇 \in L (𝑉)$ with $‖ 𝑇 𝐯 ‖ \leq ‖ 𝐯 ‖$ for all $𝐯 \in 𝑉$ .

(a)Show that $𝑇 𝐯 = 𝐯$ if and only if $𝑇^{*} 𝐯 = 𝐯$ .
(b)Deduce that $𝑉 =$ $ker (𝑇 - id) \oplus$ $im (𝑇 - id)$ , with the two subspaces orthogonal.
(c)Show that for every $𝐯 \in 𝑉$ , the averages $𝐴_{𝑁} 𝐯 =$ $(1 / 𝑁) \sum_{𝑘 = 0}^{𝑁 - 1} 𝑇^{𝑘} 𝐯$ converge to ${Pr}_{ker (𝑇 - id)} (𝐯)$ as $𝑁 \to \infty$ , in the sense that $‖ 𝐴_{𝑁} 𝐯 -$ ${Pr}_{ker (𝑇 - id)} (𝐯) ‖ \to$ $0$ .

Solution.

Uses. Exercise 6.6.42, Proposition 6.1.29 (Expanding a norm), Proposition 6.6.18 (Properties of the adjoint), Theorem 6.6.21 (Kernels and images of adjoints), Theorem 6.4.24 (Orthogonal decomposition), Theorem 6.2.11 (Triangle inequality)

Techniques. Show a vector is zero via its norm (Technique 6.1.36), Move an operator across the inner product (Technique 6.6.20)

Hint 1. By Exercise 6.6.42(b),

‖ 𝑇^{*} 𝐰 ‖ \leq ‖ 𝐰 ‖

as well. If

𝑇 𝐯 = 𝐯

, expand

{‖ 𝑇^{*} 𝐯 - 𝐯 ‖}^{2}

and compute

⟨ 𝑇^{*} 𝐯, 𝐯 ⟩

by moving

𝑇^{*}

across.

Hint 2. For (b), (a) says

ker (𝑇 - id) =

ker (𝑇^{*} - id) =

ker {(𝑇 - id)}^{*}

. Apply Theorem 6.6.21 to

𝑇 - id

Hint 3. For (c), write

𝐯 = 𝐮 + (𝑇 - id) 𝐳

by (b). The averages fix

𝐮

, and the sum over the second part telescopes.

Sketch. (a) If

𝑇 𝐯 = 𝐯

, then

⟨ 𝑇^{*} 𝐯, 𝐯 ⟩ =

⟨ 𝐯, 𝑇 𝐯 ⟩ =

{‖ 𝐯 ‖}^{2}

, so

{‖ 𝑇^{*} 𝐯 - 𝐯 ‖}^{2} =

{‖ 𝑇^{*} 𝐯 ‖}^{2} -

{‖ 𝐯 ‖}^{2} \leq

0

. The converse is the same for

𝑇^{*}

. (b)

ker (𝑇 - id) =

{ker (𝑇 - id)}^{*} =

{(im (𝑇 - id))}^{⟂}

. (c) For

𝐯 = 𝐮 + (𝑇 - id) 𝐳

with

𝑇 𝐮 = 𝐮

𝐴_{𝑁} 𝐮 = 𝐮

, and

𝐴_{𝑁} (𝑇 - id) 𝐳 =

(𝑇^{𝑁} 𝐳 - 𝐳) / 𝑁

has norm at most

2 ‖ 𝐳 ‖ / 𝑁

(a) By Exercise 6.6.42(b) with $𝑐 = 1$ , $‖ 𝑇^{*} 𝐰 ‖ \leq ‖ 𝐰 ‖$ for all $𝐰$ . Suppose $𝑇 𝐯 = 𝐯$ . Moving $𝑇^{*}$ across, $⟨ 𝑇^{*} 𝐯, 𝐯 ⟩ =$ $⟨ 𝐯, 𝑇^{*} 𝐯 ⟩ =$ $⟨ 𝑇 𝐯, 𝐯 ⟩ =$ $⟨ 𝐯, 𝐯 ⟩ =$ ${‖ 𝐯 ‖}^{2}$ , which is real. By Proposition 6.1.29,

{‖ 𝑇^{*} 𝐯 - 𝐯 ‖}^{2} = {‖ 𝑇^{*} 𝐯 ‖}^{2} - 2 Re ⟨ 𝑇^{*} 𝐯, 𝐯 ⟩ + {‖ 𝐯 ‖}^{2} = {‖ 𝑇^{*} 𝐯 ‖}^{2} - {‖ 𝐯 ‖}^{2} \leq 0,

so $𝑇^{*} 𝐯 = 𝐯$ . Conversely, $𝑇^{*}$ also satisfies $‖ 𝑇^{*} 𝐰 ‖ \leq ‖ 𝐰 ‖$ , and its adjoint is $𝑇$ (Proposition 6.6.18(c)), so the same argument applied to $𝑇^{*}$ shows that $𝑇^{*} 𝐯 = 𝐯$ implies $𝑇 𝐯 = 𝐯$ .

(b) By Proposition 6.6.18, ${(𝑇 - id)}^{*} = 𝑇^{*} - id$ , so (a) says $ker (𝑇 - id) =$ $ker {(𝑇 - id)}^{*}$ . By Theorem 6.6.21(a) applied to $𝑇 - id$ , this is ${(im (𝑇 - id))}^{⟂}$ . Since $im (𝑇 - id)$ is finite-dimensional, Theorem 6.4.24 gives $𝑉 =$ $im (𝑇 - id) \oplus$ ${(im (𝑇 - id))}^{⟂} =$ $im (𝑇 - id) \oplus$ $ker (𝑇 - id)$ , an orthogonal decomposition.

(c) Let $𝐯 \in 𝑉$ . By (b), $𝐯 = 𝐮 + (𝑇 - id) 𝐳$ with $𝐮 \in ker (𝑇 - id)$ and $𝐳 \in 𝑉$ , and $𝐮 =$ ${Pr}_{ker (𝑇 - id)} (𝐯)$ because the second piece lies in $im (𝑇 - id) =$ ${(ker (𝑇 - id))}^{⟂}$ . Since $𝑇 𝐮 = 𝐮$ , $𝐴_{𝑁} 𝐮 = 𝐮$ . The other piece telescopes:

𝐴_{𝑁} (𝑇 - id) 𝐳 = \frac{1}{𝑁} \sum_{𝑘 = 0}^{𝑁 - 1} (𝑇^{𝑘 + 1} 𝐳 - 𝑇^{𝑘} 𝐳) = \frac{1}{𝑁} (𝑇^{𝑁} 𝐳 - 𝐳) .

By the triangle inequality and $‖ 𝑇^{𝑁} 𝐳 ‖ \leq ‖ 𝐳 ‖$ , this has norm at most $2 ‖ 𝐳 ‖ / 𝑁$ , which tends to $0$ . Hence $‖ 𝐴_{𝑁} 𝐯 -$ $𝐮 ‖ \leq$ $2 ‖ 𝐳 ‖ / 𝑁 \to$ $0$ . This is the finite-dimensional case of von Neumann’s mean ergodic theorem.

More practice

The practice bank has 29 exercises on this section’s topics.