6.2 Cauchy–Schwarz and the triangle inequality

In Section 6.1.1 we found that the dot product on $ℝ^{2}$ records angles, $𝐯 \cdot$ $𝐰 =$ $‖ 𝐯 ‖ ‖ 𝐰 ‖ cos 𝜃$ , and proposed to turn this around: in any real inner product space, define the angle $𝜃$ between nonzero vectors $𝐱$ and $𝐲$ by

cos 𝜃 = \frac{⟨ 𝐱, 𝐲 ⟩}{‖ 𝐱 ‖ ‖ 𝐲 ‖} .

This defines an angle only if the right side lies in $[- 1, 1]$ . In $ℝ^{2}$ that is a property of the cosine, but for functions or matrices it is a claim about the inner product, and it needs a proof. The claim is the Cauchy–Schwarz inequality. It also gives the triangle inequality, which is what makes $‖ 𝐱 - 𝐲 ‖$ behave like a distance, and applied to the inner products of Section 6.1.3 it produces inequalities between sums and between integrals that have no visible connection to geometry.

Throughout, $𝑉$ is an inner product space over $𝔽$ , where $𝔽$ is $ℝ$ or $ℂ$ (Notation 6.1.1).

6.2.1The Cauchy–Schwarz inequality

Intuition (The shadow is shorter than the vector). Take $𝐱, 𝐲 \in ℝ^{2}$ with $𝐲 \neq 𝟎$ , and drop a perpendicular from the tip of $𝐱$ to the line through $𝐲$ (Figure 6.2.1). Its foot is a multiple $𝑐 𝐲$ , the shadow of $𝐱$ on the line, and the rest of $𝐱$ , namely $𝐰 = 𝐱 - 𝑐 𝐲$ , is perpendicular to $𝐲$ . By (6.1.1), the shadow has length

‖ 𝑐 𝐲 ‖ = ‖ 𝐱 ‖ | cos 𝜃 | = \frac{| 𝐱 \cdot 𝐲 |}{‖ 𝐲 ‖} .

The vectors $𝑐 𝐲$ , $𝐰$ , and $𝐱$ form a right triangle with hypotenuse $𝐱$ , and a leg is never longer than the hypotenuse. So

\frac{| 𝐱 \cdot 𝐲 |}{‖ 𝐲 ‖} \leq ‖ 𝐱 ‖ .

Multiplying by $‖ 𝐲 ‖$ gives $| 𝐱 \cdot 𝐲 | \leq ‖ 𝐱 ‖ ‖ 𝐲 ‖$ , with equality exactly when the other leg $𝐰$ is $𝟎$ , that is, when $𝐱$ lies on the line through $𝐲$ .

In $ℝ^{2}$ this is just $| cos 𝜃 | \leq 1$ . But the argument never used cosines. It used the decomposition $𝐱 = 𝑐 𝐲 + 𝐰$ and Pythagoras, and both make sense in every inner product space. All we need is the right $𝑐$ . The requirement that $𝐰$ be perpendicular to $𝐲$ is $⟨ 𝐱 - 𝑐 𝐲, 𝐲 ⟩ = 0$ , which by linearity in the first slot reads $⟨ 𝐱, 𝐲 ⟩ = 𝑐 {‖ 𝐲 ‖}^{2}$ . So

𝑐 = \frac{⟨ 𝐱, 𝐲 ⟩}{{‖ 𝐲 ‖}^{2}} .

Figure 6.2.1. The shadow of

𝐱

on the line through

𝐲

𝑐 𝐲

, where

𝑐 = ⟨ 𝐱, 𝐲 ⟩ / {‖ 𝐲 ‖}^{2}

, and

𝐰 = 𝐱 - 𝑐 𝐲

is perpendicular to

𝐲

. By Pythagoras,

{‖ 𝐱 ‖}^{2} =

{‖ 𝑐 𝐲 ‖}^{2} +

{‖ 𝐰 ‖}^{2}

, so the shadow is no longer than

𝐱

Theorem 6.2.1 (Cauchy–Schwarz inequality). Let $𝑉$ be an inner product space and $𝐱, 𝐲 \in 𝑉$ .

(a) $| ⟨ 𝐱, 𝐲 ⟩ | \leq$ $‖ 𝐱 ‖ ‖ 𝐲 ‖$ .
(b)Equality holds in (a) if and only if one of $𝐱, 𝐲$ is a scalar multiple of the other.

Both sides of (a) are nonnegative, so (a) is equivalent to its square, ${| ⟨ 𝐱, 𝐲 ⟩ |}^{2} \leq$ $⟨ 𝐱, 𝐱 ⟩ ⟨ 𝐲, 𝐲 ⟩$ , which avoids square roots. For the dot product on $𝔽^{𝑛}$ it says that

{| \sum_{𝑘 = 1}^{𝑛} 𝑥_{𝑘} 𝑦_{𝑘} |}^{2} \leq (\sum_{𝑘 = 1}^{𝑛} {| 𝑥_{𝑘} |}^{2}) (\sum_{𝑘 = 1}^{𝑛} {| 𝑦_{𝑘} |}^{2})

for all complex numbers $𝑥_{1},$ $\dots,$ $𝑥_{𝑛},$ $𝑦_{1},$ $\dots,$ $𝑦_{𝑛}$ , which is not obvious even for $𝑛 = 2$ .

Remark 6.2.2. In (b), “one of $𝐱, 𝐲$ ” cannot be shortened to “ $𝐱$ ”: if $𝐱 \neq 𝟎$ and $𝐲 = 𝟎$ , both sides of (a) are $0$ , but $𝐱$ is not a multiple of $𝟎$ . The condition in (b) says exactly that the list $𝐱, 𝐲$ is linearly dependent (Definition 1.5.1). Indeed, $𝐱 = 𝑎 𝐲$ gives the nontrivial relation $1 𝐱 - 𝑎 𝐲 = 𝟎$ , and $𝐲 = 𝑏 𝐱$ gives $𝑏 𝐱 - 1 𝐲 = 𝟎$ . Conversely, suppose $𝑎 𝐱 + 𝑏 𝐲 = 𝟎$ with $𝑎, 𝑏$ not both $0$ . If $𝑎 \neq 0$ , then $𝐱 = - (𝑏 / 𝑎) 𝐲$ . If $𝑎 = 0$ , then $𝑏 \neq 0$ and $𝑏 𝐲 = 𝟎$ , so $𝐲 = 𝟎 = 0 𝐱$ by Proposition 1.2.35.

Over $ℂ$ the scalar may be complex. For $𝐱 \neq 𝟎$ , $| ⟨ 𝐱, 𝑖 𝐱 ⟩ | =$ $| -$ $𝑖 | {‖ 𝐱 ‖}^{2} =$ $‖ 𝐱 ‖ ‖ 𝑖 𝐱 ‖$ , so $𝐱$ and $𝑖 𝐱$ give equality, even though in $ℂ = ℝ^{2}$ the numbers $1$ and $𝑖$ are perpendicular arrows. We return to this in Section 6.2.3.

One direction of (b) is a direct computation, which we record first.

Lemma 6.2.3 (The easy half of (b)). If one of

𝐱, 𝐲 \in 𝑉

is a scalar multiple of the other, then

| ⟨ 𝐱, 𝐲 ⟩ | =

‖ 𝐱 ‖ ‖ 𝐲 ‖

Proof.routine

Uses. Proposition 6.1.27 (Basic properties of the norm), Proposition 6.1.4 (Sesquilinearity), Definition 6.1.26 (Norm)

Hint 1. Substitute

𝐱 = 𝑎 𝐲

and take

𝑎

out of the inner product.

Hint 2. Compare

| 𝑎 | {‖ 𝐲 ‖}^{2}

with

‖ 𝑎 𝐲 ‖ ‖ 𝐲 ‖

, using Proposition 6.1.27. For

𝐲 = 𝑏 𝐱

, the scalar leaves the second slot conjugated, and

| 𝑏 | = | 𝑏 |

Sketch. If

𝐱 = 𝑎 𝐲

, both sides equal

| 𝑎 | {‖ 𝐲 ‖}^{2}

. If

𝐲 = 𝑏 𝐱

, both sides equal

| 𝑏 | {‖ 𝐱 ‖}^{2}

If $𝐱 = 𝑎 𝐲$ , then by linearity in the first slot and Proposition 6.1.27,

| ⟨ 𝑎 𝐲, 𝐲 ⟩ | = | 𝑎 | {‖ 𝐲 ‖}^{2} = ‖ 𝑎 𝐲 ‖ ‖ 𝐲 ‖ = ‖ 𝐱 ‖ ‖ 𝐲 ‖ .

If $𝐲 = 𝑏 𝐱$ , then by Proposition 6.1.4 and Proposition 6.1.27, $| ⟨ 𝐱, 𝑏 𝐱 ⟩ | =$ $| 𝑏 | {‖ 𝐱 ‖}^{2} =$ $‖ 𝐱 ‖ ‖ 𝑏 𝐱 ‖$ , since $| 𝑏 | = | 𝑏 |$ . □

So the content of the theorem is (a), together with the converse of Lemma 6.2.3: equality forces one vector to be a multiple of the other. We give three proofs. Each one finds a squared norm that is nonnegative for an obvious reason, expands it, and reads off (a). Each one gets the equality case by asking when that squared norm is $0$ .

First proof: the shadow

The first proof is the picture of Figure 6.2.1, carried out in $𝑉$ . The move it makes is used again and again in this chapter, so we name it.

Technique 6.2.4 (Orthogonal decomposition). To compare a vector $𝐱$ with a nonzero vector $𝐲$ , split $𝐱$ into a part along $𝐲$ and a part perpendicular to $𝐲$ :

𝐱 = 𝑐 𝐲 + 𝐰, 𝑐 = \frac{⟨ 𝐱, 𝐲 ⟩}{{‖ 𝐲 ‖}^{2}}, 𝐰 = 𝐱 - 𝑐 𝐲 .

This choice of $𝑐$ is exactly the one that makes $⟨ 𝐰, 𝐲 ⟩ = 0$ . Then the cross term in Proposition 6.1.29 vanishes, and ${‖ 𝐱 ‖}^{2} =$ ${| 𝑐 |}^{2} {‖ 𝐲 ‖}^{2} +$ ${‖ 𝐰 ‖}^{2}$ . So the length of $𝐱$ splits into a contribution from the direction $𝐲$ and a contribution from everything else. Dropping ${‖ 𝐰 ‖}^{2} \geq 0$ gives a lower bound for $‖ 𝐱 ‖$ that is sharp exactly when $𝐰 = 𝟎$ .

Reach for it when a statement compares $𝐱$ with one fixed direction, or asks how close $𝐱$ comes to the line through $𝐲$ . The canonical example is the first proof of Theorem 6.2.1. In Section 6.4 the line is replaced by an arbitrary subspace, the part along $𝐲$ becomes the orthogonal projection, and the same computation solves best-approximation problems (Theorem 6.4.33) and least squares (Section 6.5).

Used in 14 places and 1 practice exercise

Theorem 6.2.1, Exercise 6.2.10, Exercise 6.2.27, Exercise 6.2.45, Theorem 6.3.24, Theorem 6.4.33, Exercise 6.4.57, Exercise 6.4.62, Proposition 6.5.27, Theorem 6.6.3, Exercise 6.7.12, Problem 6.1, Problem 6.13, Proposition 8.4.21; 1 practice exercise.

Proof of Theorem 6.2.1 (by orthogonal decomposition).standard

Uses. Proposition 6.1.29 (Expanding a norm), Proposition 6.1.27 (Basic properties of the norm), Proposition 6.1.4 (Sesquilinearity), Lemma 6.2.3 (The easy half of (b))

Techniques. Orthogonal decomposition (Technique 6.2.4)

Hint 1. Dispose of

𝐲 = 𝟎

first. For

𝐲 \neq 𝟎

, write

𝐱

as a multiple of

𝐲

plus a vector

𝐰

with

⟨ 𝐰, 𝐲 ⟩ = 0

Hint 2. The multiple is

𝑐 𝐲

with

𝑐 = ⟨ 𝐱, 𝐲 ⟩ / {‖ 𝐲 ‖}^{2}

. Expand

{‖ 𝐱 ‖}^{2} = {‖ 𝐰 + 𝑐 𝐲 ‖}^{2}

with Proposition 6.1.29.

Hint 3. You should get

{‖ 𝐱 ‖}^{2} =

{‖ 𝐰 ‖}^{2} +

{| ⟨ 𝐱, 𝐲 ⟩ |}^{2} / {‖ 𝐲 ‖}^{2}

. Drop

{‖ 𝐰 ‖}^{2}

. For the equality case, ask when that term is

0

Sketch. If

𝐲 = 𝟎

, both sides vanish. Otherwise put

𝑐 = ⟨ 𝐱, 𝐲 ⟩ / {‖ 𝐲 ‖}^{2}

and

𝐰 = 𝐱 - 𝑐 𝐲

, so that

⟨ 𝐰, 𝐲 ⟩ = 0

. Expanding

{‖ 𝐰 + 𝑐 𝐲 ‖}^{2}

, the cross term vanishes, and

{‖ 𝐱 ‖}^{2} =

{‖ 𝐰 ‖}^{2} +

{| ⟨ 𝐱, 𝐲 ⟩ |}^{2} / {‖ 𝐲 ‖}^{2}

. This gives (a), with equality if and only if

𝐰 = 𝟎

, that is,

𝐱 = 𝑐 𝐲

If $𝐲 = 𝟎$ , then $⟨ 𝐱, 𝐲 ⟩ = 0$ by Proposition 6.1.4 and $‖ 𝐲 ‖ = 0$ , so both sides of (a) are $0$ . Equality holds, and $𝐲 = 0 𝐱$ is a multiple of $𝐱$ , so (b) holds as well.

Suppose $𝐲 \neq 𝟎$ , so that $‖ 𝐲 ‖ > 0$ by Proposition 6.1.27. Let

𝑐 = \frac{⟨ 𝐱, 𝐲 ⟩}{{‖ 𝐲 ‖}^{2}} and 𝐰 = 𝐱 - 𝑐 𝐲 .

By linearity in the first slot, $⟨ 𝐰, 𝐲 ⟩ =$ $⟨ 𝐱, 𝐲 ⟩ -$ $𝑐 ⟨ 𝐲, 𝐲 ⟩ =$ $0$ . Since $𝐱 = 𝐰 + 𝑐 𝐲$ , Proposition 6.1.29 with $𝑎 = 𝑐$ gives

\begin{aligned} {‖ 𝐱 ‖}^{2} & = {‖ 𝐰 ‖}^{2} + 2 Re (𝑐 ⟨ 𝐰, 𝐲 ⟩) + {| 𝑐 |}^{2} {‖ 𝐲 ‖}^{2} \\ = {‖ 𝐰 ‖}^{2} + \frac{{| ⟨ 𝐱, 𝐲 ⟩ |}^{2}}{{‖ 𝐲 ‖}^{4}} {‖ 𝐲 ‖}^{2} \\ = {‖ 𝐰 ‖}^{2} + \frac{{| ⟨ 𝐱, 𝐲 ⟩ |}^{2}}{{‖ 𝐲 ‖}^{2}} . \end{aligned}

(6.2.1)

Since ${‖ 𝐰 ‖}^{2} \geq 0$ , it follows that ${| ⟨ 𝐱, 𝐲 ⟩ |}^{2} / {‖ 𝐲 ‖}^{2} \leq$ ${‖ 𝐱 ‖}^{2}$ . Multiplying by ${‖ 𝐲 ‖}^{2} > 0$ gives ${| ⟨ 𝐱, 𝐲 ⟩ |}^{2} \leq$ ${‖ 𝐱 ‖}^{2} {‖ 𝐲 ‖}^{2}$ , and taking nonnegative square roots gives (a).

For (b), (6.2.1) shows that equality holds in (a) if and only if ${‖ 𝐰 ‖}^{2} = 0$ , that is (by Proposition 6.1.27), if and only if $𝐱 = 𝑐 𝐲$ . So if equality holds, then $𝐱$ is a multiple of $𝐲$ . The converse is Lemma 6.2.3. □

Remark 6.2.5 (The size of the gap). Multiplying (6.2.1) by ${‖ 𝐲 ‖}^{2}$ gives, for $𝐲 \neq 𝟎$ ,

{‖ 𝐱 ‖}^{2} {‖ 𝐲 ‖}^{2} - {| ⟨ 𝐱, 𝐲 ⟩ |}^{2} = {‖ 𝐲 ‖}^{2} {‖ 𝐱 - 𝑐 𝐲 ‖}^{2} .

(6.2.2)

So the first proof yields more than an inequality: it computes the gap, and the gap measures how far $𝐱$ is from the line through $𝐲$ . In $ℝ^{2}$ with the dot product, $‖ 𝐲 ‖$ is the base and $‖ 𝐱 - 𝑐 𝐲 ‖$ the height of the parallelogram spanned by $𝐱$ and $𝐲$ , so the right side is the square of its area. A direct expansion shows that the left side is ${(𝑥_{1} 𝑦_{2} - 𝑥_{2} 𝑦_{1})}^{2}$ , and Exercise 6.2.43 extends this to $ℝ^{𝑛}$ .

Second proof: a quadratic in $𝑡$

The second proof needs no picture and no clever choice. It needs one fact about real quadratics.

Lemma 6.2.6 (Nonnegative quadratics). Let $𝑎, 𝑏, 𝑐 \in ℝ$ with $𝑎 > 0$ , and let $𝑞 (𝑡) =$ $𝑎 𝑡^{2} -$ $2 𝑏 𝑡 +$ $𝑐$ for $𝑡 \in ℝ$ .

(a) $𝑞 (𝑡) \geq 0$ for all $𝑡 \in ℝ$ if and only if $𝑏^{2} \leq 𝑎 𝑐$ .
(b)If $𝑏^{2} \leq 𝑎 𝑐$ , then $𝑞$ has a real root if and only if $𝑏^{2} = 𝑎 𝑐$ , and in that case its only root is $𝑡 = 𝑏 / 𝑎$ .

Proof.routine

Uses. completing the square

Techniques. A sum of nonnegative terms (Technique 6.1.9)

Hint 1. Complete the square. Multiplying by

𝑎

first avoids fractions.

Hint 2. Check that

𝑎 𝑞 (𝑡) =

{(𝑎 𝑡 - 𝑏)}^{2} +

(𝑎 𝑐 - 𝑏^{2})

for all

𝑡

. For the converse in (a), evaluate at the

𝑡

that kills the square.

Sketch.

𝑎 𝑞 (𝑡) =

{(𝑎 𝑡 - 𝑏)}^{2} +

(𝑎 𝑐 - 𝑏^{2})

. If

𝑏^{2} \leq 𝑎 𝑐

, both terms are nonnegative, so

𝑞 \geq 0

; conversely

𝑎 𝑞 (𝑏 / 𝑎) =

𝑎 𝑐 -

𝑏^{2}

. A sum of two nonnegative terms vanishes only if both do, which happens only for

𝑡 = 𝑏 / 𝑎

and

𝑏^{2} = 𝑎 𝑐

For every $𝑡 \in ℝ$ ,

{(𝑎 𝑡 - 𝑏)}^{2} + (𝑎 𝑐 - 𝑏^{2}) = 𝑎^{2} 𝑡^{2} - 2 𝑎 𝑏 𝑡 + 𝑎 𝑐 = 𝑎 𝑞 (𝑡) .

(a) If $𝑏^{2} \leq 𝑎 𝑐$ , then $𝑎 𝑞 (𝑡)$ is a sum of two nonnegative numbers, so $𝑎 𝑞 (𝑡) \geq 0$ , and $𝑞 (𝑡) \geq 0$ since $𝑎 > 0$ . Conversely, if $𝑞 (𝑡) \geq 0$ for all $𝑡$ , then in particular $𝑎 𝑞 (𝑏 / 𝑎) =$ $0 +$ $(𝑎 𝑐 - 𝑏^{2}) \geq$ $0$ .

(b) Suppose $𝑏^{2} \leq 𝑎 𝑐$ . Then $𝑎 𝑞 (𝑡)$ is the sum of the nonnegative numbers ${(𝑎 𝑡 - 𝑏)}^{2}$ and $𝑎 𝑐 - 𝑏^{2}$ , so $𝑞 (𝑡) = 0$ if and only if both are $0$ , that is, if and only if $𝑡 = 𝑏 / 𝑎$ and $𝑏^{2} = 𝑎 𝑐$ . So if $𝑏^{2} = 𝑎 𝑐$ , then $𝑏 / 𝑎$ is a root and the only one, and if $𝑏^{2} < 𝑎 𝑐$ , there is no root. □

Technique 6.2.7 (A nonnegative quadratic has nonpositive discriminant). If a real quadratic $𝑎 𝑡^{2} - 2 𝑏 𝑡 + 𝑐$ with $𝑎 > 0$ is nonnegative for every real $𝑡$ , then $𝑏^{2} \leq 𝑎 𝑐$ , with equality exactly when the quadratic has a real root (Lemma 6.2.6). So to prove an inequality of the shape $𝑏^{2} \leq 𝑎 𝑐$ , look for an expression that is nonnegative for every real $𝑡$ for an obvious reason, usually a squared norm ${‖ 𝐱 - 𝑡 𝐲 ‖}^{2}$ or a sum of squares, and that expands to $𝑎 𝑡^{2} - 2 𝑏 𝑡 + 𝑐$ . The equality case comes for free: it says that the nonnegative expression vanishes for some $𝑡$ .

Reach for it when the target has the shape $𝑏^{2} \leq 𝑎 𝑐$ with $𝑎, 𝑐 \geq 0$ , especially when no geometric picture suggests a decomposition. The parameter $𝑡$ must be real; over $ℂ$ , first make the middle coefficient real (Technique 6.2.8). The canonical example is the second proof of Theorem 6.2.1.

Used in 5 places and 2 practice exercises

Theorem 6.2.1, Exercise 6.2.35, Problem 6.1, Problem 6.17, Lemma 8.2.10; 2 practice exercises.

Over $ℂ$ there is a complication. For real $𝑡$ , Proposition 6.1.29 gives ${‖ 𝐱 - 𝑡 𝐲 ‖}^{2} =$ ${‖ 𝐱 ‖}^{2} -$ $2 𝑡 Re ⟨ 𝐱, 𝐲 ⟩ +$ $𝑡^{2} {‖ 𝐲 ‖}^{2}$ , in which only the real part of $⟨ 𝐱, 𝐲 ⟩$ appears. The cure is to rotate $𝐱$ first.

Technique 6.2.8 (Rotate the phase). Every $𝑧 \in ℂ$ can be written $𝑧 = | 𝑧 | 𝜇$ with $| 𝜇 | = 1$ : take $𝜇 = 𝑧 / | 𝑧 |$ if $𝑧 \neq 0$ , and $𝜇 = 1$ if $𝑧 = 0$ . Then $𝜇 𝑧 = | 𝑧 |$ , so multiplying by $𝜇$ rotates $𝑧$ onto the nonnegative real axis without changing its modulus. In an inner product space, replacing $𝐱$ by $𝜇 𝐱$ changes no norm, since $| 𝜇 | = 1$ , and it turns $⟨ 𝐱, 𝐲 ⟩$ into $𝜇 ⟨ 𝐱, 𝐲 ⟩$ . So in a statement that involves only $‖ 𝐱 ‖$ , $‖ 𝐲 ‖$ , and $| ⟨ 𝐱, 𝐲 ⟩ |$ , we may assume that $⟨ 𝐱, 𝐲 ⟩$ is a nonnegative real number, and real-variable arguments apply. The same observation gives

| 𝑧 | = max_{| 𝜆 | = 1} Re (𝜆 𝑧),

since $Re (𝜆 𝑧) \leq$ $| 𝜆 𝑧 | =$ $| 𝑧 |$ for $| 𝜆 | = 1$ , with equality at $𝜆 = 𝜇$ .

Reach for it when a real-variable argument, such as a discriminant argument, must be run over $ℂ$ , or when $Re ⟨ 𝐱, 𝐲 ⟩$ and $| ⟨ 𝐱, 𝐲 ⟩ |$ must be compared. The canonical example is the second proof of Theorem 6.2.1.

Used in 6 places and 2 practice exercises

Theorem 6.2.1, Exercise 6.2.29, Exercise 6.7.6, Theorem B.1.11, Exercise B.1.44, Exercise B.1.49; 2 practice exercises.

Proof of Theorem 6.2.1 (by a quadratic in $𝑡$ ).standard

Uses. Lemma 6.2.6 (Nonnegative quadratics), Proposition 6.1.29 (Expanding a norm), Proposition 6.1.27 (Basic properties of the norm), Lemma 6.2.3 (The easy half of (b)), Notation 6.1.1 (Real and complex scalars)

Techniques. A nonnegative quadratic has nonpositive discriminant (Technique 6.2.7), Rotate the phase (Technique 6.2.8)

Hint 1. For real

𝑡

, the number

{‖ 𝐱 - 𝑡 𝐲 ‖}^{2}

is nonnegative. Expand it as a function of

𝑡

Hint 2. Over

ℝ

you get

{‖ 𝐱 ‖}^{2} -

2 𝑡 ⟨ 𝐱, 𝐲 ⟩ +

𝑡^{2} {‖ 𝐲 ‖}^{2} \geq

0

for all

𝑡

, and Lemma 6.2.6 applies when

𝐲 \neq 𝟎

. Over

ℂ

the middle coefficient is only

Re ⟨ 𝐱, 𝐲 ⟩

Hint 3. Over

ℂ

, first replace

𝐱

𝜆 𝐱

, with

| 𝜆 | = 1

chosen so that

⟨ 𝜆 𝐱, 𝐲 ⟩ =

| ⟨ 𝐱, 𝐲 ⟩ |

. This changes neither side of (a).

Sketch. Assume

𝐲 \neq 𝟎

. Choose

| 𝜆 | = 1

with

𝜆 ⟨ 𝐱, 𝐲 ⟩ =

| ⟨ 𝐱, 𝐲 ⟩ |

. For real

𝑡

0 \leq

{‖ 𝜆 𝐱 - 𝑡 𝐲 ‖}^{2} =

{‖ 𝐱 ‖}^{2} -

2 𝑡 | ⟨ 𝐱, 𝐲 ⟩ | +

𝑡^{2} {‖ 𝐲 ‖}^{2}

. A quadratic with positive leading coefficient that is never negative has

𝑏^{2} \leq 𝑎 𝑐

, which is (a). Equality means that the quadratic has a real root

𝑡_{0}

, so

𝜆 𝐱 = 𝑡_{0} 𝐲

If $𝐲 = 𝟎$ , both sides of (a) are $0$ , and $𝐲 = 0 𝐱$ , so (a) and (b) hold. Suppose $𝐲 \neq 𝟎$ . By Lemma 6.2.3, it remains to prove (a) and to show that equality forces one vector to be a multiple of the other.

Write $⟨ 𝐱, 𝐲 ⟩ =$ $| ⟨ 𝐱, 𝐲 ⟩ | 𝜇$ with $| 𝜇 | = 1$ , and let $𝜆 = 𝜇$ and $𝐱^{'} = 𝜆 𝐱$ . Since $𝜆 𝜇 = {| 𝜇 |}^{2} = 1$ ,

⟨ 𝐱^{'}, 𝐲 ⟩ = 𝜆 ⟨ 𝐱, 𝐲 ⟩ = | ⟨ 𝐱, 𝐲 ⟩ |,

a nonnegative real number, and $‖ 𝐱^{'} ‖ =$ $| 𝜆 | ‖ 𝐱 ‖ =$ $‖ 𝐱 ‖$ by Proposition 6.1.27. Let $𝑡 \in ℝ$ . By Proposition 6.1.29 with the scalar $- 𝑡$ , using $- 𝑡 = - 𝑡$ and that $⟨ 𝐱^{'}, 𝐲 ⟩$ is real,

0 \leq {‖ 𝐱^{'} - 𝑡 𝐲 ‖}^{2} = {‖ 𝐱 ‖}^{2} - 2 𝑡 | ⟨ 𝐱, 𝐲 ⟩ | + 𝑡^{2} {‖ 𝐲 ‖}^{2} .

The right side is $𝑞 (𝑡) =$ $𝑎 𝑡^{2} -$ $2 𝑏 𝑡 +$ $𝑐$ with

𝑎 = {‖ 𝐲 ‖}^{2} > 0, 𝑏 = | ⟨ 𝐱, 𝐲 ⟩ |, 𝑐 = {‖ 𝐱 ‖}^{2},

and it is nonnegative for every real $𝑡$ . By Lemma 6.2.6, $𝑏^{2} \leq 𝑎 𝑐$ , that is, ${| ⟨ 𝐱, 𝐲 ⟩ |}^{2} \leq$ ${‖ 𝐱 ‖}^{2} {‖ 𝐲 ‖}^{2}$ . Taking nonnegative square roots gives (a).

If equality holds in (a), then $𝑏^{2} = 𝑎 𝑐$ , so $𝑞$ has a real root $𝑡_{0}$ by Lemma 6.2.6. Then $‖ 𝐱^{'} - 𝑡_{0} 𝐲 ‖ = 0$ , so $𝜆 𝐱 = 𝑡_{0} 𝐲$ by Proposition 6.1.27. Multiplying by $𝜆$ and using $𝜆 𝜆 = 1$ gives $𝐱 = 𝜆 𝑡_{0} 𝐲$ , a multiple of $𝐲$ . □

Over $ℝ$ , $𝜆 = \pm 1$ , and the rotation only flips the sign of $𝐱$ when $⟨ 𝐱, 𝐲 ⟩ < 0$ ; one could instead apply Lemma 6.2.6 with $𝑏 = ⟨ 𝐱, 𝐲 ⟩$ directly. Over $ℂ$ the rotation cannot be skipped. Run on $𝐱$ itself, the argument proves only ${(Re ⟨ 𝐱, 𝐲 ⟩)}^{2} \leq$ ${‖ 𝐱 ‖}^{2} {‖ 𝐲 ‖}^{2}$ , which is weaker than (a) whenever $⟨ 𝐱, 𝐲 ⟩$ is not real. Rotating moves all of $| ⟨ 𝐱, 𝐲 ⟩ |$ into the real part. (One can also let $𝑡$ be complex. The best $𝑡$ is then $𝑐$ from the first proof, and the computation becomes the first proof.)

Third proof: normalizing

Intuition (Balancing the two sides). The cheapest inequality available is ${‖ 𝐱 - 𝐲 ‖}^{2} \geq 0$ , which by Proposition 6.1.29 says

2 Re ⟨ 𝐱, 𝐲 ⟩ \leq {‖ 𝐱 ‖}^{2} + {‖ 𝐲 ‖}^{2} .

It bounds a product by a sum, and when $‖ 𝐱 ‖$ and $‖ 𝐲 ‖$ are very different it is far from sharp. But both sides of (a) are homogeneous: replacing $𝐱$ by $𝑠 𝐱$ with $𝑠 > 0$ multiplies both $| ⟨ 𝐱, 𝐲 ⟩ |$ and $‖ 𝐱 ‖ ‖ 𝐲 ‖$ by $𝑠$ , and similarly for $𝐲$ . So it suffices to prove (a) after rescaling $𝐱$ and $𝐲$ , and we can rescale them to unit vectors. For unit vectors the right side above is $1 + 1$ , which is twice the product $‖ 𝐱 ‖ ‖ 𝐲 ‖ = 1$ , and the cheap inequality becomes sharp enough.

Equivalently, apply the cheap inequality to $𝑠 𝐱$ and $𝐲 / 𝑠$ , and choose $𝑠 > 0$ to make the bound $𝑠^{2} {‖ 𝐱 ‖}^{2} +$ ${‖ 𝐲 ‖}^{2} / 𝑠^{2}$ as small as possible. The choice $𝑠^{2} = ‖ 𝐲 ‖ / ‖ 𝐱 ‖$ makes the two terms equal, each $‖ 𝐱 ‖ ‖ 𝐲 ‖$ , and the bound becomes $2 ‖ 𝐱 ‖ ‖ 𝐲 ‖$ .

Proof of Theorem 6.2.1 (by normalizing).standard

Uses. Proposition 6.1.29 (Expanding a norm), Proposition 6.1.27 (Basic properties of the norm), Proposition 6.1.4 (Sesquilinearity), Lemma 6.2.3 (The easy half of (b))

Techniques. Rotate the phase (Technique 6.2.8)

Hint 1. Dispose of the case where

𝐱

𝐲

𝟎

. Otherwise, compare

𝐱 / ‖ 𝐱 ‖

and

𝐲 / ‖ 𝐲 ‖

Hint 2. For unit vectors

𝐮, 𝐯

, expand

0 \leq {‖ 𝐮 - 𝐯 ‖}^{2}

. You get

Re ⟨ 𝐮, 𝐯 ⟩ \leq 1

Hint 3. To get

| ⟨ 𝐮, 𝐯 ⟩ | \leq 1

, first rotate

𝐮

by a scalar of modulus

1

so that

⟨ 𝐮, 𝐯 ⟩

is a nonnegative real number.

Sketch. For nonzero

𝐱, 𝐲

, let

𝐮 = 𝜆 𝐱 / ‖ 𝐱 ‖

and

𝐯 = 𝐲 / ‖ 𝐲 ‖

, where

| 𝜆 | = 1

makes

⟨ 𝐮, 𝐯 ⟩ =

| ⟨ 𝐱, 𝐲 ⟩ | / (‖ 𝐱 ‖ ‖ 𝐲 ‖)

real and nonnegative. Then

0 \leq

{‖ 𝐮 - 𝐯 ‖}^{2} =

2 -

2 ⟨ 𝐮, 𝐯 ⟩

gives (a), and equality forces

𝐮 = 𝐯

, so

𝐲

is a multiple of

𝐱

If $𝐱 = 𝟎$ or $𝐲 = 𝟎$ , both sides of (a) are $0$ by Proposition 6.1.4, and one vector is $0$ times the other, so (a) and (b) hold. Suppose $𝐱, 𝐲 \neq 𝟎$ . By Lemma 6.2.3, it remains to prove (a) and to show that equality forces one vector to be a multiple of the other.

Choose $𝜆 \in 𝔽$ with $| 𝜆 | = 1$ and $𝜆 ⟨ 𝐱, 𝐲 ⟩ =$ $| ⟨ 𝐱, 𝐲 ⟩ |$ , as in Technique 6.2.8, and let

𝐮 = \frac{𝜆}{‖ 𝐱 ‖} 𝐱, 𝐯 = \frac{1}{‖ 𝐲 ‖} 𝐲 .

By Proposition 6.1.27, $‖ 𝐮 ‖ = ‖ 𝐯 ‖ = 1$ . Since $1 / ‖ 𝐲 ‖$ is real, it leaves the second slot unconjugated, and

⟨ 𝐮, 𝐯 ⟩ = \frac{𝜆}{‖ 𝐱 ‖ ‖ 𝐲 ‖} ⟨ 𝐱, 𝐲 ⟩ = \frac{| ⟨ 𝐱, 𝐲 ⟩ |}{‖ 𝐱 ‖ ‖ 𝐲 ‖},

a nonnegative real number. By Proposition 6.1.29,

0 \leq {‖ 𝐮 - 𝐯 ‖}^{2} = {‖ 𝐮 ‖}^{2} - 2 Re ⟨ 𝐮, 𝐯 ⟩ + {‖ 𝐯 ‖}^{2} = 2 - 2 \frac{| ⟨ 𝐱, 𝐲 ⟩ |}{‖ 𝐱 ‖ ‖ 𝐲 ‖} .

Rearranging gives (a).

If equality holds in (a), the right side is $0$ , so $‖ 𝐮 - 𝐯 ‖ = 0$ and $𝐮 = 𝐯$ by Proposition 6.1.27. That is, $𝜆 𝐱 / ‖ 𝐱 ‖ = 𝐲 / ‖ 𝐲 ‖$ , so $𝐲 =$ $(𝜆 ‖ 𝐲 ‖ / ‖ 𝐱 ‖) 𝐱$ is a multiple of $𝐱$ . □

Intuition (Comparing the three proofs). All three proofs come from one fact: ${‖ 𝐱 - 𝑡 𝐲 ‖}^{2} \geq 0$ for a scalar $𝑡$ (in the second and third proofs, after rotating the phase of $𝐱$ ). They differ in how $𝑡$ is chosen.

The first proof takes the best $𝑡$ , namely $𝑡 =$ $𝑐 =$ $⟨ 𝐱, 𝐲 ⟩ / {‖ 𝐲 ‖}^{2}$ . It makes $𝐱 - 𝑡 𝐲$ perpendicular to $𝐲$ , so $𝑐 𝐲$ is the point of the line through $𝐲$ closest to $𝐱$ . Finding $𝑐$ takes the picture or a computation, but once it is found the proof is one expansion, and it produces the identity (6.2.2) rather than just an inequality.

The second proof does not choose $𝑡$ at all. It uses every real $𝑡$ at once and lets the discriminant do the optimizing: $𝑏^{2} \leq 𝑎 𝑐$ says that the minimum value of the quadratic, attained at $𝑡 = 𝑏 / 𝑎$ , is nonnegative. For $𝐱$ rotated so that $⟨ 𝐱, 𝐲 ⟩ \geq 0$ , that minimizer $𝑏 / 𝑎 =$ $⟨ 𝐱, 𝐲 ⟩ / {‖ 𝐲 ‖}^{2}$ is exactly the coefficient of the first proof. This is the proof to remember when an inequality has the shape $𝑏^{2} \leq 𝑎 𝑐$ and there is no picture: find a quantity that is nonnegative and quadratic in a real parameter.

The third proof rescales so that $‖ 𝐱 ‖ = ‖ 𝐲 ‖ = 1$ , after which the crude choice $𝑡 = 1$ already suffices. It needs a separate zero case and a normalization step, but its idea, that a homogeneous inequality can be checked on normalized vectors, applies far beyond inner products.

The equality case works the same way in all three: equality means that the nonnegative quantity is $0$ for the chosen $𝑡$ , and that says one vector is a multiple of the other.

Example 6.2.9 (Instances of Cauchy–Schwarz). Each inner product of Section 6.1.3 gives an inequality.

(a)For the dot product on $ℝ^{𝑛}$ : ${(\sum_{𝑘} 𝑎_{𝑘} 𝑏_{𝑘})}^{2} \leq$ $(\sum_{𝑘} 𝑎_{𝑘}^{2}) (\sum_{𝑘} 𝑏_{𝑘}^{2})$ for all real $𝑎_{𝑘}, 𝑏_{𝑘}$ . For $(1, 2, 3)$ and $(4, 5, 6)$ this reads $32^{2} =$ $1024 \leq$ $14 \cdot$ $77 =$ $1078$ .
(b)For the integral inner product on $𝐶 [𝑎, 𝑏]$ (Proposition 6.1.13 and the paragraph after it):
${(\int_{𝑎}^{𝑏} 𝑓 (𝑡) 𝑔 (𝑡) d 𝑡)}^{2} \leq (\int_{𝑎}^{𝑏} {𝑓 (𝑡)}^{2} d 𝑡) (\int_{𝑎}^{𝑏} {𝑔 (𝑡)}^{2} d 𝑡) .$
For $𝑓 (𝑡) = 1$ and $𝑔 (𝑡) = 𝑡$ on $[0, 1]$ , this reads $1 / 4 \leq 1 / 3$ .
(c)For the Frobenius inner product on $𝔽^{𝑚 \times 𝑛}$ (Proposition 6.1.16): ${| tr (𝐵^{*} 𝐴) |}^{2} \leq$ $tr (𝐴^{*} 𝐴) tr (𝐵^{*} 𝐵)$ .

In each case equality holds exactly for proportional data. In (b), that means $𝑔 = 𝑐 𝑓$ or $𝑓 = 𝑐 𝑔$ as functions on $[𝑎, 𝑏]$ .

Exercise 6.2.10.routine In

ℂ^{2}

with the dot product, let

𝐱 = (1 + 𝑖, 2)

and

𝐲 = (𝑖, 1 - 𝑖)

, as in Example 6.1.11, where we found

𝐱 \cdot 𝐲 = 3 + 𝑖

{‖ 𝐱 ‖}^{2} = 6

, and

{‖ 𝐲 ‖}^{2} = 3

. Find

𝑐 \in ℂ

and

𝐰 \in ℂ^{2}

with

𝐱 = 𝑐 𝐲 + 𝐰

and

𝐰 \cdot 𝐲 = 0

, and check (6.2.1) numerically.

Solution.

Uses. Definition 6.1.8 (Dot product), (6.2.1)

Techniques. Orthogonal decomposition (Technique 6.2.4)

Hint 1.

𝑐 = (𝐱 \cdot 𝐲) / {‖ 𝐲 ‖}^{2}

Hint 2. Compute

𝑐 𝐲

coordinate by coordinate:

(3 + 𝑖) 𝑖 = - 1 + 3 𝑖

and

(3 + 𝑖) (1 - 𝑖) =

4 -

2 𝑖

Sketch.

𝑐 = (3 + 𝑖) / 3

𝑐 𝐲 =

((- 1 + 3 𝑖) / 3, (4 - 2 𝑖) / 3)

, and

𝐰 =

(4 / 3, (2 + 2 𝑖) / 3)

. Then

{‖ 𝐰 ‖}^{2} = 8 / 3

and

{| 3 + 𝑖 |}^{2} / 3 = 10 / 3

, which add to

6

By Technique 6.2.4, $𝑐 =$ $(𝐱 \cdot 𝐲) / {‖ 𝐲 ‖}^{2} =$ $(3 + 𝑖) / 3$ . Since $(3 + 𝑖) 𝑖 = - 1 + 3 𝑖$ and $(3 + 𝑖) (1 - 𝑖) =$ $4 -$ $2 𝑖$ ,

𝑐 𝐲 = (\frac{- 1 + 3 𝑖}{3}, \frac{4 - 2 𝑖}{3}), 𝐰 = 𝐱 - 𝑐 𝐲 = (\frac{4}{3}, \frac{2 + 2 𝑖}{3}) .

Check:

𝐰 \cdot 𝐲 = \frac{4}{3} 𝑖 + \frac{2 + 2 𝑖}{3} (1 - 𝑖) = \frac{- 4 𝑖}{3} + \frac{4 𝑖}{3} = 0,

using $(2 + 2 𝑖) (1 + 𝑖) = 4 𝑖$ . Now ${‖ 𝐰 ‖}^{2} =$ $16 / 9 +$ $8 / 9 =$ $8 / 3$ (as ${| 2 + 2 𝑖 |}^{2} = 8$ ), and ${| 𝐱 \cdot 𝐲 |}^{2} / {‖ 𝐲 ‖}^{2} =$ $10 / 3$ . Their sum is $18 / 3 = 6 = {‖ 𝐱 ‖}^{2}$ , as (6.2.1) predicts. In particular ${| 𝐱 \cdot 𝐲 |}^{2} =$ $10 \leq$ $18 =$ ${‖ 𝐱 ‖}^{2} {‖ 𝐲 ‖}^{2}$ .

6.2.2The triangle inequality

In a triangle, each side is at most as long as the other two together. The sides of the triangle with vertices $𝟎$ , $𝐱$ , and $𝐱 + 𝐲$ are $𝐱$ , $𝐲$ , and $𝐱 + 𝐲$ (Figure 6.2.2), so the statement to prove is $‖ 𝐱 + 𝐲 ‖ \leq ‖ 𝐱 ‖ + ‖ 𝐲 ‖$ . Both sides are nonnegative, so we may compare squares:

\begin{aligned} {‖ 𝐱 + 𝐲 ‖}^{2} & = {‖ 𝐱 ‖}^{2} + 2 Re ⟨ 𝐱, 𝐲 ⟩ + {‖ 𝐲 ‖}^{2}, \\ {(‖ 𝐱 ‖ + ‖ 𝐲 ‖)}^{2} & = {‖ 𝐱 ‖}^{2} + 2 ‖ 𝐱 ‖ ‖ 𝐲 ‖ + {‖ 𝐲 ‖}^{2} . \end{aligned}

So the triangle inequality is the statement $Re ⟨ 𝐱, 𝐲 ⟩ \leq$ $‖ 𝐱 ‖ ‖ 𝐲 ‖$ , and Cauchy–Schwarz gives this with room to spare, since $Re 𝑧 \leq | 𝑧 |$ .

Figure 6.2.2. (a) The triangle inequality: the side

𝐱 + 𝐲

is no longer than the path

𝐱

followed by

𝐲

. (b) Equality: when

𝐲

is a positive multiple of

𝐱

, the triangle is flat. (The arrow

𝐱 + 𝐲

is drawn slightly offset.)

Theorem 6.2.11 (Triangle inequality). Let $𝑉$ be an inner product space and $𝐱, 𝐲 \in 𝑉$ .

(a) $‖ 𝐱 + 𝐲 ‖ \leq ‖ 𝐱 ‖ + ‖ 𝐲 ‖$ .
(b)The following are equivalent:
1. (i) $‖ 𝐱 + 𝐲 ‖ = ‖ 𝐱 ‖ + ‖ 𝐲 ‖$ ;
2. (ii) $⟨ 𝐱, 𝐲 ⟩ = ‖ 𝐱 ‖ ‖ 𝐲 ‖$ ;
3. (iii)one of $𝐱, 𝐲$ is a nonnegative real multiple of the other.

Proof.standard

Uses. Proposition 6.1.29 (Expanding a norm), Theorem 6.2.1 (Cauchy–Schwarz inequality), Proposition 6.1.27 (Basic properties of the norm), Proposition 6.1.4 (Sesquilinearity), Notation 6.1.1 (Real and complex scalars)

Hint 1. Both sides of (a) are nonnegative, so compare their squares. Expand

{‖ 𝐱 + 𝐲 ‖}^{2}

with Proposition 6.1.29.

Hint 2. The squared inequality reduces to

Re ⟨ 𝐱, 𝐲 ⟩ \leq

‖ 𝐱 ‖ ‖ 𝐲 ‖

. Pass through

| ⟨ 𝐱, 𝐲 ⟩ |

Hint 3. For (b), equality in (a) forces both steps

Re 𝑧 \leq | 𝑧 |

and Cauchy–Schwarz to be equalities. When is

Re 𝑧 = | 𝑧 |

? For (ii)

\Rightarrow

(iii), use Theorem 6.2.1(b) and then compare

⟨ 𝑎 𝐲, 𝐲 ⟩

with

‖ 𝑎 𝐲 ‖ ‖ 𝐲 ‖

Sketch. In the expansion of

{‖ 𝐱 + 𝐲 ‖}^{2}

, the cross term

2 Re ⟨ 𝐱, 𝐲 ⟩

is at most

2 | ⟨ 𝐱, 𝐲 ⟩ |

, which is at most

2 ‖ 𝐱 ‖ ‖ 𝐲 ‖

by Cauchy–Schwarz; so

{‖ 𝐱 + 𝐲 ‖}^{2} \leq

{(‖ 𝐱 ‖ + ‖ 𝐲 ‖)}^{2}

. Equality holds if and only if both steps are equalities, that is,

⟨ 𝐱, 𝐲 ⟩

is a nonnegative real number equal to

‖ 𝐱 ‖ ‖ 𝐲 ‖

. By Cauchy–Schwarz, this forces

𝐱 = 𝑎 𝐲

(when

𝐲 \neq 𝟎

), and then

𝑎 {‖ 𝐲 ‖}^{2} = | 𝑎 | {‖ 𝐲 ‖}^{2}

gives

𝑎 \geq 0

. The converse is a direct computation.

(a) By Proposition 6.1.29, $Re 𝑧 \leq | 𝑧 |$ (Notation 6.1.1), and Theorem 6.2.1,

\begin{aligned} {‖ 𝐱 + 𝐲 ‖}^{2} & = {‖ 𝐱 ‖}^{2} + 2 Re ⟨ 𝐱, 𝐲 ⟩ + {‖ 𝐲 ‖}^{2} \\ \leq {‖ 𝐱 ‖}^{2} + 2 | ⟨ 𝐱, 𝐲 ⟩ | + {‖ 𝐲 ‖}^{2} \\ \leq {‖ 𝐱 ‖}^{2} + 2 ‖ 𝐱 ‖ ‖ 𝐲 ‖ + {‖ 𝐲 ‖}^{2} = {(‖ 𝐱 ‖ + ‖ 𝐲 ‖)}^{2} . \end{aligned}

(6.2.3)

Both $‖ 𝐱 + 𝐲 ‖$ and $‖ 𝐱 ‖ + ‖ 𝐲 ‖$ are nonnegative, so taking square roots gives (a).

(b) We prove (i) $\Leftrightarrow$ (ii) and (ii) $\Leftrightarrow$ (iii).

(i) $\Leftrightarrow$ (ii). Since both sides of (a) are nonnegative, (i) holds if and only if the first and last expressions in (6.2.3) are equal. Their difference is the sum of the two nonnegative gaps in the chain, so this happens if and only if both inequalities in the chain are equalities:

Re ⟨ 𝐱, 𝐲 ⟩ = | ⟨ 𝐱, 𝐲 ⟩ | and | ⟨ 𝐱, 𝐲 ⟩ | = ‖ 𝐱 ‖ ‖ 𝐲 ‖ .

For $𝑧 \in ℂ$ , $Re 𝑧 = | 𝑧 |$ if and only if $𝑧$ is a nonnegative real number. Indeed, writing $𝑧 = 𝑎 + 𝑏 𝑖$ with $𝑎, 𝑏 \in ℝ$ , we have $Re 𝑧 =$ $𝑎 \leq$ $| 𝑎 | \leq$ $\sqrt{𝑎^{2} + 𝑏^{2}} =$ $| 𝑧 |$ , with equality in the first step if and only if $𝑎 \geq 0$ and in the second if and only if $𝑏 = 0$ . So the two equalities together say that $⟨ 𝐱, 𝐲 ⟩$ is a nonnegative real number equal to $‖ 𝐱 ‖ ‖ 𝐲 ‖$ , which is (ii). Conversely, (ii) gives both equalities.

(iii) $\Rightarrow$ (ii). If $𝐱 = 𝑎 𝐲$ with $𝑎 \geq 0$ , then $| 𝑎 | = 𝑎$ , and by Proposition 6.1.27, $⟨ 𝐱, 𝐲 ⟩ =$ $𝑎 {‖ 𝐲 ‖}^{2} =$ $‖ 𝑎 𝐲 ‖ ‖ 𝐲 ‖ =$ $‖ 𝐱 ‖ ‖ 𝐲 ‖$ . If $𝐲 = 𝑏 𝐱$ with $𝑏 \geq 0$ , then $𝑏 = 𝑏$ , and by Proposition 6.1.4, $⟨ 𝐱, 𝐲 ⟩ =$ $𝑏 {‖ 𝐱 ‖}^{2} =$ $‖ 𝐱 ‖ ‖ 𝐲 ‖$ in the same way.

(ii) $\Rightarrow$ (iii). Assume (ii). If $𝐲 = 𝟎$ , then $𝐲 = 0 𝐱$ and (iii) holds. Suppose $𝐲 \neq 𝟎$ . By (ii), $| ⟨ 𝐱, 𝐲 ⟩ | =$ $‖ 𝐱 ‖ ‖ 𝐲 ‖$ , so by Theorem 6.2.1(b) one of $𝐱, 𝐲$ is a multiple of the other. In fact $𝐱$ is a multiple of $𝐲$ : if $𝐲 = 𝑏 𝐱$ , then $𝑏 \neq 0$ because $𝐲 \neq 𝟎$ , and $𝐱 = 𝑏^{- 1} 𝐲$ . Write $𝐱 = 𝑎 𝐲$ with $𝑎 \in 𝔽$ . Then

𝑎 {‖ 𝐲 ‖}^{2} = ⟨ 𝐱, 𝐲 ⟩ = ‖ 𝐱 ‖ ‖ 𝐲 ‖ = | 𝑎 | {‖ 𝐲 ‖}^{2},

and dividing by ${‖ 𝐲 ‖}^{2} > 0$ gives $𝑎 = | 𝑎 |$ , a nonnegative real number. □

Intuition (Same line versus same direction). Equality in Cauchy–Schwarz asks that

𝐱

and

𝐲

lie on a common line

𝔽 𝐯

through

𝟎

. Equality in the triangle inequality asks for more, a common ray

{𝑡 𝐯 | 𝑡 \geq 0}

. Over

ℝ

, if

𝐲 = - 𝐱 \neq 𝟎

, then

| ⟨ 𝐱, 𝐲 ⟩ | =

‖ 𝐱 ‖ ‖ 𝐲 ‖

but

‖ 𝐱 +

𝐲 ‖ =

0 <

2 ‖ 𝐱 ‖

. Over

ℂ

𝐲 = 𝑖 𝐱

gives equality in Cauchy–Schwarz, but

‖ 𝐱 +

𝑖 𝐱 ‖ =

| 1 +

𝑖 | ‖ 𝐱 ‖ =

\sqrt{2} ‖ 𝐱 ‖

. In the proof, it is the first inequality of (6.2.3),

Re ⟨ 𝐱, 𝐲 ⟩ \leq

| ⟨ 𝐱, 𝐲 ⟩ |

, that detects the direction.

Exercise 6.2.12.routine Let

𝐱 \neq 𝟎

be a vector in a complex inner product space. Find all

𝑎 \in ℂ

with

‖ 𝐱 +

𝑎 𝐱 ‖ =

‖ 𝐱 ‖ +

‖ 𝑎 𝐱 ‖

, and all

𝑎 \in ℂ

with

‖ 𝐱 +

𝑎 𝐱 ‖ =

| ‖ 𝐱 ‖ -

‖ 𝑎 𝐱 ‖ |

Solution.

Uses. Proposition 6.1.27 (Basic properties of the norm), Theorem 6.2.11 (Triangle inequality), Notation 6.1.1 (Real and complex scalars)

Hint 1. By Proposition 6.1.27, both conditions are equations between real multiples of

‖ 𝐱 ‖

. Divide by

‖ 𝐱 ‖

Hint 2. The first condition becomes

| 1 + 𝑎 | = 1 + | 𝑎 |

, the triangle inequality in

ℂ = ℂ^{1}

itself. For the second, square

| 1 + 𝑎 | = | 1 - | 𝑎 | |

and expand

{| 1 + 𝑎 |}^{2} =

1 +

2 Re 𝑎 +

{| 𝑎 |}^{2}

Sketch. The first condition is

| 1 + 𝑎 | = 1 + | 𝑎 |

, which by Theorem 6.2.11(b) in

ℂ

means

𝑎 \geq 0

is real. Squaring the second gives

2 Re 𝑎 = - 2 | 𝑎 |

, so

𝑎

is a real number

\leq 0

By Proposition 6.1.27, $‖ 𝐱 +$ $𝑎 𝐱 ‖ =$ $| 1 +$ $𝑎 | ‖ 𝐱 ‖$ and $‖ 𝑎 𝐱 ‖ = | 𝑎 | ‖ 𝐱 ‖$ , and $‖ 𝐱 ‖ > 0$ .

The first condition is $| 1 + 𝑎 | = 1 + | 𝑎 |$ . This is the equality case of Theorem 6.2.11 in $ℂ$ with the dot product $𝑧 𝑤$ , for the vectors $1$ and $𝑎$ . By (b), it holds if and only if one of $1, 𝑎$ is a nonnegative real multiple of the other, that is, if and only if $𝑎$ is a real number with $𝑎 \geq 0$ . (If $1 = 𝑏 𝑎$ with $𝑏 \geq 0$ , then $𝑏 > 0$ and $𝑎 = 1 / 𝑏 > 0$ .)

The second condition is $| 1 + 𝑎 | = | 1 - | 𝑎 | |$ . Both sides are nonnegative, so it is equivalent to the equality of squares $1 +$ $2 Re 𝑎 +$ ${| 𝑎 |}^{2} =$ $1 -$ $2 | 𝑎 | +$ ${| 𝑎 |}^{2}$ , that is, $Re 𝑎 = - | 𝑎 |$ . Writing $𝑎 = 𝑠 + 𝑡 𝑖$ , this says $𝑠 = - \sqrt{𝑠^{2} + 𝑡^{2}}$ , which holds if and only if $𝑡 = 0$ and $𝑠 \leq 0$ . So the second condition holds exactly for real $𝑎 \leq 0$ ; for $𝑎 < 0$ , the vectors $𝐱$ and $𝑎 𝐱$ point in opposite directions.

The triangle inequality extends to any number of vectors, and it has a reversed form.

Corollary 6.2.13 (Sums of several vectors). For $𝐱_{1}, \dots, 𝐱_{𝑛} \in 𝑉$ ,

‖ 𝐱_{1} + \dots + 𝐱_{𝑛} ‖ \leq ‖ 𝐱_{1} ‖ + \dots + ‖ 𝐱_{𝑛} ‖ .

Proof.routine

Uses. Theorem 6.2.11 (Triangle inequality)

Techniques. Induction on the length of a list (Technique 1.6.9)

Hint 1. Induct on

𝑛

Hint 2. Write

𝐱_{1} +

\dots +

𝐱_{𝑛} =

(𝐱_{1} + \dots + 𝐱_{𝑛 - 1}) +

𝐱_{𝑛}

and apply Theorem 6.2.11 to these two vectors.

Sketch. The case

𝑛 = 1

is trivial. For

𝑛 \geq 2

, the triangle inequality splits off

‖ 𝐱_{𝑛} ‖

, and the induction hypothesis bounds the norm of the remaining sum.

We induct on $𝑛$ . For $𝑛 = 1$ there is nothing to prove. Let $𝑛 \geq 2$ , and assume the claim for $𝑛 - 1$ vectors. With $𝐬 = 𝐱_{1} + \dots + 𝐱_{𝑛 - 1}$ , Theorem 6.2.11 and the induction hypothesis give

‖ 𝐬 + 𝐱_{𝑛} ‖ \leq ‖ 𝐬 ‖ + ‖ 𝐱_{𝑛} ‖ \leq ‖ 𝐱_{1} ‖ + \dots + ‖ 𝐱_{𝑛 - 1} ‖ + ‖ 𝐱_{𝑛} ‖,

as desired. □

Corollary 6.2.14 (Distances and the reverse triangle inequality). For all $𝐱, 𝐲, 𝐳 \in 𝑉$ :

(a) $‖ 𝐱 -$ $𝐳 ‖ \leq$ $‖ 𝐱 -$ $𝐲 ‖ +$ $‖ 𝐲 -$ $𝐳 ‖$ ;
(b) $| ‖ 𝐱 ‖ -$ $‖ 𝐲 ‖ | \leq$ $‖ 𝐱 -$ $𝐲 ‖$ .

Proof.routine

Uses. Theorem 6.2.11 (Triangle inequality), Proposition 6.1.27 (Basic properties of the norm)

Hint 1. For (a), write

𝐱 - 𝐳

as a sum of two vectors.

Hint 2. For (b), write

𝐱 = (𝐱 - 𝐲) + 𝐲

and apply the triangle inequality. This bounds

‖ 𝐱 ‖ - ‖ 𝐲 ‖

. Then exchange the roles of

𝐱

and

𝐲

Sketch. (a) is the triangle inequality for

(𝐱 - 𝐲) + (𝐲 - 𝐳)

. For (b),

‖ 𝐱 ‖ \leq ‖ 𝐱 - 𝐲 ‖ + ‖ 𝐲 ‖

and

‖ 𝐲 ‖ \leq ‖ 𝐲 - 𝐱 ‖ + ‖ 𝐱 ‖

, and

‖ 𝐲 - 𝐱 ‖ = ‖ 𝐱 - 𝐲 ‖

(a) Apply Theorem 6.2.11 to $𝐱 -$ $𝐳 =$ $(𝐱 - 𝐲) +$ $(𝐲 - 𝐳)$ .

(b) By Theorem 6.2.11, $‖ 𝐱 ‖ =$ $‖ (𝐱 - 𝐲) +$ $𝐲 ‖ \leq$ $‖ 𝐱 -$ $𝐲 ‖ +$ $‖ 𝐲 ‖$ , so $‖ 𝐱 ‖ - ‖ 𝐲 ‖ \leq ‖ 𝐱 - 𝐲 ‖$ . Exchanging $𝐱$ and $𝐲$ , $‖ 𝐲 ‖ -$ $‖ 𝐱 ‖ \leq$ $‖ 𝐲 -$ $𝐱 ‖ =$ $‖ 𝐱 -$ $𝐲 ‖$ , where the last step is Proposition 6.1.27 with $𝑎 = - 1$ . Since $| 𝑟 |$ is the larger of $𝑟$ and $- 𝑟$ , (b) follows. □

Part (a) is the triangle inequality for the triangle with vertices $𝐱$ , $𝐲$ , $𝐳$ : going from $𝐱$ to $𝐳$ directly is never longer than going through $𝐲$ . Part (b) says that a side of a triangle is at least the difference of the other two. Read as a statement about the function $𝐯 \mapsto ‖ 𝐯 ‖$ , it says that moving $𝐯$ by a vector of norm $𝛿$ changes $‖ 𝐯 ‖$ by at most $𝛿$ ; in particular, the norm is a continuous function. (Exercise 6.1.61 used exactly this.)

Remark 6.2.15. By Proposition 6.1.27 and Theorem 6.2.11, the norm of an inner product satisfies the three properties of a norm in the sense of Aside (Norms that do not come from inner products), so every inner product space is a normed space. By Corollary 6.2.14(a), the distance

𝑑 (𝐱, 𝐲) = ‖ 𝐱 - 𝐲 ‖

also behaves as a distance should:

𝑑 (𝐱, 𝐲) \geq 0

with equality only for

𝐱 = 𝐲

𝑑 (𝐱, 𝐲) =

𝑑 (𝐲, 𝐱)

, and

𝑑 (𝐱, 𝐳) \leq

𝑑 (𝐱, 𝐲) +

𝑑 (𝐲, 𝐳)

6.2.3Angles

Let $𝑉$ be a real inner product space and $𝐱, 𝐲 \in 𝑉$ nonzero. By Theorem 6.2.1,

- 1 \leq \frac{⟨ 𝐱, 𝐲 ⟩}{‖ 𝐱 ‖ ‖ 𝐲 ‖} \leq 1,

and the cosine is a strictly decreasing continuous function from $[0, 𝜋]$ onto $[- 1, 1]$ , hence a bijection. So the following definition makes sense, and the motivation at the start of the section is justified.

Definition 6.2.16 (Angle). Let $𝑉$ be a real inner product space, and let $𝐱, 𝐲 \in 𝑉$ be nonzero. The angle between $𝐱$ and $𝐲$ is the unique $𝜃 \in [0, 𝜋]$ with

cos 𝜃 = \frac{⟨ 𝐱, 𝐲 ⟩}{‖ 𝐱 ‖ ‖ 𝐲 ‖} .

For the dot product on $ℝ^{2}$ , this is the usual angle, by (6.1.1). The familiar properties of angles follow from what we have proved.

Proposition 6.2.17 (Properties of angles). Let $𝑉$ be a real inner product space, let $𝐱, 𝐲 \in 𝑉$ be nonzero, and let $𝜃$ be the angle between them.

(a)(Law of cosines.)
${‖ 𝐱 - 𝐲 ‖}^{2} = {‖ 𝐱 ‖}^{2} + {‖ 𝐲 ‖}^{2} - 2 ‖ 𝐱 ‖ ‖ 𝐲 ‖ cos 𝜃 .$
(b) $𝜃 = 0$ if and only if $𝐲 = 𝑎 𝐱$ for some $𝑎 > 0$ , and $𝜃 = 𝜋$ if and only if $𝐲 = 𝑎 𝐱$ for some $𝑎 < 0$ .
(c) $𝜃 = 𝜋 / 2$ if and only if $⟨ 𝐱, 𝐲 ⟩ = 0$ .

Proof.routine

Uses. Definition 6.2.16 (Angle), Proposition 6.1.29 (Expanding a norm), Theorem 6.2.11 (Triangle inequality), Proposition 6.1.4 (Sesquilinearity), Proposition 6.1.27 (Basic properties of the norm)

Hint 1. For (a), expand

{‖ 𝐱 - 𝐲 ‖}^{2}

and substitute the definition of

𝜃

Hint 2. For (b),

𝜃 = 0

means

⟨ 𝐱, 𝐲 ⟩ = ‖ 𝐱 ‖ ‖ 𝐲 ‖

. Which result characterizes that equation? For

𝜃 = 𝜋

, replace

𝐲

- 𝐲

Sketch. (a) is Proposition 6.1.29 with

⟨ 𝐱, 𝐲 ⟩ =

‖ 𝐱 ‖ ‖ 𝐲 ‖ cos 𝜃

. In (b),

cos 𝜃 = 1

means

⟨ 𝐱, 𝐲 ⟩ = ‖ 𝐱 ‖ ‖ 𝐲 ‖

, which by Theorem 6.2.11(b) means one vector is a nonnegative multiple of the other; as both are nonzero, the multiple is positive. The case

𝜃 = 𝜋

is the case

𝜃 = 0

for

𝐱

and

- 𝐲

. (c) is

cos 𝜃 = 0

(a) By Proposition 6.1.29, ${‖ 𝐱 - 𝐲 ‖}^{2} =$ ${‖ 𝐱 ‖}^{2} -$ $2 ⟨ 𝐱, 𝐲 ⟩ +$ ${‖ 𝐲 ‖}^{2}$ (over $ℝ$ , $Re ⟨ 𝐱, 𝐲 ⟩ = ⟨ 𝐱, 𝐲 ⟩$ ), and $⟨ 𝐱, 𝐲 ⟩ =$ $‖ 𝐱 ‖ ‖ 𝐲 ‖ cos 𝜃$ by Definition 6.2.16.

(b) Since $cos$ is injective on $[0, 𝜋]$ with $cos 0 = 1$ , we have $𝜃 = 0$ if and only if $⟨ 𝐱, 𝐲 ⟩ = ‖ 𝐱 ‖ ‖ 𝐲 ‖$ . By Theorem 6.2.11(b), this holds if and only if one of $𝐱, 𝐲$ is a nonnegative multiple of the other. As both are nonzero, the multiple is nonzero, hence positive, and if $𝐱 = 𝑏 𝐲$ with $𝑏 > 0$ , then $𝐲 = 𝑏^{- 1} 𝐱$ with $𝑏^{- 1} > 0$ . So $𝜃 = 0$ if and only if $𝐲 = 𝑎 𝐱$ with $𝑎 > 0$ .

Similarly, $𝜃 = 𝜋$ if and only if $⟨ 𝐱, 𝐲 ⟩ = - ‖ 𝐱 ‖ ‖ 𝐲 ‖$ , that is, $⟨ 𝐱, - 𝐲 ⟩ =$ $‖ 𝐱 ‖ ‖ -$ $𝐲 ‖$ , using Proposition 6.1.4 and Proposition 6.1.27. By the first case applied to $𝐱$ and $- 𝐲$ , this holds if and only if $- 𝐲 = 𝑎 𝐱$ with $𝑎 > 0$ , that is, $𝐲 = (- 𝑎) 𝐱$ with $- 𝑎 < 0$ .

(c) $cos 𝜃 = 0$ for $𝜃 \in [0, 𝜋]$ if and only if $𝜃 = 𝜋 / 2$ . □

Part (c) is why perpendicularity is defined by $⟨ 𝐱, 𝐲 ⟩ = 0$ in Section 6.3. In spaces without a picture, the angle still measures how far two vectors are from being proportional, and its values can be surprising.

Example 6.2.18 (Angles in

ℝ^{𝑛}

and in

𝐶 [0, 1]

(a)In $ℝ^{𝑛}$ with the dot product, let $𝐝 = (1, 1, \dots, 1)$ , the diagonal of the unit cube, and $𝐞_{1}$ , one of its edges. Then $⟨ 𝐝, 𝐞_{1} ⟩ = 1$ , $‖ 𝐝 ‖ = \sqrt{𝑛}$ , and $‖ 𝐞_{1} ‖ = 1$ , so the angle $𝜃_{𝑛}$ between them satisfies $cos 𝜃_{𝑛} = 1 / \sqrt{𝑛}$ . For $𝑛 = 2, 3, 4$ this gives $𝜃_{𝑛} = 𝜋 / 4$ , about $54.7 °$ , and $𝜋 / 3$ . As $𝑛 \to \infty$ , $cos 𝜃_{𝑛} \to 0$ and $𝜃_{𝑛} \to 𝜋 / 2$ : in high dimension the diagonal of a cube is almost perpendicular to every edge.
(b)In $𝐶 [0, 1]$ with the integral inner product, $⟨ 1, 𝑡 ⟩ =$ $\int_{0}^{1} 𝑡 d 𝑡 =$ $1 / 2$ , $‖ 1 ‖ = 1$ , and $‖ 𝑡 ‖ =$ ${(\int_{0}^{1} 𝑡^{2} d 𝑡)}^{1 / 2} =$ $1 / \sqrt{3}$ . So the angle between the functions $1$ and $𝑡$ satisfies $cos 𝜃 = \sqrt{3} / 2$ , that is, $𝜃 = 𝜋 / 6$ .

Example 6.2.19 (Correlation is a cosine). Given data points $(𝑥_{1}, 𝑦_{1}),$ $\dots,$ $(𝑥_{𝑚}, 𝑦_{𝑚}) \in$ $ℝ^{2}$ , let $𝑚_{𝑥}$ and $𝑚_{𝑦}$ be the means of the $𝑥_{𝑘}$ and of the $𝑦_{𝑘}$ , and let

\tilde{𝐱} = (𝑥_{1} - 𝑚_{𝑥}, \dots, 𝑥_{𝑚} - 𝑚_{𝑥}), \tilde{𝐲} = (𝑦_{1} - 𝑚_{𝑦}, \dots, 𝑦_{𝑚} - 𝑚_{𝑦})

be the centered data in $ℝ^{𝑚}$ . If both are nonzero, the correlation coefficient of the data is

𝑟 = \frac{\tilde{𝐱} \cdot \tilde{𝐲}}{‖ \tilde{𝐱} ‖ ‖ \tilde{𝐲} ‖},

the cosine of the angle between $\tilde{𝐱}$ and $\tilde{𝐲}$ . Cauchy–Schwarz says $- 1 \leq 𝑟 \leq 1$ , and Proposition 6.2.17(b) says that $𝑟 = \pm 1$ exactly when $\tilde{𝐲} = 𝑎 \tilde{𝐱}$ with $𝑎$ of the same sign as $𝑟$ , that is, when all data points lie on the line $𝑦 =$ $𝑚_{𝑦} +$ $𝑎 (𝑥 - 𝑚_{𝑥})$ . For the points $(0, 1)$ , $(1, 1)$ , $(2, 4)$ , we get $\tilde{𝐱} = (- 1, 0, 1)$ and $\tilde{𝐲} = (- 1, - 1, 2)$ , so $𝑟 = 3 / (\sqrt{2} \sqrt{6}) = \sqrt{3} / 2$ . Least squares (Section 6.5) finds the line that comes closest to such data.

Exercise 6.2.20.routine In

𝐶 [0, 1]

with the integral inner product, find

cos 𝜃

for the angle

𝜃

between

𝑡

and

𝑡^{2}

. Then find all

𝑎 \in ℝ

for which the angle between the functions

1

and

𝑡 + 𝑎

𝜋 / 3

Solution.

Uses. Definition 6.2.16 (Angle), Proposition 6.1.13 (Integral inner products)

Hint 1.

⟨ 𝑡, 𝑡^{2} ⟩ =

\int_{0}^{1} 𝑡^{3} d 𝑡

, and

{‖ 𝑡^{𝑘} ‖}^{2} =

\int_{0}^{1} 𝑡^{2 𝑘} d 𝑡

Hint 2. For the second part, the condition is

⟨ 1, 𝑡 + 𝑎 ⟩ =

(1 / 2) ‖ 1 ‖ ‖ 𝑡 +

𝑎 ‖

. The left side must be positive. Square and solve.

Sketch.

cos 𝜃 =

(1 / 4) / (\sqrt{1 / 3} \sqrt{1 / 5}) =

\sqrt{15} / 4

. For the second part,

1 / 2 +

𝑎 =

(1 / 2) \sqrt{1 / 3 + 𝑎 + 𝑎^{2}}

with

1 / 2 + 𝑎 > 0

; squaring gives

9 𝑎^{2} + 9 𝑎 + 2 = 0

, so

𝑎 = - 1 / 3

𝑎 = - 2 / 3

, and only

𝑎 = - 1 / 3

has

1 / 2 + 𝑎 > 0

We have $⟨ 𝑡, 𝑡^{2} ⟩ =$ $\int_{0}^{1} 𝑡^{3} d 𝑡 =$ $1 / 4$ , ${‖ 𝑡 ‖}^{2} = 1 / 3$ , and ${‖ 𝑡^{2} ‖}^{2} =$ $\int_{0}^{1} 𝑡^{4} d 𝑡 =$ $1 / 5$ . So

cos 𝜃 = \frac{1 / 4}{\sqrt{\frac{1}{3} \cdot \frac{1}{5}}} = \frac{\sqrt{15}}{4} \approx 0.968 .

For the second part, $𝑡 + 𝑎 \neq 0$ for every $𝑎$ , $‖ 1 ‖ = 1$ , $⟨ 1, 𝑡 + 𝑎 ⟩ = 1 / 2 + 𝑎$ , and

{‖ 𝑡 + 𝑎 ‖}^{2} = \int_{0}^{1} (𝑡^{2} + 2 𝑎 𝑡 + 𝑎^{2}) d 𝑡 = \frac{1}{3} + 𝑎 + 𝑎^{2} .

The angle is $𝜋 / 3$ if and only if $1 / 2 +$ $𝑎 =$ $(1 / 2) \sqrt{1 / 3 + 𝑎 + 𝑎^{2}}$ . This requires $1 / 2 + 𝑎 \geq 0$ , and then it is equivalent to the squared equation $4 {(1 / 2 + 𝑎)}^{2} =$ $1 / 3 +$ $𝑎 +$ $𝑎^{2}$ , that is, $3 𝑎^{2} + 3 𝑎 + 2 / 3 = 0$ , or $9 𝑎^{2} + 9 𝑎 + 2 = 0$ . Its roots are $𝑎 = - 1 / 3$ and $𝑎 = - 2 / 3$ . Only $𝑎 = - 1 / 3$ satisfies $1 / 2 + 𝑎 \geq 0$ ; for $𝑎 = - 2 / 3$ the angle is $2 𝜋 / 3$ . Check: ${‖ 𝑡 - 1 / 3 ‖}^{2} =$ $1 / 3 -$ $1 / 3 +$ $1 / 9 =$ $1 / 9$ , so $cos 𝜃 =$ $(1 / 6) / (1 / 3) =$ $1 / 2$ .

Over $ℂ$

Let $𝑉$ be a complex inner product space and $𝐱, 𝐲 \in 𝑉$ nonzero. Now $⟨ 𝐱, 𝐲 ⟩$ is a complex number, and there are two natural real candidates for a cosine:

Re \frac{⟨ 𝐱, 𝐲 ⟩}{‖ 𝐱 ‖ ‖ 𝐲 ‖} \in [- 1, 1] and \frac{| ⟨ 𝐱, 𝐲 ⟩ |}{‖ 𝐱 ‖ ‖ 𝐲 ‖} \in [0, 1] .

Both lie in the stated intervals by Theorem 6.2.1, since $| Re 𝑧 | \leq | 𝑧 |$ . They measure different things.

The first is the cosine of an honest real angle. Regard $𝑉$ as a real vector space, forgetting multiplication by non-real scalars. Then $Re ⟨ \cdot, \cdot ⟩$ is a real inner product on it: it is symmetric because $Re 𝑧 = Re 𝑧$ , linear in the first slot for real scalars, and $Re ⟨ 𝐱, 𝐱 ⟩ = {‖ 𝐱 ‖}^{2}$ . For $𝑉 = ℂ^{𝑛}$ this is the dot product of $ℝ^{2 𝑛}$ , and it is the inner product that appears in the expansion ${‖ 𝐱 - 𝐲 ‖}^{2} =$ ${‖ 𝐱 ‖}^{2} +$ ${‖ 𝐲 ‖}^{2} -$ $2 Re ⟨ 𝐱, 𝐲 ⟩$ . But it does not see complex scalars: $Re ⟨ 𝐱, 𝑖 𝐱 ⟩ =$ $Re (- 𝑖 {‖ 𝐱 ‖}^{2}) =$ $0$ , so $𝐱$ and $𝑖 𝐱$ are perpendicular in this sense, although $𝑖 𝐱$ is a multiple of $𝐱$ .

The second ignores phases. By Technique 6.2.8, $| ⟨ 𝐱, 𝐲 ⟩ | =$ ${max}_{| 𝜆 | = 1} Re ⟨ 𝜆 𝐱, 𝐲 ⟩$ , so it is the cosine of the smallest real angle between $𝐲$ and a unit-modulus multiple $𝜆 𝐱$ . It does not change when $𝐱$ and $𝐲$ are multiplied by arbitrary nonzero scalars (by Proposition 6.1.27, $| ⟨ 𝑎 𝐱, 𝑏 𝐲 ⟩ | =$ $| 𝑎 | | 𝑏 | | ⟨ 𝐱, 𝐲 ⟩ |$ and the norms scale the same way), so it depends only on the complex lines $ℂ 𝐱$ and $ℂ 𝐲$ , and it is really an angle between them. By Theorem 6.2.1(b) it equals $1$ exactly when these lines coincide.

Neither number is “the” angle, and we do not define one over $ℂ$ . What both notions agree on is the right angle in the strong sense: $⟨ 𝐱, 𝐲 ⟩ = 0$ if and only if $Re ⟨ 𝜆 𝐱, 𝐲 ⟩ = 0$ for every $𝜆$ with $| 𝜆 | = 1$ , that is, $𝐲$ is perpendicular, in the real sense, to every unit-modulus multiple of $𝐱$ . (For the “if”, take $𝜆 = 𝜇$ as in Technique 6.2.8.) This is the notion of orthogonality used from Section 6.3 on.

6.2.4Choosing the inner product

Every inner product gives an instance of Theorem 6.2.1, and Section 6.1.3 has many, so the theorem is a machine for producing inequalities between numbers. Using it well is a skill of its own.

Technique 6.2.21 (Apply Cauchy–Schwarz to a well-chosen inner product). To prove an inequality between sums or integrals, recognize one side as $| ⟨ 𝐱, 𝐲 ⟩ |$ and the other as $‖ 𝐱 ‖ ‖ 𝐲 ‖$ for some inner product. The work is in the choices. Pick the space and the inner product (the dot product, a weighted dot product, an integral, the Frobenius inner product), and split each term of the quantity to be bounded as a product of two factors, one belonging to $𝐱$ and one to $𝐲$ . A useful move is to insert weights: write $𝑎_{𝑘} 𝑏_{𝑘}$ as $(\sqrt{𝑤_{𝑘}} 𝑎_{𝑘}) (𝑏_{𝑘} / \sqrt{𝑤_{𝑘}})$ with $𝑤_{𝑘} > 0$ chosen so that the resulting norms are quantities you control.

Then check the equality case. Equality holds exactly when one of $𝐱, 𝐲$ is a multiple of the other, so it tells you whether the bound is sharp and which data attain it. If the data that would attain it violate your constraints, the bound is not sharp, and a different split may do better.

Reach for it when the target has the shape ${(\sum_{𝑘} 𝑎_{𝑘} 𝑏_{𝑘})}^{2} \leq$ $(\sum_{𝑘} 𝑎_{𝑘}^{2}) (\sum_{𝑘} 𝑏_{𝑘}^{2})$ after renaming, when a linear quantity such as $\sum_{𝑘} 𝑎_{𝑘}$ or $\int 𝑓$ must be bounded by a quadratic one such as $\sum_{𝑘} 𝑎_{𝑘}^{2}$ or $\int 𝑓^{2}$ , or when a quadratic quantity must be minimized subject to a linear constraint. The canonical example is Proposition 6.2.22.

Used in 21 places and 9 practice exercises

Proposition 6.2.22, Proposition 6.2.24, Corollary 6.2.26, Exercise 6.2.31, Exercise 6.2.32, Exercise 6.2.36, Exercise 6.2.37, Exercise 6.2.38, Exercise 6.2.39, Exercise 6.2.46, Exercise 6.2.47, Exercise 6.3.53, Exercise 6.4.52, Exercise 6.5.50, Exercise 6.6.49, Problem 6.15, Problem 6.16, Exercise 8.2.43, Exercise 8.3.32, Exercise 8.3.33, Exercise 8.4.36; 9 practice exercises.

Proposition 6.2.22 (The square of a sum). For $𝑎_{1}, \dots, 𝑎_{𝑛} \in 𝔽$ ,

{| 𝑎_{1} + \dots + 𝑎_{𝑛} |}^{2} \leq 𝑛 ({| 𝑎_{1} |}^{2} + \dots + {| 𝑎_{𝑛} |}^{2}),

with equality if and only if $𝑎_{1} = 𝑎_{2} = \dots = 𝑎_{𝑛}$ .

Proof.routine

Uses. Theorem 6.2.1 (Cauchy–Schwarz inequality), Proposition 6.1.10 (The dot product is an inner product)

Techniques. Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1. A sum

𝑎_{1} + \dots + 𝑎_{𝑛}

is an inner product of

(𝑎_{1}, \dots, 𝑎_{𝑛})

with a fixed vector. Which one?

Hint 2. Take

𝟏 = (1, \dots, 1)

. Then

𝐚 \cdot 𝟏 = \sum_{𝑘} 𝑎_{𝑘}

and

{‖ 𝟏 ‖}^{2} = 𝑛

. For the equality case,

𝟏 \neq 𝟎

Sketch. Apply Cauchy–Schwarz to

𝐚 = (𝑎_{1}, \dots, 𝑎_{𝑛})

and

𝟏 = (1, \dots, 1)

𝔽^{𝑛}

{| 𝐚 \cdot 𝟏 |}^{2} \leq

{‖ 𝐚 ‖}^{2} {‖ 𝟏 ‖}^{2}

is the claim. Equality holds if and only if

𝐚

is a multiple of

𝟏

, that is, all

𝑎_{𝑘}

are equal.

Let $𝐚 = (𝑎_{1}, \dots, 𝑎_{𝑛})$ and $𝟏 = (1, \dots, 1)$ in $𝔽^{𝑛}$ , with the dot product. Then $𝐚 \cdot$ $𝟏 =$ $\sum_{𝑘} 𝑎_{𝑘} 1 =$ $\sum_{𝑘} 𝑎_{𝑘}$ , ${‖ 𝟏 ‖}^{2} = 𝑛$ , and ${‖ 𝐚 ‖}^{2} = \sum_{𝑘} {| 𝑎_{𝑘} |}^{2}$ by Proposition 6.1.10. By Theorem 6.2.1,

{| \sum_{𝑘} 𝑎_{𝑘} |}^{2} = {| 𝐚 \cdot 𝟏 |}^{2} \leq {‖ 𝐚 ‖}^{2} {‖ 𝟏 ‖}^{2} = 𝑛 \sum_{𝑘} {| 𝑎_{𝑘} |}^{2} .

By Theorem 6.2.1(b), equality holds if and only if one of $𝐚, 𝟏$ is a multiple of the other. Since $𝟏 \neq 𝟎$ , this means $𝐚 = 𝑐 𝟏$ for some $𝑐 \in 𝔽$ (if $𝟏 = 𝑏 𝐚$ , then $𝑏 \neq 0$ and $𝐚 = 𝑏^{- 1} 𝟏$ ). And $𝐚 = 𝑐 𝟏$ says that every $𝑎_{𝑘}$ equals $𝑐$ . □

Corollary 6.2.23 (Arithmetic mean and quadratic mean). For $𝑎_{1}, \dots, 𝑎_{𝑛} \in ℝ$ ,

\frac{𝑎_{1} + \dots + 𝑎_{𝑛}}{𝑛} \leq \sqrt{\frac{𝑎_{1}^{2} + \dots + 𝑎_{𝑛}^{2}}{𝑛}},

with equality if and only if $𝑎_{1} = \dots = 𝑎_{𝑛} \geq 0$ .

Proof.routine

Uses. Proposition 6.2.22 (The square of a sum)

Hint 1. Bound the left side by its absolute value, then use Proposition 6.2.22.

Hint 2. For equality, both steps must be equalities: when is

𝑟 = | 𝑟 |

Sketch.

\sum_{𝑘} 𝑎_{𝑘} / 𝑛 \leq

| \sum_{𝑘} 𝑎_{𝑘} | / 𝑛 \leq

\sqrt{𝑛 \sum_{𝑘} 𝑎_{𝑘}^{2}} / 𝑛

, and the right side is the quadratic mean. Equality requires

\sum_{𝑘} 𝑎_{𝑘} \geq 0

and all

𝑎_{𝑘}

equal, hence all

𝑎_{𝑘}

equal to a common value

𝑐 \geq 0

By Proposition 6.2.22,

\frac{\sum_{𝑘} 𝑎_{𝑘}}{𝑛} \leq \frac{| \sum_{𝑘} 𝑎_{𝑘} |}{𝑛} \leq \frac{\sqrt{𝑛 \sum_{𝑘} 𝑎_{𝑘}^{2}}}{𝑛} = \sqrt{\frac{\sum_{𝑘} 𝑎_{𝑘}^{2}}{𝑛}} .

Equality holds if and only if both steps are equalities, that is, $\sum_{𝑘} 𝑎_{𝑘} \geq 0$ and (by Proposition 6.2.22) all $𝑎_{𝑘}$ are equal to some $𝑐$ . Together these say that all $𝑎_{𝑘}$ equal $𝑐$ with $𝑛 𝑐 \geq 0$ , that is, $𝑐 \geq 0$ . □

Intuition (Mean plus deviation). The proof applied Cauchy–Schwarz with $𝐲 = 𝟏$ , and the first proof of Theorem 6.2.1 computes the gap. Here $𝑐 =$ $(𝐚 \cdot 𝟏) / {‖ 𝟏 ‖}^{2} =$ $(𝑎_{1} + \dots + 𝑎_{𝑛}) / 𝑛$ is the mean $𝑚$ of the $𝑎_{𝑘}$ , the shadow $𝑚 𝟏$ is the constant vector with that mean, and $𝐰 = 𝐚 - 𝑚 𝟏$ is the vector of deviations from the mean. So (6.2.1) reads

\sum_{𝑘 = 1}^{𝑛} {| 𝑎_{𝑘} |}^{2} = 𝑛 {| 𝑚 |}^{2} + \sum_{𝑘 = 1}^{𝑛} {| 𝑎_{𝑘} - 𝑚 |}^{2} .

Dividing by $𝑛$ : the mean of the squares is the square of the mean plus the variance $(1 / 𝑛) \sum_{𝑘} {| 𝑎_{𝑘} - 𝑚 |}^{2}$ . So the gap in the squared inequality ${| 𝑚 |}^{2} \leq$ $(1 / 𝑛) \sum_{𝑘} {| 𝑎_{𝑘} |}^{2}$ is exactly the variance, and it vanishes exactly for constant data.

The same choice of $𝐲$ works for integrals. The following inequality is used constantly in analysis.

Proposition 6.2.24 ( $\int | 𝑓 |$ versus $\int {| 𝑓 |}^{2}$ ). Let $𝑎 < 𝑏$ and $𝑓 \in 𝐶 ([𝑎, 𝑏], ℂ)$ . Then

{(\int_{𝑎}^{𝑏} | 𝑓 (𝑡) | d 𝑡)}^{2} \leq (𝑏 - 𝑎) \int_{𝑎}^{𝑏} {| 𝑓 (𝑡) |}^{2} d 𝑡,

with equality if and only if $| 𝑓 |$ is constant. In particular, ${(\int_{0}^{1} | 𝑓 |)}^{2} \leq$ $\int_{0}^{1} {| 𝑓 |}^{2}$ .

Proof.routine

Uses. Theorem 6.2.1 (Cauchy–Schwarz inequality), Proposition 6.1.13 (Integral inner products)

Techniques. Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1. Work in the real space

𝐶 [𝑎, 𝑏]

with

⟨ 𝑔, ℎ ⟩ = \int_{𝑎}^{𝑏} 𝑔 ℎ

, applied to the real function

𝑔 = | 𝑓 |

Hint 2.

\int_{𝑎}^{𝑏} | 𝑓 | =

⟨ | 𝑓 |, 1 ⟩

, and

{‖ 1 ‖}^{2} = 𝑏 - 𝑎

Sketch. Apply Cauchy–Schwarz in

𝐶 [𝑎, 𝑏]

𝑔 = | 𝑓 |

and the constant function

1

{⟨ 𝑔, 1 ⟩}^{2} \leq {‖ 𝑔 ‖}^{2} {‖ 1 ‖}^{2}

is the claim. Equality holds if and only if

𝑔

is a multiple of

1

, that is,

| 𝑓 |

is constant.

Write $𝑓 = 𝑢 + 𝑖 𝑣$ with $𝑢, 𝑣 \in 𝐶 [𝑎, 𝑏]$ . Then $𝑔 = | 𝑓 | = \sqrt{𝑢^{2} + 𝑣^{2}}$ is continuous and real-valued, so $𝑔 \in 𝐶 [𝑎, 𝑏]$ . Give $𝐶 [𝑎, 𝑏]$ the integral inner product $⟨ 𝑔, ℎ ⟩ = \int_{𝑎}^{𝑏} 𝑔 ℎ$ (Proposition 6.1.13 and the paragraph after it), and let $1$ denote the constant function. Then

⟨ 𝑔, 1 ⟩ = \int_{𝑎}^{𝑏} | 𝑓 |, {‖ 𝑔 ‖}^{2} = \int_{𝑎}^{𝑏} {| 𝑓 |}^{2}, {‖ 1 ‖}^{2} = 𝑏 - 𝑎,

and Theorem 6.2.1 gives ${⟨ 𝑔, 1 ⟩}^{2} \leq {‖ 𝑔 ‖}^{2} {‖ 1 ‖}^{2}$ , which is the inequality. By Theorem 6.2.1(b), equality holds if and only if one of $𝑔, 1$ is a multiple of the other; since $1 \neq 𝟎$ , this means $𝑔 = 𝑐 \cdot 1$ for some $𝑐 \in ℝ$ , that is, $| 𝑓 |$ is constant. □

Weights let the same idea handle unequal coefficients. The next example also shows the equality case doing real work: it finds the minimizer.

Example 6.2.25 (A constrained minimum). We find the minimum of $𝑎^{2} + 2 𝑏^{2} + 3 𝑐^{2}$ over all $(𝑎, 𝑏, 𝑐) \in ℝ^{3}$ with $𝑎 + 𝑏 + 𝑐 = 1$ . The quantity to be minimized is ${‖ 𝐯 ‖}^{2}$ for $𝐯 = (𝑎, 𝑏, 𝑐)$ in the weighted inner product ${⟨ 𝐱, 𝐲 ⟩}_{𝑤} =$ $𝑥_{1} 𝑦_{1} +$ $2 𝑥_{2} 𝑦_{2} +$ $3 𝑥_{3} 𝑦_{3}$ (Example 6.1.21). So we write the constraint as an inner product in the same space: $𝑎 +$ $𝑏 +$ $𝑐 =$ ${⟨ 𝐯, 𝐳 ⟩}_{𝑤}$ for $𝐳 = (1, 1 / 2, 1 / 3)$ ; the weights $2$ and $3$ cancel the entries $1 / 2$ and $1 / 3$ . Since

{‖ 𝐳 ‖}_{𝑤}^{2} = 1 + 2 \cdot \frac{1}{4} + 3 \cdot \frac{1}{9} = \frac{11}{6},

Cauchy–Schwarz gives $1 =$ ${⟨ 𝐯, 𝐳 ⟩}_{𝑤}^{2} \leq$ ${‖ 𝐯 ‖}_{𝑤}^{2} \cdot$ $11 / 6$ , that is,

𝑎^{2} + 2 𝑏^{2} + 3 𝑐^{2} \geq \frac{6}{11} .

Equality requires $𝐯 = 𝑡 𝐳$ , and the constraint then gives $𝑡 \cdot 11 / 6 = 1$ , so $𝑡 = 6 / 11$ and $(𝑎, 𝑏, 𝑐) =$ $(6 / 11, 3 / 11, 2 / 11)$ . This point satisfies the constraint, and indeed

{(\frac{6}{11})}^{2} + 2 {(\frac{3}{11})}^{2} + 3 {(\frac{2}{11})}^{2} = \frac{36 + 18 + 12}{121} = \frac{6}{11} .

So the minimum is $6 / 11$ . Geometrically, we found the point of the plane $𝑎 + 𝑏 + 𝑐 = 1$ closest to $𝟎$ in the geometry of ${⟨ \cdot, \cdot ⟩}_{𝑤}$ .

A variant of Cauchy–Schwarz with weights in denominators is a standard tool in competition problems.

Corollary 6.2.26 (Engel’s form). Let $𝑎_{1}, \dots, 𝑎_{𝑛} \in ℝ$ and $𝑏_{1}, \dots, 𝑏_{𝑛} > 0$ . Then

\frac{𝑎_{1}^{2}}{𝑏_{1}} + \dots + \frac{𝑎_{𝑛}^{2}}{𝑏_{𝑛}} \geq \frac{{(𝑎_{1} + \dots + 𝑎_{𝑛})}^{2}}{𝑏_{1} + \dots + 𝑏_{𝑛}},

with equality if and only if $𝑎_{1} / 𝑏_{1} =$ $𝑎_{2} / 𝑏_{2} =$ $\dots =$ $𝑎_{𝑛} / 𝑏_{𝑛}$ .

Proof.routine

Uses. Theorem 6.2.1 (Cauchy–Schwarz inequality), Proposition 6.1.10 (The dot product is an inner product)

Techniques. Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1. Write

𝑎_{𝑘} =

(𝑎_{𝑘} / \sqrt{𝑏_{𝑘}}) \cdot

\sqrt{𝑏_{𝑘}}

Hint 2. Apply Cauchy–Schwarz in

ℝ^{𝑛}

𝐱 = {(𝑎_{𝑘} / \sqrt{𝑏_{𝑘}})}_{𝑘}

and

𝐲 = {(\sqrt{𝑏_{𝑘}})}_{𝑘}

Sketch. With

𝐱 = {(𝑎_{𝑘} / \sqrt{𝑏_{𝑘}})}_{𝑘}

and

𝐲 = {(\sqrt{𝑏_{𝑘}})}_{𝑘}

, we have

𝐱 \cdot 𝐲 = \sum_{𝑘} 𝑎_{𝑘}

{‖ 𝐱 ‖}^{2} =

\sum_{𝑘} 𝑎_{𝑘}^{2} / 𝑏_{𝑘}

, and

{‖ 𝐲 ‖}^{2} = \sum_{𝑘} 𝑏_{𝑘}

. Cauchy–Schwarz gives the claim after dividing by

\sum_{𝑘} 𝑏_{𝑘}

. Equality holds if and only if

𝐱 = 𝑐 𝐲

, that is,

𝑎_{𝑘} = 𝑐 𝑏_{𝑘}

for all

𝑘

Let $𝐱 =$ $(𝑎_{1} / \sqrt{𝑏_{1}}, \dots, 𝑎_{𝑛} / \sqrt{𝑏_{𝑛}})$ and $𝐲 = (\sqrt{𝑏_{1}}, \dots, \sqrt{𝑏_{𝑛}})$ in $ℝ^{𝑛}$ . Then

𝐱 \cdot 𝐲 = \sum_{𝑘} 𝑎_{𝑘}, {‖ 𝐱 ‖}^{2} = \sum_{𝑘} \frac{𝑎_{𝑘}^{2}}{𝑏_{𝑘}}, {‖ 𝐲 ‖}^{2} = \sum_{𝑘} 𝑏_{𝑘} > 0 .

By Theorem 6.2.1, ${(\sum_{𝑘} 𝑎_{𝑘})}^{2} \leq$ $(\sum_{𝑘} 𝑎_{𝑘}^{2} / 𝑏_{𝑘}) (\sum_{𝑘} 𝑏_{𝑘})$ , and dividing by $\sum_{𝑘} 𝑏_{𝑘}$ gives the inequality. Since $𝐲 \neq 𝟎$ , equality holds if and only if $𝐱 = 𝑐 𝐲$ for some $𝑐 \in ℝ$ (as in the proof of Proposition 6.2.22), that is, $𝑎_{𝑘} / \sqrt{𝑏_{𝑘}} = 𝑐 \sqrt{𝑏_{𝑘}}$ , or $𝑎_{𝑘} / 𝑏_{𝑘} = 𝑐$ , for every $𝑘$ . □

For instance, for positive reals $𝑎, 𝑏, 𝑐$ , Corollary 6.2.26 with numerators $𝑎, 𝑏, 𝑐$ and denominators $𝑏, 𝑐, 𝑎$ gives

\frac{𝑎^{2}}{𝑏} + \frac{𝑏^{2}}{𝑐} + \frac{𝑐^{2}}{𝑎} \geq \frac{{(𝑎 + 𝑏 + 𝑐)}^{2}}{𝑏 + 𝑐 + 𝑎} = 𝑎 + 𝑏 + 𝑐 .

Equality requires $𝑎 / 𝑏 = 𝑏 / 𝑐 = 𝑐 / 𝑎$ . The product of these three ratios is $1$ , so the common value is $1$ , and $𝑎 = 𝑏 = 𝑐$ .

Exercises

Exercise 6.2.27.routine Give $ℝ^{3}$ the inner product $⟨ 𝐱, 𝐲 ⟩ =$ $𝑥_{1} 𝑦_{1} +$ $2 𝑥_{2} 𝑦_{2} +$ $3 𝑥_{3} 𝑦_{3}$ (Example 6.1.21), and let $𝐱 = (1, 1, 1)$ and $𝐲 = (1, - 1, 2)$ .

(a)Verify the Cauchy–Schwarz inequality for $𝐱$ and $𝐲$ .
(b)Find $𝑐 \in ℝ$ and $𝐰 \in ℝ^{3}$ with $𝐱 = 𝑐 𝐲 + 𝐰$ and $⟨ 𝐰, 𝐲 ⟩ = 0$ , and check (6.2.1).
(c)Find the cosine of the angle between $𝐱$ and $𝐲$ in this inner product, and compare it with the cosine for the dot product.

Solution.

Uses. Theorem 6.2.1 (Cauchy–Schwarz inequality), (6.2.1), Definition 6.2.16 (Angle)

Techniques. Orthogonal decomposition (Technique 6.2.4)

Hint 1. Every inner product in this exercise carries the weights

1, 2, 3

. For instance

{‖ 𝐲 ‖}^{2} = 1 + 2 + 3 \cdot 4

Hint 2.

𝑐 = ⟨ 𝐱, 𝐲 ⟩ / {‖ 𝐲 ‖}^{2}

Sketch.

⟨ 𝐱, 𝐲 ⟩ = 5

{‖ 𝐱 ‖}^{2} = 6

{‖ 𝐲 ‖}^{2} = 15

, and

25 \leq 90

. Then

𝑐 = 1 / 3

𝐰 = (2 / 3, 4 / 3, 1 / 3)

{‖ 𝐰 ‖}^{2} = 13 / 3

, and

13 / 3 + 25 / 15 = 6

. The cosine is

5 / \sqrt{90} = \sqrt{10} / 6

; for the dot product it is

2 / \sqrt{18} = \sqrt{2} / 3

(a) $⟨ 𝐱, 𝐲 ⟩ = 1 - 2 + 6 = 5$ , ${‖ 𝐱 ‖}^{2} = 1 + 2 + 3 = 6$ , and ${‖ 𝐲 ‖}^{2} = 1 + 2 + 12 = 15$ . Indeed $5^{2} = 25 \leq 6 \cdot 15 = 90$ .

(b) By Technique 6.2.4, $𝑐 = 5 / 15 = 1 / 3$ and $𝐰 =$ $𝐱 -$ $𝐲 / 3 =$ $(2 / 3, 4 / 3, 1 / 3)$ . Check:

⟨ 𝐰, 𝐲 ⟩ = \frac{2}{3} - 2 \cdot \frac{4}{3} + 3 \cdot \frac{2}{3} = 0 .

Also

{‖ 𝐰 ‖}^{2} = \frac{4}{9} + 2 \cdot \frac{16}{9} + 3 \cdot \frac{1}{9} = \frac{39}{9} = \frac{13}{3},

and

{‖ 𝐰 ‖}^{2} + \frac{{⟨ 𝐱, 𝐲 ⟩}^{2}}{{‖ 𝐲 ‖}^{2}} = \frac{13}{3} + \frac{25}{15} = 6 = {‖ 𝐱 ‖}^{2},

as (6.2.1) predicts.

(c) $cos 𝜃 =$ $5 / \sqrt{90} =$ $5 / (3 \sqrt{10}) =$ $\sqrt{10} / 6 \approx$ $0.527$ . For the dot product, $𝐱 \cdot 𝐲 = 2$ , ${‖ 𝐱 ‖}^{2} = 3$ , ${‖ 𝐲 ‖}^{2} = 6$ , and the cosine is $2 / \sqrt{18} = \sqrt{2} / 3 \approx 0.471$ . The angle depends on the inner product.

Exercise 6.2.28.routine In $ℂ^{2}$ with the dot product, let $𝐱 = (1, 𝑖)$ and $𝐲 = (𝑧, 1)$ with $𝑧 \in ℂ$ .

(a)For which $𝑧$ does equality hold in the Cauchy–Schwarz inequality for $𝐱, 𝐲$ ?
(b)For which $𝑧$ does $‖ 𝐱 + 𝐲 ‖ = ‖ 𝐱 ‖ + ‖ 𝐲 ‖$ hold?

Solution.

Uses. Theorem 6.2.1 (Cauchy–Schwarz inequality), Theorem 6.2.11 (Triangle inequality)

Hint 1. Both vectors are nonzero for every

𝑧

. By Theorem 6.2.1(b), equality means

𝐲 = 𝑎 𝐱

for some

𝑎 \in ℂ

. Compare second coordinates first.

Hint 2. For (b), Theorem 6.2.11(b) requires a nonnegative real multiple. Is the multiple from (a) real?

Sketch. (a)

𝐲 = 𝑎 𝐱

forces

1 = 𝑎 𝑖

, so

𝑎 = - 𝑖

and

𝑧 = - 𝑖

. (b) The only dependent case has

𝐲 = - 𝑖 𝐱

, which is not a nonnegative real multiple, so there is no such

𝑧

; indeed

‖ 𝐱 + 𝐲 ‖ = 2 < 2 \sqrt{2}

for

𝑧 = - 𝑖

(a) Both $𝐱$ and $𝐲$ are nonzero, so by Theorem 6.2.1(b) equality holds if and only if $𝐲 = 𝑎 𝐱$ for some $𝑎 \in ℂ$ (a relation $𝐱 = 𝑏 𝐲$ has $𝑏 \neq 0$ and can be inverted). Comparing second coordinates, $1 = 𝑎 𝑖$ , so $𝑎 = - 𝑖$ , and then $𝑧 = 𝑎 \cdot 1 = - 𝑖$ . Conversely, $𝑧 = - 𝑖$ gives $𝐲 = - 𝑖 𝐱$ . Check: $𝐱 \cdot$ $𝐲 =$ $1 \cdot$ $- 𝑖 +$ $𝑖 \cdot$ $1 =$ $2 𝑖$ , and $| 2 𝑖 | =$ $2 =$ $\sqrt{2} \sqrt{2} =$ $‖ 𝐱 ‖ ‖ 𝐲 ‖$ .

(b) By Theorem 6.2.11(b), equality requires one of $𝐱, 𝐲$ to be a nonnegative real multiple of the other, so in particular equality in Cauchy–Schwarz. By (a), this forces $𝑧 = - 𝑖$ and $𝐲 = - 𝑖 𝐱$ , equivalently $𝐱 = 𝑖 𝐲$ , and neither $- 𝑖$ nor $𝑖$ is a nonnegative real number. So there is no such $𝑧$ . For $𝑧 = - 𝑖$ directly: $𝐱 + 𝐲 = (1 - 𝑖, 1 + 𝑖)$ has norm $\sqrt{2 + 2} = 2 < 2 \sqrt{2}$ .

Exercise 6.2.29.routine In $ℂ^{2}$ with the dot product, let $𝐱 = (1, 1)$ and $𝐲 = (1, 𝑖)$ .

(a)Compute $Re ⟨ 𝐱, 𝐲 ⟩ / (‖ 𝐱 ‖ ‖ 𝐲 ‖)$ and $| ⟨ 𝐱, 𝐲 ⟩ | / (‖ 𝐱 ‖ ‖ 𝐲 ‖)$ , and the corresponding angles in $[0, 𝜋]$ .
(b)Find $𝜆 \in ℂ$ with $| 𝜆 | = 1$ that maximizes $Re ⟨ 𝜆 𝐱, 𝐲 ⟩$ , and the maximum value.

Solution.

Uses. Definition 6.1.8 (Dot product)

Techniques. Rotate the phase (Technique 6.2.8)

Hint 1.

⟨ 𝐱, 𝐲 ⟩ = 1 \cdot 1 + 1 \cdot 𝑖

Hint 2. For (b),

Re ⟨ 𝜆 𝐱, 𝐲 ⟩ =

Re (𝜆 ⟨ 𝐱, 𝐲 ⟩)

is largest when

𝜆

rotates

⟨ 𝐱, 𝐲 ⟩

onto the positive real axis.

Sketch.

⟨ 𝐱, 𝐲 ⟩ = 1 - 𝑖

and

‖ 𝐱 ‖ = ‖ 𝐲 ‖ = \sqrt{2}

. The two ratios are

1 / 2

and

\sqrt{2} / 2

, giving angles

𝜋 / 3

and

𝜋 / 4

. In (b),

𝜆 = (1 + 𝑖) / \sqrt{2}

and the maximum is

\sqrt{2} = | 1 - 𝑖 |

(a) $⟨ 𝐱, 𝐲 ⟩ = 1 + 𝑖 = 1 - 𝑖$ , and ${‖ 𝐱 ‖}^{2} = {‖ 𝐲 ‖}^{2} = 2$ . So

\frac{Re ⟨ 𝐱, 𝐲 ⟩}{‖ 𝐱 ‖ ‖ 𝐲 ‖} = \frac{1}{2}, \frac{| ⟨ 𝐱, 𝐲 ⟩ |}{‖ 𝐱 ‖ ‖ 𝐲 ‖} = \frac{\sqrt{2}}{2},

with angles $𝜋 / 3$ and $𝜋 / 4$ . The first is the angle between $𝐱$ and $𝐲$ regarded as vectors $(1, 0, 1, 0)$ and $(1, 0, 0, 1)$ of $ℝ^{4}$ .

(b) $Re ⟨ 𝜆 𝐱, 𝐲 ⟩ =$ $Re (𝜆 (1 - 𝑖)) \leq$ $| 𝜆 (1 - 𝑖) | =$ $\sqrt{2}$ , with equality if and only if $𝜆 (1 - 𝑖)$ is a nonnegative real number. With $1 - 𝑖 = \sqrt{2} 𝜇$ , $𝜇 = (1 - 𝑖) / \sqrt{2}$ , this happens for $𝜆 = 𝜇 = (1 + 𝑖) / \sqrt{2}$ , and indeed $𝜆 (1 - 𝑖) =$ $(1 + 𝑖) (1 - 𝑖) / \sqrt{2} =$ $\sqrt{2}$ . So the maximum is $\sqrt{2} = | ⟨ 𝐱, 𝐲 ⟩ |$ , and the smallest real angle between $𝐲$ and a unit-modulus multiple of $𝐱$ is $𝜋 / 4$ , as Technique 6.2.8 predicts.

Exercise 6.2.30.routine Let $𝑧_{1},$ $\dots,$ $𝑧_{𝑛},$ $𝑤_{1},$ $\dots,$ $𝑤_{𝑛} \in$ $ℂ$ . Show that

{| \sum_{𝑘 = 1}^{𝑛} 𝑧_{𝑘} 𝑤_{𝑘} |}^{2} \leq (\sum_{𝑘 = 1}^{𝑛} {| 𝑧_{𝑘} |}^{2}) (\sum_{𝑘 = 1}^{𝑛} {| 𝑤_{𝑘} |}^{2}),

and that equality holds if and only if $(𝑧_{1}, \dots, 𝑧_{𝑛})$ and $(𝑤_{1}, \dots, 𝑤_{𝑛})$ are linearly dependent. Give an example where $(𝑧_{1}, \dots, 𝑧_{𝑛})$ and $(𝑤_{1}, \dots, 𝑤_{𝑛})$ are linearly dependent but the inequality is strict.

Solution.

Uses. Theorem 6.2.1 (Cauchy–Schwarz inequality), Definition 6.1.8 (Dot product), Remark 6.2.2

Hint 1. The dot product on

ℂ^{𝑛}

conjugates the second vector. Which vector should play the role of

𝐲

so that

𝐳 \cdot 𝐲 = \sum_{𝑘} 𝑧_{𝑘} 𝑤_{𝑘}

Hint 2. For the example, try

𝐳 = 𝐰 = (1, 𝑖)

Sketch. With

𝐰 = (𝑤_{1}, \dots, 𝑤_{𝑛})

𝐳 \cdot 𝐰 = \sum_{𝑘} 𝑧_{𝑘} 𝑤_{𝑘}

and

‖ 𝐰 ‖ = ‖ 𝐰 ‖

, so this is Cauchy–Schwarz for

𝐳, 𝐰

, with equality iff they are dependent. For

𝐳 = 𝐰 = (1, 𝑖)

, the left side is

{| 1 + 𝑖^{2} |}^{2} = 0 < 4

Let $𝐳 = (𝑧_{1}, \dots, 𝑧_{𝑛})$ and $𝐰 = (𝑤_{1}, \dots, 𝑤_{𝑛})$ . Then $𝐳 \cdot$ $𝐰 =$ $\sum_{𝑘} 𝑧_{𝑘} 𝑤_{𝑘} =$ $\sum_{𝑘} 𝑧_{𝑘} 𝑤_{𝑘}$ , and ${‖ 𝐰 ‖}^{2} =$ $\sum_{𝑘} {| 𝑤_{𝑘} |}^{2} =$ $\sum_{𝑘} {| 𝑤_{𝑘} |}^{2}$ . So the inequality is Theorem 6.2.1 for $𝐳$ and $𝐰$ (squared), and by Theorem 6.2.1(b) and Remark 6.2.2, equality holds if and only if $𝐳$ and $𝐰$ are linearly dependent.

For $𝐳 = 𝐰 = (1, 𝑖)$ , the two vectors are equal, hence dependent, but $𝑧_{1} 𝑤_{1} +$ $𝑧_{2} 𝑤_{2} =$ $1 +$ $𝑖^{2} =$ $0$ , while the right side is $2 \cdot 2 = 4$ . Here $𝐰 = (1, - 𝑖)$ is not a multiple of $𝐳$ . With $𝐰 = (1, - 𝑖)$ instead, $\sum_{𝑘} 𝑧_{𝑘} 𝑤_{𝑘} =$ $1 -$ $𝑖^{2} =$ $2$ and equality $4 = 4$ holds.

Exercise 6.2.31.standard Find the largest value of

𝑥 + 𝑦 + 𝑧

over all

(𝑥, 𝑦, 𝑧) \in ℝ^{3}

with

𝑥^{2} + 4 𝑦^{2} + 9 𝑧^{2} \leq 1

, and the point where it is attained.

Solution.

Uses. Theorem 6.2.1 (Cauchy–Schwarz inequality), Example 6.1.21 (Weighted dot products)

Techniques. Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1. The constraint says

‖ 𝐯 ‖ \leq 1

for a weighted inner product. Which weights?

Hint 2. Write

𝑥 +

𝑦 +

𝑧 =

⟨ (𝑥, 𝑦, 𝑧), 𝐮 ⟩

in the inner product

𝑥_{1} 𝑦_{1} +

4 𝑥_{2} 𝑦_{2} +

9 𝑥_{3} 𝑦_{3}

, for a suitable vector

𝐮

Hint 3.

𝐮 = (1, 1 / 4, 1 / 9)

. For the maximizer, use the equality case together with

‖ 𝐯 ‖ = 1

Sketch. In

{⟨ 𝐚, 𝐛 ⟩}_{𝑤} =

𝑎_{1} 𝑏_{1} +

4 𝑎_{2} 𝑏_{2} +

9 𝑎_{3} 𝑏_{3}

𝑥 +

𝑦 +

𝑧 =

{⟨ 𝐯, 𝐮 ⟩}_{𝑤}

with

𝐮 = (1, 1 / 4, 1 / 9)

and

{‖ 𝐮 ‖}_{𝑤} = 7 / 6

. So

𝑥 +

𝑦 +

𝑧 \leq

{‖ 𝐯 ‖}_{𝑤} \cdot

7 / 6 \leq

7 / 6

, with equality at

𝐯 =

(6 / 7) 𝐮 =

(6 / 7, 3 / 14, 2 / 21)

Let ${⟨ 𝐚, 𝐛 ⟩}_{𝑤} =$ $𝑎_{1} 𝑏_{1} +$ $4 𝑎_{2} 𝑏_{2} +$ $9 𝑎_{3} 𝑏_{3}$ , an inner product on $ℝ^{3}$ by Example 6.1.21, and let $𝐯 = (𝑥, 𝑦, 𝑧)$ and $𝐮 = (1, 1 / 4, 1 / 9)$ . Then ${⟨ 𝐯, 𝐮 ⟩}_{𝑤} =$ $𝑥 +$ $𝑦 +$ $𝑧$ , the constraint is ${‖ 𝐯 ‖}_{𝑤} \leq 1$ , and

{‖ 𝐮 ‖}_{𝑤}^{2} = 1 + 4 \cdot \frac{1}{16} + 9 \cdot \frac{1}{81} = 1 + \frac{1}{4} + \frac{1}{9} = \frac{49}{36} .

By Theorem 6.2.1,

𝑥 + 𝑦 + 𝑧 \leq | {⟨ 𝐯, 𝐮 ⟩}_{𝑤} | \leq {‖ 𝐯 ‖}_{𝑤} {‖ 𝐮 ‖}_{𝑤} \leq \frac{7}{6} .

For $𝐯 =$ $(6 / 7) 𝐮 =$ $(6 / 7, 3 / 14, 2 / 21)$ we have ${‖ 𝐯 ‖}_{𝑤} =$ $(6 / 7) (7 / 6) =$ $1$ , so the constraint holds, and $𝑥 +$ $𝑦 +$ $𝑧 =$ $(6 / 7) {‖ 𝐮 ‖}_{𝑤}^{2}$ , which is $(6 / 7) (49 / 36) = 7 / 6$ . So the maximum is $7 / 6$ .

It is attained only there: equality throughout requires ${‖ 𝐯 ‖}_{𝑤} = 1$ , equality in Cauchy–Schwarz, so $𝐯 = 𝑡 𝐮$ (as $𝐮 \neq 𝟎$ ), and ${⟨ 𝐯, 𝐮 ⟩}_{𝑤} =$ $𝑡 {‖ 𝐮 ‖}_{𝑤}^{2} \geq$ $0$ , so $𝑡 \geq 0$ ; then ${‖ 𝐯 ‖}_{𝑤} = 𝑡 \cdot 7 / 6 = 1$ gives $𝑡 = 6 / 7$ .

Exercise 6.2.32.standard

(a)Find all $(𝑥, 𝑦, 𝑧) \in ℝ^{3}$ with $𝑥 + 2 𝑦 + 3 𝑧 = 14$ and $𝑥^{2} + 𝑦^{2} + 𝑧^{2} = 14$ .
(b)Show that there is no $(𝑥, 𝑦, 𝑧) \in ℝ^{3}$ with $𝑥 + 2 𝑦 + 3 𝑧 = 14$ and $𝑥^{2} + 𝑦^{2} + 𝑧^{2} = 13$ .

Solution.

Uses. Theorem 6.2.1 (Cauchy–Schwarz inequality)

Techniques. Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1. The first equation says

𝐯 \cdot (1, 2, 3) = 14

, and

{‖ (1, 2, 3) ‖}^{2} = 14

. What does Cauchy–Schwarz say about

𝐯 \cdot (1, 2, 3)

when

{‖ 𝐯 ‖}^{2} = 14

Hint 2. The data force equality in Cauchy–Schwarz. Use the equality case.

Sketch. With

𝐯 = (𝑥, 𝑦, 𝑧)

and

𝐮 = (1, 2, 3)

14 =

𝐯 \cdot

𝐮 \leq

‖ 𝐯 ‖ ‖ 𝐮 ‖ =

\sqrt{14} \sqrt{14} =

14

, so equality holds and

𝐯 = 𝑡 𝐮

; then

14 𝑡 = 14

gives

(1, 2, 3)

. In (b),

𝐯 \cdot 𝐮 \leq \sqrt{13 \cdot 14} < 14

Let $𝐯 = (𝑥, 𝑦, 𝑧)$ and $𝐮 = (1, 2, 3)$ , so that ${‖ 𝐮 ‖}^{2} = 14$ and the first equation is $𝐯 \cdot 𝐮 = 14$ .

(a) If ${‖ 𝐯 ‖}^{2} = 14$ , then by Theorem 6.2.1, $14 =$ $𝐯 \cdot$ $𝐮 \leq$ $‖ 𝐯 ‖ ‖ 𝐮 ‖ =$ $14$ , so equality holds. By Theorem 6.2.1(b), and since $𝐮 \neq 𝟎$ , $𝐯 = 𝑡 𝐮$ for some $𝑡 \in ℝ$ . Then $𝐯 \cdot 𝐮 = 14 𝑡 = 14$ gives $𝑡 = 1$ , so $(𝑥, 𝑦, 𝑧) =$ $(1, 2, 3)$ . Conversely, $(1, 2, 3)$ satisfies both equations, since $1 + 4 + 9 = 14$ . So $(1, 2, 3)$ is the only solution.

(b) If ${‖ 𝐯 ‖}^{2} = 13$ , then $𝐯 \cdot 𝐮 \leq \sqrt{13} \sqrt{14} < 14$ , so the first equation fails.

Geometrically, the plane $𝑥 + 2 𝑦 + 3 𝑧 = 14$ is at distance $14 / \sqrt{14} = \sqrt{14}$ from the origin. It touches the sphere of radius $\sqrt{14}$ in exactly one point and misses the smaller sphere of radius $\sqrt{13}$ .

Exercise 6.2.33.standard Let $𝐱, 𝐲$ be vectors in an inner product space.

(a)Show that $| {‖ 𝐱 ‖}^{2} -$ ${‖ 𝐲 ‖}^{2} | \leq$ $‖ 𝐱 +$ $𝐲 ‖ ‖ 𝐱 -$ $𝐲 ‖$ .
(b)Show that equality holds if and only if one of $𝐱, 𝐲$ is a real multiple of the other. (Over $ℝ$ this just says that $𝐱, 𝐲$ are linearly dependent.)

Solution.

Uses. Theorem 6.2.1 (Cauchy–Schwarz inequality), Proposition 6.1.4 (Sesquilinearity), Remark 6.2.2, Notation 6.1.1 (Real and complex scalars)

Hint 1. Expand

⟨ 𝐱 + 𝐲, 𝐱 - 𝐲 ⟩

. Its real part is

{‖ 𝐱 ‖}^{2} - {‖ 𝐲 ‖}^{2}

Hint 2. Then

| {‖ 𝐱 ‖}^{2} -

{‖ 𝐲 ‖}^{2} | \leq

| ⟨ 𝐱 + 𝐲, 𝐱 - 𝐲 ⟩ |

, and Cauchy–Schwarz finishes (a).

Hint 3. For (b), equality needs

Im ⟨ 𝐱, 𝐲 ⟩ = 0

and equality in Cauchy–Schwarz for

𝐱 + 𝐲, 𝐱 - 𝐲

. Show that

𝐱 + 𝐲, 𝐱 - 𝐲

are dependent if and only if

𝐱, 𝐲

are.

Sketch.

⟨ 𝐱 + 𝐲, 𝐱 - 𝐲 ⟩ =

({‖ 𝐱 ‖}^{2} - {‖ 𝐲 ‖}^{2}) -

2 𝑖 Im ⟨ 𝐱, 𝐲 ⟩

, so

| {‖ 𝐱 ‖}^{2} - {‖ 𝐲 ‖}^{2} |

is at most

| ⟨ 𝐱 + 𝐲, 𝐱 - 𝐲 ⟩ |

, which Cauchy–Schwarz bounds by

‖ 𝐱 + 𝐲 ‖ ‖ 𝐱 - 𝐲 ‖

. Equality holds iff

Im ⟨ 𝐱, 𝐲 ⟩ = 0

and

𝐱 + 𝐲, 𝐱 - 𝐲

are dependent, iff

𝐱, 𝐲

are dependent with a real ratio.

(a) By sesquilinearity and conjugate symmetry,

\begin{aligned} ⟨ 𝐱 + 𝐲, 𝐱 - 𝐲 ⟩ & = {‖ 𝐱 ‖}^{2} - ⟨ 𝐱, 𝐲 ⟩ + ⟨ 𝐱, 𝐲 ⟩ - {‖ 𝐲 ‖}^{2} \\ = ({‖ 𝐱 ‖}^{2} - {‖ 𝐲 ‖}^{2}) - 2 𝑖 Im ⟨ 𝐱, 𝐲 ⟩, \end{aligned}

since $𝑧 - 𝑧 = 2 𝑖 Im 𝑧$ . So ${‖ 𝐱 ‖}^{2} - {‖ 𝐲 ‖}^{2}$ is the real part of $⟨ 𝐱 + 𝐲, 𝐱 - 𝐲 ⟩$ , and by $| Re 𝑧 | \leq | 𝑧 |$ and Theorem 6.2.1,

| {‖ 𝐱 ‖}^{2} - {‖ 𝐲 ‖}^{2} | \leq | ⟨ 𝐱 + 𝐲, 𝐱 - 𝐲 ⟩ | \leq ‖ 𝐱 + 𝐲 ‖ ‖ 𝐱 - 𝐲 ‖ .

(b) Equality holds if and only if both steps are equalities. The first is an equality if and only if $Im ⟨ 𝐱, 𝐲 ⟩ = 0$ , since $| Re 𝑧 | = | 𝑧 |$ exactly for real $𝑧$ . The second is an equality if and only if $𝐱 + 𝐲$ and $𝐱 - 𝐲$ are linearly dependent (Remark 6.2.2). These are dependent if and only if $𝐱, 𝐲$ are: a nontrivial relation $𝑎 (𝐱 + 𝐲) +$ $𝑏 (𝐱 - 𝐲) =$ $𝟎$ is the relation $(𝑎 + 𝑏) 𝐱 +$ $(𝑎 - 𝑏) 𝐲 =$ $𝟎$ , which is nontrivial since $𝑎 + 𝑏 = 𝑎 - 𝑏 = 0$ forces $𝑎 = 𝑏 = 0$ ; conversely, $𝛼 𝐱 + 𝛽 𝐲 = 𝟎$ becomes $((𝛼 + 𝛽) / 2) (𝐱 + 𝐲) +$ $((𝛼 - 𝛽) / 2) (𝐱 - 𝐲) =$ $𝟎$ , again nontrivial.

So equality holds if and only if $𝐱, 𝐲$ are dependent and $⟨ 𝐱, 𝐲 ⟩$ is real. If $𝐱 = 𝟎$ or $𝐲 = 𝟎$ , both conditions hold, and the zero vector is the real multiple $0$ of the other. If $𝐲 = 𝑎 𝐱$ with $𝐱 \neq 𝟎$ , then $⟨ 𝐱, 𝐲 ⟩ = 𝑎 {‖ 𝐱 ‖}^{2}$ is real if and only if $𝑎$ is real; the case $𝐱 = 𝑏 𝐲$ is the same. This proves (b).

Exercise 6.2.34.standard Let

𝐱, 𝐲, 𝐳

be vectors in an inner product space such that equality holds in the Cauchy–Schwarz inequality for the pair

𝐱, 𝐲

and for the pair

𝐲, 𝐳

. Prove or disprove: equality holds for the pair

𝐱, 𝐳

. What changes if

𝐲 \neq 𝟎

Solution.

Uses. Theorem 6.2.1 (Cauchy–Schwarz inequality), Lemma 6.2.3 (The easy half of (b))

Techniques. Finding counterexamples and minimal examples (Technique 1.4.18)

Hint 1. Equality is the same as linear dependence. Is “is dependent with” a transitive relation? Think about the zero vector.

Hint 2. If

𝐲 \neq 𝟎

, both

𝐱

and

𝐳

are multiples of

𝐲

Sketch. False: take

𝐲 = 𝟎

and

𝐱 = 𝐞_{1}

𝐳 = 𝐞_{2}

ℝ^{2}

. If

𝐲 \neq 𝟎

, then

𝐱 = 𝑎 𝐲

and

𝐳 = 𝑏 𝐲

, so one of

𝐱, 𝐳

is a multiple of the other, and equality holds by Lemma 6.2.3.

The claim is false. In $ℝ^{2}$ with the dot product, let $𝐱 = 𝐞_{1}$ , $𝐲 = 𝟎$ , and $𝐳 = 𝐞_{2}$ . Both hypotheses hold, since both sides of the inequality are $0$ for each pair containing $𝟎$ . But $| 𝐞_{1} \cdot$ $𝐞_{2} | =$ $0 <$ $1 =$ $‖ 𝐞_{1} ‖ ‖ 𝐞_{2} ‖$ .

If $𝐲 \neq 𝟎$ , the conclusion holds. By Theorem 6.2.1(b), one of $𝐱, 𝐲$ is a multiple of the other, and since $𝐲 \neq 𝟎$ , $𝐱 = 𝑎 𝐲$ for some $𝑎$ (if $𝐲 = 𝑐 𝐱$ , then $𝑐 \neq 0$ and $𝐱 = 𝑐^{- 1} 𝐲$ ). Likewise $𝐳 = 𝑏 𝐲$ . If $𝑎 = 0$ , then $𝐱 = 𝟎 = 0 𝐳$ . If $𝑎 \neq 0$ , then $𝐲 = 𝑎^{- 1} 𝐱$ and $𝐳 = (𝑏 / 𝑎) 𝐱$ . Either way one of $𝐱, 𝐳$ is a multiple of the other, and equality holds by Lemma 6.2.3.

Exercise 6.2.35.standard Find the errors in the following argument, and give a numerical example in $ℂ$ (with the inner product $⟨ 𝑧, 𝑤 ⟩ = 𝑧 𝑤$ ) showing that its first displayed formula is false.

Claim. For $𝐱, 𝐲$ in a complex inner product space, $| ⟨ 𝐱, 𝐲 ⟩ | \leq$ $‖ 𝐱 ‖ ‖ 𝐲 ‖$ .

Proof. We may assume $𝐲 \neq 𝟎$ . For every $𝑡 \in ℂ$ ,

0 \leq {‖ 𝐱 - 𝑡 𝐲 ‖}^{2} = {‖ 𝐱 ‖}^{2} - 2 𝑡 ⟨ 𝐱, 𝐲 ⟩ + 𝑡^{2} {‖ 𝐲 ‖}^{2} .

A quadratic in $𝑡$ that is never negative has nonpositive discriminant, so $4 {⟨ 𝐱, 𝐲 ⟩}^{2} -$ $4 {‖ 𝐱 ‖}^{2} {‖ 𝐲 ‖}^{2} \leq$ $0$ . Taking square roots, $| ⟨ 𝐱, 𝐲 ⟩ | \leq$ $‖ 𝐱 ‖ ‖ 𝐲 ‖$ .

Solution.

Uses. Proposition 6.1.29 (Expanding a norm), Lemma 6.2.6 (Nonnegative quadratics)

Techniques. A nonnegative quadratic has nonpositive discriminant (Technique 6.2.7)

Hint 1. Expand

{‖ 𝐱 - 𝑡 𝐲 ‖}^{2}

correctly with Proposition 6.1.29. Where do conjugates and real parts appear?

Hint 2. Is the right side of the displayed formula even a real number for complex

𝑡

? Does “

\leq 0

” make sense for

{⟨ 𝐱, 𝐲 ⟩}^{2}

Hint 3. Try

𝑥 = 1

𝑦 = 𝑖

𝑡 = 𝑖

ℂ

Sketch. The correct expansion is

{‖ 𝐱 ‖}^{2} -

2 Re (𝑡 ⟨ 𝐱, 𝐲 ⟩) +

{| 𝑡 |}^{2} {‖ 𝐲 ‖}^{2}

. The displayed expression is not real in general, so it cannot be compared with

0

; the discriminant criterion is about real quadratics in a real variable; and

{⟨ 𝐱, 𝐲 ⟩}^{2}

need not be real. In

ℂ

with

𝑥 = 1

𝑦 = 𝑖

𝑡 = 𝑖

: the formula gives

- 2

, but

{| 1 - 𝑖 \cdot 𝑖 |}^{2} = 4

. The repair is the second proof of Theorem 6.2.1.

There are three errors.

First, the expansion is wrong over $ℂ$ . By Proposition 6.1.29 with $𝑎 = - 𝑡$ ,

{‖ 𝐱 - 𝑡 𝐲 ‖}^{2} = {‖ 𝐱 ‖}^{2} - 2 Re (𝑡 ⟨ 𝐱, 𝐲 ⟩) + {| 𝑡 |}^{2} {‖ 𝐲 ‖}^{2} .

The argument’s formula drops the conjugate and the real part and writes $𝑡^{2}$ for ${| 𝑡 |}^{2}$ . For a numerical counterexample, take $𝑉 = ℂ$ with $⟨ 𝑧, 𝑤 ⟩ = 𝑧 𝑤$ , $𝑥 = 1$ , $𝑦 = 𝑖$ , and $𝑡 = 𝑖$ . Then $⟨ 𝑥, 𝑦 ⟩ = 𝑖 = - 𝑖$ , $‖ 𝑥 ‖ = ‖ 𝑦 ‖ = 1$ , and the formula gives $1 -$ $2 𝑖 (- 𝑖) +$ $𝑖^{2} =$ $1 -$ $2 -$ $1 =$ $- 2$ , while ${‖ 𝑥 - 𝑡 𝑦 ‖}^{2} =$ ${| 1 - 𝑖 \cdot 𝑖 |}^{2} =$ ${| 2 |}^{2} =$ $4$ .

Second, the discriminant criterion (Lemma 6.2.6) is a statement about real quadratics in a real variable. A polynomial in a complex variable $𝑡$ takes non-real values, and “never negative” has no meaning for it.

Third, ${⟨ 𝐱, 𝐲 ⟩}^{2}$ need not be real, so “ $4 {⟨ 𝐱, 𝐲 ⟩}^{2} -$ $4 {‖ 𝐱 ‖}^{2} {‖ 𝐲 ‖}^{2} \leq$ $0$ ” is not a meaningful inequality, and “taking square roots” of it is not a valid step. (The claim itself is true; it is Theorem 6.2.1.)

The repair is the second proof of Theorem 6.2.1: restrict to real $𝑡$ , and first rotate the phase of $𝐱$ so that $⟨ 𝐱, 𝐲 ⟩$ is a nonnegative real number. Then the expansion above becomes a real quadratic in $𝑡$ with middle coefficient $- 2 | ⟨ 𝐱, 𝐲 ⟩ |$ , and the discriminant argument applies.

Exercise 6.2.36.standard Show that for every $𝑓 \in 𝐶 [0, 1]$ ,

{(\int_{0}^{1} 𝑓 (𝑡) d 𝑡)}^{2} \leq (1 - 𝑒^{- 1}) \int_{0}^{1} 𝑒^{𝑡} {𝑓 (𝑡)}^{2} d 𝑡,

and find all $𝑓$ for which equality holds.

Solution.

Uses. Theorem 6.2.1 (Cauchy–Schwarz inequality), Proposition 6.1.13 (Integral inner products)

Techniques. Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1. The right side contains

𝑒^{𝑡} 𝑓^{2} =

{(𝑒^{𝑡 / 2} 𝑓)}^{2}

. Split

𝑓 =

(𝑒^{𝑡 / 2} 𝑓) \cdot

𝑒^{- 𝑡 / 2}

Hint 2. Apply Cauchy–Schwarz in

𝐶 [0, 1]

𝑔 = 𝑒^{𝑡 / 2} 𝑓

and

ℎ = 𝑒^{- 𝑡 / 2}

. Compute

{‖ ℎ ‖}^{2}

Sketch.

\int 𝑓 =

⟨ 𝑒^{𝑡 / 2} 𝑓, 𝑒^{- 𝑡 / 2} ⟩

, so

{(\int 𝑓)}^{2} \leq

(\int 𝑒^{𝑡} 𝑓^{2}) (\int_{0}^{1} 𝑒^{- 𝑡} d 𝑡) =

(1 - 𝑒^{- 1}) \int 𝑒^{𝑡} 𝑓^{2}

. Equality iff

𝑒^{𝑡 / 2} 𝑓 =

𝑐 𝑒^{- 𝑡 / 2}

, that is,

𝑓 (𝑡) = 𝑐 𝑒^{- 𝑡}

In $𝐶 [0, 1]$ with $⟨ 𝑔, ℎ ⟩ = \int_{0}^{1} 𝑔 ℎ$ , let $𝑔 (𝑡) =$ $𝑒^{𝑡 / 2} 𝑓 (𝑡)$ and $ℎ (𝑡) = 𝑒^{- 𝑡 / 2}$ , both continuous. Then $𝑔 ℎ = 𝑓$ , $𝑔^{2} = 𝑒^{𝑡} 𝑓^{2}$ , and $ℎ^{2} = 𝑒^{- 𝑡}$ , so

⟨ 𝑔, ℎ ⟩ = \int_{0}^{1} 𝑓, {‖ 𝑔 ‖}^{2} = \int_{0}^{1} 𝑒^{𝑡} 𝑓^{2},

and ${‖ ℎ ‖}^{2} =$ $\int_{0}^{1} 𝑒^{- 𝑡} d 𝑡 =$ $1 -$ $𝑒^{- 1}$ . By Theorem 6.2.1, ${⟨ 𝑔, ℎ ⟩}^{2} \leq {‖ 𝑔 ‖}^{2} {‖ ℎ ‖}^{2}$ , which is the inequality.

Since $ℎ \neq 𝟎$ , equality holds if and only if $𝑔 = 𝑐 ℎ$ for some $𝑐 \in ℝ$ , that is, $𝑒^{𝑡 / 2} 𝑓 (𝑡) =$ $𝑐 𝑒^{- 𝑡 / 2}$ , or $𝑓 (𝑡) = 𝑐 𝑒^{- 𝑡}$ . Check: for $𝑓 (𝑡) = 𝑒^{- 𝑡}$ , the left side is ${(1 - 𝑒^{- 1})}^{2}$ and the right side is $(1 - 𝑒^{- 1}) \int_{0}^{1} 𝑒^{- 𝑡} d 𝑡 =$ ${(1 - 𝑒^{- 1})}^{2}$ .

Exercise 6.2.37.standard Let $𝑝 =$ $𝑎_{0} +$ $𝑎_{1} 𝑥 +$ $\dots +$ $𝑎_{𝑛} 𝑥^{𝑛} \in$ $ℝ [𝑥]$ and $𝑡 \in ℝ$ .

(a)Show that
$| 𝑝 (𝑡) | \leq {(𝑎_{0}^{2} + \dots + 𝑎_{𝑛}^{2})}^{1 / 2} {(1 + 𝑡^{2} + \dots + 𝑡^{2 𝑛})}^{1 / 2},$
and that for each $𝑡$ and $𝑛$ some nonzero $𝑝$ of degree at most $𝑛$ attains equality.
(b)Deduce that if $| 𝑡 | < 1$ , then $| 𝑝 (𝑡) | \leq$ ${(𝑎_{0}^{2} + \dots + 𝑎_{𝑛}^{2})}^{1 / 2} / \sqrt{1 - 𝑡^{2}}$ for every polynomial $𝑝$ , whatever its degree.

Solution.

Uses. Theorem 6.2.1 (Cauchy–Schwarz inequality), Definition 1.2.17 (Evaluation)

Techniques. Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1.

𝑝 (𝑡) =

\sum_{𝑘} 𝑎_{𝑘} 𝑡^{𝑘}

is a dot product in

ℝ^{𝑛 + 1}

. Of which two vectors?

Hint 2. For equality, choose the coefficient vector proportional to

(1, 𝑡, \dots, 𝑡^{𝑛})

. For (b), compare

\sum_{𝑘 = 0}^{𝑛} 𝑡^{2 𝑘}

with a geometric series.

Sketch.

𝑝 (𝑡) = 𝐚 \cdot 𝛕

with

𝐚 = (𝑎_{0}, \dots, 𝑎_{𝑛})

and

𝛕 = (1, 𝑡, \dots, 𝑡^{𝑛})

, so Cauchy–Schwarz gives (a), with equality for

𝐚 = 𝛕

. For

| 𝑡 | < 1

\sum_{𝑘 = 0}^{𝑛} 𝑡^{2 𝑘} \leq

\sum_{𝑘 \geq 0} 𝑡^{2 𝑘} =

1 / (1 - 𝑡^{2})

(a) Let $𝐚 = (𝑎_{0}, \dots, 𝑎_{𝑛})$ and $𝛕 =$ $(1, 𝑡, 𝑡^{2}, \dots, 𝑡^{𝑛})$ in $ℝ^{𝑛 + 1}$ . By Definition 1.2.17, $𝑝 (𝑡) =$ $\sum_{𝑘 = 0}^{𝑛} 𝑎_{𝑘} 𝑡^{𝑘} =$ $𝐚 \cdot$ $𝛕$ , and ${‖ 𝛕 ‖}^{2} =$ $\sum_{𝑘 = 0}^{𝑛} 𝑡^{2 𝑘}$ . So Theorem 6.2.1 gives $| 𝑝 (𝑡) | \leq ‖ 𝐚 ‖ ‖ 𝛕 ‖$ , which is the inequality. For $𝑝 =$ $\sum_{𝑘 = 0}^{𝑛} 𝑡^{𝑘} 𝑥^{𝑘}$ , the coefficient vector is $𝛕$ itself, which is nonzero (its first entry is $1$ ), and $𝑝 (𝑡) =$ ${‖ 𝛕 ‖}^{2} =$ $‖ 𝛕 ‖ ‖ 𝛕 ‖$ , so equality holds.

(b) For $| 𝑡 | < 1$ , $0 \leq 𝑡^{2} < 1$ , so $\sum_{𝑘 = 0}^{𝑛} 𝑡^{2 𝑘} =$ $(1 - 𝑡^{2 𝑛 + 2}) / (1 - 𝑡^{2}) \leq$ $1 / (1 - 𝑡^{2})$ . Combining with (a), $| 𝑝 (𝑡) | \leq$ $‖ 𝐚 ‖ / \sqrt{1 - 𝑡^{2}}$ , and the bound does not depend on $𝑛$ .

Exercise 6.2.38.standard

(a)Show that $\sqrt{1} +$ $\sqrt{2} +$ $\dots +$ $\sqrt{𝑛} \leq$ $𝑛 \sqrt{(𝑛 + 1) / 2}$ for every $𝑛 \geq 1$ .
(b)Show that for all real $𝑎_{1}, \dots, 𝑎_{𝑛}$ ,
$\frac{𝑎_{1}}{1} + \frac{𝑎_{2}}{2} + \dots + \frac{𝑎_{𝑛}}{𝑛} \leq \sqrt{2} {(𝑎_{1}^{2} + \dots + 𝑎_{𝑛}^{2})}^{1 / 2} .$
(You may use $1 / 𝑘^{2} <$ $1 / (𝑘 - 1) -$ $1 / 𝑘$ for $𝑘 \geq 2$ .)

Solution.

Uses. Proposition 6.2.22 (The square of a sum), Theorem 6.2.1 (Cauchy–Schwarz inequality)

Techniques. Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1. (a) is Proposition 6.2.22 for

𝑎_{𝑘} = \sqrt{𝑘}

. Recall

1 +

2 +

\dots +

𝑛 =

𝑛 (𝑛 + 1) / 2

Hint 2. (b) The left side is

𝐚 \cdot (1, 1 / 2, \dots, 1 / 𝑛)

. Bound

\sum_{𝑘 = 1}^{𝑛} 1 / 𝑘^{2}

by telescoping.

Sketch. (a)

{(\sum_{𝑘} \sqrt{𝑘})}^{2} \leq

𝑛 \sum_{𝑘} 𝑘 =

𝑛^{2} (𝑛 + 1) / 2

. (b) Cauchy–Schwarz gives

\sum_{𝑘} 𝑎_{𝑘} / 𝑘 \leq

‖ 𝐚 ‖ {(\sum_{𝑘} 1 / 𝑘^{2})}^{1 / 2}

, and the bound

1 / 𝑘^{2} <

1 / (𝑘 - 1) -

1 / 𝑘

makes

\sum_{𝑘 = 2}^{𝑛} 1 / 𝑘^{2}

telescope to less than

1

(a) By Proposition 6.2.22 with $𝑎_{𝑘} = \sqrt{𝑘}$ ,

{(\sum_{𝑘 = 1}^{𝑛} \sqrt{𝑘})}^{2} \leq 𝑛 \sum_{𝑘 = 1}^{𝑛} 𝑘 = 𝑛 \cdot 𝑛 \frac{𝑛 + 1}{2},

and taking square roots gives the claim.

(b) Let $𝐚 = (𝑎_{1}, \dots, 𝑎_{𝑛})$ and $𝐫 = (1, 1 / 2, \dots, 1 / 𝑛)$ . By Theorem 6.2.1,

\sum_{𝑘 = 1}^{𝑛} \frac{𝑎_{𝑘}}{𝑘} = 𝐚 \cdot 𝐫 \leq ‖ 𝐚 ‖ ‖ 𝐫 ‖ = ‖ 𝐚 ‖ {(\sum_{𝑘 = 1}^{𝑛} \frac{1}{𝑘^{2}})}^{1 / 2} .

For $𝑘 \geq 2$ , $1 / 𝑘^{2} <$ $1 / (𝑘 (𝑘 - 1)) =$ $1 / (𝑘 - 1) -$ $1 / 𝑘$ , so the sum telescopes:

\sum_{𝑘 = 1}^{𝑛} \frac{1}{𝑘^{2}} \leq 1 + \sum_{𝑘 = 2}^{𝑛} (\frac{1}{𝑘 - 1} - \frac{1}{𝑘}) = 2 - \frac{1}{𝑛} < 2 .

Hence $\sum_{𝑘} 𝑎_{𝑘} / 𝑘 \leq \sqrt{2} ‖ 𝐚 ‖$ .

Exercise 6.2.39.standard

(a)Show that for every $𝐴 \in 𝔽^{𝑛 \times 𝑛}$ , ${| tr 𝐴 |}^{2} \leq$ $𝑛 tr (𝐴^{*} 𝐴)$ , with equality if and only if $𝐴$ is a scalar multiple of $𝐼_{𝑛}$ .
(b)Find all $𝐴 \in ℝ^{2 \times 2}$ with $tr 𝐴 = 2$ and $\sum_{𝑖, 𝑗} 𝑎_{𝑖 𝑗}^{2} = 2$ .

Solution.

Uses. Proposition 6.1.16 (Frobenius inner product), Theorem 6.2.1 (Cauchy–Schwarz inequality), Remark 6.2.2

Techniques. Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1. The trace is an inner product with a fixed matrix in the Frobenius inner product

⟨ 𝐴, 𝐵 ⟩ = tr (𝐵^{*} 𝐴)

. Which one?

Hint 2.

⟨ 𝐴, 𝐼_{𝑛} ⟩ = tr 𝐴

and

{‖ 𝐼_{𝑛} ‖}^{2} = 𝑛

. For (b), the data force equality.

Sketch.

tr 𝐴 =

tr (𝐼_{𝑛}^{*} 𝐴) =

⟨ 𝐴, 𝐼_{𝑛} ⟩

and

{‖ 𝐼_{𝑛} ‖}^{2} = 𝑛

, so Cauchy–Schwarz gives (a), with equality iff

𝐴 = 𝑐 𝐼_{𝑛}

. In (b),

{| tr 𝐴 |}^{2} = 4 = 2 \cdot 2

, so

𝐴 = 𝑐 𝐼_{2}

, and

tr 𝐴 = 2

gives

𝐴 = 𝐼_{2}

(a) In the Frobenius inner product (Proposition 6.1.16), $⟨ 𝐴, 𝐼_{𝑛} ⟩ =$ $tr (𝐼_{𝑛}^{*} 𝐴) =$ $tr 𝐴$ , ${‖ 𝐼_{𝑛} ‖}^{2} =$ $tr (𝐼_{𝑛}) =$ $𝑛$ , and ${‖ 𝐴 ‖}^{2} = tr (𝐴^{*} 𝐴)$ . By Theorem 6.2.1, ${| tr 𝐴 |}^{2} \leq$ ${‖ 𝐴 ‖}^{2} {‖ 𝐼_{𝑛} ‖}^{2} =$ $𝑛 tr (𝐴^{*} 𝐴)$ , with equality if and only if one of $𝐴, 𝐼_{𝑛}$ is a multiple of the other, that is (as $𝐼_{𝑛} \neq 0$ ), $𝐴 = 𝑐 𝐼_{𝑛}$ for some $𝑐 \in 𝔽$ .

(b) Here $tr (𝐴^{𝖳} 𝐴) =$ $\sum_{𝑖, 𝑗} 𝑎_{𝑖 𝑗}^{2} =$ $2$ , so ${| tr 𝐴 |}^{2} =$ $4 =$ $2 \cdot$ $2 =$ $𝑛 tr (𝐴^{𝖳} 𝐴)$ , which is the equality case of (a). So $𝐴 = 𝑐 𝐼_{2}$ , and $tr 𝐴 = 2 𝑐 = 2$ gives $𝑐 = 1$ . Conversely, $𝐼_{2}$ satisfies both conditions. So $𝐴 = 𝐼_{2}$ is the only such matrix.

Exercise 6.2.40.standard Write ${‖ 𝐴 ‖}_{𝐹} =$ ${tr (𝐴^{*} 𝐴)}^{1 / 2}$ for the Frobenius norm (Proposition 6.1.16).

(a)Show that ${‖ 𝐴 𝐵 ‖}_{𝐹} \leq$ ${‖ 𝐴 ‖}_{𝐹} {‖ 𝐵 ‖}_{𝐹}$ for all $𝐴 \in 𝔽^{𝑚 \times 𝑛}$ and $𝐵 \in 𝔽^{𝑛 \times 𝑝}$ .
(b)Deduce that every entry of $𝐴^{𝑘}$ has modulus at most ${‖ 𝐴 ‖}_{𝐹}^{𝑘}$ , for $𝐴 \in 𝔽^{𝑛 \times 𝑛}$ and $𝑘 \geq 1$ .
(c)Give $𝐴, 𝐵 \in ℝ^{2 \times 2}$ , both nonzero, with ${‖ 𝐴 𝐵 ‖}_{𝐹} <$ ${‖ 𝐴 ‖}_{𝐹} {‖ 𝐵 ‖}_{𝐹}$ .

Solution.

Uses. Proposition 6.1.16 (Frobenius inner product), Definition 2.4.1 (Matrix product), Theorem 6.2.1 (Cauchy–Schwarz inequality)

Hint 1. The

(𝑖, 𝑗)

entry of

𝐴 𝐵

\sum_{𝑘} 𝑎_{𝑖 𝑘} 𝑏_{𝑘 𝑗}

, a product of row

𝑖

𝐴

with column

𝑗

𝐵

. Bound its modulus with Cauchy–Schwarz in

𝔽^{𝑛}

(watch the conjugate).

Hint 2. Square, sum over

𝑖

and

𝑗

, and recognize

{‖ 𝐴 ‖}_{𝐹}^{2} =

\sum_{𝑖, 𝑘} {| 𝑎_{𝑖 𝑘} |}^{2}

. For (b), induct on

𝑘

Sketch.

{| {(𝐴 𝐵)}_{𝑖 𝑗} |}^{2} \leq

(\sum_{𝑘} {| 𝑎_{𝑖 𝑘} |}^{2}) (\sum_{𝑘} {| 𝑏_{𝑘 𝑗} |}^{2})

by Cauchy–Schwarz; summing over

𝑖, 𝑗

gives

{‖ 𝐴 𝐵 ‖}_{𝐹}^{2} \leq

{‖ 𝐴 ‖}_{𝐹}^{2} {‖ 𝐵 ‖}_{𝐹}^{2}

. Then

| {(𝐴^{𝑘})}_{𝑖 𝑗} | \leq

{‖ 𝐴^{𝑘} ‖}_{𝐹} \leq

{‖ 𝐴 ‖}_{𝐹}^{𝑘}

. For (c),

𝐴 = 𝐵 = (\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix})

gives

𝐴 𝐵 = 0

(a) By Definition 2.4.1, ${(𝐴 𝐵)}_{𝑖 𝑗} =$ $\sum_{𝑘 = 1}^{𝑛} 𝑎_{𝑖 𝑘} 𝑏_{𝑘 𝑗}$ . Let $𝐫_{𝑖} =$ $(𝑎_{𝑖 1}, \dots, 𝑎_{𝑖 𝑛})$ and $𝐜_{𝑗} =$ $(𝑏_{1 𝑗}, \dots, 𝑏_{𝑛 𝑗})$ in $𝔽^{𝑛}$ . Then ${(𝐴 𝐵)}_{𝑖 𝑗} =$ $𝐫_{𝑖} \cdot$ $𝐜_{𝑗}$ , and by Theorem 6.2.1,

{| {(𝐴 𝐵)}_{𝑖 𝑗} |}^{2} \leq {‖ 𝐫_{𝑖} ‖}^{2} {‖ 𝐜_{𝑗} ‖}^{2} = (\sum_{𝑘} {| 𝑎_{𝑖 𝑘} |}^{2}) (\sum_{𝑘} {| 𝑏_{𝑘 𝑗} |}^{2}) .

Summing over all $𝑖$ and $𝑗$ , and using Proposition 6.1.16,

\begin{aligned} {‖ 𝐴 𝐵 ‖}_{𝐹}^{2} & \leq \sum_{𝑖, 𝑗} (\sum_{𝑘} {| 𝑎_{𝑖 𝑘} |}^{2}) (\sum_{𝑘} {| 𝑏_{𝑘 𝑗} |}^{2}) \\ = (\sum_{𝑖, 𝑘} {| 𝑎_{𝑖 𝑘} |}^{2}) (\sum_{𝑘, 𝑗} {| 𝑏_{𝑘 𝑗} |}^{2}) = {‖ 𝐴 ‖}_{𝐹}^{2} {‖ 𝐵 ‖}_{𝐹}^{2} . \end{aligned}

(b) By (a) and induction on $𝑘$ , ${‖ 𝐴^{𝑘} ‖}_{𝐹} \leq {‖ 𝐴 ‖}_{𝐹}^{𝑘}$ : the case $𝑘 = 1$ is trivial, and ${‖ 𝐴^{𝑘 + 1} ‖}_{𝐹} =$ ${‖ 𝐴^{𝑘} 𝐴 ‖}_{𝐹} \leq$ ${‖ 𝐴^{𝑘} ‖}_{𝐹} {‖ 𝐴 ‖}_{𝐹}$ . Each entry satisfies ${| {(𝐴^{𝑘})}_{𝑖 𝑗} |}^{2} \leq$ $\sum_{𝑖^{'}, 𝑗^{'}} {| {(𝐴^{𝑘})}_{𝑖^{'} 𝑗^{'}} |}^{2} =$ ${‖ 𝐴^{𝑘} ‖}_{𝐹}^{2}$ , so $| {(𝐴^{𝑘})}_{𝑖 𝑗} | \leq$ ${‖ 𝐴 ‖}_{𝐹}^{𝑘}$ .

(c) For $𝐴 = 𝐵 = (\begin{matrix} 0 & 1 \\ 0 & 0 \end{matrix})$ , $𝐴 𝐵 = 0$ , so ${‖ 𝐴 𝐵 ‖}_{𝐹} =$ $0 <$ $1 =$ ${‖ 𝐴 ‖}_{𝐹} {‖ 𝐵 ‖}_{𝐹}$ .

Exercise 6.2.41.standard

(a)Let $𝐱, 𝐲$ be unit vectors in an inner product space with $𝐱 \neq 𝐲$ , and let $0 < 𝑠 < 1$ . Show that $‖ (1 - 𝑠) 𝐱 +$ $𝑠 𝐲 ‖ <$ $1$ . (So the segment between two points of the unit sphere lies strictly inside it, except at its endpoints.)
(b)Show that (a) fails for the taxicab norm ${‖ 𝐱 ‖}_{1} = | 𝑥_{1} | + | 𝑥_{2} |$ on $ℝ^{2}$ (Aside (Norms that do not come from inner products)). Conclude again that the taxicab norm does not come from an inner product.

Solution.

Uses. Theorem 6.2.11 (Triangle inequality), Proposition 6.1.27 (Basic properties of the norm)

Hint 1. The triangle inequality gives

\leq 1

. Equality would force

(1 - 𝑠) 𝐱

and

𝑠 𝐲

onto a common ray.

Hint 2. If

(1 - 𝑠) 𝐱 = 𝑎 𝑠 𝐲

with

𝑎 \geq 0

, take norms to find

𝑎

, and conclude

𝐱 = 𝐲

Hint 3. For (b), look at

𝐞_{1}

and

𝐞_{2}

Sketch. By Theorem 6.2.11,

‖ (1 - 𝑠) 𝐱 +

𝑠 𝐲 ‖ \leq

(1 - 𝑠) +

𝑠 =

1

, with equality only if one of

(1 - 𝑠) 𝐱, 𝑠 𝐲

is a nonnegative multiple of the other. Taking norms, the multiple is the ratio of

1 - 𝑠

and

𝑠

, and then

𝐱 = 𝐲

. In (b),

{‖ (𝐞_{1} + 𝐞_{2}) / 2 ‖}_{1} = 1

(a) By Theorem 6.2.11 and Proposition 6.1.27,

‖ (1 - 𝑠) 𝐱 + 𝑠 𝐲 ‖ \leq (1 - 𝑠) ‖ 𝐱 ‖ + 𝑠 ‖ 𝐲 ‖ = 1 .

Suppose equality holds. By Theorem 6.2.11(b), one of $(1 - 𝑠) 𝐱$ , $𝑠 𝐲$ is a nonnegative real multiple of the other. If $(1 - 𝑠) 𝐱 = 𝑎 𝑠 𝐲$ with $𝑎 \geq 0$ , taking norms gives $1 - 𝑠 = 𝑎 𝑠$ , so $(1 - 𝑠) 𝐱 = (1 - 𝑠) 𝐲$ , and $𝐱 = 𝐲$ since $1 - 𝑠 \neq 0$ . If $𝑠 𝐲 = 𝑏 (1 - 𝑠) 𝐱$ with $𝑏 \geq 0$ , the same argument gives $𝐲 = 𝐱$ . Both contradict $𝐱 \neq 𝐲$ , so the inequality is strict.

(b) The vectors $𝐞_{1}$ and $𝐞_{2}$ are distinct and ${‖ 𝐞_{1} ‖}_{1} = {‖ 𝐞_{2} ‖}_{1} = 1$ , but ${‖ (𝐞_{1} + 𝐞_{2}) / 2 ‖}_{1} =$ $1 / 2 +$ $1 / 2 =$ $1$ . (The whole segment from $𝐞_{1}$ to $𝐞_{2}$ is an edge of the diamond in Figure 6.1.4.) If ${‖ \cdot ‖}_{1}$ were the norm of an inner product, (a) would apply to it. So it is not.

Exercise 6.2.42.standard Let $𝐱, 𝐲$ be vectors in an inner product space $𝑉$ . Show that $𝐳 \in 𝑉$ satisfies

‖ 𝐱 - 𝐳 ‖ + ‖ 𝐳 - 𝐲 ‖ = ‖ 𝐱 - 𝐲 ‖

if and only if $𝐳 = (1 - 𝑠) 𝐱 + 𝑠 𝐲$ for some $𝑠 \in [0, 1]$ . That is, the points for which the detour through $𝐳$ costs nothing are exactly the points of the segment from $𝐱$ to $𝐲$ .

Solution.

Uses. Theorem 6.2.11 (Triangle inequality), Proposition 6.1.27 (Basic properties of the norm)

Techniques. Double containment (Technique 1.3.14)

Hint 1. For

\supseteq

, compute

𝐱 - 𝐳

and

𝐳 - 𝐲

for

𝐳 = (1 - 𝑠) 𝐱 + 𝑠 𝐲

: both are multiples of

𝐱 - 𝐲

Hint 2. For

\subseteq

, the condition is equality in the triangle inequality for

𝐮 = 𝐱 - 𝐳

and

𝐯 = 𝐳 - 𝐲

, whose sum is

𝐱 - 𝐲

Hint 3. If

𝐯 = 𝑎 𝐮

with

𝑎 \geq 0

, then

(1 + 𝑎) 𝐮 = 𝐱 - 𝐲

. Solve for

𝐳

Sketch. For

𝐳 = (1 - 𝑠) 𝐱 + 𝑠 𝐲

𝐱 - 𝐳 = 𝑠 (𝐱 - 𝐲)

and

𝐳 -

𝐲 =

(1 - 𝑠) (𝐱 - 𝐲)

, whose norms add up to

‖ 𝐱 - 𝐲 ‖

. Conversely, equality in the triangle inequality for

𝐮 = 𝐱 - 𝐳

𝐯 = 𝐳 - 𝐲

makes one a nonnegative multiple of the other; solving

(1 + 𝑎) 𝐮 = 𝐱 - 𝐲

gives

𝐳 = (1 - 𝑠) 𝐱 + 𝑠 𝐲

with

𝑠 = 1 / (1 + 𝑎)

, and similarly in the other case.

Let $𝐿$ be the set of $𝐳 \in 𝑉$ satisfying the equation, and $𝑆 =$ ${(1 - 𝑠) 𝐱 + 𝑠 𝐲 | 𝑠 \in [0, 1]}$ . We show $𝐿 = 𝑆$ by Technique 1.3.14.

$𝑆 \subseteq 𝐿$ . Let $𝐳 = (1 - 𝑠) 𝐱 + 𝑠 𝐲$ with $𝑠 \in [0, 1]$ . Then $𝐱 - 𝐳 = 𝑠 (𝐱 - 𝐲)$ and $𝐳 -$ $𝐲 =$ $(1 - 𝑠) (𝐱 - 𝐲)$ . Since $𝑠, 1 - 𝑠 \geq 0$ , Proposition 6.1.27 gives

‖ 𝐱 - 𝐳 ‖ + ‖ 𝐳 - 𝐲 ‖ = (𝑠 + (1 - 𝑠)) ‖ 𝐱 - 𝐲 ‖ = ‖ 𝐱 - 𝐲 ‖ .

$𝐿 \subseteq 𝑆$ . Let $𝐳 \in 𝐿$ , and put $𝐮 = 𝐱 - 𝐳$ and $𝐯 = 𝐳 - 𝐲$ , so that $𝐮 + 𝐯 = 𝐱 - 𝐲$ and $‖ 𝐮 + 𝐯 ‖ = ‖ 𝐮 ‖ + ‖ 𝐯 ‖$ . By Theorem 6.2.11(b), one of $𝐮, 𝐯$ is a nonnegative real multiple of the other.

If $𝐯 = 𝑎 𝐮$ with $𝑎 \geq 0$ , then $(1 + 𝑎) 𝐮 = 𝐱 - 𝐲$ , so $𝐮 = 𝑠 (𝐱 - 𝐲)$ with $𝑠 = 1 / (1 + 𝑎) \in (0, 1]$ , and $𝐳 =$ $𝐱 -$ $𝐮 =$ $(1 - 𝑠) 𝐱 +$ $𝑠 𝐲$ . If $𝐮 = 𝑏 𝐯$ with $𝑏 \geq 0$ , then $(1 + 𝑏) 𝐯 = 𝐱 - 𝐲$ , so $𝐯 = (1 - 𝑠) (𝐱 - 𝐲)$ with $1 - 𝑠 = 1 / (1 + 𝑏)$ , that is, $𝑠 =$ $𝑏 / (1 + 𝑏) \in$ $[0, 1)$ , and $𝐳 =$ $𝐲 +$ $𝐯 =$ $(1 - 𝑠) 𝐱 +$ $𝑠 𝐲$ . In both cases $𝐳 \in 𝑆$ .

Exercise 6.2.43.standard

(a)Prove Lagrange’s identity: for real numbers $𝑎_{1},$ $\dots,$ $𝑎_{𝑛},$ $𝑏_{1},$ $\dots,$ $𝑏_{𝑛}$ ,
$\begin{aligned} (\sum_{𝑘 = 1}^{𝑛} 𝑎_{𝑘}^{2}) (\sum_{𝑘 = 1}^{𝑛} 𝑏_{𝑘}^{2}) - {(\sum_{𝑘 = 1}^{𝑛} 𝑎_{𝑘} 𝑏_{𝑘})}^{2} \\ = \sum_{1 \leq 𝑗 < 𝑘 \leq 𝑛} {(𝑎_{𝑗} 𝑏_{𝑘} - 𝑎_{𝑘} 𝑏_{𝑗})}^{2} . \end{aligned}$
(b)Deduce the Cauchy–Schwarz inequality for the dot product on $ℝ^{𝑛}$ , including its equality case, without using Theorem 6.2.1.
(c)For $𝑛 = 2$ , compare (a) with (6.2.2), and conclude that the parallelogram spanned by $𝐚, 𝐛 \in ℝ^{2}$ has area $| 𝑎_{1} 𝑏_{2} - 𝑎_{2} 𝑏_{1} |$ .

Solution.

Uses. Remark 6.2.5 (The size of the gap), Remark 6.2.2

Techniques. A sum of nonnegative terms (Technique 6.1.9)

Hint 1. Write both products as double sums over all pairs

(𝑗, 𝑘)

, and subtract.

Hint 2. The terms with

𝑗 = 𝑘

cancel. Group the term for

(𝑗, 𝑘)

with the term for

(𝑘, 𝑗)

Hint 3. For the equality case in (b), all the numbers

𝑎_{𝑗} 𝑏_{𝑘} - 𝑎_{𝑘} 𝑏_{𝑗}

vanish. If some

𝑎_{𝑝} \neq 0

, show

𝐛 = (𝑏_{𝑝} / 𝑎_{𝑝}) 𝐚

Sketch. The left side is

\sum_{𝑗, 𝑘} (𝑎_{𝑗}^{2} 𝑏_{𝑘}^{2} - 𝑎_{𝑗} 𝑏_{𝑗} 𝑎_{𝑘} 𝑏_{𝑘})

. Diagonal terms vanish, and the terms for

(𝑗, 𝑘)

and

(𝑘, 𝑗)

with

𝑗 < 𝑘

add to

{(𝑎_{𝑗} 𝑏_{𝑘} - 𝑎_{𝑘} 𝑏_{𝑗})}^{2}

. A sum of squares is nonnegative and vanishes iff each term does, which says that

𝐚, 𝐛

are dependent. For

𝑛 = 2

the right side is

{(𝑎_{1} 𝑏_{2} - 𝑎_{2} 𝑏_{1})}^{2}

, and (6.2.2) identifies the left side with the square of base times height.

(a) Expanding the products as double sums over all pairs $(𝑗, 𝑘)$ ,

\begin{aligned} (\sum_{𝑗} 𝑎_{𝑗}^{2}) (\sum_{𝑘} 𝑏_{𝑘}^{2}) - (\sum_{𝑗} 𝑎_{𝑗} 𝑏_{𝑗}) (\sum_{𝑘} 𝑎_{𝑘} 𝑏_{𝑘}) \\ = \sum_{𝑗, 𝑘} (𝑎_{𝑗}^{2} 𝑏_{𝑘}^{2} - 𝑎_{𝑗} 𝑏_{𝑗} 𝑎_{𝑘} 𝑏_{𝑘}) . \end{aligned}

The terms with $𝑗 = 𝑘$ are $𝑎_{𝑗}^{2} 𝑏_{𝑗}^{2} -$ $𝑎_{𝑗}^{2} 𝑏_{𝑗}^{2} =$ $0$ . For $𝑗 < 𝑘$ , the terms for $(𝑗, 𝑘)$ and $(𝑘, 𝑗)$ add to

𝑎_{𝑗}^{2} 𝑏_{𝑘}^{2} + 𝑎_{𝑘}^{2} 𝑏_{𝑗}^{2} - 2 𝑎_{𝑗} 𝑏_{𝑗} 𝑎_{𝑘} 𝑏_{𝑘} = {(𝑎_{𝑗} 𝑏_{𝑘} - 𝑎_{𝑘} 𝑏_{𝑗})}^{2} .

Summing over $𝑗 < 𝑘$ gives the identity.

(b) The right side of (a) is a sum of squares, so it is nonnegative, which is the Cauchy–Schwarz inequality ${(𝐚 \cdot 𝐛)}^{2} \leq$ ${‖ 𝐚 ‖}^{2} {‖ 𝐛 ‖}^{2}$ . By Technique 6.1.9, equality holds if and only if $𝑎_{𝑗} 𝑏_{𝑘} = 𝑎_{𝑘} 𝑏_{𝑗}$ for all $𝑗 < 𝑘$ , hence for all $𝑗, 𝑘$ . We show this happens if and only if $𝐚, 𝐛$ are linearly dependent. If $𝐚 = 𝟎$ , both hold. Otherwise pick $𝑝$ with $𝑎_{𝑝} \neq 0$ . If $𝑎_{𝑝} 𝑏_{𝑘} = 𝑎_{𝑘} 𝑏_{𝑝}$ for all $𝑘$ , then $𝐛 = (𝑏_{𝑝} / 𝑎_{𝑝}) 𝐚$ . Conversely, if $𝐛 = 𝑐 𝐚$ , then $𝑎_{𝑗} 𝑏_{𝑘} -$ $𝑎_{𝑘} 𝑏_{𝑗} =$ $𝑐 (𝑎_{𝑗} 𝑎_{𝑘} - 𝑎_{𝑘} 𝑎_{𝑗}) =$ $0$ , and if $𝐚 = 𝑐 𝐛$ the same computation applies. By Remark 6.2.2, this is the equality case of Theorem 6.2.1.

(c) For $𝑛 = 2$ there is one pair $𝑗 < 𝑘$ , and (a) reads ${‖ 𝐚 ‖}^{2} {‖ 𝐛 ‖}^{2} -$ ${(𝐚 \cdot 𝐛)}^{2} =$ ${(𝑎_{1} 𝑏_{2} - 𝑎_{2} 𝑏_{1})}^{2}$ . If $𝐛 \neq 𝟎$ , (6.2.2) says the left side is ${‖ 𝐛 ‖}^{2} {‖ 𝐚 - 𝑐 𝐛 ‖}^{2}$ , the square of (base) $\times$ (height) for the parallelogram on $𝐚, 𝐛$ with base $𝐛$ . So the area is $| 𝑎_{1} 𝑏_{2} - 𝑎_{2} 𝑏_{1} |$ ; if $𝐛 = 𝟎$ , both are $0$ . This is $| det (\begin{matrix} 𝑎_{1} & 𝑏_{1} \\ 𝑎_{2} & 𝑏_{2} \end{matrix}) |$ .

Exercise 6.2.44.hard Let $𝐱_{1}, \dots, 𝐱_{𝑛}$ be vectors in an inner product space, not all $𝟎$ . Show that

‖ 𝐱_{1} + \dots + 𝐱_{𝑛} ‖ = ‖ 𝐱_{1} ‖ + \dots + ‖ 𝐱_{𝑛} ‖

if and only if there is a unit vector $𝐮$ and real numbers $𝑟_{1}, \dots, 𝑟_{𝑛} \geq 0$ with $𝐱_{𝑘} = 𝑟_{𝑘} 𝐮$ for every $𝑘$ .

Solution.

Uses. Theorem 6.2.11 (Triangle inequality), Theorem 6.2.1 (Cauchy–Schwarz inequality), Notation 6.1.1 (Real and complex scalars)

Hint 1. The “if” direction is a computation. For “only if”, first show that the sum

𝐬 = 𝐱_{1} + \dots + 𝐱_{𝑛}

is nonzero, and let

𝐮 = 𝐬 / ‖ 𝐬 ‖

Hint 2. Write

‖ 𝐬 ‖ =

⟨ 𝐬, 𝐮 ⟩ =

\sum_{𝑘} ⟨ 𝐱_{𝑘}, 𝐮 ⟩

, take real parts, and bound each term by

‖ 𝐱_{𝑘} ‖

Hint 3. Equality throughout forces

⟨ 𝐱_{𝑘}, 𝐮 ⟩ =

‖ 𝐱_{𝑘} ‖ ‖ 𝐮 ‖

for each

𝑘

. Apply Theorem 6.2.11(b).

Sketch. If

𝐱_{𝑘} = 𝑟_{𝑘} 𝐮

, both sides equal

\sum_{𝑘} 𝑟_{𝑘}

. Conversely,

𝐬 \neq 𝟎

because

‖ 𝐬 ‖ = \sum_{𝑘} ‖ 𝐱_{𝑘} ‖ > 0

. With

𝐮 = 𝐬 / ‖ 𝐬 ‖

‖ 𝐬 ‖ =

\sum_{𝑘} Re ⟨ 𝐱_{𝑘}, 𝐮 ⟩ \leq

\sum_{𝑘} ‖ 𝐱_{𝑘} ‖ =

‖ 𝐬 ‖

, so

Re ⟨ 𝐱_{𝑘}, 𝐮 ⟩ =

| ⟨ 𝐱_{𝑘}, 𝐮 ⟩ | =

‖ 𝐱_{𝑘} ‖

for each

𝑘

, and Theorem 6.2.11(b) makes

𝐱_{𝑘}

a nonnegative multiple of

𝐮

Suppose $𝐱_{𝑘} = 𝑟_{𝑘} 𝐮$ with $‖ 𝐮 ‖ = 1$ and $𝑟_{𝑘} \geq 0$ . Then $‖ \sum_{𝑘} 𝐱_{𝑘} ‖ =$ $‖ (\sum_{𝑘} 𝑟_{𝑘}) 𝐮 ‖ =$ $\sum_{𝑘} 𝑟_{𝑘} =$ $\sum_{𝑘} ‖ 𝐱_{𝑘} ‖$ .

Conversely, suppose equality holds, and let $𝐬 = 𝐱_{1} + \dots + 𝐱_{𝑛}$ . Since not all $𝐱_{𝑘}$ are $𝟎$ , $‖ 𝐬 ‖ = \sum_{𝑘} ‖ 𝐱_{𝑘} ‖ > 0$ , so $𝐬 \neq 𝟎$ . Let $𝐮 = 𝐬 / ‖ 𝐬 ‖$ , a unit vector with $⟨ 𝐬, 𝐮 ⟩ = ‖ 𝐬 ‖$ . By linearity in the first slot, $Re 𝑧 \leq | 𝑧 |$ , and Theorem 6.2.1,

\begin{aligned} ‖ 𝐬 ‖ & = Re ⟨ 𝐬, 𝐮 ⟩ = \sum_{𝑘} Re ⟨ 𝐱_{𝑘}, 𝐮 ⟩ \\ \leq \sum_{𝑘} | ⟨ 𝐱_{𝑘}, 𝐮 ⟩ | \leq \sum_{𝑘} ‖ 𝐱_{𝑘} ‖ = ‖ 𝐬 ‖ . \end{aligned}

So equality holds throughout. Since each term satisfies $Re ⟨ 𝐱_{𝑘}, 𝐮 ⟩ \leq$ $| ⟨ 𝐱_{𝑘}, 𝐮 ⟩ | \leq$ $‖ 𝐱_{𝑘} ‖$ , equality of the sums forces equality in every term. As in the proof of Theorem 6.2.11, $Re 𝑧 = | 𝑧 |$ means that $𝑧$ is a nonnegative real number, so $⟨ 𝐱_{𝑘}, 𝐮 ⟩ =$ $‖ 𝐱_{𝑘} ‖ =$ $‖ 𝐱_{𝑘} ‖ ‖ 𝐮 ‖$ . By Theorem 6.2.11(b), one of $𝐱_{𝑘}, 𝐮$ is a nonnegative real multiple of the other. If $𝐮 = 𝑏 𝐱_{𝑘}$ with $𝑏 \geq 0$ , then $𝑏 > 0$ since $𝐮 \neq 𝟎$ , and $𝐱_{𝑘} = 𝑏^{- 1} 𝐮$ . So in every case $𝐱_{𝑘} = 𝑟_{𝑘} 𝐮$ with $𝑟_{𝑘} \geq 0$ .

Exercise 6.2.45.hard Let $𝑉$ be a real inner product space, and write $𝜃 (𝐱, 𝐲)$ for the angle between nonzero vectors (Definition 6.2.16). Show that for all nonzero $𝐱, 𝐲, 𝐳 \in 𝑉$ ,

𝜃 (𝐱, 𝐳) \leq 𝜃 (𝐱, 𝐲) + 𝜃 (𝐲, 𝐳) .

(Angles measure distances on the unit sphere, and this is their triangle inequality.)

Solution.

Uses. Definition 6.2.16 (Angle), Theorem 6.2.1 (Cauchy–Schwarz inequality), (6.2.1)

Techniques. Orthogonal decomposition (Technique 6.2.4)

Hint 1. Angles do not change when vectors are scaled by positive numbers, so assume

𝐱, 𝐲, 𝐳

are unit vectors. Let

𝛼 = 𝜃 (𝐱, 𝐲)

and

𝛽 = 𝜃 (𝐲, 𝐳)

. If

𝛼 + 𝛽 \geq 𝜋

there is nothing to prove.

Hint 2. Decompose

𝐱 = (cos 𝛼) 𝐲 + 𝐰

and

𝐳 = (cos 𝛽) 𝐲 + 𝐰^{'}

with

𝐰, 𝐰^{'}

orthogonal to

𝐲

. Compute

‖ 𝐰 ‖

and

‖ 𝐰^{'} ‖

Hint 3. Expand

⟨ 𝐱, 𝐳 ⟩

and bound

⟨ 𝐰, 𝐰^{'} ⟩

below with Cauchy–Schwarz. You should get

⟨ 𝐱, 𝐳 ⟩ \geq

cos (𝛼 + 𝛽)

Sketch. Normalize. With

𝛼, 𝛽

as above,

𝐱 = (cos 𝛼) 𝐲 + 𝐰

and

𝐳 = (cos 𝛽) 𝐲 + 𝐰^{'}

, where

𝐰, 𝐰^{'} ⟂ 𝐲

‖ 𝐰 ‖ = sin 𝛼

‖ 𝐰^{'} ‖ = sin 𝛽

. Then

⟨ 𝐱, 𝐳 ⟩ =

cos 𝛼 cos 𝛽 +

⟨ 𝐰, 𝐰^{'} ⟩

, and Cauchy–Schwarz bounds the last term below by

- sin 𝛼 sin 𝛽

, so

⟨ 𝐱, 𝐳 ⟩ \geq

cos (𝛼 + 𝛽)

. If

𝛼 + 𝛽 \leq 𝜋

, since

cos

is decreasing on

[0, 𝜋]

𝜃 (𝐱, 𝐳) \leq 𝛼 + 𝛽

; otherwise the claim is trivial since

𝜃 \leq 𝜋

For $𝑠, 𝑡 > 0$ , $⟨ 𝑠 𝐱, 𝑡 𝐲 ⟩ / (‖ 𝑠 𝐱 ‖ ‖ 𝑡 𝐲 ‖) =$ $⟨ 𝐱, 𝐲 ⟩ / (‖ 𝐱 ‖ ‖ 𝐲 ‖)$ , so angles are unchanged by positive scaling, and we may assume $‖ 𝐱 ‖ = ‖ 𝐲 ‖ = ‖ 𝐳 ‖ = 1$ . Let $𝛼 = 𝜃 (𝐱, 𝐲)$ and $𝛽 = 𝜃 (𝐲, 𝐳)$ , both in $[0, 𝜋]$ . If $𝛼 + 𝛽 > 𝜋$ , the claim holds because $𝜃 (𝐱, 𝐳) \leq 𝜋$ . Assume $𝛼 + 𝛽 \leq 𝜋$ .

By Definition 6.2.16, $⟨ 𝐱, 𝐲 ⟩ = cos 𝛼$ and $⟨ 𝐳, 𝐲 ⟩ = cos 𝛽$ . As in Technique 6.2.4 (with $‖ 𝐲 ‖ = 1$ ), let $𝐰 = 𝐱 - (cos 𝛼) 𝐲$ and $𝐰^{'} = 𝐳 - (cos 𝛽) 𝐲$ , so that $⟨ 𝐰, 𝐲 ⟩ =$ $⟨ 𝐰^{'}, 𝐲 ⟩ =$ $0$ . By (6.2.1), ${‖ 𝐰 ‖}^{2} =$ $1 -$ ${cos}^{2} 𝛼 =$ ${sin}^{2} 𝛼$ , so $‖ 𝐰 ‖ = sin 𝛼$ , as $sin 𝛼 \geq 0$ on $[0, 𝜋]$ ; similarly $‖ 𝐰^{'} ‖ = sin 𝛽$ . Expanding, and using that the cross terms $⟨ 𝐲, 𝐰^{'} ⟩$ and $⟨ 𝐰, 𝐲 ⟩$ vanish,

\begin{aligned} ⟨ 𝐱, 𝐳 ⟩ & = ⟨ (cos 𝛼) 𝐲 + 𝐰, (cos 𝛽) 𝐲 + 𝐰^{'} ⟩ \\ = cos 𝛼 cos 𝛽 + ⟨ 𝐰, 𝐰^{'} ⟩ . \end{aligned}

By Theorem 6.2.1, $⟨ 𝐰, 𝐰^{'} ⟩ \geq$ $- ‖ 𝐰 ‖ ‖ 𝐰^{'} ‖ =$ $- sin 𝛼 sin 𝛽$ . So

\begin{aligned} cos 𝜃 (𝐱, 𝐳) = ⟨ 𝐱, 𝐳 ⟩ & \geq cos 𝛼 cos 𝛽 - sin 𝛼 sin 𝛽 \\ = cos (𝛼 + 𝛽) . \end{aligned}

Both $𝜃 (𝐱, 𝐳)$ and $𝛼 + 𝛽$ lie in $[0, 𝜋]$ , where $cos$ is strictly decreasing, so $𝜃 (𝐱, 𝐳) \leq 𝛼 + 𝛽$ .

Exercise 6.2.46.hard Find the minimum of $\int_{0}^{1} {𝑓 (𝑡)}^{2} d 𝑡$ over all $𝑓 \in 𝐶 [0, 1]$ with

\int_{0}^{1} 𝑓 (𝑡) d 𝑡 = 1 and \int_{0}^{1} 𝑡 𝑓 (𝑡) d 𝑡 = 1,

and all functions attaining it.

Solution.

Uses. Theorem 6.2.1 (Cauchy–Schwarz inequality), Proposition 6.1.13 (Integral inner products)

Techniques. Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1. The constraints say

⟨ 𝑓, 1 ⟩ = ⟨ 𝑓, 𝑡 ⟩ = 1

. So

⟨ 𝑓, 𝑔 ⟩

is known for every

𝑔 = 𝛼 + 𝛽 𝑡

. Apply Cauchy–Schwarz to

𝑓

and such a

𝑔

Hint 2. The bound is

{‖ 𝑓 ‖}^{2} \geq

{(𝛼 + 𝛽)}^{2} / {‖ 𝛼 + 𝛽 𝑡 ‖}^{2}

. The best

𝑔

should itself satisfy the constraints, by the equality case.

Hint 3. Find

𝑔 = 𝛼 + 𝛽 𝑡

with

⟨ 𝑔, 1 ⟩ = ⟨ 𝑔, 𝑡 ⟩ = 1

: you get

𝑔 = 6 𝑡 - 2

Sketch. For

𝑔 = 6 𝑡 - 2

⟨ 𝑓, 𝑔 ⟩ = 6 - 2 = 4

and

{‖ 𝑔 ‖}^{2} = 4

, so

16 \leq 4 {‖ 𝑓 ‖}^{2}

, that is,

\int 𝑓^{2} \geq 4

. The function

𝑔

satisfies both constraints and has

\int 𝑔^{2} = 4

. Equality forces

𝑓 = 𝑐 𝑔

, and the constraints give

𝑐 = 1

. So the minimum is

4

, attained only by

𝑓 (𝑡) = 6 𝑡 - 2

Let $𝑔 (𝑡) = 6 𝑡 - 2$ . In $𝐶 [0, 1]$ with $⟨ 𝑓, ℎ ⟩ = \int_{0}^{1} 𝑓 ℎ$ ,

\begin{aligned} ⟨ 𝑓, 𝑔 ⟩ & = 6 \int_{0}^{1} 𝑡 𝑓 - 2 \int_{0}^{1} 𝑓 = 6 - 2 = 4, \\ {‖ 𝑔 ‖}^{2} & = \int_{0}^{1} (36 𝑡^{2} - 24 𝑡 + 4) d 𝑡 = 12 - 12 + 4 = 4, \end{aligned}

for every $𝑓$ satisfying the constraints. By Theorem 6.2.1, $16 =$ ${⟨ 𝑓, 𝑔 ⟩}^{2} \leq$ ${‖ 𝑓 ‖}^{2} {‖ 𝑔 ‖}^{2} =$ $4 {‖ 𝑓 ‖}^{2}$ , so $\int_{0}^{1} 𝑓^{2} \geq 4$ .

The function $𝑔$ itself satisfies the constraints: $\int_{0}^{1} (6 𝑡 - 2) d 𝑡 =$ $3 -$ $2 =$ $1$ and $\int_{0}^{1} (6 𝑡^{2} - 2 𝑡) d 𝑡 =$ $2 -$ $1 =$ $1$ . And $\int_{0}^{1} 𝑔^{2} = 4$ . So the minimum is $4$ . If $𝑓$ attains it, equality holds in Cauchy–Schwarz, so $𝑓 = 𝑐 𝑔$ (as $𝑔 \neq 𝟎$ ), and $1 =$ $\int_{0}^{1} 𝑓 =$ $𝑐 \int_{0}^{1} 𝑔 =$ $𝑐$ . So $𝑓 = 𝑔$ is the only minimizer.

Here is how to find $𝑔$ . The constraints determine $⟨ 𝑓, 𝛼 + 𝛽 𝑡 ⟩ =$ $𝛼 +$ $𝛽$ for all $𝛼, 𝛽$ , so each $𝑔 = 𝛼 + 𝛽 𝑡 \neq 𝟎$ gives the bound ${‖ 𝑓 ‖}^{2} \geq$ ${(𝛼 + 𝛽)}^{2} / {‖ 𝑔 ‖}^{2}$ . If some such $𝑔$ satisfies the constraints itself, then $𝑓 = 𝑔$ attains its own bound, and no $𝑓$ can do better. Solving $⟨ 𝛼 + 𝛽 𝑡, 1 ⟩ =$ $𝛼 +$ $𝛽 / 2 =$ $1$ and $⟨ 𝛼 + 𝛽 𝑡, 𝑡 ⟩ =$ $𝛼 / 2 +$ $𝛽 / 3 =$ $1$ gives $𝛽 = 6$ and $𝛼 = - 2$ . (In the language of Section 6.4, the minimizer is the vector of least norm in an affine subspace, and it lies in $span (1, 𝑡)$ .)

Exercise 6.2.47.hard (Nesbitt’s inequality.) Show that for all positive real numbers $𝑎, 𝑏, 𝑐$ ,

\frac{𝑎}{𝑏 + 𝑐} + \frac{𝑏}{𝑐 + 𝑎} + \frac{𝑐}{𝑎 + 𝑏} \geq \frac{3}{2},

with equality if and only if $𝑎 = 𝑏 = 𝑐$ .

Solution.

Uses. Corollary 6.2.26 (Engel’s form)

Techniques. A sum of nonnegative terms (Technique 6.1.9), Apply Cauchy–Schwarz to a well-chosen inner product (Technique 6.2.21)

Hint 1. Engel’s form (Corollary 6.2.26) needs squares in the numerators. Multiply the numerator and denominator of

𝑎 / (𝑏 + 𝑐)

𝑎

Hint 2. You get a lower bound

{(𝑎 + 𝑏 + 𝑐)}^{2} / (2 (𝑎 𝑏 + 𝑏 𝑐 + 𝑐 𝑎))

. Compare

{(𝑎 + 𝑏 + 𝑐)}^{2}

with

3 (𝑎 𝑏 + 𝑏 𝑐 + 𝑐 𝑎)

Sketch. Write

𝑎 / (𝑏 + 𝑐) =

𝑎^{2} / (𝑎 (𝑏 + 𝑐))

. By Corollary 6.2.26, the sum is at least

{(𝑎 + 𝑏 + 𝑐)}^{2} / (2 (𝑎 𝑏 + 𝑏 𝑐 + 𝑐 𝑎))

. Finally

{(𝑎 + 𝑏 + 𝑐)}^{2} \geq

3 (𝑎 𝑏 + 𝑏 𝑐 + 𝑐 𝑎)

, because the difference is half the sum of the squares

{(𝑎 - 𝑏)}^{2}

{(𝑏 - 𝑐)}^{2}

{(𝑐 - 𝑎)}^{2}

. Equality in the last step forces

𝑎 = 𝑏 = 𝑐

, and then equality holds.

Since $𝑎, 𝑏, 𝑐 > 0$ , the denominators $𝑎 (𝑏 + 𝑐)$ , $𝑏 (𝑐 + 𝑎)$ , $𝑐 (𝑎 + 𝑏)$ are positive. By Corollary 6.2.26,

\begin{aligned} \frac{𝑎}{𝑏 + 𝑐} + \frac{𝑏}{𝑐 + 𝑎} + \frac{𝑐}{𝑎 + 𝑏} \\ = \frac{𝑎^{2}}{𝑎 (𝑏 + 𝑐)} + \frac{𝑏^{2}}{𝑏 (𝑐 + 𝑎)} + \frac{𝑐^{2}}{𝑐 (𝑎 + 𝑏)} \\ \geq \frac{{(𝑎 + 𝑏 + 𝑐)}^{2}}{2 (𝑎 𝑏 + 𝑏 𝑐 + 𝑐 𝑎)}, \end{aligned}

since the denominators add up to $2 (𝑎 𝑏 + 𝑏 𝑐 + 𝑐 𝑎)$ . Now

\begin{aligned} {(𝑎 + 𝑏 + 𝑐)}^{2} - 3 (𝑎 𝑏 + 𝑏 𝑐 + 𝑐 𝑎) & = 𝑎^{2} + 𝑏^{2} + 𝑐^{2} - 𝑎 𝑏 - 𝑏 𝑐 - 𝑐 𝑎 \\ = \frac{1}{2} ({(𝑎 - 𝑏)}^{2} + {(𝑏 - 𝑐)}^{2} + {(𝑐 - 𝑎)}^{2}) \geq 0, \end{aligned}

so the right side is at least $3 (𝑎 𝑏 + 𝑏 𝑐 + 𝑐 𝑎) / (2 (𝑎 𝑏 + 𝑏 𝑐 + 𝑐 𝑎)) =$ $3 / 2$ .

If equality holds, the last step is an equality, so ${(𝑎 - 𝑏)}^{2} +$ ${(𝑏 - 𝑐)}^{2} +$ ${(𝑐 - 𝑎)}^{2} =$ $0$ , and $𝑎 = 𝑏 = 𝑐$ by Technique 6.1.9. Conversely, for $𝑎 = 𝑏 = 𝑐$ each term is $1 / 2$ .

Exercise 6.2.48.challenge (Ptolemy’s inequality.)

(a)Show that for nonzero vectors $𝐱, 𝐲$ in an inner product space,
$‖ \frac{𝐱}{{‖ 𝐱 ‖}^{2}} - \frac{𝐲}{{‖ 𝐲 ‖}^{2}} ‖ = \frac{‖ 𝐱 - 𝐲 ‖}{‖ 𝐱 ‖ ‖ 𝐲 ‖} .$
(b)Deduce that for all $𝐱, 𝐲, 𝐳, 𝐰$ in an inner product space,
$‖ 𝐱 - 𝐲 ‖ ‖ 𝐳 - 𝐰 ‖ \leq ‖ 𝐱 - 𝐳 ‖ ‖ 𝐲 - 𝐰 ‖ + ‖ 𝐱 - 𝐰 ‖ ‖ 𝐲 - 𝐳 ‖ .$
(For four points in the plane: the product of the diagonals of a quadrilateral is at most the sum of the products of opposite sides.)

Solution.

Uses. Proposition 6.1.29 (Expanding a norm), Corollary 6.2.14 (Distances and the reverse triangle inequality), Proposition 6.1.27 (Basic properties of the norm)

Hint 1. (a) Expand the left side squared with Proposition 6.1.29; the coefficients

1 / {‖ 𝐱 ‖}^{2}

and

1 / {‖ 𝐲 ‖}^{2}

are real.

Hint 2. (b) Translating all four points by

- 𝐰

changes no distance, so you may assume

𝐰 = 𝟎

. Dispose of the case where one of

𝐱, 𝐲, 𝐳

𝟎

Hint 3. Apply the triangle inequality to the three points

𝐯 / {‖ 𝐯 ‖}^{2}

for

𝐯 = 𝐱, 𝐲, 𝐳

, rewrite each distance with (a), and clear denominators.

Sketch. (a) Expand the square of the left side with Proposition 6.1.29 and put everything over

{‖ 𝐱 ‖}^{2} {‖ 𝐲 ‖}^{2}

; the numerator is

{‖ 𝐱 - 𝐲 ‖}^{2}

. (b) Translate so that

𝐰 = 𝟎

; the cases with a zero vector are immediate. Otherwise, let

𝜄 (𝐯) = 𝐯 / {‖ 𝐯 ‖}^{2}

. The triangle inequality for the three points

𝜄 (𝐱),

𝜄 (𝐲),

𝜄 (𝐳)

becomes, by (a), the claim divided by

‖ 𝐱 ‖ ‖ 𝐲 ‖ ‖ 𝐳 ‖

(a) Let $𝛼 = 1 / {‖ 𝐱 ‖}^{2}$ and $𝛽 = 1 / {‖ 𝐲 ‖}^{2}$ , both real. By Proposition 6.1.29 (and linearity),

\begin{aligned} {‖ 𝛼 𝐱 - 𝛽 𝐲 ‖}^{2} & = 𝛼^{2} {‖ 𝐱 ‖}^{2} - 2 𝛼 𝛽 Re ⟨ 𝐱, 𝐲 ⟩ + 𝛽^{2} {‖ 𝐲 ‖}^{2} \\ = \frac{1}{{‖ 𝐱 ‖}^{2}} - \frac{2 Re ⟨ 𝐱, 𝐲 ⟩}{{‖ 𝐱 ‖}^{2} {‖ 𝐲 ‖}^{2}} + \frac{1}{{‖ 𝐲 ‖}^{2}} \\ = \frac{{‖ 𝐲 ‖}^{2} - 2 Re ⟨ 𝐱, 𝐲 ⟩ + {‖ 𝐱 ‖}^{2}}{{‖ 𝐱 ‖}^{2} {‖ 𝐲 ‖}^{2}} = \frac{{‖ 𝐱 - 𝐲 ‖}^{2}}{{‖ 𝐱 ‖}^{2} {‖ 𝐲 ‖}^{2}} . \end{aligned}

Taking nonnegative square roots gives (a).

(b) Replacing $𝐱, 𝐲, 𝐳, 𝐰$ by $𝐱 -$ $𝐰,$ $𝐲 -$ $𝐰,$ $𝐳 -$ $𝐰,$ $𝟎$ changes none of the six differences, so we may assume $𝐰 = 𝟎$ . The claim is then

‖ 𝐱 - 𝐲 ‖ ‖ 𝐳 ‖ \leq ‖ 𝐱 - 𝐳 ‖ ‖ 𝐲 ‖ + ‖ 𝐱 ‖ ‖ 𝐲 - 𝐳 ‖ .

If $𝐱 = 𝟎$ , both sides are $‖ 𝐲 ‖ ‖ 𝐳 ‖$ . If $𝐲 = 𝟎$ , both sides are $‖ 𝐱 ‖ ‖ 𝐳 ‖$ . If $𝐳 = 𝟎$ , the left side is $0$ . So assume $𝐱, 𝐲, 𝐳 \neq 𝟎$ , and write $𝜄 (𝐯) = 𝐯 / {‖ 𝐯 ‖}^{2}$ for $𝐯 \neq 𝟎$ . By Corollary 6.2.14(a),

‖ 𝜄 (𝐱) - 𝜄 (𝐲) ‖ \leq ‖ 𝜄 (𝐱) - 𝜄 (𝐳) ‖ + ‖ 𝜄 (𝐳) - 𝜄 (𝐲) ‖,

and by (a) this says

\frac{‖ 𝐱 - 𝐲 ‖}{‖ 𝐱 ‖ ‖ 𝐲 ‖} \leq \frac{‖ 𝐱 - 𝐳 ‖}{‖ 𝐱 ‖ ‖ 𝐳 ‖} + \frac{‖ 𝐳 - 𝐲 ‖}{‖ 𝐳 ‖ ‖ 𝐲 ‖} .

Multiplying by $‖ 𝐱 ‖ ‖ 𝐲 ‖ ‖ 𝐳 ‖ > 0$ gives the claim, since $‖ 𝐳 - 𝐲 ‖ = ‖ 𝐲 - 𝐳 ‖$ .

The map $𝜄$ is inversion in the unit sphere, and (a) says how it distorts distances. Ptolemy’s inequality is the triangle inequality, seen after inversion.

More practice

The practice bank has 23 exercises on this section’s topics.