Matrix Theory
Linear algebra for 21-242 at Carnegie Mellon
Reading this book
Proofs and solutions start collapsed. Reveal them one stage at a time (what the argument uses, then hints, then a sketch) or jump straight to the full argument.
The arrow at the right of a definition, theorem, or exercise folds it down to its first line. Hovering over a reference shows the statement it points to.
Proofs and exercises are marked by difficulty:
- routine
- Follows directly from the definitions; a few lines.
- standard
- Typical homework level; needs one real idea.
- hard
- Upper end of course level; several ideas or a delicate argument.
- challenge
- Harder than the course expects. Skip it if you are targeting course-level problems.
Material marked Beyond 21-242 goes past what the course covers. It is included for completeness and can be skipped.
A PDF version of the whole book is also available.
Contents
Fields, the vector space axioms, span, subspaces, linear independence, bases, and coordinates.
Linear maps, their matrices, matrix multiplication as composition, inverses, kernels and images, and the Fundamental Theorems.
Gauss–Jordan elimination as the engine of the theory, rank, and the Fundamental Theorems completed.
Dimension, rank–nullity, rank inequalities, sums and direct sums, change of basis, quotients, and affine geometry.
The determinant from its axioms: permutations, existence and uniqueness, invertibility, multiplicativity, and cofactor expansion.
Inner products and norms, Cauchy–Schwarz, orthonormal bases, projections, least squares, and adjoints.
Polynomials, eigenvalues and invariant subspaces, the characteristic polynomial, diagonalization, and Schur’s theorem.
Self-adjoint and normal operators, the real and complex spectral theorems, positive operators, and isometries.
Singular values, the SVD and its geometry, its consequences, and a survey of matrix factorizations.
Generalized eigenvectors, nilpotent operators, the generalized eigenspace decomposition, and Jordan canonical form.
Stochastic matrices, Markov chains and stationary distributions, Perron–Frobenius, and reversibility.
Bilinear forms and congruence, quadratic forms, positive definiteness and min–max, and forms over arbitrary fields.
Dual spaces and dual maps, duality for inner product spaces, tensors, and the determinant revisited through alternating forms.
Groups through matrices, field extensions, finite fields, and modules over F[x] with rational canonical form.
Hamel bases, linear algebra inside infinite-dimensional spaces, and a first look at inner product spaces of infinite dimension.
Regression, principal component analysis, low-rank models, quadratic optimization, spectral graph methods, and linear layers.
Functions and inverses, induction, equivalence relations, Zorn’s lemma, and an index of proof techniques.
Arithmetic, conjugation, modulus, polar form, and roots of unity.
Two proofs: one by minimum modulus, one by linear algebra.