Matrix World in “Linear Algebra for Everyone”

Pick favorite colors and download full screen images. One is in Japanese.

As a lifetime math lover, I’m very glad to see that Prof. Gilbert Strang recently published his new book, “Linear Algebra for Everyone.”

This book is the most easy-to-read sequel to his legend “Introduction to Linear Algebra.” He reconstructed the contents aligning the 5 factorization of a matrix and the big picture(4 subspaces) of Linear Algebra as described in “A 2020 Vision of Linear Algebra“(where 6 factorization is introduced). For more information for these books and concepts;

Linear Algebra for Everyone – http://math.mit.edu/everyone/
Introduction to Linear Algebra – http://math.mit.edu/linearalgebra/
A 2020 Vision of Linear Algebra – https://ocw.mit.edu/resources/res-18-010-a-2020-vision-of-linear-algebra-spring-2020/

As a big fan of the both of these books, I tried categorization of matrices which appear in his work, with examples, key properties and the factorizations. And the above drawing is what I came up with!

Legend

A General Matrix
C Independent Column Matrix in A = CR
R Reduced Row Echelon form in A = CR, Upper Triangular Gram-Schmidt factors(QR)
S Symmetric Matrix
Q Orthogonal Matrix
L Lower Triangular Matrix
U Upper Triangular Matrix in LU or
Orthonormal Basis Matrix of column space in SVD
P Permutation Matrix in PA = LU or Projection Matrix
Λ Diagonal Matrix with eigenvalues
Σ Diagonal Matrix with singular values
V Orthonormal Basis Matrix of row space in SVD
J Jordan Form Matrix
I Identity Matrix
O Zero Matrix
𝜆 Eigenvalue

Six Factorizations of Matrix

A = CR (Column Row) Independent Columns in C
PA = LU (LU ) Lower and Upper Triangular
A = QR(Gram-Schmidt) Orthogonal Columns in Q with Upper triangular R
A = XΛX ^-1(Diagonalization) Eigenvalues in Λ, Eigenvectors in X
S = QΛQ ^T(Diagonalization) Eigenvalues in Λ, Orthogonal Eigenvectors in Q
A=UΣV^T (SVD) Singular values in Diagonal Σ, Orthonormal vectors in U, V

Here’s thoughts behind the drawing.

All Matrices(m×n) can be factorized to A=CR (independent columns and rows to see the column rank quals the row rank).
All Matrices(m×n) can be factorized to A=UΣV^T(SVD) with orthonormal basis U and V in each space.
Square Matrices(n×n) are either Invertible or Singular.
Invertibility can be checked by whether A=LU has full pivots, det⁡(A) ≠ 0, or all the eigenvalues are none-zero.
An Invertible Matrix is factorized to A=QR using Orthogonal Matrix Q (Gram-Schmidt).
A Square Matrix is either Diagonalizable or not. If not, still Jordan-decomposable.
A Diagonalizable Matrix is factorized to A=XΛX^–¹ with a Diagonal Matrix Λ with eigenvalues and an Invertible matrix X.
A Non-diagonalizable Matrix is factorized to A=XJX^-1 with a Jordan form Matrix J and an Invertible matrix X.
A Matrix is Diagonalizable as QΛQ^T by an Orthogonal Matrix when and only when it is Normal(A^TA=AA^T).
Symmetric Matrices (S=S^T) and Orthogonal Matrices (Q^–¹=Q^T) are typical and important Normal Matrices. (Another is Anti-Symmetric which is not expressed in the drawing)
There are some matrix types both Symmetric and Orthogonal. They include Householder, Hadamard and Identity. (not in the drawing)
All eigenvalues of an Orthogonal Matrix are |λ|=1.
All eigenvalues of a Symmetric Matrix are real.
A Symmetric Matrix is called “Positive Semidefinite” if all eigenvalues λ≥0.
A Symmetric Matrix is called “Positive Definite” if all eigenvalues λ>0.
Projection Matrices(P² = P = P^T) are Symmetric and Positive Semidefinite.
All Eigenvalues of a Projection Matrix are λ=1 or 0.
A Projection Matrix to the subspace spanned by column vectors of A is P=A(A^T A)^-1A^T. (not expressed in the drawing)
A^T A for any matrix A is Positive Semidefinite, and even Positive Definite if and only if the columns of A are independent.
If a Matrix is Invertible, its inverse is expressed as A^-1=VΣ^-1U^T.
Even a Matrix is not Invertible and not even Square, there exists a pseudo inverse A^–¹=VΣ^-1U^T
To extend to Complex Matrices; (not expressed in the drawing)
- Orthogonal should be extended to Unitary
- Symmetric should be extended to Hermitian
- A^Tshould be extended to A^H(conjugated and transposed)
Lastly, Jordan is placed in Singular world in this map, but it is sometimes invertible, and other times singular(pity that I couldn’t find a better place to put it).

To start this project, I emailed to Prof. Strang, first to confirm the concept, then exchanged more than 20 emails, and finally finished it today. Thank you Prof. Gilbert Strang for the persistent support and up-cheers.

Thank you Kenji ! I can see that this figure might be interesting to a lot of people. Well done ! This is a great success. I hope you enjoyed this creation.

Very best wishes Gil
– from personal email Set.29, 2020

God, I really respect him as a great teacher and educator! I first met him on YouTube, MIT 18.06 lecture in OpenCourseWare and got inspired so much that I even made a T-shirt of his Big Picture(4 subspaces) of Linear Algebra.