Matrix World in “Linear Algebra for Everyone”

Matrix World

Pick favorite colors and download full screen images. One is in Japanese.

As a lifetime math lover, I’m very glad to see that Prof. Gilbert Strang recently published his new book, “Linear Algebra for Everyone.”

Gil Strang’s new book

This book is the most easy-to-read sequel to his legend “Introduction to Linear Algebra.” He reconstructed the contents aligning the 5 factorization of a matrix and the big picture(4 subspaces) of Linear Algebra as described in “A 2020 Vision of Linear Algebra“(where 6 factorization is introduced). For more information for these books and concepts;

As a big fan of the both of these books, I tried categorization of matrices which appear in his work, with examples, key properties and the factorizations. And the above drawing is what I came up with!

Legend

  • A    General Matrix
  • C    Independent Column Matrix in A = CR
  • R    Reduced Row Echelon form in A = CR, Upper Triangular Gram-Schmidt factors(QR)
  • S    Symmetric Matrix
  • Q    Orthogonal Matrix
  • L     Lower Triangular Matrix
  • U    Upper Triangular Matrix in LU or
    Orthonormal Basis Matrix of column space in SVD
  • P    Permutation Matrix in PA = LU or Projection Matrix
  • Λ    Diagonal Matrix  with eigenvalues
  • Σ    Diagonal Matrix with singular values
  • V    Orthonormal Basis Matrix of row space in SVD
  • J      Jordan Form Matrix
  • I      Identity Matrix
  • O    Zero Matrix
  • 𝜆     Eigenvalue

Six Factorizations of Matrix

  • A = CR (Column Row)         Independent Columns in C
  • PA = LU (LU )                        Lower and Upper Triangular
  • A = QR(Gram-Schmidt)      Orthogonal Columns in Q with Upper triangular R
  • A = XΛX -1(Diagonalization) Eigenvalues in Λ, Eigenvectors in  X
  • S = QΛQ T(Diagonalization)    Eigenvalues in Λ, Orthogonal Eigenvectors in Q
  • A=UΣVT (SVD)              Singular values in Diagonal Σ, Orthonormal vectors in U, V     

Here’s thoughts behind the drawing.

  • All Matrices(m×n) can be factorized to A=CR  (independent columns and rows to see the column rank quals the row rank).
  • All Matrices(m×n) can be factorized to A=UΣVT(SVD) with orthonormal basis U and V in each space.
  • Square Matrices(n×n) are either Invertible or Singular.
  • Invertibility can be checked by whether A=LU has full pivotsdet⁡(A) ≠ 0, or all the eigenvalues are none-zero.
  • An Invertible Matrix is factorized to  A=QR using Orthogonal Matrix Q (Gram-Schmidt). 
  • A Square Matrix is either Diagonalizable or not. If not, still Jordan-decomposable.
  • A Diagonalizable Matrix is factorized to A=XΛX1 with a Diagonal Matrix Λ with eigenvalues and an Invertible matrix X.
  • A Non-diagonalizable Matrix is factorized to A=XJX-1 with a Jordan form Matrix J and an Invertible matrix X.
  • A Matrix is Diagonalizable as QΛQT by an Orthogonal Matrix when and only when it is Normal(ATA=AAT).
  • Symmetric Matrices (S=ST) and Orthogonal Matrices (Q1=QT) are typical and important Normal Matrices. (Another is Anti-Symmetric which is not expressed in the drawing)
  • There are some matrix types both Symmetric and Orthogonal. They include Householder, Hadamard and Identity. (not in the drawing)
  • All eigenvalues of an Orthogonal Matrix are |λ|=1.
  • All eigenvalues of a Symmetric Matrix are real.
  • A Symmetric Matrix is called “Positive Semidefinite” if all eigenvalues λ≥0.
  • A Symmetric Matrix is called “Positive Definite” if all eigenvalues λ>0.
  • Projection Matrices(P2 = P = PT) are Symmetric and Positive Semidefinite.
  • All Eigenvalues of a Projection Matrix are λ=1 or 0.
  • A Projection Matrix to the subspace spanned by column vectors of A is P=A(AT A)-1AT. (not expressed in the drawing)
  • AT A for any matrix A is Positive Semidefinite, and even Positive Definite if and only if the columns of A are independent.
  • If a Matrix is Invertible, its inverse is expressed as A-1=VΣ-1UT.
  • Even a Matrix is not Invertible and not even Square, there exists a pseudo inverse A1=VΣ-1UT
  • To extend to Complex Matrices; (not expressed in the drawing)
    • Orthogonal should be extended to Unitary
    • Symmetric should be extended to Hermitian
    • AT should be extended to AH(conjugated and transposed)
  • Lastly, Jordan is placed in Singular world in this map, but it is sometimes invertible, and other times singular(pity that I couldn’t find a better place to put it).

To start this project, I emailed to Prof. Strang, first to confirm the concept, then exchanged more than 20 emails, and finally finished it today. Thank you Prof. Gilbert Strang for the persistent support and up-cheers.

             Thank you Kenji !   I can see that this figure might be interesting to a lot of people. Well done !    This is a great success. I hope you enjoyed this creation.

Very best wishes    Gil

– from personal email Set.29, 2020

God, I really respect him as a great teacher and educator! I first met him on YouTube, MIT 18.06 lecture in OpenCourseWare and got inspired so much that I even made a T-shirt of his Big Picture(4 subspaces) of Linear Algebra.

That led me to the conversations with him and the drawing. For the record, I retained the first version of the drawing by hand.

The first hand drawn version

Any feedback is welcome!

3 thoughts on “Matrix World in “Linear Algebra for Everyone”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s