Introduction to Linear Algebra
A comprehensive textbook providing a modern introduction to linear algebra, covering vectors, matrices, linear equations, vector spaces, orthogonality, determinants, and eigenvalues, with a focus on both theory and applications.
Lessons
Lesson
This lesson introduces the fundamentals of linear algebra by defining vectors as column components and exploring how scalar multiplication and vector addition create linear combinations. Students learn how these operations form the basis for solving systems of equations and how the concept of span determines the geometric dimensions reachable by a set of vectors.
This lesson explores the dual perspectives of linear systems, contrasting the row-based geometric intersection of planes with the column-based linear combination of vectors. Students will learn to solve $Ax=b$ by interpreting matrices as linear transformations and applying elimination techniques to identify singular matrices and unique solutions.
This lesson introduces vector spaces as collections of objects that satisfy eight fundamental axioms, emphasizing the importance of the zero vector and closure under addition and scalar multiplication. It further explores the column space of a matrix, defining it as the span of its columns and explaining that a system $Ax = b$ is solvable if and only if $b$ lies within that space.
This lesson explores the geometric relationship between the four fundamental subspaces of a matrix, defining them as pairs of orthogonal complements. It further introduces the concept of least squares approximations and projections, explaining how to find the best possible solution to inconsistent systems by minimizing error.
This lesson explores the fundamental properties of determinants, focusing on the product rule, the impact of transposes, and the behavior of orthogonal matrices. It also introduces efficient computation techniques, demonstrating how Gaussian elimination and pivot products simplify the process of finding determinants for complex matrices.
This lesson explores the geometry and algebra of eigenvalues and eigenvectors, explaining how they represent the principal axes of linear transformations where a matrix acts only by scaling. Students will learn to calculate these values using the characteristic equation $\det(A - \lambda I) = 0$ and apply properties like the trace and determinant to analyze matrix behavior and system stability.
This lesson defines linear transformations as mappings that preserve vector addition and scalar multiplication, governed by the principles of additivity and homogeneity. Students learn to identify linear transformations using the "origin test," construct matrices based on how transformations affect basis vectors, and apply these concepts to geometric and algebraic problems.
This lesson explores the pseudoinverse $A^+$ as a powerful tool for solving singular or overdetermined systems, acting as a bridge between the four fundamental subspaces. It further examines the $A^TCA$ framework, demonstrating how this structure models physical equilibrium and statistical least squares by projecting vectors onto the column and row spaces of a matrix.
This lesson explores how Numerical Linear Algebra optimizes computational performance by using homogeneous coordinates to convert affine translations into matrix-vector multiplications. By unifying these operations, developers can leverage hardware-accelerated BLAS routines to process complex geometric transformations with maximum efficiency and structural integrity.
This lesson introduces complex numbers as a foundational extension of the real number system, defined by the identity $i^2 = -1$. By visualizing these numbers as vectors in the complex plane, students learn to perform algebraic operations and interpret complex values as tools for modeling physical phenomena like oscillation and rotation.
Course Overview
📚 Content Summary
A comprehensive textbook providing a modern introduction to linear algebra, covering vectors, matrices, linear equations, vector spaces, orthogonality, determinants, and eigenvalues, with a focus on both theory and applications.
Master the foundations and applications of linear algebra through Gilbert Strang's intuitive and rigorous framework.
Author: Gilbert Strang
Acknowledgments: Massachusetts Institute of Technology; Wellesley-Cambridge Press
🎯 Learning Objectives
- Compute linear combinations, dot products, vector lengths, and angles between vectors.
- Describe the geometric configurations (lines, planes, or volumes) formed by sets of vectors.
- Solve linear equations using matrix-vector products and interpret the role of inverse and singular matrices.
- Differentiate between the row picture (intersecting planes) and column picture (linear combinations of vectors) of a system.
- Execute Gaussian elimination to transform a system into an upper triangular form (U) and solve via back substitution.
- Formalize elimination steps using elementary matrices (E_{ij}) and permutations (P_{ij}).
- Identify whether a subset of vectors satisfies the requirements to be a subspace.
- Perform row reduction to reach the Reduced Row Echelon Form (R) and identify the rank, pivot columns, and free variables.
- Construct the nullspace matrix N from special solutions and describe the complete solution to linear systems.
- Identify and prove the orthogonality of the four fundamental subspaces and determine orthogonal complements.
Lessons
Overview: This lesson covers the foundational pillars of Linear Algebra: vector operations and their geometric interpretations. It transitions from basic linear combinations and dot products to the algebraic structure of matrices, linear equations (Ax = b), and the critical concepts of vector independence and matrix invertibility. Students will learn to navigate between algebraic calculations and the geometric reality of vectors in \mathbb{R}^3.
Learning Outcomes:
- Compute linear combinations, dot products, vector lengths, and angles between vectors.
- Describe the geometric configurations (lines, planes, or volumes) formed by sets of vectors.
- Solve linear equations using matrix-vector products and interpret the role of inverse and singular matrices.
Overview: This lesson covers the transition from the geometric interpretation of linear systems to their computational resolution via matrix algebra. It details the mechanics of Gaussian elimination, the formalization of row operations through elementary matrices, and the culmination of these processes into fundamental matrix factorizations (LU, PA=LU, and LDL^T). The material bridges theoretical linearity with practical implementation, including computational costs and software-specific execution.
Learning Outcomes:
- Differentiate between the row picture (intersecting planes) and column picture (linear combinations of vectors) of a system.
- Execute Gaussian elimination to transform a system into an upper triangular form (U) and solve via back substitution.
- Formalize elimination steps using elementary matrices (E_{ij}) and permutations (P_{ij}).
Overview: This lesson explores the structural backbone of linear algebra, focusing on the definition and requirements of vector spaces and subspaces. Students will learn to solve the equation Ax=0 using the Reduced Row Echelon Form (R) to identify pivot and free variables, which lead to the "special solutions" that form a basis for the nullspace. Finally, the lesson culminates in the Fundamental Theorem of Linear Algebra, connecting the dimensions and properties of the four fundamental subspaces: the column space, row space, nullspace, and left nullspace.
Learning Outcomes:
- Identify whether a subset of vectors satisfies the requirements to be a subspace.
- Perform row reduction to reach the Reduced Row Echelon Form (R) and identify the rank, pivot columns, and free variables.
- Construct the nullspace matrix N from special solutions and describe the complete solution to linear systems.
Overview: This lesson explores the fundamental relationship between the four fundamental subspaces through the lens of orthogonality. Students will learn how to project vectors onto lines and subspaces using projection matrices, solve overdetermined systems via least squares approximations (fitting lines and parabolas), and utilize orthonormal bases and Gram-Schmidt orthogonalization to simplify complex linear algebra problems into A = QR factorizations.
Learning Outcomes:
- Identify and prove the orthogonality of the four fundamental subspaces and determine orthogonal complements.
- Construct projection matrices P and calculate projections of vectors onto lines and higher-dimensional subspaces.
- Apply the Normal Equations (A^T A \hat{x} = A^T b) to find the best-fit line or parabola for a set of data points.
Overview: This lesson explores the algebraic and geometric properties of determinants, transitioning from the pivot-based definition to the "Big Formula" involving permutations and cofactors. Students will apply these concepts to solve linear systems via Cramer's Rule, calculate inverse matrices, and determine areas and volumes in both linear algebra and multivariable calculus (Jacobians).
Learning Outcomes:
- Compute determinants using properties (product rule, transpose), pivot formulas, and cofactor expansion.
- Apply Cramer’s Rule and the cofactor formula to find solutions and matrix inverses.
- Geometrically interpret determinants as areas of triangles/parallelograms and volumes of parallelepipeds, extending to triple products and Jacobians.
Overview: This lesson explores the transformation of matrices into their simplest forms to solve complex problems in linear systems and dynamic equations. Students will learn how to decompose matrices using eigenvalues/eigenvectors (A = S\Lambda S^{-1}) for square matrices and the Singular Value Decomposition (A = U\Sigma V^T) for any matrix, providing the foundation for solving differential equations, testing for local minima, and performing image compression.
Learning Outcomes:
- Solve the eigenvalue equation Ax = \lambda x and relate \lambda to the matrix trace and determinant.
- Diagonalize matrices to calculate powers and solve systems of linear differential equations.
- Identify and test for positive definite matrices and calculate Cholesky factorizations.
Overview: This lesson explores the fundamental shift from viewing matrices as static data arrays to viewing them as dynamic operators called linear transformations. We will define the rules of linearity, examine how transformations map specific shapes (like the "house") in the plane, and learn to represent calculus operations (derivatives and integrals) as matrices. Finally, we conclude with advanced decompositions—including the Pseudoinverse and Polar Decomposition—which extend our ability to invert and factor transformations even when standard inverses do not exist.
Learning Outcomes:
- Define and identify linear transformations using the principles of addition and scalar multiplication.
- Construct the matrix A for a linear transformation T by mapping basis vectors.
- Perform changes of basis for signals (Wavelets and Fourier) and generalize matrix inversion using the Pseudoinverse A^+.
Overview: This lesson explores the practical application of linear algebra across diverse fields including structural engineering, network theory, stochastic processes, optimization, signal processing, and data science. Students will learn how the fundamental framework of A^TCA governs physical systems, how eigenvalues predict long-term population trends, and how orthogonal functions extend vector space concepts to functional analysis and statistics.
Learning Outcomes:
- Model physical systems of springs and masses using the stiffness matrix K = A^TCA.
- Analyze graph connectivity using incidence matrices and verify Euler’s Formula for networks.
- Calculate steady-state vectors for Markov matrices and apply the Perron-Frobenius theorem to population and economic models.
Overview: This lesson explores the practical implementation of linear algebra on computers, focusing on the transition from theoretical exactness to numerical stability and efficiency. Students will learn how to mitigate roundoff errors through partial pivoting, optimize computations using operation counts for sparse and band matrices, and evaluate system sensitivity using condition numbers. Additionally, the lesson covers iterative techniques for solving large-scale systems and methods for approximating eigenvalues.
Learning Outcomes:
- Analyze the impact of roundoff errors and apply partial pivoting to ensure numerical stability in Gaussian elimination.
- Evaluate the computational efficiency of algorithms by calculating operation counts for full, band, and sparse matrices.
- Measure the sensitivity of linear systems using norms and condition numbers, specifically identifying ill-conditioned cases like the Hilbert matrix.
Overview: This lesson covers the transition from real to complex numbers, focusing on their arithmetic, geometric representation in the complex plane, and polar forms. It culminates in the study of complex-specific matrix structures—Hermitian and Unitary matrices—which are the complex-valued counterparts to symmetric and orthogonal matrices.
Learning Outcomes:
- Perform arithmetic operations and find the conjugate/modulus of complex numbers.
- Map complex numbers to the complex plane and convert between rectangular (a + bi) and polar (re^{i\theta}) forms.
- Apply Euler’s Formula to simplify products and powers of complex numbers.