Mathematics reference
Math Wiki
Notes on mathematical concepts, with definitions, worked examples, and connections. Each entry grows as I study and revisit the subject.
Linear Algebra
-
Cholesky decomposition
Understand how geodetic network adjustment led to Cholesky factorization, how its purpose developed, and how to compute it.
-
Conjugate gradient method
Derive conjugate gradients from Hestenes and Stiefel's published system, follow its four steps, and explore residuals, energy error, and preconditioning.
-
Gram–Schmidt orthogonalization
Build perpendicular unit directions by removing projections, preserve successive spans, and understand the difference between classical and modified Gram–Schmidt.
-
Householder reflection
Derive a reflection that zeros a column tail, understand its geometry and stable sign choice, and connect compact reflectors to QR and eigenvalue algorithms.
-
LU decomposition
Factor a matrix into lower and upper triangular matrices to solve linear systems efficiently.
-
Matrix factorization and decomposition
Represent matrices as products of structured factors, and choose between LU, Cholesky, QR, and other decompositions.
-
Matrix norms
Measure matrix amplification with induced norms, distinguish spectral and Frobenius norms, and bound errors in linear systems.
-
QR factorization
Store orthogonalization as A = QR, solve least squares through a triangular system, and compare Gram–Schmidt, Householder reflections, and Givens rotations.
-
Quadratic form
Read xᵀAx as a scalar quadratic expression, understand its geometry, and connect its curvature to matrix factorizations and optimization.
-
Schur complement
Eliminate a block of variables and understand the remaining matrix through linear systems, quadratic energy, and Cholesky factorization.
-
Sparse matrices and storage formats
Represent sparse matrices with COO, CSR, and CSC, read their pointer arrays, and choose a format for assembly or computation.
-
Symmetric positive definite matrix
Recognize positive energy in every direction through quadratic forms, eigenvalues, Cholesky factors, and worked examples.
-
Vector norms
Measure vector size with the 1, 2, and infinity norms, compare their geometry, and choose a meaningful error measure.
Optimization
-
Carathéodory’s Theorem
Compress a convex combination to at most d + 1 points, starting from Carathéodory’s Fourier-coefficient problem and a constructive weight-elimination proof.
-
Convexity
Derive convex sets from mixing production plans, distinguish convex functions from feasible regions, and connect global minima to quadratic curvature.
-
Farkas’ Lemma
Prove that linear constraints are impossible by combining them into a contradiction, and connect the certificate to cones and LP duality.
-
Linear programming
Derive linear models, improving exchanges, and optimality certificates from Kantorovich's machine allocation and Dantzig's assignment problem.
-
Optimization duality
Turn production constraints into optimality certificates, prove LP strong duality with Farkas’ lemma, and derive Lagrange duals and KKT conditions.
-
Simplex method
Derive basis exchanges, reduced costs, and the ratio test from a historical production problem, then certify the answer and find a feasible start.