Symmetric positive definite matrix

Definition

The condition “symmetric positive definite” says that a matrix is mirrored across its diagonal and that its associated squared-coordinate expression is positive for every nonzero input. Before writing the condition formally, read the notation:

• A vector $x\in\mathbb R^n$ is a column of $n$ real numbers, called its coordinates. A scalar is a single number.
• A square matrix $A\in\mathbb R^{n\times n}$ has $n$ rows and $n$ columns. Its main diagonal runs from top left to bottom right; off-diagonal entries occupy the other positions.
• The transpose $A^T$ exchanges rows and columns. A matrix is symmetric when $A^T=A$. For example, a two-by-two matrix with rows $(3,1)$ and $(1,2)$ is symmetric because its two off-diagonal entries match.
• The product $x^TAx$ returns one number. Compute $Ax$, then multiply matching coordinates of $x$ and $Ax$ and add: this is their dot product. For $A=\operatorname{diag}(2,3)$, meaning entries $2,3$ on the main diagonal and zeros elsewhere, the result is $2x_1^2+3x_2^2$.

This last expression is a quadratic form: it combines squares and products of pairs of coordinates. Read quadratic forms for the coefficient rules, or the matrix-decomposition overview for basic matrix operations.

A real square matrix $A\in\mathbb R^{n\times n}$ is symmetric positive definite (SPD) when

Here nonzero means at least one coordinate is not zero; it does not require all coordinates to be positive. Strictly positive means greater than zero, not merely greater than or equal to zero. For example, $2x_1^2+3x_2^2$ is positive for every nonzero input, including $(0,-1)^T$. Both symmetry and strict positivity are required. See van de Geijn’s definition.

SPD matrices describe an energy that increases in every direction away from zero. A linear system $Ax=b$ asks which unknown coordinates $x$ produce a given output $b$. SPD structure guarantees a unique solution for every $b$. It also supports Cholesky decomposition, a specialized matrix decomposition.

Intuition: every direction costs energy

For a symmetric two-by-two matrix,

The diagonal entries penalize individual coordinates. The term $2bx_1x_2$ multiplies different coordinates, so it is a cross term; $b$ is their coupling coefficient, recording how they interact. For example, when $b>0$, equal-sign coordinates make this term positive and opposite-sign coordinates make it negative. Its effect therefore depends on the direction of $x$, rather than just its length. Testing $x=(1,0)^T$ and $x=(0,1)^T$ checks only the coordinate axes. A coupled direction can still have zero or negative energy.

Scaling a direction gives $q(tx)=t^2q(x)$. For SPD input, moving farther along any nonzero direction increases the energy. The origin is the vector with every coordinate zero. It is the unique minimum of $q$: every other input has a larger value.

A symmetric matrix is positive semidefinite (PSD) if $q(x)\ge0$ for all $x$, allowing zero energy in nonzero directions. For example, $q(x)=x_1^2$ has a flat direction: changing $x_2$ costs nothing. Indefinite means some directions have positive energy and others negative energy; $x_1^2-x_2^2$ is positive at $(1,0)^T$ and negative at $(0,1)^T$. See Higham’s semidefinite comparison.

A level curve, or contour, connects inputs with the same function value. For example, $x_1^2+x_2^2=4$ is a circle of radius $2$: every point on it has energy $4$. The figure compares ellipses (stretched circles), flat-direction pairs of lines, and hyperbolas (the two open branches of $x_1^2-x_2^2=k$ for nonzero $k$).

Worked example: prove positivity without checking infinitely many vectors

The following calculations and exercises are original illustrative examples.

Take

First, matching off-diagonal entries establish symmetry. Completing the square rewrites a square term and a cross term as a squared bracket plus a remainder. For example, $t^2+2t+3=(t+1)^2+2$, because expanding the bracket gives $t^2+2t+1$. See quadratic forms: completing the square for the two-coordinate rule. Here it gives:

Each added term (a summand) is nonnegative, meaning greater than or equal to zero. For the sum to vanish, the second term forces $x_2=0$, then the first forces $x_1=0$. Thus every nonzero vector has strictly positive energy.

For $x=(1,-1)^T$, the energy is $3-2+2=3$. This one calculation illustrates the result; the completed-square argument proves it for every direction.

A practical two-by-two test

For $a>0$, the same algebra gives

Therefore a symmetric two-by-two matrix is SPD exactly when

Necessity follows by testing $(1,0)^T$ and $(-b/a,1)^T$; sufficiency follows from the two strictly positive coefficients in the completed square. In the worked example, $a=3$ and $ac-b^2=5$.

Equivalent ways to recognize SPD matrices

Here are the terms used in the tests:

• An eigenvector is a nonzero vector $v$ satisfying $Av=\lambda v$; its eigenvalue $\lambda$ is the multiplier. For $A=\operatorname{diag}(2,3)$, $(1,0)^T$ is an eigenvector with eigenvalue $2$. See the overview’s eigenvector explanation.
• A determinant is a number associated with a square matrix. In two dimensions, the determinant of our matrix with entries $a,b,b,c$ is $ac-b^2$. A leading principal minor is the determinant of an upper-left square block. For our worked example, the first minor is $3$ and the second is $3\cdot2-1^2=5$. See LU’s determinants and initial blocks for more context.
• A lower triangular matrix has zeros above the main diagonal. A Cholesky factor is such a matrix $L$ satisfying $A=LL^T$, with positive diagonal entries. For example, $A=\operatorname{diag}(4,9)$ has $L=\operatorname{diag}(2,3)$. See Cholesky decomposition for how to construct it.

The all-positive-minors test is called Sylvester’s criterion. For a real symmetric matrix, the following tests are equivalent: passing any one guarantees the others.

For an $n\times n$ matrix, the minor test checks the blocks of sizes $1,2,\ldots,n$. See Higham’s equivalent conditions and van de Geijn’s Cholesky theorem. These statements assume symmetry; eigenvalue positivity alone does not establish SPD for a general matrix.

Why eigenvalues explain the geometry

A unit vector has length $1$. The Euclidean norm $\lVert v\rVert_2=\sqrt{v_1^2+\cdots+v_n^2}$ measures length; for example, $(3,4)^T$ has length $5$, and $(3/5,4/5)^T$ is a unit vector. Two vectors are orthogonal, or perpendicular, if their dot product is zero. An orthogonal matrix $Q$ has perpendicular unit columns, satisfies $Q^TQ=I$, and preserves lengths. Here $I$ is the identity matrix, whose diagonal entries are $1$ and all other entries $0$. See vector norms and orthogonal factors in the overview.

The spectral theorem says that a real symmetric matrix has such a set of eigenvectors spanning all coordinate directions. Its symmetric eigendecomposition stores them as $A=Q\Lambda Q^T$, with orthogonal $Q$ and diagonal $\Lambda$ (zeros off the diagonal). Its diagonal entries are the eigenvalues $\lambda_i$. Put $y=Q^Tx$: these are the coordinates measured along those eigenvector directions. Then

If every $\lambda_i>0$, this sum is positive for nonzero $y$. Conversely, an eigenvector $v_i$ satisfies $v_i^TAv_i=\lambda_i\lVert v_i\rVert_2^2$, so SPD forces $\lambda_i>0$. This derives the eigenvalue test above.

In eigenvector coordinates, the contour $q(x)=1$ has semiaxis lengths, meaning the distances from the ellipse’s center to its edge along its eigenvector directions, of $1/\sqrt{\lambda_i}$. A small eigenvalue means a long axis and a direction with weak resistance. For our worked example,

Why a Cholesky factor is a certificate

A certificate is a checkable witness for a claim: here the factor $L$ proves positivity through a sum of squares. A matrix is invertible if it has an inverse that undoes its action. For example, $\operatorname{diag}(2,3)$ is undone by $\operatorname{diag}(1/2,1/3)$; an invertible matrix cannot send a nonzero vector to zero. A triangular matrix with nonzero diagonal entries is invertible. See inverse matrices in the overview.

For $A=LL^T$ with invertible $L$,

Our example has

Multiplying $LL^T$ gives diagonal entries $3$ and $1/3+5/3=2$, and off-diagonal entries $1$. See the Cholesky page for the construction and triangular solves.

Counterexamples to common shortcuts

Positive entries do not imply SPD. The symmetric matrix

has positive entries, but $q((1,-1)^T)=-4$.

Negative off-diagonal entries are allowed. For

the energy is strictly positive for every nonzero vector.

A positive determinant is insufficient. The matrix $-I_2$ has determinant $1$, but $q(x)=-\lVert x\rVert_2^2$. Sylvester’s criterion also checks the first leading minor, which is $-1$.

A positive diagonal is insufficient. The positive-entry counterexample has positive diagonal entries but negative energy along $(1,-1)^T$.

Strict positivity of the quadratic form does not imply symmetry. Let

Then $x^TBx=x_1^2+x_2^2>0$ for nonzero $x$, but $B^T\ne B$. The off-diagonal part is skew-symmetric, meaning its transpose is its negative: $2x_1x_2$ and $-2x_2x_1$ cancel in the quadratic form. This is why the definition explicitly includes symmetry.

Where SPD matrices come from

Independent columns and least squares

Least squares fits predictions $Bx$ to measurements $b$ by minimizing the sum of squared discrepancies, $\lVert Bx-b\rVert_2^2$. For example, fitting one constant $t$ to $1,2,6$ minimizes $(t-1)^2+(t-2)^2+(t-6)^2$; its best value is $3$.

Columns are linearly independent if none can be made by multiplying the others by numbers and adding. Their number of independent directions is the rank. For instance, $(1,0)^T$ and $(0,1)^T$ are independent; identical columns are dependent, so different parameters can produce the same prediction. See the overview’s rank and least-squares examples.

For a real rectangular matrix $B$,

The matrix $B^TB$ is a Gram matrix: each entry is the dot product of a pair of columns of $B$. Its diagonal entries are their squared lengths. This matrix is always symmetric PSD. It is SPD exactly when $B$ has linearly independent columns: then $Bx=0$ forces $x=0$. See van de Geijn’s Gram-matrix exercise. This explains the rank condition on the normal equations, the linear system $B^TBx=B^Tb$ characterizing least-squares fits, discussed in LU decomposition.

For example,

The nullspace is the set of vectors $x$ satisfying $Bx=0$, meaning parameter changes that leave every prediction unchanged. For this example, the first row forces $x_1=0$ and the second forces $x_2=0$, so the nullspace contains only zero. If both columns instead equal $(1,0,1)^T$, the Gram matrix becomes

and $(1,-1)^T$ has zero energy. Dependence creates a flat direction.

Quadratic minimization

Minimization chooses inputs to make an objective function as small as possible. A gradient collects its coordinate slopes into a vector; each slope is a partial derivative, changing one coordinate while holding the others fixed. A Hessian collects second partial derivatives into a matrix, measuring how the slopes change. For example, $x_1^2+3x_2^2$ has gradient $(2x_1,6x_2)^T$ and Hessian $\operatorname{diag}(2,6)$. See quadratic forms: derivatives and optimization for a step-by-step derivation.

Consider $f(x)=\frac12x^TAx-r^Tx$ with symmetric $A$. Here $r$ is a fixed vector. Its gradient is $Ax-r$, and its Hessian is $A$. A stationary point has zero gradient, so in this example it solves $Ax=r$. If $A$ is SPD, it is invertible: $Ax=0$ would imply $x^TAx=0$, hence $x=0$.

Let $x_{\ast}$ solve $Ax_{\ast}=r$, and let $h$ be a change in its coordinates. Expanding around this point yields

The difference is strictly positive for nonzero $h$, proving that $x_{\ast}$ is the unique global minimum: it has a smaller value than every other input. For the worked-example matrix and $r=(4,3)^T$, $x_{\ast}=(1,1)^T$ and $f(x_{\ast})=-7/2$. SPD turns stationarity into a unique optimum. Convexity: the Hessian tests every direction places this quadratic result in the broader distinction between convexity, strict convexity, and existence of a minimizer.

Adding a positive penalty

If $C$ is symmetric PSD and $\alpha>0$, then

Thus $C+\alpha I$ is SPD even when $C$ has a flat direction. This derivation explains why a positive squared-length penalty removes the nullspace of a least-squares quadratic.

Numerical recognition

In software, establish symmetry and then attempt Cholesky. LAPACK’s DPOTRF reports successful factorization with INFO = 0 and a failed positive-definiteness test with INFO > 0. It reads only the selected triangle (the entries on and above the diagonal, or on and below it), so success does not independently verify that both stored triangles agree.

A square matrix is singular when it has no inverse: for example, $\operatorname{diag}(1,0)$ loses the second coordinate. An SPD matrix is invertible, but can be close to such a singular matrix. For $A_\varepsilon=\operatorname{diag}(1,\varepsilon)$ with $\varepsilon>0$, the second equation in $A_\varepsilon x=r$ gives $x_2=r_2/\varepsilon$. A change $\delta$ in $r_2$ changes $x_2$ by $\delta/\varepsilon$. Strict positivity guarantees a unique solution, but a very small positive eigenvalue can make it sensitive. Floating-point arithmetic stores numbers with limited precision, so results may be rounded; for example, representing $1/3$ as $0.3333$ introduces a small error. For nearly singular input, these errors can overwhelm a tiny positive quantity needed by the algorithm. Floating-point Cholesky can break down for nearly singular input; see Higham’s numerical discussion.

Interactive example: change the coupling

Use the existing quadratic-energy lab below for \(A=\begin{bmatrix}a&b\\b&c\end{bmatrix}\). It shows energy signs, the unit-energy contour when SPD, and a Cholesky certificate. With $a=4$ and $c=3$, predict the transition at $\lvert b\rvert=\sqrt{12}$. The absolute value $\lvert b\rvert$ ignores the sign of $b$; for example, $\lvert-2\rvert=2$. Increase $b$ through that boundary, then try the displayed zero or negative energy direction.

The controls accept keyboard input. A single positive-energy vector does not prove SPD; compare its energy with the lab’s whole-matrix test. Without JavaScript, the completed-square examples above provide the same criterion and explain its boundary.

Practice problems

1. Hand calculation: find the allowable coupling

For which real $t$ is \(A=\begin{bmatrix}2&t\\t&8\end{bmatrix}\) SPD?

2. Interpretation: what does a positive sample establish?

For \(A=\begin{bmatrix}1&3\\3&1\end{bmatrix}\), you measure positive energy along both coordinate axes and $(1,1)^T$. Can you conclude SPD? Choose a decisive next test.

3. Abstract reasoning: change coordinates

Suppose $A$ is SPD and $S$ is an invertible real square matrix. Prove $S^TAS$ is SPD. What can fail if $S$ is singular?

4. Application: remove ambiguity in a fit

A least-squares model has identical columns and Gram matrix \(C=\begin{bmatrix}2&2\\2&2\end{bmatrix}\). Why are its parameters ambiguous? Does adding $\frac\alpha2\lVert x\rVert_2^2$ to the objective remove the ambiguity for $\alpha>0$?

Further learning

• Higham: What Is a Symmetric Positive Definite Matrix?: compare eigenvalue and principal-minor tests and continue to block matrices.
• van de Geijn: Notes on Cholesky Factorization: follow a constructive proof and algorithm for the triangular factor.
• LAPACK: DPOTRF: understand triangle storage and failure reporting in a numerical implementation.

Related pages

• Vector norms
• Matrix norms

• Quadratic form

• Cholesky decomposition
• LU decomposition
• Matrix factorization and decomposition

References

1. van de Geijn, R. A. (2011). Notes on Cholesky Factorization. University of Texas at Austin. SPD definition, Gram matrices, and the Cholesky theorem.
2. Higham, N. J. (2020, July 21). What Is a Symmetric Positive Definite Matrix?. Equivalent positivity criteria and semidefinite comparison.
3. LAPACK. DPOTRF routine documentation. Symmetric triangle storage and Cholesky success or failure.
4. Higham, N. J. (2020, August 11). What Is a Cholesky Factorization?. Floating-point behavior near singularity.