8.17 Positive Definiteness

A quadratic form in \(k\) variables \(x_1,...,x_k\) is \(\mathbf{x}^T\mathbf{Ax}\), where \(\mathbf{x}^T = [x_1,...,x_k]\) and \(\mathbf{A}\) is a symmetric \(k\times k\) matrix. It can be written as \(\sum^k_{i=1}\sum^k_{j=1}a_{ij}x_ix_j\).

A symmetric matrix is positive definite if \[\mathbf{x}^T\mathbf{Ax}>0\] for all vectors \(\mathbf{x}\not= \mathbf{0}\). If a matrix is positive definite, then all of the eigenvalues will be positive.

A symmetric matrix is nonnegative definite if \[\mathbf{x}^T\mathbf{Ax}\geq 0\] for all vectors \(\mathbf{x}\not= \mathbf{0}\).

Cholesky Decomposition: For a real-valued symmetric positive definite matrix \(\mathbf{A}\), it can be decomposed as \[\mathbf{A} = \mathbf{L}\mathbf{L}^T \] where \(\mathbf{L}\) is a lower triangular matrix with real and spositive diagonal elements.