Matrix norms#

Frobenius norm#

Let \(\boldsymbol A \in \mathbb R^{m\times n}\). Frobenius norm of \(\boldsymbol A\) is defined as

\[ \Vert \boldsymbol A \Vert_F = \sqrt{\mathrm{tr}(\boldsymbol A^\top \boldsymbol A)} = \sqrt{\sum\limits_{i=1}^m \sum\limits_{j=1}^n a_{ij}^2}. \]

Spectral norm#

The spectral norm of the matrix \(\boldsymbol A \in \mathbb R^{m\times n}\) is equal to maximal eigenvalue of \(\boldsymbol A^\top \boldsymbol A\):

\[ \Vert \boldsymbol A \Vert_2 = \sqrt{\lambda_{\max}(\boldsymbol A^\top \boldsymbol A)}. \]

Condition number#

For a square matrix \(\boldsymbol A\) define its conditional number as

\[ \kappa(\boldsymbol A) = \Vert \boldsymbol A \Vert_2 \cdot\Vert\boldsymbol A^{-1}\Vert_2. \]

If \(\boldsymbol A\) is singular then \(\kappa(\boldsymbol A) = \infty\).

Properties of conditional numbers

  • \(\kappa(\boldsymbol A) \geqslant 1\);

  • \(\kappa(\boldsymbol A) = \kappa(\boldsymbol A^{-1})\);

  • \(\kappa(\boldsymbol{AB}) \leqslant \kappa(\boldsymbol A)\kappa(\boldsymbol B)\);

  • \(\kappa(\boldsymbol A) = \Big\vert\frac{\lambda_{\max}(\boldsymbol A)}{\lambda_{\min}(\boldsymbol A)}\Big\vert\) if \(\boldsymbol A^\top = \boldsymbol A\).