diff --git a/linear-algebra/eigenvalue-problem.md b/linear-algebra/eigenvalue-problem.md index a96fdc82a..a891cce80 100644 --- a/linear-algebra/eigenvalue-problem.md +++ b/linear-algebra/eigenvalue-problem.md @@ -1,5 +1,145 @@ # Eigenvalue problem +For an *n* x *n* square matrix **A**, we seek a scalar $\lambda$ and vector +**x** such that: + +\begin{align} +\vv{A} \vv{x} = \lambda \vv{x} +\end{align} + +We want a *nontrivial* solution ($\vv{x} \ne \vv{0}$). This pair represents a +vector **x** that, when multiplied into **A**, does not change its direction. +However, it may adopt a new magnitude $\lambda$. Borrowing from a German word +for "own", we call **x** an *eigenvector* and $\lambda$ an *eigenvalue* of +**A**. + +To find the eigenvalues of **A**, rearrange: + +\begin{align} +\vv{A} \vv{x} - \lambda \vv{x} &= \vv{0} \\ +(\vv{A} - \lambda \vv{I}) \vv{x} &= \vv{0} +\end{align} + +If $\vv{A} - \lambda \vv{I}$ is invertible, we have only $\vv{x} = \vv{0}$ as a +solution. Hence, we require this matrix to be singular! This occurs when its +[determinant](./determinants.md) is zero: + +\begin{align} +|\vv{A}- \lambda \vv{I}| = 0 +\end{align} + +This equation creates a characeristic polynomial of degree *n* for $\lambda$ +that can be solved. Then, the eigenvector **x** that corresponds to each root +$\lambda$ can be determined. For example, to find the eigenvalues of: + +\begin{equation} +\vv{A} = \begin{bmatrix} -5 & 2\\ 2 & -2 \end{bmatrix} +\end{equation} + +First compute the determinant: + +\begin{align} +|\vv{A} -\lambda \vv{I}| +&= \begin{vmatrix} -5-\lambda & 2 \\ 2 & -2-\lambda \end{vmatrix} \\ +&= (-5 -\lambda)(-2-\lambda) - 2 \cdot 2 \\ +&= \lambda^2 +7\lambda + 6 \\ +&= (\lambda +1)(\lambda +6) = 0 +\end{align} + +Hence, the eigenvalues of **A** are $\lambda_1 = -1$ and $\lambda_2 = -6$. Note +that the particular ordering of the eigenvalues is not important, and we are +only labeling them to make it convenient to refer to a particular eigenvalue +later. + +Next, we seek the eigenvector $\vv{x}_1$ that corresponds to $\lambda_1$. This +vector must solve the system: + +\begin{align} +(\vv{A} - \lambda_1 \vv{I}) \vv{x}_1 &= \vv{0} \\ +\begin{bmatrix} -4 & 2 \\ 2 & -1 \end{bmatrix} \vv{x}_1 &= \vv{0} +\end{align} + +We can solve for $\vv{x}_1$ using row reduction. Since the last column of the +augmented matrix would be only zeros, it is not necessary to include it, and we +row reduce only the matrix itself: + +\begin{equation} +\begin{bmatrix} +-4 & 2 \\ +2 & -1 +\end{bmatrix} +\begin{matrix}\vphantom{R_1} \\ +R_2/2 \end{matrix} +\to \begin{bmatrix} +-4 & 2 \\ +0 & 0 +\end{bmatrix} +\begin{matrix} \div -2 \\ \vphantom{R_2} \end{matrix} +\to +\begin{bmatrix} +1 & -1/2 \\ +0 & 0 +\end{bmatrix} +\end{equation} + +Note that this reduced matrix is equivalent to $x_1 = x_2/2$ (using *x* to +represent elements of $\vv{x}_1$ for convenience) with $x_2$ being free. This +means that the eigenvector is not unique, which makes sense both from the +original equation we are trying to solve and us forcing the matrix to be +singular! Equivalently, eigenvectors can be scaled up or down by an arbitrary +nonzero constant. We may then choose $x_2$ so that the eigenvector has either +nice values (like like integers) or a nice norm (1 is conventional). For +example, choosing $x_2 = 2$ gives: + +\begin{equation} +\vv{x}_1 = \begin{bmatrix} 1 \\ 2 \end{bmatrix} +\end{equation} + +```{tip} +For a 2x2 matrix, if the eigenvalue $\lambda$ has been found correctly, one row +of $\vv{A}-\lambda\vv{I}$ will always be a multiple of the other. This means +the eigenvector can be determined using only one row. Moreover, the solution +for **x** can be obtained by swapping the entries in a row and changing the sign +of one of them! For example, we had + +\begin{equation} +\begin{bmatrix} +-4 & 2 \\ +2 & -1 +\end{bmatrix} +\end{equation} + +so using the second row, we find: + +\begin{equation} +\vv{x}_1 = \begin{bmatrix}-(-1) \\ 2\end{bmatrix} = \begin{bmatrix}1 \\ 2\end{bmatrix} +\end{equation} + +However, we could also have negated the other entry: + +\begin{equation} +\vv{x}_1 = \begin{bmatrix}-1\\ -2\end{bmatrix} +\end{equation} + +or used the first row: + +\begin{equation} +\vv{x}_1 = \begin{bmatrix}2 \\ 4\end{bmatrix} +\end{equation} + +Both of these are also valid eigenvectors because they are multiples of the one +we found already. +``` + +Let's use the same strategy to get the eigenvector for $\lambda_2 = -6$. We'll +jump straight to the matrix we need: + +\begin{align} +(\vv{A} - \lambda_2 \vv{I}) = \begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix} +\to \vv{x}_2 = \begin{bmatrix}2 \\ -1 \end{bmatrix} +\end{align} + +using the swapping trick on the first row. + ## Multiple eigenvalues An *n* x *n* matrix has *n* eigenvalues, but they may not be distinct! For