Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Vectors and Linear Algebra

Linear algebra is the backbone of quantum mechanics. States are vectors, observables are matrices, and measurement outcomes are eigenvalues. Everything that follows in the course is, at bottom, an eigenvalue problem in a vector space. This page builds that vocabulary from the 2D picture up.

Vectors

What is a vector?

A vector is an ordered collection of numbers describing the state or configuration of a system:

Vectors may live in real or complex vector spaces depending on their components. In quantum mechanics the components are generally complex.

The plot shows a vector built as v=2e1+3e2\mathbf{v} = 2e_1 + 3e_2, decomposed along the two basis directions.

Source
<Figure size 600x500 with 1 Axes>

Inner products, norms, and orthogonality

The next natural operation multiplies two vectors into a number that measures how much they share a direction. Throughout this section, a⃗{\color{orange} \vec a} is orange and b⃗{\color{#3d81f6} \vec b} is blue, in every equation and every figure.

A worked example, with every number wearing its vector’s color. For a⃗=(62){\color{orange} \vec a = \begin{pmatrix} 6 \\ 2 \end{pmatrix}} and b⃗=(5−3){\color{#3d81f6} \vec b = \begin{pmatrix} 5 \\ -3 \end{pmatrix}}:

a⃗⋅b⃗=(6)(5)+(2)(−3)=30−6=24.{\color{orange} \vec a} \cdot {\color{#3d81f6} \vec b} = ({\color{orange} 6})({\color{#3d81f6} 5}) + ({\color{orange} 2})({\color{#3d81f6} -3}) = 30 - 6 = 24.

The two forms together are a machine for extracting angles from components. And the special case cos⁡θ=0\cos\theta = 0 earns its own name:

Vectors that are both orthogonal and normalized are orthonormal, compactly written with the Kronecker delta:

⟨ei∣ej⟩=δij={1i=j0i≠j.\langle e_i \mid e_j \rangle = \delta_{ij} = \begin{cases} 1 & i = j \\ 0 & i \neq j \end{cases}.

The norm is the square root of a vector’s inner product with itself, giving its length:

⟨a∣a⟩=a12+a22,∥a∥=a12+a22.\langle a \mid a \rangle = a_1^2 + a_2^2, \qquad \|a\| = \sqrt{a_1^2 + a_2^2}.

A vector with ∥a∥=1\|a\| = 1 is normalized.

Projection: the most quantum operation in this appendix

The answer defines the orthogonal projection. Choose kk so that the leftover error vector e⃗=a⃗−kb⃗{\color{#d81b60} \vec e} = {\color{orange} \vec a} - k{\color{#3d81f6} \vec b} is orthogonal to b⃗{\color{#3d81f6} \vec b}; any other choice leaves a longer error (Pythagoras). That choice gives:

Source
<Figure size 640x460 with 1 Axes>

The picture contains a decomposition. Walking along p⃗{\color{#004d40} \vec p} and then along e⃗{\color{#d81b60} \vec e} lands exactly on a⃗{\color{orange} \vec a}:

a⃗=p⃗⏟parallel to b⃗+e⃗⏟orthogonal to b⃗.{\color{orange} \vec a} = \underbrace{{\color{#004d40} \vec p}}_{\text{parallel to } {\color{#3d81f6} \vec b}} + \underbrace{{\color{#d81b60} \vec e}}_{\text{orthogonal to } {\color{#3d81f6} \vec b}}.

Now the quantum payoff. For an orthonormal basis {e1,e2,…}\{e_1, e_2, \ldots\} the projection formula collapses to ck=⟨ek∣a⟩c_k = \langle e_k \mid a \rangle, and expanding a vector in a basis is nothing but projecting it onto each basis direction in turn:

a⃗=∑kck ek,ck=⟨ek∣a⟩.\vec a = \sum_k c_k\, e_k, \qquad c_k = \langle e_k \mid a \rangle.

In quantum mechanics the state ψ\psi is the vector, the eigenstates of a measurement are the orthonormal basis, and the numbers ∣ck∣2=∣⟨ek∣ψ⟩∣2|c_k|^2 = |\langle e_k \mid \psi \rangle|^2 become the probabilities of each outcome (the Born rule). Every quantum measurement is, at its mathematical heart, the orthogonal projection drawn above.

Matrices

Linear systems as matrix equations

A system of linear equations is a matrix acting on a vector. The system

a11x1+a12x2=b1a21x1+a22x2=b2\begin{aligned} a_{11}x_1 + a_{12}x_2 &= b_1 \\ a_{21}x_1 + a_{22}x_2 &= b_2 \end{aligned}

packs into a single matrix-vector product:

Matrix times vector is the dot product of each row of AA with x\mathbf{x}:

Ax=(a11x1+a12x2a21x1+a22x2).A\mathbf{x} = \begin{pmatrix} a_{11}x_1 + a_{12}x_2 \\ a_{21}x_1 + a_{22}x_2 \end{pmatrix}.

If AA is invertible, the solution is x=A−1b\mathbf{x} = A^{-1}\mathbf{b}.

solution via inverse: [2.42857143 0.71428571]
solution via solve  : [2.42857143 0.71428571]

Eigenvalues and eigenvectors

When a matrix acts on a vector it usually changes both its length and direction. Certain special vectors keep their direction and are only rescaled. These are eigenvectors, and the scaling factors are eigenvalues.

Rearranging to (A−λI)v=0(A - \lambda I)\mathbf{v} = 0, a nonzero solution exists only when the characteristic equation holds:

det⁡(A−λI)=0.\det(A - \lambda I) = 0.

The sign and size of λ\lambda say what happens to the eigenvector: λ>1\lambda > 1 stretches it, 0<λ<10 < \lambda < 1 compresses it, λ<0\lambda < 0 flips it, and λ=0\lambda = 0 collapses it. In quantum mechanics this exact structure reappears for operators acting on wavefunctions: eigenvalues are the measurable observables (energies, momenta) and eigenvectors are the states with definite outcomes. The next page makes that leap explicit, showing that a function is just a vector with infinitely many components and the Schrodinger equation H^ψ=Eψ\hat H\psi = E\psi is this same eigenvalue problem Av=λvA\mathbf{v}=\lambda\mathbf{v}.

eigenvalues : [3.+0.j 1.+0.j]
eigenvectors (columns):
 [[ 0.70710678+0.j -0.70710678+0.j]
 [ 0.70710678+0.j  0.70710678+0.j]]

A @ v0        : [2.12132034+0.j 2.12132034+0.j]
lambda0 * v0  : [2.12132034+0.j 2.12132034+0.j]

The whole story in one picture: a matrix turns almost every vector, but an eigenvector comes back parallel to itself.

Source
<Figure size 900x440 with 2 Axes>

On the left, v⃗\vec v (gray) and Av⃗\color{steelblue} A\vec v (blue) point in different directions: the matrix rotated the vector. On the right, the eigenvector u⃗=12(1,1)\vec u = \tfrac{1}{\sqrt 2}(1,1) obeys Au⃗=3u⃗\color{crimson} A\vec u = 3\vec u: same direction, three times longer. Eigenvectors are the directions along which a matrix acts like simple multiplication.

The geometric action of a matrix

Different kinds of matrices deform a shape in characteristic ways:

Matrix typeExampleEffect
Diagonal, unequal entries[[1.8, 0], [0, 0.8]]Stretch, different scaling per axis
Diagonal, equal entries[[0.7, 0], [0, 0.7]]Uniform scaling
Rotation (orthogonal)[[cosθ, -sinθ], [sinθ, cosθ]]Rotation, preserves lengths and angles
Off-diagonal term[[1, k], [0, 1]]Shear, slants without changing area

We apply each matrix to the same small polygon to see how it transforms.

Source
<Figure size 800x800 with 4 Axes>

A matrix in new axes: diagonalization

Turning the axes changes the components of a vector but not the vector, as in the tip on bases at the start of this page. A matrix is the same: its entries depend on the axes, while what it does to an arrow does not.

In the usual axes e1=(1,0)\mathbf{e}_1 = (1, 0) and e2=(0,1)\mathbf{e}_2 = (0, 1), the entry in row mm and column nn is a dot product, Amn=em⋅(Aen)A_{mn} = \mathbf{e}_m \cdot (A\mathbf{e}_n): the matrix turns en\mathbf{e}_n into AenA\mathbf{e}_n, and the entry says how much of that lands on em\mathbf{e}_m. Turn to any two perpendicular unit vectors u1\mathbf{u}_1 and u2\mathbf{u}_2 and the same recipe gives the matrix in the new axes. In Dirac’s notation it is a “sandwich” of a bra, the matrix and a ket:

Amn′=um⋅(Aun)=⟨um∣A∣un⟩A'_{mn} = \mathbf{u}_m \cdot (A\mathbf{u}_n) = \langle \mathbf{u}_m \mid A \mid \mathbf{u}_n \rangle

Try it: drag on the plane to turn the axes for A=(2112)A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}, the matrix of the eigenvalue picture above. The orange pieces are the off-diagonal entries. They shrink as you turn, and at 45∘45^\circ they vanish: each AunA\mathbf{u}_n then lies along its own axis. The trace and the determinant never change.

For this AA, axes turned by an angle θ\theta, with u1=(cos⁡θ,sin⁡θ)\mathbf{u}_1 = (\cos\theta, \sin\theta) and u2=(−sin⁡θ,cos⁡θ)\mathbf{u}_2 = (-\sin\theta, \cos\theta), give

A′(θ)=(2+sin⁡2θcos⁡2θcos⁡2θ2−sin⁡2θ)A'(\theta) = \begin{pmatrix} 2 + \sin 2\theta & \cos 2\theta \\ \cos 2\theta & 2 - \sin 2\theta \end{pmatrix}

The off-diagonal entry cos⁡2θ\cos 2\theta vanishes at θ=45∘\theta = 45^\circ. The trace is 4 and the determinant is 4−sin⁡22θ−cos⁡22θ=34 - \sin^2 2\theta - \cos^2 2\theta = 3 at every angle, so the eigenvalues, the roots of λ2−4λ+3=0\lambda^2 - 4\lambda + 3 = 0, cannot change when the axes turn.

Finding those axes is called diagonalizing the matrix. A symmetric matrix always has perpendicular unit eigenvectors (so does a Hermitian one, see Operators as Matrices), so it can always be diagonalized. In numpy, eigh returns the eigenvectors as the columns of a matrix U, and U.T @ A @ U computes every sandwich ⟨um∣A∣un⟩\langle \mathbf{u}_m \mid A \mid \mathbf{u}_n \rangle at once: row mm of U.T is the bra, column nn of U the ket.

eigenvalues        : [1. 3.]
A in the new axes  :
 [[1. 0.]
 [0. 3.]]
trace before, after: 4.0 4.0
det   before, after: 3.0 3.0

The new matrix is diagonal, with the eigenvalues 1 and 3 in increasing order, and the trace and determinant are unchanged. This is what “diagonalizing a Hamiltonian” means in Operators 1: find the eigenvectors of HH, write HH in those axes, and read the energies off the diagonal. For complex matrices the bra carries a complex conjugate, U.conj().T @ A @ U.

Problems

Problem 1: Dot product and angle

Find ⟨a∣b⟩\langle a \mid b \rangle and the angle between a=(1,2)a = (1, 2) and b=(3,−1)b = (3, -1).

Problem 2: Normalize a vector

Normalize v=(3,4)v = (3, 4).

Problem 3: Check orthogonality

Are (1,1)(1, 1) and (1,−1)(1, -1) orthogonal? Are they orthonormal?

Problem 4: Eigenvalues by hand

Find the eigenvalues of A=(2112)A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}.

Problem 5: Solve a linear system

Solve (231−2)x=(71)\begin{pmatrix} 2 & 3 \\ 1 & -2 \end{pmatrix}\mathbf{x} = \begin{pmatrix} 7 \\ 1 \end{pmatrix} by hand and check against the code.

Problem 6: Rotation preserves length

Show that the rotation matrix R=(cos⁡θ−sin⁡θsin⁡θcos⁡θ)R = \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix} leaves the norm of any vector unchanged.

Problem 7: Eigenvector direction

Verify that (1,1)(1, 1) is an eigenvector of A=(2112)A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix} and identify its eigenvalue.

Problem 8: Complex inner product

For complex vectors the inner product conjugates the first entry: ⟨a∣b⟩=∑iai∗bi\langle a \mid b \rangle = \sum_i a_i^* b_i. Compute ⟨a∣a⟩\langle a \mid a \rangle for a=(1, i)a = (1,\ i) and confirm it is real and positive.