{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Linear Algebra 1: Basics " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Vectors, what are they?\n", "\n", "\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Vectors as Arrays of Numbers\n", "\n", "We will define **vector** as an ordered collection of numbers which describes states of physical systems. \n", "\n", " - $a = (-2, 8)$: A 2D vector.\n", " - $b = (1.34, 4.23, 5.98)$: A 3D vector.\n", " - $c = (1, -2, 4i, 3 + 2i)$: A 4D vector with complex components.\n", " - $f = (1, 2, 3, 4, 5, 6, \\ldots, \\infty)$: An infinite-dimensional vector with integer components.\n", "\n", "- **Note**: Vectors can belong to real or complex spaces, depending on their components.\n", "\n", "\n", "### Vectors Defined with Respect to a Basis\n", "\n", "- In classical physics, vectors are often visualized as arrows in space, with both **magnitude** and **direction**. A vector is typically defined with respect to a **basis**, where unit vectors ($\\vec{e_i}$) span the space. For instance, in 3D Euclidean space, the standard basis is:\n", "\n", " $$e_1 = (1, 0, 0), \\,\\, e_2 = (0, 1, 0), \\,\\, e_3 = (0, 0, 1)$$\n", "\n", "- Here is an example of vector $v$ in terms of unit vectors of euclidean space.\n", "\n", " $$v = (2,3,4) = 2e_1+3e_2+4e_3$$\n", "\n", "- In a different coordinate system or basis, the same vector will have different components, but its intrinsic properties remain unchanged. This flexibility in representation is a fundamental concept in quantum mechanics and linear algebra.\n", "\n", "### Row vectors vs column vector\n", "\n", "- We will defined row and column vectors which are related to one another by operation of transpose. They describe the same physical state or coordinate of a system.\n", "\n", "$$\n", "\\begin{pmatrix} a_1 \\\\ a_2 \\end{pmatrix} = (a_1, a_2)^{T}\n", "$$\n", "\n", "- However row vectors live with their own universe of row vectors and column vectors in their own universe of column vectors.\n", "\n", "- Later we will see that to take scalar or dot product row vector needs to be hit with column and vice versa.\n", "\n", "### Example: Vector expressed in two different basis\n", "\n", "Let’s take a 2D vector $\\mathbf{v}$ in the standard Cartesian unit basis:\n", "\n", "$$\n", "\\mathbf{v} = \\begin{pmatrix} 2 \\\\ 3 \\end{pmatrix}\n", "$$\n", "\n", "$$\n", "\\mathbf{v} = 2 \\mathbf{e_1} + 3 \\mathbf{e_2} = 2\\begin{pmatrix} 1 \\\\ 0 \\end{pmatrix}+3\\begin{pmatrix} 0 \\\\ 1 \\end{pmatrix}\n", "$$\n", "\n", "\n", "Now, suppose we switch to a new basis, where each new basis vector is a scalar multiple of the standard basis vectors. Let’s define the new basis vectors as:\n", "\n", "$$\n", "\\mathbf{b_1} = 2 \\mathbf{e_1} = \\begin{pmatrix} 2 \\\\ 0 \\end{pmatrix}, \\quad \\mathbf{b_2} = 2 \\mathbf{e_2} = \\begin{pmatrix} 0 \\\\ 2 \\end{pmatrix}\n", "$$\n", "\n", "in the new basis, the vector $\\mathbf{v}$ is represented as:\n", "\n", "$$\n", "\\mathbf{v}_{\\text{new}} = 1 \\mathbf{b_1} + 3/2 \\mathbf{b_2}\n", "$$\n", "\n", "\n", "$$\n", "\\mathbf{v}_{\\text{new}} = \\begin{pmatrix} 1 \\\\ \\frac{3}{2} \\end{pmatrix}\n", "$$\n", "\n", "\n", "- This shows that when moving to a new basis the components of the vector change accordingly, but the vector itself remains the same geometrically!\n", "\n", "\n", "\n", "### Vector Notation\n", "\n", "Depending on the context, we may emphasize the basis or omit it when the basis is implied:\n", "\n", " - In terms of a coordinate basis: \n", "\n", " $$\\vec{a} = 2\\vec{e_i} + 3\\vec{e_j}$$\n", "\n", " - As an ordered tuple of components: \n", "\n", " $$a = (2, 3)$$\n", "\n", " - Using Dirac notation (common in quantum mechanics): \n", " \n", " $$\\mid a \\rangle = 2 \\mid e_i \\rangle + 3 \\mid e_j \\rangle$$\n", "\n", "- Vectors can represent a wide range of phenomena, such as the position of a particle in space, the population of countries, or temperature variations in different regions of a forest." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Vector Operations\n", "\n", "What defines vectors in a mathematical framework are the operations performed on them. Let’s illustrate these operations using a simple 2D vectors $a = (3, 2)$ and $b=(1,1)$ as an example. \n", "\n", "#### 1. Addition or Subtraction\n", "\n", "- **Component-wise addition**:\n", "\n", " $$a + b = \\begin{pmatrix}\n", " 2\\\\\n", " 3\n", " \\end{pmatrix} + \\begin{pmatrix}\n", " 1\\\\\n", " 1\n", " \\end{pmatrix} = \\begin{pmatrix}\n", " 2+1\\\\\n", " 3+1\n", " \\end{pmatrix} = \\begin{pmatrix}\n", " 3\\\\\n", " 4\n", " \\end{pmatrix}$$\n", "\n", "- **In Dirac notation**:\n", "\n", " $$|a \\rangle + |b \\rangle = (2 + 1)\\mid e_1 \\rangle + (3 + 1)\\mid e_2 \\rangle$$\n", "\n", "#### 2. Multiplication by a Scalar\n", "\n", "Multiplying a vector by a scalar $\\beta=10$ scales the vector's magnitude without changing its direction. \n", "\n", "- **Component-wise scalar multiplication**:\n", "\n", " $$10\\cdot a = 10 \\begin{pmatrix}\n", " 2\\\\\n", " 3\n", " \\end{pmatrix} = \\begin{pmatrix}\n", " 20\\\\\n", " 30\n", " \\end{pmatrix}$$\n", "\n", "- **In Dirac notation**:\n", "\n", " $$10\\mid a \\rangle = 20 \\mid e_1 \\rangle + 30 \\mid e_2 \\rangle$$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### System of Linear Equations as Matrix Operations\n", "\n", "A system of linear equations can be interpreted as a matrix operating on a vector to produce another vector. For example, consider the system:\n", "\n", "$$\n", "\\begin{aligned}\n", "a_{11}x_1 + a_{12}x_2 &= b_1 \\\\\n", "a_{21}x_1 + a_{22}x_2 &= b_2\n", "\\end{aligned}\n", "$$\n", "\n", "This system can be written compactly in matrix form as:\n", "\n", ":::{admonition} **A system of linear equation as matrix vector product**\n", ":class: important \n", "\n", "$$\n", "A \\mathbf{x} = \\mathbf{b}\n", "$$\n", "\n", "- $A$ is a matrix containing the coefficients of the system,\n", "- $\\mathbf{x}$ is the vector of unknowns, and\n", "- $\\mathbf{b}$ is the vector of constants (right-hand side).\n", "\n", ":::\n", "\n", "For a 2D system, this becomes:\n", "\n", "$$\n", "\\begin{pmatrix}\n", "a_{11} & a_{12} \\\\\n", "a_{21} & a_{22}\n", "\\end{pmatrix}\n", "\\begin{pmatrix}\n", "x_1 \\\\\n", "x_2\n", "\\end{pmatrix}\n", "=\n", "\\begin{pmatrix}\n", "b_1 \\\\\n", "b_2\n", "\\end{pmatrix}\n", "$$\n", "\n", "### Matrix vector Multiplication\n", "\n", "**Row by column product rule**\n", "\n", "Matrix multiplication operates by taking the dot product of the rows of the matrix $A$ with the vector $\\mathbf{x}$. For the above example:\n", "\n", "$$\n", "\n", "A \\mathbf{x} = \\begin{pmatrix}\n", "a_{11}x_1 + a_{12}x_2 \\\\\n", "a_{21}x_1 + a_{22}x_2\n", "\\end{pmatrix}\n", "$$\n", "\n", "This shows how the system of equations is equivalent to multiplying the matrix by the vector.\n", "\n", "\n", "**Lower level definition of Matrix Multiplication**\n", "\n", "In general, if $A$ is an $m \\times n$ matrix and $\\mathbf{x}$ is a vector of dimension $n$, the matrix-vector multiplication $A \\mathbf{x}$ is defined as:\n", "\n", "$$\n", "b_i = (A \\mathbf{x})_i = \\sum_{j=1}^{n} a_{ij} x_j\n", "$$\n", "\n", "Where $a_{ij}$ are the elements of matrix $A$, and $x_j$ are the components of vector $\\mathbf{x}$.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ ":::{admonition} **Solving linear equation via matrix Inversion**\n", ":class: important\n", "\n", "$${\\bf x} = A^{-1}{\\bf b}$$\n", "\n", ":::" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [ "hide-input" ] }, "outputs": [], "source": [ "import numpy as np\n", "# Define the coefficients matrix A and the right-hand side vector b\n", "print('Solving Ax=b via matrix inversion')\n", "\n", "A = np.array([[2, 3],\n", " [1, -2]])\n", "\n", "b = np.array([7, 1])\n", "\n", "print('A', A)\n", "print('b', b)\n", "\n", "# Calculate the inverse of matrix A\n", "A_inv = np.linalg.inv(A)\n", "\n", "# Solve for the unknown vector x using matrix inversion: x = A_inv * b\n", "x = np.dot(A_inv, b)\n", "\n", "print(\"Solution using matrix inversion:\")\n", "print(\"x =\", x)" ] }, { "cell_type": "markdown", "metadata": { "vscode": { "languageId": "plaintext" } }, "source": [ "### Visualizing Action of a Matrix on a Vector\n", "\n", "These transformations illustrate different ways of modifying vectors and geometric shapes through scaling, shearing, and identity operations. We use a 2D vector and matrices as an example.\n", "\n", "1. **Shearing and Scaling**. $\\begin{bmatrix} 2 & 1.5 \\\\ 0.5 & 0.75 \\end{bmatrix}$\n", " - This matrix scales the $x$-direction by 2 and the $y$-direction by 0.75. Additionally, it applies both horizontal and vertical shear, distorting the object by shifting the $x$-coordinate in proportion to the $y$-coordinate and the $y$-coordinate in proportion to the $x$-coordinate. This results in a combination of scaling and shearing, altering both the size and shape of objects.\n", "\n", "2. **Stretch**. $\\begin{bmatrix} 2 & 0 \\\\ 0 & 1.5 \\end{bmatrix}$\n", " - This matrix stretches the $x$-direction by 2 and compresses $y$-direction by 1.5. It changes the size of the object, expanding it in the $x$-direction $y$-direction, while keeping its shape intact without any angular distortion.\n", "\n", "3. **Stretch/compress**. $\\begin{bmatrix} 3 & 0 \\\\ 0 & 0.75 \\end{bmatrix}$\n", " - This matrix stretches the object by a factor of 3 along the $x$-axis and compresses it by a factor of 0.75 along the $y$-axis. This operation changes the proportions of the object without distorting angles, stretching it non-uniformly.\n", "\n", "4. **Identity (No Change)**. $\\begin{bmatrix} 1 & 0 \\\\ 0 & 1 \\end{bmatrix}$\n", " - The identity matrix leaves the object unchanged. Applying this transformation results in the same object without any alteration to its size, shape, or orientation.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [ "hide-input" ] }, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "# Define the original vector\n", "vector = np.array([2, 3])\n", "\n", "# Define the transformation matrices\n", "shear_scale_matrix = np.array([[2, 1], [0, 0.5]])\n", "scaling_matrix = np.array([[2, 0], [0, 1.5]])\n", "stretching_matrix = np.array([[3, 0], [0, 0.75]])\n", "identity_matrix = np.eye(2)\n", "\n", "# Apply the transformations\n", "transformed_vector = shear_scale_matrix @ vector\n", "scaled_vector = scaling_matrix @ vector\n", "stretched_vector = stretching_matrix @ vector\n", "identity_vector = identity_matrix @ vector\n", "\n", "# Create subplots for the four transformations\n", "fig, axs = plt.subplots(2, 2, figsize=(12, 12))\n", "\n", "# Plot 1: Shearing and Scaling Transformation\n", "axs[0, 0].axhline(0, color='black', linewidth=0.5)\n", "axs[0, 0].axvline(0, color='black', linewidth=0.5)\n", "axs[0, 0].quiver(0, 0, vector[0], vector[1], angles='xy', scale_units='xy', scale=1, color='blue', label='Original Vector')\n", "axs[0, 0].quiver(0, 0, transformed_vector[0], transformed_vector[1], angles='xy', scale_units='xy', scale=1, color='red', label='Sheared & Scaled Vector')\n", "axs[0, 0].set_xlim(-6, 6)\n", "axs[0, 0].set_ylim(-6, 6)\n", "axs[0, 0].set_title('Shearing and Scaling Transformation')\n", "axs[0, 0].legend()\n", "axs[0, 0].grid(True)\n", "axs[0, 0].set_aspect('equal', adjustable='box')\n", "\n", "# Plot 2: Scaling Transformation\n", "axs[0, 1].axhline(0, color='black', linewidth=0.5)\n", "axs[0, 1].axvline(0, color='black', linewidth=0.5)\n", "axs[0, 1].quiver(0, 0, vector[0], vector[1], angles='xy', scale_units='xy', scale=1, color='blue', label='Original Vector')\n", "axs[0, 1].quiver(0, 0, scaled_vector[0], scaled_vector[1], angles='xy', scale_units='xy', scale=1, color='green', label='Scaled Vector')\n", "axs[0, 1].set_xlim(-6, 6)\n", "axs[0, 1].set_ylim(-6, 6)\n", "axs[0, 1].set_title('Scaling Transformation')\n", "axs[0, 1].legend()\n", "axs[0, 1].grid(True)\n", "axs[0, 1].set_aspect('equal', adjustable='box')\n", "\n", "# Plot 3: Stretching Transformation\n", "axs[1, 0].axhline(0, color='black', linewidth=0.5)\n", "axs[1, 0].axvline(0, color='black', linewidth=0.5)\n", "axs[1, 0].quiver(0, 0, vector[0], vector[1], angles='xy', scale_units='xy', scale=1, color='blue', label='Original Vector')\n", "axs[1, 0].quiver(0, 0, stretched_vector[0], stretched_vector[1], angles='xy', scale_units='xy', scale=1, color='purple', label='Stretched Vector')\n", "axs[1, 0].set_xlim(-6, 6)\n", "axs[1, 0].set_ylim(-6, 6)\n", "axs[1, 0].set_title('Stretching Transformation')\n", "axs[1, 0].legend()\n", "axs[1, 0].grid(True)\n", "axs[1, 0].set_aspect('equal', adjustable='box')\n", "\n", "# Plot 4: Identity Transformation\n", "axs[1, 1].axhline(0, color='black', linewidth=0.5)\n", "axs[1, 1].axvline(0, color='black', linewidth=0.5)\n", "axs[1, 1].quiver(0, 0, vector[0], vector[1], angles='xy', scale_units='xy', scale=1, color='blue', label='Original Vector')\n", "axs[1, 1].quiver(0, 0, identity_vector[0], identity_vector[1], angles='xy', scale_units='xy', scale=1, color='orange', label='Identity Vector (No Change)')\n", "axs[1, 1].set_xlim(-6, 6)\n", "axs[1, 1].set_ylim(-6, 6)\n", "axs[1, 1].set_title('Identity Transformation (No Change)')\n", "axs[1, 1].legend()\n", "axs[1, 1].grid(True)\n", "axs[1, 1].set_aspect('equal', adjustable='box')\n", "\n", "# Display the plots\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Visualize action of a matrix on several vectors" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Lets pick four vectors and act our matrix on all of them to see how the relationship between their distances and angles changes\n", "\n", "$$(0,0), (0.5,0.5), (0.5, 1.5), (0,1), (0,0)$$\n", "\n", "- We connect the points defined by this vectors to aid our visualization" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [ "hide-input" ] }, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "\n", "coords = np.array([[0,0],[0.5,0.5], [0.5,1.5], [0,1], [0,0]] )\n", "coords = coords.transpose()\n", "\n", "x = coords[0,:]\n", "y = coords[1,:]\n", "\n", "# Create the figure and axes objects\n", "fig, ax = plt.subplots()\n", "\n", "# Plot the points. x and y are original vectors, x_LT1 and y_LT1 are images\n", "ax.plot(x,y,'ro')\n", "\n", "ax.axis([-2,2,-1,2])\n", "# Connect the points by lines\n", "ax.plot(x,y,'r',ls=\"--\", label='x')\n", "ax.grid(True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Stretch matrix A\n", "\n", "$$\n", "\\begin{equation}\n", "A = \\left[ \\begin{array}{cc} 2 & 0 \\\\ 0 & 1 \\end{array}\\right]\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [ "hide-input" ] }, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "\n", "A = np.array([[2,0],[0,1]]) \n", "\n", "A_coords = A@coords\n", "\n", "###\n", "x_LT1 = A_coords[0,:]\n", "y_LT1 = A_coords[1,:]\n", "\n", "# Create the figure and axes objects\n", "fig, ax = plt.subplots()\n", "\n", "# Plot the points. x and y are original vectors, x_LT1 and y_LT1 are images\n", "ax.plot(x,y,'ro')\n", "ax.plot(x_LT1,y_LT1,'bo')\n", "\n", "# Connect the points by lines\n", "ax.plot(x,y,'r',ls=\"--\", label='x')\n", "ax.plot(x_LT1,y_LT1,'b', label='Ax')\n", "\n", "# Edit some settings \n", "ax.axvline(x=0,color=\"k\",ls=\":\")\n", "ax.axhline(y=0,color=\"k\",ls=\":\")\n", "ax.grid(True)\n", "ax.axis([-2,2,-1,2])\n", "ax.set_aspect('equal')\n", "plt.legend()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Rotation matrix R\n", "\n", "To rotate vectors in the plane, we choose an angle $\\theta$ and write down the matrix that represents the rotation counterclockwise by an angle $\\theta$. Basic trigonometry can be used to calculate the columns in this case. \n", "\n", "\\begin{equation}\n", "R = \\left[ \\begin{array}{cc} \\cos\\theta & -\\sin\\theta \\\\ \\sin\\theta & \\cos\\theta \\end{array}\\right]\n", "\\end{equation}" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [ "hide-input" ] }, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "\n", "theta = np.pi/6\n", "R = np.array([[ np.cos(theta), -np.sin(theta)],\n", " [np.sin(theta), np.cos(theta)]])\n", "R_coords = R@coords\n", "\n", "x_LT3 = R_coords[0,:]\n", "y_LT3 = R_coords[1,:]\n", "\n", "# Create the figure and axes objects\n", "fig, ax = plt.subplots()\n", "\n", "# Plot the points. x and y are original vectors, x_LT1 and y_LT1 are images\n", "ax.plot(x,y,'ro')\n", "ax.plot(x_LT3,y_LT3,'bo')\n", "\n", "# Connect the points by lines\n", "ax.plot(x,y,'r',ls=\"--\")\n", "ax.plot(x_LT3,y_LT3,'b')\n", "\n", "# Edit some settings \n", "ax.axvline(x=0,color=\"k\",ls=\":\")\n", "ax.axhline(y=0,color=\"k\",ls=\":\")\n", "ax.grid(True)\n", "ax.axis([-2,2,-1,2])\n", "ax.set_aspect('equal')\n", "ax.set_title(\"Rotation\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Shear Matrix, S\n", "\n", "In the study of mechanics, shear forces occur when one force acts on some part of a body while a second force acts on another part of the body but in the opposite direction. To visualize this, imagine a deck of playing cards resting on the table, and then while resting your hand on top of the deck, sliding your hand parallel to the table. \n", "\n", "$$\n", "\\begin{equation}\n", "S= \\left[ \\begin{array}{cc} 1 & 2 \\\\ 0 & 1 \\end{array}\\right]\n", "\\end{equation}\n", "$$" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [ "hide-input" ] }, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "\n", "S = np.array([[1,2],[0,1]])\n", "S_coords = S@coords\n", "\n", "x_LT4 = S_coords[0,:]\n", "y_LT4 = S_coords[1,:]\n", "\n", "\n", "# Create the figure and axes objects\n", "fig, ax = plt.subplots()\n", "\n", "# Plot the points. x and y are original vectors, x_LT1 and y_LT1 are images\n", "ax.plot(x,y,'ro')\n", "ax.plot(x_LT4,y_LT4,'bo')\n", "\n", "# Connect the points by lines\n", "ax.plot(x,y,'r',ls=\"--\")\n", "ax.plot(x_LT4,y_LT4,'b')\n", "\n", "# Edit some settings \n", "ax.axvline(x=0,color=\"k\",ls=\":\")\n", "ax.axhline(y=0,color=\"k\",ls=\":\")\n", "ax.grid(True)\n", "ax.axis([-2,4,-1,2])\n", "ax.set_aspect('equal')\n", "ax.set_title(\"Shear\");" ] } ], "metadata": { "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 2 }