Part I: Foundations — Chapter 3

Matrices & Determinants

Matrix operations, determinant formulas, Cramer's rule, and invertibility

3.1 Matrix Operations

A matrix $A \in \mathbb{F}^{m \times n}$ is a rectangular array of scalars with $m$ rows and $n$ columns. The entry in row $i$ and column$j$ is denoted $a_{ij}$ or $(A)_{ij}$.

Fundamental Operations

Addition: $(A + B)_{ij} = a_{ij} + b_{ij}$ (same dimensions required)
Scalar multiplication: $(\alpha A)_{ij} = \alpha \cdot a_{ij}$
Matrix multiplication: $(AB)_{ij} = \sum_{k=1}^{n} a_{ik} b_{kj}$ where $A \in \mathbb{F}^{m \times n}$, $B \in \mathbb{F}^{n \times p}$
Transpose: $(A^T)_{ij} = a_{ji}$
Conjugate transpose: $(A^*)_{ij} = \overline{a_{ji}}$

Matrix multiplication is associative ($(AB)C = A(BC)$) and distributes over addition, but is generally not commutative: $AB \neq BA$ in general. This non-commutativity is one of the most important features distinguishing linear algebra from scalar arithmetic.

Special Matrices

Identity: $I_n$ with $(I_n)_{ij} = \delta_{ij}$ (Kronecker delta)
Diagonal: $a_{ij} = 0$ for $i \neq j$
Symmetric: $A^T = A$
Skew-symmetric: $A^T = -A$
Orthogonal: $A^T A = I$ (equivalently, $A^{-1} = A^T$)
Unitary: $A^* A = I$ (complex analog of orthogonal)
Hermitian: $A^* = A$ (complex analog of symmetric)
Nilpotent: $A^k = 0$ for some $k$
Idempotent: $A^2 = A$ (projection matrices)

Derivation 1: $(AB)^T = B^T A^T$

We verify the transpose of a product reverses the order:

$$((AB)^T)_{ij} = (AB)_{ji} = \sum_{k} a_{jk} b_{ki} = \sum_{k} (B^T)_{ik} (A^T)_{kj} = (B^T A^T)_{ij}$$

This "reversal rule" extends to products of any length: $(A_1 A_2 \cdots A_k)^T = A_k^T \cdots A_2^T A_1^T$.

3.2 The Determinant

The determinant is a scalar-valued function on square matrices that encodes essential geometric and algebraic information. It measures the signed volume scaling factor of the linear transformation represented by the matrix.

Leibniz Formula

For an $n \times n$ matrix $A$:

$$\det(A) = \sum_{\sigma \in S_n} \text{sgn}(\sigma) \prod_{i=1}^{n} a_{i,\sigma(i)}$$

where the sum is over all permutations $\sigma$ of $\{1, \ldots, n\}$ and$\text{sgn}(\sigma) = (-1)^{\text{inv}(\sigma)}$ is the sign of the permutation. This sum has $n!$ terms, making it impractical for computation but theoretically fundamental.

Derivation 2: Cofactor Expansion

The cofactor expansion (or Laplace expansion) along row $i$ gives:

$$\det(A) = \sum_{j=1}^{n} (-1)^{i+j} a_{ij} M_{ij}$$

where $M_{ij}$ is the minor—the determinant of the$(n-1) \times (n-1)$ matrix obtained by deleting row $i$ and column $j$. The quantity $C_{ij} = (-1)^{i+j} M_{ij}$ is the cofactor.

To derive this from the Leibniz formula, we group the permutations by the value $\sigma(i) = j$. For each fixed $j$, the permutations with $\sigma(i) = j$ are in bijection with permutations of $\{1,\ldots,n\} \setminus \{j\}$, and the sign picks up a factor of$(-1)^{i+j}$ from the transpositions needed to move $j$ to position $i$.

For 2x2 and 3x3 Matrices

$$\det\begin{pmatrix} a & b \\ c & d \end{pmatrix} = ad - bc$$

$$\det\begin{pmatrix} a & b & c \\ d & e & f \\ g & h & i \end{pmatrix} = a(ei - fh) - b(di - fg) + c(dh - eg)$$

3.3 Properties of Determinants

Fundamental Properties

P1 (Multiplicativity): $\det(AB) = \det(A)\det(B)$
P2 (Transpose): $\det(A^T) = \det(A)$
P3 (Scaling): $\det(\alpha A) = \alpha^n \det(A)$ for $A \in \mathbb{F}^{n \times n}$
P4 (Inverse): $\det(A^{-1}) = \frac{1}{\det(A)}$
P5 (Triangular): $\det(A) = \prod_{i=1}^{n} a_{ii}$ for triangular $A$
P6 (Row swap): Swapping two rows negates the determinant
P7 (Row scaling): Multiplying a row by $c$ multiplies $\det$ by $c$
P8 (Row addition): Adding a multiple of one row to another leaves $\det$ unchanged

Derivation 3: Multiplicativity of Determinant

We sketch the proof that $\det(AB) = \det(A)\det(B)$. The key insight is that the determinant is the unique alternating multilinear function of the columns (or rows) that equals 1 on the identity matrix.

Fix $B$ and define $\phi(A) = \det(AB)$. Since the columns of $AB$are $A\mathbf{b}_1, \ldots, A\mathbf{b}_n$, and $\det$ is multilinear in columns,$\phi$ is multilinear in the columns of $B$. It is also alternating (if two columns of $B$ are equal, then two columns of $AB$ are equal, so $\det(AB) = 0$). By the uniqueness characterization, $\phi(A) = \phi(I) \cdot \det(B) = \det(A)\det(B)$.

3.4 Cramer's Rule

Theorem: Cramer's Rule

For the system $A\mathbf{x} = \mathbf{b}$ where $A$ is $n \times n$with $\det(A) \neq 0$, the unique solution is:

$$x_j = \frac{\det(A_j)}{\det(A)}$$

where $A_j$ is the matrix obtained by replacing column $j$ of $A$with the vector $\mathbf{b}$.

Derivation 4: Proof of Cramer's Rule

We use the adjugate (classical adjoint) matrix. Recall the cofactor matrix$C$ where $C_{ij} = (-1)^{i+j} M_{ij}$. The adjugate is$\text{adj}(A) = C^T$, and it satisfies:

$$A \cdot \text{adj}(A) = \det(A) \cdot I$$

This follows because the $(i,j)$ entry of $A \cdot \text{adj}(A)$ is$\sum_k a_{ik} C_{jk}$, which equals $\det(A)$ when $i = j$(cofactor expansion along row $i$) and 0 when $i \neq j$ (expansion of a matrix with two identical rows).

Therefore $A^{-1} = \frac{1}{\det(A)} \text{adj}(A)$, and the solution$\mathbf{x} = A^{-1}\mathbf{b}$ gives $x_j = \frac{1}{\det(A)} \sum_k C_{jk} b_k$, which is precisely $\frac{\det(A_j)}{\det(A)}$ by cofactor expansion of $A_j$.

Practical Note

While Cramer's rule is elegant, it is computationally expensive ($O(n \cdot n!)$ via cofactor expansion, or $O(n^4)$ if each determinant is computed by elimination). Gaussian elimination ($O(n^3)$) is far more efficient for solving systems. However, Cramer's rule is invaluable for theoretical analysis and for deriving formulas for small systems.

3.5 Invertibility

The Invertible Matrix Theorem

For an $n \times n$ matrix $A$, the following are equivalent:

1. $A$ is invertible
2. $\det(A) \neq 0$
3. $\text{rank}(A) = n$
4. $\ker(A) = \{\mathbf{0}\}$
5. The columns of $A$ are linearly independent
6. The columns of $A$ span $\mathbb{F}^n$
7. $A\mathbf{x} = \mathbf{b}$ has a unique solution for every $\mathbf{b}$
8. $A^T$ is invertible
9. The eigenvalues of $A$ are all nonzero
10. $0$ is not a singular value of $A$

Derivation 5: Inverse via Adjugate

The explicit formula for the inverse in terms of cofactors is:

$$A^{-1} = \frac{1}{\det(A)} \text{adj}(A) = \frac{1}{\det(A)} \begin{pmatrix} C_{11} & C_{21} & \cdots & C_{n1} \\ C_{12} & C_{22} & \cdots & C_{n2} \\ \vdots & \vdots & \ddots & \vdots \\ C_{1n} & C_{2n} & \cdots & C_{nn} \end{pmatrix}$$

For a $2 \times 2$ matrix, this simplifies to the well-known formula:

$$\begin{pmatrix} a & b \\ c & d \end{pmatrix}^{-1} = \frac{1}{ad - bc}\begin{pmatrix} d & -b \\ -c & a \end{pmatrix}$$

3.6 Historical Development

Seki Takakazu and Leibniz (1683–1693)

The determinant was independently discovered by the Japanese mathematician Seki Takakazu (1683) and Gottfried Wilhelm Leibniz (1693). Seki developed the concept while solving systems of equations using what he called "kihou" (literal: technique of simultaneous equations). Leibniz recognized that the sign pattern of determinants encodes permutation parity.

Gabriel Cramer (1750)

Cramer published his rule for solving systems of linear equations in his Introduction to the Analysis of Algebraic Curves. Though the formula bears his name, similar ideas had been explored by Colin Maclaurin as early as 1729.

Augustin-Louis Cauchy (1812)

Cauchy systematized the theory of determinants, proving the multiplicative property$\det(AB) = \det(A)\det(B)$ and establishing much of the modern notation. His 84-page memoir on the subject was a landmark in linear algebra.

James Joseph Sylvester (1850)

Sylvester coined the term "matrix" in 1850 (from the Latin for "womb"), viewing matrices as generating arrays from which determinants could be "born." His collaboration with Cayley established matrix algebra as a distinct branch of mathematics.

3.7 Applications

Volume and Orientation

The absolute value $|\det(A)|$ gives the volume scaling factor of the linear transformation. For vectors $\mathbf{v}_1, \ldots, \mathbf{v}_n$, the volume of the parallelepiped they span is $|\det(\mathbf{v}_1 \mid \cdots \mid \mathbf{v}_n)|$. The sign of the determinant indicates whether the map preserves or reverses orientation.

Cross Product

The cross product in $\mathbb{R}^3$ can be expressed as a determinant:$\mathbf{a} \times \mathbf{b} = \det\begin{pmatrix} \mathbf{i} & \mathbf{j} & \mathbf{k} \\ a_1 & a_2 & a_3 \\ b_1 & b_2 & b_3 \end{pmatrix}$. This extends to the $n$-dimensional setting via the exterior product.

Change of Variables in Integration

The Jacobian determinant appears in the multivariable change of variables formula:$\int_{\phi(U)} f(\mathbf{y})\, d\mathbf{y} = \int_U f(\phi(\mathbf{x})) \left|\det\left(\frac{\partial \phi}{\partial \mathbf{x}}\right)\right| d\mathbf{x}$. This is why the determinant measures volume distortion.

Characteristic Polynomial

The characteristic polynomial $p(\lambda) = \det(A - \lambda I)$ is the key to eigenvalue theory. Its roots are the eigenvalues, and its coefficients encode the trace ($\text{tr}(A) = \sum \lambda_i$) and determinant ($\det(A) = \prod \lambda_i$).

3.8 Computational Exploration

The following simulation implements determinant computation via both the Leibniz formula and cofactor expansion, verifies key determinant properties, demonstrates Cramer's rule, and visualizes how determinants relate to geometric transformations.

Matrices and Determinants: Computation and Visualization

Python

matrices_determinants.py225 lines

import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt

# ============================================================
# Matrices and Determinants
# ============================================================

# --- 1. Matrix Operations ---
print("=" * 60)
print("MATRIX OPERATIONS")
print("=" * 60)

A = np.array([[1, 2, 3],
              [4, 5, 6],
              [7, 8, 10]], dtype=float)

B = np.array([[2, 0, 1],
              [1, 3, 0],
              [0, 1, 2]], dtype=float)

print("A =")
print(A)
print("B =")
print(B)
print()
print("A + B =")
print(A + B)
print()
print("A @ B (matrix product) =")
print(A @ B)
print()
print("B @ A =")
print(B @ A)
print()
print("AB != BA (non-commutative):", not np.allclose(A @ B, B @ A))

# Transpose properties
print()
print("(AB)^T = B^T A^T:", np.allclose((A @ B).T, B.T @ A.T))

# --- 2. Determinant via cofactor expansion ---
print()
print("=" * 60)
print("DETERMINANT - COFACTOR EXPANSION")
print("=" * 60)

def cofactor_det(M):
    n = M.shape[0]
    if n == 1:
        return M[0, 0]
    if n == 2:
        return M[0, 0] * M[1, 1] - M[0, 1] * M[1, 0]
    det_val = 0.0
    for j in range(n):
        minor = np.delete(np.delete(M, 0, axis=0), j, axis=1)
        det_val += ((-1) ** j) * M[0, j] * cofactor_det(minor)
    return det_val

det_A_cofactor = cofactor_det(A)
det_A_numpy = np.linalg.det(A)
print(f"det(A) via cofactor expansion: {det_A_cofactor:.6f}")
print(f"det(A) via numpy: {det_A_numpy:.6f}")
print(f"Match: {np.isclose(det_A_cofactor, det_A_numpy)}")

# --- 3. Determinant via Leibniz formula ---
print()
print("=" * 60)
print("DETERMINANT - LEIBNIZ FORMULA (3x3)")
print("=" * 60)

from itertools import permutations

def sign_of_permutation(perm):
    perm = list(perm)
    n = len(perm)
    inversions = 0
    for i in range(n):
        for j in range(i + 1, n):
            if perm[i] > perm[j]:
                inversions += 1
    return (-1) ** inversions

def leibniz_det(M):
    n = M.shape[0]
    det_val = 0.0
    for perm in permutations(range(n)):
        sign = sign_of_permutation(perm)
        product = 1.0
        for i in range(n):
            product *= M[i, perm[i]]
        det_val += sign * product
    return det_val

det_A_leibniz = leibniz_det(A)
print(f"det(A) via Leibniz: {det_A_leibniz:.6f}")
print(f"Number of permutations for 3x3: {6}")

# --- 4. Properties of determinants ---
print()
print("=" * 60)
print("DETERMINANT PROPERTIES")
print("=" * 60)

print(f"det(A) = {np.linalg.det(A):.6f}")
print(f"det(B) = {np.linalg.det(B):.6f}")
print(f"det(AB) = {np.linalg.det(A @ B):.6f}")
print(f"det(A)*det(B) = {np.linalg.det(A) * np.linalg.det(B):.6f}")
print(f"det(AB) = det(A)*det(B): {np.isclose(np.linalg.det(A @ B), np.linalg.det(A) * np.linalg.det(B))}")
print()
print(f"det(A^T) = {np.linalg.det(A.T):.6f}")
print(f"det(A^T) = det(A): {np.isclose(np.linalg.det(A.T), np.linalg.det(A))}")
print()
alpha_val = 3.0
print(f"det({alpha_val}*A) = {np.linalg.det(alpha_val * A):.6f}")
print(f"{alpha_val}^3 * det(A) = {alpha_val**3 * np.linalg.det(A):.6f}")
print(f"det(cA) = c^n * det(A): {np.isclose(np.linalg.det(alpha_val * A), alpha_val**3 * np.linalg.det(A))}")

# --- 5. Cramer's rule ---
print()
print("=" * 60)
print("CRAMER'S RULE")
print("=" * 60)

b = np.array([1, 2, 3], dtype=float)
det_main = np.linalg.det(A)
print(f"System Ax = b, where b = {b}")
print(f"det(A) = {det_main:.6f}")

x_cramer = np.zeros(3)
for i in range(3):
    A_mod = A.copy()
    A_mod[:, i] = b
    x_cramer[i] = np.linalg.det(A_mod) / det_main

x_direct = np.linalg.solve(A, b)
print(f"Solution via Cramer: {x_cramer}")
print(f"Solution via solve:  {x_direct}")
print(f"Match: {np.allclose(x_cramer, x_direct)}")

# --- 6. Invertibility ---
print()
print("=" * 60)
print("INVERTIBILITY")
print("=" * 60)

A_inv = np.linalg.inv(A)
print("A^(-1) =")
print(A_inv)
print(f"A @ A^(-1) = I: {np.allclose(A @ A_inv, np.eye(3))}")
print(f"det(A^(-1)) = {np.linalg.det(A_inv):.6f}")
print(f"1/det(A) = {1.0/np.linalg.det(A):.6f}")
print(f"det(A^(-1)) = 1/det(A): {np.isclose(np.linalg.det(A_inv), 1.0/np.linalg.det(A))}")

# Singular matrix
C = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]], dtype=float)
print(f"\nSingular matrix C det: {np.linalg.det(C):.10f} (approximately 0)")
print(f"C is singular: {np.abs(np.linalg.det(C)) < 1e-10}")

# --- 7. Visualization ---
fig, axes = plt.subplots(1, 3, figsize=(16, 5))

# Plot 1: Determinant as area scaling
ax = axes[0]
theta = np.linspace(0, 2 * np.pi, 100)
unit_sq_x = [0, 1, 1, 0, 0]
unit_sq_y = [0, 0, 1, 1, 0]

matrices_2d = [
    np.array([[2, 0], [0, 1]]),
    np.array([[1, 1], [0, 1]]),
    np.array([[np.cos(0.5), -np.sin(0.5)], [np.sin(0.5), np.cos(0.5)]]),
]
labels_2d = ["Scale (det=2)", "Shear (det=1)", "Rotate (det=1)"]
colors_2d = ["#3b82f6", "#0ea5e9", "#38bdf8"]

ax.plot(unit_sq_x, unit_sq_y, "w--", linewidth=1.5, alpha=0.5, label="Unit square")
for M, lab, col in zip(matrices_2d, labels_2d, colors_2d):
    sq = M @ np.array([unit_sq_x, unit_sq_y])
    ax.plot(sq[0], sq[1], color=col, linewidth=2, label=lab)
    ax.fill(sq[0], sq[1], color=col, alpha=0.15)
ax.set_xlim(-1.5, 3)
ax.set_ylim(-1, 2.5)
ax.set_aspect("equal")
ax.grid(True, alpha=0.3)
ax.legend(fontsize=8)
ax.set_title("Determinant as Area Scaling", fontsize=12, fontweight="bold")
ax.set_facecolor("#0f172a")

# Plot 2: Cofactor expansion tree
ax = axes[1]
sizes = [3, 4, 5, 6, 7, 8]
from math import factorial
flops = [factorial(n) for n in sizes]
ax.semilogy(sizes, flops, "o-", color="#3b82f6", linewidth=2, markersize=8)
ax.set_xlabel("Matrix Size n", fontsize=11)
ax.set_ylabel("Operations (n!)", fontsize=11)
ax.set_title("Leibniz Formula Complexity", fontsize=12, fontweight="bold")
ax.grid(True, alpha=0.3)
ax.set_facecolor("#0f172a")

# Plot 3: det(A) as function of perturbation
ax = axes[2]
epsilons = np.linspace(-2, 2, 200)
det_vals = []
base_M = np.array([[1, 2], [3, 4]], dtype=float)
for eps in epsilons:
    M_pert = base_M.copy()
    M_pert[0, 0] += eps
    det_vals.append(np.linalg.det(M_pert))
ax.plot(epsilons, det_vals, color="#0ea5e9", linewidth=2)
ax.axhline(y=0, color="#f87171", linewidth=1, linestyle="--", alpha=0.7)
ax.set_xlabel("Perturbation epsilon", fontsize=11)
ax.set_ylabel("det(A + epsilon*E_11)", fontsize=11)
ax.set_title("Determinant Sensitivity", fontsize=12, fontweight="bold")
ax.grid(True, alpha=0.3)
ax.set_facecolor("#0f172a")

plt.tight_layout()
plt.savefig("output.png", dpi=150, bbox_inches="tight",
            facecolor="#0f172a", edgecolor="none")
plt.show()
print("\nPlot saved: Matrices and determinants visualizations")

Click Run to execute the Python code

Code will be executed with Python 3 on the server

← Linear Maps Systems of Equations →