Vector Spaces
Axioms, subspaces, span, linear independence, basis, and dimension
1.1 Axioms of a Vector Space
A vector space (or linear space) over a field $\mathbb{F}$ is a set $V$ equipped with two operations—vector addition and scalar multiplication—satisfying ten fundamental axioms. In linear algebra, we typically work over $\mathbb{F} = \mathbb{R}$ or $\mathbb{F} = \mathbb{C}$.
Definition: Vector Space
A vector space over a field $\mathbb{F}$ is a set $V$ together with operations $+: V \times V \to V$ and $\cdot: \mathbb{F} \times V \to V$ such that for all $\mathbf{u}, \mathbf{v}, \mathbf{w} \in V$ and $\alpha, \beta \in \mathbb{F}$:
- A1 (Closure under +): $\mathbf{u} + \mathbf{v} \in V$
- A2 (Commutativity): $\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}$
- A3 (Associativity): $(\mathbf{u} + \mathbf{v}) + \mathbf{w} = \mathbf{u} + (\mathbf{v} + \mathbf{w})$
- A4 (Zero vector): $\exists\, \mathbf{0} \in V$ such that $\mathbf{u} + \mathbf{0} = \mathbf{u}$
- A5 (Additive inverse): $\forall\, \mathbf{u} \in V,\, \exists\, (-\mathbf{u})$ with $\mathbf{u} + (-\mathbf{u}) = \mathbf{0}$
- S1 (Closure under scalar mult): $\alpha \mathbf{u} \in V$
- S2 (Distributivity over vectors): $\alpha(\mathbf{u} + \mathbf{v}) = \alpha\mathbf{u} + \alpha\mathbf{v}$
- S3 (Distributivity over scalars): $(\alpha + \beta)\mathbf{u} = \alpha\mathbf{u} + \beta\mathbf{u}$
- S4 (Associativity of scalars): $(\alpha\beta)\mathbf{u} = \alpha(\beta\mathbf{u})$
- S5 (Multiplicative identity): $1 \cdot \mathbf{u} = \mathbf{u}$
The axioms are not independent—some follow from others in certain formulations—but this standard list provides a complete characterization. The key insight is that vector spaces abstract the essential algebraic properties of arrows in Euclidean space to a vastly more general setting.
Examples of Vector Spaces
- $\mathbb{R}^n$: The space of n-tuples of real numbers with componentwise operations.
- $\mathcal{P}_n(\mathbb{R})$: Polynomials of degree at most $n$ with real coefficients.
- $C([a,b])$: Continuous real-valued functions on $[a,b]$.
- $\mathbb{R}^{m \times n}$: The space of $m \times n$ real matrices.
- $\ell^2$: Square-summable sequences $(a_n)$ with $\sum |a_n|^2 < \infty$.
Derivation 1: Uniqueness of the Zero Vector
We prove that the zero vector is unique. Suppose $\mathbf{0}$ and $\mathbf{0}'$ are both zero vectors. Then:
The first equality uses $\mathbf{0}$ as a zero vector (A4), and the second uses commutativity (A2), and the third uses $\mathbf{0}'$ as a zero vector (A4). Thus $\mathbf{0} = \mathbf{0}'$.
Derivation 2: $0 \cdot \mathbf{v} = \mathbf{0}$
We derive that multiplying any vector by the scalar zero yields the zero vector:
Adding $-(0\mathbf{v})$ to both sides gives $\mathbf{0} = 0\mathbf{v}$. This result, while seemingly trivial, is not an axiom but a theorem derived from the axioms.
1.2 Subspaces
A subspace of a vector space $V$ is a subset $W \subseteq V$ that is itself a vector space under the same operations. The subspace test provides an efficient way to verify this property.
Subspace Test
A nonempty subset $W \subseteq V$ is a subspace if and only if for all$\mathbf{u}, \mathbf{v} \in W$ and $\alpha \in \mathbb{F}$:
This single condition simultaneously guarantees closure under addition (set $\alpha = 1$), closure under scalar multiplication (set $\mathbf{v} = \mathbf{0}$), and the existence of the zero vector (set $\alpha = -1, \mathbf{v} = \mathbf{u}$).
Derivation 3: Intersection of Subspaces
We prove that the intersection of any collection of subspaces is itself a subspace. Let $\{W_i\}_{i \in I}$ be a family of subspaces of $V$, and let$W = \bigcap_{i \in I} W_i$.
First, $\mathbf{0} \in W_i$ for all $i$, so $\mathbf{0} \in W$ and$W \neq \emptyset$. Now let $\mathbf{u}, \mathbf{v} \in W$ and $\alpha \in \mathbb{F}$. For each $i$, since $\mathbf{u}, \mathbf{v} \in W_i$ and $W_i$ is a subspace, we have $\alpha\mathbf{u} + \mathbf{v} \in W_i$. Since this holds for all $i$,$\alpha\mathbf{u} + \mathbf{v} \in W$. By the subspace test, $W$ is a subspace.
Important Subspace Examples
- Null space: $\ker(A) = \{\mathbf{x} \in \mathbb{R}^n : A\mathbf{x} = \mathbf{0}\}$ is a subspace of $\mathbb{R}^n$.
- Column space: $\text{Col}(A) = \{A\mathbf{x} : \mathbf{x} \in \mathbb{R}^n\}$ is a subspace of $\mathbb{R}^m$.
- Solution space: Solutions to a homogeneous system form a subspace.
- Not a subspace: The unit circle $\{(x,y) : x^2 + y^2 = 1\}$ is NOT a subspace (doesn't contain $\mathbf{0}$).
1.3 Span and Linear Combinations
A linear combination of vectors $\mathbf{v}_1, \ldots, \mathbf{v}_k \in V$ is any vector of the form:
where $\alpha_1, \ldots, \alpha_k \in \mathbb{F}$ are scalars.
Definition: Span
The span of a set $S = \{\mathbf{v}_1, \ldots, \mathbf{v}_k\}$ is the set of all linear combinations of vectors in $S$:
The span is always a subspace—in fact, it is the smallest subspace containing $S$.
We say that $S$ spans (or generates) $V$ if$\text{Span}(S) = V$. This means every vector in $V$ can be written as a linear combination of vectors in $S$.
Example: Spanning $\mathbb{R}^3$
The standard basis vectors $\mathbf{e}_1 = (1,0,0)$, $\mathbf{e}_2 = (0,1,0)$,$\mathbf{e}_3 = (0,0,1)$ span $\mathbb{R}^3$ because any vector$(a,b,c)$ can be written as $a\mathbf{e}_1 + b\mathbf{e}_2 + c\mathbf{e}_3$. However, the set $\{(1,0,0), (0,1,0)\}$ does not span $\mathbb{R}^3$since vectors with nonzero third component cannot be represented.
1.4 Linear Independence
Definition: Linear Independence
Vectors $\mathbf{v}_1, \ldots, \mathbf{v}_k$ are linearly independent if the only solution to:
is the trivial solution $\alpha_1 = \alpha_2 = \cdots = \alpha_k = 0$. Otherwise, the vectors are linearly dependent.
Linear independence captures the idea that no vector in the set is "redundant"—none can be expressed as a linear combination of the others. This is fundamental: if $\mathbf{v}_j = \sum_{i \neq j} \beta_i \mathbf{v}_i$, then $\sum_{i \neq j} \beta_i \mathbf{v}_i - \mathbf{v}_j = \mathbf{0}$ is a nontrivial relation.
Derivation 4: Steinitz Exchange Lemma
The Steinitz exchange lemma states: if $\{\mathbf{v}_1, \ldots, \mathbf{v}_m\}$ is linearly independent and each $\mathbf{v}_i$ is in the span of $\{\mathbf{w}_1, \ldots, \mathbf{w}_n\}$, then $m \leq n$.
The proof proceeds by induction. Since $\mathbf{v}_1 \in \text{Span}(\mathbf{w}_1, \ldots, \mathbf{w}_n)$and $\mathbf{v}_1 \neq \mathbf{0}$ (by linear independence), at least one $\mathbf{w}_j$ has a nonzero coefficient. We can exchange $\mathbf{w}_j$ with $\mathbf{v}_1$ while preserving the span. Repeating this process $m$ times, we replace $m$ of the $\mathbf{w}_j$'s with $\mathbf{v}_i$'s, which requires $m \leq n$.
Key Consequence
The Steinitz exchange lemma implies that any two bases of a finite-dimensional vector space have the same number of elements. This number is the dimension of the space.
1.5 Basis and Dimension
Definition: Basis
A basis for a vector space $V$ is a set $\mathcal{B} = \{\mathbf{e}_1, \ldots, \mathbf{e}_n\}$ that is:
- 1. Linearly independent: No vector is a linear combination of the others.
- 2. Spanning: $\text{Span}(\mathcal{B}) = V$.
Equivalently, $\mathcal{B}$ is a basis if every $\mathbf{v} \in V$ can be writtenuniquely as a linear combination of the basis vectors. This unique representation is what makes bases so useful—they provide a coordinate system for the vector space.
Derivation 5: Dimension Theorem for Subspaces
Let $W_1, W_2$ be finite-dimensional subspaces of $V$. We prove:
Let $\{\mathbf{u}_1, \ldots, \mathbf{u}_r\}$ be a basis for $W_1 \cap W_2$. Extend to a basis $\{\mathbf{u}_1, \ldots, \mathbf{u}_r, \mathbf{v}_1, \ldots, \mathbf{v}_s\}$of $W_1$ and a basis $\{\mathbf{u}_1, \ldots, \mathbf{u}_r, \mathbf{w}_1, \ldots, \mathbf{w}_t\}$of $W_2$. Then $\{\mathbf{u}_1, \ldots, \mathbf{u}_r, \mathbf{v}_1, \ldots, \mathbf{v}_s, \mathbf{w}_1, \ldots, \mathbf{w}_t\}$is a basis for $W_1 + W_2$, giving:
Dimension of Common Spaces
- $\dim(\mathbb{R}^n) = n$
- $\dim(\mathcal{P}_n) = n + 1$ (polynomials of degree $\leq n$)
- $\dim(\mathbb{R}^{m \times n}) = mn$ (matrices)
- $\dim(\text{Sym}_n(\mathbb{R})) = \frac{n(n+1)}{2}$ (symmetric matrices)
- $\dim(\text{Skew}_n(\mathbb{R})) = \frac{n(n-1)}{2}$ (skew-symmetric matrices)
1.6 Change of Basis
Given two bases $\mathcal{B} = \{\mathbf{e}_1, \ldots, \mathbf{e}_n\}$ and$\mathcal{B}' = \{\mathbf{e}'_1, \ldots, \mathbf{e}'_n\}$ of $V$, the change-of-basis matrix $P$ relates the coordinates:
where $P$ is the matrix whose columns are the coordinates of the new basis vectors expressed in the old basis: $P = \begin{pmatrix} [\mathbf{e}'_1]_{\mathcal{B}} & \cdots & [\mathbf{e}'_n]_{\mathcal{B}} \end{pmatrix}$.
If a linear map $T: V \to V$ has matrix representation $A$ with respect to$\mathcal{B}$, then its representation with respect to $\mathcal{B}'$ is:
This is the similarity transformation, and matrices related this way are called similar. Similar matrices represent the same linear map in different coordinate systems.
1.7 Direct Sums
A vector space $V$ is the direct sum of subspaces $W_1$ and$W_2$, written $V = W_1 \oplus W_2$, if:
- 1. $V = W_1 + W_2$ (every vector is the sum of vectors from each subspace)
- 2. $W_1 \cap W_2 = \{\mathbf{0}\}$ (the intersection is trivial)
Equivalently, every $\mathbf{v} \in V$ can be written uniquely as$\mathbf{v} = \mathbf{w}_1 + \mathbf{w}_2$ with $\mathbf{w}_i \in W_i$.
Example: $\mathbb{R}^3$ as a Direct Sum
Let $W_1$ be the $xy$-plane and $W_2$ be the $z$-axis. Then $\mathbb{R}^3 = W_1 \oplus W_2$ because every vector $(a,b,c)$ can be uniquely decomposed as $(a,b,0) + (0,0,c)$. Note that $\dim(\mathbb{R}^3) = \dim(W_1) + \dim(W_2) = 2 + 1 = 3$.
The direct sum generalizes to any finite collection of subspaces: $V = W_1 \oplus W_2 \oplus \cdots \oplus W_k$means every vector has a unique decomposition, and the dimension formula becomes:
1.8 Historical Development
The concept of a vector space evolved over more than a century, emerging from disparate mathematical traditions before being unified into the abstract framework we use today.
Hermann Grassmann (1844)
In his revolutionary Ausdehnungslehre (Theory of Extension), Grassmann developed a calculus of "extensive magnitudes" that anticipated modern vector space theory. He defined operations on abstract quantities that obeyed the laws we now recognize as vector space axioms, and even developed the exterior algebra (wedge product). His work was far ahead of its time and largely unappreciated during his lifetime.
Giuseppe Peano (1888)
Peano gave the first axiomatic definition of a vector space in his Calcolo geometrico secondo l'Ausdehnungslehre di H. Grassmann. His axiom system was essentially the one we use today, making him a pioneer of the abstract algebraic approach to linear algebra.
Stefan Banach and the Polish School (1920s–1930s)
The Lviv school of mathematics, led by Banach, extended vector space theory to infinite-dimensional settings, creating functional analysis. Banach spaces (complete normed vector spaces) became the foundation of modern analysis and quantum mechanics.
Modern Formalization (1940s–1960s)
The Bourbaki group systematized linear algebra in its modern abstract form. Paul Halmos's Finite-Dimensional Vector Spaces (1942) became the standard reference, presenting the subject in the coordinate-free style that emphasizes linear maps over matrices.
1.9 Applications
Computer Graphics and 3D Rendering
Every 3D scene in computer graphics is fundamentally a collection of vectors in $\mathbb{R}^3$(or $\mathbb{R}^4$ in homogeneous coordinates). Rotations, translations, scaling, and perspective projections are all linear (or affine) maps. The GPU performs billions of vector space operations per second to render real-time graphics.
The concept of basis is directly applied when choosing coordinate systems for models, cameras, and lights. Change-of-basis transformations convert between local model coordinates, world coordinates, camera coordinates, and screen coordinates.
Quantum Mechanics
Quantum states live in complex vector spaces (specifically, Hilbert spaces). The superposition principle—that a quantum system can be in a combination of states—is precisely the statement that quantum states form a vector space. Observables are linear operators, and measurements correspond to projections onto subspaces.
A qubit, the basic unit of quantum computing, is an element of $\mathbb{C}^2$, and an n-qubit system lives in the tensor product space $(\mathbb{C}^2)^{\otimes n}$, which has dimension $2^n$—the exponential growth of this dimension is the source of quantum computing's power.
Signal Processing
Signals (audio, images, sensor data) are vectors in high-dimensional spaces. The Fourier transform is a change of basis from the time domain to the frequency domain. Compression algorithms like MP3 and JPEG exploit the fact that most signals have sparse representations in appropriate bases (Fourier, wavelet, DCT), allowing us to discard components with small coefficients.
Error-Correcting Codes
In coding theory, codewords are vectors in $\mathbb{F}_2^n$ (the vector space over the field with two elements). A linear code is a subspace of $\mathbb{F}_2^n$, and error detection and correction exploit the structure of these subspaces. The Hamming code, Reed-Solomon codes, and LDPC codes all rely on linear algebra over finite fields.
Machine Learning
Feature spaces in machine learning are vector spaces where each data point is represented as a vector of features. Dimensionality reduction techniques (PCA, t-SNE) find lower-dimensional subspaces that capture most of the variance. Neural networks compute compositions of affine maps (matrix multiplications plus bias vectors) with nonlinear activations, and the weight space of a neural network is itself a high-dimensional vector space over which we optimize.
1.10 Computational Exploration
The following simulation verifies vector space axioms numerically, tests linear independence using matrix rank, demonstrates basis representation and change of basis, and visualizes span and subspace structure. We also verify the dimension theorem for sums of subspaces.
Vector Spaces: Axioms, Independence, Basis, and Dimension
PythonClick Run to execute the Python code
Code will be executed with Python 3 on the server