Tensors & Multilinear Algebra

Historical Context

Tensor calculus was developed by Gregorio Ricci-Curbastro and Tullio Levi-Civita in the 1890s for differential geometry. Einstein adopted their notation for general relativity (1915), establishing the Einstein summation convention. The modern algebraic treatment via tensor products of vector spaces emerged from the work of Whitney, Bourbaki, and others in the mid-20th century.

Today, tensors are central to physics (stress, inertia, electromagnetic field), computer science (deep learning weight tensors), data science (multiway data analysis), and quantum information (entanglement). Tensor decompositions (CP, Tucker) generalize matrix factorization to higher-order data structures.

4.1 Multilinear Maps and Tensor Products

A tensor of type $(r, s)$ on a vector space $V$ is an element of$V^{\otimes r} \otimes (V^*)^{\otimes s}$—a multilinear map that takes $r$ covectors and $s$ vectors to produce a scalar. Matrices are $(1,1)$-tensors; the metric tensor in Riemannian geometry is a $(0,2)$-tensor.

4.2 Einstein Summation Convention

In the Einstein convention, repeated indices imply summation: $A^i{}_j B^j{}_k$ means$\sum_j A^i{}_j B^j{}_k$. This compact notation handles operations like:

• Contraction: $T^i{}_{ij} = \sum_j T^i{}_{ij}$ (trace-like operation)
• Raising/lowering indices: $v^i = g^{ij}v_j$ using the metric tensor
• Covariant derivative: $\nabla_i T^j{}_k = \partial_i T^j{}_k + \Gamma^j{}_{il}T^l{}_k - \Gamma^l{}_{ik}T^j{}_l$

NumPy's $\texttt{einsum}$ function directly implements Einstein summation, enabling efficient computation of arbitrary tensor contractions.

4.3 Symmetric and Antisymmetric Tensors

Any tensor can be decomposed into symmetric and antisymmetric parts. The wedge product $\alpha \wedge \beta = \alpha \otimes \beta - \beta \otimes \alpha$ generates theexterior algebra $\Lambda(V)$, which governs differential forms, determinants, and cross products.

In $\mathbb{R}^3$, the wedge product of two vectors is equivalent to the cross product. In general, $\Lambda^k(V)$ has dimension $\binom{n}{k}$, and the top exterior power $\Lambda^n(V)$ is 1-dimensional, explaining why the determinant is unique up to scale.

4.4 Tensor Decompositions

The CP decomposition (CANDECOMP/PARAFAC) expresses a tensor as a sum of rank-1 tensors:$\mathcal{T} = \sum_{r=1}^R a_r \otimes b_r \otimes c_r$. The minimal $R$ is the tensor rank—unlike matrix rank, tensor rank is NP-hard to compute in general.

The Tucker decomposition $\mathcal{T} = \mathcal{G} \times_1 A \times_2 B \times_3 C$generalizes PCA to higher order, with a core tensor $\mathcal{G}$ and factor matrices. Higher-order SVD (HOSVD) provides a truncated Tucker decomposition analogous to truncated SVD.

4.5 Applications

• Continuum mechanics: The stress tensor $\sigma_{ij}$ and strain tensor $\epsilon_{ij}$ are symmetric $(0,2)$-tensors
• General relativity: The Riemann curvature tensor $R^i{}_{jkl}$ encodes spacetime geometry
• Deep learning: Weight tensors in convolutional networks, attention mechanisms
• Quantum information: Entangled states are tensors that cannot be factored
• Chemometrics: Fluorescence excitation-emission matrices, multiway factor analysis

Computational Laboratory

This simulation demonstrates tensor products, Einstein summation via np.einsum, symmetric/antisymmetric decomposition, CP tensor decomposition, and stress tensor analysis.