Tensor Products and Multilinear Maps

⚡ Key Takeaways

One-line statement: The tensor product $V \otimes W$ is the universal vector space through which all bilinear maps out of $V \times W$ factor uniquely as linear maps. Key result in words: Multilinear algebra is linearized — every multilinear map becomes an ordinary linear map on a single tensor space, turning analysis of $k$ -fold linear dependence into the study of one big vector space. Where it applies: Differential geometry (tensor fields), representation theory, quantum mechanics (entanglement), homological algebra, algebraic geometry, machine learning (tensor decompositions). Who/when: Roots in Grassmann (1844) and Ricci–Levi-Civita (1900); abstracted by Whitney, Bourbaki, and category theory in the 20th century.

💡 The Core Insight

Bilinearity is not preservation of structure under a single operation — a bilinear map fails to be a homomorphism of vector spaces because $B(v_1+v_2, w) = B(v_1,w)+B(v_2,w)$ but $B$ is not additive jointly. The tensor product is the precise device that records exactly the relations forced by bilinearity and no more. It is the "freest" target: you build the largest space in which $v \otimes w$ obeys only the bilinear identities, then every other bilinear map is a quotient of this universal one. The slogan: tensor products turn bilinear maps into linear maps.

DefinitionMultilinear Map

Let $V_1, \dots, V_k, U$ be vector spaces over a field $k$ . A map $f : V_1 \times \cdots \times V_k \to U$ is multilinear ( $k$ -linear) if it is linear in each argument separately: for each $i$ , with all other arguments fixed,

1

\quad f(v_1,\dots, \alpha v_i + \beta v_i', \dots, v_k) = \alpha\, f(\dots,v_i,\dots) + \beta\, f(\dots,v_i',\dots).

Derivation hint: this is just $k$ copies of the single-variable linearity condition, one per slot, with all other slots frozen. For $k=2$ this is bilinear. The set of all $k$ -linear maps $V_1\times\cdots\times V_k \to U$ is a vector space, denoted $\mathrm{Mult}(V_1,\dots,V_k; U)$ or $L^k(V_1,\dots,V_k;U)$ .

DefinitionTensor Product via Universal Property

A tensor product of $V$ and $W$ is a pair $(T, \otimes)$ consisting of a vector space $T$ and a bilinear map $\otimes : V \times W \to T$ , written $(v,w) \mapsto v\otimes w$ , satisfying the universal property: for every vector space $U$ and every bilinear map $B : V\times W \to U$ , there exists a unique linear map $\tilde B : T \to U$ with

2

\quad B(v,w) = \tilde B(v \otimes w) \quad\text{for all } v\in V,\ w\in W.

Derivation hint: this is the factorization $B = \tilde B \circ \otimes$ ; uniqueness comes because simple tensors $v\otimes w$ span $T$ , so $\tilde B$ is determined on a spanning set. We write $T = V \otimes_k W$ . The defining bilinear identities are

3

\quad (v+v')\otimes w = v\otimes w + v'\otimes w, \quad v\otimes(w+w') = v\otimes w + v\otimes w', \quad (\alpha v)\otimes w = \alpha(v\otimes w) = v\otimes(\alpha w).

Derivation hint: these follow from requiring $\otimes$ itself to be bilinear; they are the only relations imposed in the free construction below.

DefinitionExplicit Construction

Let $F(V\times W)$ be the free vector space on the set $V\times W$ (basis = all pairs $(v,w)$ ). Let $R \subseteq F(V\times W)$ be the subspace spanned by all elements

4

\quad (v+v',w)-(v,w)-(v',w),\quad (v,w+w')-(v,w)-(v,w'),\quad (\alpha v,w)-\alpha(v,w),\quad (v,\alpha w)-\alpha(v,w).

Derivation hint: each generator of $R$ is "the failure of bilinearity"; quotienting kills exactly those failures. Then define

5

\quad V\otimes_k W := F(V\times W)/R, \qquad v\otimes w := \overline{(v,w)} \ (\text{coset of }(v,w)).

Derivation hint: the quotient map sends the formal pair to its class; the relations $R$ are precisely those of equation [3], so $\otimes$ is bilinear and universal by the universal property of free objects and quotients.

TheoremExistence, Uniqueness, and Dimension

The tensor product exists and is unique up to unique isomorphism: if $(T,\otimes)$ and $(T',\otimes')$ both satisfy the universal property, there is a unique linear isomorphism $\phi : T \to T'$ with $\phi(v\otimes w)=v\otimes' w$ . Moreover, if $\{e_i\}_{i\in I}$ is a basis of $V$ and $\{f_j\}_{j\in J}$ is a basis of $W$ , then $\{e_i \otimes f_j\}_{(i,j)\in I\times J}$ is a basis of $V\otimes_k W$ . Hence in finite dimensions

6

\quad \dim(V\otimes_k W) = (\dim V)(\dim W).

Derivation hint: the basis claim follows from the construction in [5] plus the universal property; dimension is the cardinality of the product index set $I\times J$ . There is a canonical isomorphism

7

\quad \mathrm{Mult}(V,W;U) \cong L(V\otimes_k W,\, U),

Derivation hint: send a bilinear $B$ to its unique factorization $\tilde B$ ; this map is linear and invertible by the universal property — it is the categorical meaning of "tensor product is left adjoint to Hom".

ProofUniqueness up to unique isomorphism

Step 1. Suppose $(T,\otimes)$ and $(T',\otimes')$ both satisfy the universal property. Since $\otimes': V\times W \to T'$ is bilinear, by universality of $(T,\otimes)$ there is a unique linear $\phi : T \to T'$ with $\phi\circ\otimes = \otimes'$ , i.e. $\phi(v\otimes w)=v\otimes'w$ .

Step 2. Symmetrically, since $\otimes : V\times W \to T$ is bilinear, universality of $(T',\otimes')$ yields a unique linear $\psi : T'\to T$ with $\psi(v\otimes'w)=v\otimes w$ .

Step 3. Consider $\psi\circ\phi : T\to T$ . It satisfies $(\psi\circ\phi)(v\otimes w)=v\otimes w$ , so $\psi\circ\phi$ and $\mathrm{id}_T$ are two linear maps factoring the bilinear map $\otimes$ through $T$ . By the uniqueness clause of the universal property, $\psi\circ\phi = \mathrm{id}_T$ . Symmetrically $\phi\circ\psi=\mathrm{id}_{T'}$ .

Step 4. Hence $\phi$ is the unique isomorphism intertwining the universal maps. $\blacksquare$

∎

ProofThe

e_i\otimes f_j

form a basis (finite-dim case)

Step 1 (Spanning). Any $v=\sum_i a_i e_i$ , $w=\sum_j b_j f_j$ give, by bilinearity [3], $v\otimes w = \sum_{i,j} a_i b_j\, (e_i\otimes f_j)$ . Since simple tensors span $V\otimes W$ (true in the construction [5]), the $e_i\otimes f_j$ span.

Step 2 (Independence). For each pair $(p,q)$ define the bilinear map $B_{pq}:V\times W\to k$ by $B_{pq}(v,w)=a_p b_q$ (coordinate extraction). By universality there is a linear $\tilde B_{pq}: V\otimes W\to k$ with $\tilde B_{pq}(e_i\otimes f_j)=\delta_{ip}\delta_{jq}$ .

Step 3. Suppose $\sum_{i,j} c_{ij}\, e_i\otimes f_j = 0$ . Apply $\tilde B_{pq}$ : $\sum_{i,j} c_{ij}\delta_{ip}\delta_{jq}=c_{pq}=0$ for all $(p,q)$ . Hence linear independence. $\blacksquare$

∎

mathematician · Hermann Grassmann | 1809–1877

Grassmann's Ausdehnungslehre (Theory of Extension, 1844, revised 1862) introduced the exterior (Grassmann) product and the conceptual seed of multilinear algebra decades before the tensor formalism crystallized. Working in near-total obscurity as a Gymnasium teacher in Stettin, he developed an algebra of "extensive quantities" — what we now recognize as elements of exterior and tensor algebras — including the alternating product $v\wedge w$ . His work was so far ahead of its notation that few contemporaries understood it; only after Ricci-Curbastro and Levi-Civita's "absolute differential calculus" (tensor analysis, c. 1900) and its dramatic use by Einstein in general relativity did the full architecture of multilinear algebra gain recognition. The modern universal-property definition is due to the categorical reformulation by Bourbaki and others in the mid-20th century.

🧠 Intuition

Think of $V\otimes W$ as the place where you can multiply vectors from different spaces without choosing coordinates. You cannot "multiply" a velocity vector by a force vector inside $\mathbb{R}^3$ — the product $v\otimes w$ lives in a larger $9$ -dimensional space, and the entries $v_i w_j$ are exactly the components. A cleaner mental model: $V\otimes W \cong$ matrices when $V=k^m$ , $W=k^n$ . A simple tensor $v\otimes w$ is the rank-one matrix $vw^\top$ (outer product). A general element is any matrix, written as a sum of rank-one pieces. The minimal number of summands is the matrix rank — and this is the prototype for the deep notion of tensor rank in higher orders. Bilinearity of $\otimes$ corresponds to the fact that $(v_1+v_2)w^\top = v_1 w^\top + v_2 w^\top$ . The universal property is the abstract way of saying: to define a linear map out of $V\otimes W$ , it suffices and is necessary to specify a bilinear map on $V\times W$ . You never need to check what happens on general sums — only on the building blocks $v\otimes w$ , and only that the bilinear relations are respected.

Figure

Universal property of the tensor product: every bilinear

B

factors uniquely through

\otimes

via a linear

\tilde B

.

ExampleOuter product as a simple tensor in

\mathbb{R}^2\otimes\mathbb{R}^3

Given: $v = (1,2)^\top \in \mathbb{R}^2$ with basis $\{e_1,e_2\}$ , and $w=(3,0,-1)^\top \in \mathbb{R}^3$ with basis $\{f_1,f_2,f_3\}$ . Compute $v\otimes w$ in the basis $\{e_i\otimes f_j\}$ and as a matrix.

Step 1. Expand by bilinearity [3]:

v\otimes w = (e_1 + 2e_2)\otimes(3f_1 + 0\,f_2 - f_3).

Step 2. Distribute: coefficients are $c_{ij}=v_i w_j$ .

$c_{11}=1\cdot3=3$ , $c_{12}=1\cdot0=0$ , $c_{13}=1\cdot(-1)=-1$
$c_{21}=2\cdot3=6$ , $c_{22}=0$ , $c_{23}=2\cdot(-1)=-2$

Step 3. As the outer product matrix $vw^\top$ (rows indexed by $i$ , columns by $j$ ):

\boxed{\,v\otimes w = \begin{pmatrix} 3 & 0 & -1 \\ 6 & 0 & -2 \end{pmatrix} = 3\,e_1{\otimes}f_1 - e_1{\otimes}f_3 + 6\,e_2{\otimes}f_1 - 2\,e_2{\otimes}f_3\,}

This matrix has rank 1, confirming $v\otimes w$ is a simple tensor (dimensionless coefficients).

ExampleA tensor that is NOT simple (tensor rank 2)

Given: $t = e_1\otimes f_1 + e_2\otimes f_2 \in \mathbb{R}^2 \otimes \mathbb{R}^2$ . Decide whether $t = v\otimes w$ for some $v,w$ .

Step 1. As a matrix, $t = \begin{pmatrix}1&0\\0&1\end{pmatrix}$ (the identity).

Step 2. A simple tensor corresponds to a rank-1 matrix $vw^\top$ , whose determinant is $0$ (any $2\times2$ outer product has linearly dependent columns).

Step 3. But $\det\begin{pmatrix}1&0\\0&1\end{pmatrix}=1\neq 0$ , so $t$ has matrix rank $2$ .

Step 4. Therefore $t$ cannot be written as a single $v\otimes w$ .

\boxed{\,t = e_1\otimes f_1 + e_2\otimes f_2 \text{ has tensor rank } 2; \text{ it is NOT a simple tensor.}\,}

This $t$ is exactly the maximally entangled Bell state $\tfrac{1}{\sqrt2}(|00\rangle+|11\rangle)$ up to normalization — the prototype of quantum entanglement.

⚠️ Common mistake

Pitfall 1: "Every element of $V\otimes W$ is $v\otimes w$ ." False. Simple tensors only span $V\otimes W$ ; the generic element is a sum, and (Example 2) need not collapse to a single term. The minimal number of terms is the tensor rank.

Pitfall 2: " $\dim(V\otimes W) = \dim V + \dim W$ ." That is the direct sum $V\oplus W$ . For tensor product the dimension multiplies: $\dim(V\otimes W)=\dim V\cdot\dim W$ (equation [6]).

Pitfall 3: " $v\otimes w = w\otimes v$ ." Only after choosing the swap isomorphism $V\otimes W \cong W\otimes V$ ; these are different spaces and the elements are not literally equal. The braiding $v\otimes w \mapsto w\otimes v$ is canonical but nontrivial (it is the source of symmetric vs. exterior algebra).

Pitfall 4: Forgetting the field. $V\otimes_k W$ depends on $k$ . For example $\mathbb{C}\otimes_{\mathbb{R}}\mathbb{C} \cong \mathbb{C}\oplus\mathbb{C}$ (real dimension 4), whereas $\mathbb{C}\otimes_{\mathbb{C}}\mathbb{C}\cong\mathbb{C}$ (dimension 1 over $\mathbb{C}$ ).

CounterexampleTensor product is not exact on the left over

\mathbb{Z}

Over a non-field ring the tensor product can fail to be exact. Consider $\mathbb{Z}$ -modules. The injection $\mathbb{Z}\xrightarrow{\,\times 2\,}\mathbb{Z}$ is injective, but tensoring with $\mathbb{Z}/2\mathbb{Z}$ gives

\mathbb{Z}/2 \xrightarrow{\,\times 2\,} \mathbb{Z}/2,

which is the zero map, hence not injective. So $-\otimes \mathbb{Z}/2$ is only right exact, not exact; its failure is measured by $\mathrm{Tor}_1$ . Over a field this never happens (all modules are free/flat), which is exactly why the clean dimension count [6] holds for vector spaces but not for general modules.

connection

The tensor product is the hub of multilinear algebra and threads into many areas:

Exterior Algebra: $\Lambda^k V$ is the quotient of $V^{\otimes k}$ by the subspace generated by $v_1\otimes\cdots\otimes v_k$ with a repeated factor; it linearizes alternating multilinear maps (determinants, differential forms).
Symmetric Algebra: $\mathrm{Sym}^k V = V^{\otimes k}/(\text{transposition relations})$ linearizes symmetric multilinear maps (polynomials).
Dual Space and Bilinear Forms: $V^*\otimes W \cong \mathrm{Hom}(V,W)$ in finite dimensions, and $V^*\otimes V^* \cong$ bilinear forms on $V$ .
Category Theory and Adjoint Functors: equation [7] expresses the adjunction $-\otimes W \dashv \mathrm{Hom}(W,-)$ , making $(\mathrm{Vect}_k,\otimes,k)$ a symmetric monoidal category.
Tensor Fields on Manifolds: pointwise tensor products of tangent and cotangent spaces give the global tensor bundle in differential geometry.

🔬 Where this appears

⏳ History

- 1844 — Hermann Grassmann publishes Die lineale Ausdehnungslehre, introducing exterior products and the algebra of extensive quantities; largely ignored.

- 1862 — Grassmann's revised Ausdehnungslehre, with a cleaner exposition of multilinear products.

- 1884–1885 — J. Willard Gibbs develops dyadics (essentially rank-2 tensors as

\sum v_i\otimes w_i

) in his lectures on vector analysis.

- 1890 — Woldemar Voigt coins the word tensor in the context of elasticity (the stress tensor).

- 1900 — Gregorio Ricci-Curbastro and Tullio Levi-Civita publish Méthodes de calcul différentiel absolu, the systematic tensor calculus.

- 1915–1916 — Einstein's general relativity makes tensor analysis indispensable to physics, popularizing index notation.

- 1938 — Hassler Whitney gives a rigorous abstract construction of the tensor product of abelian groups/modules ("Tensor products of Abelian groups").

- 1940s–1950s — Bourbaki (Algèbre) formalizes the universal-property definition for modules over a ring, the modern standard.

- 1958–1965 — Categorical viewpoint (Eilenberg, Mac Lane, Kelly): tensor products characterized via adjunction and monoidal categories.

open-problem

Tensor rank and complexity. While matrix rank (order-2 tensors) is computable by Gaussian elimination, for order $\geq 3$ the situation is wild:

Computing tensor rank is NP-hard (Håstad, 1990) over $\mathbb{Q}$ and $\mathbb{F}_q$ .
Rank depends on the field: a real tensor can have higher rank over $\mathbb{R}$ than over $\mathbb{C}$ .
Border rank vs. rank: a tensor can be a limit of rank- $r$ tensors without itself having rank $r$ (the set of bounded-rank tensors is not closed), which underlies fast matrix multiplication.
Matrix multiplication exponent $\omega$ : the bilinear map $M_n\times M_n\to M_n$ is a tensor whose (border) rank growth determines $\omega$ ; whether $\omega = 2$ is a major open problem. Strassen's 1969 algorithm exploits that the $2\times2$ multiplication tensor has rank $\leq 7 < 8$ .
Maximal and generic ranks of tensor spaces are unknown in general dimensions; even the maximal rank of $\mathbb{R}^3\otimes\mathbb{R}^3\otimes\mathbb{R}^3$ involves subtle algebraic geometry (secant varieties of Segre embeddings).

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also