Linear Independence, Spanning Sets, and Bases

⚡ Key Takeaways

Linear independence, spanning sets, and bases are the three interlocking concepts that capture the "size" and "coordinate structure" of a vector space. A set is linearly independent if no vector is a redundant combination of the others; it spans if every vector in the space can be built from it; a basis is a set that does both — a minimal spanning set / maximal independent set. Every vector space has a basis, and all bases have the same cardinality (the dimension). These ideas are foundational throughout linear algebra and were crystallized in the late 19th and early 20th centuries (Grassmann, Peano, Steinitz).

💡 The Core Insight

A basis is the sweet spot between two failure modes. Too few vectors and you can't reach everything (you fail to span). Too many vectors and you have redundancy (you fail to be independent). A basis is exactly the threshold where you can reach every vector in exactly one way. The "exactly one way" — uniqueness of representation — is the property that makes coordinates, matrices, and the entire computational apparatus of linear algebra possible.

DefinitionLinear Combination

Let $V$ be a vector space over a field $F$ and $v_1,\dots,v_n \in V$ . A linear combination of these vectors is any vector of the form

1

\quad w = c_1 v_1 + c_2 v_2 + \cdots + c_n v_n = \sum_{i=1}^{n} c_i v_i, \qquad c_i \in F.

Derivation hint: This is the definition; it uses only the two vector-space operations — scalar multiplication $c_i v_i$ and vector addition — applied finitely many times. Here $v_i \in V$ are vectors, $c_i \in F$ are scalars (coefficients), and $w \in V$ is the resulting vector. The scalars $c_i$ are called the weights of the combination.

DefinitionLinear Independence

A set $\{v_1,\dots,v_n\} \subseteq V$ is linearly independent if the only choice of scalars making the combination zero is the trivial one:

2

\quad c_1 v_1 + \cdots + c_n v_n = 0 \;\;\Longrightarrow\;\; c_1 = c_2 = \cdots = c_n = 0.

Derivation hint: $0$ on the left is the zero vector of $V$ ; the implication must hold for ALL scalar tuples, so the only relation among the $v_i$ is the trivial one. If some nontrivial choice (not all $c_i$ zero) gives $0$ , the set is linearly dependent. An infinite set is independent iff every finite subset is independent.

DefinitionSpan

The span of a set $S = \{v_1,\dots,v_n\}$ is the set of all linear combinations:

3

\quad \operatorname{span}(S) = \left\{ \sum_{i=1}^{n} c_i v_i \;\middle|\; c_i \in F \right\}.

Derivation hint: Take the collection of all outputs of equation [1] as the $c_i$ range over $F$ ; closure under $+$ and scalar multiplication makes this a subspace. $\operatorname{span}(S)$ is the smallest subspace of $V$ containing $S$ . If $\operatorname{span}(S) = V$ , we say $S$ spans $V$ or is a spanning (generating) set.

DefinitionBasis

A basis of $V$ is a set $B \subseteq V$ that is both linearly independent and spans $V$ . Equivalently, $B$ is a basis iff every $v \in V$ has a unique representation

4

\quad v = \sum_{i=1}^{n} c_i v_i, \qquad c_i \in F,

Derivation hint: Spanning gives existence of the $c_i$ ; independence gives uniqueness — if two representations existed, their difference would be a nontrivial relation $\sum (c_i - c_i')v_i = 0$ . The unique scalars $(c_1,\dots,c_n)$ are the coordinates of $v$ relative to $B$ .

TheoremInvariance of Dimension (Cardinality of Bases)

If $V$ has a finite basis, then every basis of $V$ has the same number of elements. This common number is the dimension $\dim V$ . More precisely (the Steinitz Exchange Lemma): if $\{u_1,\dots,u_m\}$ is linearly independent and $\{w_1,\dots,w_n\}$ spans $V$ , then $m \le n$ .

TheoremExistence and Characterizations of a Basis

Let $\dim V = n < \infty$ . Then: 1. Every linearly independent set can be extended to a basis. 2. Every spanning set can be reduced to a basis. 3. Any linearly independent set of size $n$ is automatically a basis. 4. Any spanning set of size $n$ is automatically a basis. 5. (Axiom of Choice) Every vector space — even infinite-dimensional — has a basis (a Hamel basis).

ProofSteinitz Exchange Lemma ⟹ Invariance of Dimension

We prove that an independent set is never larger than a spanning set; invariance follows.

Setup. Let $U = \{u_1,\dots,u_m\}$ be linearly independent and $W = \{w_1,\dots,w_n\}$ span $V$ . We show $m \le n$ by swapping the $u$ 's into $W$ one at a time.

Step 1 — Insert $u_1$ . Since $W$ spans, write

5

\quad u_1 = a_1 w_1 + a_2 w_2 + \cdots + a_n w_n.

Derivation hint: spanning guarantees existence of scalars $a_j$ ; this is equation [3] applied to $u_1 \in V = \operatorname{span}(W)$ . Some $a_j \neq 0$ (else $u_1 = 0$ , contradicting independence). Relabel so $a_1 \neq 0$ . Solve for $w_1$ :

6

\quad w_1 = a_1^{-1}\Big( u_1 - \sum_{j=2}^{n} a_j w_j \Big).

Derivation hint: divide [5] by $a_1$ and isolate $w_1$ ; legal because $a_1 \neq 0$ in the field $F$ . So $\{u_1, w_2,\dots,w_n\}$ still spans $V$ (we can recover $w_1$ , hence everything).

Step 2 — Inductive swap. Suppose after $k$ steps the set $\{u_1,\dots,u_k, w_{k+1},\dots,w_n\}$ spans (with $k \le n$ ). Express $u_{k+1}$ in it:

7

\quad u_{k+1} = \sum_{i=1}^{k} b_i u_i + \sum_{j=k+1}^{n} a_j w_j.

Derivation hint: again use spanning of the current set; existence of the coefficients is guaranteed. If all the $a_j$ ( $j>k$ ) were zero, then [7] would express $u_{k+1}$ as a combination of $u_1,\dots,u_k$ — a nontrivial relation contradicting independence of $U$ . So some $a_j \neq 0$ ; in particular there must still be at least one $w$ left, forcing $k+1 \le n$ . Relabel so $a_{k+1}\neq 0$ and swap $w_{k+1}$ out for $u_{k+1}$ . The new set spans.

Step 3 — Conclude. The induction can run until all $m$ of the $u$ 's are inserted, and each step required an available $w$ . Therefore $m \le n$ .

Invariance. If $B_1$ and $B_2$ are both bases, apply the lemma twice: $B_1$ (independent) vs. $B_2$ (spanning) gives $|B_1| \le |B_2|$ ; reversing roles gives $|B_2| \le |B_1|$ . Hence $|B_1| = |B_2|$ . $\qquad\blacksquare$

∎

mathematician · Hermann Grassmann | 1809–1877

A schoolteacher in Stettin, Grassmann developed in his Ausdehnungslehre (Theory of Extension, 1844; revised 1862) what we now recognize as the first abstract theory of vector spaces, independence, and dimension — decades ahead of his time and so cryptically written that it was almost ignored. He introduced the idea that geometric magnitudes ("extensive quantities") could be combined linearly, and he understood that a set of independent vectors determines the dimension of the space they generate. Giuseppe Peano gave the modern axiomatic definition of a vector space in 1888, explicitly citing Grassmann; Ernst Steinitz supplied the rigorous exchange argument for invariance of dimension in 1913. Grassmann never saw his work mainstreamed in his lifetime — he died believing his linguistic studies (he was also a noted Sanskrit scholar) were his lasting legacy.

🧠 Intuition

Think of vectors as directions you're allowed to walk in.

Spanning asks: Can I reach every point in the room using these directions? If I can only walk North–South and East–West, I span the floor but not the ceiling height.
Independence asks: Is any direction wasted? If my allowed moves are "North," "East," and "Northeast," then "Northeast" is redundant — it's already North + East. That redundancy is linear dependence.
A basis is a set of directions that is just enough: you can reach everywhere (span) with no wasted directions (independent). The number of directions you need is the dimension.

The deep payoff is uniqueness: with a basis, every destination has exactly one set of "walking instructions." That's why a basis turns an abstract space into ordinary coordinate tuples — the basis is the dictionary between $V$ and $F^n$ .

Figure

Three vectors in the plane: two form a basis, the third is dependent

ExampleTesting Independence and Finding a Basis in ℝ³

Given. The three vectors

v_1 = (1,2,3),\quad v_2 = (2,4,6),\quad v_3 = (1,0,1) \in \mathbb{R}^3.

Determine whether $\{v_1,v_2,v_3\}$ is independent, and extract a basis for $\operatorname{span}$ .

Step 1 — Set up the dependence equation. We seek $c_1,c_2,c_3$ with $c_1 v_1 + c_2 v_2 + c_3 v_3 = 0$ .

Step 2 — Spot the obvious relation. Note $v_2 = 2 v_1$ exactly. So

2 v_1 - v_2 + 0\cdot v_3 = (2,4,6)-(2,4,6) = (0,0,0).

This is a nontrivial relation ( $c_1=2,\,c_2=-1,\,c_3=0$ ). Therefore the set is linearly dependent.

Step 3 — Discard the redundant vector. Remove $v_2$ . Check whether $\{v_1, v_3\}$ is independent: from $c_1(1,2,3)+c_3(1,0,1)=0$ , the second coordinate gives $2c_1=0\Rightarrow c_1=0$ , then the first gives $c_3=0$ . Independent. ✓

Step 4 — Determine the span's dimension. Since $\{v_1,v_3\}$ is independent and spans the same set as the original three, $\dim \operatorname{span}\{v_1,v_2,v_3\} = 2$ (a plane through the origin in $\mathbb{R}^3$ ).

\boxed{\;\{v_1, v_3\} \text{ is a basis of the span; } \dim = 2\;}

ExampleCoordinates Relative to a Basis of P₂ (polynomials)

Given. In $P_2 = \{$ real polynomials of degree $\le 2\}$ , take the basis

B = \{\,1,\; 1+t,\; 1+t+t^2\,\}.

Find the coordinates of $p(t) = 3 + 2t + 5t^2$ relative to $B$ .

Step 1 — Write the coordinate equation. We want scalars $c_1,c_2,c_3$ such that

c_1(1) + c_2(1+t) + c_3(1+t+t^2) = 3 + 2t + 5t^2.

Step 2 — Collect by powers of $t$ .

\begin{aligned} t^2:&\quad c_3 = 5,\\ t^1:&\quad c_2 + c_3 = 2,\\ t^0:&\quad c_1 + c_2 + c_3 = 3. \end{aligned}

Step 3 — Back-substitute.

$c_3 = 5$ .
$c_2 = 2 - c_3 = 2 - 5 = -3$ .
$c_1 = 3 - c_2 - c_3 = 3 - (-3) - 5 = 1$ .

Step 4 — Verify. $1(1) + (-3)(1+t) + 5(1+t+t^2) = (1-3+5) + (-3+5)t + 5t^2 = 3 + 2t + 5t^2$ . ✓

\boxed{\;[p]_B = (1,\,-3,\,5)\;}

The same polynomial in the standard basis $\{1,t,t^2\}$ has coordinates $(3,2,5)$ — different basis, different coordinates, same vector.

⚠️ Common mistake

Pitfall 1 — "More vectors means more span." Adding vectors can never shrink the span, but once a set spans $V$ , throwing in more vectors gains nothing and only destroys independence. Bigger is not better.

Pitfall 2 — "The zero vector can be in a basis." Never. Any set containing $0$ is automatically dependent: $1\cdot 0 = 0$ is a nontrivial relation.

Pitfall 3 — "Independence is a property of individual vectors." It is a property of the whole set. A single nonzero vector is independent; a vector is never "independent" by itself relative to others without naming the set.

Pitfall 4 — "Spanning sets and independent sets are the same kind of thing." They pull in opposite directions: spanning is about being big enough, independence about being small enough. A basis is where they meet.

Pitfall 5 — Confusing the number of vectors with dimension. A set of $n$ vectors in an $n$ -dimensional space is a basis only if it is independent (equivalently spans). Three vectors in $\mathbb{R}^3$ might span only a plane (Example 1).

Counterexample

Claim (false): "If a set of vectors is pairwise independent — no one is a scalar multiple of another — then the whole set is linearly independent."

Counterexample. In $\mathbb{R}^2$ take

v_1 = (1,0),\quad v_2 = (0,1),\quad v_3 = (1,1).

No vector is a scalar multiple of any other (pairwise "independent" in the naïve sense). Yet

v_1 + v_2 - v_3 = (1,0)+(0,1)-(1,1) = (0,0),

a nontrivial relation. The set is dependent. Linear independence is a statement about all the vectors at once, not about pairs.

connection

Linear independence, span, and bases sit at the hub of linear algebra:

Dimension of a Vector Space is defined as the common cardinality of any basis — the whole machinery above exists to make this well-defined.
Rank of a Matrix equals the dimension of the column space (a span of columns) and the row space; the column rank = row rank theorem is independence/span in disguise.
Coordinate Vectors and Change of Basis uses the uniqueness from equation [4]; switching bases is a linear isomorphism $V \cong F^n$ .
<a class="wiki-link" href="/wiki/ndli-iit-kgp-ma10002-mathematics-ii-linear-transformations">Linear Transformations</a> are completely determined by their action on a basis — define $T$ on basis vectors arbitrarily and it extends uniquely.
<a class="wiki-link" href="/wiki/the-four-fundamental-subspaces">The Four Fundamental Subspaces</a> (column, row, null, left-null spaces) are all spans, and their dimensions are tied by the Rank–Nullity Theorem.
<a class="wiki-link" href="/wiki/inner-product-spaces-and-orthogonality">Inner Product Spaces and Orthogonality</a> refine "basis" to "orthonormal basis," via Gram–Schmidt Orthogonalization.

🔬 Where this appears

⏳ History

- 1844 — Hermann Grassmann publishes Die lineale Ausdehnungslehre, introducing extensive quantities, linear combinations, independence, and dimension — the conceptual birth of vector spaces, largely overlooked.

- 1862 — Grassmann issues a heavily rewritten second edition hoping for recognition; it again finds few readers.

- 1888 — Giuseppe Peano, in Calcolo geometrico, gives the first fully axiomatic definition of a (finite-dimensional) vector space and defines dimension and independence rigorously, crediting Grassmann.

- 1900s — Hilbert and Schmidt develop infinite-dimensional spaces; the notion of basis splinters into algebraic (Hamel) and topological (Schauder) versions.

- 1905 — Georg Hamel constructs a basis of

\mathbb{R}

as a

\mathbb{Q}

-vector space, showing infinite-dimensional bases require the Axiom of Choice.

- 1913 — Ernst Steinitz formalizes the Exchange Lemma, giving the first clean proof that all bases have equal cardinality (invariance of dimension).

- 1918 — Hermann Weyl's Raum, Zeit, Materie popularizes the abstract vector-space viewpoint in physics (relativity).

- 1920s–1930s — Banach, van der Waerden, and Bourbaki standardize the modern axiomatic treatment; bases, spans, and dimension take their current textbook form.

- 1948 — Claude Shannon's information theory and the rise of linear coding theory give independence/span a starring computational role over finite fields.

open-problem

Infinite-dimensional subtleties. In infinite dimensions the clean picture cracks:

A Hamel (algebraic) basis of an infinite-dimensional space like $\mathbb{R}$ over $\mathbb{Q}$ , or $C[0,1]$ , exists only by invoking the Axiom of Choice — it can never be explicitly written down, and such bases are necessarily uncountable for complete normed spaces (Baire category theorem). So "every vector space has a basis" is equivalent to the Axiom of Choice (Blass, 1984).
For functional analysis one instead uses a Schauder basis (allowing infinite sums via a topology), but Per Enflo (1973) constructed a separable Banach space with no Schauder basis — disproving a long-standing conjecture of Banach.
The notion of "dimension" remains invariant for Hamel bases (any two have equal cardinality), but the interplay with topology — when does an algebraic basis respect convergence? — is genuinely delicate.
Open in spirit: explicit, constructive bases for many natural infinite-dimensional spaces simply do not and cannot exist, marking a sharp boundary between the finite-dimensional theory (fully constructive) and the infinite case (entangled with set theory).

Linear Independence, Spanning Sets, and Bases

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also