Matrix Multiplication and Dot Products

⚡ Key Takeaways

Matrix multiplication combines two matrices by taking dot products of rows with columns. For matrices $A$ (size $m\times n$ ) and $B$ (size $n\times p$ ), the product $AB$ is an $m\times p$ matrix whose $(i,j)$ entry is the dot product of row $i$ of $A$ with column $j$ of $B$ . Where it applies: linear systems, geometric transformations, computer graphics, statistics, machine learning. Who/when: formalized by Arthur Cayley (1858), building on work by Jacques Binet and Gotthold Eisenstein.

💡 The Core Insight

A matrix is a machine that transforms vectors. Multiplying matrices means chaining machines together. Each entry of the product is a dot product — a measure of how much one row "agrees" with one column. The dot product is the atom; matrix multiplication is the molecule built from many such atoms arranged in rows and columns.

DefinitionDot Product

Given two vectors of the same length $n$ ,

1

\quad \mathbf{a}=(a_1,a_2,\dots,a_n),\qquad \mathbf{b}=(b_1,b_2,\dots,b_n),

Derivation hint: A vector is just an ordered list of numbers; both must have the same number of components for the dot product to be defined. their dot product (or scalar product) is the single number

2

\quad \mathbf{a}\cdot\mathbf{b}=\sum_{k=1}^{n} a_k b_k = a_1b_1 + a_2b_2 + \cdots + a_nb_n.

Derivation hint: Multiply each pair of matching entries, then add all the products together — the result is a scalar, not a vector. A second, geometric formula relates the dot product to lengths and the angle $\theta$ between the vectors:

3

\quad \mathbf{a}\cdot\mathbf{b}=\|\mathbf{a}\|\,\|\mathbf{b}\|\cos\theta,

Derivation hint: Apply the Law of Cosines to the triangle formed by $\mathbf{a}$ , $\mathbf{b}$ , and $\mathbf{a}-\mathbf{b}$ , then expand using equation [2]. where $\|\mathbf{a}\|=\sqrt{\mathbf{a}\cdot\mathbf{a}}$ is the length (magnitude) of $\mathbf{a}$ .

DefinitionMatrix Multiplication

Let $A$ be an $m\times n$ matrix with entries $a_{ik}$ and $B$ an $n\times p$ matrix with entries $b_{kj}$ . The product $C=AB$ is the $m\times p$ matrix whose entries are

4

\quad c_{ij}=\sum_{k=1}^{n} a_{ik}\,b_{kj},\qquad 1\le i\le m,\ 1\le j\le p.

Derivation hint: The entry in row $i$ , column $j$ of the product is the dot product of row $i$ of $A$ with column $j$ of $B$ — sum over the shared index $k$ .

Symbols:

$m$ = number of rows of $A$ (and of $C$ ).
$n$ = number of columns of $A$ = number of rows of $B$ (the inner dimension, which must match).
$p$ = number of columns of $B$ (and of $C$ ).
$a_{ik}$ = entry of $A$ in row $i$ , column $k$ .
$b_{kj}$ = entry of $B$ in row $k$ , column $j$ .
$c_{ij}$ = entry of the product in row $i$ , column $j$ .

TheoremCompatibility and Size Rule

The product $AB$ is defined if and only if the number of columns of $A$ equals the number of rows of $B$ . In that case:

5

\quad (m\times n)\cdot(n\times p) \;\longrightarrow\; (m\times p).

Derivation hint: The inner dimensions ( $n$ and $n$ ) must cancel; the outer dimensions ( $m$ and $p$ ) survive as the result's size.

Figure

Computing one entry of a matrix product as a row–column dot product

ProofWhy Matrix Multiplication Composes Transformations

Matrix multiplication is defined the way it is so that applying transformation $B$ , then transformation $A$ , equals applying the single matrix $AB$ .

Step 1. Treat a matrix $B$ ( $n\times p$ ) as a function that sends a vector $\mathbf{x}$ (length $p$ ) to a new vector $\mathbf{y}=B\mathbf{x}$ (length $n$ ), where each component is a dot product:

6

\quad y_k=\sum_{j=1}^{p} b_{kj}\,x_j.

Derivation hint: This is just equation [4] specialized to the case where the second factor is a single column ( $p\to 1$ ).

Step 2. Now apply matrix $A$ ( $m\times n$ ) to $\mathbf{y}$ , producing $\mathbf{z}=A\mathbf{y}$ :

7

\quad z_i=\sum_{k=1}^{n} a_{ik}\,y_k.

Derivation hint: Same dot-product rule, now applied to the intermediate vector $\mathbf{y}$ .

Step 3. Substitute equation [6] into equation [7]:

8

\quad z_i=\sum_{k=1}^{n} a_{ik}\left(\sum_{j=1}^{p} b_{kj}\,x_j\right)=\sum_{j=1}^{p}\left(\sum_{k=1}^{n} a_{ik}\,b_{kj}\right)x_j.

Derivation hint: Swap the order of summation (a finite sum lets you do this freely), then group the inner sum.

Step 4. The bracketed quantity is exactly $c_{ij}=\sum_k a_{ik}b_{kj}$ from equation [4]. Therefore $\mathbf{z}=(AB)\mathbf{x}$ , proving that the matrix product encodes the composition of the two transformations. $\blacksquare$

∎

mathematician · Arthur Cayley | 1821–1895

A British mathematician and trained lawyer, Cayley published A Memoir on the Theory of Matrices (1858), giving the first abstract treatment of matrices as objects in their own right. He defined matrix addition, multiplication, and the identity, and noted that matrix multiplication is non-commutative — a radical departure from ordinary arithmetic. Cayley also stated (and verified in small cases) what is now the Cayley–Hamilton theorem. Remarkably prolific, he produced nearly 1,000 papers, many while practicing law for 14 years before becoming Sadleirian Professor at Cambridge.

🧠 Intuition

Think of the dot product as a similarity score: when two vectors point in the same direction, $\cos\theta=1$ and the dot product is large and positive; when perpendicular, $\cos\theta=0$ and the dot product is exactly zero; when opposite, it is negative. This is why orthogonality (perpendicularity) and "zero dot product" mean the same thing. For matrix multiplication, picture $A$ as a stack of row-detectors and $B$ as a set of column-inputs. Entry $c_{ij}$ measures how strongly row $i$ responds to column $j$ . Another lens: each column of $AB$ is a combination of the columns of $A$ , weighted by a column of $B$ . The matrix $B$ tells you how to mix the columns of $A$ .

TD

The Dot Product in Physics

Physicists meet the dot product as work: $W=\mathbf{F}\cdot\mathbf{d}$ , the force times the distance in the direction of motion. Only the component of force aligned with displacement does work, which is exactly what $\|\mathbf{F}\|\|\mathbf{d}\|\cos\theta$ captures. Push perpendicular to motion and you do zero work — the dot product is zero. This single idea reappears as power ( $P=\mathbf{F}\cdot\mathbf{v}$ ), magnetic flux, and the projection of one quantity onto another throughout mechanics and electromagnetism.

ExampleDot Product with a Geometric Check

Given: $\mathbf{a}=(3,4)$ and $\mathbf{b}=(4,-3)$ .

Step 1 — Algebraic formula (eq. [2]):

\mathbf{a}\cdot\mathbf{b}=(3)(4)+(4)(-3)=12-12=0.

Step 2 — Interpret. A zero dot product means the vectors are perpendicular.

Step 3 — Verify with the geometric formula (eq. [3]). Both lengths are $\sqrt{3^2+4^2}=5$ , so

0=\mathbf{a}\cdot\mathbf{b}=5\cdot 5\cdot\cos\theta \;\Rightarrow\; \cos\theta=0\;\Rightarrow\;\theta=90^\circ.

\boxed{\mathbf{a}\cdot\mathbf{b}=0,\quad \theta=90^\circ \text{ (perpendicular)}}

ExampleA 2×2 Matrix Product

Given:

A=\begin{pmatrix} 1 & 2\\ 3 & 4\end{pmatrix},\qquad B=\begin{pmatrix} 5 & 6\\ 7 & 8\end{pmatrix}.

Step 1 — Check sizes. $A$ is $2\times 2$ , $B$ is $2\times 2$ ; inner dimensions match, so $AB$ is $2\times 2$ .

Step 2 — Compute each entry as a dot product (eq. [4]):

$c_{11}=(1)(5)+(2)(7)=5+14=19$
$c_{12}=(1)(6)+(2)(8)=6+16=22$
$c_{21}=(3)(5)+(4)(7)=15+28=43$
$c_{22}=(3)(6)+(4)(8)=18+32=50$

\boxed{AB=\begin{pmatrix} 19 & 22\\ 43 & 50\end{pmatrix}}

Step 3 — Confirm non-commutativity. Computing $BA$ instead gives $\begin{pmatrix}23 & 34\\ 31 & 46\end{pmatrix}\ne AB$ . The order genuinely matters.

⚠️ Common mistake

Matrix multiplication is NOT element-wise. Beginners often try to multiply matching entries: $(AB)_{ij}\ne a_{ij}b_{ij}$ . That element-wise operation (the Hadamard product) is a different, less common thing. The real product uses rows times columns and a sum, per equation [4].

Order matters. In general $AB\ne BA$ . Sometimes only one of the two products even exists, because dimensions must line up. Never silently swap factors.

The identity is not "all ones." The multiplicative identity is the matrix $I$ with 1's on the diagonal and 0's elsewhere, satisfying $AI=IA=A$ . A matrix of all ones does something entirely different.

Counterexample

Claim (false): "If $AB=0$ (the zero matrix), then $A=0$ or $B=0$ ." With ordinary numbers this is true, but matrices break it. Take

A=\begin{pmatrix} 1 & 0\\ 0 & 0\end{pmatrix},\qquad B=\begin{pmatrix} 0 & 0\\ 0 & 1\end{pmatrix}.

Then

AB=\begin{pmatrix} 0 & 0\\ 0 & 0\end{pmatrix},

yet neither $A$ nor $B$ is the zero matrix. Matrices can have zero divisors, so you cannot "cancel" factors the way you do with real numbers.

connection

The dot product is the seed of Vector Spaces: lengths, angles, and orthogonality are all defined through it, generalizing to the abstract inner product. Matrix multiplication realizes Linear Transformations concretely — every linear map between finite spaces is "a matrix," and composing maps is multiplying matrices. Solving Systems of Linear Equations is written compactly as $A\mathbf{x}=\mathbf{b}$ , and the search for $\mathbf{x}$ leads to the Matrix Inverse and the Determinant. The geometric side connects to Trigonometry through the $\cos\theta$ formula in equation [3].

🔬 Where this appears

⏳ History

- 1748 — Gabriel Cramer publishes Cramer's Rule using determinant-like arrays to solve linear systems.

- 1812 — Jacques Binet and Augustin-Louis Cauchy independently study products of rectangular arrays (the Binet–Cauchy identity).

- 1844 — Hermann Grassmann develops a general theory of "extensive quantities," anticipating vectors and bilinear products.

- 1850 — James Joseph Sylvester coins the word "matrix" (Latin for womb) for a rectangular array from which determinants are born.

- 1858 — Arthur Cayley's Memoir on the Theory of Matrices gives the first systematic algebra of matrices, defining multiplication and noting non-commutativity.

- 1881 — J. Willard Gibbs and Oliver Heaviside formalize vector notation, popularizing the dot product in physics and engineering.

- 1888 — Giuseppe Peano gives an axiomatic definition of a vector space, setting the abstract stage for these operations.

- 1969 — Volker Strassen shocks the field by showing matrix multiplication can be done faster than the obvious

n^3

operations, opening the study of its true complexity.

open-problem

How fast can two matrices be multiplied? The schoolbook method for $n\times n$ matrices costs about $n^3$ multiplications. Strassen (1969) lowered the exponent to $\approx 2.807$ , and successive breakthroughs have pushed it down — the current record exponent $\omega$ sits around $2.371$ (as of the early 2020s). The deep open question is whether $\omega=2$ , meaning matrices could be multiplied in essentially the time it takes just to read them. No one knows; the best lower bound is only slightly above $2$ , leaving a wide, famous gap that drives ongoing research in theoretical computer science.

Matrix Multiplication and Dot Products

Formal Statement

Why It Holds / Derivation

Building Intuition

Worked Examples

Common Misconceptions

Connections

Real-World Applications

Historical Development

Subtleties & Limits

See Also