Linear Algebra Done Right · Problem 7F.22

The key insight is that the operator norm of $T$ restricted to a $k$-dimensional subspace is minimized when that subspace is "aligned" with the smallest singular directions. The singular values order the stretching factors along orthonormal directions: $s_1$ stretches the most, $s_n$ the least. When you restrict $T$ to a $k$-dimensional subspace, you're choosing $k$ directions to keep; the worst-case (largest) stretching factor on that subspace is minimized by choosing the $k$ directions corresponding to the $k$ smallest singular values. This minimum stretching factor turns out to be exactly $s_{n-k+1}$, the largest of those $k$ smallest singular values.

The key insight is that the operator norm of $T$ restricted to a $k$-dimensional subspace is minimized when that subspace is "aligned" with the smallest singular directions. The singular values order the stretching factors along orthonormal directions: $s_1$ stretches the most, $s_n$ the least. When you restrict $T$ to a $k$-dimensional subspace, you're choosing $k$ directions to keep; the worst-case (largest) stretching factor on that subspace is minimized by choosing the $k$ directions corresponding to the $k$ smallest singular values. This minimum stretching factor turns out to be exactly $s_{n-k+1}$, the largest of those $k$ smallest singular values.