Rank, Null Space, Column Space, and Conditioning

Let

$$A \in \mathbb{R}^{m \times n}$$

and define the linear map

$$T(x)=Ax,\qquad x\in\mathbb{R}^n.$$

Write the columns of $A$ as

$$A=[a_1,\ldots,a_n],\qquad a_i\in\mathbb{R}^m.$$

The column space of $A$ is the set of all outputs the map can reach:

$$\operatorname{Col}(A) =\operatorname{Im}(T) =\{Ax:x\in\mathbb{R}^n\} =\operatorname{span}(a_1,\ldots,a_n).$$

The rank is the dimension of that reachable output space:

$$\operatorname{rank}(A)=\dim\operatorname{Col}(A).$$

The null space is the set of input directions that vanish:

$$\operatorname{Null}(A) =\ker(T) =\{x\in\mathbb{R}^n:Ax=0\}.$$

Its dimension is the nullity:

$$\operatorname{nullity}(A)=\dim\operatorname{Null}(A).$$

Rank-nullity says that every input dimension either survives into the image or is lost in the kernel:

$$\operatorname{rank}(A)+\operatorname{nullity}(A)=n.$$

For a square matrix $A\in\mathbb{R}^{n\times n}$:

• If $\operatorname{rank}(A)=n$, the map has no nonzero null-space direction and is invertible.
• If $\operatorname{rank}(A)<n$, some nonzero direction disappears, so $Ax=b$ cannot have a unique solution for every $b$.

For rectangular matrices, "full rank" means

$$\operatorname{rank}(A)=\min(m,n).$$

That can still mean different things. A tall full-column-rank matrix has no nonzero null-space direction, but it cannot reach every vector in $\mathbb{R}^m$. A wide full-row-rank matrix can reach every vector in $\mathbb{R}^m$, but it has non-unique inputs because its null space is nontrivial.

Conditioning adds a numerical layer. Let the singular values of $A$ be

$$\sigma_1\ge \sigma_2\ge \cdots \ge \sigma_r>0,$$

where $r=\operatorname{rank}(A)$. Singular values tell us how much $A$ stretches orthogonal input directions. A zero singular value is an exactly vanished direction. A tiny singular value is an almost vanished direction.

For a full-rank square matrix, the 2-norm condition number is

$$\kappa_2(A)=\frac{\sigma_{\max}(A)}{\sigma_{\min}(A)}.$$

If $\sigma_{\min}(A)=0$, the matrix is singular and this ratio is infinite. If $\sigma_{\min}(A)$ is merely tiny, the matrix may be exactly invertible but practically fragile.

That fragility appears when solving

$$Ax=b.$$

A small perturbation $\Delta b$ can produce a much larger change $\Delta x$ in the solution when $A$ has a tiny singular direction. In learning language: the predictions may barely change, but the coefficients can swing wildly. Regularization and SVD-based solvers handle this by damping or separating those fragile directions instead of pretending every direction is equally trustworthy.

Two caveats matter:

1. Exact rank is a mathematical property. Numerical rank depends on a tolerance, the data scale, and floating-point precision.
2. A large condition number is a sensitivity warning. It does not say every computation will fail; it says there are directions where small errors can be amplified.