Vector Spaces

A vector space is a set of objects you can add and scale, where those operations behave consistently.

Introductory · High-school mathematics · about 10 minutes

Intuition

The idea in plain words, before the symbols.

A vector space is a set whose elements can be added and scaled, with both operations obeying a few algebraic rules. We begin with two arrows in the plane and form their linear combinations. The figure checks whether those combinations fill the plane or collapse onto one line; the code checks several of the rules. You will need arithmetic on coordinates. The two operations are:

You can add two things of the same kind.
You can scale a thing up or down by a number.

A first example. Take two arrows in the plane, $u = (2, 1)$ and $v = (-1, 2)$ . Scaling multiplies both numbers and adding works number by number, so $2u = (4, 2)$ and $2u + v = (3, 4)$ . Because $u$ and $v$ point in different directions, every point of the plane is $au + bv$ for some numbers $a$ and $b$ . One number confirms it: $u_1 v_2 - u_2 v_1 = 2 \cdot 2 - 1 \cdot (-1) = 5$ , the signed area of the parallelogram with sides $u$ and $v$ , is not zero.

Now change one thing: take $v = (4, 2)$ . This $v$ is $2u$ , so every combination is $au + bv = (a + 2b)\,u$ , a multiple of $u$ on one line through the origin, and the area $2 \cdot 2 - 1 \cdot 4$ is $0$ . In the figure, Pair B is this collapsed case.

The rest of this page names the rules that make these calculations work for every kind of vector. Already know them? Go straight to the definition.

If those two operations obey a short list of rules (associativity, commutativity, a zero element and a few more), then one body of mathematics applies to every such space: projections, least squares, gradients, eigenvectors, Fourier features, embeddings, and more.

The idea is not "arrows". Arrows in 2D are a good mental model, but the objects can be many other things: polynomials, functions, images, token embeddings, or model parameter updates. Anything you can add and scale by these rules can be handled with the same tools.

Study prompts for this section: Intuition

Study prompts

Each button copies a prompt about this section to your clipboard, for an AI assistant you already use. Nothing is sent by this site.

Mathematics

Definitions, assumptions and the derivation.

Equation 1

Definition (Vector space). A vector space over a field $\mathbb{F}$ (usually $\mathbb{R}$ or $\mathbb{C}$ ) is a set $V$ with:

addition: $+: V \times V \to V$
scalar multiplication: $\cdot: \mathbb{F} \times V \to V$

such that for all $u,v,w \in V$ and $a,b \in \mathbb{F}$ :

$(u+v)+w = u+(v+w)$
$u+v = v+u$
There exists $0 \in V$ with $v+0=v$
For each $v$ there exists $-v$ with $v+(-v)=0$
$a(bv) = (ab)v$
$1\cdot v = v$
$a(u+v)=au+av$
$(a+b)v = av+bv$

Because both operations land back in $V$ , every linear combination stays in the space (this is called closure):

a u + b v \in V \qquad (u,v \in V,\; a,b \in \mathbb{F}).

The set of all linear combinations of $u$ and $v$ is their span, $\operatorname{span}\{u,v\} = \{au + bv : a, b \in \mathbb{F}\}$ . In $\mathbb{R}^2$ , two vectors span the whole plane unless one is a scalar multiple of the other; then their span is a line through the origin, or only the origin if both are zero. The determinant $\det[u\ v] = u_1 v_2 - u_2 v_1$ is the signed area of the parallelogram with sides $u$ and $v$ , and it is zero exactly when the span has collapsed to a line or a point.

In ML, we often work in $\mathbb{R}^d$ , but it's useful to remember that "vector" really means "element of some vector space".

Study prompts for this section: Mathematics

Study prompts

Each button copies a prompt about this section to your clipboard, for an AI assistant you already use. Nothing is sent by this site.

Code

The same calculation as runnable code, in the notation of the derivation.

Code example 1import numpy as np # A tiny sanity-check: R^3 with usual + and scalar * behaves like a vector...python

import numpy as np

# A tiny sanity-check: R^3 with usual + and scalar * behaves like a vector space.

u = np.array([1.0, 2.0, 3.0])
v = np.array([-1.0, 0.5, 4.0])
a, b = 2.0, -0.25

# Distributivity: a(u+v) = au + av
lhs = a * (u + v)
rhs = a * u + a * v
print("distributivity error:", float(np.linalg.norm(lhs - rhs)))

# Compatibility: a(bu) = (ab)u
lhs = a * (b * u)
rhs = (a * b) * u
print("compatibility error:", float(np.linalg.norm(lhs - rhs)))

# Zero + additive inverse
zero = np.zeros_like(u)
print("zero check:", bool(np.allclose(u + zero, u)))
print("inverse check:", bool(np.allclose(u + (-u), zero)))

Study prompts for this section: Code

Study prompts

Each button copies a prompt about this section to your clipboard, for an AI assistant you already use. Nothing is sent by this site.

Demo

Change an input, predict the result, and see which quantities respond.

ExploreChange one control at a time and name what responds.An optional observation guide follows the demo.

Explore Vector Spaces

Loading the interactive demo…

Observation guide

Optional and ungraded. Opening a guide saves nothing.

Choose what to inspect in Vector Spaces, then open its guide.

Try this:

Drag u and v around.
Move sliders a and b to form the linear combination $w = a u + b v$ .
Before revealing the determinant, predict whether $u$ and $v$ span a plane, nearly collapse onto one line, or collapse to a line or a point.

Notice the two linked ideas: closure keeps $w$ inside the same space, while the determinant (the parallelogram's signed area) tells you whether $u$ and $v$ span the whole plane or have collapsed into fewer directions.

Study prompts for this section: Demo

Study prompts

Each button copies a prompt about this section to your clipboard, for an AI assistant you already use. Nothing is sent by this site.

Focus on

In focus

Concept: Vector Spaces

What is the smallest example of Vector Spaces you can work through by hand, and what does it show?

BeforeNo hard prerequisiteNextDot Product

DetailsLinear Algebra

About this itemLinear Algebra

2 sources citedChecking this browser

Reference id

concept:linear-algebra/vector-spaces

Math1 equationa u + b v \in V \qquad (u,v \in V, a,b \in \mathbb{F}).CodeVector Spaces code example 1u = np.array([1.0, 2.0, 3.0])DemoVector Spaces interactive demoChange one control at a time and name what responds.

After the first pass

Go deeper

Optional material for a second reading. Sources are always shown; the other panels open from their headings or from these links, and closing one keeps what you wrote or revealed.

What to look for in each sectionChoose a section and what you expect to see in it, then read short advice.

Section by section

Choose what to look for in each section

A vector space is a set of objects you can add and scale, where those operations behave consistently.

Choose what to look for (no demo yet)01 / Intuition

Illustration of basis vectors spanning a translucent vector-space sheet.

What to look for

Start with the picture, metaphor, or geometric mechanism.

Choose first

Choose what you expect to see in this section of Vector Spaces.

Questions to ask of the pictureChoose a part of the picture and what you expect, then read short advice.

Looking at the picture

Questions to ask of the picture

A vector space is a set of objects you can add and scale, where those operations behave consistently.

4 of 4 sections writtenNo live demo yet

Visual cueWhich part of the picture should you look at first?

Highlight size2/4

Question

Which part of the picture should you look at first?

Choose first

Pick the part of the picture you expect to explain Vector Spaces.

Sources

References for this page

A reference shows where an idea comes from; it does not vouch for every step on this page.

book · 2020Mathematics for Machine LearningDeisenroth, Faisal, and Ong

Why it is cited

Grounds vector spaces, bases, coordinates, and the linear algebra vocabulary needed for ML models.

Open source

book · 2024Linear Algebra Done Right (4th ed.)Axler

Why it is cited

Springer, open access. Chapter 1 defines a vector space over R or C by the axioms used on this page; Section 2A defines linear combinations and span.

Open source

What each claim rests on1 claim, each with its sources, where it appears on this page, and its caveats.

Claims and sources

A vector space is a set of objects you can add and scale, where those operations behave consistently.

Checked against the cited passages

The checks were made by this site, not by an independent reviewer.

A vector space has addition and scalar multiplication obeying the vector-space axioms; closure guarantees that for u,v in V and scalars a,b, the linear combination au+bv is still in V.

Sources

Mathematics for Machine LearningGrounds vector spaces, bases, coordinates, and the linear algebra vocabulary needed for ML models.

What the sources support

MML defines real-valued vector spaces by addition V x V -> V, scalar multiplication R x V -> V, and the usual group/distributive/scalar axioms. It also treats closure and linear combinations as scaled sums s...

Where to see it on this page

Mathematics for Machine Learning Equation 1 Code example 1 Demo

Equation 1

a u + b v \in V \qquad (u,v \in V,\; a,b \in \mathbb{F}).

Code example 1import numpy as np # A tiny sanity-check: R^3 with usual + and scalar * behaves like a vector...

Caveat

MML works over the real numbers; the page's field F (usually R or C) is the standard, slightly broader setting. The check covers closure under linear combinations, not affine spaces, modules, norms, complete...

StatusChecked against the cited passages.

MML Def. 2.9 defines real vector spaces by +: V x V -> V and scalar multiplication R x V -> V plus Abelian-group, distributive, scalar-associative, and identity axioms; Def. 2.11 defines linear combinations sum_i lambda_i x_i in V. The page's equations state the general form over a field; the code checks the operations in R^3, and the demo forms w=a u+b v in R^2.

Explain it without the pageWrite your own explanation before asking for feedback. Your draft and any hints stay while this page is open.

Practice · Vector Spaces

Try the idea in your own words

A vector space is a set of objects you can add and scale, where those operations behave consistently.

Concept · Selected for practice

Vector Spaces

What it rests on: Sources: Mathematics for Machine Learning; Linear Algebra Done Right (4th ed.)

Context and links

Linear Algebra

Choose a task

Explain the mechanism

For Vector Spaces: What is the smallest example of Vector Spaces you can work through by hand, and what does it show? Explain your answer, including what changes, why, and which assumption matters.

Your answerNo answer yet

A rough first thought is enough. Your draft stays when you change tasks.

Your draft stays on this page and is cleared when you leave or reload.

A little help · Explain

Open one hint at a time. These are suggestions, not your answer or a grade.

0 of 3 hints shown for this question.

Clearing your answer keeps this help record. Outside help is your own report; this page cannot check it.

Where am I stuck? (optional)

Your own description, not an automatic diagnosis

Choose one, or leave this unspecified. Select it again to clear it.

Take your draft to a feedback conversation

No AI feedback runs here. You can copy a prompt to use elsewhere; nothing is sent automatically. Review the text before sharing, and leave out private information.

Write an attempt before copying a feedback prompt.

A draft, or an AI reply to it, is not a test of what you have learned. To check that, try a different case later without help.

Ask about this pagePick one item on this page and copy a prompt about it into an AI assistant you already use. Notes you write here stay in this browser.

Ask about this pageClose

ConceptVector SpacesSources: Mathematics for Machine Learning; Linear Algebra Done Right (4th ed.)u = np.array([1.0, 2.0, 3.0])

Your question

Choose what your question is about

Pick the idea, equation, source, code, claim, misconception or demo state you want to ask about. The prompt you copy includes it, so the answer can stay on that item.

Next local actionNo local draft saved yet

Open the draft below to save one note and next action in this browser.

conceptLinear Algebra

Vector Spaces

Starting question

What is the smallest example of Vector Spaces you can work through by hand, and what does it show?

SourcesCheck the 2 cited sources listed under Sources.Notes can be saved for this item

Points of view on this item

Fixed prompts generated from this item. They are not comments from people or independent reviews.

Learner’s first stepAsk what would make "Vector Spaces" feel predictable rather than familiar.

Assumption

The 2 cited sources must support this exact item, not just the surrounding topic.

Source-checking summary

Connect the definition to one equation, piece of code or demo before widening the discussion.

Proposed experiment

Change one input, then check that the same relationship holds in the mathematics, the code and the demo.

Next action

You can state the mechanism in your own words

Evidence4 checks

ObservationChecking for a saved observation

ActionReady for one action

PromptLearner prompt ready to copy

Go to this item

01ObservationChecking what this browser has saved

02EvidenceChecking for a saved observation

03SourcesCheck the 2 cited sources listed under Sources.

04Next stepSave one next action

Local action draftNo local draft saved yetOpen when you are ready to write one next action

Local action draft

This draft stays in this browser and is attached to this item only.

Draft noteNext action

No local draft saved.

Evidence to inspect

What the 2 cited sources say about this exact item
The definition, its prerequisites, and a contrasting concept
The equation or code that makes the concept concrete
One demo state that shows what stays the same as the inputs change

What would resolve this

You can state the mechanism in your own words
You can name the prerequisite that would clear up the confusion
You can predict how the result changes when one input changes

Reference id

concept/concept-notebook/linear-algebra/vector-spaces
concept:linear-algebra/vector-spaces

Return to reading

Reading map

In this concept

Change one control at a time and name what responds.

Try the demo

Continue the curriculum

Part I · unit 1 of 13

Dot Product

Connections and sources

Domain

Linear Algebra

linear-algebra

Leads to

Linear IndependenceNo page yet
Basis and SpanNo page yet
Linear TransformationsNo page yet

Dot Product
NormsNo page yet
Representation Learning & Embedding Geometry

Study aid

Ask an assistant about this notebook

Compose a question with this notebook as context, then copy it into an AI assistant you already use.

Compose a question

Your question

GoalComfortStyleStuck on

Copy-only tool: this panel composes a prompt from this page and your choices above, for an AI assistant you already use. Nothing you enter here is sent from this page.