Machine Learning

Node Embeddings: DeepWalk and node2vec

Predict how node2vec's p and q change a random walk, then see how walk context pairs pull node embeddings together.

status: reviewimportance: importantdifficulty 4/5math: graduateread: 18mlive demo

Back to Machine Learning Next: GCN, GraphSAGE, and GAT (review)

Concept Structure

Node Embeddings: DeepWalk and node2vec

01Intuition

Start with the picture, metaphor, or geometric mechanism.

02Math

Make the objects explicit and connect them with notation.

03Code

Mirror the equations with runnable implementation details.

04Interactive Demo

Manipulate the mechanism and watch the idea respond.

2prerequisites

2next concepts

3related links

Learner Contract

What this page should let you do.

You are here becausePredict how node2vec's p and q change a random walk, then see how walk context pairs pull node embeddings together.

This Machine Learning concept is the current idea: keep the same invariant visible across Intuition, Math, Code, Interactive Demo.

Before thisGraph Basics: Adjacency, Degree, and Laplacian (review)

2 prerequisites listed; refresh them before leaning on the math or code.

Graph Basics: Adjacency, Degree, and Laplacian (review)Word2Vec, GloVe, and Subword Embeddings (review)

By the end4/4 sections ready | runnable code expected | live demo

Explain the mechanism, trace the main notation, and test one prediction in the live demo.

Do this firstIntuition

Read the intuition before the notation; the math should name a mechanism you already felt.

Then go nextGCN, GraphSAGE, and GAT (review)

Follow this edge after making one prediction here; the next page should reuse the result, not restart the route.

Test the linkManipulate one control and predict the visible change.Then continue to GCN, GraphSAGE, and GAT (review)

Claim/source review status

Substantive review recorded

3/3 claims have bounded review metadata; still check caveats and source scope.Metadata-derived; review may be AI-assisted. Not a human certification.

Claims3/3 reviewed

Sources5 cited

Codeattached

Demolive

Reviewed2026-07-03

Updatedpage 2026-07-03

Learning item flow

4/4 sections readyAsk about this Research room

ConceptNode Embeddings: DeepWalk and node2vecMachine Learning EquationNode Embeddings: DeepWalk and node2vec equation 1Exact equation CodeNode Embeddings: DeepWalk and node2vec runnable code 1Exact runnable code DemoNode Embeddings: DeepWalk and node2vec interactive demoInteractive demo ClaimDeepWalk learns node embeddings by sampling truncated random walks, t...Exact claim check SourceDeepWalk: Online Learning of Social RepresentationsExact source

ConceptNode Embeddings: DeepWalk and node2vecMachine Learning

5 sources attachedLocal snapshot ready

concept:machine-learning/node-embeddings-deepwalk-node2vec

Codenearby Predictbefore reveal Roomitem handoff

01IntuitionStart with the picture, metaphor, or geometric mechanism.02MathMake the objects explicit and connect them with notation.03CodeMirror the equations with runnable implementation details.04Interactive DemoManipulate the mechanism and watch the idea respond.

Intuition

Build the mental picture first so the rest of the page has something to attach to.

Section prompt

A node embedding is a learned vector for a node ID. DeepWalk and node2vec ask: if we do not have node features yet, can the graph's walking patterns give us a useful training signal?

The trick is to turn a graph into sequences. Start random walks from nodes. Treat each walk like a short sentence. Then train a Skip-gram-style objective: a center node should predict nearby nodes in the walk window.

So the analogy to Word2Vec is real, but limited:

words become node IDs;
sentences become sampled walks;
context windows become walk-neighborhood pairs;
the learned vector stores random-walk co-occurrence evidence, not a dictionary definition of the node.

DeepWalk samples ordinary truncated random walks. If a walk often crosses from C to D, then C and D appear in each other's context windows and receive more positive updates.

node2vec keeps the same "walks become context pairs" idea, but changes the walk sampler. After the walk moves from a previous node t to a current node v, the next candidate x is not chosen only by edge weight. It is also biased by two knobs:

p, the return parameter, controls immediate backtracking to t;
q, the in-out parameter, controls whether the walk stays near t or moves outward.

High q makes outward moves less likely, so the walk behaves more locally, like a breadth-first neighborhood sampler. Low q makes outward moves more likely, so the walk explores farther, like a depth-first sampler. These are biases inside a random walk, not literal BFS or DFS traversals.

This matters because the walk corpus decides the training data. Change the walk, and you change which node pairs the embedding objective sees.

The main caveat is also important: these are shallow, transductive embeddings. The model learns a row for each node seen during training. A new node or a new graph is not handled the way a graph neural network handles features and neighborhoods; that requires a downstream encoder model.

Sources used: Perozzi, Al-Rfou, and Skiena, DeepWalk; Grover and Leskovec, node2vec; Stanford SNAP, node2vec project; Stanford CS224W Node Embeddings; Hamilton, Graph Representation Learning Chapter 3.

Math

Translate the story into symbols, assumptions, and a derivation you can inspect.

Section prompt

Equation 1 $z_u \in \mathbb{R}^d$ Equation 2 $W=(v_1,v_2,\ldots,v_L)$

Let $G=(V,E)$ be an unweighted graph. A shallow node embedding model stores a vector

z_u \in \mathbb{R}^d

for each node $u\in V$ .

DeepWalk samples walks

W=(v_1,v_2,\ldots,v_L)

where each next node is sampled from the neighbors of the current node. With context window radius $c$ , a training pair is formed whenever $|i-j|\le c$ and $i\ne j$ :

(v_i, v_j).

A simplified full-softmax Skip-gram objective maximizes

\sum_{(u,v)\in \mathcal{D}} \log P(v\mid u),

where

P(v\mid u)= \frac{\exp(z_u^\top z_v)} {\sum_{w\in V}\exp(z_u^\top z_w)}.

In practice, the denominator is expensive for large graphs, so implementations use approximations such as hierarchical softmax or negative sampling. The conceptual point is unchanged: random-walk co-occurrence supplies the positive pairs.

node2vec changes how the walk corpus $\mathcal{D}$ is sampled. Suppose the walk just moved from previous node $t$ to current node $v$ . For a candidate next node $x\in N(v)$ , define

\pi_{vx} = \alpha_{pq}(t,x)\,w_{vx},

where $w_{vx}$ is the edge weight and

\alpha_{pq}(t,x)= \begin{cases} \frac{1}{p}, & d(t,x)=0,\\ 1, & d(t,x)=1,\\ \frac{1}{q}, & d(t,x)=2. \end{cases}

Here $d(t,x)$ is the shortest-path distance from the previous node $t$ to candidate $x$ . Because $x$ is adjacent to $v$ , this distance can only be $0$ , $1$ , or $2$ .

The normalized transition probability is

P(X_{i+1}=x\mid X_i=v, X_{i-1}=t) = \frac{\pi_{vx}}{\sum_{y\in N(v)}\pi_{vy}}.

Read the knobs carefully:

low $p$ increases immediate return probability;
high $p$ discourages immediate return;
high $q$ downweights outward nodes and keeps the walk local;
low $q$ upweights outward nodes and makes farther exploration more likely.

The embedding objective then sees the resulting walk windows. The loss did not magically learn "graph meaning"; it learned to make the sampled random-walk neighborhoods predictable.

Code

Keep the implementation aligned with the notation so the algorithm is legible.

Section prompt

Runnable code 1from collections import Counter graph = { "A": {"B", "C"}, "B": {"A", "C"}, "C": {"A", "B", "...python

This script mirrors the lab transition B -> C -> ? and counts walk context pairs.

from collections import Counter

graph = {
    "A": {"B", "C"},
    "B": {"A", "C"},
    "C": {"A", "B", "D"},
    "D": {"C", "E", "F"},
    "E": {"D", "F"},
    "F": {"D", "E"},
}

def distance(prev, candidate):
    if candidate == prev:
        return 0
    return 1 if candidate in graph[prev] else 2

def transition_probs(prev, current, p=1.0, q=1.0):
    weights = {}
    for x in sorted(graph[current]):
        d = distance(prev, x)
        bias = (1 / p) if d == 0 else (1 if d == 1 else 1 / q)
        weights[x] = bias
    total = sum(weights.values())
    return {x: round(w / total, 3) for x, w in weights.items()}

print("local, high q:", transition_probs("B", "C", p=1, q=2))
print("outward, low q:", transition_probs("B", "C", p=1, q=0.5))

def context_pairs(walks, window=1):
    counts = Counter()
    for walk in walks:
        for i, center in enumerate(walk):
            for j in range(max(0, i-window), min(len(walk), i+window+1)):
                if i != j:
                    counts[tuple(sorted((center, walk[j])))] += 1
    return counts

local_walks = [["B", "C", "A", "C", "B"], ["A", "C", "B", "C", "A"]]
outward_walks = [["B", "C", "D", "E", "F"], ["A", "C", "D", "F", "E"]]

print("local C-D count:", context_pairs(local_walks)[("C", "D")])
print("outward C-D count:", context_pairs(outward_walks)[("C", "D")])

Interactive Demo

Use direct manipulation to connect the explanation to a moving system.

Section prompt

Prediction check: inspect the tiny graph and the partial walk B -> C -> ?. First predict which candidate becomes most likely when q is low. Then predict which node pair receives more context-window updates in the outward walk batch.

The lab hides the transition probabilities, context-pair counts, and embedding movement until you commit both predictions. The goal is to connect the p/q sampler to the training pairs, not to memorize node2vec as a black-box embedding recipe.

Live Concept Demo

Explore Node Embeddings: DeepWalk and node2vec

The stage is code-native and interactive. Use it to test the explanation against the mechanism.

difficulty 4/5graduatecode-aligned

Demo Prediction Checkpoint

Manipulate one control and predict the visible change.

Commit to what Node Embeddings: DeepWalk and node2vec should make visible before reading the result.

After The First Pass

Turn the concept into an inspected object.

Once the invariant is visible in the intuition, math, code, and demo, use these panels to inspect the mechanism visually, check source support, practice the idea, and attach a grounded research question.

Mechanism Storyboard

See the idea move before the page explains it

Predict how node2vec's p and q change a random walk, then see how walk context pairs pull node embeddings together.

Prediction open01 / Intuition

Prediction lens

Start with the picture, metaphor, or geometric mechanism.

Commit first

Before reading further, choose the kind of change Node Embeddings: DeepWalk and node2vec should make visible.

Visual Inquiry

Make the image answer a mathematical question

Predict how node2vec's p and q change a random walk, then see how walk context pairs pull node embeddings together.

4/4 stages readyLive demo connected

Visual cueWhich visible object should carry the first intuition?

Inspection depth2/4

Prediction

Which visible object should carry the first intuition?

Commit first

Pick the cue that should make Node Embeddings: DeepWalk and node2vec easier to reason about before the page gives the answer.

Source Grounding

Canonical references for the mechanism on this page.

paper · 2014DeepWalk: Online Learning of Social RepresentationsPerozzi, Al-Rfou, and Skiena

Primary source for treating truncated random walks as sentence-like node sequences and training node representations with Skip-gram-style context prediction.

Open source

paper · 2016node2vec: Scalable Feature Learning for NetworksGrover and Leskovec

Primary source for the second-order biased random walk, p/q return and in-out parameters, and the Skip-gram network-neighborhood objective.

Open source

reference · 2016node2vec: Scalable Feature Learning for NetworksStanford SNAP

Project reference for the p/q intuition, homophily-to-structural-equivalence framing, reference implementation, and datasets.

Open source

course-notes · 2025Stanford CS224W: Node EmbeddingsJure Leskovec, Stanford CS224W

Course source for random-walk co-occurrence statistics, embedding-space similarity, DeepWalk/node2vec examples, and limitations of shallow random-walk embeddings.

Open source

book · 2020Graph Representation Learning, Chapter 3: Neighborhood Reconstruction MethodsWilliam L. Hamilton

Book source for the encoder-decoder view, shallow embedding lookup, inner-product decoder, random-walk co-occurrence objective, and limitations.

Open source

Claim Review

Predict how node2vec's p and q change a random walk, then see how walk context pairs pull node embeddings together.

Status3 substantive reviews recorded

Claims without a substantive review badge still need exact source-support review.

Sources5 references

deepwalk-2014, node2vec-2016, snap-node2vec-project, cs224w-2025-node-embeddings, hamilton-2020-grl-node-embeddings

Local checks4 local checks

Use equations, runnable code, and demos to check whether the source support is operational.

Substantively reviewedDeepWalk learns node embeddings by sampling truncated random walks, treating the resulting node sequences like sentences, and updating node vectors so nodes that co-occur in walk context windows become easier to predict from one another.Claim metadata: source checked

DeepWalk defines short random walks as the corpus over graph vertices and applies a Skip-gram-style context objective; CS224W and Hamilton frame the same mechanism as embedding random-walk co-occurrence statistics.

Sources: DeepWalk: Online Learning of Social Representations, Stanford CS224W: Node Embeddings, Graph Representation Learning, Chapter 3: Neighborhood Reconstruction MethodsThe demo counts a few hand-written walks and uses a two-dimensional teaching projection; it does not train a full embedding matrix, tune walk length/window size, or evaluate downstream classification or link prediction.A bounded review summary is present; still check caveats and exact reference scope.

Checked DeepWalk, CS224W node-embedding slides, and Hamilton's graph representation learning chapter for truncated random walks, context-window training, shallow embeddings, and random-walk co-occurrence scope.

Reviewer: codex-local-primary-source-audit; reviewed 2026-07-03

source-span-deepwalk-2014 source-span-cs224w-2025-node-embeddings source-span-hamilton-2020-grl-node-embeddings Equation 1 Runnable code 1 Interactive demo

Substantively reviewednode2vec changes the walk corpus with a second-order transition rule: after moving from t to v, a candidate x is weighted by 1/p when x=t, by 1 when x is adjacent to t, and by 1/q when x is farther from t; high q favors local/BFS-like sampling, while low q favors outward/DFS-like sampling.Claim metadata: source checked

Grover and Leskovec define the p/q biased transition; the Stanford SNAP page and CS224W slides support the local versus outward walk intuition and the random-walk neighborhood objective.

Sources: node2vec: Scalable Feature Learning for Networks, node2vec: Scalable Feature Learning for Networks, Stanford CS224W: Node EmbeddingsThe words BFS-like and DFS-like describe a bias in a random-walk sampler, not literal breadth-first or depth-first traversal order.A bounded review summary is present; still check caveats and exact reference scope.

Checked node2vec paper and SNAP project page for p/q weighting and search-bias interpretation; checked CS224W for the random-walk similarity framing and shallow-embedding limitations.

Reviewer: codex-local-primary-source-audit; reviewed 2026-07-03

source-span-node2vec-2016 source-span-snap-node2vec-project Equation 2 Interactive demo

Substantively reviewedDeepWalk and node2vec are shallow transductive embedding methods: they learn rows for nodes seen during training, so they do not directly compute embeddings for unseen nodes or new graphs without retraining or an additional encoder.Claim metadata: source checked

CS224W explicitly lists transductive and feature-use limitations of DeepWalk/node2vec; Hamilton describes shallow node embeddings as an embedding lookup and contrasts them with encoders that use features or local graph structure.

Sources: Stanford CS224W: Node Embeddings, Graph Representation Learning, Chapter 3: Neighborhood Reconstruction MethodsInductive variants and GNNs appear downstream; this concept focuses on the random-walk shallow-embedding family.A bounded review summary is present; still check caveats and exact reference scope.

Checked CS224W limitations and Hamilton's shallow-embedding encoder discussion to avoid overclaiming node embeddings as a full graph-feature encoder.

Reviewer: codex-local-primary-source-audit; reviewed 2026-07-03

source-span-cs224w-2025-node-embeddings source-span-hamilton-2020-grl-node-embeddings Interactive demo

Source support candidates

paper 2014DeepWalk: Online Learning of Social Representations

Primary source for treating truncated random walks as sentence-like node sequences and training node representations with Skip-gram-style context prediction.

paper 2016node2vec: Scalable Feature Learning for Networks

Primary source for the second-order biased random walk, p/q return and in-out parameters, and the Skip-gram network-neighborhood objective.

reference 2016node2vec: Scalable Feature Learning for Networks

Project reference for the p/q intuition, homophily-to-structural-equivalence framing, reference implementation, and datasets.

course-notes 2025Stanford CS224W: Node Embeddings

Course source for random-walk co-occurrence statistics, embedding-space similarity, DeepWalk/node2vec examples, and limitations of shallow random-walk embeddings.

Mechanism checks

Equation 1

z_u \in \mathbb{R}^d

Equation 2

W=(v_1,v_2,\ldots,v_L)

Runnable code 1from collections import Counter graph = { "A": {"B", "C"}, "B": {"A", "C"}, "C": {"A", "B", "...Demo stateLive mechanism probe

Practice Loop

Try the idea before it explains itself

Predict how node2vec's p and q change a random walk, then see how walk context pairs pull node embeddings together.

Readiness0/3 checks ready

Predict

Before touching the demo, predict one visible change that should happen in Node Embeddings: DeepWalk and node2vec.

Hint 1

Reveal when your model needs a nudge.

Hint 2

Reveal when your model needs a nudge.

Hint 3

Reveal when your model needs a nudge.

Your answer canvas

Local checks

Claim

A concrete answer is on the canvas.

Mechanism

The answer names why the claim should hold.

Bridge

It touches the page context or a neighboring idea.

Misconception check

Grounded research drawerClose

ConceptNode Embeddings: DeepWalk and node2vecMachine Learning

Runnable code comparisonNode Embeddings: DeepWalk and node2vec runnable code 1graph = {Prediction before revealNode Embeddings: DeepWalk and node2vec interactive demoManipulate one control and predict the visible change.

Grounded room questionWhat is the smallest example that makes Node Embeddings: DeepWalk and node2vec click without losing the math?Local snapshot ready

Research Room

Attach the question to a claim, equation, code, or demo

Pick the concept, equation, source, runnable code, claim, misconception, or demo state before asking for help. The handoff keeps that page item in context.

Next local actionNo local draft saved yet

Open the draft below to save one note and next action in this browser.

conceptMachine Learning

Node Embeddings: DeepWalk and node2vec

Attached question

What is the smallest example that makes Node Embeddings: DeepWalk and node2vec click without losing the math?

Local action draftNo local draft saved yetExpand only when ready to capture one local next action

Local action draft

This draft stays in this browser, attached to the selected learning item.

Draft noteNext action

No local draft saved.

Evidence to inspect

References to inspect: attached references on this page.
Definition, prerequisite, and contrast concept links
The equation or runnable code that makes the concept operational
One demo state that shows the invariant instead of a slogan

What would resolve this

The learner can state the mechanism in their own words
The learner can name the prerequisite that would repair confusion
The learner can predict how the mechanism changes under one perturbation

Grounded AI handoff

I am working in Continuous Function's research reading room. Object: concept - Node Embeddings: DeepWalk and node2vec Selected item key: recorded for copy. Context: Machine Learning Page anchor: recorded for copy. Open question: What is the smallest example that makes Node Embeddings: DeepWalk and node2vec click without losing the math? Evidence to inspect: - References to inspect: attached references on this page. - Definition, prerequisite, and contrast concept links - The equation or runnable code that makes the concept operational - One demo state that shows the invariant instead of a slogan What would resolve this: - The learner can state the mechanism in their own words - The learner can name the prerequisite that would repair confusion - The learner can predict how the mechanism changes under one perturbation Answer as a careful research tutor: stay source-grounded, separate verified evidence from assumptions, name the relevant math objects, and end with one next action.

View it in context

concept/concept-notebook/machine-learning/node-embeddings-deepwalk-node2vec
concept:machine-learning/node-embeddings-deepwalk-node2vec

Learning Map

Before / Now / Try / Next

BeforeGraph Basics: Adjacency, Degree, and Laplacian (review)

NowIntuition -> Math -> Code -> Demo

TryManipulate one control and predict the visible change.

NextGCN, GraphSAGE, and GAT (review)

Intuitionready
Mathready
Codeready
Interactive Demoready

AI Companion

Ask beside the selected object

Predict how node2vec's p and q change a random walk, then see how walk context pairs pull node embeddings together.

Your question

GoalComfortStyleStuck on

Context prompt

You are my AI learning companion for Continuous Function. Current context: Machine Learning concept. Learning surface: Node Embeddings: DeepWalk and node2vec. What this page says: Predict how node2vec's p and q change a random walk, then see how walk context pairs pull node embeddings together. Current section: Intuition, math, code, and interactive demo. Suggested next step: Manipulate one control and predict the visible change.. Learner goal: Understand the idea. Learner comfort level: New to this. Preferred explanation style: Visual first. Task: Explain the central idea in plain language, then restate it with the exact math objects from the page. Answer in a way that helps me learn: ask one clarifying question only if needed, use intuition before notation, and end with one thing I should try on the page.

Domain

Machine Learning

graph-mlnode-embeddingsdeepwalknode2vecrepresentation-learningrandom-walks

Prerequisites

Graph Basics: Adjacency, Degree, and Laplacian (review)Word2Vec, GloVe, and Subword Embeddings (review)

Leads To

GCN, GraphSAGE, and GAT (review)Message Passing Neural Networks (review)

Graph Theory Basics for GNNs and Graphical Models (review)Embedding, Unembedding, and Weight Tying (review)Language-Model Evaluation and Perplexity (review)

Within this domain