GCN, GraphSAGE, and GAT

Let $G=(V,E)$ be a graph with node-feature matrix

$$H^{(\ell)}\in\mathbb R^{|V|\times d}.$$

We focus on one target node $v$ and its one-hop neighborhood $\mathcal N(v)$. To compare operators cleanly, define a shared transformed feature

$$z_u = W h_u^{(\ell)}, \qquad z_u\in\mathbb R^{d'}.$$

The page's finite demo uses this same $z_u$ for all three operators so the visible difference is which nodes get how much influence, not a hidden difference in learned matrices.

GCN: fixed normalized adjacency

With self-loops, let

$$\hat A = A + I, \qquad \hat D_{ii}=\sum_j \hat A_{ij}.$$

A common GCN layer has the propagation form

$$H^{(\ell+1)} = \sigma\left(\hat D^{-1/2}\hat A\hat D^{-1/2}H^{(\ell)}W\right).$$

For a single node $v$, the coefficient from source node $u$ into $v$ is

$$c^{\mathrm{GCN}}_{v,u} = \frac{1}{\sqrt{\hat d_v\hat d_u}} \quad\text{when } u\in \mathcal N(v)\cup\{v\}.$$

The key point: the coefficient is fixed by the graph degrees. It is not learned per edge in this layer.

GraphSAGE: sampled aggregate plus self/root term

GraphSAGE keeps the neighborhood-aggregation idea but emphasizes sampling and inductive aggregation. A simplified mean-aggregator view is

$$a_v = \frac{1}{|S(v)|}\sum_{u\in S(v)} z_u, \qquad S(v)\subseteq \mathcal N(v),$$

then combines the aggregate with the node's own transformed state:

$$h_v^{(\ell+1)} = \sigma\left(W_{\mathrm{self}}h_v^{(\ell)} + W_{\mathrm{nbr}}a_v\right).$$

The demo uses a transparent root blend:

$$\tilde h_v = \frac{1}{2}z_v+\frac{1}{2}\left(\frac{1}{|S(v)|}\sum_{u\in S(v)}z_u\right).$$

That is not the full GraphSAGE training recipe. It is a finite witness for the operator idea: only sampled neighbors can contribute in that step.

GAT: learned attention over the neighborhood

Graph Attention Networks compute a coefficient for each allowed neighbor:

$$\alpha_{v,u} = \operatorname{softmax}_{u\in\mathcal N(v)\cup\{v\}} \left(e_{v,u}\right),$$

where $e_{v,u}$ is a learned compatibility score computed from transformed node features. The update is an attention-weighted sum:

$$\tilde h_v = \sum_{u\in\mathcal N(v)\cup\{v\}}\alpha_{v,u}z_u.$$

The key point: the graph still masks which nodes are allowed to speak, but the relative influence inside that neighborhood is learned.

So the comparison is:

$$\begin{aligned} \text{GCN} &:\quad \text{degree-normalized fixed coefficients},\\ \text{GraphSAGE} &:\quad \text{sampled aggregate plus self/root information},\\ \text{GAT} &:\quad \text{learned attention coefficients over allowed neighbors}. \end{aligned}$$