GCN: Graph Convolutional Networks — Complete Guide to Graph Convolutional Networks

GCNs (Graph Convolutional Networks) represent a major advance in the field of deep learning applied to graph-structured data. While classical neural networks process vector data and CNNs operate on regular grids (images, text sequences), GCNs extend the convolution principle to non-Euclidean structures. This guide will explore in depth the mathematical workings of GCNs, their fundamental intuition, their practical implementation in Python with PyTorch, as well as their concrete use cases.

Summary

A GCN is a type of neural network specifically designed to process data represented as graphs. Unlike traditional architectures that assume a regular data structure (like pixels in an image or words in a sentence), GCNs exploit explicit relationships between entities to propagate and transform information across the graph.

The core mechanism relies on neighborhood aggregation: each node collects information from its direct neighbors, combines it with its own features, then applies a linear transformation followed by an activation function. By stacking multiple layers of this type, a GCN progressively captures increasingly larger structures in the graph.

These models have demonstrated their effectiveness in many domains: molecule classification in chemistry, community detection in social networks, product recommendation, or knowledge analysis through knowledge graphs. In this guide, we will see how to build, train, and deploy them.

Mathematical Principle of GCNs

The theoretical foundation of GCNs rests on an elegant formula that generalizes convolution to graphs. The propagation equation of a GCN layer is written:

H^(l+1) = σ(D̃^(-1/2) · Ã · D̃^(-1/2) · H^(l) · W^(l))

Let’s break down each term of this fundamental equation:

Detailed Definitions of Components

• Ã = A + I: this is the adjacency matrix with self-loops. The identity matrix I is added to the original adjacency matrix A of the graph. This crucial operation allows each node to retain some of its own features during aggregation, rather than only receiving information from its neighbors. Without these self-loops, a node would completely lose its own identity at each propagation step.
• D̃: this is the degree matrix of Ã. It is a diagonal matrix where each element D̃_ii corresponds to the number of connections of node i in the augmented graph (i.e., counting self-loops). For example, if a node has 3 neighbors and a self-loop, its degree is 4.
• D̃^(-1/2) · Ã · D̃^(-1/2): this is the symmetric normalized adjacency matrix. This normalization term is essential: it prevents features from highly connected nodes from dominating the aggregation. Without normalization, a node with hundreds of neighbors would transmit much more information than an isolated node, creating an imbalance harmful to learning. Symmetric normalization ensures that each neighbor contributes proportionally to its degree in the graph.
• H^(l): this is the feature matrix at level l. Each row of this matrix corresponds to a node in the graph, and each column represents a feature dimension. For the first layer (l=0), H^(0) is simply the input feature matrix X. For subsequent layers, H^(l) represents the embeddings learned by previous layers.
• W^(l): this is the learned weight matrix. It is the only true learnable parameter of each GCN layer. This matrix is optimized by gradient backpropagation during training, exactly like the weights of a classical neural network.
• σ: this is a non-linear activation function, typically ReLU (f(x) = max(0, x)). This non-linearity is essential for the model to learn complex representations. If all layers were linear, a deep GCN would reduce to a single linear transformation, negating the entire benefit of the deep architecture.

Intuitive Interpretation of Propagation

Intuitively, the operation of a GCN breaks down into two clear steps at each layer:

1. Aggregation: each node aggregates the features of its direct neighbors via the product à · H^(l). Concretely, for a given node, the sum (weighted by normalization) of the feature vectors of all its connected neighbors is calculated.
2. Transformation: the aggregation is then linearly transformed by W^(l), then passed through the activation function σ. This transformation projects the aggregated information into a new representation space more useful for the target task.

By stacking multiple layers, information propagates step by step: after one layer, a node knows its direct neighbors; after two layers, it knows the neighbors of its neighbors; after k layers, it captures a radius of k hops around it in the graph. This is the graph equivalent of the receptive field in classical CNNs.

Geometric Intuition: Why Convolution on Graphs?

On a classical image, convolution aggregates neighboring pixels in a regular grid. A 3×3 filter always looks at exactly the 8 pixels surrounding the central pixel. The geometry is fixed, predictable, and identical everywhere in the image. Each pixel has exactly 4 or 8 neighbors, depending on whether von Neumann or Moore connectivity is considered.

But what if the data is not on a grid? Consider the following situations:

• Atoms in a molecule: each atom is connected to a variable number of other atoms. Hydrogen forms one bond, carbon forms four. There is no up, down, left, or right. The structure is a free graph with no preferred orientation.
• Users in a social network: some have thousands of friends, others only a few. Connections are irregular and asymmetric. A popular person will transmit information to a large number of nodes, while an isolated person will only communicate with their few contacts.
• Scientific articles linked by citations: a foundational article may be cited thousands of times, while a recent article cites only a few. The citation network forms a directed graph with a very uneven degree distribution.

In all these cases, the natural tool to represent data is the graph: a set of nodes (entities) connected by edges (relationships). The topology of the graph captures structural dependencies that neither a list nor a matrix can naturally express.

The GCN is a convolution that adapts to the shape of the graph: it only looks at direct neighbors, no matter how many there are or how they are arranged. It is this structural flexibility that makes GCNs so powerful. Where a CNN would require forcing the data into an artificial grid (with loss of relational information), a GCN respects the natural topology of the data.

Direct Analogy with CNNs

Consider the fundamental difference between the two architectures:

• CNN: “Look at the 3 pixels to the left, the 3 to the right, the 3 above, the 3 below.” — fixed and universal grid.
• GCN: “Look at all your neighbors, regardless of their number and position.” — grid adapted to the graph.

Both operations accomplish the same thing conceptually (neighborhood aggregation), but the GCN does it on an arbitrary, non-regular, and potentially dynamic topology. This generalization is at the heart of the contribution of Kipf and Welling in their foundational 2017 paper.

Python Implementation with PyTorch

We will explore two implementation approaches: first a from-scratch version without specialized dependencies, then a version with PyTorch Geometric.

1. GCN Layer From Scratch

Here is the implementation of a single GCN layer in pure PyTorch, without any specialized library:

import torch
import torch.nn as nn
import torch.nn.functional as F


class GCNLayer(nn.Module):
    """
    A single Graph Convolutional Network layer.

    Propagation: H' = sigma(D_tilde^(-1/2) . A_tilde . D_tilde^(-1/2) . H . W)

    Arguments:
        in_features: input feature dimension
        out_features: output dimension of the layer
    """

    def __init__(self, in_features, out_features):
        super().__init__()
        # Learned weight matrix (learnable parameter)
        self.weight = nn.Parameter(torch.FloatTensor(in_features, out_features))
        # Xavier initialization for stable convergence
        nn.init.xavier_uniform_(self.weight)

    def forward(self, x, adj_normalized):
        """
        Forward pass of the GCN layer.

        Arguments:
            x: node features, tensor (N, in_features)
            adj_normalized: normalized adjacency matrix (N, N)
        Returns:
            transformed features (N, out_features) after ReLU
        """
        # Step 1: linear transformation H . W
        support = torch.matmul(x, self.weight)
        # Step 2: neighbor aggregation via normalized matrix
        output = torch.spmm(adj_normalized, support)
        # Non-linear activation function
        return F.relu(output)

And here is the utility function to build the symmetric normalized adjacency matrix from a raw adjacency matrix:

def build_normalized_adj(adj_matrix):
    """
    Computes the symmetric normalized adjacency matrix.

    Formula: D_tilde^(-1/2) . A_tilde . D_tilde^(-1/2) where A_tilde = A + I

    Arguments:
        adj_matrix: adjacency matrix (N, N)
    Returns:
        normalized matrix (N, N) ready for GCN propagation
    """
    N = adj_matrix.shape[0]
    identity = torch.eye(N, device=adj_matrix.device)

    # Add self-loops: A_tilde = A + I
    adj_with_selfloops = adj_matrix + identity

    # Compute degrees of A_tilde (row sum)
    degrees = torch.sum(adj_with_selfloops, dim=1)

    # Compute D_tilde^(-1/2) element by element
    degrees_inv_sqrt = torch.pow(degrees, -0.5)
    # Protect against division by zero (isolated nodes)
    degrees_inv_sqrt[torch.isinf(degrees_inv_sqrt)] = 0

    # Build the diagonal matrix D_tilde^(-1/2)
    D_inv_sqrt = torch.diag(degrees_inv_sqrt)

    # Complete symmetric normalization
    normalized_adj = torch.mm(
        torch.mm(D_inv_sqrt, adj_with_selfloops),
        D_inv_sqrt
    )

    return normalized_adj

2. Complete GCN Model for Node Classification (Cora)

Now let’s build a two-layer GCN model for the classic node classification task on the Cora dataset (2,708 scientific articles, 5,429 citations, 7 topic categories):

import torch
import torch.nn as nn
import torch.nn.functional as F


class TwoLayerGCN(nn.Module):
    """
    Two-layer GCN model for node classification.
    Classic architecture: Input -> GCN -> ReLU -> Dropout -> GCN -> LogSoftmax
    """

    def __init__(self, num_features, num_hidden, num_classes, dropout=0.5):
        super().__init__()

        self.gc1 = GCNLayer(num_features, num_hidden)
        self.gc2 = GCNLayer(num_hidden, num_classes)
        self.dropout = nn.Dropout(dropout)

    def forward(self, x, adj_normalized):
        # First layer: aggregation + ReLU + dropout
        x = self.gc1(x, adj_normalized)
        x = F.relu(x)
        x = self.dropout(x)

        # Second layer: final aggregation
        x = self.gc2(x, adj_normalized)

        # LogSoftmax for multi-class classification
        return F.log_softmax(x, dim=1)

Complete training loop:

def train_gcn(model, features, adj_normalized, labels,
              train_mask, num_epochs=200, lr=0.01, weight_decay=5e-4):
    """
    Complete training loop for a GCN model.

    Arguments:
        model: the GCN model to train
        features: node feature matrix X (N, F)
        adj_normalized: normalized adjacency matrix (N, N)
        labels: classification labels (N,)
        train_mask: boolean mask of training nodes
        num_epochs: number of training epochs
        lr: learning rate
        weight_decay: L2 regularization
    """
    optimizer = torch.optim.Adam(
        model.parameters(), lr=lr, weight_decay=weight_decay
    )

    for epoch in range(num_epochs):
        model.train()
        optimizer.zero_grad()

        # Forward pass
        output = model(features, adj_normalized)

        # Compute loss only on training nodes
        loss = F.nll_loss(output[train_mask], labels[train_mask])

        # Backward pass + optimization
        loss.backward()
        optimizer.step()

        # Periodic evaluation
        if (epoch + 1) % 20 == 0:
            model.eval()
            with torch.no_grad():
                output = model(features, adj_normalized)
                preds = output.argmax(dim=1)
                accuracy = (preds == labels).float().mean()

            print(f"Epoch {epoch+1}/{num_epochs} -- "
                  f"Loss: {loss.item():.4f} -- "
                  f"Overall accuracy: {accuracy:.4f}")

    return model

3. Implementation with PyTorch Geometric

PyTorch Geometric (PyG) is the reference library for deep learning on graphs. It considerably simplifies the code and offers performance optimizations:

import torch
import torch.nn.functional as F
from torch_geometric.nn import GCNConv
from torch_geometric.datasets import Planetoid


class GCNWithPyG(torch.nn.Module):
    """
    GCN model using PyTorch Geometric.
    Much more concise and optimized than the manual implementation.
    """

    def __init__(self, num_features, num_hidden, num_classes):
        super().__init__()
        # The cached=True parameter avoids recomputing normalization
        self.conv1 = GCNConv(num_features, num_hidden, cached=True)
        self.conv2 = GCNConv(num_hidden, num_classes, cached=True)
        self.dropout = 0.5

    def forward(self, x, edge_index):
        """
        Arguments:
            x: node features (N, num_features)
            edge_index: edge list in COO format (2, E)
        """
        x = self.conv1(x, edge_index)
        x = F.relu(x)
        x = F.dropout(x, p=self.dropout, training=self.training)
        x = self.conv2(x, edge_index)
        return F.log_softmax(x, dim=1)


# ---- Dataset loading and training ----
dataset = Planetoid(root="/tmp/Cora", name="Cora")
data = dataset[0]  # Single transductive graph

model = GCNWithPyG(
    num_features=dataset.num_features,
    num_hidden=16,
    num_classes=dataset.num_classes
)

optimizer = torch.optim.Adam(
    model.parameters(), lr=0.01, weight_decay=5e-4
)

for epoch in range(200):
    model.train()
    optimizer.zero_grad()
    out = model(data.x, data.edge_index)
    loss = F.nll_loss(out[data.train_mask], data.y[data.train_mask])
    loss.backward()
    optimizer.step()

    if (epoch + 1) % 20 == 0:
        model.eval()
        with torch.no_grad():
            pred = out.argmax(dim=1)
            acc = (pred == data.y).float().mean()
        print(f"Epoch {epoch+1} -- Loss: {loss:.4f} -- Accuracy: {acc:.4f}")

PyTorch Geometric automatically handles the construction of the normalized adjacency matrix, sparse storage format for large graphs, and many performance optimizations based on C++/CUDA code. For large graphs (beyond 50,000 nodes), it is strongly recommended to use this library rather than a manual implementation.

Key GCN Hyperparameters

Hyperparameter tuning is crucial to achieve good performance with a GCN. Here are the three fundamental hyperparameters to understand:

num_layers (Number of Layers) — Typically 2-3

This is the most surprising hyperparameter of GCNs. Unlike classical CNNs where dozens of layers are stacked for better performance, GCNs generally work best with only 2 or 3 layers.

Why so few? Because of the over-smoothing problem: at each layer, node representations mix with those of their neighbors. Beyond 3 or 4 layers, the embeddings of all nodes tend to converge toward nearly identical vectors, making classification impossible. This is a phenomenon fundamentally different from the vanishing gradient of classical deep networks.

Some recent architectures (GCN with residual connections, Jumping Knowledge Networks, GCNII) attempt to address this problem and allow going deeper, but for a standard GCN, 2 layers remain the default choice recommended by Kipf & Welling as early as 2017.

hidden_dim (Hidden Dimension) — Typically 16-256

The hidden layer size controls the richness of the representation learned by the model. A dimension that is too small (4-8) will not capture enough relational information between nodes. A dimension that is too large (512 and above) risks overfitting, especially if the dataset is small like Cora with only 2,708 nodes and 140 training examples per class.

Rule of thumb: for a first try, start with hidden_dim = 16 (as in the original paper) and increase gradually if validation performance is insufficient.

dropout — Typically 0.3-0.6

Dropout is even more important in GCNs than in classical networks, because GCNs tend to overfit quickly on small graphs with few labeled examples. A value of 0.5 is the recommended starting point. On larger graphs (100,000 nodes and above), you can reduce to 0.2-0.3 since the overfitting risk decreases with the amount of data.

Advantages and Limitations of GCNs

Advantages

1. Native exploitation of structural relationships: unlike other deep learning architectures, GCNs explicitly use the graph structure rather than treating it as a secondary artifact. Relationships between entities are at the heart of the model, not an afterthought.
2. Transductive and inductive operation: GCNs can work in both modes. In transductive mode, the model learns on a fixed graph and predicts on the nodes of that same graph (like classification on Cora). In inductive mode (with variants like GraphSAGE or GCNs based on neighbor sampling), the model can generalize to nodes never seen during training.
3. Remarkable parameter efficiency: a two-layer GCN on Cora has only a few thousand parameters, compared to millions for a large CNN on ImageNet. This efficiency makes GCNs accessible even with limited computing resources and allows fast training on CPU.
4. Superior interpretability compared to classical networks: unlike deep black boxes, it can be traced which contributions from which neighbors lead to which prediction. For example, in a citation graph, one can identify which cited articles most influenced the classification of a given document. This traceability is valuable for critical applications like fraud detection or medical diagnosis.

Limitations

1. Over-smoothing: as mentioned above, stacking more than 3-4 layers makes node representations indistinguishable. This is the most fundamental limitation of GCNs and an active research area since 2019.
2. Memory consumption for very large graphs: the adjacency matrix of a graph with N nodes can require O(N²) memory in the worst case (although sparse formats reduce this to O(E) where E is the number of edges). For graphs with millions of nodes, mini-batching, neighbor sampling, or gradient checkpointing techniques are absolutely necessary.
3. Homophily assumption: standard GCNs implicitly assume that connected nodes are similar (homophily). In heterophilous graphs (where connected nodes are structurally dissimilar, typically bipartite graphs), performance drops significantly. Specialized variants like heterophilous GCNs (H2GCN, CPGCN) address this problem by introducing mechanisms to separate similar and dissimilar neighbors.
4. Static graphs only: classical GCNs do not naturally handle dynamic graphs whose structure (edges) or features evolve over time. For these cases, specialized temporal architectures such as EvolveGCN, Temporal GCN, or Dynamic Graph Neural Networks are needed.
5. No native handling of weighted or typed edges: the original GCN treats all edges in a binary fashion (present or absent). For graphs with weighted edges (link intensity) or typed edges (different types of relationships), adaptations are necessary (R-GCN for relational graphs, for example).

4 Concrete and Detailed Use Cases

1. Molecule Classification in Computational Chemistry

In computational chemistry, a molecule is naturally represented as a graph: atoms are the nodes and chemical bonds are the edges. A GCN can learn to predict molecular properties (toxicity, solubility, biological activity against a protein target) directly from this graph structure.

Each atom has rich features: atom type (carbon, oxygen, nitrogen, etc.), electric charge, hybridization degree, number of implicit hydrogens. The GCN propagates this information through chemical bonds, allowing the model to capture structural motifs such as aromatic rings, functional groups, or spatial conformations.

The advantage over traditional molecular descriptors (like Morgan ECFP fingerprints) is that the GCN automatically learns the optimal representations for the task, rather than relying on manual expert encoding.

2. Graph-Based Recommendation System

In a modern recommendation system, a bipartite graph can be constructed connecting users to the products they have purchased, viewed, or rated. A GCN on this graph learns representations that capture both the implicit preferences of users and the latent characteristics of products.

The advantage over classical matrix factorization methods is that the GCN captures multi-hop relationships: “users who liked this product also like this other product” — a form of second-order collaborative filtering, automatically inferred through propagation. The company Pinterest uses this approach (via their PinSAGE model) to recommend pins to its hundreds of millions of users.

3. Financial and Cybercrime Fraud Detection

In banking, relationships between bank accounts, transactions, IP addresses, devices used, and beneficiaries can be modeled as a massive heterogeneous graph. Fraudulent accounts often exhibit specific structural patterns: connections to devices shared by many accounts, circular transaction loops, newly created accounts interacting intensively with already flagged accounts.

A GCN detects these patterns by learning embeddings that reflect the structural position of each node in the fraud graph. Nodes with “suspicious” embeddings are then classified as potentially fraudulent, often hours ahead of traditional rule-based if-then methods. This approach is deployed by several international banks and payment institutions.

4. Document Classification and Academic Citation Networks

The canonical and pedagogical example of GCNs is the classification of academic documents linked by citations (such as the Cora dataset used in the code examples). Each document is represented by a bag of words or a TF-IDF embedding (textual features), and the edges are the citations between documents.

The GCN combines the textual features of each document with the implicit categories revealed by citations between similar documents. Intuitively, if a document is cited by many articles on machine learning and itself cites articles on the subject, the GCN reinforces the probability that it belongs to that category. The result is classification that is much more accurate than using textual features alone.

This approach naturally applies to any type of documents linked by explicit relationships: legal articles citing case law, patents referring to prior patents, or support tickets linked to similar incidents.

See Also

• Master Calculating the Sum of Divisors with Python: Complete Guide and Effective Tips
• Create a Sliding Puzzle Game in Python: Complete Guide and Tips for Developers