Semi-Supervised GCN: Node Classification with Few Labels

Summary

Semi-supervised Graph Convolutional Networks (GCNs) represent one of the most elegant advances in machine learning applied to relational data. Unlike classical methods that require thousands of labeled samples, the semi-supervised GCN manages to classify all nodes in a graph by using only a tiny fraction of labels — sometimes just a single label per class. This semi-supervised approach relies on a fundamental mechanism: the propagation of information along the edges of the graph, allowing representations learned from a few known nodes to influence the classification of their neighbors, then their neighbors’ neighbors, and so on. This complete guide explores the mathematical principle of the GCN, its deep intuition, its practical implementation with PyTorch Geometric, as well as its advantages, limits, and concrete use cases. You will discover how to structure a semi-supervised training pipeline, choose optimal hyperparameters, and visualize learned embeddings using t-SNE. If you work with network data — scientific publications citing other articles, molecules where atoms are connected by chemical bonds, or social networks — the semi-supervised GCN is the tool you need.

Mathematical Principle of the Semi-Supervised GCN

The Graph Convolution Layer

The heart of the semi-supervised GCN rests on a convolution operation adapted to graphs. Whereas a classical convolution (such as in convolutional neural networks for images) aggregates information from neighboring pixels on a regular grid, graph convolution must work on an arbitrary topology. The formulation proposed by Kipf & Welling (2017) expresses a GCN layer as follows:

H⁽ˡ⁺¹⁾ = σ(D̃⁻¹/² Ã D̃⁻¹/² H⁽ˡ⁾ W⁽ˡ⁾)

Let’s break down each term of this fundamental equation:

• Ã = A + I is the adjacency matrix of the graph to which self-loops have been added (loops on each node, represented by the identity matrix I). Adding these loops allows each node to retain part of its own representation when aggregating from its neighbors — without this trick, information would only spread to neighbors and the node would “forget” its own characteristic.
• D̃ is the degree matrix of Ã, that is, a diagonal matrix where each element D̃ᵢᵢ equals the sum of the elements in row i of Ã. This matrix captures the number of connections of each node (including the self-loop).
• D̃⁻¹/² is the inverse square root of this degree matrix. This term normalizes the aggregation: without this symmetric normalization, highly connected nodes (hubs in the graph) would produce much larger activations than sparsely connected nodes, creating a numerical imbalance during training. The normalization D̃⁻¹/² Ã D̃⁻¹/² ensures that the aggregated features remain on a comparable scale.
• H⁽ˡ⁾ is the activation matrix at layer l. At the input layer (l = 0), this is simply the node feature matrix X. At each subsequent layer, H⁽ˡ⁾ contains the enriched representations of the nodes, having already integrated information from neighbors further and further away.
• W⁽ˡ⁾ is the learnable weight matrix of layer l. It is the only parameter that the network will optimize during training. Despite its apparent simplicity, this linear matrix is sufficient to learn powerful transformations of the aggregated features.
• σ is a non-linear activation function, typically ReLU (Rectified Linear Unit). It is this non-linearity that allows the GCN to learn complex functions and capture subtle patterns in the graph structure.

The Receptive Field: k Hops Across k Layers

An essential point to understand is that each GCN layer aggregates information from direct neighbors (1 hop). Stacking k layers means that at the final layer, each node’s information has been mixed with that of all nodes located at most k hops away in the graph. This is called the node’s receptive field.

In practice, GCNs generally use 2 or 3 layers. Beyond that, two problems arise:

1. Over-smoothing: after too many aggregation layers, the representations of distant nodes — even of different classes — converge to similar values, making classification impossible. This is the graph equivalent of the vanishing gradient problem in classical deep networks.
2. Computational cost: the receptive field grows exponentially with the number of layers (on a graph with average degree d). For an average social graph with degree 50, 3 layers implies a theoretical receptive field of 50³ = 125,000 nodes.

Semi-Supervised Loss Function

The loss function of the semi-supervised GCN combines two components. The first is a standard cross-entropy, but computed only on training nodes (those with a known label):

L = -Σ{l∈train} Σ{f=1}^{F} Y_{lf} · ln(Ŷ_{lf}) + λ · Ω

Where:

• Y_{lf} is the one-hot encoded label for node l and class f. If node l belongs to class f, Y_{lf} = 1, otherwise Y_{lf} = 0.
• Ŷ_{lf} is the probability predicted by the model for node l and class f, obtained after applying a softmax to the outputs of the last GCN layer.
• The cross-entropy penalizes the model when its predictions differ from the true labels, but only for nodes whose label is known (l ∈ train).
• λ · Ω is a regularization term on the weights W of all layers (typically L2 regularization: Ω = ||W||²). This term controls model complexity and avoids overfitting, which is particularly critical in semi-supervised learning where the number of available labels is very small.

The magic of the semi-supervised GCN lies in the fact that minimizing this loss, computed on a small subset of nodes, leads to an optimization of the weights W that benefits all nodes in the graph — even those without labels. This is because the representations H⁽ᴸ⁾ of all nodes are computed via the same aggregation operations: if the model learns to correctly classify the labeled nodes, it has necessarily learned useful representations for their neighbors.

Intuition: The Conference with Lost Badges

Imagine a conference where only a few people wear badges with their name and role. The others have no badge. But by observing who talks to whom, and knowing that data scientists mostly talk to other data scientists, engineers to other engineers, and designers to other designers, you can progressively guess the role of those without badges. This is exactly what the semi-supervised GCN does.

Let’s break down the analogy:

• The graph: each conference participant is a node. Each observed conversation is an edge connecting two nodes.
• Features: for each participant, we know certain attributes — the vocabulary they use, the keywords of their conversation, their style of dress. These are the node features.
• Known labels: a few participants wear a badge. We know for certain that “Marie is a data scientist” and “Pierre is a designer.” These are the labeled nodes of the training set.
• Propagation: we notice that Thomas talks intensely with Marie (known as a data scientist). He also uses a technical vocabulary close to Marie’s. In all likelihood, Thomas is also a data scientist. Then we notice that Sophie talks with Thomas — she might well be one too. This is the propagation of information through the GCN layers.
• Local regularization: the implicit assumption is that connected people (who talk to each other) are likely to share the same role. This is the smoothness assumption on the graph: nodes close in the graph’s topology should have similar labels. The GCN mechanically encodes this assumption in its aggregation layer.

This intuition captures the essence of the semi-supervised GCN: a small number of known truths, combined with a rich relational structure, is enough to deduce the unknown through iterative propagation.

Python Implementation with PyTorch Geometric

Installation and Preparation

import torch
import torch.nn.functional as F
from torch_geometric.datasets import Planetoid
from torch_geometric.nn import GCNConv
from sklearn.manifold import TSNE
import matplotlib.pyplot as plt
import numpy as np

torch.manual_seed(42)
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

Loading the Cora Dataset

The Cora dataset is the canonical testbed for GCNs. It contains 2,708 scientific publications connected by 5,429 citations, classified into 7 subject categories. Each publication is represented by a 1,433-word vector (binary bag-of-words).

dataset = Planetoid(root='/tmp/Cora', name='Cora')
data = dataset[0]

print(f"Number of nodes: {data.num_nodes}")
print(f"Number of edges: {data.num_edges}")
print(f"Number of features: {data.num_features}")
print(f"Number of classes: {dataset.num_classes}")

Creating the Semi-Supervised Mask

In the standard semi-supervised configuration, only 20 nodes per class are kept for training, totaling 140 nodes out of 2,708 (barely 5%). The next 500 nodes serve for validation, and the rest for testing.

def create_semi_supervised_mask(data, num_per_class=20, num_val=500):
    """Creates semi-supervised training masks with 20 labels per class."""
    num_classes = dataset.num_classes
    train_mask = torch.zeros(data.num_nodes, dtype=torch.bool)

    # Random selection of num_per_class nodes per class
    for c in range(num_classes):
        indices = torch.where(data.y == c)[0]
        selected = indices[torch.randperm(len(indices))[:num_per_class]]
        train_mask[selected] = True

    # Validation and test masks
    remaining = ~train_mask
    remaining_indices = torch.where(remaining)[0]
    val_mask = torch.zeros(data.num_nodes, dtype=torch.bool)
    val_indices = remaining_indices[torch.randperm(len(remaining_indices))[:num_val]]
    val_mask[val_indices] = True

    test_mask = ~train_mask & ~val_mask

    return train_mask, val_mask, test_mask

train_mask, val_mask, test_mask = create_semi_supervised_mask(data)
print(f"Training nodes: {train_mask.sum().item()}")
print(f"Validation nodes: {val_mask.sum().item()}")
print(f"Test nodes: {test_mask.sum().item()}")

Defining the GCN Model

class SemiSupervisedGCN(torch.nn.Module):
    def __init__(self, num_features, num_classes, hidden_channels=128, dropout=0.5):
        super().__init__()
        self.conv1 = GCNConv(num_features, hidden_channels, cached=True)
        self.conv2 = GCNConv(hidden_channels, num_classes, cached=True)
        self.dropout = dropout

    def forward(self, x, edge_index):
        x = self.conv1(x, edge_index)
        x = F.relu(x)
        x = F.dropout(x, p=self.dropout, training=self.training)
        x = self.conv2(x, edge_index)
        return x

model = SemiSupervisedGCN(
    num_features=dataset.num_features,
    num_classes=dataset.num_classes,
    hidden_channels=128,
    dropout=0.5
).to(device)

data = data.to(device)
print(f"Model: {model}")
print(f"Trainable parameters: {sum(p.numel() for p in model.parameters())}")

Training Loop

optimizer = torch.optim.Adam(
    model.parameters(),
    lr=0.01,
    weight_decay=5e-4
)

def train_epoch(epoch, data, train_mask):
    model.train()
    optimizer.zero_grad()
    out = model(data.x, data.edge_index)
    loss = F.cross_entropy(out[train_mask], data.y[train_mask])
    loss.backward()
    optimizer.step()

    pred = out[train_mask].argmax(dim=1)
    acc = (pred == data.y[train_mask]).sum().item() / train_mask.sum().item()

    if epoch % 20 == 0:
        print(f"Epoch {epoch:03d} | Loss: {loss.item():.4f} | Train accuracy: {acc*100:.1f}%")

    return loss.item(), acc

def test(data, mask, description="test"):
    model.eval()
    out = model(data.x, data.edge_index)
    pred = out[mask].argmax(dim=1)
    correct = (pred == data.y[mask]).sum().item()
    acc = correct / mask.sum().item()
    print(f"  {description}: {acc*100:.1f}% ({correct}/{mask.sum().item()})")
    return acc

# Training for 300 epochs
best_val_acc = 0
best_train_acc = 0

for epoch in range(301):
    _, train_acc = train_epoch(epoch, data, train_mask)

    # Periodic evaluation
    if epoch % 50 == 0 or epoch == 300:
        with torch.no_grad():
            test(data, val_mask, "  Validation")
            test(data, test_mask, "  Test")

    # Save best model
    if train_acc > best_train_acc:
        best_train_acc = train_acc
        best_state = {k: v.clone() for k, v in model.state_dict().items()}

Visualizing Embeddings with t-SNE

After training, we can visualize the representations learned by the last hidden layer (before classification) to visually check the quality of class separation.

def visualize_embeddings(data, save_path='gcn_tsne_cora.png'):
    """Visualizes the embeddings from the last hidden layer with t-SNE."""
    model.eval()

    # Get hidden layer embeddings
    model.eval()
    with torch.no_grad():
        x = model.conv1(data.x, data.edge_index)
        x = F.relu(x)
        embeddings = x.cpu().numpy()

    # Dimensionality reduction with t-SNE
    tsne = TSNE(n_components=2, random_state=42, perplexity=30)
    embeddings_2d = tsne.fit_transform(embeddings)

    class_names = [
        'Case-Based', 'Genetic Algorithms',
        'Neural Networks', 'Probabilistic Methods',
        'Reinforcement Learning', 'Rule Learning',
        'Theory'
    ]

    cmap = plt.get_cmap('tab10')
    plt.figure(figsize=(12, 8))

    for c in range(dataset.num_classes):
        mask = data.y.numpy() == c
        plt.scatter(
            embeddings_2d[mask, 0], embeddings_2d[mask, 1],
            c=[cmap(c/7)], label=class_names,
            s=30, alpha=0.7, edgecolors='white', linewidth=0.5
        )

    plt.title(
        "Semi-Supervised GCN — t-SNE Visualization of Embeddings (Cora)",
        fontsize=14, fontweight='bold'
    )
    plt.legend(bbox_to_anchor=(1.05, 1), loc='upper left', fontsize=9)
    plt.grid(True, alpha=0.3)
    plt.tight_layout()
    plt.savefig(save_path, dpi=150, bbox_inches='tight')
    plt.close()
    print(f"Visualization saved to {save_path}")

visualize_embeddings(data)

Analyzing the Impact of the Number of Labels

A crucial aspect of the semi-supervised GCN is the effect of the number of available labels on performance. Here is a systematic analysis:

def analyze_impact_labels(data, labels_per_class_list=[5, 10, 15, 20, 40, 80]):
    """Compares GCN accuracy according to the number of labels per class."""
    results = []

    for n_labels in labels_per_class_list:
        train_m, val_m, test_m = create_semi_supervised_mask(data, n_labels, 500)

        mdl = SemiSupervisedGCN(
            num_features=dataset.num_features,
            num_classes=dataset.num_classes,
            hidden_channels=128,
            dropout=0.5
        ).to(device)

        opt = torch.optim.Adam(
            mdl.parameters(), lr=0.01, weight_decay=5e-4
        )

        # Accelerated training
        for ep in range(200):
            mdl.train()
            opt.zero_grad()
            out = mdl(data.x, data.edge_index)
            loss = F.cross_entropy(out[train_m], data.y[train_m])
            loss.backward()
            opt.step()

        # Evaluation
        mdl.eval()
        with torch.no_grad():
            out = mdl(data.x, data.edge_index)
            pred = out[test_m].argmax(dim=1)
            acc = (pred == data.y[test_m]).sum().item() / test_m.sum().item()

        results.append((n_labels, acc * 100))
        print(f"  {n_labels} labels/class → {acc*100:.1f}% test accuracy")

    # Plotting the curve
    labels = [r[0] for r in results]
    accs = [r[1] for r in results]

    plt.figure(figsize=(10, 5))
    plt.plot(labels, accs, 'o-', linewidth=2.5, markersize=8, color='#2196F3')
    plt.fill_between(labels, accs, alpha=0.1, color='#2196F3')
    plt.xlabel("Number of labels per class", fontsize=12)
    plt.ylabel("Test accuracy (%)", fontsize=12)
    plt.title(
        "Semi-Supervised GCN — Impact of the Number of Labels on Cora",
        fontsize=14, fontweight='bold'
    )
    plt.grid(True, alpha=0.3)
    plt.xticks(labels)
    plt.ylim(0, 100)

    for x, y in zip(labels, accs):
        plt.annotate(f"{y:.1f}%", (x, y), textcoords="offset points",
                     xytext=(0, 12), ha='center', fontsize=10)

    plt.tight_layout()
    plt.savefig('gcn_impact_labels.png', dpi=150)
    plt.close()
    print("\nImpact curve saved to gcn_impact_labels.png")
    return results

analyze_impact_labels(data)

Hyperparameters of the Semi-Supervised GCN

The choice of hyperparameters is particularly sensitive in the semi-supervised regime, as each parameter directly influences the model’s ability to generalize from very few labeled examples.

hidden_channels

This is the dimension of the latent space of the hidden layer. Values that are too small (16-32) limit the model’s expressive capacity, while values that are too large (256-512) increase the risk of overfitting. For Cora, 128 channels is an excellent starting point — enough to capture the diversity of the 7 subject categories without over-parameterizing.

num_layers

As discussed, 2 layers are nearly universal for semi-supervised GCNs on moderate-sized graphs. The first layer aggregates direct neighbors (1 hop), the second aggregates neighbors of neighbors (2 hops). A third layer can be useful on very large graphs where information needs to travel farther, but with increased regularization to counter over-smoothing.

dropout

Dropout is the primary regularization tool for the GCN. In the semi-supervised regime, it should be high (0.4 to 0.6) because the number of available labels is not enough to constrain the network’s weights. Without sufficient dropout, the model memorizes the few training labels and generalizes very poorly. GAT uses attention coefficients, but the classical GCN relies heavily on dropout to succeed in semi-supervised mode.

learning_rate

The default learning rate of 0.01 with the Adam optimizer works remarkably well for GCNs on Cora. A higher rate (0.05) can help for larger graphs but increases the risk of instability. A lower rate (0.001) gives smoother convergence but requires many more epochs — often unnecessary in semi-supervised mode since the training signal is already weak.

weight_decay

L2 regularization with weight_decay=5e-4 (i.e., 0.0005) controls the norm of the weights, preventing the model from relying excessively on a few dominant weights. This parameter is often underestimated but proves critical: without weight_decay, test accuracy typically drops by 3 to 5 percentage points on Cora.

Advantages and Limits of the Semi-Supervised GCN

Advantages

1. Extreme label efficiency: with only 20 labels per class (140 out of 2,708), the GCN achieves accuracies in the range of 80% on Cora. That is 10 to 50 times fewer labels than a classical classifier would need for comparable results.
2. Natural information propagation: the graph’s structure is not an obstacle to the model — it is the model itself. This contrasts with classical approaches where the relational structure must be “flattened” into additional features.
3. Architectural simplicity: despite its power, a typical semi-supervised GCN has only a few thousand parameters, compared to millions for a classical deep network. This allows fast training (a few seconds on GPU, a few minutes on CPU).
4. Generalization through graph regularization: the GCN implicitly implements Laplacian regularization (the smoothness constraint on the graph), which gives it theoretical robustness in the low-supervision regime.

Limits

1. Inductive vs. transductive: the classical GCN is transductive — it requires knowing the entire graph (including unlabeled nodes) during training. To classify nodes that did not exist during training (new nodes arriving in a dynamic graph), inductive variants such as GraphSAGE or GAT are needed (see references at the end of the article).
2. Sensitivity to over-smoothing: beyond 2-3 layers, node representations tend to converge, destroying the variability necessary for classification. This is a fundamental limitation of iterative neighborhood aggregation.
3. Dependence on graph structure: if the edges are noisy, incomplete, or incorrect, information propagation becomes inefficient or even harmful. The GCN assumes that connected nodes are semantically close — an assumption that is not always verified.
4. Scalability: the original GCN requires the full adjacency matrix in memory. For graphs with hundreds of thousands of nodes, mini-batch approaches (neighborhood sampling) are necessary.

4 Concrete Use Cases

1. Classification of Scientific Articles (Cora, Pubmed, Citeseer)

This is the canonical application. Articles are nodes, citations are edges, and text content (bag-of-words) serves as features. By labeling only 20 articles per subject category, the GCN classifies the entire bibliography. In research, this allows rapid exploration of corpora of thousands of articles without exhaustive manual annotation.

2. Fraud Detection in Transaction Networks

Bank accounts form the nodes of a graph, and transactions form the edges. By manually labeling a few dozen fraudulent accounts (identified by investigators), the semi-supervised GCN propagates this information and identifies suspicious accounts connected to known fraudsters. This approach is deployed by several large banks to detect sophisticated fraud networks.

3. Gene Annotation in Protein Networks

In bioinformatics, proteins are connected by known physical interactions. The biological function of certain genes is well documented, but many remain unknown. The semi-supervised GCN uses protein-protein interactions as the graph structure and the biochemical characteristics of proteins as features to predict the function of unannotated genes. This approach has made it possible to annotate thousands of genes with reliability comparable to direct biological experimentation.

4. Product Recommendation with Prior Knowledge

In a recommendation graph, customers and products are nodes, and purchases connect customers and products. By knowing the preferences of a few customers (via a questionnaire, market research), the semi-supervised GCN can classify the preferences of other customers — not from massive purchase histories, but by exploiting behavioral similarity through the cross-purchase network.

See Also

• Finding the Largest Zero Submatrix in Python
• Master Python: Solving the ‘Birds on a Wire’ Problem Efficiently