Siamese Networks: Complete Guide — Similarity through Twin Networks

Summary — Siamese Networks are deep learning architectures designed to compare pairs of inputs and learn a similarity measure. Introduced by Bromley et al. in 1993 for signature verification, they consist of two identical subnetworks (sharing the same weights) that transform each input into an embedding, then compare these embeddings via a distance function. The key advantage: instead of classifying N categories, the network learns to say “similar or different,” which works even for classes never seen during training.

Mathematical Principle

1. Shared-Weight Architecture

A Siamese network is composed of two identical branches f_θ:

embed_1 = f_θ(x1)
embed_2 = f_θ(x2)  # Same parameters θ

The two branches share exactly the same weights: they are just a single network applied twice.

2. Distance Function

The similarity between x1 and x2 is measured by the Euclidean distance between their embeddings:

d(x1, x2) = ||f_θ(x1) - f_θ(x2)||²

3. Contrastive Loss

The contrastive loss function pushes similar pairs to be close and dissimilar pairs to be far apart:

L = y · d²
  + (1-y) · max(0, m - d)²

Where y = 1 if similar and y = 0 if dissimilar, and m is the margin (minimum desired distance between different pairs). The term max(0, m – d)² means: if the distance already exceeds the margin, no penalty. Otherwise, we apply a penalty.

4. Triplet Loss

A popular alternative: triplet loss takes a triplet (anchor, positive, negative):

L = max(0, ||f(a) - f(p)||² - ||f(a) - f(n)||² + m)

The anchor and positive are similar (same class), the negative is different. The network learns to make the anchor closer to the positive than to the negative, with a separation margin m.

5. One-shot and few-shot learning

The most powerful application of Siamese networks: recognition with a single example. If you have only one reference photo of each person, you compare the new photo to all references via the distance of the embeddings:

class(x_new) = argmin_reference d(f(x_new), f(reference))

This works for classes never seen during training, because the similarity function is generic, not class-specific.

Intuition

A Siamese network is like a twin comparing two faces. It doesn’t specifically recognize “this is Pierre” or “this is Marie.” It just asks: “are these two faces the same, yes or no?”

That’s why it’s so powerful for recognition with a single example: you don’t need thousands of photos of each person, just one reference photo and you compare. It’s the difference between memorizing a phone book (classic classification) and knowing how to compare two people (Siamese network).

Python Implementation

1. Siamese Architecture with PyTorch

import torch
import torch.nn as nn
import torch.nn.functional as F


class SiameseNetwork(nn.Module):
    """Siamese network with two branches sharing the same weights."""
    def __init__(self, input_dim=784, embed_dim=128):
        super().__init__()
        self.encoder = nn.Sequential(
            nn.Linear(input_dim, 256), nn.ReLU(),
            nn.Linear(256, 128), nn.ReLU(),
            nn.Linear(128, embed_dim)
        )

    def forward_once(self, x):
        return self.encoder(x)

    def forward(self, x1, x2):
        """Returns the embeddings of both inputs."""
        return self.forward_once(x1), self.forward_once(x2)


def contrastive_loss(embed1, embed2, label, margin=1.0):
    """Contrastive: label=1 if similar, 0 if different."""
    dist = torch.sum((embed1 - embed2) ** 2, dim=1)
    loss_pos = label * dist
    loss_neg = (1 - label) * torch.max(
        torch.zeros_like(dist),
        (margin - torch.sqrt(dist + 1e-8)) ** 2
    )
    return (loss_pos + loss_neg).mean()


def triplet_loss(anchor, positive, negative, margin=0.3):
    """Triplet loss: anchor, positive, negative."""
    dist_pos = torch.sum((anchor - positive) ** 2, dim=1)
    dist_neg = torch.sum((anchor - negative) ** 2, dim=1)
    return torch.mean(torch.clamp(dist_pos - dist_neg + margin, min=0.0))


# Creation and training
model = SiameseNetwork(input_dim=784, embed_dim=64)
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)

for epoch in range(20):
    model.train()
    total_loss = 0
    for x1, x2, labels in train_loader:
        embed1, embed2 = model(x1.view(x1.size(0), -1), x2.view(x2.size(0), -1))
        loss = contrastive_loss(embed1, embed2, labels, margin=1.0)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        total_loss += loss.item()
    avg = total_loss / len(train_loader)
    print(f'Epoch {epoch} | Loss: {avg:.4f}')

2. Similarity Search

def find_similar(query, database, model, k=5):
    """Finds the k most similar items in the database."""
    model.eval()
    with torch.no_grad():
        q_embed = model.forward_once(query.view(1, -1))
        d_embeds = model.forward_once(database)
    distances = torch.sum((d_embeds - q_embed) ** 2, dim=1)
    _, indices = torch.topk(distances, k, largest=False)
    return indices.numpy()


# One-shot learning: test on a new subject
def one_shot_test(support_img, query_img, distractors, model):
    """The model chooses the matching image from the distractors."""
    candidates = torch.cat([query_img.unsqueeze(0), distractors], dim=0)
    closest = find_similar(support_img, candidates, model, k=1)
    return (closest[0] == 0)  # The first one is the correct answer

Hyperparameters

Variants and Extensions

Siamese Networks with CNNs

For images, the MLP is replaced with a convolutional encoder (several convolutions + pooling). Each branch is a CNN that produces a compact image embedding. A pretrained VGG or ResNet architecture is often used as the backbone.

Prototypical Networks

An extension of the Siamese concept: instead of comparing to a single reference example, the prototype (mean of embeddings) of each class in the support set is computed. Classification is done by distance to the nearest prototype.

Matching Networks

These use an attention mechanism to weight the similarities between the query and each element of the support set. More sophisticated than simple Euclidean distance, especially for few-shot learning with multiple examples per class.

Advantages

1. One-shot learning: Recognizes unseen classes with a single reference example.
2. Class-independent: The similarity function is generic, not tied to a fixed set of categories.
3. Easy to add/remove: Adding a new subject = one reference photo. No retraining needed.
4. Broad applicability: Images, text, audio, signatures — as long as you can construct pairs, it works.

Limitations

1. Expensive inference: Comparing a new input to all references = O(N) embeddings and distances.
2. Lower peak performance: On a standard classification problem with many examples, a classic classifier often beats a Siamese network.
3. Pair quality is critical: The pair dataset must be well-balanced and representative.

4 Concrete Use Cases

1. Bank Signature Verification

The historical application. Banks compare a customer’s signature on a check with their reference. The Siamese network detects forgeries even when forgers imitate skillfully.

2. Facial Recognition on Smartphones

Apple’s FaceID uses a similarity principle: the stored facial imprint (from setup) is compared to the face capture. If the distance between embeddings is small enough, the phone unlocks.

3. Visual Similarity Image Search

Google Images “search by image”: you provide a reference image and the Siamese network finds the most visually similar images in an index of billions of photos.

4. Plagiarism Detection

Compare texts by encoding them via a Siamese network (with word embeddings as inputs) to detect semantic similarities even when the text has been paraphrased.

5. Product Recommendation

A Siamese network can learn to compare the preferences of two users or the similarity between two products. If user A likes product X, the network recommends product Y that is most similar to X in the learned embedding space.

6. Entity Reconciliation in Databases

When two databases contain records about the same entities (customers, companies) but in different formats, a Siamese network learns to determine whether two records refer to the same real entity despite variations in spelling.

Conclusion

Siamese networks paved the way for learning similarity through neural networks. Their fundamental principle — two weight-sharing branches to compare pairs — is now a standard component of deep learning.

Modern extensions (triplet networks, contrastive pre-training for large language models, prototypical networks) all build on the same central idea: learning a representation space where distance reflects semantic similarity.

See Also

• Mastering Pandigital Concatenated Products with Python: Complete Guide for Developers in 2024
• Creating Unique Sum Subsets in Python: Complete Guide and Advanced Tips