Siamese Network

The Siamese Network is a deep learning architecture designed to learn how to compare two inputs by measuring their degree of similarity. Unlike conventional neural networks that classify a single input, the Siamese network evaluates the relationship between two samples. This revolutionary approach, introduced by Bromley et al. in 1993 for handwritten signature verification, is today at the heart of facial recognition, biometrics, and recommendation systems.

Summary

A Siamese Network consists of two identical sub-networks sharing exactly the same weights. Each sub-network transforms an input into a compact representation called an embedding. The distance between these two embeddings determines whether the inputs are similar or different. Learning relies on specialized loss functions — notably contrastive loss and triplet loss — that penalize bad distances and reward good associations. This architecture can generalize to classes never seen during training, making it particularly well-suited to situations where certain categories have only a few examples.

Mathematical Principle

Twin-Branch Architecture

The heart of the Siamese Network rests on an elegant idea: two strictly identical sub-networks, denoted f(x₁) and f(x₂), share all their parameters (weights and biases). These two branches each process a different input — for example, two images — and produce two embedding vectors of fixed dimension.

We denote:

• f(x₁) = embedding of the first input, obtained after forward propagation through the shared network
• f(x₂) = embedding of the second input, computed with exactly the same weights
• D = ||f(x₁) − f(x₂)|| = Euclidean distance between the two embeddings
• y = binary label: 1 if the two inputs belong to the same class (positive pair), 0 otherwise (negative pair)

The fundamental principle is that the network learns to bring embeddings of positive pairs closer together (y = 1) and push embeddings of negative pairs apart (y = 0), up to a predefined margin. This strategy contrasts with traditional classification where the network learns decision boundaries between fixed classes.

Contrastive Loss

The contrastive loss function, introduced by Hadsell, Chopra, and LeCun in 2006, is the historical objective function for training Siamese networks:

L = y · D² + (1 − y) · max(0, m − D)²

Where:

• y = 1 (positive pair): the active term is D². The network minimizes the distance between similar embeddings. The ideal target is D = 0.
• y = 0 (negative pair): the active term is max(0, m − D)². If the distance D is less than the margin m, the network incurs a penalty proportional to the square of the gap. Once D ≥ m, the penalty vanishes.
• m (margin) is a hyperparameter that controls the desired minimum distance between negative pairs.

Intuitively, the contrastive loss tells the network: “Bring similar samples as close together as possible, but push different samples apart by at least m.” This is a metric distance learning mechanism rather than classical classification.

Triplet Loss

The triplet loss, popularized by Google FaceNet in 2015, is a powerful evolution of the contrastive loss. Instead of working on pairs, it uses triplets of three samples:

• a (anchor): a reference example
• p (positive): another example of the same class as the anchor
• n (negative): an example of a different class

The loss function is written:

L = max(0, ||f(a) − f(p)||² − ||f(a) − f(n)||² + margin)

This formulation requires that the distance between the anchor and the positive be strictly less than the distance between the anchor and the negative, with a minimum gap equal to the margin. In other words, not only must positive pairs be closer than negative pairs, but there must be a “safety gap” between the two distances.

The triplet loss has two major advantages over the contrastive loss. First, it directly compares positive and negative distances within the same cost term, producing more informative gradients at each learning step. Second, the margin parameter plays a more natural role: it’s not about pushing negatives to an absolute distance, but about guaranteeing a relative gap between positive and negative.

The main drawback of the triplet loss is the difficulty of selecting effective triplets. An “easy” triplet, where the anchor is already much closer to the positive than to the negative, produces no gradient (the max function saturates at zero). In practice, triplet mining strategies are used, notably semi-hard triplets, where the negative is closer to the anchor than the positive but the margin has not yet been reached. These triplets produce the most useful gradients.

Intuition

To truly understand the Siamese Network, you need to shift your perspective. A conventional classifier learns the answer to the question: “Which category does this image belong to?” The Siamese network, on the other hand, answers a different question: “Do these two images belong to the same category?”

This is a fundamental distinction. Imagine you had to recognize your friend’s face in a crowd. Individual recognition requires having memorized every face you know. But determining whether two photos show the same person is a different problem: you compare features, proportions, characteristics, without necessarily knowing who that person is.

The Siamese network learns exactly this comparison capability rather than identification. It extracts discriminating features (eye shape, bone structure, facial proportions) and encodes them in a vector space where geometric distance reflects real similarity. Two faces of the same individual will produce two embeddings close in this space, regardless of lighting, angle, or expression.

This approach provides a decisive advantage: the network can compare identities never encountered during training. You simply pass the new images through the two shared branches and measure the distance between their embeddings. No retraining is needed to add new individuals to the system. This is called one-shot or few-shot learning — recognizing a new class with only one or very few examples.

Python Implementation with Keras

Here is a complete implementation of a Siamese Network for facial recognition, built with Keras and TensorFlow.

Shared Base Network

import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers, models, optimizers
from tensorflow.keras.models import Model, Sequential
from tensorflow.keras.layers import (
    Conv2D, MaxPooling2D, Dense, Dropout,
    Flatten, Input, Lambda, BatchNormalization
)
from tensorflow.keras.optimizers import Adam
import matplotlib.pyplot as plt
import os

def create_base_network(input_shape):
    """
    Creates the shared sub-network (the "twin network").
    Two instances of this network share the same weights.
    Uses a classical CNN architecture with batch normalization.
    """
    model = Sequential([
        Conv2D(64, (3, 3), input_shape=input_shape,
               activation='relu', padding='same'),
        BatchNormalization(),
        MaxPooling2D((2, 2)),

        Conv2D(128, (3, 3), activation='relu', padding='same'),
        BatchNormalization(),
        MaxPooling2D((2, 2)),

        Conv2D(256, (3, 3), activation='relu', padding='same'),
        BatchNormalization(),
        MaxPooling2D((2, 2)),

        Flatten(),
        Dense(512, activation='relu'),
        Dropout(0.5),
        Dense(256, activation='relu'),
        Dropout(0.4),
        Dense(128, activation='relu'),
        # Final embedding: 128 dimensions
    ])

    return model

Building the Siamese Model

def create_siamese_network(input_shape=(105, 105, 1)):
    """
    Builds the complete Siamese architecture:
    two branches sharing the same weights,
    plus a Euclidean distance layer.
    """
    base_network = create_base_network(input_shape)

    # Two inputs (pair of images)
    input_a = Input(shape=input_shape, name='entree_gauche')
    input_b = Input(shape=input_shape, name='entree_droite')

    # Both inputs pass through the same network (shared weights)
    embedding_a = base_network(input_a)
    embedding_b = base_network(input_b)

    # Custom Euclidean distance layer
    def euclidean_distance(vects):
        x, y = vects
        return tf.sqrt(tf.reduce_sum(tf.square(x - y), axis=1, keepdims=True))

    def euclidean_dist_output_shape(shapes):
        shape1, shape2 = shapes
        return (shape1[0], 1)

    distance = Lambda(euclidean_distance,
                      output_shape=euclidean_dist_output_shape,
                      name='distance_euclidienne')([embedding_a, embedding_b])

    model = Model(inputs=[input_a, input_b], outputs=distance)
    return model

Custom Contrastive Loss

def contrastive_loss(y_true, y_pred, margin=1.0):
    """
    Contrastive Loss with margin.

    y_true: 1 if similar, 0 if different
    y_pred: predicted Euclidean distance
    margin: minimum distance for negative pairs
    """
    square_pred = tf.square(y_pred)
    margin_square = tf.square(tf.maximum(margin - y_pred, 0.0))

    loss = tf.reduce_mean(
        y_true * square_pred + (1.0 - y_true) * margin_square
    )
    return loss


def accuracy_siamese(y_true, y_pred, threshold=0.5):
    """
    Binary similarity metric.
    Predicts 1 if distance < threshold, 0 otherwise.
    """
    return tf.reduce_mean(
        tf.cast(
            tf.equal(y_true, tf.cast(y_pred < threshold, tf.float32)),
            tf.float32
        )
    )

Generating Pairs for Training

def generer_paires(X, y):
    """
    Generates positive and negative pairs from
    a dataset of images and labels.

    For each example, it creates:
    - 1 positive pair: same class, different image
    - 1 negative pair: different class, random image
    """
    indices_par_classe = {}
    for idx, label in enumerate(y):
        indices_par_classe.setdefault(label, []).append(idx)

    paires_gauche = []
    paires_droite = []
    etiquettes = []

    classes_disponibles = list(indices_par_classe.keys())

    for i in range(len(X)):
        classe_i = y[i]
        indices_meme_classe = indices_par_classe[classe_i]

        # 1. Positive pair: same class, different image
        if len(indices_meme_classe) > 1:
            idx_pos = np.random.choice([j for j in indices_meme_classe if j != i])
            paires_gauche.append(X[i])
            paires_droite.append(X[idx_pos])
            etiquettes.append(1)

        # 2. Negative pair: different class
        autres_classes = 
        classe_diff = np.random.choice(autres_classes)
        idx_neg = np.random.choice(indices_par_classe[classe_diff])
        paires_gauche.append(X[i])
        paires_droite.append(X[idx_neg])
        etiquettes.append(0)

    return (np.array(paires_gauche), np.array(paires_droite),
            np.array(etiquettes, dtype=np.float32))

Training and Evaluation

def entrainer_modele_siamese(X_train, y_train, X_test, y_test,
                              epochs=20, batch_size=128, margin=1.0):
    """
    Complete training pipeline for a Siamese Network
    for facial recognition or verification.
    """
    input_shape = X_train.shape[1:]

    # Model creation
    siamese = create_siamese_network(input_shape)

    # Compilation with contrastive loss
    siamese.compile(
        optimizer=Adam(learning_rate=0.001),
        loss=lambda y_t, y_p: contrastive_loss(y_t, y_p, margin),
        metrics=[accuracy_siamese]
    )

    # Architecture summary
    siamese.summary()

    # Generate training pairs
    X_left_train, X_right_train, labels_train = generer_paires(X_train, y_train)
    X_left_test, X_right_test, labels_test = generer_paires(X_test, y_test)

    print(f"Paires d'entraînement : {len(X_left_train)}")
    print(f"Paires de test : {len(X_left_test)}")

    # Training
    historique = siamese.fit(
        [X_left_train, X_right_train], labels_train,
        validation_data=(
            [X_left_test, X_right_test], labels_test
        ),
        batch_size=batch_size,
        epochs=epochs,
        verbose=1
    )

    # Plot learning curves
    plt.figure(figsize=(12, 4))

    plt.subplot(1, 2, 1)
    plt.plot(historique.history['loss'], label='Entraînement')
    plt.plot(historique.history['val_loss'], label='Validation')
    plt.title('Contrastive Loss')
    plt.xlabel('Époques')
    plt.ylabel('Perte')
    plt.legend()

    plt.subplot(1, 2, 2)
    plt.plot(historique.history['accuracy_siamese'], label='Entraînement')
    plt.plot(historique.history['val_accuracy_siamese'], label='Validation')
    plt.title('Précision de Similarité')
    plt.xlabel('Époques')
    plt.ylabel('Précision')
    plt.legend()

    plt.tight_layout()
    plt.savefig('historique_siamese.png', dpi=150)
    plt.show()

    return siamese, historique


# --- Usage example ---
# Application: facial recognition on a face dataset
# (omniglot and LFW are classical benchmarks)

# Load data (example with a fictional dataset)
# X_train, y_train = charger_dataset_visages()
# X_train = X_train.astype('float32') / 255.0
# X_train = np.expand_dims(X_train, axis=-1)

# siamese, history = entrainer_modele_siamese(X_train, y_train,
# X_test, y_test,
# epochs=20,
# batch_size=128,
# margin=1.0)

Inference and Comparison Functions

def comparer_images(siamese_model, img1, img2, seuil=0.5):
    """
    Compares two images and returns the distance
    and a similarity decision.
    """
    # Pass through both branches
    base_net = siamese_model.layers[2]  # The shared network

    emb1 = base_net(np.expand_dims(img1, axis=0))
    emb2 = base_net(np.expand_dims(img2, axis=0))

    # Compute Euclidean distance
    dist = np.linalg.norm(emb1 - emb2)

    similaire = dist < seuil
    return dist, similaire


def verification_signature(siamese_model, signature_ref, signature_test,
                           seuil=0.3):
    """
    Handwritten signature verification:
    historical application of Siamese networks.
    Compares a reference signature with
    a submitted signature to detect forgeries.
    """
    distance, est_signee = comparer_images(
        siamese_model, signature_ref, signature_test, seuil
    )
    return {
        'distance': distance,
        'authentique': bool(est_signee),
        'confiance': 1.0 - distance
    }

Siamese Network Hyperparameters

The performance of a Siamese Network depends heavily on hyperparameter tuning. Here are the most critical ones:

Margin

The margin controls the minimum distance between negative pairs. It is the most sensitive hyperparameter.

• Margin too low (0.2 – 0.3): negative pairs are not sufficiently separated, creating confusion between different classes.
• Optimal margin (0.8 – 1.5): good balance between bringing positives together and pushing negatives apart. The value 1.0 is an excellent starting point.
• Margin too high (> 2.0): the network may saturate, producing zero gradients for negative pairs as soon as the distance exceeds the margin, which slows learning.

Embedding Dimension

The dimension of the embedding vector determines the richness of the representation:

• Low dimension (16 – 32): fast but risks under-representation. Two different images may accidentally produce close embeddings (collision).
• Standard dimension (64 – 256): good compromise for most applications. 128 dimensions is a frequent and effective choice.
• High dimension (512 – 1024): necessary for complex tasks like high-precision facial recognition (FaceNet uses 128D, ArcFace uses 512D). But increases the risk of overfitting and computation time.

Sub-Network Architecture

The choice of internal architecture for the shared branch determines the quality of extracted features:

• Simple CNN (2-3 convolutional layers): sufficient for simple images like MNIST digits or Omniglot characters.
• Deep CNN (ResNet-18, VGG-16, EfficientNet): necessary for faces or complex images. Transfer learning from a network pre-trained on ImageNet can considerably accelerate convergence.
• Attention / Transformers: for non-visual data (text, sequences), attention-based architectures better capture long-range relationships.

Batch Size for Pairs

Unlike a conventional network, the Siamese Network processes pairs, meaning the effective batch size depends on the number of pairs generated:

• Batch size = 64 to 256 pairs is generally appropriate.
• Larger batches provide more stable gradients but require more GPU memory.
• It is crucial to maintain a balanced 1:1 ratio of positive to negative pairs in each batch to avoid bias toward one type of pair.

Advantages of the Siamese Network

The Siamese Network has several decisive advantages over traditional classification architectures.

The first advantage is one-shot and few-shot learning. A Siamese network can recognize new classes with a single reference example, without any retraining. This is essential for biometric applications where new users are frequently added to the system.

The second advantage is generalization to unseen classes. The model learns a universal distance function, not specific class boundaries. It is capable of comparing pairs of individuals it has never observed during training.

The third advantage is data efficiency. By generating positive and negative pairs from available data, the training set size is artificially multiplied without collecting new data. N images can produce up to N² pairs.

The fourth advantage is independence from the number of classes. A conventional classifier must modify its output layer for each new class. The Siamese Network has no class-dependent output layer — its output is always a scalar distance.

The fifth advantage is robustness to imbalances. Even when certain classes have very few samples, negative pairs can be generated in abundance, maintaining a stable learning signal.

Limits and Challenges

Despite its advantages, the Siamese Network is not a universal solution and has several important limitations.

Pair generation is the main challenge. For a dataset of N images, the number of possible pairs is approximately N²/2. Most of these pairs are negative and therefore “easy” — they don’t produce useful gradients. Intelligent sampling is needed to select the most informative pairs.

Choice of decision threshold is often arbitrary. Unlike a classifier that directly produces a probability per class, the Siamese Network returns a distance. Determining the optimal threshold to distinguish similar from non-similar requires calibration on a validation set, and this threshold may vary depending on classes and acquisition conditions.

Collapse risk is a serious problem: the network may learn to produce the same embedding for all inputs, making the distance always zero. This phenomenon is more frequent with contrastive loss than with triplet loss. Techniques like embedding normalization or data augmentation are essential to avoid it.

Dependence on data quality is strong. Poorly aligned images, uncompensated lighting variations, or noise in pair labels can significantly degrade performance. Data preprocessing is often as important as the architecture itself.

Finally, training time can be significant. Unlike a standard network that processes one image at a time, the Siamese Network must pass two images at each step, doubling the computational cost of forward propagation.

4 Concrete Use Cases

1. Handwritten Signature Verification (Historical Application)

The original application of the Siamese Network, developed by Bromley et al. for AT&T in 1993, involved verifying the authenticity of handwritten signatures on bank checks. The network compares a reference signature (the account holder’s) with a submitted signature and determines whether it is the same person or a forger. This application remains relevant in digital banking and legal verification systems.

2. Facial Recognition and Verification

This is the most well-known application among the general public. Google’s FaceNet uses a Siamese Network with triplet loss to encode faces in a 128-dimensional space. The distance between two embeddings determines whether they belong to the same individual. This technology powers smartphone facial unlock systems, automated border controls, and intelligent surveillance systems. Apple FaceID uses a similar architecture with 3D depth sensors.

3. Visual Product Search by Similarity

E-commerce platforms use Siamese networks to allow customers to search for similar products from an image. The user photographs a garment they saw on the street, and the system finds visually similar items in the catalog. The Siamese network encodes each product in a similarity space where Euclidean distance reflects visual resemblance. This approach is massively used by Amazon, Pinterest, and Alibaba.

4. Plagiarism Detection and Document Similarity

In natural language processing, variants of the Siamese Network are used to compare paragraphs, articles, or student papers. By encoding texts into embeddings (via transformers like BERT in the Siamese network branches), the semantic distance between documents is measured. This approach detects plagiarism even when the text has been rephrased, because semantic similarity is preserved even if the exact words differ. Tools like Turnitin and Compilatio use similar principles for their similarity analyses.

See Also

• Calculating the Length of the Union of Segments in Python: Complete Guide and Optimized Code
• Discover the Secrets of the Pandigital Prime with Python: Complete Guide and Programming Tips