Adversarial Training (FGSM/PGD): Robustness of Neural Networks against Perturbation Attacks

Summary

Adversarial training is a fundamental technique in deep learning aimed at strengthening the robustness of models against adversarial examples — inputs intentionally perturbed to fool neural networks. This method, introduced by Goodfellow et al. in 2014, relies on a simple but powerfully effective principle: directly integrating attacked examples into the model’s training process, much like a vaccine that exposes the immune system to weakened pathogens to prepare it for real infections.

The two most emblematic attacks used in this context are FGSM (Fast Gradient Sign Method) and PGD (Projected Gradient Descent). FGSM performs a perturbation in a single step by following the sign of the gradient of the loss function with respect to the input. PGD, on the other hand, generalizes this approach by iterating the process several times with smaller steps, projecting the result into a constraint ball at each iteration. Adversarial training combines these attacks with model parameter optimization to produce a classifier capable of resisting deliberate manipulation attempts and maintaining reliable performance on corrupted data.

Mathematical Principle

The FGSM Attack (Fast Gradient Sign Method)

The core idea of FGSM, proposed by Ian Goodfellow and his collaborators in 2014, is to exploit the gradient of the loss function with respect to the input data — not with respect to the model parameters as in classical gradient descent. The principle can be summarized in an elegant formula:

$$x_{\text{adv}} = x + \varepsilon \cdot \text{sign}(\nabla_x L(f_\theta(x), y))$$

where:

• $x$ represents the original image or input;
• $\varepsilon$ (epsilon) is a small scalar controlling the amplitude of the perturbation;
• $\nabla_x L(f_\theta(x), y)$ is the gradient of the loss function $L$ with respect to the input $x$;
• $\text{sign}(\cdot)$ denotes the sign function, applied element-wise;
• $x_{\text{adv}}$ is the resulting adversarial example, visually nearly identical to $x$.

The intuition behind this formula is remarkably intuitive: instead of modifying the network’s weights to reduce the loss (as classical gradient descent does), we modify the input in the direction that maximizes the loss, but with a small enough amplitude to remain imperceptible to the human eye. This directional perturbation applied a single time constitutes the fastest and simplest attack in the adversarial attack landscape, hence its name “Fast Gradient Sign Method.”

The PGD Attack (Projected Gradient Descent)

Madry et al. (2017) demonstrated that FGSM, while effective and very fast, is only a first-order attack (a single step). To obtain truly powerful and robust adversaries, they proposed PGD, which generalizes FGSM by iterating the adversarial update several times with a reduced step size, while projecting the result into a ball centered on the original input at each iteration:

$$x^{(t+1)} = \Pi_{[x-\varepsilon, x+\varepsilon]}(x^{(t)} + \alpha \cdot \text{sign}(\nabla_x L(f_\theta(x^{(t)}), y)))$$

where:

• $x^{(0)}$ is initialized either to $x$ (zero-perturbation attack) or to $x$ with uniform random noise added (recommended for a more aggressive attack);
• $\alpha$ represents the step size at each iteration, typically set to $\alpha = \varepsilon / K$;
• $K$ is the total number of iterations (typically 10, 20, or 40);
• $\Pi_{[x-\varepsilon, x+\varepsilon]}(\cdot)$ is the projection operator onto the $\ell_\infty$ ball of radius $\varepsilon$ centered on $x$, i.e., $\max(x-\varepsilon, \min(x^{(t)} + \alpha \cdot \text{sign}(\nabla), x+\varepsilon))$.

PGD is considered the reference attack in the machine learning literature. Indeed, Madry and his co-authors described it as the “strongest first-order attack” under the $\ell_\infty$ norm constraint, due to its conceptual simplicity and remarkable practical effectiveness.

Formulation of Adversarial Training

Adversarial training is formally expressed as a minimax optimization problem, where the attacker seeks to maximize the loss while respecting a perturbation constraint, while the defender (the model) seeks to minimize this maximum loss:

$$\min_\theta \; \mathbb{E}{(x, y) \sim \mathcal{D}}\left[ \max{|\delta|\infty \leq \varepsilon} L(f\theta(x + \delta), y) \right]$$

In this formulation:

• $\theta$ denotes the model parameters (weights and biases of the neural network);
• $\mathcal{D}$ is the training data distribution;
• $\delta$ is the adversarial perturbation applied to the input, constrained by an $\ell_\infty$ norm bounded by $\varepsilon$;
• The maximum operator ($\max$) represents the inner attack: for each example $(x, y)$, we seek the most harmful perturbation $\delta$ possible within the allowed ball;
• The minimum operator ($\min$) represents the outer optimization: we adjust the parameters $\theta$ to minimize the loss on the worst adversarial cases.

In practice, the inner problem (maximization with respect to $\delta$) is approximately solved by PGD with $K$ steps, while the outer problem (minimization with respect to $\theta$) is handled by a standard optimizer such as SGD or Adam. This bi-level approach truly constitutes the foundation of robustness in modern deep learning.

Intuition

Imagine a neural network as a student learning to recognize handwritten digits from images. With classical training, the student only sees clean, well-formed examples. But what happens if, during the exam, a nearly invisible noise is added to the images? The unprepared student could easily be fooled and classify a “7” as a “9.”

Adversarial training is like deliberately injecting this noise during study sessions. We generate altered versions of the training images — versions where an imperceptible perturbation has been added in a calculated way to fool the model — and we force the network to learn to classify them correctly despite everything. It’s the exact principle of vaccination: we expose the system to controlled threats so it develops robust defenses.

More concretely, FGSM calculates the exact direction in pixel space that, if the image is perturbed in that direction, maximizes the model’s error. PGD refines this strategy by taking several small steps in that direction, recalculating the gradient at each step to adapt to the changes in the model (the gradient evolves as the input is modified). Adversarial training therefore uses these attacks not to break the model, but to make it stronger.

There is a fundamental trade-off — an inevitable trade-off — between accuracy on clean data and robustness against attacks. In general, the more robust a model is to adversarial examples, the more its accuracy on unperturbed data tends to slightly decrease. This is a trade-off that practitioners must carefully manage depending on the usage context.

Python Implementation (PyTorch)

Here is a complete, commented implementation in Python with PyTorch, covering the FGSM attack, the PGD attack, and the adversarial training loop.

FGSM Attack

import torch
import torch.nn as nn

def fgsm_attack(model, images, labels, epsilon, loss_fn=nn.CrossEntropyLoss()):
    """Generate adversarial examples using Fast Gradient Sign Method."""
    images_adv = images.clone().detach().requires_grad_(True)
    outputs = model(images_adv)
    loss = loss_fn(outputs, labels)
    model.zero_grad()
    loss.backward()
    data_grad = images_adv.grad.data.sign()
    perturbation = epsilon * data_grad
    adversarial_images = images + perturbation
    adversarial_images = torch.clamp(adversarial_images, 0, 1)
    return adversarial_images

PGD Attack

def pgd_attack(model, images, labels, epsilon, alpha, num_steps,
               loss_fn=nn.CrossEntropyLoss(), random_init=True):
    """Generate adversarial examples using Projected Gradient Descent."""
    if random_init:
        adversarial_images = images.clone().detach() + \
            torch.randn_like(images) * epsilon
        adversarial_images = torch.clamp(adversarial_images, 0, 1).detach()
    else:
        adversarial_images = images.clone().detach()

    adversarial_images.requires_grad_(True)

    for step in range(num_steps):
        outputs = model(adversarial_images)
        loss = loss_fn(outputs, labels)
        model.zero_grad()
        loss.backward()

        grad_sign = adversarial_images.grad.data.sign()
        adversarial_images = adversarial_images.detach() + alpha * grad_sign

        delta = adversarial_images - images
        delta = torch.clamp(delta, -epsilon, epsilon)
        adversarial_images = torch.clamp(images + delta, 0, 1).detach()
        adversarial_images.requires_grad_(True)

    return adversarial_images

Adversarial Training Loop

def adversarial_training(model, train_loader, optimizer, num_epochs,
                         epsilon=0.03, alpha=0.007, pgd_steps=10,
                         device='cuda', use_pgd=True):
    """Adversarial training loop using FGSM or PGD."""
    model.to(device)
    model.train()
    criterion = nn.CrossEntropyLoss()

    for epoch in range(num_epochs):
        running_loss = 0.0
        correct_adv = 0
        total = 0

        for batch_images, batch_labels in train_loader:
            batch_images = batch_images.to(device)
            batch_labels = batch_labels.to(device)

            if use_pgd:
                adv_images = pgd_attack(
                    model, batch_images, batch_labels,
                    epsilon=epsilon, alpha=alpha,
                    num_steps=pgd_steps, random_init=True
                )
            else:
                adv_images = fgsm_attack(
                    model, batch_images, batch_labels, epsilon=epsilon
                )

            optimizer.zero_grad()
            adv_outputs = model(adv_images)
            loss = criterion(adv_outputs, batch_labels)
            loss.backward()
            optimizer.step()

            running_loss += loss.item()
            _, predicted = adv_outputs.max(1)
            total += batch_labels.size(0)
            correct_adv += predicted.eq(batch_labels).sum().item()

        avg_loss = running_loss / len(train_loader)
        adv_accuracy = 100.0 * correct_adv / total

        print(f"Epoch {epoch+1}/{num_epochs} | "
              f"Loss: {avg_loss:.4f} | "
              f"Adv Accuracy: {adv_accuracy:.2f}%")

    return model

Key Hyperparameters

The success of adversarial training crucially depends on the choice of the following hyperparameters:

Practical recommendation: For an optimal balance between robustness and accuracy, use PGD with ε = 0.03, α = 0.003 (i.e., ε/10), K = 10 steps, and random initialization. This is the standard setting used in the majority of contemporary research papers and robustness benchmarks.

Advantages and Limitations

Advantages

1. Empirically proven robustness: Adversarial training is the only method that has demonstrated significant resistance against the most sophisticated adaptive attacks. Models trained this way maintain satisfactory performance even under strong PGD attacks, where standard models collapse completely.
2. Conceptual simplicity: The fundamental principle is remarkably elegant — you just need to generate perturbed examples and train the model on them. No specialized architecture or complex structural modifications are required, making the method universally applicable.
3. Applicable to any architecture: The method works with any differentiable model — from convolutional networks (CNNs) to transformers, including recurrent neural networks (RNNs) and graph neural networks (GNNs). The only requirement is the ability to compute the gradient with respect to the input, which is universally available in modern frameworks.
4. Improved generalization: Surprisingly, adversarial training sometimes acts as a form of regularization, slightly improving the model’s generalization on unseen data, beyond just robustness to attacks.

Limitations

1. Very high computational cost: Adversarial training with PGD multiplies training time by a factor of 10 to 40 (or more) compared to standard training, because each example requires K additional forward passes to generate the perturbation. On large datasets like ImageNet, this cost can become prohibitive.
2. Accuracy-robustness trade-off: The improvement in robustness is almost always accompanied by a slight decrease in accuracy on clean, unperturbed data. This fundamental trade-off is inherent to the minimax nature of the optimization and cannot be completely eliminated, only mitigated by advanced techniques.
3. Limited transferable robustness: A model trained against a specific attack (e.g., FGSM with ε = 0.03) may not be robust to other types of perturbations (Gaussian noise, rotations, brightness variations). Robustness is not universal and remains partially dependent on the type of attack used during training.
4. Theoretical certification difficulty: Although adversarial training works well in practice, it provides no formal robustness guarantees. Formal certification methods exist but are limited to small networks or modest datasets due to combinatorial complexity reasons.

4 Concrete Use Cases

1. Autonomous Vehicles and Perception Systems

In autonomous vehicle systems, neural networks are responsible for detecting traffic signs, pedestrians, obstacles, and road markings from real-time video streams. Researchers have demonstrated that nearly invisible stickers placed on a “STOP” sign can cause it to be classified as a “Speed Limit 45 km/h” sign by an unprotected model. Adversarial training makes it possible to harden these perception systems so they resist such malicious physical manipulations, thereby ensuring the safety of road users.

2. Malware Detection and Cybersecurity

Cybercriminals are increasingly using adversarial techniques to create variants of malicious software (malware) that evade detection by machine learning-based analysis models. By training these detectors adversarially — that is, by actively generating perturbed malware samples that attempt to fool the classifier — cybersecurity companies can build detection systems that are more resilient to sophisticated evasion attempts and zero-day attacks.

3. Anti-Spam Filters and Content Moderation

Online platforms use classification models to automatically detect spam emails, hate speech, and inappropriate content. Attackers bypass these filters using subtle adversarial techniques: replacing characters with homographs, inserting innocent keywords, or making imperceptible changes to text formatting. Textual adversarial training (applying perturbations at the word embedding level) strengthens the robustness of these systems and maintains their effectiveness against increasingly elaborate bypass attempts.

4. Protection of Biometric and Facial Recognition Systems

Facial recognition systems are vulnerable to physical adversarial attacks, where specially designed patterns printed on glasses or placed on walls can trick the system into authenticating an unauthorized person or masking an individual’s identity in a surveillance camera. Adversarial training, by exposing the model to simulated adversarial variations during the learning phase, makes it more robust to these bypass attempts and strengthens the reliability of biometric authentication systems in potentially hostile environments.

See Also

• Manacher’s Algorithm: Discover How to Find All Palindromic Substrings in Python in O(N)
• Computing Comfortable Distance in Python: Techniques and Practical Applications