Projected Gradient Descent Attack
Ascend the loss with respect to the input, project back into the ε-ball after every step, and restart from random points inside it.
PGD is the standard first-order attack and, empirically, close to the strongest one within its threat model. Each step takes a signed gradient step of size and projects back onto the feasible set.
Random restarts matter: the inner maximisation is non-concave, and a single run from the clean input systematically underestimates the attainable loss. Adversarial training frames the whole thing as a saddle-point problem with PGD as the inner solver.
import torch
from torch import Tensor
def pgd(model, x: Tensor, y: Tensor, loss_fn,
eps: float = 8 / 255, alpha: float = 2 / 255,
steps: int = 20, restarts: int = 5) -> Tensor:
best_adv, best_loss = x.clone(), torch.full((x.size(0),), -1e30)
for _ in range(restarts):
delta = torch.empty_like(x).uniform_(-eps, eps)
delta = (x + delta).clamp(0, 1) - x
for _ in range(steps):
delta.requires_grad_(True)
loss = loss_fn(model(x + delta), y)
grad, = torch.autograd.grad(loss.sum(), delta)
delta = (delta.detach() + alpha * grad.sign()).clamp(-eps, eps)
delta = (x + delta).clamp(0, 1) - x # stay a valid image
with torch.no_grad():
l = loss_fn(model(x + delta), y)
improved = l > best_loss
best_loss = torch.where(improved, l, best_loss)
best_adv[improved] = (x + delta)[improved]
return best_advTwo clamps per step, in this order: the ε-ball first, then the valid input range. Reversing them lets the perturbation drift outside the threat model. Keep the per-example best rather than the last iterate — the loss is not monotone under a fixed step size, and reporting the final step understates the attack.