Adversarial Robustness

Vulnerability, transfer and accuracy

Adversarial examples and transfer - Finds small image perturbations that induce misclassification and can transfer between networks trained on different data subsets. Transfer enables attacks without differentiating the target network. Szegedy et al. 2014.
Non-robust features - Classifiers trained on adversarially relabeled images generalize to clean test data. These vision experiments show that predictive but perturbation-sensitive features can support ordinary generalization and adversarial transfer. Ilyas et al. 2019.
Robustness-accuracy trade-offs - Proves an accuracy-robustness conflict for a constructed distribution and observes a similar tension in vision experiments. The conflict depends on the data and perturbation set, so it need not hold for every classification problem. Tsipras et al. 2019.

FGSM - Takes one step in the sign of the input-loss gradient, maximizing a linearized loss under an l-infinity bound. Supplies a cheap attack and training perturbation; iterative attacks search beyond that local linearization. Goodfellow et al. 2015.
Carlini-Wagner attacks - Optimizes targeted misclassification under l0, l2 or l-infinity distance criteria. Defeats defensive distillation in the tested networks, showing that failure of earlier attacks had overstated that defense. Carlini and Wagner 2017.
Adversarial patches - Optimizes a shared visible patch over images, locations, rotations and scales to induce a target class. Demonstrates printed attacks; its localized replacement threat differs from small changes to every pixel. Brown et al. 2017.
Projected gradient attacks - Repeatedly increases input loss and projects onto the permitted perturbation set. Random starts explore different local maxima; failure to find an adversarial input supplies no certificate of its absence. Madry et al. 2018.
GCG - Uses gradient-guided discrete search for adversarial text suffixes. Optimizing across prompts and models transfers some attacks on refusal behavior to other language models; results concern the tested 2023 checkpoints and interfaces. Zou et al. 2023.

PGD adversarial training - Approximates minimax training: projected-gradient attacks maximize loss within a specified input perturbation set; parameter updates minimize that loss. Reported resistance concerns the evaluated first-order attacks. Madry et al. 2018.
TRADES - Balances clean-label loss with disagreement between predictions on clean and adversarial inputs. Derives the regularizer from natural-error and boundary-error terms, making the accuracy-robustness trade-off explicit. Zhang et al. 2019.
Robust self-training - Pseudo-labels additional unlabeled images, then uses them in robust training. Extra data improves CIFAR-10 and SVHN robustness in the tested setups, connecting robust sample requirements with semi-supervised learning. Carmon et al. 2019.
Robust overfitting - Robust test error can rise while training continues to improve. Across the studied image datasets, early stopping matches or beats the tested regularizers; selecting the final training iterate can misrank defenses. Rice et al. 2020.

Convex outer adversarial polytope - Relaxes the reachable activations of ReLU networks to bound worst-case loss under l-infinity perturbations. Optimizes a dual bound during training; a verified bound covers the specified input region. Wong and Kolter 2018.
Interval bound propagation - Propagates activation intervals to bound worst-case outputs and trains against those bounds. Cheap interval relaxations can be loose, so failure to certify an input need not imply an adversarial example exists. Gowal et al. 2018.
Randomized smoothing - Defines a classifier by majority prediction under Gaussian input noise and certifies l2 robustness from class probabilities. Finite-sample certificates use confidence bounds and may abstain; they concern the smoothed classifier. Cohen et al. 2019.

Obfuscated gradients - Seven of nine studied ICLR 2018 defenses relied on gradient masking. Approximate backward passes and averaging over randomness broke six completely and one partially, separating attack failure from robustness. Athalye et al. 2018.
Adaptive attacks - Circumvents 13 selected defenses despite their published adaptive evaluations. Successful attacks required defense-specific objectives and treatment of preprocessing or randomness; a generic attack suite was insufficient. Tramèr et al. 2020.
AutoAttack - Combines two Auto-PGD variants with FAB and a score-based Square Attack. Its evaluation of over 50 classifiers reduced reported robust accuracy by over 10 percentage points for 13 models; threat sets remain fixed. Croce and Hein 2020; code.
RobustBench - Standardizes datasets, perturbation budgets, model interfaces and attack evaluation. Reported attacked accuracy is an empirical upper bound on worst-case robust accuracy; the benchmark also accepts stronger adaptive evaluations. Croce et al. 2021; code.