Deakin University
Browse

File(s) under permanent embargo

On the convergence and robustness of adversarial training

conference contribution
posted on 2019-01-01, 00:00 authored by Y Wang, Daniel Ma, J Bailey, J Yi, B Zhou, Q Gu
Copyright © 2019 ASME Improving the robustness of deep neural networks (DNNs) to adversarial examples is an important yet challenging problem for secure deep learning. Across existing defense techniques, adversarial training with Projected Gradient Decent (PGD) is amongst the most effective. Adversarial training solves a min-max optimization problem, with the inner maximization generating adversarial examples by maximizing the classification loss, and the outer minimization finding model parameters by minimizing the loss on adversarial examples generated from the inner maximization. A criterion that measures how well the inner maximization is solved is therefore crucial for adversarial training. In this paper, we propose such a criterion, namely First-Order Stationary Condition for constrained optimization (FOSC), to quantitatively evaluate the convergence quality of adversarial examples found in the inner maximization. With FOSC, we find that to ensure better robustness, it is essential to use adversarial examples with better convergence quality at the later stages of training. Yet at the early stages, high convergence quality adversarial examples are not necessary and may even lead to poor robustness. Based on these observations, we propose a dynamic training strategy to gradually increase the convergence quality of the generated adversarial examples, which significantly improves the robustness of adversarial training. Our theoretical and empirical results show the effectiveness of the proposed method.

History

Event

Machine Learning. Conference (2019 : 36th : Long Beach, California)

Volume

2019-June

Pagination

11426 - 11438

Publisher

PMLR

Location

Long Beach, California

Place of publication

[Long Beach, Calif.]

Start date

2019-06-09

End date

2019-06-15

ISBN-13

9781510886988

Language

eng

Publication classification

E1.1 Full written paper - refereed

Title of proceedings

ICML 2019 : Proceedings of the 36th International Conference on Machine Learning

Usage metrics

    Research Publications

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC