A Perturbation Resistant Transformation and Classification System for Deep Neural Networks

Goodfellow et al. [12] proposed the Fast Gradient Sign Method as an efficient approach to generate adversarial examples based on the gradient of the classification loss function with respect to the image. An image $x$ is perturbed to obtain an adversarial image $x_{adv}$ as follows:

where $\alpha$ is an $L_{\infty }$-bounded step size, $y_t$ is the ground-true label of x, L is a classification loss function such as cross-entropy loss, and $\theta$ is the set of neural network parameters. FGSM calculates the adversarial example in a single step by taking the sign of the gradient and moving a fixed distance to stay within an $L_{\infty }$-norm boundary around $x$.

The Basic Iterative Method (BIM) [17] is an iterative version of FGSM using smaller incremental step sizes and clipping pixel values after each update as necessary. The basic update step to the input is,

where $Cli{p}_{ϵ}$ projects the image to a maximum $L_{\infty }$-bound set by $ϵ$.

Madry et al. [18] generate adversarial examples from a generalized version of BIM that projects to different $L_p$-bounds. They have also shown that networks trained using adversarial examples generated by their methods are resilient against adversarial attacks.

Dezfooli et al. [19] convert the local non-linear problem of finding an adversarial example to that of an affine problem by linearizing the decision boundary between classes and projecting the input image towards a classification boundary in an iterative manner until misclassification occurs. This process extends to the multinomial classification problem by approximating the non-linear boundaries of multiple classes surrounding $x$ as a polyhedron and finding the minimum distance to its surface.

Carlini et al. [7] formulate an optimization problem that minimizes the $L_p$-norm distance between the clean and adversarial image, subject to the constraint that the adversarial image is misclassified by the neural network. They move the misclassification constraint into the objective function to form the non-convex problem:

Although not technically new attacks, backward pass differentiable approximation (BPDA) and expectation over transformation (EOT) [2, 3] allow existing attacks to handle defense systems that apply non-differentiable or randomized techniques.

Athalye et al. point out that these defenses effectively corrupt available gradient information through shattering (making gradients unavailable at the input due to a non-differentiable operation), stochasticity (applying randomness such that a single calculation of gradients is not representative), and explosion/vanishment (artificial overflow or underflow of gradient values). From a high level, by sampling transformations in the forward pass (EOT) and approximating non-differentiable operations as the identity function in the backward pass (BPDA), gradient based attacks can still find approximate gradients at the input and over many iterations still find a successful perturbation set.

In situations where gradient information is not reliable, a decision-based attack, like the Boundary Attack proposed by Brendel et al. [4], relies only on the forward pass prediction of the network. The Boundary Attack finds an initial perturbation distribution that is adversarial to the network and then progressively draws new perturbations that both maintain misclassification and reduce the vector norm distance to the original image.

Conceptually, the adversarial training approach forces the network to learn more robust decision boundaries at training time via exposure to training examples generated using one or more adversarial attack methods or modifications to the learning process.

Madry et. al [18] increased network robustness by perturbing training examples up to a fixed $L_p$-bound using PGD at each epoch and letting the network learn from the adversarial distribution. As an extension of [18], Ding et. al [10] proposed in Max-Margin training (MMA) that the $L_p$-bound placed on adversarially generated examples need not be fixed and in fact could be optimized on a per example basis to optimize the average decision boundary margin. Zhang et. al [36] included a regularization term in their training loss that searches for an adversarial example in an $ϵ$-neighborhood that maximally changes the output of the network compared to the clean example before updating the network parameters, thus encouraging decision boundaries to move maximally away from the data.

Further advancement in adversarial training focused on crafting adversarial examples based on not only classification loss, but also on the inter-example feature relationships between clean and generated examples. In [35], a PGD adversarial distribution is selected based on maximizing an optimal transport distance from the clean example distribution at the feature map level resulting in adversarial examples that are diverse and distant from clean examples during training.

A different defensive strategy is to apply voluntary transformations to inputs or network behaviors at inference time in the hope that these effects cannot be easily approximated by the attacker. Examples of image transformation include random resizing and padding [33], JPEG compression [9], and image quilting [13].

A related defensive strategy involves attempting to eliminate adversarial perturbations through various means [9, 20, 29, 28]. The basis for this strategy relies on the observation that clean and adversarial images, despite often visually similar in pixel space, come from different manifolds in feature space and that any mapping which bridges these two distributions could reduce adversarial corruptions in an image.

In the next section, we present the polynomial kernel matrix, which we use to transform the feature maps of denoised input images.

The following discussion is based on the flowchart in Fig. 2. Conceptually similar to but substantially different from neural style transfer [11], the polynomial kernel module computes losses in one or more layers of the network comparing the kernel matrix values of the $k^{th}$ copy of $x_R$ to the $k^{th}$ sampled image $s_k$ (from class $k$). The MSE loss for a single layer is:

We compute MSE losses at chosen network feature extraction layers and backpropagate their summed loss to the input, updating $x_{R,k}$ using a gradient descent algorithm. After every pixel update, we apply both an $L_1$ [8] and $L_{\infty }$ distance constraint to the current updated input, limiting the vector distance between $x_{R,k}$ and its final transformed version $x_{RT,k}$. The values of these constraints are hyperparameters obtained from validation study. We begin the iteration process by randomly perturbing the target $x_R$ within the chosen $L_{\infty }$ ball.

We iterate on the loss-backpropagate-constraint loop for a chosen number of iterations, recalculating the layer losses, pixel gradients, and vector norm projections independently for each iteration. We emphasize that this transfer procedure is completed for each $x_{R,k}$ corresponding to a different polynomial kernel set generated from $s_k$. The end result is that we output $K$ differently transformed versions of the input, $x_{RT,1}...x_{RT,K}$, to the next step.

Having transformed all $K$ copies of $x_R$, we run inference on each transformed copy through the network and store each label prediction in a set, $\{y_p|x_{RT,1}...y_p|x_{RT,K}\}$. From this set of predictions, we create a subset $H$ keeping only the predictions that are different than the stored, original prediction on $x$. We then apply a simple hard vote type rule to the remaining predictions: if the most common predicted label, $mode\left(H\right)$, of the predictions that changed due to transformation has a count greater than or equal to a threshold hyperparameter $c_3$, we change our final prediction to this most common label. Otherwise, we keep the original network prediction on $x$.

As shown in Fig. 3, the attacker can choose to attack the network directly (option A) or simulate the full defense system (option B) by implementing the defense algorithm on the forward pass, using gradient approximations on the backward pass, and increasing their sample sizes to better estimate the polynomial kernel transformations that the defender will use at inference time. These two different choices would result in one of two different adversarial images, $x_{adv,A}$ or $x_{adv,B}$, being fed to the defense system. The defense system initializes by making a prediction on the adversarial image through the ‘bare’ network, thus storing either $f(x_{adv,A};\theta )$ or $f(x_{adv,B};\theta )$. Note how $x_{adv,B}$ was not designed to attack the bare network $f\left(\theta \right)$ without the defense transformations. Adversarial image $x_{adv,B}$ could still transfer to $f\left(\theta \right)$, but we posit that it cannot be more likely to succeed than if the attack algorithm had just attacked $f\left(\theta \right)$ directly, i.e. crafted $x_{adv,A}$ in the first place.

The ideal scenario B outcome for the attacker is that a multitude (greater than threshold parameter $c_3$) of the polynomial kernel transforms move $x_{adv,B}$ to the same incorrect label; otherwise the network effectively drops its own defense and resorts to $f(x_{adv,B};\theta )$, for which we have already stated $x_{adv,B}$ was not designed.

Recall our defense system takes $K$ (number of classes) copies of the smoothed input $x_R$ and iteratively imparts the polynomial kernel values of $K$ sampled images from the clean training set (one sample from each class). In the toy example of Fig. 4, a CIFAR10 ’dog’ is initially correctly classified as class 5, but incorrectly classified to class 3 ’cat’ when the ResNet18 network is directly attacked (BIM-scenario A). We draw a sample of images from a few classes and also include the original clean image, then apply 25 iterations of kernel transformation from the samples to the adversarial example using feature maps from each residual block in ResNet18 (early and deep layers). Several transformations are successful in restoring the correct classification and some are not.

Our results suggest several important conclusions. First, if the kernel matrix information of the original image is imparted on a perturbed version of itself, the network recovers the correct classification. Second, transformations derived from a sample class $k^{\prime }$ can recover the correct classification of images from other classes $k\ne k^{\prime }$ as can be seen from the boat sample reclassifying the dog. Third, we track the average polynomial kernel loss 9 at iteration 0 for the deepest residual block in ResNet18 as a $\left(\%\right)$ share of the total loss across all layers and recognize the original image ‘sample’ has the highest loss share compared to other samples; likewise, the early layers have relatively low kernel loss. This effect implies that adversarial attacks’ imperceptible pixel manipulations create correspondingly small changes in early layer kernel matrices, but larger changes in the deeper layers, which are closer to classification. Lastly, not all transformations result in correct classifications. An important consideration not shown in this particular example, is sometimes we receive random classifications $k^{\prime \prime }\notin \{k,k^{\prime }\}$.

Our interpretation of the toy experiment is pictured on the far right of Fig. 4, a clean example $x$ (black) is perturbed across the decision boundary to a misclassified $x_{adv}$. Several transformation directions are shown altering $x_{adv}$ to $x_{RT,k}$ depending on the sample’s class. Some transformations cross back over the decision boundary and are correctly classified, some remain misclassified, and others end up in random other classes. Let the tuple $(k^{\prime },k^{\prime \prime })$ represent the decision boundary between class $k^{\prime }$ and $k^{\prime \prime }$. We believe and have found that if the adversary is close to the boundary of the correct class or ’weak’, then $(k,k^{\prime })$ presents as a larger cross-sectional target to the kernel transforms as compared to others in $\left\{\right(k^{\prime \prime },k),(k^{\prime \prime },k^{\prime }),(k^{\prime \prime },k^{\prime \prime }\left)\right\}$. Therefore, if any reclassification is to occur, a propensity exists for a correct classification over a random classification. What about clean examples, will they be mistakenly reclassified? Particularly in the case of adversarially trained networks, the clean data is pushed away from the boundary, so the risk of any reclassification is lowered compared to a weak adversary pushed just over the boundary. Further, we perform validation studies to choose a voting threshold parameter $c_3$ that allows for a few random reclassifications of clean examples without the defense changing its prediction due to the voting committee.

How do these ideas interact with the attack scenario of Fig. 3? The polynomial kernel transforms are not attempting to cleanse adversarial perturbations, rather they are independent feature transformations to all possible classes. The scenario B attacker, to achieve the ideal scenario, must coalesce these different transformations with a single perturbation set or risk that the designed adversary is transferred back to the bare network as a weaker adversary than if it had just targeted the bare network originally.