Using machine learning models to make decisions impacting human lives has become a norm over the last decades. However, such decisions can be affected by malicious actors using adversarial examples. To mitigate this risk, the research community has developed various methods improving the robustness of machine learning systems against such attacks, with adversarial training being the most prominent approach. Despite its potential, adversarial training sees limited practical application.
In this thesis, we examine mismatches between existing research on adversarial robustness and its potential practical application in security-critical domains. We focus on three aspects that are important for practitioners but are often overlooked in the literature: realistic threat modeling, handling a wide class of machine learning methods, and the connection between adversarial robustness and non-security-related properties of machine learning systems. The goal of our research is to address these discrepancies and make adversarial robustness methods more practical and applicable in real-world scenarios.
We first show that distance-based constraints, commonly used to build threat models in the adversarial robustness literature, do not capture the realistic capabilities and limitations of realistic adversaries. In particular, in tabular domains used in many security-critical applications. We propose new cost and utility-aware threat models tailored to the financial capabilities of the adversary. We design a defense mechanism establishing security properties within these threat models, and show its effectiveness through extensive experimental evaluation.
Next, we focus on categorical feature input spaces and on the usage of non-differential models in practical applications of machine learning on tabular data. We argue that existing adversarial training methods do not adequately address the discrete and assymetrical nature of categorical features. We propose an efficient procedure for adversarial training on categorical features based on a relaxation of the model input space. Furthermore, we address the challenge of applying gradient-based adversarial training to non-differentiable models widely used in security-critical applications. We introduce a method for transferring the adversarial robustness obtained through our procedure to these classical machine learning methods.
Finally, we explore the relationship between adversarial robustness and robustness to domain shifts introduced by perturbations of different origins in the image domain. We show that carefully-performed adversarial training can improve model accuracy in the presence of noise and propose a method to enhance domain shift robustness in the image domain. Our experiments demonstrate that this approach also has cumulative gains with different data augmentation methods, leading to outstanding model performance on corrupted images.
In summary, this thesis helps to close the gap between adversarial r
