Recent advancements in Machine Learning (ML) have taught us two main lessons: a large proportion of things that humans do can actually be automated, and that a substantial part of this automation can be done with minimal human supervision. One no longer needs to select features for models to use; in many cases people are moving away from selecting the models themselves and perform a Network Architecture Search. This means non-stop search across billions of dimensions, ever improving different properties of deep neural networks (DNNs).

However, progress in automation has brought a spectre to the feast. Automated systems seem to be very vulnerable to adversarial attacks. Not only is this vulnerability hard to get rid of; worse, we often can’t even define what it means to be vulnerable in the first place.

Furthermore, finding adversarial attacks on ML systems is really easy even if you do not have any access to the models. There are only so many things that make cat a cat, and all the different models that deal with cats will be looking at the same set of features. This has an important implication: learning how to trick one model dealing with cats often transfers over to other models. Transferability is a terrible property for security because it makes adversarial ML attacks cheap and scalable. If there is a camera in the bank running a similar ML model to the camera you can get in Costco for $5, then the cost of developing an attack is $5.

As of now, we do not really have good answers to any of these questions. In the meantime, ML controlled systems are entering the human realm.

In this Three Paper Thursday I want to talk about works from the field of adversarial ML that make it much more understandable.

The platforms, providers, and infrastructures which together make up the contemporary Internet play an increasingly central role in the business of governing human societies. Although the software engineers, administrators, business professionals, and other staff working at these organisations may not have the institutional powers of state organisations such as law enforcement or the civil service, they are now in a powerful position of responsibility for the harms and illegal activities which their platforms facilitate. For this Three Paper Thursday, I’ve chosen to highlight papers which address these issues, and which explore the complex networks of different infrastructural actors and perspectives which play a role in the reporting, handling, and defining of abuse and crime online.

In this reboot of the Three Paper Thursdays, back after a hiatus of almost eight years, I consider the many different ways in which programs can be sanitised to detect, or mitigated to prevent the use of, the many programmer errors that can introduce security vulerabilities in low-level languages such as C and C++. We first look at a new binary translation technique, before covering the many compiler techniques in the literature, and finally finishing off with my own hardware analysis architecture.

I’ve just returned from the 2012 International Summer School on Information Security and Protection (ISSISP2012) held at the University of Arizona. This annual summer school brings together a mix of academic researchers and industry practitioners in the field of software protection where the main philosophy, and indeed the only viable approach available, can be summed up as “Security through Obscurity”. The goal here is to impede reverse engineering and to hide algorithms and data in the presence of disassemblers, decompilers, debuggers as well as side-channel analysis – this is the Man-at-the-End (MATE) attack. White box cryptography, I’ve learnt, is the term used to describe the protection of cryptographic primitives and keys against this kind of attack. This week I wish to highlight 3 talks/papers which I found memorable – the first 2 describe techniques to address code injection and timing side-channel attacks; the last one discusses formally verified program obfuscators.

Continue reading Three Paper Thursday: ISSISP2012 →