In this paper (winner of the eCrime 2019 Best Paper award), we consider the types of things that can go wrong when you intend to make things better and more secure. Consider this scenario. You are browsing through Internet and see a news headline on one of the presidential candidates. You are unsure if the headline is true. What you can do is to navigate to a fact-checking website and type in the headline of interest. Some platforms also have fact-checking bots that would update periodically on false information. You do some research through three fact-checking websites and the results consistently show that the news contains false information. You share the results as a comment on the news article. Within two hours, you receive hundreds of notifications with comments countering your resources with other fact-checking websites. 

Such a scenario is increasingly common as we rely on the Internet and social media platforms for information and news. Although they are meant to increase security, these cybersecurity countermeasures can result in confusion and frustration among users due to the incorporation of additional actions as part of users’ daily online routines. As seen, fact-checking can easily be used as a mechanism for attacks and demonstration of in-group/out-group distinction which can contribute further to group polarisation and fragmentation. We identify these negative effects as unintended consequences and define it as shifts in expected burden and/or effort to a group. 

To understand unintended harms, we begin with five scenarios of cyber aggression and deception. We identify common countermeasures for each scenario, and brainstorm potential unintended harms with each countermeasure. The unintended harms are inductively organized into seven categories: 1) displacement, 2) insecure norms, 3) additional costs, 4) misuse, 5) misclassification, 6) amplification and 7) disruption. Applying this framework to the above scenario, insecure norms, miuse, and amplification are both unintended consequences of fact-checking. Fact-checking can foster a sense of complacency where checked news are automatically seen as true. In addition, fact-checking can be used as tools for attacking groups of different political views. Such misuse facilitates amplification as fact-checking is being used to strengthen in-group status and therefore further exacerbate the issue of group polarisation and fragmentation. 

To allow for a systematic application to existing or new cybersecurity measures by practitioners and stakeholders, we expand the categories into a functional framework by developing prompts for each harm category. During this process, we identify the underlying need to consider vulnerable groups. In other words, practitioners and stakeholders need to take into consideration the impacts of countermeasures on at-risk groups as well as the possible creation of new vulnerable groups as a result of deploying a countermeasure. Vulnerable groups refer to user groups who may suffer while others are unaffected or prosper from the countermeasure. One example is older adult users where their non-familiarity and less frequent interactions with technologies means that they are forgotten or hidden when assessing risks and/or countermeasures within a system. 

It is important to note the framework does not propose measurements for the severity or the likelihood of unintended harm occurring. Rather, the emphasis of the framework is in raising stakeholders’ and practitioners’ awareness of possible unintended consequences. We envision this framework as a common-ground tool for stakeholders, particularly for coordinating approaches in complex, multi-party services and/or technology ecosystems.  We would like to extend a special thank you to Schloss Dagstuhl and the organisers of Seminar #19302 (Cybersafety Threats – from Deception to Aggression). It brought all of the authors together and laid out the core ideas in this paper. A complimentary blog post by co-author Dr Simon Parkin can be found at UCL’s Benthams Gaze blog. The accepted manuscript for this paper is available here.

Last week the APWG Symposium on Electronic Crime Research was held at Carnegie Mellon University in Pittsburgh. The Cambridge Cybercrime Centre was very well-represented at the symposium. Of the 12 accepted research papers, five were authored or co-authored by scholars from the Centre. The topics of the research papers addressed a wide range of cybercrime issues, ranging from honeypots to gaming as pathways to cybercrime. One of the papers with a Cambridge author, “Identifying Unintended Harms of Cybersecurity Countermeasures”, received the Best Paper award. The Honorable Mention award went to “Mapping the Underground: Supervised Discovery of Cybercrime Supply Chains”, which was a collaboration between NYU, ICSI and the Centre.

In this post, we will provide a brief description for each paper in this post. The final versions aren’t yet available, we will blog them in more detail as they appear.

Best Paper

Identifying Unintended Harms of Cybersecurity Countermeasures

Yi Ting Chua, Simon Parkin, Matthew Edwards, Daniela Oliveira, Stefan Schiffner, Gareth Tyson, and Alice Hutchings

In this paper, the authors consider that well-intentioned cybersecurity risk management activities can create not only unintended consequences, but also unintended harms to user behaviours, system users, or the infrastructure itself. Through reviewing countermeasures and associated unintended harms for five cyber deception and aggression scenarios (including tech-abuse, disinformation campaigns, and dating fraud), the authors identified categorizations of unintended harms. These categories were further developed into a framework of questions to prompt risk managers to consider harms in a structured manner, and introduce the discussion of vulnerable groups across all harms. The authors envision that this framework can act as a common-ground and a tool bringing together stakeholders towards a coordinated approach to cybersecurity risk management in a complex, multi-party service and/or technology ecosystem.

Honorable Mention

Mapping the Underground: Supervised Discovery of Cybercrime Supply Chains

Rasika Bhalerao, Maxwell Aliapoulios, Ilia Shumailov, Sadia Afroz, and Damon McCoy

Cybercrime forums enable modern criminal entrepreneurs to collaborate with other criminals into increasingly efficient and sophisticated criminal endeavors.
Understanding the connections between different products and services is currently very expensive and requires a lot of time-consuming manual effort. In this paper, we propose a language-agnostic method to automatically extract supply chains from cybercrime forum posts and replies. Our analysis of generated supply chains highlights unique differences in the lifecycle of products and services on offer in Russian and English cybercrime forums.

Honware: A Virtual Honeypot Framework for Capturing CPE and IoT Zero Day

Alexander Vetterl and Richard Clayton

We presented honware, a new honeypot framework which can rapidly emulate a wide range of CPE and IoT devices without any access to the manufacturers’ hardware.

The framework processes a standard firmware image and will help to detect real attacks and associated vulnerabilities that might otherwise be exploited for considerable periods of time without anyone noticing.

From Playing Games to Committing Crimes: A Multi-Technique Approach to Predicting Key Actors on an Online Gaming Forum

Jack Hughes , Ben Collier, and Alice Hutchings

This paper proposes a systematic framework for analysing forum datasets, which contain minimal structure and are non-trivial to analyse at scale. The paper takes a multi-technique approach drawing on a combination of features relating to content and metadata, to predict potential key actors. From these predictions and trained models, the paper begins to look at characteristics of the group of potential key actors, which may benefit more from targeted intervention activities.

Fighting the “Blackheart Airports”: Internal Policing in the Chinese Censorship Circumvention Ecosystem

Yi Ting Chua and Ben Collier

In this paper, the authors provide an overview of the self-policing mechanisms present in the ecosystem of services used in China to circumvent online censorship. We conducted an in-depth netnographic study of four Telegram channels which were used to co-ordinate various kinds of attacks on groups and individuals offering fake or scam services. More specifically, these actors utilized cybercrime tools such as denial of service attack and doxxing to punish scammers. The motivations behind this self-policing appear to be genuinely altruistic, with individuals largely concerned with maintaining a stable ecosystem of services to allow Chinese citizens to bypass the Great Firewall. Although this is an emerging phenomenon, it appears to be developing into an important and novel kind of trust mechanism within this market