Category Archives: Security economics

Social-science angles of security

Three Paper Thursday: Vulnerabilities! We’ve got vulnerabilities here! … See? Nobody cares.

Academic papers, Cybercrime, Programmable logic, Protocols, Security economics, Three Paper Thursday

Jurassic Park is often (mistakenly) left out of the hacker movie canon. It clearly demonstrated the risk of an insider attack on control systems (Velociraptor rampage, amongst other tragedies…) nearly a decade ahead of the Maroochy sewage incident, it’s the first film I know of with a digital troll (“ah, ah, ah, you didn’t say the magic word!”), and Samuel L. Jackson correctly assesses the possible consequence of a hard reset (namely, everyone dying), resulting in his legendary “Hold on to your butts”. The quotable mayhem is seeded early in the film, when biotech spy Lewis Dodgson gives a sack of money to InGen’s Dennis Nedry to steal some dino DNA. Dodgson’s caricatured OPSEC (complete with trilby and dark glasses) is mocked by Nedry shouting, “Dodgson! Dodgson! We’ve got Dodgson here! See, nobody cares…” Three decades later, this quote still comes to mind* whenever conventional wisdom doesn’t seem to square with observed reality, and today we’re going to apply it to the oft-maligned world of Industrial Control System (ICS) security.

There is plenty of literature on ICS security pre-2010, but people really sat up and started paying attention when we learned about Stuxnet. Possibly the most upsetting thing about Stuxnet (for security-complacent control system designers like me) was the apparent ease with which the “air gap” was bridged over and over again. Any remaining faith in the air gap was killed by Éireann Leverett’s demonstration (thesis and S4 presentation) that thousands of industrial systems were directly connected to the Internet — no air gap jumping required. Since then, we’ve observed a steady growth in Internet-connected ICS devices, due both to improved search techniques and increasingly-connectable ICS devices. On any given day you can find about 100,000 unique devices speaking industrial protocols on Censys and Shodan. These protocols are largely unauthenticated and unencrypted, allowing an attacker that can speak the protocol to remotely read state, issue commands, and even modify programmable logic without using an actual exploit.

This sounds (and is) bad, and people have (correctly) highlighted its badness on many occasions. The attacks, however, appear to be missing: we are not aware of a single instance of industrial damage initiated via an Internet-connected ICS device. In this Three Paper Thursday we’ll look at papers showing how easy it is to find and contextualise Internet-connected ICS devices, some evidence for lack of malicious interest, and some leading indicators that this happy conclusion (for which we don’t really deserve any credit) may be changing.

*Perhaps because guys of a certain age still laugh and say “Dodson! We’ve got Dodson here!” when they learn my surname. I try to explain that it’s spelt differently, but…
Continue reading Three Paper Thursday: Vulnerabilities! We’ve got vulnerabilities here! … See? Nobody cares. →

Three Paper Thursday – GDPR anniversary edition

Academic papers, Legal issues, Politics, Privacy technology, Security economics, Three Paper Thursday

This is a guest contribution from Daniel Woods.

This coming Monday will mark two years since the General Data Protection Regulation (GDPR) came into effect. It prompted an initial wave of cookie banners that drowned users in assertions like “We value your privacy”. Website owners hoped that collecting user consent would ensure compliance and ward off the lofty fines.

Article 6 of the GDPR describes how organisations can establish a legal basis for processing personal data. Putting aside a selection of `necessary’ reasons for doing so, data processing can only be justified by collecting the user’s consent to “the processing of his or her personal data for one or more specific purposes”. Consequently, obtaining user consent could be the difference between suffering a dizzying fine or not.

The law changed the face of the web and this post considers one aspect of the transition. Consent Management Providers (CMPs) emerged offering solutions for websites to embed. Many of these use a technical standard described in the Transparency and Consent Framework. The standard was developed by the Industry Advertising Body, who proudly claim it is is “the only GDPR consent solution built by the industry for the industry”.

All of the following studies either directly measure websites implementing this standard or explore the theoretical implications of standardising consent. The first paper looks at how the design of consent dialogues shape the consent signal sent by users. The second paper identifies disparities between the privacy preferences communicated via cookie banners and the consent signals stored by the website. The third paper uses coalitional game theory to explore which firms extract the value from consent coalitions in which websites share consent signals.

Continue reading Three Paper Thursday – GDPR anniversary edition →

Contact Tracing in the Real World

Academic papers, Cryptology, News coverage, Politics, Privacy technology, Security economics, Security psychology, ,

There have recently been several proposals for pseudonymous contact tracing, including from Apple and Google. To both cryptographers and privacy advocates, this might seem the obvious way to protect public health and privacy at the same time. Meanwhile other cryptographers have been pointing out some of the flaws.

There are also real systems being built by governments. Singapore has already deployed and open-sourced one that uses contact tracing based on bluetooth beacons. Most of the academic and tech industry proposals follow this strategy, as the “obvious” way to tell who’s been within a few metres of you and for how long. The UK’s National Health Service is working on one too, and I’m one of a group of people being consulted on the privacy and security.

But contact tracing in the real world is not quite as many of the academic and industry proposals assume.

First, it isn’t anonymous. Covid-19 is a notifiable disease so a doctor who diagnoses you must inform the public health authorities, and if they have the bandwidth they call you and ask who you’ve been in contact with. They then call your contacts in turn. It’s not about consent or anonymity, so much as being persuasive and having a good bedside manner.

I’m relaxed about doing all this under emergency public-health powers, since this will make it harder for intrusive systems to persist after the pandemic than if they have some privacy theater that can be used to argue that the whizzy new medi-panopticon is legal enough to be kept running.

Second, contact tracers have access to all sorts of other data such as public transport ticketing and credit-card records. This is how a contact tracer in Singapore is able to phone you and tell you that the taxi driver who took you yesterday from Orchard Road to Raffles has reported sick, so please put on a mask right now and go straight home. This must be controlled; Taiwan lets public-health staff access such material in emergencies only.

Third, you can’t wait for diagnoses. In the UK, you only get a test if you’re a VIP or if you get admitted to hospital. Even so the results take 1–3 days to come back. While the VIPs share their status on twitter or facebook, the other diagnosed patients are often too sick to operate their phones.

Fourth, the public health authorities need geographical data for purposes other than contact tracing – such as to tell the army where to build more field hospitals, and to plan shipments of scarce personal protective equipment. There are already apps that do symptom tracking but more would be better. So the UK app will ask for the first three characters of your postcode, which is about enough to locate which hospital you’d end up in.

Fifth, although the cryptographers – and now Google and Apple – are discussing more anonymous variants of the Singapore app, that’s not the problem. Anyone who’s worked on abuse will instantly realise that a voluntary app operated by anonymous actors is wide open to trolling. The performance art people will tie a phone to a dog and let it run around the park; the Russians will use the app to run service-denial attacks and spread panic; and little Johnny will self-report symptoms to get the whole school sent home.

Sixth, there’s the human aspect. On Friday, when I was coming back from walking the dogs, I stopped to chat for ten minutes to a neighbour. She stood halfway between her gate and her front door, so we were about 3 metres apart, and the wind was blowing from the side. The risk that either of us would infect the other was negligible. If we’d been carrying bluetooth apps, we’d have been flagged as mutual contacts. It would be quite intolerable for the government to prohibit such social interactions, or to deploy technology that would punish them via false alarms. And how will things work with an orderly supermarket queue, where law-abiding people stand patiently six feet apart?

Bluetooth also goes through plasterboard. If undergraduates return to Cambridge in October, I assume there will still be small-group teaching, but with protocols for distancing, self-isolation and quarantine. A supervisor might sit in a teaching room with two or three students, all more than 2m apart and maybe wearing masks, and the window open. The bluetooth app will flag up not just the others in the room but people in the next room too.

How is this to be dealt with? I expect the app developers will have to fit a user interface saying “You’re within range of device 38a5f01e20. Within infection range (y/n)?” But what happens when people get an avalanche of false alarms? They learn to click them away. A better design might be to invite people to add a nickname and a photo so that contacts could see who they are. “You are near to Ross [photo] and have been for five minutes. Are you maintaining physical distance?”

When I discussed this with a family member, the immediate reaction was that she’d refuse to run an anonymous app that might suddenly say “someone you’ve been near in the past four days has reported symptoms, so you must now self-isolate for 14 days.” A call from a public health officer is one thing, but not knowing who it was would just creep her out. It’s important to get the reactions of real people, not just geeks and wonks! And the experience of South Korea and Taiwan suggests that transparency is the key to public acceptance.

Seventh, on the systems front, decentralised systems are all very nice in theory but are a complete pain in practice as they’re too hard to update. We’re still using Internet infrastructure from 30 years ago (BGP, DNS, SMTP…) because it’s just too hard to change. Watch Moxie Marlinspike’s talk at 36C3 if you don’t get this. Relying on cryptography tends to make things even more complex, fragile and hard to change. In the pandemic, the public health folks may have to tweak all sorts of parameters weekly or even daily. You can’t do that with apps on 169 different types of phone and with peer-to-peer communications.

Personally I feel conflicted. I recognise the overwhelming force of the public-health arguments for a centralised system, but I also have 25 years’ experience of the NHS being incompetent at developing systems and repeatedly breaking their privacy promises when they do manage to collect some data of value to somebody else. The Google Deepmind scandal was just the latest of many and by no means the worst. This is why I’m really uneasy about collecting lots of lightly-anonymised data in a system that becomes integrated into a whole-of-government response to the pandemic. We might never get rid of it.

But the real killer is likely to be the interaction between privacy and economics. If the app’s voluntary, nobody has an incentive to use it, except tinkerers and people who religiously comply with whatever the government asks. If uptake remains at 10-15%, as in Singapore, it won’t be much use and we’ll need to hire more contact tracers instead. Apps that involve compulsion, such as those for quarantine geofencing, will face a more adversarial threat model; and the same will be true in spades for any electronic immunity certificate. There the incentive to cheat will be extreme, and we might be better off with paper serology test certificates, like the yellow fever vaccination certificates you needed for the tropics, back in the good old days when you could actually go there.

All that said, I suspect the tracing apps are really just do-something-itis. Most countries now seem past the point where contact tracing is a high priority; even Singapore has had to go into lockdown. If it becomes a priority during the second wave, we will need a lot more contact tracers: last week, 999 calls in Cambridge had a 40-minute wait and it took ambulances six hours to arrive. We cannot field an app that will cause more worried well people to phone 999.

The real trade-off between surveillance and public health is this. For years, a pandemic has been at the top of Britain’s risk register, yet far less was spent preparing for one than on anti-terrorist measures, many of which were ostentatious rather than effective. Worse, the rhetoric of terror puffed up the security agencies at the expense of public health, predisposing the US and UK governments to disregard the lesson of SARS in 2003 and MERS in 2015 — unlike the governments of China, Singapore, Taiwan and South Korea, who paid at least some attention. What we need is a radical redistribution of resources from the surveillance-industrial complex to public health.

Our effort should go into expanding testing, making ventilators, retraining everyone with a clinical background from vet nurses to physiotherapists to use them, and building field hospitals. We must call out bullshit when we see it, and must not give policymakers the false hope that techno-magic might let them avoid the hard decisions. Otherwise we can serve best by keeping out of the way. The response should not be driven by cryptographers but by epidemiologists, and we should learn what we can from the countries that have managed best so far, such as South Korea and Taiwan.

Security Engineering, and Sustainability

Academic papers, Legal issues, Politics, Security economics, Security engineering

Yesterday I got the audience at the 36th Chaos Computer Congress in Leipzig to vote on the cover art for the third edition of my textbook on Security Engineering: you can see the result here.

It was a privilege to give a talk at 36C3; as the theme was sustainability, I spoke on The Sustainability of Safety, Security and Privacy. This is a topic on which I’ve written and spoken several times in recent years, but we now have some progress to report. The EU has changed the rules to require that if you sell goods with digital components (whether embedded software, associated cloud services or smartphone apps) then these have to be maintained for as long as the customer might reasonably expect.

SHB 2019 – Liveblog

Academic papers, Legal issues, Privacy technology, Security economics, Security psychology, Social networks

I’ll be trying to liveblog the twelfth workshop on security and human behaviour at Harvard. I’m doing this remotely because of US visa issues, as I did for WEIS 2019 over the last couple of days. Ben Collier is attending as my proxy and we’re trying to build on the experience of telepresence reported here and here. My summaries of the workshop sessions will appear as followups to this post.

WEIS 2019 – Liveblog

Academic papers, Cybercrime, Legal issues, Politics, Security economics, Security psychology

I’ll be trying to liveblog the seventeenth workshop on the economics of information security at Harvard. I’m not in Cambridge, Massachussetts, but in Cambridge, England, because of a visa held in ‘administrative processing’ (a fate that has befallen several other cryptographers). My postdoc Ben Collier is attending as my proxy (inspired by this and this).

The Changing Cost of Cybercrime

Academic papers, Banking security, Cybercrime, Legal issues, Politics, Security economics

In 2012 we presented the first systematic study of the costs of cybercrime. We have now repeated our study, to work out what’s changed in the seven years since then.

Measuring the Changing Cost of Cybercrime will appear on Monday at WEIS. The period has seen huge changes, with the smartphone replacing as PC and laptop as the consumer terminal of choice, with Android replacing Windows as the most popular operating system, and many services moving to the cloud. Yet the overall pattern of cybercrime is much the same.

We know a lot more than we did then. Back in 2012, we guessed that cybercrime was about half of all crime, by volume and value; we now know from surveys in several countries that this is the case. Payment fraud has doubled, but fallen slightly as a proportion of payment value; the payment system has got larger, and slightly more efficient.

So what’s changed? New cybercrimes include ransomware and other offences related to cryptocurrencies; travel fraud has also grown. Business email compromise and its cousin, authorised push payment fraud, are also growth areas. We’ve also seen serious collateral damage from cyber-weapons such as the NotPetya worm. The good news is that crimes that infringe intellectual property – from patent-infringing pharmaceuticals to copyright-infringing software, music and video – are down.

Our conclusions are much the same as in 2012. Most cyber-criminals operate with impunity, and we have to fix this. We need to put a lot more effort into catching and punishing the perpetrators.

Our new paper is here. For comparison the 2012 paper is here, while a separate study on the emotional cost of cybercrime is here.

Security Engineering: Third Edition

Academic papers, Cybercrime, Internet censorship, Legal issues, Politics, Privacy technology, Security economics, Security engineering

I’m writing a third edition of my best-selling book Security Engineering. The chapters will be available online for review and feedback as I write them.

Today I put online a chapter on Who is the Opponent, which draws together what we learned from Snowden and others about the capabilities of state actors, together with what we’ve learned about cybercrime actors as a result of running the Cambridge Cybercrime Centre. Isn’t it odd that almost six years after Snowden, nobody’s tried to pull together what we learned into a coherent summary?

There’s also a chapter on Surveillance or Privacy which looks at policy. What’s the privacy landscape now, and what might we expect from the tussles over data retention, government backdoors and censorship more generally?

There’s also a preface to the third edition.

As the chapters come out for review, they will appear on my book page, so you can give me comment and feedback as I write them. This collaborative authorship approach is inspired by the late David MacKay. I’d suggest you bookmark my book page and come back every couple of weeks for the latest instalment!