In two weeks’ time we’re starting an open course in security economics. I’m teaching this together with Rainer Boehme, Tyler Moore, Michel van Eeten, Carlos Ganan, Sophie van der Zee and David Modic.
Over the past fifteen years, we’ve come to realise that many information security failures arise from poor incentives. If Alice guards a system while Bob pays the cost of failure, things can be expected to go wrong. Security economics is now an important research topic: you can’t design secure systems involving multiple principals if you can’t get the incentives right. And it goes way beyond computer science. Without understanding how incentives play out, you can’t expect to make decent policy on cybercrime, on consumer protection or indeed on protecting critical national infrastructure
We first did the course last year as a paid-for course with EdX. Our agreement with them was that they’d charge for it the first time, to recoup the production costs, and thereafter it would be free.
So here it is as a free course. Spread the word!
At our security group meeting on the 19th August, Sergei Skorobogatov demonstrated a NAND backup attack on an iPhone 5c. I typed in six wrong PINs and it locked; he removed the flash chip (which he’d desoldered and led out to a socket); he erased and restored the changed pages; he put it back in the phone; and I was able to enter a further six wrong PINs.
Sergei has today released a paper describing the attack.
During the recent fight between the FBI and Apple, FBI Director Jim Comey said this kind of attack wouldn’t work.
When Lying Feels the Right Thing to Do reports three studies we did on what made people less or more likely to submit fraudulent insurance claims. Our first study found that people were more likely to cheat when rejected; the other two showed that rejected claimants were just as likely to cheat when this didn’t lead to financial gain, but that they felt more strongly when there was no money involved.
Our research was conducted as part of a broader research programme to investigate the deterrence of deception; our goal was to understand how to design better websites. However we can’t help wondering whether it might shine some light on the UK’s recent political turmoil. The Brexit campaigners were minorities of both main political parties and their anti-EU rhetoric had been rejected by the political mainstream for years; they had ideological rather than selfish motives. They ran a blatantly deceptive campaign, persisting in obvious untruths but abandoning them promptly after winning the vote. Rejection is not the only known factor in situational deception; it’s known, for example, that people with unmet goals are more likely to cheat than people who are simply doing their best, and that one bad apple can have a cascading effect. But it still makes you think.
The outcome and aftermath of the referendum have left many people feeling rejected, from remain voters through people who will lose financially to foreign residents of the UK. Our research shows that feelings of rejection can increase cheating by 15-30%; perhaps this might have measurable effects in some sectors. How one might disentangle this from the broader effects of diminished social solidarity, and from politicians simply setting a bad example, could be an interesting problems for social scientists.
The Royal Society has just published a report on cybersecurity research. I was a member of the steering group that tried to keep the policy team headed in the right direction. Its recommendation that governments preserve the robustness of encryption is welcome enough, given the new Russian law on access to crypto keys; it was nice to get, given the conservative nature of the Society. But I’m afraid the glass is only half full.
I was disappointed that the final report went along with the GCHQ line that security breaches should not be reported to affected data subjects, as in the USA, but to the agencies, as mandated in the EU’s NIS directive. Its call for an independent review of the UK’s cybersecurity needs may also achieve little. I was on John Beddington’s Blackett Review five years ago, and the outcome wasn’t published; it was mostly used to justify a budget increase for GCHQ. Its call for UK government work on standards is irrelevant post-Brexit; indeed standards made in Europe will probably be better without UK interference. Most of all, I cannot accept the report’s line that the government should help direct cybersecurity research. Most scientists agree that too much money already goes into directed programmes and not enough into responsive-mode and curiosity-driven research. In the case of security research there is a further factor: the stark conflict of interest between bona fide researchers, whose aim is that some of the people should enjoy some security and privacy some of the time, and agencies engaged in programmes such as Operation Bullrun whose goal is that this should not happen. GCHQ may want a “more responsive cybersecurity agenda”; but that’s the last thing people like me want them to have.
The report has in any case been overtaken by events. First, Brexit is already doing serious harm to research funding. Second, Brexit is also doing serious harm to the IT industry; we hear daily of listings posptoned, investments reconsidered and firms planning to move development teams and data overseas. Third, the Investigatory Powers bill currently before the House of Lords highlights the fact that surveillance debate in the West these days is more about access to data at rest and about whether the government can order firms to hack their customers.
While all three arms of the US government have drawn back on surveillance powers following the Snowden revelations, Theresa May has taken the hardest possible line. Her Investigatory Powers Bill will give her successors as Home Secretary sweeping powers to order firms in the UK to hand over data and help GCHQ hack their customers. Brexit will shield these powers from challenge in the European Court of Justice, making it much harder for a UK company to claim “adequacy” for its data protection arrangements in respect of EU data subjects. This will make it still less attractive for an IT company to keep in the UK either data that could be seized or engineering staff who could be coerced. I am seriously concerned that, together with Brexit, this will be the double whammy that persuades overseas firms not to invest in the UK, and that even causes some UK firms to leave. In the face of this massive self-harm, the measures suggested by the report are unlikely to help much.
If the UK leaves the European Union, it will cost Cambridge University about £100m, or about 10% of our turnover.
I present the details in an article today in the Cambridge News.
I reckon we will lose at least £60m of the £69m we get in European grants, at least £20m of our £237m fee income (most of which is from foreign students), at least £10m from Cambridge Assessment and Cambridge University Press, and £5m each from industry and charities. Although I’m an elected member of Council (the governing body) and the committee that sets the budget, all this comes from our published accounts.
And my estimates are conservative; the outcome could easily be worse, especially if foreign students desert us, or just can’t get visas after a popular vote against immigration.
Now everyone on Britain pays on average £4 a year to the EU and gets £2 back. The net contribution of £2 amounts to £12.5m for a town the size of Cambridge. The University alone is getting more than four times that back directly, and yet more indirectly. And the same goes for many other university towns too; even Newcastle gets more than would be raised by everyone in the city paying £2 a year.
But this is not just about money; it’s about who we are, and also about what other people perceive us to be. If Britain votes to leave Europe following a xenophobic campaign against immigrants, people overseas may conclude that Britain is to longer a cool place to study, or to start a research lab. Even some of the people already here will leave. We will do the best we can to keep the flame alight, but it will be very much harder for Cambridge to remain a world-leading university.
See also the Cambridge News editorial, and my piece yesterday on Brexit and tech.
The debate on whether Britain should leave the EU has largely ignored a factor of huge importance to the tech industry – network effects.
So I’ve written an article on what Brexit means for the tech industry from the viewpoint of information economics.
Network effects mean that the value of a transaction often depends on how many other people make similar transactions. They make our industry prone to monopolies. They ensure that the UK, with 1% of world population and 3% of GDP, has little influence on tech markets, which are mostly global. But the EU has real clout; Silicon Valley sees it as the world privacy regulator, as Washington doesn’t care and no-one else is big enough to matter. And most of the other regulations that IT people find annoying, from IP laws to export controls, are also embedded in international treaties. We can’t just tear up the annoying “red tape”, as the Brexit crowd suggest.
Brexit would not only diminish our influence on the laws that affect tech – many of which reflect negative network effects. It would make startups more expensive, so UK firms would have a harder time exploiting the positive network effects that are often the key to success. And it would damage the successful tech clusters we do have in Cambridge and in London.
Tech clusters need a number of things to thrive; and it’s not just technical network effects that matter, but labour-market network effects too. And there’s quite a lot of research on that. As good engineers can earn good money and live wherever we want, we congregate in places that are good places to live. They are always open and liberal places, where it’s fine to be from an ethnic minority, or an immigrant, or gay. What would the world’s best and brightest engineers think about moving to Britain if we vote for xenophobia on Thursday?
The article is in Computer Weekly, and there’s also a pdf here.
I’m liveblogging the Workshop on Security and Human Behaviour which is being held in Harvard. The programme is here. For background, see the liveblogs for SHB 2008-15 which are linked here and here. Blog posts summarising the talks at the workshop sessions will appear as followups below.
We recently reported that the Commissioner of the Met, Sir Bernard Hogan-Howe, said that banks should not refund fraud victims as this would just make people careless with their passwords and antivirus. The banks’ desire to blame fraud victims if they can, to avoid refunding them, is rational enough, but for a police chief to support them was disgraceful. Thirty years ago, a chief constable might have said that rape victims had themselves to blame for wearing nice clothes; if he were to say that nowadays, he’d be sacked. Hogan-Howe’s view of bank fraud is just as uninformed, and just as offensive to victims.
Our spooky friends at Cheltenham have joined the party. The Register reports a story in the Financial Times (behind a paywall) which says GCHQ believes that “companies must do more to try and encourage their customers to improve their cyber security standards. Customers using outdated software – sometimes riddled with vulnerabilities that hackers can exploit – are a weak link in the UK’s cyber defences.” There is no mention of the banks’ own outdated technology, or of GCHQ’s role in keeping consumer software vulnerable.
The elegant scribblers at the Financial Times are under the impression that “At present, banks routinely cover the cost of fraud, regardless of blame.” So they clearly are not regular readers of Light Blue Touchpaper.
The spooks are slightly more cautious; according to the FT, GCHQ “has told the private sector it will not take responsibility for regulatory failings”. I’m sure the banks will heave a big sigh of relief that their cosy relationship with the police, the ombudsman and the FCA will not be disturbed.
We will have to change our security-economics teaching material so we don’t just talk about the case where “Alice guards a system and Bob pays the costs of failure”, but also this new case where “Alice guards a system, and bribes the government to compel Bob to pay the costs of failure.” Now we know how Hogan-Howe is paid off; the banks pay for his Dedicated Card and Payment Crime Unit. But how are they paying off GCHQ, and what else are they getting as part of the deal?
Commissioner Hogan-Howe of the Met said on Thursday that the banks should not refund fraud victims because it “rewards” them for being lax about internet security. This was too much to pass up, so I wrote a letter to the editor of the Times, which has just been published. As the Times is behind a paywall, here is the text.
Sir, Sir Bernard Hogan-Howe argues that banks should not refund online fraud victims as this would make people careless with their passwords and anti-virus software (p1, March 24, and letters Mar 25 & 26). This is called secondary victimisation. Thirty years ago, a chief constable might have said that rape victims had themselves to blame for wearing nice clothes; if he were to say that nowadays, he’d be sacked. Hogan-Howe’s view of bank fraud is just as uninformed, and just as offensive to victims.
About 5 percent of computers running Windows are infected with malware, and common bank fraud malware such as Zeus lets the fraudster redirect transactions. You think you’re paying £150 to your electricity bill, while the malware is actually sending £9000 to Russia. The average person is helpless against this; everything seems normal, and antivirus products usually only detect it afterwards.
Much of the blame lies with the banks, who let the users of potentially infected computers make large payments instantly, rather than after a day or two, as used to be the case. They take this risk because regulators let them dump much of the cost of the resulting fraud on customers.
The elephant in the room is that the Met has been claiming for years that property crime is falling, when in fact it’s just going online like everything else. We’re now starting to get better crime figures; it’s time we got better policing, and better bank regulation too.
Ross Anderson FRS FREng
Professor of Security Engineering
University of Cambridge
I will be trying to liveblog Financial Cryptography 2016, which is the twentieth anniversary of the conference. The opening keynote was by David Chaum, who invented digital cash over thirty years ago. From then until the first FC people believed that cryptography could enable commerce and also protect privacy; since then pessimism has slowly set in, and sometimes it seems that although we’re still fighting tactical battles, we’ve lost the war. Since Snowden people have little faith in online privacy, and now we see Tim Cook in a position to decide which seventy phones to open. Is there a way to fight back against a global adversary whose policy is “full take”, and where traffic data can be taken with no legal restraint whatsoever? That is now the threat model for designers of anonymity systems. He argues that in addition to a large anonymity set, a future social media system will need a fixed set of servers in order to keep end-to-end latency within what chat users expect. As with DNS we should have servers operated by (say ten) different principals; unlike in that case we don’t want to have most of the independent parties financed by the US government. The root servers could be implemented as unattended seismic observatories, as reported by Simmons in the arms control context; such devices are fairly easy to tamper-proof.
The crypto problem is how to do multi-jurisdiction message processing that protects not just content but also metadata. Systems like Tor cost latency, while multi-party computation costs a lot of cycles. His new design, PrivaTegrity, takes low-latency crypto building blocks then layers on top of them transaction protocols with large anonymity sets. The key component is c-Mix, whose spec up as an eprint here. There’s a precomputation using homomorphic encryption to set up paths and keys; in real-time operations each participating phone has a shared secret with each mix server so things can run at chat speed. A PrivaTegrity message is four c-Mix batches that use the same permutation. Message models supported include not just chat but publishing short anonymous messages, providing an untraceable return address so people can contact you anonymously, group chat, and limiting sybils by preventing more than one pseudonym being used. (There are enduring pseudonyms with valuable credentials.) It can handle large payloads using private information retrieval, and also do pseudonymous digital transactions with a latency of two seconds rather than the hour or so that bitcoin takes. The anonymous payment system has the property that the payer has proof of what he paid to whom, while the recipient has no proof of who paid him; that’s exactly what corrupt officials, money launderers and the like don’t want, but exactly what we do want from the viewpoint of consumer protection. He sees PrivaTegrity as the foundation of a “polyculture” of secure computing from multiple vendors that could be outside the control of governments once more. In questions, Adi Shamir questioned whether such an ecosystem was consistent with the reality of pervasive software vulnerabilities, regardless of the strength of the cryptography.
I will try to liveblog later sessions as followups to this post.