Category Archives: News coverage

Media reports that may interest you

Compartmentation is hard, but the Big Data playbook makes it harder still

A new study of Palantir’s systems and business methods makes sobering reading for people interested in what big data means for privacy.

Privacy scales badly. It’s OK for the twenty staff at a medical practice to have access to the records of the ten thousand patients registered there, but when you build a centralised system that lets every doctor and nurse in the country see every patient’s record, things go wrong. There are even sharper concerns in the world of intelligence, which agencies try to manage using compartmentation: really sensitive information is often put in a compartment that’s restricted to a handful of staff. But such systems are hard to build and maintain. Readers of my book chapter on the subject will recall that while US Naval Intelligence struggled to manage millions of compartments, the CIA let more of their staff see more stuff – whereupon Aldrich Ames betrayed their agents to the Russians.

After 9/11, the intelligence community moved towards the CIA model, in the hope that with fewer compartments they’d be better able to prevent future attacks. We predicted trouble, and Snowden duly came along. As for civilian agencies such as Britain’s NHS and police, no serious effort was made to protect personal privacy by compartmentation, with multiple consequences.

Palantir’s systems were developed to help the intelligence community link, fuse and visualise data from multiple sources, and are now sold to police forces too. It should surprise no-one to learn that they do not compartment information properly, whether within a single force or even between forces. The organised crime squad’s secret informants can thus become visible to traffic cops, and even to cops in other forces, with tragically predictable consequences. Fixing this is hard, as Palantir’s market advantage comes from network effects and the resulting scale. The more police forces they sign up the more data they have, and the larger they grow the more third-party databases they integrate, leaving private-sector competitors even further behind.

This much we could have predicted from first principles but the details of how Palantir operates, and what police forces dislike about it, are worth studying.

What might be the appropriate public-policy response? Well, the best analysis of competition policy in the presence of network effects is probably Lina Khan’s, and her analysis would suggest in this case that police intelligence should be a regulated utility. We should develop those capabilities that are actually needed, and the right place for them is the Police National Database. The public sector is better placed to commit the engineering effort to do compartmentation properly, both there and in other applications where it’s needed, such as the NHS. Good engineering is expensive – but as the Los Angeles Police Department found, engaging Palantir can be more expensive still.

National Audit Office confirms that police, banks, Home Office pass the buck on fraud

The National Audit Office has found as follows:

“For too long, as a low value but high volume crime, online fraud has been overlooked by government, law enforcement and industry. It is now the most commonly experienced crime in England and Wales and demands an urgent response. While the Department is not solely responsible for reducing and preventing online fraud, it is the only body that can oversee the system and lead change. The launch of the Joint Fraud Taskforce in February 2016 was a positive step, but there is still much work to be done. At this stage it is hard to judge that the response to online fraud is proportionate, efficient or effective.”

Our regular readers will recall that over ten years ago the government got the banks to agree with the police that fraud would be reported to the bank first. This ensured that the police and the government could boast of falling fraud figures, while the banks could direct such fraud investigations as did happen. This was roundly criticized by the Science and Technology Committee (here and here) but the government held firm. Over the succeeding decade, dissident criminologists started pointing out that fraud was not falling, just going online like everything else, and the online stuff was being ignored. Successive governments just didn’t want to know; for most of the period in question the Home Secretary was one Theresa May, who so impressed her party by “cutting crime” even though she’d cut 20,000 police jobs that she got a promotion.

But pigeons come home to roost eventually, and over the last two years the Office of National Statistics has been moving to more honest crime figures. The NAO report bears close study by anyone interested in cybercrime, in crime generally, and in how politicians game the crime figures. It makes clear that the Home Office doesn’t know what’s going on (or doesn’t really want to) and hopes that other people (such as banks and the IT industry) will solve the problem.

Government has made one or two token gestures such as setting up Action Fraud, and the NAO piously hopes that the latest such (the Joint Fraud Taskforce) could be beefed up to do some good.

I’m afraid that the NAO’s recommendations are less impressive. Let me give an example. The main online fraud bothering Cambridge University relates to bogus accommodation; about fifty times a year, a new employee or research student turns up to find that the apartment they rented doesn’t exist. This is an organised scam, run by crooks in Germany, that affects students elsewhere in the UK (mostly in London) and is netting £5-10m a year. The cybercrime guy in the Cambridgeshire Constabulary can’t do anything about this as only the National Crime Agency in London is allowed to talk to the German police; but he can’t talk to the NCA directly. He has to go through the Regional Organised Crime Unit in Bedford, who don’t care. The NCA would rather do sexier stuff; they seem to have planned to take over the Serious Fraud Office, as that was in the Conservative manifesto for this year’s election.

Every time we look at why some scam persists, it’s down to the institutional economics – to the way that government and the police forces have arranged their targets, their responsibilities and their reporting lines so as to make problems into somebody else’s problems. The same applies in the private sector; if you complain about fraud on your bank account the bank may simply reply that as their systems are secure, it’s your fault. If they record it at all, it may be as a fraud you attempted to commit against them. And it’s remarkable how high a proportion of people prosecuted under the Computer Misuse Act appear to have annoyed authority, for example by hacking police websites. Why do we civilians not get protected with this level of enthusiasm?

Many people have lobbied for change; LBT readers will recall numerous articles over the last ten years. Which? made a supercomplaint to the Payment Services Regulator, and got the usual bland non-reassurance. Other members of the old establishment were less courteous; the Commissioner of the Met said that fraud was the victims’ fault and GCHQ agreed. Such attitudes hit the poor and minorities the hardest.

The NAO is just as reluctant to engage. At p34 it says of the Home Office “The Department … has to influence partners to take responsibility in the absence of more formal legal or contractual levers.” But we already have the Payment Services Regulations; the FCA explained in response to the Tesco Bank hack that the banks it regulates should make fraud victims good. And it has always been the common-law position that in the absence of gross negligence a banker could not debit his customer’s account without the customer’s mandate. What’s lacking is enforcement. Nobody, from the Home Office through the FCA to the NAO, seems to want to face down the banks. Rather than insisting that they obey the law, the Home Office will spend another £500,000 on a publicity campaign, no doubt to tell us that it’s all our fault really.

Regulatory capture

Today’s newspapers report that the cladding on the Grenfell Tower, which appears to have been a major factor in the dreadful loss of life there, was banned in Germany and permitted in America only for low-rise buildings. It would have cost only £2 more per square meter to use fire-resistant cladding instead.

The tactical way of looking at this is whether the landlords or the builders were negligent, or even guilty of manslaughter, for taking such a risk in order to save £5000 on an £8m renovation job. The strategic approach is to ask why British regulators are so easily bullied by the industries they are supposed to police. There is a whole literature on regulatory capture but Britain seems particularly prone to it.

Regular readers of this blog will recall many cases of British regulators providing the appearance of safety, privacy and security rather than the reality. The Information Commissioner is supposed to regulate privacy but backs away from confronting powerful interests such as the tabloid press or the Department of Health. The Financial Ombudsman Service is supposed to protect customers but mostly sides with the banks instead; the new Payment Systems Regulator seems no better. The MHRA is supposed to regulate the safety of medical devices, yet resists doing anything about infusion pumps, which kill as many people as cars do.

Attempts to fix individual regulators are frustrated by lobbyists, or even by fear of lobbyists. For example, my colleague Harold Thimbleby has done great work on documenting the hazards of infusion pumps; yet when he applied to be a non-executive director of the MHRA he was not even shortlisted. I asked a civil servant who was once responsible for recommending such appointments to the Secretary of State why ministers never seemed to appoint people like Harold who might make a real difference. He replied wearily that ministers would never dream of that as “the drug companies would make too much of a fuss”.

In the wake of this tragedy there are both tactical and strategic questions of blame. Tactically, who decided that it was OK to use flammable cladding on high-rise buildings, when other countries came to a different conclusion? Should organisations be fined, should people be fired, and should anyone go to prison? That’s now a matter for the public inquiry, the police and the courts.

Strategically, why is British regulators so cosy with the industries they regulate, and what can be done about that? My starting point is that the appointment of regulators should no longer be in the gift of ministers. I propose that regulatory appointments be moved from the Cabinet Office to an independent commission, like the Judicial Appointments Commission, but with a statutory duty to hire the people most likely to challenge groupthink and keep the regulator effective. That is a political matter – a matter for all of us.

RIP smart meters

The Telegraph has just run an op-ed they asked me to write over the weekend, after I pointed out here on Friday that the Conservative manifesto had quietly downgraded the smart meter programme to a voluntary one.

Regular readers of Light Blue Touchpaper will have followed the smart meter story for almost a decade, back through the dishonest impact assessment to the fact that they pose a threat to critical infrastructure.

Manifestos and tech

The papers went to town yesterday on the Conservative manifesto but missed some interesting bits.

First, no-one seems to have noticed that the smart meter programme is being quietly put to death. We read on page 60 that everyone will be offered a smart meter by 2020. So a mandatory national programme has become voluntary, just like that. Regular readers of this blog will recall that the programme was sold in 2008 by Ed Milliband using a dishonest impact assessment, yet all the parties backed it after 2010, leaving no-one to point out that it was going to cost us all a fortune and never save any carbon. May says she wants to reduce energy costs; this was surely a no-brainer.

That was the good news for England. The good news for friends in rural Scotland is high-speed broadband for all by 2020. But there are some rather weird things in there too.

What on earth is “the right of businesses to insist on a digital signature”? Digital signatures are very 1998, and we already have the electronic signature directive. From whom will businesses be able to insist on a signature, and if I’m one of the legislated victims, how much do I have to pay to buy the apparatus?

All digital businesses will have “to support new digital proofs of identification”. That presumably means forcing firms to use Verify, a dysfunctional online authentication service whose roots lie in Blair’s obsession with identity. If a newspaper currently identifies its subscribers via a proprietary logon, will they have to offer Verify as an option? Will it have to be the only option, displacing Facebook and Twitter? The manifesto also says that local government will have to use Verify; and elsewhere that councils must publish planning applications and bus routes “without the hassle and delay that currently exists.” OK, so some councils could so with more competent webmasters, but don’t worry: “hundreds of leaders from the world of tech can come into government to help deliver better public services.”

The Land Registry, the Ordnance Survey and other quangos that do geography (our leader’s degree subject) will all band together to create the largest open repository of land data in the world. So where will the Ordnance Survey get its money from then? That small question killed the same idea in 2010 after Tim Berners-Lee sold it to Cameron.

There will be a levy on social media companies, like on gambling companies, to support awareness and preventive activity. And they must not direct users, even unintentionally, to hate speech. So will Facebook be fined whenever they let users like a xenophobic article in the Daily Mail?

No doubt in view of the delicacy of such regulatory decisions, Leveson II is killed; there will be a Data Use and Ethics Commission instead. It will advise regulators and develop the principles and rules that will give people confidence their data are being handled properly. Wow. We now have the Charter of Fundamental Rights to give us principles, the GDPR to give us rules, and the ECJ to hammer out the case law. Now the People don’t have confidence in such experts we’re going to let the Prime Minister of the day appoint a different lot.

The next government will further strengthen cyber security standards for government and public services, so presumably all such services will have to use expensive networks such as the NHS-wide network from BT which will expect them to manage their own firewalls without telling them how to.

But don’t worry. It will become “as difficult to commit a crime digitally as it is physically”. There is text about working “with international law enforcement agencies to ensure perpetrators are brought to justice” but our local police force isn’t allowed to do anything effective about online accommodation fraud committed by a gang in Germany. They have to work through the NCA – who don’t care. The manifesto signals more of the same: the NCA will get to eat the SFO, which does crimes over £100m, leaving them even less interested in online crooks who steal a thousand pounds of deposit from dozens of students a year.

In fact there is no signal anywhere in the manifesto that May understands the impact of volume cybercrime, even though it’s now most of the property crime in the UK. She rather prefers to boast of the falling crime over the past seven years, as if it were her achievement as Home Secretary. The simple fact is that crime has been going online like everything else, and until 2015 the online part of it wasn’t recorded properly. This was not the doing of Theresa May, but of Margaret Hodge.

The manifesto rather seems to have been drafted in a geek-free room. And let’s not spoil the party by mentioning the impact that tight immigration targets will have on the IT industry, or for that matter on higher education. Perhaps they want us to hope that they don’t really mean that part of it, but perhaps we’d better make a plan to open a campus in India or Canada, just in case.

Bad malware, worse reporting

The Wannacry malware that has infected some UK hospital computers should interest not just security researchers but also people interested in what drives fake news.

Some made errors of fact: the Daily Mail inititally reported the ransom demand as 300 bitcoin, or £415,000, rather than $300 in bitcoin. Others made errors of logic: the Indy, for example, reported that “Up to 90 percent of NHS computers still run XP, released in 2001”, citing as its source a BMJ article which stated that 90% of trusts run this version of Windows. And some made errors of concurrency. After dinner I found inquiries from journalists about my fight with the Prime Minister. My what? Eventually I found that the Guardian had followed something Mrs May’s spokesman had said (“not aware of any evidence that patient data has been compromised”) with something I’d said a couple of hours earlier (“The NHS are saying that patient privacy hasn’t been compromised, but if significant numbers of hospitals have been negligently running unpatched computers for two months after the patch came out, how do they know?”). The Home Secretary later helpfully glossed the PM’s stonewall as “No patient data has been accessed or transferred in any way” but leaving the get-out-of-jail card “that’s the information we’ve been given.”

Many papers caught the international political aspect: that the vulnerability was discovered by the NSA, kept secret rather than fixed (contrary to the advice of Obama’s NSA review group), then stolen from the CIA by the Russians and published via wikileaks. Scary stuff, eh? And we read of some surprising overreactions, such as the GP who switched off his networking as a precaution and found he couldn’t access any of his patients’ records.

As luck would have it, yesterday was the day that I gave my talk on entomology – the classification of software bugs and other security vulnerabilities – to my first-year security and software engineering class. So let’s try to look at it calmly as I’d expect of a student writing an assignment.

The first point is that there’s not a really lot of this malware. The NHS has over 200 hospitals, and the typical IT director is a senior clinician supported by technicians. Yet despite having their IT run by well-meaning amateurs, only 16 NHS organisations have been hit, according to the Register and Kaspersky – including several hospitals.

So the second point is that when the Indy says that “The NHS is a perfect combination of sensitive data and insecure storage. And there’s very little they can do about it” the answer is simple: in well over 90% of NHS organisations, the well-meaning amateurs managed perfectly well. What they did was to keep their systems patched up-to-date; simple hygiene, like washing your hands after going to the toilet.

The third takeaway is that it’s worth looking at the actual code. A UK researcher did so and discovered a kill switch.

Now I am just listening on the BBC morning news to a former deputy director of GCHQ who first cautions against alarmist headlines and argues that everyone develops malware; that a patch had been issued by Microsoft halfway through March; that you can deal with ransomware by keeping decent backups; and that paying ransom will embolden the bad guys. However he claims that it’s clearly an organised criminal attack. (when it could be one guy in his bedroom somewhere) and says that the NCSC should look at whether there is some countermeasure that everyone should have taken (for answer see above).

So our fourth takeaway is that although the details matter, so do the economics of security. When something unexpected happens, you should not just get your head down and look at the code, but look up and observe people’s agendas. Politicians duck and weave; NHS managers blame the system rather than step up to the plate; the NHS as a whole turns every incident into a plea for more money; the spooks want to avoid responsibility for the abuse of their stolen cyberweaponz, but still big up the threat and get more influence for a part of their agency that’s presented as solely defensive. And we academics? Hey, we just want the students to pay attention to what we’re teaching them.

Hope this helps!

Video on Edge

John Brockman of Edge interviewed me in London in March. The video of the interview, and a transcript, are now available on the Edge website. Edge runs big interviews with several dozen scientists a year, with particular interest in people who do cross-disciplinary work. For me, the interaction of economics, psychology and engineering is one of the things that makes security so fascinating, as well as the creativity driven by adversarial behaviour.

The topics covered include the last thirty years of progress (of lack of it) in information security, from the early beginnings, through the crypto wars and crime moving online, to the economics of security. We talked about how cryptography can help less developed countries; about managing complexity in big projects; about how network effects lead firms to design insecure products; about whether big data can undermine democracy by empowering elites; and about how in a future world of intelligent things, security may become more about safety than anything else. Finally I talk about our current big project, the Cambridge Cybercrime Centre.

John runs a literary agency, and he’s worked on books by many of the scientists who feature on his site. This makes me wonder: on what topic should I write my next book?

Banks biased against black fraud victims

The following is an op-ed I wrote in today’s Times. It appeared in their Thunderer column.

You’re less likely to be treated fairly by your bank if you’re elderly, poor, female or black. We’ve suspected this for years, and finally The Times has dug up the numbers to prove it.

Fraud victims who’re refused compensation often contact our security research group at Cambridge after they find we work on payment fraud. We call this stream of complaints our ‘fraud telescope’ as it gives us early warning of what the bad guys are up to. We’ve had more than 2,000 cases over 25 years.

In recent years we’ve started to realise what we weren’t seeing. The “dark matter” in the fraud universe is the missing victims: we don’t see that many middle-class white men. The victims who do come to us are disproportionately elderly, poor, female, or black. But crime surveys tell us that the middle classes and the young are more likely to be victims of fraud, so it’s hard to avoid the conclusion that banks are less generous to some of their customers.

We raised the issue of discrimination in 2011 with one of the banks and with the Commission for Racial Equality, but as no-one was keeping records, nothing could be proved, until today.

How can this discrimination happen? Well, UK rules give banks a lot of discretion to decide whether to refund a victim, and the first responders often don’t know the full story. If your HSBC card was compromised by a skimmer on a Tesco ATM, there’s no guarantee that Tesco will have told anyone (unlike in America, where the law forces Tesco to tell you). And the fraud pattern might be something entirely new. So bank staff end up making judgement calls like “Is this customer telling the truth?” and “How much is their business worth to us?” This in turn sets the stage for biases and prejudices to kick in, however subconsciously. Add management pressure to cut costs, sometimes even bonuses for cutting them, and here we are.

There are two lessons. First, banks need to train staff to be aware of unconscious bias (as universities do), and monitor their performance.

Second, the Financial Conduct Authority needs to protect all customers properly. It seems to be moving in the right direction; after the recent fraud against tens of thousands of Tesco Bank account holders, it said it expected fraud victims to be made good immediately. This has been the law in the USA since the 1980s and it must become a firm rule here too.

Government U-turn on Health Privacy

Now that everyone’s distracted with the supreme court case on Brexit, you can expect the government to sneak out something it’s ashamed of. Health secretary Jeremy Hunt has decided to ignore the wishes of over a million people who opted out of having their hospital records given to third parties such as drug companies, and the ICO has decided to pretend that the anonymisation mechanisms he says he’ll use instead are sufficient. One gently smoking gun is the fifth bullet in a new webpage here, where the Department of Health claims that when it says the data are anonymous, your wishes will be ignored. The news has been broken in an article in the Health Services Journal (it’s behind a paywall, as a splendid example of transparency) with the Wellcome Trust praising the ICO’s decision not to take action against the Department. We are assured that “the data is seen as crucial for vital research projects”. The exchange of letters with privacy campaigners that led up to this decision can be found here, here, here, here, here, here, and here.

An early portent of this u-turn was reported here in 2014 when officials reckoned that the only way they could still do administrative tasks such as calculating doctors’ bonuses was to just pretend that the data are anonymous even though they know it isn’t really. Then, after the care.data scandal showed that a billion records had been sold to over a thousand purchasers, we reported here how HES data had also been sold and how the minister seemed to have misled parliament about this.

I will be talking about ethics of all this on Thursday. Even if ministers claim that stolen medical records are OK to use, researchers must not act as if this is true; if patients end up trusting doctors as little as we trust politicians, then medical research will be in serious trouble. There is a video of a previous version of this talk here.

Meanwhile, if you’re annoyed that Jeremy Hunt proposes to ignore not just your privacy rights but your express wishes, you can send him a notice under Section 10 of the Data Protection Act forbidding him from disclosing your data. The Department has complied with such notices in the past, albeit with bad grace as they have no automated way to do it. If thousands of people serve such notices, they may finally have to stand up to the drug company lobbyists and write the missing software. For more, see here.