Why the Cabinet Office's £27bn cyber crime cost estimate is meaningless

Today the UK Cabinet Office released a report written by Detica. The report concluded that the annual cost of cyber crime in UK is £27bn. That’s less than $1 trillion, as AT&T’s Ed Amoroso testified before the US Congress in 2009. But it’s still a very large number, approximately 2% of UK GDP. If the total is accurate, then cyber crime is a very serious problem of utmost national importance.

Unfortunately, much of the total cost is based on questionable calculations that are impossible for outsiders to verify. 60% of the total cost is ascribed to intellectual property theft (i.e., business secrets not copied music and films) and espionage. The report does describe a methodology for how it arrived at the figures. However, several key details are lacking. To calculate the IP and espionage losses, the authors first calculated measures of each sector’s value to the economy. Then they qualitatively assessed how lucrative and feasible these attacks would be in each sector.

This is where trouble arises. Based on these assessments, the authors assigned a sector-specific probability of theft, one for the best-, worst- and average cases. Unfortunately, these probabilities are not specified in the report, and no detailed rationale is given for their assignment. Are the probabilities based on surveys of firms that have fallen victim to these particular types of crime? Or is it a number simply pulled from the air based on the hunch of the authors? It is impossible to determine from the report.

Yet these probabilities are absolutely crucial in estimating the true cost of cyber crime. Very small changes to the probabilities could mean the true cost of cyber crime is much smaller or larger. The authors try to account for this by also computing best- and worst-case probabilities, but there is no indication how different these values are, nor how they were derived. Consequently, stating that the true cost of cyber crime lies between the best and worst case scenarios is meaningless. To their credit, the authors essentially admit as much, stating that “the proportion of IP stolen cannot at present be measured with any degree of confidence”. But this is buried on p. 16 of the report, and the headline totals are critically dependent on the proportions selected by the authors.

I applaud the effort to measure the costs of cyber crime. In the past, Richard Clayton and I have estimated the cost of phishing, while Ben Edelman and I have estimated ad revenue attributed to typosquatting. Estimating costs is hard, because outside researchers don’t have access to the same level of information on attacks as the victims do. Estimating the cost of espionage is even harder, because victims may even be unaware that they have been attacked. Nonetheless, when measurements are made, it is essential that the entire methodology and calculations be transparent, so that the decision makers relying on the calculations are not inadvertently misled.

The report’s authors rightly call for increased incident reporting from victims so that more accurate measures may be made in future. In that spirit, I hope that the authors also consider being more forthright about how they computed their own figures.

13 thoughts on “Why the Cabinet Office's £27bn cyber crime cost estimate is meaningless

  1. But it’s not ‘theft’ and that’s the problem! The original data are still there, so it is impossible to discover what information has been seen, copied and eventually used. It may be years after the event before anything shows up as pointing to a security breach.

  2. In response to Will Godfrey, It would still be considered theft if I gained access to ARM’s plans for their newest type of processor, photocopied it and put the original back.

    The author has invested time and money into the research and development of that technology. For me to take that without permission is still theft, more-so if a competitor profits from that theft. In this example, ARM could lose a lot of money if a competitor brought a processor to market with similar or improved capabilities based on their work. As such they are stealing from ARM.

    But the original specifications/plans/etc. are still in ARM’s offices, and the breach may only be noticed when the competitor brings such a chip to market.

  3. Turner, I think that you are wrong.

    At least under British law(theft act 1968, as amended in 1978 and 1996), theft involves the intent to permanently deprive the owner of possession of the property that is stolen. Most car thefts in the UK are prosecuted as the separate, vehicle-specific offence of “taking without owner’s consent”, precisely because proving the intent to never return the vehicle is very hard.

    In the case of copying intellectual property, you are not depriving the rightful owner of the property that you take, and therefore it is NOT theft. Copying IP is frequently a harmful tort, for the reasons that you describe above, but despite what you might hear from the marketing departments of many large copyright holders it fails to meet the criteria for being theft.

  4. You ask: “Are the probabilities based on surveys of firms that have fallen victim to these particular types of crime? Or is it a number simply pulled from the air based on the hunch of the authors?”

    The summary report says: “Our assessments are,
    necessarily, based on assumptions and informed judgements rather than specific examples of cyber crime, or from data of a
    classified or commercially-sensitive origin.”

    I think that answers your questions, i.e. they made it up.

  5. @ Nicko – Theft Act 1968, s. 6(1):

    ‘A person appropriating property belonging to another without meaning the other permanently to lose the thing itself is nevertheless to be regarded as having the intention of permanently depriving the other of it if his intention is to treat the thing as his own to dispose of regardless of the other’s rights…’

  6. PS I agree that the report shows every sign of being methodologically unsound. It says that it has guessed the motivation of criminals to target different industries. But it doesn’t mention that it also apparently guesses how many criminals are doing this and how successful they are.

  7. I see Turner’s “would be considered theft” line a lot and this naturally raises the question “it would be considered theft by whom?” I consider my neighbour’s cat to be an evil little beast but my opinion does not make it so.

    I feel that, since theft is a clearly defined legal term, unless “whom” is (ideally) a judge or some other impartial legal expert we’re in the same position as someone saying “it would be considered a perpetual motion machine” — and unless it’s a physicist talking what does that possibly matter?

  8. Hmm. I think I should have worded my comment slightly differently! The point I was trying to make is that it is impossible to discover what information has been ‘removed’.

    As a comparison, of you find your front door lock broken you can quite quickly look around and see what, if anything, has been taken. Even if someone broke in leaving no trace you will quite soon realise that valuable items are missing and then to a thorough search.

    With a cyber breach you have no way of telling what, if any, information is now in another persons hands. Also, if a skilled stealth entry was made you will have no ‘missing’ items to alert you to the fact that all is not well.

  9. I think there’s a good chance this report will hurt Detica in the long term. It is a very poorly written report from someone who is supposed to be professional. My first impression is that it was a “small piece of work” that would pay for a couple of weeks of a junior consultant – I hope I am right.
    I have to agree with the author that the worst aspect is that the report looks like an official government material.

  10. You could certainly say it is theft of copyright.

    You don’t have to take any thing to break and enter and be labelled a thief. You can thieve copyright without going near a computer or taking anything. Breaking and entering is illegal physically and on a computer. Unfortunately you have to warn an electronic intruder for you to have any chance of making it stand up in court and computer access is much more difficult to prove or notice.

    Anyway, the point is mute, you have to be as proactive as you can afford, unfortunately your security often depends on other peoples security especially in business and the judicial system works around money and so is abused (6 out of 10 barristers come form rich families and so are often out of touch with reality’s) . Patenting only helps if you can afford to protect it and may just allow people to steal your copyright without even breaking and entering. Ironically if you don’t patent, you may have problems when you can afford to defend that patent.

  11. Yeah, no. The 1968 Theft Act citation isn’t talking about so-called IP in that section. It’s ensuring that various ways of dodging the spirit of the “permanently to deprive” restriction are eliminated, e.g. you can’t steal something and then sell it back to the legitimate owner and defend that as not “permanently” because they got it back after paying you.

    “Moot” not “mute” is the legal term Kevin wanted, and he also doesn’t seem to understand the difference between copyright and patent or indeed what either of them do.

    I suspect that a blog about evidence and statistics would be a better place for this topic than Light Blue Touchpaper because the audience wouldn’t get bogged down in technicalities but would focus on the actual problem of false certainty. The government now thinks it knows something when it doesn’t. We either need better evidence or better handling of the existing poor quality evidence.

  12. @Nick Lamb: I completely agree that the big problem is the false certainty that the report offers to policy makers. By including “best” and “worst” case estimates (and then suggesting that their estimate is likely on the low end to the press), the report is defining the scale of the cyber crime problem without a shred of credible empirical evidence.

  13. @Kevin:

    You do have to take something to be labeled a thief within the precise meaning of the term ‘theft’ that’s used in English Law. Of course in other contexts the word theft has a whole range of meaning dependant on who’s using it.

    You could be guilty of burglary without being guilty of theft.

    You can’t thieve a copyright – you can infringe it. It’s nothing to do with the offence of theft.

    The case of Oxford V Moss established that there is no such thing in English law as theft of information. (The defendant was acquitted of theft having taken an exam paper to read before the exam. He was planning to return the physical paper, and therefore hadn’t stolen it, and the information on the paper was not considered property that could be stolen. See http://en.wikipedia.org/wiki/Oxford_v_Moss).

    Taking information without permission may be an offence in various other ways, depending on the circumstances. If you take it by using someone’s computer system in an unauthorised way then it would probably be an offence under the Computer Misuse Act

Leave a Reply

Your email address will not be published. Required fields are marked *