Today the UK Cabinet Office released a report written by Detica. The report concluded that the annual cost of cyber crime in UK is £27bn. That’s less than $1 trillion, as AT&T’s Ed Amoroso testified before the US Congress in 2009. But it’s still a very large number, approximately 2% of UK GDP. If the total is accurate, then cyber crime is a very serious problem of utmost national importance.
Unfortunately, much of the total cost is based on questionable calculations that are impossible for outsiders to verify. 60% of the total cost is ascribed to intellectual property theft (i.e., business secrets not copied music and films) and espionage. The report does describe a methodology for how it arrived at the figures. However, several key details are lacking. To calculate the IP and espionage losses, the authors first calculated measures of each sector’s value to the economy. Then they qualitatively assessed how lucrative and feasible these attacks would be in each sector.
This is where trouble arises. Based on these assessments, the authors assigned a sector-specific probability of theft, one for the best-, worst- and average cases. Unfortunately, these probabilities are not specified in the report, and no detailed rationale is given for their assignment. Are the probabilities based on surveys of firms that have fallen victim to these particular types of crime? Or is it a number simply pulled from the air based on the hunch of the authors? It is impossible to determine from the report.
Yet these probabilities are absolutely crucial in estimating the true cost of cyber crime. Very small changes to the probabilities could mean the true cost of cyber crime is much smaller or larger. The authors try to account for this by also computing best- and worst-case probabilities, but there is no indication how different these values are, nor how they were derived. Consequently, stating that the true cost of cyber crime lies between the best and worst case scenarios is meaningless. To their credit, the authors essentially admit as much, stating that “the proportion of IP stolen cannot at present be measured with any degree of confidence”. But this is buried on p. 16 of the report, and the headline totals are critically dependent on the proportions selected by the authors.
I applaud the effort to measure the costs of cyber crime. In the past, Richard Clayton and I have estimated the cost of phishing, while Ben Edelman and I have estimated ad revenue attributed to typosquatting. Estimating costs is hard, because outside researchers don’t have access to the same level of information on attacks as the victims do. Estimating the cost of espionage is even harder, because victims may even be unaware that they have been attacked. Nonetheless, when measurements are made, it is essential that the entire methodology and calculations be transparent, so that the decision makers relying on the calculations are not inadvertently misled.
The report’s authors rightly call for increased incident reporting from victims so that more accurate measures may be made in future. In that spirit, I hope that the authors also consider being more forthright about how they computed their own figures.