February 9th, 2011 at 19:11 UTC by Joseph Bonneau
In the aftermath of Anonymous’ revenge hacking of HBGary over the weekend, some enterprising hackers used one of the stolen credentials and some social engineering to gain root access at rootkit.com, which has been down for a few days since. There isn’t much novel about the hack but the dump of rootkit.com’s SQL databases provides another password dataset for research, though an order of magnitude smaller than the Gawker dataset with just 81,000 hashed passwords.
More interestingly, due to the close proximity of the hacks, we can compare the passwords associated with email addresses registered at both Gawker and rootkit.com. This gives an interesting data point on the widely known problem of password re-use. This new data seems to indicate a significantly higher re-use rate than the few previously published estimates.
A simple intersection yielded 522 email addresses registered at both sites. This is about a 1% overlap, small but reasonable given the very different niches of the two websites. Eliminating throwaway addresses from sites like Mailinator and dubious addresses like firstname.lastname@example.org (it’s not clear that either site properly checked the validity of enrolled emails) left 456 pairs.
Analysing password re-use requires inverting the hashed passwords since the sites used different hash algorithms (and Gawker minimally salted their hashes). Rootkit.com’s password implementation is worse than Gawker’s, with no salts at all and just a single iteration of MD5, meaning it’s quick to test a huge dictionary of known passwords. I cracked 44% of the accounts using a dictionary of about 10 M entries in less than 5 minutes. I previously used the same dictionary on the Gawker dataset and cracked 54% of the accounts (despite this, the passwords at rootkit.com were generally weaker, with many more being from a smaller list of common passwords).
Of the 456 common users, 161 had their password cracked in both datasets, 46 only had their rootkit.com password cracked and 77 only had their Gawker password cracked, leaving 172 with neither password cracked. Of the accounts for which passwords were cracked at both sites, 76% used the exact same password. A further 6% used passwords differing by only capitalisation or a small suffix (e.g. ‘password’ and ‘password1′). Some of these were due to the use of crypt() at Gawker, which truncated longer passwords to 8 characters. The remainder appeared to use unrelated passwords and I saw no site-specific password tailoring such as ‘gawker-password’ and ‘rootkit-password’.
This isn’t an accurate estimate, however, because none of the users whose password was cracked at only one site could have reused the same password (since the same dictionary was used). Including these numbers, the apparent re-use rate is only 43%. If we include the similar passwords, and assume that 6% of the passwords cracked at one site but not the other were also similar but one variation was not in our dictionary, we would estimate 49% of users employed very similar passwords between the two sites.
This still isn’t quite a complete comparison because we’ve ignore the 172 users with neither password cracked. We might assume that a roughly similar proportion of these users reused their passwords. It’s likely though that these more security-conscious users had a lower re-use rate, meaning 49% is an over-estimate. Still, we have to estimate at least a 31% re-use rate even if none of this last group of users reused the same password.
Either rate is much higher than what we would estimate based on the best published studies – Flôrencio and Herley’s empirical study (about 12%) or Gaw and Felten’s user survey (about 20%). Sampling error due to random chance shouldn’t be more than about ±5%, which can’t explain the difference. It could be that users are much more likely to reuse a password between Gawker and rootkit.com, since both protect access to forums and are of relatively low value. It could also indicate that password re-use has risen significantly in the past 5 years (which Gaw and Felten specifically predicted based on their survey).
More data is clearly needed because the difference between a 10% re-use rate and a 50% re-use rate would change the economics of large-scale attacks. It would also be very interesting to study the password overlap between higher-value accounts, such as those with a large email provider or an online bank, with low-security accounts like Gawker and rootkit.com which are more likely to be compromised.