Earlier this month, I blogged about monitoring password-guessing attacks on a server, via a patched OpenSSH. This experiment has now been running for just over two weeks, and there are some interesting results. I’ve been tweeting these since the start.

As expected, the vast majority of password-guessing attempts are quite dull, and fall into one of two categories. Firstly there are attempts with a large number of ‘poor’ passwords (e.g. “password”, “1234″, etc…) against a small number of accounts which are very likely to exist (almost always “root”, but sometimes others such as “bin”).

Secondly, there were attempts on a large number of accounts which might plausibly exist (e.g. common first names and software packages such as ‘oracle’). For these, there were a very small number of password attempts, normally only trying the username as password. Well established good practices such as choosing a reasonably strong password and denying password-based log-in to the root account will be effective against both categories of attacks. Surprisingly, there were few attempts which were obviously default passwords from software packages (but they perhaps were hidden in the attempts where username equalled password). However, one attempt was username: “rfmngr”, password: “$rfmngr$”, which is the default password for Websense RiskFilter (see p.10 of the manual).

There were, however, some more interesting attempts. One category was passwords far too complicated to be in a standard password dictionary, or even found through offline-brute-force attacks on a hashed password database (e.g. “TiganilAFloriNTeleormaN”, “Fum4tulP0@t3Uc1d3R4uD3T0t!@#$%^%^&*?”, and “kx028897chebeuname+a”). The best guess is that these passwords were collected from an unhashed password database, or from a trojaned SSH server or client. Theo Markettos identified a likely source for this password database. Other odd password attempts include plain hashes (e.g. E4F89B211D997C1D5ECCE2153DC9184A which is the MD5 of “upintheair”, found by Google), salted hashes (e.g. $1$EdkQIoSn$T3gzKLxlcxF7tsTCFqC8M) and filenames (e.g. “/var/run/sshd22.pid” and “/var/run/sshd”).

One conclusion which can be drawn is that this attacker does not care enough about the quality of the password database to filter out passwords which it makes almost no sense to use. This carelessness is supported by the fact that after I initially enabled my patched SSH server, I received many log-in attempts but no passwords. It turned out that the default FreeBSD configuration is to only support keyboard-interactive authentication, rather than the more limited password authentication. The brute force attack tool only attempted password authentication, and therefore was always rejected before any password was sent, so the attack was running for days without ever having a hope of succeeding. I did enable password authentication, but some later attacks, presumably using a different tool and probably from a different attacker, attempted both keyboard-interactive and password authentication.

One attack I hadn’t seen before was to try a large number of usernames, and parts of the hostname as password. For a hostname of the style MACHINE.DOMAIN.DEPARTMENT.cam.ac.uk, the attack tried DOMAIN, DOMAIN.DEPARTMENT, MACHINE, then MACHINE.DOMAIN. This clearly isn’t a dictionary but a bit of custom code which did a reverse DNS lookup on this host then generated some possible passwords. Using the hostname as a password for a host isn’t a good idea, but I can imagine some sysadmins doing so. The fact that some attackers are taking this approach might merit some explicit statement in password selection guidance.

Another curious trend was receiving meta-data as username/passwords. This might be due to the brute force tool not properly interpreting comments in the dictionary file, or the attacker not understanding the comment notation. For example I received the following username/passwords:

• [uratu/was HERE]
• [I`m/A HaCkER ON]
• [This/Is A Blow ShiT]
• [acest/este:varza]
• [data.conf/contzine]
• [peste=6.000/de:usere]
• [setate/=<SweetSoul>
• [checking/SweetSoul]\\par

It looks like the attacker thinks that square brackets are comment notation, but the brute force tool simply sends the text as SSH username/password pairs. There also seems to be a Romanian language connection. For example, “acest este varza” according to Google means “this is cabbage” (perhaps an idiom), “contzine” means “list any”, “peste de usere” means “over the user”, “setate” means “set”. The Romanian connection also came up in the previous post where Romanian for “Handbook of Mechanical Engineering” was tried as a password.

Attentive readers will note the “\\par” in the above list perhaps indicating that the file was converted to RTF at some point. This appears indeed to be the case from the later attempt of username: “\\*\\generator”, password: “Msftedit 5.41.21.2508;}…[checking uratu]\\par”. From this we can also conclude that the attacker is using Windows WordPad.

Overall it was an interesting experiment, with some conclusions confirmed but a few surprises. However, this was only a two week experiment on a single machine, so care should be taken in drawing generalisations which assume that these results are typical.

About a moth ago I’ve presented at the Security Protocols Workshop a new idea to detect relay attacks, co-developed with Frank Stajano.

The idea relies on having a trusted box (which we call the T-Box as in the image below) between the physical interfaces of two communicating parties. The T-Box accepts 2 inputs (one from each party) and provides one output (seen by both parties). It ensures that none of the parties can determine the complete input of the other party.

Therefore by connecting 2 instances of a T-Box together (as in the case of a relay attack) the message from one end to the other (Alice and Bob in the image above) gets distorted twice as much as it would in the case of a direct connection. That’s the basic idea.

One important question is how does the T-Box operate on the inputs such that we can detect a relay attack? In the paper we describe two example implementations based on a bi-directional channel (which is used for example between a smart card and a terminal). In order to help the reader understand these examples better and determine the usefulness of our idea Mike Bond and I have created a python simulation. This simulation allows you to choose the type of T-Box implementation, a direct or relay connection, as well as other parameters including the length of the anti-relay data stream and detection threshold.

In these two implementations we have restricted ourselves to make the T-Box part of the communication channel. The advantage is that we don’t rely on any party providing the T-Box since it is created automatically by communicating over the physical channel. The disadvantage is that a more powerful attacker can sample the line at twice the speed and overcome our T-Box solution.

The relay attack can be used against many applications, including all smart card based payments. There are already several ideas, including distance bounding, for detecting relay attacks. However our idea brings a new approach to the existing methods, and we hope that in the future we can find a practical implementation of our solutions, or a good scenario to use a physical T-Box which should not be affected by a powerful attacker.

Passwords are no longer acceptable as a security mechanism. The arrogant security people ask users that passwords be memorable, unguessable, high entropy, all different and never written down. With the proliferation of the number of passwords and the ever-increasing brute-force capabilities of modern computers, passwords of adequate strength are too complicated for human memory, especially when one must remember dozens of them. The above demands cannot all be satisfied simultaneously. Users are right to be pissed off.

A number of proposals have attempted to find better alternatives for the case of web authentication, partly because the web is the foremost culprit in the proliferation of passwords and partly because its clean interfaces make technical solutions tractable.

For the poor user, however, a password is a password, and it’s still a pain in the neck regardless of where it comes from. Users aren’t fed up with web passwords but with passwords altogether. In “Pico: no more passwords, the position paper I’ll be presenting tomorrow morning at the Security Protocols Workshop, I propose a clean-slate design to get rid of passwords everywhere, not just online. A portable gadget called Pico transforms your credentials from “what you know” into “what you have”.

A few people have already provided interesting feedback on the pre-proceedings draft version of the paper. I look forward to an animated discussion of this controversial proposal tomorrow. Whenever I serve as help desk for my non-geek acquaintances and listen to what drives them crazy about computers I feel ashamed that, with passwords, we (the security people) impose on them such a contradictory and unsatisfiable set of requests. Maybe your gut reaction to Pico will be “it’ll never work”, but I believe we have a duty to come up with something more usable than passwords.

[UPDATE: the paper can also be downloaded from my own Cambridge web site, where the final version will appear in due course.]

My paper Can We Fix the Security Economics of Federated Authentication? asks how we can deal with a world in which your mobile phone contains your credit cards, your driving license and even your car key. What happens when it gets stolen or infected?

Using one service to authenticate the users of another is an old dream but a terrible tar-pit. Recently it has become a game of pass-the-parcel: your newspaper authenticates you via your social networking site, which wants you to recover lost passwords by email, while your email provider wants to use your mobile phone and your phone company depends on your email account. The certification authorities on which online trust relies are open to coercion by governments – which would like us to use ID cards but are hopeless at making systems work. No-one even wants to answer the phone to help out a customer in distress. But as we move to a world of mobile wallets, in which your phone contains your credit cards and even your driving license, we’ll need a sound foundation that’s resilient to fraud and error, and usable by everyone. Where might this foundation be? I argue that there could be a quite surprising answer.

The paper describes some work I did on sabbatical at Google and will appear next week at the Security Protocols Workshop.

During my MPhil within the Computer Lab (supervised by Markus Kuhn) I developed a card-sized device (named Smart Card Detective – in short SCD) that can monitor Chip and PIN transactions. The main goal of the SCD was to offer a trusted display for anyone using credit cards, to avoid scams such as tampered terminals which show an amount on their screen but debit the card another (see this paper by Saar Drimer and Steven Murdoch). However, the final result is a more general device, which can be used to analyse and modify any part of an EMV (protocol used by Chip and PIN cards) transaction.

Using the SCD we have successfully shown how the relay attack can be mitigated by showing the real amount on the trusted display. Even more, we have tested the No PIN vulnerability (see the paper by Murdoch et al.) with the SCD. A reportage on this has been shown on Canal+ (video now available here).

After the “Chip and PIN is broken” paper was published some contra arguments referred to the difficulty of setting up the attack. The SCD can also show that such assumptions are many times incorrect.

More details on the SCD are on my MPhil thesis available here. Also important, the software is open source and along with the hardware schematics can be found in the project’s page. The aim of this is to make the SCD a useful tool for EMV research, so that other problems can be found and fixed.

Thanks to Saar Drimer, Mike Bond, Steven Murdoch and Sergei Skorobogatov for the help in this project. Also thanks to Frank Stajano and Ross Anderson for suggestions on the project.

This is the fourth and final part in a series on password implementations at real websites, based on my paper at WEIS 2010 with Sören Preibusch.

Given the problems associated with passwords on the web outlined in the past few days, for years academics have searched for new technology to replace passwords. This thinking can at times be counter-productive, as no silver bullets have yet materialised and this has distracted attention away from fixing the most pressing problems associated with passwords. Currently, the trendiest proposed solution is to use federated identity protocols to greatly reduce the number of websites which must collect passwords (as we’ve argued would be a very positive step). Much focus has been given to OpenID, yet it is still struggling to gain widespread adoption. OpenID was deployed at less than 3% of websites we observed, with only Mixx and LiveJournal giving it much prominence.

Nevertheless, we optimistically feel that real changes will happen in the next few years, as password authentication on the web seems to be becoming increasingly unsustainable due to the increasing scale and interconnectivity of websites collecting passwords. We actually think we are already in the early stages of a password revolution, just not of the type predicted by academia.

The question of why passwords still dominate was discussed at a panel 18 months ago at Financial Crypto 2009, and already several of the major stumbling blocks to password replacement are quickly fading:

• Reluctance to deploy new technology-While few websites have replaced passwords as a primary means of authentication, there has been considerable innovation by large authentication services in the past year, mostly in backup authentication. Google has rolled out SMS-based password recovery, Microsoft is planning to do the same for one-time use at untrusted terminals, Facebook has rolled out backup authentication by identifying friends tagged in photographs, and Amazon has quietly introduced its novel Payphrases system.
• Lack of public evidence of risk-There have been a few high publicity password incidents in the last year, notably the RockYou hack and Twitter account compromises which led to forced reset of past users. These attacks are larger in scale than previous hacks but not qualitatively different (consider the now all-but-forgotten Reddit password compromise of 2007). Twitter is proving to be an interesting driver of change, however. While there have been  high-profile account takeovers of other celebrity accounts, such as Sarah Palin’s email, or Paris Hilton’s mobile phone, celebrity Twitter account hacks have become routine, happening to basketball player Ray Allen, singer Jason Mraz, astronauts with NASA, and even President Barack Obama (of course, some claims of account takeover are quite dubious). The most interesting thing about these attacks is that they increasingly seem to be financially motivated, as a high-audience compromised account is used to insert spam advertising. If account takeover becomes industrialised (along the lines of phishing and spam), it will put real pressure on changes to web authentication.
• No organisation with sufficient leverage to move the market-With the meteoric rise of Facebook, this assumption is increasingly dated. Facebook is increasingly gaining the power to impose its preferred identity solution, Facebook Connect, with some large identity providers getting on board.

This leaves one major problem in place, competing stakeholders in the market, to which our study provides valuable data. Why do small websites, particularly news websites, still collect their own passwords instead of relying on a federated identity provider? There seems to be little security reason for doing so, as reliance on email providers for backup authentication is already nearly universal. Excluding webmail providers themselves, over 97% of websites we studied used external email addresses as login identifiers and performed password reset using email as a trusted channel.

Considering newspaper websites as the prime example of password deployments with questionable utility, we found that they were significantly more likely than other websites to collect marketing data at the time of password collection. They also nearly universally, with only a single exception, insist on verifying the validity of a user’s email address as a pre-requisite for an account (this was equally true at sites which also require CAPTCHAs, so we don’t think false account prevention is the primary reason). At this point, users may be trained to accept the idea of inputting personal data along with a password. A password input field may serve as a ceremonial cue that entering personal data into a form is safe (verifying this with behavioural experiments is an important future research question).

Thus, collecting passwords provides cover for collecting personal data, which provides tangible benefits to a site operator. The costs of password collection, however, aren’t primarily borne by the server. As discussed Wednesday, the costs of insecurity are largely borne by higher-security websites. The usability costs, in contrast are borne by users themselves. Users have a limited capability to remember and manage multiple passwords securely, which deteriorates as each additional website which requests a password, but each of those websites gains unilaterally from collection. Thus, over-collection can be viewed as a tragedy of the commons, as each website races to capitalise on users’ limited resources. As a result, 85% of the 500 most popular websites collect passwords, and the average web user has over 25 password-protected accounts.

OpenID may be being held back by market forces. For users, removing the requirement of registering personal information with the site is an advantage, but for many websites, removing the ability to collect personal data eliminates a major justification for deploying passwords at all. This may explain why we saw over twice as many websites in our study which accepted Facebook Connect, a proprietary protocol which is very similar to OpenID. In addition to being simpler for users to understand, Facebook Connect allows relying websites to send data to and from the user’s Facebook profile, more than replacing the previous motivation for collecting passwords. Tapping into Facebook’s unique arsenal of user data has proved irresistible even to large websites like Amazon.

Of all places on the web though, I think the American gossip website TMZ may be the clearest picture of the future. Authentication on the site is seamless, just as OpenID advocates hope it can be. Objects on the site can be dragged and dropped into one of several external identity providers (Facebook, Google, Twitter, and Yahoo), each with heavy customisation and use of OAuth to share data, but no option to use other OpenID providers. Whether OpenID is used or similar protocols, the future may be dominated by a small oligopoly of identity providers with large amounts of personal data, in contrast to the design goals of OpenID. The economics make it inevitable, however, that most sites don’t really care about who you are, but what they can gain from who you are.

Sören Preibusch and I have finalised our in-depth report on password practices in the wild, The password thicket: technical and market failures in human authentication on the web, presented in Boston last month for WEIS 2010. The motivation for our report was a lack of technical research into real password deployments. Passwords have been studied as an authentication mechanism quite intensively for the last 30 years, but we believe ours was the first large study into how Internet sites actually implement them. We studied 150 sites, including the most visited overall sites plus a random sample of mid-level sites. We signed up for free accounts with each site, and using a mixture of scripting and patience, captured all visible aspects of password deployment, from enrolment and login to reset and attacks.

Our data (which is now publicly available) gives us an interesting picture into the current state of password deployment. Because the dataset is huge and the paper is quite lengthy, we’ll be discussing our findings and their implications from a series of different perspectives. Today, we’ll focus on the preventable mistakes. In academic literature, it’s assumed that passwords will be encrypted during transmission, hashed before storage, and attempts to guess usernames or passwords will be throttled. None of these is widely true in practice.

Storing passwords without hashing (and salting) is the first cardinal sin of password implementation; late last year it enabled the theft of 32 M RockYou users’ passwords. Yet the practice remains common: at least 29% of the sites we studied sent cleartext passwords via email, meaning that they must be stored without proper hashing (they may be stored encrypted, but this makes little difference-if the server gets rooted, the encryption keys are likely to be available).  It’s impossible to know how the rest store passwords without auditing their databases, but another telling indicator is that at least 40% of sites imposed a short maximum password length or restrictions on special characters. These are both warning signs that passwords are being stored directly in a database table without hashing, which would reduce arbitrarily long input passwords to a constant storage size. Only 1 site of 150 successfully hashed passwords in the browser using JavaScript, giving us confidence that passwords aren’t stored in a recoverable format (two other sites tried but botched the details).

Using TLS to authenticate the server and encrypt passwords in transmission is another essential aspect of password security. 41% of sites failed to use any TLS, though what’s more surprising is the number that implemented TLS inconsistently. A full password implementation will contain up to four separate forms for password entry: enrolment, login, update, and password reset. Of sites implementing TLS, 28% forgot to protect at least one of these forms (usually the update form or the enrolment form), including some big names like Facebook, MySpace, Twitter, and WordPress. Not implementing any TLS may be a (questionable) security trade-off, but implementing it inconsistently is certainly an oversight. Only 39% of sites had a complete, working TLS deployment.

For authentication by an online server, two other important security practices are preventing user probing attacks and password guessing attacks. User probing allows attackers to build up a large list  of users registered with the site. Attackers can use such a list in a trawling attack, where a few popular passwords are guessed for a large number of accounts in the hope of compromising a small number of them. 19% of sites give an “email address not registered” error message upon log-in with a bogus email address, making user probing trivial. A much more common mistake, however, made by 80% of sites, is to give an “email address not registered” error message when requesting a password reset email for a bogus email address. This is an easy detail to overlook, but it means it is trivial to collect a large membership list at most password-collecting sites.

The risk of a trawling attack, however, is predicated on the assumption that repeated password guessing for an individual account will be limited. In fact, the large majority of sites (84%) did not appear to rate-limit password guessing at all, allowing our automated script to guess 100 incorrect passwords at the rate of one per second before successfully logging in to a test account. There is some research suggesting against low guessing cutoffs, but we saw no cutoff values greater than 20, and it seems safe to assume very few of the sites which didn’t block us after 100 attempts would do so at a later point.

In the security research community, we generally assume web-based password authentication includes basic security measures: encryption during transmission, hashing for storage, and prevention of brute force attacks or user probing. There are still many ways that passwords can fail, notably key-loggers and phishing, for which we have no consensus solution. Yet implementing the basics is surprisingly rare-just 3 of  150 studied sites managed to do so successfully (Google, Microsoft Live, and ShopBop). The sites we studied were, for the most part, professionally-done sites representing multi-million dollar businesses.

In a few cases, sites may be making a defensible security/usability tradeoff. Amazon, for example, didn’t block our brute force attempts, but there’s ample reason to believe they detect account takeover by other means. On the whole though, the level of security implemented is dramatically lower than security researchers might expect. There’s an interesting parallel here. At first the insecurity of passwords was blamed on users not behaving the way security engineers wanted them to: choosing weak passwords, forgetting them, writing them down, sharing them, and typing them in to the wrong domains. It’s now generally accepted that we should design password security around users, and that users may even be wise to ignore security advice.

Web developers are people too, as was recently argued at SOUPS, and they don’t behave the way security engineers would like either when it comes to passwords. Right or wrong, we should update our thinking to take this into account.

We have a new paper on the strategic vulnerability created by the plan to replace Britain’s 47 million meters with smart meters that can be turned off remotely. The energy companies are demanding this facility so that customers who don’t pay their bills can be switched to prepayment tariffs without the hassle of getting court orders against them. If the Government buys this argument – and I’m not convinced it should – then the off switch had better be closely guarded. You don’t want the nation’s enemies to be able to turn off the lights remotely, and eliminating that risk could just conceivably be a little bit more complicated than you might at first think. (This paper follows on from our earlier paper On the security economics of electricity metering at WEIS 2010.)

Steven Murdoch, Saar Drimer, Mike Bond and I have just won the IEEE Security and Privacy Symposium’s Best Practical Paper award for our paper Chip and PIN is Broken. This was an unexpected pleasure, given the very strong competition this year (especially from this paper). We won this award once before, in 2008, for a paper on a similar topic.

Update (2010-05-28): The photo now includes the full team (original version)

The aptly-named Journal of Craptology (est. 1998) has just published a special Valentine Day issue. It contains a silly piece on Romantic Cryptography that we originally discussed in 1999 in our Friday meetings.