Today, Jonathan Anderson, Ben Laurie, Kris Kennaway, and I presented Capsicum: practical capabilities for UNIX at the 19th USENIX Security Symposium in Washington, DC; the slides can be found on the Capsicum web site. We argue that capability design principles fill a gap left by discretionary access control (DAC) and mandatory access control (MAC) in operating systems when supporting security-critical and security-aware applications.

Capsicum responds to the trend of application compartmentalisation (sometimes called privilege separation) by providing strong and well-defined isolation primitives, and by facilitating rights delegation driven by the application (and eventually, user). These facilities prove invaluable, not just for traditional security-critical programs such as tcpdump and OpenSSH, but also complex security-aware applications that map distributed security policies into local primitives, such as Google’s Chromium web browser, which implement the same-origin policy when sandboxing JavaScript execution.

Capsicum extends POSIX with a new capability mode for processes, and capability file descriptor type, as well as supporting primitives such as process descriptors. Capability mode denies access to global operating system namespaces, such as the file system and IPC namespaces: only delegated rights (typically via file descriptors or more refined capabilities) are available to sandboxes. We prototyped Capsicum on FreeBSD 9.x, and have extended a variety of applications, including Google’s Chromium web browser, to use Capsicum for sandboxing. Our paper discusses design trade-offs, both in Capsicum and in applications, as well as a performance analysis. Capsicum is available under a BSD license.

Capsicum is collaborative research between the University of Cambridge and Google, and has been sponsored by Google, and will be a foundation for future work on application security, sandboxing, and security usability at Cambridge and Google. Capsicum has also been backported to FreeBSD 8.x, and Heradon Douglas at Google has an in-progress port to Linux.

We’re also pleased to report the Capsicum paper won Best Student Paper award at the conference!

This is the fourth and final part in a series on password implementations at real websites, based on my paper at WEIS 2010 with Sören Preibusch.

Given the problems associated with passwords on the web outlined in the past few days, for years academics have searched for new technology to replace passwords. This thinking can at times be counter-productive, as no silver bullets have yet materialised and this has distracted attention away from fixing the most pressing problems associated with passwords. Currently, the trendiest proposed solution is to use federated identity protocols to greatly reduce the number of websites which must collect passwords (as we’ve argued would be a very positive step). Much focus has been given to OpenID, yet it is still struggling to gain widespread adoption. OpenID was deployed at less than 3% of websites we observed, with only Mixx and LiveJournal giving it much prominence.

Nevertheless, we optimistically feel that real changes will happen in the next few years, as password authentication on the web seems to be becoming increasingly unsustainable due to the increasing scale and interconnectivity of websites collecting passwords. We actually think we are already in the early stages of a password revolution, just not of the type predicted by academia.

The question of why passwords still dominate was discussed at a panel 18 months ago at Financial Crypto 2009, and already several of the major stumbling blocks to password replacement are quickly fading:

• Reluctance to deploy new technology-While few websites have replaced passwords as a primary means of authentication, there has been considerable innovation by large authentication services in the past year, mostly in backup authentication. Google has rolled out SMS-based password recovery, Microsoft is planning to do the same for one-time use at untrusted terminals, Facebook has rolled out backup authentication by identifying friends tagged in photographs, and Amazon has quietly introduced its novel Payphrases system.
• Lack of public evidence of risk-There have been a few high publicity password incidents in the last year, notably the RockYou hack and Twitter account compromises which led to forced reset of past users. These attacks are larger in scale than previous hacks but not qualitatively different (consider the now all-but-forgotten Reddit password compromise of 2007). Twitter is proving to be an interesting driver of change, however. While there have been  high-profile account takeovers of other celebrity accounts, such as Sarah Palin’s email, or Paris Hilton’s mobile phone, celebrity Twitter account hacks have become routine, happening to basketball player Ray Allen, singer Jason Mraz, astronauts with NASA, and even President Barack Obama (of course, some claims of account takeover are quite dubious). The most interesting thing about these attacks is that they increasingly seem to be financially motivated, as a high-audience compromised account is used to insert spam advertising. If account takeover becomes industrialised (along the lines of phishing and spam), it will put real pressure on changes to web authentication.
• No organisation with sufficient leverage to move the market-With the meteoric rise of Facebook, this assumption is increasingly dated. Facebook is increasingly gaining the power to impose its preferred identity solution, Facebook Connect, with some large identity providers getting on board.

This leaves one major problem in place, competing stakeholders in the market, to which our study provides valuable data. Why do small websites, particularly news websites, still collect their own passwords instead of relying on a federated identity provider? There seems to be little security reason for doing so, as reliance on email providers for backup authentication is already nearly universal. Excluding webmail providers themselves, over 97% of websites we studied used external email addresses as login identifiers and performed password reset using email as a trusted channel.

Considering newspaper websites as the prime example of password deployments with questionable utility, we found that they were significantly more likely than other websites to collect marketing data at the time of password collection. They also nearly universally, with only a single exception, insist on verifying the validity of a user’s email address as a pre-requisite for an account (this was equally true at sites which also require CAPTCHAs, so we don’t think false account prevention is the primary reason). At this point, users may be trained to accept the idea of inputting personal data along with a password. A password input field may serve as a ceremonial cue that entering personal data into a form is safe (verifying this with behavioural experiments is an important future research question).

Thus, collecting passwords provides cover for collecting personal data, which provides tangible benefits to a site operator. The costs of password collection, however, aren’t primarily borne by the server. As discussed Wednesday, the costs of insecurity are largely borne by higher-security websites. The usability costs, in contrast are borne by users themselves. Users have a limited capability to remember and manage multiple passwords securely, which deteriorates as each additional website which requests a password, but each of those websites gains unilaterally from collection. Thus, over-collection can be viewed as a tragedy of the commons, as each website races to capitalise on users’ limited resources. As a result, 85% of the 500 most popular websites collect passwords, and the average web user has over 25 password-protected accounts.

OpenID may be being held back by market forces. For users, removing the requirement of registering personal information with the site is an advantage, but for many websites, removing the ability to collect personal data eliminates a major justification for deploying passwords at all. This may explain why we saw over twice as many websites in our study which accepted Facebook Connect, a proprietary protocol which is very similar to OpenID. In addition to being simpler for users to understand, Facebook Connect allows relying websites to send data to and from the user’s Facebook profile, more than replacing the previous motivation for collecting passwords. Tapping into Facebook’s unique arsenal of user data has proved irresistible even to large websites like Amazon.

Of all places on the web though, I think the American gossip website TMZ may be the clearest picture of the future. Authentication on the site is seamless, just as OpenID advocates hope it can be. Objects on the site can be dragged and dropped into one of several external identity providers (Facebook, Google, Twitter, and Yahoo), each with heavy customisation and use of OAuth to share data, but no option to use other OpenID providers. Whether OpenID is used or similar protocols, the future may be dominated by a small oligopoly of identity providers with large amounts of personal data, in contrast to the design goals of OpenID. The economics make it inevitable, however, that most sites don’t really care about who you are, but what they can gain from who you are.

This is the third part in a series on password implementations at real websites, based on my paper at WEIS 2010 with Joseph Bonneau.

In our analysis of 150 password deployments online, we observed a surprising diversity of implementation choices. Whilst sites can be ranked by the overall security of their password scheme, there is a vast middle group in which sites make seemingly incongruous security decisions. We also found almost no evidence of commonality in implementations. Examining the details of Web forms (variable names, etc.) and the format of automated emails, we found little evidence that sites are re-using a common code base. This lack of consistency in technical choices suggests that standards and guidelines could improve security.

Numerous RFCs concern themselves with one-time passwords and other relatively sophisticated authentication protocols. Yet, traditional password-based authentication remains the most prevalent authentication protocol on the Internet, as the International Telecommunication Union–itself a United Nations specialized agency to standardise telecommunications on a worldwide basis–observes in their ITU-T Recommendation X.1151, “Guideline on secure password-based, authentication protocol with key exchange.” Client PKI has not seen wide-spread adoption and tokens or smart-cards are prohibitively cost-inefficient or inconvenient for most websites. While passwords have many shortcomings, it is essential deploy them as carefully and securely as possible. Formal standards and guidelines of best practices are essential to help developers.

Where Web standards come from

Traditionally, standards for secure IT systems have been developed by national standardisation agencies, such as DIN, and federal IT agencies, such as the BSI or NIST whose recommendations and guidelines first applied to government IT and whose scope was then extended to commercial installations. With the harmonisation of standardisation efforts, national standards then became European Norms (EN) or international standards (of the ISO). In some cases, technical specifications made it into codified law.

Second, new cross-national standardisation organisations have emerged, specifically targeted at the Internet and the Web. Most relevant are the Internet Engineering Task Force (IETF), with its committee the Internet Architecture Board (IAB), and the World Wide Web Consortium (W3C). Both are characterised by a fairly quick standards track and an impetus of “making things work.” More importantly, their standards are universally accessible for free. Other Internet standards bodies, such as the Internet Assigned Numbers Authority (IANA) would not include passwords on the Web into their scope of work.

Third, there are industry consortia such as Organization for the Advancement of Structured Information Standards (OASIS) and professional associations such as the Institute of Electrical and Electronics Engineers (IEEE) that shape Internet developments on an applications and a technical level, but their work does not include password advice.

Fourth, commercial entities that certify the security of websites and issue seals for a fee have (presumably) developed their own checklists. However, these lists are not publicly available and their quality cannot be the scrutinised. All the same, blatant security breaches at certified websites have cast doubt on the quality of these certification procedures (e.g. TÜV SÜD s@fer-shopping and Libri.de case, ControlScan), as has academic research into their effectiveness. Moreover, it is unclear if an evaluation checklist can also serve as a guideline during development.

Finally, large software vendors, providers of software development environments, website toolkits and frameworks can effectively shape the technical landscape on the Web. Defaults in such solutions are the most powerful way to establish a de-facto standard and potentially exercise positive paternalism.

Standards on password technologies

Government standards

Most government-track standards focus on systems security, but the recommendations on password security they contain could be analogously applied  to the Web. In the United States, NIST published FIPS 112 on “Password Usage” in 1985, which gave very specific advice on password implementations. The standard includes encrypting all passwords during transmission and storage, forcing users to change passwords every year, and forcing users to pick a new password after forgetting the previous one. The guidelines also motivate a password length of 5 to 8 characters as a compromise between memorability, security, and ease of typing, and supporting several potential character sets.

More recently, NIST published SP 800-63 “Electronic Authentication Guideline” in 2008. This document contains fewer specifics about password authentication, instead defining four assurance levels for remote authentication, for which only the lowest two levels can be based solely on passwords. It provides general requirements for password security, such as encryption during transmission and storage, but is not intended for website developers and provides no guidance on the interaction of password authentication with Web protocols. The standard also avoids recommending any specific password requirements, but does mandate upper limits on password guessability and provides a detailed algorithm for approximating the required entropy with length and character classes.

Amongst international standards, ISO/IEC 27001:2005 mandates that a secure system requires strong passwords, but provides no technical details; buyers of the accompanying implementation guide ISO/IEC 27002:2005 get some additional details on “access control,” including password considerations. The German BSI (Federal Office for Information Security) published technical standards for satisfying ISO 27001 which do contain specific recommendations. The IT-Grundschutz (IT Baseline Protection) Catalogues are organised by modules and provide details on threats and security safeguards; they can be downloaded for free. Now in its eleventh edition, it has been translated into several languages, and has grown to more than 4000 pages. On the subject of passwords, the compromise between security and usability is acknowledged again. Alphanumeric passwords should be made up of at least eight characters, and numbers-only passwords should have a minimum length of six. Further, error messages should be unspecific with no indication whether username and/or password were wrong and account locking should happen silently.

None of these standards deals with password recovery, assuming that an administrative help desk and a secure channel are available. Also they abstract away many implementation issues on the Web, such as the choice between HTTP authentication and form-based submission of login credentials.

HTTP and HTML standards

In HTTP authentication, the server detects that the requested resource requires authentication and sends an HTTP 401 status code along with a “WWW-Authenticate” header. According to RFC 2617, the browser then displays a (modal) dialog that prompts for username and password before any content is displayed. In the “Basic” variant, the credentials are submitted unencrypted; in “Digest” mode, introduced with HTTP/1.1 to “avoid the most serious flaws of Basic authentication”, salted MD5 hashes with incremental nonces can be used. Regardless of security considerations, HTTP authentication suffers from usability drawbacks and the inability to style the password prompt. As a result, we did not see a single instance of HTTP authentication in our survey.

In form-based authentication, the implementation is site-specific. The HTML (XHTML, WML, …) page includes a form with input fields for username and password. The data the user enters is submitted just as any other HTTP request, either as part of the request (GET) or in the body (POST). Consequently, transmission is unprotected by default unless the connection is encrypted via TLS (HTTPS) or hashing is done with a client-side script (we observed three websites doing so). The relevant standards are the HTML specifications that define the markup for Web forms and input fields; these elements were already part of HTML 2.0 in 1995. They came with “security considerations,” noting that the “widely deployed methods for submitting forms requests — HTTP and SMTP — provide little assurance of confidentiality. Information providers who request sensitive information via forms — especially by way of the `PASSWORD’ type input field — should be aware and make their users aware of the lack of confidentiality.” This advice was dropped in HTML 3.2, where it was instead said to obfuscate user input “using a character like * to hide the text from prying eyes when entering passwords” (suggesting a shift in the threat model). Incidentally, the example provided in the specification included a password field of “size=12″ with no “maxlength” restriction. HTML 4.0 saw the re-introduction of a note that a plaintext password “affords only light security protection” as it is “transmitted to the server in clear text”. Similar but even more explicit caveats are also included in XForms. Security notes have disappeared again from HTML5, with the exception of a vague and potentially misleading hint that a “user agent should obscure the value so that people other than the user cannot see it.”

Developers are thus unable to glean concrete help from the HTML specifications on how to implement a good password prompt. Given the strong effort to revamp forms in HTML5, it seems a lost opportunity to add better security functionality to the password input field through client-side scripting, as it was done for many other of the input fields. As early as 1999, a W3C Note on User Agent Authentication Forms proposed “new HTML capability to aid in the development of authenticated Web user interfaces”, that came down to combining the security advantages of HTTP Digest authentication with the superior user experience of Web forms (the interplay between HTTP and form-based authentication has been revisited recently from a usability perspective, resulting in a rather obscure cookie-based scheme).

IETF standards

Meanwhile, the IETF published requests for comments on stronger authentication protocols and has spent more time on password replacements than actual passwords – there are many RFCs on Internet Key Exchange (RFCs 2409, 3526, 3664, 3706, …, 5903) but not a single one on  “how to do traditional passwords.” Anecdotally, the most relevant RFC on password technology is RFC 972 (from 1986). It describes a “Password Generator Protocol” that is a service to generate and suggest to the user six strong but memorable 8-character passwords from which the user may then choose one.

More recently, the prevalence of plaintext passwords on the Web has been acknowledged. The Survey of Authentication Mechanisms by the Internet Architecture Board, currently in its 7th draft, provides a valuable critical review of authentication technologies, including (plain) passwords and their alternatives. The section on passwords lists numerous problems, with countermeasures for each, such as dictionary attacks and phishing. Resistance against brute force attacks by guessing common passwords is also discussed with countermeasure alternatives such as incorporating a limited try capability with lock-down or an increasing delay for each additional guess. In a sense, this collection of problems and countermeasures it can be read as best-practices, but requires an attentive reader to transform suggested alternatives into real resolutions. Also, this review still fails to recognise that password reset and recovery have important implications for overall system security.

Following the “log-in metaphor,” authentication on the Web is often expected to last for a session rather than for a single request. This calls for preservation and replay of credentials. When using HTTP authentication, the user agent can include authentication information in the request headers. Otherwise, cookies are the universal technique for replaying client identification, as our survey has confirmed. However RFC 2964 states that “it is generally inappropriate to use the HTTP State Management protocol as an authentication mechanism,” since cookies are transmitted in an unencrypted manner. RFC 2965 states that “While it is common practice to use them this way, cookies are not designed or intended to be used to hold authentication information, such as account names and passwords. Unless such cookies are exchanged over an encrypted path, the account information they contain is highly vulnerable to perusal and theft.” This advice suggests a troubling divide between IETF standards and near-universal practice on the Web.

Industry libraries and practical guidelines

Good drop-in solutions exist for content management systems like Drupal and Web frameworks like Django, Ruby on Rails, or ASP.NET which cover most elements of the password life cycle. The Apache project has authentication modules, but only for the uncommon HTTP authentication mechanism and not form-based authentication. Because passwords interface with many areas of a website, it remains debated whether re-using an existing password module reduces development effort for a highly customised site. Certainly, many of the professional websites we observed in our survey made mistakes not found in the modules listed above (though there are certainly dubious password modules as well), indicating that large websites prefer developing their own password solution in contrast to other elements like TLS authentication.

For developers re-inventing the wheel, formal guidelines are harder to find, being largely spread across blogs from industry, open-source projects, and Web security experts. There does not appear to be a canonical set of guidelines online which emerges highly in search engine results. Academic research on traditional passwords has mostly focused on user studies; a literature review can be found in our paper. In search of scientific results, internal validity competes with external validity: the better controlled a laboratory scenario, the less similar it becomes to real-world deployments. Consequently, conclusions from academic studies often cannot be translated easily into actual code. Although very concrete dos and dont’s of password deployments sometimes arise in academia, the main body of knowledge in academic papers is generally not accessible to developers as it resides outside their normal fora (and may be behind pay walls).

Conclusions

The value of password standards lies in them being read by developers for help in deploying better password schemes on their websites. Whilst good standards appeal as is, they aren’t l’art pour l’art. Government standards for system security tend to ignore new Web-specific pitfalls and do not cover the entire lifecycle of passwords, especially reset and recovery with no availability of inherently secure channels. Advice in existing standards may also be too general to be turned into implementations easily. Standards bodies dedicated to the Web have not embraced the pragmatic questions of password security, devoting more effort to password replacements.

Password implementation is a very common task implemented by thousands of websites, but standards are doing little to improve the process. Until good standards become available, their role is partially fulfilled by industry de-facto standards or community-driven guidelines which remain scattered on the Web amidst much noise.

This is the second part in a series on password implementations at real websites, based on my paper at WEIS 2010 with Sören Preibusch.

As we discussed yesterday, dubious practices abound within real sites’ password implementations. Password insecurity isn’t only due to random implementation mistakes, though. When we scored sites’ passwords implementations on a 10-point aggregate scale it became clear that a wide spectrum of implementation quality exists. Many web authentication giants (Amazon, eBay, Facebook, Google, LiveJournal, Microsoft, MySpace, Yahoo!) scored near the top, joined by a few unlikely standouts (IKEA, CNBC). At the opposite end were a slew of lesser-known merchants and news websites. Exploring the factors which lead to better security confirms the basic tenets of security economics: sites with more at stake tend to do better. However, doing better isn’t enough. Given users’ well-documented tendency to re-use passwords, the varying levels of security may represent a serious market failure which is undermining the security of password-based authentication.

For websites, popularity (measured by Alexa traffic data), maturity (measured by years of existence), and technical skill and expenditure (measured by low average server latency) all positively correlate with better security. The simplest explanation is that global sites can afford more (and more skilled) developers to work out the details of their password implementation than local players like the The Nashville Scene can. Controlling for this influence though, we still see sharp differences between market segments. We studied three major use-cases for passwords online: content websites like newspapers, which use them to customise content for frequent readers and speed up commenting (sometimes making them mandatory), e-commerce websites which use passwords to protect shipping and payment data and allow for order-tracking, and “identity” websites like email and social networks which use passwords to protect persistent online identities. Below is a plot of password scores showing both traffic rank and market category for each site:

Many significant trends emerged between categories. E-commerce websites are significantly more likely to deploy TLS and notify users of changes to their account. Identity websites are significantly more likely to prevent brute-forcing of users and passwords, to block the use of dictionary words as passwords, and to require CAPTCHAs to make account changes. These differences probably relate to the core implementation skills required to succeed in the two segments: social networks and webmail providers have to block automated interaction to prevent spam and false account creation while e-commerce sites have to use encryption to protect payment details. The sites we studied which store payment card details (not only e-commerce sites, but many social networking and email sites) we see significant improvement across the board. Such sites bear a direct risk of fraud if passwords are compromised, so this isn’t surprising.News websites, unfortunately, fare worse than average in every measurable aspect of security, and indeed represented most of the worst practices we found. Interestingly, news sites offering premium content behind pay-walls didn’t fare significantly better despite having a revenue stream from limiting access. They even managed to take some very simple measures less often, like opting-out of the Bugmenot database to prevent users sharing subscriptions.

We only studied websites offering free accounts, so we missed a few important segments. Falk et al. studied bank websites specifically in 2008, and found problems at a rate comparable to the e-commerce websites we studied. Flôrencio and Herley recently compared the password policies at government and university websites to commercial ones and found that commercial websites impose less strict password strength requirements because usability is more critical to their business. This was confirmed by our data-we found no significant variation in password strength requirements between the segments we studied (all commercial).

It’s not surprising that sites with different security motivations perform differently. Password security is not an isolated phenomenon though and bad practices at one site can affect unrelated websites because users reuse passwords. It’s long been said in the security community that password re-use is dangerous because it enables attackers to compromise an account at a low-security site and gain access to a higher-security one. This is increasingly true as most websites today use email addresses as identifiers (87% in our study), meaning an email/password combination can unlock many online accounts. The RockYou hacker indicated that 10% of the email/password combinations registered at RockYou were also PayPal accounts (external compromise isn’t the only threat here: a malicious insider could certainly try to profit from a database at sites like this). This very attack scenario played out earlier this year at Twitter, which forced many users to change their accounts after a torrent website’s database of email/password pairs was hacked and used to take over Twitter accounts en masse.

News websites may have very little to lose through poor password security, but they can undermine the efforts made by other sites. This is a classic negative externality; with few corrective mechanisms the market is failing as news websites are doing a shoddy job with password security. Removing these externalities is hard; it requires putting a real price tag on hosting an insecure password deployment. RockYou may have endured a public shaming on tech blogs, but its gaming business has continued along fine. One approach would be strict punitive laws for websites if user passwords are compromised due to negligence, modeled on data breach notification laws. This is problematic, as aside from public, large-scale attacks, the public may not know when passwords have been compromised (tracer accounts might be useful here). A second approach would be to raise the price of collecting passwords at all, for example by requiring sites to submit to outside auditing or to send users written annual reports about their password security. It would be a successful outcome if fewer sites chose to register passwords in the first place-currently many sites are doing so which are unable to do so securely. In the meantime, technical aids like PwdHash to generate site-specific passwords automatically, or Bugmenot to re-use credentials at very low-security sites may be the best way to alleviate the problem.

Sören Preibusch and I have finalised our in-depth report on password practices in the wild, The password thicket: technical and market failures in human authentication on the web, presented in Boston last month for WEIS 2010. The motivation for our report was a lack of technical research into real password deployments. Passwords have been studied as an authentication mechanism quite intensively for the last 30 years, but we believe ours was the first large study into how Internet sites actually implement them. We studied 150 sites, including the most visited overall sites plus a random sample of mid-level sites. We signed up for free accounts with each site, and using a mixture of scripting and patience, captured all visible aspects of password deployment, from enrolment and login to reset and attacks.

Our data (which is now publicly available) gives us an interesting picture into the current state of password deployment. Because the dataset is huge and the paper is quite lengthy, we’ll be discussing our findings and their implications from a series of different perspectives. Today, we’ll focus on the preventable mistakes. In academic literature, it’s assumed that passwords will be encrypted during transmission, hashed before storage, and attempts to guess usernames or passwords will be throttled. None of these is widely true in practice.

Storing passwords without hashing (and salting) is the first cardinal sin of password implementation; late last year it enabled the theft of 32 M RockYou users’ passwords. Yet the practice remains common: at least 29% of the sites we studied sent cleartext passwords via email, meaning that they must be stored without proper hashing (they may be stored encrypted, but this makes little difference-if the server gets rooted, the encryption keys are likely to be available).  It’s impossible to know how the rest store passwords without auditing their databases, but another telling indicator is that at least 40% of sites imposed a short maximum password length or restrictions on special characters. These are both warning signs that passwords are being stored directly in a database table without hashing, which would reduce arbitrarily long input passwords to a constant storage size. Only 1 site of 150 successfully hashed passwords in the browser using JavaScript, giving us confidence that passwords aren’t stored in a recoverable format (two other sites tried but botched the details).

Using TLS to authenticate the server and encrypt passwords in transmission is another essential aspect of password security. 41% of sites failed to use any TLS, though what’s more surprising is the number that implemented TLS inconsistently. A full password implementation will contain up to four separate forms for password entry: enrolment, login, update, and password reset. Of sites implementing TLS, 28% forgot to protect at least one of these forms (usually the update form or the enrolment form), including some big names like Facebook, MySpace, Twitter, and WordPress. Not implementing any TLS may be a (questionable) security trade-off, but implementing it inconsistently is certainly an oversight. Only 39% of sites had a complete, working TLS deployment.

For authentication by an online server, two other important security practices are preventing user probing attacks and password guessing attacks. User probing allows attackers to build up a large list  of users registered with the site. Attackers can use such a list in a trawling attack, where a few popular passwords are guessed for a large number of accounts in the hope of compromising a small number of them. 19% of sites give an “email address not registered” error message upon log-in with a bogus email address, making user probing trivial. A much more common mistake, however, made by 80% of sites, is to give an “email address not registered” error message when requesting a password reset email for a bogus email address. This is an easy detail to overlook, but it means it is trivial to collect a large membership list at most password-collecting sites.

The risk of a trawling attack, however, is predicated on the assumption that repeated password guessing for an individual account will be limited. In fact, the large majority of sites (84%) did not appear to rate-limit password guessing at all, allowing our automated script to guess 100 incorrect passwords at the rate of one per second before successfully logging in to a test account. There is some research suggesting against low guessing cutoffs, but we saw no cutoff values greater than 20, and it seems safe to assume very few of the sites which didn’t block us after 100 attempts would do so at a later point.

In the security research community, we generally assume web-based password authentication includes basic security measures: encryption during transmission, hashing for storage, and prevention of brute force attacks or user probing. There are still many ways that passwords can fail, notably key-loggers and phishing, for which we have no consensus solution. Yet implementing the basics is surprisingly rare-just 3 of  150 studied sites managed to do so successfully (Google, Microsoft Live, and ShopBop). The sites we studied were, for the most part, professionally-done sites representing multi-million dollar businesses.

In a few cases, sites may be making a defensible security/usability tradeoff. Amazon, for example, didn’t block our brute force attempts, but there’s ample reason to believe they detect account takeover by other means. On the whole though, the level of security implemented is dramatically lower than security researchers might expect. There’s an interesting parallel here. At first the insecurity of passwords was blamed on users not behaving the way security engineers wanted them to: choosing weak passwords, forgetting them, writing them down, sharing them, and typing them in to the wrong domains. It’s now generally accepted that we should design password security around users, and that users may even be wise to ignore security advice.

Web developers are people too, as was recently argued at SOUPS, and they don’t behave the way security engineers would like either when it comes to passwords. Right or wrong, we should update our thinking to take this into account.

We have a new paper on the strategic vulnerability created by the plan to replace Britain’s 47 million meters with smart meters that can be turned off remotely. The energy companies are demanding this facility so that customers who don’t pay their bills can be switched to prepayment tariffs without the hassle of getting court orders against them. If the Government buys this argument – and I’m not convinced it should – then the off switch had better be closely guarded. You don’t want the nation’s enemies to be able to turn off the lights remotely, and eliminating that risk could just conceivably be a little bit more complicated than you might at first think. (This paper follows on from our earlier paper On the security economics of electricity metering at WEIS 2010.)

I’m at SHB 2010, which brings security engineers together with psychologists, behavioral economists and others interested in deception, fraud, fear, risk perception and how we make security systems more usable.

Here is the agenda. I will be liveblogging the event in comments below this post. Here are the liveblogs for SHB 2009 and SHB 2008.

Today sees the publication of a report by Professor Trisha Greenhalgh into the Summary Care Record (SCR). There is a summary of the report in the BMJ, which also has two discussion pieces: one by Sir Mark Walport of the Wellcome Trust arguing that the future of medical records is digital, and one by me which agrees but argues that as the SCR is unsafe and unlawful, it should be abandoned.

Two weeks ago I reported here how the coalition government planned to retain the SCR, despite pre-election promises from both its constituent parties to do away with it. These promises followed our Database State report last year which demonstrated that many of the central systems built by the previous government contravened human-rights law. The government’s U-turn provoked considerable anger among doctors. NGOs and backbench MPs, prompting health minister Simon Burns to promise a review.

Professor Greenhalgh’s review, which was in fact completed before the election, finds that the SCR fails to do what it was supposed to. It isn’t used much; it doesn’t fit in with how doctors and nurses actually work; it doesn’t make consultations shorter but longer; and the project was extremely badly managed. In fact, her report should be read by all serious students of software engineering; like the London Ambulance Service report almost twenty years ago, this document sets out in great detail what not to do.

For now, there is some press coverage in the Telegraph, the Mail, E-health Insider and Computerworld UK.

Here is a liveblog of WEIS which is being held today and tomorrow at Harvard. It has 125 attendees: 59% academic, 15% govt/NGO, and 26% industry; the split of backgrounds of 47% CS, 35% econ/management and 18% policy/law. The paper acceptance rate was 24/72: 10 empirical papers, 8 theory and 6 on policy.

The workshop kicked off with a keynote talk from Tracey Vispoli of Chubb Insurance. In early 2000s, insurance industry thought cyber would be big. It isn’t yet, but it is starting to grow rapidly. There is still little actuarial data. But the tndustry can shape behaviour by being in the gap between risk aversion and risk tolerance. Its technical standards can make a difference (as with buildings, highways, …). So far a big factor is the insurance response to notification requirements: notification costs of $50-60 per compromised record mean that a 47m compromise like TJX is a loss you want to insure! So she expects healthy supply and demand model for cyberinsurance in coming years. This will help to shape standards, best practices and culture.

Questions: are there enough data to model? So far no company has enough; ideally we should bring data together from industry to one central shared point. Government has a role as with highways. Standards? Client prequalification is currently a fast-moving target. Insurers’ competitive advantage is understanding the intersection between standards and pricing. Reinsurance? Sure, where a single event could affect multiple policies. Tension between auditability and security in the power industry (NERC) – is there any role for insurance? Maybe, but legal penalties are in general uninsurable. How do we get insurers to come to WEIS? It would help if we had more specificity in our research papers, if we did not just talk about “breaches” but “breaches resulting in X” (the industry is not interested in national security, corporate espionage and other things that do not result in claims). Market evolution? She predicts the industry will follow its usual practice of lowballing a new market until losses mount, then cut back coverage terms. (E.g. employment liability insurance grew rapidly over last 20 years but became unprofitable because of class actions for discrimination etc – so industry cut coverage, but that was OK as it helped shape best employment practice). Data sharing by industry itself? Client confidentiality stops ad-hoc sharing but it would be good to have a properly regulated central depository. Who’s the Ralph Nader of this? Broad reform might come from the FTC; it’s surprising the SEC hasn’t done anything (HIPAA and GLB are too industry-specific). Quantifiability of best practice? Not enough data. How much of biz is cyber? At present it’s 5% of Chubb’s insurance business, but you can expect 8-9% in 2010-11 – rapid growth!

Future sessions will be covered in additional posts…

The coalition Government plans to keep the Summary Care Record, despite pre-election pledges by both the Conservatives and the Liberal Democrats to rip up the system – which is not compliant with the I v Finland judgement of the European Court of Human Rights.

Last year colleagues and I wrote Database State, a report for the Joseph Rowntree Reform Trust, which studied 46 systems that keep information on all of us, or at least a significant minority of us. We concluded that eleven of them were almost certainly illegal under human-rights law, and most of the rest had problems. Our report was well received by both Conservatives and Lib Dems; many of its recommendations were adopted as policy.

Old-timers may recall that back in 1996-7, many of us geeks supported New Labour enthusiastically, as Blair promised not to introduce key escrow. It took him almost a year to renege on that promise; it has taken the coalition less than a month.

Blair’s U-turn on key escrow in 1998 led to the establishment of FIPR, and a two-year fight against what became the RIP Act (where at least we limited escrow to the powers in part 3). What’s the appropriate response now to Cameron and Clegg?

It’s inconceivable that assurances given to farmers, or to soldiers, or to teachers would be tossed aside so casually. Yet half a million of us earn our living in IT in Britain – there’s a lot more of us than of any of them! And many people in other jobs care about privacy, copyright, and other digital issues. So do those of us who care about digital policy have to become more militant? Or do we have to raise money and bribe the ruling parties? Or, now that all three major parties are compromised, should we downgrade our hopes for parliament and operate through the courts and through Europe instead?