June 06, 2004

Open Source won't save e-voting

In a NYT Magazine article published may 30th, Clive Thompson argues for Open Source code for electronic voting machines:
First off, the government should ditch the private-sector software makers. Then it should hire a crack team of programmers to write new code. Then -- and this is the crucial part -- it should put the source code online publicly, where anyone can critique or debug it. This honors the genius of the open-source movement. If you show something to a large enough group of critics, they'll notice (and find a way to remove) almost any possible flaw. If tens of thousands of programmers are scrutinizing the country's voting software, it's highly unlikely a serious bug will go uncaught.

It may very well be a good thing to have Open Source software for voting, but the assumption that underlies Thompsons's argument--that Open Source is somehow a magic engine for producing bug-free software--is transparently false. Open Source software, like all software, is riddled with bugs. Many of these bugs have security implications. Moreover, these bugs can persist for long periods of time. For instance, the Linux mremap() problem (CAN-2004-0077, described here) has been in every Linux kernel since at least 2.2 (released in 1999) and was only discovered in March of 2004. Alternately, consider the OpenSSL buffer overflows. These had been around since at least 1998 and probably earlier but were only found in 2002. So much for tens of thousands of programmers finding any serious bug.

The truth of the matter is that--contrary to popular myth--practically nobody bothers to audit any Open Source code. Auditing code is a mind-destroyingly boring exercise and it's not even clear what percentage of vulnerabilities a good audit actually finds (practically no research has been done on this point). I'm probably one of the 50-100 people most qualified to audit OpenSSL: I'm a security guy specializing in SSL who uses OpenSSL on a regular basis and I've spent substantial amounts of time groveling through the source code. But the only time I look for security holes is if I happen to run into something that looks fishy. Noone I know seriously believes that we've found the last security hole in OpenSSL or Linux.

This isn't to say that auditing source code isn't worthwhile. It's just that the idea that we can audit the bugs out of a piece of code is in my view fundamentally misguided. What audits are useful for is getting an idea of how good the overall code quality is. If you audit selected pieces of a piece of software and you find a bunch of serious errors, it's safe to conclude that the company needs to shoot the programmers and start over. So, for instance, when Kohno et al. found a bunch of problems in Diebold's e-voting system, the conclusion to draw wasn't that these were all the problems that there were but rather Diebold didn't have the first clue how to write secure software. As Avi Rubin's page on e-voting points out:

To help mitigate the risks identified in the security analyses, Maryland proposed a set of technological changes to Diebold's voting machines as well as procedural changes to the election process. While this may help "raise the bar," it is impossible to know whether any security analysis identifies all the possible vulnerabilities present in an analyzed system. By only patching the known vulnerabilities, Maryland is not actually ensuring that the voting system will be secure. Rather, Maryland should follow security engineering best practices, which state that security can only be assured through a rigorous design process that considers security from a project's conception, not through a set of patches applied after the fact.

So, if we're going to have e-voting, what we really need is a procedure that allows for routinized and systematic review of voting systems. The purpose of this review is not to find vulnerabilities--though of course it would be nice to fix any that are found--but rather to assess whether the vendor is following good software engineering practices, with serious consequences if they are not. Open Source might or might not help us get that sort of review--personally, I expect that to get really thorough review you'll need to pay people to do it--but just making the code Open Source doesn't improve the situation much at all. Posted by Eric Rescorla at June 6, 2004 06:04 PM