The good news is that due to the nature of this problem, it is extremely difficult to determine the vulnerability merely by analyzing the patches; a common technique malicious individuals use to figure out security weaknesses.

The typical response I heard was “what do you mean, it can’t be reverse engineered? I’ll just look at the diffs!”

In hindsight, after examining the BIND diffs (yes, I did it too) and discussing with colleagues, all most people saw was UDP source port randomization and a better PRNG for generating the transaction ID, the latter of which would appear to be related to Amit Klein’s cache poisoning attack from about a year ago.

What Rich was really saying is that you can reverse engineer the patch until you’re blue in the face, but that won’t reveal the specifics of the vulnerability.

Dan’s blog post this morning appeared to confirm that interpretation:

DJB was right. All those years ago, Dan J. Bernstein was right: Source Port Randomization should be standard on every name server in production use.

There is a fantastic quote that guides a lot of the work I do: Luck is the residue of design. Dan Bernstein is a notably lucky programmer, and that’s no accident. The professor lives and breathes systems engineering in a way that my hackish code aspires to one day experience. DJB got “lucky” here — he ended up defending himself against an attack he almost certainly never encountered.

Such is the mark of excellent design. Excellent design protects you against things you don’t have any information about. And so we are deploying this excellent design to provide no information.

To translate the fix strategy into a more familiar domain, imagine large chunks of Windows RPC went from Anonymous to Authenticated User only, or even all the way to Admin Only. Or wait, just remember Windows XPSP2 :) This is a sledgehammer, by design. It cuts off attack surface, without necessarily saying why. Astonishingly subtle bugs can be easily hidden, or even rendered irrelevant, by a suitably blunt fix.

Nate McFeters appears to think that Tom Ptacek has figured it out. I’m going to go out on a limb and say that Tom didn’t figure anything out yet but still wanted to write a pithy blog post. I think that if Tom had figured it out, he would have written it down privately and posted the SHA-1 hash, as is the trendy thing to do these days.

Speculation aside, the title of Tom’s blog entry, Dan Kaminsky could have made hundreds of thousands of dollars with this DNS flaw!, does make an important point — Dan didn’t sell the details to ZDI, he used his influence and reputation to coordinate a massive vendor patch effort. That’s an admirable move.

Since the SVN repository remained intact, we can’t go back and examine the backdoor in detail. However, we do have a newsgroup posting that sheds a little light on the situation:

> What diff do you see between the compromised version and
> the one that is there now? I see only a comment diff in one file.

it was a small block of code that checks for a $_SERVER var. If that var was present, it would redefine SM_PATH. Under normal circumstances, this would never be executed, but we have since learned how to make it execute.

In PHP, $_SERVER is an array populated by the web server that contains information such as headers, paths, and script locations. This includes some user-supplied input such as the URL query string and the HTTP headers. SM_PATH is the filesystem path where SquirrelMail is configured to be run from. So once an attacker controls SM_PATH, it’s likely that a subsequent call to include() can be exploited to fetch and execute PHP code from a remote server. This is a typical example of a Remote File Include vulnerability.

Note that the attacker backdoored the 1.5.1 distribution as well, with the same type of vulnerability but at a different location in the codebase.

I think what’s most interesting to me about this is that so many open source projects still rely on MD5 hashes for integrity checking. The minute the Xiaoyun Wang paper on MD5 collisions was released, every security practitioner in the world considered MD5 unsafe from that point forward. Even though practical attacks had not yet been formulated, the writing was on the wall. Unfortunately, the rest of the world either didn’t notice or didn’t care.

Cryptographers have since developed increasingly sophisticated attacks stemming from Wang’s original work. Recently, researchers in the Netherlands demonstrated two examples of chosen-prefix attacks which would make it possible for an attacker to take two tarballs (one original, one backdoored) and append a series of bytes to each that result in both files having the same MD5 hash. This proves beyond a shadow of a doubt that MD5 is not an effective method for verifying software integrity. There was hardly any doubt that this attack would surface eventually, so why is MD5 still in such widespread usage?

Cryptographic weaknesses aside, a lot of people completely miss the mark with hashes. MD5 or SHA-1 (or any hash function) are not very effective if the only way a user can verify them is on the same website where the download is hosted. If the download point is compromised, chances are the attacker can modify the hashes printed on the website too. Even when it’s done correctly, hashes only help identify when the distribution point is compromised. It does nothing to protect against source code compromise or vulnerabilities in the development tool chain.