District Court Holds that Running Hash Values on Computer Is A Search: The case is United States v. Crist, 2008 WL 4682806 (M.D.Pa. October 22 2008) (Kane, C.J.). It's a child pornography case involving a warrantless search that raises a very interesting and important question of first impression: Is running a hash a Fourth Amendment search? (For background on what a "hash" is and why it matters, see here).

First, the facts. Crist is behind on his rent payments, and his landlord starts to evict him by hiring Sell to remove Crist's belongings and throw them away. Sell comes a cross Crist's computer, and he hands over the computer to his friend Hipple who he knows is looking for a computer. Hipple starts to look through the files, and he comes across child pornography: Hipple freaks out and calls the police. The police then conduct a warrantless forensic examination of the computer:

In the forensic examination, Agent Buckwash used the following procedure. First, Agent Buckwash created an "MD5 hash value" of Crist's hard drive. An MD5 hash value is a unique alphanumeric representation of the data, a sort of "fingerprint" or "digital DNA." When creating the hash value, Agent Buckwash used a "software write protect" in order to ensure that "nothing can be written to that hard drive." Supp. Tr. 88. Next, he ran a virus scan, during which he identified three relatively innocuous viruses. After that, he created an "image," or exact copy, of all the data on Crist's hard drive.

Agent Buckwash then opened up the image (not the actual hard drive) in a software program called EnCase, which is the principal tool in the analysis. He explained that EnCase does not access the hard drive in the traditional manner, i.e., through the computer's operating system. Rather, EnCase "reads the hard drive itself." Supp. Tr. 102. In other words, it reads every file-bit by bit, cluster by cluster-and creates a index of the files contained on the hard drive. EnCase can, therefore, bypass user-defined passwords, "break down complex file structures for examination," and recover "deleted" files as long as those files have not been written over. Supp. Tr. 102-03.

Once in EnCase, Agent Buckwash ran a "hash value and signature analysis on all of the files on the hard drive." Supp. Tr. 89. In doing so, he was able to "ingerprint" each file in the computer. Once he generated hash values of the files, he compared those hash values to the hash values of files that are known or suspected to contain child pornography. Agent Buckwash discovered five videos containing known child pornography. Attachment 5. He discovered 171 videos containing suspected child pornography.

One of the interesting questions here is whether the search that resulted was within the scope of Hipple's private search; different courts have approached this question differently. But for now the most interesting question is whether running the hash was a Fourth Amendment search. The Court concluded that it was, and that the evidence of child pornography discovered had to be suppressed:

The Government argues that no search occurred in running the EnCase program because the agents "didn't look at any files, they simply accessed the computer." 2d Supp. Tr. 16. The Court rejects this view and finds that the "running of hash values" is a search protected by the Fourth Amendment.

Computers are composed of many compartments, among them a "hard drive," which in turn is composed of many "platters," or disks. To derive the hash values of Crist's computer, the Government physically removed the hard drive from the computer, created a duplicate image of the hard drive without physically invading it, and applied the EnCase program to each compartment, disk, file, folder, and bit.2d Supp. Tr. 18-19. By subjecting the entire computer to a hash value analysis-every file, internet history, picture, and "buddy list" became available for Government review. Such examination constitutes a search.

I think this is generally a correct result: See my article Searches and Seizures in a Digital World, 119 Harv. L. Rev. 531 (2005), for the details. Still, given the lack of analysis here it's somewhat hard to know what to make of the decision. Which stage was the search — the creating the duplicate? The running of the hash? It's not really clear. I don't think it matters very much to this case, because the agent who got the positive hit on the hashes didn't then get a warrant. Instead, he immediately switched over to the EnCase "gallery view" function to see the images, which seems to be to be undoudtedly a search. Still, it's a really interesting question.

I find that legacy code poses a unique challenge for organizations rolling out a new security process.  Often, the resources dedicated to maintaining older code are a small fraction of those devoted to new features or products.  Furthermore, the original developers for such features have often moved on, leaving no subject matter experts to drive reviews.  The astute reader will ask “How do I apply the principles of the Microsoft SDL to legacy code when I have no development resources and nobody knows how it works?”

The answer is “Start small, and build expertise over time.”

A Rising Tide Lifts All Boats

The best thing a security engineering team can do to improve security in the short term is to drive code quality, and the first step in this process is to define and enforce a secure coding standard.  This helps on two fronts: 

1.       It will improve code quality and reduce implementation flaws across the entire code base.  Unlike other security processes, driving a secure coding standard is relatively easy to accomplish across an entire code base, regardless of the code’s age, by a focused security team.  That is not to say that it is easy without qualification – a large batch of spaghetti code will require a lot of work to untangle!  Such an effort can only be called “easy” when compared to, say, comprehensive identification and remediation of design flaws across legacy features.  Even so, improving code quality through the use of secure coding standards offers a unique combination of high impact, applicability to features, and ability to be carried out by a core team that makes it a sensible first step.

2.       The security team might notice that some sections of code have more standards violations or outright flaws than others.  This is an instance of vulnerability clustering, a concept that has been used to predict vulnerability rates and improve quality in the functional realm.  The evidence is anecdotal, but it stands to reason that portions of code that consistently violate secure coding standards are good places to start looking for other classes of security flaw.  These are security hotspots, and should be high on the prioritized list for further review.

Security testing may also be applied to legacy code, but initial activities should be considered on a case-by-case basis based on the expected return on investment.  Such testing ranges from using inexpensive off-the-shelf tools to exercise common interfaces to rather expensive custom testing and formal analysis.  It is worthwhile to begin with off-the-shelf tools, such as those that target file parsers or web applications, and tools created as part of your greater secure development efforts.  These can help identify easily-found flaws and suggest improvements to the coding standards.  Comprehensive security testing, on the other hand, is best tackled after the Legacy Security Push.

The Legacy Security Push

Coding standards and basic testing provide bang for the buck, but formal security processes seek to provide security assurance.  The challenge for legacy code is that it needs to play catch-up.  Security processes that occur early in the development cycle, such as requirements analysis, design review, and threat modeling, are particularly difficult to achieve years after the fact.  The main goal of the Legacy Security Push is to create the deliverables from these efforts, the most important of which are security requirements and a full risk analysis.

It may sound trivial, but security requirements are essential.  Not only do they define proper operation for the system in question, they also define assumptions that are suitable for relying systems.   It is very common to find security flaws in legacy systems that arise from well-intentioned but incorrect assumptions such as “I assume that the Foo authenticates server Bar when initiating a bank transfer.”  It stands to reason that Foo would do so for such an important activity, but this assumption must be validated.  It is very common for older features to have been deployed in and written for different environments where the security assumptions that are "obvious" today just didn't apply at the time.

When reviewing legacy systems, the first step is to identify such requirements.  If the original architects, developers or managers are available, they can provide valuable insight at this stage.  More often than not this is not the case, and analysis must instead rely on what documentation is present and interaction between the software and its consumers.  The goal is the same as in requirements analysis during project inception, except that in this case one must turn the process on its head and reverse engineer requirements from system behavior.  At the conclusion of this effort, requirements can be theorized – “Foo must authenticate its server Bar before initiating a bank transfer.” 

Risk analysis can be performed once a plausible set of requirements have been identified.  Threat modeling is a more structured means of performing such an analysis, with the eventual goal of identifying means by which requirements can be violated by an attacker. 

As with requirements analysis, original developers would be a valuable resource to consult.  With or without such help, the first step is to identify how the software works.  In many cases, help is not available and performing this task requires a great deal of effort.  For features of moderate size, this author has spent upwards of a month reading code, using process profiling tools, and walking through the software with a debugger to identify program flow and security-sensitive functionality.

Once completed, actual system behavior should be documented and compared against the requirements theorized.   It might be that the requirements should be re-evaluated (New requirement:  Do not assume that Foo requires server authentication) or the system may need to be changed (New bug:   Foo does not verify the CN for Bar).  At the end, this information should be sufficient to support a comprehensive threat modeling exercise where security requirements, risks, and their mitigations can be documented.

Next Steps

Bringing a legacy feature up to par with its newer kin requires a relatively small number of items:  improved code quality, clear security requirements, and a thorough threat model.  As we have seen, performing even these tasks is quite the effort!  I am sure that it is little comfort to be reminded that accomplishing these tasks has simply laid the foundation, and that the true benefit is that the newly-reviewed legacy feature is able to participate fully in the security processes that remain: reviewing cross-component security requirements and assumptions, comprehensive testing, and incident planning, to name a few.

Unfortunately, there is no silver bullet in security assurance.  The soundness of the design and implementation of legacy software is just as important as in newer software, which is why any complete secure software development process will look backwards as well as forwards.  Feature by feature, from higher priority to lower, the overall security of the software improves as legacy code receives the full security treatment it deserves.

In a new paper, Bernd Roellgen of Munich-based encryption outfit PMC Ciphers, explains how it is possible to compare an encrypted backup image file made with almost any commercial encryption program or algorithm to an original that has subsequently changed so that small but telling quantities of data 'leaks'.

Here's the paper. Turns out that if you use a block cipher in Electronic Codebook Mode, identical plaintexts encrypt to identical ciphertexts.

Yeah, we already knew that.

And -1 point for a security company requiring the use of Javascript, and not failing gracefully for a browser that doesn't have it enabled.

And -- ahem -- what is it with that photograph in the paper? Couldn't the researchers have found something a little less adolescent?

For the record, I doghoused PMC Ciphers back in 2003:

PMC Ciphers. The theory description is so filled with pseudo-cryptography that it's funny to read. Hypotheses are presented as conclusions. Current research is misstated or ignored. The first link is a technical paper with four references, three of them written before 1975. Who needs thirty years of cryptographic research when you have polymorphic cipher theory?

EDITED TO ADD (10/9): I didn't realize it, but last year PMC Ciphers responded to my doghousing them. Funny stuff.

Proxy caches, combined with poorly written session management code, can easily leads to serious security flaws similar to what we highlighted in A New Security Breach in Google Docs Revealed.

Web developers have no control over proxy caches in the Internet. However, developers do have control of the code they write and their admin teams have configuration control of their web servers. Developers must assume the worst case Internet scenario with aggressive Internet cache management policies that serve cached data for economic and performance reasons.

As a consequence, this fact-of-life on the Internet sometimes results in multiple web clients being sent the same Set-Cookie HTTP headers, for example.  Caching proxy servers should obtain a fresh cookie for the each new client request. Ideally, proxy caches should not cache session management cookies and distribute cached cookies to multiple clients. However, application developers cannot assume that proxy caches are well behaved, especially for applications where security and privacy are required.

Web developers cannot know whether their content is consumed directly or via a proxy cache. Developers also cannot assume that the HTTP responses will be delivered to the intended browser. Moreover, developers cannot be sure that the intended browser even receives the intended content.  For example, a session ID issued to a client gets used while it is valid or until abandoned and expired. If it is served and delivered in response to an unencrypted HTTP GET request, there’s no guarantee it will be consumed by the intended web browser.

Ideally, SSL should be used on all web transactions that require confidentiality and privacy, including our recent Google Docs breach.  On the other hand, even SSL is not foolproof. For example, many web developers do not correctly set the “Encrypted Sessions Only” cookie property. These incorrectly configured “secure” servers will send HTTPS cookies in the open, unencrypted.

There be dragons …

Proxy Caches and Web Application Security

Abstract:  Proxy caches, combined with poorly written session management code, can easily lead to serious Internet security breaches. Web application developers cannot know whether their content is consumed directly or via a proxy cache. Developers cannot assume that the HTTP responses will be delivered to the intended browser. Moreover, developers cannot be sure that the intended browser even receives the intented content. Consequently, proxy caches are a serious theat to web application security.  In the presentation, we will discuss the recent security breach Tim found in Google Docs and review web application security and session management topics related to proxy caching.