Hash functions are the most commonly used cryptographic primitive, and the most poorly understood. You can think of them as fingerprint functions: They take an arbitrary long data stream and return a fixed length, and effectively unique, string. The security comes from the fact that while it's easy to generate the fingerprint from a file, it's infeasible to go the other way and generate a file given a fingerprint.

Originally created to make digital signatures more efficient, hashes are now used to secure the very fundamentals of our information infrastructure: in password logins, secure web connections, encryption key management, virus and malware scanning, and almost every cryptographic protocol in current use. Without cryptographic hash functions, the internet would simply not work. At the same time, there isn't a good theory of hash functions. Unlike encryption algorithms, there are no secret keys involved; this makes it harder to mathematically define exactly what hash functions are.

The National Institute of Standards and Technology, NIST, is holding a competition to replace the SHA family of hash functions. "SHA" stands for "Secure Hash Algorithm." It was developed by the NSA in 1993 to replace the commercial MD4 and MD5 algorithms, and has been updated several times since then. All the SHA algorithms are very similar, and have been increasingly under attack, so NIST wants to replace them.

The competition is important because, unlike other technological standards, committee design — balancing the interests of diverse constituents — isn't conducive to good security. Security is best when it's designed by expert teams and then subjected to public review. And cryptography is best when it's chosen by competition.

In 1997, NIST held a competition for a block cipher to replace DES. Fifteen candidates and three-and-a-half years later, Rijndael became the new Advanced Encryption Standard — AES. NIST is doing the same thing for what it's calling SHA-3 (not, for some unexplained reason, the Advanced Hash Standard or AHS).

The deadline was October 31, and NIST received 64 submissions. This isn't surprising — I predicted 80 — as most of the 15 AES submitters were professors, whose students at the time have become professors themselves, with their own students. (If NIST does a stream cipher competition in another ten years, they should expect about 256 submissions.) These submissions came from academia, from industry, and from hobbyists. CIO magazine recently interviewed one of the submitters, who is 15. Twenty-eight submissions have been made public by the submitters, and six of those have been broken.

NIST is going through all the submissions right now, making sure they are complete and proper. Their goal is to publish all accepted submissions by the end of November, in advance of the First Hash Function Candidate Conference, to be held in Belgium right after the Fast Software Encryption workshop in February.

The group expects to quickly make a first cut of algorithms — hopefully to about a dozen — and give the community a year of cryptanalysis before making a second cut in 2010. After another year of cryptanalysis, NIST will choose a winner in 2011. Expect a final standard by 2012.

My advice for software developers is to let the process run its course. While it's tempting to use the new cool algorithms in your designs, it's far too soon to trust any of them. This process is likely to result in all sorts of new research results in hash function security, and some real cryptanalytic surprises. Give the community a few years to figure out which ones are good and which aren't.

I've previously called this sort of thing a cryptographic demolition derby: The last one left standing wins. But that's only partially true. Certainly all the groups will spend the next few years trying to cryptanalyze each other, but in the end there will be a bunch of unbroken algorithms. NIST will select one based on performance and features.

NIST has stated that the goal of this process is not to choose the best standard but to choose a good standard. I think that's smart; in this process, the best is the enemy of the good. While there's no rush to choose a new standard — the SHA-2 algorithms will remain secure for the foreseeable future — we don't want to analyze the candidates forever.

Personally, I was part of a group of eight cryptographers that submitted Skein to the competition. A decade ago, writing Twofish and participating in the AES process was the most fun I had ever had in cryptography. These next few years promise to be even more fun.

---

Bruce Schneier is chief security technology officer of BT. His new book is Schneier on Security.

I find that legacy code poses a unique challenge for organizations rolling out a new security process.  Often, the resources dedicated to maintaining older code are a small fraction of those devoted to new features or products.  Furthermore, the original developers for such features have often moved on, leaving no subject matter experts to drive reviews.  The astute reader will ask “How do I apply the principles of the Microsoft SDL to legacy code when I have no development resources and nobody knows how it works?”

The answer is “Start small, and build expertise over time.”

A Rising Tide Lifts All Boats

The best thing a security engineering team can do to improve security in the short term is to drive code quality, and the first step in this process is to define and enforce a secure coding standard.  This helps on two fronts: 

1.       It will improve code quality and reduce implementation flaws across the entire code base.  Unlike other security processes, driving a secure coding standard is relatively easy to accomplish across an entire code base, regardless of the code’s age, by a focused security team.  That is not to say that it is easy without qualification – a large batch of spaghetti code will require a lot of work to untangle!  Such an effort can only be called “easy” when compared to, say, comprehensive identification and remediation of design flaws across legacy features.  Even so, improving code quality through the use of secure coding standards offers a unique combination of high impact, applicability to features, and ability to be carried out by a core team that makes it a sensible first step.

2.       The security team might notice that some sections of code have more standards violations or outright flaws than others.  This is an instance of vulnerability clustering, a concept that has been used to predict vulnerability rates and improve quality in the functional realm.  The evidence is anecdotal, but it stands to reason that portions of code that consistently violate secure coding standards are good places to start looking for other classes of security flaw.  These are security hotspots, and should be high on the prioritized list for further review.

Security testing may also be applied to legacy code, but initial activities should be considered on a case-by-case basis based on the expected return on investment.  Such testing ranges from using inexpensive off-the-shelf tools to exercise common interfaces to rather expensive custom testing and formal analysis.  It is worthwhile to begin with off-the-shelf tools, such as those that target file parsers or web applications, and tools created as part of your greater secure development efforts.  These can help identify easily-found flaws and suggest improvements to the coding standards.  Comprehensive security testing, on the other hand, is best tackled after the Legacy Security Push.

The Legacy Security Push

Coding standards and basic testing provide bang for the buck, but formal security processes seek to provide security assurance.  The challenge for legacy code is that it needs to play catch-up.  Security processes that occur early in the development cycle, such as requirements analysis, design review, and threat modeling, are particularly difficult to achieve years after the fact.  The main goal of the Legacy Security Push is to create the deliverables from these efforts, the most important of which are security requirements and a full risk analysis.

It may sound trivial, but security requirements are essential.  Not only do they define proper operation for the system in question, they also define assumptions that are suitable for relying systems.   It is very common to find security flaws in legacy systems that arise from well-intentioned but incorrect assumptions such as “I assume that the Foo authenticates server Bar when initiating a bank transfer.”  It stands to reason that Foo would do so for such an important activity, but this assumption must be validated.  It is very common for older features to have been deployed in and written for different environments where the security assumptions that are "obvious" today just didn't apply at the time.

When reviewing legacy systems, the first step is to identify such requirements.  If the original architects, developers or managers are available, they can provide valuable insight at this stage.  More often than not this is not the case, and analysis must instead rely on what documentation is present and interaction between the software and its consumers.  The goal is the same as in requirements analysis during project inception, except that in this case one must turn the process on its head and reverse engineer requirements from system behavior.  At the conclusion of this effort, requirements can be theorized – “Foo must authenticate its server Bar before initiating a bank transfer.” 

Risk analysis can be performed once a plausible set of requirements have been identified.  Threat modeling is a more structured means of performing such an analysis, with the eventual goal of identifying means by which requirements can be violated by an attacker. 

As with requirements analysis, original developers would be a valuable resource to consult.  With or without such help, the first step is to identify how the software works.  In many cases, help is not available and performing this task requires a great deal of effort.  For features of moderate size, this author has spent upwards of a month reading code, using process profiling tools, and walking through the software with a debugger to identify program flow and security-sensitive functionality.

Once completed, actual system behavior should be documented and compared against the requirements theorized.   It might be that the requirements should be re-evaluated (New requirement:  Do not assume that Foo requires server authentication) or the system may need to be changed (New bug:   Foo does not verify the CN for Bar).  At the end, this information should be sufficient to support a comprehensive threat modeling exercise where security requirements, risks, and their mitigations can be documented.

Next Steps

Bringing a legacy feature up to par with its newer kin requires a relatively small number of items:  improved code quality, clear security requirements, and a thorough threat model.  As we have seen, performing even these tasks is quite the effort!  I am sure that it is little comfort to be reminded that accomplishing these tasks has simply laid the foundation, and that the true benefit is that the newly-reviewed legacy feature is able to participate fully in the security processes that remain: reviewing cross-component security requirements and assumptions, comprehensive testing, and incident planning, to name a few.

Unfortunately, there is no silver bullet in security assurance.  The soundness of the design and implementation of legacy software is just as important as in newer software, which is why any complete secure software development process will look backwards as well as forwards.  Feature by feature, from higher priority to lower, the overall security of the software improves as legacy code receives the full security treatment it deserves.

Lots being written about the Cloud, most of it quite dark and gloomy.  In fact I’m surprised, that Hoff hasn’t got a preso spooled up called “The Toxic Cloud” or something similarly ominous for his next speaking tour.
That said, the Economist does a great job distilling the issue into a simple statement -

Cloud computing is a trade-off between sovereignty and efficiency.

Let me ask you -  if you had to put your money on one of those horses, considering your average profit-preoccupied business, which would it be?  I’d put my bottom dollar on the thoroughbred named “Cost Center Reduction”, to place.

WHO ARE WE TO STAND IN THE WAY OF “PROGRESS”?

I’m always fond of Jack’s rule that the role of information risk management boils down to three deceptively simple premises:

• Reduce Risk.
• Reduce Loss.
• Create Operational Efficiencies.

So it would seem antithetical to the charter of the Chief Security Officer to stand in the way of progress as embodied by “cloud computing” (not to mention dangerous to long-term job security).  And I think that this presents opportunities to discuss strategies for managing risk, strategies that aren’t too theoretical and have practical application (though actual “cloud” use by enterprises may be rare at this point).

ON RISK REDUCTION IN THE CLOUD (or, How To Learn From the Shortcomings of PCI DSS)

The good news is, there’s already a well-established model for managing the risk around outsourcing the processing of “confidential” information.  The bad news is, that model kinda sucks it.

The Payment Card Industry, known as the “PCI” or “meal ticket” to many in the industry, faced a similar problem with the introduction of GLBA.  As I see it (and I’m not at all close to the PCI, at all, so this is all just abstract soliloquy) the PCI had one of two choices when faced with the prospect of other people managing their sensitive information:

1. Accept the *massive* amount of GLBA risk their business creates and spend a TON of money to build out the infrastructure (both process and IT) to manage the consumer data themselves (in conjunction with the banks, of course) and never have it grace the computing systems of the retailer.  Or,
2. Transfer the GLBA risk down to the retailer and have them bear the majority of the risk (and cost of reducing risk to a level that might be tolerable to the US Government).

(Martin, you may recall our Twittering about PCI a while back.  This is the crux of my view on the subj.)

Now fortunately, the CSO’s of the world are going to be a little more “invested” in protecting the information they are stewards over, and unlike the PCI, will remain primarily responsible for the C, I, & A of the data in the Cloud.  The cool thing is, this actually presents a great opportunity to start building a meaningful model for co-management of risk!  In fact, we can take the PCI model of contractual risk transference but modify where it goes all wrong, and start working to create something better.  And we can start by euthanizing some faulty assumptions.

JUST HOW INFORMATIVE IS PCI DSS?

What might be the.greatest.mistake of the standards compliance mentality is the assumption of value for the past-state measurement.  That is, I believe that the CSO needs more than some “past-state” assurance in order to understand their risk.    If you look at the concept of “PCI compliance” it really is an examination of a past state of nature that is assumed to be relevant to current and future states.   Many people (myself included) are not at all convinced that this past-state is nearly as informative as those who mandate it’s measurement believe it to be.

That’s not to condemn past-state measurements as completely non-informative,  they most certainly are useful.  It’s just that no self-respecting CSO sleeps well because they were deemed “PCI compliant” 10 months ago.  They sleep well because they have good visibility into current-state information and confidence in their strategy concerning future-state (based on that visibility and the outcomes of sound IRM models).

MOVING PAST THE VULNERABILITY SCANNER INTO INTELLIGENCE AND WISDOM

So realizing this new importance (to me, at least) concerning visibility and IRM models, I’m lead to the conclusion that if we are to manage risk in the Cloud, we’ll have to move beyond “PCI Compliance” or the concept that some regular “audit” of controls in place at the host is all we need to understand our ability to manage risk.  No, the CSO must have good information concerning current and probable future states.   This is that “visibility” I spoke of above.  In fact, we’ll need significant amounts of piercing, transparent visibility.  And in order to gain that visibility, our insight into Cloud Risk Management must include significant provisions for understanding a joint ability to Prevent/Detect/Respond as well as provisions for managing the risk that one of the participants won’t provide that visibility or ability via SLA’s and penalties . These SLA’s must be expressed in measurable terms (more visibility), and those metrics must have their roots in the things that help understand how we manage risk (those aforementioned IRM models).

THE CLOUD COMPUTING SECURITY SILVER LINING (sorry couldn’t resist)

As I mentioned earlier, I do see an opportunity to create insight.  The need for visibility and IRM models would allow us to create a “guidance” if you’ll allow me to use the term.  Not a standard or a “best practice” to audit by, but simply a reference document that says “if you’re going to put information on somebody else’s systems and still hold some significant responsibility for that information, here’s the considerations, why they are considerations, and how you might go about collaborating on the management of risk”.

And I think that if we undertake this journey, there is going to be a lot of growth and risk management innovation along the way.  But keen insights into what it means to manage risk will be necessary, and secure and forthright collaboration will be of absolute importance.

I say that last bit because, if these pundits are right about the utility of a hosted computing model - the Cloud will happen regardless of the CSO’s ability or desire to manage it.