Hash functions are the most commonly used cryptographic primitive, and the most poorly understood. You can think of them as fingerprint functions: They take an arbitrary long data stream and return a fixed length, and effectively unique, string. The security comes from the fact that while it's easy to generate the fingerprint from a file, it's infeasible to go the other way and generate a file given a fingerprint.

Originally created to make digital signatures more efficient, hashes are now used to secure the very fundamentals of our information infrastructure: in password logins, secure web connections, encryption key management, virus and malware scanning, and almost every cryptographic protocol in current use. Without cryptographic hash functions, the internet would simply not work. At the same time, there isn't a good theory of hash functions. Unlike encryption algorithms, there are no secret keys involved; this makes it harder to mathematically define exactly what hash functions are.

The National Institute of Standards and Technology, NIST, is holding a competition to replace the SHA family of hash functions. "SHA" stands for "Secure Hash Algorithm." It was developed by the NSA in 1993 to replace the commercial MD4 and MD5 algorithms, and has been updated several times since then. All the SHA algorithms are very similar, and have been increasingly under attack, so NIST wants to replace them.

The competition is important because, unlike other technological standards, committee design — balancing the interests of diverse constituents — isn't conducive to good security. Security is best when it's designed by expert teams and then subjected to public review. And cryptography is best when it's chosen by competition.

In 1997, NIST held a competition for a block cipher to replace DES. Fifteen candidates and three-and-a-half years later, Rijndael became the new Advanced Encryption Standard — AES. NIST is doing the same thing for what it's calling SHA-3 (not, for some unexplained reason, the Advanced Hash Standard or AHS).

The deadline was October 31, and NIST received 64 submissions. This isn't surprising — I predicted 80 — as most of the 15 AES submitters were professors, whose students at the time have become professors themselves, with their own students. (If NIST does a stream cipher competition in another ten years, they should expect about 256 submissions.) These submissions came from academia, from industry, and from hobbyists. CIO magazine recently interviewed one of the submitters, who is 15. Twenty-eight submissions have been made public by the submitters, and six of those have been broken.

NIST is going through all the submissions right now, making sure they are complete and proper. Their goal is to publish all accepted submissions by the end of November, in advance of the First Hash Function Candidate Conference, to be held in Belgium right after the Fast Software Encryption workshop in February.

The group expects to quickly make a first cut of algorithms — hopefully to about a dozen — and give the community a year of cryptanalysis before making a second cut in 2010. After another year of cryptanalysis, NIST will choose a winner in 2011. Expect a final standard by 2012.

My advice for software developers is to let the process run its course. While it's tempting to use the new cool algorithms in your designs, it's far too soon to trust any of them. This process is likely to result in all sorts of new research results in hash function security, and some real cryptanalytic surprises. Give the community a few years to figure out which ones are good and which aren't.

I've previously called this sort of thing a cryptographic demolition derby: The last one left standing wins. But that's only partially true. Certainly all the groups will spend the next few years trying to cryptanalyze each other, but in the end there will be a bunch of unbroken algorithms. NIST will select one based on performance and features.

NIST has stated that the goal of this process is not to choose the best standard but to choose a good standard. I think that's smart; in this process, the best is the enemy of the good. While there's no rush to choose a new standard — the SHA-2 algorithms will remain secure for the foreseeable future — we don't want to analyze the candidates forever.

Personally, I was part of a group of eight cryptographers that submitted Skein to the competition. A decade ago, writing Twofish and participating in the AES process was the most fun I had ever had in cryptography. These next few years promise to be even more fun.

---

Bruce Schneier is chief security technology officer of BT. His new book is Schneier on Security.

Police had no idea that one SIM card could be used simultaneously from two handsets before the detention of Nazir Ahmed for interrogation. Nazir was picked up from Morigaon after an SMS from his mobile number in the name of ISF-IM claimed responsibility for Thursday's blasts in Assam.

Nazir had a Reliance connection and an Eve handset. Each handset of this particular model has a unique International Mobile Equipment Identity (IMEI) number. Cops found that two IMEI numbers were using the same SIM. Accordingly there were two record sheets of calls and SMSes from Nazir's mobile number. The record of the SMS to the media was found in only one sheet, which forced police to believe that Nazir's SIM might have been cloned and someone else was using the duplicate card, with or without the owner's knowledge.

"We stumbled upon this technological surprise that Nazir Ahmed's SIM card was used in two handsets," Assam IG (Law and Order) Bhaskarjyoti Mahanta said.

So far, not that interesting. There are lots of vulnerabilities in technological systems, and it's generally a race between the good guys and the bad guys to see who finds them first. It's the last sentence of this article that's significant:

The experts said no one has actually done any research on SIM card cloning because the activity is illegal in the country.

If the good guys can't even participate, the bad guys will always win.

"Risk management" is just a fancy term for the cost-benefit tradeoff associated with any security decision. It's what we do when we react to fear, or try to make ourselves feel secure. It's the fight-or-flight reflex that evolved in primitive fish and remains in all vertebrates. It's instinctual, intuitive and fundamental to life, and one of the brain's primary functions.

Some have hypothesized that humans have a "risk thermostat" that tries to maintain some optimal risk level. It explains why we drive our motorcycles faster when we wear a helmet, or are more likely to take up smoking during wartime. It's our natural risk management in action.

The problem is our brains are intuitively suited to the sorts of risk management decisions endemic to living in small family groups in the East African highlands in 100,000 BC, and not to living in the New York City of 2008. We make

systematic risk management mistakes -- miscalculating the probability of rare events, reacting more to stories than data, responding to the feeling of security rather than reality, and making decisions based on irrelevant context. And that risk cockpit of ours? It's not nearly as finely tuned as we might like it to be.

Like a rabbit that responds to an oncoming car with its default predator avoidance behavior -- dart left, dart right, dart left, and at the last moment jump -- instead of just getting out of the way, our Stone Age intuition doesn't serve us well in a modern technological society. So when we in the security industry use the term "risk management," we don't want you to do it by trusting your gut. We want you to do risk management consciously and intelligently, to analyze the tradeoff and make the best decision.

This means balancing the costs and benefits of any security decision -- buying and installing a new technology, implementing a new procedure or forgoing a common precaution. It means allocating a security budget to mitigate different risks by different amounts. It means buying insurance to transfer some risks to others. It's what businesses do, all the time, about everything. IT security has its own risk management decisions, based on the threats and the technologies.

There's never just one risk, of course, and bad risk management decisions often carry an underlying tradeoff. Terrorism policy in the U.S. is based more on politics than actual security risk, but the politicians who make these decisions are concerned about the risks of not being re-elected.

Many corporate security decisions are made to mitigate the risk of lawsuits rather than address the risk of any actual security breach. And individuals make risk management decisions that consider not only the risks to the corporation, but the risks to their departments' budgets, and to their careers.

You can't completely remove emotion from risk management decisions, but the best way to keep risk management focused on the data is to formalize the methodology. That's what companies that manage risk for a living -- insurance companies, financial trading firms and arbitrageurs -- try to do. They try to replace intuition with models, and hunches with mathematics.

The problem in the security world is we often lack the data to do risk management well. Technological risks are complicated and subtle. We don't know how well our network security will keep the bad guys out, and we don't know the cost to the company if we don't keep them out. And the risks change all the time, making the calculations even harder. But this doesn't mean we shouldn't try.

You can't avoid risk management; it's fundamental to business just as to life. The question is whether you're going to try to use data or whether you're going to just react based on emotions, hunches and anecdotes.

This essay appeared as the first half of a point-counterpoint with Marcus Ranum in Information Security magazine.

The convergence of voice and video traffic are quickly transforming the growing complexity of networks at a torrid pace. IDC estimates that the skills gap in VOIP should grow to 19% by 2011.

This changing profile in the role of the network plays a key role in the skills shortage. Network enabled collaboration tools such as social networking apps and the Webex conferencing/collaboration solutions we use in our business each and every day are demanding a new set of IT skills to deliver business value.

My perspective is two-fold on this issue; the first is what I have seen in the resources we have attempted to hire! We give a very straightforward quick written/oral test to all new technical hires. This requires basic networking knowledge and some Unix commands. On average, (after filters from reputable recruiting firms, some with 5-10 years experience) less than 10% pass muster for the first filter we use in our hiring process. This is a troubling fact, which has cost us considerable time and effort to secure the right resources with competent skills. So I can say from our market assessment in a very strong technological job skills market, core Unix and networking foundation skills are slipping.

The second is that we as an IT Operations Management (ITOM) industry need to keep pushing hard to build better proactive and intuitive solutions to aggregate instrumentation from all Data Center tools, including more work around VOIP, video streaming, and collaboration so that we can ease this transition. If ITOM solutions become more proactive across the typical Cisco infrastructure that is commonly installed in the Data Center, we can free up some additional time for advanced “emerging technologies” training where existing IT workers can enhance their core skills and re-invigorate their careers. We have to do a much better job of getting our existing IT professionals trained on emerging technologies!

While there’s less that ScienceLogic can do around training, we certainly strive to do our part to enhance a day in the life of the networking engineers who use our solutions to simplify monitoring of increasingly complex networking, Wireless, VOIP, and collaboration needs.

It's basically what you would expect: Appoint a national cyber security advisor, invest in math and science education, establish standards for critical infrastructure, spend money on enforcement, establish national standards for securing personal data and data-breach disclosure, and work with industry and academia to develop a bunch of needed technologies.

I could comment on the plan, but with security the devil is always in the details -- and, of course, at this point there are few details. But since he brought up the topic -- McCain supposedly is "working on the issues" as well -- I have three pieces of policy advice for the next president, whoever he is. They're too detailed for campaign speeches or even position papers, but they're essential for improving information security in our society. Actually, they apply to national security in general. And they're things only government can do.

One, use your immense buying power to improve the security of commercial products and services. One property of technological products is that most of the cost is in the development of the product rather than the production. Think software: The first copy costs millions, but the second copy is free.

You have to secure your own government networks, military and civilian. You have to buy computers for all your government employees. Consolidate those contracts, and start putting explicit security requirements into the RFPs. You have the buying power to get your vendors to make serious security improvements in the products and services they sell to the government, and then we all benefit because they'll include those improvements in the same products and services they sell to the rest of us. We're all safer if information technology is more secure, even though the bad guys can use it, too.

Two, legislate results and not methodologies. There are a lot of areas in security where you need to pass laws, where the security externalities are such that the market fails to provide adequate security. For example, software companies who sell insecure products are exploiting an externality just as much as chemical plants that dump waste into the river. But a bad law is worse than no law. A law requiring companies to secure personal data is good; a law specifying what technologies they should use to do so is not. Mandating software liabilities for software failures is good, detailing how is not. Legislate for the results you want and implement the appropriate penalties; let the market figure out how -- that's what markets are good at.

Three, broadly invest in research. Basic research is risky; it doesn't always pay off. That's why companies have stopped funding it. Bell Labs is gone because nobody could afford it after the AT&T breakup, but the root cause was a desire for higher efficiency and short-term profitability -- not unreasonable in an unregulated business. Government research can be used to balance that by funding long-term research.

Spread those research dollars wide. Lately, most research money has been redirected through DARPA to near-term military-related projects; that's not good. Keep the earmark-happy Congress from dictating how the money is spent. Let the NSF, NIH and other funding agencies decide how to spend the money and don't try to micromanage. Give the national laboratories lots of freedom, too. Yes, some research will sound silly to a layman. But you can't predict what will be useful for what, and if funding is really peer-reviewed, the average results will be much better. Compared to corporate tax breaks and other subsidies, this is chump change.

If our research capability is to remain vibrant, we need more science and math students with decent elementary and high school preparation. The declining interest is partly from the perception that scientists don't get rich like lawyers and dentists and stockbrokers, but also because science isn't valued in a country full of creationists. One way the president can help is by trusting scientific advisers and not overruling them for political reasons.

Oh, and get rid of those post-9/11 restrictions on student visas that are causing (.pdf) so many top students to do their graduate work in Canada, Europe and Asia instead of in the United States. Those restrictions will hurt us immensely in the long run.

Those are the three big ones; the rest is in the details. And it's the details that matter. There are lots of serious issues that you're going to have to tackle: data privacy, data sharing, data mining, government eavesdropping, government databases, use of Social Security numbers as identifiers, and so on. It's not enough to get the broad policy goals right. You can have good intentions and enact a good law, and have the whole thing completely gutted by two sentences sneaked in during rulemaking by some lobbyist.

Security is both subtle and complex, and -- unfortunately -- it doesn't readily lend itself to normal legislative processes. You're used to finding consensus, but security by consensus rarely works. On the internet, security standards are much worse when they're developed by a consensus body, and much better when someone just does them. This doesn't always work -- a lot of crap security has come from companies that have "just done it" -- but nothing but mediocre standards come from consensus bodies. The point is that you won't get good security without pissing someone off: The information broker industry, the voting machine industry, the telcos. The normal legislative process makes it hard to get security right, which is why I don't have much optimism about what you can get done.

And if you're going to appoint a cyber security czar, you have to give him actual budgetary authority -- otherwise he won't be able to get anything done, either.

This essay originally appeared on Wired.com.