Hash functions are the most commonly used cryptographic primitive, and the most poorly understood. You can think of them as fingerprint functions: They take an arbitrary long data stream and return a fixed length, and effectively unique, string. The security comes from the fact that while it's easy to generate the fingerprint from a file, it's infeasible to go the other way and generate a file given a fingerprint.

Originally created to make digital signatures more efficient, hashes are now used to secure the very fundamentals of our information infrastructure: in password logins, secure web connections, encryption key management, virus and malware scanning, and almost every cryptographic protocol in current use. Without cryptographic hash functions, the internet would simply not work. At the same time, there isn't a good theory of hash functions. Unlike encryption algorithms, there are no secret keys involved; this makes it harder to mathematically define exactly what hash functions are.

The National Institute of Standards and Technology, NIST, is holding a competition to replace the SHA family of hash functions. "SHA" stands for "Secure Hash Algorithm." It was developed by the NSA in 1993 to replace the commercial MD4 and MD5 algorithms, and has been updated several times since then. All the SHA algorithms are very similar, and have been increasingly under attack, so NIST wants to replace them.

The competition is important because, unlike other technological standards, committee design — balancing the interests of diverse constituents — isn't conducive to good security. Security is best when it's designed by expert teams and then subjected to public review. And cryptography is best when it's chosen by competition.

In 1997, NIST held a competition for a block cipher to replace DES. Fifteen candidates and three-and-a-half years later, Rijndael became the new Advanced Encryption Standard — AES. NIST is doing the same thing for what it's calling SHA-3 (not, for some unexplained reason, the Advanced Hash Standard or AHS).

The deadline was October 31, and NIST received 64 submissions. This isn't surprising — I predicted 80 — as most of the 15 AES submitters were professors, whose students at the time have become professors themselves, with their own students. (If NIST does a stream cipher competition in another ten years, they should expect about 256 submissions.) These submissions came from academia, from industry, and from hobbyists. CIO magazine recently interviewed one of the submitters, who is 15. Twenty-eight submissions have been made public by the submitters, and six of those have been broken.

NIST is going through all the submissions right now, making sure they are complete and proper. Their goal is to publish all accepted submissions by the end of November, in advance of the First Hash Function Candidate Conference, to be held in Belgium right after the Fast Software Encryption workshop in February.

The group expects to quickly make a first cut of algorithms — hopefully to about a dozen — and give the community a year of cryptanalysis before making a second cut in 2010. After another year of cryptanalysis, NIST will choose a winner in 2011. Expect a final standard by 2012.

My advice for software developers is to let the process run its course. While it's tempting to use the new cool algorithms in your designs, it's far too soon to trust any of them. This process is likely to result in all sorts of new research results in hash function security, and some real cryptanalytic surprises. Give the community a few years to figure out which ones are good and which aren't.

I've previously called this sort of thing a cryptographic demolition derby: The last one left standing wins. But that's only partially true. Certainly all the groups will spend the next few years trying to cryptanalyze each other, but in the end there will be a bunch of unbroken algorithms. NIST will select one based on performance and features.

NIST has stated that the goal of this process is not to choose the best standard but to choose a good standard. I think that's smart; in this process, the best is the enemy of the good. While there's no rush to choose a new standard — the SHA-2 algorithms will remain secure for the foreseeable future — we don't want to analyze the candidates forever.

Personally, I was part of a group of eight cryptographers that submitted Skein to the competition. A decade ago, writing Twofish and participating in the AES process was the most fun I had ever had in cryptography. These next few years promise to be even more fun.

---

Bruce Schneier is chief security technology officer of BT. His new book is Schneier on Security.

The panelists were:
William P. Crowell – former Deputy Director of the National Security Agency
Michael P. Jackson – Deputy Secretary, Department of Homeland Security
Jose A. Rodriguez, Jr. – former Director CIA, National Clandestine Service & CIA, DCI Counterterrorist Center

Overall, it was a very nicely arranged event on a brisk fall evening with about 100 CXO attendees; mostly large but some small government contractors and a few product companies like ScienceLogic that conduct business with military, intelligence and the public sector.

No surprise, given the financial crisis the economy is suffering from that the panelists said we also have a crisis coming on the Federal budget front. This will put enormous pressure on the way Administration thinks, and how and where to spend the $$.

Obama’s tone regarding the issues he will be confronting in the world during the election was encouraging. Make the world more non-partisan and take on the threats that we have in front of us head-on!

The panel was very upfront about current threats. William Crowell said,

“It is highly imprudent to believe that there will not be another 9-11. We have to fund and support the work to stop other attacks. We can only mitigate risk but we can’t eliminate risk. We have to try to absorb the sense of urgency and wake up every day looking at the intelligence screens as if 9-11 happened within the last couple of months.”

He added,

“They (the intelligence community) need the innovation, sense of commitment and urgency that comes from the private sector – a sense of mutual commitment to that mission.”

Predicted Priorities for investment for DHS:

1. Cyber attack as the top issue
2. Nuclear threats including dirty bomb
3. Chemical and biological attacks
4. Explosive attacks against critical infrastructure with maximum # of lives and or financial disruption / loss.
5. Large scale natural disasters – hurricane + earthquakes
6. Border penetration - identity management and border management issues

An Obama administration will spend dollars around these threat vectors. They will want to spend $$ to help state and local governments. Grants to state and local governments should significantly increase with the Obama administration, so think about how you will increase your focus on the state and local government spending initiatives.

Secure border investments – the panelists believe that the new administration will feel compelled to invest here. Michael P. Jackson bluntly said, “You have to make investments in border tools to get meaningful immigration reform.”

Panelists agreed that the 1^st year will be an intense period of scrutiny about fundamental directions. We can’t afford it all at DHS; it is dramatically under budgeted. At TSA/DOT and then at DHS, we spent about $4 Billion on technology investments since 9-11; those investments are now reaching the end of the original service life.

One gripe from the panel that I found humorous: “We don’t have a group of people who think like entrepreneurs.” It is insane how long things last when you buy things in the government. As an example, we are still replacing vacuum tubes in some of the very old FAA gear… this is well beyond what any reasonable person would think these initial investments should/would last.

Final Thoughts:
I actually think that the Obama Administration will be quite favorable to COTS software products, SaaS offerings, and creative financing initiatives from the private sector. The government just won’t have the capital budget to do everything it wants to accomplish. I would say if you look at how intelligently and aggressively Obama used technology to assist his campaign, the odds are good that this new breed of IT talent (which is already really comfortable with SaaS products, blogs, wiki’s, hosted/outsourced Cloud solutions… this team really understands the latest technology trends) will quickly work to bring these new IT paradigms to the Federal marketplace. Clearly the private sector can help the Government achieve more with lower capital budgets – beginning to provide services rather than transaction-based selling. Another clear idea is to think about leasing as a better way to work with the government which going forward will have increased budgets restrictions.

They will likely be in confrontation with members of Congress that won’t change fast enough, however the future of our nation’s ability to fight terror lies in becoming more efficient and effective. It requires the government be flexible enough to figure out what jobs and IT functions to outsource in a nimble and smart way. My prediction: this is great news for Service Providers. Overall the next 4 years should be great for our business as well as the Managed Service Provider/SaaS industry!

After Interop Las Vegas you may have seen our analysis of the data that we collected and delivered in our NOC view.  I thought I’d recreate the same data for NY and do a short comparison.

1) Like in Vegas, uptime for the network 100%.  This is no small feat considering that we introduced a new wrinkle in NY, taking down the primary NOC while the education portion of the show was still going on.  This was a forced failover to the backup systems, and it went flawlessly.  I’d like to give a little credit to EM7 on the 100% uptime as it caught a failover to battery power that allowed AC to be restored before a series of critical equipment would have gone down.

2) Again like Vegas, the average monitored device in the show network didn’t even hit 10% CPU utilization.  Still lots of computing overhead availabe in the show network.

3) The NY show network wasn’t nearly as busy as in Las Vegas, sustaining an average of only 27Mbps of usage (versus 56 Mbps) in Vegas.

4) Power consumption for the network and NOC in NY clocked in at 445kwh per day, about 25% less than the Las Vegas show.  This wasn’t because the equipment was any more power efficient, but instead because the show was smaller and therefore there was less network gear.

5) Finally, a stat we didn’t track too carefully in Las Vegas, but that I find interesting.  During show hours the wireless network average 1,100 users attached.  That’s a lot of people and a lot of wireless devices.

The good news is there was nothing too unexpected in the data, overall the smaller show led to a smaller number of tickets and smaller consumption of resources across the board.  We hope to have the opportunity to work with the InteropNet team again next year and take a look at this data year-over-year for each show.

The problem is that (Chris’ analysis is) completely impractical. I’ll take a recent, and fairly typical situation as an example. I was taking issue with the manner in which remote access was being provisioned for a third party vendor to connect to a system hosted by one of our European business units. To cut a long story short, it was not only a breach of policy but highly insecure. I wanted the access to be disconnected, the business unit director wanted my risk assessment. And he didn’t want to wait for it.

To quote Chris Hayes, spending time on working out the expected effectiveness of controls, over a given timeframe, as measured against a baseline level of force was not going to pacify an angry Italian fearful that my decision was going to cost him money. He wanted my explanation of the risk and more importantly, what I was going to offer as a solution to keep his business functioning

As Chris is someone who actually does this for a living in a large company, and this is typical of his actual day job, I really find Stuart’s “impractical” comment to be, um, misinformed.

Also, I think Stuart mistakes the purpose of a risk analysis.  The purpose of the risk analysis is not to force someone to be “secure”, but to provide knowledge for decision making.  Using it as a “hammer” to knock in the nail of your personal risk tolerance impairs efficiency and in the long run retards “security” as it creates political resentment.  Seriously, who cares if something might violate policy or not in a pre-implementation discussion?  Policies are not stone tablets handed down from on high, they are state-in-time codification of the data owners risk tolerance.  This risk tolerance changes sometimes, and that’s OK.

To that extent, I appreciate (and I’m sure Chris does, as well) that risk analysis does not create rationality in the data owner.  Someone who sees you as a speedbump on the route to progress they may not be ready to appreciate your point of view even if it is stated in the most rational manner possible.   But it’s worth noting (and Stuart’s example is indicative of this point) that risk analysis does not create rationality in the analyst, either.  If one is being so “security minded” as to ignore the risk tolerance of the business owner - we’re bound to get a reaction similar to that Stuart encountered.  In fact, a practical risk analysis like Chris performed on his blog, done in 30 minutes, should identify the critical point of disagreement between Stuart and the data owner (again, Stuart doesn’t own the data, the agitated Italian does).

But let’s stay rational and open to alternatives to what Chris offers.  Stuart states his approach to risk analysis as:

When I need to document a risk assessment I use a very simple form: list the threats, state the level of vulnerability, list the associated operational costs and potential revenue hits. High, medium, or low risk? Describe the controls and options. Write up who needs to do what, and how much of their time it’s going to take. Job done.

At first glance, I don’t think what Chris has done is any less efficient, and it provides greater insight (using Frequency and Capability instead of just ‘listing the threats’).  But what is key here is that Chris’ approach is consistent and defensible.  Less generous risk geeks and CSO’s I know would have no little difficulty with Stuart’s approach.  But to particularly answer Stuart’s main objection (impracticality) I would offer that with practice, Chris’ work is probably quicker and easier than Stuart’s described process as it eliminates much of the ambiguity an immature risk analysis creates - reducing the need for further discussion and arguments with data owners (regardless of disposition or nationality).

Finally the irony of Stuart’s post is that the reason he had this confrontation may in fact be because he was incapable of bringing a salient model for risk to the table, one that identified the factors that create risk and developed a defensible belief statement concerning risk.   We’ll never know if one would have helped him in this isolated instance, but I can tell you that in organizations like Chris’, good risk models and strong risk anlayses create operational efficiencies, reduce costs, and streamlines intra-departmental communications.

After meetings and calls, I was finally able to slip into a conference session – just in time to catch uber-smart analysts Rachel Chalmers (The 451 Group) and Dan Golding (Tier1 Research) engage in a lively and not-so-mock debate on “Hosting Meets the Cloud”.

Now this doesn’t cover the entire debate – and part II is coming tomorrow. But what it does cover is the most interesting questions (to me) and paraphrase the points made by the analysts. I thought they both had very interesting points and more similarities than differences in the end; the real difference is how they thought about the issues and through what lens – for Rachel it was the enterprise and for Dan it was managed hosting providers. (image from inmagine)

Question: What is a cloud and why?

Dan: Shared infrastructure leveraged/run by third parties for the benefit of enterprises, developers, etc. This is not a new idea – just recently “rebranded.” Given all the discussion and disagreement over this now, what will the cloud end up looking like?

Rachel: The cloud is “IT infrastructure as a service” down to the level of a server operating system. Take the example of Amazon web services – in this case it’s not just the infrastructure but also the internal processes built around service delivery, e.g., provisioning, that are being exposed as a commodity to external customers.

Dan’s Question for Rachel: In your opinion, how much is the cloud a fad versus CIOs really trying to solve a problem?

Rachel: For the practical, roll-up-your-sleeves types of CIOs – those coming up from the engineering ranks – that I talk to, the cloud is real, as opposed to SOA and middleware.

What about “internal” cloud computing – built and maintained by an enterprise versus a third-party provider?

Dan: Cloud computing is done by providers for customers. Certainly there are enterprises that have made internal computing investments, e.g., for publishing, large-scale phone systems, etc - but they were stupid ideas made by companies that have too much money. A better question here is does it make any sense for an enterprise to create their own cloud? While an enterprise can play at it, they can’t do it cost-effectively, not in a way that a third party provider can do it.

Rachel: Many CIOs have “managed-hoster” envy – for things like chargeback and billing that hosters understand a do better. Of course there has been a rise in automation and virtualization tools in the enterprise which may not be as efficient and built for scalability as a hoster can achieve, but what is important is that they are customized/specialized for that business.

Dan: Can you give a specific example of optimization to make it worthwhile for enterprises to do it themselves?

Rachel: One example is sovereignty. The privacy laws around financial and healthcare information are not the same everywhere. Clouds and their geographically-dispersed data centers don’t necessarily have “national” borders. This is definitely a concern for the CIO that has to comply with regulations in their industry around privacy protection, for instance. Another example is security. Dow Chemical does a lot of work via joint ventures and has a need to provide but lock down desktops given to contractors as corporate workspaces. For their level of security, they need to “own” their computing resources.

Dan: But why can’t someone like SunGard provide that as they do for many other large companies?

Rachel: It comes down to a question of trust.

Do people trust their hosting providers?

Dan: Yes. Whether it’s for a content delivery network or collocation, hosting the customers of hosting providers are some of the largest companies in the world in industries like energy and financial services. Give me a case when there was a major security issue with a hosting company. In fact, managed hosting providers usually provide better security than enterprises are capable of.

And a question provided by an attendee from EMC: A few years ago, this would have been a grid discussion. How is the cloud different?

Rachel: Grid computing ended up being applicable only for niches – which I predicted. The real opportunity for everyone else with the cloud only comes up when you combine the kinds of automation tools (originally developed for grid computing) with x86 virtualization.

Dan: I agree. Grid was a niche play. There were very few orgs that needed it and that the economics worked for. There were very few enterprises for whom it made sense to build their own for. The cloud is shared/leveraged versus grid computing. It economically makes sense in a way grid never did.

Of course, virtualization can do its bit to help companies save on hardware, power and cooling costs. We have this straight from a customer’s mouth – see the video here. And having your data center be 100% wind-powered certainly does its bit as well.

But the winner – according to eWeek and and an EMA study in September – is to turn equipment off when they’re not in use. I would say this is a no-brainer, but apparently we all need to be reminded…

Who remembers that line from the Schoolhouse Rock video on Energy? “So don’t get cross, when momma says turn that extra light off.” The full video for your viewing pleasure here.

(image from freedomroadproject)

Skein is our submission (myself and seven others: Niels Ferguson, Stefan Lucks, Doug Whiting, Mihir Bellare, Tadayoshi Kohno, Jon Callas, and Jesse Walker). Here's the paper:

Executive Summary

Skein is a new family of cryptographic hash functions. Its design combines speed, security, simplicity, and a great deal of flexibility in a modular package that is easy to analyze.

Skein is fast. Skein-512 -- our primary proposal -- hashes data at 6.1 clock cycles per byte on a 64-bit CPU. This means that on a 3.1 GHz x64 Core 2 Duo CPU, Skein hashes data at 500 MBytes/second per core -- almost twice as fast as SHA-512 and three times faster than SHA-256. An optional hash-tree mode speeds up parallelizable implementations even more. Skein is fast for short messages, too; Skein-512 hashes short messages in about 1000 clock cycles.

Skein is secure. Its conservative design is based on the Threefish block cipher. Our current best attack on Threefish-512 is on 25 of 72 rounds, for a safety factor of 2.9. For comparison, at a similar stage in the standardization process, the AES encryption algorithm had an attack on 6 of 10 rounds, for a safety factor of only 1.7. Additionally, Skein has a number of provably secure properties, greatly increasing confidence in the algorithm.

Skein is simple. Using only three primitive operations, the Skein compression function can be easily understood and remembered. The rest of the algorithm is a straightforward iteration of this function.

Skein is flexible. Skein is defined for three different internal state sizes -- 256 bits, 512 bits, and 1024 bits -- and any output size. This allows Skein to be a drop-in replacement for the entire SHA family of hash functions. A completely optional and extendable argument system makes Skein an efficient tool to use for a very large number of functions: a PRNG, a stream cipher, a key derivation function, authentication without the overhead of HMAC, and a personalization capability. All these features can be implemented with very low overhead. Together with the Threefish large-block cipher at Skein core, this design provides a full set of symmetric cryptographic primitives suitable for most modern applications.

Skein is efficient on a variety of platforms, both hardware and software. Skein-512 can be implemented in about 200 bytes of state. Small devices, such as 8-bit smart cards, can implement Skein-256 using about 100 bytes of memory. Larger devices can implement the larger versions of Skein to achieve faster speeds.

Skein was designed by a team of highly experienced cryptographic experts from academia and industry, with expertise in cryptography, security analysis, software, chip design, and implementation of real-world cryptographic systems. This breadth of knowledge allowed them to create a balanced design that works well in all environments.

Here's source code, text vectors, and the like for Skein. Watch the Skein website for any updates -- new code, new results, new implementations, the proofs.

NIST's deadline is Friday. It seems as if everyone -- including many amateurs -- is working on a hash function, and I predict that NIST will receive at least 80 submissions. (Compare this to the 21 submissions NIST received -- five were rejected as not being complete -- for the AES competition in 1998.) I expect people to start posting their submissions over the weekend. (Ron Rivest already presented MD6 at Crypto in August.) Probably the best place to watch for new hash functions is here; I'll try to keep a listing of the submissions myself.

The selection process will take around four years. I've previously called this sort of thing a cryptographic demolition derby -- last one left standing wins -- but that's only half true. Certainly all the groups will spend the next couple of years trying to cryptanalyze each other, but in the end there will be a bunch of unbroken algorithms; NIST will select one based on performance and features.

NIST has stated that the goal of this process is not to choose the best standard but to choose a good standard. I think that's smart of them; in this process, "best" is the enemy of "good." My advice is this: immediately sort them based on performance and features. Ask the cryptographic community to focus its attention on the top dozen, rather than spread its attention across all 80 -- although I also expect that most of the amateur submissions will be rejected by NIST for not being "complete and proper." Otherwise, people will break the easy ones and the better ones will go unanalyzed.

Setting the context for this post, it is a tough economic situation all over the world. IT spending has reduced and will reduce significantly. In one of earlier posts, I had referred to information security as an overhead of an overhead (IT). What is a good approach for security practice in this type of economy?

I don't have a magic wand to pull a rabbit out of a hat. I have always been told that: tough economy is the time for real smart people to make money. Coming back to information security topic, with a bit of common sense, it is wise for information security professionals to offer services in those areas that does not involve capital expenditure. As a Security Manager, you may be already aware that your people are willing to go an extra mile in the current economic times.

- No budget or lack of budget, means no new capital expenditure. Spend time wisely in building a future technology strategy and keep it in the back pocket when the economy turns around.

- This is a good time to create roles/responsibilities and ownership for various areas. Create operating procedures. Make your team to automate tasks. This will help your operations become more efficient.

- This is time for security awareness  education. Create pamphlets/brochures/presentations for an online or classroom training. Engage your and your team's time to impart training.

- Leverage already invested technology platforms. Leverage utilized features that reduce costs. If you have already invested in technology such as VMware, this is the time to get the best out of it. You can use VMware's toolkit to build your lab and staging environment and optimize on hardware cost.

- Off shoring has been the mantra of senior executives, this is the time to revisit those services and measure their performance closely and assess your satisfaction level. This is a good time to build a case for not off shoring if it makes sense.

- Companies are more vulnerable in bad economic times. You are in a better position to influence senior management about information security risks under these circumstances and drive home the value of protecting your intellectual property under these kinds of circumstances. management will be all ears for such a pitch.

- Time to engage your architect to optimize your security architecture, revisit standards and optimize design for cost efficiency.

- Revisit various controls and see if there are some risks that you could optimize spending on.

- Training budget is an unfortunate victim of this type of economy. Encourage employees to take free webinars offered by various security vendors and encourage them to share the summary across the team. This will put your employees in touch with latest happenings in security at the same time there is some learning that is imparted despite zero training budget.

- Since there are very few projects in action, this is a good time to have conversations with cross functional teams and educate them about your services and solicit feedback on how to do better.

- Revisit your vendor logistics and identify whether you can renegotiate some of your already existing contracts.

The above are some good ways by which you can optimize costs, this will also enhance your team's competence level in the long run. And this approach is better than letting people go, if you can pull this.