The report was written by a committee whose members include William Perry, a professor at Stanford University; Charles Vest, the former president of MIT; W. Earl Boebert, a retired senior scientist at Sandia National Laboratories; Cynthia Dwork of Microsoft Research; R. Gil Kerlikowske, Seattle's police chief; and Daryl Pregibon, a research scientist at Google.

They admit that far more Americans live their lives online, using everything from VoIP phones to Facebook to RFID tags in automobiles, than a decade ago, and the databases created by those activities are tempting targets for federal agencies. And they draw a distinction between subject-based data mining (starting with one individual and looking for connections) compared with pattern-based data mining (looking for anomalous activities that could show illegal activities).

But the authors conclude the type of data mining that government bureaucrats would like to do--perhaps inspired by watching too many episodes of the Fox series 24--can't work. "If it were possible to automatically find the digital tracks of terrorists and automatically monitor only the communications of terrorists, public policy choices in this domain would be much simpler. But it is not possible to do so."

A summary of the recommendations:

• U.S. government agencies should be required to follow a systematic process to evaluate the effectiveness, lawfulness, and consistency with U.S. values of every information-based program, whether classified or unclassified, for detecting and countering terrorists before it can be deployed, and periodically thereafter.

• Periodically after a program has been operationally deployed, and in particular before a program enters a new phase in its life cycle, policy makers should (carefully review) the program before allowing it to continue operations or to proceed to the next phase.

• To protect the privacy of innocent people, the research and development of any information-based counterterrorism program should be conducted with synthetic population data... At all stages of a phased deployment, data about individuals should be rigorously subjected to the full safeguards of the framework.

• Any information-based counterterrorism program of the U.S. government should be subjected to robust, independent oversight of the operations of that program, a part of which would entail a practice of using the same data mining technologies to "mine the miners and track the trackers."

• Counterterrorism programs should provide meaningful redress to any individuals inappropriately harmed by their operation.

• The U.S. government should periodically review the nation's laws, policies, and procedures that protect individuals' private information for relevance and effectiveness in light of changing technologies and circumstances. In particular, Congress should re-examine existing law to consider how privacy should be protected in the context of information-based programs (e.g., data mining) for counterterrorism.

Here are more news articles on the report. I explained why data mining wouldn't find terrorists back in 2005.

I’m assuming the author wants us to read the title as “Things to Look Out For in Performing Risk Analysis” and not “Risk Management is Folly - Stop, Stop, Stop!” The former is fine, the latter isn’t supported by the evidence presented by the subjects of the article.
The subjects of the article are a good study from Wade Baker & Co. at Verizon, and a report from RSA’s Security for Business Innovation Council. Let’s take a look at each of these and examine why what they’re saying might contribute to poor risk management, shall we?

1.)  THE VERIZON REPORT

The Verizon report is an analysis of some 530 forensic investigations their company performed.  It is well worth your time as it’s chock full of interesting information.  As it relates to the Dark Reading piece, a coarse summary would be that “likelihood” is “different” for different people and so you can’t use the same “likelihood” across different industries.

Distilled through the lens of FAIR:

“different threat communities may be applicable based on Probability of Action factors which include: Value, Level of Effort and Risk (of Getting Caught).”

Or, even further distilled and in the words of my six year old son,

“Duh-uh”.

With regards to what I assume is the purpose of the article (What Doesn’t Work in Risk Analysis) this concept  seems just to rehash the old GIGO argument regarding risk analysis.  Great.  Can’t argue with that, nor it’s corollary QIQO (quality in, quality out).

But let me ask you -  is this really a problem common in your analysis?  Did reading this article make you go “Crap, we’ve been using data normalized across multiple industries in our analysis! They’re all wrong!”  Or have you already been accounting for the unique value proposition your company has to the specific threat community you’re worried about?  See, maybe I’m just not your average analyst, but even in my NIST/OCTAVE days, this has *never* been an issue for me.

Let me be specific, this is not a problem with Verizon’s very cool report.  It’s just that I don’t see what the big deal is.  This article is starting to feel like someone is running through the motions, trying to play the ” a crazy title gets people to read a boring article” game.

Speaking of cool reports - You know what would be cool?  I think it would be interesting to see is the quality of these companies’ “risk management process” established using good criteria,  and then correlated to the frequency and magnitude of real-world losses across the aggregate sample.  In other words, can we establish evidence that strong risk management practices not just reduce “risk” but also reduce actual incidents.

2.)  THE RSA COUNCIL “EXPLORES WHY LEGACY METHODS OF EVALUATING INFORMATION SECURITY RISK DON’T WORK IN TODAY’S CONNECTED WORLD, IN WHICH ANY NEW BUSINESS INNOVATION INHERENTLY CARRIES SOME LEVEL OF RISK TO INFORMATION.”

This report from the RSA council puts forth a seemingly obvious proposition, that risk must be balanced by reward.  Why is this news?  Now as I read the article it’s not clear if:

• The RSA Council is claiming that the CISO’s office should be the ones determining reward.  Absurd.

or

• Businesses aren’t doing a good job at determining risk and reward.

Let’s go with the latter.  So I’m pretty sure (good) businesses do a good job at estimating reward.  Businesses I’ve been a part of?  We LOVE(D) estimating reward.  We don’t tend to start projects all willy-nilly. No we tend to be careful to identify the size of the market and what it will cost to address the market.  So what could the problem be that this RSA council is trying to address?  Maybe it has to do with something like the following:

Yesterday, I got a demo of an IT-GRC application that shall remain nameless.  It seemed to be very good at the “C” bits - lots of information on regulations and expectations and even what sorts of controls would answer the regulations (which is goofy, but we’ll have to talk about that later).  It also gave you the ability to build workflow quite nicely.  But it measured NOTHING.  There really was no observable “G” and “R” was really Medium X Low X Low = High sorts of stuff.  So let’s use this relatively expensive tool as evidence of what your average CISO is armed with going into a Risk/Reward sort of meeting.  I imagine a nice board room with wood-grain paneling and glass bowls filled with little chocolate covered mints designed to give everyone involved in the meeting (CEO, CFO, CIO, CSO, VP S&M, etc…) a little sugar rush when needed and fresh breath.  The conversation goes a little something like this (apologies to Rich):

Business Guy Who Wants to Make Money Because That’s What Businesses Do: Based on market studies, we believe that initial gross revenues from the new product and technology rollout will be eleventy gazillion dollars based on a 37% market penetration in Scandinavia, alone.

CSO: Well now, we have a likelihood of “High” and a “C” impact of Medium, and an “I” impact of Low, and an “A” impact of “High” and because we are a (bank/hospital/retailer/basically any business that breathes anymore) we weight “C” by a factor of 2 - we multiplied those all together and got a “High”.

So can you guys delay the product rollout by 9 months and give me a bunch more money that’s not in the budget so that I can get this thing down to a “Medium”, please?

Again, I just don’t see the problem with Information Risk Management being that our businesses have no idea what the rewards of business might be.  Now maybe we need get a seat in that boardroom just to be able to talk about our “Mediums”, sure.  And maybe we’re infantile in our ability to describe our problem space.  But I cannot fathom that “Risk Management Doesn’t Work” because businesses haven’t been considering “reward”.

WHY RISK MANAGEMENT MAY  NOT BE WORKIN’ FOR YOU

Two meta-categories of causation:

• No skills

and/or

• No resources

Any ancillary “cause” can be mapped to one of these categories.  You could have significant resources but crappy models, and have conversations like our imaginary CSO, above.  You could have really good models and people trained and motivated to use them, but scarce time & money, so no conversation happens.

Now my question for you is - which does it make sense to acquire *first* to solve the “Why Risk Management Doesn’t Work” problems, skills or resources?

Interop NY 2008 was up considerably in size from the show in 2007.  According to Lenny Heymann, the GM of Interop, this is a trend that they expect to continue.  My personal experience was that the size of the vendors was also up this year.  I think there were so few startups that “Startup City” was pulled from the show completely.  In any case, the show floor was full and there was plenty of attendee traffic to go around.

Definitely helping out from a traffic and draw perspective was the addition of the Web 2.0 Expo - Interop was co-located with both Mobile Business Expo and the Web 2.0 show. It seems like that buzzword still hasn’t lost most of its luster.

From the InteropNet perspective, the main feeling was one of being rushed.  With the show only lasting two days, and the InteropNet team only having a couple of days of ramp up time, everything was compressed into a much shorter period than in Las Vegas.  While this would normally be a challenge, it’s an even bigger challenge at the Javits where the InteropNet team was allowed to do almost nothing ourselves because of union rules.  You’d be surprised how frustrated you can make a network guy who’s told that he has to stand there and watch the electrician plug things in, rather than just doing it himself.  The only thing faster than the InteropNet team getting the Interop NY network up, was my pedicab ride to the InteropNet Booze Cruise.

In any case, everything came off without a hitch, and EM7 performed flawlessly catching a couple of power outages that last day and alerting everyone before the batteries on the UPSes had a chance to run down.

Over the next couple of weeks I’ll analyze the data from the show to see how many tickets were handled, amount of bandwidth consumed, etc and we’ll do a comparison to Interop Las Vegas.

We’re (both ScienceLogic and me personally) looking forward to Interop 2009.

So, in conjunction with the BlueHat team, I am pleased to announce that the SDL team will be organizing the sessions for the second day of the fall BlueHat conference. The BlueHat SDL sessions will be laser-focused on not just describing vulnerabilities but also solving them. Every attendee should leave every presentation with a clear idea of exactly what he or she needs to do to protect themselves from the threat that was discussed during the session.

The sessions will begin, appropriately, with the topic of secure design. Danny Dhillon of EMC and the SDL team’s own Adam Shostack will each present their organization’s approach to threat modeling. As a bonus, Adam will also be demonstrating the new SDL Threat Modeling tool that you might have heard about last week.

Next up is Matt Miller, a recent and very welcome addition to the Microsoft Security Science team. Matt has a fantastic presentation on the evolution of buffer overflow attacks and on the corresponding development of overflow mitigations. From there we will switch gears to look at some managed code implementation issues: iSEC Partners’ Scott Stender and Alex Vidergar will demonstrate coding techniques to mitigate elusive concurrency vulnerabilities in web applications.

At this point we will have covered the Design and Implementation phases of the SDL; where better to go from here than Verification? One of the most important activities in the Verification phase is fuzzing, and we have a trio of security experts from the Microsoft Security Science team to talk about it. Jason Shirk, Lars Opstad, and Dave Weinstein will answer three of the most common fuzzing questions: How should I fuzz? When have I fuzzed enough? And what do I do now that I’ve fuzzed?

Finally, we will wrap up the Verification phase talks with a return appearance to BlueHat by Stach & Liu’s Vinnie Liu. Vinnie will compare different approaches to security verification – static code analysis, blackbox analysis, and manual code review – and make recommendations as to when each approach is best used.

Even if you can’t make it in to BlueHat in person, you can still watch the sessions via streaming media on TechNet. Additionally, webcast interviews with the speakers – condensed “Cliff’s Notes” versions of their full presentations – will be posted on Channel 9. And we’ll be continuing the BlueHat tradition of inviting speakers and other industry notables to guest blog about their topics and the latest security trends. More information on all of these resources will be posted here when it becomes available.