The report was written by a committee whose members include William Perry, a professor at Stanford University; Charles Vest, the former president of MIT; W. Earl Boebert, a retired senior scientist at Sandia National Laboratories; Cynthia Dwork of Microsoft Research; R. Gil Kerlikowske, Seattle's police chief; and Daryl Pregibon, a research scientist at Google.

They admit that far more Americans live their lives online, using everything from VoIP phones to Facebook to RFID tags in automobiles, than a decade ago, and the databases created by those activities are tempting targets for federal agencies. And they draw a distinction between subject-based data mining (starting with one individual and looking for connections) compared with pattern-based data mining (looking for anomalous activities that could show illegal activities).

But the authors conclude the type of data mining that government bureaucrats would like to do--perhaps inspired by watching too many episodes of the Fox series 24--can't work. "If it were possible to automatically find the digital tracks of terrorists and automatically monitor only the communications of terrorists, public policy choices in this domain would be much simpler. But it is not possible to do so."

A summary of the recommendations:

• U.S. government agencies should be required to follow a systematic process to evaluate the effectiveness, lawfulness, and consistency with U.S. values of every information-based program, whether classified or unclassified, for detecting and countering terrorists before it can be deployed, and periodically thereafter.

• Periodically after a program has been operationally deployed, and in particular before a program enters a new phase in its life cycle, policy makers should (carefully review) the program before allowing it to continue operations or to proceed to the next phase.

• To protect the privacy of innocent people, the research and development of any information-based counterterrorism program should be conducted with synthetic population data... At all stages of a phased deployment, data about individuals should be rigorously subjected to the full safeguards of the framework.

• Any information-based counterterrorism program of the U.S. government should be subjected to robust, independent oversight of the operations of that program, a part of which would entail a practice of using the same data mining technologies to "mine the miners and track the trackers."

• Counterterrorism programs should provide meaningful redress to any individuals inappropriately harmed by their operation.

• The U.S. government should periodically review the nation's laws, policies, and procedures that protect individuals' private information for relevance and effectiveness in light of changing technologies and circumstances. In particular, Congress should re-examine existing law to consider how privacy should be protected in the context of information-based programs (e.g., data mining) for counterterrorism.

Here are more news articles on the report. I explained why data mining wouldn't find terrorists back in 2005.

I’m assuming the author wants us to read the title as “Things to Look Out For in Performing Risk Analysis” and not “Risk Management is Folly - Stop, Stop, Stop!” The former is fine, the latter isn’t supported by the evidence presented by the subjects of the article.
The subjects of the article are a good study from Wade Baker & Co. at Verizon, and a report from RSA’s Security for Business Innovation Council. Let’s take a look at each of these and examine why what they’re saying might contribute to poor risk management, shall we?

1.)  THE VERIZON REPORT

The Verizon report is an analysis of some 530 forensic investigations their company performed.  It is well worth your time as it’s chock full of interesting information.  As it relates to the Dark Reading piece, a coarse summary would be that “likelihood” is “different” for different people and so you can’t use the same “likelihood” across different industries.

Distilled through the lens of FAIR:

“different threat communities may be applicable based on Probability of Action factors which include: Value, Level of Effort and Risk (of Getting Caught).”

Or, even further distilled and in the words of my six year old son,

“Duh-uh”.

With regards to what I assume is the purpose of the article (What Doesn’t Work in Risk Analysis) this concept  seems just to rehash the old GIGO argument regarding risk analysis.  Great.  Can’t argue with that, nor it’s corollary QIQO (quality in, quality out).

But let me ask you -  is this really a problem common in your analysis?  Did reading this article make you go “Crap, we’ve been using data normalized across multiple industries in our analysis! They’re all wrong!”  Or have you already been accounting for the unique value proposition your company has to the specific threat community you’re worried about?  See, maybe I’m just not your average analyst, but even in my NIST/OCTAVE days, this has *never* been an issue for me.

Let me be specific, this is not a problem with Verizon’s very cool report.  It’s just that I don’t see what the big deal is.  This article is starting to feel like someone is running through the motions, trying to play the ” a crazy title gets people to read a boring article” game.

Speaking of cool reports - You know what would be cool?  I think it would be interesting to see is the quality of these companies’ “risk management process” established using good criteria,  and then correlated to the frequency and magnitude of real-world losses across the aggregate sample.  In other words, can we establish evidence that strong risk management practices not just reduce “risk” but also reduce actual incidents.

2.)  THE RSA COUNCIL “EXPLORES WHY LEGACY METHODS OF EVALUATING INFORMATION SECURITY RISK DON’T WORK IN TODAY’S CONNECTED WORLD, IN WHICH ANY NEW BUSINESS INNOVATION INHERENTLY CARRIES SOME LEVEL OF RISK TO INFORMATION.”

This report from the RSA council puts forth a seemingly obvious proposition, that risk must be balanced by reward.  Why is this news?  Now as I read the article it’s not clear if:

• The RSA Council is claiming that the CISO’s office should be the ones determining reward.  Absurd.

or

• Businesses aren’t doing a good job at determining risk and reward.

Let’s go with the latter.  So I’m pretty sure (good) businesses do a good job at estimating reward.  Businesses I’ve been a part of?  We LOVE(D) estimating reward.  We don’t tend to start projects all willy-nilly. No we tend to be careful to identify the size of the market and what it will cost to address the market.  So what could the problem be that this RSA council is trying to address?  Maybe it has to do with something like the following:

Yesterday, I got a demo of an IT-GRC application that shall remain nameless.  It seemed to be very good at the “C” bits - lots of information on regulations and expectations and even what sorts of controls would answer the regulations (which is goofy, but we’ll have to talk about that later).  It also gave you the ability to build workflow quite nicely.  But it measured NOTHING.  There really was no observable “G” and “R” was really Medium X Low X Low = High sorts of stuff.  So let’s use this relatively expensive tool as evidence of what your average CISO is armed with going into a Risk/Reward sort of meeting.  I imagine a nice board room with wood-grain paneling and glass bowls filled with little chocolate covered mints designed to give everyone involved in the meeting (CEO, CFO, CIO, CSO, VP S&M, etc…) a little sugar rush when needed and fresh breath.  The conversation goes a little something like this (apologies to Rich):

Business Guy Who Wants to Make Money Because That’s What Businesses Do: Based on market studies, we believe that initial gross revenues from the new product and technology rollout will be eleventy gazillion dollars based on a 37% market penetration in Scandinavia, alone.

CSO: Well now, we have a likelihood of “High” and a “C” impact of Medium, and an “I” impact of Low, and an “A” impact of “High” and because we are a (bank/hospital/retailer/basically any business that breathes anymore) we weight “C” by a factor of 2 - we multiplied those all together and got a “High”.

So can you guys delay the product rollout by 9 months and give me a bunch more money that’s not in the budget so that I can get this thing down to a “Medium”, please?

Again, I just don’t see the problem with Information Risk Management being that our businesses have no idea what the rewards of business might be.  Now maybe we need get a seat in that boardroom just to be able to talk about our “Mediums”, sure.  And maybe we’re infantile in our ability to describe our problem space.  But I cannot fathom that “Risk Management Doesn’t Work” because businesses haven’t been considering “reward”.

WHY RISK MANAGEMENT MAY  NOT BE WORKIN’ FOR YOU

Two meta-categories of causation:

• No skills

and/or

• No resources

Any ancillary “cause” can be mapped to one of these categories.  You could have significant resources but crappy models, and have conversations like our imaginary CSO, above.  You could have really good models and people trained and motivated to use them, but scarce time & money, so no conversation happens.

Now my question for you is - which does it make sense to acquire *first* to solve the “Why Risk Management Doesn’t Work” problems, skills or resources?

Developers have the objective of building a functional application. They are focused on building more functionality into applications. Moreover, building security creates more workload  for Developers which is a disincentive and moreover, Developers are rewarded for building more functionality than building more security. I have never seen a Developer in my professional life for being rewarded for building a secure application.

Hackers are focused on how to break the application. They look for weak links in application that will enable them to access application data. Developers usually follow process to build application, but Hackers have no process and all they have is multitude of possibilities. Hackers are innovative in trying various permutations in compromising the application.

A million dollar question is whether we can build secure applications when a Developer is focused on functionality but not on breaking the application?

There is a school of thought about Inside-out security where the application is built securely from scratch. Unfortunately,  this approach won't suffice because hackers traverse Outside-in. A little reflection will highlight the importance of vulnerability scanning and penetration testing of application. This will bring the perspective of what developers do not know already.

Building a secure application inside out is not enough. In order to address unknown unknowns (or blind spots of developers), penetration testing should be done. Both whitebox style penetration testing (where components of an application is known)  and also blackbox style penetration testing which mi micks an Hacker who may not have any knowledge of the application, should be carried out.

An application of higher level of security is not built just by Developers. It is built by integrative process of Developer mindset and Hacker mindset.  This is a constant struggle for years to come.
 

Originally, researchers in CEP were not trying to solve a problem of streaming data or streaming events.   Often we read this mischaracterization by folks in the database/streaming domain, as they were focused on the low latency processing of streaming events.   A natural extension of this research has been stream processing software (often called “engines”) that process streaming data with continuous queries, for example market data feeds for algo-trading or best market order execution.  This mischaracterization is partly responsible for why we see many order processing applications in market data stream processing mislabled as “complex event processing” applications.

The genesis of complex event processing was not the stream processing need for “feeds and speed” but the processing capability to solve what can be characterized as the “problem of asymmetric capabilties”.   The term “asymmetric” has been used in the military domain. For example we often hear the term “asymmetric warfare.”  However, in general the concept of “asymmetrical processing capablities” is the true genesis for CEP and related processing concepts and domains.   It is this genesis that distinguishes CEP from EDA, SOA, SOR, and so many other technology oriented concepts.

In order to illustrate what I mean by “asymmetrical processing capablities” we will take the example of the evolution of rocketry.    In the early days, scientists learned how to make rockets, I assume with gunpowder and similar chemical compounds to launch rockets.   Over many years the application of rocketry advanced much faster than the ability to understand the situations created in the sky.    In other words, folks could fill the skies with rockets long before they had the capability to track and identify (or sense and respond to)  the rockets in real time.

Therefore, the concept of “asymmetrical processing capablities” is the situation where there is a capability, such as “launch a rocket, sense-and-respond,” that is asymmetric in nature.    In other words, the capability to detect multiple rocket launches creates an asymmetric situation where it is easy to launch rockets, but hard to detect and defend against those launches.

The same concept can be applied to everyday air travel.   If we could only fly airplanes, but did not have the capability to track the planes, understand situations in airspace, and then respond to changing situations, air travel would be quite difficult.   Lucky for us, the global traveller, there is symmetry in the capabilities to build and fly aircraft and the capabilities to detect, track and follow the evolving situations in the sky.

The genesis of CEP was to solve the problem of asymmetry in cyberspace, or if you prefer, distributed data networks.   The folks who identified, early on,  the problems associated with asymmetry in cyberspace were folks working the the field of network and security management.    This is because there has been, and is currently, a great asymmetry between the capablities to “launch a process or transaction” in cyberspace and the capabilties to detect and track what is going on in the same domain.

In my next post on this topic, we will go into some details of this asymmetry and review the first CEP projects from Stanford University in the context of asymmetric processing capabilities in cyberspace.