The report was written by a committee whose members include William Perry, a professor at Stanford University; Charles Vest, the former president of MIT; W. Earl Boebert, a retired senior scientist at Sandia National Laboratories; Cynthia Dwork of Microsoft Research; R. Gil Kerlikowske, Seattle's police chief; and Daryl Pregibon, a research scientist at Google.

They admit that far more Americans live their lives online, using everything from VoIP phones to Facebook to RFID tags in automobiles, than a decade ago, and the databases created by those activities are tempting targets for federal agencies. And they draw a distinction between subject-based data mining (starting with one individual and looking for connections) compared with pattern-based data mining (looking for anomalous activities that could show illegal activities).

But the authors conclude the type of data mining that government bureaucrats would like to do--perhaps inspired by watching too many episodes of the Fox series 24--can't work. "If it were possible to automatically find the digital tracks of terrorists and automatically monitor only the communications of terrorists, public policy choices in this domain would be much simpler. But it is not possible to do so."

A summary of the recommendations:

• U.S. government agencies should be required to follow a systematic process to evaluate the effectiveness, lawfulness, and consistency with U.S. values of every information-based program, whether classified or unclassified, for detecting and countering terrorists before it can be deployed, and periodically thereafter.

• Periodically after a program has been operationally deployed, and in particular before a program enters a new phase in its life cycle, policy makers should (carefully review) the program before allowing it to continue operations or to proceed to the next phase.

• To protect the privacy of innocent people, the research and development of any information-based counterterrorism program should be conducted with synthetic population data... At all stages of a phased deployment, data about individuals should be rigorously subjected to the full safeguards of the framework.

• Any information-based counterterrorism program of the U.S. government should be subjected to robust, independent oversight of the operations of that program, a part of which would entail a practice of using the same data mining technologies to "mine the miners and track the trackers."

• Counterterrorism programs should provide meaningful redress to any individuals inappropriately harmed by their operation.

• The U.S. government should periodically review the nation's laws, policies, and procedures that protect individuals' private information for relevance and effectiveness in light of changing technologies and circumstances. In particular, Congress should re-examine existing law to consider how privacy should be protected in the context of information-based programs (e.g., data mining) for counterterrorism.

Here are more news articles on the report. I explained why data mining wouldn't find terrorists back in 2005.

I’m assuming the author wants us to read the title as “Things to Look Out For in Performing Risk Analysis” and not “Risk Management is Folly - Stop, Stop, Stop!” The former is fine, the latter isn’t supported by the evidence presented by the subjects of the article.
The subjects of the article are a good study from Wade Baker & Co. at Verizon, and a report from RSA’s Security for Business Innovation Council. Let’s take a look at each of these and examine why what they’re saying might contribute to poor risk management, shall we?

1.)  THE VERIZON REPORT

The Verizon report is an analysis of some 530 forensic investigations their company performed.  It is well worth your time as it’s chock full of interesting information.  As it relates to the Dark Reading piece, a coarse summary would be that “likelihood” is “different” for different people and so you can’t use the same “likelihood” across different industries.

Distilled through the lens of FAIR:

“different threat communities may be applicable based on Probability of Action factors which include: Value, Level of Effort and Risk (of Getting Caught).”

Or, even further distilled and in the words of my six year old son,

“Duh-uh”.

With regards to what I assume is the purpose of the article (What Doesn’t Work in Risk Analysis) this concept  seems just to rehash the old GIGO argument regarding risk analysis.  Great.  Can’t argue with that, nor it’s corollary QIQO (quality in, quality out).

But let me ask you -  is this really a problem common in your analysis?  Did reading this article make you go “Crap, we’ve been using data normalized across multiple industries in our analysis! They’re all wrong!”  Or have you already been accounting for the unique value proposition your company has to the specific threat community you’re worried about?  See, maybe I’m just not your average analyst, but even in my NIST/OCTAVE days, this has *never* been an issue for me.

Let me be specific, this is not a problem with Verizon’s very cool report.  It’s just that I don’t see what the big deal is.  This article is starting to feel like someone is running through the motions, trying to play the ” a crazy title gets people to read a boring article” game.

Speaking of cool reports - You know what would be cool?  I think it would be interesting to see is the quality of these companies’ “risk management process” established using good criteria,  and then correlated to the frequency and magnitude of real-world losses across the aggregate sample.  In other words, can we establish evidence that strong risk management practices not just reduce “risk” but also reduce actual incidents.

2.)  THE RSA COUNCIL “EXPLORES WHY LEGACY METHODS OF EVALUATING INFORMATION SECURITY RISK DON’T WORK IN TODAY’S CONNECTED WORLD, IN WHICH ANY NEW BUSINESS INNOVATION INHERENTLY CARRIES SOME LEVEL OF RISK TO INFORMATION.”

This report from the RSA council puts forth a seemingly obvious proposition, that risk must be balanced by reward.  Why is this news?  Now as I read the article it’s not clear if:

• The RSA Council is claiming that the CISO’s office should be the ones determining reward.  Absurd.

or

• Businesses aren’t doing a good job at determining risk and reward.

Let’s go with the latter.  So I’m pretty sure (good) businesses do a good job at estimating reward.  Businesses I’ve been a part of?  We LOVE(D) estimating reward.  We don’t tend to start projects all willy-nilly. No we tend to be careful to identify the size of the market and what it will cost to address the market.  So what could the problem be that this RSA council is trying to address?  Maybe it has to do with something like the following:

Yesterday, I got a demo of an IT-GRC application that shall remain nameless.  It seemed to be very good at the “C” bits - lots of information on regulations and expectations and even what sorts of controls would answer the regulations (which is goofy, but we’ll have to talk about that later).  It also gave you the ability to build workflow quite nicely.  But it measured NOTHING.  There really was no observable “G” and “R” was really Medium X Low X Low = High sorts of stuff.  So let’s use this relatively expensive tool as evidence of what your average CISO is armed with going into a Risk/Reward sort of meeting.  I imagine a nice board room with wood-grain paneling and glass bowls filled with little chocolate covered mints designed to give everyone involved in the meeting (CEO, CFO, CIO, CSO, VP S&M, etc…) a little sugar rush when needed and fresh breath.  The conversation goes a little something like this (apologies to Rich):

Business Guy Who Wants to Make Money Because That’s What Businesses Do: Based on market studies, we believe that initial gross revenues from the new product and technology rollout will be eleventy gazillion dollars based on a 37% market penetration in Scandinavia, alone.

CSO: Well now, we have a likelihood of “High” and a “C” impact of Medium, and an “I” impact of Low, and an “A” impact of “High” and because we are a (bank/hospital/retailer/basically any business that breathes anymore) we weight “C” by a factor of 2 - we multiplied those all together and got a “High”.

So can you guys delay the product rollout by 9 months and give me a bunch more money that’s not in the budget so that I can get this thing down to a “Medium”, please?

Again, I just don’t see the problem with Information Risk Management being that our businesses have no idea what the rewards of business might be.  Now maybe we need get a seat in that boardroom just to be able to talk about our “Mediums”, sure.  And maybe we’re infantile in our ability to describe our problem space.  But I cannot fathom that “Risk Management Doesn’t Work” because businesses haven’t been considering “reward”.

WHY RISK MANAGEMENT MAY  NOT BE WORKIN’ FOR YOU

Two meta-categories of causation:

• No skills

and/or

• No resources

Any ancillary “cause” can be mapped to one of these categories.  You could have significant resources but crappy models, and have conversations like our imaginary CSO, above.  You could have really good models and people trained and motivated to use them, but scarce time & money, so no conversation happens.

Now my question for you is - which does it make sense to acquire *first* to solve the “Why Risk Management Doesn’t Work” problems, skills or resources?

1. Great paper that complements the whole "SIEM is dead?" saga - "Most enterprises are looking for a product that will solve all of their problems in some sort of off-the-shelf miracle, and when they find out that the currently available tools can't do it, they either postpone their deployment or put them on the back burner. "
2. "The Mess: looking for someone to blame?" is an awesome piece on Internet security and its architecture - and so is Gunnar's follow-up ("If a tree falls in someone else's silo...")
3. Mike call to "Rise up against Mediocrity."  - "Dilbert makes the risk of the lowest common denominator approach abundantly clear."; in other words, you say 'best practices', I say 'mediocrity!' Mike also remind us, in vain, to do "Security FIRST!" (and compliance second)
4. A great piece from Burton: "On Response" - I think the world needs another 10-20 million reminders that PREVENTION FAILS. This is definitely a good one for those still in the "we'll just block the threat world" - "we will not win a continuing war of escalation" and "using response can be more cost effective than installing the latest and greatest preventative tool"
5. More on metrics, including the highly-awaited ISO27004.
6. Pretty dumb paper by a person confused by why PCI DSS exists (the guy needs to read this). PCI doesn't "fall short," it helps people who will otherwise not do anything and their systems will "power" those botnets of the future...
7. While we are on this subject: a really good coverage of PCI 1.2. changes, released Oct 1st. More PCI fun here. And more here ("PCI Compliance - dispelling some common myths"). And, more PCI myths. And more good ideas on PCI from Mike R. Sorry, can't stop thinking about PCI :-)  - also this is good.
8. Adrian on behavioral monitoring; mostly in DAM, but also elsewhere in security.
9. "Premature Chasm-Crossing"  - a must-read for all security vendors and especially their marketing (and  their easily-excitable PR teams...) - "Shouldn't vendors be spending more time fighting the problems that security managers are facing today, right this minute?" (Mike R also comments on that). A related - and  just as interesting point is made here: "Security is not a solution"
10. More on compliance and security checklists, good and bad: "I think this is a dangerous trend unless the "checklist" is all inclusive." (how can a checklist include ALL? :-))
11. "SANS Top 7 New IR/Forensic Trends In 2008"
12. Read "The three approaches to computer security!"  Why? Come on, it is from Joanna! :-)
13. A fun discussion about a hot new technology: network IDS. Is IDS absolutely indispensable to ALL companies? No. Can it be incredibly useful? You bet. End of discussion.
14. On an unrelated note, are lasers the future of warfare? Some say no.
15. Finally, some security humor from Gartner (!): "Get Rich Quick With Network Security"

Enjoy!

Previous security reading.

Does Oracle run the world? I would have to say no but Raj (Larry Ellison is his idol) and the 40,000 Oracle customers that descended upon SF last week might beg to differ. What do James Carville and Mary Matalin have to do with enterprise software? Pretty much nothing, except for the fact that they delivered the opening keynote for Oracle OpenWorld. (And that’s the only and last politically-oriented thing you’ll hear from me as we run up to the election). For a surprisingly funny and extensive photo gallery of the eye-popping event, check out cdye1’s photostream on Flickr.

But UB40, Elvis Costello and Seal aside, Oracle OpenWorld did offer training, certifications, and always entertaining speeches by Ellison. Ben Worthen’s favorite – “Larry Ellison’s Brilliant Anti-Cloud Computing Rant” delivered to analysts on Thursday. From Ben’s slightly-edited excerpt:

“The interesting thing about cloud computing is that we’ve redefined cloud computing to include everything that we already do. I can’t think of anything that isn’t cloud computing with all of these announcements. The computer industry is the only industry that is more fashion-driven than women’s fashion. Maybe I’m an idiot, but I have no idea what anyone is talking about. What is it? It’s complete gibberish. It’s insane. When is this idiocy going to stop?

“We’ll make cloud computing announcements. I’m not going to fight this thing. But I don’t understand what we would do differently in the light of cloud computing other than change the wording of some of our ads. That’s my view.”

So did everyone catch that? Cloud computing is complete gibberish and idiocy, but apparently Oracle’s already been doing enough around it to advertise the fact. I will have my cake and eat it too!

We’ve been pumping out the posts from the shows we went to – let me tell you, live-blogging is hard when you’re trying to share apparently miniscule amounts of bandwidth with 14,000 other attendees – and we have even more to share as we step back, contemplate and describe how some of the announcements, info and especially roadmaps fit into our overall picture over here at ScienceLogic.

For example, we released the results of our annual industry IT survey last week. Twice a year – at FOSE (for Government IT) and at Interop NY (for enterprises) – we take advantage of the fact that we have a big beautiful booth at these shows and offer a fabulous ScienceLogic t-shirt in return for a couple of minutes time with attendees living the problems we try to solve. Instead of telling people what their problems and priorities are, we like to ask.
Interop NY Survey - Trends and Challenges
Detailed Reports on Trends and Comparison to Government IT

And I just had to share this one because it is so bizarre. Are VMware and Paul Maritz guilty of plagiarism? You have to check this out to get even part of the picture. Apparently this guy has posted his slides (we know they are from VMworld 2007 because it says so in the lower-right-hand corner…) which prove that the “virtual datacenter operating system” idea was his idea a year before it showed up on Maritz’s keynote this year. Hmmm. And then after posting all these slides and making all the connections between his presentation and Maritz’s, he says he’s just kidding about the plagiarism. Can anyone sort this out and let me know?

I’ll tell you who wasn’t kidding when I went by their booth at VMworld – a certain chargeback vendor and VMware “partner” who was quite shocked two months ago when they walked into a meeting with VMware about future roadmap. Apparently, the slides they saw (preview of VMware’s announcement re adding extended chargeback capability within vCenter management services) were mighty might similar to slides they had given in a presentation to VMware about their own roadmap. Coincidence? I’ll let you decide. And I’ll also say, their strategy to combat this – support for Hyper-V coming early in 2009.

The Results:

Which hypervisor(s) are you currently using?

• 72% VMware
• 17% Using something else
• 9% Hyper-V and VMware
• 2% Hyper-V

(based on 46 responses)

So the VMware responses were in line with what we thought, although I’ve seen numbers up to 90% share of the market. And about 10% are at least playing with Hyper-V – pretty good numbers just a few months out from launch. But look at 17% using a hypervisor other than Hyper-V and VMware!

We know from talking with people that several brought up Xen. I have to tell you that other than from media and analysts, we never hear about Xen (Citrix), which is why we didn’t include it in the survey as a specific selection. Perhaps it took the introduction of Hyper-V, with the attendant marketing juggernaut, to break people of the VMware-only habit. Xen couldn’t really carry that “heterogeneous” hypervisor environment message on its own, but now that Hyper-V is available, the genie’s out of the bottle. Bears watching.

On another note: We were more successful in hanging onto our marbles on day two – people seemed more in tune to the poll and less focused on collecting giveaways than on day one! [Note: no attendees were irrevocably harmed during the execution of the polls. :)] At Interop Vegas, May 17 – 19, 2009, we’ll be about a year out from Microsoft launching Hyper-V and will make sure to ask the same question then to track changes in hypervisor adoption.