1)  Gunnar has put up his speech as the Quality of Protection Keynote:  “The Economics of Finding and Fixing Vulnerabilities in Distributed Systems.” Don’t worry if that title doesn’t turn you on, his post is one of the best this year.  I wanted to make today’s blog post some reflection on what he says there, but I haven’t the time today and we’ll have to table that until next week.  Anyway, it’s excellent.

2)  Aleks Jakulin writes about The Future of Data Analysis.  I spoke with a CSO who is morphing into a CRO role and one of the things he plans on doing is hiring about  a half dozen data analysts.  If you think better use of Security Information is in your future, you’ll want to take a look at that blog.

3)  Brent Huston of the Ohio voting machine fame writes about an incident he just worked on and risk and rational security.

4)  Our friend Mike Rothman and our friends at Business Of Security/Cisco are doing a Pragmatic CSO thing.  Mike is always entertaining and practical (dare I say, pragmatic) so I think this should be a fun webex.  Hope you’ll sign up.

Namaste Risk Geeks!

UPDATE:  @MYRCURIAL from the great site Liquidmatrix says that I need to post the following warning:

YOU MAY NOT WANT TO PROCESS THIS PRIOR TO YOUR 11TH CUP OF COFFEE

==================================

Carrying on from yesterday’s post a bit, I’m happy to admit that Chris’ poem is right: we don’t have nearly the information we need now when we’re supposed to have “control” over our assets, putting things in a hosted/asp/cloud/buzzword model ain’t going to help our quest for visibility. My intention was/is to show that you need visibility (in part one) and then today explain that unfortunately, that’s only half the picture.

Today’s follow-on is about the fact that whatever visibility we can contractually enforce (be it in the “cloud” or in our own perimeter) has to be informative (Amrit, this is why I was plugging you with those variance questions on Twitter yesterday).  That is, we can ask whatever IT department (ours, theirs, whomever) for all sorts of information, and maybe they’ll even give it to us.  But we’re not really ready to:

• Know what to ask for
• Use it to create wisdom

A really salient example of this from outside IT hit my browser this morning.  Now it’s not at all my intention to be political or endorse one candidate over another.  Those who know me know I’m fiercely independent.  But this morning there’s a headline on a well-read news website about how one candidate is now “+2″ over another in a Gallup poll of “likely voters”. The source is here.

That is a screen grab from Gallup’s website that shows the “+2″.   I have to ask - how informative is this information?  Part of the problem is that Gallup’s methods are hidden as some sort of “secret sauce” (their FAQ section doesn’t help much, either).  But regardless of the quality of the measurement, this “+2″ has no context - we don’t really know what this information means with regards to an actual election.  Nor is there any predictive element (I hate the using the word predictive, but it’s common nomenclature - so there you go).  We don’t have what we need from this Gallup poll to create wisdom about the ability of either candidate to be elected.

Allow me show you what I mean by way of contrast.  Take a look at Nate Silver’s work at http://www.fivethirtyeight.com/.  Now I’ve been long familiar with Nate due to his work in baseball.  He’s been at these sorts of ‘predictive’ analytics around our shared passion: creating wisdom from baseball statistics.

What Nate is doing at 538 is applying that acumen from his baseball work to the political process.  He’s breaking down the vote not just on popularity among likely voters, but in the context of the electoral college, accounting for variance and uncertainty, running Monte Carlo simulations and taking into account all sorts of polling information.  The result is really quite amazing. Here’s just one graph he presents - it’s the most similar to the Gallup one above, but you should really visit the site to understand the difference in quality of information and to check out the predictive elements he creates.

NOT ALL INFORMATION IS CREATED EQUAL, AND NOT ALL  JUDGMENTS ARE CREATED EQUALLY

And take a look at the contrast, here:

On one hand you have Gallup giving us a “+2″ advantage to a particular candidate.  Now Gallup themselves draws no conclusion but, as digested, how many readers do you think take this as evidence that the election is *really* close?

On the other hand, 538’s predictions show a 348/189 electoral college split, and one candidate winning 96% of the time in simulated elections.  That doesn’t seem close at all!

RISK MANAGEMENT

It is these predictive elements that we need in order to make better strategy and decisions.  I’ve been talking in the past about risk management’s inability to link current state to systemic causes, and this “context” is what predictive analytics provide.  We might have all sorts of visibility into our environment, and measurement of various amounts of variability that visibility gives us. But unless we have context to create wisdom, it’s all just, as Chris says, “machinations”.  We have to move beyond “+2″.

So Cloud/Grid/Utility/ASP/TimeShare/Whatever you want to call it - security will have to clean up our own mess first before we can do a good job with or without a perimeter.  Once we can start moving beyond “+2″ statements, then we can know what sort of visibility we require into an ability to Prevent, Detect, and Respond.

Is it supposed to make me feel better about getting my problem fixed if I’m talking to someone in the Midwest versus someone in Bangalore? (Please no hate mail – I’m from the Midwest.) Honestly, I just want my computer to stop showing me a blue screen of death.

But apparently, I might be in the minority. According to the Black Book of Outsourcing (yes, outsourcing has a black book), reverse outsourcing is on the rise with “India’s leading service providers opening offices on Main Street, USA” to be closer to customers (mainly North American) and draw from the “local talent pools”.

The one area of outsourcing bucking this trend – infrastructure management. Co-writer Scott Wilson says that infrastructure management is largely automated, low touch and does not involve a lot of interaction.

Speaking as a vendor of infrastructure management tools, that’s a bunch of malarkey. Perhaps at a very low level this is true (i.e., is the device responding), but that’s just the tip of the iceberg when it comes to monitoring performance, availability and SLAs for today’s networks, systems and applications.

Certainly as vendors, we try to put as much automation as possible into our toolsets – helping our customers to simplify IT management wherever possible, enabling them to be proactive by setting up “intelligent” alarms and thresholds that warn of problems before they become showstoppers and reacting at a speed in this increasingly virtual world that simply is not possible for human manual interaction.

But infrastructure management doesn’t happen in a vacuum and you can bet when something goes wrong which affects some mission-critical app state-side, that there is a LOT of communication and interaction. And it takes a lot of work and setup to get to a level of automation where the alerting is proactive and intelligent and customized for each business.

One of the main points of tools like ours is to automate where possible in order to free up the valuable time of the sysadmins, network engineers, IT managers, etc to do the higher order work – which is how they’ll get to the next level of infrastructure management. Beyond “is it up”, infrastructure management should be providing answers to questions like: “is it always up”, “is it doing what I expected it to do” and “will it still be working as expected as my company grows”.

We have been conducting polls of our customers and of IT professionals at technology trade shows for the past two years and the results are in: Pulling together all of the management pieces and processes is even more crucial in a virtualized environment.

So what does this mean for you? You will need to refine your incident and problem management processes with new technologies in order to reduce downtime and maintain end user performance. But of course even the most basic technologies are not well integrated even in today’s world.

I recently participated in a Gartner Conference and watched to my amazement a real-time electronic survey of the audience. To my disbelief, the audience, filled with 300+ people from Fortune 2000 companies provided real-time responses to the question:

What level of integration does your IT org have between event management and service desk applications?

• None: 10%
• Manual Phone call from IT ops to IT service desk staff member: 46%
• Manual click button on event manager to open trouble ticket: 20%
• Automated event management system automatically opens trouble ticket without requiring human oversight or approval: 24%

Unbelievable… still very few of the survey respondents have yet to formalize problem management systems with event management systems. For 56% of the audience the process is still manual!

Another interesting real-time survey question at the Gartner Conference was:

Who in your organization is responsible for critical problem processes and resolution?

• IT Service Desk 13%
• IT Operations 49%
• Process Team 12%
• Other 9%
• Responsibility not formalized 17%

Virtualization adoption and the speed with which things change in a virtualized environment require automation and will transform Incident and Problem Management. Clearly with this new technology we are required to re-think Organizational, Behavioral and Cultural Challenges required to take advantage of the opportunities that virtualization provides.

Incident and problem management processes and metrics must bridge organizational silos that have been the norm within IT. With virtualization, people have to work more closely together in the different silos than ever before. IT leaders need to break down the walls between the technology-centric silo mentalities.

Business Imperative Action Plan:

1. What can you do today? –Understand the impact of virtualization on incident and problem mgt. workload, provide technology training for helpdesk/service desk staff.
2. What can you do in the next 12 months?

Formalize problem management processes, metrics and personnel.
Invest in tools and processes for systems on virtualized servers.
Long term: On the Radar Screen!
Instill teamwork into all groups responsible for the virtualized environment service and support. Map components and configuration items directly to end user services.

Final Thoughts: Know the management pieces and ensure that they fit together. It’s great to buy new technology, but be demanding to ensure that your vendors show you have they will help to link all these pieces together - Change, Inventory, Incident, Problem, Server, Capacity, Performance, Configuration, Event, and Integrated Workflow.

I’m assuming the author wants us to read the title as “Things to Look Out For in Performing Risk Analysis” and not “Risk Management is Folly - Stop, Stop, Stop!” The former is fine, the latter isn’t supported by the evidence presented by the subjects of the article.
The subjects of the article are a good study from Wade Baker & Co. at Verizon, and a report from RSA’s Security for Business Innovation Council. Let’s take a look at each of these and examine why what they’re saying might contribute to poor risk management, shall we?

1.)  THE VERIZON REPORT

The Verizon report is an analysis of some 530 forensic investigations their company performed.  It is well worth your time as it’s chock full of interesting information.  As it relates to the Dark Reading piece, a coarse summary would be that “likelihood” is “different” for different people and so you can’t use the same “likelihood” across different industries.

Distilled through the lens of FAIR:

“different threat communities may be applicable based on Probability of Action factors which include: Value, Level of Effort and Risk (of Getting Caught).”

Or, even further distilled and in the words of my six year old son,

“Duh-uh”.

With regards to what I assume is the purpose of the article (What Doesn’t Work in Risk Analysis) this concept  seems just to rehash the old GIGO argument regarding risk analysis.  Great.  Can’t argue with that, nor it’s corollary QIQO (quality in, quality out).

But let me ask you -  is this really a problem common in your analysis?  Did reading this article make you go “Crap, we’ve been using data normalized across multiple industries in our analysis! They’re all wrong!”  Or have you already been accounting for the unique value proposition your company has to the specific threat community you’re worried about?  See, maybe I’m just not your average analyst, but even in my NIST/OCTAVE days, this has *never* been an issue for me.

Let me be specific, this is not a problem with Verizon’s very cool report.  It’s just that I don’t see what the big deal is.  This article is starting to feel like someone is running through the motions, trying to play the ” a crazy title gets people to read a boring article” game.

Speaking of cool reports - You know what would be cool?  I think it would be interesting to see is the quality of these companies’ “risk management process” established using good criteria,  and then correlated to the frequency and magnitude of real-world losses across the aggregate sample.  In other words, can we establish evidence that strong risk management practices not just reduce “risk” but also reduce actual incidents.

2.)  THE RSA COUNCIL “EXPLORES WHY LEGACY METHODS OF EVALUATING INFORMATION SECURITY RISK DON’T WORK IN TODAY’S CONNECTED WORLD, IN WHICH ANY NEW BUSINESS INNOVATION INHERENTLY CARRIES SOME LEVEL OF RISK TO INFORMATION.”

This report from the RSA council puts forth a seemingly obvious proposition, that risk must be balanced by reward.  Why is this news?  Now as I read the article it’s not clear if:

• The RSA Council is claiming that the CISO’s office should be the ones determining reward.  Absurd.

or

• Businesses aren’t doing a good job at determining risk and reward.

Let’s go with the latter.  So I’m pretty sure (good) businesses do a good job at estimating reward.  Businesses I’ve been a part of?  We LOVE(D) estimating reward.  We don’t tend to start projects all willy-nilly. No we tend to be careful to identify the size of the market and what it will cost to address the market.  So what could the problem be that this RSA council is trying to address?  Maybe it has to do with something like the following:

Yesterday, I got a demo of an IT-GRC application that shall remain nameless.  It seemed to be very good at the “C” bits - lots of information on regulations and expectations and even what sorts of controls would answer the regulations (which is goofy, but we’ll have to talk about that later).  It also gave you the ability to build workflow quite nicely.  But it measured NOTHING.  There really was no observable “G” and “R” was really Medium X Low X Low = High sorts of stuff.  So let’s use this relatively expensive tool as evidence of what your average CISO is armed with going into a Risk/Reward sort of meeting.  I imagine a nice board room with wood-grain paneling and glass bowls filled with little chocolate covered mints designed to give everyone involved in the meeting (CEO, CFO, CIO, CSO, VP S&M, etc…) a little sugar rush when needed and fresh breath.  The conversation goes a little something like this (apologies to Rich):

Business Guy Who Wants to Make Money Because That’s What Businesses Do: Based on market studies, we believe that initial gross revenues from the new product and technology rollout will be eleventy gazillion dollars based on a 37% market penetration in Scandinavia, alone.

CSO: Well now, we have a likelihood of “High” and a “C” impact of Medium, and an “I” impact of Low, and an “A” impact of “High” and because we are a (bank/hospital/retailer/basically any business that breathes anymore) we weight “C” by a factor of 2 - we multiplied those all together and got a “High”.

So can you guys delay the product rollout by 9 months and give me a bunch more money that’s not in the budget so that I can get this thing down to a “Medium”, please?

Again, I just don’t see the problem with Information Risk Management being that our businesses have no idea what the rewards of business might be.  Now maybe we need get a seat in that boardroom just to be able to talk about our “Mediums”, sure.  And maybe we’re infantile in our ability to describe our problem space.  But I cannot fathom that “Risk Management Doesn’t Work” because businesses haven’t been considering “reward”.

WHY RISK MANAGEMENT MAY  NOT BE WORKIN’ FOR YOU

Two meta-categories of causation:

• No skills

and/or

• No resources

Any ancillary “cause” can be mapped to one of these categories.  You could have significant resources but crappy models, and have conversations like our imaginary CSO, above.  You could have really good models and people trained and motivated to use them, but scarce time & money, so no conversation happens.

Now my question for you is - which does it make sense to acquire *first* to solve the “Why Risk Management Doesn’t Work” problems, skills or resources?

Yesterday, they posted a terrific blog post about what they saw at Interop. Fortunately, ScienceLogic was one of the technologies that they highlighted from the show. I have written earlier posts about how difficult it has been to gain smart, insightful coverage for our solutions with technology media.

I have to say that they really got it! And it feels so good. We know that we have a bit of a hidden gem of a product here at ScienceLogic and will be working overtime in the coming months to take our business and products to a “Blue Ocean” environment that will shock and surprise many others in the media. However Sevick and Wetzel will be amongst the first to get a close-up on why and how we will deliver a new paradigm to this marketplace in 2009!

A few excerpts from their post:

“We noticed yet more specialty network management vendors, leading us to wonder how the market can support such a plethora of them, and we felt empathy for IT teams that have to master yet more interfaces.”

“Application performance management and application acceleration vendors were well represented. Such products play well in today’s climate because they allow enterprises to get the most out of existing IT investments instead of buying more “stuff”. One particularly interesting vendor we talked to was ScienceLogic. They are integrating IT infrastructure and application monitoring into a single, not-very-expensive platform that will serve mainstream business well. This is smart, and we predict they will give the CA’s, BMC’s, HP’s and IBM’s of the world a run for their money.”

Check out the blog post here and keep App Performance View on your radar..