UPDATE:  @MYRCURIAL from the great site Liquidmatrix says that I need to post the following warning:

YOU MAY NOT WANT TO PROCESS THIS PRIOR TO YOUR 11TH CUP OF COFFEE

==================================

Carrying on from yesterday’s post a bit, I’m happy to admit that Chris’ poem is right: we don’t have nearly the information we need now when we’re supposed to have “control” over our assets, putting things in a hosted/asp/cloud/buzzword model ain’t going to help our quest for visibility. My intention was/is to show that you need visibility (in part one) and then today explain that unfortunately, that’s only half the picture.

Today’s follow-on is about the fact that whatever visibility we can contractually enforce (be it in the “cloud” or in our own perimeter) has to be informative (Amrit, this is why I was plugging you with those variance questions on Twitter yesterday).  That is, we can ask whatever IT department (ours, theirs, whomever) for all sorts of information, and maybe they’ll even give it to us.  But we’re not really ready to:

• Know what to ask for
• Use it to create wisdom

A really salient example of this from outside IT hit my browser this morning.  Now it’s not at all my intention to be political or endorse one candidate over another.  Those who know me know I’m fiercely independent.  But this morning there’s a headline on a well-read news website about how one candidate is now “+2″ over another in a Gallup poll of “likely voters”. The source is here.

That is a screen grab from Gallup’s website that shows the “+2″.   I have to ask - how informative is this information?  Part of the problem is that Gallup’s methods are hidden as some sort of “secret sauce” (their FAQ section doesn’t help much, either).  But regardless of the quality of the measurement, this “+2″ has no context - we don’t really know what this information means with regards to an actual election.  Nor is there any predictive element (I hate the using the word predictive, but it’s common nomenclature - so there you go).  We don’t have what we need from this Gallup poll to create wisdom about the ability of either candidate to be elected.

Allow me show you what I mean by way of contrast.  Take a look at Nate Silver’s work at http://www.fivethirtyeight.com/.  Now I’ve been long familiar with Nate due to his work in baseball.  He’s been at these sorts of ‘predictive’ analytics around our shared passion: creating wisdom from baseball statistics.

What Nate is doing at 538 is applying that acumen from his baseball work to the political process.  He’s breaking down the vote not just on popularity among likely voters, but in the context of the electoral college, accounting for variance and uncertainty, running Monte Carlo simulations and taking into account all sorts of polling information.  The result is really quite amazing. Here’s just one graph he presents - it’s the most similar to the Gallup one above, but you should really visit the site to understand the difference in quality of information and to check out the predictive elements he creates.

NOT ALL INFORMATION IS CREATED EQUAL, AND NOT ALL  JUDGMENTS ARE CREATED EQUALLY

And take a look at the contrast, here:

On one hand you have Gallup giving us a “+2″ advantage to a particular candidate.  Now Gallup themselves draws no conclusion but, as digested, how many readers do you think take this as evidence that the election is *really* close?

On the other hand, 538’s predictions show a 348/189 electoral college split, and one candidate winning 96% of the time in simulated elections.  That doesn’t seem close at all!

RISK MANAGEMENT

It is these predictive elements that we need in order to make better strategy and decisions.  I’ve been talking in the past about risk management’s inability to link current state to systemic causes, and this “context” is what predictive analytics provide.  We might have all sorts of visibility into our environment, and measurement of various amounts of variability that visibility gives us. But unless we have context to create wisdom, it’s all just, as Chris says, “machinations”.  We have to move beyond “+2″.

So Cloud/Grid/Utility/ASP/TimeShare/Whatever you want to call it - security will have to clean up our own mess first before we can do a good job with or without a perimeter.  Once we can start moving beyond “+2″ statements, then we can know what sort of visibility we require into an ability to Prevent, Detect, and Respond.

You can read all the details on their website (except for the part about the certification not being a measure of practical skills). From what I can tell, the CSSLP is just the CISSP with different CBKs, or Common Bodies of Knowledge. As with the CISSP, they are going for broad knowledge, not depth. Starting in June 2009, you can get certified by taking a paper exam, likely a multiple choice test similar to the CISSP. Why June? Because the test isn’t even written yet — I’ve heard from several sources that they are actively soliciting their existing pool of CISSPs to help write test questions.

Ah, but what if you can’t wait that long and want to get certified right away? You’re in luck. If you act before March 31, 2009, you can get grandfathered in without even having to take the exam! That’s right, they call it the CSSLP Experience Assessment, and here are the requirements:

• Upload a resume showing three years of experience related to software security, or four years if you don’t have a college degree
• Write short essays (500 words maximum) discussing four CBKs of your choice
• Get a CISSP to vouch for you
• Pay $650

Let’s examine these requirements one at a time.

Three years of experience. (ISC)2 doesn’t provide any requirements on depth of experience, other than citing the broadly-defined CBKs. Considering they are targeting everyone from software developers to security assessors to business analysts (yes, really), chances are they are going to accept any experience that is even tangential to the SDLC or software security.

Short essays on four of the CBKs. I asked the (ISC)2 exhibitors specifically what they are looking for to satisfy this requirement, and they said the essays should be a general discussion of the CBK topic, optionally citing your personal experience in that area if you have any. This messaging is not quite aligned with the website guidance, which states that the essays should be “Accomplishment Records” which are self-reported descriptions of experience. Either way, with a maximum essay length of 500 words, it’s pretty obvious that substance is not (ISC)2’s first priority. Here’s one data point for you: I spoke to someone who has already submitted the CSSLP Experience Assessment, and he said it took about an hour to write the essays.

Get a CISSP to vouch for you. Actually this can be any (ISC)2 certified person, not just CISSPs. Contrary to what you’d expect, though, the person isn’t vouching for your skillset so much as they are confirming that the attestations on your resume are accurate.

Pay $650. You knew it was coming. After all, there is money to be made. How is it that qualifying for the CSSLP through professional experience should cost $650? If you’re taking the written exam, fair enough, (ISC)2 does incur the cost of administering and grading that exam (even though the Scantron machine is probably paid off by now). But $650 for the submitted-online Experience Assessment? If we assume that the person reading these essay submissions makes a rather generous $100k per year, then $650 accounts for roughly a day and a half. Will it really take that long to read a maximum of 2,000 words and pass judgment? Of course not. (ISC)2 wants to get as many people as possible to qualify based on “experience”, seeding the initial pool of CSSLPs and netting them $650 per head for doing next to nothing.

As Lee Kushner stated during his OWASP AppSec presentation (7 Habits of Highly Effective Career Managers), “the more people who own a cert, the less relevant it becomes.” Irrelevant — that’s exactly what the CISSP has become, and it’s exactly where the CSSLP is headed. Meanwhile, (ISC)2 will sit back and watch while you and your employers continue to fill their coffers.

In closing, let me acknowledge that this blog entry probably comes across as judgmental. I accept that. I’m not ranting against the idea of certifications, though admittedly I’m not a fan of them either. I am disappointed that (ISC)2, an organization with tremendous influence, could have created something more meaningful but chose not to. Why bother when people will just fork over the cash anyway?