Today, allow me to update you on FAIR and the movement towards a formal, open standard.  There’s a couple of cool things going on in our little risk-world.

First, The Open Group Security Forum continues to move towards a formal adoption of FAIR.

WHAT DO YOU MEAN “WE” - YOU GOT A STANDARDS BODY IN YOUR POCKET OR SOMETHING?

Our meeting in Chicago a few weeks ago was great, but also slightly disturbing for me. I got pronoun-confusion syndrome.   I’m used to using the “we” pronoun to refer to RMI, or Jack and myself as we vet the models.  So without even thinking I would said “we have been looking at how loss occurs, and may want to change the model some” and The Open Group Members freaked out (rightfully so).  Adrian Seccombe gently reminded me that the “we” was now the Security Forum, and that “we” didn’t go changing things at will without vetting against each other.  Man I love this stuff.  I get to run our thoughts and ideas past some great folks now - you know, those smart people who tend to have really complex problems and are trying hard to solve them.

Formal Adoption:  Soon, Very Soon Now

Formal Adoption basically means we’ve made this document, everyone is close to saying that they generally like it, and once that finally happens then “bam”, we’re ready to move onward and upward with better things (see Cookbooks, below).  We’ve got a couple of changes to the current document that have been requested that aren’t a big deal.  For example, one request is that we make some statement about general applicability of FAIR to risk domains outside of the IT realm.   But once additions like that and others are done, this long process should be complete.

New Document Moving Towards Public Release:

We’ve got a basic document that should be public in the next few weeks on “What Makes a Good Risk Assessment Methodology” - written by yours truly and Jack.  It’s a very high-level document, and serves two purposes:

• For novices it helps parse out what is important in any undertaking to understand corporate risk (the repeated discussions on the ISO 27001 mailing list make me think it would be a place ripe for such a document).
• For those who “know” risk, it helps to re-establish some fundamental principles like the use of scales (ratio, please), the implications of dealing in probabilities, what attributes like consistency and defensibility mean, how “risk” should be reported to the business (something you know, meaningful) and so on.

When this doc is deemed ready for public consumption I’ll be sure to post on this blog here.

COOKBOOKS, EUROPEAN AGENCIES, AND, IRON CHEF “RISK” - WHOSE CUISINE WILL REIGN SUPREME?

One interesting thing that came up in the Chicago meeting was that ENISA (The European Network and Information Security Agency) developed a very nice document that reviewed something like 18 different risk assessment methodologies against their Criteria for Goodness.  FAIR was one of the ones they reviewed, and we (the royal “we” used there to include all us FAIR-Folk) did awfully well.  Things of interest:

1. They based their work on the current introduction paper which is not at all a step-by-step guide towards an organizational risk assessment (what ENISA really wanted) and we did pretty well.  Well enough that if we had developed a paper along the lines of NIST 800-30 or OCTAVE for the use of FAIR in a formal process, we could have done really, really well.  Like won-the-bake-off kind of well.
2. FAIR is actually not at all incongruous to many of the risk assessment methodologies offered, and in fact compliments many of them by letting those methodologies develop real, structured probabilities.  Think OCTAVE, where they basically say “math is (probabilities are) hard, so if you want to do them for reals, good luck!  But here’s a nonsensical way to do things if you want to believe in magic-fairy risk“.  FAIR fits right in there by stomping on the magic-fairy risk with the jack-boots of rationality.  FAIR similarly helps other risk standards that might lack structured probability development.

So The Open Group Security Forum decided that though we could create a new document and totally p0wn any future ENISA bake-off, there wasn’t much demand for the development of that documentation by the membership  - a point which was made quite apparent at the beginning of the discussion when one large European company CISO asked “What’s ENISA?”  Relevancy is everything, I suppose.

But that second item up there - the one about helping rather than competing with other “risk assessment methodologies” - really struck a chord.  So “we” (The Security Forum) are going to develop some “Cookbooks” that basically are high-level documents that say “If you want to use FAIR with (OCTAVE/COSO/CoBIT/Whatever) here’s how it fits, makes it better, and improves your life.  I’m pretty excited about these, and our first document looks like it’s going to be COSO integration.

THE OPEN GROUP SECURITY FORUM - THEY’RE A TRUSTING BUNCH (WITH QUALIFICATION, OF COURSE)

Finally, many people have asked me “Why work with The Open Group?”  There are many reasons, to be sure, but I will give you one example.  Members of the Security Forum there are not only great at vetting the model and getting consensus on risk and risk factors - but they’re quick to start applying.  So in Chicago, I thought I’d be talking about FAIR and the standard and fighting groupthink.  Nope.  Not at all.  In fact, the forum members spent more time suddenly discussing use of FAIR in a new Trust Model they’re developing.  So all of the sudden, I’m part of a new and exciting project to develop a Trust Model - how cool is that?  While formal adoption of the Trust Model will be necessarily long and deliberate - the collaboration and development is happening much faster than I can keep up with.  But if you all will allow me, it will help me get my head around it all by blogging about it later this week.  So be prepared to read about me dealing in “Trust” a little bit.

Had a great time Sunday with Rob Newby. We solved the world’s problems over deep fried whitefish and french fries (fish & chips to him).  It was a very good time, even if my driving did make him a bit uneasy.  If I may quote myself (said in an attempt to soothe Rob’s uneasyness about being lost in the car of a complete stranger in a strange country):

If your life doesn’t imitate the surreal aspects of a Douglas Adams book at least once a day, you’re just not living right.

Aside:  Bruce Scheier already has too many awards and too much recognition, so go vote for Rob instead :)   :  http://robnewby.blogspot.com/2008/07/award-up-for-grabs.html

SEPARATION OF CHURCH AND (CURRENT) STATE

Rob and I spent some time discussing risk and security,  and our conversation circled around the (now) recurring blogo-topic concerning the State of the Practice.  It’s a favorite topic of mine, so I’ve been delighted that it has reappeared in blogodom.

Rob writes about it some here in PCI the Priest.  LonerVamp’s and Richard Bejtlich’s blogs talk about Galileo, his confrontation with his church, and lessons we can learn from history (there’s nothing wrong with them recycling the meme, IMHO - because I, for one, never got closure the first time). Jon added a nice quote from Feynman today that’s also inline with the meme.

I’m not going to belabor the analogy, the “art vs. science” misnomer, nor discuss the problems with our various canon (PCI, ISO, CoBTI, COSO, blah, blah, blah).  Rather I’d like to talk about some essential things I think our industry needs to “sort out”  before it can move on towards a more scientific view of the world.  And by “sort out” of course, I mean agree with me on

CAN’T WE ALL JUST GET ALONG?

1 - Can we agree that risk is a probability issue?
Now obviously, you can retreat in probability theory a century or so and claim that risk is a Knightian uncertainty and that we just can’t “know” it.  Have fun.  But you should know that there’s the catch - “security” is also a probability issue.  So I’m betting that you can’t know “secure” for much of the same reasons Frank Knight would argue we can’t know “risky”.

But if risk (and security) is a probability issue, however, then we’re going to have to do better than “A’s in three college courses in statistics” to address the problem.  We will have to do as Curphey (and others) suggest and bring elements of other disciplines to bear on our problem space.  Let me suggest probability theory and economics as fine, fine places to start.

2 - Can we agree to stop measuring stupidly?
We have to agree that Ordinal Scales are not measurements, and Interval Scales are not useful measurements?

I had a post titled “More Ways To Confuse Your Auditor/Assessor” but it turned out to be a pretty cruel discussion about how we tend to try to act like our calculations based on ordinal or interval scales are useful (hint:  insist that your auditor/assessor/consultant replace the label “one” with the label “zero”).

Note that if risk is a probability issue, then we’re going to have to throw out the concepts of measuring in any scale other than a ratio anyhow.

3 - Can we agree on a (good) taxonomy?
We’re going to have to do (much) better than ISO 27005 (nudge, nudge).

4 - Can we agree we need to do a better job with our data?
We’re going to have to do better with measurements, metrics, models and testing.

It’s a shame that honeypots tend to be under appreciated.

5 - Can we agree to test that data and share it with each other?
We may not need to share specific data, but we will need to share when a model falls down.

I’d like to be as idealistic as some of my fellow ‘New Schoolers’ and suggest we’ll someday all be sharing data together, but I’m skeptical.  But that doesn’t mean we can’t demonstrate where results from the models we use are not repeatable, consistent or logical.   One thing Rob and I talked about at length yesterday was the ability to disprove a model using realistic but “substitute” or sanitized data.  There’s gonna be a TON of work to be done here, and that work will take not years but careers.  Which begs a great question:

Is it the sharing of data that we need, or the sharing of models?

HELP ME OUT, HERE
That’s my list of 5 fundamental concepts I wish we could move past.  Let me ask you - what else am I missing?  What’s it going to take to get past our current malaise?  How does the New School reach critical mass?  Who is going to help us agree in a centralized manner?

Your comments or own blog posts are most welcome (please include a trackback or post here)

CISCO WEBINAR UPDATE
First, Jack’s Webinar with Cisco is Thursday.  If you were lucky enough to get a slot, be sure to catch it.  If you didn’t get a slot but would like to still go, let me know (info –at– riskmanagementinsight–dot–com - subject Webinar).

RISK MANAGEMENT STANDARDS AND FAIR

Second, The Open Group has a Press Release out this morning:

“The Open Group Security Forum Initiates Development of Risk Management and Analysis Taxonomy”

You might know The Open Group from their efforts with UNIX or SOA or helping the Jericho Forum.  You’ll recall that a while back I had mentioned that RMI was working withThe Open Group, and today’s announcement is a culmination of about a year and a half worth of effort there.   Today The Open Group formally announces our (we’re members) intent to put a stake in the ground concerning risk and risk management.

Our goal is common language and common models to create meaning.  This has the capacity to change everything - the way we audit, the way we talk to other lines of business, the way we gather metrics… a Herculean effort, to be sure, but I think that The Open Group is one organization that can effect change because it is:

• Open & Participatory - Unlike many organizations developing security standards, anyone can join and anyone can contribute.  Because there are real people (doing real risk work) as members of the forum, you won’t sit back at the end of some work day working on risk and think, “Who are these people, and why are they making my life so miserable with all these unnecessary hoops to jump through?”

• Authoritative and Structured - That is, change is welcome but carefully instituted.

These are important qualities to me.  When you look around at some of the risk management efforts out there, too often you’ll find that the people instituting models and standards are removed from the actual practitioner, and/or the institution creating these standards are autocratic.  The change our profession needs cannot happen from one vendor or from one  bureaucracy that takes little account for the wishes and opinions of it’s constituency.

YET ANOTHER RISK MANAGEMENT EFFORT?

Some folks may be thinking “do we really need another risk management effort?” And really, I sympathize with the thought.  There’s ISO risk management stuff, there’s OCTAVE and NIST 800-30 and AS/NZ 340 and CRAM and FRAP and others…

And this is where I think FAIR and The Open Group have a good fit.  FAIR as a model for analysis, does not compete but rather compliments OCTAVE and NIST 800-30 and ISO 2700x (That reminds me, Rybolov, I’ve got to respond to your 800-30 article). In fact, one of the goals for the work with The Open Group is supporting documentation (call them white papers or guidance letters or whatever) that talks about how to use FAIR and the work of The Open Group Forum with ISO 27001 or as probability determination within OCTAVE, or in context with COSO efforts, etc…

SO WHAT DOES THIS MEAN TO YOU?

Well, it means a couple of things.  First, you have somewhere to go where people are vetting the models.  There is a forum of users and people with the same risk management issues and challenges as you have, but that are committed to working together to make things better.  A forum in which you can contribute and work to vet models against experience.  A forum that is a “vendor- and technology-neutral consortium” with experience building standards that work to interoperate across organizational and industrial boundaries.

Second, it means that you have a nice reference point for people who want it.  Defending the use of FAIR over some other analysis method got a little easier thanks to the increased credibility of The Open Group.

Third, new and exciting things are already happening at The Open Group in the Security Forum surrounding new standards and new ways of doing business.  Even if Risk Analysis isn’t your primary passion, let me encourage you to get involved with The Open Group’s Security Forum. Mike Jerbic and Ian Dobson there both have a passion to help codify what works and what helps security and risk management departments, regardless of “silo” or discipline.

WHAT DOES THIS MEAN TO RMI?
If you’re an employee, or client, or just a well-wisher, today’s announcement is just one culminating factor of the past year of changes RMI has undergone.  The announcement means that we’re now no longer the sole custodians of FAIR, but simply part of a larger effort to drive a better understanding of risk in our industry.  We (RMI) have a responsibility support and contribute to the effort, but the journey is no longer ours alone.  We’ve got friends.

New Website

I think our new website reflects who we are and what we do better now.  It takes into account not just what we can do because of FAIR, but also what we’ve been able to synthesize because of it (and the use of our other models and frameworks to create a whole picture of what is Risk Management).  The primary focus of our message no longer needs be that we’ve got something new and cool that makes you better - we’re freer to talk about our experience and abilities - very much reflecting the maturity we’re experiencing as a company.

In order for the framework to have a solid risk-based foundation we will be using many of the principles of COSO. In particular, the framework will be based on event identification, risk assessment, risk response, and control activities. The IT context is established by utilizing the ITIL framework for IT service delivery. IT services are used to identify risk events. Scenarios are developed for each identified risk outlining the actions necessary to realize the risk. Controls are then mapped to each scenario to either prevent or detect the actions.

Since COSO and ITIL will be used to develop the framework, a common industry-agnostic set of key risk indicators can be established and bring some consistency and clarity to IT risk management.

COSO was originally formed in 1985 to sponsor the National Commission on Fraudulent Financial Reporting, an independent private sector initiative which studied the cause-and-effects that can lead to fraudulent financial reporting. 

COSO developed enterprise risk management (ERM) recommendations for public companies and their independent auditors, and also for the SEC, other regulators, and for educational institutions.

At the heart of COSO is events and how events, both opportunity and threat-related events, in context, effect enterprise risk management.

Detecting opportunity and threats in real-time, both mentioned in COSO, is a core CEP concept; so I will be blogging on how CEP relates to COSO and ERM (and also Basel II ORM) in a future blog post.

Please stay tuned …

• ...numerous standards organizations have issued leading or “best” practices for control design and implementation; however, neither SOX (Sarbanes-Oxley Section 404) nor the PCAOB (Public Company Accounting Oversight Board) recommends a specific set of controls.


• ...In 2004, (PCAOB) issued a statement that COSO (“Committee of Sponsoring Organizations’ Internal Control—Integrated Framework"), or any other generally accepted control framework could be used. Note: it did not say COSO was the only one.


• But COSO can pose a problem...COSO doesn’t set out details. As its name implies, it is a framework.


• Each organization must still go through the difficult process of setting out its own system of internal control to meet its perception of COSO—which, in broad terms, is more of a philosophy than a set of rules.


• To fill the gap between theories and practice in implementing effective general IT controls, managers have turned to other externally developed standards and frameworks, such as the Information Technology Infrastructure Library (ITIL) from OGC, CobiT from ISACA, or the 20000-series of information security standards from the ISO/IEC