“How I can communicate the value of an ISO implementation to non-security management?”

This question came to me after one of the posters on the ISO Google Group asked about KPIs for ISO implementation.  Got great responses in email, blog comments, and on Twitter from current/former CISO folks and consultants and analysts.  Some really great thought and effort, by the way - thank you.  It’s really great to be able to have these sorts of conversations online.

First, I have to point out some resources Brian Honan linked to from Gary Hinson, just because they’re so cool.  Gary has invested gobs of time and effort to become one of the defacto resources on the ISO (you might also want to read or re-read Gary’s web post on the 7 myths of metrics).   Brian links to an implementation guidance document(pdf) and a metrics example(pdf) document.

As full of awesomeness as they are, though, these are simply metrics “mapped” to the ISO (i.e. the ISO isn’t a pre-requisite for generating this information).  They are not KPI’s that express the value of ISO implementation.  Problem is the metrics created here still require some level of “translation” in order to create some value statement that data owners can understand.  As Myrcurial twittered me “27001 is orthoganal to process” meaning (I hope) that metrics have their foundation in events that are generated by processes.  27001 by itself was never meant to create metrics (see above), and so we’re asking a question the ISO can’t answer.  But the desire, the need to measure still exists.  To that extent we can google “ISO compliance” (whatever that means) and if something can be certifiable or deemed “compliant” we can and are “measuring”.  But does that have value? Rybolov (my favorite Guerilla CISO) wrote:

“Whatever you do, don’t start measuring percentage of compliance. Eventually, that’s what all metrics efforts around a framework devolve into.”

I have to agree.  Being ISO “compliant/certified” has little expressive business value prima facia. I find that one KPI that absolutely asserts value when expressed properly is risk - and similarly  Shrdlu wrote:

“I really have no idea. I personally wouldn’t try to justify an ISO implementation by itself. If I could show traceability on how it affected our overall security risk, then that’s what I’d do.”

And that’s a delightful answer.  That “traceability” (geeze-louise Shrdlu - what a word!) is absolutely what I’m after here.  How do I get that?  

If you’re going to do something with corporate budget (time, money - and goodness knows an ISO implementation is time & money) you better be able to communicate the value.  And while the zealotry for ISO implementation differs from person to person, I have yet to come across someone who says that ISO adoption is totally without value.  It’s just not apparent what that value of adoption is and how we can measure (metrics) and express it (KPIs).

Jenean Paschalidis wrote what he thought that value was in a very nice email in which he puts a qualitative name on the value of adoption:

“Transparency and accountability-this is what all executive/senior management (the company) is on the hook for. ISO provides that. If you want to understand and have confidence in your operations as supported by security (because you will know the who, what, where, when, why and how of a system (human, technical etc.) and you want to be able to trace back why a decision (risk-vetted) had been made - then adoption of this best international practice will assist in providing these answers.”

So working with our above thoughts a little here - if we agree with Shrdlu that the only value of an ISO implementation can only be expressed if we can say how said implementation affected our overall security risk - and we agree with Jenean that the primary benefit is an ability to have confidence in operations as supported by security, then….

The value of the ISO should be expressed as a KPI or set of KPIs that cleary explain how the confidence it generates helps us understand (and then reduce) our risk.

If risk is a probability issue,  ISO adoption helps generate confidence in our predictive analytics.  The dollar value the ISO generates (the ultimate KPI) is part of the cost of being able to make wise risk decisions.

So what is that (making wise risk decisions) worth to you?

SOME CONCLUDING THOUGHTS

First, it occurs to me that this is a real shame.  In a sense, an inability to generate a quantitative value statement for ISO use is simply more witch-doctory (“use it because we, the wise men of the tribe say you should”).  In some future version, the ISO should include some mechanism for measuring and expressing the worth of adoption to the organization (a better reason to use the ISO than “because we said so”).

Second, It should be noted that of Jack Jones’ 3 true value statements from which all metrics/KPIs should point to - we’re only talking about one of those value statements - the ability to reduce risk.  Using the ISO in an organization most certainly could create operational efficiencies (help us do more with less) - but the ISO isn’t a standard that creates operational efficiencies as a primary goal, nor does it give implicit direction on how to create operational efficincies.    The ISO folks do, however, play fast and loose with the idea of “risk” and “risk management” so it’s within this context that I interpreted our conversation.

Finally if you’re going to hire someone to help you with ISO adoption in your organization, the deliverables you ask for in your RFP/SOW/what-have-you should include quantitative (probability) statments about risk reduction and the creation of operational efficiencies.  If the firms answering can’t tell you what value their work will be to your company, then drop me a note and I’ll gladly point you to some friends of RMI’s that know FAIR & all our Risk Management frameworks and also do great ISO work.

Jon Swartz, USA Today’s award-winning technology reporter and Pulitzer Prize nominee, is Gary’s guest on the 25th episode of The Silver Bullet Security Podcast. They discuss Jon’s new book, Zero Day Threat: The Shocking Truth of How Banks and Credit Bureaus Help Cyber Crooks Steal Your Money and Identity and the research that went into writing it. Gary and Jon also cover how cybercrime is driven by capitalist principals, why the general public’s attitude is so lax about software security, and how, even though it’s hard to get an accurate count of identity theft instances, they tend to show a sharp upward trend. Jon ends the episode by disclosing his secret dream career.

(Apologies for the below-average sound quality on this episode.)

• Zero Day Threat
• Jon’s USA Today articles
• Three recent articles:
• Microsoft still seen with a win
• Online crime’s impact spreads
• AOL, News Corp. join battle over Yahoo
• The New Face of Cybercrime trailer

In the fourth episode of the Silver Bullet Security Podcast, Gary’s guest is Dana Epp, CEO and founder of Scorpion Software. Dana also runs a popular software security blog and is a jazz trumpeter. On this show, Dana and Gary talk about past programming disasters (”code lives forever”), the security implications of systems with ever-increasing complexity, suggestions for new developers interested in learning about software security, regulation’s role in information security, and Miles Davis.

• SilverStr’s blog - Dana’s blog
• It’s Pat!
• RemoteAccess BBS
• The 5 Rules of the Regulatory Process
• Chris Botti
• SC-L List
• Bitches Brew
• Subscribe to IEEE Security & Privacy