Sounds like the beginning of a joke, right?  So these two guys walk into a bar…

“The” Bruce Schneier and Marcus Ranum have an article up on TechTarget/Information Security Magazine called, creatively enough, “Bruce Schenier, Marcus Ranum debate risk management“.

Unfortunately, to get to the article, you’ll have to either already be a subscriber to IT Security, a subscriber to TechTarget, or go through the 20 minute process of signing up by giving TechTarget all sorts of “market information” about how you’re really Brandon Walsh, CSO of “The Peach Pit” Industries in Beverly Hills, CA 90210 (phone 714-867-5309).

For those of you who are already a TechTarget person, the link is above.  For those who aren’t, or those who just don’t have the time, I’ll summarize.  The “debate” is kind of awkward because both authors seem come to the same conclusion:

Risk Management, it’s something our profession should do, something humans do naturally, it’s necessary in business, but gosh - we don’t have enough data.

I’m not a cryptographer.  I don’t *nearly* have the insight on privacy and politics that Bruce has.  I’m not deep in IP communications.  I haven’t got a proven track record of innovation in IP Security products like Marcus has.  But here’s the thing, I hope you’ll never hear me pretend that I have the skill set to speak authoritatively on those subjects.  Heck, I wouldn’t claim to be a “risk” expert because I have a some insight into my shortcomings and what is needed to tackle such a complex problem.  But such a tepid article on something that (at least I think) is so important kind of, well, confuses me.

Why is it such a boring article?  I’m not sure.  Maybe because they’re just two guys who would rather debate the merits of specific controls or control activities (after all, their penetration testing debate was a huge success), but there’s no new information in the “debate”.  It’s the same old “insurance companies know risk because they have scads of data and we don’t have that” complaint. You know what?  I’m tired of hearing that line, so let’s talk about it.

HOW DO YOU KNOW WE DON’T HAVE THE AMOUNT OF DATA WE NEED TO DO RISK MANAGEMENT WELL?

Not particularly picking on Marcus, but in the article he uses the common complaint, “We lack the data to do risk management well.”  This mantra is repeated to the point where I’m blase’ about it.  But for some reason, this sentence really jumped out at me this time for two reasons.  It made me ask:

1.)  How do you know we don’t have the proper amount of data?

2.)  Can we even define “well” (i.e. what “good” risk management is) yet?

I really don’t know that the industry, especially concerning IT risk, is mature enough to really conclude that we don’t know (in the case of the former), nor that we can define (latter), conclusively.

PLAYING THE CONTRARIAN

Just because I’m feeling kind of zany this morning, let me suggest something.  Maybe there actually is lots of evidence out there for us to use.  Maybe:

1.)  It’s just that we don’t have particularly good models that provide context.

2.)  When that evidence isn’t an obvious phenomena that lends itself to easy measurement, we throw our hands up in disgust and fall back on “lack of data”, “can’t quantify risk”, “best practices work just fine” or any other number of arguments, no, excuses we use to justify our inability to be precise about the past (more or less the present or future - apologies to Niels Bohr).

IT’S IN THE WAY THAT YOU USE IT

Now I actually am happy to acknowledge that we don’t have enough data to be precise.  You, me, even smart guys like Marcus and Bruce - we’ll never be able to “engineer” risk management.  But you know what?  Neither can Insurance companies.  Sure, there are plenty of places where they have enough data to apply a traditional frequentist approach to risk valuations.   But there are plenty of times Insurers actually insure and they don’t have centuries or decades of data.  There are plenty of times when they rely on the “estimates” of subject matter experts.  There are many times they have enough information to be accurate rather than precise, and that’s good enough for them.

For that matter, it’s worth noting that there are plenty of scientific disciplines that have to deal in imprecise prior information, or evidence that’s fraught with uncertainty (what Ranum calls “squishy”, and what I’ve heard real honest to goodness physicists call “noisy”).  Unfortunately, we’re going to be like them.  Until we can read minds and predict the future, there will always be uncertainty in our measurements and posterior conclusions.  The trick is in how you deal with it and express it.  And while I really don’t know how much time Marcus or Bruce have really spent in the deep end on the subject of risk and its management - I have seen people doing brilliant things around risk (though they just aren’t mainstream).  Whether the tools are Bayesian methods, Monte Carlo engines, reductionist models of complex problems, there are risk analysts trying to deal with the problem.  These analysts are applying scientific method(s) and developing reasonable approaches to a very complex problem.  There are people trying, and our body of knowledge is growing, growing well beyond “gee, I haven’t got an obvious solution so I’ll blame it on lack of data”.  Heck, I’ve seen readers of this blog suggest Douglas Hubbard’s book in other security forums!*

I’VE GOT YOUR DATA RIGHT HERE…

But we don’t have enough data?  I have to ask, how much more do we need?  I mean crikey, JPMC just visited our ISSA chapter claiming, like, a bajillion events an hour.  There’s not one, but several companies out there that will want to tell you about how they have deep “insight” into the attacker community.  The boundaries of IT Risk losses are pretty well established by events that happen to public companies.  We have pretty mature testing/assessment tools and methodologies now that help us test our ability to resist the force an attacker can apply to us.  So what part of the Threat Landscape, Asset (Controls) Landscape, or Loss Magnitude landscape is too incomplete (and what are you doing to find the information you need)?

SO WHY DO WE FAIL?

Which brings me to a final, somewhat depressing conclusion.  Maybe there’s data, and maybe we’re starting to see the means to use it.  But in the end I do have to agree with Marcus that the vast majority of the infosec world *is* doing a really, really bad job with regards to “risk” and “risk management”.  The majority of people I know consider GRC to be a cruel, expensive joke.  Risk Assessment Methodologies tend to be built on the faulty premise that if we create a repeatable process, our measurements and conclusions will magically become accurate and wise.  Risk models tend to be factors loosely measured by ordinal scales and then somehow “multiplied” together to create a relatively meaningless qualitative value.  The State of the Union here is not good.  But after reading such a superficial treatment of an important and complex subject, I am left wondering if Bruce and Marcus were the right people to write about risk management in a mainstream publication.  As Inspector Callahan says, “A man’s got to know his limitations.”

===============================

* Speaking of which, if you want to do one cost effective thing to address your uncertainty - go find Douglas Hubbard’s book. It’s even got a nice recommendation from Peter Tippett.  The book is called “How To Measure Anything” - the title sounds rather hyperbolic, but there are good techniques in it we can use to identify useful information and refine our ability to frame that qualitative information into quantitative values. The key is how Hubbard has you deal with your uncertainty.  For those of you who are more scientific minded and want to dig deep into the subject, I have on good authority that E.T. Jaynes “Probability Theory, The Logic of Science” is a rather under appreciated work.

Almost as bad as the Sprint PCS password reset fiasco that made the news in April, here is the Yahoo Mail password reset screen:

As you can see, you need to know the user’s birthday, country of residence, and postal code. Not difficult information to dig up in Palin’s case, as shown here. After you enter this information correctly, you are asked to type in the alternate e-mail address that’s associated with the account. But they give you hints — so if your alternate e-mail was sarah@alaska.gov, they would show you s****@a*****.gov.

Assuming you guess the alternate e-mail correctly, Yahoo mails a password reset link to that address. So it’s likely that the attacker may have also had to gain access to her alternate e-mail account. Either that, or they exploited a vulnerability in the Yahoo password reset mechanism itself, which seems less likely but not implausible.

So Yahoo itself probably didn’t get hacked, per se, even though there will probably be a lot of FUD in the media about that.

FP: And what about drilling? Republican presidential candidate Sen. John McCain, his running mate Gov. Sarah Palin, and President George W. Bush are implying that lifting environmental restrictions on drilling is the way to promote energy independence.

TF: Well, I think it’s patent nonsense. No one believes that somehow offshore, there’s enough oil in any near term and even the long term to provide us oil independence. It’s the wrong approach because in a world that’s hot, flat, and crowded, fossil fuels—and particularly crude oil—are going to be expensive and exhausting. Therefore the focus should be on the next great global industry: clean energy technology. When I hear McCain pounding the table for “drill, drill, drill,” it reminds me of someone pounding the table for IBM Selectric typewriters on the eve of the IT revolution.

I’m not against offshore drilling, by the way, because I believe the technology and the safety has improved far beyond where it was back in the 70s, 80s, and 90s, even. What I’m against is making it the centerpiece of our energy policy. If all McCain said was, “Let’s drill, but let’s also throw everything into innovating the next generation of clean-energy technologies,” I’d say, “You’ve got it exactly right, pal.”

Here’s an interesting post on High Earth Orbit about the usage and promotion of open source software for defense contracts. As a developer of open source tools, Andrew Turner of course brings up some “pros” for the government to push open source, but it’s the “cons” that are really interesting. A big “con” – the US government having something called “sovereign immunity” which apparently means something like it can’t be sued unless it consents to be sued. Hunh – the Republic of ScienceLogic-Land? Closing the loop here, a federal appeals court just boosted open-source software licenses by saying that any infringements can now get more severe remedies under copyright law (instead of contract law); here’s the case, Jacobsen v Katzer. But apparently not if it’s the US government?? Who knows more?

Does Linus Torvalds hate everyone except for developers? You have to check out this article on an email exchange he had with Network World this week, talking about how fed up he is with the “security circus”. Over the course of the exchange and some other comments from last month, he manages to blast security folk, OpenBSD (on security) in particular, vendors and PR people (of course). In the midst of the barrage of colorful language, it’s difficult to really get his point – which if you can dig it out, ends up being surprisingly sensible.

Sharon Taylor, Chief Architect of ITIL V3, recently wrote that with the release of the latest version of ITIL, BSM is now an ‘ITIL best practice.’ You say potato… “The distinction between IT and the business has blurred, and the language of IT has been replaced with the language of the business.”

After my “used car salesman of NAC” series I was going to give Ray and the gang a break.  But the depths they sink to just never cease to amaze me! Today I received a Google alert on NAC with a link to a press release announcing the NAC used car sales guys continuing to deliver best in class security management solutions, yada, yada, yada.  The basis for this claim was that “SC Magazine awarded ForeScout’s CounterACT a four-out-of-five star rating, lauding the product’s ability to “function like a firewall, an IPS and a NAC device all rolled into one”.  They wrapped some customer quote (that had nothing to do with the SC magazine story) and voila!, can they put you in this car today?

So why do I call this out? No, no sour grapes here.  Actually StillSecure Safe Access received the same 4 out of 5 stars and when we dig into the rating here are some interesting facts:

In actuality, our friends the used car salesmen only received a 2 star rating in ease of use, a 2 star rating in documentation and a 3 star rating in support.  In contrast StillSecure Safe Access received 5 stars across the board, except for a 4 star grade in documentation.  How both products finish up with a 4 star rating overall based upon this is frankly baffling to me. I think it has more to do with the reviewer not wanting to spank any of the products too badly.  I have already asked for a clarification and will let you know what I find out.  But being a slick marketing machine, I thought it the height of chutzpah that they would put out a release around this, considering the best buy and editors choice were two different products.  But I guess that is why they did not have a quote or a link to the actual review.  The review starts out with this memorable quote, “The ForeScout CounterACT was the device which took the most time to install and configure.”  Later on the reviewers had this to say, “The second part of the configuration was far more difficult. The initial screens for the GUI made us feel lost and we immediately began looking for the documentation CD.”  Now does that sound like a review to be touting?  Only those master car salesman would seek to put out a press release trumpeting the results of this review.  They are counting by wrapping enough other quotes (and frankly who knows about those) around it, no one will bother to dig into the facts here. Hey, thats what you guys pay me for, telling it like it is!

After my ???used car salesman of NAC??? series I was going to give Ray and the gang a break.  But the depths they sink to just never cease to amaze me! Today I received a Google alert on NAC with a link to a press release announcing the NAC used car sales guys continuing to deliver best in class security management solutions, yada, yada, yada.  The basis for this claim was that ???SC Magazine awarded ForeScout???s CounterACT a four-out-of-five star rating, lauding the product???s ability to ???function like a firewall, an IPS and a NAC device all rolled into one???.  They wrapped some customer quote (that had nothing to do with the SC magazine story) and voila!, can they put you in this car today?

So why do I call this out? No, no sour grapes here.  Actually StillSecure Safe Access received the same 4 out of 5 stars and when we dig into the rating here are some interesting facts:

In actuality, our friends the used car salesmen only received a 2 star rating in ease of use, a 2 star rating in documentation and a 3 star rating in support.  In contrast StillSecure Safe Access received 5 stars across the board, except for a 4 star grade in documentation.  How both products finish up with a 4 star rating overall based upon this is frankly baffling to me. I think it has more to do with the reviewer not wanting to spank any of the products too badly.  I have already asked for a clarification and will let you know what I find out.  But being a slick marketing machine, I thought it the height of chutzpah that they would put out a release around this, considering the best buy and editors choice were two different products.  But I guess that is why they did not have a quote or a link to the actual review.  The review starts out with this memorable quote, ???The ForeScout CounterACT was the device which took the most time to install and configure.???  Later on the reviewers had this to say, ???The second part of the configuration was far more difficult. The initial screens for the GUI made us feel lost and we immediately began looking for the documentation CD.???  Now does that sound like a review to be touting?  Only those master car salesman would seek to put out a press release trumpeting the results of this review.  They are counting by wrapping enough other quotes (and frankly who knows about those) around it, no one will bother to dig into the facts here. Hey, thats what you guys pay me for, telling it like it is!