At the centerof the Smithsonian Institution's National Museum of Natural History's gleaming new Sant Ocean Hall lies a preserved giant female squid -- the arresting, spineless star among the vibrant exhibition's animal specimens. Tentacles menacingly outstretched and seemingly frozen in time, the 24-foot squid embodies humans' fascination with the briny deep. But this squid also symbolizes something else: an ongoing experiment in the chemistry of preservation, without which the Smithsonian's new exhibition would not have been possible.

Also note the terrorism tie-in:

To create the exhibit, the Smithsonian had to work around post-9/11 rules restricting flammable materials, while maximizing the lifelike appearance of the squid for public display. They turned not to formalin or ethanol, but to a new fluorinated chemical called Novec, developed by 3M.

If we give up our preserved giant squids, then surely the terrorists have won.

One easy, inexpensive way to actually increase your effectiveness in 2009 is to, right now, make a quick review your risk management processes.  As you take a look at how you’re using risk in your organization, I’d ask you to make sure that those processes are providing value for the energy you’re spending.  If they’re not - if you’re not successfully using risk within security and with the other lines of business that you serve - then I’d like to invite you to  come take advantage of RMI’s public training session for 2008, held in Columbus Ohio on December 10-12.  >A brochure is here<.

For three days and $1,995 - you’ll get real answers to many of the commonly voiced frustrations RMI hears concerning risk & risk management.  Answers around measurement, application, communicating risk to other lines of business, heck, basic answers as to what risk is and how to get consistent, defensible values that actually mean something.

Not to mention - Strengthening your Risk Management processes increases your ability to manage risk, which reduces the amount of risk you actually face.

NEW TO THE PUBLIC STUFF!

I’m personally excited because this is the first time that our public training we’ll feature measurement “calibration” exercises and include excel tools to take home and use for quantitative FAIR analysis.  These are benefits we’ve only previously reserved for private client workshops.

I know that FAIR can help you and your organization, but as the sales guys always say, “don’t take my word for it”.  Here’s something we recently received (unsolicited) from the CSO of one of the 10 largest banks in the US, who has had several of his analysts receive this same basic training:

I would like to also add my deep appreciation for what FAIR and RMI has brought to (us) and how we go about the business of risk analysis. We have had some great conversations around risk with the lines of business that have ended very favorably for us.

More information can be found on RMI’s website here:  http://www.riskmanagementinsight.com/12_2008_training.html

Thanks.

Oh and tomorrow, we’ll talk a little bit about quantitative and qualitative risk.

But that credit card weirdness is nothing compared to the one I’m about to describe.

Before we do that, let’s take a moment to discuss the design principle of failing securely. The general idea is that if a security mechanism fails, it should fail closed. If your firewall crashes, it should block all traffic, not allow all the packets through. If the power source to your card key system is interrupted, it shouldn’t unlock all the doors. If the connection between your application server and your LDAP directory is severed, subsequent authentication requests should be rejected, not approved. This is not rocket science.

So back to credit cards. I had a conversation last night with an old friend who related a bizarre situation they had encountered during the QA process for one of their web applications. One of their tests involved repeatedly attempting a credit card transaction using a canceled/expired American Express card. Here’s what they saw in their logs, paraphrased by me:

Attempt 1: Denied
Attempt 2: Denied
Attempt 3: Denied
 .
 .
 .
Attempt 49: Denied
Attempt 50: Denied
Attempt 51: Approved

What the…? Approved? That can’t be right. So they ran the test again. Every time, after multiple consecutive rejected attempts, the transaction would inexplicably go through. The threshold wasn’t always 50, but the general pattern was consistent — keep trying and eventually it’ll work. Clearly, this had to be a bug in the code, but a deep-dive into the guts of the application turned up nothing. The application security group got American Express on the phone to see if they had any insight on this odd behavior. The answer? They didn’t concede the failure was on their end, despite log data showing the successful authorization codes.

My gut instinct would be that the application requesting the transactions wasn’t failing securely (e.g. network connection to AmEx timed out, so just approve the transaction). But that explanation wouldn’t account for authorization codes coming back.

So what in the world is going on here? Why would the system behave this way? Is it by design? I can’t think of a single legitimate use case for failing open like this. If this is actually a design decision by the credit card companies, I have no doubt that someone in our audience knows the rest of the story.

Sounds like the beginning of a joke, right?  So these two guys walk into a bar…

“The” Bruce Schneier and Marcus Ranum have an article up on TechTarget/Information Security Magazine called, creatively enough, “Bruce Schenier, Marcus Ranum debate risk management“.

Unfortunately, to get to the article, you’ll have to either already be a subscriber to IT Security, a subscriber to TechTarget, or go through the 20 minute process of signing up by giving TechTarget all sorts of “market information” about how you’re really Brandon Walsh, CSO of “The Peach Pit” Industries in Beverly Hills, CA 90210 (phone 714-867-5309).

For those of you who are already a TechTarget person, the link is above.  For those who aren’t, or those who just don’t have the time, I’ll summarize.  The “debate” is kind of awkward because both authors seem come to the same conclusion:

Risk Management, it’s something our profession should do, something humans do naturally, it’s necessary in business, but gosh - we don’t have enough data.

I’m not a cryptographer.  I don’t *nearly* have the insight on privacy and politics that Bruce has.  I’m not deep in IP communications.  I haven’t got a proven track record of innovation in IP Security products like Marcus has.  But here’s the thing, I hope you’ll never hear me pretend that I have the skill set to speak authoritatively on those subjects.  Heck, I wouldn’t claim to be a “risk” expert because I have a some insight into my shortcomings and what is needed to tackle such a complex problem.  But such a tepid article on something that (at least I think) is so important kind of, well, confuses me.

Why is it such a boring article?  I’m not sure.  Maybe because they’re just two guys who would rather debate the merits of specific controls or control activities (after all, their penetration testing debate was a huge success), but there’s no new information in the “debate”.  It’s the same old “insurance companies know risk because they have scads of data and we don’t have that” complaint. You know what?  I’m tired of hearing that line, so let’s talk about it.

HOW DO YOU KNOW WE DON’T HAVE THE AMOUNT OF DATA WE NEED TO DO RISK MANAGEMENT WELL?

Not particularly picking on Marcus, but in the article he uses the common complaint, “We lack the data to do risk management well.”  This mantra is repeated to the point where I’m blase’ about it.  But for some reason, this sentence really jumped out at me this time for two reasons.  It made me ask:

1.)  How do you know we don’t have the proper amount of data?

2.)  Can we even define “well” (i.e. what “good” risk management is) yet?

I really don’t know that the industry, especially concerning IT risk, is mature enough to really conclude that we don’t know (in the case of the former), nor that we can define (latter), conclusively.

PLAYING THE CONTRARIAN

Just because I’m feeling kind of zany this morning, let me suggest something.  Maybe there actually is lots of evidence out there for us to use.  Maybe:

1.)  It’s just that we don’t have particularly good models that provide context.

2.)  When that evidence isn’t an obvious phenomena that lends itself to easy measurement, we throw our hands up in disgust and fall back on “lack of data”, “can’t quantify risk”, “best practices work just fine” or any other number of arguments, no, excuses we use to justify our inability to be precise about the past (more or less the present or future - apologies to Niels Bohr).

IT’S IN THE WAY THAT YOU USE IT

Now I actually am happy to acknowledge that we don’t have enough data to be precise.  You, me, even smart guys like Marcus and Bruce - we’ll never be able to “engineer” risk management.  But you know what?  Neither can Insurance companies.  Sure, there are plenty of places where they have enough data to apply a traditional frequentist approach to risk valuations.   But there are plenty of times Insurers actually insure and they don’t have centuries or decades of data.  There are plenty of times when they rely on the “estimates” of subject matter experts.  There are many times they have enough information to be accurate rather than precise, and that’s good enough for them.

For that matter, it’s worth noting that there are plenty of scientific disciplines that have to deal in imprecise prior information, or evidence that’s fraught with uncertainty (what Ranum calls “squishy”, and what I’ve heard real honest to goodness physicists call “noisy”).  Unfortunately, we’re going to be like them.  Until we can read minds and predict the future, there will always be uncertainty in our measurements and posterior conclusions.  The trick is in how you deal with it and express it.  And while I really don’t know how much time Marcus or Bruce have really spent in the deep end on the subject of risk and its management - I have seen people doing brilliant things around risk (though they just aren’t mainstream).  Whether the tools are Bayesian methods, Monte Carlo engines, reductionist models of complex problems, there are risk analysts trying to deal with the problem.  These analysts are applying scientific method(s) and developing reasonable approaches to a very complex problem.  There are people trying, and our body of knowledge is growing, growing well beyond “gee, I haven’t got an obvious solution so I’ll blame it on lack of data”.  Heck, I’ve seen readers of this blog suggest Douglas Hubbard’s book in other security forums!*

I’VE GOT YOUR DATA RIGHT HERE…

But we don’t have enough data?  I have to ask, how much more do we need?  I mean crikey, JPMC just visited our ISSA chapter claiming, like, a bajillion events an hour.  There’s not one, but several companies out there that will want to tell you about how they have deep “insight” into the attacker community.  The boundaries of IT Risk losses are pretty well established by events that happen to public companies.  We have pretty mature testing/assessment tools and methodologies now that help us test our ability to resist the force an attacker can apply to us.  So what part of the Threat Landscape, Asset (Controls) Landscape, or Loss Magnitude landscape is too incomplete (and what are you doing to find the information you need)?

SO WHY DO WE FAIL?

Which brings me to a final, somewhat depressing conclusion.  Maybe there’s data, and maybe we’re starting to see the means to use it.  But in the end I do have to agree with Marcus that the vast majority of the infosec world *is* doing a really, really bad job with regards to “risk” and “risk management”.  The majority of people I know consider GRC to be a cruel, expensive joke.  Risk Assessment Methodologies tend to be built on the faulty premise that if we create a repeatable process, our measurements and conclusions will magically become accurate and wise.  Risk models tend to be factors loosely measured by ordinal scales and then somehow “multiplied” together to create a relatively meaningless qualitative value.  The State of the Union here is not good.  But after reading such a superficial treatment of an important and complex subject, I am left wondering if Bruce and Marcus were the right people to write about risk management in a mainstream publication.  As Inspector Callahan says, “A man’s got to know his limitations.”

===============================

* Speaking of which, if you want to do one cost effective thing to address your uncertainty - go find Douglas Hubbard’s book. It’s even got a nice recommendation from Peter Tippett.  The book is called “How To Measure Anything” - the title sounds rather hyperbolic, but there are good techniques in it we can use to identify useful information and refine our ability to frame that qualitative information into quantitative values. The key is how Hubbard has you deal with your uncertainty.  For those of you who are more scientific minded and want to dig deep into the subject, I have on good authority that E.T. Jaynes “Probability Theory, The Logic of Science” is a rather under appreciated work.

In any case, if you do pentests, think about all the RECENT cases where you break in to a major corporation through:

• a Solaris system with Internet-exposed telnet with a guessable password OR a telnet vulnerability (circa 1994!)
• an exposed VPN appliance with a manufacturer's administrator password
• a router with default "enable" password
• or, something else entirely - but something that rivals the above example in its unparalleled, unbelievable, abysmal, deep idiocy.

Indeed, many of my pentesting friends still report plenty of such cases (one was also featured in the presentation mentioned above). Whenever I hear about it from a pentester, I always ask:

Do you think "somebody bad" had already passed through the hole you just discovered?

Maybe an hour ago, a day ago - or a year ago?!

I cannot see how the answer can be "no."

Even though pentesters usually don't focus on forensics (no time for this), it is not uncommon to notice "your predecessor's" intrusion traces while you break through systems, "plant flags", change screen backgrounds [for the admins to notice that you've been there...], etc.

Let's think what this situation really means? Here are the choices I see:

1. Nobody discovered the hole - a law of large  numbers (aka "dumb luck") have "shielded" the company from an incident. Yes, Virginia, dumb luck IS a security strategy for some companies... AND it works for them.
2. It was discovered, but not used/abused by the attacker - maybe he was busy hacking other systems, or saved this for later and never came back due to his ADD. Congratulation, you win! The immense power of dumb luck wrapped you in a protective "security" blanket ... again :-)
3. It was discovered; the attacker went in, looked around and compromised a few others systems, but found nothing of interest (no low hanging fruits)  - and he was not a bot herder. Again, you win. Next time you are in Vegas, bet on "00."
4. It was discovered; the attacker went in and deployed a bot on "your" system - given how many botnets are there, this situation is clearly acceptable to many organizations. In this case, dumb luck strategy, apparently, still work: so they use your box to spam and phish somebody else ... big deal!
5. It was discovered; the attacker went in and stole all your credit card information (it is now for sale) - even in this case, the user of "the dumb luck strategy" still "wins" (in some perverse sense)! Unless and until the stolen information IS tracked back to you OR a friendly neighborhood PCI auditor come and jams a broomstick up your ..., you can still continue to be stupid at your leisure and ignore basic security practices.
6. It was discovered; the attacker went in and stole your CEO's Inbox, including the email related to his affair (it is now on CNN) - now, in this case, you lose AND it is time to stop being stupid! Welcome to the "0wned world." Time to launch (relaunch?) your security program and get serious.

What does this teach us about RISK? The lesson here is important:

• For a security professional, an Internet-exposed system with "root/root" is an obvious HUGE risk!
• For your boss's boss's boss, it is NOT!

This is exactly why I think that the most critical problem in security today is METRICS. Metrics that a) work AND mean something to decision makers and b) can be clearly communicated to said decision makers [BTW, a) and b) are two separate problems.] Metrics that cover not only threats and vulnerabilities we face, but also the effectiveness of security countermeasures we deploy. Metrics you can act on - and ones your boss (and his boss) will act on. Metrics that lead to correct decisions about which risks to accept, which to  mitigate (all while knowing with what efficiency such mitigation occurs) and which to transfer.

Until that time, the dreaded "C-word" (compliance) will trump "the other C-word" (common sense) as a driver for security ... and we will continue to live in the "0wned world."

Possibly related posts:

• Risk vs Risk