1. We don’t believe that the goal of Quantitative Risk Analysis is to be precise.  We believe the goal is to be accurate. Subtle but important difference.
2. FAIR can be used both Quantitatively and Qualitatively.   The decision on which method to be used depends on various factors that Steve lays out nicely in the article there.
3. We believe that Risk Management is more than looking at specific vulnerabilities, their likelihood and impact.  It must encompass all aspects of the organizations ability to effect the probable frequency and magnitude of loss on an aggregate level, not just within the context of a discreet technical or policy issue.

That last point is important.  And it’s related to my post today.

WHAT DO YOU MANAGE TOWARDS?
This blog is blessed to have some very smart people be part of it.  There are security managers from all sorts of industries that read and comment and contribute.   And so today’s blog is more of an open-ended question for you all.  It’s a question that, if I have a comfortable relationship with the organization I like to first ask the senior manager, and then subsequently ask the direct reports.

When you think about it, Sales & Marketing managers have goals they manage towards.  CFO’s have goals that they manage towards.  COO’s have goals and measurement that they manage towards (cost management, production, etc…).  So what does the CSO manage towards?  I’m guessing if we took a national poll, we’d get all sorts of very nice sounding answers to that question.  I thought I’d list some of the answers I’ve heard and talk about them with you today.

1.)  Being Secure or “Managing to Security”

Generally, this concept of being secure is the most common answer.  And when I’m given that answer, it generally means that management focuses on Vulnerability Management, Patch Management, and to some degree, log analysis from various sources.  These are very basic core security functions, and the  belief is that if we do these well, we will be “secure”.  Ok, well… what does this “secure” mean, and how can we talk to management about whether we are meeting this goal?   If you examine that question, you actually find out what a “Being Secure” organization is really managing towards, another answer I hear often:

2.)  Being Incident-Free or “Managing to Perfection”

Security Person:  “Alex, our goal is not to have any incidents.”  Alex:  “Good luck with that.”

OK, that’s not what I really say, but here’s the problem I see with this common answer and the one above both of these common answers:  How do you know if you’re good or just lucky?

Well, are you, punk? (youtube link)

In my six years of working with a Penetration Testing team, nobody ever really “passed” with a perfect score*.  Some did better than others, some folks looked really, really good - but the degree  of good/bad was really more dependent on scope than the actual state of controls or the ability of the team to overcome them.  That is to say, when pressed, the mature security professional must admit that, given a strong, capable threat community -  there is no secure.   And therefore any state of “incidentlessness” deals with some combination of amount of control strength, and some lack of attacks (frequency!) by someone with enough skills and resources to overcome those controls.  If that last sentence sounds very FAIR-Like, that’s because it is.  If FAIR really accounts for those things that create Risk, then Managing to security or lack of incident means that you’re primarily concerned with FAIR Vulnerability, and ignoring other critical aspects of risk (like frequency of attacks, controls that reduce the probable impact of an event due to an ability to respond well to external stakeholders, etc…).

3.) Being Compliant or “Managing to Compliance” (External Compliance Pressures)

Because that’s what business buy, right?  They buy compliance!   Or so I’m told.  So let’s say that you go out and actually twist senior managements arm to get them to cough up enough dough so that you can be as compliant as Large Accounting Firm says you need to be.  Good on you!

But what I always wonder is, what happens when you want to manage something beyond compliance?  What happens when the checklist you’re managing towards is run by a bureaucracy that can’t keep up with a changing threat landscape?   For many people, the answer is “GOTO 1″ and try to sell upper management using FUD (hey, it used to work, maybe it’ll work again).  An alternative is to get to the next step:

4.)  Being Measured or “Managing to Metrics”

Say what you will, but “quants” have one thing right.  What gets measured gets done.  And a few mature organizations have spent a ton of time and effort on being able to create dashboards of KPI’s that attempt to measure security.  Problem is, that we don’t know if a 98% on patch levels is good or bad or just right.  We don’t know what value, if any, does creating metrics around the number and severity of vulnerabilities found in a monthly scan actually have.  So we’ve come up with this thing called “GRC” that’s supposed to make sense of those things we can measure empirically and help you find out if/when you’ve fixed them. And while GRC tools can tell you some good information about systems out of compliance, they tend to give you fantastic information like how your “risk = 57“.

Wha….?

Risk = 57 means very little to someone who doesn’t spend their life in the machinations of the GRC indicies.  So again, measurement without a (good) model still falls down when faced with that ultimate business decision.  Or, as Shurdlu so eloquently puts it in her post on GRC:

“By contract, risk is personal.  It’s variable as hell.  It “governs” what you spend your money on, and therefore, with or without a dashboard, your CEO is already doing risk assessment every time she decides what your security budget is going to be.  Will you really be able to change her mind by showing her the dashboard and saying, “But—but—the needle is pointing to RED!” when you’re sitting there with your line items in your fiscal shopping cart? “

5.)  Using Risk or “Risk Management”

Which brings us to my favorite, using risk (as defined as the probable frequency & probable magnitude of loss event(s)) as a means to manage.  Now many industry veterans will tell you how jaded we all are on the term “Risk Management”.  And we have every right to be, as Risk Management has been horribly abused by vendors, committees and standards bodies alike.

These days, the term has been narrowly defined to mean an extension of vulnerability management.   This is small, small thinking, IMHO.  To me, Risk Management isn’t the management of individual issues deemed as “risky” as much as it is measuring (see 4) our ability to make decisions through the lens of risk.  Maybe I should start saying “Risk-Based Management” instead of “Risk Management”.

This Risk-Based Management approach provides meaning to metrics. We can know what we’re measuring and why we care about it.  And why we care about it needs to match what management cares about.  If your approach to Risk Management results in some metric or KPI that non-IT (or non-security) management doesn’t understand or speak to them in an evident language, it’s time to find a new model.  This is why “Quants will win” and where risk = 57 is wrong.  Risk, expressed as “expect a once in 5 year chance to lose $875,000 if we don’t spend $90,000 now” actually gives executives something beyond an arbitrary ordinal number or color to work with.  And what’s interesting is, if your model does the right things in getting you to that expression - then metrics and KPIs - those “why/when/where” questions we have a tough time answering about metrics - they become easier to discover.

DISPROVING RISK MANAGEMENT

As a side note, originally I was going to write today a completely different post on how we can disprove whether or not OCTAVE or 800-30 or ISO 27001 risk management efforts are really “Risk Management” - and one significant point was “Does your non-IT management really care about the deliverable?”   This thought came to me after seeing a few too many emails into the ISO27001 mailing list asking “How can I get management to fund ISO 27001 certification?”  Of course, the value of implementing the ISMS and the value of certification are two separate business propositions, but if you can’t sell the first, then are those efforts really good risk management?  You know, the kind of effort that we can use to make meaningful reporting?

=============================

* I can tell you that at times we were asked to test products out for clients before they made a significant investment.  One biometric device stands out in memory as not being “hacked” in the time alloted for the engagement by a defense contractor.  After it passed the “Gummi Finger” test - we were going to try using a recently severed finger, but oddly enough nobody on the team volunteered their digit for the sake of security.  Bunch of slackers.

Those so-called Afghan National Auxiliary Police (ANAP), all formerly in the service of local warlords, had received two months of training by Dutch and American soldiers and were now the first line of defense against the Taliban.

Arming tribesmen was a risky idea. True, this sort of tribal initiative had been effective in Iraq. But NATO commanders feared that Afghan loyalties to their warlords ran too deep. NATO was “arming people who were not necessarily in line with the [Afghan] government,” U.S. Brig. Gen. Robert Cone told Wired.com.

So, last month, NATO fired the auxiliary cops and scrapped the tribal strategy, leaving gaping holes in Afghanistan's defenses. The fix? Marines, of course, armed with fingerprint pads, iris scanners and electronic databases.

With these biometric tools, the Marines are planning to recruit new cops who have no ties to tribal warlords. “We know there are some shadow police and some militia-type police,” Lt. Col. Ray Hall, the Marine commander, said. “Once we go through the vetting process, we'll have everybody screened … so that problem should go away.”

That means scanning every new recruit's unique iris “eye prints,” logging their thumb prints and feeding it all into a growing, but still very spotty, national database linked to criminal and intelligence records. If a cop has any known warlord ties, he's disqualified from serving.

CIA teams used FBI biometrics while hunting for known Al Qaeda operatives in Afghanistan in 2001, and since then, the military has gathered data on almost every Afghan it comes in regular contact with.

There's one more problem. Not all the military databases can talk to one another. “We haven't standardized,” said Larry Schneider, a Northrop Grumman VP who last year was working on collapsing many biometrics systems into just one.

Until everyone is looking at the same data, seditious Afghan cops will probably keep falling through the cracks.

Here's the story directly from the Chaos Computer Club (in German), and its Engligh-language guide to lifting and using fingerprints. And me on biometrics from 10 years ago.

In fact, I found the article to be rather upsetting. Not because of the article’s thesis that strong authentication through a national ID program would not necessarily pose a threat to privacy; but rather, because of their naive (and irresponsible) handling of the realities of the biometric authentication challenge. They gloss over the real security challenges with creating a national biometric infrastructure. Here are the two quotes that are most misleading:

• “Confusing privacy with anonymity has delayed implementation of robust, virtually tamper-proof biometric authentication to replace paper-based forms of ID that neither assure privacy nor reliably prove identity.”
• “This emerging technology makes it virtually impossible to assume someone else’s unique identity.”

The problem that the authors are glossing over is that no such technology exists today, and it is unlikely to ever exist. Now, to be fair, I am assuming that a critical success factor for any national biometric program, as described, would be that the authentication devices have to be available, and usable, anyplace paper-based IDs can be used today. This of course implies that the authenticator must be an inexpensive, commodity device, easy to purchase, maintain, and operate. Such a device would have to be even more ubiquitous than the electronic credit card machine.

The problem is that the authenticator itself may be in the possession of the attacker (Perhaps after you authenticate your legitimate purchase the clerk desires to use your identity herself…). In the history of security controls, when the attacker has unsupervised at-will physical access, the attacker wins. Here are a few examples:

• Defeated copy protection on DVDs ( more & more info)
• Cold Boot Crypto Attack on hard disk encryption (more info)
• MiFare RFID Cards (more info)
• Skimming devices attached to ATM machines to steal card and PIN data (more info)

Of course, all of these systems worked in the lab. But when a security system is widely deployed, it has to withstand an enormous amount of scrutiny, and minor flaws will be exploited. And of course, the greater the financial gain, the greater the time and energy attackers invest in trying to defeat the system. The authors of the article ignore these issues, idealistically assuming biometrics will just work.

Now, of course there are lots of examples where biometrics work very effectively. But I would propose that biometric authentication is most useful when the authentication device is physically secure and the authentication itself is supervised. The MiFare example above also demonstrates two other issues:

• The system chose not to implement a reviewed and standard cryptographic algorithm - always a bad idea
• MiFare was able to sell 1 billion cards and authenticators before the system failed

The cost of investing in a national biometric authentication program, and then having the security fail, is enormous. Can you imagine deploying a biometric authentication infrastructure to every bank, police car, restaurant, shop, etc. and then having video on YouTube of it being defeated ?

- Erik

BTW, Maybe the attacker doesn’t even need to tamper with the device -> ftp://ftp.ccc.de/pub/video/Fingerabdruck_Hack/fingerabdruck.mpg

Art of Information Security would love your feedback !

What do the Cold Boot Crypto Attack, DVD Players, and MiFare tell us about the Future of Biometrics?

Today I can announce new measures at Heathrow and other airports to ensure that a greater use of biometric technology speeds up the time it takes passengers to get through immigration control.