I’ve often read people who say that we (security, risk management) need to “think like the attacker”.  And when you read this sort of article, that usually alludes to trying to anticipate the tactics an attacker might use to mess with your C, I, or A.  Smart stuff, that, and very useful when architecting security solutions.  But as I was training some folks Monday, I was thinking in the back of my head about Threat Capability (TCap) in FAIR.  As you might know, we like to estimate the capability of a threat to apply some level of “force” against our assets.  This ability to apply force is a byproduct of the attacker’s skills and resources.  And thinking of how an attacker applies skills and resources, I came across another way we might “think” like an attacker.

Traditionally, I’ve thought of “skills” as being a byproduct of the toolset an attacker has.  This mindset probably stems from my time with Penetration Testing teams, where in the process of scoping the  PenTest I would ask our clients to select the level of effort that they wanted us to throw at them.  If a client chose “high” we’d throw every ‘spoit we had at them.  If they chose “low” we’d limit ourselves to a more commonly available toolset.

But while the resources part of TCap is time & materials (money) - the skills are really more than just the toolset.  Skills would include the ability of the attacker to be creative and innovative.    As an example of that innovation from those PenTesting days - when we got a “high” effort request, we would always try to couple that with some “social engineering”-type of attack, or some unique means of delivering an existing exploit.  Our creativity was not necessarily a byproduct of a unique exploit or tool we had, but the process by which we might deliver pre-existing or commonly available exploits.  I remember when we first got ahold of a handful of 32mb thumb drives (hey, 32mb was huge back then) and “dropped” a few in the lobby of a client’s retail space.  The keystroke loggers and phone-home script weren’t new, but using the thumb drive as delivery vehicle certainly was.

So I’ve started to really think about this concept of innovation, and how if “thinking like an attacker” means to be innovative, we ought to do the same.  I’ve been thinking of two main categories of innovation this morning.

INNOVATION

The first I’ll call Technology Innovation.  And by Technology Innovation, I mean some new, unique, “ahead of the curve” technology that an attacker can use against us.  The obvious example of which is a zero-day.  It’s that “high” tool set our PenTesters would use against the clients.  For security departments, this might be the latest security product designed to enhance our ability to P, D, and/or R.

Alternately, we can be creative in the way we deliver (manage) existing technology.  I think of this as Process Innovation.  It’s doing more with what we already have, just like the PenTest team would be creative in the delivery of an existing exploit.

Unfortunately for us - attackers have traditionally had quite a leg up on us in terms of Process Innovation.  It is much easier fro them to be creative, as they are free of political constraints and bureaucracy.  In contrast, when the security industry tries Process Innovation, the results are checklists and “standards”.  It’s committees and consensus.  An extreme example of which might be something like SABSA - a great work if you want to understand some very smart people’s comprehensive understanding of organizational security  - but the “adoption”of which will do very little to help you be innovative in P/D/R.

It’s worth noting that ultimately, this is one reason I don’t like regulatory compliance efforts - they simply serve to prove how mundane your security department is,  wasting valuable resources that could be spent on creating ways to be more effective.

PROCESS INNOVATION AS A SUBSTITUTE FOR TECHNOLOGY INNOVATION

As we come to the close of 2009, some surveys suggest that security spending isn’t horribly impacted yet by the economy (the latest from E&Y points to only 5% of their respondents getting budget cuts).  But if this is a protracted downturn, and because InfoSec is an operational expense, I would expect cash to become more and more difficult to keep.  And regardless if technology spends do slow, I believe it makes sense to think about Process Innovation because I see Process Innovation as a means to increase effectiveness without significant capital expenditures (effectiveness increases because our ability to manage risk has a direct correlation to the amount of risk we have).

The bad news is, of course, that great innovation is hard.  It is R & D.  Failure is usually a pre-requisite to success.

The good news is, our current state is so bad that many of us don’t need to come up with a whizbang new way of reducing software defects in the SDLC as innovation.  Simply inserting a risk analyst into the PMO’s processes might count as a big enough victory. Be cautioned, though,  that if we’re substituting the risk reductions provided by technology acquisition - Process Innovation might actually be even more “expensive” as it requires us to expend political capital.   But there are (forgive the term) innovative ways to spend this political capital.

For example, by taking a second now and figuring out the 3 things that the rest of the organization can do to make your life easier, when that “I need to reduce your budget” talk comes, you can be prepared to negotiate.  Get a political capital “loan” or “investment” from the C-Suite reducing your budget.  Something to the effect of: “I expected this, and am happy to give up my budget.  But if our tolerance for risk hasn’t changed, what I’d like to do is get you to personally back my office on three projects I’ve identified that can reduce our risk without requiring significant capital expenditure.”

One interesting outcome is that the insecure wireless WEP protocol will be banned…but not until June 2010. Says Ars Technica:

Although TJX has become the poster-child for consumer data theft over WiFi, it is (by far) not the only company to use insecure wireless technologies. Wireless security manufacturer AirDefense released a report in late 2007 saying that a quarter of the 4,748 retail access points it surveyed across the US had no security whatsoever, while another quarter only used WEP, “one of the weakest protocols for wireless data encryption.” Just under half (49 percent) of the surveyed hotspots used WiFi Protected Access (WPA) or WPA 2—much stronger encryption protocols than WEP.

If you’re wondering about what other impacts will have, you might want to read through the PCI site or sign up for the SecureWorks webcast on October 14th to learn more.

It does not matter if Chris is right or I am right. The outcome of IDS/IPS is all determined by random drift of market forces. There is no conspiracy to make IDS/IPS this way or that way. I would like to wrap up with a quote from Arthur Chandler : "We can tell when a technology has truly arrived when the new problems it gives rise  to approach in magnitude the problem it was designed to solve".

The name reveals some of its creepy appeal: Spykee = Spy Camera. I suppose the nanny you're trying to make sure isn't shaking your baby might be freaked out when it suddenly starts emitting Star Wars music, or such like. Made by Meccano under the Erector brand, its control software is Mac and Windows compatible.

I, for one, welcome our new Spykee overlords--on 15-Oct-2008 when it starts to ship generally.

Silicon Valley project finally gets underway: It's a still a pilot, small, with no promised outcome. And after all this time, a switch of partners, and new parameters, they've still mounted just 20 of 28 access points.

Now I’ve been “accused” of being a quant in the past (hi rybolov!) but in reality the only dogs I have in this fight are the model and the application of scientific method - and really, ethically speaking, I have to be tied to the latter while applying the former.

And I see a false dichotomy in this whole Quant vs. Qual thing.  We, as a profession, tend to create a political divide between the two which, if it even exists, I’d say is based more on our ignorance rather than our expertise.  After all, we are the profession that regularly multiplies across ordinal scales and uses wonderful models like R=VxTxI.   As someone  learning to deal in probabilities and rationalism, I have to recognize that this discussion is really just about the act of observation using different metrics of measurement.

But how we’re going about observing does not change the fact that there is measurement based on observation.  So if I’m working with you I can easily turn your qualitative scale into a quantitative one, and vice-versa.  Yes, Shrdlu, if we had the time, even your most seemingly Qual things could be Quant! (This flexible world view, btw, is an outcome of that new-fangled Bayesian thing).

COGNITIVE BIAS A-PLENTY

But back to what Rich is saying there about information security and risk - and he isn’t/won’t be the only one saying these sorts of things - we should try to understand what’s really going on rather than get caught up in the emotional hurricane.  Our profession suffers several forms of cognitive bias.  The nature of our jobs and what we do can cause us to be focused on the outcome and not the quality of the decision at the time it was made.  We want to bring in things from other professions that are useful, but at times we do view things outside our profession with false correlation to our own (unfortunately for those who write these sorts of articles, financial risk is completely different than operational risk).  We also have the tendency to focus on negative outcomes without acknowledging the positive outcomes (For example, I hear that Alan Greenspan’s new firm is up a couple of $billion in all this mess since he joined them, short sellers are doing quite well - must be because they have qualitative models or something -grin-).  The effect of these biases are compounded by the facts that proper correlation takes more work than we usually give it, and rational thought is not that easy when there’s a witch-hunt mentality.

What also floats in water? (link to Youtube)

WHAT SHOULD WE BE THINKING ABOUT?

So as you and I read opinions that seem to be the polar opposite of irrational exuberance (and there will be plenty between now and the election) we’ll have to ask ourselves, “what really failed here?”  At the risk (pun) of over-simplification:

• Was There an Error on the part of Probability Theory?

After all, Probability Science like all other fields of knowledge is always “advancing” as they say.  So perhaps probability theory is wrong somehow?

I’m personally disinclined to put the blame here, primarily because I would think that there would be evidence from other fields (like Quantum Mechanics) that something is amiss waaaaay before it hit a field like economics.

• Was There Error In The Model Used to Determine Risk?

Some people who understand real estate valuation and complex derivatives and financial risk want to put the blame here.  It’s a little too early to tell, but one thing is for sure - Financial risk is so different from operational risk I couldn’t begin to hazard an opinion on the subject.   But it would seem that this is really somewhere we might look.

• Was There Error In The  Scale Used (Quantitative vs. Qualitative)?

Honestly?  I find it extremely difficult to understand how this could be the source of financial ruin.

• Was There Error on the part of the Decision Maker?

What if all of the above were just fine, and the decision maker chose short term gain over long term stability?  What if this was (to simplify the matter greatly) a choice of “heads” over “tails” and the coin landed on tails?  What if the model represented the right risk (probability of negative outcome vs. positive outcome), but the complex derivative was sold to someone else who had poor “risk management” (ability to make a good decisions)?

Now I have no clue about complex derivatives, and I’m oversimplifying to be sure - chances are like most things, there are several problems that helped create the primary cause. But it seems to me that as we go into incident response mode for the economy, it’s more helpful to do so in a rational, logical manner.

OTHER THINGS WE MIGHT WANT TO CONSIDER

Consider the Source
Some authors (who I think tend to exploit outcome and hindsight bias,and then combine those with indirect ad hominem attacks in order to sell their books), are actually putting forth arguments against the use of analytics.  The source of this is a current epistemic debate between those who believe that only falsification is certain, and those who maintain that neither proof nor falsification are certain, there are only probabilities.    So before you go believing any “quadrants” of usefulness on faith - I encourage you to understand what is at the heart of the discussion.

We All Have to Live In The Real World
The sun will rise tomorrow, and someone will try to find the source of the problem and do a better job.  Now chances are, they’ll be doing it in a quantitative manner.  Chances are also that at some point their models will fail and we’ll need to build new ones.  And this will happen whether the field is cosmology, economics, meteorology, information security, or professional baseball.

WHAT ABOUT YOU, ALEX?

I’m far from certain and subject to change, but these days I lean towards Robin Hanson & MIchael Lewis w/regards to placing blame.

I say today is a good day for two reasons:  1.)  BT’s CSO Jill Knesek wrote an article called “Keys to establishing an end-to-end security strategy” which begs some discussion within context, and 2.)  Sara Peters on Twitter last night wanted to know why I thought “risk management” requires more than what most “best practices” around the subject suggest the effort requires.

WHAT SHOULD WE BE REFLECTING ABOUT?

Jill Knesek’s article gives us a rough outline of how to develop a security strategy.  It’s fairly high-level, Pragmatic CSO-ish type stuff.  It gives us a nice outline of

• Get a seat at the table
• Process
• People
• Technology

Nothing earth-shattering there.  But it is a very nice broad CISO-level taxonomy about what we have to reflect on.  The need to reflect is driven by something Jack told me long ago,

“The amount of risk we have is a function of the decisions we made and our ability to execute on them from some point in the past”.

As an Aside:  So Sarah if you’re reading, this quote does much to explain why I said I disagree with much of what our industry calls “risk management”.  We tend to define the process of risk management as essentially a tactical “issue whack-a-mole” exercise. Find the issue.  Analyze the “risk” around the issue.  Fix the issue.  Repeat. This hamster-wheel-of-pain, while sometimes an effective tool for the CISO, is incongruous with addressing root causes (the ability to match a tactical issue to the strategic shortcoming that created the issue is up to the expertise of the analyst or consultant).  It is only Kaizen without (good) Hansei, if you will.

Back to what Jill is writing - the sorts of things we should be reflecting about can be thought of in context of her outline.  Namely:

1. Once you have a seat at the table, what is the nature of that relationship?  Who are you reporting to and what are their concerns? What and how are you reporting and how might that be addressing their concerns?
2. What processes are in place?, How do you know that those are the processes that should be in place? If they are, what kind of job am I doing at those processes?
3. What is the quality of the skills and resources I have from a people perspective, and how do I know if they are adequate?  How do I know that the training they petition me for will effectively reduce organizational risk?
4. Are the Technology solutions I have in place effective, are we managing them effectively, and what sort of States of Knowledge could they provide me with (to make good decisions and execute upon them, from above)?

This, for the CISO, is Hansei.  The continuous management of it is Kaizen.  Not to particularly pick on Jill’s article, but creating a “risk register expressed in ALE” might be fine if you’re trying to explain to the board what your “first 100 days in office” will be like - but these sorts of lists are usually not very strategic in nature, and as such, depending on the outcome of that risk register (and the models used to create it) it might not actually be useful.

WHAT IS NEEDED FOR REFLECTION?

So what is needed for this sort of CISO-level Hansei?

The CISO must understand the

• Current State of Nature

turn that into a

• State of Knowledge

and use that to create a

• State of Wisdom.

CREATING A STATE OF NATURE FOR THE IRM PROGRAM

This Current State of Nature determination be done by applying analytical methods to a program audit.  We must understand questions like,  “What is in that program and how is it structured?”  before we can answer questions about “how (good/bad) are we at managing risk?”

There are many ways to structure an IRM program, but as an example - below is a graphic shared with me by Adrian Seccombe.  For those who know Adrian and the Trust Model - this is classified as “white” so it’s OK for public display and consumption.  But here’s what Adrian is trying to build at a high level:

So regarding Adrian’s program diagram:

1. Is a governance framework.  Think ITIL.
2. Is a risk framework.  Think ISO 27002 using FAIR as an analytical engine.  To be fair (pun) I believe this is really issue management, and it’s a process, but that’s OK.
3. Reg compliance should be self explanatory.  That’s essentially what GRC products do for you.
4. With architecture, I think Adrian is inclined towards TOGAF.
5. Security is the ISMS in place (27001, ISM^3, PCI, whatever…)
6. Are the processes that drive execution
7. Monitor (audit) is creating a State of Nature and Evaluate is creating a State of Knowledge from that State of Nature around items 1-6.

EVALUATE - CREATING A STATE OF KNOWLEDGE ABOUT THE IRM PROGRAM

That evaluate is Hansei/Kaizen.  Evaluation, done effectively, will drive actual organizational risk exposure.  Evaluate will even answer those four questions we raised in the “What Should We Be Reflecting About” section above:

1. Once you have a seat at the table, what is the nature of that relationship?  Who are you reporting to and what are their concerns? What and how are you reporting and how might that be addressing their concerns?
2. What processes are in place?, How do you know that those are the processes that should be in place? If they are, what kind of job am I doing at those processes?
3. What is the quality of the skills and resources I have from a people perspective, and how do I know if they are adequate?
4. Are the Technology solutions I have in place effective, are we managing them effectively, and what sort of States of Wisdom do they provide me with (to make good decisions and execute upon them, from above)?

If we could have a nice metric (or set of metrics) that answers these questions, we might call it something like “My Ability To Manage Risk” or MATMR for short.

GETTING TO A STATE OF WISDOM

What’s then missing is how you create a State of Wisdom around the State of Knowledge developed - your “MATMR” metric.  That is, given the current State of Knowledge - how can I be most effective?  This State of Wisdom requires proper models for what risk is, and what you can do to manage it applied in a probabilistic manner (because we can’t intrinsically *know* the future, we can only say with some degree of certainty what the desired course should be).

So the outcome of Hansei/Kaizen should be to create a State of Wisdom about Risk Management.  This is why reflection must be relentless - because your wisdom must be similarly abundant.

This is no small part of the reason RMI exists, why we build software and help organizations understand the things they do.

It's unclear if this error is random or systemic. If it's random -- a small percentage of all votes are dropped -- then it is highly unlikely that this affected the outcome of any election. If it's systemic -- a small percentage of votes for a particular candidate are dropped -- then it is much more problematic.

Ohio is trying to sue:

Ohio Secretary of State Jennifer Brunner is seeking to recover millions of dollars her state spent on the touch-screen machines and is urging the state legislature to require optical scanners statewide instead.

In a lawsuit, Brunner charged on Aug. 6 that touch-screen machines made by the former Diebold Election Systems and bought by 11 Ohio counties "produce computer stoppages" or delays and are vulnerable to "hacking, tampering and other attacks." In all, 44 Ohio counties spent $83 million in 2006 on Diebold's touch screens.

In other news, election officials sometimes take voting machines home for the night.

My 2004 essay: "Why Election Technology is Hard."