I’m assuming the author wants us to read the title as “Things to Look Out For in Performing Risk Analysis” and not “Risk Management is Folly - Stop, Stop, Stop!” The former is fine, the latter isn’t supported by the evidence presented by the subjects of the article.
The subjects of the article are a good study from Wade Baker & Co. at Verizon, and a report from RSA’s Security for Business Innovation Council. Let’s take a look at each of these and examine why what they’re saying might contribute to poor risk management, shall we?

1.)  THE VERIZON REPORT

The Verizon report is an analysis of some 530 forensic investigations their company performed.  It is well worth your time as it’s chock full of interesting information.  As it relates to the Dark Reading piece, a coarse summary would be that “likelihood” is “different” for different people and so you can’t use the same “likelihood” across different industries.

Distilled through the lens of FAIR:

“different threat communities may be applicable based on Probability of Action factors which include: Value, Level of Effort and Risk (of Getting Caught).”

Or, even further distilled and in the words of my six year old son,

“Duh-uh”.

With regards to what I assume is the purpose of the article (What Doesn’t Work in Risk Analysis) this concept  seems just to rehash the old GIGO argument regarding risk analysis.  Great.  Can’t argue with that, nor it’s corollary QIQO (quality in, quality out).

But let me ask you -  is this really a problem common in your analysis?  Did reading this article make you go “Crap, we’ve been using data normalized across multiple industries in our analysis! They’re all wrong!”  Or have you already been accounting for the unique value proposition your company has to the specific threat community you’re worried about?  See, maybe I’m just not your average analyst, but even in my NIST/OCTAVE days, this has *never* been an issue for me.

Let me be specific, this is not a problem with Verizon’s very cool report.  It’s just that I don’t see what the big deal is.  This article is starting to feel like someone is running through the motions, trying to play the ” a crazy title gets people to read a boring article” game.

Speaking of cool reports - You know what would be cool?  I think it would be interesting to see is the quality of these companies’ “risk management process” established using good criteria,  and then correlated to the frequency and magnitude of real-world losses across the aggregate sample.  In other words, can we establish evidence that strong risk management practices not just reduce “risk” but also reduce actual incidents.

2.)  THE RSA COUNCIL “EXPLORES WHY LEGACY METHODS OF EVALUATING INFORMATION SECURITY RISK DON’T WORK IN TODAY’S CONNECTED WORLD, IN WHICH ANY NEW BUSINESS INNOVATION INHERENTLY CARRIES SOME LEVEL OF RISK TO INFORMATION.”

This report from the RSA council puts forth a seemingly obvious proposition, that risk must be balanced by reward.  Why is this news?  Now as I read the article it’s not clear if:

• The RSA Council is claiming that the CISO’s office should be the ones determining reward.  Absurd.

or

• Businesses aren’t doing a good job at determining risk and reward.

Let’s go with the latter.  So I’m pretty sure (good) businesses do a good job at estimating reward.  Businesses I’ve been a part of?  We LOVE(D) estimating reward.  We don’t tend to start projects all willy-nilly. No we tend to be careful to identify the size of the market and what it will cost to address the market.  So what could the problem be that this RSA council is trying to address?  Maybe it has to do with something like the following:

Yesterday, I got a demo of an IT-GRC application that shall remain nameless.  It seemed to be very good at the “C” bits - lots of information on regulations and expectations and even what sorts of controls would answer the regulations (which is goofy, but we’ll have to talk about that later).  It also gave you the ability to build workflow quite nicely.  But it measured NOTHING.  There really was no observable “G” and “R” was really Medium X Low X Low = High sorts of stuff.  So let’s use this relatively expensive tool as evidence of what your average CISO is armed with going into a Risk/Reward sort of meeting.  I imagine a nice board room with wood-grain paneling and glass bowls filled with little chocolate covered mints designed to give everyone involved in the meeting (CEO, CFO, CIO, CSO, VP S&M, etc…) a little sugar rush when needed and fresh breath.  The conversation goes a little something like this (apologies to Rich):

Business Guy Who Wants to Make Money Because That’s What Businesses Do: Based on market studies, we believe that initial gross revenues from the new product and technology rollout will be eleventy gazillion dollars based on a 37% market penetration in Scandinavia, alone.

CSO: Well now, we have a likelihood of “High” and a “C” impact of Medium, and an “I” impact of Low, and an “A” impact of “High” and because we are a (bank/hospital/retailer/basically any business that breathes anymore) we weight “C” by a factor of 2 - we multiplied those all together and got a “High”.

So can you guys delay the product rollout by 9 months and give me a bunch more money that’s not in the budget so that I can get this thing down to a “Medium”, please?

Again, I just don’t see the problem with Information Risk Management being that our businesses have no idea what the rewards of business might be.  Now maybe we need get a seat in that boardroom just to be able to talk about our “Mediums”, sure.  And maybe we’re infantile in our ability to describe our problem space.  But I cannot fathom that “Risk Management Doesn’t Work” because businesses haven’t been considering “reward”.

WHY RISK MANAGEMENT MAY  NOT BE WORKIN’ FOR YOU

Two meta-categories of causation:

• No skills

and/or

• No resources

Any ancillary “cause” can be mapped to one of these categories.  You could have significant resources but crappy models, and have conversations like our imaginary CSO, above.  You could have really good models and people trained and motivated to use them, but scarce time & money, so no conversation happens.

Now my question for you is - which does it make sense to acquire *first* to solve the “Why Risk Management Doesn’t Work” problems, skills or resources?

Charlie Rose:  
And so when you look at where we are going, there seems to be two issues that are apparent to me at least, risk and leverage.  We just lost sight of risk and leverage of what was appropriate?

Warren Buffett:  
Yeah.  Again, because it pays off for a while.  You know, you can lose leverage, and it's the only way a smart guy can go broke.  If you owe money, you can't pay them out.  You just pay for everything, you do smart things, you eventually get very rich.  If you do smart things and use leverage and do one wrong thing along the way, it could wipe you out, because anything times zero is zero.  But it's reinforcing when the people around you are doing it successfully, you're doing it successfully, and it's a lot like Cinderella at the ball.  I mean you know at midnight everything is going to turn to pumpkins and mice; right?  But if the evening goes along, I mean, you know, the guys look better all the time, the music sounds better, it's more and more fun, you think why the hell should I leave at quarter of 12.  I'll leave at two minutes to 12.  But the trouble is, there are no clocks on the wall.  And everybody thinks they're going to leave at two minutes to 12.

Charlie Rose:  
And should wise people have known better?

Warren Buffett:  
People should always know better.

Charlie Rose:  
Yeah.

Warren Buffett:  
I mean people -- people don't get -- they don't get smarter about things that get as basic as greed and you can't stand to see your neighbor getting rich.  You know you're smarter than he is, and he's doing these things, you know, and he's getting rich, and your spouse is getting unhappy with you because you aren't doing -- pretty soon you start doing it.  And so you get what I call the natural progression, the three Is.  The innovators, the imitators, and the idiots.  And that's what happens.  Everybody just kind of goes along.  And you look kind of silly if you disagree.  I mean, you know, you could have these crazy Internet valuations in the late 1990s, but they prove themselves out in the market.  The next day they were selling for more than they were the day before, and people said, you know, you're crazy if you don't get in on this.  So it's very human.  Now, with housing it's something even more dramatic than that, because most people aspire to own their own home.  And if you really think that houses prices are going to go up next year and the year after, you feel if I don't buy it this year, I'm going to have to buy it next year.  That's not true of an Internet stock.  But it's true of a home.  And when somebody makes it very easy for you to do it by saying you don't really have to put up my money, you can lie about your income a little, or we'll give you 100 percent mortgage, you're going to do it, because everybody that's done it has been proven right.  You have what they call social tools, and, you know, you're going to feel like an idiot if you didn't do it, because the house cost more.

Warren Buffett:  
Oh, I think confidence will come back.  I will tell you this.  This country is going -- be living better ten years from now than it is now.  It will be living better in 20 years from now than ten years from now.  The ingredients that made this country, you know, the miracle of the world -- I mean we had a seven for one improvement in the average American standard of living in the 20th century.  Now, we had the great depression, we had two world wars, we had the flu epidemic.  You know, we had oil shock.  You know, we had all these terrible things happen.  But something about the American system unleashed more and of a potential to human beings over that hundred years so that we had a seven for one improvement in -- there's never been any -- I mean, you have centuries where if you've got a 1 percent improvement, then it's something.  So we've got a great system.  And we've got more productive capacity now than we ever have.  The American worker is more productive than he's ever been.  We've got more people to do it.  We've got all the ingredients for a sensational future.  It's just that right now the athlete's on the floor.  But we -- this is a super athlete.

A blog with one of the biggest followings on the SBN is the GNUCitizen blog. Today in a post called "Fear" the author states, "The entire information security industry today is based on fear." He then goes on to say, "This is what gives security vendors the power to sell you useless products which you don???t really need."  So of course I don't agree with the later statement, not all of those products are useless, but is it really fear that is motivating buyers?

Fear of what is a good first question. The blog post talks about fear of being hacked, fear of harm to reputation.  To that we can add fear of jail or fines and by doing so cover the compliance issue. So yeah, at first blush it does appear that fear is the prime motivator in security.  But think a bit deeper on this and you come to the conclusion that fear is a primary driver for so much of what we do besides security.  Fear of failure, fear of loss, fear, fear, fear. Is there anything besides fear that motivates people?

For me it comes down to the carrot or the stick.  The carrot being the reward.  So making money or however you measure success is certainly motivating.  The stick is failure.  Their are consequences of failure.  But really isn't success and failure two heads of the same coin.  Aren't the rewards of success and the consequences of failure a Zoroastic type of Yin and Yang? 

So if in the final analysis, success and failure are intrinsically linked. There really is nothing wrong with saying security sales are motivated by fear, because by the same token they are motivated by success.  Now as to useless security products, lets discuss that a bit later. All of this philosophy is hurting my head.

A blog with one of the biggest followings on the SBN is the GNUCitizen blog. Today in a post called "Fear" the author states, "The entire information security industry today is based on fear." He than goes on to say, "This is what gives security vendors the power to sell you useless products which you don’t really need."  So of course I don't agree with the later statement, not all of those products are useless, but is it really fear that is motivating buyers?

Fear of what is a good first question. The blog post talks about fear of being hacked, fear of harm to reputation.  To that we can add fear of jail or fines and by doing so cover the compliance isssue. So yeah, at first blush it does appear that fear is the prime motivator in security.  But think a bit deeper on this and you come to the conclusion that fear is a primary driver for so much of what we do besides security.  Fear of failure, fear of loss, fear, fear, fear. Is there anything besides fear that motivates people?

For me it comes down to the carrot or the stick.  The carrot being the reward.  So making money or however you measure success is certainly motivating.  The stick is failure.  Their are consequences of failure.  But really isn't success and failure two heads of the same coin.  Aren't the rewards of success and the consequences of failure a zoroastic type of Yin and Yang? 

So if in the final analysis, success and failure are intrinsically linked there really is nothing wrong with saying security sales are motivated by fear, because by the same token they are motivated by success.  Now as to useless security products, lets discuss that a bit later.

The statement from Starbucks reads: "T-Mobile, AT&T and Starbucks have entered into a memorandum of understanding to resolve their disputes and are committed to providing a high quality WiFi experience for customers, including Starbucks Rewards Customers, at Starbucks locations nationwide."

My interpretation is Starbucks said, oops, our bad, and they're figuring out the dollars and cents. Sometimes companies move too rapidly. T-Mobile is a quasi-jilted suitor, although they get something out of AT&T transition, too, so they're not likely to cut any slack.

Reuters confirms that AT&T confirms the statement. I separately confirmed with T-Mobile that the statement is accurate as well.

The lawsuit, which I've read, says that T-Mobile never agreed to nor was compensated for providing free service in stores. A link to AT&T's network in all markets except San Antonio, Tex., and Bakersfield, Calif., is handled on the backend entirely by T-Mobile. The suit notes, "If AT&T or Starbucks wanted to offer 'free' Wi-Fi in non-transitioned stores for Starbucks customers, as they are now doing, they should have--and, indeed, were contractually required to--negotiate such an arrangement with T-Mobile."

The crux is that while T-Mobile did agree to provide free roaming to AT&T subscribers, as defined in a bilateral roaming agreement the two firms signed, T-Mobile states the agreement doesn't allow other parties to roam for free. (That's most likely why we haven't seen AT&T's roaming partners, like Boingo and iPass, appear in the login menu, too.)

Representatives of Starbucks immediately available on a Friday night. A Reuters report quotes a Starbucks spokesperson who doesn't comment directly on the suit.

An AT&T spokesperson said via email that the company doesn't comment on other companies' lawsuits. AT&T is not a party to the suit, although it is mentioned throughout.

The lawsuit provides quite a bit of previously private detail about the transition agreement. T-Mobile says that the transition contract signed by all three parties, T-Mobile still had responsibility for and ownership of a market until all equipment in all stores in a defined market belong to AT&T. The agreement also called for exclusive roaming only for each party's existing subscribers in markets that were converted or still under T-Mobile's control until 4-Jan-2009.

T-Mobile states in the suit that they didn't learn of the planned launch of the free Wi-Fi service until 30-May-2008.

T-Mobile wants money, release from current obligations, and other damages. I expect that things have gone quite far for them to file a suit.

"We hope to come to an amicable solution, and sometimes you do have to file a complaint in order to make that happen," T-Mobile's Dobrow said. "It's easy to give something away for free if it's not yours."

The Card Rewards program allows anyone with a Starbucks Card to register it with Starbucks for freebies, including Wi-Fi. There's an interesting choice (when it worked) where you can select whether to have freebies like free exotic milk options or brewed coffee refills by themselves or with Wi-Fi on top. If you choose Wi-Fi, you're redirected to SBC servers (for nostalgia's sake), at which point everything seems to fall apart.

Trying two separate cards, I was unable to set up an account and get the cards to take. The errors weren't clearly spelled out. Clearly, the system was neither designed to handle demand, nor designed to fail gracefully, blocking users until capacity was available.

For loyal Starbucks patrons, this doesn't come across very well at all.

[Photo by Matt Davis. Used under Creative Commons license.]

Have you ever sat in a corporate training? Some are good, some are bad, but did you ever say, “man I can’t wait for training today.” What about mandatory training? What about mandatory training in a subject that you really don’t think is your area? What if you had to do it every year, and got harassed if you didn’t do it? What if you were, say, an audio engineer and were dragged into a security class?

I ran the SDL training program at Microsoft for a long time, and developed and taught a big chunk of the training. I spent hundreds of hours in front of thousands of developers, testers, and program managers. I got some really good reviews (and a few bad ones) on the classes I offered. And I tried to do a lot of things to try to make the trainings interesting. I handed out dozens of fresh peaches in an early class on fuzz testing, for example.  The room smelled really nice after that, and there are probably still a few people around Microsoft who think of fuzz testing when they see a peach.

But even on my best day, I was under no illusion that the majority of the audience was excited to be there, and I was certain that they weren’t going to go back to their offices and spend weeks applying the lessons from the class, setting aside other things that are causing present and immediate problems in favor of something that is far off into the future.

You have to work at getting people’s attention – especially as it relates to security and privacy. From time to time, I would see people reading their mail in class, and I would point to them and ask them a question. That did not endear me to the audience as much as the peaches, but embarrassment is always fresh and in season.  J

One student wrote of one of my classes, “the basics for secure design - could be replaced by non-anonymous site-wide exam with open material.”  He was not alone, I assure you.

Is that an indication that our training, or any training, is pointless? Hardly, but training alone is not a change agent.

Richard Derwent Cooke wrote,   “It is a first principle of Change Management that people will act in what they perceive as being their best interests.”

At best, training can provide people with insight into what they need to do to solve a security problem if they believe that solving that security problem is in their best interests.

To be effective, training needs to happen in an environment:

·         Where expectations are clearly set (the SDL sets specific minimum requirements).

·         People have appropriate incentives and consequences (security is a great career path at Microsoft, and nobody wants to be the one holding up a ship schedule for failure to meet a security requirement).

·         Where tools and resources to accomplish the goals are available (we build a whole variety of tools that map to the SDL requirements).

·         Where management models the behavior (recall the original BillG TWC memo).

·         Where the environment reflects and supports the values presented in the training (apparent in everything Microsoft does).

Don’t make the mistake of thinking that a bunch of training, even really high quality training done periodically, will result in actual behavior change. It won’t. You have to build an environment where people perceive solving security problems as being in their best interests. You have to make security their problem – not in the sense of passing the buck, but in the sense of changing their behavior so they will bring security problems to you.

To illustrate further, I’ll cite two examples. First, fuzz testing. Fuzz testing has been a success story here at Microsoft. Tools arise spontaneously to solve new fuzzing challenges, written by people who believe the challenges are their challenges. There are people who feel ownership for our fuzzing strategy and on-going research and science, there are specific goals and requirements, we have training (remember the peaches?), and internally developed fuzzers have won prestigious awards within the company, handed out by members of the executive staff, and all of this gets revisited periodically as part of the SDL.

 By contrast, I’ll choose a less successful area – defect estimation. On my own volition, I created (based mostly on some excellent material from Microsoft Research) and taught a class called “Defect Estimation and Management” and added it to the SDL curriculum. Microsoft is a great place to work in that regard. It was pretty close to the best-reviewed class I taught. But, we have not yet been able to establish a set of tools to estimate security defect density effectively, and establish a fair set of expectations, incentives, and consequences, or even  decide what we should do if we had the data. We discovered some things, though. For example, based on what I observed (which should not be construed as rigorous research), it does not appear as if the density of general defects correlates closely with the density of security defects.  And Microsoft Research found higher code coverage in testing correlates with higher bug rates in the field.

And so even though people like the idea of defect estimation, and we’ve got some interesting and surprising data, we’ve not yet been successful in changing people’s behavior.  Generally speaking, an individual test manager does not feel that establishing a high quality estimate of their defect density is in his or her best interests, as compared to, say, improving the time in which an established series of tests can be performed .  

We need to build an environment that has the tools, training, rewards and incentives, and expectations and consequences to change people’s behavior. Not that we’re not trying. But training won’t solve it alone, nor would tools, trophies, rants, testing, code review, or some edict from on high. The SDL is as much about changing the culture and influencing the behavior of individual engineers as it is anything else.

I’m convinced that Microsoft’s SDL process works because it addresses the end-to-end problem - from training through servicing, and provides a complete environment where people feel ownership of their part of the security problem and have the resources to solve it.

So the next time you find yourself sitting in some mandatory training, remember the lessons of the SDL (and most of the research on human performance management): training alone won’t cut it. If you want real behavior change, there have to be things outside the lecture room to influence people to change their behavior.

Evolutionarily, presumably it is a better survival strategy to -- all other things being equal, of course -- accept small gains rather than risking them for larger ones, and risk larger losses rather than accepting smaller losses. Lions chase young or wounded wildebeest because the investment needed to kill them is lower. Mature and healthy prey would probably be more nutritious, but there's a risk of missing lunch entirely if it gets away. And a small meal will tide the lion over until another day. Getting through today is more important than the possibility of having food tomorrow.

Similarly, it is evolutionarily better to risk a larger loss than to accept a smaller loss. Because animals tend to live on the razor's edge between starvation and reproduction, any loss of food -- whether small or large -- can be equally bad. That is, both can result in death. If that's true, the best option is to risk everything for the chance at no loss at all.

This behavior has been demonstrated in animals as well: "species of insects, birds and mammals range from risk neutral to risk averse when making decisions about amounts of food, but are risk seeking towards delays in receiving food."

A recent study examines the relative risk preferences in two closely related species: chimanzees and bonobos.

Abstract

Human and non-human animals tend to avoid risky prospects. If such patterns of economic choice are adaptive, risk preferences should reflect the typical decision-making environments faced by organisms. However, this approach has not been widely used to examine the risk sensitivity in closely related species with different ecologies. Here, we experimentally examined risk-sensitive behaviour in chimpanzees (Pan troglodytes) and bonobos (Pan paniscus), closely related species whose distinct ecologies are thought to be the major selective force shaping their unique behavioural repertoires. Because chimpanzees exploit riskier food sources in the wild, we predicted that they would exhibit greater tolerance for risk in choices about food. Results confirmed this prediction: chimpanzees significantly preferred the risky option, whereas bonobos preferred the fixed option. These results provide a relatively rare example of risk-prone behaviour in the context of gains and show how ecological pressures can sculpt economic decision making.

The basic argument is that in the natural environment of the chimpanzee, if you don't take risks you don't get any of the high-value rewards (e.g., monkey meat). Bonobos "rely more heavily than chimpanzees on terrestrial herbaceous vegetation, a more temporally and spatially consistent food source." So chimpanzees are less likely to avoid taking risks -- as most species are.

Fascinating stuff, but there are at least two problems with this study. The first one, the researchers explain in their paper. The animals studied -- five of each species -- were from the Wolfgang Koehler Primate Research Center at the Leipzig Zoo, and the experimenters were unable to rule out differences in the "experiences, cultures and conditions of the two specific groups tested here."

The second problem is more general: we know very little about the life of bonobos in the wild. There's a lot of popular stereotypes about bonobos, but they're sloppy at best.

Even so, I like seeing this kind of research.