I’ve often read people who say that we (security, risk management) need to “think like the attacker”.  And when you read this sort of article, that usually alludes to trying to anticipate the tactics an attacker might use to mess with your C, I, or A.  Smart stuff, that, and very useful when architecting security solutions.  But as I was training some folks Monday, I was thinking in the back of my head about Threat Capability (TCap) in FAIR.  As you might know, we like to estimate the capability of a threat to apply some level of “force” against our assets.  This ability to apply force is a byproduct of the attacker’s skills and resources.  And thinking of how an attacker applies skills and resources, I came across another way we might “think” like an attacker.

Traditionally, I’ve thought of “skills” as being a byproduct of the toolset an attacker has.  This mindset probably stems from my time with Penetration Testing teams, where in the process of scoping the  PenTest I would ask our clients to select the level of effort that they wanted us to throw at them.  If a client chose “high” we’d throw every ‘spoit we had at them.  If they chose “low” we’d limit ourselves to a more commonly available toolset.

But while the resources part of TCap is time & materials (money) - the skills are really more than just the toolset.  Skills would include the ability of the attacker to be creative and innovative.    As an example of that innovation from those PenTesting days - when we got a “high” effort request, we would always try to couple that with some “social engineering”-type of attack, or some unique means of delivering an existing exploit.  Our creativity was not necessarily a byproduct of a unique exploit or tool we had, but the process by which we might deliver pre-existing or commonly available exploits.  I remember when we first got ahold of a handful of 32mb thumb drives (hey, 32mb was huge back then) and “dropped” a few in the lobby of a client’s retail space.  The keystroke loggers and phone-home script weren’t new, but using the thumb drive as delivery vehicle certainly was.

So I’ve started to really think about this concept of innovation, and how if “thinking like an attacker” means to be innovative, we ought to do the same.  I’ve been thinking of two main categories of innovation this morning.

INNOVATION

The first I’ll call Technology Innovation.  And by Technology Innovation, I mean some new, unique, “ahead of the curve” technology that an attacker can use against us.  The obvious example of which is a zero-day.  It’s that “high” tool set our PenTesters would use against the clients.  For security departments, this might be the latest security product designed to enhance our ability to P, D, and/or R.

Alternately, we can be creative in the way we deliver (manage) existing technology.  I think of this as Process Innovation.  It’s doing more with what we already have, just like the PenTest team would be creative in the delivery of an existing exploit.

Unfortunately for us - attackers have traditionally had quite a leg up on us in terms of Process Innovation.  It is much easier fro them to be creative, as they are free of political constraints and bureaucracy.  In contrast, when the security industry tries Process Innovation, the results are checklists and “standards”.  It’s committees and consensus.  An extreme example of which might be something like SABSA - a great work if you want to understand some very smart people’s comprehensive understanding of organizational security  - but the “adoption”of which will do very little to help you be innovative in P/D/R.

It’s worth noting that ultimately, this is one reason I don’t like regulatory compliance efforts - they simply serve to prove how mundane your security department is,  wasting valuable resources that could be spent on creating ways to be more effective.

PROCESS INNOVATION AS A SUBSTITUTE FOR TECHNOLOGY INNOVATION

As we come to the close of 2009, some surveys suggest that security spending isn’t horribly impacted yet by the economy (the latest from E&Y points to only 5% of their respondents getting budget cuts).  But if this is a protracted downturn, and because InfoSec is an operational expense, I would expect cash to become more and more difficult to keep.  And regardless if technology spends do slow, I believe it makes sense to think about Process Innovation because I see Process Innovation as a means to increase effectiveness without significant capital expenditures (effectiveness increases because our ability to manage risk has a direct correlation to the amount of risk we have).

The bad news is, of course, that great innovation is hard.  It is R & D.  Failure is usually a pre-requisite to success.

The good news is, our current state is so bad that many of us don’t need to come up with a whizbang new way of reducing software defects in the SDLC as innovation.  Simply inserting a risk analyst into the PMO’s processes might count as a big enough victory. Be cautioned, though,  that if we’re substituting the risk reductions provided by technology acquisition - Process Innovation might actually be even more “expensive” as it requires us to expend political capital.   But there are (forgive the term) innovative ways to spend this political capital.

For example, by taking a second now and figuring out the 3 things that the rest of the organization can do to make your life easier, when that “I need to reduce your budget” talk comes, you can be prepared to negotiate.  Get a political capital “loan” or “investment” from the C-Suite reducing your budget.  Something to the effect of: “I expected this, and am happy to give up my budget.  But if our tolerance for risk hasn’t changed, what I’d like to do is get you to personally back my office on three projects I’ve identified that can reduce our risk without requiring significant capital expenditure.”

No longer are OS vendors and other large infrastructure technology providers the main source of vulnerabilities. It’s the thousands of applications, produced by thousands of software vendors, that make up this huge 3rd wave. ISS reported that in 2007 that the top five sources of vulnerabilities: Microsoft, Apple, Oracle, IBM, and Cisco, had dropped to supplying us with only 13.6% of our vulnerabilities. 86.4% came from the other thousands of software vendors that supply our computers with a seemingly unending supply of vulnerabilities for attackers to exploit.

In a recent report Microsoft has congratulated itself on doing a good job securing Windows. And by all accounts they have done a good job. But then they state this:

“Unless software development practices change throughout the industry, any improvements in the security of Windows would be meaningless.”

Whoa. Millions of dollars spent on securing the most prevalent piece of software and it could be meaningless? Yes, it’s true. Since attackers typically only need one vulnerability, if it isn’t in the network, and it isn’t in the host configuration, and it isn’t in the OS, they will happily exploit a vulnerability in an application.

At every shift of exploit target the problem has gotten more difficult to solve. Networks had choke points and could be centrally managed. It took a while but eventually host configurations became centrally managed and automated tools could scan configurations. Although OSes were huge and complex beasts with 10’s of millions of lines of code, with enough effort, their vulnerabilities have been largely tamed as Microsoft’s Windows and the Linux kernel track record shows. This was a very substantial, over five year effort, which used some of the most talented security people anywhere.
But now what to do? Instead of a few OSes we now have thousands of applications with vulnerabilities. As Microsoft found out, the attackers don’t go away, they just move on to the next incrementally less juicy vulnerability. In the world of exploits that typically means the vulnerability with the next smallest target population.

Attackers have started with the common client applications that can be found on almost every machine: Acrobat, Flash, RealPlayer, Quicktime, popular antivirus software. And they will continue down the popularity slope until they get to application populations down in the thousands which is getting to fairly small software vendors. Attackers can do this because they can bundle many vulnerabilities together, exploiting the statistical fact that you must have some vulnerable software installed. Compromised web sites have been found attacking visitors with over ten client side exploits preying on multiple versions of vulnerable client software.

The solution to this problem is all software must be written securely, not just the software from the big guys. Small vendors think they aren’t a target just like home users used to think they weren’t a target. People thought, “Why would someone want to attack my home computer?” Then they realized they did home banking, or had a fast internet connection that could be used for DDoS attacks or sending spam. All software vendors need to get the same wakeup call. Attackers don’t want to find a vulnerability in your software to make you look bad. They want any vulnerability. If the population of your software is small they will just bundle your vulnerability together with others in an exploit pack. The days of the average software vendor not having to worry about application security are officially over.

Howard Schmidt, who was once the CSO of Microsoft, knows a thing or two about vendors shipping insecure software.  He offers this advice relating to his iPhone, “Just because a piece of software was distributed through Apple’s App Store, don’t assume that it is vulnerability free.”  I think that sums up the problem pretty well.  Customers assume the software they are getting is vulnerability free until it is proved otherwise.

If it’s distributed by the Apple Store it is coming from a trusted brand. “It must be secure”, many think.  The same thinking is used by people who install social networking applets and give them access to their personal data.  Someone, somewhere is taking care of the software security so I don’t have to.  It must be the platform provider, the store, some industry body, my antivirus provider, or maybe even the government.

You can see how this thinking pervades the consumer space because there are regulatory bodies governing all other aspects of safety and security in our personal lives.  I’m safe in a plane or car because the government is looking out for me with standards and testing requirements.  I am safe in the mall parking lot because the men in the white SUV are patrolling.

This thinking also pervaded the b2b space.  I talk to companies which are outsourcing critical applications to offshore development companies and they assume that security testing is taking place as part of the development process.  I ask them if they have made security quality part of the requirements of the project and they say no.  Then I ask them what evidence does the offshore developer provide to demonstrate they have a certain level of security quality in the software they are producing and they tell me they have never asked.

I can tell you what would happen if they did ask because I have also spoken with the offshore developers.  They have no evidence.  Their concern is getting the software functionality done on time and on budget. They consider fixing security vulnerabilities, once discovered, rework which the customer pays for.  So not only are they not looking for vulnerabilities and relying on the customer to find them, they are charging the customer to fix the problems.  The customer has to this date accepted this model.

The same goes for commercial off the shelf software and open source.  Surely the developers writing the software are trained in secure software engineering.  Surely commercial software companies are using 3rd parties to test their software just like the banks have the big 4 audit their accounting or auto manufacturers submit to testing by the NHTSA. And of course open source has “many eyes” reviewing the code for security defects and informing the developers.  The customer has accepted a model where this is almost never true.

But times are changing and it is partially due to the availability of software that can automate the process of looking for security vulnerabilities. David Rice, the author of “Geekanomics: The Real Cost of Insecure Software” was interviewed recently by Drazin Drazic his Beast or Buddha blog.  He said the trend is toward a future of secure software and automated security analysis is one of the sparks:

BorB: I recently wrote in a post that little is changing. We are not learning from the lessons of the past. There are few, if any new technologies that exist today, that we have great faith and trust in as being secure now, and expecting them to continue to be secure in the future. Any solutions to even basic security issues need a starting point and a significant change to current thinking, and even then, it will takes years to see the impacts of this. What are your thoughts on this? Are we seeing anything at present to make us more confident of the future?

DR: It is true that it takes years to see the positive impacts of a change of mindset. And we are in the unfortunate position of repeating many old lessons.

At base, human history is a collection of exhaustive, expensive, and protracted engagements; only the relentless survive and have a chance at succeeding (notice no guarantee here). Confronting some of our most complex problems like highway safety, nuclear proliferation, or insecure software is painful, difficult, complicated, and troublesome. Human endeavors of any significance are like this. But we must do it. The inertia of culture and status quo is difficult to overcome, but overcome it we can; otherwise, we would not have the better parts of the world we enjoy today.

I believe the technology space is no different. We are just a little dazed and bewildered by all the changes technology has introduced so quickly and on such a grand scale. For every change we react to, another two or three rapidly appear.

I do see sparks of hope emerging. In the United States some members of government are beginning to understand the problem and are willing to start discussing how to approach insecure software from a policy perspective. On the technology front, companies like Ounce, Fortify, and Veracode are beginning to give software buyers an automated method of evaluating assurance levels of software. While not complete in and of themselves, these solutions are, as I stated, “sparks” that can help us progress down paths that were once not easily open to us.

As for the larger issue of cyber security, which software assurance is only a part of, society has a lot of adjusting to do. The Internet is a new environment for many still, and many more to come. There is a learning curve that must be confronted. It took the United States almost 80 years to develop the highway system we know and enjoy today. Nearly $400 billion was spent on this endeavor with hundreds of thousands of lives lost. As this shows, learning how to govern and navigate a new environment is expensive. Failing to learn even more so.

Independent, automated, and repeatable software security testing is an essential component of a safe and secure online environment.  Without it we are stuck with the assumption of vendors perfoming software security as our imaginary security blanket that allows us to operate in the current online world.

Howard Schmidt, who was once the CSO of Microsoft, knows a thing or two about vendors shipping insecure software. He offers this advice relating to his iPhone, “Just because a piece of software was distributed through Apple’s App Store, don’t assume that it is vulnerability free.” I think that sums up the problem pretty well. Customers assume the software they are getting is vulnerability free until it is proved otherwise.

If it’s distributed by the Apple Store it is coming from a trusted brand. “It must be secure”, many think. The same thinking is used by people who install social networking applets and give them access to their personal data. Someone, somewhere is taking care of the software security so I don’t have to. It must be the platform provider, the store, some industry body, my antivirus provider, or maybe even the government.

You can see how this thinking pervades the consumer space because there are regulatory bodies governing all other aspects of safety and security in our personal lives. I’m safe in a plane or car because the government is looking out for me with standards and testing requirements. I am safe in the mall parking lot because the men in the white SUV are patrolling.

This thinking also pervaded the b2b space. I talk to companies which are outsourcing critical applications to offshore development companies and they assume that security testing is taking place as part of the development process. I ask them if they have made security quality part of the requirements of the project and they say no. Then I ask them what evidence does the offshore developer provide to demonstrate they have a certain level of security quality in the software they are producing and they tell me they have never asked.

I can tell you what would happen if they did ask because I have also spoken with the offshore developers. They have no evidence. Their concern is getting the software functionality done on time and on budget. They consider fixing security vulnerabilities, once discovered, rework which the customer pays for. So not only are they not looking for vulnerabilities and relying on the customer to find them, they are charging the customer to fix the problems. The customer has to this date accepted this model.

The same goes for commercial off the shelf software and open source. Surely the developers writing the software are trained in secure software engineering. Surely commercial software companies are using 3rd parties to test their software just like the banks have the big 4 audit their accounting or auto manufacturers submit to testing by the NHTSA. And of course open source has “many eyes” reviewing the code for security defects and informing the developers. The customer has accepted a model where this is almost never true.

But times are changing and it is partially due to the availability of software that can automate the process of looking for security vulnerabilities. David Rice, the author of “Geekanomics: The Real Cost of Insecure Software” was interviewed recently by Drazin Drazic his Beast or Buddha blog. He said the trend is toward a future of secure software and automated security analysis is one of the sparks:

BorB: I recently wrote in a post that little is changing. We are not learning from the lessons of the past. There are few, if any new technologies that exist today, that we have great faith and trust in as being secure now, and expecting them to continue to be secure in the future. Any solutions to even basic security issues need a starting point and a significant change to current thinking, and even then, it will takes years to see the impacts of this. What are your thoughts on this? Are we seeing anything at present to make us more confident of the future?

DR: It is true that it takes years to see the positive impacts of a change of mindset. And we are in the unfortunate position of repeating many old lessons.

At base, human history is a collection of exhaustive, expensive, and protracted engagements; only the relentless survive and have a chance at succeeding (notice no guarantee here). Confronting some of our most complex problems like highway safety, nuclear proliferation, or insecure software is painful, difficult, complicated, and troublesome. Human endeavors of any significance are like this. But we must do it. The inertia of culture and status quo is difficult to overcome, but overcome it we can; otherwise, we would not have the better parts of the world we enjoy today.

I believe the technology space is no different. We are just a little dazed and bewildered by all the changes technology has introduced so quickly and on such a grand scale. For every change we react to, another two or three rapidly appear.

I do see sparks of hope emerging. In the United States some members of government are beginning to understand the problem and are willing to start discussing how to approach insecure software from a policy perspective. On the technology front, companies like Ounce, Fortify, and Veracode are beginning to give software buyers an automated method of evaluating assurance levels of software. While not complete in and of themselves, these solutions are, as I stated, “sparks” that can help us progress down paths that were once not easily open to us.

As for the larger issue of cyber security, which software assurance is only a part of, society has a lot of adjusting to do. The Internet is a new environment for many still, and many more to come. There is a learning curve that must be confronted. It took the United States almost 80 years to develop the highway system we know and enjoy today. Nearly $400 billion was spent on this endeavor with hundreds of thousands of lives lost. As this shows, learning how to govern and navigate a new environment is expensive. Failing to learn even more so.

Independent, automated, and repeatable software security testing is an essential component of a safe and secure online environment. Without it we are stuck with the assumption of vendors perfoming software security as our imaginary security blanket that allows us to operate in the current online world.

Now for those of you who don’t know me personally, here’s a little bit of trivia for you:  Every week I go back and forth between “wow, we’re doing great things above and beyond what the private sector knows about” and “culturally, security in the Government will never work because you’re trying to do risk management in a zero-defects world”.

Bookmark to:

Have you ever sat in a corporate training? Some are good, some are bad, but did you ever say, “man I can’t wait for training today.” What about mandatory training? What about mandatory training in a subject that you really don’t think is your area? What if you had to do it every year, and got harassed if you didn’t do it? What if you were, say, an audio engineer and were dragged into a security class?

I ran the SDL training program at Microsoft for a long time, and developed and taught a big chunk of the training. I spent hundreds of hours in front of thousands of developers, testers, and program managers. I got some really good reviews (and a few bad ones) on the classes I offered. And I tried to do a lot of things to try to make the trainings interesting. I handed out dozens of fresh peaches in an early class on fuzz testing, for example.  The room smelled really nice after that, and there are probably still a few people around Microsoft who think of fuzz testing when they see a peach.

But even on my best day, I was under no illusion that the majority of the audience was excited to be there, and I was certain that they weren’t going to go back to their offices and spend weeks applying the lessons from the class, setting aside other things that are causing present and immediate problems in favor of something that is far off into the future.

You have to work at getting people’s attention – especially as it relates to security and privacy. From time to time, I would see people reading their mail in class, and I would point to them and ask them a question. That did not endear me to the audience as much as the peaches, but embarrassment is always fresh and in season.  J

One student wrote of one of my classes, “the basics for secure design - could be replaced by non-anonymous site-wide exam with open material.”  He was not alone, I assure you.

Is that an indication that our training, or any training, is pointless? Hardly, but training alone is not a change agent.

Richard Derwent Cooke wrote,   “It is a first principle of Change Management that people will act in what they perceive as being their best interests.”

At best, training can provide people with insight into what they need to do to solve a security problem if they believe that solving that security problem is in their best interests.

To be effective, training needs to happen in an environment:

·         Where expectations are clearly set (the SDL sets specific minimum requirements).

·         People have appropriate incentives and consequences (security is a great career path at Microsoft, and nobody wants to be the one holding up a ship schedule for failure to meet a security requirement).

·         Where tools and resources to accomplish the goals are available (we build a whole variety of tools that map to the SDL requirements).

·         Where management models the behavior (recall the original BillG TWC memo).

·         Where the environment reflects and supports the values presented in the training (apparent in everything Microsoft does).

Don’t make the mistake of thinking that a bunch of training, even really high quality training done periodically, will result in actual behavior change. It won’t. You have to build an environment where people perceive solving security problems as being in their best interests. You have to make security their problem – not in the sense of passing the buck, but in the sense of changing their behavior so they will bring security problems to you.

To illustrate further, I’ll cite two examples. First, fuzz testing. Fuzz testing has been a success story here at Microsoft. Tools arise spontaneously to solve new fuzzing challenges, written by people who believe the challenges are their challenges. There are people who feel ownership for our fuzzing strategy and on-going research and science, there are specific goals and requirements, we have training (remember the peaches?), and internally developed fuzzers have won prestigious awards within the company, handed out by members of the executive staff, and all of this gets revisited periodically as part of the SDL.

 By contrast, I’ll choose a less successful area – defect estimation. On my own volition, I created (based mostly on some excellent material from Microsoft Research) and taught a class called “Defect Estimation and Management” and added it to the SDL curriculum. Microsoft is a great place to work in that regard. It was pretty close to the best-reviewed class I taught. But, we have not yet been able to establish a set of tools to estimate security defect density effectively, and establish a fair set of expectations, incentives, and consequences, or even  decide what we should do if we had the data. We discovered some things, though. For example, based on what I observed (which should not be construed as rigorous research), it does not appear as if the density of general defects correlates closely with the density of security defects.  And Microsoft Research found higher code coverage in testing correlates with higher bug rates in the field.

And so even though people like the idea of defect estimation, and we’ve got some interesting and surprising data, we’ve not yet been successful in changing people’s behavior.  Generally speaking, an individual test manager does not feel that establishing a high quality estimate of their defect density is in his or her best interests, as compared to, say, improving the time in which an established series of tests can be performed .  

We need to build an environment that has the tools, training, rewards and incentives, and expectations and consequences to change people’s behavior. Not that we’re not trying. But training won’t solve it alone, nor would tools, trophies, rants, testing, code review, or some edict from on high. The SDL is as much about changing the culture and influencing the behavior of individual engineers as it is anything else.

I’m convinced that Microsoft’s SDL process works because it addresses the end-to-end problem - from training through servicing, and provides a complete environment where people feel ownership of their part of the security problem and have the resources to solve it.

So the next time you find yourself sitting in some mandatory training, remember the lessons of the SDL (and most of the research on human performance management): training alone won’t cut it. If you want real behavior change, there have to be things outside the lecture room to influence people to change their behavior.

• Architectural/Design Defects
• Implementation Defects

Microsoft has been releasing security bulletins since 1999. Based on some informal analysis that members of our organization have done, we believe well over 50% of *all* security bulletins have resulted from implementation vulnerabilities and by some estimates as high as 70-80%. (Some cases are questionable and we debate if they are truly “implementation issues” vs. “design issues” – hence this metric isn’t precise, but still useful). I have also heard similar ratios described in casual discussions with other software developers.