When a data owner's (a business) sensitive data is breached it is difficult to quantify the monetary loss. According to respectable survey sources, the average cost of sensitive data breach for a large size company is about $50,000. I am attempting here to think about this in simple mathametical terms:

There is a data breach. From the data owner's perspective the loss is:

Loss = Cost to protect data + Loss of business due to data theft aka cost of competitive disadvantage

From the data thief's perspective

Net Gain= [Cost of producing the data  *  Data freshness factor] - Cost to steal the data + Profit of business due to data aka gain of competitive advantage

From the above two equations it is very clear that this is not a zero sum game. There is a clear cost asymmetry for a data owner and for a data thief. When there is an asymmetry there is an opportunity. Data owner would not even know that the data is lost because the original copy of the data may be still intact - data thief could have simply copied the data. Data theft does not look like a car theft, there is no vacuum left behind. 

This motivates a data thief to keep the cost to steal low, steal highly valuable data that has a long shelf life and in a way that data owner will never even be aware of theft.

From a data thief's perspective, the cost to steal data if kept high would disincentive him. Moreover, Data freshness factor, i.e. how valuable this data is over period of time plays an important role. A good example is content of today's newspaper is hardly valuable tomorrow, but the content of newspaper two days ahead (if can be procured)would be invaluable. Data relevance is a function of time and other marketplace variables -  Data freshness Factor accounts for that variable. A good way to discourage data thief is to increase his/her cost to steal the data. There are other inferences from the above equation. If there exists no competitive advantage with the stolen data, hardly any thief would even venture to steal the data in the first place. If the cost of producing data is very low, then probably thief can just produce the data himself and would not attempt to steal the data. If the cost of theft is kept high, it would definitely deter the data thief from stealing data using technical mechanisms, then the data thief would exploit weak links in data security such as use of social engineering to get access to the data.

From data owner perspective protecting data becomes very important. How much would the owner be willing to spend? Not definitely the cost equal to cost of producing the data. 1% to 10% of cost of producing data is considered prudent. For a data owner it is difficult to estimate cost of data protection of a specific data, because it is not easy to chunkify data protection costs. Moreover, as Dan Geer says in his book, a data owner has to protect himself from number of intruders not just one.

It pays for a data owner to: be aware of data breaches (or data leaks), employ appropriate mechanisms to protect the data; the cost of protection which is fractional cost of the valuable data and enhance information security awareness of personnel who handle the data.

Data loss is not a zero sum game. The advantage is in favor of a data thief (data thieves rather). Data owner does not give much thought on the value of data unless there is a data theft. But, a data thief has every reason to think about economics of data theft before he acts to steal the data else data thief won't survive in this game and he is very well aware of his advantageous position.

I’d love to have another blog even more specific - “Ok, that Assurance is Evidence Of What, Exactly?

Today he has a great article called:

What’s the matter with Risk Management?

And “in short, it’s everything.” It pretty much sums up why I had to grow to re-evaluate how our industry does risk, risk management, approaches controls & vulnerability and find a new way.   A couple of things jump out at me in reading Chris’ article:

1.)  Just because that Deming cycle sucks and is full of unknowns doesn’t mean “risk” doesn’t exist, nor that it isn’t of primary importance. Nor does it mean that in the absence of model & methodology, we won’t be “doing” risk analysis anyway - just in an ad hoc method and completely from “the gut”.

Our industry calls these unstructured risk analysis “Best Practices”, as it’s an easy and convenient way of sweeping the unknowns under the rug of bureaucracy and enforcing it via peer pressure.

2.)  What this “suckiness” does mean is that your model and methodology aren’t helping you. As Chris intimates, there is too much uncertainty in the inputs for his model (they are, in the language of Bayesians - too subjective to be useful priors).

Take for example how we might be approaching the “controls” part of our analysis.  Chris writes:

“2. What are the controls that we have to employ?
800-53, ISO 27001, PCI, etc.

Still kinda good, but we basically know that ISO is relatively voluntary and NIST supplies a control catalog and not policies. So here we have to take the control catalog, and mash our policies into it.”

I wouldn’t call this “kinda good” at all :)  These control catalogs only provide a hierarchy within which to look for evidence of  our ability to resist an attacker.  They are incapable of making any claim about the effectiveness of the controls when they are operated at 100% efficiency, or more importantly, what % efficiency our specific organization operates at.

Let’s use Chris Hayes’ Initech as our fictional example.

Initech has a control (a back door on a loading dock).  Now the locks on the door are 100% capable of locking the door.  This is different than saying that they are capable of frustrating all but the top 5% of lockpicking burgalars.  It is also diffferent than saying that in a sample of several “walk around audits” the doors are left open 20% of the time (they are not in compliance with policy 100% of the time).  Even worse, that 80% of the time the door is not propped open?  Yeah, tailgating is a known issue.

So we have several different variables here that we need to account for (and it’s just a door).  But the analogy stands that most “risk management” methodologies are “We have a door, yes/no?” And most GRC platforms, when asked for their “opinion” will simply say “door is needed” or, even worse, “a door policy is needed”.

3.)  Criticality and the Source of Value is all messed up in these Risk Management models.

Chris writes:

Someone wants me to tell them which boxes are more critical than others. This is mainly because of budgetary or operational reasons. To which I usually say “All of them, it is a system after all”.

This literally made me laugh out loud.  And this sort of “rate the firewall as Risk = 500 but rate the actual business application as Risk = 157″ thing is also endemic.  Now Chris is very smart here.  He correctly identifies that the value is tied to the business process the systems support, and not to a specific box.  Oh, we scan at the specific box level - but because of the nature of systemic failures - all the boxes in the process are inexorably interrelated.

One of the reasons I really like FAIR is that the losses are quantified (or qualified) based not on some amorphous value of the box or the process itself, but losses are linked to the actions that the threat will take. Take systems in a highly regulated industries as an example.  Usually the most probable losses aren’t due to system compromise per se, but in the disclosure the compromise causes (regulators are a threat source, after all).  But many “risk management” methodologies will say “online banking is worth $2 billion, the value of the systems is therefore $2 billion”.  And suddenly we’re telling executive management that there’s a 60% probability that they’ll lose $2 billion.

4.)  If the primary source of prior information for your “risk management” methodology is a vulnerability scanner - you’re doing it wrong.  Chris writes:

So we ran a scan and now we have a report. A snapshot in time to make all decisions. Where did these vulnerability ratings come from? Do I even know if my system is at risk? What if I spend my time on vulnerabilities that have no threat?

So first, my thoughts are that actual “vulnerability” must be a comparison of the force a threat can apply, and our ability to resist that force (this is a probability statement, btw).

Changing your thinking about vulnerability now helps us understand the problem in several new ways.  First, you can start to divorce yourself from the scanner.  After all, the scanner is simply providing you with current state information that is usually just relevant variance from policy. It doesn’t really tell you about real “weakness in a system” because the system is an interrelated mess of people, processes and IT assets.

5.)  Finally, most “risk management” approaches just *don’t* do a good job of helping us understand the how’s and why’s of managing risk. In the past, I’ve referred to these standards as really being “issue management” because they are at their heart, an act of discovery - a formal process around gathering prior information.  They are not, in and of themselves, capable of linking the issues discovered to the root cause.  And these root causes?  Yeah, they’re the things that create “risk”.  Not a threat, not a vulnerability, not the existence of an asset - the amount of risk that we have stems from our capability to manage it.

So Chris, I completely agree - but I wouldn’t give up yet.  There actually are a few of us who are focused on what you suggest:

Where to go from here: A fundamental revamp of how to deal with Risk. Where risk professionals focus on the treating the sickness and not the symptoms, and come up with some new success/actionable metrics.

Chris, there’s nothing I want to do more than that.

I have to admit that I was already a fan of Collins’ quantitative style blended with clever insight, but this was the first time that I had seen him in person, and he was just spectacular. He has a vivid, animated way of telling a story, and had a great sense of humor. This combination of presentation skill was put to immediate use with his first statement drawing a hearty laugh from the audience full of entrepreneurs.

“How many of you in the room are constitutionally unemployable?”

Much of his remaining presentation provided interesting stories and insight from the research that he has done to understand the make-up of exceptional companies.

As Jim said, he has spent years studying the contrast between average companies and exceptional companies. They faced the same set of variables… similar economic conditions, similar competition for top human resources, and a similar set of huge unknowns.

What is the single biggest element of difference?

Not a function of the cards you are dealt, or circumstance… it is conscious choice and discipline.

Jim’s key principles & disciplines that have come from the studies we have worked on:

1. Building greatness is a cumulative never ending process! The idea that no matter how exceptional, you are always only relatively as good as to what you can do next.
2. Most overnight successes are 20 years in the making…. Wal-mart  took 13 years to get to 125 stores. Starbucks required 17 years to get to 38 stores.

“If you start to break Packard’s law, and there are very few laws of business, it is like breaking a law of physics for building great companies.” - David Packard (Co-founder of HP)

If you allow growth to exceed your ability to get enough of the right people to fill the key seats to execute on the growth brilliantly, you will fall as surely as a stone dropped from your hand. This is one of those timeless truths that extends beyond technology and economics.

The number one constraint on growth and sustained success…

An ability to get enough of the right people in the key seats to achieve that sustained growth.

The discipline that WHO comes before WHAT. Collins always kept coming back to the “who” thing over and over again. He said, “The more turbulent the world, (given the great current economic uncertainty of our financial system) the more important this issue is.”

A question from the audience came near the end of his session… How do you figure out who are the right people to put in key seats on the bus?

Collins responded with “Given that I stand here amidst a room full of unmotivated people… the right people are self motivated, self disciplined, self managed, The task is not to motivate unmotivated people, the task is not to have to manage people… self motivated, figured it out from there… self motivated people don’t need tons of management … when you have to start managing, you know that you have the wrong person at the task.”

Final thoughts:

Greatness is not a function of circumstance. Greatness is a function of conscious choice and discipline. It is not a matter of circumstance, it is one of choices.

I believe that every one of the Inc. 500 companies that I met at this conference achieved the list because they did not embrace the status quo. Incredible passion, an unwillingness to accept failure and an excessive and compulsive willingness to solve customer’s problems were key ingredients in the business building formula for the entrepreneurs that were at the conference.

To kick off our celebration of our first five years, we asked ScienceLogic founders Dave Link, Richard Chart and Chris Cordray for their thoughts and memories on events leading to today’s milestone. How and why did they set out on this venture? What happened along the way – expected and unexpected? Why were they successful in times when other new (and established) businesses have come and gone?

How did you three put together this team?

We all worked together at a large Managed Service Provider for a couple of years before leaving to start ScienceLogic, so we all knew each other and knew our collective strengths. More importantly, each of us had worked with network management tools on some level (sales and marketing, engineering and product development), and knew first-hand all of the customer pain points, from every perspective. So we left and began rapidly figuring out how to build a better network management solution based upon our real world operational experience..

Dave: One interesting aspect is that our areas of expertise don’t overlap, which has contributed to our success. Chris is excellent with developing the product front-end and interface, Richard handled the backend architecture and engineering and I focused on the technical business side of sales and marketing. Our roles have been to build a product that works well and that provides real value to operations teams that experience the same day to day frustrations that we felt.

Whose idea was it to start the company?

Dave: It was really a collective effort. We were all passionate about “getting it right” and not just starting a company. We knew the industry need and between us, we had the knowledge and skill sets to address all of the right aspects of developing a product and a building a business around it.

What process did you go through to get started?

Richard: From the beginning we knew the type of solution the market needed and we knew that we wanted to build it as an appliance. From different vantage points, we had each experienced the effects of long, difficult and expensive installations that still exist with traditional network tools. Every install has unique variations: there are always different server types, varying hardware and software versions, different patches installed, and on and on. Every installation was time consuming and unpredictable. We knew that an appliance model would address all of these variables and save a lot of time on how quickly customers could achieve immediate value.

The harder decisions were around actually starting the business, assessing the market and of course determining the product pricing.

EM7 completely flips the traditional model of complex, lengthy and expensive deployments. How did you convince others that the EM7 Meta-Appliance product was valid?

Dave: Yes, EM7 totally disrupts the traditional model for network management. While others take a narrow approach, we intentionally designed EM7 to focus on the broad problem – managing the data center. How do you cover a variety of technologies and make sure they work seamlessly together? The vision was to make it easier, not harder, for customers.

Chris: I have to give it to Dave – very early on, he realized the power of a demo. If Dave could get in front of someone, he’d make them a believer. He’d use the Peter Falk/Columbo technique of “let me show you one more thing.” It was very effective. It’s getting easier, but even today people sometimes have to see EM7 in action before they become believers.

Can you describe the early days of running a new business?

Dave: ScienceLogic is a classic case of entrepreneurship. For the first year we worked out of our basements. We kept the costs low in every conceivable way and spent the first year developing the product before we even made a sale.

Chris: We stayed at lots of odd places when we were on the road, took cheap flights with multiple layovers and purchased lots of our first test equipment on eBay. This was during the dot-com bust so there was lots of equipment for sale on eBay, really cheap!

Richard: The amount of equipment I had in my house was absolutely crazy. Back then, servers were huge – I had a Cisco 6509 Catalyst, a Compaq Proliant DL380, Brocade switch, IBM Netfinity 4500R, and tons of other machines.

Chris: I had to install a new circuit box at home because I was blowing breakers. I remember when that 6509 crashed, we revived it and it died again. The second death was final.

So you started in your houses – what was your first office space?

Dave: My friend, the CEO at Ernst & Young Technology had a few extra cubes and a data center in their office that they graciously allowed us to use. Their help was an important step in helping us really formalize the business. We started doing well and adding people, but ironically, their company was downsizing. Before long, many of their original YET people were gone and the ScienceLogic team kept growing in to the open cubes.

Our first leased space was converted warehouse space in Chantilly, VA that once housed an internet radio station. It was cool – it had a large salt water fish tank, a loft, a spiral staircase and a Star Trek door that retracted into the walls with the customary lights and “whooshing” sound.

We outgrew the Chantilly space, leading to our current office in Reston, VA.

Who was the first ScienceLogic customer?

Our first paying customer was Martins Point Health Care. We deployed there in July 2004 and are pleased to say they continue to be a ScienceLogic customer. Other early (and still) EM7 customers include Navy Knowledge Online and the Department of Transportation. Nearly all of our customers are still actively using EM7 and renewing their maintenance.

Where do you see the company in the next 5, 10 or 15 years?

Well, our revenue has doubled year-over-year in each of the last three years, so of course we’d like to continue to grow like that or even faster. In five years we’ve gone from three founders to the point where Dave does not know everyone’s fondest childhood memory. We’ll continue to scale our growth to cover the demands of our growing customer base.

Where do you see the industry going over the coming years?

Chris: IT is always moving and gaining in complexity, so network management is also becoming more complicated. There’s increasing diversity, new standards, virtualization and cloud computing. All of these are today’s technologies. Customers have a mix of the old and the new, so EM7 has to accommodate and support both.

Richard: Each generation of products has a new set of ways to monitor, but the “old” doesn’t go away. Even when a new, hot technology comes along, the old technologies still need to be supported. We work to ensure EM7 keeps up with both.

Dave: After five years we’re just hitting our stride and we’re just now reaching the tipping point in awareness of ScienceLogic and EM7. We’re all still passionate about the product and as Chris and Rich said, there’s still a lot do. We’ll continue disrupting the market with EM7. Our vision hasn’t changed, and with the increasing levels of automation that customers demand, the market needs are greater than ever. Our future is as bright, or brighter, than ever and we’ll continue to be looking for smart ways to automate traditionally manual IT Operations processes.

What’s your advice for someone interested in starting their own business?

Chris: Be passionate. That’s what has gotten me through the tough times. I didn’t really appreciate this thought when I heard others say it before. But it’s very true.

Richard: I agree. We met and talked with lots of people who told us, “That’s been done before.” But we kept going because we truly believed in what we were doing and we knew that while our approach was different, that it would be successful.

Richard: Be fearless. You can’t be too nervous and you need to be able to expect and handle the stress because it will be there. You have to learn to accept the stressful times as a necessary part of the process of starting out on your own.

Dave: Know your niche from the beginning and give potential customers a compelling reason to trust you and really benefit from your solution. You have to know the problem, see the gap and have a clear and consistent vision of how to solve the problem. Then you have to execute. If you don’t build your team with “doers” you won’t make it.

Chris: It helps to have friends. ScienceLogic was built on friendships and relationships, starting with the three of us. If you look at our team, most of our hires are referrals – people who developed and maintained great connections with other great people throughout their careers. Maintain your connections and keep in touch with your network of friends.

For June, we have an 802.1X hat-trick to blame for my slack blogging habits. Over the past few weeks, I’ve had back-to-back 802.1X implementations, one wired, one wireless and one with both. Two government customers and one commercial, not in that order. And I even did one semi-training-slash-semi-implementation-quick-start for another customer.

It’s been fun, but 1X is always challenging. The variety of components, the nature of the interactions and the ‘newness’ of actual implementations make it difficult to work from any type of cookbook or implementation guide. There are just too many variables.

When will it be easier? I think as 1X is more widely implemented in the real world, customers will become more familiar with the concepts and integrators will have more experience to make it go smoothly. For now, everyone has to just take it one step at a time and address issues as they arise. And, for now, I’ll enjoy the job security that 1X offers ;)

Luckily, I’ve had the opportunity to work with a variety of customers and a variety of environments and equipment while hammering out 802.1X. The experience and exposure has certainly given me a unique insight into the issues, complications and solutions that come along with a 1X project.

At present, I think we’ve successfully configured 1X on about a dozen different types of equipment, both switches and wireless APs and controllers, from a variety of vendors. It may not sound like much, but in the world of 1X, that’s quite a variety when you consider each manufacturer has their own ‘system’ for configuring 1X and the commands and procedures can vary greatly even from product-to-product from the same vendor.

Is the 1X streak over? Not at all. We have several customers with NAC and 802.1X projects that we had to queue up for after June 30. I’ll keep you posted!

# # #

Blogger: Pete Lindstrom

Microsoft’s Security Development Lifecycle (SDL) is the main product of its Trustworthy Computing Initiative, launched from the now-famous Bill Gates memo in 2002. Six years into the initiative, Microsoft surely must be reaping the benefits of, for example, the well-publicized security training every developer went through.

So, how do we determine whether the SDL is working? Microsoft suggests that this is a simple exercise – simply compare the number of public vulnerabilities disclosed for products prior to SDL with similar products developed after SDL. The most recent case was comparing Windows XP SP2 to Vista vulnerabilities in the first year. The count is down and Microsoft provides a quick and easy example of the logical fallacy “post hoc ergo propter hoc” which in this case means “public perception is ripe for deception.”

The biggest problem with Microsoft’s assertion is simply that there are too many variables that are uncontrolled and could just as easily be making the difference. There are too many unknowns related to effort of independent researchers and focus on a specific Microsoft platform. At the very least, Microsoft has done an admirable job in making people feel more secure. (I happen to believe the SDL is working as well, but that belief is a matter of conjecture without strong evidence).

If Microsoft wants to use public vulnerability counts as the ultimate arbiter, it needs to create an environment where independent researchers are encouraged to find bugs. Creating a controlled bounty program for a limited time period would increase incentives and at least provide circumstantial evidence of SDL effectiveness. Interestingly, if the number of vuln counts was higher, it still wouldn’t mean SDL is ineffective,  but the framing of the conversation would be entirely different.

The plot thickens when Microsoft makes claims that spending more time and leveraging external resources are a part of SDL. Whether they are or not, there is a big difference between making programmers more secure developers and simply spending more money on a problem. You don’t really need SDL if the latter is more beneficial.

But if public vulnerability counts are not the answer, what should Microsoft be doing to demonstrate the effectiveness of its SDL? Well, it is much easier to determine causality by controlling for all other variables, and conducting a test of two groups – one with SDL training and one without. Comparing vulnerability creation rates per unit output (either developer-hours or lines of code, for example) would go a long way to answering the effectiveness question.

At this stage, it might be difficult to find a group of developers in-house that aren’t SDL trained, and Microsoft is fully vested in the program such that it wouldn’t allow an untrained developer on a real project, so a new experiment may need to be set up using some arbitrary project created solely for the experiment. Alternatively, Microsoft could measure the differences in development skills after an acquisition and during the transition to SDL-trained developers. Or a final option is to conduct a private benchmarking exercise where the effectiveness is compared among multiple groups.

At this stage, it may be even harder to figure out the effectiveness of an SDL-trained QA group. Presumably, QA training will help the group find more bugs earlier, but if the developers are getting better, then the rate of finding vulnerabilities will go down. There are techniques associated with defect density that could be leveraged to determine this effectiveness level as well.

Creating fewer bugs and finding more bugs early, I believe, are the real expectations of SDL, and finding those numbers would provide much stronger evidence for or against its effectiveness. Not only that, but this information would better frame discussions around ultimate effectiveness of software development: Microsoft is likely to have spent more money than anyone else on its SDL efforts, so the benchmarks provided by the company would serve as an upper limit for expectations.

Blogger: Pete Lindstrom

Microsoft???s Security Development Lifecycle (SDL) is the main product of its Trustworthy Computing Initiative, launched from the now-famous Bill Gates memo in 2002. Six years into the initiative, Microsoft surely must be reaping the benefits of, for example, the well-publicized security training every developer went through.

So, how do we determine whether the SDL is working? Microsoft suggests that this is a simple exercise ??? simply compare the number of public vulnerabilities disclosed for products prior to SDL with similar products developed after SDL. The most recent case was comparing Windows XP SP2 to Vista vulnerabilities in the first year. The count is down and Microsoft provides a quick and easy example of the logical fallacy ???post hoc ergo propter hoc??? which in this case means ???public perception is ripe for deception.???

The biggest problem with Microsoft???s assertion is simply that there are too many variables that are uncontrolled and could just as easily be making the difference. There are too many unknowns related to effort of independent researchers and focus on a specific Microsoft platform. At the very least, Microsoft has done an admirable job in making people feel more secure. (I happen to believe the SDL is working as well, but that belief is a matter of conjecture without strong evidence).

If Microsoft wants to use public vulnerability counts as the ultimate arbiter, it needs to create an environment where independent researchers are encouraged to find bugs. Creating a controlled bounty program for a limited time period would increase incentives and at least provide circumstantial evidence of SDL effectiveness. Interestingly, if the number of vuln counts was higher, it still wouldn???t mean SDL is ineffective,  but the framing of the conversation would be entirely different.

The plot thickens when Microsoft makes claims that spending more time and leveraging external resources are a part of SDL. Whether they are or not, there is a big difference between making programmers more secure developers and simply spending more money on a problem. You don???t really need SDL if the latter is more beneficial.

But if public vulnerability counts are not the answer, what should Microsoft be doing to demonstrate the effectiveness of its SDL? Well, it is much easier to determine causality by controlling for all other variables, and conducting a test of two groups ??? one with SDL training and one without. Comparing vulnerability creation rates per unit output (either developer-hours or lines of code, for example) would go a long way to answering the effectiveness question.

At this stage, it might be difficult to find a group of developers in-house that aren???t SDL trained, and Microsoft is fully vested in the program such that it wouldn???t allow an untrained developer on a real project, so a new experiment may need to be set up using some arbitrary project created solely for the experiment. Alternatively, Microsoft could measure the differences in development skills after an acquisition and during the transition to SDL-trained developers. Or a final option is to conduct a private benchmarking exercise where the effectiveness is compared among multiple groups.

At this stage, it may be even harder to figure out the effectiveness of an SDL-trained QA group. Presumably, QA training will help the group find more bugs earlier, but if the developers are getting better, then the rate of finding vulnerabilities will go down. There are techniques associated with defect density that could be leveraged to determine this effectiveness level as well.

Creating fewer bugs and finding more bugs early, I believe, are the real expectations of SDL, and finding those numbers would provide much stronger evidence for or against its effectiveness. Not only that, but this information would better frame discussions around ultimate effectiveness of software development: Microsoft is likely to have spent more money than anyone else on its SDL efforts, so the benchmarks provided by the company would serve as an upper limit for expectations.