In any case, if you do pentests, think about all the RECENT cases where you break in to a major corporation through:

• a Solaris system with Internet-exposed telnet with a guessable password OR a telnet vulnerability (circa 1994!)
• an exposed VPN appliance with a manufacturer's administrator password
• a router with default "enable" password
• or, something else entirely - but something that rivals the above example in its unparalleled, unbelievable, abysmal, deep idiocy.

Indeed, many of my pentesting friends still report plenty of such cases (one was also featured in the presentation mentioned above). Whenever I hear about it from a pentester, I always ask:

Do you think "somebody bad" had already passed through the hole you just discovered?

Maybe an hour ago, a day ago - or a year ago?!

I cannot see how the answer can be "no."

Even though pentesters usually don't focus on forensics (no time for this), it is not uncommon to notice "your predecessor's" intrusion traces while you break through systems, "plant flags", change screen backgrounds [for the admins to notice that you've been there...], etc.

Let's think what this situation really means? Here are the choices I see:

1. Nobody discovered the hole - a law of large  numbers (aka "dumb luck") have "shielded" the company from an incident. Yes, Virginia, dumb luck IS a security strategy for some companies... AND it works for them.
2. It was discovered, but not used/abused by the attacker - maybe he was busy hacking other systems, or saved this for later and never came back due to his ADD. Congratulation, you win! The immense power of dumb luck wrapped you in a protective "security" blanket ... again :-)
3. It was discovered; the attacker went in, looked around and compromised a few others systems, but found nothing of interest (no low hanging fruits)  - and he was not a bot herder. Again, you win. Next time you are in Vegas, bet on "00."
4. It was discovered; the attacker went in and deployed a bot on "your" system - given how many botnets are there, this situation is clearly acceptable to many organizations. In this case, dumb luck strategy, apparently, still work: so they use your box to spam and phish somebody else ... big deal!
5. It was discovered; the attacker went in and stole all your credit card information (it is now for sale) - even in this case, the user of "the dumb luck strategy" still "wins" (in some perverse sense)! Unless and until the stolen information IS tracked back to you OR a friendly neighborhood PCI auditor come and jams a broomstick up your ..., you can still continue to be stupid at your leisure and ignore basic security practices.
6. It was discovered; the attacker went in and stole your CEO's Inbox, including the email related to his affair (it is now on CNN) - now, in this case, you lose AND it is time to stop being stupid! Welcome to the "0wned world." Time to launch (relaunch?) your security program and get serious.

What does this teach us about RISK? The lesson here is important:

• For a security professional, an Internet-exposed system with "root/root" is an obvious HUGE risk!
• For your boss's boss's boss, it is NOT!

This is exactly why I think that the most critical problem in security today is METRICS. Metrics that a) work AND mean something to decision makers and b) can be clearly communicated to said decision makers [BTW, a) and b) are two separate problems.] Metrics that cover not only threats and vulnerabilities we face, but also the effectiveness of security countermeasures we deploy. Metrics you can act on - and ones your boss (and his boss) will act on. Metrics that lead to correct decisions about which risks to accept, which to  mitigate (all while knowing with what efficiency such mitigation occurs) and which to transfer.

Until that time, the dreaded "C-word" (compliance) will trump "the other C-word" (common sense) as a driver for security ... and we will continue to live in the "0wned world."

Possibly related posts:

• Risk vs Risk 

In a tale rich with incongruities, the Communist-run government of West Bengal State invited the Tata Group, a symbol of Indian capitalism, to set up its plant in an area called Singur. It acquired 1,000 acres from farmers on the company’s behalf.
As the project advanced, some farmers who had sold their land demanded it back. The main state-level opposition party, the Trinamool Congress, led protests demanding that the land be returned. Most people sympathetic to Tata accused the opposition of inducing the farmers to protest, while Tata’s critics said the farmers had legitimate grievances.

The issue simmered for months. But in recent days, protesters began surrounding the plant, blocking roads and preventing Tata workers from reaching the plant. “The existing environment of obstruction, intimidation and confrontation has begun to impact the ability of the company to convince several of its experienced managers to relocate and work in the plant,” Tata said in a statement on Tuesday.

The halt to the plant has caused many Indian business people to warn of a chilling effect on investment in the country. It is also unclear how Tata will be able to keep the Nano’s cost so low, since part of the affordable price reflects the company’s savings on the land in Singur.

There is a fundamental asymmetry between state and markets. It is easier to create markets than it is to create state capacity or to prevent its deterioration. Creating markets is a lot about letting go, establishing a reasonable policy framework, and allowing the natural hustling instinct to take over. In other words, hustling is the natural state. Building state capacity, on the other hand, is quite different. It involves overcoming collective action problems, mediating conflict, creating accountability mechanisms where outputs are multiple and fuzzy and links between inputs and outputs murky, and contending with the deep imprints of history. In Weber’s memorable words, building public institutions is like the “slow boring of hard boards”.

In that light, China’s task of improving its private sector seems easier to accomplish than India’s task of arresting institutional decline. So, while China and India can probably both count on more years of high growth, the odds still favour China pulling off that feat than India. That, and not just the meagre medal tally, should be what India mulls over after the Beijing Olympics.

It's easier to liberalise a functional state than it is to functionalise a dysfunctional one, of any ideological stripe.

It's become a big deal in IT security, too. Many corporate customers are demanding ROI models to demonstrate that a particular security investment pays off. And in response, vendors are providing ROI models that demonstrate how their particular security solution provides the best return on investment.

It's a good idea in theory, but it's mostly bunk in practice.

Before I get into the details, there's one point I have to make. "ROI" as used in a security context is inaccurate. Security is not an investment that provides a return, like a new factory or a financial instrument. It's an expense that, hopefully, pays for itself in cost savings. Security is about loss prevention, not about earnings. The term just doesn't make sense in this context.

But as anyone who has lived through a company's vicious end-of-year budget-slashing exercises knows, when you're trying to make your numbers, cutting costs is the same as increasing revenues. So while security can't produce ROI, loss prevention most certainly affects a company's bottom line.

And a company should implement only security countermeasures that affect its bottom line positively. It shouldn't spend more on a security problem than the problem is worth. Conversely, it shouldn't ignore problems that are costing it money when there are cheaper mitigation alternatives. A smart company needs to approach security as it would any other business decision: costs versus benefits.

The classic methodology is called annualized loss expectancy (ALE), and it's straightforward. Calculate the cost of a security incident in both tangibles like time and money, and intangibles like reputation and competitive advantage. Multiply that by the chance the incident will occur in a year. That tells you how much you should spend to mitigate the risk. So, for example, if your store has a 10 percent chance of getting robbed and the cost of being robbed is $10,000, then you should spend $1,000 a year on security. Spend more than that, and you're wasting money. Spend less than that, and you're also wasting money.

Of course, that $1,000 has to reduce the chance of being robbed to zero in order to be cost-effective. If a security measure cuts the chance of robbery by 40 percent -- to 6 percent a year -- then you should spend no more than $400 on it. If another security measure reduces it by 80 percent, it's worth $800. And if two security measures both reduce the chance of being robbed by 50 percent and one costs $300 and the other $700, the first one is worth it and the second isn't.

The Data Imperative

The key to making this work is good data; the term of art is "actuarial tail." If you're doing an ALE analysis of a security camera at a convenience store, you need to know the crime rate in the store's neighborhood and maybe have some idea of how much cameras improve the odds of convincing criminals to rob another store instead. You need to know how much a robbery costs: in merchandise, in time and annoyance, in lost sales due to spooked patrons, in employee morale. You need to know how much not having the cameras costs in terms of employee morale; maybe you're having trouble hiring salespeople to work the night shift. With all that data, you can figure out if the cost of the camera is cheaper than the loss of revenue if you close the store at night -- assuming that the closed store won't get robbed as well. And then you can decide whether to install one.

Cybersecurity is considerably harder, because there just isn't enough good data. There aren't good crime rates for cyberspace, and we have a lot less data about how individual security countermeasures -- or specific configurations of countermeasures -- mitigate those risks. We don't even have data on incident costs.

One problem is that the threat moves too quickly. The characteristics of the things we're trying to prevent change so quickly that we can't accumulate data fast enough. By the time we get some data, there's a new threat model for which we don't have enough data. So we can't create ALE models.

But there's another problem, and it's that the math quickly falls apart when it comes to rare and expensive events. Imagine you calculate the cost -- reputational costs, loss of customers, etc. -- of having your company's name in the newspaper after an embarrassing cybersecurity event to be $20 million. Also assume that the odds are 1 in 10,000 of that happening in any one year. ALE says you should spend no more than $2,000 mitigating that risk.

So far, so good. But maybe your CFO thinks an incident would cost only $10 million. You can't argue, since we're just estimating. But he just cut your security budget in half. A vendor trying to sell you a product finds a Web analysis claiming that the odds of this happening are actually 1 in 1,000. Accept this new number, and suddenly a product costing 10 times as much is still a good investment.

It gets worse when you deal with even more rare and expensive events. Imagine you're in charge of terrorism mitigation at a chlorine plant. What's the cost to your company, in money and reputation, of a large and very deadly explosion? $100 million? $1 billion? $10 billion? And the odds: 1 in a hundred thousand, 1 in a million, 1 in 10 million? Depending on how you answer those two questions -- and any answer is really just a guess -- you can justify spending anywhere from $10 to $100,000 annually to mitigate that risk.

Or take another example: airport security. Assume that all the new airport security measures increase the waiting time at airports by -- and I'm making this up -- 30 minutes per passenger. There were 760 million passenger boardings in the United States in 2007. This means that the extra waiting time at airports has cost us a collective 43,000 years of extra waiting time. Assume a 70-year life expectancy, and the increased waiting time has "killed" 620 people per year -- 930 if you calculate the numbers based on 16 hours of awake time per day. So the question is: If we did away with increased airport security, would the result be more people dead from terrorism or fewer?

Caveat Emptor

This kind of thing is why most ROI models you get from security vendors are nonsense. Of course their model demonstrates that their product or service makes financial sense: They've jiggered the numbers so that they do.

This doesn't mean that ALE is useless, but it does mean you should 1) mistrust any analyses that come from people with an agenda and 2) use any results as a general guideline only. So when you get an ROI model from your vendor, take its framework and plug in your own numbers. Don't even show the vendor your improvements; it won't consider any changes that make its product or service less cost-effective to be an "improvement." And use those results as a general guide, along with risk management and compliance analyses, when you're deciding what security products and services to buy.

This essay previously appeared in CSO Magazine.