The name reveals some of its creepy appeal: Spykee = Spy Camera. I suppose the nanny you're trying to make sure isn't shaking your baby might be freaked out when it suddenly starts emitting Star Wars music, or such like. Made by Meccano under the Erector brand, its control software is Mac and Windows compatible.

I, for one, welcome our new Spykee overlords--on 15-Oct-2008 when it starts to ship generally.

Silicon Valley project finally gets underway: It's a still a pilot, small, with no promised outcome. And after all this time, a switch of partners, and new parameters, they've still mounted just 20 of 28 access points.

I’d love to have another blog even more specific - “Ok, that Assurance is Evidence Of What, Exactly?

Today he has a great article called:

What’s the matter with Risk Management?

And “in short, it’s everything.” It pretty much sums up why I had to grow to re-evaluate how our industry does risk, risk management, approaches controls & vulnerability and find a new way.   A couple of things jump out at me in reading Chris’ article:

1.)  Just because that Deming cycle sucks and is full of unknowns doesn’t mean “risk” doesn’t exist, nor that it isn’t of primary importance. Nor does it mean that in the absence of model & methodology, we won’t be “doing” risk analysis anyway - just in an ad hoc method and completely from “the gut”.

Our industry calls these unstructured risk analysis “Best Practices”, as it’s an easy and convenient way of sweeping the unknowns under the rug of bureaucracy and enforcing it via peer pressure.

2.)  What this “suckiness” does mean is that your model and methodology aren’t helping you. As Chris intimates, there is too much uncertainty in the inputs for his model (they are, in the language of Bayesians - too subjective to be useful priors).

Take for example how we might be approaching the “controls” part of our analysis.  Chris writes:

“2. What are the controls that we have to employ?
800-53, ISO 27001, PCI, etc.

Still kinda good, but we basically know that ISO is relatively voluntary and NIST supplies a control catalog and not policies. So here we have to take the control catalog, and mash our policies into it.”

I wouldn’t call this “kinda good” at all :)  These control catalogs only provide a hierarchy within which to look for evidence of  our ability to resist an attacker.  They are incapable of making any claim about the effectiveness of the controls when they are operated at 100% efficiency, or more importantly, what % efficiency our specific organization operates at.

Let’s use Chris Hayes’ Initech as our fictional example.

Initech has a control (a back door on a loading dock).  Now the locks on the door are 100% capable of locking the door.  This is different than saying that they are capable of frustrating all but the top 5% of lockpicking burgalars.  It is also diffferent than saying that in a sample of several “walk around audits” the doors are left open 20% of the time (they are not in compliance with policy 100% of the time).  Even worse, that 80% of the time the door is not propped open?  Yeah, tailgating is a known issue.

So we have several different variables here that we need to account for (and it’s just a door).  But the analogy stands that most “risk management” methodologies are “We have a door, yes/no?” And most GRC platforms, when asked for their “opinion” will simply say “door is needed” or, even worse, “a door policy is needed”.

3.)  Criticality and the Source of Value is all messed up in these Risk Management models.

Chris writes:

Someone wants me to tell them which boxes are more critical than others. This is mainly because of budgetary or operational reasons. To which I usually say “All of them, it is a system after all”.

This literally made me laugh out loud.  And this sort of “rate the firewall as Risk = 500 but rate the actual business application as Risk = 157″ thing is also endemic.  Now Chris is very smart here.  He correctly identifies that the value is tied to the business process the systems support, and not to a specific box.  Oh, we scan at the specific box level - but because of the nature of systemic failures - all the boxes in the process are inexorably interrelated.

One of the reasons I really like FAIR is that the losses are quantified (or qualified) based not on some amorphous value of the box or the process itself, but losses are linked to the actions that the threat will take. Take systems in a highly regulated industries as an example.  Usually the most probable losses aren’t due to system compromise per se, but in the disclosure the compromise causes (regulators are a threat source, after all).  But many “risk management” methodologies will say “online banking is worth $2 billion, the value of the systems is therefore $2 billion”.  And suddenly we’re telling executive management that there’s a 60% probability that they’ll lose $2 billion.

4.)  If the primary source of prior information for your “risk management” methodology is a vulnerability scanner - you’re doing it wrong.  Chris writes:

So we ran a scan and now we have a report. A snapshot in time to make all decisions. Where did these vulnerability ratings come from? Do I even know if my system is at risk? What if I spend my time on vulnerabilities that have no threat?

So first, my thoughts are that actual “vulnerability” must be a comparison of the force a threat can apply, and our ability to resist that force (this is a probability statement, btw).

Changing your thinking about vulnerability now helps us understand the problem in several new ways.  First, you can start to divorce yourself from the scanner.  After all, the scanner is simply providing you with current state information that is usually just relevant variance from policy. It doesn’t really tell you about real “weakness in a system” because the system is an interrelated mess of people, processes and IT assets.

5.)  Finally, most “risk management” approaches just *don’t* do a good job of helping us understand the how’s and why’s of managing risk. In the past, I’ve referred to these standards as really being “issue management” because they are at their heart, an act of discovery - a formal process around gathering prior information.  They are not, in and of themselves, capable of linking the issues discovered to the root cause.  And these root causes?  Yeah, they’re the things that create “risk”.  Not a threat, not a vulnerability, not the existence of an asset - the amount of risk that we have stems from our capability to manage it.

So Chris, I completely agree - but I wouldn’t give up yet.  There actually are a few of us who are focused on what you suggest:

Where to go from here: A fundamental revamp of how to deal with Risk. Where risk professionals focus on the treating the sickness and not the symptoms, and come up with some new success/actionable metrics.

Chris, there’s nothing I want to do more than that.

• Moderator: Jim Metzler, Vice President, Ashton, Metzler & Associates
• Rowan Snyder, CIO, KPMG
• David Michael, CIO, United Business Media Group
• Joanna Young, Chief Information Officer, Corporate Information Systems & Enterprise Services, Liberty Mutual

Jim: Is the CIO a technical job anymore? For example, inside Liberty there are business projects with an IT component.

Joanna: We are organized to partner with internal business clients or vendors who provide objectives and business requirements. We strive to figure out the smallest amount of an IT investment we can make to get this to work.

Rowan: We have both. Part of the dilemma is that the thing that sells the best is fear. I don’t want to use that to get business.

Joanna: One good example is security from an application perspective. It’s hard to talk about security investments in business terms. We put it into terms like “this is what it will cost us if we DON’T do this.” For example, a solution for spam required us to do research into what it was costing us overall. Once we put it together, the business was all for it. You have to put your business hat on and think “how can I make this important for a businessperson?” If you can’t, you may need to ask yourself why you’re pushing services on them that they may not need.

Jim: Can you give us insight into business-IT alignment? What about governance?

Rowan: Governance is the hardest part of IT. It’s not like the technology is easy. If it’s a business project with an IT component, I don’t usually get involved. It comes down to overall budget. The infrastructure we own and let people know exactly what it will cost to do it. We are a distributed IT firm, there are multiple groups. This is the most distributed and risk-prone organization I’ve worked in. It can be difficult for the business to exert control. It demonstrates risk, in security, compliance, methodologies, etc.

Joanna: Governance has become a word that nobody wants to use. It suddenly implies that IT is the holder of all the money and they are the ones that get to decide. We stopped using that word and position IT as a strategic business partner.

David: We have a highly decentralized IT set-up. We have about 600 globally and around 40 in the headquarters. We have 10 CIOs for each division, and within each division it is decentralized. We try to run each unit as autonomous. This is a close alignment with IT and business. However, then the problem of how do you have commonality between divisions and collaboration?

Jim: How can you minimize risk in distributed environment using standards and procedures?

David: The reality is it can be impractical for an organization. You end up with a patchwork of platforms and technologies. We have to accept that we’ll have multiple solutions. We can attempt to push a standard, but overall have a much more relaxed approach to manage everything. There is a lot of equality between divisions in what they can choose to purchase.

Joanna: Standards are easier to apply the further down the staff you are. The most important thing with any of this is to understand why you are making the decisions. If there is a process and pros and cons are identified, there is a clear record of why decisions were made.

Audience Poll: Everyone raised their hand that MORE standards were needed.

Audience Question: Are there inefficiencies in the data center in terms of energy and green IT? What are you doing about it?

Joanna: Everyone focuses on cars for carbon footprints. But, it’s really buildings…and then data centers. The data center has the same importance as any other efficiency. They need to be running as cheaply as possible. Corporations have a responsibility to make sure they are energy efficient.

Rowan: We recently did a carbon footprint analysis, and found that half of carbon comes from electricity, with half of that from the data center.

David: Every company does have a responsibility to look at its carbon emission globally. Consider international travel, flying, etc. As much as possible, we are not building data centers. We are using other people’s data centers in an effort to get out of the data center business.

Audience Question: How do you balance the good from standards with agile development and possible roadblocks?

Joanna: Luckily agile development is under the CIO’s control. You can see the lifecycle and savings that occur. When I look, I check what the standards are that I’m measuring by.

Jim: Does web 2.0 have any business meaning in your environment? If so, what are you doing about it?

Joanna: I’ve been in IT for 20 years. It’s another component to business IT investment, and has to be presented as such. As IT professionals we have a responsibility to identify what Web 2.0 is, and then translate to see if there is anything the company should be doing with it. Monitor it based on your current portfolio, and consider its impact.

David: It’s pretty important to our business as a media company. I don’t think it means one thing, it’s a term people use to talk about the web and what’s going on online. From mobile, to ajax, cloud computing or mashups - you can draw multiple conclusions. More and more business is being done online. We have a lot of growth opportunities online.

Rowan: Compliance, security, and privacy issues just explode with Web 2.0.

The Zune doesn't include a Web browser or any Internet focused features; it's not an iPod touch. But you can use Wi-Fi to browse the Zune Marketplace for music and games, and download new songs in programmed channels, music selections created by a variety of artists and stations. Zune offers both music purchases and a subscription for unlimited music listening. The new models range from $149 for an 8 GB flash model to $249 for a 120 GB hard drive-based player.

The feature I'm most interested in is Buy from FM, which leverages the built-in FM tuner and very low-bandwidth data that's already pushed over analog AM/FM. (See my write-up of this feature from last week.) With Buy from FM, when you're listening to radio stations that participate, you'll be able to click a button and buy the song you're listening to if you're connected to a Wi-Fi network. Zune Pass subscribers can download the song at no additional charge. If there's no Wi-Fi network, the song download or purchase is queued.

Wayport's marketing head Dan Lowden said, "Obviously, it's cool because folks who already own a Zune device and just need to do an upgrade will be able to use this just as with any of the new Zune devices that they start selling as soon as possible." (Microsoft may have a little accounting work to do: Sarbanes-Oxley doesn't let you enhance a product in the market without a fee if you realize the revenue all at once.)

The benefit for Wayport is to have yet another hefty but undisclosed fixed sum underlying its fixed infrastructure costs. In the past, Wayport has done deals with Nintendo, ZipIt, and Eye-Fi to allow all devices in a category unlimited access at McDonald's locations. McDonald's obviously gets more customers, or existing customers who spend more time or visit more frequently.

A partnership with a hotspot operator means that Microsoft doesn't have to provide tools and their users endure frustration in joining a network. "We're experts enabling one click to get this network connected," Lowden said. He noted that Wayport has opened test labs to work with manufacturers in Japan, San Francisco, San Diego, and Seattle. "We're working with these guys from day 1 to make sure it's one click to get connected," he said. I'd also note that San Diego happens to be where Qualcomm's headquarters are located, not that Lowden gave me any tip-off there.

And I have to just say: burn, burn, burn on Apple. Despite Apple partnership with AT&T, which relies on Wayport to operate the AT&T-branded hotspot network and resells access to Wayport's own network, iPhone and iPod touch users have no inclusive Wi-Fi service. AT&T slipped a few times and ostensibly opened up their network or released details that iPhone users would gain free hotspot access--like all AT&T's fiber and all its standard and premium DSL customers.

As Wi-Fi becomes an expected part of any handheld gadget, the venues in which Wi-Fi is used multiply beyond cafes and hotels. Lifestyle locations--which could be clothing stores, nightclubs, ski resorts, and the tops of mountains suddenly become places where people want the same kind of access they have at home. Ultima thule is already unwired.

Today, however, I discovered a huge security breach in Google Docs. While I was in my account working on a spreadsheet I suddenly found my Google Doc account listing many documents that did not belong to me. I clicked on one of the documents and the results are in the image below, where my Google Doc session appears to have “crossed over” with another users.

I decided to do a bit more exploring and take a few more screenshots, because I don’t yet know how to reproduct this security breach. The image below show a Google document (fifth from the top) which is not owned by me, “owned by me”. However, when I click on this mysterious “owned by me” document, it is owned by another user. Here is another screenshot below; you can click on the image for the full-screen version.

Again, here is another example of the same security violation with two documents. As above, you can click on the image for a full-screen version.

I contacted the owner of the Google Docs account which I had suddenly and mysteriously “crossed sessions” with today. I asked him if he was in Thailand (since a few of the documents were in Thai) and he said yes, however he say he did not have any Thai language documents in his account. However, as you can see from the screenshot, the Google Docs menu shows this person as “the owner” of a Thai language document. He also mentioned that, today, he saw “wierd documents” in his account that did not belong to him (or “normally” shared with him).

Unfortunately, I was having problems with the Internet connection in my hotel room so I could not continue to investigate the breach. When I logged back in a few hours later, everything was back to normal. So far, all is “normal” and I have not been able to repeat this breach.

I suspect the Google Docs flaw comes from a JavaScript error in how Google manages user sessions. The bottom line is that the security breach is real and dangerous. Your Google Docs, and I suspect other Google applications that use the same session management code, are vulnerable. There may be an underlying XSS vulnerability as well.

Note: Reposted from my original post on the ISC2 blog.

It's become a big deal in IT security, too. Many corporate customers are demanding ROI models to demonstrate that a particular security investment pays off. And in response, vendors are providing ROI models that demonstrate how their particular security solution provides the best return on investment.

It's a good idea in theory, but it's mostly bunk in practice.

Before I get into the details, there's one point I have to make. "ROI" as used in a security context is inaccurate. Security is not an investment that provides a return, like a new factory or a financial instrument. It's an expense that, hopefully, pays for itself in cost savings. Security is about loss prevention, not about earnings. The term just doesn't make sense in this context.

But as anyone who has lived through a company's vicious end-of-year budget-slashing exercises knows, when you're trying to make your numbers, cutting costs is the same as increasing revenues. So while security can't produce ROI, loss prevention most certainly affects a company's bottom line.

And a company should implement only security countermeasures that affect its bottom line positively. It shouldn't spend more on a security problem than the problem is worth. Conversely, it shouldn't ignore problems that are costing it money when there are cheaper mitigation alternatives. A smart company needs to approach security as it would any other business decision: costs versus benefits.

The classic methodology is called annualized loss expectancy (ALE), and it's straightforward. Calculate the cost of a security incident in both tangibles like time and money, and intangibles like reputation and competitive advantage. Multiply that by the chance the incident will occur in a year. That tells you how much you should spend to mitigate the risk. So, for example, if your store has a 10 percent chance of getting robbed and the cost of being robbed is $10,000, then you should spend $1,000 a year on security. Spend more than that, and you're wasting money. Spend less than that, and you're also wasting money.

Of course, that $1,000 has to reduce the chance of being robbed to zero in order to be cost-effective. If a security measure cuts the chance of robbery by 40 percent -- to 6 percent a year -- then you should spend no more than $400 on it. If another security measure reduces it by 80 percent, it's worth $800. And if two security measures both reduce the chance of being robbed by 50 percent and one costs $300 and the other $700, the first one is worth it and the second isn't.

The Data Imperative

The key to making this work is good data; the term of art is "actuarial tail." If you're doing an ALE analysis of a security camera at a convenience store, you need to know the crime rate in the store's neighborhood and maybe have some idea of how much cameras improve the odds of convincing criminals to rob another store instead. You need to know how much a robbery costs: in merchandise, in time and annoyance, in lost sales due to spooked patrons, in employee morale. You need to know how much not having the cameras costs in terms of employee morale; maybe you're having trouble hiring salespeople to work the night shift. With all that data, you can figure out if the cost of the camera is cheaper than the loss of revenue if you close the store at night -- assuming that the closed store won't get robbed as well. And then you can decide whether to install one.

Cybersecurity is considerably harder, because there just isn't enough good data. There aren't good crime rates for cyberspace, and we have a lot less data about how individual security countermeasures -- or specific configurations of countermeasures -- mitigate those risks. We don't even have data on incident costs.

One problem is that the threat moves too quickly. The characteristics of the things we're trying to prevent change so quickly that we can't accumulate data fast enough. By the time we get some data, there's a new threat model for which we don't have enough data. So we can't create ALE models.

But there's another problem, and it's that the math quickly falls apart when it comes to rare and expensive events. Imagine you calculate the cost -- reputational costs, loss of customers, etc. -- of having your company's name in the newspaper after an embarrassing cybersecurity event to be $20 million. Also assume that the odds are 1 in 10,000 of that happening in any one year. ALE says you should spend no more than $2,000 mitigating that risk.

So far, so good. But maybe your CFO thinks an incident would cost only $10 million. You can't argue, since we're just estimating. But he just cut your security budget in half. A vendor trying to sell you a product finds a Web analysis claiming that the odds of this happening are actually 1 in 1,000. Accept this new number, and suddenly a product costing 10 times as much is still a good investment.

It gets worse when you deal with even more rare and expensive events. Imagine you're in charge of terrorism mitigation at a chlorine plant. What's the cost to your company, in money and reputation, of a large and very deadly explosion? $100 million? $1 billion? $10 billion? And the odds: 1 in a hundred thousand, 1 in a million, 1 in 10 million? Depending on how you answer those two questions -- and any answer is really just a guess -- you can justify spending anywhere from $10 to $100,000 annually to mitigate that risk.

Or take another example: airport security. Assume that all the new airport security measures increase the waiting time at airports by -- and I'm making this up -- 30 minutes per passenger. There were 760 million passenger boardings in the United States in 2007. This means that the extra waiting time at airports has cost us a collective 43,000 years of extra waiting time. Assume a 70-year life expectancy, and the increased waiting time has "killed" 620 people per year -- 930 if you calculate the numbers based on 16 hours of awake time per day. So the question is: If we did away with increased airport security, would the result be more people dead from terrorism or fewer?

Caveat Emptor

This kind of thing is why most ROI models you get from security vendors are nonsense. Of course their model demonstrates that their product or service makes financial sense: They've jiggered the numbers so that they do.

This doesn't mean that ALE is useless, but it does mean you should 1) mistrust any analyses that come from people with an agenda and 2) use any results as a general guideline only. So when you get an ROI model from your vendor, take its framework and plug in your own numbers. Don't even show the vendor your improvements; it won't consider any changes that make its product or service less cost-effective to be an "improvement." And use those results as a general guide, along with risk management and compliance analyses, when you're deciding what security products and services to buy.

This essay previously appeared in CSO Magazine.