Two of the more controversial topics in information security are return on security investment (or ROSI) and the related subject of security metrics. I will talk to ROSI in this column and metrics in the next one.

There are a number of opponents to the ROSI approach. One is Jos Pols who, in his recent article “The Fallacy of Information Security ROI” in the February 2008 issue of the ISSA Journal (membership required to access link resource), claims that one cannot have a return where there is no income. In my opinion, this is an overly restrictive view of the meaning of the word “income.” The avoidance of potential losses redounds to the bottom line, as does revenue, so that a cost saving is a return on an investment, just as much as a corresponding revenue enhancement would be.

Pols prefers to use the term “insurance” in referring to spending on security. He asks the question “How do you value what was not lost?” While I am not saying that it is easy to measure the losses from a potential security breach, I do believe that one can come up with workable estimates of the magnitude and probability of losses and then calculate an expected loss number as the product of estimates of the size of loss and probability of occurrence. Better than point estimates is the inclusion of probability distributions and Monte Carlo simulations. You can read how this may be achieved in Douglas Hubbard’s excellent book, How to Measure Anything: Finding the Value of Intangibles in Business (John Wiley, 2007).

Donn Parker, for whom I have the highest regard, also argues against the ROSI approach. In an article in the May 2006 issue of the ISSA Journal (membership required to access link resource) with the title “Making the Case for Replacing Risk-Based Security” he states that “ … security based on risk management, risk reduction, and risk assessment is a failed concept.” He argues that it is not possible to measure the probability or the magnitude of infrequent but very damaging security events. But Hubbard specifically addresses IT security and provides confirming evidence as to the viability of the probabilistic approach from his work at a major government agency. On page 47 of his book, Hubbard writes: “When we say that security has improved, we generally mean that particular risks have decreased … a reduction in risk must mean that the probability and/or severity (loss) of a certain list of events decrease.”

So what is the answer? In one sense, detractors such as Pols and Parker are correct in questioning the feasibility of measuring what might be lost as a result of a security breach. It is neither a simple nor obvious task and requires some measure of training (or “calibration,” using Hubbard’s term). However, I think that it is not only possible, but imperative, to come up with reasonable estimates of the probability and magnitude of potential losses in order to achieve reductions in risk. Not doing so precludes an important method from the risk manager’s toolbox. I address this whole issue in my chapter on ROSI in the book Managing Information Assurance for Financial Services (IGI, 2007). In the chapter, I demonstrate that, if one does in fact derive reasonable estimates of potential gains and losses of various approaches, one can then go on to optimize the mix of security measures to be taken.

Risk reduction and avoided losses are not easy concepts to apply, but that does not mean that they should be discarded. A relatively small effort in this area can lead to major benefits and savings. Look into it. You’ll be pleasantly surprised.