As a reference, you may have seen this briefing, one of many where I show these functional relationships, Mythbusters: Event Stream Processing Versus Complex Event Processing, from DEBS2007.  For example slide 23 shows the functional relationship between events, pre-processing, event tracking, situational detection, historical patterns (the output of BI tools, for example), visualization and business process management.

In Faithful Representation, Richard Veryard reminds his readers that the most challenging part is in the situation models (not the system integration).  Unfortunately, by accident, Richard incorrectly attributes Opher Etzion’s “first order situation model approximation” to both Opher and I in this quote from Richard’s post, “a simple situation model of complex events, in which events (including derived, composite and complex events) represent the “situation”.   

Actually, that simple situation model above is Opher’s, not mine.  I have offered a more general and comprehensive (first draft) situation model, in A Simple Situation Model for Complex Events based on a cognitive situation model used by researchers at the University of Notre Dame.  I do not believe that complex events and situations can be modelled accurately using Opher’s simple model of derived, composite and complex events.   This model is overly simple, in my opinion. to represent the vast majority of CEP classes of problems, perhaps explaining why Opher and I do not agree on the state-of-the-art of CEP.  Opher tends to view CEP as mostly an extension of active database technology where I see CEP as a technology that is much more closely aligned with the cognitive models represented in the art-and-science of multi-sensor data fusion (MSDF).  

Complex events represent situations, and situations must be accurately modelled if we are going to accurately detect them in real-time.  If your business cannot model a complex event (situation) then it does not matter what software you buy, how much money you spend, or what event processing and integration platform you use.   The models are hard.  The system integration is relatively easy.

The secret sauce is the situation and complex event models.

As mentioned here a few times, it does not matter how fast you process events in real-time, if your model is wrong, you just detect the wrong thing very fast.  This is very bad and quite dangerous.  You will make bad decisions fast.  You will waste time, money and resources.

This is why CEP benchmarks should be based on accuracy in situation detection, not in latency and other low-level performance metrics.   First, get the models right; then refine to detect faster, if speed is required.   What has happened in CEP to date, is that the models are so simple, they do not really detect complex events, they just process and act on simple events that are easy to model. 

“PCI Risk Management”

Now usually when I see a term like this, I can only imagine that such things are the byproduct of rapidly decaying brain cells.  In my mind I imagine there’s a conference room somewhere with some marketing types all hopped up on the vapors from industrial solvents spewing terms like “protectivity” or “advanced adaptive deep packet inspection” into the ether with all the acumen of an intoxicated long-horned bovine.

BUT

I thought about this, and it’s really not a bad idea - depending on how you define it.  Now I just couldn’t make the effort to read how the author used the term (I have a short pain threshold), but here’s my thoughts on what PCI Risk Management should be.  If we define Risk as the probable frequency and probable magnitude of future loss.

Then managing the risk inherent in PCI DSS compliance could mean:

1.)  The expected frequency of being out of compliance and how much that will cost us.

Because let’s face it - being in or out of PCI compliance is still a subjective judgment.  First, we have what our ever-qualified assessor says.  But in the case of an incident, it’s really someone else who has the final say in whether or not we were “compliant” at the time of incident.  So we can only know for certain if we’re in compliance after the fact - i.e. after there’s an incident.  So if we cannot really “know” if we’re compliant - we have a probability problem to solve!  Sounds like “risk” or “secure” doesn’t it?

So we could view the PCI as a threat community to deal with.  This gives us the first angle of what we could call PCIRM (this sort of term begs to be it’s own acronym, doesn’t it?) - the simple creation of a probability statement that says there is some belief that we could be found out of compliance - regardless of our efforts - and the calculation of what the impact would be to our organization (like defending frivolous 90 bajillion $ law suits from tiny financial institutions whose lawyers smell blood in the water).  Note that you may or may not want to add the value of the money and time spent on PCI compliance into your loss magnitude calculations.  It’s a sunk cost at that point.

However, there’s another side of the coin.  We can find out the risk of being out of compliance, but is there risk in being *in* compliance?  I think there is.  So our second aspect of PCI Risk Management might be:

2.)  The expected frequency of being in compliance and how much that will cost us.

An alternate view of how we could view the Payment Card Industry as a threat community would involve trying to figure out the probable frequency with which they will make onerous demands of our security budget, and the impact of those demands.

Now note that we would have a “secondary risk” to measure here.  I’m thinking that it’s not improbable that our PCI efforts may not be the most efficient use of or time and money.  So if we’re spending money on what PCI says we must, and neglecting areas of our IRM landscape that would actually reduce organizational risk more than those PCI efforts - then PCI compliance is costing us some real value by reducing our capability to manage real risk.  However,  and it’s quite a long tail event but, imagine that we’re unlucky and an incident happens!  This incident may become, in no small probability, the byproduct of PCI requirements.  Being diligent in risk management, we might want to study this likelihood, too.

So there you have it.  In both cases PCI Risk Management involves looking at the Payment Card Industry as a threat community, and determining the probable impact of having to deal with PCI DSS.

Now if you’ll excuse me, I have a white paper to write and I’m fresh out of acetone-based paint remover.

POST SCRIPT

I should make it clear that Risk Management should (and is) obviously being performed by those with PCI concerns.  PCI, if you will, is simply a sort of ISMS.  And the development of an ISMS can assist IT management with the process of developing metrics and analysis concerning the organizations capability to manage risk.  There’s nothing wrong with PCI in this regard.

But I figured I should make the effort to read what the author was advocating, and the document this “PCI Risk Management” term was drawn from was really a set of “best practices” for PCI and “best practices” above and beyond what PCI requires.  This is not risk management (and no, adding “risk assessment” - in quotes because the author is really referring to vulnerability management - to the list of best practices doesn’t make it risk management, either).  It is more witch-doctory.

This blog gives us a place to talk about our experiences from using the SDL here at Microsoft and hopefully provide useful information that will help you implement it more effectively at your company.  So, I would encourage you to use the Comments section at the bottom of each post to ask questions, give us feedback, or request other topics for us to cover.

Some quick definitions before we dive in. I’ve been using the imagery of learning to “crawl, walk and run” as a way to provide some basic starting points that would move your organization toward implementing the Security Development Lifecycle (SDL).  “Walking” is the point where your security development practices become a lifecycle – a repeatable, reusable process that makes security a part of your development culture. To relate the analogy to SDL a bit more closely, think of crawling as the “SD” in SDL. For this post, we’ll continue to talk about walking – or adding the “L” in SDL.

Let’s jump into another component for adopting the Microsoft SDL to expand your own Security Development Lifecycle.

Expand Security Training

Once you have management approval, it is necessary to gain grassroots acceptance of the changes – at the developer, QA/test, and PM levels. If you have been “crawling”, you have probably already implemented some sort of discipline-specific training around things like threat modeling, using compiler defenses, and fuzz testing. Now that you are building a lifecycle, your goal for security training should expand. Security training should be about creating an environment where writing secure software is everyone’s mission. While security training should be undertaken with the goal of understanding security issues and how to address them, good training (and instructors) will also explain why solving security problems is in their best interests and create an environment where they know voicing security concerns is encouraged.

Training has been one of the earliest and most important elements of the SDL at Microsoft. From our experience, we learned that the most effective approach is to divide your training into two tracks: general security principles and role-specific security practices. Before I jump into the details, I want to encourage you to also read Shawn Hernan’s very good post about SDL training that highlights some of the ways to make security training effective.

The general security principles should explain why security is important, how you define security requirements, the process you will use for writing and validating secure code, and how security relates to each phase of the lifecycle or unique roles contributing to the development process. A key factor for building a development lifecycle is educating your individual contributors on the value of investing in security. Of course changing culture takes time, but using the opportunity of structured training to explain your principles will be one of your most effective platforms for influencing change.

At this point in your organizational maturity, you are also beginning to expand your security thinking by focusing on each role in the development process. Discipline-specific security training is where you dig into the details of implementing a Security Development Lifecycle.

·         The developer needs to understand the practical details of how to write code securely, how to set compiler flags, what a security code review means, how to avoid using banned APIs, and what tools are available for them to perform security analysis before checking in their code.

·         The QA/tester needs to know how to set security rules in test tools, how to perform penetration testing, and what the security quality criteria is for your product, or how to file a security bug.

·         The PM needs to understand how to define measurable goals or how security policies can be factored into feature design.

·         The business decision maker of your organization should understand how to track security metrics alongside other product measurements or how security policy plays a critical role in the overall quality and value of your product.

·         Finally, it is critical for the employees occupying all job roles to understand the value of threat modeling – both as a tool for understanding threats early in the design phase and throughout the development process as a key barometer to the security pulse of your product.

Discipline-specific training will be the place to address these issues for your organization. In case you were wondering, all job roles should be required to attend both types of security training before working on your product.

Our new SDL website [http://www.microsoft.com/sdl] will be a very good place to watch for future training materials. The SDL Training and Resources page has some useful material up now and more will be coming in the future.

That’s Part Two. In Part Three, I will discuss the important “walk” components of formalizing security requirements and reusing threat models and attack surface reviews. Then we will close with the discussions on conducting final security reviews, and managing post-release documentation.

I’d like to hear if anyone is using the concept of “crawling” and “walking” to implement SDL in your company.

?        Do you provide security training to your employees today?

?        Do these additional training topics make sense in your organization?

?        What would you add to this that is unique to your application or company?