As a reference, you may have seen this briefing, one of many where I show these functional relationships, Mythbusters: Event Stream Processing Versus Complex Event Processing, from DEBS2007.  For example slide 23 shows the functional relationship between events, pre-processing, event tracking, situational detection, historical patterns (the output of BI tools, for example), visualization and business process management.

In Faithful Representation, Richard Veryard reminds his readers that the most challenging part is in the situation models (not the system integration).  Unfortunately, by accident, Richard incorrectly attributes Opher Etzion’s “first order situation model approximation” to both Opher and I in this quote from Richard’s post, “a simple situation model of complex events, in which events (including derived, composite and complex events) represent the “situation”.   

Actually, that simple situation model above is Opher’s, not mine.  I have offered a more general and comprehensive (first draft) situation model, in A Simple Situation Model for Complex Events based on a cognitive situation model used by researchers at the University of Notre Dame.  I do not believe that complex events and situations can be modelled accurately using Opher’s simple model of derived, composite and complex events.   This model is overly simple, in my opinion. to represent the vast majority of CEP classes of problems, perhaps explaining why Opher and I do not agree on the state-of-the-art of CEP.  Opher tends to view CEP as mostly an extension of active database technology where I see CEP as a technology that is much more closely aligned with the cognitive models represented in the art-and-science of multi-sensor data fusion (MSDF).  

Complex events represent situations, and situations must be accurately modelled if we are going to accurately detect them in real-time.  If your business cannot model a complex event (situation) then it does not matter what software you buy, how much money you spend, or what event processing and integration platform you use.   The models are hard.  The system integration is relatively easy.

The secret sauce is the situation and complex event models.

As mentioned here a few times, it does not matter how fast you process events in real-time, if your model is wrong, you just detect the wrong thing very fast.  This is very bad and quite dangerous.  You will make bad decisions fast.  You will waste time, money and resources.

This is why CEP benchmarks should be based on accuracy in situation detection, not in latency and other low-level performance metrics.   First, get the models right; then refine to detect faster, if speed is required.   What has happened in CEP to date, is that the models are so simple, they do not really detect complex events, they just process and act on simple events that are easy to model. 

1. Person A has position X.

2. Person B ignores X and instead presents position Y.
Y is a distorted version of X and can be set up in several ways, including:

1. Presenting a misrepresentation of the opponent’s position and then refuting it, thus giving the appearance that the opponent’s actual position has been refuted.
2. Quoting an opponent’s words out of context — i.e., choosing quotations that are not representative of the opponent’s actual intentions.
3. Presenting someone who defends a position poorly as the defender and then refuting that person’s arguments, thus giving the appearance that every upholder of that position, and thus the position itself, has been defeated.
4. Inventing a fictitious persona with actions or beliefs that are criticized, such that the person represents a group of whom the speaker is critical.
5. Oversimplifying an opponent’s argument, then attacking the simplified version.

3. Person B attacks position Y.

4. Person B draws a conclusion that X is false/incorrect/flawed.
This sort of “reasoning” is fallacious because attacking a distorted version of a position simply does not constitute an attack on the position itself.

For example, there has been some lively discussions recently around the notion that CEP is overhyped.

Debate:      “CEP is Overhyped.”

Person A:   “CEP has been overhyped.”

Person B:     “CEP is just hype.”

The point of the discussion by person A was to point out that CEP has been overhyped.  Person B has exaggerated this to a harder to defend position, “CEP is mere hype.” or “CEP is just hype.”

From the customer perspective, I don’t think that fallacies and red-herring arguments are good for CEP.   Believe me, if we could take an “out of the box” stream processing rules-engine and bolt it on to a network and insure a client it would detect complex fraud, or diagnose network faults accurately, and not put my entire professional reputation on the line, I would do it in a heartbeat.

It is not the speed of the an engine which makes a good CEP engine, it is the capability of the analytics to deliver high-quality, high-confidence complex event detection in real-time.

Greg Hoglund

Cory Feldman

a

Greg Hoglund = Cory Feldman

First, let me disclose something - my frantic efforts with the Paint allow me to proudly proclaim: I am a certified, trusted "Good Guy":

Good guys, let me tell you, do not need any controls placed on them; they are "trusted." Don't you have to trust somebody? Why not trust a sysadmin, for example?

So, what about controls? Ah, glad that you asked! "Controls" are for the bad guys; they are in place to prevent the bad guys from doing "an unspeakable evil" (tm) :-) on you. On the other hand, good guys are doing "the right thing" every time - why monitor them? It goes without saying that nobody ever moves between these groups, especially, not from "good guys" to "bad guys."

As I am rambling about this, many of my security-minded readers are wondering "what is Anton up to? Isn't it kind of OBVIOUS that controls are for everybody?" Controls know no good/bad! For example, a network control, say a NIPS, will block malicious web access due to a typo in a URL (by - gasp! - a good guy) or due to determined malicious hacking.

I think a few of my readers have watched one too many "Batman" movies and have acquired the dark side of the "IT hero" mentality." How about getting an "IT employee" mentality? If your boss is an idiot (and Terry's managers definitely seem pretty far gone in that direction...), than your "heroic duty" is to let them impale themselves on a sword of their idiocy, not to commit crimes (even if cybercrimes) to prevent that idiocy. Really, go find another job if you do not like the environment; good admins are needed in many places. For example, if your boss insists on posting all VPN passwords for all users publicly out of his sheer and unfathomable stupidity, it is your duty to tell him that it is "a very bad idea" - and not to change all passwords and not let him see it. "Doing you job" despite your boss and despite the law just doesn't work...

In other words, I want a banker making policy decisions at a bank, not a sysadmin. If a banker makes a wrong decision, his will suffer. If he is an idiot, he will most likely make the wrong decision. However, it is NOT the admin's decision to make - he does not "own" the business.  BTW, the fact that it is a city, not a bank, and it is taxpayer funded, does not change it.

Am I "anti-admin" for saying that admins should not run the business?  Am I "anti-admin" for saying controls (at least logging/auditing) on administrator activities are needed?  You call it "anti-admin", I call it common sense!!  Pray tell me, what makes admins float above accountability, control and  IT governance?

Please also read what Randy Smith said about this issue; a lot of good thoughts that I agree with.

Now I would like to respond to specific comments from my readers:

"What rankles your readers is how blithely you imply this problem has a simple or effective solution. It doesn't, all the processes or tools you advocate can do is speed up the time it takes to detect the lock-out, but not actually prevent it - i.e. they are ineffective at tackling the primary problem."

That is correct; the rogue admin problem has NO simple solution. You might prevent some (few, really) things, you might log some of them and then figure what happened, but there is no simple solution (it goes without saying that "just trust them" is NOT a solution...)

"We all know companies run without sane risk management all the time and are rarely held accountable in America. What makes you think anyone is "screwed"?"

Well, this is a good point; maybe I let my idealistic side take over. But, come on, just the fact that bad IT governance is somewhat common, doesn't make it right!

"Now ask yourself who is "screwed" by one person at a small company having all access and no accountability on a network. That's how I run my home network. Big deal."

Nobody is. I addressed it here. The risk is acceptable for smaller environments, usually. I don't have an overseeing body set up to control my home passwords :-)

"You seem to forget that sometimes the management just has to trust somebody. "

Addressed above.

"Chuvakin, you're a tool. Given the recent idiocy of the releasing of the VPN names and codes, it obviously shows that any sort of detest that Childs had against his superiors at the city were justified."

The fact that his bosses are idiots (which seems fairly well established!) does not make him right!

Bad boss + admin out of control =/= right :-)

"This is not a private organization. His superiors don't own the company and are NOT entitled to the data. We are, the taxpayers. And as a California taxpayer I fully support someone with the paranoia and technical skill of Terry Childs over a group of bureaucrats who release secure information to the public."

Properly evaluating this statement requires a law degree. Thus, no comment. Bureaucrats suck, but rogue admins are not a solution to that. Really!

"The guy was doing his job and doing it incredibly well, and keeping it out of the hands of those who, given their most recent choices, would bring potential disaster to the city."

He was NOT, unless crime is part of his job :-) Also, see comments on "IT heroes" above. If your boss is an idiot AND you don't like it, quit.

"Anton Chuvakin seems to think that all admins should be kept underneath management's boot at all times. [...]  Managers can't and don't understand what we do, and thus eventually come to the conclusion that we can't be trusted with our own knowledge. [...] Perhaps it's human nature to fear what you don't know or understand -- and that's why management can develop a fear of their own employees."

You say 'fear of employees', I say "insider risk management." You say "trust employees", I say "trust but [be able to] verify (=log)"

"his blog leads the casual reader to infer that their businesses are in danger of being hijacked by disgruntled Sys Admins and that isn’t the case." (from here)

Eh, not all businesses, but some businesses - definitely (hmm, see Terry Childs story or other published insider attack cases, all the way back to Omega Engineering case and maybe all the way back to ancient history)

"I despise people like Terry Childs, but despise Chicken Little’s like Anton Chuvakin even more." (from here)

You say  I am 'chicken little', I say "if your boss ignores insider risk management, he is stupid and deserves his business to fail."  I also add "if you think admins are 'above the law', you have a good chance of 'turning rogue' yourself AND then ending in jail."

Finally, this and my other posts about the case are inspired by on the media reporting; I possess no "insider knowledge" on this case  whatsoever.

Possibly related posts:

• "On Doomsaying (Terry Childs case)"
• "So ... Am I? Maybe I Am!"