To solve a complex problem in the blackboard-style, the intelligent agents cooperate as functional specialists, observing updates to the blackboard and self-actualizing in an event driven process) when there is new information to process.  Agents continually update the blackboard with partial solutions when the agents capabilities for processing match the state of the blackboard. 

The blackboard architecture is a distributed computing model for a metaphor describing how people work together to collaboratively solve a problem around a blackboard (whiteboard in todays lingo).   For example, one person is standing at the whiteboard working on a solution while three other people are sitting (or standing) around watching.   One of the observers sees new information on the whiteboard, thinks of how he (or she) can contribute, and then jumps up, takes the whiteboard marker from the person working, and adds to the solution.  This process is repeated in various scenarios.  

The blackboard architecture can be very effective in solving complex distributed computing problems, including event processing problems; however, scheduling the self-actuating agents can be a key challenge.   Another core challenge is how to model and manage the blackboard itself, especially in distributed blackboard architectures.  

John McManus, former CTO of NASA, wrote an excellent PhD dissertation in 1992,  Design and Analysis Techniques for Concurrent Blackboard Systems, at the College of William and Mary, addressing challenges in BB systems.

The table below lists two books that focus on blackboard architecture:

One of the thought leaders in blackboard architecture is Daniel D. Corkill a professor at the University of Massachusetts Amherst. 

Blackboard architecture is relevant to the field of event processing, and in particular complex event processing.   I will go into more details in future blog posts on this topic, including how blackboard architectures relate to grid computing, distributed object caching (of the blackboard), and CEP.

A few folks have tried to tie “maturity” to “if the code is robust” or “if the product has certain product features.” The way we have addressed this emerging controversy over at The CEP blog is to center the discussion around the Gartner Hype Cycle, which is a pretty good model for representing the maturity, adoption and business application of specific technologies.

On CEP Maturity and the Gartner Hype Cycle

Since many folks work very closely with Gartner, I expect they are keenly aware of Gartner’s view on technology adoption maturity models and their definitions. Just for our readers who might not be as familar, I quote Gartner’s definitions below to be complete from here:

A hype cycle is a graphic representation of the maturity, adoption and business application of specific technologies. The term was coined by Gartner[citation needed], an analyst/research house, based in the United States, that provides opinions, advice and data on the global information technology industry.

Since 1995, Gartner has used hype cycles to characterize the over-enthusiasm or “hype” and subsequent disappointment that typically happens with the introduction of new technologies. Hype cycles also show how and when technologies move beyond the hype, offer practical benefits and become widely accepted. According to Gartner, hype cycles aim to separate the hype from the reality, and enable CIOs and CEOs to decide whether or not a particular technology is ready for adoption. A longer-term historical perspective on such cycles can be found in the research of the economist Carlota Perez.

A hype cycle in Gartner’s interpretation comprises 5 steps:

“Technology Trigger” — The first phase of a hype cycle is the “technology trigger” or breakthrough, product launch or other event that generates significant press and interest.

“Peak of Inflated Expectations” — In the next phase, a frenzy of publicity typically generates over-enthusiasm and unrealistic expectations. There may be some successful applications of a technology, but there are typically more failures.

“Trough of Disillusionment” — Technologies enter the “trough of disillusionment” because they fail to meet expectations and quickly become unfashionable. Consequently, the press usually abandons the topic and the technology.

“Slope of Enlightenment” — Although the press may have stopped covering the technology, some businesses continue through the “slope of enlightenment” and experiment to understand the benefits and practical application of the technology.

“Plateau of Productivity” — A technology reaches the “plateau of productivity” as the benefits of it become widely demonstrated and accepted. The technology becomes increasingly stable and evolves in second and third generations. The final height of the plateau varies according to whether the technology is broadly applicable or benefits only a niche market.

The term is now used more broadly in the marketing of new technologies.

We used the Gartner Hype Cycle in Two-Thirds of Our Readers Say CEP is Still Immature as a basis for having interested readers vote, and in a unscientific straw poll, the readers indicated that, in their view, CEP is still immature.

At the CEP Blog we ground our discussions and terminology on maturity in Gartner’s models on maturity, and we ground our discussions on event processing in the art-and-science of a long standing domain in event processing - multisensor data fusion (MSDF).

1. The number of potential terrorist targets is essentially infinite. 2. The probability that any individual target will be attacked is essentially zero. 3. If one potential target happens to enjoy a degree of protection, the agile terrorist usually can readily move on to another one. 4. Most targets are "vulnerable" in that it is not very difficult to damage them, but invulnerable in that they can be rebuilt in fairly short order and at tolerable expense. 5. It is essentially impossible to make a very wide variety of potential terrorist targets invulnerable except by completely closing them down.

1. Any protective policy should be compared to a "null case": do nothing, and use the money saved to rebuild and to compensate any victims. 2. Abandon any effort to imagine a terrorist target list. 3. Consider negative effects of protection measures: not only direct cost, but inconvenience, enhancement of fear, negative economic impacts, reduction of liberties. 4. Consider the opportunity costs, the tradeoffs, of protection measures.

This paper attempts to set out some general parameters for coming to grips with a central homeland security concern: the effort to make potential targets invulnerable, or at least notably less vulnerable, to terrorist attack. It argues that protection makes sense only when protection is feasible for an entire class of potential targets and when the destruction of something in that target set would have quite large physical, economic, psychological, and/or political consequences. There are a very large number of potential targets where protection is essentially a waste of resources and a much more limited one where it may be effective.

The good news is that due to the nature of this problem, it is extremely difficult to determine the vulnerability merely by analyzing the patches; a common technique malicious individuals use to figure out security weaknesses.

The typical response I heard was “what do you mean, it can’t be reverse engineered? I’ll just look at the diffs!”

In hindsight, after examining the BIND diffs (yes, I did it too) and discussing with colleagues, all most people saw was UDP source port randomization and a better PRNG for generating the transaction ID, the latter of which would appear to be related to Amit Klein’s cache poisoning attack from about a year ago.

What Rich was really saying is that you can reverse engineer the patch until you’re blue in the face, but that won’t reveal the specifics of the vulnerability.

Dan’s blog post this morning appeared to confirm that interpretation:

DJB was right. All those years ago, Dan J. Bernstein was right: Source Port Randomization should be standard on every name server in production use.

There is a fantastic quote that guides a lot of the work I do: Luck is the residue of design. Dan Bernstein is a notably lucky programmer, and that’s no accident. The professor lives and breathes systems engineering in a way that my hackish code aspires to one day experience. DJB got “lucky” here — he ended up defending himself against an attack he almost certainly never encountered.

Such is the mark of excellent design. Excellent design protects you against things you don’t have any information about. And so we are deploying this excellent design to provide no information.

To translate the fix strategy into a more familiar domain, imagine large chunks of Windows RPC went from Anonymous to Authenticated User only, or even all the way to Admin Only. Or wait, just remember Windows XPSP2 :) This is a sledgehammer, by design. It cuts off attack surface, without necessarily saying why. Astonishingly subtle bugs can be easily hidden, or even rendered irrelevant, by a suitably blunt fix.

Nate McFeters appears to think that Tom Ptacek has figured it out. I’m going to go out on a limb and say that Tom didn’t figure anything out yet but still wanted to write a pithy blog post. I think that if Tom had figured it out, he would have written it down privately and posted the SHA-1 hash, as is the trendy thing to do these days.

Speculation aside, the title of Tom’s blog entry, Dan Kaminsky could have made hundreds of thousands of dollars with this DNS flaw!, does make an important point — Dan didn’t sell the details to ZDI, he used his influence and reputation to coordinate a massive vendor patch effort. That’s an admirable move.

Fellow blogger Opher Etzion, replies to  On Elephants and Analytics with On Unicorn, Professor and Infant.   Opher is kindly giving us another metaphor to consider, the Infant and the Profession, since we are both big fans of big gentle elephants, babies and our universities.  

Opher and I agree that Infants are not Professors, and we also agree that CEP is in its Infancy and there is overhype by folks often implying CEP is a Professor.     So it seems we all have a huge elephant in the room with an Infant Professor hanging on the end of a wildly swinging Elephant’s trunk!

To keep the blogopoints interesting, I should point out that with all this agreement and Kumbaya campfire singing, there are a couple of things I do disagree with in Opher’s amusing counterpoint. 

First of all, Opher uses the well know debate technique of falsely attributing some easily refutable discussion point and then offering a slam dunk counterpoint.   He does this in this clever, but completely inaccurate Opher quote,

 “I [Opher] respectfully disagree with Tim … in his claim that what has been done until today is just hype and hence totally worthless…”

Folks reading my blog know that I have never said “what has been done until today is … totally worthless.”    This is a misfortunate misquote.  Shame on you Opher!  

What I said, easily read in the blog, was that CEP is overhyped and that most of the self-described CEP software on the market today does not live up to the inflated claims we read and hear from CEP software vendors, the analysts and reporters they influence.

The second counterpoint that I find interesting is Opher’s consistent attempt to redress the dramatic lack of capability and analytics in current generation self-described CEP software by repositioning CEP as “intelligent event processing” (IEP) as he is continues in On Intelligent Event Processing.   

Perhaps Opher will be successful in repositioning the vast majority of the original CEP problem space as IEP.   This is a interesting slippery slope, in my opinion.   The new positioning that Opher is offering is that when “event processing” has advanced analytics, it is not CEP anymore, it becomes IEP because CEP is really “Simple Event Processing” (SEP) - event processing with little to no analytical capability.

I don’t know about most of our readers, but all this positioning and repositioning to match the capabilities, or lack of capabilities, in the current portfolio of self-described CEP software vendors is fascinating.

Here is the next logical question is:

What is the difference between a “Complex Event” and an “Intelligent Event” ?

This could get quite interesting, so stay tuned!

Opher and I agree that Infants are not Professors, and we also agree that CEP is in its Infancy and there is overhype by folks often implying CEP is a Professor.     So it seems we all have a huge elephant in the room with an Infant Professor hanging on the end of a wildly swinging Elephant’s trunk!

To keep the blogopoints interesting, I should point out that with all this agreement and Kumbaya campfire singing, there are a couple of things I do disagree with in Opher’s amusing counterpoint. 

First of all, Opher uses the well know debate technique of falsely attributing some easily refutable discussion point and then offering a slam dunk counterpoint.   He does this in this clever, but completely inaccurate Opher quote,

 “I [Opher] respectfully disagree with Tim … in his claim that what has been done until today is just hype and hence totally worthless…”

Folks reading my blog know that I have never said “what has been done until today is … totally worthless.”    This is a misfortunate misquote.  Shame on you Opher!  

What I said, easily read in the blog, was that CEP is overhyped and that most of the self-described CEP software on the market today does not live up to the inflated claims we read and hear from CEP software vendors, the analysts and reporters they influence.

The second counterpoint that I find interesting is Opher’s consistent attempt to redress the dramatic lack of capability and analytics in current generation self-described CEP software by repositioning CEP as “intelligent event processing” (IEP) as he is continues in On Intelligent Event Processing.   

Perhaps Opher will be successful in repositioning the vast majority of the original CEP problem space as IEP.   This is a interesting slippery slope, in my opinion.   The new positioning that Opher is offering is that when “event processing” has advanced analytics, it is not CEP anymore, it becomes IEP because CEP is really “Simple Event Processing” (SEP) - event processing with little to no analytical capability.

I don’t know about most of our readers, but all this positioning and repositioning to match the capabilities, or lack of capabilities, in the current portfolio of self-described CEP software vendors is fascinating.

Here is the next logical question is:

What is the difference between a “Complex Event” and an “Intelligent Event” ?

This could get quite interesting, so stay tuned!