ATC is an excellent working example of complex event processing.   Radar and GPS provide the basic sensory information to accurately track and trace the position of each aircraft in the area of responsibility (AOR) of a particular control tower/zone.     Naturally,  sensory information is preprocessed and formatted in such a way that the data can be processed upstream by multiple real-time applications.

Before we look at complex ATC scenarios, such as “potential collision” or “aircraft off approach vector” we must trace and trace individual objects, aircraft-objects, accurately with very high confidence.    In addition to tracking aircraft-objects, there is a database of information about the aircraft (ideally), such as make, model, age, range, passengers and other properties about the aircraft-object.      In addition, there is a state-model for each aircraft, for example the aircraft might be “on the ground”, “approaching the runway”, “cleared for takeoff”, “cruising altitude”, “approaching runway”, “final decent” etc.  

Tracking and tracing individual aircraft is what is generally referred to as “object refinement” in our CEP/EP reference architecture.   The reason we call this function “object refinement” is that system engineers are focused on optimizing the situational knowledge about individual objects.     Sometimes we refer to this function as “track and trace” because that is what we are doing to  each object in the model.  In Marc Adler’s recent shoplifting scenario, Marc was interested in tracking and tracing people in a store using imaging processing techniques to estimate their behavioral patterns.  In the same way, before we can process for scenarios such as “potential shoplifter” or “suspicious criminal gang activity” we must be able to accurately process (track and trace) individual object, such as people or merchandise.

Back to aircraft and ATC, the “complex event processing” begins when we are looking about object-object relationships, in this model, aircraft-to-aircraft, but this is an overly simplistic model, as we have not yet added (to our model) ground features (towers, buildings, power lines), weather (storm cells, wind) and other flying objects (known migratory bird paths, swarms of insects) to our simple model.  

Complex event processing occurs when we are processing multiple objects in our model looking for threats in real-time.     Practically speaking, all ATC applications are CEP applications.  This means that vendors and integrators who build ATC applications are also CEP vendors.   

Editorial Note: CEP/EP has been around for a long time and was not recently invented in the past decade as some “inventors” would like for us to believe. 

As you can imagine, there is considerable “complex event processing” that goes on “behind the scenes” to provide air traffic controllers and pilots situational knowledge into the “friendly skies”.   As you might further imagine, the situation is more complex when the skies are “not so friendly”, for example, in air combat situations.   

Processing myriad objects is not the end of the processing “chain”.  For example, decisions are being made constantly about potential damage, alternative airports, and more.    In our reference model, we refer to this, generally speaking, as “impact assessment” because we must take an estimated detected complex event, for example “aircraft collision,” and estimate potential damage based on numerous factors such as, the amount of jet fuel in the aircrafts and the location of the aircrafts (over a large city or rural area, near a hospital and emergency services).   Regardless of the scenario, an impact assessment is normally required before optimal decisions can be made.

This is true, by the way, for our shoplifting example (the impact is different if a piece of gum is stolen versus a $1,000,000 diamond necklace or weapons-grade nuclear material) and other scenarios and models.  Static data (information about objects) is required for accurate decision processing.  

Impact assessment is not the end of the “knowledge chain”.    Decisions are constantly being made that effect resources.  For example, suggestion an alternative route for an aircraft is a resource management decision.    Turning on and off radar or switching to alternative tracking devices is a resource management function.  In our CEP/EP reference model (based on the JDL data fusion model), we call this “resource management”.   This function includes contacting emergency services and directing them to a potential crash location or sending out a message to instruct all aircraft to stay off a certain radio frequency.  Resource management is critical.

Our simple ATC model today is by no means complete, it just scratches the surface.  In fact, I have a very close friend, Mark Secrist, who is a former Marine fighter pilot and currently a senior captain for American Airlines.   I have asked Mark to read this post and help me further refine this crude “laymans” ATC model (Thanks Mark!).

Solace provides sophisticated middleware functionality in hardware to monitor, filter, route, transform and secure very large volumes of events in real time and with minimal processing overhead.  Solace uses leading-edge FPGA, ASIC and network processor technology to increase throughput and lower latency of event processing. Applications such as fraud detection, algorithmic trading, compliance, insider trade monitoring, risk management and more can be tackled more effectively by separating the simple monitoring, filtering and normalization of raw events from the complex processing of select events. This event pre-processing takes the burden off CEP engines allowing individual engines to be much more effective. 

The Cisco 7600 OSR consists of a 256 Gbps switching fabric and a 30 million packets per second (mpps) forwarding engine. Its breadth of IP services comes from Cisco IOS, which provides features such as security, enhanced QoS, and destination sensitive services. In addition, the Cisco 7600 OSR allows the migration of existing port adapters from Cisco 7500 series routers, via the Cisco FlexWAN module, giving service providers one the industry’s widest array of interface options in any single platform. This provides service providers great flexibility in deploying the Cisco 7600 OSR for a variety of applications, protects their investment in existing systems, and gives them a practical migration path to the New World Optical Internet.

A Revolutionary Platform For Evolving Networks

The Cisco 7600 OSR helps service providers break through service and bandwidth barriers today, while designing networks to scale for future growth. The Cisco 7600 OSR achieves this through “adaptive network processing,” or the ability to evolve the platform for new IP services without hardware upgrades. Unlike fixed, ASIC-based platforms, which are hardware encoded, the Cisco 7600 OSR relies on the highly flexible Parallel eXpress Forwarding (PXF) technology for scalable performance of services. PXF is a patented, Cisco-developed network processor capable of line-rate IP services delivery that can support new IP services through periodic software upgrades. Each OSM has two PXF processors capable of 12 mpps of IP services delivery per interface card.

“IP+Optical combines the dynamism of the Internet world with the foundation of the transport world, creating an infrastructure that can deliver the services that service providers need,” said Lele Nardin, vice president of the Internet Systems Business Unit at Cisco. “Cisco will continue to add innovative solutions on top of this solid foundation to make service providers better equipped to meet the constantly escalating and changing customer demands for new networking services.”

Pricing and Availability

The base Cisco 7600 OSR system is list priced at $73,000 and the entry level system, with interfaces, start at $100,000. The interfaces modules are priced between $27,000 to $180,000. The Cisco 7600 OSR is available now worldwide.

I agree that these gentlemen are top notch researchers, witnessed by the fact that the authors do not mention nor claim to be “complex event processing” anywhere in their paper!  This paper is not about CEP, nor does it claim to be about CEP, it is about stream processing and unifying SQL standards.

ABSTRACT: This paper describes a unification of two different SQL extensions for streams and its associated semantics. We use the data models from Oracle and StreamBase as our examples. Oracle uses a time-based execution model while StreamBase uses a tuple-based execution model. Time-based execution provides a way to model simultaneity while tuple-based execution provides a way to react to primitive events as soon as they are seen by the system.

Asmentioned on numerous occasions, stream processing is a very important area in CEP/EP.   It is important not to confuse the higher situational knowledge from object-object correlation and state management with the single-object event refinement that occurs in stream processsing.    Event stream processing is fundamentally different than complex event processing. 

Event stream processing performs operations on streaming event objects.   In almost all advanced CEP/EP applications is is necessary to perform robust track and trace operations on streaming event objects, like tracking the position of an airplane.    Tracking the position of an aircraft can be modelled very nicely with event stream processing.  Tracking individual event objects is a precuror to multiclass object situation refinement.

When we manage the state of all the aircraft in the skies over New York, you need more than a stream processing construct.  You need to manage the state of all the aircraft.  Paul Vincent of TIBCO Software being to address this important point in The Value of State…  

Again, we will be better equiped to solve complex distributed event processing problems if we do not confuse the notion of event stream processing and complex event processing.   These technologies are indeed complimentary, both very important, but they are not the same.

I applaud Oracle and StreamBase’s work toward a unified standard for SQL extensions for streams.

So that's what he asked for, and that's what he got. And now it's time to create some cryptographic applications for the rings. Cory and I are holding an open contest for the cleverest application.

I don't think we can invent any encryption algorithms that will survive computer analysis -- there's just not enough entropy in the system -- but we can come up with some clever pencil-and-paper ciphers that will serve them well if they're ever stuck back in time. And there are certainly other cryptographic uses for the rings.

Here's a way to use the rings as a password mnemonic: First, choose a two-letter key. Align the three wheels according to the key. For example, if the key is "EB" for eBay, align the three wheels AEB. Take the common password "PASSWORD" and encrypt it. For each letter, find it on the top wheel. Count one letter to the left if there is a dot over the letter, and one letter to the right if there is a dot under it. Take that new letter and look at the letter below it (in the middle wheel). Count two letters to the left if there is a dot over it, and two letters to the right if there is a dot under it. Take that new letter (in the middle wheel), and look at the letter below it (in the lower wheel). Count three letters to the left if there is a dot over it, and three letters to the right if there is a dot under it. That's your encrypted letter. Do that with every letter to get your password.

"PASSWORD" and the key "EB" becomes "NXPPVVOF."

It's not very good; can anyone see why? (Ignore for now whether or not publishing this on a blog makes it no longer secure.)

How can I do that better? What else can we do with the rings? Can we incorporate other elements -- a deck of playing cards as in Solitaire, different-sized coins to make the system more secure?

Post your contest entries as comments to Cory's blog post -- you can post them here, but they're not going to count as contest submissions -- or send them to cryptocontest@craphound.com. Deadline is October 1st.

Good luck, and have fun with this.

First of all, let’s ground the discussion a bit by translating “smart order routing” to “rule-based message routing” since in this application “smart”  translates to “using rules” and “order” translates to “message”.    Basically, Mark (and other “new on the routing scene” stream processing players) argue that rule-based message routing is CEP.  I will argue that routing is not even close to CEP.  Here is why,

Let’s take a look at a router on the backbone of the global Internet.   A backbone router has very sophisticated software developed over many decades.   These routers run sophisticated, mature algorithms to determine how to route messages (packets) and use these algorithms to build complex routing tables. 

In addition, these routers process messages (packets) from countless sources and route messages (packets) to countless destinations.  Using some of the terms in early posts (above), there is a great “confluence of events” processed by routers.    Futhermore, there are normally quite complex authentication, authorization and other security parameters managed in a router, all in real time.   Routers do much more, but I don’t want to get too deep into routing in this post.

My point is that, without any doubt, global Internet routers process very “cloudy” “confluence of events” with much more sophistication than order routing applications.    However, we do not call Internet routing “CEP”, regardless of how many connections are processed or how much sophisticated processing occurs.  The reason is because the “C” in “CEP” defines a complexity that is at a higher abstraction than messaging and routing.

If you study the literature on CEP, some of which I posted recently, CEP was envisioned to solve complex event processing problems “on top of the routing layer” because the routing layer is a mature technology layer.  We can route, pure and simple.  Of course, we are always seeking faster, more scaleable and more secure routing. 

I admire some of the startups in the CEP/ESP/EP space for working hard to make money and for aggressively positioning their products and attempting to build market share.   However, issues surface when these same companies seem to believe they are the first companies to work in the event processing or message routing space and that they can define whatever they want as “complex event processing” as long as it benefits their sales targets.

There is no doubt that a router does much more sophisticated event processing than the new rule-based stream processing systems running continuous queries across streaming data.  There is no doubt that a router processes a complex “confluence of events”.   However, we don’t call routers “CEP”. 

We do not call routers “CEP” because CEP is about a higher level of knowledge processing.  CEP was created to detect the “complex events” that happen above the mediation and routing layer.     The literature and original examples on CEP are quite clear on this.

PaperBack is a free application that allows you to back up your precious files on the ordinary paper in the form of the oversized bitmaps. If you have a good laser printer with the 600 dpi resolution, you can save up to 500,000 bytes of uncompressed data on the single A4/Letter sheet. Integrated packer allows for much better data density - up to 3,000,000+ (three megabytes) of C code per page.

You may ask - why? Why, for heaven’s sake, do I need to make paper backups, if there are so many alternative possibilities like CD-R’s, DVD±R’s, memory sticks, flash cards, hard disks, streamer tapes, ZIP drives, network storages, magnetooptical cartridges, and even 8-inch double-sided floppy disks formatted for DEC PDP-11? (I still have some). The answer is simple: you don’t. However, by looking on CD or magnetic tape, you are not able to tell whether your data is readable or not. You must insert your medium into the drive (if you have one!) and try to read it.

Paper is different. Do you remember the punched cards? EBCDIC and all this stuff. For years, cards were the main storage medium for the source code. I agree that 100K+ programs were… unhandly, but hey, only real programmers dared to write applications of this size. And used cards were good as notepads, too. Punched tapes were also common. And even the most weird codings, like CDC or EBCDIC, were readable by humans (I mean, by real programmers).

Read the whole thing here.