Marc’s initial shoplifting model in his post is based on John Colapinto’s concepts of matching a pattern of customer movements in the store with their estimated patterns of shoplifting behavioral patterns.    Marc’s asks how Coral8 might address this.   We are not ready to seek a vendor solution.  We do not yet have a workable detection model.

As indicated above, I don’t think the example situation cited by John and Marc is a viable model for automated processing.    Tracking the behavior of customer’s movements, by machine, would require some very sophisticated image processing technology that would be too expensive compared to any possible loss at most retails stores.    This type of behavioral pattern recognition. in retail stores, is performed by people (security personnel), not machines, observing people.  

To develop a machine pattern recognition application to detect retail shoplifting we need to build detection models that are economically feasible.  If we are going to use a model of shoplifting pattern recognition versus anomaly detection, we need to define the objects we must track.  

In the most simple model, we have merchandise-objects.   Stores normally (physically) track merchandise-objects only at the exit/entry points of the store using some electromagnetic proximity detection technology.   In this model, the detection configuration is a combination of simple alerting with humans watching the store (”minding the store”).    This is not complex event processing.

However, if we added another object to our model, the customer-object, then we start to get more “complex,” but we have not defined “complexity” yet because we have not defined the object properties, the possible states of the objects, and the relationships between the objects that are the basis for estimated situations.

Hence, model building is constrained by available resources, simple economics and risk (cost-benefit).  If we are detecting shoplifting in Walmart the cost-benefit model for implementing an automated shoplifting detection system would be different than at a top diamond store on 5th Avenue in NYC.   Protecting loss at a weapons-grade uranium respository follows a different model than protecting loss at a handicraft shop, naturally.

Like Marc, I find models to automatically detect shoplifting interesting, so permit me to close with a general discussion of shoplifting in the context of our CEP/EP reference model.

One approach would be do determine what objects will be represented in our model.   For example, if we are going to track merchandise, we need to model the ”merchandise-object”.  If we are going to track people, we need to define the properties of this “person object.”  If we are going to represent the store layout, we need to define all these objects (store-object, table-object, shelf-object, entry-object and so forth).  The model can get “complex” quite quickly. 

Editorial Note:  An object-oriented approach greatly assists complex model building because we can benefit from OO properties such as encapsulation and polymorphism.  For example, we can define a basic “person object class” and then create superclasses of this object for “customer-object”, “manager-object”, “or criminal-object.”

Generally speaking, each object we define will require a state-model, for example, in Marc’s example of a customer moving around the store, we would need to model the possible states (customer at the entrance, at table 1, at table 2, at shelf 1, in the bathroom, at the cashier, etc.)  Indeed Marc, this is complex event processing if we have modelled multiple objects and defined object-object relationships that indicate situations of interest.   For example, customer-object at table2 where merchandise-object has the property of  ”very expensive, high risk” and then customer-object changes state to “in bathroom”.  Of course, we need more key indicators, but you get the idea.

Right now, I am typing from the Taste from Heaven Vegetarian Restaurant in Chiang Mai and my battery is running low.  The owner of this excellent restaurant also runs the Elephant Nature Park, a non-profit organization advocating and acting on behalf of the rights of the mighty elephants in Thailand.  Would be great if we could also automatically detect the situation of “elephant abuse” by poachers and other crimes against nature.   Time to get back to my delicious mushroom salad, Northeastern Thai style.

As always, thanks for reading, time for me to get back to eating!

The Cisco 7600 OSR consists of a 256 Gbps switching fabric and a 30 million packets per second (mpps) forwarding engine. Its breadth of IP services comes from Cisco IOS, which provides features such as security, enhanced QoS, and destination sensitive services. In addition, the Cisco 7600 OSR allows the migration of existing port adapters from Cisco 7500 series routers, via the Cisco FlexWAN module, giving service providers one the industry’s widest array of interface options in any single platform. This provides service providers great flexibility in deploying the Cisco 7600 OSR for a variety of applications, protects their investment in existing systems, and gives them a practical migration path to the New World Optical Internet.

A Revolutionary Platform For Evolving Networks

The Cisco 7600 OSR helps service providers break through service and bandwidth barriers today, while designing networks to scale for future growth. The Cisco 7600 OSR achieves this through “adaptive network processing,” or the ability to evolve the platform for new IP services without hardware upgrades. Unlike fixed, ASIC-based platforms, which are hardware encoded, the Cisco 7600 OSR relies on the highly flexible Parallel eXpress Forwarding (PXF) technology for scalable performance of services. PXF is a patented, Cisco-developed network processor capable of line-rate IP services delivery that can support new IP services through periodic software upgrades. Each OSM has two PXF processors capable of 12 mpps of IP services delivery per interface card.

“IP+Optical combines the dynamism of the Internet world with the foundation of the transport world, creating an infrastructure that can deliver the services that service providers need,” said Lele Nardin, vice president of the Internet Systems Business Unit at Cisco. “Cisco will continue to add innovative solutions on top of this solid foundation to make service providers better equipped to meet the constantly escalating and changing customer demands for new networking services.”

Pricing and Availability

The base Cisco 7600 OSR system is list priced at $73,000 and the entry level system, with interfaces, start at $100,000. The interfaces modules are priced between $27,000 to $180,000. The Cisco 7600 OSR is available now worldwide.

First of all, let’s ground the discussion a bit by translating “smart order routing” to “rule-based message routing” since in this application “smart”  translates to “using rules” and “order” translates to “message”.    Basically, Mark (and other “new on the routing scene” stream processing players) argue that rule-based message routing is CEP.  I will argue that routing is not even close to CEP.  Here is why,

Let’s take a look at a router on the backbone of the global Internet.   A backbone router has very sophisticated software developed over many decades.   These routers run sophisticated, mature algorithms to determine how to route messages (packets) and use these algorithms to build complex routing tables. 

In addition, these routers process messages (packets) from countless sources and route messages (packets) to countless destinations.  Using some of the terms in early posts (above), there is a great “confluence of events” processed by routers.    Futhermore, there are normally quite complex authentication, authorization and other security parameters managed in a router, all in real time.   Routers do much more, but I don’t want to get too deep into routing in this post.

My point is that, without any doubt, global Internet routers process very “cloudy” “confluence of events” with much more sophistication than order routing applications.    However, we do not call Internet routing “CEP”, regardless of how many connections are processed or how much sophisticated processing occurs.  The reason is because the “C” in “CEP” defines a complexity that is at a higher abstraction than messaging and routing.

If you study the literature on CEP, some of which I posted recently, CEP was envisioned to solve complex event processing problems “on top of the routing layer” because the routing layer is a mature technology layer.  We can route, pure and simple.  Of course, we are always seeking faster, more scaleable and more secure routing. 

I admire some of the startups in the CEP/ESP/EP space for working hard to make money and for aggressively positioning their products and attempting to build market share.   However, issues surface when these same companies seem to believe they are the first companies to work in the event processing or message routing space and that they can define whatever they want as “complex event processing” as long as it benefits their sales targets.

There is no doubt that a router does much more sophisticated event processing than the new rule-based stream processing systems running continuous queries across streaming data.  There is no doubt that a router processes a complex “confluence of events”.   However, we don’t call routers “CEP”. 

We do not call routers “CEP” because CEP is about a higher level of knowledge processing.  CEP was created to detect the “complex events” that happen above the mediation and routing layer.     The literature and original examples on CEP are quite clear on this.

In a tale rich with incongruities, the Communist-run government of West Bengal State invited the Tata Group, a symbol of Indian capitalism, to set up its plant in an area called Singur. It acquired 1,000 acres from farmers on the company’s behalf.
As the project advanced, some farmers who had sold their land demanded it back. The main state-level opposition party, the Trinamool Congress, led protests demanding that the land be returned. Most people sympathetic to Tata accused the opposition of inducing the farmers to protest, while Tata’s critics said the farmers had legitimate grievances.

The issue simmered for months. But in recent days, protesters began surrounding the plant, blocking roads and preventing Tata workers from reaching the plant. “The existing environment of obstruction, intimidation and confrontation has begun to impact the ability of the company to convince several of its experienced managers to relocate and work in the plant,” Tata said in a statement on Tuesday.

The halt to the plant has caused many Indian business people to warn of a chilling effect on investment in the country. It is also unclear how Tata will be able to keep the Nano’s cost so low, since part of the affordable price reflects the company’s savings on the land in Singur.

There is a fundamental asymmetry between state and markets. It is easier to create markets than it is to create state capacity or to prevent its deterioration. Creating markets is a lot about letting go, establishing a reasonable policy framework, and allowing the natural hustling instinct to take over. In other words, hustling is the natural state. Building state capacity, on the other hand, is quite different. It involves overcoming collective action problems, mediating conflict, creating accountability mechanisms where outputs are multiple and fuzzy and links between inputs and outputs murky, and contending with the deep imprints of history. In Weber’s memorable words, building public institutions is like the “slow boring of hard boards”.

In that light, China’s task of improving its private sector seems easier to accomplish than India’s task of arresting institutional decline. So, while China and India can probably both count on more years of high growth, the odds still favour China pulling off that feat than India. That, and not just the meagre medal tally, should be what India mulls over after the Beijing Olympics.

It's easier to liberalise a functional state than it is to functionalise a dysfunctional one, of any ideological stripe.

Fourth US airline to go Wi-Fi: Aircell says they have a fourth airline--after American, Delta, and Virgin America--on board for its in-flight Wi-Fi service. The aerial broadband provider's latest partner will be announced soon. Aircell's service went live in 15 American Airlines planes two weeks ago, and there's been a surprising lack of reporting from regular travelers or journalists since the big splash at the launch.

Microsoft, two universities research methods for better Wi-Fi handoff for vehicles: The researchers developed a method they call Vi-Fi, writes the Seattle Post-Intelligencer's Todd Bishop, which allows a system to maintain connections with several base stations at once, using a primary access point for traffic until a discontinuity is predicted or encountered. This allows seamless handoffs and continuous voice conversations.

Speaking of autos and Wi-Fi, concerns raised about Chrysler's in-car Wi-Fi option: Randall Stross wrote nearly two weeks ago in The New York Times about the problem of distraction. With the Internet at your fingertips, can you restrain yourself? The only problem with the humorous and accurate analysis is that millions of business travelers have 3G access via laptop cards already, so you'd think we'd already be seeing the bad effects of automotive area networks.

A Wi-Fi booster can't post availability signs on highway: The Nebraska town of Louisville has free Wi-Fi downtown, and wanted to post "Visitor Wi-Fi" on a highway sign as another amenity. The state highway department has a policy that doesn't allow the promotion of Wi-Fi, because they believe they'd be inundated. A resident who runs a local Internet firm installed his own signs on the highway; the roads department removed them; he remounted them; they were removed again. The idea of zoning and mounting a billboard apparently hasn't come to the city officials' minds (or perhaps they're prohibited).

The folks spreading misinformation about Wi-Fi health effects cause Ulster school to disable network: I can understand why non-technical folks might think that Wi-Fi has been proven to be unsafe, given the kind of information that's available on the Internet about wireless safety. While there are ongoing studies about the safety of cellular signals--and I'm convinced at this point there's no increased risk to an adult's health by using a cell phone--there is no specific and credible research linked to Wi-Fi, which broadcasts signals at a far lower level than a cell phone, most of the time in most uses.

Washington state shuts down rest-area Wi-Fi: The $3 for 15 minutes, $7 per day, or $30 per month Wi-Fi service at 28 of Washington's 42 rest areas has been turned off after a year for lack of use. Figures. The fees charged by Parsons and Road Connect aren't unreasonable for a nationally scoped plan, but are ridiculous for limited use. States should either bite the bullet and offer these service for free, partner with national roaming operators who can resell service into large networks of business travelers, or use ads to support the service. Highways in remote areas can typically pick up cell data networks, and ongoing costs should be minimal to operate such networks.

IEEE approves fast-roaming standard, 802.11r: This new standard is designed to improve the handoff of devices between base stations. This is accomplished in part by allowing base stations to communicate security and quality of service information so that a VoIP over WLAN phone can immediately reassociate without the delay of authentication and other handshaking.

Denver airport sees 7,000 connections on a single day last week due to Democratic National Convention: FreeFi released the usage figures recently to show how their service is operating. The network started with about 600 daily users when the switchover from fee to free happened 10 months ago, and now carries about 3,500 daily connections.

Coffee Bean & Tea Leaf goes free: The chain of about 700 cafes will have free Wi-Fi installed by now in all its company-owned stores (about 300).

The Anchorage Daily News reports back in September 2004:

Sarah Palin never thought of herself as an investigator. Yet there she was, hacking uncomfortably into Randy Ruedrich’s computer, looking for evidence that the state Republican Party boss had broken the state ethics law while a member of the Alaska Oil & Gas Conservation Commission.

The next week, when Palin went back to work at the AOGCC, she noticed that Ruedrich had removed his pictures from the walls and the personal effects from his desk. But as she and an AOGCC technician worked their way around his computer password at the behest of an assistant attorney general in Fairbanks, they found his cleanup had not extended to his electronic files.

The technician “said it looked like he tried to delete this, but she knew a way to go around and get some of the deleted stuff,” Palin said in an interview. “I didn’t know what I was looking for, but I was there.”

And this is how Salon reports the same incident:

“In a neat symbolic fit, the agent responsible for Alaska’s current moment of reform and modernization is a woman, a breed once nearly as rare in far Northwest politics as a Democrat. Sarah Palin, a libertarian and hockey mom from the fast-growing suburbs of Anchorage, began her political career — as an appointed member of the state’s Oil and Gas Commission — by hacking into the computer of another commissioner, Randy Ruedrich, chairman of the Alaska Republican Party. Palin was seeking the evidence that she would eventually use to charge him with an improper relationship with lobbyists. (Ruedrich would later settle state ethics charges against him by paying a $12,000 fine.)”

Is this where the McCain administration is going to get their computer security expertise? She’s not a security expert but it is nice to see someone at the level of state govenor who knows their way around a computer.

“Event processing has been going on for more than fifty years.”

However, in On Event Processing as a Discipline and Some Subsets another colleague mistakenly blogs,

“… people who dealt in this area [network management and event correlation] have never investigated event processing in the larger sense (e.g. looking at additional patterns), and this area has also not spawned the event processing discipline.”

If you examine just one page from the CEP history at Stanford, researchers there outlined their view of the future applications for CEP, as follows:

• Instant Insight  - hierarchical event viewing applied to the Enterprise IT layer.
  • Analysing business processes
• Network Level Monitoring and Management
• Cyber Security: Network Intrusion Detection
• Enterprise Monitoring and Management
• Modeling and Simulation of Collaborative Business Processes
• Business Policy Monitoring
• Analysis and Debugging of Distributed Systems

These applications areas mentioned by Stanford researchers, including Professor Luckham, support and validate our recent discussion Magic Quadrant for IT Event Correlation and Analysis, 2007 where we concluded that “event correlation and event analysis is Gartner’s closest magic quadrant (MQ)  [...] relates directly to complex event processing (and event processing in general).”  

If you take a detailed look at the 1999 CEP presentation, Defeating Large Scale Attacks: Technology and Strategies for Global Network Monitoring you will readily see that our colleagues are incorrect when they says that event correlational and network management folks have never investigated event processing in the “larger sense”.  For example, the 1999 slides above, Stanford, slide 6, is titled “Complex Event Processing,” defining CEP from the application perspective of event correlation;

Complex Event Processing

• Accept network ‘events’ from any source
  • CISCO NetFlow FlowCollector, tcpdump
• Correlates events based on content and temporal relationship between events
• Event Processing Agents (EPAs) connected in an Event Processing Network (EPNs)
• Both post-mortem and real-time processing

This single event correlational project example from David’s team at Stanford examined the challenging event correlation problems in the context of hierarchical events, maps, patterns, visualization tools, event processing models, patterns languages, network management abstraction layers, and more.  Those core event processing problems from this 1999 example, very large and complex then, still exist today and are much more large and complex - precisely why it is called “complex event processing.”

It is quite obvious, in just this one example, that many folks have been looking at event correlation as a motivating application for event processing, in a larger context, for a long time, contrary to what our colleagues write in their “history of event processing” posts.  

In a future post I will completely debuke these event processing “history revisionists.”   I will illustrate very clearly how the history of event processing goes back at least a decade, and perhaps two (twenty years) before the history outlined in posts like On Research and Practice in Event Processing and The History of Complex Event Processing. 

David Luckam stated that the art-and-science of event processing goes back around 50 years. 

I am not sure I will go all the way back to 1960 in my next post on the history of event processing.  However,  I will go back at least to the early days of Internet Protocol (IP) networking and illustrate why distributed IP networking, network management and network security, is one of the key  motivating factors for what we now call “event processing” and “complex event processing.”