ATC is an excellent working example of complex event processing.   Radar and GPS provide the basic sensory information to accurately track and trace the position of each aircraft in the area of responsibility (AOR) of a particular control tower/zone.     Naturally,  sensory information is preprocessed and formatted in such a way that the data can be processed upstream by multiple real-time applications.

Before we look at complex ATC scenarios, such as “potential collision” or “aircraft off approach vector” we must trace and trace individual objects, aircraft-objects, accurately with very high confidence.    In addition to tracking aircraft-objects, there is a database of information about the aircraft (ideally), such as make, model, age, range, passengers and other properties about the aircraft-object.      In addition, there is a state-model for each aircraft, for example the aircraft might be “on the ground”, “approaching the runway”, “cleared for takeoff”, “cruising altitude”, “approaching runway”, “final decent” etc.  

Tracking and tracing individual aircraft is what is generally referred to as “object refinement” in our CEP/EP reference architecture.   The reason we call this function “object refinement” is that system engineers are focused on optimizing the situational knowledge about individual objects.     Sometimes we refer to this function as “track and trace” because that is what we are doing to  each object in the model.  In Marc Adler’s recent shoplifting scenario, Marc was interested in tracking and tracing people in a store using imaging processing techniques to estimate their behavioral patterns.  In the same way, before we can process for scenarios such as “potential shoplifter” or “suspicious criminal gang activity” we must be able to accurately process (track and trace) individual object, such as people or merchandise.

Back to aircraft and ATC, the “complex event processing” begins when we are looking about object-object relationships, in this model, aircraft-to-aircraft, but this is an overly simplistic model, as we have not yet added (to our model) ground features (towers, buildings, power lines), weather (storm cells, wind) and other flying objects (known migratory bird paths, swarms of insects) to our simple model.  

Complex event processing occurs when we are processing multiple objects in our model looking for threats in real-time.     Practically speaking, all ATC applications are CEP applications.  This means that vendors and integrators who build ATC applications are also CEP vendors.   

Editorial Note: CEP/EP has been around for a long time and was not recently invented in the past decade as some “inventors” would like for us to believe. 

As you can imagine, there is considerable “complex event processing” that goes on “behind the scenes” to provide air traffic controllers and pilots situational knowledge into the “friendly skies”.   As you might further imagine, the situation is more complex when the skies are “not so friendly”, for example, in air combat situations.   

Processing myriad objects is not the end of the processing “chain”.  For example, decisions are being made constantly about potential damage, alternative airports, and more.    In our reference model, we refer to this, generally speaking, as “impact assessment” because we must take an estimated detected complex event, for example “aircraft collision,” and estimate potential damage based on numerous factors such as, the amount of jet fuel in the aircrafts and the location of the aircrafts (over a large city or rural area, near a hospital and emergency services).   Regardless of the scenario, an impact assessment is normally required before optimal decisions can be made.

This is true, by the way, for our shoplifting example (the impact is different if a piece of gum is stolen versus a $1,000,000 diamond necklace or weapons-grade nuclear material) and other scenarios and models.  Static data (information about objects) is required for accurate decision processing.  

Impact assessment is not the end of the “knowledge chain”.    Decisions are constantly being made that effect resources.  For example, suggestion an alternative route for an aircraft is a resource management decision.    Turning on and off radar or switching to alternative tracking devices is a resource management function.  In our CEP/EP reference model (based on the JDL data fusion model), we call this “resource management”.   This function includes contacting emergency services and directing them to a potential crash location or sending out a message to instruct all aircraft to stay off a certain radio frequency.  Resource management is critical.

Our simple ATC model today is by no means complete, it just scratches the surface.  In fact, I have a very close friend, Mark Secrist, who is a former Marine fighter pilot and currently a senior captain for American Airlines.   I have asked Mark to read this post and help me further refine this crude “laymans” ATC model (Thanks Mark!).

Solace provides sophisticated middleware functionality in hardware to monitor, filter, route, transform and secure very large volumes of events in real time and with minimal processing overhead.  Solace uses leading-edge FPGA, ASIC and network processor technology to increase throughput and lower latency of event processing. Applications such as fraud detection, algorithmic trading, compliance, insider trade monitoring, risk management and more can be tackled more effectively by separating the simple monitoring, filtering and normalization of raw events from the complex processing of select events. This event pre-processing takes the burden off CEP engines allowing individual engines to be much more effective. 

First of all, let’s ground the discussion a bit by translating “smart order routing” to “rule-based message routing” since in this application “smart”  translates to “using rules” and “order” translates to “message”.    Basically, Mark (and other “new on the routing scene” stream processing players) argue that rule-based message routing is CEP.  I will argue that routing is not even close to CEP.  Here is why,

Let’s take a look at a router on the backbone of the global Internet.   A backbone router has very sophisticated software developed over many decades.   These routers run sophisticated, mature algorithms to determine how to route messages (packets) and use these algorithms to build complex routing tables. 

In addition, these routers process messages (packets) from countless sources and route messages (packets) to countless destinations.  Using some of the terms in early posts (above), there is a great “confluence of events” processed by routers.    Futhermore, there are normally quite complex authentication, authorization and other security parameters managed in a router, all in real time.   Routers do much more, but I don’t want to get too deep into routing in this post.

My point is that, without any doubt, global Internet routers process very “cloudy” “confluence of events” with much more sophistication than order routing applications.    However, we do not call Internet routing “CEP”, regardless of how many connections are processed or how much sophisticated processing occurs.  The reason is because the “C” in “CEP” defines a complexity that is at a higher abstraction than messaging and routing.

If you study the literature on CEP, some of which I posted recently, CEP was envisioned to solve complex event processing problems “on top of the routing layer” because the routing layer is a mature technology layer.  We can route, pure and simple.  Of course, we are always seeking faster, more scaleable and more secure routing. 

I admire some of the startups in the CEP/ESP/EP space for working hard to make money and for aggressively positioning their products and attempting to build market share.   However, issues surface when these same companies seem to believe they are the first companies to work in the event processing or message routing space and that they can define whatever they want as “complex event processing” as long as it benefits their sales targets.

There is no doubt that a router does much more sophisticated event processing than the new rule-based stream processing systems running continuous queries across streaming data.  There is no doubt that a router processes a complex “confluence of events”.   However, we don’t call routers “CEP”. 

We do not call routers “CEP” because CEP is about a higher level of knowledge processing.  CEP was created to detect the “complex events” that happen above the mediation and routing layer.     The literature and original examples on CEP are quite clear on this.

This makes the compliance area rife for scammers, who send letters or emails claiming that businesses owe them penalties or haven’t filed the right forms.

Tony Mancuso at Nolo, the publisher of books about corporate forms and law, writes about how to recognize some of these scams.

One official-looking legal letter came into the controller’s office recently. It was from an “agency” calling itself the Corporate Minutes Compliance Counsel, or somesuch, and it strongly advised (warned, really) that Nolo send the Board a payment of $125 to prepare its state-mandated domestic corporation statement. Failure to do so could result in dire consequences, the letter advised, including a loss of corporate status with the Secretary of State.
Nolo’s controller, who has years of experience with real and bogus corporate service solicitations, shredded this letter immediately.

This scam also seems particularly nasty because not only can the scammers steal money from you, they will get valuable information and insight into your private business!

One smart way to avoid these scams is to check anything suspicious–call your secretary of state office or look up the organization who sends you that nasty letter. Or learn more by reading the full article here.

PaperBack is a free application that allows you to back up your precious files on the ordinary paper in the form of the oversized bitmaps. If you have a good laser printer with the 600 dpi resolution, you can save up to 500,000 bytes of uncompressed data on the single A4/Letter sheet. Integrated packer allows for much better data density - up to 3,000,000+ (three megabytes) of C code per page.

You may ask - why? Why, for heaven’s sake, do I need to make paper backups, if there are so many alternative possibilities like CD-R’s, DVD±R’s, memory sticks, flash cards, hard disks, streamer tapes, ZIP drives, network storages, magnetooptical cartridges, and even 8-inch double-sided floppy disks formatted for DEC PDP-11? (I still have some). The answer is simple: you don’t. However, by looking on CD or magnetic tape, you are not able to tell whether your data is readable or not. You must insert your medium into the drive (if you have one!) and try to read it.

Paper is different. Do you remember the punched cards? EBCDIC and all this stuff. For years, cards were the main storage medium for the source code. I agree that 100K+ programs were… unhandly, but hey, only real programmers dared to write applications of this size. And used cards were good as notepads, too. Punched tapes were also common. And even the most weird codings, like CDC or EBCDIC, were readable by humans (I mean, by real programmers).

Read the whole thing here.

Blogger: Randall Gamby

Two weeks ago the PCI Security Standards Council released the preliminary details of the PCI Data Security Standard (DSS) V1.2 that’s due out in October.  While many Analysts and Reporters have already written on the topic (I’ll be releasing an extensive update on Burton Group’s PCI coverage around the October release date), they really haven’t commented on what’s still not been addressed by the standard for enterprises still working on attaining compliance.

While I applaud the PCI Security Standards Council in further clarifying and adjusting the standard, a lot of work still needs to be done.  I receive about one or two PCI questions a week from our clients and they seem to revolve around a couple of topics I’ve yet to see addressed:

• Guidelines for selecting a Qualified Security Assessor (QSA) – while there are a large number of QSA organizations listed on the PCI Security Standards Council web site; they can’t really recommend a particular QSA for an individual organization.  This leads a lot of organizations to struggle with determining what criteria they should use in selecting a QSA for their certification.
• The role of the QSA – organizations are also still trying to understand the role of a QSA.  Should they get a QSA involved in the gap and remediation process in advance of certification?  If so, should it be the same QSA that will do their certification (knowing there’s a risk that the QSA will be pre-disposed to only care about certain vulnerabilities)?
• Industry-specific best practices – while each organization may have different infrastructures, in general, most industries try to be consistent with the major functions they perform.  So are credit card transactions handled differently between say, a major retailer with 10,000 POS systems and an insurance company that has hundreds of independent agents receiving remittances? Probably, so what are best practices around these industry-specific configurations?
• Virtualized environments – while the PCI Security Standards Council recognizes that some organizations have moved to virtual services for consolidation and management, the DSS really doesn’t provide guidelines for QSAs to evaluate and certify these environments.
• Monitoring and audit – while the PCI DSS recommends minimum timeframes for scanning, doing pen tests, etc. what are the real levels of monitoring and audit needed for ensuring security?  With the Hannaford and Okemo breaches that occurred (both where PCI compliant), neither discovered the problem until months after the breaches had happened.  So identifying what should be scanned and tested and if some of this should be on a continuous basis still requires refinement.
• PCI as part of an overall security model – what are the best practices around merging PCI security requirements into an enterprise’s overall security model?  Should it be maintained separately? Should some components be integrated with similar security mechanisms?  Should PCI be at the top of the security model and other configurations be based upon its requirements?  There are really no answers coming forth on this topic and the other question is where will they come from? Surely enterprises won’t expect the PCI Security Standards Council to tell them how to run their security services.

I will be providing Burton Group’s perspective on most of these questions in my upcoming report, but rather than relying on third parties to resolve these, I’d hope that the PCI Security Standards Council will be able to continue to provide answers to the questions they can in future updates, and releases, of the PCI DSS.

In a tale rich with incongruities, the Communist-run government of West Bengal State invited the Tata Group, a symbol of Indian capitalism, to set up its plant in an area called Singur. It acquired 1,000 acres from farmers on the company’s behalf.
As the project advanced, some farmers who had sold their land demanded it back. The main state-level opposition party, the Trinamool Congress, led protests demanding that the land be returned. Most people sympathetic to Tata accused the opposition of inducing the farmers to protest, while Tata’s critics said the farmers had legitimate grievances.

The issue simmered for months. But in recent days, protesters began surrounding the plant, blocking roads and preventing Tata workers from reaching the plant. “The existing environment of obstruction, intimidation and confrontation has begun to impact the ability of the company to convince several of its experienced managers to relocate and work in the plant,” Tata said in a statement on Tuesday.

The halt to the plant has caused many Indian business people to warn of a chilling effect on investment in the country. It is also unclear how Tata will be able to keep the Nano’s cost so low, since part of the affordable price reflects the company’s savings on the land in Singur.

There is a fundamental asymmetry between state and markets. It is easier to create markets than it is to create state capacity or to prevent its deterioration. Creating markets is a lot about letting go, establishing a reasonable policy framework, and allowing the natural hustling instinct to take over. In other words, hustling is the natural state. Building state capacity, on the other hand, is quite different. It involves overcoming collective action problems, mediating conflict, creating accountability mechanisms where outputs are multiple and fuzzy and links between inputs and outputs murky, and contending with the deep imprints of history. In Weber’s memorable words, building public institutions is like the “slow boring of hard boards”.

In that light, China’s task of improving its private sector seems easier to accomplish than India’s task of arresting institutional decline. So, while China and India can probably both count on more years of high growth, the odds still favour China pulling off that feat than India. That, and not just the meagre medal tally, should be what India mulls over after the Beijing Olympics.

It's easier to liberalise a functional state than it is to functionalise a dysfunctional one, of any ideological stripe.

The TSA's useless photo ID rules

No-fly lists and photo IDs are supposed to help protect the flying public from terrorists. Except that they don't work.

By Bruce Schneier

August 28, 2008

The TSA is tightening its photo ID rules at airport security. Previously, people with expired IDs or who claimed to have lost their IDs were subjected to secondary screening. Then the Transportation Security Administration realized that meant someone on the government's no-fly list -- the list that is supposed to keep our planes safe from terrorists -- could just fly with no ID.

Now, people without ID must also answer personal questions from their credit history to ascertain their identity. The TSA will keep records of who those ID-less people are, too, in case they're trying to probe the system.

This may seem like an improvement, except that the photo ID requirement is a joke. Anyone on the no-fly list can easily fly whenever he wants. Even worse, the whole concept of matching passenger names against a list of bad guys has negligible security value.

How to fly, even if you are on the no-fly list: Buy a ticket in some innocent person's name. At home, before your flight, check in online and print out your boarding pass. Then, save that web page as a PDF and use Adobe Acrobat to change the name on the boarding pass to your own. Print it again. At the airport, use the fake boarding pass and your valid ID to get through security. At the gate, use the real boarding pass in the fake name to board your flight.

The problem is that it is unverified passenger names that get checked against the no-fly list. At security checkpoints, the TSA just matches IDs to whatever is printed on the boarding passes. The airline checks boarding passes against tickets when people board the plane. But because no one checks ticketed names against IDs, the security breaks down.

This vulnerability isn't new. It isn't even subtle. I first wrote about it in 2006. I asked Kip Hawley, who runs the TSA, about it in 2007. Today, any terrorist smart enough to Google "print your own boarding pass" can bypass the no-fly list.

This gaping security hole would bother me more if the very idea of a no-fly list weren't so ineffective. The system is based on the faulty notion that the feds have this master list of terrorists, and all we have to do is keep the people on the list off the planes.

That's just not true. The no-fly list -- a list of people so dangerous they are not allowed to fly yet so innocent we can't arrest them -- and the less dangerous "watch list" contain a combined 1 million names representing the identities and aliases of an estimated 400,000 people. There aren't that many terrorists out there; if there were, we would be feeling their effects.

Almost all of the people stopped by the no-fly list are false positives. It catches innocents such as Ted Kennedy, whose name is similar to someone's on the list, and Islam Yusuf (formerly Cat Stevens), who was on the list but no one knew why.

The no-fly list is a Kafkaesque nightmare for the thousands of innocent Americans who are harassed and detained every time they fly. Put on the list by unidentified government officials, they can't get off. They can't challenge the TSA about their status or prove their innocence. (The U.S. 9th Circuit Court of Appeals decided this month that no-fly passengers can sue the FBI, but that strategy hasn't been tried yet.)

But even if these lists were complete and accurate, they wouldn't work. Timothy McVeigh, the Unabomber, the D.C. snipers, the London subway bombers and most of the 9/11 terrorists weren't on any list before they committed their terrorist acts. And if a terrorist wants to know if he's on a list, the TSA has approved a convenient, $100 service that allows him to figure it out: the Clear program, which issues IDs to "trusted travelers" to speed them through security lines. Just apply for a Clear card; if you get one, you're not on the list.

In the end, the photo ID requirement is based on the myth that we can somehow correlate identity with intent. We can't. And instead of wasting money trying, we would be far safer as a nation if we invested in intelligence, investigation and emergency response -- security measures that aren't based on a guess about a terrorist target or tactic.

That's the TSA: Not doing the right things. Not even doing right the things it does.