Partial Disclosure - The Good

The responsible disclosure process tends to break down in rare occasions where the vendor doesn’t want to fix the issue. When this occurs, the researcher is put into a difficult position whereby full disclosure could put users’ systems at high risk of compromise. The other case where partial disclosure becomes an alternative is when the researcher has discovered a design flaw in a protocol or underlying multiple vendor component. Examples of this case include the DNS flaws published this past summer by Dan Kaminsky and the TCP denial of service condition discovered by Robert E. Lee and Jack Louis that is currently in the disclosure process. When the flaw affects a very large number of vendors and the actual problem is located within the underlying protocols that support the communications of the Internet as a whole, one possible solution is to follow a partial disclosure model where phasing the details to the general public can be used to encourage adoption and creation of patches throughout the enormous target audience.

Partial Disclosure - The Bad

What is driving the fear surrounding partial disclosure is the potential for abuse. When a major flaw is partially disclosed, a number of potential issues may occur. First and foremost, the further along the partial disclosure path we are, the more details will be released to the public, and the higher the probability that someone (either good or bad intentioned) will figure out the exploit and disclose the details. Second, when partially disclosing, the vendor’s hand is being forced into a situation that could speed up fixes, reduce testing, and cause ripple problems elsewhere within the infrastructure. It is difficult enough to dance the fine time line when doing responsible disclosure, but if we are escalated to the point of partial disclosure, additional fuel is added to the fire.

The Ugly

The real ugly part of partial disclosure is when we add to the equation the ability to spread fear, uncertainty, and doubt into the normal user community. It is generally well accepted that FUD can be used to drive additional revenue. If it is possible to increase the perceived magnitude of the “problem” that your product or service solves, it is possible to directly impact the demand for that product or service. That is the major fear imposed by the growing trend of partial disclosure. By releasing just enough information to trigger wide scale speculation into the flaw, it is possible to create buzz and garner media attention resulting in a lot of speculation and very little hard facts around the issue. The potential for abuse by the security industry at large is enormous.

The Fix

Some have suggested a group of security researchers be convened to vet the requirement of partial disclosure and to allow for independent peer review of any security research that requires the partial disclosure process. This suggestion leaves questions regarding who would stand on this group and who would be impartial enough to ensure that the right thing was always done regardless of profit potential. It also leaves open the opportunity for member researchers to utilize the information gathered during the vetting process to position themselves to profit from the data upon release. It might be wiser to rely on a higher level authority or government entity to manage this process and use the services of security researchers as required for subject matter expertise. While a group of this type wouldn’t ensure that all partial disclosure is appropriate, it would hopefully limit the potential for abuse and the ever present chance that people try to profit from the FUD that surrounds the current partial disclosure process.

Around the same time that Professor Luckham and his team was working on CEP applications in network management and security management, I was leading efforts to build network and security management control centers for the United States Air Force.  In the beginning, dating back to 1994, my Internet-related work was for Air Combat Command (ACC), working out of ACC headquarters at Langley Air Force Base.

In 1997, I lead a technical team that developed countermeasures against an actual distributed Internet-based attack on the Langley AFB SMTP email infrastructure.  This attack was documented in a technical paper, E-Mail Bombs and Countermeasures: Cyber Attacks on Availability and Brand Integrity, IEEE Network Magazine, Vol. 12, No. 2, pp. 10-17, March/April 1998.  In addition, this attackand countermeasures I designed was featured in Popular Science Magazine in an 1998 article, War.Com and other news channels.  I also published a number of related papers on this topic.

Our team used a rule-based approach for countermeasures against massive email bombs attacks on the Langley Air Force Base email infrastructure.   We called this rule-based system, BombShelter. and it was written in PERL.  I developed both the original software architecture and the original working prototype for BombShelter (in two days) and then we turned the software over to our team who used the rule-based approach for daily attack countermeasures.

I watched for days, and then weeks, as my team designed rules, and the attackers wrote new attacks that circumvented the rules.  Some folks in the Pentagon used to say that I “lead the effort to fight the first war in cyberspace”.   It might have have been the first cyberwar, I am not sure, but it was certainly the first publicly documented cyberwar.  There is no doubt about this.

Without getting into all the historical footnotes and significance of this cyberwar that was fought with experts and rule-based systems, I would like to jump to an important conclusion.

Rule-based systems are useful, but have limited functionality and scaleability in most complex event processing applications.

Rule-based systems are human resource intensive because rule-based systems cannot learn and adapt on their own, humans learn and then write new rules.  This is how rule-based systems work.

This is the motivation behind why I spend a lot of time to search for new, more efficient and adaptive methods as alternatives to rule-based systems.   After extensive research, I published a series of papers on the future of intrusion detection in the Internet.  Intrusion Detection Systems & Multisensor Data Fusion - Creating Cyberspace Situational Awareness [1], helped lead an evolution in Internet security, particularly in the area of network-based intrusion detection systems (IDS).

In my published research work, motivated by limitations with rule-based approaches, I used the same mature functional model that is used to process missile attacks, control global air traffic, and other complex event processing applications in physical space; but I applied these concepts to cyberspace.

Around the same time, Professor Luckham and others were working on similar problems, all related to real-time detection and response to threats in cyberspace.  They were also funded by the US government.

Sidebar: Stream processing of transaction- based systems (databases), another area of interest, was focused on a totally different problem, which was the low latency processing of straight-thru processing in databased-oriented systems.   These stream processing systems were, and remain however,  rule-based systems.  The problems we were trying to solve in cyberspace, however, cannot be efficiently and pragmatically solved by rule-based systems alone.  Only relatively simple scenarios can be efficiently detected by rule-based stream processing systems.

The vast majority of complex event processing classes of problems require rules plus advanced algorithms that can learn and adapt in real-time.    I know this, not from reading papers or taking university classes on rule-bases systems, but from working on some very challenging operational problems in real-time.    This is why I remain interested in complex event processing and why I continue to elaborate on why rule-based systems have limitations.

Charlie Rose:  
And so when you look at where we are going, there seems to be two issues that are apparent to me at least, risk and leverage.  We just lost sight of risk and leverage of what was appropriate?

Warren Buffett:  
Yeah.  Again, because it pays off for a while.  You know, you can lose leverage, and it's the only way a smart guy can go broke.  If you owe money, you can't pay them out.  You just pay for everything, you do smart things, you eventually get very rich.  If you do smart things and use leverage and do one wrong thing along the way, it could wipe you out, because anything times zero is zero.  But it's reinforcing when the people around you are doing it successfully, you're doing it successfully, and it's a lot like Cinderella at the ball.  I mean you know at midnight everything is going to turn to pumpkins and mice; right?  But if the evening goes along, I mean, you know, the guys look better all the time, the music sounds better, it's more and more fun, you think why the hell should I leave at quarter of 12.  I'll leave at two minutes to 12.  But the trouble is, there are no clocks on the wall.  And everybody thinks they're going to leave at two minutes to 12.

Charlie Rose:  
And should wise people have known better?

Warren Buffett:  
People should always know better.

Charlie Rose:  
Yeah.

Warren Buffett:  
I mean people -- people don't get -- they don't get smarter about things that get as basic as greed and you can't stand to see your neighbor getting rich.  You know you're smarter than he is, and he's doing these things, you know, and he's getting rich, and your spouse is getting unhappy with you because you aren't doing -- pretty soon you start doing it.  And so you get what I call the natural progression, the three Is.  The innovators, the imitators, and the idiots.  And that's what happens.  Everybody just kind of goes along.  And you look kind of silly if you disagree.  I mean, you know, you could have these crazy Internet valuations in the late 1990s, but they prove themselves out in the market.  The next day they were selling for more than they were the day before, and people said, you know, you're crazy if you don't get in on this.  So it's very human.  Now, with housing it's something even more dramatic than that, because most people aspire to own their own home.  And if you really think that houses prices are going to go up next year and the year after, you feel if I don't buy it this year, I'm going to have to buy it next year.  That's not true of an Internet stock.  But it's true of a home.  And when somebody makes it very easy for you to do it by saying you don't really have to put up my money, you can lie about your income a little, or we'll give you 100 percent mortgage, you're going to do it, because everybody that's done it has been proven right.  You have what they call social tools, and, you know, you're going to feel like an idiot if you didn't do it, because the house cost more.

Warren Buffett:  
Oh, I think confidence will come back.  I will tell you this.  This country is going -- be living better ten years from now than it is now.  It will be living better in 20 years from now than ten years from now.  The ingredients that made this country, you know, the miracle of the world -- I mean we had a seven for one improvement in the average American standard of living in the 20th century.  Now, we had the great depression, we had two world wars, we had the flu epidemic.  You know, we had oil shock.  You know, we had all these terrible things happen.  But something about the American system unleashed more and of a potential to human beings over that hundred years so that we had a seven for one improvement in -- there's never been any -- I mean, you have centuries where if you've got a 1 percent improvement, then it's something.  So we've got a great system.  And we've got more productive capacity now than we ever have.  The American worker is more productive than he's ever been.  We've got more people to do it.  We've got all the ingredients for a sensational future.  It's just that right now the athlete's on the floor.  But we -- this is a super athlete.

Originally, researchers in CEP were not trying to solve a problem of streaming data or streaming events.   Often we read this mischaracterization by folks in the database/streaming domain, as they were focused on the low latency processing of streaming events.   A natural extension of this research has been stream processing software (often called “engines”) that process streaming data with continuous queries, for example market data feeds for algo-trading or best market order execution.  This mischaracterization is partly responsible for why we see many order processing applications in market data stream processing mislabled as “complex event processing” applications.

The genesis of complex event processing was not the stream processing need for “feeds and speed” but the processing capability to solve what can be characterized as the “problem of asymmetric capabilties”.   The term “asymmetric” has been used in the military domain. For example we often hear the term “asymmetric warfare.”  However, in general the concept of “asymmetrical processing capablities” is the true genesis for CEP and related processing concepts and domains.   It is this genesis that distinguishes CEP from EDA, SOA, SOR, and so many other technology oriented concepts.

In order to illustrate what I mean by “asymmetrical processing capablities” we will take the example of the evolution of rocketry.    In the early days, scientists learned how to make rockets, I assume with gunpowder and similar chemical compounds to launch rockets.   Over many years the application of rocketry advanced much faster than the ability to understand the situations created in the sky.    In other words, folks could fill the skies with rockets long before they had the capability to track and identify (or sense and respond to)  the rockets in real time.

Therefore, the concept of “asymmetrical processing capablities” is the situation where there is a capability, such as “launch a rocket, sense-and-respond,” that is asymmetric in nature.    In other words, the capability to detect multiple rocket launches creates an asymmetric situation where it is easy to launch rockets, but hard to detect and defend against those launches.

The same concept can be applied to everyday air travel.   If we could only fly airplanes, but did not have the capability to track the planes, understand situations in airspace, and then respond to changing situations, air travel would be quite difficult.   Lucky for us, the global traveller, there is symmetry in the capabilities to build and fly aircraft and the capabilities to detect, track and follow the evolving situations in the sky.

The genesis of CEP was to solve the problem of asymmetry in cyberspace, or if you prefer, distributed data networks.   The folks who identified, early on,  the problems associated with asymmetry in cyberspace were folks working the the field of network and security management.    This is because there has been, and is currently, a great asymmetry between the capablities to “launch a process or transaction” in cyberspace and the capabilties to detect and track what is going on in the same domain.

In my next post on this topic, we will go into some details of this asymmetry and review the first CEP projects from Stanford University in the context of asymmetric processing capabilities in cyberspace.

Defining Cloud Computing

Cloud computing is a very active community. The Google Group gets 600 posts per month and many bloggers are covering the space. However, “cloud computing” is impossible to define in a way that satisfies everyone (or even most). Cloud computing is not alone in this controversy, consider the definition and meaning of “Web 2.0″, “mashups” or “RESTful architecture”. All of these terms are relatively recent. According to Google Trends, these terms became popular to the general public sometime between 2005 and 2007:

• Web 2.0 - often confused with RIA, AKA Social Computing, Long-Tail Apps, Crowdware (2005 by O’Reilly Media)
• Mashup - made popular by Google Maps, AKA Composite/Situational Apps. (2005)
• REST - Has a strict definition, but many don’t understand it and abuse the term. (2006 by R. Fielding)
• Cloud computing - collides with many other terms, such as SaaS, Grid, Utility, PaaS, etc. (2007)

The definition of cloud computing is in progress:

There’s a Darwinian evolution of the exact definition of cloud computing running around. We’re about a country mile away from “knowing when I see it”, which is excellent progress. The cloud to everyone’s silver-lining has enough material to write a 3 volume desktop reference at this point. - Michael Cote, June 2008

Definition #1 - “Cloud computing is the realisation of Internet (”Cloud”) based development and use of computer technology (”Computing”) delivered by an ecosystem of providers. - Sam Johnston, July 2008

Definition #2 - “Cloud computing = network computing. I love the idea of cloud computing, the next evolution of the most network intensive architecture possible, but one that if it works well, is transparent. It’s all about the transparency.” - Douglas Gourlay, Cisco, May 2008

Definition #3 - “There seems to be a group myopia around so-called “cloud computing” and its definitions. What we’re really talking about are “cloud services” of which, “computing” is only a subset…Cloud services are not SaaS. They are far more akin to web services…” - Randy Bias, neoTactics, May 2008

(Anti-)Definition #4 - “Note that I refer to cloud services, not to the could. I am not interested in defining cloud as a term, because I don’t think it’s very useful. For those of us in the distributed computing’s pace

The Working Definition (Winner!):

“…the notion of providing easily accessible compute and storage resources on a pay-as-you-go, on-demand basis, from a virtually infinite infrastructure managed by someone else. As a customer, you don’t know where the resources are, and for the most part, you don’t care. What’s really important is the capability to access your application anywhere, move it freely and easily, and inexpensively add resources for instant scalability.” - Mitchell Crandell, Rightscale, June 2008

Taxonomies of the Cloud Space

Taxonomies are useful to provide insight into a market. It classifies a multitude of players into a smaller bucket.

Andreessen’s Platforms - September 2007

Provided an early taxonomy model for emerging cloud platforms

Platform being a system that can be programmed

• Access API - platform that provides web service endpoints
• Plug-In API - platform invokes your code, that you have deployed remotely
• Runtime Environment - your code runs inside the platform’s process space.

Mehta 11 Layer Stack, April 2008

1. Facilities (space, power, cooling)
2. Network
3. Hardware (e.g. servers Amazon EC2 runs)
4. Hardware virtualization (e.g. Xen for EC2) - optional
5. O/S (e.g. Linux)
6. Systems Management (e.g., tools to manage EC2 instances)
7. Application Middleware (e.g., MySQL on EC2)
8. Application Code
9. Application APIs / Web Services
10. GUI for Application
11. GUI for Application Development / Customization

Croll Cloud Stack, June 2008

7 layer stack within Turnkey app and Generic Platform.

Turnkey app

• SaaS
• Extensible app
• Generic IDE
• Constrained APIs
• App Cluster
• Virtual Data Center
• Virtual Servers

Generic Platform

The bottom of Alistair’s stack includes “root access “style compute clouds.

Robert Anderson, July 2008

3 layer stack

• Software (SaaS)
• Platform (PaaS)
• Infrastructure (IaaS)

This is the model taxonomy for this session.

Related Concepts and Terms

• Infrastructure as a Service (IaaS), Hardware as a Service (HaaS) are synonyms to cloud infrastructure.
• Virtualization
• Hosting
• Autonomic computing
• Distributed computing
• Grid computing

Cloud Applications

• SaaS
• S+S (Software+Services)
• Managed Service Provider (MSP)

Marc’s initial shoplifting model in his post is based on John Colapinto’s concepts of matching a pattern of customer movements in the store with their estimated patterns of shoplifting behavioral patterns.    Marc’s asks how Coral8 might address this.   We are not ready to seek a vendor solution.  We do not yet have a workable detection model.

As indicated above, I don’t think the example situation cited by John and Marc is a viable model for automated processing.    Tracking the behavior of customer’s movements, by machine, would require some very sophisticated image processing technology that would be too expensive compared to any possible loss at most retails stores.    This type of behavioral pattern recognition. in retail stores, is performed by people (security personnel), not machines, observing people.  

To develop a machine pattern recognition application to detect retail shoplifting we need to build detection models that are economically feasible.  If we are going to use a model of shoplifting pattern recognition versus anomaly detection, we need to define the objects we must track.  

In the most simple model, we have merchandise-objects.   Stores normally (physically) track merchandise-objects only at the exit/entry points of the store using some electromagnetic proximity detection technology.   In this model, the detection configuration is a combination of simple alerting with humans watching the store (”minding the store”).    This is not complex event processing.

However, if we added another object to our model, the customer-object, then we start to get more “complex,” but we have not defined “complexity” yet because we have not defined the object properties, the possible states of the objects, and the relationships between the objects that are the basis for estimated situations.

Hence, model building is constrained by available resources, simple economics and risk (cost-benefit).  If we are detecting shoplifting in Walmart the cost-benefit model for implementing an automated shoplifting detection system would be different than at a top diamond store on 5th Avenue in NYC.   Protecting loss at a weapons-grade uranium respository follows a different model than protecting loss at a handicraft shop, naturally.

Like Marc, I find models to automatically detect shoplifting interesting, so permit me to close with a general discussion of shoplifting in the context of our CEP/EP reference model.

One approach would be do determine what objects will be represented in our model.   For example, if we are going to track merchandise, we need to model the ”merchandise-object”.  If we are going to track people, we need to define the properties of this “person object.”  If we are going to represent the store layout, we need to define all these objects (store-object, table-object, shelf-object, entry-object and so forth).  The model can get “complex” quite quickly. 

Editorial Note:  An object-oriented approach greatly assists complex model building because we can benefit from OO properties such as encapsulation and polymorphism.  For example, we can define a basic “person object class” and then create superclasses of this object for “customer-object”, “manager-object”, “or criminal-object.”

Generally speaking, each object we define will require a state-model, for example, in Marc’s example of a customer moving around the store, we would need to model the possible states (customer at the entrance, at table 1, at table 2, at shelf 1, in the bathroom, at the cashier, etc.)  Indeed Marc, this is complex event processing if we have modelled multiple objects and defined object-object relationships that indicate situations of interest.   For example, customer-object at table2 where merchandise-object has the property of  ”very expensive, high risk” and then customer-object changes state to “in bathroom”.  Of course, we need more key indicators, but you get the idea.

Right now, I am typing from the Taste from Heaven Vegetarian Restaurant in Chiang Mai and my battery is running low.  The owner of this excellent restaurant also runs the Elephant Nature Park, a non-profit organization advocating and acting on behalf of the rights of the mighty elephants in Thailand.  Would be great if we could also automatically detect the situation of “elephant abuse” by poachers and other crimes against nature.   Time to get back to my delicious mushroom salad, Northeastern Thai style.

As always, thanks for reading, time for me to get back to eating!