“I am generally not an alarmist, but I have become more and more concerned about the state of our country and its innovation,” she said last week, explaining why she wrote her book, “Closing the Innovation Gap,” which arrives in bookstores Tuesday. “We have a national innovation deficit.”

Ms. Estrin’s book is the latest call to action during the last several years by scientists, technologists and political leaders worried about the country’s future competitiveness in technology.

In 2005, the National Academies published “Rising Above the Gathering Storm,” a report requested by Congress, which found that federal financing of research in the physical sciences was 45 percent less in 2004 than in 1976 and that 93 percent of students in grades five through eight learn science from teachers who do not hold degrees or certifications in the topics.

...

“There is a remarkable telescoping in of vision and an unwillingness to make long-term bets,” said Vinton G. Cerf, the chief Internet evangelist at Google.

Geez, its like no one ever read "The Only Sustainable Edge" or something...

“Event processing has been going on for more than fifty years.”

However, in On Event Processing as a Discipline and Some Subsets another colleague mistakenly blogs,

“… people who dealt in this area [network management and event correlation] have never investigated event processing in the larger sense (e.g. looking at additional patterns), and this area has also not spawned the event processing discipline.”

If you examine just one page from the CEP history at Stanford, researchers there outlined their view of the future applications for CEP, as follows:

• Instant Insight  - hierarchical event viewing applied to the Enterprise IT layer.
  • Analysing business processes
• Network Level Monitoring and Management
• Cyber Security: Network Intrusion Detection
• Enterprise Monitoring and Management
• Modeling and Simulation of Collaborative Business Processes
• Business Policy Monitoring
• Analysis and Debugging of Distributed Systems

These applications areas mentioned by Stanford researchers, including Professor Luckham, support and validate our recent discussion Magic Quadrant for IT Event Correlation and Analysis, 2007 where we concluded that “event correlation and event analysis is Gartner’s closest magic quadrant (MQ)  [...] relates directly to complex event processing (and event processing in general).”  

If you take a detailed look at the 1999 CEP presentation, Defeating Large Scale Attacks: Technology and Strategies for Global Network Monitoring you will readily see that our colleagues are incorrect when they says that event correlational and network management folks have never investigated event processing in the “larger sense”.  For example, the 1999 slides above, Stanford, slide 6, is titled “Complex Event Processing,” defining CEP from the application perspective of event correlation;

Complex Event Processing

• Accept network ‘events’ from any source
  • CISCO NetFlow FlowCollector, tcpdump
• Correlates events based on content and temporal relationship between events
• Event Processing Agents (EPAs) connected in an Event Processing Network (EPNs)
• Both post-mortem and real-time processing

This single event correlational project example from David’s team at Stanford examined the challenging event correlation problems in the context of hierarchical events, maps, patterns, visualization tools, event processing models, patterns languages, network management abstraction layers, and more.  Those core event processing problems from this 1999 example, very large and complex then, still exist today and are much more large and complex - precisely why it is called “complex event processing.”

It is quite obvious, in just this one example, that many folks have been looking at event correlation as a motivating application for event processing, in a larger context, for a long time, contrary to what our colleagues write in their “history of event processing” posts.  

In a future post I will completely debuke these event processing “history revisionists.”   I will illustrate very clearly how the history of event processing goes back at least a decade, and perhaps two (twenty years) before the history outlined in posts like On Research and Practice in Event Processing and The History of Complex Event Processing. 

David Luckam stated that the art-and-science of event processing goes back around 50 years. 

I am not sure I will go all the way back to 1960 in my next post on the history of event processing.  However,  I will go back at least to the early days of Internet Protocol (IP) networking and illustrate why distributed IP networking, network management and network security, is one of the key  motivating factors for what we now call “event processing” and “complex event processing.”

For those of you who aren’t familiar with XSS Filter, a brief summary is that it is a client-side defense against reflected cross-site scripting (XSS) attacks.  It works by recognizing that reflected XSS attacks inject script into the string that the browser sends to the targeted web server.  If the server doesn’t neuter or strip out the injected script, it gets sent back to the browser and executed in the context of the target web page.  Bad things then happen.  At a high level, XSS Filter remembers the string that the browser sent to the server, and looks at the server’s response to see if any of the script was actually in that string.  If it was, then XSS Filter decides that it got there because it was injected by an XSS attack and blocks the script from executing.  The rest of the web page renders as usual.  This is a vastly oversimplified sketch of XSS Filter – for details, see the post by David Ross, inventor of XSS Filter on the Security Vulnerability Research and Defense blog.

So what does XSS Filter have to do with the SDL?  Well, for almost nine years, since XSS was first discovered at Microsoft, we’ve been trying to figure out effective ways to reduce vulnerability to XSS attacks.  Our focus has been on improving the ways that web page developers code their pages, and we’ve developed a lot of tools and techniques for making web content safer from XSS attacks and for detecting XSS vulnerabilities in live pages.  The SDL requires the use of many of these tools and techniques, and we’re sure we’ve prevented a lot of XSS vulnerabilities from being introduced into Microsoft web pages as a result. 

But while we identify (and the SDL requires) measures that allow developers to avoid classes of vulnerabilities, we also look to identify more sweeping solutions that can either 1) eliminate classes of vulnerabilities, 2) reduce their severity, or 3) reduce the likelihood of attacks being successful.  The process usually starts from deep understanding of a class of vulnerabilities and attacks, and then we broaden defenses from there.  In the case of XSS Filter, David’s years of work researching XSS led him to come up with an approach that blocks many of the most common vulnerabilities to reflected attacks found on the web today.  The solution is compatible with existing web pages (doesn’t “break the web”) and thus we were able to enable it by default for users of Internet Explorer 8.  Because it’s a client-side mitigation, it will help protect users from attacks even though the sites they visit may be vulnerable to XSS. 

Our work on buffer overrun defenses follows a somewhat similar pattern – we started by prescribing coding techniques, banning the use of some APIs, and building tools that detect coding constructs that look like buffer overruns.  As we gained a deeper understanding of how buffer overruns can be exploited, we enhanced the /GS compiler flag and added ASLR in a quest to cause classes of exploits to fail even if a buffer overrun remains.  We’re not yet close to eliminating the SDL requirements for use of tools and coding techniques, but the SDL also requires the use of the mitigations to reduce the severity of vulnerabilities that slip past.  Will we ever get to the point where the mitigating technologies are so strong that we can relax the coding requirements?  Maybe not, but we will continue to introduce technologies that reduce the chances of a successful attack.

Similarly, in the case of XSS, even after IE8 ships, the SDL will continue to require the use of safe web site coding practices and tools such as the Anti-XSS library both to protect users of browsers other than IE8 and to provide protection in recognition of the fact that XSS Filter is a mitigation or defense in depth rather than a complete solution.  But we’ll also be keeping our eyes open (and doing active research) in the quest for an even more effective defense – whether client or server side – that eliminates XSS for good.

This post is a little far afield from the normal content of the SDL blog, but I thought it was important to provide a picture of the role of security science and security research in defining SDL requirements and in making major improvements in software security.  You can read more about our work in security science in the Security Vulnerability Research and Defense blog.

A good place to being to look for clues to an answer is Wikipedia, where the opinion of the author there is,

 ”A simple and practical definition, however, would be how an entity (i.e., business) arrives at an optimal or realistic decision based on existing data.”

Quoting the Wikipedia author(s) further,

“Common applications of Analytics include the study of business data using statistical analysis in order to discover and understand historical patterns with an eye to predicting and improving business performance in the future. Also, some people use the term to denote the use of mathematics in business. Others hold that field of analytics include the use of Operations Research, Statistics and Probability. However, it would be erroneous to limit the field of analytics to only statistics and mathematics.”

The Wikipedia author(s) continue their discussion of analytics, as follows;

“Analytics closely resembles statistical analysis and data mining, but tends to be based on modeling involving extensive computation. Some fields within the area of analytics are enterprise decision management, marketing analytics, predictive science, strategy science, credit risk analysis and fraud analytics.”

All of these topics above are CEP-related areas involving complex events and situations based on the need for optimal and reliable real-time capabilities to make meaningful (business) decisions. 

Simple pattern matching, event mediation and routing, and basic mathematical calculations do not really fall into the realm of complex event processing.  Instead, CEP is real-time decision support based on modeling and “extensive” computation.  In a nutshell, complex events and situations require analytical models that are non-trivial and that is why without analytics, there is no true “complex event processing.”

See also:

WIkipedia on Predictive Analytics

The site was inspired by Margaret Atwood's infamous comment that Oryx and Crake isn't really science fiction, because science fiction is "talking squids in outer space." This prompted a hunt for science fiction which actually did feature talking squids in outer space.

“I have quite a few other concerns the with EPTS Member Agreement.   Basically, the agreement needs to be written with an eye toward a more flexible, open and inclusive process that puts the future of the EPTS square into the hands of the event processing community, not a small group of well intended folks who represent a small part of the overall event processing community and worldview.”

Opher’s reply was to just dismiss these comments, a bit surprising since I served the CEP/EP community on the EPTS steering committee; worked quite hard as a matter of fact, for a number of years.   Opher’s appreciation for the years of work is to just off-handly dismiss my comments.

Then in On faithfull representation and other comments and On Top Down and Bottom Up Opher does the same thing, he simply dismisses my comments, defensively, adding humor, sarcasm and fallacy.

I am sorry Opher is so defensive of his narrow society; however I will not yield, because I do not need to resort to sarcasm, fallacy and ad hominums; the facts obviously support my view.  For proof that Opher has a narrow view of event processing, go no further than look at the companies he hand-picked for his EPTS Steering Committee; most startups (or with startup products) in the event processing space, working on common messages to distinguish themselves in a market with much more mature players excluded - classic “not invented here,” isn’t it?

Opher’s claims the EPTS view on event processing is quite general, but the  majority of vendors on the EPTS Steering Committee members are selling similar platforms, a very narrow segment of the CEP/EP space.    Opher claims that he agrees that other domains (like sensor fusion) are significant to CEP/EP, but he simply dismisses my advice to create a true, general EPTS, inclusive of the prior-art and science of CEP/EP (before the marketing folks took over).  He insists on having the EPTS “reinvent the wheel” and develop their own vocabulary, as if event processing did not exist prior to one book on CEP.

Opher’s fun-to-read blog counterpoints to my concerns are evolving to a mixture of ad hominums and sarcasm, sometime wrapped in a defensive tone.   I think we can do better and we must be more inclusive of the other prior-art.  I say we, because I am also a founding member of the EPTS, althought I suspect Opher will banish my name from the membership for trying to diminish the “not invented here” attitude that seems to dominate the EPTS since inception.

The truth of the matter is that the EPTS has a relatively narrow view of event processing, evident by the makeup of the steering committee and the focus of their discussions.    It is not a technical society about event processing, per se; it is a marketing society with a narrowly focused membership that discounts most of the prior-art in the event processing space, it is really, an Event Processing Marketing Society (EPMS) for a narrow group of niche players.

The event processing domain is much, much larger.   The art-and-science of event processing is deep and mature, much more mature (and inclusive) than what we see in the EPTS. 

I think Opher (and the EPTS committee) should take these comments seriously and not discount them with sarcasm and subtle ad hominum replies.