The following excellent discussion is reposted from Streaming SQL approaches insist in ignoring causality by PatternStorm.

The recent paper “Towards a Streaming SQL Standard” by Oracle and Streambase unifies and generalizes two different execution models of Streaming SQL: Oracle’s and StreamBase’s.

While it’s true that the generalization succeeds in overcoming the unability of both execution models of producing correct results for astonishing simple queries (showing evidence of the actual limitations of these two Streaming SQL languages) it is also true that the generalization is closer to being overly complex than natural and intuitive.

The root cause behind the actual limitations of these two Streaming SQL languages is that their execution models “hardcode” the way events can be related to each other: in the Oracle case events are partially ordered by timestamp, in the StreamBase case events are totally ordered by time of arrival. These design decisions (natural in a stream oriented lamguage) have strong implications on what queries can be answered correctly, particularly when these queries involve joins of derived streams.

The generalization, of course, mainly consists in providing a new operator that allows the user to establish custom ordering relationships among the events (the SPREAD operator), which is good news but takes us to the fundamental issue: event processing cannot be reduced to stream processing, that is, to the processing of events that are totally or partially ordered by a pre-defined relationship (as Oracle and StreamBase actual implementations do), on the contrary, no particular ordering can be assumed because the user needs to be able to order the events in different ways in order to solve different problems. This is what event processing is about and the paper provides evidence that Streaming SQL approaches have found the need to move towards that direction and are having trouble in their way.

For instance, one of the queries used in the paper as an example of a query that StreamBase cannot solve (but Oracle can) is the following: correlate the stream that contains the total number of cars on the road for each time interval with the stream that contains the total average speed of the cars on the road for each time interval in order to detect the situation where the avergae speed is below 45 and the total number of cars is two or more. This query can be very easily and more robustly solved if you order the events by causality rather than by time, that is, if you have each position report update the average speed stream and the total number of cars stream and then you causally relate each position report to the new average speed event and the new total number of cars event that it generates; then the query is just a matter of detecting all report speeds that are causally related both to an average speed event below 45 and a total number of cars event of two or more (notice that this approach is more robust than Oracle’s time-based one because it works without requiring derived streams to be synchronized with the report speed stream)

Conclusions:

• Event Processing is a generalization of Stream Processing (as the paper shows)
• Event Processing requires providing the ability to the user of creating custom relationships among events and then define patterns/queries using those custom relationships.
• Causality is more often than not a more robust and easier criteria to order events than time or order of arrival.
• Event Processing Languages should support causality.

Regards,
PatternStorm

As we get closer to the conference dates, I wish that I had made plans to fly back to the US to meet everyone.    However, I have been cutting back on public speaking, taking a break since May.  In addition, Gartner did not ask me to speak at their Event Processing Summit this year, I assume because they did not want to pay airfare for my flight from Thailand to the US.    Also, Gartner always likes to fill their conference speaking slots with as many Gartner speakers as they can, unless you are a paid sponsor; and I noticed a number of Gartner employees speaking in multiple slots.

(Editorial Note) Then again, maybe I complained to much about the lack of organization and conference problems when I was invited at be a Gartner keynote speaker last time - reservations not made propertly,  problems with the guest speaker registration list at sign-in, rooms shifted without notifying the speakers and panelists.   Admittedly, I was not happy with the conference organizers at the last get together.  This was my fault, as I am accustomed to better conference execution and am probally too “picky” about details these days - my bad.  Anyway, the Gartner organizers apologized numerous times, saying they had too many conferences going on at the same time and not enough people to cover them all.

One of the problems with spending so much time in Asia, especially in Thailand, is that guest speakers are really treated as VIPs.  There are usually special comfy couches set up for the speakers and the conference staff really treat you very nice, taking care of you every step of the way.   In fact, there is an entire very nice culture around how guest speakers are treated in Thailand.   Often, they pin flowers on the VIP speakers and take your photos like you are a star.    Very nice culture.

I absolutely look forward to speaking on event processing or CEP at a future venue and meeting everyone face-to-face instead of over the net.  My sincere and deepest apologies for not attending either the Gartner or the EPTS event this year.   

PS:  If you take up a collection and send me a RT business class air ticket, I might change my mind

Image by goddam via Flickr

Ah, its almost August.  Football training camps are open and the Yankees and Red Sox are battling. Does it get any better?  For most of this year I thought the Yankees were going to be out of it this year and content to have a rebuilding year.  We have several veteran players who past their prime and whose contracts are up after this year.  We have a some great young talent that need to grow into their potential.  It looked like the Bosox and Tampa Rays were going to run away with the division and wild card this year.

But like inevitable turning of the seasons, sometime after July 4th and then the All Star break, the Yankees beginninng their drive. Those old bones warm up in the heat of the summer and the bats come alive. This year the pitching is carrying them too.  Old pros Andy Pettite and Mike Mussina are joined by Jobba Chamberlin.  Mariano Rivera is still the best closer in baseball.  Just like old times the Yanks went out and fleeced some 2nd division team for a bunch of minor leaguers and added a quality hitter and pitcher right before the trade deadline.  Look around and we are one game behind the Red Sox for the wild card slot and only three games behind the Rays for first place!

I still think Tampa is going to stumble and it will come down to the Sox and the Yanks. Just the way it is supposed to be. I am heading up to NY next Friday, taking my sons to the shrine that is Yankee Stadium to see it in person in its last year.  The rest of the baseball season is going to be very exciting.  Again, just the way it is supposed to be!

Image by goddam via Flickr

Ah, its almost August.  Football training camps are open and the Yankees and Red Sox are battling. Does it get any better?  For most of this year I thought the Yankees were going to be out of it this year and content to have a rebuilding year.  We have several veteran players who past their prime and whose contracts are up after this year.  We have a some great young talent that need to grow into their potential.  It looked like the Bosox and Tampa Rays were going to run away with the division and wild card this year.

But like inevitable turning of the seasons, sometime after July 4th and then the All Star break, the Yankees beginninng their drive. Those old bones warm up in the heat of the summer and the bats come alive. This year the pitching is carrying them too.  Old pros Andy Pettite and Mike Mussina are joined by Jobba Chamberlin.  Mariano Rivera is still the best closer in baseball.  Just like old times the Yanks went out and fleeced some 2nd division team for a bunch of minor leaguers and added a quality hitter and pitcher right before the trade deadline.  Look around and we are one game behind the Red Sox for the wild card slot and only three games behind the Rays for first place!

I still think Tampa is going to stumble and it will come down to the Sox and the Yanks. Just the way it is supposed to be. I am heading up to NY next Friday, taking my sons to the shrine that is Yankee Stadium to see it in person in its last year.  The rest of the baseball season is going to be very exciting.  Again, just the way it is supposed to be!

But in the end, it seems like Tucci didn’t have faith that Greene had the chops to run the successfully growing company anymore. So she could build it to the stature it has now but just as MS comes out of the gates, all of a sudden she’s no good? Boy, I can’t wait for Greene’s book on this. CEOs, take heed – don’t be too successful or the board will fire you. (Or alternatively don’t let the guy who doesn’t like you stack the board!)

So where does VMware go from here? Rachel Chalmers, Research Director for Infrastructure Management at The 451 Group, places a bet on cloud computing – saying that VMware plans to offer a new suite of cloud computing at the next VMworld Conference. And here’s a nice piece on the Burton Group’s Data Center Strategies Blog that suggests another multi-pronged winning strategy.

Oh no. The virtualization management space, if it didn’t before, is beginning to remind me of the Internet boom time when everyone and their brother (literally, ask me about it sometime) got into the act. Introducing, DynamicOps and their product, Virtual Resource Manager (VRM). The two-week old company and product are spinouts from Credit Suisse, where the original solution was home-grown and in production for more than 2 years, managing thousands of virtual machines. I’m really interested in taking a closer look at it and seeing just what VRM does differently to meet the unique requirements of virtualization management at such a scale.

Forrester Research released a research report on “the Five Essential Metrics for Managing IT.” The study relates the “Operational Health” metric to the measuring of IT failures. Dave will be happy to note that the report uses one of his favorite phrases – talking about the “dial-tone reliability of IT services”.

ShareThis

One of the cool things about the first dot com bubble was the "ebay millionaire". These were people who built businesses around selling goods at auction on ebay.  There has been much written and said about the methods of these people and certainly it was a big attraction to people selling on ebay.  I had an interesting plane ride home today where I met someone and discovered todays equivalent. I call it the StubHub millionaire. It  is a testament to American ingenuity and shows that given the tools, people will find a way to exploit and make money.

Up until fairly recently you bought tickets to sporting events and other entertainment from a box office or ticket agent such as ticketron.  The "after market" in ticket sales or scalping as it was called in NY was often times illegal.  There were though some legal ticket brokers that you could buy tickets from. Now with the advent of StubHub and similar type of ticket reselling outlets on the web though, the infrastructure is in place for anyone to sell tickets on line.  You would think that most of these people selling tickets were people who had either extra tickets to an event or perhaps a season ticket holder looking to unload some tickets to help defray the costs. Not the case!

There is a now a whole class of businessman who buys season tickets to multiple teams, sports and cities and than uses outlets like StubHub and others to sell these tickets.  The guy I spoke to today had season tickets to 6 different NFL teams, 3 major league baseball teams and multiple basketball and hockey teams.  Many of his tickets are sold months and weeks before the event. If any are left within 14 days of the event he puts them on ebay.  His average mark up is about 40 to 50% of face value, but by buying season tickets he pays below face, so his actual margin is closer to 60 to 70%. He keeps a few tickets for him and his family to go to a few games a year. 

This started as a hobby for him with Yankee season tickets, but he has done an analysis and compared to what he would make investing that money in the market, he has come out way, way ahead.  He thinks that on a 12,500 investment, he makes about 40k!  That is not bad.  This year when all is said and done he will make six figure income from the resale of tickets he bought.  Think about it, no office or anything.  Just list your tickets and let people buy them.  Take some of the money and buy more tickets.

So what the heck am I doing trying to show people why it is important that they put good security in place on their computers?  There has got to be a better way.

One of the cool things about the first dot com bubble was the "ebay millionaire". These were people who built businesses around selling goods at auction on ebay.  There has been much written and said about the methods of these people and certainly it was a big attraction to people selling on ebay.  I had an interesting plane ride home today where I met someone and discovered todays equivalent. I call it the StubHub millionaire. It  is a testament to American ingenuity and shows that given the tools, people will find a way to exploit and make money.

Up until fairly recently you bought tickets to sporting events and other entertainment from a box office or ticket agent such as ticketron.  The "after market" in ticket sales or scalping as it was called in NY was often times illegal.  There were though some legal ticket brokers that you could buy tickets from. Now with the advent of StubHub and similar type of ticket reselling outlets on the web though, the infrastructure is in place for anyone to sell tickets on line.  You would think that most of these people selling tickets were people who had either extra tickets to an event or perhaps a season ticket holder looking to unload some tickets to help defray the costs. Not the case!

There is a now a whole class of businessman who buys season tickets to multiple teams, sports and cities and than uses outlets like StubHub and others to sell these tickets.  The guy I spoke to today had season tickets to 6 different NFL teams, 3 major league baseball teams and multiple basketball and hockey teams.  Many of his tickets are sold months and weeks before the event. If any are left within 14 days of the event he puts them on ebay.  His average mark up is about 40 to 50% of face value, but by buying season tickets he pays below face, so his actual margin is closer to 60 to 70%. He keeps a few tickets for him and his family to go to a few games a year. 

This started as a hobby for him with Yankee season tickets, but he has done an analysis and compared to what he would make investing that money in the market, he has come out way, way ahead.  He thinks that on a 12,500 investment, he makes about 40k!  That is not bad.  This year when all is said and done he will make six figure income from the resale of tickets he bought.  Think about it, no office or anything.  Just list your tickets and let people buy them.  Take some of the money and buy more tickets.

So what the heck am I doing trying to show people why it is important that they put good security in place on their computers?  There has got to be a better way.