Defining Cloud Computing

Cloud computing is a very active community. The Google Group gets 600 posts per month and many bloggers are covering the space. However, “cloud computing” is impossible to define in a way that satisfies everyone (or even most). Cloud computing is not alone in this controversy, consider the definition and meaning of “Web 2.0″, “mashups” or “RESTful architecture”. All of these terms are relatively recent. According to Google Trends, these terms became popular to the general public sometime between 2005 and 2007:

• Web 2.0 - often confused with RIA, AKA Social Computing, Long-Tail Apps, Crowdware (2005 by O’Reilly Media)
• Mashup - made popular by Google Maps, AKA Composite/Situational Apps. (2005)
• REST - Has a strict definition, but many don’t understand it and abuse the term. (2006 by R. Fielding)
• Cloud computing - collides with many other terms, such as SaaS, Grid, Utility, PaaS, etc. (2007)

The definition of cloud computing is in progress:

There’s a Darwinian evolution of the exact definition of cloud computing running around. We’re about a country mile away from “knowing when I see it”, which is excellent progress. The cloud to everyone’s silver-lining has enough material to write a 3 volume desktop reference at this point. - Michael Cote, June 2008

Definition #1 - “Cloud computing is the realisation of Internet (”Cloud”) based development and use of computer technology (”Computing”) delivered by an ecosystem of providers. - Sam Johnston, July 2008

Definition #2 - “Cloud computing = network computing. I love the idea of cloud computing, the next evolution of the most network intensive architecture possible, but one that if it works well, is transparent. It’s all about the transparency.” - Douglas Gourlay, Cisco, May 2008

Definition #3 - “There seems to be a group myopia around so-called “cloud computing” and its definitions. What we’re really talking about are “cloud services” of which, “computing” is only a subset…Cloud services are not SaaS. They are far more akin to web services…” - Randy Bias, neoTactics, May 2008

(Anti-)Definition #4 - “Note that I refer to cloud services, not to the could. I am not interested in defining cloud as a term, because I don’t think it’s very useful. For those of us in the distributed computing’s pace

The Working Definition (Winner!):

“…the notion of providing easily accessible compute and storage resources on a pay-as-you-go, on-demand basis, from a virtually infinite infrastructure managed by someone else. As a customer, you don’t know where the resources are, and for the most part, you don’t care. What’s really important is the capability to access your application anywhere, move it freely and easily, and inexpensively add resources for instant scalability.” - Mitchell Crandell, Rightscale, June 2008

Taxonomies of the Cloud Space

Taxonomies are useful to provide insight into a market. It classifies a multitude of players into a smaller bucket.

Andreessen’s Platforms - September 2007

Provided an early taxonomy model for emerging cloud platforms

Platform being a system that can be programmed

• Access API - platform that provides web service endpoints
• Plug-In API - platform invokes your code, that you have deployed remotely
• Runtime Environment - your code runs inside the platform’s process space.

Mehta 11 Layer Stack, April 2008

1. Facilities (space, power, cooling)
2. Network
3. Hardware (e.g. servers Amazon EC2 runs)
4. Hardware virtualization (e.g. Xen for EC2) - optional
5. O/S (e.g. Linux)
6. Systems Management (e.g., tools to manage EC2 instances)
7. Application Middleware (e.g., MySQL on EC2)
8. Application Code
9. Application APIs / Web Services
10. GUI for Application
11. GUI for Application Development / Customization

Croll Cloud Stack, June 2008

7 layer stack within Turnkey app and Generic Platform.

Turnkey app

• SaaS
• Extensible app
• Generic IDE
• Constrained APIs
• App Cluster
• Virtual Data Center
• Virtual Servers

Generic Platform

The bottom of Alistair’s stack includes “root access “style compute clouds.

Robert Anderson, July 2008

3 layer stack

• Software (SaaS)
• Platform (PaaS)
• Infrastructure (IaaS)

This is the model taxonomy for this session.

Related Concepts and Terms

• Infrastructure as a Service (IaaS), Hardware as a Service (HaaS) are synonyms to cloud infrastructure.
• Virtualization
• Hosting
• Autonomic computing
• Distributed computing
• Grid computing

Cloud Applications

• SaaS
• S+S (Software+Services)
• Managed Service Provider (MSP)

To solve a complex problem in the blackboard-style, the intelligent agents cooperate as functional specialists, observing updates to the blackboard and self-actualizing in an event driven process) when there is new information to process.  Agents continually update the blackboard with partial solutions when the agents capabilities for processing match the state of the blackboard. 

The blackboard architecture is a distributed computing model for a metaphor describing how people work together to collaboratively solve a problem around a blackboard (whiteboard in todays lingo).   For example, one person is standing at the whiteboard working on a solution while three other people are sitting (or standing) around watching.   One of the observers sees new information on the whiteboard, thinks of how he (or she) can contribute, and then jumps up, takes the whiteboard marker from the person working, and adds to the solution.  This process is repeated in various scenarios.  

The blackboard architecture can be very effective in solving complex distributed computing problems, including event processing problems; however, scheduling the self-actuating agents can be a key challenge.   Another core challenge is how to model and manage the blackboard itself, especially in distributed blackboard architectures.  

John McManus, former CTO of NASA, wrote an excellent PhD dissertation in 1992,  Design and Analysis Techniques for Concurrent Blackboard Systems, at the College of William and Mary, addressing challenges in BB systems.

The table below lists two books that focus on blackboard architecture:

One of the thought leaders in blackboard architecture is Daniel D. Corkill a professor at the University of Massachusetts Amherst. 

Blackboard architecture is relevant to the field of event processing, and in particular complex event processing.   I will go into more details in future blog posts on this topic, including how blackboard architectures relate to grid computing, distributed object caching (of the blackboard), and CEP.

The two CEP/ESP related Water’s categories were, Best Streaming Data Management Solution and Best Complex Event Processing Solution.    Water’s awards Best Streaming Data Management Solution to data/event stream processing company StreamBase; and then awards Best Complex Event Processing Solution to Oracle’s BEA product, which is built on top of another data/event stream processing engine.  Confused?   

Alexander Alves,  currently employed by Oracle, previously having worked for BEA Systems, in Best Complex Event Processing Solution, observes:

“Regardless, I find it intriguing that Waters not only does not state the differences between the categories, but also uses the term CEP several times in the SDMS category.

I guess the verdict is that there is still confusion amongst the experts regarding event and stream processing… And that both products must be very good.”

Of course, Alex must be politically correct, and rightly so, since he works for Oracle/BEA and Water’s gave them an award.  But on what tangible, objective basis for Best Complex Event Processing Solution?

According to our 2007 survey, CEP/EP Reference Customers 2005-2007, BEA was in last place based on public CEP/EP reference clients.  

Waters’ award  for Best Streaming Data Management Solution to StreamBase is a much more credible.   Congratulations StreamBase.   Most would agree that StreamBase is a streaming data management solution (SDMS), but so are Apama and Coral8 (and BEA etc etc).   

Waters simply muddies the water, unfortunately.  

In On the Maturity of Complex Event Processing, the author concludes:

“I think [... the. ...] comment at the end of [... the. ...] post “we shouldn’t feel compelled to thwart that growth with a claim that the products are not ‘mature’ when they actually are in a lot of ways” is quite revealing. The fact that such a level of debate about CEP’s maturity is taking place, and the fact that [... someone ...] is concerned that the debate might stifle growth, is itself indicative of an immature market segment in my opinion.”

This quote is compelling.  When vendors disagree with the direction and tone a debate is going and they call to end the debate, labelling the discussion “a distraction” - it tends to prove the premise of the original post Deciphering the Myths Around Complex Event Processing  by Ivy Schmerken;  the CEP market, both exciting and promising, is today, mostly immature and brittle. 

For more conclusive evidence, I turn our readers attention to this post, An Overture to the 2007 CEP Blog Awards,  That analysis was based, in part, on CEP/EP Reference Customers 2005-2007 where we documented 18 public “CEP reference clients” in 2007 (27 for the entire period 2005 - 2007).

Twenty seven public reference clients over a three year period with 18 last year (2007) do not demonstrate a mature market or technology domain.

————————

Footnote:

Here were the results of the CEP/EP Reference Customers Survey” for 2005-2007:

Reducing complex problems sets to simple problem sets is an interesting, and sometimes valid, approach to complex event processing.    Transformations can be useful, especially when well defined.

For example, CEP was evisioned as a new technology to debug relatively large distributed systems, discover hidden causal relationships in seemingly disconnected event space.    This “discovery” requires backwards chaining with uncertainty, for example.  Most of the current so-called “CEP software” (on the market today (including Marc Adler’s SQL-based examples) do not perform backwards chaining (with uncertainty).   This is also true from other so-called CEP products, like most forward chaining RETE engines - for example, see this post.

Marc Adler says he is, “hunting for advice from people who might have implemented event clouds in Coral8, Streambase, and Aleri, all three which are based on SQL.”

Current steaming SQL engines cannot model true event clouds without reducing the cloud to causal-ordered sets of linear steaming data.   These software tools are stream processing engines that process events in a time window of continuous streaming data.  These products are not, in reality, CEP engines - calling them ”CEP engines” is marketing-speak, not technology-speak!

Reducing complex models to simple ones is a valid technique for problem solving.  Likewise, eliminating uncertainty and assuming causality is a way to reduce complexity. 

CEP was envisioned to discover causal relationships in complex, uncertain, ”cloudy data” and the current state-of-the-art of software from the streaming SQL vendors do not have this capability, unless you reduce all event models to ordered sets of streaming data (reduce POSETS to TOSETS).

Reductionism can be a valid technique, of course.  We can eliminate uncertainty, force strict determinism, demand apriori system rules and perform all sorts of tricks to permit us to reduce complex problems to simple ones.  

However this also resulting in reducing CEP  (complex event processing) to SEP (simple event processing).  

Today is an especially joyful occasion on The CEP Blog.    I am pleased to announce that one of the world’s top experts on CEP, Dr. Rainer von Ammon, has joined the blog.

Dr. Rainer von Ammon is managing director of the Centrum für Informations-Technology Transfer (CITT) in Regensburg. Until October 2005 he was Professor for Software Engineering, specializing in E-Business infrastructures and distributed systems, at the University of Applied Sciences Upper Austria. Rainer is still teaching there and at the University of Applied Sciences of Regensburg. From 1998 to 2002, he worked as Principal Consultant and Manager for R+D Cooperations at BEA Systems (Central and Eastern Europe). Prior to this, he was Professor for Software Engineering in Dresden with a focus on development of applications with event driven object oriented user interfaces and component based application development. Before this Rainer was acting as manager of the field Basic Systems at the Mummert + Partner Unternehmensberatung, Hamburg. After finishing his studies of Information Sciences at the University of Regensburg, he started as project leader of Computer Based Office Systems (COBIS) from 1978 to 1983 and afterward founded a start up company with some of his colleagues.

Some of you may recall my recent musings, A Bitter Pill To Swallow: First Generation CEP Software Needs To Evolve.   When you read Rainer’s excellent reply, you will quickly see why we are very pleased to have his thought leadership here at The CEP Blog.  Dr. von Ammon and his team are leading experts in CEP and related business integration domains.  Not only does he provide thought leadership, his team  researches, develops, implements and tests CEP solutions.   

In another example of  his thought leadership, some of you might recall this post, Brandl and Guschakowski Deliver Excellent CEP/BAM Report, where Hans-Martin Brandl and David Guschakowski of the University of Applied Sciences Regensburg, Faculty of Information Technology/Mathematics (CITT), advised by Dr. von Ammon, completed an excellent CEP thesis, Complex Event Processing in the context of Business Activity Monitoring. 

Please join me in extending a warm welcome for Dr. Rainer von Ammon to The CEP Blog.

I was wondering when IBM would actually jump into the event processing market.  

Well, it was announced today that IBM will acquire Aptsoft, adding an event stream processing (ESP) platform to the IBM WebSphere porfolio.  IBM will also gain AptSoft’s event processing reference customers.  This was a smart move by IBM.

Oracle is acquiring BEA which uses Esper under the hood, another stream processing engine. 

However, AptSoft has a more advanced user interface and graphical design-time environment when compared to the Oracle/BEA/Esper platform,; so the IBM/WebSphere/AptSoft offering will propel IBM to the top of the event stream processing market.

This also means the ESP market leaders will be Progress Apama and IBM.  It is a sure bet that IBM will compete directly with Apama in the financial services sector with their acquisition of AptSoft.

Now, it’s TIBCO’s turn to acquire an ESP platform to compliment their process-driven event processing offering.

Today I noticed that SL Corporation has revamped their website with a new page, Solutions for CEP Engine Users.    The page is well written, reinforcing some of my earlier posts on the value proposition for CEP; so I hope the folks at SL don’t mind if I repost their excellent thoughts on BAM and CEP here. 

Solutions for CEP Engine Users by SL Corporation

© 1999-2008 Sherrill-Lubinski Corporation. All rights reserved.

Complex Event Processing (CEP) is a relatively new technology that is used to help companies detect both opportunities and threats in real-time with minimal coding and reusable key performance indicators (KPIs) and business models. Just as services are shared and reused in a SOA, CEP permits the sharing and reuse of KPIs in business activity monitoring while efficiently processing events so businesses can act on situations that impact business and take advantage of real-time processing.

Business activity monitoring, often referred to as BAM, is the capability that Gartner and other distinguished analysts use to describe this visualization capability in the business world. BAM introduces a human element to CEP. It is well-established that the human mind is, today and for the foreseeable future, far superior to machine intelligence in making sense out of complicated situations and events. Therefore, BAM is critical to the success of any complex event processing (CEP) solution.

Depending on an organization’s mission, BAM can be used in various levels within an event processing solution to help users visualize and understand the dynamics behind rapidly changing situations and critical business events. In other words, BAM plays a key role wherever there is a need for better insight into the myriad events that effect your business operations.

BAM provides real-time visualization and alerting capabilities for users to better understand how business events impact their organization. BAM software permits users to quickly prototype, build and deploy event processing business solutions. For example, a telecommunications company would find BAM useful to achieve event-driven SLA monitoring and management; and a large retailer would find BAM important as they stay on top of business-critical events in their supply chain.

Insight gained from BAM, in concert with event processing solutions, enable organizations to make better and faster business decisions so they can rapidly sense and respond to threats, problems and opportunities. BAM solutions permit applications to be designed, deployed and modified rapidly with minimal or no coding resulting in significantly lower development costs. Therefore, a key benefit of BAM in real-time event processing solutions is that KPIs can be deployed, monitored, revised, reused and utilized, economically and rapidly.

Depending on the business application, BAM-enabled visualization is required at numerous levels in an event processing architecture. For example, events from across the enterprise are typically processed by a CEP software platforms from companies such as TIBCO, BEA (soon to be Oracle), Progress Apama, StreamBase, Aleri, and Coral8.

Long before KPIs are displayed to the business users, BAM tools can be configured to assist application developers to monitor and visualize the raw event stream. For the developer, their business is developing applications, and BAM can be very useful when designing KPIs for event processing applications.

Fine-tuned KPIs that have been derived from an event processing application are displayed to the business user. These KPIs can indicate risks, threats, problems, opportunities and other emerging business situations that impact the business.

BAM, in concert with state-of-the-art event processing software, provides the framework for a complete sense-and-respond capability for businesses. Processing raw events and event streams for business opportunities and threats requires robust and rapidly deployable visualization solutions. This is the reason that many distinguished analysts believe that BAM and CEP are complementary and critically interdependent core business capabilities. We at SL Corporation agree, and are pleased to be the leading BAM visualization platform in the event processing/CEP ecosystem today.

© 1999-2008 Sherrill-Lubinski Corporation. All rights reserved.

We all know that Oracle just bought BEA.

Personally, I would have recommended Oracle to buy TIBCO instead of BEA. TIBCO has great technology and their software stack is richer and more diverse that BEA’s. TIBCO spends a lot of  development resources on their graphical user interfaces and design-time and modelling environment to make business integration very easy.  TIBCO’s stockholders, like most great companies with a long history of the same executive management and management style, would greatly benefit from the acquisition.

Citigroup’s John Reilly Walsh upgraded TIBCO (TIBX) shares to Buy from Hold based on Oracle’s purchase of BEA. John thinks TIBCO, the last real middleplayer on the block, will also be purchased, and names IBM as the most likely candidate, “but adds that SAP, Hewlett-Packard (HPQ), Oracle (ORCL), Sun Microsystems (JAVA), EMC and Cisco (CSCO) all could potentially be interested.”

If Oracle bought TIBCO, a very interesting idea, that would leave Oracle the King of Integration (KOI). I don’t think HP, EMC or Cisco would want to purchase a company that is so fundamentally different than their core business.

This begs the question if Sun would want to buy TIBCO and challenge Oracle now that Sun has purchased MySQL?

So the three most likely scenarios are:

• IBM buys TIBCO
• Sun buys TIBCO
• Oracle buys TIBCO

In my opinion, the most interesting scenario would be Sun following their purchase of MySQL with a purchase of TIBCO. This could create a strong competitor to Oracle.

On the other hand, Oracle would benefit from the purchase, if only a defensive mechanism against a Sun/MySQL/TIBCO triple-threat, and they would get great technology at the same time.

I would be surprised if IBM buys TIBCO, but if they do, this would also keep things interesting!

From a cultural perspective, the TIBCO culture and the Sun culture are the best match.   I don’t think that the SAP or  IBM cultures are very suitable for TIBCO employess.  So, if you toss in the cultural perspective, Sun, cash rich and in acquisition mode, seems the most likely candidate to buy TIBCO.

After three months of wrangling over prices, Oracle Corp. will acquire BEA Systems in a $8.5 billion deal.

This means that Oracle will now have an event processing platform, the Oracle WebLogic Event Server to compliment their product line.

Reference:  Oracle Strikes Deal to Buy BEA Systems for $8.5 Billion  (Wall Street Journal)

Oracle Corp. said it will acquire BEA Systems in a $8.5 billion deal three months after BEA slapped away an Oracle takeover offer as too low.

Oracle would pay $19.38 for each BEA share, a 24% premium to Tuesday’s close price of $15.58.

Oracle made an unsolicited $6.7 billion, or $17 a share, takeover proposal in October, but the company let it expire weeks later after BEA said the bid was unacceptable. At the same time, BEA added it was looking to start negotiations with interested parties willing to pay at least $21 a share.

“The addition of BEA products and technology will significantly enhance and extend Oracle’s Fusion middleware software suite,” said Oracle Chief Executive Larry Ellison. “Middleware” is a general term for any programming that serves to mediate between two separate and often already existing programs.

BEA Chairman and CEO Alfred Chuang called the deal the culmination of a “diligent and thoughtful process” to maximize stockholder value. The company’s largest shareholder, billionaire Carl Icahn, had called for an auction to sell the business-management-software firm.

BEA is one of the few independent, midsize software companies left in Silicon Valley as the technology industry consolidates. Oracle has for years eyed BEA as an acquisition target.

BEA has been battling Oracle, International Business Machines Corp. and others in the market for middleware. BEA, with a product called WebLogic, pioneered one category of middleware called application servers that are used to build Web services.

Oracle expects the buyout to boost earnings by one cent to two cents a share, excluding items, in the first year after the deal closes. That is slated to happen by midyear.

Shares of Oracle fell in premarket trading to $20.80 after closing Tuesday at $21.31.