I agree with Mark that SOR is important and very interesting; but in his reply he seems to be confusing CEP with “complex EAI” or a “complex messaging” application.  For example, Mark says,

“It’s not uncommon for a single SOR system to connect to 10 or more markets and multiple asset classes.  Not only is this a confluence of events, it’s a stunningly complicated environment in which to create a complex, real-time model in which to apply “simple” routing decisions. On this basis alone, SOR needs CEP.”

Connecting to many market feeds with multiple asset classes might be complicated, but “complicated connections” are an EAI  (adaptation layer) function, not a core CEP function.   In fact, TIBCO Software has been doing this type of low latency back-office order routing for many years, and TIBCO historically calls this “messaging.”  Adding some rules to high speed, low latency messaging does not make it a “CEP” application.

Mark goes on to set up a counter argument to ILOG’s Changhai Ke, comments with,

“SOR operates by analyzing the confluence of events from market data feeds, order flows from OMS systems, and executions, aggregating and analyzing those events in real time, and adjust routing decisions on the fly.”

This is the well travelled argument the “new stream processing vendors in capital markets” have been saying, still unconvincingly, for the last few years.  Basically their perspective is that if you have a lot of ”feeds” and a core requirement for “speed” - “feeds and speed” - you are doing “complex event processing.” 

Mark Palmer forcefully stated his opinon that the folks who do not agree with him do not “understand” modern day SOR.    However,  a strong counter argument can be made that the “newcomers” to capital markets like StreamBase do not understand that “feeds and speeds” with order routing is little more than moderan day EAI.   This is a basic message routing capability and it has been around for a long time.  After all, Wall Street operated quite well before the term CEP was coined!  TIBCO technology was providing Wall Street back office, low latency, smart order routing a decade ago, and they called this technology “messaging”.  

So, I remain unconvinced, at least by Mark’s passionate counter post, that SOR is CEP.   SOR, as Mark and other have described it, is a low latency messaging technology.  Message routing rules have exisited in this technology space for decades.

I agree with Mark completely that low latency EAI (like SOR has been described) can be quite complex, from a “feeds and speeds” perspective.   However,  I remain skeptical that “feeds and speeds” is much more than  modern day messaging and message routing.

In closing, in the network and security management world we have been dealing with “myriad feeds and speeds” for as long as I can remember, but admitted not like capital markets.    Taking myriad feeds, running rules against the feeds and then routing the messages/events for further processing, regardless of the complexity of the feeds and the data, is actually more of a messaging/ESB technology than a CEP technology. 

I remain completely open minded to any convincing counter arguments.

In a tale rich with incongruities, the Communist-run government of West Bengal State invited the Tata Group, a symbol of Indian capitalism, to set up its plant in an area called Singur. It acquired 1,000 acres from farmers on the company’s behalf.
As the project advanced, some farmers who had sold their land demanded it back. The main state-level opposition party, the Trinamool Congress, led protests demanding that the land be returned. Most people sympathetic to Tata accused the opposition of inducing the farmers to protest, while Tata’s critics said the farmers had legitimate grievances.

The issue simmered for months. But in recent days, protesters began surrounding the plant, blocking roads and preventing Tata workers from reaching the plant. “The existing environment of obstruction, intimidation and confrontation has begun to impact the ability of the company to convince several of its experienced managers to relocate and work in the plant,” Tata said in a statement on Tuesday.

The halt to the plant has caused many Indian business people to warn of a chilling effect on investment in the country. It is also unclear how Tata will be able to keep the Nano’s cost so low, since part of the affordable price reflects the company’s savings on the land in Singur.

There is a fundamental asymmetry between state and markets. It is easier to create markets than it is to create state capacity or to prevent its deterioration. Creating markets is a lot about letting go, establishing a reasonable policy framework, and allowing the natural hustling instinct to take over. In other words, hustling is the natural state. Building state capacity, on the other hand, is quite different. It involves overcoming collective action problems, mediating conflict, creating accountability mechanisms where outputs are multiple and fuzzy and links between inputs and outputs murky, and contending with the deep imprints of history. In Weber’s memorable words, building public institutions is like the “slow boring of hard boards”.

In that light, China’s task of improving its private sector seems easier to accomplish than India’s task of arresting institutional decline. So, while China and India can probably both count on more years of high growth, the odds still favour China pulling off that feat than India. That, and not just the meagre medal tally, should be what India mulls over after the Beijing Olympics.

It's easier to liberalise a functional state than it is to functionalise a dysfunctional one, of any ideological stripe.

Parlez-vous open source? While wide-spread open source usage is still debated in many companies, the French have been advocating for all open source all the time in government and education. French President Nicolas Sarkozy set up an economic commission that recommended tax benefits to stimulate more open source development. Lesson learned from France: start ‘em early. “All students in France use open source.”

Just in time for Labor Day, John Edwards (no, not that one) comes out with an informative guide on “Who provides what in the cloud”. No doubt, this will be a rapidly expanding list, but what’s really interesting is the comment on the article. People have very strong opinions on the cloud…

Research firm Aberdeen Group reports that network costs will increase slightly more than 5 percent over 2007. Contributing factors: “need for speed”, shift from standard to mobile PCs (more end points of connectivity), and the ever-expanding network. And of course the hidden costs of multiple tools with multiple management consoles – if you’re not smart enough to choose say a comprehensive network management solution that is vendor agnostic…One tool to monitor them all…

And just because I miss the Olympics already, here’s an irreverent take on what it’s like to lose to Michael Phelps. http://www.thetechstop.net/?p=1503

Enjoy your long Labor Day Weekend!

More from Info Week:

Rather than indicating a deteriorating security situation, the rising number of reported data breaches may just mean corporate security auditors are better at finding compromised systems, [ITRC founder Linda Foley ] suggested.

The Identity Theft Resource Center points out that the actual number of breaches this year is probably higher than 449 so far because of underreporting and because breaches affecting multiple businesses tend to be reported as a single event.

Read the whole story here.

So, Anton Security Tip of the Day #16: Virtually Screwed - Journey Into VMWare ESX Log Analysis

CISecurty guide for VMWare (here) and DISA STIG for virtual machines (here) both mandate collection and analysis of VM platform logs; none goes into enough details on what to look for in logs. Let's try to shed some light on security-focused log analysis of VMWare ESX v. 3.x logs.

First, at least until ESXi becomes the default choice, one needs to keep in mind that ESX as "Linux-inside" and thus diving into /var/log will not reveal any "alien technology" (well, not much :-)). However, one of the most useful logs is /var/log/hostd.N which is not a descendant of Linux standard logs. Extensive VM event records are written into this file.

Let's focus on various types of logins to the ESX platform and identify logs that indicate a successful and failed attempts to log in. Here are a few useful examples to analyze:

Successful logins:

• May 30 09:20:42 esx2 su(pam_unix)[9405]: session opened for user root by jhonny(uid=1626)

This is a classic Linux root login message; you can watch for these by searching VMWare ESX logs for "session AND opened AND user AND root."  Notice the user name of the user who switched to root.

• May 30 09:20:34 esx2 sshd(pam_unix)[9364]: session opened for user jhonny by (uid=0)

This is also a classic Linux message for a normal (non-root) user login.

• [2008-05-25 06:57:48.774 'ha-eventmgr' 111639472 info] Event 40645 : User jhonny@1.1.1.1 logged in

This is a VMWare -specific application login to ESX. You can track such events by username, by event ID or by keywords "event AND logged AND user" (if you are using search)

Failed logins:

• May 30 09:20:31 esx2 sshd[9356]: Failed password for jhonny from 1.1.1.1 port 54773 ssh2

Another classic Linux message from the ESX system; a failure to login due to incorrect password.

• May 27 12:06:59 esx2 sshd[4756]: Failed password for illegal user jonny from 1.1.1.1 port 30594 ssh2

A message indicating a failure to login due to incorrect username (note a typo).

• May 25 07:03:48 esx1 sudo:     jhonny : 3 incorrect password attempts ; TTY=pts/0 ; PWD=/var/log ; USER=root ; COMMAND=/bin/bash

This ESX Linux platform message should also be familiar to Linux/Unix admins: it indicates multiple sudo password failures; look for such messages in the logs.

BTW, do you need to be reminded to track NOT only failed, but also successful login events?!

Overall, you must prepare for the future by learning to analyze  VMWare logs, just like you handled "legacy OS", such as Linux/Unix and Windows.

As I said before, I am tagging all the tips on my del.icio.us feed; here is the link: All Security Tips of the Day.

adaptive process management (n.) an element of resource and business process management, adaptive search and event processing. Sometimes referred to as “Level 4” event processing or process refinement.

application concept (n.) a definition of a set of properties that represent the data fields of an application entity. An application concept can describe relationships among themselves. For example, an order concept might have a parent/child relationship with an item concept. A department concept might be related to a purchase requisition concept based on the shared property, department_id. Application concepts can include an application state model.

application state modeler (n.) a UML-compliant application that allows you to model the life cycle of a concept instance — that is, for each instance of a given concept, you can define which states it will pass through and how it will transition from state to state. States have entry actions, exit actions, and conditions, providing precision control over the behavior of an event processing agent. Transitions between states also may have rules. Multiple types of states and transitions maximize the versatility and power of the application state modeler.

derived event (n.) an event that is created as a result of processing one or more other events.

complex event (n.) an event that is a situation-entity abstraction of two or more simple, derived or other complex events.

complex event processing (n.) CEP is a technology for extracting information from message-based systems. CEP is primarily an event processing concept that deals with the task of processing multiple events from an event cloud with the goal of identifying the meaningful events within the event cloud. CEP employs techniques such as detection of complex patterns of many events, event correlation and abstraction, event hierarchies, and relationships between events such as causality, membership, and timing, and event-driven processes.

event (n.) a instance of an event definition. It is an immutable object that represents a business activity that happened at a single point in time. Just as one cannot change the fact that a given activity occurred, one cannot change an event — events are immutable.

event aggregation (n.) the aggregation of simple, derived or complex events into higher levels of event abstractions.

event definition (n.) a set of properties related to a given activity that represents an important or interesting change of state in a human, system or computational activity. An event definition includes event properties such as event priority, event time to live (TTL), and a description of the payload, which is comprehensive information related to the activity that occurred. Events expire when the TTL has elapsed, unless the event processing agent has instructions to consume them prior to that time.

event channel (n.) a communications channel in which events are transmitted from event source to event receivers, typically received as electronic messages. Each channel can have multiple destination and. events can be configured to transmit to a default destination. JMS is an example of an event channel.

event cloud (n.) a partially ordered set of events (poset), either bounded or unbounded, where the partial orderings are imposed by the causal, timing and other relationships between the events. Typically an event cloud is created by the events produced by one or more distributed systems. An event cloud may contain many event types, event streams and event channels. The difference between a cloud and a stream is that there is no event relationship that totally orders the events in a cloud.

event-driven (n.) the behavior of a human, system or computational entity whose execution or actuation is in response to events, typically received as electronic messages.

event-driven architecture (n.) an architectural style for distributed computing applications in which some of the components are event-driven and communicate by means of events.

event processing (n.) computing that performs operations on events, including modifying, creating and destroying events.

event-object (n.) an software object that represents an event, generally for the purpose of computer processing, that exhibits both encapsulation, inheritance and polymorphism.

event prediction (n.) computational activity where the impact of events, complex events, and situations caused by events identified, including both opportunity or threat. Sometimes referred to as “Level 2” event processing, impact assessment or predictive analytics.

event pre-processing (n.) computational activity where events are cleansed or normalized to produce semantically understandable data. Sometimes referred to as “Level 0” event processing.

event processing (n.) computational activities on events dealing with the association, correlation, and combination of event data and information from single and multiple event sources to achieve refined identity and situation estimates for observed event objects, and to achieve complete and timely assessments of opportunities, threats, and their significance. Event processing is characterized by continuous refinements of event estimates and assessments and by evaluation of the need for additional sources, or modification of the process itself, to achieve improved results.

event processing agent (n.) an EPA is a computational entity that performs event processing.

event processing network (n.) a set of event processing agents and a set of event channels connecting them.

event properties (n.) data representation of an event, typically by name-value pairs of type string, integer, real, boolean or a complex data type.

event refinement (n.) filter, identify and track events & make initial processing decisions based on association, correlation and state estimation. Sometimes referred to as “Level 1” event, or event-object, track and trace.

event stream (n.) a time-ordered sequence of events. An event stream may be bounded by a certain time interval or other contextual dimension (content, space, source, certainty), or be open ended and unbounded.

event stream processing (n.) a time-ordered sequence of events. An event stream may be bounded by a certain time interval or other contextual dimension (content, space, source, certainty), or be open ended and unbounded.

rule (n.) defines what triggers unusual, suspicious, problematic, or advantageous activity within an event processing agent and what the EPA does when it discovers these types of activities. Rules execute actions based on certain conditions on events, instances, or a combination of both. A rule includes a group of condition-rule statements and action-rule statements. The condition statements instruct the EPA what to look for in events, and action statements instruct the EPA how to respond when conditions are met. If all the conditions in a rule are satisfied by events or instances or both, the EPA fires the actions. The action might be to execute tasks, create an event instance, modify property values in an event instance, create and send an event, or something else.

rules engine (n.) a type of event processing agent that uses a declarative programming model to process events. Formally described as “an abstract structure that describes a formal language precisely, i.e., a set of rules that mathematically delineates a (usually infinite) set of finite-length strings over a (usually finite) alphabet“. Informally, it can be any system that uses rules, in any form, that can be applied to data to produce outcomes.

rule language (n.) is an artificial language that is used to control the behavior of an event processing agent. Rules languages, like human languages, have syntactic and semantic rules to define meaning.

situation refinement (n.) identify situations, or complex events, based on event clustering, event-event relationships and relationship analysis and context. Sometimes referred to as “Level 2” event processing.

simple event (n.) an event that is not an abstraction or composition of other events.

virtual event (n.) an event that is imagined, modeled or simulated.

To kick off our celebration of our first five years, we asked ScienceLogic founders Dave Link, Richard Chart and Chris Cordray for their thoughts and memories on events leading to today’s milestone. How and why did they set out on this venture? What happened along the way – expected and unexpected? Why were they successful in times when other new (and established) businesses have come and gone?

How did you three put together this team?

We all worked together at a large Managed Service Provider for a couple of years before leaving to start ScienceLogic, so we all knew each other and knew our collective strengths. More importantly, each of us had worked with network management tools on some level (sales and marketing, engineering and product development), and knew first-hand all of the customer pain points, from every perspective. So we left and began rapidly figuring out how to build a better network management solution based upon our real world operational experience..

Dave: One interesting aspect is that our areas of expertise don’t overlap, which has contributed to our success. Chris is excellent with developing the product front-end and interface, Richard handled the backend architecture and engineering and I focused on the technical business side of sales and marketing. Our roles have been to build a product that works well and that provides real value to operations teams that experience the same day to day frustrations that we felt.

Whose idea was it to start the company?

Dave: It was really a collective effort. We were all passionate about “getting it right” and not just starting a company. We knew the industry need and between us, we had the knowledge and skill sets to address all of the right aspects of developing a product and a building a business around it.

What process did you go through to get started?

Richard: From the beginning we knew the type of solution the market needed and we knew that we wanted to build it as an appliance. From different vantage points, we had each experienced the effects of long, difficult and expensive installations that still exist with traditional network tools. Every install has unique variations: there are always different server types, varying hardware and software versions, different patches installed, and on and on. Every installation was time consuming and unpredictable. We knew that an appliance model would address all of these variables and save a lot of time on how quickly customers could achieve immediate value.

The harder decisions were around actually starting the business, assessing the market and of course determining the product pricing.

EM7 completely flips the traditional model of complex, lengthy and expensive deployments. How did you convince others that the EM7 Meta-Appliance product was valid?

Dave: Yes, EM7 totally disrupts the traditional model for network management. While others take a narrow approach, we intentionally designed EM7 to focus on the broad problem – managing the data center. How do you cover a variety of technologies and make sure they work seamlessly together? The vision was to make it easier, not harder, for customers.

Chris: I have to give it to Dave – very early on, he realized the power of a demo. If Dave could get in front of someone, he’d make them a believer. He’d use the Peter Falk/Columbo technique of “let me show you one more thing.” It was very effective. It’s getting easier, but even today people sometimes have to see EM7 in action before they become believers.

Can you describe the early days of running a new business?

Dave: ScienceLogic is a classic case of entrepreneurship. For the first year we worked out of our basements. We kept the costs low in every conceivable way and spent the first year developing the product before we even made a sale.

Chris: We stayed at lots of odd places when we were on the road, took cheap flights with multiple layovers and purchased lots of our first test equipment on eBay. This was during the dot-com bust so there was lots of equipment for sale on eBay, really cheap!

Richard: The amount of equipment I had in my house was absolutely crazy. Back then, servers were huge – I had a Cisco 6509 Catalyst, a Compaq Proliant DL380, Brocade switch, IBM Netfinity 4500R, and tons of other machines.

Chris: I had to install a new circuit box at home because I was blowing breakers. I remember when that 6509 crashed, we revived it and it died again. The second death was final.

So you started in your houses – what was your first office space?

Dave: My friend, the CEO at Ernst & Young Technology had a few extra cubes and a data center in their office that they graciously allowed us to use. Their help was an important step in helping us really formalize the business. We started doing well and adding people, but ironically, their company was downsizing. Before long, many of their original YET people were gone and the ScienceLogic team kept growing in to the open cubes.

Our first leased space was converted warehouse space in Chantilly, VA that once housed an internet radio station. It was cool – it had a large salt water fish tank, a loft, a spiral staircase and a Star Trek door that retracted into the walls with the customary lights and “whooshing” sound.

We outgrew the Chantilly space, leading to our current office in Reston, VA.

Who was the first ScienceLogic customer?

Our first paying customer was Martins Point Health Care. We deployed there in July 2004 and are pleased to say they continue to be a ScienceLogic customer. Other early (and still) EM7 customers include Navy Knowledge Online and the Department of Transportation. Nearly all of our customers are still actively using EM7 and renewing their maintenance.

Where do you see the company in the next 5, 10 or 15 years?

Well, our revenue has doubled year-over-year in each of the last three years, so of course we’d like to continue to grow like that or even faster. In five years we’ve gone from three founders to the point where Dave does not know everyone’s fondest childhood memory. We’ll continue to scale our growth to cover the demands of our growing customer base.

Where do you see the industry going over the coming years?

Chris: IT is always moving and gaining in complexity, so network management is also becoming more complicated. There’s increasing diversity, new standards, virtualization and cloud computing. All of these are today’s technologies. Customers have a mix of the old and the new, so EM7 has to accommodate and support both.

Richard: Each generation of products has a new set of ways to monitor, but the “old” doesn’t go away. Even when a new, hot technology comes along, the old technologies still need to be supported. We work to ensure EM7 keeps up with both.

Dave: After five years we’re just hitting our stride and we’re just now reaching the tipping point in awareness of ScienceLogic and EM7. We’re all still passionate about the product and as Chris and Rich said, there’s still a lot do. We’ll continue disrupting the market with EM7. Our vision hasn’t changed, and with the increasing levels of automation that customers demand, the market needs are greater than ever. Our future is as bright, or brighter, than ever and we’ll continue to be looking for smart ways to automate traditionally manual IT Operations processes.

What’s your advice for someone interested in starting their own business?

Chris: Be passionate. That’s what has gotten me through the tough times. I didn’t really appreciate this thought when I heard others say it before. But it’s very true.

Richard: I agree. We met and talked with lots of people who told us, “That’s been done before.” But we kept going because we truly believed in what we were doing and we knew that while our approach was different, that it would be successful.

Richard: Be fearless. You can’t be too nervous and you need to be able to expect and handle the stress because it will be there. You have to learn to accept the stressful times as a necessary part of the process of starting out on your own.

Dave: Know your niche from the beginning and give potential customers a compelling reason to trust you and really benefit from your solution. You have to know the problem, see the gap and have a clear and consistent vision of how to solve the problem. Then you have to execute. If you don’t build your team with “doers” you won’t make it.

Chris: It helps to have friends. ScienceLogic was built on friendships and relationships, starting with the three of us. If you look at our team, most of our hires are referrals – people who developed and maintained great connections with other great people throughout their careers. Maintain your connections and keep in touch with your network of friends.