Now you know why I have not been travelling the world, speaking in conferences this year. Event processing is great fun, but there is more to life than processing complex events!

“Experienced end users are very intelligent.

These end users know the complex event processing problems they need to solve; and they know the limitations of the current COTS approaches marketed by the CEP community.  Even in Thailand, a country many of you might mistakenly think is not very advanced technologically, there are experts in telecommunications (who run large networks) who are working on very difficult fraud detection applications, and they use neural networks and say the results are very good.   However, there is not one CEP vendor, that I know of, who offers true CEP capability in the form of neural nets.

Almost every major bank, telco, etc. has the same opinion, and the same problem. They need much more capability than streaming joins, selects and rules to solve their complex event processing problems that Dr. Luckham outlined in his book.   The software vendors are attempting to define the CEP market to match their capability; unfortunately, their capabilities do not meet the requirements of the vast majority of end users who have CEP problems to solve.

If the current CEP platforms were truely solving complex event processing problems, annual sales would be orders of magnitudes higher.  Hence, the users have already voted.   The problem is that the CEP community is not listening.”

Not to be overly repetitive,  but the last part of this quote from a year ago is worth highlighting:

“If the current CEP platforms were truely solving complex event processing problems, annual sales would be orders of magnitudes higher.  Hence, the users have already voted.   The problem is that the CEP community is not listening.”

Frankly speaking, nothing in the “CEP world” has changed, technologically speaking, since this September 2007 post was written.  From a sales perspective, we have seen less CEP-related sales in 2008 than in prior years.   If these so called CEP products were actually capability of detecting “real” complex network-centric situations (threats) in real-time, they would be selling faster than a cup of ice water in the blazing hot Sahara desert.

Don’t shoot the messenger.  Build better detection engines!

On the other hand, maybe complex detection is too hard for most of these companies and that is why they focus on routing, mediation and relatively simple rule-based scenarios, versus complex event processing?

Today, however, I discovered a huge security breach in Google Docs. While I was in my account working on a spreadsheet I suddenly found my Google Doc account listing many documents that did not belong to me. I clicked on one of the documents and the results are in the image below, where my Google Doc session appears to have “crossed over” with another users.

I decided to do a bit more exploring and take a few more screenshots, because I don’t yet know how to reproduct this security breach. The image below show a Google document (fifth from the top) which is not owned by me, “owned by me”. However, when I click on this mysterious “owned by me” document, it is owned by another user. Here is another screenshot below; you can click on the image for the full-screen version.

Again, here is another example of the same security violation with two documents. As above, you can click on the image for a full-screen version.

I contacted the owner of the Google Docs account which I had suddenly and mysteriously “crossed sessions” with today. I asked him if he was in Thailand (since a few of the documents were in Thai) and he said yes, however he say he did not have any Thai language documents in his account. However, as you can see from the screenshot, the Google Docs menu shows this person as “the owner” of a Thai language document. He also mentioned that, today, he saw “wierd documents” in his account that did not belong to him (or “normally” shared with him).

Unfortunately, I was having problems with the Internet connection in my hotel room so I could not continue to investigate the breach. When I logged back in a few hours later, everything was back to normal. So far, all is “normal” and I have not been able to repeat this breach.

I suspect the Google Docs flaw comes from a JavaScript error in how Google manages user sessions. The bottom line is that the security breach is real and dangerous. Your Google Docs, and I suspect other Google applications that use the same session management code, are vulnerable. There may be an underlying XSS vulnerability as well.

Note: Reposted from my original post on the ISC2 blog.

Marc’s initial shoplifting model in his post is based on John Colapinto’s concepts of matching a pattern of customer movements in the store with their estimated patterns of shoplifting behavioral patterns.    Marc’s asks how Coral8 might address this.   We are not ready to seek a vendor solution.  We do not yet have a workable detection model.

As indicated above, I don’t think the example situation cited by John and Marc is a viable model for automated processing.    Tracking the behavior of customer’s movements, by machine, would require some very sophisticated image processing technology that would be too expensive compared to any possible loss at most retails stores.    This type of behavioral pattern recognition. in retail stores, is performed by people (security personnel), not machines, observing people.  

To develop a machine pattern recognition application to detect retail shoplifting we need to build detection models that are economically feasible.  If we are going to use a model of shoplifting pattern recognition versus anomaly detection, we need to define the objects we must track.  

In the most simple model, we have merchandise-objects.   Stores normally (physically) track merchandise-objects only at the exit/entry points of the store using some electromagnetic proximity detection technology.   In this model, the detection configuration is a combination of simple alerting with humans watching the store (”minding the store”).    This is not complex event processing.

However, if we added another object to our model, the customer-object, then we start to get more “complex,” but we have not defined “complexity” yet because we have not defined the object properties, the possible states of the objects, and the relationships between the objects that are the basis for estimated situations.

Hence, model building is constrained by available resources, simple economics and risk (cost-benefit).  If we are detecting shoplifting in Walmart the cost-benefit model for implementing an automated shoplifting detection system would be different than at a top diamond store on 5th Avenue in NYC.   Protecting loss at a weapons-grade uranium respository follows a different model than protecting loss at a handicraft shop, naturally.

Like Marc, I find models to automatically detect shoplifting interesting, so permit me to close with a general discussion of shoplifting in the context of our CEP/EP reference model.

One approach would be do determine what objects will be represented in our model.   For example, if we are going to track merchandise, we need to model the ”merchandise-object”.  If we are going to track people, we need to define the properties of this “person object.”  If we are going to represent the store layout, we need to define all these objects (store-object, table-object, shelf-object, entry-object and so forth).  The model can get “complex” quite quickly. 

Editorial Note:  An object-oriented approach greatly assists complex model building because we can benefit from OO properties such as encapsulation and polymorphism.  For example, we can define a basic “person object class” and then create superclasses of this object for “customer-object”, “manager-object”, “or criminal-object.”

Generally speaking, each object we define will require a state-model, for example, in Marc’s example of a customer moving around the store, we would need to model the possible states (customer at the entrance, at table 1, at table 2, at shelf 1, in the bathroom, at the cashier, etc.)  Indeed Marc, this is complex event processing if we have modelled multiple objects and defined object-object relationships that indicate situations of interest.   For example, customer-object at table2 where merchandise-object has the property of  ”very expensive, high risk” and then customer-object changes state to “in bathroom”.  Of course, we need more key indicators, but you get the idea.

Right now, I am typing from the Taste from Heaven Vegetarian Restaurant in Chiang Mai and my battery is running low.  The owner of this excellent restaurant also runs the Elephant Nature Park, a non-profit organization advocating and acting on behalf of the rights of the mighty elephants in Thailand.  Would be great if we could also automatically detect the situation of “elephant abuse” by poachers and other crimes against nature.   Time to get back to my delicious mushroom salad, Northeastern Thai style.

As always, thanks for reading, time for me to get back to eating!

First of all, as background information, I learned the Thai alphabet (script with 44 consonants and 32 vowels) nearly 20 years ago, so I have have a pretty decent foundation for the Thai language compared to most foreigners visting or working in Thailand.   I can read (slowly) and speak better than 99.99+ percent of all foreigners in Thailand.  For this reason, I thought it was ”the right thing to do” to redirect my career to a “new challenge” in the business climate of Thailand as I continue to improve my foreign language skills.   I wanted to help Thailand progress in IT and IT security, so where else would I go but where I have second language skills?

This was no small decision as you can imagine.  Your career and life changes quite dramatically when you give up a long established consulting practice in the US and dive into business in a foreign land, seeking a new challenge.  I can frankly tell you thatit is more difficult to do business in Thailand (as a foreigner) than I expected, for a number of reasons.  Here is my first off-topic post on this topic.

First of all, it is not legal for foreigners to directly own land in Thailand.  Foreigners can ”own” land using a variety of legal loopholes, proxy owners and shell companies; but all of this is risky and not advised.  Many foreigners lose a lot of money coming to Thailand and attempting to buy land via various “structures”.  Some get lucky, but the entire process of foreigners buying and selling land is quite risky and not recommended.

Foreigners can legally own condominiums, under certain conditions, but this “foreign market” results in inflated prices for condos in Thailand that are traded in an “artificial market place” designed for foreigners.   Condos in Bangkok and major resort areas that are up-to-par with condos in the US can easily cost more than condos in major cities in the US.  Hence, the cost of living in Thailand is not as economical as some might believe when you visit Thailand as a tourist.

Second, business in Thailand can best be described as protectionism with discrimination where the government has placed many barriers to entry to foreigners working and competing in Thailand.     Every foreigner must have a work permit and these work permits are expensive and time consuming to maintain.   If you own a business you must pay high professional service fees for “auditors” to perform annual and semiannual audits regardless of how much income you have (including zero).   Firms in Thailand charge thousands of dollars for these ”audits”.      

Third, if you operate a business in Thailand, you must have a place of business (you cannot legally work from your condo you bought at high prices!), so you are forced, by law, to lease office space.   Foreigners from the US, for example, must be paid a minimum of 50,000 Thai Baht per month, so the government will take 10 percent of that each month as their share of tax withholdings.  Startups with no income simply pay income taxes against their personal savings to comply with the law.  Therefore, to start a company and maintain the business in Thailand, you are required to pay significant startup, monthly, semi-annual and annual fees, permits, tax, leases, visas, etc. 

Forth, generating incoming revenue in Thailand can be quite difficult in a climate of both protectionism and discrimination.   In Thailand, it is easy when you are spending money.  This is the ”Land of Smiles” that tourists see and experience.   However, when you are legally permitted to work in Thailand and trying to generate in-country income, you cannot help but notice the protectionism and discrimination against foreigners working and living here.  Many foreigners working in Thailand just “give up” because the barriers to business success are quite high.

Fifth, on top of the challenges of protectionism/discrimination regarding foreigners and foreign investments, which I have only just scratched the surface here, is the overall global business slowdown combined with a climate of political instability which I am sure you have seen in the news.  Thailand has seen 18 coups since 1932.   Currently, Thailand is under a State-of-Emergency  which negatively impacts business even more.  Sound challenging? 

Most people who live and work in Thailand have the opinion that it is far better to enjoy being a tourist here. Working in Thailand is very difficult for many reasons.   Being a tourist in Thailand is completely different than working here.  When you are a tourist, foreign currently flows from you into Thailand, so life in Thailand as a tourist is fun and friendly, hence the “Land of Smiles” you have heard about or experienced.     However, when you are working in Thailand and trying to generate income from Thailand versus bringing in foreign currency, you don’t see the “Land of Smiles” quite the same anymore.

Without getting into too many details in this post, I can simply say that a foreigner doing business in Thailand experiences both protectionism and discrimination.  I came to Thailand hoping to contribute my experience to help the Kingdom.  However, sometimes it feels like foreigners are only welcome if you are working for free, giving seminars for free, and bringing in lots of foreign currency here.

In a future post on business in Thailand I will dive into some details on a number of topics that might be of interest to readers who will never have a chance to come and work here.   

First of all, I learned the Thai alphabet nearly 20 years ago, so I have have a pretty good foundation for the Thai language.   I can read (slowly) and speak better than 99.99+ percent of all foreigners in Thailand; so, I thought it was time to redirect my career to a “new challenge” in the business climate of Thailand.   

This was no small decision.  Your career changes dramatically when you give up a successful consulting practice in the US and dive into business in a foreign land for a new challenge.  I can frankly tell you that often the challenge is sometimes overwhelming.    It is quite difficult as a foreigner to do business in Thailand.

First of all, it is not legal for foreigners to own land in Thailand.  Foreigners can ”own” land using a variety of legal loopholes, proxy owners and shell companies; but all of this is risky and not advised.  Foreigners lose a lot of money coming to Thailand and attempting to buy land.  Some get lucky, but the entire process of foreigners buying and selling land is quite risky.

Foreigners can own condos, under certain conditions, but this results in  inflated prices for condos in Thailand that are traded in an artificial market place.   Condos that are up-to-par with condos in the US can easily cost more than condos in major cities in the US.  Hence, the cost of living is not as cheap as some might believe.

Business can best be described as “protectism” where the government has placed many barriers to entry to foreigners working in Thailand.     Every foreigner must have a work permit and these work permits are expensive and time consuming to maintain.   If you own a business you must pay high professional service fees for auditors to perform annual and semiannual audits even if your business has no income yet.   Firms in Thailand charge thousands of dollars for these ”audits”.      

In addition, if you operate a business, you must have a place of business, so you are forced to lease office space.   Foreigners from the US must be paid a minimum of 50,000 Thai Baht per month, so the government will take 10 percent of that each month as their share of tax withholdings.   Therefore, to start a company, you will pay a lot of money in startup fees, permits, tax, leases, visas, etc.  The entire system is designed to secure money from you, even if you do not have a penny of incoming revenue.

Of course, generating incoming revenue can be quite difficult in a climate of protectionism.   In Thailand, it is easy when you are spending money.  When you are trying to generate income from Thailand, as a foreigner the challenge can seem overwhelming at times.   Many foreigners here give up because the barriers to business here are very high.

On top of all these challenges, which I have not described in detail, is the overall global business slowdown combined with a climate of political instability, which I am sure you have seen in the news.  

Most people I know say it is better to be a tourist here.   Being a tourist is completely different.  Money flows from you, so life in Thailand is fun and friendly, complimentary to the “Land of Smiles” you have heard about.     However, when you are working to have money flow the other direction, flow to you versus away from you, you don’t see the “Land of Smiles” as tourists experience.

Without getting into too many details, I can simply say that a foreigner doing business in Thailand experiences protectionism and, to a certain degree, discrimination, and sometimes I wonder if coming here for a “business challenge” was a good idea.    I was seeking a “new challenge” and I got more than I bargained for!

In a future post on business in Thailand I will discuss issues regarding how little value is placed in intellectual property in Thailand and how this adversely impacts professional services.    I will also touch on how this lack of regard for intellectual property impacts a consulting practice.   Also, I will touch on some cultural differences in how Thais appear to view teamwork, which is very different than in the US.

As we get closer to the conference dates, I wish that I had made plans to fly back to the US to meet everyone.    However, I have been cutting back on public speaking, taking a break since May.  In addition, Gartner did not ask me to speak at their Event Processing Summit this year, I assume because they did not want to pay airfare for my flight from Thailand to the US.    Also, Gartner always likes to fill their conference speaking slots with as many Gartner speakers as they can, unless you are a paid sponsor; and I noticed a number of Gartner employees speaking in multiple slots.

(Editorial Note) Then again, maybe I complained to much about the lack of organization and conference problems when I was invited at be a Gartner keynote speaker last time - reservations not made propertly,  problems with the guest speaker registration list at sign-in, rooms shifted without notifying the speakers and panelists.   Admittedly, I was not happy with the conference organizers at the last get together.  This was my fault, as I am accustomed to better conference execution and am probally too “picky” about details these days - my bad.  Anyway, the Gartner organizers apologized numerous times, saying they had too many conferences going on at the same time and not enough people to cover them all.

One of the problems with spending so much time in Asia, especially in Thailand, is that guest speakers are really treated as VIPs.  There are usually special comfy couches set up for the speakers and the conference staff really treat you very nice, taking care of you every step of the way.   In fact, there is an entire very nice culture around how guest speakers are treated in Thailand.   Often, they pin flowers on the VIP speakers and take your photos like you are a star.    Very nice culture.

I absolutely look forward to speaking on event processing or CEP at a future venue and meeting everyone face-to-face instead of over the net.  My sincere and deepest apologies for not attending either the Gartner or the EPTS event this year.   

PS:  If you take up a collection and send me a RT business class air ticket, I might change my mind