Around the same time that Professor Luckham and his team was working on CEP applications in network management and security management, I was leading efforts to build network and security management control centers for the United States Air Force.  In the beginning, dating back to 1994, my Internet-related work was for Air Combat Command (ACC), working out of ACC headquarters at Langley Air Force Base.

In 1997, I lead a technical team that developed countermeasures against an actual distributed Internet-based attack on the Langley AFB SMTP email infrastructure.  This attack was documented in a technical paper, E-Mail Bombs and Countermeasures: Cyber Attacks on Availability and Brand Integrity, IEEE Network Magazine, Vol. 12, No. 2, pp. 10-17, March/April 1998.  In addition, this attackand countermeasures I designed was featured in Popular Science Magazine in an 1998 article, War.Com and other news channels.  I also published a number of related papers on this topic.

Our team used a rule-based approach for countermeasures against massive email bombs attacks on the Langley Air Force Base email infrastructure.   We called this rule-based system, BombShelter. and it was written in PERL.  I developed both the original software architecture and the original working prototype for BombShelter (in two days) and then we turned the software over to our team who used the rule-based approach for daily attack countermeasures.

I watched for days, and then weeks, as my team designed rules, and the attackers wrote new attacks that circumvented the rules.  Some folks in the Pentagon used to say that I “lead the effort to fight the first war in cyberspace”.   It might have have been the first cyberwar, I am not sure, but it was certainly the first publicly documented cyberwar.  There is no doubt about this.

Without getting into all the historical footnotes and significance of this cyberwar that was fought with experts and rule-based systems, I would like to jump to an important conclusion.

Rule-based systems are useful, but have limited functionality and scaleability in most complex event processing applications.

Rule-based systems are human resource intensive because rule-based systems cannot learn and adapt on their own, humans learn and then write new rules.  This is how rule-based systems work.

This is the motivation behind why I spend a lot of time to search for new, more efficient and adaptive methods as alternatives to rule-based systems.   After extensive research, I published a series of papers on the future of intrusion detection in the Internet.  Intrusion Detection Systems & Multisensor Data Fusion - Creating Cyberspace Situational Awareness [1], helped lead an evolution in Internet security, particularly in the area of network-based intrusion detection systems (IDS).

In my published research work, motivated by limitations with rule-based approaches, I used the same mature functional model that is used to process missile attacks, control global air traffic, and other complex event processing applications in physical space; but I applied these concepts to cyberspace.

Around the same time, Professor Luckham and others were working on similar problems, all related to real-time detection and response to threats in cyberspace.  They were also funded by the US government.

Sidebar: Stream processing of transaction- based systems (databases), another area of interest, was focused on a totally different problem, which was the low latency processing of straight-thru processing in databased-oriented systems.   These stream processing systems were, and remain however,  rule-based systems.  The problems we were trying to solve in cyberspace, however, cannot be efficiently and pragmatically solved by rule-based systems alone.  Only relatively simple scenarios can be efficiently detected by rule-based stream processing systems.

The vast majority of complex event processing classes of problems require rules plus advanced algorithms that can learn and adapt in real-time.    I know this, not from reading papers or taking university classes on rule-bases systems, but from working on some very challenging operational problems in real-time.    This is why I remain interested in complex event processing and why I continue to elaborate on why rule-based systems have limitations.

Note that if I want to opt out of further communications, I have to do two separate things -- which actually becomes three things.

• First I have to click the last link to opt out of future TechTarget spam. (Yes, I deleted the actual links. But certainly none of my trustworthy readers would attempt to re-subscribe me, right...? <g>
• But that isn't enough -- I also have to separately opt out of future Avaya spam! (Why does the no-more-from-Avaya link live on a techtargetmail.com server? Whatever.) Clicking on that link eventually does land me on an avaya.com page, where I have to confirm my email address and indicate they don't have my permission to send me spam. Hmm, too difficult to embed my email in that link, when the other techtargetmail.com link did embed my email?
• Then after submitting it, another page pops up telling me that I'll soon receive an email with additional instructions! In this email there's a link -- to avaya.com with my email address embedded -- that I must click, I guess to double plus confirm that yes, I really really really do wish never to hear from you again. Clicking that link takes me to a page that promises my "permissions have successfully been set. Thank you."

A pox on both your houses, TechTarget and Avaya. I never asked for your stuff. Go away.

Spam, my friends, is only going to get worse. It was so easy to ban junk faxes in 1991. But even those regulations were weakened in 2005. So do you really think we'll see anything even remotely logical for outlawing spam? I doubt it, unless we the citizens foment a revolt. Let's get cracking!

Received: from SVC-EXGWY-E801.partners.extranet.microsoft.com (10.251.24.242)
by tk5-exhub-c102.redmond.corp.microsoft.com (157.54.18.53) with Microsoft
SMTP Server (TLS) id 8.1.291.1; Tue, 16 Sep 2008 11:27:56 -0700
Received: from mail139-wa4-R.bigfish.com (216.32.181.113) by
mail04.microsoft.com (10.253.160.184) with Microsoft SMTP Server (TLS) id
8.1.291.1; Tue, 16 Sep 2008 11:27:55 -0700
Received: from mail139-wa4 (localhost.localdomain [127.0.0.1])    by
mail139-wa4-R.bigfish.com (Postfix) with ESMTP id 018C11184C2    for
<steriley@microsoft.com>; Tue, 16 Sep 2008 18:27:50 +0000 (UTC)
X-BigFish: ps16(zz18c1K1936K2b7wcak69jzzzz2af1jz2fh6bh5eh65h)
X-Spam-TCS-SCL: 4:0
Received: by mail139-wa4 (MessageSwitch) id 1221589667478982_28100; Tue, 16
Sep 2008 18:27:47 +0000 (UCT)
Received: from pp.techtargetmail.com (pp.techtargetmail.com [65.211.80.227])
    by mail139-wa4.bigfish.com (Postfix) with SMTP id 46566978071    for
<steriley@microsoft.com>; Tue, 16 Sep 2008 18:27:47 +0000 (UTC)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=pp.techtargetmail.com; b=iOmibOrM91/1Ugy2gj3QbWo74T2m3GuhmwxZCXJQpFT+nwRES8QKg+4vjt48SNp7WWJExG61Ge+DtnKD3KVI3KwqTKzkPRVrEBF0DCHhYot6VAG/EyEr5vb5RhBz+91yvNhbIqITzGnuQ+uBDJzyc6gU0FHfBl0Fa3S/phcPELM=;
Message-ID: <a818b044.724694.236c8ee748f7dd97.1.n.4.2971370188@pp.techtargetmail.com>
Date: Tue, 16 Sep 2008 14:27:47 -0400
thread-index: a818b044.724694.236c8ee748f7dd97.1.n.4
Reply-To: Avaya <a818b044.724694.236c8ee748f7dd97.1.n.4@pp.techtargetmail.com>
From: Avaya <Avaya@pp.techtargetmail.com>
To: Steve Riley <steriley@microsoft.com>
Subject: 7 Tips to Ensure Readiness for UC Deployment
MIME-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
Content-Class: urn:content-classes:message
Importance: normal
Priority: normal
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.4133
Return-Path: a818b044.724694.236c8ee748f7dd97.1.n.4@pp.techtargetmail.com
X-MS-Exchange-Organization-PRD: pp.techtargetmail.com
Received-SPF: Pass (SVC-EXGWY-E801.partners.extranet.microsoft.com: domain
of Avaya@pp.techtargetmail.com designates 65.211.80.227 as permitted sender)
receiver=SVC-EXGWY-E801.partners.extranet.microsoft.com;
client-ip=65.211.80.227; helo=mail139-wa4-R.bigfish.com;
X-MS-Exchange-Organization-PCL: 2
X-MS-Exchange-Organization-Antispam-Report: DV:3.3.6916.600;SV:3.3.6916.813;SID:SenderIDStatus Pass;OrigIP:65.211.80.227
X-MS-Exchange-Organization-SCL: 2
X-MS-Exchange-Organization-SenderIdResult: PASS

The following message was sent to you as a subscriber to third party offers from a TechTarget property, including our network of Search sites, Bitpipe.com, CIO Decisions Magazine, Information Security Magazine, Storage Magazine, KnowledgeStorm, TheServerSide.com and/or TheServerSide.NET. To unsubscribe, see below.
____________________________________________________________

How should you evaluate the move to unified communications (UC)? Who within which parts of an organization will benefit? Will UC reduce the time to market? Read this E-Guide for answers to these questions and a better look at how the value of UC will, at first, be less of a financial issue and more of a productivity improvement issue that translates into financial benefits. Download this white paper now: http://pp.techtargetmail.com/c.asp?724694&236c8ee748f7dd97&1

When implementing unified communications, there are a number of important issues to think about and questions to ask. This E-Guide analyzes seven phases to ensure you reap the full benefits of UC in each. If you're ready to take the plunge but you're not sure your business or your infrastructure is - download this E-Guide now.

Click here to learn more: http://pp.techtargetmail.com/c.asp?724694&236c8ee748f7dd97&1

"If you do not wish to receive future promotions directly from Avaya please forward this e-mail to {link removed} ; please note that there is a separate opt-out procedure below to be removed from the list from which this email originated."
____________________________________________________________

Please do not reply to this email.  To unsubscribe from all future third party offers from all TechTarget properties, simply click here: {link removed}

TechTarget | 117 Kendrick Street, Suite 800 | Needham, MA 02494

ScienceLogic: How long have you been involved in InteropNet?

Katsev: I started at Coyote Point 3 years ago and InteropNet 2006 was my first “big” assignment.  This was the first time Coyote Point had put in a proposal to participate, so we were very excited when we were selected.

ScienceLogic: How long has Coyote Point been involved in Interop overall?

Katsev: We’ve been exhibiting at Interop for a number of years, and after seeing the InteropNet in action, we decided to submit a proposal in ‘06.  We were actually one of the first companies in the load balancing/traffic management space (we’ve been doing this for almost 10 years), so we have a lot of experience to share with InteropNet.

ScienceLogic: What is your role at Coyote Point?

My official title is “Engineering Project Manager”.  Basically, that means that I’m in charge of product releases and maintenance.  It sounds like a weird title for someone participating in InteropNet, but I’ve actually found it extremely useful since my position means that I don’t get to see our systems out in the field a lot.  We’ve added several features and have ideas for others just from my experiences at InteropNet.

ScienceLogic: What do the Coyote Point products do?

Katsev: Coyote Point makes a Traffic Management appliance called Equalizer.  What this means is that any traffic destined for a datacenter’s servers goes through our appliances and we make sure that the server which is best equipped to handle it, does.  Our systems sit between the clients and the servers and monitor the client traffic and the state of the servers.  If the clients start sending more traffic, we’ll balance it out so that no server is overloaded.  If one of the servers stops responding or starts responding very slowly, we’ll steer traffic away from that server.

ScienceLogic: In what way are your products being used as part of InteropNet?

Katsev: In the InteropNet, we’re utilizing a lot of our expertise:  We’re making sure that traffic is balanced and servers are redundant for show services such as DNS and SMTP.  We’re also using our geographic load balancing technology to ensure that the ScienceLogic EM7 appliances and some other internal NOC services are available from anywhere, with the lowest latency, with our SSL acceleration and GZIP compression technology.  Finally, we’re helping logistics in the NOC by allowing a physical separation between systems located in the NOC and those in an emergency rack outside of the NOC.  If either of these two locations were to fail, the network will continue operating without a glitch.

ScienceLogic: Are there any special considerations for Interop that cause you to deploy your systems there differently that any other place?

Katsev: Interop is definitely different than most of our customer installations.   One difference from a standard environment is that the network (at least this year) is one large flat network, with pieces carved out where extra security is needed.  Because of this, we can actually run our failover pairs of Equalizer systems in a non-standard configuration where the two peers are in different racks, or even on different floors.  That’s one of the things that I really like about InteropNet — it definitely brings new ideas to mind, which end up becoming ’special configuration’ white papers after the show.

ScienceLogic: Has InteropNet taught you anything that caused you to actually change your product?

Katsev: In addition to the failover configuration differences I mentioned above, participating in InteropNet has actually caused us to add several new features and allowed configurations.  One example is the “no-spoof” option for Layer 4 clusters.  Prior to the 2006 shows, we always ’spoofed’ the client’s IP address when talking to a server so that the server would see the client’s IP address instead of our own.  At Interop, we ran into a special configuration which would’ve been very difficult to set up in this manner, so our engineers added this feature, and it’s been very a very popular configuration with our customers ever since.

We have also had a couple of business relationships that extended outside of the show.  In 2006, we had a good experience using Spirent Communications gear to benchmark the network, so we ended up purchasing a couple of these systems to test our products.  More recently, we have found a way to bundle our Equalizer e350si load balancers with the ScienceLogic EM7 collector appliances to help ScienceLogic get the best performance in load balancing large quantities of syslog messages to be processed.  If it wasn’t for our participation in InteropNet, neither of these relationships would’ve happened.

ScienceLogic: What’s the best part of being involved with InteropNet?  What do you most look forward to?

Katsev: InteropNet is an amazing networking opportunity (no pun intended).  The group of engineers that put the network together every year is, well, amazing.  There is so much combined experience that any question instantly has several possible answers, and the best answer is chosen very quickly.  One of the ’sayings’ at Interop is “if you run into a problem, ask someone… we’ve probably seen that problem before… five times.”  One would think that being part of InteropNet is the same thing, year after year.  However, in the two years that I’ve been part of this (for four shows), there have been huge differences in the way that the network is designed and put together.  These are both because the vendors selected every year are different, and because the engineers who design the network change from year to year.  Somehow, though, when all is said and done, we have a network that works.

ScienceLogic: You don’t have to answer this one if you’re not comfortable… What would you like to see changed with the way things are done at InteropNet?

Katsev: This isn’t a cop-out… I really can’t think of anything I would do differently.  Sure, there are small problems that pop up sometimes, but every project has those, and the people at InteropNet are more than capable of figuring them all out.  In fact, I know that Interop started out as a show to test the interoperability of devices… but I’m still amazed that all of these devices actually talk to each other and “play nice” together.

ShareThis

The trick to doing this is not to hardcode things like host, port, etc. for your SMTP server when you use System.Net.Mail to send mail. Instead, use the default ctor for SmtpClient as I've done in the code below.

static void Main(string[] args)
{
    // note the use of the MailAddress class
    // this allows me to specify display names as well as email addresses
    MailAddress from = new MailAddress("admin@fabrikam.com", "Fabrikam Website");
    MailAddress to = new MailAddress("mari@fabrikam.com", "Mari Joyce");

    MailMessage msg = new MailMessage(from, to);
    msg.Subject  = "Testing 123";
    msg.Body = "This is only a test!";

    // note use of default ctor
    // this looks in config to figure out how to send mail
    new SmtpClient().Send(msg);
}

What you're telling .NET by using the default ctor for SmtpClient is, "please use my config file to figure out how to send mail". Now you can use the system.net/mailSettings/smtp section in config to specify the details of your mail server, and all of the code in your app that is written to use the default SmtpClient ctor will inherit these settings. Here's an example of what the config on a production server might look like (if you put passwords in your config files, be sure to encrypt those sections):

<configuration>
  <system.net>
    <mailSettings>
      <smtp deliveryMethod="Network">
        <network host="mail.fabrikam.com"
                 port="25"
                 userName="WebsiteMailAccount"
                 password="whatever"/>
      </smtp>
    </mailSettings>
  </system.net>
</configuration>

During development, I use different settings because I don't usually want to deal with the hassle of installing an SMTP server on my development box. Instead, I want email messages delivered as individual files in a directory on my hard drive (I always have a c:\mail directory on my development box for just this purpose):

<configuration>
  <system.net>
    <mailSettings>
      <smtp deliveryMethod="SpecifiedPickupDirectory">
        <specifiedPickupDirectory pickupDirectoryLocation="c:\mail"/>
      </smtp>
    </mailSettings>
  </system.net>
</configuration>

Now when I run the program above, I get a .EML file in my c:\mail directory:

Outlook Express is normally registered as the viewer for .EML files, so double-click the file to view it:

If you've never seen this method of simulating email before, I hope you find it as useful as I have. Happy coding!

Most event processing systems would be incomplete without the ability to process events in the form of messages.   Messages can be delivered in either a connection-oriented protocol or a connectionless protocol.   Most enterprise-class messaging systems have both.   Many messaging systems have features like guarenteed delivery, which are important to many applications.

On the other hand, you do not have to work with a messaging system or enterprise service bus (ESB) to process events, because the transport layer is independent from the event processing layer, theoretically.  Most enterprise-class event processing system architectures will use a combination of both asynchronous and synchronous messaging. 

To understand event processing I recommend you turn to network management and the practical use of Simple Network Management Protocol (SMNP) for a basic undertanding of event processing.   SNMP uses both synchronous event-based messaging, called polling, and asynchronous messaging, called traps.   Network management systems engineers use a combination of both polling and trapping in all enterprise-class operational NMS.  Optimizing polling and trapping is one of the tasks good NMS engineers do well. The same holds true in most distributed event processing architectures.  

For example, look at the CEP/EP reference architecture on this site.  You will notice that the mechanism for event transport is generic, represented as an event bus, but it does not specify the transport protocol.  If you are receiving raw events and comparing correlated results against a signature in a database, you are using both asynchronous and synchronous messaging.    In theory, you could build an event processing system with only connection-oriented protocols, but this would be an exeception, not the rule.

Event processing is generally associated with messaging because we generally represent event-objects as electronic messages.   In theory, we could call these cyber event-objects anything we want; for example, we could call them “packets.” However, packets are generally associated with the underlying Internet Protocol (IP) layer by network engineers.  

Moving up the stack, we think in terms of a complete message-object, which we generally call “a message.”  This message could be an SNMP event-object, an SMTP event-object (an email message), or an HTML request to a web server, to only name a few.    In fact, the basic unit of work at the application level of a distributed network application is what we call “a message.”  

So, in On Messaging and Events Opher asks, “Is event processing just fancy name to message processing ?”

Events are generally represented in some electronic format.  The event-object must be transported electronically in cyberspace, and the way that it is transported is in what network engineers generally call “a message.”   It make no difference what we call it, really; because whatever we call it, it is still binary data representing information we are interested in, hopefully in a format we can efficiently process.    Enterprise-class event processing systems are designed to work with myriad formats, protocols and transports.   One size does not fit all.