We've probably all heard that you can't secure what you can't see and that statement is even more profound when it comes to virtual environments.  This is because it is extremely challenging to see what is going on at a micro vs. macro level within a virtual environments network.  The virtualization vendors such as VMWare and Citrix have provided embedded tools into their management consoles that show a macro level of visibility but its not enough to identify security events in the environment.  Take a look at the attached picture.  It simply shows VMWare's ability to monitor virtual network performance statistics from a bits per second perspective.

<-Click To Enlarge

With only this level of detail how can one determine which network applications are causing spikes.  Is it FTP traffic that is occuring at a high volume at an unuseal time of day?  If that were occuring, could that be indicative of either a breach or some sort of problem? What if FTP isn't even an authorized service in the virtual environment but there is a high volume of it?  Did someone install a rouge FTP service so they could steal information from the server at will?

These types of questions can't really be answered without a micro level of detail into the packets flowing in, out and within the virtual environment.  Now, what I am highlighting is not security in the traditional sense of prevention but using visibility as a means to first identify, then pin point the source of an issue so that it can properly be mitigated.  Having constant visibility can also ensure that other security products in the environment are performing as expected.  What if a Montego HyperSwitch with firewalling enabled is configured with many policies but someone forgot to create an FTP block policy.  One could think they are protected from rouge FTP services transmiting data out of the network, but without constant visibility monitoring, can you be certain?

Some vendors, namely Reflex Security will get you to believe that their IPS / IDS solution that is inline and running in the virtual environment is the right and only approach.  Or they will tell you to hang a virtual IDS off a span port in the virtual environment and you will at least have visibility into the attacks that are taking place.  Well, sure... You now have attack visibility but at the performance cost of your virtual environment.  Signature matching technologies are great, I'm a huge believer; however they don't scale very well in shared computing environments such as virtual ones.  IDS systems also don't typically track protocol and network service (FTP, HTTP, etc.) utilizations; which is another important part of visibility.

So, what do we do to gain visibility without the performance headache?  Well, for starters its probably best to put your IDS/IPS solutions in the physical environment where performance will be less of a concern.  In fact, you can span a virtual switch's traffic out to a physical NIC as easy as you can to a virtual one.  So why do it virtual and have to pay a 60% CPU utilization tax?  Another solution is to IDS inspect only the things you care about.  Why IDS inspect SSL traffic if you know your solution can't unencrypt SSL.  Its just a waste of compute cycles isnt it?  Policy based switching helps you with directing only the things you care about to an IDS (attack visualization product).  Montego's HyperSwitch also can help you with the traffic redirection of only the things you care about.

Another method of visibility which I tend to be a fan of is one of packet analysis (aka NetFlow).  NetFlow was invented by Cisco some time ago and has gained popularity in the physical world and definately has a use in the virtual world.  NetFlow is lightweight.  Let me say that again, its light weight!  It only sends a summation of packet detail to an analytical engine which can do some number crunching, packet comparison, etc. etc. to make some sense out of whats going on.  Lancope, an Atlanta based visibility company that provides Network Visibility, Security Visibility and User Visibility has this tool on their website that is a Netflow Bandwidth calculator.  You'll see from playing with this ( http://www.lancope.com/netflowcalculator.aspx ) calculator that it doesn't consume a lot of network bandwidth to transmit these network accounting records.  It also doesn't cause a lot of CPU overhead to send these records to an analytical engine sitting somewhere in the network.

Lancope's analytical engines have the ability to do the following for you within your virtual environment:

...and probably a slew of other things I'm not aware of.  A screen shot of their product is bellow:

<- Click to enlarge

You'll notice from the screenshot that you are able to visualize who is talking to who, how much traffic they have sent and received and something called a concern index (not seen on this screenshot).

Now, a concern index is a number that increases as Lancopes analytical engines monitor suspicious activity on a session.  A high counter can be indicative of a security problem.  Its another way of identifying (visualizing) compromised hosts (virtual machines) without having to do signature matching like a heavy weight IPS engine.  Example:  Lets say you have a VM that has a BOT on it and is "owned".  The Lancope product is monitoring this long life session.  Let's say that session is established for several hours or maybe even days or months.  Lets also say that the conversation appears to be mostly unidirectional from a public ip address not belonging to your enterprise.  Lancope would increase a the concern index on this since this server hasn't typically had this type of behavior.  Once the concern index reached a certain level it could then fire off an email, send you a text message or something saying:  Warning, Warning, Danger, Danger Will Robinson!!! You're virtual server may be infected with a BOT, please investigate immediately!!!

This example is VISIBILITY which helps you with SECURITY.  There are a number of other things you can do with NetFlow and Lancope products that have less to do with security and more to do with operational efficiencies.  Things like, helping you answer questions of:  How do I know what network applications are taking up the most bandwidth?  When should I move those applications over to a server with more horsepower?  When did these VM's vmotion over here and was there a traffic condition / CPU condition that caused that to occur?  I could go on and on but thats a topic for another blog entry.

So, my suggestion is to take a look at what NetFlow has to offer.  Montego Networks supports NetFlow transmission and Lancope supports NetFlow analytics and with both you can regain what was lost visibility.

I hope this was helpful to you all!

-John Peterson

We've probably all heard that you can't secure what you can't see and that statement is even more profound when it comes to virtual environments.  This is because it is extremely challenging to see what is going on at a micro vs. macro level within a virtual environments network.  The virtualization vendors such as VMWare and Citrix have provided embedded tools into their management consoles that show a macro level of visibility but its not enough to identify security events in the environment.  Take a look at the attached picture.  It simply shows VMWare's ability to monitor virtual network performance statistics from a bits per second perspective.

<-Click To Enlarge

With only this level of detail how can one determine which network applications are causing spikes.  Is it FTP traffic that is occuring at a high volume at an unuseal time of day?  If that were occuring, could that be indicative of either a breach or some sort of problem? What if FTP isn't even an authorized service in the virtual environment but there is a high volume of it?  Did someone install a rouge FTP service so they could steal information from the server at will?

These types of questions can't really be answered without a micro level of detail into the packets flowing in, out and within the virtual environment.  Now, what I am highlighting is not security in the traditional sense of prevention but using visibility as a means to first identify, then pin point the source of an issue so that it can properly be mitigated.  Having constant visibility can also ensure that other security products in the environment are performing as expected.  What if a Montego HyperSwitch with firewalling enabled is configured with many policies but someone forgot to create an FTP block policy.  One could think they are protected from rouge FTP services transmiting data out of the network, but without constant visibility monitoring, can you be certain?

Some vendors, namely Reflex Security will get you to believe that their IPS / IDS solution that is inline and running in the virtual environment is the right and only approach.  Or they will tell you to hang a virtual IDS off a span port in the virtual environment and you will at least have visibility into the attacks that are taking place.  Well, sure... You now have attack visibility but at the performance cost of your virtual environment.  Signature matching technologies are great, I'm a huge believer; however they don't scale very well in shared computing environments such as virtual ones.  IDS systems also don't typically track protocol and network service (FTP, HTTP, etc.) utilizations; which is another important part of visibility.

So, what do we do to gain visibility without the performance headache?  Well, for starters its probably best to put your IDS/IPS solutions in the physical environment where performance will be less of a concern.  In fact, you can span a virtual switch's traffic out to a physical NIC as easy as you can to a virtual one.  So why do it virtual and have to pay a 60% CPU utilization tax?  Another solution is to IDS inspect only the things you care about.  Why IDS inspect SSL traffic if you know your solution can't unencrypt SSL.  Its just a waste of compute cycles isnt it?  Policy based switching helps you with directing only the things you care about to an IDS (attack visualization product).  Montego's HyperSwitch also can help you with the traffic redirection of only the things you care about.

Another method of visibility which I tend to be a fan of is one of packet analysis (aka NetFlow).  NetFlow was invented by Cisco some time ago and has gained popularity in the physical world and definately has a use in the virtual world.  NetFlow is lightweight.  Let me say that again, its light weight!  It only sends a summation of packet detail to an analytical engine which can do some number crunching, packet comparison, etc. etc. to make some sense out of whats going on.  Lancope, an Atlanta based visibility company that provides Network Visibility, Security Visibility and User Visibility has this tool on their website that is a Netflow Bandwidth calculator.  You'll see from playing with this ( http://www.lancope.com/netflowcalculator.aspx ) calculator that it doesn't consume a lot of network bandwidth to transmit these network accounting records.  It also doesn't cause a lot of CPU overhead to send these records to an analytical engine sitting somewhere in the network.

Lancope's analytical engines have the ability to do the following for you within your virtual environment:

...and probably a slew of other things I'm not aware of.  A screen shot of their product is bellow:

<- Click to enlarge

You'll notice from the screenshot that you are able to visualize who is talking to who, how much traffic they have sent and received and something called a concern index (not seen on this screenshot).

Now, a concern index is a number that increases as Lancopes analytical engines monitor suspicious activity on a session.  A high counter can be indicative of a security problem.  Its another way of identifying (visualizing) compromised hosts (virtual machines) without having to do signature matching like a heavy weight IPS engine.  Example:  Lets say you have a VM that has a BOT on it and is "owned".  The Lancope product is monitoring this long life session.  Let's say that session is established for several hours or maybe even days or months.  Lets also say that the conversation appears to be mostly unidirectional from a public ip address not belonging to your enterprise.  Lancope would increase a the concern index on this since this server hasn't typically had this type of behavior.  Once the concern index reached a certain level it could then fire off an email, send you a text message or something saying:  Warning, Warning, Danger, Danger Will Robinson!!! You're virtual server may be infected with a BOT, please investigate immediately!!!

This example is VISIBILITY which helps you with SECURITY.  There are a number of other things you can do with NetFlow and Lancope products that have less to do with security and more to do with operational efficiencies.  Things like, helping you answer questions of:  How do I know what network applications are taking up the most bandwidth?  When should I move those applications over to a server with more horsepower?  When did these VM's vmotion over here and was there a traffic condition / CPU condition that caused that to occur?  I could go on and on but thats a topic for another blog entry.

So, my suggestion is to take a look at what NetFlow has to offer.  Montego Networks supports NetFlow transmission and Lancope supports NetFlow analytics and with both you can regain what was lost visibility.

I hope this was helpful to you all!

-John Peterson

Is there security needed between virtual machines?  Some say no, some say yes.  I've been out talking to a number of virtualization users and non users on this topic and I'm finding that some say no and some say yes.  The users of virtualization technology tend to say yes while others looking at virtualization from the outside tend to say no.  Why is this?

Well, I thought I'd blog on my thoughts on this!

You see, in the physical datacenter there is no firewalling between servers plugged into the same switch and because of this some people think, well if its not done in the physical world why should it be done in the virtual world.  I believe that its not done in the physical world today because there are no solutions today that embed security into datacenter switches.  Should it be done in the physical world?  I think so!  It never hurts to get security as close as possible to the things you are trying to protect and what better place than the switch port in which the critical asset are connected to.  This is why people have HOST BASED FW/IPS ON SERVERS!  To get security as close as possible!  Is that needed? 

So my first response to those that say, security between virtual machines is not needed because its not done in the physical world is:  Well, just because people have done things one way for many years doesn't mean there isn't a better way.

Would environments be more secure if there was security between servers?  I tend to think so.  You see, many of the attacks that are taking place these days are not attacks for fame but attacks for fortune and gone are the days where people just hacked to spread nasty viruses.  Its all about the data these days (ie. credit cards, social security numbers, etc).  We've all heard about the TJ Max security breach where customer data was compromised and many others like banks that have had credit cards compromised. 

How and the heck do you think most of these things happened?  Attackers are targeting the datacenter these days.  Physical or Virtual.  Their gateway into these environments are the Web Front End Servers.  Let me say that again.  The Web Front End Servers!  Hackers get to the data from the web front end server that talks to the database backend server.  This useually occurs by something called "Cross-Site Scripting" or "SQL Injection" breaches. 

Here is a trival way of how this happens:

A hacker finds a vulnerable web site.  He sometimes does this by something called Google Hacking.  He uses Google to search for sites that has vulnerabilities on it.  Say a web site has some content on one of the pages that says "Powered by Drupal 4.1".  If a hacker knows that Drupal 4.1 software has a vulnerability in it, he can now target all the search results related to this.  Click Here for more detail.

Now lets say Drupal 4.1 on a web site has a SQL-Injection vulnerability because the developer of the Drupal software didn't do Form Field Validation properly.  A Form field is something you fill out on a web page like a form that asks for the user name and password.  User names and passwords to log into the web site are stored on whats called a Database Server.  Hmmm... So this means the web server needs to talk to the database server right?  Yes!  Keep this in the back of our head for now.  The hacker enters in "Admin" for the user ID and "password doesn't matter 'or 1=1--" for the password.  And presto!  He is logged in to the server as Admin.

The reason he was able to log in is because the web site sends a SQL Database command to the Database server and because the developer of the Drupal software didn't do "Form Field Validation" properly (method of checking for invalid characters like the ' (single quote)  symbol), the user was able to bypass the password.  Notice the 'OR 1=1 command appended to the password.  One does equal one so therefore it will return a TRUE result to the password checker and the OR says use the password typed in (password doesnt matter) OR check to see if one is equal to one.  If its true then the password is valid for this user which is Admin.

Now that the user is on the web server, he probably has the ability to connect to the database server or other servers in the network.  Why?  Because there is connectivity from the web front end to all of the backend servers.  He essently can backdoor his way throughout the network.

Another method is for him to append some SQL statement to another SQL statement.  Lets say their is a FORM FIELD on the website that collects some information from the database to display it to web site users.  It could be entering in the Zip code to find store locations in your area.  Instead of putting in the zip code you could put in "95123 'UNION SELECT * FROM credit_card_table--".  The hacker is injecting via the UNION command (which means join one SQL statement with another one) a command that says grab all (via the asterisk) information out the credit card table.

Lastly, the hacker can use the UNION command to write text of his desire to a text file on the database server.  He may write some nasty code, tell the database to write the code to a file and then tell the server to execute that file.  The code could be used to do a denial of service attack to the other virtual machines or whatever.  The possibilities are endless!!

Anyway, these are high level examples.  I think you get the point.

The Web Front End Virtual Machine has a need to talk to the Web Back End Virtual Machine and security such as Firewalling, Intrusion Prevention definately needs to be in place to have a higher level of security.

Another reason to have security between virtual machines is because servers are now mobile in the virtual world.  They move between trust domains to take advantage of computing resources that may be available on a given piece of hardware.  Lets say one PHYSICAL server was hosting database VM's and another PHYSICAL server was hosting file server VM's.  The file server VM could VMOTION to the same environment as the database VM's.   Now where is your isolation between trust domains or unlike resources?

People should think about this problem in greater detail.  I'd love to hear everyones comments as to whether or not they think security between VM's is needed.

John Peterson
Montego Networks

Is there security needed between virtual machines?  Some say no, some say yes.  I've been out talking to a number of virtualization users and non users on this topic and I'm finding that some say no and some say yes.  The users of virtualization technology tend to say yes while others looking at virtualization from the outside tend to say no.  Why is this?

Well, I thought I'd blog on my thoughts on this!

You see, in the physical datacenter there is no firewalling between servers plugged into the same switch and because of this some people think, well if its not done in the physical world why should it be done in the virtual world.  I believe that its not done in the physical world today because there are no solutions today that embed security into datacenter switches.  Should it be done in the physical world?  I think so!  It never hurts to get security as close as possible to the things you are trying to protect and what better place than the switch port in which the critical asset are connected to.  This is why people have HOST BASED FW/IPS ON SERVERS!  To get security as close as possible!  Is that needed? 

So my first response to those that say, security between virtual machines is not needed because its not done in the physical world is:  Well, just because people have done things one way for many years doesn't mean there isn't a better way.

Would environments be more secure if there was security between servers?  I tend to think so.  You see, many of the attacks that are taking place these days are not attacks for fame but attacks for fortune and gone are the days where people just hacked to spread nasty viruses.  Its all about the data these days (ie. credit cards, social security numbers, etc).  We've all heard about the TJ Max security breach where customer data was compromised and many others like banks that have had credit cards compromised. 

How and the heck do you think most of these things happened?  Attackers are targeting the datacenter these days.  Physical or Virtual.  Their gateway into these environments are the Web Front End Servers.  Let me say that again.  The Web Front End Servers!  Hackers get to the data from the web front end server that talks to the database backend server.  This useually occurs by something called "Cross-Site Scripting" or "SQL Injection" breaches. 

Here is a trival way of how this happens:

A hacker finds a vulnerable web site.  He sometimes does this by something called Google Hacking.  He uses Google to search for sites that has vulnerabilities on it.  Say a web site has some content on one of the pages that says "Powered by Drupal 4.1".  If a hacker knows that Drupal 4.1 software has a vulnerability in it, he can now target all the search results related to this.  Click Here for more detail.

Now lets say Drupal 4.1 on a web site has a SQL-Injection vulnerability because the developer of the Drupal software didn't do Form Field Validation properly.  A Form field is something you fill out on a web page like a form that asks for the user name and password.  User names and passwords to log into the web site are stored on whats called a Database Server.  Hmmm... So this means the web server needs to talk to the database server right?  Yes!  Keep this in the back of our head for now.  The hacker enters in "Admin" for the user ID and "password doesn't matter 'or 1=1--" for the password.  And presto!  He is logged in to the server as Admin.

The reason he was able to log in is because the web site sends a SQL Database command to the Database server and because the developer of the Drupal software didn't do "Form Field Validation" properly (method of checking for invalid characters like the ' (single quote)  symbol), the user was able to bypass the password.  Notice the 'OR 1=1 command appended to the password.  One does equal one so therefore it will return a TRUE result to the password checker and the OR says use the password typed in (password doesnt matter) OR check to see if one is equal to one.  If its true then the password is valid for this user which is Admin.

Now that the user is on the web server, he probably has the ability to connect to the database server or other servers in the network.  Why?  Because there is connectivity from the web front end to all of the backend servers.  He essently can backdoor his way throughout the network.

Another method is for him to append some SQL statement to another SQL statement.  Lets say their is a FORM FIELD on the website that collects some information from the database to display it to web site users.  It could be entering in the Zip code to find store locations in your area.  Instead of putting in the zip code you could put in "95123 'UNION SELECT * FROM credit_card_table--".  The hacker is injecting via the UNION command (which means join one SQL statement with another one) a command that says grab all (via the asterisk) information out the credit card table.

Lastly, the hacker can use the UNION command to write text of his desire to a text file on the database server.  He may write some nasty code, tell the database to write the code to a file and then tell the server to execute that file.  The code could be used to do a denial of service attack to the other virtual machines or whatever.  The possibilities are endless!!

Anyway, these are high level examples.  I think you get the point.

The Web Front End Virtual Machine has a need to talk to the Web Back End Virtual Machine and security such as Firewalling, Intrusion Prevention definately needs to be in place to have a higher level of security.

Another reason to have security between virtual machines is because servers are now mobile in the virtual world.  They move between trust domains to take advantage of computing resources that may be available on a given piece of hardware.  Lets say one PHYSICAL server was hosting database VM's and another PHYSICAL server was hosting file server VM's.  The file server VM could VMOTION to the same environment as the database VM's.   Now where is your isolation between trust domains or unlike resources?

People should think about this problem in greater detail.  I'd love to hear everyones comments as to whether or not they think security between VM's is needed.

John Peterson
Montego Networks

I've blogged on this topic of Netflow enabling visibility within virtual environments in the past but thought I'd discuss this topic once again as I feel visibility within virtual networks is VERY important.

One of the big problems that comes along with virtualization is the inability to see "hidden" traffic flows within virtual networks created by VMWare, Citrix and others.  There are a number of companies off building "agents" or visibility products that can drop inside VMWare to bring back that visibility that was once had in the physical network and I think this is great!  But! These solutions although needed are yet another tool that needs to be purchased, managed and maintained.  These new solutions also only work within the virtual environment.

Wouldn't it be great if you could leverage either existing tools that give you physical network visibility or being able to use a tool that could give you both physical and virtual visibility?  It would be one less thing to manage right?  It could also probably correlate information for your entire network vs. just a subset of it.

Well, look no further.  With the enablement of a feature called Netflow within virtual switches from Montego Networks and an experimental version that exists in VMWare ESX 3.5 you can now export Netflow records to physical network monitoring solutions from the likes of Lancope, Plixer International, Mazu Networks, Arbor Networks and others.

What triggered my blog on this topic today was a webinar I listened in on this morning from CTO Adam Powers of Lancope.  He did an excellent job explaining the how VM 2 VM communications are hidden and how you can bring back that visibility by leveraging Netflow and Lancope. 

I would suggest everyone interested in this topic CLICK HERE to register for the next Webinar by Lancope on this topic.  It starts at 2:00 PM EST today May 21st 2008. 

Bellow are a couple of screenshots from the webinar that was hosted earlier today.

CLICK IMAGES TO ENLARGE

Also the picture bellow shows a nice graphic of how the Montego Networks HyperSwitch interacts with Netflow devices.

Again, I would suggest everyone interested in this topic CLICK HERE to register for the next Webinar by Lancope on this topic.  It starts at 2:00 PM EST today May 21st 2008.

John Peterson

I've blogged on this topic of Netflow enabling visibility within virtual environments in the past but thought I'd discuss this topic once again as I feel visibility within virtual networks is VERY important.

One of the big problems that comes along with virtualization is the inability to see "hidden" traffic flows within virtual networks created by VMWare, Citrix and others.  There are a number of companies off building "agents" or visibility products that can drop inside VMWare to bring back that visibility that was once had in the physical network and I think this is great!  But! These solutions although needed are yet another tool that needs to be purchased, managed and maintained.  These new solutions also only work within the virtual environment.

Wouldn't it be great if you could leverage either existing tools that give you physical network visibility or being able to use a tool that could give you both physical and virtual visibility?  It would be one less thing to manage right?  It could also probably correlate information for your entire network vs. just a subset of it.

Well, look no further.  With the enablement of a feature called Netflow within virtual switches from Montego Networks and an experimental version that exists in VMWare ESX 3.5 you can now export Netflow records to physical network monitoring solutions from the likes of Lancope, Plixer International, Mazu Networks, Arbor Networks and others.

What triggered my blog on this topic today was a webinar I listened in on this morning from CTO Adam Powers of Lancope.  He did an excellent job explaining the how VM 2 VM communications are hidden and how you can bring back that visibility by leveraging Netflow and Lancope. 

I would suggest everyone interested in this topic CLICK HERE to register for the next Webinar by Lancope on this topic.  It starts at 2:00 PM EST today May 21st 2008. 

Bellow are a couple of screenshots from the webinar that was hosted earlier today.

CLICK IMAGES TO ENLARGE

Also the picture bellow shows a nice graphic of how the Montego Networks HyperSwitch interacts with Netflow devices.

Again, I would suggest everyone interested in this topic CLICK HERE to register for the next Webinar by Lancope on this topic.  It starts at 2:00 PM EST today May 21st 2008.

John Peterson

Simon Crosby, CTO of Citrix/XenSource made a pretty bold statement yesterday that has some people agreeing with his position and others disagreeing.  In an interview with searchsecurity.com he publicy stated that virtualization vendors are not competent to try and secure virtual environments and therefore looks to 3rd party security companies to solve these concerns. 

Listen to the podcast here

Who are these 3rd party security companies?  Well, there are a number of startup companies such as Montego Networks, Blue Lane, Catbird, Altor Networks as well as some of the big guys that are working on helping the virtualization vendors with these security concerns.

I tend to agree with Simon that the virtualization vendors don't currently have the expertise to deliver appropriate security controls for virtual environments BUT should they?

Well, Chris Hoff who blogs on the topic of virtualization security a lot seems to think that they should deliver security tools and and by not delivering solutions to secure the environment they are doing their customers a disservice.

"Further, I don't expect that the hypervisor should be the place in which all security functionality is delivered, but simply transferring the lack of design and architecture forethought from the hypervisor provider to the consumer by expecting someone else to clean up the mess is just, well, typical."  Said Chris Hoff in his blog on this topic

I've spoken with a number of research analysts, venture capitalists and customers on this topic over the last several months and whenever I tell them what Montego Networks is off building they ALL seem to ask the same questions.  One of those questions is:  Why isn't VMWare or Citrix/Xensource doing this?  My response has always been that "they have publicly stated they do not want to and plan on leveraging an eco-system of security vendors to provide this". 

Well, Simon's public statement is right in line with what I've been saying all along.  The other question I get when I describe how Montego has security built into a virtual switch we've created is; shouldn't this technology be in the VMWare Virtual Switch?  And my response is "absolutely!  But it isn't!  so, someones got to do it."

So, I agree with Chris Hoff and I also agree with Simon Crosby.  The virtualization vendors don't have the expertise BUT I feel they should provide SOME security tools to ensure the environment is safe. 

There are some virtualization vendors that I have spoken with that are planning on using security as a differentiator and its my prediction that one of them will acquire security technology to do this.   Its often easier to acquire vs. try and built it yourself given you don't currently have the expertise.

So who's problem is it to solve??  Virtualization Vendors or Security Vendors??

I see the finger pointing game starting!

-John Peterson

CTO / Montego Networks

Simon Crosby, CTO of Citrix/XenSource made a pretty bold statement yesterday that has some people agreeing with his position and others disagreeing.  In an interview with searchsecurity.com he publicy stated that virtualization vendors are not competent to try and secure virtual environments and therefore looks to 3rd party security companies to solve these concerns. 

Listen to the podcast here

Who are these 3rd party security companies?  Well, there are a number of startup companies such as Montego Networks, Blue Lane, Catbird, Altor Networks as well as some of the big guys that are working on helping the virtualization vendors with these security concerns.

I tend to agree with Simon that the virtualization vendors don't currently have the expertise to deliver appropriate security controls for virtual environments BUT should they?

Well, Chris Hoff who blogs on the topic of virtualization security a lot seems to think that they should deliver security tools and and by not delivering solutions to secure the environment they are doing their customers a disservice.

"Further, I don't expect that the hypervisor should be the place in which all security functionality is delivered, but simply transferring the lack of design and architecture forethought from the hypervisor provider to the consumer by expecting someone else to clean up the mess is just, well, typical."  Said Chris Hoff in his blog on this topic

I've spoken with a number of research analysts, venture capitalists and customers on this topic over the last several months and whenever I tell them what Montego Networks is off building they ALL seem to ask the same questions.  One of those questions is:  Why isn't VMWare or Citrix/Xensource doing this?  My response has always been that "they have publicly stated they do not want to and plan on leveraging an eco-system of security vendors to provide this". 

Well, Simon's public statement is right in line with what I've been saying all along.  The other question I get when I describe how Montego has security built into a virtual switch we've created is; shouldn't this technology be in the VMWare Virtual Switch?  And my response is "absolutely!  But it isn't!  so, someones got to do it."

So, I agree with Chris Hoff and I also agree with Simon Crosby.  The virtualization vendors don't have the expertise BUT I feel they should provide SOME security tools to ensure the environment is safe. 

There are some virtualization vendors that I have spoken with that are planning on using security as a differentiator and its my prediction that one of them will acquire security technology to do this.   Its often easier to acquire vs. try and built it yourself given you don't currently have the expertise.

So who's problem is it to solve??  Virtualization Vendors or Security Vendors??

I see the finger pointing game starting!

-John Peterson

CTO / Montego Networks

I blogged on this topic a few weeks ago but given the huge interest in this topic I’ve decided to blog on it again. One of the major concerns in virtualized environments is the lack of visibility of the communication between virtual machines. With this lack of visibility a number of challenges start to appear such as security, monitoring and capacity planning.  It’s hard to secure what you can’t see or don’t know about and it’s hard to determine when you need to add more resources when you don’t have a clear picture into what applications are consuming them.

This problem is widely known and as a result there are a few companies that are starting to pop up that are building Virtual Network Visibility tools. But should you buy yet another tool to gain visibility into your Virtual Network communication when you may already have a tool for your physical network? Should you have to have separate tools for your physical network and virtual network?

One common method of gaining visibility into network communication is through a technology called Netflow. Netflow was originally developed by Cisco Systems but has since become a defacto standard for Network Monitoring and Network Behavioral Analysis. Companies such as Lancope, Mazu Networks, Plixer International and Arbor Networks all have products that enable network visibility, monitoring and analysis. These tools typicaly take Netflow feeds from a switch of some sort.  Knowing that some of these tools may have already been deployed in physical environments, IT staff will now need to consider  whether or not to buy new visibility tools to give them visibility into their virtual environment communication or try and leverage existing solutions already deployed in their physical environments.

Up until recently there has been no elegant way to export Netflow records from virtual environments such as VMWare and as a result companies have had consider purchasing new visibility tools that would often antiquate their existing physical solutions. This is due to their migration from physical environments to virtual environments.

Montego Networks now has Netflow capability in its HyperSwitch product which runs inside VMWare and enables security, visibility and control for the virtual environment by leveraging existing tools. Through its API’s and standards based methods Montego can enable customers to leverage existing infrastructure purchases to gain visibility and control within the virtual environment.

So, enough of the commercial and lets get on to the technical meat of this new Netflow enablement within the virtual environment.

Let’s say that you have a virtual machine that is infected with a BOT and it is communicating to a Command and Control Site of a BOT-Army. How would you know this? Well, you could have a NetFlow tap at a network switch close to your internet connection. But what if you have some sort of communication between VM’s on a non standard port that you are not aware of? Maybe a machine got infected and is sending data from the database virtual machine to a web server virtual machine and then feeding that info from the web server virtual machine to the internet. Your Netflow tap on the internet facing switch would see traffic coming from the web server virtual machine to the internet but wouldn’t see that data was being taken from the database, put on the web server and then fed out to the internet. Kinda tricky to hunt this problem down isn’t it?

So, whats needed is Netflow all the way into the virtual environment so that it can be fed to the same tools in your physical environment for easy correlation.

Take a look at the attached screen shot which shows Lancope and Montego Networks in action.

<---Click to Enlarge

With this level of visibility now you can see who is talking to who, when are they communicating and how much traffic is being consumed by which applications and which virtual machines.  This can now all be done by leveraging existing Netflow analytics tools.

This screen shot is showing flow data of Virtual Machines talking either to the Internet or to other virtual machines within the same environment.  You will notice from the flow data that one of the Virtual Machines has iTunes running on it.  An IT Administrator may have not sanctioned this or even know about it.  But with Flow records you can now see!  Like a new pair of glasses for your virtual environment.  With this visibility you can now go in to the Montego HyperSwitch and enable a firewall policy to block that iTunes traffic as an example.
 

Lancope is just one example here and its important to note that, because Netflow is a defacto standard for this type of visibility, other tools such as those from Mazu Networks, Plixer International and others can be used as well.  They all have their unique advantages and disadvantages but the point here is that dependent upon your prior network purchases in this area you will now be able to leverage existing tools vs. having to purchase new ones in many cases.

Check out Montego Networks at Networld Interop 2008 in the Lancope booth to see the solution in action!

John Peterson
CTO Montego Networks

I blogged on this topic a few weeks ago but given the huge interest in this topic I???ve decided to blog on it again. One of the major concerns in virtualized environments is the lack of visibility of the communication between virtual machines. With this lack of visibility a number of challenges start to appear such as security, monitoring and capacity planning.  It???s hard to secure what you can???t see or don???t know about and it???s hard to determine when you need to add more resources when you don???t have a clear picture into what applications are consuming them.

This problem is widely known and as a result there are a few companies that are starting to pop up that are building Virtual Network Visibility tools. But should you buy yet another tool to gain visibility into your Virtual Network communication when you may already have a tool for your physical network? Should you have to have separate tools for your physical network and virtual network?

One common method of gaining visibility into network communication is through a technology called Netflow. Netflow was originally developed by Cisco Systems but has since become a defacto standard for Network Monitoring and Network Behavioral Analysis. Companies such as Lancope, Mazu Networks, Plixer International and Arbor Networks all have products that enable network visibility, monitoring and analysis. These tools typicaly take Netflow feeds from a switch of some sort.  Knowing that some of these tools may have already been deployed in physical environments, IT staff will now need to consider  whether or not to buy new visibility tools to give them visibility into their virtual environment communication or try and leverage existing solutions already deployed in their physical environments.

Up until recently there has been no elegant way to export Netflow records from virtual environments such as VMWare and as a result companies have had consider purchasing new visibility tools that would often antiquate their existing physical solutions. This is due to their migration from physical environments to virtual environments.

Montego Networks now has Netflow capability in its HyperSwitch product which runs inside VMWare and enables security, visibility and control for the virtual environment by leveraging existing tools. Through its API???s and standards based methods Montego can enable customers to leverage existing infrastructure purchases to gain visibility and control within the virtual environment.

So, enough of the commercial and lets get on to the technical meat of this new Netflow enablement within the virtual environment.

Let???s say that you have a virtual machine that is infected with a BOT and it is communicating to a Command and Control Site of a BOT-Army. How would you know this? Well, you could have a NetFlow tap at a network switch close to your internet connection. But what if you have some sort of communication between VM???s on a non standard port that you are not aware of? Maybe a machine got infected and is sending data from the database virtual machine to a web server virtual machine and then feeding that info from the web server virtual machine to the internet. Your Netflow tap on the internet facing switch would see traffic coming from the web server virtual machine to the internet but wouldn???t see that data was being taken from the database, put on the web server and then fed out to the internet. Kinda tricky to hunt this problem down isn???t it?

So, whats needed is Netflow all the way into the virtual environment so that it can be fed to the same tools in your physical environment for easy correlation.

Take a look at the attached screen shot which shows Lancope and Montego Networks in action.

<---Click to Enlarge

With this level of visibility now you can see who is talking to who, when are they communicating and how much traffic is being consumed by which applications and which virtual machines.  This can now all be done by leveraging existing Netflow analytics tools.

This screen shot is showing flow data of Virtual Machines talking either to the Internet or to other virtual machines within the same environment.  You will notice from the flow data that one of the Virtual Machines has iTunes running on it.  An IT Administrator may have not sanctioned this or even know about it.  But with Flow records you can now see!  Like a new pair of glasses for your virtual environment.  With this visibility you can now go in to the Montego HyperSwitch and enable a firewall policy to block that iTunes traffic as an example.
 

Lancope is just one example here and its important to note that, because Netflow is a defacto standard for this type of visibility, other tools such as those from Mazu Networks, Plixer International and others can be used as well.  They all have their unique advantages and disadvantages but the point here is that dependent upon your prior network purchases in this area you will now be able to leverage existing tools vs. having to purchase new ones in many cases.

Check out Montego Networks at Networld Interop 2008 in the Lancope booth to see the solution in action!

John Peterson
CTO Montego Networks