Virtual Environment performance has been a widely discussed topic when it comes to running security within virtual environments and there is this concept that I have had in my head for a while now that I thought I'd share with the public to get feedback on.  Its called the Virtual Security Nic and is intended to move security out of the shared computing layer (virtual environment) and into the physical layer with dedicated processors.  By doing this the performance challenge goes away and you are able to get security as close as possible to the VM's.  All traffic going from VM to VM will have to traverse the bus and be inspected by this security NIC before it is delivered to its final destination.

Take a look at the picture bellow and feel free to comment either on this blog or email me at:  jpeterson@montegonetworks.com

Is there security needed between virtual machines?  Some say no, some say yes.  I've been out talking to a number of virtualization users and non users on this topic and I'm finding that some say no and some say yes.  The users of virtualization technology tend to say yes while others looking at virtualization from the outside tend to say no.  Why is this?

Well, I thought I'd blog on my thoughts on this!

You see, in the physical datacenter there is no firewalling between servers plugged into the same switch and because of this some people think, well if its not done in the physical world why should it be done in the virtual world.  I believe that its not done in the physical world today because there are no solutions today that embed security into datacenter switches.  Should it be done in the physical world?  I think so!  It never hurts to get security as close as possible to the things you are trying to protect and what better place than the switch port in which the critical asset are connected to.  This is why people have HOST BASED FW/IPS ON SERVERS!  To get security as close as possible!  Is that needed? 

So my first response to those that say, security between virtual machines is not needed because its not done in the physical world is:  Well, just because people have done things one way for many years doesn't mean there isn't a better way.

Would environments be more secure if there was security between servers?  I tend to think so.  You see, many of the attacks that are taking place these days are not attacks for fame but attacks for fortune and gone are the days where people just hacked to spread nasty viruses.  Its all about the data these days (ie. credit cards, social security numbers, etc).  We've all heard about the TJ Max security breach where customer data was compromised and many others like banks that have had credit cards compromised. 

How and the heck do you think most of these things happened?  Attackers are targeting the datacenter these days.  Physical or Virtual.  Their gateway into these environments are the Web Front End Servers.  Let me say that again.  The Web Front End Servers!  Hackers get to the data from the web front end server that talks to the database backend server.  This useually occurs by something called "Cross-Site Scripting" or "SQL Injection" breaches. 

Here is a trival way of how this happens:

A hacker finds a vulnerable web site.  He sometimes does this by something called Google Hacking.  He uses Google to search for sites that has vulnerabilities on it.  Say a web site has some content on one of the pages that says "Powered by Drupal 4.1".  If a hacker knows that Drupal 4.1 software has a vulnerability in it, he can now target all the search results related to this.  Click Here for more detail.

Now lets say Drupal 4.1 on a web site has a SQL-Injection vulnerability because the developer of the Drupal software didn't do Form Field Validation properly.  A Form field is something you fill out on a web page like a form that asks for the user name and password.  User names and passwords to log into the web site are stored on whats called a Database Server.  Hmmm... So this means the web server needs to talk to the database server right?  Yes!  Keep this in the back of our head for now.  The hacker enters in "Admin" for the user ID and "password doesn't matter 'or 1=1--" for the password.  And presto!  He is logged in to the server as Admin.

The reason he was able to log in is because the web site sends a SQL Database command to the Database server and because the developer of the Drupal software didn't do "Form Field Validation" properly (method of checking for invalid characters like the ' (single quote)  symbol), the user was able to bypass the password.  Notice the 'OR 1=1 command appended to the password.  One does equal one so therefore it will return a TRUE result to the password checker and the OR says use the password typed in (password doesnt matter) OR check to see if one is equal to one.  If its true then the password is valid for this user which is Admin.

Now that the user is on the web server, he probably has the ability to connect to the database server or other servers in the network.  Why?  Because there is connectivity from the web front end to all of the backend servers.  He essently can backdoor his way throughout the network.

Another method is for him to append some SQL statement to another SQL statement.  Lets say their is a FORM FIELD on the website that collects some information from the database to display it to web site users.  It could be entering in the Zip code to find store locations in your area.  Instead of putting in the zip code you could put in "95123 'UNION SELECT * FROM credit_card_table--".  The hacker is injecting via the UNION command (which means join one SQL statement with another one) a command that says grab all (via the asterisk) information out the credit card table.

Lastly, the hacker can use the UNION command to write text of his desire to a text file on the database server.  He may write some nasty code, tell the database to write the code to a file and then tell the server to execute that file.  The code could be used to do a denial of service attack to the other virtual machines or whatever.  The possibilities are endless!!

Anyway, these are high level examples.  I think you get the point.

The Web Front End Virtual Machine has a need to talk to the Web Back End Virtual Machine and security such as Firewalling, Intrusion Prevention definately needs to be in place to have a higher level of security.

Another reason to have security between virtual machines is because servers are now mobile in the virtual world.  They move between trust domains to take advantage of computing resources that may be available on a given piece of hardware.  Lets say one PHYSICAL server was hosting database VM's and another PHYSICAL server was hosting file server VM's.  The file server VM could VMOTION to the same environment as the database VM's.   Now where is your isolation between trust domains or unlike resources?

People should think about this problem in greater detail.  I'd love to hear everyones comments as to whether or not they think security between VM's is needed.

John Peterson
Montego Networks

I've blogged on this topic of Netflow enabling visibility within virtual environments in the past but thought I'd discuss this topic once again as I feel visibility within virtual networks is VERY important.

One of the big problems that comes along with virtualization is the inability to see "hidden" traffic flows within virtual networks created by VMWare, Citrix and others.  There are a number of companies off building "agents" or visibility products that can drop inside VMWare to bring back that visibility that was once had in the physical network and I think this is great!  But! These solutions although needed are yet another tool that needs to be purchased, managed and maintained.  These new solutions also only work within the virtual environment.

Wouldn't it be great if you could leverage either existing tools that give you physical network visibility or being able to use a tool that could give you both physical and virtual visibility?  It would be one less thing to manage right?  It could also probably correlate information for your entire network vs. just a subset of it.

Well, look no further.  With the enablement of a feature called Netflow within virtual switches from Montego Networks and an experimental version that exists in VMWare ESX 3.5 you can now export Netflow records to physical network monitoring solutions from the likes of Lancope, Plixer International, Mazu Networks, Arbor Networks and others.

What triggered my blog on this topic today was a webinar I listened in on this morning from CTO Adam Powers of Lancope.  He did an excellent job explaining the how VM 2 VM communications are hidden and how you can bring back that visibility by leveraging Netflow and Lancope. 

I would suggest everyone interested in this topic CLICK HERE to register for the next Webinar by Lancope on this topic.  It starts at 2:00 PM EST today May 21st 2008. 

Bellow are a couple of screenshots from the webinar that was hosted earlier today.

CLICK IMAGES TO ENLARGE

Also the picture bellow shows a nice graphic of how the Montego Networks HyperSwitch interacts with Netflow devices.

Again, I would suggest everyone interested in this topic CLICK HERE to register for the next Webinar by Lancope on this topic.  It starts at 2:00 PM EST today May 21st 2008.

John Peterson

I've blogged on this topic of Netflow enabling visibility within virtual environments in the past but thought I'd discuss this topic once again as I feel visibility within virtual networks is VERY important.

One of the big problems that comes along with virtualization is the inability to see "hidden" traffic flows within virtual networks created by VMWare, Citrix and others.  There are a number of companies off building "agents" or visibility products that can drop inside VMWare to bring back that visibility that was once had in the physical network and I think this is great!  But! These solutions although needed are yet another tool that needs to be purchased, managed and maintained.  These new solutions also only work within the virtual environment.

Wouldn't it be great if you could leverage either existing tools that give you physical network visibility or being able to use a tool that could give you both physical and virtual visibility?  It would be one less thing to manage right?  It could also probably correlate information for your entire network vs. just a subset of it.

Well, look no further.  With the enablement of a feature called Netflow within virtual switches from Montego Networks and an experimental version that exists in VMWare ESX 3.5 you can now export Netflow records to physical network monitoring solutions from the likes of Lancope, Plixer International, Mazu Networks, Arbor Networks and others.

What triggered my blog on this topic today was a webinar I listened in on this morning from CTO Adam Powers of Lancope.  He did an excellent job explaining the how VM 2 VM communications are hidden and how you can bring back that visibility by leveraging Netflow and Lancope. 

I would suggest everyone interested in this topic CLICK HERE to register for the next Webinar by Lancope on this topic.  It starts at 2:00 PM EST today May 21st 2008. 

Bellow are a couple of screenshots from the webinar that was hosted earlier today.

CLICK IMAGES TO ENLARGE

Also the picture bellow shows a nice graphic of how the Montego Networks HyperSwitch interacts with Netflow devices.

Again, I would suggest everyone interested in this topic CLICK HERE to register for the next Webinar by Lancope on this topic.  It starts at 2:00 PM EST today May 21st 2008.

John Peterson

This week has been an interesting week in the virtual security blog world!  Simon Crosby of Citrix/XenSource stated in his podcast that he felt the virtualization vendors like VMWare and Citrix didn't have the competence to address the security challenges of virtualization and Chris Hoff blogged about it saying that the statement is a cop-out and that they should do more in securing their platforms. Alan Shimel also blogged on the topic and agreed with Hoff and I blogged about it agreeing with both Simon and Hoff. 

To restate my position on it I think that Simon is correct in that virtualization vendors like VMWare and Citrix do not have the expertise today to address all of the security challenges.  I also agree with Hoff that they should address more of the security challenges.  So this leads me to my own opinion that some of the virtualization vendors will acquire security technologies to differentiate  themselves from others and acquire the expertise.  Many say that the virtualization market will become commoditized and  that security can help protect its value. 

Think about it.  Would you rather buy a Virtual Environment or a Secure Virtual Environment?!

So.. Onto the topic of this blog!  Is Virtual Security Technology A Prime Target For Acquisition?

I'd love your opinion so please comment!!

What triggered my blog on this topic was this rumor I heard today.  Some buzz started today that one of the virtual security startups just agreed behind closed doors to be acquired by one of the big guys.  But, who could it be?  Reflex Security, Catbird, Blue Lane, Altor Networks, VMSight, Embotics, etc.

I have an idea of who it could be but don't want to spread rumors that could be false.  The other question is whether or not there is an atmosphere of acquisition frenzy brewing in the virtualization market. 

Please comment on your thoughts - Just click the comments link bellow.

This week has been an interesting week in the virtual security blog world!  Simon Crosby of Citrix/XenSource stated in his podcast that he felt the virtualization vendors like VMWare and Citrix didn't have the competence to address the security challenges of virtualization and Chris Hoff blogged about it saying that the statement is a cop-out and that they should do more in securing their platforms. Alan Shimel also blogged on the topic and agreed with Hoff and I blogged about it agreeing with both Simon and Hoff. 

To restate my position on it; I think that Simon is correct in that virtualization vendors like VMWare and Citrix do not have the expertise today to address all of the security challenges.  I also agree with Hoff that they should address more of the security challenges.  So this leads me to my own opinion that some of the virtualization vendors will acquire security technologies to differentiate  themselves from others and acquire the expertise.  Many say that the virtualization market will become commoditized and  that security can help protect its value. 

Think about it.  Would you rather buy a Virtual Environment or a Secure Virtual Environment?!

So.. Onto the topic of this blog!  Is Virtual Security Technology A Prime Target For Acquisition?

I'd love your opinion so please comment!!

What triggered my blog on this topic was this rumor I heard today.  Some buzz started today that one of the virtual security startups just agreed behind closed doors to be acquired by one of the big guys.  But, who could it be?  Reflex Security, Catbird, Blue Lane, Altor Networks, VMSight, Embotics, etc.

I have an idea of who it could be but don't want to spread rumors that could be false.  The other question is whether or not there is an atmosphere of acquisition frenzy brewing in the virtualization market. 

Please comment on your thoughts - Just click the comments link bellow.

Simon Crosby, CTO of Citrix/XenSource made a pretty bold statement yesterday that has some people agreeing with his position and others disagreeing.  In an interview with searchsecurity.com he publicy stated that virtualization vendors are not competent to try and secure virtual environments and therefore looks to 3rd party security companies to solve these concerns. 

Listen to the podcast here

Who are these 3rd party security companies?  Well, there are a number of startup companies such as Montego Networks, Blue Lane, Catbird, Altor Networks as well as some of the big guys that are working on helping the virtualization vendors with these security concerns.

I tend to agree with Simon that the virtualization vendors don't currently have the expertise to deliver appropriate security controls for virtual environments BUT should they?

Well, Chris Hoff who blogs on the topic of virtualization security a lot seems to think that they should deliver security tools and and by not delivering solutions to secure the environment they are doing their customers a disservice.

"Further, I don't expect that the hypervisor should be the place in which all security functionality is delivered, but simply transferring the lack of design and architecture forethought from the hypervisor provider to the consumer by expecting someone else to clean up the mess is just, well, typical."  Said Chris Hoff in his blog on this topic

I've spoken with a number of research analysts, venture capitalists and customers on this topic over the last several months and whenever I tell them what Montego Networks is off building they ALL seem to ask the same questions.  One of those questions is:  Why isn't VMWare or Citrix/Xensource doing this?  My response has always been that "they have publicly stated they do not want to and plan on leveraging an eco-system of security vendors to provide this". 

Well, Simon's public statement is right in line with what I've been saying all along.  The other question I get when I describe how Montego has security built into a virtual switch we've created is; shouldn't this technology be in the VMWare Virtual Switch?  And my response is "absolutely!  But it isn't!  so, someones got to do it."

So, I agree with Chris Hoff and I also agree with Simon Crosby.  The virtualization vendors don't have the expertise BUT I feel they should provide SOME security tools to ensure the environment is safe. 

There are some virtualization vendors that I have spoken with that are planning on using security as a differentiator and its my prediction that one of them will acquire security technology to do this.   Its often easier to acquire vs. try and built it yourself given you don't currently have the expertise.

So who's problem is it to solve??  Virtualization Vendors or Security Vendors??

I see the finger pointing game starting!

-John Peterson

CTO / Montego Networks

Simon Crosby, CTO of Citrix/XenSource made a pretty bold statement yesterday that has some people agreeing with his position and others disagreeing.  In an interview with searchsecurity.com he publicy stated that virtualization vendors are not competent to try and secure virtual environments and therefore looks to 3rd party security companies to solve these concerns. 

Listen to the podcast here

Who are these 3rd party security companies?  Well, there are a number of startup companies such as Montego Networks, Blue Lane, Catbird, Altor Networks as well as some of the big guys that are working on helping the virtualization vendors with these security concerns.

I tend to agree with Simon that the virtualization vendors don't currently have the expertise to deliver appropriate security controls for virtual environments BUT should they?

Well, Chris Hoff who blogs on the topic of virtualization security a lot seems to think that they should deliver security tools and and by not delivering solutions to secure the environment they are doing their customers a disservice.

"Further, I don't expect that the hypervisor should be the place in which all security functionality is delivered, but simply transferring the lack of design and architecture forethought from the hypervisor provider to the consumer by expecting someone else to clean up the mess is just, well, typical."  Said Chris Hoff in his blog on this topic

I've spoken with a number of research analysts, venture capitalists and customers on this topic over the last several months and whenever I tell them what Montego Networks is off building they ALL seem to ask the same questions.  One of those questions is:  Why isn't VMWare or Citrix/Xensource doing this?  My response has always been that "they have publicly stated they do not want to and plan on leveraging an eco-system of security vendors to provide this". 

Well, Simon's public statement is right in line with what I've been saying all along.  The other question I get when I describe how Montego has security built into a virtual switch we've created is; shouldn't this technology be in the VMWare Virtual Switch?  And my response is "absolutely!  But it isn't!  so, someones got to do it."

So, I agree with Chris Hoff and I also agree with Simon Crosby.  The virtualization vendors don't have the expertise BUT I feel they should provide SOME security tools to ensure the environment is safe. 

There are some virtualization vendors that I have spoken with that are planning on using security as a differentiator and its my prediction that one of them will acquire security technology to do this.   Its often easier to acquire vs. try and built it yourself given you don't currently have the expertise.

So who's problem is it to solve??  Virtualization Vendors or Security Vendors??

I see the finger pointing game starting!

-John Peterson

CTO / Montego Networks

I blogged on this topic a few weeks ago but given the huge interest in this topic I’ve decided to blog on it again. One of the major concerns in virtualized environments is the lack of visibility of the communication between virtual machines. With this lack of visibility a number of challenges start to appear such as security, monitoring and capacity planning.  It’s hard to secure what you can’t see or don’t know about and it’s hard to determine when you need to add more resources when you don’t have a clear picture into what applications are consuming them.

This problem is widely known and as a result there are a few companies that are starting to pop up that are building Virtual Network Visibility tools. But should you buy yet another tool to gain visibility into your Virtual Network communication when you may already have a tool for your physical network? Should you have to have separate tools for your physical network and virtual network?

One common method of gaining visibility into network communication is through a technology called Netflow. Netflow was originally developed by Cisco Systems but has since become a defacto standard for Network Monitoring and Network Behavioral Analysis. Companies such as Lancope, Mazu Networks, Plixer International and Arbor Networks all have products that enable network visibility, monitoring and analysis. These tools typicaly take Netflow feeds from a switch of some sort.  Knowing that some of these tools may have already been deployed in physical environments, IT staff will now need to consider  whether or not to buy new visibility tools to give them visibility into their virtual environment communication or try and leverage existing solutions already deployed in their physical environments.

Up until recently there has been no elegant way to export Netflow records from virtual environments such as VMWare and as a result companies have had consider purchasing new visibility tools that would often antiquate their existing physical solutions. This is due to their migration from physical environments to virtual environments.

Montego Networks now has Netflow capability in its HyperSwitch product which runs inside VMWare and enables security, visibility and control for the virtual environment by leveraging existing tools. Through its API’s and standards based methods Montego can enable customers to leverage existing infrastructure purchases to gain visibility and control within the virtual environment.

So, enough of the commercial and lets get on to the technical meat of this new Netflow enablement within the virtual environment.

Let’s say that you have a virtual machine that is infected with a BOT and it is communicating to a Command and Control Site of a BOT-Army. How would you know this? Well, you could have a NetFlow tap at a network switch close to your internet connection. But what if you have some sort of communication between VM’s on a non standard port that you are not aware of? Maybe a machine got infected and is sending data from the database virtual machine to a web server virtual machine and then feeding that info from the web server virtual machine to the internet. Your Netflow tap on the internet facing switch would see traffic coming from the web server virtual machine to the internet but wouldn’t see that data was being taken from the database, put on the web server and then fed out to the internet. Kinda tricky to hunt this problem down isn’t it?

So, whats needed is Netflow all the way into the virtual environment so that it can be fed to the same tools in your physical environment for easy correlation.

Take a look at the attached screen shot which shows Lancope and Montego Networks in action.

<---Click to Enlarge

With this level of visibility now you can see who is talking to who, when are they communicating and how much traffic is being consumed by which applications and which virtual machines.  This can now all be done by leveraging existing Netflow analytics tools.

This screen shot is showing flow data of Virtual Machines talking either to the Internet or to other virtual machines within the same environment.  You will notice from the flow data that one of the Virtual Machines has iTunes running on it.  An IT Administrator may have not sanctioned this or even know about it.  But with Flow records you can now see!  Like a new pair of glasses for your virtual environment.  With this visibility you can now go in to the Montego HyperSwitch and enable a firewall policy to block that iTunes traffic as an example.
 

Lancope is just one example here and its important to note that, because Netflow is a defacto standard for this type of visibility, other tools such as those from Mazu Networks, Plixer International and others can be used as well.  They all have their unique advantages and disadvantages but the point here is that dependent upon your prior network purchases in this area you will now be able to leverage existing tools vs. having to purchase new ones in many cases.

Check out Montego Networks at Networld Interop 2008 in the Lancope booth to see the solution in action!

John Peterson
CTO Montego Networks

I blogged on this topic a few weeks ago but given the huge interest in this topic I???ve decided to blog on it again. One of the major concerns in virtualized environments is the lack of visibility of the communication between virtual machines. With this lack of visibility a number of challenges start to appear such as security, monitoring and capacity planning.  It???s hard to secure what you can???t see or don???t know about and it???s hard to determine when you need to add more resources when you don???t have a clear picture into what applications are consuming them.

This problem is widely known and as a result there are a few companies that are starting to pop up that are building Virtual Network Visibility tools. But should you buy yet another tool to gain visibility into your Virtual Network communication when you may already have a tool for your physical network? Should you have to have separate tools for your physical network and virtual network?

One common method of gaining visibility into network communication is through a technology called Netflow. Netflow was originally developed by Cisco Systems but has since become a defacto standard for Network Monitoring and Network Behavioral Analysis. Companies such as Lancope, Mazu Networks, Plixer International and Arbor Networks all have products that enable network visibility, monitoring and analysis. These tools typicaly take Netflow feeds from a switch of some sort.  Knowing that some of these tools may have already been deployed in physical environments, IT staff will now need to consider  whether or not to buy new visibility tools to give them visibility into their virtual environment communication or try and leverage existing solutions already deployed in their physical environments.

Up until recently there has been no elegant way to export Netflow records from virtual environments such as VMWare and as a result companies have had consider purchasing new visibility tools that would often antiquate their existing physical solutions. This is due to their migration from physical environments to virtual environments.

Montego Networks now has Netflow capability in its HyperSwitch product which runs inside VMWare and enables security, visibility and control for the virtual environment by leveraging existing tools. Through its API???s and standards based methods Montego can enable customers to leverage existing infrastructure purchases to gain visibility and control within the virtual environment.

So, enough of the commercial and lets get on to the technical meat of this new Netflow enablement within the virtual environment.

Let???s say that you have a virtual machine that is infected with a BOT and it is communicating to a Command and Control Site of a BOT-Army. How would you know this? Well, you could have a NetFlow tap at a network switch close to your internet connection. But what if you have some sort of communication between VM???s on a non standard port that you are not aware of? Maybe a machine got infected and is sending data from the database virtual machine to a web server virtual machine and then feeding that info from the web server virtual machine to the internet. Your Netflow tap on the internet facing switch would see traffic coming from the web server virtual machine to the internet but wouldn???t see that data was being taken from the database, put on the web server and then fed out to the internet. Kinda tricky to hunt this problem down isn???t it?

So, whats needed is Netflow all the way into the virtual environment so that it can be fed to the same tools in your physical environment for easy correlation.

Take a look at the attached screen shot which shows Lancope and Montego Networks in action.

<---Click to Enlarge

With this level of visibility now you can see who is talking to who, when are they communicating and how much traffic is being consumed by which applications and which virtual machines.  This can now all be done by leveraging existing Netflow analytics tools.

This screen shot is showing flow data of Virtual Machines talking either to the Internet or to other virtual machines within the same environment.  You will notice from the flow data that one of the Virtual Machines has iTunes running on it.  An IT Administrator may have not sanctioned this or even know about it.  But with Flow records you can now see!  Like a new pair of glasses for your virtual environment.  With this visibility you can now go in to the Montego HyperSwitch and enable a firewall policy to block that iTunes traffic as an example.
 

Lancope is just one example here and its important to note that, because Netflow is a defacto standard for this type of visibility, other tools such as those from Mazu Networks, Plixer International and others can be used as well.  They all have their unique advantages and disadvantages but the point here is that dependent upon your prior network purchases in this area you will now be able to leverage existing tools vs. having to purchase new ones in many cases.

Check out Montego Networks at Networld Interop 2008 in the Lancope booth to see the solution in action!

John Peterson
CTO Montego Networks