The Azure announcement is the culmination of years of planning for Microsoft’s “software-plus-services approach to computing.” According to Debra Chrapaty, the woman who runs Microsoft’s data center infrastructure, plans started about four to five years ago to build out data center capacity for the new initiatives. The best place to build a new data center: Quincy, Washington – whose hydroelectric power and commitment to fiber made it a winner. (Click here for Mayor Hernberry’s update on the impact of the new data centers and apparently new wineries popping up in Quincy.)

Thank goodness for Microsoft. In this economy, we should all be grateful to companies that can still spend between $300 million to $700 million to build just one data center.

Martin MC Brown at ComputerWorld has a great post on capacity planning and cloud computing. He discusses a new book “The Art of Capacity Planning”. The problem with the current model of data center management is that often a large number of machines may sit relatively idle while waiting for the traffic spike that causes them to be used. This is a problem because it’s simply a waste of time and resources on a whole number of levels. Enter the cloud – or at least the “hope of cloud computing”.

Numbers – what do they really mean? IDC released a statement with a whole bunch of them from their “Worldwide Quarterly Server Virtualization Tracker”. The most interesting stat: x86 Virtualization License Market Standings. VMware owns 44% of the market, but Microsoft, in its first quarter of general availability for Microsoft Hyper-V (plus Virtual Server 2005), has 23% of the market of new shipments.

We have been conducting polls of our customers and of IT professionals at technology trade shows for the past two years and the results are in: Pulling together all of the management pieces and processes is even more crucial in a virtualized environment.

So what does this mean for you? You will need to refine your incident and problem management processes with new technologies in order to reduce downtime and maintain end user performance. But of course even the most basic technologies are not well integrated even in today’s world.

I recently participated in a Gartner Conference and watched to my amazement a real-time electronic survey of the audience. To my disbelief, the audience, filled with 300+ people from Fortune 2000 companies provided real-time responses to the question:

What level of integration does your IT org have between event management and service desk applications?

• None: 10%
• Manual Phone call from IT ops to IT service desk staff member: 46%
• Manual click button on event manager to open trouble ticket: 20%
• Automated event management system automatically opens trouble ticket without requiring human oversight or approval: 24%

Unbelievable… still very few of the survey respondents have yet to formalize problem management systems with event management systems. For 56% of the audience the process is still manual!

Another interesting real-time survey question at the Gartner Conference was:

Who in your organization is responsible for critical problem processes and resolution?

• IT Service Desk 13%
• IT Operations 49%
• Process Team 12%
• Other 9%
• Responsibility not formalized 17%

Virtualization adoption and the speed with which things change in a virtualized environment require automation and will transform Incident and Problem Management. Clearly with this new technology we are required to re-think Organizational, Behavioral and Cultural Challenges required to take advantage of the opportunities that virtualization provides.

Incident and problem management processes and metrics must bridge organizational silos that have been the norm within IT. With virtualization, people have to work more closely together in the different silos than ever before. IT leaders need to break down the walls between the technology-centric silo mentalities.

Business Imperative Action Plan:

1. What can you do today? –Understand the impact of virtualization on incident and problem mgt. workload, provide technology training for helpdesk/service desk staff.
2. What can you do in the next 12 months?

Formalize problem management processes, metrics and personnel.
Invest in tools and processes for systems on virtualized servers.
Long term: On the Radar Screen!
Instill teamwork into all groups responsible for the virtualized environment service and support. Map components and configuration items directly to end user services.

Final Thoughts: Know the management pieces and ensure that they fit together. It’s great to buy new technology, but be demanding to ensure that your vendors show you have they will help to link all these pieces together - Change, Inventory, Incident, Problem, Server, Capacity, Performance, Configuration, Event, and Integrated Workflow.

Here we talk to John Proctor, who started out as one of our first customers (and the first MSP customer). And he believed in it so much, he eventually became part of the ScienceLogic team. (Remember “I’m not only the President, I’m also a client” from the Hair Club for Men?)

John shares his perspectives about the service provider world and why he took a chance on a little-known product called EM7.

ScienceLogic: What is your background? How many years have you worked as a service provider and for what types of companies?

John Proctor: I have been working with Service providers for over twelve years. I worked at a major regional service provider for six years and before that I designed and built national and international networks for ISP’s and Fortune 500 companies as a consultant for PriceWaterhouseCoopers and WorldComm.

ScienceLogic: You were one of the first customers of EM7 – why did you choose it and how did you get over the hurdles associated with using a start-up company’s product?

John Proctor: We were actually customer number five. Back in 2004 when we evaluated and purchased EM7 we could see that EM7 provided about 80% of what we were looking for in one integrated solution right out of the box. One of the things that sold us on EM7 was that the ScienceLogic founders had all previously worked for a service provider, so we knew they understood our business and our challenges. But in the end, it comes down to features. Once we compared EM7 functionality to the alternatives, it was clearly a “no brainer.”

ScienceLogic: What other alternatives were being considered?

John Proctor: Well, we had started with a few point solutions, but as our business and product offerings matured, this resulted in a growing number of point solutions. What started with 3 or 4 ended up as 14 separate tools. They all had strengths but what they didn’t have was integration and because of this they could not scale. And, if the tools could not scale, our business could not grow.

So, naturally we started looking at framework solutions, but they are expensive to buy, expensive to implement, and expensive to maintain. At one point, we even considered some open source projects. There were several that showed promise, but we would still be stuck with tools that were not integrated. So then we considered hiring developers to cobble something together that would work for our business. The only problem with this alternative was that we felt it would take 6 to 8 months before we could have something viable to work with.

ScienceLogic: What products were you using before EM7? What were your goals?

John Proctor: Before we purchased EM7 we used 14 different point solutions to deliver our products and services to the marketplace. Tools like NetCool, Openview, Argent, Heat, What’s Up Gold as well as several other point solutions, vendor specific applications and manually updated spreadsheets. And, as I mentioned before, this does not scale. This also adds a great deal of complexity when you begin to consider business continuity and disaster recovery. All these tools were vital to the delivery of our products and services. Any service provider will tell you it is all about uptime. So if the product is uptime, the tools used to deliver it have to be available 24×7x365.

Our goals were simple: scale and redundancy. As it turns out, the solution was simple as well. EM7 provided a tool that could replace the functionality of almost half of the existing point solutions and the applications that could not be replaced were integrated with EM7 to provide our staff with a “single pane of glass” to see the status and performance of each area of the business from one application. We had visibility into everything from facility systems to applications using EM7.

ScienceLogic also delivers an extensible configuration that addressed uptime and redundancy. We deployed collectors throughout our network that reported back to a central pair of redundant database servers and with this configuration we were able to perform backups and add capacity without taking the system down.

ScienceLogic: Why are service providers different from enterprises? How are their needs different?

John Proctor: First and foremost, service providers face the same challenges that only the largest enterprises ever face and they also have many unique challenges that only service providers experience.

One challenge we faced was that we had multiple datacenters in different states. They were all interconnected with plenty of bandwidth between each site, but the tools were not designed to be used across the WAN. Our staff in our remote data center did not have the same access as our staff in the corporate office. Since EM7 is web-based, it immediately eliminated this problem.

Another challenge is that service providers must manage systems across multiple domains. Back in the early version of a specific tool we were using before EM7, the only way you could implement it across multiple domains was to put the same username and password on every computer that you monitored. Beyond the security concerns, maintenance was a nightmare. Anytime we had to change the password, we would get locked out of dozens upon dozens of systems. When the password was changed on the monitoring server, it would attempt to login to the remote machines and fail. Repeated attempts would result in the account getting locked. I think that vendor eventually addressed this issue, but service providers seldom find tools that were designed for their unique situations.

ScienceLogic: How is EM7 geared to service providers?

John Proctor: Enterprise IT is a trusted part of the business; they are one of the team. Service providers are outsiders that must earn trust by showing the customer exactly what they are doing.

EM7 provides a multi-tenant environment that allows service providers to manage systems across many different customers while at the same time providing the customer access to see the same information but only what’s relevant to them.

EM7 was built by service providers and even includes a few features just for them. Two of my favorites are bandwidth billing and the emergency notification system. Take bandwidth billing, for instance. EM7 provides a way to collect bandwidth utilization, store subscription information, and calculate a bill from any one of about 10 different methodologies. And at the end of the billing period, EM7 sends the completed report out to whomever you chose via email.

Another unique service provider feature is the emergency notification system. EM7 allows the provider to track what customers used their unique infrastructure components. If they have to perform maintenance on the infrastructure component or have a problem they can send an email to all of the impacted customers in a matter of minutes.

ScienceLogic: What trends do you see for service providers? What about big trends such as virtualization and cloud computing – how will they impact service providers?

John Proctor: Virtualization is really hot for service providers right now and for the same reasons as in the enterprise. Service providers run data centers and data centers must be powered and cooled. So, anytime they can use a virtual server instead of adding physical equipment it is a good thing. But then you add the complexity that multiple customers reside on the same host and you must track things like bandwidth utilizations by guest OS, and it all gets a little harder. Lucky for us this is not a problem for EM7.

I still think it’s early days for cloud computing. Depending on who you talk to, much of what service providers (especially the big ones) have already been doing with SAAS offerings and hosted applications could be described as cloud computing already. In which case, service providers are ahead of the game. But whatever the “final” definition, cloud computing actually shares many similarities with virtualization – in that service providers (or enterprises) will need to be able to manage far more “devices” in real-time with “zero downtime” expectations by customers. What this really means is that you’re going to see much more automation in provisioning and IT monitoring tools to handle the scale and speed with which things can change in the data center given vm migration and the talked-about switching between “clouds” that can be used for high availability.

Charlie Rose:  
And so when you look at where we are going, there seems to be two issues that are apparent to me at least, risk and leverage.  We just lost sight of risk and leverage of what was appropriate?

Warren Buffett:  
Yeah.  Again, because it pays off for a while.  You know, you can lose leverage, and it's the only way a smart guy can go broke.  If you owe money, you can't pay them out.  You just pay for everything, you do smart things, you eventually get very rich.  If you do smart things and use leverage and do one wrong thing along the way, it could wipe you out, because anything times zero is zero.  But it's reinforcing when the people around you are doing it successfully, you're doing it successfully, and it's a lot like Cinderella at the ball.  I mean you know at midnight everything is going to turn to pumpkins and mice; right?  But if the evening goes along, I mean, you know, the guys look better all the time, the music sounds better, it's more and more fun, you think why the hell should I leave at quarter of 12.  I'll leave at two minutes to 12.  But the trouble is, there are no clocks on the wall.  And everybody thinks they're going to leave at two minutes to 12.

Charlie Rose:  
And should wise people have known better?

Warren Buffett:  
People should always know better.

Charlie Rose:  
Yeah.

Warren Buffett:  
I mean people -- people don't get -- they don't get smarter about things that get as basic as greed and you can't stand to see your neighbor getting rich.  You know you're smarter than he is, and he's doing these things, you know, and he's getting rich, and your spouse is getting unhappy with you because you aren't doing -- pretty soon you start doing it.  And so you get what I call the natural progression, the three Is.  The innovators, the imitators, and the idiots.  And that's what happens.  Everybody just kind of goes along.  And you look kind of silly if you disagree.  I mean, you know, you could have these crazy Internet valuations in the late 1990s, but they prove themselves out in the market.  The next day they were selling for more than they were the day before, and people said, you know, you're crazy if you don't get in on this.  So it's very human.  Now, with housing it's something even more dramatic than that, because most people aspire to own their own home.  And if you really think that houses prices are going to go up next year and the year after, you feel if I don't buy it this year, I'm going to have to buy it next year.  That's not true of an Internet stock.  But it's true of a home.  And when somebody makes it very easy for you to do it by saying you don't really have to put up my money, you can lie about your income a little, or we'll give you 100 percent mortgage, you're going to do it, because everybody that's done it has been proven right.  You have what they call social tools, and, you know, you're going to feel like an idiot if you didn't do it, because the house cost more.

Warren Buffett:  
Oh, I think confidence will come back.  I will tell you this.  This country is going -- be living better ten years from now than it is now.  It will be living better in 20 years from now than ten years from now.  The ingredients that made this country, you know, the miracle of the world -- I mean we had a seven for one improvement in the average American standard of living in the 20th century.  Now, we had the great depression, we had two world wars, we had the flu epidemic.  You know, we had oil shock.  You know, we had all these terrible things happen.  But something about the American system unleashed more and of a potential to human beings over that hundred years so that we had a seven for one improvement in -- there's never been any -- I mean, you have centuries where if you've got a 1 percent improvement, then it's something.  So we've got a great system.  And we've got more productive capacity now than we ever have.  The American worker is more productive than he's ever been.  We've got more people to do it.  We've got all the ingredients for a sensational future.  It's just that right now the athlete's on the floor.  But we -- this is a super athlete.

Traveling towards VMworld 2008

I, along with thousands of others, wended my way through a vast dimly lit cavern of a place helped along by the strangely surreal sight of ushers in black waving wispy red flags to guide us not to the empty seats in front of us, but to the ones 50 yards on. (Ah Vegas, my feet hurt already.) Perhaps the point was to live in the moment, soak in the pre-rock concert atmosphere complete with a hip and cool soundtrack ripped off from Apple commercials. (Do they all use the same ad firm?) A better way to build the anticipation for, yes, the kickoff keynote session at VMworld 2008. (photo credit: lodev)

To the sounds of Hey Ya (Shake it like a Polaroid picture), we shifted forward in our uncomfortable temporary seating placed, as at all tech conferences, too close for all but the skinny girls. The moment was here – one of those videos started playing on the dozen or so huge monitors floating above the convention crowd. You know this video; you’ve probably seen it before from HP or someone like that. One of those videos with instrumental Coldplay music in the background with time lapse/speeded-up video of people in motion and floating captions dropping into the images that leave you with a slight smile on your face as you “get” the relationship between image and text. (Do they all use the same ad firm?)

And here he is, announced like a Vegas headliner, Paul Maritz, the new CEO of VMware. Hmm. After all that hype, I rather expected someone in a black turtleneck and jeans to come out. Instead here’s this guy with pleat-front pants and an admittedly cool accent (New Zealand?) who looks a little like Al from Home Improvement. Not that there’s anything wrong with that – everyone likes Al.

And then the real fun begins.

• 30 years ago, Paul Maritz started off his business career as a developer
• 10 years ago, VMware was founded by Diane Greene and Mendel Rosenblum (BTW, 10 seconds spent showing a slide with cartoon-ized images of the founders, “thanks for what you did for the company for the past 10 years”. 10 seconds after 10 years…but maybe more would have been hypocritical…)
• a retrospective of centralized vs. decentralized computing initiatives from the 1960’s to today
• of course VMware milestones from 1998 to today
• and then an analyst-ready diagram showing the product roadmap (to be delivered in 2009) with, you guessed it, finally a connection between VMware and cloud computing (remember Maritz’s cloud-computing company was bought by EMC just a couple of years ago and that’s the section he headed up at EMC before being brought into VMware).

Forward Looking

2008 (and probably much of 2009) will be a very busy year for VMware. If you believe the roadmap, VMware seems to be taking on the management of everything – from chargeback and capacity planning to virtual storage and virtual networking (more to come on just what the planned vStorage and vNetwork will deliver) – but all of it VMware-centric. As we said in an earlier post, they’ve moved away from “defending” the hypervisor business proposition to focusing on management services on top of their own hypervisor platform. Revenue pressures must be excruciating – who wants to be a public company these days?

The best part of that new “Virtual Data Center Operating System” diagram/roadmap was the addition (and I mean addition) of something called Cloud vServices. (Did anyone else find it odd that Cloud vServices is kind of on its own in the Infrastructure vServices area? AND, I’ll have to get the other version of the diagram/roadmap I actually saw at the show because that one shows an inexplicable 4^th box in the Application vServices area titled “…”. Really. Maybe to balance out the addition of Cloud vServices?)

What was clear is that the move from VirtualCenter to vCenter –and the new vServices for rolled-up management of virtualization components/capability to span multiple VirtualCenters (or future vCenters) for reporting, monitoring and management at scale – has been in the works for a bit (but in tech time, that could mean 6 months), but the cloud stuff…not so much.

Beyond the very high-level speak appropriate to a keynote (100+ service provider partners for off-premise cloud…suspended VM’s that you don’t have to pay for until you need it), the details are uber-fuzzy. There was a session that Dave went to which was supposed to shed more light, but when questions were asked about how it really works, the answers seemed to be TBD. Does anyone know more? If VMware really has figured out practical cloud computing for enterprises, kudos to them. But I fear they’re like everyone else (except maybe AT&T) and are still working out the details.

IBM’s presentation was cleverly titled 2mor0@Wrk - Tomororow work and Web 2.0.

Overview

Web 2.0 is delivering a whole different paradigm of communication. The slide is Lotus Symphony - NOT PPT. Over 2 million downloads.

There is an information overload that impacts individual productivity in the workplace. It has a profound effect on organizational productivity. A more complex organization entity provides more pressure and more inefficiencies in workplace. Up to 70% of time can be used looking for the WRONG information.

Collaboration mitigates information overload. It allows you to identify experts and opinions.

The collaboration agenda. Enterprises are at the onset of exploring these features. Web 2.0 is giving us the capacity to do more. Collaboration optimizes business outcomes - global, secure and dynamic.The most progressive companies are looking at UNIFIED COMMUNICATIONS. Making sure that directories and profiles are fully mobile.

Collaboration should be a contextual part of the workflow, going directly into applications.

IBM’s collaboration strategy is to deliver these services through online or offline services.

Demonstration

Executive IT architect Ron Sebastian provided a demonstration of IBM’s collaboration strategy. IBM’s Web 2.0 solutions span delivery platforms:

• Platform - web as  platform
• Application - development
• People - social computing

Lotus Connections - a family of social computing software that provides profile lookup and community capabilities. Think of Facebook, Yahoo Groups, and delicious combined in one portal.

Ron demonstrated these social services embedded into a healthcare provider application. Semantic tagging is available, contact information and commenting. Not only are we providing service to customers, you can integrate sync capability to directly call the person you want.

The biggest aspect of Lotus Connection? It’s all integrated.

A new service - Project Bluehouse. This is a SaaS delivery of these collaborated capabilities. The store and share can manage and share documents within and outside the company. Access control is no longer an issue.

Collaborative Web 2.0 services available as standalone products that also work in a mobile environment.

Case Study: Natural Disaster Management Mashup

Boeing came up with twenty different scenarios that they could handle through their systems. The problem was the one they didn’t count on. One example was Katrina - how to deliver supplies to the area: what airports were open? Where could they land? The problem was they could not find one list of public, private and military airports, nor what was open. The mashup took different feeds to allow the deacon maker to make a more rapid and intelligent decision based on information on where they could fly in the appropriate supplies. From open information sites like AirNav.com and personal contacts, users were able to mashup the information to make better decisions.

Conclusion

IBM announced the IBM Center for Social Software, proving their commitment to connect, collaborate, and innovate. Users and academics can work together to how these innovations can be applied to businesses and provide value to the market.

There has been some question of whether or not IBM can pull this off and move into the collaborative Web 2.0 market. Despite some criticism, it looks like IBM has really taken a step forward in advancing their products and services to meet market needs.

People drive better business outcomes. Connecting, collaboration, and innovation is key. Having the right tools and information to do that eases pressure that many organizations feel and brings Web 2.0 technologies to the heart of businesses.

The session took place at the beginning of Partner Day. The “regular” conference sessions actually begin tomorrow. Today is spent focusing on partner issues and enablement.

The panel for this session included:

• Mark Chuang Group Manager, Product Marketing, VMware, Inc.
• Kenon Owens Staff Systems Engineer, VMware, Inc.

You have to remember that most of the Partners here are not vendors like ScienceLogic, but big and small shops that are selling IT, networking and now virtualization solutions into end-customer environments. For these guys, understanding what virtualization partner programs and tools are at NetApp, for example, is very useful. And many of these companies are already selling Microsoft software and surrounding services for Microsoft products. So if you’re VMware, what’s the message to these partners in the face of the Microsoft juggernaut?

Microsoft to partners: “You may not like to admit it, but you’re probably already in bed with us.”

VMware to partners: “Our hypervisor technology outperforms Hyper-V and Xen, especially at scale. And anyway, it’s not about the battle at the hypervisor. It’s about the V-services on top of the hypervisor – VMotion, Storage VMotion, DRS, etc.”

Interesting and what we all already know, or think we know. The scale issue is an interesting one – too soon for Hyper-V and who uses Xen? But also interestingly enough, no announcement or even talk about extending VMware management tools to other hypervisors. The point, as the VMware product marketing guy made a point of saying, is that the question they needed to answer used to be “Why Virtualization?” and now it’s “Why VMware?”.

One more tidbit – this survey run by VMware asking their customers:

What are the top 6 apps you are running on VMware today

• IIS
• Apache
• Active Directory
• SQL Server
• Sharepoint
• Exchange

That means, 5 of 6 are Microsoft applications. Certainly it makes it even more challenging for VMware to navigate a path here.

The change since 2004 – would have talked about why virtualize. And now why VMware. (Duh.)

Talking to partners – many of which already have a successful Microsoft business. How VMware enhances your existing Microsoft business.

Top 6 apps running on VMware today (5 of 6 are Microsoft applications)

• IIS
• Apache
• AD
• Sql server
• Sharepoint
• Exchange

Source: VMware survey

Esxi - VMware – true thin hypervisor; maximizes resources utilization (over 100% memory commitment – allows avg of 2:1 memory overcommit) – host system memory is usually the resource bottleneck – plus Advanced Scheduler runs VMs better under load and to a greater capacity (hard to show this part); performance acceleration – using binary translation (32bit), para-virtualization and Hardware Assist (for 64-bit)

(rvi – rapid virtualization indexing)

No parent partition that all hypervisors have to go through

Vs ms/xen

Parent partition – dom 0 => potentially problem at scale; i/o that could be a bottleneck

Hyper-v SPECjbb comparison

= 9 vms on VMware and hyper-v hypervisors

Outperform (CPU) by 50% - general purpose scheduler isn’t able to keep up? “got to be”

(cpu only test)

Also used VMmark – to demonstrate again that VMware is performance tuned and designed to run at scale vs Hyper-V

Size Does Matter:

Vmware ESXi: 32MB

Hyper-v – 2.6 GB

Xen – 1.2 GB

Hyper-V uses Microsoft Server Core – so the last two Patch Tuesdays had to make changes to Server Core (nothing to do with Hyper-V) but service interruption for Hyper-V.

VMware VMsafe – “Provides an unprecedented level of security” “virtual is more secure than Real” (uh oh – clearly didn’t read about the

*****************

VMware TEST:512 mb vms on server w/ 4gb ram –

7 vms - xensource (w/no memory overcommit)

6vms – hyper-v before error (w/no memory overcommit)

14vms - w/memory overcommit and management

Running sql io sim – heavy workloads

TCO – not just license; now ESXi is free – so hardware

809 - ESXi

871 – vi3 foundation ($995)

1168- vi3 enterprise ($5750)

1621 – hyper-v – 2x cost because of hw

Xen – 1618

Memory overcommit (89% in production vs. test/dev)

Survey – 37% of respondents at 2:1 RATIO OR HIGHER; real average is around 1.8: 1

*********************

This guy Mark sounds like a used car salesman:

“Always On, On Demand Data Center”

Hypervisor is very important but what is more important are the v-services on top of this. Manage shared, pooled resources. “Value Above the Hypervisor”

How does all this save “your customers” $$?

VMotion – saves cost on planned maintenance: no more overtime, no more time scheduling maintenance windows (see cost framework below)

10 (# of servers) x 6 (@ of updates) x [ (overtime cost 2hrs x $150/hr) + (scheduling downtime # of apps per server 15 x time spend scheduling per app 0.75 hr x $50/hr)] = $58,500

Same thing with using VMware Storage VMotion

Overtime cost + scheduling downtime + planning move + alternative tool cost - $68,750 (2.5 TeraBytes)

The Value of High Availability

- cost of lost business, lost work

- cost of lost productive time

4 hours of downtime x # of users per vm 10 x number of vms per host 15 x cost of user productive time $50/hr x failures per year in 10-host cluster 2 = $60K

(10 servers, 150 vms)

SAVINGS (using enterprise version)

Update management 149,760

HA 60K

DRS, VMotion Storage VMotion 187,250

808,259 – hw, power cooling, etc.