Before we delve into the intricacies of individual methods of anti-debugging let’s use this post to define the classes of anti-debugging that we will be discussing. While other classes may exist, the definition of these classes is an attempt to include the majority of anti-debugging methods in use today. There is some overlap between classifications and we may have left out some methods due to limited exposure or effectiveness.

API Based Anti-Debugging
API based anti-debugging is the most straightforward and possibly the easiest to understand for a typical developer. Using both documented and undocumented API calls, these methods query process and system information to determine the existence or operation of a debugger. From single line calls such as IsDebuggerPresent() and CheckRemoteDebugger() to slightly more complex methods including debugger detaching and CloseHandle() checks. These methods are generally trivial to add to an existing code base and many can even be implemented in as few as two or three lines.

Exception Based Anti-Debugging
Exception based anti-debugging is slightly different than your basic API based techniques. Many times when a debugger is attached to a process, exceptions are trapped and handled by the debugger without regard to passing the exception back to the application for continued execution. Occasionally these exceptions can even crash or terminate a process when run under a debugger and be handled gracefully when running clean. It is these discrepancies that makes exception based anti-debugging techniques possible.

Process and Thread Block Anti-Debugging
Some of the API based anti-debugging methods use published functions to query information from within the process and thread blocks for our running code. Many API based detections can be subverted within a debugger by hooking the API call and returning values that indicate a clean process. One way around this subversion is to directly query the process and thread blocks, bypassing the API calls. Direct analysis of the process and thread blocks, while more complex, can lead to a more accurate and high assurance result.

Modified Code Anti-Debugging
One of the methods that a debugger uses to signal a breakpoint is to insert a break byte into the running code at the location that it wishes to stop execution. The process execution breaks when this value is seen, giving control to the debugger. When the program is resumed, the breakpoint value is removed and replaced with the original byte, the execution backed up one byte, and the program is resumed. Detection of software based breakpoints can be achieved by analyzing the process for modifications from the expected norm.

Hardware and Register Based Anti-Debugging
A second way that a debugger can break the execution of a process is by using a hardware breakpoint. A hardware breakpoint relies upon CPU registers to store the pertinent information and to detect when the target break addresses are seen on the bus. A break interrupt is triggered at the appropriate time based on these register values. Reading or modifying the hardware can allow for the detection of a debugger.

Timing and Latency Anti-Debugging
Finally timing and latency can be used as an effective anti-debugging method. When executing a program within a debugger, specifically when single stepping, a much larger latency occurs between execution of instructions. This latency can be detected and compared against a reasonable threshold to detect the existence of a debugger attached to our process.

Each of the classes of anti-debugging outlined above has merit when used individually to protect a process. While none of them can be assured to ever protect a program from a determined reverse engineer or debugger, implementation of these techniques (or many of them if appropriate) can sufficiently slow down the debugging process and hopefully make the attacker spend his time on other, easier, ventures. In the remainder of this series on anti-debugging we will review in depth some of the more interesting methods of each of the above classes. So bring along your debugger and your development environment and let the games begin.

So I was very interested when I heard that Opus Interactive, a customer of ours, had “joined the VMware vCloud initiative as a VMware Service Provider”. I talked to Eric Hulbert, CTO of Opus Interactive, to get some details directly from the source.

Eric shared our own caution about making “cloud-ready” announcements. There have simply been too many companies talking about cloud solutions that lack any substance – usually based on definitions of cloud computing that are hazy or just too broad. The backlash against the cloud hype is often quite justified. But in Opus’ case, there are real components that if they don’t add up to a “full” cloud computing solution just yet, are well on their way – and enabled by VMware’s program for service providers (VSPP).

Opus Interactive is serious about virtualization, which is an indispensable tool in their stated goal of creating a high-density micro-data center with the smallest footprint possible. They are 100% wind-powered and have already virtualized much of their data center, reducing the amount of hardware necessary to run the business and driving down costs to produce even more competitive advantage in a crowded marketplace.

VSPP for vCloud provides a rental model of VMware licenses – e.g., for Enterprise ESX or VDI. VMware Service Providers report on their customers’ virtual machines (vm) and pay only for what is actually used. This model lets Opus Interactive quickly spin up a vm to get a new customer up and running in about an hour and stay very cost competitive at the same time; Opus offers their vClustr entry-level virtual server for only $99.

Cost-effective, rapidly scalable computing “on-demand” based on shared resources, managed by “expert” third-parties, enabled by virtualization technology and pay-per-use vm licenses. Cloud computing? Instead of thinking about a single definition of cloud computing, perhaps it’s more relevant as the market matures to think about a continuum of cloud computing. And by that definition, Opus Interactive is providing cloud services, enabled by VMware’s VSP program. Next on the schedule, automated provisioning and perhaps in the future, API’s that make it even easier for application developers to test and deploy apps on Opus Interactive’s cloud platform – which, by the way, uses EM7 for its core management solution.

The analysts paired up today:
Antonio Piraino (Tier1 Research)
William Fellows (The 451 Group)

My usual disclaimers on live-blogging: doesn’t include everything covered (just what was most interesting to me) and had to paraphrase some answers because I simply cannot type that fast.

Quick definition of Cloud Computing
WF: The cloud is a continuum of grid, virtualization and utility done right. It is about provisioning services instead of servers; flexible computing instead of fixed assets. Done right, the cloud abstracts users from the complexity of grid. Cloud computing is IT as a service. Cloud computing is the Third Way – not entirely in-house or outsourced, but an optimized hybridized version of both. In light of the Goldman Sachs report out resetting IT spending forecast from up 6% to down 1%, don’t underestimate the ability for enterprises to move from capex to opex by buying cloud computing instead of building it themselves.

The 451 Group conducted a survey on cloud computing in March, and then revisited it a month ago. Some interesting results:

• 84% have no plans to develop an internal cloud. 5% had no answer to this question. And for the 10% who did answer – the uses for a private/internal cloud were the same as those for a public cloud.
• Top 6 vendors they look to help them develop an internal cloud: Microsoft, IBM, Cisco, HP, Oracle, VMware

Is it all “upside” when it comes to cloud computing?

WF: Watch out for the Trojan horse, the red flag. What about the software needed to manage all this stuff? Any management software needs to take a holistic approach to solve the problem.

AP: Increased management requirements and capability – this is actually a great story for managed hosters who can hold your hand while getting you up into the cloud. Hosters alleviate the pain points, and this is why we’re going to see continued growth and focus in the managed hosting sector.

WF: I would argue that they’re too expensive. Look at Amazon – 10 cents a hit adds up.

AP: It’s almost impossible to do an apples-to-apples comparison between cloud providers. One reason is that they charge differently. I’d say that when you’re talking about the big cloud providers, you are right – that they are expensive over the long-term, but for use in the short-term, they can be optimal.

WF: The cloud is setting big expectations. Can IT deliver? It’s nice to talk about “shared resources for the greater good” but in any organization, you will still run into issues of power and control! Plus it’s still early days for resolution of regulatory issues and compliance around the cloud.

Final Thoughts

AP: Think of the opportunities of using cloud computing resources in the areas of testing and pre-production – short-term use/environment (quick up/quick down), inexpensive, opex not capex. We’re already seeing the cloud fostering much innovation.

WF: “It’s okay to fall in love with the term.” It is real but keep the expectations lower and realistic.

AP: I agree with you. The reality is that the cloud is driving a very fundamental underlying platform change. This is not just a term or something that will fall out of fashion. There’s a real need to build trust in the cloud and leveraging shared resources in this way – so use the cloud computing term cautiously; don’t abuse it and make the cloud seem like IT’s new toy.

Here we talk to John Proctor, who started out as one of our first customers (and the first MSP customer). And he believed in it so much, he eventually became part of the ScienceLogic team. (Remember “I’m not only the President, I’m also a client” from the Hair Club for Men?)

John shares his perspectives about the service provider world and why he took a chance on a little-known product called EM7.

ScienceLogic: What is your background? How many years have you worked as a service provider and for what types of companies?

John Proctor: I have been working with Service providers for over twelve years. I worked at a major regional service provider for six years and before that I designed and built national and international networks for ISP’s and Fortune 500 companies as a consultant for PriceWaterhouseCoopers and WorldComm.

ScienceLogic: You were one of the first customers of EM7 – why did you choose it and how did you get over the hurdles associated with using a start-up company’s product?

John Proctor: We were actually customer number five. Back in 2004 when we evaluated and purchased EM7 we could see that EM7 provided about 80% of what we were looking for in one integrated solution right out of the box. One of the things that sold us on EM7 was that the ScienceLogic founders had all previously worked for a service provider, so we knew they understood our business and our challenges. But in the end, it comes down to features. Once we compared EM7 functionality to the alternatives, it was clearly a “no brainer.”

ScienceLogic: What other alternatives were being considered?

John Proctor: Well, we had started with a few point solutions, but as our business and product offerings matured, this resulted in a growing number of point solutions. What started with 3 or 4 ended up as 14 separate tools. They all had strengths but what they didn’t have was integration and because of this they could not scale. And, if the tools could not scale, our business could not grow.

So, naturally we started looking at framework solutions, but they are expensive to buy, expensive to implement, and expensive to maintain. At one point, we even considered some open source projects. There were several that showed promise, but we would still be stuck with tools that were not integrated. So then we considered hiring developers to cobble something together that would work for our business. The only problem with this alternative was that we felt it would take 6 to 8 months before we could have something viable to work with.

ScienceLogic: What products were you using before EM7? What were your goals?

John Proctor: Before we purchased EM7 we used 14 different point solutions to deliver our products and services to the marketplace. Tools like NetCool, Openview, Argent, Heat, What’s Up Gold as well as several other point solutions, vendor specific applications and manually updated spreadsheets. And, as I mentioned before, this does not scale. This also adds a great deal of complexity when you begin to consider business continuity and disaster recovery. All these tools were vital to the delivery of our products and services. Any service provider will tell you it is all about uptime. So if the product is uptime, the tools used to deliver it have to be available 24×7x365.

Our goals were simple: scale and redundancy. As it turns out, the solution was simple as well. EM7 provided a tool that could replace the functionality of almost half of the existing point solutions and the applications that could not be replaced were integrated with EM7 to provide our staff with a “single pane of glass” to see the status and performance of each area of the business from one application. We had visibility into everything from facility systems to applications using EM7.

ScienceLogic also delivers an extensible configuration that addressed uptime and redundancy. We deployed collectors throughout our network that reported back to a central pair of redundant database servers and with this configuration we were able to perform backups and add capacity without taking the system down.

ScienceLogic: Why are service providers different from enterprises? How are their needs different?

John Proctor: First and foremost, service providers face the same challenges that only the largest enterprises ever face and they also have many unique challenges that only service providers experience.

One challenge we faced was that we had multiple datacenters in different states. They were all interconnected with plenty of bandwidth between each site, but the tools were not designed to be used across the WAN. Our staff in our remote data center did not have the same access as our staff in the corporate office. Since EM7 is web-based, it immediately eliminated this problem.

Another challenge is that service providers must manage systems across multiple domains. Back in the early version of a specific tool we were using before EM7, the only way you could implement it across multiple domains was to put the same username and password on every computer that you monitored. Beyond the security concerns, maintenance was a nightmare. Anytime we had to change the password, we would get locked out of dozens upon dozens of systems. When the password was changed on the monitoring server, it would attempt to login to the remote machines and fail. Repeated attempts would result in the account getting locked. I think that vendor eventually addressed this issue, but service providers seldom find tools that were designed for their unique situations.

ScienceLogic: How is EM7 geared to service providers?

John Proctor: Enterprise IT is a trusted part of the business; they are one of the team. Service providers are outsiders that must earn trust by showing the customer exactly what they are doing.

EM7 provides a multi-tenant environment that allows service providers to manage systems across many different customers while at the same time providing the customer access to see the same information but only what’s relevant to them.

EM7 was built by service providers and even includes a few features just for them. Two of my favorites are bandwidth billing and the emergency notification system. Take bandwidth billing, for instance. EM7 provides a way to collect bandwidth utilization, store subscription information, and calculate a bill from any one of about 10 different methodologies. And at the end of the billing period, EM7 sends the completed report out to whomever you chose via email.

Another unique service provider feature is the emergency notification system. EM7 allows the provider to track what customers used their unique infrastructure components. If they have to perform maintenance on the infrastructure component or have a problem they can send an email to all of the impacted customers in a matter of minutes.

ScienceLogic: What trends do you see for service providers? What about big trends such as virtualization and cloud computing – how will they impact service providers?

John Proctor: Virtualization is really hot for service providers right now and for the same reasons as in the enterprise. Service providers run data centers and data centers must be powered and cooled. So, anytime they can use a virtual server instead of adding physical equipment it is a good thing. But then you add the complexity that multiple customers reside on the same host and you must track things like bandwidth utilizations by guest OS, and it all gets a little harder. Lucky for us this is not a problem for EM7.

I still think it’s early days for cloud computing. Depending on who you talk to, much of what service providers (especially the big ones) have already been doing with SAAS offerings and hosted applications could be described as cloud computing already. In which case, service providers are ahead of the game. But whatever the “final” definition, cloud computing actually shares many similarities with virtualization – in that service providers (or enterprises) will need to be able to manage far more “devices” in real-time with “zero downtime” expectations by customers. What this really means is that you’re going to see much more automation in provisioning and IT monitoring tools to handle the scale and speed with which things can change in the data center given vm migration and the talked-about switching between “clouds” that can be used for high availability.

By all accounts the show did go on, and we have some very interesting results to share with you all.

Take the Top Challenges question. Once again, “Supporting New Technologies/Enabling Innovation” was most popular. But that’s a no-brainer and as one memorable respondent told me, “the definition of what I do”. What was more important was seeing the big jump that “Reducing Management Costs” made on the list, from #5 last year to #2 this year and only 1 percentage point behind #1. Tightening the belt is top of mind for everyone. (As I write, the Dow closed down today over 700 points)

Overall, IT professionals told us they were tackling the practical projects that should and could get done – from deploying Security Information Management solutions to getting Asset Management and Inventory Tools in place. For the first time, we saw a close correlation between what people said was important and what actually got done. Of low importance and even lower actual deployments – ITIL and CMDB, IPv6, Green IT and Cloud Computing.

And perhaps people “fessed” up about virtualization. Instead of the usual “high importance, not so many deployments now, but more deployments planned” theme we’ve been seeing around virtualization adoption, this year the very hot trend seemed to lose a bit of steam. Across the board, the numbers were down for virtualization management, with close to 50% of respondents telling us that their businesses were less than 10% virtualized (4% of that with no virtualization at all).

2008 Detailed Results – showing trends year over year

Comparison of Results from Interop NY 2008 vs FOSE 2008 (government IT)

John talked about Microsoft’s mission, his perspectives on key industry trends and market opportunity; he touched on Cloud Computing and Virtualization and took some Q&A from the audience of Managed Service Provider executives.

One of his first proclamations - Microsoft has really embraced the heterogeneous environment. Really? How in the world is Microsoft going to help convince IT line managers, or mid level managers to believe this statement? I think they have a long way to go to achieve this vision with any credibility in the marketplace.  I do know that they are making small strides.

Microsoft has been widely credited with some very good blogs that are self critical and introspective. They have also been quite active in the standards boards within DMTF and many others such as Open WSMAN and CIMON (Open Pegasus). Microsoft in February published 30,000 pages detailed technical specifications – protocol documentation for Exchange, since that time they have published another 15,000 pages. They have had over 224,000 downloads since February 21, 2008. Thus they are trying to be more open by making some of these secret sauce protocol resources directly available on the web.

So for now, I will take a very cautious wait and see approach to this proclamation. Time will tell.

Trends

• Rapid growth continues
• Hosting Competition has a new face
  • Platform gorillas (amazooglesoft)
  • Ad supported Web 2.0 hosters (Google, Facebook,)
• Utility Cloud Computing models are expanding to non-traditional hosting companies
  • Wells Fargo vSafe - hard to believe that a big bank would start to offer a SaaS offering
  • New tools and markets digital ribbon, CohesiveIT

IDC Data shows that growth of SaaS ISV’s is the biggest layer of growth. The fastest growing services are complex, custom applications. IDC says this area will be bigger than the hosting area in the next 5 years. John said that Microsoft is spending a lot of time, money and energy on this right now.

John said:

“when Microsoft thinks about the building blocks that make-up the cloud, virtualization is a core piece of the puzzle. However you also need also identity services, Operating system with standard set of libraries to tap into… or remote storage that application developers will tap into.. Developers will consume these set of services, but you will also need a set of tools to manage your physical, virtual and geographically distributed datacenter infrastructure.” (that is where ScienceLogic comes in!!)

He went on to say,

“In some ways, virtualization enables decentralization – allows you to move from data centers, enables fast scaling out, business to move from on premise to the cloud and off again…. Automation is very important – this will help you scale your business – this is core to your future success.”

He talked about a new breed of knowledge worker: He called them Digital Natives (compared to grey haired guys like me who are left out of this category).

Definition of a Digital natives? A young adult who has grown up with cellphone, web based applications, Facebook account, as their primary mode of communications.

John commented that we are 5 years into a 10 year journey. Only 12% of all servers in the world are virtualized today… in the next 4 years it will double to 25%. This is the time to think through how this business will affect you.

‘Virtualization without good management is more dangerous than not using virtualization in the first place.” Thomas Bittman, Analyst Gartner

Patching and provisioning nightmare – no scalable administration – sprawl chaos.

John posed a question to the audience: How do you partner to provide the ISV support in application development with specific market needs… partner by keeping the hosting to SaaS solution providers up and running and provide the quality of service that their customers expect…. Complimentary services of storage and backup is a big win with a huge market-upside over the next 5 years..

John said that Microsoft continues to make  huge investments with Managed Service Providers.

• Investing in the windows hosting platform
• Hyper V and SQL2008 GoLive program - getting beta code out to service provides to find as many bugs as early as possible.
• Software + Services (S+S) incubation center program
• Partnering for cloud platform market offers
• Cloud platform guidance and best practices

During the Q&A, David Burns from Cincinnati Bell asked the very best question… “when are you going to make it easier for the Service Provider market to deal with the Microsoft Service Provider Licensing Agreement (SPLA) quarterly statistics pull and change the SPLA pricing to be more efficient and creative for the new Virtualization and Cloud offerings you have talked about?”

John’s response: “We hear your frustrations loud and clear and are working on some new ideas for the future version of SPLA.” My interpretation – “Dear Service Providers don’t expect anything new or easier to deal with in the next 6 months!”

His closing remarks: “Cloud is evolving = very early stages, lots of hype, but think of how this evolution will effect your business and how you can plug into it.”