In this interview, cloudsecurity.org talks to Guido van Rossum about Python, Google App Engine and security.

Guido is the creator of the Python programming language and more recently, Google App Engine team member.  His involvement with the App Engine project was pretty late - the code “was almost ready for release” when he get involved.  The security architect of App Engine was primarily project lead, Kevin Gibbs, supported by the rest of the App Engine crew and the Google Security Team.

The Interview

cloudsecurity.org: What security principles did you follow for App Engine?

GvR: While I can’t share any specifics on what we’re doing to secure App Engine, I can say that the main principle we’ve followed could be called “defense in depth”. We’re not relying exclusively on a secure interpreter, or any other single security layer, to protect our users.

cloudsecurity.org: Please provide some examples of how those principles played out in terms of the current implementation?

GvR: Sorry, we don’t divulge such information.

cloudsecurity.org: What criteria did you apply to Python module selection?

GvR: We first looked for modules that were useful and straightforward to audit. If a module was large or complex, we’d only audit it (fixing things we found) if it was deemed essential or at least useful for a large number of users; otherwise we’d exclude it.

cloudsecurity.org: What do you see as the security risks inherent in exposing an interpreter runtime in a shared environment?

GvR: I presume you’re asking about risks to users, like providing accidental access to data belonging to another app. We’ve taken extensive measures to isolate different apps from each other. For example, each app runs in a separate process, and the datastore prevents an app from accessing data belonging to other apps.

cloudsecurity.org: I recently attended a fascinating talk by Justin Ferguson (a Seattle based security consultant) at eusecwest in London.  He gave a great talk exploring security vulnerabilities in language interpreters and specifically highlighted some security weaknesses in Python App Engine.  What are your thoughts on his research and specifically the Python issues he highlighted?  When do you anticipate they will get fixed?

GvR: We’ve anticipated all of the possibilities raised in Justin’s talk, and took measures to protect our users. Justin highlighted weaknesses in Python, but not in App Engine. Furthermore, our security model does not rely solely upon protections within the Python interpreter; there are additional protections that these external analyses have missed.

cloudsecurity.org: How do you contain an attacker that exploits bugs in App Engine from exploiting the underlying OS and potentially interfering with other users processes or attacking backend systems?

GvR: You are correct that there are strong measures in place, but I’m not at liberty to discuss details.

cloudsecurity.org: Python was the first language to get the App Engine treatment, what language is next and what are some of the language specific security challenges the team has had to deal with?

GvR: Although I can’t comment on what language is next, we are working on this, and have gotten a lot of great feedback from our developers. As far as language-specific security challenges, they stemmed mostly from the complexity of the Python interpreter. We spent a lot of time auditing this, and did a great deal more than just identifying buffer overflows.  I can also add that Google is actively researching the security of interpreted languages.  Google engineers routinely contribute security fixes to open source projects, including but not limited to Python.

cloudsecurity.org: How does the team decide when ‘enough is enough’ in terms of hardening the interpreter?

GvR: That’s not really how we approach it. We realize that security is an ongoing effort, and try to stay ahead of threats through continuous monitoring and testing.

cloudsecurity.org: Some commentators have suggested that perhaps the difficulty of auditing the implementation led to some modules being more heavily restricted than perhaps necessary.  What are your thoughts on that and what plans, if any, are there to bring back code objects/functions that were eliminated in the initial release?  (with the benefit of hindsight).

GvR: The only thing we are likely to put back is the _ast module, which was not audited based upon an underestimation of its usefulness (see my answer to question #3 above).  We will also put back some dummy functions and other objects whose absence currently prevents some popular frameworks from being loaded without modifications. For example, some harmless functionality in the imp module will come back. We’re also looking into making urllib2 work (to some extent), though that’s not really a security issue but merely a matter of API adjustment.

cloudsecurity.org: It is reported that Google encourages small groups to go off and create.  How involved were the Google security team with App Engine in terms of design and implementation review/testing?  Given the dynamics, is it possible to have a meaningful security process that shadows the development process?

GvR: The Google Security team is involved in everything we do. They have been extremely helpful.

cloudsecurity.org: How can people report security weaknesses they discover in App Engine?  What commitment does Google give in terms of dealing vulnerability reports?

GvR: There is a standard process for submitting security issues. See http://www.google.com/corporate/security.html. Google moves very fast to protect its users when a verifiable security vulnerability is reported.

cloudsecurity.org: One concern is the potential misuse of App Engine to exploit security vulnerabilities in visitors browsers.  This is not a new problem per se, shared hosting providers know all about this.  But with Google and other Cloud providers, the scalability potential is much higher.  What are your thoughts on this and what pro-active steps is Google taking to detect and terminate evil apps?

GvR: This is high on our list of concerns. We deal with this through a combination of restrictions on what you can do (e.g. certain HTTP headers and ports are off-limits) and, again, monitoring.

cloudsecurity.org: Beyond App Engine, what role do you think Python will play in the Cloud both now and in the future?

GvR: Sorry, I’m not prone to philosophizing about the future.

cloudsecurity.org: Trust is often cited as a barrier to enterprise adoption of Cloud Computing.  What role do you personally think Google can play in building that trust?

GvR: I think trust is built up over a long period of experience. Our actions in terms of being open to our users will be the most important factor in establishing trust. Of course, Google’s reputation also helps: everybody understands that Google doesn’t want its name associated with a bad product.

cloudsecurity.org: Looking at the Cloud Computing landscape beyond Google, what are your thoughts on the current state of Cloud Computing and Security?

GvR: It’s obvious that Cloud Computing is only just taking off. The next few years will be very exciting.

cloudsecurity.org: Lastly, what are some of your favourite App Engine apps?

GvR: There are too many to enumerate. If you insist on a highlight, well, I like Rietveld (http://codereview.appspot.com), a tool for collaborative code review which I (largely) wrote myself. It is open source and includes some essential components from Mondrian, a similar internal tool which I created before I joined the App Engine team.

Thanks

My thanks to Guido for his time and sharing his views.

Research Director Patricia Adams and VP and Distinguished Analyst Ronnie Colville presented this thought provoking session. It seemed to echo what ScienceLogic has been talking about regarding our thinking around the practical ways to efficiently accomplish key tactical gains against your Configuration Management Data Base (CMDB) initiatives.

They started out with, what are the prerequisites to a successful CMDB implementation?

Garbage in = Garbage out

There is no miracle occurring in all of these new fancy framework tools; these complex databases are only as good as the trusted source of information inserted. You have to put a bunch of elbow grease into figuring out what to actually put in the CMDB.

So how do you define the metrics?

First you need to know where you are starting from – you will need to baseline the environment. Then baseline what your state is 3, 6, and 12 months after installing CMDB.

Next: break metrics down to 2 strategic areas:

1. Strategic
   1. Operational Costs
   2. Application performance
   3. Compliance - internal auditors doing analysis – keep track of their findings and incorporate into your elements for data gathering
2. Operational Metrics
   1. Changes unplanned (typically 80% unplanned or emergency)
   2. Changes withdrawn (how many changes were withdrawn / roll back)
   3. Application downtime (what did it cost from app being down)
   4. Server downtime (before and after)
   5. Tickets generated (before and after)

Having the data to show how you are performing makes it much easier to show why you need more budget to improve performance in specific areas. Having metrics allows IT managers to do marketing back to the business units about the value you are delivering.

Gartner said that from their Enterprise customers they often hear “I haven’t quantified the value yet”…That is not the right answer.

During the session, Gartner did a real-time wireless poll of the audience with some interesting questions:

What are the tools to build and populate your CMDB with IT services?

Focus of CMDB?

• Inventory 20%
• IT service relationships 68%
• Other 6%
• Don’t know 6%

Interesting to note, a very consistent set of information from year to year polling which equals a mature understanding of the CMDB’s role for analysis and decision process.

Have you heard of ITIL V.2 & V.3 and considered how it impacts this discussion?

ITIL is a process framework, it is not a technology automation framework. Just because something is pink ITIL certified does not mean that it will help at all with the automation of the process framework.

Gartner quantified the market as being about 2 years old this month. So the point here is we are in early days of this technology. The way they see it, the Large Enterprise/Framework vendors selling you is like a lock-in, but the interesting thing about CMDB is that the tools that you need to integrate and federate were only recently acquired, so the entire framework vendor integration and alignment story is mostly incomplete.

Gartner’s Evolution of the CMDB deployment

On average it takes 12 – 18 months to get up and running.

Through 2011 enterprise should recognize that any of the CMDB tools bought today may require significant upgrades to offer near real time service views to support decision support analytics.

Several items from this presentation jump out at me:

1. IT Organizations need to deploy tools that will help to automate the continuous collection of IT asset inventory, configuration and business impact analysis. That is a big gap that exists in the marketplace today… the speed at which information is collected and updated into the CMDB.
2. Investing too much into this immature market before the official standards are set and then adopted by the industry (estimated 18 months after final adoption) is quite risky.

The conclusion that I made from this presentation is that you are better off with our 80 – 20 rule around CMDB’s. Use a tool that will collect 80% of what you need to operate the business in 20% of the time it takes to deploy these heavy, less than automated framework tools!

ShareThis

We found that when we were in VMware VirtualCenter, we can add permissions via the inventory datastore & networks view but once we did that there was no easy way to view or delete the permissions within the same view. You need to go back and navigate the hosts/clusters view, one at a time, in order to view where these permissions showed up and if necessary delete/modify them one at a time as well, or check where that role is applied within the administration/roles view.

While this might work for small environments or for a couple of administrators, it absolutely wouldn’t work for large environments with hundreds of hosts or thousands of virtual machines or a complex resources structure with complex storage. Or what about environments with multiple administrators? One administrator makes a change to permissions, but the next administrator has no idea and a change to permissions here cascades through and impacts all VMs in that datacenter. Sounds like a good way to shoot yourself in the foot!

So is this a design flaw? Was the point of the “Add Permissions” feature for datastores and networks to prevent users from getting to those datastores/networks? Or was it to maybe give the appearance of ACL functionality? Or something like a poor man’s quota management? And if you’re going to let administrators add permissions in a view, why not let them view and delete just as in the other views?

Does anyone know why this feature is even available here for datastores and networks in VirtualCenter without really taking the feature all the way? Maybe I’m not seeing the forest for the trees at the moment but if you know or have used this, please do share…

ShareThis

First it was the EU looking at passing a law that would require bloggers to disclose their identity and affiliation. Now the AP is looking to enforce a new license that would require payments when a blogger puts an excerpt from an AP article in their blog.  My friend Kevin McLaughlin blogged on this over at Channel Web blog today. Basically the AP says that if you excerpt more than 5 words you need to start paying them fees.  Kevin reached out to me and I gave him my views on this one.

I think that it is a really short sighted move by the AP.  First of all it shows they really don't understand blogging.  Blogging is about taking an idea which often comes from another source and putting the bloggers own spin and ideas behind it. In this way topics are built on one blog at a time with each blogger adding a bit more to the conversation. Each additional blog on topic enriches those blogs and articles that preceded it.  As I said in the Channel Web article, it is like a jazz musician playing a riff on top of a line already laid down.

In real terms blogging on the AP content will only generate more views and interest in the AP content.  AP is just a dinosaur with this type of view and will soon go the way of dinosaurs if they try to enforce this. In the meantime bloggers can talk about an AP article, but don't link to it and don't excerpt from it. I suspect that the next thing is we will have a replay of the inbound links litigation we had 8 years ago.  In the meantime blogging will continue to march on with AP or not.

First it was the EU looking at passing a law that would require bloggers to disclose their identity and affiliation. Now the AP is looking to enforce a new license that would require payments when a blogger puts an excerpt from an AP article in their blog.  My friend Kevin McLaughlin blogged on this over at Channel Web blog today. Basically the AP says that if you excerpt more than 5 words you need to start paying them fees.  Kevin reached out to me and I gave him my views on this one.

I think that it is a really short sighted move by the AP.  First of all it shows they really don't understand blogging.  Blogging is about taking an idea which often comes from another source and putting the bloggers own spin and ideas behind it. In this way topics are built on one blog at a time with each blogger adding a bit more to the conversation. Each additional blog on topic enriches those blogs and articles that preceded it.  As I said in the Channel Web article, it is like a jazz musician playing a riff on top of a line already laid down.

In real terms blogging on the AP content will only generate more views and interest in the AP content.  AP is just a dinosaur with this type of view and will soon go the way of dinosaurs if they try to enforce this. In the meantime bloggers can talk about an AP article, but don't link to it and don't excerpt from it. I suspect that the next thing is we will have a replay of the inbound links litigation we had 8 years ago.  In the meantime blogging will continue to march on with AP or not.

There is a great article in Business Week this week that talks about a scam that bank and credit card companies are pulling on consumers.  It has resulted in the banks winning arbitration cases against consumers to the tune of a 99.998% clip.  That is right, 99.998%.  It has turned arbitration, where an impartial judge makes determination into the biggest home field advantage this side of the NBA play offs.

It seems many of the credit card agreements that govern your use of credit cards call for arbitration to settle any disputes between you and the credit card company.  Well the credit card company gets to pick the arbitration company. Many pick the National Arbitration Forum, which markets itself to the credit card companies as a form of collection agency.  The whole system is basically stacked against the consumer, which results in the credit card companies getting their way.  Business Week does a great job of digging in here and finding out all of the dirty secrets of this scam.  I highly recommend you read the article for all of the details.

I don't think too many people disagree that over the last years there has been a big swing in the pendulum favoring business's over the consumer. Many of the laws and rules that were put in place to protect consumers over the years have either been thrown out or ignored.  Our bankruptcy laws have been totally rewritten to the disadvantage of the consumer.  Lazes-fare attitudes toward regulating business has seen oil companies raking in billions of dollars a quarter while we pay 4 dollars a gallon.  Health insurance companies raising rates higher than inflation while hospitals have to close for not making enough money.  A mortgage industry that without oversight has written loans that has our finance system to the brink of disaster. A return of inflation and recession at the same time.

Not too advertise my own political views, but do I think it is time for a change?  Your damn right I do!  I hope that the press shining the light on some of these injustices will make it easier for a new era in Washington to make right (no pun intended) some of the wrongs in our system.

There is a great article in Business Week this week that talks about a scam that bank and credit card companies are pulling on consumers.  It has resulted in the banks winning arbitration cases against consumers to the tune of a 99.998% clip.  That is right, 99.998%.  It has turned arbitration, where an impartial judge makes determination into the biggest home field advantage this side of the NBA play offs.

It seems many of the credit card agreements that govern your use of credit cards call for arbitration to settle any disputes between you and the credit card company.  Well the credit card company gets to pick the arbitration company. Many pick the National Arbitration Forum, which markets itself to the credit card companies as a form of collection agency.  The whole system is basically stacked against the consumer, which results in the credit card companies getting their way.  Business Week does a great job of digging in here and finding out all of the dirty secrets of this scam.  I highly recommend you read the article for all of the details.

I don't think too many people disagree that over the last years there has been a big swing in the pendulum favoring business's over the consumer. Many of the laws and rules that were put in place to protect consumers over the years have either been thrown out or ignored.  Our bankruptcy laws have been totally rewritten to the disadvantage of the consumer.  Lazes-fare attitudes toward regulating business has seen oil companies raking in billions of dollars a quarter while we pay 4 dollars a gallon.  Health insurance companies raising rates higher than inflation while hospitals have to close for not making enough money.  A mortgage industry that without oversight has written loans that has our finance system to the brink of disaster. A return of inflation and recession at the same time.

Not too advertise my own political views, but do I think it is time for a change?  Your damn right I do!  I hope that the press shining the light on some of these injustices will make it easier for a new era in Washington to make right (no pun intended) some of the wrongs in our system.