Economists have long understood the corollary concept of Coase's ceiling, a point above which organizations collapse under their own weight -- where hiring someone, however competent, means more work for everyone else than the new hire contributes. Software projects often bump their heads against Coase's ceiling: recall Frederick P. Brooks Jr.'s seminal study, The Mythical Man-Month (Addison-Wesley, 1975), which showed how adding another person onto a project can slow progress and increase errors.

What's new is something consultant and social technologist Clay Shirky calls "Coase's Floor," below which we find projects and activities that aren't worth their organizational costs -- things so esoteric, so frivolous, so nonsensical, or just so thoroughly unimportant that no organization, large or small, would ever bother with them. Things that you shake your head at when you see them and think, "That's ridiculous."

Sounds a lot like the Internet, doesn't it? And that's precisely Shirky's point. His new book, Here Comes Everybody: The Power of Organizing Without Organizations, explores a world where organizational costs are close to zero and where ad hoc, loosely connected groups of unpaid amateurs can create an encyclopedia larger than the Britannica and a computer operating system to challenge Microsoft's.

Shirky teaches at New York University's Interactive Telecommunications Program, but this is no academic book. Sacrificing rigor for readability, Here Comes Everybody is an entertaining as well as informative romp through some of the Internet's signal moments -- the Howard Dean phenomenon, Belarusian protests organized on LiveJournal, the lost cellphone of a woman named Ivanna, Meetup.com, flash mobs, Twitter, and more -- which Shirky uses to illustrate his points.

The book is filled with bits of insight and common sense, explaining why young people take better advantage of social tools, how the Internet affects social change, and how most Internet discourse falls somewhere between dinnertime conversation and publishing.

Shirky notes that "most user-generated content isn't 'content' at all, in the sense of being created for general consumption, any more than a phone call between you and a sibling is 'family-generated content.' Most of what gets created on any given day is just the ordinary stuff of life -- gossip, little updates, thinking out loud -- but now it's done in the same medium as professionally produced material. Unlike professionally produced material, however, Internet content can be organized after the fact."

No one coordinates Flickr's 6 million to 8 million users. Yet Flickr had the first photos from the 2005 London Transport bombings, beating the traditional news media. Why? People with cellphone cameras uploaded their photos to Flickr. They coordinated themselves using tools that Flickr provides. This is the sort of impromptu organization the Internet is ideally suited for. Shirky explains how these moments are harbingers of a future that can self-organize without formal hierarchies.

These nonorganizations allow for contributions from a wider group of people. A newspaper has to pay someone to take photos; it can't be bothered to hire someone to stand around London underground stations waiting for a major event. Similarly, Microsoft has to pay a programmer full time, and Encyclopedia Britannica has to pay someone to write articles. But Flickr can make use of a person with just one photo to contribute, Linux can harness the work of a programmer with little time, and Wikipedia benefits if someone corrects just a single typo. These aggregations of millions of actions that were previously below the Coasean floor have enormous potential.

But a flash mob is still a mob. In a world where the Coasean floor is at ground level, all sorts of organizations appear, including ones you might not like: violent political organizations, hate groups, Holocaust deniers, and so on. (Shirky's discussion of teen anorexia support groups makes for very disturbing reading.) This has considerable implications for security, both online and off.

We never realized how much our security could be attributed to distance and inconvenience -- how difficult it is to recruit, organize, coordinate, and communicate without formal organizations. That inadvertent measure of security is now gone. Bad guys, from hacker groups to terrorist groups, will use the same ad hoc organizational technologies that the rest of us do. And while there has been some success in closing down individual Web pages, discussion groups, and blogs, these are just stopgap measures.

In the end, a virtual community is still a community, and it needs to be treated as such. And just as the best way to keep a neighborhood safe is for a policeman to walk around it, the best way to keep a virtual community safe is to have a virtual police presence.

Crime isn't the only danger; there is also isolation. If people can segregate themselves in ever-increasingly specialized groups, then they're less likely to be exposed to alternative ideas. We see a mild form of this in the current political trend of rival political parties having their own news sources, their own narratives, and their own facts. Increased radicalization is another danger lurking below the Coasean floor.

There's no going back, though. We've all figured out that the Internet makes freedom of speech a much harder right to take away. As Shirky demonstrates, Web 2.0 is having the same effect on freedom of assembly. The consequences of this won't be fully seen for years.

Here Comes Everybody covers some of the same ground as Yochai Benkler's Wealth of Networks. But when I had to explain to one of my corporate attorneys how the Internet has changed the nature of public discourse, Shirky's book is the one I recommended.

This essay previously appeared in IEEE Spectrum.

SaaS is taking a bite out of the $18 billion IT management market. A new Forrester Research report forecasts SaaS-based IT management accounts will be 10% of the market by 2013. The reason: high level of interest from medium-sized and large enterprises. Forrester also predicts that enterprises with 1,000 or more employees will account for 50% of SaaS installations in 2009. We’ve seen this on the service desk side with the rapid growth of upstart Service-now.com. Companies are looking for easier and rapid deployment, lower upfront and capital costs and rapid time to value – all benefits of SaaS as well as our own appliance model.

IBM snapped up Transitive this week. Their QuickTransit software dynamically translates native code between architectures, enabling apps compiled for one processor to be run on another without any modification. Apple was the first licensee and used it to build Rosetta, a translation system that allowed users of Intel Macs to seamlessly run legacy PowerPC apps. IBM plans to use the technology to move workloads onto IBM systems without recompiling, allowing customers to “save on energy costs due to hardware consolidation and reduced TCO.”

At CA World, CA announced a partnership with Amazon to provide “management capabilities around Amazon’s EC2 utility computing platform, potentially including discovery of software running on EC2 instances, performance monitoring, configuration management, software deployment capabilities and provisioning”. John Willis, in spite of some pretty funny potshots and stories about CA (don’t we all have them), writes that “CA is the first of the Big Four to take the cloud serious”.

The panelists were:
William P. Crowell – former Deputy Director of the National Security Agency
Michael P. Jackson – Deputy Secretary, Department of Homeland Security
Jose A. Rodriguez, Jr. – former Director CIA, National Clandestine Service & CIA, DCI Counterterrorist Center

Overall, it was a very nicely arranged event on a brisk fall evening with about 100 CXO attendees; mostly large but some small government contractors and a few product companies like ScienceLogic that conduct business with military, intelligence and the public sector.

No surprise, given the financial crisis the economy is suffering from that the panelists said we also have a crisis coming on the Federal budget front. This will put enormous pressure on the way Administration thinks, and how and where to spend the $$.

Obama’s tone regarding the issues he will be confronting in the world during the election was encouraging. Make the world more non-partisan and take on the threats that we have in front of us head-on!

The panel was very upfront about current threats. William Crowell said,

“It is highly imprudent to believe that there will not be another 9-11. We have to fund and support the work to stop other attacks. We can only mitigate risk but we can’t eliminate risk. We have to try to absorb the sense of urgency and wake up every day looking at the intelligence screens as if 9-11 happened within the last couple of months.”

He added,

“They (the intelligence community) need the innovation, sense of commitment and urgency that comes from the private sector – a sense of mutual commitment to that mission.”

Predicted Priorities for investment for DHS:

1. Cyber attack as the top issue
2. Nuclear threats including dirty bomb
3. Chemical and biological attacks
4. Explosive attacks against critical infrastructure with maximum # of lives and or financial disruption / loss.
5. Large scale natural disasters – hurricane + earthquakes
6. Border penetration - identity management and border management issues

An Obama administration will spend dollars around these threat vectors. They will want to spend $$ to help state and local governments. Grants to state and local governments should significantly increase with the Obama administration, so think about how you will increase your focus on the state and local government spending initiatives.

Secure border investments – the panelists believe that the new administration will feel compelled to invest here. Michael P. Jackson bluntly said, “You have to make investments in border tools to get meaningful immigration reform.”

Panelists agreed that the 1^st year will be an intense period of scrutiny about fundamental directions. We can’t afford it all at DHS; it is dramatically under budgeted. At TSA/DOT and then at DHS, we spent about $4 Billion on technology investments since 9-11; those investments are now reaching the end of the original service life.

One gripe from the panel that I found humorous: “We don’t have a group of people who think like entrepreneurs.” It is insane how long things last when you buy things in the government. As an example, we are still replacing vacuum tubes in some of the very old FAA gear… this is well beyond what any reasonable person would think these initial investments should/would last.

Final Thoughts:
I actually think that the Obama Administration will be quite favorable to COTS software products, SaaS offerings, and creative financing initiatives from the private sector. The government just won’t have the capital budget to do everything it wants to accomplish. I would say if you look at how intelligently and aggressively Obama used technology to assist his campaign, the odds are good that this new breed of IT talent (which is already really comfortable with SaaS products, blogs, wiki’s, hosted/outsourced Cloud solutions… this team really understands the latest technology trends) will quickly work to bring these new IT paradigms to the Federal marketplace. Clearly the private sector can help the Government achieve more with lower capital budgets – beginning to provide services rather than transaction-based selling. Another clear idea is to think about leasing as a better way to work with the government which going forward will have increased budgets restrictions.

They will likely be in confrontation with members of Congress that won’t change fast enough, however the future of our nation’s ability to fight terror lies in becoming more efficient and effective. It requires the government be flexible enough to figure out what jobs and IT functions to outsource in a nimble and smart way. My prediction: this is great news for Service Providers. Overall the next 4 years should be great for our business as well as the Managed Service Provider/SaaS industry!

I’ve often read people who say that we (security, risk management) need to “think like the attacker”.  And when you read this sort of article, that usually alludes to trying to anticipate the tactics an attacker might use to mess with your C, I, or A.  Smart stuff, that, and very useful when architecting security solutions.  But as I was training some folks Monday, I was thinking in the back of my head about Threat Capability (TCap) in FAIR.  As you might know, we like to estimate the capability of a threat to apply some level of “force” against our assets.  This ability to apply force is a byproduct of the attacker’s skills and resources.  And thinking of how an attacker applies skills and resources, I came across another way we might “think” like an attacker.

Traditionally, I’ve thought of “skills” as being a byproduct of the toolset an attacker has.  This mindset probably stems from my time with Penetration Testing teams, where in the process of scoping the  PenTest I would ask our clients to select the level of effort that they wanted us to throw at them.  If a client chose “high” we’d throw every ‘spoit we had at them.  If they chose “low” we’d limit ourselves to a more commonly available toolset.

But while the resources part of TCap is time & materials (money) - the skills are really more than just the toolset.  Skills would include the ability of the attacker to be creative and innovative.    As an example of that innovation from those PenTesting days - when we got a “high” effort request, we would always try to couple that with some “social engineering”-type of attack, or some unique means of delivering an existing exploit.  Our creativity was not necessarily a byproduct of a unique exploit or tool we had, but the process by which we might deliver pre-existing or commonly available exploits.  I remember when we first got ahold of a handful of 32mb thumb drives (hey, 32mb was huge back then) and “dropped” a few in the lobby of a client’s retail space.  The keystroke loggers and phone-home script weren’t new, but using the thumb drive as delivery vehicle certainly was.

So I’ve started to really think about this concept of innovation, and how if “thinking like an attacker” means to be innovative, we ought to do the same.  I’ve been thinking of two main categories of innovation this morning.

INNOVATION

The first I’ll call Technology Innovation.  And by Technology Innovation, I mean some new, unique, “ahead of the curve” technology that an attacker can use against us.  The obvious example of which is a zero-day.  It’s that “high” tool set our PenTesters would use against the clients.  For security departments, this might be the latest security product designed to enhance our ability to P, D, and/or R.

Alternately, we can be creative in the way we deliver (manage) existing technology.  I think of this as Process Innovation.  It’s doing more with what we already have, just like the PenTest team would be creative in the delivery of an existing exploit.

Unfortunately for us - attackers have traditionally had quite a leg up on us in terms of Process Innovation.  It is much easier fro them to be creative, as they are free of political constraints and bureaucracy.  In contrast, when the security industry tries Process Innovation, the results are checklists and “standards”.  It’s committees and consensus.  An extreme example of which might be something like SABSA - a great work if you want to understand some very smart people’s comprehensive understanding of organizational security  - but the “adoption”of which will do very little to help you be innovative in P/D/R.

It’s worth noting that ultimately, this is one reason I don’t like regulatory compliance efforts - they simply serve to prove how mundane your security department is,  wasting valuable resources that could be spent on creating ways to be more effective.

PROCESS INNOVATION AS A SUBSTITUTE FOR TECHNOLOGY INNOVATION

As we come to the close of 2009, some surveys suggest that security spending isn’t horribly impacted yet by the economy (the latest from E&Y points to only 5% of their respondents getting budget cuts).  But if this is a protracted downturn, and because InfoSec is an operational expense, I would expect cash to become more and more difficult to keep.  And regardless if technology spends do slow, I believe it makes sense to think about Process Innovation because I see Process Innovation as a means to increase effectiveness without significant capital expenditures (effectiveness increases because our ability to manage risk has a direct correlation to the amount of risk we have).

The bad news is, of course, that great innovation is hard.  It is R & D.  Failure is usually a pre-requisite to success.

The good news is, our current state is so bad that many of us don’t need to come up with a whizbang new way of reducing software defects in the SDLC as innovation.  Simply inserting a risk analyst into the PMO’s processes might count as a big enough victory. Be cautioned, though,  that if we’re substituting the risk reductions provided by technology acquisition - Process Innovation might actually be even more “expensive” as it requires us to expend political capital.   But there are (forgive the term) innovative ways to spend this political capital.

For example, by taking a second now and figuring out the 3 things that the rest of the organization can do to make your life easier, when that “I need to reduce your budget” talk comes, you can be prepared to negotiate.  Get a political capital “loan” or “investment” from the C-Suite reducing your budget.  Something to the effect of: “I expected this, and am happy to give up my budget.  But if our tolerance for risk hasn’t changed, what I’d like to do is get you to personally back my office on three projects I’ve identified that can reduce our risk without requiring significant capital expenditure.”

After meetings and calls, I was finally able to slip into a conference session – just in time to catch uber-smart analysts Rachel Chalmers (The 451 Group) and Dan Golding (Tier1 Research) engage in a lively and not-so-mock debate on “Hosting Meets the Cloud”.

Now this doesn’t cover the entire debate – and part II is coming tomorrow. But what it does cover is the most interesting questions (to me) and paraphrase the points made by the analysts. I thought they both had very interesting points and more similarities than differences in the end; the real difference is how they thought about the issues and through what lens – for Rachel it was the enterprise and for Dan it was managed hosting providers. (image from inmagine)

Question: What is a cloud and why?

Dan: Shared infrastructure leveraged/run by third parties for the benefit of enterprises, developers, etc. This is not a new idea – just recently “rebranded.” Given all the discussion and disagreement over this now, what will the cloud end up looking like?

Rachel: The cloud is “IT infrastructure as a service” down to the level of a server operating system. Take the example of Amazon web services – in this case it’s not just the infrastructure but also the internal processes built around service delivery, e.g., provisioning, that are being exposed as a commodity to external customers.

Dan’s Question for Rachel: In your opinion, how much is the cloud a fad versus CIOs really trying to solve a problem?

Rachel: For the practical, roll-up-your-sleeves types of CIOs – those coming up from the engineering ranks – that I talk to, the cloud is real, as opposed to SOA and middleware.

What about “internal” cloud computing – built and maintained by an enterprise versus a third-party provider?

Dan: Cloud computing is done by providers for customers. Certainly there are enterprises that have made internal computing investments, e.g., for publishing, large-scale phone systems, etc - but they were stupid ideas made by companies that have too much money. A better question here is does it make any sense for an enterprise to create their own cloud? While an enterprise can play at it, they can’t do it cost-effectively, not in a way that a third party provider can do it.

Rachel: Many CIOs have “managed-hoster” envy – for things like chargeback and billing that hosters understand a do better. Of course there has been a rise in automation and virtualization tools in the enterprise which may not be as efficient and built for scalability as a hoster can achieve, but what is important is that they are customized/specialized for that business.

Dan: Can you give a specific example of optimization to make it worthwhile for enterprises to do it themselves?

Rachel: One example is sovereignty. The privacy laws around financial and healthcare information are not the same everywhere. Clouds and their geographically-dispersed data centers don’t necessarily have “national” borders. This is definitely a concern for the CIO that has to comply with regulations in their industry around privacy protection, for instance. Another example is security. Dow Chemical does a lot of work via joint ventures and has a need to provide but lock down desktops given to contractors as corporate workspaces. For their level of security, they need to “own” their computing resources.

Dan: But why can’t someone like SunGard provide that as they do for many other large companies?

Rachel: It comes down to a question of trust.

Do people trust their hosting providers?

Dan: Yes. Whether it’s for a content delivery network or collocation, hosting the customers of hosting providers are some of the largest companies in the world in industries like energy and financial services. Give me a case when there was a major security issue with a hosting company. In fact, managed hosting providers usually provide better security than enterprises are capable of.

And a question provided by an attendee from EMC: A few years ago, this would have been a grid discussion. How is the cloud different?

Rachel: Grid computing ended up being applicable only for niches – which I predicted. The real opportunity for everyone else with the cloud only comes up when you combine the kinds of automation tools (originally developed for grid computing) with x86 virtualization.

Dan: I agree. Grid was a niche play. There were very few orgs that needed it and that the economics worked for. There were very few enterprises for whom it made sense to build their own for. The cloud is shared/leveraged versus grid computing. It economically makes sense in a way grid never did.