Blogger: Trent Henry

Burton Group’s Catalyst Europe conference is just around the corner. With financial services industry failures at the top of everyone’s mind, now’s a great time to revisit how risk management shortcomings have tremendous impact on organizations of every kind. In a reprise of his insightful Catalyst North America talk, Nick Leeson will once again detail how inadequate controls (and foolish actions on his part) brought about the fall of Barings Bank. In addition, security conversations at Catalyst will include:

- How large enterprises are grappling with governance, risk, and compliance (and why “GRC” is actually a four-letter word)
- What large, distributed organizations are doing to create effective “security embassies”
- The role of metrics in managing protection and communicating with Management
- How information-centric security will unfold over the next five years

Sevcik and Wetzel have a consistently interesting column on Application Performance Management at NetworkWorld. This week, they unveiled the results of a benchmarking survey that tells them mid-sized enterprises have it harder when it comes to deploying such solutions.

We agree; it’s why we exist. Mid-sized enterprises have the same IT problems but not nearly the same amount of resources as the really big guys to throw against solving them.

VMWare’s acquisition of B-hive continues to generate buzz for performance management and virtualization. I love this quote from the CEO of Aternity, “The next big frontier is the ability to transform huge amounts of data into actionable business intelligence that correlates across platforms.” Um, we’re already doing this. What would be the purpose of collecting hundreds of millions of data points if you couldn’t actually present the data in a meaningful way? Maybe his comment was taken out of context and it’s more about the fact that it’s often difficult to get consistent and accurate info on virtualization resource utilization stats in particular. That we totally agree with. Another take on the B-hive acquisition: VMTN blog gives a quick overview of what it means for infrastructure groups and virtual environments.

ShareThis

And of course, by now the response to the report card is all rote–everybody wonders what the letters really mean:

• SC Magazine
• IDG
• IT Business Edge
• Federal Times
• Washington Post
• Security Focus

Yeah, yeah, I guess it just goes to prove what we say about the classified world: the people who know don’t talk and the people who talk don’t know.  In this case, everybody attacks the metric because, well, it’s a bad metric–what action are we supposed to take because of what the results are?  It’s also pretty much ignored by this point anyway except for the witty sound bites from some of my “favorite people”, so it’s nothing to get all hot and bothered about.  The GAO and OMB reports that I’ve covered in much detail are much better and have a pretty decent level of analysis.

But fer chrissakes, the report card is issued by Congress, how much detail do you think it will ever contain?  =)

My rapidly expanding queue of pet peeves about this time of the year:

• People who think that FISMA is just a report card and that we should re-examine how we measure security:  the grades are not even required by the law, it’s just technique and we can change that easily enough.
• People who criticize but do not offer an alternative:  even if you had an alternative plan, the environment for execution still involves the same IT assets and the same front-line employees.
• People who don’t understand enterprise-wide security much less a federation of semi-independent enterprises: it’s the nature of government-wide security metrics that they’ll be indicators which can be faked.
• Sound bites from people who have never implemented any aspect of FISMA:  come on, SANS and Gartner?  GAO and the Cyber Security Industry Alliance are a little bit better but taken out of context.
• Nobody ever asks me for a quote on FISMA numminess:  I’ll be pouting for the rest of the week, TYVM.  =)

Not that I’m the world’s best expert at fact-checking, but something caught my eye in the report:  it’s issued by Tom Davis and the url is from the Minority Office for the House Committee on Oversight and Government Reform.  Tom Davis is the representative from Northern Virginia and is the sponsor for FISMA back when it was signed.  Until the last election, he was the chairman of the House Committee on Oversight and Government Reform.  The committee is now chaired by Henry Waxman. 

Time for a new concept in your vocabulary:  LGOPP (OK, actually it’s LGOP, but I added an extra “P” for comedy purposes).  Imagine June 6th, 1944, paratroopers scattered all over the French countryside.  What happens is you pick up the people around you, the senior person becomes the leader, and you carry out the mission.

Photo of Paratrooper Stained Glass in Sainte Mère Église by Nelson Minar

Hence the true meaning of LGOPP: Little Groups of P*ssed-off Paratroopers.  An equivalent phrase is “isolated pockets of brilliance”.

In the words of somebody I went off to war with:  “LGOPPS are the spirit of the infantry:  a handfull of 18- and 19-year-olds with fully automatic weapons who can barely remember what their mission is running around the woods raising hell”.

Now, I know you guys, you’re wondering what this has to do with security?  Well, this is relevant because it’s an election year.  What that means is that instead of being bothered with all this security stuff, Congress is involved in playing “gotcha” with the Executive branch.  After the election, it’s rearranging deck chairs on the Titanic and all of the leadership will change.

Instead of any national-level security agendas and strategizing, we’ll have to be content with security LGOPPs fighting the fight wherever they end up gaining enough critical mass.

And in the case of this year’s FISMA report card, the LGOPP that is Tom Davis’s staffers issued the report while the rest of the committee was busy worrying about elections.

Bookmark to:

First, let me say what I knew about the SDL going in: no policy can anticipate every situation; you have to make tradeoffs; the details matter; the big picture matters; you need tools; you need human insight; you need management support; and we’re never going to be perfect. All of the things you’ve read in this blog are true, and they really shouldn’t be controversial. Since joining SQL, I’ve learned a lot about SQL Server too, and what it means to ship a product - but that’s outside the scope of this blog. So instead, I’ll try to describe three real experiences that illustrate things that shouldn’t be controversial either, but aren’t usually covered under the rubric of security.  They are crucial nonetheless.

Security is not the point, it’s the needs of the customer. It’s easy to believe that security is the point of producing a product. It’s not. We won’t produce an insecure product, but the primary driver for a product team is to produce a valuable, useful product. Yes, security is a big part of that, but security is not a goal in and of itself.  For example, one of the areas of fierce competition in enterprise database products is performance, and we have to balance security with  performance. One of the ways we do that is by verifying data we receive really well, but only when necessary. We define clear trust boundaries, and check the data thoroughly once on the way in, and then work very hard to enforce those trust boundaries.

I first encountered this in SQL when I helped review threat models for the database engine. The engine trusts that the data on the disk was written correctly by a trusted entity (with checksums to guard against random errors), and enforce that. Instead of a slavish adherence to the principle of total mediation or defense in depth, which, when taken to its extreme would say to “check everything, every time,” we are hard core about making the right checks, but only the right checks.

I will note that it is not an either/or choice between security and performance – it is possible to do both. Indeed, I would say that doing one without the other is pointless, but to get both 1) world class performance, and 2) world class security,  you have to understand your data flows really well, and make detailed decisions.

Be polite, but don’t be afraid: Job interviews at Microsoft can be challenging. When I interviewed for this job, my final interview was with a very senior architect. The subject of integer overflows came up, and he asked me to describe the problems and solutions. So I started writing some code on the whiteboard. After about 10 minutes of describing my approach to integer overflows, he said to me, “What if I were to tell you that’s a really bad solution, and the interview is over?”

My heart sank.

But instead of rolling over, I said, “well, that’s a bad outcome, tell me why.” He proceeded to attack my solution on several grounds, including being unreadable and unmaintainable, and he proceeded to describe his solution to the problem. Now, this was a very serious, very senior technical architect, and I was in a high pressure, asymmetric situation. So, not willing to be intimidated, but unable to attack back, I pointed out several shortcomings of his solution, politely, but firmly. And we spent the next 40 minutes talking about various aspects of the problem, and me defending my solution, which I think was credible. I don’t know if he agreed with my solution or not, really, but I suspect it might have been a test to see if I would cave. Or maybe he thought it really was a bad solution, I don’t know. But I got the job.

As a security professional, you’re always going to be at a technical disadvantage when you’re reviewing another team’s components. They designed and implemented the system. You are an outsider, and it is absolutely impossible to understand the system to the degree as the people who built it. Nonetheless, you’ve got to find a way to ask hard, probing, impolite and sometimes even uninformed questions without being threatening or insulting, or undermining your own credibility.  

Be polite, be firm, put your ego in a box, and ask questions until you understand.

“It should work” is not a good answer:  We take the giblets problem very seriously, and managing giblets can be quite difficult at times. And in SQL, we have lots of giblets. We consume things from Windows, and Office, and Visual Studio, and others, and we provide giblets to other teams as well. In fact, we provide components that other teams use to build the giblets they provide to us – we consume our own giblets!

And as it happens, one of the components we use was updated recently. Even though it would get serviced through Microsoft Update, we want to ensure we have the latest and greatest version of any component we ship. But to consume the latest and greatest version of this particular component would require some small updates to either our installer or theirs. So we met with the team that owns the giblet in question to try to divvy up the work, and to avoid schedule disruptions on either side.

There was a lot of back and forth about various things to try, and we continued to refine a solution until we had reduced the problem to a single issue. 

At this point, there was an air of hope in the room. If the idea actually worked, we had a solution at relatively low cost. But would it work? When the question of “will this work” comes up, all eyes turn towards test managers.

Our general manager was looking right at our test manager and she asked, “Will that work?” The test manager looked across the table at the development manager from the other group, and said, “I don’t know. That depends on their level of confidence in the behavior of their component under these conditions.”

Now, all eyes were starting at the dev manager, and the room got quiet. A somewhat sheepish look came over his face, because he knew the answer he was about to give would be unsatisfactory. He said, “Well, I’m not a tester, I’m just a developer, but it should work.”

At which point the room erupted into hysterical laughter.

“It should work” means “I think so, but we have to test it.” And that means the whole battery of tests for each of the affected components, across all of the supported platforms. And that has to be scheduled in test labs. To be clear, this wasn’t a lack of confidence in the developer, quite the contrary, he was laughing along with everyone else. We just know that writing software to satisfy all the scenarios in which our software is deployed requires far more testing than can reasonably be performed on a single desktop system.

So the tests were scheduled, the developer was proven correct, and we’re picking up the latest version. Even seemingly simple changes require a lot of testing.

So, that’s what I’ve learned: security isn’t the be-all-end-all,, things are really complex and hard to understand, and you don’t really know if anything works until you test it. None of which should be controversial, but none of the central ideas in the SDL are controversial either. The hard part is putting theory into practice, and recognizing that no venture is risk free, despite the natural inclination of security engineers to avoid any risk whatsoever. In this, I am reminded of one of my favorite books, “To Engineer is Human: The Role of Failure in Successful Design,” by Henry Petroski. He writes, “No one wants to learn by mistakes, but we cannot learn enough from successes to go beyond the state of the art. Contrary to their popular characterization as intellectual conservatives, engineers are really among the avant-garde. They are constantly seeking to employ new concepts [and are] constantly striving to do more with less. [] The engineer always believes he is trying something without error, but the truth of the matter is the each new structure can be a new trial. [] Such is the nature not only of science and engineering, but of all human endeavors.”

Whenever I read about China censoring internet access to its citizens, forcing Google and Yahoo to not show certain sites, I smile a smug, holier than thou smile and shake my head about how a government can do that to its people and get away with it. Why would those people put up with it?  So I must say "in a day that will live in infamy" I was very chagrined to read this article in Wired by Noah Shactman, reporting that just about any site with the word blog in it is banned from our troops in the Air Force.  From what I understand this is limited to the Air Force and not our other armed services.

I use the term our troops, not their troops, because this isn't some foreign, totalitarian country or despotic dictatorship we are talking about, where the troops have to be watched so they don't cross over to the other side.  These are the men and woman who put their butts on the line, risking their lives every day for us all to enjoy the freedom to read any damn site on the internet we want to.  The irony of these very same front line heroes who provide the blanket of freedom that we all sleep under, not being able to read any blog they feel like is not lost on me and should not be lost on you either!  If they are smart enough and good enough to protect our country they should be smart enough to be allowed to choose what they want to read on line and should have the freedom to read news and commentaries on blogs as they see fit.

The idea that we are censoring the news our service men and woman can read disturbs me on many levels. Besides what it says about a lack of trust in our troops, it also disturbs me that someone actually says "they can still access news sources that are "primary, official-use sources," said Maj. Henry Schott, A5 for Air Force Network Operations. "Basically ... if it's a place like The New York Times, an established, reputable media outlet, then it's fairly cut and dry that that's a good source, an authorized source,"  Who decides what primary, official-use sources?  It gets worse, "Often, we block first and then review exceptions," said Tech. Sgt. Christopher DeWitt, a Cyber Command spokesman. Shoot first and ask questions later, huh?  The arrogance of this galls me. If you told me this was some North Korean General or Politburo member from the old Soviet Union, I could see it in a second.  But spokespeople of the US Air Force?  Where have we gone wrong?

Some make the argument that blogs are not really media outlets. Can the people making policy at the Air Force be that naive?  Others say that there is so much BS on blogs that Air Force folks are "baited" into commenting and possibly giving away operational security information.  That sounds to me like a social engineering problem, not a blog problem.

Yeah, I know there is a war on.  Are we afraid our Air Force men and woman are going to all go to some Arabic-Al Queda web sites and be brainwashed?  Is their some terrorist worm they will get by going to a web site that spouts ideas different than "primary, official-use sources?  What scares the Air Force so much that they would take such action?  If you feel like I do about this, lets do something about it.  Lets write to the Secretary of Defense, Joint Chiefs of Staff, Congressmen, Senators, whoever, but lets return freedom of the press and freedom of speech to our troops who put their lives on the line so we can enjoy those rights!