Today, allow me to update you on FAIR and the movement towards a formal, open standard.  There’s a couple of cool things going on in our little risk-world.

First, The Open Group Security Forum continues to move towards a formal adoption of FAIR.

WHAT DO YOU MEAN “WE” - YOU GOT A STANDARDS BODY IN YOUR POCKET OR SOMETHING?

Our meeting in Chicago a few weeks ago was great, but also slightly disturbing for me. I got pronoun-confusion syndrome.   I’m used to using the “we” pronoun to refer to RMI, or Jack and myself as we vet the models.  So without even thinking I would said “we have been looking at how loss occurs, and may want to change the model some” and The Open Group Members freaked out (rightfully so).  Adrian Seccombe gently reminded me that the “we” was now the Security Forum, and that “we” didn’t go changing things at will without vetting against each other.  Man I love this stuff.  I get to run our thoughts and ideas past some great folks now - you know, those smart people who tend to have really complex problems and are trying hard to solve them.

Formal Adoption:  Soon, Very Soon Now

Formal Adoption basically means we’ve made this document, everyone is close to saying that they generally like it, and once that finally happens then “bam”, we’re ready to move onward and upward with better things (see Cookbooks, below).  We’ve got a couple of changes to the current document that have been requested that aren’t a big deal.  For example, one request is that we make some statement about general applicability of FAIR to risk domains outside of the IT realm.   But once additions like that and others are done, this long process should be complete.

New Document Moving Towards Public Release:

We’ve got a basic document that should be public in the next few weeks on “What Makes a Good Risk Assessment Methodology” - written by yours truly and Jack.  It’s a very high-level document, and serves two purposes:

• For novices it helps parse out what is important in any undertaking to understand corporate risk (the repeated discussions on the ISO 27001 mailing list make me think it would be a place ripe for such a document).
• For those who “know” risk, it helps to re-establish some fundamental principles like the use of scales (ratio, please), the implications of dealing in probabilities, what attributes like consistency and defensibility mean, how “risk” should be reported to the business (something you know, meaningful) and so on.

When this doc is deemed ready for public consumption I’ll be sure to post on this blog here.

COOKBOOKS, EUROPEAN AGENCIES, AND, IRON CHEF “RISK” - WHOSE CUISINE WILL REIGN SUPREME?

One interesting thing that came up in the Chicago meeting was that ENISA (The European Network and Information Security Agency) developed a very nice document that reviewed something like 18 different risk assessment methodologies against their Criteria for Goodness.  FAIR was one of the ones they reviewed, and we (the royal “we” used there to include all us FAIR-Folk) did awfully well.  Things of interest:

1. They based their work on the current introduction paper which is not at all a step-by-step guide towards an organizational risk assessment (what ENISA really wanted) and we did pretty well.  Well enough that if we had developed a paper along the lines of NIST 800-30 or OCTAVE for the use of FAIR in a formal process, we could have done really, really well.  Like won-the-bake-off kind of well.
2. FAIR is actually not at all incongruous to many of the risk assessment methodologies offered, and in fact compliments many of them by letting those methodologies develop real, structured probabilities.  Think OCTAVE, where they basically say “math is (probabilities are) hard, so if you want to do them for reals, good luck!  But here’s a nonsensical way to do things if you want to believe in magic-fairy risk“.  FAIR fits right in there by stomping on the magic-fairy risk with the jack-boots of rationality.  FAIR similarly helps other risk standards that might lack structured probability development.

So The Open Group Security Forum decided that though we could create a new document and totally p0wn any future ENISA bake-off, there wasn’t much demand for the development of that documentation by the membership  - a point which was made quite apparent at the beginning of the discussion when one large European company CISO asked “What’s ENISA?”  Relevancy is everything, I suppose.

But that second item up there - the one about helping rather than competing with other “risk assessment methodologies” - really struck a chord.  So “we” (The Security Forum) are going to develop some “Cookbooks” that basically are high-level documents that say “If you want to use FAIR with (OCTAVE/COSO/CoBIT/Whatever) here’s how it fits, makes it better, and improves your life.  I’m pretty excited about these, and our first document looks like it’s going to be COSO integration.

THE OPEN GROUP SECURITY FORUM - THEY’RE A TRUSTING BUNCH (WITH QUALIFICATION, OF COURSE)

Finally, many people have asked me “Why work with The Open Group?”  There are many reasons, to be sure, but I will give you one example.  Members of the Security Forum there are not only great at vetting the model and getting consensus on risk and risk factors - but they’re quick to start applying.  So in Chicago, I thought I’d be talking about FAIR and the standard and fighting groupthink.  Nope.  Not at all.  In fact, the forum members spent more time suddenly discussing use of FAIR in a new Trust Model they’re developing.  So all of the sudden, I’m part of a new and exciting project to develop a Trust Model - how cool is that?  While formal adoption of the Trust Model will be necessarily long and deliberate - the collaboration and development is happening much faster than I can keep up with.  But if you all will allow me, it will help me get my head around it all by blogging about it later this week.  So be prepared to read about me dealing in “Trust” a little bit.

You can read plenty of other blogs about some of the great presentations at Black Hat.  So I thought I would take another angle and talk about some of the other stuff that may be important to you.

1.  secure@microsoft.com – This years hottest party was again the Microsoft party.  This year it was at the LAX club in the Luxor.  As usual there were quite a number of people at the door who thought they could talk their way in or worse yet were told they “were one the list”.  I was happy to be able to go and saw many of the usual suspects there as well. I had to leave the party early to go catch my red eye flight home, so went right to the airport from the party.  As I wrote earlier, Microsoft is trying really hard on security.  But I couldn’t help but notice the irony of this grainy, lousy picture of the DJ booth at the party.  If you can, notice the computers that the secure@microsoft.com DJs are using. That’s right they are Macs!

2. A new low for booth babes – What would a Shimel review of a trade show be without a booth babe rant.  Hey I recognize it is Vegas and all, but EdgeOS went way over the line this year.  A booth babe dressed as a Las Vegas showgirl or some other type of costume makes a statement.  I personally don’t like exploiting woman to make that statement, but I understand.  However, these guys had woman who were dressed so raunchy and classless, that I could not bring myself to post a picture of them.  Come on guys!  You want to resort to the booth babe thing (and BTW I think the Black Hat crowd does not respond to that), at least have a little class.  These girls looked like street walkers and do you and your company no favors.  Is that really the image you want to promote?  Grow up!

3.  The Security Bloggers Network – We are back!  With the end of the Black Hat show, the SBN is going back to being the SBN.  The old logo is back and our promotion with Black Hat is at an end.  However, I want to personally thank so many of you SBN members who blogged about Black Hat.  The Black Hat marketing folks made it a point to come over to me and thank us for the overwhelming support and help of the community.  Our network delivered big time with them and they are already thinking about ways we can work together next year.  I will keep you all posted on that.

We have several new promotions we are working on with the SBN and will have more on that soon. Also, we learned some valuable lessons.  Next time we will work with the network members more closely in doing these affiliations.  Also, for any show like this we need to have an official bloggers get together.  Not because we don’t want to buy our own drinks (thanks to Chris Hoff for doing more than his share in picking up a big bar tab), but frankly we need to reserve a place that has enough space for us.  Security bloggers are big time. We have a great community of people who get together. Lets make it better.

I have some other ideas around the SBN I am working on too and want to form a committee to help. If you are a member and want to get involved, please drop me a line or comment.

Anyway, another year of Black Hat is in the books. It was a good one and I can’t wait until next year!

Q1: How do you handle variety of log sources? There are so many, almost beyond my capability.

A1: Sorry to ponder the meaning of "is" here, but what is meant by "handle"? It is really not that hard to collect logs from a large number of diverse sources (as long as the logs can be delivered via syslog or exist as files and can be collected). Now, there will certainly be challenges  when the volume of logs gets large, but if by "handle" you mean "collect + store", it is really not that hard, given the right tools. Now, if "handle" means "make sense of what all those logs are trying to tell you," it is a different story altogether.

Q2: You talked about the importance of logging; however for an intermediate or novice admin what are the starting steps .. what are the minimal logs they should start at once?

A2: Answered in "Log Management - Day 1" If you want a simple list of things to "enable today,"  I cannot really answer it since I know neither your needs, nor your environment. In other words, this is the "what is the meaning of life question?" :-)

Q3: What regulations, rules or guidance exist regarding sharing or visibility of logs to users?

A3: PCI DSS says in Requirement 10.5:  "Secure audit trails so they cannot be altered.
10.5.1 Limit viewing of audit trails to those with a job-related need
10.5.2 Protect audit trail files from unauthorized modifications
10.5.3 Promptly back-up audit trail files to a centralized log server or media that is difficult to
alter"

NIST guidance for FISMA also says something similar (for example, look in NIST 800-92 doc). Overall, log protection and security are mentioned in many other regulations as well.

Q4: Privileged groups membership monitoring in AD one of the most important from my point of view. However I did not find effective way to monitor/report on changes in those groups. Any recommendations?

A4: This is indeed a tricky one which might take more space to answer than I have here; it might also take you 'beyond logs.' One good source of information is Randy Smith's site and, specifically, his webinar on 'Active Directory "Logging Gap"' (here somewhere) - which covers how to audit things of that sort when then native logging is not sufficient.

Q5: How I can learn what exactly I need to log?

A5: OMG, this is a $1,000,000 question :-) Let me answer "how can I learn" part and not the "what exactly I need to log part,"  (also see discussion on "MUST-DO Logging for PCI?") as it is actually answerable. To learn what you need to log, first ask "Why?" (and then see this) - basically establish what you want to accomplish with logs, catalogue your systems, figure how to tweak the logging knobs - and then do it!

Q6: How granular should logging be? What is your recommendation for enterprise servers like domain servers and Windows servers?

A6: Again, too long to answer here in details (it will become a subject of a longer blog post later), but some pointers follow: here for Windows (MS site also have a few recommendations on audit policies)

Q7: What is "more control" and what is "less control" that you mention in the webcast? Can you give an example?

A7: OK, I did say that "sometimes when you implement more controls, you actually have less control." What do I mean? If you buy a firewall (a network security control) and then - over time, of course - configure it with 7800 rules (!) that are supposed to give you control over who can and cannot access your network, you will not gain control over your environment. You will actually be less in control of who is touching your network, compared to, say, having only 20 rules.

Q8: What about mandated NIST controls for government systems? Auditing is a specific control for Moderate and High risk systems. What list of events do you recommend for auditing?

A8: This is too long to answer here, but NIST 800-92 Guide is a really good source of such info ("Guide to Computer Security Log Management [PDF]") Also, see my presentation on NIST 800-92 Guide in the Real World.

Q9: The issue that many organizations get stuck on, is the monitoring process, and defining what exceptions to monitor for? Is there guidance / framework for this? How much of it is system specific and how much is applicable generally to all systems?

A9: I outlined some general ideas back in 2004 via this presentation (note to self - update that to be more 2008-relevant); it is mostly general, but also has pointers to specific system. Keep in mind that it is focused on security, not operational monitoring (which is often no less important - in fact, often MORE important)

Enjoy! Sorry for being brief with some of the answers - I am woefully late with this even as they are...

Other questions that I answered in the past:

• More Log Management Questions - Answered!
• Some Burning Logging Questions - Answered!

ScienceLogic: How long have you been involved in InteropNet?

Katsev: I started at Coyote Point 3 years ago and InteropNet 2006 was my first “big” assignment.  This was the first time Coyote Point had put in a proposal to participate, so we were very excited when we were selected.

ScienceLogic: How long has Coyote Point been involved in Interop overall?

Katsev: We’ve been exhibiting at Interop for a number of years, and after seeing the InteropNet in action, we decided to submit a proposal in ‘06.  We were actually one of the first companies in the load balancing/traffic management space (we’ve been doing this for almost 10 years), so we have a lot of experience to share with InteropNet.

ScienceLogic: What is your role at Coyote Point?

My official title is “Engineering Project Manager”.  Basically, that means that I’m in charge of product releases and maintenance.  It sounds like a weird title for someone participating in InteropNet, but I’ve actually found it extremely useful since my position means that I don’t get to see our systems out in the field a lot.  We’ve added several features and have ideas for others just from my experiences at InteropNet.

ScienceLogic: What do the Coyote Point products do?

Katsev: Coyote Point makes a Traffic Management appliance called Equalizer.  What this means is that any traffic destined for a datacenter’s servers goes through our appliances and we make sure that the server which is best equipped to handle it, does.  Our systems sit between the clients and the servers and monitor the client traffic and the state of the servers.  If the clients start sending more traffic, we’ll balance it out so that no server is overloaded.  If one of the servers stops responding or starts responding very slowly, we’ll steer traffic away from that server.

ScienceLogic: In what way are your products being used as part of InteropNet?

Katsev: In the InteropNet, we’re utilizing a lot of our expertise:  We’re making sure that traffic is balanced and servers are redundant for show services such as DNS and SMTP.  We’re also using our geographic load balancing technology to ensure that the ScienceLogic EM7 appliances and some other internal NOC services are available from anywhere, with the lowest latency, with our SSL acceleration and GZIP compression technology.  Finally, we’re helping logistics in the NOC by allowing a physical separation between systems located in the NOC and those in an emergency rack outside of the NOC.  If either of these two locations were to fail, the network will continue operating without a glitch.

ScienceLogic: Are there any special considerations for Interop that cause you to deploy your systems there differently that any other place?

Katsev: Interop is definitely different than most of our customer installations.   One difference from a standard environment is that the network (at least this year) is one large flat network, with pieces carved out where extra security is needed.  Because of this, we can actually run our failover pairs of Equalizer systems in a non-standard configuration where the two peers are in different racks, or even on different floors.  That’s one of the things that I really like about InteropNet — it definitely brings new ideas to mind, which end up becoming ’special configuration’ white papers after the show.

ScienceLogic: Has InteropNet taught you anything that caused you to actually change your product?

Katsev: In addition to the failover configuration differences I mentioned above, participating in InteropNet has actually caused us to add several new features and allowed configurations.  One example is the “no-spoof” option for Layer 4 clusters.  Prior to the 2006 shows, we always ’spoofed’ the client’s IP address when talking to a server so that the server would see the client’s IP address instead of our own.  At Interop, we ran into a special configuration which would’ve been very difficult to set up in this manner, so our engineers added this feature, and it’s been very a very popular configuration with our customers ever since.

We have also had a couple of business relationships that extended outside of the show.  In 2006, we had a good experience using Spirent Communications gear to benchmark the network, so we ended up purchasing a couple of these systems to test our products.  More recently, we have found a way to bundle our Equalizer e350si load balancers with the ScienceLogic EM7 collector appliances to help ScienceLogic get the best performance in load balancing large quantities of syslog messages to be processed.  If it wasn’t for our participation in InteropNet, neither of these relationships would’ve happened.

ScienceLogic: What’s the best part of being involved with InteropNet?  What do you most look forward to?

Katsev: InteropNet is an amazing networking opportunity (no pun intended).  The group of engineers that put the network together every year is, well, amazing.  There is so much combined experience that any question instantly has several possible answers, and the best answer is chosen very quickly.  One of the ’sayings’ at Interop is “if you run into a problem, ask someone… we’ve probably seen that problem before… five times.”  One would think that being part of InteropNet is the same thing, year after year.  However, in the two years that I’ve been part of this (for four shows), there have been huge differences in the way that the network is designed and put together.  These are both because the vendors selected every year are different, and because the engineers who design the network change from year to year.  Somehow, though, when all is said and done, we have a network that works.

ScienceLogic: You don’t have to answer this one if you’re not comfortable… What would you like to see changed with the way things are done at InteropNet?

Katsev: This isn’t a cop-out… I really can’t think of anything I would do differently.  Sure, there are small problems that pop up sometimes, but every project has those, and the people at InteropNet are more than capable of figuring them all out.  In fact, I know that Interop started out as a show to test the interoperability of devices… but I’m still amazed that all of these devices actually talk to each other and “play nice” together.

ShareThis

During the presentation I mentioned in some depth how to perform fuzz testing, and what parts of an application should be fuzz testing targets. I also introduced an idea (that's not new) to help people who have never performed fuzz testing begin fuzz testing with very little cost and friction. The idea is to add a small layer of code to an application to automatically mutate untrusted data as it comes into an application; I called that code layer "a layer of hurt."

Before I continue, I want to point out that fuzzing is an SDL requirement, but the idea in this blog post is not an SDL requirement, it's just another way to help meet SDL fuzzing requirements.

Adding a layer of hurt, as shown in the picture below, is pretty simple as it involves adding code to an application to tweak data as it comes into an application. You can work out where to place the fuzzing code by looking at your threat models to see where data crosses trust boundaries. You could also simply grep the code looking for APIs that read data, for example:

• Read from files: fread, ReadFile
• Reading from sockets: recv, recvfrom
• For .NET code, any stream.Read

You get the picture.

The fuzzing code should appear right after the API that reads that data.

For example, C or C++ code that reads from a UDP socket and then fuzzes the data before it's consumed by the rest of the application might look like this:

Or, in C#, code that reads from an untrusted file:

In both code examples, Fuzz() mutates the incoming data. In the C++ case, the fuzzing code looks like this:

The sample C# and C++ fuzzing code is available as a ZIP file at the end of this post.

This code is an example of dumb-fuzzing, which is fuzzing with little or no regard for the data structure being manipulated. If you've never performed any kind of fuzz testing in the past, then you will probably find bugs with this simple fuzzing technique. Once you have weeded out the low-hanging bugs, you may need to turn your attention to smarter fuzzers. For example, in theory, this code would find few if any bugs in a PNG parser, because PNG files have a built in check-sum, so if you fuzz a PNG file, you'd have to recalculate the checksum to get decent code coverage.

When I showed this code during my presentation, I urged people to add it to their applications today if they currently don't do fuzz testing, and simply run their applications through their normal testing processes. Within three days of my presentation I received emails from people saying they had found bugs. I have no doubt others did too.

One of the comments I made during the session was,"If you can't spend the time on great fuzzing, fuzz anyway" and adding a "layer of hurt" is a reasonable start.

Please feel free to sound off if you have ideas to help improve the code and let us know what you think, either through email or comments to this post.

Formalize your Final Security Review (FSR) Process

A Final Security Review is your final security audit to ensure your software is secure enough to deliver to your customers. I will assume the idea of an FSR is a new concept and try to provide some FAQ-style detail on this topic.

Who is the FSR team? An FSR Team usually consists of a non-product-team security expert (for impartial perspective), a security representative from the product team, and individual representatives from the separate disciplines. However, that size team may not scale to your company. If that is the case, at a minimum, you should have an impartial “outsider” separate from the product team who understands the security requirements as well as the measurements used to validate them. This person along with a project manager can probably perform the bulk of the FSR with development or test leadership providing input as needed.

What is needed to do an FSR? All threat models should be revised to reflect the final product, the code should be complete, and all security-related testing should be completed and documented. In addition, everyone involved in the FSR should have full access to the bug database to review status or exceptions to security bugs.

What does an FSR team do?

1. Re-review threat models to verify all mitigations identified in those exercises were fixed or went through an exception process.
2. Verify that all security issues uncovered during the development process were fixed or granted exceptions by the appropriate people. This is where you verify whether the state of your security bugs meets the “bug bar” requirements you have defined for your products.
3. If there is any output from security tools that you have used to define requirements, the FSR team would verify that the results of the tools meet the security requirements.
4. Review all exceptions to verify that they approve these decisions in the context of the final product. If they identify risks associated with the exceptions, they should communicate those to the business ownership for a final decision before signoff. Any decisions related to known risks should also be reflected in the response plan for future reference.
5. Finally, there should be a final signoff exercise where all security people and project leadership jointly approve the decision of the Final Security Review.

How long does an FSR take? If done correctly, the FSR will likely take some time. You should schedule this review well in advance of your release date to give your FSR team some time to complete the review, push issues back to the product team, and respond to any serious issues that may be discovered.

Final security reviews are a crucial piece to your Security Development Lifecycle. It would be easy to encourage secure development in your team, but as you expand your process to include formal security requirements and begin enforcing those requirements, it is necessary to perform a final audit of your product before it is released. Your customers will thank you for taking the time to add this layer of quality control to your operations and you will likely save yourself some security embarrassment down the road by adding a FSR to the end of your product cycle.

Document security work for reference

After the FSR is complete, there is still work for the security team. The final FSR documentation should be archived along with the symbols and code that represents the finished project. This becomes the time-stamped “snapshot” of your product. Your post-release process should include archiving the following documents in an easily accessible location:

·         All final threat models for future reference.

·         Bug bars, tool settings, and test results related to your project and the supporting tools used to validate. These will be referenced and reused in the next product cycle.

·         All documented security bug exceptions. These need to be rolled into your next product cycle to ensure they are addressed.

·         The final symbols that reflect the product shipped should be archived.

·         The Final Security Report and project signoffs to validate your security audit activity

·         Your Incident Response Plan (discussed in the Crawl post). This must be accessible for quick reference if security incidents occur.

Archiving this evidence serves a few critical purposes: it shows historic evidence of the work you did to ensure a secure product, allows you to postmortem the results and improves your process each time, and reduces the amount of time your team will have to spend next time around by making the existing resources reusable.

In closing…

I hope this long series has provided some practical steps you can take to move your Security Development Lifecycle practices to the next level. At Microsoft, creating a lifecycle to match security development practices has faced a fair share of challenges. However, the investment and time has resulted in more secure products. We’ll continue refining how we execute the Security Development Lifecycle and hope to share those ideas with you along the way. We welcome your thoughts and questions as you start “Walking” with the SDL in your own company and look forward to seeing more secure products and customers as a result.

I’ve created a unique tag on the SDL Blog to cover this series. To get a full list of the related posts, click the “Crawl Walk Run” tag on the left column. I’ll post a Word document version of the full “Walk” series sometime in the next week.