Well, the results are in! Our servers nearly crashed thanks to the influx of responses, but, fortunately, that wasn't the case. Here are the results...]]> Thu, 28 Aug 2008 05:00:00 +0000 iso results piece question posed adoption weeks ago standard rule influx ISO 27001 Adoption Poll Results are In http://securityratty.com/article/40a098c4c848de62a0921d68f8cef2e7 http://securityratty.com/article/40a098c4c848de62a0921d68f8cef2e7 In eerily similar cases in the Netherlands and the United States, courts have recently grappled with the computer-security norm of "full disclosure," asking whether researchers should be permitted to disclose details of a fare-card vulnerability that allows people to ride the subway for free.

The "Oyster card" used on the London Tube was at issue in the Dutch case, and a similar fare card used on the Boston "T" was the center of the U.S. case. The Dutch court got it right, and the American court, in Boston, got it wrong from the start -- despite facing an open-and-shut case of First Amendment prior restraint.

The U.S. court has since seen the error of its ways -- but the damage is done. The MIT security researchers who were prepared to discuss their Boston findings at the DefCon security conference were prevented from giving their talk.

The ethics of full disclosure are intimately familiar to those of us in the computer-security field. Before full disclosure became the norm, researchers would quietly disclose vulnerabilities to the vendors -- who would routinely ignore them. Sometimes vendors would even threaten researchers with legal action if they disclosed the vulnerabilities.

Later on, researchers started disclosing the existence of a vulnerability but not the details. Vendors responded by denying the security holes' existence, or calling them just theoretical. It wasn't until full disclosure became the norm that vendors began consistently fixing vulnerabilities quickly. Now that vendors routinely patch vulnerabilities, researchers generally give them advance notice to allow them to patch their systems before the vulnerability is published. But even with this "responsible disclosure" protocol, it's the threat of disclosure that motivates them to patch their systems. Full disclosure is the mechanism (.pdf) by which computer security improves.

Outside of computer security, secrecy is much more the norm. Some security communities, like locksmiths, behave much like medieval guilds, divulging the secrets of their profession only to those within it. These communities hate open research, and have responded with surprising vitriol to researchers who have found serious vulnerabilities in bicycle locks, combination safes (.pdf), master-key systems and many other security devices.

Researchers have received a similar reaction from other communities more used to secrecy than openness. Researchers -- sometimes young students -- who discovered and published flaws in copyright-protection schemes, voting-machine security and now wireless access cards have all suffered recriminations and sometimes lawsuits for not keeping the vulnerabilities secret. When Christopher Soghoian created a website allowing people to print fake airline boarding passes, he got several unpleasant visits from the FBI.

This preference for secrecy comes from confusing a vulnerability with information about that vulnerability. Using secrecy as a security measure is fundamentally fragile. It assumes that the bad guys don't do their own security research. It assumes that no one else will find the same vulnerability. It assumes that information won't leak out even if the research results are suppressed. These assumptions are all incorrect.

The problem isn't the researchers; it's the products themselves. Companies will only design security as good as what their customers know to ask for. Full disclosure helps customers evaluate the security of the products they buy, and educates them in how to ask for better security. The Dutch court got it exactly right when it wrote: "Damage to NXP is not the result of the publication of the article but of the production and sale of a chip that appears to have shortcomings."

In a world of forced secrecy, vendors make inflated claims about their products, vulnerabilities don't get fixed, and customers are no wiser. Security research is stifled, and security technology doesn't improve. The only beneficiaries are the bad guys.

If you'll forgive the analogy, the ethics of full disclosure parallel the ethics of not paying kidnapping ransoms. We all know why we don't pay kidnappers: It encourages more kidnappings. Yet in every kidnapping case, there's someone -- a spouse, a parent, an employer -- with a good reason why, in this one case, we should make an exception.

The reason we want researchers to publish vulnerabilities is because that's how security improves. But in every case there's someone -- the Massachusetts Bay Transit Authority, the locksmiths, an election machine manufacturer -- who argues that, in this one case, we should make an exception.

We shouldn't. The benefits of responsibly publishing attacks greatly outweigh the potential harm. Disclosure encourages companies to build security properly rather than relying on shoddy design and secrecy, and discourages them from promising security based on their ability to threaten researchers. It's how we learn about security, and how we improve future security.

This essay previously appeared on Wired.com.

EDITED TO ADD (8/26): Matt Blaze has a good essay on the topic.

The "Oyster card" used on the London Tube was at issue in the Dutch case, and a similar fare card used on the Boston "T" was the center of the U.S. case. The Dutch court got it right, and the American court, in Boston, got it wrong from the start -- despite facing an open-and-shut case of First Amendment prior restraint.

The U.S. court has since seen the error of its ways -- but the damage is done. The MIT security researchers who were prepared to discuss their Boston findings at the DefCon security conference were prevented from giving their talk.

The ethics of full disclosure are intimately familiar to those of us in the computer-security field. Before full disclosure became the norm, researchers would quietly disclose vulnerabilities to the vendors -- who would routinely ignore them. Sometimes vendors would even threaten researchers with legal action if they disclosed the vulnerabilities.

Later on, researchers started disclosing the existence of a vulnerability but not the details. Vendors responded by denying the security holes' existence, or calling them just theoretical. It wasn't until full disclosure became the norm that vendors began consistently fixing vulnerabilities quickly. Now that vendors routinely patch vulnerabilities, researchers generally give them advance notice to allow them to patch their systems before the vulnerability is published. But even with this "responsible disclosure" protocol, it's the threat of disclosure that motivates them to patch their systems. Full disclosure is the mechanism (.pdf) by which computer security improves.

Outside of computer security, secrecy is much more the norm. Some security communities, like locksmiths, behave much like medieval guilds, divulging the secrets of their profession only to those within it. These communities hate open research, and have responded with surprising vitriol to researchers who have found serious vulnerabilities in bicycle locks, combination safes (.pdf), master-key systems and many other security devices.

Researchers have received a similar reaction from other communities more used to secrecy than openness. Researchers -- sometimes young students -- who discovered and published flaws in copyright-protection schemes, voting-machine security and now wireless access cards have all suffered recriminations and sometimes lawsuits for not keeping the vulnerabilities secret. When Christopher Soghoian created a website allowing people to print fake airline boarding passes, he got several unpleasant visits from the FBI.

This preference for secrecy comes from confusing a vulnerability with information about that vulnerability. Using secrecy as a security measure is fundamentally fragile. It assumes that the bad guys don't do their own security research. It assumes that no one else will find the same vulnerability. It assumes that information won't leak out even if the research results are suppressed. These assumptions are all incorrect.

The problem isn't the researchers; it's the products themselves. Companies will only design security as good as what their customers know to ask for. Full disclosure helps customers evaluate the security of the products they buy, and educates them in how to ask for better security. The Dutch court got it exactly right when it wrote: "Damage to NXP is not the result of the publication of the article but of the production and sale of a chip that appears to have shortcomings."

In a world of forced secrecy, vendors make inflated claims about their products, vulnerabilities don't get fixed, and customers are no wiser. Security research is stifled, and security technology doesn't improve. The only beneficiaries are the bad guys.

If you'll forgive the analogy, the ethics of full disclosure parallel the ethics of not paying kidnapping ransoms. We all know why we don't pay kidnappers: It encourages more kidnappings. Yet in every kidnapping case, there's someone -- a spouse, a parent, an employer -- with a good reason why, in this one case, we should make an exception.

The reason we want researchers to publish vulnerabilities is because that's how security improves. But in every case there's someone -- the Massachusetts Bay Transit Authority, the locksmiths, an election machine manufacturer -- who argues that, in this one case, we should make an exception.

We shouldn't. The benefits of responsibly publishing attacks greatly outweigh the potential harm. Disclosure encourages companies to build security properly rather than relying on shoddy design and secrecy, and discourages them from promising security based on their ability to threaten researchers. It's how we learn about security, and how we improve future security.

---

Bruce Schneier is Chief Security Technology Officer of BT Global Services and author of Beyond Fear: Thinking Sensibly About Security in an Uncertain World. You can read more of his writings on his website.

The first post described the problem: ASP.NET wasn't reporting inner exception stack traces.

The second post described my solution.

This post shows the code I used to solve the problem: a custom email provider for the Health Monitoring system in ASP.NET. Enjoy!

Here's the provider. Note that I opted *not* to build a buffering provider to keep things simple:

public class MyMailWebEventProvider : WebEventProvider
{
    string to;
    string from;
    string subjectPrefix;

    public override void Initialize(string name,
        NameValueCollection config)
    {
        base.Initialize(name, config);

        to = GetAndRemoveStringAttribute(config, "to", true);
        from = GetAndRemoveStringAttribute(config, "from", true);
        subjectPrefix = GetAndRemoveStringAttribute(config,
            "subjectPrefix", false);
    }
    public override void ProcessEvent(WebBaseEvent raisedEvent)
    {
        SendMail(raisedEvent);
    }

    private void SendMail(WebBaseEvent raisedEvent)
    {
        string subject = ComputeEmailSubject(raisedEvent);
        string body = ComputeEmailBody(raisedEvent);

        MailMessage msg = new MailMessage(from, to, subject, body);
        new SmtpClient().Send(msg);
    }

    private string ComputeEmailBody(WebBaseEvent raisedEvent)
    {
        WebRequestErrorEvent errorEvent =
            raisedEvent as WebRequestErrorEvent;
        if (null != errorEvent)
            return ErrorEventFormattingHelper.FormatRequestErrorEvent(errorEvent);
        else return raisedEvent.ToString();
    }

    private string ComputeEmailSubject(WebBaseEvent raisedEvent)
    {
        StringBuilder subjectBuilder = new StringBuilder();

        // surface some details in subject about error events
        WebBaseErrorEvent errorEvent = raisedEvent as WebBaseErrorEvent;
        if (null != errorEvent)
        {
            Exception unhandledException = errorEvent.ErrorException;

            // drill through reflection exceptions to show the root cause
            TargetInvocationException invocationException =
                unhandledException as TargetInvocationException;
            if (null != invocationException)
            {
                Exception innerException =
                    DrillIntoTargetInvocationException(invocationException);
                subjectBuilder.AppendFormat("{0}",
                    (innerException ?? invocationException).GetType().Name);
                if (null != innerException)
                    subjectBuilder.Append(" (via reflection)");
            }
            else subjectBuilder.Append(unhandledException.GetType().Name);
        }

        // if we've not got anything better
        // just show the event type in the subject
        if (0 == subjectBuilder.Length)
            subjectBuilder.AppendFormat("Event type: {0}",
                raisedEvent.GetType().Name);

        if (!string.IsNullOrEmpty(subjectPrefix)) {
            subjectBuilder.Insert(0, ' ');
            subjectBuilder.Insert(0, subjectPrefix);
        }
        return subjectBuilder.ToString();
    }

    /// <summary>
    /// Reflection often hides exception details, so we try to drill down
    /// through the plumbing exceptions to find a likely cause
    /// </summary>
    private Exception DrillIntoTargetInvocationException(
        TargetInvocationException outerException)
    {
        Exception innerException = outerException.InnerException;
        TargetInvocationException innerInvocationException =
            innerException as TargetInvocationException;
        if (null != innerInvocationException)
            return DrillIntoTargetInvocationException(innerInvocationException);
        else if (null != innerException)
            return innerException;
        else return null;
    }

    private static string GetAndRemoveStringAttribute(NameValueCollection config,
        string attributeName, bool required)
    {
        string value = config.Get(attributeName);
        if (required && string.IsNullOrEmpty(value))
            throw new ConfigurationErrorsException(string.Format(
                "Expected attribute {0}, which is missing or empty.",
                attributeName));
        config.Remove(attributeName);
        return value;
    }

    public override void Flush()
    {
        // nothing to do - this is not a buffering provider
    }

    public override void Shutdown()
    {
        // nothing to do here either
    }
}

Here's a helper class that formats the error messages the way I want to see them. Note that I've omitted some fields that I personally didn't care about, and I've reordered things a bit, so you might want to tweak this if you're going to use it in your own system.

internal static class ErrorEventFormattingHelper
{
    internal static string FormatRequestErrorEvent(
        WebRequestErrorEvent errorEvent)
    {
        CustomEventFormatter formatter = 
            new CustomEventFormatter();

        formatter.AppendLine(string.Format(
            "Unhandled Exception in {0}:",
            WebBaseEvent.ApplicationInformation
            .ApplicationVirtualPath));
        formatter.Indent();
        EmitExceptionAtAGlance(formatter, 
            errorEvent.ErrorException);
        formatter.RevertIndent();

        formatter.AppendLine();
        formatter.AppendLine("Exception stack trace(s):");
        EmitExceptionStackTrace(formatter, 
            errorEvent.ErrorException);

        formatter.AppendLine();
        formatter.AppendLine("Event information:");
        formatter.Indent();
        EmitEventInfo(formatter, errorEvent);
        formatter.RevertIndent();

        formatter.AppendLine();
        formatter.AppendLine("Application information:");
        formatter.Indent();
        EmitApplicationInfo(formatter, 
            WebBaseEvent.ApplicationInformation);
        formatter.RevertIndent();

        formatter.AppendLine();
        formatter.AppendLine("Process/thread information:");
        formatter.Indent();
        EmitProcessInfo(formatter, 
            errorEvent.ProcessInformation);
        formatter.RevertIndent();

        formatter.AppendLine();
        formatter.AppendLine("Request information:");
        formatter.Indent();
        EmitRequestInfo(formatter, 
            errorEvent.RequestInformation);
        formatter.RevertIndent();

        return formatter.ToString();
    }

    private static void EmitEventInfo(
        CustomEventFormatter formatter,
        WebBaseEvent theEvent)
    {
        formatter.AppendLine(string.Format(
            "Event code: {0}",
            theEvent.EventCode.ToString(
            CultureInfo.InvariantCulture)));
        formatter.AppendLine(string.Format(
            "Event message: {0}", 
            theEvent.Message));
        formatter.AppendLine(string.Format(
            "Event time: {0}", 
            theEvent.EventTime.ToString(
            CultureInfo.InvariantCulture)));
        formatter.AppendLine(string.Format(
            "Event ID: {0}", 
            theEvent.EventID.ToString("N", 
            CultureInfo.InvariantCulture)));
    }

    private static void EmitApplicationInfo(
        CustomEventFormatter formatter, 
        WebApplicationInformation appInfo)
    {
        formatter.AppendLine(string.Format(
            "Application domain: {0}", 
            appInfo.ApplicationDomain));
        formatter.AppendLine(string.Format(
            "Application Virtual Path: {0}", 
            appInfo.ApplicationVirtualPath));
        formatter.AppendLine(string.Format(
            "Application Physical Path: {0}", 
            appInfo.ApplicationPath));
    }

    private static void EmitProcessInfo(
        CustomEventFormatter formatter, 
        WebProcessInformation webProcessInfo)
    {
        formatter.AppendLine(string.Format(
            "Process ID: {0}", 
            webProcessInfo.ProcessID.ToString(
            CultureInfo.InvariantCulture)));
        formatter.AppendLine(string.Format(
            "Process name: {0}", 
            webProcessInfo.ProcessName));
        formatter.AppendLine(string.Format(
            "Account name: {0}", 
            webProcessInfo.AccountName));
    }

    private static void EmitRequestInfo(
        CustomEventFormatter formatter, 
        WebRequestInformation webRequestInfo)
    {
        string name = null;
        if (webRequestInfo.Principal != null)
            name = webRequestInfo.Principal.Identity.Name;

        formatter.AppendLine(string.Format(
            "Request URL: {0}", 
            webRequestInfo.RequestUrl));
        formatter.AppendLine(string.Format(
            "Request path: {0}", 
            webRequestInfo.RequestPath));
        formatter.AppendLine(string.Format(
            "User name: {0}", 
            name ?? "[ANONYMOUS]"));
        formatter.AppendLine(string.Format(
            "User host address: {0}", 
            webRequestInfo.UserHostAddress));
    }

    private static void EmitExceptionAtAGlance(
        CustomEventFormatter formatter, 
        Exception exception)
    {
        formatter.AppendLine(string.Format(
            "Type: {0}", 
            exception.GetType().Name));
        formatter.AppendLine(string.Format(
            "Message: {0}", 
            exception.Message));
        if (null != exception.InnerException)
        {
            formatter.Indent();
            formatter.AppendLine("-->Inner Exception");
            EmitExceptionAtAGlance(formatter, 
                exception.InnerException);
            formatter.RevertIndent();
        }
    }

    private static void EmitExceptionStackTrace(
        CustomEventFormatter formatter, Exception exception)
    {
        formatter.AppendLine(exception.StackTrace);

        if (null != exception.InnerException)
        {
            // no point indenting
            // since stack traces typically wrap like crazy
            formatter.AppendLine();
            formatter.AppendLine("-->Inner exception stack trace:");
            EmitExceptionStackTrace(formatter, exception.InnerException);
        }
    }
}

And finally, here's a helper class that manages indentation levels for the output email message:

public class CustomEventFormatter
{
    const int TabSpaces = 4;

    StringBuilder sb = new StringBuilder();
    private int indentLevel;
    private bool startingNewLine = true;

    public void Indent()
    {
        ++indentLevel;
    }

    public void RevertIndent()
    {
        if (indentLevel > 0)
            --indentLevel;
    }

    public void Append(string text)
    {
        if (startingNewLine)
            EmitIndent();
        sb.Append(text);
        startingNewLine = false;
    }

    public void AppendLine(string lineOfText)
    {
        if (startingNewLine)
            EmitIndent();
        EmitIndent();
        sb.AppendLine(lineOfText);
        startingNewLine = true;
    }

    private void EmitIndent()
    {
        sb.Append(' ', TabSpaces * indentLevel);
    }

    public void AppendLine()
    {
        AppendLine(string.Empty);
    }

    public override string ToString()
    {
        return sb.ToString();
    }
}

Build this into a library application and reference it in your config file. Here's an example:

<healthMonitoring>
  <providers>
    <add name="mailWebEventProvider"
         type="MyMailWebEventProvider"
         to="web-fault@fabrikam.com"
         from="website@fabrikam.com"
         buffer="false"
         subjectPrefix="[WEB-ERROR]"
       />
  </providers>
  <rules>
    <add name="All Errors Email"
         eventName="All Errors"
         provider="mailWebEventProvider"
         profile="Default"
         minInstances="1"
         maxLimit="Infinite"
         minInterval="00:01:00"
         custom=""/>
  </rules>
</healthMonitoring>

What I found is that the two providers were using entirely different means to format the events! The email provider calls ToString(bool, bool) on the event to ask it to format itself. But the EventLogWebEventProvider does its own formatting of individual fields of the event. Indeed, its ProcessEvent method has a big list of checks:

if (eventRaised is WebBaseErrorEvent)
    AddErrorStuff();
if (eventRaised is WebAuthenticationSuccessAuditEvent)
    AddLogonStuff();

So it seemed like a better approach would be to write my own provider. I left the event log provider alone, and I wrote a custom email provider to display errors in a more useful way. This also allowed me to drop some fields from the event report that aren't useful for us. And I was able to construct a much more concise and useful subject line (the subject line that SimpleMailWebEventProvider uses is rather clunky since it assumes it might be spitting out a whole bunch of buffered events in one go).

Not only does my provider include the stack traces for all of the exceptions in the chain, but in the subject line, I display the type of error that is at the root of the problem. So if I am formatting a TargetInvocationException, I drill into its InnerException chain until I find a different exception type, and display that exception type instead.

Oh, one other benefit of building the custom provider instead of using a custom WebEvent was that I was then able to remove the Error handler from global.asax. All I had to do was replace the SimpleMailWebEventProvider with my own provider, and I got the behavior I wanted. Now my email notifications include detailed stack traces.

I'll post the code for this provider once it's run for a little while in production and I'm satisfied that it works reasonably well.

ASP.NET's health monitoring system does list the inner exceptions (apparently up to a maximum depth of two, from spelunking the code with reflector), but it does not emit the stack traces for these exceptions, which would be really helpful. I've spent some time this morning trying to figure out how I'd customize things to emit this, and it looks like what I'd have to do is catch the exception and generate a custom WebEvent that overrides ToString(bool, bool) and does everything that WebRequestErrorEvent does, but also generate the inner stack trace.

That seems a bit ugly. A search for "ASP.NET web event inner exception stack trace" yielded no interesting results, so if you've dealt with this and have a cleaner solution, let me know. I'll post my solution once I get it worked out.

Formalize your Final Security Review (FSR) Process

A Final Security Review is your final security audit to ensure your software is secure enough to deliver to your customers. I will assume the idea of an FSR is a new concept and try to provide some FAQ-style detail on this topic.

Who is the FSR team? An FSR Team usually consists of a non-product-team security expert (for impartial perspective), a security representative from the product team, and individual representatives from the separate disciplines. However, that size team may not scale to your company. If that is the case, at a minimum, you should have an impartial “outsider” separate from the product team who understands the security requirements as well as the measurements used to validate them. This person along with a project manager can probably perform the bulk of the FSR with development or test leadership providing input as needed.

What is needed to do an FSR? All threat models should be revised to reflect the final product, the code should be complete, and all security-related testing should be completed and documented. In addition, everyone involved in the FSR should have full access to the bug database to review status or exceptions to security bugs.

What does an FSR team do?

1. Re-review threat models to verify all mitigations identified in those exercises were fixed or went through an exception process.
2. Verify that all security issues uncovered during the development process were fixed or granted exceptions by the appropriate people. This is where you verify whether the state of your security bugs meets the “bug bar” requirements you have defined for your products.
3. If there is any output from security tools that you have used to define requirements, the FSR team would verify that the results of the tools meet the security requirements.
4. Review all exceptions to verify that they approve these decisions in the context of the final product. If they identify risks associated with the exceptions, they should communicate those to the business ownership for a final decision before signoff. Any decisions related to known risks should also be reflected in the response plan for future reference.
5. Finally, there should be a final signoff exercise where all security people and project leadership jointly approve the decision of the Final Security Review.

How long does an FSR take? If done correctly, the FSR will likely take some time. You should schedule this review well in advance of your release date to give your FSR team some time to complete the review, push issues back to the product team, and respond to any serious issues that may be discovered.

Final security reviews are a crucial piece to your Security Development Lifecycle. It would be easy to encourage secure development in your team, but as you expand your process to include formal security requirements and begin enforcing those requirements, it is necessary to perform a final audit of your product before it is released. Your customers will thank you for taking the time to add this layer of quality control to your operations and you will likely save yourself some security embarrassment down the road by adding a FSR to the end of your product cycle.

Document security work for reference

After the FSR is complete, there is still work for the security team. The final FSR documentation should be archived along with the symbols and code that represents the finished project. This becomes the time-stamped “snapshot” of your product. Your post-release process should include archiving the following documents in an easily accessible location:

·         All final threat models for future reference.

·         Bug bars, tool settings, and test results related to your project and the supporting tools used to validate. These will be referenced and reused in the next product cycle.

·         All documented security bug exceptions. These need to be rolled into your next product cycle to ensure they are addressed.

·         The final symbols that reflect the product shipped should be archived.

·         The Final Security Report and project signoffs to validate your security audit activity

·         Your Incident Response Plan (discussed in the Crawl post). This must be accessible for quick reference if security incidents occur.

Archiving this evidence serves a few critical purposes: it shows historic evidence of the work you did to ensure a secure product, allows you to postmortem the results and improves your process each time, and reduces the amount of time your team will have to spend next time around by making the existing resources reusable.

In closing…

I hope this long series has provided some practical steps you can take to move your Security Development Lifecycle practices to the next level. At Microsoft, creating a lifecycle to match security development practices has faced a fair share of challenges. However, the investment and time has resulted in more secure products. We’ll continue refining how we execute the Security Development Lifecycle and hope to share those ideas with you along the way. We welcome your thoughts and questions as you start “Walking” with the SDL in your own company and look forward to seeing more secure products and customers as a result.

I’ve created a unique tag on the SDL Blog to cover this series. To get a full list of the related posts, click the “Crawl Walk Run” tag on the left column. I’ll post a Word document version of the full “Walk” series sometime in the next week.