So I put together a simple class that holds an XmlDocument and implements ISerializable and called it SerializableXmlDocument. I'm sharing the source code here in the hopes that

a) somebody will find it useful, and

b) somebody smarter than I am will point out how I screwed it up and help me make it better.

SerializableXmlDocument includes implicit conversion operators to make it easy to convert to/from an XmlDocument. It holds the actual document in a property called Value. This "isomorph" pattern is one that I picked up from Craig.

While writing this code, I also wrote a helpful extension method for getting a byte array out of a MemoryStream that is exactly the length of the data written to the stream so far (CopyUpToSeekPointer). So don't go looking in the docs for MemoryStream for this method :) This is obviously not the most efficient way to consume bytes written to a MemoryStream since it copies the data into a new byte array, but it's very convenient in many scenarios.

Here is SerializableXmlDocument.cs:

using System;
using System.Runtime.Serialization;
using System.Xml;
using System.IO;

namespace Pluralsight.Samples
{
    [Serializable]
    public class SerializableXmlDocument : ISerializable
    {
        public SerializableXmlDocument() { }
        public SerializableXmlDocument(XmlDocument value)
        {
            this.Value = value;
        }

        public XmlDocument Value { get; set; }

        #region ISerializable implementation
        public SerializableXmlDocument(SerializationInfo info,
                                       StreamingContext context)
        {
            byte[] serializedData = (byte[])info.GetValue("doc",
                typeof(byte[]));
            if (null != serializedData)
                this.Value = Deserialize(serializedData);
        }

        public void GetObjectData(SerializationInfo info,
                                  StreamingContext context)
        {
            byte[] serializedData = null;
            if (null != Value)
                serializedData = Serialize(Value);
            info.AddValue("doc", serializedData);
        }
        #endregion

        #region implicit conversion to/from XmlDocument
        public static implicit operator SerializableXmlDocument(
            XmlDocument doc)
        {
            return new SerializableXmlDocument(doc);
        }
        public static implicit operator XmlDocument(
            SerializableXmlDocument sdoc)
        {
            return sdoc.Value;
        }
        #endregion

        #region Xml serialization helper methods
        private static byte[] Serialize(XmlDocument doc)
        {
            MemoryStream stream = new MemoryStream();
            doc.Save(stream);
            return stream.CopyUpToSeekPointer();
        }
        private static XmlDocument Deserialize(byte[] serializedData)
        {
            XmlDocument doc = new XmlDocument();
            doc.Load(new MemoryStream(serializedData, false));
            return doc;
        }
        #endregion
    }
}

...and here's the CopyUpToSeekPointer extension method for MemoryStream:

using System;
using System.IO;

namespace Pluralsight.Samples
{
    public static class MemoryStreamExtensionMethods
    {
        public static byte[] CopyUpToSeekPointer(
            this MemoryStream stream)
        {
            // copy only the part of the buffer
            // that contains the serialized document
            long length = stream.Position;
            byte[] buffer = stream.GetBuffer();
            byte[] result = new byte[length];
            for (int i = 0; i < length; ++i)
                result[i] = buffer[i];
            return result;
        }
    }
}

...and here's a sample object that uses SerializableXmlDocument:

using System;

namespace Pluralsight.Samples
{
    [Serializable]
    public class Item
    {
        public string Name { get; set; }
        public SerializableXmlDocument Data { get; set; }

        public void Print()
        {
            Console.WriteLine("Name: {0}", Name);
            Console.WriteLine(Data.Value.OuterXml);
        }
    }
}

...and here's a sample program that creates an instance of Item, serializes it, then deserializes it, printing diagnostics along the way to show that it's working properly.

using System;
using System.Xml;
using System.Runtime.Serialization.Formatters.Binary;
using System.IO;
using Pluralsight.Samples;

class DemoProgram
{
    static void Main(string[] args)
    {
        XmlDocument doc = new XmlDocument();
        doc.LoadXml("<root><child>text</child></root>");

        Item item = new Item
        {
            Name = "Testing 123",
            Data = doc,
        };

        // print object before serialization
        item.Print();

        BinaryFormatter formatter = new BinaryFormatter();
        MemoryStream stream = new MemoryStream();
        formatter.Serialize(stream, item);

        byte[] serializedItem = stream.CopyUpToSeekPointer();

        Console.WriteLine("Serialized data (base64): {0}",
            Convert.ToBase64String(serializedItem));

        item = (Item)formatter.Deserialize(
            new MemoryStream(serializedItem, false));

        // print object after deserialization
        item.Print();
    }
}

Here's the output of the previous sample program:

Flame away!

Bind("AmountCharged", "{0:C}")

While this displays just as you'd expect (e.g., $200), it doesn't do so well when you submit an edit that includes the same value ($200):

Input string was not in a correct format.

I searched around and didn't find much in the way of a clean solution, but I did solve the problem with just a few lines of code. The trick is to handle the data-bound control's Updating event. Since I was working with a GridView, my solution looked a bit like this:

<asp:GridView DataSourceID='myDataSource'
              OnRowUpdating='FixFormatting'
              AutoGenerateColumns='false'
              CellPadding="3" ...>

Notice the OnRowUpdating handler that I've installed in my grid view. That code looks like this:

protected void FixFormatting(object sender, GridViewUpdateEventArgs args)
{
    decimal amountPaid = ParseDecimal((string)args.NewValues["AmountPaid"]);
    args.NewValues["AmountPaid"] = amountPaid;
}

When you handle this event, you're given a dictionary of old and new values, which appear to come directly from the controls (in my case, a TextBox was used to gather the updated data AmountPaid, so the type of object that I found in NewValues["AmountPaid"] was a string. I wrote a little helper method called ParseDecimal that parses a string into a decimal value, allowing currency characters, decimal points, and thousands separators. I also allowed a blank value to indicate zero:

public static decimal ParseDecimal(string value)
{
    if (string.IsNullOrEmpty(value))
        return 0;
    return Decimal.Parse(value,
        NumberStyles.AllowThousands |
        NumberStyles.AllowDecimalPoint |
        NumberStyles.AllowCurrencySymbol,
        CultureInfo.InstalledUICulture);
}

This solved the problem quite nicely. Now two-way binding works with formatted data.

The first post described the problem: ASP.NET wasn't reporting inner exception stack traces.

The second post described my solution.

This post shows the code I used to solve the problem: a custom email provider for the Health Monitoring system in ASP.NET. Enjoy!

Here's the provider. Note that I opted *not* to build a buffering provider to keep things simple:

public class MyMailWebEventProvider : WebEventProvider
{
    string to;
    string from;
    string subjectPrefix;

    public override void Initialize(string name,
        NameValueCollection config)
    {
        base.Initialize(name, config);

        to = GetAndRemoveStringAttribute(config, "to", true);
        from = GetAndRemoveStringAttribute(config, "from", true);
        subjectPrefix = GetAndRemoveStringAttribute(config,
            "subjectPrefix", false);
    }
    public override void ProcessEvent(WebBaseEvent raisedEvent)
    {
        SendMail(raisedEvent);
    }

    private void SendMail(WebBaseEvent raisedEvent)
    {
        string subject = ComputeEmailSubject(raisedEvent);
        string body = ComputeEmailBody(raisedEvent);

        MailMessage msg = new MailMessage(from, to, subject, body);
        new SmtpClient().Send(msg);
    }

    private string ComputeEmailBody(WebBaseEvent raisedEvent)
    {
        WebRequestErrorEvent errorEvent =
            raisedEvent as WebRequestErrorEvent;
        if (null != errorEvent)
            return ErrorEventFormattingHelper.FormatRequestErrorEvent(errorEvent);
        else return raisedEvent.ToString();
    }

    private string ComputeEmailSubject(WebBaseEvent raisedEvent)
    {
        StringBuilder subjectBuilder = new StringBuilder();

        // surface some details in subject about error events
        WebBaseErrorEvent errorEvent = raisedEvent as WebBaseErrorEvent;
        if (null != errorEvent)
        {
            Exception unhandledException = errorEvent.ErrorException;

            // drill through reflection exceptions to show the root cause
            TargetInvocationException invocationException =
                unhandledException as TargetInvocationException;
            if (null != invocationException)
            {
                Exception innerException =
                    DrillIntoTargetInvocationException(invocationException);
                subjectBuilder.AppendFormat("{0}",
                    (innerException ?? invocationException).GetType().Name);
                if (null != innerException)
                    subjectBuilder.Append(" (via reflection)");
            }
            else subjectBuilder.Append(unhandledException.GetType().Name);
        }

        // if we've not got anything better
        // just show the event type in the subject
        if (0 == subjectBuilder.Length)
            subjectBuilder.AppendFormat("Event type: {0}",
                raisedEvent.GetType().Name);

        if (!string.IsNullOrEmpty(subjectPrefix)) {
            subjectBuilder.Insert(0, ' ');
            subjectBuilder.Insert(0, subjectPrefix);
        }
        return subjectBuilder.ToString();
    }

    /// <summary>
    /// Reflection often hides exception details, so we try to drill down
    /// through the plumbing exceptions to find a likely cause
    /// </summary>
    private Exception DrillIntoTargetInvocationException(
        TargetInvocationException outerException)
    {
        Exception innerException = outerException.InnerException;
        TargetInvocationException innerInvocationException =
            innerException as TargetInvocationException;
        if (null != innerInvocationException)
            return DrillIntoTargetInvocationException(innerInvocationException);
        else if (null != innerException)
            return innerException;
        else return null;
    }

    private static string GetAndRemoveStringAttribute(NameValueCollection config,
        string attributeName, bool required)
    {
        string value = config.Get(attributeName);
        if (required && string.IsNullOrEmpty(value))
            throw new ConfigurationErrorsException(string.Format(
                "Expected attribute {0}, which is missing or empty.",
                attributeName));
        config.Remove(attributeName);
        return value;
    }

    public override void Flush()
    {
        // nothing to do - this is not a buffering provider
    }

    public override void Shutdown()
    {
        // nothing to do here either
    }
}

Here's a helper class that formats the error messages the way I want to see them. Note that I've omitted some fields that I personally didn't care about, and I've reordered things a bit, so you might want to tweak this if you're going to use it in your own system.

internal static class ErrorEventFormattingHelper
{
    internal static string FormatRequestErrorEvent(
        WebRequestErrorEvent errorEvent)
    {
        CustomEventFormatter formatter = 
            new CustomEventFormatter();

        formatter.AppendLine(string.Format(
            "Unhandled Exception in {0}:",
            WebBaseEvent.ApplicationInformation
            .ApplicationVirtualPath));
        formatter.Indent();
        EmitExceptionAtAGlance(formatter, 
            errorEvent.ErrorException);
        formatter.RevertIndent();

        formatter.AppendLine();
        formatter.AppendLine("Exception stack trace(s):");
        EmitExceptionStackTrace(formatter, 
            errorEvent.ErrorException);

        formatter.AppendLine();
        formatter.AppendLine("Event information:");
        formatter.Indent();
        EmitEventInfo(formatter, errorEvent);
        formatter.RevertIndent();

        formatter.AppendLine();
        formatter.AppendLine("Application information:");
        formatter.Indent();
        EmitApplicationInfo(formatter, 
            WebBaseEvent.ApplicationInformation);
        formatter.RevertIndent();

        formatter.AppendLine();
        formatter.AppendLine("Process/thread information:");
        formatter.Indent();
        EmitProcessInfo(formatter, 
            errorEvent.ProcessInformation);
        formatter.RevertIndent();

        formatter.AppendLine();
        formatter.AppendLine("Request information:");
        formatter.Indent();
        EmitRequestInfo(formatter, 
            errorEvent.RequestInformation);
        formatter.RevertIndent();

        return formatter.ToString();
    }

    private static void EmitEventInfo(
        CustomEventFormatter formatter,
        WebBaseEvent theEvent)
    {
        formatter.AppendLine(string.Format(
            "Event code: {0}",
            theEvent.EventCode.ToString(
            CultureInfo.InvariantCulture)));
        formatter.AppendLine(string.Format(
            "Event message: {0}", 
            theEvent.Message));
        formatter.AppendLine(string.Format(
            "Event time: {0}", 
            theEvent.EventTime.ToString(
            CultureInfo.InvariantCulture)));
        formatter.AppendLine(string.Format(
            "Event ID: {0}", 
            theEvent.EventID.ToString("N", 
            CultureInfo.InvariantCulture)));
    }

    private static void EmitApplicationInfo(
        CustomEventFormatter formatter, 
        WebApplicationInformation appInfo)
    {
        formatter.AppendLine(string.Format(
            "Application domain: {0}", 
            appInfo.ApplicationDomain));
        formatter.AppendLine(string.Format(
            "Application Virtual Path: {0}", 
            appInfo.ApplicationVirtualPath));
        formatter.AppendLine(string.Format(
            "Application Physical Path: {0}", 
            appInfo.ApplicationPath));
    }

    private static void EmitProcessInfo(
        CustomEventFormatter formatter, 
        WebProcessInformation webProcessInfo)
    {
        formatter.AppendLine(string.Format(
            "Process ID: {0}", 
            webProcessInfo.ProcessID.ToString(
            CultureInfo.InvariantCulture)));
        formatter.AppendLine(string.Format(
            "Process name: {0}", 
            webProcessInfo.ProcessName));
        formatter.AppendLine(string.Format(
            "Account name: {0}", 
            webProcessInfo.AccountName));
    }

    private static void EmitRequestInfo(
        CustomEventFormatter formatter, 
        WebRequestInformation webRequestInfo)
    {
        string name = null;
        if (webRequestInfo.Principal != null)
            name = webRequestInfo.Principal.Identity.Name;

        formatter.AppendLine(string.Format(
            "Request URL: {0}", 
            webRequestInfo.RequestUrl));
        formatter.AppendLine(string.Format(
            "Request path: {0}", 
            webRequestInfo.RequestPath));
        formatter.AppendLine(string.Format(
            "User name: {0}", 
            name ?? "[ANONYMOUS]"));
        formatter.AppendLine(string.Format(
            "User host address: {0}", 
            webRequestInfo.UserHostAddress));
    }

    private static void EmitExceptionAtAGlance(
        CustomEventFormatter formatter, 
        Exception exception)
    {
        formatter.AppendLine(string.Format(
            "Type: {0}", 
            exception.GetType().Name));
        formatter.AppendLine(string.Format(
            "Message: {0}", 
            exception.Message));
        if (null != exception.InnerException)
        {
            formatter.Indent();
            formatter.AppendLine("-->Inner Exception");
            EmitExceptionAtAGlance(formatter, 
                exception.InnerException);
            formatter.RevertIndent();
        }
    }

    private static void EmitExceptionStackTrace(
        CustomEventFormatter formatter, Exception exception)
    {
        formatter.AppendLine(exception.StackTrace);

        if (null != exception.InnerException)
        {
            // no point indenting
            // since stack traces typically wrap like crazy
            formatter.AppendLine();
            formatter.AppendLine("-->Inner exception stack trace:");
            EmitExceptionStackTrace(formatter, exception.InnerException);
        }
    }
}

And finally, here's a helper class that manages indentation levels for the output email message:

public class CustomEventFormatter
{
    const int TabSpaces = 4;

    StringBuilder sb = new StringBuilder();
    private int indentLevel;
    private bool startingNewLine = true;

    public void Indent()
    {
        ++indentLevel;
    }

    public void RevertIndent()
    {
        if (indentLevel > 0)
            --indentLevel;
    }

    public void Append(string text)
    {
        if (startingNewLine)
            EmitIndent();
        sb.Append(text);
        startingNewLine = false;
    }

    public void AppendLine(string lineOfText)
    {
        if (startingNewLine)
            EmitIndent();
        EmitIndent();
        sb.AppendLine(lineOfText);
        startingNewLine = true;
    }

    private void EmitIndent()
    {
        sb.Append(' ', TabSpaces * indentLevel);
    }

    public void AppendLine()
    {
        AppendLine(string.Empty);
    }

    public override string ToString()
    {
        return sb.ToString();
    }
}

Build this into a library application and reference it in your config file. Here's an example:

<healthMonitoring>
  <providers>
    <add name="mailWebEventProvider"
         type="MyMailWebEventProvider"
         to="web-fault@fabrikam.com"
         from="website@fabrikam.com"
         buffer="false"
         subjectPrefix="[WEB-ERROR]"
       />
  </providers>
  <rules>
    <add name="All Errors Email"
         eventName="All Errors"
         provider="mailWebEventProvider"
         profile="Default"
         minInstances="1"
         maxLimit="Infinite"
         minInterval="00:01:00"
         custom=""/>
  </rules>
</healthMonitoring>

I realize that security is a system level issue and it takes a long time to change things at that level, but what's more concerning to me is that the typical infosec mindset remains the same. Should we be surprised by rampant phishing and fraud? I am frankly surprised the numbers are so low given the opportunities that the attackers have via the glacial pace of security improvements. Its been three years since that list and I could write the same exact one today for SOAP, REST, SOA, Web 2.0 whatever. Maybe the main reason, beyond failure of imagination, why infosec is so far behind developers is that infosec lacks tools. Developers automate everything possible. Security doesn't. The most promising thing about static analysis is not the ability to find everything, its the ability to find many important things in an automated way. Infosec needs to stop giving people fish and teaching people to fish. Look at Fortify's vulncat site which has a Taxonomy of Coding Errors. Fortify's Seven (plus one) pernicious kingdoms are:

The trick to doing this is not to hardcode things like host, port, etc. for your SMTP server when you use System.Net.Mail to send mail. Instead, use the default ctor for SmtpClient as I've done in the code below.

static void Main(string[] args)
{
    // note the use of the MailAddress class
    // this allows me to specify display names as well as email addresses
    MailAddress from = new MailAddress("admin@fabrikam.com", "Fabrikam Website");
    MailAddress to = new MailAddress("mari@fabrikam.com", "Mari Joyce");

    MailMessage msg = new MailMessage(from, to);
    msg.Subject  = "Testing 123";
    msg.Body = "This is only a test!";

    // note use of default ctor
    // this looks in config to figure out how to send mail
    new SmtpClient().Send(msg);
}

What you're telling .NET by using the default ctor for SmtpClient is, "please use my config file to figure out how to send mail". Now you can use the system.net/mailSettings/smtp section in config to specify the details of your mail server, and all of the code in your app that is written to use the default SmtpClient ctor will inherit these settings. Here's an example of what the config on a production server might look like (if you put passwords in your config files, be sure to encrypt those sections):

<configuration>
  <system.net>
    <mailSettings>
      <smtp deliveryMethod="Network">
        <network host="mail.fabrikam.com"
                 port="25"
                 userName="WebsiteMailAccount"
                 password="whatever"/>
      </smtp>
    </mailSettings>
  </system.net>
</configuration>

During development, I use different settings because I don't usually want to deal with the hassle of installing an SMTP server on my development box. Instead, I want email messages delivered as individual files in a directory on my hard drive (I always have a c:\mail directory on my development box for just this purpose):

<configuration>
  <system.net>
    <mailSettings>
      <smtp deliveryMethod="SpecifiedPickupDirectory">
        <specifiedPickupDirectory pickupDirectoryLocation="c:\mail"/>
      </smtp>
    </mailSettings>
  </system.net>
</configuration>

Now when I run the program above, I get a .EML file in my c:\mail directory:

Outlook Express is normally registered as the viewer for .EML files, so double-click the file to view it:

If you've never seen this method of simulating email before, I hope you find it as useful as I have. Happy coding!

In this series I am going to talk about “Walking” with the SDL. Walking is the point where your security development practices become a lifecycle – a repeatable, mostly reusable process that makes security a part of your development culture. To relate the analogy to SDL a bit more closely, think of crawling as the “SD” in SDL. For this post, we’ll talk about walking – or adding the “L” in SDL.

I will be covering quite a bit on this topic, so I intend to split it up in to a multi-part series over a few days. I’ll condense it all into one big doc at the end. In Part One, I will review “crawling” and the foundation you need to have in place as well as discuss getting management approval. In Part Two we’ll cover the topic of expanding your security training. In the additional posts, we’ll discuss formalizing requirements, reusing threat modeling and attack surface review data, the importance of final security reviews, and managing post-release documentation. All of these are components to “walking” with the SDL.

Before I jump into detailing what you can do to “walk” with the SDL, let’s look back at a snapshot of what you should already have in place from learning to “crawl.” At a high level, crawling involved three components. Each of these components requires specific activities or tools that your team must implement to begin developing secure code:

1.       Detailed awareness of your architecture and its attack surface.

a.       Threat Modeling

2.       Tools that will perform security analysis on your application.

a.       Strengthen compiler defenses

b.      Use code analysis or static analysis tools such as PREfast, FxCop, AppVerif

c.       Build a strong fuzz testing capability

3.       Results that show how the analysis resulted in improved security

a.       Response planning and response process in place

b.      Use bugs to gather evidence and show that your work improved security

Think of these pieces as the “gross motor skills” you need to start walking. You should already be using these components and have reached a conscious decision to start building a lifecycle around your secure development practices. As you start figuring out how to “walk”, I want to point out that each of the concepts I discuss in this post is a critical component of the Microsoft Security Development Lifecycle. Adopting the SDL in your company involves a combination of integrating the existing SDL principles and the creating of unique requirements and components specific to your environment to build your own Security Development Lifecycle.

With that in place, let’s start talking about what it means to “Walk with SDL.”

Obtain Management Approval/Endorsement

Creating a Security Development Lifecycle will cost time and money. In addition, it will likely require some process changes. In most organizations, this change will not happen unless you obtain the management approval and endorsement necessary to compel the organization to act.

The key to successfully pitching SDL to your management can be found in the data you have been accumulating during the “crawl” phase. As you may recall from my crawling post, the simplest way to create evidence that clearly illustrates improved application security is to “mine” the data from your bug database. Connecting those bugs to known security vulnerabilities or to what would have been bad security issues that were avoided by fixing them in development is a powerful story. Of course your pitch should include other necessary components like anticipated costs, new software acquisition, possible vendor and consulting contracts and anticipated return on investment.

However, the heart of your argument will be the story you tell. The story is quite simply “If we hadn’t done this basic work in security, here is what we would have missed and how much it would have hurt…” followed by “if we continue to expand our security practices and make them a part of our process, we can better predict measurable security improvements that reduce the likelihood of future risks.”

The new SDL website [http://www.microsoft.com/sdl] provides some valuable reference material on the Business Case for SDL. I would recommend that looking through that information for some good supporting material. In Part Two, I will discuss expanding your security training as another component of “walking” with SDL.

I’d like to hear if anyone is using the concept of “crawling” and “walking” to implement SDL in your company.

?        What unique challenges are you facing as you try to push for SDL adoption?

?        What have you used to successfully communicate the importance of security to your management?