• ValidationSummary controls look at the ErrorMessage field to figure out what to display, so always use ErrorMessage in a verbose enough way that it will be helpful from a ValidationSummary control.
• If you need a shorter message to display inline (i.e., where the validation control is on the form, as opposed to the ValidationSummary) use the body of the control to define it.

In the past, I've used RequiredFieldValidator controls on my web forms to remind users that certain fields are required. I would set the ErrorMessage to something vanilla like, "This field is required", or even something simpler like "*" (an asterisk) if I didn't have much room on the form to display more prose for an error.

A friend was recently testing a new feature that I'd built for our sales team and she had a hard time seeing the little red asterisks that were showing up next to required fields. It felt to her as though she was pushing the submit button on the form but nothing was happening. It was clear that a ValidationSummary control would be helpful, especially if placed close to the submit button for the form.

I've been a bit lazy in the past about using ValidationSummary controls, partially because most of my forms are simple enough that they feel a bit redundant. But on a more complicated form, they can be very helpful to guide users back to the places on the form where there's problems.

So I threw one of those puppies on the form and immediately saw that there was a problem - my error message was set to "*", which meant that my validation summary was pretty useless - it just displayed a bunch of red asterisks! And in places where I'd used the prose, "This field is required", well that was pretty useless as an error message in the summary.

After a bit of research and experimentation, I discovered that the ValidationSummary control looks at the ErrorMessage property on each validation control in order to figure out what to display in the summary. So it's important to use ErrorMessage with a summary in mind! Don't use text like "*" or "This field is required". Be more specific so the user can find her way up to the problem field, as in, "PostalCode is required".

But if you make ErrorMessage verbose so that it's helpful in a summary, it may make your form really ugly when displayed inline next to the control being validated. The trick is to use the body of the validation control element to specify the inline error message. Then you end up with two messages: a verbose one that's used in your summary, and a more localized, brief message that shows up right next to the control being validated. Note the asterisk that's in the body of the RequiredFieldValidator below:

<asp:RequiredFieldValidator
      ErrorMessage="Zip/postal code is required"
      ControlToValidate='txtPostalCode'
      ValidationGroup='BasicInfo'
      Display="Dynamic"
      runat='server'>*</asp:RequiredFieldValidator>

I've learned a lesson from all of this. In the future when I use validation controls I'll always provide a summary-friendly message in the ErrorMessage field, and if I need something different (typically shorter) to display inline, I'll put it in the body of the validation control element.

Hope this helps!

adaptive process management (n.) an element of resource and business process management, adaptive search and event processing. Sometimes referred to as “Level 4” event processing or process refinement.

application concept (n.) a definition of a set of properties that represent the data fields of an application entity. An application concept can describe relationships among themselves. For example, an order concept might have a parent/child relationship with an item concept. A department concept might be related to a purchase requisition concept based on the shared property, department_id. Application concepts can include an application state model.

application state modeler (n.) a UML-compliant application that allows you to model the life cycle of a concept instance — that is, for each instance of a given concept, you can define which states it will pass through and how it will transition from state to state. States have entry actions, exit actions, and conditions, providing precision control over the behavior of an event processing agent. Transitions between states also may have rules. Multiple types of states and transitions maximize the versatility and power of the application state modeler.

derived event (n.) an event that is created as a result of processing one or more other events.

complex event (n.) an event that is a situation-entity abstraction of two or more simple, derived or other complex events.

complex event processing (n.) CEP is a technology for extracting information from message-based systems. CEP is primarily an event processing concept that deals with the task of processing multiple events from an event cloud with the goal of identifying the meaningful events within the event cloud. CEP employs techniques such as detection of complex patterns of many events, event correlation and abstraction, event hierarchies, and relationships between events such as causality, membership, and timing, and event-driven processes.

event (n.) a instance of an event definition. It is an immutable object that represents a business activity that happened at a single point in time. Just as one cannot change the fact that a given activity occurred, one cannot change an event — events are immutable.

event aggregation (n.) the aggregation of simple, derived or complex events into higher levels of event abstractions.

event definition (n.) a set of properties related to a given activity that represents an important or interesting change of state in a human, system or computational activity. An event definition includes event properties such as event priority, event time to live (TTL), and a description of the payload, which is comprehensive information related to the activity that occurred. Events expire when the TTL has elapsed, unless the event processing agent has instructions to consume them prior to that time.

event channel (n.) a communications channel in which events are transmitted from event source to event receivers, typically received as electronic messages. Each channel can have multiple destination and. events can be configured to transmit to a default destination. JMS is an example of an event channel.

event cloud (n.) a partially ordered set of events (poset), either bounded or unbounded, where the partial orderings are imposed by the causal, timing and other relationships between the events. Typically an event cloud is created by the events produced by one or more distributed systems. An event cloud may contain many event types, event streams and event channels. The difference between a cloud and a stream is that there is no event relationship that totally orders the events in a cloud.

event-driven (n.) the behavior of a human, system or computational entity whose execution or actuation is in response to events, typically received as electronic messages.

event-driven architecture (n.) an architectural style for distributed computing applications in which some of the components are event-driven and communicate by means of events.

event processing (n.) computing that performs operations on events, including modifying, creating and destroying events.

event-object (n.) an software object that represents an event, generally for the purpose of computer processing, that exhibits both encapsulation, inheritance and polymorphism.

event prediction (n.) computational activity where the impact of events, complex events, and situations caused by events identified, including both opportunity or threat. Sometimes referred to as “Level 2” event processing, impact assessment or predictive analytics.

event pre-processing (n.) computational activity where events are cleansed or normalized to produce semantically understandable data. Sometimes referred to as “Level 0” event processing.

event processing (n.) computational activities on events dealing with the association, correlation, and combination of event data and information from single and multiple event sources to achieve refined identity and situation estimates for observed event objects, and to achieve complete and timely assessments of opportunities, threats, and their significance. Event processing is characterized by continuous refinements of event estimates and assessments and by evaluation of the need for additional sources, or modification of the process itself, to achieve improved results.

event processing agent (n.) an EPA is a computational entity that performs event processing.

event processing network (n.) a set of event processing agents and a set of event channels connecting them.

event properties (n.) data representation of an event, typically by name-value pairs of type string, integer, real, boolean or a complex data type.

event refinement (n.) filter, identify and track events & make initial processing decisions based on association, correlation and state estimation. Sometimes referred to as “Level 1” event, or event-object, track and trace.

event stream (n.) a time-ordered sequence of events. An event stream may be bounded by a certain time interval or other contextual dimension (content, space, source, certainty), or be open ended and unbounded.

event stream processing (n.) a time-ordered sequence of events. An event stream may be bounded by a certain time interval or other contextual dimension (content, space, source, certainty), or be open ended and unbounded.

rule (n.) defines what triggers unusual, suspicious, problematic, or advantageous activity within an event processing agent and what the EPA does when it discovers these types of activities. Rules execute actions based on certain conditions on events, instances, or a combination of both. A rule includes a group of condition-rule statements and action-rule statements. The condition statements instruct the EPA what to look for in events, and action statements instruct the EPA how to respond when conditions are met. If all the conditions in a rule are satisfied by events or instances or both, the EPA fires the actions. The action might be to execute tasks, create an event instance, modify property values in an event instance, create and send an event, or something else.

rules engine (n.) a type of event processing agent that uses a declarative programming model to process events. Formally described as “an abstract structure that describes a formal language precisely, i.e., a set of rules that mathematically delineates a (usually infinite) set of finite-length strings over a (usually finite) alphabet“. Informally, it can be any system that uses rules, in any form, that can be applied to data to produce outcomes.

rule language (n.) is an artificial language that is used to control the behavior of an event processing agent. Rules languages, like human languages, have syntactic and semantic rules to define meaning.

situation refinement (n.) identify situations, or complex events, based on event clustering, event-event relationships and relationship analysis and context. Sometimes referred to as “Level 2” event processing.

simple event (n.) an event that is not an abstraction or composition of other events.

virtual event (n.) an event that is imagined, modeled or simulated.

We write to express our firm belief that research on security vulnerabilities, and the sensible publication of the results of the research, are critical for scientific advancement, public safety and a robust market for secure technologies. Generally speaking, the norm in our field is that researchers take reasonable steps to protect the individuals using the systems studied. We understand that the student researchers took such steps with regard to their research, notably by planning not to present a critical element of a flaw they found. They did this so that their audience would be unable to exploit the security flaws they uncovered. . . .

The restraining order at issue in this case also fosters a dangerous information imbalance. In this case, for example, it allows the vendors of the technology and the MBTA to claim greater efficacy and security than their products warrant, then use the law to silence those who would reveal the technologies’ flaws. In this case, the law gives the public a false sense of security, achieved through law, not technical effectiveness. Preventing researchers from discussing a technology’s vulnerabilities does not make them go away - in fact, it may exacerbate them as more people and institutions use and come to rely upon the illusory protection. Yet the commercial purveyors of such technologies often do not want truthful discussions of their products’ flaws, and will likely withhold the prior approval or deny researchers access for testing if the law supports that effort. . . .

Yet at the same time that researchers need to act responsibly, vendors should not be granted complete control of the publication of such information, as it appears MBTA sought here. As noted above, vendors and users of such technologies often have an incentive to hide the flaws in the system rather than come clean with the public and take the steps necessary to remedy them. Thus, while researchers often refrain from publishing the technical details necessary to exploit the flaw, a legal ban on discussion of security flaws, such as that contained in the temporary restraining order, is especially troubling.

It will be interesting to see what arguments the MBTA uses to keep the students from speaking on a topic where all the important vulnerability information seems to have already disclosed. Sure the students haven’t presented a cookbook exploit tool but they have also stated they have no intention of doing so.

Perhaps the court will investigate what the MBTA’s and their technology vendors response has been to the MiFare card vulnerabilities that were disclosed responsibly. If there has been no vigorous response to responsibly disclosed vulnerabilities of many months ago how can they say with a straight face that are truly responding to new security information and just need more time.

We write to express our firm belief that research on security vulnerabilities, and the sensible publication of the results of the research, are critical for scientific advancement, public safety and a robust market for secure technologies. Generally speaking, the norm in our field is that researchers take reasonable steps to protect the individuals using the systems studied. We understand that the student researchers took such steps with regard to their research, notably by planning not to present a critical element of a flaw they found. They did this so that their audience would be unable to exploit the security flaws they uncovered. . . .

The restraining order at issue in this case also fosters a dangerous information imbalance. In this case, for example, it allows the vendors of the technology and the MBTA to claim greater efficacy and security than their products warrant, then use the law to silence those who would reveal the technologies’ flaws. In this case, the law gives the public a false sense of security, achieved through law, not technical effectiveness. Preventing researchers from discussing a technology’s vulnerabilities does not make them go away - in fact, it may exacerbate them as more people and institutions use and come to rely upon the illusory protection. Yet the commercial purveyors of such technologies often do not want truthful discussions of their products’ flaws, and will likely withhold the prior approval or deny researchers access for testing if the law supports that effort. . . .

Yet at the same time that researchers need to act responsibly, vendors should not be granted complete control of the publication of such information, as it appears MBTA sought here. As noted above, vendors and users of such technologies often have an incentive to hide the flaws in the system rather than come clean with the public and take the steps necessary to remedy them. Thus, while researchers often refrain from publishing the technical details necessary to exploit the flaw, a legal ban on discussion of security flaws, such as that contained in the temporary restraining order, is especially troubling.

It will be interesting to see what arguments the MBTA uses to keep the students from speaking on a topic where all the important vulnerability information seems to have already disclosed. Sure the students haven’t presented a cookbook exploit tool but they have also stated they have no intention of doing so.

Perhaps the court will investigate what the MBTA’s and their technology vendors response has been to the MiFare card vulnerabilities that were disclosed responsibly. If there has been no vigorous response to responsibly disclosed vulnerabilities of many months ago how can they say with a straight face that are truly responding to new security information and just need more time.

The SQL [structured query language] inserts that came earlier were just pablum intended to lull the Army cadets into a false sense of security. But then the bad guys unleashed a stealthy kernel-level rootkit that burrowed into one workstation, started scraping data and "calling home."

It was a highly sophisticated attack, but this time the bad guys were really good guys in wolves' clothing.

For four days in late April, the National Security Agency -- the nation's most secretive repository of spooks, snoops and electronic eavesdroppers -- directed coordinated assaults on custom-built networks at seven of the nation's military academies, including West Point, the Army university 50 miles north of New York City.

It was all part of the seventh annual Cyber Defense Exercise, a training event for future military IT specialists. The exercise offered a rare window into the NSA's toolkit for infiltrating, corrupting or destroying computer networks.

The 34 Army cadets comprising the West Point IT team operated in a different kind of battlefield, but their combat skills and instincts need to be every bit as sharp. Like George Washington said: "There is nothing so likely to produce peace as to be well prepared to meet the enemy."

The SQL injections, targeting their Fedora Core 8 Web server, were a piece of cake for these IT combatants. Each injection tried to smuggle malicious code inside the seemingly harmless language used by the network’s MySQL software. The cadets handily defended with open source Apache web server modules, plus some manual tweaking of the SQL database to "avoid any surprises," in the words of Lt Col. Joe Adams, a West Point instructor who helped coach the team.

But the kernel-level rootkit was much more dangerous. This stealthy operating-system hijacker can open unseen "back doors" into even highly protected networks. When they detected the rootkit's "calls home" the cadets launched Sysinternal's security software to find the hijacker, then they manually scoured the workstation to find the unwelcome executable file.

Then they terminated it. With extreme prejudice.

"This was probably the most challenging part of the exercise, since it required them to use some advanced techniques to find the rootkit," Adams says. And rooting it out helped boost the West Point team to the top of the pile when, in the aftermath of the exercise, the referees rated all the universities' network defenses.

For the second year in a row, the Army placed first over the Navy, Air Force, Coast Guard and others, winning geek bragging rights and the privilege of holding onto a gaudy, 60-pound brass trophy festooned with bald eagles and American flags. Adams credits the team’s thorough preparation and their excellent teamwork despite the round-the-clock schedule.

At the network control room on the second floor of West Point’s 200-year-old engineering building (which once was an indoor horse corral and still smells like it in some remote corners, according to one instructor), the IT team set up cots and, just for the hell of it, camouflaged netting. They worked in shifts, with one team member always monitoring incoming and outgoing traffic. He or she would alert other cadets -- "router guys" -- to block any suspicious addresses. Meanwhile, off-shift cadets would make food and coffee runs to keep everyone fueled up and alert. Together, the team was "faster than anyone else," Adams says.

But the way the cadets designed their network was a big factor in their victory, too. The NSA dictated some terms: All networks had to be capable of e-mail, chat and other services and had to be up and running at all times despite any attacks or defensive measures. Beyond that, the teams were free to come up with their own designs.

West Point's took three weeks to build. The cadets settled on a fairly standard Linux and FreeBSD-based network with advanced routing techniques for steering incoming traffic in directions of the IT team's choosing.

The choices in software tools for responding to any attack really boiled down to "automatic" versus "custom," says Eric Dean, a civilian programmer and instructor. He adds that while automatic tools that do most of their own work are certainly easier, custom tools that allow more manual tweaking are more effective. "I expect one of the 'lessons learned' will be the use of custom tools instead of automatics."

Even with a solid network design and passable software choices, there was an element of intuitiveness required to defend against the NSA, especially once it became clear the agency was using minor, and perhaps somewhat obvious, attacks to screen for sneakier, more serious ones.

"One of the challenges was when they see a scan, deciding if this is it, or if it’s a cover," says Dean. Spotting "cover" attacks meant thinking like the NSA -- something Dean says the cadets did quite well. "I was surprised at their creativity."

Legal limitations were a surprising obstacle to a realistic exercise. Ideally, the teams would be allowed to attack other schools' networks while also defending their own. But only the NSA, with its arsenal of waivers, loopholes, special authorizations (and heaven knows what else) is allowed to take down a U.S. network.

And despite the relative sophistication of the NSA's assaults, the agency told Wired.com that it had tailored its attacks to be just "a little too hard for the strongest undergraduate team to deal with, so that we could distinguish the strongest teams from the weaker ones."

In other words, grasshopper, nice work -- but the NSA is capable of much craftier network take-downs.

As I touched on in my December posting on Common Criteria, and as Michael Howard discussed in his post on security metrics, trying to objectively quantify and measure “How secure is secure” is far more difficult than one might think. I’d like to share my perspective that there are two “dimensions” useful to consider when characterizing software security metrics: security functional requirements and security engineering quality requirements. While the SDL is focused primarily (but not exclusively) on the latter, both are ultimately important when assessing the security of a given bit of software. However, for reasons I’ll elaborate on below, the SDL does focus on trying to prevent the most common causes of vulnerabilities today and hence looking at the ways in which Microsoft tracks and measures individual products teams’ compliance with SDL requirements offers some interesting fodder for the security metrics debate. I’m not offering a complete solution, but am sharing our experience at Microsoft with measuring how development teams actually follow the SDL. It’s helped us deliver more secure software, and sharing this will hopefully help others as well as putting more data on the table for consideration when discussing security metrics.

Putting aside computer security for just a moment, it’s interesting to look at other ways in which we attempt to measure security in our society. Padlocks offer security protections, and organizations such as the American Standard for Testing and Materials (ASTM) provide standards like F883-04 Standard Performance Specification for Padlocks that characterize padlock security ratings. Prisons provide security protections as well. Prisoners reside in different facilities that vary by security level. The US Bureau of Prisons uses a numbered scale from one to six to represent the security level. Both of these examples are similar in that the threats and risks each of them must protect against are reasonably well understood and relatively static (meaning the threats don’t change much over time). Computer security is still evolving with new classes of attacks still being discovered, and while hackers understand how to exploit known types of vulnerabilities – software developers are still catching up in learning how to modify engineering practices to be resilient against both new and old types of attacks. Hence, metrics are more challenging for computer security.

Several attempts have been made by governments to come up with a security rating system similar to the examples listed above. In the 1980’s, the US Department of Defense created the “Trusted Computer System Evaluation Criteria (TCSEC)” that tried to establish a standard for measure operating system security. The “Orange Book” offered a relatively simple system for assigning “score” summarized below:

D (Minimal Protection)

C (Discretionary Protection)

C1: Discretionary Security Protection

C2: Controlled Access Protection

B (Mandatory Protection)

B1: Labeled Security Protection

B2: Structured Protection

B3: Security Domains

A (Verified Protection)

A1: Verified Design

In the 1990’s, the US and other nations combined their efforts to create an international security standard for software known as the Common Criteria (ISO 15408). Common Criteria also has a rating system that scores products with “evaluation assurance levels” (EALs):

EAL 1: Functionally Tested

EAL 2: Structurally Tested 

EAL 3: Methodically Tested and Checked 

EAL 4: Methodically Designed, Tested, and Reviewed 

EAL 5: Semi-formally Designed and Tested 

EAL 6: Semi-formally Verified Design and Tested 

EAL 7: Formally Verified Design and Tested 

Both TCSEC and Common Criteria (CC) are primarily focused on “security functional requirements” (as called out earlier, distinct from “security engineering quality requirements”). The EALs reflect the amount of rigor and attention to claimed security functional requirements a developer applied while creating a product. Furthermore, the EALs also reflect increasing levels of effort and resources necessary by anyone reviewing a product in order to evaluate the product’s claimed security functional requirements. However, EAL ratings for commercial products have historically not correlated with the number of vulnerabilities found in commercial products after release. As I discussed in my December posting on Common Criteria, this is because CC is primarily focused on “security functional requirements” and fails to adequately address “security engineering quality requirements”.  This leads a question on how to measure those aspects of software security that earlier efforts have been unable to successfully address. 

Microsoft has been releasing security bulletins since 1999. Based on some informal analysis that members of our organization have done, we believe well over 50% of *all* security bulletins have resulted from implementation vulnerabilities and by some estimates as high as 70-80%. (Some cases are questionable and we debate if they are truly “implementation issues” vs. “design issues” – hence this metric isn’t precise, but still useful). I have also heard similar ratios described in casual discussions with other software developers. In other words, most vulnerabilities can be addressed by the “security engineering quality requirements” described via SDL. This is not to say that “security functional requirements” are unimportant or that SDL ignores secure design (as Adam has described in his threat modeling series), but rather that it is not where vulnerabilities are being most frequently encountered. With SDL, we adopt a pragmatic approach in looking at identifying the root causes of security vulnerabilities, and trying to prevent those root causes from reoccurring. The challenge lies in how we actually validate that development teams are indeed adopting and executing whatever changes SDL requires in engineering (either in terms of process or tools). Process changes are often difficult to quantify, as we must rely upon development teams truthfully attesting they have followed the process. As long as development teams believe the process results in better code, they generally will adopt and follow such practices. Tool usage becomes more interesting and valuable in that using tools becomes a vehicle for objectively and independently verifying if code satisfies requirements or not. But that is just the tip of the iceberg…

As I said above in my comments on EALs, the amount of time required by anyone reviewing a product to assess “security” is relevant since security review can be a very time and resource intensive activity. However, running static code analysis tools, verifying build tools and switches, searching for banned APIs, and recording the output of other tools that inspect code and/or binaries for potential implementation vulnerabilities is a key element in how we approach the challenge of trying to measure compliance with SDL requirements from product groups at Microsoft today. While not every technique required by SDL has a corresponding tool, we try to provide both tools and automation if and wherever possible. There is still much work to be done in terms of standardizing tool output formats and creating automation to assess tool output. However, these “grass roots” metrics derive from practical experience of changing engineering requirements based on actual vulnerabilities. We look objectively at what is causing vulnerabilities, and target solutions to address the root causes of those issues. As the saying goes, “If it hurts when you do that, stop doing that”. If what we have done in the past has hurt our customers by creating vulnerabilities requiring security bulletins, we want to stop doing that. J

The challenge in using a plethora of individual detailed metrics such as I describe above (that we do internally at Microsoft for measuring SDL compliance), is that they don’t roll up into a nice aggregate score that customers can easily understand.  However, they have translated into reduced numbers of vulnerabilities as Michael Howard wrote a few weeks ago. Coupling these types of scores with assessment of compliance with “security functional requirements” might be the basis for coming up with a metric that is useful to customers, both in the government and private sector.

What do you think?

1. Microsoft's customers, competitors, and the public at large.
2. Security folks, especially software security folks that want to improve the quality of their software.
3. People who want more metrics about all things generally, the costs of security, etc.

The court heard the bombers intended to use hydrogen peroxide and mix it with a product called Tang, used in soft drinks, to turn it into an explosive.

They intended to carry it on board disguised as 500ml bottles of Oasis or Lucozade by using food dye to recreate the drinks' distinctive colour.

The detonator would have been disguised as AA 1.5 batteries. The contents of the batteries would have been removed and an electric element such as a lightbulb or wiring would have been inserted.

A disposable camera would have provided a power source.

Any chemists want to take a crack at this one?