There is another reason why the article got my attention; my manager, Steve Lipner, is one of the few people to have designed and built a TCSEC A1 assured system and lived to tell the tale. None were sold, but they built one!

The NSA-directed project, the Tokeneer ID Station (TIS), involved building a low-defect system that conforms "to the Common Criteria requirements for Evaluation Assurance Level 5 (EAL5)" in a "cost effective manner." I'm all for this, because building high-assurance solutions is not cheap.

There's a paper with more technical detail about the project that is worth a read.

In my opinion, the project is only a science project, an experiment, for the following reasons:

• It's tiny. Weighing in at a little under 10 KLOC.
• It's only a very small portion of a much larger solution which has not been developed using the same rigor. This bit of context makes the solution as a whole moot. Call me cynical, but my question is "can the entire solution be built with same rigor in a ‘cost effective manner'?" Perhaps it can, but that is not what is presented.
• It sits on top of many operating systems (Windows, Mac OS X and Linux) that are not EAL5 certified. So it would be a little like having an EAL5 certified CharMap application running on EAL4 Windows Vista.
• It's written in a subset of Ada called SPARK, and SPARK skills are not common in the marketplace. Interestingly, SPARK makes use of annotations to help drive the static analysis process. While not a total analog, we also recommend Microsoft development teams use annotations (SAL) to help drive the required static analysis process.
• The application has a large number of dependencies that are not part of the project:

In the SDL we call these files ‘giblets' because they are components needed for your application to operate, but they do not belong to your team. Some of the files look old and highly vulnerable, such as libpng-3.dll from 2001! OSVDB lists 23 vulnerabilities since 2002 in libpng!

In summary, the TIS project is very interesting to a small number of important but specialized customers, such as the NSA, for whom this kind of research is critical. I too found it interesting, but the process is far from a set of "secrets to writing secure code" and the tools are certainly not within reach of day-to-day applications and not applicable to developing complete solutions.

As part of the United Arab Emirates, Dubai censors the Internet:

The government of the United Arab Emirates (UAE) pervasively filters Web sites that contain pornography or relate to alcohol and drug use, gay and lesbian issues, or online dating or gambling. Web-based applications and religious and political sites are also filtered, though less extensively. Additionally, legal controls limit free expression and behavior, restricting political discourse and dissent online.

More detail here.

What was interesting to me about how reasonable the execution of the policy was. Unlike some countries -- China for example -- that simply block objectionable content, the UAE displays a screen indicating that the URL has been blocked and offers information about its appeals process.

No doubt you are aware of the out-of-band security bulletin issued by the Microsoft Security Response Center today, and like all security vulnerabilities, this is a vulnerability we can learn from and, if necessary, can use to shape future versions of the Security Development Lifecycle (SDL).

Before I get into some of the details, it's important to understand that the SDL is designed as a multi-pronged security process to help systemically reduce security vulnerabilities. In theory, if one facet of the SDL process fails to prevent or catch a bug, then some other facet should prevent or catch the bug. The SDL also mandates the use of security defenses, because we know full well that the SDL process will never catch all security bugs. As we have said many times, the goal of the SDL is to "Reduce vulnerabilities, and reduce the severity of what's missed."

In this post, I want to focus on the SDL-required code analysis, code review, fuzzing and compiler and operating system defenses and how they fared.

Code Analysis and Review

I want to start by analyzing the code to understand why we did not find this bug through manual code review nor through the use of our static analysis tools. First, the code in question is reasonably complex code to canonicalize path names; for example, strip out ‘..' characters and such to arrive at the simplest possible directory name. The bug is a stack-based buffer overflow inside a loop; finding buffer overruns in loops, especially complex loops, is difficult to detect with a high degree of probability without producing many false positives. At a later date I will publish more of the source code for the function.

The loop inside the function walks along an incoming string to determine if a character in the path might be a dot, dot-dot, slash or backslash and if it is then applies canonicalization algorithms.

The irony of the bug is it occurs while calling a bounded function call:

_tcscpy_s(previousLastSlash, pBufferEnd - previousLastSlash, ptr + 2);

This function is a macro that expands to wcscpy_s(dest, len, source); technically, the bug is not in the call to wcscpy_s, but it's in the way the arguments are calculated. As I alluded to, all three arguments are highly dynamic and constantly updated within the while() loop. There is a great deal of pointer arithmetic in this loop. Without going into all the gory attack details, given a specific path, and after the while() loop has been passed through a few times, the pointer, previousLastSlash, gets clobbered.

In my opinion, hand reviewing this code and successfully finding this bug would require a great deal of skill and luck. So what about tools?  It's very difficult to design an algorithm which can analyze C or C++ code for these sorts of errors.  The possible variable states grows very, very quickly.  It's even more difficult to take such algorithms and scale them to non-trivial code bases. This is made more complex as the function accepts a highly variable argument, it's not like the argument is the value 1, 2 or 3! Our present toolset does not catch this bug.

Ok, now I'm really going out on a limb with this next section.

Over the last year or so I've noticed that the security vulnerabilities across Microsoft, but most noticeably in Windows have become bugs of a class I call "onesey - twosies" in other words, one-off bugs. There is a good side and a bad side to this. First the good news; I think perhaps we have removed a good number of the low-hanging security vulnerabilities from many of our products, especially the newer code. The bad news is, we'll continue to have vulnerabilities because you cannot train a developer to hunt for unique bugs, and creating tools to find such bugs is also hard to do without incurring an incredible volume of false positives. With all that said, I will add detail about one-off bugs to our internal education; I think it's important to make people aware that even with great tools and great security-savvy engineers, there are still bugs that are very hard to find.

Fuzz Testing

I'll be blunt; our fuzz tests did not catch this and they should have. So we are going back to our fuzzing algorithms and libraries to update them accordingly. For what it's worth, we constantly update our fuzz testing heuristics and rules, so this bug is not unique.

Defenses

If you want the full details of the defenses, and how they come into play on Windows Vista and Windows Server 2008, I urge you to read teh SVRD team's in-depth analysis once it is posted.

A big focus of the SDL is to define and require defenses because we have no allusions about finding or preventing all security vulnerabilities by attempting to get the code right all the time, because no-one can do that. No one.  See my comment above about one-off bugs!

Let's look at each SDL mandated requirement and how they fared in light of this vulnerability.

-GS

The -GS story is not so simple. A lot of code is executed before a cookie check is made and the attacker can control the overflow because the overflow starts at an offset before the stack buffer, rather than at the stack buffer itself. So the attacker can overwrite other frames on the call stack, corresponding to functions that return before a cookie check is made. That's a long way of saying that -GS was not meant to prevent this type of scenarios.

ASLR and NX

The code fully complies with the SDL, and is linked with /DYNAMICBASE and /NXCOMPAT on Windows Vista and Windows Server 2008. There are great defenses when used together, and reduce the chance of a successful attack substantially. Also, the stack offset is randomized too, making a deterministic attack even more unlikely.

Service Restart Policy

By default the affected service is marked to restart only twice after a crash on Windows Vista and Windows Server 2008, which means the attacker has only two attempts to get the attack right. Prior to Windows Vista, the attacker has unlimited attempts because the service restarts indefinitely.

Authentication

Thanks to mandatory integrity control (MIC) settings (which comes courtesy of UAC) the networking endpoint that leads to the vulnerable code requires authentication on Windows Vista and Windows Server 2008 by default. Prior to Windows Vista, the end point is always anonymous, so anyone can attack it, so long as the attacker can traverse the firewall. This is a great example of SDL's focus on attack surface reduction; requiring authentication means the number of attackers that can access the entry point is dramatically reduced.

Firewall

We enabled the firewall by default in Windows XP SP2 and later, this was a direct learning from the Blaster worm. By default, ports 139 and 445 are not opened to the Internet on Windows XP SP2, Windows Vista and Windows Server 2008.

Summary

The $64,000 question we ask ourselves when we issue any bulletin is "did SDL fail?" and the answer in this case is categorically "No!" No because as I said earlier the goal of the SDL is "Reduce vulnerabilities, and reduce the severity of what you miss." Windows Vista and Windows Server 2008 customers are protected by the defenses in the operating system that have been crafted in part by the SDL. The development team who built the affected component compiled and linked with the appropriate settings as described in "Windows Vista ISV Security" and Writing Secure Code for Windows Vista so that their service is protected by the operating system.

The team did not poke holes through the firewall unnecessarily, in accordance with the SDL.

The team reduced their attack surface, in accordance with the SDL, by requiring authenticated connections rather than anonymous connections by default.

We know that the SDL-mandated -GS has very strict heuristics so some functions are not protected by a stack cookie, but in this case, there is no buffer on the stack, so there will be no cookie. We know this. There are no plans to remedy this in the short term.

Fuzzing missed the bug, so we will update our fuzz testing heuristics, but we continually update our fuzzing heuristics anyway.

In short, based on what we know right now, Windows Vista and Windows Server 2008 customers are protected because of the SDL-mandated defenses in the operating system, and because the development team adhered to the letter of the SDL to take advantage of those defenses.

Chalk one up for Windows Vista and later and the SDL!

As usual, questions and comments are very welcome.

Blogger: Trent Henry

Burton Group’s Catalyst Europe conference is just around the corner. With financial services industry failures at the top of everyone’s mind, now’s a great time to revisit how risk management shortcomings have tremendous impact on organizations of every kind. In a reprise of his insightful Catalyst North America talk, Nick Leeson will once again detail how inadequate controls (and foolish actions on his part) brought about the fall of Barings Bank. In addition, security conversations at Catalyst will include:

- How large enterprises are grappling with governance, risk, and compliance (and why “GRC” is actually a four-letter word)
- What large, distributed organizations are doing to create effective “security embassies”
- The role of metrics in managing protection and communicating with Management
- How information-centric security will unfold over the next five years

I arrived in Austin Sunday night to get ready for a factory tour on Monday, a kickoff dinner and then two days of briefings from Dell executives, including Michael Dell himself! Dell’s ISG business is growing at a very fast pace and continues to build momentum and focus within the broader organization.

We had a nice overview of the product roadmap, including some of the exciting enhancements Dell is making to their storage products such as the MD3000 and the new EqualLogic PS5000 series iSCSI solutions.

I really enjoyed the Council meeting and it reminds me all over again; what I admire about Dell is the way they and Michael Dell himself stay close to the customer. The entire purpose of this event is to “get it right” and determine meaningful ways to embrace change (including change in the manufacturing process) in order to make their customers more successful. Ah shucks, you may say that all companies behave this way… well I must tell you that is not true and at times, I find it difficult as we continue to grow to stay as close as I would like to all of our customers varying needs and directions.

This concept of gathering, internalizing and embracing customer feedback is a simple principle of Business Success stories. Always trying to improve the pace of change and build meaningful sticky relationships with customers. Dell’s very successful Ideastorm site where customers post product feedback and are active participants in the Dell community is a great example of how to do this right. No other hardware vendor that we have worked with or attempted to work with has ever gone to the extent of embracing change that Dell has during our 5-year relationship.

From the custom factory integration services to the attention to detail in the order and manufacturing, and logistics processes, Dell helps us execute for our customers and I must admit that we could not have built the business as quickly or efficiently without Dell!

So thank you Michael Dell for building a business that embraces change and is focused on helping your ISG customers succeed.

First of all, I learned the Thai alphabet nearly 20 years ago, so I have have a pretty good foundation for the Thai language.   I can read (slowly) and speak better than 99.99+ percent of all foreigners in Thailand; so, I thought it was time to redirect my career to a “new challenge” in the business climate of Thailand.   

This was no small decision.  Your career changes dramatically when you give up a successful consulting practice in the US and dive into business in a foreign land for a new challenge.  I can frankly tell you that often the challenge is sometimes overwhelming.    It is quite difficult as a foreigner to do business in Thailand.

First of all, it is not legal for foreigners to own land in Thailand.  Foreigners can ”own” land using a variety of legal loopholes, proxy owners and shell companies; but all of this is risky and not advised.  Foreigners lose a lot of money coming to Thailand and attempting to buy land.  Some get lucky, but the entire process of foreigners buying and selling land is quite risky.

Foreigners can own condos, under certain conditions, but this results in  inflated prices for condos in Thailand that are traded in an artificial market place.   Condos that are up-to-par with condos in the US can easily cost more than condos in major cities in the US.  Hence, the cost of living is not as cheap as some might believe.

Business can best be described as “protectism” where the government has placed many barriers to entry to foreigners working in Thailand.     Every foreigner must have a work permit and these work permits are expensive and time consuming to maintain.   If you own a business you must pay high professional service fees for auditors to perform annual and semiannual audits even if your business has no income yet.   Firms in Thailand charge thousands of dollars for these ”audits”.      

In addition, if you operate a business, you must have a place of business, so you are forced to lease office space.   Foreigners from the US must be paid a minimum of 50,000 Thai Baht per month, so the government will take 10 percent of that each month as their share of tax withholdings.   Therefore, to start a company, you will pay a lot of money in startup fees, permits, tax, leases, visas, etc.  The entire system is designed to secure money from you, even if you do not have a penny of incoming revenue.

Of course, generating incoming revenue can be quite difficult in a climate of protectionism.   In Thailand, it is easy when you are spending money.  When you are trying to generate income from Thailand, as a foreigner the challenge can seem overwhelming at times.   Many foreigners here give up because the barriers to business here are very high.

On top of all these challenges, which I have not described in detail, is the overall global business slowdown combined with a climate of political instability, which I am sure you have seen in the news.  

Most people I know say it is better to be a tourist here.   Being a tourist is completely different.  Money flows from you, so life in Thailand is fun and friendly, complimentary to the “Land of Smiles” you have heard about.     However, when you are working to have money flow the other direction, flow to you versus away from you, you don’t see the “Land of Smiles” as tourists experience.

Without getting into too many details, I can simply say that a foreigner doing business in Thailand experiences protectionism and, to a certain degree, discrimination, and sometimes I wonder if coming here for a “business challenge” was a good idea.    I was seeking a “new challenge” and I got more than I bargained for!

In a future post on business in Thailand I will discuss issues regarding how little value is placed in intellectual property in Thailand and how this adversely impacts professional services.    I will also touch on how this lack of regard for intellectual property impacts a consulting practice.   Also, I will touch on some cultural differences in how Thais appear to view teamwork, which is very different than in the US.

We've probably all heard that you can't secure what you can't see and that statement is even more profound when it comes to virtual environments.  This is because it is extremely challenging to see what is going on at a micro vs. macro level within a virtual environments network.  The virtualization vendors such as VMWare and Citrix have provided embedded tools into their management consoles that show a macro level of visibility but its not enough to identify security events in the environment.  Take a look at the attached picture.  It simply shows VMWare's ability to monitor virtual network performance statistics from a bits per second perspective.

<-Click To Enlarge

With only this level of detail how can one determine which network applications are causing spikes.  Is it FTP traffic that is occuring at a high volume at an unuseal time of day?  If that were occuring, could that be indicative of either a breach or some sort of problem? What if FTP isn't even an authorized service in the virtual environment but there is a high volume of it?  Did someone install a rouge FTP service so they could steal information from the server at will?

These types of questions can't really be answered without a micro level of detail into the packets flowing in, out and within the virtual environment.  Now, what I am highlighting is not security in the traditional sense of prevention but using visibility as a means to first identify, then pin point the source of an issue so that it can properly be mitigated.  Having constant visibility can also ensure that other security products in the environment are performing as expected.  What if a Montego HyperSwitch with firewalling enabled is configured with many policies but someone forgot to create an FTP block policy.  One could think they are protected from rouge FTP services transmiting data out of the network, but without constant visibility monitoring, can you be certain?

Some vendors, namely Reflex Security will get you to believe that their IPS / IDS solution that is inline and running in the virtual environment is the right and only approach.  Or they will tell you to hang a virtual IDS off a span port in the virtual environment and you will at least have visibility into the attacks that are taking place.  Well, sure... You now have attack visibility but at the performance cost of your virtual environment.  Signature matching technologies are great, I'm a huge believer; however they don't scale very well in shared computing environments such as virtual ones.  IDS systems also don't typically track protocol and network service (FTP, HTTP, etc.) utilizations; which is another important part of visibility.

So, what do we do to gain visibility without the performance headache?  Well, for starters its probably best to put your IDS/IPS solutions in the physical environment where performance will be less of a concern.  In fact, you can span a virtual switch's traffic out to a physical NIC as easy as you can to a virtual one.  So why do it virtual and have to pay a 60% CPU utilization tax?  Another solution is to IDS inspect only the things you care about.  Why IDS inspect SSL traffic if you know your solution can't unencrypt SSL.  Its just a waste of compute cycles isnt it?  Policy based switching helps you with directing only the things you care about to an IDS (attack visualization product).  Montego's HyperSwitch also can help you with the traffic redirection of only the things you care about.

Another method of visibility which I tend to be a fan of is one of packet analysis (aka NetFlow).  NetFlow was invented by Cisco some time ago and has gained popularity in the physical world and definately has a use in the virtual world.  NetFlow is lightweight.  Let me say that again, its light weight!  It only sends a summation of packet detail to an analytical engine which can do some number crunching, packet comparison, etc. etc. to make some sense out of whats going on.  Lancope, an Atlanta based visibility company that provides Network Visibility, Security Visibility and User Visibility has this tool on their website that is a Netflow Bandwidth calculator.  You'll see from playing with this ( http://www.lancope.com/netflowcalculator.aspx ) calculator that it doesn't consume a lot of network bandwidth to transmit these network accounting records.  It also doesn't cause a lot of CPU overhead to send these records to an analytical engine sitting somewhere in the network.

Lancope's analytical engines have the ability to do the following for you within your virtual environment:

...and probably a slew of other things I'm not aware of.  A screen shot of their product is bellow:

<- Click to enlarge

You'll notice from the screenshot that you are able to visualize who is talking to who, how much traffic they have sent and received and something called a concern index (not seen on this screenshot).

Now, a concern index is a number that increases as Lancopes analytical engines monitor suspicious activity on a session.  A high counter can be indicative of a security problem.  Its another way of identifying (visualizing) compromised hosts (virtual machines) without having to do signature matching like a heavy weight IPS engine.  Example:  Lets say you have a VM that has a BOT on it and is "owned".  The Lancope product is monitoring this long life session.  Let's say that session is established for several hours or maybe even days or months.  Lets also say that the conversation appears to be mostly unidirectional from a public ip address not belonging to your enterprise.  Lancope would increase a the concern index on this since this server hasn't typically had this type of behavior.  Once the concern index reached a certain level it could then fire off an email, send you a text message or something saying:  Warning, Warning, Danger, Danger Will Robinson!!! You're virtual server may be infected with a BOT, please investigate immediately!!!

This example is VISIBILITY which helps you with SECURITY.  There are a number of other things you can do with NetFlow and Lancope products that have less to do with security and more to do with operational efficiencies.  Things like, helping you answer questions of:  How do I know what network applications are taking up the most bandwidth?  When should I move those applications over to a server with more horsepower?  When did these VM's vmotion over here and was there a traffic condition / CPU condition that caused that to occur?  I could go on and on but thats a topic for another blog entry.

So, my suggestion is to take a look at what NetFlow has to offer.  Montego Networks supports NetFlow transmission and Lancope supports NetFlow analytics and with both you can regain what was lost visibility.

I hope this was helpful to you all!

-John Peterson