During the presentation I mentioned in some depth how to perform fuzz testing, and what parts of an application should be fuzz testing targets. I also introduced an idea (that's not new) to help people who have never performed fuzz testing begin fuzz testing with very little cost and friction. The idea is to add a small layer of code to an application to automatically mutate untrusted data as it comes into an application; I called that code layer "a layer of hurt."

Before I continue, I want to point out that fuzzing is an SDL requirement, but the idea in this blog post is not an SDL requirement, it's just another way to help meet SDL fuzzing requirements.

Adding a layer of hurt, as shown in the picture below, is pretty simple as it involves adding code to an application to tweak data as it comes into an application. You can work out where to place the fuzzing code by looking at your threat models to see where data crosses trust boundaries. You could also simply grep the code looking for APIs that read data, for example:

• Read from files: fread, ReadFile
• Reading from sockets: recv, recvfrom
• For .NET code, any stream.Read

You get the picture.

The fuzzing code should appear right after the API that reads that data.

For example, C or C++ code that reads from a UDP socket and then fuzzes the data before it's consumed by the rest of the application might look like this:

Or, in C#, code that reads from an untrusted file:

In both code examples, Fuzz() mutates the incoming data. In the C++ case, the fuzzing code looks like this:

The sample C# and C++ fuzzing code is available as a ZIP file at the end of this post.

This code is an example of dumb-fuzzing, which is fuzzing with little or no regard for the data structure being manipulated. If you've never performed any kind of fuzz testing in the past, then you will probably find bugs with this simple fuzzing technique. Once you have weeded out the low-hanging bugs, you may need to turn your attention to smarter fuzzers. For example, in theory, this code would find few if any bugs in a PNG parser, because PNG files have a built in check-sum, so if you fuzz a PNG file, you'd have to recalculate the checksum to get decent code coverage.

When I showed this code during my presentation, I urged people to add it to their applications today if they currently don't do fuzz testing, and simply run their applications through their normal testing processes. Within three days of my presentation I received emails from people saying they had found bugs. I have no doubt others did too.

One of the comments I made during the session was,"If you can't spend the time on great fuzzing, fuzz anyway" and adding a "layer of hurt" is a reasonable start.

Please feel free to sound off if you have ideas to help improve the code and let us know what you think, either through email or comments to this post.

Really, when I've heard about it first, I was like "ah, come on, I am sure the journalists are just mis-reporting it; nobody is that dumb in their approach to system security."

Well, they really were that dumb.

Honestly, from the "blatant disregard of common sense", this is very, very high on the list (many in security agree, some in IT disagree). This is where the words "a huge data security risk" really sound like a mild understatement. 

But you know what is the most scary about this case? The fact that there are MANY organizations who manage their networks the same way: one admin with ALL the access and NONE of the monitoring.

One person + ALL access + NO accountability = you are screwed!

Also, in light of this, do you still think that "insider attacks" is some kinda security vendor propaganda? Well, go tell Terry Childs that :-) Even though some people still think that he is a good guy (more on that on Slashdot)

What also caught my attention is that some retard called his bail ($5m) "ridiculously high." Well, if he was an outside hacker, say a Romanian script kiddie, jailed for hacking SF network, would they release him for $5m? Maybe not!  Now, do you get that this case is actually MUCH WORSE! Hacker might have gained access to many assets; this guy did have access.

So, think, think, think: CAN YOUR SYSADMINS "0WN" YOUR BUSINESS? (BTW, some people think that IT "owns" you already!)

Are you OK with it?

If not, do something - start logging and monitoring (and then controlling) their actions! If you think you cannot control them, then just monitor; if you think you can neither control nor monitor, then at least log them so they will know that there will be enough good evidence to let them rot in jail for many years ... Or, if you prefer an easier alternative, stop calling your business YOUR business.

Possibly related posts:

• "Protecting logs from admins"

So my next iteration of fun reading on security, logging and other topics.

1. "Security-as-control" vs "security-as-assurance" - a very useful idea (more here), which is often confused with bad results (e.g. "secure" software = has password authentication OR has has no overflow bugs)
2. Rich Mogul grabs GRC by the balls and kicks it, hard, again. A Burton Group guy comes and helps him by doing a nice roundhouse kick in its butt. Still, it doesn't die, as more people kick it ... Maybe 'cause Andy "loves or hates it?"
3. Good advice from Andy IT Guy: "We need to step back from time to time and evaluate what we are doing to determine if it still makes sense." (more)
4. BBC on cloud security, actually interesting. More on the same subject, albeit with a dumb name
5. Breach disclosure laws and security study by CMU, that SANS called idiotic ("What a silly study. It measures the wrong outcome. What matters about data breach notification is what it does to the quality of defenses.") AND "badly flawed" as well. More fun comments on it are here.  More discussion of this complicated subject. Rick kicks it too here.
6. Along the same line, "Data breaches at retailers are the top cause of credit and debit card theft, accounting for about 20% of all incidents." Wow!
7. "The biggest issue in both Audit and IT is a lack of strategic thought." (maybe) When I read it, it reminded me of the old wisdom from Ms Trunk: "if you think you are a 'strategist' - check maybe you think that 'cause your execution sux"
8. A very fun read: "Facing The Monster: The Labors Of Log Management." I am happy that log management has been granted a monster status :-)
9. Role of compliance for SCADA security puzzles me: think about it - you need a law to make people protect systems that control utilities EVEN THOUGH you already demonstrated (kind of) that hackers can explode generators remotely. So, people fear fines from regulators more than exploded power generators? Yep.
10. Is it time to regulate the security of cloud computing?
11. "How to Sell Security" by Bruce Schneier - a MUST read. BTW, FUD is NOT dead, and won't be dead. Ever!
12. OMG, this is huge and will grow: PCI Compliance and Virtualization (think "only one primary function per server" mandated in PCI). Same source on costs of PCI (also fun!) - still, IMHO, PCI is cheaper than properly securing your environment ... And while we are on the subject of PCI, check out Rich's "The Good (Yes, Good) And Bad Of PCI" and the discussion that followed.
13. New wave of compliance is incoooooooooooooming. Take cover!!!
14. Please shut up about ALL security being rolled into the network. Hoff says it best here.  If you want to join this bandwagon, say "all NETWORK security will be in the network."  (you'd probably still be wrong, but less embarassed :-))
15. Finally, some "Unintentional hilarity" from David: this is sooooo the world we live in :-)
    

• State of Nature. State of Nature just means what the current state is.  There are two ISSA Journals on my desk right now - State of Nature statement.

• State of Knowledge:  Analysis derived from examination of State of Nature.  “One of these ISSA Journals has an article co-authored Donn Parker on ROI.  I’ve read it, and it makes some statements he regards as truth.  Looking at those, well, I know that risk is quantifiable, best practices have significant issues, and there are many, many other statements of authority in the article that I can refute on evidence.” - Analysis or State of Knowledge.

• State of Wisdom:  Synthesis from the analysis.  The “So” moment.  “So since there are many statements of authority made in the article that I can refute on evidence, I should be open but skeptical about whether the conclusions of this article are likely to have much value to me in my quest to understand the value of risk reducing investments.”  What I’ve synthesized from the quality of the article - State of Wisdom.

(Just a clue for our readers, anytime you read someone talk about risk and mention the term “actuarial” - be skeptical about the conclusions they have you draw from the statement using that word. 9 times out of 10 what I’ve read after someone says actuarial is made as authoritative but shows a level of ignorance on the subject.  If you really want to mess with them - say “Really! Well, tell me how you feel about the use of non-parametric Bayesian Methods” and wait… )

MMMMM-MMMMMMM CHECKLISTS!

So what about Checklists?  They’re worth discussing because we’re swamped by them!  Heck, we’ve got people in love with the idea of checklists of checklists and claiming GRC nirvana is not in the checklist itself, but in the mapping of checklists.

Here ya go:  Checklists have one of two uses -

First they can give us a path to accomplish something.  I make a checklist every morning I call a “Todo List”.   Useful Checklists could be as Curphey mentions - steps for operating machinery or performing a certain task (heck, scientific method could be said to be a checklist of steps in analysis).  Checklists are useful in this way because, well, we’re fallible, absent minded, and novices.  They serve to reduce some level of variability in a process.

Second, they can help us develop a State of Nature.  PCI or the ISO are very nice checklists that, once you’re done, certifies that you have the existence of a certain amount of control.  Again, this serves to reduce some level of variability, comparing you to a “best practice”.

And so…..

They are both useful in each use - as long as the limitations therein are understood!   And that’s where we get into trouble.  Too many times we believe that checklists are a State of Knowledge.  Checklists allow for some limited analysis, just like the use of ordinal numbers to describe “risk” - they only serve to identify some level of variability, nothing more.

But outside of that they usually offer us no analytical function at all, they cannot provide a State of Knowledge and therefore, more succinctly, Checklists are dumb.

As slightly paranoid, skeptical and jaded risk management professionals, we know this to be true.  A PCI compliant company may or may not be at all “secure” or “risk-free” or even “risk-reduced”.  That’s an aspect of analysis that the checklist is some prior information for, but not nearly all the information we need for an analysis of risk or even a statement about the ability to control or resist.  We know an ISO certified organization did what they claim they do enough to at least fool an auditor once, but cannot arrive at any other State of Knowledge without more effort.

Make no mistake, the checklists we commonly deal with provide a very, very limited State of Knowledge.  Only analysis (with rigor and testing) will provide that.  And note that a State of Wisdom (what we’re really after, after all) is predicated on a strong State of Knowledge.

WHAT ARE YOU MANAGING TOWARDS, REDUX
So if checklists only provide a State of Nature, and are incapable of really giving us Knowledge or Wisdom - then let me encourage you to think about the amount of time you spend just getting a certain State of Nature and the relative return on that investment vs. the amount of time you spend in analysis and synthesis.  Is your time best spent mapping checklist to checklist - or is it better spent developing the analytics that allow us to synthesize wisdom?

AMAZE AND CONFUSE YOUR FRIENDS AUDITORS
Let me finish by encouraging you to have a frank discussion with those who perform your audit function.  You must really pin them down if they are out to give you any analysis at all - and when/if they do provide analysis - press them on what rigor they use to create a State of Nature, and then the means by which they create a State of Knowledge (that belief statement based on the State of Nature they see).

Deciphering the Myths Around Complex Event Processing  by Ivy Schmerken stimulated a recent flurry of blog posts about the maturity of CEP, including; Mark Palmer’s CEP Myths: Mature or Not? and Opher Etzion’s On Maturity.

I agree with Ivy.  CEP is not yet a mature technology by any stretch of the imagination.  In fact, I agree with all three of Ivy’s main points about CEP.

In 1998 David C. Luckham and Brian Frasca published a paper, Complex Event Processing in Distributed Systems on a new technology called complex event processing, or CEP (Postscript Version).  In that seminal paper on CEP, the authors said, precisely:

“Complex event processing is a new technology for extracting information from message-based systems.”

Ten years later there are niche players, mostly self-proclaimed CEP vendors,  whom do very little in the way of extracting critical, undiscovered, information from message-based, or event-based, systems.  

A handful of these niche players have informally redefined CEP as “performing low latency calculations across streaming market data.”  The calculations they perform are still relatively straight forward and they focus on how to promote white-box algo trading with commercial-off-the-shelf (COTS) software.  In this domain, we might be better off not using the term CEP at all, as this appears to be simply a type of new-fangled COTS algo trading engine.

The real domain of CEP, we thought, was in detecting complex events, sometime referred to as situations, from your digital event-driven infrastructure - the “event soup” for a lack of a better term.    In this domain, CEP, as COTS software, is still relatively immature and the current self-styled COTS CEP software on the market today is not yet tooled to perform complex situational analysis.

This perspective naturally leads to more energy flowing in-and-around the blogosphere, as folks “dumb down” CEP to be redefined as it benefits their marketing strategy, causing more confusion with customers who want CEP capabilties that have zero to do with low latency, high throughput algo trading, streaming market data processing, which maybe we should call “Capital Market Event Stream Processing” or CESP - but wait we don’t really need more acronyms!

Hold on just a minute!  Wasn’t it just a short couple of years ago that folks were arguing that, in capital markets, it was really ESP, not CEP, remember?  Now folks are saying that it is really CEP and that CEP is mature?   

CEP is mature?  CEP is really not ESP?  CEP is really event-driven SOA?  CEP is really real-time BI?  CEP is really low latency, high throughput, white-box COTs algo trading?  CEP is really not a type of BPM?  CEP is not really for detecting complex events?   Complex does not really  mean complex? 

Come on guys, give us a break! 

(Anyway, no one is going to give us a break….  so stay tuned!)

The answer to this is usually “yes and no”. I know, helpful, right?

The first thing we have to understand before answering is- is this a completely light AP solution, or is it ‘semi-light’. These are my terms and each manufacturer has their own verbiage they’ll use, but the concepts are the same.

In a completely light AP product, the controller has the brains, and the APs are dumb. For all practical purposes here, the APs are just radio antennas. They know nothing, and every packet is sent back through the controller for processing. Generally a fully light AP will not even have an IP address.

With a semi-light AP product, the controller does most of the work (usually anything routed or not local) and the APs have enough sense to process local traffic.

Scenario. Imagine a controller at a central office, connected to a light AP at another location (across the WAN). If it’s a completely light AP, it will send every bit of traffic over the WAN, to the controller. Not a great idea if you have medium-heavy wireless usage and a small WAN pipe. You’ll find you can quickly eat your bandwidth with your wireless traffic. If it’s a semi-light solution, the AP can process local traffic, for example a wireless user that wants to send a print job locally.

Processing local requests at the AP cuts down on the amount of traffic that has to traverse the WAN and is generally the way to go if you want a single central controller and remote APs.

If you decide you just have to run a completely light AP solution across the WAN, be sure your pipe is big enough and your usage low enough to support that configuration. Note that ‘big enough’ and ‘low enough’ are always relative and you’ll need to do a little experimenting to get the right threshold for your environment.

# # #

Today's entry is about throwing free money and free work [of somebody else, mind you] down the proverbial crapper.

So, the other day I was at one security conference which had a bit of a vendor expo. Since I work for a log management vendor, I am always on the lookout for new log-producing technologies. Typically, I just ask the vendor to send some log samples so that we can either create an official support package for this new log source or, at least, see how such logs will fare with our log indexer (that enables LogLogic index searches and  Index Reports).

Obviously, every vendor I ever approached loved it: after all, they might get something for nothing. If they are small, integrating with LogLogic might help their business. If they are big, they are typically happy that their "partner ecosystem" is growing. All it takes for them is sending a small sample of their logs - and we will do the rest.

While cruising that show I noticed a booth of a relatively well-known (but still pretty small) security appliance vendor. So I chatted with them a bit and in the end asked the engineer to connect  me with their core  folks so that we [LogLogic] can get a sample of logs and then develop support for it.  We don't really have to do it for them, but, then again, it might come handy, who knows.

Imagine my surprise (nah, shock!) when an email came that they "don't really want that."  I thought long and hard about the possible benefits of NOT having your logs in a log management system, but only one stood above the rest - and that is STUPIDITY! Thus, this entry :-)

Dmarti's blog over on LinuxWorld has an article up titled "Dumbest networking vendor idea since Network Access Control", which talks about what a dumb idea it is for Cisco to allow Linux apps to run on their ISR routers. Besides the fact that the title of the article alone is enough to make me want to tear this one apart, the underlying logic of the authors argument is just weak.

On one hand he talks about why would someone want to run Linux apps on a router, it is potentially bad design. On the other hand he says it is better to run them on a cheaper router alternative like Vyatta and than spouts some PR by Vyatta about their price/performance advantage over Cisco.  They back up this advantage with "3rd party testing".  Turns out the testing is by Tolly Group.  Oh, now that changes everything.  Have any of you ever had a Tolly evaluation done? Anytime you submit a form that contains what you would like to see the testing show in the final report and the final report shows it, well you know what I am saying. But seriously if it is good for Vyatta, why would it not be also good for Cisco?

Here is the real issue though that the author misses.  We live in an age of convergence!  The idea of having a stand alone box that only does routing is history and when Cisco themselves acknowledge it, you know it is fact.  People want more functionality out of their hardware.  Now that is not to say that your router should be your database server or mail server.  But there are certainly network functions that make sense to put on a router. Security is a no brainer to start. IPS, VPN, firewall, gateway AV- easy.  What about network functionality like DHCP, DNS, Radius, etc.  How about some next gen network stuff like WAP and VOIP?  That would make sense. By embracing Linux on the router all of these things and more are possible.  By the way you can do all of this now with our own Cobia platform.

That's right, we had this idea 2 years ago and have been working on it since.  With the convergence of networking, security, VOIP and wireless technologies, why wouldn't you want a multi-use box that can deliver all of this.

Dmarti's blog over on LinuxWorld has an article up titled "Dumbest networking vendor idea since Network Access Control", which talks about what a dumb idea it is for Cisco to allow Linux apps to run on their ISR routers. Besides the fact that the title of the article alone is enough to make me want to tear this one apart, the underlying logic of the authors argument is just weak.

On one hand he talks about why would someone want to run Linux apps on a router, it is potentially bad design. On the other hand he says it is better to run them on a cheaper router alternative like Vyatta and than spouts some PR by Vyatta about their price/performance advantage over Cisco.  They back up this advantage with "3rd party testing".  Turns out the testing is by Tolly Group.  Oh, now that changes everything.  Have any of you ever had a Tolly evaluation done? Anytime you submit a form that contains what you would like to see the testing show in the final report and the final report shows it, well you know what I am saying. But seriously if it is good for Vyatta, why would it not be also good for Cisco?

Here is the real issue though that the author misses.  We live in an age of convergence!  The idea of having a stand alone box that only does routing is history and when Cisco themselves acknowledge it, you know it is fact.  People want more functionality out of their hardware.  Now that is not to say that your router should be your database server or mail server.  But there are certainly network functions that make sense to put on a router. Security is a no brainer to start. IPS, VPN, firewall, gateway AV- easy.  What about network functionality like DHCP, DNS, Radius, etc.  How about some next gen network stuff like WAP and VOIP?  That would make sense. By embracing Linux on the router all of these things and more are possible.  By the way you can do all of this now with our own Cobia platform.

That's right, we had this idea 2 years ago and have been working on it since.  With the convergence of networking, security, VOIP and wireless technologies, why wouldn't you want a multi-use box that can deliver all of this.

• One person said that 'a common theme recently is that "those breached were compliant"' (meaning TJX and Hannaford). I question: is this really so? I think the truth is everybody, compliant or not, is 0wned, not that "those compliant are 0wned"
• All panelists predicted that governments (US and European) will be influencing security more in the near future: more laws, more regulation, more enforcement (and that governments will do more to secure their own systems)
• One person proclaimed that 'law enforcement model of security (detect->respond) doesn't work anymore', but said nothing about what comes next, instead, etc. I just hate empty posturing like that ... but wait! There is more from the posturing department: one more panel member said 'we need to not buy software products unless "absolutely secure".'  Hellooooo, is anybody home? :-)
• ISO27001 is hot. Really? A lot of people in the audience seemed to like ISO27001. So, is it enough to predict its takeoff in the US? Somehow I am still skeptical ...
• GRC was mentioned... in passing.  Everybody heard about it - and nobody cared. One person said "GRC... hmmm... so, how do you know you have it?'  :-)
• One more person said that "plausible deniability [about security] is dead" - companies cannot pretend that information security doesn't exist anymore ... Again, no matter how much we want this to be the case, is this really true? I think many smaller companies are kinda still in the same bin?
• A bizarro opinion on PCI DSS was voiced by one panel member: she said that she dislikes PCI since it is "too prescriptive" and it got turned into a mindless checklist (losing the original intent of improving security). She also disliked that PCI compliance evaluation is bad: based on a "dumb" control checklist, not on measuring effectiveness of "meaningful controls." I think this is true to some extent; but I'd hate to blame it on PCI DSS standard itself.
• Finally, panels' take on "What will happen in 5 years?" Their predictions: catastrophic events ("Estonia-like" - eeeeh, you mean somebody is fined $1642?), 'integrity of data' attacks which are "exceptionally scary" (data loss -> data change!), growth in data volume (huge!) with total lack  of how to control it, increased dependency on the Internet - without a corresponding increase in security, SaaS and Web 2.0 will change security and so will virtualization (now, that's original :-))

So, it was all good fun!