Pouring over endless details of risks, regulations, taxonomies, and technologies can sometimes give us a narrow view of the world, so it seems worthwhile to take a minute to mark the 125th anniversary of the cataclysmic eruption of Krakatoa this week. For those of us that want to think big but can’t remember that far back, this week is also the 3rd anniversary of Hurricane Katrina’s devastating sweep across a wide stretch of the US Gulf Coast.

By now, I expect that most of you have read or are familiar with the 2007 book, The Black Swan, by Nassim Nicholas Taleb, which argues that these kinds of unpredictable, outlying occurrences are the ones that really shape businesses, countries, economies, and people. Taleb argues that although these “Black Swan” events are almost completely unforeseeable, we mistakenly try to explain the circumstances at the time and make predictions about similar events in the future.

In my ERM work with clients, and especially in the context of research I’ve been doing with my colleague Stephanie Balaouras on business continuity and resiliency, questions come up about how to plan for catastrophes... and they’re good questions. Were the CardSystems or TJX data breaches foreseeable? What about the Societe General debacle or the 2004 Indian Ocean tsunami? What’s next? Should these types of events be included in our risk assessments?

We’d like to get your opinion on these and other risks that may be on the very edge of the statistical tail. At what point do they belong in your risk register?

Of course, it’s possible to define mitigating controls for crises, disasters, or incidents without knowing for sure what they’re going to look like. That’s one of the hallmarks of a good crisis management plan. And that’s an important point, because trying to predict the next unforeseeable event can be a real challenge sometimes.

People ask why do you blog?  In the final analysis I blog because I like to. Every once in a while though you get a comment from a reader that reminds you why it is all worth while.  Here is one I received today from a person alleging to be a Julie Peterson:

Julie Peterson commented on Safe Access wins SC Magazine Award Reader Trust Award, again!:

Dressed in a tuxedo and chewing those rubber chicken breasts at the award ceremony is your idea of fun? Aren't you the same mentally retarded idiot who said in 2007 that you hated SC awards and that anyone can buy the SC awards with a sponsorship? Why do you think people give over $10k as sponsorship for the SC awards? Who is watching the awards except other vendors? By the way you suck big time with your rubbish blogs. Didn't networld magazine give you the boot within 3 months? Think before you write Mr. mental. Well done on winning, but please, dont give the impression that you cant buy an award from SC! And don't forget to eat your medication pills tonight, otherwise from your hair it is obvious you ran away from a mental hospital.

First of all Julie, let me thank you for your kind words! You made the statement and let me answer your questions for you.

1. Is dressing in a tuxedo and chewing rubber chicken breasts my idea of fun?  Actually, I do enjoy dressing up in a tuxedo once in a while.  The food at the awards ceremony was actually pretty good, if not diet friendly, as were the cocktails.  The entertainment at the awards show was pretty good as well. Catching up with friends you had not seen for a while and networking with industry peers was pretty worthwhile too.  Maybe your idea of a good time is putting on a bowling shirt and swilling a couple of beers and pretzels before going home and undressing into your dirty ripped underwear. Hey I say to each his own.

2. I am not the idiot who in 2007 said that I hated the SC awards and that anyone can buy the SC awards with a sponsorship.  I am the idiot who said that about the InfoSec Products Guide award by the folks at Silicon Valley Communications.  In contrast I have always said nice things about the SC awards. I actually have a lot of respect for them.  Also for the record, StillSecure has never been a sponsor of the SC Magazine awards. I have seen sponsors who did not win awards as well.  So looks like you got that one wrong Julie, but it happens.

3. ???Networld??? magazine didn???t give me the boot within 3 months.  They never had the chance, as I never wrote for ???networld, network world or any other magazine. Maybe you have me confused with Mike Rothman or Mitchell Ashley, who do and did write for Network World. But let me assure you that I do try and think before I write.

4. Regarding what medication pills I take and does my hair make it obvious I ran away from a mental hospital. I don???t take any medication, maybe I should.  Better living through chemistry you know ;-)  As to my hair, what can I say.  At this stage I am happy I have any hair at all.  My wife always says when I get my haircut it looks like a Buzz Lightyear style, but no one ever mentioned a mental hospital look to it.  In any event sorry it doesn???t appeal to you.

So who is this troll Julie Peterson?  Could it be Richard Stiennon in drag?  Maybe his wife striking out?  Maybe another one of my fans?  Who knows, but these sort of comments keep me juiced about blogging and remind me of how much fun I have doing it.  Thanks again Julie!

People ask why do you blog?  In the final analysis I blog because I like to. Every once in a while though you get a comment from a reader that reminds you why it is all worth while.  Here is one I received today from a person alleging to be a Julie Peterson:

Julie Peterson commented on Safe Access wins SC Magazine Award Reader Trust Award, again!:

Dressed in a tuxedo and chewing those rubber chicken breasts at the award ceremony is your idea of fun? Aren't you the same mentally retarded idiot who said in 2007 that you hated SC awards and that anyone can buy the SC awards with a sponsorship? Why do you think people give over $10k as sponsorship for the SC awards? Who is watching the awards except other vendors? By the way you suck big time with your rubbish blogs. Didn't networld magazine give you the boot within 3 months? Think before you write Mr. mental. Well done on winning, but please, dont give the impression that you cant buy an award from SC! And don't forget to eat your medication pills tonight, otherwise from your hair it is obvious you ran away from a mental hospital.

First of all Julie, let me thank you for your kind words! You made the statement and let me answer your questions for you.

1. Is dressing in a tuxedo and chewing rubber chicken breasts my idea of fun?  Actually, I do enjoy dressing up in a tuxedo once in a while.  The food at the awards ceremony was actually pretty good, if not diet friendly, as were the cocktails.  The entertainment at the awards show was pretty good as well. Catching up with friends you had not seen for a while and networking with industry peers was pretty worthwhile too.  Maybe your idea of a good time is putting on a bowling shirt and swilling a couple of beers and pretzels before going home and undressing into your dirty ripped underwear. Hey I say to each his own.

2. I am not the idiot who in 2007 said that I hated the SC awards and that anyone can buy the SC awards with a sponsorship.  I am the idiot who said that about the InfoSec Products Guide award by the folks at Silicon Valley Communications.  In contrast I have always said nice things about the SC awards. I actually have a lot of respect for them.  Also for the record, StillSecure has never been a sponsor of the SC Magazine awards. I have seen sponsors who did not win awards as well.  So looks like you got that one wrong Julie, but it happens.

3. “Networld” magazine didn’t give me the boot within 3 months.  They never had the chance, as I never wrote for “networld, network world or any other magazine. Maybe you have me confused with Mike Rothman or Mitchell Ashley, who do and did write for Network World. But let me assure you that I do try and think before I write.

4. Regarding what medication pills I take and does my hair make it obvious I ran away from a mental hospital. I don’t take any medication, maybe I should.  Better living through chemistry you know ;-)  As to my hair, what can I say.  At this stage I am happy I have any hair at all.  My wife always says when I get my haircut it looks like a Buzz Lightyear style, but no one ever mentioned a mental hospital look to it.  In any event sorry it doesn’t appeal to you.

So who is this troll Julie Peterson?  Could it be Richard Stiennon in drag?  Maybe his wife striking out?  Maybe another one of my fans?  Who knows, but these sort of comments keep me juiced about blogging and remind me of how much fun I have doing it.  Thanks again Julie!

Nearly 40% of U.S. information technology workers would accept a reduced salary to have the ability to telecommute, a Dice Holding survey revealed Tuesday.

In a poll of more than 1,500 IT workers, 37% of respondents said they would be willing to take “slightly less” pay to telecommute full time. The survey defined “slightly less” as up to a 10% reduction in salary.

The article does mention that workers can save some costs at the gas pump — but there are a lot more savings from telecommuting too. Workers who commute have to pay not just for gas or bus fees, but also for parking, the cost of lunch from eating out often, and the time they spend in the commute. In the end, the amount that workers might save by telecommuting might make up for the potential pay cut.

Of course, there are other costs to working remotely — getting a good business phone line and Internet connection, setting up a home office that you can stand to sit in all day and the costs of coffee for renting a table at your local coffee shop. But those are certainly worthwhile for the convenience and flexibility of working remotely.

Don’t get me wrong.  I greatly admire the work Mike does and wish he and his book had been around when I started out as a CISO.  Would have saved me significant pain and suffering.  On the other hand, if I’d had Mike’s P-CSO I might have become complacent and ended up believing that’s all there was to being a CISO.  Not that I think Mike is advocating complacency — he’s not.  I also don’t think he discounts risk analysis concepts.  He’s simply focused on helping that component of our profession who’s just getting started or who faces other practical constraints in dealing with our very complex problem space.  His is a necessary and highly valuable contribution, and he provides it in an entertaining way that’s too rare.

Let me set this discussion in a medical analogy context.  If I was in the middle of nowhere or didn’t have the resources for a physician, then a medic who’s skilled in lifesaving basics would do just fine.  However, if the situation called for a deeper understanding of the complex, sometime subtle health considerations, then I’d prefer a physician.  Someone who didn’t say;  “Boy, this anatomy and physiology stuff is complicated.  I’m just going to stick with ‘The hip bone is connected to the back bone…’”   My physician may, of course, choose to follow a pragmatic, commonly-used course of treatment, but they’d be able to do so with a deeper understanding of the problem space, greater (but not perfect) certainty that the course of treatment would work, and a better ability to explain to me, the patient, why I had to swallow this bitter pill, undergo the knife, or have this long tube snaked into one of my orifices.  

Yes, I realize that physicians sometimes get it wrong, sometimes get wrapped up in fancy and even unnecessary procedures, and can drive up costs.  That’s just as true as what can happen at the other end of the spectrum — the shaman who operates entirely by superstition, faith, FUD, and intuition.  The point is, there’s absolutely a need for both medics and physicians (and levels in between).  We, as professionals, can choose where we want to be within that continuum.  With this in mind, a few things to consider are:

• In the heat of battle, when resources are limited, or when it just makes sense, physicians always have the option of behaving as medics and sticking with the bare essentials (the reverse isn’t true).  In fact, the best physicians I’ve encountered are pragmatic in their approach but have the deeper knowledge to leverage when need arises
• Medics might effectively deal with 80+% of our problems, but that remaining ~20% can be critical 
• A person can start out as a medic and then become a physician later, as need and resources dictate  
• Physicians tend to be paid more

Bottom line — knowledge and understanding are never a bad thing, but it requires extra effort to acquire them.  And, as Mike points out, the simple approach is often good enough and may be all we can hope for given our individual circumstances.  For myself though, I prefer a deeper understanding of our complex problem space.  I want to be able to answer the hard questions about why and how.  But that’s just me.

BTW - I was amused at Mike’s characterization of risk analysis as Black Magic, as this phrase would also have been used in the past to describe medical and scientific concepts/practices we take for granted today.  

So, can we trust that the time stamp in the log file or the one added by the log management system correctly describes when the event actually happened?

We will start from locating the timestamps in logs. Most of the log formats, such as file-based logs (web, application, some security gear, etc) and syslog, Windows event logs, database audit tables, proprietary ones, contain a timestamp. In fact, once I saw somebody use a timestamp to define logs as "timed records of IT activity." So, time is critical for logs being, well, logs :-) At this point it is worthwhile to note that file-based logs will contain a timestamp IN the file, while syslog records arriving over the UDP or TCP port 514 connection are usually timestamped upon arrival BY the syslog daemon (using its own "knowledge" of time) - and then it shows up in the syslog files in  /var/log.

Let's assess whether this "in-log timestamp" provides an adequate way of timing the actual event that is being logged. Answering this question is important for investigations and troubleshooting, but becomes  nearly a matter of life and death in case of log forensics.

Here are some fun cases and issues to consider:

First, what are the chances of a  completely false timestamp in logs (BTW, today is Jan 1, 1970!) When might that happen? Typically when a logging system own clock is reset or not set correctly. This timestamp clearly should NOT be trusted. 

Second, we can say that it’s always 5PM somewhere: in other words, what time zone are your logs in? EST? PDT? GMT? UTC? Or any of more than 24 other possibilities. If you have no idea, you should not trust the timestamp.

Third, are you in drift? Is your system clock? Those pesky drift seconds turn into minutes which then work to undermine the accuracy of timing the records (and thus your certainly and trust in evidence quality)

Fourth, syslog forwarder mysteries are plenty: some of the syslog messages will be delayed in transit and the  be timestamped by the final recipient daemon, thus completely losing when the event was originally logged. Admittedly, this delayed syslog is rare, but as more people employ buffering syslog daemons (e.g. syslog-ng), it might happen more often.

Fifth, more esoteric, but still real (and really annoying): some system logs will contain two timestamps. If you don't possess in-depth knowledge of this specific log, confusion has a chance to cut the trust as well (so, which timestamp should I use?)

Sixth, most people will not think that they will fall to something that stupid: 24 vs 12 hour time. However, when facing an unknown (and poorly designed!) log format, beware that 5:17 might well be 17:17...

Finally, if you know that something got logged at 5:17AM, then when did it happen? Beware of "Log lag!" This issues is actually to tricky to give it justice here... The simplest example is when the process leaves a log records when it exits not when it starts, possibly days earlier (thus creating a log lag).

As we dive into more issues with timing logs, we also need to think about sequence timing and absolute timing. Sequence of logged events is a critical fact! Miss the sequence and the whole “house of cards” goes …  But! Absolute time is also important! Can we be assured of both all the time? (hint: no)

So,  when you look at logs next time and you see a timestamp there - start thinking about all this :-)