This is just ridiculous. Of course the bad guys will use all the communications tools available to the rest of us. They have to communicate, after all. They'll also use cars, water faucets, and all-you-can-eat buffet lunches. So what?

This commentary is dead on:

Steven Aftergood, a veteran intelligence analyst at the Federation of the American Scientists, doesn't dismiss the Army presentation out of hand. But nor does he think it's tackling a terribly seriously threat. "Red-teaming exercises to anticipate adversary operations are fundamental. But they need to be informed by a sense of what's realistic and important and what's not," he tells Danger Room. "If we have time to worry about 'Twitter threats' then we're in good shape. I mean, it's important to keep some sense of proportion."

This is a quick and dirty video documentary of the things that when on around the talks and event at Phreaknic 12 (2008). Don't watch if you get sick at shaky cam movies like Blair Witch or Cloverfield. A rough timeline of the content in the video is as follows:

        Intro and leaving Louisville with Brian. Morgellon talks about hacking the Arduino micro controller platform. Sorteal talks about the LiVes Open Source video editor. AT&T Batman building by night. Mojo-JoJo soldering some stuff for the shooting range. The patron gods of hackerdom. Registration. Con swag overview. Morgellon  gets his discreet logic on. AK-47 building with HandGrip and Buttstock. Froggy talks up Notacon, which I plan to go to next year. Skydog explains the Jware chair toss event, and then we compete. Rootwars hacker wargames. I ask Int80 about using his nerdcore music in some of my videos. NotLarry explains rootwars. Some iPhone hacking with Lee Baird and John Skinner. I do a little Bluecaseing/Warnibbling with the Bluetooth on my Nokia n810. John, Lee, Brian and I go to the German restaurant. I blind DOSman with the light from my camera and check out what folks are doing with the Arduinos Droops brought for folks to play with. I check back in on R00tW4rz. I blind Droops. I talk Ettercap filters with operat0r. USB door key fun with the Arduino. More breadboard fun. Nokia n810 + Ettercap Filter + Lemon-part = win. Int80 gets down with his own bad self, and the rest of Phreaknic. I find an energy drink with protein. Folks play with the hardware keyloggers I brought, and we have some epic fail with the IBM Model M + USB adapter + Mac OS 10.5. Winn Schwartau joins in on the keylogger fun. DOSman and Zack use a directional antenna from the 9th floor to search downtown Nashville for WiFi access points. Zoom in on Al. John and Lee eat jerky. Daren and Shannon from Hak5 blind me this time. :) Then they do a quick interview. I interview TRiP about the legalities of wardriving, sniffing and leaving your access point open so you have plausible deniability of copyright infringement (most likely it won't hold water in court if you are a computer geek). I give Hak5 Daren beef jerky. Ziplock had more con badges than God. I meet up with Iridium. I talk with Nightcarnage about the audio/video setup at Phreaknic. As I predicted, the Potters won the WiFi Race. I say why this was the best Phreaknic ever. Using green lasers on crack dealers. Techno in the dark, the Aiptek action HD does not do well in low light. Nicodemius shows off his Minority Report like multi-touch table. Hula hoop contest. I check back in with Jeff Cotton and his USB keyed door. I strap on my gear to leave the con. Brian and I do a wrap up of our thoughts on Phreaknic 2008.

Unfortunately, there's not really anything to his response. It's obvious he doesn't want to admit that they've been checking ID's all this time to no purpose whatsoever, so he just emits vague generalities like a frightened squid filling the water with ink. Yes, some of the stunts in article are silly (who cares if people fly with Hezbollah T-shirts?) so that gives him an opportunity to minimize the real issues.

Watch-lists and identity checks are important and effective security measures. We identify dozens of terrorist-related individuals a week and stop No-Flys regularly with our watch-list process.

It is simply impossible that the TSA catches dozens of terrorists every week. If it were true, the administration would be trumpeting this all over the press -- it would be an amazing success story in their war on terrorism. But note that Hawley doesn't exactly say that; he calls them "terrorist-related individuals." Which means exactly what? People so dangerous they can't be allowed to fly for any reason, yet so innocent they can't be arrested -- even under the provisions of the Patriot Act.

And if Secretary Chertoff is telling the truth when he says that there are only 2,500 people on the no-fly list and fewer than 16,000 people on the selectee list -- they're the ones that get extra screening -- and that most of them live outside the U.S., then it is statistically impossible that the TSA identifies "dozens" of these people every week. The math just doesn't make sense.

And I also don't believe this:

Behavior detection works and we have 2,000 trained officers at airports today. They alert us to people who may pose a threat but who may also have items that could elude other layers of physical security.

It does work, but I don't see the TSA doing it properly. (Fly El Al if you want to see it done properly.) But what I think Hawley is doing is engaging in a little bit of psychological manipulation. Like sky marshals, the real benefit of behavior detection isn't whether or not you do it but whether or not the bad guys believe you're doing it. If they think you are doing behavior detection at security checkpoints, or have sky marshals on every airplane, then you don't actually have to do it. It's the threat that's the deterrent, not the actual security system.

This doesn't impress me, either:

Items carried on the person, be they a 'beer belly' or concealed objects in very private areas, are why we are buying over 100 whole body imagers in upcoming months and will deploy more over time. In the meantime, we use hand-held devices that detect hydrogen peroxide and other explosives compounds as well as targeted pat-downs that require private screening.

Optional security measures don't work, because the bad guys will opt not to use them. It's like those air-puff machines at some airports now. They're probably great at detecting explosive residue off clothing, but every time I have seen the machines in operation, the passengers have the option whether to go through the lane with them or another lane. What possible good is that?

The closest thing to a real response from Hawley is that the terrorists might get caught stealing credit cards.

Using stolen credit cards and false documents as a way to get around watch-lists makes the point that forcing terrorists to use increasingly risky tactics has its own security value.

He's right about that. And, truth be told, that was my sloppiest answer during the original intervied. Thinking about it afterwards, it's far more likely is that someone with a clean record and a legal credit card will buy the various plane tickets.

This is new:

Boarding pass scanners and encryption are being tested in eight airports now and more will be coming.

Ignoring for a moment that "eight airports" nonsense -- unless you do it at every airport, the bad guys will choose the airport where you don't do it to launch their attack -- this is an excellent idea. The reason my attack works, the reason I can get through TSA checkpoints with a fake boarding pass, is that the TSA never confirms that the information on the boarding pass matches a legitimate reservation. If all TSA checkpoints had boarding pass scanners that connected to the airlines' computers, this attack would not work. (Interestingly enough, I noticed exactly this system at the Dublin airport earlier this month.)

Stopping the ‘James Bond’ terrorist is truly a team effort and I whole-heartedly agree that the best way to stop those attacks is with intelligence and law enforcement working together.

This isn't about "Stopping the 'James Bond' terrorist," it's about stopping terrorism. And if all this focus on airports, even assuming it starts working, shifts the terrorists to other targets, we haven't gotten a whole lot of security for our money.

FYI: I did a long interview with Kip Hawley last year. If you haven't read it, I strongly recommend you do. I pressed him on these and many other points, and didn't get very good answers then, either.

Now I’ve been “accused” of being a quant in the past (hi rybolov!) but in reality the only dogs I have in this fight are the model and the application of scientific method - and really, ethically speaking, I have to be tied to the latter while applying the former.

And I see a false dichotomy in this whole Quant vs. Qual thing.  We, as a profession, tend to create a political divide between the two which, if it even exists, I’d say is based more on our ignorance rather than our expertise.  After all, we are the profession that regularly multiplies across ordinal scales and uses wonderful models like R=VxTxI.   As someone  learning to deal in probabilities and rationalism, I have to recognize that this discussion is really just about the act of observation using different metrics of measurement.

But how we’re going about observing does not change the fact that there is measurement based on observation.  So if I’m working with you I can easily turn your qualitative scale into a quantitative one, and vice-versa.  Yes, Shrdlu, if we had the time, even your most seemingly Qual things could be Quant! (This flexible world view, btw, is an outcome of that new-fangled Bayesian thing).

COGNITIVE BIAS A-PLENTY

But back to what Rich is saying there about information security and risk - and he isn’t/won’t be the only one saying these sorts of things - we should try to understand what’s really going on rather than get caught up in the emotional hurricane.  Our profession suffers several forms of cognitive bias.  The nature of our jobs and what we do can cause us to be focused on the outcome and not the quality of the decision at the time it was made.  We want to bring in things from other professions that are useful, but at times we do view things outside our profession with false correlation to our own (unfortunately for those who write these sorts of articles, financial risk is completely different than operational risk).  We also have the tendency to focus on negative outcomes without acknowledging the positive outcomes (For example, I hear that Alan Greenspan’s new firm is up a couple of $billion in all this mess since he joined them, short sellers are doing quite well - must be because they have qualitative models or something -grin-).  The effect of these biases are compounded by the facts that proper correlation takes more work than we usually give it, and rational thought is not that easy when there’s a witch-hunt mentality.

What also floats in water? (link to Youtube)

WHAT SHOULD WE BE THINKING ABOUT?

So as you and I read opinions that seem to be the polar opposite of irrational exuberance (and there will be plenty between now and the election) we’ll have to ask ourselves, “what really failed here?”  At the risk (pun) of over-simplification:

• Was There an Error on the part of Probability Theory?

After all, Probability Science like all other fields of knowledge is always “advancing” as they say.  So perhaps probability theory is wrong somehow?

I’m personally disinclined to put the blame here, primarily because I would think that there would be evidence from other fields (like Quantum Mechanics) that something is amiss waaaaay before it hit a field like economics.

• Was There Error In The Model Used to Determine Risk?

Some people who understand real estate valuation and complex derivatives and financial risk want to put the blame here.  It’s a little too early to tell, but one thing is for sure - Financial risk is so different from operational risk I couldn’t begin to hazard an opinion on the subject.   But it would seem that this is really somewhere we might look.

• Was There Error In The  Scale Used (Quantitative vs. Qualitative)?

Honestly?  I find it extremely difficult to understand how this could be the source of financial ruin.

• Was There Error on the part of the Decision Maker?

What if all of the above were just fine, and the decision maker chose short term gain over long term stability?  What if this was (to simplify the matter greatly) a choice of “heads” over “tails” and the coin landed on tails?  What if the model represented the right risk (probability of negative outcome vs. positive outcome), but the complex derivative was sold to someone else who had poor “risk management” (ability to make a good decisions)?

Now I have no clue about complex derivatives, and I’m oversimplifying to be sure - chances are like most things, there are several problems that helped create the primary cause. But it seems to me that as we go into incident response mode for the economy, it’s more helpful to do so in a rational, logical manner.

OTHER THINGS WE MIGHT WANT TO CONSIDER

Consider the Source
Some authors (who I think tend to exploit outcome and hindsight bias,and then combine those with indirect ad hominem attacks in order to sell their books), are actually putting forth arguments against the use of analytics.  The source of this is a current epistemic debate between those who believe that only falsification is certain, and those who maintain that neither proof nor falsification are certain, there are only probabilities.    So before you go believing any “quadrants” of usefulness on faith - I encourage you to understand what is at the heart of the discussion.

We All Have to Live In The Real World
The sun will rise tomorrow, and someone will try to find the source of the problem and do a better job.  Now chances are, they’ll be doing it in a quantitative manner.  Chances are also that at some point their models will fail and we’ll need to build new ones.  And this will happen whether the field is cosmology, economics, meteorology, information security, or professional baseball.

WHAT ABOUT YOU, ALEX?

I’m far from certain and subject to change, but these days I lean towards Robin Hanson & MIchael Lewis w/regards to placing blame.