In the paper, we propose a simple solution: take-down companies should share their raw, unverified feeds of phishing URLs with their competitors. Each company can examine the raw feed, pick out the websites impersonating their clients, and focus on removing these sites.

Since we presented our findings to the Anti-Phishing Working Group eCrime Researchers Summit, we have received considerable feedback from take-down companies. Take-down companies attending the APWG meeting understood that sharing would help speed up response times, but expressed reservations at sharing their feeds unless they were duly compensated. Eric Olsen of Cyveillance (another company offering take-down services) has written a comprehensive rebuttal of our recommendations. He argues that competition between take-down companies drives investment in efforts to detect more websites. Mandated sharing of phishing URL feeds, in his view, would undermine these detection efforts and cause take-down companies such as Cyveillance to exit the business.

I do have some sympathy for the objections raised by the take-down companies. As we state in the paper, free-riding (where one company relies on another to invest in detection so they don’t have to) is a concern for any sharing regime. Academic research studying other areas of information security (e.g., here and here), however, has shown that free-riding is unlikely to be so rampant as to drive all the best take-down companies out of offering service, as Mr. Olsen suggests.

While we can quibble over the extent of the threat from free free-riding, it should not detract from the conclusions we draw over the need for greater sharing. In our view, it would be unwise and irresponsible to accept the current status quo of keeping phishing URL feeds completely private. After all, competition without sharing has approximately doubled the lifetimes of phishing websites! The solution, then, is to devise a sharing mechanism that gives take-down companies the incentive to keep detecting more phishing URLs.

Here is our stab at devising a suitable sharing mechanism. We propose the creation of a members-only sharing club with compensation for net contributors paid for by net receivers. Take-down companies submit real-time copies of their entire feeds to a trusted third party (for the sake of argument, let’s assume that the APWG takes on this role). The APWG collates the individual feeds, marks the source of each submission (i.e., which take-down company) along with a timestamp. The APWG makes the amalgamated feed available immediately to all members. The members pick out phishing URLs impersonating their own clients, while ignoring the rest. Crucially, the expensive task of verifying phishing URLs and initiating take-down continues to be performed by the take-down company.

Periodically, the combined feed is audited to determine the reciprocity of contributions. Take-down companies provide a list of their clients to the auditor. The auditor then computes the number of phishing websites impersonating each take-down company’s clients that are missed by the takedown company but identified by others. The auditor also tallies the time difference for phishing websites that are identified by others first.

For example, suppose bank A1 has hired take-down company A to remove phishing sites on its behalf, and bank B1 has hired take-down company B. Suppose 500 phishing sites impersonate A1, and that A identifies 400 while B identifies an additional 100 sites missed by A. Likewise, suppose another 500 phishing sites impersonate bank B1, and that B identifies 300 while A identifies an additional 200 sites missed by B. B has received a net of 100 useful phishing sites more from A than B has given to A. Consequently, B should pay A a previously-agreed ‘finder’s fee’ for identifying these extra 100 websites.

The ‘finder’s fee’ provides additional incentive for take-down companies to invest in better phishing website detection. Designed properly, such a sharing club can overcome the potential for free-riding that companies such as Cyveillance fret about, while increasing sharing to shorten phishing website lifetimes.

Some subtleties must be mentioned, however. If the finder’s fee is big enough, some companies may be tempted to cheat to minimize their payout. For instance, underperforming take-down companies could claim to have independently discovered missing data from their feed shortly after collecting it from the shared feed. This can be mitigated by adding a credible threat of detection — inserting a few dubious fake phishing URLs that only appear in the shared feed. If the company claims to have ‘independently’ rediscovered these URLs, then they will be caught cheating. Another issue is that the auditing system does incur some overhead, which could be avoided if sharing was made unconditional.

To sum up, we recognize that many take-down companies will be reticent to share. However, we feel that sharing is too important to the goal of tackling phishing to brush aside because of a few inevitable complications. For the good of protecting consumers, the anti-phishing industry should learn to co-operate!

Is it supposed to make me feel better about getting my problem fixed if I’m talking to someone in the Midwest versus someone in Bangalore? (Please no hate mail – I’m from the Midwest.) Honestly, I just want my computer to stop showing me a blue screen of death.

But apparently, I might be in the minority. According to the Black Book of Outsourcing (yes, outsourcing has a black book), reverse outsourcing is on the rise with “India’s leading service providers opening offices on Main Street, USA” to be closer to customers (mainly North American) and draw from the “local talent pools”.

The one area of outsourcing bucking this trend – infrastructure management. Co-writer Scott Wilson says that infrastructure management is largely automated, low touch and does not involve a lot of interaction.

Speaking as a vendor of infrastructure management tools, that’s a bunch of malarkey. Perhaps at a very low level this is true (i.e., is the device responding), but that’s just the tip of the iceberg when it comes to monitoring performance, availability and SLAs for today’s networks, systems and applications.

Certainly as vendors, we try to put as much automation as possible into our toolsets – helping our customers to simplify IT management wherever possible, enabling them to be proactive by setting up “intelligent” alarms and thresholds that warn of problems before they become showstoppers and reacting at a speed in this increasingly virtual world that simply is not possible for human manual interaction.

But infrastructure management doesn’t happen in a vacuum and you can bet when something goes wrong which affects some mission-critical app state-side, that there is a LOT of communication and interaction. And it takes a lot of work and setup to get to a level of automation where the alerting is proactive and intelligent and customized for each business.

One of the main points of tools like ours is to automate where possible in order to free up the valuable time of the sysadmins, network engineers, IT managers, etc to do the higher order work – which is how they’ll get to the next level of infrastructure management. Beyond “is it up”, infrastructure management should be providing answers to questions like: “is it always up”, “is it doing what I expected it to do” and “will it still be working as expected as my company grows”.

The report was written by a committee whose members include William Perry, a professor at Stanford University; Charles Vest, the former president of MIT; W. Earl Boebert, a retired senior scientist at Sandia National Laboratories; Cynthia Dwork of Microsoft Research; R. Gil Kerlikowske, Seattle's police chief; and Daryl Pregibon, a research scientist at Google.

They admit that far more Americans live their lives online, using everything from VoIP phones to Facebook to RFID tags in automobiles, than a decade ago, and the databases created by those activities are tempting targets for federal agencies. And they draw a distinction between subject-based data mining (starting with one individual and looking for connections) compared with pattern-based data mining (looking for anomalous activities that could show illegal activities).

But the authors conclude the type of data mining that government bureaucrats would like to do--perhaps inspired by watching too many episodes of the Fox series 24--can't work. "If it were possible to automatically find the digital tracks of terrorists and automatically monitor only the communications of terrorists, public policy choices in this domain would be much simpler. But it is not possible to do so."

A summary of the recommendations:

• U.S. government agencies should be required to follow a systematic process to evaluate the effectiveness, lawfulness, and consistency with U.S. values of every information-based program, whether classified or unclassified, for detecting and countering terrorists before it can be deployed, and periodically thereafter.

• Periodically after a program has been operationally deployed, and in particular before a program enters a new phase in its life cycle, policy makers should (carefully review) the program before allowing it to continue operations or to proceed to the next phase.

• To protect the privacy of innocent people, the research and development of any information-based counterterrorism program should be conducted with synthetic population data... At all stages of a phased deployment, data about individuals should be rigorously subjected to the full safeguards of the framework.

• Any information-based counterterrorism program of the U.S. government should be subjected to robust, independent oversight of the operations of that program, a part of which would entail a practice of using the same data mining technologies to "mine the miners and track the trackers."

• Counterterrorism programs should provide meaningful redress to any individuals inappropriately harmed by their operation.

• The U.S. government should periodically review the nation's laws, policies, and procedures that protect individuals' private information for relevance and effectiveness in light of changing technologies and circumstances. In particular, Congress should re-examine existing law to consider how privacy should be protected in the context of information-based programs (e.g., data mining) for counterterrorism.

Here are more news articles on the report. I explained why data mining wouldn't find terrorists back in 2005.

And while I’m at it, I wanted to draw attention to some of the other presentations that I thought were very interesting, including one by Karine Peralta “Specifying Security Aspects in UML Models” and “Curriculum for Modelling Security: Experiences and Lessons Learned.”

Interop NY 2008 was up considerably in size from the show in 2007.  According to Lenny Heymann, the GM of Interop, this is a trend that they expect to continue.  My personal experience was that the size of the vendors was also up this year.  I think there were so few startups that “Startup City” was pulled from the show completely.  In any case, the show floor was full and there was plenty of attendee traffic to go around.

Definitely helping out from a traffic and draw perspective was the addition of the Web 2.0 Expo - Interop was co-located with both Mobile Business Expo and the Web 2.0 show. It seems like that buzzword still hasn’t lost most of its luster.

From the InteropNet perspective, the main feeling was one of being rushed.  With the show only lasting two days, and the InteropNet team only having a couple of days of ramp up time, everything was compressed into a much shorter period than in Las Vegas.  While this would normally be a challenge, it’s an even bigger challenge at the Javits where the InteropNet team was allowed to do almost nothing ourselves because of union rules.  You’d be surprised how frustrated you can make a network guy who’s told that he has to stand there and watch the electrician plug things in, rather than just doing it himself.  The only thing faster than the InteropNet team getting the Interop NY network up, was my pedicab ride to the InteropNet Booze Cruise.

In any case, everything came off without a hitch, and EM7 performed flawlessly catching a couple of power outages that last day and alerting everyone before the batteries on the UPSes had a chance to run down.

Over the next couple of weeks I’ll analyze the data from the show to see how many tickets were handled, amount of bandwidth consumed, etc and we’ll do a comparison to Interop Las Vegas.

We’re (both ScienceLogic and me personally) looking forward to Interop 2009.