In the paper, we propose a simple solution: take-down companies should share their raw, unverified feeds of phishing URLs with their competitors. Each company can examine the raw feed, pick out the websites impersonating their clients, and focus on removing these sites.

Since we presented our findings to the Anti-Phishing Working Group eCrime Researchers Summit, we have received considerable feedback from take-down companies. Take-down companies attending the APWG meeting understood that sharing would help speed up response times, but expressed reservations at sharing their feeds unless they were duly compensated. Eric Olsen of Cyveillance (another company offering take-down services) has written a comprehensive rebuttal of our recommendations. He argues that competition between take-down companies drives investment in efforts to detect more websites. Mandated sharing of phishing URL feeds, in his view, would undermine these detection efforts and cause take-down companies such as Cyveillance to exit the business.

I do have some sympathy for the objections raised by the take-down companies. As we state in the paper, free-riding (where one company relies on another to invest in detection so they don’t have to) is a concern for any sharing regime. Academic research studying other areas of information security (e.g., here and here), however, has shown that free-riding is unlikely to be so rampant as to drive all the best take-down companies out of offering service, as Mr. Olsen suggests.

While we can quibble over the extent of the threat from free free-riding, it should not detract from the conclusions we draw over the need for greater sharing. In our view, it would be unwise and irresponsible to accept the current status quo of keeping phishing URL feeds completely private. After all, competition without sharing has approximately doubled the lifetimes of phishing websites! The solution, then, is to devise a sharing mechanism that gives take-down companies the incentive to keep detecting more phishing URLs.

Here is our stab at devising a suitable sharing mechanism. We propose the creation of a members-only sharing club with compensation for net contributors paid for by net receivers. Take-down companies submit real-time copies of their entire feeds to a trusted third party (for the sake of argument, let’s assume that the APWG takes on this role). The APWG collates the individual feeds, marks the source of each submission (i.e., which take-down company) along with a timestamp. The APWG makes the amalgamated feed available immediately to all members. The members pick out phishing URLs impersonating their own clients, while ignoring the rest. Crucially, the expensive task of verifying phishing URLs and initiating take-down continues to be performed by the take-down company.

Periodically, the combined feed is audited to determine the reciprocity of contributions. Take-down companies provide a list of their clients to the auditor. The auditor then computes the number of phishing websites impersonating each take-down company’s clients that are missed by the takedown company but identified by others. The auditor also tallies the time difference for phishing websites that are identified by others first.

For example, suppose bank A1 has hired take-down company A to remove phishing sites on its behalf, and bank B1 has hired take-down company B. Suppose 500 phishing sites impersonate A1, and that A identifies 400 while B identifies an additional 100 sites missed by A. Likewise, suppose another 500 phishing sites impersonate bank B1, and that B identifies 300 while A identifies an additional 200 sites missed by B. B has received a net of 100 useful phishing sites more from A than B has given to A. Consequently, B should pay A a previously-agreed ‘finder’s fee’ for identifying these extra 100 websites.

The ‘finder’s fee’ provides additional incentive for take-down companies to invest in better phishing website detection. Designed properly, such a sharing club can overcome the potential for free-riding that companies such as Cyveillance fret about, while increasing sharing to shorten phishing website lifetimes.

Some subtleties must be mentioned, however. If the finder’s fee is big enough, some companies may be tempted to cheat to minimize their payout. For instance, underperforming take-down companies could claim to have independently discovered missing data from their feed shortly after collecting it from the shared feed. This can be mitigated by adding a credible threat of detection — inserting a few dubious fake phishing URLs that only appear in the shared feed. If the company claims to have ‘independently’ rediscovered these URLs, then they will be caught cheating. Another issue is that the auditing system does incur some overhead, which could be avoided if sharing was made unconditional.

To sum up, we recognize that many take-down companies will be reticent to share. However, we feel that sharing is too important to the goal of tackling phishing to brush aside because of a few inevitable complications. For the good of protecting consumers, the anti-phishing industry should learn to co-operate!

These “take-down” companies are generally specialist offshoots of more general “brand protection” companies, and are hired by banks to handle removal of fake phishing websites.

When we examined our data more carefully we found that we were receiving “feeds” of phishing website URLs from several different sources — and the “take-down” companies that were passing the data to us were not passing the data to each other.

So it often occurs that take-down company A knows about a phishing website targeting a particular bank, but take-down company B is ignorant of its existence. If it is company B that has the contract for removing sites for that bank then, since they don’t know the website exists, they take no action and the site stays up.

Since we were receiving data feeds from both company A and company B, we knew the site existed and we measured its lifetime — which is much extended. In fact, it’s somewhat of a mystery why it is removed at all! Our best guess is that reports made directly to ISPs trigger removal.

The paper contains all the details, and gives all the figures to show that website lifetimes are extended by about 5 days when the take-down company is completely unaware of the site. On other occasions the company learns about the site some time after it is first detected by someone else; and this extends the lifetimes by an average of 2 days.

Since extended lifetimes equate to more unsuspecting visitors handing over their credentials and having their bank accounts cleaned out, these delays can also be expressed in monetary terms. Using the rough and ready model we developed last year, we estimate that an extra $326 million per annum is currently being put at risk by the lack of data sharing. This figure is from our analysis of just two companies’ feeds, and there are several more such companies in this business.

Not surprisingly, our paper suggests that the take-down companies should be sharing their data, so that when they learn about websites attacking banks they don’t have contracts with, they pass the details on to another company who can start to get the site removed.

We analyse the incentives to make this change (and the incentives the companies have not to do so) and contrast the current arrangements with the anti-virus/malware industry — where sample suspect code has been shared since the early 1990s.

In particular, we note that it is the banks who would benefit most from data sharing — and since they are paying the bills, we think that they may well be in a position to force through changes in policy. To best protect the public, we must hope that this happens soon.

Without replacing these poles, the city would be unable to use the municipal services from which it still plans to save $3.5m over the 10-year contract life, and thus it would be pennywise and pound foolish to leave the status quo.

In the case of phishing, it is relatively clear. The developers believe the PKI book. The PKI people believe in the efficacy of digital signatures to prove stuff. The cryptographers believe in the perfection of mathematics, and the security world believes in the completeness of their own learning. They are all wrong, but only at the large level of generalisations, not at the detailed level of particular claims. Any one of the claims, in isolation can be shown to be true. But, generalising these brittle claims to be solid building blocks is a completely different question. Few of the claims are strong enough to partake in a general model without severe support; the general model of secure browsing is the best evidence of how it is secure in name only.

How then is it built? By accident or by design, a series of claims meet together in a holy ring of righteous architecture. Each of the proponents claim loudly that their part is strong, but the ring has no strength. Eventually, one of the claims in the links is broken. For phishing, the browsers never did have the potential to show authenticity; not only did they not have the security strength to do it (c.f., Skype v. CSRF), they didn't even do it in practice (recall the lost padlock?), and their recent efforts to show authenticity (c.f. colour debate) reveal how far they are from understanding even the goal, let alone the implementation. Once that link was broken, and money was made, all the others revealed their weaknesses, as crooks systematically worked to breach the lot.

If we look at the wider financial collapse, now underscored by the nationalisation of the worlds biggest financiers of mortgages ($ 5.3 trillion.... or is it $ 5.4 ?), we see the same pattern. The bankers believed in their product. The originators believed in their origination, the securitizers believed in their free market and accurate price, and the holders believed in the assets. The CDO, the subprime, the other 100 special names, each was a contract. Each was clear in and of itself. But, when placed end-to-end, in a line, with a bunch of other agreements, the claims that were good in isolation were not strong enough to participate in the super-claim made of the overall edifice.
The financial system was built like a bridge; each piece rested on the previous one. And then, the clever architects bent the bridge around ... and around again, until the first piece met the last. The elegant keystone of finance was to finally lift up the first one to rest on the last.

Thus, the banks themselves invested their capital in their own product.