A recent article in the London Review of Books revealed that a number of private companies now sell off-the-shelf data-mining solutions to government spies interested in analyzing mobile-phone calling records and real-time location information. These companies include ThorpeGlen, VASTech, Kommlabs, and Aqsacom--all of which sell "passive probing" data-mining services to governments around the world.

ThorpeGlen, a U.K.-based firm, offers intelligence analysts a graphical interface to the company's mobile-phone location and call-record data-mining software. Want to determine a suspect's "community of interest"? Easy. Want to learn if a single person is swapping SIM cards or throwing away phones (yet still hanging out in the same physical location)? No problem.

In a Web demo (PDF) (mirrored here) to potential customers back in May, ThorpeGlen's vice president of global sales showed off the company's tools by mining a dataset of a single week's worth of call data from 50 million users in Indonesia, which it has crunched in order to try and discover small anti-social groups that only call each other.

Learn how file integrity monitoring solutions work and the capabilities you should expect your solution to have. Then review a detailed checklist you should complete before purchasing your solution. Finally, discover how Tripwire Enterprise effectively combines file integrity monitoring with configuration assessment-a single configuration control solution that proactively assesses and monitors the IT infrastructure and enables organizations to achieve and maintain compliance with standards and regulations.

A good place to being to look for clues to an answer is Wikipedia, where the opinion of the author there is,

 ”A simple and practical definition, however, would be how an entity (i.e., business) arrives at an optimal or realistic decision based on existing data.”

Quoting the Wikipedia author(s) further,

“Common applications of Analytics include the study of business data using statistical analysis in order to discover and understand historical patterns with an eye to predicting and improving business performance in the future. Also, some people use the term to denote the use of mathematics in business. Others hold that field of analytics include the use of Operations Research, Statistics and Probability. However, it would be erroneous to limit the field of analytics to only statistics and mathematics.”

The Wikipedia author(s) continue their discussion of analytics, as follows;

“Analytics closely resembles statistical analysis and data mining, but tends to be based on modeling involving extensive computation. Some fields within the area of analytics are enterprise decision management, marketing analytics, predictive science, strategy science, credit risk analysis and fraud analytics.”

All of these topics above are CEP-related areas involving complex events and situations based on the need for optimal and reliable real-time capabilities to make meaningful (business) decisions. 

Simple pattern matching, event mediation and routing, and basic mathematical calculations do not really fall into the realm of complex event processing.  Instead, CEP is real-time decision support based on modeling and “extensive” computation.  In a nutshell, complex events and situations require analytical models that are non-trivial and that is why without analytics, there is no true “complex event processing.”

See also:

WIkipedia on Predictive Analytics

You will discern a certain amount of enthusiasm for blocking, and for a “something must be done” approach. However, in coming to their conclusions, they do not, in my view, seem to have listened too hard to the evidence, or sought out expertise elsewhere in the world…

Google/YouTube told them that 10 hours of video was posted every minute, and the amount is increasing. In the oral evidence session an MP helpfully suggested: “That video content is tagged. You do not need to look at every single minute of video content. Surely you could have people who would look at the video content which is tagged with labels which suggest it could be inappropriate.” Of course “happy_slapping.wmv” or “fluffy_bunnies.avi” must always contain exactly what it says on the tin (not!) but unaccountably Google said it was a “fair suggestion”, so perhaps my cynicism is misplaced.

However, back to blocking.

I submitted some evidence of my own, which the committee summarised, reasonably accurately:

Dr Richard Clayton, a researcher in the Security Group of the Computer Laboratory at Cambridge University and author of several academic papers on methods for blocking access to Internet content, pointed out that there was no single blocking method which was both inexpensive and discerning enough to block access to only one part of a large website (such as FaceBook). In his view, the fatal flaw of all network-level blocking schemes was the ease with which they could be overcome, either by encrypting content or by the use of proxy services hosted outside the UK.

The committee’s conclusion, having read this was:

At a time of rapid technological change, it is difficult to judge whether blocking access to Internet content at network level by Internet service providers is likely to become ineffective in the near future. However, this is not a reason for not doing so while it is still effective for the overwhelming majority of users.

which I suppose logically means that the committee thinks that blocking should now be discarded as a policy option — but somehow I think that isn’t their intended meaning.

The Committee should perhaps have a look at this Australian report, which found that ISP level content filtering (and in Australia the politicians want to use ISP level filtering to provide a child-friendly Internet) did work (up to a point) at Tier 3 (the smallest) ISPs. The up-to-a-point is that unlike previous tests the systems didn’t completely wreck the browsing experience by slowing it down. However, the systems blocked only 85-98% of illegal material and similar percentages of material suitable for adults but not for younger children. Interestingly some products were better at different categories.

Getting that many sites wrong is really quite significant, so it’s difficult to see this as a ringing endorsement for blocking the web. Additionally, the Australian report found that the blocking was useless on “non-web” protocols (such as peer-to-peer) and their report specifically didn’t consider cost, or ease of circumvention — so it’s not just UK politicians not wanting to consider evidence on that topic!

Finally, I should note that the Culture Media and Sport Committee has also ignored some rather more recent academic work. The MPs have put into their report that they were horrified to discover that child sexual abuse images took 24 hours to remove in the UK. What (should they ever learn of it) will they make of the recent discovery by Tyler Moore and myself that shows that if the website is hosted abroad then a month is more to be expected?

Here's the story. Every Oyster card has a radio-frequency identification chip that communicates with readers mounted on the ticket barrier. That chip, the "Mifare Classic" chip, is used in hundreds of other transport systems as well — Boston, Los Angeles, Brisbane, Oslo, Amsterdam, Taipei, Shanghai, Rio de Janeiro — and as an access pass in thousands of companies, schools, hospitals, and government buildings around Britain and the rest of the world.

The security of Mifare Classic is terrible. This is not an exaggeration; it's kindergarten cryptography. Anyone with any security experience would be embarrassed to put his name to the design. NXP attempted to deal with this embarrassment by keeping the design secret.

The group that broke Mifare Classic is from Radboud University Nijmegen in the Netherlands. They demonstrated the attack by riding the Underground for free, and by breaking into a building. Their two papers (one is already online) will be published at two conferences this autumn.

The second paper is the one that NXP sued over. They called disclosure of the attack "irresponsible," warned that it will cause "immense damages," and claimed that it "will jeopardize the security of assets protected with systems incorporating the Mifare IC." The Dutch court would have none of it: "Damage to NXP is not the result of the publication of the article but of the production and sale of a chip that appears to have shortcomings."

Exactly right. More generally, the notion that secrecy supports security is inherently flawed. Whenever you see an organization claiming that design secrecy is necessary for security — in ID cards, in voting machines, in airport security — it invariably means that its security is lousy and it has no choice but to hide it. Any competent cryptographer would have designed Mifare's security with an open and public design.

Secrecy is fragile. Mifare's security was based on the belief that no one would discover how it worked; that's why NXP had to muzzle the Dutch researchers. But that's just wrong. Reverse-engineering isn't hard. Other researchers had already exposed Mifare's lousy security. A Chinese company even sells a compatible chip. Is there any doubt that the bad guys already know about this, or will soon enough?

Publication of this attack might be expensive for NXP and its customers, but it's good for security overall. Companies will only design security as good as their customers know to ask for. NXP's security was so bad because customers didn't know how to evaluate security: either they don't know what questions to ask, or didn't know enough to distrust the marketing answers they were given. This court ruling encourages companies to build security properly rather than relying on shoddy design and secrecy, and discourages them from promising security based on their ability to threaten researchers.

It's unclear how this break will affect Transport for London. Cloning takes only a few seconds, and the thief only has to brush up against someone carrying a legitimate Oyster card. But it requires an RFID reader and a small piece of software which, while feasible for a techie, are too complicated for the average fare dodger. The police are likely to quickly arrest anyone who tries to sell cloned cards on any scale. TfL promises to turn off any cloned cards within 24 hours, but that will hurt the innocent victim who had his card cloned more than the thief.

The vulnerability is far more serious to the companies that use Mifare Classic as an access pass. It would be very interesting to know how NXP presented the system's security to them.

And while these attacks only pertain to the Mifare Classic chip, it makes me suspicious of the entire product line. NXP sells a more secure chip and has another on the way, but given the number of basic cryptography mistakes NXP made with Mifare Classic, one has to wonder whether the "more secure" versions will be sufficiently so.

This essay originally appeared in the Guardian.

In fact, it’s not just great artists like Mr. Lucas and I starting up old projects, our honourable colleagues wearing the black hats have got the same idea. We have new victims reporting in, rumours abound of an auth system compromise at Citi, the Ombudsman is backlogged with months of disputed withdrawal cases, and some like Alain Job are even going to court.

One original contributor to the phantom case histories has just been hit by a second phantom withdrawal five years on and is chalking up another case in the files. While her new phantom is a bread-and-butter skim incident (a magstripe clone used in the far east), amongst this mass, true phantoms — the real mystery cases — are on the rise too. Two new victims with whom I have been corresponding very kindly offered to fund the hosting for the revamped site.

Let’s consider one of these mysteries. The McGaughey case has been reported in the media in Northern Ireland: dozens of withdrawals taking place over four weeks, totaling almost five thousand pounds, all within a ten mile radius of the McGaughey’s home. Summarised that way it looks like a classic first party fraud (couple short on cash withdraw money, then deny it later). But no-one in the family is short on cash, the McGaugheys look after their card details carefully, and have solid alibis at the time of many of the withdrawals, and the interlocking pattern of real and disputed withdrawals is such that any third party would have a hard time taking and returning the card (whether covertly or in collusion with the McGaugheys). No-one appears to have either the means or the motive.

Unusually the bank has been very cooperative, providing logs from their authorisation system (BASE24), including all of the cryptograms, input data and transaction parameters covering the affected transactions. Everything turns on the Application Transaction Counter (ATC), an on-card counter which increments with every transaction initiated. If an EMV chip can be fully cloned (secret keys and all), then it will have to submit an ATC value when transacting, and if used in parallel with the real card, it won’t be long before the same number pops up twice in the auth system, or large gaps in the sequence appear. The McGaughey’s ATC sequence appears to interlock perfectly: clearly the original card was used?

Of course logs can be misinterpreted (Badger) or even faked, auth systems may not work as expected, and customers may lie and cheat following all sorts of agendas; just around the corner the missing piece of the jigsaw may lie, which reveals the truth behind the case. And there is the totally separate matter of who should suffer the loss in the interim, whilst the truth remains unclear. Liability for disputed withdrawals is the most hotly contested issue of all.

phantomwithdrawals.com can’t do much more for the McGaugheys, but it can bear witness. Documenting the incidence of phantoms and the experiences of customers disputing them adds much needed transparency to the process, and helps researchers and experts seek out the really interesting cases.

Maybe we can lift the lid and discover the truth behind the “phantom menace” — everyone is united in that goal at least — but let’s also hope that Episode 2: Attack of the Clones has not yet started shooting!

Mike.