John Poindexter, convicted felon and retired Admiral, tried to create a huge data mining network of government and corporate databases for the Bush administration, but was forced to resign by an avalanche of bad publicity.

Now he seems to have sold this plan to Singapore, with a few changes.

Peterson, (Poindexter’s associate in this venture) provides the money quote, feeding the dream with words that every tech believer longs to hear. “Essentially, [RAHS is] a strategic tool that ties together every one of the agencies in a government into a large network that is constantly scanning the horizon looking for weak signals that point toward the possibility of a significant event that would have important implications for Singapore,” Wired quotes him as saying.

There are other, more meaningful ways to talk about what Peterson means by his exceptionally vague phrase, “weak signals”. Vague phrases are useful whenever a more accurate, more precise one would exude an unfortunate air of truth, and here one is deployed with care. “False positives” would give us some precision here in place of “weak signals”, (emphasis mine) and it invokes one of the defining features of the process of data mining, which is a moderately useful marketing tool now promoted to the status of a national security crystal ball.

Only, it’s never going to work. For example, in the past five years we’ve seen our airports become hubs of data mining and analysis. Not surprisingly, we’ve seen many thousands of innocent passengers detained, questioned, bullied, inconvenienced, and embarrassed, while not one terrorist has ever been caught. The rate of false positives appears to be one hundred per cent. (emphasis mine)

And as for false negatives, surely, in the past five years, at least a few terrorists have flown commercially, and perhaps quite a few. They’re not being caught because, unlike the dumb technological tools deployed against them, human adversaries learn. When one thinks of data mining as a threat, one takes steps to avoid detection. Innocent people don’t take steps to protect themselves so they get “caught” every day. Meanwhile, the terrorists run rings around the national security agencies and their magic machinery.

It is beginning to look as though the only thing that changed with Poindexter’s resignation was that we lost the few fig leaves of privacy protection it retained. The NSA has continued and expanded a program of domestic spying.


Bush has built a secret system, without enabling legislation, justified by executive fiat and presidential findings alone, deliberately operating beyond the oversight of Congress and the courts, and existing outside the law.

Many times I have heard or seen discussions of individuals or pundits or reporters saying that they, or most people, are willing to give up some privacy in order to make us more secure. Now it looks like we get nothing for being detained, questioned, bullied, inconvenienced, and embarrassed by data mining and security measures. All we get for all the intrusive domestic spying, data mining, and loss of privacy, is to add ourselves to the huge statistical bank of false positives.

Data mining is good for feeding targeted advertisements to likely punters. It can improve returns on an advertising investment by increasing the likelihood that a consumer will actually find a particular product or service interesting, although it is still an incredibly blunt instrument. Still, if it increases the response rate to an advertisement from, say, two per 1,000 to six per 1,000, it’s a real money saver.

But is it a real life saver? We have seen data mining in action in airports, and it appears that every single “detection” has been a false positive. Meanwhile, an unknown number of undesirables continue to move about via commercial air travel. We can’t know how many times this has happened, but it doesn’t matter. Even if it’s happened only once, the rate of false negatives, too, is 100 per cent.

Terrorists and criminals are caught when they make mistakes. They confide in the wrong person and are ratted, or their communications are intercepted, or they arouse suspicion in the real world because of their behaviour.
. . .

Governments across the globe are already engaged in data mining and analysis to a degree unimaginable a decade ago. But much of it is confined to single agencies. The next logical step is to unite the databases, according to Poindexter’s ambition. It’s not going to work, and it can well be criticised on grounds of wasting money and resources — but from a privacy point of view, really, who cares at this point? If the FBI is already reading my email and listening to my phone calls without a warrant – if the TSA is already scouring my credit history every time I book a flight – why should I care if the DOD can as well?

We might as well invite everyone to the privacy-invasion party.