Australian police report 9 errors across 1.4 million faces

Five months of camera vans

1,380,882 faces scanned, 9 false alerts. That is the tally Western Australia Police published when its real-time facial recognition trial ended on Saturday 31 October. For five months, police vans fitted with cameras went out 100 times to 43 sites, from Perth to Geraldton, including the big agricultural show in Perth. Every face was compared against a watchlist of wanted people.

Nine errors in 1,380,882 works out at 0.00065%. The police put that on their official page, updated on 2 November, and draw a conclusion from it: the rate, they say, reflects a high level of system accuracy (the page's wording is "reflecting a high level of system accuracy"). The progress update of 6 October had already made the same claim, in the same words, at 0.0009%.

The number is correct. But there is a second division to do, and we didn't find it in any of the coverage we read.

What the coverage did with it

The Guardian ran the tally on 2 November. Its article attributes the rate to a police spokesperson and names the base plainly: nine false alerts set "against the total 1.38m scans". It also gives critics their say. Bill Rowlings of Civil Liberties Australia accuses the police of "severely gilding the lily". Sarah Schwartz of the Human Rights Law Centre points out that every false match is "a real person who may be stopped, questioned or investigated because an algorithm got it wrong".

IDTechWire, a trade site covering digital identity, does much the same: it presents the rate as the one the police announce. At the progress update, the same site had gone a step further and spelled out that the rate was calculated across all faces scanned.

The groundwork is solid. Yet in every piece we read, the same number is missing. The criticism that circulates, from Civil Liberties Australia, proposes a different test: wait and see how many of the 122 people arrested end up convicted. It's a fair question, but it mixes up two things, the machine that recognises a face and the court that judges a case.

Nine out of 292

The missing number sits in the same sentence of the official tally: the police write that the system generated 292 alerts, 9 of them incorrect. None of the coverage we read cites it.

Nine out of 292 is 3.1%. About one alert in 32.

Both figures are true. They answer different questions.

Picture a sniffer dog at an airport. Thousands of travellers walk past it every day, and now and then it sits down in front of a suitcase. Count its mistakes against every traveller who passed, and it looks almost infallible. The customs officer only cares about the moments when the dog sits: that's when someone has to decide whether to open the bag.

The 0.00065% is the traveller's number. The 3.1% belongs to the officer who gets the alert on a tablet, and who is about to walk up to someone in the street.

A choice built into the spreadsheet

That denominator isn't an accident of press-release wording. The police also publish a district-by-district table, dated 12 October, covering the trial up to 11 October. Its column headed "Incorrect Rate %" is calculated on faces seen. The choice is baked into the tracking tool itself.

The table has a telling example. In Cannington, in suburban Perth, over what looks like the four weeks from mid-August to mid-September (the period is our inference from the table), the system raised five alerts, one of them false. Displayed rate: 0.0088%. One alert in five was wrong, and the spreadsheet shows a rate that rounds to zero.

What the 0.00065% does tell you

The official figure shouldn't be thrown out. For a passer-by who walks past a van, the risk of being wrongly flagged is very low: about one chance in 150,000 per face scanned. That's useful, and it is the right answer to the question a Perth resident asks on seeing the van.

According to the Guardian, Commissioner Col Blanch told a press conference: "No one was arrested. No one made a complaint." The wrongly flagged people were approached by officers and questioned. The official page puts it its own way: "An alert is not, by itself, grounds to arrest or exercise powers."

What doesn't follow is the conclusion. A rate calculated on passers-by says little about the system's accuracy, because almost none of them are on the watchlist. Mostly it tells you the machine stays quiet most of the time.

The alerts nobody checked

The 12 October table has one more column that the final tally leaves out: alerts marked "Unconfirmed (person not engaged)", where the officer never approached the person. As of 11 October there were 34 of them against 252 alerts, or 13.5%. Those alerts are neither right nor wrong: nobody went to look.

So the 3.1% counts as correct every alert not declared false, including the ones never checked. It's a bit like marking every unmarked exam paper as a pass. What you get is a floor, not a measurement. And the final tally doesn't say how many unconfirmed alerts remain among the 292.

One last question goes unanswered, for lack of data: how many wanted people walked past the cameras without being recognised? The system immediately deletes faces that match nothing, which is the privacy guarantee it offers. It therefore can't count what it missed.

An evaluation begins

The vans are off the road while an internal evaluation looks, according to the police, at the system's performance, its effectiveness, its effect on the community and compliance with the rules. In the first week, Col Blanch told the press, as reported by the news agency AAP, that his counterparts in other states were watching the trial closely. In France, the debate is taking a different route for now, through police databases, as with Gérald Darmanin's plan for the TAJ criminal records file.

That evaluation may eventually say which of the two figures matters. Until then, the choice has already been made, in a spreadsheet column: errors are divided by the people being reassured, not by the people stopped in the street.