Amazon’s facial recognition matched 28 members of Congress to criminal mugshots

Facial recognition, there's no escaping it- this technology is going to be increasingly used in a plethora of ways. But, how accurate is it? The ACLU decided to put Amazon's facial recognition system to the test. The results, in my opinion, are hilarious:

The Verge Source Article

The American Civil Liberties Union tested Amazon’s facial recognition system — and the results were not good. To test the system’s accuracy, the ACLU scanned the faces of all 535 members of congress against 25,000 public mugshots, using Amazon’s open Rekognition API. None of the members of Congress were in the mugshot lineup, but Amazon’s system generated 28 false matches, a finding that the ACLU says raises serious concerns about Rekognition’s use by police. “An identification — whether accurate or not — could cost people their freedom or even their lives,” the group said in an accompanying statement. “Congress must take these threats seriously, hit the brakes, and enact a moratorium on law enforcement use of face recognition.” Reached by The Verge, an Amazon spokesperson attributed the results to poor calibration. The ACLU’s tests were performed using Rekognition’s default confidence threshold of 80 percent — but Amazon says it recommends at least a 95 percent threshold for law enforcement applications where a false ID might have more significant consequences.

“While 80% confidence is an acceptable threshold for photos of hot dogs, chairs, animals, or other social media use cases,” the representative said, “it wouldn’t be appropriate for identifying individuals with a reasonable level of certainty.” Still, Rekognition does not enforce that recommendation during the setup process, and there’s nothing to prevent law enforcement agencies from using the default setting.

Members of congress took notice (I wonder how many of them are hiding out now?):

The test has already inspired significant reaction from three members of Congress. Shortly after the test was published, Sen. Markey (D-MA), Rep. Gutiérrez (D-IL) and Rep. DeSaulnier (D-CA) signed onto an open letter to Amazon CEO Jeff Bezos asking for a full list of law enforcement agencies using the technology and inquiring about safeguards for using it on children younger than thirteen. “Serious concerns have been raised about the dangers facial recognition can pose to privacy and civil rights,” the letter reads, “especially when it is used as a tool of government surveillance.”

a reply to: seattlerat


LOCK EM UP!

Good enough for us, good enough for THEM!

a reply to: seattlerat

Who needs facial recognition? I could point out 535 criminals in Congress with 100% accuracy.

originally posted by: sine.nomine
a reply to: seattlerat

Who needs facial recognition? I could point out 535 criminals in Congress with 100% accuracy.

ZING!

28?

Out of 535.

Amazon gots some more work to do.

a reply to: seattlerat

ACLU has it's panties in a bunch. I haven't read anywhere that a match equals criminal. It's a starting point or additional clues to an investigation. I don't believe any prosecutor would convict someone solely on a facial match in a computer.

a reply to: seattlerat

Any convictions based solely on Amazon's program? If not then who cares?

originally posted by: neo96
28?

Out of 535.

Amazon gots some more work to do.

The other 507 are far more discreet; they wear Guy Fawkes masks or shape-shift before committing their crimes.

They busted 28 amateurs...

Clearly Amazon's software is inefficient. 28 out of 535 crooks. Pitiful results really.

originally posted by: sine.nomine
a reply to: seattlerat

Who needs facial recognition? I could point out 535 criminals in Congress with 100% accuracy.

You would probably be only be 99.6% accurate.

Bravo for the post!

a reply to: seattlerat

lol. Good.

originally posted by: OccamsRazor04
a reply to: seattlerat

Any convictions based solely on Amazon's program? If not then who cares?

Do we really need to wait for someone to be shot dead over a mis-identification? Amazon match, reaches for ID, another one bites the dust.

originally posted by: Bramble Iceshimmer
a reply to: seattlerat

ACLU has it's panties in a bunch. I haven't read anywhere that a match equals criminal. It's a starting point or additional clues to an investigation. I don't believe any prosecutor would convict someone solely on a facial match in a computer.

So looking for vague reasons to investigate random people does not seem a bit Orwellian to you?

originally posted by: Bramble Iceshimmer
a reply to: seattlerat

ACLU has it's panties in a bunch. I haven't read anywhere that a match equals criminal. It's a starting point or additional clues to an investigation. I don't believe any prosecutor would convict someone solely on a facial match in a computer.

Well, basically, you could get pulled over or stopped and searched just because you fit the description.

But this has to be more accurate than what they do now.

I've been stopped with a friend walking home from work one night and had to pull out our IDs so they could check for warrants, because they said there had been break ins in the area recently and we fit the description.

So, I can see how people would complain but it still has to be better than stopping every single group that "fits the description".

I guess it would depend on how it was used.

ETA- But this was a pretty small sample size (I thought), so, using this in a larger database would lead to a higher number of mismatches. But still, as long as the real criminal is included in the group, it would narrow down a list of possible suspects much faster than a human.

I'm still on the fence until more info is available.


EDIT2- Even the ACLU original source does not include any examples showing what the person looked like and who they were matched to. All they did was post the pictures of the Congress members who were mismatched. I want to know, if it was an "honest mistake" or did they look nothing much alike and were still matched?




But yeah... I do not trust the ACLU at all and would not be surprised to find something fishy with the data or the conclusion.