Amazon’s facial recognition tool mistakenly matched the faces of 28 members of Congress with criminals in mugshots and disproportionately misidentified people of color, a new test by the American Civil Liberties Union revealed.
For the test, the civil rights organization compared photos of congressional lawmakers with a database of roughly 250,000 mug shots, Huffington Post reported. Of the 28 misidentified legislators, 39 percent were non-white, including Reps. John Lewis (D-Ga.), Lacy Clay (D-Mo.) and Luis Gutiérrez (D-Ill.).
Of the 28 wrongly identified lawmakers, 39 percent were people of color. (Photo by CNN Money)
The AI blunder has raised serious concerns about racial bias and intensified calls for the tech giant to pull out of the facial recognition industry. Amazon has already pitched its facial “Rekognition” service to local police departments, however.
“It’s not hard to imagine a police officer getting a ‘match’ indicating that a person has a previous concealed-weapon arrest, biasing the officer before an encounter even begins,” the ACLU wrote in its report. “People of color are already disproportionately harmed by police practices, and it’s easy to see how [Amazon] Rekognition could exacerbate that.”
Multiple lawmakers, including those who were mistakenly ID’d, have penned letters to the company and requested a meeting with CEO Jeff Bezos to address their concerns, tech site Ars Technica reported. Rep. Mark DeSaulnier of California (D) wrote a letter to Amazon on Thursday, giving the company until Aug. 20 to answer specific questions and requests about the service.
Despite the pushback, Amazon has defended its facial recognition tool, arguing its used to help prevent human trafficking, avoid package theft and locate missing children. Per Huff Post, the company also said the ALCU’s test results could’ve been improved if they upped the tool’s “confidence threshold,” or the likelihood that a given prediction is correct.
“When using facial recognition for law enforcement activities, we guide customers to set a higher threshold of at least 95 percent or higher,” Amazon explained.
The technology’s default threshold is 80 percent, however, and the ACLU argued increasing the threshold wouldn’t have made much of a difference.
“In addition to remaining silent on these very real concerns that members of Congress, community groups, and Amazon’s own employees … have raised repeatedly, Amazon is acknowledging that Rekognition – a product it aggressively markets to law enforcement – can and will misidentify people by default,” Jacob Snow, an attorney with the ACLU of Northern California, said in a statement. “That’s downright dangerous.”
In May, the Congressional Black Caucus wrote to Bezos, urging the company against selling Rekognition to law enforcement agencies. The caucus argued that communities of color are heavily, and aggressively policed and warned that implementing the tool could exacerbate already problematic policing tactics.
“We’re worried deployment of technology like the one you have developed has a high propensity for misuse,” the letter read. “Surveillance of perfectly legitimate and constitutionally protected activity will only further erode the public’s trust in law enforcement.”
Amazon has yet to respond to lawmakers’ concerns.
