Rep. Rashida Tlaib (D-Mich.) touched off controversy by suggesting the Detroit Police Department should let only African Americans analyze facial recognition results. (Alex Edelman/Bloomberg)
When Rep. Rashida Tlaib (D-Mich.) was invited to tour the Detroit Police Department’s Real Time Crime Center, the purpose was to explain how officers use facial recognition when policing the streets of a city that is more than 80 percent black.
But the meeting quickly deteriorated when Tlaib told Chief James Craig that “analysts need to be African Americans, not people that are not,” because “non-African Americans think African Americans all look the same.”
Craig, who is African American, said the suggestion that white analysts would be less adept at their jobs than people of color was “insulting.”
Tlaib’s comments, however, were consistent with an enduring debate that rages around facial recognition software: the systems more accurately identify lighter-skinned faces than they do people of color. Researchers and numerous studies argue that’s because the software is trained on vast sets of images that skew heavily toward white men, leaving women and minorities vulnerable to holes in mammoth databases.
That can be especially risky, critics argue, as facial recognition is embraced by government and law enforcement.
Critics also worry that people aren’t being trained adequately in how to use the technology and interpret its results. Researchers say that law enforcement agencies don’t always disclose how its analysts are taught to use the systems, or who is conducting the training. And they worry that even if a department claims a strong training protocol, people will inevitably let biases about gender and race creep into how they assess a match.
“There’s a huge amount of reliance that this is going to be accurate if it spits out a match, or a candidate list of five people,” said Jake Laperruque, senior counsel at The Constitution Project at the Project on Government Oversight. “And that’s just not the case.”
Camera quality, lighting and the size of a system’s database can all affect facial recognition’s accuracy. But researchers argue that improving those factors doesn’t erase a system’s hard-wired biases. One 2018 study conducted by Joy Buolamwini of the M.I.T. Media Lab found that the technology is correct 99 percent of the time with photos of white men. But the software misidentified the gender as often as 35 percent of the time when viewing an image of a darker-skinned woman.
In January, researchers with M.I.T. Media Lab reported that facial-recognition software developed by Amazon and marketed to local and federal law enforcement also fell short on basic accuracy tests, including correctly identifying a person’s gender. Specifically, Amazon’s Rekognition system was perfect in predicting the gender of lighter-skinned men, the researchers said, but misidentified the gender of darker-skinned women in roughly 30 percent of their tests.
Amazon disputed those findings, saying the research used algorithms that work differently from the facial-recognition systems used by police departments. (Amazon founder and chief executive Jeff Bezos owns The Washington Post.)
But the results, researchers argue, offer a cautionary tale for millions of Americans. A 2016 report by Georgetown Law researchers found that the facial images of half of all American adults, or more than 117 million people, were accessible in a law-enforcement facial-recognition database.
Greater scrutiny on these databases has spurred some progress. ImageNet, an online image database, recently said it would remove 600,000 pictures of people from its system after an art project showed the severity of the bias wired into its artificial intelligence. Artist Trevor Paglen and AI researcher Kate Crawford showed how the system could generate derogatory results when people uploaded photos of themselves. A woman might be called a “slut,” for example, and an African American user could be labeled a “wrongdoer” or with a racial epithet.
Unlike many social and policy debates gripping Washington, facial-recognition has drawn sharp criticism from Republican and Democratic lawmakers alike. In May, members of the House Oversight and Reform Committee jointly condemned the technology, charging that it was inaccurate and threatened Americans’ privacy and freedom of expression. But there are no current federal rules governing artificial intelligence or facial recognition software.
“We have a technology that was created and designed by one demographic, that is only mostly effective on that one demographic, and they’re trying to sell it and impose it on the entirety of the country,” Rep. Alexandria Ocasio-Cortez (D-N.Y.) said earlier this year.
Detroit’s police board approved the use of facial recognition software last month. But the technology has not been embraced by all locales. San Francisco and Oakland, Calif., along with Somerville, Mass., have banned local government agencies, including police departments, from using the software. In September, California lawmakers temporarily banned state and local law enforcement from using facial-recognition software in body cameras.
Beyond the software itself, critics worry that users will put too much faith in facial recognition, even as they acknowledge the software’s pitfalls. Laperruque pointed to the “CSI Effect” — when people come to believe in the technology’s infallibility because of how they see it used in a crime shows on TV.
“Training in general is seen as a pretty essential feature of getting this to work,” Laperruque said. “There’s an expectation when law enforcement and the public see a new sci-fi-looking tool to say, ‘This is a magical, futuristic technology,’ when in reality, facial recognition is a lot more akin to outsourcing policing to glitchy computers.”
Jennifer Lynch, surveillance litigation director of the Electronic Frontier Foundation, pointed to studies showing how poorly people identify images of people they don’t know — especially when it comes to people of different races or ethnicity.
Researchers argue that among police departments that use the software, there aren’t always clear or transparent standards for how officials are trained on the systems, or how much weight is given to the results.
“The police departments say, ‘we are not considering this an exact match because we have humans that look at this after the fact and verify the technology,’” Lynch said, “which is problematic because humans are not good at identifying people.”
The back and forth between Tlaib and Craig was tense, The Detroit News reported. Tlaib described seeing people on the House floor misidentify longtime Democratic congressmen John Lewis and Elijah Cummings, both of whom are black.
But Craig said that the department had “a diverse group of crime analysts” and that Tlaib’s criticism was “a slap in the face to all the men and women in the crime center.”
Speaking to a local news channel, Tlaib said she stood by her comments “that facial recognition technology is broken.” Tlaib said that as an elected official, her job was to make sure residents “are not going to be misidentified and detained or falsely arrested because [Craig] is using broken technology.”
Tlaib’s office did not respond to multiple requests for comment by The Post. Tlaib is a sponsor of a House bill that would ban facial and biometric recognition in public housing, plus another that would bar federal funds from being used to buy or use the technology.
The Detroit Police Department did not respond to multiple requests for comment, including about how the department trains its analysts who use facial recognition technology.
But Craig said his trust was in the analysts “who are trained, regardless of race, regardless of gender. It’s about the training.” In his meeting with Tlaib, Craig emphasized that “a match is a tool only.”
“As a police chief who happens to be African American in this city, if I made a similar statement, people would be calling for my resignation right now,” Craig told a local news station. “So is that a double standard? That’s the number one question I have.”