Researchers point out that Amazon and Apple's voice recognition algorithm has a problem that 'I can not hear black voices well'

Speech recognition algorithms have been adopted in various devices and applications such as smart speakers and smartphones, and are now part of everyday life. However, in experiments using voice recognition algorithms of Apple, Amazon, Google, IBM, Microsoft, etc., it was found that there is a problem that 'a voice recognition algorithm can not recognize black voices better than white voices'.

Racial disparities in automated speech recognition | PNAS

Voice recognition algorithms are used in various applications such as smart assistant operations, voice input, and transcription services. Machine learning algorithms are used in speech recognition systems, and machine learning algorithms are trained with voice data and text data prepared by developers.

In order to investigate the accuracy of such a speech recognition algorithm, the research team of Stanford University conducted an experiment to convert the speech spoken by various people into characters for the speech recognition algorithm of Apple, Amazon, Google, IBM, Microsoft . The voices used in the experiment totaled 19.8 hours and consisted of 2141 voices spoken by 42 whites and 73 blacks. Also, 44% of the speakers were men, and the average age was 45 years old.

Experiments have shown that on average, speech recognition algorithms misidentify 19% of the words spoken by whites, but 35% of the words spoken by blacks are misidentified. The error rate was 41% for black men and 30% for black women.

The graph below compares the error rates of white speakers and black speakers by the voice recognition algorithms of each company. You can see that the error rate for black spoken words exceeds the error rate for white spoken words for all speech recognition algorithms. Apple's voice recognition algorithm has the highest error rate, with a black speaker error rate of 45% and a white speaker error rate of 23%. Even with the best performing Microsoft speech recognition algorithms, black speakers had an error rate of 27% and white speakers had an error rate of 15%.

'The results are not limited to a specific company. We found a pattern similar to all five companies,' said Sharad Goel , an associate professor of engineering at Stanford University.

In the past, cases where algorithms and software have racial bias have been reported, cases where Google Photos recognized black people as `` gorillas '' and tagged them, and medical data that does not have data on race There are also known cases where blacks are evaluated unequal in the system.

Why did racial discrimination occur in healthcare systems that lack race data? -GIGAZINE

It is believed that the set of problems is likely due to the bias present in the dataset when training the machine learning algorithm. If the data used for training itself contains a large amount of white-speaker speech and not much of a black-speaker's speech, the speech recognition algorithm will not be able to successfully learn the black-speaker's accents and speaking styles, resulting in an It will be expensive. 'Our paper suggests that developers need to use more diverse data to train speech recognition algorithms,' said research team Allison Koenecke .

Justin Burr, a spokeswoman for Google, said that fairness is one of the basic principles of Google AI and has been working on improving the accuracy of speech recognition algorithms for many years. 'We've been working on the challenge of accurately recognizing different types of speech over the years, and will continue,' Burr said. An IBM spokesperson also commented, 'IBM continues to develop, improve, and advance our natural language and speech processing capabilities, increasing the level of user functionality through IBM Watson ,' Amazon said. We are presenting a web page explaining that we are continuously improving our recognition algorithms.

Apple and Microsoft declined to comment on the results of this survey.