May 18, 2022 06:00:00

AI can distinguish race from X-rays with 90% probability, but how to distinguish is unknown

It is difficult for human doctors to distinguish the patient's race from X-rays that do not know the color of the skin, but in a new paper published in the medical journal

The Lancet Digital Health , 'AI is from chest X-ray images. You can distinguish races with 90% accuracy. ' Researchers have also investigated how AI discriminates races, but so far it is unclear how AI discriminates races.

AI recognition of patient race in medical imaging: a modeling study --The Lancet Digital Health
https://www.thelancet.com/journals/landig/article/PIIS2589-7500(22)00063-2/fulltext

MIT, Harvard scientists find AI can recognize race from X-rays — and nobody knows how --The Boston Globe
https://www.bostonglobe.com/2022/05/13/business/mit-harvard-scientists-find-ai-can-recognize-race-x-rays-nobody-knows-how/

AI Can Now Predict a Person's Race From Medical X-Rays | Psychology Today
https://www.psychologytoday.com/us/blog/the-future-brain/202205/ai-can-now-predict-persons-race-medical-x-rays

Medical diagnosis using AI has already produced promising results in various fields, but Associate Professor Leo Anthony Celi of Harvard University School of Medicine said, 'For some reason, AI programs that examine chest X-ray images are used in black patients. There is a high possibility that the signs of illness will be overlooked. '

'I asked myself why computers can't tell the difference between races,' Celi said. Therefore, an international research team was established by researchers from the United States, Canada, Taiwan, Australia, etc., and research began on whether AI could distinguish races from chest X-ray images.

First, the research team created a dataset of medical chest X-rays taken by X-ray or CT scan, labeled as 'white,' 'Asian,' or 'black,' based on the subject's self-report. I used it to train AI. The images in the dataset did not include clear indicators of race, such as skin color or hair texture.

Then, the AI trained with a large number of racially labeled images was then raced from another unlabeled chest x-ray image. Then, AI succeeded in identifying the race shown in the X-ray image from whites, Asians, and blacks with an accuracy of 90% or more. It seems that this result did not depend on the subject's physique, age, and gender, and even when AI was trained with X-ray images of various parts of the body such as hands and spine other than the chest, the chest X-ray image was also obtained. The research team says that it was possible to determine the race from.

The research team investigated why AI can distinguish race from chest X-ray images, but even if we exclude factors that are biased by race such as disease, bone density, and breast density of the subject, AI Was still able to identify the race.

Marzyeh Ghassemi, co-author of the paper and assistant professor of computer science at the Massachusetts Institute of Technology, said, 'When graduate students in the lab showed some of the results in this paper, I was sure. I thought, 'This must be a mistake.' To be honest, I thought the students' stories were crazy. ' Ghassemi thinks that 'information on the melanin pigment contained in the skin is embedded in the X-ray image, and AI may use it as a clue to identify the race,' but to confirm this. Needs further research.

In today's world where AI is increasingly used as a diagnostic criterion for doctors, the fact that 'AI can infer race from medical diagnostic images' makes an unintentionally racially biased diagnosis. It suggests that there is a possibility. For example, AI may show a biased treatment strategy for all black patients, even though they may not know if it is optimal for a particular patient. On the other hand, human doctors using AI are unaware that the diagnosis is based on racial data that is not loaded by AI.

Biological anthropology at Hampshire College, author of Racism, Not Race, which argues that genetic differences within races are greater than genetic differences between races. Professor Alan Goodman is skeptical of the conclusions of this paper and doubts whether other research teams can reproduce the same results. Goodman also points out that even if it can be reproduced, it is not a race but a 'difference in where the ancestors lived.'

According to Goodman, there is no evidence that there is a substantial 'racial difference' in the human genome, but we know that there are differences between people depending on 'where their ancestors lived.' thing. In other words, if AI can infer geographical information from X-ray images, such as 'This person's ancestor is Scandinavian, this person's ancestor is Central Africa, this person's ancestor is Japan', the result is race with high accuracy. It can be determined. 'People call this'race', but I call it'geographical variation',' Goodman commented.

We don't know exactly how AI can determine race, but Celi argues that he should be reluctant to use AI diagnostic tools that can give biased results. 'We need to pause. We can't rush to bring the algorithm to a hospital or clinic until we're sure we're not making racist or sexist decisions,' he said. ..