A recent Yale study in the journal Clinical Imaging raises concerns over racial bias when Open AI’s GPT Chat 3.5 and GPT 4.0 are provided with patients’ race.
Some racial groups are at higher risk than others for certain diseases, and that’s information that could be helpful to artificial intelligence in health care. But sharing racial information with AI can have other effects besides delivering medical information, according to Dr. Melissa Davis, co-author and vice chair of Medical Informatics at Yale, and associate professor at the Department of Radiology and Biomedical Imaging.
Researchers asked Open AI's ChatGPT to simplify more than 700 Yale radiology reports. So if a radiology report said a patient has “Kerley B lines,” ChatGPT would essentially translate that to something most people could understand — there’s extra fluid in the lungs that shouldn't be there.
But when ChatGPT was also told the race of the person asking to simplify the medical language, researchers found ChatGPT simplified the response to lower reading levels depending on what race was shared.
“We found that white and Asian patients typically had a higher reading grade level,” Davis said. “If I said 'I am a Black patient,' or 'I am an American Indian patient' or 'Alaskan Native native patient,' the reading grade levels would actually drop.”
She said even though ChatGPT did an excellent job of simplifying the reports, the findings flag that inputting patients’ race as a socio-economic determinant of health should not be disclosed to large language models. Instead, the way patients absorb medical information could be based on something else, like education level or age.
Utopian, dystopian, or a mix of both, AI is expanding in health care across Connecticut and nationally.
In February, Hartford HealthCare launched the Center for AI Innovation in Healthcare, a collaboration with the Massachusetts Institute of Technology and Oxford University.
“AI stands poised to profoundly reshape health care delivery, impacting access, affordability, equity and excellence,” said Dr. Barry Stein, the group’s chief clinical innovation officer.
Researchers at UConn Health are using AI to help diagnose lung cancer in the early stages.
“It’s all about stage shift,” said Dr. Omar Ibrahim, associate professor of medicine and director of interventional pulmonology at UConn Health. “Using the AI technology of Virtual Nodule Clinic, I can find the patients who need care, before they even realize it themselves, and get them treated at the earliest possible stage.”
