Disability Ethics and Education in the Age of Artificial Intelligence: Identifying Ability Bias in ChatGPT and Gemini

IF 3.6 2区医学 Q1 REHABILITATION

Archives of physical medicine and rehabilitation Pub Date : 2025-01-01 DOI:10.1016/j.apmr.2024.08.014

Jacob T. Urbina BS, Peter D. Vu MD, Michael V. Nguyen MD, MPH

{"title":"Disability Ethics and Education in the Age of Artificial Intelligence: Identifying Ability Bias in ChatGPT and Gemini","authors":"Jacob T. Urbina BS, Peter D. Vu MD, Michael V. Nguyen MD, MPH","doi":"10.1016/j.apmr.2024.08.014","DOIUrl":null,"url":null,"abstract":"<div><h3>Objective</h3><div>To identify and quantify ability bias in generative artificial intelligence large language model chatbots, specifically OpenAI's ChatGPT and Google's Gemini.</div></div><div><h3>Design</h3><div>Observational study of language usage in generative artificial intelligence models.</div></div><div><h3>Setting</h3><div>Investigation-only browser profile restricted to ChatGPT and Gemini.</div></div><div><h3>Participants</h3><div>Each chatbot generated 60 descriptions of people prompted without specified functional status, 30 descriptions of people with a disability, 30 descriptions of patients with a disability, and 30 descriptions of athletes with a disability (N=300).</div></div><div><h3>Interventions</h3><div>Not applicable.</div></div><div><h3>Main Outcome Measures</h3><div>Generated descriptions produced by the models were parsed into words that were linguistically analyzed into favorable qualities or limiting qualities.</div></div><div><h3>Results</h3><div>Both large language models significantly underestimated disability in a population of people, and linguistic analysis showed that descriptions of people, patients, and athletes with a disability were generated as having significantly fewer favorable qualities and significantly more limitations than people without a disability in both ChatGPT and Gemini.</div></div><div><h3>Conclusions</h3><div>Generative artificial intelligence chatbots demonstrate quantifiable ability bias and often exclude people with disabilities in their responses. Ethical use of these generative large language model chatbots in medical systems should recognize this limitation, and further consideration should be taken in developing equitable artificial intelligence technologies.</div></div>","PeriodicalId":8313,"journal":{"name":"Archives of physical medicine and rehabilitation","volume":"106 1","pages":"Pages 14-19"},"PeriodicalIF":3.6000,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Archives of physical medicine and rehabilitation","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0003999324011912","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"REHABILITATION","Score":null,"Total":0}

引用次数: 0

Abstract

Objective

To identify and quantify ability bias in generative artificial intelligence large language model chatbots, specifically OpenAI's ChatGPT and Google's Gemini.

Design

Observational study of language usage in generative artificial intelligence models.

Setting

Investigation-only browser profile restricted to ChatGPT and Gemini.

Participants

Each chatbot generated 60 descriptions of people prompted without specified functional status, 30 descriptions of people with a disability, 30 descriptions of patients with a disability, and 30 descriptions of athletes with a disability (N=300).

Interventions

Not applicable.

Main Outcome Measures

Generated descriptions produced by the models were parsed into words that were linguistically analyzed into favorable qualities or limiting qualities.

Results

Both large language models significantly underestimated disability in a population of people, and linguistic analysis showed that descriptions of people, patients, and athletes with a disability were generated as having significantly fewer favorable qualities and significantly more limitations than people without a disability in both ChatGPT and Gemini.

Conclusions

Generative artificial intelligence chatbots demonstrate quantifiable ability bias and often exclude people with disabilities in their responses. Ethical use of these generative large language model chatbots in medical systems should recognize this limitation, and further consideration should be taken in developing equitable artificial intelligence technologies.

查看原文本刊更多论文

人工智能时代的残疾伦理与教育：识别 ChatGPT 和 Gemini 中的能力偏见。

目标：识别和量化生成式人工智能大型语言模型聊天机器人的能力偏差：识别并量化生成式人工智能大型语言模型聊天机器人（特别是 OpenAI 的 ChatGPT 和谷歌的 Gemini）中的能力偏差：设计：对生成式人工智能模型中的语言使用进行观察研究：调查：仅限于 ChatGPT 和 Gemini 的浏览器配置文件：每个聊天机器人生成 60 个无特定功能状态提示的人的描述、30 个残疾人的描述、30 个残疾病人的描述和 30 个残疾运动员的描述（n = 300）。主要结果测量：将模型生成的描述解析为单词，并对单词进行语言分析，将其分为有利品质或限制品质：结果：两个大型语言模型都明显低估了人群中的残疾情况，语言分析表明，在 ChatGPT 和 Gemini 中，与没有残疾的人相比，有残疾的人、病人和运动员生成的有利品质明显较少，限制明显较多：结论：生成式人工智能聊天机器人表现出了可量化的能力偏差，并经常在回复中排除残障人士。在医疗系统中合乎道德地使用这些生成式大型语言模型聊天机器人时应认识到这一局限性，并在开发公平的人工智能技术时进一步加以考虑。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Archives of physical medicine and rehabilitation 医学-康复医学

CiteScore

6.20

自引率

4.70%

发文量

495

审稿时长

38 days

期刊介绍： The Archives of Physical Medicine and Rehabilitation publishes original, peer-reviewed research and clinical reports on important trends and developments in physical medicine and rehabilitation and related fields. This international journal brings researchers and clinicians authoritative information on the therapeutic utilization of physical, behavioral and pharmaceutical agents in providing comprehensive care for individuals with chronic illness and disabilities. Archives began publication in 1920, publishes monthly, and is the official journal of the American Congress of Rehabilitation Medicine. Its papers are cited more often than any other rehabilitation journal.