Algorithmic Disability Discrimination

Abstract

Prior to the Digital Age, disability-related information flowed between people with disabilities and their doctors, family members, and friends. However, in the 21st century, artificial intelligence tools allow corporations that collect and analyze consumer data to bypass privacy and antidiscrimination laws, such as HIPAA and the ADA, and infer consumers’ disabilities without their knowledge or consent. When people make purchases, browse the Internet, or post on social media, they leave behind trails of digital traces that reflect where they have been and what they have done. Companies aggregate and analyze those traces using AI to reveal details about people’s physical and mental health. I describe this process as mining for “emergent medical data” (EMD) because digital traces have emergent properties; when analyzed by machine learning, they reveal information that is greater than the sum of their parts. EMD collected from disabled people can serve as a means of sorting them into categories that are assigned positive or negative weights before being used in automated decision making. Negatively weighted categories can stigmatize disabled people and contribute to the narrative that disabilities are bad. Moreover, by negatively weighting categories into which disabled people are sorted, algorithms may stigmatize disabled people and screen them out of life opportunities without considering their desires or qualifications. This chapter explains how AI disrupts the traditional flow of disability-related data to promote algorithmic disability discrimination. It presents and analyzes four legislative solutions to the problem: Amend Title III of the ADA to include internet business within the law’s definition of places of public accommodation, expand the scope of HIPAA’s covered entities to include companies that mine for EMD, impose fiduciary duties on internet platforms and other businesses that infer health data, and establish general data protection regulations in the US inspired by the EU’s General Data Protection Regulation (GDPR) and the California Consumer Protection Act of 2018 (CCPA). Regardless of the regulatory path chosen, we must evolve our understanding of health information and disability-related data. Whether it is exchanged between patients and doctors or pieced together by AI from the digital traces scattered throughout the Internet, the data of people with disabilities deserves protection. Health data has the potential to harm people if used to exploit rather than to heal, and companies can increasingly mine EMD and use it to reduce the autonomy of people with disabilities. Members of this group should be able to control when and how their data is used to draw conclusions about them and make decisions for them. Otherwise, AI-based inferences will contribute to the obstacles that people with disabilities must overcome in their daily lives.