Making It Possible for the Auditing of AI: A Systematic Review of AI Audits and AI Auditability

Artificial intelligence (AI) technologies have become the key driver of innovation in society. However, numerous vulnerabilities of AI systems can lead to negative consequences for society, such as biases encoded in the training data and algorithms and lack of transparency. This calls for AI systems to be audited to ensure that the impact on society is understood and mitigated. To enable AI audits, auditability measures need to be implemented. This study provides a systematic review of academic work and regulatory work on AI audits and AI auditability. Results reveal the current understanding of the AI audit scope, audit challenges, and auditability measures. We identify and categorize AI auditability measures for each audit area and specific process to be audited and the party responsible for each process to be audited. Our findings will guide existing efforts to make AI systems auditable across the lifecycle of AI systems.