Artificial intelligence auditability and auditor readiness for auditing artificial intelligence systems

Abstract

As the business community races to implement artificial intelligence (AI), there are several challenges that need to be addressed such as fairness and biases, transparency, denial of individual rights, and dilution of privacy. AI audits are expected to ensure that AI systems function lawfully, robustly, and follow ethical standards (e.g., fairness). While the auditability for financial audits and information system audits has been well addressed in the literature, auditability of AI systems has not been sufficiently addressed. AI auditability and auditors’ competencies are crucial for ensuring AI audits are conducted with high quality. Research on the auditability of AI and the competencies of AI auditors is gravely lacking leaving risks in AI systems unmitigated. The primary reason is that the field is nascent and the rapid growth has left the audit profession struggling to catch up. Foundational work on establishing parameters for such research would help advance this research. In this paper, we explore AI auditability measures and competencies required for conducting AI audits. We conducted semi‐structured interviews with 23 experienced AI professionals who have direct involvement or indirect exposure to AI audits. Based on our findings, we propose a framework of AI auditability and identify the competencies required to conduct AI audits. Our study serves as the first formal attempt to systematically identify and classify auditability measures and auditors’ expertise demanded for AI audits based on practitioners’ perspectives. Our findings contribute to the AI audit literature, inform AI developers about implementing auditability, guide the training of new AI auditors, and establish a foundation for further research in the field.