From black box to glass box: algorithmic explainability as a strategic decision

Abstract

The best-performing algorithms are often the least explainable. In parallel, there is growing concern and evidence that algorithms may autonomously engage in misconduct. Inspired by recent regulatory proposals, we propose a simple model of firm compliance and explainability decisions under the threat of (costly and imperfect) regulatory audits. When audit efficacy is independent of explainability, audits and transparency always encourage investment in explainability, with transparency signaling compliance. However, if explainability strongly improves audit efficacy, firms may hide misconduct behind opaque algorithms, a phenomenon exacerbated by opportunistic auditing policies. In these cases, audits may stimulate the proliferation of black box algorithms.