ASRA-Q: AI Security Risk Assessment by Selective Questions

Abstract

In this paper, we propose a new framework for security risk assessment. To conduct security analysis efficiently, it is necessary for developers to assess the security risks of machine learning based system (MLS) by themselves, but existing technologies cannot be used to such a purpose. Using the proposed framework, MLS developers can assess the security risks of MLSs by themselves. Our framework consists of two phases. In the preparation phase, a machine learning security expert extracts conditions of adversarial attacks for each adversarial attack method and makes an attack tree for each attack method using the extracted conditions. In addition, they prepare yes/no questions corresponding to extracted conditions. In the assessment phase, MLS developers just answer yes/no questions, and the assessment results are shown. We asked some developers to evaluate our proposal by implementing the proposed framework. As a result, they found some vulnerabilities in MLSs they chose to analyze. We received positive comments from them as results of the questionnaire.