A Methodology of Face Validation with Domain Experts for Agent-Based Crime Risk Prediction

Abstract

In modeling and simulation, face validation with domain experts is a recommended validation technique applied after implementing the conceptual model. Experts can offer early feedback on the prediction results. Experts’ empirical experiences can identify mistakes in the conceptual model and help estimate initial values of key parameters to save time for later calibrations. This work proposes a three-phase method to conduct face validation with police investigators and crime analysts to validate an agent-based crime risk prediction model. The three phases focus on verifying the prediction results, the behavioral design of target individuals, and the risk estimation process respectively. We present each phase’s goals, details, and types of validity addressed to provide clear guidance on adopting the method. Quantitative results of the feedback have shown that our face validation improves the correctness of the conceptual model and boosts intended users’ confidence in the model results. Our work also provides additional empirical experiences in urban crime prediction. The experimental results show the potential of the proposed method for validating other software systems with domain experts’ knowledge.