POSTER: A Teacher-Student with Human Feedback Model for Human-AI Collaboration in Cybersecurity

We have developed a novel ’Teacher-Student with human feedback’ model for Human-Artificial Intelligence (AI) collaborations in cybersecurity tasks. In our model, AI furnishes sufficient information about its decision-making process to enable human agents to provide feedback to improve the model. Our key innovations include: enhancing the interpretability of AI models by analyzing falsely detected samples using LIME and SHAP values; developing a novel posthoc explanation-based dynamic teacher-student model to address concept drift or concept shift; integrating human experts’ feedback on falsely detected samples to increase accuracy, precision, and recall values, without retraining the entire model; establishing a list of attack-based feature values for human experts to promote reproducibility. We show in experiments with real data and threat detection tasks that our model significantly improves the accuracy of existing AI algorithms for these tasks.