Timing Strategy for Active Detection of APT Attack Based on FlipIt Model and Q-learning Method

At present, APT attack detection mainly focuses on the specific realization of detection technology, and there is little research on the timing strategy for active detection of APT attack. This paper focuses on the problem of APT dynamic detection with timing decision intelligence. We use the FlipIt game model and reinforcement learning method of Q-learning algorithm, consider the APT attack events as an exponentially distributed random process, and the defender using the attacker’s last move time (LM-defender) to learn his optimal timing strategy using Q-Learning algorithm. The defender's reinforcement learning mechanism for the timing of the APT attack action and outputs its timing selection strategy. Simulation experiments show that compared with the existing greedy algorithm, the timing strategy based on the Q-Learning algorithm does not require the attacker's prior information under the premise, it can achieve a higher average benefit than the greedy strategy. This result provides a theoretical reference for the design of timing strategy to defend APT attack intelligently and efficient.