Optimizing the decision to expel attackers from an information system

Abstract

The conventional reaction after detecting an attacker in an information system is to expel the attacker immediately. However the attacker is likely to attempt to reenter the system, and if the attacker succeeds in reentering, it might take some time for the defender's intrusion detection system (IDS) to re-detect the attacker's presence. In this interaction, both the attacker and defender are learning about each other — their vulnerabilities, intentions, and methods. Moreover, during periods when the attacker has reentered the system undetected, he is likely learning faster than the defender. The more the attacker learns, the greater the chance that he succeeds in his objective — whether it be stealing information, inserting malware, or some other objective. Conversely, the greater the defender's knowledge, the more likely that the defender can prevent the attacker from succeeding. In this setting, we study the defender's optimal strategy for expelling or not expelling an intruder. We find that the policy of always expelling the attacker can be far from optimal. Furthermore, by formulating the problem as a Markov decision process (MDP), we find how the optimal decision depends on the state variables and model parameters that characterize the IDS's detection rate and the attacker's persistence.