Opponent modeling with incremental active learning: A case study of Iterative Prisoner's Dilemma

What's the most important sources of information to guess the internal strategy of your opponents? The best way is to play games against them and infer their strategy from the experience. For novice players, they should play lot of games to identify other's strategy successfully. However, experienced players usually play small number of games to model other's strategy. The secret is that they intelligently design their plays to maximize the chance of discovering the most uncertain parts. Similarly, in this paper, we propose to use an incremental active learning for modeling opponents. It refines the other's models incrementally by cycling “estimation (inference)“ and “exploration (playing games)” steps. Experimental results with Iterative Prisoner's Dilemma games show that the proposed method can reveal other's strategy successfully.