A Baseline Approach for Goalkeeper Strategy using Sarsa with Tile Coding on the Half Field Offense Environment

Abstract

Much research in RoboCup 2D Soccer Simulation has used the Half Field Offense (HFO) environment. This work proposes a baseline approach for goalkeeper strategy using Reinforcement Learning on HFO. The proposed approach uses Sarsa with eligibility traces and Tile Coding for the discretization of state variables. Two comparative studies were conducted to validate the proposed baseline. First, a comparative study between the Agent2D's goalkeeper strategy and a random decision strategy was performed. The second comparative study verified the performance of the proposed approach against a random decision strategy. Wilcoxon's Signed-Rank test was used for measuring the statistical significance of performance differences. Experiments showed that the Agent2D's goalkeeper strategy is inferior to a random decision, and the proposed baseline delivers a performance superior to a random decision strategy with a confidence level of 95%.