Online Judge Support for Programming Teaching

Online programming judges are considered useful and sometimes indispensable tools to support competitive programming, professionals' recruiting, and programming education. In this last field, the scientific literature on these tools focuses on the learners' needs, but neglects the requirements of the professors, even though they are who mainly decide whether or not an educational tool is adopted in the courses they teach. This article collected 132 functional requirements for educative online judges, from the scientific literature and programming teachers with experience in the use of this type of tool. To know the degree of support, the requirements were grouped into 27 categories, and a requirements verification was performed with four available educative online judges reported in a recent systematic literature review. A low degree of satisfaction of requirements was found. This result encourages future research to create tools that better support teaching-learning processes and the requirements collected are a useful contribution as a starting point for such research.