A domain-specific language for automatic generation of checkers

Abstract

One of the important modules of a black-box automatic program grader is a "checker". In programming competition environment, a checker is a program written for the purpose to check the output of the contestant's program for a task that has many solutions. Usually, a checker is written manually as needed. In this paper, the idea of the output checker in the programming competition environment is extended to input checker and source code checker as a part of the automatic grader in our programming learning environment. Input checker validates the input coverage. The source code checker is used to validate a set of properties from a source code against the given coding specification. A Domain-Specific Language (DSL) grammar is designed and developed as a specification for the automatic generation of the output, input, and source code checkers. The DSL grammar and the checker generator tool set are used to evaluate source codes in our programming class. By writing the checkers specification in DSL, the specification is automatically documented and can be reused for similar properties.