Improved Testing of PrairieLearn Question Generators

Abstract

With many institutions forced online due to the pandemic, assessments became a challenge for many educators. Take-home exams provided the flexibility required for varied student needs (and time zones), but they were vulnerable to cheating. In response, many turned to tools that could present a different exam for every student. PrairieLearn is a feature-rich open-source package that allows educators to author randomized Question Generators; we have been using the tool extensively for the last two years, and it has a fast-growing educator user base. One of the first issues we noticed with the system was that the only way to quality assure (QA) a question was to click the new variant button, which would spin whatever internal random number generators were used again to produce a new question. Sometimes it was the same one you had just seen, and other times it would never seem to "hit'' on the variant you were looking to debug. This poster describes our team's work to solve this problem through the design of an API that would allow a question to declare how many total variants it had, and be asked to render variant i. The user interface could then be extended to list what variant the QA team was viewing out of the total (e.g., 7/50), and a next, previous and go to a particular variant buttons would allow for the team to easily QA all variants.