On the Cover: Sequential Progression and Item Review in Timed Tests: Patterns in Process Data

Abstract

We are excited to announce the winners of the 12th EM:IP Cover Graphic/Data Visualization Competition. Each year, we invite our readers to submit visualizations that are not only accurate and insightful but also visually compelling and easy to understand. This year's submissions explored key topics in educational measurement, including process data, item characteristics, test design, and score interpretation. We extend our sincere thanks to everyone who submitted their work, and we are especially grateful to the EM:IP editorial board for their thoughtful review and feedback in the selection process.

Winning entries may be featured on the cover of a future EM:IP issue. Previous winners who have not yet appeared on a cover remain eligible for upcoming issues.

This issue's cover features Sequential Progression and Item Review in Timed Tests: Patterns in Process Data, a compelling visualization created by Christian Meyer from the Association of American Medical Colleges and the University of Maryland, along with Ying Jin and Marc Kroopnick, both from the Association of American Medical Colleges.

The visualization, developed using R, presents smoothed density plots derived from process data collected during a high-stakes admissions test. It illustrates how examinees navigated one section of the test within a 95-minute time limit. The x-axis represents elapsed time in minutes. The y-axis segments item positions into five groups: 1 to 15, 16 to 25, 26 to 35, 36 to 45, and 46 to 59. Meyer and his colleagues explain that, for each item group, the height of the plot indicates density. The supports of the estimated densities extend beyond the start and end of the test to allow the plots to approach zero smoothly at the extremes.

Color is used effectively to distinguish between initial engagement and item review. Blue areas indicate when items were first viewed, while red areas show when examinees revisited those same items. The authors describe, “The figure illustrates a common test-taking strategy: examinees initially progress sequentially through the test, as shown by the early blue density peaks for each group. Toward the end of the session, they frequently revisit earlier items, as evidenced by the red peaks clustering near the time limit.” This pattern reflects deliberate time management, with examinees dividing their approach into two distinct phases.

They continue, “In the first phase, they assess each item, either attempting a response or skipping it for later review. In the second phase, they revisit skipped or uncertain items, providing more considered answers when time permits or resorting to random guessing if necessary.”

According to Meyer and his colleagues, the visualization offers valuable insight into examinees’ time management and engagement strategies during timed tests. They conclude, “It captures temporal strategies, such as sequential progression and end-of-session reviews, offering valuable insights into how test-takers interact with exam structure and constraints.”

Although the figure does not include information regarding individual items and is limited to item position ranges, it demonstrates how temporal behavioral data can be represented in an accessible and interpretable format. The clarity and design make it a useful tool for communicating test-taking patterns revealed through process data.

If you are interested in learning more about this data visualization, please contact Christian Meyer at [email protected]. We also invite you to participate in the annual EM:IP Cover Graphic/Data Visualization Competition. Details are available on the NCME and journal websites, and your entry could be featured on the cover of a future issue. For questions or feedback, feel free to reach out to Yuan-Ling Liaw at [email protected].