Analyzing the influence of block highlighting on beginning programmers' reading behavior using eye tracking

To improve programming education, it is worthwhile to investigate how understanding of source code takes place. Reading is a foundational skill essential to tracing, debugging and understanding code. As readers' working memory is limited, their proficiency strongly relies on the ability to chunk and organize source code material. There are several alternative visual representations aiming to facilitate code reading, of which syntax highlighting is the most common. We hypothesized that highlighting code blocks in the style of the language Stride would scaffold students' reading process more effectively. In this exploraratory study, the code reading behavior of seven students was investigated in the context of a code comprehension task using eye tracking, stimulated Retrospective Think Aloud and interviews. The tasks were either presented using plain code, with standard syntax highlighting or with block highlighting. We found no significant differences in the students' comprehension scores. Their reading patterns, however, turned out to be strongly affected by the highlighting conditions. Remarkably, this effect appears to be subconscious, as participants consistently claimed to not have been influenced by the block highlighting. This also implies that in order to study code reading behavior and comprehension, participant reports alone are not sufficient. An approach combining objective data, such as that of eye tracking, combined with classic stimulated retrospective think aloud sessions is needed to paint the full picture.