Comparing The Readability Level of Biology Exams Texts in Malta

Abstract

We used a quantitative approach to evaluate the readability level of comprehension test passages in high-stakes, advanced-level biology exams in Malta. We studied 37 passages from a public institution and 54 from national exams from the years 1996 to 2022. Using an online computational tool called Coh-Metrix we calculated three readability formulas, Flesch Reading Ease, Flesch-Kincaid Grade Level, and Coh-Metrix L2 Reading Index. All three indices indicated that all texts were "difficult" and understood by English-speaking undergraduates; however, for Maltese students, English is a second language (L2). We computed ANOVA to analyse differences in readability indices in the two high-stakes exams. The results revealed no significant difference in the readability of the texts, implying that the institution exposes students to the same level of difficulty as the national average. The Pearson correlation coefficient revealed that both exams maintained the same degree of readability throughout the study period. We found that the average word count and sentence length were significantly shorter in the public institution exams. Lexical diversity is an important indicator of how complex and difficult a text is to read. It measures how many different words occur in the text. The two indices of lexical diversity used showed that the texts of the institution were less complicated and difficult to read. Thus, in a way, paper setters at the institution attempt to match difficulty to the students’ level, keeping in mind that they have been learning the subject for one year. This study benefits paper-setters by allowing them to evaluate and match the readability of the comprehension test passages with 17-to 18-year-old L2 students.