Secondary Students’ Reading of Socio-Scientific Image-Texts on Climate Change in a GPT-4 Scenario

Abstract

The prominence of multimodal generative artificial intelligence (GenAI) facilitates students’ comprehension of scientific knowledge through linguistic and visual modes. However, there is a lack of research that investigates how students read image-text outputs created in GenAI. We conceptualize a model of image-text reading of GenAI scientific texts that comprises the interpretation, exchange, and evaluation domains. Based on this theoretical model, we explored how 68 junior secondary students read two image-text socio-scientific texts created by GPT-4 with DALL.E plugins, one focusing on cognitive-epistemic aspects and another focusing on social-institutional aspects of climate change. Our findings indicated that these domains did not exhibit a hierarchical structure, while students’ performance in the evaluation domain in the cognitive-epistemic text was better than that in the social-institutional text. More importantly, students expressed a range of uninformed ideas regarding the nature of GenAI when they read the two texts, including equating GenAI to an Internet search engine, picture creators, and human. We discussed how teaching and learning can foster students’ image-text and epistemic reading by targeting the three domains of our theoretical model.