Three applications of semantic network analysis to individual student think-aloud data

Abstract

Student during-learning data such as think-alouds or writing are often coded for use of strategies or moves, but less often for what knowledge the student is using. However, analyzing the content of such products could yield much valuable information. A promising technique for analyzing the content of student products is semantic network analysis, more widely used in political science, communication, information science, and some other social science disciplines. We reviewed the small literature on semantic network analysis (SemNA) of individuals with relevant outcomes to identify which network analysis metrics might be suitable. The Knowledge Integration (KI) framework from science education is discussed as focusing on amount and structure of student knowledge, and therefore especially relevant for testing with SemNA metrics. We then re-analyze three published think-aloud data sets from undergraduate students learning introductory biology with the metrics found in the literature review. Significant relations with posttest comprehension score are found for number of nodes and edges; degree and betweenness centrality; diameter, and mean distance. Inconsistent results possibly due to text-specific features were found for number of clusters, LCC, and density, and null results were found for PageRank centrality and centralization degree. Basic principles from the KI framework are supported—amount of information (nodes), connections (edges, average degree), key ideas (degree and betweenness centrality) and length of causal chains (mean distance and diameter) are related to posttest comprehension, but not density or LCC. Possible explanations for slight variations across data sets are discussed, and alternative theories and metrics are offered.