How Timing and Authority in Peer Review Impact STEM Students: A Comparative Assessment of Writing and Critical Thinking in Kinesiology Courses

Abstract

This comparative, mixed-methods study illustrates the impact of weekly facilitated peer review (“Writing Circles”) in STEM courses across time: 1) in a lowerdivision course, Circles improve all learning outcomes for writing and critical thinking, and most significantly, writing; 2) in an upper-division course, Circles are most effective at improving learning outcomes for critical thinking; 3) when comparing scores in the lowerand upper-division courses, we see that critical thinking improves significantly from second to fourth year; 4) finally, we see that upper-division students grant their peers more disciplinary authority during the Circles peer review. Thinking about pedagogy for writing in the disciplines (WID) with respect to time, we distinguish three kinds of questions. First, how much time should students spend on learning to write in the discipline (Deans, 2017; Graham, 1992; Kramer et al., 2019)? Second, given some quantity of time, how should students spend it (Armstrong & Paulson, 2008; Bruffee, 1984; Gere, 1987; Kramer et al., 2019); that is, what should they be doing? Bruffee (1984), Gere (1987), Kramer et al. (2019), and others recommend collaborative learning, specifically peer review. Brieger & Bromley (2014), reporting on their model for facilitating peer review, summarize many of the advantages of incorporating peer review in undergraduate science, technology, engineering, and math (STEM) writing assignments. These include more critical thinking “through improved conceptualization, synthesis, evaluation, and application of new information” (p. 2). Indeed, as vehicles for collaborative learning, peer review groups are one way to decentralize authoritative barriers to entry (Keating, 2019), which can frustrate the efforts of STEM students seeking to join disciplinary discourse communities. Proceeding from this premise, then, the third question arises: the timing of time, or more specifically, at what points in a college education should a student spend time learning and practicing peer review (Beaufort, 2007; Gere, 2019; Keating, 2019)? This third question includes multiple considerations. Are learning outcomes for writing and critical thinking differently impacted by peer review at different stages of a college career? And how do psycho-socio-epistemic dimensions of peer review manifest at different Kramer, Zeccardi, Emhoff, Williams, Dunn, & Rose 306 ATD, VOL18(ISSUE3/4) stages; specifically, how do students perceive each other’s authority in the discipline, and how does that perception impact their learning? Given the ubiquity and necessity of peer review in the sciences generally, an observer might expect to find it similarly situated in STEM classrooms devoted to preparing students for writing and research in the sciences. Instead, as is the case in many disciplines but perhaps in STEM more acutely, the learning and practice of content-area knowledge takes precedence over the learning and practice of peer review. Furthermore, content-area experts rarely are also experts in teaching the writing process, including peer review. Therefore, when and how to effectively develop students’ ability to write is a central concern for educators invested in preparing students for successful careers in the sciences. Kramer et al. (2019) note that “student peer review can be unhelpful or even backfire...We ourselves have witnessed unor understructured peer review resulting in students giving each other too little, incorrect, or otherwise unhelpful advice” (p. 29). While forms of student peer review have long been adopted and researched in the field of writing studies (e.g., Gere, 1987; DiPardo & Warshauer Freedman, 1988; Cho & Schunn, 2007), there continues to be a need to better understand its benefits and how they are realized over time in discipline-specific courses. Kimberly Baker (2016), studying peer review within individual sociology courses, notes that “while numerous studies have investigated peer review, these studies have focused primarily on outcomes rather than process” (p. 4). In STEM disciplines, Julia Reynolds and colleagues (2012) say there is a general lack of writing-to-learn strategies, which employ writing as means for students to gain understanding of science concepts and methods. Undertaking a National Science Foundation-funded study and drawing on the expertise of 12 STEM research and education experts, Reynolds et al. built on a previous review (Rivard, 1994) to identify empirically validated writing-to-learn practices in STEM (Gerdeman et al., 2007; Pelaez, 2002; Walvoord et al., 2008). More recent research on the adoption of peer review in STEM courses suggests that it can be beneficial (Finkenstaedt-Quinn et al., 2019; Finkenstaedt-Quinn et al., 2021; Reddy et al., 2021; Yalch et al., 2019). Important considerations revolve around the guiding and timing of student peer review. Gere (2019) and Keating (2019) show that students understand, experience, and respond to peer review in different ways at different stages of their education. In the early stages of their studies, students regard neither themselves nor their peers as legitimate, authoritative sources of knowledge. As a result, they tend to resist peer review, experience it negatively, and describe it as a frustrating, inauthentic waste of time. Conversely, students in the later stages of their education begin to understand themselves and their peers as authorized sources of knowledge, and are therefore more likely to value the feedback they receive in peer review. Given Keating’s (2019) account of students’ evolving understanding and experience of peer review, it remains to be discerned not just how and to what extent these different perspectives manifest in student writing but also when. Whereas Gere’s (2019) and Keating’s (2019) studies are broad surveys of students writing in 47 different majors, our research focused in a detailed way on a single discipline, kinesiology. Further, whereas our previous work, Kramer et al. 2019, examined the impacts of iterated, structured, facilitated peer review groups (“Writing Circles” or “Circles”) at a single developmental point, this study compares the impact of Circles on student writing and critical thinking at different stages in their major. Accordingly, this study analyzes student writing from a lower-division WID course in Research Methods & Writing (RMW) typically taken in the second year and an upper-division lab course in Exercise Physiology (EP) typically taken during the fourth year. We employed mixed methods, using quantitative analyses to compare disciplinary writing produced with Circles at both stages to writing produced without Circles, and qualitative analyses of reflective writing to compare lower-division Kinesiology students’ understanding and experience of Circles to that of upper-division Kinesiology students. Our quantitative research questions focused on the ways in which these differences manifest in student writing, specifically: How does peer review impact students’ writing at different stages of their disciplinary and psycho-socio-epistemic How Timing and Authority in Peer Review Impact STEM Students 307 ATD, VOL18(ISSUE3/4) development? And how do the impacts of peer review practiced among novice groups differ from those with more disciplinary expertise? Our qualitative research questions asked how and to what extent students’ understanding and experience of Circles changed as they progressed through the major. Below, we report the quantitative results first, followed by the qualitative results. Our data indicate that the benefits of peer review advance in parallel with evolutions in students’ understanding and experience of peer review, and that peer review manifests in different ways at different times in students’ cognitive and disciplinary development. Context on Kinesiology and Circles Kinesiology at Saint Mary’s College of California is an interdisciplinary major. In addition to courses taken through the Department of Kinesiology, students take courses in the natural sciences, social sciences, and business, based upon their chosen area of emphasis: Exercise Science, Health Promotion, or Sport & Recreation Management. As a consequence, students majoring in kinesiology exhibit a broad spectrum of writing abilities, styles, processes, and backgrounds that might be found in other majors across campus. The lower-division course Research Methods & Writing in Kinesiology (RMW) provides students the opportunity to consider fundamental research questions in kinesiology, read and interpret research articles, and explore issues related to evaluation and measurement techniques. As their final project, students write a research proposal consisting of an extensive literature review and a detailed proposal for an experiment (Kramer et al., 2019). The RMW course immerses students in empirical studies in order to familiarize them with the research methodology used in kinesiology. The course includes an overview of various types of discipline-specific writing and an introduction to research, measurement, and evaluation within the kinesiology discipline. The upper-division course Exercise Physiology (EP), which has RMW as one of its prerequisites, requires students to enroll concurrently in a lecture and lab to learn theoretical concepts and gain proficiencies in applied laboratory skills, respectively. Throughout the EP course, students write four lab reports, each progressing in complexity and breadth of content, based on hands-on experiments conducted in labs, incorporating theory learned in the lecture portion of the course. These lab reports are intended to provide a deeper inquiry and discussion into main EP concepts while also providing the opportunity to improve scientific writing skills. Excerpts of the EP syllabus illustrate how the course teaches students to both acquire disciplinary expertise and write as experts in their field: “Students are expected