A new concept of replication

ABSTRACTThe replication crisis has spawned discussions on the meaning of replication. In fact, in order to determine whether an experiment fails to replicate, it is necessary to establish what replication is. This is, however, a difficult task, as it is possible to attribute different meanings to it. This paper offers a solution to this problem of ambiguity by engineering a concept of replication that, if compared to other proposals, stands out for being not only broadly applicable but also sufficiently specific. It features a minimal level of operationalism, which would otherwise limit its applicability, while it heavily relies on replication’s specific epistemic functions, which are inter-disciplinary. Another merit is its context sensitivity, which enables it to differentiate instances of replication from non-instances of replication in every scientific discipline according to the discipline’s own standards.KEYWORDS: Replicationambiguityconceptual engineeringresampling account of replicationcontextuality AcknowledgmentsI would like to express my sincere gratitude to Claus Beisbart and Matthias Rolffs from the University of Bern, and to two anonymous reviewers of this journal for reading my paper. Their valuable insights and constructive feedback significantly contributed to the refinement of this paper. I am also indebted to the audience at the reading group in Philosophy of Science at Caltech (California Institute of Technology). Their thoughtful questions and engaging discussions helped me enhance the quality of this work. Furthermore, I extend my heartfelt appreciation to the team of the Group of Materials Research with Neutron and Ion Beams (MRNIB) led by Dr. Jiří Vacík at the Czech Academy of Sciences for their feedback on my project on replicability. Their input, particularly the suggestion of the case of replication for thermal neutron-induced reactions in nuclear physics, significantly enriched the content of this paper.Disclosure statementNo potential conflict of interest was reported by the author(s).Notes1 ‘Experiment’ should be understood here as a general term encompassing measurements, studies, and observations. The same is true for the use of ‘experiment’ in the definition of Replication2.2 That replication is an ambiguous term is discussed also in Schmidt (Citation2009): ‘The word replication is used as […] to describe various meanings’ (p. 90); ‘the notion of replication has several meanings and is a very ambiguous term.’ (p. 98). See Leonelli (Citation2018) for a more exhaustive list of meanings of replication.3 See for instance in the abstract, where Machery (Citation2020) states: ‘This article develops a new, general account of replication (“the Resampling Account of replication”)’ (p. 545).4 A factor of an experiment is a controlled independent variable. Levels should be understood as the states, values, or magnitudes that a factor could take. For the case of treatment, for instance, a medication could be given in different doses (e.g., 10 , 20 , 50 mg). These three doses constitute the three levels of the treatment.5 An experiment that resamples some experimental component from a different population, for instance, should count as ‘extension,’ rather than ‘replication.’ (Machery Citation2020, 557).6 Indeed, Schmidt implies that the function of controlling for sampling error, which is done by resampling, merely targets participant selection (Schmidt Citation2009, 93).7 In this paper, I contrast operational definitions with functional ones. Operational definitions are those that include a specific characterization of which set of procedures are constitutive of the concept being defined. Functional definitions are those that include a specific characterization of which set of epistemic functions are fulfilled by the concept being defined. In this sense, I am in line with the use of ‘operational’ that can be found in Nosek and Errington (Citation2020).8 In addition to this, physicists also use tools and data generators, but these two categories overlap with the ones just mentioned.9 It is stricter because Replication1 allows for repetition of exactly the same experiment, without resampling, while Replication (RAR) requires an act of resampling.10 Nosek and Errington (Citation2020) actually provide a definition of replication which is neutral with respect to whether it is successful. For this reason, they claim that any outcome of replication should either increase or decrease our confidence in a claim of prior research. Here I provide what their definition would be for successful replications, as this is what the other concepts previously discussed do. If a replication is successful, then its outcome should increase (and not decrease) our confidence in the claim that we have replicated.11 Debates on how we should conceptualize reliability and validity in qualitative research is still open. Different proposals have been put forward. See for instance Altheide and Johnson (Citation1994) ; Guba and Lincoln (Citation1981) ; Kuzel and Like (Citation1991) . For more recent accounts, see Franklin and Ballan (Citation2001) .12 A test for semantic ambiguity proposed by Aristotle is to try to construct a concept that encompasses all the different meanings. If this is possible, then the apparent ambiguity gets dissolved. See the entry ‘Ambiguity’ in the Stanford Encyclopedia of Philosophy (Sennet Citation2016).