On Convenience, Diversity, and Generalisability: A Commentary on Scaff et al. (2025)

The Child Language Data Exchange System (CHILDES, MacWhinney 2000) is the jewel in the crown of child language research. Emerging in the 1980s to archive and facilitate the sharing and re-use of precious and laborious-to-collect-and-process corpus data (MacWhinney and Snow 1985), its forward-thinking ethos predated the modern Open Science movement by decades. Thanks to the hard work of Brian MacWhinney and others, it has continued to expand, has birthed similar repositories (AphasiaBank, Forbes et al. 2012; HomeBank, VanDam et al. 2016), and no doubt inspired others (e.g., WordBank, Frank et al. 2017). Progress in the field of child language acquisition has unquestionably accelerated because of its existence. Yet, as Scaff et al. (2025) show in their paper, the data in CHILDES are not fully representative of the languages of the world and the children who learn them. Despite containing corpora on many dozens of languages, those languages are predominantly Indo-European, with the data mostly coming from affluent urban nuclear families in wealthy countries. They conclude that, because of this skew in the data, researchers should be mindful of generalising from the data.

In this commentary, we discuss two issues that the Scaff et al. (2025) paper raises, addressing (i) the importance of data coverage in studies of child language, and (ii) the extent to which demographic variables allow generalisations from existing corpus data.

Research in the cognitive and psychological sciences overwhelmingly relies on convenience sampling. In studies of language, convenience sampling comes in two forms—selection of the target language(s) and selection of participants. Both contain variability. The circa 7000 languages spoken across the world vary on many different dimensions, such that it is difficult to identify substantive universals (Evans and Levinson 2009), and the boundaries of what is possible in language continue to expand as language documentation uncovers new phenomena (Seifart et al. 2018). However, we are a long way from understanding how numerous features of language are acquired. In a paper that analysed language coverage in child language journals across 45 years of publishing, we found a large skew in the literature towards English and a handful of other, mostly Indo-European languages (Kidd and Garcia 2022a). Passmore et al. (2025) juxtaposed those data against two forms of data: (i) the Expanded Graded Intergenerational Disruption Scale (EGIDS, Eberhard et al. 2021; Lewis and Simons 2010), which ranks languages according to their vitality (i.e., their vulnerability to loss, from healthy national languages like English and Spanish to ‘sleeping’ languages with no current speakers), and (ii) large language databases used to represent the design space of phonology (Phoible: Moran and McCloy 2019) and grammar (Grambank: Skirgård et al. 2023). They showed that studies of child language overwhelmingly concentrate on conveniently sampled big (i.e., widely spoken by many speakers) national languages at the expense of smaller vulnerable languages, and that the languages studied occupy a small subset of the estimated design space of language. Thus, convenience sampling has a scientific cost, limiting our capacity to understand how the vast corners of the design space for human language are acquired. There is also a humanitarian cost—ignoring vulnerable languages limits the potential to contribute to efforts to stem the continual flow of language loss (Evans 2022; Pye 2021).

Scaff et al. (2025) identify limitations in language coverage and in participant selection. Skewed language coverage is not unique to child language and is indicative of a broader problem in the cognitive science of language (see Berghoff and Bylund 2025; Bylund et al. 2024). There is no easy solution, but it almost certainly involves diversifying the researchers in the field (Aravena-Bravo et al. 2023; Kidd and Garcia 2022b). In cases of vulnerable languages, widening participation to include community members to create projects that align with community goals (e.g., creating knowledge to inform educational priorities) adds social value (Passmore et al. 2025). Corpus data are an important resource in this endeavour, but the creation of corpora takes considerable time and resources. The gold standard in the field is to collect longitudinal corpora from several children at a sampling density that allows the researcher to capture the acquisition of a large range of frequent and infrequent phenomena. However, achieving such depth comes at the expense of breadth of language coverage. Initiatives like the Sketch Acquisition Project attempt to increase breadth of coverage by sacrificing depth (Hellwig et al. 2023), which will hopefully address some of the problems in language coverage. Technological development in recording will also speed up corpus development (e.g., Scaff et al. 2024), with much promise for relieving the transcription bottleneck.

Increasing language coverage and the rapid pace of technological development demands a reassessment of ethical processes. The CHILDES database pioneered data sharing, the value of which has been immeasurable. That this has been possible is remarkable given the ethics of working with children, who cannot give direct consent for their participation. The standard use of video and the advent of daylong recordings, which capture families in their most unguarded moments, raise considerable issues concerning privacy, an issue that promises to become more complicated to navigate now that an individual's digital footprint presents potentially unimaginable uses as technology develops (see Léon and Cristia 2024). Governments are now protecting their citizens through important privacy laws, giving individuals power to control their data. The end result is that posting data on CHILDES and similar repositories is likely to evolve into a process of continual consent and heightened access restrictions. A related point that is relevant to Indigenous cultures is the issue of data sovereignty (Russo Carroll et al. 2021). Groups whose languages and cultures have been decimated by colonialism quite rightly demand sovereignty over any data collected in their communities (e.g., in Australia, it is considered best practice). Many groups see anything less as a form of epistemic violence that perpetuates colonialism (Woods 2022).

The skew in language coverage raises clear difficulties in generalisation: if we sample from a restricted corner of the total design space of language then we cannot know how phenomena outside of the design space are acquired. However, the degree to which the other demographic biases identified by Scaff et al. (2025)—i.e., socioeconomic status (SES), urbanisation, and family structure—limit generalisation is less clear. As sources of within-language variation, their influence on acquisition will be via the way they change the language environment and other more distal variables that might have a broader influence on cognitive development (e.g., due to increased neurotoxin levels in urban environments, Shadbegian et al. 2019). Distal environmental variables will affect acquisition via complex developmental cascades, and we leave discussion of these for another time. Concerning variation in the input, the key questions are: (i) in what ways does variation in the input affect acquisition? And (ii) can we build theoretical models that account for such variability?

Concerning (i), the answer is complex and, in some cases, not without controversy. Take, for example, SES, the most well-studied variable on the list. One underappreciated problem in discussions of SES and its influence on development is that, as a kind of formative measurement (i.e., it is only measurable through the measurement of other variables, e.g., Bainter and Bollen 2014), it is defined based on several many interacting indicators (e.g., parental education, income, postcode), making it difficult to pin down exactly why it may influence an outcome variable. Additionally, as a distinctly culturally-specific concept, it will be defined differently and will likely have a variable influence on development across countries (Singh et al. 2025). Ambiguity about how it influences acquisition has led to intense debate in countries like the USA (e.g., Sperry et al. 2019; Golinkoff et al. 2019). Debates on its role in acquisition have centred on attainment, but we might instead ask: does variation in the input due to SES have a significant enough effect on the process of acquisition, such that we should not expect to see the same underlying developmental mechanisms at play? If not, then we may be able to make cautious generalisations beyond individual cases. Regardless, we agree with Scaff et al. that good metadata for corpora should include this kind of background information.

Concerning (ii), building a theory of how variation in the input influences acquisition is no small task, in part because it will interact with endogenous sources of variation (e.g., Eghbalzad et al. 2021). Regardless, understanding when and how variation matters has the potential to provide valuable insights into mechanisms underlying acquisition (Kidd and Donnelly 2020).

The cognitive and psychological sciences are currently within an extended period of soul-searching concerning their ability to explain how all humans learn, think, and feel. The crux of this is the realisation that the modern study of human psychology has primarily been a Western (or one might say, WEIRD, Henrich et al. 2010) concern (or perhaps more accurately, dominated by Western European academic traditions). In most cases, ‘the field’, broadly construed, is poorly equipped to engage with issues of cultural influences on behaviour because of a range of barriers, including a lack of researcher diversity, a bias for inferring universals from skewed data, and a lack of meaningful interaction with adjacent disciplines like anthropology (see Núñez et al. 2019). Child language has always had a proud history of crosslinguistic research (e.g., Slobin 1985), and its location at the intersection of psychology, linguistics, anthropology, and computer science means it has the potential to act as a leader in discussions about interdisciplinarity, generalisability, and the necessity of inclusion. Raising awareness of gaps in our data coverage, as Scaff et al. (2025) have done, is an important step in this process.