Open Mind最新文献

{"title":"Large Language Models and the Wisdom of Small Crowds.","authors":"Sean Trott","doi":"10.1162/opmi_a_00144","DOIUrl":"10.1162/opmi_a_00144","url":null,"abstract":"<p><p>Recent advances in Large Language Models (LLMs) have raised the question of replacing human subjects with LLM-generated data. While some believe that LLMs capture the \"wisdom of the crowd\"-due to their vast training data-empirical evidence for this hypothesis remains scarce. We present a novel methodological framework to test this: the \"number needed to beat\" (NNB), which measures how many humans are needed for a sample's quality to rival the quality achieved by GPT-4, a state-of-the-art LLM. In a series of pre-registered experiments, we collect novel human data and demonstrate the utility of this method for four psycholinguistic datasets for English. We find that NNB > 1 for each dataset, but also that NNB varies across tasks (and in some cases is quite small, e.g., 2). We also introduce two \"centaur\" methods for combining LLM and human data, which outperform both stand-alone LLMs and human samples. Finally, we analyze the trade-offs in data cost and quality for each approach. While clear limitations remain, we suggest that this framework could guide decision-making about whether and how to integrate LLM-generated data into the research pipeline.</p>","PeriodicalId":32558,"journal":{"name":"Open Mind","volume":"8 ","pages":"723-738"},"PeriodicalIF":0.0,"publicationDate":"2024-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11142632/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141201704","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"How Do You Know If You Were Mind Wandering? Dissociating Explicit Memories of Off Task Thought From Subjective Feelings of Inattention.","authors":"Nathan K Mathews, Umer Bin Faiz, Nicholaus P Brosowsky","doi":"10.1162/opmi_a_00142","DOIUrl":"10.1162/opmi_a_00142","url":null,"abstract":"<p><p>Mind wandering is a common experience in which your attention drifts away from the task at hand and toward task-unrelated thoughts. To measure mind wandering we typically use experience sampling and retrospective self-reports, which require participants to make metacognitive judgments about their immediately preceding attentional states. In the current study, we aimed to better understand how people come to make such judgments by introducing a novel distinction between explicit memories of off task thought and subjective feelings of inattention. Across two preregistered experiments, we found that participants often indicated they were \"off task\" and yet had no memory of the content of their thoughts-though, they were less common than remembered experiences. Critically, remembered experiences of mind wandering and subjective feelings of inattention differed in their behavioral correlates. In Experiment 1, we found that only the frequency of remembered mind wandering varied with task demands. In contrast, only subjective feelings of inattention were associated with poor performance (Experiments 1 and 2) and individual differences in executive functioning (Experiment 2). These results suggest that the phenomenology of mind wandering may differ depending on how the experiences are brought about (e.g., executive functioning errors versus excess attentional resources), and provide preliminary evidence of the importance of measuring subjective feelings of inattention when assessing mind wandering.</p>","PeriodicalId":32558,"journal":{"name":"Open Mind","volume":"8 ","pages":"666-687"},"PeriodicalIF":0.0,"publicationDate":"2024-05-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11142633/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141200924","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Instruction on the Scientific Method Provides (Some) Protection Against Illusions of Causality.","authors":"Julie Y L Chow, Micah B Goldwater, Ben Colagiuri, Evan J Livesey","doi":"10.1162/opmi_a_00141","DOIUrl":"10.1162/opmi_a_00141","url":null,"abstract":"<p><p>People tend to overestimate the efficacy of an ineffective treatment when they experience the treatment and its supposed outcome co-occurring frequently. This is referred to as the <i>outcome density</i> effect. Here, we attempted to improve the accuracy of participants' assessments of an ineffective treatment by instructing them about the scientific practice of comparing treatment effects against a relevant base-rate, i.e., when no treatment is delivered. The effect of these instructions was assessed in both a trial-by-trial contingency learning task, where cue administration was either decided by the participant (Experiments 1 & 2) or pre-determined by the experimenter (Experiment 3), as well as in summary format where all information was presented on a single screen (Experiment 4). Overall, we found two means by which base-rate instructions influence efficacy ratings for the ineffective treatment: 1) When information was presented sequentially, the benefit of base-rate instructions on illusory belief was mediated by reduced sampling of cue-present trials, and 2) When information was presented in summary format, we found a <i>direct</i> effect of base-rate instruction on reducing causal illusion. Together, these findings suggest that simple instructions on the scientific method were able to decrease participants' (over-)weighting of cue-outcome coincidences when making causal judgements, as well as decrease their tendency to over-sample cue-present events. However, the effect of base-rate instructions on correcting illusory beliefs was incomplete, and participants still showed illusory causal judgements when the probability of the outcome occurring was high. Thus, simple textual information about assessing causal relationships is partially effective in influencing people's judgements of treatment efficacy, suggesting an important role of scientific instruction in debiasing cognitive errors.</p>","PeriodicalId":32558,"journal":{"name":"Open Mind","volume":"8 ","pages":"639-665"},"PeriodicalIF":0.0,"publicationDate":"2024-05-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11142631/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141200925","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Toward the Emergence of Intelligent Control: Episodic Generalization and Optimization.","authors":"Tyler Giallanza, Declan Campbell, Jonathan D Cohen","doi":"10.1162/opmi_a_00143","DOIUrl":"10.1162/opmi_a_00143","url":null,"abstract":"<p><p>Human cognition is unique in its ability to perform a wide range of tasks and to learn new tasks quickly. Both abilities have long been associated with the acquisition of knowledge that can generalize across tasks and the flexible use of that knowledge to execute goal-directed behavior. We investigate how this emerges in a neural network by describing and testing the Episodic Generalization and Optimization (EGO) framework. The framework consists of an episodic memory module, which rapidly learns relationships between stimuli; a semantic pathway, which more slowly learns how stimuli map to responses; and a recurrent context module, which maintains a representation of task-relevant context information, integrates this over time, and uses it both to recall context-relevant memories (in episodic memory) and to bias processing in favor of context-relevant features and responses (in the semantic pathway). We use the framework to address empirical phenomena across reinforcement learning, event segmentation, and category learning, showing in simulations that the same set of underlying mechanisms accounts for human performance in all three domains. The results demonstrate how the components of the EGO framework can efficiently learn knowledge that can be flexibly generalized across tasks, furthering our understanding of how humans can quickly learn how to perform a wide range of tasks-a capability that is fundamental to human intelligence.</p>","PeriodicalId":32558,"journal":{"name":"Open Mind","volume":"8 ","pages":"688-722"},"PeriodicalIF":0.0,"publicationDate":"2024-05-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11142636/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141200926","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Two Determinants of Dynamic Adaptive Learning for Magnitudes and Probabilities.","authors":"Cedric Foucault, Florent Meyniel","doi":"10.1162/opmi_a_00139","DOIUrl":"10.1162/opmi_a_00139","url":null,"abstract":"<p><p>Humans face a dynamic world that requires them to constantly update their knowledge. Each observation should influence their knowledge to a varying degree depending on whether it arises from a stochastic fluctuation or an environmental change. Thus, humans should dynamically adapt their learning rate based on each observation. Although crucial for characterizing the learning process, these dynamic adjustments have only been investigated empirically in magnitude learning. Another important type of learning is probability learning. The latter differs from the former in that individual observations are much less informative and a single one is insufficient to distinguish environmental changes from stochasticity. Do humans dynamically adapt their learning rate for probabilities? What determinants drive their dynamic adjustments in magnitude and probability learning? To answer these questions, we measured the subjects' learning rate dynamics directly through real-time continuous reports during magnitude and probability learning. We found that subjects dynamically adapt their learning rate in both types of learning. After a change point, they increase their learning rate suddenly for magnitudes and prolongedly for probabilities. Their dynamics are driven differentially by two determinants: change-point probability, the main determinant for magnitudes, and prior uncertainty, the main determinant for probabilities. These results are fully in line with normative theory, both qualitatively and quantitatively. Overall, our findings demonstrate a remarkable human ability for dynamic adaptive learning under uncertainty, and guide studies of the neural mechanisms of learning, highlighting different determinants for magnitudes and probabilities.</p>","PeriodicalId":32558,"journal":{"name":"Open Mind","volume":"8 ","pages":"615-638"},"PeriodicalIF":0.0,"publicationDate":"2024-05-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11093407/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140921748","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Neural Networks as Cognitive Models of the Processing of Syntactic Constraints.","authors":"Suhas Arehalli, Tal Linzen","doi":"10.1162/opmi_a_00137","DOIUrl":"10.1162/opmi_a_00137","url":null,"abstract":"<p><p>Languages are governed by <i>syntactic constraints</i>-structural rules that determine which sentences are grammatical in the language. In English, one such constraint is <i>subject-verb agreement</i>, which dictates that the number of a verb must match the number of its corresponding subject: \"the dog<i>s</i> run\", but \"the dog run<i>s</i>\". While this constraint appears to be simple, in practice speakers make agreement errors, particularly when a noun phrase near the verb differs in number from the subject (for example, a speaker might produce the ungrammatical sentence \"the key to the cabinets are rusty\"). This phenomenon, referred to as <i>agreement attraction</i>, is sensitive to a wide range of properties of the sentence; no single existing model is able to generate predictions for the wide variety of materials studied in the human experimental literature. We explore the viability of neural network language models-broad-coverage systems trained to predict the next word in a corpus-as a framework for addressing this limitation. We analyze the agreement errors made by Long Short-Term Memory (LSTM) networks and compare them to those of humans. The models successfully simulate certain results, such as the so-called number asymmetry and the difference between attraction strength in grammatical and ungrammatical sentences, but failed to simulate others, such as the effect of syntactic distance or notional (conceptual) number. We further evaluate networks trained with explicit syntactic supervision, and find that this form of supervision does not always lead to more human-like syntactic behavior. Finally, we show that the corpus used to train a network significantly affects the pattern of agreement errors produced by the network, and discuss the strengths and limitations of neural networks as a tool for understanding human syntactic processing.</p>","PeriodicalId":32558,"journal":{"name":"Open Mind","volume":"8 ","pages":"558-614"},"PeriodicalIF":0.0,"publicationDate":"2024-05-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11093404/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140921183","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Examining the Impact of Reading Fluency on Lexical Decision Results in French 6th Graders.","authors":"Marie Lubineau, Cassandra Potier Watkins, Hervé Glasel, Stanislas Dehaene","doi":"10.1162/opmi_a_00140","DOIUrl":"10.1162/opmi_a_00140","url":null,"abstract":"<p><strong>Purpose: </strong>How does lexical decision behavior vary in students with the same grade level (all students were in their first year of middle-school), but different levels of reading fluency? Here, we tested a prediction of the dual-route model: as fluency increases, variations in the results may reflect a decreasing reliance on decoding and an increasing reliance on the lexical route.</p><p><strong>Method: </strong>1,501 French 6^th graders passed a one-minute speeded reading-aloud task evaluating fluency, and a ten-minute computerized lexical decision task evaluating the impact of lexicality, length, word frequency and pseudoword type.</p><p><strong>Results: </strong>As predicted, the word length effect varied dramatically with reading fluency, with the least fluent students showing a length effect even for frequent words. The frequency effect also varied, but solely in proportion to overall reading speed, suggesting that frequency affects the decision stage similarly in all readers, while length disproportionately impacts poor readers. Response times and errors were also affected by pseudoword type (e.g., letter substitutions or transpositions), but these effects showed minimal variation with fluency. Overall, lexical decision variables were excellent predictors of reading fluency (r = 0.62).</p><p><strong>Conclusion: </strong>Our results highlight the variability in middle-school reading ability and describe how a simple lexical decision task can be used to assess students' mental lexicon (vocabulary) and the automatization of reading skills.</p>","PeriodicalId":32558,"journal":{"name":"Open Mind","volume":"8 ","pages":"535-557"},"PeriodicalIF":0.0,"publicationDate":"2024-05-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11093403/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140920953","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Does Distance Matter? How Physical and Social Distance Shape Our Perceived Obligations to Others.","authors":"Julia Marshall, Matti Wilks","doi":"10.1162/opmi_a_00138","DOIUrl":"10.1162/opmi_a_00138","url":null,"abstract":"<p><p>Debates within moral philosophy have long centered on the question of whether we are more obligated to help those close to us compared to those who are farther away. Despite these debates, we have little understanding of our psychological intuitions about these issues. In the current study, we presented adults and children (5- to 9-year-olds) in the United States (<i>N</i> = 406) with hypothetical scenarios involving pairs of socially and physically close and far strangers and asked about their obligations to help one another. In general, younger children (∼6-year-olds) were more inclined to describe strangers as obligated to help one another compared to older children (∼8-year-olds) and adults. For physical distance, we documented an age-related trend where younger children were less inclined to consider physical distance when ascribing obligations to help compared to older children and adults. For social distance, we found different results depending on how social distance was manipulated. In Study 1, where social distance was manipulated via mere similarity, we found an age-related effect where adults, but not younger or older children, judged that individuals are more obligated to help socially close others relative to far ones. In Study 2, where social distance was manipulated via explicit group membership, we did not find an age trend. Instead, participants generally described individuals as more obligated to help an ingroup member relative to an outgroup one. These results demonstrate that the tendency to deny obligations towards distant others is a belief that emerges relatively late in development.</p>","PeriodicalId":32558,"journal":{"name":"Open Mind","volume":"8 ","pages":"511-534"},"PeriodicalIF":0.0,"publicationDate":"2024-05-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11093409/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140920632","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Toddlers Prefer Agents Who Help Those Facing Harder Tasks.","authors":"Brandon M Woo, Shari Liu, Hyowon Gweon, Elizabeth S Spelke","doi":"10.1162/opmi_a_00129","DOIUrl":"https://doi.org/10.1162/opmi_a_00129","url":null,"abstract":"<p><p>Capacities to understand and evaluate others' actions are fundamental to human social life. Infants and toddlers are sensitive to the costs of others' actions, infer others' values from the costs of the actions they take, and prefer those who help others to those who hinder them, but it is largely unknown whether and how cost considerations inform early understanding of third-party prosocial actions. In three experiments (<i>N</i> = 94), we asked whether 16-month-old toddlers value agents who selectively help those who need it most. Presented with two agents who attempted two tasks, toddlers preferentially looked to and touched someone who helped the agent in greater need, both when one agent's task required more effort and when the tasks were the same but one agent was weaker. These results provide evidence that toddlers engage in need-based evaluations of helping, applying their understanding of action utilities to their social evaluations.</p>","PeriodicalId":32558,"journal":{"name":"Open Mind","volume":"8 ","pages":"483-499"},"PeriodicalIF":0.0,"publicationDate":"2024-04-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11045033/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140864930","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Role of Uniform Textures in Making Texture Elements Visible in the Visual Periphery.","authors":"Marco Bertamini, Carolina Maria Oletto, Giulio Contemori","doi":"10.1162/opmi_a_00136","DOIUrl":"https://doi.org/10.1162/opmi_a_00136","url":null,"abstract":"<p><p>There are important differences between central and peripheral vision. With respect to shape, contours retain phenomenal sharpness, although some contours disappear if they are near other contours. This leads to some uniform textures to appear non-uniform (Honeycomb illusion, Bertamini et al., 2016). Unlike other phenomena of shape perception in the periphery, this illusion is showing how continuity of the texture does not contribute to phenomenal continuity. We systematically varied the relationship between central and peripheral regions, and we collected subjective reports (how far can one see lines) as well as judgments of line orientation. We used extended textures created with a square grid and some additional lines that are invisible when they are located at the corners of the grid, or visible when they are separated from the grid (control condition). With respects to subjective reports, we compared the region of visibility for cases in which the texture was uniform (Exp 1a), or when in a central region the lines were different (Exp 1b). There were no differences, showing no role of objective uniformity on visibility. Next, in addition to the region of visibility we measured sensitivity using a forced-choice task (line tilted left or right) (Exp 2). The drop in sensitivity with eccentricity matched the size of the region in which lines were perceived in the illusion condition, but not in the control condition. When participants were offered a choice to report of the lines were present or absent (Exp 3) they confirmed that they did not see them in the illusion condition, but saw them in the control condition. We conclude that mechanisms that control perception of contours operate differently in the periphery, and override prior expectations, including that of uniformity. Conversely, when elements are detected in the periphery, we assign to them properties based on information from central vision, but these shapes cannot be identified correctly when the task requires such discrimination.</p>","PeriodicalId":32558,"journal":{"name":"Open Mind","volume":"8 ","pages":"462-482"},"PeriodicalIF":0.0,"publicationDate":"2024-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11045036/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140868266","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}