Behavior Research Methods最新文献

{"title":"Comparative judgement as a research tool: A meta-analysis of application and reliability.","authors":"George Kinnear, Ian Jones, Ben Davies","doi":"10.3758/s13428-025-02744-w","DOIUrl":"10.3758/s13428-025-02744-w","url":null,"abstract":"<p><p>Comparative judgement (CJ) provides methods for constructing measurement scales, by asking assessors to make a series of pairwise comparisons of the artefacts or representations to be scored. Researchers using CJ need to decide how many assessors to recruit and how many comparisons to collect. They also need to gauge the reliability of the resulting measurement scale, with two different estimates in widespread use: scale separation reliability (SSR) and split-halves reliability (SHR). Previous research has offered guidance on these issues, but with either limited empirical support or focused only on education research. In this paper, we offer guidance based on our analysis of 101 CJ datasets that we collated from previous research across a range of disciplines. We present two novel findings, with substantive implications for future CJ research. First, we find that collecting ten comparisons for every representation is generally sufficient; a more lenient guideline than previously published. Second, we conclude that SSR can serve as a reliable proxy for inter-rater reliability, but recommend that researchers use a higher threshold of .8, rather than the current standard of .7.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 8","pages":"222"},"PeriodicalIF":4.6,"publicationDate":"2025-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12246014/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144607263","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"An investigation of the convergent validity and test-retest reliability of three uncertainty preference measures.","authors":"Guangyu Zhu, Yiyun Shou, Michael Smithson","doi":"10.3758/s13428-025-02729-9","DOIUrl":"10.3758/s13428-025-02729-9","url":null,"abstract":"<p><p>Establishing a reliable and valid measure is crucial for ensuring the accuracy and replicability of research findings on risk and uncertainty preference. However, few studies have assessed the reliability and validity of behavioral measures of uncertainty preference. This study examined the convergent validity and test-retest reliability of three commonly used uncertainty preference measures: forced binary choice, certainty equivalent, and matching probability tasks. Experiments 1 (N = 302) and 2 (N = 366) tested the convergent validity and test-retest reliability of one-off assessment of these measures and found that the three measures did not demonstrate satisfactory convergent validity and test-retest reliability for the one-off assessment. Experiment 3 (N = 311) increased the number of repeats to explore whether repeated measurements could enhance convergent validity and test-retest reliability by addressing the attenuation effect of lack of reliability. The convergent validity between certainty equivalent and matching probability improved in the repeated measurement condition. However, the test-retest reliability of the three measures was still not satisfactory in repeated measurement conditions. These findings highlight the measurement issues in the behavioral measures of uncertainty preferences. The potential causes of this low validity and reliability of behavioral measures of uncertainty preference are discussed.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 8","pages":"221"},"PeriodicalIF":4.6,"publicationDate":"2025-07-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12241273/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144599235","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multidimensional normative ratings for the body emotional stimulus set in Bochum Emotional Stimulus Set in Chinese samples.","authors":"Li Zhang, Pengyu Liu, Yan Shi, Xiaoyi Liu, Yunpeng Liu, Huazhan Yin","doi":"10.3758/s13428-025-02742-y","DOIUrl":"https://doi.org/10.3758/s13428-025-02742-y","url":null,"abstract":"<p><p>The body serves as a vital medium for emotional expression. In this study, we aimed to extend the Bochum Emotional Stimulus Set (BESST), which comprises 565 body pictures depicting expressions of neutral emotions and six basic emotions (happiness, sadness, fear, anger, disgust, and surprise). This study included two tasks. One hundred college students (M_age = 18.82; SD_age = 0.89) underwent the emotion recognition task of pictures (choose one out of seven emotions) and 103 college students (M_age = 19.83; SD_age = 0.84) underwent the rating task of emotional dimensions (motivation, valence, arousal, and dominance; rated on a scale of 1-9). The results showed that: (1) On average, neutrality had the highest recognition rate (94%), followed by sadness (89.04%) and happiness (81.26%), anger (74.03%), disgust (69.61%), fear (67.97%), and surprise (64.45%). (2) The main effects of emotional type in ratings of four dimensions were all significant. Specifically, fear, disgust, and sadness have similar emotional characteristics (withdrawal-motivation, high arousal, low pleasure, and low dominance), and emotions with higher biological relevance (such as disgust) have a greater degree of emotional characteristics. Happiness and anger share similar emotional characteristics, including approach motivation, high arousal, and dominance; however, they represent opposite ends of the valence dimension. (3) There were gender differences in motivation dimension ratings of surprised pictures and arousal dimension ratings of neutral pictures. We provided the recognition rate and rating values for each of the four dimensions for each picture. This set serves as a valuable resource for cross-cultural research and behavioral or clinical studies related to emotional disorders.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 8","pages":"220"},"PeriodicalIF":4.6,"publicationDate":"2025-07-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144590365","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The UCI Phonotactic Calculator: An online tool for computing phonotactic metrics.","authors":"Connor Mayer, Arya Kondur, Megha Sundara","doi":"10.3758/s13428-025-02725-z","DOIUrl":"10.3758/s13428-025-02725-z","url":null,"abstract":"<p><p>This paper presents the UCI Phonotactic Calculator (UCIPC), a new online tool for quantifying the occurrence of segments and segment sequences in a corpus. This tool has several advantages compared to existing tools: it allows users to supply their own training data, meaning it can be applied to any language for which a corpus is available; it computes a wider range of metrics than most existing tools; and it provides an accessible point-and-click interface that allows researchers with more modest technical backgrounds to take advantage of phonotactic models. After describing the metrics implemented by the calculator and how to use it, we present the results of a proof-of-concept study comparing how well different types of metrics implemented by the UCIPC predict human responses from eight published nonce word acceptability judgment studies across four different languages. These results suggest that metrics that take into account the relative position of sounds and include word boundaries are better at predicting human responses than those that are based on the absolute position of sounds and do not include word boundaries. We close by discussing the usefulness of tools like the UCIPC in experimental design and analysis and outline several areas of future research that this tool will help support.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 8","pages":"218"},"PeriodicalIF":4.6,"publicationDate":"2025-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12226677/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144558859","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A toolbox for generating multidimensional 3D objects with fine-controlled feature space: Quaddle 2.0.","authors":"Xuan Wen, Leo Malchin, Thilo Womelsdorf","doi":"10.3758/s13428-025-02736-w","DOIUrl":"10.3758/s13428-025-02736-w","url":null,"abstract":"<p><p>Multidimensional 3D-rendered objects are an important component of vision research and video-gaming applications, but it has remained challenging to parametrically control and efficiently generate those objects. Here, we describe a toolbox for controlling and efficiently generating 3D-rendered objects composed of 10 separate visual feature dimensions that can be fine-adjusted using Python scripts. The toolbox defines objects as multidimensional feature vectors with primary dimensions (object body related features), secondary dimensions (head related features), and accessory dimensions (including arms, ears, or beaks). The toolbox interfaces with the freely available Blender software to create objects. The toolbox makes it possible to gradually morph features of multiple feature dimensions, determine the desired feature similarity among objects, and automatize the generation of multiple objects in 3D object and 2D image formats. We document the use of multidimensional objects in a sequence learning task that embeds objects in a 3D-rendered augmented reality environment controlled by the gaming engine Unity. Together, the toolbox features enable the efficient generation of multidimensional objects with fine control of low-level features and higher-level object similarity useful for visual cognitive research and immersive visual environments.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 8","pages":"219"},"PeriodicalIF":4.6,"publicationDate":"2025-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12226688/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144558858","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Moving beyond button presses to enhance the reliability of congruency tasks.","authors":"Christopher D Erb, Samara Morrison, Alexandra Nicholson-Brown","doi":"10.3758/s13428-025-02740-0","DOIUrl":"10.3758/s13428-025-02740-0","url":null,"abstract":"<p><p>Developmental and individual differences in conflict processing are commonly assessed with button-press measures of response time and accuracy in congruency tasks such as the Eriksen flanker task. However, when the reliability of response time and accuracy congruency effects are reported, the measures frequently fall short of the reliability levels needed to draw firm conclusions. In Study 1, we re-analyzed data from nine hand-tracking experiments. We found that reaching behavior generated high congruency effect reliabilities across different age groups (children, young adults, and older adults) in arrow versions of the flanker task, with split-half reliabilities often above .85 in response time, initiation time, movement time, or movement curvature. Adult performance of the Stroop task revealed congruency effect reliabilities ranging from .76 in movement curvature to .89 in initiation time. In contrast, the Simon task revealed a good reliability estimate in movement curvature for children (.84) and acceptable estimates in response time (.72), movement time (.76), and curvature (.75) for adults. Study 2 tested the reliability of a release-and-press version of the arrow flanker task using a keyboard in one condition and a touchscreen in another condition (N = 51). The keyboard condition generated high congruency effect reliability estimates in response time (.91), initiation time (.94), and movement time (.88), while the touchscreen condition generated high-reliability estimates in response time (.85) and initiation time (.94), and a lower estimate for movement time (.78). These results indicate that the reliability of congruency tasks can be enhanced by moving beyond traditional button-press measures of performance.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 8","pages":"215"},"PeriodicalIF":4.6,"publicationDate":"2025-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12222348/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144551831","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Comparing Likert and visual analogue scales in ecological momentary assessment.","authors":"Jonas M B Haslbeck, Alberto Jover Martínez, Anne J Roefs, Eiko I Fried, Lotte H J M Lemmens, Esmee Groot, Peter A Edelsbrunner","doi":"10.3758/s13428-025-02706-2","DOIUrl":"10.3758/s13428-025-02706-2","url":null,"abstract":"<p><p>Measuring subjective experiences in ecological momentary assessment (EMA) studies has become pervasive in psychological science. A design choice that has to be made in all of these studies is which response scale to use. However, to date there is little guidance on this choice in the context of EMA. As a first step towards understanding the effects of different response scales, we experimentally vary the response scale and assess whether the resulting time series of subjective experiences are systematically different. We conducted a between-person experiment comparing a seven-point Likert scale ( <math><mrow><mi>n</mi> <mo>=</mo> <mn>63</mn></mrow> </math> ) with a Visual Analogue Scale (VAS; <math><mrow><mi>n</mi> <mo>=</mo> <mn>56</mn></mrow> </math> ) in an EMA study measuring affective states over 14 days. Using Bayesian multilevel models, we found that the VAS resulted in moderately higher within-person item means, lag-0 correlations, lag-1 autocorrelations, as well as lower within-person skewnesses and response frequencies of exact zeros. We found the largest difference in correlations with external criteria related to psychopathology, where correlations for the VAS were much higher. We did not observe reliable differences in within-person item variances, root mean squared successive differences, missing data, duration of measurements, and ratings about the experiences with the EMA survey. Apart from higher within-person means and higher correlations with external criteria in the VAS group, the differences were relatively small. While more research on response scales in EMA is needed, based on our results we conclude that the VAS should be preferred in studies aiming at capturing affective states relating to general psychopathology, as well as for items whose variation occurs close to scale limits. We conclude by discussing how our findings may contribute to a larger research agenda that addresses the fit of different response scales for different research aims.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 8","pages":"217"},"PeriodicalIF":4.6,"publicationDate":"2025-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12222338/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144551830","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The fundamentals of eye tracking part 5: The importance of piloting.","authors":"Roy S Hessels, Diederick C Niehorster, Marcus Nyström, Richard Andersson, Gijs A Holleman, Ignace T C Hooge","doi":"10.3758/s13428-025-02737-9","DOIUrl":"10.3758/s13428-025-02737-9","url":null,"abstract":"<p><p>The goal of this article is to demonstrate the importance of pilot studies in empirical eye-tracking research. First, we show what can go wrong when proper pilot experiments are omitted for all phases of an eye-tracking study, from testing an experiment, conducting the data collection, to building, revising, and interpreting the data analysis. Second, we describe a series of eye-tracking studies as a case study, and elaborate on all the pilot experiments that were conducted. We highlight what was learned from each pilot experiment when conceiving, designing, and conducting the research. Finally, we give practical advice for eye-tracking researchers on planning and conducting pilot experiments. This advice can be summarized as (1) take enough time, (2) be problem-oriented, (3) pilots are of an iterative nature, (4) many questions are empirical, and (5) apply the four-eyes principle. We envision that the present article helps early career researchers discover, and more established researchers rediscover, the utility of pilot experiments.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 8","pages":"216"},"PeriodicalIF":4.6,"publicationDate":"2025-07-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12222426/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144551832","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A cognitive diagnosis model for disengaged behaviors.","authors":"Benjamin Lugu, Wenjing Guo, Wenchao Ma","doi":"10.3758/s13428-025-02734-y","DOIUrl":"https://doi.org/10.3758/s13428-025-02734-y","url":null,"abstract":"<p><p>Cognitive diagnosis assessments are frequently used for formative purposes. Due to the low-stakes nature of these assessments, students may exhibit disengaged behaviors, such as rapid guessing and item omissions. Most existing studies in cognitive diagnosis models assume that item responses are reflections of students' proficiency without considering their engagement levels. This study proposes a disengaged behavior cognitive diagnosis model (DB-CDM) that accounts for both disengaged and engaged behaviors simultaneously. We examined the performance of the DB-CDM through simulation and empirical studies. The simulation showed that the item parameters of the DB-CDM were recovered well, especially when the sample size was large and the proportion of disengaged students was small. The DB-CDM can also accurately identify disengaged students, even under some unfavorable conditions involving a large number of disengaged students. By comparing DB-CDM with the compensatory reparameterized unified model in terms of attribute classifications, we observed that the DB-CDM yielded similar if not higher attribute classifications. In the real data analysis, we found that engaged students had a lower probability of omission and guessing and a higher probability of exhibiting solution behavior compared to disengaged students. This paper provides some initial evidence to support the use of DB-CDM when disengaged behaviors occur.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 8","pages":"213"},"PeriodicalIF":4.6,"publicationDate":"2025-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144537950","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A validation study on the accuracy and precision of gaze and vergence using stereoscopic eye-tracking technology.","authors":"Arthur R Pijpaert, H H L M Jeroen Goossens, Bob W van Dijk, L J Bert Roetman, Ruth M A van Nispen, Laurentius J René van Rijn","doi":"10.3758/s13428-025-02731-1","DOIUrl":"10.3758/s13428-025-02731-1","url":null,"abstract":"<p><p>Binocular video-based eye-tracking allows for gaze and vergence measurements, but the accuracy and precision of vergence are barely addressed. Here, we investigate the accuracy and precision of both gaze and vergence measurements using a stereoscopic eye-tracking system. Previous studies have evaluated stereoscopic eye-trackers for gaze eccentricities up to 16°. We validated a custom-built stereoscopic eye-tracker with two cameras and two infrared light sources for gaze eccentricities up to 21°. Additionally, we studied the impact of fixation distance and pupil size on vergence accuracy. Participants with normal binocular vision (N = 8) performed fixation tasks, enabling the assessment of both gaze and vergence errors. The stereoscopic system provided gaze estimates with a mean absolute error (MAE) of less than 1° within the central visual field. However, the accuracy decreased for peripheral angles larger than 14°. We found a MAE of 0.89 ± 0.58° in measuring vergence and a strong linear association between target vergence and measured vergence, with a slope of 0.99 ± 0.05. In contrast to previous studies using single-camera eye-trackers, we found no systematic influence of pupil size on the vergence measurements. Although there was high agreement between estimated and ground truth vergence in the central field, the system did struggle to maintain accuracy at larger eccentricities. This limitation arises primarily from the loss of reliable glints rather than technical constraints, indicating the need for alternative approaches to enhance accuracy in wider fields of view.</p>","PeriodicalId":8717,"journal":{"name":"Behavior Research Methods","volume":"57 8","pages":"214"},"PeriodicalIF":4.6,"publicationDate":"2025-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12213999/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144537951","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}