Field MethodsPub Date : 2024-09-14DOI: 10.1177/1525822x241280574
Francisco Olivos, Minhui Liu
{"title":"ChatGPTest: Opportunities and Cautionary Tales of Utilizing AI for Questionnaire Pretesting","authors":"Francisco Olivos, Minhui Liu","doi":"10.1177/1525822x241280574","DOIUrl":"https://doi.org/10.1177/1525822x241280574","url":null,"abstract":"The rapid advancements in generative artificial intelligence have opened new avenues for enhancing various aspects of research, including the design and evaluation of survey questionnaires. However, the recent pioneering applications have not considered questionnaire pretesting. This article explores the use of GPT models as a useful tool for pretesting survey questionnaires, particularly in the early stages of survey design. Illustrated with two applications, the article suggests incorporating GPT feedback as an additional stage before human pretesting, potentially reducing successive iterations. The article also emphasizes the indispensable role of researchers’ judgment in interpreting and implementing AI-generated feedback.","PeriodicalId":48060,"journal":{"name":"Field Methods","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2024-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142252672","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Field MethodsPub Date : 2024-08-01Epub Date: 2023-08-09DOI: 10.1177/1525822x231193311
Htay-Wah Saw, Brady T West, Mick P Couper, William G Axinn
{"title":"What predicts willingness to participate in a follow-up panel study among respondents to a national web/mail survey?","authors":"Htay-Wah Saw, Brady T West, Mick P Couper, William G Axinn","doi":"10.1177/1525822x231193311","DOIUrl":"10.1177/1525822x231193311","url":null,"abstract":"<p><p>The American Family Health Study (AFHS) collected family health and fertility data from a national probability sample of persons aged 18-49 between September 2021 and May 2022, using web and mail exclusively. In July 2022, we surveyed AFHS respondents and gauged their willingness to become part of a national web panel that would create novel longitudinal data on these topics. We focus on predictors of willingness to participate, identifying the potential selection bias that this type of approach may introduce. We found that efforts of this type to create a national web panel may introduce potential selection bias in estimates based on the panel respondents, with individuals having higher socio-economic status being more cooperative. Thus, alternative recruitment strategies and re-weighting of the subsample may be needed to further reduce selection bias. We present methodological implications of our results, limitations of our approach, and suggestions for further research on this topic.</p>","PeriodicalId":48060,"journal":{"name":"Field Methods","volume":null,"pages":null},"PeriodicalIF":1.1,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11262570/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48213738","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Field MethodsPub Date : 2024-07-26DOI: 10.1177/1525822x241267171
David L. Morgan
{"title":"Invited Review: Collecting Data through Dyadic Interviews: A Systematic Review","authors":"David L. Morgan","doi":"10.1177/1525822x241267171","DOIUrl":"https://doi.org/10.1177/1525822x241267171","url":null,"abstract":"This article reports on a systematic review of journal articles that used dyadic interviews, also known as paired or joint interviews. The two basic formats for these interviews involve either interviewing participants separately or together, plus the additional possibility of using both formats in the same study. A search using the Social Science Citations Index yielded 471 articles that reported on such interviews. Of these articles, 38% reported interviewing the dyad members separately, 55% reported interviewing them together, and 7% used both formats. Over 40% reported on interviews of married couples or equivalents, and an additional 30% of the interviews involved other pairs of family members. Less common pairings involved coworkers, patients and providers, or friends. These results indicate an almost unanimous use of dyadic interviews based on pre-existing relationships and family members in particular, with only limited use as a broader interviewing method.","PeriodicalId":48060,"journal":{"name":"Field Methods","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2024-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141782658","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Field MethodsPub Date : 2024-07-20DOI: 10.1177/1525822x241263508
Hafsteinn Einarsson
{"title":"Offering Web Response as a Refusal Conversion Technique in a Mixed-mode Survey","authors":"Hafsteinn Einarsson","doi":"10.1177/1525822x241263508","DOIUrl":"https://doi.org/10.1177/1525822x241263508","url":null,"abstract":"Survey organizations aiming to improve response rates in the later stages of fieldwork often attempt refusal conversions. However, reestablishing contact with units that have refused participation at prior stages of fieldwork may prove costly and time consuming. In this article, the potential of using a refusal conversion procedure in a single contact is investigated. In a mixed-mode survey of young adults in Iceland, individuals who were contacted by telephone and refused participation were instantly offered the option of self-completing the survey on the web at their time of choice. Results indicate that this procedure can meaningfully improve response rates in a survey of young adults in Iceland. The additional respondents gathered using this procedure were mostly similar to those who responded by using other modes when considering demographic background and responses to survey items.","PeriodicalId":48060,"journal":{"name":"Field Methods","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2024-07-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141738905","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Field MethodsPub Date : 2024-07-20DOI: 10.1177/1525822x241262678
Mireia Bolíbar, Julia Martínez-Ariño, Maria Schiller
{"title":"Network of Categories: A Method to Aggregate Egocentric Network Survey Data into a Whole Network Structure","authors":"Mireia Bolíbar, Julia Martínez-Ariño, Maria Schiller","doi":"10.1177/1525822x241262678","DOIUrl":"https://doi.org/10.1177/1525822x241262678","url":null,"abstract":"We propose a new method for analyzing and visualizing information on a large collection of personal networks to uncover the socio-centric structure of relationships among aggregated actors that we clustered into categories. The network of categories identifies the links between groups of sampled ego actors sharing a given attribute (e.g., “being immigrant organizations”) and abstract attribute-based groups or types of alters (e.g., “any immigrant organization”). This method takes advantage of both whole networks and ego-centered networks analytical approaches since it gains insights into the context’s whole network structure while generalizing the results to a wider population. The method can be applied to data obtained through modified versions of instruments such as position or resource generators, which are easy to collect within standard large- N representative surveys. We present a concrete application of the method to the analysis of immigrant organizations' position in local governance networks in 40 French and German cities.","PeriodicalId":48060,"journal":{"name":"Field Methods","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2024-07-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141738906","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Field MethodsPub Date : 2024-06-24DOI: 10.1177/1525822x241260994
Christopher Antoun, Xin (Rosalynn) Yang, Brady T. West, Ai Rene Ong
{"title":"Developing a Modular Survey App Using Co-design Principles","authors":"Christopher Antoun, Xin (Rosalynn) Yang, Brady T. West, Ai Rene Ong","doi":"10.1177/1525822x241260994","DOIUrl":"https://doi.org/10.1177/1525822x241260994","url":null,"abstract":"The objective of our project was to develop a smartphone app for administering shorter (“modular”) surveys. Given the paucity of research on this topic, we decided to use “co-design” techniques to generate design solutions. To implement these techniques, we recruited respondents in the survey target population to work in small groups and provide design ideas for the app. In this article, we present their input and examine whether we were successful in engaging them to contribute useful ideas. Although participation levels were uneven across the different groups, we found no evidence that group engagement was low overall or declined over the course of the sessions. In addition, the groups generated a diverse set of ideas, some of which informed the final app design. These findings underscore the feasibility of applying co-design principles to survey research, though further investigation is needed with different samples and design topics.","PeriodicalId":48060,"journal":{"name":"Field Methods","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2024-06-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141532494","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Field MethodsPub Date : 2024-05-28DOI: 10.1177/1525822x241252220
Michael C. Robbins, Zhuping (Zoe) Li
{"title":"The Nolan Index: An R Program for Calculating List Resemblances","authors":"Michael C. Robbins, Zhuping (Zoe) Li","doi":"10.1177/1525822x241252220","DOIUrl":"https://doi.org/10.1177/1525822x241252220","url":null,"abstract":"The Nolan Index (NI) is a normed, quantitative measure for comparing the degree of resemblance (similarity or dissimilarity) between free listings with an Excel program for calculating it. This article enhances that effort with the addition of an R program and additional applications. Free-list resemblance measures have been used to investigate and compare historical and cultural connections, ecological relationships, cultural item utilities, practices, knowledge, and values both within and between various groups. Their utility for cultural domain analysis and the theory of cognitive cultural models is also considered.","PeriodicalId":48060,"journal":{"name":"Field Methods","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2024-05-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141169857","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Field MethodsPub Date : 2024-05-16DOI: 10.1177/1525822x241244421
Michelle C. Pasco, A. Roque, Brittany Romanello, Emir Estrada
{"title":"Photovoice: Methodological Insights From a Multi-site Online Design","authors":"Michelle C. Pasco, A. Roque, Brittany Romanello, Emir Estrada","doi":"10.1177/1525822x241244421","DOIUrl":"https://doi.org/10.1177/1525822x241244421","url":null,"abstract":"Photovoice involves respondents taking photographs of their environment to promote critical discussions and reflect on their experiences. Photovoice empowers marginalized communities and serves to reach policymakers. The Arizona Youth Identity Project (AZYIP) used photovoice with an innovative approach in a multisite research design with a large sample size and completely online research implementation using video conferencing, mobile phones, and video messages. We outline our process for other researchers interested in utilizing this dynamic method. We also reflect on the challenges and opportunities of engaging in this research design for future projects.","PeriodicalId":48060,"journal":{"name":"Field Methods","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2024-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140969174","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Field MethodsPub Date : 2024-05-11DOI: 10.1177/1525822x241247441
Chan Zhang, Shuaiying Cao, Minglei Wang, Jiangyan Wang, Lirui He
{"title":"Lexical Similarity in Grid Questions: An Experiment of the Effect of “Same Leading” on Respondent Behaviors","authors":"Chan Zhang, Shuaiying Cao, Minglei Wang, Jiangyan Wang, Lirui He","doi":"10.1177/1525822x241247441","DOIUrl":"https://doi.org/10.1177/1525822x241247441","url":null,"abstract":"Previous research on grid questions has mostly focused on their comparability with the item-by-item method and the use of shading to help respondents navigate through a grid. This study extends prior work by examining whether lexical similarity among grid items affects how respondents answer the questions in an experiment where we manipulated lexical similarity by adding the same leading phrase to the items in a grid (“same leading” condition). Both interpretative heuristics and survey satisficing theory suggest that such a design could increase non-differentiated answers. Based on an experiment using respondents from a Chinese opt-in panel ( n = 2,000+), we confirmed more non-differentiation in the same leading condition (than the control) for low-education respondents, while the effects were not significant for the others. This study adds to the literature that incidental survey design features can make an impact and highlights the need for more research into the effects of wording styles.","PeriodicalId":48060,"journal":{"name":"Field Methods","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2024-05-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140987307","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Field MethodsPub Date : 2024-05-11DOI: 10.1177/1525822x241247444
H. Cigdem Bulut, Okan Bulut, Ashley Clelland
{"title":"A Psychometric Network Analysis Approach for Detecting Item Wording Effects in Self-report Measures across Subgroups","authors":"H. Cigdem Bulut, Okan Bulut, Ashley Clelland","doi":"10.1177/1525822x241247444","DOIUrl":"https://doi.org/10.1177/1525822x241247444","url":null,"abstract":"In this study, we explored psychometric network analysis (PNA) as an alternative method for identifying item wording effects in self-report instruments. We examined the functioning of negatively worded items in the network structures of two math-related scales from the 2019 Trends in International Mathematics and Science Study (TIMSS); Students Like Learning in Mathematics (SLLM); and Students Confident in Mathematics (SCM). We also explored how the negatively worded items functioned in network structures across demographic subgroups. Data were drawn from eight countries that represented diverse levels of math performance and cultural attitudes toward school ( n = 75,972). We found that negatively worded items were distinct from the positively worded items in the SLLM and SCM item networks, and that this effect was consistent across all age- and country-level subgroups. Based on these findings, we recommend PNA as a data-driven approach for detecting wording effects effectively.","PeriodicalId":48060,"journal":{"name":"Field Methods","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2024-05-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140925066","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}