Survey practicePub Date : 2025-01-01Epub Date: 2025-03-02DOI: 10.29115/sp-2024-0029
Sharan Sharma, Bijay Chouhan, O P Sharma, Jaya Koti, Sonalde Desai
{"title":"Can I Interview Her? Gatekeeping in a Telephone Survey of Female Migrants in India.","authors":"Sharan Sharma, Bijay Chouhan, O P Sharma, Jaya Koti, Sonalde Desai","doi":"10.29115/sp-2024-0029","DOIUrl":"10.29115/sp-2024-0029","url":null,"abstract":"<p><p>This paper explores 'gatekeeping', the phenomenon where access to a sample person in the household is controlled by another person. Gatekeeping of female persons is especially an issue in societies governed by gendered social norms. It can increase survey error by reducing response rates and potentially increasing non-response bias, and can increase measurement error when gatekeepers insist on providing proxy responses. We contribute to the sparse literature on gatekeeping by using data from a telephone survey of migrants in India and focusing on the sample of female married-out migrants. We estimate the prevalence of gatekeeping, compare outcomes of calls made to gatekept versus non-gatekept cases, and model the likelihood of a proxy interview. We estimate that gatekeeping of females in telephone surveys is in the range of 56%-65%, with gatekept calls much more likely to result in proxy interviews. We find that older and more-educated women are less subject to gatekeeping and that male and married interviewers are associated with a greater likelihood of gatekeeping. We discuss implications of these results for survey practice.</p>","PeriodicalId":74893,"journal":{"name":"Survey practice","volume":"19 ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12362564/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144982023","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Survey practicePub Date : 2024-07-17DOI: 10.29115/sp-2024-0009
Ádám Stefkovics, Jan Karem Höhne
{"title":"Examining the Link between the ‘Middle Means Typical’ Heuristic and Answer Behavior","authors":"Ádám Stefkovics, Jan Karem Höhne","doi":"10.29115/sp-2024-0009","DOIUrl":"https://doi.org/10.29115/sp-2024-0009","url":null,"abstract":"Question interpretation in web surveys may not only depend on the textual content but also on visual design aspects. Research has shown that respondents seem to make use of interpretative heuristics when answering questions potentially influencing their answer behavior. In this study, we investigate the implications of the ‘middle means typical’ (MMT) heuristic, which suggests that respondents perceive the middle option of a scale as the most typical one. For this purpose, we use data from a survey experiment embedded in the probability-based German Internet Panel (N = 4,679) varying the inclusion of a non-substantive “Don’t know” option (with or without separation from the substantive options) and scale polarity (unipolar or bipolar). The four questions under investigation were adopted from the Big5 inventory dealing with agreeableness and openness. The results suggest that the MMT heuristic has a minor impact on answer behavior, as the separation of non-substantive options did not affect answer distributions and response times (as a measure of response effort). However, scale polarity influenced answer behavior and response times. Similar to what has been observed in previous studies, unipolar scales elicited more middle answers and bipolar scales elicited more positive answers. Bipolar scales also resulted in longer response times. Although design violations against the MMT heuristic do not seem to impact answer behavior, we still recommend exercising caution when designing scales with non-substantive options. We also highlight the necessity of testing scales differing with respect to polarity.","PeriodicalId":74893,"journal":{"name":"Survey practice","volume":"172 ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141828599","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Survey practicePub Date : 2024-05-23DOI: 10.29115/sp-2024-0005
Rachel A. Gordon, Amelia R. Branigan, M. Khan, Johanna G. Nunez
{"title":"Best Practices for Measuring Skin Color in Surveys","authors":"Rachel A. Gordon, Amelia R. Branigan, M. Khan, Johanna G. Nunez","doi":"10.29115/sp-2024-0005","DOIUrl":"https://doi.org/10.29115/sp-2024-0005","url":null,"abstract":"Surveys that assess skin color support evidence building about colorism and related systemic inequalities that affect health and wellbeing. Methodologists have increasing choices for such assessments, including a growing array of digital images for rating scales and increasingly cost-effective handheld mechanical devices based on color science. Guidance is needed for choosing among these growing options. We used data from a diverse sample of 102 college students to produce new empirical evidence and practical guidance about various options. We compared three handheld devices that ranged in price, considering variations in their reliabilities and how their results differed by where on the body and with what device settings readings were taken. We also offered evidence regarding how reliably interviewers and participants could choose from a large array of color swatches offering variation in skin undertone (redness, yellowness) in addition to skin shade (lightness-to-darkness). Overall, the results were promising, demonstrating that modern handheld devices and rating scales could be feasibly and reliably used. For instance, results demonstrated that just one or two device readings were needed at any given location, and, the device readings and rating scale scores similarly captured the relative darkness of skin. In other cases, recommendations were less certain. For instance, skin undertones of redness and yellowness were more sensitive to device choices and body locations. We encourage future studies that pursue why such variability exists and for which substantive questions it matters most.","PeriodicalId":74893,"journal":{"name":"Survey practice","volume":"38 50","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141103729","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Survey practicePub Date : 2024-05-09DOI: 10.29115/sp-2024-0006
Ting Yan, Hanyu Sun, Anil Battalahalli
{"title":"Applying Machine Learning to Survey Question Assessment","authors":"Ting Yan, Hanyu Sun, Anil Battalahalli","doi":"10.29115/sp-2024-0006","DOIUrl":"https://doi.org/10.29115/sp-2024-0006","url":null,"abstract":"Sun and Yan (2023) described a Computer-Assisted Recorded Interviewing (CARI) Machine Learning (ML) pipeline that efficiently processes 100% of recorded interviews as quickly as possible and as inexpensively as possible. The CARI ML pipeline leads to automatic identification of recordings that are at a higher risk of being falsified or exhibiting undesirable interviewer behaviors. This paper describes an extension to the pipeline that can be used to automatically detect survey questions at a higher risk of poor performance. A proof-of-concept study was conducted and showed that the enhanced pipeline was able to detect worst performing items judged by experts. The results demonstrated the potential of the enhanced pipeline to screen and select problematic items for conventional behavior coding and to improve the efficiency of using CARI for question evaluation and testing.","PeriodicalId":74893,"journal":{"name":"Survey practice","volume":" 10","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140997613","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Survey practicePub Date : 2024-04-10DOI: 10.29115/sp-2023-0032
William L. Rosenberg
{"title":"Processes and Challenges Associated with Conducting Survey Research in Conflict Zones","authors":"William L. Rosenberg","doi":"10.29115/sp-2023-0032","DOIUrl":"https://doi.org/10.29115/sp-2023-0032","url":null,"abstract":"A wide variety of organizations domestically and internationally conduct surveys in conflict zones. Their timely findings are essential for policy makers. Survey research in conflict zones is an essential tool for policy makers in government, humanitarian, private sector, and non-governmental organizations to develop and evaluate programs, services, and policies. Such research is often even more challenging in less developed areas of the world due to issues related to access, technology, and staffing. Other challenges include population movements, lack of societal data structures such as a recent census, communication systems such as telephones, as well as issues related to sampling, design, and the omnipresent issue of security for the researchers and respondents. Often, researchers must make trade-offs regarding internal and external validity which are tempered by issues related to cost, speed, and accuracy. Policy makers need to feel comfortable in the reliability and the generalizability of the findings they receive or develop. The data collected in conflict zones must strive to use appropriate techniques to quantify the data and evaluate the results. Furthermore, flexibility and adaptability are important concepts to consider when conducting research in conflict zones.","PeriodicalId":74893,"journal":{"name":"Survey practice","volume":"15 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-04-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140717797","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Survey practicePub Date : 2024-02-01DOI: 10.29115/sp-2024-0002
Zachary Neal
{"title":"Large sampling errors when using the Unmatched Count Technique to estimate prevalence: A simulation study","authors":"Zachary Neal","doi":"10.29115/sp-2024-0002","DOIUrl":"https://doi.org/10.29115/sp-2024-0002","url":null,"abstract":"The Unmatched Count Technique (UCT) is a method for ensuring respondent anonymity and thereby providing an unbiased estimate of the prevalence of a characteristic in a population. I illustrate that under realistic conditions UCT estimates can have ten times more sampling error than estimates derived from direct questions, and that UCT estimates can take nonsensical negative values. Therefore, the UCT should be used with caution.","PeriodicalId":74893,"journal":{"name":"Survey practice","volume":"34 2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139889992","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Survey practicePub Date : 2024-02-01DOI: 10.29115/sp-2024-0002
Zachary Neal
{"title":"Large sampling errors when using the Unmatched Count Technique to estimate prevalence: A simulation study","authors":"Zachary Neal","doi":"10.29115/sp-2024-0002","DOIUrl":"https://doi.org/10.29115/sp-2024-0002","url":null,"abstract":"The Unmatched Count Technique (UCT) is a method for ensuring respondent anonymity and thereby providing an unbiased estimate of the prevalence of a characteristic in a population. I illustrate that under realistic conditions UCT estimates can have ten times more sampling error than estimates derived from direct questions, and that UCT estimates can take nonsensical negative values. Therefore, the UCT should be used with caution.","PeriodicalId":74893,"journal":{"name":"Survey practice","volume":"264 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139830259","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Survey practicePub Date : 2024-01-04DOI: 10.29115/sp-2023-0019
Zoe Padgett, Sam Gutierrez, Laura Wronski, Soubhik Barari
{"title":"Measuring the Growth of Gender-Inclusive Surveys Around the World","authors":"Zoe Padgett, Sam Gutierrez, Laura Wronski, Soubhik Barari","doi":"10.29115/sp-2023-0019","DOIUrl":"https://doi.org/10.29115/sp-2023-0019","url":null,"abstract":"As ideas about gender identity evolve, survey researchers around the world are working to understand how best to measure sex and gender in a way that is both accurate and inclusive. Emerging best practices differ widely between countries based on cultural and societal norms and the construction of language around gender. In this paper, we examine how survey creators have changed how they ask about gender in the past decade across 11 linguistically and culturally diverse countries. We measure the number of answer options included in gender questions created by SurveyMonkey users between 2012 and 2022. Our findings show that the number of gender questions with more than two answer options increased in all countries examined in our research. Canada, the United Kingdom, and Australia show the highest levels of gender questions with more than two answer options in 2022, while Egypt and Nigeria have the lowest levels.","PeriodicalId":74893,"journal":{"name":"Survey practice","volume":"62 4","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-01-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139386908","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Survey practicePub Date : 2023-11-09DOI: 10.29115/sp-2023-0021
Dan Cassino
{"title":"Symposium on Adding a Middle Eastern or North African category to the US Census","authors":"Dan Cassino","doi":"10.29115/sp-2023-0021","DOIUrl":"https://doi.org/10.29115/sp-2023-0021","url":null,"abstract":"","PeriodicalId":74893,"journal":{"name":"Survey practice","volume":" 38","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135293247","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}