Jakob Ohme, Theo Araujo, L. Boeschoten, Deen Freelon, Nilam Ram, Byron B. Reeves, Thomas N. Robinson
{"title":"Digital Trace Data Collection for Social Media Effects Research: APIs, Data Donation, and (Screen) Tracking","authors":"Jakob Ohme, Theo Araujo, L. Boeschoten, Deen Freelon, Nilam Ram, Byron B. Reeves, Thomas N. Robinson","doi":"10.1080/19312458.2023.2181319","DOIUrl":"https://doi.org/10.1080/19312458.2023.2181319","url":null,"abstract":"","PeriodicalId":47552,"journal":{"name":"Communication Methods and Measures","volume":"1 1","pages":""},"PeriodicalIF":11.4,"publicationDate":"2023-02-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42634187","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Systematic Literature Review of Latent Variable Mixture Modeling in Communication Scholarship","authors":"Colton E. Krawietz, Rudy C. Pett","doi":"10.1080/19312458.2023.2179612","DOIUrl":"https://doi.org/10.1080/19312458.2023.2179612","url":null,"abstract":"ABSTRACT Recently, latent variable mixture modeling has gained traction in many disciplines, given its unique ability to discover unknown groups within a broader population. Indeed, this method assumes that a finite number of mixtures (i.e. unknown groups) exist within the population and can be discovered by evaluating participants’ response patterns to a set of manifest indicators. Despite the intuitive approach, recommendations have been proposed to overcome some methodological concerns associated with latent variable mixture modeling. The primary purpose of this study was to understand the characteristics of latent variable mixture modeling in communication research and to evaluate the extent to which the existing research meets these recommendations. Ninety-five manuscripts published between 2010 and 2022 in 18 communication journals were identified and systematically analyzed. The review found that (1) the use of latent variable mixture modeling has increased; (2) latent class analysis and latent profile analysis are the most common models; and (3) most manuscripts did not meet the proscribed standards for random start values, auxiliary variable procedures, indicator requirements, and missing data procedures. These findings are discussed more in comparison with the proscribed standards. In addition, conceptual and applicable recommendations are provided to improve communication scholarship.","PeriodicalId":47552,"journal":{"name":"Communication Methods and Measures","volume":"17 1","pages":"83 - 110"},"PeriodicalIF":11.4,"publicationDate":"2023-02-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42567266","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Correction","authors":"","doi":"10.1080/19312458.2023.2182983","DOIUrl":"https://doi.org/10.1080/19312458.2023.2182983","url":null,"abstract":"Published in Communication Methods and Measures (Vol. 17, No. 2, 2023)","PeriodicalId":47552,"journal":{"name":"Communication Methods and Measures","volume":"16 4","pages":""},"PeriodicalIF":11.4,"publicationDate":"2023-02-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"50165780","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dror K. Markus, Guy Mor-Lan, Tamir Sheafer, Shaul R. Shenhav
{"title":"Leveraging Researcher Domain Expertise to Annotate Concepts Within Imbalanced Data","authors":"Dror K. Markus, Guy Mor-Lan, Tamir Sheafer, Shaul R. Shenhav","doi":"10.1080/19312458.2023.2182278","DOIUrl":"https://doi.org/10.1080/19312458.2023.2182278","url":null,"abstract":"ABSTRACT As more computational communication researchers turn to supervised machine learning methods for text classification, we note the challenge in implementing such techniques within an imbalanced dataset. Such issues are critical in our domain, where, in many cases, researchers attempt to identify and study theoretically interesting categories that can be rare in a target corpus. Specifically, imbalanced distributions, with a skewed distribution of texts among the categories, can lead to a lengthy and expensive annotation stage, forcing practitioners to sample and label large numbers of texts to train a classification model. In this paper, we provide an overview of the issue, and describe existing strategies for mitigating such challenges. Noting the pitfalls of previous solutions, we then provide a semi-supervised method – Expert Initiated Latent Space Sampling – that complements researcher domain expertise with a systematic, unsupervised exploration of the latent semantic space to overcome such limitations. Utilizing simulations to systematically evaluate our method and compare it to existing approaches, we show that our procedure offers significant advantages in terms of efficiency and accuracy in many classification tasks.","PeriodicalId":47552,"journal":{"name":"Communication Methods and Measures","volume":"17 1","pages":"250 - 271"},"PeriodicalIF":11.4,"publicationDate":"2023-02-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44019352","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Developing an Incivility Dictionary for German Online Discussions – a Semi-Automated Approach Combining Human and Artificial Knowledge","authors":"Anke Stoll, L. Wilms, Marc Ziegele","doi":"10.1080/19312458.2023.2166028","DOIUrl":"https://doi.org/10.1080/19312458.2023.2166028","url":null,"abstract":"ABSTRACT Incivility in online discussions has become an important issue in political communication research. Instruments and tools for the automated analysis of uncivil content, however, are rare, especially for non-English user-generated text. In this study, we present a) an extensive dictionary (DIKI - Diktionär für Inzivilität, English: Dictionary for Incivility) to detect incivility in German-language online discussions, and b) a semi-automated two-step-approach that combines manual content analysis with automated keyword collection using a pre-trained word embedding model. We show that DIKI clearly outperforms comparable dictionaries that have been used as alternative instruments to measure incivility (e.g., the LIWC) as well as basic machine learning approaches to text classification. Further, we provide evidence that pre-trained word embeddings can fruitfully be employed in the explorative phase of creating dictionaries. Still, the manual evaluation of DIKI confirms that detecting complex and context-dependent forms of incivility remains challenging and constant update would be needed to maintain performance. Finally, the detailed documentation of the developing and evaluation process of DIKI may serve as a guideline for further research. We therefore provide DIKI as a freely available instrument that also will be applicable in a web interface for drag-and-drop data analysis (diki.limitedminds.org).","PeriodicalId":47552,"journal":{"name":"Communication Methods and Measures","volume":"17 1","pages":"131 - 149"},"PeriodicalIF":11.4,"publicationDate":"2023-02-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43968197","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yingying Chen, Zhao Peng, Sei-Hill Kim, Chang Won Choi
{"title":"What We Can Do and Cannot Do with Topic Modeling: A Systematic Review","authors":"Yingying Chen, Zhao Peng, Sei-Hill Kim, Chang Won Choi","doi":"10.1080/19312458.2023.2167965","DOIUrl":"https://doi.org/10.1080/19312458.2023.2167965","url":null,"abstract":"ABSTRACT Topic modeling has become an effective tool for communication scholars to explore large amounts of texts. However, empirical studies applying topic modeling often face the critical question of making meaningful theoretical contributions. In this study, we highlighted the importance of theoretical underpinning, the research design, and the methodological details of topic modeling studies. We summarized five normative arguments that address critical issues in theory building and testing, research design, and reliability and validity assessments. Using these normative arguments as criteria, we systematically reviewed 105 communication studies that applied topic modeling. We identified gaps and missed opportunities in previous studies and discussed potential pitfalls for the field.","PeriodicalId":47552,"journal":{"name":"Communication Methods and Measures","volume":"17 1","pages":"111 - 130"},"PeriodicalIF":11.4,"publicationDate":"2023-01-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47252637","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Brahim Zarouali, Theo Araujo, Jakob Ohme, Claes H. de Vreese
{"title":"Comparing Chatbots and Online Surveys for (Longitudinal) Data Collection: An Investigation of Response Characteristics, Data Quality, and User Evaluation","authors":"Brahim Zarouali, Theo Araujo, Jakob Ohme, Claes H. de Vreese","doi":"10.1080/19312458.2022.2156489","DOIUrl":"https://doi.org/10.1080/19312458.2022.2156489","url":null,"abstract":"","PeriodicalId":47552,"journal":{"name":"Communication Methods and Measures","volume":" ","pages":""},"PeriodicalIF":11.4,"publicationDate":"2023-01-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45616538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Conceptualizing and Examining Change in Communication Research.","authors":"Miriam Brinberg, David M Lydon-Staley","doi":"10.1080/19312458.2023.2167197","DOIUrl":"10.1080/19312458.2023.2167197","url":null,"abstract":"<p><p>Communication research often focuses on <i>processes</i> of communication, such as how messages impact individuals over time or how interpersonal relationships develop and change. Despite their importance, these change processes are often implicit in much theoretical and empirical work in communication. Intensive longitudinal data are becoming increasingly feasible to collect and, when coupled with appropriate analytic frameworks, enable researchers to better explore and articulate the types of change underlying communication processes. To facilitate the study of change processes, we (a) describe advances in data collection and analytic methods that allow researchers to articulate complex change processes of phenomena in communication research, (b) provide an overview of change processes and how they may be captured with intensive longitudinal methods, and (c) discuss considerations of capturing change when designing and implementing studies. We are excited about the future of studying processes of change in communication research, and we look forward to the iterations between empirical tests and theory revision that will occur as researchers delve into studying change within communication processes.</p>","PeriodicalId":47552,"journal":{"name":"Communication Methods and Measures","volume":"17 1","pages":"59-82"},"PeriodicalIF":6.3,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10139745/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9404666","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Christina Viehmann, Tilman Beck, Marcus Maurer, Oliver Quiring, Iryna Gurevych
{"title":"Investigating Opinions on Public Policies in Digital Media: Setting up a Supervised Machine Learning Tool for Stance Classification","authors":"Christina Viehmann, Tilman Beck, Marcus Maurer, Oliver Quiring, Iryna Gurevych","doi":"10.1080/19312458.2022.2151579","DOIUrl":"https://doi.org/10.1080/19312458.2022.2151579","url":null,"abstract":"ABSTRACT Supervised machine learning (SML) provides us with tools to efficiently scrutinize large corpora of communication texts. Yet, setting up such a tool involves plenty of decisions starting with the data needed for training, the selection of an algorithm, and the details of model training. We aim at establishing a firm link between communication research tasks and the corresponding state-of-the-art in natural language processing research by systematically comparing the performance of different automatic text analysis approaches. We do this for a challenging task – stance detection of opinions on policy measures to tackle the COVID-19 pandemic in Germany voiced on Twitter. Our results add evidence that pre-trained language models such as BERT outperform feature-based and other neural network approaches. Yet, the gains one can achieve differ greatly depending on the specific merits of pre-training (i.e., use of different language models). Adding to the robustness of our conclusions, we run a generalizability check with a different use case in terms of language and topic. Additionally, we illustrate how the amount and quality of training data affect model performance pointing to potential compensation effects. Based on our results, we derive important practical recommendations for setting up such SML tools to study communication texts.","PeriodicalId":47552,"journal":{"name":"Communication Methods and Measures","volume":"17 1","pages":"150 - 184"},"PeriodicalIF":11.4,"publicationDate":"2022-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46967481","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A New Scale for Measuring Identity Insecurity","authors":"Zachary B. Massey, Ioana A. Cionea","doi":"10.1080/19312458.2022.2144631","DOIUrl":"https://doi.org/10.1080/19312458.2022.2144631","url":null,"abstract":"ABSTRACT Although theory and research have discussed identity insecurity as a factor that influences intercultural communication behaviors, no reliable measure has been presented to capture this construct. This paper reports results from three studies designed to test and validate a new measure of identity insecurity. Study 1 (N = 173) included item generation and exploratory factor analysis, revealing five unidimensional identity insecurity factors, termed individual identity insecurity, public presentation insecurity, dissimilar others insecurity, reactive insecurity, and social identity insecurity. Study 2 (N = 524) confirmed this five-factor structure. Study 3 (N = 807) further examined the structure and construct validity of the identity insecurity scale. Results confirmed the five-factor solution obtained in Study 2 and revealed the measure had good construct, convergent, and discriminant validity. Thus, we conclude that the proposed new measure for identity insecurity has a clear factor structure, strong factor loadings, and good reliability and validity.","PeriodicalId":47552,"journal":{"name":"Communication Methods and Measures","volume":"17 1","pages":"40 - 58"},"PeriodicalIF":11.4,"publicationDate":"2022-11-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48445683","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"文学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}