L. J. Williams, Aaron R. Williams, Ernest H. O’Boyle
{"title":"Assessment of Path Model Fit: Evidence of Effectiveness and Recommendations for use of the RMSEA-P","authors":"L. J. Williams, Aaron R. Williams, Ernest H. O’Boyle","doi":"10.1177/10944281221124946","DOIUrl":"https://doi.org/10.1177/10944281221124946","url":null,"abstract":"We review the development of path model fit measures for latent variable models and highlight how they are different from global fit measures. Next, we consider findings from two published simulation articles that reach different conclusions about the effectiveness of one path model fit measure (RMSEA-P). We then report the results of a new simulation study aimed at resolving the questions of whether and how the RMSEA-P should be used by organizational researchers. These results show that the RMSEA-P and its confidence interval is very effective with multiple indicator models at identifying misspecifications across large and small sample sizes and is effective at identifying true models at moderate to large sample sizes. We conclude with recommendations for how the RMSEA-P can be incorporated along with other information into model evaluation.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":" ","pages":""},"PeriodicalIF":9.5,"publicationDate":"2022-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44950558","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Should Moderated Regressions Include or Exclude Quadratic Terms? Present Both! Then Apply Our Linear Algebraic Analysis to Identify the Preferable Specification","authors":"A. Kalnins","doi":"10.1177/10944281221124945","DOIUrl":"https://doi.org/10.1177/10944281221124945","url":null,"abstract":"Organizational research increasingly tests moderated relationships using multiple regression with interaction terms. Most research does so with little concern regarding curvilinear relationships. But methodologists have established that omitting quadratic terms of correlated primary variables may create false interaction positives (type 1 errors). If dependent variables are generated by the canonical process where fully specified regressions satisfy the Gauss-Markov assumptions, including quadratics solves the problem. But our empirical analysis of published organizational research suggests that dependent variables are often generated by processes where, even with quadratics included, regression analyses will remain Gauss-Markov non-compliant. In such cases, our linear algebraic analysis demonstrates that including quadratics—even those motivated by compelling theory—may exacerbate rather than mitigate the incidence of false interaction positives. The interaction coefficient may substantially change its magnitude and even flip sign once quadratics are included, and not necessarily for the better. We encourage researchers to present two full sets of results when testing moderating hypotheses—one with, and one without, quadratic terms. Researchers should then answer five questions developed here in order to determine the preferable set of results.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":" ","pages":""},"PeriodicalIF":9.5,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45785943","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
S. Certo, Chunhu Jeon, Kristen A. Raney, Wookyung Lee
{"title":"Measuring What Matters: Assessing how Executives Reference Firm Performance in Corporate Filings","authors":"S. Certo, Chunhu Jeon, Kristen A. Raney, Wookyung Lee","doi":"10.1177/10944281221125160","DOIUrl":"https://doi.org/10.1177/10944281221125160","url":null,"abstract":"We know very little about the performance measures executives use to make decisions. To fill this void, we investigate the performance variables that executives reference in corporate filings with the SEC. Our analyses suggest that executives refer to monetary variables (i.e., revenue, profit, and cash flow) in over 98% of filings. In contrast, executives refer to the unitless performance measures scaled by size (i.e., return on assets, return on equity), which are favored by organizational scholars, in less than 15% of filings. We find that this preference for unscaled measures remains across market capitalization and actual firm performance. In other words, even observations with the highest levels of ROA and ROE are more likely to include monetary measures as opposed to ratios. In fact, we find that almost every observation that references ratios also includes monetary measures of firm performance. Stated differently, our findings suggest executives use ratios in addition to—and not instead of—monetary measures. We discuss research opportunities for scholars to further align with the practitioner perspective and to revisit conceptualizations of firm performance.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":" ","pages":""},"PeriodicalIF":9.5,"publicationDate":"2022-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45607947","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Advancing Reproducibility and Accountability of Unsupervised Machine Learning in Text Mining: Importance of Transparency in Reporting Preprocessing and Algorithm Selection","authors":"L. Valtonen, S. Mäkinen, J. Kirjavainen","doi":"10.1177/10944281221124947","DOIUrl":"https://doi.org/10.1177/10944281221124947","url":null,"abstract":"Machine learning (ML) enables the analysis of large datasets for pattern discovery. ML methods and the standards for their use have recently attracted increasing attention in organizational research; recent accounts have raised awareness of the importance of transparent ML reporting practices, especially considering the influence of preprocessing and algorithm choice on analytical results. However, efforts made thus far to advance the quality of ML research have failed to consider the special methodological requirements of unsupervised machine learning (UML) separate from the more common supervised machine learning (SML). We confronted these issues by studying a common organizational research dataset of unstructured text and discovered interpretability and representativeness trade-offs between combinations of preprocessing and UML algorithm choices that jeopardize research reproducibility, accountability, and transparency. We highlight the need for contextual justifications to address such issues and offer principles for assessing the contextual suitability of UML choices in research settings.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":" ","pages":""},"PeriodicalIF":9.5,"publicationDate":"2022-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44146497","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Reviewer Resources: Confirmatory Factor Analysis","authors":"Christopher D. Nye","doi":"10.1177/10944281221120541","DOIUrl":"https://doi.org/10.1177/10944281221120541","url":null,"abstract":"Confirmatory factor analyses (CFA) are widely used in the organizational literature. As a result, understanding how to properly conduct these analyses, report the results, and interpret their implications is critically important for advancing organizational research. The goal of this paper is to summarize the complexities of CFA models and, therefore, to provide a resource for journal reviewers and researchers who are using CFA in their research. The topics covered in this paper include the estimation process, power analyses, model fit, and model modifications, among other things. In addition, this paper concludes with a checklist that summarizes the key points that are discussed and can be used to evaluate future studies that incorporate CFA.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":" ","pages":""},"PeriodicalIF":9.5,"publicationDate":"2022-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47697424","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Construct Development and Validation in Three Practical Steps: Recommendations for Reviewers, Editors, and Authors*","authors":"L. Lambert, Daniel A. Newman","doi":"10.1177/10944281221115374","DOIUrl":"https://doi.org/10.1177/10944281221115374","url":null,"abstract":"We review contemporary best practice for developing and validating measures of constructs in the organizational sciences. The three basic steps in scale development are: (a) construct definition, (b) choosing operationalizations that match the construct definition, and (c) obtaining empirical evidence to confirm construct validity. While summarizing this 3-step process [i.e., Define-Operationalize-Confirm], we address many issues in establishing construct validity and provide a checklist for journal reviewers and authors when evaluating the validity of measures used in organizational research. Among other points, we pay special attention to construct conceptualization, acknowledging existing constructs, improving existing measures, multidimensional constructs, macro-level constructs, and the need for independent samples to confirm construct validity and measurement equivalence across subpopulations.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":" ","pages":""},"PeriodicalIF":9.5,"publicationDate":"2022-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41912909","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yemisi Bolade-Ogunfodun, Lebene Richmond Soga, B. Laker
{"title":"Entwined Positionality and Interpretive Frames of Reference: An Autoethnographic Account","authors":"Yemisi Bolade-Ogunfodun, Lebene Richmond Soga, B. Laker","doi":"10.1177/10944281221111401","DOIUrl":"https://doi.org/10.1177/10944281221111401","url":null,"abstract":"This paper investigates the ethnographic researcher's positionality and its role in sensemaking within the research process. Using autoethnographic data of the first author - a black female West African (Yoruba) scholar in a Western organizational context, we adopt a critical sensemaking approach to make sense of the researcher's field experience. We propose a conceptualization of the researcher's positionality as one that is entwined in the field, being an active interaction of the researcher's formative context with her sensory capabilities. We demonstrate how openness to the researcher's entwined positionality generates interpretive frames of reference and uncovers nuances in the sensemaking process, which widens the scope for reflexivity. We offer a methodological roadmap for engaging entwined positionality in reflexive practice and contribute to the body of research which challenges the idea of the detached researcher; thus, we respond to the growing calls for integrating the elements of a researcher's positionality into research in a way that enhances reflexivity.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":" ","pages":""},"PeriodicalIF":9.5,"publicationDate":"2022-06-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47035015","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"How Rare Is Rare? How Common Is Common? Empirical Issues Associated With Binary Dependent Variables With Rare Or Common Event Rates","authors":"H. Woo, John P. Berns, Pol Solanelles","doi":"10.1177/10944281221083197","DOIUrl":"https://doi.org/10.1177/10944281221083197","url":null,"abstract":"The use of logit and probit models when examining binary dependent variables including those in the form 0/1 (i.e., dummy variables), yes/no, and true/false (hereafter binary DVs) is commonplace. Yet, the appropriateness and effectiveness of such models are challenged when the event rate of a binary DV is rare or common. To better understand the impact on the field of strategy, we undertook a literature review and assessed recently published research in the Strategic Management Journal. We then utilized Monte Carlo simulations with results showing that as event rates become rarer or more common, issues including biased coefficients, standard error inflation, low statistical power to detect significant effects, and model convergence failure increasingly arise. In addition, small sample sizes amplified these empirical issues. Using a strategy example study, we also show how various analytic tools can lead to different findings when empirical models face an extreme event rate with small sample sizes. Based on our findings, we provide step-by-step guidance for strategy researchers going forward.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"1 1","pages":""},"PeriodicalIF":9.5,"publicationDate":"2022-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41573269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Attitude Toward the Color Blue: An Ideal Marker Variable","authors":"B. Miller, Marcia J. Simmering","doi":"10.1177/10944281221075361","DOIUrl":"https://doi.org/10.1177/10944281221075361","url":null,"abstract":"Researchers often turn to post hoc statistical techniques to identify common method variance (CMV) in same source data and one viable option is to use a marker variable. The choice of marker variable is important, yet these variables are difficult to find, primarily because they must be theoretically unrelated to study variables but measured in the same way (e.g., perceptual; on a Likert scale). This manuscript uses scale development best practices to create a marker variable—attitude toward the color blue—that can be applied in a wide variety of social science research. Scale reliability and validity are addressed, discriminant validity with other measures that detect CMV is tested, and the Confirmatory Factor Analysis Marker Technique is applied with this scale. An experiment designed to analyze the effect of the placement of the scale in surveys is reported. Recommendations to researchers for use of this new scale to detect CMV are provided.","PeriodicalId":19689,"journal":{"name":"Organizational Research Methods","volume":"26 1","pages":"409 - 440"},"PeriodicalIF":9.5,"publicationDate":"2022-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44421742","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"管理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}