A Scale for Evaluating the Methodological Quality of Studies Based on Observational Methodology.

IF 3.2 2区心理学 Q1 PSYCHOLOGY, MULTIDISCIPLINARY

Psicothema Pub Date : 2024-07-01 DOI:10.70478/psicothema.2025.37.01

Susana Sanduvete-Chaves, Daniel López-Arenas, M Teresa Anguera, Salvador Chacón-Moscoso

{"title":"A Scale for Evaluating the Methodological Quality of Studies Based on Observational Methodology.","authors":"Susana Sanduvete-Chaves, Daniel López-Arenas, M Teresa Anguera, Salvador Chacón-Moscoso","doi":"10.70478/psicothema.2025.37.01","DOIUrl":null,"url":null,"abstract":"Background: To date, no studies have shown validity evidence of a scale that measures the methodological quality of studies based on observational methodology (SBOM). This study presents validity evidence of the Methodological Quality Scale for Studies based on Observational Methodology (MQSOM) based on its internal structure.Method: MQSOM was applied to 650 journal articles that used observational methodology. The Intraclass Correlation Coefficient (ICC) was calculated to obtain evidence of inter- and intracoder reliability. Parallel analysis was done using optimal implementation to study the dimensionality of the scale before conducting an exploratory factor analysis with a randomly-selected half of the sample, followed by a confirmatory factor analysis with the remaining half.Results: Both inter- and intra-coder reliability were adequate, ICC > .73. Parallel analysis suggested a lack of unidimensionality, UniCo = .41; ECV = .31. A second-order factor structure (general factor: Methodological quality) with two first-order factors (F1 Design, F2 Measurement and Analysis) was obtained, RMSEA = 0.000, NNFI = 1, GFI = .98, AGFI = .97, with adequate reliability and discrimination results.Conclusions: MQSOM is a short (11 items), useful instrument for professionals, researchers or assessment commissions when designing, implementing, or evaluating SBOM.","PeriodicalId":48179,"journal":{"name":"Psicothema","volume":"37 1","pages":"1-10"},"PeriodicalIF":3.2000,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Psicothema","FirstCategoryId":"102","ListUrlMain":"https://doi.org/10.70478/psicothema.2025.37.01","RegionNum":2,"RegionCategory":"心理学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PSYCHOLOGY, MULTIDISCIPLINARY","Score":null,"Total":0}

引用次数: 0

Abstract

Background: To date, no studies have shown validity evidence of a scale that measures the methodological quality of studies based on observational methodology (SBOM). This study presents validity evidence of the Methodological Quality Scale for Studies based on Observational Methodology (MQSOM) based on its internal structure.

Method: MQSOM was applied to 650 journal articles that used observational methodology. The Intraclass Correlation Coefficient (ICC) was calculated to obtain evidence of inter- and intracoder reliability. Parallel analysis was done using optimal implementation to study the dimensionality of the scale before conducting an exploratory factor analysis with a randomly-selected half of the sample, followed by a confirmatory factor analysis with the remaining half.

Results: Both inter- and intra-coder reliability were adequate, ICC > .73. Parallel analysis suggested a lack of unidimensionality, UniCo = .41; ECV = .31. A second-order factor structure (general factor: Methodological quality) with two first-order factors (F1 Design, F2 Measurement and Analysis) was obtained, RMSEA = 0.000, NNFI = 1, GFI = .98, AGFI = .97, with adequate reliability and discrimination results.

Conclusions: MQSOM is a short (11 items), useful instrument for professionals, researchers or assessment commissions when designing, implementing, or evaluating SBOM.

查看原文本刊更多论文

基于观察方法学的研究方法学质量评价量表。

背景：到目前为止，还没有研究显示出有效的证据来衡量基于观察方法学（SBOM）的研究的方法学质量。本研究从观察方法学研究方法学质量量表的内部结构出发，提出了该量表的效度证据。方法：采用观察法对650篇期刊论文进行MQSOM分析。计算类内相关系数（ICC）以获得类间和类内信度的证据。在对随机选择的一半样本进行探索性因素分析之前，使用优化实现进行平行分析，以研究量表的维度，然后对剩余的一半样本进行验证性因素分析。结果：编码器间和编码器内的可靠性良好，ICC >；收。平行分析显示缺乏单维性，UniCo = .41；Ecv = 0.31。得到一个二阶因子结构（一般因子：方法学质量），有两个一阶因子（F1 Design, F2 Measurement and Analysis）， RMSEA = 0.000, NNFI = 1, GFI = 0.98, AGFI = 0.97，具有足够的信度和判别结果。结论：MQSOM是专业人员、研究人员或评估委员会在设计、实施或评估SBOM时使用的一个简短（11个条目）的有用工具。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Psicothema PSYCHOLOGY, MULTIDISCIPLINARY-

CiteScore

6.50

自引率

16.70%

发文量

审稿时长

24 weeks

期刊介绍： La revista Psicothema fue fundada en Asturias en 1989 y está editada conjuntamente por la Facultad y el Departamento de Psicología de la Universidad de Oviedo y el Colegio Oficial de Psicólogos del Principado de Asturias. Publica cuatro números al año. Se admiten trabajos tanto de investigación básica como aplicada, pertenecientes a cualquier ámbito de la Psicología, que previamente a su publicación son evaluados anónimamente por revisores externos. Psicothema está incluida en las bases de datos nacionales e internacionales más relevantes, entre las que cabe destacar Psychological Abstracts, Current Contents y MEDLINE/Index Medicus, entre otras. Además, figura en las listas de Factor de Impacto del Journal Citation Reports. Psicothema es una revista abierta a cualquier enfoque u orientación psicológica que venga avalada por la fuerza de los datos y los argumentos, y en la que encuentran acomodo todos los autores que sean capaces de convencer a los revisores de que sus manuscritos tienen la calidad para ser publicados. Psicothema es una revista de acceso abierto lo que significa que todo el contenido está a disposición de cualquier usuario o institución sin cargo alguno. Los usuarios pueden leer, descargar, copiar, distribuir, imprimir, buscar, o realizar enlaces a los textos completos de esta revista sin pedir permiso previo al editor o al autor, siempre y cuando la fuente original sea referenciada. Para acervos y repositorios, se prefiere que la cobertura se realice mediante enlaces a la propia web de Psicothema. Nos parece que una apuesta decidida por la calidad es el mejor modo de servir a nuestros lectores, cuyas sugerencias siempre serán bienvenidas.