Phi-Yen Nguyen, Joanne E. McKenzie, Daniel G. Hamilton, David Moher, Peter Tugwell, Fiona M. Fidler, Neal R. Haddaway, Julian P. T. Higgins, Raju Kanukula, Sathya Karunananthan, Lara J. Maxwell, Steve McDonald, Shinichi Nakagawa, David Nunan, Vivian A. Welch, Matthew J. Page
{"title":"系统评价者对系统评价复制的看法：一项调查","authors":"Phi-Yen Nguyen, Joanne E. McKenzie, Daniel G. Hamilton, David Moher, Peter Tugwell, Fiona M. Fidler, Neal R. Haddaway, Julian P. T. Higgins, Raju Kanukula, Sathya Karunananthan, Lara J. Maxwell, Steve McDonald, Shinichi Nakagawa, David Nunan, Vivian A. Welch, Matthew J. Page","doi":"10.1002/cesm.12009","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Background</h3>\n \n <p>Replication is essential to the scientific method. It is unclear what systematic reviewers think about the replication of systematic reviews (SRs). Therefore, we aimed to explore systematic reviewers' perspectives on (a) the definition and importance of SR replication; (b) incentives and barriers to conducting SR replication; and (c) a checklist to guide when to replicate an SR.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>We searched PubMed for SRs published from January to April 2021, from which we randomly allocated 50% to this survey and 50% to another survey on data sharing in SRs. We sent an electronic survey to authors of these SRs (<i>n</i> = 4669) using Qualtrics. Quantitative responses were summarized using frequency analysis. Free-text answers were coded using an inductive approach.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>The response rate was 9% (<i>n</i> = 409). Most participants considered “replication of SRs” as redoing an SR (68%) or reanalyzing originally collected data (61%), using the same or similar methods. Participants also considered updating an SR, either one's own (42%) or others (43%), equivalent to replication. Most participants agreed that replication of SRs is important (89%). Although 54% of participants reported having conducted a replication of a SR, only 22% have published a replication within 5 years. Those who published a replication (<i>n</i> = 89) often found their replication supported (47%) or expanded the generalizability of the original review (51%). The most common perceived barriers to replicating SRs were difficulty publishing (75%), less prestige (65%), fewer citations (56%), and less impact on career advancement (55%) compared to conducting an original SR. A checklist to assess the need for replication was deemed useful (79%) and easy to apply in practice (69%) by participants.</p>\n </section>\n \n <section>\n \n <h3> Conclusion</h3>\n \n <p>Reviewers have various perceptions of what constitutes a replication of SRs. Reviewers see replication as important and valuable but perceive several barriers to conducting replications. Institutional support should be better communicated to reviewers to address these perceptions.</p>\n </section>\n </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"1 2","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2023-04-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12009","citationCount":"2","resultStr":"{\"title\":\"Systematic reviewers' perspectives on replication of systematic reviews: A survey\",\"authors\":\"Phi-Yen Nguyen, Joanne E. McKenzie, Daniel G. Hamilton, David Moher, Peter Tugwell, Fiona M. Fidler, Neal R. Haddaway, Julian P. T. Higgins, Raju Kanukula, Sathya Karunananthan, Lara J. Maxwell, Steve McDonald, Shinichi Nakagawa, David Nunan, Vivian A. Welch, Matthew J. Page\",\"doi\":\"10.1002/cesm.12009\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div>\\n \\n \\n <section>\\n \\n <h3> Background</h3>\\n \\n <p>Replication is essential to the scientific method. It is unclear what systematic reviewers think about the replication of systematic reviews (SRs). Therefore, we aimed to explore systematic reviewers' perspectives on (a) the definition and importance of SR replication; (b) incentives and barriers to conducting SR replication; and (c) a checklist to guide when to replicate an SR.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Methods</h3>\\n \\n <p>We searched PubMed for SRs published from January to April 2021, from which we randomly allocated 50% to this survey and 50% to another survey on data sharing in SRs. We sent an electronic survey to authors of these SRs (<i>n</i> = 4669) using Qualtrics. Quantitative responses were summarized using frequency analysis. Free-text answers were coded using an inductive approach.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Results</h3>\\n \\n <p>The response rate was 9% (<i>n</i> = 409). Most participants considered “replication of SRs” as redoing an SR (68%) or reanalyzing originally collected data (61%), using the same or similar methods. Participants also considered updating an SR, either one's own (42%) or others (43%), equivalent to replication. Most participants agreed that replication of SRs is important (89%). Although 54% of participants reported having conducted a replication of a SR, only 22% have published a replication within 5 years. Those who published a replication (<i>n</i> = 89) often found their replication supported (47%) or expanded the generalizability of the original review (51%). The most common perceived barriers to replicating SRs were difficulty publishing (75%), less prestige (65%), fewer citations (56%), and less impact on career advancement (55%) compared to conducting an original SR. A checklist to assess the need for replication was deemed useful (79%) and easy to apply in practice (69%) by participants.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Conclusion</h3>\\n \\n <p>Reviewers have various perceptions of what constitutes a replication of SRs. Reviewers see replication as important and valuable but perceive several barriers to conducting replications. Institutional support should be better communicated to reviewers to address these perceptions.</p>\\n </section>\\n </div>\",\"PeriodicalId\":100286,\"journal\":{\"name\":\"Cochrane Evidence Synthesis and Methods\",\"volume\":\"1 2\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2023-04-10\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12009\",\"citationCount\":\"2\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Cochrane Evidence Synthesis and Methods\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1002/cesm.12009\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Cochrane Evidence Synthesis and Methods","FirstCategoryId":"1085","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/cesm.12009","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Systematic reviewers' perspectives on replication of systematic reviews: A survey
Background
Replication is essential to the scientific method. It is unclear what systematic reviewers think about the replication of systematic reviews (SRs). Therefore, we aimed to explore systematic reviewers' perspectives on (a) the definition and importance of SR replication; (b) incentives and barriers to conducting SR replication; and (c) a checklist to guide when to replicate an SR.
Methods
We searched PubMed for SRs published from January to April 2021, from which we randomly allocated 50% to this survey and 50% to another survey on data sharing in SRs. We sent an electronic survey to authors of these SRs (n = 4669) using Qualtrics. Quantitative responses were summarized using frequency analysis. Free-text answers were coded using an inductive approach.
Results
The response rate was 9% (n = 409). Most participants considered “replication of SRs” as redoing an SR (68%) or reanalyzing originally collected data (61%), using the same or similar methods. Participants also considered updating an SR, either one's own (42%) or others (43%), equivalent to replication. Most participants agreed that replication of SRs is important (89%). Although 54% of participants reported having conducted a replication of a SR, only 22% have published a replication within 5 years. Those who published a replication (n = 89) often found their replication supported (47%) or expanded the generalizability of the original review (51%). The most common perceived barriers to replicating SRs were difficulty publishing (75%), less prestige (65%), fewer citations (56%), and less impact on career advancement (55%) compared to conducting an original SR. A checklist to assess the need for replication was deemed useful (79%) and easy to apply in practice (69%) by participants.
Conclusion
Reviewers have various perceptions of what constitutes a replication of SRs. Reviewers see replication as important and valuable but perceive several barriers to conducting replications. Institutional support should be better communicated to reviewers to address these perceptions.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
