American Journal of Evaluation最新文献_第4页

Digital Evaluation Stories: A Case Study of Implementation for Monitoring and Evaluation in an Australian Community not-for-Profit 数字评估故事：澳大利亚非营利社区监测和评估实施案例研究

IF 1.7 3区社会学

American Journal of Evaluation Pub Date : 2023-02-20 DOI: 10.1177/10982140221138031

Samantha Abbato

{"title":"Digital Evaluation Stories: A Case Study of Implementation for Monitoring and Evaluation in an Australian Community not-for-Profit","authors":"Samantha Abbato","doi":"10.1177/10982140221138031","DOIUrl":"https://doi.org/10.1177/10982140221138031","url":null,"abstract":"The merit of narrative film methods to support participatory approaches and professional development has been increasingly demonstrated by research in several fields and education. However, the use of digital storytelling and other film methods in evaluation remains largely unchartered territory. This article provides a case study of a digital storytelling evaluation initiative in monitoring and evaluation (M&E) in an Australian community not-for-profit. The aim is to offer practical insights for evaluators and organizations considering digital storytelling and other film narrative methods for participant-centered evaluation. Embedding digital evaluation stories into M&E evolved through collaboration between the external evaluation team and organizational leadership, requiring capacity building in evaluation, digital and qualitative methods, and new systems and processes. Benefits include transformation into a participant-centered evaluation and learning culture. Several challenges are discussed, including the extent of organizational change required, the associated time, energy, and cost, and the positive bias of visual narratives.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":" ","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-02-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47377110","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Challenges and Lessons Learned Conducting a Formative Evaluation of a Multicomponent Care Delivery Intervention 对多成分护理提供干预进行形成性评估的挑战和经验教训

IF 1.7 3区社会学

American Journal of Evaluation Pub Date : 2023-02-13 DOI: 10.1177/10982140221116096

Rosalind E. Keith, Shannon Heitkamp, J. Little, Victoria Peebles, Rumin Sarwar, Dana M. Petersen, A. O'Malley

{"title":"Challenges and Lessons Learned Conducting a Formative Evaluation of a Multicomponent Care Delivery Intervention","authors":"Rosalind E. Keith, Shannon Heitkamp, J. Little, Victoria Peebles, Rumin Sarwar, Dana M. Petersen, A. O'Malley","doi":"10.1177/10982140221116096","DOIUrl":"https://doi.org/10.1177/10982140221116096","url":null,"abstract":"Formative evaluation provides stakeholders with timely feedback to support an intervention's improvement during implementation to maximize its effectiveness. We describe the qualitative methods that guided one study within a formative evaluation of a multicomponent care delivery intervention. We then describe the challenges and lessons learned that emerged from this study, organizing them by the study's four overarching challenges: (1) addressing multiple research questions, (2) working with a large interdisciplinary team, (3) triangulating qualitative results with quantitative results, and (4) studying implementation in real-world delivery settings. Overall, the evaluation generated important findings to support improvement of the intervention during implementation. We hope that sharing the lessons learned will increase the rigor and efficiency with which formative evaluations of complex care delivery interventions are conducted and the likelihood that they will improve implementation in real time. We also hope the lessons learned will enhance the satisfaction of the researchers working on these evaluations.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":" ","pages":""},"PeriodicalIF":1.7,"publicationDate":"2023-02-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48626015","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The Importance of Context for Determining Causal Mechanisms in Program Evaluation: The Case of Medical Male Circumcision for HIV Prevention Among the Luo in Western Kenya 在项目评估中确定因果机制的背景的重要性:医学男性包皮环切术在肯尼亚西部罗人中预防艾滋病毒的案例

IF 1.7 3区社会学

American Journal of Evaluation Pub Date : 2023-02-13 DOI: 10.1177/10982140211062267

M. Kabare, Jeremy Northcote

{"title":"The Importance of Context for Determining Causal Mechanisms in Program Evaluation: The Case of Medical Male Circumcision for HIV Prevention Among the Luo in Western Kenya","authors":"M. Kabare, Jeremy Northcote","doi":"10.1177/10982140211062267","DOIUrl":"https://doi.org/10.1177/10982140211062267","url":null,"abstract":"The importance of considering wider contexts when evaluating the success or failure of programs has been increasingly acknowledged with the shift towards culturally responsive evaluation. But one of the important advantages of contextual approaches has been mostly overlooked—that they can provide more “realist” evaluations for why programs fail or succeed. The careful identification of causal mechanisms involved in program delivery is important for avoiding spurious conclusions about the effectiveness of programs. Drawing on findings from a mixed-methods study conducted in Western Kenya among the Luo to evaluate the impacts of a HIV prevention program involving voluntary male medical circumcision (VMMC), it is shown that the VMMC program was one of several variables that contributed to the desired outcome, being not so much the cause but a catalyst for accelerating the desired behavioral change that the surrounding context was already amenable to and contributing to, even before the program was introduced. The need for context evaluations is particularly obvious when programs are part of broader campaigns involving scale-up from one context to another.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":"44 1","pages":"221 - 235"},"PeriodicalIF":1.7,"publicationDate":"2023-02-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45108186","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Decolonizing Community Development Evaluation in Rakhine State, Myanmar 缅甸若开邦非殖民化社区发展评估

IF 1.7 3区社会学

American Journal of Evaluation Pub Date : 2023-02-13 DOI: 10.1177/10982140221146140

Leanne M. Kelly, Phyo Pyae Thida (aka Sophia) Htwe

引用次数: 0

Cultural Competence: 10-Year Comparison of Program Evaluators’ Perceptions 文化能力：项目评估者认知的10年比较

IF 1.7 3区社会学

American Journal of Evaluation Pub Date : 2023-02-13 DOI: 10.1177/10982140221122767

Krystall Dunaway, K. Gardner, Karly Grieve

引用次数: 1

The Oral History of Evaluation: The Influence of Edmund Wyatt Gordon on Evaluation 评价的口述历史:埃德蒙·怀亚特·戈登对评价的影响

IF 1.7 3区社会学

American Journal of Evaluation Pub Date : 2023-02-05 DOI: 10.1177/10982140221148432

M. Mark, R. Hopson, Valerie J. Caracelli, R. Miller

{"title":"The Oral History of Evaluation: The Influence of Edmund Wyatt Gordon on Evaluation","authors":"M. Mark, R. Hopson, Valerie J. Caracelli, R. Miller","doi":"10.1177/10982140221148432","DOIUrl":"https://doi.org/10.1177/10982140221148432","url":null,"abstract":"Since 2003, the Oral History Project Team has conducted interviews with individuals who have made substantial contributions to evaluation theory and practice. The previous interviews were conducted with individuals who have a major identification within the field of evaluation and whose professional development has been intertwined with the history of evaluation as a distinct field. Over a similar period some members in the field of evaluation have worked to highlight more of the field’s history, especially in pointing out the contributions of individuals from traditionally underrepresented groups, including those who were early in addressing how perceptions and realities of race and class affect our programs and their evaluations. This is especially the case in educational evaluation, where a “collective ignorance” about the scholarship of African Americans has sparked efforts to more fully represent voices that can enlighten and enrich our scholarship and our recorded history (e.g., Hood, 2001; Hood & Hopson, 2008). In keeping with this endeavor, the present interview extends the previous scope of the oral history project to celebrate the life and work of Dr. Edmund Wyatt Gordon, a leading intellectual in the field of education. Dr. Gordon is a centenarian who remains actively engaged in research at The Edmund W. Gordon Institute for Urban and Minority Education (IUME) within Teachers College at Columbia University. This center, founded by Dr. Gordon in 1974, was renamed in his honor in 2021 to recognize his contributions in educational justice, equity, and education. Long-time members of the Oral History Project Team (Robin Lin Miller, Melvin M. Mark, Valerie J. Caracelli) along with Rodney K. Hopson conducted three interviews with Dr. Gordon between October 2021 and December 2021. The interview transcripts have been combined and edited for clarity, length, and content. Dr. Gordon reviewed and approved the final product prior to its submission to the American Journal of Evaluation.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":"44 1","pages":"175 - 189"},"PeriodicalIF":1.7,"publicationDate":"2023-02-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42021756","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Evaluating Collective Impact Initiatives: A Systematic Scoping Review 评估集体影响计划:系统的范围审查

IF 1.7 3区社会学

American Journal of Evaluation Pub Date : 2023-02-01 DOI: 10.1177/10982140221130266

S. Panjwani, Taylor Graves-Boswell, W. Garney, Daenuka Muraleetharan, Mandy N. Spadine, Sara A Flores

{"title":"Evaluating Collective Impact Initiatives: A Systematic Scoping Review","authors":"S. Panjwani, Taylor Graves-Boswell, W. Garney, Daenuka Muraleetharan, Mandy N. Spadine, Sara A Flores","doi":"10.1177/10982140221130266","DOIUrl":"https://doi.org/10.1177/10982140221130266","url":null,"abstract":"Collective impact (CI) is a structured approach that helps drive multi-sector collaborations to address social problems through systems changes. While the CI approach is gaining popularity, practitioners experience challenges in evaluating its implementation and intended outcomes. We conducted a systematic scoping review to understand evaluation methods specific to CI initiatives, identify challenges or limitations with these evaluations, and provide recommendations for the design of CI evaluations. Eighteen studies met the inclusion criteria. Process evaluations were the most frequently used evaluation design. Most studies collected cross-sectional data to evaluate their efforts. The complexity of CI was most frequently cited as the greatest evaluation challenge. Study recommendations primarily focused on improvements during the evaluation planning phase. Taking careful consideration in the planning of CI evaluations, developing context-specific data collection methods, and communicating results intentionally and effectively could prove useful to sufficiently capture and assess this systems-level approach to address social problems.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":"44 1","pages":"406 - 423"},"PeriodicalIF":1.7,"publicationDate":"2023-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41379577","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Outcome Trajectory Evaluation (OTE): An Approach to Tackle Research-for-Development’s Long-Causal-Chain Problem 结果轨迹评价(OTE):一种解决研发长因果链问题的方法

IF 1.7 3区社会学

American Journal of Evaluation Pub Date : 2023-01-31 DOI: 10.1177/10982140221122771

B. Douthwaite, C. Proietti, V. Polar, G. Thiele

引用次数: 1

The Mechanistic Rewards of Data and Theory Integration for Theory-Based Evaluation 基于理论的评价的数据和理论整合的机制奖励

IF 1.7 3区社会学

American Journal of Evaluation Pub Date : 2023-01-23 DOI: 10.1177/10982140221122764

Corrado Matta, J. Lindvall, A. Ryve

引用次数: 0

How Many Cases per Cluster? Operationalizing the Number of Units per Cluster Relative to Minimum Detectable Effects in Two-Level Cluster Randomized Evaluations with Linear Outcomes 每个集群有多少个案例?具有线性结果的两级聚类随机评估中相对于最小可检测效应的每簇单位数的可操作性

IF 1.7 3区社会学

American Journal of Evaluation Pub Date : 2023-01-23 DOI: 10.1177/10982140221134618

E. Hedberg

{"title":"How Many Cases per Cluster? Operationalizing the Number of Units per Cluster Relative to Minimum Detectable Effects in Two-Level Cluster Randomized Evaluations with Linear Outcomes","authors":"E. Hedberg","doi":"10.1177/10982140221134618","DOIUrl":"https://doi.org/10.1177/10982140221134618","url":null,"abstract":"In cluster randomized evaluations, a treatment or intervention is randomly assigned to a set of clusters each with constituent individual units of observations (e.g., student units that attend schools, which are assigned to treatment). One consideration of these designs is how many units are needed per cluster to achieve adequate statistical power. Typically, researchers state that “about 30 units per cluster” is the most that will yield benefit towards statistical precision. To avoid rules of thumb not grounded in statistical theory and practical considerations, and instead provide guidance for this question, the ratio of the minimum detectable effect size (MDES) to the larger MDES with one less unit per cluster is related to the key parameters of the cluster randomized design. Formulas for this subsequent difference effect size ratio (SDESR) at a given number of units are provided, as are formulas for finding the number of units for an assumed SDESR. In general, the point of diminishing returns occurs with smaller numbers of units for larger values of the intraclass correlation.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":"44 1","pages":"153 - 168"},"PeriodicalIF":1.7,"publicationDate":"2023-01-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42152247","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0