American Journal of Evaluation最新文献

{"title":"Corrigendum to Evaluator Education Curriculum: Which Competencies Ought to be Prioritized in Master's and Doctoral Programs?","authors":"","doi":"10.1177/10982140221126774","DOIUrl":"https://doi.org/10.1177/10982140221126774","url":null,"abstract":"","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42784676","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evaluating Youth Empowerment: The Construction and Validation of an Inventory of Dimensions and Indicators","authors":"Anna Planas-Lladó, Xavier Úcar","doi":"10.1177/10982140211055643","DOIUrl":"https://doi.org/10.1177/10982140211055643","url":null,"abstract":"Empowerment is a concept that has become increasingly used over recent years. However, little research has been undertaken into how empowerment can be evaluated, particularly in the case of young people. The aim of this article is to present an inventory of dimensions and indicators of youth empowerment. The article describes the various phases in the construction and validation of the inventory. These phases were (1) a contrast of the inventory of dimensions and indicators against specialized published writings on youth empowerment; (2) the validation of the resulting inventory by experts; and (3) a contrast with young people through four participatory evaluation processes and six life stories. The tool is scientifically and practically useful and enables the impact of youth empowerment programmes to be evaluated; it also serves to plan and implement socio-educational processes aimed at influencing the empowerment of young people.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45886326","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Statistical Power for Detecting Moderation in Partially Nested Designs","authors":"Kyle Cox, Ben Kelcey","doi":"10.1177/1098214020977692","DOIUrl":"https://doi.org/10.1177/1098214020977692","url":null,"abstract":"Analysis of the differential treatment effects across targeted subgroups and contexts is a critical objective in many evaluations because it delineates for whom and under what conditions particular programs, therapies or treatments are effective. Unfortunately, it is unclear how to plan efficient and effective evaluations that include these moderated effects when the design includes partial nesting (i.e., disparate grouping structures across treatment conditions). In this study, we develop statistical power formulas to identify requisite sample sizes and guide the planning of evaluations probing moderation under two-level partially nested designs. The results suggest that the power to detect moderation effects in partially nested designs is substantially influenced by sample size, moderation effect size, and moderator variance structure (i.e., varies within groups only or within and between groups). We implement the power formulas in the R-Shiny application PowerUpRShiny and demonstrate their use to plan evaluations.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41597293","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Developing Evaluation Approaches for an Anti-Human Trafficking Housing Program","authors":"Rebecca J. Macy, A.L. Eckhardt, Christopher J. Wretman, Ran Hu, Jeongsuk Kim, Xinyi Wang, Cindy Bombeeck","doi":"10.1177/10982140211056913","DOIUrl":"https://doi.org/10.1177/10982140211056913","url":null,"abstract":"The increasing number of anti-trafficking organizations and funding for anti-trafficking services have greatly out-paced evaluative efforts resulting in critical knowledge gaps, which have been underscored by recent recommendations for the development of greater evaluation capacity in the anti-trafficking field. In response to these calls, this paper reports on the development and feasibility testing of an evaluation protocol to generate practice-based evidence for an anti-trafficking transitional housing program. Guided by formative evaluation and evaluability frameworks, our practitioner-researcher team had two aims: (1) develop an evaluation protocol, and (2) test the protocol with a feasibility trial. To the best of our knowledge, this is one of only a few reports concerning anti-trafficking housing program evaluations, particularly one with many foreign-national survivors as evaluation participants. In addition to presenting evaluation findings, the team herein documented decisions and strategies related to conceptualizing, designing, and conducting the evaluation to offer approaches for future evaluations.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47023787","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Transforming the Paradigm for LGBTQ+ Evaluation: Advancing a Praxis of LGBTQ+ Inclusion and Liberation in Evaluation","authors":"Gregory Phillips, D. Felt, Esrea Pérez-Bill, Megan M. Ruprecht, Erik Elías Glenn, Peter Lindeman, R. Miller","doi":"10.1177/10982140211067206","DOIUrl":"https://doi.org/10.1177/10982140211067206","url":null,"abstract":"Lesbian, gay, bisexual, transgender, queer, intersex, Two-Spirit, and other sexual and gender minority (LGBTQ+) individuals encounter numerous obstacles to equity across health and healthcare, education, housing, employment, and other domains. Such barriers are even greater for LGBTQ+ individuals who are also Black, Indigenous, and People of Color (BIPOC), as well as those who are disabled, and those who are working-class, poor, and otherwise economically disadvantaged, among other intersecting forms of oppression. Given this, an evaluation cannot be equitable for LGBTQ+ people without meaningfully including our experiences and voices. Unfortunately, all evidence indicates that evaluation has systematically failed to recognize the presence and value of LGBTQ+ populations. Thus, we propose critical action steps and the articulation of a new paradigm of LGBTQ+ Evaluation. Our recommendations are grounded in transformative, equitable, culturally responsive, and decolonial frameworks, as well as our own experiences as LGBTQ+ evaluators and accomplices. We conclude by inviting others to participate in the articulation and enactment of this new paradigm.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-06-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43246592","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Systematic Review of Meta-Evaluations: Lessons for Evaluation and Impact Analysis","authors":"J. Qian-Khoo, K. Hiruy, Rebecca Hutton, Jo Barraket","doi":"10.1177/10982140211018276","DOIUrl":"https://doi.org/10.1177/10982140211018276","url":null,"abstract":"Impact evaluation and measurement are highly complex and can pose challenges for both social impact providers and funders. Measuring the impact of social interventions requires the continuous exploration and improvement of evaluation approaches and tools. This article explores the available evidence on meta-evaluation—the “evaluation of evaluations”—as an analytical tool for improving impact evaluation and analysis in practice. It presents a systematic review of 15 meta-evaluations with an impact evaluation/analysis component. These studies, taken from both the scholarly and gray literature, were analyzed thematically, yielding insights about the potential contribution of meta-evaluation in improving the methodological rigor of impact evaluation and organizational learning among practitioners. To conclude, we suggest that meta-evaluation is a viable way of examining impact evaluations used in the broader social sector, particularly market-based social interventions.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-06-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43935959","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"From the Interim Co-Editors","authors":"J. Hall, Laura R. Peck","doi":"10.1177/10982140221098626","DOIUrl":"https://doi.org/10.1177/10982140221098626","url":null,"abstract":"After the untimely passing of the American Journal of Evaluation’s (AJE’s) Editor-in-Chief, George Julnes, the two of us—Jori Hall and Laura Peck—agreed to step in and serve as Interim Co-Editors-in-Chief while the American Evaluation Association (AEA) secured a new, permanent lead for our journal. We are grateful to George and Rachael Lawrence, AJE’s most recent Managing Editor, for ushering through the publication process of the six articles and the teaching note that appear in this issue. The articles reflect the diversity of the field of program evaluation with attention to the concept of the counterfactual (Reichardt), evaluation policy (Kinarsky & Christie), program dosage in evaluation (Hewawitharana et al.), policy advocacy (Albert et al.), evaluation capacity (Hudib & Cousins), qualitative data collection and analysis (LaChenaye & McCarthy), and evaluation competencies (Montrosse-Moorhead et al.). In addition, this issue presents an “In Memoriam” section, dedicated to and reflecting on the scholarly life of George Julnes. In our experiences with George, we believed him to be an exceptional editor for our journal because of his passion for pushing the boundaries of the field of program evaluation. He truly valued the diversity of approaches and perspectives that operate in our field and aimed to ensure that all of those approaches and perspectives earned attention in our journal. It is for that reason that he established, for example, the Experimental Methodology Section (which Laura edits) and the Economic Evaluation Section (which Brooks Bowden edits); and that he reconceptualized the Ethics, Values and Culture Section (formerly known as professional values and ethics, which Jill Anne Chouinard and Fiona Cram edit). He diversified the journal’s editorial team to ensure global representation of the many varied parts of our field, as represented by the inclusion of the International Developments Section (which Deborah Rugg and Zenda Ofir edit) and the appointment of Apollo Nkwake as an Associate Editor to bring more attention to evaluation scholars and practitioners in the Global South. George maintained the Method Note (which Tarek Azzam and Dana Wanzer edit), Teaching and Learning (which Anne Vo and Phung Pham edit), and Book Review (which Leslie Cooksy edits) Sections as having ongoing importance. In addition to Jori, Leah Neubauer, Gregory Phillips, II and Justus Randolph served as Associate Editors with George, and we have been grateful for their continued service during this transition. Upon his first “From the Editor” note, kicking off volume 40, George stated his aspirations for the journal. He desired AJE to be “(1) a top source for the most important and relevant information for members of the evaluation community and (2) an influential voice representing the expertise and values of evaluators in policy discussions that affect the evaluation community” (Julnes 2019, 158). It is our hope—during our time as Interim Co-Editors-in-C","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47802016","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Conversations with George","authors":"S. Rallis","doi":"10.1177/10982140221078750","DOIUrl":"https://doi.org/10.1177/10982140221078750","url":null,"abstract":"George Julnes has long been a friend and colleague to whom I often turned for provocative, engaging, theoretically grounded, complex (albeit convoluted at times), and fun conversations. I could count on George to question any point and to offer alternative perspectives. Thus, when I became editor of the American Journal of Evaluation (AJE) in 2014, I asked George to serve as editor of the section that we decided to title Professional Ethics and Values. A few words from what George wrote introducing the section in December 2014 illustrate his ability to raise critical questions:","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48119815","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Remembering George Julnes: My Friend, Colleague, and Pragmatic Theorist","authors":"D. Rog","doi":"10.1177/10982140221078747","DOIUrl":"https://doi.org/10.1177/10982140221078747","url":null,"abstract":"","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44903239","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Teaching Evaluation Through Community-Engaged Learning Courses","authors":"Corrie B. Whitmore","doi":"10.1177/10982140221100448","DOIUrl":"https://doi.org/10.1177/10982140221100448","url":null,"abstract":"This paper describes a framework for educating future evaluators and users of evaluation through community-engaged, experiential learning courses and offers practical guidance about how such a class can be structured. This approach is illustrated via a reflective case narrative describing how an introductory, undergraduate class at a mid-size, public university in the northwest partnered with a community agency. In the class, students learned and practiced evaluation principles in the context of a Parents as Teachers home visiting program, actively engaged in course assignments designed to support the program's evaluation needs, and presented meta-evaluative findings and recommendations for future evaluation work to the community partner to conclude the semester. This community-engaged approach to teaching evaluation anchors student learning in an applied context, promotes social engagement, and enables students to contribute to knowledge about effective human action, as outlined in the American Evaluation Association's Mission.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43901136","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}