American Journal of Evaluation最新文献

筛选
英文 中文
Research on Evaluation Influence in India: Theorizing Beyond Process and Results to Design 印度评价影响研究:超越过程和结果的设计理论
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-20 DOI: 10.1177/10982140211050912
M. Goodnight
{"title":"Research on Evaluation Influence in India: Theorizing Beyond Process and Results to Design","authors":"M. Goodnight","doi":"10.1177/10982140211050912","DOIUrl":"https://doi.org/10.1177/10982140211050912","url":null,"abstract":"This study analyzes evaluation influence theories to understand their unified contributions to a conceptual framework for research on evaluation influence (RoEI) in non-western contexts. Specifically, these theories are analyzed according to their usefulness for interpreting the consequences of the Annual Status of Education Report (ASER)—a cyclical, large-scale educational monitoring and evaluation effort in India. An ethnographic case study of ASER facilitates an empirical investigation of the field's evaluation influence theories and uncovers a new source of influence—evaluation design. The study's findings challenge a common belief in the field: that evaluation process and results are the two fountainheads of evaluation impact. Design, as a distinct third source of influence, gives rise to a novel form of evaluation consequence—design diffusion. The phenomenon of design diffusion illuminates why ASER's model for evaluation has been adopted by several other Global South nations.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45109498","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Social Justice as Ontology: The Intersection of Black Evaluators’ Identities, Roles, and Practice 作为本体论的社会正义:黑人评价者身份、角色和实践的交汇
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-20 DOI: 10.1177/10982140221108664
A. Boyce, Aileen Reid, Cherie M. Avent, Adeyemo Adetogun, J. R. Moller, Brianna Hooks Singletary, Ph.D Melvin Hall
{"title":"Social Justice as Ontology: The Intersection of Black Evaluators’ Identities, Roles, and Practice","authors":"A. Boyce, Aileen Reid, Cherie M. Avent, Adeyemo Adetogun, J. R. Moller, Brianna Hooks Singletary, Ph.D Melvin Hall","doi":"10.1177/10982140221108664","DOIUrl":"https://doi.org/10.1177/10982140221108664","url":null,"abstract":"Violence, marginalization, oppression, exploitation, erasure, and injustice are cornerstones of the Black experience in the United States. Despite seemingly insurmountable challenges, Black people have worked diligently and competently to earn spaces within the present-day professional arena. While the experiences of Black professionals have been investigated in multiple fields, little is known about Black evaluators. This study aimed to investigate Black evaluators’ experiences in an attempt to understand the intersectionality of their identities, roles, and practice and to ensure voices and contributions of Black evaluators are highlighted. We conducted 26 interviews with Black evaluators across academic, government, nonprofit, philanthropic, and private sectors. We found race and advocate for social justice are central to Black evaluators’ identities, roles, and practice. We developed a thematic framework as a result of our analysis and interpretations of the ways in which Black evaluators’ identities impacted their perceptions of their professional role and practice.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49313083","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
Measuring Evaluator Competencies: Developing and Validating the Evaluator Competencies Assessment Tool 测量评估员能力:开发和验证评估员能力评估工具
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-20 DOI: 10.1177/10982140211056539
Minji Cho, Ann Marie Castleman, Haley Umans, Mike Osiemo Mwirigi
{"title":"Measuring Evaluator Competencies: Developing and Validating the Evaluator Competencies Assessment Tool","authors":"Minji Cho, Ann Marie Castleman, Haley Umans, Mike Osiemo Mwirigi","doi":"10.1177/10982140211056539","DOIUrl":"https://doi.org/10.1177/10982140211056539","url":null,"abstract":"Evaluation scholars have committed decades of work to the development of evaluator competencies. The 2018 American Evaluation Association (AEA) Evaluator Competencies may be useful for evaluators to identify their strengths and weaknesses to improve their practice; however, a few empirically validated self-assessment tools based on the competencies exist. Two studies were conducted to develop a validated tool. The first study (N = 170) developed the Evaluator Competencies Assessment Tool (ECAT), a self-assessment tool based on the AEA, 2018 Evaluator Competencies. This study provided evidence for structural validity via confirmatory factor analysis. The second study (N = 142) reconfirmed structural validity with a new sample and examined variables that are associated with evaluator competencies through correlation and t-test analyses. Having a mentor, years of evaluation experience, age, evaluation training, and education level were positively related to evaluator competencies. The ECAT can be used to foster self-reflection for practitioners to improve evaluation competence.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47459902","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
The Process of Applying Principles-Focused Evaluation to the Sexual Violence Prevention Field: Implications for Practice in Other Social Services Fields 以原则为中心的评估在性暴力预防领域的应用过程:对其他社会服务领域实践的启示
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-20 DOI: 10.1177/10982140211056935
E. Casey, J. Vanslyke, B. Beadnell, N. Tatiana Masters, Kirstin McFarland
{"title":"The Process of Applying Principles-Focused Evaluation to the Sexual Violence Prevention Field: Implications for Practice in Other Social Services Fields","authors":"E. Casey, J. Vanslyke, B. Beadnell, N. Tatiana Masters, Kirstin McFarland","doi":"10.1177/10982140211056935","DOIUrl":"https://doi.org/10.1177/10982140211056935","url":null,"abstract":"Principles focused evaluation (PFE) can complement existing formative and outcome evaluation plans by identifying Effectiveness Principles (EPs), an operationalization of values and standards that guide practitioners during program implementation. To date, however, few examples of PFE are available in the literature. This description of the application of PFE to the Washington State Rape Prevention and Education (RPE) sexual violence prevention program provides an example of how this flexible approach can augment an existing evaluation plan to distill shared evaluation components across different organizations implementing diverse prevention programming. Specifically, we describe the process used by a team of practitioners, funders, evaluation consultants and state-level sexual violence prevention technical assistance providers to identify EPs, operationalize indicators for each EP, and develop and test an EP measurement approach. In this process, the seven very different RPE-funded organizations, each serving a unique community, were able to identify and endorse shared, core EPs. This description illustrates PFE's promise for augmenting a shared evaluation approach and identifying common guiding tenets across uniquely situated organizations in a larger community of practice.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44700944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Fidelity and Adaptation of Programs: Does Adaptation Thwart Effectiveness? 节目的忠实性和适应性:适应性增强有效性吗?
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-20 DOI: 10.1177/10982140221138604
Kate L. Nolt, L. Leviton
{"title":"Fidelity and Adaptation of Programs: Does Adaptation Thwart Effectiveness?","authors":"Kate L. Nolt, L. Leviton","doi":"10.1177/10982140221138604","DOIUrl":"https://doi.org/10.1177/10982140221138604","url":null,"abstract":"Evidence-based programs and grassroots programs are often adapted during implementation. Adaptations are often hidden, ignored, or punished. Although some adaptations stem from lack of organizational capacity, evaluators report other adaptations happen in good faith or are efforts to better fit the local context. Program implementers, facilitators who need to adapt during implementation, do not always report adaptations because they fear losing funding if the program is not implemented with fidelity. Program personnel including program evaluators need this information to improve effectiveness of programs, and to determine whether an adaptation is still consistent with the theory of change. Evaluators also need this information for generalizing results to varied settings and populations. Following the PRECEDE–PROCEED model, we recommend a hybrid approach to fidelity and adaptation. We argue in favor of advance planning to accommodate potential adaptations. Such planning also establishes evaluation criteria for determining whether adaptations are helpful, harmful, and appropriate to the context. We illustrate some types of adaptations that can occur, why they may be needed, and how to structure transparent reporting about adaptations to program developers and funding organizations.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45018945","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Principles for Stakeholder Involvement in Evaluation in Latin America 拉丁美洲利益攸关方参与评估的原则
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-19 DOI: 10.1177/10982140221123010
Julia Espinosa-Fajardo, Pablo Rodríguez-Bilella, Esteban Tapella
{"title":"Principles for Stakeholder Involvement in Evaluation in Latin America","authors":"Julia Espinosa-Fajardo, Pablo Rodríguez-Bilella, Esteban Tapella","doi":"10.1177/10982140221123010","DOIUrl":"https://doi.org/10.1177/10982140221123010","url":null,"abstract":"In the last three decades, the promotion of stakeholder involvement in evaluation has been gaining relevance in the Latin American and internationally, across varied agencies, institutions, and civic organizations. The 2030 Agenda and the Global Evaluation Agenda have also recognized the centrality of participation in evaluation. This article explores stakeholder involvement in evaluation based on collaborative work with stakeholders from 15 evaluative experiences. It shows what characterizes participatory evaluation in the region today and the principles of this practice.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48929468","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Supporting Evaluation Practice Through Mindfulness 通过正念支持评估练习
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-19 DOI: 10.1177/10982140221116094
J. Pann, E. DiLuzio, A. Coghlan, Scott D. Hughes
{"title":"Supporting Evaluation Practice Through Mindfulness","authors":"J. Pann, E. DiLuzio, A. Coghlan, Scott D. Hughes","doi":"10.1177/10982140221116094","DOIUrl":"https://doi.org/10.1177/10982140221116094","url":null,"abstract":"This article explores the utility of mindfulness in the field of evaluation. Mindfulness is a translation of the ancient Indian word, Sati, which means awareness, attention, and remembering. While definitions vary, a practical definition of mindfulness is present-moment awareness in an open and nonjudgmental manner. Mindfulness-based interventions have been employed by a wide variety of professions. Although it has received limited attention in the writings of evaluators, we argue that mindfulness can improve the practice of evaluation and support the development of the professional practice and interpersonal domains of American Evaluation Association (AEA) evaluator competencies. We review several mindfulness-based practices and how they can be used by evaluators in their work. Thus, we posit that far from being an esoteric concept, mindfulness practices can serve the pragmatic end of improving our discipline. We also discuss the limits of mindfulness and propose recommendations for future efforts.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47726105","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Professionalizing Evaluation: A Time-Bound Comparison of the American Evaluation Association's Foundational Documents 专业化评估:美国评估协会基础文件的时间范围比较
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-04 DOI: 10.1177/10982140221136486
S. Tucker, L. Stevahn, J. King
{"title":"Professionalizing Evaluation: A Time-Bound Comparison of the American Evaluation Association's Foundational Documents","authors":"S. Tucker, L. Stevahn, J. King","doi":"10.1177/10982140221136486","DOIUrl":"https://doi.org/10.1177/10982140221136486","url":null,"abstract":"This article compares the purposes and content of the four foundational documents of the American Evaluation Association (AEA): the Program Evaluation Standards, the AEA Public Statement on Cultural Competence in Evaluation, the AEA Evaluator Competencies, and the AEA Guiding Principles. This reflection on alignment is an early effort in the third step of professionalization: defining how to use and recognize evaluator competencies. The analysis intentionally focuses on content and reflects on the implications of the differences and similarities across documents. The comparison reveals important questions of interest at both the micro level (individual evaluator) and the macro level (evaluation). The article concludes with challenges, learnings, and proposed next steps of AEA's Professionalization and Competencies Working Group.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44055415","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Integrating Validity Evidence to Revise a Child Abuse Knowledge Test for Early Childhood Education Providers: A Mixed Methods Approach. 整合有效性证据,修订针对幼儿教育机构的虐待儿童知识测试:混合方法。
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-01 Epub Date: 2021-11-22 DOI: 10.1177/10982140211002901
Carlomagno C Panlilio, Lisa Famularo, Jessica Masters, Sarah Dore, Nicole Verdiglione, Chengwu Yang, Erik Lehman, Robert M Hamm, Richard Fiene, David Bard, Karl M Kapp, Benjamin H Levi
{"title":"Integrating Validity Evidence to Revise a Child Abuse Knowledge Test for Early Childhood Education Providers: A Mixed Methods Approach.","authors":"Carlomagno C Panlilio, Lisa Famularo, Jessica Masters, Sarah Dore, Nicole Verdiglione, Chengwu Yang, Erik Lehman, Robert M Hamm, Richard Fiene, David Bard, Karl M Kapp, Benjamin H Levi","doi":"10.1177/10982140211002901","DOIUrl":"10.1177/10982140211002901","url":null,"abstract":"<p><p>Knowledge tests used to evaluate child protection training program effectiveness for early childhood education providers may suffer from threats to construct validity given the contextual variability inherent within state-specific regulations around mandated reporting requirements. Unfortunately, guidance on instrument revision that accounts for such state-specific mandated reporting requirements is lacking across research on evaluation practices. This study, therefore, explored how collection and integration of validity evidence using a mixed methods framework can guide the instrument revision process to arrive at a more valid program outcome measure.</p>","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9733792/pdf/nihms-1810124.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10353343","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Empowering Indigenous Communities Through a Participatory, Culturally Responsive Evaluation of a Federal Program for Older Americans. 通过对美国老年人联邦计划的参与性、文化响应性评估赋予土著社区权力。
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-01 DOI: 10.1177/10982140211030557
Gretchen S Clarke, Elizabeth B Douglas, Marnie J House, Kristen E G Hudgins, Sofia Campos, Elizabeth E Vaughn
{"title":"Empowering Indigenous Communities Through a Participatory, Culturally Responsive Evaluation of a Federal Program for Older Americans.","authors":"Gretchen S Clarke,&nbsp;Elizabeth B Douglas,&nbsp;Marnie J House,&nbsp;Kristen E G Hudgins,&nbsp;Sofia Campos,&nbsp;Elizabeth E Vaughn","doi":"10.1177/10982140211030557","DOIUrl":"https://doi.org/10.1177/10982140211030557","url":null,"abstract":"<p><p>This article describes our experience of conducting a 5-year, culturally responsive evaluation of a federal program with Indigenous communities. It describes how we adapted tenets from \"participatory evaluation models\" to ensure cultural relevance and empowerment. We provide recommendations for evaluators engaged in similar efforts. The evaluation included <i>stakeholder engagement</i> through a Steering Committee and an Evaluation Working Group in designing and implementing the evaluation. That engagement facilitated <i>attention to Indigenous cultural values</i> in developing a program logic model and medicine wheel and in gathering local perspectives through storytelling to facilitate understanding of community traditions. Our ongoing assessment of program grantees' needs shaped our approach to <i>evaluation capacity building</i> and development of a diverse array of experiential learning opportunities and user-friendly tools and resources. We present practical strategies from lessons learned during the evaluation design and implementation phases of our project that might be useful for other evaluators.</p>","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/28/a3/10.1177_10982140211030557.PMC9732784.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10330796","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信