发布求助
文献互助 智能选刊 最新文献

American Journal of Evaluation最新文献

筛选
英文 中文
The Mechanistic Rewards of Data and Theory Integration for Theory-Based Evaluation 基于理论的评价的数据和理论整合的机制奖励
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2023-01-23 DOI: 10.1177/10982140221122764
Corrado Matta, J. Lindvall, A. Ryve
{"title":"The Mechanistic Rewards of Data and Theory Integration for Theory-Based Evaluation","authors":"Corrado Matta, J. Lindvall, A. Ryve","doi":"10.1177/10982140221122764","DOIUrl":"https://doi.org/10.1177/10982140221122764","url":null,"abstract":"In this article, we discuss the methodological implications of data and theory integration for Theory-Based Evaluation (TBE). TBE is a family of approaches to program evaluation that use program theories as instruments to answer questions about whether, how, and why a program works. Some of the groundwork about TBE has expressed the idea that a proper program theory should specify the intervening mechanisms underlying the program outcome. In the present article, we discuss in what way data and theory integration can help evaluators in constructing and refining mechanistic program theories. The paper argues that a mechanism is both a network of entities and activities and a network of counterfactual relations. Furthermore, we argue that although data integration typically provides information about different parts of a program, it is the integration of theory that provides the most important mechanistic insights.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2023-01-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42816075","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
How Many Cases per Cluster? Operationalizing the Number of Units per Cluster Relative to Minimum Detectable Effects in Two-Level Cluster Randomized Evaluations with Linear Outcomes 每个集群有多少个案例?具有线性结果的两级聚类随机评估中相对于最小可检测效应的每簇单位数的可操作性
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2023-01-23 DOI: 10.1177/10982140221134618
E. Hedberg
{"title":"How Many Cases per Cluster? Operationalizing the Number of Units per Cluster Relative to Minimum Detectable Effects in Two-Level Cluster Randomized Evaluations with Linear Outcomes","authors":"E. Hedberg","doi":"10.1177/10982140221134618","DOIUrl":"https://doi.org/10.1177/10982140221134618","url":null,"abstract":"In cluster randomized evaluations, a treatment or intervention is randomly assigned to a set of clusters each with constituent individual units of observations (e.g., student units that attend schools, which are assigned to treatment). One consideration of these designs is how many units are needed per cluster to achieve adequate statistical power. Typically, researchers state that “about 30 units per cluster” is the most that will yield benefit towards statistical precision. To avoid rules of thumb not grounded in statistical theory and practical considerations, and instead provide guidance for this question, the ratio of the minimum detectable effect size (MDES) to the larger MDES with one less unit per cluster is related to the key parameters of the cluster randomized design. Formulas for this subsequent difference effect size ratio (SDESR) at a given number of units are provided, as are formulas for finding the number of units for an assumed SDESR. In general, the point of diminishing returns occurs with smaller numbers of units for larger values of the intraclass correlation.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2023-01-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42152247","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Anthropology in Evaluation: Free-Listing to Generate Cultural Models 评价中的人类学:生成文化模型的自由清单
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2023-01-22 DOI: 10.1177/10982140221116095
Caitlyn D. Placek, Eric Budzielek, Lillian White, Deanna Williams
{"title":"Anthropology in Evaluation: Free-Listing to Generate Cultural Models","authors":"Caitlyn D. Placek, Eric Budzielek, Lillian White, Deanna Williams","doi":"10.1177/10982140221116095","DOIUrl":"https://doi.org/10.1177/10982140221116095","url":null,"abstract":"Free-listing is a quick, semi-quantitative methodology commonly used by anthropologists to uncover information within a cultural domain. In this method note, we review how anthropologists have used free-listing in a variety of research settings. We then apply the social-ecological framework to describe how free-listing can be used for formative, process, outcome, and impact stages of program evaluation. Each type of evaluation includes a set of example free-list prompts to help researchers begin using this tool. We propose that free-listing is a beneficial data collection method in program evaluation. The free-listing method has identified barriers to treatment within our own work with clients recovering from either substance use disorder (SUD) or opioid use disorder (OUD) and has aided in providing flexible, individualized services. We conclude by providing recommendations for collecting free-list data and demonstrating the ease of computing free-list results by providing instructions and an example on how to analyze free-list responses.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2023-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47428091","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Enabling Young People’s Involvement in Research: Principles for a Family Youth Alcohol and Other Drug Program Evaluation 促进青少年参与研究:家庭、青少年、酒精和其他药物项目评估原则
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-21 DOI: 10.1177/10982140211033653
L. Berends, Horace Wansbrough
{"title":"Enabling Young People’s Involvement in Research: Principles for a Family Youth Alcohol and Other Drug Program Evaluation","authors":"L. Berends, Horace Wansbrough","doi":"10.1177/10982140211033653","DOIUrl":"https://doi.org/10.1177/10982140211033653","url":null,"abstract":"We developed principles drawn from empowerment and realist evaluation theories to guide a family youth alcohol and drug program evaluation, before reflecting on our experience to develop additional principles for future work. The research team included the program clinician and an independent researcher. A purposive sample of young people and one parent took part in interviews. The principles we applied include practical and transformative elements: a collaborative, multidisciplinary team; valuing client perspectives; diffusing power; the belief that programs aim to achieve positive results; capacity building; and sharing information for personal benefit. Additional principles were having young people on collaborative research teams, addressing organizational readiness to embed young people’s involvement in research, accounting for gatekeepers in recruitment processes, and using information communication technology. While principles should be flexibly applied at project level, they can guide project design and encourage the development of organizational systems that support collaborative inquiry.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42038418","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Integrative Evaluation 综合评价
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-20 DOI: 10.1177/10982140221079181
O. Feinstein
{"title":"Integrative Evaluation","authors":"O. Feinstein","doi":"10.1177/10982140221079181","DOIUrl":"https://doi.org/10.1177/10982140221079181","url":null,"abstract":"“Integrative evaluation” is an approach with two main phases: identification of plausible rival hypotheses and integration of rival hypotheses. The first phase may correspond to traditional adversary evaluation, whereas the second phase, that is not included in adversary evaluation, requires integrative thinking which can be applied when they are compatible and complementary. Integrative evaluation may facilitate “evaluative thinking” and contribute to deliberative or integrative democracy.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49306336","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Beyond the Buzzword: A Framework for an Indigenous Relational Evaluation in Traditional Communities in Ghana 超越流行语:加纳传统社区原住民关系评估框架
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-20 DOI: 10.1177/10982140211048459
E. Boadu, I. Ile
{"title":"Beyond the Buzzword: A Framework for an Indigenous Relational Evaluation in Traditional Communities in Ghana","authors":"E. Boadu, I. Ile","doi":"10.1177/10982140211048459","DOIUrl":"https://doi.org/10.1177/10982140211048459","url":null,"abstract":"The notions of indigenous knowledge and cultural philosophies are becoming ubiquitous in many social inquiries and evaluation is no exception. Nonetheless, the pursuit to making evaluation impulses embedded in indigenous philosophies relevant in evaluation activities is yet to succeed. Thus, the article discusses the indigenous relational philosophies, approaches and practices of evaluation. Using qualitative research approaches, the study interviewed 43 indigenous development leaders and other local representatives in three local government areas in Ghana. Utilising evidence synthesis approaches through a triangulation process, the paper realised that indigenous knowledge and other cultural ethos were distinct in community-based development evaluation process. The study grasped that there is an elusive intersection between indigenous and contemporary evaluation paradigms. It was observed that former has principles such as community spirit, mutual trust, self-organisation, relational patterns or networks, “ubuntu” ideals, consensus building, collectiveness inter alia that can complement the latter for effective and efficient evaluation of community development programmes and social policies. The article identified key indigenous elements and other indigenous relational assessment patterns to design an indigenously driven relational evaluation framework. The evaluative competencies embedded in indigenous philosophies are vast, thus, a call for future research is proposed.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47205355","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
The Prevalence of Evaluator Anxiety in Practice: An Empirical Examination 实践中评估者焦虑的患病率：一项实证检验
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-20 DOI: 10.1177/10982140221108665
Jessica Renger, S. Donaldson
{"title":"The Prevalence of Evaluator Anxiety in Practice: An Empirical Examination","authors":"Jessica Renger, S. Donaldson","doi":"10.1177/10982140221108665","DOIUrl":"https://doi.org/10.1177/10982140221108665","url":null,"abstract":"Anxiety is a multifaceted force that can negatively impact the ability of evaluators to succeed in practice. In the evaluation literature, discussions concerning anxiety have primarily been limited to strategies to reduce stakeholder anxiety to encourage positive and productive working relationships with evaluators. This study was among the first to explicitly recognize and empirically assess the anxiety evaluators feel during practice. Using a mixed-methods design, a random sample of 109 American Evaluation Association members was surveyed. There was overwhelming agreement (96.30%) that evaluator anxiety is a commonly experienced phenomenon, often citing reasons such as lack of experience, imposter syndrome, unreasonable scope, and difficulties with stakeholder interactions as key contributors to increased anxiety. Evaluator anxiety varied across stages of the evaluation process, with some stages eliciting more anxiety than others. Further discussion and training concerning how to reduce evaluator anxiety could be useful in the preparation of evaluators.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44314603","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Evaluation Quality Assessment Frameworks: A Comparative Assessment of Their Strengths and Weaknesses 评估质量评估框架：优缺点的比较评估
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-20 DOI: 10.1177/10982140211062815
A. Pollard, K. Forss
{"title":"Evaluation Quality Assessment Frameworks: A Comparative Assessment of Their Strengths and Weaknesses","authors":"A. Pollard, K. Forss","doi":"10.1177/10982140211062815","DOIUrl":"https://doi.org/10.1177/10982140211062815","url":null,"abstract":"Quality Assessment Frameworks (QAFs) are standardized templates that include pre-defined criteria, open questions, or a combination to assess the quality of evaluations. They support organizations to limit risks associated with poor-quality evaluation. Qualitative research focused on four QAFs shows that five dimensions of quality are assessed within the frameworks that include pre-defined criteria. These are aligned with the criteria that audiences of evaluation identify as important to assessing quality: substantive findings, robust and appropriate methodology, accessibility, inclusive processes and analysis of wider systems. Consequently, these QAFs are likely to support critical assessment of the main dimensions of quality that are relevant to audiences of evaluation. Audiences also note synergies between these five dimensions of quality and appropriate contextualization as crucial to assessing quality. Consequently, the structure and categorization of quality within QAFs which only include pre-defined criteria risks limiting reviewers’ ability to reflect on these synergies and contextualize assessments.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46950588","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A Look at Power Issues in Collaborative Program Evaluations Under Michel Foucault´s Conception of Power-Knowledge 从福柯的权力-知识观看合作项目评估中的权力问题
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-20 DOI: 10.1177/10982140211060311
Maria Alejandra Torres-Cuello, L. Pinzón‐Salcedo
{"title":"A Look at Power Issues in Collaborative Program Evaluations Under Michel Foucault´s Conception of Power-Knowledge","authors":"Maria Alejandra Torres-Cuello, L. Pinzón‐Salcedo","doi":"10.1177/10982140211060311","DOIUrl":"https://doi.org/10.1177/10982140211060311","url":null,"abstract":"Power has always been an element of the program evaluation field, however, it has traditionally been attributed a negative connotation. With that in mind, power is seen as a force possessed to some extent by different stakeholders that can be imposed over others, and by which certain views, ideas, and decisions can be adopted. Given that program evaluation has a political character in which power is always present, we seek to explore a different notion of this concept. In this paper, we intend to take advantage of Michel Foucault's ideas to explore power in the program evaluation field. We illustrate the introduction of these concepts in collaborative evaluations and in doing so, we also explore power as well as power-knowledge relations, adopting a transformative perspective on power. The introduction of these concepts in collaborative evaluation is useful in illustrating how they can be used in non-collaborative evaluations.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47432474","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
How to Train Better in Evaluation: Teaching Landscape and Lessons Learned From Latin America 如何更好地进行评估培训：拉丁美洲的教学景观和经验教训
IF 1.7 3区 社会学
American Journal of Evaluation Pub Date : 2022-12-20 DOI: 10.1177/10982140211059373
Cinthia Josette Arévalo Gross, Pablo Rodríguez-Bilella, C. Olavarría
{"title":"How to Train Better in Evaluation: Teaching Landscape and Lessons Learned From Latin America","authors":"Cinthia Josette Arévalo Gross, Pablo Rodríguez-Bilella, C. Olavarría","doi":"10.1177/10982140211059373","DOIUrl":"https://doi.org/10.1177/10982140211059373","url":null,"abstract":"Evaluation has been expanding as an important discipline in Latin America, yet there are significant challenges regarding capacity development through educational and training initiatives in the region. This paper first analyzes the evolution and state of the field in terms of the teaching of, and training in evaluation in Latin America, with a special focus on young and emerging evaluators. The paper then draws on published literature, interviews with evaluation experts, and the results of a survey with VOPE leaders and a survey focused on young and emerging evaluators to illustrate Latin American evaluators’ challenges and needs in terms of capacity building and training in evaluation. Lastly, this paper provides some examples of innovative activities and strategies that are being put in place by different actors to address training and teaching needs of Latin American evaluators.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":null,"pages":null},"PeriodicalIF":1.7,"publicationDate":"2022-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42706574","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
首页 上一页 下一页 尾页
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整，为了账户安全，请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们：info@booksci.cn Book学术提供免费学术资源搜索服务，方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号：481959085
Book学术官方微信