Pathways and Structures: Evaluating Systems Changes in an NSF INCLUDES Alliance

IF 1.1 3区社会学 Q2 SOCIAL SCIENCES, INTERDISCIPLINARY

American Journal of Evaluation Pub Date : 2022-01-05 DOI:10.1177/10982140211041606

C. Howley, Johnavae Campbell, Kimberly S. Cowley, Kimberly Cook

{"title":"Pathways and Structures: Evaluating Systems Changes in an NSF INCLUDES Alliance","authors":"C. Howley, Johnavae Campbell, Kimberly S. Cowley, Kimberly Cook","doi":"10.1177/10982140211041606","DOIUrl":null,"url":null,"abstract":"In this article, we reflect on our experience applying a framework for evaluating systems change to an evaluation of a statewide West Virginia alliance funded by the National Science Foundation (NSF) to improve the early persistence of rural, first-generation, and other underrepresented minority science, technology, engineering, and mathematics (STEM) students in their programs of study. We begin with a description of the project and then discuss the two pillars around which we have built our evaluation of this project. Next, we present the challenge we confronted (despite the utility of our two pillars) in identifying and analyzing systems change, as well as the literature we consulted as we considered how to address this difficulty. Finally, we describe the framework we applied and examine how it helped us and where we still faced quandaries. Ultimately, this reflection serves two key purposes: (1) to consider a few of the challenges of measuring changes in systems, and (2) to discuss our experience applying one framework to address these issues.","PeriodicalId":51449,"journal":{"name":"American Journal of Evaluation","volume":"43 1","pages":"632 - 646"},"PeriodicalIF":1.1000,"publicationDate":"2022-01-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"American Journal of Evaluation","FirstCategoryId":"90","ListUrlMain":"https://doi.org/10.1177/10982140211041606","RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"SOCIAL SCIENCES, INTERDISCIPLINARY","Score":null,"Total":0}

引用次数: 1

Abstract

In this article, we reflect on our experience applying a framework for evaluating systems change to an evaluation of a statewide West Virginia alliance funded by the National Science Foundation (NSF) to improve the early persistence of rural, first-generation, and other underrepresented minority science, technology, engineering, and mathematics (STEM) students in their programs of study. We begin with a description of the project and then discuss the two pillars around which we have built our evaluation of this project. Next, we present the challenge we confronted (despite the utility of our two pillars) in identifying and analyzing systems change, as well as the literature we consulted as we considered how to address this difficulty. Finally, we describe the framework we applied and examine how it helped us and where we still faced quandaries. Ultimately, this reflection serves two key purposes: (1) to consider a few of the challenges of measuring changes in systems, and (2) to discuss our experience applying one framework to address these issues.

查看原文本刊更多论文

路径和结构：评估NSF联盟中的系统变化

在这篇文章中，我们回顾了我们的经验，将评估系统变化的框架应用于由美国国家科学基金会（NSF）资助的西弗吉尼亚州全州联盟的评估，以提高农村、第一代和其他代表性不足的少数族裔科学、技术、工程和数学（STEM）学生在其学习项目中的早期坚持性。我们首先对项目进行描述，然后讨论我们对该项目进行评估的两个支柱。接下来，我们介绍了我们在识别和分析系统变化方面面临的挑战（尽管我们的两个支柱很有用），以及我们在考虑如何解决这一困难时查阅的文献。最后，我们描述了我们应用的框架，并考察了它是如何帮助我们的，以及我们在哪些方面仍然面临困境。最终，这种反思有两个关键目的：（1）考虑衡量系统变化的一些挑战，以及（2）讨论我们应用一个框架来解决这些问题的经验。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

American Journal of Evaluation SOCIAL SCIENCES, INTERDISCIPLINARY-

CiteScore

4.40

自引率

11.80%

发文量

期刊介绍： The American Journal of Evaluation (AJE) publishes original papers about the methods, theory, practice, and findings of evaluation. The general goal of AJE is to present the best work in and about evaluation, in order to improve the knowledge base and practice of its readers. Because the field of evaluation is diverse, with different intellectual traditions, approaches to practice, and domains of application, the papers published in AJE will reflect this diversity. Nevertheless, preference is given to papers that are likely to be of interest to a wide range of evaluators and that are written to be accessible to most readers.