Evaluation and Program Planning最新文献

{"title":"Impact of a monitoring and evaluation training in 3 PEPFAR-supported countries","authors":"Anna Russell ,&nbsp;Smita Ghosh ,&nbsp;Niharika Tiwari ,&nbsp;Clarissa Valdez ,&nbsp;Leigh Tally ,&nbsp;Lindsay Templin ,&nbsp;Danielle Pappas ,&nbsp;Samantha Gross ,&nbsp;Biniyam Eskinder ,&nbsp;Sisay Alemayehu Abayneh ,&nbsp;Elizabeth Kamga ,&nbsp;Colince Keleko ,&nbsp;Spence Lloyd ,&nbsp;Nasim Farach ,&nbsp;Sherri Pals ,&nbsp;Eboni Galloway ,&nbsp;Sadhna Patel ,&nbsp;John Aberle-Grasse","doi":"10.1016/j.evalprogplan.2024.102479","DOIUrl":"10.1016/j.evalprogplan.2024.102479","url":null,"abstract":"<div><h3>Background</h3><p>The second phase of the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR) transitioned from scaling up HIV prevention and treatment to promoting sustainability and capacity building for programs monitoring performance and evaluating key program indicators. We assessed the success of a monitoring and evaluation (M&amp;E) curriculum designed to build capacity in three PEPFAR-supported countries.</p></div><div><h3>Methods</h3><p>We customized M&amp;E trainings based on country-specific epidemic control priorities in Ethiopia, Guatemala, and Cameroon. The M&amp;E curriculum included five modules and three evaluation activities to assess impact: (i) in-person pre-post confidence assessment surveys (CAS), (ii) in-person pre-post knowledge tests (PPKT), and (iii) electronic 6–12 months post-training translating knowledge into practice (TKP) surveys. Pre- and post-training results were compared within and across countries and triangulation with the qualitative data evaluated overall success.</p></div><div><h3>Results</h3><p>Among 188 participants attending M&amp;E trainings, 154 (82 %) responded to CAS and 165 (88 %) participants from Ethiopia and Cameroon completed PPKT. Overall CAS scores between pre- and post-test improved [Score mean difference:1.5–1.9]. PPKT indicated statistically significant knowledge gained. One out of five TKP respondents provided direct application examples from the M&amp;E training.</p></div><div><h3>Conclusion</h3><p>While feedback was predominantly positive overall, revisions were recommended for three of the five modules. Developing a customizable and adaptable M&amp;E curriculum may sustain countries’ ability to monitor their progress towards epidemic control.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"108 ","pages":"Article 102479"},"PeriodicalIF":1.5,"publicationDate":"2024-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0149718924000818/pdfft?md5=917aa6a7c123652b45ece7302674c8b7&pid=1-s2.0-S0149718924000818-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142241787","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evaluating food security and nutritional pathways of rural farm families: Empirical evidence from northeast India","authors":"S.T. Pavan Kumar ,&nbsp;Silkame N. Sangma ,&nbsp;Ch. Basanti Devi ,&nbsp;Biswajit Lahiri ,&nbsp;H.G. Kencharaddi ,&nbsp;Jyoti Vastrad","doi":"10.1016/j.evalprogplan.2024.102478","DOIUrl":"10.1016/j.evalprogplan.2024.102478","url":null,"abstract":"<div><p>The study aimed to ascertain a relationship between agricultural status, socioeconomic factors, and nutrition of farm families. The study was conducted in selected villages in the West Garo Hills district of Meghalaya, using Stratified Random Sampling (St. RS). Using pretested interview schedules, we collected primary data from respondents in 2020 and 2021, focusing on socioeconomic variables, body mass index, and income from agriculture and related sectors. The data was analysed using correlation analyses and separate combined regression estimates for each year and month were obtained. Results from the study indicate that agricultural income significantly influenced nutritional status (p &lt; 0.05) and household income growth was also found significant. The region's agricultural production of cereals, pulses, and vegetables was insufficient, as was the production of meat and meat products, milk, and milk products. Hence, expenditure towards purchasing the above food groups from the market was found to be significant (p &lt; 0.05). Therefore, the markets near the mainland especially in the hilly region play a crucial role in the nutritional pathway of rural farm families.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"107 ","pages":"Article 102478"},"PeriodicalIF":1.5,"publicationDate":"2024-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142122902","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Learning from graduate students’ non-formal evaluation experiences in university-based centers","authors":"Devon Charlier ,&nbsp;Suness Hall ,&nbsp;Hannah Kinzer ,&nbsp;John M. LaVelle","doi":"10.1016/j.evalprogplan.2024.102477","DOIUrl":"10.1016/j.evalprogplan.2024.102477","url":null,"abstract":"<div><p>This paper examines graduate research assistants' informal applied learning experiences on the evaluation team at a university-based agricultural safety and health research center. The case study aims to identify the specific learning outcomes derived from the experience, as described by students, and the factors that facilitated them. The research team used a semi-structured focus group tool, and the focus group participants conducted their own analysis and interpretation of the data. An inductive qualitative analysis revealed that students had new perspectives on evaluation, greater evaluation knowledge, and comfort using evaluative tools. Students expressed that key factors like trust, respect, and reflective practice effectively fostered learning. These data were corroborated using deductive analysis using Fink’s Taxonomy of Significant Learning. Ultimately, the study supports the value of informal applied learning experiences for students, particularly in the field of evaluation, and describes a model of support to facilitate learning in various contexts.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"107 ","pages":"Article 102477"},"PeriodicalIF":1.5,"publicationDate":"2024-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142011652","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The impact evaluation of short repeated programs: The case of parenting skills programs","authors":"Daniela Del Boca ,&nbsp;Chiara Daniela Pronzato ,&nbsp;Lucia Schiavon","doi":"10.1016/j.evalprogplan.2024.102476","DOIUrl":"10.1016/j.evalprogplan.2024.102476","url":null,"abstract":"<div><p>In this article, we estimate a series of models to evaluate the effects of a short program in a context where randomization is not feasible but the program follows a cyclical pattern. We will focus empirically on a case study involving parenting skills courses, which have become increasingly popular. In order to evaluate the impact of the program on parents’ and children outcomes, we employ two different methods. The first method compares the outcomes of families who have just finished the program with those who are about to start it; the second compares the outcomes of the same families over time. Furthermore, we propose a model to test whether families who enrolled early were systematically different from those who enrolled later. We find beneficial effects of the program on the importance of living in an area that offers opportunities and of having good quality relationships with friends and family; on the level of self-confidence in sharing one's experiences with other parents; and, in general, on the opinion that tablets and cell phones can be useful for learning, can give parents the opportunity to do something and can calm children. Moreover, we show that the families who access the course early are not random in every respect: they consistently assign higher importance to being well-integrated into a community and having access to culture for their well-being, and eventually, they utilize their time with children differently.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"106 ","pages":"Article 102476"},"PeriodicalIF":1.5,"publicationDate":"2024-08-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0149718924000788/pdfft?md5=075aa3d6601e7c8f3a3bfff3c7f9067a&pid=1-s2.0-S0149718924000788-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141996699","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Public healthcare efficiency in India: Estimates and determinants using two stage DEA approach","authors":"Khursheed Hussain Dar ,&nbsp;Shahid Hamid Raina","doi":"10.1016/j.evalprogplan.2024.102472","DOIUrl":"10.1016/j.evalprogplan.2024.102472","url":null,"abstract":"<div><h3>Background</h3><p>Nearly 400 million Indians (30 % of the total population) lack any financial protection for health which leads to very high out of pocket expenditure. In India more than 90 million people spend 10–25 % of household expenses on healthcare. This is a serious threat to Sustainable Development Goal 3, which aims at providing universal health coverage along with protection from catastrophic spending on health.</p></div><div><h3>Aim</h3><p>The aim of this paper is to estimate the efficiency and determinants of public health in India at subnational level.</p></div><div><h3>Methods</h3><p>To estimate the efficiency of public healthcare, input oriented bias corrected DEA model has been used. In this model life expectancy at birth and infant survival rate have been treated as outputs. Public health spending and per capita income are treated as inputs. In the second stage Tobit regression is used to analyse the determinants of efficiency.</p></div><div><h3>Results</h3><p>The mean bias corrected efficiency score across Indian states is 0.60, implying that on average there is a 40 % inefficiency in public healthcare in India. Maharashtra and Mizoram are the most and least efficient states with efficiency score of 0.921 and 0.218, respectively. Fourteen states have efficiency scores less than 0.60, two states have efficiency score of 0.60 and 15 states have the efficiency score greater than 0.60. Socio economic factors outweigh the medical factors in determining the public healthcare efficiency in India.</p></div><div><h3>Conclusion</h3><p>There are 40 % inefficiencies in public health in India implying significant wastages in public health. By improving public health efficiency, there would be savings of 48 % in terms of government expenditure on health per capita. It will also improve infant survival rate by 27.19 % and life expectancy by 20.65 %.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"106 ","pages":"Article 102472"},"PeriodicalIF":1.5,"publicationDate":"2024-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141990852","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Faculty Online Teaching Effectiveness Scale (FOTES): Instrument development and content validation","authors":"Yea-Jyh Chen ,&nbsp;Yvonne M. Smith ,&nbsp;Andrea Warner Stidham","doi":"10.1016/j.evalprogplan.2024.102474","DOIUrl":"10.1016/j.evalprogplan.2024.102474","url":null,"abstract":"<div><p>This study aimed to develop and psychometrically test the Faculty Online Teaching Effectiveness Scale (FOTES) based upon both student and faculty perspectives of online teaching and learning in higher education. Online teaching effectiveness is a crucial component of quality education, but it has not been well-defined conceptually, and few studies have been conducted, using relevant domains, to accurately measure online teaching effectiveness. The impact of online course delivery on teaching effectiveness remains unclear. An exploratory sequential mixed methods design was employed with three phases of instrument development and psychometric testing. The FOTES comprises 50 items in seven domains: teaching philosophy, self-efficacy, relationships, course content, learning activities, teaching practices, and satisfaction. The instrument underwent initial testing, yielding positive expert appraisals with good-excellent psychometrics. All domains of the scale were significantly correlated, except for teaching philosophy. The preliminary results of the FOTES provide the empirical evidence to advance additional psychometric validation. This newly developed instrument has the potential to enhance faculty capacity and skill in self-evaluating their teaching effectiveness in online courses, providing a valid and reliable measure. The resulting instrument is poised to promote outcome evaluation and strengthen teaching and learning processes.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"106 ","pages":"Article 102474"},"PeriodicalIF":1.5,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141976891","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Using Outcome Harvesting to evaluate socio-economic development and social innovation generated by Social Enterprises in complex areas. The case of BADAEL project in Lebanon","authors":"Gabriele Tomei ,&nbsp;Linda Terenzi ,&nbsp;Enrico Testi","doi":"10.1016/j.evalprogplan.2024.102475","DOIUrl":"10.1016/j.evalprogplan.2024.102475","url":null,"abstract":"<div><p>This paper delves into the utility of Outcome Harvesting as an effective methodology for evaluating development projects that seek to cultivate and advance social enterprises within intricate and multifaceted contexts. Amidst the ever-evolving socio-economic, political, and cultural dynamics characterizing complex environments, traditional evaluation approaches often fall short in capturing the nuanced outcomes of such projects. By adopting Outcome Harvesting, this study argues that evaluators gain a robust tool to systematically collect, analyze, and interpret the intended and unintended impacts of development initiatives in complex and changing contexts. The utilization of mixed methods in the gathering of both primary and secondary data to effectively implement the Outcome Harvesting method has been influenced by overarching theoretical frameworks in the realm of social enterprises. This approach resulted in a comprehensive research methodology that seeks to synergize the strengths of the Outcome Harvesting method within the broader context of understanding and advancing the concept of social entrepreneurship.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"106 ","pages":"Article 102475"},"PeriodicalIF":1.5,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141914267","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Technical efficiency and its determinants in healthcare systems of underdeveloped countries","authors":"Gülizar Gülcan Şeremet ,&nbsp;Murat Konca","doi":"10.1016/j.evalprogplan.2024.102473","DOIUrl":"10.1016/j.evalprogplan.2024.102473","url":null,"abstract":"<div><p>This study aims to evaluate the technical efficiency levels of healthcare systems in underdeveloped countries and to reveal the impact of COVID-19 along with some other factors on the efficiency levels via a two-stage data envelopment analysis. The study covers panel data from 2013 to 2020. The results show that technical efficiency scores decreased during the pandemic period. It is also understood from the results that immunization and the ratio of population ages 65 and above have significant and negative effects on inefficiency scores while carbon dioxide emissions have significant and positive effects. It is thought that the results of this study can guide decision-makers regarding efficiency gains in healthcare systems.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"107 ","pages":"Article 102473"},"PeriodicalIF":1.5,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142021548","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A theory of change evaluation of doctoral admission methods: A case study of a Chinese university","authors":"Lu Zhao","doi":"10.1016/j.evalprogplan.2024.102471","DOIUrl":"10.1016/j.evalprogplan.2024.102471","url":null,"abstract":"<div><p>In response to China's incentive for \"connotative development\" in doctoral education, universities are actively fostering creative and innovative PhD candidates with enhanced research productivity and superior educational quality. A Chinese university, aligning with this objective, has initiated significant reforms in its doctoral admission methods, introducing a new type of student and altering the proportion of existing ones. Recognizing the complexity and uncertainty inherent in the doctoral admission process, this theory-based study employs a Theory of Change framework to systematically evaluate the immediate, intermediate, and long-term outcomes of this university's interventions. It assesses the transformative impact of these changes by integrating secondary data with qualitative insights, including focus group discussions with twelve students and semi-structured interviews with three supervisors. The study reveals that while this university has made substantial progress in achieving short-term goals, these reforms have not uniformly benefited all student categories, presenting distinct challenges and opportunities for stakeholders. Consequently, it advocates for diversifying the student composition and emphasizes inclusive pedagogical strategies to facilitate high-quality doctoral education. Importantly, this research extends beyond this university, promoting a balanced mix of student profiles and a holistic approach to shaping effective policies and practices within doctoral admission, essential for navigating global competition.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"106 ","pages":"Article 102471"},"PeriodicalIF":1.5,"publicationDate":"2024-07-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141898655","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Impact of contextual factors on the evaluation outcomes of national virtual simulation experimental teaching projects in biology and medicine","authors":"Zhengmei Zhou ,&nbsp;Shidi Zhao ,&nbsp;Ye Lin ,&nbsp;Yan Li","doi":"10.1016/j.evalprogplan.2024.102470","DOIUrl":"10.1016/j.evalprogplan.2024.102470","url":null,"abstract":"<div><p>The number of applications for National Virtual Simulation Experimental Teaching Projects (NVSETPs) in China has seen a significant increase. Consequently, the influence of contextual factors and their potential personal connections on the evaluation results, whether for national or non-national NVSETPs, has become a prominent concern. In this study, we employed a modified back-chaining method using logistic regression to examine whether contextual factors in NVSETP applications could explain the evaluation outcomes. Our analysis was based on data available on the open platform of China's Ministry of Education (MOE). We identified several significant influencing factors, including the score on a five-point rating system, the number of clicks on the application page, school quality, school region, and the gender, title, and position of the applicants. Our results shed light on the impact of contextual factors on the evaluation results of NVSETPs in the fields of biology and medicine, using a modified back-chaining method. We conclude that enhancing the transparency of the assessment process and implementing standardized, detailed scoring guidelines for NVSETPs would mitigate the negative influence of contextual factors.</p></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"106 ","pages":"Article 102470"},"PeriodicalIF":1.5,"publicationDate":"2024-07-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141761705","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}