Journal of multidisciplinary evaluation最新文献

{"title":"Evaluation Competencies and Evaluation Use: Some Reflections","authors":"Soma De Silva","doi":"10.56645/jmde.v19i46.891","DOIUrl":"https://doi.org/10.56645/jmde.v19i46.891","url":null,"abstract":"This article aimed to identify the emphasis on evaluation use within evaluation competency frameworks. A review of evaluation competency frameworks shows an underlying focus on evaluation use in all frameworks. Nevertheless, specific competencies explicitly focusing on evaluation use were incorporated in more recent frameworks, reflecting the increasing attention to evaluation use. A theory of change for evaluation use is proposed depicting the role of competencies of evaluators and critical users. The article argues for more emphasis on directly use related competencies and extending the competencies beyond evaluators to users. It proposes establishing standardized and up-to-date evaluation training in academic institutions to professionalize evaluation and thereby promote the integration of evaluation in development interventions.   \u0000Keywords: competency frameworks, evaluation competencies, use of evaluation, evaluation training, development interventions.","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":"39 4","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138980653","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Reorienting Evaluator Competencies: Learnings from Evaluation Practice During the COVID-19 Pandemic","authors":"Rajib Nandi, Aparajita De","doi":"10.56645/jmde.v19i46.941","DOIUrl":"https://doi.org/10.56645/jmde.v19i46.941","url":null,"abstract":"Background: The Covid-19 Pandemic has emphasized the need for evaluators to reorient their skills based on ongoing learnings from evaluations. As a result, evaluators must possess a variety of competencies to meet the challenges created by such unprecedented circumstances. \u0000Purpose: Using existing literature about evaluators' competencies and the experiences of conducting evaluations during pandemic, this paper proposes a set of competencies that enable an evaluator to collaborate meaningfully with grassroots organizations and co-design evaluations with communities that empower evaluators to think beyond the boundaries of a project intervention towards achieving a larger goal. \u0000Setting: Covid 19 Pandemic and evaluation practice in India \u0000Intervention: NA \u0000Research Design: Case study with interpretive approach \u0000Data Collection and analysis: NA \u0000Findings: The paper suggests that acquiring 'strategic thinking' and 'emotional intelligence and resilience' strengthens evaluators' competencies to be flexible and innovative in evolving evaluation methods.","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":"15 24","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139010473","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Reflections on Required Competencies For Health Systems Monitoring, Evaluation, and Learning In India","authors":"Neethi Rao, Devaki Nambiar","doi":"10.56645/jmde.v19i46.873","DOIUrl":"https://doi.org/10.56645/jmde.v19i46.873","url":null,"abstract":"The movement towards systems thinking and complexity-informed monitoring and evaluation has been ongoing for some time. Covid-19 has accelerated this shift and increased the salience of contextually aware, adaptive forms of evaluation. Drawing from our experience over three years of providing monitoring, evaluation, research and learning (MERL) to a large international philanthropic organization’s health systems portfolio of work in India, we identify three types of evaluator capacities and associated set of competencies that derive from complexity-driven practice in a developing country. \u0000The first type of capacity relates to deriving evidence in contexts where there are very few traditional forms of data. The MERL portfolio related to health systems outcomes of access, financial protection, and equity even as the health programs themselves were evolving in a period of rapid transformation and dynamism of policy priorities and implementation mechanisms. This required an integration of traditional performance indicators drawn from government datasets with qualitative ‘signals’ focused on drivers of change within the system. Assessment of signals in turn required synthesizing different sources of information, including informal channels of obtaining evidence such as partner meetings or government-sponsored events. The evaluating team thus consciously empowered different kinds of researchers with differential expertise with the goal of building a much more pixelated picture. The goal was not to identify a single source of truth but rather a patchwork of validated information where the relevance of different pieces of data were dependent on evolving outcomes of interest.   \u0000The second set of competencies related to the skills required to play a convening role for donors and implementing partners, supporting better understanding of the changing operating context and help inform decision-making by program officers and partners. This involved building and sustaining relationships across different stakeholders at different stages of the project – from proposal development to review. Competencies relating to effective dialogue and developing an understanding of the core interests of international, national and sub-national partners as well as international donors and experts could only be developed iteratively and over time, but this was crucial in a distributed health decision-making ecosystem like India. \u0000The third and final set of competencies relate to operational adaptiveness, while evaluating an ecosystem with few constants. This can be the hardest competency to acquire because it is the farthest from the traditional notions embedded in the training of evaluation scientists and MERL practitioners. We found that the degree of buy-in and commitment to previously agreed upon frames of reference for evaluation can be shifted by changes in personnel or internal organizational structures. These shifts can lead to chain reactions of mismatched expectation","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":"6 2","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138981470","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Rethinking Evaluator Competencies in an Age of Discontinuity – Implications of Inequities, Sustainability and the Pandemic for Training Evaluators: An Introduction to a Special Volume","authors":"Sanjeev Sridharan, Rachael Gibson, April Nakaima","doi":"10.56645/jmde.v19i46.969","DOIUrl":"https://doi.org/10.56645/jmde.v19i46.969","url":null,"abstract":"","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":"19 7","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138980108","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evaluator Education for the Twenty-First Century: The Centrality of Developing Evaluators’ Interpersonal Competencies","authors":"Jean King","doi":"10.56645/jmde.v19i46.877","DOIUrl":"https://doi.org/10.56645/jmde.v19i46.877","url":null,"abstract":"This paper first traces the development of competencies for professional evaluators and identifies interpersonal competencies as a key component of evaluator education in the future. It then outlines two roles for evaluators that may affect the application of interpersonal competencies, discusses four ideas for their development, and briefly addresses the possibility of evaluation capacity building. It concludes with a brief discussion of where and how evaluator training or education might effectively teach these competencies and know that people had learned them.","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":"17 6","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138979060","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Sustainability, Evaluation, and Credentials","authors":"Andy Rowe, J. Uitto","doi":"10.56645/jmde.v19i46.887","DOIUrl":"https://doi.org/10.56645/jmde.v19i46.887","url":null,"abstract":" While there is clear demand for sustainability-ready evaluation in which environmental impacts are integrated, a major gap exists between this desire and reality. Stock takings show that environmental sustainability is rarely addressed by evaluations. For this to happen, a significant adaptation in how evaluation is organized and conducted is needed, and evaluators, commissioners and evaluation users must take an interest in sustainability. Competencies of individual evaluators and firms must be enhanced, but this is not sufficient. Sustainability-ready evaluation requires interdisciplinary competencies and collaboration.","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":"67 12","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138979112","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Building Spaces for Dialogues to Rethink Evaluator Competencies: Lessons from the Webinars Organized by the Evaluation Centre for Complex Health Interventions","authors":"Sanjeev Sridharan, April Nakaima, Rachael Gibson, Claudeth White, A. Kalugampitiya, Randika L. De Mel, Madhuka Liyanagamage, Ian MacDougall","doi":"10.56645/jmde.v19i46.971","DOIUrl":"https://doi.org/10.56645/jmde.v19i46.971","url":null,"abstract":"Background:  There is a need to rethink evaluator competencies given the harsh and paralyzing realities of COVID.  The pandemic was a time where there was a need to balance diverse perspectives given the limited scientific evidence that existed when faced with a genuinely unprecedented time. In the Fall of 2021 (September to October), the Evaluation Centre for Complex Health Interventions in partnership with the Asia Pacific Evaluation Association organized a three-part webinar series in response to the multiple issues that surfaced during COVID-19, and specifically, the implications of the pandemic for rethinking evaluator competencies and evaluator training. The presenters were from multiple countries including India, Canada, USA, UK, and South Africa. \u0000Purpose: The presenters pushed for more responsive evaluation approaches to address inequities and sustainability and for a decolonized approach to knowledge building.  The webinar raised a number of themes that have potential implications for future discussions on evaluator competencies including:  enhancing evaluation contributions to the Sustainable Development Goals (SDGs), the need to rethink evaluation criteria,  the need to embrace and address varieties of uncertainties,  focus on diversity and heterogeneity;  understanding the role of contexts in complex programs and policies;  the need to reconceptualize sustainability;  being more explicit about inequities and vulnerabilities; and the need to pay attention to systems and system dynamics. \u0000Setting:  The webinars were organized by the Evaluation Centre and the Asia Pacific Evaluation Association on a Zoom platform. \u0000Intervention: Not applicable. \u0000Research Design: Not applicable. \u0000Data Collection and Analysis: Not applicable. \u0000Findings: Not applicable.","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":"19 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138979258","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Competencies for Evaluation as a Civic Science","authors":"Thomas Schwandt","doi":"10.56645/jmde.v19i46.859","DOIUrl":"https://doi.org/10.56645/jmde.v19i46.859","url":null,"abstract":"This paper relocates the practice of evaluation from its traditional intellectual home in the applied social sciences to an interdisciplinary intellectual community that draws on concepts and practices from civic studies, political science, and studies of coproduction and citizen engagement in public administration and management. It offers an overview of the competencies for evaluation practice once relocated in this way.","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":"16 9","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138979718","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Developing the Wisdom of a Mindfulness Competency","authors":"Michael A. Harnar","doi":"10.56645/jmde.v19i46.979","DOIUrl":"https://doi.org/10.56645/jmde.v19i46.979","url":null,"abstract":"","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":"31 8","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138980022","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Fostering Values-Driven Sustainability Through an Ex-Post Capacities Lens","authors":"Jindra Cekan/ova","doi":"10.56645/jmde.v19i46.875","DOIUrl":"https://doi.org/10.56645/jmde.v19i46.875","url":null,"abstract":"Monitoring and evaluation (M&E) work is driven by an array of values held by funders, implementers, M&E experts, and the project participants themselves. Some are explicit, such as embedding equity or democratization values into aid projects. Some are assumed, such as the truth of “values-neutral” evaluation or that long-term sustainability will result from projects. The author espouses Quinn Patton’s “activist interventionist change-committed evaluation” in pushing for ex-post evaluation of much development aid programming’s untested hypothesis about the sustainability of results without proof post-project. Programming approaches can make development results more sustained and can be monitored and evaluated. One set of activities is the participatory involvement of national and local stakeholders from the onset and during all stages of programming. The activism includes M&E experts pushing to include listening to local participants and partners via mixed-methods evaluations, especially by hiring local evaluators, and asking not only about donor-expected results but also about locally emerging outcomes, along with sharing learning at all levels, from local to donors. Focusing on the sustainability of outcomes and impacts involves those doing M&E and those involved in explicit sustainability planning from design, with national project stakeholders at every step. This chapter focuses on a roadmap of capacities and tools needed to foster sustainability pre-exit drawn from a decade of ex-post project closure evaluations. Current evaluators can help improve development practice and the durability of results by honing their capacities to evaluate the natural system on which most projects rest and how to foster resilience to climate change via M&E. Once this is built, transformations toward locally-driven development become possible.","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":"207 3","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139010165","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}