Journal of multidisciplinary evaluation最新文献

{"title":"Taking Evaluation Contexts Seriously: A Cross-Cultural Evaluation in Extreme Unpredictability","authors":"H. Laperrière","doi":"10.56645/jmde.v3i4.77","DOIUrl":"https://doi.org/10.56645/jmde.v3i4.77","url":null,"abstract":"","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49511527","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Beyond GDP: Tracking and Evaluating National Contributions to Social and Environmental Sustainability","authors":"R. Picciotto","doi":"10.56645/jmde.v17i41.717","DOIUrl":"https://doi.org/10.56645/jmde.v17i41.717","url":null,"abstract":"Background: The Gross Domestic Product (GDP) emerged as a convenient measure of national economic activity during the Great Depression. It was subsequently adopted by international development economists to track developing countries’ progress so that, despite its severe deficiencies, it became ‘locked in’ by habit, convenience, and policy makers’ preferences. \u0000Purpose: This article conceives of GDP as a social intervention fit for evaluation. It shows that the GDP has had a pervasive and pernicious influence on policy making. Since past strategies aimed at dethroning the GDP have failed, it proposes new, evaluator-driven approaches designed to undermine the GDP’s dominance in the global market economy. \u0000Setting: The Stiglitz report commissioned in the wake of the 2008 financial crisis launched a ‘Beyond GDP’ movement. Since then, public alarm about the GDP growth addiction has escalated: the drawbacks of GDP as a free-market policy tool have become self-evident as the rich get richer, the ranks of the poor swell and the future of the planet hangs in the balance. \u0000Research Design: Not applicable. \u0000Data Collection and Analysis: For the twenty largest economies in the world, the article estimates climate change discounts to the GDP based on official CO2 emissions statistics and a social cost of carbon estimate derived from a 2015 survey of eminent climatologists. It also draws on composite indexes generated by four reputable social research organizations to rank countries for their contributions to the 5 Ps of the Sustainable Development Goals (SDGs): people, planet, prosperity, peace, and partnership. \u0000Findings: Pending the results of on-going efforts to upgrade worldwide statistics focused on the 169 SDG targets, the proposed GDP discounts help track progress towards the SDGs. But monitoring is not enough. In a policy world dominated by vested interests, the new ‘Beyond GDP’ indicators should be combined with principled, evaluator-directed evaluations. \u0000Keywords: Beyond GDP; climate change; evaluator-directed evaluation; Gross Domestic Product; indicators; Sustainable Development Goals","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48560529","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Choosing an Evaluation Theory: A Supplement to Evaluation Roots (3rd Edition)","authors":"Marvin C. Alkin, Christina A. Christie, Naomi A. Stephen","doi":"10.56645/jmde.v17i41.709","DOIUrl":"https://doi.org/10.56645/jmde.v17i41.709","url":null,"abstract":"Background: Unlike scientific theories, evaluation theories are prescriptive: a set of actions and approaches that should be followed when conducting an evaluation. While evaluation theorists have offered a variety of writings describing their theories and approaches, few have offered a specific outline of what the theory looks like in practice. Thus, Alkin and Christie formulated a book to aid evaluators in how to apply theories in evaluations (Alkin & Christie, forthcoming). This book culminates in a series of prototypes that outline each theory’s goals, appropriate contexts, prescriptions, and observable actions in application. \u0000Purpose: In order to aid evaluators in applying theories, this article seeks to provide a basis for comparison that can be used to help evaluators select which theory would be most appropriate in their practice. \u0000Setting: This comparison can be applied in any setting where evaluations fit the context prescribed by each of the theories. \u0000Intervention: Not applicable. \u0000Research Design: Not applicablre. \u0000Data Collection and Analysis: Not applicable. \u0000Findings: In order for theories to influence practice effectively, theories must be displayed in a way that allows for easy comparison. This comparison of three theory prototypes demonstrates that prototypes can be an effective way for selecting a prescriptive theory when conducting an evaluation. \u0000Keywords: prescriptive theories; practice; empowerment evaluation; learning centered model; developmental evaluation","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42378666","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Complexity-Aware Monitoring and Evaluation","authors":"Tilman Hertz, Eva Brattander, L. Rose","doi":"10.56645/jmde.v17i41.679","DOIUrl":"https://doi.org/10.56645/jmde.v17i41.679","url":null,"abstract":"Background: Addressing today’s sustainability challenges requires adopting a systemic approach where social and ecological systems are treated as integrated social-ecological systems. Such systems are complex, and the international development sector increasingly recognises the need to account for the complexity of the systems that they seek to transform. \u0000Purpose: This paper sketches out the elements of a complexity-aware monitoring and evaluation (M&E) system for international development programmes in the area of sustainable development.  \u0000Setting: Not applicable. \u0000Data Collection and Analysis: The authors draw on existing literature on complexity and evaluation and on their own experience from working in the field of M&E. \u0000Findings: An M&E system should not be seen simply as a tool to track compliance against a pre-determined theory of change. Instead, it is most useful as a real-time approach, constantly defining and re-defining narratives for change that help push systems along trajectories of interest. Dealing with complexity involves embracing uncertainty; and this challenges established notions of accountability—something which funders and implementers must begin to redefine together. \u0000Keywords: monitoring; evaluation; complexity; social-ecological systems; international development programmes; narratives for change; theory of change","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-06-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43042690","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Emergent Developmental Evaluation Developments","authors":"Michael Q. Patton","doi":"10.56645/jmde.v17i41.699","DOIUrl":"https://doi.org/10.56645/jmde.v17i41.699","url":null,"abstract":"Background: Developmental Evaluation (DE) continues to develop. After reviewing the niche and history of DE, this article identifies five new and emergent purposes for DE to add to the original five – and forecasts future developments for diverse, innovative, and adaptive DE applications. An extended example of DE in supporting pivoting in the pandemic illustrates DE as adaptation to crisis as distinct from supporting social innovation. \u0000Purpose: To update DE. \u0000Setting: Global. \u0000Data Collection and Analysis: Not applicable. \u0000Findings: The applications and uses of DE have expanded beyond the original framing. \u0000Keywords: adaptation; developmental evaluation; pivot; social innovation","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44835475","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evaluating the Content of Palestinian Curricula in Light of the Sustainable Development Goals 2030","authors":"Rabiha M. I. Elyan, A. Al-Doulat","doi":"10.56645/jmde.v17i41.687","DOIUrl":"https://doi.org/10.56645/jmde.v17i41.687","url":null,"abstract":"Background: Curriculum evaluation is a dynamic and indispensable process necessary to develop the curriculum, and to support decision-makers with evidences to guide the curriculum towards the intended goals. The Sustainable Development Goals (SDGs) are the most important universal goals to be taken into consideration by curriculum. SDGs represent a call to end poverty, protect the planet and ensure that all people enjoy peace and prosperity by 2030. Because education, through curriculum, is the nations’ key to accelerate the achievement of sustainable development, this study evaluates the content of the Palestinian curriculum and assesses the degree to which SDGs are included in the curriculum. \u0000Purpose: The study aimed to evaluate the Palestinian school curricula, through using national standards for each SDG, to examine the degree to which the SDGs are incorporated in the curricula, and to know whether there is variation in this incorporation among the different curricular subjects. \u0000Setting: The study evaluates the content of Palestinian curricula of the main subjects based on national curriculum standards developed and published in light of the SDGs. These subjects are: Arabic Language for grades 1-12, Science for grades 1-12, Mathematics for grades 1-12, Social Studies for grades 1-12, Technology for grades 5-12, Agricultural Sciences for grades 11-12, Renewable Energy for grades 11-12, Entrepreneurship and Business for grades 11-12, Management and Economy for grades 11-12, and Smart Buildings for grades 11-12. In addition, timeframe delimit is the academic year 2018-2019. \u0000Data Collection and Analysis: In order to evaluate content of the Palestinian curricula, descriptive-analytical methodology was used by utilizing content analysis of the guideline document for each curricular subject. \u0000Findings: The results showed variation in the inclusion of the SDGs, and absence of essential aspects. The fourth SDG (Quality Education) obtained the highest inclusion percentage with 28.5%. While the fourteenth SDG (Life below Water) obtained the lowest inclusion percentage with 0.8%. In light of the results, the study highlighted variation of the curriculum from SDGs and recommended for the development and enrichment of the Palestinian curriculum to ensure the inclusion of SDGs, with its all dimensions, considering that education in Palestine defined as the main gateway towards progressing achievement of the SDGs. \u0000Keywords: the 2030 agenda for sustainable development; content analysis; evaluation; SDGs; standards","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47191481","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Concepts and Contexts of Creative Evaluation Approaches","authors":"Elisavet Christou, Violet Owen, Pinar Ceyhan","doi":"10.56645/jmde.v17i40.675","DOIUrl":"https://doi.org/10.56645/jmde.v17i40.675","url":null,"abstract":"Background: Creative Evaluation (CE) is an unformed constellation of evaluation approaches that is based on varied understandings of creativity. Although creativity in evaluation has been consistently valued and needed by evaluation practitioners and researchers, the use of the term CE is currently limited in its applications as it has been developed inconsistently and often in isolation to the broader evaluation practice. The authors review examples where the term CE is being used, to present this constellation of approaches and group them under concepts and contexts, aiming at fostering an exploration into CE. \u0000Purpose: This paper examines the use of term Creative Evaluation by reviewing a set of examples from varied settings like educational programmes, cultural industries and social work and to present the reader with a better understanding about the contexts and concepts associated with CE. The authors discuss the transdiciplinary nature of evaluation along with a review of historical developments in evaluation theory and practice that help identify the emergence of responsive, flexible and problem-solving driven approaches to evaluation that can showcase a growing connection between creativity and evaluation. \u0000Setting: Not applicable. \u0000Intervention: Not applicable. \u0000Research Design: Desk review was utilised for the preparation of this paper. \u0000Findings: There are different conceptualisations of the term CE applied in a variety of disciplinary and practice contexts. The term CE is used to describe both the process of applying creative thinking and the process of employing creative methods in the evaluation process. Creative thinking indicates a broader application of creativity that shapes the evaluation process and often results in the development of novel methods while creative methods indicate a more targeted use of methods that employ creative practices to achieve a specific evaluation goal like increasing engagement in the evaluation process. \u0000Keywords: creative evaluation; creative methods; creative approaches; transdisciplinary evaluation; evaluation","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41976255","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Developmental Evaluation in Theory versus Practice: Lessons from Three Developmental Evaluation Pilots","authors":"Heather Esper, Y. Fatehi, Rebecca Baylor","doi":"10.56645/jmde.v17i40.685","DOIUrl":"https://doi.org/10.56645/jmde.v17i40.685","url":null,"abstract":"Background. Developmental Evaluation (DE) practitioners turn to DE theory to make design and implementation decisions. However, DE practitioners can experience difficulty in fully understanding how to implement DE using theory because it is method agnostic (Patton, 2016). Instead, DE is a principle-based approach. \u0000Purpose. This article presents an empirical examination of how DE theory was (or was not) applied during three DE pilots. Our analysis aims to better understand how DE theory is used in practice to expand the evidence base and strengthen future DE implementation. \u0000Setting. A consortium of three organizations implemented three DE pilots through the United States Agency for International Development (USAID) from November 2016 to September 2019. The authors—who participated in the consortium—did not implement the DEs but instead conducted a study or meta-evaluation across the DE pilots. \u0000Data Collection and Analysis. This article focuses on the results of an ex post facto analysis of three DE pilots based on the entire DE implementation experience. For each DE studied, we used mixed methods to collect data on the effectiveness of the DE approach, to identify adaptations to strengthen DE implementation in the USAID context, and to measure its value to stakeholders. Data included more than 100 hours of interviews, 465 pages of qualitative data, and 30 surveys completed by DE participants. \u0000Findings. We find that the ability to apply the DE principles in practice is influenced, in no particular order, by DE participant buy-in to the DE, the Developmental Evaluator’s aptitude, support and resources available to the Developmental Evaluator, and the number of DE participants. We also find that buy-in can change and this should be closely monitored throughout a DE to inform whether a DE should be paused or prematurely ended. \u0000Keywords: Developmental Evaluation; developmental evaluator skills; buy-in; DE practice; DE funder; meta-evaluation","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43229559","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evaluating Environment in International Development (2nd ed.)","authors":"R. Picciotto","doi":"10.56645/jmde.v17i40.551","DOIUrl":"https://doi.org/10.56645/jmde.v17i40.551","url":null,"abstract":"","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46959092","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Measuring Mental Health Literacy: Development of the Mental Health Awareness and Advocacy Assessment Tool","authors":"T. Aller, E. Fauth, J. Novak, Sarah Schwartz","doi":"10.56645/jmde.v17i39.671","DOIUrl":"https://doi.org/10.56645/jmde.v17i39.671","url":null,"abstract":"Background: Mental health literacy programs are a common community-based approach used to address the prevention of mental health issues on college campuses. Current assessment strategies used to evaluate the effectiveness of these programs often lack strong theoretical rational and psychometric rigor. \u0000Purpose: The purpose of this study was twofold. First, based upon extant literature, theory, and standard clinical practice, we propose a process-based model of mental health literacy that includes three macro factors—identifying mental health issues, locating empirically based resources, and responding to mental health issues—and three micro processes of how they unfold—acquiring knowledge, building self-efficacy, and applying skills (behavior). The second aim was to test the psychometric properties of a new tool created to evaluate this process-based model—the Mental Health Awareness and Advocacy Assessment Tool (MHAA-AT). \u0000Setting: Not applicable. \u0000Intervention: Not applicable. \u0000Research Design: A national sample of 296 college attending participants were recruited from Amazon’s Mechanical Turk. Participants responded to a demographic questionnaire and the newly developed MHAA-AT. Psychometric properties were examined through item response theory, exploratory factor analyses, and bivariate correlations. \u0000Findings: Results suggest the MHAA-AT is a sound measure and demonstrates appropriate item, person, and trait characteristics on declarative knowledge items, and single factor structures on self-efficacy and behavior items with moderate to high reliability and validity. While additional testing is need among other samples, results suggest that the MHAA-AT is a quality assessment tool. \u0000Keywords: College students; mental health literacy; item response theory; measurement","PeriodicalId":91909,"journal":{"name":"Journal of multidisciplinary evaluation","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-03-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47647774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}