EvaluationPub Date : 2023-09-10DOI: 10.1177/13563890231188743
Rick Davies, Tom Hobson, Lara Mani, Simon Beard
{"title":"ParEvo: A methodology for the exploration and evaluation of alternative futures","authors":"Rick Davies, Tom Hobson, Lara Mani, Simon Beard","doi":"10.1177/13563890231188743","DOIUrl":"https://doi.org/10.1177/13563890231188743","url":null,"abstract":"Evaluators’ main encounter with views of the future is in the form of theories of change, about how a programme will work to achieve a desired end, in a given context. These are typically focussed on specific relatively short-term futures, which are both desired and expected. But even in the short term, reality often involves unpredictable events which must be responded to. Other ways of thinking about the future may be helpful and complementary, notably those developed by foresight practitioners working in the field of futures studies. These pay more attention to a range of possible futures, rather than a single perspective. One way of exploring such futures is by using ParEvo.org, an online process that enables the participatory exploration of alternative futures. This article explains how the ParEvo process works, the theory informing its design, and its usage to date. Attention is given to three evaluation challenges, and methods to address them: (a) optimising exercise design, (b) analysis of immediate results and (c) identifying longer-term impacts. Two exercises undertaken by the Cambridge-based Centre for the Study of Existential Risk (CSER) in 2021–2022 are used as illustrative examples.","PeriodicalId":47511,"journal":{"name":"Evaluation","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"136071454","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
EvaluationPub Date : 2023-07-01DOI: 10.1177/13563890231185015
{"title":"French language abstracts","authors":"","doi":"10.1177/13563890231185015","DOIUrl":"https://doi.org/10.1177/13563890231185015","url":null,"abstract":"Restricted accessAbstractFirst published online July 13, 2023French language abstractsVolume 29, Issue 3https://doi.org/10.1177/13563890231185015","PeriodicalId":47511,"journal":{"name":"Evaluation","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"135507381","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
EvaluationPub Date : 2022-10-01Epub Date: 2022-09-15DOI: 10.1177/13563890221124637
Ana Manzano
{"title":"Conducting focus groups in realist evaluation.","authors":"Ana Manzano","doi":"10.1177/13563890221124637","DOIUrl":"10.1177/13563890221124637","url":null,"abstract":"<p><p>Focus groups are valuable tools for evaluators to help stakeholders to clarify programme theories. In 1987, R.K. Merton, often attributed with the birth of focus groups, wrote about how these were 'being mercilessly misused'. In the 1940s, his team had conceived focus groups as tools for developing middle-range theory, but through their astonishing success focus groups have metamorphosed and are often an 'unchallenged' choice in many evaluation approaches, while their practice seems to provide a philosophically diverse picture. This article examines what knowledge focus group data generate, and how they support theory development. It starts with an overview of the history of focus groups, establishing a relationship between their emergence as a data collection method and the evaluation profession. Practical lessons for conducting groups in realist evaluation are suggested, while exploring how qualitative data can support programme and middle-range theory development using the example of realist evaluation.</p>","PeriodicalId":47511,"journal":{"name":"Evaluation","volume":"28 4","pages":"406-425"},"PeriodicalIF":2.4,"publicationDate":"2022-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9530522/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"33497308","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
EvaluationPub Date : 2018-01-01Epub Date: 2018-01-19DOI: 10.1177/1356389017750194
Martijn Felder, Hester van de Bovenkamp, Antoinette de Bont
{"title":"Politics of policy learning: Evaluating an experiment on free pricing arrangements in Dutch dental care.","authors":"Martijn Felder, Hester van de Bovenkamp, Antoinette de Bont","doi":"10.1177/1356389017750194","DOIUrl":"https://doi.org/10.1177/1356389017750194","url":null,"abstract":"<p><p>In Dutch healthcare, new market mechanisms have been introduced on an experimental basis in an attempt to contain costs and improve quality. Informed by a constructivist approach, we demonstrate that such experiments are not neutral testing grounds. Drawing from semi-structured interviews and policy texts, we reconstruct an experiment on free pricing in dental care that turned into a critical example of market failure, influencing developments in other sectors. Our analysis, however, shows that (1) different market logics and (2) different experimental logics were reproduced simultaneously during the course of the experiment. We furthermore reveal how (3) evaluation and political life influenced which logics were reproduced and became taken as <i>the</i> lessons learned. We use these insights to discuss the role of evaluation in learning from policy experimentation and close with four questions that evaluators could ask to better understand <i>what</i> is learned from policy experiments, <i>how</i>, and <i>why</i>.</p>","PeriodicalId":47511,"journal":{"name":"Evaluation","volume":"24 1","pages":"6-25"},"PeriodicalIF":2.4,"publicationDate":"2018-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1356389017750194","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"35940620","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
EvaluationPub Date : 2018-01-01Epub Date: 2017-10-04DOI: 10.1177/1356389017733210
Bodille Arensman, Margit van Wessel
{"title":"Negotiating effectiveness in transnational advocacy evaluation.","authors":"Bodille Arensman, Margit van Wessel","doi":"10.1177/1356389017733210","DOIUrl":"https://doi.org/10.1177/1356389017733210","url":null,"abstract":"<p><p>International development organizations increasingly use advocacy as a strategy to pursue effectiveness. However, establishing the effectiveness of advocacy is problematic and dependent on the interpretations of the stakeholders involved, as well as the interactions between them. This article challenges the idea of objective and rational evaluation, showing that advocacy evaluation is an inherently political process in which space for interactions around methods, processes and results defines how effectiveness is interpreted, measured and presented. In addition, this article demonstrates how this space for interaction contributes to the quality and accuracy of evaluating advocacy effectiveness by providing room to explore and address the multiplicities of meaning around identifying, measuring and presenting outcomes.</p>","PeriodicalId":47511,"journal":{"name":"Evaluation","volume":"24 1","pages":"51-68"},"PeriodicalIF":2.4,"publicationDate":"2018-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1356389017733210","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"36614289","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
EvaluationPub Date : 2014-10-01DOI: 10.1177/1356389014551484
Susan Nayiga, Deborah DiLiberto, Lilian Taaka, Christine Nabirye, Ane Haaland, Sarah G Staedke, Clare I R Chandler
{"title":"Strengthening patient-centred communication in rural Ugandan health centres: A theory-driven evaluation within a cluster randomized trial.","authors":"Susan Nayiga, Deborah DiLiberto, Lilian Taaka, Christine Nabirye, Ane Haaland, Sarah G Staedke, Clare I R Chandler","doi":"10.1177/1356389014551484","DOIUrl":"https://doi.org/10.1177/1356389014551484","url":null,"abstract":"<p><p>This article describes a theory-driven evaluation of one component of an intervention to improve the quality of health care at Ugandan public health centres. Patient-centred services have been advocated widely, but such approaches have received little attention in Africa. A cluster randomized trial is evaluating population-level outcomes of an intervention with multiple components, including 'patient-centred services.' A process evaluation was designed within this trial to articulate and evaluate the implementation and programme theories of the intervention. This article evaluates one hypothesized mechanism of change within the programme theory: the impact of the Patient Centred Services component on health-worker communication. The theory-driven approach extended to evaluation of the outcome measures. The study found that the proximal outcome of patient-centred communication was rated 10 percent higher (<i>p</i> < 0.008) by care seekers consulting with the health workers who were at the intervention health centres compared with those at control health centres. This finding will strengthen interpretation of more distal trial outcomes.</p>","PeriodicalId":47511,"journal":{"name":"Evaluation","volume":"20 4","pages":"471-491"},"PeriodicalIF":2.4,"publicationDate":"2014-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1177/1356389014551484","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"33312540","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
EvaluationPub Date : 2012-01-01DOI: 10.1177/1356389011430371
Damien Contandriopoulos, Astrid Brousselle
{"title":"Evaluation models and evaluation use.","authors":"Damien Contandriopoulos, Astrid Brousselle","doi":"10.1177/1356389011430371","DOIUrl":"10.1177/1356389011430371","url":null,"abstract":"<p><p>The use of evaluation results is at the core of evaluation theory and practice. Major debates in the field have emphasized the importance of both the evaluator's role and the evaluation process itself in fostering evaluation use. A recent systematic review of interventions aimed at influencing policy-making or organizational behavior through knowledge exchange offers a new perspective on evaluation use. We propose here a framework for better understanding the embedded relations between evaluation context, choice of an evaluation model and use of results. The article argues that the evaluation context presents conditions that affect both the appropriateness of the evaluation model implemented and the use of results.</p>","PeriodicalId":47511,"journal":{"name":"Evaluation","volume":"18 1","pages":"61-77"},"PeriodicalIF":2.4,"publicationDate":"2012-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3605074/pdf/nihms2438.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"31334845","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
EvaluationPub Date : 2009-04-01DOI: 10.1177/1356389008101967
Astrid Brousselle, Damien Contandriopoulos, Marc Lemire
{"title":"Using Logic Analysis to Evaluate Knowledge Transfer Initiatives: The Case of the Research Collective on the Organization of Primary Care Services.","authors":"Astrid Brousselle, Damien Contandriopoulos, Marc Lemire","doi":"10.1177/1356389008101967","DOIUrl":"10.1177/1356389008101967","url":null,"abstract":"<p><p>Models that shift more responsibility onto researchers for the process of incorporating research results into decision-making have greatly gained in popularity during the past two decades. This shift has created a new area of research to identify the best ways to transfer academic results into the organizational and political arenas. However, evaluating the utilization of information coming out of a knowledge transfer (KT) initiative remains an enormous challenge. This article demonstrates how logic analysis has proven to be a useful evaluation method to assess the utilization potential of KT initiatives. We present the case of the evaluation of the Research Collective on the Organization of Primary Care Services, an innovative experiment in knowledge synthesis and transfer. The conclusions focus not only on the utilization potential of results coming out of the Research Collective, but also on the theoretical framework used, in order to facilitate its application to the evaluation of other knowledge transfer initiatives.</p>","PeriodicalId":47511,"journal":{"name":"Evaluation","volume":"15 2","pages":"165-183"},"PeriodicalIF":2.4,"publicationDate":"2009-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3610640/pdf/nihms2546.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"40238877","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
EvaluationPub Date : 2004-04-01DOI: 10.1177/1356389004046292
Astrid Brousselle
{"title":"What Counts is not Falling … but Landing: Strategic Analysis: An Adapted Model for Implementation Evaluation.","authors":"Astrid Brousselle","doi":"10.1177/1356389004046292","DOIUrl":"10.1177/1356389004046292","url":null,"abstract":"<p><p>Implementation evaluations, also called process evaluations, involve studying the development of programmes, and identifying and understanding their strengths and weaknesses. Undertaking an implementation evaluation offers insights into evaluation objectives, but does not help the researcher develop a research strategy. During the implementation analysis of the UNAIDS drug access initiative in Chile, the strategic analysis model developed by Crozier and Friedberg was used. However, a major incompatibility was noted between the procedure put forward by Crozier and Friedberg and the specific characteristics of the programme being evaluated. In this article, an adapted strategic analysis model for programme evaluation is proposed.</p>","PeriodicalId":47511,"journal":{"name":"Evaluation","volume":"10 2","pages":"155-173"},"PeriodicalIF":2.4,"publicationDate":"2004-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3605071/pdf/nihms2509.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"31425666","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Toward a theory of planned social change: alternative perspectives and ideas.","authors":"R F Rich, G Zaltman","doi":"","DOIUrl":"","url":null,"abstract":"","PeriodicalId":47511,"journal":{"name":"Evaluation","volume":" ","pages":"41-7"},"PeriodicalIF":2.4,"publicationDate":"1978-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"21109712","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}