{"title":"Population Health Challenges and Evaluative Thinking: Rapid Responses in the Time of COVID-19","authors":"K. D’Onise, Katherine Pontifex","doi":"10.1177/1035719X231175884","DOIUrl":"https://doi.org/10.1177/1035719X231175884","url":null,"abstract":"There are multiple complex challenges that the population health sector faces to improve the health of the community, and the sector must work efficiently and effectively to make the most of the committed funds, sometimes in contexts of uncertainty. This article explores, through the case study of the South Australian COVID-19 Vaccine Hesitancy Project, the application of pillars from the Lean Start-up model of setting a clear vision, and creation of a series of minimum viable products to enable work with the community to commence as quickly as possible. This article draws parallels between this corporate model and the notion of evaluative thinking as well as adaptive evaluation approaches, demonstrates the interest from population health program and policy actors in adaptive practices and proposes a continuing opportunity for the evaluation profession in this space.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-05-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48000057","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An asset-based evaluation of a novel New Zealand rural health service","authors":"Katie E McMenamin, J. McMenamin, A. Towers","doi":"10.1177/1035719X231175874","DOIUrl":"https://doi.org/10.1177/1035719X231175874","url":null,"abstract":"This article describes the background behind the development of a novel evaluation approach of an Aotearoa New Zealand rural health service using an equity-focused, strengths-based approach to complement the traditional audit process. This approach avoids a deficit-based, gap-analysis of service shortcomings in favour of a strengths-based model through which services can draw on identified assets in planning future service initiatives. The approach discussed aims to identify and mobilise assets and strengths of people and services that promote quality outcomes, with a particular focus on those that address the deeply embedded health inequities for Māori (the indigenous people of Aotearoa). The asset model will be used to evaluate a rural service that is uniquely co-located with other health-related hospital services. This article aims to provide a commentary on the benefits of an asset-based approach to rural general practice evaluation.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"23 1","pages":"126 - 138"},"PeriodicalIF":0.0,"publicationDate":"2023-05-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45762702","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Editorial June 2023","authors":"Jon Guenther","doi":"10.1177/1035719x231168702","DOIUrl":"https://doi.org/10.1177/1035719x231168702","url":null,"abstract":"","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42545608","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Practice of Evaluation: Partnership Approaches for Community Change","authors":"L. Curran","doi":"10.1177/1035719x231164871","DOIUrl":"https://doi.org/10.1177/1035719x231164871","url":null,"abstract":"","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48925192","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Procedural and participatory ethics: community-based evaluation in practice","authors":"Tony O’Connor","doi":"10.1177/1035719X231166206","DOIUrl":"https://doi.org/10.1177/1035719X231166206","url":null,"abstract":"This article argues that it can be beneficial for institution-based, procedural ethics review of evaluation design in the planning stage to be followed by community-based oversight of ethical issues in the field. Deferring to an institutional review board (IRB) for ethical assessment when a project is underway can be impractical for community-based projects that are designed to be responsive to local needs and interests, especially when community leaders expect to have a meaningful say in determining what is the right thing to do. This article discusses a 2-year project in New Zealand, where community leaders and the project funder formed a project steering group to, among other things, provide ethical oversight. Ethical issues that arose during the project and the steering group’s role in considering the most suitable response are discussed and linked to literature about participatory ethics.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"23 1","pages":"91 - 100"},"PeriodicalIF":0.0,"publicationDate":"2023-03-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43968560","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Evaluative thinking","authors":"Michael J. Cole","doi":"10.1177/1035719X231163932","DOIUrl":"https://doi.org/10.1177/1035719X231163932","url":null,"abstract":"Evaluative thinking is a core skill used by evaluators. It brings evaluation practice to a higher level of sophistication and utility and helps to ensure that evaluation findings, interpretations and recommendations are contextualised, evidence-based and practical. This article draws upon decades of the work of evaluators who have published on thinking critically and evaluatively. It identifies and explores the five primary elements of evaluative thinking – critical, contextual, creative, practical thinking and reflective practice – and offers a practical framework for examining and applying these elements of evaluative thinking. It also distinguishes evaluative thinking from evaluation planning and implementation, which it precedes and guides. Finally, it proposes that evaluative thinking should be examined and learned intentionally as a core evaluation skill just as, for example, various methodologies or methods of analysis are taught.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"23 1","pages":"70 - 90"},"PeriodicalIF":0.0,"publicationDate":"2023-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41369581","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An interview with an emerging evaluator – Amanda Mottershead","authors":"A. Rutter","doi":"10.1177/1035719x231161899","DOIUrl":"https://doi.org/10.1177/1035719x231161899","url":null,"abstract":"The ‘ Evaluator Perspective ’ series cover a range of perspectives from evaluators working in the indigenous area; evaluators with a long career in evaluation; or evaluators experienced in a certain methodology. This particular perspective is that of","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"23 1","pages":"163 - 166"},"PeriodicalIF":0.0,"publicationDate":"2023-03-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46669148","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
R. Renger, Jessica Renger, Richard N. Van Eck, M. Basson, Jirina Renger
{"title":"A comparative case study illustrating the influence of a system perspective on the outcome evaluation of simple versus complex interventions","authors":"R. Renger, Jessica Renger, Richard N. Van Eck, M. Basson, Jirina Renger","doi":"10.1177/1035719X231160584","DOIUrl":"https://doi.org/10.1177/1035719X231160584","url":null,"abstract":"When evaluating the effectiveness of an intervention, the evaluation approach must match the intervention complexity, ensuring that the chosen evaluation is “fit for purpose.” For simple interventions, evaluating short-, mid-, and long-term outcomes is appropriate. However, for complex interventions, an additional outcome that must be considered is the essential system property that emerges as a result of the interaction of interdependent intervention components. By focusing on the emergent system property, evaluators are better able to assess the holistic effectiveness of a complex intervention. This article illustrates this principle through a comparative case study of a simple intervention and a complex intervention within a National Institute of Health (NIH) funded Clinical Translational Research center. The analysis illustrates that a more effective and appropriate evaluation results when a complex intervention, deemed to be operating and functioning as a system, is evaluated as a system than could have been achieved by treating each component as independent and evaluating the short-, mid-, or long-term outcomes of each component.","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"23 1","pages":"101 - 110"},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45653594","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ihoghosa Iyamu, Mai Berger, Saranee Fernando, M Elizabeth Snow, Amy Salmon
{"title":"Developmental evaluation during the COVID-19 pandemic: Practice-based learnings from projects in British Columbia, Canada.","authors":"Ihoghosa Iyamu, Mai Berger, Saranee Fernando, M Elizabeth Snow, Amy Salmon","doi":"10.1177/1035719X221119841","DOIUrl":"10.1177/1035719X221119841","url":null,"abstract":"<p><p>In this article, we explore experiences and learnings from adapting to challenges encountered in implementing three Developmental Evaluations (DE) in British Columbia, Canada within the evolving context of the COVID-19 pandemic. We situate our DE projects within our approach to the DE life cycle and describe challenges encountered and required adaptations in each phase of the life cycle. Regarding foundational aspects of DEs, we experienced challenges with relationship building, assessing and responding to the context, and ensuring continuous learning. These challenges were related to suboptimal embeddedness of the evaluators within the evaluated projects. We adapted by leveraging online channels to maintain communications and securing stakeholder engagement by assuming non-traditional DE roles based on our knowledge of the context to support project goals. Additional challenges experienced with mapping the rationale and goals of the projects, identifying domains for assessment, collecting data, making sense of the data and intervening were adapted to by facilitating online workshops, collecting data online and through proxy evaluators, while sharing methodological insights within the evaluation team. During evolving crises, like the COVID-19 pandemic, evaluators must embrace flexibility, leverage, and apply their knowledge of the evaluation context, lean on their strengths, purposefully reflect and share knowledge to optimise their DEs.</p>","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"23 1","pages":"23-39"},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9379587/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"65871859","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Reflections on COVID-19 and internal evaluation in a humanitarian non-profit.","authors":"Leanne M Kelly","doi":"10.1177/1035719X221109310","DOIUrl":"10.1177/1035719X221109310","url":null,"abstract":"<p><p>This article uses reflective practice and social interdependence theory to unpack the ways in which our practice as internal evaluators was impacted by the COVID-19 pandemic, using the development of a monitoring and evaluation framework as a case study. The reflections are separated into six stages: description, feelings, evaluation, analysis, conclusion and action plan. Initial reflections on the impact of COVID-19 drew out the negative effects of the pandemic and associated restrictions, which limited our ability to build rapport and stretched our capacity to balance work and home commitments. Deeper investigation revealed that the disaster pushed us to develop new ways of working that will augment and improve our future efforts. We foresee that these learnings will enable a future with greater ability to offer hybrid online/face-to-face collaborative opportunities that will enhance inclusion and active participation, thereby promoting monitoring and evaluation with greater collective ownership and relevance to a wider audience.</p>","PeriodicalId":37231,"journal":{"name":"Evaluation Journal of Australasia","volume":"23 1","pages":"40-58"},"PeriodicalIF":0.0,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9237647/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48802163","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}