{"title":"Comprehensive evaluation framework building for integrated development program","authors":"Jae Eun Shin, Mi Joung Yu, Suk-won Lee","doi":"10.1177/13563890231183990","DOIUrl":"https://doi.org/10.1177/13563890231183990","url":null,"abstract":"The integrated development program, a cross-sectional development approach targeting a specific region or village, is receiving increasing attention as the 2030 agenda emphasizes a holistic approach in solving the poverty problem. The conventional evaluation designed for a causal inference of project-level outcome is incapable of reflecting the characteristics of the integrated development program. In this regard, we analyzed the unique features of the integrated development program by applying policy design theory to identify the challenges of evaluating this type of program. We then derived the logic of causal inference from the generative causation approach to resolve these challenges and developed practical evaluation tools, including causal maps and the evaluation matrix. These tools can be applied to respond to the evaluation questions related to the characteristics of the integrated development program, such as chain of causality, interactions between the outcomes, and many-to-many correspondence between aid-providers and recipients.","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87839959","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sara Van Belle, I. Abejirinde, A. Ssennyonjo, P. Srinivas, P. Hebbar, B. Marchal
{"title":"How to develop a realist programme theory using Margaret Archer’s structure–agency–culture framework: The case of adolescent accountability for sexual and reproductive health in urban resource-constrained settings","authors":"Sara Van Belle, I. Abejirinde, A. Ssennyonjo, P. Srinivas, P. Hebbar, B. Marchal","doi":"10.1177/13563890231185167","DOIUrl":"https://doi.org/10.1177/13563890231185167","url":null,"abstract":"Realist evaluation is in essence a theory-building and testing approach. We argue that in practice, the theory-building potential of realist evaluation, review and research is not fully exploited in the field of global health. Our assumption is that the Structure-Agency-Culture explanatory framework of critical realist Margaret Archer could stimulate realist evaluators to conceptualize and systematically explore how structural and cultural conditions interact with programmes that aim at introducing social change. We propose step-wise guidance towards integrating the Structure–Agency–Culture framework into the development of realist programme theories. We present a worked example from an urban adolescent health study in poor neighbourhoods of Kampala, Mumbai, New Delhi and Cotonou. The guidance aims to bring to the fore the role of agency and context through the analysis of the interactions between structure, culture, agency and mechanisms. This is helpful in realist research in general, and in evaluations of complex interventions oriented towards social change.","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2023-06-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"78644832","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Understanding evaluation use from an organisational perspective: A review of the literature and a research agenda","authors":"T. Kupiec, Dorota Celińska-Janowicz, V. Pattyn","doi":"10.1177/13563890231185164","DOIUrl":"https://doi.org/10.1177/13563890231185164","url":null,"abstract":"Why do some types of evaluation use prevail in certain contexts and not in others? The aim of this article is to advance knowledge about organisational factors of evaluation use, that is, determinants of evaluation use grounded in organisational theories. We critically review existing frameworks of organisational factors of evaluation use, highlighting key differences between them and pointing out discrepancies with empirical insights. We discuss the merits of two potential areas for future research that can help concretise theoretical stances: considering organisational legitimacy as a potential direct determinant of evaluation use and incorporating a dynamic perspective in organisational frameworks of evaluation use.","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2023-06-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77871560","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Challenges and strategies for implementers and evaluators working in conflict settings","authors":"Laura Bond, Elizabeth K. Klein, Emily F. Gates","doi":"10.1177/13563890231171608","DOIUrl":"https://doi.org/10.1177/13563890231171608","url":null,"abstract":"Practitioners implementing and evaluating humanitarian programs in conflict zones face unique challenges requiring context-sensitive guidance. Conflict zones—geographical areas with militarized and non-militarized violence, widespread political instability, and state-sanctioned intimidation and genocidal violence—are rife with logistical, methodological, and ethical challenges that impact implementation and evaluation. Although challenges have been well-documented, few solutions or evaluation frameworks exist to help evaluators in conflict zones confront these challenges. This article examines how the application of implementation science frameworks, specifically, Reach, Effectiveness, Adoption, Implementation, and Maintenance and the Consolidated Framework for Implementation Research, offers useful strategies to mitigate some challenges in conducting evaluations in conflict zones. Areas for future research include collaborative team approaches to using these frameworks, ethical guidance, and reporting and dissemination.","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2023-05-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"87436242","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An analysis of probabilistic forwarding of coded packets on random geometric graphs","authors":"B.R. Vinay Kumar , Navin Kashyap , D. Yogeshwaran","doi":"10.1016/j.peva.2023.102343","DOIUrl":"https://doi.org/10.1016/j.peva.2023.102343","url":null,"abstract":"<div><p><span><span>We consider the problem of energy-efficient broadcasting on large ad-hoc networks. Ad-hoc networks are generally modelled using random geometric graphs (RGGs). Here, nodes are deployed uniformly in a square area around the origin, and any two nodes which are within </span>Euclidean distance of 1 are assumed to be able to receive each other’s broadcast. A source node at the origin encodes </span><span><math><mi>k</mi></math></span> data packets of information into <span><math><mrow><mi>n</mi><mspace></mspace><mrow><mo>(</mo><mo>></mo><mi>k</mi><mo>)</mo></mrow></mrow></math></span> coded packets and transmits them to all its one-hop neighbours. The encoding is such that, any node that receives at least <span><math><mi>k</mi></math></span> out of the <span><math><mi>n</mi></math></span> coded packets can retrieve the original <span><math><mi>k</mi></math></span><span> data packets. Every other node in the network follows a probabilistic forwarding protocol; upon reception of a previously unreceived packet, the node forwards it with probability </span><span><math><mi>p</mi></math></span> and does nothing with probability <span><math><mrow><mn>1</mn><mo>−</mo><mi>p</mi></mrow></math></span>. We are interested in the minimum forwarding probability which ensures that a large fraction of nodes can decode the information from the source. We deem this a <em>near-broadcast</em>. The performance metric of interest is the expected total number of transmissions at this minimum forwarding probability, where the expectation is over both the forwarding protocol as well as the realization of the RGG. In comparison to probabilistic forwarding with no coding, our treatment of the problem indicates that, with a judicious choice of <span><math><mi>n</mi></math></span>, it is possible to reduce the expected total number of transmissions while ensuring a near-broadcast.</p></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49901373","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lu Pang , Anis Alazzawe , Madhurima Ray , Krishna Kant , Jeremy Swift
{"title":"Adaptive Intelligent Tiering for modern storage systems","authors":"Lu Pang , Anis Alazzawe , Madhurima Ray , Krishna Kant , Jeremy Swift","doi":"10.1016/j.peva.2023.102332","DOIUrl":"https://doi.org/10.1016/j.peva.2023.102332","url":null,"abstract":"<div><p><span>Enterprise systems routinely use tiered storage consisting of a hierarchy of storage devices that vary in speed and size. One key to obtaining good performance in such a hierarchy is to migrate data elements intelligently to the appropriate tier. For example, moving the most used data towards the fastest tier and the least used data towards the slowest tier. Tiering is typically done based on usage </span>statistics<span><span> over relatively long time periods. In this paper, we consider a much more agile tiering mechanism called Adaptive Intelligent Tiering (AIT). It can dynamically adapt to the changing behavior of storage accesses by the running applications. The AIT mechanism uses a deep learning model to generate a set of candidate movements and employs a </span>reinforcement learning mechanism to further refine the candidates. Based on extensive simulations in a 3-tier system, we show that the proposed scheme, compared with several other methods, enhances workload performance up to 85% on storage traces with a wide range of characteristics.</span></p></div>","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2023-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49901375","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"French language abstracts","authors":"","doi":"10.1177/13563890231167708","DOIUrl":"https://doi.org/10.1177/13563890231167708","url":null,"abstract":"dans les centres de santé pour améliorer la santé des membres de la communauté. Les analy-ses corrigées démontraient l’effet de l’intervention sur certains dispositifs des centres de santé. Cependant, cela ne se traduisait pas par une amélioration de l’état de santé de la communauté. Nous examinons les enjeux pratiques et épistémologiques de l’utilisation de l’analyse de médiation pour évaluer une intervention complexe. Ces conclusions serviront à des évaluations à venir","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83003686","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"DOME: A mixed-method evaluation of the impacts of complex programmes against educational poverty","authors":"G. Tomei","doi":"10.1177/13563890231166979","DOIUrl":"https://doi.org/10.1177/13563890231166979","url":null,"abstract":"Developmental Outcome Monitoring and Evaluation (DOME) is a mixed and participatory method for the impact assessment of complex and innovative programmes. It has been developed and is being applied in Italy for the evaluation of interventions to combat juvenile educational poverty. The methodological process focuses on the detection of strategic change (outcomes) implemented by key actors in complex programmes, implemented in diverse geographical areas and managed by public–private partnerships. DOME combines methodological approaches already in use in the field of monitoring and evaluation – theory of change, counterfactual analysis and developmental evaluation and trailing research – in new ways.","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81065902","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Gráinne Hickey, S. McGilloway, Yvonne Leckey, T. Bywater, M. Donnelly
{"title":"Understanding the theoretical underpinnings of preventative early parenting interventions: A theory-based evaluation","authors":"Gráinne Hickey, S. McGilloway, Yvonne Leckey, T. Bywater, M. Donnelly","doi":"10.1177/13563890231165814","DOIUrl":"https://doi.org/10.1177/13563890231165814","url":null,"abstract":"Early parenting programmes which promote nurturing care for children aged 0–3 years are popular in community-based primary health care settings. However, little research has explored the causal and theoretical assumptions underpinning these programmes. This paper outlines a programme theory evaluation which was conducted to explore the change principles inherent in a complex early parenting intervention, the parent and infant programme, which aims to support parents and infants aged 0–2 years. A documentary analysis and qualitative interviews (n = 19) with key stakeholders were undertaken to describe programme components, outline intervention objectives and identify the mechanisms assumed to be important to programme success. The interpretation of the findings was informed by the Consolidated Framework for Implementation Research, within which we explore and describe the theoretical perspectives that underpin the programme and guide its operationalisation and delivery. The importance of attending to the causal mechanisms underlying early parenting interventions is outlined.","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84031060","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Strengthening evaluation capacity in Government: Why institutionalisation matters and why it is not sufficient","authors":"D. Lamarque","doi":"10.1177/13563890231166981","DOIUrl":"https://doi.org/10.1177/13563890231166981","url":null,"abstract":"If we are interested in promoting and strengthening evaluation in a country, we know that the extent to which evaluation is institutionalised is key. Investigating the strengths and weaknesses of institutional factors and identifying their limits help us understand under which conditions these factors will make a difference – and what is still needed. I will be drawing on several studies and reports – as well as my own experience – that address the issue of how evaluation is institutionalised internationally. While all these sources confirm the positive influence of institutional arrangements, they also show that evaluation develops unevenly and slowly and these developments are often difficult to specify or measure.","PeriodicalId":19964,"journal":{"name":"Performance Evaluation","volume":null,"pages":null},"PeriodicalIF":2.2,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"84542916","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}