Using novel data and ensemble models to improve automated labeling of Sustainable Development Goals.

IF 5.1 2区 环境科学与生态学 Q1 ENVIRONMENTAL SCIENCES
Sustainability Science Pub Date : 2024-01-01 Epub Date: 2024-07-24 DOI:10.1007/s11625-024-01516-3
Dirk U Wulff, Dominik S Meier, Rui Mata
{"title":"Using novel data and ensemble models to improve automated labeling of Sustainable Development Goals.","authors":"Dirk U Wulff, Dominik S Meier, Rui Mata","doi":"10.1007/s11625-024-01516-3","DOIUrl":null,"url":null,"abstract":"<p><p>A number of labeling systems based on text have been proposed to help monitor work on the United Nations (UN) Sustainable Development Goals (SDGs). Here, we present a systematic comparison of prominent SDG labeling systems using a variety of text sources and show that these differ considerably in their sensitivity (i.e., true-positive rate) and specificity (i.e., true-negative rate), have systematic biases (e.g., are more sensitive to specific SDGs relative to others), and are susceptible to the type and amount of text analyzed. We then show that an ensemble model that pools SDG labeling systems alleviates some of these limitations, exceeding the performance of the individual SDG labeling systems considered. We conclude that researchers and policymakers should care about the choice of the SDG labeling system and that ensemble methods should be favored when drawing conclusions about the absolute and relative prevalence of work on the SDGs based on automated methods.</p>","PeriodicalId":49457,"journal":{"name":"Sustainability Science","volume":null,"pages":null},"PeriodicalIF":5.1000,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11366727/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Sustainability Science","FirstCategoryId":"93","ListUrlMain":"https://doi.org/10.1007/s11625-024-01516-3","RegionNum":2,"RegionCategory":"环境科学与生态学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/7/24 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"ENVIRONMENTAL SCIENCES","Score":null,"Total":0}
引用次数: 0

Abstract

A number of labeling systems based on text have been proposed to help monitor work on the United Nations (UN) Sustainable Development Goals (SDGs). Here, we present a systematic comparison of prominent SDG labeling systems using a variety of text sources and show that these differ considerably in their sensitivity (i.e., true-positive rate) and specificity (i.e., true-negative rate), have systematic biases (e.g., are more sensitive to specific SDGs relative to others), and are susceptible to the type and amount of text analyzed. We then show that an ensemble model that pools SDG labeling systems alleviates some of these limitations, exceeding the performance of the individual SDG labeling systems considered. We conclude that researchers and policymakers should care about the choice of the SDG labeling system and that ensemble methods should be favored when drawing conclusions about the absolute and relative prevalence of work on the SDGs based on automated methods.

利用新数据和集合模型改进可持续发展目标的自动标注。
为了帮助监测联合国可持续发展目标(SDGs)的工作,人们提出了许多基于文本的标签系统。在此,我们利用各种文本资源对著名的可持续发展目标标注系统进行了系统比较,结果表明,这些系统在灵敏度(即真阳性率)和特异性(即真阴性率)方面存在很大差异,存在系统性偏差(例如,相对于其他系统,对特定的可持续发展目标更敏感),并且易受所分析文本的类型和数量的影响。然后,我们展示了一个集合可持续发展目标标注系统的集合模型,该模型缓解了其中的一些局限性,其性能超过了所考虑的单个可持续发展目标标注系统。我们的结论是,研究人员和政策制定者应该关注对可持续发展目标标注系统的选择,并且在基于自动化方法得出可持续发展目标工作的绝对和相对普遍性的结论时,应该优先考虑集合方法。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Sustainability Science
Sustainability Science 环境科学-环境科学
CiteScore
11.30
自引率
10.00%
发文量
174
审稿时长
3 months
期刊介绍: The journal Sustainability Science offers insights into interactions within and between nature and the rest of human society, and the complex mechanisms that sustain both. The journal promotes science based predictions and impact assessments of global change, and seeks ways to ensure that such knowledge can be understood by society and be used to strengthen the resilience of global natural systems (such as ecosystems, ocean and atmospheric systems, nutrient cycles), social systems (economies, governments, industry) and human systems at the individual level (lifestyles, health, security, and human values).
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信