Banamali Panigrahi, Saman Razavi, Lorne E. Doig, Blanchard Cordell, Hoshin V. Gupta, Karsten Liber
{"title":"On Robustness of the Explanatory Power of Machine Learning Models: Insights From a New Explainable AI Approach Using Sensitivity Analysis","authors":"Banamali Panigrahi, Saman Razavi, Lorne E. Doig, Blanchard Cordell, Hoshin V. Gupta, Karsten Liber","doi":"10.1029/2024wr037398","DOIUrl":null,"url":null,"abstract":"Machine learning (ML) is increasingly considered the solution to environmental problems where limited or no physico-chemical process understanding exists. But in supporting high-stakes decisions, where the ability to <i>explain</i> possible solutions is key to their acceptability and legitimacy, ML can fall short. Here, we develop a method, rooted in formal <i>sensitivity analysis</i>, to uncover the primary drivers behind ML predictions. Unlike many methods for <i>explainable artificial intelligence</i> (XAI), this method (a) accounts for complex multi-variate distributional properties of data, common in environmental systems, (b) offers a global assessment of the input-output response surface formed by ML, rather than focusing solely on local regions around existing data points, and (c) is scalable and data-size independent, ensuring computational efficiency with large data sets. We apply this method to a suite of ML models predicting various water quality variables in a pilot-scale experimental pit lake. A critical finding is that subtle alterations in the design of some ML models (such as variations in random seed, functional class, hyperparameters, or data splitting) can lead to different interpretations of how outputs depend on inputs. Further, models from different ML families (decision trees, connectionists, or kernels) may focus on different aspects of the information provided by data, despite displaying similar predictive power. Overall, our results underscore the need to assess the explanatory robustness of ML models and advocate for using model ensembles to gain deeper insights into system drivers and improve prediction reliability.","PeriodicalId":23799,"journal":{"name":"Water Resources Research","volume":"214 1","pages":""},"PeriodicalIF":4.6000,"publicationDate":"2025-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Water Resources Research","FirstCategoryId":"89","ListUrlMain":"https://doi.org/10.1029/2024wr037398","RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENVIRONMENTAL SCIENCES","Score":null,"Total":0}
引用次数: 0
Abstract
Machine learning (ML) is increasingly considered the solution to environmental problems where limited or no physico-chemical process understanding exists. But in supporting high-stakes decisions, where the ability to explain possible solutions is key to their acceptability and legitimacy, ML can fall short. Here, we develop a method, rooted in formal sensitivity analysis, to uncover the primary drivers behind ML predictions. Unlike many methods for explainable artificial intelligence (XAI), this method (a) accounts for complex multi-variate distributional properties of data, common in environmental systems, (b) offers a global assessment of the input-output response surface formed by ML, rather than focusing solely on local regions around existing data points, and (c) is scalable and data-size independent, ensuring computational efficiency with large data sets. We apply this method to a suite of ML models predicting various water quality variables in a pilot-scale experimental pit lake. A critical finding is that subtle alterations in the design of some ML models (such as variations in random seed, functional class, hyperparameters, or data splitting) can lead to different interpretations of how outputs depend on inputs. Further, models from different ML families (decision trees, connectionists, or kernels) may focus on different aspects of the information provided by data, despite displaying similar predictive power. Overall, our results underscore the need to assess the explanatory robustness of ML models and advocate for using model ensembles to gain deeper insights into system drivers and improve prediction reliability.
期刊介绍:
Water Resources Research (WRR) is an interdisciplinary journal that focuses on hydrology and water resources. It publishes original research in the natural and social sciences of water. It emphasizes the role of water in the Earth system, including physical, chemical, biological, and ecological processes in water resources research and management, including social, policy, and public health implications. It encompasses observational, experimental, theoretical, analytical, numerical, and data-driven approaches that advance the science of water and its management. Submissions are evaluated for their novelty, accuracy, significance, and broader implications of the findings.