D. Zhang, Q. Lin, H. Yao, Y. R. He, J. Deng, X. X. Zhang, Bellwood Road Dorset P A E Ontario Canada Parks
{"title":"Accelerating SWAT Simulations Using An In-Memory NoSQL Database","authors":"D. Zhang, Q. Lin, H. Yao, Y. R. He, J. Deng, X. X. Zhang, Bellwood Road Dorset P A E Ontario Canada Parks","doi":"10.3808/jei.201900425","DOIUrl":null,"url":null,"abstract":"Due to its versatility, the Soil and Water Assessment Tool (SWAT) has been widely applied to investigate the effects of management activities and climate change on water availability and quality. However, the use of high spatial resolution data and the advantages of SWAT itself have significantly increased the input/output (I/O) demand and thus the runtime of modeling routines that require a large number of iterative simulations. In this study, we proposed a generic scheme to reduce the SWAT runtime by caching the model inputs using the in-memory NoSQL database Redis. Then the SWAT source codes (rev 488) was modified according to this proposed scheme to develop the MA-SWAT (memory accelerated SWAT) model by incorporating a new subroutine known as Fortran_calls_c to retrieve the cached inputs. We then evaluated MA-SWAT with four synthetic hydrological models and five different parallel schemes in a quad-core commodity laptop. The test results showed that when applied with a parallel simulation program, MA-SWAT could achieve a speedup by a factor of 8.4 ~ 10.9 depending on model complexity. Compared with the original SWAT, MA-SWAT significantly improved the computation speed, indicating that the proposed scheme is a desirable method for solving high computational demand problems such as calibration, sensitivity and uncertainty analysis. Moreover, the proposed concept of linking the SWAT model with Redis via the minimalistic C client driver of Redis is a generic method, and it is possible to apply this method to other Fortran-implemented environmental model to alleviate I/O demands.","PeriodicalId":54840,"journal":{"name":"Journal of Environmental Informatics","volume":null,"pages":null},"PeriodicalIF":6.0000,"publicationDate":"2019-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"5","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Environmental Informatics","FirstCategoryId":"93","ListUrlMain":"https://doi.org/10.3808/jei.201900425","RegionNum":1,"RegionCategory":"环境科学与生态学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENVIRONMENTAL SCIENCES","Score":null,"Total":0}
引用次数: 5
Abstract
Due to its versatility, the Soil and Water Assessment Tool (SWAT) has been widely applied to investigate the effects of management activities and climate change on water availability and quality. However, the use of high spatial resolution data and the advantages of SWAT itself have significantly increased the input/output (I/O) demand and thus the runtime of modeling routines that require a large number of iterative simulations. In this study, we proposed a generic scheme to reduce the SWAT runtime by caching the model inputs using the in-memory NoSQL database Redis. Then the SWAT source codes (rev 488) was modified according to this proposed scheme to develop the MA-SWAT (memory accelerated SWAT) model by incorporating a new subroutine known as Fortran_calls_c to retrieve the cached inputs. We then evaluated MA-SWAT with four synthetic hydrological models and five different parallel schemes in a quad-core commodity laptop. The test results showed that when applied with a parallel simulation program, MA-SWAT could achieve a speedup by a factor of 8.4 ~ 10.9 depending on model complexity. Compared with the original SWAT, MA-SWAT significantly improved the computation speed, indicating that the proposed scheme is a desirable method for solving high computational demand problems such as calibration, sensitivity and uncertainty analysis. Moreover, the proposed concept of linking the SWAT model with Redis via the minimalistic C client driver of Redis is a generic method, and it is possible to apply this method to other Fortran-implemented environmental model to alleviate I/O demands.
期刊介绍:
Journal of Environmental Informatics (JEI) is an international, peer-reviewed, and interdisciplinary publication designed to foster research innovation and discovery on basic science and information technology for addressing various environmental problems. The journal aims to motivate and enhance the integration of science and technology to help develop sustainable solutions that are consensus-oriented, risk-informed, scientifically-based and cost-effective. JEI serves researchers, educators and practitioners who are interested in theoretical and/or applied aspects of environmental science, regardless of disciplinary boundaries. The topics addressed by the journal include:
- Planning of energy, environmental and ecological management systems
- Simulation, optimization and Environmental decision support
- Environmental geomatics - GIS, RS and other spatial information technologies
- Informatics for environmental chemistry and biochemistry
- Environmental applications of functional materials
- Environmental phenomena at atomic, molecular and macromolecular scales
- Modeling of chemical, biological and environmental processes
- Modeling of biotechnological systems for enhanced pollution mitigation
- Computer graphics and visualization for environmental decision support
- Artificial intelligence and expert systems for environmental applications
- Environmental statistics and risk analysis
- Climate modeling, downscaling, impact assessment, and adaptation planning
- Other areas of environmental systems science and information technology.