SapFlower: an automated tool for sap flow data preprocessing, gap-filling, and analysis using deep learning

Abstract

Introduction

Carbon and water cycles are fundamental to the Earth's climate system, with plants playing a key role through processes such as transpiration and carbon sequestration. Measuring plant water use is essential for understanding how ecosystems respond to changing environmental conditions, particularly in the context of climate change (Niu et al., 2011). Sap flow, a measure of the movement of water through a plant's vascular system, is a critical indicator of plant transpiration and water use (Meinzer et al., 2004). Understanding sap flow dynamics is essential for studying plant physiology, plant hydraulic functioning, budget of watersheds, ecosystem water cycles, and their responses to environmental stressors such as drought (Wilson et al., 2001; Meinzer et al., 2004; Steppe et al., 2015; Zhu et al., 2017). While recent efforts have advanced the synthesis of global sap flow data, such as SAPFLUXNET data, challenges in raw data cleaning and gap-filling continue to limit its broader accessibility and utility within the global research community (Poyatos et al., 2016; Peters et al., 2018).

The thermal dissipation probe (TDP) method, introduced by Granier (1985), is a widely used, cost-effective technique for measuring sap flow in plants, particularly trees, by quantifying thermal dissipation (TD) caused by xylem sap flow (Granier, 1987; Smith & Allen, 1996). It operates on the principle that sap flow cools a heated probe inserted into the sapwood, with the cooling rate proportional to sap flow velocity. The method employs two radially inserted probes: a heated probe, continuously warmed by an electric current, and a reference probe, which measures ambient wood temperature. The temperature difference ($\mathrm{\Delta }T$) between the probes decreases as sap flow increases, with the maximum temperature difference ($\mathrm{\Delta }{T}_{\mathrm{M}}$) occurring when sap flow is absent. Sap flux density ($F$) is commonly expressed as Eqn 1.

$$ (Eqn 1)

where $$ represents sap flux density, and $$ and $$ are the empirical coefficients derived from Granier's original formulation. The flow index $$ is defined as Eqn 2.

$$ (Eqn 2)

This method is particularly effective for studying plant–water relations and transpiration, although accurate probe placement, calibration, and consideration of wood thermal properties are crucial for reliable measurements (Lu et al., 2004; Masmoudi et al., 2012; Alizadeh et al., 2018). Despite TDP's widespread adoption, sap flow data collected via this method often contain noise, outliers, and gaps due to environmental factors, sensor failures, or data collection interruptions (Oishi et al., 2016). These challenges necessitate efficient data processing, cleaning, and gap-filling methods to ensure robust, accurate, and reliable analysis.

Current tools and methods for sap flow data processing range from simple spreadsheet-based approaches to more sophisticated software/packages designed for specific datasets (Oishi et al., 2016; Peters et al., 2021). However, existing tools require substantial manual intervention, particularly for data cleaning, modeling, and gap-filling. These manual processes often introduce variability in data quality, increasing the risk of generating inconsistent or unstandardized data across different sap flow processing approaches. This, in turn, can lead to inaccurate outcomes. Moreover, the absence of standardized protocols significantly hinders the feasibility of large-scale or long-term studies, in which reliable and automated approaches are essential. Recent studies have shown that state-of-the-art machine learning and deep learning algorithms are superior in gap-filling time-series flux measurements such as eddy covariance and sap flow data, and the most used algorithms are recurrent neural networks (RNNs) and machine learning models, such as support vector regression (SVR) and random forest (RF) (Li et al., 2022; Zhang et al., 2023; Lucarini et al., 2024; Yu et al., 2024). However, most of those algorithms are not available or accessible to those who have no or limited programming skills. The necessity for standardized, automated, and flexible tools for sap flow data processing, gap-filling, and analysis using state-of-the-art techniques has become increasingly apparent. Large ecological datasets, particularly those involving continuous long-term monitoring, are challenging to manage without efficient and reproducible workflows. Automated tools that incorporate advanced machine learning techniques for gap-filling and analysis have the potential to substantially improve the accuracy and efficiency of sap flow data handling, providing new insights into plant water use under varying environmental conditions.

In this study, we present SapFlower, a novel application designed to meet the growing demand for automated sap flow data processing. SapFlower integrates a comprehensive pipeline for data cleaning, preprocessing, model training, gap-filling, sapwood area modeling, and water use analysis. By leveraging advanced machine learning models, including RNNs and RF, SapFlower predicts and fills missing data points using time stamps and user-defined key environmental variables as predictors. By providing a standardized framework, it streamlines sap flow data processing, reducing inconsistencies and minimizing the impact of methodological variations. Beyond automation, SapFlower offers a versatile tool for researchers to analyze sap flow dynamics across different species, ecosystems, and environmental conditions. This study highlights SapFlower's effectiveness in efficiently processing and analyzing sap flow data, supporting its application in long-term plant water use studies and ecosystem monitoring.