Learning From Limited Temporal Data: Dynamically Sparse Historical Functional Linear Models With Applications to Earth Science

Abstract

Scientists and statisticians often seek to understand the complex relationships that connect two time-varying variables. Recent work on sparse functional historical linear models confirms that they are promising as a tool for obtaining complex and interpretable inferences, but several notable limitations exist. Most importantly, previous works have imposed sparsity on the historical coefficient function, but have not allowed the sparsity, hence lag, to vary with time. We simplify the framework of sparse functional historical linear models by using a rectangular coefficient structure along with Whittaker smoothing, then reduce the assumptions of the previous frameworks by estimating the dynamic time lag from a hierarchical coefficient structure. We motivate our study by aiming to extract the physical rainfall–runoff processes hidden within hydrological data. We show the promise and accuracy of our method using eight simulation studies, further justified by two real sets of hydrological data.