{"title":"A neural ordinary differential equation framework for modeling inelastic stress response via internal state variables","authors":"R. Jones, A. Frankel, K. Johnson","doi":"10.1615/jmachlearnmodelcomput.2022042917","DOIUrl":"https://doi.org/10.1615/jmachlearnmodelcomput.2022042917","url":null,"abstract":"We propose a neural network framework to preclude the need to define or observe incompletely or inaccurately defined states of a material in order to describe its response. The neural network design is based on the classical Coleman-Gurtin internal state variable theory. In the proposed framework the states of the material are inferred from observable deformation and stress. A neural network describes the flow of internal states and another represents the map from internal state and strain to stress. We investigate tensor basis, component, and potential-based formulations of the stress model. Violations of the second law of thermodynamics are prevented by a constraint on the weights of the neural network. We extend this framework to homogenization of materials with microstructure with a graph-based convolutional neural network that preprocesses the initial microstructure into salient features. The modeling framework is tested on large datasets spanning inelastic material classes to demonstrate its general applicability.","PeriodicalId":320604,"journal":{"name":"Journal of Machine Learning for Modeling and Computing","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125003280","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Leveraging Local Variation in Data: Sampling and Weighting Schemes for Supervised Deep Learning","authors":"Paul Novello, G. Poette, D. Lugato, P. Congedo","doi":"10.1615/JMachLearnModelComput.2022041819","DOIUrl":"https://doi.org/10.1615/JMachLearnModelComput.2022041819","url":null,"abstract":"In the context of supervised learning of a function by a neural network, we claim and empirically verify that the neural network yields better results when the distribution of the data set focuses on regions where the function to learn is steep. We first traduce this assumption in a mathematically workable way using Taylor expansion and emphasize a new training distribution based on the derivatives of the function to learn. Then, theoretical derivations allow constructing a methodology that we call Variance Based Samples Weighting (VBSW). VBSW uses labels local variance to weight the training points. This methodology is general, scalable, cost-effective, and significantly increases the performances of a large class of neural networks for various classification and regression tasks on image, text, and multivariate data. We highlight its benefits with experiments involving neural networks from linear models to ResNet and Bert.","PeriodicalId":320604,"journal":{"name":"Journal of Machine Learning for Modeling and Computing","volume":"154 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-01-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134035864","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hannah Lu, D. Ermakova, H. Wainwright, Liange Zheng, D. Tartakovsky
{"title":"Data-informed Emulators for Multi-Physics Simulations","authors":"Hannah Lu, D. Ermakova, H. Wainwright, Liange Zheng, D. Tartakovsky","doi":"10.1615/jmachlearnmodelcomput.2021038577","DOIUrl":"https://doi.org/10.1615/jmachlearnmodelcomput.2021038577","url":null,"abstract":"Machine learning techniques are powerful tools for construction of emulators for complex systems. We explore different machine learning methods and conceptual methodologies, ranging from functional approximations to dynamical approximations, to build such emulators for coupled thermal, hydrological, mechanical and chemical processes that occur near an engineered barrier system in the nuclear waste repository. Two nonlinear approximators, random forests and neural networks, are deployed to capture the complexity of the physics-based model and to identify its most significant hydrological and geochemical parameters. Our emulators capture the temporal evolution of the Uranium distribution coefficient of the clay buffer, and identify its functional dependence on these key parameters. The emulators' accuracy is further enhanced by assimilating relevant simulated predictors and clustering strategy. The relative performance of random forests and neural networks shows the advantage of ensemble learning in random forests algorithm, especially for highly nonlinear problems with limited data.","PeriodicalId":320604,"journal":{"name":"Journal of Machine Learning for Modeling and Computing","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-12-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132649122","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
R. Tipireddy, P. Perdikaris, P. Stinis, A. Tartakovsky
{"title":"Multistep and continuous physics-informed neural network methods for learning governing equations and constitutive relations","authors":"R. Tipireddy, P. Perdikaris, P. Stinis, A. Tartakovsky","doi":"10.1615/jmachlearnmodelcomput.2022041787","DOIUrl":"https://doi.org/10.1615/jmachlearnmodelcomput.2022041787","url":null,"abstract":"We investigate the applicability and relative merit of discrete and continuous versions of physics-informed neural network (PINN) methods for learning unknown governing equations or constitutive relations in a nonlinear dynamical system. In the case of unknown dynamics, entire right-hand-side (RHS) equations of the ordinary differential equations are unknown. In the case of unknown constitutive relations, however, the RHS equations are known up to the specification of constitutive relations (that may depend on the state of the system). We use a deep neural network to model unknown governing equations or constitutive relations. The discrete PINN approach combines clas-sical multistep discretization methods for dynamical systems with neural-network-based machine learning methods. On the other hand, the continuous versions utilize deep neural networks to minimize the residual function for the continuous governing equations. We use the case of a fedbatch bioreactor system to study the effectiveness of these approaches and discuss conditions for their applicability. Our results indicate that the accuracy of the trained neural network models is much higher for the cases where we only have to learn a constitutive relation instead of all dynamics. This finding corroborates the well-known fact from scientific computing that building as much structural information as is available into an algorithm can enhance its efficiency and/or accuracy.","PeriodicalId":320604,"journal":{"name":"Journal of Machine Learning for Modeling and Computing","volume":"27 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131466905","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Preface for Special Issue on Machine Learning Techniques for Applications in Sustainability Research","authors":"V. Ciriello, D. Tartakovsky","doi":"10.1615/jmachlearnmodelcomput.2022045267","DOIUrl":"https://doi.org/10.1615/jmachlearnmodelcomput.2022045267","url":null,"abstract":"","PeriodicalId":320604,"journal":{"name":"Journal of Machine Learning for Modeling and Computing","volume":"122 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116839423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Junqi Yin, Siyan Liu, Viktor Reshniak, Xiaoping Wang, Guannan Zhang
{"title":"A scalable transformer model for real-time decision making in neutron scattering experiments","authors":"Junqi Yin, Siyan Liu, Viktor Reshniak, Xiaoping Wang, Guannan Zhang","doi":"10.1615/jmachlearnmodelcomput.2023048607","DOIUrl":"https://doi.org/10.1615/jmachlearnmodelcomput.2023048607","url":null,"abstract":"","PeriodicalId":320604,"journal":{"name":"Journal of Machine Learning for Modeling and Computing","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124112360","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Assessment of Neuroimaging Data and Identification of Alzheimer's Disease using Extreme Learning Machines","authors":"D. Raghavan, Suma K V, Puneeth N Ganesh","doi":"10.1615/jmachlearnmodelcomput.2023048413","DOIUrl":"https://doi.org/10.1615/jmachlearnmodelcomput.2023048413","url":null,"abstract":"Alzheimer’s disease (AD), one of the most common forms of dementia, is a cognitive disorder that is progressive in nature and causes a dynamic deterioration of the mental state of an individual. It severely damages the brain cells, neurotransmitters, and nerves, leading to irreparable damage to the brain, which is one of the major causes of dementia. Early identification, assessment, and timely diagnosis are of paramount importance to slow down the progression of the disease, which calls for the design and development of algorithms and technology-aided tools for accurate detection, diagnosis, and prediction of the severity of Alzheimer’s disease. To provide a solution to this, we propose an extreme learning machine (ELM) algorithm that is trained on neuroimaging data from longitudinal MRI scans obtained from the OASIS database. We adopt an extensive feature engineering pipeline to choose the most significant features for early identification of the onset of dementia. We obtain an overall accuracy of 98.3%, sensitivity of 0.956, specificity of 0.962, and F1 score of 0.972. We also show that our proposed ELM algorithm outperforms several other contemporary classifiers based on a range of evaluation metrics. The paper also provides a feasibility analysis of the proposed model for real-time clinical deployment.","PeriodicalId":320604,"journal":{"name":"Journal of Machine Learning for Modeling and Computing","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128841592","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Physics-Informed Neural Networks for Informed Vaccine Distribution in Meta-Populations","authors":"Alvan Arulandu, P. Seshaiyer","doi":"10.1615/jmachlearnmodelcomput.2023047642","DOIUrl":"https://doi.org/10.1615/jmachlearnmodelcomput.2023047642","url":null,"abstract":"","PeriodicalId":320604,"journal":{"name":"Journal of Machine Learning for Modeling and Computing","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116326786","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Letter from the Editors: The emerging need for biologically-inspired and mathematically-guided machine learning for knowledge discovery in biology","authors":"Ritambhara Singh, P. Seshaiyer","doi":"10.1615/jmachlearnmodelcomput.2023049680","DOIUrl":"https://doi.org/10.1615/jmachlearnmodelcomput.2023049680","url":null,"abstract":"","PeriodicalId":320604,"journal":{"name":"Journal of Machine Learning for Modeling and Computing","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130431525","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
D. Dwivedi, U. Mital, B. Faybishenko, B. Dafflon, C. Varadharajan, Deb Agarwal, K. Williams, C. Steefel, Susan Hubbard
{"title":"Imputation of Contiguous Gaps and Extremes of Subhourly Groundwater Time Series Using Random Forests","authors":"D. Dwivedi, U. Mital, B. Faybishenko, B. Dafflon, C. Varadharajan, Deb Agarwal, K. Williams, C. Steefel, Susan Hubbard","doi":"10.1615/jmachlearnmodelcomput.2021038774","DOIUrl":"https://doi.org/10.1615/jmachlearnmodelcomput.2021038774","url":null,"abstract":"Machine learning can provide sustainable solutions to gap-fill groundwater (GW) data needed to adequately constrain watershed models. However, imputing missing extremes is more challenging than other parts of a hydrograph. To impute missing subhourly data, including extremes, within GW time-series data collected at multiple wells in the East River watershed, located in southwestern Colorado, we consider a single-well imputation (SWI) and a multiple-well imputation (MWI) approach. SWI gap-fills missing GW entries in a well using the same well’s time-series data; MWI gap-fills a specific well’s missing GW entry using the time series of neighboring wells. SWI takes advantage of linear interpolation and random forest (RF) approaches, whereas MWI exploits only the RF approach. We also use an information entropy framework to develop insights into how missing data patterns impact imputation. We discovered that if gaps were at random intervals, SWI could accurately impute up to 90% of missing data over an approximately two-year period. Contiguous gaps constituted more complex scenarios for imputation and required the use of MWI. Information entropy suggested that if gaps were contiguous, up to 50% of missing GW data could be estimated accurately over an approximately two-year period. The RF-feature importance suggested that a time feature (months) and a space feature (neighboring wells) were the most important predictors in the SWI and MWI. We also noted that neither SWI nor MWI methods could capture the missing extremes of a hydrograph. To counter this, we developed a new sequential approach and demonstrated the imputation of missing extremes in a GW time series with high accuracy.","PeriodicalId":320604,"journal":{"name":"Journal of Machine Learning for Modeling and Computing","volume":"81 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133636231","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}