ACM Transactions on Modeling and Computer Simulation最新文献_第5页

DSMC Evaluation Stages: Fostering Robust and Safe Behavior in Deep Reinforcement Learning – Extended Version DSMC评估阶段:在深度强化学习中培养稳健和安全的行为-扩展版

IF 0.9 4区计算机科学

ACM Transactions on Modeling and Computer Simulation Pub Date : 2023-07-12 DOI: 10.1145/3607198

Timo P. Gros, D. Höller, Jörg Hoffmann, M. Klauck, Hendrik Meerkamp, Verena Wolf

{"title":"DSMC Evaluation Stages: Fostering Robust and Safe Behavior in Deep Reinforcement Learning – Extended Version","authors":"Timo P. Gros, D. Höller, Jörg Hoffmann, M. Klauck, Hendrik Meerkamp, Verena Wolf","doi":"10.1145/3607198","DOIUrl":"https://doi.org/10.1145/3607198","url":null,"abstract":"Neural networks (NN) are gaining importance in sequential decision-making. Deep reinforcement learning (DRL), in particular, is extremely successful in learning action policies in complex and dynamic environments. Despite this success, however, DRL technology is not without its failures, especially in safety-critical applications: (i) the training objective maximizes average rewards, which may disregard rare but critical situations and hence lack local robustness; (ii) optimization objectives targeting safety typically yield degenerated reward structures which for DRL to work must be replaced with proxy objectives. Here we introduce a methodology that can help to address both deficiencies. We incorporate evaluation stages (ES) into DRL, leveraging recent work on deep statistical model checking (DSMC), which verifies NN policies in Markov decision processes. Our ES apply DSMC at regular intervals to determine state space regions with weak performance. We adapt the subsequent DRL training priorities based on the outcome, (i) focusing DRL on critical situations, and (ii) allowing to foster arbitrary objectives. We run case studies on two benchmarks. One of them is the Racetrack, an abstraction of autonomous driving that requires navigating a map without crashing into a wall. The other is MiniGrid, a widely used benchmark in the AI community. Our results show that DSMC-based ES can significantly improve both (i) and (ii).","PeriodicalId":50943,"journal":{"name":"ACM Transactions on Modeling and Computer Simulation","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43824767","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 6

Optimizing reachability probabilities for a restricted class of Stochastic Hybrid Automata via Flowpipe-Construction 基于流动管道构造的受限类随机混合自动机可达概率优化

IF 0.9 4区计算机科学

ACM Transactions on Modeling and Computer Simulation Pub Date : 2023-07-11 DOI: https://dl.acm.org/doi/10.1145/3607197

Carina da Silva, Stefan Schupp, Anne Remke

{"title":"Optimizing reachability probabilities for a restricted class of Stochastic Hybrid Automata via Flowpipe-Construction","authors":"Carina da Silva, Stefan Schupp, Anne Remke","doi":"https://dl.acm.org/doi/10.1145/3607197","DOIUrl":"https://doi.org/https://dl.acm.org/doi/10.1145/3607197","url":null,"abstract":"Stochastic hybrid automata (SHA) are a powerful tool to evaluate the dependability and safety of critical infrastructures. However, the resolution of nondeterminism, which is present in many purely hybrid models, is often only implicitly considered in SHA. This paper instead proposes algorithms for computing maximum and minimum reachability probabilities for singular automata with urgent transitions and random clocks which follow arbitrary continuous probability distributions. We borrow a well-known approach from hybrid systems reachability analysis, namely flowpipe construction, which is then extended to optimize nondeterminism in the presence of random variables. Firstly, valuations of random clocks which ensure reachability of specific goal states are extracted from the computed flowpipes and secondly, reachability probabilities are computed by integrating over these valuations. We compute maximum and minimum probabilities for history-dependent prophetic and non-prophetic schedulers using set-based methods. The implementation featuring the library HyPro and the complexity of the approach are discussed in detail. Two case studies featuring nondeterministic choices show the feasibility of the approach.","PeriodicalId":50943,"journal":{"name":"ACM Transactions on Modeling and Computer Simulation","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138523776","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Optimizing reachability probabilities for a restricted class of Stochastic Hybrid Automata via Flowpipe-Construction 用Flowpipe构造优化一类受限随机混合自动机的可达性概率

IF 0.9 4区计算机科学

ACM Transactions on Modeling and Computer Simulation Pub Date : 2023-07-11 DOI: 10.1145/3607197

Carina Pilch, Stefan Schupp, Anne Remke

{"title":"Optimizing reachability probabilities for a restricted class of Stochastic Hybrid Automata via Flowpipe-Construction","authors":"Carina Pilch, Stefan Schupp, Anne Remke","doi":"10.1145/3607197","DOIUrl":"https://doi.org/10.1145/3607197","url":null,"abstract":"Stochastic hybrid automata (SHA) are a powerful tool to evaluate the dependability and safety of critical infrastructures. However, the resolution of nondeterminism, which is present in many purely hybrid models, is often only implicitly considered in SHA. This paper instead proposes algorithms for computing maximum and minimum reachability probabilities for singular automata with urgent transitions and random clocks which follow arbitrary continuous probability distributions. We borrow a well-known approach from hybrid systems reachability analysis, namely flowpipe construction, which is then extended to optimize nondeterminism in the presence of random variables. Firstly, valuations of random clocks which ensure reachability of specific goal states are extracted from the computed flowpipes and secondly, reachability probabilities are computed by integrating over these valuations. We compute maximum and minimum probabilities for history-dependent prophetic and non-prophetic schedulers using set-based methods. The implementation featuring the library HyPro and the complexity of the approach are discussed in detail. Two case studies featuring nondeterministic choices show the feasibility of the approach.","PeriodicalId":50943,"journal":{"name":"ACM Transactions on Modeling and Computer Simulation","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-07-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46721864","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Toward Data Center Digital Twins via Knowledge-based Model Calibration and Reduction 基于知识的数据中心数字孪生模型校正与约简

IF 0.9 4区计算机科学

ACM Transactions on Modeling and Computer Simulation Pub Date : 2023-06-10 DOI: https://dl.acm.org/doi/10.1145/3604283

Ruihang Wang, Deneng Xia, Zhiwei Cao, Yonggang Wen, Rui Tan, Xin Zhou

{"title":"Toward Data Center Digital Twins via Knowledge-based Model Calibration and Reduction","authors":"Ruihang Wang, Deneng Xia, Zhiwei Cao, Yonggang Wen, Rui Tan, Xin Zhou","doi":"https://dl.acm.org/doi/10.1145/3604283","DOIUrl":"https://doi.org/https://dl.acm.org/doi/10.1145/3604283","url":null,"abstract":"Computational fluid dynamics (CFD) models have been widely used for prototyping data centers. Evolving them into high-fidelity and real-time digital twins is desirable for online operations of data centers. However, CFD models often have unsatisfactory accuracy and high computation overhead. Manually calibrating the CFD model parameters is tedious and labor-intensive. Existing automatic calibration approaches apply heuristics to search the model configurations. However, each search step requires a long-lasting process of repeatedly solving the CFD model, rendering them impractical especially for complex CFD models. This paper presents Kalibre, a knowledge-based neural surrogate approach that calibrates a CFD model by iterating four steps of i) training a neural surrogate model, ii) finding the optimal parameters through neural surrogate retraining, iii) configuring the found parameters back to the CFD model, and iv) validating the CFD model using sensor-measured data. Thus, the parameter search is offloaded to the lightweight neural surrogate. To speed up Kalibre’s convergence, we incorporate prior knowledge in training data initialization and surrogate architecture design. With about ten hours computation on a 64-core processor, Kalibre achieves mean absolute errors (MAEs) of 0.57°C and 0.88°C in calibrating the CFD models of two production data halls hosting thousands of servers. To accelerate CFD-based simulation, we further propose Kalibreduce that incorporates the energy balance principle to reduce the order of the calibrated CFD model. Evaluation shows the model reduction only introduces 0.1°C to 0.27°C extra errors, while accelerating the CFD-based simulations by thousand times.","PeriodicalId":50943,"journal":{"name":"ACM Transactions on Modeling and Computer Simulation","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138523750","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Toward Data Center Digital Twins via Knowledge-based Model Calibration and Reduction 基于知识的数据中心数字孪生模型校正与约简

IF 0.9 4区计算机科学

ACM Transactions on Modeling and Computer Simulation Pub Date : 2023-06-10 DOI: 10.1145/3604283

Ruihang Wang, Deneng Xia, Zhi-Ying Cao, Yonggang Wen, Rui Tan, Xiaoxia Zhou

{"title":"Toward Data Center Digital Twins via Knowledge-based Model Calibration and Reduction","authors":"Ruihang Wang, Deneng Xia, Zhi-Ying Cao, Yonggang Wen, Rui Tan, Xiaoxia Zhou","doi":"10.1145/3604283","DOIUrl":"https://doi.org/10.1145/3604283","url":null,"abstract":"Computational fluid dynamics (CFD) models have been widely used for prototyping data centers. Evolving them into high-fidelity and real-time digital twins is desirable for online operations of data centers. However, CFD models often have unsatisfactory accuracy and high computation overhead. Manually calibrating the CFD model parameters is tedious and labor-intensive. Existing automatic calibration approaches apply heuristics to search the model configurations. However, each search step requires a long-lasting process of repeatedly solving the CFD model, rendering them impractical especially for complex CFD models. This paper presents Kalibre, a knowledge-based neural surrogate approach that calibrates a CFD model by iterating four steps of i) training a neural surrogate model, ii) finding the optimal parameters through neural surrogate retraining, iii) configuring the found parameters back to the CFD model, and iv) validating the CFD model using sensor-measured data. Thus, the parameter search is offloaded to the lightweight neural surrogate. To speed up Kalibre’s convergence, we incorporate prior knowledge in training data initialization and surrogate architecture design. With about ten hours computation on a 64-core processor, Kalibre achieves mean absolute errors (MAEs) of 0.57°C and 0.88°C in calibrating the CFD models of two production data halls hosting thousands of servers. To accelerate CFD-based simulation, we further propose Kalibreduce that incorporates the energy balance principle to reduce the order of the calibrated CFD model. Evaluation shows the model reduction only introduces 0.1°C to 0.27°C extra errors, while accelerating the CFD-based simulations by thousand times.","PeriodicalId":50943,"journal":{"name":"ACM Transactions on Modeling and Computer Simulation","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42854944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Uncertainty-aware Simulation of Adaptive Systems 自适应系统的不确定性感知仿真

IF 0.9 4区计算机科学

ACM Transactions on Modeling and Computer Simulation Pub Date : 2023-05-13 DOI: https://dl.acm.org/doi/10.1145/3589517

Jean-Marc Jézéquel, Antonio Vallecillo

引用次数: 0

NIM: Generative Neural Networks for Automated Modeling and Generation of Simulation Inputs NIM:用于自动建模和生成仿真输入的生成神经网络

IF 0.9 4区计算机科学

ACM Transactions on Modeling and Computer Simulation Pub Date : 2023-04-19 DOI: 10.1145/3592790

Wang Cen, Peter J. Haas

{"title":"NIM: Generative Neural Networks for Automated Modeling and Generation of Simulation Inputs","authors":"Wang Cen, Peter J. Haas","doi":"10.1145/3592790","DOIUrl":"https://doi.org/10.1145/3592790","url":null,"abstract":"Fitting stochastic input-process models to data and then sampling from them are key steps in a simulation study but highly challenging to non-experts. We present Neural Input Modeling (NIM), a Generative Neural Network (GNN) framework that exploits modern data-rich environments to automatically capture simulation input processes and then generate samples from them. The basic GNN that we develop, called NIM-VL, comprises (i) a variational autoencoder architecture that learns the probability distribution of the input data while avoiding overfitting and (ii) long short-term memory components that concisely capture statistical dependencies across time. We show how the basic GNN architecture can be modified to exploit known distributional properties—such as independent and identically distributed structure, nonnegativity, and multimodality—to increase accuracy and speed, as well as to handle multivariate processes, categorical-valued processes, and extrapolation beyond the training data for certain nonstationary processes. We also introduce an extension to NIM called Conditional Neural Input Modeling (CNIM), which can learn from training data obtained under various realizations of a (possibly time series valued) stochastic “condition,” such as temperature or inflation rate, and then generate sample paths given a value of the condition not seen in the training data. This enables users to simulate a system under a specific working condition by customizing a pre-trained model; CNIM also facilitates what-if analysis. Extensive experiments show the efficacy of our approach. NIM can thus help overcome one of the key barriers to simulation for non-experts.","PeriodicalId":50943,"journal":{"name":"ACM Transactions on Modeling and Computer Simulation","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-04-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48355138","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

NIM: Generative Neural Networks for Automated Modeling and Generation of Simulation Inputs NIM:用于自动建模和生成仿真输入的生成神经网络

IF 0.9 4区计算机科学

ACM Transactions on Modeling and Computer Simulation Pub Date : 2023-04-19 DOI: https://dl.acm.org/doi/10.1145/3592790

Wang Cen, Peter J. Haas

{"title":"NIM: Generative Neural Networks for Automated Modeling and Generation of Simulation Inputs","authors":"Wang Cen, Peter J. Haas","doi":"https://dl.acm.org/doi/10.1145/3592790","DOIUrl":"https://doi.org/https://dl.acm.org/doi/10.1145/3592790","url":null,"abstract":"Fitting stochastic input-process models to data and then sampling from them are key steps in a simulation study, but highly challenging to non-experts. We present Neural Input Modeling (NIM), a generative-neural-network (GNN) framework that exploits modern data-rich environments to automatically capture simulation input processes and then generate samples from them. The basic GNN that we develop, called NIM-VL, comprises (i) a variational-autoencoder (VAE) architecture that learns the probability distribution of the input data while avoiding overfitting and (ii) Long Short-Term Memory (LSTM) components that concisely capture statistical dependencies across time. We show how the basic GNN architecture can be modified to exploit known distributional properties—such as i.i.d. structure, nonnegativity, and multimodality—in order to increase accuracy and speed, as well as to handle multivariate processes, categorical-valued processes, and extrapolation beyond the training data for certain nonstationary processes. We also introduce an extension to NIM called “conditional” NIM (CNIM), which can learn from training data obtained under various realizations of a (possibly time-series-valued) stochastic “condition”, such as temperature or inflation rate, and then generate sample paths given a value of the condition not seen in the training data. This enables users to simulate a system under a specific working condition by customizing a pre-trained model; CNIM also facilitates what-if analysis. Extensive experiments show the efficacy of our approach. NIM can thus help overcome one of the key barriers to simulation for non-experts.","PeriodicalId":50943,"journal":{"name":"ACM Transactions on Modeling and Computer Simulation","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-04-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138523754","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Compositional safe approximation of response time probability density function of complex workflows 复杂工作流响应时间概率密度函数的组合安全逼近

IF 0.9 4区计算机科学

ACM Transactions on Modeling and Computer Simulation Pub Date : 2023-04-05 DOI: https://dl.acm.org/doi/10.1145/3591205

Laura Carnevali, Marco Paolieri, Riccardo Reali, Enrico Vicario

{"title":"Compositional safe approximation of response time probability density function of complex workflows","authors":"Laura Carnevali, Marco Paolieri, Riccardo Reali, Enrico Vicario","doi":"https://dl.acm.org/doi/10.1145/3591205","DOIUrl":"https://doi.org/https://dl.acm.org/doi/10.1145/3591205","url":null,"abstract":"We evaluate a stochastic upper bound on the response time Probability Density Function (PDF) of complex workflows through an efficient and accurate compositional approach. Workflows consist of activities having generally distributed stochastic durations with bounded supports, composed through sequence, choice/merge, and balanced/unbalanced split/join operators, possibly breaking the structure of well-formed nesting. Workflows are specified using a formalism defined in terms of Stochastic Time Petri Nets (STPNs), that permits decomposition into a hierarchy of subworkflows with positively correlated response times, guaranteeing that a stochastically larger end-to-end response time PDF is obtained when intermediate results are approximated by stochastically larger PDFs and when dependencies are simplified by replicating activities appearing in multiple subworkflows. In particular, an accurate stochastically larger PDF is obtained by combining shifted truncated Exponential terms with positive or negative rates. Experiments are performed on sets of manually and randomly generated models with increasing complexity, illustrating under which conditions different decomposition heuristics work well in terms of accuracy and complexity, and showing that the proposed approach outperforms simulation having the same execution time.","PeriodicalId":50943,"journal":{"name":"ACM Transactions on Modeling and Computer Simulation","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138523747","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Compositional safe approximation of response time probability density function of complex workflows 复杂工作流响应时间概率密度函数的组合安全逼近

IF 0.9 4区计算机科学

ACM Transactions on Modeling and Computer Simulation Pub Date : 2023-04-05 DOI: 10.1145/3591205

L. Carnevali, Marco Paolieri, R. Reali, E. Vicario

{"title":"Compositional safe approximation of response time probability density function of complex workflows","authors":"L. Carnevali, Marco Paolieri, R. Reali, E. Vicario","doi":"10.1145/3591205","DOIUrl":"https://doi.org/10.1145/3591205","url":null,"abstract":"We evaluate a stochastic upper bound on the response time Probability Density Function (PDF) of complex workflows through an efficient and accurate compositional approach. Workflows consist of activities having generally distributed stochastic durations with bounded supports, composed through sequence, choice/merge, and balanced/unbalanced split/join operators, possibly breaking the structure of well-formed nesting. Workflows are specified using a formalism defined in terms of Stochastic Time Petri Nets (STPNs), that permits decomposition into a hierarchy of subworkflows with positively correlated response times, guaranteeing that a stochastically larger end-to-end response time PDF is obtained when intermediate results are approximated by stochastically larger PDFs and when dependencies are simplified by replicating activities appearing in multiple subworkflows. In particular, an accurate stochastically larger PDF is obtained by combining shifted truncated Exponential terms with positive or negative rates. Experiments are performed on sets of manually and randomly generated models with increasing complexity, illustrating under which conditions different decomposition heuristics work well in terms of accuracy and complexity, and showing that the proposed approach outperforms simulation having the same execution time.","PeriodicalId":50943,"journal":{"name":"ACM Transactions on Modeling and Computer Simulation","volume":null,"pages":null},"PeriodicalIF":0.9,"publicationDate":"2023-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47466124","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1