Journal of Verification, Validation and Uncertainty Quantification最新文献

{"title":"A Unifying Framework for Probabilistic Validation Metrics","authors":"P. Gardner, C. Lord, R. Barthorpe","doi":"10.1115/1.4045296","DOIUrl":"https://doi.org/10.1115/1.4045296","url":null,"abstract":"\u0000 Probabilistic modeling methods are increasingly being employed in engineering applications. These approaches make inferences about the distribution for output quantities of interest. A challenge in applying probabilistic computer models (simulators) is validating output distributions against samples from observational data. An ideal validation metric is one that intuitively provides information on key differences between the simulator output and observational distributions, such as statistical distances/divergences. Within the literature, only a small set of statistical distances/divergences have been utilized for this task; often selected based on user experience and without reference to the wider variety available. As a result, this paper offers a unifying framework of statistical distances/divergences, categorizing those implemented within the literature, providing a greater understanding of their benefits, and offering new potential measures as validation metrics. In this paper, two families of measures for quantifying differences between distributions, that encompass the existing statistical distances/divergences within the literature, are analyzed: f-divergence and integral probability metrics (IPMs). Specific measures from these families are highlighted, providing an assessment of current and new validation metrics, with a discussion of their merits in determining simulator adequacy, offering validation metrics with greater sensitivity in quantifying differences across the range of probability mass.","PeriodicalId":52254,"journal":{"name":"Journal of Verification, Validation and Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":0.6,"publicationDate":"2019-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42413232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Assessment of Model Validation, Calibration, and Prediction Approaches in the Presence of Uncertainty","authors":"N. W. Whiting","doi":"10.1115/1.4056285","DOIUrl":"https://doi.org/10.1115/1.4056285","url":null,"abstract":"\u0000 Model validation is the process of determining the degree to which a model is an accurate representation of the true value in the real world. The results of a model validation study can be used to either quantify the model form uncertainty or to improve/calibrate the model. However, the model validation process can become complicated if there is uncertainty in the simulation and/or experimental outcomes. These uncertainties can be in the form of aleatory uncertainties due to randomness or epistemic uncertainties due to lack of knowledge. Four different approaches are used for addressing model validation and calibration: 1) the area validation metric (AVM), 2) a modified area validation metric (MAVM) with confidence intervals, 3) the standard validation uncertainty from ASME V&V 20, and 4) Bayesian updating of a model discrepancy term. Details are given for the application of the MAVM for accounting for small experimental sample sizes. To provide an unambiguous assessment of these different approaches, synthetic experimental values is generated from computational fluid dynamics simulations of a multi-element airfoil. A simplified model is then developed using thin airfoil theory. This simplified model is then assessed using the synthetic experimental data. Each of these validation/calibration approaches are assessed for the ability to tightly encapsulate the true value in nature at locations both where experimental results are provided and prediction locations where no experimental data are available.","PeriodicalId":52254,"journal":{"name":"Journal of Verification, Validation and Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":0.6,"publicationDate":"2019-07-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43124512","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Statistics for Testing Under Adverse Conditions","authors":"L. Pease, K. Anderson, J. Bamberger, M. Minette","doi":"10.1115/1.4045117","DOIUrl":"https://doi.org/10.1115/1.4045117","url":null,"abstract":"\u0000 Here, we develop a statistical basis for limited adverse testing. This type of testing simultaneously evaluates system performance against minimum requirements and minimizes costs, particularly for large-scale engineering projects. Because testing is often expensive and narrow in scope, the data obtained are relatively limited—precisely the opposite of the recent big data movement but no less compelling. Although a remarkably common approach for industrial and large-scale government projects, a statistical basis for adverse testing remains poorly explored. Here, we prove mathematically, under specific conditions, that setting each independent variable to an adverse condition leads to a similar level of adversity in the dependent variable. For example, setting all normally distributed independent variables to at least their 95th percentile values leads to a result at the 95th percentile. The analysis considers sample size estimates to clarify the value of replicates in this type of testing, determines how many of the independent variables must be set to adverse condition values, and highlights the essential assumptions, so that engineers, statisticians, and subject matter experts know when this statistical framework may be applied successfully and design testing to satisfy statistical requisites.","PeriodicalId":52254,"journal":{"name":"Journal of Verification, Validation and Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":0.6,"publicationDate":"2019-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49505621","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Verification of Stress-Intensity Factor Solutions by Uncertainty Quantification","authors":"J. Sobotka, R. Mcclung","doi":"10.1115/1.4044868","DOIUrl":"https://doi.org/10.1115/1.4044868","url":null,"abstract":"\u0000 This paper summarizes an emerging process to establish credibility for surrogate models that cover multidimensional, continuous solution spaces. Various features lead to disagreement between the surrogate model's results and results from more precise computational benchmark solutions. In our verification process, this disagreement is quantified using descriptive statistics to support uncertainty quantification, sensitivity analysis, and surrogate model assessments. Our focus is stress-intensity factor (SIF) solutions. SIFs can be evaluated from simulations (e.g., finite element analyses), but these simulations require significant preprocessing, computational resources, and expertise to produce a credible result. It is not tractable (or necessary) to simulate a SIF for every crack front. Instead, most engineering analyses of fatigue crack growth (FCG) employ surrogate SIF solutions based on some combination of mechanics, interpolation, and SIF solutions extracted from earlier analyses. SIF values from surrogate solutions vary with local stress profiles and nondimensional degrees-of-freedom that define the geometry. The verification process evaluates the selected stress profiles and the sampled geometries using the surrogate model and a benchmark code (abaqus). The benchmark code employs a Python scripting interface to automate model development, execution, and extraction of key results. The ratio of the test code SIF to the benchmark code SIF measures the credibility of the solution. Descriptive statistics of these ratios provide convenient measures of relative surrogate quality. Thousands of analyses support visualization of the surrogate model's credibility, e.g., by rank-ordering of the credibility measure.","PeriodicalId":52254,"journal":{"name":"Journal of Verification, Validation and Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":0.6,"publicationDate":"2019-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47970700","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Numerical Errors in Unsteady Flow Simulations","authors":"L. Eça, G. Vaz, S. Toxopeus, M. Hoekstra","doi":"10.1115/1.4043975","DOIUrl":"https://doi.org/10.1115/1.4043975","url":null,"abstract":"This article discusses numerical errors in unsteady flow simulations, which may include round-off, statistical, iterative, and time and space discretization errors. The estimation of iterative and discretization errors and the influence of the initial condition on unsteady flows that become periodic are discussed. In this latter case, the goal is to determine the simulation time required to reduce the influence of the initial condition to negligible levels. Two one-dimensional, unsteady manufactured solutions are used to illustrate the interference between the different types of numerical errors. One solution is periodic and the other includes a transient region before it reaches a steady-state. The results show that for a selected grid and time-step, statistical convergence of the periodic solution may be achieved at significant lower error levels than those of iterative and discretization errors. However, statistical convergence deteriorates when iterative convergence criteria become less demanding, grids are refined, and Courant number increased.For statistically converged solutions of the periodic flow and for the transient solution, iterative convergence criteria required to obtain a negligible influence of the iterative error when compared to the discretization error are more strict than typical values found in the open literature. More demanding criteria are required when the grid is refined and/or the Courant number is increased. When the numerical error is dominated by the iterative error, it is pointless to refine the grid and/or reduce the time-step. For solutions with a numerical error dominated by the discretization error, three different techniques are applied to illustrate how the discretization uncertainty can be estimated, using grid/time refinement studies: three data points at a fixed Courant number; five data points involving three time steps for the same grid and three grids for the same time-step; five data points including at least two grids and two time steps. The latter two techniques distinguish between space and time convergence, whereas the first one combines the effect of the two discretization errors.","PeriodicalId":52254,"journal":{"name":"Journal of Verification, Validation and Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":0.6,"publicationDate":"2019-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1115/1.4043975","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45726190","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Prediction of Transient Statistical Energy Response for Two-Subsystem Models Considering Interval Uncertainty","authors":"Chen Qiang, Q. Fei, Shaoqing Wu, Yanbin Li","doi":"10.1115/1.4045201","DOIUrl":"https://doi.org/10.1115/1.4045201","url":null,"abstract":"\u0000 The transient response analysis is important for the design and evaluation of uncertain engineering systems under impact excitations. In this paper, statistical energy analysis (SEA) is developed to evaluate the high-frequency transient energy response of two-subsystem models considering interval uncertainties. Affine arithmetic (AA) and a subinterval technique are introduced into SEA to improve the computational accuracy. Numerical simulations on a coupled-plate and a plate-cavity system considering interval uncertainties are performed. The analysis precision of the proposed approach is validated by Monte Carlo (MC) method. The results show that the analysis precision of the proposed method decreases with the increasing uncertainty level of parameters. The computational accuracy of the proposed method can be significantly improved by employing AA and subinterval technique.","PeriodicalId":52254,"journal":{"name":"Journal of Verification, Validation and Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":0.6,"publicationDate":"2019-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48079516","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Bayesian Inference-Based Approach to Empirical Training of Strongly Coupled Constituent Models","authors":"G. Flynn, Evan Chodora, S. Atamturktur, D. Brown","doi":"10.1115/1.4044804","DOIUrl":"https://doi.org/10.1115/1.4044804","url":null,"abstract":"\u0000 Partitioned analysis enables numerical representation of complex systems through the coupling of smaller, simpler constituent models, each representing a different phenomenon, domain, scale, or functional component. Through this coupling, inputs and outputs of constituent models are exchanged in an iterative manner until a converged solution satisfies all constituents. In practical applications, numerical models may not be available for all constituents due to lack of understanding of the behavior of a constituent and the inability to conduct separate-effect experiments to investigate the behavior of the constituent in an isolated manner. In such cases, empirical representations of missing constituents have the opportunity to be inferred using integral-effect experiments, which capture the behavior of the system as a whole. Herein, we propose a Bayesian inference-based approach to estimate missing constituent models from available integral-effect experiments. Significance of this novel approach is demonstrated through the inference of a material plasticity constituent integrated with a finite element model to enable efficient multiscale elasto-plastic simulations.","PeriodicalId":52254,"journal":{"name":"Journal of Verification, Validation and Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":0.6,"publicationDate":"2019-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49587963","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"An Adaptive Response Surface Methodology Based on Active Subspaces for Mixed Random and Interval Uncertainties","authors":"Xingzhi Hu, Yanhui Duan, Ruili Wang, Xiao Liang, Jiangtao Chen","doi":"10.1115/1.4045200","DOIUrl":"https://doi.org/10.1115/1.4045200","url":null,"abstract":"\u0000 The popular use of response surface methodology (RSM) accelerates the solutions of parameter identification and response analysis issues. However, accurate RSM models subject to aleatory and epistemic uncertainties are still challenging to construct, especially for multidimensional inputs, which is widely existed in real-world problems. In this study, an adaptive interval response surface methodology (AIRSM) based on extended active subspaces is proposed for mixed random and interval uncertainties. Based on the idea of subspace dimension reduction, extended active subspaces are given for mixed uncertainties, and interval active variable representation is derived for the construction of AIRSM. A weighted response surface strategy is introduced and tested for predicting the accurate boundary. Moreover, an interval dynamic correlation index is defined, and significance check and cross validation are reformulated in active subspaces to evaluate the AIRSM. The effectiveness of AIRSM is demonstrated on two test examples: three-dimensional nonlinear function and speed reducer design. They both possess a dominant one-dimensional active subspace with small estimation error, and the accuracy of AIRSM is verified by comparing with full-dimensional Monte Carlo simulates, thus providing a potential template for tackling high-dimensional problems involving mixed aleatory and interval uncertainties.","PeriodicalId":52254,"journal":{"name":"Journal of Verification, Validation and Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":0.6,"publicationDate":"2019-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48853050","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Prediction of Structural Reliability Through an Alternative Variability-Based Methodology","authors":"K. Haas","doi":"10.1115/vvs2019-5150","DOIUrl":"https://doi.org/10.1115/vvs2019-5150","url":null,"abstract":"\u0000 The often-competing goals of optimization and reliability design amplify the importance of verification, validation, and uncertainty quantification (VVUQ) to achieve sufficient reliability. Evaluation of a system's reliability presents practical challenges given the large number of permutations of conditions that may exist over the system's operational lifecycle. Uncertainty and variability sources are not always well defined and are sometimes not possible to predict, yielding traditional uncertainty quantification (UQ) techniques insufficient. A variability-based method is proposed to bridge this gap in state-of-the-art UQ practice where sources of uncertainty and variability cannot be readily quantified. At the point of incipient structural failure, the structural response becomes highly variable and sensitive to minor perturbations in conditions. This characteristic provides a powerful opportunity to determine the critical failure conditions and to assess the resulting structural reliability through an alternative variability-based method. Nonhierarchical clustering, proximity analysis, and the use of stability indicators are combined to identify the loci of conditions that lead to a rapid evolution of the response toward a failure condition. The method's utility is demonstrated through its application to a simple nonlinear dynamic single-degree-of-freedom structural model. In addition to the L2 norm, a new stability indicator is proposed called the “instability index,” which is a function of both the L2 norm and the calculated proximity to adjacent loci of conditions with differing structural response. The instability index provides a rapidly achieved quantitative measure of the relative stability of the system for all possible loci of conditions.","PeriodicalId":52254,"journal":{"name":"Journal of Verification, Validation and Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":0.6,"publicationDate":"2019-05-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41794438","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Data Analysis and Model Validation of Natural Gas Transmission Pipeline With Compressor Station","authors":"David Cheng","doi":"10.1115/1.4045386","DOIUrl":"https://doi.org/10.1115/1.4045386","url":null,"abstract":"\u0000 Data from the distributed control system (DCS) or supervisory control and data acquisition (SCADA) system provide useful information critical to the evaluation of the performance and transportation efficiency of a gas pipeline system with compressor stations. The pipeline performance data provide correction factors for compressors as part of the operation optimization of natural gas transmission pipelines. This paper presents methods, procedures, and an example of model validation-based performance analysis of a gas pipeline based on actual system operational data. An analysis approach based on statistical methods is demonstrated with actual DCS gas pipeline measurement data. These methods offer practical ways to validate the pipeline hydraulics model using the DCS data. The validated models are then used as performance analysis tools in assessing the pipeline hydraulics parameters that influence the pressure drop in the pipeline such as corrosion (inside diameter change), roughness changes, or basic sediment and water deposition.","PeriodicalId":52254,"journal":{"name":"Journal of Verification, Validation and Uncertainty Quantification","volume":null,"pages":null},"PeriodicalIF":0.6,"publicationDate":"2019-05-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47763221","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}