Abstract
Due to inherent variability (i.e., aleatory uncertainty) in material properties, loading conditions, manufacturing processes, etc., simulation output responses should follow certain distributions, which are the outcome of input distribution models and simulation models. Thus, uncertainty quantification (UQ) using simulation-based methods is accurate only if we have (1) accurate input distribution models and (2) an accurate simulation model. However, in practical engineering applications, only limited numbers of input test data are available for modeling input distributions. Moreover, the simulation model could be biased due to assumptions and idealizations in the modeling process. Thus, the simulation model needs to be validated to correctly predict the output response of the physical system. The statistical validation of the simulation model would require large numbers of physical test data, which is extremely expensive. This study presents a computational method to obtain a target output distribution, which is a good approximation of the true output distribution given limited test data and a biased model. Using Bayesian analysis, possible candidates of output distribution are obtained, and a target output distribution is selected at the posterior median. The target output distribution is used to measure the bias of the simulation models and the surrogate models for UQ and statistical model validation. Furthermore, the cumulative distribution function of the simulation model prediction error is presented to provide (1) the median of model prediction error and (2) model confidence at a user-specified error level. To demonstrate the proposed statistical model validation method, a simulation model of an offshore jacket structure panel is generated using ANSYS. This example has eight uncertain input parameters (six of them are geometrical parameters and two of them are material properties). For input distributions of geometrical parameters, the standard deviation of manufacturing tolerances is used, and three material tests are carried out to obtain input distributions of two material properties. The output response of interest is a reaction force. The simulation model is validated using three sets of material test data and experimental test data to determine model confidence on output prediction and the validity of the simulation model. The result of statistical model validation shows how much bias the simulation model has and how much confidence engineers can have in the model prediction at a user-specified error level.










Similar content being viewed by others

Abbreviations
- AKDE:
-
Adaptive KDE
- AM:
-
Addictively manufactured
- ASTM:
-
American Society for Testing and Material
- CAE:
-
Computer-aided engineering
- CDF:
-
Cumulative distribution function
- DKG:
-
Dynamic kriging
- FEA:
-
Finite element analysis
- f x :
-
Input PDF
- h(ye; h0):
-
Local bandwidth in AKDE
- h 0 :
-
Global fixed bandwidth for modeling output distribution
- K :
-
Kernel
- KDE:
-
Kernel density estimation
- M :
-
Number of MCS samples
- MAE:
-
Mean absolute error
- MAP:
-
Maximum a posteriori probability
- MCMC:
-
Markov chain Monte Carlo
- MCS:
-
Monte Carlo simulation
- MLE:
-
Maximum likelihood estimation
- MSE:
-
Mean squared error
- nx, ny :
-
Number of input and output test data
- UTM:
-
Universal testing machine
- UQ:
-
Uncertainty quantification
- PDF:
-
Probability density function
- P(h0):
-
Prior distribution of bandwidth
- P(h0|ye):
-
Posterior distribution of bandwidth given output data
- STD:
-
Standard deviation
- \( {\mathbf{y}}_i^e,{\mathbf{y}}^e \) :
-
ith output data and output data vector
- x e :
-
Input data vector
- \( \hat{\boldsymbol{\uptheta}} \) :
-
Input distribution parameter vector
References
American Institute of Steel Construction (2016) ANSI/AISC 303-16-The 2016 AISC Code of Standard Practice for Steel Buildings and Bridges. AISC, Chicago, Illinois
American Petroleum Institute (2002) API Recommended Practice 2A-WSD. API Publishing Service, Washington, D.C.
Cho H, Choi KK, Gaul N, Lee I, Lamb D, Gorsich D (2016) Conservative reliability-based design optimization method with insufficient input data. Struct Multidiscip Optim 54(6):1–22. https://doi.org/10.1007/s00158-016-1492-4
Chowdhury FN, Kolber ZS, Barkley MD (1991) Monte Carlo convolution method for simulation and analysis of fluorescence decay data. Rev Sci Instrum 62(1):47–52
Ferson S, Oberkampf WL, Ginzburg L (2008) Model validation and predictive capability for the thermal challenge problem. Comput Methods Appl Mech Eng 197:2408–2430. https://doi.org/10.1016/j.cma.2007.07.030
Gunawan S, Papalambros PY (2006) A Bayesian approach to reliability-based optimization with incomplete information. J Mech Des 128(4):909–918. https://doi.org/10.1115/1.2204969
He Q (2019) Model validation based on probability boxes under mixed uncertainties. Adv Mech Eng 11(5):1687814019847411
Hess PE, Bruchman D, Assakkaf IA, Ayyub BM (2002) Uncertainties in material and geometric strength and load variables. Nav Eng J 114(2):139–166
Jekel C, Romero V (2019) Bootstrapping and jackknife resampling to improve sparse-sample UQ methods for tail probability estimation. Proceedings of the ASME 2019 Verification and Validation Symposium. ASME 2019 Verification and Validation Symposium. Las Vegas, Nevada, USA. May 15–17, 2019. V001T06A003. ASME. https://doi.org/10.1115/VVS2019-5127
Jiang Z, Chen W, Fu Y, Yang RJ (2013) Reliability-based design optimization with model bias and data uncertainty. SAE Int J Mater Manuf 6(2013-01-1384):502–516. https://doi.org/10.4271/2013-01-1384
Jones TA (1977) A computer method to calculate the convolution of statistical distributions. J Int Assoc Math Geol 9(6):635–647
Kennedy MC’ O’Hagan A (2001) Bayesian calibration of computer models. JR Stat Soc Series B Stat Methodol 63(3):425–464. https://doi.org/10.1111/1467-9868.00294
Kim HS, Lee K, Park B, Kim D (2017) A comparative study of offshore platform design based on rule scantling and topology optimization. International Mechanical Engineering Congress & Exposition Tampa, Florida, November 3-9
Kim HS, Lee K, Park B, and Kim D (2018) Topology optimization of offshore platform structure panel and experimental validation. 41st Solid Mechanics Conference, Warsaw, Poland, August 27–31
Lai CD, Murthy D, Xie M (2006). Weibull distributions and their applications. Springer Handbook of Engineering Statistics. Chapter 3. pp. 63–78. https://doi.org/10.1007/978-1-84628-288-1_3
Lee G, Son H, Youn BD (2019) Sequential optimization and uncertainty propagation method for efficient optimization-based model calibration. Structural and Multidisciplinary Optimization 60:1355–1372. https://doi.org/10.1007/s00158-019-02351-2
Li W, Chen W, Jiang Z, Lu Z, Liu Y (2014) New validation metrics for models with multiple correlated responses. Reliab Eng Syst Saf 127:1–11
McFarland J, Mahadevan S, Romero V, Swileir L (2008) Calibration and uncertainty analysis for computer simulations with multivariate output. AIAA J 46(5):1253–1265
Moon MY, Choi KK, Cho H, Gaul N, Lamb D, Gorsich D (2017) Reliability-based design optimization using confidence-based model validation for insufficient experimental data. J Mech Des 139(3):031404. https://doi.org/10.1115/1.4035679
Moon MY, Cho H, Choi KK, Gaul N, Lamb D, Gorsich D (2018) Confidence-based reliability assessment considering limited numbers of both input and output test data. Struct Multidiscip Optim 57(5):2027–2043
Moon M, Choi KK, Gaul N, Lamb D (2019a) Treating epistemic uncertainty using bootstrapping selection of input distribution model for confidence-based reliability assessment. ASME J Mech Des 2019:141(3)
Moon M, Choi KK, Lamb D (2019b) Target output distribution and distribution of bias for statistical model validation given a limited number of test data. Struct Multidiscip Optim 60(4):1327–1353
Mourelatos ZP, Zhou J (2005) Reliability estimation and design with insufficient data based on possibility theory. AIAA J 48(8):1696–1705
Nelsen RB (2006) An introduction to copulas, second edition. New York, NY 10013, USA: Springer Science+Business Media Inc. ISBN 978-1-4419-2109-3
Noh Y, Choi KK, Lee I, Gorsich D (2011) Reliability-based design optimization with confidence level for non-Gaussian distributions using bootstrap method. ASME J Mech Design 133(9):091001. https://doi.org/10.1115/1.4004545
Oh H, Choi H, Jung JH, Youn BD (2019) A robust and convex metric for unconstrained optimization in statistical model calibration—probability residual (PR). Structural and Multidisciplinary Optimization 1-7
Pan H, Xi Z, Yang RJ (2016) Model uncertainty approximation using a copula-based approach for reliability based design optimization. Struct Multidiscip Optim 54(6):1543–1556. https://doi.org/10.1007/s00158-016-1530-2
Picheny V, Kim NH, Haftka RT (2010) Application of bootstrap method in conservative estimation of reliability with limited samples. Struct Multidiscip Optim 41(2):205–217. https://doi.org/10.1007/s00158-009-0419-8
RAMDO Software (2018) RAMDO solutions, LLC, Iowa City, IA, https://wwwramdosolutionscom August 8
Romero VJ, Weirs VG (2018) A class of simple and effective uq methods for sparse replicate data applied to the cantilever beam end-to-end uq problem. In 2018 AIAA Non-Deterministic Approaches Conference. P. 1665
Roy CJ, Oberkamph WL (2011) A comprehensive framework for verification, validation, and uncertainty quantification in scientific computing. Comput Methods Appl Mech Eng 200:2131–2144
Sen O, Davis S, Jacobs G, Udaykumar HS (2015) Evaluation of convergence behavior of metamodeling techniques for bridging scales in multi-scale multimaterial simulation. J Comput Phys 294:585–604. https://doi.org/10.1019/j.jcp.2015.03.043
Silverman BW (1986) Density estimation for statistics and data analysis. Chapman & Hall, London
Volpi S, Diez M, Gaul NJ, Song H, Iemma U, Choi KK, Campana EF, Stern F (2014) Development and validation of a dynamic metamodel based on stochastic radial basis functions and uncertainty quantification. Struct Multidiscip Optim 51(2):347–368. https://doi.org/10.1007/s00158-014-1128-5
Wang P, Youn BD, Xi Z, Kloess A (2009) Bayesian reliability analysis with evolving, insufficient, and subjective data sets. J Mech Des 131(11):111008
Xi Z (2019) Model-based reliability analysis with both model uncertainty and parameter uncertainty. J Mech Des 141(5):051404-051404-11. https://doi.org/10.1115/1.4041946
Youn BD, Jung BC, Xi Z, Kim SB, Lee W (2011) A hierarchical framework for statistical model calibration in engineering product development. Comput Methods Appl Mech Eng 200:1421–1431
Zaman K, Mahadevan S (2017) Reliability-based design optimization of multidisciplinary system under aleatory and epistemic uncertainty. Struct Multidiscip Optim 55(2):681–699. https://doi.org/10.1007/s00158-016-1532-0
Zhao L, Choi KK, Lee I (2011) Metamodeling method using dynamic kriging for design optimization. AIAA J 49(9):2034–2046. https://doi.org/10.2514/1.J051017
Acknowledgments
This research was supported by a grant from Endowment Project of “Study on the Core Technology of Structural Design, Engineering and Test for Establishment of Structural Evaluation System for Offshore Structure” funded by Korea Research Institute of Ships and Ocean engineering (PES3250). The authors wish to acknowledge the technical and financial support of a joint project between the University of Iowa and the Korea Research Institute of Ships and Ocean Engineering (KRISO), as well as technical support from RAMDO Solutions, LLC.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Replication of results
As the model cannot be published due to confidential issue of the funded project, detail explanation about how the simulation model for the offshore structure panel can be modeled using ANSYS is given in Section 2. Detail algorithm of sampling-based Bayesian analysis used to obtain target output distribution is given in the reference Moon et al. (2019b). Algorithm to obtain approximated input distribution given limited number of data (Fig. 5) is presented as below.
-
Step 1:
Obtain sample mean and standard deviation of input data for each of input random variables for which limited number of data is provided. If paired data is correlated, sample Kendall’s tau is evaluated.
-
Step 2:
Transform sample mean and standard deviation into two distribution parameters for each of six candidates of marginal input distribution (Normal, Lognormal, Weibull, Gumbel, Extreme1, and Extreme II) (Table 4).
-
Step 3:
Evaluate likelihood value for each of six candidate distributions and select one that provides maximum likelihood value.
Additional information
Responsible Editor: Erdem Acar
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Moon, MY., Kim, HS., Lee, K. et al. Uncertainty quantification and statistical model validation for an offshore jacket structure panel given limited test data and simulation model. Struct Multidisc Optim 61, 2305–2318 (2020). https://doi.org/10.1007/s00158-020-02520-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00158-020-02520-8