UQ[PY]LAB USER MANUAL
SENSITIVITY ANALYSIS
S. Marelli, C. Lamas, K. Konakli, C. Mylonas, P. Wiederkehr, B. Sudret
CHAIR OF RISK, SAFETY AND UNCERTAINTY QUANTIFICATION
STEFANO-FRANSCINI-PLATZ 5
CH-8093 Z
¨
URICH
Risk, Safety &
Uncertainty Quantification
How to cite UQ[PY]LAB
C. Lataniotis, S. Marelli, B. Sudret, Uncertainty Quantification in the cloud with UQCloud, Proceedings of the 4th
International Conference on Uncertainty Quantification in Computational Sciences and Engineering (UNCECOMP
2021), Athens, Greece, June 27–30, 2021.
How to cite this manual
S. Marelli, C. Lamas, K. Konakli, C. Mylonas, P. Wiederkehr, B. Sudret, UQ[py]Lab user manual – Sensitivity anal-
ysis, Report UQ[py]Lab -V1.0-106, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland,
2024
BIBT
E
X entry
@TechReport{UQdoc_10_106,
author = {Marelli, S. and Lamas, C. and Konakli, K. and Mylonas, C. and Wiederkehr,
P. and Sudret, B.},
title = {{UQLab user manual -- Sensitivity analysis }},
institution = {Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich,
Switzerland},
year = {2024},
note = {Report UQ[py]Lab - V1.0-106}
}
List of contributors:
Name Contribution
A. Hlobilov
´
a Translation from the UQLab manual
Document Data Sheet
Document Ref. UQ[PY]LAB-V1.0-106
Title: UQ[PY]LAB user manual – Sensitivity analysis
Authors: S. Marelli, C. Lamas, K. Konakli, C. Mylonas, P. Wiederkehr, B. Sudret
Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich,
Switzerland
Date: 27/05/2024
Doc. Version Date Comments
V0.8 07/12/2022 Initial release
V1.0 27/05/2024 Minor update for UQ[PY]LAB release 1.0
Abstract
The UQ[PY]LAB Sensitivity Analysis module is a powerful and flexible tool for performing
sensitivity analysis with a number of different techniques. This user manual includes a review
of several advanced sensitivity analysis methods and algorithms available in UQ[PY]LAB,
also serving as an overview of the relevant up-to-date literature. It also provides an in-
depth, example-driven user guide to help new users to properly set up and solve sensitivity
analysis problems with the techniques described. Finally, a comprehensive reference list of
the methods and functions available in the UQ[PY]LAB sensitivity module is given at the end
of the manual.
Keywords: Global sensitivity analysis - Sobol’ indices - Morris method - Cotter measure - SR-
C/SRRC coefficients - Polynomial Chaos Expansions - Low-Rank Approximations - Borgonovo
Indices - ANCOVA indices - Kucherenko indices
Contents
1 Theory 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.3 Sample-based methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3.1 Input/Output Correlation . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3.2 Standard Regression Coefficients . . . . . . . . . . . . . . . . . . . . . 3
1.4 Linearization methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4.1 Perturbation method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4.2 Cotter method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.5 Global methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5.1 Morris method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5.2 Borgonovo indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5.3 Sobol’ indices (ANOVA) . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.5.4 ANCOVA indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.5.5 Kucherenko indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2 Usage 25
2.1 Independent input variables: borehole function . . . . . . . . . . . . . . . . . 26
2.1.1 Problem statement and set-up . . . . . . . . . . . . . . . . . . . . . . . 26
2.1.2 Input/output correlation . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.1.3 Standard Regression Coefficients . . . . . . . . . . . . . . . . . . . . . 29
2.1.4 Perturbation method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.1.5 Cotter Measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.1.6 Morris Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.1.7 Borgonovo indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.1.8 MC-based Sobol’ indices . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.1.9 PCE-based Sobol’ indices . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.1.10 LRA-based Sobol’ indices . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.2 Dependent input variables: short column function . . . . . . . . . . . . . . . . 48
2.2.1 Problem statement and set-up . . . . . . . . . . . . . . . . . . . . . . . 48
2.2.2 ANCOVA indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.2.3 Kucherenko indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.3 Sensitivity analysis of models with multiple outputs . . . . . . . . . . . . . . . 54
2.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.3.2 Accessing multi-output results . . . . . . . . . . . . . . . . . . . . . . . 54
2.4 Excluding parameters from the analysis . . . . . . . . . . . . . . . . . . . . . 55
2.4.1 Using a factor index . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.4.2 Using constant input variables . . . . . . . . . . . . . . . . . . . . . . . 56
3 Reference List 57
3.1 Create a sensitivity analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.1.1 Input/Output correlation options . . . . . . . . . . . . . . . . . . . . . 61
3.1.2 Standard Regression Coefficients (SRC and SRRC) options . . . . . . . 61
3.1.3 Perturbation Method options . . . . . . . . . . . . . . . . . . . . . . . 61
3.1.4 Cotter Measure options . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.1.5 Morris Measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.1.6 Borgonovo indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.1.7 MC-based Sobol’ indices . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.1.8 PCE-based Sobol’ indices . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.1.9 LRA-based Sobol’ indices . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.1.10 ANCOVA indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.1.11 Kucherenko indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.2 Accessing the results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.2.1 Input/Output correlation . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.2.2 Standard Regression Coefficients . . . . . . . . . . . . . . . . . . . . . 70
3.2.3 Perturbation Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.2.4 Cotter Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.2.5 Morris Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.2.6 Borgonovo indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.2.7 MC-based Sobol’ indices . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.2.8 PCE-based Sobol’ indices . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.2.9 LRA-based Sobol’ indices . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.2.10 ANCOVA indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.2.11 Kucherenko indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.3 Printing/Visualizing of the results . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.3.1 Printing the results: uq.print . . . . . . . . . . . . . . . . . . . . . . . 78
3.3.2 Graphically display the results: uq.display . . . . . . . . . . . . . . . . 78
Chapter 1
Theory
1.1 Introduction
Let M(x) be a mathematical model, depending on a set of input variables (x
1
, ..., x
M
) de-
scribed by a random vector X. In general, the aim of sensitivity analysis is to describe how
the variability of the model response, y = M(x) is affected by the variability of each input
variable or combinations thereof.
Apart from the interesting information that such an analysis provides to the modeller in a
theoretical sense, sensitivity analysis is useful to spot unimportant input variables and help
reduce the dimension of the problem (model reduction), among other usages. This kind of
analyses are performed with a black-box approach, i.e. only based on the model response
evaluations for a certain sample of inputs, selected in such a way that they will maximize
the output information about the model structure. Furthermore, it is often the case that each
run of the model is expensive in terms of computer time and, therefore, sensitivity methods
generally aim at reducing the number of model evaluations as much as possible.
1.2 Problem statement
Numerous approaches are available nowadays to perform sensitivity analysis. Broadly speak-
ing, we can either look at the correlation of samples of the input parameters with samples
of the output (correlation-based measures), at the values of partial derivatives of the model
or at a given point (linearisation methods) or directly at the distribution of the model out-
put. More specifically, global sensitivity analysis (many of which are variance decomposition
techniques) aims at decomposing the variance of the model output in terms of contributions
of each single input parameter, or combinations thereof.
Here, the following approaches will be discussed (for an up-to-date review and classification
of sensitivity analysis techniques, see Iooss and Lema
ˆ
ıtre (2015)):
• Sample-based methods - They are based on the post-processing of available Monte
Carlo samples of the model. They are rather simple as they do not require special
sampling strategies. The following methods are available in UQ[PY]LAB:
1
UQ[PY]LAB user manual
– Input/output correlation
– Standard regression coefficients
• Linearisation methods - They are based on the assumption that the model is linear
or can be linearised around a central value, usually referred to as the nominal value in
the literature. They are simple and usually require few model evaluations. However,
they fail to identify non-linear behaviours and higher-order interactions between input
variables. Within this category, we will review the following methods:
– Perturbation method
– Cotter method
• Global methods - These methods take into account the whole input domain. They may
refer to different features of the model output, such as variance or distribution. In the
current version of UQ[PY]LAB the following methods are implemented:
– Morris’ elementary effects
– Borgonovo sensitivity indices
– Sobol’ sensitivity indices
– ANCOVA sensitivity indices
– Kucherenko sensitivity indices
Note that other authors use a different classification of the aforementioned methods, by
identifying the class of so-called screening methods. Screening methods aim at providing a
qualitative ranking of the importance of the input variables w.r.t. the model response, with
relatively low computational costs. Cotter and Morris’ methods are commonly grouped in this
class. In this context, we could include PCE- and LRA-based Sobol’ indices (Section 1.5.3.4
and Section 1.5.3.5) and the ANCOVA indices (Section 1.5.4) in the class of screening meth-
ods.
Note: Some methods are specifically developed for the case of independent input vari-
ables and are only to be applied in case this premise is fulfilled. While it is also
remarked in the respective sections, the methods only applicable in the case of
independent input variables are:
– Standard regression coefficients
– Perturbation method
– Sobol’ sensitivity indices
UQ[PY]LAB-V1.0-106 - 2 -
Sensitivity analysis
1.3 Sample-based methods
1.3.1 Input/Output Correlation
Perhaps the most intuitive measure of sensitivity of model response Y to the components of
the input random vector X is the component-wise correlation coefficients between the two.
Due to its simplicity this method is applicable regardless of dependence in the input variables.
Consider a sampling of the input random vector X =
x
(1)
, x
(2)
, ..., x
(N)
and the corre-
sponding model responses Y =
y
(1)
, y
(2)
, ..., y
(N)
. The linear correlation coefficient ρ
i
be-
tween the i
th
input and the output is defined as
ρ
i
def
= ρ(X
i
, Y ) =
E [(X
i
− µ
i
)(Y − µ
Y
)]
σ
i
σ
Y
, (1.1)
where µ
i
def
= E [X
i
] and µ
Y
def
= E [Y ] and σ
i
and σ
Y
are the corresponding standard deviations.
The unbiased Monte Carlo estimator of the sample correlation reads:
bρ
i
=
P
N
j=1
(x
(j)
i
− bµ
i
)(Y
(j)
− bµ
Y
)
bσ
i
bσ
Y
, (1.2)
where the hat notation
b
· identifies the sample estimates of the corresponding quantities in
Eq. (1.1).
Linear correlation, however, is known to be inaccurate in the presence of strongly non-linear
dependence between variables. A more stable estimator that relies on the monotonicity
instead of the linearity of the dependence of two variables, is the Spearman’s rank correlation
index ρ
S
.
To calculate Spearman’s ρ
S
, each component of the input sample and its response are trans-
formed into their rank-equivalents:
R
i
=
n
r
(j)
i
∈ {1, ..., N } : r
(j)
i
> r
(k)
i
⇐⇒ x
(j)
i
> x
(k)
i
∀j, k ∈ {1, ..., N}
o
. (1.3)
The rank-transformed model response then reads R
Y
=
n
r
(1)
Y
, r
(2)
Y
, ..., r
(N)
Y
o
. The Spearman
correlation coefficient is defined as the linear correlation coefficients between the ranks:
ρ
S
i
def
= ρ
S
(X
i
, Y ) = ρ(R
i
, R
Y
). (1.4)
1.3.2 Standard Regression Coefficients
Another simple measure of sensitivity of the model response to each of the input variables
is given by the Standard Regression Coefficients (SRC). Since it consists of a simplified vari-
ance decomposition, it only leads to useful results for uncorrelated input variables. For this
method, the model M(X) is approximated by a linear regression:
Y = M(X) ≈ β
0
+
M
X
i=1
β
i
X
i
, (1.5)
UQ[PY]LAB-V1.0-106 - 3 -
UQ[PY]LAB user manual
The advantage of this formulation is that, in case of independent variables X
i
, the total
variance of the model can be estimated by:
bσ
2
Y
=
M
X
i=1
β
2
i
σ
2
X
i
, (1.6)
where bσ
2
Y
and σ
2
X
i
are the variances of the response and each input, respectively.
Given a sample of the input X =
x
(1)
, x
(2)
, ..., x
(N)
and the corresponding sample of model
responses Y =
y
(1)
, y
(2)
, ..., y
(N)
, the least-square estimator of the β = {β
1
, β
2
, ..., β
M
} is
given by:
b
β =
A
T
A
−1
A
T
Y , (1.7)
where the matrix A has for columns the M components of the input sampling X .
The SRC indices are then defined as:
SRC
i
=
b
β
i
σ
i
σ
Y
(1.8)
A straightforward extension to this method consists in substituting both the experimental
design samples and their response with the corresponding ranks so as to obtain the standard
rank regression coefficients (SRRC). Applying a rank-transform to both Y and X in Eq. (1.5)
yields:
r
Y
≈ γ
0
+
M
X
i=1
γ
i
r
i
, (1.9)
where the r
Y
are the ranks of the model responses, r
i
the ranks of the input coordinates and
γ
i
the rank-regression coefficients. Eq. (1.7) then becomes:
b
γ =
R
T
R
−1
R
T
R
Y
, (1.10)
where the matrix R has for columns the M components of the rank-transformed experimen-
tal design R
i
(Eq. (1.4)). The corresponding indices read:
SRRC
i
= bγ
i
. (1.11)
Note that the standard deviations in Eq. (1.8) cancel out after the rank transform because
each rank-transformed variable is a permutation of the vector of integer ranks {1, 2, . . . , N}.
1.4 Linearization methods
1.4.1 Perturbation method
The basis of the perturbation method is to substitute the model by its Taylor expansion around
a value of interest, usually the mean value of the input distribution. Once the first-order ex-
pansion is found, it can be used to analytically find the mean and variance of the model, and
UQ[PY]LAB-V1.0-106 - 4 -
Sensitivity analysis
to develop an importance measure of each of the input variables, by looking at their contribu-
tion to the model variance in terms of their own initial variance. The decomposition leading
to the importance measures is based on the assumption of independent input variables.
Let µ
X
= {µ
1
, ..., µ
M
}
T
denote the vector containing the means of the marginal distributions
of the input variables and C be the covariance matrix. The first-order expansion of the model
M(x) around the mean value is as follows:
M(x) = M(µ
X
) +
M
X
i=1
∂M
∂x
i
|
x=µ
X
(x
i
− µ
i
) + o(∥x − µ
X
∥
2
) .
The mean and variance can be estimated from this expression as:
b
E[M(X)] = M(µ
X
) , (1.12)
d
Var [M(X)] = ∇M
T
(µ
X
) · C · ∇M
T
(µ
X
) . (1.13)
If the variables are independent, C contains only the variances of the input variables on the
diagonal, so the estimate of the total variance in (1.13) simplifies to:
d
Var [M(X)] =
M
X
i=1
∂M
∂x
i
(µ
X
)
2
σ
2
i
. (1.14)
As this is a sum of positive terms over the number of variables, each of the terms represents
its contribution to the estimate of the total variance. Finally, to construct the perturbation
method sensitivity estimates η
i
, these terms are divided by the total variance, so that they
sum up to one:
η
i
=
∂M
∂x
i
(µ
X
)
2
σ
2
i
d
Var [M(X)]
, i = 1, ..., M. (1.15)
Note that these estimates are based on the assumption of independent input variables and are
thus only applicable in case this is assumption verified.
1.4.2 Cotter method
The Cotter method (Cotter, 1979) is a simple and low-cost method that allows for ranking
input parameters. It can be applied regardless of dependence between the input variables.
Let us consider an input vector x = {x
1
, ..., x
M
}
T
∈ D
x
. Each input variable X
i
is assumed
to vary in a known interval described by its low and high levels X
i
∈ [x
−
i
, x
+
i
]. Then, a
systematic fractional replicate design is carried out, in which the following 2M + 2 model
evaluations are performed:
• One run with all variables at their low levels, say y
0
= M(x
−
1
, ..., x
−
M
)
• M runs at low levels, switching one variable at a time to its high level:
y
i
= M(x
−
1
, ..., x
+
i
, ..., x
−
M
), i = 1, ..., M
UQ[PY]LAB-V1.0-106 - 5 -
UQ[PY]LAB user manual
• M runs at high levels, switching one variable at a time to its low level:
y
i
= M(x
+
1
, ..., x
−
i
, ..., x
+
M
), i = M + 1, ..., 2M
• One run with all variables at their high levels, y
2M+1
= M(x
+
1
, ..., x
+
M
)
The importance of the variables is then measured by:
I
Cotter
(i) = |C
o
(i)| + |C
e
(i)| , (1.16)
where C
o
(i) and C
e
(i) denote the expectation of the importance of the odd and even order
effects involving the input variable X
i
respectively, and are given by:
C
o
(i) =
1
4
[(y
2M+1
− y
M+i
) + (y
i
− y
0
)] , (1.17)
C
e
(i) =
1
4
[(y
2M+1
− y
M+i
) − (y
i
− y
0
)] . (1.18)
The great advantage of this method is that it only requires 2M + 2 runs of the model, yet it
has the drawback that, if the odd or even orders cancel each other out among dimensions,
this can lead to an incorrect importance measure.
1.5 Global methods
1.5.1 Morris method
The Morris method (Morris, 1991) is somehow similar to the Cotter method, but tries to
cover more exhaustively the input space. Instead of looking at a single global perturbation,
it creates a grid where multiple different perturbations are possible. The sensitivity measure
is then built based on the mean and standard deviation of that set of perturbations. Like the
Cotter method, it is applicable for any kind of dependence in the input random vector.
With no loss of generality, let us assume that the input parameters X vary in an M -dimensional
unit hypercube, i.e. X
i
∈ [0, 1], i = 1, ..., M (in the practical case, these sets can be rescaled
to the real input space).
A grid of p values is created for each parameter. It consists of the set:
G
i
=
0,
1
p − 1
,
2
p − 1
, ..., 1
. (1.19)
The whole input set is then defined by:
G = G
1
× ... × G
M
(1.20)
Let us now define a perturbation parameter ∆ such that it is a multiple of
1
p−1
,
∆ =
¯c
p − 1
, for a fixed ¯c ∈ {1, ..., p − 1}. (1.21)
For a given value of x = {x
1
, ..., x
M
}
T
∈ G, such that x
i
+ ∆ ≤ 1, the elementary effect for the
UQ[PY]LAB-V1.0-106 - 6 -
Sensitivity analysis
variable x
i
is defined by:
d
i
(x) =
M(x
1
, ..., x
i
+ ∆, ..., x
M
) − M(x)
∆
(1.22)
For each input variable, there exist a finite number of p
M−1
(p − ¯c) elementary effects that
can be computed. Morris’ original approach suggests to estimate the distribution of the
elementary input variables by randomly sampling r values in G and computing (1.22) for
each of them. Once these values are found, the estimators of the moments of the distribution
can be analysed as follows:
• Mean, bµ
i
: A high value for the mean indicates that the input variable X
i
is important
and, thus, its input values cause variability on the outputs of the model. Low mean
would indicate an unimportant input variable.
• Standard deviation, bσ
i
: A high value of the standard deviation indicates that the effect
of the variable X
i
is significant, either because of a nonlinear behaviour with respect to
this variable or because of interactions with other input variables.
Note that using this method with samples of r values would require a total of 2Mr runs (2r
runs per elementary effect).
1.5.1.1 Trajectory-based sampling strategy
Using a trajectory-based sampling strategy, the required number of model evaluations can be
reduced to (M + 1)r. The key idea is as follows: since two model evaluations are needed
for computing the elementary effect of each variable, one of them can be shared between
two different elementary effects by perturbing the current point properly. This means that
only one new model evaluation is needed for each variable after the starting point, leading
to M + 1 model evaluations in total. Let us introduce the following vectors and matrices:
• Random starting point, x
∗
It is a random point (dimension M × 1) of the grid, constructed in such a way that
increasing its coordinates in any dimension will lead to a point that still lies in the grid.
• Base matrix, B
It is a M × (M + 1) matrix containing ones in its strict lower triangular part and zeros
on the rest.
• Matrix J
i,j
It denotes a matrix made of ones, with dimension i × j.
• Random displacements matrix D
It consists of a diagonal M × M matrix that contains −1 or 1 on its diagonal, randomly
chosen with equal probability.
UQ[PY]LAB-V1.0-106 - 7 -
UQ[PY]LAB user manual
• Identity permutation P
It is a random permutation of the identity matrix of dimension M × M
The trajectory can be represented by a M × (M + 1) matrix B
∗
that contains in each column
a point, starting from the random point x
∗
. Each successive trajectory point is obtained from
the previous by sequentially increasing one of its coordinate by ∆. Each coordinate can be
increased only once.
A matrix B
′
satisfying such properties is given by:
B
′
=
J
M+1,1
(x
∗
)
T
+ ∆B
T
. (1.23)
However, such trajectory is still deterministic except for the starting point. The following
formula provides a properly randomized version (Morris, 1991):
B
∗
=
J
M+1,1
(x
∗
)
T
+
∆
2
[(2B − J
M+1,M
)D
∗
+ J
M+1,1
]
P
T
(1.24)
1.5.1.2 Modified estimator
As it is based on the mean, the estimator bµ fails to identify the effects of those variables that
can cancel each other out (i.e., alternate signs). To avoid this drawback, a new estimator bµ
∗
was introduced in Campolongo et al. (2007) which, instead of computing the mean of the
elementary effects of (1.22), computes the mean of its absolute values:
d
∗
i
(x) =
M(x
1
, ..., x
i
+ ∆, ..., x
M
) − M(x)
∆
(1.25)
This estimator, however, lacks the information about the sign of the effect introduced by
the input variable; yet combined with the previous bµ
i
and bσ
i
it can provide more complete
information about the actual importance of each input variable, the sign of its effect and the
nature of this effect (linear, non-linear or interactions).
1.5.1.3 Graphical representation
Morris sensitivity analysis provides two estimators for each variable ˆµ
i
and ˆσ
i
. It is customary
to represent them in a scatter-plot, often referred to as a Morris plot. Figure 1 shows a Morris
plot for an 8-dimensional problem (see Section 2.1.6 for details on the model).
Each point on the plot is the short name of one of input variables. Its position on the µσ-plane
is given by the corresponding ˆµ
i
and ˆσ
i
. Points close to the origin correspond to unimportant
variables. Points far away from the origin in the µ direction are generally important. Points
far away from the origin in the σ direction represent variables with significantly non-linear
contributions, hence sometimes difficult to properly assess.
UQ[PY]LAB-V1.0-106 - 8 -
Sensitivity analysis
−20 0 20 40
0
10
20
30
40
50
60
rw
r
Tu
Hu
Tl
Hl
L
Kw
Elementary effects
μ
σ
0 10 20 30 40
0
10
20
30
40
50
60
rw
r
Tu
Hu
Tl
Hl
L
Kw
Elementary effects
μ
*
σ
Figure 1: Morris plots for an 8-dimensional problem (see Section 2.1.6 for details). The µ of
each index is given by its position on the x−axis, while the corresponding σ by that on the
y−axis.
1.5.2 Borgonovo indices
The Borgonovo index δ
i
(Borgonovo, 2007) of a random input variable X
i
is a measure of
the expected shift in the probability distribution of the model output when X
i
is set to a fixed
value. If the variable is not important, the expected shift is close to zero, otherwise it takes a
larger value. Its derivation is not based on any assumptions on the dependence structure of
the input variables X, which makes it applicable in either case, dependent or independent
input variables. The definition of the (first-order
1
) Borgonovo index reads:
δ
i
=
1
2
E
X
i
"
Z
D
y
|f
Y
(y) − f
Y |X
i
(y)|dy
#
, (1.26)
where f
Y
is the probability distribution function (PDF) of the model output and f
Y |X
i
is the
conditional distribution on X
i
. The expectation value in Eq. (1.26) can be written as:
δ
i
=
1
2
Z
D
x
i
f
X
i
(x
i
)
Z
D
y
|f
Y
(y) − f
Y |X
i
(y|x
i
)|dydx
i
. (1.27)
1.5.2.1 Estimation of Borgonovo indices
Accurate estimation of the Borgonovo index δ
i
for an input variable X
i
relies on the precise
approximation of both the unconditional PDF f
Y
and the conditional PDF f
Y |X
i
of the model
output. UQ[PY]LAB provides two methods to estimate the inner integral in Eq. (1.27),
Histogram- and CDF-based. The approximation of the conditional distribution of Y |X
i
is simi-
lar for both methods. N samples are drawn from the input random vector X and binned into
so-called classes on variable X
i
. Each of the P classes is defined by its lower and upper limits
{a
p
, b
p
} with a
p
< b
p
, p ∈ {1, . . . , P }. Class C
p
includes all samples x
(j)
, j ∈ {1, . . . , N},
whose realizations of X
i
, x
(j)
i
, lie between between a
p
and b
p
. In short, this can be written as
1
At this point, higher-order Borgonovo indices are not defined yet.
UQ[PY]LAB-V1.0-106 - 9 -
UQ[PY]LAB user manual
C
p
=
n
x
(j)
: a
p
≤ x
(j)
i
< b
p
o
. The distribution of Y in each class is then:
f
Y |X
i
=x
i
(y) ≈
ˆ
f
Y |C
p
⊃x
i
(y) , (1.28)
where C
p
⊃ x
i
means that all the points in the same class C
p
as x
i
are considered to estimate
the conditional PDF.
Class building
The classes can either be disjoint (if b
p
= a
p+1
, p = 1, . . . , P −1) or overlapping (if b
p
>
a
p+1
, p = 1, . . . , P −1). Overlapping classes can improve the estimation in two ways. First,
for a given input sample size N, one can define more classes each containing a sufficient
amount of samples, thus leading to a better estimate. Second, samples in the overlapping
regions are included in both classes and correlate the estimates in neighbouring classes and
smooth the change. Another possibility to improve the estimation is to allow the class widths
(b
p
− a
p
) to be dynamically set by equally-spaced quantiles of X
i
, ensuring the same number
of samples in each class.
Histogram-based estimation
In the original approach of Borgonovo (2007), the distributions for the inner integral in
Eq. (1.27) are estimated from a sample of the input random vector X =
x
(1)
, · · · , x
(N)
and the corresponding model evaluations Y = M(x
(1)
, ..., M(x
(N)
)). The unconditional
PDF values are directly calculated through a histogram of Y. In order to estimate the PDF
values of the conditional distributions, a histogram is set up in each class. In this setting, the
sum of the absolute differences between the conditional and unconditional PDF values yields
an estimation of the inner integral in Eq. (1.27). A graphical representation of this approach
is given in Figure 2.
Since this can be expected to be a rough approximation in the presence of small samples,
Plischke et al. (2013) and Caniou (2012) proposed the use of kernel density estimators to
improve the estimate in Eq. (1.28). After extensive testing, a choice has been made in UQLAB
to avoid the usage of kernel density estimation for the conditional PDF estimation due to
an observed tendency of such estimators to produce biased estimates in the inner integral
in Eq. (1.27). Consequently, computations are based on histograms only and a reasonable
sample size (e.g. N ≥ 10
3
) is recommended. For the histograms to accurately estimate
the conditional PDF, it is important to have enough samples in each class and to choose a
reasonable number of histogram bins. While the number of samples per class is ensured by
quantile-based classes, the number of histogram bins and the bin widths should be chosen
for each class separately.
Once the absolute difference of the histograms is calculated for each class, it is then averaged
by taking the mean of the estimates. This averaging corresponds to the outer integral in
Eq. (1.27).
UQ[PY]LAB-V1.0-106 - 10 -
Sensitivity analysis
Figure 2: Approximating conditional distribution f
Y |X
1
=1.1
with conditional distribution
ˆ
f
Y |C
p
where Y |C
p
denotes the model responses where x
1
lies between a
p
= 1.0 and b
p
= 1.2.
CDF-based estimation
An alternative method to estimate the inner integral in Eq. (1.27) is based on the computation
of cumulative distribution functions and was introduced in Liu and Homma (2009). It can
be shown that:
Z
S(y)dy =
Z
|f
Y
(y) − f
Y |X
i
(y|x
i
)|dy (1.29)
can be replaced with a summation of the difference of the unconditioned and conditional
cumulative density function (CDF) at the crossing points {a
1
, · · · , a
l
} of f
Y
and f
Y |X
i
. The
summation formula reads:
Z
S(y)dy =
2 ·
l
X
k=1
(−1)
k−1
[F
Y
(a
k
) − F
Y |X
i
(a
k
)]
if F
Y
(a
1
) > F
Y |X
i
(a
1
)
2 ·
l
X
k=1
(−1)
k−1
[−F
Y
(a
k
) + F
Y |X
i
(a
k
)]
if F
Y
(a
1
) < F
Y |X
i
(a
1
)
(1.30)
To evaluate Eq. (1.30), kernel smoothing estimators of the CDFs F
Y
and F
Y |X
i
are used as in
Liu and Homma (2009) and Caniou (2012). Again, this is done for each class C
p
⊃ x
i
and
averaged by taking the mean.
1.5.2.2 Comparison of histogram- and CDF-based estimation methods
Considering a simple model M (X
1
, X
2
) = X
1
+ X
2
with marginal distributions X
1
∼ N (0, 1)
and X
2
∼ N (1, 2) the Borgonovo indices were estimated 50 times with the number of classes
equal to 10.
The histogram-based implementation yields as robust results as the CDF-based, as is demon-
UQ[PY]LAB-V1.0-106 - 11 -
UQ[PY]LAB user manual
Figure 3: Boxplots of the Borgonovo histogram- and CDF-based estimate for indices of
M(X
1
, X
2
) = X
1
+ X
2
based on 50 re-samplings for each sample size.
strated in Figure 3. The two methods yield roughly the same results. Since for the CDF-based
approach the crossing points of the CDF’s have to be found, which seems to unnecessarily
complicate the estimation, the histogram-based approach is the default in UQLAB, but both
methods are available.
1.5.3 Sobol’ indices (ANOVA)
Sobol’ indices (Sobol’, 1993) are based on the idea of defining the expansion of the compu-
tational model into summands of increasing dimension. Analogously, the total variance of
the model is described in terms of the sum of the variances of the summands. This decom-
position only holds for independent input variables and this method is often referred to as
ANOVA (ANalysis Of VAriance).
To simplify the notation and without loss of generality, all the equations in the following will
assume that the input variables are uniform variables in [0, 1] and, therefore, that the sup-
port of the input set is D
X
= [0, 1]
M
. (In the general case, the integration domains would
need to be properly defined and all of the expectation value integrals to be weighted by the
appropriate marginal PDFs.)
UQ[PY]LAB-V1.0-106 - 12 -
Sensitivity analysis
1.5.3.1 Sobol’ decomposition
The Sobol’ decomposition is defined as:
f(x
1
, ..., x
M
) = f
0
+
M
X
i=1
f
i
(x
i
) +
X
1≤i<j≤M
f
ij
(x
i
, x
j
) + ... + f
1,2,...,M
(x
1
, ..., x
M
), (1.31)
where the following conditions hold:
i) The term f
0
is constant and equal to the expected value of f(X).
ii) The integrals of the summands with respect to their own variables f
i
1
,...,i
s
vanish:
Z
1
0
f
i
1
,...,i
s
(x
i
1
, ..., x
i
s
) dx
i
k
= 0 if 1 ≤ k ≤ s . (1.32)
For integrable functions in D
X
and independent input variables X, this expansion is shown
to exist and to be unique in Sobol’ (1993). It is also shown that all the summands of the
expansion can be computed by integrals in a recursive manner as follows:
f
0
=
Z
D
X
f(x)dx,
f
i
(x
i
) =
Z
1
0
...
Z
1
0
f(x)dx
∼i
− f
0
, (1.33)
f
ij
(x
i
, x
j
) =
Z
1
0
...
Z
1
0
f(x)dx
∼(ij)
− f
0
− f
i
(x
i
) − f
j
(x
j
),
where x = {x
1
, ..., x
M
} and the notation ∼ indicates that variables are excluded, e.g.
x
∼i
= (x
1
, ..., x
i−1
, x
i+1
, ..., x
M
) . (1.34)
The higher-order summands can be constructed in an analogous way. However, the use
of Eq. (1.31) is only for computing the variance decomposition, which is presented next.
Following the previous results, the total variance of f(X) is defined as:
D =
Z
D
X
f
2
(x)dx − f
2
0
, (1.35)
and the partial variances are computed as:
D
i
1
,...,i
s
=
Z
1
0
...
Z
1
0
f
2
i
1
,...,i
s
(x
i
1
, ..., x
i
s
)dx
i
1
...dx
i
s
1 ≤ i
1
< ... < i
s
≤ M ; s = 1, ..., M
(1.36)
since the summands have zero mean due to Eq. (1.32). The partial variances have the prop-
erty that they sum up to the total variance.
UQ[PY]LAB-V1.0-106 - 13 -
UQ[PY]LAB user manual
1.5.3.2 Sensitivity indices
First- and higher-order indices
The variance decomposition leads to a natural definition of the sensitivity measures:
S
i
1
,...,i
s
=
D
i
1
,...,i
s
D
, (1.37)
which represent the relative contribution of each group of variables {X
i
1
, . . . , X
i
s
} to the
total variance. The index with respect to one input variable X
i
is called the first-order Sobol’
index and represents the effect of X
i
alone. Multiple-term indices, e.g. S
ij
, i ̸= j, are referred
to as higher-order Sobol’ indices and are interaction indices that account for the effects of the
interactions of the variables X
i
and X
j
that cannot be decomposed into the contributions of
those variables separately. Further higher-order indices can be interpreted analogously.
Total indices
The total Sobol’ index of input variable X
i
, denoted S
T
i
, is the sum of all the Sobol’ indices
involving this variable:
S
T
i
=
X
{i
1
,...,i
s
}⊃i
S
i
1
,...,i
s
. (1.38)
This definition is not practical for actual computations since it would result in computing
each index separately. Instead, by denoting S
∼i
the sensitivity measure of all the variables
excluding variable X
i
, i.e. S
∼i
= S
v
, where v = {1, ..., i − 1, i + 1, ..., M }, the total index can
be written as:
S
T
i
= 1 − S
∼i
, (1.39)
From this, the idea of computing indices of groups of variables is developed.
Closed indices
If the components of the input vector x ∈ [0, 1]
M
are separated into two groups x
v
and x
w
,
where v = {i
1
, ..., i
s
} ⊂ {1, ..., M} and w = ∼v such that y = f(x) = f (x
v
, x
w
), the Sobol’
index with respect to X
v
can be written as:
S
v
=
Var [E [Y |X
v
= x
v
]]
Var [Y ]
. (1.40)
This index accounts for the sum of all the single effects, also referred to as structural effects,
as well as interactions between the variables X
v
. In the univariate case, i.e. v = {i}, |v| = 1,
this index leads to the previously introduced first-order Sobol’ index. In the general case
with |v| > 1, this measure is called the closed index of X
v
, since it includes all univariate and
interaction effects of X
v
but no interactions with X
w
.
To compute higher-order indices, lower-order indices can be subtracted from the closed in-
dex. For example, the second order index of (X
i
, X
j
) is the closed index of these two variables
UQ[PY]LAB-V1.0-106 - 14 -
Sensitivity analysis
(denoted by S
{ij}
) minus the single effect of each of them:
S
ij
= S
{ij}
− S
i
− S
j
. (1.41)
1.5.3.3 Monte Carlo-based estimation
The variances described in Eqs. (1.35) and (1.36) may be computed by means of Monte Carlo
simulation using the following estimators:
b
f
0
=
1
N
N
X
n=1
f(x
(n)
), (1.42)
b
D =
1
N
N
X
n=1
f
2
(x
(n)
) −
b
f
2
0
, (1.43)
b
D
i
=
1
N
N
X
n=1
f(x
(n)
i
, x
(n)
∼i
)f(x
(n)
i
, x
′
(n)
∼i
) −
b
f
2
0
, (1.44)
where x
′
denotes a realization of X independent of x =
n
x
(n)
i
, x
(n)
∼i
o
T
, and the subscript
x
j,∼i
indicates the j-th realization of x which does not contain the input variable i.
To compute total indices it is easier to use group indices in combination with Eq. (1.39).
Let us now denote by y
v
= f(x
v
, x
′
w
), where x
′
w
is a sample of X
w
independent of x
w
.
Analogously, X
′
w
denotes an independent copy of X
w
and Y
v
= f(X
v
, X
′
w
). Following Janon
et al. (2014), we can rewrite the index in Eq. (1.40) in terms of sample covariance as follows:
S
v
=
Cov [Y, Y
v
]
Var [Y ]
, (1.45)
which leads to a first estimator, hereafter referred to as the Homma estimator, initially pro-
posed by Saltelli and Homma (1996):
S
v
=
1
N
P
y
i
y
v
i
− (
1
N
P
y
i
)(
1
N
P
y
v
i
)
1
N
P
y
2
i
− (
1
N
P
y
i
)
2
, (1.46)
where the sums without super- or subscript are considered to be
P
def
=
P
N
i=1
. A similar
estimator shown to perform better in most cases, hereafter referred to as the Janon estimator,
is described in Janon et al. (2014):
S
v
=
1
N
P
y
i
y
v
i
−
1
N
P
h
y
i
+y
v
i
2
i
2
1
N
P
h
(y
i
)
2
+(y
v
i
)
2
2
i
−
1
N
P
h
y
i
+y
v
i
2
i
2
. (1.47)
NB: It is clear that for computing the effect of one group of variables, only one estimator is
needed.
UQ[PY]LAB-V1.0-106 - 15 -
UQ[PY]LAB user manual
1.5.3.4 PCE-based Sobol’ indices
Sobol’ indices are traditionally evaluated by Monte Carlo simulation which make them dif-
ficult to use with computationally expensive models M. In order to bypass this problem,
Sudret (2008) proposed an original post-processing of polynomial chaos expansions (PCE)
for sensitivity analysis. As it will be demonstrated below, the Sobol’ decomposition of a trun-
cated PCE:
by = M
P C
(x) =
X
α∈A
by
α
Ψ
α
(x) , (1.48)
can be established analytically. For more information about PCE refer to the UQ[PY]LAB User
Manual – Polynomial Chaos Expansions.
The truncated expansion in Eq. (1.48) can be rearranged so as to reflect the decomposition
into summands of increasing order. For any non-empty set u ⊂ {1, . . . , M} and any finite
truncation set A ⊂ N
M
we define:
A
u
= {α ∈ A : α
k
̸= 0 ⇔ k ∈ u, k = 1, . . . , M } . (1.49)
In other words A
u
contains all the multi-indices within the truncation set A which have non-
zero components α
k
̸= 0 if and only if k ∈ u. The sum of the associated PCE terms forms a
function which depends only on input variables x
u
. The reordering of Eq. (1.48) reads:
M
A
(x) = y
0
+
X
u⊂{1,...,M}
u̸=∅
X
α∈A
u
by
α
Ψ
α
(x) . (1.50)
The Sobol’ decomposition described in Eq. (1.31) can also be written as follows, by introduc-
ing the generic index set v
def
= {i
1
, . . . , i
k
} ⊂ {1, . . . , M } and denoting by x
v
the subvector of
x obtained by extracting the components labelled by the indices in v:
f(x) = f
0
+
X
v⊂{1,...,M}
v̸=∅
f
v
(x
v
) . (1.51)
As mentioned in Section 1.5.3, the Sobol’ decomposition is unique, thus the comparison of
Eqs. (1.50) and (1.51) readily provides the expression of each summand for the PCE:
f
v
(x
v
) =
X
α∈A
v
by
α
Ψ
α
(x) . (1.52)
Due to the orthonormality of the polynomial chaos basis, the variance of the truncated PCE
(Y
A
= M
P C
(X)) reads:
UQ[PY]LAB-V1.0-106 - 16 -
Sensitivity analysis
Var [Y
A
] =
X
α∈A
α̸=0
by
2
α
, (1.53)
Var [f
v
(x
v
)] =
X
α∈A
v
α̸=0
by
2
α
, (1.54)
and the associated Sobol’ indices in Eq. (1.40) is just the ratio of the above two quantities.
1.5.3.5 LRA-based Sobol’ indices
The Sobol’ sensitivity indices can also be computed by post-processing the coefficients of
a canonical low-rank approximation (LRA) metamodel. A canonical LRA metamodel with
polynomial basis has the form:
by = M
LRA
(X) =
R
X
l=1
b
l
M
Y
i=1
p
i
X
k=0
z
(i)
k,l
ϕ
(i)
k
(X
i
)
!!
, (1.55)
where ϕ
(i)
k
is a univariate polynomial of degree k in the i-th input variable, and z
(i)
k,l
and b
l
de-
note polynomial and weighing coefficients, respectively. For more information on canonical
LRA metamodels, refer to the UQLAB User Manual – Canonical low-rank approximations.
The computation of the LRA-based Sobol’ sensitivity indices relies on the following expres-
sions for the first-order and the total indices, respectively:
S
i
=
E
h
E [M(X)|X
i
]
2
i
− E [M(X)]
2
Var [M(X)]
(1.56)
and
S
T
i
= 1 −
E
h
E [M(X)|X
∼i
]
2
i
− E [M(X)]
2
Var [M(X)]
, (1.57)
where X
∼i
denotes the set of variables {X
j
∈ X : X
j
̸= X
i
} (see Konakli and Sudret (2016)
for further details on the above equations). Note that each of Eq. (1.56) and Eq. (1.57)
involves the mean and variance of the model response and additionally, the mean-square of
a conditional expectation. Konakli and Sudret (2016) have shown that by substituting the
model M with the LRA metamodel M
LRA
, we can compute these quantities in terms of the
LRA coefficients. In particular, the mean and variance of the model output are respectively
approximated by:
E
M
LRA
(X)
=
R
X
l=1
b
l
M
Y
i=1
z
(i)
0,l
!
(1.58)
and
UQ[PY]LAB-V1.0-106 - 17 -
UQ[PY]LAB user manual
Var
M
LRA
(X)
=
R
X
l=1
R
X
l
′
=1
b
l
b
l
′
M
Y
i=1
p
i
X
k=0
z
(i)
k,l
z
(i)
k,l
′
!!
−
M
Y
i=1
z
(i)
0,l
z
(i)
0,l
′
!!
. (1.59)
The quantity E
h
E [M(X)|X
i
]
2
i
, appearing in the expression for S
i
, is approximated by:
E
h
E
M
LRA
(X)|X
i
2
i
=
R
X
l=1
R
X
l
′
=1
b
l
b
l
′
M
Y
j̸=i
z
(j)
0,l
z
(j)
0,l
′
p
i
X
k=0
z
(i)
k,l
z
(i)
k,l
′
!
, (1.60)
while the quantity E
h
E [M(X)|X
∼i
]
2
i
, appearing in the expression for S
T
i
, is approximated
by:
E
h
E
M
LRA
(X)|X
∼i
2
i
=
R
X
l=1
R
X
l
′
=1
b
l
b
l
′
z
(i)
0,l
z
(i)
0,l
′
Y
j̸=i
p
j
X
k=0
z
(j)
k,l
z
(j)
k,l
′
!
. (1.61)
Expressions for higher-order indices can also be obtained in terms of the LRA coefficients;
details of the computation can be found in Konakli and Sudret (2016).
1.5.3.6 Confidence intervals
The bootstrap method (Efron, 1979) provides a simple and relatively inexpensive way of es-
timating confidence intervals on the estimations of the Sobol’ indices. In its simplest form,
bootstraping consists of resampling with replacement new points from the original sample
set without additional sampling of the original model. The estimator of interest
b
θ is then
computed for each of the B bootstrap samples. From the resulting collection of estimators,
namely
b
Θ = {
b
θ
1
,
b
θ
2
, . . . ,
b
θ
B
}, empirical quantiles can be calculated to get confidence inter-
vals on
b
θ.
1.5.4 ANCOVA indices
ANCOVA (ANalysis Of COVAriance), also called SCSA (Structural and Correlated Sensitivity
Analysis), aims to produce helpful sensitivity indices for correlated input variables using a
covariance decomposition method similar to the ANOVA introduced in Section 1.5.3. It is
a way to generalize the formulation of the Sobol’ indices for the case of dependent input
variables.
The ANCOVA method was introduced by Xu and Gertner (2008). They proposed to decom-
pose the variance contribution of X
i
into two parts, one including the contributions due to in-
dependent variation and the other including the contributions due to dependence with other
variables. Caniou (2012) and Sudret and Caniou (2013) proposed to use polynomial chaos
expansion (PCE, see UQ[PY]LAB User Manual – Polynomial Chaos Expansions) to derive a
UQ[PY]LAB-V1.0-106 - 18 -
Sensitivity analysis
high dimensional model representation (HDMR), originally proposed by Li et al. (2010):
M
P C
(x) = M
P C
0
+
M
X
i=1
M
P C
i
(x
i
) +
X
1≤i≤i≤M
M
P C
ij
(x
i
, x
j
) + · · · + M
P C
1...M
(x) . (1.62)
where the M
P C
v
correspond to the definition provided in Eq. (1.52). By evaluating the terms
of this decomposition on samples of the dependent input variables, the first-order sensitivity
index can be defined as the covariance of the univariate PC component functions and the
output normalized by the output variance:
S
i
=
Cov
M
P C
i
(X
i
), Y
Var [Y ]
, (1.63)
where M
P C
i
(X
i
) represents the univariate component function(s) and Y = M
P C
(X) repre-
sents the full PCE (see Eq. (1.62)). The covariance term in the numerator can be decomposed
into:
S
i
=
Var
M
P C
i
(X
i
)
Var [Y ]
+
Cov
"
M
P C
i
(X
i
),
P
v⊂{1,...,M}
v̸={i}
M
P C
v
(X
v
)
#
Var [Y ]
. (1.64)
This result provides the first-order sensitivity index of X
i
as the sum of two partial variances,
one including independent contributions and the other dependent ones, as was suggested by
Xu and Gertner (2008). In the absence of correlation between the inputs, the second term
vanishes due to the inherent orthogonality of the component functions and the first-order
Sobol’ index remains. In case of dependent variables, however, the second term remains.
A limitation of the decomposition in Eq. (1.31) is that it only exists in the case of independent
input variables. On this account, Caniou (2012) proposed to set up a PCE assuming the input
variables are independent X
ind
and to use it as a surrogate of the full model. This PCE can be
used as a predictor of the values of the HDMR in Eq. (1.62) but cannot be used to analytically
derive sensitivity indices as in Section 1.5.3.5, since it will be evaluated on samples of the
dependent input variables. Caniou (2012) further pointed out that in the case of dependent
input variables the covariance between a univariate component function M
P C
i
(x
i
) and an-
other function M
P C
v
(x
v
), where i ∈ v, can originate not only from correlation but also from
X
i
acting in both terms. Thus, the covariance term in Eq. (1.64) consists of interactive and
correlative contributions and can be further split up into two parts. The first-order index of
X
i
finally reads:
S
i
= S
U
i
+ S
I
i
+ S
C
i
, (1.65)
where S
U
i
, S
I
i
and S
C
i
represent the uncorrelative, interactive and correlative index of X
i
,
respectively. They are defined as:
S
U
i
=
Var
M
P C
i
(X
i
)
Var [Y ]
, (1.66)
UQ[PY]LAB-V1.0-106 - 19 -
UQ[PY]LAB user manual
S
I
i
=
Cov
"
M
P C
i
(X
i
),
P
v⊂{1,...,M}
{i}∈v
M
P C
v
(X
v
)
#
Var [Y ]
(1.67)
and:
S
C
i
=
Cov
"
M
P C
i
(X
i
),
P
w⊂{1,...,M}
{i}/∈w
M
P C
w
(X
w
)
#
Var [Y ]
. (1.68)
The variances and covariances needed for the indices are computed using MC estimators.
Note that the only computational cost involved in this method is the experimental design
used to calculate the PCE metamodel M
P C
.
Splitting the first-order index into three parts aims to separate the effects of X
i
in more
detail than other methods. The interactive index S
I
i
includes the structural influence of X
i
in
M
P C
i
(X
i
) and
P
i∈v
M
P C
v⊂{1,...,M}
{i}∈v
(X
v
), as well as correlation effects between the component
functions. Thus, some confusion between interactive and correlative influence of X
i
remains.
The same holds for total indices, because lower-order indices are summed up. For this reason,
the definition of proper higher-order and total ANCOVA indices remains open.
Note: Since the interactive and correlative indices are defined as covariances, they can
take on negative values. This is related to the fact that correlation between input
variables can lead to a decrease of the output variance, which can also result in
ANCOVA indices that exceed absolute values of 1. This is not an artefact of the
ANCOVA decomposition, but rather one of the effects of dependence between
input variables. Notably, the reduction of the variance in one input variable can
result in an increase of the total variance.
1.5.5 Kucherenko indices
Kucherenko et al. (2012) proposed to define sensitivity indices using a direct decomposition
of the output variance with the law of total variance. As in the case of ANCOVA, it is a way to
generalize the Sobol’ indices to the case of dependent input variables. If the input variables
X are divided into two complementary subsets X
v
and X
w
= X
∼v
, the total variance of
Y = M(X) is the following sum:
Var [Y ] = Var [E [Y |X
v
]] + E [Var [Y |X
v
]] . (1.69)
After normalization by Var [Y ], the first summand is the closed index of X
v
(Eq. (1.40)),
including all univariate and interaction effects between these variables:
S
v
=
Var [E [Y |X
v
= x
v
]]
Var [Y ]
. (1.70)
Consequently, the second summand represents the total effect of the variables X
w
, which
includes all univariate and interaction effects of the variables X
w
plus the interaction effects
UQ[PY]LAB-V1.0-106 - 20 -
Sensitivity analysis
between X
v
and X
w
:
S
T
w
=
E [Var [Y |X
v
= x
v
]]
Var [Y ]
, w =∼v . (1.71)
This is the same definition of the sensitivity indices as in Section 1.5.3.2. However, in
the presence of correlation between input variables, the indices cannot be estimated with
Eq. (1.45) and the following. If X
v
is set equal to a certain value x
v
, the remaining depen-
dent variables have to be sampled conditionally to them and will be denoted as X
v
in the
following. Using this notation and the full expressions of variance and expectation, Eq. (1.70)
can be rewritten as:
S
v
=
1
D
Z
R
|v|
f
X
v
(x
v
) dx
v
Z
R
M−|v|
M(x
v
, x
w
)f
X
w
(x
v
, x
w
) dx
w
2
− M
2
0
, (1.72)
where D = Var [Y ], M
0
= E [Y ], f
X
v
is the PDF of X
v
, and f
X
w
is the conditional PDF of
X
w
given X
v
= x
v
.
1.5.5.1 Estimation of Kucherenko indices
Using the expression in Eq. (1.72), a Monte Carlo (MC) estimator of the first-order effect in
Eq. (1.70) can be formulated:
S
v
=
1
N
v
P
N
v
j=1
1
N
w
P
N
w
k=1
M(x
v,j
, x
w,k
)
2
− M
2
0
D
. (1.73)
This formula uses a double-loop MC estimation. It requires N
v
sample points of X
v
and for
each of them N
w
sample points of X
w
conditioned on X
v
= x
v,j
. Additionally, it requires N
samples of X to estimate the moments:
M
0
=
1
N
N
X
l=1
M(x
l
) , (1.74)
D =
1
N
N
X
l=1
M(x
l
)
2
− M
2
0
. (1.75)
The estimator in Eq. (1.73) converges for large values of both N
v
and N
w
. The final cost
in terms of model evaluations is N
tot
= N
v
N
w
+ N and can get extremely large in realistic
problems.
Kucherenko et al. (2012) estimators
Following Kucherenko et al. (2012), more efficient MC estimators of the first-order effect,
which circumvents the expensive double-loop, can be formulated. To this end, a new set of
notations is introduced. The samples x = (x
v
, x
w
) and x
′
= (x
′
v
, x
′
w
) are i.i.d. samples of X.
In (x
v
, x
′
w
), the overbar means that the marked sample x
′
w
is conditioned on x
v
.
UQ[PY]LAB-V1.0-106 - 21 -
UQ[PY]LAB user manual
The first estimator, hereafter referred to as the standard estimator, has the following form:
S
v
=
1
N
P
N
j=1
M(x
v,j
, x
w,j
)M(x
v,j
, x
′
w,j
)
−
1
N
P
N
j=1
M(x
v,j
, x
w,j
)
2
D
, (1.76)
where x
′
w
is conditioned on x
v
. Using the fact that S
T
v
= 1 − S
w
, a more efficient estimator
for the total effect can be formulated (Kucherenko et al., 2012):
S
T
v
=
1
N
P
N
j=1
M(x
v,j
, x
w,j
) − M(x
′
v,j
, x
w,j
)
2
2D
. (1.77)
In Eqs. (1.76) and (1.77), the estimators in Eqs. (1.74) and (1.75) are used. Per MC trial of
S
v
and S
T
v
, three model evaluations are required. Thus, the cost for the estimations of the
first-order and total indices of M variables
S
i
, S
T
i
, i = 1, . . . , M is N
tot
= N(2M + 1). The
samples
x
′
w
and x
′
v
are sampled conditionally to x
v
and x
w
, respectively.
Kucherenko et al. (2012) also proposed a modified MC estimator for the first-order effect:
S
v
=
1
N
P
N
j=1
M(x
v,j
, x
w,j
)
M(x
v,j
, x
′
w,j
) − M(x
′
v,j
, x
′
w,j
)
D
, (1.78)
This modified estimator improves convergence for small indices but requires one additional
model evaluation per MC trial. The total cost for the estimations of
S
i
, S
T
i
, i = 1, . . . , M
is therefore N
tot
= N(2M + 2).
Sample-based estimator
The Kucherenko indices can also be estimated purely based on an existing sample. This is
especially useful if conditional sampling, needed for the estimators in Eqs. (1.76)-(1.78),
is cumbersome, or even unavailable. Sample-based estimation is performed with a similar
approach to that of estimating Borgonovo indices (Section 1.5.2.1): exact conditioning on
specific values X
v
= x
v
is replaced by approximate conditioning on intervals a
l
v
< x
v
<
a
u
v
, where a
l
v
and a
u
v
are the multidimensional upper and lower bounds, respectively. The
approximate conditioning is achieved by selecting subsets of the available sample X that
belong to the relevant interval.
Given a sample of inputs X and outputs Y of size N, the estimator of the total effect is the
weighted mean of conditional variance estimates and takes the following form:
S
T
v
=
P
B
j=1
N
b,j
N
d
Var [Y|X
w,j
]
D
, (1.79)
where B is the total number of bins, N
b,j
is the number of sample points in the j-th bin, and
d
Var [Y|X
w,j
] is the sample variance of the outputs Y
j
that correspond to input sample points
X
w
that lie in the j-th bin. To compute this quantity, the input sample X is binned along
each conditioning set of variables X
w
. In another words, X is divided into B hyperrectangles
(bins). Figure 4 provides a visualization of this process for the estimation of S
T
1
in the case
of three input variables (X
1
, X
2
, X
3
) in which the conditioning variables X
2
and X
3
are used
to bin the sample points into B bins.
UQ[PY]LAB-V1.0-106 - 22 -
Sensitivity analysis
Analogously, the first-order index is the weighted variance of conditional mean estimates and
can be formulated as:
S
v
=
P
B
j=1
N
b,j
N
b
E [Y|X
v,j
] − bµ
b
E
2
D
, (1.80)
where
b
E [Y|X
v,j
] is the sample mean of the outputs Y
j
using the input sample points in
which X
v
lie in the j-th bin; while the term bµ
b
E
is the weighted sample mean of the bin
means
b
E [Y|X
v,j
], that is
bµ
b
E
=
B
X
j=1
N
b,j
N
b
E [Y|X
v,j
] . (1.81)
Contrary to the estimation of the total effect index, the input sample X is now binned along
X
v
. Since all indices can be estimated based on one set of sample (by binning it in different
ways), the total cost of estimation is N
tot
= N (i.e., if no output sample is provided).
Figure 4: Approximate conditional sampling from a given sample used to calculate closed
Kucherenko indices. The blue lines show the binning along X
w
= (X
2
, X
3
). The j-th bin /
hyperrectangle is highlighted in green. The sample points lying in this bin are highlighted in
red.
UQ[PY]LAB-V1.0-106 - 23 -
Chapter 2
Usage
In this section, a reference problem will be set up to showcase how each of the techniques
in Chapter 1 can be deployed in UQ[PY]LAB. This includes specification and running all
the different sensitivity analyses, so that the corresponding results can be visualized. The
following methods are showcased in this section:
• Input/output correlations: Section 2.1.2
• Standard regression coefficients (SRC/SRRC): Section 2.1.3
• Perturbation method: Section 2.1.4
• Cotter method: Section 2.1.5
• Morris elementary effects: Section 2.1.6
• Borgonovo sensitivity indices: Section 2.1.7
• Sobol’ sensitivity indices: Section 2.1.8
• ANCOVA sensitivity indices: Section 2.2.2
• Kucherenko sensitivity indices: Section 2.2.3
Note: As mentioned in Section 1.2, the SRC, the Perturbation method and the Sobol’
indices are only applicable in case the input variables are independent. All other
methods can be used regardless of dependence between inputs.
The goal of sensitivity analysis is to identify which parameters contribute the most to the
variability of the model output. To this end, we will make use of well known benchmarks
for sensitivity analysis: the borehole function and the short column limit state function
(see, e.g., www.sfu.ca/
˜
ssurjano/borehole.html and www.sfu.ca/
˜
ssurjano/
shortcol.html). Methods specifically developed to deal with dependent input random
vectors (i.e. ANCOVA and Kucherenko indices) are applied on the short column function, the
other methods are applied on the borehole function.
25
UQ[PY]LAB user manual
2.1 Independent input variables: borehole function
2.1.1 Problem statement and set-up
The so-called borehole function (http://www.sfu.ca/
˜
ssurjano/borehole.html) is
an analytical 8-dimensional function that models the flow of water through a borehole with
uncertain, independent parameters, given by the following equation:
f(x) =
2πT
u
(H
u
− H
l
)
ln(r/r
w
)
1 +
2LT
u
ln(r/r
w
)r
2
w
K
w
+
T
u
T
l
(2.1)
where the description and ranges of each parameter are given in Table 1.
Table 1: Distributions and descriptions of the parameters of the borehole function in
Eq. (2.1). The parameters refer to µ and σ for Gaussian distributions, to those of the un-
derlying Gaussian for lognormal distributions and to the bounds for uniform distributions.
Name Distributions Parameters Description
r
w
Gaussian [0.1, 0.0161812] radius of the borehole (m)
r Lognormal [7.71, 1.0056] radius of influence (m)
T
u
Uniform [63070, 115600] transmissivity of upper aquifer (m
2
/yr)
H
u
Uniform [990, 1110] potentiometric head of upper aquifer (m)
T
l
Uniform [63.1, 116] transmissivity of lower aquifer (m
2
/yr)
H
l
Uniform [700, 820] potentiometric head of lower aquifer (m)
L Uniform [1120, 1680] length of borehole (m)
K
w
Uniform [9855, 12045] hydraulic conductivity of borehole (m/yr)
The model in Eq. (2.1) is implemented as a Python file in:
borehole.py
To use it in a UQ[PY]LAB analysis, we need to define a MODEL object:
ModelOpts = {
'Type': 'Model',
'ModelFun':'borehole.model'
}
myModel = uq.createModel(ModelOpts)
For more details about the configuration options available for a MODEL object, refer to
UQ[PY]LAB User Manual – the MODEL module.
Similarly, the distributions in Table 1 can be implemented in UQ[PY]LAB as follows:
IOpts = {
'Marginals': [
{
'Name': 'rw', # Radius of the borehole
'Type': 'Gaussian',
'Parameters': [0.10, 0.0161812] # (m)
},
{
'Name': 'r', # Radius of influence
'Type': 'Lognormal',
'Parameters': [7.71, 1.0056] # (m)
UQ[PY]LAB-V1.0-106 - 26 -
Sensitivity analysis
},
{
'Name': 'Tu', # Transmissivity, upper aquifer
'Type': 'Uniform',
'Parameters': [63070, 115600] # (mˆ2/yr)
},
{
'Name': 'Hu', # Potentiometric head, upper aquifer
'Type': 'Uniform',
'Parameters': [990, 1110] # (m)
},
{
'Name': 'Tl', # Transmissivity, lower aquifer
'Type': 'Uniform',
'Parameters': [63.1, 116] # (mˆ2/yr)
},
{
'Name': 'Hl', # Potentiometric head , lower aquifer
'Type': 'Uniform',
'Parameters': [700, 820] # (m)
},
{
'Name': 'L', # Length of the borehole
'Type': 'Uniform',
'Parameters': [1120, 1680] # (m)
},
{
'Name': 'Kw', # Borehole hydraulic conductivity
'Type': 'Uniform',
'Parameters': [9855, 12045] # (m/yr)
},
]
}
# Create an INPUT object based on the specified marginals:
myInput = uq.createInput(IOpts)
For more details about the configuration options available for an INPUT object, refer to the
UQ[PY]LAB User Manual – the INPUT module.
2.1.2 Input/output correlation
Specifying an input/output correlation analysis as described in Section 1.3.1 only requires the
size of the sample from which to calculate the correlations. To create and run a correlation
analysis based on 10,000 points, the following syntax is used:
CorrSensOpts = {
'Type': 'Sensitivity',
'Method': 'Correlation',
'Correlation': {
'SampleSize': 1e4
}
}
CorrAnalysis = uq.createAnalysis(CorrSensOpts)
Once the analysis is performed, a report with the sensitivity results can be printed on screen
by:
UQ[PY]LAB-V1.0-106 - 27 -
UQ[PY]LAB user manual
uq.print(CorrAnalysis)
which produces the following:
-------------------------------------------
Correlation-based sensitivity indices:
-------------------------------------------
rw r Tu Hu Tl Hl L Kw
0.80726 -0.01936 -0.00695 0.31120 0.00329 -0.31218 -0.30322 0.15717
-------------------------------------------
Rank-Correlation-based sensitivity indices:
-------------------------------------------
rw r Tu Hu Tl Hl L Kw
0.81548 -0.01043 -0.00539 0.31197 0.00559 -0.31190 -0.29201 0.15002
Total cost (model evaluations): 10000
The results can be visualized graphically as follows:
uq.display(CorrAnalysis)
which produces the image in Figure 5.
rw r Tu Hu Tl Hl L Kw
−0.2
0
0.2
0.4
0.6
0.8
linear
rank
Correlation-based indices
Input Variable
ρ
Figure 5: Input/output linear and rank correlation results from the analysis described in
Section 2.1.2.
Note: In the preceding example only the sample size for the analysis is provided. If not
further specified the input/output correlation analysis runs with the following
default options:
– Sampling method: 'LHS'.
2.1.2.1 Accessing the results
The analysis results can be accessed in the CorrAnalysis['Results'] dictionary:
{
'CorrIndices': [0.8072546426041787, -0.019359815338160388, ...],
'RankCorrIndices': [0.8154775487587754, -0.010427128436271284, ...],
UQ[PY]LAB-V1.0-106 - 28 -
Sensitivity analysis
'ExpDesign': {'X': [[...], [...], ...], 'Y': [...]},
'Cost': 10000,
'VariableNames': ['rw', 'r', 'Tu', 'Hu', 'Tl', 'Hl', 'L', 'Kw']
}
In the dictionary, the CorrIndices and RankCorrIndices keys are arrays containing the
indices in Eq. (1.1) and (1.4), respectively. The Cost key represents the number of model
evaluations needed to calculate the indices, in this case equal to the sample size. A full
description of the format of the output can be found in Table 22.
2.1.2.2 Advanced options
The input/output correlation method is extremely simple. Thus there are only a few options
available for configuration. The most relevant are related to the generation of the sample
from which the coefficients are computed. In the following, two typical advanced usage
scenarios are presented:
• Specify the sampling scheme: by default, model realizations are sampled via simple
Monte Carlo from the input distributions. It is possible, however, to specify a different
sampling scheme (e.g.Latin Hypercube Sampling or Sobol’ pseudo-random sequences),
by adding the following option:
CorrSensOpts['Correlation']['Sampling'] = 'LHS'
Any of the sampling schemes supported by the default input module can be specified
(see Section 3.2, Page 37 of the UQ[PY]LAB User Manual – the INPUT module).
• Manually provide the samples: if input samples are already available and no addi-
tional sampling is needed, it is possible to use them directly for the analysis. In case
the samples are available in variables X_ED and the corresponding model evaluations
in variable Y_ED, they can be added as:
CorrSensOpts['Correlation'] = {
'Sample': {
'X': X_ED.tolist(),
'Y': Y_ED.tolist()
}
}
Note that in this case no SampleSize needs to be specified.
For a comprehensive list of the options available to the Correlation method, see Table 4.
2.1.3 Standard Regression Coefficients
Standard regression coefficients (SRC) and standard rank-regression coefficients (SRRC),
described in Section 1.3.2 only require a single configuration option: the size of the sample
on which to perform regression.
To create and run a SRC analysis based on a sample of 10,000 points, the following syntax is
used:
UQ[PY]LAB-V1.0-106 - 29 -
UQ[PY]LAB user manual
SRCSensOpts = {
'Type': 'Sensitivity',
'Method': 'SRC',
'SRC': {
'SampleSize': 1e4
}
}
SRCAnalysis = uq.createAnalysis(SRCSensOpts)
Once the analysis is performed, a report with the sensitivity results can be printed on screen
by:
uq.print(SRCAnalysis)
which produces the following:
-------------------------------------------
Standard Regression sensitivity indices:
-------------------------------------------
rw r Tu Hu Tl Hl L Kw
0.81735 0.00143 0.00274 0.30640 0.00448 -0.30980 -0.30117 0.14681
-------------------------------------------
Standard Rank-Regression sensitivity indices:
-------------------------------------------
rw r Tu Hu Tl Hl L Kw
0.82342 -0.00504 0.00458 0.30699 0.00174 -0.30573 -0.29028 0.14366
-------------------------------------------
The results can be visualized graphically as follows:
uq.display(SRCAnalysis)
which produces the image in Figure 6.
rw r Tu Hu Tl Hl L Kw
−0.2
0
0.2
0.4
0.6
0.8
SRC
SRRC
SRC results
Input Variable
Sensitivity
Figure 6: SRC and SRRC sensitivity results from the analysis described in Section 2.1.3.
UQ[PY]LAB-V1.0-106 - 30 -
Sensitivity analysis
Note: In the preceding example only the sample size for the analysis is provided. If not
further specified the standard regression analysis runs with the following default
options:
– Sampling method: 'LHS'.
2.1.3.1 Accessing the results
The analysis results can be accessed in the SRCAnalysis['Results'] dictionary:
{
'SRCIndices': [0.817349524184763, 0.0014320273246466594, ...],
'SRRCIndices': [0.8234237493126195, -0.005044069170986693, ...],
'Cost': 1000,
'VariableNames': ['rw', 'r', 'Tu', 'Hu', 'Tl', 'Hl', 'L', 'Kw']
}
In the dictionary, the SRCIndices and SRRCIndices keys are arrays containing the indices
in Eq. (1.8) and (1.11), respectively. The Cost key represents the number of model evalua-
tions needed to calculate the indices, in this case equal to the sample size. A full description
of the format of the output can be found in Table 23.
2.1.3.2 Advanced options
In the following, two typical advanced usage scenarios of the SRC/SRRC method are pre-
sented:
• Specify the sampling scheme: by default, model realizations are sampled via simple
Monte Carlo from the input distributions. It is possible, however, to specify a different
sampling scheme (e.g. Latin Hypercube Sampling or Sobol’ pseudorandom sequences),
by adding the following option:
SRCSensOpts['SRC']['Sampling'] = 'Sobol'
Any of the sampling schemes supported by the default input module can be specified
(see Section 3.2, Page 37 of the UQ[PY]LAB User Manual – the INPUT module).
• Manually provide the samples: if module samples are already available and no
additional sampling is needed, it is possible to use them directly for the analysis. In case
the samples are available in variables X_ED and the corresponding model evaluations
in variable Y_ED, they can be added as:
SRCSensOpts['SRC'] = {
'Sample': {
'X': X_ED.tolist(),
'Y': Y_ED.tolist()
}
}
Note that in this case no SampleSize needs to be specified.
For a comprehensive list of options available to the SRC method, see Table 6.
UQ[PY]LAB-V1.0-106 - 31 -
UQ[PY]LAB user manual
2.1.4 Perturbation method
Perturbation-based sensitivity analysis (Section 1.4.1) does not require any configuration
options since it is based on the numerical evaluation of the gradient of the model around the
mean value. To create and run a perturbation analysis, the following syntax is used:
PerturbationSensOpts = {
'Type': 'Sensitivity',
'Method': 'Perturbation'
}
PerturbationAnalysis = uq.createAnalysis(PerturbationSensOpts)
Once the analysis is performed, a report with the sensitivity results can be printed on screen
by:
uq.print(PerturbationAnalysis)
which produces the following:
-------------------------------------------
Perturbation-based sensitivity indices:
-------------------------------------------
rw r Tu Hu Tl Hl L Kw
0.69742 0.00000 0.00000 0.09584 0.00000 0.09584 0.08872 0.02218
-------------------------------------------
The results can be visualized graphically as follows: which produces the image in Figure 7.
rw r Tu Hu Tl Hl L Kw
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Perturbation-based indices
Input Variable
Sensitivity
Figure 7: Perturbation-based sensitivity results from the analysis described in Section 2.1.4.
Note: In the preceding example no specifications are provided. The perturbation
method runs with the following default options:
– Gradient steps: 'relative';
– Relative step size: 10
−3
;
– Finite difference scheme: 'forward' .
UQ[PY]LAB-V1.0-106 - 32 -
Sensitivity analysis
2.1.4.1 Accessing the results
The analysis results can be accessed in the PerturbationAnalysis['Results'] dictio-
nary:
{
'Mu': 70.93190986636091,
'Var': 749.103875762616,
'Sensitivity': [0.6974241956904639, 2.221361209777998e-06, ...],
'Cost': 9,
'VariableNames': ['rw', 'r', 'Tu', 'Hu', 'Tl', 'Hl', 'L', 'Kw']
}
In the dictionary, the Mu and Var keys report the estimated mean and variance of the model
output, while the Sensitivity key is the array containing the indices in Eq. (1.15). The
Cost key represents the number of model evaluations needed to calculate the indices, in this
case equal to the cost of evaluating partial derivatives of the model. A full description of the
format of the output can be found in Table 24.
2.1.4.2 Advanced options
The advanced options for the perturbation method are related to the numerical computation
of the gradient of the model. They allow one to:
• Specify the gradient method: by default, the gradient is calculated with the “forward”
method (to minimize the number of model evaluations). It can be set to any of “For-
ward”, “Centered” (more accurate, but more expensive to calculate) or “Backwards” as
follows:
PerturbationSensOpts['Gradient'] = {
'Method': 'Centered'
}
• Specify the gradient step mode and its value: by default, the gradient is calculated
numerically in the standardized space (i.e.in units of standard deviation), and the finite
difference step is set to h = 10
−3
. It is possible, however to specify both the mode and
the value as follows:
# specify the FD gradient step as absolute (not normalized)
# and give it a value of 0.0001
PerturbationSensOpts['Gradient'] = {
'Step': 'absolute',
'h': 0.0001
}
Note that no SampleSize needs to be specified. The 'absolute' option should be
used with caution when the magnitude of the various input parameters varies from one
another.
For a comprehensive list of options available to the perturbation method, see Table 8.
UQ[PY]LAB-V1.0-106 - 33 -
UQ[PY]LAB user manual
2.1.5 Cotter Measure
The Cotter measure does not require any specific configuration option, because the input
bounds (see Section 1.4.2) are set by default to the following values, depending on the input
distributions:
• Uniform, beta and truncated distributions: x
±
i
match the lower and upper bounds of the
corresponding input distribution.
• Lognormal distribution: x
−
i
= max(µ
i
− σ
i
, 0), x
+
i
= µ
i
+ σ
i
.
• Unbounded distributions: x
−
i
= µ
i
− σ
i
, x
+
i
= µ
i
+ σ
i
.
where µ
i
and σ
i
are the mean and standard deviation of variable X
i
, respectively, as defined
in the INPUT object (see Section 2.1.1). The Cotter sensitivity analysis is then created as
follows:
CotterSensOpts = {
'Type': 'Sensitivity',
'Method': 'Cotter'
}
CotterAnalysis = uq.createAnalysis(CotterSensOpts)
Once the analysis is performed, a report with the sensitivity results can be printed on screen
by:
uq.print(CotterAnalysis)
which produces the following:
-------------------------------------------
Cotter sensitivity indices:
-------------------------------------------
rw r Tu Hu Tl Hl L Kw
20.99107 43.87921 0.00017 18.15692 0.17056 18.15692 21.78663 7.94760
-------------------------------------------
The results can also be visualized graphically as follows:
uq.display(CotterAnalysis)
which produces the image in Figure 8.
UQ[PY]LAB-V1.0-106 - 34 -
Sensitivity analysis
rw r Tu Hu Tl Hl L Kw
0
5
10
15
20
25
30
35
40
45
Cotter indices
Input Variable
Cotter Index
Figure 8: Cotter sensitivity results from the analysis described in Section 2.1.5.
2.1.5.1 Accessing the results
The analysis results can be accessed in the CotterAnalysis['Results'] dictionary:
{
'CotterIndices': [],
'EvenOrder': [],
'OddOrder': [],
'Cost': ,
'VariableNames': ['rw', 'r', 'Tu', 'Hu', 'Tl', 'Hl', 'L', 'Kw']
}
The CotterIndices key is the array containing the indices in Eq. (1.16), while the OddOrder
and EvenOrder arrays contain the odd and even order effects in Eq. (1.17) and (1.18), re-
spectively. The Cost key represents the number of model evaluations needed to calculate
the indices, in this case (2M + 2), where M is the number of input variables.
A full description of the format of the output can be found in Table 25.
2.1.5.2 Advanced options
The Cotter method only allows for the customization of the low and high factor bounds
[x
−
i
, x
+
i
] (see Section 1.4.2). The bounds can be specified in two ways: absolute or relative
to the standard deviation of the inputs. The behaviour of the factors can be specified in the
CotterOptions['Factors'] dictionary as follows:
• Absolute bounds: they can be specified by setting the 1 × 2 array
CotterOptions['Factors'][ii]['Boundaries'] that contains lower and upper
bounds for each variable. As an example, to set all of the M inputs factors low and
high level to 0 and 1, respectively, the following code can be used:
CotterSensOpts['Factors'] = [{'Boundaries': [0, 1]} for i in range(M)]
• Relative bounds: bounds can be specified relative to the standard deviation according
UQ[PY]LAB-V1.0-106 - 35 -
UQ[PY]LAB user manual
to the following equation:
x
±
i
= µ
i
± k
i
σ
i
(2.2)
where k
i
is a positive scalar, µ
i
and σ
i
are the i−th factor mean and standard deviations
specified as in Section 2.1.1. There are two ways to specify the k
i
:
– If the Factors dictionary is a 1-element dictionary and Factors['Boundaries']is
a scalar, all the k
i
are set to its value, e.g.:
CotterSensOpts['Factors'] = {
'Boundaries': 0.5
}
will set all k
1
= · · · = k
m
= 0.5.
– If the Factors dictionary has M elements and each
CotterSensOpts['Factors'][ii]['Boundaries'] is a scalar b
i
, the k
i
= b
i
,
e.g.:
CotterSensOpts['Factors'] = [{'Boundaries': ii} for ii in range(M)]
will manually set each of the k
i
= i.
For a comprehensive list of options available to the Cotter method, see Table 9.
2.1.6 Morris Method
The Morris method only requires one configuration option, namely the maximum number of
full-model evaluations to be performed to estimate the elementary effects. Hence, a Morris
analysis with default settings and a maximum of 10
4
model evaluations can be configured as
follows:
MorrisSensOpts = {
'Type': 'Sensitivity',
'Method': 'Morris',
'Moris': {
'Cost': 1e4
}
}
MorrisAnalysis = uq.createAnalysis(MorrisSensOpts)
Note that the MorrisSensOpts['Morris']['Cost'] represents the maximum allowed cost.
UQ[PY]LAB will use this constraint conservatively, so that the effective cost is lower than or
equal to the specified value. Once the analysis is performed, a report with the sensitivity
results can be printed on screen by:
uq.print(MorrisAnalysis)
which produces the following:
--------------------------------------------------
Morris sensitivity indices:
--------------------------------------------------
rw r Tu Hu Tl Hl L Kw
UQ[PY]LAB-V1.0-106 - 36 -
Sensitivity analysis
mu: 40.3683 30.9515 0.0002 26.5961 0.2025 -26.260 -25.285 12.7802
mu
*
: 40.3683 31.0639 0.0002 26.5961 0.2025 26.2596 25.2849 12.7802
sigma: 18.6245 59.2618 0.0001 11.9724 0.1507 12.6968 12.7066 6.64024
--------------------------------------------------
Total cost (model evaluations): 9999
The results can also be visualized graphically as follows:
uq.display(MorrisAnalysis)
which produces the image in Figure 9.
−20 0 20 40
0
10
20
30
40
50
60
rw
r
Tu
Hu
Tl
Hl
L
Kw
Elementary effects
μ
σ
0 10 20 30 40
0
10
20
30
40
50
60
rw
r
Tu
Hu
Tl
Hl
L
Kw
Elementary effects
μ
*
σ
Figure 9: Morris sensitivity results from the analysis described in Section 2.1.6.
The results in Figure 9 are shown with the standard Morris format: the x-axis represents
the mean value of the elementary effects related to each variable, while the y-axis repre-
sents the corresponding standard deviation. Values close to the origin (lower left corner)
represent unimportant parameters. Parameters far from the origin in the x-direction are
important, while a large y-coordinate means that the parameter has a strong degree of non-
linearity/interaction.
Note: In the preceding example only the maximum cost for the analysis is provided. If
not further specified the Morris method runs with the following default options:
– Number of grid levels: 2×⌈
M
2
⌉ (closest even number of the number of input
variables);
– Perturbation steps: one-half of the number of grid levels.
2.1.6.1 Accessing the results
The analysis results can be accessed in the MorrisAnalysis['Results'] dictionary:
{
'Mu': [40.3682587794485, 30.95148367002072, ...],
'MuStar': [40.3682587794485, 31.06385822252627, ...],
'Std': [18.62453103316619, 59.261767090822104, ...],
'Cost': 9999,
'VariableNames': ['rw', 'r', 'Tu', 'Hu', 'Tl', 'Hl', 'L', 'Kw']
}
UQ[PY]LAB-V1.0-106 - 37 -
UQ[PY]LAB user manual
The mean value of the elementary effects for each variable is given in the Mu key , while
the corresponding standard deviation in the Std key (see Eq. (1.22)). The MuStar key
corresponds instead to the mean of the modified estimator in Eq. (1.25), especially useful for
ranking purposes. The Cost key represents instead the effective number of model evalua-
tions used to calculate the elementary effects. A full description of the format of the output
can be found in Table 26.
2.1.6.2 Advanced options
There are several parameters that can be fine-tuned in a Morris’ sensitivity analysis. They
include:
• Min-max boundaries of the parameters: they are similar to the boundaries for the
Cotter measure (see Section 2.1.5.2). They can be specified with the SensOpts['Factors']
key, with the same syntax and the same default values as for the Cotter sensitivity case.
• Perturbation grid: the input space between the variable boundaries is explored through
a regular grid as defined in Eqs. (1.19) to (1.21). Its parameters can be specified as
follows:
MorrisSensOpts['GridLevels'] = 6
MorrisSensOpts['PerturbationSteps'] = 3
where GridLevels and PerturbationSteps correspond to p in Eq. (1.19) and ¯c in
Eq. (1.21), respectively. Note that according to Morris (1991), GridLevels should be
equal to twice of PerturbationSteps.
The number of replications for each elementary effect (variable r, in Section 1.5.1) is deter-
mined directly from the specified Cost according to:
cost = r(M + 1), r =
cost
M + 1
(2.3)
where cost is the value specified in Section 2.1.6.
For a comprehensive list of options available to the Morris method, see Table 10.
2.1.7 Borgonovo indices
In order to set up a Borgonovo analysis only the size of the used sample needs to be specified.
As mentioned, a large amount of samples is recommended in order to achieve converging
results:
BorgonovoOpts = {
'Type': 'Sensitivity',
'Method': 'Borgonovo',
'Borgonovo': {
'SampleSize': 1e4
}
}
BorgonovoAnalysis = uq.createAnalysis(BorgonovoOpts)
UQ[PY]LAB-V1.0-106 - 38 -
Sensitivity analysis
Once the analysis is performed, a report with the sensitivity results can be printed on screen
by:
uq.print(BorgonovoAnalysis)
which produces the following:
--------------------------------------------------
Borgonovo indices for output component 1
--------------------------------------------------
rw r Tu Hu Tl Hl L Kw
0.41931 0.05497 0.05172 0.12057 0.05649 0.12333 0.11974 0.077731
--------------------------------------------------
Total cost (model evaluations): 10000
The results can be visualized graphically as follows:
uq.display(BorgonovoAnalysis)
which produces the image in Figure 10.
rw r Tu Hu Tl Hl L Kw
0
0.2
0.4
0.6
0.8
1
Borgonovo indices
Input Variable
𝛿
i
Figure 10: Borgonovo indices for the borehole model.
Note: In the preceding example only the sample size for the analysis is provided. If not
further specified the Borgonovo analysis runs with the following default options:
– Sampling method: 'LHS';
– Number of classes: 20;
– Binnig strategy to create classes: 'Quantile'
– Overlap between neighbouring classes: 2%;
– Method to estimate the inner integral: 'HistBased';
– Number of bins in the histograms: chosen separately for each class ('auto').
2.1.7.1 Accessing the results
The analysis results can be accessed in the BorgonovoAnalysis['Results']dictionary:
{
'Delta': [0.41931193454325266, 0.05496656494161635, ...],
'JointPDF': [[[...], [...], ...], ...],
UQ[PY]LAB-V1.0-106 - 39 -
UQ[PY]LAB user manual
0 5 10 15
0
2
4
6
8
10
12
14
16
18
0
0.1
0.2
0.3
0.4
0.5
Joint Pdf Y|rw
rw
Y
Figure 11: A histogram estimate of the joint PDF of Y and X
1
= r
w
. This is the estimate
used to compute the conditional distributions f
Y |X
1
in the computation of the corresponding
index.
'Cost': 100000,
'ExpDesign': {
'X': [[0.07720691938816371, 2490.152253059321, ...], ...],
'Y': [46.36637656340143, 158.34032805265318, ...]
},
'VariableNames': ['rw', 'r', 'Tu', 'Hu', 'Tl', 'Hl', 'L', 'Kw']
}
The sensitivity indices can be found in Delta. The estimated joint PDF is also stored in the
field JointPDF.
2.1.7.2 Advanced Options
In order to qualitatively assess the accuracy of the computation of a Borgonovo index, it is
useful to inspect the estimated joint probability distribution function of the model output
with the corresponding variable. For example, the estimate of the joint PDF f
Y,X
1
can be
visualized by:
uq.display(BorgonovoAnalysis, outidx=1, Joint_PDF=True, inidx=1)
In this advanced uq.display command, the second argument is the index of the model
output, the third one is a flag determining the type of visualization and the last is the input
variable index. The previous command produces the plot that is shown in Figure 11.
The Borgonovo analysis allows for many additional options to be specified, which can be
found in Section 3.1.6:
• CDF-based estimation of the inner integral: instead of the default histogram-based
estimation method 'HistBased', the CDF-based estimation method in Eq. (1.30) can
UQ[PY]LAB-V1.0-106 - 40 -
Sensitivity analysis
be chosen:
BorgonovoOpts['Borgonovo']['Method'] = 'CDFBased'
• Specify the binning strategy for the classes: by default UQ[PY]LAB creates non-
constant classes of the X
i
values by taking the quantiles of f
X
i
into account. This
binning strategy is labeled 'Quantile'. In order to use constant width classes specify:
BorgonovoOpts['Borgonovo']['BinStrat'] = 'Constant'
• Specify the amount and overlap of classes (in the X
i
-direction): by default UQ[PY]LAB
uses 20 classes with a 2% overlap. The user can modify both values:
BorgonovoOpts['Borgonovo']['NClasses'] = 15
BorgonovoOpts['Borgonovo']['Overlap'] = 0.1
Note that Overlap specifies the ratio of samples that overlap between two neighbouring
classes and therefore must be between 0 and 1.
• Specify the amount of bins for the unconditional histogram (in the Y -direction):
by default the number of bins for the histogram of f
Y
is optimally chosen for each class
by the histcounts MATLAB function. To fix the amount of bins to a certain number
specify:
BorgonovoOpts['Borgonovo']['NHistBins'] = 60
Note: it is important that the number of samples in each class is sufficiently large
to allow the binning in the specified amount of bins. If the chosen amount
of bins is too large for the available samples in a class, the histograms will
not be an accurate depiction of the conditional distribution which, conse-
quently, will lead to wrong estimates of the indices.
• Provide the samples to be analysed: instead of sampling, the user can provide sam-
ples to be used directly. If input and output samples are stored in variables X and Y,
they can be provided:
BorgonovoOpts['Borgonovo']['Sample'] = {
'X': X,
'Y': Y
}
Note that in this case SampleSize is set equal to the length of the provided samples.
For a complete account of the available computational options, refer to Section 3.1.6.
2.1.8 MC-based Sobol’ indices
The Sobol’ measure only requires one configuration option, namely the number of model
evaluations needed to calculate each index. Hence, a Sobol’ analysis with default settings
UQ[PY]LAB-V1.0-106 - 41 -
UQ[PY]LAB user manual
(Total + first order indices only, Janon estimator in Eq. (1.47)) and a maximum of 10, 000
model evaluations per dimension can be configured as follows:
SobolSensOpts = {
'Type': 'Sensitivity',
'Method': 'Sobol',
'Sobol': {
'SampleSize': 10000
}
}
SobolAnalysis = uq.createAnalysis(SobolSensOpts)
Note that the SobolSensOpts['Sobol']['SampleSize'] represents the number of model
evaluations used for each index. This is important as the number of indices that can be
calculated increases with the input dimension. The increase is linear for total and first order
indices, but increases factorially with the order of the indices. Once the analysis is performed,
a report with the sensitivity results can be printed on screen by:
uq.print(SobolAnalysis)
which produces the following:
--------------------------------------------------
Total Sobol’ indices for output component 1
--------------------------------------------------
rw r Tu Hu Tl Hl L Kw
0.6989 0.0000 0.0000 0.1126 0.0000 0.1085 0.1031 0.0252
--------------------------------------------------
--------------------------------------------------
First Order Sobol’ indices for output component 1
--------------------------------------------------
rw r Tu Hu Tl Hl L Kw
0.6600 0.0006 0.0006 0.1005 0.0007 0.0908 0.0875 0.0209
--------------------------------------------------
--------------------------------------------------
Total cost (model evaluations): 100000
The results can also be visualized graphically as follows:
uq.display(SobolAnalysis)
which produces the images in Figure 12.
rw r Tu Hu Tl Hl L Kw
0
0.2
0.4
0.6
0.8
1
Total Sobol' indices
Sobol' indices Order 1
Sobol' indices
Input Variable
Sensitivity
Figure 12: Sobol’ sensitivity results from the analysis described in Section 2.1.8.
UQ[PY]LAB-V1.0-106 - 42 -
Sensitivity analysis
Note: In the preceding example only the sample size for the analysis is provided. If not
further specified the Sobol’ analysis runs with the following default options:
– Sample size: 10, 000;
– Sampling method: 'MC';
– Estimator: 'Janon';
– Calculation of Sobol’ indices up to order: 1.
2.1.8.1 Accessing the results
The analysis results can be accessed in the SobolAnalysis['Results'] dictionary:
{
'AllOrders': [[0.6600183782322024, 0.0006050971249769527, ...], ...],
'Total': [0.6988861128146353, 3.024512600036111e-06, ...],
'FirstOrder': [0.6600183782322024, 0.0006050971249769527, ...],
'VarIdx': [[1, 2, 3, 4, 5, 6, 7, 8]],
'TotalVariance': 796.9635264054062,
'FactorIndex': [True, True, True, True, True, True, True, True],
'Cost': 100000,
'ExpDesign': {'Sample1': {'X': [...], 'Y': [...]}, ...},
'CoefficientBased': False,
'VariableNames': ['rw', 'r', 'Tu', 'Hu', 'Tl', 'Hl', 'L', 'Kw']
}
The Total and FirstOrder lists contain the total and first order Sobol’ indices, respectively.
All the indices (excluding the total Sobol’ indices but including the first order ones) are
grouped by order in the AllOrders list . The value of the indices of order j are given as
column vectors in AllOrders[j] The corresponding list of variables for each order is given
in the VarIdx[j] lists. To access the first order indices and the corresponding list of variables
in our example, e.g.:
> SobolAnalysis['Results']['AllOrders'][0]
[0.6600183782322024, 0.0006050971249769527, 0.00061142445380011,
0.10052167809932894, 0.0006623164420608658, 0.09075670396723774,
0.08751305180071746, 0.020899582241371322]
> SobolAnalysis['Results']['VarIdx'][0]
[1, 2, 3, 4, 5, 6, 7, 8]
Note: because the number of indices depends both on M and on the index order, AllOrders
and VarIdx are Python nested lists.
As in previous cases, Cost gives the total cost (in model evaluations) of the analysis. TotalVariance
is an list that contains the total variance of the model response. The CoefficientBased
flag is False if the indices are calculated via Sampling as in Section 1.5.3 or True if they
are calculated via PCE expansion as in Section 1.5.3.4. Finally, the ExpDesign key contains
information about the model evaluations used to calculate the Sobol’ indices.
For a complete overview of the output format of the Sobol’ analysis module, refer to Table 28.
UQ[PY]LAB-V1.0-106 - 43 -
UQ[PY]LAB user manual
2.1.8.2 Advanced Options
Several options can be fine-tuned for the calculation of Sobol’ indices. The most common
customization options are reported below:
• Specify the maximum Sobol’ index order: the maximum Sobol’ index order can be
specified with the Sobol['Order']option:
SobolSensOpts['Sobol']['Order'] = 2
• Specify an estimator: to use the estimator T in Eq. (1.47), set the following:
SobolSensOpts['Sobol']['Estimator'] = 'T'
• Specify the sampling scheme: the Monte-Carlo sampling scheme can be specified via
the Sobol['Sampling']option. Note that all of the options Section 3.2, Page 37 of
the UQ[PY]LAB User Manual – the INPUT module are valid:
SobolSensOpts['Sobol']['Sampling'] = 'LHS'
• Calculate confidence intervals: bootstrap-based confidence intervals can be calcu-
lated by UQ[PY]LAB by specifying the SobolSensOpts['Boostrap'] option. The
following code creates a bootstrap resampling with B = 100 samples and calculates
confidence bounds corresponding to the 0.025 and 0.975 quantiles:
SobolSensOpts['Bootstrap'] = {'Replications': 100}
SobolSensOpts['Alpha'] = 0.05
If bootstrap is used, an additional Bootstrap key is added to
SobolAnalysis['Results']:
{
'Total': {
'CI': [[[...]], [[...]], ...],
'Mean': [...],
'ConfLevel': [...]
}
'FirstOrder': {
'CI': [[[...]], [[...]], ...],
'Mean': [...],
'ConfLevel': [...]
}
'AllOrders': [['Empty']]
}
Details on how the Bootstrap results are reported are given in Table 29.
• Do not store the model evaluations: by default, the full model evaluations that are
run to calculate the Sobol’ indices are saved, as they can be expensive to calculate.
However, when higher order indices are calculated from inexpensive models, a very
large number of model evaluations may be generated (order of 10
9
), hence creating
UQ[PY]LAB-V1.0-106 - 44 -
Sensitivity analysis
taxing storage needs. It is possible, however, to remove the model evaluations after the
index calculation is complete, by setting:
SobolSensOpts['SaveEvaluations'] = False
For a complete reference list of options available for the calculation of Sobol’ indices, refer to
Table 14.
2.1.9 PCE-based Sobol’ indices
If a polynomial chaos expansion (PCE) metamodel created by UQ[PY]LAB is available, it is
possible to calculate Sobol’ indices analytically from its coefficients. A comparison between
Sobol’ indices calculated with Sampling (see Section 1.5.3) and directly from PCE (see Sec-
tion 1.5.3.4) for the borehole model is provided in the file:
2-Sobol Indices.ipynb
The problem is set up as in Section 2.1.1, but, in addition, a PCE metamodel is created before
defining the sensitivity analysis. This can be accomplished as follows (for details on how to
use the UQ[PY]LAB’s metamodelling toolbox, refer to the Section 2 of the UQ[PY]LAB User
Manual – Polynomial Chaos Expansions). The following code creates a sufficiently accurate
PCE of the borehole model based on an experimental design of 200 samples.
PCEOpts = {
# Select the 'metamodel tool' in UQ[py]Lab
'Type': 'Metamodel',
# Choose the Polynomial Chaos Expansion module
'MetaType': 'PCE',
# PCE requires an input to be defined
'Input': myInput['Name'],
# As well as a full model, if the Experimental Design is not provided
'FullModel': myModel['Name'],
# Run 5th degree PCE (default: sparse PCE expansion)
'Degree': 5,
# Specify the experimental design size (actual cost of the metamodel)
'ExpDesign': {
'NSamples': 200
}
}
# Calculate the PCE coefficients
myPCE = uq.createModel(PCEOpts)
To create a second order Sobol’ analysis the following is sufficient:
PCESobol = {
'Type': 'Sensitivity',
'Method': 'Sobol',
'Sobol': {
'Order': 2
}
}
PCESobolAnalysis = uq.createAnalysis(PCESobol)
UQ[PY]LAB-V1.0-106 - 45 -
UQ[PY]LAB user manual
Note that there is no necessity to specify that the Sobol’ indices should be calculated from the
PCE coefficients: if the current model is of type “PCE metamodel”, PCE-based Sobol’ indices
will be calculated. A comparison between Monte Carlos-based and PCE-based Sobol’ indices
is shown in Figure 13.
2.1.9.1 Accessing the results
The results are stored in the same format as for MC-based Sobol’ indices, see Section 2.1.8.1.
{
'Total': [0.6966366177968193, 3.158896013026147e-05, ...],
'FirstOrder': [0.6668262803174965, 0, ...],
'AllOrders': [[0.6668262803174965, 0, ...], ...],
'VarIdx': [[1, 2, 3, 4, 5, 6, 7, 8], [[...], ...],
'TotalVariance': 811.9572152146179,
'VariableNames': ['rw', 'r', 'Tu', 'Hu', 'Tl', 'Hl', 'L', 'Kw'],
'CoefficientBased': True
}
Note that the CoefficientBased flag is now automatically set to True and that there are no
ExpDesign nor Cost keys , because no new model evaluations are run.
2.1.9.2 Advanced Options
Except for the Sobol’ order, that is specified as in Section 2.1.8.2, only one option can be
specified for PCE-based Sobol’ analysis:
• Force Monte-Carlo based estimation of the indices: for comparison/validation, it
is possible to force the calculation of the Sobol’ indices through MC estimator as in
Section 2.1.8. The CoefficientBased flag is then specified as follows:
PCESobol['Sobol'] = {
'PCEBased': 0
}
2.1.10 LRA-based Sobol’ indices
As explained in Section 1.5.3.5, the Sobol’ indices can also be computed by post-processing
the coefficients of a canonical low-rank approximation (LRA) metamodel. A comparison be-
tween Sobol’ indices calculated with Sampling and directly from LRA for the borehole model
is provided in the file:
2-Sobol Indices.ipynb.
The problem is set up as in Section 2.1.9, but using a canonical LRA metamodel instead of a
PCE one. The following code creates a sufficiently accurate LRA of the borehole model based
on an experimental design of 200 samples.
LRAOpts = {
# Select the 'metamodel tool' in UQ[py]Lab
'Type': 'Metamodel',
UQ[PY]LAB-V1.0-106 - 46 -
Sensitivity analysis
# Choose the low-rank approximation module
'MetaType': 'LRA',
# Specify the input and full model for LRA
'Input': myInput['Name'],
'FullModel': myModel['Name'],
# Specify the rank and degree ranges
'Rank': list(range(1,21)),
'Degree': list(range(1,21)),
# Specify the experimental design size (total cost of the metamodel)
'ExpDesign': {
'NSamples': 200
}
}
# Calculate the LRA coefficients
myLRA = uq.createModel(LRAOpts)
To create a second order Sobol’ analysis the following is sufficient:
LRASobol = {
'Type': 'Sensitivity',
'Method': 'Sobol',
'Sobol': {
'Order': 2
}
}
LRASobolAnalysis = uq.createAnalysis(LRASobol)
Note that if the current model is of type “LRA metamodel”, LRA-based Sobol’ indices will
be calculated analytically from the expansion coefficients. A comparison between Monte
Carlo-based and LRA-based Sobol’ indices is shown in Figure 13.
Figure 13: Comparison between MC-based, PCE-based and LRA-based Sobol indices for the
borehole model. Left: total Sobol’ indices; Right: first order Sobol’ indices.
The results are stored in the same format as for MC-based and PCE-based Sobol’ indices (see
Sections 2.1.8.1 and 2.1.9.1).
Similarly to the PCE-based Sobol analysis, it is possible to force Monte-Carlo based calcula-
tion of the Sobol’ indices even with a LRA meta-model. This is specified with the command:
LRASobol['Sobol'] = {
UQ[PY]LAB-V1.0-106 - 47 -
UQ[PY]LAB user manual
'PCEBased': 0
}
2.2 Dependent input variables: short column function
2.2.1 Problem statement and set-up
The so-called short column function (http://www.sfu.ca/
˜
ssurjano/shortcol.html)
is an analytical 3-dimensional function that models the limit state of a short steel column with
a rectangular (b = 5 mm, h = 15 mm) cross-section that is subjected to a bending moment
and an axial force, given by the following equation:
g(x) = 1 −
4M
bh
2
Y
−
P
2
b
2
h
2
Y
2
(2.4)
where the description and the ranges of the non-constant parameters are given in Table 2.
Table 2: Distributions and descriptions of the parameters of the short column function in
Eq. (2.4). The parameters refer to µ and σ for Gaussian distributions and to those of the
associated Gaussian for the lognormal distribution.
Name Distributions Parameters Description
Y Lognormal [5, 0.5] yield stress (MPa)
M Gaussian [2000, 400] bending moment (Nmm)
P Gaussian [500, 100] axial force (N)
The axial force and the bending moment are assumed to be dependent. The dependence
is modeled by a Gaussian copula with a copula parameter of 0.72 between the two input
variables M and P .
The model in Eq. (2.4) is implemented as a Python file in:
shortcol.py
To use it in a UQ[PY]LAB analysis, we need to configure a MODEL module:
ModelOpts = {
'Type': 'Model',
'ModelFun': 'shortcol.model',
'isVectorized': 'true'
}
myModel = uq.createModel(ModelOpts)
The distributions in Table 2 and the dependence can be implemented as follows:
# Marginals
ModelOpts = {
'Marginals': [
{
'Name': 'Y', # yield stress
'Type': 'Lognormal',
'Parameters': [5, 0.5] # (MPa)
},
{
UQ[PY]LAB-V1.0-106 - 48 -
Sensitivity analysis
'Name': 'M', # bending moment
'Type': 'Gaussian',
'Parameters': [2000, 400] # (N.mm)
},
{
'Name': 'P', # axial force
'Type': 'Gaussian',
'Parameters': [500, 100] # (N)
}]
}
# Dependence
ModelOpts['Copula'] = {
'Type': 'Gaussian',
'RankCorr': [[1, 0, 0], [0, 1, 0.72], [0, 0.72, 1]]
}
myInput = uq.createInput(ModelOpts)
2.2.2 ANCOVA indices
To run an ANCOVA analysis for the problem introduced in Section 2.2.1, one specifies:
ANCOVAOpts = {
'Type': 'Sensitivity',
'Method': 'ANCOVA',
'ANCOVA': {
'SampleSize': 150
}
}
ANCOVAAnalysis = uq.createAnalysis(ANCOVAOpts)
When performing the analysis, UQ[PY]LAB will create a PCE metamodel (required by the
method, see Section 1.5.4) with default options. Advanced PCE Options can also be set
manually, see Section 2.2.2.2. Once the analysis is performed, a report with the results can
be printed on screen by:
uq.print(ANCOVAAnalysis)
which produces the following:
--------------------------------------------------
ANCOVA indices for output component 1
--------------------------------------------------
Indices Y M P
S 0.853 0.100 0.010
SˆU 0.849 0.092 0.001
SˆI 0.003 -0.000 -0.000
SˆC 0.002 0.009 0.008
--------------------------------------------------
Total cost (model evaluations): 150
The results can also be visualized graphically as follows:
uq.display(ANCOVAAnalysis)
which produces the plots in Figure 14.
UQ[PY]LAB-V1.0-106 - 49 -
UQ[PY]LAB user manual
Y M P
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Uncorrelated ANCOVA indices
Input Variable
S
i
U
Y M P
−0.15
−0.1
−0.05
0
0.05
0.1
0.15
Interactive ANCOVA indices
Input Variable
S
i
I
Y M P
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
Correlated ANCOVA indices
Input Variable
S
i
C
Y M P
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
First-order ANCOVA indices
Input Variable
S
i
Figure 14: ANCOVA sensitivity results from the analysis described in Section 2.2.2.
Note: In this example only the size of the experimental design was provided. If not
further specified, the ANCOVA runs with the following default options:
– Size of the experimental design for the PCE: 200;
– Sampling strategy: 'LHS';
– Number of samples used for the ANCOVA: 10,000;
– PCE of degree 1-10 based on 'LARS' (see also UQ[PY]LAB User Manual –
Polynomial Chaos Expansions).
2.2.2.1 Accessing the results
The results of the ANCOVA can be accessed in the ANCOVAAnalysis['Results']dictionary:
{
'Cost': 150,
'PCE': 'Model 2',
'TotalVariance': 0.0012798022801078928,
'Uncorrelated': [0.849081, 0.091899, 0.001359],
'Interactive': [0.002722, -0.000216, -3.301961e-05],
'Correlated': [0.001517, 0.008718, 0.008233],
'FirstOrder': [0.853320, 0.100401, 0.009559],
'VariableNames': ['Y', 'M', 'P']
}
The Uncorr, Interact and Corr lists contain the uncorrelated, interactive and correlated
UQ[PY]LAB-V1.0-106 - 50 -
Sensitivity analysis
ANCOVA indices, respectively. The list FirstSum contains the first-order indices (the sum
of the aforementioned indices) per variable and output. The Cost key shows the number
of model runs and is equal to the size of the experimental design. The name of the PCE
metamodel is stored in the PCE key . The used PCE metamodel can be accessed as follows:
PCEResults = uq.getModel(ANCOVAAnalysis['Results']['PCE'])
2.2.2.2 Advanced Options
The ANCOVA analysis requires the set-up of a PCE metamodel and sampling of the dependent
variables (see Section 1.5.4). Different specifications are possible:
• Specify the sampling strategy of the input variables: if desired, one can specify the
sampling strategy for the input variables using the ANCOVAOpts['ANCOVA']['Sampling']
option. Note that all of the options in Section 3.2, Page 37 of the UQ[PY]LAB User Man-
ual – the INPUT module are valid.
ANCOVAOpts['ANCOVA']['Sampling'] = 'LHS'
• Specify the number of samples used for the ANCOVA: the number of samples used
for the Monte Carlo estimation of the indices can be specified:
ANCOVAOpts['ANCOVA']['MCSamples'] = 10000
Note that these samples are used only with the PCE, which does not increase the com-
putational cost in terms of full model evaluations.
• Provide the experimental design for the set-up of the PCE: the main cost of the
method stems from the full model evaluations that are needed for the experimental
design. If a sample set of the input random vector and the corresponding model evalu-
ations are available, they can be provided as the experimental design for the PCE:
ANCOVAOpts['ANCOVA']['Samples'] = {
'X': X_ED.tolist(),
'Y': Y_ED.tolist()
}
• Specify the set-up of the PCE metamodel: to ensure the success of the ANCOVA
method, UQ[PY]LAB replaces the provided model by a PCE metamodel. For this set-
up, several options are available (see also UQ[PY]LAB User Manual – Polynomial Chaos
Expansions). For example, to specify an adaptive degree range of 5 : 10 and to force
the ordinary least squares method to compute the coefficients, the following shall be
used:
ANCOVAOpts['ANCOVA']['PCE'] = {
'Degree': list(range(5,11)),
'Method': 'OLS'
}
UQ[PY]LAB-V1.0-106 - 51 -
UQ[PY]LAB user manual
• Use an existing UQ[PY]LAB PCE metamodel: by default UQ[PY]LAB replaces the
provided model by a PCE before the ANCOVA analysis is performed, even if the pro-
vided model already is a PCE metamodel. If the user wishes to provide a PCE meta-
model myPCE that should be used directly in order to avoid the cost arising from the
set-up, the latter can be specified as follows:
ANCOVAOpts['ANCOVA']['CustomPCE'] = myPCE
Note: If this option is used, it is essential that the provided PCE is a UQ[PY]LAB
PCE object created with uq.createModel (see UQ[PY]LAB User Manual
– Polynomial Chaos Expansions). Any other MODEL object provided in
ANCOVASensOpts['Model'] will be ignored.
For a complete reference list of options available for the calculation of ANCOVA indices, refer
to Section 3.1.10.
2.2.3 Kucherenko indices
The main parameter for the Kucherenko analysis is the number of sample points N to be
used. The total cost of the analysis depends on the chosen estimator (see Section 1.5.5.1):
KucherenkoSensOpts = {
'Type': 'Sensitivity',
'Method': 'Kucherenko',
'Kucherenko': {
'SampleSize': 1e4
}
}
KucherenkoAnalysis = uq.createAnalysis(KucherenkoSensOpts)
Once the analysis is performed, a report with the results can be printed on screen by:
uq.print(KucherenkoAnalysis)
which produces the following:
--------------------------------------------------
Total Kucherenko indices for output component 1
--------------------------------------------------
Y M P
0.885428 0.057982 0.001890
--------------------------------------------------
--------------------------------------------------
First Order Kucherenko indices for output component 1
--------------------------------------------------
Y M P
0.895352 0.065015 0.045809
--------------------------------------------------
Total cost (model evaluations): 80000
The results can also be visualized graphically as follows:
uq.display(KucherenkoAnalysis)
which produces the images in Figure 15.
UQ[PY]LAB-V1.0-106 - 52 -
Sensitivity analysis
Note: In this example, only the number of samples was specified. If nothing else is
specified, the Kucherenko analysis runs with the following default options:
– Number of samples used for the Kucherenko analysis: 10,000;
– Sampling strategy: 'LHS';
– Estimator: 'Modified'.
Y M P
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Total Indices
First Order Indices
Kucherenko indices
Input Variable
Sensitivity
Figure 15: Kucherenko sensitivity results from the analysis described in Section 2.2.3.
2.2.3.1 Accessing the results
The results of the Kucherenko analysis are stored in KucherenkoAnalysis['Results'],
similarly to the other global sensitivity analyses.
{
'FirstOrder': [0.895352, 0.065015, 0.045809],
'Total': [0.885428, 0.057982, 0.001890],
'TotalVariance': 0.0013156867463267677,
'Cost': 80000,
'ExpDesign': {
'Sample1': {'X': [[...], ...], 'Y': [...]},
'Sample2': {'X': [[...], ...], 'Y': [...]},
'CondSamples': [{'X': [...], 'Y': [...]}, ...]
},
'VariableNames': ['Y', 'M', 'P']
}
2.2.3.2 Advanced Options
The following advanced options can be configured by the user:
• Specify the first-order index estimator: different estimators were presented in Sec-
tion 1.5.5. The 'Standard' estimator can be specified as follows:
KucherenkoSensOpts['Kucherenko']['Estimator'] = 'Standard'
UQ[PY]LAB-V1.0-106 - 53 -
UQ[PY]LAB user manual
• Provide a sample to estimate the indices : by specifying the 'SampleBased' esti-
mator (see sample-based estimation, Section 1.5.5.1), the user can provide the input X
(i.e., X ) and (optionally) the output samples Y (i.e., Y) used for the estimation:
KucherenkoSensOpts['Kucherenko']['Samples'] = {
'X': sampleX.tolist(),
'Y': sampleY.tolist()
}
In case only X is specified, sampleY = uq.evalModel(sampleX) will be automatically
calculated.
In alternative, the user can also define the size of the sample to be generated and used
for the estimation:
KucherenkoSensOpts['Kucherenko']['SampleSize'] = 1e5
For a complete reference list of options available for the calculation of Kucherenko indices,
refer to Section 3.1.11.
2.3 Sensitivity analysis of models with multiple outputs
2.3.1 Introduction
If a model has a vector-valued response Y = M(X) ∈ R
N
out
, UQ[PY]LAB will treat each
of the N
out
outputs independently and automatically. Full-model evaluations are of course
performed only once, regardless of the number of output components. Hence, no changes in
the syntax are needed to run the analyses presented throughout this chapter.
An example analysis calculating Sobol’ indices on a multi-component model (deflection of a
simply supported beam at multiple points) can be found in:
4-Multiple Outputs.ipynb
2.3.2 Accessing multi-output results
Results are stored in the same format as described in the previous section, with the only
difference that each output is stored in a different column of the results array. Consider as
an example the Sobol’ indices results in Section 2.1.8.1. If the number of output components
were increased to N
out
= 5, the results structure would read:
{
'AllOrders': [[[...], [...], [...], [...], [...]]],
'Total': [[0.04438143405414774, 0.04438143405421047, ...], ...],
'FirstOrder': [[0.033681191868571554, 0.033681191868414, ...], ...],
'VarIdx': [[1, 2, 3, 4, 5]],
'TotalVariance': [0, 0, 0, 0, 0, 0, 0, 0, 6.727057922321131e-06],
'FactorIndex': [True, True, True, True, True],
'Cost': 70000,
'ExpDesign': {
'Sample1': {'X': [[...], ...], 'Y': [[...], ...]},
UQ[PY]LAB-V1.0-106 - 54 -
Sensitivity analysis
'Sample2': {'X': [[...], ...], 'Y': [[...], ...]},
'Shuffled': {'X': [[...], ...], 'Y': [[...], ...]}
},
'CoeffiecientBased': False,
'VariableNames': ['rw', 'r', 'Tu', 'Hu', 'Tl', 'Hl', 'L', 'Kw']
}
The only difference with respect to the scalar model output case (N
out
= 1) lies in the fact
that now all of the result arrays have one column for each output.
Note: the VarIdx list for Sobol’ analysis is independent on the output component, hence its
dimension is unchanged, as it corresponds to v in Eq. (1.40).
2.4 Excluding parameters from the analysis
In some analyses, one may want to perform sensitivity analysis on a reduced set of input vari-
ables only. This can be important for methods like Morris’, MC-based Sobol’ or Kucherenko
sensitivity indices, whose costs (in terms of model evaluations) increase significantly with
the number of input variables. There are two ways to achieve this within UQ[PY]LAB: using
a parameter index (factor masking) or using constant input variables.
This process is transparent to the user as the analysis results will still show the excluded
variables, but their sensitivities will be set to 0. Whenever applicable, UQ[PY]LAB will
automatically and appropriately account for the set of input parameters which were declared
constant so as to avoid unnecessary model evaluations.
2.4.1 Using a factor index
Sensitivity calculation for selected factors can be included/excluded explicitly by means of
the SensOpts['FactorIndex'] configuration variable. SensOpts['FactorIndex'] is a
logical index of size (1 × M) that is True for the factors that are to be included and True
those that are to be excluded. As an example, to remove the calculation of the first index in
the MC-based Sobol’ analysis in Section 2.1.8, one can write:
SobolSensOpts['FactorIndex'] = [False, True, True, True, True, True, True,
True]
which results in the following SobolAnalysis['Results'] dictionary:
{
'AllOrders': [[0, 0.008229865900657347, ...]],
'Total': [0, 5.425110364831731e-06, ...],
'FirstOrder': [0, 0.008229865900657347, ...],
'VarIdx': [[1, 2, 3, 4, 5, 6, 7, 8]],
'TotalVariance': 237.46593910943335,
'FactorIndex': [False, True, True, True, True, True, True, True],
'Cost': 90000,
'ExpDesign': {
'Sample1': {'X': [[...], ...], 'Y': [[...], ...]},
'Sample2': {'X': [[...], ...], 'Y': [[...], ...]},
'Shuffled': {'X': [[...], ...], 'Y': [[...], ...]}
},
'CoefficientBased': False,
UQ[PY]LAB-V1.0-106 - 55 -
UQ[PY]LAB user manual
'VariableNames': ['rw', 'r', 'Tu', 'Hu', 'Tl', 'Hl', 'L', 'Kw']
}
Comparing to the same printout in Section 2.1.8, all output structures have the same format.
However, the total cost is lower (90, 000 vs 100, 000 model runs), as well as the total variance.
2.4.2 Using constant input variables
In some analyses, one may need to assign a constant value to one or to a set of parameters
input. When this is the case, the sensitivity analysis is built in UQ[PY]LAB by appropri-
ately setting the SensOptions['FactorIndex'] parameter so as to exclude the constant
variables from the analysis.
To set a parameter to constant, the following command can be used when the probabilistic
input is defined (See UQ[PY]LAB User Manual – the INPUT module):
inputOpts = {
'Marginals' = {
'Type': 'Constant',
'Parameters': value
}
}
Furthermore, when the standard deviation of a parameter is set to zero, UQ[PY]LAB auto-
matically sets this parameter’s marginal to the type Constant. For example, the following
uniformly distributed variable whose upper and lower bounds are identical is automatically
set to a constant with value 1:
inputOpts = {
'Marginals' = {
'Type': 'Uniform',
'Parameters': [1, 1]
}
}
UQ[PY]LAB-V1.0-106 - 56 -
Chapter 3
Reference List
How to read the reference list
Python dictionaries play an important role throughout the UQ[PY]LAB syntax. They offer
a natural way to semantically group configuration options and output quantities. Due to
the complexity of the algorithms implemented, it is not uncommon to employ nested dictio-
naries to fine-tune the inputs and outputs. Throughout this reference guide, a table-based
description of the configuration dictionaries is adopted.
The simplest case is given when a value of a dictionary key is a simple value or a list:
Table X: Input
Name String A description of the field is put here
which corresponds to the following syntax:
Input = {
'Name' : 'My Input'
}
The columns, from left to right, correspond to the name, the data type and a brief description
of each key-value pair. At the beginning of each row a symbol is given to inform as to whether
the corresponding key is mandatory, optional, mutually exclusive, etc. The comprehensive
list of symbols is given in the following table:
Mandatory
□ Optional
⊕ Mandatory, mutually exclusive (only one of
the keys can be set)
⊞ Optional, mutually exclusive (one of them
can be set, if at least one of the group is set,
otherwise none is necessary)
57
UQ[PY]LAB user manual
When the value of one of the keys of a dictionary is a dictionary itself, a link to a table that
describes the structure of that nested dictionary is provided, as in the case of the Options
key in the following example:
Table X: Input
Name String Description
□ Options Table Y Description of the Options
dictionary
Table Y: Input['Options']
Key1 String Description of Key1
□ Key2 Double Description of Key2
In some cases, an option value gives the possibility to define further options related to that
value. The general syntax would be:
Input = {
'Option1' : 'VALUE1',
'VALUE1' : {
'Val1Opt1' : ... ,
'Val1Opt2' : ...
}
}
This is illustrated as follows:
Table X: Input
Option1 String Short description
'VALUE1' Description of 'VALUE1'
'VALUE2' Description of 'VALUE2'
⊞ VALUE1 Table Y Options for 'VALUE1'
⊞ VALUE2 Table Z Options for 'VALUE2'
Table Y: Input['VALUE1']
□ Val1Opt1 String Description
□ Val1Opt2 Float Description
Table Z: Input['VALUE2']
□ Val2Opt1 String Description
□ Val2Opt2 Float Description
UQ[PY]LAB-V1.0-106 - 58 -
Sensitivity analysis
3.1 Create a sensitivity analysis
Syntax
myAnalysis = uq.createAnalysis(SOpts)
Input
The Dictionary variable SOpts contains the configuration information for a sensitivity anal-
ysis. The detailed list of options available is reported in Table 3.
Table 3: SOpts
Type 'Sensitivity' Identifier of the sensitivity module.
Method String Sensitivity analysis method. The
available options are listed below:
'Correlation' I/O correlation method
(Section 2.1.2).
'SRC' Standard Regression Coefficients
(Section 2.1.3).
'Perturbation' Perturbation Method (Section 2.1.4).
'Cotter' Cotter Method (Section 2.1.5).
'Morris' Morris method (Section 2.1.6).
'Borgonovo' Borgonovo indices (Section 2.1.7).
'Sobol' Sobol’ indices (Section 2.1.8, 2.1.9
and 2.1.10).
'ANCOVA' ANCOVA indices (Section 2.2.2).
'Kucherenko' Kucherenko indices (Section 2.2.3).
□ Input INPUT object Specification of the used input object
for the analysis. If not specified, the
currently active one is used.
□ Model MODEL object Specification of the used model
object for the analysis. If not
specified, the currently active one is
used.
□ FactorIndex 1 × M Boolean list
default: [True
for i in range(M)]
Vector with M elements that
contains True if the factor must be
taken into account or False if it
should be ignored in the
computations. Because the total
computational costs are related to
the number of factors, this option
can help save model evaluations.
□ SaveEvaluations Boolean
default: True
Store or not the performed model
evaluations.
UQ[PY]LAB-V1.0-106 - 59 -
UQ[PY]LAB user manual
True Model evaluations are saved.
False Model evaluations are not saved.
□ Display String
default: 'standard'
Level of information displayed by the
methods.
'quiet' Minimum display level, displays
nothing or very few information.
'standard' Default display level, shows the most
important information.
'verbose' Maximum display level, shows all the
information on runtime, like updates
on iterations, etc.
□ Gradient See Table 8 Define the options for computing the
gradient. It applies only to the
perturbation method, when
SOpts['Method'] =
'Perturbation'.
□ Factors See Table 9 Defines the intervals for input
parameters. It applies only when
SOpts['Method'] = 'Cotter'
or
SOpts['Method'] = 'Morris'.
□ Morris See Table 10 Define the options for the Morris
method. It applies only when
SOpts['Method'] = 'Morris'.
□ Sobol See Table 14, Table 16
and Table 17
Define the options for the Sobol’
indices. It applies only when
SOpts['Method'] = 'Sobol'.
□ Borgonovo See Table 12 Define the options for the Borgonovo
indices. It applies only when,
SOpts['Method'] =
'Borgonovo'.
□ ANCOVA See Table 18 Define the options for the ANCOVA
indices. It applies only when
SOpts['Method'] = 'ANCOVA'.
□ Kucherenko See Table 20 Define the options for the
Kucherenko indices. It applies only
when
SOpts['Method'] =
'Kucherenko'.
UQ[PY]LAB-V1.0-106 - 60 -
Sensitivity analysis
3.1.1 Input/Output correlation options
For details on the usage, please refer to Section 2.1.2.
Table 4: SOpts['Correlation']
□ Sampling String
default: 'LHS'
Sampling method to generate the
sample. All methods in Table 11,
page 37 of the UQ[PY]LAB User
Manual – the INPUT module are
supported.
□ Sample See Table 5 Manual specification of samples.
□ SampleSize Integer Number of samples to be generated.
Table 5: SOpts['Correlation']['Sample']
X N × M Float Manually provided input sampling.
Y N × N
out
Float Manually provided model response.
3.1.2 Standard Regression Coefficients (SRC and SRRC) options
For details on the usage, please refer to Section 2.1.3.
Table 6: SOpts['SRC']
SampleSize Integer Number of samples to be generated.
□ Sampling String
default: 'LHS'
Sampling method to generate the
sample. All methods in Table 11,
page 37 of the UQ[PY]LAB User
Manual – the INPUT module are
supported.
□ Sample See Table 7 Manually provided input sampling.
Table 7: SOpts['SRC']['Sample']
X N × M Float Manually provided input sampling.
Y N × N
out
Float Manually provided model response.
3.1.3 Perturbation Method options
For details on the usage, please refer to Section 2.1.4.
Table 8: SOpts['Gradient']
□ Step String
default: 'relative'
Defines how the perturbation h used
in the approximation is calculated.
UQ[PY]LAB-V1.0-106 - 61 -
UQ[PY]LAB user manual
'relative' h is an amount relative to the
standard deviation of each variable
h
i
= (Gradient['h']) · σ
i
'absolute' h is defined as an absolute value.
□ h Float
default: 10
−3
Value of the difference for the
scheme. If Gradient['Step'] is
set to 'relative', the difference
in each coordinate is computed as
h
i
= (Gradient['h']) · σ
i
□ Method String
default: 'forward'
Specifies the type of finite differences
scheme to be used.
'forward' Forward finite differences. Two
model evaluations M(x) and
M(x
1
, . . . , , x
i
+ h
i
, x
i+1
, . . . , x
M
)
are used.
'backward' Same as forward, but the increment
is backwards, M(x − h).
'centred' or
'centered'
Evaluates the model at two points in
the neighbourhood of the point of
interest, but not at the point itself. It
is more accurate, but it needs more
model evaluations.
3.1.4 Cotter Measure options
For details on the usage, please refer to Section 2.1.5.
Table 9: SOpts['Factors'][i]
Boundaries 1 × 2 Float lower and upper boundaries for the
i-th variable [x
−
i
, x
+
i
]
Float
default: 1
If Factors[i]['Boundaries'] is
a single scalar k, the intervals for
each factor are as x
±
i
= µ
i
± kσ
i
UQ[PY]LAB-V1.0-106 - 62 -
Sensitivity analysis
3.1.5 Morris Measure
For details on the usage, please refer to Section 2.1.6.
Table 10: SOpts['Morris']
⊕ Cost Integer Maximum number of evaluations
allowed for the Morris method,
i.e., Cost = (M + 1)r.
⊕ FactorSamples Integer Number of replications r for each
elementary effect, i.e.,
Cost = (M + 1)r.
□ GridLevels Integer
default:
2×PerturbationSteps
if the latter is defined,
2 × ⌈
M
2
⌉ otherwise
Number of points used to
discretize each interval where the
factors vary.
□ PerturbationSteps Integer
default: ⌊
GridLevels
2
⌋
Number of grid levels used for
computing each elementary
effect.
□ Factors[i] See Table 11 Minimum and maximum
boundaries for the i-th parameter.
Table 11: SOpts['Morris']['Factors'][i]
□ Boundaries 1 × 2 Float Absolute lower and upper
boundaries for the i-th parameter
(factor), i.e., [x
−
i
, x
+
i
].
Float
default: 1.0
A scalar that defines the relative
boundaries for the i-th parameter
(similar to the one for the Cotter
method, Eq. (2.2)). If a scalar k
i
is
defined for each parameter, the
upper and lower boundaries for the
i-th parameter are x
±
i
= µ
i
± k
i
σ
i
,
where µ
i
and σ
i
are the mean and
standard deviation of the i-th
parameter.
If a single scalar k is defined for all
parameters (i.e.,
SOpts['Factors']
['Boundaries'] = k), the upper
and lower boundaries for the i-th
parameter are x
±
i
= µ
i
± kσ
i
.
3.1.6 Borgonovo indices
For details on the usage, please refer to Section 2.1.7.
Table 12: SOpts['Borgonovo']
UQ[PY]LAB-V1.0-106 - 63 -
UQ[PY]LAB user manual
⊕ SampleSize Integer
default: 10000
Specify the number of random
samples.
□ Sampling String
default: 'LHS'
Sampling strategy for generating the
samples. (see Table 14)
⊕ Sample See Table 13 Provide the samples on which the
analysis will be conducted.
□ Sampling String
default: 'LHS'
Sampling strategy for generating the
samples. (see Table 14)
□ BinStrat String
default: 'Quantile'
Specify the binning strategy for the
creation of the classes.
'Quantile' Classes are created based on
quantiles to ensure enough samples
in each class.
'Constant' Classes have a constant width.
□ NClasses Integer
default: 20
Specify the amount of classes (in
X
i
-direction).
□ Overlap Float
default: 0.02
Specify the ratio of shared samples
between two neighbouring classes
(in X
i
-direction).
□ NHistBins Integer or String
default: 'auto'
Specify the amount of bins for the
histogram (in Y -direction).
'auto' Automatically choses a suitable
amount of bins for each class.
□ Method String
default: 'HistBased'
Specify the method used for the
estimation of the inner integral.
'HistBased' Estimation based on the absolute
differences between histograms.
'CDFBased' Estimation based on approximated
CDF’s.
Table 13: SOpts['Borgonovo']['Sample']
X N × M Float Manually provided input sample.
Y N × N
out
Float Manually provided model response.
UQ[PY]LAB-V1.0-106 - 64 -
Sensitivity analysis
3.1.7 MC-based Sobol’ indices
For details on the usage, please refer to Section 2.1.8.
Table 14: SOpts['Sobol'](MC-Sampling indices)
SampleSize Integer
default: 1000
Size of each of the samples to be
generated. The total number of
evaluations is N(M + 2), for a
problem of dimension M.
□ Estimator String
default: 't'
The type of estimator used to
approximate the Sobol’ indices.
'sobol', 'classic' Original estimator presented in
Sobol’ (1993), based on equations
(1.42)-(1.44).
'homma', 's' The estimator presented in Saltelli
and Homma (1996), as described in
(1.46).
'janon', 't' Estimator proposed by Janon et al.
(2014), described in equation (1.47).
It is the default estimator.
□ Order Integer
default: 1
Maximum order of the interactions
for which the Sobol’ indices are
estimated.
□ Sampling String
default: 'mc'
Sampling strategy for generating the
initial samples.
'halton' The samples are created with Halton
pseudo-random sampling.
'lhs' The samples are created with Latin
Hypercube sampling.
'mc' The samples are created with Monte
Carlo random sampling.
'sobol' The samples are created with Sobol’
pseudo-random sampling.
□ Bootstrap See Table 15 Specify the options for generating
bootstrap confidence intervals for the
estimates.
3.1.7.1 Boostrap error estimate on the Sobol’ indices
For details on the usage, please refer to Section 2.1.8.2.
Table 15: SOpts['Bootstrap']
□ Replications Integer
default: 0
Number of bootstrap replicates to
carry out for generating confidence
intervals.
UQ[PY]LAB-V1.0-106 - 65 -
UQ[PY]LAB user manual
□ Alpha Float
default: 0.05
The bootstrap confidence intervals
are constructed with confidence level
1 − α.
3.1.8 PCE-based Sobol’ indices
For details on the usage, please refer to Section 2.1.9.
Table 16: SOpts['Sobol'](PCE-based indices)
□ PCEBased Boolean
default: True
If set to False , UQ[PY]LAB will
compute the Monte Carlo estimates
of the Sobol’ indices, not the
analytical ones.
□ Order Integer
default: 1
Maximum order of the interactions
for which the Sobol’ indices are
calculated.
3.1.9 LRA-based Sobol’ indices
For details on the usage, please refer to Section 2.1.10.
Table 17: SOpts['Sobol'](LRA-based indices)
□ LRABased Boolean
default: True
If set to False , UQ[PY]LAB will
compute the Monte Carlo estimates
of the Sobol’ indices, not the
analytical ones.
□ Order Integer
default: 1
Maximum order of the interactions
for which the Sobol’ indices are
calculated.
3.1.10 ANCOVA indices
For details on the usage, please refer to Section 2.2.2.
Table 18: SOpts['ANCOVA']
⊞ SampleSize Float
default: 200
Size of the experimental design for
the PCE. Drives the cost of the
analysis.
□ Sampling String
default: 'LHS'
Sampling strategy for generating the
samples (see Table 14).
⊞ Samples See Table 19 Provide the experimental design for
the PCE.
□ MCSamples Integer
default: 10000
Specify how many samples are used
to get the MC estimations of the
indices.
UQ[PY]LAB-V1.0-106 - 66 -
Sensitivity analysis
□ CustomPCE UQ[PY]LAB MODEL object
of type PCE metamodel
Provide a PCE MODEL object to be
directly used by the analysis. Any
MODEL object in SOpts['Model']
will be ignored.
□ PCE See UQ[PY]LAB User
Manual – Polynomial
Chaos Expansions
Define the options for the set-up of
the PCE. By default
PCEOpts['Degree'] =
list(range(1,11))
PCEOpts['Method'] = 'LARS'
Table 19: SOpts['ANCOVA']['Samples']
X N × M Float Input sample. Its length will be set as
SampleSize.
□ Y N × N
out
Float Model evaluations of X.
3.1.11 Kucherenko indices
For details on the usage, please refer to Section 2.2.3.
Table 20: SOpts['Kucherenko']
⊕ SampleSize Float
default: 10
4
Size of the samples to be generated
and evaluated. If the
'SampleBased' estimator is
chosen, the default value is 10
5
.
□ Sampling String
default: 'LHS'
Sampling strategy for generating the
samples (see Table 14).
□ Estimator String
default: 'Modified'
Estimator for the indices.
'Standard' The first-order index is estimated
with Eq. (1.76) and the total index
with Eq. (1.77). The total cost is
N(2M + 1).
'Modified' The first-order index is estimated
with Eq. (1.78) and the total index
with Eq. (1.77). The total cost is
N(2M + 2).
'SampleBased' The first-order index is estimated
with Eq. (1.80) and the total index
with Eq. (1.79). If no output sample
is provided, the total cost is N.
□ Samples See Table 21 Manually provided input and output
samples. Only valid for the
'SampleBased' estimator.
Note: Both the 'Standard' and the 'Modified' estimators use Eq. (1.77) for the
estimation of the total index.
UQ[PY]LAB-V1.0-106 - 67 -
Sensitivity analysis
3.2 Accessing the results
Syntax
myAnalysis = uq.createAnalysis(SOpts)
Output
Each of the sensitivity analyses presented in Chapter 2 produces different results, albeit with
an overall similar structure. The details for each one of them are given in the following
tables:
• Input/output correlation - Table 22
• Standard regression coefficients - Table 23
• Perturbation - Table 24
• Cotter - Table 25
• Morris - Table 26
• Borgonovo indices - Table 27
• Sobol’ indices - Table 28
• PCE-based Sobol’ indices - Table 32
• LRA-based Sobol’ indices - Table 33
• ANCOVA indices - Table 34
• Kucherenko indices - Table 35
UQ[PY]LAB-V1.0-106 - 69 -
UQ[PY]LAB user manual
3.2.1 Input/Output correlation
See also Section 2.1.2.1.
Table 22: myAnalysis['Results']
CorrIndices M × N
out
Float Correlation-based sensitivity indices, according
to Eq. (1.2).
RankCorrIndices M × N
out
Float Rank-correlation-based sensitivity indices,
according to Eq. (1.4).
Cost Integer Total model evaluations carried out by the
method.
ExpDesign See Table 31 Model evaluations used during the calculation.
VariableNames 1 × M List List with the names of the input variables.
3.2.2 Standard Regression Coefficients
See also Section 2.1.3.1.
Table 23: myAnalysis['Results']
SRCIndices M × N
out
Float SRC sensitivity indices, according to Eq. (1.8).
SRRCIndices M × N
out
Float Rank-correlation-based sensitivity indices,
according to Eq. (1.11).
Cost Integer Total model evaluations carried out by the
method.
ExpDesign See Table 31 Model evaluations used during the calculation.
VariableNames 1 × M List List with the names of the input variables.
3.2.3 Perturbation Method
See also Section 2.1.4.1.
Table 24: myAnalysis['Results']
Mu Float First order estimate of the model mean.
Var Float First order estimate of the model variance.
Sensitivity M × N
out
Float Sensitivity estimates of the perturbation
method, from equation (1.15).
Cost Integer Total model evaluations carried out by the
method.
ExpDesign See Table 31 If SOpts['SaveEvaluations'] was set to
True , the model evaluations are saved in this
struct.
VariableNames 1 × M List List with the names of the input variables.
UQ[PY]LAB-V1.0-106 - 70 -
Sensitivity analysis
3.2.4 Cotter Method
See also Section 2.1.5.1.
Table 25: myAnalysis['Results']
CotterIndices M × N
out
Float Vector containing the value of the Cotter
sensitivity measure for each of the factors from
equation (1.16).
EvenOrder M × N
out
Float Vector containing the value of the even order
effect for each of the factors.
OddOrder M × N
out
Float Vector containing the value of the odd order
effect for each of the factors.
Cost Integer Total model evaluations carried out by the
method.
ExpDesign See Table 31 If SOpts['SaveEvaluations'] was set to
True , the model evaluations are saved in this
struct.
VariableNames 1 × M List List with the names of the input variables.
UQ[PY]LAB-V1.0-106 - 71 -
UQ[PY]LAB user manual
3.2.5 Morris Method
See also Section 2.1.6.1.
Table 26: myAnalysis['Results']
Mu M × N
out
Float Original Morris sensitivity measure, µ, mean of
the elementary effects.
MuStar M × N
out
Float Improved Morris sensitivity measure, µ
∗
(Eq. (1.25)).
Std M × N
out
Float Standard deviation of the elementary effects.
Cost Integer Total model evaluations performed.
ExpDesign See Table 31 Model evaluations used for the estimation
(if SOpts['SaveEvaluations']=1).
VariableNames 1 × M List List with the names of the input variables.
3.2.6 Borgonovo indices
See also Section 2.1.7.
Table 27: myAnalysis['Results']
Delta M × N
out
Float Borgonovo index estimations for each input
variable and output component.
JointPDF M × N
out
List 2D histogram approximations of the joint
distributions of each input variable and output
component.
Cost Integer Total model evaluations carried out by the
method.
ExpDesign Similar to
Table 31
The used experimental design. Does not
contain a Shuffled field!
VariableNames 1 × M List List with the names of the input variables.
UQ[PY]LAB-V1.0-106 - 72 -
Sensitivity analysis
3.2.7 MC-based Sobol’ indices
See also Section 2.1.8.1.
Table 28: myAnalysis['Results']
Total M × N
out
Float Array of total Sobol’ indices. Each column
corresponds to an output variable.
FirstOrder M × N
out
Float Array of first order Sobol’ indices. Each column
corresponds to an output variable.
AllOrders List Each element AllOrders[i] is a (M
i
× N
out
)
array of i-th order Sobol’ indices for each of the
J outputs.
VarIdx List Each element VarIdx[i] is a (M
i
× i) array
with interactions considered in the Sobol’
indices in AllOrders[i]. For Example, if
VarIdx[1] = [1 2], then AllOrders[1]
contains the second order Sobol’ index S
1,2
(see Eq. (1.40)).
TotalVariance 1 × N
out
Float Total variance of each output of the model.
Cost Integer Total model evaluations carried out by the
method.
Bootstrap See Table 29 Results of the bootstrap method.
ExpDesign See Table 31 If SOpts['SaveEvaluations'] is True ,
the model evaluations are saved in this Struct.
VariableNames 1 × M List List with the names of the input variables.
CoefficientBased Boolean If True , the indices are calculated analytically
for either a PCE or a LRA metamodel,
otherwise they are Monte Carlo estimates.
UQ[PY]LAB-V1.0-106 - 73 -
UQ[PY]LAB user manual
3.2.7.1 Bootstrap
See also Section 2.1.8.1.
Table 29: myAnalysis['Results']['Bootstrap']
Total See Table 30 Dictionary containing the bootstrap
information for the total Sobol’ indices.
FirstOrder See Table 30 Dictionary containing the bootstrap
information for the first order Sobol’ indices.
AllOrders See Table 30 List containing the bootstrap information for
all the calculated Sobol’ indices, grouped by
order. Each element AllOrders[i] refers the
i-th order indices.
Table 30: myAnalysis['Results']['Bootstrap']['Total']/['FirstOrder']/['AllOrders'][i]
CI N
ind
× N
out
× 2
Float
Upper and lower bounds of the confidence
interval for each of the N
ind
indices and N
out
output variables.
ConfLevel N
ind
× N
out
Float
Confidence level for each of the N
ind
indices
and N
out
output variables.
Mean N
ind
× N
out
Float
Means of the bootstrap estimates for each of
the N
ind
indices and N
out
output variables.
3.2.7.2 Model Evaluations
If the SOpts['SaveEvaluations'] option is set to True , the model evaluations will be
saved in the myAnalysis['Results']['ExpDesign'] dictionary detailed in Table 31.
Table 31: myAnalysis['Results']['ExpDesign']
X N × M Float Coordinates of the experimental design.
Y N × N
out
Float Model responses.
Shuffled[ii] Dictionary
Experimental design for higher order indices.
(Monte Carlo estimators only.)
UQ[PY]LAB-V1.0-106 - 74 -
Sensitivity analysis
3.2.8 PCE-based Sobol’ indices
See also Section 2.1.9.1.
Note: this table is essentially identical to Table 28, with the exclusion of the model-evaluation
and LRA related fields (Cost, ExpDesign, Bootstrap and CoefficientBased).
Table 32: myAnalysis['Results']
Total M × N
out
Float Array of total Sobol’ indices. Each column
corresponds to an output variable.
FirstOrder M × N
out
Float Array of first order Sobol’ indices. Each column
corresponds to an output variable.
AllOrders List Each element AllOrders[i] is a (M
i
× N
out
)
array of i-th order Sobol’ indices for each of the
J outputs.
VarIdx List Each element VarIdx[i] is a (M
i
× i) array
with interactions considered in the Sobol’
indices in AllOrders[i]. For Example, if
VarIdx[1] = [1 2], then AllOrders[1]
contains the second order Sobol’ index S
1,2
(see Eq. (1.40)).
TotalVariance 1 × N
out
Float Total variance of each output of the model.
VariableNames 1 × M List List with the names of the input variables.
PCEBased Boolean If True , the indices are analytical from the
PCE coefficients, otherwise they are Monte
Carlo estimates.
3.2.9 LRA-based Sobol’ indices
Note: this table is essentially identical to Table 28, with the exclusion of the model-evaluation
and PCE related fields (Cost, ExpDesign, Bootstrap and PCEBased).
Table 33: myAnalysis['Results']
Total M × N
out
Float Array of total Sobol’ indices. Each column
corresponds to an output variable.
FirstOrder M × N
out
Float Array of first order Sobol’ indices. Each column
corresponds to an output variable.
AllOrders List Each element AllOrders[i] is a (M
i
× N
out
)
array of i-th order Sobol’ indices for each of the
J outputs.
VarIdx List Each element VarIdx[i] is a (M
i
× i) array
with interactions considered in the Sobol’
indices in AllOrders[i]. For Example, if
VarIdx[1] = [1 2], then AllOrders[1]
contains the second order Sobol’ index S
1,2
(see Eq. (1.40)).
TotalVariance 1 × N
out
Float Total variance of each output of the model.
UQ[PY]LAB-V1.0-106 - 75 -
UQ[PY]LAB user manual
VariableNames 1 × M List List with the names of the input variables.
CoefficientBased Boolean If True , the indices are analytical from the
LRA coefficients, otherwise they are Monte
Carlo estimates.
UQ[PY]LAB-V1.0-106 - 76 -
Sensitivity analysis
3.2.10 ANCOVA indices
See also Section 2.2.2.1
Table 34: myAnalysis['Results']
PCE Dictionary The set up PCE model for the analysis.
Cost Float Total model evaluations carried out by the
method.
TotalVariance 1 × N
out
Float Total variance of each output of the model.
Uncorrelated M × N
Out
Float
Array of the uncorrelated ANCOVA indices.
Each column corresponds to an output
variable.
Interactive M × N
Out
Float
Array of the interactive ANCOVA indices. Each
column corresponds to an output variable.
Correlated M × N
Out
Float
Array of the correlated ANCOVA indices. Each
column corresponds to an output variable.
FirstOrder M × N
Out
Float
Array of the first order ANCOVA indices which
are equal to the sum of the uncorrelated,
interactive and correlated indices. Each
column corresponds to an output variable.
VariableNames 1 × M List List with the names of the input variables.
3.2.11 Kucherenko indices
See also Section 2.2.3.1
Table 35: myAnalysis['Results']
Cost Float Total model evaluations carried out by the
method.
TotalVariance 1 × N
out
Float Total variance of each output of the model.
FirstOrder M × N
out
Float Array of first order Kucherenko indices. Each
column corresponds to an output variable.
Total M × N
out
Float Array of total Kucherenko indices. Each
column corresponds to an output variable.
VariableNames 1 × M List List with the names of the input variables.
UQ[PY]LAB-V1.0-106 - 77 -
UQ[PY]LAB user manual
3.3 Printing/Visualizing of the results
UQ[PY]LAB offers two commands to conveniently print reports containing contextually rele-
vant information for a given object:
3.3.1 Printing the results: uq.print
Syntax
uq.print(myAnalysis)
uq.print(myAnalysis, outidx)
Description
uq.print(myAnalysis) print a report on the results of the sensitivity analysis in object
myAnalysis. If the model has multiple outputs, only the results for the first output
variable are printed.
uq.print(myAnalysis, outidx) print a report on the results of the sensitivity analysis
in object myAnalysis for the output variables specified in the array outidx.
Examples:
uq.print(myAnalysis, [1, 3]) will print the sensitivity analysis results for output vari-
ables 1 and 3.
3.3.2 Graphically display the results: uq.display
Syntax
uq.display(myAnalysis)
uq.display(myAnalysis, outidx)
Description
uq.display(myAnalysis) create a visualization of the results of the sensitivity analysis in
object myAnalysis, if possible. If the model has multiple outputs, only the results for
the first output variable are visualized.
uq.display(myAnalysis, outidx) create a visualization of the results of the sensitivity
analysis in object myAnalysis for the output variables specified in the array outidx.
Examples:
uq.display(myAnalysis, [1, 3]) will display the sensitivity analysis results for output
variables 1 and 3.
See also the examples in Part 2.
UQ[PY]LAB-V1.0-106 - 78 -
References
Borgonovo, E. (2007). A new uncertainty importance measure. Reliability Engineering &
System Safety 92, 771–784. 9, 10
Campolongo, F., J. Cariboni, and A. Saltelli (2007). An effective screening design for sen-
sitivity analysis of large models. Environmental modelling & software 22(10), 1509–1518.
8
Caniou, Y. (2012). Global sensitivity analysis for nested and multiscale models. Ph. D. thesis,
Universit
´
e Blaise Pascal, Clermont-Ferrand. 10, 11, 18, 19
Cotter, S. (1979). A screening design for factorial experiments with interactions.
Biometrika 66, 317–320. 5
Efron, B. (1979). Bootstrap methods: another look at the Jackknife. Annals of Statistics 7(1),
1–26. 18
Iooss, B. and P. Lema
ˆ
ıtre (2015). A review on global sensitivity analysis methods. In C. Mel-
oni and G. Dellino (Eds.), Uncertainty management in Simulation-Optimization of Complex
Systems: Algorithms and Applications. Springer. 1
Janon, A., T. Klein, A. Lagnoux, M. Nodet, and C. Prieur (2014). Asymptotic normality and
efficiency of two Sobol’ index estimators. ESAIM: Probability and Statistics 18, 342–364.
15, 65
Konakli, K. and B. Sudret (2016). Global sensitivity analysis using low-rank tensor approxi-
mations. Reliability Engineering & System Safety 156, 64 –83. 17, 18
Kucherenko, S., S. Tarantola, and P. Annoni (2012). Estimation of global sensitivity indices
for models with dependent variables. Computer Physics Communications 183, 937–946.
20, 21, 22
Li, G., H. Rabitz, P. Yelvington, O. Oluwole, F. Bacon, C. Kolb, and J. Schoendorf (2010).
Global sensitivity analysis for systems with independent and/or correlated inputs. The
Journal of Physical Chemistry A 114, 6022–6032. 19
Liu, Q. and T. Homma (2009). A new computational method of a moment-independent
uncertainty importance measure. Reliability Engineering & System Safety 94(7), 1205–
1211. Special Issue on Sensitivity Analysis. 11
79
UQ[PY]LAB user manual
Morris, M. D. (1991). Factorial sampling plans for preliminary computational experiments.
Technometrics 33(2), 161–174. 6, 8, 38
Plischke, E., E. Borgonovo, and L. S. Smith (2013). Global sensitivity measures from given
data. European Journal of Operational Research 226(3), 536 – 550. 10
Saltelli, A. and T. Homma (1996). Importance measures in global sensitivity analysis of
model output. Reliability Engineering & System Safety 52, 1–17. 15, 65
Sobol’, I. M. (1993). Sensitivity estimates for nonlinear mathematical models. 1(4), 407–
414. 12, 13, 65
Sudret, B. (2008). Global sensitivity analysis using polynomial chaos expansions. Reliability
Engineering & System Safety 93(7), 964–979. 16
Sudret, B. and Y. Caniou (2013). Analysis of covariance (ANCOVA) using polynomial chaos
expansions. In G. Deodatis (Ed.), Proc. 11th Int. Conf. Struct. Safety and Reliability (ICOS-
SAR’2013), New York, USA. 18
Xu, C. and G. Z. Gertner (2008). Uncertainty and sensitivity analysis for models with corre-
lated parameters. Reliability Engineering & System Safety 93(10), 1563–1573. 18, 19
UQ[PY]LAB-V1.0-106 - 80 -
