Description

A MATLAB Implementation of the Minimum Relative Entropy Method for Linear Inverse Problems* 1

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.

Related Documents

Share

Transcript

A MATLAB Implementation of theMinimum Relative Entropy Methodfor Linear Inverse Problems
Roseanna M. Neupauer
1
and Brian Borchers
2
1
Department of Earth and Environmental Science
2
Department of MathematicsNew Mexico Institute of Mining and Technology, Socorro, NM 87801, USAborchers@nmt.edu
March 31, 2000
Abstract
The minimum relative entropy (MRE) method can be used to solve linear inverseproblems of the form
Gm
=
d
, where
m
is a vector of unknown model parameters and
d
is a vector of measured data. The MRE method treats the elements of
m
as randomvariables, and obtains a multivariate probability density function for
m
. The proba-bility density function is constrained by prior information about the upper and lowerbounds of
m
, a prior expected value of
m
, and the measured data. The solution of the
1
inverse problem is the expected value of
m
, based on the derived probability densityfunction. We present a MATLAB implementation of the MRE method. Several nu-merical issues arise in the implementation of the MRE method and are discussed here.We present the source history reconstruction problem from groundwater hydrology asan example of the MRE implementation.
Keywords
Maximum Entropy, Minimum Relative Entropy, Inverse Problems
1 Introduction
In this paper, we present a MATLAB implementation of the minimum relative entropy(MRE) method to solve linear inverse problems of the form
Gm
=
d
,
(1)where
G
is a matrix of size
N
by
M
,
m
is a vector of length
M
containing unknownmodel parameters, and
d
is a data vector of length
N
. The vector
d
consists of measured data, and typically includes measurement error. The matrix
G
is typicallyvery badly conditioned, making a conventional least squares approach impractical.In the MRE method, the unknown model parameters,
m
, are treated as randomvariables, and the solution to the inverse problem is obtained from the multivariateprobability density function (pdf) of
m
. The pdf is selected in a two-step process.First, we generate a prior distribution whose entropy is minimized subject to constraints
2
imposed by the lower and upper bounds of
m
(
l
and
u
, respectively) and by an estimate,
s
, of the expected value of
m
. This prior distribution ensures that the solution of theinverse problem is within the speciﬁed bounds on
m
, but does not guarantee thatEq. (1) is satisﬁed.In the second step, we select a posterior distribution for
m
, whose entropy is min-imized relative to the prior distribution and subject to the constraint imposed by themeasured data
d
. The mean of the posterior pdf,ˆ
m
, can be used as a “solution” tothe inverse problem. The 5th and 95th percentiles of the posterior distribution deﬁnea 90% probability interval, similar to the 90% conﬁdence interval in classical statisti-cal methods. We can investigate the properties of the solution by using the posteriordistribution to randomly generate many realizations of
m
.The goal of this paper is to present the MRE method in the context of a generaldiscrete linear inverse problem, to discuss some important issues in the implementationof the method, and to describe our MATLAB implementation of the MRE method.We also present an example taken from groundwater hydrology.
2 The MRE Method
In this section, we describe the generation of the prior distribution,
p
(
m
), and theposterior distribution,
q
(
m
), of
m
. The elements of
m
are assumed to be independent,so
p
(
m
) =
M i
=1
p
(
m
i
) and
q
(
m
) =
M i
=1
q
(
m
i
), where
M
is the number of modelparameters.In this discussion, we assume that the lower bound is zero. As we show later,
3
non-zero lower bounds can easily be incorporated with a change of variables. Fora continuous random variable with ﬁnite upper bounds, zero lower bounds, and aﬁnite expected value within the bounds, it can be shown that the maximum entropydistribution is a truncated exponential distribution (Kapur and Kesavan, 1992). Usingthis fact we can derive the prior distribution
p
(
m
i
) =
β
i
exp(
−
β
i
m
i
)1
−
exp(
−
β
i
U
i
)for
β
i
= 0 (2)
p
(
m
i
) =1
U
i
for
β
i
= 0 ;
p
(
m
) =
M
i
=1
p
(
m
i
)
,
where
U
i
is the upper bound of parameter
m
i
, and
β
i
is a Lagrange multiplier whosevalue is determined by satisfying the expected value constraint,
m
i
p
(
m
i
)
dm
i
=
s
i
,where
s
i
is the prior expected value of
m
i
for
i
= 1
,
2
,...,M
. By integrating Eq. (2),we obtain the expected value equation that is used to evaluate
β
i
−
(
β
i
U
i
+ 1)exp(
−
β
i
U
i
) + 1
β
i
[1
−
exp(
−
β
i
U
i
)]=
s
i
.
(3)Details of this derivation can be found in Neupauer (1999).To obtain the posterior distribution,
q
(
m
), we minimize its entropy relative to theprior distribution,
p
(
m
). The entropy to be minimized is
H
(
q,p
) =
m
q
(
m
)ln
q
(
m
)
p
(
m
)
d
m
,
(4)where
p
(
m
) is given in Eq. (2). This minimization is subject to two constraints—thenormalization requirement
m
q
(
m
)
d
m
= 1, and the requirement that the posterior
4
mean solution ﬁt the data within a speciﬁed tolerance
||
d
−
G
ˆ
m
||
2
≤
ξ
2
ǫ
2
,
(5)where
||·||
denotes the L
2
norm,ˆ
m
is the mean of the posterior distribution,
ǫ
is themeasurement error, and
ξ
is a parameter that depends on the assumed error model.This constrained optimization problem is solved by the method of Lagrange multipliersby minimizing
φ
=
H
(
q,p
) +
µ
m
q
(
m
)
d
m
−
1
+
γ
N
j
=1
M
i
=1
g
ji
ˆ
m
i
−
d
j
2
−
ξ
2
ǫ
2
,
(6)where
µ
and
γ
are Lagrange multipliers. We have replaced the data inequality con-straint in Eq. (5) with an equality constraint. If the initial guess of the solution doesnot satisfy the data constraint, the estimate will be modiﬁed until the data constraintis satisﬁed, which will ﬁrst occur when
||
d
−
G
ˆ
m
||
2
=
ξ
2
ǫ
2
.This objective function,
φ
, is minimized relative to
q
(
m
) when the following equalityholds (Johnson and Shore, 1984):0 = ln
q
(
m
)
p
(
m
)
+ 1 +
µ
+
N
j
=1
λ
j
M
i
=1
g
ji
m
i
−
d
j
,
(7)where
λ
j
= 2
γ
M i
=1
g
ji
m
i
−
d
j
are Lagrange multipliers on the individual measureddata points. In terms of the Lagrange multipliers,
λ
j
, the data constraint in Eq. (5) canbe rewritten as
||
λ
||
2
= 4
γ
2
ξ
2
ǫ
2
, showing that
γ
=
||
λ
||
/
(2
ξǫ
). With this deﬁnition of
γ
and the deﬁnition of
λ
j
above, the data constraint holds when
λ
satisﬁes the nonlinear
5

Search

Similar documents

Related Search

Modernist Idea of a Single Style of the EpochDesign and Implementation of the Closed Loop FATE AND FREEWILL: A CRTICAL OVERVIEW OF THEA Re-evaluation of the Keyboard Sonatas of DoRoman copy of a Greek statue of the fifth cenLabor as a Performformative Language of the Simplementation of the govt schemes in the triA Portrait Of The Artist As A Young ManA Vindication Of The Rights Of WomanThe Black Arrow: A Tale Of The Two Roses

We Need Your Support

Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks