Graphics & Design

A MATLAB Implementation of the Minimum Relative Entropy Method for Linear Inverse Problems* 1

A MATLAB Implementation of the Minimum Relative Entropy Method for Linear Inverse Problems* 1
of 22
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
  A MATLAB Implementation of theMinimum Relative Entropy Methodfor Linear Inverse Problems Roseanna M. Neupauer 1 and Brian Borchers 2 1 Department of Earth and Environmental Science 2 Department of MathematicsNew Mexico Institute of Mining and Technology, Socorro, NM 87801, March 31, 2000 Abstract The minimum relative entropy (MRE) method can be used to solve linear inverseproblems of the form Gm = d , where m is a vector of unknown model parameters and d is a vector of measured data. The MRE method treats the elements of  m as randomvariables, and obtains a multivariate probability density function for m . The proba-bility density function is constrained by prior information about the upper and lowerbounds of  m , a prior expected value of  m , and the measured data. The solution of the 1  inverse problem is the expected value of  m , based on the derived probability densityfunction. We present a MATLAB implementation of the MRE method. Several nu-merical issues arise in the implementation of the MRE method and are discussed here.We present the source history reconstruction problem from groundwater hydrology asan example of the MRE implementation. Keywords Maximum Entropy, Minimum Relative Entropy, Inverse Problems 1 Introduction In this paper, we present a MATLAB implementation of the minimum relative entropy(MRE) method to solve linear inverse problems of the form Gm = d , (1)where G is a matrix of size N  by M  , m is a vector of length M  containing unknownmodel parameters, and d is a data vector of length N  . The vector d consists of measured data, and typically includes measurement error. The matrix G is typicallyvery badly conditioned, making a conventional least squares approach impractical.In the MRE method, the unknown model parameters, m , are treated as randomvariables, and the solution to the inverse problem is obtained from the multivariateprobability density function (pdf) of  m . The pdf is selected in a two-step process.First, we generate a prior distribution whose entropy is minimized subject to constraints 2  imposed by the lower and upper bounds of  m ( l and u , respectively) and by an estimate, s , of the expected value of  m . This prior distribution ensures that the solution of theinverse problem is within the specified bounds on m , but does not guarantee thatEq. (1) is satisfied.In the second step, we select a posterior distribution for m , whose entropy is min-imized relative to the prior distribution and subject to the constraint imposed by themeasured data d . The mean of the posterior pdf,ˆ m , can be used as a “solution” tothe inverse problem. The 5th and 95th percentiles of the posterior distribution definea 90% probability interval, similar to the 90% confidence interval in classical statisti-cal methods. We can investigate the properties of the solution by using the posteriordistribution to randomly generate many realizations of  m .The goal of this paper is to present the MRE method in the context of a generaldiscrete linear inverse problem, to discuss some important issues in the implementationof the method, and to describe our MATLAB implementation of the MRE method.We also present an example taken from groundwater hydrology. 2 The MRE Method In this section, we describe the generation of the prior distribution, p ( m ), and theposterior distribution, q ( m ), of  m . The elements of  m are assumed to be independent,so p ( m ) =  M i =1 p ( m i ) and q ( m ) =  M i =1 q ( m i ), where M  is the number of modelparameters.In this discussion, we assume that the lower bound is zero. As we show later, 3  non-zero lower bounds can easily be incorporated with a change of variables. Fora continuous random variable with finite upper bounds, zero lower bounds, and afinite expected value within the bounds, it can be shown that the maximum entropydistribution is a truncated exponential distribution (Kapur and Kesavan, 1992). Usingthis fact we can derive the prior distribution  p ( m i ) = β  i exp( − β  i m i )1 − exp( − β  i U  i )for β  i  = 0 (2)  p ( m i ) =1 U  i for β  i = 0 ;  p ( m ) = M   i =1  p ( m i ) , where U  i is the upper bound of parameter m i , and β  i is a Lagrange multiplier whosevalue is determined by satisfying the expected value constraint,   m i  p ( m i ) dm i = s i ,where s i is the prior expected value of  m i for i = 1 , 2 ,...,M  . By integrating Eq. (2),we obtain the expected value equation that is used to evaluate β  i − ( β  i U  i + 1)exp( − β  i U  i ) + 1 β  i [1 − exp( − β  i U  i )]= s i . (3)Details of this derivation can be found in Neupauer (1999).To obtain the posterior distribution, q ( m ), we minimize its entropy relative to theprior distribution, p ( m ). The entropy to be minimized is H  ( q,p ) =   m q ( m )ln  q ( m )  p ( m )  d m , (4)where p ( m ) is given in Eq. (2). This minimization is subject to two constraints—thenormalization requirement   m q ( m ) d m = 1, and the requirement that the posterior 4  mean solution fit the data within a specified tolerance || d − G ˆ m || 2 ≤ ξ 2 ǫ 2 , (5)where ||·|| denotes the L 2 norm,ˆ m is the mean of the posterior distribution, ǫ is themeasurement error, and ξ is a parameter that depends on the assumed error model.This constrained optimization problem is solved by the method of Lagrange multipliersby minimizing φ = H  ( q,p ) + µ   m q ( m ) d m − 1  + γ   N    j =1  M   i =1 g  ji ˆ m i − d  j  2 − ξ 2 ǫ 2  , (6)where µ and γ  are Lagrange multipliers. We have replaced the data inequality con-straint in Eq. (5) with an equality constraint. If the initial guess of the solution doesnot satisfy the data constraint, the estimate will be modified until the data constraintis satisfied, which will first occur when || d − G ˆ m || 2 = ξ 2 ǫ 2 .This objective function, φ , is minimized relative to q ( m ) when the following equalityholds (Johnson and Shore, 1984):0 = ln  q ( m )  p ( m )  + 1 + µ + N    j =1 λ  j  M   i =1 g  ji m i − d  j  , (7)where λ  j = 2 γ   M i =1 g  ji m i − d  j  are Lagrange multipliers on the individual measureddata points. In terms of the Lagrange multipliers, λ  j , the data constraint in Eq. (5) canbe rewritten as || λ || 2 = 4 γ  2 ξ 2 ǫ 2 , showing that γ  = || λ || / (2 ξǫ ). With this definition of  γ  and the definition of  λ  j above, the data constraint holds when λ satisfies the nonlinear 5
Similar documents
View more...
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks