Education

Performance of the maximum likelihood estimators for the parameters of multivariate generalized Gaussian distributions

Description
Performance of the maximum likelihood estimators for the parameters of multivariate generalized Gaussian distributions
Categories
Published
of 4
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
  PERFORMANCES OF THE MAXIMUM LIKELIHOOD ESTIMATORS OF THEMULTIVARIATE GENERALIZED GAUSSIAN DISTRIBUTION  Lionel Bombrun 1  , Fr ´ ed ´ eric Pascal 2  ,Jean-Yves Tourneret  3 and Yannick Berthoumieu 1 1 : Universit´e de Bordeaux, UB1, IPB, ENSEIRB-Matmeca, Laboratoire IMS, UMR 5218Groupe Signal et Image, 351 cours de la lib´eration, 33402 Talence { lionel.bombrun, yannick.berthoumieu  } @ims-bordeaux.fr  2 : SONDRA, Sup´elec, Plateau du Moulon, 3 rue Joliot-Curie, 91190 Gif-sur-Yvette  frederic.pascal@supelec.fr  2 : Universit´e de Toulouse, IRIT/INP-ENSEEIHT, 2 rue Charles Camichel, 31071 Toulouse  jean-yves.tourneret@enseeiht.fr  ABSTRACT This paper address the problem of estimating the parameter of a Multivariate Generalized Gaussian Distribution (MGGD).After a brief introduction of the MGG distribution, the max-imum likelihood estimators of the MGGD parameters aregiven. For  β   ∈ ]0 , 1] , which corresponds to most of thereal-life problems, we prove theoretically that the maximumlikelihood estimator of the normalized covariance matrix ex-ists and is unique up to a scalar factor. Some experiments areconducted to evaluate the convergence speed of the proposedestimation algorithm. With simulations results, we show thatthe maximum likelihood estimator of the normalized covari-ance matrix is unbiased and consistent. Concerning the shapeparameter  β  , we show experimentally that the variance of its maximum likelihood estimator reaches the Cram´er-Raolower bound.  Index Terms —  Multivariate Generalized Gaussian Dis-tribution, performance estimators. 1. INTRODUCTION In the framework of natural textured image processing,stochastic models have been widely proposed in the liter-ature to model the texture. In content-based image retrieval,the Univariate Generalized Gaussian (UGG) distribution hasbeen introduced in [1] to represent the wavelet coefficients.Its more peaky and heavy-tailed behaviour compared to theunivariate Gaussian distribution are enough attributes to suit-ably represent the texture.Many authors have hence proposed to used some multi-variate distributions which generalizes this UGG distributionin order to capture inter-band dependencies at a wide-sense(multiscale, multichannel, spatial dependencies). A copula-based model with UGG distributed marginal have notablybeen proposed in [2] for the modeling the multichannel de-pendencies of wavelet coefficients.Another possible extension of the UGG distribution tothe multivariate case has been proposed in [3] (referred asanisotropic multivariate generalized Gaussian distribution).But, this model does not belong to the class of ellipticallycontoured distributions.Another extension of the UGG distribution is the Mul-tivariate Generalized Gaussian Distribution (MGGD) intro-duced in [4], also known as the multivariate power exponen-tial distribution [5]. This model belongs to the family of el-liptically contoured distributions [6] [7], and presents a lep-tokurtic behaviour which is highly appreciated for the model-ing of texture. The MGGD is completely characterized byits covariance matrix  Σ  and its shape parameter  β  . Manyworks have recently been proposed in the literature to adaptthis MGGD in image processing applications such as multi-spectral image indexing [8], image denoising [9] and textureimageretrieval[10][11]. MGGDparametersareestimatedbyminimizinga χ 2 distancein[8], andbyminimizinga L 2 normin [9]. A moment-based and a maximum likelihood (ML) ap-proach have been proposed in [10] [11]. But, in those works,no study of the estimators performances has been carried out.The aim of this paper is to address the problem of MGGDparameters estimation and to study the performances of theestimators, both for the covariance matrix  Σ  and for the shapeparameter  β  .The remainder of this paper is as follows. We introduce insection 2 the MGG distribution and present its ML estimators.In section 3, we prove that the ML estimator of the covariancematrix exists and is unique up to a scalar factor for  β   ∈ ]0 , 1] .In section 4, some experiments are conducted to evaluate theconvergence speed of the algorithm, to evaluate the bias andconsistency of the ML covariance matrix estimator, and tostudy the variance of   ˆ β  . Finally, section 5 concludes the paperand suggests an outlook on future works.  2. THE MULTIVARIATE GENERALIZED GAUSSIANDISTRIBUTION (MGGD)2.1. Definition and stochastic representation The probability density function of the Multivariate General-ized Gaussian Distribution (MGGD) is defined by [4]:  p x ( x | [ M  ] ,m,β  ) = Γ   p 2  π p 2 Γ   p 2 β  2 p 2 β β  | [ M  ] | 12 1 m p 2 × exp  −  12 m β  x T  [ M  ] − 1 x  β  = 1 | [ M  ] | 12 h  x T  [ M  ] − 1 x  ,  (1)where  h ( · )  is called density generator. In the following, thecovariance matrix  [ M  ]  is normalized such that  tr ([ M  ]) =  p where  p  is the dimension of vector  x .The case  β   = 0 . 5  corresponds to the multivariate Laplacedistribution, while  β   = 1  corresponds to multivariate Gaus-sian distributions. When  β   tends toward infinity, the MGGdistribution reduced to the multivariate uniform distribution.Let x be a random vector distributed according to a MGGdistribution of covariance matrix  Σ =  m [ M  ]  and shape pa-rameter  β  . G´omez  et al.  have shown in [5] that  x  admits thefollowing stochastic representation: x  d =  τ   Σ 12 u ,  (2)where  d =  means equality in distribution.  u  is a uniformly dis-tributed random vector on the unit sphere R  p , and τ   is a scalarpositive random variable such that: τ  2 β ∼ Gamma   p 2 β , 2  ,  (3)Gamma being the univariate Gamma distribution. 2.2. MGGD parameters estimators Let  ( x 1 ,..., x N  )  be  N   realizations of an independent andidentically distributed random vector x following a MGG dis-tribution. For any elliptical distribution, the Maximum Like-lihood (ML) estimator of the covariance matrix  [ M  ]  is given,in the real case, by [12]: [ ˆ M  ] = 2 N  N   i =1 − g ( x T i  [ ˆ M  ] − 1 x i ) h ( x T i  [ ˆ M  ] − 1 x i ) x i x T i  ,  (4)where g ( v ) =  ∂h ( v ) ∂v  . It yields that for the MGG distribution: [ ˆ M  ] =ˆ β N   ˆ m ˆ βN   i =1 x i x T i  x T i  [ ˆ M  ] − 1 x i  1 − ˆ β with  tr ([ ˆ M  ]) =  p. (5)The first term outside the sum is a scalar, hence the ML esti-mator of the normalized covariance matrix reduces to: [ ˆ M  ] = N   i =1 x i x T i  x T i  [ ˆ M  ] − 1 x i  1 − ˆ β with  tr ([ ˆ M  ]) =  p.  (6)The ML estimators of the scale  m  and shape  β   parameters of the MGG distributions are solutions of: f  (ˆ β  ) =  pN  2 N   i =1 u ˆ βiN   i =1 ln u i u ˆ βi  −  pN  2ˆ β   Ψ   p 2ˆ β   + ln2  − N   −  pN  2ˆ β  ln   ˆ β  pN  N   i =1 u ˆ βi   = 0 ,  (7)and ˆ m  =   ˆ β  pN  N   i =1 u ˆ βi  1ˆ β ,  (8)where  u i  =  x T i  [ ˆ M  ] − 1 x i , and  Ψ( · )  is the digamma function.Equations (6) and (7) show that  [ M  ]  and  β   can be estimatedindependently from the scale parameter  m . To solve (7), aNewton-Raphson procedure is considered. It yields: ˆ β  n +1  = ˆ β  n −  f  (ˆ β  n ) f  ′ (ˆ β  n ) ,  (9)where  ˆ β  n  is the estimator of   β   at step  n , and the function f  ( β  )  is defined by (7).In practice, the following algorithm isimplemented to estimate the parameters of the MGG distribu-tion. 1:  Initialisation of   β   and  [ M  ] . 2:  for  k  = 1 :  N iter max  do 3:  Estimation of   [ M  ]  by (6). 4:  Estimation of   β   by a Newton-Raphson procedure bycombining (7) and (9). 5:  end for 6:  Estimation of   m  by (8). 3. PROPERTIES OF THE M-ESTIMATORTheorem1.  Let  ( x 1 ,..., x N  ) bea N  -sampleof   p -dimensionalreal independent vectors, with zero mean and  x i  ∼ E   p ( 0 , Λ)  , i  = 1 ,...,N   where  E   p  denotes the elliptical distribution.The M-estimator of   Λ  is defined as the solution of the follow-ing equation: V N   = 1 N  N   n =1 u  x T n V − 1 N   x n  x n x T n  (10) where u ( · )  is a function satisfying a set of general assumptionstated in [13] and recalled here below:  (i)  u  is non-negative, non increasing, and continuous on [0 , ∞ ) .(ii) Let   ψ ( s ) =  s u ( s )  and   K   =  sup s ≥ 0 ψ ( s ) .  m < K < ∞  ,  ψ  is non decreasing and strictly increasing in theinterval where  ψ < K  .(iii) Let  P  N   ( · ) denotestheempiricaldistributionof  x 1 ,..., x N  .There exists  a >  0  such that for every hyperplane H,dim ( H  ) ≤ m − 1 P  N   ( H  ) ≤ 1 −  mK   − a.  (11)  Let us consider the following equation, which is roughlyspeaking the limit of   (10) . V  =  E  u ( x T  V − 1 x ) xx T   .  (12) Then, under the above condition, Maronna has shown in [13]that: •  Equation  (12)  , respectively  (10)  , admits a unique solu-tion  V  , respectively  V N   , such that   V  is equal to  Λ  upto a scale factor and is therefore equal to the covari-ance matrix up to another scale factor, •  a simple iterative procedure provides  V N  . Proposition 1.  Let   ( x 1 ,..., x N  )  be  N   independent realiza-tions of a  p -dimensional random vector  x distributed accord-ing to a Multivariate Generalized Gaussian Distribution of  parameter   m  ,  β   and   [ M  ] . Hence, for   β   ∈ ]0 , 1]  , the maxi-mum likelihood estimator of the covariance matrix  [ M  ]  givenby  (6)  exists and is unique up to a scalar factor.Proof.  The proof is directly obtained byconsidering thefunc-tion  u ( s ) =  s β − 1 with  β   ∈ ]0 , 1]  in Theorem 1.It is important to note that the particular case where  β   be-longs to the interval  ]0 , 1]  corresponds to most of the real-lifeproblems, such as the modeling of spatial or color dependen-cies of wavelet coefficients of texture images [11]. 4. SIMULATIONS In this section, some simulations results are presented to eval-uate the performances of the ML estimators of the MGG dis-tributions. Let  ( x 1 ,..., x N  )  be  N   realizations of an indepen-dent and identically distributed random vector  x  following aMGG distribution. MGGD realizations are obtained accord-ing to its stochastic representation given by (2). In the follow-ing, the covariance matrix  [ M  ]  is chosen such that [ M  ]( i,j ) =  ρ | i − j | .  (13) 4.1. Convergence of the ML covariance matrix estimator Fig. 1 shows the convergence results of the ML covariancematrix estimator  [ ˆ M  ]  when the length of each vector  x i  is  p  = 3 . In this simulation, we have considered  β   = 0 . 2  and ρ  = 0 . 8 . Convergence results are analyzed by evaluating thecriterion  C   defined as: C  ( k ) =  || [ ˆ M  k +1 ] − [ ˆ M  k ] |||| [ ˆ M  k ] || ,  (14)where ||·|| is the Frobenius norm and  [ ˆ M  k ]  is the ML estima-tor of   [ M  ]  at step  k .Fig. 1.(a) shows the convergence of   [ ˆ M  ]  for various ini-tialization points  [ M  ] 0  (moment based estimator [10], iden-tity matrix and the true covariance matrix  [ M  ] ). After 20 it-erations, all curves merge. Hence, the convergence speed isindependent of the algorithm initialization. Fig. 1.(b) showsthe evolution of criterion  C  ( k )  for various number  N   of sec-ondary data. It can be observed that the convergence speedincreases with  N  .(a) (b) Fig. 1 . Convergence of the covariance matrix  [ ˆ M  ]  for  p  = 3 , β   = 0 . 2  and  ρ  = 0 . 8 . (a)  C  ( k )  as a function of the number k  of iterations for different starting point for  N   = 200 . (b) C  ( k )  as a function of   k  for various values of   N  . 4.2. Bias and consistency analysis Fig. 2.(a) shows the estimates bias for different values of   β  ( 0 . 2 ,  0 . 5  and  0 . 8 ). For that purpose, a plot of  || [ ˆ M  ] − [ M  ] || as a function of the number of samples  N   is proposed [14],where operator  [ ˆ M  ]  is defined as the empirical mean of theestimated covariance matrices [ ˆ M  ] = 1 I  I   i =1 [ ˆ M  ]( i ) .  (15)In this experiment,  1000  Monte Carlo runs have been consid-ered. As observed in Fig. 2.(a), it can be noticed that the biasis close to  0  for each value of  N  . This bias is also independentfrom  β  .Fig. 2.(b) presents results of estimates consistency. Here,a plot of  || [ ˆ M  ] − [ M  ] || as a function of the number of samples  N   is shown for different values of   β   ( 0 . 2 ,  0 . 5  and  0 . 8 ). It canbe noticed that this criterion tends to  0  when  N   tends toward + ∞ independently of   β  .(a) (b) Fig. 2 . (a) estimates bias for different values of   β  , (b) esti-mates consistency for different values of   β  . 4.3.  β   parameter The Fisher information matrix has been recently derived forthe MGG distribution in [10]. It has been shown that it de-pends only on the number  N   of secondary data and on theshape parameter  β  . The Cram´er-Rao lower bound (in black)can hence be computed to evaluate the estimation perfor-mances. A comparison of the performance estimations of  β   between the moment based method [10] (dashed red line)and the ML method (solid blue line) is achieved. In thisexperiment,  1000  Monte Carlo runs have been considered.Fig. 3.(a) shows the evolution of the variance of   ˆ β   as afunction of the number of samples  N   for  β   = 0 . 2 ,  ρ  = 0 . 8 and  p  = 3 , while Fig. 3.(b) shows the variance of   β   as afunction of   β   for  N   = 10 000 ,  ρ  = 0 . 8  and  p  = 3 . Asexpected, the ML method yields a lower estimation variancecompared to the moment-based approach. The Cram´er-Raolower bound is reached with the ML method and depends on N   and  β  .(a) (b) Fig. 3 . Performance estimation of   β  . (a) Variance of   ˆ β   as afunction of the number of samples  N   for  β   = 0 . 2 ,  ρ  = 0 . 8 and  p  = 3 , (b) Variance of   ˆ β   as a function of   β   for  N   =10 000 ,  ρ  = 0 . 8  and  p  = 3 . 5. CONCLUSION In this paper, the problem of Multivariate Generalized Gaus-sian Distribution (MGGD) parameter estimation has been ad-dressed. For  β   ∈ ]0 , 1] , which corresponds to most of the real-life problems, we have proved theoretically that the maximumlikelihood (ML) estimator of the normalized covariance ma-trix exists and is unique up to a scalar factor. Simulationsresults have shown that this estimator is unbiased and con-sistent. For the shape parameter  β  , the variance of its MLestimator reaches the Cram´er-Rao lower bound.Further works will concern the use of the MGG distribu-tion in remote sensing applications such as change detection,classification, ... 6. REFERENCES [1] M. N. Do and M Vetterli, “Wavelet-Based Texture Retrieval UsingGeneralized Gaussian Density and Kullback-Leibler Distance,”  IEEE Transactions on Image Processing , vol. 11, pp. 146–158, 2002.[2] S. Sarra and A. Benazza-Benyahia, “Indexing of Multichannel Imagesin the Wavelet Transform Domain,” in  International Conference on Information nad Communication Technologies: From Theory to Appli-cations , 2008, pp. 1–6.[3] L. Boubchir and J. Fadili, “Multivariate Statistical Modeling of Imageswith the curvelet transform,” in  Proceedings of the Eighth InternationalSymposium on Signal Processing and its Application , 2005, pp. 747–750.[4] S. Kotz,  Statistical Distributions in Scientific Work, I. , Dordrecht:Reidel, 1968, ch. Multivariate distributions at a cross road, pp. 247-270.[5] E. G´omez, M.A. Gomez-Villegas, and J.M. Mar´ın, “A Multivari-ate Generalization of the Power Exponential Family of Distributions,” Communications in Statistics - Theory and Methods , vol. 27, no. 3, pp.589–600, 1998.[6] K.-C.Chu, “EstimationandDecisionforLinearSystemswithEllipticalRandomProcesses,”  IEEETransactionsonAutomaticControl , October1973.[7] S. Cambanis, S. Huang, and G. Simons, “On the Theory of EllipticallyContoured Distributions,”  Journal of Multivariate Analysis , vol. 11, no.3, pp. 368–385, 1981.[8] N. Khelil-Cherif and A. Benazza-Benyahia, “Wavelet-based Multivari-ate Approach for Multispectral Image Indexing,” in  SPIE Conferenceon Wavelet Applications in Industrial Processing , 2004.[9] D. Cho and T. Bui, “Multivariate Statistical Modeling for Image De-noising using Wavelet Transforms,”  Signal Processing: Image Com-munications , vol. 20, no. 1, pp. 77–89, 2005.[10] G. Verdoolaege and P. Scheunders, “On the Geometry of MultivariateGeneralized Gaussian Models,”  Journal of Mathematical Imaging and Vision , 2011.[11] G. Verdoolaege and P. Scheunders, “Geodesics on the Manifold of Multivariate Generalized Gaussian Distributions with an Applicationto Multicomponent Texture Discrimination,”  International Journal of Computer Vision , 2011.[12] F. Gini and M. V. Greco, “Covariance Matrix Estimation for CFARDetection in Correlated Heavy Tailed Clutter,”  Signal Processing , vol.82, no. 12, pp. 1847–1859, 2002.[13] R.A. Maronna, “Robust M-estimators of Multivariate Location andScatter,”  Annals of Statistics , vol. 4, no. 1, pp. 51–67, January 1976.[14] F. Pascal, P. Forster, J. P. Ovarlez, and P. Larzabal, “Performance Anal-ysis of Covariance Matrix Estimates in Impulsive Noise,”  IEEE Trans-actions on Signal Processing , vol. 56, no. 6, pp. 2206–2216, 2008.
Search
Similar documents
View more...
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks
SAVE OUR EARTH

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!

x