Documents

VARIOUS RELATIONS ON NEW INFORMATION DIVERGENCE MEASURES

Description
Divergence measures are useful for comparing two probability distributions. Depending on the nature of the problem, different divergence measures are suitable. So it is always desirable to develop a new divergence measure. Recently, Jain and Chhabra [6] introduced new series ( ( , ) m x P Q , ( , ) m z P Q and ( , ) m w P Q for mÎN ) of information divergence measures, defined the properties and characterized, compared with standard divergences and derived the new series ( ( ) * , m x P Q formÎN ) of metric spaces. In this work, various important and interesting relations among divergences of these new series and other well known divergence measures are obtained. Some intra relations among these new divergences are evaluated as well and bounds of new divergence measure ( ( ) 1 x P,Q ) are obtained by using Csiszar’s information inequalities. Numerical illustrations (Verification) regarding bounds are done as well.
Categories
Published
of 18
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
  International Journal on Information Theory (IJIT),Vol.3, No.4, October 2014 DOI : 10.5121/ijit.2014.3401 1 V ARIOUS  R ELATIONS  O N  N EW I NFORMATION  D IVERGENCE  M EASURES K.C. Jain 1 , Praphull Chhabra 2 Department of Mathematics, Malaviya National Institute of Technology, Jaipur- 302017 (Rajasthan), INDIA.  Abstract  Divergence measures are useful for comparing two probability distributions. Depending on the nature of the problem, different divergence measures are suitable. So it is always desirable to develop a new divergence measure.  Recently,  Jain and Chhabra    [6]   introduced new series (  ( ) , m  P Q ξ   ,  ( ) , m  P Q ζ   and   ( ) , m  P Q ω   for  m N  ∈  ) of information divergence measures, defined the properties and characterized, compared with standard divergences and derived the new series (  ( ) * , m  P Q ξ   for  m N  ∈  ) of metric spaces.  In this work, various important and interesting relations among divergences of these new series and other well known divergence measures are obtained. Some intra relations among these new divergences are evaluated as well and bounds of new divergence measure (  ( ) 1 , P Q ξ   ) are obtained by using Csiszar’s information inequalities. Numerical illustrations (Verification) regarding bounds are done as well.  Index terms:   Convex function, Divergence measure, Algebraic inequalities, Csiszar’s inequalities, Bounds, Mean divergence measures, Difference of mean divergences, Difference of divergences.  Mathematics subject classification :   Primary 94A17, Secondary 26D15. 1.   Introduction   Let ( ) 1231 ,,...,:0,1,2 nn n i ii P p p p p p p n =   Γ = = > = ≥    ∑  be the set of all complete finite discrete probability distributions. If we take i  p  ≥ 0 for some 1,2,3,..., i n = , then we have to suppose that  ( ) 000000  f f     = =    . Csiszar’s f- divergence [2]  is a generalized information divergence measure, which is given by (1.1), i.e.,  International Journal on Information Theory (IJIT),Vol.3, No.4, October 2014 2 ( ) 1 , ni f ii i  pC P Q q f q =   =     ∑ . (1.1) Where f: (0, ∞ ) →  R (set of real no.) is real, continuous and convex function and ( ) ( ) 123123 ,,...,,,,..., n n P p p p p Q q q q q = =  ∈   Γ n , where i  p  and i q  are probability mass functions. Many known divergence measures can be obtained from this generalized measure by suitably defining the convex function f. Some of those are as follows. ( )  ( ) ( ) ( ) 22221/221 ,,1,2,3... mni im mmii i i  p qP Q m p q q ξ  −= −= = ∑ (Jain and Chhabra [6]) (1.2) ( )  ( ) ( ) ( ) ( ) 2224223421/2221 2,,1,2,3... mni i i i i i i im mmii i i  p q p p q p q qP Q m p q q ζ  ++= − − + += = ∑ (Jain and Chhabra [6]) (1.3) ( )  ( ) ( ) ( ) ( ) 222222321/221 ,exp,1,2,3... mni i i im mmi i ii i i  p q p qP Q m pq pq q ω  −=   − −   = =     ∑ (Jain and Chhabra [6]) (1.4)   ( ) ( )( ) ( ) 2*21/21 ,,1,2,3,... mni im mii i  p q E P Q m p q  −= −= = ∑ (Jain and Srivastava [11])  (1.5) ( ) ( )( ) ( ) ( ) 22*21/21 ,exp,1,2,3,... mni i i im mi i ii i  p q p q J P Q m p q p q  −=   − −   = =     ∑ (Jain and srivastava [11]) (1.6) ( ) ( )( )( )( ) 22*2121 ,exp,1,2,3,... mni i i im mii i i i  p q p q N P Q m p q p q −=   − −   = =   + +    ∑ (Jain and Saraswat [10]) (1.7) ( )  ( ) ( ) 2223/21 ,2 ni i M ii i  p qP Q p q ψ  = −= ∑  (Kumar and Johnson [16])  (1.8) ( ) ( ) 21 ,log2 ni i i ii i i i i  p q p q L P Q p q p q =   − +=     +   ∑  (Kumar and Hunter [15]) (1.9) Puri and Vineze divergence (Kafka,Osterreicher and Vincze   [13]) ( ) ( )( ) 2211 ,,1,2,3... mni im mii i  p qP Q m p q  −= −∆ = =+ ∑   (1.10) Where  ( ) ( ) 21 , ni ii i i  p qP Q p q = −∆ =+ ∑ = Triangular discrimination, is a particular case of (1.10) at m=1. Relative Arithmetic- Geometric divergence (Taneja [19])   International Journal on Information Theory (IJIT),Vol.3, No.4, October 2014 3 ( ) 1 ,log22 ni i i ii i  p q p qG P Q p =   + +   =        ∑ . (1.11) Arithmetic- Geometric mean divergence (Taneja [19]) ( ) ( ) ( ) 1 1,,,log222 ni i i iii i  p q p qT P Q G P Q G Q P p q =   + += + =         ∑ . (1.12) Where ( ) , G P Q is given by (1.11). d- Divergence (Taneja [20]) ( ) 1 ,122 ni i i ii  p q p qd P Q =   + += −      ∑ . (1.13) Symmetric Chi- square divergence (Dragomir, Sunde and Buse [4]) ( ) ( ) ( ) ( ) ( ) 2221 ,,, ni i i ii i i  p q p qP Q P Q Q P p q  χ χ  = − +Ψ = + = ∑ . (1.14) Where  ( ) 2 , P Q  χ  is given by (1.18). Relative J- divergence (Dragomir, Gluscevic and Pearce [3]) ( ) ( ) ( ) ( ) 1 ,2,,log2 ni i R i ii i  p q J P Q F Q P G Q P p qq =   += + = −       ∑ . (1.15) Where ( ) ( ) ,, F P Q and G P Q  are given by (1.16) and (1.11) respectively. Relative Jensen- Shannon divergence (Sibson [18]) ( ) 1 2,log niii i i  pF P Q p p q =   =    +   ∑ . (1.16) Hellinger discrimination   (Hellinger [5])   ( ) ( )  ( ) 2*1 1,1,2 ni ii h P Q G P Q p q = = − = − ∑ . (1.17) Where  ( ) * , G P Q is given by (1.29). Chi- square divergence (Pearson [17]) ( ) ( ) 221 , ni ii i  p qP Qq  χ  = −=  ∑ . (1.18) Relative information (Kullback and Leibler [14]) ( ) 1 ,log niii i  pK P Q pq =   =     ∑ . (1.19)  J- Divergence (Jeffreys, Kullback and Leibler [12, 14]) ( ) ( ) ( ) ( ) ( ) ( ) 1 ,,,,,log ni R R i ii i  p J P Q K P Q K Q P J P Q J Q P p qq =   = + = + = −     ∑ . (1.20) Where  ( ) ( ) ,,  R  J P Q and K P Q are given by (1.15) and (1.19) respectively. Jensen- Shannon divergence (Burbea, Rao and Sibson [1, 18])    International Journal on Information Theory (IJIT),Vol.3, No.4, October 2014 4 ( ) ( ) ( ) 11 2211,,,loglog22 n ni ii ii ii i i i  p q I P Q F P Q F Q P p q p q p q = =      = + = +         + +      ∑ ∑ . (1.21) Where  ( ) , F P Q is given by (1.16). Some mean divergence measures and difference of particular mean divergences can be seen in literature (Taneja [21]) , these are as follows. Root mean square divergence =  ( ) 221 ,2 ni ii  p qS P Q = +=  ∑ . (1.22) Harmonic mean divergence =  ( ) 1 2, ni ii i i  p q H P Q p q = =+ ∑ . (1.23) Arithmetic mean divergence =  ( ) 1 ,12 ni ii  p q A P Q = += = ∑ . (1.24) Square root mean divergence =  ( ) 211 ,2 ni ii  p q N P Q =   +=      ∑ . (1.25) ( ) 21 ,22 ni i i ii  p q p q N P Q =   + +=      ∑ . (1.26) Heronian mean divergence =  ( ) 31 ,3 ni i i ii  p p q q N P Q = + +=  ∑ . (1.27) Logarithmic mean divergence =  ( ) *1 ,,loglog ni ii ii i i  p q L P Q p q i p q = −= ≠ ∀− ∑ . (1.28) Geometric mean divergence =  ( ) *1 , ni ii G P Q p q = =  ∑ . (1.29) Square root- arithmetic mean divergence ( ) ( ) ( ) 221 ,,,12 ni iSAi  p q M P Q S P Q A P Q =   += − = −     ∑ . (1.30) Where  ( ) ( ) ,, S P Q and A P Q are given by (1.22) and (1.24) respectively. Square root- geometric mean divergence ( ) ( ) ( ) * 22*1 ,,,2 ni ii iSGi  p q M P Q S P Q G P Q p q =   += − = −     ∑ . (1.31) Where  ( ) ( ) * ,, S P Q and G P Q are given by (1.22) and (1.29) respectively. Square root- harmonic mean divergence ( ) ( ) ( ) 221 2,,,2 ni i i iSH i i i  p q p q M P Q S P Q H P Q p q =   += − = −    +   ∑ . (1.32)
Search
Tags
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks