Description

Divergence measures are useful for comparing two probability distributions. Depending on the nature of
the problem, different divergence measures are suitable. So it is always desirable to develop a new
divergence measure.
Recently, Jain and Chhabra [6] introduced new series ( ( , ) m x P Q , ( , ) m z P Q and ( , ) m w P Q for
mÎN ) of information divergence measures, defined the properties and characterized, compared with
standard divergences and derived the new series ( ( ) * , m x P Q formÎN ) of metric spaces.
In this work, various important and interesting relations among divergences of these new series and other
well known divergence measures are obtained. Some intra relations among these new divergences are
evaluated as well and bounds of new divergence measure ( ( ) 1 x P,Q ) are obtained by using Csiszar’s
information inequalities. Numerical illustrations (Verification) regarding bounds are done as well.

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.

Related Documents

Share

Transcript

International Journal on Information Theory (IJIT),Vol.3, No.4, October 2014 DOI : 10.5121/ijit.2014.3401 1
V
ARIOUS
R
ELATIONS
O
N
N
EW
I
NFORMATION
D
IVERGENCE
M
EASURES
K.C. Jain
1
, Praphull Chhabra
2
Department of Mathematics, Malaviya National Institute of Technology, Jaipur- 302017 (Rajasthan), INDIA.
Abstract
Divergence measures are useful for comparing two probability distributions. Depending on the nature of the problem, different divergence measures are suitable. So it is always desirable to develop a new divergence measure. Recently,
Jain and Chhabra
[6]
introduced new series (
( )
,
m
P Q
ξ
,
( )
,
m
P Q
ζ
and
( )
,
m
P Q
ω
for
m N
∈
) of information divergence measures, defined the properties and characterized, compared with standard divergences and derived the new series (
( )
*
,
m
P Q
ξ
for
m N
∈
) of metric spaces. In this work, various important and interesting relations among divergences of these new series and other well known divergence measures are obtained. Some intra relations among these new divergences are evaluated as well and bounds of new divergence measure (
( )
1
,
P Q
ξ
) are obtained by using Csiszar’s information inequalities. Numerical illustrations (Verification) regarding bounds are done as well.
Index terms:
Convex function, Divergence measure, Algebraic inequalities, Csiszar’s inequalities, Bounds, Mean divergence measures, Difference of mean divergences, Difference of divergences.
Mathematics subject classification
:
Primary 94A17, Secondary 26D15.
1.
Introduction
Let
( )
1231
,,...,:0,1,2
nn n i ii
P p p p p p p n
=
Γ = = > = ≥
∑
be the set of all complete finite discrete probability distributions. If we take
i
p
≥
0 for some
1,2,3,...,
i n
=
, then we have to suppose that
( )
000000
f f
= =
.
Csiszar’s f- divergence [2]
is a generalized information divergence measure, which is given by (1.1), i.e.,
International Journal on Information Theory (IJIT),Vol.3, No.4, October 2014 2
( )
1
,
ni f ii i
pC P Q q f q
=
=
∑
. (1.1) Where f: (0,
∞
)
→
R (set of real no.) is real, continuous and convex function and
( ) ( )
123123
,,...,,,,...,
n n
P p p p p Q q q q q
= =
∈
Γ
n
, where
i
p
and
i
q
are probability mass functions. Many known divergence measures can be obtained from this generalized measure by suitably defining the convex function f. Some of those are as follows.
( )
( )
( )
( )
22221/221
,,1,2,3...
mni im mmii i i
p qP Q m p q q
ξ
−=
−= =
∑
(Jain and Chhabra [6])
(1.2)
( )
( ) ( )
( )
( )
2224223421/2221
2,,1,2,3...
mni i i i i i i im mmii i i
p q p p q p q qP Q m p q q
ζ
++=
− − + += =
∑
(Jain and Chhabra [6])
(1.3)
( )
( )
( )
( )
( )
222222321/221
,exp,1,2,3...
mni i i im mmi i ii i i
p q p qP Q m pq pq q
ω
−=
− −
= =
∑
(Jain and Chhabra [6])
(1.4)
( ) ( )( )
( )
2*21/21
,,1,2,3,...
mni im mii i
p q E P Q m p q
−=
−= =
∑
(Jain and Srivastava [11])
(1.5)
( ) ( )( )
( )
( )
22*21/21
,exp,1,2,3,...
mni i i im mi i ii i
p q p q J P Q m p q p q
−=
− −
= =
∑
(Jain and srivastava [11])
(1.6)
( ) ( )( )( )( )
22*2121
,exp,1,2,3,...
mni i i im mii i i i
p q p q N P Q m p q p q
−=
− −
= =
+ +
∑
(Jain and Saraswat [10])
(1.7)
( )
( )
( )
2223/21
,2
ni i M ii i
p qP Q p q
ψ
=
−=
∑
(Kumar and Johnson [16])
(1.8)
( ) ( )
21
,log2
ni i i ii i i i i
p q p q L P Q p q p q
=
− +=
+
∑
(Kumar and Hunter [15])
(1.9)
Puri and Vineze divergence (Kafka,Osterreicher and Vincze
[13])
( ) ( )( )
2211
,,1,2,3...
mni im mii i
p qP Q m p q
−=
−∆ = =+
∑
(1.10) Where
( ) ( )
21
,
ni ii i i
p qP Q p q
=
−∆ =+
∑
= Triangular discrimination, is a particular case of (1.10) at m=1.
Relative Arithmetic- Geometric divergence (Taneja [19])
International Journal on Information Theory (IJIT),Vol.3, No.4, October 2014 3
( )
1
,log22
ni i i ii i
p q p qG P Q p
=
+ +
=
∑
. (1.11)
Arithmetic- Geometric mean divergence (Taneja [19])
( ) ( ) ( )
1
1,,,log222
ni i i iii i
p q p qT P Q G P Q G Q P p q
=
+ += + =
∑
. (1.12) Where
( )
,
G P Q
is given by (1.11).
d- Divergence (Taneja [20])
( )
1
,122
ni i i ii
p q p qd P Q
=
+ += −
∑
. (1.13)
Symmetric Chi- square divergence (Dragomir, Sunde and Buse [4])
( ) ( ) ( ) ( ) ( )
2221
,,,
ni i i ii i i
p q p qP Q P Q Q P p q
χ χ
=
− +Ψ = + =
∑
.
(1.14) Where
( )
2
,
P Q
χ
is given by (1.18).
Relative J- divergence (Dragomir, Gluscevic and Pearce [3])
( ) ( ) ( ) ( )
1
,2,,log2
ni i R i ii i
p q J P Q F Q P G Q P p qq
=
+= + = −
∑
. (1.15) Where
( ) ( )
,,
F P Q and G P Q
are given by (1.16) and (1.11) respectively.
Relative Jensen- Shannon divergence (Sibson [18])
( )
1
2,log
niii i i
pF P Q p p q
=
=
+
∑
. (1.16)
Hellinger discrimination
(Hellinger [5])
( ) ( )
( )
2*1
1,1,2
ni ii
h P Q G P Q p q
=
= − = −
∑
. (1.17) Where
( )
*
,
G P Q
is given by (1.29).
Chi- square divergence (Pearson [17])
( ) ( )
221
,
ni ii i
p qP Qq
χ
=
−=
∑
. (1.18)
Relative information (Kullback and Leibler [14])
( )
1
,log
niii i
pK P Q pq
=
=
∑
. (1.19)
J- Divergence (Jeffreys, Kullback and Leibler [12, 14])
( ) ( ) ( ) ( ) ( ) ( )
1
,,,,,log
ni R R i ii i
p J P Q K P Q K Q P J P Q J Q P p qq
=
= + = + = −
∑
. (1.20) Where
( ) ( )
,,
R
J P Q and K P Q
are given by (1.15) and (1.19) respectively.
Jensen- Shannon divergence (Burbea, Rao and Sibson [1, 18])
International Journal on Information Theory (IJIT),Vol.3, No.4, October 2014 4
( ) ( ) ( )
11
2211,,,loglog22
n ni ii ii ii i i i
p q I P Q F P Q F Q P p q p q p q
= =
= + = +
+ +
∑ ∑
. (1.21) Where
( )
,
F P Q
is given by (1.16). Some mean divergence measures and difference of particular mean divergences can be seen in literature
(Taneja [21])
, these are as follows.
Root mean square divergence =
( )
221
,2
ni ii
p qS P Q
=
+=
∑
.
(1.22)
Harmonic mean divergence =
( )
1
2,
ni ii i i
p q H P Q p q
=
=+
∑
.
(1.23)
Arithmetic mean divergence =
( )
1
,12
ni ii
p q A P Q
=
+= =
∑
.
(1.24)
Square root mean divergence =
( )
211
,2
ni ii
p q N P Q
=
+=
∑
.
(1.25)
( )
21
,22
ni i i ii
p q p q N P Q
=
+ +=
∑
.
(1.26)
Heronian mean divergence =
( )
31
,3
ni i i ii
p p q q N P Q
=
+ +=
∑
.
(1.27)
Logarithmic mean divergence =
( )
*1
,,loglog
ni ii ii i i
p q L P Q p q i p q
=
−= ≠ ∀−
∑
.
(1.28)
Geometric mean divergence =
( )
*1
,
ni ii
G P Q p q
=
=
∑
.
(1.29)
Square root- arithmetic mean divergence
( ) ( ) ( )
221
,,,12
ni iSAi
p q M P Q S P Q A P Q
=
+= − = −
∑
.
(1.30) Where
( ) ( )
,,
S P Q and A P Q
are given by (1.22) and (1.24) respectively.
Square root- geometric mean divergence
( ) ( ) ( )
*
22*1
,,,2
ni ii iSGi
p q M P Q S P Q G P Q p q
=
+= − = −
∑
.
(1.31) Where
( ) ( )
*
,,
S P Q and G P Q
are given by (1.22) and (1.29) respectively.
Square root- harmonic mean divergence
( ) ( ) ( )
221
2,,,2
ni i i iSH i i i
p q p q M P Q S P Q H P Q p q
=
+= − = −
+
∑
.
(1.32)

Search

Similar documents

Tags

Related Search

Social Control On New TechnologiesWorking on New Communication TechnologyNew Information and Communication TechnologiePastoral care on new converts in IndiaCouncil On Foreign RelationsCouncil On American–Islamic RelationsForeign relations during Ancient Egypt`s New Research papers on information systemsNew Trends \u0026 Technologies of InformationNicopolis of Epirus. New studies on the mint

We Need Your Support

Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks