Internet & Technology

A direct geometrical method for bounding the error exponent for any specific family of channel codes. I. Cutoff rate lower bound for block codes

Description
A direct geometrical method for bounding the error exponent for any specific family of channel codes. I. Cutoff rate lower bound for block codes
Published
of 12
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
  1548 zyxwvutsrqponmlkjih EEE TRANSACTIONS ON INFORMATTON THEORY, VOL. 38, NO. 4, SEPTEMBER 1992 zy A Direct Geometrical Method for Bounding the Error Exponent for Any Specific Family of Channel Codes-Part I: Cutoff Rate Lower Bound for Block Codes Dejan E. LaziC, zyxwvutsr ember, IEEE, and Vojin Senk, Member, IEEE Abstract-A direct, general and conceptually simple geometri- cal method for determining lower and upper bounds on the error exponent of any specific family of channel block codes is presented. It is considered that a specific family of codes is characterized by a unique distance distribution exponent, de- fined as the asymptotic (in codelength) negative logarithm of the expected (according to the codewords a zyxwvutsr riori probabilities) Bhattacharyya distance distribution, normalized by the code length. Depending on the channel characteristics, every such family has its own error exponent. The maximum error exponent over all the realizable families of channel block codes is the usual channel error exponent. The tight linear lower bound of slope - zyxwv   on the code family error exponent represents the code family cutoff rate bound. It is always a minimum of a sum of three functions; the first of these functions depends on the asymptotic Bhattacharyya distance characteristics of the code family, the second on the channel characteristics, and the third is the negative value of the code rate. The intrinsic asymptotic properties of channel block codes are revealed by analyzing these functions and their relationships. It is shown that the well-known random coding technique for lower-bounding the channel error exponent, when adequately interpreted, is but a special case of this general method. The (indirect) random coding technique uses the notion of an ensemble of all possible codes, thus concealing the requirements that a code family should meet in order to have a positive error exponent and at best attain the channel error exponent. These requirements are now stated in a limpid way using the (direct) distance distribu- tion method presented. Index Terms-Channel coding theorem, error exponent, non- random coding argument, distance distribution. 1. INTRODUCTION VER since Claude Shannon published his famous E 948 papers [ll, information theorists used and devel- Manuscript received October IO, 1990; revised September 6, 1991. This work was presented in part at the IEEE International Symposium on Information Theory, Budapest, Hungary, June 24-28, 1991. D. E. LaziC was with Universitit Karlsruhe, Fakultit fur Informatik, Institut fur Algorithmen und Kognitive Systeme, zyxwvutsr m Fasanengarten 5 (Geb. 5034), D-7500 Karlsruhe, FR Germany, Alexander von Humboldt Fellow on leave from Faculty of Technical Sciences, Institute for Com- puter Science, Control and Measurements, Trg Dositeja 0bradovii.a 6, 21000 Novi Sad, Yugoslavia. He is now with Universitat Karlsruhe, Fakultat fur Informatik, Institut fur ProzeRrechentechnik und Robotik (Geb. 4028), D-7500 Karlsruhe, FR Germany, as a Fellow of European Commy-ity-DGXII. V. Senk is with the Faculty of Technical Sciences, Institute for Computer Science, Control and Measurements, Trg Dositeja Obradoviia 6, 21000 Novi Sad, Yugoslavia. IEEE Log Number 9107507. oped his random coding technique of obtaining an asymp- totic (in code length) upper bound on the probability of block decoding error for the optimal channel block code. Formally, it consists of calculating the average probability of block decoding error over the ensemble of all sets of M codewords with dimensionality (length) zy   hat are possi- ble over an encoding space. This technique was enthusias- tically welcomed as the “way out of the impasse” and frequently designated as the central technique of informa- tion theory. For a large number of coding channels it has really succeeded in obtaining lower bounds on channel error exponents (reliability functions) defined as where R is the code rate (measured in bits/symbol), and P, opt( R, N) s the smallest possible probability of block decoding error for codes of code rate R and length N used on the channel considered. Thus, the upper bound on Peopt(R, N) s given by PeOpt(R P,,,,l(R> N) = exp2[-N.E(R) + o(N)], exp2(x) = 2”, (1b) where o(N) s a term of order less than N; s N + zy C it becomes small relative to N. (R). Three aspects of this technique were especially satisfac- tory: 1) the obtained lower bound on E(R) was positive for all code rates below the channel capacity, R,, and equal to zero at R,, 2) it was tight in the most important range of code rates, i.e., it agreed with the upper bound on E(R) for all code rates above R,,,, (critical rate), and below R,. (R,,,, is typically well below Rc), nd 3) it guaranteed that almost every code of code rate R and great enough N atisfied (lb). The most unsatisfactory aspect of the random coding technique is that it cannot determine the performance of 0018-9448/92 03.00 0 Y92 IEEE 11  LAZIC AND SENK: DIRECT GEOMETRICAL METHOD zyxwvutsrq OR BOUNDING THE ERROR EXPONENT zyxwvu 549 a specific family of channel block codes used on the channel considered, nor can it determine the require- ments that a specific family of channel block codes should meet in order to attain the probability of error guaranteed by Ob). In order to overcome this problem, it is necessary to define the code family error exponent where zyxwvutsr ? s a specific family of channel block codes with a unique distance distribution exponent, defined as the asymptotic (in zyxwvutsrqp ) negative logarithm of the expected (according to the codewords a priori probabilities) Bhat- tacharyya distance distribution, normalized by the code length, and zyxwvutsrqpo ,,(R, N) s the probability of block decod- ing error for the code of code rate R and dimensionality N that belongs to 9. ince P,,,(R, N) cannot be smaller than Peopt(R, ), it is obvious that 0 5 E(R)g 5 E(R). (3) It is the aim of this paper to present a new technique for the derivation of the tight linear lower bound of slope 1 on the code family error exponent, E,(R),. This direct method does not use the random coding argument, i.e., ensemble averaging, and is applicable to any specific family 35’ of channel blocks codes decoded using the maximum likelihood rule. The bound thus obtained has the form Eo(R)g = R,, R, zyxwvu 4) where R,, is a function independent of R that repre- sents the cutoff rate of the code family on the coding channel considered. The bound (4) is called the cutoff rate lower bound on the code family error exponent. The general distance distribution method for bounding E(R), is presented in Section 11, and used for the deter- mination of ,(RI,. In Section zyxwvuts 11, this method is applied to derive ,(RI, for several families of spherical and binary codes used on the additive white Gaussian noise (AWGN) channel and binary symmetric channel (BSC), as well as to establish the impact of the minimum distance on E,(R),. zyxwvutsr n improvement of the method for obtaining <R)a is analyzed in a companion paper [2] (hereafter denoted as Part 11). This improvement gives the exact solution for E( R), of some important families of channel block codes, and tight solution at low code rates for any specific family of block codes. It contains the full geometrical explana- tion of all the phenomena encountered, including the impact of the minimum distance on E(R),,. 11. DISTANCE ISTRIBUTION ETHOD ND ITS APPLICATION TO THE DETERMINATION F THE CUTOFF RATE LOWER BOUND N E(R), This method will be demonstrated on the standard model for studying the reliability of communications over noisy transmission channels. The model consists of a coding channel, Z’, with an encoder at its input and a decoder at its output. When the input to the channel is an n-tuple from X”, = 1,2;.., it outputs a random n-tuple from Y“, where input and output alphabets X nd Y may be of different cardinality, 1x1 = q, IYI = q’, 1 < q, q’ 5 x A coding channel is completely defined by X, Y and the set of likelihood functions, {P[y‘”’ x‘“)], x “) E X“, y(”) E Y”, n = 1,2;..). The encoder generates a code- word of the channel block code B(R, N,P) = {xm x,,, € PC ”, m = 1,2;.., M}, where zyx is the available encod- ing space. The decoder uses the received random vector y = (yl;.., yN) o produce the estimate 3 of the transmit- ted codeword from B( R, N, 29 according to a convenient decoding rule. Denote by e,(x,) the probability that 3 equals x, when x,, m f , is sent over 5 i.e., e,,(x,) ~~P[i=x,Ix,,j#rn], x,, x, E B( R, N,Z), m = l;.. ,M. (5) This quantity will be called the error effect of the code- word x, on the probability of erroneous decoding when x,, is sent over since M P,, = P[i z x, x,,] = e, x,), r = I,..., M. I= I (6) The overall probability of block decoding error for B(R, N,X), used on is M M zyx   P,(R>N) = c P[x,lP,,, = c c P[x,Ie,,(x,). m= 1 m=l /=I J Wl (7) An upper bound, P,(R, N), on P,(R, N) for any B(R, N,Z) used on can be obtained upper bounding e,(x,) for each pair of codewords from the code. Sorting the M(M I) upper bounds on the error effects of each codeword on all others by descending order, L different values of such bounds are obtained (1 I 2 M(M 1)). Denote by S he Ith value of the upper bound on the error effec_t (e, > e,, ,, = l;.., L - l), where there are A?, (1 5 M, 5 M(M 1_>) ounds- on it that have the value S out of which MnII 0 5 M,, 5 M 1) have the sole influence on the determination of P,,. P,(R, N) s thus overbounded by Mi P,(R,N) = c CP[x,I~,,~, m= I I= 1 iM = e4 c P[.,ln;l,,. (8) l=l m=l Denoting by (A?,) the expected number of codewords in B(R, N,Z) hat have the same upper bound on the error  1550 zyxwvutsrqponmlkjih EEE TRANSACTIONS ON INFORMATION THEORY, VOL. 38, NO. 4, SEPTEMBER 1992 effect on an another codeword, evaluated as zyxwvuts   m=l zyxwvutsrqpo x, E zyxwvutsrq (R,N,X), zyxwvuts   = l;.. zyxwv L 9) one obtains that zyxwvuts   FS,(R,N) = LQ/>Z,. (loa) I= 1 It is obvious that using the same sorting procedure one can also obtain the lower bound on P,(R, N) whose form is c FJR,N) = c M>g/, (lob) I= zyxwvutsrqponmlkjihgf   where &, (MI>, nd e, are defined analogously to their counterparts in (loa). Upper and lower bounds on the error effects are best determined and sorted according to a convenient distance between different codewords. Such a distance will be introduced in the sequel, assuming that a maximum likelihood decoder (MLD) is used. MLD selects as its estimate the codeword x, for which the likelihood function P[y x,] is largest. Though its decision is optimum only when all the codewords are equally likely, it is often robust in the sense that it gives the same or nearly the same P,(R, N) egardless of the a priori probability distribution of the codewords, {P[ ,], m = l;.., MI. MLD sets 3 = x, E B(R, N,Z) if P[y x,] > P[y x,), j = I;.., M, J f m. Ties are re- solved arbitrarily, for instance in the favor of smaller index. This decision criterion can be given a geometrical interpretation using the appropriate decision regions (also called Voronoi regions), that perform the partitioning of the whole decoding space YN. MLD sets 2 = x,, if and only if the channel output vector y belongs to the Voronoi region V(x,), defined as zyxwvuts   VX, n qx, I x,), qx, I XI) I= I I'm (smaller index tie resolving rule is assumed). An attempt to visualize the partition of a part of the channel output space YN nto Voronoi regions is shown in Fig. 1. The error effect of the codeword x, on the probability of erroneously decoding when codeword x, is sent through the channel can be bounded using the two iso- lated codewords bound (see Fig. 1) Fig. 1. Visualization of the partition of a part of the channel output space Y into Voronoi regions, and the expansion of the Voronoi region Wx,) into V(x, Ix,) that enables the determination of the two isolated codewords upper bound on the error effect. y, denotes the most probable channel output when the codeword x, has been trans- mitted over it. that can be further overbounded by the Bhattacharyya upper bound [3, pp. 62-631 C x,) = exp2[ -~d,(X,,X,)], (13) where is the normalized (by N hattacharyya distance. It is readily seen that dR(xn,, ,) depends on x,, x , and the characteristics of the channel considered. If YN1is contin- uous, the sum in (14) is substituted with an integral. For the AWGN channel, the normalized Bhattacharyya distance is equal to [3, p. 631 4,( 9,) = [SNR/(81n(2))ld~(x,,x,), zy 15) where 4 is the squared Euclidean distance normalized by the expected codeword energy, E = C, P[x, IIx,~ 2 and SNR is the signal-to-noise ratio (ratio of E and noise energy, EAWGN Nu2, where (+? is the variance of the AWGN component). For the BSC the normalized Bhat- tacharyya distance is equal to gA xni,x,) = [-1dJ.lp l-p)]d, xrn,xj), (16) where p is the crossover probability of the BSC, and dH denotes the normalized (by N amming distance. We can now define the expected Bhattacharyya dis- tance distribution of the block code B(R, N,Z) as M,, P, = Pl(dsr, R, N P[x,]- M-1 m= 1 Here L 5 M(M 1)/2 is the number of different nor- malized Bhattacharyya distances in the code (dR1 < ds2 < < &), M,nl is the number of codewords on dis- I 11  uzic AND SENK: DIRECT GEOMETRICAL METHOD FOR BOUNDING THE ERROR EXPONENT 1551 tance zyxwvutsrqpo ,, from zyxwvutsr ,, and (MI) s the expected (according to {P[x,], m zyxwvutsrqp   l;*.,M}) number of codewords on dis- tance d,, from a codeword in zyxwvutsrq (R, N,t% ). EBDD is the set of relative frequencies of the expected (according to {P[x,], m = l,..., zyxwvutsrq }) umbers of codewords on distance d,, from a codeword in B(R, N,Z). It is obvious that these relative frequencies sum up to 1. In the case when B(R, N,2) is distance invariant, i.e., when the set of Bhattacharyya distances is the same from any codeword in the code, EBDD is independent of zyxwvut P[n,], zyxwvu   = 1;--, MI. Note also that EBDD remains unchanged if the normal- ized Bhattacharyya distance is substituted by any mono- tonically increasing function of &. For instance, LI in the case of the AWGN channel and 4 in the case of the BSC are suitable for this substitution, as can be seen from (15) and (16). The upper and lower bounds on the error effect (used in (10)) can always be simply related to the normalized Bhattacharyya distance d,, (a simple upper bound is given by (13)). Generally, introducing these bounds, denoted by e', zyxwvutsrq   '(&,, R, zyxwvuts ) nd e, e(d,,, R, NI, together with (17) into (10) and overbounding (M 1) by M one obtains that the form of upper and lower bounds on P,(R, N) s L 2 e( R, N, ) P(dB,, R7 N, I= 1 < Few N) L kf c P(d,,, R, N) l= 1 *e (dB,, R, N), (18) I since L = = & and M,) = M - l)P(dB,, R, N) = k,) zyxwvut y,). ntroducing the notion of the expected Bhattacharyya distance density, (19) p(dB, R7 4/31 5 dB dRL9 0, elsewhere, ebdd 2 these bounds attain the form 2 ee(~, ) = (M I)/ ~P(W, R, N) zyxwvut A1 e'(x,R,N)&. (20) Since EBDD is essentially discrete for any block code, (20) has practical sense only if a function p(d,, R, N), continuous in 4 can be associated to EBDD of B(R,N,Z), whose features are that the difference be- tween bounds in (18) and (20) s negligible. This is always the case when L 00, if the associated continuous func- tion represents an adequate interpolation of EBDD of B(R, N,Z). L + CD implies M + 00, which further implies that R for R finite. Only this later case is of interest. The method of obtaining the upper and lower bounds on P,(R, N) sing (18) and (20) will be called the distance distribution method. The expressions (18) and (20) are, as can be easily seen from their derivation, absolutely gen- eral and can be applied to any channel, regardless of whether random or deterministic codes are considered. For random codes EBDD corresponds to the probability distribution of the expected number of codewords on the normalized Bhattacharyya distance d,, from a codeword in the code. Naturally, the bounds (18) and (20) will have sense only if the existence of a block code with underlying EBDD (ebdd) can be proved. Finally, it can be observed that (18) and (20) represent linear transformations of EBDD and ebdd into upper and lower bounds on P,(R, N) that are relatively simple to evaluate, as will be seen in the sequel. Since the aim of this paper is obtaining the cutoff rate lower bound on the code family error expo- nent, from now on we will consider only the upper bound on P,(R, NI, though the expressions analogous to those derived in the rest of this section are also valid for the lower bound on it. for N finite, and that N The expression (18) can always be overbounded by while an overbound on (20) is Applying (21a) to all the members of a specific family 9 of channel block codes, and introducing it into (21, the lower bound on E(R), is obtained as supposing that the limit in (22) exists. Note that the term ( /N)ld( L) is omitted, since when L exponentially grows with N all the conditions for switching to continu- ous case (21b) are met, and that introducing (21b) into (2) gives (22) again (the only difference is that P(&, R, N) is substituted by p(&, R, NI). Thus, in what follows, the discrete and continuous case need not be treated sepa- rately. We will use the continuous denotation in the sequel. It is convenient, in order to proceed with the analysis of (22), to define the distance density exponent (dde) (or the I 11  1552 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 38, NO. zyxw , SEPTEMBER 1992 distance distribution exponent (DDE) in the discrete case) as zyxwvutsrq \E, elsewhere, where &, = limN+m{cfBl} nd zyxwvutsr jBL = 1imN+%{dBL}, s well as the lower bound on the error effect exponent (EEE) as Supposing that the limits in (23) and (24) exist, and that (23) specifies the code family zyxwvut , xchanging lim and min operations in zyxwvutsrqp 22) gives the following lower bound on zyxwvut ( R), E WB = <?in {E,(d,,R) E,(dB, R,g) zy . (25) zyxwvut BI ~ -B 5 I Here, E,(&,, R) s dependent on the code family charac- teristics (that may, but often does not, depend on the code rate), and &e(d,, R, 9 s dependent on the channel characteristics and, eventually, the code family character- istics and the code rate, subject to the choice of the way of overbounding Z . Such an overbound that depends on 5 B nd R is used in Part 11. The expression (25) is the basic formula for evaluating lower (upper) bounds on E(R),, that displays the intrin- sic dependence of the code family error exponent on the characteristics of B , and on R. This expression is very simple, general and conspicuous. In order to make it even simpler, at least in notation, denote by &B.9eff that nor- malized Bhattacharyya distance at which the minimum in (25) is obtained, so that awL3 =-WdB.@cff?R) MdB.@Cff?R>9) -R. (26) Maximizing E(R), over all realizable families of block codes, one would obtain the lower bound on the channel error exponent. It is now easily seen that the usual ran- dom coding technique for bounding the channel error exponent when adequately interpreted is but a special case of this general technique. Namely, in that case the DDE (dde) is defined for codes whose codewords are drawn at random according to an arbitrary probability distribution on 2, nd then optimized over all possible probability distributions on zyxwvuts   he random coding tech- nique is usually interpreted as averaging P,(R, N) over an ensemble of all possible codes of given cardinality and dimension, though it is almost never carried out in that way, but according to the procedure just described. This point will be discussed again in the conclusion, after exhibiting the power of the distance distribution method on several examples usually encountered in the literature. The simplest lower bound on EEE is obtained introduc- ing (13) into (24), so that Ce:e(d,, R, 9 EAdR) = dB, (27) i.e., it represents simply the normalized Bhattacharyya distance. This bound does not depend on R or 9. There are families 9 f block codes whose DDE (dde) is not a function of the code rate. Having such a code rate invariant family, and using (271, the corresponding lower bound on E(R), is, according to (251, given by Since neither E,(d,) nor dR depend on R, his is a linear function of R, of slope - 1. Since (28) is always tight in a certain range of code rates, as shown in Part zy 1, it is the desired cutoff rate lower bound F,(R), on E(R),. Ac- cording to (26), expression (28) can also be written as e0(R)L8 = E (d;,df) + d;;*,ff - R e R , - R, (29) where dg9 cff is that normalized Bhattacharyya distance at which the minimum in (28) is obtained, and R,, = E 8( deff) + dg eft represents the cutoff rate of the family 9 sed on g. bviously, if E,(&,) is differen- tiable and dgBeff s in the interval [ ,, I, it satisfies the relation Maximizing (28) over all realizable code rate invariant families of block codes, one would obtain the cutoff rate lower bound on error exponent for the channel itself. Fig. 2 gives a simple graphical interpretation of the presented method of obtaining Ro~* or a typical code rate invariant family of block codes that have a unique dde. It is noteworthy that the asymptotic minimum Bhat- tacharyya distance, &,, of 9 oes not have any influence on Ro,,, unless d;;,,cff is equal to l. Examples of code rate invariant families of spherical and binary codes, used in the AWGN and binary symmet- ric channel will be used in the next section to illustrate the power of the distance distribution method in obtain- ing the appropriate cutoff rate lower bounds on the code family error exponent. 111. EXAMPLES F THE APPLICATION F THE DETERMINATION F THE CUTOFF ATE LOWER DISTANCE ISTRIBUTION ETHOD OR BOUND N CODE FAMILY RROR XPONENT In this section, the distance distribution method will be applied to determination of the cutoff rate lower bound on E( R), for some important code rate invariant families of channel block codes used on the AWGN and binary symmetric channels. Among the most important families of block codes for any symmetric channel and encoding space are those that are uniformly distributed over F, I I1
Search
Similar documents
View more...
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks