Description

entropy

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.

Related Documents

Share

Transcript

90 Entropy Rates of a Stochastic ProcessLet S n be the waiting time for the nthhead to appear.Thus,S 0 = 0S n+1 = S n +X n+1where X 1,X 2,X 3,... are i.i.d according to the distribution above.(a) Is the process {S n} stationary?(b) Calculate H (S 1,S 2,...,S n).(c) Does the process {S n} have an entropy rate? If so, what is it? If not, why not?(d) What is the expected number of fair coin ?ips required to generate a randomvariable having the same distribution as S n?Solution: Waiting time process.(a) For the process to be stationary, the distribution must be time invariant. It turnsout that process {
S n} is not stationary. There are several ways to show this.
S 0 is always 0 while S i, i ?= 0 can take on several values. Since the marginalsfor S 0 and S 1, for example, are not the same, the process can
t be stationary.
It
s clear that the variance of S n grows with n, which again implies that themarginals are not time-invariant.
Process {S n} is an independent increment process. An independent incrementprocess is not stationary (not even wide sense stationary), since var(S n) =var(X n) + var(S n-1) > var(S n-1).(b) We can use chain rule and Markov properties to obtain the following results.H (S 1,S 2
,...,S n) = H (S 1) +n?i=2H (S i|S i-1)= H (S 1) +n?i=2H (S i|S i-1)= H (X 1) +n?i=2H (X i)=n?i=1H
(X i)= 2n(c) It follows trivially from the previous part thatH(S ) = limn?8H (S n)n= limn?82nn= 2 Entropy Rates of a Stochastic Process 91Note that the entropy rate can still exist even when the process is not stationary.Furthermore, the entropy rate (for this problem) is the same as the entropy of X.(d) The expected number of ?ips required can be lower-bounded by H (S n) and upper-bounded by H (S n) + 2 (Theorem 5.12.3, page 115). S n has a negative binomialdistribution; i.e., Pr(S n = k) =? k-1n-1?(12

We Need Your Support

Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks

SAVE OUR EARTH

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...Sign Now!

We are very appreciated for your Prompt Action!

x