Description

Stochastic Processes
A stochastic process is a model that evolves in time or space subject to probabilistic laws.
The simplest example is the one-dimensional simple random walk.. The process starts in
state X
0
at time t = 0. Independently, at each time instance, the process takes a jump Z
n
:
Prob { Z
n
= -1} = q, Prob { Z
n
= +1} = p and Prob { Z
n
= 0 } = 1 - p - q.
The state of the process at time n is
X
n
= X
0
+ Z
1
+ Z
2
+ … + Z
n.
Assume for convenience that X

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.

Related Documents

Share

Transcript

Stochastic Processes
A
stochastic process
is a model that evolves in time or space subject to
probabilistic laws.
The simplest example is the one-dimensional simple
random walk..
The process starts in state
X
0
at time t = 0. Independently, at each time instance, the process takes a jump Z
n
: Prob { Z
n
= -1} = q, Prob { Z
n
= +1} = p and Prob { Z
n
= 0 } = 1 - p - q. The state of the process at time n is X
n
= X
0
+ Z
1
+ Z
2
+ … + Z
n.
Assume for convenience that X
0
= 0. Since E[ Z
n
] = p - q and VAR [Z
n
] = p + q - ( p - q)
2
, then E[ X
n
] = n (p - q) and Var [ X
n
] = n { p + q - ( p - q )
2
}. A stochastic process, such as the simple random walk, has the
memoryless
or
Markov property
if the conditional distribution of X
n
only depends on the most recent information: Prob { X
n
= k | X
n-1
= a, X
n-2
= b, … } = Prob { X
n
= k | X
n-1
= a } We can think of random walks as representing the position of a particle on an infinite line. The position of the particle can be
unrestricted,
or can be restricted by the presence of
barriers
. A barrier is
absorbing
if the process stops once the particle reaches the barrier, or
reflecting
if the particle remains at the barrier until a jump in the appropriate direction causes it to move away. Problems of interest are What is the expected time to absorption at a barrier, if one exists ? What is the distribution of time spent at a reflecting barrier, if one exists ?
Examples of Stochastic Processes.
Example [ Reservoir Systems] Here Z
n
is the inflow of water into a reservoir on day n. Once a particular water threshold a is reached, an amount of water b is released. The system is a random walk on the range [0, a] with a reflecting barrier at a. Example [ Company Cash Flow] X
0
is the initial capital of the company. During trading period i, the company receives revenue r
i
and incurs costs c
i
, so the change in liquidity is z
i
= r
i
- c
i
. The company will continue to trade profitably as long as its accumulated capital is non zero. The underlying process is defined on the positive real line with an absorbing barrier at zero. Example [Building Society Funds]. This is similar to the last example, except that the company pays out an amount b if the accumulated funds on a particular day exceeds an amount a. Building societies are designed to provide a steady flow of funds into the housing market and relatively simple models give insight into how the market can be regulated. Example [Market Share] we are given the srcinal market P Final States shares p
i
of three companies and the
transition matrix
(Initial) 1 2 3 P = [ p
i, j
] 1 p
1, 1
p
1, 2
p
1, 3
where p
i, j
= Prob { that a customer of company i transfers 2 p
2, 1
p
2, 2
p
2, 3
to j over a single trading period} 3 p
3, 1
p
3, 2
p
3, 3
Stochastic processes of this type always reach a
steady state
which is an absorbing barrier and is independent of the
starting distribution
. The rate of convergence to the steady state depends on the values in the transition matrix.
The Infinite Single Server Queue M|M|1
In the simplest queue, customers arrive at an average rate to a queue with infinite capacity and one server. Assuming the Markov property holds, by taking very small time slices t, Prob { 1 arrival in the interval [t, t + t] } = t,
Prob { 0 arrivals in “ [t, t + t] } = 1
- t, Prob { More than 1 arrival in [t, t + t] } = 0. These are the classical conditions for the Poisson distribution, so P
n
( t ) = Prob { n arrivals in the interval [0, t] } = ( t )
n
/ n ! exp ( - t). and Prob {Interarrival time t} = Prob {First arrival t} = 1-Prob {No arrival in [ 0, t]} = 1 - P
0
(t) = 1 - exp ( - t ) so the interarrival time has an exponential distribution with parameter . By the same token, if on average customers are served per unit time, then the service times have an exponential distribution with parameter . Since both the arrival and service distributions, this single server queue is designated M | M | 1. The traffic intensity = / is an important characteristic of queuing networks. Unless < 1, the queue is unstable (i.e.) the expected queue size is infinite. In queuing models, the
system
consists of those in the queue plus those, if any, being served. The main items of interest in queuing models are the means and variances of the
Waiting time
for customers and the Queue or System
Sizes.
Kirkhoff’s Law
is useful in analysing queues. It states that if the queue is in equilibrium ( P
n
(t) is independent of t ) , the rates at which the states of the system are being incremented must equal the rates at which they are being decremented.

We Need Your Support

Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks