Math 5652: Introduction to Stochastic Processes Homework 6: due

Math 5652: Introduction to Stochastic Processes
Homework 6: due Tuesday, May 5. This is the last homework.
You are welcome and encouraged to discuss the problems with your classmates; please write
up your own solutions, and indicate collaborators on your write-up.
(1) (10 points) (Durrett 4.27) This is an analysis of the Duke–UNC Basketball game. (It was
UMN–UST in lecture.) As a reminder, there were 4 states to the chain, which I’ll write
DO, DS, UO, US for Duke or UNC on Offence or Scoring. When Duke was on offence,
it took them on average 1/3 of a minute to part with the ball; after Duke scored, it took
UNC 1/5 of a minute to take the ball down the court, and then they were on Offence.
(If Duke lost the ball, UNC ended up on offence immediately.) Similarly, when UNC
was on offence, it took them 0.4 minutes to part with the ball, and it took Duke 1/6 of a
minute to take the ball back down the court after UNC scored. The full transition rate
matrix (from which you can tell also the probabilities of scoring) was as follows:
 DO
DO −3
DS 
 0
UO 1
US
6
DS
2
−5
0
0
UO US 
1
0
5
0 

−2.5 1.5 
0
−6
(a) Find Ei [TDS ], the expected time to hit state DS from each of the other states.
(b) Use your solution to show that the number of visits to state DS by time t satisfies
N1 (t)/t → 0.6896 (baskets per minute). (Hint: think about an alternating renewal
process with states “in DS” and “not in DS”.)
(c) Similarly compute the long-time average rate at which UNC scores.
(2) (10 points) (Durrett 4.34) Customers arrive at the Rainbow convenience store at a rate
of 20 per hour. When two or fewer customers are in the checkout line, a single clerk
works and the mean service time is 3 minutes. When there are three or more customers
in the line, an assistant comes over to bag up the groceries, and the mean service time
is reduced to 2 minutes. Assuming the service times are exponentially distributed, find
the stationary distribution for the number of people in the checkout line.
(3) (10 points) (Durrett 4.31) Excited by the recent warm weather, Jill and Kelly are doign
spring cleaning at their house. Jill takes an exponentially distributed amount of time with
mean 30 minutes to clean the kitchen. Kelly takes an exponentially distributed amount
of time with mean 40 minutes to clean the bathroom. The first one to complete their
task will go outside and start raking leaves, a task that takes exponentially distributed
amount of time with mean 1 hour.1 When the second person is done inside, they will
help with the raking, which will then be happening at rate 2. (If, of course, it isn’t done
already.)
What is the expected time until all three chores are done?
1They
must have an oddly shaped yard...
1
(4) (20 points) (Durrett 4.42) At the registration at a very small college, students arrive at
the English table at rate 10 per hour, and at the Math table at rate 5 per hour. A student
who completes service at the English table goes to the Math table with probability 1/4,
and to the cashier with probability 3/4. A student who completes service at the Math
table goes to the English table with probability 2/5, and to the cashier with probability
3/5. (We’re assuming that it’s possible to visit the same table multiple times, so you
can go Math to English to Math to English...) Students who reach the cashier leave the
system after they pay. Note that each table (and the cashier) functions as a queue, so if
you’re not the first person there, you’re waiting.
Suppose the average service times for the English table, the Math table, and the cashier
are 25 seconds, 30 seconds, and 20 seconds respectively, and the distributions are exponential. We claimed in lecture that the stationary distribution for such a network should
be the product of M/M/1 queues with appropriate arrival rates.
(a) Find the effective arrival rates to each of the stations, and therefore write down the
stationary distribution.
(b) Check that your stationary distribution satisfies the equations of stationarity. (This
requires writing out the equations of stationarity!)
(c) On average, how many students are waiting in front of each of the three stations?
(Include the student in service.)
(5) (10 points) (Durrett 5.3) Let Xi = eηi with ηi ∼ N (µ, σ 2 ) iid random variables. Classify
for which pairs of values (µ, σ) the process
Mn = X0 X1 . . . Xn
is a martingale.
(6) (20 points) (Durrett 5.7) Consider a general birth-death chain. This is a countable statespace Markov chain: the state space is {0, 1, 2, . . . }. The transitions are only from a state
to one of the adjacent states:
p(x, x + 1) = px ,
p(x, x) = 1 − px − qx ,
p(x, x − 1) = qx if x ≥ 1.
(You are allowed to stay in the state you were in.)
Let
z y−1
X
Y qx
φ(z) =
.
p
y=1 x=1 x
Show that φ(Xn ) is a martingale. Conclude (similarly to the gambler’s ruin example)
that
φ(N ) − φ(x)
Px (reach state 0 before state N ) =
.
φ(N ) − φ(0)
Use this to derive conditions that determine when this will be a recurrent Markov chain;
your condition should treat the behavior of φ(N ) as N → ∞.
2
(7) (20 points) (Durrett 5.11) In lecture, we computed the probability of winning or losing
Gambler’s ruin game. Here, we’ll compute the mean and variance of the time it takes.
(We talked about the mean as a hitting time earlier in the course.)
Recall that Gambler’s ruin has states 0, 1, . . . , N , with P(move up) = p(x, x + 1) = p
and P(move down) = p(x, x − 1) = 1 − p. We will write q = 1 − p, and assume p < 1/2
(so this is a losing game).
(a) Use Wald’s equation to find
Ex [T0 ] = Ex [first time when X = 0].
Your answer should depend on x and p.
(b) Show that
Qn = (Xn − (p − q)n)2 − n(1 − (p − q)2 )
is a martingale. (Hint: let ξn = Xn − Xn−1 ; these are iid random variables with
mean p − q. Express everything in terms of them.)
(c) Similarly to Wald’s equation, use the martingale stopping theorem on Q to find
Ex [T02 ] and hence Varx (T0 ).
Remark: This problem uses the martingale stopping theorem twice on unbounded martingales. This is actually OK (just as in Wald’s equation), but the non-cheating method
here would be to look at the first time to reach 0 or N , compute Px (XT = N ), and check
that this term is insignificant as N → ∞.
3