Random Algorithms Assignments

Random Algorithms
Assignments
Computer Science
Faculty of Fundamental Problems of Technology
Wroclaw University of Technology
Wroclaw 2014/2015
1
Probability Theory revision
Ex. 1 — Out of a batch of N pieces of an article, including M pieces meeting the standard requirements we
randomly choose n pieces (a) with replacement, (b) without replacement. Compute the probability, that among
the selected pieces we will have exactly k pieces satisfying the standard requirements. Define the probability
space of these experiments.
Ex. 2 — We choose with an uniform distribution a random point from a disc with a center O and radius R
(B(O,R) ). For what value of r the probability that we selected random point lies inside the disc B(O,r) is equal
to 12 ? Define the probability space which describes this experiment.
Ex. 3 — Prove, that if random events A and B are independent, then the events A and B are also independent.
Ex. 4 — Out of three candidates we select one chairman. To do this we place three ballots in a bin, but only
one ballot says “chairman”. The candidates one-by-one choose randomly ballot from the bin and the one, that
draws the ballot saying “chairman” wins. What is the probability, that a candidate wins if he draws as the (a)
first, (b) second, (c) third?
Ex. 5 — Let us suppose, that a drug test correctly detects drugs with probability 0.99 and correctly confirms
the lack of a drug with probability 0.99. Assume, that only 0.5% of the population takes drugs. What is the
probability, that an individual is under the influence of drugs if the test was positive?
Ex. 6 — Huckleberry Hound bought insecticide to kill a termite. We know two types of termites: woodeating termites and non-wood-eating termites. We know that the insecticide kills the wood-eating termite with
probability 0.8. Huckleberry Hound assumes that his small enemy is a wood-eating termite with probability 14
and is a non wood-eating termite with probability 34 . Huckleberry Hound calculated that the chance of killing
the termite is 0.45. What is the probability of killing a termite if it is a non-wood-eating termite?
Ex. 7 — As in the previous exercise we focus on the adventures of Huckleberry Hound and the termite. The
insecticide kills the termite with probability 45 if it is a wood-eating termite and with probability 13 otherwise.
Huckleberry Hound assumes that his small enemy is a wood-eating termite with probability 14 and is non woodeating termite with probability 43 . After using the insectcide the termite is still alive. What is the probability,
that the termite is a wood-eating type, since he survived?
Ex. 8 — (Monty Hall Problem) A player is taking part in a TV game show. The player is given a choice of
three doors. A prize is hidden behind one randomly chosen doors, while behind the other two doors are hidden
goats. The player selects one gate and the host, who knows what is behind each door, opens one door, behind
which there is a goat. Then, the host ask the player if he wants to switch his choice? What should the player
do, should he switch or stick with his choice?
Ex. 9 — Compute following quantities:
Pn
(a) k=0 nk ak bn−k = . . .,
Pn
k
(b) k=0 λk! = . . .,
Pn
|q|<1
(c) k=0 q k = . . ..
1
Ex. 10 — Let p ∈ (0, 1), λ > 0. Compute:
Pn
(a) k=0 k nk pk (1 − p)n−k ,
Pn
−λ k
(b) k=0 k e k!λ ,
Pn
(c) k=0 k(1 − p)k−1 p.
What are the quantities you calculated?
Ex. 11 — Define formally the random variable X which counts the number of pieces satisfying the requirements of standard in a random trial in Exercise 1. For scenarios (a) and (b) define the distribution of X.
Ex. 12 — We roll a dice. If we obtain a number of pips k 6= 3, we win £k. If we have k = 3, we toss a coin.
If we get heads, we lose £100. If we get tails we lose £5. Define the formal model describing this experiment
(i.e., the probability sample space Ω, σ-algebra F and P({ω}) for each random event). Compute E(X), where
X is the payoff after the game.
Ex. 13 — For what values of c the sequence pn = 3cn , (n = 1, 2, 3, . . .) describes the distribution of a random
variable X? What is the name of this distribution? Give an example of such random experiment, which result
is described by the random variable X. Compute P(3 ≤ X < 5).
cn
(n+1)! , (n
Ex. 14 — For what values of c the sequence pn =
random variable?
= 1, 2, 3, . . .) describes the distribution of a
Ex. 15 — We know that Var[X] = E[X − E(X)]2 . Prove, that Var[X] = E[X 2 ] − (EX)2 . For which random
variables Var[X] = 0?
Ex. 16 — Let a, b ∈ R and let X be a random variable with a finite 2nd moment (E[X 2 ] < ∞). Compute
E[aX + b] = . . ., Var[aX + b] = . . ..
Ex. 17 — Let X and Y be independent random variables. Compute
1.Var[X + Y ],
2.Var[X − Y ].
Ex. 18 — Let Y be a random variable with the following distribution: P[Y = π2 ] = P[Y = 0] = 41 , P[Y = π] =
1
2 . Describe the distribution of a random variable X, such that X = sin Y . Compute E[X] and Var[X].
Ex. 19 — Define and prove the Markov inequality.
Ex. 20 — Prove that the Markov inequality cannot be improved, i.e., for a fixed integer value k find a random
variable X, which takes only non-negative values and for which holds
P[X ≥ kEX] =
1
.
k
Ex. 21 — Prove the Chebyshev inequality using the Markov inequality.
Ex. 22 — Using the Markov inequality and the Chebyshev inequality estimate the probability, that in n
independent tosses of a fair coin we get at least 3n
4 heads.
2
Random Quicksort
Ex. 23 — For a set S = {1, 2, 3, 4} conduct the RQS algorithm and describe the sample space Ω and values
P({ω}) for each event. Give all elementary events which belong to the event A = [1 is compared with 4].
Compute P(A) by calculating probabilities of the relevant elementary events.
Ex. 24 — Let Y be a random variable describing the depth of the tree which we receive as a result of
conducting the RQS on a set S = {1, 2, 3, 4}. Compute EY .
Ex. 25 — Let Hn = 1 +
1
2
+
1
3
+ . . . + n1 . Prove that
Hn n→∞
−−−→
log n −
2
1.
S∞
Ex. 26 — Give the linear order on n=1 {0, 1}n which allows to read the order on S = {a1 , a2 , . . . , an } from
left to right after conducting the RQS algorithm.
Ex. 27 — Let C be a random variable describing the number of comparisons in the RQS. Compute E[C]
exactly.
3
AND-OR Trees
Ex. 28S— Let (Ω, F, P) be a finite probability space. Let Ai , i = 1, . . . , n be a set partition of Ω. For all Ai
∞
holds: n=1 Ai = Ω, Ai ∩ Aj = ∅ for i 6= j and P(Ai ) > 0. Prove that E(X|A1 )P(A1 ) + E(X|A2 )P(A2 ) + . . . +
E(X|An )P(An ) = E(X).
Ex. 29 — Compute maxx∈[0,1] Ax + (1 − x)B, where A and B are fixed real numbers.
Ex. 30 — For a given tree T4 follow the algorithm A from the lecture.
1 0
0 1 0 0
1 1
0 0
0 0 0 0
0 1
Ex. 31 — Compute E(X), where X is a random variable which counts the number of read operations in the
tree T2 , if the values 0 and 1 are set in leaves as follows
(a) with probability 12 in each leaf,
(b) 0 with probability 23 , 1 with probability
(c) following the schema
1
1
3
in each leaf,
0
0
0
Ex. 32 — Discuss if the algorithm A from the previous exercise can be improved in scenario (b).
Ex. 33 — Analogously to the AND-OR tree define the ⇒ − ⇐ tree and the AND-AND tree. Check, if the
expected value E(X) of readings in the ⇒ − ⇐ tree is equal to the E(X) in the AND-OR tree.
4
NOR-Trees
Let A∗ be a recursive algorithm defined as follows:
To find the Boolean value of a node v in a given NOR-tree, we first check the value of its left son. If value of
its left son is 0, we have to evaluate the value of the right son of v. If the value of the right son is also 0 we set
the value of v to 1, otherwise the value of v is 0. If the value of the left son of v is 1, we cut the whole subtree
rooted at the right son of v and the value of v is set to 0 (we don’t need to evaluate the value of its right son).
Theorem 1. Let A be a family of all deterministic algorithms which help to find the value of a root in a
NOR-tree. Let I be a family of all inputs for the leaves. By PI we denote some probability over the space I.
Let A ∈ A and I ∈ I. Let a random variable XA (I) denote number of readings after conducting algorithm A
on input I. Then,
min EPI (XA (·)) = EPI (XA∗ (·)).
A∈A
3
Ex. 34 — Prove the equivalence between NOR-trees and AND-OR trees, which were defined in the lecture.
Ex. 35 — Conduct the algorithm A∗ on the following NOR-trees.
Ex. 36 — Let us assume, that in a NOR-tree value 1 is assigned to every leaf with probability p. Compute
the value of p for which every node in this tree has value 1 with the same probability. Prove by induction that
your calculations are correct.
(k)
Ex. 37 — Let a random variable XA∗ describe the cost (number of leaves readings) of finding the boolean
value of the root in the NOR-tree Tk by conducting the algorithm A∗ . Show, that for the probability PI defined
in Exercise 36 it holds that
h
i
h
i
(k)
(k−1)
EPI XA∗ = (2 − p)EPI XA∗
.
Ex. 38 — For the hprobability
PI as in Exercise 36 prove, that in the NOR tree T2k (so also in the tree of
i
(2k)
AND-OR type) EPI XA∗
≥ n0.694 , where n = 22k . Compare this results with the result obtained for the
AND-OR tree after conducting algorithm A.
(This result can be improved, to show that the algorithm A gives the best possible result. If your are interested, you
can find more information in the following article: M. Saks, A. Wigderson, Probabilistic Boolean decision trees and the
complexity of evaluating game trees, Proceedings of the 27th Annual IEEE Symposium on Foundations of Computer
Science(1986), 29-38, Toronto, Ontario )
4
5
Optimal stopping
Ex. 39 — For the secretary problem we know the formula to compute the probability of success in the strategy
when the selector chooses the first cadidate that is best after the k interviews. We also know the time T (n)
starting from which (according to the optimal strategy) we should make a choice. Use both these information
to show, that if t(n) describes the optimal stopping time for the secretary problem with n candidates, then
T (n)
T (n)
1
+ ... +
1 + (n)
,
P[π(t(n) ) = 1] =
n
n−1
T
+1
where T (n) = min{t :
1
t
+ ... +
1
n−1
< 1}.
Ex. 40 — Using the observation that
1
T (n)
+
1
T (n)
+1
+ ... +
1
1
1
1
+ (n) + . . . +
< 1 ≤ (n)
n−1
n−1
T
−1 T
show why this inequality holds and prove that
Ri 1
R i+1 1
1
HINT: i
x dx < i < i−1 x dx.
(1)
T (n) n→∞ 1
−−−→ e .
n −
n→∞
Ex. 41 — Prove that P[π(t(n) ) = 1] −−−−→ 1e .
Ex. 42 — Prove that for all n holds P[π(t(n) ) = 1] > 1e .
Ex. 43 — Find the values T (n) for n = 5, 8, 10, 20, 100.
Ex. 44 — Draw the Hasse diagram for a tuple (X, ≺), where X = {a, b, c, d, e},
≺= {(a, b), (c, d), (c, e), (d, e), (a, d), (a, e)}.
Ex. 45 — For the given Hasse diagram for the tuple (Y, ≺) (1) give ≺.
Figure 1: Hasse diagram for tuple (Y, ≺)
Ex. 46 — For the order as in Exercise 45, the elements of the set Y in the secretary problem come in the
following order: a, c, e, d, b. Define the Hasse diagram which represents the scenario observed by the selector at
consecutive time steps.
Ex. 47 — A partial order with a Hasse diagram as shown in Figure 2 is called a star. Give an optimal
algorithm for the secretary problem on Sn .
Figure 2: Star Sn
Ex. 48 — Let us cosider a standard secretary problem. Define a simple algorithm in which the probability of
success is equal p ≥ 14 (regardless of the number of elements). How would the probability change if we apply
this algorithm for bigger families of partially ordered sets.
5
Ex. 49 — We roll a hexagonal, fair dice at most 6 times. After each roll we can stop the game. When we stop
the game, we receive as many krugerrands as many pips we currently see on the dice. Give a strategy which
maximizes the expected value of the payoff after the game.
Ex. 50 — Pikachu writes two arbitrary real numbers on two separate ballots and places both ballots in a bin.
We randomly choose one ballot from the bin. Now, we have two possibilities: 1) stick with the selected ballot,
2) reject this ballot and select the second one from the bin. We win, if we selected the ballot with the higher
number. Give a strategy, which guarantees the win with probability p > 12 .
HINT: Select randomly one more number R from the normal distribution.
Ex. 51 — You need a new staff assistant, and you have n people to interview. You want to hire the best
candidate for the position. When you interview a candidate, you can give them a score, with the highest score
being the best and no ties being possible. You interview the candidates one by one. Because of your company’s
hiring practices, after you interview the kth candidate, you either offer the candidate the job before the next
interview or you forever lose the chance to hire that candidate. We suppose the candidates are interviewed in
a random order, chosen uniformly at random from all n! possible orderings.
We consider the following strategy. First, interview m candidates but reject them all, these candidates give you
an idea of how strong the field is. After the mth candidate, hire the first candidate you interview who is better
than all of the previous candidates you have interviewed.
a) Let E be the event that we hire the best assistant, and let Ei be the event that ith candidate is the best
and we hire him. Determine P[Ei ], and show that
P[E] =
b) Bound
Pn
1
j=m+1 j−1
n
1
m X
.
n j=m+1 j − 1
to obtain
m
m
(ln n − ln m) ≤ P[E] ≤ (ln (n − 1) − ln (m − 1)).
n
n
c) Show that m(ln n − ln m)/n is maximized when m = n/e, and explain why this means P[E] ≥ 1/e for the
choice of m.
Ex. 52 — The Parking Problem. (MacQueen and Miller (1960)) Let’s assume that you are driving
along an infinite street toward your destination, for example the A-1 building. There are parking places along
the street, some of them are free and some of them are taken. You want to park as close to the A-1 building as
possible. What is more, you cannot turn around on this street. If you see an empty parking place at a distance
d before the A-1 building, should you take it?
Let us model a problem as follows. We assume that we start at point 0 and that there are parking places at
all integer points of the real line. Let X0 , X1 , X2 , . . . be independent Bernoulli random variables with common
probability of success p, where Xj = 1 means that parking place j is filled and Xj = 0 means that it is available.
Let T > 0 denote your target parking place. You can park at place j if Xj = 0 and you have to pay |T − j|.
You cannot see parking place j + 1 when you are at j. You cannot also return to a certain parking place j once
you pass it up. Notice, that if you reach T , you should choose the next available parking place.
The optimal algorithm (which minimizes the expected value of a payment) which should be used to solve the
described problem has a threshold character. We continue until r places from the destination and park at the
first available place from then on. As E[Nr ] we denote the cost of this strategy. Try to solve the following tasks:
(a)Compute E[N0 ],
(b)Define the recursion formula of E[Nr ],
(c)Prove by induction that E[Nr ] = r + 1 +
2pr+1 −1
1−p ,
(d)Find the optimal value ropt which minimizes the expected value fo the cost E[Nr ]. HINT: Investigate the
monotonicity of the sequence E[Nr+1 ] − E[Nr ]. When the elements of this sequence become no longer
negative?.
(e)Compute values ropt and E[Nropt ] for p ≤ 1/2 and for p = 0.7, p = 0.8, p = 0.9.
6
Ex. 53 — The Cayley Problem We have a population of n objects with values x1 , x2 , x3 , . . . , xn respectively.
From this population, a simple random sample of size at most m, where m ≤ n, is drawn sequentially without
replacement. We may stop the sampling at any time. Stopping is equivalent to selecting the last chosen object
and you receive the value of that object as a reward ( if we don’t stop the sampling before the stage, we treat
the m0 th element as selected). Find the optimal stopping rule (which maximizes the expected payoff) for the
population with 4 elements with the following values {1, 2, 3, 4} for m = 1, m = 2, m = 3, m = 4.
Ex. 54 — Suppose that we roll ten standard six-sided dice. What is the probability that their sum will be
divisible by 6, assuming that the rolls are independent?
HINT: Use the principle of deferred decisions, and consider the situation after rolling all but one of the dice.
Ex. 55 — Prove, that if A ⊆ B oraz P[B] > 0, then P[A] ≤ P[A|B].
7