UMD Probability Qualifying Exams/Aug2010Probability

Problem 1Edit

Two persons, A and B, are playing a game. If A winsa a round, he gets $4 from B and wins the next round with probability 0.7. If A loses the round, he pays $5 to B and wins the next round with probability 0.5.

(i) Write downt he transition matrix of the Markov chain with two states, {A won the current round, B won the current round} and find the stationary probabilities of the states

(ii) Find  .


(i) The Markov transition matrix will be the 2x2 matrix   where   corresponds to a win for Player A and   corresponds to a loss for Player A. For example,   is the probability that Player A wins after winning in the previous hand;   is the probability that Player A wins after losing in the previous hand; etc. This will give


The stationary distribution will be the tuple   such that  . We can calculate this explicitly:

  yields the following system of equations:   Using the fact that   must be a probability (i.e.  ) we get  .

(ii) Since   is positive, and hence ergodic, then any initial probability distribution will converge to the stationary distribution just calculated,  . Thus as   Player A will win with probability  . Can Player A expect to have more money though? For sufficiently large   we can compute Player A's expected winnings in one round:


Thus Player A should expect to have more money than before the game with probability 1.

Problem 2Edit

(i) Let   be a random variable with zero mean and finite variance  . Show that for any  


(ii) Let   be a square-integrable martingale with  . Show that for any  



(i)   were the second-to-last inequality is the standard Chebyshev's inequality.

Problem 3Edit


Problem 4Edit

Let   be random variables with finite expectations.

(i) Show that   implies  .

(ii) Show that if   is identically distributed with  , then



(i) Let  . Easy to see that   is convex.

Then by Jensen's Inequality we have


 . Taking the expectation on both sides gives