# UMD Probability Qualifying Exams/Aug2010Probability

## Problem 1

 Two persons, A and B, are playing a game. If A winsa a round, he gets $4 from B and wins the next round with probability 0.7. If A loses the round, he pays$5 to B and wins the next round with probability 0.5. (i) Write downt he transition matrix of the Markov chain with two states, {A won the current round, B won the current round} and find the stationary probabilities of the states (ii) Find ${\displaystyle \lim _{n\to \infty }P(A{\text{ has more money after }}n{\text{ rounds than before the game}})}$ .

### Solution

(i) The Markov transition matrix will be the 2x2 matrix ${\displaystyle Q=(q_{ij})}$  where ${\displaystyle i,j=1}$  corresponds to a win for Player A and ${\displaystyle i,j=0}$  corresponds to a loss for Player A. For example, ${\displaystyle q_{11}}$  is the probability that Player A wins after winning in the previous hand; ${\displaystyle q_{01}}$  is the probability that Player A wins after losing in the previous hand; etc. This will give

${\displaystyle Q=\left({\begin{array}{cc}0.7&0.3\\0.5&0.5\end{array}}\right)}$

The stationary distribution will be the tuple ${\displaystyle \pi =(a,b)}$  such that ${\displaystyle \pi Q=\pi }$ . We can calculate this explicitly:

${\displaystyle (a,b)\left({\begin{array}{cc}0.7&0.3\\0.5&0.5\end{array}}\right)=(a,b)}$  yields the following system of equations: ${\displaystyle .7a+.5b=a;.3a+.5b=b}$  Using the fact that ${\displaystyle \pi }$  must be a probability (i.e. ${\displaystyle a+b=1}$ ) we get ${\displaystyle a={\frac {5}{8}},b={\frac {3}{8}}}$ .

(ii) Since ${\displaystyle Q}$  is positive, and hence ergodic, then any initial probability distribution will converge to the stationary distribution just calculated, ${\displaystyle \pi }$ . Thus as ${\displaystyle n\to \infty }$  Player A will win with probability ${\displaystyle 5/8}$ . Can Player A expect to have more money though? For sufficiently large ${\displaystyle n}$  we can compute Player A's expected winnings in one round:

${\displaystyle E[{\text{winnings}}]=5/8*4+3/8*(-5)=5/8>0}$

Thus Player A should expect to have more money than before the game with probability 1.

## Problem 2

 (i) Let ${\displaystyle X}$  be a random variable with zero mean and finite variance ${\displaystyle \sigma ^{2}}$ . Show that for any ${\displaystyle c>0}$  ${\displaystyle P(X>c)\leq {\frac {\sigma ^{2}}{\sigma ^{2}+c^{2}}}}$ . (ii) Let ${\displaystyle \{X_{n},n\geq 1\}}$  be a square-integrable martingale with ${\displaystyle E(X_{1})=0}$ . Show that for any ${\displaystyle c>0}$  ${\displaystyle P(\max _{1\leq i\leq n}X_{i}\geq c)\leq {\frac {var(X_{n})}{var(X_{n})+c^{2}}}}$ .

### Solution

(i) ${\displaystyle P(X\geq c)\leq P(|X|\geq c)=P(|X-E[x]|\geq c)\leq P(|X-E[X]|\geq c+\sigma )\leq {\frac {\sigma ^{2}}{c^{2}+2c\sigma +\sigma ^{2}}}\leq {\frac {\sigma ^{2}}{c^{2}+\sigma ^{2}}}}$  were the second-to-last inequality is the standard Chebyshev's inequality.

## Problem 4

 Let ${\displaystyle X,y}$  be random variables with finite expectations. (i) Show that ${\displaystyle E(X|Y)=0}$  implies ${\displaystyle E(|X+Y|)\geq E(|Y|)}$ . (ii) Show that if ${\displaystyle (X,Y)}$  is identically distributed with ${\displaystyle (Y,X)}$ , then ${\displaystyle E(|3X-Y|)\geq E(|X+Y|).}$

### Solution

(i) Let ${\displaystyle f(x)=|x+Y|}$ . Easy to see that ${\displaystyle f}$  is convex.

Then by Jensen's Inequality we have

${\displaystyle f(E(X|Y))\leq E(f(X)|Y)}$

${\displaystyle |E(X|Y)+Y|\leq E(|X+Y||Y)}$ . Taking the expectation on both sides gives

${\displaystyle E(|Y|)\leq E(|X+Y|)}$ .