NEED THE STEPS Problem 4. Consider the Markov chain with three states l,2, and 3
ID: 1926784 • Letter: N
Question
NEED THE STEPS
Problem 4. Consider the Markov chain with three states l,2, and 3, and with state transition matrix as 1/2 1/3 1/61 3/4 0 1/4 a) Show that this is a regular Markov chain, Solution: By taking powers of P, we find that TO 5000 0.3333 0.16671 P3 3 0.5625 0.2500 0.1875 which has all positive entries. Therefore, this L0.3750 0.5000 0 1250J is a regular Markov chain. b Assume that the process starts in State 1, find the probability that it is in State 3 in two steps, 0.5000 0.3333 0.1667 Solution: We can obtain that P 0.3750 0.5000 0.1250 The probability 0.7500 0.2500 corresponds to t (1,3)-th entry of this matrix, which is 0.1667. c Find the limiting probability vector w. Solution Since this is a 3 x 3 matrix, we assume that w Gw, w. wal. And we can use wP w and w1 w2 wa 1 to obtain that the limitingExplanation / Answer
regular markov chain means that the probability of going from one state to every other state is non -zero in some finite number of steps so to show that a given markov chain is regular, we are gonna calculate p^2,p^3 and so on until we see every element of the matrix non zero. p^2=p.p=[ 0.5000 0.3333 0.1667 0.3750 0.5000 0.1250 0.7500 0 0.2500] still one term is zero so, we will have to try again p^3=p.p^2= [0.5000 0.3333 0.1667 0.5625 0.2500 0.1875 0.3750 0.5000 0.1250] so each term is non zero, hence we can say that the markov chain is regular for 3 steps. b.) to calculate probability of going from one state to another state in 2 steps , we will have to find out the 2 step transition matrix which is p^2. so p^2 is as shown above the probability of going from state 1 t 3 in two steps will be(p^2)13 ie the the element in row 1 and column 3. c.) the limiting vector the limiting vector is a row vector which upon multiplication with the tansition probability matrix gives the same vector as itself and sum of its elements should be equal to 1. so, w.P=w or [w1 w2 w3].[1/2 1/3 1/6 3/4 0 1/4 0 1 0]=[w1 w2 w3] we get three linear equations from this matrix equation which are homogenius and the fourth equation is w1+w2+w3=1 upon solving these fourth equation, you will get the result. this is it. i guess you were not asking how to solve these linear equations or find matrix square or cube cuz that can be done using any scientific calculator.
Related Questions
Navigate
Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.