Lemma:
Consider a collection of random variables Interpret as the State of the system at time n, and suppose that the set of possible values of the —that is, the possible states of the system is the set 1,…N. if there exists a set of bumber $P_{ij},i,j = 1$$,…N, such that whenever the process is in state i then,