STOCHASTIC PROCESSES 07
Transition Matrices
A transition matrix stores all one-step transition probabilities of a finite-state discrete-time Markov chain.
The matrix
For states \(1,\ldots,m\), define
Using this row convention, row \(i\) contains probabilities of leaving state \(i\).
1. Row sums
\[p_{ij}\ge0,\qquad \sum_jp_{ij}=1.\]Such a matrix is called stochastic.
2. Example
\[P=\begin{pmatrix}0.8&0.2\\0.3&0.7\end{pmatrix}.\]From state \(1\), the next state is \(1\) with probability \(0.8\) and \(2\) with probability \(0.2\).
3. Two-step transitions
The matrix
\[P^2=P\,P\]contains two-step transition probabilities. More generally, \(P^n\) gives \(n\)-step probabilities.
4. Why matrix multiplication appears
To reach a future state through an intermediate state, probabilities along a route are multiplied and alternative intermediate routes are summed. Matrix multiplication performs exactly these operations.
5. Absorbing state
If state \(i\) cannot be left, then \(p_{ii}=1\) and all other entries in row \(i\) are zero.
6. Convention warning
Some texts use column-stochastic matrices instead. Either convention is valid if the probability-vector convention is kept consistent.