STOCHASTIC PROCESSES 08
Probability Vectors
A probability vector records the probability that a Markov chain occupies each possible state at a particular time.
The distribution vector
For states \(1,\ldots,m\), write
using a row-vector convention.
1. Components sum to one
\[\pi_{n,j}\ge0,\qquad \sum_j\pi_{n,j}=1.\]2. Initial condition
If the process starts with certainty in state \(2\),
\[\pi_0=(0,1,0,\ldots,0).\]An uncertain initial state can instead be represented by any valid probability vector.
3. Updating the distribution
With the row-stochastic convention of the previous lesson,
and therefore
\[\pi_n=\pi_0P^n.\]4. Meaning of the multiplication
For destination state \(j\),
\[\pi_{n+1,j}=\sum_i\pi_{n,i}p_{ij}.\]We sum the probability of being in each possible current state multiplied by the probability of moving from that state to \(j\).
5. Distribution versus simulation
The vector calculation gives the full state distribution exactly for the finite model. A simulation gives one randomly selected trajectory.
6. Stationary distribution
A probability vector \(\pi\) is stationary when
\[\pi=\pi P.\]If the chain starts with this distribution, its state distribution remains unchanged at later steps.