STOCHASTIC PROCESSES 11
Generator Matrices
The generator matrix stores the transition rates of a finite-state continuous-time Markov chain.
The generator
Using a row convention, for \(i\ne j\),
\[q_{ij}\ge0\]is the rate from state \(i\) to state \(j\). The diagonal is defined by
so every row sums to zero.
1. Example
\[Q=\begin{pmatrix}-\lambda&\lambda\\ \mu&-\mu\end{pmatrix}.\]The off-diagonal entries are transition rates; the negative diagonal records the total rate of leaving each state.
2. Why the diagonal is negative
For a small time \(h\),
\[P(h)\approx I+hQ.\]The diagonal entry becomes
\[1+q_{ii}h=1-a_i h,\]which is the approximate probability of remaining in state \(i\).
3. Generator versus transition matrix
A DTMC transition matrix contains probabilities and rows sum to one. A CTMC generator contains rates and rows sum to zero.
4. Transition probabilities through time
For a time-homogeneous finite CTMC,
where the matrix exponential propagates transition probabilities through continuous time.
5. Forward equation
With row probability vectors,
\[\frac{d\pi(t)}{dt}=\pi(t)Q.\]This is a Kolmogorov forward equation.