STOCHASTIC PROCESSES 05
The Markov Property
A Markov process has a state that contains all the information needed to describe the probability law of its future evolution.
The main idea
Once the current state is known, the earlier history provides no additional information for predicting the future within the model.
1. Discrete-time form
For a Markov chain,
\[P(X_{n+1}=j\mid X_n=i,X_{n-1},\ldots,X_0)=P(X_{n+1}=j\mid X_n=i).\]2. What memoryless means
It does not mean the process has no history. It means the current state summarises the relevant history for future transition probabilities.
3. State choice matters
A system may fail to be Markov if its state omits important information. Expanding the state can sometimes restore the Markov property.
4. Epidemic example
If infection and recovery rates depend only on the current numbers \((S,I,R)\), then those compartment counts can form a Markov state under suitable assumptions.
5. Time-homogeneous Markov processes
If transition probabilities depend on the time difference but not the absolute time, the process is time homogeneous.
6. Why the property is useful
It allows the process to be described through transitions from the current state rather than probabilities for every complete history.