Applications of Hidden Markov Chains

Given a Markov transition matrix and an invariant distribution on the states, we can impose a probability measure on the set of subshifts. For example, consider the Markov chain given on the left on the states $A,B_{1},B_{2}$, with invariant distribution $\pi =(2/7,4/7,1/7)$. If we "forget" the distinction between $B_{1},B_{2}$, we project this space of subshifts on $A,B_{1},B_{2}$ into another space of subshifts on $A,B$, and this projection also projects the probability measure down to a probability measure on the subshifts on $A,B$.

The hidden part of a hidden Markov model, whose observable states is non-Markovian.

The curious thing is that the probability measure on the subshifts on $A,B$ is not created by a Markov chain on $A,B$, not even multiple orders. Intuitively, this is because if one observes a long sequence of $B^{n}$, then one would become increasingly sure that the $Pr(A|B^{n})\to {\frac {2}{3}}$, meaning that the observable part of the system can be affected by something infinitely in the past.

Conversely, there exists a space of subshifts on 6 symbols, projected to subshifts on 2 symbols, such that any Markov measure on the smaller subshift has a preimage measure that is not Markov of any order (Example 2.6).