Primer
In a previous article, we utilized a very important assumption before we began using the concept of a random walk (which is an example of a Markov chain) to predict stock price movements;
The assumption here of course, is that the movement in a stocks price is random.
Be sure to check out this article to see how we used coin tosses to predict stock price movements by using a geometric random walk to yield surprising results.
Here we will carry on with that assumption which allows us to call upon the use of ideas from probability — namely we will explore how to predict stock market trends using the mathematical concept of Markov chains.
What is a Markov chain?
A Markov chain is a type of stochastic process. Therefore, to understand what a Markov chain is, we must first define what a stochastic process is.
A stochastic process is one where a random variable evolves over time. There are two ideas of time, the discrete and the continuous. Discrete time is countable, whilst the continuous time is not.
Enter the discrete-time stochastic process. It is a stochastic process in which the state of the system can be observed at discrete instants in time. In contrary the states of a continuous-time stochastic process can be observed at any instant in time. For simplification purposes, we will utilize the power of a discrete-time stochastic process for our Markov chain model.
Now, a discrete-time stochastic process is a Markov chain if, for t=0, 1, 2… and all states:
Essentially this means that a Markov chain is a stochastic process containing random variables transitioning from one state to another depending only on certain assumptions and definite probabilistic rules — having the Markov property.
The Markov property is a simple statement where we say: given the present, the future is independent of the past. It is a property belonging to a memoryless process as it is solely dependent on the current state and the randomness of transitioning to the next states. This property can be seen in the figure above on the left hand side of the equation where there is a conditional probability of what the future will be, given the outcome of the present time t.
Practically speaking, a discrete-time Markov chain deals with only discrete-time random variables. As we’ve stated, in our case we will use a discrete-time Markov model to predict market trends; to do so we must ask —
What is a Markov chain model?
A Markov chain model is a stochastic model that models random variables in such a way that the variables follow the Markov property. Basically the purpose of our model will be to predict the future state, the only requirement would be to know the current state.
There are three measures we need to be aware of so we may construct a Markov chain:
- States: All the states (occurrences) within the state-space ‘s’ of the dynamical system
- The initial state distribution: The initial probability distribution of the starting state. It is encoded into a column vector denoted as ‘q’
- State transition probabilities: transition probability of moving from one state to another. It is encoded into an s-by-s matrix denoted as ‘P’
In the matrix P, it is important to note that the rows denote the current state Xt and the columns denote the next state Xt+1. Xij denotes the conditional probability that Xt+1=j given the current state Xt=i. The probability of moving for state i to state j is outlined below:
The Stock Markets Movement as a Markov Chain Model
We want to model the stock markets trend. To do so we must first recall our assumption that a stocks market movement is random.
Therefore there is a dynamical system we want to examine — the stock markets trend.
We can identify that this system transitions randomly, between a set number of states.
All possible collections of all these states is called a state-space. For our case, we can identify that a stock markets movement can only take on three states (the state-space):
- Bull: the price is increasing
- Bear: the price is decreasing
- Stagnant: there is no change in price, neither increasing nor decreasing
Each of these states are unique occurrences. The total number of occurrences of a state is the frequency, and it is reflected by the number arrows pointing to it on a transition state diagram.
In our case we shall adopt the premise that stock market trends are independent of past events and only the current state can determine the future state.
Since the system contains states, is random, and satisfies Markov’s property — we may therefore model our system as a Markov chain. Additionally, we will choose to model in discrete-time.
Since we have three states we will henceforth have a three-state Markov chain.
Let us draft the transition state diagram,
The reason this is a draft is because we are yet to determine the probabilities of transition between each state. To do so we must use historical data to ascertain patterns and and thence find their estimated probabilities. As we have a three-state Markov chain, it follows that our state transition probabilities can be encoded in a three-by-three matrix.
Our goal is to find out the transition matrix P; then to complete the transition state diagram, so as to have a complete visual image of our model.
To serve our example, we will cut to the chase and rely on hypothetical data put together in the table below:
Let us encode it into a transition matrix P:
Now we can complete our transition state diagram, which will look like this:
Now the question that all but asks itself —
What now?
Now we have a Markov chain described by a state transition diagram and a transition matrix P.
The real gem of this Markov model is the transition matrix P.
The reason for this is that the matrix itself predicts the next time step. P² gives us the probability of two time steps in the future. P³ gives the probability of three time steps in the future, and so on. It is this constant self matrix multiplication that produces the future outcomes of the Markov model. However we must keep in mind that we ought to multiply it by the column vector q denoting the initial conditions as either bull, bear, or stagnant.
Let us specify some hypothetical data regarding the initial state probability distribution. Suppose we start at bull market conditions (at the bull state), this means that q=[1 0 0].
Then suppose we wanted to know the market trend 2 days from now. In effect we’re trying to find out the probability of a Time Series,(fig 7)
In conclusion we can deduce from the final 1-by-3 matrix that two days from today there is a higher chance of getting a bull market p=0.6745 compared to p=0.1690 for a bear market or p=0.1565 for a stagnant market.
Using Matlab to Determine The Series of Chains
To make things interesting we will simulate 100 days from now with the starting state as a bull market. This is very attainable if we are to use a computer program. Enter faithful Matlab, who will help us with our task. We will go on to iterate 100 times and create a chain of ones twos and threes which signify bull, bear, and stagnant; respectively.
After running 100 simulations we get the following chain:
We started at bull (1) and after 100 simulations we ended with bear (2) as the final state.
How about we ask the question, what happens if we increase the number of simulations? Observe:
For 10 Simulations
For 100 Simulations
For 1,000 Simulations
For 100,000 Simulations
For 1 million Simulations
Did you notice that as we increased the number of simulations we get an interesting phenomenon? A pattern perhaps? Starting from 1000 , 100,000 and 1 million simulations we can see that the bar charts look very similar, no? This is because as we increased the number of simulations, we saw lots of fluctuations in the frequency of the states but eventually they will stabilize to what is called a stationary distribution — ours being a bull market trend; hooray!
Conclusion
A Markov chain is a stochastic process containing random variables transitioning from one state to another which satisfy the Markov property which states that the future state is only dependent on the present state.
We can construct a model by knowing the state-space, initial probability distribution q, and the state transition probabilities P. To know a future outcome at time n away from now, we carry out the basic matrix multiplication: q*P^n
To figure out the stationary distribution which ultimately tells us where the probabilities effectively tilt to a certain state more than others, we run several simulations to create a long series chain. The frequency of states in a series chain is proportional to its number of connections in the state transition diagram. The P matrix plays a huge role here too.
Markov chains were initially discovered as a result of proving that even dependent outcomes follow a pattern. Historically it was believed that only independent outcomes follow a distribution. As we have seen, even Markov chains eventually stabilize to produce a stationary distribution.
Nowadays Markov chains are used in everything, from weather forecasting to predicting market movements and much much more.