Skip to content

Using the Hidden Markov Model to Classify the S&P 500: summary and key takeaways

The article fits a hidden Markov model to daily SPY volatility to label S&P 500 market regimes, first with two states and then with three. It then feeds a one-day-ahead volatility forecast from a DD-EWMA model into the HMM to project the next day's regime.

Johnson-Skinner, E. · 2021 · MediumSummary updated 20 November 2026Practitioner research
Published2021Web article
Asset focusEquitiesRegimes / HMM
TypeResearchRegimes & Safe Havens
Reading time3 minthis summary

Using the Hidden Markov Model to Classify the S&P 500 [web] at a glance

The question
Can a hidden Markov model fitted to volatility classify the S&P 500 into bullish and bearish regimes, and can the regime be forecast one day ahead?
Dataset & period
SPY ETF as a proxy for the S&P 500, daily data from Yahoo, starting 2017-01-01 and running to the article date (October 2021). Exact sample size: Not stated in the article.
Method
Daily volatility is defined as the absolute value of the log return. An HMM with Gaussian emissions is fitted by expectation maximization, with forward and backward recursions and Viterbi decoding of the state path. The author runs it with two states and with three states (bearish, neutral, bullish). For a forward view, a DD-EWMA model predicts next-day volatility, and that forecast is passed into the two-state HMM.

Key findings of Using the Hidden Markov Model to Classify the S&P 500 [web]

  • The two-state HMM, run on SPY data from 2017 onward, picks out the bearish or downward phases of the S&P 500 according to the author.
  • The three-state version labels bearish, neutral and bullish phases. The article presents this only as a chart, with no statistics.
  • The DD-EWMA volatility forecast uses a look-back of 30 days for alpha and reports an RMSE of 0.00301.
  • The one-day-ahead volatility forecast for the next trading day is 0.00807, which maps to state 1 (bearish) in the two-state HMM, so the regime is predicted to stay unchanged.
  • The author notes that the HMM can only classify the current state from current data. Forecasting a future state needs the added volatility forecast.

Robustness, as reported in the paper

  • The DD-EWMA forecast is described as consistent with ARMA and GARCH baseline models, but no comparison figures are given in the text.
  • No out-of-sample test of the regime labels, no sub-period analysis and no trading-cost analysis are reported.

Caveats and limits

  • The article is a blog post. Regime accuracy is judged by eye from charts, with no hit rate, no confusion measure and no backtest of a trading rule (inferred).
  • Volatility is taken as the absolute log return, a noisy single-day proxy, and the states are labelled bullish or bearish from that alone (inferred).
  • The only forward example is one day, where the state stays the same. The author says the method matters most at regime transitions, which are not shown.
  • Gaussian emissions are used, while the author notes that student-t distributions would be a better extension for financial data.

Items marked (inferred) are our reading of the design, not statements by the authors.

Why it matters

It shows a simple, reproducible way to turn volatility into discrete market regimes with an HMM. It also shows the known limit that HMM states describe the present, not the future. Readers should treat it as an illustration, not validated evidence.

This is a firm research note or web article, not a peer-reviewed paper.

Questions about this paper

How many regimes does the article use?

It fits a two-state HMM and a three-state HMM, the latter labelling bearish, neutral and bullish phases.

How does the article forecast the next regime?

It predicts next-day volatility with a DD-EWMA model (30-day look-back for alpha, RMSE 0.00301) and feeds the forecast of 0.00807 into the HMM, which returns the bearish state.

Does the article test a trading strategy?

No. It suggests the state forecast could be used to adjust algorithmic strategies a day ahead but reports no backtest.

Where can I read the original paper?

The original is at its publisher: see the citation on this page. We link to the paper and do not host the PDF.

Follow new quant research

Download MacroFinance market data as CSV or Excel, or pull it by API, and test the ideas in these papers yourself. New summaries are added as we review them.

Create a free account
Summaries are written by MacroFinance from the published papers, in our own words, for information only. They are not investment advice and may omit detail. Papers are not hosted here: each page links to the original at its publisher.