Stock index crash prediction using ML model

AI & Machine learning
1

I developed a RUT crash prediction system using the random forest model to guide trading and assist decision-making.

Feature engineering includes

1 Rut ohlcv price data, and some technical indicators, e.g. macd, rsi, etc.,

2 Macroeconomic data, such as: non-agricultural employment index, CPI, PPI, GDP, etc.

The system obtains daily RUT daily OHLCV data from Yahoo Finance, and gives the probability of crash >7% in the future 20 trading days (30 natural days) through statistical analysis.

On the second day, the code collects the data of the previous trading day and retrain the model. The prediction probability graph is as follows

On February 24, 2025, the system issued a warning signal on February 3, 2025, which means: there will be a >6% drop in the future 20 trading days from Feb 3.

On February 26, 2025, the system issued a warning signal on February 5, 2025, which means: there will be a >6% drop in the future 20 trading days from Feb 5.

On February 28, 2025, the system issued a warning signal on February 6, 2025, which means: there will be a >6% drop in the future 20 trading days from Feb 6.

One possible explanation is that when new price data enters the model, the model will be retrained and adjust the past historical predictions so that the results of the test set can be self-consistent.

Problem: The model’s behavior of modifying the historical signal date after new price data enters the model makes it impossible for traders to use the system accurately.
In other words, the model is more about explaining itself using new data instead of making predictions.

For example: When the trader is on February 24, 2025, the system issues a warning signal on February 3, 2025. At this time, the system prompts a decline, the decline has occurred, the short kinetic energy is released, and the trader opens a long position.

As a result, within a week, the model continuously modifies the date of the warning signal, which confuses the trader.

How to solve this problem?

Thanks

2025.2.24 crash prediction - signal date: 2025.2.3
2025.2.26 crash prediction - signal date: 2025.2.5
2025.2.28 crash prediction - signal date: 2025.2.6

2

Follow up about this question, Here is what I figured out, feel free to correct me or add more comments.

On February 24, 2025, the model said: Starting from February 3, 2025, a decline of >6% will occur in the next 20 trading days;
On February 26, 2025, the model said: Starting from February 5, 2025, a decline of >6% will occur in the next 20 trading days;
On February 28, 2025, the model said: Starting from February 6, 2025, a decline of >6% will occur in the next 20 trading days;

Knowing that the model will modify the historical signal per the latest incoming data, what is the right way to deal with it for trader? What should the trader do next time a similar situation occurs?
Answer: On February 24, 2025, the model predicted a decline, and the market(RUT) fell. At this time, the energy for the decline has been released. There is a high probability that there will not be a big drop in the future. Trader can long the market with a heavy position, but the trader must keep a safe margin.

1 Like
3

Hi Rong,

Great work on implementing a walk-forward optimization approach! A few thoughts:

  • Consider researching why tree-based models struggle with extrapolation, as this may affect crash prediction accuracy.
  • Did you apply the same dynamic retraining method during backtesting? If not, this could cause inconsistencies in real-world performance.
  • Your model keeps issuing crash signals until there’s a model-expected positive lift in features—this is expected.
    • for example:
      • On February 28, 2025, the model said: Starting from February 6, 2025, a decline of >6% will occur in the next 20 trading days; this signal will continue even after the Feb 24 when crash has already happened, unless it see good degree positive momentum.
    • It will provide start point of signal but fail at fading the signal. for reversal part you will need different model.

Improving the Trading Strategy
To improve accuracy, consider combining two models:

  • Long-Term Model → Detects potential crashes using macroeconomic + technical indicators.
  • Short-Term Model → Uses lagged crash signals + price action (e.g., multiple rainbow crossovers) to assess short-term trends (e.g., detecting dead cat bounces, losing steam, etc.).

As quant traders, we can further enhance accuracy by integrating inputs from these models + additional technical indicators, then developing a rule-based strategy on top.

Looking forward to your thoughts!

1 Like
4

Recently, as more facts have been clarified, the reasons have gradually become clear.

RUT 2024.11.25: 2466.49 → RUT 2025.3.14: 2010, a decline of approximately 18%.
In this case, the model only provides a self-consistent explanation in hindsight but essentially fails to offer any effective prediction,

The main reasons for the decline are:

  • Federal Reserve rate change out of expectation: Investors worry that the Fed will cut rates fewer times than expected.
  • Tariff tensions: Ongoing U.S. trade frictions have increased concerns about a slowdown in global economic growth.
  • Tech stock pullback: The decline in large technology companies’ stock prices has led to a reassessment of high-valuation stocks, impacting overall market sentiment.

These factors are largely driven by news and sentiment rather than historical prices, price indicators, or macroeconomic data.

Since the Machine Learning model is driven by technical Indicators and macro economic data, instead of news and sentiment indicators, it naturally fails to reflect corresponding predictions.

Thanks for the responded insights. As the next step, I will attempt to input news and sentiment indicators and observe the model’s future prediction performance.

1 Like
5

Hi Rong,

Great work! You might be able to identify sentiment shifts by analyzing changes in skewness across market returns. Try experimenting with different time periods—1 day, 3 days, a week, a month, and a year—and see if lagging the data provides additional insights.

If you’re comfortable sharing, I’d love to see the graphs, feature importance, or dendrogram related to this analysis.

Looking forward to your thoughts!