Machine Learning and Prediction of US Stock Index

AI & Machine learning
1

Background

Assuming that the historical daily-level SPX Ohlcv data is known, and we want to predict the price one month later, it is enough to know the probability of increase or decrease. It is better if we can predict the increase or decrease.



Question

1 Which algorithm should be chosen? and why?

  • Logistic Regression
  • Random Forests
  • Gradient Boosting Machines (GBM)
  • Support Vector Machines (SVM)
  • Recurrent Neural Networks (RNNs) / Long Short-Term Memory (LSTM)
2 What details should be paid attention to when using algorithm prediction?
3 Are there any other feasible ideas? For example: Will the historical OHLCV data combined with news improve accuracy?
4 Are there any related blogs recommended for reading?

Thanks
 
2

After the brief investigation, I found part of the result

1 LSTM with 3 - 5  years historical daily data is a good choice

2 Indicators, e.g. RSI, ADX, are suitable selections.



Further question, any article/course to learn more detail to reproduce the prediction?

 

 
3

Hello Rong,



While logistic regression and SVMs are effective for binary classification, they might struggle with predictions for time series data such as index/stock data. Similarly, random forests and GBMs have limitations with time series as they do not retain the memory of previous data points.



Recurrent Neural Networks (RNNs) and Long Short-Term Memory networks (LSTMs) are better suited for this problem because they can model temporal dependencies. LSTMs, in particular, are designed to learn from long-term patterns and are highly effective for time series predictions, such as predicting SPX price changes over a month.



Since you're predicting price movements over time and need to capture sequential patterns, LSTMs would be the most suitable choice. They can learn the dependencies in sequential OHLCV data better than tree-based models or logistic regression.



You can refer to the courses Neural Networks in Trading by Dr. Ernest Chan and Deep Reinforcement Learning in Trading by Dr. Thomas Starke to learn more about the applications of RNNs and LSTMs for predicting market trends. Please let us know if you have any further questions.

4

Cool? thanks for the valuable answer

5

I occasionally Sell SPX, RUT or other index’s Bull Put Spread strategy to earn premiums. Question: In the future, for example: 20 trading days (30 natural days), when the market falls sharply (for example: the decline is > 10%), this strategy will have a stop loss scenario, resulting in losses.
Expectation: Through random forest or LSTM, or other ML algorithms, combined with technical indicators such as RSI, MACD, SMA, and macroeconomic data, such as: Federal Reserve interest rates, CPI, non-agricultural employment data, etc. Train the model and give the probability of a sharp decline in the future. For example: if the probability of decline > the given threshold, notify the trader to stop opening BPS positions, danger ahead!
Can the above course( Neural Networks in Trading by Dr. Ernest Chan) answer my question? Or does the course only cover pure knowledge, and I need to solve the problem based on knowledge learned?
Thank you in advance for your reply.

6

Hello,

The course on Neural Networks will enable you to create an LSTM network to predict future prices. Thus, it might not directly answer your question but it will definitely help you move towards your objective. You can check the free preview to understand more.

I hope that helps.