Course Name: Deep Reinforcement Learning in Trading, Section No: 18, Unit No: 14, Unit type: Notebook
What is the difference between the notebooks in Unit No 13 and in Unit No 14? I can't find any in the code except the file names in rl_config.
The reason I am asking is: I can't reproduce the result in the notebook in Section No: 18, Unit No: 14 on my local computer after turning off TEST_MODE. The model just doesn't converge, even the notebook is using synthetic market data.
Hello Bo Peng,
We really appreciate you for bringing this to our notice.
Initially, there were two separate notebooks under Section 18 - Units 13 and 14.
In the first notebook, we applied reinforcement learning over a synthetic sine wave pattern (Unit 13).
And in the following notebook, we applied reinforcement learning to a synthetic trending pattern (Unit 14).
But later on, we decided to try out a slightly complex, yet interesting approach.
Hence, we created a notebook that walks you through steps to apply to reinforcement learning to mixed wave patterns. This includes both the synthetic sine wave, as well as the trending pattern within a single notebook.
Thus, we have now removed the duplicate notebook from this section to avoid any confusion.
Now, coming to your second query, which is regarding difficulty faced in reproducing the results:
Reinforcement learning models explore the patterns in the data and often perform a completely unique set of actions for each time that you run it. This means, that there will always be an element of randomness that is associated with what the model would have learnt and hence, the model will come up with unique results for the most part.
I hope this answers your question as to why you were facing difficulty in reproducing the exact same results on your local computer. Also, do let us know if you face any difficulties in the future.
Thanks Kevin. Yeah that makes sense. I can see now the result in the course notebook is similar to my result
If you don't mind me asking: what kind of hardware do you guys run the notebook on? I have a 5 year old Windows PC and 2 days later it has done only 3 years of data.
Hi Bo Peng,
The notebook was run on a system that had the following specifications:
- i5 processor
- 32 GB RAM
- 2 GB Graphic Card
- 160 GB SSD
Even with the specs mentioned above, the notebook needed to be kept running overnight.
If you're looking to reduce the time for training the RL models, using systems with a better configuration will certainly help.
I hope this answers your question.