Deep reinforcement learning course

AI & Machine learning
1

Hi everyone,



Two queries:



I tried to use csv file instead of pickle files in the capstone project  but I failed. Does anyone try this types of files and succeed?



I wish to adapt the game environment in an openAI gym environment to test with different agents. Does anyone work already on this type of idea?



Best

Laurent

2

Hi Laurent,



Thanks for your question. And good to see that you have reached the capstone part of the course.



We have used pickle files to read the data as it is memory efficient. But the codes should work with CSV files too.



For example, the below pickle code can be replaced with a read_csv and you should be able to read the CSV file.

 

# Read the price data
bars5m = pd.read_pickle('../data_modules/fx_pair_data_1.bz2')

# Display the last 5 entries of price data
bars5m.tail()

CSV file code
  
# Read the price data
bars5m = pd.read_csv('../data_modules/fx_pair_data_1.csv')

Display the last 5 entries of price data


bars5m.tail()

We have also shared a model solution that you can refer to if you get stuck. This solution can be found at the following link: Quantra by QuantInsti | Courses on Algorithmic and Quantitative Trading



However, we strongly want you to succeed in completing the capstone project. Would you be able to share more details on the problem you are facing while working with CSV file such as the screenshot of the error message, a sample of the CSV file you are working with and place where you are getting the error or stuck.



With this information, we would be able to help you better.



On the OpenAI gym framework, we will pass the feedback to Dr Thomas Starke.



But generally, in the course, we have used a package independent approach to develop the codes for RL. In this way, you get a lot of flexibility in implementing the RL model as per your need. And by using a python package for RL implementation you would lose out on some flexibility. But I do understand that you are looking for help in implementing using OpenAI gym, So I believe the below links might be helpful to explore that package.

  1. RL with OpenAI gym: https://towardsdatascience.com/reinforcement-learning-with-openai-d445c2c687d2
  2. Get started with OpenAi Gym: https://blog.paperspace.com/getting-started-with-openai-gym/
I hope this helps.

Thanks,
Ishan

 
3

Dear Ishan,



Thanks for all your answers.



When I use csv files I got the error below

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_29112/358897055.py in <module>
      4 `False` in `rl_config`
      5 """
----> 6 run(bars5m, rl_config)

~\AppData\Local\Temp/ipykernel_29112/1387846044.py in run(bars5m, rl_config)
     18     }
     19 
---> 20     bars1h = bars5m.resample(
     21         '1H', label='right', closed='right').agg(ohlcv_dict).dropna()
     22     bars1d = bars1h.resample(

~\anaconda3\envs\quantra_py\lib\site-packages\pandas\core\generic.py in resample(self, rule, axis, closed, label, convention, kind, loffset, base, on, level, origin, offset)
   8367 
   8368         axis = self._get_axis_number(axis)
-> 8369         return get_resampler(
   8370             self,
   8371             freq=rule,

~\anaconda3\envs\quantra_py\lib\site-packages\pandas\core\resample.py in get_resampler(obj, kind, **kwds)
   1309     """
   1310     tg = TimeGrouper(**kwds)
-> 1311     return tg._get_resampler(obj, kind=kind)
   1312 
   1313 

~\anaconda3\envs\quantra_py\lib\site-packages\pandas\core\resample.py in _get_resampler(self, obj, kind)
   1474             return TimedeltaIndexResampler(obj, groupby=self, axis=self.axis)
   1475 
-> 1476         raise TypeError(
   1477             "Only valid with DatetimeIndex, "
   1478             "TimedeltaIndex or PeriodIndex, "

TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'RangeIndex'
4

Hey Laurent,



Thanks for sharing the error.



While resampling the data (changing from 5-minute to 1-hour frequency), the index needs to be of datetime. In a CSV file, the datatype of the column is not stored. Therefore, after the CSV file is read, you need to convert the index to datetime to solve the problem.



You can use the below code to fix the issue which you are facing.

 

import pandas as pd

# Read the price data
bars5m = pd.read_csv('../data_modules/fx_pair_data_1.csv')

# Convert index to datetime type
bars5m.index = pd.to_datetime(bars5m.index)

# Display the last 5 entries of price data
bars5m.tail()

I hope this helps. If you continue to face error then feel free to let us know and we would be very happy to help you.



Thanks!