Reinforcement Learning running Capstone Model Solution Fx generates errors

Generic
1

Running the model run(bars5m, rl_config) generates error’s like,



1/1 [==============================] - 0s 33ms/step

1/1 [==============================] - 0s 33ms/step

1/1 [==============================] - 0s 13ms/step



On the 12th run Trade 001 and after an hour it will generate only Trade 002, followed by a series of

1/1 [==============================] - 0s 13ms/step



The gameover is never set to True, stuck in this loop after second game.



William Porter

2

Hi William,



Can I ask if you are running the notebook on the portal or the local computer? Since the reinforcement learning model is resource-intensive, we always mention that the notebook should be run on the local system.

You can find the capstone notebook solution as a downloadable zip file in section 22, unit 7.



There are certain hyperparameters which can be modified to reduce computation time. The updated zip file has the hyperparameter "BATCH_SIZE" set to 1. Do let me know if this works.



 

3

Changing the "BATCH_SIZE" set to 1, does nothing. I still get l huge list of 

1/1 [==============================] - 0s 92ms/step

1/1 [==============================] - 0s 31ms/step

1/1 [==============================] - 0s 13ms/step

. . .

1/1 [==============================] - 0s 92ms/step

1/1 [==============================] - 0s 31ms/step

1/1 [==============================] - 0s 13ms/step

. . .

William

4

Yes the code is running on my local computer. This computer is built just for for this. I am running on Jupyter Lab, with 'test mode' set to False. I have used all of the code downloaded. This particular is being ran on Capstone Model Solution. and in adition i woulds prefer to run the downloaded program not on Fx_pair but on the SPY for which I have 12-years+ of five month data, that is clean. William

5

How do I send you my Capstone Model Solution.ipynb and the fx_pair_data_1.bz2 files that I ran. It has errors and I do not understand the errors.



William

6

how do I stop generating all of these blank lines. There are hundreds of them.

1/1 [==============================] - 0s 13ms/step
1/1 [==============================] - 0s 13ms/step
1/1 [==============================] - 0s 13ms/step
1/1 [==============================] - 0s 12ms/step
1/1 [==============================] - 0s 13ms/step
7

Q. On the 12th run Trade 001 and after an hour it will generate only Trade 002, followed by a series of 1/1 [==============================] - 0s 13ms/step



A. To speed up the execution, you can try reducing the number of iterations or setting the batch size to a lower number. For example, you can set the batch_size to 1 in rl_config.

 

    'LKBK': 30,
    'BATCH_SIZE': 1,
    'MAX_MEM': 600,  # You can change the maxmimum memory buffer here
    'EPSILON': 0.0001,


Q. i woulds prefer to run the downloaded program not on Fx_pair but on the SPY for which I have 12-years+ of five month data
A. To run the program on SPY data instead of Fx_pair data, you can change the dataset used in the program. Specifically, you can update the following line of code:
bars5m = pd.read_pickle(path + 'fx_pair_data_1.bz2')
to use your SPY data file instead:
bars5m = pd.read_pickle(path + <Name of your SPY data file>)

Note:
1. You need to change the method from read_pickle to read_csv if you are reading the data from a CSV file and so on.
2. You may also need to adjust the code to handle the different format of the SPY data, such as specifying the date column as the index and converting it to datetime using pd.to_datetime method.

Q. How do I send you my Capstone Model Solution.ipynb and the fx_pair_data_1.bz2 files that I ran. It has errors and I do not understand the errors.
A. You can upload the files to a cloud storage service such as Google Drive or Dropbox and share the link with us. Make sure to set the link to be publicly accessible so that we can access the files. 

Q. how do I stop generating all of these blank lines. There are hundreds of them. 1/1 [==============================] - 0s 13ms/step
A. The message "1/1 [==============================] - 0s 13ms/step" is a progress bar indicating the progress of the current iteration. You can suppress these messages by setting the verbose parameter to False when calling the predict method. For example, you can update the following lines of code and also at other places where predict method is called: 
targets[i] = modelR.predict(state_t)[0]
Q_sa = np.max(modelQ.predict(state_tp1)[0])


to:
  
targets[i] = modelR.predict(state_t, verbose=False)[0]
Q_sa = np.max(modelQ.predict(state_tp1, verbose=False)[0])


This should suppress the progress bar messages and only show the final result.

I hope this helps.