How can I develop a reward function for penalizing the model for reaching below the defined percent drawdown?
For example:
I want to penalize the model if the drawdown has reached a 5% daily limit.
Hello Marcus, to do this, you need to add an additional condition in the backtester whenever the game is getting updated.
In the backtesting implementation notebook, in the run function, the code line 'reward, game_over = env.act(action)' updates the reward and game_over status.
You can add an addition code here (if-else) to check if env.pnl*100 is less than -5% and if it is, you can change the status of game_over to TRUE and reward to -5. If it is false, you can use env.act(action) to update the reward, game_over.
If you want to change the reward from the current pnl to other big numbers to penalize the drawdown, you can create a reward function that calculates the reward based on the current pnl and pass it to the game class. I.e. reward would be pnl if pnl is not less than -5% and a big number like -10000 if pnl is less than -5%.
Hope this helps!