Q-Table vs Rewards Table vs State-Action Table

Generic
1

Course Name: Deep Reinforcement Learning in Trading, Section No: 4, Unit No: 2, Unit type: Quiz



Question 1:

I don't understand what's the difference between "Q-Table" vs "Rewards Table" vs "State-Action Table". They all list the rewards for each action. May I know what's the difference?



Question 2: 

A Q-table shows us the difference in reward between our decision and the best decision to be taken

Answer:
Not correct. A Q-table is used for updating the perceived reward for an action and thus, cannot tell us the difference in reward between our decision and best decision.

Why?
If we can see all the perceived rewards for each action, then can't we decide what is the best action to take? e.g. take the action that has the highest perceived reward... (?)

Thank you for your help!!
richard
 
2

Hi Richard,



The State-Action table is commonly called as the Q-table. This table gives you the perceived (or expected) reward for different actions for different states. So the three components of this table are State, Action and Reward.



The reward table, on the other hand, is a lookup table for the actual rewards for a given State and Action.



With regard to the MCQ (Section 4, Unit 2), the second option does exactly what you say! Between option 2 and option 4, the sense on how to use the Q-table might be the same, but the statement is incorrect in option 4 which states that the Q-table contains the difference between the perceived reward and the best possible reward for any action. Hence the option 2 is the correct answer.



Hope this helped!

Please let me know if you need any more help!



Thanks,

Gaurav