What is the acceptable value of MSE while training a RNN or LSTM model?
Does this value matter ? or we should only look at accuracy score for testing the validity of a model?
Example, is 2.4 an acceptable value of MSE?
MSE used in any models, regression or deep NN, is essentially an absolute measure - it is just the sqaured sum of y (dependent) vs y-hat (predicted) averaged over the observations. The root of it (RMSE) has the same unit as the y. As with any absolute measure, there is no threshold that is good or bad. If you are predicting stock price which trades around daily range of $20, an RMSE of $0.5 is impressive. If you are predicting penny stocks, that same value is quite useless. You can possibly make it dimension-less by dividing MSE of your model vs MSE of a trivial model (say that predicts y to be just the average of observed y, in which case this dimension-less number is exactly one minus the famous r-squred of linear regression. Note an average model as a base model may not make sense in all cases.)
So 2.4 is meaningless without knowing the variance of the dependent you are predicting. On the other hand mathematically MSE is a very nice function - symmetric, with well-defined grad and hess, so fits nicely in most optimization algorithms. What you want to use for training and testing/ validation is a business decision. If your business metrics is captured closely by MSE by all means use that to validate as well (keeping in mind it is absolute and not relative and hence not comparable acoss models with different y). On the other hand if accuracy is your business metrics, that should be used for validation.