Cross Validated Asked by Tiago Dias on January 1, 2022

I would like to understand the difference between Root Mean Squared Error and the Standard Prediction Error.

The SEP formula is simillar to the RMSE, but with an aditional term called bias inside the root squared.

Is expected to give similar values for my model?

$$RMSE = sqrt {{sum_{i=1}^n(y_i -hat y_i)^2} over n}$$

$$SEP = sqrt {{sum_{i=1}^n(y_i -hat y_i-bias)^2} over {n-1}}$$

$$bias = {{sum_{i=1}^n(y_i -hat y_i)} over n} $$

SEP functions similarly to RMSE, but the bias term acts to adjust the mean of the predictions to match the mean of the actuals. That is, if you were to add a constant term to all of your predictions, you would degrade your RMSE would would not affect your SEP.

Another way to express SEP is:

$$ SEP(y, hat{y}) equiv RMSE(y, hat{y} + bar{y} - bar{hat{y}}) $$

I can think of a few cases where predicting the mean accurately is less relevant and you might prefer a metric like SEP:

- You care more about the rank order of your predictions than their absolute magnitude (in this case there are other metrics like normalized gini that are also useful).
- You care more about the relative difference between predictions than their absolute magnitude.
- You expect that the mean varies noisily and is hard to get right, though that it is still be possible to predict the differences accurately, hence you favour a metric that places less emphasis on accurately predicting the mean.
- Some other process over which you have no control will adjust the mean of your predictions later.
- Your particular holdout set has a large shift in mean versus your training set and you do not want to favour models that skew in that direction.

Answered by Dex Groves on January 1, 2022

0 Asked on December 1, 2021 by user989761

1 Asked on December 1, 2021 by josh-davis

0 Asked on December 1, 2021 by yy-shi

1 Asked on December 1, 2021

1 Asked on December 1, 2021 by diana-petitti

1 Asked on November 29, 2021

dependent variable logistic multiple regression predictive models sensitivity specificity

4 Asked on November 29, 2021 by zhi-zhao

1 Asked on November 29, 2021

non negative matrix factorization optimization overfitting regularization

1 Asked on November 29, 2021 by ama-perera

4 Asked on November 29, 2021 by lelouche-lamperouge

2 Asked on November 29, 2021 by user292024

1 Asked on November 29, 2021

1 Asked on November 29, 2021 by user292068

0 Asked on November 29, 2021

0 Asked on November 29, 2021 by user810739

confidence interval prediction prediction interval predictive models

1 Asked on November 29, 2021 by afternoon

hypothesis testing normal distribution probability sample sample size

1 Asked on November 29, 2021

1 Asked on November 29, 2021 by atakan

4 Asked on November 29, 2021 by richie-cotton

Get help from others!

Recent Answers

- haakon.io on Why fry rice before boiling?
- Jon Church on Why fry rice before boiling?
- Peter Machado on Why fry rice before boiling?
- Joshua Engel on Why fry rice before boiling?
- Lex on Does Google Analytics track 404 page responses as valid page views?

Recent Questions

- How Do I Get The Ifruit App Off Of Gta 5 / Grand Theft Auto 5
- Iv’e designed a space elevator using a series of lasers. do you know anybody i could submit the designs too that could manufacture the concept and put it to use
- Need help finding a book. Female OP protagonist, magic
- Why is the WWF pending games (“Your turn”) area replaced w/ a column of “Bonus & Reward”gift boxes?
- Does Google Analytics track 404 page responses as valid page views?

© 2023 AnswerBun.com. All rights reserved. Sites we Love: PCI Database, MenuIva, UKBizDB, Menu Kuliner, Sharing RPP, SolveDir