# What is the difference between RMSE and SEP

Cross Validated Asked by Tiago Dias on January 1, 2022

I would like to understand the difference between Root Mean Squared Error and the Standard Prediction Error.

The SEP formula is simillar to the RMSE, but with an aditional term called bias inside the root squared.

Is expected to give similar values for my model?

$$RMSE = sqrt {{sum_{i=1}^n(y_i -hat y_i)^2} over n}$$

$$SEP = sqrt {{sum_{i=1}^n(y_i -hat y_i-bias)^2} over {n-1}}$$

$$bias = {{sum_{i=1}^n(y_i -hat y_i)} over n}$$

SEP functions similarly to RMSE, but the bias term acts to adjust the mean of the predictions to match the mean of the actuals. That is, if you were to add a constant term to all of your predictions, you would degrade your RMSE would would not affect your SEP.

Another way to express SEP is:

$$SEP(y, hat{y}) equiv RMSE(y, hat{y} + bar{y} - bar{hat{y}})$$

I can think of a few cases where predicting the mean accurately is less relevant and you might prefer a metric like SEP:

• You care more about the rank order of your predictions than their absolute magnitude (in this case there are other metrics like normalized gini that are also useful).
• You care more about the relative difference between predictions than their absolute magnitude.
• You expect that the mean varies noisily and is hard to get right, though that it is still be possible to predict the differences accurately, hence you favour a metric that places less emphasis on accurately predicting the mean.
• Some other process over which you have no control will adjust the mean of your predictions later.
• Your particular holdout set has a large shift in mean versus your training set and you do not want to favour models that skew in that direction.

Answered by Dex Groves on January 1, 2022

## Related Questions

### Is there a word in statistics for “mean divided by absolute difference”?

0  Asked on December 1, 2021 by user989761

### SPSS – Automatic Linear Modeling “Importance” Numbers

1  Asked on December 1, 2021 by josh-davis

### Is the pooled AUC calculation for imputated data in (psfmi package) mivalext_lr() correct?

0  Asked on December 1, 2021 by yy-shi

### Am I okay in not using EC model when series are co-integrated?

1  Asked on December 1, 2021

### How does propensity score matching that uses only a small proportion of eligible patients affect generalizability?

1  Asked on December 1, 2021 by diana-petitti

### Logistic regression model predicts only one outcome, producing a high specificity but very low sensitivity. How do I improve the model?

1  Asked on November 29, 2021

### Why does the Lasso provide Variable Selection?

4  Asked on November 29, 2021 by zhi-zhao

### Why do increasing regularization weights make objective function not monotonically decrease?

1  Asked on November 29, 2021

### Do we need to demean and standardize all variables in a model?

1  Asked on November 29, 2021 by ama-perera

### linear causal model

1  Asked on November 29, 2021 by markowitz

### What is the point of test set in ML?

4  Asked on November 29, 2021 by lelouche-lamperouge

### Proof that Cov(W+Y, Y-V) = 0 given that W, Y, and V are uncorrelated but not independent

2  Asked on November 29, 2021 by user292024

### Can linear and logistic regression coefficients be combined using an inverse variance weighted average?

1  Asked on November 29, 2021

### How to construct one sided CI for Superiority Randomized Controlled Trial?

1  Asked on November 29, 2021 by user292068

### Working out expected steps of absorbent Markov Chain with more than one sink

0  Asked on November 29, 2021

### How do I calculate confidence level or interval?

0  Asked on November 29, 2021 by user810739

### Power of two-sample test of binomial proportions

1  Asked on November 29, 2021 by afternoon

### What is the most sound way to perform variable selection on an lmer() model?

1  Asked on November 29, 2021

### Comparing AUC and classification loss for binary outcome in LASSO cross validation

1  Asked on November 29, 2021 by atakan

### Examples of Simpson’s Paradox being resolved by choosing the aggregate data

4  Asked on November 29, 2021 by richie-cotton