# What is the difference between RMSE and SEP

Cross Validated Asked by Tiago Dias on January 1, 2022

I would like to understand the difference between Root Mean Squared Error and the Standard Prediction Error.

The SEP formula is simillar to the RMSE, but with an aditional term called bias inside the root squared.

Is expected to give similar values for my model?

$$RMSE = sqrt {{sum_{i=1}^n(y_i -hat y_i)^2} over n}$$

$$SEP = sqrt {{sum_{i=1}^n(y_i -hat y_i-bias)^2} over {n-1}}$$

$$bias = {{sum_{i=1}^n(y_i -hat y_i)} over n}$$

SEP functions similarly to RMSE, but the bias term acts to adjust the mean of the predictions to match the mean of the actuals. That is, if you were to add a constant term to all of your predictions, you would degrade your RMSE would would not affect your SEP.

Another way to express SEP is:

$$SEP(y, hat{y}) equiv RMSE(y, hat{y} + bar{y} - bar{hat{y}})$$

I can think of a few cases where predicting the mean accurately is less relevant and you might prefer a metric like SEP:

• You care more about the rank order of your predictions than their absolute magnitude (in this case there are other metrics like normalized gini that are also useful).
• You care more about the relative difference between predictions than their absolute magnitude.
• You expect that the mean varies noisily and is hard to get right, though that it is still be possible to predict the differences accurately, hence you favour a metric that places less emphasis on accurately predicting the mean.
• Some other process over which you have no control will adjust the mean of your predictions later.
• Your particular holdout set has a large shift in mean versus your training set and you do not want to favour models that skew in that direction.

Answered by Dex Groves on January 1, 2022

## Related Questions

### How to fit a piecewise assembly of nonlinear functions?

1  Asked on November 6, 2021

### Linear model what is $p(x|y_0)$

1  Asked on November 6, 2021 by chasmani

### Bayes Estimator for Bernoulli Variance

3  Asked on November 6, 2021 by probability-stats-optimisation

### “Dominance” condition for consistency of MLE

0  Asked on November 6, 2021

### Can I use matching weights to check that treatment endogeneity is eliminated after exact matching?

2  Asked on November 6, 2021 by stefano-testoni

### Number of MC Simulations in Multivariate Model with Copulas

0  Asked on November 6, 2021

### Permutation testing for significance of a predictor

0  Asked on November 6, 2021

### How to estimate cut off percentiles to classify cost per metric?

1  Asked on November 6, 2021 by keith-siopes

### Are there convenient methods/tricks to make calculations with non-independent terms? (two examples here in particular)

0  Asked on November 6, 2021

### Are segments painted randomly respective to previously painted segments?

0  Asked on November 6, 2021

### What could be the reasons that making validation loss jumping up and down?

1  Asked on November 2, 2021 by haitao-du

### Between-subject design and within-subject anlyses

1  Asked on November 2, 2021 by giorgio-p

### Likelihood function when there is no common dominating measure?

1  Asked on November 2, 2021 by kjetil-b-halvorsen

### Are these the major response variable types?

0  Asked on November 2, 2021 by chris-science

### Exploding probability under simple hierarchical Bayesian formulation

1  Asked on November 2, 2021

### mixed model variance-covariance matrix| parameter estimation

1  Asked on November 2, 2021 by hedayat

### Looking to build the Mathematical proof that $Var(hat{y}) = sigma^2textbf{H}$

2  Asked on November 2, 2021 by seve-martinez

### How do I calculate confidence intervals on an elastic net regression in R

1  Asked on November 2, 2021 by alberto-pascale

### Is there an intuition behind the formula of chi-square?

1  Asked on November 2, 2021

### Finding C for a PMF of a frequency distribution

1  Asked on November 2, 2021