Prediction Intervals (Conformal Predictions) for Regression Problems

Cross Validated Asked by bioinformatics_student on December 29, 2021

So I’ve been looking into the idea of looking into conformal predictions, or to obtain prediction intervals instead of just single point predictions. Basically my take is that I would have my deep learning model (NN) that predicts the residual for this to be useful. However by using some distance function for this it will only be as meaningful as it is correlated to the error, which begs the question, wouldn’t it better to predict the error directly with a separate model?

I came across this post Prediction Intervals for ML Models and thought it would be interesting to apply to my own models. In my case I am dealing with a ML regression model to predict the stability of my proteins and wish to define at the end in 5 steps or less.

  1. Data Splitting
  2. Training the Model
  3. Calibrating the model or measuring the skewness of the predictions (original labels vs predicted labels).

I was thinking of using a secondary model such as the KNN to measure the differences between labels.

  1. Obtaining prediction intervals
  2. Evaluating prediction intervals

Basically my question at the end of this is that would this be the correct way to do this or is this even smart to do? If so what would be the best approach for this?

Thanks for any input.

Possible Output

Sequence  Prediction Interval

AAADD     [0.34, 0.67]
AABBD     [0.24, 0.77]
AABCD     [0.36, 0.75]

enter image description here

Conformal Predictions Overview

Add your own answers!

Related Questions

ARIMA(0,0,0) model but residuals not white noise

0  Asked on November 30, 2020 by mathias-schinnerup-hejberg


Bayes Optimal Classification Rule

1  Asked on November 28, 2020 by john


Bounds for the expectation of the max

1  Asked on November 27, 2020 by user3285148


Cohen’s Kappa for more than two categories

1  Asked on November 25, 2020 by asra-khalid


Ask a Question

Get help from others!

© 2023 All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP