Sampling uncertainty of posterior probability distribution

Cross Validated Asked by Milan Bosnic on January 5, 2022

I’m working on a problem with 3 possible outcomes and a bunch of features. I have a regression model that outputs probabilities for each category and I’d like to extend these probabilities to probability distributions. So instead of getting an output of [0.2,0.3,0.5], I’d get a probability density function or at least a quantification of the uncertainty of the prediction. I’ve looked into some models that give these distributions but haven’t found any way to use it on a multiple output problem. Is there any way to do this?

One Answer

Interesting problem - which is most often overlooked in data science and machine learning. The output probabilities $bf{y}$ are indeed estimates of the underlying (true) posterior probabilities (your $[0.2,0.3,0.5]$). Sampling a different training set (from your presupposed 'oracle'), will yield a slightly different set of output probabilities, when the identical input feature vector $bf{x}$ is presented to the classifier.

The distributions of $hat{P}(bf{y} mid bf{x},bf{theta})$ - they have been studied for linear and quadratic discriminant analysis ($theta$ is the parameter vector of the discriminant classifier).

And yes, also the sufficient parameters of these distributions of $hat{P}(bf{y} mid bf{x},bf{theta})$ have been derived. Specifically the variance of each posterior probability has been derived. A mathematically sound description (with the relevant references to papers in the statistical literature), can be found in Chapter 11 in the book: Discriminant analysis and statistical pattern recognition by G.J. McLachlan, Wiley (2004).

Answered by Match Maker EE on January 5, 2022

Add your own answers!

Related Questions

How to calculate ARIMA(1,0,0)(1,0,1)12 prediction by hand

1  Asked on December 10, 2021 by code_diy


ANOVA determining percentage of variation

2  Asked on December 8, 2021 by unistudent87


what is Multivariate Data

2  Asked on December 8, 2021


Ask a Question

Get help from others!

© 2023 All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP