Sampling uncertainty of posterior probability distribution

Cross Validated Asked by Milan Bosnic on January 5, 2022

I’m working on a problem with 3 possible outcomes and a bunch of features. I have a regression model that outputs probabilities for each category and I’d like to extend these probabilities to probability distributions. So instead of getting an output of [0.2,0.3,0.5], I’d get a probability density function or at least a quantification of the uncertainty of the prediction. I’ve looked into some models that give these distributions but haven’t found any way to use it on a multiple output problem. Is there any way to do this?

One Answer

Interesting problem - which is most often overlooked in data science and machine learning. The output probabilities $bf{y}$ are indeed estimates of the underlying (true) posterior probabilities (your $[0.2,0.3,0.5]$). Sampling a different training set (from your presupposed 'oracle'), will yield a slightly different set of output probabilities, when the identical input feature vector $bf{x}$ is presented to the classifier.

The distributions of $hat{P}(bf{y} mid bf{x},bf{theta})$ - they have been studied for linear and quadratic discriminant analysis ($theta$ is the parameter vector of the discriminant classifier).

And yes, also the sufficient parameters of these distributions of $hat{P}(bf{y} mid bf{x},bf{theta})$ have been derived. Specifically the variance of each posterior probability has been derived. A mathematically sound description (with the relevant references to papers in the statistical literature), can be found in Chapter 11 in the book: Discriminant analysis and statistical pattern recognition by G.J. McLachlan, Wiley (2004).

Answered by Match Maker EE on January 5, 2022

Add your own answers!

Related Questions

report output GLMER and do contrasts

0  Asked on November 24, 2021 by chiara-toschi


Ask a Question

Get help from others!

© 2023 All rights reserved. Sites we Love: PCI Database, MenuIva, UKBizDB, Menu Kuliner, Sharing RPP, SolveDir