# Sampling uncertainty of posterior probability distribution

Cross Validated Asked by Milan Bosnic on January 5, 2022

I’m working on a problem with 3 possible outcomes and a bunch of features. I have a regression model that outputs probabilities for each category and I’d like to extend these probabilities to probability distributions. So instead of getting an output of [0.2,0.3,0.5], I’d get a probability density function or at least a quantification of the uncertainty of the prediction. I’ve looked into some models that give these distributions but haven’t found any way to use it on a multiple output problem. Is there any way to do this?

Interesting problem - which is most often overlooked in data science and machine learning. The output probabilities $$bf{y}$$ are indeed estimates of the underlying (true) posterior probabilities (your $$[0.2,0.3,0.5]$$). Sampling a different training set (from your presupposed 'oracle'), will yield a slightly different set of output probabilities, when the identical input feature vector $$bf{x}$$ is presented to the classifier.

The distributions of $$hat{P}(bf{y} mid bf{x},bf{theta})$$ - they have been studied for linear and quadratic discriminant analysis ($$theta$$ is the parameter vector of the discriminant classifier).

And yes, also the sufficient parameters of these distributions of $$hat{P}(bf{y} mid bf{x},bf{theta})$$ have been derived. Specifically the variance of each posterior probability has been derived. A mathematically sound description (with the relevant references to papers in the statistical literature), can be found in Chapter 11 in the book: Discriminant analysis and statistical pattern recognition by G.J. McLachlan, Wiley (2004).

Answered by Match Maker EE on January 5, 2022

## Related Questions

### Capacity of neural network with one hidden neuron?

1  Asked on November 24, 2021

### What does Y-axis of Normal Distribution’s plot denote?

2  Asked on November 24, 2021 by dmittal

### Confidence intervals and multiple regression for a multiply imputed data set

2  Asked on November 24, 2021 by appleseed

### Universal approximation theorem on limited precision arithmetic

0  Asked on November 24, 2021 by mrmartin

### Intercept interpretation in multi-level model when first-level predictor discrete

1  Asked on November 24, 2021

### Factors given by DoE can experimentally not be reached

0  Asked on November 24, 2021

### Why should we compare estimates of generalized linear model with its corresponding standard errors?

3  Asked on November 24, 2021

### Is it possible to view sequential independent trials as pre-determined with unknown outcome?

1  Asked on November 24, 2021 by jack-arthur

### Interpret coefficient of negative binomial regression

1  Asked on November 24, 2021

### Gaussian process smoothers (bs = “gp”) in GAMs

0  Asked on November 24, 2021 by doug-sponsler

### simulation of logistic regression sensitivity to prior probability: Brier score vs accuracy

0  Asked on November 24, 2021

### What is the best structure (Accuracy of the text extracted) for building an OCR? ATTENTION, CRNNN, DRAM,RAM, CTC based

0  Asked on November 24, 2021

### Nearest-neighbor returns different results based on coordinates chosen

0  Asked on November 24, 2021 by zhutchens1

### Counterexample where E(u|x)=0 in a regression model cannot hold in the population?

1  Asked on November 24, 2021

### report output GLMER and do contrasts

0  Asked on November 24, 2021 by chiara-toschi

### How determine the bandwidth of a gaussian kernel such that k nearest points represent a certain % of sum weight

1  Asked on November 24, 2021 by tzirtzi

### Comparing ISOMAP residual variance to PCA explained variance

1  Asked on November 21, 2021 by user3358740

### How to optimize Gaussian-process parameters for multiple tasks with GPML?

1  Asked on November 21, 2021 by scott-thibault

### For conjoint attribute importance calculation, should insignificant attribute levels be included in the calculation?

1  Asked on November 21, 2021 by arctan27

### Multilevel Poisson Regression

1  Asked on November 21, 2021