Sampling uncertainty of posterior probability distribution

Interesting problem - which is most often overlooked in data science and machine learning. The output probabilities $bf{y}$ are indeed estimates of the underlying (true) posterior probabilities (your $[0.2,0.3,0.5]$). Sampling a different training set (from your presupposed 'oracle'), will yield a slightly different set of output probabilities, when the identical input feature vector $bf{x}$ is presented to the classifier.

The distributions of $hat{P}(bf{y} mid bf{x},bf{theta})$ - they have been studied for linear and quadratic discriminant analysis ($theta$ is the parameter vector of the discriminant classifier).

And yes, also the sufficient parameters of these distributions of $hat{P}(bf{y} mid bf{x},bf{theta})$ have been derived. Specifically the variance of each posterior probability has been derived. A mathematically sound description (with the relevant references to papers in the statistical literature), can be found in Chapter 11 in the book: Discriminant analysis and statistical pattern recognition by G.J. McLachlan, Wiley (2004).