TransWikia.com

How to calculate log likelihood for a model which outputs log probability?

Cross Validated Asked by Joff on December 11, 2021

I am trying to compare some different models, some of them take samples and some of them make a single prediction. I have a reference here https://github.com/yaringal/ConcreteDropout/blob/master/concrete-dropout-keras.ipynb in the Eval Function box. I was using this, but I get some answers that don’t make sense and I cannot justify why.

The link above calculates log probability like this…

$$
frac{1}{N}[logsum_kexpBig(sum_n log p(y | x)Big) – log(K)]
$$

where $K$ is the number of samples taken, and you could view that as a division in exponential space or a subtraction in log space.

If I write this expression myself while I am coding I write it like this, but the answers come out different and as far as I can tell they are not equivalent expressions.

$$
frac{1}{N}sum_n[logsum_kexp(log p(y | x)) – log(K)]
$$

Which one is the correct way to write this?

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP