Cross Validated Asked on December 27, 2021

Just wanted to ask whether the following derivation is correct:

Suppose $X$ is a vector of observed random variables, $Z$ is a vector of unobserved random variables and $theta$ is a vector of parameters. Let $S$ be the set of values $Z$ could take on; we’ll assume this is discrete. We’ll also assume the following are known

$f(x|z,theta)$ = the probability density function of X given Z and theta

$P(Z=z|theta)$ = the probability that $Z=z$ given $theta$

The likelihood function I want is

$prod_{zin S}[f(x|z,theta)P(Z=z|theta)]^{1_{Z=z}}$

Taking the logarithm:

$sum_{zin S}1_{Z=z}[log f(x|z,theta)+log P(Z=z|theta)]$

Now suppose I’m performing the EM algorithm and my current estimate of $theta$ is $theta^{(n)}$. Then I need to apply $E_{Z|theta^{(n)},x}$ to the above expression. This gives:

$sum_{zin S}P(Z=z|theta^{(n)})[log f(x|z,theta)+log P(Z=z|theta)]$

That is the expression I need to maximize with respect to $theta$ in the M step.

Is that correct, or did I make a mistake somethere? Thanks

Yes, your derivation looks correct. Just a minor clarification in the final expression -

In the expression, $sum_{zin S}P(Z=z|x,theta^{(n)})[log f(|z,theta)+log P(Z=z|theta)]$, the term $P(Z=z|x,theta^{(n)})$ would be computed in Expectation step; hence, during the maximization step, it is treated as constant value. In the M-step, derivatives are computed for only the terms within the bracket.

Answered by honeybadger on December 27, 2021

2 Asked on November 24, 2021 by dmittal

2 Asked on November 24, 2021 by appleseed

confidence interval missing data multiple imputation multiple regression regression

0 Asked on November 24, 2021 by mrmartin

1 Asked on November 24, 2021

mixed model multilevel analysis multiple regression regression coefficients repeated measures

0 Asked on November 24, 2021

3 Asked on November 24, 2021

generalized linear model regression regression coefficients standard error statistical significance

1 Asked on November 24, 2021 by jack-arthur

1 Asked on November 24, 2021

0 Asked on November 24, 2021 by doug-sponsler

0 Asked on November 24, 2021

0 Asked on November 24, 2021

computer vision neural networks optical character recognition pattern recognition

0 Asked on November 24, 2021 by zhutchens1

1 Asked on November 24, 2021

conditional expectation conditional probability econometrics expected value regression

0 Asked on November 24, 2021 by chiara-toschi

1 Asked on November 24, 2021 by tzirtzi

1 Asked on November 21, 2021 by user3358740

1 Asked on November 21, 2021 by scott-thibault

1 Asked on November 21, 2021 by arctan27

1 Asked on November 21, 2021

multilevel analysis multiple regression poisson regression regression repeated measures

Get help from others!

Recent Answers

- Joshua Engel on Why fry rice before boiling?
- Jon Church on Why fry rice before boiling?
- haakon.io on Why fry rice before boiling?
- Lex on Does Google Analytics track 404 page responses as valid page views?
- Peter Machado on Why fry rice before boiling?

Recent Questions

- How Do I Get The Ifruit App Off Of Gta 5 / Grand Theft Auto 5
- Iv’e designed a space elevator using a series of lasers. do you know anybody i could submit the designs too that could manufacture the concept and put it to use
- Need help finding a book. Female OP protagonist, magic
- Why is the WWF pending games (“Your turn”) area replaced w/ a column of “Bonus & Reward”gift boxes?
- Does Google Analytics track 404 page responses as valid page views?

© 2023 AnswerBun.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP