Mean-field variational inference is a method for approximate Bayesianposterior inference. It approximates a full posterior distribution with afactorized set of distributions by maximizing a lower bound on the marginallikelihood. This requires the ability to integrate a sum of terms in the logjoint likelihood using this factorized distribution. Often not all integralsare in closed form, which is typically handled by using a lower bound. Wepresent an alternative algorithm based on stochastic optimization that allowsfor direct optimization of the variational lower bound. This method usescontrol variates to reduce the variance of the stochastic search gradient, inwhich existing lower bounds can play an important role. We demonstrate theapproach on two non-conjugate models: logistic regression and an approximationto the HDP.
translated by 谷歌翻译