Minor notes about variational methods

Choice of quantities in KL divergence

There are two choices using KL divergence given an optimal distribution \(Q^*\) and an approximating distribution \(Q\).
Since the KL divergence is asymetric, two choices can not be interchanged. For example, if we let \(g=dQ^{*}/dQ\), then
\[D_{KL}(Q^{*}||Q):=\int \log \frac{dQ^{*}}{dQ}dQ^{*}=-\int \left(\log \frac{dQ}{dQ^{*}}\right)\frac{dQ^{*}}{dQ}dQ=-\int \log \frac{dQ}{dQ^{*}}gdQ.\]

Q*||Q

\(D_{KL}(Q^{*}||Q)\) yields a convex optimization problem. It is suited for online computation.
Examples: Information-Theoretic Model Predictive Control (IT-MPC)

Q||Q*

\(D_{KL}(Q||Q^{*})\) results in a nonconvex optimization problem that can be solved by gradient methods.
Examples: Auto-Encoding Variational Bayes (VAE)