Distilling intractable generative models

George Papamakarios and Iain Murray.

A generative model’s partition function is typically expressed as an intractable multi-dimensional integral, whose approximation presents a challenge to numerical and Monte Carlo integration. In this work, we propose a new estimation method for intractable partition functions, based on distilling an intractable generative model into a tractable approximation thereof, and using the latter for proposing Monte Carlo samples. We empirically demonstrate that our method produces state-of-the-art estimates, even in combination with simple Monte Carlo methods.

Appeared in Probabilistic Integration Workshop at the Neural Information Processing Systems Conference, 2015. [PDF, DjVu, GoogleViewer, BibTeX, Code]

See also: George’s MSc thesis.