Distilling intractable generative models
A generative model’s partition function is typically expressed as an intractable multi-dimensional integral, whose approximation presents a challenge to numerical and Monte Carlo integration. In this work, we propose a new estimation method for intractable partition functions, based on distilling an intractable generative model into a tractable approximation thereof, and using the latter for proposing Monte Carlo samples. We empirically demonstrate that our method produces state-of-the-art estimates, even in combination with simple Monte Carlo methods.
See also: George’s MSc thesis.