RNADE: The real-valued neural autoregressive density-estimator

Benigno Uría, Iain Murray, and Hugo Larochelle.

We introduce RNADE, a new model for joint density estimation of real-valued vectors. Our model calculates the density of a datapoint as the product of one-dimensional conditionals modeled using mixture density networks with shared parameters. RNADE learns a distributed representation of the data, while having a tractable expression for the calculation of densities. A tractable likelihood allows direct comparison with other methods and training by standard gradient-based optimizers. We compare the performance of RNADE on several datasets of heterogeneous and perceptual data, finding it outperforms mixture models in all but one case.

Advances in Neural Information Processing Systems 26, 2013. [PDF, DjVu, GoogleViewer, arXiv, BibTeX]

See also: the original NADE for binary vectors, and a deep version.