Publications by Iain Murray

See also lists on arXiv, Google scholar, Semantic scholar, Edinburgh research explorer, and DBLP.

Prospective students: Please read my page for prospective group members before emailing me.

Papers, published and unpublished

• Don’t recommend the obvious: Estimate probability ratios
  Roberto Pellegrini, Wenjie Zhao, Iain Murray
  RecSys, 2022.
  [ACM, Amazon Science]
• Maximum Likelihood Training of Score-Based Diffusion Models
  Yang Song*, Conor Durkan*, Iain Murray, Stefano Ermon
  * Equal contribution.
  Advances in Neural Information Processing Systems 34, 2021.
  [NeurIPS, arXiv]
• Regularising Fisher Information Improves Cross-lingual Generalisation
  Asa Cooper Stickland, Iain Murray
  1st Workshop on Multilingual Representation Learning, 2021.
  [ACL anthology]
• Lossless compression with state space models using bits back coding
  James Townsend and Iain Murray.
  Neural Compression Workshop, ICLR 2021.
  [arXiv]
• Density Deconvolution with Normalizing Flows
  Tim Dockhorn*, James A. Ritchie*, Yaoliang Yu and Iain Murray.
  * Equal contribution.
  Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models Workshop, ICML 2020.
  [arXiv]
• Ordering Dimensions with Nested Dropout Normalizing Flows
  Artur Bekasov and Iain Murray.
  Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models Workshop, ICML 2020.
  [arXiv, code]
• Diverse Ensembles Improve Calibration
  Asa Cooper Stickland, and Iain Murray.
  Uncertainty and Robustness in Deep Learning Workshop, ICML 2020.
  [arXiv]
• On Contrastive Learning for Likelihood-free Inference
  Conor Durkan, Iain Murray, and George Papamakarios.
  Proceedings of the 37th International Conference on Machine Learning (ICML), 2020.
  [PMLR, arXiv]
• Scalable Extreme Deconvolution
  James A. Ritchie, and Iain Murray.
  Machine Learning and the Physical Sciences Workshop, NeurIPS 2019.
  [arXiv]
• Neural Spline Flows
  Conor Durkan*, Artur Bekasov*, Iain Murray, and George Papamakarios.
  * Equal contribution.
  Advances in Neural Information Processing Systems 32, 2019.
  [NeurIPS, arXiv, our pytorch code, TensorFlow Probability's code, Available in Pyro]
• Cubic-Spline Flows
  Conor Durkan*, Artur Bekasov*, Iain Murray, and George Papamakarios.
  * Equal contribution.
  First workshop on Invertible Neural Networks and Normalizing Flows (ICML), 2019. Superseded by Neural Spline Flows.
  [available on arXiv]
• Dynamic Evaluation of Transformer Language Models
  Ben Krause, Emmanuel Kahembwe, Iain Murray, and Steve Renals.
  Research note, 2019.
  [available on arXiv, code]
• BERT and PALs: Projected Attention Layers for Efficient Adaptation in Multi-Task Learning
  Asa Cooper Stickland, and Iain Murray
  Proceedings of the 36th International Conference on Machine Learning (ICML), 2019.
  [PMLR, arXiv, code]
• Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows
  George Papamakarios, David C. Sterratt, and Iain Murray
  The Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 89–837–848, 2019.
  [PMLR page, arXiv, code]
• Mode Normalization
  Lucas Deecke, Iain Murray, and Hakan Bilen
  ICLR 2019.
  [available on OpenReview, code]
• Model Criticism in Latent Space
  Sohan Seth, Iain Murray, and Christopher K. I. Williams
  Bayesian Analysis, 14(3):703–725 2019.
  [Journal page, arXiv]
• Sequential Neural Methods for Likelihood-free Inference
  Conor Durkan, George Papamakarios, and Iain Murray
  NeurIPS Bayesian Deep Learning Workshop, 2018.
  Control experiments comparing a few recent methods.
  [available on arXiv]
• Bayesian Adversarial Spheres:
  Bayesian Inference and Adversarial Examples in a Noiseless Setting
  Artur Bekasov and Iain Murray
  NeurIPS Bayesian Deep Learning Workshop, 2018.
  [available on arXiv]
• Dynamic Evaluation of Neural Sequence Models
  Ben Krause, Emmanuel Kahembwe, Iain Murray, and Steve Renals.
  Proceedings of the 35th International Conference on Machine Learning (ICML), 2018.
  [Proceedings page, code]
• Masked Autoregressive Flow for Density Estimation
  George Papamakarios, Theo Pavlakou, and Iain Murray
  Advances in Neural Information Processing Systems 30, 2017.
  [Abstract, PDF, Supplementary, DjVu, GoogleViewer, arXiv, BibTeX, code]
• A determinant-free method to simulate the parameters of large Gaussian fields
  Louis Ellam, Heiko Strathmann, Mark Girolami, and Iain Murray.
  Stat, 6(1):271–281 2017. [Abstract, Journal Site, arXiv, BibTeX]
• Multiplicative LSTM for sequence modelling
  Ben Krause, Iain Murray, Steve Renals, and Liang Lu.
  Appeared in ICLR Workshop track, 2017. [Abstract, PDF, DjVu, GoogleViewer, arXiv, BibTeX, code]
• Markov Chain Truncation for Doubly-Intractable Inference
  Colin Wei, and Iain Murray.
  The Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 54–776-784, 2017. [Abstract, PDF, DjVu, GoogleViewer, arXiv, BibTeX]
• Aye or naw, whit dae ye hink? Scottish independence and linguistic identity on social media
  Philippa Shoemark, Debnil Sur, Luke Shrimpton, Iain Murray, and Sharon Goldwater.
  Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, 2017. [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• Fast ε-free Inference of Simulation Models with Bayesian Conditional Density Estimation
  George Papamakarios and Iain Murray
  Advances in Neural Information Processing Systems 29, 2016.
  [Abstract, PDF, DjVu, GoogleViewer, arXiv, BibTeX, code]
• Neural Autoregressive Distribution Estimation
  Benigno Uría, Marc-Alexandre Côté, Karol Gregor, Iain Murray, and Hugo Larochelle.
  Journal of Machine Learning Research 17(205):1−37, 2016. [Abstract, PDF, DjVu, GoogleViewer, arXiv, BibTeX]
• Differentiation of the Cholesky decomposition
  Iain Murray, 2016.
  TensorFlow, Theano, Autograd, MXNet, PyTorch, and possibly other packages include algorithms citing this note. [PDF, DjVu, GoogleViewer, arXiv, github, BibTeX]
  
  See also: differentiating celerite, Daniel Foreman-Mackey follows a similar process for the Cholesky factor in 1D Gaussian process models.
• Pseudo-Marginal Slice Sampling
  Iain Murray and Matthew M. Graham.
  Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, JMLR W&CP 9:541–548, 2016. [Abstract, PDF, DjVu, GoogleViewer, arXiv, BibTeX]
• Distilling intractable generative models
  George Papamakarios and Iain Murray
  Probabilistic Integration Workshop at the Neural Information Processing Systems Conference, 2015. [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• On the efficiency of recurrent neural network optimization algorithms
  Ben Krause, Liang Lu, Iain Murray and Steve Renals.
  OPT2015 Optimization for Machine Learning at the Neural Information Processing Systems Conference, 2015. [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• MADE: Masked Autoencoder for Distribution Estimation
  Mathieu Germain, Karol Gregor, Iain Murray and Hugo Larochelle.
  Proceedings of the 32nd International Conference on Machine Learning, JMLR W&CP 37:881–889, 2015.
  [Abstract, PDF, DjVu, GoogleViewer, Supplementary, arXiv, BibTeX]
• Modelling Acoustic-Feature Dependencies with Artificial Neural-Networks: Trajectory-RNADE
  Benigno Uría, Iain Murray, Steve Renals, Cassia Valentini-Botinhao and John Bridle.
  IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp4465–4469, 2015.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• Dissecting magnetar variability with Bayesian hierarchical models
  Daniela Huppenkothen et al.
  ApJ, 810, 66. 2015. [arXiv/1501.05251]
• A Deep and Tractable Density Estimator
  Benigno Uría, Iain Murray and Hugo Larochelle.
  Proceedings of the 31st International Conference on Machine Learning, JMLR W&CP 32(1):467–475, 2014.
  [Abstract and code, PDF, DjVu, GoogleViewer, arXiv, BibTeX]
• Parallel MCMC with Generalized Elliptical Slice Sampling
  Robert Nishihara, Iain Murray and Ryan P. Adams.
  The Journal of Machine Learning Research, 15(Jun):2087–2112 2014.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX, Code]
• RNADE: The real-valued neural autoregressive density-estimator
  Benigno Uría, Iain Murray and Hugo Larochelle.
  Advances in Neural Information Processing Systems 26, 2013.
  [Abstract and code, PDF, DjVu, GoogleViewer, arXiv, BibTeX]
• Attention as Reward-Driven Optimization of Sensory Processing
  Matthew Chalk, Iain Murray and Peggy Seriès.
  Neural Computation, 25(11):2904–2933, 2013. [NECO page]
• A Composable Strategy for Shredded Document Reconstruction
  Razvan Ranca and Iain Murray.
  Computer Analysis of Images and Patterns (CAIP), 2013. [LNCS page]
• Estimation Bias in Maximum Entropy Models
  Jakob H. Macke, Iain Murray and Peter E. Latham.
  Entropy 15(8):3109–3219, 2013. [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• A Framework for Evaluating Approximation Methods for Gaussian Process Regression
  Krzysztof Chalupka, Christopher K. I. Williams and Iain Murray.
  The Journal of Machine Learning Research, 14:333–350, 2013.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• Deep architectures for articulatory inversion
  Benigno Uría, Iain Murray, Steve Renals and Korin Richmond.
  The Proceedings of InterSpeech, 2012.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• Driving Markov chain Monte Carlo with a dependent random stream
  Iain Murray and Lloyd T. Elliott.
  I think this is a fun paper, but not that useful by itself as pseudo-random number generators work well in practice. A machine learning venue wanted more demos, and a stats venue wanted more rigour. As the work stands, it didn’t seem worth spending time on either, so I’ve just left it on arXiv (2012). [arXiv/1204.3187]
• How biased are maximum entropy models?
  Jakob H. Macke, Iain Murray and Peter E. Latham.
  Advances in Neural Information Processing Systems 24, 2011. [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• The Neural Autoregressive Distribution Estimator
  Hugo Larochelle and Iain Murray.
  The Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, JMLR W&CP 15:29–37, 2011.
  [Abstract and code, PDF, DjVu, GoogleViewer, BibTeX, Discussion]
  Notable paper award.
• Slice sampling covariance hyperparameters of latent Gaussian models
  Iain Murray and Ryan Prescott Adams.
  Advances in Neural Information Processing Systems 23, 2010.
  [Abstract and code, PDF, DjVu, GoogleViewer, arXiv, Related Poster, BibTeX, VideoLecture]
• Incorporating Side Information in Probabilistic Matrix Factorization with Gaussian Processes
  Ryan Prescott Adams, George E. Dahl and Iain Murray.
  Proceedings of the 26th Annual Conference on Uncertainty in Artificial Intelligence (UAI), 2010.
  [Abstract, PDF, DjVu, GoogleViewer, arXiv, BibTeX]
• Elliptical slice sampling
  Iain Murray, Ryan Prescott Adams and David J.C. MacKay.
  The Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, JMLR W&CP 9:541–548, 2010.
  [Abstract and code, PDF, DjVu, GoogleViewer, arXiv, Poster, BibTeX]
• Dynamical inference from a kinematic snapshot: the force law in the solar system
  Jo Bovy, Iain Murray and David W. Hogg.
  ApJ 711(2):1157–1167, 2010.
  [ApJ, arXiv/0903.5308, BibTeX]
• Tractable Nonparametric Bayesian Inference in Poisson Processes with Gaussian Process Intensities.
  Ryan Prescott Adams, Iain Murray and David J.C. MacKay.
  Proceedings of the 26th International Conference on Machine Learning (ICML), 2009.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
  
  ICML Best Student Paper Honourable Mention
• Evaluation Methods for Topic Models.
  Hanna M. Wallach, Iain Murray, Ruslan Salakhutdinov and David Mimno.
  Proceedings of the 26th International Conference on Machine Learning (ICML), 2009.
  [Abstract and code, PDF, DjVu, GoogleViewer, BibTeX]
• Gaussian Processes and Fast Matrix-Vector Multiplies.
  Iain Murray
  Presented at the Numerical Mathematics in Machine Learning workshop at the 26th International Conference on Machine Learning (ICML 2009), Montreal, Canada.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
  
  The talk, available online, covered slightly different material.
• Nonparametric Bayesian Density Modeling with Gaussian Processes.
  Ryan Prescott Adams, Iain Murray and David J.C. MacKay.
  A longer treatment of a project (below) that was first reported at Neural Information Processing Systems, 2009. [arXiv/0912.4896]
• Evaluating probabilities under high-dimensional latent variable models.
  Iain Murray and Ruslan Salakhutdinov.
  Advances in Neural Information Processing Systems 21, 2009.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• The Gaussian Process Density Sampler.
  Ryan Prescott Adams, Iain Murray and David J.C. MacKay.
  Advances in Neural Information Processing Systems 21, 2009.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• Characterizing response behavior in multisensory perception with conflicting cues.
  Rama Natarajan, Iain Murray, Ladan Shams and Richard Zemel.
  Advances in Neural Information Processing Systems 21, 2009.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• On the Quantitative Analysis of Deep Belief Networks.
  Ruslan Salakhutdinov and Iain Murray.
  Proceedings of the 25th International Conference on Machine Learning (ICML), 2008.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX, Code]
• MCMC for doubly-intractable distributions.
  Iain Murray, Zoubin Ghahramani, David J.C. MacKay.
  Proceedings of the 22nd Annual Conference on Uncertainty in Artificial Intelligence (UAI), 2006.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• Nested sampling for Potts Models.
  Iain Murray, David J.C. MacKay, Zoubin Ghahramani,
  John Skilling. Advances in Neural Information Processing Systems 18, 2006.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• A pragmatic Bayesian approach to predictive uncertainty
  Iain Murray and Edward Snelson.
  Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Textual Entailment.: First PASCAL Machine Learning Challenges Workshop. Springer Lecture Notes in Computer Science. 2006.
  [Abstract and PDF free from Springer, GoogleViewer, BibTeX]
• Bayesian Learning in Undirected Graphical Models: Approximate MCMC algorithms.
  Iain Murray and Zoubin Ghahramani.
  Proceedings of the 20th Annual Conference on Uncertainty in Artificial Intelligence (UAI), 2004.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX]

Contributed discussions

• Discussion on the paper: Catching up faster by switching sooner…, by Erven, Grünwald and Rooij
  Iain Murray.
  Journal of the Royal Statistical Society: Series B (Statistical Methodology), 74(3):409–411, 2012. [PDF, DjVu, GoogleViewer]
• Discussion on the paper: Riemann manifold Langevin and Hamiltonian Monte Carlo methods, by Girolami and Calderhead
  Iain Murray and Ryan Prescott Adams.
  Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(2):191–192, 2011. [Full paper, PDF, DjVu, GoogleViewer]
• Using TPA for Bayesian inference — Discussion
  Iain Murray.
  Bayesian Statistics 9, pp. 273–275, Oxford University Press, 2011. [PDF, DjVu, GoogleViewer]

PhD thesis

My thesis contains more work on nested sampling, doubly-intractable distributions and Markov chain Monte Carlo (MCMC) in general than in my earlier publications. The thesis received an honorable mention for the Savage award.

Advances in Markov chain Monte Carlo methods, Iain Murray, PhD thesis, Gatsby computational neuroscience unit, University College London, 2007.
[Abstract, PDF, DjVu, GoogleViewer, BibTeX]

Short Notes and Technical Reports

• A Bayesian approach to Observing Dark Worlds. Iain Murray, 2012.
  [HTML note]
• Smooth histograms from MCMC. Iain Murray, 2011.
  [HTML note]
• Notes on unknown uniform distributions in hierarchical probabilistic models. Iain Murray, 2009.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• Notes on the KL-divergence between a Markov chain and its equilibrium distribution.
  Iain Murray and Ruslan Salakhutdinov, 2008.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• A note on the evidence and Bayesian Occam’s razor.
  Iain Murray and Zoubin Ghahramani.
  Gatsby Unit Technical Report GCNU-TR 2005-003. August 2005.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX]
• Note on Rejection sampling and exact sampling with the Metropolised Independence Sampler.
  Iain Murray, 2004.
  [Abstract, PDF, DjVu, GoogleViewer, BibTeX]

See also materials on my teaching page.

My Erdős number is 3 through:
David J.C. MacKay → Robert James McEliece → Paul Erdős.