Research

Update Sept 2018: I have now moved to Google and I am no longer taking on students at Edinburgh.

My research concerns a broad range of applications of probabilistic methods for machine learning, including software engineering, natural language processing, computer security, queueing theory, and sustainable energy. Although these applications are disparate, they are connected by an underlying statistical methodology in probabilistic modelling and techniques for approximate inference in graphical models.

My research strategy is based on the idea that sufficiently difficult applications motive the development of new methodology. I aim to develop new machine learning methods based on this interplay of theory and practice.

I am part of a large machine learning group at Edinburgh. Here is some information for prospective students in the group.

My position is funded through the Scottish Informatics and Computer Science Alliance.

Recent Publications

Please see my full list of publications, or my list of publications, sorted by topic.

Here are a few recent highlights:

  • Can Large Language Models Reason about Program Invariants?. Kexin Pei, David Bieber, Kensen Shi, Charles Sutton and Pengcheng Yin. In International Conference on Machine Learning. 2023.

    [ .pdf | bib | abstract ]

  • Natural Language to Code Generation in Interactive Data Science Notebooks. Pengcheng Yin, Wen-Ding Li, Kefan Xiao, Abhishek Rao, Yeming Wen, Kensen Shi, Joshua Howland, Paige Bailey, Michele Catasta, Henryk Michalewski, Alex Polozov and Charles Sutton. In Proceedings of the Association of Computational Linguistics (ACL). 2023.

    [ arXiv | bib | abstract | source code ]

  • A Survey of Machine Learning for Big Code and Naturalness. Miltiadis Allamanis, Earl T. Barr, Premkumar Devanbu and Charles Sutton. ACM Computing Surveys 51 (4). 2018.

    [ arXiv | bib ]

  • Autoencoding Variational Inference for Topic Models. Akash Srivastava and Charles Sutton. In International Conference on Learning Representations (ICLR). 2017.

    [ .pdf | arXiv | bib | discussion | source code ]

  • VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learning. Akash Srivastava, Lazar Valkov, Chris Russell, Michael Gutmann and Charles Sutton. In Advances in Neural Information Processing Systems (NIPS). 2017.

    [ .pdf | bib | abstract | code and data ]

  • An Introduction to Conditional Random Fields. Charles Sutton and Andrew McCallum. Foundations and Trends in Machine Learning 4 (4). 2012.

    [ .pdf | bib | abstract ]

Finally, I have a collection of brief, tutorial-style research notes (very old).

Research Group

I collaborate with a wonderful group of students and researchers who have, for whatever reason, chosen to go under the name CUP: Charles's Uncertain People. We have a weekly reading group, to which all are welcome.

A subgroup of CUP, called MAST (Machine learning for the Analysis of Source code Text), focuses on machine learning for software engineering and programming languages. Our software in this area is available via the MAST Github group.

Members of my research group

Current and former members of my group at the CUP group web site.

Projects

Some of my research projects have dedicated pages.

But not all of my research fits into one of these web sites. To get the whole story, read all of my papers!

Advisors, Mentors, Collaborators

  • My graduate advisor was Andrew McCallum at the University of Massachusetts Amherst.
  • I did a postdoc at the University of California, Berkeley working with Michael I. Jordan. I also collaborated with Dave Patterson, Randy Katz, Armando Fox, and Anthony Joseph in networking and systems. I participated in the RAD Lab, which focused on issues in the design and management of data center applications.
  • I worked as a intern at Microsoft Research with Tom Minka.
  • Other collaborators include Earl Barr (UCL), Zoubin Ghahramani (Cambridge), Max Welling (University of Amsterdam), Chris Pal (Ecole Polytechnique de Montréal), Khashayar Rohanimanesh (UMass), Yanlei Diao (Ecole Polytechnique), Prashant Shenoy (UMass), Hanna Wallach (Microsoft Research), Peter Bodik (Microsoft Research), Rob Hall (TripAdvisor), Michael Sindelar (Uber).

Personal

Hobbies: I live with cats and fish, who don't interact as much as you might think. I've played a few computer games, mostly adventure games and RPGs. I play Go (圍棋, 囲碁, 바둑). If you would like to know where to play Go in person, try the American Go Association or the British Go Association. I enjoy cooking.

When I was in university, I was a bit sillier than I am now, so I created a silly web site called al.oysi.us. The URL is easy to remember, because as I'm sure you're aware, Aloysius is my middle name. Warning: May not suitable for the silliness-challenged.

Does this page seem a bit boring? That's because you haven't cracked the Easter egg yet.