Charles Sutton Lecturer (== US Assistant Professor) School of Informatics University of Edinburgh Publications Office: IF 3.26 Voice (W): +44 (0) 131 651 5634 Skype: casutton (calls forward to my mobile) Email: csutton@inf.ed.ac.uk

Recent Publications

My full list of publications is available. Or you might be interested in these recent highlights:

• Bayesian Inference in Queueing Networks. Charles Sutton, Michael I. Jordan. 2010. arXiv stat.ML/1001.3355

  [ .pdf | bib ]

  @Unpublished{	sutton10qnet,
  	title	= {Bayesian Inference in Queueing Networks},
  	author	= {Charles Sutton and Michael I. Jordan},
  	year	= {2010}
  }

• Piecewise Training for Structured Prediction. Charles Sutton, Andrew McCallum. Machine Learning 77 (2--3). 2009. (Train undirected graphical model by splitting into overlapping parts that are trained independently. Connections to pseudolikelihood and Bethe free energy. Journal version of UAI and ICML papers below.)

  [ .pdf | bib | abstract ]

  A drawback of structured prediction methods is that parameter estimation requires repeated inference, which is intractable for general structures. In this paper, we present an approximate training algorithm called piecewise training that divides the factors into tractable subgraphs, which we call pieces, that are trained independently. Piecewise training can be interpreted as approximating the exact likelihood using belief propagation, and different ways of making this interpretation yield different insights into the method. We also present an extension to piecewise training, called piecewise pseudolikelihood, designed for when variables have large cardinality. On several real-world NLP data sets, piecewise training performs superior to Besag's pseudolikelihood and sometimes comparably to exact maximum likelihood. In addition, PWPL performs similarly to piecewise and superior to standard pseudolikelihood, but is five to ten times more computationally efficient than batch maximum likelihood training.

  @Article{	sutton08piecewise,
  	number	= {2--3},
  	title	= {Piecewise Training for Structured Prediction},
  	author	= {Charles Sutton and Andrew McCallum},
  	journal	= {Machine Learning},
  	pages	= {165--194},
  	volume	= {77},
  	year	= {2009}
  }

• Probabilistic Inference in Queueing Networks. Charles Sutton, Michael I. Jordan. In Workshop on Tackling Computer Systems Problems with Machine Learning Techniques (SysML). 2008.

  [ .pdf | bib ]

  @InProceedings{	sutton08mm1,
  	title	= {Probabilistic Inference in Queueing Networks},
  	author	= {Charles Sutton and Michael I. Jordan},
  	booktitle	= {Workshop on Tackling Computer Systems Problems with Machine Learning Techniques (SysML)},
  	year	= {2008}
  }

• Unsupervised Deduplication using Cross-field Dependencies. Robert Hall, Charles Sutton, Andrew McCallum. In Conference on Knowledge Discovery and Data Mining (KDD). 2008. (Hierarchical DP model that jointly clusters citation venue strings based on both string-edit distance and title information.)

  [ .pdf | bib | abstract ]

  Recent work in deduplication has shown that collective deduplication of different attribute types can improve performance. But although these techniques cluster the attributes collectively, they do not model them collectively. For example, in citations in the research literature, canonical venue strings and title strings are dependent---because venues tend to focus on a few research areas---but this dependence is not modeled by current unsupervised techniques. We call this dependence between fields in a record a cross-field dependence. In this paper, we present an unsupervised generative model for the deduplication problem that explicitly models cross-field dependence. Our model uses a single set of latent variables to control two disparate clustering models: a Dirichlet-multinomial model over titles, and a non-exchangeable string-edit model over venues. We show that modeling cross-field dependence yields a substantial improvement in performance---a 58% reduction in error over a standard Dirichlet process mixture.

  @InProceedings{	hall08unsupervised,
  	title	= {Unsupervised Deduplication using Cross-field Dependencies},
  	author	= {Robert Hall and Charles Sutton and Andrew McCallum},
  	booktitle	= {Conference on Knowledge Discovery and Data Mining (KDD)},
  	year	= {2008}
  }

• An Introduction to Conditional Random Fields for Relational Learning. Charles Sutton, Andrew McCallum. In Lise Getoor and Ben Taskar, editors. Introduction to Statistical Relational Learning. MIT Press. 2007. (Detailed tutorial on conditional random fields. Includes motivation, background, mathematical foundations, linear-chain form, general-structure form, inference, parameter estimation, and tips and tricks. NOTE: In Equation (1.22), there is a small error. There should not be a summation over k in the final term, just lambda_k / sigma_2.)

  [ .pdf | bib ]

  @InCollection{	sutton07introduction,
  	title	= {An Introduction to Conditional Random Fields for Relational Learning},
  	author	= {Charles Sutton and Andrew McCallum},
  	editor	= {Lise Getoor and Ben Taskar},
  	publisher	= {MIT Press},
  	booktitle	= {Introduction to Statistical Relational Learning},
  	year	= {2007}
  }

Finally, I have a collection of brief, tutorial-style research notes.

Courses

• Spring 2010: Data Mining and Exploration

Advisors, Mentors, Collaborators

• My graduate advisor was Andrew McCallum at the University of Massachusetts Amherst.
• I did a postdoc at the University of California, Berkeley working with Michael I. Jordan. I also collaborated with Dave Patterson, Randy Katz, Armando Fox, and Anthony Joseph in networking and systems. I participated in the RAD Lab, which focuses on issues in the design and management of data center applications.
• I worked as a intern at Microsoft Research with Tom Minka.
• Other collaborators include Max Welling (UCI), Chris Pal (Rochester), Khashayar Rohanimanesh (UMass), Yanlei Diao (UMass), Prashant Shenoy (UMass), Hanna Wallach (UMass), Peter Bodik (graduate student; Berkeley), Rob Hall (graduate student; CMU), Michael Sindelar (graduate student; UMass).

News

• I am fundraising chair for AISTATS 2010.

Software I Write

• I have contributed to MALLET, a Java package for machine learning on text. MALLET includes implementations of several classification algorithms, conditional random fields, and latent Dirichlet allocation.
• I have also written GRMM, a Java toolkit for inference in graphical models with discrete variables. It includes facilities for easily constructing and training CRFs with arbitrary graphical structure. I have used it to produce results for several of my papers, including those on DCRFs and skip-chain CRFs.

Software I Use

Here is a list of low-cost and open-source software that I enjoy using.

Personal

• What do I have in common with a popular social bookmarking site? If this question intrigues you, then explore the web site of my non-work alter ego, al.oysi.us.
• Why doesn't this page have color? Because you haven't cracked the Easter egg yet.