Probabilistic Modelling Reading Group

There is also a page based form of this document

Introduction

Probability theory forms the cornerstone for many modelling methods. These methods are used in a variety of fields and are continuously being developed. A paper discussion group has been formed to aid the communication of novel approaches and techniques within probabilistic modelling. The group will be

Resources

There are many Web resources available to probabilistic modellers. The following few pointers might be helpful

Association for Uncertainty in Artificial Intelligence

Probabilistic Methods in AI Course

bayes.html: An outline of Bayesian methods with some information on the junction tree algorithm.

Jordan, M.I, Ghahramani, Z., Jaakkola, T.S., and Saul, L.K. (1998) An introduction to variational methods for graphical models. This paper also has an introduction with an outline of the junction tree method.

Books

For a good introduction to probabilistic graphical models see:

Castillo E., J. M. Gutierrez and A. S. Hadi (1997) Expert Systems and Probabilistic Network Models. Springer.

Unfortunately Edinburgh University Library does not appear to stock this book. It can be ordered from bookshops at 37.50 pounds sterling.

Meetings and Papers

The Probabilistic Modelling Reading Group meets fortnightly on fridays at 12:30 pm in the C floor seminar room, 5 Forrest Hill. Papers read at one meeting are proposed in the previous meeting. Please feel free to suggest papers which might be worth discussing.

The past and present meetings are listed below (latest at bottom):

Friday 15 October

Geoffrey Hinton (1999) Products of Experts. ICANN99.

Introduced by Paul Taylor

Friday 29 October

A presentation by William Chesters

Friday 12 November

Te-Won Lee, Mark Girolami, Anthony Bell and Terrence Sejnowski A Unifying Information-theoretic Framework for Independent Component Analysis.

Introduced by Stephen Felderhof

Friday 26 November

Dellaportas, Forster, and Ntzoufras On Bayesian Model and Variable Sepection using MCMC

Introduced by Joe Frankel

Spring/Summer 1999

Previous meetings included

Friday 5 February

Boyen & Koller (1998), Tractable inference for Complex Stochastic Processes
Proceedings of the Fourteenth Annual Conference on Uncertainty in AI, 33-42.

Introduced by Amos Storkey

Friday 19 February

Ghahramani, Z. and Jordan, M.I. (1997)  Factorial Hidden Markov Models , Machine Learning 29, 245-273.

Introduced by Will Lowe

Friday 5 March (NOTE MEETING TIME IS 1.00pm THIS WEEK)

J.F.G. de Freitas, M. Niranjan, A.H. Gee and A. Doucet (1998)
Sequential Monte Carlo Methods for Optimisation of Neural Network Models. Technical report CUED/F-INFENG/TR 328 ,
Cambridge University, Department of Engineering, July 1998.

Introduced by Matthias Seeger.

Friday 19 March

M. Ostendorf and V. Digalakis and O. Kimball (1996)
 From HMMs to Segment Models: A Unified View of Stochastic Modeling for Speech Recognition (Local copy)
IEEE Trans. on Speech and Audio Processing 4, 360-378

Introduced by Simon King

Friday 2 April - This is Good Friday. No meeting this week.
Friday 16 April: Postponed to Friday 23 April

Friday 23 April

Learning multi-class dynamics, Andrew Blake, Ben North and Michael Isard. Advances in Neural Information Processing Systems 11, in press, MIT Press, (1999).

Friday 7 May

An introduction to the junction tree algorithm: discussion.

Friday 21 May

David MacKay (1998) Introduction to Monte Carlo Methods (appears in Learning in Graphical Models, ed. M. I. Jordan, Kluwer 1998)

Introduced by Chris Williams

Friday 11 June

F. C. N. Pereira (1999) Speech Recognition by Composition of Weighted Finite Automata.

Introduced by Paul Taylor

Friday 15 October

Geoffrey Hinton (1999) Products of Experts. ICANN99.

Introduced by Paul Taylor

Friday 29 October

A presentation by William Chesters

Friday 12 November

Te-Won Lee, Mark Girolami, Anthony Bell and Terrence Sejnowski A Unifying Information-theoretic Framework for Independent Component Analysis.

Introduced by Stephen Felderhof

Friday 26 November

Dellaportas, Forster, and Ntzoufras On Bayesian Model and Variable Sepection using MCMC

Introduced by Joe Frankel

Friday 10 December

Andreas Stolcke An efficient probabilistic context-free parsing algorithm that computes prefix probabilities, Computational Linguistics 21(2), 165-201.

Introduced by Chris Brew

2000

Friday 21 January

Note: This meeting will be held at 12:30 Faculty Room North, David Hume Tower.

The first meeting of this term will involve a talk rather than a paper discussion (non-PMRG members very welcome):

Sam Roweis from the Gatsby Computational Neuroscience Group at UCL, London will be talking on

Constrained Hidden Markov Models for Sequence Modeling

By thinking of each state in a hidden Markov model as corresponding to some spatial region of a fictitious "topology space" it is possible to naturally define the neighbouring states of any state as those which are connected in that space. The transition matrix of the HMM can then be constrained to allow transitions only between neighbours; this means that all valid state sequences correspond to connected paths in the topology space. This strong constraint makes structure discovery in sequences easier. I show how such *constrained HMMs* can learn to discover underlying structure in complex sequences of high dimensional data, and apply them to the problem of recovering mouth movements from acoustic observations in continuous speech and to learning character sequences in text.

Fiday 3 March

Applying Collins' Models for Categorial Grammars

Julia Hockenmaier will be introducing her work. She has suggested discussing the paper "Three Generative, Lexicalised Models for Statistical Parsing", by Michael Collins; which can be found at http://xxx.lanl.gov/abs/cmp-lg/9706022 She will introduce this paper and say how it relates to her work.

Friday 12 May

Exact Sampling

P.J. Green and Duncan J. Murdoch (1998) Exact Sampling for Bayesian Inference: Towards general purpose algorithms.

From this I hope we will get some overview of exact sampling. For those who want more detail I have also provided the Propp and Wilson reference here (quite long).

New Term September 2000

Friday 8 September

The first meeting of term will involve looking at two papers. The first is a short tutorial:

Adam Berger "A gentle introduction to iterative scaling" available from http://www.cs.cmu.edu/People/aberger/maxent.html

which will then lead on to wider discussion regarding

S. Della Pietra, V. Della Pietra, and J. Lafferty, Inducing features of random fields, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(4), April 1997, pp. 380-393. available from http://www.cs.cmu.edu/~lafferty/pubs.html. See also Lafferty's home page
...