Probabilistic Modelling Reading Group
There is also a page based form of this document
Introduction
Probability theory forms the cornerstone for many modelling methods. These
methods are used in a variety of fields and are continuously being developed.
A paper discussion group has been formed to aid the communication of novel
approaches and techniques within probabilistic modelling. The group will
be
-
Discussing new or formative papers related to probabilistic modelling.
-
Allowing the interaction of those using probabilistic methods in different
research fields.
-
Providing a forum for mutual questioning about technical and methodological
aspects of papers.
Resources
There are many Web resources available to probabilistic modellers. The
following few pointers might be helpful
Association for Uncertainty in Artificial
Intelligence
Probabilistic Methods
in AI Course
bayes.html:
An outline of Bayesian methods with some information on the
junction tree algorithm.
Jordan, M.I, Ghahramani, Z., Jaakkola, T.S., and Saul, L.K. (1998) An
introduction to
variational methods for graphical models.
This paper also has an introduction with
an outline of the junction tree method.
Books
For a good introduction to probabilistic graphical models see:
Castillo E., J. M. Gutierrez and A. S. Hadi (1997) Expert Systems and
Probabilistic Network Models. Springer.
Unfortunately Edinburgh University Library does not appear to stock
this book. It can be ordered from bookshops at 37.50 pounds sterling.
Meetings and Papers
The Probabilistic Modelling Reading Group meets fortnightly on fridays
at 12:30 pm in the C floor seminar room, 5 Forrest Hill. Papers read
at one meeting are proposed in the previous meeting. Please feel free
to suggest papers which might be worth discussing.
The past and present meetings are listed below (latest at bottom):
Friday 15 October
Geoffrey Hinton (1999) Products of
Experts. ICANN99.
Introduced by Paul Taylor
Friday 29 October
A presentation by William Chesters
Friday 12 November
Te-Won Lee, Mark Girolami, Anthony Bell and Terrence Sejnowski
A Unifying Information-theoretic Framework for Independent Component
Analysis.
Introduced by Stephen Felderhof
Friday 26 November
Dellaportas, Forster, and Ntzoufras On
Bayesian Model and Variable Sepection using MCMC
Introduced by Joe Frankel
Spring/Summer 1999
Previous meetings included
Friday 5 February
Boyen & Koller (1998), Tractable
inference for Complex Stochastic Processes
Proceedings of the Fourteenth Annual Conference on Uncertainty in AI,
33-42.
Introduced by Amos Storkey
Friday 19 February
Ghahramani, Z. and Jordan, M.I. (1997) Factorial
Hidden Markov Models , Machine Learning 29, 245-273.
Introduced by Will Lowe
Friday 5 March (NOTE MEETING TIME IS 1.00pm THIS WEEK)
J.F.G. de Freitas, M. Niranjan, A.H. Gee and A. Doucet (1998)
Sequential
Monte Carlo Methods for Optimisation of Neural Network Models. Technical
report CUED/F-INFENG/TR 328 ,
Cambridge University, Department of Engineering, July 1998.
Introduced by Matthias Seeger.
Friday 19 March
M. Ostendorf and V. Digalakis and O. Kimball (1996)
From
HMMs to Segment Models: A Unified View of Stochastic Modeling for Speech
Recognition
(Local
copy)
IEEE Trans. on Speech and Audio Processing 4, 360-378
Introduced by Simon King
Friday 2 April - This is Good Friday. No meeting this week.
Friday 16 April: Postponed to Friday 23 April
Friday 23 April
Learning multi-class dynamics,
Andrew Blake, Ben North and Michael Isard.
Advances in Neural Information Processing Systems 11, in press, MIT Press,
(1999).
Friday 7 May
An introduction to the junction tree algorithm: discussion.
Friday 21 May
David MacKay (1998) Introduction
to Monte Carlo Methods
(appears in Learning in Graphical Models, ed. M. I. Jordan, Kluwer 1998)
Introduced by Chris Williams
Friday 11 June
F. C. N. Pereira (1999) Speech
Recognition by Composition of Weighted Finite Automata.
Introduced by Paul Taylor
Friday 15 October
Geoffrey Hinton (1999) Products of
Experts. ICANN99.
Introduced by Paul Taylor
Friday 29 October
A presentation by William Chesters
Friday 12 November
Te-Won Lee, Mark Girolami, Anthony Bell and Terrence Sejnowski
A Unifying Information-theoretic Framework for Independent Component
Analysis.
Introduced by Stephen Felderhof
Friday 26 November
Dellaportas, Forster, and Ntzoufras On
Bayesian Model and Variable Sepection using MCMC
Introduced by Joe Frankel
Friday 10 December
Andreas Stolcke An
efficient probabilistic context-free parsing algorithm that computes
prefix probabilities, Computational Linguistics 21(2), 165-201.
Introduced by Chris Brew
2000
Friday 21 January
Note: This meeting will be held at 12:30 Faculty Room North, David
Hume Tower.
The first meeting of this term will involve a talk rather than a paper
discussion (non-PMRG members very welcome):
Sam Roweis from the Gatsby Computational Neuroscience Group at UCL, London
will be talking on
Constrained Hidden Markov Models for Sequence Modeling
By thinking of each state in a hidden Markov model as corresponding to
some spatial region of a fictitious "topology space" it is
possible to naturally define the neighbouring states of any state
as those which are connected in that space.
The transition matrix of the HMM can then be constrained to allow
transitions only between neighbours; this means that all valid state
sequences correspond to connected paths in the topology space.
This strong constraint makes structure discovery in sequences easier.
I show how such *constrained HMMs* can learn to discover underlying
structure in complex sequences of high dimensional data, and apply them
to the problem of recovering mouth movements from acoustic observations
in continuous speech and to learning character sequences in text.
Fiday 3 March
Applying Collins' Models for Categorial Grammars
Julia Hockenmaier will be introducing her work. She has suggested
discussing the paper
"Three Generative, Lexicalised Models for Statistical Parsing", by
Michael Collins;
which can be found at
http://xxx.lanl.gov/abs/cmp-lg/9706022
She will introduce this paper and say how it relates to her work.
Friday 12 May
Exact Sampling
P.J. Green and Duncan J. Murdoch (1998) Exact Sampling for Bayesian
Inference: Towards general purpose algorithms.
From this I hope we will get some overview of exact sampling. For those
who want more detail I have also provided the Propp and Wilson reference
here
(quite long).
New Term September 2000
Friday 8 September
The first meeting of term will involve looking at two papers.
The first is a short tutorial:
Adam Berger "A gentle introduction to iterative scaling"
available from
http://www.cs.cmu.edu/People/aberger/maxent.html
which will then lead on to wider discussion regarding
S. Della Pietra, V. Della Pietra, and J. Lafferty, Inducing features
of random fields, IEEE Transactions on Pattern Analysis and Machine
Intelligence, 19(4), April 1997, pp. 380-393.
available from http://www.cs.cmu.edu/~lafferty/pubs.html.
See also Lafferty's home page
...