Paul A. Crook


I've moved to the newly formed Interaction Lab part of the School of Mathematical and Computer Sciences (MACS) at Heriot-Watt University, Edinburgh. Employed as a Research Fellow I am working on applying POMDP models to Spoken Dialogue Systems under a new three year EPSRC grant. In this new work I am looking at synergies between symbolic and statistical representations in order to compactly represent and plan over large POMDP belief spaces.

Until recently I was employed as Research Associate in the School of Informatics in the University of Edinburgh working on a project to apply partially observable (POMDP) models of Reinforcement Leaning to statistical spoken dialogue systems. Previously, in 2007, I was working on a project looking into the application of machine learning and machine vision to the task of automatically labelling animal behaviours from video footage (iBehave project).

I was previously a member of the Institute of Perception Action and Behaviour (IPAB) (also part of the School of Informatics) where I completed my PhD thesis in 2006. My thesis examined learning in robots who could exert control over their own perceptions - see research interest below for details and links to my thesis.

My CV / Résumé can be found here (pdf document) and publications can be found here.


Research Interests

My main academic interests are focused on machine learning, especially reinforcement learning and leaning in Partially Observable Markov Decision Processes (POMDPs). I have worked on applying machine learning and machine vision to the interpretation of animal behaviour. More recently I have been looking into the application of machine learning techniques to multi-modal, especially spoken, human machine interfaces. I am also very interested in work on embodied, embedded and situated agents, especially those agents that attempt to deal with the real world, i.e. robots.

At a very general level I'm interested in problem solving by artificial agents embodied and situated in the real world. As an example I'd love to be involved in the development of a robotic squirrel which can learn to solve various physical puzzles to obtain food - like the real squirrels you see tackling problems on nature programmes. Unfortunately the field of A.I. is nowhere near achieving that apparent level of intelligence, not to mention such a project is an engineering feat way beyond one person, so I focus my interest on research questions whose answers will hopefully contribute towards my "embodied problem solving agent".

Link to Publications Page

PhD Research

My recent academic interest has focused on ideas about attention and its application to embodied and situated agents. My ideas on attention have been shaped by a 1989 paper by Alan Allport (in Foundations of Cognitive Science, pp631-682, MIT Press) suggesting that what we call attention might not be a single bottleneck or filter, but instead a set of separate functions each of which has evolved to provided a useful computational process. My MSc work looked at detection of novelty (see publications page), arguably one of the functions of attention.

My PhD work looked at whether an agent can learn to deploy its attention in order to differentiate between states which can appear identical in order to successfully learn a given task. I focused on active perception, that is the control of an agent's sensory input by the agent, which I regard as a function of an attentional process. In this work I demonstrate the benefits provided by active perception to simple reactive agents who are learn through reinforcement (i.e using reinforcement learning techniques). Feel free to browse my publications page.

The finished version of my PhD thesis can be found in the Edinburgh Research Archive at This is a persistent URL which should always resolve to my thesis, i.e. you can safely cite it. Slides from a talk that I gave at the IDSIA institute (in Switzerland) give a summary of my PhD work.

The Reinforcement Learning - Java Test Platform (RL-JTP) that I developed as part of my PhD is available to download. Documentation and download details can be found here.

Science Festival / Robot Demos

Over the years have been involved in demonstrating the University's robots at the annual Edinburgh Science Festival which is always great fun. Worked on designing, building and programming many of the demos that I ran. A recent creation is is a Connect 4 playing robot arm.

Any thoughts or suggestions or cool demo projects that you'd like some help with, please feel free to email me:

Other interesting links: