Contents || Introduction || Background || Research || Group || Publications|| Demonstrations || Teaching || Interests || Links || Contact || email
This gives an overview of the different areas of research. In some cases more details can be obtained by selecting the releavnt links.
The challenge for the scaling of Machine Learning methods is not primarily the challenge of Big Data, although large amounts of data will inevitably be used. Rather it is the challenge of moving from a monolithic, centrally designed machine learning process, that typically makes a number of assumptions of data uniformity, to a large scale process that can handle disparate knowledge, information and data. This process will need to integrate data from different sources, enable particular models and methods to perform analyses on subparts of the problem and then integrate those analyses. This needs to be done in an extensible, seamless way that can react to the needs and demands for particular analysis.
This transition feels very much like the difference between a large scale planned economy, and a market economy where local decision are made and integrated via trade. At the same time there is a clear theoretical correspondence between the predictions and components in a machine learning system and agents acting in information markets. We argue that information markets can provide the incentivisation mechanism for a distributed machine learning, where the systems designer has only to design the market that allows disparate agents with different knowledge bases to trade dependent on that knowledge. The result of the subsequent information market is equivalent to a machine learning model that aggregates the broader information to provide inferential predictions or estimated.
This project examines both the links between machine learning and information markets and the methods of utilising information markets for machine learning goals.
What does neural dynamics actually do? One interesting hypothesis is that neural dynamics effectively implements a sampling method for representing a posterior distribution implemented by the impact of sensory data a neural system. This hypothesis has many levels, from the representation form of a neural system, to the sampling process the neural system actually implements. We believe that knowledge and understanding of neural processes as sampling will shed some light on what happens in real neural systems, but will also introduce improved sampling methods to the probabilistic modelling community.
This project is about continuous time stochastic inference and learning (CTSIL). CTSIL is useful in continuously evolving environments that need to be modelled to fit with particular observations. Examples of this include weather systems where a stochastic model of the development of the local weather must explain the observed meteorological recordings well, and neural systems where models for neural spike patterns need to match intracellular recordings. Using CTSIL will provide models that can predict future data better and enable more accurate scientific explanations. In general, the capability of doing inference in continuous time stochastic systems reaps benefits in situations where:
By moving from simulation to continuous time stochastic inference, it is possible to perform rigourous parameter and model estimation that produces better models than matching limited simulation runs with observational statistics. By using stochastic instead of deterministic systems, the poor fit that results from trying to model a stochastic signal by a limited deterministic model is avoided. Consider a trivial example: fitting a deterministic oscillator (i.e. sinusoidal signal) to observations of a \emph{noisy} oscillator. Due to the various phase shifts that the noisy oscillator produces the fit of the deterministic oscillator will be poor, even if the parameters are exactly correct.
Continuous-time rather than discrete-time inference results in greater accuracy, especially in non-linear systems where the dynamics of the discrete iterated system can poorly match the underlying nonlinear dynamical system as the underlying attractors, limit cycles and chaotic dynamics will be potentially be different. In dynamic rather than static models, temporal constraints on causality can be used: even limited information from time-indexed signals can help establish causal relationships between variables that are indistinguishable in static models due to the large equivalence classes.
More recently I have been working in this area. Watch this space!