First International Workshop on Intelligence and Multimodality
in Multimedia Interfaces: Research and Applications

Human Communication Research Centre
EdCAAD, Dept. of Architecture

University of Edinburgh
Edinburgh, Scotland
Thursday 13th - Friday 14th July 1995

IMMI-1 Workshop Programme

Programme Summary

Thursday 13th July 1995

0900-0930 Registration & Coffee
0930-1100 Session 1: Language and Learning
1100-1130 Break
1130-1300 Session 2: Speech and Retrieval
1300-1400 Lunch
1400-1530 Session 3: Intelligence in Presentation
1530-1600 Break
1600-1730 Session 4: Software and Interaction Design

Friday 14th July 1995
0900-0930 Coffee
0930-1100 Session 5: The Relevance of Formal Theory
1100-1130 Break
1130-1300 Session 6: Cognitive Design and Evaluation 1
1300-1400 Lunch
1400-1530 Session 7: Cognitive Design and Evaluation 2
1530-1600 Break
1600-~1630 Wrap-up session


Programme Details

The intention is that in each session there are about four presentations, which should be short and focussed on issues that it would be useful to discuss. Some suggestions for issues are given below. In assigning papers to the session groups, there is the inevitable difficulty that many (or even most) of the papers would fit into more than one group. However, since all the sessions are plenary, there is plenty of scope for people to involve themselves in discussion wherever they have a contribution to make. Each session is 90 mins, where the idea is that presentations should take up 40-60 minutes -- no more than that, i.e. 10-15 mins each -- allowing at least 30 mins for discussion. In the few cases where papers have been accepted but the author is unable to attend the workshop, the papers will be in any case included in the pre-proceedings to inform discussion in the relevant sessions (such cases are denoted below by an asterisk [*]).

Session 1: Language and Learning.

(Back to Summary)

The papers in this group begin with Natural Language based multimodal interfaces, then shift topic via explanation to multimedia in learning. Pursuit of the relation between language and other information resources will be a common theme through many of the sessions.

Reference interpretation in a multimodal environment combining speech and gesture
Nadia Bellalem & Laurent Romary
CRIN-CNRS & INRIA Lorraine, France.

Producing coordinated natural language and graphical explanations in the context of a geometric problem-solving task
Sergio Santana & Luis A Pineda
IIE, Cuernavaca, Mexico.

Natural language interaction as a vehicle for learning in a hypermedia environment
Gavin Long, Mark Edwards, Heather Powell, & Dominic Palmer-Brown.
Nottingham Trent University, England, UK.

Providing educational resources through the internet
Arthur C Huntley
University of California, Davis, CA, USA.

Dimensions of learner control: A reappraisal of interactive multimedia instruction.
* R Sims & J Hedberg
University of Technology, Sydney & University of Wollongong,

Session 2: Speech and Retrieval

(Back to Summary)

This group has considerable overlap of interest with the previous one, but here a major topic is the relative merits of language and other means for describing information one wishes to retrieve from database systems, as well as wider usability issues for speech interfaces.

Oral and gestural activities of the users in the GEORAL system
J Siroux, M Guyomard, Multon, & C Remondeau
IUT-IRISA, Lannion, France.

Applying a cognitive model of the user to the design of a multimodal speech interface
John Dowell, Yael Shmueli & Ian Salter
University College, London, UK.

Video mail retrieval by voice: Towards intelligent retrieval and browsing of multimedia documents
J T Foote, M G Brown, G J F Jones, K Sparck Jones, & S J Young
Cambridge University & Olivetti Research Ltd. Cambridge, UK.

Structuring indexes for video clips
* Eric A Domeshek & Andrew S Gordon
Northwestern University, Evanston, IL, USA.

Session 3: Intelligence in Multimodal Presentation

(Back to Summary)

Various kinds of rules, guidelines and strategies have been proposed for deciding how to present information, making the best use of multimodal options. This session opens the issues (which will also recur later) of how to define good theory in this area, and then how to relate it to practical approaches to the problem.

Rule-based multimodal interface design
Niels Ole Bernsen
Roskilde University, Denmark.

Media/Modalities allocation in intelligent multimedia user interfaces: Towards a theory of media and modalities
Charalampos Karagiannidis, Adamantios Koumpis & Constantine Stephanidis
Foundation for Research & Technology, Crete, Greece.

Intent-based layout in interactive multimedia communication.
Winfried H Graf
German Research Center for AI (DFKI), Saarbruecken, Germany.

Intelligent presentation of information retrieved from heterogeneous multimedia databases
M Hare, A Doubleday, I Bennett & M Ryan
City University, London, UK.

Session 4: Software and Interaction Design Techniques

(Back to Summary)

Multimodal systems demand innovative approaches in terms both of software architectures and of interactive techniques and devices. The papers in this session all address some of the technical problems and possibilities that arise.

Multifeature systems: From HCI properties to software design
Laurence Nigay & Joelle Coutaz
LGI-IMAG, Grenoble, France.

M: An architecture of integrated agents
Doug Riecken
AT&T Bell Laboratories, Holmdel, NJ, USA.

Toward engineering for multimodal interactive systems
Hans-W Gellersen
University of Karlsruhe, Germany.

Expectation-based user interaction
F L Engel & Reinder Haakma
Philips Research Laboratories, New York, USA.

Session 5: The Relevance of Formal Theory

(Back to Summary)

Those who use, and even many of those who create multimodal systems, are often unaware of the formal techniques that can be applied to deepen our understanding of how representation works, how it can support reasoning, etc. A natural topic for discussion in this session is what these techniques can contribute to practical system dvelopment.

Architectures for heterogeneous reasoning
Dave Barker-Plummer & Mark Greaves
Stanford University, USA.

Reasoning on multimodal interactive systems
Fabio Paterno & Menica Mezzanotte
CNUCE-CNR, Pisa, Italy.

Minimal sorts for interpreting pictures
Henk Zeevat & Dejuan Wang
University of Amsterdam, Netherlands.

Sublanguages in text and graphics
Ehud Reiter
CoGenTex Inc, Ithaca, NY, USA.

Session 6: Cognitive Design and Evaluation 1

(Back to Summary)

The final two sessions concentrate on various aspects of the application of cognitive theories to the design and/or evaluation of multimodal or multimedia systems. This is what is becoming known as "Cognitive Engineering", but a central discussion point is how much we can say clearly about the cognitive underpinnings of multimodal communication, and how this can help us with e.g. predicting usability, user modelling or task-oriented interaction in practical contexts. In this session, general considerations about diverse modalities are addressed.

Graphs and sentences: distinguishing semantic from perceptual explanations of their usability
Keith Stenning
University of Edinburgh, Scotland, UK.

Multimodal interfaces based on types and goals of cooperation between modalities
Jean-Claude Martin & Dominique Beroule

The effect of interactive multimedia interfaces upon representation selection
Richard Cox, Keith Stenning, & Jon Oberlander
University of Edinburgh, Scotland, UK.

Automatic composition of intention-based music for multimedia interfaces
Detlev Zimmerman
German Research Center for AI (DFKI), Saarbruecken, Germany.

Session 7: Cognitive Design and Evaluation 2

(Back to Summary)

The issues of the previous session are taken up again with particular attention to evaluation, user-modelling and hypertext.

Evaluating multimedia presentations
Peter Faraday & Alistair Sutcliffe
City University, London, UK.

External Cognition as an alternative framework for evaluating single, multi and virtual media
Yvonne Rogers & Mike Scaife
University of Sussex, Brighton, UK.

A glass box intelligent help interface
Kristina Hook, Jussi Karlgren & Annika Waern
SICS, Sweden.

Intelligent support for navigation in hypermedia: discourse structure and the Web
Robert Inder, Jon Oberlander & Richard Tobin
University of Edinburgh, Scotland, UK.

Towards the generation of hypermedia structure
* Lynda Hardman & Dick C A Bulterman
CWI, Amsterdam, Netherlands.


(Back to Summary)


If you have further queries, please do not hesitate to contact:

John Lee
Human Communication Research Centre
University of Edinburgh
2 Buccleuch Place
Edinburgh EH8 9LW
Scotland, UK.

Tel: +44 131 650 4420
Fax: +44 131 650 4587