Alahverdzhieva, K. and A. Lascarides [2011] An HPSG Approach to Synchronous Speech and Deixis, in Stefan Muller (ed.), Proceedings of the 18th International Conference on Head-Driven Phase Structure Grammar (HPSG), pages 6--24, Seattle, CSLI Publications.
In this paper, we present a constraint-based analysis of multimodal communicative signals consisting of deictic gestures and speech. Our approach re-uses standard devices from linguistics to map multimodal form to meaning, thereby accounting for the gestural ambiguity by means of established underspecification mechanisms. To specify this mapping, we use empirically extracted grammar construction rules which capture the conditions under which the speech-deixis signal is grammatical and semantically intended. We demonstrate that the constraint-based grammar framework of HSPG is expressive enough to produce multimodal lfs from syntax.
@inproceedings{alahverdzhieva:lascarides:2011,
author = {Katya Alahverdzhieva and Alex Lascarides},
year = {2011},
title = {An HPSG Approach to Synchronous Speech and Deixis},
booktitle = {Proceedings of the
18th International Conference on Head-Driven Phase Structure Grammar
(HPSG)},
editor = {S.\ M\"uller},
pages = {6--24},
publisher = {CSLI Publications},
address = {Seattle}
}