Alahverdzhieva, K. and A. Lascarides [2011] Semantic Composition of Multimodal Actions in Constraint-based Grammars, Proceedings of Constraints in Discourse (CID) 2011, Agay-Roches Rouges, Var, France.
The use of speech-accompanying hand gestures to depict objects, to structure the discourse or to give directions is ubiquitous in face-to-face interaction. In this paper, we analyse multimodal signals consisting of speech and gesture from the perspective of semantic composition in constraint-based grammars: we elevate standard methods from linguistics to description of multimodal input so as to connect the semantics of the gestural signal to the semantics of the speech signal and to produce an integrated logical form.
@inproceedings{alahverdzhieva:lascarides:2011,
author = {Katya Alahverdzhieva and Alex Lascarides},
title = {Semantic Composition of Multimodal Actions in Constraint-based Grammars},
year = {2011},
booktitle = {Proceedings of Constraints in Discourse (CID) 2011},
address = {Agay-Roches Rouges, Var, France}
}