SpeakerKathy McCoy
DateOct 27, 2011
Time12:00PM 01:30PM
TitleSceneTalker: A Research-Driven, Utterance-Based System Design for AAC.

The field of Augmentative and Alternative Communication (AAC) is concerned with improving the communicative abilities of individuals who have severe speech impairments or other complex communication needs. Speech generating devices allow a user to enter text which is then synthesized as speech – but the process of inputting text is often very slow, especially for users who have limited motor skills. Utterance-based AAC systems, which allow the selection of full messages, have the potential to significantly speed-up communication rate, but they pose interesting challenges, including: anticipating text needs, remembering what text is stored, and accessing desired text when needed. Moreover, using such systems has profound pragmatic implications as a prestored message may or may not capture exactly what the user wishes to say in a particular discourse situation.

In this talk, I describe a series of research studies that were undertaken to investigate pragmatic issues in using utterance-based AAC systems. Our focus was on goal-oriented situations with unfamiliar partners and we investigated how pragmatic mismatches between the situational context and the prestored messages affected conversational outcomes. Our findings, coupled with cognitive theories, have led us to the design of a prototype AAC system called SceneTalker. The system incorporates and supports a schema-based organization of messages into scenes, and is intended to be used in goal-oriented, routine situations (such as going to a restaurant). The content of the prestored messages take advantage of conversational adjacency pairs which consider partner interaction. Based on our findings, we suggest ways to produce messages where systematic consideration is given to what the partner is likely to say. Specific examples will be shown.

BioKathleen F. McCoy is a Professor in the Department of Computer and Information Sciences at the University of Delaware, with a joint appointment in the Department of Linguistics and Cognitive Science.  Her research interests include natural language generation and understanding, discourse, augmentative communication, assistive technologies, accessibility, and user modeling. McCoy received her PhD in Computer Science from the University of Pennsylvania in 1985 and has been on the faculty of the University of Delaware ever since. She has published extensively in the fields of computational linguistics, speech and language pathology, user modeling, computer-aided language learning, assistive technology, and accessibility.

She served from 1995-2008 as a member of the executive committee, secretary-treasurer, and treasurer of the Association for Computational Linguistics. She was the program chair of the International Conference on User Modeling in 2007, and the ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) in 2009. She is the general chair of the ASSETS conference in 2011. She has a great interest in applying natural language processing technologies to applications for people with disabilities, and has been a founding & continuing organizer of a series of workshops in Speech and Language Processing for Assistive Technologies for 3 years.

Previous Next