Home - CV - Research - Publications - Life

Pei-Yun (Sabrina) Hsueh PhD Student/Research Associate Institute of Computing and Communicative Systems Human Commmunication Research Centre School of Informatics University of Edinburgh 2 Buccleuh Place Edinburgh EH9 8LW Office: 2-BP 4R04 Tel: +44-131-650-8289 Email: p dot hsueh at ed dot ac dot uk

I am now based in IBM T.J.Watson Center at Yorktown Heights, New York. The Edinburgh email address will still be valid for a while. You can reach me also at pshsueh at us.ibm.com.

I am a PhD student at the School of Informatics in University of Edinburgh. My research interest ties closely to the marriage of artificial intelligence and human computer interaction, with a focus on integrating machine learning and empirical analysis approaches for natural language understanding. My recent research concerns the development of spoken language understanding applications in spontaneous speech, using a variety of approaches ranging from statistical analysis, empirical study to machine learning. This is no secret that people speak differently under different circumstances. Some of the differences are systematic and can be attributed to deeper differences, such as the intention of speaker. My job is then to develop a learning framework that can be used to identify multimodal features (and patterns) that are characteristics of the systematic differences in human conversations and to build automatic detection mechanisms that are robust to spontaneous speech effects. Current projects include automatic topic segmentation and labeling and automatic decision detection. The overarching goal is to provide visual aids at the right level of details for the users to find information from the often-lengthy archives of conversation recordings.

• I have been invited to attend Grace Hoppers Celebration (Oct, 2007) in Florida.
• I have recently been selected as Google Anita Borg Scholar (June, 2007).
• I am now also attending Dialog Systems Group and Statistical Natural Language Processing Reading Group in Edinburgh.
• I am on the founding committee of the bi-monthly Language at Edinburgh Lunch. We initiated this series of events as a platform for members of the University of Edinburgh interested in language to showcase their research and discuss ideas and socialise with researchers in other fields. Departments already participating include Informatics, Philosophy, Psychology and Linguistics and English Language.
• I have attended the 17th European Summer School in Logic, Language and Information (ESSLLI 2005) and the fifth IGK Annual Research Meeting in 2005, and the 8th Summer School of Euromasters in Language and Speech in 2007.
• I have been awarded top scholar award (University of Washington) in 2003 and Elite Scholarship (Ministry of Education, Taiwan) in 2004.

Research

Thesis Proposal: Toward Automatic Decision Detection: Empirical Study, Statistical Analysis, and Machine Learning Approach (Abstract)

• This study addresses the problem of detecting and labeling decision-making dialogue (DM detection and labeling) from a lengthy archive of meeting recordings. The problem is central to the automatic extraction and summarization of information from the recorded conversation speech. It has also posed a challenge to the mainstream spoken language understanding techniques, which have made assumptions violated by the spontaneous, face-to-face dialogues in meetings. In this proposal, I present a learning framework which identifies systematic differences in speech for automatic detection of decision-making dialogue acts, extracts the decision-making dialogue acts, and presents them in diagrammatic form. (Read more...)

Spoken Language Understanding

• AMI Project: The project aims at substantially advancing the state-of-the-art, within important underpinning technologies (such as human-human communication modeling, speech recognition, computer vision, multimedia indexing and retrieval). It will also produce tools for off-line and on-line browsing of multi-modal meeting data, including meeting structure analysis and summarizing functions. The project also makes recorded and annotated multimodal meeting data widely available for the European research community, thereby contributing to the research infrastructure in the field. My work here focuses on developing automatic mechanisms for segmenting conversations into a number of coherent segments and labelling them with similar agenda-based items. (Supervisors: Dr. Johanna Moore and Steve Renals, UoE)
• AMIDA (Augmented Multiparty Interaction with Distance Access) Project: The project focuses more on multimodal structure and content analysis, including the modelling of individuals and groups, through the joint processing of multiple (multimodal) information channels (audio, visual, slides, handwriting, and white board activity). My work here focuses on statistically analyzing the systematic differences (e.g., words, prosody, meeting context, topics) in human decision-making conversations and developing automatic mechanisms for extracting decision points for summarisation and question answering. (Supervisors: Dr. Johanna Moore and Dr. Steve Renals, UoE)

Medical Informatics

• Telemakus Project: Telemakus is an information management and knowledge discovery tool that provides a unique way for search both within and across research documents that form the intellectual output of a domain. Each knowledgebase contains data extracted from scientific reports in a specific domain. In this project, I analysed the linguistic patterns exhibited in the captions and used the patterns found to boostrap the extraction of relations in scientific publications. (Supervisor: Dr. Sherrilynne Fuller, Health Sciences Libraries and Information Center, U of Washington)
• GeneDepot Project: I conducted an organisational study which aims to create system design guidelines through observing how knowledge workers in a parmaceutical company organise information for subsequent use. (Supervisor: Dr. Carol Morita, Genentech Inc.)

Computer Supported Collaborative Work

• KFTF Project: The goal of this study is to understand better the ways in which people manage information for subsequent re-access and re-use. AddToFavorites2 (ATF2) was developed as a tool to support the kind of “super” keeping method. My project is to conduct an evaluation to test whether a "super" keeping method actually adds more values and increases users' interests in using Favorites to keep information found. I was also involved in the initial stage of desigining a standalone personal information management application which aims at provide universal labeling of personal information items. (Supervisors: Dr. William Jones and Dr. Harry Bruce, iSchool, U of Washington)
• CoBrowse Project: CoBrowse is a social bookmarking tool similar to that provided by del.li.cious back in 2002. This tool aims to collect recommendations of quality websites from users' circles of friends and display recommendations on the fly. (Collabotors: Chan Jean Lee and Bin Xin) (Supervisor: Dr. Warren Sack, SIMS, UC.Berkeley)

Publications

• Hsueh, P., Melville, P., Sindhwani, V. (2009). Data Quality from Crowdsourcing: A Study of Annotation Selection Criteria. In Proceedings of NAACL-HLT Workshop of Active Learning, 2009.
• Liu, Y., Hsueh, P., Lai, J., Sangin, M., Nüssli, M-A., Pillenbourg, P. (2009). Who is the expert? Analyzing gaze data to predict expertise level in collaborative applications. In Proceedings of IEEE International Conference on Multimedia and Expo (ICME 2009).
• Hsueh, P. and Moore, J. (2009). Improving Meeting Summarization by Focusing on User Needs: A Task-Oriented Evaluation. In Proceedings of the International Conference on Intelligent User Interfaces (IUI), 2009. pdf presentation
• Hsueh, P. (2008). Audio-based Unsupervised Segmentation of Meeting Dialogue. In Proceedings of the 33rd International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2008.
• Pei-Yun Hsueh, Jonathan Kilgour, Jean Carletta, Johanna Moore, Steve Renals. (2007). Automatic Decision Detection in Meeting Speech. In the Proceedings of MLMI 2007. In Popescu-Belis, Renals, Bourlard (Eds.), Lecture Notes in Computer Science, Springer-Verlag, 2007. pdf
• Hsueh, P. and Moore, J. (2007). Combining Multiple Knowledge Sources for Dialogue Segmentation in Multimedia Archives. In the Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL 2007), Prague, Czech Republic. ps
• Hsueh, P. and Moore, J. (2007). What Decisions Have You Made: Automatic Decision Detection in Conversational Speech. In the Proceedings of NACCL/HLT 2007, Rochester, NY. Erratum pdf
• Hsueh, P. and Moore, J. (2006). Automatic Topic Segmentation and Lablelling in Multiparty Dialogue. In the first IEEE/ACM workshop on Spoken Language Technology (SLT 2006), Aruba. pdf
• Hsueh, P., Moore, J. and Renals, S. (2006). Automatic Segmentation of Multiparty Dialogue. In the Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2006), Trento, Italy. pdf ppt

• Streiter, O., Hsueh, P. (2000). A Case Study on Example-based Parser. In Proceedings of the International Chinese Computational Linguistics Conference, Chicago, USA.

Demo, Poster, and Presentation (Selected)

• Pei-Yun Hsueh. (2007). Decision-Based Summarization for Meetings. In the Workshop of Meeting Modelling, 2007.
• Pei-Yun Hsueh, Jonathan Kilgour, Jean Carletta, Johanna Moore, Steve Renals. (2007). Automatic Decision Detection in Meeting Speech. In the Demo Session of MLMI 2007, Brno, Czech Republic.
• Gabriel Murray, Pei-Yun Hsueh, Simon Tucker, Jonathan Kilgour, Jean Carletta, Johanna Moore, Steve Renals. (2007). Automatic Segmentation and Summarization of Meeting Speech. In the Proceedings of NAACL/HLT 2007, Rochester, NY.
• Hsueh, P. (2007). Toward Automatic Decision Detection: Empirical, Statistical and Machine Learning Approach. Poster presentation at MMKM Workshop (Multimedia Knowledge Management): Industry meets academia, UK.
• Hsueh, P. and Phuwanartnurak, A.J. (2004). AddToFavorites2 (ATF2): Prototype and Evaluation. The American Society for Information Science and Technology, 2004.
• Efthimis N. Efthimiadis and Hsueh, P. (2004). The Web Searching of Greeks: An Exploratory Study. The American Society for Information Science and Technology, 2004.
• Hsueh, P. S., Lee, C., Xin, B. (2003). Weaving Searching into Browsing: A Perspective-Taking Collaborative Web Resource Mediator. The Twelfth International World Wide Web Conference, Budapest, Hungary, 2003.

Techinal Report

• AMI Annual Report D5.1 Report on InitialWork in Segmentation, Structuring, Indexing and Summarization. pdf

Life

Outside of my academic life, I enjoy hanging out with my dear friends, traveling around the world, and studying recipes I got from my trips. I am also interested in trying out different sports. Among them, what I like the most are hill walking, sailing, rock climbing, and swimming. I have been wanting to try gliding, but have not been lucky enough with the weather in Edinburgh. When I am allowed to stay on the green lines, I also enjoy skiing. Edinburgh is such a lively city, especially during the festival time. I recently performed Indian classical Khatha dance on stage in the Edinburgh festival. That was a great experience. At least, I could never picture myself looking like this before I committed myself to this.

Guru Surangama and us on stage	Dreamy Edinburgh during Christmas	Beltane

The view from my window			Winter ice rinks for birds

• Albums of some of my recent trips..

Budapest	Trier (IGK 2005)	Trento (EACL 2006)	Milano
Firbush	Pentland	Fort Williams	Paris
Spain (Toledo)	Aruba (SLT 2006)		Brno
Portmoak (Gliding Society)	Hong Kong	The Alps	Mt. Pilatus (Lucerne)
Prague (ACL 2007)	Google Zurich	Lugano	Kairouan (Tunisia)
Isle of Skye	El Jem (Tunisia)