SPEECH PERCEPTION BY EAR AND EYE

/ FACIAL ANIMATION @ PSL


News and Publications

PSL On ABC PrimeTIME, Thurs March 15, 2001

NSF Challenge Grant
Ron Coleand Mike Macon at CSLU, in conjunction with Dominic Massaro at the University of Santa Cruz and Alex Waibel at Carnegie Mellon University, have been awarded a challenge grant for a project entitled "Creating Conversational Agents for Language Training: Technologies for the Next Generation of Interactive Systems." The project goal is to create a realistic conversational agent for language training, in particular for the hearing-impaired. The conversational agent includes the following components:

  1. A 3-D model of a talking face with accurate articulator movements and facial expressions.
  2. Natural text-to-speech generation based on concatenative synthesis.
  3. Visual speech recognition using a desktop camera in order to provide information about the student's articulators.
  4. Auditory speech recognition to understand what is being said and to correct mispronunciations.

The conversational agent is initially being used to teach hearing-impaired students at the Tucker-Maxon Oral School in Portland, Oregon.


Demos


MPEG Movies


QuickTime Movies



Papers


This material is based upon work supported by the National Science Foundation under Grants Nos. CDA-9726363, BCS-9905176, and IIS-0086107, the Public Health Service under Grant No. PHS R01 DC00236, cooperative grants from the Intel Corporation and the University of California Digital Media Program, and grants from the University of California-Santa Cruz.
This page has been accessed 1286 times since Tue Jan 6 16:12:21 PST 1998.