Summary of Baldi as a Language Tutor

The social psychologist Kurt Lewin said that there is nothing so challenging as a practical problem. However, there is also nothing so practical as a good theory. Given our theoretical framework, we have applied our knowledge to language training for children with language deficits. These include children with hearing impairment, as well as children diagnosed with autism.

We have developed a computer animated talking head, Baldi, who can serve as a language tutor, reading tutor, or personal agent in human machine interaction, interpersonal communication over the internet, or in multi-person game environments. Baldi produces realistic facial speech and emotion, and is appropriately aligned with either synthesized or natural speech. Baldi's speech and emotion have been developed to mimic the speech of real persons. The quality and intelligibility of the talking head have been repeatedly modified and evaluated to be as realistic as possible. Our talking head now accurately simulates much of the behavior of naturally talking heads. It is also possible to control different morphological heads and to texture map the image of a real person onto the talking head. Baldi also has a tongue and palate, which have been modeled on real tongues and palates.

We have been using our animated talking head, Baldi, as a language tutor for deaf children in the Tucker Maxon Oral School (TMOS). We have also collaborated with researchers in Mexico at UDLA and researchers in Chile at Playa Ancha University to improve Spanish speech recognition, text to speech synthesis, and facial animation systems, and to implement these systems in the language training applications. Given that Baldi now speaks and understands Spanish, this TMOS project has also expanded to include international cooperation though the efforts of individuals in Rotary Clubs in Portland Oregon and Renaca, Chile. Rotary has donated 12 PCs to Deaf School F-258 in Renaca Chile to enable language training with the CSLU Toolkit. The school has about 300 children, with about 60 of them severely hearing-impaired. Rotary also sponsored a two-week training course at TMOS attended by 10 Chileans, including a physician, audiologist, speech scientist and educators from deaf school F258.

We have also been applying our psychological research on technology in language learning with autistic children at Natural Bridges School and the Bay School in Santa Cruz.