From Technology Awards - 1995

Visualizing Speech University of California at Santa Cruz's Animated Talking Face.

Innovators: Michael Cohen and Dominic Massaro.     Part of the fun of old Godzilla movies lies in hearing the actors speak in english but seeing their lips move in Japanese. Such lip-synch comedy may soon be a thing of the past. Psychologists Dominic Massaro and Michael Cohen at the University of California at Santa Cruz have created a computer program that reproduces the facial expressions people make when they talk.     Cohen and Massaro originally intended to design a computer-generated "talking face" that deaf and hard-of-hearing people could use to practice their lip-reading skills. For more than four years, they videotaped people as they talked, converted the videotape to digital images, and broke down the images into segments that correspond to the parts of speech. Then they reassembled the segments to create a computer program that can convert typewritten into a computer generated talking face whose facial muscles, jaw, and tongue move with an accuracy unprecedented for an animated figure. "Our lip-reading subjects tell us that ours is the only face they can understand," Massaro said.     The inventors believe that the talking face has potential for a wide range of uses, such as computer games with more lifelike characters, friendlier computers that can talk, and even the dubbing of foreign so the actors mouths move in a way that is consistent with their speech. The program can also help students learn foreign languages by showing them how the mouth is supposed to move during speech. "It turns out that the visual clues - the position of the lips and jaw of the speaker, for example - can be very important to understanding what a person says," says Massaro.