« Supported Reading: Ambedkar Multimedia Study Environment | Main | Virtual Village Allows Virtual "Fieldwork" »

December 7, 2005

Team Up With Timo: Vocabulary Builder With Speech Synthesis


Children with speech difficulties - whether stemming from hearing impairment, autism, or language impediment - often benefit from drills in which they practice mimicking the facial muscle, jaw and tongue movements of instructors. However, as anyone who has taught or taken speech classes is well aware, such exercises are often fraught: children can become frustrated or embarassed by the face-to-face encounter such drills require.

Team Up With Timo, a collaboration between psychology professor Dom Massaro and software developer Dan Feshbach, is an interactive tutorial that allows K-4 students to closely study the mouth movements of an animated 3-D vocal coach and practice both pronounciation and reading comprehension. Timo, the software-driven language tutor, is a commercial spinoff of Baldi, a similar language tutor created more than 15 years ago in Dr. Dominic Massaro's Perceptual Science Lab at the University of California at Santa Cruz.

The software interface is very straightforward: after choosing from one of 127 vocabulary lessons, students are directed to a screen with three central elements: a central window in which the lesson takes place and two side windows containing Timo's talking head and a scoreboard. The lesson begins with Timo pronouncing the name of an object pictured in the lesson screen - in the screenshot above, a pot of gravy from a Thanksgiving meal - while the word itself appears below the set of images. The student is asked first to select the image based on a visual cue; the image they must select is highlighted). Then, they are asked to select the image according to a verbal cue from Timo; if they are hearing impaired, they will need to lip-read in order to understand what to select. After this, they must select the vocabulary word for each image based on the same verbal cue; finally, they must pronounce the word themselves following both verbal and visual cues (in the screenshot below, Timo is asking students to say the words planets, sun and moon: because this is a trial version of the program, the microphone function is turned off).


I originally assumed that Timo would contain a speech recognition program that would be able to evaluate students' pronounciations, but there isn't - which makes sense, because, to my knowledge, speech recognition software isn't sophisticated yet enough to recognize or evaluate speech impairments. Instead, Timo has a recording function, so that teachers and therapists can evaluate student's speech on their own.

The lack of internal speech evaluation is thus understandable, but it does lead to one peculiarity in the program: while students only get 'points' for correct answers in the first three stages of the tutorial, (these points indicated in the scoring window, somewhat mysteriously, by the accumulation of sand dollars on the beach) they get points every time the microphone records them trying to pronounce a word, no matter how it is pronounced. This design quirk, in turn, might lead the student to decide that their pronounciation of a word is ultimately emphasized less than their recognition of it, but several academic studies using Timo have shown that it's effective at teaching pronounciation.

Personally, I wanted Timo to show a little more soul: his facial expressions are doubtless constrained by the software's focus on modeling speech, but it seemed like it would be possible for him to be a bit happier-looking when students get the answers right -- sand dollars are well and good, but what about a smile? According to the product website, Baldi (the original version of Timo) has been used to teach austitic children the meaning of facial expressions, so it seems like more expressiveness is certainly possible.

Massaro and Feschbach are now developing a Timo authoring program, which is currently looking for beta testers: it's designed to let speech therapists and parents create their own lessons with Timo. If you're interested, contact them at pilot@animatedspeech.com.

Posted by lisa lynch at December 7, 2005 2:49 PM