Open Collections

Design of a computer-assisted speechreading training system for Japanese

University of British Columbia

Speechreading is known to play an important role in speech perception, particularly for hearing-impaired individuals. Computer-Aided Speechreading Training (CAST) systems have been implemented for English and French, to train adults with mild-to-moderate hearing loss to improve their communicative ability with an effective use of visual speech information. Recent tests have shown that the CAST system is effective in improving speechreading performance. The purpose of this study is to design a similar system for Japanese, which will be called CAST-J. CAST-J follows the basic principles used for the English and French versions of CAST: (1) it uses the method of discourse tracking, in which training is focused on the perception of ongoing speech; (2) it aims at training subjects to integrate visual information, auditory information, and linguistic redundancy of the message; (3) it is designed for post-lingually hearing-impaired adults. Adults with cochlear implants are also considered as a possible target population for CAST-J, as long as they use auditory information of speech as a primary source in speech perception. The lessons of CAST-J are organized around the concept of viseme. Fifteen visemes of Japanese have been determined for this study, more specifically, five vowel visemes: { i }, { e }, { a }, { o }, and { u }, seven consonant visemes: { p, b, m }, { w, Φ }, { r }, { s, z, t, d, n, ts, dz }, { k, g, h }, { f }, and { j, q, ∫, t∫, d₃, η }, and three haku visemes: { N }, { Q }, and { R }. The four sections of each lesson are designed (1) to review previously taught visemes; (2) to practice the recognition of a new viseme; (3) to practice new and old visemes by the tracking method; and (4) to recap the lesson. The lessons progress from easy-to-recognize visemes to difficult-to-recognize visemes. There are sixteen lessons, and two extra lessons are available for pre-and post-training comparison. The hardware required by CAST-J includes a microcomputer, a video disc player, a high resolution video monitor, and a printer. The system allows the instructor (an audiologist) to select the appropriate speaking rate for the video (fast or slow), whether the trainee will practice with or without audio signal, and to skip if wanted the optional lessons or part of a lesson, depending upon the level of the trainee.

Text

2010-08-16

For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.

http://hdl.handle.net/2429/27426

Audiology and Speech Sciences

University of British Columbia

Graduate