[ Technology ] | [ Get Involved ] | [ Papers ] | [ Home ]

Speechreading Study

Speechreading can enhance the intelligibility of speech for the hearing impaired. More generally it is thought that most individuals (with or without a hearing impairment) use lipreading to some extent, especially if the auditory signal is degraded in noisy environments.

We have developed a prototype speechreading aid, LipCam, to assist the hearing impaired in an everyday classroom setting. The system provides a student continuous access to the educator's visual (lips) and auditory speech. If an educator were to move about the room or turn to the blackboard a student who benefits from lipreading would maintain a consistent and close up view of the educator's lips. The visual information is provided to them through a small display which can be worn on a head band or displayed on a small monitor.

Various modes of speechreading tasks which relate to the use of LipCam are being investigated with particular emphasis on how lip and face information can enhance perception in communication. Work will include field testing and further development of the prototype.

This research may also have implications on how video or animated models of the lips or face may be incorporated into applications to increase speech perception and language acquisition. A new generation of computers will provide improved processing and animation capabilities. With these abilities, visual information may be used for the hearing impaired in current and future technologies such as speech recognizers, speech synthesizers, educational software, and telecommunications.

Papers and Publications

1. "A Preliminary Investigation of a Lipreading Aid Intended for a Regular Educational Classroom Setting", Phalanga, Barner, Joyce; RESNA 96. RTF

Staff

Anna Phalangas, Research Engineer
Ken Barner PhD., Principle Investigator