![[Speech to Braille home page]](images/back.gif)
Communication is the basis of human interaction. The barriers to communication that deaf-blind individuals face deny them of many independent interactions with the world. The project combines several recently available technologies into a system which reduces the need for an interpreter, thus allowing greater independence and interaction with the world.
Deaf-blindness is any condition in which a combination of visual and hearing impairments cause a severe communication and/or developmental gap (1990, IDEA, Sec. 622). This dual-sensory impairment causes deaf-blind individuals to experience the world in a unique way.
The problems deaf-blind individuals face include a feeling of isolation, a lack of independence, difficulty in obtaining education, and inadequate employment opportunities. In the past, overcoming these obstacles has been difficult, if not impossible. The underlying foundation of these problems is a barrier to communication. Combining recently developed technologies to facilitate communication will allow deaf-blind individuals to bridge the communication gap.
Recent surveys estimate that there are 45,000-50,000 individuals who are deaf-blind in the United States (Baldwin, 1994; Watson, 1993). Communication is one of the largest barriers to independence that this population must overcome. One common approach to the communication barrier is to use an interpreter. However, due to circumstances such as scheduling difficulties and high demand for their services, interpreters are often a limited resource.
If the need for interpreters could be eliminated or reduced, the independence of deaf-blind individuals would be greatly improved. We have designed a computer based system which allows deaf-blind individuals to achieve this freedom.
In order to be successful, the design process for this new communication technology has incorporated significant user input. This project involves both authors, (undergraduate students in Biochemistry and Computer Science, respectively), one of whom is deaf-blind, as both developers and users. Furthermore, it employed a technique known as scenario-based design (Carroll, 1995) in which the intended user does not have to articulate his/her needs, but participates in the development of performance scenarios. The project team brainstorms and writes a trial scenario describing the technology and the interactions of the users. (Users include both the deaf-blind individual and the those with whom he/she communicates.) The team assesses the scenario by "acting it out" and identifying aspects that are useful as well as noting problems. The scenario is then modified and evaluated in a repeat performance. This process is continued until a truly user-responsive scenario is achieved. This approach identifies important design issues that often would not have been obvious to users in more traditional interviews and questionnaires.
Initial scenarios determined that current computer technology cannot replace the skills of human interpreters entirely. Scenarios where technology attempted to replace interpreters were unsuccessful. More appropriate were situations where technology could provide services that allowed effective conversation when an interpreter was unavailable. After a number of iterations, the following scenario was found to be very useful. It incorporates the flow of movement and actions comfortable to users, it respects personal space and normal conventions of social interaction, and it accommodates the limitations of commercial computer technologies.
The deaf-blind user is working in an office with a computer. A co-worker enters the room, triggering a device that sends a signal to the user, announcing the co-worker's presence. The user's choice of responses includes: acknowledging the co-worker, ignoring the co-worker, or indicating that the co-worker should come back at a more convenient time.
Once acknowledged, the co-worker lifts the handset, says, "Hello," and several processes begin simultaneously. In the background, without interrupting currently open applications, the computer chooses the appropriate voice file for the co-worker. Additionally, a second signal indicates that the co-worker wishes to speak to the user.
Once the user is ready to converse, a pre-defined key combination accesses the communication software. The communication window is a split screen with the co-worker's remarks occupying one half and the user's comments in the other. The Braille terminal displays the co-worker's name based on the selected voice file and conversation can then commence. As the user types comments into the window, the speech synthesizer converts the text to spoken language. As the co-worker speaks into the handset, speech recognition software translates the spoken words into text, which is output to both the monitor and the Braille terminal. Once the conversation is finished, the co-worker leaves, and the user returns to work.
The system described in the scenario was implemented as a laboratory prototype. The front end software integrates several components to translate speech to Braille, and keyboard input to synthesized speech. After careful consideration, the following hardware and software components were determined to best meet our needs.
One possible limitation of the system is that DragonDictate requires speaking in isolated words. However, we found that the speed of conversation was still close to 50% of spoken conversations (100-150 wpm) and equivalent to interpreted conversations. Deaf-blind interpretation consists mainly of finger spelling at approximately 60 wpm. DragonDictate can translate trained voices at speeds of 60 wpm, and untrained voices at 40 wpm. The speech synthesizer is limited only by the typist's speed.
Based upon both feedback from deaf-blind users and the number of deaf-blind people in the United States, we are confident that a market exists for this product. One of the many benefits of this system comes from eliminating the costs associated with hiring an interpreter. This, in turn, opens many doors, including increased educational opportunities and a wider range of available jobs.
In the future, many additions could be made to the system to enhance independence even further:
The purpose of this system is to assist deaf-blind individuals in solving the unique communication problems they experience. The portability, user friendly design, and state-of-the-art technology inherent to the system combine to give the deaf-blind individual greater control over interactions with the everyday world and to establish personal independence.
Baldwin, V. (1994). Annual deaf-blind census. Monmouth: Teaching Research Division.
Carroll, J., (Ed.). (1995). Scenario-based design: Envisioning work and technology in system development. New York: Wiley.
U.S. Department of Education, Office of Special Education Programs, Severe Disabilities Branch, Services for Children with Deaf-Blindness Program. IDEA, Part C; Section 622; CFDA 84.025. Washington, DC.
Watson, D. & Taff-Watson, M., (Eds.). (1993). A model service delivery system for persons who are deaf-blind, Second edition. Arkansas, University of Arkansas.
Funding for this project was provided by NSF Grant #HRD-9450019 to the Applied Science and Engineering Laboratories, and supported by the Nemours Foundation. The authors wish to thank Tamara Black, JoAnne King, and Richard Foulds.
Beth E. Finn
Applied Science and Engineering Laboratories
duPont Hospital for Children
University of Delaware
1600 Rockland Road/P.O. Box 269
Wilmington, DE 19899
(302) 651-6830
bethf@Udel.Edu