Page 742 - AI for Good Innovate for Impact

AI for Good Innovate for Impact

2 Use Case Description

2�1 Description

As a developing country, China has 27.8 million hearing-impaired individuals who struggle
with bidirectional communication through sign language [31], making them a vulnerable group
in society. Most deaf individuals physiologically retain speech capacity but gradually lose
verbal abilities due to auditory deprivation- unable to hear, unwilling to speak, and ultimately
incapable of articulation [42]. Therefore, we target hearing-impaired individuals with residual
speech capacity, applied in daily social, educational, and rehabilitation scenarios to aid their
vocal learning and social integration.

Lip-reading forms the foundation for vocal rehabilitation. However, the lack of commercial
lip-reading tools forces deaf learners to rely solely on special education teachers in limited
institutions. Severe shortages and uneven distribution of educational resources create
significant barriers. For instance, Wuhan (capital of Hubei Province) hosts 15,334 hearing-
impaired residents [53], yet its provincial rehabilitation center employs merely a dozen special
educators.

This pioneering project leverages AI technology to empower deaf individuals in lip-reading
acquisition and active vocalization, enabling bidirectional communication. We have developed
the world’s first AI-powered lip-reading training system, allowing autonomous learning through
real-time visual feedback. Users imitate lip movements from instructional videos while our
computer vision-based algorithm analyzes and evaluates their articulatory gestures, providing
immediate corrective feedback. This system particularly benefits remote communities through
web access, reducing dependence on human instructors according to field tests. To date,
343 deaf users have regained speech capabilities, with data showing 3× learning efficiency
improvement. Our lip-reading recognition algorithm demonstrates exceptional performance
in complex scenarios, achieving 10% higher accuracy than state-of-the-art methods on
international benchmarks [64].

Currently deployed in rehabilitation centers and special schools across China through non-
profit partnerships, the system remains free for users. Looking forward, we aim to harness AI’s
potential in ensuring equitable educational opportunities for all hearing-impaired individuals,
empowering them to express, learn, and thrive equally in our digital society - transforming
impossibility into reality. The system protects user rights and data privacy by anonymizing
user information and implementing access control. Additionally, video data is only used for
model inference on the server side and not stored long-term, ensuring compliance with data
security standards. We are also promoting the localized deployment of models to achieve
local processing priority, minimizing the need for transmitting sensitive lip-reading video data.

Use Case Status: The case is part of larger research projectArtificial Intelligence, Assist Disabled
Communities

Partners: Huazhong University of Science and TechnologyN/A

2�2 Benefits of the use case

Quality Education: The project leverages AI technology to overcome the traditional dependence
on special education teachers for lip-reading instruction. It enables deaf individuals—especially

706