Page 769 - AI for Good Innovate for Impact

AI for Good Innovate for Impact

2 Use Case Description

2�1 Description 4.9: Accessibility

Context and Background

Millions of people in India who are deaf or hard of hearing rely on ISL as their main form of
communication. Nonetheless, there is still a big communication gap between people who speak
ISL and those who don’t. For the deaf community, social integration and information access are
hampered by the absence of real-time translation tools. The language barrier that separates
ISL from commonly spoken languages like Hindi and English also makes communication
more difficult and restricts the participation of deaf people in public services, education, and
mainstream conversations. While the core focus of this work is on developing an accurate ISL
recognition model, we recognize that effective communication solutions must reflect the rich
sociocultural diversity of India's deaf and hard-of-hearing communities. Incorporating detailed
user archetypes and narratives that capture lived experiences and regional linguistic diversity
will be essential for real-world adoption and accessibility. These qualitative insights will guide
future phases of this project to ensure the technology is inclusive and responsive to community-
specific communication barriers.

Objective and Aims

This research aims to develop an AI-powered system that can effectively detect and translate
ISL into English, and vice versa. The core objectives include:

• Real-time ISL Recognition and Translation: Enabling real-time translation between ISL and
English, allowing smooth interaction between deaf individuals and others.
• Bidirectional Translation: The system will not only translate from ISL to English but also
facilitate translation from English to ISL, making it versatile.
• Multi-language Support: Since translation from other languages (e.g., Hindi) to ISL will
be enabled by first converting the text to English, this approach will support indirect
translation to ISL from a variety of languages.
• Proof-of-Concept (PoC) Development: The project will create a PoC system to
demonstrate the feasibility of ISL recognition and translation in real-time using open-
domain ISL datasets and lightweight machine learning models. Use models like the
CLIP-SLA method [5], which offers parameter efficient adaptations for continuous ISL
recognition.

Use Case Status: In Development (PoC Stage) and this are currently developing a
PoC system�

Partner: N/A

2�2 Benefits of the use case

Quality Education: Integrating ISL translation tools into educational settings enhances
accessibility for deaf and hard-of-hearing learners. This ensures that individuals who rely on sign
language can fully engage with classroom activities, online learning platforms, and academic
programs, contributing to a more inclusive and equitable education system.

733