Page 656 - AI for Good Innovate for Impact

AI for Good Innovate for Impact

(continued)

Category Education
Technology Speech Recognition, Natural Language Processing (NLP), Text-to-Sign
keywords Translation, Sign Language Animation, Real-Time Processing, 3D Avatar
Rendering, Machine Learning (ML), Deep Learning, Sequence-to-Sequence
Model, Transformer Models, Speech-to-Text Engine, Gesture Mapping,
Sign Language Grammar, Multimodal AI, Contextual Translation, Avatar
Animation Pipeline, Sign Language Dataset, Pose Estimation, Low-Latency
Inference, Human-Computer Interaction (HCI), Semantic Parsing, Language
Modeling, Phoneme Recognition, Audio Signal Processing, Accessibility
Technology, Inclusive Education Tools
Data availability • ASL Database for training the system models: [5]
• • Dataset 2: [6]

Metadata (type of Text, real-time audio, visual, images
data)

Model training In this real-time speech-to-sign language translation system, artificial
and fine-tuning intelligence and machine learning technologies are integrated to facili-
tate seamless communication. The process begins with automatic speech
recognition (ASR) models, such as Google Speech, which transcribe spoken
language into text. This textual data is then processed using natural language
processing (NLP) techniques, employing the BERT & T5 models, to under-
stand context and structure. Subsequently, sequence-to-sequence models,
often based on transformer architectures, translate the processed text into
sign language glosses. These guide the generation of corresponding sign
gestures, which are animated using 3D avatars. The avatars' movements are
driven by pose estimation and animation models trained on sign language
datasets, ensuring accurate and culturally appropriate representations. To
optimize performance for real-time applications, the system undergoes
fine-tuning through transfer learning and employs techniques like model
pruning and quantization, enhancing efficiency and responsiveness all with
the use of an internet connection.

Testbeds or pilot www .littlesigns .co .zw
deployments King Gorge The 4th Secondary school

2 Use Case Description

2�1 Description

Communication barriers between deaf individuals and hearing communities remain a
significant challenge worldwide. Spoken languages often exclude individuals who rely on
sign language for communication, limiting access to essential services, education, employment
opportunities, and social interactions. This challenge is amplified in environments requiring
rapid exchange of information, such as classrooms, workplaces, or emergencies. Real-time
speech-to-sign-language translators aim to bridge this gap, promoting inclusivity and fostering
a sense of belonging. Moreover, the philosophy that "disability is not an inability" reinforces
the importance of creating tools that enable individuals to fully express their potential without
limitations imposed by communication barriers.

620