Page 771 - AI for Good Innovate for Impact

AI for Good Innovate for Impact 4.9: Accessibility

Figure 1: Proposed Data Collection Pipeline
• REQ-02: It must handle temporal differences between consecutive signs, including when
to update the predicted letter on the screen.
• REQ-03: The model should join letters to form words and verify if the word belongs to a
dictionary.
• REQ-04: The system should understand basic key statement expressions in sign language.
• REQ-05: The system should use pose estimation and embedding representations for
better accuracy.
• REQ-06: The AI model should support translation in local languages such as Hindi and
English.
• REQ-07: Comparing outputs of existing Transformer and CNN models with our model
to understand their performance differences using interpretability techniques, finding
feature importance in each of the models.
• REQ-08: Develop a pipeline by addressing regional variations, linguistic nuances, and
real-world usage scenarios.
• REQ-09: Preserve the context and emotion of the message to be conveyed throughout
the translation and identification process.

4 Sequence Diagram

The following Diagram (High-Level System Architecture for ISL Detection and Translation)
Illustrates the proposed System Pipeline:

735