Novosibirsk State Technical University (NSTU NETI) has advanced a system designed to recognize and vocalize Russian sign language. The Interfax news agency reports findings from Mikhail Grif, a professor in NSTU NETI’s Department of Automatic Control Systems, describing a technology that could redefine accessibility for Deaf and hard-of-hearing communities.
The core idea is a mobile-ready translator that works in real time. A camera on a smartphone tracks continuous hand and body movements, while software analyzes these gestures to produce sign language output. In the opposite direction, spoken language can be converted into sign language gestures, enabling two-way communication. If everything goes as planned, this bidirectional translator could become available toward the end of 2024 or early 2025, opening new avenues for everyday conversations, education, and public services.
Grif emphasized the practical requirement for reliable performance: the system needs to recognize roughly five thousand distinct movements with about ninety percent accuracy to function smoothly across diverse conversations. At present, NSTU NETI has mapped and trained roughly four hundred movements, a significant step but far from the target. This gap highlights the complexity of sign languages, where subtleties in motion, timing, and trajectory carry meaning just as spoken words do.
As Grif noted, while the team has cataloged eight and a half thousand movements within various gesture dictionaries, real-world usage will demand a broader palette. The researchers project that the eventual vocabulary may approach ten thousand distinct gestures to cover common phrases, expressions, and domain-specific terms. This scale promises broad utility but also presents challenges in data collection, annotation, and ensuring consistent interpretation across users with different signing styles.
Another obstacle is the reverse translation—from voice to gestures. Ambiguity in speech, synonyms, and context can cause misinterpretations in the produced signs. Addressing these nuances will require sophisticated natural language processing, contextual awareness, and perhaps user-adjustable personalization to adapt to individual signing preferences and regional variations.
Earlier reports from Russia highlighted parallel momentum in autonomous technology, including the development of a new protection system for drones. While distinct in purpose, such projects collectively illustrate the rapid pace of defense, robotics, and accessibility technologies advancing within the country and the region.