Researchers in South Korea developed a wearable system that uses seven smart rings to read finger and hand motions to translate American Sign Language and International Sign Language into text. The purpose is to make communicating easier between those who sign and nonsigners without needing a separate human interpreter.
According to the study, published Friday in the journal Science Advances, the system reliably recognized 100 ASL and ISL words during testing. It also performed well with users the system had not seen before, and it didn’t require recalibration for each person. Because the system detects words in sequence, it can produce sentence-level translations without extra training on grammar.
ASL and ISL are the everyday languages of more than 72 million deaf and hard-of-hearing people. However, most hearing people do not know any words in these languages or have a very basic understanding. That gap makes certain tasks, like ordering at a restaurant or asking for help, much more difficult.
A concept of how the rings work in the real world.
Existing sign language translator prototypes often rely on bulky gloves that can distract from or block natural hand movement or feel uncomfortable for the wearer, which limits real word adaption. Camera-based technologies can work well in controlled environments but are often limited to those places where a camera can be set up with a clear line of sight, the researchers wrote.
To solve these problems, the researchers designed sensing rings for each finger that can capture precise motion and finger position while letting the hands move naturally. The rings can detect both signs that involve movement, like the words for “dance,” “fly” and “sun,” and signs that are held still, like “I” and “you.”
“These advances suggest that [the device could enable] barrier-free public translation systems for unseen users and unrestricted daily assistive interfaces,” the authors wrote in the study.
The authors are affiliated with Yonsei University, Hankuk University of Foreign Studies and the Korea Institute of Science and Technology, among others. While the technology is still experimental, the authors wrote that the technology has the potential to ease communication difficulties. The underlying idea could also help improve controls for other systems, like virtual or augmented reality.
“Beyond sign language translation, the ring-type, wireless, and modular architecture of (wirelessly connected, ring-type sign language translators) may also be extended to other gesture-driven applications such as virtual or augmented reality control, touchless device interfaces, or rehabilitation monitoring systems where fine-grained hand movement tracking is essential,” they wrote.