New glove can translate sign language instantly through an app, researchers say
California researchers developed a glove embedded with electronic sensors that can translate American Sign Language into English in real time through an app on your smartphone.
Like most wearable technologies, past devices used to translate ASL are heavy, bulky and uncomfortable to wear, but the new glove is lightweight, inexpensive and “long-lasting,” according to a news release from the University of California, Los Angeles.
The device could help eliminate communication barriers between signers and non-signers, a study published Monday in the journal Nature Electronics says.
“Our hope is that this opens up an easy way for people who use sign language to communicate directly with non-signers without needing someone else to translate for them,” Jun Chen, an assistant professor of bioengineering at the UCLA Samueli School of Engineering and the principal investigator on the research, said in the news release.
“In addition, we hope it can help more people learn sign language themselves.”
Sign language is the use of gestures, hand shapes and facial expressions to convey concepts and ideas, according to StartASL, and is considered the third most commonly used language in America.
About 500,000 people who are deaf and hard of hearing in the U.S. and Canada use ASL to communicate, but the majority of hearing people have not learned the language.
The team worked with four people who are deaf and use ASL and had them repeat gestures up to 15 times while wearing the glove-like device, simultaneously teaching a machine how to recognize them.
The glove has thin, stretchable sensors made from “electrically conducting yarns” that run up each finger and convert the movements into signals, the researchers said.
These signals are sent to a “dollar-coin-sized” circuit board placed on the person’s wrist, which then wirelessly transfers over to a smartphone.
The system was able to recognize 660 hand motions and finger placements that stand for each letter in the English alphabet, different phrases and each number, according to the release.
The algorithm recognized the correct signs about 99% of the time, and in less than one second — or about at one word per second, the study said.
The researchers also placed sensors in between the participants’ eyebrows and on the sides of their mouths to “capture facial expressions that are a part of American Sign Language,” according to the release.
Facial expressions play a significant role in communicating certain feelings in sign language, just how tone and volume can convey meanings in other languages.
“The same exact hand-shape and movement can totally change meaning because of the facial expression that is used to accompany it,” according to Signing Savvy. “One example of this is the word MUCH. The degree of how much can totally be determined by the facial expression alone while the sign stays the same.”
Chen said the glove would need to learn more vocabulary and be able to translate signs faster in order to get a commercial version of it in stores.
The UCLA team said it has filed a patent on the technology.
This story was originally published July 1, 2020 at 3:20 PM with the headline "New glove can translate sign language instantly through an app, researchers say."
