The AI sonar glasses can read lips with about 95 percent accuracy

     Researchers at Cornell University have developed a pair of sonar glasses that use acoustic sensing and artificial intelligence to track lip and mouth movements and recognize up to 31 silent commands in a sequence. The system could help those who are temporarily unable to speak or have a voice.

     The glasses are a silent speech recognition interface called EchoSpeech. Equipped with a pair of microphones and speakers smaller than a pencil eraser, the glasses become a wearable AI-powered sonar system that sends and receives sound waves across the face and senses mouth movements. The deep learning algorithm then analyzes these echo profiles in real time with about 95 percent accuracy. The low-power, wearable glasses require only a few minutes of user training data to recognize commands and run on a smartphone.

     EchoSpeech can be used to communicate with others via smartphone in places where speaking is inconvenient or inappropriate, such as in a noisy restaurant or a quiet library. The silent voice interface also pairs with a stylus and works with design software such as CAD, requiring almost no keyboard or mouse.

     Researchers say they are moving sonar technology to the human body. Its small size, low power consumption and sensitivity to privacy are all important features for deploying new wearable technology in the real world. Most techniques in silent speech recognition are limited to a selected set of predetermined commands and require the user to face or wear a camera, which is neither practical nor feasible. Wearable cameras also pose significant privacy concerns, both for users and those who interact with them. Acoustic sensing technologies like EchoSpeech eliminate the need for wearable cameras.

     In addition, because audio data is much smaller than image or video data, EchoSpeech uses less bandwidth, transmits it to a smartphone in real time via Bluetooth, and keeps sensitive privacy information safe by staying local rather than in the cloud.

     Since the beginning of this year, artificial intelligence has once again attracted great attention. A few years ago, experts were discussing how to make artificial intelligence more accurate at understanding people's intentions and engaging in multiple rounds of conversation. Once upon a time, artificial intelligence reading lips was the stuff of science fiction movies. Today, this is fast becoming a reality. In the face of the unexpected changes brought by AI, people need to really think about how to better dominate AI, use it for the benefit of mankind, and avoid the trouble it may bring.