Functional sketch: Future hearing aids can read lips

September 12, 2022

Researchers at the University of Glasgow http://gla.ac.uk have developed a precise lip-reading system for wearers of protective masks that paves the way for a new generation of hearing aids. The approach combines radio frequency sensing with artificial intelligence for the first time to identify lip movements. It could, when integrated with conventional hearing aid technology, help combat the “cocktail party effect”. Hearing aid users often only understand “stationary” in a clamour of voices because they cannot identify the directions from which the individual voices are coming.
Privacy at risk

The problem could be solved if cameras were used to assist lip-reading, but collecting video footage of people without their explicit consent raises concerns about individual privacy. Cameras are also unable to read lips through masks, an everyday challenge for people who wear face coverings for cultural or religious purposes, and a broader problem in the age of COVID-19. For the development, researchers asked male and female volunteers to pronounce the five vowel sounds A, E, I, O and U with their lips uncovered and then repeat them while wearing a surgical mask.

In the process, radar sensors, which operate in the high-frequency range and therefore easily penetrate the masks, recorded the faces and lip movements. For comparison, they were also scanned while the lips remained immobile. The 3,600 data samples were used to teach machine learning and deep learning algorithms to recognise characteristic facial movements associated with each vowel sound. The hit rate achieved by the scientists was up to 95 per cent for uncovered lips and 83 per cent for covered lips.

Related Articles

“E-wallet”: US banks arm themselves against PayPal

“E-wallet”: US banks arm themselves against PayPal

Joint digital payment system also to compete with providers such as Apple Pay: The largest banks in the USA want to jointly establish an electronic wallet to take on competitors such as Apple Pay (https://www.apple.com/at/apple-pay/) and PayPal...

New law for critical infrastructures

New law for critical infrastructures

KÖTTER Security calls for greater consideration of private security service providers Acts of sabotage against railway lines and cyber-attacks on public institutions have recently brought security for critical infrastructures (CRITIS) back into the public focus....

Walking stick 2.0 for the visually impaired finds muesli

Walking stick 2.0 for the visually impaired finds muesli

An intelligent walking stick developed by researchers at the University of Colorado Boulder (https://www.colorado.edu) is designed to make life easier for blind and visually impaired people. It is equipped with a camera and software that can evaluate images. In...