Functional sketch: Future hearing aids can read lips

September 12, 2022

Researchers at the University of Glasgow http://gla.ac.uk have developed a precise lip-reading system for wearers of protective masks that paves the way for a new generation of hearing aids. The approach combines radio frequency sensing with artificial intelligence for the first time to identify lip movements. It could, when integrated with conventional hearing aid technology, help combat the “cocktail party effect”. Hearing aid users often only understand “stationary” in a clamour of voices because they cannot identify the directions from which the individual voices are coming.
Privacy at risk

The problem could be solved if cameras were used to assist lip-reading, but collecting video footage of people without their explicit consent raises concerns about individual privacy. Cameras are also unable to read lips through masks, an everyday challenge for people who wear face coverings for cultural or religious purposes, and a broader problem in the age of COVID-19. For the development, researchers asked male and female volunteers to pronounce the five vowel sounds A, E, I, O and U with their lips uncovered and then repeat them while wearing a surgical mask.

In the process, radar sensors, which operate in the high-frequency range and therefore easily penetrate the masks, recorded the faces and lip movements. For comparison, they were also scanned while the lips remained immobile. The 3,600 data samples were used to teach machine learning and deep learning algorithms to recognise characteristic facial movements associated with each vowel sound. The hit rate achieved by the scientists was up to 95 per cent for uncovered lips and 83 per cent for covered lips.

Related Articles

Euralarm releases new white paper on fire alarm sensors

Euralarm releases new white paper on fire alarm sensors

Euralarm has published a White Paper on multi-sensor fire detectors and how these devices can help to reduce false alarms. The document is intended for fire safety professionals, building managers, and regulatory authorities. Fire detection is a critical component of...

Face recognition 2.0 from a great distance

Face recognition 2.0 from a great distance

LiDAR system from researchers at Heriot-Watt University impresses with extremely high resolution: Comparison of a LiDAR image with the original (Photo: Aongus McCarthy, hw.ac.uk) In the future, it will be possible to recognise a face from a distance of hundreds of...

‘SUPER’ races safely through treacherous terrain

‘SUPER’ races safely through treacherous terrain

Drones developed by engineers at the University of Hong Kong use LiDAR technology to orient themselves ‘SUPER’ is what roboticists at the University of Hong Kong (https://www.hku.hk/ ) call their new flying robot, which is designed to move through unfamiliar terrain...

Share This