Functional sketch: Future hearing aids can read lips

September 12, 2022

Researchers at the University of Glasgow have developed a precise lip-reading system for wearers of protective masks that paves the way for a new generation of hearing aids. The approach combines radio frequency sensing with artificial intelligence for the first time to identify lip movements. It could, when integrated with conventional hearing aid technology, help combat the “cocktail party effect”. Hearing aid users often only understand “stationary” in a clamour of voices because they cannot identify the directions from which the individual voices are coming.
Privacy at risk

The problem could be solved if cameras were used to assist lip-reading, but collecting video footage of people without their explicit consent raises concerns about individual privacy. Cameras are also unable to read lips through masks, an everyday challenge for people who wear face coverings for cultural or religious purposes, and a broader problem in the age of COVID-19. For the development, researchers asked male and female volunteers to pronounce the five vowel sounds A, E, I, O and U with their lips uncovered and then repeat them while wearing a surgical mask.

In the process, radar sensors, which operate in the high-frequency range and therefore easily penetrate the masks, recorded the faces and lip movements. For comparison, they were also scanned while the lips remained immobile. The 3,600 data samples were used to teach machine learning and deep learning algorithms to recognise characteristic facial movements associated with each vowel sound. The hit rate achieved by the scientists was up to 95 per cent for uncovered lips and 83 per cent for covered lips.

Related Articles

Mobile Road Blocker M30 from Hörmann

Mobile Road Blocker M30 from Hörmann

Flexible and certified protection for events Public festivals, music events or Christmas markets - open-air events require appropriate security concepts to provide the best possible protection for the people on site. An important part of this concerns the protection...

Share This