Functional sketch: Future hearing aids can read lips

September 12, 2022

Researchers at the University of Glasgow http://gla.ac.uk have developed a precise lip-reading system for wearers of protective masks that paves the way for a new generation of hearing aids. The approach combines radio frequency sensing with artificial intelligence for the first time to identify lip movements. It could, when integrated with conventional hearing aid technology, help combat the “cocktail party effect”. Hearing aid users often only understand “stationary” in a clamour of voices because they cannot identify the directions from which the individual voices are coming.
Privacy at risk

The problem could be solved if cameras were used to assist lip-reading, but collecting video footage of people without their explicit consent raises concerns about individual privacy. Cameras are also unable to read lips through masks, an everyday challenge for people who wear face coverings for cultural or religious purposes, and a broader problem in the age of COVID-19. For the development, researchers asked male and female volunteers to pronounce the five vowel sounds A, E, I, O and U with their lips uncovered and then repeat them while wearing a surgical mask.

In the process, radar sensors, which operate in the high-frequency range and therefore easily penetrate the masks, recorded the faces and lip movements. For comparison, they were also scanned while the lips remained immobile. The 3,600 data samples were used to teach machine learning and deep learning algorithms to recognise characteristic facial movements associated with each vowel sound. The hit rate achieved by the scientists was up to 95 per cent for uncovered lips and 83 per cent for covered lips.

Related Articles

BITKOM: Many use AI, but few pay for it

BITKOM: Many use AI, but few pay for it

8 percent of users of generative AI use paid services 6 out of 10 want to continue using free services in the future AI is now used to create images almost as often as text Free services currently dominate the artificial intelligence market in Germany. Only 10 percent...

Recession keeps Germany in a stranglehold

Recession keeps Germany in a stranglehold

Economic forecast by the German Economic Institute predicts contraction of 0.2 per cent According to the latest economic forecast in German language by the German Economic Institute (IW), the German economy will shrink by 0.2 percent this year. Experts attribute the...

Share This