Microsoft has filed a patent for AI tech that could change what people say during live broadcasts in near real time, then alter their lip movements to match.
The filing, published on October 29, describes a system that scans live video and audio, picks out words to change, and swaps them for new ones before viewers see the footage.
Microsoft patent swaps words and redraws lips
According to the filing, as detailed by Patentlyze, an AI agent identifies spoken words to modify based on the profile of the person watching, such as their age, location, and the languages they speak.
A generative AI system then replaces those words in the audio, smooths over the edits so they sound natural, and generates new video frames so the speaker’s mouth matches the new words.
The patent says this would happen in “near-real-time,” which Patentlyze says would mean a delay of a few seconds. The system could also run on phones, tablets, cameras and AR glasses.
Patent lists translation and profanity filters
The filing gives several examples of how it could be used, including cutting out filler words like “um,” replacing profanity or offensive language, and fixing slang, mumbled or mispronounced words.
Replacement words could even be pulled from an internet search, which Patentlyze says could allow for instant fact-checking.
A separate section covers translation, where words spoken in a language the viewer doesn’t understand are swapped for translated words, with the speaker’s lips redrawn to match.
Because edits are tied to each viewer’s profile, two people watching the same broadcast could hear different versions of what was said.
The patent also describes using AI to condense long recordings into shorter summary videos, generating new footage of the speakers saying a summarized script.
As with any patent, there’s no guarantee Microsoft will turn it into a real product.
I’m Abhishek Sharma, an author at TigerJek.com. I enjoy exploring games, testing different strategies, and turning what I learn into clear, useful guides. My goal is to help players understand the game better and improve without the usual confusion.
