Microsoft patent filing describes AI that rewrites speakers’ words in live video

Microsoft has applied for a US patent on a system that edits live video while it airs. AI agents pick out words in a speaker’s speech and cut or replace them, and a generative model redraws the speaker’s face and mouth to match the new text.

Microsoft filed the application, US 2026/0341309 A1, on 29 April 2025, and the USPTO published it on 29 October. The tracker Patentlyze spotted it first, and Futurism reported it on 1 November.

How the system works

An AI agent listens to the live audio and chooses which words to change “according to a first set of parameters and based on a profile of a user”, claim 1 says. The profile can hold your age, the languages you speak and your location. A generative model then swaps the words in both sound and picture, and reworks the moments just before and after each swap so you don’t see the join.

The filing puts “near-real-time” at a delay of a few seconds, roughly 1 to 10. Its targets include lectures, corporate presentations, award ceremonies, cooking shows and live sports. The processing can run on AR glasses, phones, tablets or cameras.

Replacement words may come from an “internet search”, and the inventors float on-the-fly fact-checking of the speaker. The filing lists what the system could change:

• filler words such as “um” and “ah”, and stutters;
• colloquial phrases, with “y’all” and “bite the bullet” as examples;
• mumbled, mispronounced and misused words;
• “offensive words, insensitive words, or inappropriate words”;
• speech in a language you don’t know, translated with the lips redrawn.

The safeguard is optional

Only a dependent claim says the system may store the original feed as “ground truth”. Claim 1 doesn’t require anyone to tell viewers that a speaker’s words changed, Patentlyze notes. The patent examiner has pushed back on the claims as filed, and Microsoft can answer by narrowing them. Microsoft did not respond to Futurism’s request for comment.

Microsoft ships parts of this

The Microsoft patent builds on products the company ships. The Interpreter agent in Teams translates meetings live and can simulate each speaker’s own voice.

In April 2024 Microsoft Research showed VASA-1, a model that animates a face from one photo and an audio clip at up to 40 frames per second. Microsoft called it a research demo and kept it unreleased, citing the risk of impersonation.

What it means for you

Crypto scammers use the same family of tools. TRM Labs found that AI use in crypto crime rose 40% in a year, with deepfake scams in the lead. In 2024 an Arup employee in Hong Kong paid out about $25 million after a video call with a deepfaked finance chief.

Many patent applications never turn into products. If edited live streams become normal, you will have less reason to trust a video of a founder announcing a token or an exchange promising a payout. Check such news on the project’s official channels before you send money. In the UK, actors campaign for legal ownership of your voice.

Patent details come from the published application as described by Patentlyze and Futurism. A filing does not mean Microsoft will release such a product.