Latest in Gaming

Image credit: CD Projekt Red

Cyberpunk 2077's dialogue was lip-synced by AI

Procedural generation provides accurate animations for 10 languages.
Kris Holt, @krisholt
October 20, 2020
415 Shares
Share
Tweet
Share

Sponsored Links

Cyberpunk 2077
CD Projekt Red

Cyberpunk 2077 is just a few weeks away and the action-packed RPG will launch with support for dialogue in 10 languages and subtitle options for several others. CD Projekt Red is aiming to add a deeper layer of immersion for players by using artificial intelligence to lip sync the dialogue in multiple languages.

The game’s lead character technical director, Mateusz Popławski, said in a presentation (via @shinobi602 on Twitter) that CDPR targeted better lip sync quality than in The Witcher 3. It wanted to do so in 10 languages: English, German, Spanish, French, Italian, Polish, Brazilian Portuguese, Russian, Mandarin and Japanese.

The goal was to do all of that for every character in the open world. Because of the game’s enormous scope, CDPR needed to do all of that with zero facial motion capture. To do so, the studio tapped into Jali Research’s lip syncing and facial animation tech to procedurally generate how the characters’ faces move.

Some of the characters in Cyberpunk 2077 speak multiple languages, sometimes switching between them in the same sentence. To account for that, transcript tagging was used. The tags also helped to adjust a character’s facial expressions when their emotional state changed within a line of dialogue. The system also used audio analysis to replicate the emotions of a vocal performance in the animations.

CDPR used algorithms to help it animate the cutscenes in The Witcher 3, and this is an evolution of that approach. Many games use motion capture to lip sync dialogue in just one language, which can break the immersion a bit for players who switch to a different one.

While the procedurally generated animations in Cyberpunk 2077 might not be quite as detailed or as expressive as those in some other AAA games, they could add to the experience for many players who’d prefer to play in different languages. You’ll be able to take a closer look at how that works in practice when the game arrives on November 19th.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
415 Shares
Share
Tweet
Share

Popular on Engadget

The 2020 Engadget Holiday Gift Guide

The 2020 Engadget Holiday Gift Guide

View
Amazon’s free news app on Fire TV now features local stations

Amazon’s free news app on Fire TV now features local stations

View
The first phone with an under-display camera goes on sale December 21st

The first phone with an under-display camera goes on sale December 21st

View
Apple will replace unresponsive iPhone 11 screens for free

Apple will replace unresponsive iPhone 11 screens for free

View
The best Cyber Week tech deals you can still get today

The best Cyber Week tech deals you can still get today

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr