Powered by Machine Learning, the Photorealistic Digital Human Can Answer Questions and Mimics Moods in Real Time; FMX Demo Running on Unreal Engine

Today at FMX, Oscar-winning VFX studio and leaders in digital human technology, Digital Domain, announces “Zoey,” the world’s most advanced autonomous human. Powered by machine learning and created using an advanced version of the technology and process that helped bring Thanos to the big screen, the photorealistic Zoey can engage in conversations with multiple participants at once, remember people, access the internet to answer questions and more, paving the way for the next step in the evolution of AI.

“With the potential of the metaverse and continued advancements in AI, the desire to interact with autonomous humans face-to-face is becoming more important and part of modern life,” said Daniel Seah, Global CEO of Digital Domain. “For decades, Digital Domain has been pioneering digital human technology, and Zoey takes the concepts of virtual assistants like Alexa and Siri several steps further, creating a digital helper you can truly interact with.”

Marketing Technology News: Venly Raises 21 Million Euros in Series A to Bring More Users Into Web3

Building on its proof-of-concept autonomous human, “Douglas,” Zoey is the result of years of research and development from the award-winning visual effects house, spearheaded by its internal Digital Humans Group. The physical appearance of Zoey is based on actress Zoey Moses, who worked with Digital Domain to create a comprehensive set of facial movements and mannerisms, as well as a full range of emotive expressions. Using that data and its proprietary facial animation tool “Charlatan” — recently seen in blockbuster films, including the Oscar-nominated Free Guy — artists were able to take the real footage of Moses and create a flexible digital face, capable of reacting in real time.

To give Zoey her personality, Digital Domain used multiple forms of machine learning to ensure she could understand most questions and formulate proper replies, and then access the internet (or stored data) for its answer. But rather than simply give a verbal or text response, Zoey can fully emote. If a question is confusing, she will look perplexed; if she is told a joke, she’ll smile. Throughout a conversation Zoey will also move and fidget in accordance with her answers, and even get annoyed when interrupted. She can engage with participants, asking them questions and opinions, and facial recognition software lets her recognize and remember people. Chatbots can also be added, giving Zoey the potential to quickly assume the role of an established virtual assistant for deployment in hospitality, the service industry and much more.

Marketing Technology News: MarTech Interview with Amanda Mountain, Global Vice President at SAP Digital Commerce

To provide Zoey with a robust and flexible voice system capable of more than just reciting scripted answers, Digital Domain employed AI-powered text-to-speech technology from WellSaid Labs. Using WellSaid Labs’ tools, Zoey can access a vast vocabulary of words and phrases, ensuring that no answer is beyond its capacity to vocalize. There are also multiple options to control the tone of her replies, including different levels of enthusiasm to reflect the goal of the interactions. For a retail setting, for instance, a more upbeat approach would be required, where a conversation would be more level. Zoey also has the ability to expand her speech capabilities by incorporating multiple language packs, making her fully bilingual.

Once Zoey is “educated” and ready to interact with the real world, she can then be added to multiple platforms, including custom systems designed by potential users, or game engines like Unity and Epic Games’ software. For today’s introductory presentation at FMX, Zoey can be seen running in real time on Epic Games’ Unreal Engine 4.

“For the last 30 years, Digital Domain has been one of the world leaders behind some of the most sophisticated and memorable visual effects ever seen, so it was a natural progression for us to become a leader in digital and virtual humans and to create ‘Zoey,’ the world’s most advanced autonomous human,” said John Fragomeni, global president of Digital Domain. “We will continue to push the boundaries of visual effects in all fields and on any screen, as we look for ways to offer people a better experience, regardless of the medium or delivery platform.”

Marketing Technology News: Why the Next Big Marketing Opportunity is: In-App Customer Service