Tag: deepfake

China has a flourishing market for deepfakes that clone the dead

[ad_1]

Deepfake technologies have evolved to the point where it’s now easy and affordable to clone people’s looks and voices with AI. Meanwhile, large language models mean it’s more feasible than ever before to conduct full conversations with AI chatbots.

I just published a story today about the burgeoning market in China for applying these advances to re-create deceased family members. Thousands of grieving individuals have started turning to dead relatives’ digital avatars for conversations and comfort.

It’s a modern twist on a cultural tradition of talking to the dead, whether at their tombs, during funeral rituals, or in front of their memorial portraits. Chinese people have always liked to tell lost loved ones what has happened since they passed away. But what if the dead could talk back? This is the proposition of at least half a dozen Chinese companies offering “AI resurrection” services. The products, costing a few hundred to a few thousand dollars, are lifelike avatars, accessed in an app or on a tablet, that let people interact with the dead as if they were still alive.

I talked to two Chinese companies that, combined, have provided this service for over 2,000 clients. They describe a growing market of people accepting the technology. Their customers usually look to the products to help them process their grief.

To read more about how these products work and the potential implications of the technology, go here.

However, what I didn’t get into in the story is that the same technology used to clone the dead has also been used in other interesting ways.

For one, this process is being applied not just to private individuals, but also to public figures. Sima Huapeng, CEO and cofounder of the Chinese company Silicon Intelligence, tells me that about one-third of the “AI resurrection” cases he has worked on involve making avatars of dead Chinese writers, thinkers, celebrities, and religious leaders. The generated product is not intended for personal mourning but more for public education or memorial purposes.

Last year, Silicon Intelligence replicated Mei Lanfang, a renowned Peking opera singer born in 1894. The avatar of Mei was commissioned to address a 2023 Peking opera festival held in his hometown, Taizhou. Mei talked about seeing how drastically Taizhou had changed through modern urban development, even though the real artist died in 1961.

[ad_2]

Source link

May 8, 2024
Deepfakes of the dead are a growing Chinese business

[ad_1]

In Chinese homes, it’s common to put up a portrait of a deceased relative for a few years after the death. Zhang Zewei, founder of a Shanghai-based company called Super Brain, says he and his team wanted to revamp that tradition with an “AI photo frame.” They create avatars of deceased loved ones that are pre-loaded onto an Android tablet, which looks like a photo frame when standing up. Clients can choose a moving image that speaks words drawn from an offline database or from an LLM.

“In its essence, it’s not much different from a traditional portrait, except that it’s interactive,” Zhang says.

Zhang says the company has made digital replicas for over 1,000 clients since March 2023 and charges $700 to $1,400, depending on the service purchased. The company plans to release an app-only product soon, so that users can access the avatars on their phones, and could further reduce the cost to around $140.

The purpose of his products, Zhang says, is therapeutic. “When you really miss someone or need consolation during certain holidays, you can talk to the artificial living and heal your inner wounds,” he says.

And even if that conversation is largely one-sided, that’s in keeping with a strong cultural tradition. Every April during the Qingming festival, Chinese people sweep the tombs of their ancestors, burn joss sticks and fake paper money, and tell them what has happened in the past year. Of course, those conversations have always been one-way.

But that’s not the case for all Super Brain services. The company also offers deepfaked video calls in which a company employee or a contract therapist pretends to be the relative who passed away. Using DeepFace, an open-source tool that analyzes facial features, the deceased person’s face is reconstructed in 3D and swapped in for the live person’s face with a real-time filter.

Example of a deepfake video call Super Brain did in July 2023. The face in the top right corner is from the deceased son of the woman.
SUPER BRAIN

At the other end of the call is usually an elderly family member who may not know that the relative has died—and whose family has arranged the conversation as a ruse.

Jonathan Yang, a Nanjing resident who works in the tech industry, paid for this service in September 2023. His uncle died in a construction accident, but the family hesitated to tell Yang’s grandmother, who is 93 and in poor health. They worried that she wouldn’t survive the devastating news.

[ad_2]

Source link

May 7, 2024
An AI startup made a hyperrealistic deepfake of me that’s so good it’s scary

[ad_1]

The more data points the AI system has on facial movements, microexpressions, head tilts, blinks, shrugs, and hand waves, the more realistic the avatar will be.
DAVID VINTINER

He then asks me to read a script for a fictitious YouTuber in different tones, directing me on the spectrum of emotions I should convey. First I’m supposed to read it in a neutral, informative way, then in an encouraging way, an annoyed and complain-y way, and finally an excited, convincing way.

“Hey, everyone—welcome back to Elevate Her with your host, Jess Mars. It’s great to have you here. We’re about to take on a topic that’s pretty delicate and honestly hits close to home—dealing with criticism in our spiritual journey,” I read off the teleprompter, simultaneously trying to visualize ranting about something to my partner during the complain-y version. “No matter where you look, it feels like there’s always a critical voice ready to chime in, doesn’t it?”

Don’t be garbage, don’t be garbage, don’t be garbage.

“That was really good. I was watching it and I was like, ‘Well, this is true. She’s definitely complaining,’” Oshinyemi says, encouragingly. Next time, maybe add some judgment, he suggests.

We film several takes featuring different variations of the script. In some versions I’m allowed to move my hands around. In others, Oshinyemi asks me to hold a metal pin between my fingers as I do. This is to test the “edges” of the technology’s capabilities when it comes to communicating with hands, Oshinyemi says.

Historically, making AI avatars look natural and matching mouth movements to speech has been a very difficult challenge, says David Barber, a professor of machine learning at University College London who is not involved in Synthesia’s work. That is because the problem goes far beyond mouth movements; you have to think about eyebrows, all the muscles in the face, shoulder shrugs, and the numerous different small movements that humans use to express themselves.

The motion capture process uses reference patterns to help align footage captured from multiple angles around the subject.
DAVID VINTINER

Synthesia has worked with actors to train its models since 2020, and their doubles make up the 225 stock avatars that are available for customers to animate with their own scripts. But to train its latest generation of avatars, Synthesia needed more data; it has spent the past year working with around 1,000 professional actors in London and New York. (Synthesia says it does not sell the data it collects, although it does release some of it for academic research purposes.)

The actors previously got paid each time their avatar was used, but now the company pays them an up-front fee to train the AI model. Synthesia uses their avatars for three years, at which point actors are asked if they want to renew their contracts. If so, they come into the studio to make a new avatar. If not, the company will delete their data. Synthesia’s enterprise customers can also generate their own custom avatars by sending someone into the studio to do much of what I’m doing.

[ad_2]

Source link

April 25, 2024