93 points by sinak 7 days ago
Does anyone know what models / software this person is using? I assume it's something off the shelf, because there are an abundance of other videos in this space produced by different people:
I've been doing ML for audio (Tacotron, WaveRNN, etc.), but haven't stepped into the video world. I also work in film and want to apply this technique to my work.
How do they keep the frames from blurring or not needing to be rekeyed? The motion tracking is spot-on.
Would someone operating in this space be open to meeting over Hangouts or Skype to discuss the tech? I'd be happy to pay someone for a short survey of the space.
Pretty sure most of these crew are using DeepFaceLab, https://github.com/iperov/DeepFaceLab
This was in the news for being the first(?) public GitHub repo that you needed an account to access.
I believe to try and stem the tide of deepfake revenge porn or celebrity porn.
The shots in the gallery aren't nearly as impressive, though...
There's something on this user in this Guardian article : https://www.theguardian.com/news/shortcuts/2019/aug/13/dange...
This video is helped by some superficial similarity between Hader and Cruise, and the transition to and from Seth Rogen's face is slightly less smooth. Still, this is incredibly well done.
The unsettling part is how fast the quality of deep fakes seems to have improved, to me that's unsettling.
The Arnold Schwarzenegger video from the same guys is arguably better: https://www.youtube.com/watch?v=bPhUhypV27w
Why have they not made the actual video they discuss with a baby's body in a diaper with Arnie's head on it. That would be classic.
This is the first deep fake I've seen that was good enough to fool a random Facebook user that wasn't looking for fakeness. Cool how far it's come.
Your friend was fooled to think that Hader did that on his own?
I'm technologically experienced enough to be posting and reading on HN.
It fooled me on the first view, I just didn't even see it, it was seamless.
Me as well, I thought to my self 'that dude has some resemblance to Tom Cruise' for a second and the next moment the similarity had dissapeared and I fully dismissed my initial reaction since he afterwards had no real resemblance.. Dismissed it and thought my mind had played a trick on me because of sleepiness or something. Only after glancing the video comments (after first full view) did I understand what was going on.
Same. I came across this clip a few days ago without the context.
It's a little embarrassing frankly. I kept thinking that he's such a good impersonator that he's able to manipulate his facial muscles in just such a way as to embody Tom Cruise.
My defenses are now up a bit higher.
Hypothetically, I meant, if someone were to style their hair as Cruise and did an entire video impersonating his speech - it would be convincing it was him
Same creator as "Bill Hader impersonates Arnold Schwarzenegger", which I think is easier to see the shift in since there's less resemblance and higher quality: https://www.youtube.com/watch?v=bPhUhypV27w
I think the subtlety of Hader/Cruise makes it even more disturbing.
What's disturbing is that people think he's literally shape shifting. Or they are trolling and I am getting too affected by it.
Same user also did the Jim Carrey in Shining deepfake which is as well done as this and at times really unsettling. From now on I can safely say that I can't trust any video footage I see. This is that well done. The algos will only get better and do a convincing blend in 4K in the future.
Those The Shining clips are simply the best. Here is another that always make me chuckle:
https://youtu.be/aUphMqs1vFw [Full House of Mustaches]
Thank you for that link. As a kid growing up in the 90s, that is just pure gold. Cant believe it only has 3000 views...
It is a repost as YouTube removed the original.
Once that deep fake technology algorithm gets in the hands of the masses (Adobe Premiere After Effects plugin, Instagram/Snapchat filter, Apple Facetime filter, etc) ... so many obvious directions it will take:
- dating / flirting apps
- job candidates being interviewed remotely via webcams will use software to make them look younger
- "facelift" every television personality (news anchors, etc) and every movie actor. Both for creative purposes (younger/older timeline aging), for vanity, and for extending careers.
- generate fake but convincing political ads to put words in the mouth of opponent that he/she never said
- a growing distrust and skepticism of all videos including archive footage because we never had SHA256 authenticated hash of the old videos widely disseminated before deepfakes became ultrarealistic. E.g. There will be a subset of tomorrows kids that will not be convinced JFK ever actually said, "landing a man on the moon". The WTC towers were never hit by airplanes. All that old footage will be dismissed as fake. Video evidence becomes useless. Yes, society will try to counteract it with software the detects deep fakes but some of the public still won't be convinced. (Same as evidence against anti-vaccine being rejected.)
1. We don't have videos of The Gettysburg address. History existed before video.
2. There will never be another generation (in the near future) like boomers who didn't grow up tech savvy
3. Videos, like any form of documentation, are anecdotal evidence easily manipulated by what's "not in the frame"
4. The general problem you are describing comes from a faulty attribution of truth to video in our current day and age. The sooner we dismiss that the better
Yes, the transition will be awkward, but no need for hysteria.
I don't know if there's need for hysteria, but I think this is a much bigger change than you're making it out to be. It's a different league than artful editing and framing.
> 2. There will never be another generation (in the near future) like boomers who didn't grow up tech savvy
You're living in a bubble.
On the first day, God created The Deepfake Algorithm. On days 2-6, he waited patiently as the The Algorithm auto-generated history up to and including yesterday. Day 7 is today.
There are only 7 days because on Day 8 God saw the AWS bill and realized the unit economics were doomed from the start.
> - growing distrust and skepticism of all videos including archive footage
That's an excellent, and very terrifying point. Governments are going to have a hay-day with misinformation campaigns to rewrite history (even more than they do now).
I've been thinking a lot about this one, and it gets a ton of help in the fact that Bill Hader is actually good at facial impressions, and that's what really sells it.
Yeah, he really nails Tom Cruise's mannerisms when he shakes his head at 1:27.
Directors can use any actor to portray any character, and then make the character appear any way they want. Decouples acting talent from appearance.
Peter Jackson's Bolg and Azog could have been much more convincing, just 7 years on.
It isn't too hard to image a future where actors no longer sell their skills but rather just their name and likeness. And – similar to photoshop or autotune — expressions, dialog, posture and everything else can be edited as needed.
I would guess the opposite: the surface appearance is the easy part; articulating human mannerisms in a way that's not just convincing, but compelling, will remain one of the very hardest things for computers to replicate.
Acting, as a craft, only ever had coincidental ties to physical appearance. What it really is is the talent of being human. I think what we'll see is a lot of pretty faces who aren't actually that good at acting exiting the business, while upcoming talent won't have its career limited by birth attributes. It should be very interesting.
How does one become a name actor then?
Assuming you mean they build up a name and then license their image ... I'm not sure you've met an actor before. This goes against every impulse of a great actor. They're completely obsessed with controlling their image and practicing their craft.
I think you may have meant celebrities.
>They're completely obsessed with controlling their image and practicing their craft.
I'm sure textile workers said the same thing before their skill and craft was replaced by a machine. Every artist and artisan thinks they're irreplaceable. Currently, many programmers think theirs is the only industry that can't be automated because there's some magical golden juice in their minds that produces some creative and intellectual quality no machine will ever be able to replicate. Like textile workers of old, their time will come too.
As much as actors may want their work to be art, their work is also product. When it becomes possible to manufacture an acceptable equivalent product through automation, then artists of all stripes will simply have to adapt to the new terms of the market like everyone else, or find other work. No one is going to care about the effort an actor puts into their performance when it becomes feasible to pay a few dollars to stream Marlon Brando into the original Star Wars.
Elite actors of the world are extremely rare, talented individuals. They compete in one of the most cut-throat fields I can think of. How is that analogous to a textile worker?
As long as there are humans, then actors, musicians, dancers, etc will be fine. People are fascinated with artists because they demonstrate a spectacle of skill and a genius which we don't have.
>Elite actors of the world are extremely rare, talented individuals. They compete in one of the most cut-throat fields I can think of. How is that analogous to a textile worker?
The textile workers replaced by automation were also once extremely rare, talented individuals. Rugs were once prized works of art, infused with culture and artisanal techniques passed down through generations. Now you can buy them at Wal-Mart. Unfortunately, the post-industrial market doesn't optimize for skill, it optimizes against it.
>People are fascinated with artists because they demonstrate a spectacle of skill and a genius which we don't have.
Most people aren't fascinated with skill and genius, they're fascinated with celebrity. Art isn't made to be appreciated nowadays, it's mass produced to be consumed like fast food, with little regard beyond the immediate satisfaction it provides, and the familiar comfort of mediocrity.
Sure, there will always be a place for human artists, dancers, musicians, etc, just as there is still a place for people who can weave rugs. But I do believe that most mainstream art will be automated as soon as the result is acceptable to the mainstream, and that includes automating the persona of the artist as well.
Artisanal textile workers, categorically, were not as talented as elite actors because the size of their competitive pool and the barriers to creating work were minuscule compared to that of Hollywood actors - who emigrate from all over the globe and assimilate to US culture to use their talent.
While a lot of cultural products (not art) like movies and shows are created like fast food, people are drawn to them because of the unique talent, charisma, looks, presence etc of the actors that participate. I don't see Avengers: End Game breaking box office records if it didn't have those rarefied individuals.
If you go back to my initial post, I pondered whether the original comment meant celebrity.
You mean like Robin Wright in The Congress
Interestingly in that movie, it was Robin Wright and Tom Cruise who were the very last actors to hold out on selling their likeness for entirely generated films.
Or maybe imagine a future were actors only sell their skills, since it's decoupled from their face, race, gender, and other appearance related factors.
This is the better future. It opens the door to actors without favoring those that have "the look".
Physical features will be cosmetic and plastic, and they will be engineered to match the desired setting.
there are thousands of dead actors to resurrect. but more importantly nobody can be as perfect as an artificially drawn or generated face
Directors will be bored with all of the one take wonders. Maybe they'll have just go ahead an eliminate the director position as well.
Youtube kids will use that to turn YT to hollywood
Some of the tracking is still janky, but I'm surprised by the transitions - I can see when they shift, but I don't see any weird morph.
I think our brains are helping the transition along, given that before and after are both believably configured faces.
Totally, but that's exact same for our brain fooling us about the motion itself. And in motions case our brain throws in interpolated middle frames, not so in the face morph.
Is the (poor) video quality intentional? I'm curious if that is used to disguise the effects, or just due to the original video.
I believe it is due to the quality of the original video: https://www.youtube.com/watch?v=zS1Aee2X3Yc
Fake images are often downsampled. I assume the same reasons apply here.
These are super cool, but it makes me wonder how much longer before we get to an era where video evidence is no-longer admissible in court.
Yeah, you can see occasional artifacts and glitches with current deepfakes, but surely the software will only get better.
Eh, photoshop and PDF editors exist and we can still use documents and photos as evidence.
But, there will definitely be a new cohort of expert witnesses with software tools (functional or just appearing so) to testify about the veracity or video.
Clipping a video, reordering scenes, or reversing it, or just showing it in the wrong context can be enough to manipulate the viewer.
We should add integrity checks and digital signatures to videos.
That's an interesting idea; how would you go about suggesting we do that? Have some kind of hardware-signing thing that we put into video cameras, and sign every keyframe with it before its written to the SD card?
Reusing concepts from existing DRM technology.
I feel like it was just last year when some deep fakes were coming out and people waved them off as having obvious artifacts and too easy to ignore
I thought these effects were going to be real fun, but it's kind of spooky and scary now. Perhaps good for horror movies.
I don't see any change throughout the video. I don't see tom cruise even once. Where is he?
Tom Cruise was the source video. His face has been replaced by Bill Hader.
Ah, I've never seen the source video so that's why I didn't pick up on it.
I can't tell if you both are joking or not but watch the whole video. The source video is of Bill Hader. Bill hader's face changes to Tom Cruise everytime he talks from Tom's perspective. And then once to Seth Rogan for that impersonation.
I honestly can't see much change in the face throughout the video.
It probably doesn't help that I am so horrible at celebrities that I couldn't picture Tom Cruise or Bill Hader in my head if I tried, nor do I know what they're famous for. I just know that Tom Cruise sounds familiar and everyone seems to act like Bill Hader is significant so I go along with it.
I guess there's a some change in the face when he switches "voices". It's not nearly as exciting as the one of Obama saying stuff he's never said before though.
If you know Bill Hader's face, the change to Cruise is pretty obvious. The Cruise resemblance is subtle but strong.
Part of the point with fakes like this is it's not just gluing a head on someone else's body, like a bad photoshop. It's actually changing the facial features of the target to resemble someone else.
You’re correct. If you listen to what he’s actually saying it’s told from the perspective of Hader and it only shifts when he’s doing an impression of Cruise or Rogan.
Ah, I didn't listen to it with audio. My mistake!
As an aside, I feel really weird having completely convinced myself that the original video was deepfaked. It's kind of unnerving given the subject matter.
Only on LSD have I seen faces change that seamlessly. There were a couple of spots that I noticed the transition, but that was like the 3rd viewing. I hope I never piss off someone with these abilities to make me say/do things I'd never imagine. The damage that can be done is mind boggling when used for that purpose.
Photos do not lie, it is said. But frequently they tell partial
truths. They tear moments out of time's flowing river and separate
them from what has come before and what will follow. They can
misrepresent as well as represent.
- JIM HOAGLAND
And that's before we even begin to discuss intentional manipulation.
I liked this Tom Cruise Deepfake by Node from a month back: https://www.youtube.com/watch?v=3vHvOyZ0GbY