Image animation aims to generate video sequences such that the person in the source image is animated according to the motion of a video.
This technology lies within the field of computer vision, and AI researchers have been working on to produce more realistic videos. It leverages on machine learning to manipulate and generate visual images or videos that replaces a person with someone else’s.
For the love of science and from a research standpoint, Aliaksandr’s work is certainly impressive. It has been published in NeurIPS, and the source codes are available online.
His work outperforms state of the art on all the benchmarks, and it works on a variety of images (faces, body, cartoon and robot). The model is so flexible that you can create good quality Deepfakes with a single image of the target object.
Its ability to learn the facial movements is unbelievable. You can see that it can identify key points on the face, and it follows these key points to the movements in the video very well.
In previous works, we need additional information such as facial landmarks to map head movement and pose estimation to map full-body movement.
In this work, it can work without using any annotation or prior information about the specific object to animate. Once the model has trained on faces, the model can transfer any motion onto any faces.
You can record a video of yourself and animate the person in the photo. Yes, even a painting portrait of Mona Lisa.
You can look up and turn your head around. You can say something, and the mouth movements look great. You can roll your eyes, and it maps the eye movements nicely onto the target video.
It works for videos with full-body movement as well! Theoretically, this means that you can take the Billie Jean video and make Donald Trump do moonwalk like Michael Jackson.
As the person covers the part of the image, the algorithm needs to figure out the background behind the person. In this work, it automatically generates the background that is covered by the moving person — absolutely fantastic.
Aliaksandr work consists of the motion extractor which learns to extract key points along with their local affine transformations. There is a generator network that models occlusions in the target motions and combines the appearance extracted from the source image and the motion derived from the driving video.
To understand how it works, I suggest you visit the GitHub page and examine the research paper. You can also watch his video explaining how it works. Solid cool stuff.
Want to make your own? Check out this Colab notebook.
Deepfakes have garnered widespread attention for their uses in fake news, frauds, scams, and many other illegal activities.
People used to share their Deepfakes videos which they have created in the subreddit, r/deepfakes. Many of these videos are swapping celebrities faces, such as Gal Gadot and Taylor Swift, onto pornography performers’ bodies.
Many Deepfakes videos are also shared depicting politicians. It has affected politics by being authoritarian governments to spread false information, hate and fear.
This technology has concerned both industry and government to control and limit the use of Deepfakes. In February 2018, Reddit suspended r/deepfakes for violating policies. In June 2019, it elicited attention from the government to combat the spread of disinformation through the limitation of Deepfakes video alteration technology.
We build amazing solutions for different industries including in the financial, government and telecommunication sectors.
