Look! Now there are four little sisters dancing in front of you:
Think it is the work released by some anchors on the short video platform?
No,No,No . The real answer is: fake, generated, and only rely on a picture of the kind!
The real way to open it is like this:
This is the latest study from the National University of Singapore and byte beat, called MagicAnimate.
Its function can be summed up simply as a formula: a picture + a set of actions = a video with no sense of violation.
Then, as soon as this technology was announced, it made a lot of waves in the science and technology circle, and many technology bigwigs and geeks got out of the game one after another.
Even HuggingFace CTO tried it with his own avatar:
By the way, he made a funny joke:
This is a workout, right? I can skip the gym this week.
There are also netizens who keep pace with the times, playing with characters from the newly released GTA6 trailer:
Even emojis have become the objects of netizens' pick.
MagicAnimate can be said to have focused the attention of the technology circle on himself, so some netizens joked:
OpenAI can take a break.
Fire [thumb] is really fire.
A picture can generate a dance so popular MagicAnimate, how to "eat"?
Needless to say, let's try it hand in hand now.
At present, the project team has opened the page of online experience in HuggingFace:
The operation is also very simple, with only three steps:
Upload a still character photo
Upload the action demo video you want to generate
To adjust the parameters, click "Animate".
For example, here are my photos and a dance clip of "subject 3" that has recently swept the world:
△ video source: Douyin (ID:QC0217) can also select the template provided at the bottom of the page to experience:
It should be noted, however, that due to the current popularity of MagicAnimate, "downtime" may occur during the generation process:
Even if you succeed in "eating", you may have to stand in line.
That's right! As of the press release, still did not wait for the result! ) in addition, MagicAnimate also gives a way of local experience in GitHub. Interested partners can give it a try.
So the next question is:
How did you do that? Overall, MagicAnimate adopts a framework based on the Diffusion Model (diffusion), which aims to enhance time consistency, maintain the authenticity of reference images, and improve animation fidelity.
To this end, the team first developed a video diffusion model (Temporal Consistency Modeling) to encode time information.
This model encodes the time information by adding the time attention module to the diffusion network, so as to ensure the time consistency between the frames in the animation.
Second, in order to maintain the appearance consistency between frames, the team introduced a new appearance encoder (Appearance Encoder) to retain the complex details of the reference image.
Different from the previous methods using CLIP coding, this encoder can extract dense visual features to guide animation, so as to better retain identity, background, clothing and other information.
Based on these two innovative technologies, the team further adopted a simple video fusion technology (Video Fusion Technique) to promote the smooth transition of long video animation.
Finally, experiments on two benchmarks show that the result of MagicAnimate is much better than that of previous methods.
Especially on the challenging TikTok dance dataset, MagicAnimate is more than 38% higher than the strongest baseline in terms of video fidelity!
The qualitative comparison given by the team is as follows:
And compared with the SOTA baseline of cross-ID, the results are as follows:
One More Thing has to say that projects such as MagicAnimate have been a bit hot lately.
Well, shortly before its debut, the Ali team also released a project called Animate Anyone, as long as "one picture" and "desired action":
As a result, some netizens also raised questions:
This seems to be a war between MagicAnimate and AnimateAnyone. Who is better?
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
CTOnews.com, May 8, Nothing recently announced that the Nothing Phone (2) phone will be launched this summer, which is the second mobile phone product of Nothing and its first entry into the US market.
CTOnews.com, December 28 (Xinhua)-- although Tesla's share price suffered the worst plunge in history, it did not deter South Korean retail investors, who bought a net $2.8 billion worth of shares this year. Tuyuan Pexels despite the stock's popularity so far this month
Thank CTOnews.com netizens for the delivery of sweet peanuts! CTOnews.com August 13 news, Xiaomi service announced that the mobile phone screen maintenance quality upgrade, since August 14, Xiaomi / Redmi series mobile phone screen maintenance by 4
Thanks to CTOnews.com netizens Pianke Suohuang 4100 eyes, rain and snow on the way, Xiao Zhan cut, South China Daniel Wu, aurora meteor clue delivery! CTOnews.com May 25 news, Xiaomi mobile phone official Weibo has announced that Xiaomi Civi 3 hands
CTOnews.com, September 15 (Xinhua)-- Kaihua & Saifan Science Fiction Space announced the joint launch of the "wandering Earth 2"-themed mechanical keyboard, model MPE65, which will go on sale in mid-September 2023. According to reports, the key disk body is jointly determined with Kaihua.