Shulou(Shulou.com)12/24 Report--

CTOnews.com, December 20, Apple Machine Learning Research team recently released a blog post showing a new generative AI technology, HUGS, which can analyze short videos in 30 minutes and then map to create new actions and angles.

Anurag Ranjan, an Apple researcher, tweeted that HUGS, whose full name is Human Gaussian Splats, uses machine learning and computer vision to create realistic portrait elements with little raw input data.

According to Apple's official introduction, neural rendering technology has made great progress, but the most suitable scene is still photogrammetry of static scenes, which can not be extended to human beings who move freely in the environment.

HUGS uses 3D Gaussian Splatting technology to create mobile humans in the scene.

The method itself requires a small amount of subject video, usually moves in the scene, and displays as many surfaces as possible for the system to work.

In some cases, the technology requires very low source input data, at least as low as 50 to 100 frames of monocular video, equivalent to 2 to 4 seconds of 24fps video.

Apple claims that the system has been trained to "unlock static scenes and fully animated human avatars in 30 minutes".

Apple said that although the SMPL body model is used to initialize the human Gaussian model, it cannot capture every detail. For unmodeled elements, such as cloth and hair, this process can deviate from the SMPL model to fill the gaps in the model capture.

Apple officials say that from training videos to rendering speed output at 60fps, human body modeling and "state-of-the-art rendering quality" animation can be completed in half an hour, which is 100 times faster than other methods such as NeuMan and Vid2Avatar.

The reference address of the paper attached to CTOnews.com is as follows:

Project: https://machinelearning.apple.com/research/hugs

Xiv: https://arxiv.org/abs/2311.17910

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.