Nvidia unveils the new AI tool Magic3D: automatically renders 3D models based on text

CTOnews.com November 22, last Friday, NVIDIA launched a new AI tool, Magic3D, which can automatically generate 3D models based on text descriptions. For example, after a user enters "a blue poison dart frog sitting on a water lily", Magic3D can generate a 3D mesh model with a colored texture in about 40 minutes.

Nvidia pitted Magic3D against Google's DreamFusion. DreamFusion is a text-to-3D model AI tool announced by Google researchers in September. Similar to the way DreamFusion uses a text-to-image model to generate a 2D image and then optimizes it to volume NeRF (neural radiation field) data, the rendering process of Magic3D is divided into two steps, optimizing the rough model generated at low resolution to high resolution.

Magic3D can also make prompt-based edits to 3D models. Given a low-resolution 3D model and a basic hint, you can change the text to change the resulting model. In addition, the authors of Magic3D show themes that have been retained in past versions (a concept often referred to as consistency) and apply the style of two-dimensional images (such as cubist painting) to 3D models.

CTOnews.com learned that Nvidia did not publish any Magic3D code with its academic papers.