Tuesday, December 3, 2024
ad
HomeNewsUniversity of Hong Kong Introduces Text2NeRF, an AI Framework that Turns Text...

University of Hong Kong Introduces Text2NeRF, an AI Framework that Turns Text Descriptions into 3D Scenes

Text2NeRF can generate realistic texture and delicate geometric shapes in 3D scenes.

University of Hong Kong has introduced Text2NeRF, a text-driven 3D scene synthesis system that combines the Neural Radiance Field (NeRF) and the best characteristics of a trained text-to-image diffusion model. 

Researchers picked NeRF as the 3D representation because of its superiority in modeling fine-grained and lifelike characteristics in a variety of circumstances, which may significantly reduce the artifacts generated by a triangular mesh. They replaced older methods like DreamFusion, which used semantic priors to govern the 3D creation, with finer-grained picture priors inferred from the diffusion model. 

Because of this, Text2NeRF can generate realistic texture and delicate geometric shapes in 3D scenes. A pre-trained text-to-image diffusion model is used as the image-level prior, and they constrain the NeRF optimization from scratch without the requirement for additional 3D supervision or multiview training data. 

Read More: Microsoft Announces AI Personal Assistant Windows Copilot for Windows 11

Priors for depth and content are used to optimize the NeRF representation’s parameters. To be more explicit, they build a text-related picture as the content prior using a diffusion model and a monocular depth estimation approach to offer the geometric prior of the constructed scene. In order to guarantee consistency across numerous viewpoints, they also recommend a progressive inpainting and updating technique (PIU) for the unique view synthesis of the 3D scene. 

Text2NeRF developed a variety of 3D settings, including artistic, indoor, and outdoor scenes, due to the method’s universality. Text2NeRF can also create 360-degree views and is not limited by the view range. Their Text2NeRF performs qualitatively and statistically better than the preceding approaches, according to numerous tests. 

Subscribe to our newsletter

Subscribe and never miss out on such trending AI-related articles.

We will never sell your data

Join our WhatsApp Channel and Discord Server to be a part of an engaging community.

Sahil Pawar
Sahil Pawar
I am a graduate with a bachelor's degree in statistics, mathematics, and physics. I have been working as a content writer for almost 3 years and have written for a plethora of domains. Besides, I have a vested interest in fashion and music.

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Most Popular