Nvidia Research have revealed new AI technology that could change 3D object modeling and video game design. Nvidia Neuralangelo, which utilizes a deep neural network, is an AI model that can successfully transform objects from 2D videos into complex 3D graphics structures with intricate details and impeccable accuracy.
This new AI model allowed the team from Nvidia Research to generate virtual replicas of different real-time objects shown in 2D videos including statues and sculptures, buildings, and other different constructions.
The team was inspired by the sculptor himself – Michelangelo, who breathed life into his sculptures David, Madonna of Bruges, Angel, and others. Once the system transforms 2D objects into 3D graphics, designers and developers will be able to export them and then import them into their own projects.
The team behind Nvidia Neuralangelo is hoping to offer a solution to generating complex structures like buildings without compromising time and other resources. The new AI system is suitable to be used for video game development, 3D modeling, architecture, robotics, and industrial digital twins.
Neural rendering
Nvidia Neuralangelo is not the first AI model of this kind, but it’s certainly more effective than its predecessors and other similar tech.
Previous AI models couldn’t accurately replicate repetitive texture patterns, strong color variations, or a homogenous color palette. However, because Neuralangelo uses deep neural networks and neural graphics primitives, it can replicate finer details that some structures require.
That same technology powers the Nvidia Instant NeRF which can also transform 2D images into immersive 3D scenes. This technology won the best paper reward at SIGGRAPH in 2022. With Nvidia Neuralangelo, researchers hope to extend this technology to architecture and 3D modeling and achieve even better results.
To test the app, researchers took a 2D video of an object from different angles and then had the model calculate and select the frames that would capture different viewpoints. First, it’d create a rough 3D representation of the objects and then fine-tunes it to get sharper details, similarly to how sculptors shape and hew their sculptures.
Nvidia Neuralangelo at CVPR
The innovative AI-powered solution by Nvidia Research will make a lot of room for game designers, VR developers, architects, exterior, and interior designers to showcase impressive 3D structures without spending weeks or months building their projects.
It will allow new technologies to emerge, and potentially transform the video gaming industry by allowing more competitive games to come out, featuring world-known architecture with fine details and other additions.
The research team will present Nvidia Neuralangelo at the Conference on Computer Vision and Pattern Recognition (CVPR) 2023 taking place from June 18-22 in Vancouver.
