Source: NVIDIA
After announcing its new graphics processors from the GeForce RTX 40 series, alongside announcements about developments in the field of autonomous vehicles and the cloud – Nvidia announced a short time ago at the GTC conference another interesting development, one that can save developers hours of work and turns 2D objects into 3D.
Turn images into 3D
NVIDIA’s new tool, NVIDIA GET3D, enables the creation of 3D figures and objects by combining a dataset of 2D images directly within a dedicated engine developed by the researchers. Thus, on paper, Nvidia can remove from the equation a significant barrier to creating content in 3D – the need for professional personnel such as designers, animators or 3D artists – and allows anyone (technically) to replace them. And even if not actually replace – at least do a large part of the work.
The model developed by Nvidia was trained on a database of about a million two-dimensional images that includes various objects, so that even today, as soon as images of a vehicle are fed into it, it will be able to create a three-dimensional model of that vehicle in just a few seconds. In accordance with the trend of recent years, this model was also not trained on “real” images, but on synthetic data created for it – which includes a huge amount of 2D images alongside certain 3D shapes that were “photographed” from several different angles. According to Nvidia’s researchers, they needed exactly two days to train the model using the company’s A100 chips. From this moment, the model is ready to receive a two-dimensional image of a vehicle and create a new model from it.
Among other things, Nvidia allows you to change the textures of objects created in 3D by receiving a Prompt (yes, as in the popular AI-based image generators). So for example you could create a line of dogs in 3D, but make them all spotted, blue or any other request (that would only make sense). This ability comes thanks to the implementation of another Nvidia tool – StyleGAN-NADA – developed by Israeli researchers from Tel Aviv University and Nvidia, which allows the application of different styles on top of an existing image.
In addition, all the products of the model come in formats that are used today by popular design software – so that they can be easily transferred for further editing with professional tools after the model emits for you the 3D model you wanted. The tool will be available to developers for free – without waiting lists (unlike models such as those of OpenAI).
Another tool in Ominverse
The announcement follows another announcement by Nvidia from GTC about the Omniverse Cloud service – which enables the creation of virtual worlds in the cloud, without the need to purchase and operate powerful on-prem hardware. It’s another tool in Nvidia’s strategy, which goes hand-in-hand with Mark Zuckerberg’s hype-filled, content-less vision of Meta and the Metaverse. The use of NVIDIA GET3D can, on paper, allow anyone to create a virtual world without having to look for artists or graphic artists to build all the objects that will be in it, when everything is done through the new tool.