The resulting objects can be used to render 3D-shaped buildings, outdoor spaces, or entire cities for use in industries such as gaming, robotics, architecture, and social media.
GET3D can generate an unlimited number of 3D shapes based on the data used to train it. Like an artist’s delicate sculptural clay, GET3D models transform numbers into complex 3D shapes.
Like using a dataset of 2D car images to train a GET3D model, it will create 3D shapes of cars, trucks, race cars, and pickups. When trained with 2D animal images, it produces animals such as 3D shaped foxes, rhinos, horses, and bears. When trained with 2D chair images, it produces swivel chairs, dining chairs, and comfortable reclining chairs in various 3D shapes.
Film:
Virtual worlds need to be created with AI
The real world is rich in appearance, the streets are lined with unique buildings, vehicles of all kinds whizz past, and people of all kinds shuttle through it. It takes time to simulate a 3D virtual world that reflects these features manually, and it is difficult to add details to this digital environment.
AI models are faster at building 3D virtual worlds than manual methods in the past, but they are still not precise enough. Even the latest inverse rendering methods can only generate 3D objects from 2D images taken from different angles, and developers can only create one 3D shape at a time.
Enables creators to modify shapes, textures and materials
GET3D gets its name from its ability to generate Explicit Textured 3D (Generate Explicit Textured 3D) meshes, meaning a triangular mesh is used to build shapes, like a papier-mâché model covered with textured material. This allows users to easily import and edit objects into game engines, 3D modelers, and movie renderers.
The researchers point out that a future version of GET3D could use camera pose estimation techniques, allowing developers to train models using real-world data rather than synthetic data. The researchers will also improve the GET3D model to support generic generative techniques, allowing developers to train GET3D with a variety of 3D shapes at a time, rather than one object class at a time.
Further reading: