3D-GPT is a pioneering framework that simplifies 3D asset modeling in the metaverse era by utilizing large language models (LLMs). Developed collaboratively by teams from the Australian National University, University of Oxford, and Beijing Academy of Artificial Intelligence, 3D-GPT breaks down complex 3D modeling tasks into manageable segments, employing LLMs as adept problem-solvers. The framework consists of three key agents – task dispatch, conceptualization, and modeling – working together to enhance initial scene descriptions and seamlessly integrate procedural generation. Demonstrating reliability and effective collaboration with human designers, 3D-GPT not only streamlines traditional 3D modeling but also integrates smoothly with Blender, expanding manipulation possibilities. This innovative approach underscores the substantial potential of language models in shaping the future of 3D modeling, particularly in scene generation and animation.

Paint3D introduces a groundbreaking generative framework for creating high-resolution, lighting-less, and diverse 2K UV texture maps for untextured 3D meshes, conditioned on text or image inputs. Addressing the challenge of generating quality textures without embedded illumination information, the method employs a coarse-to-fine approach. It initially utilizes a pre-trained depth-aware 2D diffusion model for multi-view texture fusion, producing a coarse texture map. To overcome incomplete areas and illumination artifacts, separate UV Inpainting and UVHD diffusion models are trained for shape-aware refinement. The resulting process yields high-quality 2K UV textures with semantic consistency, allowing for significant advancements in texturing 3D objects and providing flexibility for re-lighting and editing within modern graphics pipelines.

The Gsgen project introduces a novel approach called Gaussian Splatting based text-to-3D GENeration (Gsgen) that generates high-quality, multi-view consistent 3D assets. Previous methods lacked accurate geometry and fidelity, so Gsgen leverages 3D Gaussian Splatting as a representation technique to address these limitations. Their approach involves a progressive optimization strategy, including a geometry optimization stage and an appearance refinement stage. The geometry optimization establishes a coarse representation under a 3D geometry prior, while the appearance refinement iteratively refines the obtained Gaussians to enhance details. The method proves effective in generating 3D assets with accurate geometry and delicate details