``Text2Room'' will be developed that can automatically generate a 3D room just by entering text into AI
With the development of AI, many sites and tools have been developed that generate
Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models
(PDF file) https://lukashoel.github.io/text-to-room/static/images/paper.pdf
Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models
GitHub - lukasHoel/text2room: Text2Room generates textured 3D meshes from a given text prompt using 2D text-to-image models.
https://github.com/lukasHoel/text2room
Developed by Lucas Helein et al. of the Technical University of Munich, 'Text2Room' aims to generate a textured 3D mesh of the room from the given text prompt. The following video explains the mechanism of Text2Room and the generated 3D mesh.
Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models-YouTube
As a preparation for 3D mesh generation, we first use the image generation AI 'Stable Diffusion' to generate images of the room from various angles according to the input prompts. For each generated image, we render a 3D mesh to create an RGB rendering and depth map for that part. Mesh filters and depth alignment are applied to these processed images, and all images are combined into one.
Text2Room then fills in the blank areas of the merged image by sampling the appropriate camera positions according to the prompts entered.
Next, create the main parts of the room, such as the layout of the furniture. Then follow the prompts to render the furniture, appliances, miscellaneous goods, etc. that cover the entire room.
Ultimately, the blank areas that could not be interpolated are sampled post-hoc to complete the perfect textured 3D mesh of the room.
The image below is a screenshot of the 3D model of the room generated by Text2Room. The prompt I typed in is 'Living room with a lit fireplace, a sofa, cozy curtains, and a lamp to brighten the room.'
The 3D mesh generated by inputting `` eye level, coastal bathroom, bathtub, shell,
The generated data for 'rustic house, living room, stone fireplace, wood, leather work, wool' is the image below.
'A library with tall bookshelves, tables, chairs and reading lights'
'Modern children's room, table lamp,
'A small office consisting of chairs, desks and displays'
'A living room with lots of bookshelves, sofas and small tables.'
'Text2Room is the first tool that can generate attractively textured, room-scale 3D geometry from text-only input,' said Helein et al.
Related Posts:
