Open Collections

Perryman, Olivia

University of British Columbia

Enabling computers to understand and interpret visual information is crucial for the development of more sophisticated and interactive technologies. Learning structure, such as 3D shape, can help computers understand visual information more effectively. Our model disentangles object structure and appearance in a weakly supervised manner from multiview images within a single category. We extract a 3D pointcloud from images and reconstruct consistent novel views by rendering the pointclouds from different perspectives. Using a much simpler model and far fewer training examples than costly state-of-the-art diffusion models, we can recover 3D structure from single images of objects and quickly reconstruct them from unseen viewpoints. Our findings suggest that understanding 3D constraints of the real world can enhance performance on visual tasks and make models more robust and generalizable to a wider variety of inputs. This approach enables downstream tasks such as pose transfer, spatially-guided conditional image generation, and paves the way for commonsense reasoning. Our work has potential applications in augmented reality and visual effects, with further exploration of the model's capabilities and integration into broader systems for enhanced visual understanding.

Text

2024-08-30

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/89126

Computer Science

University of British Columbia

UBCV

http://creativecommons.org/licenses/by-nc-nd/4.0/