Open Collections

He, Xingzhe

University of British Columbia

Object correspondence and structure play critical roles in image generation, 3D reconstruction and animation. In recent years, supervised algorithms have dramatically improved the accuracy of the learned correspondence. However, these approaches are expensive due to manual annotation and do not generalize well to new domains. We propose several methods in this dissertation on unsupervised structure learning from casually recorded images and videos. To be specific, we propose a Generative Adversarial Network (GAN)-based unsupervised keypoint detector and extend it for object part segmentation. Furthermore, we introduce a representation for unsupervised keypoints relationship estimation. We later adapted this technique for few-shot keypoint learning, depth prediction, and occlusion handling. In addition, we propose a dataset generation approach for diffusion model personalization to implicitly learn the object structure and appearance. The overarching goal of this dissertation is to make progress in the field of unsupervised object correspondence and structure learning. Our proposed methods outperform existing unsupervised methods on the established keypoint estimation and part segmentation benchmarks and paves the way for structure-conditioned generative models on more diverse datasets.

Text

2024-03-06

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/87531

Computer Science

University of British Columbia

UBCV

http://creativecommons.org/licenses/by-nc-nd/4.0/