UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Modularizing deep learning for geometry-aware registration and reconstruction Jiang, Wei

Abstract

In this work, we explore the modularization of deep learning for geometry-aware registration and reconstruction, with a particular focus on cameras registration and human reconstruction from videos. The traditional methods for these tasks have been challenged by deep learning approaches, but end-to-end learning can be limited in terms of generalization, transparency, and controllability. Modularization breaks the task into smaller subtasks and allows each to be addressed individually using traditional methods or deep learning techniques. Through modularization, we are able to embed knowledge from the real world, enabling better generalization, simpler and more effective learning, explainable and transparent models, and geometry-awareness. Specifically, this work consists of four major chapters, each presenting a modularized approach to solve a specific geometric problem. Firstly, a novel linearized multi-sampling method is proposed to enable better image alignment and learning. Secondly, the homography warping is modularized out of the pipeline allowing optimization through the learned error for accurate sports field registration. Thirdly, by modularizing the robust estimation and 3D map from the pose estimation pipeline, the neural network can focus on learning accurate image correspondences. Finally, the modularization of human scene positioning and mesh skinning allows for the reconstruction of animatable human avatar from video. Overall, our work demonstrates the power of modularization, and we hope it will inspire future research on modularization and its potential applications to other areas.

Item Media

Item Citations and Data

Rights

Attribution-NonCommercial-NoDerivatives 4.0 International