Gaussian shadow casting for neural characters
department or school: Science, Faculty of
department or school: Computer Science, Department of
data provider: DSpace
degree campus: UBCV
creator: Bolanos, Luis
issued: 2024-04-24T22:02:27Z
issued: 2024
related degree: Master of Science - MSc
degree grantor: University of British Columbia
description: High-fidelity character models form an integral part in multiple domains, including visual effects, animation, gaming, telepresence, fashion, and more. Recently, approaches for learning and creating 3D animatable avatars through neural rendering have obtained impressive rendering results, even when learning from monocular video data. These neural characters, while achieving highly detailed geometry and texture reconstruction, bake lighting and shadow information into the model, leading to artifacts when generating novel views, novel poses, or when relighting the model. The highest quality models are trained on data that is uniformly lit, and will otherwise contain artifacts in the shape reconstruction if using illuminated data. Consequently, the use of these neural characters is limited and has not seen widespread adoption in the aforementioned industries. In this thesis, we address the limitations of neural character lighting, specifically shadow casting, enabling explicit and efficient light calculations while disentangling shadows in the learned texture. Our primary contributions include: 1) A framework for approximating the shape of neural characters using a sum of 3D anisotropic Gaussians; 2) The derivation of a simple analytic formula for integrating the sum of anisotropic Gaussians along a ray; 3) A framework to perform light and shadow calculations using a deferred rendering approach for easy integration on existing models. These contributions enable the training of neural characters from outdoor lit data, and the relighting of the neural characters in a wide array of environments, increasing their utility in industry.
aggregated CHO: https://circle.library.ubc.ca/rest/handle/2429/88000
note: Gaussian Shadow Casting for NeuralCharactersbyLuis BolanosB.Sc., The University of British Columbia, 2022A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFMASTER OF SCIENCEinThe Faculty of Graduate and Postdoctoral Studies(Computer Science)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)April 2024© Luis Bolanos 2024The following individuals certify that they have read, and recommend tothe Faculty of Graduate and Postdoctoral Studies for acceptance, the thesisentitled:Gaussian Shadow Casting for Neural Characterssubmitted by Luis Bolanos in partial fulfillment of the requirements forthe degree of Master of Sciencein Computer ScienceExamining Committee:Helge Rhodin, Assistant Professor, Department of Computer Science, UBCSupervisorLeonid Sigal, Professor, Department of Computer Science, UBCSupervisory Committee MemberDinesh K. Pai, Professor, Department of Computer Science, UBCSupervisory Committee MemberiiAbstractHigh-fidelity character models form an integral part in multiple domains,including visual effects, animation, gaming, telepresence, fashion, and more.Recently, approaches for learning and creating 3D animatable avatars throughneural rendering have obtained impressive rendering results, even when learn-ing from monocular video data. These neural characters, while achiev-ing highly detailed geometry and texture reconstruction, bake lighting andshadow information into the model, leading to artifacts when generatingnovel views, novel poses, or when relighting the model. The highest qualitymodels are trained on data that is uniformly lit, and will otherwise containartifacts in the shape reconstruction if using illuminated data. Consequently,the use of these neural characters is limited and has not seen widespreadadoption in the aforementioned industries.In this thesis, we address the limitations of neural character lighting,specifically shadow casting, enabling explicit and efficient light calculationswhile disentangling shadows in the le…
has type: Thesis/Dissertation
date issued: 2024-05
is shown at: 10.14288/1.0441531
language: eng
degree discipline: Computer Science
provider: Vancouver : University of British Columbia Library
publisher: University of British Columbia
rights: Attribution-NonCommercial-ShareAlike 4.0 International
rights URI: http://creativecommons.org/licenses/by-nc-sa/4.0/
scholar level: Graduate
contributor: Rhodin, Helge
type: Text
identifier URI: http://hdl.handle.net/2429/88000