Open Collections

Johnson, Joel

University of British Columbia

This work aims to produce a biologically and physically accurate model for the relationship between skin structure and color that is practical for real-time rendering and editing of skin appearance. The motivation behind this endeavour stems from the need to enhance visual realism in digital media and to improve the inclusivity of training data for computer vision applications. As virtual and augmented realities become more prevalent, the demand for realistic skin rendering grows, necessitating a model that captures the subtle nuances of skin tones and textures and allows for dynamic modifications in real-time. Monte Carlo light transport simulations are utilized to generate a comprehensive tabu- lar dataset encompassing skin chromophore amounts, thicknesses, reflectance distributions, and corresponding RGB colors. This dataset serves as a Lookup Table (LUT) for convert- ing diffuse 3D images and albedo textures into their corresponding biophysical parameter maps. Supervised autoencoders (SAEs) are used to learn the relationship between opto- structural skin parameters and color to facilitate a more efficient mapping of the data. We then evaluate the SAE in parameter map generation, image recovery, and image editing, aiming for the most biophysically accurate representation of skin color. We iterate on the simulation inputs and SAE model parameters, using these evaluations to find the ideal training data and SAE architecture. We then create a procedural data pipeline for generating synthetic human/landmark pairs, utilizing SAEs for biophysically based skin color augmentation of diffuse albedo tex- tures. Using this synthetic data, we train a DLM for facial landmark detection. Finally, we demonstrate a novel combination of DLMs and SAEs to facilitate spatially coherent alter- ations within the latent space, ensuring realistic and accurate transfer of latent biophysical properties between different images of human skin. This novel approach facilitates photo- realistic, real-time skin rendering and enables a biologically based spatial distribution of skin characteristics. This comprehensive methodology sets a new standard for skin rendering, editing, and generation, combining biological accuracy with practical applicability to enhance realism and inclusivity in digital humans.

Text

2024-08-28

Attribution-ShareAlike 4.0 International

http://hdl.handle.net/2429/89075

Electrical Engineering

University of British Columbia

UBCO

http://creativecommons.org/licenses/by-sa/4.0/