Plausible Cloth Wrinkling for Captured Mesh Sequences by Xi Chen B.Sc. in Engineering, Zhejiang University, 2007 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF Master of Science in THE FACULTY OF GRADUATE STUDIES (Computer Science) The University Of British Columbia (Vancouver) October 2011 c© Xi Chen, 2011 Abstract Cloth modeling has become an important research topic in computer graphics, as garments are ubiquitous in virtual environments used in movies or games. While recent studies can successfully capture the general shape of cloth as well as a con- tinuous cloth motion, they usually lose high frequency details such as wrinkles or folds. This work provides a method of recovering high frequency details for cap- tured cloth animation sequences without any aids of additional information. The method analyzes the stretch/shrinkage tensor fields in 4D space-time domain for the original cloth animation sequences. Based on the assumption of cloth devel- opability, the stretch/shrinkage information is utilized to recover fold shapes and directions. Finally, a space-time consistent deformation method is applied to re- cover the high frequency folds. The work is generally applicable to most of the cloth capture methods applied either on scanned point clouds or video sequences. ii Preface The algorithm described in Chapter 4 was developed together with Dr. Alla Shef- fer. Dr. Sheffer supervised me on the project while I performed research and implementation. Dr. Hao (Richard) Zhang and Kevin Xu participated in various discussions of the project. The algorithm was combined with a deformation frame- work developed by Dr. Tiberiu Popa and Dr. Alla Sheffer [28]. This thesis extends a draft paper coauthored by me, Dr. Alla Sheffer and Dr. Hao (Richard) Zhang, which is being prepared for publication. iii Table of Contents Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Garment Modeling . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1 Stretch Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Developable Surfaces . . . . . . . . . . . . . . . . . . . . . . . . 5 2.3 Isometric Deformation . . . . . . . . . . . . . . . . . . . . . . . 5 3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.1 Deformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.1.1 Surface Deformation . . . . . . . . . . . . . . . . . . . . 7 3.1.2 Space Deformation . . . . . . . . . . . . . . . . . . . . . 8 3.2 Cloth Animation . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.2.1 Cloth Simulation . . . . . . . . . . . . . . . . . . . . . . 9 3.2.2 Cloth Capture . . . . . . . . . . . . . . . . . . . . . . . . 10 3.2.3 Modeling with Wrinkles and Folds . . . . . . . . . . . . . 11 iv 4 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.2 Stretch Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4.3 Phase detection and Stretch Integration . . . . . . . . . . . . . . . 17 4.4 Fold Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.5 Space-time Consistent Deformation . . . . . . . . . . . . . . . . 22 5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 6 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 6.1 Complexity Analysis and Implementation Details . . . . . . . . . 27 6.2 Discussion of Adding Folds on Cloth Simulation Data and Cloth Captured Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 6.3 Limitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 7 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . 30 7.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 v List of Figures Figure 2.1 Unfolding sphere and cone . . . . . . . . . . . . . . . . . . . 5 Figure 4.1 System pipeline . . . . . . . . . . . . . . . . . . . . . . . . . 15 Figure 4.2 Tensor smooth . . . . . . . . . . . . . . . . . . . . . . . . . 17 Figure 4.3 Tensor accumulation of the emergence and disappearing of a fold . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Figure 5.1 Result of a hand model . . . . . . . . . . . . . . . . . . . . . 24 Figure 5.2 Result of a shirt model . . . . . . . . . . . . . . . . . . . . . 25 Figure 5.3 Result of a dress model . . . . . . . . . . . . . . . . . . . . . 26 vi Acknowledgments First of all, I would like to thank my supervisor Dr. Alla Sheffer, who brought me to the world of geometry. I am grateful for her ongoing guidance on this work. Her never-ending enthusiasm in research inspired me the most. I would also like to thank my second reader, Dr. Wolfgang Heidrich, for his valuable feedback. Thank you to Tiberiu Popa for providing the deformation source code and var- ious discussions with him. Thank you to Cheryl Lau, Mushfiqur (Nasa) Rouf, Ron Maharik, Jia Gu, Li Ma and Caoyu Wang for helping me revise the thesis. Thank you to my mother for understanding my decision to study abroad, where is more than one third of the earth away from her. Thank you to all my friends for their accompany and encouragement. Xi Chen The University of British Columbia September 2011 vii Chapter 1 Introduction 1.1 Garment Modeling Realistic modeling of cloth contributes significantly to the natural look and feel of dressed characters and other fabric objects in an animated scene. One of the crucial aspects of realistic cloth behavior is the existence of wrinkles. Traditional methods for modeling wrinkles or folds on fabrics rely on physical simulations [6][7][8]. The ensuing problem is computationally demanding, as deforming clothes offer possibly the worst-case scenario in terms of collision handling[6]. Also, deriving and modeling the correct physical properties of different kinds of fabrics, in order to simulate realistic wrinkle behavior, are both highly delicate processes. With the advent of advanced geometry and motion acquisition technologies, systems for capturing garments or moving characters with loose-fitting clothes have emerged [5][12][40], allowing for a more direct, data-driven approach for obtaining realistic cloth behavior. Using multi-view video, high-frequency folds may be captured with the aid of color markers, but this requires custom-made fab- rics [45]. Markerless garment capture simplifies the acquisition setup and applies directly to off-the-shelf clothing. However, it typically does not reproduce high- frequency folds, as they are more difficult to capture and distinguish from noise or registration error. Recently, more and more works focus on recovering high frequency folds for cloth animation sequences. Some researchers rely on additional information such 1 as video footage of captured garments [28] or pre-simulated wrinkle examples [41] to reproduce detailed wrinkles. Others study the relationship between cloth stretch and wrinkles formulation, and propose methods based on the low-stretch char- acteristic of cloth. This thesis follows the spirit of the marker-less, data-driven approaches to generate dynamic wrinkles on captured clothes. By making the as- sumption that garments are made of low-stretch material, we are able to estimate the location and magnitude of wrinkles across time by analyzing the captured geo- metric sequences only. The method is based on three observations: • Cloth can undergo very little stretch and no shrinkage. From the geometric point of view, cloth animation can be modeled isometric deformation. This provides a further information for cloth capture methods besides the acquired data inputs (scans or videos) and the time continuity constraint. • The emergence and attenuation of cloth wrinkles are correlated with the vari- ations of cloth shapes across time. It can be illustrated by gradually com- pressing a piece of cloth and then gradually restoring it, where folds are gen- erated and then disappear. Imagine if we are compressing a paper for which stretch/shrinkage can occur. Fold emergence will be replaced by shrinkage and fold disappearance will be replaced by stretch. • Folds and wrinkles can be modeled by a generalized cylindrical pattern. This narrows down the elements for modeling folds to fold direction (the axis) and fold width (diameter). The local fold direction is perpendicular to the stretch/shrinkage direction. Our method takes as input a deforming mesh sequence with consistent vertex- to-vertex correspondence, but without a known rest pose. Absolute rest poses are necessary to compute the absolute stretch/shrinkage, we usually do not have it in results from garment capture systems. A naive attempt to identify a global rest shape for each triangle cannot lead to satisfactory results, since noise are accu- mulated by the temporal registration step which produced the mesh sequence and global rest shape does not exist. Instead, we identify local changes in triangle 2 shapes both spatially and temporally and integrate them across time for fold infer- ence. Fold emergence and attenuation phases are identified separately by abrupt change in triangle shape variations. When large patches of similar stretch are de- tected, regions to recover folds are extracted. Folds are then located by following the perpendicular direction of stretch, and fold width is adaptively set according to the stretch magnitude. The consistent space-time deformation method described in [28] is followed for the final deformation setup. 1.2 Contribution The main contributions of this thesis are as follows: • This work proposes and studies a simple but effective method of recovering high frequency folds based on the low-stretch constraint of cloth for captured cloth mesh sequences, without user-defined rest poses. • It provides a postprocessing of garment capture to enhance the results with- out the aid of any additional inputs such as video sequences or pre-simulated wrinkle examples. • It further demonstrates that geometry constraints can successfully enhance the captured results, making geometry modeling an effective alternative to cloth simulation. 3 Chapter 2 Background 2.1 Stretch Metrics Given two triangles T and T ′, the stretch/shrinkage between them is computed as follows. First, triangle T is projected into 2D. The 2D coordinates of the three vertices in T are denoted as pi = (si, ti) , with i = 1, ...,3. Correspondingly, the 3D coordinates of vertices in T ′ are denoted as qi , with i = 1..3. The unique affine mapping between the two triangles S(p) = q is defined as S(p) = 〈p, p2, p3〉q1 + 〈p, p3, p1〉q2 + 〈p, p1, p2〉q3 〈p1, p2, p3〉 where 〈a,b,c〉 denotes the area of triangle abc. The partial derivatives are derived as: Ss = ∂S ∂ s = q1(t2− t3)+q2(t3 − t1)+q3(t1 − t2) 2〈p1, p2, p3〉 St = ∂S ∂ t = q1(s3 − s2)+q2(s3− s1)+q3(s1 − s2) 2〈p1, p2, p3〉 The 3×2 Jacobian matrix J = [Ss,St ] defines a unique mapping between vec- tors in the spaces that the two triangles are in. Applying Singular Value Decompo- sition(SVD) to J yields get J =UΣV T . Σ is a diagonal matrix containing singular values along its diagonal. The maximum singular value Γ and minimum singular value γ represent the largest and smallest lengths obtained when mapping unit- 4 1(a) 1(b) 2(a) 2(b) Figure 2.1: Unfolding sphere and cone. 1(a) original sphere; 1(b) unfolding sphere; 2(a) original cone; 2(b) unfolding cone. L2 stretch norm is visu- alized on the unfolded model. Stretch to compression is represented by color from red to blue. White represents no stretch. length vectors from T to T ′, i.e. the largest and smallest local stretch/shrinkage, respectively. U and T are two sets of orthogonal singular vectors. They are both the directions of the largest and smallest local stretch/shrinkage but in different spaces. The L2 stretch norm is defined as L2(T ) = √ (Γ2 + γ2)/2 2.2 Developable Surfaces Developable surfaces are surfaces that can be unfolded onto a plane without any distortion. For example, the cone in Figure 2.1, can be unfolded onto planes with no stretch, by cutting perpendicular to the cross-section. In comparison, a sphere is not a developable surface. No stretch means that every unit disk should be mapped onto a unit disk after unfolding, i.e. Γ = γ = 1. Cloth can be modeled as a quasi- developable surface, since cloth can undergo very little stretch and no shrinkage. 2.3 Isometric Deformation Isometric deformation refers to deformation in which only rotations or translations occur whereas every triangle/quad’s shape is preserved (no stretch). For an an- imated developable surface, if the developability characteristic is preserved, the animation is an isometric deformation. For example, the animation of unfolding a 5 paper is an isometric deformation. Let V ti be the local frame of a triangle i at time t. A deformation is isometric if and only if V ti+1(V ti )−1 = Ri where Ri = Rix(α)Riy(β )Riz(γ) is a rotation matrix with 3 degrees of freedom. Rix(α) is the rotation matrix representing the rotation around the x axis with an- gle α . As cloth can be modeled as a quasi-developable surface, deformation of animated cloth is isometric deformation. 6 Chapter 3 Related Work 3.1 Deformation 3.1.1 Surface Deformation Surface deformation aims at propagating the deformation of a set of handles to other surface areas while preserving the local details of the original shape. It ap- plies directly on the surface by minimizing a deformation energy which constrains both local properties and handle locations. Early works focus on multi-resolution editing, which deforms the low-resolution surface while adding the surface de- tails afterwards. Recently, many researchers proposed approaches of represent- ing local details using local differential coordinates. These methods remove the time-consuming prerequisites of subdivision, simplification and reconstruction for multi-resolution methods. Moreover, these methods resolved the problem of basis elements with large support in multi-resolution methods [37] and achieve better results. Yu et al. [46] introduced a method of implicitly manipulating the gradient field to preserve the local shape details. Rotations of handles are propagated to the region of interest explicitly using a uniform, linear or Gaussian scheme. Surface is then reconstructed by solving a Poisson equation which relates the gradient and the edited local surface normal. Sorkine et al. [37] and Lipman et al. [24] repre- sented the local frame using Laplacian coordinates. Laplacian coordinates are not rotation invariant and need explicit formulation of the rotation. This forms a non- 7 linear deformation energy function, which is linearized in [37] and heuristically approximated in [24]. Sheffer et al. [33] designed a set of rotation and scale invari- ant coordinates, they call pyramid coordinates. However, this set of coordinates involves implicit rotation of normals, which makes the deformation energy func- tion non-linear and can only be solved by iterative method. To achieve interactive editing while dealing with the non-linear constraints involved in the deformation energy, Huang et al. [21] presented a deformation framework on subspace edit- ing, which dramatically reduces the size of the system, while also improving the numerical stability. As a similar concern, Au et al. [2] found that the isolines of a harmonic scalar field representing the deformation propagation can capture the low-frequency shape deformation. Deformation is computed over the reduced domain, while transformations for original mesh is interpolated afterwards. In addition to building fast deformation framework, some other problems are also important to be solved for achieving better deformation results. Zhou et al. [47] studied volumetric graph Laplacian method, which removed the unnatural volume changes. Sorkine et al. [36] designed a framework to preserve local rigid- ity. Botsch et al. [4] provided a method of embedding the surface in a layer of rigid volumetric prisms and minimized the elastic energy, which is inspired by the physical behavior. Shi et al. [35] proposed a system combining mesh deformation and skeleton-based rigging, which balances the character animation and the local shape preservation. 3.1.2 Space Deformation Another family of methods applies deformation on the space instead of the surface. A shape is enclosed by a polyhedron cage of a small number of points that also act as the control points. The enclosed surface is represented by a barycentic inter- polation of the outer cage points. This family of methods focuses on introducing different coordinates with various properties. Floater [16] introduced mean-value coordinates for 2D general polygons as a closed form scheme, followed by a gener- alized form in 3D domain by Hormann and Floater [19]. To achieve non-negative weights, harmonic coordinates [22] and maximum entropy coordinates [20] are presented to guarantee that the deformed surface lies in the convex hull of the outer 8 polyhedron cage. While harmonic coordinates do not have a closed form, maxi- mum entropy coordinates have one and can be evaluated efficiently by solving a convex optimization problem. Recently, several researchers have been focusing on studying shape preserving coordinates. They observed that if the control points on outer cages are relaxed as soft constraints, there exist coordinates to achieve conformal or quasi-conformal deformation. Lipman et al [25] introduced green coordinates derived from the the- ory of Green functions, where Weber et al. [44] presented an equivalent Cauchy coordinates. Weber et al. [43]make a further study to prevent foldovers by intro- ducing Hilbert coordinates, which shows a connection between harmonic coordi- nates and Cauchy coordinates. 3.2 Cloth Animation Cloth animation is a well researched topic in computer graphics. The existing methods can be roughly classified into two main categories: simulation and cap- ture. Physically-based cloth simulation models the dynamics (e.g. velocity and acceleration) and material characteristics. Cloth capture focuses on capturing the geometric shapes and their deformation along time. 3.2.1 Cloth Simulation Traditional physically-based methods have been very successful in simulation of cloth physics [6][7][8]. Details such as folds and wrinkles are preserved by simu- lating bendings and collisions. Bounded stretch constraints are generally used as a rule to obtain better accuracy. Most current simulation techniques apply isotropic elastic model. Bridson et al. [6] limited the strain and strain rate by iteratively adjusting the velocity. English and Bridson [15] presented a new discrimination of non-conforming elements for developable surfaces to constrain zero deforma- tion in-plane. Their model reflects motion of developable dynamics by allowing more degrees of freedom. However, the attached conforming mesh is still in low resolution and is not developable and not smooth enough. As opposed to isomet- ric conforming constraints presented in previous works, Thomaszewski et al. [39] pointed out the anisotropic stretch nature of cloth and proposes an anisotropic strain 9 limiting method based on a continuum deformation measure. Besides, sometimes parameters are difficult to be tuned in physically-based simulations. This is because of the large number of parameters adjusted for less ef- ficient non-linear optimizations. Several methods have attempted to automatically extract the parameters from either pre-simulated cloth sequences or cloth captured data. Bhat et al. [3] presented a method of estimating parameters from video data by optimization, while Wang et al. [42] and de Aguiar et al. [13] proposed to apply machine learning to the modeling process. They treated existing simulation results or measurements data as a training set to learn parameters for future use. Their models are simplified to fit in a machine learning mechanism, making it more ef- ficient to process but also less accurate. Overall, the acquisition of real material parameters are generally difficult. 3.2.2 Cloth Capture Cloth capture or capturing moving human in loose-fitting clothes aim at acquir- ing animated high resolution cloth sequences with geometric details at a low cost [5][12][40]. This category of methods takes video capture from multiple points of view or a sequence of scanned point clouds as an input. It is presented as an alternative simpler way of cloth modeling to simulation. While several methods establish dense correspondences and reconstruct a high resolution surface in one step, other methods register and capture the motion of a smooth template first, and recover high frequency details in a follow-up step. At the early stage, marker-based methods were used for tracking the correspon- dence between frames. Scholz et al. [32] used a color-coded pattern attached to a piece of cloth to track the correspondences of features in video captures from different points of view. Surface is then reconstructed by exploring 3D transforma- tions. White et al. [45] established similar pipeline but focused on dealing with the problem of occlusions. In their system, the correspondence matching step is en- hanced by improving the color pattern used and by pruning inaccurate matchings using strain constraints. They also use MeshIK[38] to recover the occluded regions produced by folding. These marker-based methods are restricted to custom-made cloth and cannot be generalized to general cloth capture settings. 10 In addition to marker-based methods, Hernandez et al. [18] set up red, green and blue light from different directions and extracted surface normals with pho- tometric stereo. This normal map is first integrated independently for each frame to produce a sequence of depth-maps. In addition, they apply a 2D optical flow method on the normal map to produce dense correspondences. The surface is then registered by this optical flow and with the help of a rigidity constraint to produce a coherent sequence. Bradley et al. [5] proposed the method for generating a high resolution model sequence directly from multi-view stereo. Consistent cross-parameterizations with isometric constraints are used to establish dense correspondence. However, their method still fails to capture high-frequency details. This thesis further explores the isometric constraints of cloth to generate higher frequency folds. Vlasic et al. [40] and Aguiar et al. [12] explored template-based methods in a coarse-to-fine manner. Geometric details are reconstructed by deforming the scanned higher-resolution template utilizing methods such as multi-view stereo and silhouettes fitting. Although these methods produce a dense captured mesh sequence, geometric details tend to be copied from the scanned template [28]. To recover more geometric details, Ahemd et al. [1] apply markerless silhouette-based motion estimation method on a smooth template. Isometric BRDF (bidirectional reflectance distribution function) reflectance model is assumed and estimated from a sequence of rotating body under a static pose. From the estimated reflectance map, the normal map is computed and transferred onto the smooth template by a spatio-temporal coherent optimization to recover surface details. In order to ac- curately estimate the reflectance model, light calibration is needed for their acqui- sition system. Li et al.[23] made a further study along this trend, but work with scanned data. As a coarse template is used for registration step, the distance map between the deformed template and the original scans is computed later to recover high frequency details. 3.2.3 Modeling with Wrinkles and Folds Several methods focus explicitly on modeling fine-scale folds and wrinkles to en- hance the realism. 11 Combaz et al.[9] and Decaudin et al.[14] study approaches of painting or adding user-defined folds. Since their methods are based on procedural modeling with lit- tle information from the real cloth, they are apt to generate unnatural regular folds. Beside of painting directly by a user, some researchers utilize machine learn- ing to learn relationships between fold patterns and joints. Cordier and Magnenat- Thalmann [10] identified cloth-to-joint relations and learn joint coefficients by di- mension reduction from a pre-simulated cloth animation. Wrinkles are then in- terpolated from a fine mesh using joint coefficients. Wang et al. [41] utilize a machine learning approach to generate real-time wrinkles and folds. They learn different fold patterns in different joints and poses, and then transfer them to new simulations. Popa et al.[28] proposed a method of reintroducing folds by edge detection from a captured video. Rotation anchors are put along a pair of parallel bright and dark edges extracted. Rotations are then prorogate into nearby areas by using a space-time consistent deformation method. The space-time deformation gener- ates more consistent and natural folds, and therefore is utilized in this thesis as the deformation setup after fold pattern extraction. By substituting fold extraction method, this thesis generalizes the application of the method to reconstruction se- quences from single-views or scanned point clouds by removing the aid of video sequence. Several methods focus on extracting folds and wrinkles using a geometric point of view. This category of methods can be considered as a counterpart of low- stretch constraint in physically-based methods. They add wrinkles to cloth ani- mation sequences by either directly preserving the cloth shapes or analyzing ab- normal large stretch. The methods can be applied as a post-processing of various modeling and/or simulation approaches. This thesis can be classified as belonging in this category. It targets generating realistic folds by fully exploiting the local stretch/shrinkage information in both direction and magnitude. Early methods used user-defined pattern as examples to help wrinkle modeling. Hadap et al. [17] transfer a user-defined texture of wrinkle patterns to an animation sequence as bump mapping. The transformation is defined by a wrinkle function satisfying area preservation. Cutler et al. [11] design an art-directed wrinkling system. The system requires some extreme poses and rest pose identified by user. 12 Wrinkle patterns for the extreme poses are also designed and modeled by user. The system then blends these inputs to create folds of consequent frames guided by the stretch changes to achieve wrinkle animation. Their system is developed for artists to easily blend the wrinkle animation, but is unsuitable for non-expert users. Also, the system cannot deal with wrinkles moving and interference of wrinkles in different phases. Sauvage et al.[31] presented a method of modeling wrinkles based on preserv- ing curve length. The method first deforms the curve with length preservation and then propagate the deformation on the surface. Loviscach et al.[26] extract compression direction and magnitude from vertex-based local shape deformation analysis. The folds are modeled as local sine waves by bump mapping. Their re- sults appear somehow artificial due to the assumption of the sine wave profile of the folds. Muller and Chentanez[27] directly solved a global optimization prob- lem on a subdivided mesh by setting edge-length constraints and vertex-position constraints. Edge-length constraints limits cloth isometry whereas vertex-position constraint limits fold frequency. However, they do not take anisotropic character- istic into consideration, and as a result visual artifacts may arise when edge stretch is not consistent to the main direction of folds. In addition, they need to paint the vertex position constraints, i.e. manually set the fold frequency. Rohmer et al.[29] proposed a procedural wrinkle modeling approach to refine coarse simulation sequences. They trace fold lines along the local shape com- pression direction with respect to a rest pose while preserving time consistency. Implicit deformers guided by the extracted fold lines are applied in order to han- dle complex folds interactions. A single parameter, the minimum curvature of the fold is used to determine the fold shape as well as the frequency of parallel folds. Their assumption on the knowledge of the rest shape and the minimum curvature of fold is reasonable for simulation sequences. However, in the case of captured data, a global rest shape may not exist. An adaptive fold shape is also more reason- able than a manually setting one, since different parts of cloth may have different fold shape due to different forces. This thesis develops a similar fold extraction method base on stretch tensor and focuses on studies in this trend with an adaptive frequency setting and no rest shape. 13 Chapter 4 Algorithm 4.1 Overview The pipeline of the method is shown in Figure 4.1. The inputs to the method are coarse garment animations generated by capture techniques. The per frame meshes are required to have one to one correspondences and same connectivity for all vertices. Nearly all of the acquisition processes take advantage of surface deformation, which makes the shape variations space-time consistent for exacting continuous stretch information. First, the stretch analysis stage captures the spatially local surface deformation between two adjacent frames (Sec. 3.1). As addressed, cloth is piecewise devel- opable and cloth deformation is isometric. Thus, the local stretch/shrinkage de- tected is a good indication of a fold missing during the capture process. By doing a stretch tensor analysis, the principal directions with maximum shape deforma- tion are extracted to represent the local deformation. It can be stretch, shrinkage or isometry. To achieve space continuity in each frame, the tensor is smoothed for each independent component, and the outliers are removed. Second, the local stretch/shrinkage is integrated, so that the stretching and shrinking changes gradually over time (Sec. 3.2). As addressed before, stretch indicates folds to disappear and shrinkage reveals fold to emerge. Two differ- ent phases are identified during the integration step, i.e. integration is stopped if stretch/shrinkage state is changed. At the end of this step, our method have 14 Captured sequence Stretch direction & magnitude Wrinkle direction & amplitude Wrinkle control lines Space-time deformation Wrinkled system Integration & phase detection Fold tracing Stretch analysis Figure 4.1: System pipeline computed the two key elements to generate wrinkles: directions (perpendicular to stretch/shrinkage direction) and amplitude. In addition, we identify wrinkle regions as well, which are regions with relatively large stretch/shrinkage extracted. After the direction and magnitude is extracted, a fold tracing algorithm is de- signed to get the final control lines to generate folds (Sec. 3.3). The fold is traced in a 2D parameterized domain in order to force the control lines to be perpendicular to the normal direction. This restricts the deformation degree of freedom and greatly reduces the possibility of artifacts. The width between the control lines and the fold amplitude are both derived from the stretch magnitude. [28] is followed as the deformation method for generating folds. It constraints the space-time continuity a second time. 4.2 Stretch Analysis The stretch/shrinkage tensor field is analyzed between neighboring frames. This analysis focuses on small 2D shape deformation to get a sense of the shape de- formation transition. The tensor field provides the direction and magnitude of the local shape change, which will in turn be used to guide the missing fold’s shape and direction. The stretch/shrinkage tensor field is computed between a pair of correspond- ing triangles over time in order to obtain information about the local deformation both in time and space. We follow [30] to compute the stretch/shrinkage between two triangles. However, stretch/shrinkage is acquired as the local deformation in- formation instead of texture stretch. Besides, we are interested in the direction in addition to the magnitude. As we addressed in Section 2.1, we can derive largest 15 and smallest local stretches Γ and γ as well as two sets of corresponding stretch or shrinkage directions. One of the two principal directions is selected as the dominant direction of the local deformation. Since stretch and shrinkage should be treated equally, we convert both stretch and shrinkage each to absolute change, which is a value greater than or equal to 1 as in the following equation: |v|= { v v ≥ 1, 1/v v < 1. (4.1) where v is the stretch/shrinkage computed. The absolute change of a stretch is equal to itself, whereas the absolute change of a shrinkage is the reciprocal of the shrinkage. The absolute change is 1 when there is no stretch/shrinkage. We select one of the two principal directions and its corresponding stretch/shrinkage which has larger absolute change to represent the local deformation. Since the capture data usually has noise due to instrument precision in the acquisition setup, the stretch analysis results are smoothed to achieve a more con- tinuous input in space for better fold extraction. The tensor field has three inde- pendent components: maximum singular value Γ, minimum singular value γ , and singular vectors. Although there are two sets of singular vectors, each set has two perpendicular vectors, and two sets are different vectors in different spaces and is convertible to each other. Therefore, we can convert one of the directions to polar coordinates as d = (cosθ ,sin θ) and represent the third component as the angle θ . To smooth the tensor, each component is smoothed separately. Figure 4.2 gives an example of a comparison between the original and the smoothed tensor field. Note that after smoothing, the vector directions are more continuous and the dominant direction are more consistent. Captured data sometimes also has outliers due to various reasons. For exam- ple, captured results based on multi-view stereo often have outliers on the bound- aries, since the boundaries usually cannot be extracted accurately. Outliers can be removed simply by defining a valid range on stretch/shrinkage magnitude. More- 16 before smooth after smooth Figure 4.2: Tensor smooth. Only dominate directions are visualized by short lines. Stretch is represented by red line and shrinkage is represented by green line. Magnitude is also visualized by the length of the line. In order to easily find the difference of the magnitude, magnitude is represented by the difference of the absolute value and 1, and is scaled to fit the size of the triangle. over, we can analyze the stretch/shrinkage magnitude using a histogram and re- move those stretch and shrinkage with little frequency in the range near highest stretch and lowest shrinkage. 4.3 Phase detection and Stretch Integration Ideally, every triangle has a rest state when there is no local stretch. However, due to the error accumulation in time during the cloth sequence reconstruction pipeline, it is infeasible to estimate the rest state of a triangle for the whole sequence. There- fore, we pay more attention to the variation tendency over time. A consistent fold in a cloth sequence has three possible time phases. At first, 17 a fold starts to emerge and becomes more and more clear. Secondly, when a fold reach its maximum, it may hold its shape for a small period of time. Finally, the fold gradually disappears. A consistent shrinkage over time implies the fold emerging phase whereas a consistent stretch implies the fold disappearing phase, where consistency is determined by a small direction difference. Thus, phases are differentiated easily by an abrupt change in direction or state (stretch, shrinkage or isometry). Let vi(t) and di(t) denote the principal magnitude and direction of the stretch or shrinkage of a triangle t at time i−1 with respect to its corresponding triangle at time i. Relatively, v′i(t) and d′i(t) represent the stretch or shrinkage of a triangle t at time i+1 with respect to its corresponding triangle at time i. The boolean function defining the continuity of a triangle t at time i is as follows: continuous(t, i) = |di(t) ˙d′i(t)|> cosθ0 and (vi(t)−1)(vi(t)′−1)< 0 (4.2) where θ0 is the maximum angle difference allowed to ensure consistency. This function defines the stretch/shrinkage as continuous if magnitude is in the same phase and direction is consistent. Note that when we test whether two subsequent stretch directions are consistent, we need to transform the directions with respect to the same triangle and compute the angle difference in the local frame. When the function evaluates as true, neighboring triangles are classified in the same phase, and otherwise, a phase boundary occurs. Fold emerging or disappearing usually lasts a few frames. This leads to our further interest in the gradual changes of stretch or shrinkage along time, i.e. the stretch or shrinkage integration. Figure 4.3 is an example of the accumulated stretch or shrinkage of the three phases of a fold. Note that fold disappearing can be considered as fold emerging as well when watching the sequence in reverse. Therefore, we can do local stretch/shrinkage analysis both forwards and backwards in time and only consider areas with shrinkage. From now on, we do not need to differentiate between the fold emerging phase and the fold disappearing phase. Once we identify the boundaries of the phases using the function defined in Equa- tion 4.2, the shrinkage is accumulated in the same phase. Instead of accumulating the shrinkage along time, the shrinkage analysis is recomputed with respect to the 18 first triangle in the phase. The principal direction and magnitude are extracted to be used in the fold generation steps. When a fold reach its maximum, there will be a very small period of time (usu- ally 1-2 frames) that the fold will hold its maximum position. Intuitively, we expect no stretch in this phase. Since a fold emerging phase is always followed by a fold disappearing phase, it seems that we can integrate the tensor field across all three phases of an animated fold by simply changing the continuous function to ignore isometric tensor and the state difference. However, in practice, the deformation in the transition frames between the fold emerging and disappearing phases is not iso- metric. This is because the captured deformation is continuous and so is the tensor field. The tensor field always abruptly change in the direction and magnitude in order to achieve the transition from shrinkage to stretch. Therefore, fold phases are identified and treated separately. 4.4 Fold Tracing From the results of stretch accumulation, gradually changing stretch/shrinkage with consistent directions always occurs in the 3D space-time region where a fold is missing. The stretch/shrinkage direction and the missing fold direction are roughly perpendicular to each other. Also the absolute change of stretch/shrinkage always has a local maximum/minimum roughly at the fold peak. According to these ob- servations, we find fold regions by first detecting these local maximums, and then we locate individual fold regions guided by stretch/shrinkage information. Seed Detection: A triangle with a local shrinkage minimum is selected as the seed for detecting the whole region of a fold. If we detect all local minimums using a pre-defined neighborhood, we probably can get several seeds in one fold region. This is because before the region is located, we do not know how large areas can be considered as “local”. Thus, we choose to detect seeds one at a time. For each frame, we put all the absolute changes into a priority queue and pick a seed triangle with the largest value every time. We then locate a fold region using this seed and remove all the seed triangles that has been located inside this fold region from the priority queue. Axis Tracing: As we addressed, the fold direction is perpendicular to the 19 Figure 4.3: Tensor accumulation of the emergence and disappearing of a fold. Visualizations are sampled every three frames. For the first three rows, folds are emerging and the accumulated shrinkage is getting larger. In the fourth row, some of the folds are at their maximum, where there are no stretch or shrinkage. In the last line, folds are disappearing, making stretch appears instead of shrinkage. These pictures are visualized in the same way as in Figure4.2. 20 stretch/shrinkage direction. Therefore we can trace the fold axis along the shrink- age direction field. In order to compare the two neighboring directions for identify- ing the stopping criterion in tracing and to easily locate the control lines in the next step, the tracing is carried out in the 2D parameterized domain. We use ABF++[34] as the parameterization method to achieve minimum stretch and scaling. A line is traced in 2D along the direction perpendicular to the shrinkage direction until we reach two edges of the seed triangle. For the two ends of this tracing, we keep tracing in the neighboring triangle, using its local shrinkage direction. The tracing is continued in both directions until we get to a triangle with little stretch/shrinkage or an abrupt direction change. This gives us a piece-wise linear line on the surface, which defines the axis of the fold. Width Location: From [28], fold width is inversely proportional to the local rotation that forms the fold. In addition, a larger absolute change implies a larger local rotation. Thus, the fold width is set to be inversely proportional to the absolute change of shrinkage. By parameterizing the surface in 2D, we can get a pair of parallel control lines that are offset from the axis on both sides by a distance equal to half of the fold’s width . The location of these two lines is projected back to 3D using barycentric coordinates and plugged into the subsequent deformation framework in [28]. Region of Influence: As addressed in the seed detection part, we need to rule out all the triangles whose deformation is part of the origin from this particular fold. The width of the region is different from the width of control lines, for which a larger absolute change will lead to a smaller width of control lines. This width is set to be proportional to the absolute change since a more protuberant missing fold will generally influence a bigger region. The region is located by growing a the patch in 2D parameterized domain. The stopping criterion is ‖pt − pseed‖> α |v| (4.3) where pt is the 2D center position of a newly grown triangle, pseed is the 2D center position of the seed triangle, and ‖ · ‖ is the Euclidean distance between the two positions. v is the shrinkage magnitude of the seed triangle, and α is proportional factor. 21 Multi-Resolution Processing: One of the most important reasons for why high resolution folds are missing is that they are beyond the resolution of the ac- quisition setup. This makes adding higher folds on higher resolution mesh than the original sequence necessary. This is the case when the width of the fold is smaller than a single triangle’s size. Therefore, the model sequences are subdivided to recover more high frequency folds. The seed detection, axis tracing, and region of influence detection are carried out as usual while the pair of control lines are relocated on up-sampled model sequences. 4.5 Space-time Consistent Deformation After the pair of lines which represent the fold’s location and direction are acquired, the fold recovery is done using the space-time consistent deformation algorithm in [28]. The triangles which has offset points on the two parallel control lines are selected as the anchor triangles whereas the local interpolations of the directions are computed as the rotation axes. The rotation angle is set to proportional to the stretch/shrinkage magnitude. In [28], the space-time deformation method aims at propagating the rotations of the anchor triangles to their neighbouring triangles in both space and time. The method defines the rotation of each triangle as a convex combination of anchor rotations and identity rotations. The weight vectors for each triangle are calculated by minimizing an objective function which constraints both space and time con- sistency for neighbouring triangles. Soft constraints of full rotations (weight one) and identity rotations (weight zero) are set for anchor and non-anchor rotations, separately. After rotations of all triangles are computed, a quadratic function is minimized to recalculate the positions of vertices to meet the local rotation changes as well as stitching the mesh together. This method implicitly smoothes the fold generated in the previous step. 22 Chapter 5 Results We show a variety of wrinkled mesh sequences on simulated deformation sequence as well as captured cloth sequences. The captured inputs are reconstruction results by applying the method in [5]. The method applies multiview stereo followed by a consistent cross-parametrization to establish the correspondence. However, our system is not restricted to results generated using this method, and can be applied to results from other captured methods as well. Since we need detect consistent stretch or shrinkage regions in order to model fold animation, occlusions, depth information missing or incorrect correspondence mapping can cause a failure of folds detection in the local area. Figure 5.1 shows wrinkled results of a simulated deformation sequence. The folds pattern seems very regular, because the simulated deformation has nearly the same shrinkage directions in the shape changing area. Figure 5.2 shows part of the results for a mesh sequence of a shirt. This figure shows gradually emerging folds. Note that folds are traced along the shrinkage direction detected, which are not fully parallel, making the results more realistic. Figure 5.3 shows part of the results for a mesh sequence of a dress. This example illustrates a more complex set of folds and their interactions. 23 Figure 5.1: Wrinkled simulated mesh sequence of a hand model. Frames from 6 to 20. 24 Figure 5.2: Wrinkled captured mesh sequence of a shirt model. Frames from 44 to 58. 25 Figure 5.3: Wrinkled captured mesh sequence of a dress model. Frames from 206 to 220. 26 Chapter 6 Evaluation 6.1 Complexity Analysis and Implementation Details During the stretch analysis and accumulation stages, for every two corresponding triangles in adjacent frames, a stretch analysis is conducted. For every three corre- spondence triangles in three consequent frames, continuous function is evaluated. For every triangle except in the first frame, a new accumulated tensor is evaluated at most one time. Therefore, time complexity is in O(NT), where N is the total number of frames and T is the number of triangles in each frames. To save memory, one frame of model is loaded during the first two stages. In the stretch accumulation stage, for each triangle, a reference triangle is stored. It records the earliest corresponding triangle in time, when the continuous function is still being conformed at the current frame. This saves us from storing all the models of previous frames, and thus reduce the space complexity from O(NT) to O(T ). Since we are mostly dealing with are dense models containing over 20k triangles each, it is impossible to store the whole sequence of several hundreds of frames. This implementation makes the space complexity invariant to the num- ber of frames, making the wrinkle extraction part of the algorithm applicable to arbitrary long sequence on a single machine. At the wrinkle generation stage, for each seed, a patch is grown to locate the wrinkle influence region. Every triangle could probably be a seed and the patch can also be as large as the order of the model size. It seems that the time complexity 27 become O(T 2). However, a triangle already be in one of the influence region can no longer be consider as a seed, i.e. every triangle can only contribute for one fold region. Thus, the two nested square search space can actually be reduced to linear search space, making the the time complexity still O(T ) for each frame. At this stage, only stretch/shrinkage information and seed information in previous frame is loaded for each triangle at a time. The space complexity involved is still O(T ). According to [28], the bottle neck of the fold generation step is to solve the weight minimization energy function. Gradient conjugate is applied to solves equa- tions of Mx = b where M is a m∗n sparse matrix. Both m and n is of size O(AT), where A is the size of anchors. Solving of these batch of sparse matrices takes O(N(AT)2) time and is dramatically slower than the extraction steps. Therefore, substituting the fold extraction with the method proposed in this thesis will not add any burden to the original method. Moreover, this indicates that developing a more efficient generation step will strongly increase the whole pipeline efficiency. 6.2 Discussion of Adding Folds on Cloth Simulation Data and Cloth Captured Data This thesis focus working on adding folds on cloth captured data instead of simu- lation data. Although, in general the method is applicable for all cloth sequence, there are several key points to deal with for captured data. Rest State Cloth simulation usually has the rest state at hand, which simplifies the pipeline of the fold extraction steps. All the stretch/shrinkage analysis can be easily calculated with respect to the rest state of the cloth. However, captured data usually does not have rest state. Even though there are several frames where the model is standing still, the cloth is being stretched by gravity. Therefore, we need to analyze the change tendency instead of the absolute change of cloth stretches in the captured data. Stretch/Shrikage Regions When we compute tensor field in the simulation data, we can easily find a large area of similar deformation. The fold frequency we can put on this area is undefined, because we have not specified the materials yet. This is usually controlled either by deformation distance constraint to limit the amplitude of the fold or the curvature constraint to limit the bending angle. 28 However, for captured data, the stretch detected usually are more localized to a single fold. This is because the local information is captured instead of modeling the underlying mechanism. 6.3 Limitation The limitation of our algorithm stems from the reliance on the shape variation. Some wrinkles or folds that stay for the entire sequence, as they are formed by gravity or the underlying body does not change part of the poses during the moving. At this circumstance, in fact the shrinkage exist as opposed to the rest shape of the cloth. However, as we do not know the rest shape, we cannot discover any shape variations thus cannot discover the folds. 29 Chapter 7 Conclusion and Future Work 7.1 Conclusion This thesis proposes and develops a system of generating plausible realistic folds in a purely geometric way. It takes the low-stretch characteristic of cloth into con- sideration in the cloth capture. The relationship between fold shape and the stretch tensor field is established and evaluated. The system can be easily plugged in as a post-processing of most of the cloth capture system. Compared to many previous approaches, rest shape poses or examples of folds are not required, making it ap- plicable for capture sequence without the aid of any additional information, such as videos or simulated examples of folds. A stretch/shrinkage tensor is captured and analyzed in both space and time. Fold phases are automatically identified and stretch/shrinkage is integrated along time to reveal the gradually change of the fold. The algorithm is based on a very intuitive assumption on the linkage between fold shape and stretch/shrinkage ten- sor field, where fold direction is perpendicular to the shrinkage direction and fold location and width is related to the shrinkage magnitude as well. While fold is ex- tracted separately for each frame, seed information of previous frame is memorized and gives an indication of fold location in the current frame. The runtime of fold extraction part is reasonable on a single machine. Fold generation by deformation is the bottleneck of the system, but can be run in par- allel on cluster. The overall runtime is around 3-4 minutes per frame. As demon- 30 strated in Chapter 5, the method robustly recovers high frequency folds on captured sequences from different captured methods. The method is not sensitive to the se- quence length and start point, since the tendencies of variation of folds instead of the absolute changes are captured. 7.2 Future Work In the stretch/shrinkage tensor field analysis, a simple but effective smoothing scheme is applied to get a better shrinkage vector field in order to prevent fold fragmentation. A more dedicate smoothing algorithm can be studied to achieve so as well as better preserve the feature location(local maximum) of the fold. Although some information is memorized and propagated from previous frame to improve the continuity of the fold transition, a more elaborate 3D region detec- tion algorithm is demanding to achieve more continuity in time domain. Possible ideas is segment mesh triangles of different fold regions in 3D. The structure of the the sequence models can be modeled as a graph. Each triangle is represented as a node in the graph. Space and time adjacency is represented as edges between nodes. An energy function can be established by considering the consistency of shrinkage as well as time consistency and space consistency. Standard segmenta- tion methods may be generalized into 3D to solve this problem. Multi-dimensional algorithm could be used to deal with the potential high runtime. As the bottleneck of the system, a more efficient deformation algorithm would be an interesting endeavor. Instead of solving a batch of frames at a time, a single frame could be solved at a time. Prorogation and blending can be an alternative to establish the time consistency. Another possible improvement is to restricted the influence region of each anchor to a local area depending on the shrinkage extracted. This will dramatically reduce the number of triangle being solved for the weights function. From the modeling point of view, a more dedicate fold modeling method can improve the reality of the folds. Developing global modeling methods can advance the interaction between different patterns of folds. 31 Bibliography [1] N. Ahmed, C. Theobalt, P. Dobrev, H. Seidel, and S. Thrun. Robust fusion of dynamic shape and normal capture for high-quality reconstruction of time-varying geometry. 2008. → pages 11 [2] O. Au, H. Fu, C. Tai, and D. Cohen-Or. Handle-aware isolines for scalable shape editing. In ACM SIGGRAPH 2007 papers, pages 83–es. ACM, 2007. → pages 8 [3] K. Bhat, C. Twigg, J. Hodgins, P. Khosla, Z. Popovic, and S. Seitz. Estimating cloth simulation parameters from video. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, pages 37–51, 2003. → pages 10 [4] M. Botsch, M. Pauly, M. Gross, and L. Kobbelt. Primo: coupled prisms for intuitive surface modeling. In Proceedings of the fourth Eurographics symposium on Geometry processing, pages 11–20. Eurographics Association, 2006. → pages 8 [5] D. Bradley, T. Popa, A. Sheffer, W. Heidrich, and T. Boubekeur. Markerless garment capture. ACM Transactions on Graphics, 27(3):99, 2008. → pages 1, 10, 11, 23 [6] R. Bridson, R. Fedkiw, and J. Anderson. Robust treatment of collisions, contact and friction for cloth animation. ACM Transactions on Graphics, 21 (3):594–603, 2002. → pages 1, 9 [7] R. Bridson, S. Marino, and R. Fedkiw. Simulation of clothing with folds and wrinkles. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, pages 28–36. Eurographics Association Aire-la-Ville, Switzerland, Switzerland, 2003. → pages 1, 9 [8] K. Choi and H. Ko. Stable but responsive cloth. In ACM SIGGRAPH 2002 papers, pages 604–611. ACM, 2002. → pages 1, 9 32 [9] J. Combaz and F. Neyret. Painting folds using expansion textures. In Pacific Graphics, pages 176–183, 2002. → pages 12 [10] F. Cordier and N. Magnenat-Thalmann. A data-driven approach for real-time clothes simulation. In Computer Graphics Forum, volume 24, pages 173–183. Wiley Online Library, 2005. → pages 12 [11] L. Cutler, R. Gershbein, X. Wang, C. Curtis, E. Maigret, L. Prasso, and P. Farson. An art-directed wrinkle system for cg character clothing. In Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, pages 117–125. ACM, 2005. → pages 12 [12] E. de Aguiar, C. Stoll, C. Theobalt, N. Ahmed, H. Seidel, and S. Thrun. Performance capture from sparse multi-view video. In International Conference on Computer Graphics and Interactive Techniques. ACM New York, NY, USA, 2008. → pages 1, 10, 11 [13] E. de Aguiar, L. Sigal, A. Treuille, and J. Hodgins. Stable spaces for real-time clothing. In ACM SIGGRAPH 2010 papers, pages 1–9. ACM, 2010. → pages 10 [14] P. Decaudin, D. Julius, J. Wither, L. Boissieux, A. Sheffer, and M. Cani. Virtual garments: A fully geometric approach for clothing design. In Computer Graphics Forum, volume 25, pages 625–634. Blackwell Publishing, Inc, 2006. → pages 12 [15] E. English and R. Bridson. Animating developable surfaces using nonconforming elements. In ACM SIGGRAPH 2008 papers, pages 1–5. ACM, 2008. → pages 9 [16] M. Floater. Mean value coordinates. Computer Aided Geometric Design, 20 (1):19–27, 2003. → pages 8 [17] S. Hadap, E. Bangerter, P. Volino, and N. Magnenat-Thalmann. Animating wrinkles on clothes. In Proceedings of the conference on Visualization’99: Celebrating ten years, pages 175–182. IEEE Computer Society Press, 1999. → pages 12 [18] C. Hernández, G. Vogiatzis, G. Brostow, B. Stenger, and R. Cipolla. Non-rigid photometric stereo with colored lights. In Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on, pages 1–8. IEEE, 2007. → pages 11 33 [19] K. Hormann and M. Floater. Mean value coordinates for arbitrary planar polygons. ACM Transactions on Graphics (TOG), 25(4):1424–1441, 2006. → pages 8 [20] K. Hormann and N. Sukumar. Maximum entropy coordinates for arbitrary polytopes. In Computer Graphics Forum, volume 27, pages 1513–1520. Wiley Online Library, 2008. → pages 8 [21] J. Huang, X. Shi, X. Liu, K. Zhou, L. Wei, S. Teng, H. Bao, B. Guo, and H. Shum. Subspace gradient domain mesh deformation. ACM Transactions on Graphics (TOG), 25(3):1126–1134, 2006. → pages 8 [22] P. Joshi, M. Meyer, T. DeRose, B. Green, and T. Sanocki. Harmonic coordinates for character articulation. ACM Transactions on Graphics (TOG), 26(3):71–es, 2007. → pages 8 [23] H. Li, B. Adams, L. Guibas, and M. Pauly. Robust single view geometry and motion reconstruction. ACM Transactions on Graphics, 2009. → pages 11 [24] Y. Lipman, O. Sorkine, D. Levin, and D. Cohen-Or. Linear rotation-invariant coordinates for meshes. ACM Transactions on Graphics (TOG), 24(3):479–487, 2005. → pages 7, 8 [25] Y. Lipman, D. Levin, and D. Cohen-Or. Green coordinates. ACM Transactions on Graphics (TOG), 27(3):1–10, 2008. → pages 9 [26] J. Loviscach. Wrinkling coarse meshes on the GPU. In Computer Graphics Forum, volume 25, pages 467–476. Blackwell Publishing, Inc, 2006. → pages 13 [27] M. Müller and N. Chentanez. Wrinkle meshes. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pages 85–92. Eurographics Association, 2010. → pages 13 [28] T. Popa, Q. Zhou, D. Bradley, V. Kraevoy, H. Fu, A. Sheffer, and W. Heidrich. Wrinkling Captured Garments Using Space-Time Data-Driven Deformation. In Computer Graphics Forum, volume 28, pages 427–435. Blackwell Publishing Ltd, 2009. → pages iii, 2, 3, 11, 12, 15, 21, 22, 28 [29] D. Rohmer, T. Popa, M. Cani, S. Hahmann, and A. Sheffer. Animation wrinkling: augmenting coarse cloth simulations with realistic-looking wrinkles. In ACM Transactions on Graphics (TOG), volume 29, page 157. ACM, 2010. → pages 13 34 [30] P. Sander, J. Snyder, S. Gortler, and H. Hoppe. Texture mapping progressive meshes. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, page 416. ACM, 2001. → pages 15 [31] B. Sauvage, S. Hahmann, G. Bonneau, and F. LMC-IMAG. Length constrained multiresolution deformation for surface wrinkling. In IEEE International Conference on Shape Modeling and Applications, 2006. SMI 2006, pages 21–21, 2006. → pages 13 [32] V. Scholz, T. Stich, M. Keckeisen, M. Wacker, and M. Magnor. Garment motion capture using color-coded patterns. In Computer Graphics Forum, volume 24, pages 439–447. Wiley Online Library, 2005. → pages 10 [33] A. Sheffer and V. Kraevoy. Pyramid coordinates for morphing and deformation. In 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004. Proceedings. 2nd International Symposium on, pages 68–75. IEEE, 2004. → pages 8 [34] A. Sheffer, B. Lévy, M. Mogilnitsky, and A. Bogomyakov. ABF++: fast and robust angle based flattening. 2004. → pages 21 [35] X. Shi, K. Zhou, Y. Tong, M. Desbrun, H. Bao, and B. Guo. Mesh puppetry: cascading optimization of mesh deformation with inverse kinematics. In ACM SIGGRAPH 2007 papers, pages 81–es. ACM, 2007. → pages 8 [36] O. Sorkine and M. Alexa. As-rigid-as-possible surface modeling. In Proceedings of the fifth Eurographics symposium on Geometry processing, pages 109–116. Eurographics Association, 2007. → pages 8 [37] O. Sorkine, D. Cohen-Or, Y. Lipman, M. Alexa, C. Rössl, and H. Seidel. Laplacian surface editing. In Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing, pages 175–184. ACM, 2004. → pages 7, 8 [38] R. Sumner, M. Zwicker, C. Gotsman, and J. Popović. Mesh-based inverse kinematics. ACM Transactions on Graphics (TOG), 24(3):488–495, 2005. → pages 10 [39] B. Thomaszewski, S. Pabst, and W. Straßer. Continuum-based strain limiting. In Computer Graphics Forum, volume 28, pages 569–576. Wiley Online Library, 2009. → pages 9 [40] D. Vlasic, I. Baran, W. Matusik, and J. Popović. Articulated mesh animation from multi-view silhouettes. 2008. → pages 1, 10, 11 35 [41] H. Wang, F. Hecht, R. Ramamoorthi, and J. O’Brien. Example-based wrinkle synthesis for clothing animation. ACM Transactions on Graphics (TOG), 29(4):1–8, 2010. → pages 2, 12 [42] O. J. R. R. Wang, H. Data-driven elastic models for cloth: modeling and measurement. In ACM SIGGRAPH 2011 papers, pages 1–11. ACM, 2011. → pages 10 [43] O. Weber and C. Gotsman. Controllable conformal maps for shape deformation and interpolation. ACM Transactions on Graphics (TOG), 29 (4):78, 2010. → pages 9 [44] O. Weber, M. Ben-Chen, and C. Gotsman. Complex barycentric coordinates with applications to planar shape deformation. In Computer Graphics Forum, volume 28, pages 587–597. Wiley Online Library, 2009. → pages 9 [45] R. White, K. Crane, and D. Forsyth. Capturing and animating occluded cloth. In ACM SIGGRAPH 2007 papers, pages 34–es. ACM, 2007. → pages 1, 10 [46] Y. Yu, K. Zhou, D. Xu, X. Shi, H. Bao, B. Guo, and H. Shum. Mesh editing with poisson-based gradient field manipulation. In ACM Transactions on Graphics (TOG). → pages 7 [47] K. Zhou, J. Huang, J. Snyder, X. Liu, H. Bao, B. Guo, and H. Shum. Large mesh deformation using the volumetric graph laplacian. In ACM Transactions on Graphics (TOG), volume 24, pages 496–503. ACM, 2005. → pages 8 36
- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Theses and Dissertations /
- Plausible cloth wrinkling for captured mesh sequences
Open Collections
UBC Theses and Dissertations
Featured Collection
UBC Theses and Dissertations
Plausible cloth wrinkling for captured mesh sequences Chen, Xi 2011
pdf
Page Metadata
Item Metadata
Title | Plausible cloth wrinkling for captured mesh sequences |
Creator |
Chen, Xi |
Publisher | University of British Columbia |
Date Issued | 2011 |
Description | Cloth modeling has become an important research topic in computer graphics, as garments are ubiquitous in virtual environments used in movies or games. While recent studies can successfully capture the general shape of cloth as well as a continuous cloth motion, they usually lose high frequency details such as wrinkles or folds. This work provides a method of recovering high frequency details for captured cloth animation sequences without any aids of additional information. The method analyzes the stretch/shrinkage tensor fields in 4D space-time domain for the original cloth animation sequences. Based on the assumption of cloth developability, the stretch/shrinkage information is utilized to recover fold shapes and directions. Finally, a space-time consistent deformation method is applied to recover the high frequency folds. The work is generally applicable to most of the cloth capture methods applied either on scanned point clouds or video sequences. |
Genre |
Thesis/Dissertation |
Type |
Text |
Language | eng |
Date Available | 2011-10-25 |
Provider | Vancouver : University of British Columbia Library |
Rights | Attribution-NonCommercial-NoDerivatives 4.0 International |
DOI | 10.14288/1.0052095 |
URI | http://hdl.handle.net/2429/38243 |
Degree |
Master of Science - MSc |
Program |
Computer Science |
Affiliation |
Science, Faculty of Computer Science, Department of |
Degree Grantor | University of British Columbia |
GraduationDate | 2011-11 |
Campus |
UBCV |
Scholarly Level | Graduate |
Rights URI | http://creativecommons.org/licenses/by-nc-nd/4.0/ |
AggregatedSourceRepository | DSpace |
Download
- Media
- 24-ubc_2011_fall_chen_xi.pdf [ 1.72MB ]
- Metadata
- JSON: 24-1.0052095.json
- JSON-LD: 24-1.0052095-ld.json
- RDF/XML (Pretty): 24-1.0052095-rdf.xml
- RDF/JSON: 24-1.0052095-rdf.json
- Turtle: 24-1.0052095-turtle.txt
- N-Triples: 24-1.0052095-rdf-ntriples.txt
- Original Record: 24-1.0052095-source.json
- Full Text
- 24-1.0052095-fulltext.txt
- Citation
- 24-1.0052095.ris
Full Text
Cite
Citation Scheme:
Usage Statistics
Share
Embed
Customize your widget with the following options, then copy and paste the code below into the HTML
of your page to embed this item in your website.
<div id="ubcOpenCollectionsWidgetDisplay">
<script id="ubcOpenCollectionsWidget"
src="{[{embed.src}]}"
data-item="{[{embed.item}]}"
data-collection="{[{embed.collection}]}"
data-metadata="{[{embed.showMetadata}]}"
data-width="{[{embed.width}]}"
async >
</script>
</div>
Our image viewer uses the IIIF 2.0 standard.
To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.24.1-0052095/manifest