M u l t i p l e L i g h t S o u r c e O p t i c a l F l o w fo r T r a n s l a t i o n s a n d R o t a t i o n s i n the I m a g e P l a n e by Shingo Jason Takagi Honours B. Sc., University of Toronto, 1998 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR T H E DEGREE OF Master of Science in THE F A C U L T Y OF G R A D U A T E STUDIES (Department of Computer Science) We accept this thesis as conforming - to the required standard T h e U n i v e r s i t y o f B r i t i s h C o l u m b i a October 2000 © Shingo Jason Takagi, 2000 UBC Special Collections - Thesis Authorisation Form Page 1 of 1 In presenting t h i s thesis i n p a r t i a l f u l f i l m e n t of the requirements for an advanced degree at the University of B r i t i s h Columbia, I agree that the Library s h a l l make i t f r e e l y a v a i l a b l e for reference and study. I further agree that permission for extensive copying of th i s thesis for sc h o l a r l y purposes' may be granted by the head of my department or by his or her representatives. I t i s understood that copying or p u b l i c a t i o n of t h i s thesis for f i n a n c i a l gain s h a l l not be allowed .without my written permission. Department of Co The U n i v e r s i t y of B r i t i s h Columbia Vancouver, Canada Date http://www.library.ubc.ca/spcoll/thesauth.html 10/22/00 Abstract The classic optical flow constraint equation is accurate under conditions of translation and distant light sources, but becomes inaccurate under conditions where the object may rotate or deform. The inaccuracies are generally a result of the changing intensities at points under rotation or deformation. The changing intensities of these points are associated with the changing surface gradients. We investigate a novel approach to multiple light source optical flow under known reflectance properties. This novel approach is specialized for translation and for rotation around axes which are parallel to the optical axis. The assumption that the object is moving under translation, rotation, or a combination of translation androtation in the image plane allows us to introduce a physical surface area constraint. Even with this additional constraint, however, the problem still remains locally underdetermined. At each time frame, our multiple light source optical flow approach assumes that three images are acquired from the same viewpoint, but under three different illumination conditions. Photometric stereo determines many of the coefficients in our underdetermined system, which has six equations in seven unknowns at each pixel in the image. Two of the unknowns are the optical flow components. Another three of the unknowns are the total derivatives of the three intensities with respect to time. The last two are the total derivatives of the surface gradients with respect to time. A variety of local regularization methods were investigated to select optical flow estimates which best matched the known motion fields. All the experimental results for this approach were obtained from synthetic data, in which the motion fields were known. 11 Contents Abstract » Contents »i List of Tables v List of Figures xi Acknowledgements xviii 1 Optical Flow 1 1.1 . Introduction • 1 2 Multiple Light Source Optical Flow 4 2.1 Introduction ...4 3 Physical Surface Area Based Approach 7 3.1 Introduction 7 3.2 Photometric Surface Constraint 10 3.3 The Physical Surface Area Constraint For Rotations 13 3.4 Degenerate Cases 16 4 Implementation 20 4.1 Hardware and Software Setup 20 4.2 Local Regularization 25 5 Synthetic Image Data 35 5.1 Calibration sphere 35 5.2 Translation of a Sphere, 38 5.3 Curving Sheet 40 5.4 Curved Translating Sheet 47 5.5 Rotation of a Surface With Negative Gaussian Curvature 48 5.6 Translation of a Surface With Negative Gaussian Curvature 50 i n 5.7 Rotation and Translation of a Surface With Negative Gaussian Curvature 52 5.8 Rotation of a Surface With Positive Gaussian Curvature 54 5.9 Translation of a Surface With Positive Gaussian Curvature 56 5.10 Rotation and Translation of a Surface With Positive Gaussian Curvature... 58 6 Results..... 6 0 6.1 Optical Flow Estimation Techniques Tested 60 6.2 Optical Flow Estimation Quality Measures 63 6.3 Translation of the Calibration Sphere 70 6.4 Curving Sheet 80 6.5 Curved Translating Sheet 89 6.6 Rotation of a Surface With Negative Gaussian Curvature 98 6.7 Translationof a Surface With Negative Gaussian Curvature 108 6.8 .Rotation and Translation of a Surface with Negative Gaussian Curvature .... 118 6.9 Rotation of a Surface With Positive Gaussian Curvature 128 6.10 Translation of a Surface With Positive Gaussian Curvature 138 6.11 Rotation and Translation of a Surface with Positive Gaussian Curvature.. 148 7 Conclusions and Future Work 158 Bibliography . 162 iv List of Tables Table 6.1.1: The different optical flow estimation algorithms implemented and tested.. 62 Table 6.3.1: The averages and standard deviations of measures associated with the known motion field. The units for all measures are pixels 70 Table 6.3.2: The averages and standard deviations of measures associated with the type 1 optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels .71 Table 6.3.3. The averages and standard deviations of measures associated with the type 2A Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 72 Table 6.3.4: The averages and standard deviations of measures associated with the type 2A Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians All other measures are in pixels 73 Table 6.3.5: The averages and standard deviations of measures associated with the type 2A Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 74 Table 6.3.6: The averages and standard deviations of measures associated with the type 2 A Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 75 Table 6.3.7: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels. 76 Table 6.3.8. The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels. 77 Table 6.3.9: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 78 Table 6.3.10: The averages and standard deviations of measures associated with the type 2B^ 2B\ and 2B" Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 79 Table 6.4.1: The averages and standard deviations of measures associated with the known motion field. The measure r is unitless. All measures are in pixels. 80 Table 6.4.2: The averages and standard deviations.of measures associated with the type 2A Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 81 Table 6.4.3: The averages and standard deviations of measures associated with the type 2A Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 82 Table 6.4 4: The averages and standard deviations of measures associated with the type 2A Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 83 Table 6.4.5: The averages and standard deviations of measures associated with the type 2A Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 84 Table 6.4.6: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 85 Table 6.4.7: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 86 Table 6.4.8: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels .....87 Table 6.4.9: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 88 Table 6.5.1: The averages and standard deviations of measures associated with the known motion field. All measures are in pixels 89 Table 6.5.2: The averages and standard deviations of measures associated with the type 2A Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 90 Table 6.5.3: The averages and standard deviations of measures associated with the type 2A Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels ... 91 Table 6.5.4: The averages and standard deviations of measures associated with the type 2A Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians, All other measures are in pixels 92 Table 6.5.5: The averages and standard deviations of measures associated with the type 2A Min. D optical flow estimation. The measure r is unitless. The measure Z.'is in radians. All other measures are in pixels 93 Table 6.5.6: The averages and standard deviations of measures associated with the type • 2B, 2B', and 2B" Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 94 Table 6.5.7: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 95 Table 6.5.8: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 96 Table 6.5.9: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 97 Table 6.6.1: The averages and standard deviations of measures associated with the known motion field. The measure r is unitless. The measure Z is in radians.. All other measures are in pixels 98 vi Table 6.6.2: The averages and standard deviations of measures associated with the type 1 optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 99 Table 6.6.3: The averages and standard deviations of measures associated with the type 2A Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 100 Table 6.6.4: The averages and standard deviations of measures associated with the type 2A Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 101 . Table 6;6.5: The averages and standard deviations of measures associated with the type 2A Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 102 Table 6.6.6: The averages and standard deviations of measures associated with the type 2A Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 103 Table 6.6.7: The averages and standard deviations of measures associated with the type : 2B, 2B', and 2B" Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels...: :. 104 Table 6.6.8: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels...:.: 105 Table 6.6.9: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 106 Table 6.6. JO: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 107 . Table 6.7.1: The averages and standard deviations of measures associated with the known motion field. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 108 Table 6.7.2: The averages and standard deviations of measures associated with the type 1 optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 109 Table 6:7.3: The averages and standard deviations of measures associated with the type 2A Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 110 Table 6.7.4: The averages and standard deviations of measures associated with the type 2A Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels I l l Table 6.7.5: The averages and standard deviations of measures associated with the type 2A Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 112 Table 6.7.6: The averages and standard deviations of measures associated with the type 2A Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 113 vn Table 6.7.7: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 1 14 Table 6.7.8: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 115 Table 6.7.9: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 116 Table 6.7.10: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 117 Table' 6.8.1: The averages and standard deviations of measures associated with the known motion field. All measures are in pixels 118 Table 6.8.2: The averages and standard deviations of measures associated with the type 1 optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 119 Table 6.8.3: The averages and standard deviations of measures associated with the type 2A Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels...; ..: 120 Table 6.8.4: The averages and standard deviations of measures associated with the type 1 2A Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 121 Table 6.8.5: The averages and standard deviations of measures associated with the type 2A Min. C optical flow estimation. The measure r is unitless. The measure Z is in .s radians. All other measures are in pixels 122 Table 6.8.6: The averages and standard deviations of measures associated with the type • 2A Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 123 Table 6.8.7: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. A optical flow estimation. The measure r is unitless. The . measureZ is in radians. All other measures are in pixels 124 Table 6.8.8: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels...: 125 Table 6.8.9: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians All other measures are in pixels 126 Table 6.8.10: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 127 Table 6.9.1: The averages and standard deviations of measures associated with the known motion field. All measures are in pixels 128 viu Table 6.9.2: The averages and standard deviations of measures associated with the type 1 optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 129 Table 6.9.3: The averages and standard deviations of measures associated with the type 2A Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 130 Table 6.9.4: The averages and standard deviations of measures associated with the type 2A Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 131 Table 6.9.5: The averages and standard deviations of measures associated with the type 2A Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixets 132 Table 6.9.6: The averages and standard deviations of measures associated with the type 2A Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 133 Table 6.9.7:. The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 134 Table 6.9.8: The averages and standard deviations.of measures associated with the type 2B, 2B', and 2B" Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels. 135 Table 6.9.9: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels. 136 Table 6.9.10: The averages and standard deviations of measures associated with the type , 2B, 2B', and 2B" Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 137 Table 6.10.1: The averages and standard.deviations of measures associated with the known motion field. All measures are in pixels 138 Table 6.10.2: The averages and standard deviations of measures associated with the type . 1 optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 139 Table 6.10.3: The averages and standard deviations of measures associated with the type 2A Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians: All other measures are in pixels 140 Table 6.1.0.4: The averages and standard deviations of measures associated with the type 2A Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 141 Table 6.10.5: The averages and standard deviations of measures associated with the type 2A Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 142 Table 6.10.6: The averages and standard deviations of measures associated with the type 2A Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 143 ix Table 6.10.7: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 144 Table 6.10.8: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 145 Table 6.10.9: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 146 Table 6.10.10: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians: All other measures are in pixels 147 Table 6.11.1: The averages and standard deviations of measures associated with the known motion field. All measures are in pixels 148 Table 6.11.2: The averages and standard deviations of measures associated with the type 1 optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 149 Table 6.11.3: The averages and standard deviations of measures associated with the type "' 2A Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. AJI other measures are in pixels......:... 150 Table 6.11.4: The averages and standard deviations of measures associated with the type 2A Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 151 Table 6.11.5: The averages and standard deviations of measures associated with the type 2A Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 152 Table 6.11.6: The averages and standard deviations of measures associated with the type 2A Min. D. optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 153 Table 6.11:7: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. A optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 154 Table 6.1 1.8: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. B optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 155 Table 6.11.9: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. C optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels..: 156 Table 611.10: The averages and standard deviations of measures associated with the type 2B, 2B', and 2B" Min. D optical flow estimation. The measure r is unitless. The measure Z is in radians. All other measures are in pixels 157 List of Figures Figure 1.1.1 Figure 5.1:1 Figure 5.1.2 Figure 5.1.3 Figure 5.2.1 Imaging axis under orthographic projection 1 The calibration sphere under the red illumination 37 The calibration sphere under the green illumination 37 The calibration sphere under the blue illumination 38 The first of 11 images in the translating calibration sphere sequence. The red, green, and blue illuminations have been superimposed 39 Figure 5.2^ 2: The last of 11 images in the translating calibration sphere sequence. The red, green, and blue illuminations have been superimposed 39 Figure 5.3.1: Diagram depicting the parameterization of curving surface ...40 Figure 5.3:2: A wire mesh visualization of the curved sheet, at the end of its deformation. Units are in pixels". .....44 Figure 5.3.3: The first of 11 images in the curving sheet sequence. The red, green, and blue-illuminations-have been superimposed. 45 Figure 5.3.4: The last of 11 images in the curving sheet sequence. The red, green, and blue illuminations have been superimposed 45 Figure 5.4 I; The first of 11 images in the curved translating sheet sequence. The red, green, and blue illuminations have been superimposed .47 Figure 5.4.2: The last of 11 images in the curved translating sheet sequence. The red, green, and blue illuminations have been superimposed 48 Figure 5.5.1: A wire mesh visualization of the surface with negative Gaussian curvature. Units are in pixels. 49 Figure 5.5>2: The first of 11 images in the sequence of a rotating surface with negative Gaussian curvature. The red, green, and blue illuminations have been superimposed. ..; : 49 Figure 5.5.3: The last of 11 images in the sequence of a rotating surface with negative Gaussian curvature. The red,, green, and blue illuminations have been superimposed. : : '. 50 Figure 5.6,1: The first of 11 images in the sequence of a translating surface with negative Gaussian curvature. The red, green, and blue illuminations have been superimposed. t .....51 Figure 5.6.2: The last of 11 images in the sequence of a translating surface with negative Gaussian curvature. The red, green, and blue illuminations have been superimposed. . .' '. 52 Figure 5.7.1: The first of 11 images in the sequence of a rotating and translating surface with negative Gaussian curvature. The red, green, and blue illuminations have been superimposed 53 Figure 5.7.2: The last of 1 1 images in the sequence of a rotating and translating surface with negative Gaussian curvature. The red, green, and blue illuminations have been superimposed : 53 Figure 5.8.1: A wire mesh visualization of the surface with positive Gaussian curvature. Units are in pixels 54 Figure 5.8.2: The first of 11 images in the sequence of a rotating surface with positive Gaussian curvature. The red, green, and blue illuminations have been superimposed. ", ...55 XI Figure 5.8.3: The last of 11 images in the sequence of a rotating surface with positive Gaussian curvature. The red, green, and blue illuminations have been superimposed. , 56 Figure 5.9.1: The first of 11 images in the sequence of a translating surface with positive Gaussian curvature. The red, green, and blue illuminations have been superimposed. 57 Figure 5.9.2: The last of 11 images in the sequence of a translating surface with positive Gaussian curvature. The red, green, and blue illuminations have been superimposed. 58 Figure 5.10.1: The first of 11 images in the sequence of a rotating and translating surface with positive Gaussian curvature. The red, green, and blue illuminations have been superimposed 59 Figure 5.10.2: The last of 11 images in the sequence of a rotating and translating surface with positive Gaussian curvature. The red, green, and blue illuminations have been superimposed : 59 Figure 6.2.1 The 5 by 5 neighborhood of the pixel P. All cells labelled "N" are in the 5 by 5 neighborhood 63 Figure 6.3.1: The known motion field for the translating calibration sphere. Vectors are magnified 10 times and sampled every 20 pixels 70 Figure 6.3.2: The type 1 optical flow estimation for the translating calibration sphere. Vectors are magnified 10 times and sampled every 20 pixels 71 Figure 6.3.3: The type 2A Min. A optical flow estimation for the translating calibration sphere. Vectors are magnified 10 times and sampled every 20 pixels 72 Figure 6.3.4: The type 2A Min. B optical flow estimation for the translating calibration sphere. Vectors are magnified 10 times and sampled every 20 pixels 73 Figure 6.3.5: The type 2A Min. C optical flow estimation for the translating calibration sphere. Vectors are magnified 10 times and sampled every 20 pixels 74 Figure 6.3.6: The type 2A Min. D optical flow estimation for the translating calibration sphere. Vectors are magnified 10 times and sampled every 20 pixels 75 Figure 6.3.7: The type 2B Min. A optical flow estimation for the translating calibration sphere. Vectors are magnified 10 times and sampled every 20 pixels 76 Figure 6.3.8: The type 2B Min. B optical flow estimation for the translating calibration sphere. Vectors are magnified 10 times and sampled every 20 pixels 77 Figure 6.3.9: The type 2B Min. C optical flow estimation for the translating calibration sphere. Vectors are magnified 10 times and sampled every 20 pixels 78 Figure 6.3.10: The type 2B Min. D optical flow estimation for the translating calibration sphere. Vectors are magnified 10 times and sampled every 20 pixels 79 Figure 6.4.1: The known motion field for the curving sheet. Vectors are magnified 35 times and sampled every 20 pixels 80 Figure 6.4.2: The type 2A Min. A optical flow estimation for the curving sheet. Vectors are magnified 0.000000000000004 times and sampled every 20 pixels 81 Figure 6.4.3: The type 2 A Min. B optical flow estimation for the curving sheet. Vectors are magnified 2 times and sampled every 20 pixels 82 Figure 6.4.4: The type 2A Min. C optical flow estimation for the curving sheet. Vectors are magnified 2 times and sampled every 20 pixels 83 xn Figure 6.4.5: The type 2 A Min. D optical flow estimation for the curving sheet. Vectors are magnified 0.000000000000004 times and sampled every 20 pixels 84 Figure 6.4.6: The type 2B Min. A optical flow estimation for the curving sheet. Vectors are magnified 2 times and sampled every 20 pixels 85 Figure 6.4.7: The type 2B Min. B optical flow estimation for the curving sheet. Vectors are magnified 2 times and sampled every 20 pixels 86 Figure 6.4.8: The type 2B Min. C optical flow estimation for the curving sheet. Vectors are magnified 2 times and sampled every 20 pixels 87 Figure 6.4.9: The type 2B Min. D optical flow estimation for the curving sheet. Vectors are magnified 2 times and sampled every 20 pixels 88 Figure 6.5.1: The known motion field for the translating curved sheet. Vectors are magnified 10 times and sampled every 20 pixels 89 Figure 6.5.2: The type 2A Min. A optical flow estimation for the translating curved sheet. Vectors are magnified 0.00000000000001 times and sampled every 20 pixels 90 Figure 6.5.3: The type 2A Min. B optical flow estimation for the translating curved sheet. Vectors are magnified 10 times and sampled every 20 pixels 91 Figure 6.5.4: The type 2A Min. C optical flow estimation for the translating curved sheet. Vectors are magnified 10 times, and. sampled every 20 pixels 92 Figure 6.5.5: The type 2A Min. D optical flow estimation for the translating curved sheet. Vectors are magnified 0.00000000000001 times and sampled every 20 pixels. ...... .. : 93 Figure 6.5.6: The type 2B Min. A optical flow estimation for the translating curved sheet. Vectors are magnified 10 times and sampled every 20 pixels 94 Figure 6.5.7: The type 2B Min. B optical flow estimation for the translating curved sheet. Vectors are magnified 1.0 times and sampled every 20 pixels 95 Figure 6.5.8: The type 2B Min. C optical flow estimation for the translating curved sheet. Vectors are magnified 10 times and sampled every 20 pixels 96 Figure 6.5.9: The type 2B Min. D optical flow estimation for the translating curved sheet. Vectors are magnified 10 times and sampled every 20 pixels 97 Figure 6.6.1: The known motion field for the rotating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 98 Figure 6.6.2: The type 1 optical flow estimation for the rotating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels. 99 Figure 6.6.3: The type 2 A Min. A optical flow estimation for the rotating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels. : 100 Figure 6.6.4: The type 2A Min. B optical flow estimation for the rotating surface with negative Gaussian, curvature. Vectors are magnified 10 times and sampled every 20 pixels : 101 Figure 6.6.5: The type 2A Min. C optical flow estimation for the rotating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 102 Figure 6.6.6: The type 2A Min. D optical flow estimation for the rotating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels •. 103 xiii Figure 6.6.7: The type 2B Min. A optical flow estimation for the rotating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 104 Figure 6.6.8: The type 2B Min. B optical flow estimation for the rotating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 105 Figure 6.6.9: The type 2B Min. C optical flow estimation for the rotating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels.. . 106 Figure 6.6.10; The type 2B Min. D optical flow estimation for the rotating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels . . : 107 Figure.6.7.1: The known motion field for the translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 108 Figure 6.7.2: The type 1 optical flow estimation for the translating surface with negative Gaussian curvature. Vectors are magnified 1.0 times and sampled every 20 pixels. • : 109 Figure 6.7.3: The type 2A Min. A optical flow estimation for the translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels.! •. ; 110 Figure 6.7.4: The type 2A Min. B optical flow estimation for the translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels...: .' : 111 Figure 6.7.5: The type 2A Min. C optical flow estimation for the translating surface with negative Gaussian curvature. Vectors are, magnified 10 times and sampled every 20 pixels .: 112 Figure 6.7.6: The type 2A Min. D optical flow estimation for the translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels.... : :, 113 Figure 6.7.7: The type 2B Min. A optical flow estimation for the translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels.... : : 114 Figure 6.7.8: The type 2B Min. B optical flow estimation for the translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels •. 115 Figure 6.7.9: The type 2B Min. C optical flow estimation for the translating surface with negative Gaussian curvature. Vectors;are magnified 10 times and sampled every 20 pixels 116 Figure 6.7.10: The type 2B Min. D optical flow estimation for the translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 117 Figure 6.8.1: The known motion field for the rotating and translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 118 xiv Figure 6.8.2: The type 1 optical flow estimation for the rotating and translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 1 19 Figure 6.8.3: The type 2A Min. A optical flow estimation for the rotating and translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 120 Figure 6.8.4. The type 2A Min. B optical flow estimation for the rotating and translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 121 Figure 6.8.5: The type 2A Min. C optical flow estimation for the rotating and translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 122 Figure 6.8.6: The type 2A Min. D optical flow estimation for the rotating and translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 123 Figure 6.8.7: The type 2B Min. A optical flow estimation for the rotating and translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels... 124 Figure 6.8.8: The type 2B Min. B optical flow estimation for the rotating and translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 125 Figure 6.8.9: The type 2B Min. C optical flow estimation for the rotating and translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 126 Figure 6,8.10: The type 2B Min. D optical flow estimation for the rotating and translating surface with negative Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 127 Figure 6.9.1: The known motion field for the rotating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 128 Figure 6.9.2: The type 1 optical flow estimation for the rotating surface with positive Gaussian curvature. Vectors are magnified 20 times and sampled every 20 pixels. 129 Figure 6.9.3: The type 2 A Min, A optical flow estimation for the rotating surface with positive Gaussian curvature. Vectors are magnified 20 times and sampled every 20 pixels 130 Figure 6:9.4: The type 2A Min. B optical flow estimation for the rotating surface with positive Gaussian curvature. Vectors are magnified 20 times and sampled every 20 pixels.... 131 Figure 6.9.5: The type 2A Min. C optical flow estimation for the rotating surface with positive Gaussian curvature. Vectors are magnified 20 times and sampled every 20 pixels 132 Figure 6.9.6: The type 2A Min. D optical flow estimation for the rotating surface with positive Gaussian curvature. Vectors are magnified 20 times and sampled every 20 pixels 133 xv Figure 6.9.7: The type 2B Min. A optical flow estimation for the rotating surface with positive Gaussian curvature. Vectors are magnified 20 times and sampled every 20 pixels 134 Figure 6.9.8: The type 2B Min. B optical flow estimation for the rotating surface with positive Gaussian curvature. Vectors are magnified 20 times and sampled every 20 pixels... 135 Figure 6.9.9: The type 2B Min. C optical flow estimation for the rotating surface with positive Gaussian curvature. Vectors are magnified 20 times and sampled every 20 pixels 136 Figure 6.9.10: The type 2B Min. D optical flow estimation for the rotating surface with positive Gaussian curvature. Vectors are magnified.20 times and sampled every 20 pixels 137 Figure 6.10.1: The known motion field for the translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 138 Figure 6.10.2: The type 1 optical flow estimation for the translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels. .' : ' 139 Figure 6.10.3: The type 2A Min. A optical flow estimation for the translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 140-Figure 6.10.4: The type 2A Min. B optical flow estimation for the translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 141 Figure 6.10.5: The type 2A Min. C optical flow estimation for the translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels • :. 142 Figure 6.10.6: The type 2A Min. D optical flow estimation for the translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 143 Figure 6.10.7: The type 2B Min. A optical flow estimation for the translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 144 Figure 6.10.8: The type 2B Min. B optical flow estimation for the translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 145 Figure 6.10.9: The type 2B Min. C optical flow estimation for the translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 146 Figure 6.10.10: The type 2B Min. D optical flow estimation for the translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels .147 Figure'6.11.1: The known motion field for the rotating and translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 148 xvi Figure 6.11.2: The type 1 optical flow estimation for the rotating and translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 149 Figure 6.11.3: The type 2A Min. A optical flow estimation for the rotating and translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 150 Figure 6.11.4: The type 2A Min. B optical flow estimation for the rotating and translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 151 Figure 6.11.5: The type 2A Min. C optical flow estimation for the rotating and translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels. 152 Figure 6.11.6. The type 2A Min. D optical flow estimation for the rotating and translating surface with positive Gaussian curvature. Vectors are magnified 10 '•' times and sampled every 20 pixels •: 153 Figure 6.11.7: The type 2B Min. A optical flow estimation for the rotating and translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 154 Figure 6.11.8: The type 2B Min. B optical flow estimation for the rotating and translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 155 Figure 6.11.9: The type 2B Min C optical flow estimation for the rotating and translating surface with positive Gaussian curvature. Vectors are magnified 10 .times and sampled every 20 pixels 156 Figure 6.11.10: The type 2B Min. D optical flow estimation for the rotating and translating surface with positive Gaussian curvature. Vectors are magnified 10 times and sampled every 20 pixels 157 XVI1 Acknowledgements I would like to thank my supervisor Dr. Robert J. Woodham for his guidance, support and insightful comments on. all aspects of this thesis, Dr. James Little, for being my second reader, and providing valuable feedback on this thesis, and the technical and support staff in the Computer Science department, who helped solve some of my major computer problems. The Laboratory for Computational Intelligence (LCI), the Active Measurement Facility (ACME), and the entire Computer Science department has been a fun and exciting place to work. Last but not least, I would like to thank all my family and friends. They have all helped me to complete this thesis, in one way or another. Shingo Jason Takagi The Univesity of British Columbia October 2000 xviii C h a p t e r O n e 1 Optical Flow 1.1 I n t r o d u c t i o n In this thesis we mainly investigate multiple light source optical flow [18] for objects which are translating or rotating around an axis parallel to the optical axis. The optical axis is the axis which passes through the image center and the optical center. In Figure 1.1.1, the imaging axis is the z-axis. Optical flow [7,9] is the apparent motion of the image intensity pattern. Often the optical flow will correspond with the actual motion field in the scene, but there are many cases in which this is not true. The classic example is that of a sphere that is rotating. In the absence of distinct surface markings, the image sequence of a sphere that is rotating will •1 my Figure 1.1.1: Imaging axis under orthographic projection. be constant since there are no changes in the image intensity pattern. Hence the optical flow will be zero at points on the sphere. However the actual motion field is a rotation of the sphere. Often in the calculation of the optical flow, it is assumed that the radiance of a point E(x,yJ) does not change over time. This can be stated mathematically as dE(x,yJ) 3E dE dE A — 7 =—it+-—v + — = 0 dt dx dy dt Equation 1.1.1 where u, and v are dxl dt, and dyldt respectively. This assumption implies that no matter how the surface normal at that point changes over time its radiance is constant. This is true under translation under distant light sources, but in general this assumption is not correct. For example, under rotation, the radiance at a point changes with time as its surface normal changes with respect to the lights sources and the viewer. It is enlightening to see that it is possible to solve Equation 1.1.1 for the normal velocity. The normal velocity is the component of (w, v) in the direction of the intensity gradient. If we let the normal velocity be vn, then _ -dEldt VE V" ~ | V E | X | V £ | Equat ion 1.1.2 However the velocity perpendicular to the normal velocity cannot be recovered, without additional information. Essentially the problem is that the system is under-determined. Suppose we let a surface be represented by z = /(x, y), under orthographic projection, where z points along the optical axis, and x and^ form the image coordinates, 2 as in Figure 1.1.1. Letting the partial derivatives of/be represented by p and q, we have . df df p = — , and q = — . The general image irradiance equation can be written as dx dy E(x,y) = R(p,q) Equa t ion 1.1.3 where E is the image irradiance at the point (x,y) in the image and R is the scene radiance produced by a surface point with p and q as partial derivatives. Note that x and y are functions of time t, and p and q are functions of x,y and hence t. Therefore compared to Equation 1.1.1, the general equation for dEldt is dt dx dy dt dp Equa t ion 1.1.4 In Equation 1.1.1 and Equation 1.1.4, it is the u and v quantities that form the optical flow. Equation 1.1.1 is an equation in two unknowns, u and v, so it is not possible to solve for u and v uniquely, without regularization. Often, the minimum norm solution of (v,v) is chosen as an initial approximation for some global regularization method, in these cases. dp dp dp — U+ — V + — dx dy dt dR dq dq dq dq — u + — v + — dx dy dt C h a p t e r T w o 2 Multiple Light Source Optical Flow 2 . 1 Introduction Photometric stereo is used in our optical flow estimation to help introduce more knowledge of the three dimensional world into pur optical flow equations. Photometric stereo allows us to determine surface gradients at all points on an imaged object, and hence its curvature. It involves the use of multiple images of the same scene taken ' simultaneously from the same view point, but under different illumination conditions. This is typically accomplished by illuminating the scene with three light sources from different directions. The light sources are isolated in the red, green, and blue spectrums. A colour camera is then used to capture the scene. The red, green, and blue planes of the colour image records the scene under the red, green, and blue illuminations conditions, respectively. In photometric stereo [17], the situation can be described with three image irradiance equations. E](x,y) = Rx{p,q) E2(x,y) = R2(p,q) E3(x,y) = R3(p,q) Equat ion 2.1.1 4 C. E. Siegerist's work [14,15] uses the classic optical flow constraint equation, and assumes that the image irradiance E is constant over time. dE = E * + E ± + E 0 dt dt dt . „ dE „ 8E dE where Ex - — E v = -— Et = — dx ' dy dt Equation 2.1.2 If three images are acquired, we end up with a system of three linear equations in two unknowns. The unknowns are dx/dt, and dyldt, which form the optical flow. The equations are as follows. E]xii + E]yv + Eu =0 E 2 J ' + E 2 v V + E2< = .° E3x" + Eiyv + E3t =0 Equation 2.1.3 dx dv where, /./= — , v- — . In general this is an over determined system, and the least dt df squares solution for v and v can be found as = (A1 AylAT where A E E F F x > 2 . v z - ' 2 v E3, Equation 2.1.4 5 However, under certain conditions, Equation 2.1.3 will not be over determined, and may be under determined. This will be discussed later in section 3.4 on degenerate cases. C. E . Siegerist's work [14,15] on real time multiple light source optical flow shows that this method works quite well for translational motion. However the assumption that dE/ dt -0 is only correct under translation and infinitely distant light sources. In order to avoid making this assumption, Equation 1.1.4, the derivative of the full image irradiance equation with respect to time, was used. David Hsu's work [11] extends the classic optical flow constraint equation to an equation similar to Equation 1.1.4 for the three light sources. It tries to account for cases which are not just translational in nature by accounting for the changes in the intensity of a point due to surface normal changes at that point. This leads to an underdetermined system, and he was able to show that by tuning the regularization parameters, a minimal rotation, or a maximal rotation solution for the optical flow could be obtained. The minimal rotation solution favored a solution in which the surface normal did not change. This tends to yield the same results as if the optical flow constraint equation, Equation 1.1.1, had been used, and is accurate under conditions of translations. The maximal rotation solution favors a solution in which the surface normals change significantly. This yields results which are quite different from those had the optical flow constraint equation been used. 6 C h a p t e r T h r e e 3 Physical Surface Area Based Approach 3.1 Introduction In our method, we also extend the optical flow constraint equations, and create an initial system of equations which are similar to that used by David Hsu [11], except that we have constructed the reflectance map, R, as a function of the surface gradient, p and q, instead of the unit surface normal, which can represented by three variables, and an additional constraint equation. We use the full image irradiance equation, Equation 1.1.4, for the three imaging conditions in multiple light source optical flow. The only unknowns in these equations are •;./ and v. The other entities can be calculated from the image sequence or knowledge of the reflectance map. E]xu + E,vv + Eu =RX dp . dq dt dt Equation 3.1.1 E2xu + E2vv + E2l =R. dp dt Equation 3.1.2 E , n + E, v + £ , , =R, dp + R, dq dt dt Equation 3.1.3 Note that 7 dp dp dp dp — = —u + — v + — dt dx dy dt Equation 3.1.4 and dq dq dq dq — = —u + — v + — dt dx dy dt Equation 3.1.5 Note that dq dx £>, dq dq Yt 2p ^ 2q OX dq dx E2y dq 'dy E2, dq ~dt Eix dq dx dq_ dy E3! dq dt Equation 3.1.6 The above equations can be rearranged as 8 ( Equa t ion 3.1.7 E2xu + E2yv-a2=-E2l Equat ion 3.1.8 E3xu + E3yv-a3 =-E3t Equa t ion 3.1.9 PS< + P?v-a = -P< Equa t ion 3.1.10 Equat ion 3.1.11 And in matrix form the above can be represented as ll Eu - 1 0 0 0 0 " V E2x E2y 0 - 1 0 0 0 or. -E2t E3x E3y 0 0 - 1 0 0 a2 = ~E3l Px Py 0 0 0 - 1 0 a3 ~ Pt qx 0 0 0 0 - 1 a b Equat ion 3.1.12 9 • • , • • ndp dq dR dR7 where a, b,a., a7, and a-, are equal to the total derivatives of — , — — L , — - , and dt dt dt df dR- , , dE, dE7 , dE, — respectively. The variables a], a2, and a 3 are also equal to — - , — - , and dt " 1 ' 1 dt' dt' dt . , , dE dR respectively, because -—- = — L , for / = 1,2, and 3. • dt dt 3.2 Photometric Surface Constraint R. J. Woodham [17] introduced the notion that all the intensity triples from a lambertian surface, under photometric stereo conditions, lie on an ellipsoid in the space of intensity triples. Elli Angelopoulou and Lawrence B. Wolff [1,2,3] extended this to show that the intensity triples from a generalized diffuse surface lie on a closed convex surface in the space of intensity triples. This closed convex surface will be referred to as the photometric surface. At any given point in the image, its intensity triple will correspond to a point on this photometric surface. As time progresses, the surface normal (A7, ,n2, /?3) and the intensity triple at a point in the image will change and the respective point on the photometric surface will move to a new point on the photometric surface. Because the point is constrained to lie on this photometric surface, its velocity vector, .dR, dRn dR, x , . . . . . . . , ,. , (— L ,———), on the photometric surface will always be perpendicular to the surface dt dt di normal of this photometric surface. From the notation used in Equation 3.1.7, Equation 3.1.8, and Equation 3.1.9, the velocity vector can be written as (ax,a2,ai). Let us define the photometric surface as 10 F(E],E2,E3) = 0 . , Equa t ion 3.2.1 Then the normal (nx,n2,n3) to.this surface is , .dF dF dF, . (n^n2,n3)-(-.—, ,—-) . dE. dE2 dh, Equa t ion 3.2.2 Because of the relation in Equation 2.1.1, the surface normal can be equivalently expressed as a cross product of vector derivatives of the reflectance with respect to p and q. (n„n2,n3) = (Rlp,R2p,R3p)x(R]q,R2q,R3q) ' • = (R2pR3q ~R3pR2q^R3pR\q -R\pR3crR^R2y-R2pRUi) Equa t ion 3.2.3 Therefore the photometric constraint can be written as (R2PR3,-R^R3pK - / J i A^A ' -^A) , ( a " O i ' a 3 . ) = 0 Equa t ion 3.2.4 Hence it appears that knowledge of all the photometric surface normals will allow us to introduce an additional constraint on our optical flow equations. However this is not the case. In fact the photometric constraint is a consequence of Equation 3.1.7, Equation 3.1.8, Equation 3.1.9, Equation 3.1.10, and Equation 3.1.11. To see this, let us first convert Equation 3.1.7, Equation 3.1.8, and Equation 3.1.9 into equations using derivatives of the reflectance, using the identities in Equation 3.1.6. Equation 3.1.7, Equation 3.1.8, and Equation 3.1.9 become 11 (RiPpx + K,qx > + (R,ppy + V / , ) , ; - « . = - R\,>P> - R ^ , Equat ion 3.2.5 (R2PPx + R2qQX >' + (R2pPv + R2cfly > ~a2= - R 2 p P , ~ R2qq, Equa t ion 3.2.6 (R3pPx + R3,qx>' + ( R 3 p P y + R i q « y > - a, = -R3pp, - R3qq, Equa t ion 3.2.7 Now we can subtract Equation 3.1.10 multiplied byi?1;)and Equation 3.1.11 multiplied by RUl, from Equation 3.2.5 to get -a,+Ripa + R]qb = 0~ Equa t ion 3.2.8 Subtracting Equation 3.1.10 multiplied by R2p and Equation 3.1.11 multiplied by R2 from Equation 3.2.6, we get - a 2 + R 2 p a + R2qb = 0 Equa t ion 3.2.9 Subtracting Equation 3.1.10 multiplied byR 3 p and Equation 3.1.11 multiplied by R3q, from Equation 3.2.7, we get •a3 +R3pa + R3qb - 0 Equa t ion 3.2.10 12 Finally summing up Equation 3.2.8 multiplied by -(R2pR^q - R^pR2q), Equation 3.2.9 multiplied by -(R3pRlq -R]pR^ ), and Equation 3.2.10 multiplied by -(R]pR2q-R2pR]q)y\c\ds (R2pR3q-R3pR2q)a, +(R3pR]q-RlpR3q)a2 + (RlpR2q -R2pRhl)a3 = 0 / Equation 3.2.11 which is just the photometric surface constraint, Equation 3.2.4. Hence the photometric surface constraint equation does not provide any additional constraint to our existing equations. 3.3 The Physical Surface Area Constraint For Rotations Accounting for physical surface area in the scene however does provide an additional constraint on our optical flow equations. Let us represent a surface as z = f(x, y), a function on x and y. Now let us consider trying to find the area of a patch • df df Q. Let us assume that we know the value ofp and q at (x,y\ where p = — ,and q = — . dx . • dy The area S of the surface over a patch Q in the x and y plane can be expressed as follows. Let n be the surface normal. w = ( l , 0 J J p ) x ( 0 , l , ^ ) = ( - j p - ^ l ) Equation 3.3.1 13 S - jj \rtyxdy - jjp2 + cf + \dxdy n Q Equat ion 3.3.2 Then a small area ds, at (xj/) can be expressed as ds = T]\ + p2 + q2dxdy Equa t ion 3.3.3 The surface gradients p and q are functions of x and y, and the x and y are themselves functions of the time, /. So the infinitesimal area, ds, will remain constant under conditions of translation. This is because the object is only translating and the surface gradients at a given (x,y), with respect to the viewer, are not changing. The infinitesimal area, ds, will also remain constant under conditions of rotation around an axis parallel to the optical axis, since, in this case, (p2 +q2) is constant. Therefore under conditions of translations and rotations around axes parallel to. ;the optical axis, the area ds should not change as time moves along. This can be expressed as d(ds) _ Q dt Equat ion 3.3.4 Let us denote ds by A for convenience. Then we have. dA(x,y,t) . A A — J = A ii + A v+ A, dt x Equat ion 3.3.5 14 ^ - ' ( 1 + ^ + ^ ( 2 ^ + 2 ^ ) = » - + « -Equat ion 3.3.6 dA ppv+qqx dy yj\ + p2 + q2 Equa t ion 3.3.7 dA _ pp,+qq, dt J\ + p2+q2 Equat ion 3.3.8 So combining these equations, we get: (ppx + qqx >' + (pp., + qqy ) V+PP, + qq, = o Equat ion 3.3.9 Combining Equation 3.3.9 with Equation 3.1.7, Equation 3.1.8, Equation 3.1.9, Equation 3.1.10, and Equation 3.1.11, we get a system of 6 equations in 7 unknowns in the best case. This will be the system that we use in our approach to solve for optical flow. 15 E2x Px q* _PPx + 3.4 Degenerate Cases In order to solve for the optical flow using Equation 2.1.3 or Equation 3.3.10, we generally need to calculate the matrix product Ar A . For solving Equation 2.1.3, we use ATA as described in Equation 2.1.4. For solving Equation 3.3.10, AT A is used in a local regularization process which is described in section 4.2 on local regularization. Hence knowledge of the rank of AT A is helpful in determining whether a solution can be found. In Equation 2.1.3, u and v are over determined. There are, however, several cases in which this system of equations will degenerate, and it becomes impossible to solve for the optical flow using the method described after the definition of the systems of Equation 2.1.3. These degeneracies occur when the object's surface is planar or developable. When the surface being imaged is planar, the matrix A has no linearly independent rows. To see this we just need to note that if the surface is planar then Eix =Eiy =0 for all i=l, 2, and 3 Equat ion 3.4.1 -1 0 0 0 0 E2y 0 -1 0 0 0 0 0 -1 0 0 Py 0 0 0 -1 0 0 0 0 0 -1 pp. + qqv 0 0 0 0 0 II V a, a2 a b - P , - pp, - qq, Equat ion 3.3.10 16 as the image intensity is the same at all points on the planar surface. Hence A T A , will have a rank of zero. When the surface is developable, the matrix A has only one linearly independent row. The reason why the matrix A has one linearly independent row is due to the following property which occurs when the surface is developable [16]. E E — = for 7=1, 2, 3, and /=!, 2, 3 E E IX J* Equation 3.4.2 To see this we first note that X " ' R ' Px Pv = H p where H = i y A. 3* Equation 3.4.3 H is known as the Hessian matrix, and for developable surfaces it can be shown that the determinant of the Hessian is zero. Because H is positive definite, it can be factored as H = RT X 0 0 0 R, where 7^ is a rotation matrix. Equation 3.4.4 Hence no matter what ( R p , R q ) is, it will be mapped to a point, (Ex,£v),ona straight line. From this it can be shown that Equation 3.4.2 holds. Since A has one linearly independent row, the matrix product, A T A , will have a rank less than or equal to one. This follows from the following inequality of matrix ranks [13]. 17 rank(BC) < rank(B), and rank(BC) < rank(C) Equat ion 3.4.5 Equation 3.3.10 also has similar degenerate cases. Before we delve into these degenerate cases, which can decrease the rank of AT A , let us first prove another lemma concerning the number of independent rows of A and the rank of AT A . Lemma 1. If ,4 is an m by p matrix with m<p and has m independent rows, then AT A has full rank. (In other words, the rank of ATA is equal to m.) Proof: This follows from Sylvester's inequality [10, page 13] rank(B) + rank(C) - k < rank{BC) < mm(rank(B),rarik(C) Equa t ion 3.4.6 where B is an m by k matrix with k independent columns, and C is an arbitrary k by n matrix. If we let B equal A7, and G equal A, we get m < rank(AT A) < m Equa t ion 3.4.7 as k is equal to m, here. Hence the rank of AT A is m, if A is m by p with m independent rows. Let us consider the Equation 3.3.10 as^ 4x=Z>. The maximum number of independent rows A can have is 6. In this case, from lemma 1, A1 A will have full rank. 18 The minimium number of independent rows ,4 can have is 5. In this case, from the inequalities in Equation 3.4.5, AT A will have a rank less than or equal to 5. A has a minimum of 5 independent rows because the first five rows has an element, -1, in a column where all other rows have zero. Hence these 5 rows are able to span a space with dimension 5, and are linearly independent. The matrix. A, will have 5 independent rows when (ppx + qqx) and (ppv + qqv) from the last row of Equation 3.3?10 are zero. If (PPx + cI1x) o r (PPV + acly ) a r e n ° t equal to zero, the matrix A will have 6 independent rows. There are many conditions which might cause (ppx +qqx) and (ppy +qqv) to be zero. An obvious case is when the surface is planar. In this case thep and q are constant and the partial derivatives ofp and q are zero. It is interesting to note however that developable surfaces do not in general cause (ppx + qqx) and (ppv +qqv) to be zero, and hence do not in general decrease the rank of matrix A as they did when we were considering the matrix A from Equation 2.1.3, which uses the classic optical flow constraint equations. 19 C h a p t e r F o u r 4 Implementation 4.1 Hardware and Software Setup Optical flow calculations were carried out on a Pentium III 300 MHz computer. The programs were written in Visual C++ 6.0, and made use of C L A P A C K (C Linear Algebra Package), and Microsoft's Vision Software Development Kit. One synthetic image was synthesized using Kinetic's 3D Studio Max, but most of the synthetic data was created by self made programs through a knowledge of surface gradients and reflectance maps. In order to estimate the optical flow, the Equation 3.3.10 is solved through a local regularization at each pixel for the seven unknowns. All the elements in the 6 by 7 matrix, A, and the 6 by 1 matrix, b, can be calculated. The image gradients Elx, Eu,, Eu, E2x, E2y, E2t, E3x, E3v, and E3l, can be determined by taking partial derivatives in the image sequence. The surface gradients, p and q, can be determined from the photometric stereo lookup table. The derivatives of the surface gradients px, pv, pt, qx, qy, and qt can be determined by taking discrete derivatives ofp and q, but a method which is less sensitive to noise is used. Before the image gradients are calculated, all the images are smoothed with a 5 by 5 discrete Gaussian filter, with a - 1. The image gradients are calculated by using a kernel of size 5. The following are the exact values in this difference kernel. 20 diff_kernel[-2] = (float)-l. 0/12.0 diff_kernel[-l] = (float)8.0/12.0 diff_kernel[0] = (float)O.O diff_kernel[l] = (float)-8.0/12.0 diff_kernel[2] = (float)l. 0/12.0 This kernel has element 0 as the center element, and is convolved with the smoothed images in the x, and orientations, and also in time. The gradients in time are taken over 5 images, 2 before and after in the time sequence to the image of interest. Photometric stereo [17] allows us to obtain surface gradient information from intensity triples. The photometric, stereo lookup table is a table containing surface gradient doubles, (p,q), associated with intensity triples, and is indexed by these intensity triples. This photometric stereo lookup table yields important information about other potential imaged objects provided that they are made of the same material, have the same reflectance properties, and are imaged under the same imaging geometry. The imaging geometry refers to the camera, and the relative positions of the three light sources with respect to the camera. To build the photometric stereo lookup table, a real or synthetic image of a calibration sphere is used. The image is analysed and the associations between intensity triples and gradient doubles (p,q) are made. As in R. J. Woodham's work [17], we use the following equations to make the association between intensity triples and the gradient doubles (p,q). The sphere is represented as 21 x 2 + (Ay)2 + z2 = r2 Equation 4.1.1 where A takes the aspect ratio into account. The intensity triple at the point (x,y) in the image is associated with the gradient double (p,q), by the following equation. x A2 y p=— q= z z Equation 4.1.2 Hence if the imaging geometry or the material of interest changes, the photometric stereo lookup table must be recalibrated again in order to be useful. Photometric stereo [17] utilizes spectral multiplexing. Spectral multiplexing involves the use of multiple images taken from the same viewpoint, but under different conditions of illuminations. As in R. J. Woodham's work [17], we use three images in our spectral multiplexing, under three illumination conditions. For real images, the three images are acquired simultaneously in a colour image, with a Sony DXC-950 3 CCD 24-bit colour camera with a Sony VCL-714BXEA zoom lens. The red, green, and blue fields of the colour image form the three images. The red, green, and blue fields of the colour image can capture a scene under three different illumination conditions, if the three light sources illuminate the scene from different directions, and are sufficiently isolated in the red, green, and blue parts of the visible spectrum. This photometric stereo setup for real images used three Newport MP-1000 Moire (white-light) projectors with associated Nikon lenses and spectral filters. The filters separted the white light into the red, blue, and green parts of the spectrum. The filters were manufactured by Corion. R. Woodham's work [17] showed that there was negligible overlap in the wavelengths of 22 the red, blue, and green filtered light sources. The light sources were placed so as to illuminate the camera's area of interest from different directions. Spectral multiplexing was also artificially introduced into the synthetic images. This was accomplished by using different reflectance maps for the intensities in the red, green, and blue fields of the image. The different reflectance maps are specific to the different illumination directions of the red, green, and blue filtered light sources, and are derived from the calibration sphere used in the photometric stereo setup. This is done by inverting the association between and intensity triple (r,g,b), and its surface gradient (p,q). The red, green, and blue reflectance maps are tables which map the surface gradient (p,q) to the r, g, and b elements respectively of the intensity triple. Surface gradients (p,q) which are not explicitly in the table can be interpolated at run time from the recorded data. In order to calculate the partial derivates of the surface gradient (p,q) in a manner which is less sensitive noise we make use the derivatives of the three reflectance maps for the three different illumination conditions, which in our case is the red, green, and blue illumination conditions. The derivatives of the reflectance maps /?, Ru R2p, R2q, R, , and R, are calculated at run time, as needed, from the reflectance maps. The derivatives of the surface gradient, px, p pt, qx, qv, and q, are calculated as follows. The equations in Equation 3.1.6 in matrix form are 23 Equation 4.1.3 Considering the above as E = OMT Equation 4.1.4 we can solve for Q as O = EM(MTMyl Equation 4.1.5 The derivatives of the surface gradient, px, py, pt, qx, qy, and ql qt can be extracted from Q. Because tables used in gradient lookup are potentially very long, it became important to determine how many significant bits are in the images taken by our system. This was accomplished by looking at the least significant bits of the image and deciding whether there was a non-random pattern in the image. If there was a non-random pattern in the image, then it was likely that it reflected the underlying scene, and therefore contained significant information. We started by looking at the least significant bit, then the last two significant bits, etc. We found that the two least significant bits did not contain much information. Hence instead of indexing the tables with 8 bit intensity values, we use 6 bit intensity values. Hence instead of having 16,777,216 (2A8 * 2A8 * 2A8) entries in our tables, we have 262,144 (2A6 * 2 A6 * 2A6) entries in our tables.' 24 4.2 Local Regularization Given that all the elements besides the unknowns can be estimated in Equation 3.3.10, local regularization can to be used. This local regularization does not consider information from any other points in the image, as in global regularization, or any other non-local regularization. Again, let us represent Equation 3.3.10 as Ax = b Equa t ion 4.2.1 The regularization that is used is one which finds the x which minimizes /l 0|Px|| 2 + \ \Ax - b\ Equa t ion 4.2.2 where Xx > 0 and X7 > 0 are regularization parameters chosen beforehand. P represents a stabilization function, and is also chosen in advance. Two types of regularization were implemented and tested. Both of which were studied by D. Hsu's work [11] on multiple light source optical flow. The regularizations make use of QR factorization and Singular Value Decomposition (SVD). The standard OR factorization [13] of a real m by n matrix A is given by (R) A=0 ~{o) Equa t ion 4.2.3 25 where R is an n by n upper triangular matrix , 0 is an m by m orthogonal matrix, and m> n. \fA has n linearly independent rows, then R is non-singular. Often the factorization is written as which reduces to R\ Equat ion 4.2.4 A = 0,R Equa t ion 4.2.5 where 0, consists of the first n columns of 0, and 02 consists of the remaining columns. The Singular Value Decomposition [13] of a real m by n matrix A is given by A = UZYT Equat ion 4.2.6 Here, the U and V matrices are orthogonal, and _ is an m by n diagonal matrix with real diagonal elements ai. a, >a2 >.->cTmm<mjf) >0 Equa t ion 4.2.7 These real diagonal elements cr, are known as the singular values of ,4, and the first min(w,») columns of U and V are known as the left and right singular vectors of A. The mathematical relation between singular values and singular vectors is described below. 26 Av. =ajuj and A1w, =cr.v. Equa t ion 4.2.8 where ui is the rth left singular vector, and v. is the /th right singular vector.' Method 1 This method assumes that the measurements are exact. So Ax- b\ is weighted heavily. In fact if we set Xx—<x>, and A0 - 1 , the problem essentially becomes one of finding the x which.minimizes \\Px\\, subject to Ax = b . This regularization problem has a unique solution up to the nullspace of P. The algorithm used is as follows. First we use QR factorization on AT to get AT = 0 = f e . & ) . A is a 7 by 6 matrix, Qx is a 7 by 6 matrix, Q2 is a 7 by 1 matrix, and R is a 6 by 6 upper triangular matrix. The matrix Q which is equal to {Q]Q2) is a 7 by 7 orthogonal matrix. Now (R 0Tx = b So if we let we get Equat ion 4.2.9 z =0Tx Equat ion 4.2.10 27 Equa t ion 4.2.11 Now let z - where z, is a vector of six elements, and z2 is a scalar. Then RTz, =b Equat ion 4.2.12 can be solved uniquely, because R, in general, will have a rank of 6. Note that z2 can be any value and Equation 4.2.11 will still be satisfied. This means that z2 can be adjusted so as to minimize \\Px\\ Let h(z2)= Px Equat ion 4.2.13 and O = [Ot :02] where Ox is the first 6 columns of O and 02 is the last column of Q. Substituting Equation 4.2.10 into Equation 4.2.13 we get / 2 ( z 2 ) = | | J p o z | | 2 = | p ( o , z 1 + a z 2 ) | 2 Equat ion 4.2.14 28 We only consider stabilizing functions, P, such that h is convex and has only one minimum. This way the minimum can be found at the point where the derivative of h is zero. 0 = hXz2) = 2[P(Q]zt+Q2z2)]T(PQ2)-Equat ion 4.2.15 The solution is obtained by solving (QT2PrP02)z2 =-Q2rPTPO]z, Equat ion 4.2.16 If P is positive definite, the solution is unique. If P is only positive semi-definite then we find the solution with minimum l2 norm, by using Singular Value Decomposition (SVD) to solve Equation 4.2.16. Method 2 The assumption, that the measurements are exact, in general will not be correct. In this case A] should not be set to oo. Instead it should be set to a finite value. So here we set A() = A and/I, = I, because only the ratio between A(1 and A] is important. In our implementations of this second method, we experiment with three different values for A . The three different values are 0.00000001, 0.0001, and 0.1. Essentially here we are minimizing h{x) -JAx - /3||2 + AJPxf Equat ion 4.2.17 29 Again, we only consider stabilizing functions, P, such that h is convex and has only one minimum. The minimum occurs when the derivative of h(x) is equal to zero. 0 = h\x) = 2(Ax - b)T A + 2X{Px)T P Equat ion 4.2.18 0= Ar(Ax-b) + APTPx Equa t ion 4.2.19 Therefore (Ar A + APTP)x = ATb Equa t ion 4.2.20 Sox may be solved if (ATA + APTP)is invertible. When it is not invertible, the solution x may not be unique, and so we chose the solution with the smallest norm. So Equation 4.2.20 is solved using the standard Singular Value Decomposition algorithm (SVD). One of the problems with this method is that (AT A + XPT P) has a large condition number and inverting (A' A + / L P ' P) is ill conditioned. 30 Stabilizing Functions In general P can be taken to be any positive semi-definite matrix. In our applications P is a 7 by 7 diagonal matrix with diagonal entries dl,d2,d3,d4,ds,d6, and d7. So V 1=1 ..7 Equa t ion 4.2.21 i The relative weights of the di 's determines the contribution weight of the elements of the vector*. Because x corresponds to the 7 by 1 matrix in Equation 3.3.10 x - [u, v, a,, a2, a 3 , a, bj Equa t ion 4.2.22 The first two components of the solution, x, describe the translational component of the solution. The middle three components of the solution x, describe the change in the • • • . , / x ,dEi dE0 dE,. _ , intensity triple, because (a,, a2, a,) = (—-, — - , — - ) . The last two components or the dt dt dt solution x describe the rotational component of the solution, because a and b are equal to the total derivatives of the surface gradients p and q with respect to time. From here on the first two elements will be grouped and referred to as the translational component of the solution, the middle three components as the intensity change component of the solution, and the last two components as the rotational components of the solution. In our implementations we have implemented four different types of stabilizing functions. The first is a stabilizing function that minimizes the intensity change and the rotational component of the solution. The intensity change and rotational component of 31 the solution are minimized by placing infinite weight on the intensity change and rotational components with respect to the other components, which are weighted with zero weight. This in effect tries to account for observed motion mainly by translation, and the associated Px has the following form. 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 i 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 Equa t ion 4.2.23 The second stabilizing function minimizes the translational component of the solution. The translational component of the solution is minimized by placing infinite weight on the translational components with respect to the other components which are weighted with zero weight. This in effect tries to account for the observed motion mainly by rotation, and the associated P2 has the following form. 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Equat ion 4.2.24 32 The third stabilizing function minimizes the translational, intensity change, and rotational components of the solution. This is accomplished by weighting all components with a weight of one. This has the effect of selecting a solution to the optical flow problem with the minimum amount of translation and rotation, which are equally weighted. 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 p : J o 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 , 0 0 0 0 1 Equat ion 4.2.25 The fourth stabilizing function minimizes the intensity change components of the solution. This is accomplished by weighting the intensity change components with a weight of one. This has the effect of selecting a solution to the optical flow problem with the minimum amount of intensity change. It is expected that the fourth stabilizing function will yield similar results to the minimal intensity change and rotation solution, because the less rotation there is, the less the intensity change will be. 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 P* = 10 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Equat ion 4.2.26 In the four different stabilizing functions, the matrices Px, P2, P3, and P4 have different null spaces. The local regularization selects the minimum norm solution within these null spaces. The minimal intensity change and rotation solution is correct under translation under distant light sources, and yields a motion estimate which is similar to that obtained had we assumed the classic optical flow constraint equations. The minimal translation solution is correct for deforming surfaces with minimal translation but has significant surface normal changes. The minimal translation, intensity change, and rotation solution is more difficult to analyze. The assumptions on the observed motion for the minimal translation, and the minimal rotation case contradict each other. Hence the minimal translation, intensity change, and rotation solution is likely to be correct under a certain balance of rotation and translation in the scene. 34 Chapter Five 5 Synthetic Image Data Experimentation and data collection were accomplished using synthetic image data. The true optical flow for the synthetic data could be calculated and then compared with the estimated optical flow from our algorithm. The motions in the sequences described below include translation, rotation, and deformation. 5.1 Calibration sphere The synthetic calibration sphere was created using a program called "3D Studio Max 2.0" which is made by Kinetix. This is a three dimensional graphics program which allows one to place objects and light sources at arbitrary locations in a virtual environment, and to render this virtual scene using various shading models. Information on the various shading models can be obtained on the internet at the Uniform Resource Locator (URL), www.ktx.com. In this program, the calibration sphere was created by placing a sphere at the center of the scene, and placing red, green, and blue light sources behind the camera. The light sources were arranged so that they illuminated the sphere from three different directions. In this artificial environment the light sources were separated into the red, green, and blue parts of the spectrum with no overlaps. The calibration sphere was made of a dull white material, and a lambertian or diffuse, reflectance model was used. The placement of the light sources illuminated the sphere 35 from the bottom left, top, and bottom right, for the red, blue, and green light sources respectively. 36 Figure 5.1.1: The calibration sphere under the red illumination. 37 Figure 5.1.3: The ca l ib ra t ion sphere under the blue i l l umina t ion . 5.2 Translation of a Sphere This sequence shows the synthetic calibration sphere translating horizontally to the right. The translational speed is constant at 1 pixel per frame. The sphere is a doubly curved object and hence the multiple light source optical flow from Siegerist's work [14,15] can be used. 38 Figure 5.2.1: The first of 11 images in the translating calibration sphere sequence. The red, green, and blue illuminations have been superimposed. Figure 5.2.2: The last of 11 images in the translating calibration sphere sequence. The red, green, and blue illuminations have been superimposed. 39 5.3 Curving Sheet The curving sheet image sequence shows a white sheet gradually curving. The surface normal at the center of the sheet is pointed towards the viewer. At the start of the sequence the sheet is almost flat, but then gradually the edges start curving away from the viewer. In order to determine the true optical flow of the sequence, it is important to understand how the curving sheet deformation was parameterized and formulated. The curving sheet is parameterized by its length / in pixels, with 1=0 representing the vertical line at the center of the sheet. The sheet is also parameterized by its width w in pixels, with w=0 representing the horizontal edge at the top of the sheet. S=hngW2 F igu re 5.3.1: D i a g r a m depict ing the parameter izat ion of cu rv ing surface. The curving deformation on the sheet takes place only along the 1 axis. Since we are assuming an orthographic projection all horizontal lines represented by w=k1 where k is some value, will, continue to be projected to horizontal lines in the image sequence as the sheet deforms. Let x and y be the image coordinates where x increases towards the right from x=0 at the left edge, and_y increases downwards from _y=0 at the top edge. Also we will 40 let time be represented by t. If we model the curving of the sheet as being wrapped around a cylinder which decreases in radius, then the radius decreases with time. When the radius is at infinity the sheet is flat, but as time moves on and the radius decreases, and the sheet wrapped around the cylinder appears to curve away from the viewer. We also let the radius r=k-mt, where k and m are some constants. Then under orthographic projection the sheet will be represented as x = x„ + r cos - + - ' V 2 J y = yc + w z = r sin / K - + — Equa t ion 5.3.1 where (xc,yc) is the center of the image and the z-axis points towards the viewer along the optical center. The / and w parameters of the sheet represent actual lengths. Since we are modeling a sheet that is deforming but not stretching, at each time step the / and w parameters are constrained to be within certain ranges at all times / in the image sequence. - length! 2 <l < length 12 0 < w < width Equa t ion 5.3.2 The following proves that / and w represent actual lengths on the sheet surface. Because of orthographic projection and the fact that there is no deformation occurring along the w axis, it is obvious that w represents actual length on the curved sheet. To show that / 41 represents actual length, we will determine the length along the / axis of the deforming sheet. This is equivalent to finding the arc length of the curve in the x and z plane. From elementary calculus, we know that, if X and Y are parameterized by /, then arclength = jVx'2 + Y'2dl Equa t ion 5.3.3 So in our case we have arclength = j V x 1 + z 2 dl Equa t ion 5.3.4 arclength = j " I - sin n xX)2 f fl TT^ 2 + r > 2 + cosl V \r+ 2JJ dl Equa t ion 5.3.5 arclength = jldl = I Equa t ion 5.3.6 Hence it is clear that the parameter, /, is also a parameter which represents actual length in the horizontal direction on the surface of the curved sheet. In order to synthesize the images of the deforming sheet we have just parameterized, we need to know the surface gradients (p,q), of the curving sheet. 42 Equa t ion 5.3.8 as z is constant along y parameter. Now that we have the gradients, (p,q), at all points on the sheet, we can use the red, green, and blue reflectance maps estimated from the synthetic calibration sphere to convert the gradient (p,q) into red, green, and blue intensity values at all points on the sheet in the image. The image sequence is created by repeating this process for a certain range of time values. 43 wire mesh visualization of the curved sheet, at the end of its deformation. Units in pixels. 44 Figure 5.3.3: The first of 11 images in the curving sheet sequence. The red, green, and blue illuminations have been superimposed. Figure 5.3.4: The last of 11 images in the curving sheet sequence. The red, green, and blue illuminations have been superimposed. 4 5 In order to determine how accurate our estimated optical flow is, we determine the known motion field. The known motion field (w,v) for the curving image sequence is given below. dx dx dr it = — = = -m dt dr dt cos (1 >0 >0 — + — + sin . — 2j ^, 2) Equa t ion 5.3.9 v _dy _dy dry _ dt dr dt Equa t ion 5.3.10 In our actual implementation r deforms according to the following equation. r = 210-10/, where t is time. \ 46 5.4 Curved Translating Sheet This translation sequence consists of a non-deforming curved sheet which translates to the right. The curved sheet is parameterized and created in a similar manner to the sequence above, except that it does not deform. The speed o f the horizontal motion was set to a constant value of 1 pixel per frame. The curved translating sheet is exactly the same as the curved sheet at the last frame in the curving sheet sequence which is shown in Figure 5.3.4, and has the same wire mesh visualization as in Figure 5.3.2. Figure 5.4.1: The first of 11 images in the curved t rans la t ing sheet sequence. The red , green, and blue i l luminat ions have been super imposed. 47 Figure 5.4.2: The last of 11 images in the curved translating sheet sequence. The red, green, and blue illuminations have been superimposed. 5.5 Rotation of a Surface With Negative Gaussian Curvature This sequence shows a saddle shaped object being rotated around the center of the image at a rate of 0.01 radians per frame. A l l points on the surface have negative Gaussian curvature. Along the length of the object, the edges curve away from the viewer, while along the width of the object, the edges curve toward the viewer. This can be parameterized in a similar manner to the curved sheet discussed above, and curvature is simply introduced in a perpendicular direction to that which is already present. The effects of inter-reflection which would occur in real life for this saddle shaped object were not taken into account. 48 Figure 5.5.1: A wire mesh visualization of the surface with negative Gaussian curvature. Units are in pixels. Figure 5.5.2: The first of 11 images in the sequence of a rotating surface with negative Gaussian curvature. The red, green, and blue illuminations have been superimposed. 49 Figure 5.5.3: The last of 11 images in the sequence of a rotating surface with negative Gaussian curvature. The red, green, and blue illuminations have been superimposed. 5.6 Translation of a Surface With Negative Gaussian Curvature This sequence shows the same saddle shaped object as that depicted in Figure 5.5.1 being translated at a rate of 1 pixel per frame. The effects of inter-reflection, which would occur in real life for this saddle shaped object were not taken into account. 50 Figure 5.6.1: The first of 11 images in the sequence of a translating surface with negative Gaussian curvature. The red, green, and blue illuminations have been superimposed. 51 Figure 5.6.2: The last of 11 images in the sequence of a t rans la t ing surface w i th negative Gauss ian cur \ ature. The red , green, and blue i l luminat ions have been super imposed. 5.7 Rotation and Translation of a Surface With Negative Gaussian Curvature This sequence shows the same saddle shaped object as that depicted in Figure 5.5.1 being translated at a rate of 0.5 pixels per frame and also being rotated at a rate 0.01 radians per frame. The effects of inter-reflection, which would occur in real life for this saddle shaped object were not taken into account. 52 Figure 5.7.1: The first of 11 images in the sequence of a rotating and translating surface with negative Gaussian curvature. The red, green, and blue illuminations have been superimposed. 5.8 Rotation of a Surface With Positive Gaussian Curvature This sequence shows a surface with positive Gaussian curvature being rotated around the center of the image at a rate of 0.01 radians per frame. Along all the edges of this rectangular object, the surface curves away from the viewer. This can be parameterized in a similar manner to the curved sheet discussed above, and curvature is simply introduced in a perpendicular direction to that which is already present. Figure 5.8.1: A wire mesh visualization of the surface with positive Gaussian curvature. Units are in pixels. 54 Figure 5.8.2: The first of 11 images in the sequence of a rotating surface with positive Gaussian curvature. The red, green, and blue illuminations have been superimposed. 55 Figure 5.8.3: The last of 11 images in the sequence of a rotat ing surface w i th posit ive Gauss ian curva ture . T h e red , green, and blue i l luminat ions have been super imposed. 5.9 Translation of a Surface With Positive Gaussian Curvature This sequence uses the same surface with positive Gaussian curvature as that depicted in Figure 5.8.1. In this sequence, it is being translated at a rate of 1 pixel per frame. 56 Figure 5.9.1: The first of 11 images in the sequence of a translating surface with positive Gaussian curvature. The red, green, and blue illuminations have been superimposed. 57 Figure 5.9.2: The last of 11 images in the sequence of a translating surface with positive Gaussian curvature. The red, green, and blue illuminations have been superimposed. 5.10 Rotation and Translation of a Surface With Positive Gaussian Curvature This sequence uses the same surface with positive Gaussian curvature as that depicted in Figure 5.8.1. It has positive Gaussian curvature at all points and is being translated at a rate of 0.5 pixels per frame and is also being rotated at a rate 0.01 radians per frame. 58 Figure 5.10.1: The first of 11 images in the sequence of a rotating and translating surface with positive Gaussian curvature. The red, green, and blue illuminations have been superimposed. Figure 5.10.2: The last of 11 images in the sequence of a rotating and translating surface with positive Gaussian curvature. The red, green, and blue illuminations have been superimposed. 59 Chapter S i x 6 Results 6.1 Optical Flow Estimation Techniques Tested Two types of optical flow estimation techniques were implemented and tested. The two types of optical flow will be identified as type 1 and type 2 optical flows. Type 1 is the multiple light source optical flow and solves the optical flow problem by using Equation 2.1.4. Type 2 solves for the optical flow by using Equation 3.3.10. The type 2 optical flow estimation has sub-types identified as A, B, B', and B" according to the type of regularization used. Type A uses method 1 from the discussion on regularization, while type B uses method 2 with X equal to 0.00000001. Type B' uses method 2 with X equal to 0.0001 and Type B" uses method 2 with X equal to 0.1. Also all the sub-types of the type 2 optical flow estimation will be further sub-classified based on the stabilizing function used. "Min. A" will refer to the minimum intensity change and rotation stabilizing function, "Min. B" will refer to the minimum translation stabilizing function, "Min. C" will refer to the minimum intensity change, translation and rotation stabilizing function, and "Min. D" wilkrefer to the minimum intensity change stabilizing function. 60 Algorithm Equations Used Regularizat ion Method Lambda (A) Minimize Translation Minimize Intensity Change Minimize Rotation Type 1 Equation 2.1.4 N/A N/A N/A N/A Type 2A - Equation 1 N/A • No Yes Yes Min. A 3.3.10 Type 2A - Equation 1 N/A Yes No No Min. B 3.3.10 Type 2A - Equation 1 N/A Yes Yes Yes Min. C 3.3.10 Type 2A - Equation 1 N/A No Yes No Min. D 3.3.10 Type 2B - Equation • 2 0.00000001 No Yes Yes Min. A 3.3.10 Type 2B - Equation 2 0.00000001 Yes No No Min. B 3.3.10 Type 2B - Equation 2 0.00000001 Yes Yes Yes Min. C 3.3.10 Type 2B - Equation 2 0.00000001 No Yes No Min. D 3.3.10 61 Type 2B' -Min. A Equation 3.3.10 2 0.0001 No Yes Yes Type 2B' -Min. B Equation 3.3.10 2 0.0001 Yes No No Type 2B' -Min. C Equation 3.3.10 2. 0.0001 Yes Yes Yes Type 2B' -Min. D Equation 3.3.10 2 0.0001 No Yes No Type 2B" -Min. A Equation 3.3.10 2 0.1 No Yes Yes Type 2B" -Min. B Equation 3.3.10 2 0.1 Yes No No Type 2B" -Min. C Equation 3.3.10 2 0.1 Yes Yes Yes Type 2B" -Min. D Equation 3.3.10 2 0.1. No Yes No Tab le 6.1.1: The different opt ical f low est imation a lgor i thms implemented and tested. 62 6.2 Optical Flow Estimation Quality Measures All the variations on the optical flow estimation techniques described above were tested using the synthetic image sequences described in chapter 5. For each optical flow estimation a number of statistical averages were calculated. At a given pixel (x,y), we denote the it and v components of the estimated optical flow as ne(x,y) , and ve(x,y) . The average of ue(x,y) is denoted as ue and the average of ve(x,y) is denoted as ve . The averages are calculated by summing over all the pixels corresponding to the object's surface and dividing by the number of pixels considered. The pixel is considered if it is illuminated by all three light sources and has a neighborhood of pixels which are also illuminated by all three sources. This neighborhood condition considers pixels which are far enough away from the edge of the object that the edge effects from image smoothing do not contribute to the statistics. The neighborhood is a 5 by 5 grid with the pixel of interest at the center. N N N N N N N N N N N N P N N N N N N N N N N N N Figure 6.2.1 The 5 by 5 neighborhood of the pixel P. A l l cells label led " N " are in the 5 by 5 ne ighborhood. 63 Here all the cells labeled with "N" are part of the neighborhood of the pixel P. A 5 by 5 neighborhood was used because a 5 by 5 Gaussian smoothing kernel was used in our implementation to smooth the images. (jc,y)3 surface \{(x,y)\{.x,y) ^ surface}\ Equa t ion 6.2.1 ^ (x,y)5surfhce \{(x,y)\(x,y)^surface}\ Equa t ion 6.2.2 where "| |" is the "size o f operator which returns the number of elements in a set. The standard deviation of u and v is also calculated and will be denoted as cr and cr ue e respectively. Standard deviations in our analysis are always calculated according to the following formula. _](T(x,y)-j)2 ^_ (x,y)Gsurface \ y)\(x, y) G surface^ -1 Equa t ion 6.2.3 where T is some quantity at pixel, (x,y), like ue or \>e. We will denote the u and v components of the known motion field vector at a pixel (x,y), as uk (x,y) and vk (x,y) . 64 The averages and standard deviations are also calculated for the known motion field vectors. They will be denoted as uk , vk , c and a respectively. If — l ^ J . J, , _ \{(x,y%x,y) 3 surface^ Equa t ion 6.2.4 ^ (x,y)5surface \{(x,y)\(x,y)3surface}\ Equa t ion 6.2.5 These it and v component averages of the estimated optical flow make sense for translating objects, as all the estimated optical flow vectors should be constant if the estimation is completely accurate. If the estimation is not completely accurate, the averaging tends to get rid of some of the noise that is present in the estimation. A number of error measures will be used to gauge the quality of the estimated optical flow. Our formulation for the estimation of optical flow is under-constrained and ambiguous. We are simply selecting a solution out of all the ambiguous cases. Hence part of the error is due to the ambiguous nature of the problem. Nonetheless we will still refer to these comparison measures as error measures, and the difference between the known motion field vector and estimated optical flow vectors at a point (x,y) will be referred to as the error vector. The first set of error measures involves the error vectors between the known motion field vectors and the estimated optical flow vectors. Let us denote the error vector as e and define it as follows. 65 e(x,y) = (ue(x,y),ve(x,y))-(uk(x,y),vk(x,y)) Equa t ion 6.2.6 where ue(x,y) and ve(x,y) are the u and v components of the estimated optical flow vector at the pixel, (x,y), and uk (x,y) and vk (x,y) are the a and v components of the known motion field vector at the pixel, (x,y). The average norm of the error vectors will be denoted as . e = ZIH*^ )| (x,y)Gsurface \{(x,y)\(x,y)<=surface}\ Equa t ion 6.2.7 The standard deviation of the norms of the error vectors will be denoted as cr» •. H The u component of e(x,y)Wi\\ be denoted as eu(x,y) and the v component of e(x,y) will be denoted as ev(x,y). The average of the magnitude of eu(x,y) over all the pixels corresponding to the object's surface will be denoted as ||eu|| and the average magnitude of ev (x,y) over all.the pixels corresponding to the object's surface will be denoted as \\evj. The average of eu (x,y) over all the pixels corresponding to the object's surface will be denoted as eu and the average of ev(x,y) over all the pixels corresponding to the object's surface will be denoted as ev 66 We.. = ZIK(x^)ll (x,y)£surface e., = \{{x,y%x,y)& surface^ Equa t ion 6.2.8 ZIK(^^)|| (x,y)£ surf ace \{(x,y^(x,y)esurface}\ Equa t ion 6.2.9 ]_eu(x,y) (xty)G surface \{(x,y)\(x,y)Gsurface}\ Equat ion 6.2.10 _ __ev(x,y) ^ (x,y)Gsurface \{(x,y)\(x,y)Gsurface}\ Equat ion 6.2.11 The standard deviations of eu(x,y), ev(x,y), |e„ >0|| and | | e v ( x , w i l l be denoted as °%> °"«,' % u \ a n d a i K i r e s P e c t i v e , y -At a given pixel (x,y), the norm of the error, e, divided by the norm of the known motion field vector will be denoted as r. This quantity expresses the amount of error per known amount of motion. 67 Equat ion 6.2.12 One of the most informative error measures is the average of r over all pixels that are a part of the object's surface being imaged and will be denoted as r . This measure r expresses the amount of error per known amount of motion on average over all the pixels corresponding to the object's surface. ^ (x,y)&sutface \{(x,y)\(x,y)esurface}\ Equa t ion 6.2.13 The above measure r encodes both the errors in magnitude and direction of the estimated optical flow vectors. In order to measure only the directional error between the estimated optical flow vectors and the known motion field vectors we will introduce one more measure to gage the quality of the directions of the estimated optical flow vectors. At a given pixel (x,y), let Z denote the angle between estimated optical flow vector and the known motion field vector. Z(x, y) = cos (ue (x, y), ve (x,y))» (uk (x, y), vk (x, y)) (x, y\ v, {x,y))guk (x,y), vk (x,y)\ Equat ion 6.2.14 Here we are only interested in the positive angles returned by the inverse cosine function, because our directional measure will be the average magnitude of Z(x,y) over all pixels 68 corresponding the object's surface and will be denoted as Z . So we assume that the inverse cosine function always returns values in the range of 0 to/r. Z: (x.y)Gsurface \{(x,y)\(x,y)esurface}\ . Equat ion 6.2.15 The standard deviation of Z(x,y) over all pixels corresponding the object's surface will be denoted as . / 69 6.3 Translation of the Calibration Sphere Known Motion Field Figure 6.3.1: The known mot ion field f o r the t rans la t ing ca l ibrat ion sphere. Vec to rs are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation 1.000000 0.000000 vk 0.000000 0.000000 1.000000 0.000000 IM 0.000000 0.000000 i( W A-> V A- l 1.000000 0.000000 Tab le 6.3.1: The averages and s tandard deviat ions of measures associated wi th the known mot ion field. The units f o r al l measures are pixels. 70 Type 1 Optical Flow Estimation Figure 6.3.2: The type 1 opt ical f low est imat ion fo r the t rans la t ing ca l ib ra t ion sphere. Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation l'e 1.02011 0.155768 V e - 0 . 0 0 5 8 9 8 0.323023 1 0 0 0 e u 0.02011 0.155768 e V - 0 . 0 0 5 8 9 8 0.323023 e 0.177143 0.312516 e „ 0.098314 0.122484 ev 0.122239 0.299058 r 0.177143 0.312516 Z 0.107476 0.142329 Tab le 6.3.2: The averages and s tandard deviat ions of measures associated wi th the type 1 opt ical f low est imat ion. The measure r is unitless. The measure Z is in radians. A l l other measures are in pixels. 71 Type 2A Min. A Optical Flow Estimation Figure 6.3.3: The type 2 A M i n . A opt ical f low estimation fo r the t rans la t ing ca l ib ra t ion sphere. Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation U e 0.965659 0.261974 V e -0.001434 0.145695 Uk 1 0 Vk 0 0 e u -0.034341 0.261974 ev -0.001434 0.145695 e 0.198312 0.227399 eu 0.141533 0.223109 0.095068 0.110413 r 0.198312 0.227399 Z 0.097455 0.099299 Tab le 6.3.3: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . A opt ical flow est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 72 Type 2A Min. B Optical Flow Estimation r r / r f S S \ \ •• *• * * S S / S ' s. \. v > v ^ * J* s> % s S. W V - '* -*• ^ X \ v v ^ ^ ^* ^+ —». *-> \ -'' -^ -^ ^ w _k. S .^ ^ ^ -* \ \ . ^ ^ -v ^-v ^ ^ > »- \ \ ^ ^ ^ s V V X ^ s ^ . s s / / r " v, S N N /• j » ... * ' ^ N, V N Figure 6.3.4: The type 2 A M i n . B opt ical f low est imation fo r the t rans lat ing ca l ib ra t ion sphere. Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation Ue 0.484021 0.373708 V e 0.005511 0.355688 l'k 1 0 Vk 0 0 e u -0.515979 0.373708 g v 0.005511 0.355688 e 0.663219 0.30426 e» 0.526835 0.358241 ev 0.308737 0.176704 r 0.663219 0.30426 Z 0.784984 0.457188 Tab le 6.3.4: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . B opt ical f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l o ther measures are in pixels. 73 Type 2A Min. C Optical Flow Estimation Figure 6.3.5: The type 2 A M i n . C opt ical f low est imation f o r the t rans lat ing ca l ib ra t ion sphere. Vec to rs are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation 0.682772 0.284131 V e 0.003327 0.222653 "A- 1 0 VA- • 0 0 -0.317228 0.284131 e V 0.003327 0.222653 , e 0.417219 0.238487 e» 0.333361 0.265018 ev 0.185223 0.1236 r 0.417219 0.238487 Z 0.285751 0.18296 Tab le 6.3.5: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . C opt ical flow est imat ion. The measure r is unit less. The measure Z is in radians. A l l o ther measures are in pixels. 74 T y p e 2A M i n . D O p t i c a l F l o w E s t i m a t i o n F igu re 6.3.6: The type 2 A M i n . D opt ica l flow est imation fo r the t rans la t ing ca l ib ra t ion sphere. Vectors are magni f ied 10 t imes and sampled every 20 pixels. M e a s u r e A v e r a g e S t a n d a r d D e v i a t i o n 0.965658 0.261973 V e -0.001434 0.145695 uk 1 0 0 0 -0.034342 0.261973 -0.001434 0.145695 e 0.198312 0.227399 e« 0.141533 0.223109 0.095068 0.110413 r 0.198312 0.227399 Z 0.097455 0.0993 Tab le 6.3.6: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . D opt ical f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 75 Type 2B, 2B\ and 2B" Min. A Optical Flow Estimation F igu re 6.3.7: The type 2B M i n . A opt ical f low est imation fo r the t rans la t ing ca l ib ra t ion sphere. Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2B Min. A 2B' Min. A 2B" Min. A Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation ». 1.021277 0.15875 1.020507 0.158327 1.020034 0.156021 V e -0.005378 0.32523 -0.005464 0.324829 -0.005825 0.323202 Uk 1 0 1 0 1 0 Vk 0 0 0 ' 0 0 0 e u 0.021277 0.15875 0.020507 0.158327 0.020034 0.156021 ev -0.005378 0.32523 -0.005464 0.324829 -0.005825 0.323202 e 0.17975 0.314876 0.178938 0.314662 0.177393 0.31268 *» 0.099899 0.125198 0.099435 0.124901 0.098472 0.122666 ev 0.12404 0.300694 0.123464 0.300499 0.12242 0.299177 r 0.17975 0.314876 0.178938 0.314662 0.177393 0.31268 Z 0.109054 0.145002 0.108604 0.144731 0.107693 0.142775 Tab le 6.3.7: The averages and s tandard deviat ions of measures associated wi th the type 2B, 2B', and 2B" M i n . A opt ical f low est imation. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 76 Type 2B, 2B', and 2B" Min. B Optical Flow Estimation r .. . f- / V V > r ? S S \ •, S . v -A / S S .. \ >. * v t f S ^> ' ~» ~ . N N > —' ~T ~" / — ~* " • ^. - * > K *v ~v — ^ ^ ^ ^ S t r , v \ S. ~v ~> ^ ;• S -> ' ' * .-. V x x 1 - % S S / ' .-. - \ S S ">> v N ' F igu re 6.3.8: The type 2B M i n . B opt ica l f low est imation fo r the t rans la t ing ca l ib ra t ion sphere. Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2B Min. B 2B'M [in.B 2B" Min. B Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation V e 0 . 4 8 3 4 2 1 0 . 3 7 3 3 4 5 0 . 3 0 7 2 7 4 0 . 3 2 3 4 7 6 0 . 0 4 4 6 3 7 0 . 1 3 4 7 2 8 V e 0 . 0 0 5 5 0 7 0 . 3 5 5 1 8 8 0 . 0 0 3 9 4 9 0 . 2 5 6 8 9 8 0 . 0 0 1 2 4 6 0 . 0 7 9 4 6 8 "k 1 0 1 0 1 0 vk 0 0 0 0 0 0 e„ - 0 . 5 1 6 5 7 9 0 . 3 7 3 3 4 5 - 0 . 6 9 2 7 2 6 0 . 3 2 3 4 7 6 - 0 . 9 5 5 3 6 3 0 . 1 3 4 7 2 8 K 0 . 0 0 5 5 0 7 0 . 3 5 5 1 8 8 0 . 0 0 3 9 4 9 0 . 2 5 6 8 9 8 0 . 0 0 1 2 4 6 0 . 0 7 9 4 6 8 e 0 . 6 6 3 2 2 3 0 . 3 0 4 2 3 8 0 . 7 6 2 0 4 5 0 . 2 6 4 2 0 3 0 . 9 6 1 3 4 2 0 . 1 1 4 0 5 e « 0 . 5 2 7 2 7 3 0 . 3 5 8 0 8 2 0 . 6 9 5 7 7 1 0 . 3 1 6 8 7 4 0 . 9 5 5 5 6 6 0 . 1 3 3 2 7 8 0 . 3 0 8 2 5 9 0 . 1 7 6 5 3 3 0 . 1 9 4 6 5 1 0 . 1 6 7 6 9 7 0 . 0 2 7 2 1 6 0 . 0 7 4 6 7 2 r 0 . 6 6 3 2 2 3 0 . 3 0 4 2 3 8 0 . 7 6 2 0 4 5 0 . 2 6 4 2 0 3 0 . 9 6 1 3 4 2 0 . 1 1 4 0 5 Z 0 . 8 2 6 0 1 9 0 . 5 4 6 0 0 9 0 . 8 2 8 9 1 2 0 . 5 4 9 5 4 6 0 . 8 2 7 2 3 7 0 . 5 4 7 4 0 5 Tab le 6.3.8: The averages and s tandard deviat ions of measures associated w i th the type 2B, 2B', and 2B" M i n . B opt ica l f low est imat ion. The measure r is unitless. The measure Z is in radians. A l l 1 o ther measures are in pixels. 77 Type 2B, 2 B \ and 2B" Min. C Optical Flow Estimation > * > -> . - - * ? — ' ^ ~r ~r ~* ^ -> - "V — ^ ^ ^ ^ S „ J - -> •» •V ~v ~*> ^ *> ^ -» * \ "> -v * * «• <• •v *r -> N Figure 6.3.9: The type 2B M i n . C opt ical flow est imation fo r the t rans la t ing ca l ib ra t ion sphere. Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2B Min. C 2B' Min. C 2B" Min. C Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation "e 0.715898 0.246847 0.635354 0.214035 0.574245 0.194891 V e 0.006586 0.234884 0.004522 0.201468 -0.000261 0.1951 »A 1 0 1 0 1 0 Vk 0 0 0 0 0 0 e „ -0.284102 0.246847 -0.364646 0.214035 -0.425755 0.194891 e V 0.006586 0.234884 0.004522 0.201468 -0.000261 0.1951 e 0.393332 0.205306 0.430975 0.183432 0.47691 0.172831 e „ 0^30395 0.221951 0.37373 0.197746 0.430625 0.183882 0.197835 0.126784 0.16277 0.118808 0.158475 0.113795 r 0.393332 0.205306 0.430975 0.183432 0.47691 0.172831 Z 0.291177 0.190262 0.263782 0.186978 0.281585 0.194536 Tab le 6.3.9: The averages and s tandard deviat ions of measures associated wi th the type 2B, 2B', and 2B" M i n . C opt ical flow est imat ion. The measure r is,unit less. The measure Z is in radians. A l l other measures arc in pixels. 78 Type 2B, 2B\ and 2B" Min. D Optical Flow Estimation —»-» -> ^ F igu re 6.3.10: The type 2B M i n . D opt ica l f low est imat ion fo r the t rans la t ing ca l ibra t ion sphere. Vec to rs are magni f ied 10 t imes and sampled every 20 pixels. 2B Min. D 2B' Min. D 2B" Min. D Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation , ue 1.021276 0.15875 1.020506 0.158326 1.020033 0.156019 Ve -0.005377 0.32523 -0.005464 0.324829 -0.005826 0.323202 Uk 1 0 1 0 1 0 Vk 0 0 0 0 0 0 e « 0.021276 0.15875 0.020506 0.158326 0.020033 0.156019 ev -0.005377 0.32523 -0.005464 0.324829 -0.005826 0.323202 e 0.17975 0.314876 0.178938 0.314662 0.177392 0.312679 e u 0.099899 0.125197 0.099435 0.124901 0.098471 0.122665 0.12404 0.300694 0.123464 0.300499 0.122419 0.299176 r 0.17975 0.314876 0.178938 0.314662 0.177392 0.312679 Z 0.109054 0.145002 0.108604 0.144732 0.107692 0.142773 Tab le 6.3.10: The averages and s tandard deviat ions of measures associated w i th the type 2B, 2B', and 2B" M i n . D opt ica l flow est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 79 6.4 Curving Sheet The type 1 multiple light source optical flow method cannot estimate the flow properly for developable surfaces, so only the type 2 estimation technique for optical flow wil l be shown. Known Motion Field Figure 6.4.1: The known mot ion f ie ld fo r the cu rv ing sheet. Vectors are magni f ied 35 t imes and sampled every 20 pixels. Measure Average Standard Deviation uk 0 0 .134642 : vk 0 0 "k 0 .088119 0 .101799 Ik 0 0 0 .088119 0 .101799 Tab le 6.4.1: The averages and s tandard deviat ions of measures associated wi th the known mot ion f ie ld . The measure r is unit less. A l l measures are in pixels. 80 Type 2A Min. A Optical Flow Estimation 4 '" <• - -•• + ' 4 •i I '" -•• v + * 4 4-I " - + ' 4 4-4 '" -•• -' - + ' 4 I ' - + * 4 I '" ~ V A + * 4 •I F igu re 6.4.2: The type 2 A M i n . A opt ical flow estimation for the cu rv i ng sheet. Vectors are magni f ied 0.000000000000004 t imes and sampled every 20 pixels. Measure Average Standard Deviation - 0 .162539 3 .240363 ' V 3 .45929E+14 2 . 8 2 9 3 3 E + 1 5 0 0 .134642 vk 0 0 -0 .162539 3 .1258 e • V 3 .45929E+14 2 . 8 2 9 3 3 E + 1 5 e 1.45225E+15 2 . 4 5 2 6 6 E + 1 5 2 .63142 1.694745 1 .45225E+15 2 . 4 5 2 6 6 E + 1 5 r 1.00325E+28 1 .20386E+29 Z 1.51226 0 .289457 Tab le 6.4.2: The averages and s tandard deviat ions of measures associated w i th the type 2 A M i n . A opt ica l f low est imat ion. The measure r is unitless. The measure Z is in rad ians. A l l other measures arc in pixels. 8 1 Type 2A Min. B Optical Flow Estimation Figure 6.4.3: The type 2 A M i n . B opt ical flow est imation fo r the cu rv ing sheet Vectors are magni f ied 2 t imes and sampled every 20 pixels. Measure Average Standard Deviation -0 .14445 3 .205273 Ve 0 0 .287939 l'k 0 0 .134642 Vk 0 0 • e u -0 .14445 3 .090808 . ev 0 0 .287939 e 2 .613752 1.680674 2 .591166 1.690954 0 .048586 0.28381 r 2 .01542E+12 2.36781 E+13 Z 0 .053333 0 .30824 Tab le 6.4.3: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . B opt ical f low est imat ion. The measure r is unitless. The measure Z is in radians. A l l o ther measures are in pixels. 82 T y p e 2A M i n . C O p t i c a l F l o w E s t i m a t i o n - * • - » * > < * • « - < — -«— -J- -> * > < + +- 4--*— F igu re 6 .4.4: The type 2a M i n . C opt ical flow estimation for the cu rv ing sheet. Vectors are magni f ied 2 t imes and sampled every 20 pixels. M e a s u r e A v e r a g e S t a n d a r d D e v i a t i o n -0.145185 3.218087 V e 0 0.202182 l'k 0 0.134642 Vk 0 0 e u -0.145185 3.103231 e V 0 0.202182 e 2.623962 1.675204 2.617133 1.673686 0.029572 0.200007 r 2.0171E+12 2.36968E+13 Z 0.03445 0.272545 Tab le 6 .4.4: The averages and standard deviat ions of measures associated wi th the type 2a M i n . C opt ical flow est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 83 Type 2 A Min. D Optical Flow Estimation 1 '•• -• * - + ' 4 + . 4 '• - + ' 4 I ^ .J, 1 4 I " - A + ' 4 + I ' - - + I - + * 4 4 . F igu re 6.4.5: The type 2 A M i n . D opt ical flow est imation fo r the cu rv ing sheet Vectors are magni f ied 0.000000000000004 t imes and sampled every 20 pixels. Measure Average Standard Deviation -0 .162539 3 .240363 ve 3 .45929E+14 2 . 8 2 9 3 3 E + 1 5 uk 0 0 .134642 vk 0 0 -0 .162539 3 .1258 3 .45929E+14 2 .82933E+15 e 1.45225E+15 2 .45266E+15 eu 2 .63142 1.694745 ev '•: 1 .45225E+15 2 .45266E+15 r 1.00325E+28 1 .20386E+29 Z 1.51226 0 .289457 Tab le 6.4.5: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . D opt ical f low est imat ion. The measure r is unitless. The measure Z is in rad ians. A l l other measures are in pixels. 84 Type 2 B , 2 B \ and 2 B " Min. A Optical Flow Estimation Figure 6.4.6: The type 2B M i n . A opt ical f low est imation fo r the cu rv ing sheet Vectors are magni f ied 2 t imes and sampled every 20 pixels. 2 B Min. A 2 B ' 1 V J in. A 2 B " Min. A Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation - 0 .142662 3 .257643 -0 .160225 3 .247889 -0 .162388 3 .250522 ve 0 1.320262 0 1.150765 0 • 1.078429 . "k 0 0 .134642 0 0 .134642 0 0 .134642 0 0 0 0 0 0 ; e u - 0 .142662 3 .142417 -0 .160225 3 .132997 -0 .162388 3 .135713 e V 0 1.320262 0 1.150765 0 1.078429 e 2.678971 2 .112064 2 .656735 2 .026534 2 .655547 1.992391 e„ 2 .652187 1.691308 2 .632291 1.706421 2 .632132 1.711852 0 .040767 1.319633 0.038561 1.150119 0 .037602 1.077773 r 1.69259E+12 1 .98869E+13 1 .55237E+12 1 .82432E+13 1 .55235E+12 1 .82429E+13 Z 0 .028017 0 .269778 0 .028054 0 .269747 0 .028064 0 .269748 Tab le 6.4.6: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . A opt ical f low est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 85 Type 2B, 2B', and 2B" Min. B Optical Flow Estimation - » - - » * > < * *- *— ->— - » • - * * > •<— -f -r * f t t- t. t— -i— - » - - » * > < * * . * _ F igu re 6.4.7: The type 2B M i n . B opt ical flow est imat ion fo r the cu rv ing sheet Vectors are magni f ied 2 t imes and sampled every 20 pixels. 2B Min. B 2B'M [in. B 2B" Min. B Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation Ue - 0 .139623 3 .201663 -0 .014633 0 .73972 -0 .005184 0 .04495 V e 0 0 .287783 0 0 .11293 0 0 .000502 uk 0 0 .134642 0 0 .134642 0 0 .134642 Vk 0 0 0 0 0 0 e « - 0 .139623 3 .087124 -0 .014633 0 .612937 -0 .005184 0 .127064 ev 0 0 .287783 0 0 .11293 0 0 .000502 e 2.6024 1.691053 0 .360718 0 .50846 0 .083905 0 .09556 e„ 2 .579842 1.701104 0.352401 0 .501707 0 .083904 0 .09556 0 .048487 0 .283668 0 .01442 0 .112005 0 .000039 0.000501 r 7.4112E+11 8 .70654E+12 117229792 .8 1377282367 117237 .5699 1377362 .236 Z 0 .053333 0 .30824 0 .053333 0 .30824 0 .053333 0 .30824 Tab le 6.4.7: The averages and s tandard deviat ions of measures associated wi th the type 2B, 2B', and 2B" M i n . B opt ical flow est imat ion. The measure r is unitless. The measure Z is in radians. A l l other measures are in pixels. 86 Type 2B, 2B\ and 2B" Min. C Optical Flow Estimation -f -t V <. * * • 1 - • * — —t- * > <. * -tr- + — — - » • - » • * . ^ 4 * * - I T - • •<— F igu re 6.4.8: The type 2 B M i n . C opt ical f low est imation fo r the cu rv i ng sheet Vectors are magni f ied 2 times and sampled every 20 pixels. 2B Min. C 2 B ' M in. C 2B" Min. C Measure Average Standard Deviation Average N Standard Deviation Average Standard Deviation Ue -0 .142102 3 .214436 -0 .093626 2.195431 -0 .092035 2 .055316 Ve 0 0.202111 0 0 .172778 0 0 .169902 Uk 0 0 .134642 0 0 .134642 0 0 .134642 . Vk 0 0 0 0 0 0 Su -0 .142102 3 .099506 -0 .093626 2 .077482 -0 .092035 1.937986 ev 0 0.202111 0 0 .172778 0 0 .169902 e 2 .614689 1.68252 1.773133 1.100143 1.668613 1.004318 2 .60786 1.68098 1,766003 1.098043 1.661131 1.002349 0 .02954 0.199941 0 .024846 0 .170982 0 .024666 0 .168102 r 1.11357E+12 1 .30834E+13 8 .89828E+11 1 .04562E+13 8 .53426E+11 1 .00285E+13 A 0.034504 0 .272319 0 .036308 0 .275083 0 .037047 0 .276253 Tab le 6.4.8: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . C opt ical f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 87 Type 2B, 2B', and 2B" Min. D Optical Flow Estimation < t t - 1— •* * f * * *• Figure 6.4.9: The type 2 B M i n . D opt ical f low est imation for the cu rv ing sheet Vectors are magni f ied 2 t imes and sampled every 20 pixels. 2B Min. D 2B'1VJ [in. D 2B"M [in.D Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation V - 0 .142662 3 .257643 -0 .160225 3.247891 -0 .162389 3 .250524 V e • 0 1.320274 0 1.150754 0 1.078398 Uk 0 0 .134642 0 0 .134642 0 0 .134642 Vk 0 0 0 0 0 0 e „ . -0 .142662 3 .142418 -0 .160225 3 .132999 -0 .162389 3 .135716 e V 0 1.320274 0 1.150754 0 1.078398 e 2 .678972 2.112071 2 .656735 2 .02653 2 .655548 1.992378 eu 2 .652187 1.691308 2 .632292 1.706424 2 .632133 1 711855 0 .040767 1.319645 0.038561 1.150108 0 .037602 1:077742 R 1.69259E+12 1 .98869E+13 1 .55237E+12 1 .82432E+13 1 .55235E+12 1 .8243E+13. Z 0 .028017 0 .269778 0 .028054 0 .269747 0 .028064 0 .269748 Tab le 6.4.9: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . D opt ical f low est imation. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 88 6.5 Curved Translating Sheet The type 1 multiple light source optical flow method cannot determine the flow for developable surfaces, so only the type 2 optical flow estimation technique results will be shown. Known Motion Field Figure 6.5.1: The known mot ion f ie ld fo r the t rans la t ing curved sheet. Vectors are magni f ied 10 times and sampled every 20 pixels. Measure Average Standard Deviation l'k 1.000000 0.000000 vk 0.000000 . 0.000000 " t 1.000000 0.000000 vk 0.000000 0.000000 ("*>v*)| 1.000000 0.000000 Tab le 6.5.1: The averages and standard deviat ions of measures associated wi th the k n o w n mot ion f ie ld . A l l measures are in pixels. 89 Type 2A Min. A Optical Flow Estimation t t 4 ' + ' t I ' " + t +• ' .+. V + t 4- ' V + t 4- ' 4 ' A * t ' F igu re 6.5.2: The type 2 A M i n . A opt ical flow est imation fo r the t rans la t ing curved sheet Vectors are magni f ied 0.00000000000001 t imes and sampled every 20 pixels. Measure Average Standard Deviation 1.036104 0 .263672 V e - 2 .66135E+13 9 .60369E+14 llk 1 0 Vk 0 0 eu 0.036104 0 .263672 ev - 2 .66135E+13 9 .60369E+14 e 4 .14421E+14 8 .66752E+14 e» 0 .104323 0.244831 ev 4 .14421E+14 8 .66752E+14 r 4 .14421E+14 8 .66752E+14 Z 1.46612 0.38511 Tab le 6.5.2: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . A opt ical f low est imat ion. The measure r is unitless. The measure Z is in radians. A l l other measures are in pixels. 90 Type 2A Min. B Optical Flow Estimation F igu re 6.5.3: The type 2A M i n . B opt ica l flow est imation fo r the t rans la t ing curved sheet Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation ue 1.019631 0.271611 Ve 0 0 .090596 Uk 1 0 Vk 0 0 eu 0.019631 0.271611 0 0 .090596 e 0 .118612 0 .261335 eu 0.109976 0 .249123 ev 0 .017412 0 .088907 R 0 .118612 0 .261335 Z 0.026161 0 .138672 Tab le 6.5.3: The averages and s tandard deviat ions of measures associated w i th the type 2 A M i n . B opt ical f low est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 91 Type 2 A Min. C Optical Flow Estimation — — > — F igu re 6.5.4: The type 2 A M i n . C opt ical flow est imation fo r the t rans la t ing curved sheet Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation Ue 1.028629 0 .257969 V e 0 0 .059492 ltk 1 0 Vk 0 0 eu 0 .028629 . 0 .257969 ev 0 0 .059492 e 0 .106015 0 .244268 eu 0 .101082 0 .239059 0 .009012 0 .058805 r 0 .106015 0 .244268 Z 0 .010296 0.059511 Tab le 6.5.4: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . C opt ica l flow est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 92 Type 2A Min. D Optical Flow Estimation t A A t I ' A A t \ ' A A " + t I ' + " A A t I ' f ' ' t 4 ' " * • * + ''• A A I ' + " V + F igu re 6.5.5: The type 2 A M i n . D opt ica l flow est imation fo r the t rans la t ing curved sheet. Vectors are magni f ied 0.00000000000001 t imes and sampled every 20 pixels. Measure Average Standard Deviation 1 1 e ' 1.036104 0.263671 V e - 2 .66135E+13 9 .60369E+14 uk 1 0 vk 0 0 0 .036104 0.263671 -2 .66135E+13 9 .60369E+14 e 4 .14421E+14 8 .66752E+14 eu 0 .104323 0.244831 ev 4 .14421E+14 8 .66752E+14 r 4 .14421E+14 8 .66752E+14 Z 1.46612 0.38511 Tab le 6.5.5: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . D opt ical f low est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 93 Type 2B, 2B\ and 2B" Min. A Optical Flow Estimation Figure 6.5.6: The type 2 B M i n . A opt ical f low est imation fo r the t rans la t ing curved sheet Vectors are magni f ied 10 times and sampled every 20 pixels. 2B Min. A 2B' Min. A 2B" Min. A Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation ti e 1.03519 0 .253348 1.033541 0 .263507 1.033738 0 .270427 V e 0 0 .342174 0 0 .34486 0 0 .351724 >'k 1 0 1 0 1 0 Vk • 0 0 0 0 0 0 • e « 0 .03519 0 .253348 0.033541 0 .263507 0 .033738 0 .270427 e V 0 0 .342174 0 0 .34486 0 0 .351724 e 0.109014 0 .413064 0 .108739 0 .421503 0 .109902 0 .43116 e„ 0 .098012 0 .236255 0 .097678 0.247021 0 .098678 0 .25403 0 .013043 0 .341925 0 .012938 0 .344617 0 .013015 0 .351483 r 0.109014 0 .413064 0 .108739 0 .421503 0 .109902 0 .43116 Z 0 .005392 0 .057209 0 .005237 0 .056437 0 .005165 0 .056098 Tab le 6.5.6: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . A opt ical f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 94 Type 2 B , 2 B ' , and 2 B " Min. B Optical Flow Estimation F igu re 6.5.7: The type 2 B M i n . B opt ica l f low est imat ion fo r the t rans la t ing curved sheet Vectors are magni f ied 10 times and sampled every 20 pixels. 2 B Min. B 2 B ' Min. B 2 B " Min. B Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation 1.004322 0.279711 0 .281178 0 .378537 0 .002568 0 .015323 V e 0 0 .090325 0 0 .047219 0 0 .00052 llk 1 0 1 0 1 0 Vk 0 0 0 0 0 0 eu 0 .004322 0.279711 -0 .718822 0 .378537 -0 .997432 0 .015323 ev 0 0 .090325 0 0 .047219 0 0 .00052 e 0 .128627 0 .264328 0 .761678 0 .286473 0 .997432 0 .015323 e« 0 .120163 0 .25262 0 .760147 0 .286649 0 .997432 0 .015323 0.017311 0 .08865 0 .006504 0 .046769 0 .000033 0 .000519 r 0 .128627 0 .264328 0 .761678 0 .286473 0 .997432 0 .015323 Z 0.026161 0 .138672 0.026161 0 .138672 0.026161 0 .138672 Tab le 6.5.7: The averages and standard deviat ions of measures associated wi th the type 2 B , 2 B \ and 2 B " M i n . B opt ica l f low est imation. The measure r is unitless. The measure Z is in rad ians. A l l other measures are in pixels. 95 Type 2 B , 2 B \ and 2 B " Min. C Optical Flow Estimation Figure 6.5.8: The type 2B M i n . C opt ica l f low est imation fo r the t rans la t ing curved sheet Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2 B Min. C 2 B'Min. C 2 B " Min. C Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation "e 1.019292 0 .260423 0 .823345 0.267081 0 .789354 0 .26337 V e 0 0 .059477 0 0 .056755 0 0 .056336 11 k 1 0 1 0 1 0 . vk 0 0 0 0 0 0 e u 0.019292 0 .260423 -0 .176655 0.267081 -0 .210646 0 .26337 • e V 0 0 .059477 0 0 .056755 0 0 .056336 e 0.110236 0 .244084 0 .232095 0 .227795 0 .26196 0 .219738 0 .105329 0 .23895 0 .229212 0 .223605 0 .259287 0 .215647 ev 0.009011 0 .05879 0 .00827 0 .056149 0 .008293 0 .055722 r 0.110236 0 .244084 0 .232095 0 .227795 0 .26196 0 .219738 Z 0.010318 0 .059533 0 .011658 0 .067029 0 .01235 0.070991 Tab le 6.5.8: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . C opt ical f low est imat ion. T h e measure r is unit less. The measure Z is in radians. A l l o ther measures are in pixels. 96 Type 2B, 2B\ and 2B" Min. D Optical Flow Estimation F igu re 6.5.9: The type 2B M i n . D opt ical flow est imation fo r the t rans la t ing curved sheet. Vectors are magni f ied 10 times and sampled every 20 pixels. 2B Min. D 2B '1M [in. D 2B" Min. D Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation 1.03519 0 .253348 1.033541 0 .263507 1.033738 0 .270428 V e 0 0 .342174 0 0.344861 0 0 .351729 1 0 1 0 1 • 0 vk 0 0 0 0 0 0 0 .03519 0 .253348 0.033541 0 .263507 0 .033738 0 .270428 e V 0 0 .342174 0 0.344861 0 0 .351729 e 0.109014 0 .413064 0 .108739 0 .421504 0 .109902 0 .431165 eu 0.098012 0 .236255 0 .097678 0 .247022 0 .098678 0.254031 0 .013043 0 .341926 0 .012938 0 .344619 0 .013015 0 .351488 r 0.109014 0 .413064 0 .108739 0 .421504 0 .109902 0 .431165 Z 0 .005392 0 .057209 0 .005237 0 .056437 0 .005165 0 .056098 Tab le 6.5.9: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B \ and 2 B " M i n . D opt ical flow est imat ion. The measure r is unit less. The measure Z is in radians. A l l o ther measures are in pixels. 97 6.6 Rotation of a Surface With Negative Gaussian Curvature Known Motion Field / / S s ^. f / S S s'^. f / ; s ^ /' /• / s ^ ft/* t << , ^ -v •I •! i ' •' * $ J ' ' t * * / •/ J *• * s / / +'*''•'•// ~ ** s- s / F igu re 6.6.1: The known mot ion f ie ld for the rotat ing surface wi th negative Gauss ian curvature . Vectors are magni f ied 10 times and sampled every 20 pixels. Measure Average Standard Deviation uk 0 0 .732816 0 0 .468766 uk 0.631154 0 .372356 IK 0 .390208 0 .259756 II(M*>V*)| 0.784106 0 .376717 Tab le 6.6.1: The averages and s tandard deviat ions of measures associated wi th the known motion field. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 9 8 Type 1 Optical Flow Estimation X ' , 1 'V - \ Y \ S P \ " ^ y s y y Figure 6.6.2: The type 1 opt ical flow est imation fo r the rotat ing surface w i th negative Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation -0 .026634 1.207233 -0 .003204 1.200599 "A- 0 0 .732816 0 0 .468766 -0 .026634 0 .622309 . e v -0 .003204 0 .889667 e 0.77432 0 .761513 e„ 0 .445683 0 .435129 ev 0.573548 0 .680109 r 1.294162 1.580457 Z 0.283987 0 .231137 Tab le 6.6.2: The averages and s tandard deviat ions of measures associated wi th the type 1 opt ical flow est imat ion. The measure r is unitless. The measure Z is in radians. A l l other measures are in pixels. 99 Type 2 A Min. A Optical Flow Estimation F igu re 6.6.3: The type 2 A M i n . A opt ical f low est imation fo r the rotat ing surface w i th negative Gauss ian curvature . Vectors are magni f ied 10 times and sampled every 20 pixels. Measure Average Standard Deviation -0 .020448 1.233756 V e -0 .004997 1.291821 llk 0 0 .732816 Vk 0 0 .468766 eu - 0 .020448 0 .67678 ev - 0 .004997 1.016785 e 0 .805597 0 .918322 e« 0.462881 0 .494148 ev 0 .600366 0 .820624 r 1.34301 1.76883 Z 0 .290014 0 .238149 Tab le 6.6.3: The averages and s tandard deviat ions of measures associated w i th the type 2 A M i n . A opt ical f low est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 100 Type 2 A Min. B Optical Flow Estimation \ v \ f •v /' y •? f /• y -+• \ •i * -+ •'- -v *> 7 j -- > r *• .> ? ,- y y .r < < i V * <r -r / v *-' ' f > r r " •\ y y ^ / V y V \ Figure 6.6.4: The type 2 A M i n . B opt ical flow est imation fo r the rotat ing surface wi th negative Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation «. -0 .012674 .0 .415968 V e -0 .011052 0.31971 1 1 k 0 0 .732816 • Vk 0 0 .468766 e « - -0 .012674 0 .603756 ev -0 .011052 0 .495564 e 0 .672638 0 .397403 eu 0 .450845 0 .401764 ev 0.41674 0 .268378 r 0 .837118 0.177931 Z 1.03181 0 .348888 Tab le 6.6.4: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . B opt ical f low est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures a rc in pixels. 101 Type 2A Min. C Optical Flow Estimation .---'.-^ s ^ z ; s s ^-^ I / / *r / /• f S ^ _^ t ? t r r _ ^ ^ f f ' - - -> , " f '• •> K V s, * * * t f •/ \ , ' •/ / * *- * " r • r *" * " V ^ J +~ f <^ ' s / . *~~ Figure 6.6.5: The type 2 A M i n . C opt ical flow est imation fo r the rotat ing surface w i th negative Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation Ue - 0 .013993 0 .777264 Ve, -0 .02398 0 .519127 Uk 0 0 .732816 Vk 0 0 .468766 e« -0 .013993 0 .230545 ev -0 .02398 0 .217927 e 0 .212287 0 .237374 eu 0 .131485 0 .189888 ev 0 .143358 0 .165876 r 0.321901 0 .337653 Z 0.175804 0 .16936 Tab le 6.6.5: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . C opt ical f low est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 102 Type 2A Min. D Optical Flow Estimation F igu re 6.6.6: The type 2 A M i n . D opt ica l flow est imation fo r the rotat ing surface w i th negative Gauss ian curvature. Vectors are magni f ied 10 times and sampled every 20 pixels. Measure Average Standard Deviation - 0 .020448 1.233743 V e - 0 . 004993 1.291802 uk 0 0 .732816 vk 0 0 .468766 -0 .020448 0 .676764 -0 .004993 1.016762 e 0 .805586 0 .918294 eu 0.462874 0 .494133 0 .600359 0.820601 r 1.342995 1.768789 Z 0 .290012 0 .238148 Tab le 6.6.6: The averages and s tandard deviat ions of measures associated w i th the type 2 A M i n . D opt ica l flow est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 103 Type 2B, 2B', and 2B" Min. A Optical Flow Estimation Figure 6.6.7: The type 2B M i n . A opt ical flow est imation fo r the rotat ing surface wi th negative Gauss ian curva ture . Vectors are magni f ied 10 times and sampled every 20 pixels. 2B Min. A 2B'IVJ in. A 2B" Min. A Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation ii e -0 .028704 1.240773 -0 .027702 1.227127 -0 .026708 1.209576 Ve 0.001175 1.298493 -0 .000047 1.241082 -0 .002604 1:202041 '. 0 0 .732816 0 0 .732816 0 0 .732816 0 0 .468766 0 0 .468766 0 0 .468766 e -0 .028704 0 .676498 -0 .027702 0 .651597 -0 .026708 0 .625467 e V 0.001175 1.014995 -0 .000047 0.941671 -0 .002604 0 .891089 e 0.802091 0.919411 0 .79199 0 .827538 0 .775465 0.764591 e„ 0.46076 0.496151 0 .456399 0 .465873 0.446551 0 .438757 ev 0.597054 0.820811 0 .58768 0 .735774 0 .574218 0.681401 r 1.340487 1.770217 1.321145 1.66008 1,295127 1.581188 Z 0.290073 0 .238385 0 .287389 0 .234418 0 .284067 0 .231103 Tab le 6.6.7: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . A opt ical flow est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 1 0 4 Type 2B, 2 B \ and 2B" Min. B Optical Flow Estimation V •• V t •v /• / •f f • r ••. v. > -r --+ \ 4 * r y > A- - > > * y > > y y •• .: < < < < * t * i j j , -, * t <• J- -r r "'• • J N •> t -</ V V J - •> i " V \ F igu re 6.6.8: The type 2 B M i n . B opt ical f low est imat ion fo r the rotat ing surface With negative Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2B Min. B 2B' Min. B 2B" Min. B Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation Ue - 0 . 0 1 2 8 4 5 0 .416126 -0 .011902 0 .374295 -0 .005776 0 .120229 Ve - 0 . 011035 0 .31977 -0 .007854 0 .282625 0 .000868 0 .087973 Uk 0 0 .732816 0 0 .732816 0 0 .732816 V 0 0 .468766 0 0 .468766 0 0 .468766 -0 .012845 0 .603585 -0 .011902 0 .618994 -0 .005776 0 .716097 ev - 0 .011035 0 .495566 -0 .007854 0 .482088 0 .000868 0 .464062 e 0.672501 0 .397383 0 .6824 0 .3874 0 .770124 0 .367517 e« 0.450686 0 . 4 0 1 6 8 9 ' 0 .48155 0 .389097 0 .616056 0 .365089 0.416741 0.268381 0 .398224 0 .271813 0 .385168 0 .258833 r 0.836891 0 .176773 0 .862873 0.153251 0 .987626 0 .055354 Z 1.035537 0 .358807 1.035809 0 .359235 1.035719 0 .358924 Tab le 6.6.8: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . B opt ica l f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 105 Type 2 B , 2 B \ and 2 B " Min. C Optical Flow Estimation \ / z ^ ^ ^ / / / s ^ „ _ t / / /• r „ ^ ^ f f ' * * -> , ' F ' • - > * » N 1 * ' * » > v * * * <• t y •/ ^ - , , . , v ^ ^ •/ / * * *- S S S * - « - ^ ^ ' s / -<"" ^ • — +— F igu re 6.6.9: The type 2B M i n . C opt ical f low est imation fo r the rotat ing surface w i th negative Gauss ian curvature . Vectors are magni f ied 10 times and sampled every 20 pixels. 2 B Min. C 2 B ' M in. C 2 B " Min. C Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation "e -0 .019062 0 .781198 -0 .002108 0.770561 0 .019229 0 .695406 V e - 0 .020856 0 .525582 -0 .04164 0 .506823 -0 .071462 0 .455809 l'k 0 0 .732816 0 0 .732816 0 0 .732816 0 0 .468766 0 0 .468766 0 0 .468766 e it -0 .019062 0 .219673 -0 .002108 0 .212329 0 .019229 0 .206943 ev -0 .020856 . 0 .203386 -0 .04164 0 .179475 -0 .071462 0 .152258 e 0 .206308 0 .218759 0 .188312 0 .208736 0 .20242 0 .174669 e „ 0.127724 0 .179738 0 .117953 0 .176563 0 .129566 0 .162503 ev 0.13817 0 .150695 0 .123363 0 .136843 0 .128487 0 .108537 r 0 .317293 0.33171 . 0 .285666 0 .303132 0 .305908 0 .281594 Z 0.176481 0 .169816 0.162721 0 .159338 0 .191072 0.180111 Tab le 6.6.9: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . C opt ical f low est imation. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 106 Type 2B, 2 B \ and 2B" Min. D Optical Flow Estimation Figure 6.6.10: The type 2 B M i n . D opt ical f low est imat ion fo r the rotat ing surface wi th negative Gauss ian curva ture . Vectors are magni f ied 10 times and sampled every 20 pixels. 2B Min. D 2B'M [in. D 2B" Min. D Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation l'e - 0 . 028703 1.240757 -0 .027701 1.227115 -0 .026706 1.209543 V e 0 .001179 1.298459 -0 .000044 1.241064 -0 .002605 1.202019 l>k 0 0 .732816 0 0 .732816 0 0 .732816 Vk 0 0 .468766 0 0 .468766 0 0 .468766 eu - 0 .028703 0 .676476 -0 .027701 0.651581 -0 .026706 0 .625427 0 .001179 1.014954 -0 .000044 0 .94165 -0 .002605 0 .891064 e 0 .80208 0 .919358 0 .791978 0 .827513 0.775441 0 .764553 e« 0 .460753 0 .496128 0.456391 0 .465859 0 .446532 0 .438719 0 .597046 0 .820765 0 .587672 0 .735754 0 .574204 0 .68138 r 1.340471 1.770151 1.321126 1.660039 1.295093 1.581133 Z 0 .290072 0 .238384 0 .287387 0 .234417 0 .284065 0 .231105 Tab le 6.6.10: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B \ and 2 B " M i n . D opt ical flow est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 107 6.7 Translation of a Surface With Negative Gaussian Curvature Known Motion Field Figure 6.7.1: The known mot ion field fo r the t rans la t ing surface wi th negative Gauss ian curvature . Vec to rs are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation " A -1.000000 0.000000 V . 0.000000 0.000000 " A -1.000000 0.000000 . v*ll 0.000000 0.000000 . ( » A ^ A - } | 1.000000 0.000000 Tab le 6.7.1: The averages and s tandard deviat ions of measures associated wi th the known mot ion field. The measure r is unit less. The measure Z is in radians. A l l o ther measures are in pixels. 108 Type 1 Optical Flow F igu re 6.7.2: The type 1 opt ical flow est imation for the t rans la t ing surface w i th negative Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation u e 1,026016 0 .22315 V e -0.0031 1.253734 1 0 0 0 e it 0.026016 0 .22315 e V -0.0031 1.253734 e 0 .302459 1.237274 e „ 0.09086 0 .205467 ev 0 .266413 1.225104 r 0 .302459 1.237274 .z 0 .165259 0 .242102 Tab le 6.7.2: The averages and s tandard deviat ions of measures associated wi th the type 1 opt ical f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 109 Type 2A Min. A Optical Flow - ' * - * - - V Figure 6.7.3: The type 2 A M i n . A opt ical flow est imation for the t rans la t ing surface wi th negative Gauss ian curvature . Vectors are magni f ied 10 times and sampled every 20 pixels. Measure Average Standard Deviation 1.015535 0 .21744 V e 0 .019208 1.198455 1 0 0 0 e u 0 .015535 0 .21744 e V 0 .019208 1.198455 e 0.30576 1.179276 0 .093898 0 .196734 0 .263546 1.169275 r 0.30576 1.179276 Z 0 .166728 0 .241619 Tab le 6.7.3: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . A opt ical flow est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 110 Type 2A Min. B Optical Flow \ ••: v y -> - ^ - 1 , "> 7 V •* \ \- * 7 / J-^ --> —f j** —* t -t —v„ v — — - * s _» -y ~y ** — s —> ~> -» -> —*• —-t /. ^ -± ~A . -r- ^ - J -s —r —r S ... f * ;. / * - . ^ - s - » > V ^ V V F igu re 6.7.4: The type 2 A M i n . B opt ical f low est imation fo r the t rans la t ing surface wi th negative Gauss ian curvature . Vectors are magni f ied 10 times and sampled every 20 pixels. Measure Average Standard Deviation 0 .643305 0 .454303 V e - 0 . 005065 0 .295743 Uk 1 0 Vk 0 0 e it - 0 . 356695 0 .454303 e, - 0 . 005065 0 .295743 e 0.52506 0 .381338 0.415091 0 .401648 ev 0 .228468 0 .187856 r 0.52506 0 .381338 Z 0 .607632 0 .546113 Tab le 6.7.4: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . B opt ical f low est imation. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. I l l Type 2A Min. C Optical Flow Figure 6.7.5: The type 2 A M i n . C opt ical flow est imation fo r the t rans lat ing surface wi th negative Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation l'c 0.885491 0 .265396 V e 0.003547 0.234221 "A- 1 0 0 0 e » : -0 .114509 0 .265396 e V 0 .003547 0.234221 e . 0.252921 0 .272853 e„ 0.17713 0 .228412 ev 0.133488 0 .192489 r 0.252921 0 .272853 Z 0 .141913 0 .13829 Tab le 6.7.5: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . C opt ical flow est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 1 1 2 Type 2 A Min. D Optical Flow Figure 6.7.6: The type 2 A M i n . D opt ica l flow est imation for the t rans la t ing surface w i th negative Gauss ian curva ture . Vectors are magni f ied 10 times and sampled every 20 pixels. Measure Average Standard Deviation u e 1.015535 0 .217443 V e 0.019201 1.19849 "k 1 0 "* . 0 0 e u 0 .015535 0 .217443 ev 0.019201 1.19849 e 0.305764 1.179311 e„ 0 .093899 0 .196737 ev 0 .26355 1.16931 r 0.305764 1.179311 Z 0 .166729 0.241621 Tab le 6.7.6: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . D opt ical flow est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 113 Type 2B, 2B\ and 2B" Min. A Optical Flow F igu re 6.7.7: The type 2 B M i n . A opt ical flow est imation fo r the t rans la t ing surface wi th negative Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2B Min. A 2B'M in. A 2B" Min. A Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation Ue 1.023672. 0 .196776 1.023972 0 .200033 1.025944 0 .222412 : Ve 0 .019119 1.198595 0 .016228 1.193974 -0 .003017 1.251757 l'k 1 0 1 0 1 0 Vk 0 0 0 0 0 0 •: eu 0 .023672 0 .196776 0 .023972 0 .200033 0 .025944 0 .222412 0 .019119 1.198595 0 .016228 1.193974 -0 .003017 1.251757 e 0.2988 1.177707 0.296611 1.174071 0 .302286 1.235178 0 .086637 0 .178256 0 .08703 0 .181695 0 .090726 0 .204716 0 .26434 1.169238 0.26204 1.164976 0 .266409 1.223082 r 0.2988 1 .177707 0.296611 1.174071 0 .302286 1.235178 Z 0.166417 0 .241852 0 .165343 0 .241126 0.165451 0 .242477 Tab le 6.7.7: The averages and s tandard deviat ions of measures associated w i th the type 2 B , 2 B ' , and 2 B " M i n . A opt ical flow est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 114 Type 2B, 2B', and 2B' Min. B Optical Flow f s> 7 7 >- -> —t ^* -> - 1 . - r - r —r -> f- -> _ ^ y. ->* — V ^ V —+• - ^ v - J " ^ —v -^ - ^ V — • T+ ' * ;• ^ 7 * „ > ^ * * V V * N " ^ ' * •* Figure 6 . 7 . 8 : The type 2B M i n . B opt ica l f low est imation fo r the t rans la t ing surface wi th negative Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2B Min. B 2B'M [in.B 2B" Min. B Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation "e 0 .643225 0 .454243 0 .463769 0.427441 0 .022268 0.084161 V e -0 .004903 0.295401 -0 .004165 0 .21493 -0 .000826 0 .042337 »A- 1 0 1 0 1 0 0 0 0 , 0 0 0 e -0 .356775 0 .454243 -0 .536231 0.427441 -0 .977732 0.084161 ; ev -0 .004903 0.295401 -0 .004165 0 .21493 -0 .000826 0 .042337 e 0 .524843 0 .381374 0 .625652 0 .353578 0 .979886 0 .068252 e „ 0.415127 0 :401613 0 .5785 0 .368219 0 .978909 0 .069136 ev 0.228161 0 .187687 0.150371 0 .153623 0 .007963 0 .04159 r 0 .524843 0 .381374 0 .625652 0 .353578 0 .979886 0 .068252 Z 0.615816 0.556611 0 .616878 0 .55712 0.613821 0 .554227 Tab le 6.7.8: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . B opt ical flow est imat ion. The measure r is unit less. The measure Z is in radians. A l l o ther measures are in pixels. 115 Type 2B, 2B\ and 2B" Min. C Optical Flow ^ r** -—•*• ^ ^ ^ S --r —r ^-y —? - s — * • -> »• —f j - J -F igu re 6.7.9: The type 2B M i n . C opt ical flow est imat ion fo r the t rans la t ing surface w i th negative Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2B Min. C 2B'M in. C 2B" Min. C Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation Ue 0 .888538 0 .257078 0 .854424 0.250651 0 .808472 0 .262075 V e 0 .003804 0 .234443 0.007841 0 .208503 0 .011213 0 .158672 Uk 1 0 1 0 1 0 Vk 0 0 0 0 0 0 e « - 0 . 1 1 1 4 6 2 0 .257078 -0 .145576 0.250651 -0 .191528 0 .262075 eV 0 .003804 0 .234443 0^007841 0 .208503 0 .011213 0 .158672 e 0 .249805 0 .26662 0 .241806 0 .262835 0 .264099 0 .246819 0 .174002 0 .219625 0 .192678 0 .216548 0 .233544 0 .22544 0 .134093 0 .192345 0.106851 0 .179214 0 .08487 0 .134534 r 0 .249805 0 .26662 0 .241806 0 .262835 0 .264099 0 .246819 Z 0 .14243 0 .139455 0 .117646 0 .129043 0 .100125 0 .107209 Tab le 6.7.9: The averages and standard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . C opt ical flow est imation. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 116 Type 2B, 2B', and 2B" Min. D Optical Flow Figure 6.7.10: The type 2 B M i n . D opt ica l f low est imat ion fo r the t rans la t ing surface wi th negative Gauss ian curva ture . Vectors are magni f ied 10 times and sampled every 20 pixels. 2B Min. D 2B'1V] [in. D 2B" Min. D Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation l'e 1.023672 0 .19678 1.023972 0 .200036 1.025945 0 .222433 Ve 0 .019112 1.198631 0.016221 1.194017 -0 .003035 1.251958 "k 1 0 1 0 1 0 Vk 0 0 0 0 0 0 e u 0 .023672 0 .19678 0 .023972 0 .200036 0 .025945 0 .222433 0 .019112 1.198631 0.016221 1.194017 -0 .003035 1.251958 e 0.298804 1.177742 0 .296616 1.174114 0.30231 1.23538 e» 0.086638 0 .178259 0.087031 0 .181698 0 .09073 0 .204737 0 .264344 1.169274 0 .262045 1.165019 0 .266429 1.223283 r 0.298804 1.177742 0 .296616 1.174114 0.30231 1.23538 Z 0.166417 0 .241853 0 .165343 0 .241128 0 .165448 0 .242478 Tab le 6.7.10: The averages and s tandard deviat ions of measures associated w i th the type 2 B , 2 B ' , and 2 B " M i n . D opt ical f low est imat ion. The measure r is unitless. The measure Z is in radians. A l l other measures are in pixels. 117 6.8 Rotation and Translation of a Surface with Negative Gaussian Curvature Known Motion Field F igu re 6.8.1: The known motion f ie ld for the rotat ing and t rans la t ing surface wi th negative Gauss ian curvature . Vectors are magni f ied 10 times and sampled every 20 pixels. Measure Average Standard Deviation "A- 0.5 0 .828117 yk 0 0 .280359 »A- 0.80427 0 .537506 . V*ll 0 .241545 0 .142319 0 .873972 0 .500532 Tab le 6.8.1: The averages and standard deviat ions of measures associated wi th the known motion f ie ld. A l l measures are in pixels. 118 Type 1 Optical Flow F igu re 6.8.2: The type 1 opt ical flow est imat ion fo r the rotat ing and t rans la t ing surface wi th negative Gauss ian curvature. Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation 0 .513119 1.252835 V e - 0 . 064763 1.500357 llk 0.5 0 .828117 ' Vk 0 0 .280359 e „ 0 .013119 0 .502478 -0 .064763 1.360187 e 0 .829368 1.191253 eu 0.337401 0 .372575 0 .63273 1.205795 r 1.295993 1.822922 Z 0 .311989 0 .312019 Tab le 6.8.2: The averages and s tandard deviat ions of measures associated wi th the type 1 opt ical flow est imat ion. The measure r is unit less. The measure Z is in radians. A l l o ther measures are in pixels. 1 1 9 Type 2A Min. A Optical Flow Figure 6.8.3: The type 2 A M i n . A opt ical flow est imation fo r the rotat ing and t rans lat ing surface wi th negative Gauss ian curvature. Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation l'e 0.49656 1.223665 V e -0 .05245 1.583047 Uk 0.5 0 .828117 Vk 0 0 .280359 e u -0 .00344 0 .515875 -0 .05245 1.448678 e 0.855991 1.278597 0 .345342 0.383241 ev 0.652982 . 1 .294225 r 1.33508 2 .025876 Z 0.320975 0 .323646 Tab le 6.8.3: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . A opt ical f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 120 Type 2A Min. B Optical Flow „ t t * , , , , ; / U - » - » . - * > -J- * ± -t -v i- J- ' •> v •'. , .r -. •> •:• I -I V * < T> -\ ^ \ ^ :- ,-. V \ •» * •* * v v F igu re 6.8.4: The type 2 A M i n . B opt ical f low est imation fo r the rotat ing and t rans la t ing surface wi th negative Gauss ian curvature. Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation l'e 0 .321193 0 .527403 V e -0 .002764 0 .288226 ' V k 0.5 0 .828117 Vk 0 0 .280359 e u -0 .178807 0 .697287 -0 .002764 0 .453419 e 0.686994 0 .501798 e„ 0.472894 0 .54272 er 0 .371909 0 .259375 r 0 .787508 0 .32358 Z 0 .962267 0 .467942 Tab le 6.8.4: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . B opt ical f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 121 Type 2 A Min. C Optical Flow ^ "> x > ^ N J J - > \ \ * y. J-* * + r \ I' \ *• <- J V *• * /-> _ f ^ Figure 6.8.5: The type 2 A M i n . C opt ical f low estimation fo r the rotat ing and t rans lat ing surface wi th negative Gauss ian curvature . Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation ii e 0 .449203 0 .863007 V e -0 .007631 0.40771 "k 0.5 0 .828117 0 0 .280359 -0 .050797 0 .302624 -0 .007631 0 .293145 e 0 .26348 0.332761 e „ 0 .156646 0.263861 ev 0 .172373 0 .237232 r 0 .396028 0 .705978 Z 0.200626 0.237431 Tab le 6.8.5: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . C opt ical f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 122 Type 2A Min. D Optical Flow Figure 6.8.6: The type 2 A M i n . D opt ica l flow est imat ion fo r the rotat ing and t rans lat ing surface wi th negative Gauss ian curvature. Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure L Average Standard Deviation l'e 0.496559 1.223656 V e • .• -0 .052447 1.58304 "A , 0.5 0 .828117 0 0 .280359 -0 .003441 0 .515865 e V -0 .052447 1.448674 e 0 .855979 1.278596 e„ 0.345337 0.383231 ev 0.652974 1.294224 r 1.335063 2 .025853 Z 0 .320975 0 .323646 Tab le 6.8.6: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . D opt ical flow est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 123 Type 2B, 2B', and 2B" Min. A Optical Flow F igu re 6.8.7: T h e type 2 B M i n . A opt ical f low est imat ion fo r the rotat ing and t rans la t ing surface wi th negative Gauss ian curvature. Vectors are magni f ied 10 times and sampled every 20 pixels. 2B Min. A 2B'1VJ in. A 2B" Min. A Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation u. 0.508156 1.252049 0 .509795 1.252176 0 .513128 1.253075 ve - 0 . 060525 1.603328 -0 .058934 1.568129 -0 .06535 1.503125 0.5 0 .828117 0.5 0 .828117 0.5 0 .828117 0 0 .280359 0 0 .280359 0 0 .280359 0 .008156 0 .500724 0 .009795 0 .50053 0 .013128 0 .502684 -0 .060525 1.467881 -0 .058934 1.431114 -0 .06535 1.363113 e 0 .854805 1.295535 0.844671 1.260434 0 .830347 1.194031 eu 0 .336659 0 .370739 0 .336445 0.370711 0 .337452 0 .372808 0 .659103 1.312976 0 .648693 1.277006 0 .633654 1.208642 r 1.3352 2 .032264 1.319977 1.9745 1.296579 1.823741 A 0 .320373 0 .323518 0 .317202 0 .320135 0 .312157 0 .312289 Tab le 6.8.7: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . A opt ical f low est imation. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 124 Type 2B, 2B', and 2B" Min. B Optical Flow «. 1 t * * * , v / - » - » . - » . ' -»- - * — ^ . > ^ v *• -••>•> \ \ t •: > . ? - . - > ». t •< * : ' ,-. V ^ ' > * , ~. ^ , * , \ i Figure 6.8.8: The type 2 B M i n . B opt ica l flow est imation for the rotat ing and t rans lat ing surface w i th negative Gauss ian curvature . Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2B Min. B 2B'M [in. B 2B" Min. B Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation ' "e 0 .320937 0 .527756 0 .230534 0 .481639 0 .007905 0 .126686 V e -0 .002787 0 .288014 -0 .002414 0 .239796 -0 .001839 0 .072969 "k 0.5 0 .828117 0.5 0 .828117 0.5 0 .828117 Vk 0 0 .280359 0 0 .280359 0 0 .280359 e u -0 .179063 0 .697319 -0 .269466 0 .712076 -0 .492095 0 .806567 e V -0 .002787 0.453241 -0 .002414 0 .413926 -0 .001839 0 .287376 e 0.686977 0 .501796 0 .71983 0 .482528 0 .857747 0 .489437 e „ 0.473039 0 .54272 0 .550749 0 .525673 0.783811 0 .527575 ev 0.37171 0 .259349 0.332621 0 .246373 0 .244238 0 .151439 r 0.787473 0.323461 0 .835815 0 .214807 0.987001 0.056331 Z 0 .97095 0 .48489 0.969691 0 .483186 0 .969523 0 .483958 Tab le 6.8.8: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . B opt ical f low est imat ion. T h e measure r is unitless. The measure Z is in radians. A l l other measures are in pixels. 125 Type 2B, 2B\ and 2B" Min. C Optical Flow s * y /• ' ' > , v. 1 " V - v " V v ^ -v \ \ ' * + f <v * S $ *• * S S ' ' " / ^ y y y F igu re 6.8.9: The type 2B M i n . C opt ical f low estimation fo r the rotat ing and t rans la t ing surface wi th negative Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2B Min. C 2B'1V] [in. C 2B" Min. C Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation Ue 0 .456767 0 .869273 0 .46346 0 .86489 0 .468949 0 .809016 Ve -0 .010012 0.41121 0 .001592 0 .372644 0 .02336 0 .308418 0.5 0 .828117 0.5 0 .828117 0.5 0 .828117 0 0 .280359 0 0 .280359 0 0 .280359 -0 .043233 0 .250047 -0 .03654 0 .24857 -0 .031051 0 .257685 -0 .010012 0.291751 0 .001592 0 .24948 0 .02336 0 .183147 e 0.249407 0 .295644 0 .227504 0 .271302 0.216301 0 .233809 0 .142346 0 .21007 0 .139035 0 .209263 0 .14382 0 .216058 0 .170508 0 .236949 0 .147312 0 .201348 0 .12619 0 .134774 r 0.386666 0 .702356 0 .354518 0 .65066 0.335881 0 .620093 Z 0.20077 0 .236467 0.184561 0 .238145 .0 .189116 0 .228745 Tab le 6.8.9: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . C opt ical flow est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 126 Type 2B, 2B\ and 2B" Min. D Optical Flow Figure 6.8.10: The type 2B M i n . D opt ical f low est imat ion fo r the rotat ing and t rans la t ing surface wi th negative Gauss ian curvature. Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2B Min. D 2B' M [in. D 2B" Min. D Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation Ue 0 .508155 1.252039 0 .509794 1.252167 0 .513129 1.253066 V e - 0 .060522 1.603315 -0 .058931 1.568119 -0 .065345 1.503139 Uk 0.5 0 .828117 0.5 0 .828117 0.5 0 .828117 . Vk 0 0 .280359 0 0 .280359 0 0 .280359 eu 0 .008155 0 .500713 0 .009794 0 .500519 0 .013129 0 .502676 ev -0 .060522 1.467869 -0 .058931 1.431107 -0 .065345 1.363136 e 0 .854793 1.295526 0 .844657 1.260431 0 .83033 1.194065 eu 0 .336655 0 .370729 0 .33644 0 .370702 0 .337446 0 .372802 0 .659094 1.312967 0 .648683 1.277003 0.633641 1.208674 r 1.335183 2 .032226 1.319958 1.974473 1.296556 1.823716 Z 0 .320373 0 .323518 0.317201 0 .320134 0 .312152 0 .312288 Tab le 6.8.10: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . D opt ica l f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures arc in pixels. 127 6.9 Rotation of a Surface With Positive Gaussian Curvature Known Motion Field > --r* -^ „ / S S S / / / s / / ; >• .-<> -•> ~> f r / s f r / ' > v v t ' ' s s ^ i ' ' * X > \ * ' •/ , , \ " ^ " ' * V ^ " ^ * - ' • *• *• *• s / / * - * - ' S / / y *- *- " s s / F igu re 6.9.1: The known motion f ie ld fo r the rotat ing surface w i th posit ive Gauss ian curvature . Vectors are magni f ied 10 times and sampled every 20 pixels. Measure Average Standard Deviation w * 0 0 .732816 vk 0 0 .468766 "k 0.631154 0 .372356 IK 0 .390208 0 .259756 \\(i'k^'k) 0.784106 0 .376717 Tab le 6.9.1: The averages and s tandard deviat ions of measures associated wi th the known motion field. A l l measures are in pixels. 128 Type 1 Optical Flow Estimation F igu re 6.9.2: The type I opt ica l flow est imat ion fo r the rotat ing surface wi th posit ive Gauss ian curva ture . Vectors are magni f ied 20 times and sampled every 20 pixels. Measure Average Standard Deviation ue -0 .041499 0 .655506 Ve -0 .049725 0 .77928 0 0 . 7 3 2 8 1 6 . Vk . 0 0 .468766 -0 .041499 0 .608952 ' ev -0 .049725 0 .902984 e 0.754114 0 .788472 0 .435246 0 .4279 0 .562232 0 .708334 r 1.25925 1.490273 Z 0.986764 0 .876978 Tab le 6.9.2: The averages and standard deviat ions of measures associated wi th the type 1 opt ical flow est imat ion. The measure r is unitless. The measure Z is in rad ians. A l l other measures are in pixels. 129 Type 2A Min. A Optical Flow Estimation F igu re 6.9.3: The type 2 A M i n . A opt ical flow est imation fo r the rotat ing surface wi th positive Gauss ian curvature . Vectors are magni f ied 20 times and sampled every 20 pixels. Measure Average Standard Deviation l'e -0 .03137 0 .609187 V e - 0 .033372 0 .72762 WA- 0 0 .732816 Vk 0 0 .468766 e „ -0 .03137 0 .574276 e y - 0 .033372 0.847721 e 0 .752568 0 .695803 e„ 0 .433163 0 .378338 ev 0 .561019 0 .636389 r 1.264576 1.381854 Z 0 .986883 0 .878407 Tab le 6.9.3: The averages and s tandard deviat ions of measures associated wi th the type 2A M i n . A opt ical flow est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 1 3 0 Type 2 A Min. B Optical Flow Estimation Figure 6 .9 . 4 : The type 2 A M i n . B opt ical flow est imation fo r the rotat ing surface wi th positive Gauss ian curvature . Vectors are magni f ied 20 times and sampled every 20 pixels. Measure Average Standard Deviation it e -0 .017124 0 .3845 • V e - 0 .004853 0 .254874 "A 0 0 .732816 ,;A- 0 0 .468766 e a. -0 .017124 0 .616834 e V - 0 .004853 0 .475946 e 0.671933 0 .394735 e « 0.46289 0 .408047 0 .407473 0 .245983 r 0 .837352 0 .178655 Z 1.037083 0.356841 Tab le 6.9 .4: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . B opt ical f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 131 Type 2 A Min. C Optical Flow Estimation 4, ^ y / > / / / - y ~> y / / ' / *^-^>? ft / _ / z , r / /• - - ^ -4-> ^ i i J " *-^-t— I / / y v * > ^ / y > -"•>— j s s <r / Figure 6.9.5: The type 2 A M i n . C opt ical f low est imation fo r the rotat ing surface wi th posit ive Gauss ian curvature . Vectors are magni f ied 20 times and sampled every 20 pixels. Measure Average Standard Deviation -0 .023736 0 .453832 V e -0 .015198 0 .304343 0 0 .732816 0 0 .468766 e -0 .023736 0 .490333 e V -0 .015198 0 .460246 e 0.609 0.286621 0 .389686 0 .298544 ev 0.406519 0 .216317 /: 0.88026 0 .418327 Z 0 .876148 0 .694527 Tab le 6.9.5: The averages and s tandard deviat ions of measures associated wi th the type 2A M i n . C opt ical flow est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 1 3 2 Type 2 A Min. D Optica! Flow Estimation F igu re 6.9.6: The type 2 A M i n . D opt ica l f low est imation fo r the rotat ing surface wi th positive Gauss ian curva ture . Vectors are magni f ied 20 times and sampled every 20 pixels. Measure Average Standard Deviation l'e -0 .031369 0 .609194 V e -0 .033371 0 .727648 uk 0 0 .732816 vk 0 0 .468766 e „ - 0 .031369 0 .574285 e -0 .033371 0 .847745 e 0 .752569 0 .695839 0.433163" 0 .378352 0 .561019 0.636421 r 1.264574 1.381895 Z 0 .986882 0 .878406 Tab le 6.9.6 opt ica l f low : The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . D est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 133 Type 2B, 2B\ and 2B" Min. A Optical Flow Estimation F igu re 6.9.7: The type 2B M i n . A opt ical f low est imation fo r the rotat ing surface w i th positive Gauss ian curva ture . Vectors are magni f ied 20 times and sampled every 20 pixels. 2B Min. A 2B'1V1 in. A 2B" Min. A Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation -0 .034119 0 .618244 -0 .034284 0 .619814 -0 .041133 0 .653982 V e - 0 .034848 0 .734132 -0 .036499 0.734301 -0 .04907 0 .778102 llk 0 0 .732816 0 0 .732816 0 0 .732816 Vk 0 0 .468766 0 0 .468766 0 0 .468766 eu -0 .034119 0 .575265 -0 .034284 0 .576482 -0 .041133 0 .607585 eV -0 .034848 0 .852512 -0 .036499 0 .853057 -0 .04907 0 .901227 e 0.752741 0 .702456 0 .749403 0 .707757 0.75331 0 .786112 e» 0 .432005 0.381391 0 .430673 0 .384735 0 .434779 0 .426394 0 .56265 0 .641409 0 .55952 0 .644953 0.561541 0 .706596 r 1.265056 1.384341 1.257897 1.39099 '1.258519 1.488815 Z 0 .986302 0 .877607 0 .985243 0 .87738 0 .986416 0 .876944 Tab le 6.9.7: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . A opt ical f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 134 Type 2B, 2B\ and 2B" Min. B Optical Flow Estimation Figure 6.9.8: The type 2 B M i n . B opt ica l f low est imat ion fo r the rotat ing surface w i th posit ive Gauss ian curva ture . Vectors are magni f ied 20 times and sampled every 20 pixels. 2B Min. B 2B'M [in.B 2B" Min. B Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation - 0 . 0 1 7 0 7 3 0 .384558 -0 .014975 0 .341235 -0 .008858 0 .120254 V e - 0 .004904 0 .254803 -0 .001787 0 .215919 0 .003439 0.063951 l'k 0 0 .732816 0 0 .732816 0 0 .732816 ' Vk 0 0 .468766 0 0 .468766 0 0 .468766 e - 0 . 017073 0 .616772 -0 .014975 0 .632305 -0 .008858 0 .726078 ev -0 .004904 0 .475876 -0 .001787 0 .463655 0 .003439 0 .459389 e 0 .671855 0 .394686 0 .683304 0 .384828 0 .774848 0 .371358 e« 0 .462847 0.408001 0 .493397 0 .395708 0 .624157 0 .371055 ev 0 .407402 0 :245967 0 .390987 0 .249204 0 .383592 0 .252788 r 0 .837169 0 .177333 0 .864187 0 .149412 0 .991242 0 .041432 Z 1.041031 0 .368464 1.041566 0 .368379 1.04158 0 .368559 Tab le 6.9.8: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . B opt ical f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 135 Type 2B, 2B\ and 2B" Min. C Optical Flow Estimation >/ / / / -» » >• " *" * V -< V * i ^ >• ^ / / / ' v*> • y / ^ * / / ^ - ^ ^ - ^ F igu re 6.9.9: The type 2B M i n . C opt ical f low est imation fo r the rotat ing surface w i th posit ive Gauss ian curva ture . Vectors are magni f ied 20 times and sampled every 20 pixels. 2B Min. C 2B' Min. C 2B"Min. C Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation Ue - 0 .026647 0 .459427 -0 .022188 0 .439344 -0 .024229 0 .411696 V e -0 .014404 0 .307725 -0 .013404 0 .282412 -0 .015622 0 .24474 «* 0 0 .732816 0 0 .732816 0 0 .732816 Vk 0 0 .468766 0 0 .468766 0 0 .468766 eu -0 .026647 0 .485939 -0 .022188 0 .485339 -0 .024229 0 .498795 ev -0 .014404 0 .458205 -0 .013404 0 .433852 - 0 . 0 1 5 6 2 2 0 .411448 e 0 .605692 0.283071 0 .585675 0 .285352 0 .579586 0 .288072 eu 0 .386608 0 .295593 0 .383176 0 .298693 0 .391427 0 .310102 0.405271 0 .214264 0 .380002 0 .209763 0 .361834 0 .196482 r 0 .877475 0 .418385 0 .835206 0 .386688 0 .815678 0 .358595 Z 0 .87495 0 .693722 0 .829868 0 .698959 0 .792386 0 .683293 Tab le 6.9.9: The averages and s tandard deviat ions of measures associated w i th the type 2 B , 2 B ' , and 2 B " M i n . C opt ical f low est imation. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 136 Type 2B, 2B\ and 2B" Min. D Optical Flow Estimation ,/ y , •v-Figure 6.9.10: The type 2 B M i n . D opt ical f low est imation for the rotat ing surface w i th posit ive Gauss ian curva ture . Vectors are magni f ied 20 times and sampled every 20 pixels. 2B Min. D 2B'M [in.D 2B" Min. D Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation Ue - 0 .034118 0 .618236 -0 .034283 0 .619824 -0 .041139 0 .654038 V e - 0 .034846 0 .734097 -0 .036498 0 .734338 -0 .049079 0 .778224 l<k 0 0 .732816 0 0 .732816 0 0 .732816 Vk 0 0 .468766 0 0 .468766 0 0 .468766 e u -0 .034118 0 .575259 - 0 . 0 3 4 2 8 3 0 .576495 -0 .041139 0 .607644 -0 .034846 0 .852482 -0 .036498 0 .853088 -0 .049079 0 .901342 e 0.752741 0 .702416 0 .749403 0 .707804 0 .753325 0 .786276 0 .432004 0 .381382 0 .430673 0 .384754 0 .434787 0.426471 0 .562649 0 .641369 0 .559519 0 .644994 0 .561553 . 0 .706734 r 1.265052 1.384237 1 .257895 1.391045 1.25853 1:48899 Z 0 .986302 0 .877606 0 .985242 0 .877379 0 .98642 0 .876943 Tab le 6.9.10: The averages and s tandard deviat ions of measures associated w i th the type 2 B , 2 B \ and 2 B " M i n . D opt ica l f low est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels 1 3 7 6.10 Translation of a Surface With Positive Gaussian Curvature Known Motion Field Figure 6.10.1: The known motion f ie ld fo r the t rans la t ing surface w i th posit ive Gauss ian curvature. Vectors are magni f ied 10 times and sampled every 20 pixels. Measure Average Standard Deviation uk 1.000000 0.000000 vk 0.000000 ' 0.000000 11 k i. oooooo 0.000000 IK 0.000000 0.000000 ll(«*>v*) 1.000000 0.000000 Tab le 6.10.1: The averages and standard deviat ions of measures associated w i th the known mot ion f ie ld . A l l measures are in pixels. 138 Type 1 Optical Flow / -~ -A. ^ ^ ^> F igu re 6.10.2: The type 1 opt ical flow est imation for the t rans la t ing surface wi th positive Gauss ian curva ture . Vectors are magni f ied 10 times and sampled every 20 pixels. Measure Average Standard Deviation Ue 1.025517 0 .227234 Ve 0 .006833 1.366753 uk 1 0 VA- 0 0 e » 0 .025517 0 .227234 ev 0 .006833 1.366753 e ' 0 .295394 1.353915 e « 0.090913 0 .209812 ev 0 .259299 1.341947 r 0.295394 1.353915 Z 0.165197 0.241831 Tab le 6.10.2: The averages and s tandard deviations of measures associated wi th the type 1 opt ical f low est imat ion. The measure r is unitless. The measure Z is in radians. A l l other measures are in pixels. 139 Type 2A Min. A Optical Flow / Figure 6.10.3: The type 2 A M i n . A opt ical f low est imation fo r the t rans la t ing surface wi th positive Gauss ian curva ture . Vectors are magni f ied 10 times and sampled every 20 pixels. Measure Average Standard Deviation Ue 1.015957 0 .225237 Ve -0 .013608 1.301582 Ut 1 0 Vk 0 0 eu 0.015957 0 .225237 ev -0 .013608 1.301582 e 0.308788 1.284498 0.094621 0 .20502 0 .266682 1.27404 r 0.308788 1.284498 Z 0.16787 0 .245508 Tab le 6.10.3: The averages and standard deviat ions of measures associated wi th the type 2 A M i n . A opt ical f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 140 Type 2A Min. B Optical Flow V > > "V \ . "< --V \ v -<x -\ - V •-4. % *• -> V -* t-— - -> —* ~* * ' -•• > V ^' '"' '"" > — * F igu re 6.10.4: The type 2 A M i n . B opt ical f low est imat ion fo r the t rans la t ing surface wi th posit ive Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation 0.644317 0 .458264 Ve 0.004278 0 .295703 llk 1 0 Vk 0 0 e« -0 .355683 0 .458264 ev 0 .004278 0 .295703 e 0.525498 0 .384477 0 .415829 0 .404476 0 .228316 0 .187959 r 0.525498 0 .384477 Z 0.607421 0 .546064 Tab le 6.10.4: The averages and standard deviat ions of measures associated wi th the type 2A M i n . B opt ical f low est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 141 Type 2 A Min. C Optical Flow F igu re 6.10.5: The type 2 A M i n . C opt ical flow est imation fo r the t rans la t ing surface wi th positive Gauss ian curvature . Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation 0 .886484 0 .269735 Ve - 0 .003978 0 .248934 uk 1 0 0 0 -0 .113516 0 .269735 -0 .003978 0 .248934 e 0 .25546 0 .286993 e„ . 0 .177741 0 .232487 ev 0 .136184 0 .208416 r 0 .25546 0 .286993 Z 0.143571 0 .144375 Tab le 6.10.5: The averages and s tandard deviat ions of measures associated w i th the type 2A M i n . C opt ical f low est imat ion. The measure r is unit less. The measure Z is in rad ians . A l l other measures are in pixels. 142 Type 2A Min. D Optical Flow Figure 6.10.6: The type 2 A M i n . D opt ical How est imation fo r the t rans la t ing surface wi th posit ive Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation Uc 1.015957 0 .22524 Ve -0 .013604 1.301629 l,k 1 0 VA- 0 0 0 .015957 0 .22524 ev -0 .013604 1.301629 e 0 .308792 1.284544 e. 0.094621 0 .205023 ev 0.266685 1.274087 r 0 .308792 1.284544 Z 0.167869 0 .245507 Tab le 6.10.6: The averages and standard deviations of measures associated wi th the type 2 A M i n . D opt ical f low est imat ion. The measure r is unitless. The measure Z is in rad ians. A l l other measures are in pixels. 143 Type 2B, 2B\ and 2B" Min. A Optical Flow Figure 6.10.7: The type 2 B M i n . A opt ica l f low est imat ion fo r the t rans lat ing surface wi th posit ive Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2B Min. A 2B' JVl in. A 2B" Min. A Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation u. 1.024111 0 .205278 1.024233 0 .207827 1.025478 0 .226717 V e - 0 .013479 1.301753 -0 .01136 1.303557 0 .006757 1.365383 U't 1 0 1 0 1 0 V\- 0 0 0 0 0 0 e „ 0.024111 0 .205278 0 .024233 0 .207827 0 .025478 0 .226717 e V - 0 .013479 1.301753 -0 .01136 1.303557 0 .006757 1.365383 e 0.301854 1.283099 0 .298837 1.286025 0 .295243 1.352477 eu 0 .087338 0 .187329 0.087551 0 .190036 0 .090824 0 .209286 ev 0 .267532 1.274035 0 .264303 1.276531 0 .259288 1.340554 r 0 .301854 1.283099 0 .298837 1.286025 0 .295243 1.352477 Z 0.167584 0 .245748 0 .166093 0 .244278 0 .165339 0 .242044 Tab le 6.10.7: The averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . A opt ica l flow est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 144 Type 2B, 2B\ and 2B" Min. B Optical Flow v s v v ~ v V "< ' \ v ' A * y * * >• • * - * A v * >• - » - - i . -v. •- —*• - * ^ V ~"-.v . — * - ^ ^ -4 - V- 4- - .^ —^- — t ^ 4 — » • / t-y —>• v . . ^ ^-v >. - > - * ' V V ~ * y* *• ... V "> ^y f- j. > >, ^ f J- > v y * /• F igu re 6.10.8: The type 2B M i n . B opt ical flow est imat ion fo r the t rans la t ing surface wi th posit ive Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2B Min. B • • 2B' M [in.B 2B" Min. B Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation 0 .644237 0 .458205 0 .464752 0 .431849 0 .02266 0 .090479 , Ve 0 .004116 0.295361 0 .003403 0.214856 0 .001088 0 .045915 1 0 1 0 1 0 vk 0 0 0 0 0 0 - 0 .355763 0 .458205 -0 .535248 0 .431849 -0 .97734 0 .090479 0 .004116 0.295361 0 .003403 0 .214856 0 .001088 0 .045915 e 0 .52528 0 .384512 0 .62606 0 .356657 0.980031 0 .070918 eu 0 .415866 0.404441 0 .579204 0 .370816 0.978921 0 .071369 0 .228009 0 .18779 0 .150215 0 .153653 0 .008067 0 .045214 r 0 .52528 0 .384512 0 .62606 0 .356657 0.980031 0 .070918 •Z 0 .615197 0 .555658 0 .61643 0 .55666 0 .614235 0 .555405 Tab le 6.10.8: The averages and standard deviat ions of measures associated w i th the type 2B, 2B', and 2B" M i n . B opt ica l flow est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 145 Type 2B, 2B\ and 2B" Min. C Optical Flow F igu re 6.10.9: The type 2B M i n . C opt ical f low est imation for the t rans la t ing surface wi th posit ive Gauss ian curva ture . Vectors are magni f ied 10 times and sampled every 20 pixels. 2BMin. C 2B'M in. C 2B" Min. C Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation 0 .889569 0 .261477 0 .855194 0 .254277 0.808671 0 .262887 • V e -0 .004228 0 .249166 -0 .008183 0 .221797 -0 .011086 0 .160312 » A - 1 0 1 0 1 0 Vk 0 0 0 0 0 0 eu -0.110431 0 .261477 -0 .144806 0 .254277 -0 .191329 0 .262887 e v -0 .004228 0 .249166 -0 .008183 0 .221797 -0 .011086 0 .160312 e 0 .252315 0 .281073 0 .243787 0 .274687 0 .264394 0.248261 e« 0 .174575 0 .223804 0 .193088 0 .219868 0 .233625 0 .22613 ev 0.136806 0.208291 0 .109119 0 .19327 0 .085334 0 .136164 0 .252315 0 .281073 0 .243787 0 .274687 0 .264394 0.248261 z 0 .144089 0 .145502 0 .119124 0.135191 0 .100342 0 .107289 Tab le 6.10.9: The averages and s tandard deviat ions of measures associated w i th the type 2 B , 2 B ' , and 2 B " M i n . C opt ical flow est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 146 T y p e 2 B , 2 B ' , a n d 2 B " M i n . D O p t i c a l F l o w Figure 6.10.10: The type 2 B M i n . D opt ica l f low est imation fo r the t rans la t ing surface wi th positive Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2 B M i n . D 2 B ' M [ i n . D 2 B " M i n . D M e a s u r e A v e r a g e S t a n d a r d D e v i a t i o n A v e r a g e S t a n d a r d D e v i a t i o n A v e r a g e S t a n d a r d D e v i a t i o n ne 1.024111 0.205281 1.024233 0 .20783 1.025479 0 .226738 Ve - 0 . 013475 1.301802 -0 .011356 1.303614 0 .006772 .1 .365593 ltk 1 0 1 0 1 0 Vk 0 0 0 0 0 0 e u 0.024111 0.205281 0 .024233 0 .20783 0 .025479 0 .226738 g v - 0 . 013475 1.301802 -0 .011356 1.303614 0 .006772 1.365593 e 0 .301858 1.283148 0.298841 1.286083 0 .29526 1.352689 e u 0 .087339 0 .187333 0.087551 0 .19004 0 .090827 0 .209307 ev 0 .267535 1.274085 0 .264306 1.276589 0 .259302 1.340765 r 0 .301858 1.283148 0.298841 1.286083 0 .29526 1.352689 Z 0 .167582 0 .245747 0 .166092 0 .244277 0 .165339 0 .242045 Tab le 6.10.10: The averages and standard deviations of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . D opt ica l f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 147 6.11 Rotation and Translation of a Surface with Positive Gaussian Curvature Known Motion Field F igu re 6.11.1: The known mot ion f ie ld f o r the rotat ing and t rans la t ing surface wi th posit ive Gauss ian curva ture . Vectors are magni f ied 10 times and sampled every 20 pixels. i Measure Average Standard Deviation uk 0.5 0 .828117 vk 0 0 .280359 "k 0.80427 0 .537506 vk 0 .241545 0 .142319 0 .873972 0 .500532 Tab le 6.11.1: The averages and s tandard deviat ions of measures associated w i th the known mot ion f ie ld . A l l measures are in pixels. 148 Type 1 Optical Flow > \ \ \ j - s y . ' / , > f , * f , , f , + r F igu re 6.11.2: The type 1 opt ica l f low est imat ion fo r the rotat ing and t rans la t ing surface w i th posit ive Gauss ian curva ture . Vectors are magni f ied 10 times and sampled every 20 pixels. Measure Average Standard Deviation Ue 0.507621 0 .566288 V e - 0 .049223 1.005767 Uk 0.5 0 .828117 Vk 0 0 .280359 e u 0.007621 0 .431403 ev - 0 .049223 1.137235 e 0 .782229 0 .932732 eu 0.308546 0 .301599 ev 0.601414 0 .966444 r 1.269681 1.813566 Z 1.020951 0 .900852 Tab le 6.11.2: The averages and s tandard deviat ions of measures associated w i th the type 1 opt ica l f low est imat ion. The measure r is unitless. The measure Z is in rad ians. A l l other measures are in pixels. 149 Type 2A Min. A Optical Flow F igu re 6.11.3: T h e type 2 A M i n . A opt ical f low est imation fo r the rotat ing and t rans la t ing surface wi th posit ive Gauss ian curvature . Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation «. : 0 .497357 0 .560068 Ve -0 .056957 1.07724 l'k 0.5 0 .828117 Vk 0 0 .280359 K -0 .002643 0 .445587 -0 .056957 1.206419 e • 0 .806202 1.003624 eu 0 .312235 0 .3179 0 .622167 1.035174 r 1.310182 1.998251 Z 1.029846 0 .902167 Tab le 6.11.3: The averages and standard deviat ions of measures associated w i th the type 2 A M i n . A opt ical f low est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 150 I Type 2A Min. B Optical Flow < v X * " J-^ N " % '•' /• / f —V -v •- -* •* ->-+ -v •? A A v + *• —— V V f> »• V > % > v > 4 ^ i- > ^-4. •-. V ^ < ; : . - < < . S , . \ ~. 1 - V ' t * F igu re 6.11.4: The type 2 A M i n . B opt ical flow est imation for the rotat ing and t rans la t ing surface wi th posit ive Gauss ian curvature . Vectors are magni f ied 10 times and sampled every 20 pixels. Measure Average Standard Deviation lle 0 .306493 0 .471503 Ve -0 .001064 0 .245655 l'k 0.5 0 .828117 Vk 0 0 .280359 e „ -0 .193507 0 .701012 ev -0 .001064 0.435231 e 0 .68733 0 .495835 e „ 0 .479903 0 .546397 ev 0 .360952 0 .243179 r 0 .795609 0.351071 Z 0 .972872 0 .48828 Tab le 6.11.4: The averages and s tandard deviat ions of measures associated w i th the type 2 A M i n . B opt ical f low est imat ion. The measure r is unit less. The measure Z is in rad ians. A H other measures are in pixels. 1 5 1 Type 2A Min. C Optical Flow —*• -+ 4- - v —4 ~ s ->• * —v ~* • * - i " » " * ^ "* •~> -v ^> -* •* - V * v -v -v v v •> * •> V v v V > V v \ V - •> • t < * > "Is < * ** F i g u r e 6.11.5: The type 2 A M i n . C opt ical f low est imation fo r the rotat ing and t rans la t ing surface w i th posit ive Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation l'e 0 .425582 0 .500532 V e - 0 .030096 0 .304835 uk 0.5 0 .828117 vk 0 0 .280359 e - 0 . 074418 0 .524139 -0 .030096 0 .505238 e 0 .635109 0 .364764 eu 0 .364147 0 .384256 ev 0 .397148 0 .313749 r 0 .906237 0.741561 Z 0 .913032 0 .764985 Tab le 6.11.5: The averages and s tandard deviat ions of measures associated wi th the type 2 A M i n . C opt ica l f low est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 152 Type 2A Min. D Optical Flow Figure 6.11.6: The type 2 A M i n . D opt ical f low est imation fo r the rotat ing and t rans la t ing surface wi th positive Gauss ian curvature . Vectors are magni f ied 10 t imes and sampled every 20 pixels. Measure Average Standard Deviation U. 0.497355 0 .560068 V e -0 .056948 1.07723 l{k 0.5 0 .828117 Vk 0 0 .280359 eu -0 .002645 0 .445589 ev -0 .056948 1.206409 e 0 .806199 1.003614 0 .312236 0.317901 0 .622163 1.035164 r 1.310174 1.998207 Z 1.029844 0 .902165 Tab le 6.11.6: The averages and standard deviat ions of measures associated w i th the type 2 A M i n . D opt ical f low est imat ion. The measure r is unit less. The measure Z is in rad ians. A l l other measures are in pixels. 153 Type 2B, 2B \ and 2B" M in . A Optical Flow f _, ^-y \ '•A i. -v t — * ^ s s I • / V > V > ' f r V , * f , , , f . + t , z< * Figure 6.11.7: The type 2B M i n . A opt ica l f low est imat ion fo r the rotat ing and t rans la t ing surface w i th posit ive Gauss ian curvature . Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2B Min . A 2B'1V] in. A 2B" Min. A Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation lle 0 .505935 0.5641.89 0 .506717 0.563791 0.507751 0 .566453 Ve . - 0 . 0 5 9 6 7 5 1.079824 -0 .05745 1.042139 -0 .049459 1.005777 l'k 0.5 0 .828117 0.5 0 .828117 0.5 0 .828117 Vk 0 0 .280359 0 0 .280359 0 0 .280359 e u 0 .005935 0.426 0 .006717 0 .426607 0.007751 0 .431269 - 0 . 0 5 9 6 7 5 1.208647 -0 .05745 1.173326 -0 .049459 1.137146 e 0 .801466 1.001762 0 .793565 0 .965539 0 .7823 0 .932516 0 .304359 0 .298117 0 .305072 0 .298272 0 .308466 0 .301494 0 .623707 1.036997 0 .615187 1.000763 0 .601594 0 .96624 r 1.30895 1.999626 1.29059 1.898176 1.269828 1.813396 Z 1.029662 0 .903543 1 .027422 0 .902852 1.021109 0 .90082 Tab le 6.11.7: T h e averages and s tandard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . A opt ica l f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 154 Type 2B, 2B', and 2B" Min. B Optical Flow V v v » -v % '•' /• / "->.v /• " 4 -v ••. -4- —t -y V -V -V -y +• v > r > v v > "> > v > . "• •> > s : •:- V • ' <, < * u * ^ :' ..• -\ -\ ^ * V ^ A * T ,'. v F i g u r e 6.11.8: The type 2B M i n . B opt ical flow est imation fo r the rotat ing and t rans la t ing surface w i th posit ive Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2B Min. B 2B' Min. B 2B" Min. B Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation 0 .306049 0 .471849 0 .215673 0 .421748 0.004231 0 .117099 ve -0 .001117 0 .245555 0 .000042 0 .19006 -0 .001267 0.046651 l'k 0.5i 0 .828117 0.5 0 .828117 0.5 0 .828117 Vk 0 0 .280359 0 0 .280359 0 0 .280359 eu -0.193951 0 .700683 -0 .284327 0.71691 -0 .495769 0 .813087 ev -0 .001117 0 .435192 0 .000042 0 .393499 -0 .001267 0 .283724 e 0.687094 0 .495836 0 .721574 0 .478496 0 .863075 0 .492422 eu 0 .479642 0.546361 0.561 0 .529218 0 .790977 0 .530319 0 .360878 0 .24322 0.320031 0 .228949 0 .241435 0 .149023 r 0 .795308 0 .350908 0 .841983 0 .232489 0 .991672 0 .055754 Z 0 .981682 0 .506227 0 .982106 0 .506385 0 .981977 0 .505957 Tab le 6.11.8: The averages and s tandard deviat ions of measures associated w i th the type 2 B , 2 B ' , and 2 B " M i n . B opt ical f low est imat ion. The measure r is unitless. The measure Z is in radians. A l l other measures are in pixels. 155 Type 2 B , 2 B \ and 2 B " Min. C Optical Flow ^ -r • V 1 -t •<* ^ / 7 r" > V F igu re 6.11.9: The type 2B M i n . C opt ical flow est imat ion fo r the rotat ing and t rans la t ing surface wi th posit ive Gauss ian curva ture . Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2 B Min. C 2 B ' M in. C 2 B " Min. C Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation l'e 0.43066 0 .504944 0 .41747 0 .48343 0.397221 0 .451114 V e -0 .031968 0 /306354 -0 .025905 0 .272953 -0 .016183 0 .215445 0.5 0 .828117 0.5 0 .828117 . 0.5 0 .828117 ' 0 0 .280359 0 0 .280359 0 0 .280359 -0 .06934 0 .510124 -0 .08253 0.519961 -0 .102779 0.552031 e • V -0 .031968 0 .505366 -0 .025905 0 .47472 -0 .016183 0 .424729 e 0.629224 0 .354279 0 .617952 0 .348314 0 .617508 0 .338568 e » 0 .357135 0 .370789 0 .376969 0 .367505 0 .409595 0 .384094 ev 0 .396422 0 .315057 0 .369417 0 .29926 0 .341282 0 .253335 r 0.902814 0,742346 0.862428 0 .64093 0 .841073 0 .594276 z 0 .913482 0 .765897 0 .883429 0 .75185 0 .844306 0 .724234 Tab le 6.11.9: The averages and s tandard deviations of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . C opt ical f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 156 Type 2 B , 2 B \ and 2 B " Min. D Optical Flow Figure 6.11.10: The type 2B M i n . D opt ical f low est imation fo r the rotat ing and t rans la t ing surface wi th posit ive Gauss ian curvature . Vectors are magni f ied 10 t imes and sampled every 20 pixels. 2 B Min. D 2B'M [in.D 2 B " Min. D Measure Average Standard Deviation Average Standard Deviation Average Standard Deviation l'e 0 .505933 0 .564188 0 .506716 0 .56379 0 .507745 0 .566452 Ve - 0 .059666 1.079833 -0 .057441 1.042132 -0 .049438 1.005785 l'k 0.5 0 .828117 0.5 0 .828117 0.5 0 .828117 Vk 0 0 .280359 0 0 .280359 0 0 .280359 eu 0 .005933 0 .426002 0 .006716 0 .426609 0 .007745 0 .431285 ev -0 .059666 1.208653 -0 .057441 1.173318 -0 .049438 1.137147 e 0.801464 1.001772 0 .793562 0 .965532 0 .782296 0 .932527 eu 0.30436 0 .298118 0 .305073 0 .298273 0 .308474 0 .301508 ev 0 .623704 1.037006 0 .615183 1.000755 0 .601582 0 .966247 r 1.308943 1.999653 1.290581 1.898132 1.269809 1.813351 Z 1.029661 0.903541 1.027421 0 .90285 1.021098 0 .900816 Tab le 6.11.10: The averages and standard deviat ions of measures associated wi th the type 2 B , 2 B ' , and 2 B " M i n . D opt ica l f low est imat ion. The measure r is unit less. The measure Z is in radians. A l l other measures are in pixels. 157 C h a p t e r 7 7 Conclusions and Future Work Although our approach is specialized for translations and rotations in the image plane, the problem of optical flow estimation still remains under-determined locally. We have attempted to select the optical flow estimate which best matches the known motion field by using a number of local regularization schemes. This was met with varying degrees of success. • For the curving sheet sequence results of section 6.4, all the optical flow estimation techniques found a solution which poorly matched the known motion field. This sequence yielded the largest r error measure for all optical flow estimation techniques implemented. All the r error measures were over 10 n. The underlying motion in this sequence was neither a translation nor a rotation. The motion was a curving deformation of the sheet. This violated the assumption that the underlying motion was a rotation, translation, or a combination of both. Many translating motion fields were estimated very well by the optical flow estimation techniques implemented. For the translating calibration sphere results of section 6.3, the type 1 and the type 2B" Min. A optical flow estimation techniques do very well with the r error measures being less than 0.18. The type 1 technique assumes the classic optical flow constraint which is correct for this sequence. The type 2B" Min. A minimizes the rotational component of the solution which is appropriate for this sequence. For the translating curved sheet sequence results of section 6.5, the type 2B' 158 M i n . A and the type 2 A M i n . C optical flow estimation techniques do very well with the r error measures being less than 0.109. The 2B' M i n . A minimizes the rotational component of the solution which is appropriate for this sequence. However, the type 2 A M i n . C minimizes all the components of the solution. In sections 6.6, 6.7 and 6.8, we experimented with sequences involving a surface with negative Gaussian curvature under rotation, translation, and combined motions: In these sequences the type 2B, 2B', and 2B" M i n . C selects the optical flow estimate which comes closest to the known motion field. For this particular surface with negative Gaussian curvature, the stabilizing function which minimizes all the components of the solution is able to select the best optical f low estimate under conditions of rotation, translation, and combined motions. In sections 6.9, 6.10 and 6.11, we experimented with sequences involving the surface with positive Gaussian curvature under rotation, translation, and combined motions. In the sequences involving rotation, all the optical flow estimation techniques selected an optical flow estimate which poorly matched the known motion field. This was the case even under different light source directions. This leads us to hypothesize that the sign of the Gaussian curvature of a surface plays an important role in how well our optical flow estimation techniques perform under rotational cases. Under rotation, our optical flow estimation techniques performed very well for the surface with negative Gaussian curvature, but performed poorly for the surface with positive Gaussian curvature. Further research is necessary to validate this hypothesis. . There were a number of general trends which were noticeable among the techniques implemented. In general, the type 2A and 2B methods yielded similar results, 159 under the same stabilizing functions. The only time they differ significantly is in the sequence involving a translating curved sheet. In this case the type 2A Min. A and 2A Min. D yield very bad results, whereas the type 2B estimation does not. So the assumption of exact measurements which is made in the regularization of type 2A, is not acceptable under all conditions. Also in the type 2A and 2B optical flow estimation techniques, the minimum intensity change and rotation stabilization yields a similar, if not the same result, as if just the minimum intensity change stabilization had been used. This is evident from all the test cases. Also noticeable was the fact that the type 2B, 2B', or 2B" optical flow estimation techniques yield similar r values when the optical flow estimate is quite good. The three different A values used have little effect in these cases. An example is in Table 6.6.9. When the type 2B optical flow estimate was not so good, the type 2B', and the type 2B" optical flow also yield bad optical estimates, but the r values can vary more significantly. An example is in Table 6.3.9. Our approach to optical flow estimation is very local, in which only information at a given pixel is considered. This information includes images derivatives and surface gradients at that pixel. Because our approach solves a linear system, it has the potential to be very fast. There are many possible improvements to this approach which have yet.. to be investigated. The stabilizing fiinction plays a very important role in determining the most appropriate optical flow estimation. In our implementations, the stabilizing function for a certain optical flow estimation technique was constant for all pixels in the image. However, customizing the stabilizing function for each pixel may yield better results. Knowledge or estimates of the general underlying motion gained from such 160 methods as feature tracking or previous optical flow iterations could be used to customize the stabilizing function to each pixel. Another area of further research is the use of global regularization, instead of the local regularization techniques which we investigated. 161 Bibliography 1. Elli Angelopoulou, "Gaussian Curvature from Photometric Scatter Plots", Proceeding of IEEE Workshop on Photometric Modeling for Computer Vision and Graphics (PMCVG, CVPR Workshop) pp. 12-19, 1999. 2. Elli Angelopoulou, Lawrence B. Wolff, "Photometric Computation of the Sign of Gaussian Curvature Using a Curve-Orientation Invariant", Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 432-437, 1997. 3. Elli Angelopoulou, Lawrence. B. Wolff, Sign of Gaussian Curvature from Curve Orientation in Photometric Space, IEEE Transactions On Pattern Analysis and Machine Intelligence, Vol. 20, No. 10, pp. 1056-66, October 1998. 4. J. L. Barron, D. J. Fleet, S. S. Beauchemin, "Performance of Optical Flow Techniques", IJCV, 12(1), pp. 43-77,1994. 5. Richard.L. Burden, J. Douglas Faires, "Numerical Analysis Sixth Edition", Brooks/Cole Publishing Company, 1997. 6. James D. Foley, Andries van Dam, Steven K. Feiner, John F. Hughes, "Computer Graphics Principles and Practice Second Edition in C", Addison-Wesley Publishing Company, Inc., 1996. 7. E.C. Hildreth, "The Measurement of Visual Motion", MIT Press, Cambridge, Mass., 1984. 8. B.K.P Horn, "Robot Vision", The MIT Press 1986. 9. B.K.P. Horn, B . G Schunck, "Determining Optical Flow", Artificial lntelligence(17), pp. 185-203, 1981. 162 10. R. A. Horn and C. R. Johnson, Matrix Analysis, Cambridge University Press, 1990. 11. David Hsu, "Multiple light source estimation of the 2-D motion field when reflectance properties are known", Bachelor's Thesis, Computer Science, UBC, April 1995. 12. Yuji Iwahori, R. J. Woodham, Ardeshir Bagheri, "Principal Components Analysis and Neural Network Implementation of Photometric Stereo", Proceedings of the IEEE Workshop on Physics-Based Modeling in Computer Vision (PBMCV ), pp. 117-125,1995. 13. W. Keith Nicholson, "Linear Aigebra With Applications Third Edition", PWS Publishing Company, Boston, 1995. 14. Cristina Elena Siegerist, Master's Thesis, Computer Science, UBC, April 1996. 15. Cristina Elena Siegerist, "Parallel near-real-time implementation of multiple light source optical flow", Vl-96, pp. 167-174, 1996. 16. R. J. Woodham, "Analysing Images of Curved Surfaces", Artificial Intelligence, pp. 117-140, 1981. 17. R. J. Woodham, "Gradient and Curvature from Photometric Stereo Including Local Confidence Estimation", J. Opt. Soc. Am. A, vol 11, pp. 3050-3068, 1994. 18. R. J. Woodham, "Multiple Light Optical Flow", Proceedings of the Third International Conference in Computer Vision, pp. 42-45, Osaka, Japan, 1990. 163
- Library Home /
- Search Collections /
- Open Collections /
- Browse Collections /
- UBC Theses and Dissertations /
- Multiple light source optical flow for translations...
Open Collections
UBC Theses and Dissertations
Featured Collection
UBC Theses and Dissertations
Multiple light source optical flow for translations and rotations in the image plane Takagi, Shingo Jason 2001
pdf
Page Metadata
Item Metadata
Title | Multiple light source optical flow for translations and rotations in the image plane |
Creator |
Takagi, Shingo Jason |
Date Issued | 2001 |
Description | The classic optical flow constraint equation is accurate under conditions of translation and distant light sources, but becomes inaccurate under conditions where the object may rotate or deform. The inaccuracies are generally a result of the changing intensities at points under rotation or deformation. The changing intensities of these points are associated with the changing surface gradients. We investigate a novel approach to multiple light source optical flow under known reflectance properties. This novel approach is specialized for translation and for rotation around axes which are parallel to the optical axis. The assumption that the object is moving under translation, rotation, or a combination of translation androtation in the image plane allows us to introduce a physical surface area constraint. Even with this additional constraint, however, the problem still remains locally underdetermined. At each time frame, our multiple light source optical flow approach assumes that three images are acquired from the same viewpoint, but under three different illumination conditions. Photometric stereo determines many of the coefficients in our underdetermined system, which has six equations in seven unknowns at each pixel in the image. Two of the unknowns are the optical flow components. Another three of the unknowns are the total derivatives of the three intensities with respect to time. The last two are the total derivatives of the surface gradients with respect to time. A variety of local regularization methods were investigated to select optical flow estimates which best matched the known motion fields. All the experimental results for this approach were obtained from synthetic data, in which the motion fields were known. |
Extent | 11950220 bytes |
Genre |
Thesis/Dissertation |
Type |
Text |
FileFormat | application/pdf |
Language | eng |
Date Available | 2009-07-29 |
Provider | Vancouver : University of British Columbia Library |
Rights | For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use. |
DOI | 10.14288/1.0051694 |
URI | http://hdl.handle.net/2429/11457 |
Degree |
Master of Science - MSc |
Program |
Computer Science |
Affiliation |
Science, Faculty of Computer Science, Department of |
Degree Grantor | University of British Columbia |
GraduationDate | 2001-05 |
Campus |
UBCV |
Scholarly Level | Graduate |
AggregatedSourceRepository | DSpace |
Download
- Media
- 831-ubc_2001-0120.pdf [ 11.4MB ]
- Metadata
- JSON: 831-1.0051694.json
- JSON-LD: 831-1.0051694-ld.json
- RDF/XML (Pretty): 831-1.0051694-rdf.xml
- RDF/JSON: 831-1.0051694-rdf.json
- Turtle: 831-1.0051694-turtle.txt
- N-Triples: 831-1.0051694-rdf-ntriples.txt
- Original Record: 831-1.0051694-source.json
- Full Text
- 831-1.0051694-fulltext.txt
- Citation
- 831-1.0051694.ris
Full Text
Cite
Citation Scheme:
Usage Statistics
Share
Embed
Customize your widget with the following options, then copy and paste the code below into the HTML
of your page to embed this item in your website.
<div id="ubcOpenCollectionsWidgetDisplay">
<script id="ubcOpenCollectionsWidget"
src="{[{embed.src}]}"
data-item="{[{embed.item}]}"
data-collection="{[{embed.collection}]}"
data-metadata="{[{embed.showMetadata}]}"
data-width="{[{embed.width}]}"
async >
</script>
</div>
Our image viewer uses the IIIF 2.0 standard.
To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0051694/manifest