Applied Science, Faculty of
Civil Engineering, Department of
DSpace
UBCV
Yang, Jian Song
Master of Applied Science - MASc
University of British Columbia
The atmospheric river event in November 2021 is one of the costliest natural disasters in Canadian history. Among others, the Coldwater River in Merritt, British Columbia breached its banks on November 15th, 2021, resulting in extensive damage to the infrastructure and total evacuation of the residents. Estimating the magnitude of this flood is difficult, as it damaged the local flow monitoring station and altered the surrounding landscape. Parts of this flooding event, including the flow close to its peak, were filmed by local residents using mobile devices or drones. Though with significant perspective distortion and imprecision, they still provide valuable information on this extreme event, which would have otherwise been neglected. This study aims to apply image velocimetry techniques to some of these videos, with limited resources and outdated geodata, for reconstructing surface velocities and discharges during the flood. The analysis method consists of using Large Scale Particle Image Velocimetry and Farneback optical flow on the original clips where possible. The extreme and post-event nature of the flood requires changes to many aspects of the conventional image velocimetry workflow. Ground Control Points are identified in the videos, then geolocated or surveyed after the flood, for rectification of raw velocities from image to real-world coordinates. This conservative measure allows unlimited iterations in orthorectification. Discharges are then calculated using surveyed transects, with water surface elevations estimated from the video frames. Results from both methods show a maximum of 20% difference against estimates from from the Water Survey of Canada, proving the versatility of image velocimetry under adverse conditions. Uncertainties in one standard deviation of all four transect discharges, at a maximum of 47%, are higher than expected but still reasonable, likely due to deviations in estimating the stage directly from the videos of poor quality. Extensive testing on the Farneback method show a different response on velocity estimation, especially when surface features are not as rich as those from flooding. Edge pixels are tested and proven to be a promising metric for quantifying natural surface features, without the need for image binarization which does not work well with dense optical flow.
STREAMFLOW MONITORING IN A TIME OF CHANGE: USING IMAGE VELOCIMETRY METHODS ON CITIZEN VIDEOS OF THE NOVEMBER 2021 FLOODING IN MERRITT, BRITISH COLUMBIA by Jian Song Yang B.A.Sc., The University of British Columbia, 2020 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED SCIENCE in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Civil Engineering) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) October 2023 © Jian Song Yang, 2023 ii The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, a thesis entitled: Streamflow monitoring in a time of change: using image velocimetry methods on citizen videos of the November 2021 flooding in Merritt, British Columbia submitted by Jian Song Yang in partial fulfillment of the requirements for the degree of Master of Applied Science in Civil Engineering Examining Committee: Dr. Steven V. Weijs, Associate professor, Department of Civil Engineering, UBC Supervisor Dr. Gregory A. Lawrence, Professor, Department of Civil Engineering, UBC Supervisory Committee Member iii Abstract The atmospheric river event in November 2021 is one of the costliest natural disasters in Canadian history. Among others, the Coldwater River in Merritt, British Columbia breached its banks on November 15th, 2021, resulting in extensive damage to the infrastructure and total evacuation of the residents. Estimating the magnitude of this flood is difficult, as it damaged the local flow monitoring station and altered the surrounding landscape. Parts of this flooding event, including the flow close to its peak, were filmed by local residents using mobile devices or drones. Though with significant perspective distortion and imprecision, they still provide valuable information on this extreme event, which would have otherwise been neglected. This study aims to apply image velocimetry techniques to some of these videos, with limited resources and outdated geodata, for reconstructing surface velocities and discharges during the flood. The analysis method consists of using Large Scale Particle Image Velocimetry and Farneback optical flow on the original clips where possible. The extreme and post-event nature of the flood requires changes to many aspects of the conventional image velocimetry workflow. Ground Control Points are identified in the videos, then geolocated or surveyed after the flood, for rectification of raw velocities from image to real-world coordinates. This conservative measure allows unlimited iterations in orthorectification. Discharges are then calculated using surveyed transects, with water surface elevations estimated from the video frames. Results from both methods show a maximum of 20% difference against estimates from from the Water Survey of Canada, proving the versatility of image velocimetry under adverse conditions. Uncertainties in one standard deviation of all four transect discharges, at a maximum of 47%, are higher than expected but still reasonable, likely due to deviations in estimating the iv stage directly from the videos of poor quality. Extensive testing on the Farneback method show a different response on velocity estimation, especially when surface features are not as rich as those from flooding. Edge pixels are tested and proven to be a promising metric for quantifying natural surface features, without the need for image binarization which does not work well with dense optical flow. v Lay Summary The November 2021 flooding event in British Columbia is the costliest natural disaster in Canadian history. This includes the Coldwater River in Merritt, which caused extensive damage. Such an extreme event presents adverse conditions for post-event discharge estimates. Videos from local citizens on parts of this event are used to reconstruct surface velocities and discharges, processed with Large Scale Particle Image Velocimetry (LSPIV) and Farneback optical flow, with applications of the latter on extreme flows almost absent from literature. The results, when compared against estimated values from the Water Survey of Canada, show acceptable levels of accuracy but with higher uncertainties likely from poor quality of the videos. Farneback method produced similar results to LSPIV under similar processing parameters and can be directly incorporated into existing workflows. Orthorectifying raw velocity vectors which are determined from the original frames is recommended when georeferencing data isn’t immediately available or with high uncertainties. vi Preface This thesis consists of original research conducted by the author, J. Yang. Steven Weijs is the supervisory author who collaborated with the author throughout the project. Chen Shi was involved with J.Y. in the collection of freshet videos and field surveying data. The videos and photos used in image processing and analysis, were provided by Simon Rizzardo and Ross Stewart of Merritt, B.C, under verbal agreement for use in research. The drone videos were made public by the City of Merritt on YouTube. Figures Figure 2.1, Figure 2.2, Figure 2.3, Figure 2.4, Figure 2.5, Figure 2.6, Figure 2.7, Figure 2.8, Figure 2.10, Figure 3.1, Figure 3.3, and Figure 3.6 are used with permission from applicable sources, or under applicable Creative Commons licenses. Parts of this thesis, specifically a version of Chapters 3 and 4, as well as Sections 5.1, 5.2 (excluding Section 5.2.1), 5.4.1, 5.6, 5.7, and 5.9 (excluding Section 5.9.3), have been compiled into a publication named “Reconstruction an extreme flow event with image velocimetry techniques on citizen videos – the November 2021 flooding in Merritt, British Columbia, Canada” by Jason J.S. Yang and Steven V. Weijs. As of October 1, 2023, it is currently under revision following peer review. J.Y. conducted all the experiments and wrote the entire manuscript, while S.W. reviewed and made changes to the manuscript. vii Table of Contents Abstract ......................................................................................................................................... iii Lay Summary .................................................................................................................................v Preface ........................................................................................................................................... vi Table of Contents ........................................................................................................................ vii List of Tables ................................................................................................................................ xi List of Figures ............................................................................................................................. xiii List of Abbreviations ...................................................................................................................xx Acknowledgements .................................................................................................................... xxi Dedication .................................................................................................................................. xxii Chapter 1: Introduction ................................................................................................................1 Chapter 2: Literature Review .......................................................................................................5 2.1 Image velocimetry methods ............................................................................................ 5 2.1.1 Particle Image Velocimetry ........................................................................................ 5 2.1.2 Optical flow .............................................................................................................. 10 2.1.3 Eulerian vs Lagrangian frames of reference ............................................................. 16 2.1.4 Other methods ........................................................................................................... 16 2.2 Currently established workflow .................................................................................... 18 2.2.1 Video collection platforms ........................................................................................ 18 2.2.2 Site setup and video sampling................................................................................... 20 2.2.3 Processing ................................................................................................................. 22 2.2.4 Orthorectification types ............................................................................................ 29 2.3 Application of image velocimetry in extreme events ................................................... 32 viii 2.3.1 Previous cases ........................................................................................................... 32 2.3.2 Challenges with citizen videos.................................................................................. 35 2.3.3 Adaptations for post-event studies ............................................................................ 36 2.3.4 Accuracy and uncertainties ....................................................................................... 37 2.4 Tuning analysis parameters........................................................................................... 38 2.4.1 Existing tests with analysis parameters ..................................................................... 38 2.4.2 Techniques in optimizing analysis parameters ......................................................... 41 2.5 Research gaps summary ................................................................................................ 43 Chapter 3: Data collection...........................................................................................................45 3.1 The flooding event and citizen videos .......................................................................... 45 3.2 Challenges with access, safety, and resources .............................................................. 48 3.3 Video data source .......................................................................................................... 52 3.4 Site areas and description.............................................................................................. 56 3.5 Geodata collection on GCPs and transects ................................................................... 61 3.5.1 Ground Control Points (GCPs) ................................................................................. 62 3.5.2 Cross sections............................................................................................................ 65 3.6 Cameras and lens distortion correction ......................................................................... 66 Chapter 4: Image data processing ..............................................................................................71 4.1 Before processing.......................................................................................................... 72 4.1.1 Video selection and trimming ................................................................................... 72 4.1.2 Camera lens distortion correction and stabilization .................................................. 72 4.2 Image processing for raw surface velocities ................................................................. 74 4.3 Orthorectification workflow for surfacy velocity vectors ............................................ 76 ix 4.4 Estimation of stage and discharge................................................................................. 80 4.5 Surface features ............................................................................................................. 83 4.6 Parameter tuning ........................................................................................................... 84 4.6.1 CLAHE block sizes and interrogation area or filter sizes ......................................... 84 4.6.2 Camera calibration .................................................................................................... 86 4.6.3 Pre-analysis vs post-analysis orthorectification ........................................................ 86 Chapter 5: Results........................................................................................................................88 5.1 iPhone videos on atmospheric river at Site 1 ................................................................ 88 5.1.1 November 15, 2021 ................................................................................................... 89 5.1.2 November 16, 2021 ................................................................................................... 92 5.2 Drone videos on Sites 3 and 4....................................................................................... 96 5.2.1 Oblique waves ........................................................................................................... 98 5.3 iPhone videos on freshet at Site 1 ............................................................................... 101 5.4 Camera calibration ...................................................................................................... 106 5.4.1 Flooding videos ....................................................................................................... 106 5.4.2 Freshet video ........................................................................................................... 108 5.5 Computational costs .................................................................................................... 109 5.6 Orthorectification parameters ..................................................................................... 111 5.6.1 2D orthorectification and choice of GCPs .............................................................. 111 5.6.2 3D orthorectification ............................................................................................... 115 5.7 Water surface elevations (WSEs) ............................................................................... 117 5.8 Pre- vs post-analysis orthorectification ....................................................................... 118 5.8.1 Atmospheric River videos....................................................................................... 118 x 5.8.2 Freshet videos ......................................................................................................... 121 5.9 Tuning of processing parameters ................................................................................ 123 5.9.1 LSPIV velocities ..................................................................................................... 123 5.9.2 Farneback velocities................................................................................................ 127 5.9.3 Freshet videos ......................................................................................................... 131 5.10 Edges and image complexity ...................................................................................... 136 Chapter 6: Discussion ................................................................................................................139 6.1 Accuracy and uncertainty of the measurements ......................................................... 139 6.2 Effectiveness of the velocimetry methods .................................................................. 141 6.3 Orthorectification types .............................................................................................. 143 6.4 Surface hydraulic features and edges .......................................................................... 145 6.5 Applications and recommendations ............................................................................ 147 Chapter 7: Conclusion and future work ..................................................................................150 Bibliography ...............................................................................................................................152 Appendices ..................................................................................................................................171 Appendix A 2D Orthorectification errors for GCPs in Site 1 video on 2021-11-15 .............. 171 A.1 GCPs under orthorectification Version 1 ................................................................ 171 A.2 GCPs under orthorectification Version 2 ................................................................ 172 Appendix B Examples of 3D orthorectification errors ........................................................... 174 B.1 Errors in 3D orthorectification on drone video over Sites 3&4 on 2021-11-15 ..... 174 B.2 Errors in 3D orthorectification on iPhone video over Site 1 on 2022-06-16 .......... 175 Appendix C Supplementary data ............................................................................................ 177 Appendix D Example of velocity orthorectification code ...................................................... 184 xi List of Tables Table 3.1. An example of calculated camera parameters from the Camera Calibrator App. ....... 68 Table 4.1. List of all videos used in analysis and their information. ............................................ 71 Table 4.2. List of all videos’ frame properties. ............................................................................. 71 Table 4.3. LSPIV analysis parameters for 4K videos. .................................................................. 85 Table 4.4. LSPIV analysis parameters for HD videos. ................................................................. 85 Table 5.1. Transect WSEs and discharges at Site 1 on 2021-11-15. ............................................ 91 Table 5.2. Errors and uncertainties for the averaged discharges at Site 1 on 2021-11-15. ........... 92 Table 5.3. Transect WSEs and discharges at site 1 on 2021-11-16. ............................................. 95 Table 5.4. Errors and uncertainties for the averaged discharges at site 1 on 2021-11-15. ........... 96 Table 5.5. Averaged transect surface velocities for both methods on 2022-05-22 and 2022-06-16...................................................................................................................................................... 101 Table 5.6. Summarized discharge estimates of Site 1 videos, with and without distortion correction. ................................................................................................................................... 106 Table 5.7. Differences in transect WSE and discharge for the two versions of orthorectification...................................................................................................................................................... 113 Table 5.8. Results for WSE sensitivity analysis on site 1 video in 2021-11-15. ........................ 117 Table 5.9. Averaged edge pixel ratios for all five videos at site 1. ............................................. 137 Table A.1 List of GCPs' image and world coordinates used in version 1, as well the errors in real-world distances. ................................................................................................................... 171 Table A.2 List of GCPs' image and world coordinates used in version 2, as well the errors in real-world distances. ................................................................................................................... 173 xii Table A.3 List of GCPs' image and world coordinates for drone video over Sites 3&4 on 2021-11-15, with camera reprojection errors in pixels. ....................................................................... 174 Table A.4 List of GCPs' image and world coordinates for iPhone video over Site 1 on 2022-06-16, with camera reprojection errors in pixels.............................................................................. 176 xiii List of Figures Figure 2.1. Illustration of velocity components resulted from PIV, on two consecutive image pairs. Figure from Atkins (2016). Copyright Elsevier (2016). Reproduced with permission. ....... 6 Figure 2.2. A typical experimental setup of 2D PIV. Figure from Atkins (2016). Copyright Elsevier (2016). Reproduced with permission. ............................................................................... 7 Figure 2.3. Mapping of the correlation peak in the correlation plane. Figure from Atkins (2016). Copyright Elsevier (2016). Reproduced with permission............................................................... 8 Figure 2.4. Conceptualization of the LSPIV image processing algorithm. Cross-correlation is calculated between the IAs P and Q or R in the search area. Figure from Muste et al (2008). Copyright John Wiley and Sons (2008). Reproduced with permission. ....................................... 10 Figure 2.5. The production and detection of optical flow in the retina (a) and image plane (b). Figure from Raudies (2016). Reproduced under CC-BY 3.0 License. ......................................... 11 Figure 2.6. PIV in Eulerian (A) and PTV in Lagrangian (B) frames of reference. Figure from Jolley et al (2021). Reproduced under CC-BY 4.0 license. .......................................................... 17 Figure 2.7. Steps of a generic processing workflow. Redrawn based on Jolley et al (2021). ...... 23 Figure 2.8. LSPIV-based discharge measurement procedure. Figure from Muste et al (2008). Copyright John Wiley and Sons (2008). Reproduced with permission. ....................................... 28 Figure 2.9. Frames extracted from the YouTube video analyzed in Le Boursicaud et al (2016), showing before the flood (a) and the pulsed flow during the flood (b-d). Video source: Modesto (2011). ........................................................................................................................................... 34 Figure 2.10 The DADR process. a) Enhanced image. b) Subtraction of image pair. c) The binary image. d) The morphological process. e) The ROI mask. f) The ROI region. Figure from Yeh et al (2019). Copyright Elsevier (2019). Reproduced with permission. ........................................... 43 xiv Figure 3.1. Sentinel-2 satellite image over Merritt on November 16, 2021. Major flow directions have been marked in blue arrows, and WSC’s station location marked with red arrow. Image taken from the Copernicus Open Access Hub. (https://scihub.copernicus.eu/) ............................ 46 Figure 3.2. WSC Hydrograph from November to December 2021 (top) and zoomed into the atmospheric river event (bottom). Source of data: WSC & ECCC (2022). .................................. 47 Figure 3.3. Map of highway damage due to the November 15, 2021 storm event. Copyright the Province of British Columbia. All rights reserved. Reproduced with permission of the Province of British Columbia....................................................................................................................... 48 Figure 3.4. Site 1 area photographed on February 27, 2022. ........................................................ 49 Figure 3.5. River bank at site 1 photographed during a field visit in August 2022. ..................... 50 Figure 3.6. A comparison of Google Earth imagery on Sites 3 and 4 in 2007 (top) and 2022 (bottom)......................................................................................................................................... 52 Figure 3.7. Frames from smartphone videos (a,b) and drone videos (c,d) on the atmospheric river flows used in analysis, marked with a general direction of flow. ................................................. 55 Figure 3.8. Overview of the measurement sites in Merritt, over satellite image on 2022-07-30. 57 Figure 3.9. Site 2 in the morning of November 15 (a) and 16 (b), 2021, extracted from the videos recorded from the left bank by Simon Rizzardo. .......................................................................... 58 Figure 3.10. The remains of the bridge at Site 2, photographed on August 5, 2022 from the right bank. .............................................................................................................................................. 58 Figure 3.11. Site 5 as covered by a drone in the morning of November 16, 2021, by Simon Rizzardo. ....................................................................................................................................... 60 Figure 3.12. Overview of Site 0, extracted from drone video recorded on November 29, 2021, by Simon Rizzardo. ............................................................................................................................ 61 xv Figure 3.13. Labelled GCPs in red for Site 1 video filmed on November 15, 2021. ................... 63 Figure 3.14. Labelled GCPs in red for Site 1 video filmed on June 29, 2022. ............................. 64 Figure 3.15. Overview of datums, instrument stations, and surveyed GCPs at Site 1. ................ 65 Figure 3.16. Overview of surveyed transects at Site 1. ................................................................ 66 Figure 3.17. Pattern-centric camera positions as processed from the Camera Calibrator App. ... 67 Figure 3.18. HEIF (left) for photos vs HEVC (right) for videos, as displayed from an iPhone. The photo and video were provided by Ross Stewart. ......................................................................... 69 Figure 4.1. PIVlab GUI, showing analysis parameters, the mask, and enhanced image. ............. 74 Figure 4.2. Contour plot of time-averaged raw velocities determined through PIVlab. .............. 75 Figure 4.3. Contour plot of time-averaged raw velocities from Farneback optical flow. ............. 76 Figure 4.4. Rectified velocity vectors and contours from RIVeR, the original frame and ROI are on the left. ..................................................................................................................................... 80 Figure 4.5. Plots of rectified velocity vectors (a) and magnitude contours (b) from the recreated code. .............................................................................................................................................. 80 Figure 4.6. Defined waterline on original image with surveyed transects (a), and on orthorectified image with extrapolated transects (b). .......................................................................................... 82 Figure 5.1. Results for video recorded on 2021-11-15: LSPIV surface velocity vector (a) and contour (b) plots; Farneback surface velocity vector (c) and contour (d) plots. ........................... 89 Figure 5.2. Transect surface velocities for LSPIV (a) and Farneback (b) on 2021-11-15. .......... 90 Figure 5.3. Site 1 transect profiles with respective WSE values on 2021-11-15. ......................... 92 Figure 5.4. Results for video recorded on 2021-11-16: LSPIV surface velocity vector (a) and contour (b) plots; Farneback surface velocity vector (c) and contour (d) plots. ........................... 94 Figure 5.5. Transect surface velocities for LSPIV (a) and Farneback (b) on 2021-11-16. .......... 95 xvi Figure 5.6. LSPIV result for drone videos on Sites 3 & 4: surface velocity contours (a) and vectors (b) over surveyed or nominal transects on 2021-11-15; LSPIV surface velocity vector field (c) and contours (d) for Site 3 on 2021-11-16. ..................................................................... 97 Figure 5.7. Oblique waves identified with red arrows in the original frame (a) and plotted on the rectified frame (b) with the averaged flow direction and calculated Froude numbers. .............. 100 Figure 5.8. Surface velocity results for freshet video on 2022-05-22: LSPIV vectors (a) and contours (b); Farneback vectors (c) and contours (d). ................................................................ 102 Figure 5.9. Transect velocity from LSPIV (a) and Farneback (b) for video on 2022-05-22, plotted on a rectified frame as background image. ................................................................................. 103 Figure 5.10. Surface velocity results for freshet video on 2022-06-16: LSPIV vectors (a) and contours (b); Farneback vectors (c) and contours (d). ................................................................ 104 Figure 5.11. Transect velocity from LSPIV (a) and Farneback (b) for video on 2022-06-16, plotted on a rectified frame as background image. ..................................................................... 105 Figure 5.12. Transect surface velocities for video on 2021-11-15, with and without lens distortion correction. LSPIV are (a) to (d), Farneback are (e) to (h). ......................................... 107 Figure 5.13. Transect surface velocities for video on 2022-06-16, with and without lens distortion correction. LSPIV are (a) to (d), Farneback are (e) to (h). ......................................... 108 Figure 5.14. Total computing time in PIVlab per different IA sizes on 4K video (a), computing time per different CLAHE block sizes (b) and Farneback averaging filter sizes (c) for both 4K and HD videos............................................................................................................................. 110 Figure 5.15. LSPIV surface velocity contours of orthorectification version 1 (a) and version 2 (b). GCPs outside of the image bounds are also included. Note Version 2 has GCPs #3 and #9 removed....................................................................................................................................... 112 xvii Figure 5.16. LSPIV transect velocities at site 1 on 2021-11-15, under two versions of orthorectification: CS#4 (a), CS#5 (b), CS#6 (c), and CS#8 (d). ............................................... 114 Figure 5.17. Farneback transect velocities at site 1 on 2021-11-15, under two versions of orthorectification: CS#4 (a), CS#5 (b), CS#6 (c), and CS#8 (d). ............................................... 114 Figure 5.18. 3D reprojection of GCPs in drone video on 2021-11-15. ...................................... 115 Figure 5.19. 3D reprojection of GCPs for iPhone video at site 1 on 2022-06-16. ..................... 116 Figure 5.20. Surface velocity vector and contour plots for the video on 2021-11-15 under pre-analysis orthorectification: LSPIV are (a) and (b), Farneback are (c) and (d). .......................... 119 Figure 5.21. Transect velocities for video on 2021-11-15 under the two types of orthorectification. LSPIV are (a) to (d), Farneback are (e) to (h). .............................................. 120 Figure 5.22. Results from pre-analysis orthorectification for video on 2022-06-16: LSPIV vectors (a) and contours (b); Farneback vectors (c) and contours (d). ....................................... 122 Figure 5.23. LSPIV surface velocity profiles at Site 1 CS#5: only changing CLAHE block size (top) and only changing IA size (bottom). PP stands for “Preprocessing”, numbers in pixels. . 124 Figure 5.24. LSPIV surface velocity profiles at Site 1 CS#6 (top) and CS#8 (bottom), under different IA sizes in pixels. ......................................................................................................... 125 Figure 5.25. LSPIV velocity contour plots for Site 1 video on 2021-11-16 under different IA sizes: a) to i) are assigned to increasing IA sizes from 4px to 128px. ........................................ 126 Figure 5.26. Farneback surface velocity profiles at site 1 CS#5: only changing CLAHE block size (top) and only changing filter size (bottom). “PP” stands for Pre-Processing, “F” stands for filter, “n” stands for not used, numbers in pixels. ....................................................................... 127 Figure 5.27. Farneback surface velocity profiles at site 1 CS#6 (top) and CS#8 (bottom), under different CLAHE block sizes in pixels. ...................................................................................... 128 xviii Figure 5.28. Farneback velocity contour plots for Site 1 video on 2021-11-16 under different filter sizes: a) to i) are assigned to increasing IA sizes from 4px to 128px. ............................... 130 Figure 5.29. LSPIV surface velocity vectors and contour plots for video on 2022-06-16 for combinations PP2-IA4 (a,b) and PP2-IA12 (c,d). ...................................................................... 131 Figure 5.30. LSPIV surface velocity vectors and contour plots for video on 2022-06-16 for combinations PP32-IA4 (a,b) and PP32-IA12 (c,d). .................................................................. 133 Figure 5.31. Farneback surface velocity vectors and contour plots for video on 2022-06-16 for combinations PP2-F4 (a,b) and PP2-F12 (c,d). .......................................................................... 134 Figure 5.32. Farneback surface velocity vectors and contour plots for video on 2022-06-16 for combinations PP32-F4 (a,b) and PP32-F12 (c,d). ...................................................................... 135 Figure 5.33. Edge pixels for frame 100/298 in video on 2021-11-15, zoomed in to the full region of interest. Edge pixels for the frame are at 16.43%. .................................................................. 136 Figure 5.34. Examples of edge pixels in freshet videos: on 2022-05-22 (a) and on 2022-06-16 (b). ............................................................................................................................................... 138 Figure 6.1 Likely flow paths including overland flow away from Site 1. Red circle indicates the approximate location where Irnie (2021)’s video was recorded. Drawn over Sentinel-2 image taken from Copercius Open Access Hub. (https://scihub.copernicus.eu/).................................. 140 Figure 6.2. Original frame (a) and orthorectified frame (b) from the video on 2021-11-15. ..... 144 Figure A.1 GCPs identified for orthorectification version 1, over an original frame of the video on 2021-11-15. This Figure is identical to Figure 3.13. ............................................................. 171 Figure A.2 GCPs identified for orthorectification version 2, on an original frame of the video on 2021-11-15. ................................................................................................................................. 172 xix Figure A.3 GCPs identified and numbered on the first frame of drone video over Sites 3&4 on 2021-11-15. ................................................................................................................................. 174 Figure A.4 GCPs identified and numbered on the first frame of iPhone video over Site 1 on 2022-06-16. ................................................................................................................................. 175 xx List of Abbreviations ABoM Australian Bureau of Meteorology BCMOTI British Columbia Ministry of Transportation and Infrastructure DEM Digital Elevation Model ECCC Environment and Climate Change Canada fps Frames per second GCP Ground Control Point GIS Geographic Information System IA Interrogation area (a parameter in PIV/LSPIV processing) LSPIV Large Scale Particle Image Velocimetry OF Optical Flow PIV Particle Image Velocimetry PTV Particle Tracking Velocimetry px pixel STIV Space-time image velocimetry WSC Water Survey of Canada (a division of ECCC) xxi Acknowledgements I would like to offer my gratitude to my thesis advisor, Dr. Steven Weijs, for his guidance on my work. I am grateful to him for accepting my transfer to MASc thesis under his supervision, and allowing more freedom on which I conducted this study. He provided many important suggestions over my methods and code, some of which kept my work on the right track. I would also like to thank Dr. Gregory Lawrence for providing suggestions on surface hydraulic features, specifically oblique waves. Other tips from Drs Alexandre Hauet, Guillaume Bodart, Robert Ljubičić, and Elizabeth Jamieson that arose during conversations at conferences are appreciated here as well. In addition, some ideas and information are provided by Dr. Tobias Müller, Derek Brzoza, and Ben Wahl. I offer my enduring gratitude to Yuzhe (Zack) Xiao and Chun Ting (Jonathan) Lee for assisting me in field visits to Merritt. I owe particular thanks to Chen Shi, who volunteered himself in assisting me in collecting freshet videos and total station surveying data. This work would not materialize without their help. This research was funded by the UBC Civil Engineering Flood Fund, the Earl R Peterson Memorial Scholarship, and Dr. Weijs’ NSERC Discovery Grant. All of which are greatly appreciated. Special thanks are owed to my parents, whose have supported me throughout my years of education. xxii Dedication To my parents 1 Chapter 1: Introduction Streamflow gauging is important in collecting discharge and stage data for monitoring and forecasting. The data collected, especially on extreme flows, is important in correcting rating curve shifts, as well as parameter estimation of streamflow forecasting models. Many of the current techniques used in discharge measurement, utilizing the velocity-area method, require on-site measurements of cross-sectional areas and velocities, which is expensive and labour intensive. This is made more difficult and dangerous under extreme flow events, as the fast-flowing river poses severe hazards to the operators. Remote sensing methods, such as surface radar or image velocimetry, minimize such risk. The latter of which is gaining importance amid increasing frequency of floods and other water-related disasters in Canada (Ness et al, 2021) and around the world (WMO, 2021) due to climate change, complemented by advents in camera technology. Previous applications on image-based methods over the past three decades have culminated into many techniques and workflows, the most notable and widely applied being Large Scale Particle Image Velocimetry (Fujita & Komura, 1994; Hauet et al, 2008b; Muste et al, 2008), Particle Tracking Velocimetry (Tauro et al, 2017), and Space Time Image Velocimetry (Fujita et al, 2007). Originally developed for fluid measurements in laboratory settings, these methods have adapted to field environments by detecting the propagation of turbulence features or artificial tracers on the free surface. Developments in computer vision, specifically optical flow and feature tracking, have also been implemented for streamflow measurements in recent years (Weijs et al, 2014; Tauro et al, 2018; Rousseau, 2019; Perks, 2020; Hutley et al, 2023). Though not originally designed with 2 fluid mechanics principles, and with different detection mechanisms, they have proved to bear strong resemblance to Eulerian or Lagrangian fluid motions to their older counterparts. There have been over a thousand of applications in riverine environments for PIV or PTV methods alone (Jolley et al, 2021), most focused on applications in experimental settings, using stationary or fixed cameras and drones or helicopters. Among which are cases where citizen videos on flash floods, filmed with smart phones or drones, were used for analysis (Le Boursicaud et al, 2016; Perks et al, 2016; Guillén et al, 2017), but this source of data is usually not widely used due to higher imprecision, requiring extensions in the software workflow (Winsemius et al, 2022). Those existing studies on citizen videos were restricted to mainly covering artificial cross sections. They also emphasized the importance of coordination with local authorities. Le Boursicaud et al (2016) indicated that more difficult videos need to be tested, specifically on cases in natural environments with few GCPs. While applications on PIV-based method are extensive, the same cannot be established for optical flow. This is especially true for the Farneback technique, which estimate the velocity vectors of every pixel in images using polynomial expansion and pyramid-based scheme for increasing resolution (Farnebäck, 2003). Applications using this technique on riverflow, though proven to have great potential (Wu et al, 2019; Hutley et al, 2023), still remain sparse, especially on extreme events. Patalano et al (2017) developed RIVeR (Rectification of Image Velocity Results), a free MATLAB-based software to rectify raw image velocities from PIV or PTV. It remarked a future extension on the software to optical flow results, but the implementation has not been carried out or made available to date. The study featured in this thesis aims to build on previous studies using citizen videos and applications on optical flow through a recent flooding case in British Columbia. In 3 November 2021, an atmospheric river swept across Southern British Columbia, causing total estimated damage up to 11 billion dollars (Lee & Parfitt, 2022). Among the affected locations is Merritt, where the Coldwater River caused extensive damage to the infrastructure and evacuation of all 7000 residents in the city. The river significantly altered its surrounding landscape and damaged the local flow monitoring station, making estimates on its peak flow discharge difficult. Parts of that event were captured by local residents using smartphones or drones, making it an excellent, yet challenging, case study. It has great significance due to the amount of damage caused, along with limited resources, up-to-date geodata, and surface velocity data for validation. Data collection and analysis were carried under a restricted budget and timeline, resulting in high uncertainties on georeferencing data. The detailed objectives of the project are as follows: • Reconstruct surface velocities and discharges during the event using imprecise citizen videos, and compare the latter against estimates from the Water Survey of Canada; • Investigate the differences in flooding flow surface velocities resulted from LSPIV and OF with different analysis parameters; • Investigate the effect on surface velocities from extrinsic parameters or settings usually not controlled in the citizen videos. This project only analyzed video data covering areas around the Coldwater River in the Merritt city area, thus flooding in urban areas was excluded. Also, only high flow events were considered, this included freshet videos for comparison against those from the atmospheric river event. This thesis is structured as such. First, a literature review on the background of image velocimetry methods, the consideration in its applications, the applications on extreme events, 4 and studies on fine-tuning analysis parameters. Second, methodology on data collection and analysis. Third, results on estimated surface velocities from LSPIV and Farneback optical flow, comparison of estimated discharge values between this study and the Water Survey of Canada, as well as the effect of camera calibration or orthorectification mode on the results. Last, the discussion on the sources of error, future improvement, and recommendations on similar applications. 5 Chapter 2: Literature Review This chapter reviews the image velocimetry methods and their applications, with focus on extreme events and tuning of analysis parameters for detection of optical surface velocity. It provides a background for the thesis through an introduction to particle image velocimetry and its adaptation to field environments, optical flow methods, techniques for field image collection and analysis, applications on extreme flooding events, and analysis parameters. A summary of research gaps follows. 2.1 Image velocimetry methods This section contains an overview on Particle Image Velocimetry and Optical Flow estimation methods, which fall under the Eulerian method for fluid motion. Also, methods that utilize the Lagrangian method, among others, such as Particle Tracking Velocimetry are also mentioned. 2.1.1 Particle Image Velocimetry Particle Image Velocimetry, or PIV in short, is an optical flow measurement and visualization technique. It measures the velocity field of the moving fluid through its seeding or tracer particles under transport. The end result is velocity vectors in both x and y directions, between two consecutive images spaced at time interval ∆𝑡. As illustrated in Figure 2.1, displacements ∆𝑥 and ∆𝑦 are determined between a certain area in frames 1 and 2 timed at 𝑡1 and 𝑡2, with 𝑡2 = 𝑡1 + ∆𝑡. 𝑢 and 𝑣, velocity components in x and y direction respectively, are calculated by dividing ∆𝑥 and ∆𝑦 over ∆𝑡. This technique is nonintrusive, as it measures the flow indirectly, without disturbance from probes (e.g., pitot tubes) placed directly into the fluid flow. 6 Figure 2.1. Illustration of velocity components resulted from PIV, on two consecutive image pairs. Figure from Atkins (2016). Copyright Elsevier (2016). Reproduced with permission. The PIV technique and experimental apparatus have been refined and improved over the last two decades thanks to advances in technology, leading to their maturity and widespread applications using systems that are commercially feasible. A typical setup for PIV in a laboratory setting is shown in Figure 2.2, including a test rig, seeding generator, pulsed laser system with light sheet, charged-coupled device (CCD) camera, and data acquisition and processing software (Atkins, 2016). The light sheet and CCD camera are installed to be perpendicular to their respective planes of the test body (x-z plane in Figure 2.2). During the experiment, seeding particles are introduced into the fluid. As they move with the flow, they are illuminated by the light sheet. Recordings are taken with synchronized laser light sheet pulses and CCD camera. The particles scatter laser light and are captured by the camera, resulting in digital images as output. As data is converted from analog to digital, the amount of light exposure is expressed in grayscale pixel intensity, from 0 (black) to 255 (white). Particles with less exposure to laser light will have brighter pixels over a dark background. 7 Figure 2.2. A typical experimental setup of 2D PIV. Figure from Atkins (2016). Copyright Elsevier (2016). Reproduced with permission. PIV analysis on the images is performed through statistical methods, calculating spatial cross-correlation between a subregion within two consecutive images (or frames). This is represented as interrogation areas (IA), sub-windows divided within the image. Cross-correlation is a measure of “matching” properties between the two windows when one is spatially displaced from the other, specifically the discrete pixel intensities and image patterns within those windows. In practical settings, Fast Fourier Transform (FFT) is usually used to convert grayscale intensity distribution to signal frequencies, which are then used for cross-correlation calculations. The cross-correlation values are calculated using (1) for example, as a function of spatial displacement variables: 8 𝛷𝐼𝐼(𝑥, 𝑦) = ∑ ∑[𝐼(𝑖, 𝑗)] ∗ [𝐼′(𝑖 + 𝑥, 𝑗 + 𝑦)]𝑛𝑗=1𝑚𝑖=1 (1) where 𝛷𝐼𝐼 is the cross-correlation value, (𝑥, 𝑦) are the spatial displacement variables in x and y directions respectively, and (𝑖, 𝑗) are the horizontal and vertical positions of the sub-window, with the respective window sizes denoted (𝑚, 𝑛). Equation (1) is evaluated through the summation of pixel intensities of the sub-window in the first frame 𝐼(𝑖, 𝑗) and in the second frame 𝐼′(𝑖 + 𝑥, 𝑗 + 𝑦). A peak value occurs when the features between the two windows match up, and is a direct estimate of their displacement (Atkins, 2016). Figure 2.3 plots the peak correlation at a displacement of 8 pixels in the x direction. Figure 2.3. Mapping of the correlation peak in the correlation plane. Figure from Atkins (2016). Copyright Elsevier (2016). Reproduced with permission. PIV has a high popularity for measurement of fluid flows in laboratories, but applications on natural flows were limited in comparison. There have been multiple studies on the 9 implementation of PIV technique on the surfaces of natural river flow, starting in the 1990s (Fujita & Komura, 1994; Fujita et al, 1997; Fujita et al, 1998) and subsequent refinements with field measurements and proof-of-concept experiments (Fujita & Aya, 2000; Muste et al, 2004). The term “Large Scale Particle Image Velocimetry” – LSPIV, culminated from their works on this type of measurements, as the measured flows are on a much larger scale than conventional PIV. It has been regarded as the benchmark for velocimetry methods (Khalid et al, 2019). Note in studies such as Hauet et al (2008b), PIV was used instead of LSPIV for the application of the latter. The general approach to LSPIV is essentially very similar to PIV, with some aspects adapted to field environments. First, natural lighting is used for illumination instead of laser light sheets. Second, visible features on the free surface induced by turbulence, such as ripples, foams, boils, and waves, replace seeding particles as visualization tracers for cross-correlation calculations. Substitutes for those tracers could also be reflections from incident sunlight interacting with the free surface deformations. However, as reported in Muste et al (2008), under unfavourable conditions when naturally-occurring surface features are lacking, artificial tracers need to be introduced to the flow to facilitate detection. They must accurately follow and represent local flow patterns. While the laboratory setting for PIV enjoys the benefit of producing orthogonal images ready for processing, it may not always be the case for LSPIV, as some videos are recorded from an oblique view. This requires one extra treatment, orthorectification, to be applied. Details are described in Section 2.2. For velocity estimation, LSPIV uses a different equation from (1) to calculate cross-correlation values (Fujita et al, 1998; Jodeau et al, 2022): 10 𝑅(𝑎𝑖,𝑗, 𝑏𝑖,𝑗) =∑ ∑ [(𝐴𝑖𝑗 − 𝐴𝑖𝑗̅̅ ̅̅ ) ∗ (𝐵𝑖𝑗 − 𝐵𝑖𝑗̅̅ ̅̅ )]𝑀𝑗𝑗=1𝑀𝑖𝑖=1[∑ ∑ (𝐴𝑖𝑗 − 𝐴𝑖𝑗̅̅ ̅̅ )2𝑀𝑗𝑗=1 ∗𝑀𝑖𝑖=1∑ ∑ (𝐵𝑖𝑗 − 𝐵𝑖𝑗̅̅ ̅̅ )2𝑀𝑗𝑗=1𝑀𝑖𝑖=1 ]12 (2) where 𝑅 is the cross-correlation coefficient, as a function of the location of interrogation area in the first frame 𝑎𝑖,𝑗 and in the second frame 𝑏𝑖,𝑗. 𝑀𝑖and 𝑀𝑗 are sizes of the interrogation area in pixels. 𝐴𝑖𝑗 and 𝐵𝑖𝑗 are the distribution of grayscale pixel intensities in the first and second windows. Figure 2.4 illustrates the process of dividing of an image into interrogation areas and determining all possible displacements within a larger search area. Both PIV and LSPIV employ this process in cross-correlation calculations. Figure 2.4. Conceptualization of the LSPIV image processing algorithm. Cross-correlation is calculated between the IAs P and Q or R in the search area. Figure from Muste et al (2008). Copyright John Wiley and Sons (2008). Reproduced with permission. 2.1.2 Optical flow Optical flow is used in computer vision to represent the perception of apparent motion of objects in a visual scene. It is defined as the distribution of velocities on movement of brightness patterns in an image (Horn & Schunck, 1981). Optical flow can arise from relative motion 11 between the sensor and the object as images are being captured on the retina or camera sensor. Using a sequence of image frames (from a video), optical flow is estimated as pixel displacements, which can then be calculated as velocities. Figure 2.5 illustrates the detection of optical flow in the retina (Figure 2.5a) through displacements of visual features, and in the image plane (Figure 2.5b) by corresponding movements to the pixel patterns. Figure 2.5. The production and detection of optical flow in the retina (a) and image plane (b). Figure from Raudies (2016). Reproduced under CC-BY 3.0 License. 12 For generic optical flow estimations on a 2D case, at a location (𝑥, 𝑦 ) of time 𝑡, image intensity 𝐼(𝑥, 𝑦, 𝑡), and movement ∆𝑥, ∆𝑦 between two frames spaced at ∆𝑡, the brightness constancy constraint is defined as: 𝐼(𝑥, 𝑦, 𝑡) = 𝐼(𝑥 + ∆𝑥, 𝑦 + ∆𝑦, 𝑡 + ∆𝑡) (3) assuming the movement is small, the constraint at 𝐼(𝑥, 𝑦, 𝑡) is expanded into a Taylor series: 𝐼(𝑥 + ∆𝑥, 𝑦 + ∆𝑦, 𝑡 + ∆𝑡) = 𝐼(𝑥, 𝑦, 𝑡) +𝜕𝐼𝜕𝑥∆𝑥 +𝜕𝐼𝜕𝑦∆𝑦 +𝜕𝐼𝜕𝑡∆𝑡 +12𝜕2𝐼𝜕𝑥2∆𝑥 +12𝜕2𝐼𝜕𝑦2∆𝑦 +12𝜕2𝐼𝜕𝑡2∆𝑡 + ⋯ (4) the higher-order terms are truncated from the series in (4), so only the first-order terms remain: 𝜕𝐼𝜕𝑥∆𝑥 +𝜕𝐼𝜕𝑦∆𝑦 +𝜕𝐼𝜕𝑡∆𝑡 = 0 (5) dividing (5) by ∆𝑡 converts ∆𝑥, ∆𝑦 into 𝑉𝑥, 𝑉𝑦: 𝜕𝐼𝜕𝑥𝑉𝑥 +𝜕𝐼𝜕𝑦𝑉𝑦 +𝜕𝐼𝜕𝑡= 0 (6) where 𝑉𝑥, 𝑉𝑦 are horizontal and vertical velocity components of 𝐼(𝑥, 𝑦, 𝑡), 𝜕𝐼𝜕𝑥,𝜕𝐼𝜕𝑦,𝜕𝐼𝜕𝑡 are first-order derivatives (or gradients) of pixel intensities at (𝑥, 𝑦, 𝑡). (6) is then with the gradients expressed as 𝐼𝑥, 𝐼𝑦, 𝐼𝑡: 𝐼𝑥𝑉𝑥 + 𝐼𝑦𝑉𝑦 = −𝐼𝑡 (7) combining the x and y components into a matrix produces: ∇𝐼 ∗ 𝑉 = −𝐼𝑡 (8) Equation (8) has two unknowns and cannot be solved without additional equations. This is also known as the aperture problem, which a contoured pattern moving behind an aperture with both ends obstructed, appears to move in a perpendicular direction to its own orientation. An example is the barber’s pole, with the perceived motion pointing upwards (y-direction) while 13 the helix of stripes rotate horizontally (x-direction). To solve this problem and estimate optical flow, additional conditions are required. Many methods on estimating optical flow were proposed during the 1980s, mainly falling under four categories: differential techniques (Lucas & Kanade, 1981; Horn & Schunck, 1981), region-based matching (Anadan, 1989), energy-based methods (Heeger, 1988), and phased-based techniques (Fleet et al, 1990). Barron, Fleet and Beauchemin experimented nine optical flow algorithms on the accuracy and density of 2D velocities, their study in 1994 showed that Lucas-Kanade and Fleet & Jepson methods to be the most reliable. The former is a first-order local differential method while the latter is a local phase-based method. Though optical flow’s underlying algorithms are not directly derived from fluid mechanics principles, it is adaptable to physical equations that can account for different fluid motions. Khalid et al (2019) applied a modified scalar transport equation in its own optical flow estimation algorithm, with a weighted diffusion term to compensate for small-scale non-observed parts of turbulence. Some existing algorithms have been used in detecting velocities in moving fluids, the methods with high prevalence are Lucas-Kanade, Horn-Schunck, and Farneback. The Lucas-Kanade method assumes the brightness is constant (per (3)), and the displacement of pixels between two frames is small and constant within a defined neighbourhood area, then all pixel intensities in the area are formulated as in (7) and solved using the least squares method. The resulting equation for velocity, on a 2x2 system, is below. [𝑉𝑥𝑉𝑦] = [∑ 𝐼𝑥(𝑞𝑖)2𝑖∑ 𝐼𝑥(𝑞𝑖)𝑖𝐼𝑦(𝑞𝑖)∑ 𝐼𝑥(𝑞𝑖)𝑖𝐼𝑦(𝑞𝑖) ∑ 𝐼𝑦(𝑞𝑖)2𝑖]−1[− ∑ 𝐼𝑥(𝑞𝑖)𝑖𝐼𝑡(𝑞𝑖)− ∑ 𝐼𝑦(𝑞𝑖)𝑖𝐼𝑡(𝑞𝑖)] (9) 14 The Horn-Schunck method also assumes the constancy of brightness as per (3), while introducing a global smoothness constraint equation to address the aperture problem in (8). ∫ ∫ [(𝐼𝑥𝑢 + 𝐼𝑦𝑣 + 𝐼𝑡)2+ 𝛼2(‖∇𝑢‖2 + ‖∇𝑣‖2)] 𝑑𝑥𝑑𝑦 (10) Where 𝛼 is the regularization constant. Velocities 𝑢, 𝑣 are solved by minimizing the value calculated in (10), through iterations. The Farneback method was proposed by Gunnar Farneback in his 2003 paper, by calculating local optical flow of a pixel between two frames through a second-order polynomial. A weight function for the points in a neighbourhood area, which is then minimized through iterations in a pyramid until convergence is reached: 𝑑(𝑋) = (∑ 𝜔𝐴𝑇𝐴)−1∑ 𝜔𝐴𝑇∆𝑏 (11) The detailed steps of Lucas-Kanade, Horn & Schunck, and Farneback methods are explained in the respective authors’ publications (Lucas & Kanade,1981; Horn & Schunck, 1981; Farnebäck, 2003). Wu et al (2019) applied the Farneback method on the surface ripples of the flow in a rectangular channel, under natural lighting. The setup was inclined towards laboratory settings, as the camera was fixed onto a tripod and pointed orthogonally to the water surface. Hutley et al (2023) incorporated this technique in a stereo computer vision gauging system on estimating the discharge of natural river flows in Ontario, Canada and several others in Australia. Horn & Schunck method was used in multiple studies conducted by Bung & Valero (2016a, 2016b, 2016c) on aerated flows and flow interactions at hydraulic structures, using air bubbles as seeding for detection. The experimental setup was illustrated by synchronized LED for constant lighting intensity. Videos were recorded with a high-speed camera on cross-sections 15 of the flow apparatus. Results compared to PIV indicated that optical flow produces dense velocity fields for every pixel, which is not possible in PIV without interpolation due to the use of block-based interrogation areas. The application of Lucas-Kanade method in fluid mechanics is more limited in quantity compared to the previous two. Zhang & Chanson (2018) experimented this method, along with the Farneback technique, in a stepped spillway model on complex air-water flow patterns. Rousseau (2019) developed the OpyFlow algorithm using this method to calculate displacement of trackers detected through Good Features to Track, testing results on image sequences indicated it is more accurate and efficient than PIV. Additional adaptations of Lucas-Kanade method are on constructing flow particle trajectories. An example is Optical Tracking Velocimetry (OTV), developed by Tauro et al (2018). It also uses Good Features to Track, with tracking done using pyramidal Lucas-Kanade method, starting from a lower resolution, through multiple levels, and propagating up to the resolution of the original image. The identified features are then matched between consecutive images in constructing trajectories, with spurious trajectories filtered out in post-processing. Shi et al (2020) compared the effectiveness of Lucas-Kanade, Horn & Schunck, and Farneback techniques on the velocities from breaking surges in an experimental setting. It reported Farneback method the most suitable for producing smooth ensemble-averaged 2D velocity fields in air-entrained flow regions. Meanwhile, Horn & Schunck method had larger regions of discontinuity in its velocity fields due to changes in the brightness intensity, suggesting its high sensitivity to changes in lighting conditions. Although natural flows on a larger scale may not exhibit the same amount of deformation as breaking surges (comparable to hydraulic jumps) or air-entrained flows, natural lighting 16 intensities for those measurements are likely to be more locally variable than the previously mentioned studies. Also, consistency of lighting between videos, even if they are recorded at the same location in different times or weather conditions, are almost impossible to maintain. Therefore, Farneback method appears to be more reliable for applications in a natural environment, producing dense velocity fields. 2.1.3 Eulerian vs Lagrangian frames of reference Eulerian and Lagrangian are two frames of reference commonly used in fluid dynamics. In the Eulerian frame of reference, the control volume is sized infinitely small at (𝑑𝑥, 𝑑𝑦, 𝑑𝑧) with the position fixed relative to the viewer’s coordinates, as (𝑥𝑜 , 𝑦𝑜 , 𝑧𝑜). In the Lagrangian frame, however, the control volume is an infinitely small fluid particle, which is tracked along the motion of the fluid with the same number of particles between points a and b. PIV or LSPIV, as well as Farneback method, both use Eulerian frame of reference to estimate flow velocities. Zhang & Chanson (2018) noted the latter being similar to LSPIV in nature. They estimate the velocities for either the same block or pixel between two frames, on the entire flow area covered in the frames’ field of view. A key assumption for Eulerian flow equations is steady flow, i.e., the change in flow rate over time is zero. Lagrangian method can be found in applications using the Lucas-Kanade technique (Tauro et al, 2018). Some methods are explained in the next sub-section. However, opyf, an adaptation of Lucas-Kanade, appears to be Eulerian, as velocity fields, not particle trajectories, are calculated. 2.1.4 Other methods Apart from PIV/LSPIV and optical flow, there exist other methods which are briefly explained in this separate sub-section. 17 Particle Tracking Velocimetry (PTV) is the Lagrangian counterpart to PIV, utilizing similar setups and the statistical technique for calculating cross-correlation coefficients. But it determines the velocity vector of every particle, and builds the trajectories between subsequent frames (Dabiri & Pecora, 2019). Figure 2.6 illustrates the comparison between PIV (A) and PTV (B). For tracers to be effectively detected and tracked, they must be in uniform shape, which limits the application in field environments where only artificial tracers could satisfy that requirement. Despite being sparsely tested than LSPIV, Tauro et al (2017) conducted comparisons between the LSPIV and PTV, using a modified PTV procedure to filter out nonrealistic particle trajectories. It reported the potential for PTV to mitigate issues in LSPIV such as non-uniform distribution or cluttering of tracers, unsteady flows, and poorly seeded surfaces. Experiments conducted on 12 rivers in Italy showed that PTV results were closer to benchmark data while LSPIV resulted in underestimates. Figure 2.6. PIV in Eulerian (A) and PTV in Lagrangian (B) frames of reference. Figure from Jolley et al (2021). Reproduced under CC-BY 4.0 license. Perks (2020) developed KLT-IV, a particle tracking technique using Good Features to Track in detecting features, and subsequent tracking using the Kanade-Lucas-Tomasi method. It also involves a pyramid refinement scheme mentioned in Tauro et al (2018). 18 Fujita et al (2007) developed a different technique, named Space Time Image Velocimetry (STIV). Space time images (STI’s) are generated on a local area of the image, with surface ripples on the water as changes in image brightness get detected in line segments. Orientation angles and displacements are calculated from the changes in pixel brightness patterns using eigenvalue analysis. This method can be directly applied to oblique images though input of different image scales to the different line segments for actual surface velocities. 2.2 Currently established workflow This section summarizes the currently established steps in implementing image velocimetry techniques on natural flow measurements, culminated from the recent two decades of existing studies. They include video collection platforms, site setup and video sampling, steps of the processing workflow, and orthorectification modes. 2.2.1 Video collection platforms Early applications on image velocimetry had videos recorded from cameras attached to fixed posts (Hauet et al, 2008b). Such an arrangement had also been used in more recent studies (Ran et al, 2016; Stumpf et al, 2016, Carrel et al, 2021) for long-term flow monitoring. This method is advantageous in minimizing camera movement, also allowing the site area in its field of view to be more accurately calibrated during low flows. The camera parameters are then able to be used for a longer time period, including peak flows when the area becomes inaccessible. Usually, the cameras are set up at an oblique angle, thus orthorectification is required through registered ground control points (GCPs) (Muste et al, 2008). However, in Tauro et al (2014), both the camera and two lasers as GCPs were fixed to a mast, which was then installed to a bridge. Both the cameras and lasers were pointing to the river surface in nadir perspective (i.e. 19 orthogonal), minimizing errors from perspective distortion and surveying. The details of orthorectification are elaborated in Section 2.2.2. Mobile collection platforms on the ground started from cameras installed to masts on vehicles (Kim et al, 2008), whose video collection process was similar to fixed installations, as the platform was also stationary during the recordings and GCPs were measured and calibrated. This evolved into video collection using handheld devices, including smartphones. Both fixed and mobile platforms have the ability to be processed onsite, especially remotely, minimizing maintenance work. An example of the latter is the DischargeApp on Android phones (Carrel et al, 2019). Though being cost-effective, as Winsemius et al (2022) noted, mobile devices produce more imprecision on videos and consequently require additional steps in processing. Another type of mobile platform is aerial vehicles, which are especially convenient as they can cover both river banks in its field of view with relative ease, plus they can operate in remote environments, where access is limited. Legleiter & Kinzel (2020) demonstrated this by using high-resolution cameras installed on helicopters to measure sediment-laden flows in Alaska. Unmanned aerial vehicles (UAVs), or drones, are being increasingly used for video collection under many different applications involving surface velocity and/or discharge (Eltner et al, 2020; Perks et al, 2020; Strelnikova et al, 2020; Han et al, 2021; Liu et al, 2021). Videos recorded on these platforms are more commonly in nadir perspective. Apart from conventional video cameras that operate under visible light, infrared or thermal cameras have been tested in data collection. Schweitzer & Cowen (2021) applied LSPIV (named by the authors as QIV – Quantitative Image Velocity) on videos from infrared cameras recorded in field experimental setups. Since those cameras operate on different wavelengths, they have less dependence on illumination and seeding and could operate at night. 20 2.2.2 Site setup and video sampling This subsection covers the factors in video data collection that have an influence on the quality of measurements. Before video data on riverflow is collected, one or more suitable site(s) should be selected. For image velocimetry to perform effectively for measurements, full width of the river should be visible and the flow surface should have no obstructions, such as vegetation and taglines for other methods (Detert, 2021). The collection platform should remain steady during video collection to minimize the influence shaking on the measurements, through fastening cameras on a post or keeping the drone steady in flight (ABoM, 2021). Cross-sections within the frames are required for discharge calculations. Cameras should also be intrinsically calibrated onsite to remove lens distortions at similar conditions to the measurements (Detert, 2021). As video collections take place in the field, environmental conditions onsite can affect image quality, and they could not be easily controlled. They mainly consist of glare and shadows affecting the visibility of natural tracers, wind affecting the flow’s surface velocity and magnitude (Peña-Haro et al, 2020), and precipitation blurring the images. The impact of those conditions on surface velocity is difficult to predict (Jolley et al, 2021) and likely to be site-specific. Current recommendations such as Detert (2021) and ABoM (2021) emphasized the importance of users in minimizing the effect of disadvantageous conditions. The perspective of which the videos are recorded from is critical in transformations. ABoM recommended nadir perspective for drone videos as this only requires pixel scaling across the entire image. For oblique videos, the device should be at a sufficient elevation from the water surface in order to minimize perspective distortion. A minimum critical angle of 15 degrees between the camera and farthest search area is needed for LSPIV and 2 degrees to the farthest 21 search line for STIV. Angles less than those could introduce errors caused by excessive distortion. The reader is referred to Figure 9 in the ABoM guide for an illustration of the critical tilt angle on a camera installed at a river bank. The setup and distribution of GCPs onsite have a significant impact on the accuracy of orthorectification. Bodart et al (2022) highlighted errors introduced by poor spatial distribution of GCPs. The points should be distributed within the area of interest (Detert, 2021), with additional points marked and surveyed for better reliability of the results. Simulation results from Le Coz et al (2021) showed increasing GCPs from 3 to 6 significantly reduced the relative uncertainty in discharge estimates caused by parametric errors from orthorectification. The minimum required number of GCPs are explained in the Section 2.2.3. Video sampling is another important aspect in controlling data quality, including image resolution, frame rate, and video length. Image resolution is represented through the size of the frames (e.g., HD is usually 1920*1080 pixels) and it affects tracer quality and computational time. Decreasing the resolution can increase the processing speed, an advantage for expedited onsite processing. Leitão et al (2018) showed a decrease from 1280*720 pixels to 256*144 pixels had little effect on surface velocity measurements, only further decreases started to have more significant impacts. Le Boursicaud et al (2016) had similar findings from YouTube videos as decreasing the resolution from 1080 pixels to 240 pixels resulted in less than 5% difference over 50 frames. On the other hand, higher resolutions, such as 4K, allow more detailed visualization of tracers but has the risk of burdening the computer hardware (Jolley et al, 2021). Adjusting the frame rate, however, introduces more errors to surface velocity than frame sizes. Leitão et al (2018) found that decreasing the rate from the original 27 fps to 25 or 20 increased image noise, while the results became unusable when fps was decreased to 15 or 10. A 22 frame rate at 30 fps should be sufficient (Fujita et al, 2013). Detert (2021) noted it is more important to maintain a consistent frame rate by checking the frames’ timestamps. Changing the length of videos affects the quality of time-averaged velocity results as decreasing the length is detrimental to the accuracy of estimates. Dramais et al (2011) demonstrated significant errors in discharge when only 10 or less frames were processed, and the variations decreased to 1% at 100 frames. 2.2.3 Processing Images are processed for velocity and discharge through the following steps of a generic workflow in sequential order: camera calibration, stabilization, orthorectification, pre-processing, image velocimetry analysis, post-processing, and discharge calculation. These steps are shown in Figure 2.7. Camera calibration is the process to remove lens distortion in the images and register pixels to real-world distances, so the cameras are able to be used as measurement instruments Detert (2021). In the case of common video camera lens, barrel distortion is removed. There are two types of calibration: intrinsic and extrinsic. Intrinsic calibration is performed by taking photos on pre-defined patterns at different angles, and extrinsic calibration is performed by registering objects in the images to real-world coordinates, similar to orthorectification. Stabilization is aimed to remove movements, shaking, and/or vibrations from the cameras, so in theory the fixed objects remain fixed and the only movements are from the river surface. Though it is possible to achieve this through mechanical means by fixing the platforms themselves during video collection, it could only work on low intensities and digital image stabilization (DIS) is commonly used for the higher ones (Ljubičić et al, 2021). The latter is achieved by detecting static features in the frames, then estimating their displacements through 23 feature matching. Correction is then performed using similarity, affine, or projective transformation. Only the river bank areas are used for feature detection (Jodeau et al, 2022) as the flow area introduces errors from its own movement. Perks et al (2016) used a different approach, as the image movement from a drone was estimated by registering and tracking fixed control points, then the vectors deducted from raw results. It was applied on KLT-IV to produce trajectories, and not seen in block-based counterparts (i.e., LSPIV). Figure 2.7. Steps of a generic processing workflow. Redrawn based on Jolley et al (2021). Orthorectification is to correct distortions caused by the landscape and camera tilt angles, producing orthogonal (i.e. plan view) images fit for mapping. This is critical for oblique images 24 as they are taken at an angle, and thus the pixel distances within the images are unequal. It is, however, unnecessary for images under nadir perspective as the camera is pointing directly towards the surface at 90 degrees. In that case, only scaling between the image and world distances is needed, which is achieved by defining two points on an image then input the real-life distance between them. Full orthorectification can be carried out in two modes: 3D and 2D, both requiring ground control points (GCPs) defined in the images and their real-world coordinates as inputs. Under 3D orthorectification, the pixel projection equations are expressed as below: 𝑖 =𝐶11𝑥 + 𝐶12𝑦 + 𝐶13𝑧 + 𝐶14𝐶31𝑥 + 𝐶32𝑦 + 𝐶33𝑧 + 𝐶34 (12) 𝑗 =𝐶21𝑥 + 𝐶22𝑦 + 𝐶23𝑧 + 𝐶24𝐶31𝑥 + 𝐶32𝑦 + 𝐶33𝑧 + 𝐶34 (13) where (𝑖, 𝑗) are the GCP’s image coordinates in pixels, and (𝑥, 𝑦, 𝑧) are their real-world coordinates (usually in meters). The coefficients 𝐶11 … 𝐶34 are elements of a 3*4 projection matrix, of which 𝐶34 = 1. There are 11 unknowns, requiring at least 6 GCPs (Jodeau et al, 2008) to form a system of linear equations, which is solved through least squares method. The projection matrix, also known as a camera matrix (Patalano et al, 2017), transforms world coordinates (𝑥, 𝑦, 𝑧) into image coordinates (𝑖, 𝑗) as below: [𝐶11 𝐶12 𝐶13 𝐶14𝐶21 𝐶22 𝐶23 𝐶24𝐶31 𝐶32 𝐶33 𝐶34] ∗ [𝑥𝑦𝑧1] = [𝑖̃𝑗̃?̃?] (14) (𝑖, 𝑗) = (𝑖̃?̃?,𝑗̃?̃?) (15) where (𝑖̃, 𝑗̃, ?̃?) are intermediate coordinates used to calculate the image coordinates. 25 This projection only transforms in one way, and cannot be reversed as the matrix’s inverse is not found. To perform inverse mapping, the GCP elevations can be expressed as: 𝑧 = 𝑏1𝑥 + 𝑏2𝑦 + 𝑏3 (16) provided the river surface is considered as planar (Jodeau et al, 2008). The coefficients 𝑏1, 𝑏2, 𝑏3 are solved as a system of bivariate linear equations. Assuming the surface slope is small for the section of river under study, the previous equation is reduced to: 𝑧 = 𝑏3 (17) as the difference between the GCP plane to the actual water level. It is then applied into Equations (12) and (13) to get: 𝑖 =𝐶11𝑥 + 𝐶12𝑦 + 𝐶13𝑏3 + 𝐶14𝐶31𝑥 + 𝐶32𝑦 + 𝐶33𝑏3 + 𝐶34 (18) 𝑗 =𝐶21𝑥 + 𝐶22𝑦 + 𝐶23𝑏3 + 𝐶24𝐶31𝑥 + 𝐶32𝑦 + 𝐶33𝑏3 + 𝐶34 (19) They essentially correct the image points to the same elevation plane, which is solved for inverse mapping of image coordinates to real-life Cartesian coordinates through the equations below: 𝑥 =𝐻11𝑖 + 𝐻12𝑗 + 𝐻13𝐻31𝑖 + 𝐻32 + 𝐻33 (20) 𝑦 =𝐻21𝑖 + 𝐻22𝑗 + 𝐻23𝐻31𝑖 + 𝐻32 + 𝐻33 (21) where the coefficients 𝐻11 … 𝐻33 are elements of a 3*3 projection matrix. The reader is referred to Jodeau et al (2008) for the calculation of those coefficients. For equations (18) and (19), the flow’s slope must be mild for the elevation reduction to work, as the GCPs’ elevation plane is parallel to the surface plane. Therefore, this technique is not applicable to steep slopes because 26 the elevation plane will be easily out of alignment with the surface slope and consequently introduce errors. Dramais et al (2011) noted the importance of water levels in generating accurate orthophotos. Decreasing water levels result in increasing pixel scales, and subsequently increasing surface velocities and decreasing wetted areas. The opposite is also true for increasing water levels. Under 2D orthorectification, the GCPs are at the same plane as the water surface, removing the elevations as inputs. Equations (12) and (13) are reduced to: 𝑖 =𝐻11𝑥 + 𝐻12𝑦 + 𝐻13𝐻31𝑥 + 𝐻32𝑦 + 𝐻33 (22) 𝑗 =𝐻21𝑥 + 𝐻22𝑦 + 𝐻23𝐻31𝑥 + 𝐻32𝑦 + 𝐻33 (23) The coefficients form a 3*3 2D projection matrix (AKA homography matrix; Khan et al 2014; Patalano et al, 2017), calculated through direct linear transformation. It transforms real-life coordinates to image coordinates as below: [𝐻11 𝐻12 𝐻13𝐻21 𝐻22 𝐻23𝐻31 𝐻32 𝐻33] ∗ [𝑥𝑦1] = [𝑖𝑗1] (24) where 𝐻33 = 1. This is similar to equation (14), but it can be inversed to transform image to world coordinates as in equations (20) and (21), which cannot be performed directly under the 3D technique. There are 8 unknowns, requiring at least 4 GCPs (Fujita et al, 1998) to solve under a system of linear equations. The calculated transformation matrix can be used in either orthorectifying the video frames or the velocities vectors (Hauet et al, 2008a), the details are elaborated in Section 2.2.4. 27 Pre-processing, in this case, is defined as increasing the visibility of surface tracers before image processing for surface velocity. Jolley et al (2021) suggested a minimum step of isolating out the flow area as a region of interest (ROI), and applying contrast enhancement to the image. The former is to minimize the amount of image noise induced from the redundant pixel intensities. If such step is insufficient, additional methods could be used, such as Gaussian filters to remove image noise or binarization to isolate the surface tracers. One of the commonly used enhancement methods for image velocimetry is Contrast-Limited Adaptive Histogram Equalization (CLAHE). When it is applied to the grayscale images, it enhances their contrast using small rectangular areas of the image, named as tiles. The tiles’ intensity values are distributed on a histogram, then interpolated using bilinear interpolation, with a clip limit to reduce amplification of contrast (Zuiderveld, 1994). The number of titles per side is defined as the total number of pixels of the image length or width divided by a defined block size in pixels. This way, all tiles in the frames should be square at the aforementioned block size. For example, if the block size is specified to be 8px, then the tiles are sizes at 64px^2. The image processing step is when the frames are processed through the algorithms in section 2.1, producing velocity fields or trajectories. Analysis parameters applied in this step will affect the resolution and quality of the surface velocity field. This is elaborated in Section 2.4. Post-processing is the correction or reduction of erroneous outputs, usually achieved through statistical methods. For LSPIV, velocity vectors are compared against surrounding neighbours, with outliers filtered detected through different types of statistical calculations or thresholds (Jolley et al, 2021). Spurious vectors can then be removed and replaced with interpolated data. 28 Discharge calculation is the most common application of image velocity methods, applying the surface velocities from LSPIV/LSPTV/STIV etc. to velocity-area methods: mid-section or mean-section. The general concept is illustrated in Figure Figure 2.8. Two critical inputs are required: water surface elevation (WSE) to calculate depths, and mean velocity coefficient to convert to depth-averaged velocities. As image methods only generate velocities, WSEs are usually measured separately through monitoring stations or staff gauges (Jodeau et al, 2008; Kim et al, 2008; Le Coz et al, 2010; Bomhof, 2022). Figure 2.8. LSPIV-based discharge measurement procedure. Figure from Muste et al (2008). Copyright John Wiley and Sons (2008). Reproduced with permission. The mean velocity coefficient, denoted as alpha (α), is a conversion from surface velocity to depth-averaged velocity, calculated as below: 𝛼 =𝑉𝑑𝑒𝑝𝑡ℎ−𝑎𝑣𝑒𝑟𝑎𝑔𝑒𝑑𝑉𝑠𝑢𝑟𝑓𝑎𝑐𝑒 (25) 29 As the velocity profile over depth follows a log or power curve, simply using surface velocity will introduce erroneous discharge results. The commonly used value for α is 0.85 (Le Coz et al, 2010). For different river depths in natural beds, Hauet et al (2018) concluded from 3611 measurements on 176 sites in France, concluded with a rule of thumb: α is 0.8 for depths of less than 2 meters, and 0.9 for 2 meters or higher. The relationship between α and other factors, such as the river’s aspect ratio or Manning’s roughness coefficient, was not clear. There are cases where α is more than 1.0 due to wind forcing (Bomhof, 2022) or flows upstream of weirs (Bodart et al, 2022), requiring onsite measurements for more accurate results. 2.2.4 Orthorectification types There are two prevailing types which the transformation matrix is applied in orthorectification as named in Jolley et al (2021): explicit and implicit. Explicit is directly to the video frames before image analysis, and implicit to the velocity vectors after image analysis. They are therefore respectively named “pre-analysis” and “post-analysis” in this thesis. “Pre-analysis” orthorectification mode is to orthorectify the video frames, then applying orthophotos under the velocimetry algorithms to calculate surface velocity. It is the conventional mode explained in section 2.2.3, generating a consistent pixel scale before image processing (Perks et al, 2016; Le Coz et al, 2021), easier for parameters to be quantified in real-life distances. However, it has the potential of exaggerating errors from spurious tracers as those are transformed before processing (Highham & Brevis, 2019), as well as requiring a lengthy computational process to orthorectify all the frames (Patalano et al, 2017). An example of “pre-analysis” mode in freely available LSPIV software is Fudaa-LSPIV, with its current version at 1.9.2 in May 2022 (Jodeau et al, 2022). It operates in Java, with a built-in workflow including 30 stabilization and orthorectification of video frames before applying LSPIV algorithm onto the orthoimages for surface velocity. “Post-analysis” orthorectification, on the other hand, has the original oblique video frames processed for raw velocities, then orthorectified from image space to real-world space. This process treats the raw velocity vectors as displacements in image coordinates. Mesh grids of the original image coordinates are generated, then a new set of coordinates are calculated by adding the velocity vectors in: 𝑖𝑎,𝑏1 = 𝑖𝑎,𝑏0 + 𝑈 (26) 𝑗𝑎,𝑏1 = 𝑗𝑎,𝑏0 + 𝑉 (27) where (𝑖𝑎,𝑏0 , 𝑗𝑎,𝑏0 ) are the original image coordinates (in pixels) at position (𝑎, 𝑏) in the mesh grid matrix, and (𝑖𝑎,𝑏1 , 𝑗𝑎,𝑏1 ) are the new image coordinates at the same position. 𝑈 and 𝑉 are horizontal and vertical raw velocities in the image coordinate system. As PIV results are based on interrogation areas, the vector matrices are in different sizes to the images, thus requiring a mesh grid for plotting. Both mesh grids are orthorectified using equation (24), then subtracted for displacements in world coordinates ∆𝑥 and ∆𝑦, which are the velocity vectors in real-world coordinate system (Patalano et al, 2017): ∆𝑥 = 𝑥𝑐,𝑑1 − 𝑥𝑐,𝑑0 (28) ∆𝑦 = 𝑦𝑐,𝑑1 − 𝑦𝑐,𝑑0 (29) where (𝑥𝑐,𝑑0 , 𝑦𝑐,𝑑0 ) and (𝑥𝑐,𝑑1 , 𝑦𝑐,𝑑1 ) are the original and new real-world coordinates after orthorectification, at position (𝑐, 𝑑) in the mesh grid matrix. Note the different notation to the previous, as the mesh grids also changed in size after orthorectification. 31 The real-life vectors are then mapped out onto an orthorectified video frame as a background and multiplied with the frame rate: 𝑝𝑖𝑥𝑒𝑙𝑓𝑟𝑎𝑚𝑒→𝑚𝑒𝑡𝑒𝑟𝑓𝑟𝑎𝑚𝑒∗𝑓𝑟𝑎𝑚𝑒𝑠𝑒𝑐𝑜𝑛𝑑=𝑚𝑒𝑡𝑒𝑟𝑠𝑒𝑐𝑜𝑛𝑑 (30) RIVeR (Rectification of Image Velocity Results; Patalano et al, 2017) is the “post-analysis” counterpart to Fudaa-LSPIV, applying orthorectification to raw image vectors from PIV or PTV. For PIV analysis, PIVlab (Thielicke & Stamhuis, 2014) is used, with the ability to apply pre-processing and post-processing features. The latest version is 2.62 published on April 17, 2023. From version 2.60 (published November 9, 2022) onwards, the code has been optimized to increase processing speed by up to a factor of 3 (ref). If the videos need to be corrected for lens distortion and movement, they could be processed in RIVeR before loaded into PIVlab. The latest version, 2.6, was released in October 2022 (Patalano, 2022). Schweitzer & Cowen (2022) applied this mode independently. For optical flow, the same process has already been applied to videos on walking pedestrians (Khan et al, 2014). Though the rectification on optical flow results to be included in future updates of RIVeR (Patalano et al, 2017), such feature has not been implemented or available to date, and therefore poses an obstacle for applications of “post-analysis” rectification on Farneback results. A potential conversion to world coordinate systems without orthorectification is the use of variable scaling in the images. Points are registered in the images at different distances to the camera, then their respective scales are calculated. Experimented in parking lots (Khan et al, 2014), it was discovered to be more sensitive to ground truth data than homography-based 32 orthorectification, especially for objects farther from the camera. This should be more difficult for riverine environments as it is more difficult to identify parallel lines than parking lots. 2.3 Application of image velocimetry in extreme events Extreme flows have high information content from a hydrological perspective (Fannin et al, 2021), as they cause shifts and subsequent adjustments in the river’s rating curves. Also, parameter estimation for hydrological streamflow forecasting models and extreme value distributions is very sensitive to the highest flows. Therefore, estimating the magnitude of the peak of extreme flows is of great importance. On the other hand, the dangerous conditions posed by floods render conventional stream-gauging techniques impossible to be conducted, with existing monitoring stations damaged or even destroyed by the fast, sediment-laden flows. Conventional measurements of peaks are performed post-flood, using the slope-area method or hydraulic modelling. Such circumstances necessitate the use of non-intrusive gauging techniques, including image velocimetry. This section summarizes existing applications of image velocimetry in extreme events, including citizen videos as a source of data in crowdsourcing campaigns. The following are reviewed: challenges of citizen videos, adaptations for post-event studies, as well as accuracy and uncertainties of image velocimetry methods. 2.3.1 Previous cases Extreme flows have been captured from both fixed and mobile platforms, then processed with image velocimetry. For fixed platforms, as elaborated in section 2.2.1, were attached to posts or masts, and usually calibrated before the high flows, ensuring higher degree of accuracy. Jodeau et al (2008) recorded high flows on a river downstream of a dam spillway and processed the videos with LSPIV. Tsubaki et al (2011) captured high flows from a CCTV camera installed 33 by the river bank. Al-mamari et al (2019) captured videos on a river flash flood in a wadi area and processed with both LSPIV and STIV. Winsemius et al (2022) captured parts of the Summer 2021 flooding in the Netherlands and processed in OpenRiverCam, a LSPIV-based software. Jamieson et al (2023) captured high flows in Nova Scotia in Canada caused by Hurricane Fiona in September 2022, and processed them mainly using STIV. Aerial mobile platforms mainly consist of helicopters and drones. Studies by Fujita & Kunita (2011) and Fujita et al (2018) used flooding footage on rivers or dam breaches recorded from helicopters. Perks et al (2016) processed drone videos in KLT-IV for surface velocity trajectories, noting the drones’ advantage in rapid deployment on flash floods, without requiring prior calibration. Handheld devices, including smartphone cameras, have become an emerging source for video data collection. The advent of camera technology means better devices are getting more ubiquitous among the general population. Local citizens witnessing floods can take videos with their phones, and share them as evidence in crowdsourcing campaigns, notably for flood inundation mapping in Dresden, Germany (Fohringer et al, 2015) and in Christchurch, New Zealand (Le Coz et al, 2016). As more of these videos are becoming available on social media and video-sharing platforms, it will be easier to preserve and share evidence of the event, from which important information could be extracted for post-event analysis. This is especially useful when no hydrographers could be present onsite, or unable to take measurements without being endangered (Le Coz et al, 2016). Some citizen videos were able to be processed through image velocimetry for post-flood surveys (Le Coz et al, 2015; 2016), and not limited to smartphones. An earlier application on an urban flash flood in Japan used footage taped by a TV cameraman using a news camera (Fujita et 34 al, 2013). Later notable examples including a YouTube video on flash flooding at an ungauged mountainous torrent in France (Le Boursicaud et al, 2016; in Figure 2.9) and a smartphone recording a flash flooding at a street intersection in Argentina (Guillén et al, 2017). The increasing accessibility of drones to citizens, thanks to their cheaper prices, also allows them to record videos with drones, such as in an urban flooding case (Perks et al, 2016) and the case in France in Le Coz et al (2016). Compared to phones with their easy access to citizens, drones are able to cover most areas with relative ease. Also, if the dry riverbed is filmed in the video (i.e., before a flash flood), such as in Figure 2.9a, that particular frame can be used as reference to stabilize subsequent frames and post-event surveying of GCPs with higher accuracy and relatively ease. Figure 2.9. Frames extracted from the YouTube video analyzed in Le Boursicaud et al (2016), showing before the flood (a) and the pulsed flow during the flood (b-d). Video source: Modesto (2011). 35 Though cases of processing under LSPIV and STIV are relatively abundant, few extreme cases are processed using optical flow methods, especially with Farneback, other than testing cases under milder flows (Yagi et al, 2020; Hutley et al, 2023; Ansari et al, 2023). 2.3.2 Challenges with citizen videos As the witnesses usually record flooding videos without training and under non-ideal conditions, it should be emphasized that those videos present significant challenges to processing than those from conventional setups. Le Boursicaud et al (2016) summarized a number of major points, listed below. First is the ensuring the visibility of both river banks in the camera’s field of view, and relatively planar water surfaces for LSPIV analyses. Second is the lack of camera parameters for lens calibration plus abundance of shaking. However, the shaking may not be as significant with increasing length of video clips, as the averaged magnitude converges closer to zero, provided the camera’s position remains unchanged. Le Coz et al (2016) noted continuous movement of cameras in some citizen videos, rendering them unusable as no GCPs could be identified. Third is the need to estimate water level directly from the image, instead of using data from monitoring stations as explained in section 2.2.3. This is due to the stations likely to be damaged during flooding. Fourth is the change in hydraulic conditions, including mean velocity coefficient α, due to river morphology during the event. Additionally, such videos are usually recorded from an oblique perspective with more perspective distortion. Orthorectifying them introduces additional errors from georeferencing. 36 Though the authors highlighted the lack of visible GCPs being detrimental to analyses, attempts on such cases, which are more likely to occur in natural channels, are close to none. 2.3.3 Adaptations for post-event studies If site areas have been set up and calibrated prior to the extreme events, then routine corrections and measurements on cross-sections should be measured post-event to update on any changes, as performed in a long-term monitoring case in Stumpf et al (2016). However, if sites have not been calibrated before, especially the ones covered in citizen videos, such measurements can only be performed post-event. The considerations in section 2.2.2 thus become largely unapplicable, but some are, instead, useful as selection criteria on videos suitable for analysis. An important example mentioned in Fujita et al (2013) is the selection of videos with relatively stable frames. Post event measurements mainly involve the collection of data on lens distortion, GCP coordinates, and transect profiles. Le Boursicaud et al (2016) identified GCPs in the images, then surveyed them with a total station. Some of them were located in the torrent channel, which could be identified thanks to the pre-flood part of the video, as the channel bed was entirely visible. Other examples with handheld devices (Fujita et al, 2013; Guillén et al, 2017) implemented a similar process to identify and survey visible GCPs in the images. Environmental conditions during the event might be difficult to quantify without onsite real-time weather data, which could especially affect the estimation of velocity coefficients. Dramais et al (2011) noted that wind likely does not have high influence on fast flows during floods. Water level data during extreme events could be unavailable, as they could be damaged or destroyed. Direct estimation of WSE’s from images is possible by calculating the pixel scale 37 of two known points near the river bank for their elevations, then extrapolating it to the water surface (Le Boursicaud et al, 2015). Stumpf et al (2016) incorporated a digital elevation model into the camera’s field of view, along with an automated waterline estimation algorithm, was able to estimate the water levels automatically, albeit with issues of underestimating under unfavourable lighting conditions. As observed from those examples, many focused on artificial channels or urban streets with uniform cross-sections, very few were on natural flow channels with changing riverbeds during flooding. The cases also implemented orthorectification with GCP coordinates already measured. Under extreme cases, such coordinates could be unavailable due to limited access to the areas, let alone the lack of identifiable GCPs in natural areas. Though Le Boursicaud et al (2016) remarked those more difficult conditions should be tested, such examples are almost unavailable, and the according adjustments to the processing workflow are therefore unknown. 2.3.4 Accuracy and uncertainties The numerous applications of LSPIV have provided information on the accuracy and uncertainties on the measurements of this method. A study on existing application cases in Muste et al (2008) summarized the error on velocity is at an average of 10% and a maximum of 35%, with errors in discharge are less, at a range from 1% to 6%, thanks to the inherent spatial averaging involved in velocity-area methods. Le Coz et al (2010) compared LSPIV results to those at a gauging station, and estimated the uncertainty to be less than 10% for velocity and 20% for discharge, with good videos. Poorer quality of videos could increase the uncertainty in discharge to a maximum of 30-80%. Dramais et al (2011) concluded that LSPIV measurements on discharge is up to 10% under favourable flooding conditions, which is considered accurate for conventional velocity-area methods. 38 As reported by Muste et al (2008), errors are generated in all stages of the processing. The authors recommended using more accurate means of surveying, including Real Time Kinematic (RTK) GPS receivers, than total stations, as accuracy of orthorectification is directly related to surveying accuracy. It should be noted that post-event surveys on the flooding coverage area are usually deemed undesirable (Tauro et al, 2017) due to the higher amount of uncertainties. For accuracy and uncertainties of optical flow measurements, Hutley et al (2023) produced raw (i.e. before being applied to a learning model) Farneback discharge measurements at a maximum 6% error under normal flows. Yagi et al (2020) reported discharge measurement errors of DeepFlow at an overestimate of 2.25%, while LSPIV was at an underestimate of 1.61%. As few cases were processed with Farneback, its uncertainty in velocities and discharges are unknown. 2.4 Tuning analysis parameters This section covers existing work on tuning analysis parameters for LSPIV and optical flow, consisting of the behaviour of LSPIV results under different interrogation area sizes (Section 2.1.1) and pre-processing parameters, along with strategies in optimizing IA sizes for surface velocity measurements. Similar examples on optical flow are also included. 2.4.1 Existing tests with analysis parameters Under a more sophisticated setup, adjusting the interrogation area for conventional PIV relies more on the size of seeding particles to ascertain a certain number of them are contained within the IA, as well as the inter-frame displacement. For LSPIV in the field, the task becomes more complicated. Illumination and seeding conditions are key controls to LSPIV accuracy, and experimental knowledge on those, along with velocity, are critical in determining the sizes 39 (Tauro et al, 2017). The Austrian guideline recommended IA sizes to be “large enough to contain tracer/particle patterns, but small enough to be representative of the local flow and tracer movement”. The values for these two thresholds, however, require case-specific experimentation. Though decreasing interrogation area sizes increases the resolution of results, as more vectors are calculated for smaller subdivisions of the image, this action is detrimental to the quality of the results under normal flow conditions. In Legleiter & Kinzel (2020), the interrogation areas for helicopter videos on sediment-laden flow ranged were 2.30m, 6.4m, 12.8m and 25.6m, resulted from the same interrogation area of 128 pixels but under different pixel sizes: 0.018m, 0.05m, 0.1m and 0.2m respectively. At the smallest IA size, the velocities contained the most amount of error, attributed to image noise and relatively smaller size than the surface features. As the size increased, the velocities increased in accuracy, albeit at larger spacings, with the largest IA size producing results too wide spread to be used for further applications. Similar results have also been reported in Liu et al (2021) using drone videos on normal and low flow conditions. As interrogation area decreased from 16px to 8px, decreasing the number of features, and consequently increasing errors in cross-correlation and measurement. When different heights of drone flight were tested, the maximum of 112m had the most errors due to low image resolution. The authors recommended IA sizing at 16px given its highest accuracy. This value may appear to be widely applicable, but the authors did recommend caution due to site-specific conditions. Pearce et al (2020) evaluated multiple methods also using drone videos on low flow, with artificial surface tracers. They concluded that LSPIV is very sensitive to changes in IA size and 40 feature extraction rate, with the smallest IA size at 32px exhibiting higher variations than 64px and 128px. KLT and OTV exhibited the least amount of variance. Seeding density for this example was high, while changes in density on the previous one resulted minor change in measurement accuracy, particularly concerning the potential enhancement with an increase in density. Extreme flows contain abundant surface features, suitable for processing under image velocimetry without the need for artificial tracers. Fujita et al (2011) classified them into three types: Type A: surface ripples due to turbulence in the flow; Type B: floating debris and other objects such as tree branches; Type C: the difference in water’s colour caused by varying concentration of suspended sediment. Despite the natural advantage, when Fujita et al sized the IA at 40*40 pixels (10.56m) on helicopter footage, missing velocities occurred in the transect. With the video recorded from higher altitude than drones, its spatial resolution decreased even for HD-sized frames. One possible way to increase detectability of tracers is to record videos at oblique angles. Fujita & Kunita (2011) noted such videos contain more tracers as surface ripples are more visible from sunlight reflections, or type A features. Similar occurrence was also observed under normal flows in Bomhof et al (2022), as nadir videos recorded from drones resulted in erroneous search lines under STIV due to lack of visible features, and had to be manually corrected. The oblique counterpart from a smartphone, however, didn’t have this issue. If drones are moving during filming, it can be possible to have oblique angles between the lens and the water surface, but risking a decrease in measurement precision (Liu et al, 2021). 41 For optical flow, parameter tuning is different to LSPIV due to the different method in calculating velocity. Bung & Valero (2016c) experimented filter sizes of 5px, 10px, and 15px on Horn-Schunck results of highly aerated flows, and found 15px producing the best results. As the flow regime has sufficient features for detection, similar to natural extreme flows, the high frame rate (732Hz) of videos and controlled lighting inside a lab makes this finding difficult to be directly applied in the field. On the other hand, it offered a potential parameter for tuning on Farneback, as similar tunings with this method are scarce. Hutley et al (2023) applied a Savitzky-Golay filter window to the results to improve accuracy, while Ansari et al (2023) did not specify similar parameters for Farneback. 2.4.2 Techniques in optimizing analysis parameters Interest has been growing on determining the optimal spatial distribution of tracers. Pizzaro et al (2020) developed the Seeding Distribution Index (SDI), a dimensionless metric aimed to minimize errors in LSPIV and LSPTV measurements. They modelled the metric and measurement errors as such: 𝜖 = 𝑐1(1 − 𝑒−𝑐2∗𝑆𝐷𝐼) (31) 𝑆𝐷𝐼 = 𝜈𝑘1 (𝜌𝜌𝑐𝜈1)𝑘2 (32) where 𝜖 is error in velocity measurement, 𝑆𝐷𝐼 is Seeding Distribution Index, 𝜈 is the spatial clustering level (achieved with Poisson random distribution for simulated results), 𝜌 and 𝜌𝑐𝜈1 are the seeding density and reference seeding density at 𝜈 = 1. 𝑐1, 𝑐2, 𝑘1, 𝑘2 are coefficients that are fitted through calibration. The SDI was applied for its relationship and velocity errors, and the results showed a significant decrease in LSPIV error for SDI between approximately 2 to 30. Dal Sasso et al 42 (2021) used SDI in identifying parts of a flow video for analysis – the maximum number of frames under minimum SDI. Cases in Switzerland and France were analyzed, of which the results could be used in reducing the length of footage for the most accurate surface velocities, eliminating the need for post-processing. However, the IA sizes remained fixed so other parameters were adjusted instead. Also, artificial tracers were used for the low flow conditions analyzed, in lieu of natural surface features. Yeh et al (2019) noted the potential in reducing IA sizes for LSPIV under favourable conditions, of which image noise is controllable. They proposed a flexible determination algorithm as below: 𝑑𝐼𝐴 = 𝑉𝑚𝑒𝑎𝑛′ ∗ ∆𝑡 ∗ 𝑓𝑡 ∗ 𝑚𝑠𝑛 (33) 𝑓𝑡 =𝑉𝑚𝑒𝑎𝑛′ + √(𝑉𝑚𝑎𝑥′ − 𝑉𝑚𝑒𝑎𝑛′ )2 + (𝑉𝑚𝑒𝑎𝑛′ − 𝑉𝑚𝑖𝑛′ )2𝑉𝑚𝑒𝑎𝑛′ (34) 𝑚𝑠𝑛 =𝐹𝐶𝐷𝑠̅̅ ̅ (35) where 𝑑𝐼𝐴 is the IA size in pixels, 𝑓𝑡 is the flow turbulence parameter, 𝑉𝑚𝑒𝑎𝑛′ is mean velocity over the entire detected area, ∆𝑡 is time difference between two frames (reciprocal of frame rate), 𝑚𝑠𝑛 is magnification factor, 𝐹𝐶 is detected environment factor, 𝐷𝑠̅̅ ̅ is average tracer density. Testing over videos on an artificial river in Taiwan yielded higher accuracy of surface velocity, under a variety of environmental conditions including rain and nighttime. Both SDI and adaptable IA algorithm used image binarization to isolate the tracers for calculating density, a critical value in their respective equations. The latter included additional steps prior to binarization – first CLAHE, Gaussian high-pass filter, and adaptive Wiener filter for enhancement and noise removal, then subtraction of image pairs (Figure 2.10). The resulting 43 tracers are suitable for LSPIV as they are contained within blocks, but may not exactly perform in the same fashion under optical flow for pixel-wise velocity fields. Also, those enhancement steps require processing time in addition to that of the existing workflow (Section 2.2.3), which could be a burden on the hardware. A simpler metric on surface features that could be used for both optimizing LSPIV and optical flow results should therefore be sought. Figure 2.10 The DADR process. a) Enhanced image. b) Subtraction of image pair. c) The binary image. d) The morphological process. e) The ROI mask. f) The ROI region. Figure from Yeh et al (2019). Copyright Elsevier (2019). Reproduced with permission. 2.5 Research gaps summary This section summarizes the research gaps resulted from the literature review. There have been many applications of image velocimetry methods on extreme flow events. They mainly involving LSPIV and STIV as these two methods, especially the former, has been refined and matured over the recent decades. However, more challenging cases are still lacking, particularly involving natural channels and a lack of visible GCPs. The destructive 44 nature of extreme floods also causes site areas to be unreachable, making GCP coordinates unavailable for orthorectification. Their effects and consequent adjustments to the existing processing workflow are unknown. Interest has been growing on using optical flow methods, including Farneback, on surface velocity measurement. Despite having application cases under mild flows, the behaviour of Farneback results under extreme flow conditions, has not been studied in the literature. This includes the change in surface velocity under different parameters, and their uncertainties. RIVeR, an existing software which did plan to accommodate rectification of optical flow vectors, has not had this feature implemented, posing an obstacle for the implementation of post-analysis orthorectification on Farneback results. Existing methods to optimize parameters based on seeding density used binarization to cater the needs of LSPIV, but such treatment method may not work with Farneback as it calculates movement for every single pixel instead of image blocks. With its additional processing time, a simpler metric on the density of natural surface features is needed for both LSPIV and Farneback velocities. 45 Chapter 3: Data collection 3.1 The flooding event and citizen videos In early November 2021, three atmospheric rivers hit the Pacific Northwest region, bringing over excessive amounts of rainfall that broke daily records in 20 places in Southern British Columbia (CBC News, 2021). The rainfall also resulted in melting of snowpack (Fannin et al, 2021). Erosion was further exacerbated by the summer 2021 wildfires, as the soil became water-repellent. The areas around Merritt, a city in the Nicola Valley of B.C.’s Southern Interior, were both ravaged by the extreme wildfires and then deluged by the extreme rainfall (Chung, 2021). All of them culminated in the sudden increase of flow rate in the Coldwater River flowing through Merritt, breaching the dykes and causing widespread flooding in the city area. The flooding also caused the collapse of the Middlesboro Bridge on Voght Street (thus a.k.a. Voght Street Bridge) and the failure of the city’s wastewater treatment plant. The latter resulted in the total evacuation of more than 7000 residents, for a second time after the summer wildfires. After the initial flooding, the Canadian Armed Forces arrived in to help repair and reinforce the breached dykes before additional high flows arrived in December (Little, 2021). A Sentinel 2 satellite image over the area on November 16th, 2021 (Figure 3.1) shows the extent of the flood inundation after the peak flow passed. 46 Figure 3.1. Sentinel-2 satellite image over Merritt on November 16, 2021. Major flow directions have been marked in blue arrows, and WSC’s station location marked with red arrow. Image taken from the Copernicus Open Access Hub. (https://scihub.copernicus.eu/) The Water Survey of Canada (WSC) has a long-term flow monitoring station in the city: Coldwater River at Merritt (08LG010). It is located in Merritt’s residential area, marked with a red arrow on Figure 3.1, well within the inundation area. The Coldwater at this point has a gross drainage area of 917 km^2. The station’s corrected hydrograph for the flooding events, displayed on Figure 3.2, shows an instantaneous peak flow of 324 m^3/s, occurring at 17:05 on November 15th, 2021. Closer examination of the hydrograph showed two peaks on November 15th, with the first one at 317 m^3/s at 05:55. The peak flow value contains a large amount of uncertainty, as the initial provisional value was over 400 m^3/s (Weijs & Kovacek, 2022), but corrected to 324 m^3/s later. 47 Figure 3.2. WSC Hydrograph from November to December 2021 (top) and zoomed into the atmospheric river event (bottom). Source of data: WSC & ECCC (2022). This event was captured by many local residents and their videos were posted online or featured in the news channels. Most videos were recorded in the morning of Nov 15, 2021, thus very close to the first peak. The sunset on November 15th was at 16:18, so the second or higher peak occurred almost an hour after sunset with little visibility. Other videos were filmed on days after the event. These videos provide an excellent source of data for applications involving image velocimetry. The details of video data are elaborated in Section 3.3. This event was chosen for three reasons. First, the damage caused by this event is significant. Total damage in the city is estimated to be 150 million CAD by the City of Merritt, while overall damages is around 11 billion (Lee & Parfitt, 2022), making it the costliest natural disaster in Canadian history. Second, the Coldwater River is relatively suitable for image techniques. Despite having a small drainage area, it has sufficient flow rates during high flows. Some sections of the river exhibit relatively uniform bathymetry compared to mountain streams, 48 rendering a more planar water surface. Third, the event took place in a relatively sudden manner, with little warning given to the citizens and local government alike. Therefore, it is likely no one there received prior training on image velocimetry, and all the videos were recorded for sharing on social media instead of for crowdsourcing platforms, making it representative of the commonly encountered situation. 3.2 Challenges with access, safety, and resources The first and foremost challenge is limited or restricted road access to the site areas. Province-wide, all the highways connecting Vancouver to the rest of British Columbia were closed due to washouts (Figure 3.3), and emergency repairs to reopen them were completed months later. The Coquihalla Highway, for example, wasn’t open to non-essential traffic until January 19th, 2022 (BC MoTI, 2023). Locally, the entire city was cordoned off following the evacuation, forbidding entry without authorization. Figure 3.3. Map of highway damage due to the November 15, 2021 storm event. Copyright the Province of British Columbia. All rights reserved. Reproduced with permission of the Province of British Columbia. 49 Figure 3.4. Site 1 area photographed on February 27, 2022. A short field visit to Merritt in late February found most parts of the Coldwater River froze at the surface (Figure 3.4), thus it was impossible to perform field surveying in winter without additional rescue training. Onsite surveying was not possible until August 2022, as the river was then under minimal flows (less than 1m^3/s) after frozen surfaces in winter and freshet in spring. The second challenge is the significant alterations to the natural landscape resulted from the high flow. The river changed its course at several locations in the city area alone. Many fixed objects present in some videos that could be used as GCPs, such as trees or even houses, disappeared after the flood. Some areas near the river are heavily vegetated and could only to be accessed through wading. The field visits showed many parts of the river bank were turned into 1-2m high cutbanks due to erosion, making access difficult. One example is shown in Figure 3.5. 50 Figure 3.5. River bank at site 1 photographed during a field visit in August 2022. The third one is the poor quality of most citizen videos. Apart from shaking, many have panning movement as described in the Argentinian case under Le Coz et al (2016). Though providing excellent overviews on the site, they complicate the stabilization process as shorter clips are trimmed in an effort to contain all the identified GCPs. The fourth challenge is limited resources posed to data collection. For citizen videos, it is important to have the collaboration of local authorities and other relevant stakeholders in order to facilitate with onsite data collection and monitoring (Le Coz et al, 2016). Unfortunately, the local government was contacted, but no replies arrived before surveying was commenced. Also, this sudden event precluded any prior coordination with the local governments regarding long-term monitoring or video data collection, and subsequent communications on this matter were slow as disaster response and recovery were the respective governments’ priorities. WSC has performed image velocimetry measurements with fixed cameras or drones in British Columbia, including at 51 Pemberton Creek in Pemberton (Bomhof, 2021), and Salmon River on Vancouver Island (Perks et al, 2020). However, there was, and still is, no calibrated monitoring sites installed or pre-event image-based measurements done on the Coldwater River in Merritt. The fifth challenge, arising from the previous, is the lack of up-to-date geodata publicly available for the region. BC’s provincial government maintains a freely-available LiDAR dataset, but data for the Merritt area is currently absent. The province’s existing digital elevation model is at a scale of 1:250,000, i.e., a resolution 25 meters per pixel. It was used by BGC Engineering in their peak estimates. Though suitable for large-scale for entire city areas, it is too coarse for such type of applications, as many areas covers in the camera’s field of view is around 50-100 meters in length. Consequently, the desired contour data needs to have a maximum scale of 1:50 for sufficient precision. Moreover, the best Google Earths imagery on pre-event Merritt was made in 2007, more than 15 years prior; post-event imagery was photographed on July 30, 2022 (Figure 3.6) and was not available until October, one month after surveying was finished. The satellite imagery in City of Merritt’s GIS portal was taken in 2016. With both imageries taken years before the flooding event, many objects near the old riverbanks have disappeared, causing confusion onsite. Lastly, due to the lack of prior installation of velocity monitoring apparatus, there is no velocity data for validation of image velocimetry results. The only data available for comparison is the corrected hydrograph from WSC, which contains instantaneous discharges during the event. 52 Figure 3.6. A comparison of Google Earth imagery on Sites 3 and 4 in 2007 (top) and 2022 (bottom). 3.3 Video data source Citizen videos on the November 2021 flooding event flows were collected from two methods: local residents with direct contact and YouTube, where some flooding videos were 53 uploaded by local residents. Field visits were made to the site in May and June 2022 to record videos on freshet flows, plus to copy the original video files from the established contacts. Direct contacts came from two sources. The first was then reached by phone after some of his videos were posted on his company’s Facebook page. The transfer of original videos was arranged later, and verbal authorization was obtained for using his videos for research. His videos were either filmed with an iPhone 11 Pro Max, or a DJI Phantom 4 Pro drone. Examples of the former are included in Figures Figure 3.7a and Figure 3.7b. The phone videos were filmed at 4K resolution (frame dimensions 3840*2160 pixels) and 60 frames per second, while the drone videos were also filmed at 4K resolution (frame dimensions 2720*1530 pixels) but at 30 frames per second. All of his videos were originally stored in the iMac at his office, then transferred through an external hard drive. It was discovered after returning from Merritt, that the phone videos’ datetimes created were not the datetimes when they were recorded, but when he copy-pasted the video files into a separate folder on June 16, 2022. Those incorrect datetimes were corrected during the surveying data collection trip in August 2022. The second contact had parts of the flooding event filmed from the left bank of the Voght Street Bridge, with an iPad Air 3rd Generation. All of their photos and videos on the flooding event were transferred through AirDrop (a wireless sharing function between Apple devices in close ranges), after verbal permission from them had been obtained for use in research purposes. Their videos on the flows are at HD resolution (frame dimensions 1920*1080 pixels) and 30 frames per second. One video was featured in Section 4.3. For the YouTube videos, they were downloaded from the website, with their resolutions and frame rates set by YouTube, normally at HD resolution and 30 frames per second. The 54 authors were unable to be contacted for the actual datetime recorded, nor the original videos. Most videos on the flooding flows are covering the city area, but some, specifically parts of videos, cover sections of the river. Notably, there are drone videos made available by the City of Merritt on YouTube (Greg InBC 2021; 2022), included in Figures Figure 3.7c and Figure 3.7d. Other videos include ones filmed with phones, often contained significant amount of shaking. A potential source for video data is footage from the traffic webcams installed on BC’s highways. However, a cursory inspection on some of the contemporary footage indicates a low likelihood of flooding flows being included, as those cameras’ field of view is almost exclusively on road conditions. Three field visits were conducted in spring of 2022 to record videos on freshet flows: May 22, June 15, and June 29. All videos were taken with a personal phone, an iPhone 11 Pro Max, in HD resolution with frame dimensions at 1920*1080 pixels, and frame rate at 30 frames per second. It is a mere coincidence that the same model of phone was used. Because iPhone 11 Pro Max has three camera lenses, but lens information isn’t stored in the video encoding (see Section 3.5 for details), the lens type and magnification level for every video were manually recorded. Most videos were filmed with ultra-wide lens at 0.5x zoom level. 55 Figure 3.7. Frames from smartphone videos (a,b) and drone videos (c,d) on the atmospheric river flows used in analysis, marked with a general direction of flow. 56 3.4 Site areas and description This section describes the designated sites in the section of the river within the city area, where videos on the flooding flows have been recorded. The site areas are designated 1 to 5, at ascending order from upstream to downstream. Figure 3.8 shows an overview of the five sites’ locations in Merritt. Site 1 is located at the section of river directly upstream of the Houston Street bridge. The approximate coordinates for this site are 50° 5'47.54'' N, 120°46'48.48'' W. The videos were taken on top of the bridge facing upstream. The left bank is mostly a natural area with dense vegetation, and a cliff of 10-meter height further upstream. Only the section closest to the left bank bridge abutment is accessible from the bridge by going down a cliff, while all upstream areas could only be accessed by wading across the river from the right bank. The right bank is at lower elevation than left bank, with a multiple housing park right next to it. The river channel in this site has a relatively constant width of approximately 25-30 metres. Site 2 is a bend of the river about 400m downstream of Site 1, coordinates 50°5’55.18” N, 120°47’6.43” W. Houston Street runs parallel to the section directly upstream of it, on top of a cliff. It is where all the videos covering this area were recorded. The river flows between two industrial areas, with its left bank a steep cutbank at a height of 3-4m, and right bank a densely vegetated area resembling a point bar. Further morphology has resulted in smaller alternating cutbanks and point bars on both banks. There was a decommissioned railway bridge connecting the two banks, but it was washed away by flooding sometime in the evening of Nov 15th, as it was still visible in the morning of Nov 15th (Figure 3.9a) but disappeared the following morning (Figure 3.9b). Only the abutments remained after the event, as shown in Figure 3.10. 57 Figure 3.8. Overview of the measurement sites in Merritt, over satellite image on 2022-07-30. 58 Figure 3.9. Site 2 in the morning of November 15 (a) and 16 (b), 2021, extracted from the videos recorded from the left bank by Simon Rizzardo. Figure 3.10. The remains of the bridge at Site 2, photographed on August 5, 2022 from the right bank. Sites 3 and 4 are respective areas upstream and downstream of the collapsed Voght Street bridge, delineated by the bridge itself, at the coordinates of 50°6’15.78” N, 120°47’38.66” W. Site 3 has experienced significant changes to its landscape, as the river changed its course during the flooding event. Before the flood, the left bank was a flat area with several trees along the waterline, a cliff followed further from the river. The right bank consisted of a line of trees parallel to the flow direction, separating the river and an RV park. During the flood, the river flow shifted towards the left bank, completely eroding away the flat area and the bridge’s left 59 bank abutment, causing the bridge to collapse. The new left bank was eroded into a cliff. In March and April 2022, the City of Merritt performed partial restoration work on this area, removing the bridge’s collapsed span, building a dyke at the left bank which stopped further erosion at the left bank, and restoring the flow direction to the original, adjacent to the line of trees. It was discovered during the field visits that the freshet changed the river’s flow direction to that along the newly built dyke. The river also shifted its course upstream, washing away several houses. A dyke was built by the municipal government at the current flow channel. The entire area has an overall size of around 275m by 120m. Directly downstream of the Voght Street Bridge, Site 4 is a lot smaller than Site 3, at a maximum of 100m in length by 55m in width. There are old bridge abutments present on both banks, with the right bank abutment directly adjacent to the flow. The left bank is slightly higher, dotted with trees along the waterline, before a cliff further from the water. Videos on the atmospheric river event were recorded on top of the cliff. No restoration work on this area has been carried out. Further downstream, at the Main Street Bridge, is where Site 5 is located, at 50°6’36.83” N, 120°48’12.43” W. It is specifically an area directly upstream of the bridge, with the camera facing upstream. It is in a very flat residential area, with both banks dotted by trees along the flow direction. During the flooding event, the river breached both banks, inundated the entire area, and changed its flow direction downstream of the bridge – through the residential area towards the confluence with Nicola River. Figure 3.11 shows an aerial overview of the site in late November 2021, the pumphouse is the blue building next to the river. Both the City’s pumphouse and WSC’s flow monitoring station are located in this area, inside a building on the left bank. 60 Figure 3.11. Site 5 as covered by a drone in the morning of November 16, 2021, by Simon Rizzardo. One additional site was brought forward by one of the citizen witnesses, tentatively designated Site 0. It is located at the Patchett Road Bridge (49°59’1.17” N, 120°55’40.77” W), approximately 30km upstream from the city area. He recorded videos at the area in late November of 2021 during emergency diking work after the flooding event, with both his phone and drone (Figure 3.12). The bridge remained intact after the flood, but the river eroded away the right bank ramp, which was later restored. This site was not extensively surveyed, however, due to the lack of any precise elevation data required to set up datums for the total station. As there are many more videos covering the flooding in the city’s residential area, they are easier to be georeferenced than the river sections, thanks to the large number of fixed objects present in their fields of view. Though providing valuable data on urban flooding, they were ruled out for the current study as going beyond its scope. 61 Figure 3.12. Overview of Site 0, extracted from drone video recorded on November 29, 2021, by Simon Rizzardo. 3.5 Geodata collection on GCPs and transects Collection of sites’ geodata, specifically the coordinates of GCPs and cross-sections, was achieved through onsite surveying with a Leica TC705 total station, borrowed from the Department of Civil Engineering. Field surveying was performed in Sites 1 to 5 between August 26th to September 2nd, 2022. One prior visit was conducted on August 5th to further explore the areas, whose access was precluded due to high freshet discharges. To conduct topographical surveys, local datums are required for the respective site areas. Survey monuments in the city area were considered as benchmarks for routing to the different sites. Unfortunately, none were found during site visits as they were likely destroyed by the flooding. 62 Therefore, for every site area, several points were selected as makeshift datums, with their coordinates in UTM determined from Google Maps, then their elevations estimated from the elevation contours in the City of Merritt’s GIS system, with the most precise elevations at 1m, extending to the city’s boundary. This is standard for municipal datasets, but not precise enough for datum setups. There is no direct indication on when elevation data was collected, but the elevations of city utility assets, such as light poles, fire hydrants, etc., were collected in 2013, which are considered outdated. Moreover, many areas near the river were flooded, altering their elevations. Despite these shortcomings, there was no other publicly available data to use. The datums’ elevations were estimated by linear interpolation between the two closest contour lines. The total station was then set up at locations where at least two datums were visible, or the prism placed on top of it was visible. Then the instrument’s location was determined through free-stationing, that is, using two or more datums to triangulate the instrument’s coordinates. 3.5.1 Ground Control Points (GCPs) Ground control points, or GCPs, were identified from the video frames, using fixed objects such as built structures, utility poles, houses, etc. Many of them could be easily identified on Google Maps, thus their coordinates determined. Some were also designated as makeshift datums. Due to the post-event nature, no artificial GCPs were set up prior to the event. Conversely, for more natural areas, such as Site 1, few artificial objects were visible in the video frames. The only identifiable objects were certain trees along the river banks with distinguishable shapes, which were selected as GCPs. Figures Figure 3.13 and Figure 3.14 show the GCPs in the video frame at Site 1, marked and numbered in red. Many trees were grouped close together that they became almost indistinguishable on Google Maps. Attempts in 63 determining their coordinates without onsite surveying resulted in confusions and consequently, errors. Figure 3.13. Labelled GCPs in red for Site 1 video filmed on November 15, 2021. 64 Figure 3.14. Labelled GCPs in red for Site 1 video filmed on June 29, 2022. Surveying the coordinates of trees as GCPs, plotted in Figure 3.15, wouldn’t significantly reduce the errors to minimal levels. The coordinates could only provide accurate locations on where the trees are, but when the GCPs were chosen on the video frame, errors from the actual surveyed point were inevitable. Furthermore, many trees only had their canopies visible due to the dense vegetation around them. Two-dimensional projective transformations performed on Site 1 show that errors at a minimum of 0.5m are normal, and higher errors should be expected as the points are further away from the camera, especially for points that were not clearly distinguishable. 65 Figure 3.15. Overview of datums, instrument stations, and surveyed GCPs at Site 1. 3.5.2 Cross sections Cross sections, or transects, were surveyed for Sites 1,4, and 5 as they experienced less morphological change compared to Site 2 and 3. For each site, the designated transects were spaced to be 5-10m apart, and must be visible from the location where videos were recorded. For Site 1, nine transects were selected, seven were actually surveyed, designated numbers 0, 1, 2, 4, 5, 6, and 8, with ascending order from downstream to upstream, i.e., increasing distance from the camera. Figure 3.16 lays out all seven surveyed transects on a rectified Google Earth image of Site 1. It should be noted that not all transects were visible in the videos. 66 Figure 3.16. Overview of surveyed transects at Site 1. For Site 4, three out of four transects were actually surveyed. For Site 5, eight transects were surveyed. All cross-sections’ coordinates were corrected to project on the same bearing between the surveyed left and right ends. Points beyond the surveyed ends were reconstructed or extrapolated, with their elevations estimated from City’s elevation contours, similar to those of the datums. 3.6 Cameras and lens distortion correction Camera calibration was performed by taking pictures of a calibration pattern at different angles. In this case, a black-and-white checkerboard pattern of size 46mm is used. Sets of calibration photos were taken using the same lens at the same magnification level(s). Autofocus was turned off. For every set of calibration photos, the checkerboard was located in the centre 67 and all eight edges of the photo frame (top, bottom, left, right, top left, top right, bottom left, bottom right), with the camera pointing straight at the pattern, at its left, and at its right. The total is twenty-seven photos per set, with the camera positions are displayed in Figure 3.17, producing matrices and vectors such as the ones in Table 3.1. All photos were then processed in the Camera Calibrator App in MATLAB. Figure 3.17. Pattern-centric camera positions as processed from the Camera Calibrator App. For iPhone 11 pro max, there are three back camera lenses for different zoom levels: • ultra-wide lens: for 0.5-0.9x magnification, ƒ/2.4 aperture and 120° field of view; 68 • wide-angle lens: for 1.0-1.9x magnification, ƒ/1.8 aperture; and • telephoto lens: for 2.0-10.0x magnification, ƒ/2.0 aperture. Calibration photos for all three lenses in both aspect ratios (4:3 and 16:9) were taken on December 19th, 2022 inside the Hydraulics Lab. Distance between the lens and pattern was between 2 to 3 metres. The app-generated camera parameters for 16:9 photos were used to correct lens distortion to all videos taken with this device, as they are all at that aspect ratio. Table 3.1. An example of calculated camera parameters from the Camera Calibrator App. Camera intrinsic matrix Rotation vector (image 1 of 27, transposed) Translation vector (image 1 of 27, transposed) [2888.746 0 00 2905.403 01984.374 1130.154 1] [0.0145−0.23943.1239] [183.9654−71.52372756.664] For iPad Air (3rd Generation), photos were taken with its only back camera lens (ƒ/2.4 aperture), outside of Ross Stewart’s house on September 2nd, 2022. However, all calibration photos are at an aspect ratio of 4:3. Due to how the app is designed to incorporate camera parameters in image processing, the app-generated camera parameters are unusable for videos with different aspect ratios. Consequently, all the iPad videos were not corrected for lens distortion. One unexpected issue occurred with determining the lens information in the videos. The photos on Apple devices are encoded in High Efficiency Image File Format (HEIF), storing lens data, such as type of lens used, focal length, exposure time, etc., in the image files. On the other hand, all the videos on Apple devices are encoded in High Efficiency Video Coding (HEVC), with only frame rate and dimensions but not the lens information. The difference between HEIF and HEVC formats displayed on an iPhone is shown in Figure 3.18. Le Coz et al (2021) noted the convenience of retrieving camera information from the smartphone sensors for effective 69 camera calibration, and subsequent facilitation of image velocimetry applications by the general public. However, the lack of lens information in the video files is still an obstacle for full implementation. Figure 3.18. HEIF (left) for photos vs HEVC (right) for videos, as displayed from an iPhone. The photo and video were provided by Ross Stewart. Similar to the iPad videos, lens distortion correction was also not performed on all the drone videos. This is due to the difficulties in determining the specific drone model used to record the video(s), or having access to the same model of drone for camera calibration with checkerboards. Streßer et al (2017) noted that the rectilinear lens used in drones, specifically DJI 70 Phantom III, have very little distortion. This indicates calibration for drone videos may not be required, as was part of the processing applied on the video data in McIlvenny et al (2022). 71 Chapter 4: Image data processing Eight videos were used for detailed analysis, five of which covered the flow at Site 1. The first two were recorded during the atmospheric river event, the other three during the freshet season in 2022, one for each field visit. The last two videos in Tables Table 4.1 and Table 4.2, being downloaded from YouTube, were used only for determining surface velocity. Many of the videos have actual frame rates that are slightly less than the nominal frame rates of 60 or 30. Table 4.1. List of all videos used in analysis and their information. Video file name Date Time recorded (hh:mm:ss) Location Device Lens (if applicable) IMG_0713 2021-11-15 07:44:00 Site 1 iPhone 11 pro max Wide IMG_0807 2021-11-16 08:46:30 Site 1 iPhone 11 pro max Wide IMG_7832 2022-05-22 15:04:18 Site 1 iPhone 11 pro max Ultra-wide IMG_8338 2022-06-16 12:38:23 Site 1 iPhone 11 pro max Ultra-wide IMG_8478 2022-06-29 11:53:47 Site 1 iPhone 11 pro max Ultra-wide IMG_5892 2021-11-15 07:16:17 Site 4 iPad Air (3rd Generation) N/A N/A 2021-11-15 N/A Sites 3,4 Drone N/A N/A 2021-11-16 N/A Sites 3,4 Drone N/A Table 4.2. List of all videos’ frame properties. Video file name Resolution Frame dimensions (px) Nominal frame rate (fps) Actual frame rate (fps) IMG_0713 4K 3840*2160 60 59.97 IMG_0807 4K 3840*2160 60 59.98 IMG_7832 HD 1920*1080 30 29.99 IMG_8338 HD 1920*1080 30 29.99 IMG_8478 HD 1920*1080 30 29.99 IMG_5892 (clip 1) HD 1920*1080 30 30.00 Drone (2021-11-15) HD 1920*1080 30 30.00 Drone (2021-11-16) HD 1920*1080 30 30.00 72 The analysis workflow implemented in this study is as follows, with LSPIV and Farneback optical flow selected for calculating surface velocities. The overall process is very similar to that in Section 2.2, with the code almost exclusively written and run in MATLAB environment. 4.1 Before processing 4.1.1 Video selection and trimming For videos in 4K and 60fps, they were trimmed to at most about 5 seconds, which translates to around 300 frames. For HD 30fps videos, they were used in their entirety of 10-13 seconds, or between around 300 to 380 frames. Videos with significant movements, including panning, were trimmed to frames containing all the designated GCPs. For all videos or their clips, it was assumed that the flow during a video (or clip)’s entire duration is steady, thus satisfying the constraint for Eulerian frame of reference. For the drone videos, due to the amount of movement present, and both of them focused on surveying the extent of the flooding and its damage in the city area, there existed only short moments showing the riverflow with relatively minimal obstruction. The former, recorded on November 15th, was trimmed to 4.164 seconds (125 frames) and the latter to 2.733 seconds (81 frames). Though the numbers of frames are significantly less than the iPhone videos, they should still be sufficient in minimizing velocity variations as per Dramais et al (2011). 4.1.2 Camera lens distortion correction and stabilization First, the video was loaded and its frame rate and dimensions were extracted. Lens distortion correction, if necessary, was applied to all original video frames with the camera parameters calculated from MATLAB’s Camera Calibration App (details in Section 3.5). 73 Image stabilization was applied to all video frames to remove any shaking or movement during video collection. This was achieved through the respective feature included in Fudaa-LSPIV, requiring a manually-defined mask to cover over the flow area, then matching the points generated from stationary objects, and correcting the image frames by aligning those points through affine or projective transformations, with the first frame as reference. The type of transformation was selected depending on the type and magnitude of movement. The stabilized frames were exported and compiled back into a video, keeping the same frame rate and dimensions as the original. Before the images were processed for raw surface velocity, pre-processing was performed to increase or enhance the visibility of surface features for detection. For PIV, preprocessing was done through PIVlab GUI while a similar procedure was implemented in custom code for Farneback. The pre-processing steps were defined and executed as such: 1. Define a region of interest (ROI) for analysis, and applied to all video frames. For PIVlab GUI, the area of interest is created through defining masks to exclude areas of non-interest, and thus isolating the area for analysis. The last frame is used as a background image for the ROI to be defined, as it usually has the maximum amount of stabilization correction to match the first frame. This is to minimize erroneous vectors induced by black areas introduced by the stabilization algorithm. 2. Apply contrast-limited adaptive histogram equalization (CLAHE) to all the cropped video frames, achieved by using the MATLAB function “adapthisteq”. This enhances the contrast of the frames’ grayscale pixel intensities using the aforementioned method. Inspection of the PIVlab GUI’s code shows this feature under the function “PIVlab_preproc” uses uniform distribution with a clip limit of 0.01. Such parameters are carried over in the Farneback code. 74 3. The pixel intensities at the bottom and top 1% are then automatically stretched using the “imadjust” function. Other enhancement techniques, such as Gaussian filters, adaptive binarization, subtract mean intensity, were not implemented as flooding flows already had sufficient surface features to be detectable with those basic techniques. 4.2 Image processing for raw surface velocities The workflow for both LSPIV and Farneback utilized the mask or region of interest mentioned in Section Error! Reference source not found. for image processing. Post-processing for filtering out spurious vectors was not applied to the raw velocity vectors. Figure 4.1. PIVlab GUI, showing analysis parameters, the mask, and enhanced image. Processing for LSPIV was performed in the PIVlab GUI (Thielicke & Stamhuis, 2014), in Figure 4.1, where the analysis parameters were defined. For all videos, Fast Fourier Transform (FFT, see Section 2.1.1) was selected, with high correlation coefficient. Passes and their interrogation area sizes were different, with the details in Section 4.4. Step sizes were defined as 75 50% of those of their respective interrogation areas. Image processing under PIVlab includes parallel computing for shortened analysis time, as well as batch processing for running the software uninterrupted at night. The output was horizontal and vertical velocity vectors in the image coordinates, Figure 4.2 shows their time-averaged magnitude being plotted as a contour plot. Figure 4.2. Contour plot of time-averaged raw velocities determined through PIVlab. For Farneback optical flow, custom code was written to process the videos for raw velocities, with the parameters defined in the code every time before the code was run. The only adjustable parameter for Farneback in the MATLAB environment was the size of averaging filters. All other parameters were fixed for all the videos. They are as follows: • Number of pyramid levels: 3 • Pyramid scale: 0.5 • Number of iterations: 3 76 • Neighbourhood size: 5 pixels Then, time-averaged velocity vectors and magnitude were calculated and saved into a .mat file, a contour plot is shown as Figure 4.3, similar to Figure 4.2. The velocities for individual frames were not saved in order to minimize the use of disk space. Contrary to PIVlab, parallel computing was not incorporated into the Farneback code, as the attempts resulted in flow velocities with a reversed direction. Figure 4.3. Contour plot of time-averaged raw velocities from Farneback optical flow. 4.3 Orthorectification workflow for surfacy velocity vectors Orthorectification after analysis was adapted, to transform velocity vectors from the image coordinate system to real-world coordinate system. This method was chosen over the conventional orthorectification mode, i.e., orthorectifying video frames before analysis, for allowing multiple iterations on orthorectification with minimal changes on the original velocity 77 results. It was a conservative measure accounting for the high amounts of uncertainties present in the geodata. The workflow consists of two separate parts: setting up orthorectification parameters and implementation of them in orthorectifying raw velocimetry results. It was recreated based on RIVeR from Patalano et al (2017). The detailed steps for both steps were designed and implemented as below, respectively. An example of the code is included in Appendix D For determining orthorectification parameters: 1. Both an original video frame and one with marked GCPs are loaded. 2. GCPs are manually defined on both the original and marked frames, achieved using MATLAB’s cpselect function. Their image coordinates (i,j) are stored, with only the ones in the original frame used for orthorectification calculations. 3. Real-life GCP coordinates (x,y,z), stored in a csv file, are loaded. 4. For 2D orthorectification, a 3x3 projective transformation matrix and errors in meters are calculated, using only the GCPs’ 2D coordinates (x,y). See steps 5 to 7 for 3D orthorectification. 5. For 3D orthorectification, a 3x4 camera projection matrix is calculated along with reprojection errors in pixels. This transforms coordinates in (x,y,z) to (i,j). 6. An interpolated elevation is calculated for all the GCPs. Their image coordinates are then “corrected” to the same elevation plane with Equations (18) and (19). 7. Calculate a new 3x3 projective transformation matrix between the corrected image coordinates and 2D real-life coordinates. 8. The calculated parameters are saved into a .mat file. For orthorectifying velocity vectors: 78 1. Both the raw velocity results (PIVlab session for LSPIV) and the previously mentioned orthorectification parameters are loaded. 2. A polygon region of interest is defined by user for the desired velocities, without restrictions on the number of vertices. In RIVeR, this region of interest can only be a quadrangle, i.e., with four vertices. This is difficult to define as not all river sections, especially natural ones, fit into a quadrangle without residuals. 3. The video’s frame rate is manually defined. However, for Farneback results, the frame rate is already saved into the raw velocity results file and is loaded along with everything else at step 1. 4. The raw velocities, in both horizontal and vertical directions, are isolated from the region of interest, then rectified and plotted to real-world coordinate system using Equations (26) through (30). As optical flow shows velocity at every single pixel, the velocity and image matrices are of the same size. For LSPIV results, however, they are different, and thus require additional steps detailed in steps 5 to 8. The size of LSPIV velocity matrices and meshgrids is smaller than the image size by a factor of approximately the final step size. For example, if the IA size is 16px, step size is thus 8px, and consequently velocity and meshgrid matrices are approximately 1/8 of that of the image. Interestingly, it was discovered that the larger the IA size, the smaller the size of meshgrid and velocity matrices, with the scaling factor higher than that of the step size. For an IA sized at 128px, step size is 64px, but velocity matrices are scaled down to more than 65 times from the image. 5. A scaling factor between image size and meshgrid size is calculated, rounded down to the closest whole number. 79 6. The image matrices, including the region of interest as it has the same size of the background image, are downscaled to get it as close as the velocity matrices as possible. They will likely be slightly larger than the velocity ones, so cropping is performed to bring them to the same size as the latter. 7. The scaling factor is also applied to the GCP’s image coordinates, with a new 3x3 transformation matrix calculated. 8. The LSPIV velocity vectors in both horizontal and vertical directions are rectified still using the original transformation matrix as in step 4, but plotted using the new matrix for LSPIV results in step 7. 9. Velocity magnitude is calculated from the rectified vectors. Both the rectified vectors and magnitude are mapped on an orthorectified background image. This is different from that of RIVeR, which a completely new meshgrid matrix sized at 256px-by-256px, based from the quadrangle ROI, is generated for plotting the rectified vectors. 2D rectification was applied on November 2021 phone videos as most of the GCPs are very close to the water surface, as seen in Figure 3.13. 3D rectification, however, was mainly applied on freshet videos and the November 2021 drone videos, because more GCPs are visible and the elevation range between them is often greater than 3m. The water surface elevations in many videos were unavailable or could not be determined, an interpolated elevation was calculated to correct all GCPs for the new 2D transformation matrix. The rectified PIV results from RIVeR and recreated code were compared for verification of the code, using the same orthorectification parameters, shown in Figures Figure 4.4 and Figure 4.5. The velocities on both figures were nearly identical, especially the velocity vectors, 80 which are crucial in displaying the surface flow patterns. This indicated that the recreation had been a success, allowing further orthorectifications to be performed on the custom code. Figure 4.4. Rectified velocity vectors and contours from RIVeR, the original frame and ROI are on the left. Figure 4.5. Plots of rectified velocity vectors (a) and magnitude contours (b) from the recreated code. 4.4 Estimation of stage and discharge Because WSC’s monitoring station is downstream of all designated sites, no other staff gauges were available for estimation of water surface elevations (WSEs) at the respective sites. 81 Therefore, they had to be estimated, either using fixed objects by the river banks or a manually defined waterline. If fixed objects were available to interpolate/extrapolate the stage, then they could be used to estimate the water surface elevation using the method in Le Boursicaud et al (2015): a pixel scale was calculated for a known height, then interpolated/extrapolated to the water surface. However, not all sites have the fixed objects by the river’s bank. In this case, a waterline was manually defined on a video’s first frame as the background image (Figure 4.6a). The waterline vertices, in image coordinates, were transformed into world coordinates and plotted on orthorectified image (Figure 4.6b). A corrected transect is imported, and the water surface coordinate for the particular transect should intersect the transect line, either by interpolation or extrapolation. In the latter scenario, the water surface elevation was estimated from the City’s elevation contours using the intersect coordinates. The WSE’s for all transects were then corrected for a fitted slope in the correct direction. For velocity-area methods in calculating discharge, depth-averaged velocities for the transect verticals are required, estimated from surface velocity using the coefficient 𝛼. Different 𝛼 values, 0.9 and 0.85, were applied to the November 2021 and Spring 2022 videos respectively. The former was deemed to be a reasonable estimate given no velocity data was collected during the events. Transect discharge was calculated using the mean-section method. Transect coordinates were imported from a csv file, with the coordinates of the existing velocities applied into the transect profile to estimate their depths by linear interpolation. Velocities for the transect were extracted using their end coordinates, and interpolated for the surveyed locations by the nearest value. 82 Figure 4.6. Defined waterline on original image with surveyed transects (a), and on orthorectified image with extrapolated transects (b). 83 4.5 Surface features Though binarization produces features suitable for calculating tracer density on LSPIV results (Section 2.4.2), it negates the greyscale variations from natural surface features during extreme events. The features from the raw images alone could indicate density and distribution of surface tracers under different environmental conditions. The visual complexity of an image is defined as the level of detail or intricacy contained within it (Forsythe, 2009). In the case of videos on river flow, image complexity could correspond to the amount of features present on the water surface. For this study, edge pixels were used as a metric for image complexity within the flow area of video frames, inspired by the study on the surface of tidal streams in McIlvenny et al (2022). Sobel method was applied on unenhanced stabilized grayscale video frames. It is a gradient-based method looking for strong changes in the first derivative of an image, specifically the pixel intensity (Farber, 2011). The pixel gradient threshold for Sobel edge detection was defined at 0.025 to allow for less prominent gradient changes to be detected. McIlvenny et al (2022) did not disclose a threshold for processing. An ROI was defined over major flow area within the video frames. Fixed objects such as trees were excluded to minimize the detection of edge pixels other than surface flow features. For all frames in the video, edge pixels were first detected on the entire frame, then isolated for the ones inside the ROI. This was designed to minimize the occurrence of edge pixels on the ROI’s boundary, ensuring most if not all the edge pixels were from the surface features. The number of edge pixels were then divided by the total number of pixels in the ROI for the percentage of edge pixels in one frame. The percentage was then averaged across all frames for the entire video. 84 An attempt was made to correlate parameter combination having the least amount of error to the percentage of surface features present in the flow. It was hypothesized for this study that more surface features within the flow area indicates using smaller IAs or filters will induce less erroneous vectors or noise. 4.6 Parameter tuning This section describes the tuning of parameters used on analyses under both LSPIV and Farneback optical flow. Specifically, CLAHE and IA or filter sizes, camera calibration, as well as pre-analysis vs post-analysis orthorectifications are mentioned, for they require separate explanations from what is usually included in the results chapters. 4.6.1 CLAHE block sizes and interrogation area or filter sizes The CLAHE block sizes were 4, 8, 16, 32, 64, 128, and 256 pixels for 4k videos. Those values were halved for HD videos, i.e., 2, 4, 8, 16, 32, 64, and 128 pixels. In addition, Farneback analyses also included no CLAHE treatment, meaning the unenhanced video was analyzed for velocities in the image coordinates. For LSPIV this was deemed unnecessary, as Jolly et al (2021) already noted CLAHE treatment on the frames a required minimum for PIV or PTV analysis. As noted in Section 4.2, the size of interrogation areas was variable for LSPIV while the size of the averaging filter was variable for Farneback. Both have an effect in reducing image noise, but increasing IA’s in LSPIV also decrease the resolution of results (Bung & Valero, 2016a). For Farneback applied on 4K videos, the filter sizes were selected as 4, 8, 16, 24, 32, 48, 64, 96, and 128 pixels. The sizes for HD videos were halved from those on 4K: 2, 4, 8, 12, 16, 24, 32, 48, and 64 pixels. Those pixels sizes are similar to what were used in Legleiter & Kinzel 85 (2020) and Pearce et al (2020), indicating a reasonable range on these videos. The IA sizes for LSPIV were identical to Farneback’s filter sizes, with one exception for HD videos: 2 pixels was removed because it was unable to be processed due to the step size being 1 pixel. Tables Table 4.3 and Table 4.4 show the IA sizes and their respective passes implemented in PIVlab, with “N/A” indicating the particular pass was not used. Table 4.3. LSPIV analysis parameters for 4K videos. Final IA size (px) Pass 1 (px) Pass 2 (px) Pass 3 (px) Pass 4 (px) 4 256 96 24 4 8 256 128 48 8 16 256 128 64 16 24 256 128 64 24 32 256 128 64 32 48 256 128 64 48 64 256 128 64 N/A 96 256 128 96 N/A 128 256 128 N/A N/A Table 4.4. LSPIV analysis parameters for HD videos. Final IA size (px) Pass 1 (px) Pass 2 (px) Pass 3 (px) Pass 4 (px) 4 128 64 24 4 8 128 64 32 8 12 128 64 32 12 16 128 64 32 16 24 128 64 32 24 32 128 64 32 N/A 48 128 64 48 N/A 64 128 64 N/A N/A Incorporating the both sets of parameters resulted in a matrix of parameter combinations, with the rows occupied with CLAHE block sizes and the columns with IA or filter sizes. For 86 LSPIV results, it produced a 7-by-9 matrix on 4K videos and 7-by-8 matrix on HD videos. For Farneback results, an 8-by-9 matrix of parameter combinations for both 4K and HD videos. 4.6.2 Camera calibration To evaluate the effect of camera calibration on image velocimetry results, nine parameter combinations were selected from the matrices in Section 4.5.4, for both LSPIV and Farneback results on videos recorded on iPhone 11 Pro Max. This was intended to evaluate the changes on surface velocity and discharge due to an absence of camera lens parameters, in case the video’s author(s) may not be able to be contacted directly. The chosen parameters are as follows: For 4K videos: • CLAHE block sizes: 16, 64, 256 pixels • IA or Filter sizes: 16, 32, 64 pixels For HD videos: • CLAHE block sizes: 8, 32, 128 pixels • IA or Filter sizes: 8, 16, 32 pixels Because the images before and after distortion correction exhibited little visible difference, especially in the displacement of objects, all raw velocity results were rectified using the same control point coordinates as those from the undistorted videos. Other parameters, such as stage and alpha, were also kept the same, isolating camera calibration to be the only variable parameter. 4.6.3 Pre-analysis vs post-analysis orthorectification An evaluation of the difference of surface velocities and discharges between the two orthorectification modes was attempted, specifically between the conventionally used pre-analysis orthorectification and post-analysis orthorectification implemented in this study. 87 To orthorectify video frames for velocimetry analysis, the distortion-undistorted and stabilized frames were orthorectified using the same orthorectification parameters, i.e., transformation matrix, calculated from Section 4.3. The imwarp function in MATLAB was used, with cubic interpolation of pixel, as specified in the latest version of Fudaa-LSPIV’s User Manual. The orthorectified frames were then upsized to the same height in pixels of the original video frame. For example, if the original frame dimensions are 3840*2160 pixels, then the orthorectified frames will be upsized to a height of 2160 pixels, with the length varying subject to orthorectification parameters. The following parameter combinations were selected for analysis, with the IA sizes are applied to all videos. They correspond to different real-life distances in different videos from their respective pixel scales. • IA sizes: 4,8,16,32,64; • CLAHE block sizes for 4K videos: 16, 64, 256, and none • CLAHE block sizes for HD videos: 8, 32, 128, and none Similar to Section 4.6.1, IA sizes smaller than 4 pixels resulted in no result produced from PIV and thus not used for analysis. The behaviour of both velocimetry algorithms under those parameter combinations were analyzed separately from their counterparts in post-analysis orthorectification, as it is difficult to exactly match the sizes of pre-processing and image processing parameters in real-life distances. 88 Chapter 5: Results This chapter shows and explains the results from image velocimetry analyses, including velocities and discharges on the atmospheric river videos, as well as the freshet videos. Results from different parameters in the workflow follow, consisting of camera calibration, computation times, orthorectification parameters, water surface elevations, orthorectification types, tuning of image processing parameters, and edge pixels as a metric for flow area image complexity. 5.1 iPhone videos on atmospheric river at Site 1 This section shows the results processed from the iPhone videos on Site 1 recorded on both November 15 and 16, 2021. Both videos were corrected for lens distortion before processing under LSPIV and Farneback methods. Following the classification by Fujita et al (2011), the surface features in these two videos were mainly type A, with type B in the form of branches partially visible. As Site 1 is located approximately 2.7km upstream of the gauge station, the lag time between these two locations must be estimated. This was estimated through interpolation of the peak flow travel time between the aforementioned station and Coldwater River near Brookmere (08LG048), a WSC’s gauge station 43.5km upstream. The 6-hour lag time between Brookmere and Merritt corresponded to a 23-minute lag time for the 2.7km stretch. Accounting for uncertainties of the real-time hydrograph when measuring highly unsteady flows, reference values were selected 25-30 minutes after the times recorded. To evaluate the spread of the transect discharges, root mean square error (RMSE) was used in addition to standard deviation. WSE’s value was used as the actual value from which the differences were calculated. Uncertainties were expressed through coefficient of variation, i.e., standard deviation divided by 𝑄𝑚𝑒𝑎𝑛, as a measure of spread among the transect discharges 89 relative to the mean. This is different to that of Le Coz et al (2021), which used discharges at 97.5% and 2.5% for the maximum and minimum ends of the two-tail 95% confidence interval, for a sample size of 500. Because only four transect discharges were extracted, such a small sample size was insufficient for t-tests. 5.1.1 November 15, 2021 Figure 5.1. Results for video recorded on 2021-11-15: LSPIV surface velocity vector (a) and contour (b) plots; Farneback surface velocity vector (c) and contour (d) plots. The rectified LSPIV and Farneback surface velocities are shown in Figure 5.1, both bear a strong resemblance to each other as similar analysis parameters were used: CLAHE was 90 applied at 64-pixel blocks with a 16-pixel interrogation area (LSPIV) or filter size (Farneback). The latter had slightly higher surface velocities thanks to its higher resolution at pixel size: a maximum magnitude of 12.954 m/s compared to LSPIV’s 11.794 m/s. The red dots in Figures Figure 5.1b and Figure 5.1d indicate high surface velocities, which correspond to the downward travelling slope of the waves. Taking the overall nature of the flow into account, the video likely depicted undular jumps as per Dingman (2009, p354), with the Froude number in the range between 1 to 1.7. The crest and upward travelling slopes are represented as areas of low surface velocity immediately behind those crests. This is likely attributed to the fact that the crests as surface features exhibit less movement than the downward slopes, whose motion was relatively perpendicular to the flow as captured from the upstream-facing camera, and its velocities increased due to rectification. The waves’ upward slopes were not visible in the camera located in the Northwest (top left) direction away from the rectified images. Figure 5.2. Transect surface velocities for LSPIV (a) and Farneback (b) on 2021-11-15. 91 The general resemblance and detailed difference in surface velocities were also reflected in the calculated discharges of the transects. From the values in Table 5.1, Farneback is marginally higher than LSPIV except for CS#8 where Farneback has a 6% higher discharge than LSPIV, and CS#4 where it is 4.6% lower. When all transect discharges were averaged, Farneback was only 1.4% higher than LSPIV. As the WSEs originally estimated from the waterline were not following a correct slope, they were corrected to a slope of 0.0184 between CS#4 and CS#8. Table 5.1. Transect WSEs and discharges at Site 1 on 2021-11-15. Transect number WSE original (m) WSE corrected (m) Q_LSPIV (m^3/s) Q_Farneback (m^3/s) CS#4 602.099 602.099 205.8 196.4 CS#5 602.157 602.204 216.5 218.5 CS#6 602.005 602.402 236.8 237.5 CS#8 602.701 602.701 332.4 353.2 Averaged Q (m^3/s) 247.9 251.4 The reference value on instantaneous discharge, from WSC’s corrected real-time hydrograph, at 08:15 on November 15th, 2021, is 297 m^3/s. The averaged discharges from both methods are compared against the former, in Table 5.2. Both methods have discharges underestimated from that of WSC, with Farneback method exhibiting less difference at 15.35% compared to LSPIV’s 16.54%. Apart from higher discharge estimates, Farneback also has higher standard deviation and RMSE compared to LSPIV. The transect discharges displayed an interesting phenomenon of increasing discharge values in the upstream direction, likely caused by two factors. First, the downstream transects CS#4 and CS#5 are significantly shallower in the middle than the banks, shown in Figure 5.3, which likely wasn’t the case during the flood. The velocities are also higher in the middle, 92 reducing the weight of their respective discharge verticals. The upstream transects, however, are relatively uniform in terms of depth, especially in the middle. Second factor is the likely possibility of overestimating the WSE in the upstream transects from the corrections. Table 5.2. Errors and uncertainties for the averaged discharges at Site 1 on 2021-11-15. Method LSPIV Farneback Q – averaged (m^3/s) 247.9 251.4 Difference to WSC value (%) -16.54% -15.35% Standard deviation (m^3/s) 57.81 69.92 Uncertainty (%) 23.32% 27.81% RMSE (m^3/s) 70.13 75.80 Figure 5.3. Site 1 transect profiles with respective WSE values on 2021-11-15. 5.1.2 November 16, 2021 As this video was recorded at the same location as the previous one, it was processed in a similar fashion, the velocity vector and contour fields are shown in Figure 5.4, (a) and (b) for LSPIV, (c) and (d) for Farneback. Maximum surface velocity for LSPIV is 8.314 m/s, and for Farneback 10.749 m/s, which is a larger difference than in the previous video. 93 Both methods produced the distinct pattern of standing waves present in the video, as the crests were static features on the surface. On first sight, they may be similar to the undular jumps that were depicted on the previous day, but onsite inspections revealed that parts of the river bed are not uniform in depth, which could explain the cause behind those standing waves occurring only at a portion of the river’s width. Similar to the video the day before, higher velocities occur at the waves’ downward slope. A closer examination shows the dots of high velocities in Farneback’s contour plot are more prominent throughout the river’s length. While for LSPIV, the dots further away from the camera, near the bottom right corner of Figure 5.4b, are stretched and smoothed, as the pixel scale for that portion of the original image is higher than that at a closer distance, therefore the same IA covered larger areas there. This smoothing effect could be a factor causing the larger difference in surface velocities between the two methods. The original waterline-induced WSEs, in Table 5.3, caused the same issue of incorrect slope, and they were corrected using the slope among CS#5, CS#6, and CS#8, then extrapolated and corrected for the same WSE at CS#4. After transect velocities were extracted for discharge calculation, the results are in a similar trend to the previous video. Farneback’s discharges are slightly higher than those of LSPIV, except for CS#6 where it is marginally less. The averaged discharge across all transects are similar on both methods, with Farneback’s at 3.3% higher than LSPIV. 94 Figure 5.4. Results for video recorded on 2021-11-16: LSPIV surface velocity vector (a) and contour (b) plots; Farneback surface velocity vector (c) and contour (d) plots. The effect of standing waves on the transect velocities is also visible through Figure 5.5, as there is a substantial increase of surface velocity slightly right from the middle at CS#8, caused by the transect cutting through one of the standing waves. However, as the waves decrease in magnitude as they travel downstream, all three downstream transects are not visibly affected. 95 Figure 5.5. Transect surface velocities for LSPIV (a) and Farneback (b) on 2021-11-16. WSC’s value for reference at 09:10 on November 16th, 2021 is 119 m^3/s. The differences against WSC’s value are significantly reduced, as both methods fall within a maximum of 2%, or 2.38 m^3/s, and continue to resemble similar discharge estimates, with Farneback being higher and at an overestimate in this case. Conversely, standard deviation and RMSE increased for both methods, listed in Table 5.4. The transect discharges exhibit a wider spread, especially from the significant overestimate at CS#8, potentially caused by over-corrected WSEs. Table 5.3. Transect WSEs and discharges at site 1 on 2021-11-16. Transect number WSE original (m) WSE corrected (m) Q_LSPIV (m^3/s) Q_Farneback (m^3/s) CS#4 601.345 601.345 76.97 77.14 CS#5 599.770 601.586 89.16 93.23 CS#6 600.070 601.885 112.0 111.8 CS#8 600.603 602.418 191.6 203.2 Averaged Q (m^3/s) 117.4 121.3 96 Table 5.4. Errors and uncertainties for the averaged discharges at site 1 on 2021-11-15. Method LSPIV Farneback Q – averaged (m^3/s) 117.4 121.3 Difference to WSC value (%) -1.30% +1.97% Standard deviation (m^3/s) 51.55 56.39 Uncertainty (%) 43.89% 46.47% RMSE (m^3/s) 44.67 48.89 5.2 Drone videos on Sites 3 and 4 Both drone videos were only processed in LSPIV for surface velocities. For most sections of the river in these two site areas, discharge was unable to be estimated as the bathymetry was changed significantly during this event and subsequent restoration works. The surface velocity profile in the morning of November 15th, illustrated in Figures Figure 5.6a and Figure 5.6b, shows the river just started to shift towards the left bank at Site 3 due to erosion. It is the result of higher velocities on the left bank, a maximum of 3.94 m/s. This later eroded away the bridge’s left bank abutment, causing its collapse. Despite having a large part of the surface flow obstructed by trees, the vectors in Figure 5.6b indicate the higher velocities transition from the left to right bank just upstream of the bridge. When the same section of river was filmed on the next day, illustrated in Figures Figure 5.6c and Figure 5.6d, a new curvy channel had been eroded to the left of the original, and some of the flow passed over where previously was the bridge’s left abutment. Because the video was recorded from a different location than the previous, the section of river at Site 3 was less obstructed, allowing a better view on its surface flow structure. 97 Figure 5.6. LSPIV result for drone videos on Sites 3 & 4: surface velocity contours (a) and vectors (b) over surveyed or nominal transects on 2021-11-15; LSPIV surface velocity vector field (c) and contours (d) for Site 3 on 2021-11-16. The new flow has a maximum surface velocity of 2.62 m/s directing to west then north, but suddenly decreases to almost nil at the end of the curve, then increasing again to just upstream of the collapsed bridge. Visual inspection on the video shows the decrease is likely representative of a hydraulic jump or series of standing waves, with the relatively stationary 98 features exhibiting minimal displacement. The vectors after the jump are more spread out, and some flowing towards the right side constitute backflow that eventually rejoins the main flow slightly upstream of the curve. 5.2.1 Oblique waves The drone video on November 15th captured complex surface flow features, including oblique shock waves occurring at some trees. These waves were used as a proof of concept to estimate instantaneous water depths. Those stationary waves are formed when fluids under supercritical flow passes through stationary objects. The angle of the shock waves is inversely proportional to Froude number in the equation below (Henderson, 1995): sin 𝛽 =𝑐𝑣=1𝐹𝑟 (36) where 𝛽 is the angle between the flow direction and the shock wave front (the former is originally the side wall of a flume, which is parallel to the direction of flow), 𝑐 is the celerity or wave speed, 𝑣 is the velocity of the flow, and 𝐹𝑟 is Froude number, a dimensionless ratio between inertial and gravitational forces of open channel flow, which is usually calculated as such: 𝐹𝑟 = 𝑣√𝑔𝐷 (37) where 𝑣 is the mean velocity of the flow, 𝑔 is gravitational constant, 𝐷 is hydraulic depth where: 𝐷 =𝑐𝑟𝑜𝑠𝑠 − 𝑠𝑒𝑐𝑡𝑖𝑜𝑛 𝑎𝑟𝑒𝑎𝑠𝑢𝑟𝑓𝑎𝑐𝑒 𝑤𝑖𝑑𝑡ℎ (38) For channels with high aspect ratios (1:15 or higher), 𝐷 can just be regarded as flow depth. 99 In this study, the direction of flow was determined through extracting and averaging the velocity vectors within a user-defined region near the wave. The region was large enough such that it represented an accurate representation on the flow direction. The wave front was then defined on the original image. Both lines were orthorectified into the real-world space, of which the angle 𝛽 and Froude number were calculated. Two prominent oblique waves were identified: the first one on the left bank immediately upstream of the bridge, the other on the right bank immediately upstream of the old bridge abutment. For the former, its 𝛽 was determined to be 35.325 degrees, corresponding to Fr of 1.7295. The latter had its 𝛽 at 46.302 degrees, corresponding to Fr of 1.3831. Both trees are identified in Figure 5.7a and their waves’ Froude number shown in Figure 5.7b. The estimated depths for the two locations were 0.291m and 0.169m, respectively. This was under the assumption of a high aspect ratio and 𝛼 of 0.9, as accurate surveying on the channel bathymetry during that time was precluded by significant erosion from the flood. Those results provide a general estimation on the flow depth and characteristic at those locations. The supercritical nature of the flow is consistent with the scouring exhibited, with the upstream tree uprooted and disappeared entirely, despite at a lesser Froude number. Although it would be possible, in theory, to validate the estimated depths with surveyed data, the drastic changes on the landscape caused by the flood rendered such data inaccurate for validation. 100 Figure 5.7. Oblique waves identified with red arrows in the original frame (a) and plotted on the rectified frame (b) with the averaged flow direction and calculated Froude numbers. 101 5.3 iPhone videos on freshet at Site 1 Surface velocity results from May 22 and June 16 in 2022 are included in this section, both using a parameter combination of 32px for CLAHE and 16px for IA or filter. For all freshet flows, manual waterline estimation was unable to produce reliable results despite having no overbank flows. The riprap-covered right bank caused the corresponding WSEs to produce significantly lower discharges, and slope-based corrections similar to the results in Section 5.1 were not possible. Also, estimating the actual lag times was more difficult between Site 1 and WSC’s station at Site 5, due to the flow peaks being less prominent and consequently unable to be identified in the hydrograph at Brookmere station. Therefore, no discharge comparisons were performed against WSC’s values. Due to different parameters for 3D orthorectification, the two cases had a different number of transects visible in the rectified range of coordinates: for May 22, three transects, CS#0, CS#1, and CS#2; for June 16, four transects, the aforementioned three plus CS#4. Table 5.5. Averaged transect surface velocities for both methods on 2022-05-22 and 2022-06-16. Date Method Velocity @ CS#0 (m/s) Velocity @ CS#1 (m/s) Velocity @ CS#2 (m/s) Velocity @ CS#4 (m/s) 2022-05-22 LSPIV 0.6696 0.6418 0.5001 N/A Farneback 0.6718 0.6268 0.5880 N/A Difference (%) 0.33 -2.33 17.58 N/A 2022-06-16 LSPIV 1.1484 0.9297 0.7223 0.5368 Farneback 1.0925 0.8731 0.7616 0.5739 Difference (%) -4.86 -6.09 5.45 6.91 The surface velocity contours on May 22 in Figures Figure 5.8b and Figure 5.8d show that Farneback has a slightly higher peak than LSPIV: 1.4372 m/s versus 1.3197 m/s, which is similar to the atmospheric river cases. The flow exhibits a more uniform and homogeneous 102 pattern, with a V-shape likely caused by the channel slope, or the middle section of the river exhibiting less surface features. For reference, WSC’s gauge recorded 13.9 m^3/s at 15:45 on the same day, i.e., approximately 40 minutes after the video. Figure 5.8. Surface velocity results for freshet video on 2022-05-22: LSPIV vectors (a) and contours (b); Farneback vectors (c) and contours (d). 103 When the transect velocities were extracted and plotted in Figure 5.9, both methods display a similar pattern of vectors, though Farneback vectors are less homogeneous and some appear to scatter more towards the banks. When those velocities were averaged, as presented in Table 5.5, Farneback results were 17.6% higher than LSPIV at CS#2, the farthest of the three. The other two had differences of less than 2.5%. When the absolute differences were compared, that difference at CS#2 was at 0.088 m/s, but at a higher proportion due to the overall lower surface velocity of the flow. Figure 5.9. Transect velocity from LSPIV (a) and Farneback (b) for video on 2022-05-22, plotted on a rectified frame as background image. For the video on June 16, peak surface velocity from LSPIV was at 1.8133 m/s, higher than Farneback’s 1.7772 m/s. WSC’s gauge recorded 27.9 m^3/s at 13:15 on 2022-06-16, i.e., approximately 40 minutes after the video. The flow in this video was showing less visible features in the middle, as sunlight was shining directly towards the camera during filming. Such 104 is exhibited in Figure 5.10 for both methods, the middle of the channel shows less magnitude than the sides, but the tree shadow on the left bank (near bottom right of the subfigures) produced higher velocity than its surroundings. Figure 5.10. Surface velocity results for freshet video on 2022-06-16: LSPIV vectors (a) and contours (b); Farneback vectors (c) and contours (d). 105 The extracted transect velocities, displayed in Figure 5.11, show Farneback velocities in the middle of the channel at CS#4 decreased in magnitude and homogeneity compared to LSPIV, more vectors point towards the banks than in the direction of the flow. For the two transects closer to the camera, CS#0 and CS#1, Farneback’s vectors appear to be similar to those of LSPIV, albeit slightly more scattered. The averaged velocities in Table 5.5 show the differences fall consistently between 5% to 7%, with Farneback less than LSPIV at CS#0 and CS#1, and higher at CS#2 and CS#4. The scattering of Farneback’s vector directions may be attributed to the method’s higher sensitivity to image noise, caused by a lack of prominent surface features in the flow. Figure 5.11. Transect velocity from LSPIV (a) and Farneback (b) for video on 2022-06-16, plotted on a rectified frame as background image. 106 5.4 Camera calibration 5.4.1 Flooding videos To investigate the effect of lens distortion on discharge estimates, the iPhone videos were also processed in their original form, i.e., without distortion correction. For both videos under both LSPIV and Farneback, all analysis parameters were unchanged, therefore isolating lens distortion to be the only independent variable. Averaged transect velocity was not used due to influence from the many verticals located in the overland flow area. The results in Table 5.6 show distortion correction introduces differences in averaged discharge at 1% maximum, less than “a few percent” noted in Le Boursicaud et al (2016). When discharges from different parameter combinations were compared, the maximum difference of 1.25% occurred for LSPIV at 64px for both CLAHE and IA size. For both videos in general, LSPIV results exhibits higher differences than Farneback, though the specific reason behind this is unknown. Table 5.6. Summarized discharge estimates of Site 1 videos, with and without distortion correction. Date Method Q_mean with distortion correction (m^3/s) Q_mean without distortion correction (m^3/s) Difference from Q_mean with correction (%) 2021-11-15 LSPIV 247.9 245.5 -0.493 Farneback 251.4 251.9 +0.216 2021-11-16 LSPIV 117.5 118.5 +0.892 Farneback 121.3 121.9 +0.448 The difference in discharge is reflected through transect velocities. As shown in Figure 5.12, the transect velocities generally exhibit minimal difference between with and without distortion correction, for both methods. Given the distance between the lens and water surface is a lot longer than the calibration distance of 2 to 3m in Section 3.6, the actual difference in discharge estimation should be close to negligible as the longer distance allows the entire flow 107 area to be visible and centred within the lens’ field of view. In contrast, videos recorded closer to the river are more likely to exhibit higher distortion, especially near the frame’s sides. Figure 5.12. Transect surface velocities for video on 2021-11-15, with and without lens distortion correction. LSPIV are (a) to (d), Farneback are (e) to (h). 108 5.4.2 Freshet video Figure 5.13. Transect surface velocities for video on 2022-06-16, with and without lens distortion correction. LSPIV are (a) to (d), Farneback are (e) to (h). The same technique was applied to freshet videos as well, with the one on June 16th, 2022 used as an example. Since discharge could not be estimated and compared due to difficulties in 109 estimating WSEs, only transect velocities were compared for the effect of distortion correction applied upon the ultra-wide lens. From Figure 5.15, the differences in velocity magnitude appear to be similar to that under wide angle lens, but the overall proportion is higher due to lower velocity under milder flows. This is ascertained through average transect velocities, as the maximum difference is around 5.3% for both methods, but at different transects: CS#4 for LSPIV, CS#1 for Farneback. If the ultra-wide lens was used to film those flooding videos in addition to wide-angle lens, the resulting difference in velocity would be very similar. Therefore, the effect of lens distortion correction on surface velocity measurements may also depend on the general velocity of the flow. 5.5 Computational costs To compare the computation costs of both LSPIV and Farneback, their processing times were recorded for analyses under some parameter combinations. All the analysis cases were performed on Lenovo ThinkPad T490s, with an 8th-generation Intel i7-8565U CPU at a frequency of 1.80 GHz, and a RAM of 16.0 GB. For LSPIV, only total time was recorded as it was the output from PIVlab, and six data points were available for CLAHE block size of 8px, as the majority of cases were processed on batch at nighttime. For Farneback, CLAHE and image processing times were able to be recorded separately, so the processing time of tuning individual steps were also analyzed. As the results are plotted in Figure 5.14, some general trends emerged. For LSPIV, shown in Figure 5.14a, larger IA sizes result in shorter processing time, this is also subject to the number of passes used to narrow down the IA size. However, due to insufficient data points collected, the exact relationship is unknown. For pre-processing, smaller CLAHE block sizes led 110 to increasing computing time in what appears to be exponential in Figure 5.14b. For Farneback, larger filter sizes result in longer computing time, which is the opposite to the IAs in LSPIV. Figure 5.14. Total computing time in PIVlab per different IA sizes on 4K video (a), computing time per different CLAHE block sizes (b) and Farneback averaging filter sizes (c) for both 4K and HD videos. Overall, the total computation time for LSPIV and Farneback on original frames are similar, 4K and HD videos alike. However, PIVlab has already improved the algorithm that speeds up calculations by a factor of 3 while the custom written Farneback code could be further streamlined and adapted for batch processing. As two video resolutions were tested, being HD and 4K, Figures Figure 5.14b and Figure 5.14c show computing times increased by approximately a factor of 2 from HD to 4K, though it wasn’t the same video being tested at different resolutions, but instead different videos with their own regions of interest. Video in 4K have frames with 4 times as many pixels than those of HD, which was indeed a burden during processing, as the computer’s RAM was running at capacity, and 4K videos with more than 300 frames (or 5 seconds in 60fps) were unable to be processed. 111 5.6 Orthorectification parameters 5.6.1 2D orthorectification and choice of GCPs Two versions of orthorectification were tested on site 1’s video on November 15th. The first was used to rectify the vector results in section 4.1, and the second to reduce the error. Due to the difficulty in pinpointing precise coordinates of the GCPs, the projection errors of GCPs between image and real-world coordinates are significantly higher than usual cases. Version 1 has 14 GCPs, with errors ranging between 0.0564m to 7.8398m, with a 95% confidence interval of 2.7276m. Higher errors are usually at GCPs further away from the camera, with the highest of 7.8398m at GCP#3, outside of the image bounds in Figure 5.15Figure 5.16a at the bottom right. Version 2 is a revision from version 1 and has 12 GCPs, i.e., GCPs #3 and #9 in version 1 were removed (shown in Figure 5.15). Errors in the projection range from 0.1000m to 3.6559m, RMSE at 1.5737m, with a 95% confidence interval of 1.7697m. It was a significant improvement from the previous. The details results are in Appendix A The rectified results for LSPIV and Farneback show insignificant difference between the two versions. For LSPIV, maximum surface velocity of version 2 is 11.6505 m/s against version 1’s 11.7943 m/s. For Farneback, maximum surface velocity of version 2 is 12.8233 m/s against version 1’s 12.954 m/s. The same are for discharge estimates, with both averaged estimates under version 2 slightly less than those in version 1: 0.29% for LSPIV and 1.2% for Farneback. It should be noted, since the estimate of WSE from images require orthorectification, a new set of transect WSEs need to be estimated from version 2. But the results in Table 5.7 show a maximum difference of only a few millimeters. 112 Figure 5.15. LSPIV surface velocity contours of orthorectification version 1 (a) and version 2 (b). GCPs outside of the image bounds are also included. Note Version 2 has GCPs #3 and #9 removed. 113 Both changes in velocity and depth contribute to the differences in discharge. When transect velocities were plotted, shown in Figures Figure 5.16 and Figure 5.17 respectively for LSPIV and Farneback, it appears that velocities under version 2 are reduced in magnitude than version 1, especially in the peaks. At CS#8, peak transect velocity for LSPIV decreased from 9.602 m/s in version 1 to 9.352 in version 2, or 2.60%; for Farneback, the decrease is higher, from 10.838 m/s in version 1 to 10.060 m/s in version 2, or 7.18%. When compared against the decrease in discharge at the same transect (2.60% for LSPIV and 7.45% for Farneback), it clearly showed varying velocity had more influence on discharge than depth. CS #4 and CS#6 in the figures appear slightly shifted towards the left bank. For CS#5 and CS#6, decreases in depths corresponded to increases in discharge, further indicating that orthorectification accuracy caused more change on velocity than discharge. Table 5.7. Differences in transect WSE and discharge for the two versions of orthorectification. Transect number WSE corrected version 1 (m) WSE corrected version 2 (m) Q_LSPIV version 1 (m^3/s) Q_LSPIV version 2 (m^3/s) Q_Farneback version 1 (m^3/s) Q_Farneback version 2 (m^3/s) CS#4 602.099 602.102 205.8 214.4 196.4 209.5 CS#5 602.204 602.202 216.5 219.1 218.5 218.7 CS#6 602.402 602.397 236.8 231.3 237.5 238.1 CS#8 602.701 602.693 332.4 323.8 353.2 326.9 Averaged Q (m^3/s) 247.9 247.2 251.4 248.3 114 Figure 5.16. LSPIV transect velocities at site 1 on 2021-11-15, under two versions of orthorectification: CS#4 (a), CS#5 (b), CS#6 (c), and CS#8 (d). Figure 5.17. Farneback transect velocities at site 1 on 2021-11-15, under two versions of orthorectification: CS#4 (a), CS#5 (b), CS#6 (c), and CS#8 (d). 115 5.6.2 3D orthorectification For the drone videos, due to a larger area covered in the field of view with more fixed objects distinctly visible, 3D orthorectification was used instead of 2D. This also allowed more coordinates able to be estimated from Google Maps and the City’s GIS portal. Figure 5.18. 3D reprojection of GCPs in drone video on 2021-11-15. The reprojection errors for GCPs in both drone clips are in a few pixels, with the maximum less than 9 pixels. Averaged error for the November 15th clip was 3.7435 pixels, RMSE at 4.4821 pixels, illustrated in Figure 5.18. For November 16th, averaged error was 3.0240 pixels and RMSE at 3.6584 pixels. A similar challenge existed for pinpointing GCPs in the images post-event, but not as severe as in the smartphone videos. Trees as GCPs generally exhibit higher errors than manmade fixed objects like bridges or telephone poles, as many were in the inundation area where the landscape was altered by the flood. The left bank telephone poles, GCPs #6 to #9 have an averaged error of 3.1077 pixels with the maximum being 6.0058 116 pixels at GCP #9. For the left bank trees in the inundation area, GCPs #10 to #13, they have an averaged error of 6.5794 pixels, with the maximum at GCP #13 at 8.8788 pixels. For the freshet videos under 3D orthorectification, the difficulty of pinpointing coordinates of trees still existed similar to the previous subsection, despite having more trees visible under the ultra-wide lens with no overland flows. The video on 2022-06-16, for example shown in Figure 5.19, had an averaged projection error of 5.2726 pixels for 11 GCPs, and RMSE of 5.5302 pixels. Compared to atmospheric river videos, the main challenge was obstruction by some of the trees, including GCP#5 with only its canopy being visible. The errors from other freshet videos were similar, with the video on 2022-05-22 had an average of 6.3438 pixels and RMSE of 7.0868 pixels, and the one on 2022-06-29 at an average of 4.1789 pixels and RMSE of 4.5920 pixels. The detailed results for both cases are included in Appendix B Figure 5.19. 3D reprojection of GCPs for iPhone video at site 1 on 2022-06-16. 117 5.7 Water surface elevations (WSEs) To investigate the effect of changes in WSE on discharge estimates, WSEs were increased and decreased by increments of 0.1m or 10cm, up to a maximum of 0.8m or 80cm, using the same surface velocity field in world coordinate system. Such an extreme range can occur in case the waterline is difficult to locate, which did occur for both videos due to frequent obstructions. Although the 2D orthorectification matrix in theory needs to be recalculated due to different image coordinates of GCPs following a change in WSE, the exact adjustments were too difficult to quantify without additional fixed objects as a reliable reference. Table 5.8. Results for WSE sensitivity analysis on site 1 video in 2021-11-15. ΔWSE (m) Wetted area (m^2) Δ (%) Q_LSPIV (m^3/s) Δ (%) Q_Farneback (m^3/s) Δ (%) -0.8 42.74 -42.02 150.985 -39.09 153.419 -38.97 -0.7 46.24 -37.28 163.071 -34.22 165.619 -34.12 -0.6 49.80 -32.45 175.164 -29.34 177.825 -29.27 -0.5 53.43 -27.52 187.262 -24.46 190.042 -24.41 -0.4 57.22 -22.38 199.369 -19.57 202.279 -19.54 -0.3 61.12 -17.09 211.481 -14.69 214.531 -14.67 -0.2 65.15 -11.63 223.603 -9.80 226.797 -9.79 -0.1 69.31 -5.98 235.737 -4.90 239.084 -4.90 0.0 73.72 --- 247.886 --- 251.399 --- +0.1 78.38 +6.33 260.049 +4.91 263.740 +4.91 +0.2 83.07 +12.69 272.213 +9.81 276.078 +9.82 +0.3 87.77 +19.06 284.378 +14.72 288.421 +14.73 +0.4 92.48 +25.45 296.544 +19.63 300.765 +19.64 +0.5 97.19 +31.84 308.709 +24.54 313.110 +24.55 +0.6 101.90 +38.23 320.874 +29.44 325.455 +29.46 +0.7 106.62 +44.64 333.039 +34.35 337.802 +34.37 +0.8 111.35 +51.05 345.205 +39.26 350.149 +39.28 From Table 5.8, for the case in Section 4.1.1, the changes in both LSPIV and farneback discharges are slightly less than those in the wetted area. At an increase of 0.4m, the wetted area 118 increased by 25.45%, both LSPIV and Farneback discharges increased by around 19.6%. The opposite is observed when WSE decreases. This suggests a slight decrease in surface velocity with an increase in wetted areas and vice versa, corroborating the result from Le Boursicaud et al (2016). Interestingly, as a 2D orthorectification case, though surface velocities remained constant as rectification is at the same plane as the free surface, yet the results still exhibit a similar behaviour as in 3D orthorectification. 5.8 Pre- vs post-analysis orthorectification 5.8.1 Atmospheric River videos For the iPhone video at site 1 on November 15th, 2021, its upsized orthorectified frames had a pixel distance of 0.07267 m/px. Surface velocity results for the combination of CLAHE at 64px and IA or filter size at 32px (i.e., PP64-IA32 for LSPIV or PP64-F32 for Farneback) were selected for comparison in this subsection. The IA or filter size in real-life distance was therefore 2.325m. Showing the vectors and contours in Figure 5.20, the surface velocities resulted from LSPIV and Farneback are noticeably different. The former (Figure 5.20b) are significantly reduced in magnitude while the latter (Figure 5.20d) are the opposite. Both vector patterns in Figures Figure 5.20a and Figure 5.20c are more scattered than the “post-analysis” counterparts in Figures Figure 5.1a and Figure 5.1c. The likely cause of this change is the decrease of surface tracer visibility due to correction of high perspective distortion from orthorectification, converting the flow area into a more pixelated appearance. This effect was possibly worsened by enlarging the orthophotos through interpolation of grayscale intensities. 119 Figure 5.20. Surface velocity vector and contour plots for the video on 2021-11-15 under pre-analysis orthorectification: LSPIV are (a) and (b), Farneback are (c) and (d). Comparing the transect velocities in Figure 5.21 displays the drastic difference between the two orthorectification types, as well as the different behaviour of the two velocimetry methods on the orthorectified frames. For LSPIV, the “pre-analysis” velocities in the main channel were consistently in the range of 1-2 m/s, compared to the 4-6 m/s in “post-analysis”. The former also smoothed out all the peaks present in the latter, which represented the downward slopes of waves in the flow (Section 5.1.1). The pixelated features caused cross-correlation algorithm to detect movements significantly less than from the original frames. Near the right 120 bank, “pre-analysis” produced slightly higher velocities than “post-analysis”, as image noise from the trees was amplified. Figure 5.21. Transect velocities for video on 2021-11-15 under the two types of orthorectification. LSPIV are (a) to (d), Farneback are (e) to (h). 121 For Farneback, in contrast, velocities increased to a maximum of 16-22 m/s for “pre-analysis”, twice or three times higher than the “post-analysis” counterpart. The right bank noises were amplified in even higher magnitude. This disproportionately high velocity is likely the result from enlargement of orthorectified frames and image enhancement, introducing excessive image noise in the already “noisy” orthorectified flow area. Velocities at areas closer to the camera (Figure 5.21d), including the transects (Figures Figure 5.21e, Figure 5.21f), are higher than the ones further away. Such is inconsistent with the pattern from “post-analysis” in Figure Figure 5.21d. The “pre-analysis” contour plot also shows many small patches, indicating the Farneback algorithm detected pixelwise movement for the highly pixelated surface features. In this case, the averaging filter in Farneback failed to smooth out the noise. This can be fixed with larger filter areas, but at a higher cost in computation. Generally, both methods perform better with original video frames for citizen flooding videos, especially under lower tilt angles, as the complex surface features are better preserved and more consistent surface velocity results are maintained. The more conventional “pre-analysis” type of orthorectification, which applies image velocimetry to orthorectified frames, may require additional steps in the workflow to account for the pixelation of the flow area and the consequent introduction of excessive image noise. 5.8.2 Freshet videos To investigate the effect of “pre-analysis” orthorectification on flows with less surface features, the video from June 16th, 2022 was selected for illustration thanks to the particular lack of features in the middle section. The upsized orthorectified frames had a pixel distance of 0.1490 m/px. Results for the combination of CLAHE at 64px and IA or filter size at 16px (i.e., 122 PP64-IA16 for LSPIV or PP64-F16 for Farneback) were selected for comparison in this subsection, therefore the IA or filter size in real-life distance was 2.384m. Figure 5.22. Results from pre-analysis orthorectification for video on 2022-06-16: LSPIV vectors (a) and contours (b); Farneback vectors (c) and contours (d). Shown in Figure 5.22, compared to “post-analysis”, LSPIV surface velocities are lower in magnitude, while Farneback velocities are higher and especially for areas closer to the camera. 123 A closer visual examination on the LSPIV vectors shows a more scattered pattern, and higher surface velocity concentrated to a few spots in the middle section, indicating that a lack of surface features compounds the issues of excessive pixelation and image noise already present in the orthorectified frames under flooding conditions. 5.9 Tuning of processing parameters Parameter tuning was performed on both atmospheric river and freshet videos at site 1. In this section, some results from the matrix of parameter combinations were selected to visualize the effect of varying one of the two parameters (CLAHE block size, or IA/Filter size). For atmospheric river videos, when CLAHE block size changed, IA or filter size was fixed at 16 pixels. When the latter changed, the former was fixed at 64 pixels. For both methods, CS#5 at site 1 was used for comparison unless otherwise specified. The approximate length scale at this transect is 0.0153m per pixel. The details for freshet videos are explained in Section 5.9.3. 5.9.1 LSPIV velocities As shown in Figure 5.23 for the video on November 15, 2021, interrogation area sizes have more influence on LSPIV velocities than CLAHE block sizes. When IA is sized at 128 pixels, or 1.958m, there is visible decrease of surface velocity in the middle as the peaks, between 17 to 25m from the left bank, are smoothed over. This decrease is less widespread in varying CLAHE sizes, with the maximum range of differences occurring over the peak, about 20m from the left bank. As CLAHE size decreases, peak velocity increases, from 9.473 m/s at 4 pixels to 8.496 m/s at 256 pixels. The decrease in peaks is reflected through discharge, 216.0 m^3/s for CS#5, a 0.25% decrease. When the averaged transect discharges were compared, 128px IA produced 233.7 m^3/s, a 6.54% decrease. CS#8 contributed the most, at a decrease of 14.35%, as it is the most upstream of the four with a higher pixel distance of 0.0238m/px. 124 Figure 5.23. LSPIV surface velocity profiles at Site 1 CS#5: only changing CLAHE block size (top) and only changing IA size (bottom). PP stands for “Preprocessing”, numbers in pixels. An inspection over all four transects showed no missing surface velocities, even at the smallest interrogation area of 4 pixels, corresponding to real-life distances of 0.0508m, 0.0612m, 0.0732m, and 0.0952m for CS#4, CS#5, CS#6, and CS#8 respectively. This is a stark contrast to the helicopter videos in Fujita & Kunita (2011) with sudden gaps in the transect velocity profile at 40 pixels, or 10.56m. It also to some extent corroborates an occurrence in Bomhof (2022), when oblique videos from a smartphone exhibited more surface features for detection than nadir drone videos covering the same site. The detectability also depends on lighting conditions and spatial resolution, as Ansari et al (2023) found though having similar velocities between nadir and oblique videos when those conditions are met. When velocity profiles at farther away transects were inspected, it appeared that smoothing of peak velocities occurred at smaller IA sizes with increasing distance from the camera. In Figure 5.24, the smoothing starts at 96 pixels on CS#6, and at 64 pixels on CS#8. 125 Their real-life distances are 1.758m and 1.523m respectively. Combining with CS#5, they show a trend of decreasing real-life size of IAs with increasing distance from the camera. Figure 5.24. LSPIV surface velocity profiles at Site 1 CS#6 (top) and CS#8 (bottom), under different IA sizes in pixels. As all four transects were extracted from the raw velocities within the same image space, and pixel scales are different in oblique images, it is impractical to apply different IA sizes solely for obtaining optimal velocities. The result from this case implies that reasonable sized interrogation areas are sufficient to avoid both missing velocities and over-smoothing of peaks. When the velocity magnitude contours were examined for over-smoothing effects, the video on November 16, 2021 was used thanks to the many standing wave slopes present. Shown in Figure 5.25, smoothing of the upstream section appears to start at IA size 16px (Figure 5.25c or case PP64-IA16), and it propagates towards the downstream areas as IA size increases, eventually smoothing out all individual standing waves at 96px (Figure 5.25h or case PP64-IA96). 126 Figure 5.25. LSPIV velocity contour plots for Site 1 video on 2021-11-16 under different IA sizes: a) to i) are assigned to increasing IA sizes from 4px to 128px. 127 5.9.2 Farneback velocities Farneback’s velocity profiles under changes in both CLAHE and filter sizes, shown in Figure 5.26, exhibit continued variations along the entire transect. Contrary to the smoothing of velocity peaks under higher IA sizes in LSPIV, changes in filter size did not produce a similar result. Only at CS#8, where it is the furthest from the camera, did velocity gradually decrease starting at filter sizes of 96 pixels and higher. The spread of peak velocities in varying CLAHE sizes is similar to that in LSPIV, with the maximum peak velocity at 4 pixels was 9.459 m/s. Figure 5.26. Farneback surface velocity profiles at site 1 CS#5: only changing CLAHE block size (top) and only changing filter size (bottom). “PP” stands for Pre-Processing, “F” stands for filter, “n” stands for not used, numbers in pixels. Quite notably, when CLAHE was not applied, in case “PPn-F16” of Figure 5.26, the peak velocity was significantly lower, from 8.574 m/s in 256px to 6.923 m/s. The decreasing in peaks without applying CLAHE is also reflected through discharge. For CS#5, case “PPn-F16” produced 205.3 m^3/s, a 6.02% decrease. The averaged discharge is 244.1 m^3/s, a 2.9% 128 decrease. Interestingly, the trend of decreases is reversed to increasing IA sizes under LSPIV, as CS#4’s discharge decreased by 5.89%, but CS#8’s discharge increased by 1%, likely due to additional image noise introduced by CLAHE. This implies that applying contrast enhancement to the frames, even at larger block sizes, increases surface velocity estimates under Farneback method. This phenomenon should be similar to that in LSPIV, even as the two methods use different mechanisms to calculate vector fields. Figure 5.27. Farneback surface velocity profiles at site 1 CS#6 (top) and CS#8 (bottom), under different CLAHE block sizes in pixels. When the effect of CLAHE was inspected for transects farther away from the camera, namely CS#8, the difference between with and without CLAHE becomes negligible compared to CS#6, as shown in Figure 5.27. This suggests CLAHE is more effective on Farneback velocities in areas closer to the camera than those farther away, however, it is unknown if such effect is the same for LSPIV due to CLAHE being the prespecified minimum in preprocessing. 129 The aforementioned smoothing effect on velocity magnitude contours, when plotted for the standing waves on November 16, 2021 in Figure 5.28, shows that averaging filter also introduces an over-smoothing effect the downward wave slopes but to a lesser extent than the IAs of LSPIV. Only at a size of 48px (Figure 5.28f, or case PP64-F48) does the smoothing appear at the upstream reach, and even at the largest size of 128px (Figure 5.28i, or case PP64-F128) those dots of high velocity can still be distinguished. On the other hand, when the filter size is too small, such as in Figure 5.28a for size 4px, it becomes insufficient for noise reduction and unreasonably high velocities appear in the bottom right corner. But from case “PP64-F4” in Figure 5.26, transect velocities appear not to be affected.130 Figure 5.28. Farneback velocity contour plots for Site 1 video on 2021-11-16 under different filter sizes: a) to i) are assigned to increasing IA sizes from 4px to 128px. 131 5.9.3 Freshet videos Figure 5.29. LSPIV surface velocity vectors and contour plots for video on 2022-06-16 for combinations PP2-IA4 (a,b) and PP2-IA12 (c,d). The video from 2022-06-16 was used to further investigate the effect of parameter tuning on surface velocity, as its flow contained less visible surface features than in other videos. Analyses were performed on all parameter combinations, four of which were selected for 132 visualization in this subsection: CLAHE block sizes (PP) at 2px and 32px, IA or Filter size at 4px or 12px. For LSPIV, when CLAHE block size was fixed at 2px, decreasing IA size resulted in erroneous or missing velocity vectors, especially in the middle section of the river, where surface features were lacking. The plots in Figures Figure 5.29a and 5.29b show at the section of river farther from the camera, there exist patches of missing velocities surrounded with large areas of vectors in reversed direction. As the river was flowing in a relatively homogeneous direction towards the Northwest, such result is incorrect even through a cursory visual inspection. The missing velocities, expressed in the velocity matrices as “NaN” (Not a Number), resemble what was reported in Fujita & Kunita (2011). It was likely caused by the cross-correlation algorithm failing to determine the movement due to too few features present in the interrogation area. The situation improved when IA was sized at 12px, as there are no missing or reversed flow vectors in the flow, shown in Figures Figure 5.29c and Figure 5.29d. Increasing the IA size encompassed more features for the algorithm to calculate correlation values for determining movement. A similar phenomenon was also observed for the cases under 32px CLAHE block size, showing vectors in a reversed direction in Figure 5.30a at 4px IA, and a more homogeneous pattern at 12px in Figure 5.30c. However, there are no missing velocities as in Figures 5.30a and 5.30b, suggesting applying smaller CLAHE sizes under insufficient surface features create excessive image noise that can hinder the effectiveness of the PIV algorithm. Though the video recorded a relatively high flow, the lack of surface features is similar to low flows mentioned by Ljubičić (2023), in which applying CLAHE alone generated unnecessary image noise. 133 Figure 5.30. LSPIV surface velocity vectors and contour plots for video on 2022-06-16 for combinations PP32-IA4 (a,b) and PP32-IA12 (c,d). When the same parameters were applied in Farneback analyses, the response on surface velocity was different to that from LSPIV. Under a smaller filter size for both CLAHE block sizes of 2px and 32px, shown in Figures Figure 5.31a and Figure 5.32a respectively, some vectors in the middle section exhibit unreasonably high surface velocity, which is likely caused 134 by image noise not being sufficiently reduced by the small filter. Unlike LSPIV, only a few vectors exhibit a reversed direction against the flow. When both CLAHE block and filter sizes were increased, as in case PP32-F12, the vectors and magnitude plotted in Figures Figure 5.32c and Figure 5.32d closely resemble that from Section 5.3, which is at a larger filter size. Figure 5.31. Farneback surface velocity vectors and contour plots for video on 2022-06-16 for combinations PP2-F4 (a,b) and PP2-F12 (c,d). 135 Figure 5.32. Farneback surface velocity vectors and contour plots for video on 2022-06-16 for combinations PP32-F4 (a,b) and PP32-F12 (c,d). Decreasing CLAHE block size appears to have exacerbated the image noise issue in Farneback velocities. Under a filter size of 4px, maximum velocity magnitude was at 9.1325 m/s for CLAHE block size of 2px (Figure 5.31b), and 3.0133 m/s for 32px (Figure 5.32b). The former is more than three times higher, even given the latter is already considered as unreasonably high for this specific case. Increasing filter size to 12px, as in case PP2-F12, only 136 reduced the maximum to 3.3086 m/s in Figure 5.31d, while it still lies in the middle section, making it the result of image noise. This meant that Farneback method is more prone to image noise to LSPIV, and averaging filters are not as effective as interrogation areas in noise reduction. Using CLAHE at smaller block sizes worsened the problem, and it could not be effectively mitigated by increasing the filter to the same size as the IA in LSPIV. 5.10 Edges and image complexity Figure 5.33. Edge pixels for frame 100/298 in video on 2021-11-15, zoomed in to the full region of interest. Edge pixels for the frame are at 16.43%. When the edge pixels were plotted, such as the example shown in Figure 5.33, the edge pixels appear to trace the outer edges of ripples and waves, and likely changes in the water’s colour from different sediment concentrations. All of them are surface features of flooding flows classified in Fujita & Kunita (2021), indicating that edge pixels have the potential for 137 visualization of the surface features and the eventual use as a metric in measuring the density of surface features. Further investigation using the same video but under a decreased resolution resulted in a higher edge pixel percentage. For Site 1’s video on 2021-11-15, the averaged edge pixel ratio was 15.80% for 4K resolution, but increased to 24.55% for HD resolution (frame size 1920px*1080px). The increase was less than twice, the ratio at which the frame sizes decreased. Meanwhile, the approximate decrease in the number of pixels in the frame was 4 times, as both the length and width were halved. This suggests when more pixels are present as video resolution increases, more details become visible, and therefore do not necessarily constitute a linear or quadratic decrease. Moreover, this difference in ratios indicate that such metric should be compared under the same frame size for consistency. When the atmospheric river and freshet videos were compared for their edge pixel ratios, under the same frame sizes, the different flows and their surface velocities were reflected in the edge pixel ratios, summarized in Table 5.9. The atmospheric river videos, rich with complex features on the surface, had ratios at a minimum of 3 times higher than all freshet videos, which, along with the results in Section 5.9, validated the initial hypothesis that more surface features allow for the use of smaller interrogation areas or filters. Table 5.9. Averaged edge pixel ratios for all five videos at site 1. Date 2021-11-15 2021-11-16 2022-05-22 2022-06-16 2022-06-29 Averaged edge pixel ratio (%) 24.55 26.84 8.49 2.91 1.77 Inspecting the freshet videos’ edge pixels showed their distribution, illustrated in Figure 5.34, was not as homogeneous than the atmospheric river counterparts (Figure 5.33). Especially 138 for 2022-06-16 in Figure 5.34b, the edges were more concentrated towards the left bank closer to the camera, with some clustered within the tree shadow slightly further. The middle section was almost devoid of edges due to sunlight reflection, explaining why surface velocities in that area resulted in more errors for both LSPIV and Farneback in Section 5.9.3. This suggests that lighting, in addition to flow and velocity, has an influence on edge pixels. Other possible factors include camera tilt angle, which in these videos, resulted in disproportionately distributed edges in oblique views, as areas closer to the camera have more edges than the those farther away. Figure 5.34. Examples of edge pixels in freshet videos: on 2022-05-22 (a) and on 2022-06-16 (b). These preliminary results show the potential of edge pixels for optimizing image processing parameters such as in Section 5.9. Processing speed on the stabilized HD grayscale videos was around 30-40 seconds for a length of 300-380 frames, making this technique a relatively fast addition to the existing workflow to adapt for optimizing IA or filter sizes, such as in Yeh et al (2019). 139 Chapter 6: Discussion This chapter discusses the accuracy and uncertainty of the results, effectiveness the two image velocimetry methods, the two types of orthorectification, surface hydraulic features, and applications of image velocimetry on citizen videos. 6.1 Accuracy and uncertainty of the measurements Applying image velocimetry techniques to the smartphone videos on Site 1 yielded instantaneous discharge measurements at a maximum 15-20% difference compared WSC’s estimate. This is similar to the 20% under “good conditions of operation” specified in Le Coz et al (2010), but higher than Dramais et al (2011)’s 5-10% under “favourable flood conditions”. On the contrary, the range of uncertainties between 23% to 47% is significantly higher than previous studies, caused by deviations in estimating water level from the images. The averaging of all transect discharges attributed to the acceptable level of accuracy. But as Le Coz et al (2010) noted a maximum of 80% for poor videos, the findings in this study are still acceptable. Since the conditions in Merritt were adverse, with a lack of good GCPs and videos recorded by citizens without prior exposure to image velocimetry, the accuracy level in this study has met expectations, therefore proving the versatility of these methods. The accuracy level between the discharges on November 15th and 16th shows external factors, apart from errors in surveying and outdated geodata, are likely causing a higher difference to the measurements during and after the event, 17% vs 2% respectively. Satellite imagery on November 16th (Figure 3.1) and part of a video by Irnie (2021) show overland flow occurring north of Site 1 (Figure 6.1), suggesting not the entire flow was visible in the videos 140 and thus the actual instantaneous flow through the bridge should be closer to what was measured. Figure 6.1 Likely flow paths including overland flow away from Site 1. Red circle indicates the approximate location where Irnie (2021)’s video was recorded. Drawn over Sentinel-2 image taken from Copercius Open Access Hub. (https://scihub.copernicus.eu/) Deviations in discharges are also caused by bathymetry changes. Rapid changes in the riverbed during extreme flows cause great uncertainty (Dramais et al 2011) which demands onsite surveying, as performed in Stumpf et al (2016). Due to the actual surveying performed after two high flow seasons, there has been sediment deposited in the middle of the downstream transects (CS#4 and CS#5), while both banks experienced erosion. In theory, if the transects were surveyed shortly after the flood, those estimates would be even closer to the actual. Among the tested processing parameters, stage estimation had the most influence on discharges, with a 10cm change in WSE resulting in a 5% deviation on discharge estimate. This 141 corroborates findings from Le Boursicaud et al (2016) and Dramais et al (2011), that specific deviations in discharge may vary depending on the river’s aspect ratio. Though orthorectification presented substantial influence on velocity measurements, noted in Dramais et al (2011) as the most influential, the exact deviation in surface velocity caused by orthorectification errors remains unknown due to the challenges in this step. But the experience from Site 1 videos shows that determining the GCPs’ image coordinates introduces more uncertainties than actual surveying errors, as natural objects are difficult to pinpoint pixel-wise and it is easy to for the pixel coordinates to deviate. It should be noted that even the best videos featured in this study had errors that make them only suited for visualization and less accurate work (ABoM, 2021), which is actually the nature of citizen videos used for post-event reconstructions. Surprisingly, though 2D orthorectification does not require stage elevations as an input, and consequently not affecting surface velocity, the discharge results from Table 5.8 still implied there were potentially minor differences on surface velocity, similar to the cases under 3D orthorectification. The issue with multiple lenses in the same mobile device and lack of lens data in video encoding posed an unexpected challenge, but camera calibration presented minimal influence (less than 1%) on discharge estimates, likely because the water surface was at a long distance away, and consequently the entire flow area was well-centred in the frames. However, this issue will be more critical in the future as smartphones with multiple cameras are being more ubiquitous. 6.2 Effectiveness of the velocimetry methods The workflow commonly used in LSPIV was applied to Farneback in this study. When the image processing parameters are similar, results between LSPIV and Farneback are very close to each other, with the latter slightly higher, at a maximum 4% difference in discharge. 142 LSPIV results from both iPhone videos indicated underestimates in discharge, similar to the findings in Tsubaki et al (2011) and Yagi et al (2020). Though LSPIV’s high sensitivity to changes in surface patterns could be the cause, such a small difference, along with sufficient amount of surface features under flooding and oblique perspectives, indicating the likely cause to be the different resolution in velocity fields. This demonstrates that Farneback method is effective in image velocimetry of extreme flows, whose ample surface features provide sufficient tracers for generating dense velocity fields. The different techniques in velocity calculation are reflected through their behaviour under different parameters, with Farneback more sensitive to image noise than LSPIV, but less sensitive to over-smoothing surface velocities with increasing filter size (or IA size for LSPIV). When CLAHE was not applied before Farneback processing, peak velocity was reduced in areas closer to the camera, implying the necessity of contrast enhancement for this method, similar to LSPIV. However, in freshet videos with less surface features, applying CLAHE at small block sizes, induced additional image noise that increased errors for both LSPIV and Farneback results. Care should be taken in the selection of block sizes, especially when less complex features are present on the flow surface. When flows exhibit less surface features, using Farneback resulted in substantial underestimates (Hutley et al, 2023; Ansari et al, 2023). This is somewhat different to the results in Section 5.9.3, as decreasing surface complexity resulted in increasing image noise and therefore required larger filter sizes. Similar tests should be applied on different videos, preferably at calibrated sites, for better understanding of tuning parameters on the Farneback method. A large number of samples, preferably generated through Monte Carlo simulation 143 similar to Le Coz et al (2021), could be used to test the behaviour of Farneback surface velocities under different parameters, especially image coordinates of GCPs. The lack of existing hydrometric stations, staff gauges, or elevation references at site 1 called for the use of waterline estimation from the image, resulting in high amounts of uncertainty to discharges. In this case, the waterlines were more difficult to define due to obstructions from trees. This method is independent of velocimetry, with its reliance on accuracy of orthorectification introduces additional errors. Interestingly, due to erosion at the banks, manual definition of waterlines proved to be more effective on overbank flows, as the area was previously not inundated and thus able to have existing DEMs applied to obtain WSEs for stage and discharge estimation. 6.3 Orthorectification types The results from “pre-analysis” orthorectification, in Section 5.8, show substantial inconsistencies on velocities between LSPIV and Farneback, and even under the same method between “pre-analysis” and “post-analysis” orthorectification types. Bomhof (2021) reported discharge differences in the range of 5-10% between Fudaa-LSPIV and PIVlab-RIVeR, representing the two different types of orthorectification implemented in their workflows. Ansari et al (2023) also reported similar differences in surface velocity using videos collected from the same source, noting this small difference was possible thanks to sufficient spatial resolution in both nadir and oblique images. Conversely, when videos are recorded in extremely low angles, as in the case at Site 1, the distortion correction in orthorectification removes many detailed surface features present in the flow area. This effect appears to be more profound than that from simulated images in Highham & Brevis (2019), as natural flow structures are more subtle in pixel intensity differences and consequently less distinguishable. 144 Figure 6.2. Original frame (a) and orthorectified frame (b) from the video on 2021-11-15. Due to the drastic reduction in the sizes of orthoimages, they were enlarged through cubic interpolation of the grayscale pixels. Shown in Figure 6.2b, the orthoimage’s size reduced from the original’s 3840px*2160px to 206px*157px. The enlargement exacerbated the pixelation of the flow area and further magnified the errors. Specifically, the pixelation introduced excessive noise to the images, resulting in unreasonable measurements through Farneback, as it detects movements of every single pixel. Removing this step could mitigate those errors, but introduces additional challenges in analysis, as interrogation areas or filters of the same pixel size now occupy larger portions of the orthoimage. The aforementioned issue was not present in the “post-analysis” type, as both methods were used on the original frames, and therefore able to produce measurements with relatively minimal difference. Overall, the workflow involving “post-analysis” orthorectification proved to be effectual under challenging circumstances for geodata collection. Apart from reducing computational time as the transformation of vectors requires a lot less time than generating orthophotos (Patalano et al, 2017), it also allows the raw velocities to be unaltered while rectifications could be attempted with unlimited iterations. 145 With high uncertainties in georeferencing and potentially in surveying, caused by poor quality of citizen videos or lack of geodata, this type of orthorectification is a conservative, yet practical consideration in applications involving citizen videos as they are expected to be from oblique perspective, and camera tilt angles are usually unable to be optimized. Thus, it is better to not introduce additional errors, especially from orthorectification, into the results. In contrast, the conventional “pre-analysis” workflow will not be able to proceed past stabilization due to a lack of GCP coordinates, or produce erroneous orthorectified frames. However, processing oblique videos in their raw form results in differential spatial resolutions, complicating detailed analyses. If the accuracy of orthorectification could be maintained to a high level, “pre-analysis” orthorectification could be used, for a consistent spatial resolution that is necessary for analysis (Le Coz et al, 2021). 6.4 Surface hydraulic features and edges The videos presented also contain complex surface flow structures, including standing waves and oblique waves, which could be studied separately. The standing waves on November 16th, introduced bias to surface velocity measurements as well as complications to estimating peak discharges. Specifically, the riverbed profile at the wave locations should be different to cause the flow to form standing waves. Unfortunately, the exact conditions during the time of the video are unknown as no simultaneous measurements took place. This is likewise true for oblique waves, as significant scouring changed the surrounding landscape and consequently rendered the validation data inaccurate, therefore unable to use the wave front angle to estimate the depth of flow. Their closest imitations would have to be performed in flumes. 146 Comiti & Lenzi (2006) concluded that the wavelengths of undular hydraulic jumps at artificial sills are correlated to the Froude number. Tinkler (1997) concluded that measuring wavelengths of standing wave trains at rock bed rivers leads to accurate estimation of mean velocity and minimum discharge. These findings have great potential to be used to reduce the errors in estimating of velocity and discharge around hydraulic jumps and standing waves, however, site-specific conditions should be taken into consideration, especially given the high likelihood of sediment transport and bed scour during floods. The estimation of mean velocity coefficient (alpha) requires detailed considerations on hydraulic and riverbed conditions. While the estimated value of 0.9 was reasonable to be applied for all the transects during the atmospheric river flows, the vertical-specific values may differ due to bed widths and waves. The latter should have a higher stage and therefore a different alpha than 0.9, but such changes were neglected during discharge calculations. Johnson & Cowen (2016) developed a method to remotely estimate the alpha coefficient through surface turbulence metrics, and tested in various flow conditions in a lab flume. This is a promising prospect of which applications to field environments are possible, with the eventual goal of improving the accuracy of discharge estimates, albeit introducing many additional variables. It was verified in this study that edge pixel ratios could be used as metric on the amount of surface features in the flow, and consequently to facilitate tuning of image processing parameters. Preliminary results from Section 5.10 show being the result of pixel intensity gradients, edge pixels could be affected by the flow’s turbulence, turbidity, background lighting, and camera tilt angle, all of which are critical variables in video data collection. The first two coincide with Fujita & Kunita (2011)’s classification of surface features of flooding flows, making edge pixels a valid alternative to tracers visualized through image binarization and/or 147 artificial placements. However, those pixels only trace the “edges” of features, and are susceptible to changing video resolutions, so further investigations on this topic should note their influence on the ratios and subsequent effects on parameter tuning. 6.5 Applications and recommendations As demonstrated in previous studies, citizen videos are a valuable source in post-event peak estimates. Though requiring more corrections and thus unsuitable for operational deployment (Jamieson, 2023), citizen videos could still be used as a stopgap, especially in areas where extreme events not captured in a timely manner. Such gaps in monitoring are particularly large in Canada, especially in remote areas. Though WSC already has started testing remote sensing in sites across Canada, including a surface velocity radar installed in Merritt. But it took place in April 2022 (Jamieson et al, 2023), months after the flood had passed. This meant the citizen videos were likely the only source able to capture parts of the actual flooding event, and with extreme events happening more frequently in the future, similar occurrences with citizen videos as stopgaps are very likely to increase in number. A common limitation with images is the dependance on lighting and consequent inability to capture flows at nighttime (Fujita & Konita, 2011). In Merritt’s case, the estimated peak occurred after dusk, meaning videos at that time, if any, would be extremely difficult to process due to insufficient lighting. Also, since the banks were already breached upstream of the city area, what most citizen videos covered were only parts of the flooding flow instead of its entirety, with their surface velocity results able to be used to validate those from hydraulic simulations. Though it proved to be possible to carry out this study with minimal involvement from all levels of government (federal, provincial, and municipal), while purely using videos from onsite 148 witnesses and publicly available geodata, the amount of effort spent on correcting movements and identifying GCPs made it difficult to be widely applied without extensive training or time-consuming processing. Improving the video quality by stabilizing the device and including visible fixed objects should require less training and effort, but greatly reduce the amount of work needed for processing. Surface velocity on drone videos provided opportunities to implement image velocimetry for onsite flood mapping and emergency response. In addition to the aforementioned damage inspection featured in the original videos, drones could be flown over the flow area for several minutes to measure surface velocity. Such information could then be used in determining imminent risks to critical infrastructure, including scouring and/or failure of bridges (Hauet et al, 2009), highways, and dikes, such as the case in Section 5.2. The recommendations from the study of citizen videos are summarized as follows: • For extreme flows, Farneback method can be applied in the existing workflow with minimal adjustment; • CLAHE should be applied before Farneback processing, regardless of block size, but preferably at larger sizes to minimize processing time and image noise, especially for flows exhibiting less complex surface structure; • When tuning image processing parameters, especially interrogation areas or filters, as a rule of thumb, the less features present on the surface of the flow, the larger the parameter should be. Edge pixels ratio could be used to measure the amount of surface features present, and as a rule of thumb for tuning image processing parameters. • When latest geodata could not be collected or existing data exhibits high amount of uncertainty, “post-analysis” orthorectification workflow should be used; 149 • For smartphone videos, if the device has more than one lens and camera calibration needs to be performed, the specific one used in filming should also be recorded in addition to the exact time. Videos taken with fisheye lens should be corrected for distortion as per Le Boursicaud (2015; 2016). • Normal parameters for video cameras on smartphones are sufficient for image velocimetry, e.g., 30 frames per second and HD resolution. If desired, 4K resolution can be used, but preferably not at a higher frame rate to prevent excessive computing time. 150 Chapter 7: Conclusion and future work In this thesis, citizen videos on the November 2021 atmospheric river event in Merritt were collected for measurement of surface velocity and discharge. Videos recorded during the freshet season of 2022 were also used for comparison during parameter turning. Two methods for image velocimetry were used in the same processing workflow: LSPIV and Farneback optical flow. Despite the challenges in site access and data collection, the methods were successfully applied using outdated geodata in part, producing discharge measurements with acceptable accuracy levels with up to 20% difference against Water Survey of Canada’s estimate. The application of Farneback method to videos on extreme flows was also successful, with its results similar to those of LSPIV under similar parameters in processing. However, when the main processing parameter, interrogation area for LSPIV and averaging filter for Farneback, was adjusted for flows with less surface features, their velocity results changed as the two methods used different mechanisms to determine movement, with the latter more easily affected by image noise. The variation of other parameters in processing produced results similar to previous studies, except in estimating water surface elevation, whose results under 2D orthorectification were similar to those in 3D, with differences in surface velocity. Uncertainty levels in the measured discharges were higher due to deviations in estimating the waterline from the images. Accurate estimations of the stage are critical in minimizing the uncertainties in discharge. The results of this study further proved the versatility of image velocimetry methods and the worth of citizen videos in gauging extreme flows. With extreme events occurring more frequently due to climate change, and more advanced devices are becoming more accessible to 151 citizens, citizen videos will increase in value as complements or stopgaps to existing hydrometric monitoring efforts. The lack of up-to-date DEMs and high uncertainties in orthorectification could be mitigated with conservative measures incorporated into the processing workflow. It should be noted that given citizen videos tend to have lower tilt angles that result in pixelation of the frames after orthorectification, and therefore rectifying the original vectors, not the images, ensures more accurate results. Local governments could also implement imaged-based methods, coupled with existing geospatial data, in real-time flood mapping for emergency response and risk assessment. Though it proved to be possible to carry out studies without collaboration from all levels of government, future work in this area should focus on a consistent and coordinated approach, eventually having citizen observers or municipal staff producing better quality videos, as well as compiling standardized procedures in improving onsite disaster response actions. The custom code will also need to be streamlined and possibly implemented in python, so it could be freely available and wouldn’t rely on costly MATLAB licenses to run. Another direction in improving the quality of citizen videos is stereo imaging using two separate devices, with both focusing on the river flow. Briefly mentioned in Le Boursicaud et al (2016), this technique has seen seldom applications, likely due to the difficulty in synchronization. A possible way to solve it is to introduce a sound at a certain frequency to measure or calibrate for the approximate delay between the two viewers. Furthermore, automated edge detection methods can be used in improving the estimation of the waterline directly from images, leading to the eventual automation of image-based stream gauging and real-time update on rating curves. 152 Bibliography ABoM (Australian Bureau of Meteorology). (2021, October). National Industry Guidelines for hydrometric monitoring, Part 11: Application of Surface Velocity Methods for Velocity and Open Channel Discharge Measurements. (Publication No. NI GL 100 11-2021). http://www.bom.gov.au/water/standards/documents/NI_GL_100.11-2021.pdf Al-mamari, M., Kantoush, S., Kobayashi, S., Sumi, T., & Saber, M. (2019). Real-time measurement of flash-flood in a wadi area by LSPIV and STIV. Hydrology, 6(1), 27. https://doi.org/10.3390/hydrology6010027 Anandan, P. (1989). A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision, 2(3), 283–310. https://doi.org/10.1007/BF00158167 Ansari, S., Rennie, C. D., Jamieson, E. C., Seidou, O., & Clark, S. P. (2023). Rivqnet: Deep learning-based river discharge estimation using close-range water surface imagery. Water Resources Research, 59(2), e2021WR031841. https://doi.org/10.1029/2021WR031841 Apple Inc. (2021, April 26). iPad Air (3rd Generation) - Technical Specifications. Official Apple Support. https://support.apple.com/kb/SP787?locale=en_US Apple Inc. (2022, July 15). iPhone 11 Pro Max - Technical Specifications. Official Apple Support. https://support.apple.com/kb/SP806?locale=en_US 153 Associated Engineering (B.C.) Ltd. (2022, November). City of Merritt. Final Report. Flood Mitigation Plan. Project Number 2021-2385-02. https://pub-merritt.escribemeetings.com/filestream.ashx?DocumentId=8181 Atkins, M. D. (2016). Chapter 5 – Velocity Field Measurement Using Particle Image Velocimetry (PIV). In T. Kim, T. J. Lu, & S. J. Song (Eds.), Application of Thermo-Fluidic Measurement Techniques: An Introduction (pp. 125 – 166). Elsevier. https://doi.org/10.1016/C2015-0-01881-0 Barron, J. L., Fleet, D. J., & Beauchemin, S. S. (1994). Performance of optical flow techniques. International Journal of Computer Vision, 12(1), 43–77. https://doi.org/10.1007/BF01420984 BC MoTI (British Columbia Ministry of Transportation and Infrastructure). (2023, March 21). B.C. Highway Flood Recovery Projects - Highway 5 – Coquihalla. B.C. Highway Flood Recovery Projects. https://www2.gov.bc.ca/gov/content/transportation-projects/bc-highway-flood-recovery/2021-flood-road-recovery-projects-highway-5 Bodart, G., Le Coz, J., Jodeau, M., and Hauet, A. (2022, May 23-27). Quantifying the operator effect in LSPIV image-based velocity and discharge measurements. EGU General Assembly 2022, Vienna, Austria and online. https://doi.org/10.5194/egusphere-egu22-4457 Bomhof, J. (2021, May 4-6). Preliminary Comparison of Image Velocimetry Software and Methods. International Surface Velocimetry Workshop 2022. Online. 154 Bomhof, J. (2022, October 3-5). Investigation of Image Velocimetry techniques using Unmanned Aerial Vehicles for discharge measurements in Canadian Rivers. International Surface Velocimetry Workshop 2022, Lancaster, United Kingdom and online. Bung, D., Valero, D. (2016a). Application of the Optical Flow Method to Velocity Determination in Hydraulic Structure Models. In B. Crookston & B. Tullis (Eds.), Hydraulic Structures and Water System Management. 6th IAHR International Symposium on Hydraulic Structures, Portland, OR, 27-30 June (pp. x-x). doi:10.15142/T3150628160853 (ISBN 978-1-884575-75-4). Bung., D. B., & Valero, D. (2016b). Image processing techniques for velocity estimation in highly aerated flows: Bubble Image Velocimetry vs. Optical Flow. In S. Erpicum, B. Dewals, P. Archambeau, & M. Pirotton (Eds.), Sustainable Hydraulics in the Era of Global Change. Proceedings of the 4th IAHR European Congress (Liege, 2016) (pp. 151-157). doi: 10.1201/b21902-31. Bung, D. B., & Valero, D. (2016c). Optical flow estimation in aerated flows. Journal of Hydraulic Research, 54(5), 575–580. https://doi.org/10.1080/00221686.2016.1173600 Carrel, M., Peña-Haro, S., Lüthi, B., Hansen, I. (2021). Image-based flow monitoring in a remote alpine catchment. 14th Congress INTERPRAEVENT 2021, Norway. Carrel, M., Detert, M., Peña-Haro, S., Luethi, B. (2019, February 26 – March 1). Evaluation of the DischargeApp: A Smartphone Application for Discharge Measurements. HydroSenSoft, International Symposium and Exhibition on Hydro-Environment Sensors and Software. Madrid, Spain. 155 CBC news. (2021, November 18). These graphics show just how much record-breaking rain hit southern B.C. recently. CBC News – British Columbia. https://www.cbc.ca/news/canada/british-columbia/bc-record-rainfall-graphics-1.6254271 Chung, E. (2021, November 18). Why are the B.C. floods so bad? Blame the wildfires, at least in part. CBC News – Science. https://www.cbc.ca/news/science/post-wildfire-floodring-1.6253544 City of Merritt. (2022). 2021 Flood Review + Repair. https://www.merritt.ca/flood/ City of Merritt. (2020). Maps. Retrieved August 26, 2022, from https://www.merritt.ca/maps/ Comiti, F., & Lenzi, M. A. (2006). Dimensions of standing waves at steps in mountain rivers. Water Resources Research, 42(3). https://doi.org/10.1029/2004WR003898 Dabiri, D., & Pecora, C. (2019). Particle Tracking Velocimetry. IOP Publishing. https://iopscience.iop.org/book/mono/978-0-7503-2203-4 Dal Sasso, S. F., Pizarro, A., Pearce, S., Maddock, I., & Manfreda, S. (2021). Increasing LSPIV performances by exploiting the seeding distribution index at different spatial scales. Journal of Hydrology, 598, 126438. https://doi.org/10.1016/j.jhydrol.2021.126438 Detert, M. (2021). How to avoid and correct biased riverine surface image velocimetry. Water Resources Research, 57(2). https://doi.org/10.1029/2020WR027833 Dingman, S. L. (2009). Fluvial hydraulics. Oxford University Press. 156 Dramais, G., Le Coz, J., Camenen, B., & Hauet, A. (2011). Advantages of a mobile LSPIV method for measuring flood discharges and improving stage–discharge curves. Journal of Hydro-Environment Research, 5(4), 301–312. https://doi.org/10.1016/j.jher.2010.12.005 Eltner, A., Sardemann, H., & Grundmann, J. (2020). Technical Note: Flow velocity and discharge measurement in rivers using terrestrial and unmanned-aerial-vehicle imagery. Hydrology and Earth System Sciences, 24(3), 1429–1445. https://doi.org/10.5194/hess-24-1429-2020 Fannin, J., Kim, A., McDougall, S., Weijs, S., & Wijewickerme, D. (2022, January/February). Landslides, Floods and Infrastructure Impacts. Innovation, 26 (1), 34-41. Farber, R. (2011). CUDA Application Design and Development. Elsevier Science. Farnebäck, G. (2003). Two-Frame Motion Estimation Based on Polynomial Expansion. In: Bigun, J., Gustavsson, T. (eds) Image Analysis. SCIA 2003. Lecture Notes in Computer Science, vol 2749. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45103-X_50 Fleet, D. J., & Jepson, A. D. (1990). Computation of component image velocity from local phase information. International Journal of Computer Vision, 5(1), 77–104. https://doi.org/10.1007/BF00056772 Fohringer, J., Dransch, D., Kreibich, H., & Schröter, K. (2015). Social media as an information source for rapid flood inundation mapping. Natural Hazards and Earth System Sciences, 15(12), 2725–2738. https://doi.org/10.5194/nhess-15-2725-2015 157 Forsythe, A. (2009). Visual Complexity: Is That All There Is? In: Harris, D. (eds) Engineering Psychology and Cognitive Ergonomics. EPCE 2009. Lecture Notes in Computer Science, vol 5639. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02728-4_17 Fujita, I., & Aya, S. (2000). Refinement of LSPIV technique for monitoring river surface flows. Building Partnerships, 1–9. https://doi.org/10.1061/40517(2000)312 Fujita, I., Aya, S., & Deguchi, T. (1997). Surface velocity measurement of river flow using video images of an oblique angle. In: T. Carstens & S. S. Y. Wang (Eds.), Environmental and Coastal Hydraulics: Protecting the Aquatic Habitat (pp. x-x). American Society of Civil Engineers, New York, NY. Fujita, I., & Komura, S. (1994). Application of video image analysis for measurements of river-surface flows. Proceedings of Hydraulic Engineering (in Japanese), 38, 733–738. https://doi.org/10.2208/prohe.38.733 Fujita, I., & Kunita, Y. (2011). Application of aerial LSPIV to the 2002 flood of the Yodo River using a helicopter mounter high density video camera. Journal of Hydro-environment Research, 5, 323-331. Fujita, I., Kunita, Y. & Tsubaki, R. (2013). “Image analysis and reconstruction of the 2008 Toga River Flash Flood in an urbanised area”, Australian Journal of Water Resources, 16(2), 151-162, http://dx.doi.org/10.7158/W12-011.2013.16.2. 158 Fujita, I., Muste, M., & Kruger, A. (1998). Large-scale particle image velocimetry for flow analysis in hydraulic engineering applications. Journal of Hydraulic Research, 36(3), 397–414. https://doi.org/10.1080/00221689809498626 Fujita, I., Notoya, Y., Furuta, T. (2018). Measurement of Inundating Flow from a Broken Embankment by Using Video Images Shoot from a Media Helicopter. Proceedings of the 9th River Flow (Lyon, 2018). https://doi.org/10.1051/e3sconf/20184006001 Fujita, I., Watanabe, H., & Tsubaki, R. (2007). Development of a non-intrusive and efficient flow monitoring technique: The space-time image velocimetry (STIV). International Journal of River Basin Management, 5(2), 105–114. https://doi.org/10.1080/15715124.2007.9635310 Google. (2007, July 19). [Google Earth Merritt, BC]. Retrieved August 26, 2022. Google. (2022, July 30). [Google Earth Merritt, BC]. Retrieved October 15, 2022. Greg InBC. (2022, November 14). Anniversary - 2021 Merritt flooding (Created for 2022 UBCM conference). [Video]. YouTube. https://youtu.be/FSYtDSiyqq0?si=lTwPVJfxTE4IF7Ba Greg InBC. (2021, November 16). City of Merritt - 2021-11-16. [Video]. YouTube. https://youtu.be/O6hQL3se_U8?si=4r2FESZaJIsZxuzJ Guillén, N. F., Patalano, A., García, C. M., & Bertoni, J. C. (2017). Use of LSPIV in assessing urban flash flood vulnerability. Natural Hazards, 87(1), 383–394. https://doi.org/10.1007/s11069-017-2768-8 159 Han, X., Chen, K., Zhong, Q., Chen, Q., Wang, F., & Li, D. (2021). Two-dimensional space-time image velocimetry for surface flow field of mountain rivers based on UAV video. Frontiers in Earth Science, 9, 686636. https://doi.org/10.3389/feart.2021.686636 Hauet, A., Creutin, J.-D., & Belleudy, P. (2008a). Sensitivity study of large-scale particle image velocimetry measurement of river discharge using numerical simulation. Journal of Hydrology, 349(1–2), 178–190. https://doi.org/10.1016/j.jhydrol.2007.10.062 Hauet, A., Kruger, A., Krajewski, W. F., Bradley, A., Muste, M., Creutin, J.-D., & Wilson, M. (2008b). Experimental system for real-time discharge estimation using an image-based method. Journal of Hydrologic Engineering, 13(2), 105–110. https://doi.org/10.1061/(ASCE)1084-0699(2008)13:2(105) Hauet, A., Morlot, T., & Daubagnan, L. (2018). Velocity profile and depth-averaged to surface velocity in natural streams: A review over a large sample of rivers. E3S Web of Conferences, 40, 06015. https://doi.org/10.1051/e3sconf/20184006015 Hauet, A., Muste, M., & Ho, H.-C. (2009). Digital mapping of riverine waterway hydrodynamic and geomorphic features. Earth Surface Processes and Landforms, 34(2), 242–252. https://doi.org/10.1002/esp.1709 Heeger, D. J. (1988). Optical flow using spatiotemporal filters. International Journal of Computer Vision, 1(4), 279–302. https://doi.org/10.1007/BF00133568 Henderson, F. M. (1995). Open Channel Flow (Nachdr.). Prentice Hall. 160 Higham, J. E., & Brevis, W. (2019). When, what and how image transformation techniques should be used to reduce error in Particle Image Velocimetry data? Flow Measurement and Instrumentation, 66, 79–85. https://doi.org/10.1016/j.flowmeasinst.2019.02.005 Horn, B. K. P., & Schunck, B. G. (1981). Determining Optical Flow. Artificial Intelligence, 17(1–3), 185–203. https://doi.org/10.1016/0004-3702(81)90024-2 Hutley, N. R., Beecroft, R., Wagenaar, D., Soutar, J., Edwards, B., Deering, N., Grinham, A., & Albert, S. (2023). Adaptively monitoring streamflow using a stereo computer vision system. Hydrology and Earth System Sciences, 27(10), 2051–2073. https://doi.org/10.5194/hess-27-2051-2023 Irnie, M. (2021, November 15). Merritt BC Flooded Evacuation Order 4:30pm November 15, 2021 | Irnieracing News. [Video]. YouTube. https://www.youtube.com/watch?v=gwUMPs6ULtI Jodeau, M., Hauet, A., Le Coz, J., Faure, J., Bodart, G., (2022, April 12). Fudaa-LSPIV Version 1.9.2. User Manual. EDF & INRAE. https://forge.irstea.fr/attachments/download/3031/User%20Manual%20Fudaa-LSPIV%201.9.2.pdf Jodeau, M., Hauet, A., Paquier, A., Le Coz, J., & Dramais, G. (2008). Application and evaluation of LS-PIV technique for the monitoring of river surface velocities in high flow conditions. Flow Measurement and Instrumentation, 19(2), 117–127. https://doi.org/10.1016/j.flowmeasinst.2007.11.004 161 Jolley, M. J., Russell, A. J., Quinn, P. F., & Perks, M. T. (2021). Considerations when applying large-scale PIV and PTV for determining river flow velocity. Frontiers in Water, 3, 709269. https://doi.org/10.3389/frwa.2021.709269 Khalid, M., Pénard, L., & Mémin, E. (2019). Optical flow for image-based river velocity estimation. Flow Measurement and Instrumentation, 65, 110–121. https://doi.org/10.1016/j.flowmeasinst.2018.11.009 Khan, S.D., Porta, F., Vizzari, G., Bandini, S. (2014). Estimating Speeds of Pedestrians in Real-World Using Computer Vision. In: Wąs, J., Sirakoulis, G.C., Bandini, S. (Eds.) Cellular Automata. ACRI 2014. Lecture Notes in Computer Science, vol 8751. Springer, Cham. https://doi.org/10.1007/978-3-319-11520-7_55 Kim, Y., Muste, M., Hauet, A., Krajewski, W. F., Kruger, A., & Bradley, A. (2008). Stream discharge using mobile large-scale particle image velocimetry: A proof of concept. Water Resources Research, 44(9). https://doi.org/10.1029/2006WR005441 Le Boursicaud, R., Le Coz, J., Dramais, G. (2015). Guide méthodologique Collecte et préparation de film amateur en vue d’une exploitation par analyse d’image [Methodological guide: Collection and preparation of amateur film for exploitation by image analysis]. IRSTEA. Technical report. Le Boursicaud, R., Pénard, L., Hauet, A., Thollet, F., & Le Coz, J. (2016). Gauging extreme floods on YouTube: Application of LSPIV to home movies for the post-event determination of stream discharges: Application of LSPIV to Flood Home Movies. Hydrological Processes, 30(1), 90–105. https://doi.org/10.1002/hyp.10532 162 Le Coz, J., Patalano, A., Collins, D., Guillén, N. F., García, C. M., Smart, G. M., Bind, J., Chiaverini, A., Le Boursicaud, R., Dramais, G., & Braud, I. (2016). Crowdsourced data for flood hydrology: Feedback from recent citizen science projects in Argentina, France and New Zealand. Journal of Hydrology, 541, 766–777. https://doi.org/10.1016/j.jhydrol.2016.07.036 Le Coz, J., Hauet, A., Pierrefeu, G., Dramais, G., & Camenen, B. (2010). Performance of image-based velocimetry (LSPIV) applied to flash-flood discharge measurements in Mediterranean rivers. Journal of Hydrology, 394(1–2), 42–52. https://doi.org/10.1016/j.jhydrol.2010.05.049 Le Coz, J., Renard, B., Vansuyt, V., Jodeau, M., & Hauet, A. (2021). Estimating the uncertainty of video‐based flow velocity and discharge measurements due to the conversion of field to image coordinates. Hydrological Processes, 35(5). https://doi.org/10.1002/hyp.14169 Le Coz, J., Hauet, A., Le Boursicaud, R., Pénard, L., Bonnifait, L., Dramais, G., Thollet, F., Braud, I. (2015, April 12-17). Estimating flood discharge using witness movies in post-flood hydrological surveys. EGU General Assembly 2015, Vienna, Austria. https://meetingorganizer.copernicus.org/EGU2015/EGU2015-5296.pdf Lee, M., & Parfitt, B. (2022, November). A Climate Reckoning The economic costs of BC’s extreme weather in 2021. Canadian Centre for Policy Alternatives. BC Office. https://policyalternatives.ca/climate-reckoning Legleiter, C. J., & Kinzel, P. J. (2020). Inferring surface flow velocities in sediment-laden alaskan rivers from optical image sequences acquired from a helicopter. Remote Sensing, 12(8), 1282. https://doi.org/10.3390/rs12081282 163 Leitão, J. P., Peña-Haro, S., Lüthi, B., Scheidegger, A., & Moy De Vitry, M. (2018). Urban overland runoff velocity measurement with consumer-grade surveillance cameras and surface structure image velocimetry. Journal of Hydrology, 565, 791–804. https://doi.org/10.1016/j.jhydrol.2018.09.001 Little, S. (2021, November 28). Flood warning issued for Merritt, B.C. as military helps fortify defenses. Global News – Environment. https://globalnews.ca/news/8408547/more-merritt-flooding-november-28/ Liu, W.-C., Lu, C.-H., & Huang, W.-C. (2021). Large-scale particle image velocimetry to measure streamflow from videos recorded from unmanned aerial vehicle and fixed imaging system. Remote Sensing, 13(14), 2661. https://doi.org/10.3390/rs13142661 Ljubičić, R. (2023, May 17). Personal communication. Ljubičić, R., Strelnikova, D., Perks, M. T., Eltner, A., Peña-Haro, S., Pizarro, A., Dal Sasso, S. F., Scherling, U., Vuono, P., & Manfreda, S. (2021). A comparison of tools and techniques for stabilising unmanned aerial system (UAS) imagery for surface flow observations. Hydrology and Earth System Sciences, 25(9), 5105–5132. https://doi.org/10.5194/hess-25-5105-2021 Lucas, B.D., Kanade, T. (1981). An Iterative Image Registration Technique with an Application to Stereo Vision. In: Proceedings of the Seventh International Joint Conference on Artificial Intelligence, pp. 674-679. Morgan Kaufmann Publishers Inc. McIlvenny, J., Williamson, B. J., Fairley, I. A., Lewis, M., Neill, S., Masters, I., & Reeve, D. E. (2022). Comparison of dense optical flow and PIV techniques for mapping surface current flow 164 in tidal stream energy sites. International Journal of Energy and Environmental Engineering. https://doi.org/10.1007/s40095-022-00519-z Modesto, A. (2011, August 22). debris flow - 22 août 2011 - Crue torrentielle à Saint Julien Montdenis. [debris flow – 22 august 2011 - Torrential flood in Saint Julien Montdenis]. [Video]. YouTube. https://youtu.be/jgWbYDUm-ME?si=nbLa6paGG2cRY9Cp Muste, M., Xiong, Z., Schöne, J., & Li, Z. (2004). Validation and extension of image velocimetry capabilities for flow diagnostics in hydraulic modeling. Journal of Hydraulic Engineering, 130(3), 175–185. https://doi.org/10.1061/(ASCE)0733-9429(2004)130:3(175) Muste, M., Fujita, I., & Hauet, A. (2008). Large-scale particle image velocimetry for measurements in riverine environments. Water Resources Research, 44(4). https://doi.org/10.1029/2008WR006950 Ness, R., Clark, D.G., Bourque, J., Coffman D., Beugin D. (2021, September). Under Water: The Costs of Climate Change for Canada’s Infrastructure. Canadian Institute for Climate Choices. Ottawa, ON. Patalano, A., García, C. M., & Rodríguez, A. (2017). Rectification of Image Velocity Results (RIVeR): A simple and user-friendly toolbox for large scale water surface Particle Image Velocimetry (PIV) and Particle Tracking Velocimetry (PTV). Computers & Geosciences, 109, 323–330. https://doi.org/10.1016/j.cageo.2017.07.009 Patalano, A., (2022, October 18). Update RIVeR 2.6 ! #BatchProcessing #ParallelComputing. RIVeR. https://riverdischarge.blogspot.com/2022/10/update-river-26-batchprocessing.html 165 Pearce, S., Ljubičić, R., Peña-Haro, S., Perks, M., Tauro, F., Pizarro, A., Dal Sasso, S., Strelnikova, D., Grimaldi, S., Maddock, I., Paulus, G., Plavšić, J., Prodanović, D., & Manfreda, S. (2020). An evaluation of image velocimetry techniques under low flow conditions and high seeding densities using unmanned aerial systems. Remote Sensing, 12(2), 232. https://doi.org/10.3390/rs12020232 Peña-Haro, S., Lüthi, B., Lukes, R., & Carrel, M. (2020). Wind effect on image-based river surface velocity measurements. EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-9943. https://doi.org/10.5194/egusphere-egu2020-9943. Perks, M. T. (2020). KLT-IV v1.0: Image velocimetry software for use with fixed and mobile platforms. Geoscientific Model Development, 13(12), 6111–6130. https://doi.org/10.5194/gmd-13-6111-2020 Perks, M. T., Dal Sasso, S. F., Hauet, A., Jamieson, E., Le Coz, J., Pearce, S., Peña-Haro, S., Pizarro, A., Strelnikova, D., Tauro, F., Bomhof, J., Grimaldi, S., Goulet, A., Hortobágyi, B., Jodeau, M., Käfer, S., Ljubičić, R., Maddock, I., Mayr, P., … Manfreda, S. (2020). Towards harmonisation of image velocimetry techniques for river surface velocity observations. Earth System Science Data, 12(3), 1545–1559. https://doi.org/10.5194/essd-12-1545-2020 Perks, M. T., Russell, A. J., & Large, A. R. G. (2016). Technical Note: Advances in flash flood monitoring using unmanned aerial vehicles (UAVs). Hydrology and Earth System Sciences, 20(10), 4005–4015. https://doi.org/10.5194/hess-20-4005-2016 166 Pizarro, A., Dal Sasso, S. F., Perks, M. T., & Manfreda, S. (2020). Identifying the optimal spatial distribution of tracers for optical sensing of stream surface flow. Hydrology and Earth System Sciences, 24(11), 5173–5185. https://doi.org/10.5194/hess-24-5173-2020 Ran, Q., Li, W., Liao, Q., Tang, H., & Wang, M. (2016). Application of an automated LSPIV system in a mountainous stream for continuous flood flow measurements: LSPIV for Mountainous Flood Monitoring. Hydrological Processes, 30(17), 3014–3029. https://doi.org/10.1002/hyp.10836 Raudies, F. (2013). Optic flow. Scholarpedia. http://www.scholarpedia.org/article/Optic_flow Rousseau, G.P.D.M. (2019). Turbulent Flows over Rough Permeable Beds in Mountain Rivers: Experimental Insights and Modeling. [Doctoral dissertation, École Polytechnique Fédérale de Lausanne]. InfoScience. https://infoscience.epfl.ch/record/264790 Schweitzer, S. A., & Cowen, E. A. (2021). Instantaneous river‐wide water surface velocity field measurements at centimeter scales using infrared quantitative image velocimetry. Water Resources Research, 57(8). https://doi.org/10.1029/2020WR029279 Shi, R., Leng, X., & Chanson, H. (2020). On optical flow techniques applied to breaking surges. Flow Measurement and Instrumentation, 72, 101710. https://doi.org/10.1016/j.flowmeasinst.2020.101710 Strelnikova, D., Paulus, G., Käfer, S., Anders, K.-H., Mayr, P., Mader, H., Scherling, U., & Schneeberger, R. (2020). Drone-based optical measurements of heterogeneous surface velocity 167 fields around fish passages at hydropower dams. Remote Sensing, 12(3), 384. https://doi.org/10.3390/rs12030384 Streßer, M., Carrasco, R., & Horstmann, J. (2017). Video-based estimation of surface currents using a low-cost quadcopter. IEEE Geoscience and Remote Sensing Letters, 14(11), 2027–2031. https://doi.org/10.1109/LGRS.2017.2749120 Stumpf, A., Augereau, E., Delacourt, C., & Bonnier, J. (2016). Photogrammetric discharge monitoring of small tropical mountain rivers: A case study at Rivière des Pluies, Réunion Island. Water Resources Research, 52(6), 4550–4570. https://doi.org/10.1002/2015WR018292 Tauro, F., Piscopia, R., & Grimaldi, S. (2017). Streamflow observations from cameras: Large‐scale particle image velocimetry or particle tracking velocimetry? Water Resources Research, 53(12), 10374–10394. https://doi.org/10.1002/2017WR020848 Tauro, F., Porfiri, M., & Grimaldi, S. (2014). Orienting the camera and firing lasers to enhance large scale particle image velocimetry for streamflow monitoring. Water Resources Research, 50(9), 7470–7483. https://doi.org/10.1002/2014WR015952 Tauro, F., Tosi, F., Mattoccia, S., Toth, E., Piscopia, R., & Grimaldi, S. (2018). Optical tracking velocimetry (OTV): Leveraging optical flow and trajectory-based filtering for surface streamflow observations. Remote Sensing, 10(12), 2010. https://doi.org/10.3390/rs10122010 Thielicke, W. (2022, November 9). PIVlab 2.60. GitHub. https://github.com/Shrediquette/PIVlab/releases/tag/2.60 168 Thielicke, W., & Stamhuis, E. J. (2014). PIVlab – towards user-friendly, affordable and accurate digital particle image velocimetry in MATLAB. Journal of Open Research Software, 2. https://doi.org/10.5334/jors.bl Tinkler, K. J. (1997). Indirect velocity measurement from standing waves in rockbed rivers. Journal of Hydraulic Engineering, 123(10), 918–921. https://doi.org/10.1061/(ASCE)0733-9429(1997)123:10(918) Tsubaki, R., Fujita, I., & Tsutsumi, S. (2011). Measurement of the flood discharge of a small-sized river using an existing digital video recording system. Journal of Hydro-Environment Research, 5(4), 313–321. https://doi.org/10.1016/j.jher.2010.12.004 Weijs, S.V., Chen, Z., Brauchli, T., Huwald, H. (2014, April 27 – May 2). Measuring surface flow velocity with smartphones: potential for citizen observatories. EGU General Assembly 2014, Vienna, Austria. https://meetingorganizer.copernicus.org/EGU2014/EGU2014-15882.pdf Weijs, S.V., Kovacek, D. (2022, May 23-27). The November 2021 floods in British Columbia, Canada: observations, mechanisms and probabilities. EGU General Assembly 2022, Vienna, Austria and online. https://doi.org/10.5194/egusphere-egu22-10998 Winsemius, H., Annor, F., Hagenaars, R., Luxemburg, W., Van den Munckhof, G. V., Heeskens, P., & Van de Giesen, N. (2022, May 23-27). Capturing the Europe July 2021 flood event flows with an IP camera and OpenRiverCam. EGU General Assembly 2022, Vienna, Austria and online. https://doi.org/10.5194/egusphere-egu22-11627 169 WMO (World Meteorological Organization). (2021, August 31). Weather-related disasters increase over past 50 years, causing more damage but fewer deaths. https://public.wmo.int/en/media/press-release/weather-related-disasters-increase-over-past-50-years-causing-more-damage-fewer WSC (Water Survey of Canada), & ECCC (Environment and Climate Change Canada). (2022). Real-Time Hydrometric Data Graph for COLDWATER RIVER AT MERRITT (08LG010) [BC]. Extracted from the Environment and Climate Change Canada Real-time Hydrometric Data web site (https://wateroffice.ec.gc.ca/mainmenu/real_time_data_index_e.html) on March 6, 2023. Wu, H., Zhao, R., Gan, X., & Ma, X. (2019). Measuring Surface Velocity of Water Flow by Dense Optical Flow Method. Water, 11(11), 2320. https://doi.org/10.3390/w11112320 Yagi, J., Tani, K., Fujita, I., Nakayama, K., (2020). Application of Optical Flow Techniques for River Surface Flow Measurements. In Y. Shimizu (Ed.), Proceedings of the 22nd IAHR APD Congress (Sapporo, 2020) (pp.x-x). Yeh, M.-T., Chung, Y.-N., Huang, Y.-X., Lai, C.-W., & Juang, D.-J. (2019). Applying adaptive LS-PIV with dynamically adjusting detection region approach on the surface velocity measurement of river flow. Computers & Electrical Engineering, 74, 466–482. https://doi.org/10.1016/j.compeleceng.2017.12.013 Zhang, G., & Chanson, H. (2018). Application of local optical flow methods to high-velocity free-surface flows: Validation and application to stepped chutes. Experimental Thermal and Fluid Science, 90, 186–199. https://doi.org/10.1016/j.expthermflusci.2017.09.010 170 Zuiderveld, K. (1994). Contrast Limited Adaptive Histogram Equalization. In P. S. Heckbert (Ed.), Graphics Gems (pp. 474–485). Academic Press. https://doi.org/10.1016/B978-0-12-336156-1.50061-6 171 Appendices Appendix A 2D Orthorectification errors for GCPs in Site 1 video on 2021-11-15 This section contains detailed results of 2D orthorectification mentioned in Section 5.6.1. A.1 GCPs under orthorectification Version 1 Figure A.1 GCPs identified for orthorectification version 1, over an original frame of the video on 2021-11-15. This Figure is identical to Figure 3.13. Table A.1 List of GCPs' image and world coordinates used in version 1, as well the errors in real-world distances. GCP# Image coordinates World coordinates Errors i(pixel) j(pixel) x(m) y(m) Δx(m) Δy(m) distance(m) 1 3469.25 1498.25 658803.150 5551684.337 0.831 -0.008 0.832 2 2712.00 1040.00 658860.830 5551662.430 1.113 -3.205 3.393 3 2540.25 908.25 658922.040 5551628.540 -6.228 4.761 7.840 4 1002.75 1821.75 658804.775 5551722.926 -1.020 -0.510 1.140 5 1129.25 1704.75 658807.553 5551719.837 0.136 0.591 0.607 172 GCP# Image coordinates World coordinates Errors i(pixel) j(pixel) x(m) y(m) Δx(m) Δy(m) distance(m) 6 1205.25 1617.75 658811.125 5551718.068 0.207 0.809 0.835 7 1301.75 1278.25 658837.106 5551713.086 -0.098 -0.309 0.325 8 1453.25 1195.25 658846.848 5551707.170 0.018 -0.054 0.056 9 1782.75 1008.25 658882.750 5551686.230 3.188 -1.019 3.347 10 1944.25 938.75 658909.020 5551669.210 3.435 -1.016 3.582 11 280.25 1726.25 658811.171 5551730.656 -0.134 0.078 0.156 12 216.25 1393.25 658834.845 5551734.119 -1.266 -0.276 1.296 13 526.75 1288.25 658842.507 5551728.785 0.216 -0.075 0.229 14 721.75 1276.25 658842.402 5551724.763 0.101 -0.071 0.124 A.2 GCPs under orthorectification Version 2 Figure A.2 GCPs identified for orthorectification version 2, on an original frame of the video on 2021-11-15. 173 Table A.2 List of GCPs' image and world coordinates used in version 2, as well the errors in real-world distances. GCP# Image coordinates World coordinates Errors i(pixel) j(pixel) x(m) y(m) Δx(m) Δy(m) distance(m) 1 3446.75 1485.75 658803.150 5551684.337 0.989 -0.421 1.075 2 2726.00 1027.00 658860.830 5551662.430 0.445 -1.144 1.227 4 1001.50 1820.83 658804.775 5551722.926 -1.864 -0.545 1.942 5 1128.17 1703.50 658807.553 5551719.837 -0.403 0.451 0.605 6 1201.50 1609.50 658811.125 5551718.068 0.288 0.587 0.654 7 1298.83 1272.17 658837.106 5551713.086 0.888 -0.604 1.074 8 1451.50 1192.83 658846.848 5551707.170 0.313 -0.161 0.352 10 1971.25 931.75 658909.020 5551669.210 -3.155 1.846 3.656 11 300.25 1728.75 658811.171 5551730.656 -0.737 0.201 0.764 12 221.25 1389.75 658834.845 5551734.119 -0.093 -0.036 0.100 13 520.83 1278.83 658842.507 5551728.785 2.385 0.065 2.385 14 726.25 1272.75 658842.402 5551724.763 1.216 -0.295 1.251 174 Appendix B Examples of 3D orthorectification errors This section contains detailed results of 3D orthorectification mentioned in Section 5.6.2. B.1 Errors in 3D orthorectification on drone video over Sites 3&4 on 2021-11-15 Figure A.3 GCPs identified and numbered on the first frame of drone video over Sites 3&4 on 2021-11-15. Table A.3 List of GCPs' image and world coordinates for drone video over Sites 3&4 on 2021-11-15, with camera reprojection errors in pixels. GCP# i(pixel) j(pixel) x(m) y(m) z(m) Reprojection error (px) 1 487.250 768.625 657724.499 5552587.893 596.362 0.3082 2 233.125 607.625 657746.706 5552594.556 597.000 1.9432 3 270.125 472.875 657768.038 5552588.005 596.163 1.6230 4 734.063 507.563 657743.397 5552565.528 596.515 4.1712 5 1136.375 406.125 657737.564 5552537.174 599.000 1.8713 6 1280.125 300.125 657754.961 5552505.948 597.000 2.0514 7 1616.125 217.375 657747.643 5552468.158 599.941 2.3260 8 1895.375 148.625 657740.988 5552429.691 603.033 2.0474 175 GCP# i(pixel) j(pixel) x(m) y(m) z(m) Reprojection error (px) 9 1792.125 318.125 657719.142 5552487.615 601.548 6.0058 10 1066.125 644.375 657721.436 5552555.388 593.699 6.5239 11 1109.125 707.375 657714.841 5552557.706 594.049 4.2500 12 1304.125 719.625 657708.087 5552550.862 594.058 6.6650 13 1042.375 524.625 657735.113 5552547.310 594.568 8.8788 B.2 Errors in 3D orthorectification on iPhone video over Site 1 on 2022-06-16 Figure A.4 GCPs identified and numbered on the first frame of iPhone video over Site 1 on 2022-06-16. 176 Table A.4 List of GCPs' image and world coordinates for iPhone video over Site 1 on 2022-06-16, with camera reprojection errors in pixels. GCP# i(pixel) j(pixel) x(m) y(m) z(m) Reprojection error (px) 1 1651.875 959.375 658765.138 5551703.893 602.855 5.6011 2 1407.75 917.875 658769.809 5551704.376 601.683 8.1474 3 1260.625 663.875 658780.621 5551699.282 602.000 5.7674 4 1470.875 511.375 658781.060 5551688.446 605.091 1.3721 5 1181.625 458.875 658803.150 5551684.337 602.183 5.6166 6 434.625 667.375 658795.343 5551725.920 600.580 4.7330 7 540.625 557.375 658804.775 5551722.926 601.236 7.1467 8 589.625 519.625 658808.241 5551720.746 601.716 5.7849 9 701.625 374.875 658846.848 5551707.170 602.165 4.7389 10 814.625 328.375 658879.973 5551684.827 602.341 5.3362 11 859.375 303.875 658913.541 5551667.456 602.450 3.7543 177 Appendix C Supplementary data The processing code files, original video clips, surveying data, and some of the image velocimetry results featured in this thesis can be found at: https://github.com/yangjs97/Image_Velocimetry_Merritt Screenshots of the repository follow. 178 179 180 181 182 183 184 Appendix D Example of velocity orthorectification code The screenshots in this section are from the code and corresponding displays for rectification of PIVlab velocity results in 3D orthorectification mode (“PIV_3Dorthorect_ver2_1.m” in Appendix C ). The code was written in MATLAB R2021b, with Image Processing Toolbox and Camera Calibration App. 185 186 187 188 189 190 191
Thesis/Dissertation
eng
Civil Engineering
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Weijs, Steven
Streamflow monitoring in a time of change : using image velocimetry methods on citizen videos of the November 2021 flooding in Merritt, British Columbia
Text