Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Using motes for high resolution hydrological measurement Trubilowicz, Joel William 2008

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata

Download

Media
24-ubc_2008_spring_trubilowicz_joel.pdf [ 4.16MB ]
Metadata
JSON: 24-1.0066329.json
JSON-LD: 24-1.0066329-ld.json
RDF/XML (Pretty): 24-1.0066329-rdf.xml
RDF/JSON: 24-1.0066329-rdf.json
Turtle: 24-1.0066329-turtle.txt
N-Triples: 24-1.0066329-rdf-ntriples.txt
Original Record: 24-1.0066329-source.json
Full Text
24-1.0066329-fulltext.txt
Citation
24-1.0066329.ris

Full Text

Using motes for high resolution hydrological measurement by Joel William Trubilowicz B.Sc., Michigan Technological University, 2004  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  MASTER OF APPLIED SCIENCE in THE FACULTY OF GRADUATE STUDIES (Forestry)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) March 2008 © Joel Trubilowicz, 2008  Abstract  Low cost, low power wireless sensors (motes) promise to revolutionize environmental data collection, but are they currently refined enough for widespread use by hydrologists? Their viability as a replacement for traditional data collection techniques was investigated in a 7 ha forested watershed in south-western British Columbia. The watershed included 41 instrument clusters measuring air and soil temperature, humidity, throughfall, soil moisture content, overland flow and groundwater head. The foundation of each cluster was a data box containing a MDA300 data acquisition board and a MICA2 processor board from Crossbow Technologies, Inc.™ that allowed for short range wireless data collection. The 41 motes each recorded data every 15 minutes from July, 2006, to April, 2007. In addition to reporting on the reliability of the motes and sensors during the 10 months deployment, the high spatial and temporal resolution data collected by this study gave the opportunity for many analyses of catchment processes. As soil moisture and throughfall are two influential processes in the exchange of water between the earth and the atmosphere, these were the focus of the data analysis. The first analysis was a resampling experiment on seven different events selected from the full data set. Comparing 100 different subsamples each of 5, 10 and 20 points for throughfall and soil moisture showed if increasing the sample size eventually produced diminishing returns in the ability to reproduce the true catchment mean. With significant differences in prediction ability for both soil moisture and throughfall at times of differing hydrologic activity, this analysis provides further support for the theories of changing moisture states of soil moisture and threshold values for throughfall. The second analysis described how the organization of soil moisture and throughfall changed during a range of  ii  weather conditions and timescales. Spatial representation of normalized values and Pearson correlation coefficients showed that there were distinct differences between wet and dry periods for soil moisture and between long and short analysis periods for throughfall.  iii  Table of Contents  Abstract ..................................................................................................................................... ii Table of Contents..................................................................................................................... iv List of Tables .......................................................................................................................... vii List of Figures ........................................................................................................................ viii List of Abbreviations ................................................................................................................ x Acknowledgements.................................................................................................................. xi Co-Authorship Statement........................................................................................................ xii Chapter 1  Introduction........................................................................................................1  1.1  Introduction........................................................................................................1  1.2  Motivation for high resolution monitoring ........................................................3  1.3  Objectives ..........................................................................................................7  1.4  References..........................................................................................................8  Chapter 2 2.1  2.2  The viability of motes for hydrological measurement.....................................10 Introduction......................................................................................................10 2.1.1  Why motes? .........................................................................................10  2.1.2  Technological potential of motes.........................................................13  2.1.3  Current state of development ...............................................................15  2.1.4  Objectives ............................................................................................17  Methods............................................................................................................17 2.2.1  Study Site .............................................................................................17  2.2.2  Hardware Description ..........................................................................20  2.2.3  Software Description ...........................................................................23 2.2.3.1 The Framework........................................................................23 2.2.3.2 Data Conversion Software .......................................................23  2.3  Timeline ...........................................................................................................23  2.4  Results..............................................................................................................24  2.5  2.4.1  Power Usage ........................................................................................24  2.4.2  Reliability.............................................................................................25  Discussion ........................................................................................................28  iv  2.6  Conclusion .......................................................................................................31  2.7  References........................................................................................................32  Chapter 3  Effects of sample size on catchment averaged soil moisture and throughfall  measurement ..........................................................................................................................35 3.1  Introduction......................................................................................................35  3.2  Methods............................................................................................................37  3.3  3.2.1  Study Site .............................................................................................37  3.2.2  Data Analysis .......................................................................................40  Results..............................................................................................................46 3.3.1  3.4  Statistical Results .................................................................................49  Discussion ........................................................................................................55 3.4.1  Soil Moisture........................................................................................55  3.4.2  Throughfall ..........................................................................................58  3.4.3  Sampling Design..................................................................................60  3.5  Conclusions......................................................................................................60  3.6  References........................................................................................................61  Chapter 4  The dynamics of spatial variability in soil moisture and throughfall ..............63  4.1  Introduction......................................................................................................63  4.2  Methods............................................................................................................67  4.3  4.4  4.5  4.2.1  Study Site .............................................................................................67  4.2.2  Data Analysis .......................................................................................70  Results..............................................................................................................75 4.3.1  Soil Moisture........................................................................................75  4.3.2  Throughfall ..........................................................................................80  4.3.3  Variogram Analysis .............................................................................83  Discussion ........................................................................................................85 4.4.1  Frequency distribution .........................................................................85  4.4.2  Persistent values...................................................................................86  4.4.3  Pearson correlation...............................................................................89  4.4.4  Variogram Analysis .............................................................................90  Conclusions......................................................................................................90  v  4.6 Chapter 5  Reference .........................................................................................................92 Conclusions......................................................................................................95  5.1  Discussion ........................................................................................................95  5.2  Weaknesses ......................................................................................................96  5.3  Strengths and Application................................................................................97  5.4  Future Work .....................................................................................................99  5.5  References......................................................................................................101  Appendix A............................................................................................................................102 Appendix B ............................................................................................................................107 Appendix C ............................................................................................................................111 Appendix D............................................................................................................................115 Appendix E ............................................................................................................................118 Appendix F.............................................................................................................................121 Appendix G............................................................................................................................125 Appendix H............................................................................................................................132 Appendix I .............................................................................................................................138 Appendix J .............................................................................................................................140 Appendix K............................................................................................................................144 Appendix L ............................................................................................................................146 Appendix M ...........................................................................................................................149 Appendix N............................................................................................................................151  vi  List of Tables Table 2.1  Instrument cluster functions.............................................................................22  Table 2.2  Run time results (winter). ................................................................................25  Table 2.3  Percentage of usable data during the main sampling period............................27  Table 3.1  Summary of analyzed events. Start and end dates are inclusive.....................41  Table 3.2  Testing Scheme................................................................................................44  Table 4.1  Event Summary................................................................................................71  Table J.1  Soil Moisture F and T test results. .................................................................140  Table J.2  Soil Moisture Wilcoxon Rank-Sum Test Results for Pearson r. ...................141  Table J.3  Throughfall Test Results (RMSE). ................................................................142  Table J.4  Throughfall Wilcoxon Rank-Sum Test Results for Pearson r. ......................143  vii  List of Figures  Figure 2.1  Map of the test catchment in Malcom Knapp Research Forest, British Columbia, Canada............................................................................................19  Figure 2.2  Measurement station ready for deployment.....................................................21  Figure 3.1  Measurement locations within the study catchment. .......................................40  Figure 3.2  Ensemble Plots for Event A.............................................................................46  Figure 3.3  Ensemble Plots for Event C. ............................................................................47  Figure 3.4  Ensemble Plots for Event D.............................................................................48  Figure 3.5  Ensemble Plots for Event G.............................................................................49  Figure 3.6  Distribution of RMSE of each realization to the true sample mean for Soil Moisture (all events). .......................................................................................50  Figure 3.7  Distributions of Pearson correlation coefficients between each realization and the true sample mean for Soil Moisture (all events). .......................................51  Figure 3.8  Distribution of RMSE of each realization to the true sample mean for throughfall (all events). ....................................................................................53  Figure 3.9  Distributions of Pearson correlation coefficients between each realization and the true sample mean for Throughfall (all events)...........................................54  Figure 4.1  Data measurement locations on the study area DEM. .....................................70  Figure 4.2  Distribution of hourly soil moisture measurement values through event S1 (left) and S2 (right). .........................................................................................75  Figure 4.3  Distribution of daily soil moisture measurement values through events L1 (left) and L2 (right). .........................................................................................76  Figure 4.4  Mean normalized soil moisture values for each event.....................................77  Figure 4.5  Mean normalized soil moisture values shown at their measurement location on the catchment DEM. ........................................................................................78  Figure 4.6  Pearson r values for soil moisture shown at their measurement location on the catchment DEM. ..............................................................................................80  Figure 4.7  Mean normalized throughfall values for each event........................................81  Figure 4.8  Mean normalized throughfall values shown at their measurement location on the catchment DEM. ........................................................................................82  viii  Figure 4.9  Pearson r values for throughfall shown at their measurement location on the catchment DEM. ..............................................................................................83  Figure 4.10  Variograms shown for soil moisture in all events. ..........................................84  Figure 4.11  Variograms shown for throughfall in all events. .............................................85  Figure I.1  Ensemble plots for event B showing 100 different realizations at each sample size. ................................................................................................................138  Figure I.2  Ensemble plots for event E showing 100 different realizations at each sample size. ................................................................................................................138  Figure I.3  Ensemble plots for event E showing 100 different realizations at each sample size. ................................................................................................................139  ix  List of Abbreviations DEM -  Digital Elevation Model  GCM -  Global Climate Model  IDL -  Interactive Data Language  MKRF -  Malcolm Knapp Research Forest  RMSE -  Root Mean Square Error  x  Acknowledgements  Dr. Markus Weiler provided assistance, guidance and ideas in all aspects of the project, from hardware development and field installation to data analysis and writing.  Dr. Dan Moore and Dr. Violeta Martin provided guidance and ideas for the general goals of the project, particularly in the data analysis phase of the project.  Cheryl So began development of the software for operation of instrumentation on the motes.  Kan Cai refined the software used for mote operation and assisted in solving problems with mote operations.  Anthony Bier assisted in initial field installation of sensors and weirs, general field maintenance of the experiment and decisions on catchment delineation and measurement location choice, and surveying of measurement locations.  Alpha Reghan Wong assisted in proofreading and editing of the thesis.  Charlotte Argue and Stephanie Ewen assisted in the installation and maintenance of field instrumentation.  xi  Co-Authorship Statement  Dr. Markus Weiler, Department of Forest Resource Management, University of British Columbia Markus Weiler is included as a co-author for all 3 research papers included in this thesis. As my graduate advisor, he has assisted with many ideas throughout the entire experiment. In the paper included in chapter 2, he had the initial motivation for using motes in the field of hydrology, as this had not yet been done in an experiment with actual scientific goals beyond just research on motes themselves. He provided many ideas for bringing the monitoring network into operation and troubleshooting. In the papers in chapters 3 and 4 he provided ideas for the data analysis and general format and goals of the papers. He did not perform any of the data analysis or writing for any chapter of this thesis.  Kan Cai, Department of Computer Science, University of British Columbia Kan Cai is included as a co-author for the paper in chapter 2 of this thesis. He spent a large amount of time refining software for operation of the monitoring experiment and was integral to the overall success of the experiment. He has written a summary of the software framework that is included in chapter 2 of this thesis. I have indicated in the text where his portion of the writing begins and ends. This portion is included because it will be included when chapter 2 is submitted for publication to Water Resources Research and should not be considered as a part of my thesis.  xii  Chapter 1  1.1  Introduction  Introduction  Data collection in hydrology takes two forms: remote sampling, where data are collected using instrumentation that is not in physical contact with the area being studied, and direct sampling, where data are collected through direct measurement at the point of interest. The most basic form of direct sampling in hydrology is manual data collection. Each individual measurement is taken and recorded by someone in the field. If only a few measurements are needed, this is the quickest and easiest form of data collection, but if a large number of samples are needed, it becomes more difficult. The next step beyond manual sampling is a programmable data logger that can take and record samples over long periods of time at an individual point. Data loggers make data collection over time much easier for scientific researchers. Current widely used data loggers, such as the Campbell Scientific CR1000, are robust, reliable, and easy to configure for operation of different instrumentation, but they are also expensive ($2000+) and require a 12 V power supply for operation (Campbell 2006). Since they are stationary, they are only useful for sampling in one location at a high temporal resolution, and since deploying a large amount of these data loggers throughout a catchment would be very costly, manual sampling at many different points is still the main option for high spatial resolution projects such as the Tarrawarra project (Western et al. 1999).  This thesis focuses on technology that has the potential to be the next step in the evolution of direct sampling. In May 2005, I began a hydrologic data collection project using low-cost, low-power environmental sensors, known as motes (Hill and Culler 2002) that operate the  1  open source operating system TinyOS (Levis et al. 2005). Their low price and ability to network with one another has the potential to eliminate the compromise hydrologists have to make between high spatial resolution and high temporal resolution. My goal was to determine if these motes can currently deliver on their promises of a very flexible environmental sensor network that collects spatially and temporally rich data, as they had not yet been used in a full scale hydrologic catchment study. I wanted to determine if they are a viable alternative to more common data collection techniques in hydrology. I was particularly focused on data reliability and overall stability of the motes because the most important feature of any kind of data collector is its ability to successfully collect and record data.  Throughout the data collection portion of the study, from December 2005 to April 2007, the mote network collected over 6 million data points. As there are many possibilities for the use of all of these collected points, but not nearly enough time to try them all, it was necessary to choose specific purposes. As soil moisture and throughfall were two of the most reliable measurements in this study and both are essential to the understanding of the movement of water between the atmosphere and the earth, I chose to focus the data analysis portion of the project on these two parameters. In particular, I used a resampling analysis to determine how important sample size was on the ability to measure the catchment mean value and a range of spatial analysis methods to illustrate how patterns of soil moisture and throughfall change over time.  2  1.2  Motivation for high resolution monitoring  In certain hydrological studies it may be beneficial to have a dense network of sensors to demonstrate spatial variability. There are potential benefits when dealing with a wide range of spatial scales. A few previous catchment experiments are described below, and some possible areas where a higher resolution network may have helped in the experiment are discussed.  In order to determine the most efficient way to perform a catchment study, many researchers have attempted to determine a minimum area of an experiment that can be thought of as one homogenous unit. This homogenous unit was termed the representative elementary area (REA) by Wood et al. in 1988. It was considered the most fundamental block of the catchment model, meaning that when study scales were greater than this area, the REA could be thought of as one unit (Wood et al. 1988). Seyfried and Wilcox (1995) have focused on areas smaller than the REA and used the term deterministic length scale to describe scales smaller than the REA. This scale is where “the actual patterns of rainfall, topography, soil, etc. must be explicitly considered” (Seyfried and Wilcox 1995). In their description, the REA represents the upper limit of the deterministic length scale, above which, spatial variation may be considered stochastic. It has been stated that a full analysis of the variation of spatial patterns should include both deterministic and stochastic analyses (Keim et al. 2005), but this has typically been a difficult task to accomplish. Seyfried and Wilcox (1995) included measurements of groundwater recharge via 4 piezometers in the summit catchment (a small segment of the larger study catchment). These wells gave some indication of the large variance in recharge in the summit catchment of the Reynolds Creek watershed of  3  Southwest Idaho, and they indicated that it would be difficult to estimate recharge from a single parameter. Applying a denser network of sensors (in this case piezometers) might help to determine the scale at which a homogenous recharge parameter could be used or allow for the discretization of the summit catchment into different recharge zones.  Hydrologic macro modeling (Motovilov et al. 1999) is a method of modeling that involves applying the same hydrological model repeatedly to different areas in a region using the same set of parameters. This type of modeling requires the definition of a fundamental unit of the model for validation. Motovilov et al. (1999) used data collected from 5 smaller basins within their study area to determine parameters and verify the macro scale model in the NOPEX experiment in southern Sweden. The smaller basins were used as the representative elementary area and the grid size of the model was 2 km by 2 km. In the NOPEX catchment study, figures are shown to represent the determination of the fundamental unit for the macro model. It can be seen that as the area measured becomes smaller, the variation of soil moisture and groundwater levels becomes larger. These variations are the highest at the smallest scales (10-100 m distances) and there are roughly 20 samples in this range of area. The catchment study proposed here would fall directly in that range of distance between points. It may be possible to provide more information on this variation in soil moisture and its causes since it has been stated that below the REA (in this case 2 km by 2 km) the nature of variability is most likely deterministic (Seyfried and Wilcox 1995).  In a much smaller watershed in Pennsylvania, the spatial distribution of a water balance was evaluated in a distributed approach by simulation of spatial and temporal variation using  4  remotely sensed data (Yu et al. 2000). Precipitation and temperature data for this 7.3 km2 catchment was collected at one point and applied to the entire model. Soil moisture content was evaluated at different depths by gypsum blocks over a period of two months. Yu et al. (2000) recognized that soil, climate, crop type and topography all affect the distribution of hydrological variables, but in this case the actual effect was only simulated. With the use of higher resolution sampling techniques, it may have been possible to comment specifically on the variability of these environmental factors.  In another catchment experiment different soil moisture spatial regimes have been encountered during different seasons, and an abrupt (within approximately one month) shift between regimes/patterns can be seen (Grayson et al. 1997; Western et al. 1999; Western et al. 1999; Western et al. 2001). The Tarawarra soil moisture study used high density soil moisture readings (>1500) in a relatively small (10.5 ha) area. These moisture readings were taken multiple times throughout the year in order to investigate spatial patterns of soil moisture. Grayson et al. (1997) found that soil moisture in the Tarawarra catchment followed two distinct patterns depending on the overall moisture in the system. In the dry season, soil moisture was largely dominated by “local” controls, such as the soil properties, and most moisture moved vertically through the soil. Because of this, there was no discernable spatial pattern in the catchment. However, during the wetter seasons, soil moisture appeared to be dominated by more “non-local” controls, such as topography, and more soil moisture was supplied by lateral flow through the soil. Because of this, a much more organized spatial pattern of soil moisture that appeared to be more associated with terrain features was evident (Grayson et al. 1997). The actual association with terrain  5  features and other physical characteristics of the catchment was investigated through the use of different hydrological models, and terrain indices (Western et al. 1999). Further study determined that different models and indices performed better depending on which preferred moisture state the catchment was in (Western et al. 1999). Indicator geostatistics and connectivity statistics were also employed to determine the level of connectivity of the different soil moisture values (Western et al. 2001).  Similar spatial patterns to those seen in the Tarawarra catchment have been simulated numerically (Ridolfi et al. 2003). The simulations performed by Ridolfi et al (2003) showed a higher degree of spatial organization and dependence on topography during a more humid regime than during a dry regime. This agrees with field results of studies such as the Tarrawarra project.  The wireless sensor network is not as detailed spatially as the Tarawarra project (Western et al. 1999), or other projects focusing on soil moisture (Schume et al. 2003) but it is much more detailed temporally, and allows for a better understanding of the transition between moisture states.  6  1.3  Objectives  This thesis has 3 objectives, and each of these objectives is covered in its own stand alone scientific paper intended for journal submission (Chapters 2, 3 and 4). The objectives are:  1.  To evaluate the practical application of low cost, low power wireless transmitters  (motes) in their current form to a hydrologic catchment experiment.  2.  To evaluate the effects of changing the sample size on the ability to estimate the  catchment mean value of soil moisture and throughfall and how these effects change with changing weather conditions.  3.  To use different methods of describing the spatial variability in order to show how the  organization of throughfall and soil moisture changed over short and long time scales with changing weather conditions  7  1.4  References  Campbell (2006). "CR1000 Specifications." Retrieved February 2, 2008, from http://www.campbellsci.ca/Catalogue/CR1000_Specs.pdf. Grayson, R. B., A. W. Western, G. Bloschl and F. H. S. Chiew (1997). "Preferred states in spatial soil moisture patterns: local and nonlocal controls." Water Resources Research 33(12): 2897-2908. Hill, J. L. and D. E. Culler (2002). "Mica: A Wireless Platform for Deeply Embedded Networks." IEEE Micro 22(6): 12-24. Keim, R. F., A. E. Skaugset and M. Weiler (2005). "Temporal persistence of spatial patterns in throughfall." Journal of Hydrology 314: 263-274. Levis, P., S. Madden, J. Polastre, R. Szewczyk, K. Whitehouse, A. Woo, D. Gay, J. Hill, M. Welsh, E. Brewer and D. Culler (2005). TinyOS: An Operating System for Sensor Networks. Ambient Intelligence. W. Weber, J. M. Rabaey and E. Aarts. New York, NY, Springer: 115148. Motovilov, Y. G., L. Gottschalk, K. Engeland and A. Rodhe (1999). "Validation of a distributed hydrological model against spatial observations." Agricultural and Forest Meteorology 98-99: 257-277. Ridolfi, L., P. D'Orico, A. Porporato and I. Rodgriquez-Iturbe (2003). "Stochastic soil moisture dynamics along a hillslope." Journal of Hydrology 272: 264-275. Schume, H., G. Jost and K. Katzensteiner (2003). "Spatio-temporal analysis of the soil water content in a mixed Norway spruce (Picea abies (L.) Karst.)-European beech (Fagus sylvatica L.) stand." Geoderma 112: 273-287. Seyfried, M. S. and B. P. Wilcox (1995). "Scale and the Nature of Spatial Variability: Field examples having implications for hydrologic modeling." Water Resources Research 31(1): 173-184. Western, A. W., R. B. Grayson and G. Bloschl (2001). "Towards capturing hydrologically significant connectivity in spatial patterns." Water Resources Research 37(1): 83-97. Western, A. W., R. B. Grayson, G. Bloschl, G. R. Willgoose and T. A. McMahon (1999). "Observed spatial organization of soil moisture and its relation to terrain indices." Water Resources Research 35(3): 797-810. Western, A. W., R. B. Grayson and T. R. Green (1999). "The Tarrawarra project: high resolution spatial measurement, modelling and analysis of soil moisture and hydrologic responses." Hydrological Processes 13: 633-652.  8  Wood, E. F., M. Sivapalan, K. Beven and L. Band (1988). "Effects of spatial variability and scale with implications to hydrologic modelling." Journal of Hydrology 102: 29-47. Yu, Z., W. J. Gburek and F. W. Schwartz (2000). "Evaluating the spatial distribution of water balance in a small scale watershed, Pennsylvania." Hydrological Processes 14: 941-956.  9  Chapter 2:  The viability of motes for hydrological measurement1  2.1  Introduction  2.1.1  Why Motes?  Catchment sampling studies in hydrology always require compromise. It is necessary to strike a balance between the amount of time and resources available and the necessary amount of instrumentation needed. Previously, striking this balance meant having less instrumentation than might be desired, but the emerging technology of low-cost, low-power wireless sensors, known as motes, offers the possibility of much denser instrumentation and hence much more detailed data collection. Motes also allow for the possibility of a surplus of measurement stations to be deployed in an area of interest, resulting in a network where sensors are only activated when needed, and can remain dormant and unobtrusive when not in use. This will allow for an extremely flexible network that can be adapted for different projects and scales of measurement with little to no modification of actual hardware. As this idea is still limited by the size and cost of sensors that will be attached to motes, it is a hope for the future of hydrologic data collection (Glaser 2004). While researchers would probably not refuse higher resolution measurement, it may be more important for some experiments than others. In the following paragraphs we will explore a few scenarios where data collection at a high spatial and temporal resolution would be especially desirable.  1  A version of this Chapter will be submitted for publication.  J.Trubilowicz, M. Weiler and K. Cai. The viability of motes for hydrological measurement. Water Resources Research  10  Seyfried and Wilcox (1995) stated that variability can be either deterministic (based on theoretically derived or empirical relationships) or stochastic (random). They also stated that “accurate description of hydrologic responses requires a deterministic description of spatial variability over a particular range of scales” and refer to this as the deterministic length scale. This deterministic length scale depends on the scale of interest, so with a flexible data collection network, such as one promised using motes, different sensors could be activated or deactivated as the scale of interest changes from small scale hill slopes or soil plots to large scale river basins. Whenever a few measurement locations indicate a large amount of spatial variation (Seyfried and Wilcox 1995) it would be useful to researchers to know more about the nature of this spatial variability and specifically determine the scale that homogenous parameters could be used. With high resolution sampling it would also be possible to use actual field data on spatial variability for different parameters such as precipitation and temperature instead of turning to simulation(Yu et al. 2000; Wilson et al. 2005) for use in modeling.  Motes also have the potential to let us know more about how spatial patterns change over time. Recent experiments in the Tarrawarra project of Australia measuring soil moisture at high spatial resolution have shown evidence of different soil moisture states depending on the overall moisture level in the catchment (Grayson et al. 1997; Western et al. 1999; Western et al. 1999; Western et al. 2001). This study had data collection at a very high spatial resolution, but low temporal resolution (~2 week intervals). Employing motes in such an experiment would be the next step in the evolution of this type of study. Motes give much  11  richer data temporally, allowing for much better knowledge of the way that soil moisture switches between moisture states.  Distributed physical hydrological models take into consideration the spatial variability of hydrologic parameters within a catchment. This is in opposition to lumped models that consider a catchment as one unit (Refsgaard 1997). While it would be desirable to use a model based on physical characteristics, physically based hydrological models present many difficulties that stem from our inability to accurately describe the spatial and temporal variability of parameters in a catchment (Seyfried and Wilcox 1995). A cheaper, simpler and easier method for data collection at high spatial and temporal variability, such as motes, would be helpful in collecting data for use in a distributed physical hydrologic model.  Another model type where selective high-resolution data would be desirable is known as hydrologic macro modeling. Macro modeling is a method of modeling that involves applying the same hydrological model repeatedly to different areas in a region using the same set of parameters. They are used for large scale projects such as assessing the impact of climate on water resources. By definition, macro models are “models that are compatible with the scale of a GCM (Global Climate Model) grid square (e.g. 105 km2) and can accept atmospheric model data as input.” (Motovilov et al. 1999) This type of modeling requires the definition of a fundamental unit of the model for validation. Motovilov et al. (1999) used data collected from 5 smaller basins within the study area to determine parameters and verify the macro scale model in the NOPEX experiment in southern Sweden. The smaller basins were used as fundamental units and their parameters were subsequently applied to other  12  ungauged sections within the modeled region. Easily available high resolution data in a macro model situation such as the NOPEX experiment (Motovilov et al. 1999) would provide a comparatively easy way to get information from more hydrologic catchments for use as fundamental units. More fundamental units would inspire more confidence in the results of a macro model like the NOPEX experiment and its use in the forecasting impacts from floods and other natural disasters.  2.1.2  Technological potential of motes  Low-cost, low-power wireless sensor technology that we evaluate in this paper has the potential to greatly decrease the time and effort required to collect spatially and temporally rich data. We believe that the idea that a large number of inexpensive sensors are better and more useful than a few expensive sensors (Delin 2002; Mainwaring et al. 2002) is applicable to segments of hydrology interested in small scale spatial variability. We think that wireless sensors can change the way researchers design hydrologic experiments. Most current catchment experiments require that researchers make a choice between high spatial resolution, where many measurements are taken manually at whatever time intervals are possible for the researcher (Western et al. 1999) or high temporal resolution, where a few expensive logging stations are deployed at strategic locations and record information over time. With an inexpensive and reliable method for sensing and data collection, it would not be necessary to choose between spatial and temporal resolution. Current prices for motes are around $200 U.S., and prices are anticipated to drop to as low as $5 in the near future (Glaser 2004). This means that data collection with motes as the centerpiece of a full complement of hydrologic sensors (rainfall, groundwater level, soil moisture, overland flow, temperature,  13  humidity, etc.) can be assembled at a fraction of the cost of existing data loggers alone, which can cost in upwards of $2000. An overwhelming amount of sensors could cover an area and self organize into a network where information is processed in place (Hill and Culler 2002). It would then be possible to extract information from this exhaustive data set to answer a specific research question. As data can be transferred between motes and filtered down to a high powered wireless communication device, there is no limit on the ability to expand the network size. This type of data transfer is known as multi-hop networking. This type of sensor network will result in a very adaptable environmental monitoring system that does not require complete redesign when experimental goals change; instead it simply requires activation of different sensors that are already deployed in the field.  Real time data collection from the wireless network will allow researchers to view data during a storm event rather than having to wait until data are manually collected from field loggers. This will greatly help to speed along the process of collecting hydrological data, and hopefully proceed towards integrating the steps of collecting and processing data. Also, real time collection will allow for on-line data quality control and on-demand maintenance as it has already been shown that anomalies in sensor network readings can reliably predict sensor failures (Szewczyk et al. 2004). This will lead to less data loss and better data quality as the malfunctioning instruments can be repaired right away instead of left running in the field because they are assumed to still be functioning.  14  2.1.3  Current state of development  The possibility of inexpensive and simple high resolution hydrologic monitoring is getting closer and closer to reality. Crossbow Technology, Inc. ™ began selling the world’s first open architecture wireless sensing platform in 2002 (Lee 2007). The Mica platform (known as a mote) is a flexible and inexpensive device made of off the shelf components that is capable of sensing, data routing, communicating with other devices, storing data and on site processing (Hill and Culler 2002).  Since they became available in 2002, Crossbow motes have been experimented with in hundreds of lab and indoor settings (Hill and Culler 2002), but not nearly as many field settings. Field experimentation with the Crossbow motes is necessary to evaluate if they can deliver on their promises of a flexible and inexpensive wireless environmental monitoring system in situations more challenging than the sterile lab setting. One of the first field experiments with goals beyond just testing the Crossbow motes is the Great Duck Island experiment in Maine, USA. This experiment focused on non-intrusive monitoring of the nesting habits of the Storm Petrel by using light sensors to determine when the birds are inside their burrows (Szewczyk et al. 2004). Researchers in this experiment are part of the computer science department of the University of California, Berkeley. UC Berkeley was the initial developer of both the Crossbow motes (Hill and Culler 2002) and the TinyOS open source operating system (Levis et al. 2005). This experiment included 43 nodes, each measuring light, temperature and humidity. In the Great Duck Island experiment, the initial 4 months of data had an average daily success rate ranging from 70 to 95% (excluding an approximately 2 week interval where the entire system was down. Considering that this type  15  of monitoring would not have been possible at all with traditional logging it can be considered quite successful; however, they did experience many node failures for unexplained reasons.  Outside of the natural environment, there have also been industrial field experiments with motes to determine their readiness for use beyond the realm of scientific research. Glaser, in 2004 investigated the applicability of motes, in their current development form to specific industrial monitoring applications. This experiment was one of the first to focus on how ready the TinyOS operating system and wireless sensor nodes are for real world applications. On paper, the motes offered by Crossbow seem like an ideal candidate for identifying structural damage in buildings, as “dense, inexpensive instrumentation is needed to identify structural damage and prognosticate future behaviour” (Glaser 2004). However, Glaser concluded that “the hardware is not electronically robust”, and the software is not user friendly enough to be used by most engineers in an industrial scenario yet. Of the twenty deployed wireless nodes, only nine gave useful data. Seven of the nine working nodes worked the entire time, with the other two only partially working. For the working nodes, the wireless network was unsuccessful; Glaser had to resort to individually downloading each node, thus eliminating a main advantage of using motes for such an experiment.  Anastasi and Falchi et al (2004) focused on field testing the motes to determine how environmental factors impacted their performance as a wireless network. They evaluated the main indicators of embedded sensor network health including node interference, power  16  consumption and effects of weather. A major finding of this study was the drastic decrease of transmission distances from 55 m down to 10 m of the motes when exposed to rain or fog.  2.1.4  Objectives  The objective of this study was to test the viability of configurable sensor network hardware (motes) in a hydrological data collection experiment in a complex environment (forested watershed). Since their lesser cost and power requirements when compared to traditional data loggers (e.g. Campbell Scientific ™) are already known and there has already been evaluation of their wireless networking performance (Anastasi et al. 2004), our particular interest is their ability to reliably collect and store quality data. We are also interested in determining if the potential of greatly reduced data collection time and effort is worth the increased complexity of initial setup for scientists who do not have extensive knowledge of computer science (Glaser 2004).  2.2  Methods  2.2.1  Study Site  Field experiments were carried out in a 7 ha rain dominated forested catchment in Malcolm Knapp Research Forest, a research forest operated by the University of British Columbia in Maple Ridge, BC, Canada. The catchment base point is located at 49.26º N and 122.55º W. Surface makeup of the catchment is a thin layer of glacial till (0-100 cm deep) with protruding granitic outcroppings. Much of the forest floor is covered with a layer (10 or more centimeters thick) of degrading organic matter as well as a large amount of degrading fallen trees and branches. Trees are predominately western red cedar (Thuja plicata),  17  Douglas fir (Pseudotsuga menziesii) and western hemlock (Tsuga heterophylla); stand age is approximately 70 years. The catchment is located in the Coastal Western Hemlock biogeoclimatic zone (Egan 1999), has an annual average temperature of 9.6ºC and receives an average of 2200mm of rain per year (Canada 2006). This catchment receives the majority of precipitation in the form of rain, with the greatest amount of precipitation in November, December and January. An intermittent headwater stream in the catchment functions as a tributary to the North Allouette River and flows roughly from October to June. A forest service road bisects the catchment at roughly the middle elevation; this road is also shown on Figure 2.1.  18  5456750  UTM U10 Northing  5456700  5456650  5456600  Gauged Base Weir Gauged Culvert Instrument  5456550  Road  5456500  *5 meter contour intervals 532450  532500  532550  532600  532650  532700  532750  UTM U10 Easting  Figure 2.1. Map of the test catchment in Malcom Knapp Research Forest, British Columbia, Canada.  The study catchment included 41 mote-based measurement stations. Measurement station locations were chosen with the desire to cover the different range of topographic features present in the catchment at the same time as spatially covering the catchment as evenly as possible. As a result, they cover different slope angles, soil depths, and distances from trees. Instrument cluster locations were determined using a Trimble Pathfinder™ GPS with a stated accuracy of 20 cm.  19  2.2.2  Hardware Description  Each measurement station in this study catchment included a powered data box with a Crossbow Technologies, Inc. MDA300 data acquisition board, and MICA2 processor board. Each parameter was measured and recorded every 15 minutes and stored in flash memory on the MICA2 processor boards. When fully operational, the MICA2 processors (motes) will communicate with each other over a radio frequency of 433 MHz to form a wireless sensor network. However, our first priority was to develop a well functioning instrument cluster that can store the measured data reliably and communicate only directly with a base station to download the data, and as such must be individually polled by a wireless base station attached to a laptop computer. The motes run the open source operating system TinyOS, which uses the NesC programming language (Gay et al. 2003).  A custom enclosure to house the Mica2 motes, MDA300 environmental data acquisition boards and power supply (two D-cell alkaline batteries) was constructed from robust GSI Industries ™ lexan plastic boxes. Using screw in rubber cable grommets, and silica desiccant, it was possible to maintain a low humidity level in the mote enclosure even throughout the winter months. The use of clear lexan made it possible to view the MICA2 processors and power supply without opening the box, which eliminated the need to open mote boxes to make sure they were still operating. The entire instrument cluster is shown in Figure 2.2.  20  Figure 2.2. Measurement station ready for deployment. (1) Pressure Transducer; (2) Mote with power supply; (3) Tipping Bucket; (4) Soil Moisture Probe; (5) Air Temperature/Humidity probe with radiation shield; (6) Ground Temperature Thermistor; (7) Overland flow weir.  Each measurement station included the following measurements: air temperature, humidity, soil temperature, rainfall, soil moisture, and groundwater head. Sixteen of the 41 locations also included a sensor to measure overland flow (Srinivasan et al. 2000). Air temperature and humidity were measured using Humirel HTM 2500 transducers (Humirel 2002); soil temperature was measured using a thermistor encased in a thin layer of epoxy and inserted 15 cm into the ground. Throughfall was measured using Rainwise, Inc. Rainew tipping buckets  21  (Rainwise 2007). The tipping buckets were factory calibrated to tip every 0.25 mm of rainfall and mounted on posts 1 m above the forest floor. Soil moisture was measured using Decagon Devices ECH2O dielectric aquameters (Decagon 2006). ECH20 aquameters have an advertised accuracy of ± 4%. Groundwater level was measured at each site in pvc piezometers inserted into the ground down to bedrock (usually 1 to 1.5 m) with a porous section in the bottom 10cm of length. The water level was recorded with 4 m SensorTechnics pressure transducers with a range of 0-3 psig and a sensitivity of 0.1% of the total length (SensorTechnics 2007). Overland flow was recorded using small (5 cm) v-notch weirs inserted into the ground perpendicular to the fall line and recorded in binary (yes/no) using a simple float switch. See Table 2.1 for a summary of instrumentation at each measurement location.  Table 2.1. Instrument cluster functions. Parameter Soil Moisture Througfall Groundwater level Air Temperature Humidity Overland Flow Ground Temperature Battery Power Internal Temperature Internal Humidity  Unit % by volume mm cm ºC % Yes/No ºC V ºC %  Instrument Decagon Devices ECH2O dielectric aquameter Rainwise, Inc. Rainew tipping bucket Sensor Technics pressure transducer Humirel HTM 2500 transducer Humirel HTM 2500 transducer Custom Weir with binary float switch Thermistor Internal to MDA300 Internal to MDA300 Internal to MDA300  One of the major goals of this project was evaluating the viability of deploying dense sensor networks in a minimum amount of time and with minimum cost. A dense sampling network of lower cost sensors can provide as much or more useful data than a less dense network of expensive instruments (Delin 2002; Mainwaring et al. 2002). In the spirit of this goal, we  22  relied upon the factory supplied calibration of the relatively low-cost instrumentation that we used.  2.2.3  Software Description  Operation of a mote network currently requires the ability to develop custom software for the TinyOS operating system using the NesC programming language (Gay et al. 2003).  2.2.3.1 Was written by Kan Cai and is located in Appendix A.  2.2.3.2 Data conversion software Data was downloaded from each mote in hexadecimal format with only raw voltage readings for analog sensors (everything except the tipping buckets and overland flow sensors) and relative time starting from 0 at power up. Using IDL software (Stern 2003), a custom program was created that would convert the raw data from hexadecimal voltage readings into usable information as well as insert the actual date and time of each reading. This program is shown in Appendix B.  2.3  Timeline  Implementation of the crossbow sensor network has gone through multiple development stages. Initial testing of the mote platform was indoor, lab style testing. This began in June 2005, and was the sole location of mote operations until January 2006. During this time, TinyOS software was developed to operate all necessary instruments as well as save collected information in the onboard flash memory of the Mica2 sensors.  23  In January 2006, a pilot field study of 10 identically programmed motes was deployed to Malcolm Knapp Research Forest. Of the first 10 motes deployed for 2 weeks, 8 functioned somewhat successfully, and for unclear reasons two did not collect any data. After this initial two week data collection, the experiment was expanded to 20 sensors that each alternated between 2 locations every month until June, 2006. During this time each mote had to be hardwired and reprogrammed to enable downloading, which proved to be much more time consuming than anticipated. In June 2006 we moved TinyOS operation from a Windows XP system to an Ubuntu Linux operating system. At this time the experiment was expanded to 41 measurement locations and wireless downloading was enabled. Data collection continued in this manner until April 20, 2007.  Implementation of multi-hop networking was and is a major goal of this project. Initial testing of a multi-hop wireless network began in July 2006, with successful lab based testing occurring in January/February 2007.  2.4  Results  2.4.1  Power Usage  Each mote was powered with 2 D-cell alkaline batteries (14000 mAh each). Batteries were replaced approximately every 30 days throughout the course of the experiment. During one interval during the winter of 06/07, however, batteries were left in the motes until they were completely discharged due to our inability to access the catchment. The benefit to this was that it gave us a chance to evaluate battery performance during the harshest conditions  24  encountered (very wet conditions with temperatures generally between -2 and 5º C). Results of this test can be seen in Table 2.2. Though Crossbow motes have an advertised operating range of 2.7 to 3.3 V, our experiment showed reliable operation at as low as 2.55 V and some operation as low as 2.23 V.  Table 2.2 Run time results (winter)  Run time Value at Failure  Unit Days Volts  AVG 34.30 2.37  STDEV 4.90 0.09  MIN 24.50 2.23  MAX 42.00 2.55  It would be possible to greatly increase the run time of the motes to well over 1 year with the same batteries with the use of the sleep function, where motes are only running for a few seconds at every measurement time (Anastasi et al. 2004). Putting the motes to sleep was not feasible in this experiment due to the use of tipping buckets, because each mote had to be awake at all times to record when tips happened and sum the total amount of tips that occurred in each 15 minute interval.  2.4.2  Reliability  As with any field experiment, uncontrollable environmental issues affected this project. Specifically, there were four environmental factors that affected this project. The first of these factors was wind. Individual instrument clusters were knocked over (or just knocked out of alignment enough to go off line) by falling branches, and during the most severe winter storm event, fallen trees affected two sensing locations. Black bear interaction with  25  tipping buckets also happened numerous times. These interactions ranged from minor cases of unlevel tipping buckets, to a case of complete destruction of the tipping bucket and dislodging of the MICA2 processor board. Falling debris from the forest canopy was very problematic to the tipping bucket throughfall measurements. Numerous times, particularly in autumn, tipping bucket filter screens became completely clogged by debris; tipping bucket data collected during this time was unusable for these locations back to the most recently known unclogged time. During winter, snow and ice blockage of tipping buckets was another source of malfunction.  Some of the temperature probes produced erroneous readings when exposed to temperatures near or below freezing temperatures. Because the Humirel HTM 2500 has an operating range of -30 to 70ºC (Humirel 2002), we suspect that this error resulted from the Mica2 rather than errors from the temperature probes themselves. Different motes had this error at different times, and it affected six motes in total.  A major disappointment in this experiment was with the groundwater head measurement. Pressure transducers had problems throughout the entire experiment. The SensorTechnics pressure transducers that were used in this experiment arrived with a factory conversion formula for converting the raw output voltage to meters. The calibration method requires combining this formula with an offset number experimentally determined by the 0 m level of water in each probe. During the developmental stages of the TinyOS software, pressure transducers were tested with this technique using a Campbell Scientific CR10 logger. Results from these tests indicated a very high degree of linearity between output voltage and  26  water depth (r2 = 0.999). When the identical program for operating the SensorTechnics pressure transducers was written for TinyOS the transducers seemed to function normally, but in the field we could not get the pressure transducers to operate at an acceptable level. Of the 41 operational instrument clusters, four had normally operating pressure transducers, 25 gave unreliable data and twelve had no response whatsoever. While we have so far been unable to determine the exact reason for this failure, we suspect that an unstable supply voltage from the MDA300 may be the source of the problem.  Aside from the data reliability problems described above, there were numerous times when instruments and instrument clusters ceased operation for unknown reasons. These nondeterministic errors are summarized in Table 2.3. These statistics were compiled with a custom IDL program shown in Appendix C.  Table 2.3. Percentage of usable data during the main sampling period. Parameter Soil moisture Air Temperature Humidity Soil Temperature Battery Voltage (int.) Humidity (int.) Temperature (int.)  AVG 84.60 99.37 96.01 98.32 100.00 100.00 99.96  MED 100.00 99.74 99.99 100.00 100.00 100.00 100.00  STDEV 29.81 1.14 17.13 10.45 0.00 0.00 0.21  MIN 0.01 95.31 0.01 34.71 99.99 99.99 98.78  As illustrated in Table 2.3, measurements that were internal to the MDA300 had very good reliability. They were functional for almost the entire time that the Mica2 was functional. With a few exceptions, the soil temperature readings and readings from the Humirel probes (air temperature and humidity) gave very good results as well. Soil moisture readings were somewhat less reliable; the large standard deviation shows that there were numerous motes  27  with poor reliability results for soil moisture. Because of previously described problems with the pressure transducers, groundwater level was not included in this chart, as this measurement was almost completely unreliable. Instruments run on the digital channels of the MDA300 also were not included, as it was impossible to know for sure if a reading of 0 simply meant that there was no activity, or if there was a malfunction with the instrument. However, having 41 different motes in operation did allow for cross checking of throughfall measurements for data quality; if there were individual measurements that did not show any measured throughfall when all others were recording we knew if there was a problem with that individual instrument.  2.5  Discussion  This paper describes the first stage in our project of developing an automated monitoring system for a hydrologic catchment. The main task left in this project is bringing the catchment experiment online. As stated previously, initial lab testing of the multi-hop wireless functionality of our motes began in January 2007. When redeployed to a field experiment in the spring of 2008, this network will allow for data to be transmitted from all instrument clusters to a base point somewhere in the catchment while still being recorded on the internal memory of the MICA2 processor board for backup. A more powerful radio modem will then transmit these data to a computer located in the Malcolm Knapp field office in order to connect to the University of British Columbia network and make the data available in real time. This will allow for easy monitoring of the data being collected in the catchment in nearly real time from anywhere with internet access, without the necessity of being in the catchment.  28  Another issue that needs addressing from this experiment is our inability to use the sleep function of motes to save battery life. Using separate loggers designed specifically for counting tips in the throughfall gauges and connecting them to the MDA300 is a possible solution for the inability to use the sleep function of the Mica2. This will move us much further forward towards the goal of a fully functioning data collection network that requires little field maintenance.  While motes certainly have an extremely large potential for hydrologic measurement, results of this experiment have shown that they may not yet be reliable or user-friendly enough for widespread use by hydrologists. Motes in this experiment often gave erroneous readings for unknown reasons. Because the software and hardware setup was identical for each mote, inconsistencies in the function of the MICA2 and/or MDA300 are the only places left to look for reasons for the unknown data failure. We hope and expect that future generations of the MICA platform will be more robust and consistent in this area.  Aside from inconsistency in the hardware, we feel that a major factor preventing motes from revolutionizing environmental data collection right now is software. Currently, an in-depth knowledge of computer programming in the NesC language is necessary for operation of motes in TinyOS. They are much more difficult to configure for operation than a traditional data logger and are still in the research stage instead of the operational phase. This fact makes motes less appealing for most hydrologists, who are interested in spending time understanding what collected data means than configuring a sensor network. This will most  29  likely change in the near future, however, as software is being developed to make motes easier to use, such as MoteWorks 2.0, a software developing environment containing many tools that will allow for easier initialization and operation of environmental monitoring systems using motes. MoteWorks 2.0 also promises better stability and much greater data transmission reliability than TinyOS (Crossbow 2007).  Another attribute of mote technology that needs to be considered is the extremely rapid advancements being made in the field. Multiple new versions of hardware have been available, and each has been an improvement on the last. This rapid advancement in technology is good for development, as problems can be fixed with new hardware, but it may actually be a detriment to widespread use of motes in hydrology or other environmental sensing. These types of data collection projects require stability in hardware and the ability to replace equipment if it becomes damaged by uncontrollable environmental factors. Even through the course of this project, the MICA2 processor boards that we have used are already being phased out of production for newer hardware. While newer hardware will likely solve some problems that we had during this study, this means that replacing any equipment that breaks in the field will be difficult. It is not feasible for hydrologists to purchase new hardware every few years for long term measurement projects, and technology development will have to stabilize before mote networks become a viable option for large scale environmental sensing projects.  30  2.6  Conclusion  We see a great potential for motes to allow hydrologists to perform experiments that were previously logistically impossible. They will be able to supply extremely rich data in both space and time at a low price that is only limited by the price of the sensors that attach to them. This potential, however, has not been reached yet; hydrologists are interested in reliability, stability and ease of use for data collection. So far in this project we have found that these three important qualities have not yet reached a point that warrants widespread use of motes in hydrology; they need to be refined into more mature environmental data collection hardware before they can compete with the current methods of data collection and recording.  31  2.7  References  Anastasi, G., A. Falchi and A. Passarella (2004). Performance Measurements of Motes Sensor Networks. Proceedings of the 7th ACM international symposium on Modeling, analysis and simulation of wireless and mobile systems. Venice, Italy: 174-181. Canada (2006). "National Climate Data and Information Archive." Retrieved October 2, 2007, from http://climate.weatheroffice.ec.gc.ca/climateData/dailydata_e.html. Crossbow (2007). "MoteWorks 2.0 Software Platform." Retrieved January 10, 2008, from http://www.xbow.com/Products/Product_pdf_files/Wireless_pdf/MoteWorks_OEM_Edition. pdf. Decagon (2006). "ECH2O Soil Moisture Probe Operator's Manual." Retrieved November 10, 2007, from http://www.decagon.com/manuals/echomanual.pdf. Delin, K. A. (2002). "The Sensor Web: A Macro-Instrument for Coordinated Sensing." Sensors 2: 270-285. Egan, B. (1999). "The Ecology of the Coastal Western Hemlock Zone." Ecosystems of British Columbia Retrieved February 10, 2008, from http://www.for.gov.bc.ca/hfd/pubs/docs/Bro/bro31.pdf. Gay, D., P. Levis, R. v. Behren, M. Welsh, E. Brewer and D. Culler (2003). The nesC Language: An Holistic Approach to Networked Embedded Systems. ACM SIGPLAN 2003 conference on Programming language design and implementation, San Diego, CA, USA. Glaser, S. D. (2004). Some real-world applications of wireless sensor nodes. SPIE Symposium on Smart Structures & Materials, San Diego, California. Grayson, R. B., A. W. Western, G. Bloschl and F. H. S. Chiew (1997). "Preferred states in spatial soil moisture patterns: local and nonlocal controls." Water Resources Research 33(12): 2897-2908. Hill, J. L. and D. E. Culler (2002). "Mica: A Wireless Platform for Deeply Embedded Networks." IEEE Micro 22(6): 12-24. Humirel (2002). "Relative Humidity/ Temperature Module Technical Data." Retrieved January 25, 2008, from http://www.humirel.com/product/fichier/HTM2500.pdf. Lee, S. (2007). "Crossbow Technology, Inc. Quick Facts." Retrieved January 7, 2008, from http://www.xbow.com/General_info/Info_pdf_files/Crossbow_Quick_Facts.pdf. Levis, P., S. Madden, J. Polastre, R. Szewczyk, K. Whitehouse, A. Woo, D. Gay, J. Hill, M. Welsh, E. Brewer and D. Culler (2005). TinyOS: An Operating System for Sensor Networks.  32  Ambient Intelligence. W. Weber, J. M. Rabaey and E. Aarts. New York, NY, Springer: 115148. Mainwaring, A., J. Polastre, R. Szewczyk, D. Culler and J. Anderson (2002). Wireless Sensor Networks for Habitat Monitoring. Proceedings of the 1st ACM International Workshop on Wireless Sensor Networks and Applications. Atlanta, GA, ACM Press: 88-97. Motovilov, Y. G., L. Gottschalk, K. Engeland and A. Rodhe (1999). "Validation of a distributed hydrological model against spatial observations." Agricultural and Forest Meteorology 98-99: 257-277. Rainwise (2007). "Wired Rain Gauge." from http://www.rainwise.com/products/detail.php?ID=6697&Category=Rain_Gauges:Wired&pa geNum_cart=/products/index.php. Refsgaard, J. C. (1997). "Parameterisation, calibration and validation of distributed hydrological models." Journal of Hydrology 198: 69-97. SensorTechnics (2007). "OEM Stainless Steel Submersible Pressure Transducers." Retrieved January 10, 2007, from http://www.sensortechnics.com/download/cte-ctu9000cs594.pdf. Seyfried, M. S. and B. P. Wilcox (1995). "Scale and the Nature of Spatial Variability: Field examples having implications for hydrologic modeling." Water Resources Research 31(1): 173-184. Srinivasan, M. S., M. A. Whittman, J. M. Hamlett and W. J. Gburek (2000). "Surface and subsurface sensors to record variable runoff generation areas." Transactions of the American Society of Agricultural Engineers 43(3): 651-660. Stern, D. (2003). IDL Student Edition. Boulder, CO, Research Systems, Inc./ ITT Visual Information Solutions. Szewczyk, R., J. Polastre, A. Mainwaring and D. Culler (2004). "Lessons from a Sensor Network Expedition." Lecture Notes in Computer Science 2920: 307-322. Western, A. W., R. B. Grayson and G. Bloschl (2001). "Towards capturing hydrologically significant connectivity in spatial patterns." Water Resources Research 37(1): 83-97. Western, A. W., R. B. Grayson, G. Bloschl, G. R. Willgoose and T. A. McMahon (1999). "Observed spatial organization of soil moisture and its relation to terrain indices." Water Resources Research 35(3): 797-810. Western, A. W., R. B. Grayson and T. R. Green (1999). "The Tarrawarra project: high resolution spatial measurement, modelling and analysis of soil moisture and hydrologic responses." Hydrological Processes 13: 633-652.  33  Wilson, D. J., A. W. Western and R. B. Grayson (2005). "A terrain and data-based method for generating the spatial distribution of soil moisture." Advances in Water Resources 28: 4354. Yu, Z., W. J. Gburek and F. W. Schwartz (2000). "Evaluating the spatial distribution of water balance in a small scale watershed, Pennsylvania." Hydrological Processes 14: 941-956.  34  Chapter 3:  Effects of Sample Size on Catchment Averaged Soil Moisture and  Throughfall Measurement1  3.1  Introduction  As the main input of water into a hydrologic catchment, throughfall is an important factor in the interaction between soil and the atmosphere. Rainwater that is not intercepted by the canopy and returned to the atmosphere enters a catchment as throughfall (Viessman and Lewis 2003). Throughfall is spatially variable due to both the variability of the storms producing rainfall (Shah et al. 1996) and structure of the canopy (Keim et al. 2005). Because of this, obtaining a spatially averaged throughfall value for use in a hydrologic model necessitates multiple measurements throughout a forested catchment. A question that hydrologists must answer when designing experiments to measure throughfall is: how many measurements are adequate, and are the same number of measurements suitable for the entire range of storm events encountered?  Water in the unsaturated zone of soil is the boundary layer between the water table and the soil surface and as such is integral to understanding where and how water moves between different reservoirs in the water cycle. Water from the surface infiltrates into the soil and either percolates through the unsaturated zone into the saturated zone (Viessman and Lewis 2003) or remains in the unsaturated zone and moves through this layer  1  A version of this Chapter will be submitted for publication.  J.Trubilowicz and M. Weiler. “Effects of Sample Size on Catchment Averaged Soil Moisture and Throughfall Measurement.”  35  as unsaturated flow (Bedient and Huber 2002). Measurement of soil moisture content in the unsaturated zone is necessary for determining how much water exists in the unsaturated zone and if this amount is changing. Because soil moisture content is thought to be controlled by different scales of factors at different wetness levels, ranging from small scale soil properties during dry periods to topography at a larger scale during wet periods (Grayson et al. 1997), it is necessary to monitor soil moisture at a wide range of spatial resolutions. Ideally, the spatial resolution used will depend on the scale of data desired. Sometimes, however, it is not possible to monitor soil moisture at the resolution required. In these cases it is necessary to either upscale or downscale measurement data. The effectiveness of changing scale and potential to introduce bias based on the ratio of the measurement scale to the scale of natural variability was explored on extremely high resolution soil moisture data of the Tarrawarra catchment in south-eastern Australia (Western and Bloschl 1999).  In field experiments, a balance must be struck between the many different factors that affect the number and location of samples. These are often based on available resources (time, financial, human, etc.) and environmental factors such as inaccessible monitoring locations (Talzi et al. 2007). Because of these limiting factors, it is important to know the minimum amount of monitoring points that are necessary to obtain the desired result.  With this experiment our objective was twofold. The first objective was to evaluate if increasing the sample size (and consequently the sampling density) through resampling always increased our ability to measure the true mean soil moisture and throughfall values  36  for the entire catchment. The second goal was to determine whether there is a change in the necessary sampling density to accurately estimate the mean value in the same catchment for different events. In contrast to studies that have worked to optimize the individual sampling scheme to get the best results from each catchment in terms of spatial interpolation within the catchment (Olea 1984; Ferreyra et al. 2002; Zio et al. 2004), we focused purely on sampling point density and its effects on measured values. Decreasing variance as sample density increases has been explored before, in a resampling analysis of the soil moisture data at the Tarrawarra catchment (Park and Giesen 2004), but the main focus of that analysis was the differences in standard error between stratified and random sampling. Our analysis added in throughfall and focused more specifically on whether or not increasing the density wass always useful and the variation in our ability to predict the catchment mean values across different storm events.  3.2  Methods  3.2.1  Study Site  Field data were collected in a 7 ha rain dominated forested catchment in Malcolm Knapp Research Forest, a research forest operated by the University of British Columbia in Maple Ridge, BC, Canada. The base of the catchment is located at 49.26º N and 122.55º W. Surface makeup of the catchment is a thin layer of glacial till (0-100 cm deep) with protruding granitic outcroppings. Much of the forest floor is covered with a layer (10 or more centimeters thick) of degrading organic matter as well as a large amount of degrading fallen trees and branches. Trees are predominately western red cedar (Thuja plicata), Douglas fir (Pseudotsuga menziesii) and western hemlock (Tsuga heterophylla). Stand age  37  is approximately 70 years. The catchment is located in the Coastal Western Hemlock biogeoclimatic zone (Egan 1999), has an average annual temperature of 9.6º C and receives an average of 2200 mm of precipitation per year. This catchment receives the majority of precipitation in the form of rain, with the heaviest precipitation in November, December and January (Canada 2006). An intermittent headwater stream in the catchment functions as a tributary to the North Allouette River and flows roughly from October to June. A forest service road bisects the catchment at roughly the middle elevation and is shown on Figure 3.1. The digital elevation model (DEM) of the catchment shown in Figure 3.1 was derived in a previous field experiment in the same catchment (Moore and Thompson 1996).  The study catchment featured 41 measurement stations using a novel method of data collection and storage. Each measurement station included a powered data box with a Crossbow Technologies, Inc.™ MDA300 data acquisition board, and MICA2 processor board, known as a mote (Hill and Culler 2002; Lee 2007). The motes ran the open source operating system TinyOS (Levis et al. 2005), which is configured using the NesC programming language (Gay et al. 2003). The functions of TinyOS can be tailored to the needs of a specific application; in this case it operated all sensors attached to the MDA300, collected and stored data for seven external variables every 15 minutes, and allowed for wireless downloading of data from each measurement station.  Measurements relevant to this experiment were soil moisture and throughfall. Both of these parameters were recorded every 15 minutes, from July 10, 2006, to April 22, 2007. Throughfall was collected using Rainwise, Inc. Rainew tipping buckets which have an  38  advertised resolution of 0.25 mm (Rainwise 2007). Soil moisture was measured using Decagon Devices ECH2O dielectric aquameters. The ECH2O probes are 20cm long and were inserted so that the top was 10cm into the inorganic layer between 20 and 50cm from the measurement station. The organic layer of soil was removed for insertion and then replaced afterwards. Factory calibrations of the soil moisture probes give an advertised accuracy of ± 4% (Decagon 2006). Lab testing showed that the precision was ± 0.5%.  Measurement locations were chosen with the desire to have a fairly uniform spatial coverage; however, we could not use a random number generation or uniform grid because there were many locations that would be impossible to install a measurement location (due to exposed bedrock, standing trees or fallen trees. As a result, measurement locations were chosen while walking through the catchment with the desire to cover the different range of topographic features present in the catchment. They covered different slope angles, soil depths, and distances from trees. The (+) symbols in Figure 3.1 show the location of each measurement location in the study catchment. Instrument cluster locations were determined using a Trimble Pathfinder™ GPS with a stated accuracy of 20 cm. There is a lack of measurement points in the upper catchment because we initially spaced out locations to fit 50 measurement stations, but ended up with only 41.  Other measured variables included at each measurement station were: air temperature, humidity, soil temperature, and groundwater head. Sixteen of the 41 locations included a sensor to measure overland flow (Srinivasan et al. 2000). Other instrumentation in this catchment included 3 V-notch weirs with water level logged every 10 minutes for the  39  duration of the experiment. Two of these weirs were used to measure the flow through two culverts beneath a road that is intersecting the study catchment. The third weir was located at the base of the catchment in order to measure outflow from the catchment. These weirs are shown with the (▼) symbol in Figure 3.1.  5456750  4 19 17 18 2  UTM U10 Northing  5456700  9  29 5456650  8  5 6  3  16 11  21  15 13 7 12 10 20 14  35  34  26 32 33 31 2240 30 24 38  5456600  Gauged Base Weir Gauged Culvert  25 37 28  5456550  41  Instrument  23  Road  27  36 39 43 5456500  *5 meter contour intervals 532450  532500  532550  532600  532650  532700  532750  UTM U10 Easting  Figure 3.1. Measurement locations within the study catchment.  3.2.2. Data Analysis We selected 4 events and 3 full months from the dataset collected between July 10, 2006 and April 22, 2007 for analysis. The 4 events were selected to give a representation of the different hydrologic situations encountered in the catchment. As the use of emerging  40  technology always has some operational issues, we also had to factor in the quality of collected data available when choosing events. For the full month selections, we chose the 2 wettest months and the driest month that occurred during our sampling period. In the interest of streamlining visuals and calculations, we used hourly data for the shorter events (A-D) and daily data for the entire months (E-G). Raw data from each mote were selected and reorganized into files containing throughfall and soil moisture for all motes throughout the event using IDL (Hill and Culler 2002). The program used for combining raw data into events is shown in Appendix D. A summary of each event is shown in Table 3.1.  Table 3.1 Summary of analyzed events. Start and end dates are inclusive. ID  Start Day  End Day  dt  A B C D E F G  18-Sep-06 21-Mar-07 25-Mar-07 7-Apr-07 1-Aug-06 1-Nov-06 1-Mar-07  20-Sep-06 24-Mar-07 29-Mar-07 8-Apr-07 31-Aug-06 30-Nov-06 31-Mar-07  Hourly Hourly Hourly Hourly Daily Daily Daily  Total Precip. mm 64.2 169.0 0.0 30.6 17.0 438.2 371.1  Avg. Temp. Deg. C 12.70 12.23 10.70 11.75 17.72 4.13 6.50  *(Canada 2006)  Description Large storm in dry season Large storm in wet season Dry period in wet season Moderate storm in wet season Dry month Wettest month Wet month  The first step in data analysis was finding the mean measured value (whether throughfall or soil moisture) for the entire catchment and plotting it over time throughout each entire event. This overall catchment average served as the best-known or “true” value for the entire catchment; it was the mean value of all functioning measurement stations over time.  To determine how different sample sizes affected the catchment mean value, we created data subsets of different sample sizes from our complete sample of 41 measurement stations. We randomly selected groups of 5, 10 and 20 sampling points 100 times each (we refer to each  41  randomly selected group as a realization) to evaluate how successful randomly selected measurement points can be in determining an overall catchment value. These sample sizes corresponded to densities of 0.7, 1.4 and 2.8 measurements/ha, respectively. Comprehensive sampling density was 5.9 points/ha. The mean value for all of the functioning randomly selected points (whether it was 5, 10, or 20) was calculated and plotted over time for each realization. Each of the 100 realizations for the same sample size was plotted on the same graph with the best-known catchment mean. Creating these three ensemble plots for each event allowed us to visualize the effect that increasing the sample size had on our ability to reproduce the best-known mean for both soil moisture and throughfall.  After creating the ensemble plots for each sample size and each event, we calculated the root mean square error (RMSE) and the Pearson product-moment correlation coefficient (r) (McBean and Rovers 1998; Stern 2003) between each realization and the best-known mean. The ensemble plots and RMSE were calculated using the IDL program included in Appendix E and the Pearson r for each realization was calculated using the IDL program in Appendix F.  This resulted in groups of 100 RMSE values and 100 Pearson r values for each ensemble. We used box plots to visually represent how the RMSE decreased and Pearson r increased as sample size increased from 5 to 10 to 20 points. These representative box plots show the minimum, lower quartile, median, upper quartile and maximum value of each respective group of RMSE and Pearson r values.  42  Beyond just visual comparison, a more robust method was necessary to compare the box plots. These comparisons were made with two goals for both soil moisture and throughfall. The first goal was to determine if increasing the subset sample size from 5 to 10 to 20 points significantly decreased the RMSE (or increased Pearson r) for the ensemble of 100 random realizations. To answer this question, we performed statistical tests to compare between the different sample sizes for the same event. The second goal was to determine if sample subsets of equal size had significantly different RMSE (or Pearson r) values during different events. For these comparisons, events were separated between full months (Events E-G) and shorter events (A-D) for relevance. The full testing scheme used for comparing both RMSE and Pearson’s r distributions is shown in Table 3.2 below. All comparisons involving events C and E were omitted for throughfall because no precipitation was recorded during these times. All tests used a significance level of 0.05.  43  Table 3.2. Testing Scheme Short Events Full Months Changing Sample Size Tests A - 5 pt vs. A - 10 pt E - 5 pt vs. E - 10 pt A - 10 pt vs. A - 20 pt E - 10 pt vs. E - 20 pt B - 5 pt vs. B - 10 pt F - 5 pt vs. F - 10 pt B - 10 pt vs. B - 20 pt F - 10 pt vs. F - 20 pt C - 5 pt vs. C - 10 pt G - 5 pt vs. G - 10 pt C - 10 pt vs. C - 20 pt G - 10 pt vs. G - 20 pt D - 5 pt vs. D - 10 pt D - 10 pt vs. D - 20 pt Across Event Tests A - 5 pt vs. B - 5 pt E - 5 pt vs. F - 5 pt A - 5 pt vs. C - 5 pt E - 5 pt vs. G - 5 pt A - 5 pt vs. D - 5 pt F - 5 pt vs. G - 5 pt B - 5 pt vs. C - 5 pt E - 10 pt vs. F - 10 pt B - 5 pt vs. D - 5 pt E - 10 pt vs. G - 10 pt C - 5 pt vs. D - 5 pt F - 10 pt vs. G - 10 pt A - 10 pt vs. B - 10 pt E - 20 pt vs. F - 20 pt A - 10 pt vs. C - 10 pt E - 20 pt vs. G - 20 pt A - 10 pt vs. D - 10 pt F - 20 pt vs. G - 20 pt B - 10 pt vs. C - 10 pt B - 10 pt vs. D - 10 pt C - 10 pt vs. D - 10 pt A - 20 pt vs. B - 20 pt A - 20 pt vs. C - 20 pt A - 20 pt vs. D - 20 pt B - 20 pt vs. C - 20 pt B - 20 pt vs. D - 20 pt C - 20 pt vs. D - 20 pt  The Shapiro-Wilk W test for normality (McBean and Rovers 1998) showed that the RMSE distributions represented in the box plots were lognormal. Because of this we logtransformed the datasets and performed the f-Test to test for equal variances according to the testing scheme described in Table 2. If the selected group passed the f-Test, we performed a student’s t-test for significantly different means assuming equal variance. If the group failed the F-test, we performed the t-test assuming unequal variance (McBean and Rovers 1998; Stern 2003). The F and T tests were performed using the IDL program in Appendix G.  44  The Shapiro-Wilk W test showed that none of the distributions of Pearson r values represented in the box plots were normal or lognormal. Because of this we used the nonparametric Wilcoxon rank-sum test (Stern 2003) to test for significant differences of Pearson r distributions according to the sampling scheme in Table 3.2. The Wilcoxon tests were performed using the IDL program in Appendix H.  45  3.3  Results  Figures 3.2-3.5 are examples of how increasing the sample size decreased noise in the catchment mean when compared to the “true” (best known) catchment mean. The true catchment mean is shown on each figure as the thicker grey line.  Figure 3.2 illustrates Event A (64 mm storm during the dry season) and shows a visual improvement in our ability to estimate the catchment mean, for both soil moisture and throughfall as the sample size increased from 5 to 10 points and from 10 to 20 points. In the throughfall graphs, the greatest improvements shown are during the hours of peak throughfall.  Soil Moisture (%)  5 random points  20 random points  70  70  70  60  60  60  50  50  50  40  40  40  30  30  30  9-18 0:00  Throughfall (mm)  10 random points  9-19 0:00  9-20 0:00  9-21 0:00  9-18 0:00  9-19 0:00  9-20 0:00  9-21 0:00  9-18 0:00  10  10  10  8  8  8  6  6  6  4  4  4  2  2  2  0 9-18 0:00  9-19 0:00  9-20 0:00  9-21 0:00  0 9-18 0:00  9-19 0:00  9-20 0:00  9-21 0:00  0 9-18 0:00  9-19 0:00  9-20 0:00  9-21 0:00  9-19 0:00  9-20 0:00  9-21 0:00  Figure 3.2. Ensemble Plots for Event A. Soil moisture and hourly throughfall results are shown for 5 point, 10 point and 20 point subsets.  46  Figure 3.3 shows that the catchment mean value of soil moisture decreased throughout Event C, which was a drying period in a wet catchment. As the catchment mean decreased, there was no significant visual evidence of the distribution of realizations coming closer to predicting the true mean value. Increasing the sample size did show an improved ability to predict the catchment mean. There appears to be just as significant a change from 5 points to 10 points as from 10 to 20 (with one outlier in the 20 point sample size).  Soil Moisture (%)  5 random points  10 Random Points  20 Random Points  70  70  70  60  60  60  50  50  50  40  40  40  30  30  30  3-25 0:00  3-27 0:00  3-29 0:00  3-25 0:00  3-27 0:00  3-29 0:00  3-25 0:00  3-27 0:00  3-29 0:00  Figure 3.3. Event C. Only soil moisture is shown, as there was no precipitation during Event C.  47  Aside from the one outlier in the 5 point sample size for soil moisture (above the mean), Figure 3.4 does not visually give much evidence of a benefit of increasing the sample size during Event D (30 mm storm). There is only a slightly narrower distribution of realizations above and below the mean value for soil moisture and only slightly lower peaks in the throughfall measurement for this event. 10 random points  20 random points  70  70  60  60  60  50  50  50  40  40  40  30  30  30  4-7 0:00 4-7 12:00 4-8 0:00 4-8 12:00 4-9 0:00  4-7 0:00 4-7 12:00 4-8 0:00 4-8 12:00 4-9 0:00  4-7 0:00 4-7 12:00 4-8 0:00 4-8 12:00 4-9 0:00  Throughfall (mm)  Soil Moisture (%)  5 random points 70  5  5  5  4  4  4  3  3  3  2  2  2  1  1  1  0 4-7 0:00 4-7 12:00 4-8 0:00 4-8 12:00 4-9 0:00  0 4-7 0:00 4-7 12:00 4-8 0:00 4-8 12:00 4-9 0:00  0 4-7 0:00 4-7 12:00 4-8 0:00 4-8 12:00 4-9 0:00 Date  Figure 3.4. Ensemble Plots for Event D. Soil moisture and hourly throughfall results are shown for 5 point, 10 point and 20 point subsets.  48  Figure 3.5 (all of March 2007) shows quite an improvement in the ability of our randomly selected samples to predict the mean catchment value, both for soil moisture and throughfall at the change from 5 to 10 samples. There is a somewhat less pronounced change in the transition from 10 to 20 samples. Events B, E and F are shown in Appendix I, Figures I.1I.3.  Soil Moisture (%)  5 random points  20 random points  70  70  60  60  60  50  50  50  40  40  40  30  30  30  3/1/07  Throughfall (mm)  10 random points  70  3/8/07  3/15/07 3/22/07 3/29/07  3/1/07  3/8/07  3/15/07 3/22/07 3/29/07  3/1/07  100  100  100  80  80  80  60  60  60  40  40  40  20  20  20  0 3/1/07  3/8/07  3/15/07 3/22/07 3/29/07  0 3/1/07  3/8/07  3/15/07 3/22/07 3/29/07  0 3/1/07  3/8/07  3/15/07 3/22/07 3/29/07  3/8/07  3/15/07 3/22/07 3/29/07  Figure 3.5. Ensemble Plots for Event G. Soil moisture and daily throughfall results are shown for 5 point, 10 point and 20 point subsets.  3.3.1  Statistical Results  The box plots in Figure 3.6 summarize the RMSE calculations for soil moisture. These datasets were log-transformed and compared to each other using the student’s t-test to evaluate where there were significant changes in dataset means (McBean and Rovers 1998).  49  Soil Moisture (%)  RMSE in Event A Analysis  RMSE in Event B Analysis  RMSE in Event D Analysis  16  16  16  12  12  12  12  8  8  8  8  4  4  4  4  0  0 5 Points  10 Points  20 Points  0 5 Points  RMSE in Event E Analysis  Soil Moisture (%)  RMSE in Event C Analysis  16  10 Points  20 Points  RMSE in Event F Analysis 10  10  8  8  8  6  6  6  4  4  4  2  2  2  0 5 Points  10 Points  20 Points  10 Points  20 Points  5 Points  10 Points  20 Points  RMSE in Event G Analysis  10  0  0 5 Points  0 5 Points  10 Points  20 Points  5 Points  10 Points  20 Points  Figure 3.6: Distribution of RMSE of each realization to the true sample mean for Soil Moisture (all events).  Test results showed that for almost every event (A-G), increasing the sample size resulted in a significantly decreased mean of the resultant RMSE dataset. The one exception to this was the change from 5 sampling points to 10 in event D (P=.072). However, since there was still a significant decrease from 10 to 20 points, this result does not indicate that 5 measurement points would have been sufficient for an accurate result.  Of the 18 different t-tests performed to compare between events A-D (see Table 3.2), 10 had a significant difference between data sets. In every sample size, events A and D had no significant difference. In the 10 and 20 point sample sizes, events A and D had no significant difference from event C. In all cases but one, event B was significantly different from all other events. This one exception was for the 5 point sample size comparison of events B and C (P=.1190).  50  Of the 9 different t-tests performed to compare between events E, F and G, 7 had significant differences between datasets. The only comparisons that had no significant difference were between events F and G for the 10 and 20 point sample sizes. Full Results for these comparison tests are shown in Appendix J, Table J.1.  Figure 3.7 shows the box plots of distributions of Pearson r values for soil moisture. Wilcoxon Rank Sum tests showed that increasing the sample size almost always resulted in significant changes in Pearson r distributions (for events A-G). The one exception to this was the change from 5 to 10 points for event E (P = 0.42).  Pearson r  Pearson r: Event A  Pearson r: Event B  Pearson r: Event D  1  1  1  0.9  0.9  0.8  0.8  0.8  0.9 0.8  0.7  0.7  0.7  0.7  0.6 0.5  0.6 0.5  0.6 0.5  0.6  0.4  0.4  0.4  0.3  0.3  0.3  0.4 0.3  0.2 0.1  0.2 0.1  0.2 0.1  0.2  0  0  0  -0.1  -0.1  -0.1  0 -0.1  -0.2  -0.2  -0.2  5 Point  10 Point  20 Point  5 Point  Pearson r: Event E  Pearson r  Pearson r: Event C  1 0.9  10 Point  20 Point  1  1  0.9  0.9  0.9  0.8 0.7  0.8 0.7  0.8 0.7  0.6  0.6  0.6  0.5  0.5  0.5  0.4 0.3  0.4 0.3  0.4 0.3  0.2  0.2  0.2  0.1  0.1  0.1  0 -0.1  0 -0.1  0 -0.1  -0.2 10 Point  20 Point  -0.2 10 Point  20 Point  5 Point  10 Point  20 Point  Pearson r: Event G  1  5 Point  0.1  5 Point  Pearson r: Event F  -0.2  0.5  -0.2 5 Point  10 Point  20 Point  5 Point  10 Point  20 Point  Figure 3.7: Distributions of Pearson correlation coefficients between each realization and the true sample mean for Soil Moisture (all events).  51  Of the 18 comparisons made between events A-D, 17 showed significant differences between mean values. The one test did not have a significant difference was the comparison between the 5 point samples of events B and C, though it was very close to the 95% confidence threshold (P = 0.0564). Of the 9 comparisons made between events E-G, 4 had significant differences and 5 had no significant difference. Events F and G had no significant difference at any sample size. There was also no significant difference between any of the events at the 5 point sample size. Event E was significantly different than events F and G at both the 10 and 20 point sample size. Full Results for these analyses can be seen in Appendix J, Table J.2.  RMSE datasets for throughfall shown in Figure 3.8 were log-transformed and compared using the student’s t-test. As with soil moisture, comparisons were made between different sample sizes for the same event and equal sample sizes across events. Events C and E were not included in these tests as there was no precipitation recorded during these events.  52  Hourly Throughfall (mm)  RMSE in Event A Analysis  RMSE in Event B Analysis 1.6  1.6  1.2  1.2  1.2  0.8  0.8  0.8  0.4  0.4  0.4  0  0 5 Points  10 Points  20 Points  0 5 Points  RMSE in Event F Analysis  Daily Throughfall (mm)  RMSE in Event D Analysis  1.6  10 Points  20 Points  5 Points  10 Points  20 Points  RMSE in Event G Analysis  25  25  20  20  15  15  10  10  5  5  0  0 5 Points  10 Points  20 Points  5 Points  10 Points  20 Points  Figure 3.8: Distribution of RMSE of each realization to the true sample mean for throughfall (all events).  Increasing the sample size always resulted in a significantly decreased mean RMSE for datasets of events A, B, D, F and G. Of the 9 t-tests performed to compare between events A, B and D, 6 showed significant differences. There were no significant differences between events A and B at any sample size. Event D was significantly different from events A and B at every sample size. Comparisons between events F and G showed no significant differences in RMSE dataset means for any sample size. Full results for the throughfall comparison tests are shown in Appendix J, Table J.3.  The distributions of Pearson r results for throughfall are shown in Figure 3.9. Visually, these box plots confirm the high amount of correlation by the random realizations shown in the throughfall plots in Figures 1, 3 and 4. More than 75% of realizations have a Pearson r value of 0.85 or higher. All Wilcoxon rank-sum tests for the effects of changing sample sizes 53  showed significant differences as sample size was increased (Events A, B, D, F and G). Of the twelve tests performed to compare between events, all but one showed significant differences between events. This was the comparison between the 20 point sample size for events F and G (P = 0.2450). Full results are shown in Appendix J, Table J.4.  Pearson r  Pearson r: Event A  Pearson r: Event B  1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 -0.1 -0.2 -0.3 5 Point  10 Point  20 Point  1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 -0.1 -0.2 -0.3 5 Point  Pearson r: Event F  Pearson r  Pearson r: Event D  1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 -0.1 -0.2 -0.3 10 Point  20 Point  5 Point  10 Point  20 Point  Pearson r: Event G  1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 -0.1 -0.2 -0.3  1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 -0.1 -0.2 -0.3 5 Point  10 Point  20 Point  5 Point  10 Point  20 Point  Figure 3.9: Distributions of Pearson correlation coefficients between each realization and the true sample mean for Throughfall (all events).  54  3.4  Discussion  3.4.1  Soil Moisture  T-tests and Wilcoxon rank-sum tests indicated that there was a significant decrease in RMSE and a significant increase in Pearson r for soil moisture when sample size was increased for almost all events. In the case or RMSE, the one exception to this was the change from 5 points to 10 points for event D. However, Figure 3.4 shows that the change from 5 points to 10 points does negate one outlier random realization. Removal of the possibility of this outlier does show a benefit of increasing the sample size in the case of event D, and there was a significant decrease in RMSE when the same event was increased from 10 to 20 points, so a point of diminishing returns was not yet reached. The exception for Pearson r test results was the change from 5 points to 10 points in event E. A possible explanation for this result is that there was very little activity to give strength to the correlation coefficients during this time; the soil moisture value was essentially a straight line. Event E was the full month of August 2006, when no throughfall was recorded in the catchment experiment and there was a minimum amount of hydrologic activity, the mean soil moisture during this time was static, minimizing the meaning of this result. Also, (as with the RMSE exception) since there was a significant increase in Pearson correlation when sample size was increased from 10 to 20 points, this exception does not indicate reaching a point of diminishing returns.  Even though there are significant changes from 5 points to 10 points and 10 points to 20 points for almost all events, there is a more prominent visual decrease in noise when increasing from 5 measurement points to 10 than from 10 to 20 (Figures 1-4). This indicates in this particular catchment, that there is less benefit when sample size is increased beyond  55  10 points in the case of soil moisture. The test results for correlation and RMSE indicate that there is a significant though smaller benefit.  Comparing RMSE between storm events indicates a shift in the factors controlling soil moisture. This switch seems to be somewhat different than the switch between small and large scale controls described in the Tarrawarra project (Western et al. 1999). Our results indicate that this shift may occur much more often in this study catchment. At the 10 and 20 point sample size, there was no significant difference in RMSE for soil moisture, and hence no difference in our ability to accurately predict catchment average soil moisture for events A, C, and D. Since these 3 events all experienced less rainfall and had significantly different RMSE than event B, this is an indication that event B was a large enough storm to shift into a wet state and the others were not. It is interesting that the size of the storm had more of an influence than the background general wetness of the catchment in determining how well were able to predict the catchment average soil moisture. The exception to this finding is the result that there was no significant difference between Events B and C at the 5 point sample size.  Evidence provided by the Wilcoxon tests for comparisons of soil moisture across events A-D shows that the correlation of measurement points was probably much more specific to the antecedent conditions in the catchment during each event and the spatial nature of water input throughout each event. The only test that did not have significant differences between the two Pearson r distributions was the 5 point sample size of events B and C, and it was very close to our 95% threshold (P = 0.0564).  56  When comparing soil moisture between events E, F, and G (full months), there is evidence of a shift between dry and wet states in the distribution as RMSE for events F and G were both significantly different from event E (dry event) and equal to each other at the 10 and 20 point sample sizes. All events were significantly different from each other at the 5 point sample size. These results are very similar for the tests of event E-G for the differences in Pearson r distributions as events F and G were equal at all sample sizes and significantly different than E at the 10 and 20 point sample size. The major discrepancy was at the 5 point sample size. Though the strength was somewhat low (E5 vs. F5, p = 0.1331 and E5 vs. G5, p = 0.0915) all events had no significant differences at the 5 point sample size. These differences in results provide more evidence that the 5 point sample size may be too small to obtain reliable and reproducible results for soil moisture measurements. It is likely that the optimal measurement density for soil moisture in this catchment is at least 1.4 points/ha (10 points), because we were able to obtain more predictable results for comparing between events with the 10 and 20 point sample size. The fact that the results for the Pearson r distribution on cross event comparisons of events E-G largely agree with the results for the RMSE tests whereas the results for A-D do not, can be explained by the switch from hourly measurements in events A-D to daily measurements in events F-G. The larger temporal scale of events F-G may act to dampen the effects of antecedent conditions and spatial variations of water input that contributed to the results for events A-D.  In the case of soil moisture, nearly every time that an individual realization had a large difference from the best known catchment mean it was a case of values higher than the mean.  57  This could be an indication that soil moisture in most of the catchment stays near a baseline, or “rest” level during dryer periods. When the catchment became wetter, a few points increased greatly from their typical resting value while the rest of the measurement points had a much smaller change from their resting value. When these points that increased were part of a random selection, especially at the 5 point sample size, they had a large effect on the average value of that realization. A shift in the controlling factors of the points with large responses from local controls such as soil properties to larger, non-local controls such as topography (Grayson et al. 1997) could be an explanation for their response.  3.4.2  Throughfall  For throughfall measurements, there was always a significant decrease in RMSE and increase in correlation as sample size increased from 5 to 10 to 20 points. This is evidence that if there is a point of diminishing returns on increasing the sample size we have not achieved it in this experiment. It may exist even at densities higher than 41 points in 7 ha. Although we have shown that more data is always better in the case of throughfall measurements in this catchment, we also show that in every event, greater than 75% of randomly chosen sample subsets have a correlation of 0.85 or higher and a RMSE of less than 0.8 mm/hr (for events A, B and D) or less than 8 mm/day (event F and G).  When RMSE was compared across events, events A and B were shown to have no significant differences at any sample size. This is not very surprising, as these were both large rainstorms (B was the largest rainstorm), with the main difference that event A occurred during a generally very dry period in the catchment. Unlike soil moisture, this wouldn’t be  58  expected to have a large effect on the spatial variation of throughfall provided there wasn’t already water being stored in the canopy. Even though Event D was closer in size to Event A than Event A was to B, Event D was significantly different from A and B. One possible explanation for this is that events A and B were both large enough to fully exceed the canopy interception storage (with A being just large enough) whereas event D was not (Viessman and Lewis 2003). Comparisons between the full months with significant rainfall (Events F and G) showed that they did not have significantly different RMSE at any sample size. These were the two months with the largest precipitation recorded during the sampling period, so it is not surprising that they experienced very similar RMSE values for predicting the best known mean throughfall. Most likely any canopy storage value was exceeded multiple times during each month, and when averaging RMSE for an entire month it is likely that any differences in the spatial distribution of individual storms was dampened when averaged among the multiple storms experienced each month.  The correlation (Pearson r values) results between events F and G for throughfall agree with these results at the RMSE for the 20 point sample size, but not at the 5 or 10 point sample size; we do, however, have the greatest confidence in the correlation at the 20 point sample size, and defer to this test result for support. This is evidence that a higher sample size may be necessary in an experiment where the interest is more in the actual timing of precipitation input rather than just the overall amount of water input throughout an event.  59  3.4.3  Sampling Design  Even in the smallest sample size (5 point) there were only a few individual realizations that did a poor job of staying near the best known catchment value. Since sampling choices for this experiment were completely random, these few individual realizations that were the furthest away could be thought of as the worst-case sampling scenario. It may have been the worst case scenario simply because of bad luck with choices, but more likely is that they were far away from the catchment value due to poor distribution throughout the catchment. Because of the many different theories on choosing optimal sample locations based on geostatistical methods (Olea 1984; Ferreyra et al. 2002; Zio et al. 2004) and the unlikelihood that someone would design a sampling experiment with all sampling locations grouped in one section of a catchment, it is likely that most real world sampling designs could avoid the worst case scenario. With a conscious effort of choosing a proper array of sampling points it should be possible to have a small RMSE at even the 5 point sample size.  3.5  Conclusions  Results of this study indicate that if sampling resources are limited, it is still possible to measure values that are accurate. Our evidence also shows, however, that increasing the sample size always decreases the measurement error and increases the samples correlation with the catchment mean values, so if possible, a larger sample size will still be better. When comparing across events, the significant changes in RMSE and Pearson r are evidence that when certain thresholds (moisture states for soil moisture and interception storage for throughfall) are exceeded, there is a significant decrease in our ability to accurately measure the catchment average value with a constant sample size.  60  3.6  References  Bedient, P. B. and W. C. Huber (2002). Hydrology and Floodplain Analysis. Upper Saddle River, NJ, Prentice Hall. Canada (2006). "National Climate Data and Information Archive." Retrieved October 2, 2007, from http://climate.weatheroffice.ec.gc.ca/climateData/dailydata_e.html. Decagon (2006). "ECH2O Soil Moisture Probe Operator's Manual." Retrieved November 10, 2007, from http://www.decagon.com/manuals/echomanual.pdf. Egan, B. (1999). "The Ecology of the Coastal Western Hemlock Zone." Ecosystems of British Columbia Retrieved February 10, 2008, from http://www.for.gov.bc.ca/hfd/pubs/docs/Bro/bro31.pdf. Ferreyra, R. A., H. P. Apezteguia, R. Sereno and J. W. Jones (2002). "Reduction of soil water spatial sampling density using scaled semivariograms and simulated annealing." Geoderma 110: 265-289. Gay, D., P. Levis, R. v. Behren, M. Welsh, E. Brewer and D. Culler (2003). The nesC Language: An Holistic Approach to Networked Embedded Systems. ACM SIGPLAN 2003 conference on Programming language design and implementation, San Diego, CA, USA. Grayson, R. B., A. W. Western, G. Bloschl and F. H. S. Chiew (1997). "Preferred states in spatial soil moisture patterns: local and nonlocal controls." Water Resources Research 33(12): 2897-2908. Hill, J. L. and D. E. Culler (2002). "Mica: A Wireless Platform for Deeply Embedded Networks." IEEE Micro 22(6): 12-24. Keim, R. F., A. E. Skaugset and M. Weiler (2005). "Temporal persistence of spatial patterns in throughfall." Journal of Hydrology 314: 263-274. Lee, S. (2007). "Crossbow Technology, Inc. Quick Facts." Retrieved January 7, 2008, from http://www.xbow.com/General_info/Info_pdf_files/Crossbow_Quick_Facts.pdf. Levis, P., S. Madden, J. Polastre, R. Szewczyk, K. Whitehouse, A. Woo, D. Gay, J. Hill, M. Welsh, E. Brewer and D. Culler (2005). TinyOS: An Operating System for Sensor Networks. Ambient Intelligence. W. Weber, J. M. Rabaey and E. Aarts. New York, NY, Springer: 115148. McBean, E. A. and F. A. Rovers (1998). Statistical Procedures for Analysis of Environmental Monitoring Data & Risk Assessment. Upper Saddle River, New Jersey, Prentice Hall PTR.  61  Moore, R. D. and J. C. Thompson (1996). "Are water table variations in a shallow forest soil consistent with the TOPMODEL concept?" Water Resources Research 32(3): 663-669. Olea, R. A. (1984). "Sampling Design Optimization for Spatial Functions." Mathematical Geology 16(4): 369-392. Park, S. J. and N. v. d. Giesen (2004). "Soil-landscape delineation to define spatial sampling domains for hillslope hydrology." Journal of Hydrology 295: 28-46. Rainwise (2007). "Wired Rain Gauge." from http://www.rainwise.com/products/detail.php?ID=6697&Category=Rain_Gauges:Wired&pa geNum_cart=/products/index.php. Shah, S. M. S., P. E. O'Connell and J. R. M. Hosking (1996). "Modelling the effects of spatial variability in rainfall on catchment response. 1. Formulation and calibration of a stochastic rainfall field model." Journal of Hydrology 175: 67-88. Srinivasan, M. S., M. A. Whittman, J. M. Hamlett and W. J. Gburek (2000). "Surface and subsurface sensors to record variable runoff generation areas." Transactions of the American Society of Agricultural Engineers 43(3): 651-660. Stern, D. (2003). IDL Student Edition. Boulder, CO, Research Systems, Inc./ ITT Visual Information Solutions. Talzi, I., A. Hasler, S. Gruber and C. Tschudin (2007). "PermaSense: Investigating Permafrost with a WSN in the Swiss Alps." In Press. Viessman, W. and G. L. Lewis (2003). Introduction to Hydrology. Upper Saddle River, NJ, Prentice Hall. Western, A. W. and G. Bloschl (1999). "On the spatial scale of soil moisture." Journal of Hydrology 217: 203-224. Western, A. W., R. B. Grayson and T. R. Green (1999). "The Tarrawarra project: high resolution spatial measurement, modelling and analysis of soil moisture and hydrologic responses." Hydrological Processes 13: 633-652. Zio, S. D., L. Fontanella and L. Ippoliti (2004). "Optimal spatial sampling schemes for environmental surveys." Environmental and Ecological Statistics 11: 397-414.  62  Chapter 4: The dynamics of spatial variability in soil moisture and throughfall1  4.1  Introduction  Throughfall and soil moisture are two hydrologic processes that have been shown to have a high level of variability in space and time. Understanding the spatial variability of these parameters and how this variability changes with changing environmental conditions is critical to our understanding of the effects that the physical characteristics of a hydrologic catchment have on the water cycle. Understanding the spatial reorganization of rainfall and subsequent water loss due to interception is an important factor in the creation of water budgets for hydrologic catchments (Whelan and Anderson 1996), and is an important link between rainfall and stream flow response in a catchment (Shah et al. 1996; Singh 1997). Along with throughfall, knowledge of the spatial variability of soil moisture is necessary to further understand the role it plays in the exchange of water between the atmosphere and ground at many different scales (Western and Bloschl 1999). It has been stated that “the relationship between soil moisture and land use and topography needs to be studied in a variety of places and over a large range of scales” (Qiu et al. 2001). This is due to the importance of soil moisture in the prediction of many different hydrological process, and its typically high degree of variance, both in space and in time (Grayson et al. 1997). This experiment will seek to evaluate the spatial and temporal (stochastic) relationships of soil moisture and throughfall measurements in a variety of ways.  1  A version of this chapter will be submitted for publication.  J.Trubilowicz and M. Weiler. The dynamics of spatial variability in soil moisture and throughfall.  63  A basic representation of stochastic variability of measurements in a catchment is through the frequency distribution. It has been employed for the description of snapshot measurements of soil moisture that show different organizations of soil moisture at different locations and times (Robertson et al. 1993; Pellenq et al. 2003) and averaged over long periods of time (Huang et al. 1996; D'Odorico and Porporato 2004). Viewing how frequency distributions change over time is a simple way to demonstrate how a catchment is responding to changing conditions. A drawback to simply using frequency distributions to represent variability of measurements is that the do not differentiate between measurement points; they only represent how often a certain value was measured.  A method to demonstrate variability in a catchment that does incorporate individual measurement points is the calculation of normalized values of a measurement for a certain study period. This calculation shows which measurement locations are consistently above or below the catchment mean, and how they are distributed about the mean. This type of calculation has been performed previously in experiments that investigated the spatial persistence of throughfall and soil moisture among different storms (Raat et al. 2002) and throughfall in different forest stands (Keim et al. 2005). The drawback of these studies was that they only included storm total precipitation (or a single soil moisture measurement) for each analyzed event. Because of this it could not show how variable each normalized value was in time within an event. Similarly, scaled water content at multiple measurement sites measured at multiple times were plotted for the Tarrawarra soil moisture data (Western et al. 1999) giving a visualization of how consistently different measurement points related to each other. This analysis also had the drawback of a low temporal resolution as soil moisture was  64  only measured at each point once every few weeks. With a similar experiment for soil moisture and throughfall performed at a higher temporal resolution, it would be possible to show the standard deviation at each measurement location on top of a plot of mean normalized parameter value.  The next step beyond showing how measurement locations typically relate to one another and how they relate to the catchment mean is to incorporate their location in relation to each other. A variogram can estimate the separation distance at which measurement variables no longer have a correlation based on distance, and describe the direction in which the best correlation exists. Variograms have been used to describe the spatial variability of soil moisture (Western and Bloschl 1999; Western et al. 2001) and throughfall (Keim et al. 2005). A limit to these previous variogram analyses has been that they described the spatial continuity of a hydrologic parameter at one moment in time. In the case of soil moisture measurements in the Tarrawarra project (Western et al. 1999; Western et al. 1999; Western et al. 2001), the changing patterns of spatial organization have been shown in an experiment featuring very high spatial resolution (>1500 data points), but low temporal resolution (>2 weeks per measurement). They demonstrated a shift between two different organizational states with different controlling factors, where a wet state is controlled by larger scale parameters such as topography, and a dry state is controlled by small scale parameters such as specific soil properties. Investigation is needed to see if and how these patterns persist throughout wetting and drying periods using shorter time intervals, necessitating an experiment with higher temporal resolution. Experiments describing the spatial continuity of throughfall also had this drawback, where measurement resolution only gave the storm total  65  precipitation, and could not describe how consistent this spatial organization was throughout the entire storm event (Keim et al. 2005), or during longer term analysis periods. Higher temporal resolution is necessary to compare individual rain events to long term wet periods to determine if spatial organization of throughfall was indeed dependent mostly on the canopy structure (Whelan and Anderson 1996) and hence did not change much whether it was analyzed over one storm event or a longer period of multiple storms.  Previous spatial descriptions of both soil moisture and throughfall have demonstrated a need for data measurement at a higher temporal resolution. Through the use of the emerging technology of low-cost, low-power wireless sensor modules, known as motes (Hill and Culler 2002), we were able to operate a hydrologic experiment that measured soil moisture and throughfall at high spatial (~15 m) and temporal (15 min) resolution.  Our objective for this analysis was to investigate how the spatial organization of soil moisture and throughfall changes within storm events and over longer time periods. We also wanted to determine if there were individual measurement points that were representative of the entire catchment for both soil moisture and throughfall. We investigated if these representative points were consistent over different catchment moisture conditions, and how the spatial and frequency distribution of measurements changed over time.  66  4.2  Methods:  4.2.1  Experiment Description:  Field data collection was carried out in a 7 ha rain dominated forested catchment in Malcolm Knapp Research Forest, a research forest operated by the University of British Columbia in Maple Ridge, BC, Canada. The base of the catchment is located at 49.26º N and 122.55º W. The ground surface of the catchment is a thin layer of glacial till (0-100 cm deep) with protruding granitic outcroppings. Much of the forest floor is covered with a layer (10 or more centimeters thick) of degrading organic matter as well as a large amount of degrading fallen trees and branches. Trees are predominately western red cedar (Thuja plicata), Douglas fir (Tseudotsuga menziesii) and western hemlock (Tsuga heterophylla) (Thompson 1994); stand age is approximately 70 years. The catchment is located in the Coastal Western Hemlock biogeoclimatic zone (Egan 1999), has an average temperature of 9.6ºC and receives an average of 2200 mm of precipitation per year. This catchment receives the majority of precipitation in the form of rain, with the heaviest precipitation in November, December and January (Canada 2006). An intermittent headwater stream in the catchment functions as a tributary to the North Allouette River and flows roughly from October to June. A forest service road bisects the catchment and is shown on Figure 4.1. The digital elevation model (DEM) of the catchment shown in Figure 4.1 was derived in a previous field experiment in the same catchment (Moore and Thompson 1996).  The study catchment featured 41 measurement stations using a novel method of data collection and storage. Each measurement station included a powered data box with a Crossbow Technologies, Inc. MDA300 data acquisition board, and MICA2 processor board,  67  known as a mote (Hill and Culler 2002; Lee 2007). The motes ran the open source operating system TinyOS (Levis et al. 2005), which is configured using the NesC programming language (Gay et al. 2003). The functions of TinyOS can be tailored to the needs of a specific application; in this case it operated all sensors attached to the MDA300, collected and stored data for seven external variables every 15 minutes, and allowed for wireless downloading of data from each measurement station.  Measurement locations were chosen with the desire to have a fairly uniform spatial coverage; however, we could not use a random number generation or uniform grid because there were many locations that would be impossible to install a measurement location (due to exposed bedrock, standing trees or fallen trees. As a result, measurement locations were chosen while walking through the catchment with the desire to cover the different range of topographic features present in the catchment. They covered different slope angles, soil depths, and distances from trees. The (+) symbols in Figure 4.1 show the location of each measurement location in the study catchment. Instrument cluster locations were determined using a Trimble Pathfinder™ GPS with a stated accuracy of 20 cm. There is a lack of measurement points in the upper catchment because we initially spaced out locations to fit 50 measurement stations, but ended up with only 41.  Measurements used in this experiment were soil moisture and throughfall. Both of these parameters were recorded every 15 minutes, from July 10, 2006 to April 22, 2007. Throughfall was collected using Rainwise, Inc. Rainew tipping buckets which feature a resolution of .25 mm (Rainwise 2007). Soil moisture was measured using Decagon Devices  68  ECH2O dielectric aquameters. The ECH2O probes are 20cm long and were inserted so that the top was 10cm into the inorganic layer between 20 and 50cm from the measurement station. The organic layer of soil was removed for insertion and then replaced afterwards. Factory calibrations of the soil moisture probes give an advertised accuracy of ± 4% (Decagon 2006). Lab testing showed that the precision was ± 0.5%.  Other variables measured at each station included: air temperature, humidity, soil temperature, and groundwater head. Sixteen of the 41 locations also included a sensor to measure overland flow. Other instrumentation in this catchment included 3 V-notch weirs with water levels logged every 10 minutes. Two of these weirs were used to measure the flow of two culverts beneath a road that is intersecting the study catchment. The third weir was located at the base of the catchment in order to measure outflow from the catchment. These weirs are shown with the (▼) symbol in Figure 4.1.  69  5456750  4 19 5 6  17 18 2  5456700  UTM U10 Northing  7 9  29 5456650  8  10  16 11  3  15  12  21  13  20 14  35  34  26 32 33 31 2240 30 24 38  5456600  Gauged Base Weir Gauged Culvert  25 37 28  5456550  41  Instrument  23  Road  27  36 39 43 5456500  *5 meter contour intervals 532450  532500  532550  532600  532650  532700  532750  UTM U10 Easting  Figure 4.1. Data measurement locations on the study area DEM.  4.2.2  Data analysis  We selected four events for analysis from the full dataset collected between July 2006 and April 2007. Two of these were short periods lasting a few days and two were full months. The short-term analysis periods are of a large individual storm event and a short-term dry period. The long-term analysis periods (full months) represent the driest month during the sample period (August 2006) and the wettest month during the sample period (November 2006). In the interest of streamlining computation and simplifying plots for the results, we resampled the 15-minute original dataset into hourly data for the two shorter events, and  70  daily data for the two longer events using a custom IDL program (Stern 2003). The program used for combining raw data into events is shown in Appendix D. Summaries of the four events are shown in Table 4.1 below. We analyzed throughfall and soil moisture data for events S1 and L2. Only soil moisture data was analyzed for events S2 and L1 as there was negligible precipitation and no recorded throughfall during these periods.  Table 4.1. Event Summary. ID  Start Day  End Day  dt  Total Precip. mm  Avg. Temp. Deg. C  S1 S2 L1 L2  21-Mar-07 25-Mar-07 1-Aug-06 1-Nov-06  24-Mar-07 29-Mar-07 31-Aug-06 30-Nov-06  Hourly Hourly Daily Daily  169.0 0.0 17.0 438.2  12.23 10.70 17.72 4.13  Description Large storm in wet season Dry period in wet season Dry month Wettest month  After selecting the events for analysis, the next step in data processing was removal of data measurement errors. As we were testing the emerging technology of wireless sensors for data collection and trying to resolve its various operational issues, there was a substantial amount of data filtering necessary to remove any erroneous measurements. Some measurement points had random intermittent failures, and others were rendered completely inoperable at different times; all of these errors had to be detected and removed. Once the data errors were filtered out, we calculated the catchment mean values of soil moisture and throughfall over time for each event. The catchment mean was used in all of the other analyses performed.  The first representation of changing spatial patterns was an analysis of the changing frequency distribution of soil moisture throughout the event. We created grayscale shaded  71  raster images using the data analysis software IDL (Stern 2003) with time as the x dimension and soil moisture percentage as the y dimension. Each pixel in the image had the height of 1% soil moisture and the width of 1 time interval (hour for S1 and S2 and day for L1 and L2). The shade of each pixel represented the fraction of the data points that recorded a value within each respective bin size (inclusive on the lower end)at each time step. The IDL program used is shown in Appendix K. The catchment mean soil moisture value was then plotted on top of the image files to represent the general response of the whole catchment. We did not create these graphics for throughfall as it was represented by a one tailed distribution (going up from 0) that was not well represented by this type of graphic.  The second analysis was an investigation of persistent normalized values of soil moisture and throughfall for each measurement point. Each measurement point was normalized to the catchment mean using the following equation:  (x − x ) ~ xi = i Sx  (1)  In this equation ~ xi is the normalized measurement value, xi is the individual measurement value, x is the catchment mean and S x is the standard deviation for the entire catchment (Keim et al. 2005). We included the catchment standard deviation in the calculation as it minimized the effects of extreme values on the normalized measurement value. The IDL program used for this analysis is shown in Appendix L.  72  The mean normalized values for each measurement location throughout an event were then ranked and plotted along with error bars representing their standard deviation, allowing us to show their stability over time (for soil moisture and throughfall) throughout multiple events.  The mean normalized data values for each measurement location were grouped into 5 different bins for spatial representation on the catchment DEM. We wanted to visually analyze and determine any possible spatial organization of either soil moisture or throughfall and how this organization changed between wet and dry moisture states. The IDL program used for this analysis is shown in Appendix M.  In addition to using the persistent values of soil moisture and throughfall for each event to relate each individual point to the catchment mean, we also calculated the Pearson product moment correlation coefficient (Pearson r) between each measurement point and the catchment mean (soil moisture and throughfall) over time for each event (McBean and Rovers 1998). The IDL program used for this analysis is shown in Appendix N. This was a different way of understanding how well an individual measurement point predicted what was happening in the entire catchment. A point that had a mean normalized value far different from the mean might still be very useful as an indicator for the whole catchment if it is correlated well (r close to 1) with the catchment mean over time. Pearson r values for each point were separated into 5 bins for graphical representation of each event on the DEM for the same purpose as the persistent normalized values. This also allowed for comparison between Pearson r values and persistent normalized values for the same event.  73  In order to better describe how the spatial organization in the study catchment changed between events and evaluate how similar the spatial patterns related between persistent values and Pearson r values, we created variograms for each spatial plot. Variograms are a commonly used geostatistical method of describing spatial continuity and how it changes with distance. They show the average semivariance for groups of paired measurements over a range of distances. These distance separations are known as lags. They are defined by the equation:  γ ( h) =  1 (vi − v j )2 ∑ 2 N (h) (i , j ) hij ≈ h  (2)  Where γ (h) is the semivariance for measurements h distance apart, N (h) is the number of pairs in the lag group, h is the lag distance, vi and v j are the measurement values at points i and j, and (i, j ) | hij ≈ h represents the tolerance of the distance h for pairs of data points to be  included in a lag group (Isaaks and Srivastava 1989). Variograms can describe spatial continuity averaged over all directions (omnidirectional) or in specific directions. Critical information obtained from a variogram includes the range; which is the distance at which measurements are correlated with distance; the sill, which is the semivariance at the range; and the nugget effect, which is the effect of short scale discontinuity that is not dependent on the separation between measurement pairs (Isaaks and Srivastava 1989). While there are no set regulations for the number of lags used, or the spacing between lags, a common starting point is the average distance between nearest neighboring measurement points and the number of lags resulting in the furthest lag being equal to the largest distance between pairs of measurements. In this experiment, we used omnidirectional (not directionally dependent) variograms with 9 lags spaced in 10 m increments.  74  4.3  Results  4.3.1  Soil Moisture  Distribution of soil moisture in event S1 (Figure 4.2, left) appears more organized in values below the catchment mean, with 2 distinct modes visible throughout most of the event. The distribution is more erratic in values above the catchment mean, only becoming organized at approximately the 70th hour of the event. The distinct change in organization shown at the 12 hour of the event was due to a mote battery change, where many more sensors became active.  S1  S2  Figure 4.2. Distribution of hourly soil moisture measurement values through event S1 (left) and S2 (right). Events are separated by the vertical blue line. Catchment mean is shown in red, hourly throughfall is shown in green. A completely white pixel represents 0% of measurements and a completely black pixel represents 30% of measurements.  Event S2 was a drying period in a wet catchment as it occurred directly after event S1. Figure 4.2 (right) shows that the distribution of soil moisture values became more organized throughout the event, as the unsaturated soil zone dried out. There are distinct modes shown in Figure 4.2 that show the soil moisture distribution trending more toward the catchment mean and becoming more organized over time. The highest moisture values do not appear to  75  change throughout the event. Soil moisture values below the catchment mean appear mainly static and evenly distributed throughout the event.  The extremely dry month L1 (Figure 4.3 left) showed a much more even distribution of soil moisture values than the rainy event L2 (Figure 4.3 right) just as S2 was more evenly distributed than S1. Event L2 was organized in a few distinct bands between 42 and 45%, but also had a separate much higher grouping around 62% and some scattered higher values later in the month. Event L2 showed very erratic soil moisture distribution through time, with no distinctive bands.  L1  L2  Figure 4.3. Distribution of daily soil moisture measurement values through events L1 (left) and L2 (right). Catchment means are shown in red, daily throughfall is shown in green. A completely white pixel represents 0% of measurements and a completely black pixel represents 30% of measurements.  Figure 4.4 shows the normalized soil moisture values for each event. The overall shape of all of the plots was similar, with an essentially linear increase in persistent value as the rankings moved up. There was a slight upward curve at the upper extreme and downward curve at the lower extreme. Other notable observations from Figure 4.4 are that motes 11 and 32 76  appeared consistently closest to the catchment value throughout the events, though mote 32 had consistently smaller standard deviations than mote 11. During the dry events, S2 and L1 motes 37 and 10 had consistently much higher soil moisture values than the other measurement locations; however, during the wet events they stayed in the upper rankings but did not stand out from the other higher end values.  Soil Moisture Persistence for Event S1  Soil Moisture Persistence for Event S2  4  4 10 37 30 18  0  -2  19 16 31 41 5 32 17 21 3 22 35 40 4 38 36 9 34 11 13 29 8 14 25 23 33 6 28  2 3710  2  41  2  S.D.  S.D.  2  0  -2  -4  4 9 28 33 6 34 23  18 3 31 36 30 21 32 13 14 35 38 8 19 16 11 17 29 25 22 40  -4  Soil Moisture Persistence for Event L1  Soil Moisture Persistence for Event L2  4  4 7  10 37 2  21  2  0 15 6 -2  -4  4 12 5  20 16 14 25 11 27 3 29 33 22 28 18 9 31 13 21 32 35 19 17 23 8  S.D.  S.D.  36 30  0  -2  23 33 9  12  14 35 43 25 29  3 31 16 36  32 40 20 8 4 22 13 6  19  18 10 11 17  41  37  30 27  -4  Figure 4.4. Mean normalized soil moisture values for each event. Error bars represent the standard deviation at each station throughout the event.  Figure 4.5 shows that a noticeable change in the spatial organization of persistent soil moisture values does not exist between wet and dry periods in the catchment. When comparing between events S1 and S2, the only noticeable difference is slightly more  77  measurements above the mean value in the upper catchment. Comparing events L1 and L2 shows no noticeable differences in the distribution of soil moisture values. In all events, the most consistent location of over predicting values was near the base of the catchment. Event S1 Soil Moisture Persistence  Event S2 Soil Moisture Persistence  Event L1 Soil Moisture Persistence  Event L2 Soil Moisture Persistence  Standard Deviation 1.5 to 4 0.5 to 1.5 -0.5 to 0.5 -1.5 to -0.5 -4 to -1.5  Figure 4.5. Mean normalized soil moisture values shown at their measurement location on the catchment DEM.  Figure 4.6 shows that individual soil moisture measurement locations had a higher correlation to the catchment mean during the shorter events (S1 and S2) than during the long  78  term events (L1 and L2). The only noticeable difference between events S1 and S2 were a few very poorly correlated measurement points in event S1. Comparison between events L1 and L2 show a wider range of correlation for the much wetter event L2. In event L1, most points have a Pearson r ranging from 0.4 to 0.8, whereas in L2 there is a range from almost no correlation, to correlation of greater than 0.8. There appears to be a ridge of highly correlated points down the center of the catchment in event L2. These points fall very near the intermittent headwater stream in the catchment.  79  Event S1. Soil Moisture Pearson r  Event S2. Soil Moisture Pearson r  Event L1. Soil Moisture Pearson r  Event L2. Soil Moisture Pearson r  Pearson r 0.8 to 1 0.6 to 0.8 0.4 to 0.6 0.2 to 0.4 0.01 to 0.2  Figure 4.6. Pearson r values for soil moisture shown at their measurement location on the catchment DEM.  4.3.2  Throughfall  Ranked plots of normalized throughfall shown in Figure 4.7 (particularly event S1) show a somewhat different shape than the normalized soil moisture plots. In event S1 there is evidence of a bimodal distribution split between underpredicting measurement stations and  80  over predicting measurement stations. Event L2 shows a more linear increase in mean normalized throughfall, similar to the increases seen for soil moisture in Figure 4.4. The distribution of mean normalized throughfall values was closer to the catchment mean than with soil moisture for both events, with most mean values within one standard deviation, but the standard deviation of each individual point was in general much larger than for soil moisture. Mote 4 was consistently the closest to the catchment mean value for both events, but it had a wide standard deviation indicating that it may not be as reliable of a predictor of total catchment throughfall as motes 32 or 11 were for soil moisture.  Throughfall Persistence for Event S1  Throughfall Persistence for Event L2  3  3  2  2  0  12 19 40 20  33  21  9  -1 -2  18 28 43 17 8  29  5  4  23 22 31 16 30 35 14 32 37 2  1  S.D.  S.D.  1  41 38 3 11 34 13 25 36 10  0  29 25  -1  21  6  -3  -2  6 43 40  3  7  22 15 30 4 14 13 41 24 16  37 10 36 23 9 38 35 31 33 12 5 32  11  -3  Figure 4.7. Mean normalized throughfall values for each event. Events S1 and L2 were the only analyzed events with measured throughfall. Error bars represent the standard deviation throughout the event.  In event S1, shown in Figure 4.8, there appears to be a slight trend of lower throughfall in the upper catchment and higher throughfall in the lower catchment. The close proximity of high and low values, particularly in the southwest corner of the catchment, however, indicates that smaller scale variability in the canopy structure is probably a more important control. Event  81  L2 was an average normalized throughfall over multiple storms, as opposed to the single storm for event S1. Results in Figure 4.8 show that the consistent under and over predicting measurement locations are evenly distributed throughout the catchment, and that there are more values with a normalized mean near the catchment mean. This is evidence that the slight inverse relationship to elevation seen in S1 was most likely due to the particular spatial organization of precipitation during event S1.  Event S1 Througfall Persistence  Event L2 Througfall Persistence  Standard Deviation 1 to 2 0.25 to 1 -0.25 to 0.25 -1 to -0.25 -2 to -1  Figure 4.8. Mean normalized throughfall values shown at their measurement location on the catchment DEM.  Individual measurement points for throughfall in events S1 and L2 (Figure 4.9) had a generally very high level of correlation with the catchment mean. There were only a few measurement locations that had low Pearson r values. While both events showed a high level  82  of correlation, the shorter event S1 did not have the few very poorly correlated points that event L2 did.  Event S1. Throughfall Pearson r  Event L2. Throughfall Pearson r  Pearson r 0.8 to 1 0.6 to 0.8 0.4 to 0.6 0.2 to 0.4 0.01 to 0.2  Figure 4.9. Pearson r values for throughfall shown at their measurement location on the catchment DEM.  4.3.3 Variogram Analysis Results for the variogram analysis, shown in Figure 4.10, show little spatial correlation for either the persistence values or the Pearson r values. In the case of persistent soil moisture, only events L1 and L2 show any evidence of a correlation range. A spherical model was fitted with a least squares fit to both of these events for reference, but neither was a very good fit. The model fitted (with the least squares method) to event L1 had a range of 21 m and the model fitted to L2 had a range of 35 m. Events S1 and S2 only showed an apparent nugget effect, as spatial variation was not evident on the scales described by a variogram. Variogram results for the spatially distributed Pearson r plots had very little spatial continuity as well. Only event S2 had evidence (albeit poor) of a correlation range; a spherical model is  83  shown on the experimental variogram for event S2 with a range of 30 m. All other events showed evidence purely of the nugget effect.  1.4  1.2  1  1.2  1  0.8 0.6  Variogram  1.2  0.4  1 0.8 0.6  10 20 30 40 50 60 70 80 90  0 0  10 20 30 40 50 60 70 80 90  Lag Distance  0.022  0.007  0.018  0.006  0.016  0.015  0.004 0.003  0.01  0.002  0.005  0.001  0 0  10 20 30 40 50 60 70 80 90 Lag Distance  0 0  0.4  0.1 0 0  Event L1: Soil Moisture Pearson r  Event L2: Soil Moisture Pearson r 0.12 0.1  0.014 0.012 0.01 0.008  0.06  0.02  0.002 0 0  0.08  0.04  0.004  10 20 30 40 50 60 70 80 90  10 20 30 40 50 60 70 80 90 Lag Distance  0.006  Lag Distance  0.5  0.2  0.02  Variogram  Variogram  0.02  0.6  Lag Distance  Event S2: Soil Moisture Pearson r  0.005  0.7  0.3  0 0 10 20 30 40 50 60 70 80 90  0.04  0.025  0.8  0.6  0.008  0.03  0.9  0.8  0.045  0.035  Event L2: Soil Moisture Persistence  1  Lag Distance  Event S1: Soil Moisture Pearson r  1.1  0.2  0.2  0 0  Event L1: Soil Moisture Persistence  0.4  0.4  0.2  Variogram  1.4  Variogram  Event S2: Soil Moisture Persistence 1.6  Variogram  Event L1: Soil Moisture Persistence  Variogram  Variogram  1.4  10 20 30 40 50 60 70 80 90 Lag Distance  0 0  10 20 30 40 50 60 70 80 90 Lag Distance  Figure 4.10. Variograms shown for soil moisture in all events. The upper row is results for the normalized soil moisture values and the lower row if the Pearson r values.  As with soil moisture, there was very little evidence of spatial continuity at these measurement scales for either throughfall persistence (Figure 4.11), or throughfall correlation to the catchment mean. A spherical model was fitted to the persistent throughfall analysis of event L2 as it was the only variogram displaying evidence of a correlation range. Correlation range of the model fitted by a least squares analysis was 14 m. All other variograms showed apperent nugget effects.  84  Event S1: Throughfall Persistence  1.4  0.6  1.2 1 Variogram  Variogram  0.5 0.4 0.3  0.8 0.6  0.2  0.4  0.1  0.2  0 0  Event L2: Throughfall Persistence  0 0  10 20 30 40 50 60 70 80 90 Lag Distance  Lag Distance  Event S1: Throughfall Pearson r  Event L2: Througfall Pearson r  0.14  0.1  Variogram  Variogram  0.12  0.08 0.06 0.04 0.02 0 0  10 20 30 40 50 60 70 80 90 Lag Distance  10 20 30 40 50 60 70 80 90  0.022 0.02 0.018 0.016 0.014 0.012 0.01 0.008 0.006 0.004 0.002 0 0  10 20 30 40 50 60 70 80 90 Lag Distance  Figure 4.11. Variograms shown for throughfall in all events. The upper row is results for the normalized soil moisture values and the lower row if the Pearson r values.  4.4  Discussion  4.4.1  Frequency distribution  For both the long and short events, there is a distinct difference between the wet (S1 and L2) and dry (S2 and L1) periods. There is a larger standard deviation and a more skewed distribution of measured values for the wetter events. When comparing events S1 and S2 there is a distinct baseline of lower soil moisture values that exist both during the storm event (S1) and after the storm event has passed (S2). The major difference is in the higher soil moisture values, which are much more unevenly distributed during event S1 and trend back towards the mean during event S2. The long term dry event, L1 is similar to the event S2 in that there is a uniform distribution of soil moisture values. Event L2, however, appears different than S1 because it does not display a steady baseline of normal values. There are  85  large changes in the entire distribution between time intervals. As events S1 and S2 were recorded during a very wet period in the catchment, similar to event L2, (March 2007 was second only to November 2006 in total precipitation on the catchment (Canada 2006) it is most likely that the lack of a smooth baseline throughout event L2 is due to the interval of measurement for that particular event (daily). Event L2 shows that the lowest recorded values are reacting to the water input, but event S1 shows that they react much slower than the upper extreme measurements. These results agree in part with a soil moisture frequency distribution analysis on the Nerrigundah data set (Walker et al. 2001) used for the validation of a model “designed for low resolution remote sensing data assimilation into hydrological modeling” (Pellenq et al. 2003). Pellenq, et al (2001) created frequency distributions for 12 different days of measurement. For 11 of 12 of those days there was a normal distribution of observed soil moisture data, but the final day, which was the only analyzed day with a large storm event, had a much more erratic distribution. Our results for events S1 and L2 agree with this, and S2 adds the ability to see how the frequency distribution returns to a more normal distribution.  4.4.2  Persistent values  Results in Figures 4.4 and 4.5 show no evidence for changing states of soil moisture recorded in this catchment. The spatial organization of persistent soil moisture values appears very similar across all events. If we were to see evidence of changing moisture states, we would expect it to be in the comparison of events L1 and L2, as Grayson, et al (1997) found that the changing patterns were a result of long term dry or wet periods as opposed to short term storms. There are two possible explanations for why we do not see switching between  86  moisture states such as those reported in previous catchment studies (Grayson et al. 1997; Western et al. 1999; Ridolfi et al. 2003; D'Odorico and Porporato 2004) when comparing the spatial distribution of persistent values in events L1 and L2.  The first explanation is that this study catchment does not experience the same switching between moisture states that the Tarrawarra catchment did. In our experiment, the general difference in soil moisture content between the wet season (event L2) and dry season (event L1) was only about 10-12%, whereas the difference between wet and dry seasons in the Tarrawarra project was greater than 20% (Western et al. 1999). There just may not be enough variation between seasons in our catchment to see a difference in states. This conclusion is backed up by the results of the variograms shown in Figure 4.10. There was only nugget effect shown in events S1 and S2 for the persistent soil moisture, and very little spatial correlation with the longer events. Variograms representing the soil moisture of the Tarrawarra project showed that there was purely nugget effect for the dry state, and a defined spatial correlation for the wet moisture state (Western et al. 2001). This is evidence that our catchment always stays in a moisture organizational state similar to the dry state of the Tarrawarra project, where soil moisture is controlled mostly by local soil properties.  The second explanation is that we simply did not measure soil moisture at a high enough resolution; while this experiment did have high spatial resolution, it was not nearly as high as that of the Tarrawarra project, which included over 1500 measurements in a 10.5 ha catchment (Western et al. 1999). The resolution in our catchment may not be enough to capture the difference between the extremely short scale variability during dry events, which  87  Western et al. (1999) found to be mostly dependent on local soil properties. Higher spatial resolution measurement is necessary to determine if either one of these explanations is the case (or if it is a combination of both).  There were a few points that were consistently very near the mean, with a low standard deviation for all events. Motes 11 and 32 stood out in particular as very near the catchment mean throughout all events, and hence could be reliable indicators of the overall catchment soil moisture. Motes 16, 17 and 31 also had good prediction of the catchment mean, but not quite as good as 11 and 32. Possible deterministic reasons for these motes being the most representative points over a number of long and short term measurement periods are not apparent from simply looking at the distribution on the catchment DEM (Figure 4.1). All of these points, except 17, are arranged in general along the centerline of the catchment. A more extensive deterministic analysis of each of these measurement locations, including local small scale topography and soil properties would be necessary to the properties that they have in common to make them the most representative of the catchment.  For throughfall, there was a noticeable difference between the persistent values for event S1, which was one individual storm vs. event L2 which was a series of many storms over the course of a month. There is a change from a somewhat bimodal distribution in the short event to a more even distribution of persistent values for the long event shown in Figure 4.6. Persistent throughfall values are distributed about the catchment in event S1 (Figure 4.7), with over-predicting points and under-predicting points mixed together throughout the catchment whereas all points trended to even out more toward an average in event L2, which  88  experienced multiple storms in very short succession. This leads us to believe that the short scale variability of canopy structure and its ability to store water has a dominant influence on rainfall interception in an individual storm whereas this storage capacity was eventually exceeded in all locations as multiple storms crossed the catchment. This change in dominant forces in the catchment means that we cannot reliably use any of our individual measurement points as an indication of the overall catchment throughfall, as individual measurements changed relative to the mean depending on how much water was already stored in the canopy.  4.4.3  Pearson correlation  The spatial display of Pearson r values (Figure 4.6) for soil moisture did not always mirror the results for persistence (Figure 4.5). There was a noticeably wider range of correlation values for events L1 and L2 than the short term events, but the distributions were not much different. The one result that stands out in the spatial display of Pearson r values of soil moisture is the line of very high Pearson r values that followed the general backbone of the catchment, which was also the area of the intermittent stream in the catchment (Figure 4.6). This is some evidence of water accumulating towards the stream in synchronization with the fluctuation of catchment mean soil moisture. This was probably only visible in event L2 because it was the only event with multiple large fluctuations in soil moisture.  Pearson r correlation levels were as a whole much closer to one for throughfall than they were for soil moisture. Both throughfall events had very high levels of correlation between individual points and the catchment mean that were evenly distributed throughout the  89  catchment. This high level of correlation was probably due in part to the way the Pearson product-moment correlation coefficient works. Throughfall values have a range of multiple orders of magnitude, whereas soil moisture only changes by at most 20% of the measured value, this give more strength to the correlation of throughfall values than soil moisture.  4.4.4  Variogram Analysis  The variogram analyses generally described the spatial variability of Pearson r values and persistent normalized values of soil moisture and throughfall as purely nugget effect, meaning that organization was due to factors not described at our sampling resolution. Only four out of twelve variograms showed any evidence of a correlation length, and all twelve had a high amount of noise between lags. This noise in the variograms was most likely due to having too few measurement pairs for each lag, because a tradeoff had to be made between having too few lags to see a trend and a low (~30) amount of measurement pairs per lag. In general, about 50 measurements have been recommended to begin to evaluate correlation length of soil moisture in a catchment (Bardossy and Lehmann 1998). With perfect function of all motes in this experiment, we would have been close to this recommended amount, but operational issues of an experimental measurement method have left us with slightly less than the desired amount.  4.5  Conclusions  We have used multiple different types of spatial distribution analysis to demonstrate how spatial organization of soil moisture and throughfall changes with the temporal scale and the overall wetness level of a catchment. The distribution of measured values was much larger  90  for soil moisture during wet periods than dry periods, but the organization of normalized individual measurements did not change very much between wet and dry analysis periods or with the timescale of analysis. Organization of Pearson correlation values between the individual measurement and the catchment mean soil moisture differed from the organization of mean normalized soil moisture in that it was mostly dependent on the timescale of measurement. Organization of normalized throughfall values was also dependent on the timescale of measurement, whereas the organization of Pearson r values for throughfall was similar for both short and long term timescales. Our inability to describe the spatial organization (for soil moisture or throughfall) by variogram analysis, points to the variability being mostly controlled by factors local to each measurement point.  91  4.6  References  Bardossy, A. and W. Lehmann (1998). "Spatial Distribution of soil moisture in a small catchment. Part 1: geostatistical analysis." Journal of Hydrology 206: 1-15. Canada (2006). "National Climate Data and Information Archive." Retrieved October 2, 2007, from http://climate.weatheroffice.ec.gc.ca/climateData/dailydata_e.html. D'Odorico, P. and A. Porporato (2004). "Preferential states in soil moisture and climate dynamics." Proceedings of the National Academy of Sciences 101(24): 8848-8851. Decagon (2006). "ECH2O Soil Moisture Probe Operator's Manual." Retrieved November 10, 2007, from http://www.decagon.com/manuals/echomanual.pdf. Egan, B. (1999). "The Ecology of the Coastal Western Hemlock Zone." Ecosystems of British Columbia Retrieved February 10, 2008, from http://www.for.gov.bc.ca/hfd/pubs/docs/Bro/bro31.pdf. Gay, D., P. Levis, R. v. Behren, M. Welsh, E. Brewer and D. Culler (2003). The nesC Language: An Holistic Approach to Networked Embedded Systems. ACM SIGPLAN 2003 conference on Programming language design and implementation, San Diego, CA, USA. Grayson, R. B., A. W. Western, G. Bloschl and F. H. S. Chiew (1997). "Preferred states in spatial soil moisture patterns: local and nonlocal controls." Water Resources Research 33(12): 2897-2908. Hill, J. L. and D. E. Culler (2002). "Mica: A Wireless Platform for Deeply Embedded Networks." IEEE Micro 22(6): 12-24. Huang, J., H. M. V. D. Dool and K. P. Georgakakos (1996). "Analysis of Model-Calculated Soil Moisture over the United States (1931-1993) and Applications to Long-Range Temperature Forecasts." Journal of Climate 9: 1350-1362. Isaaks, E. H. and R. M. Srivastava (1989). Applied Geostatistics. New York, NY, Oxford University Press. Keim, R. F., A. E. Skaugset and M. Weiler (2005). "Temporal persistence of spatial patterns in throughfall." Journal of Hydrology 314: 263-274. Lee, S. (2007). "Crossbow Technology, Inc. Quick Facts." Retrieved January 7, 2008, from http://www.xbow.com/General_info/Info_pdf_files/Crossbow_Quick_Facts.pdf. Levis, P., S. Madden, J. Polastre, R. Szewczyk, K. Whitehouse, A. Woo, D. Gay, J. Hill, M. Welsh, E. Brewer and D. Culler (2005). TinyOS: An Operating System for Sensor Networks. Ambient Intelligence. W. Weber, J. M. Rabaey and E. Aarts. New York, NY, Springer: 115148.  92  McBean, E. A. and F. A. Rovers (1998). Statistical Procedures for Analysis of Environmental Monitoring Data & Risk Assessment. Upper Saddle River, New Jersey, Prentice Hall PTR. Moore, R. D. and J. C. Thompson (1996). "Are water table variations in a shallow forest soil consistent with the TOPMODEL concept?" Water Resources Research 32(3): 663-669. Pellenq, J., J. Kalma, G. Boulet, G.-M. Saulnier, S. Wooldridge, Y. Kerr and A. Chehbouni (2003). "A disaggregation scheme for soil moisture based on topography and soil depth." Journal of Hydrology 276: 112-127. Qiu, Y., B. Fu, J. Wang and L. Chen (2001). "Soil moisture variation in relation to topography and land use in a hillslope catchment of the Loess Plateau, China." Journal of Hydrology 240: 243-263. Raat, K. J., G. P. J. Draaijers, M. G. Shaap, A. Tietema and J. M. Verstraten (2002). "Spatial variability of throughfall water and chemistry and floor water content in a Douglas fir forest stand." Hydrology and Earth System Sciences 6(3): 363-374. Rainwise (2007). "Wired Rain Gauge." from http://www.rainwise.com/products/detail.php?ID=6697&Category=Rain_Gauges:Wired&pa geNum_cart=/products/index.php. Ridolfi, L., P. D'Orico, A. Porporato and I. Rodgriquez-Iturbe (2003). "Stochastic soil moisture dynamics along a hillslope." Journal of Hydrology 272: 264-275. Robertson, G. P., J. R. Crum and B. G. Ellis (1993). "The spatial variability of soil resources following long-term disturbance." Oecologia 96: 451-456. Shah, S. M. S., P. E. O'Connell and J. R. M. Hosking (1996). "Modelling the effects of spatial variability in rainfall on catchment response. 1. Formulation and calibration of a stochastic rainfall field model." Journal of Hydrology 175: 67-88. Singh, V. P. (1997). "Effect of spatial and temporal variability in rainfall and watershed characteristics on stream flow hydrograph." Hydrological Processes 11: 1649-1669. Stern, D. (2003). IDL Student Edition. Boulder, CO, Research Systems, Inc./ ITT Visual Information Solutions. Thompson, J. C. (1994). Water Table Behaviour and Prediction. Department of Geography. Vancouver, BC, Simon Fraser University. Master of Science: 156. Walker, J. P., G. R. Willgoose and J. D. Kalma (2001). "The Nerrigundah data set: soil moisture patterns, soil characteristics and hydrological flux measurements." Water Resources Research 37(11): 2653-2658.  93  Western, A. W. and G. Bloschl (1999). "On the spatial scale of soil moisture." Journal of Hydrology 217: 203-224. Western, A. W., R. B. Grayson and G. Bloschl (2001). "Towards capturing hydrologically significant connectivity in spatial patterns." Water Resources Research 37(1): 83-97. Western, A. W., R. B. Grayson, G. Bloschl, G. R. Willgoose and T. A. McMahon (1999). "Observed spatial organization of soil moisture and its relation to terrain indices." Water Resources Research 35(3): 797-810. Western, A. W., R. B. Grayson and T. R. Green (1999). "The Tarrawarra project: high resolution spatial measurement, modelling and analysis of soil moisture and hydrologic responses." Hydrological Processes 13: 633-652. Whelan, M. J. and J. M. Anderson (1996). "Modelling spatial patterns of throughfall and interception loss in a Norway spruce (Picea abies) plantation at the plot scale." Journal of Hydrology 186: 335-354.  94  Chapter 5:  5.1  Conclusions  Discussion  The analyses in Chapters 3 and 4 both required data that was collected at high spatial and temporal resolution for enough time that the full range of moisture conditions typical to Coastal British Columbia would be experienced. Without the development of the hardware and data collection network described in Chapter 2 of this thesis, the data analyses of Chapters 3 and 4 would not have been possible. The relatively low price of motes made it possible to have many more measurement locations than with the use of more common data loggers. The tradeoff of using a less expensive emerging method for data collection is that the reliability was not well known before the experiment. While I did have issues with data collection and operation of the mote network, as a whole it succeeded in collecting the data necessary to perform the analyses in Chapters 3 and 4.  Along with the dependence that Chapters 3 and 4 have on Chapter 2 of this thesis, they are also connected. The differences in results among different sized storms and between wet and dry periods in both Chapters reinforce the need to investigate many different events when trying to make general conclusions about moisture processes within a catchment. Also, Chapters 3 and 4 show that the timescale of measurement can affect results even if everything else is the same. Changing the timescale can make results seem different even though nothing has physically changed.  95  The analysis of Chapters 3 and 4 do differ in their results on recognizing changing scales of soil moisture controls. In Chapter 3, results lead to conclusions that the catchment is switching between controls much more often than the Tarrawarra catchment described by Western, et al (2001), whereas in Chapter 4, results show that either we do not have the necessary spatial resolution to detect a change, or no change is occurring at all. As the analysis in Chapter 4 is much more similar to what was done in the Tarrawarra catchment, these results are probably a better indicator than Chapter 3.  5.2  Weaknesses  There are three main weaknesses in this experiment. The first is the reliability of the motes themselves. As explained in Chapter 2, there were many instrument malfunctions for no apparent reason. This leaves many random gaps in data measurements throughout the entire collection period. For Chapters 3 and 4, I chose events for analysis that minimized these random instrument error problems, but they still existed.  The second weakness in this experiment was most apparent in the spatial analysis in Chapter 4. While I attempted to compare soil moisture spatial distribution in the same manner as the Tarrawarra project (Western et al. 1999), but with a much better temporal resolution, the spatial resolution was not high enough. This experiment had an excellent combination of spatial and temporal variability, but the spatial variability on its own was not high enough to display changing controls of soil moisture (or throughfall) in the same manner as the Tarrawarra project.  96  The third weakness in this experiment is inherent to many hydrologic experiments that are only performed in one location. That weakness is the unknown applicability to other locations. It is unknown if the results I describe in Chapters 3 and 4 are going to be reproduced in a different hydrologic catchment. This would only be possible if the same experiment was performed in many different catchments; doing this was beyond the scope of this M.A.Sc. thesis.  5.3  Strengths and Application  Chapter 2 provides one of the few accounts of the application of motes to actual hydrologic measurement. While many hydrologists are aware of the emerging technology of motes, most of the information that exists about them is that provided by the manufacturer, or by computer scientists experimenting with the motes themselves (instead of practical uses for motes). Upon speaking with hydrologists (and other scientists), I found that they did not realize the amount of time and effort that was required to start a research project with motes and the amount of instability that they currently exhibit. Before I began this project I thought that setup would not be all that different from other more established data collection methods. Through the course of the setup I found that this was not the case. The long and involved setup time is probably the reason that most of the descriptions of projects using motes are much more focused on the setup and operation of the motes themselves than the actual data that they collected (Mainwaring et al. 2002; Glaser 2004; Talzi et al. 2007).  There are rapid advancements happening in the hardware and software for mote networks that promise to make mote experiments much more reliable and easy to use (Crossbow  97  2007). While this advancement is very positive, it is not conducive to the operation of a long term experiment using motes at this time. Since the time I began this project two years ago, the exact hardware that I used is no longer available and there are completely new software packages available. This rapid advancement needs to slow down before motes become more practical for use in hydrology field experiments, where hardware is often damaged and need to be replaced. I hope that this thesis (Chapter 2 in particular) can help scientists realize that the use of motes for environmental data collection is still very much in the experimental phase and it needs more time before it as easy to use and reliable as other methods.  While motes do still have some issues with implementation and operation, this experiment was reasonably successful in collecting hydrological data at a combined level of spatial and temporal resolution that is quite rare. The analysis that I describe in Chapter 3 can potentially be useful to hydrologists designing catchment sampling experiments. I have provided evidence that there is not huge benefit of extremely high measurement resolution (greater than ~1.5 points/ha) if the information required is a homogeneous parameter for throughfall or soil moisture, even with completely random sampling locations.  Whereas Chapters 2 and 3 provide information on hardware and experimental design, the results of Chapter 4 describe actual hydrological process. There have been previous studies showing how spatial variability is different in different conditions for throughfall (Whelan and Anderson 1996; Raat et al. 2002; Keim et al. 2005) and soil moisture (Western et al. 1999; Western et al. 2001), but these analyses have shown storm total spatial variability for throughfall and snapshot measurements for soil moisture. With the benefit of the mote  98  network for measurement, I have been able to describe the spatial variability of soil moisture and throughfall with more emphasis on how it changes over time, and how consistent it is within a specific storm event.  For the study catchment in Malcolm Knapp Research Forest, I have duplicated the ranking of persistent throughfall values by Keim et al, (2005) for throughfall and soil moisture with the addition of the standard deviation of the persistent value throughout the entire event. Particularly for throughfall, this addition has shown that there is a high level of variability of throughfall at individual points throughout a storm event. This could not be seen with only storm total throughfall measurements.  5.4  Future Work  There are two separate directions for future work stemming from the results of this project. The first is further development of the mote network to move closer to the ultimate goal of an entirely automated data collection system. The next planned evolution of this network will allow for data to be transmitted from all instrument clusters to a base point somewhere in the catchment. A more powerful radio modem (ELPRO 905U-D) will then transmit this data nightly to a computer located in the Malcolm Knapp field office in order to connect to the UBC network and display the data online. This will allow for easy monitoring of the data being collected in the catchment in nearly real time from anywhere with internet access, without the necessity of being in the catchment. With a fully automated data collection system, the next issue that needs to be addressed is power consumption. Tipping buckets were the major limiter of power consumption in this project, and there are two possible  99  solutions. The first option is automatic battery charging. Instrument clusters could be powered using small solar panels to continuously recharge lead acid batteries regulated to deliver 3.3 volts. Using separate loggers for measuring tips in the throughfall gauges and connecting them to the MDA300 is another possible solution for the inability to use the sleep function of the Mica2. With these two advancements, the mote network will be much closer to fully realizing its potential as a fully automated hydrological data collection network.  The second direction for further analysis of the results of this experiment would be the use of the dataset I collected in Malcolm Knapp Research Forest for different types of analyses. While I chose to do two analyses that I felt would be useful and unique, there are many other options for analysis of this dataset. Aside from soil moisture and throughfall, there were multiple other processes successfully monitored in the catchment at the same time, while monitoring of groundwater head was a failure. These include soil and air temperature, humidity, overland flow measurements, and stream flow measurements at the 3 installed Vnotch weirs. A number of options are possible for this dataset, such as further investigating into changing spatial variability of parameters, deterministic links between processes, and a detailed hydrologic budget for the entire catchment.  100  5.5  References  Crossbow (2007). "MoteWorks 2.0 Software Platform." Retrieved January 10, 2008, from http://www.xbow.com/Products/Product_pdf_files/Wireless_pdf/MoteWorks_OEM_Edition. pdf. Glaser, S. D. (2004). Some real-world applications of wireless sensor nodes. SPIE Symposium on Smart Structures & Materials, San Diego, California. Keim, R. F., A. E. Skaugset and M. Weiler (2005). "Temporal persistence of spatial patterns in throughfall." Journal of Hydrology 314: 263-274. Mainwaring, A., J. Polastre, R. Szewczyk, D. Culler and J. Anderson (2002). Wireless Sensor Networks for Habitat Monitoring. Proceedings of the 1st ACM International Workshop on Wireless Sensor Networks and Applications. Atlanta, GA, ACM Press: 88-97. Raat, K. J., G. P. J. Draaijers, M. G. Shaap, A. Tietema and J. M. Verstraten (2002). "Spatial variability of throughfall water and chemistry and floor water content in a Douglas fir forest stand." Hydrology and Earth System Sciences 6(3): 363-374. Talzi, I., A. Hasler, S. Gruber and C. Tschudin (2007). "PermaSense: Investigating Permafrost with a WSN in the Swiss Alps." In Press. Western, A. W., R. B. Grayson and G. Bloschl (2001). "Towards capturing hydrologically significant connectivity in spatial patterns." Water Resources Research 37(1): 83-97. Western, A. W., R. B. Grayson and T. R. Green (1999). "The Tarrawarra project: high resolution spatial measurement, modelling and analysis of soil moisture and hydrologic responses." Hydrological Processes 13: 633-652. Whelan, M. J. and J. M. Anderson (1996). "Modelling spatial patterns of throughfall and interception loss in a Norway spruce (Picea abies) plantation at the plot scale." Journal of Hydrology 186: 335-354.  101  Appendix A.  A.1  Section of Chapter 2 Written by Kan Cai.  The Framework  The framework of our software system consists of four components: (1) Data collecting (2) Data Logging (3) Data Polling (4) Data Communication. The data collecting component gathers data from the sensors; the data logging component records all the information locally to the EPROM; the data polling component pulls data from each mote's EPROM to the polling node; and the communication component is in charge of sending/forwarding the data from each mote to a lab machine five miles away from the sensing field. The software system is implemented using NesC language under TinyOS version 1.1.10. In this section, we will describe the implmention of each of these components in details.  A.1.1  Data Collecting  Our monitoring system consists of seven sensors, two of which are digital sensors while the rest are analog ones. Each sensor data is connected to one channel of the data board MDA300. The data collecting subsystem collects data from these sensors once every 15 minutes. The time interval is so big that we do not need to worry if the system cannot finsh reading between runs.  The data is collected sequentially, i.e., one reading after another, by calling getSample function of NesC's DataTakeC component. Everytime a reading is done, the system will be notified and trapped in the dataReady function, in which it issues another read for the next channel. The reading results are stored in an array of type int (2-byte long each) temporially. When all the readings are done, the results will be passed to the logger component for  102  recording.  A.1.2  Data Logging  When one round of readings is done, our system needs to record the data to the EPROM at each sensor. This is done using the Logger component of NesC. The EPROM of each mote can store totally 512 Kbytes, consisting of 32K lines, 16 bytes per line. After each round of reading, a mote obtains a 24-byte record, including 7 sensing results, 3 mote health information as well as a 4-byte long timestamp. This means that, in our system, each mote consumes the EPROM two lines per 15 minutes. At this rate, it will take a mote more than 160 days to use all the lines in the EPROM.  Every time a mote records data to the EPROM, it issues two sequential writes. The mote is notified and trapped in the writeDone function after each write is done. The mote keeps a flag for each of the two writes; when the first flag is set and the second write returns successfully, the mote knows that both writes have succeed, and thus stops writing.  The motes will not immediately reset the logging position to the beginning of the EPROM after being polled. The reason behind this is our concern of data safety, i.e., we want the motes to keep the data as long as possible in case the data is lost for any possible reason after being polled to our computer, hard disk failure, for example. This approach, however, raises two issues. One, where to store the data when all the lines in the EPROM are eventually used? In this case, we simply wrap the writing pointer back to the beginning of the EPROM, starting to overwrite the oldest data in the EPROM.  103  Second, how can we determine the position in the EPROM where a mote stores the last set of readings? If we cannot get this right, we might read the oldest record in the mote right after the latest one and thus cannot capture the changing continuation in time. To address this issue, we designate the line 19 as the special line to record the last reading line number. Every time a writing is done, the mote needs to update the value at the line 19 so that, when it is being polled, it knows where it should stop. Note that the real data starts from line 20 in the EPROM.  A.1.3  Data Polling  The data polling sub-system allows us to manually collect data from each sensor, one at a time. The implementation of communication between the polling mote and the polled mote is based on the GenericComm component of NesC. The data polling sub-system consists of three components. The first component is running on the laptop to which the polling mote is connected via a serial/USB cable. It is a Java file, which simply writes the data, received from the serial/USB port, to a file on the local hard disk.  The second component is running at the polling mote, which continues to send polling message to the polled mote until it receives data packets. This polling message not only includes the polled mote ID but also specifies the starting polling line for that mote. Therefore, when the polled mote receives such a message, it knows which line to start reporting. When the polling mote receives the data, it forwards the data to the computer immediately.  104  The last compnent is running at the polled mote, which has collected the data and stored them in its EPROM. Once receiving a polling message, it reads data from its EPROM, wraps each line in a message and then sends it to the polling mote. It continues doing so until it reaches the final line that stores the last set of readings, as indicated by the value stored at line 19.  A.2  Data Communication  The data communication subsystem consists of two components: (1) multi-hop data routing from motes to the sink (2) Long-range communication between radio modems.  A.2.1  Multi-Hop Data Routing  Our multip-hop routing component is based on the TinyOS's example multiple-hop application, surge, which in turn is built upon the three multi-hop modules of TinyOS: MultiHopEngineM, MultiHopLEPSM, and MultiHopRouter. The beauty of doing so is that our system does not need to care about how the underlying topology/connection is built. For example, our application does not need to worry about how a mote can detect its neighbours and find its way to reach the sink node (the mote with node ID 0).  We have implemented this multi-hop data routing/forwarding facility. Every time a mote finishes recoding the readings to its EPROM, it sends the data to the sink as well. When an intermediate mote receives such a data message, it forwards the message to the next-hop until the the sink mote receives it.  105  A.2.2  Long-Range Communication Between Radio Modems  We rely on two radio modems, ELPRO 905U-D, to deliver the data from the sink mote to a far-away lab machine. The sending frequency is set to 900 Mhz and the rate is fixed to 9.6 Kbps, which is fast enough to handle our data collecting rate. One modem is connect to the MIB510 programming board via serial cable, while another is connected to the lab machine which is 2 km away. It is claimed that these radio modems can communicate over 15 miles under ideal conditions. We have tried this setting in the lab, and it works as expected. Fieldtest is needed for our future work.  106  Appendix B.  IDL Program read_motes_linux_final.pro  Used for converting raw hexadecimal data from the motes into usable information. path='C:\Documents and Settings\Hydrology\My Documents\MKRF-A41\Mote Data\Combined_stat\' file='node_6_all' ; Length of text file openr, unit, path+file, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE close, unit array=dblarr(17,n_lines-2) temp='' openr, unit, path+file, /get_lun skip_lun, unit, 2, /LINES correct_lines=intarr(1) for i=0, n_lines-3 do begin READF, unit, temp temp1 = STRSPLIT(temp, ' ', /EXTRACT) IF n_elements(temp1) eq 18 THEN BEGIN for j=0,15 do begin temp11=temp1(j) res=0 ReadS, temp11, res, Format='(Z)' array(j,i)=res endfor  107  ReadS, strmid(temp1(17),2,2)+strmid(temp1(17),0,2)+strmid(temp1(16),2,2)+strmid(temp1(16),0,2), res1, Format='(z)' array(16,i)=res1/1000./60./60./24. correct_lines=[correct_lines,i] endif endfor ;print, i, array(*,i) close, unit ; only use correct lines correct_lines=correct_lines(1:*) array=array(*,correct_lines) ;Data Processing ;Internal Voltage array(2,*)=1252.32 / array(2,*) ;Internal Temperature array(4,*)=-38.4+.0098*array(4,*) ;Internal Humidity array(3,*)=(array(4,*)-25)*(.01+.0008*array(3,*))+(-4 + 0.0405*array(3,*) - 0.0000028*array(3,*)^2) ;Air Temperature array(5,*)=625*array(5,*) / 1024000 array(5,*)=15*array(5,*) / (2.5 - array(5,*)) array(5,*)=0.00002*array(5,*)^4 - 0.0033*array(5,*)^3 + 0.1813*array(5,*)^2 - 5.0781*array(5,*) + 61.051 ;Humidity array(6,*)=625*array(6,*) / 1024000 array(6,*)=57.5*array(6,*) - 38.987 ;Soil Temperature array(7,*)=625*array(7,*) / 1024000 array(7,*)=13*array(7,*) / (2.5 - array(7,*)) array(7,*)= -10.528*aLog(array(7,*)*1000) + 122.22 ;Soil Moisture array(8,*)=625*array(8,*) / 1024000 array(8,*)=(100*array(8,*)/1.5)-33.33 ;Groundwater Level piez_depth = .476  108  array(9,*)= 12.5*(array(9,*)/2048 - 1) array(9,*)= -0.25*(array(9,*))-.5-piez_depth ;Rainfall array(11,*)=array(11,*)*.25 ;Time dummy = LABEL_DATE(DATE_FORMAT=['%D-%M, %H:%I']) startday= JULDAY(7,14,2006,10,14,0) batt1= JULDAY(8,9,2006,10,55,0) batt2= JULDAY(9,11,2006,12,01,0) batt3= JULDAY(10,17,2006,10,36,0) batt4= JULDAY(11,20,2006,10,33,0) batt5= JULDAY(11,27,2006,14,41,0) array(16,*)=((array(16,*) LT -0.0000001)*49.7103)+array(16,*) vector=intarr(n_elements(correct_lines)) for i=1, n_elements(correct_lines)-1 do begin vector(i)=array(16,i) lt array(16,i-1) endfor restart = WHERE (vector eq 1) ;plot, array(16,*) array(16,0:restart(0)-1)=array(16,0:restart(0)-1)+startday array(16,restart(0):restart(1)-1)=array(16,restart(0):restart(1)-1)+batt1 array(16,restart(1):restart(2)-1)=array(16,restart(1):restart(2)-1)+batt2 array(16,restart(2):restart(3)-1)=array(16,restart(2):restart(3)-1)+batt3 array(16,restart(3):restart(4)-1)=array(16,restart(3):restart(4)-1)+batt4 array(16,restart(4):n_elements(array(16,*))-1) = array(16,restart(4):n_elements(array(16,*))-1)+batt5 ;Plot window,0, xsize=700, ysize=700 !p.multi=[0,3,3] plot, array(16,*), array(2,*), TITLE = "Internal Voltage", YTITLE="Volts", XTITLE="Time", XTICKFORMAT = 'LABEL_DATE', MIN_VALUE=0, MAX_VALUE=4 plot,array(16,*), array(3,*), TITLE = "Internal Humidity", psym=3, YTITLE="%", XTITLE="Time", XTICKFORMAT = 'LABEL_DATE'  109  plot,array(16,*), array(4,*), TITLE = "Internal Temperature", YTITLE="Deg C", XTITLE="Time", XTICKFORMAT = 'LABEL_DATE' plot,array(16,*), array(5,*), TITLE = "Air Temperature", YTITLE="Deg C", XTITLE="Time", XTICKFORMAT = 'LABEL_DATE', MAX_VALUE=50 plot,array(16,*), array(6,*), TITLE = "Humidity", YTITLE="%", XTITLE="Time", XTICKFORMAT = 'LABEL_DATE' plot,array(16,*), array(7,*), TITLE = "Soil Temperature", psym=3, YTITLE="Deg C", XTITLE="Time", XTICKFORMAT = 'LABEL_DATE' plot,array(16,*), array(8,*), TITLE = "Soil Moisture", psym=3, YTITLE="%", XTITLE="Time", XTICKFORMAT = 'LABEL_DATE', MIN_VALUE=0, MAX_VALUE=100 plot,array(16,*), array(9,*), TITLE = "Groundwater Level", psym=3, YTITLE="Meters Below Surface", XTITLE="Time", XTICKFORMAT = 'LABEL_DATE' ;plot,array(16,*), array(10,*), TITLE = "Overland Flow", psym=3, XTITLE="Time", XTICKFORMAT = 'LABEL_DATE' plot,array(16,*), array(11,*), TITLE = "Rainfall",YTITLE="Millimeters", XTITLE="Time", XTICKFORMAT = 'LABEL_DATE' ;write content into a file openw, 1, path+file+'_output_date.csv' printf, 1, 'Date, Internal Voltage, Internal Humidity, Internal Temperature, Air Temperature, Humidity, Soil Temperature, Soil Moisture, Groundwater Level, Rainfall, Overland Flow' for i=0, n_elements(array(0,*))-1 do begin printf, 1, array(16,i)-JULDAY(12,30,1899,0,00), ',', array(2,i), ',', array(3,i), ',', array(4,i), $ ',', array(5,i), ',', array(6,i), ',', array(7,i), ',', array(8,i), ',', array(9,i), ',', array(11,i), $ ',', array(10,i), FORMAT='(23A)' Endfor close, 1 end  110  Appendix C.  IDL program qualitydata.pro  Used to determine the percentage of usable values for each mote in chapter 2 outputarray = dblarr(43,8) outputarray(*) = -9999 ;Mote Files to Use mote = strarr(39) mote(0)='2' mote(1)='3' mote(2)='4' mote(3)='5' mote(4)='6' mote(5)='7' mote(6)='8' mote(7)='9' mote(8)='10' mote(9)='11' mote(10)='12' mote(11)='13' mote(12)='14' mote(13)='15' mote(14)='16' mote(15)='17' mote(16)='18' mote(17)='19' mote(18)='20' mote(19)='21' mote(20)='22' mote(21)='23' mote(22)='24' mote(23)='25' mote(24)='28' mote(25)='29' mote(26)='30' mote(27)='31' mote(28)='32' mote(29)='33'  111  mote(30)='34' mote(31)='35' mote(32)='36' mote(33)='37' mote(34)='38' mote(35)='39' mote(36)='40' mote(37)='41' mote(38)='43' inpathA='C:\Documents and Settings\Hydrology\My Documents\MKRF-A41\Mote$ Data\Combined_stat\' infileA='node_' infileB='_all_output_date.csv' for k=0, n_elements(mote)-1 do begin ;read from files openr, unit, inpathA+infileA+mote(k)+infileB, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE array=fltarr(11,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,10 do begin temp11=temp1(j) array(j,i)=temp11  112  endfor endfor  close, unit FREE_Lun, unit outputarray(mote(k)-1,0) = n_elements(array(7,*)) dummy = WHERE(array(7,*) GE 0 and array(7,*) LE 100) outputarray(mote(k)-1,1) = (n_elements(dummy)/outputarray(mote(k)$ -1,0))*100 dummy = WHERE(array(4,*) GE -10 and array(4,*) LE 40) outputarray(mote(k)-1,2) = (n_elements(dummy)/outputarray(mote(k)$ -1,0))*100 dummy = WHERE(array(5,*) GE 0 and array(5,*) LE 100) outputarray(mote(k)-1,3) = (n_elements(dummy)/outputarray(mote(k)$ -1,0))*100 dummy = WHERE(array(6,*) GE -10 and array(6,*) LE 40) outputarray(mote(k)-1,4) = (n_elements(dummy)/outputarray(mote(k)$ -1,0))*100 dummy = WHERE(array(1,*) GE 0 and array(1,*) LE 4) outputarray(mote(k)-1,5) = (n_elements(dummy)/outputarray(mote(k)$ 1,0))*100 dummy = WHERE(array(2,*) GE 0 and array(2,*) LE 100) outputarray(mote(k)-1,6) = (n_elements(dummy)/outputarray(mote(k)$ -1,0))*100 dummy = WHERE(array(3,*) GE -10 and array(3,*) LE 40) outputarray(mote(k)-1,7) = (n_elements(dummy)/outputarray(mote(k)$ -1,0))*100 endfor  113  ;write to file outpath='C:\Documents and Settings\Hydrology\My Documents\MKRF-A41\Mote$ Data\Organized Data\' outfile='gooddata.csv' openu, 51, outpath+outfile, /APPEND printf, 51, outputarray, format="(43(f10.2, ','))" close, 51 END  114  Appendix D.  IDL Program transposerange.pro  Used for selecting a single parameter over time from all motes and reorganizing it into one file in Chapters 3 and 4. ;date range extracttimestart = JULDAY(11,1,2006,00,00)-JULDAY(12,30,1899,0,00) extracttimeend = JULDAY(11,30,2006,24,00)-JULDAY(12,30,1899,0,00) ;time interval (hours) interval = 1. numlines = (extracttimeend-extracttimestart)/interval + 1 numlines = ROUND(numlines) ;create output array outputarray = dblarr(43,numlines) outputarray(*) = -9999 for i=0, numlines-1 do begin outputarray(0,i)= extracttimestart+interval*i endfor ;Mote Files to Use mote = strarr(38) mote(0)='2' mote(1)='3' mote(2)='4' mote(3)='5' mote(4)='6' mote(5)='10' mote(6)='11' mote(7)='12' mote(8)='13' mote(9)='14' mote(10)='15' mote(11)='16' mote(12)='17' mote(13)='18'  115  mote(14)='19' mote(15)='20' mote(16)='21' mote(17)='22' mote(18)='23' mote(19)='24' mote(20)='25' mote(21)='27' mote(22)='28' mote(23)='29' mote(24)='30' mote(25)='31' mote(26)='32' mote(27)='33' mote(28)='35' mote(29)='36' mote(30)='37' mote(31)='38' mote(32)='40' mote(33)='41' mote(34)='43' mote(35)='7' mote(36)='8' mote(37)='9' inpathA='C:\Documents and Settings\Hydrology\My Documents\MKRF-A41\ $ Mote Data\Combined_stat\' infileA='node_' infileB='_all_output_date.csv' for k=0, n_elements(mote)-1 do begin ;read from files openr, unit, inpathA+infileA+mote(k)+infileB, /get_lun n_lines=0. dummy=''  116  WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE array=fltarr(11,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,10 do begin temp11=temp1(j) array(j,i)=temp11 endfor endfor close, unit FREE_Lun, unit for l=0, numlines-1 do begin desiredrows = Where((array(0,*) GT outputarray(0,l)-.006) $ and (array(0,*) LT outputarray(0,l)+.006)) If desiredrows(0) GT -1 then begin outputarray(mote(k) $ 1,l)=array(7,desiredrows(n_elements(desiredrows)-1)) endif endfor endfor ;write to file outpath='C:\Documents and Settings\Hydrology\My Documents\MKRF-A41\ $ Mote Data\Organized Data\' outfile='soilmoisturemulti.csv' openu, 51, outpath+outfile, /APPEND printf, 51, outputarray, format="(43(f10.4, ','))" close, 51 END  117  Appendix E.  IDL program randomselectionCI.pro  Used for creating data ensembles and calculating the RMSE of each ensemble for Chapter 3. path='C:\Documents and Settings\Hydrology\My Documents\MKRF-A41\Mote Data\Organized Data\paper data\' file='apr7_8_soil suff='.csv' ;graph string labels ;xlabel='Days after March 1, 2007 0:00' ;title='Ensemble of 100 for 10 Randomly Selected Points' ;t-distribution values for 90% CI (2 tailed) conf=fltarr(21) conf=[1.812,1.796,1.782,1.771,1.761,1.753,1.746,1.740,1.734,1.729,1.725,1.721,1.717,1.714,1.711, 1.708,1.706,1.703,1.701,1.699,1.697] ;number of points selected num = 5 number = '5' ;read from files openr, unit, path+file+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE array=dblarr(43,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES  118  for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,42 do begin temp11=temp1(j) array(j,i)=temp11 endfor endfor close, unit FREE_Lun, unit dummy = LABEL_DATE(DATE_FORMAT=['%D/%N/%Z, %H%A']) realvalue=dblarr(n_lines-1,105) ;transpose date realvalue(*,0)=array(0,0:n_lines-2) ;realvalue(*,0)=FINDGEN(n_lines-1) ;Average, SD, and 90% CI of all collected values (True Value) for n=0,n_lines-2 do begin temp=WHERE(array(1:*,n) GE 0 AND array(1:*,n) LE 100, count) ;IF count GT 0 then begin MEANSM = mean(array(temp+1,n)) SD = STDDEV(array(temp+1,n)) ;endif realvalue(n,1)=MEANSM realvalue(n,2)=SD IF count GT 10 and count LE 31 then begin realvalue(n,3)=MEANSM+SD*conf(count-11) realvalue(n,4)=MEANSM-SD*conf(count-11) endif if count GT 31 then begin realvalue(n,3)=MEANSM+SD*1.69 realvalue(n,4)=MEANSM-SD*1.69 endif if count LE 10 then begin  119  realvalue(n,3)=MEANSM+SD*1.83 realvalue(n,4)=MEANSM-SD*1.83 endif endfor ;Fill Array with average value from variable number of points ;some points may not contain valid values For i=5,104 Do begin points=Round(RANDOMU(seed,num)*40+2) SM=fltarr(num) for n=0,n_lines-2 do begin SM=array(points,n) temp=WHERE(SM GE 0 AND SM LE 100, count) IF count GT 0 then MEANSM= mean(SM(temp)) realvalue(n,i)=MEANSM endfor endfor ;Calculate Mean error for each realization rmse= fltarr(100) for i=0,99 do begin rmse(i)=SQRT(MEAN((realvalue(*,i+5)-realvalue(*,1))^2)) endfor sorted=sort(rmse) rmse=rmse(sorted) ;;write content into a file transreal=TRANSPOSE(realvalue) ;openw, 1, path+file+number+'_real.csv' ;printf, 1, transreal, format="(105(f10.3, ','))" ;close, 1 ;openw, 2, path+file+number+'_rmse.csv' ;printf, 2, rmse, format="(f10.3, ',')" ;close, 2 END  120  Appendix F.  IDL program soilpearson.pro  Used for calculating the Pearson r value for each ensemble in chapter 3. path='C:\Documents and Settings\Hydrology\My Documents\MKRF-A41\Mote Data\Organized Data\paper data\events\' filepre='apr7_8_soil' filepost='_real' file=strarr(3) file(0)='5' file(1)='10' file(2)='20' suff='.csv' ;read from files openr, unit, path+filepre+file(0)+filepost+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE five=fltarr(105,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,104 do begin temp11=temp1(j) five(j,i)=temp11 endfor endfor close, unit FREE_Lun, unit  121  openr, unit, path+filepre+file(1)+filepost+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE ten=fltarr(105,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,104 do begin temp11=temp1(j) ten(j,i)=temp11 endfor endfor close, unit FREE_Lun, unit openr, unit, path+filepre+file(2)+filepost+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE twenty=fltarr(105,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES  122  for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,104 do begin temp11=temp1(j) twenty(j,i)=temp11 endfor endfor close, unit FREE_Lun, unit five=transpose(five) ten=transpose(ten) twenty=transpose(twenty) ;Calculate Mean error for each realization pearson= fltarr(3,100) for i=0,99 do begin pearson(0,i)=correlate(five(*,i+5),five(*,1)) endfor for i=0,99 do begin pearson(1,i)=correlate(ten(*,i+5),ten(*,1)) endfor for i=0,99 do begin pearson(2,i)=correlate(twenty(*,i+5),twenty(*,1)) endfor sorted1=sort(pearson(0,*)) pearson(0,*)=pearson(0,sorted1) sorted2=sort(pearson(1,*)) pearson(1,*)=pearson(1,sorted2) sorted3=sort(pearson(2,*)) pearson(2,*)=pearson(2,sorted3)  123  ;;;write content into a file openw, 2, path+filepre+'_pearson.csv' printf, 2, '5 Point', ',', '10 Point', ',', '20 Point' printf, 2, pearson, format="(3(f10.3, ','))" close, 2 END  124  Appendix G.  IDL program rmsestatsevents.pro  Used for F and T test values in Chapter 3. path='C:\Documents and Settings\Hydrology\My Documents\MKRF-A41\Mote Data\Organized Data\paper data\events\' suff='.csv' file=strarr(4) file(0)='sep18_20_rain_rmse' file(1)='mar21_24_rain_rmse' file(2)='mar25_29_rain_rmse' file(3)='apr7_8_rain_rmse' ;read from files openr, unit, path+file(0)+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE a=fltarr(3,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,2 do begin temp11=temp1(j) a(j,i)=temp11 endfor endfor  125  close, unit FREE_Lun, unit openr, unit, path+file(1)+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE b=fltarr(3,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,2 do begin temp11=temp1(j) b(j,i)=temp11 endfor endfor close, unit FREE_Lun, unit openr, unit, path+file(2)+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE c=fltarr(3,n_lines-1)  126  temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,2 do begin temp11=temp1(j) c(j,i)=temp11 endfor endfor close, unit FREE_Lun, unit openr, unit, path+file(3)+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE d=fltarr(3,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,2 do begin temp11=temp1(j) d(j,i)=temp11 endfor endfor close, unit FREE_Lun, unit  127  ;log transform a=ALOG10(a) b=ALOG10(b) c=ALOG10(c) d=ALOG10(d)  ;f-test, then t-test combos f=fltarr(2,26) t=fltarr(2,26) f(*,0)=FV_TEST(a(0,*),a(1,*)) IF f(1,0) GT .05 then t(*,0)=TM_TEST(a(0,*),a(1,*)) ELSE t(*,0)=TM_TEST(a(0,*),a(1,*), /UNEQUAL) f(*,1)=FV_TEST(a(1,*),a(2,*)) IF f(1,1) GT .05 then t(*,1)=TM_TEST(a(1,*),a(2,*)) ELSE t(*,1)=TM_TEST(a(1,*),a(2,*), /UNEQUAL) f(*,2)=FV_TEST(b(0,*),b(1,*)) IF f(1,2) GT .05 then t(*,2)=TM_TEST(b(0,*),b(1,*)) ELSE t(*,2)=TM_TEST(b(0,*),b(1,*), /UNEQUAL) f(*,3)=FV_TEST(b(1,*),b(2,*)) IF f(1,3) GT .05 then t(*,3)=TM_TEST(b(1,*),b(2,*)) ELSE t(*,3)=TM_TEST(b(1,*),b(2,*), /UNEQUAL) f(*,4)=FV_TEST(c(0,*),c(1,*)) IF f(1,4) GT .05 then t(*,4)=TM_TEST(c(0,*),c(1,*)) ELSE t(*,4)=TM_TEST(c(0,*),c(1,*), /UNEQUAL) f(*,5)=FV_TEST(c(1,*),c(2,*)) IF f(1,5) GT .05 then t(*,5)=TM_TEST(c(1,*),c(2,*)) ELSE t(*,5)=TM_TEST(c(1,*),c(2,*), /UNEQUAL) f(*,6)=FV_TEST(d(0,*),d(1,*)) IF f(1,6) GT .05 then t(*,6)=TM_TEST(d(0,*),d(1,*)) ELSE t(*,6)=TM_TEST(d(0,*),d(1,*), /UNEQUAL) f(*,7)=FV_TEST(d(1,*),d(2,*)) IF f(1,7) GT .05 then t(*,7)=TM_TEST(d(1,*),d(2,*)) ELSE t(*,7)=TM_TEST(d(1,*),d(2,*), /UNEQUAL) f(*,8)=FV_TEST(a(0,*),b(0,*))  128  IF f(1,8) GT .05 then t(*,8)=TM_TEST(a(0,*),b(0,*)) ELSE t(*,8)=TM_TEST(a(0,*),b(0,*), /UNEQUAL) f(*,9)=FV_TEST(a(0,*),c(0,*)) IF f(1,9) GT .05 then t(*,9)=TM_TEST(a(0,*),c(0,*)) ELSE t(*,9)=TM_TEST(a(0,*),c(0,*), /UNEQUAL) f(*,10)=FV_TEST(a(0,*),d(0,*)) IF f(1,10) GT .05 then t(*,10)=TM_TEST(a(0,*),d(0,*)) ELSE t(*,10)=TM_TEST(a(0,*),d(0,*)$ , /UNEQUAL) f(*,11)=FV_TEST(b(0,*),c(0,*)) IF f(1,11) GT .05 then t(*,11)=TM_TEST(b(0,*),c(0,*)) ELSE t(*,11)=TM_TEST(b(0,*),c(0,*)$ , /UNEQUAL) f(*,12)=FV_TEST(b(0,*),d(0,*)) IF f(1,12) GT .05 then t(*,12)=TM_TEST(b(0,*),d(0,*)) ELSE t(*,12)=TM_TEST(b(0,*),d(0,*)$ , /UNEQUAL) f(*,13)=FV_TEST(c(0,*),d(0,*)) IF f(1,13) GT .05 then t(*,13)=TM_TEST(c(0,*),d(0,*)) ELSE t(*,13)=TM_TEST(c(0,*),d(0,*)$ , /UNEQUAL) f(*,14)=FV_TEST(a(1,*),b(1,*)) IF f(1,14) GT .05 then t(*,14)=TM_TEST(a(1,*),b(1,*)) ELSE t(*,14)=TM_TEST(a(1,*),b(1,*)$ , /UNEQUAL) f(*,15)=FV_TEST(a(1,*),c(1,*)) IF f(1,15) GT .05 then t(*,15)=TM_TEST(a(1,*),c(1,*)) ELSE t(*,15)=TM_TEST(a(1,*),c(1,*)$ , /UNEQUAL) f(*,16)=FV_TEST(a(1,*),d(1,*)) IF f(1,16) GT .05 then t(*,16)=TM_TEST(a(1,*),d(1,*)) ELSE t(*,16)=TM_TEST(a(1,*),d(1,*)$ , /UNEQUAL) f(*,17)=FV_TEST(b(1,*),c(1,*)) IF f(1,17) GT .05 then t(*,17)=TM_TEST(b(1,*),c(1,*)) ELSE t(*,17)=TM_TEST(b(1,*),c(1,*)$ , /UNEQUAL)  129  f(*,18)=FV_TEST(b(1,*),d(1,*)) IF f(1,18) GT .05 then t(*,18)=TM_TEST(b(1,*),d(1,*)) ELSE t(*,18)=TM_TEST(b(1,*),d(1,*)$ , /UNEQUAL) f(*,19)=FV_TEST(c(1,*),d(1,*)) IF f(1,19) GT .05 then t(*,19)=TM_TEST(c(1,*),d(1,*)) ELSE t(*,19)=TM_TEST(c(1,*),d(1,*)$ , /UNEQUAL) f(*,20)=FV_TEST(a(2,*),b(2,*)) IF f(1,20) GT .05 then t(*,20)=TM_TEST(a(2,*),b(2,*)) ELSE t(*,20)=TM_TEST(a(2,*),b(2,*)$ , /UNEQUAL) f(*,21)=FV_TEST(a(2,*),c(2,*)) IF f(1,21) GT .05 then t(*,21)=TM_TEST(a(2,*),c(2,*)) ELSE t(*,21)=TM_TEST(a(2,*),c(2,*)$ , /UNEQUAL) f(*,22)=FV_TEST(a(2,*),d(2,*)) IF f(1,22) GT .05 then t(*,22)=TM_TEST(a(2,*),d(2,*)) ELSE t(*,22)=TM_TEST(a(2,*),d(2,*)$ , /UNEQUAL) f(*,23)=FV_TEST(b(2,*),c(2,*)) IF f(1,23) GT .05 then t(*,23)=TM_TEST(b(2,*),c(2,*)) ELSE t(*,23)=TM_TEST(b(2,*),c(2,*)$ , /UNEQUAL) f(*,24)=FV_TEST(b(2,*),d(2,*)) IF f(1,24) GT .05 then t(*,24)=TM_TEST(b(2,*),d(2,*)) ELSE t(*,24)=TM_TEST(b(2,*),d(2,*)$ , /UNEQUAL) f(*,25)=FV_TEST(c(2,*),d(2,*)) IF f(1,25) GT .05 then t(*,25)=TM_TEST(c(2,*),d(2,*)) ELSE t(*,25)=TM_TEST(c(2,*),d(2,*)$ , /UNEQUAL) ft=fltarr(4,26) ft(0:1,*)=f ft(2:3,*)=t print, ft  130  ;write content into a file openw, 1, path+'f_t_stat_event.csv' printf, 1, ft, format="(4(f13.9, ','))" close, 1 END  131  Appendix H.  IDL program pearsonstatsevents.pro  Used for calculating Wilcoxon rank-sum test results in chapter 3. path='C:\Documents and Settings\Hydrology\My Documents\MKRF-A41\Mote Data\Organized Data\paper data\events\' suff='.csv' file=strarr(4) file(0)='sep18_20_soil_pearson' file(1)='mar21_24_soil_pearson' file(2)='mar25_29_soil_pearson' file(3)='apr7_8_soil_pearson' ;read from files openr, unit, path+file(0)+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE a=dblarr(3,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,2 do begin temp11=temp1(j) a(j,i)=temp11 endfor endfor  132  close, unit FREE_Lun, unit openr, unit, path+file(1)+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE b=dblarr(3,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,2 do begin temp11=temp1(j) b(j,i)=temp11 endfor endfor close, unit FREE_Lun, unit openr, unit, path+file(2)+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1  133  ENDWHILE c=dblarr(3,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,2 do begin temp11=temp1(j) c(j,i)=temp11 endfor endfor close, unit FREE_Lun, unit openr, unit, path+file(3)+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE d=dblarr(3,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT)  134  for j=0,2 do begin temp11=temp1(j) d(j,i)=temp11 endfor endfor close, unit FREE_Lun, unit ;reform arrays a5=reform(a(0,*)) b5=reform(b(0,*)) c5=reform(c(0,*)) d5=reform(d(0,*)) a10=reform(a(1,*)) b10=reform(b(1,*)) c10=reform(c(1,*)) d10=reform(d(1,*)) a20=reform(a(2,*)) b20=reform(b(2,*)) c20=reform(c(2,*)) d20=reform(d(2,*)) ;f-test, then t-test combos t=dblarr(4,26) t(0:1,0)=RS_TEST(a5,a10, ux=ux, uy=uy) t(2:3,0)=[ux,uy] t(0:1,1)=RS_TEST(a10,a20, ux=ux, uy=uy) t(2:3,1)=[ux,uy] t(0:1,2)=RS_TEST(b5,b10, ux=ux, uy=uy) t(2:3,2)=[ux,uy] t(0:1,3)=RS_TEST(b10,b20, ux=ux, uy=uy) t(2:3,3)=[ux,uy] t(0:1,4)=RS_TEST(c5,c10, ux=ux, uy=uy) t(2:3,4)=[ux,uy] t(0:1,5)=RS_TEST(c10,c20, ux=ux, uy=uy) t(2:3,5)=[ux,uy]  135  t(0:1,6)=RS_TEST(d5,d10, ux=ux, uy=uy) t(2:3,6)=[ux,uy] t(0:1,7)=RS_TEST(d10,d20, ux=ux, uy=uy) t(2:3,7)=[ux,uy] t(0:1,8)=RS_TEST(a5,b5, ux=ux, uy=uy) t(2:3,8)=[ux,uy] t(0:1,9)=RS_TEST(a5,c5, ux=ux, uy=uy) t(2:3,9)=[ux,uy] t(0:1,10)=RS_TEST(a5,d5, ux=ux, uy=uy) t(2:3,10)=[ux,uy] t(0:1,11)=RS_TEST(b5,c5, ux=ux, uy=uy) t(2:3,11)=[ux,uy] t(0:1,12)=RS_TEST(b5,d5, ux=ux, uy=uy) t(2:3,12)=[ux,uy] t(0:1,13)=RS_TEST(c5,d5, ux=ux, uy=uy) t(2:3,13)=[ux,uy] t(0:1,14)=RS_TEST(a10,b10, ux=ux, uy=uy) t(2:3,14)=[ux,uy] t(0:1,15)=RS_TEST(a10,c10, ux=ux, uy=uy) t(2:3,15)=[ux,uy] t(0:1,16)=RS_TEST(a10,d10, ux=ux, uy=uy) t(2:3,16)=[ux,uy] t(0:1,17)=RS_TEST(b10,c10, ux=ux, uy=uy) t(2:3,17)=[ux,uy] t(0:1,18)=RS_TEST(b10,d10, ux=ux, uy=uy) t(2:3,18)=[ux,uy] t(0:1,19)=RS_TEST(c10,d10, ux=ux, uy=uy) t(2:3,19)=[ux,uy] t(0:1,20)=RS_TEST(a20,b20, ux=ux, uy=uy) t(2:3,20)=[ux,uy] t(0:1,21)=RS_TEST(a20,c20, ux=ux, uy=uy) t(2:3,21)=[ux,uy] t(0:1,22)=RS_TEST(a20,d20, ux=ux, uy=uy) t(2:3,22)=[ux,uy] t(0:1,23)=RS_TEST(b20,c20, ux=ux, uy=uy) t(2:3,23)=[ux,uy] t(0:1,24)=RS_TEST(b20,d20, ux=ux, uy=uy)  136  t(2:3,24)=[ux,uy] t(0:1,25)=RS_TEST(c20,d20, ux=ux, uy=uy) t(2:3,25)=[ux,uy] print, t ;write content into a file openw, 1, path+'wilco_stat_event.csv' printf, 1, t, format="(4(f20.10, ','))" close, 1 END  137  Appendix I.  Ensemble plots for events not included in the text of Chapter 3.  Soil Moisture (%)  5 random points  20 random points  70  70  70  60  60  60  50  50  50  40  40  40  30  30  30  3-21 0:00  Throughfall (mm)  10 random points  3-22 12:00  3-24 0:00  3-21 0:00  3-22 12:00  3-24 0:00  3-21 0:00  10  10  10  8  8  8  6  6  6  4  4  4  2  2  2  0 3-21 0:00  3-22 12:00  Figure I.1  3-24 0:00  0 3-21 0:00  3-22 12:00  3-24 0:00  0 3-21 0:00  3-22 12:00  3-24 0:00  3-22 12:00  3-24 0:00  Ensemble plots for event B showing 100 different realizations at each  sample size.  Soil Moisture (%)  5 random points  10 random points  20 random points  70  70  70  60  60  60  50  50  50  40  40  40  30  30  30  8/1/06  8/8/06  Figure I.2  8/15/06 8/22/06 8/29/06  8/1/06  8/8/06  8/15/06 8/22/06 8/29/06  8/1/06  8/8/06  8/15/06 8/22/06 8/29/06  Ensemble plots for event E showing 100 different realizations at each  sample size. No throughfall is shown because no throughfall was measured during this time.  138  Soil Moisture (%)  5 random points  20 random points  70  70  70  60  60  60  50  50  50  40  40  40  30  30  30  11/1/06  Throughfall (mm)  10 random points  11/8/06 11/15/06 11/22/06 11/29/06  11/1/06  11/8/06 11/15/06 11/22/06 11/29/06  11/1/06  100  100  100  80  80  80  60  60  60  40  40  40  20  20  20  0 11/1/06  11/8/06 11/15/06 11/22/06 11/29/06  Figure I.3  0 11/1/06  11/8/06 11/15/06 11/22/06 11/29/06  0 11/1/06  11/8/06 11/15/06 11/22/06 11/29/06  11/8/06 11/15/06 11/22/06 11/29/06  Ensemble plots for event E showing 100 different realizations at each  sample size.  139  Appendix J.  Specific test results not shown in the text of chapter 3.  Table J.1 Soil Moisture F and T test results Events A - 5 pt A - 10 pt A - 10 pt A - 20 pt B - 5 pt B - 10 pt B - 10 pt B - 20 pt C - 5 pt C - 10 pt C - 10 pt C - 20 pt D - 5 pt D - 10 pt D - 10 pt D - 20 pt A - 5 pt B - 5 pt A - 5 pt C - 5 pt A - 5 pt D - 5 pt B - 5 pt C - 5 pt B - 5 pt D - 5 pt C - 5 pt D - 5 pt A - 10 pt B - 10 pt A - 10 pt C - 10 pt A - 10 pt D - 10 pt B - 10 pt C - 10 pt B - 10 pt D - 10 pt C - 10 pt D - 10 pt A - 20 pt B - 20 pt A - 20 pt C - 20 pt A - 20 pt D - 20 pt B - 20 pt C - 20 pt B - 20 pt D - 20 pt C - 20 pt D - 20 pt Full Months E - 5 pt E - 10 pt E - 10 pt E - 20 pt F - 5 pt F - 10 pt F - 10 pt F - 20 pt G - 5 pt G - 10 pt G - 10 pt G - 20 pt E - 5 pt F - 5 pt E - 5 pt G - 5 pt F - 5 pt G - 5 pt E - 10 pt F - 10 pt E - 10 pt G - 10 pt F - 10 pt G - 10 pt E - 20 pt F - 20 pt E - 20 pt G - 20 pt F - 20 pt G - 20 pt  F-stat 1.1533 1.0111 1.0364 1.4139 1.2936 1.0031 1.4355 1.0177 1.2735 1.0834 2.3916 1.3797 3.0457 2.2076 1.4171 1.2151 1.4445 1.7220 2.0471 1.1888 1.9817 1.2248 1.4864 2.4273 2.9457 1.2136 F-stat 1.6515 1.0754 1.0589 1.4460 1.2981 1.1192 1.5921 1.9371 1.2167 1.0208 1.5226 1.4916 1.3725 1.2651 1.0850  P 0.4792 0.9564 0.8590 0.0864 0.2021 0.9879 0.0736 0.9305 0.2308 0.6912 0.0000 0.1110 0.0000 0.0001 0.0844 0.3340 0.0688 0.0073 0.0004 0.3911 0.0008 0.3146 0.0499 0.0000 0.0000 0.3371 P 0.0133 0.7183 0.7766 0.0680 0.1961 0.5765 0.0216 0.0011 0.3308 0.9188 0.0377 0.0480 0.1169 0.2439 0.6858  T-stat 3.7708 4.6763 5.0922 5.3513 4.9563 5.6789 1.8042 3.9683 -5.1703 -3.3033 0.8725 1.5657 4.7774 3.4197 -4.2794 -1.5570 -0.1740 2.3606 3.6260 1.2541 -5.1730 -0.0447 -0.0589 4.7743 4.4311 -0.0159 T-stat 2.4363 8.1884 5.8458 7.9605 3.2975 7.2196 -6.8098 -3.9114 3.6203 -4.8552 -4.7433 0.5674 -6.2777 -6.9466 -0.8450  P 0.0002 0.0000 0.0000 0.0000 0.0000 0.0000 0.0727 0.0001 0.0000 0.0011 0.3842 0.1190 0.0000 0.0008 0.0000 0.1211 0.8620 0.0193 0.0004 0.2113 0.0000 0.9644 0.9531 0.0000 0.0000 0.9872 P 0.0158 0.0000 0.0000 0.0000 0.0012 0.0000 0.0000 0.0001 0.0004 0.0000 0.0000 0.5711 0.0000 0.0000 0.3991  140  Table J.2  Soil Moisture Wilcoxon Rank-Sum Test Results for Pearson r Events A - 5 pt A - 10 pt A - 10 pt A - 20 pt B - 5 pt B - 10 pt B - 10 pt B - 20 pt C - 5 pt C - 10 pt C - 10 pt C - 20 pt D - 5 pt D - 10 pt D - 10 pt D - 20 pt A - 5 pt B - 5 pt A - 5 pt C - 5 pt A - 5 pt D - 5 pt B - 5 pt C - 5 pt B - 5 pt D - 5 pt C - 5 pt D - 5 pt A - 10 pt B - 10 pt A - 10 pt C - 10 pt A - 10 pt D - 10 pt B - 10 pt C - 10 pt B - 10 pt D - 10 pt C - 10 pt D - 10 pt A - 20 pt B - 20 pt A - 20 pt C - 20 pt A - 20 pt D - 20 pt B - 20 pt C - 20 pt B - 20 pt D - 20 pt C - 20 pt D - 20 pt Full Months E - 5 pt E - 10 pt E - 10 pt E - 20 pt F - 5 pt F - 10 pt F - 10 pt F - 20 pt G - 5 pt G - 10 pt G - 10 pt G - 20 pt E - 5 pt F - 5 pt E - 5 pt G - 5 pt F - 5 pt G - 5 pt E - 10 pt F - 10 pt E - 10 pt G - 10 pt F - 10 pt G - 10 pt E - 20 pt F - 20 pt E - 20 pt G - 20 pt F - 20 pt G - 20 pt  Z stat 2.7451 7.6930 2.7122 5.0322 4.9711 6.2416 6.7486 6.7865 -7.4206 -6.6643 -11.8651 1.5858 -7.6344 -10.0045 -8.4285 -7.1665 -11.9555 3.1495 -8.3747 -11.2042 -9.9092 -9.6331 -12.1607 2.6645 -9.4889 -11.5511 Z stat -0.2052 4.4946 4.2026 6.7560 5.1470 6.6008 1.1117 1.3316 0.6157 5.8092 6.7193 0.9969 7.5941 7.7724 0.5742  P (Equal Mean) 0.0030 0.0000 0.0033 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0564 0.0000 0.0000 0.0000 0.0000 0.0000 0.0008 0.0000 0.0000 0.0000 0.0000 0.0000 0.0039 0.0000 0.0000 P (Equal Mean) 0.4187 0.0000 0.0000 0.0000 0.0000 0.0000 0.1331 0.0915 0.2690 0.0000 0.0000 0.1594 0.0000 0.0000 0.2829  Ux 6123.5 8148.5 6110.0 7059.5 7034.5 7554.5 7762.0 7777.5 1963.0 2272.5 144.0 5649.0 1875.5 905.5 1550.5 2067.0 107.0 6289.0 1572.5 414.5 944.5 1057.5 23.0 6090.5 1116.5 272.5 Ux 4916.0 6839.5 6720.0 7765.0 7106.5 7701.5 5455.0 5545.0 5252.0 7377.5 7750.0 5408.0 8108.0 8181.0 5235.0  Uy 3876.5 1851.5 3890.0 2940.5 2965.5 2445.5 2238.0 2222.5 8037.0 7727.5 9856.0 4351.0 8124.5 9094.5 8449.5 7933.0 9893.0 3711.0 8427.5 9585.5 9055.5 8942.5 9977.0 3909.5 8883.5 9727.5 Uy 5084.0 3160.5 3280.0 2235.0 2893.5 2298.5 4545.0 4455.0 4748.0 2622.5 2250.0 4592.0 1892.0 1819.0 4765.0  141  Table J.3  Throughfall Test Results (RMSE) Events A - 5 pt A - 10 pt A - 10 pt A - 20 pt B - 5 pt B - 10 pt B - 10 pt B - 20 pt D - 5 pt D - 10 pt D - 10 pt D - 20 pt A - 5 pt B - 5 pt A - 5 pt D - 5 pt B - 5 pt D - 5 pt A - 10 pt B - 10 pt A - 10 pt D - 10 pt B - 10 pt D - 10 pt A - 20 pt B - 20 pt A - 20 pt D - 20 pt B - 20 pt D - 20 pt Full Months F - 5 pt F - 10 pt F - 10 pt F - 20 pt G - 5 pt G - 10 pt G - 10 pt G - 20 pt F - 5 pt G - 5 pt F - 10 pt G - 10 pt F - 20 pt G - 20 pt  F-stat 1.2639 1.3142 1.0717 1.1250 1.1188 1.0135 1.8961 1.0488 1.8078 1.3997 1.0772 1.5077 1.0563 1.3969 1.3224 F-stat 1.0866 1.0765 1.4102 1.1058 2.0018 1.3064 1.3420  P 0.2456 0.1758 0.7310 0.5590 0.5777 0.9471 0.0016 0.8130 0.0035 0.0960 0.7122 0.0423 0.7858 0.0980 0.1662 P 0.6802 0.7145 0.0889 0.6177 0.0006 0.1853 0.1450  T-stat 4.9977 4.5499 5.5938 7.6349 4.8238 5.3830 -0.5188 18.3332 21.5832 -1.2731 19.8266 22.5937 1.3612 22.5196 21.0134 T-stat 7.2539 6.9363 6.4643 6.8026 -0.6376 0.2363 1.0372  P 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.6045 0.0000 0.0000 0.2045 0.0000 0.0000 0.1750 0.0000 0.0000 P 0.0000 0.0000 0.0000 0.0000 0.5246 0.8135 0.3009  142  Table J.4  Throughfall Wilcoxon Rank-Sum Test Results for Pearson r Events A - 5 pt A - 10 pt A - 10 pt A - 20 pt B - 5 pt B - 10 pt B - 10 pt B - 20 pt D - 5 pt D - 10 pt D - 10 pt D - 20 pt A - 5 pt B - 5 pt A - 5 pt D - 5 pt B - 5 pt D - 5 pt A - 10 pt B - 10 pt A - 10 pt D - 10 pt B - 10 pt D - 10 pt A - 20 pt B - 20 pt A - 20 pt D - 20 pt B - 20 pt D - 20 pt Full Months F - 5 pt F - 10 pt F - 10 pt F - 20 pt G - 5 pt G - 10 pt G - 10 pt G - 20 pt F - 5 pt G - 5 pt F - 10 pt G - 10 pt F - 20 pt G - 20 pt  Z Stat 4.0072 5.1617 6.5825 6.9123 5.4048 6.5984 -2.1893 3.2228 6.0242 -1.7507 4.4897 7.0052 -2.7794 6.3956 9.5683 Z stat 4.3053 4.6815 6.0144 5.5538 -3.0640 -2.4898 -0.6903  P (Equal Mean) 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0143 0.0006 0.0000 0.0400 0.0000 0.0000 0.0027 0.0000 0.0000 P (Equal Mean) 0.0000 0.0000 0.0000 0.0000 0.0011 0.0064 0.2450  Ux 6640.0 7112.5 7694.0 7829.0 7212.0 7700.5 4104.0 6319.0 7465.5 4283.5 6837.5 7867.0 3862.5 7617.5 8916.0 Ux 6762.0 6916.0 7461.5 7273.0 3746.0 3981.0 4717.5  Uy 3360.0 2887.5 2306.0 2171.0 2788.0 2299.5 5896.0 3681.0 2534.5 5716.5 3162.5 2133.0 6137.5 2382.5 1084.0 Uy 3238.0 3084.0 2538.5 2727.0 6254.0 6019.0 5282.5  143  Appendix K.  IDL program distributiongraph.pro  Used for creating frequency distribution graphics in chapter 4. path='C:\Documents and Settings\Hydrology\My Documents\MKRF-A41\Mote Data\Organized Data\paper data\distribution\soil\' file='november_daily_soil' suff='.csv' ;read from files openr, unit, path+file+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE array=dblarr(43,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,42 do begin temp11=temp1(j) array(j,i)=temp11 endfor endfor close, unit FREE_Lun, unit disarray=dblarr(n_lines,52) disarray(0,1:*)=INDGEN(51)+25 disarray(1:*,0)=array(0,*) realvalue=dblarr(n_lines-1,2) ;transpose date  144  realvalue(*,0)=array(0,0:n_lines-2) for i=0, n_lines-2 do begin dummy1=Where(array(1:*,i) GE 0 and array(1:*,i) LE 100, count) norm=count norm=float(norm) for j=25,75 do begin dummy2 = Where(array(1:*,i) GE j and array(1:*,i) LT j+1, count) datum = count datum=FLOAT(datum) ;print, count If datum GT 0 then datum=datum/norm datum= float(datum) disarray(i+1,j-24)=datum endfor temp=WHERE(array(1:*,i) GE 0 AND array(1:*,i) LE 100, count) ;IF count GT 0 then begin MEANSM = mean(array(temp+1,i)) realvalue(i,1)=MEANSM endfor surf= disarray(1:*,1:*) x= disarray(1:*,0) y=REFORM(disarray(0,1:*)) surf= (1-surf) IIMAGE, surf, x, y, YTITLE = 'Soil Moisture %', XTITLE = 'Day', XTICKUNITS = "Days" iplot, realvalue(*,0), realvalue(*,1), OVERPLOT=1, THICK=3, COLOR=[255,0,0] ;;write content into a file ;openw, 1, path+file+'_distribution.csv' ;printf, 1, disarray, format="(31(f10.3, ','))" ;close, 1  END  145  Appendix L.  IDL program standardize.pro  Used for creating ranked persistence plots in Chapter 4. path='C:\Documents and Settings\Hydrology\My Documents\MKRF-A41\Mote Data\Organized Data\paper data\persistence\soil\' file='march_daily_soil' suff='.csv' ;read from files openr, unit, path+file+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE array=fltarr(43,n_lines) temp='' point_lun, unit, 0 ;skip_lun, unit, 1, /LINES for i=0, n_lines-1 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,42 do begin temp11=temp1(j) array(j,i)=temp11 endfor endfor close, unit FREE_Lun, unit ;Average of all collected values (True Value) avg=fltarr(3,n_lines) ;transpose date  146  avg(0,*)=array(0,0:n_lines-1) for n=0,n_lines-1 do begin temp=WHERE(array(1:*,n) GE 0 AND array(1:*,n) LE 100, count) IF count GT 0 then begin MEANSM = mean(array(temp+1,n)) SDSM = STDDEV(array(temp+1,n)) endif avg(1,n)=MEANSM avg(2,n)=SDSM endfor ;standardize the values for i=1, n_lines-1 do begin array(1:42,i)=(array(1:42,i) GT 0)*(array(1:42,i) LE 100)*(array(1:42,i)-AVG(1,i))/AVG(2,i) nondata=WHERE(array(*,i) EQ 0) array(nondata,i)= -9999 endfor ;get average values for measurement interval plotarray=fltarr(42,3) plotarray(*,0)=array(1:42,0) for i=0, 41 do begin good=WHERE(array(i+1,1:n_lines-1) NE -9999, count) ;print, count ;plotarray(i,*)=(count EQ 0)*-9999 IF count GT 1 then begin plotarray(i,1)=mean(array(i+1,good+1)) plotarray(i,2)=STDDEV(array(i+1,good+1)) endif else begin plotarray(i,1:2)=-9999 endelse endfor ;sort data  147  sorted=sort(plotarray(*,1)) plotarray(*,0)=plotarray(sorted,0) plotarray(*,1)=plotarray(sorted,1) plotarray(*,2)=plotarray(sorted,2) ;remove -9999 values good = WHERE(plotarray(*,1) NE -9999) newplotarray=fltarr(n_elements(good),3) newplotarray(*,0:2)=plotarray(good,0:2) name=FIX(newplotarray(*,0)) names=STRARR(n_elements(good)) names(*)=name(*)  ;write to file transplot=TRANSPOSE(newplotarray) outpath='C:\Documents and Settings\Hydrology\My Documents\MKRF-A41\Mote Data\Organized Data\paper data\persistence\soil\' outfile=file+'_standard.csv' openw, 51, outpath+outfile printf, 51, transplot, format="(3(f10.3, ','))" close, 51 END  148  Appendix M.  IDL program pointsoilrmse.pro  Used for creating files used in the spatial display of RMSE in Chapter 4. path='C:\Documents and Settings\Hydrology\My Documents\MKRF-A41\Mote Data\Organized Data\paper data\months\' file='march_daily_soil' suff='.csv' ;read from files openr, unit, path+file+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE array=dblarr(43,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,42 do begin temp11=temp1(j) array(j,i)=temp11 endfor endfor close, unit FREE_Lun, unit MEANSM=fltarr(n_lines-1) for n=0,n_lines-2 do begin temp=WHERE(array(1:*,n) GE 0 AND array(1:*,n) LE 100, count)  149  MEANSM(n) = mean(array(temp+1,n)) endfor ;Calculate Mean error for each point rmse= fltarr(42) dummy = 0 for i=1,42 do begin good=where(array(i,*) GE 0. and array(i,*) LE 100., count) If count GT 0 then rmse(i-1)=SQRT(MEAN((array(i,good)-meansm(good))^2)) dummy=FLOAT(dummy+count) endfor all = FLOAT(n_elements(array(1:*,*))) print, dummy/all sorted=sort(rmse) rmse=rmse(sorted) use=where(rmse gt 0) rmse2=rmse(use) ;;write content into a file openw, 2, path+file+'_rmsepoints.csv' printf, 2, rmse2, format="(f10.4, ',')" close, 2 END  150  Appendix N.  IDL program pointsoilpearson.pro  Used for creating files needed for spatial display of Pearson r values in chapter 4. path='C:\Documents and Settings\Hydrology\My Documents\MKRF-A41\Mote Data\Organized Data\paper data\months\' file='august_daily_soil' suff='.csv' ;read from files openr, unit, path+file+suff, /get_lun n_lines=0. dummy='' WHILE ~ EOF(unit) DO BEGIN ; Read a line of text: READF, unit, dummy n_lines=n_lines+1 ENDWHILE array=dblarr(43,n_lines-1) temp='' point_lun, unit, 0 skip_lun, unit, 1, /LINES for i=0, n_lines-2 do begin READF, unit, temp temp1 = STRSPLIT(temp, ',', /EXTRACT) for j=0,42 do begin temp11=temp1(j) array(j,i)=temp11 endfor endfor close, unit FREE_Lun, unit MEANSM=fltarr(n_lines-1) for n=0,n_lines-2 do begin temp=WHERE(array(1:*,n) GE 0 AND array(1:*,n) LE 100, count) MEANSM(n) = mean(array(temp+1,n))  151  endfor ;Calculate Mean error for each realization pearson= fltarr(42,2) pearson(*,0)=INDGEN(42)+2 dummy = 0 for i=1,42 do begin good=where(array(i,*) GE 0. and array(i,*) LE 100., count) If count GT 0 then pearson(i-1,1)=correlate(MEANSM(good), array(i,good)) dummy=FLOAT(dummy+count) endfor all = FLOAT(n_elements(array(1:*,*))) print, dummy/all pearson2=transpose(pearson) ;sorted=sort(pearson) ;pearson=pearson(sorted) ;use=where(pearson gt 0) ;pearson2=pearson(use) ;write content into a file openw, 2, 'C:\Documents and Settings\Hydrology\My Documents\MKRFA41\surfer\thesis3\r_persistence\'+file+'_r_pers.csv' printf, 2, pearson2, format="(2(f10.4, ','))" close, 2 END  152  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            data-media="{[{embed.selectedMedia}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.24.1-0066329/manifest

Comment

Related Items