UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Shannon’s information theory in hydrologic network design and estimation Husain, Tahir 1979

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1979_A1 H98.pdf [ 7.89MB ]
Metadata
JSON: 831-1.0062559.json
JSON-LD: 831-1.0062559-ld.json
RDF/XML (Pretty): 831-1.0062559-rdf.xml
RDF/JSON: 831-1.0062559-rdf.json
Turtle: 831-1.0062559-turtle.txt
N-Triples: 831-1.0062559-rdf-ntriples.txt
Original Record: 831-1.0062559-source.json
Full Text
831-1.0062559-fulltext.txt
Citation
831-1.0062559.ris

Full Text

SHANNON'S INFORMATION THEORY IN HYDROLOGIC NETWORK DESIGN AND ESTIMATION by TAHIR HUSAIN B.Sc. Engg. (Hons.) ( C i v i l ) , A l i g a r h Muslim University, India, 1969 M.Eng. (Systems Eng. and Management), Asian I n s t i t u t e of Technology, Thailand, 1972 UNESCO Postgraduate C e r t i f i c a t e (Water Resources Eng.), Research I n s t i t u t e f o r Water Resources Development, Hungary, 1975 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY i n THE FACULTY OF GRADUATE STUDIES DEPARTMENT OF CIVIL ENGINEERING We accept t h i s thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA A p r i l 1979 0 Tahir Husain, 1979 In p r e s e n t i n g t h i s t h e s i s i n p a r t i a l f u l f i l m e n t o f the r e q u i r e m e n t s f o r an advanced degree a t the U n i v e r s i t y of B r i t i s h C o l u m b i a , I agree t h a t the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r r e f e r e n c e and s t u d y . I f u r t h e r agree t h a t p e r m i s s i o n f o r e x t e n s i v e c o p y i n g o f t h i s t h e s i s f o r s c h o l a r l y purposes may be g r a n t e d by the Head of my Department o r by h i s r e p r e s e n t a t i v e s . I t i s u n d e r s t o o d t h a t c o p y i n g o r p u b l i c a t i o n of t h i s t h e s i s f o r f i n a n c i a l g a i n s h a l l not be a l l o w e d w i t h o u t my w r i t t e n p e r m i s s i o n . (Tahir Husain) Department o f C i v i l Engineering The U n i v e r s i t y o f B r i t i s h Columbia 2075 Wesbrook P l a c e V a n c o u v e r , Canada V6T 1W5 Date A p r i l 6, 1979 (i) SHANNON'S INFORMATION THEORY IN HYDROLOGIC NETWORK DESIGN AND ESTIMATION ABSTRACT The hydrologic basin and i t s data c o l l e c t i o n network i s treated as a communication system. The s p a t i a l and temporal c h a r a c t e r i s t i c s of the hydrologic events throughout the basin are represented as a message source and t h i s message i s transmitted by the network stations to a data base. A measure of the basin information transmitted by the hydrologic network i s derived using Shannon's multivariate information. An optimum network s t a t i o n s e l e c t i o n c r i t e r i o n , based on Shannon's methodology, i s established and i s shown to be independent of the estimation of the events at ungauged l o c a t i o n s . M u l t i v a r i a t e information transmission f o r the hydrologic network i s i n i t i a l l y computed using the di s c r e t e entropy concept. The computa-t i o n of the mu l t i v a r i a t e entropy i s then extended to the case of va r i a b l e s represented by continuous d i s t r i b u t i o n s . B i v a r i a t e and multivariate forms of the normal and lognormal d i s t r i b u t i o n s and the b i v a r i a t e form of gamma, extreme value and exponential p r o b a b i l i t y density functions are considered. Computational requirements are substantial when dealing with large numbers of g r i d points i n the basin representation, and i n the combin-a t o r i a l search for optimum networks. Computational aids are developed which reduce the computational load to a p r a c t i c a l l e v e l . The performance of optimal information transmission networks i s compared with networks designed by e x i s t i n g methods. The a b i l i t y of Shannon's theory to cope with the multivariate nature of the output from a network i s shown to provide network designs with generally superior estimation performance. Although the optimal information transmission c r i t e r i o n avoids the necessity of specifying the estimators for events at ungauged locati o n s , the c r i t e r i o n can also be applied to the determination of optimal estimators. The a p p l i c a b i l i t y of the information transmission c r i t e r i o n i n determining optimal estimation parameters i s demonstrated for simple and multiple l i n e a r regression and Kalman f i l t e r estimation. Information transmission c r i t e r i o n i s applied to design the l e a s t cost network where a choice of instrument p r e c i s i o n e x i s t s . ( i i ) TABLE OF CONTENTS ABSTRACT (i) LIST OF TABLES (v) LIST OF FIGURES (x) NOTATION (xi) ACKNOWLEDGEMENTS ( x i i i ) CHAPTER I INTRODUCTION 1 II A REVIEW OF HYDROLOGIC NETWORK DESIGN 6 2.1 Review of the E x i s t i n g Methods 6 2.2 Limitations of the E x i s t i n g Methods 12 III SHANNON'S INFORMATION THEORY AND HYDROLOGIC NETWORK DESIGN 15 3.1 Hydrologic Network as a Communciation System . . . 15 3.2 Entropy, Information and Uncertainty 16 3.3 A Measure of Information Transmitted 17 3.4 Information Transmitted by a Hydrologic Network . . 22 3.4.1 Transmission of information provided by a temporary network 23 3.4.2 Transmission of hydrologic information throughout the region 26 3.5 Hydrologic Network Design for Optimum Information Transmission 29 3.6 Implications of the Optimum Information Transmission 30 3.7 Relative Measures of Network Performance and Size 31 IV NETWORK ANALYSIS BY INFORMATION THEORY 34 4.1 Ap p l i c a t i o n of the Optimum Information Transmission C r i t e r i o n 34 4.1.1 C o r r e l a t i o n matrix approach 34 4.1.2 Mu l t i v a r i a t e synthetic approach 34 ( i i i ) 4.2 Analysis 37 4.2.1 M u l t i v a r i a t e c o r r e l a t i o n matrix approach . 37 4.2.2 Mul t i v a r i a t e synthetic approach 39 4.2.3 Lower Mainland region of B.C 40 4.2.4 I n t e r i o r B r i t i s h Columbia region 41 4.3 Further Discussions on Results 42 V COMPUTATIONAL AIDS 59 5.1 Introduction 59 5.2 Elimination Based on the Entropy of the Individual Station 60 5.3 Elim i n a t i o n Based on the J o i n t Entropies of the Pairs Point Locations 61 5.4 Dynamic Elimination Based on Station Persistence . 62 5.5 Comparison of the Elimination Techniques 63 5.6 Washita River Watershed, Chickasha, Oklahoma . . . 66 VI DERIVATIONS OF INFORMATION TRANSMISSION FROM MULTIVARIATE CONTINUOUS DISTRIBUTIONS . 83 6.1 Univariate Entropy Derivations for Continuous D i s t r i b u t i o n s 83 6.2 Information Transmission at the B i v a r i a t e Level. . 85 6.2.1 Normal d i s t r i b u t i o n 85 6.2.2 Lognormal d i s t r i b u t i o n 86 6.2.3 Gamma d i s t r i b u t i o n 87 6.2.4 Exponential d i s t r i b u t i o n 88 6.2.5 Extreme value d i s t r i b u t i o n 89 6.3 Information Transmission at the Mul t i v a r i a t e Level 91 6.3.1 Normal d i s t r i b u t i o n 91 6.3.2 Lognormal d i s t r i b u t i o n . 93 VII COMPARISON OF SHANNON'S AND FISHER'S INFORMATION MEASURES IN THEORY AND NETWORK DESIGN APPLICATIONS . . . 95 7.1 Comparative Study 95 7.1.1 Bi v a r i a t e case 95 7.1.2 Mul t i v a r i a t e case 98 (iv) 7.2 Estimation Performance C r i t e r i a 100 7.2.1 Multiple l i n e a r regression 100 7.2.2 Estimation by Kalman f i l t e r model 100 7.3 Analysis 101 7.3.1 Simulated data 101 7.3.2 Daily p r e c i p i t a t i o n data from Lower Mainland region 102 7.4 Conclusions 103 VIII ESTIMATION AND INFORMATION TRANSMISSION 116 8.1 Introduction 116 8.2 General Estimation and Information Transmission . 117 8.3 Parameter Estimation f o r a Multi p l e Linear Regression Model Using Information Transmission C r i t e r i o n 120 8.4 Parameter Estimation f o r a Kalman F i l t e r Model . . 122 IX INFORMATION TRANSMISSION AND NETWORK COST 126 9.1 Introdution 126 9.2 Data Simulation Based on Instrument Accuracy . . . 127 9.3 Theoretical Development and Analysis 128 X SUMMARY AND CONCLUSIONS 135 BIBLIOGRAPHY 137 APPENDIX 14 3 (v) LIST OF TABLES Table No. T i t l e Page 4.1a C o r r e l a t i o n Matrix f o r a Low Correlated Simulated Basin 44 4.1b Optimum Information Transmission and the Optimal Station Locations i n a Simulated Low Correlated Basin 44 4.2a C o r r e l a t i o n Matrix f o r a Simulated Highly Correlated Basin 45 4.2b Optimum Information Transmission and the Optimal Station Locations i n a Simulated High Correlated Basin 45 4.3a Co r r e l a t i o n Matrix f o r a Typical Simulated Basin Using a Mu l t i v a r i a t e Synthetic Approach 46 4.3b Optimum Information Transmission and the Optimal Station Locations i n a Typical Simulated Basin 46 4.4 Description of the P r e c i p i t a t i o n Gauges i n Lower Mainland 47 4.5a Optimum Information Transmission and the Optimal Station Locations i n Lower Mainland - Subregion I 48 4.5b Optimum Information Transmission and the Optimal Station Locations i n Lower Mainland - Subregion II 48 4.5c Optimum Information Transmission and the Optimal Station Locations i n Lower Mainland - Subregion III 49 4.6 Description of the P r e c i p i t a t i o n Gauges i n I n t e r i o r B.C. Region 50 4.7 Optimum Information Transmission and the Optimal S t a t i o n Locations i n I n t e r i o r B.C. Region 51 5.1 V a l i d i t y of Elimination Technique Based on the Univariate Entropy Concept -Weakly Correlated Simulated Region 68 (vi) Table No. T i t l e Page 5.2a V a l i d i t y of Elimination Technique Based on the Univariate Entropy Concept -Highly Correlated Simulated Region 69 5.2b V a l i d i t y of Elimination Technique Based on the B i v a r i a t e Entropy Concept -Highly Correlated Simulated Region 69 5.3a V a l i d i t y of Elimination Technique Based on the Univariate Entropy Concept -I n t e r i o r B.C. Region 70 5.3b V a l i d i t y of Elimination Technique Based on the B i v a r i a t e Entropy Concept -I n t e r i o r B.C. Region 70 5.4a V a l i d i t y of Elimination Technique Based on the Univariate Entropy Concept -Lower Mainland Subregion III 71 5.4b V a l i d i t y of Elimination Technique Based on the B i v a r i a t e Entropy Concept -Lower Mainland Subregion III 71 5.4c Optimum Information Transmission and the Optimal Station Locations i n Lower Mainland (Elimination Method Based on Single and B i v a r i a t e Entropy Concept: A l l 25 Locations) 72 5.4d Optimum Information Transmission and the Optimal Station Locations i n Lower Mainland Region (Selection Based on Dynamic Elimination Method: A l l 25 Locations) . . 72 5.5 Raingauge Network Station Data, Chickasha, Oklahoma 73 5.6a Optimum Information Transmission and the Optimal Station Locations i n Chickasha, Oklahoma - Subregion I (Selection Based on B i v a r i a t e Entropy Concept) 77 5.6b Optimum Information Transmission and the Optimal Station Locations i n Chickasha, Oklahoma - Subregion I (Selection Based on Dynamic Elim i n a t i o n Method) 77 5.7a Optimum Information Transmission and the Optimal Station Locations i n Chickasha, Oklahoma - Subregion II (Selection Based on B i v a r i a t e Entropy Concept) 78 ( v i i ) Table No. T i t l e Page 5.7b Optimum Information Transmission and the Optimal Station Locations i n Chickasha, Oklahoma - Subregion II (Selection Based on Dynamic Elim i n a t i o n Method) 78 5.8a Optimum Information Transmission and the Optimal Station Locations i n Chickasha, Oklahoma - Subregion III (Selection Based on B i v a r i a t e Entropy Concept) 79 5.8b Optimum Information Transmission and the Optimal Sta t i o n Locations i n Chickasha, Oklahoma 9 Subregion III (Selection Based on Dynamic Elimination Method) 79 5.9a Optimum Information Transmission and the Optimal Station Locations i n Chickasha, Oklahoma - Subregion IV (Selection Based on B i v a r i a t e Entropy Concept) 80 5.9b Optimum Information Transmission and the Optimal Station Locations i n Chickasha, Oklahoma - Subregion IV (Selection Based on Dynamic Elimination Method). . 80 5.10a Optimum Information Transmission and the Optimal Station Locations i n Chickasha, Oklahoma - Subregion V (Selection Based on B i v a r i a t e Entropy Concept) 81 5.10b Optimum Information Transmission and the Optimal Station Locations i n Chickasha, Oklahoma - Subregion V (Selection Based on Dynamic Elimination Method) 81 6.1 Univariate Entropy f o r Some Common Continuous D i s t r i b u t i o n s 84 7.1a Co r r e l a t i o n Matrix of Simulated Data (300 Observations at 8 Locations) 105 7.1b Shannon's Information Matrix Using Discrete Entropy Concept (Based on 150 Observations) 105 7.1c Optimum Retained Stations and the Optimum Information Transmission 105 7.Id Fisher's Information Matrix (Based on 150 Observations at 8 Locations) 106 ( v i i i ) Table No. T i t l e Page 7.1e Optimum Retained Stations and the Optimum Information Transmission 106 7.If Comparison of Shannon and Fisher's Methodologies Using Estimation Error Method (Single Station Estimation) 107 7.1g Comparison of Shannon and Fisher's Methodologies Using Estimation Error Method (Two-Station Estimation) 107 7.2a Fisher's Information Matrix (Based on Real Observations Obtained from Lower Mainland Region: F i r s t Eight Stations with 350 Observations) 108 7.2b Optimum Retained Stations and the Optimum Information Transmission 108 7.2c Shannon's Information Matrix Using Discrete Entropy Concept (Based on Real Observations Obtained from Lower Mainland Region: F i r s t Eight Stations with 350 Observations) 109 7.2d Optimum Retained Stations and the Optimum Information Transmission 109 7.2e Comparison of Shannon and Fisher's Methodologies Using Estimation Error Method (One-Station Estimation) 110 7.2f Comparison of Shannon and Fisher's Methodologies Using Estimation Error Method (Two-Station Estimation) 110 7.2g Comparison of Shannon and Fisher's Methodologies Using Estimation Error Method (Three-Station Estimation) 110 7.3a Shannon's Information Matrix (Based on Real Observations Obtained from Lower Mainland Region: F i r s t Eight Stations with 350 Observations) (Bivariate Normal Distribution) I l l 7.3b Optimum Retained Stations and the Optimum Information Transmission I l l 7.3c Shannon's Information Matrix (Based on Real Observations Obtained from Lower Mainland Region: F i r s t Eight Stations with 350 Observations) (Bivariate Log-normal Distribution) 112 (ix) Table No. T i t l e Page 7.3d Optimum Retained Stations and the Optimum Information Transmission 112 7.3e Shannon's Information Matrix (Based on Real Observations Obtained from Lower Mainland Region: F i r s t Eight Stations with 350 Observations) (Bivariate Gamma Distribution) 113 7.3f Optimum Retained Stations and the Optimum Information Transmission 113 7.3g Comparison of Shannon and Fisher's Methodologies Using Estimation Er r o r Method (One-Station Estimation) 114 7.3h Comparison of Shannon and Fisher's Methodologies Using Estimation Error Method (Two-Station Estimation) 115 7.3i Comparison of Shannon and Fisher's Methodologies Using Estimation Error Method (Three-Station Estimation) 115 9.1 C o r r e l a t i o n Matrix of the Simulated Basin with Most Precise Instruments 130 (x) LIST OF FIGURES Figure No. T i t l e Page 3.1a G r i d Point Representation of Basin Hydrology . . . . 32 3.1b Conventional Representation of Communication System (After Shannon 1949) 32 3.1c Hydrologic Network as a Communication System . . . . 32 3.2 Cascaded Channel Communication System 33 4.1 Optimum Information Transmission and the Optimum Stati o n Selection i n a Poorly Correlated Simulated Region 52 4.2 Optimum Information Transmission and the Optimum Station Selection i n a Highly Correlated Simulated Region 53 4.3 Comparison of Information Transmission i n Highly Correlated and Poorly Correlated Simulated Region 54 4.4 Coordinates of Various Point Locations Used i n M u l t i v a r i a t e Synthetic Approach 55 4.5 Optimum Information Transmission and the Optimal Station Selection i n a Lower Mainland Region - Subregion III 56 4.6 Optimum Information Transmission and the Optimal Station S e l e c t i o n i n an I n t e r i o r B.C. Region 57 4.7 Comparison of Information Transmission and the Optimal Station Selection i n a I n t e r i o r B.C. Region 58 5.1 Raingauge Network Washita River Experimental Watershed, Chickasha, Oklahoma 82 9.1 Cost per Instrument Against Instrument Error C o e f f i c i e n t e, 131 k 9.2 Information Transmission by Various Optimum Statio n Sets from the Network of Various Precisions . 132 9.3 Cost of Network with Less Precise Instruments Equivalent to Networks Having Highly Precise Instruments 133 9.4 Cost Comparison Networks 134 (xi) NOTATIONS USED The following notations are frequently used: A^ Kalman estimation matrix ct + ^ Matrix i n t e r r e l a t i n g ungauged points to gauged points. C C o e f f i c i e n t of v a r i a t i o n of the var i a b l e s i n the ve e x i s t i n g network. C C o e f f i c i e n t of v a r i a t i o n of the variables desired vr ,_ . i n a network of N sta t i o n s . [P| Determinant of P ijj(q) Digamma function of q E( ) Expected value f ( ) Beta p r o b a b i l i t y density function B ^EXP ^ ^  Exponential p r o b a b i l i t y density function f „ v m ( ) Extreme value p r o b a b i l i t y density function EXT f ( ) Gamma p r o b a b i l i t y density function G f ( ) Lognormal p r o b a b i l i t y density function LN f^ ( ) Normal p r o b a b i l i t y density function F Matrix i n Kalman state-space equation T(p) Gamma function of p G Identity matrix i n state space equation H( ) Single and j o i n t entropy (I) 1 Inverse of £ 1^( ) Fisher's information y,a Respectively, s t a t i s t i c a l mean and standard deviation o) Random normal deviate P[x.] P r o b a b i l i t y of occurrence of the i t h outcome of the v a r i a b l e X ( x i i ) Posterior p r o b a b i l i t y or p r o b a b i l i t y of occurrence of the outcome x. given that the outcome y. has occurred. 1 2 Cross c o r r e l a t i o n c o e f f i c i e n t between i t h and j t h point i n the basin Variance covariance matrix of measurement error at time t The ensemble of the v a r i a b l e s representing the measured hydrologic events i n a basin The ensemble of n v a r i a b l e s representing the measured hydrologic events i n a basin at the i t h combination of n locati o n s The ensemble of v a r i a b l e s which describes the true events at very large number of locations i n the basin Estimation matrix used i n estimation equation Information transmitted by Y about X Information transmitted by s t a t i o n set about S Transpose Variance covariance matrix of state v a r i a l b e at time t=0 Random var i a b l e Discrete i t h outcome of the random v a r i a b l e X^_ State v a r i a b l e (single point) I n i t i a l state v a r i a b l e vector at t=0 Estimate of state v a r i a b l e vector X Measurement vector ( x i i i ) ACKNOWLEDGEMENTS The work described i n t h i s t h e s i s was conducted under the supervision of Dr. W.F. Caselton. The author wishes to thank him for h i s advice and guidance i n the research and for h i s continual i n t e r e s t and optimism. The author i s also indebted to Dr. S.O. Russe l l , Department of C i v i l Engineering; Dr. J.V. Zidek, Department of Mathematics and Applied S t a t i s t i c s ; and Dr. G.B. Anderson, Department of E l e c t r i c a l Engineering; for t h e i r valuable suggestions i n the course of t h i s study. Thanks go also to Mr. A.D. Nicks, Department of Agr i c u l t u r e , Chickasha, Oklahoma i n supplying p r e c i p i t a t i o n data from the experi-mental basin i n Oklahoma. The able assistance of Mr. A. Pipes, Department of C i v i l Engineering i n supplying data on Carnation Creek Watershed i s also highly approciated. Appreciation i s extended to the Canadian Commonwealth Scholarship and Fellowship Committee i n providing the f i n a n c i a l support for the enti r e period of study. Appreciation i s also extended to Ms. Ruth S t . C l a i r e f o r the excell e n t typing of the manuscript and to Mr. Richard Brun for the excell e n t d r a f t i n g . 1. CHAPTER I INTRODUCTION The problem of a t t a i n i n g an optimal balance between the economic r i s k a r i s i n g from inadequate information on the one hand and the economic costs of a hydrologic network capable of meeting a l l requirements f o r information on the other, has not yet been resolved on a r a t i o n a l and s c i e n t i f i c b a s i s . Although the hydrologic network system i s c l e a r l y a fundamental to o l i n the design, development, and operation of water resources projects, t h i s unresolved problem has been the p r i n c i p a l reason why no universal network design method has yet emerged. Without adequate hydrologic information, the uncertainty associated with the operation of a pro j e c t may r e s u l t i n serious d e f i c i e n c i e s i n performance. The f a i l u r e of many c a p i t a l intensive water resources projects throughout the world can be p a r t l y a t t r i b u t e d to the inadequacy of the hydrologic, meteorologic, and clima t a l o g i c information on which t h e i r designs were based (59). The hydrologic network i s an organized system f o r the c o l l e c t i o n of information of s p e c i f i c kinds, such as point p r e c i p i t a t i o n , runoff, sediment, et c . Other than hydrologic networks, there are environmental monitoring and meteorological networks but, from the standpoint of t h e i r s p a t i a l and temporal c h a r a c t e r i s t i c s , they present s i m i l a r network design problems. Much of the methodology i n t h i s t h e s i s i s therefore applicable to a number of d i f f e r e n t types of networks. The main  emphasis throughout the study i s on hydrologic networks. There are a number of fa c t o r s t r a d i t i o n a l l y considered i n designing a hydrologic network. The economic development of the region, population, p o t e n t i a l water use; and the physiographic and c l i m a t i c 2 . v a r i a b i l i t i e s of the region; are a few among the main contributing f a c t o r s . The density of the drainage channel network i s also a factor contributing towards the hydrologic complexity of a region. Due to the large physiographic and c l i m a t i c v a r i a t i o n s i n a mountainous region, the hydrologic network should be r e l a t i v e l y dense. For example, i n the mountainous regions of Bulgaria and Hungary, the network density i s as small as 100-200 km2 per s t a t i o n . This i s also due to the topographic p e c u l i a r i t i e s and intense economic developments i n the region. On the other hand, i n the mountainous areas of the USSR, an average network density of 500 to 1000 km2 per s t a t i o n i s considered reasonable. In the alpine region of the USSR, where extreme topographic v a r i a t i o n s e x i s t , a density of 300 to 500 km2 per s t a t i o n has been recommended (42). Judging the status of a hydrologic network on the basis of i t s density alone must be done with caution. But reviewing the network density at the global l e v e l does give a broad view of the extent of network development i n both developed and developing nations of the world. The world average network density i s 2650 km2 per s t a t i o n . I t v aries from 1000 km2 per s t a t i o n i n North America to 14,500 km2 per s t a t i o n i n the A f r i c a n continent. In other continents the network d e n s i t i e s are: Europe 1750 km2 per st a t i o n ; A u s t r a l i a 2600 km2 per stat i o n ; A s i a 3600 km2 per station; and South America 5000 km2 per s t a t i o n (76). The minimum network density f o r an i d e a l i z e d basin was proposed by Huff and Changnon (35) to be approximately 900 km2 per s t a t i o n . In comparison with t h i s minimum value, i t might be concluded that most of the nations i n the world are undergauged. 3. Depending upon the t e r r a i n c h a r a c t e r i s t i c s , the minimum raingauge d e n s i t i e s which have been recommended by WMO (76) are as follows: In f l a t regions of temperate, mediterranean and t r o p i c a l zones, the minimum density i s recommended as 600-900 km^ per s t a t i o n . In the mountainous regions, the density should be as high as 100-250 km2 per s t a t i o n . For small mountainous islands with the area l e s s than 20,000 km2, a network density of 25 km2 per s t a t i o n i s recommended. In the case of a r i d and polar zones, network density should be as much as 1500 to 10,000 km2 per s t a t i o n . These recommendations reveal the large v a r i a t i o n s of network density which are considered d e s i r a b l e . I t i s equally important to study the regional d i s t r i b u t i o n s of the hydrologic stations with a l t i t u d e s . In most countries having mountainous t e r r a i n s , there are higher density hydrologic networks at lower a l t i t u d e than at higher elevations. For example i n Korea, 85% of the land area i s below 500 m el e v a t i o n but approximately 98% of the stations are i n s t a l l e d below 500 m a l t i t u d e . In Switzerland, 17.9% of the t o t a l land area i s below 500 m a l t i t u d e but more than 30% of the stations are located i n t h i s area. With the high v a r i a b i l i t y i n the hydrologic data at higher a l t i t u d e s , i t could be argued that the network density should increase with a l t i t u d e . However, there are p r a c t i c a l d i f f i c u l t i e s i n l o c a t i n g instruments at higher elevations and, 4. due to the high operating cost and sparse or non-existent development, i t i s not often f e a s i b l e to locate a dense network at higher a l t i t u d e s . Although budgetary constraints are a prime cause of network inadequacy, i t i s quite probable that the absence of u n i v e r s a l l y recognized network design methods and performance c r i t e r i a a lso bear some r e s p o n s i b i l i t y . But j u s t i f i c a t i o n s or recommendations f o r the budgetary a l l o c a t i o n s to provide adequate hydrologic network density could be strengthened i f i t could be c l e a r l y demonstrated that measurable and acceptable information standards would be achieved. The problems a r i s i n g from inadequate networks are perhaps more prominant i n devel-oping countries but also continue to a r i s e i n the developed nations of the world. There are several ways of d e f i n i n g the hydrologic network design objective (31,59) but the fundamental theme i n most cases i s the s e l e c t i o n of the optimum number of the stations and t h e i r optimum lo c a t i o n s . Other considerations which can a r i s e i n network design concerns are: achieving adequate record length p r i o r to u t i l i z i n g the data, developing estimation models which u t i l i z e the network data to provide estimates of events at ungauged locat i o n s , and assessing the probable magnitudes of errors which can a r i s e i n any estimation dependent upon the data provided by the network. Chapter II gives a review of a number of e x i s t i n g hydrologic design methods. In Chapter I I I an informational basis of network performance i s presented and a c r i t e r i o n f o r s e l e c t i n g the optimum s t a t i o n s i t e s i n a network i s developed. Chapter IV deals with the t e s t i n g of the methodology developed using both simulated and r e a l d a i l y p r e c i p i t a t i o n data. In Chapter V and VI, combinatorial and 5. dimensional d i f f i c u l t i e s which do a r i s e i n computational phase are resolved to a p r a c t i c a l l e v e l . Chapter VII compares the estimation performance of the methodology with one of the e x i s t i n g methods. Based on the optimal information transmission c r i t e r i o n , an estimation model i s proposed to estimate runoff. This model i s discussed i n Chapter V I I I . A possible further refinement of the network design i s to supplement the primary network by the secondary networks from the economic point of view. This refinement i s discussed i n context of the network design methodology provided i n t h i s t hesis i n Chapter IX. 6. CHAPTER II A REVIEW OF HYDROLOGIC NETWORK DESIGN 2.1 A Review of the E x i s t i n g Methods A number of methods of hydrologic network design are c i t e d i n the l i t e r a t u r e . Some representative approaches are b r i e f l y described here. Hydrologic network design was i n i t i a l l y based on the r e l a t i o n s h i p of the mean square error, between the observed and estimated values of the hydrologic events, to the distance between the s t a t i o n points (16,23,32). This approach was f i r s t established by Horton (32), and l a t e r by Drozdov (16), for two s t a t i o n points. Ganguli et a l (23) generalized t h i s approach and proved that the number of stations i n a region would be inv e r s e l y proportional to the square of the c o e f f i c i e n t of v a r i a t i o n of the hydrologic v a r i a b l e s . The generalized form was expressed as: N r = N e - ( C v e / C v r > 2 ( 2 - D where N g i s the e x i s t i n g number of stations; C v e i s the c o e f f i c i e n t of v a r i a t i o n of the var i a b l e s i n the e x i s t i n g network; C v r i s the c o e f f i c i e n t of v a r i a t i o n desired i n a network of N r s t a t i o n s . A second approach was based on the inter-distance c o r r e l a t i o n r e l a t i o n s h i p . I t was assumed that the c o r r e l a t i o n between the hydrologic events at two stations decreased with the distance between the st a t i o n s . The decay function of the c o e f f i c i e n t of c o r r e l a t i o n between two stations 7. was assumed to be exponential (15,67). The r e l a t i v e e f f i c i e n c y was then obtained from the following r e l a t i o n s h i p : r i r = [ r ( x , t ) / p ( t ) ] 2 (2.2) where n i s the r e l a t i v e e f f i c i e n c y ; r ( x , t ) i s the c o r r e l a t i o n c o e f f i c i e n t for the i n t e r -s t a t i o n distance x at time t; p(t) i s the c o r r e l a t i o n c o e f f i c i e n t between two point l o c a t i o n s f o r very dense network at time t. The method of i s o c o r r e l a t i o n proposed by Hershfield (29) was also based on the i n t e r s t a t i o n c o r r e l a t i o n approach. The co r r e l a t i o n s between one or more key stations and a l l other stations i n a network were ca l c u l a t e d and l i n e s of equal c o r r e l a t i o n p l o t t e d around each of the key st a t i o n s . For storm r a i n f a l l , H e r s h f i e l d (29) selected a value f o r the c o e f f i c i e n t of c o r r e l a t i o n of 0.9, i n an a r b i t r a r y fashion, to determine the raingauge spacing. A t h i r d approach to network design, c a l l e d the simulation approach, was based on the improvement of the estimates of selected s t a t i s t i c a l parameters, such as the mean and variance of the hydrologic time seri e s data, by the use of primary and secondary stations (7,8,9,19,43,52). Primary stations, often r e f e r r e d to as time sampling stations, were used to a s c e r t a i n the hydrologic time seri e s r e l a t i o n s h i p s . Secondary stations, r e f e r r e d to as space sampling stations, were used f o r a short time to e s t a b l i s h s p a t i a l r e l a t i o n s h i p s . A decision could be made to ei t h e r continue or discontinue a s t a t i o n i n a region based on the requirement of the accuracy of the s t a t i s t i c a l estimates of the parameters (43). 8. A p a r a l l e l approach to the above was based on the synthesis of time ser i e s data by manipulating the auto-correlation and cross-c o r r e l a t i o n of h i s t o r i c a l data (20). I t was found that the length of the correlated s e r i e s would be much greater than the uncorrelated s e r i e s for the equivalent information content (43). Nick (57) also proposed a data generation scheme for an adequate estimate of the r a i n f a l l f o r a number of watersheds within a region. The estimation u t i l i z e d the a d d i t i o n a l information obtained by i n s t a l l i n g more p r e c i p i t a t i o n gauges on a temporary bas i s . A f t e r e s t a b l i s h i n g the p r i n c i p a l r e l a t i o n s h i p s , the network siz e could be reduced considerably. The r e g i o n a l i z a t i o n approach, an advancement i n mapping techniques by d i v i d i n g the whole area into square g r i d s , proposed by Solomon and others, i s also applicable to hydrologic network design (65,66). The proposed method was used to process the hydrologic information from a large area and was used to r e l a t e hydrologic variables to p h y s i c a l c h a r a c t e r i s t i c s . The square g r i d system was applied to estimate the runoff d i s t r i b u t i o n i n a large area using meteorologic and hydrologic informations. Estimates of the hydrologic parameters at the ungauged s i t e s could also be obtained on the basis of estimates of the parameters of r a i n f a l l and runoff at the gauged points. Another approach to network design has been named the r a t i o n a l approach and t h i s takes into account the demographic, economic, meteorologic and basin c h a r a c t e r i s t i c f a c t o r s (15,70). Multiple l i n e a r regression was used i n assessing the network performance on the basis of the estimation error at the ungauged points i n the basin (3). In ad d i t i o n to the above f a c t o r s , the drainage area could also be accounted for i n the multiple regression analysis (4). The variance of the 9. t e r r i t o r i a l mean of the hydrologic sampling was used as the basic c r i t e r i o n f o r determining the density of the p r e c i p i t a t i o n network r a t i o n a l l y (15). This approach i s empirical i n nature and i s not u n i v e r s a l l y a p p l i c a b l e . The l a t e s t development i n network planning i s the a p p l i c a t i o n of systems analysis and d e c i s i o n theory (12,14,36,50,53,64). Langbein has been able to formulate the network design problem and determined the number of primary as well as secondary stations by taking into account the desired accuracy and a budgetary cons t r a i n t (4 3) . The method proposed by F i e r i n g (18) i s si m i l a r to Langbein's except for the introduction of an objective function and deciding whether the stations are to be continued or not. The a p p l i c a b i l i t y of each of the approaches discussed above i s dependent upon fa c t o r s such as the nature and the extent of the resources development; the required p r e c i s i o n i n estimating the hydrologic c h a r a c t e r i s t i c s ; the physical s i z e of the basin, etc. Among these approaches, systems analysis and dec i s i o n theoretic approaches have been adopted i n the past due to t h e i r a b i l i t y to cope with the economic aspects of the network design, although i n a l i m i t e d manner. Attempts have been made to define the value of information i n s p e c i f i c s i t u a t i o n s but a generalized r e l a t i o n s h i p between the benefits and the costs of information has yet to be developed (25,63). A number of important contributions i n the economic context of the network design are described below. Jacobi (36) applied a dec i s i o n t h e o r e t i c a l approach to determine the optimum economic record lengths using the concept of the economic balance between the information value and the information cost. The 10. optimum economic record length was defined as a function of the si z e of the data sample. The methodology which u t i l i z e d a Bayesian decision framework, was then applied to a sediment deposition problem. The population mean and the variance of the annual sediment load was assumed to be known. The penalty function describing the consequences of underdesign or overdesign of sediment storage was dependent upon the true population mean and variance. F i n a l l y the expected opportunity l o s s was computed as a function of the record length. Decision theory has also been applied i n the optimum design of a mountainous raingauge network (12). A conceptual framework i n t e r -r e l a t i n g the uncertainty of a model parameter with the record length was established. Moss and Dawdy (53) attempted to improve t h i s methodology by combining i t with a Monte Carlo simulation. I n i t i a l l y the s t a t i s t i c a l properties were assumed to be known but, at a l a t e r stage, the condition of assumed s t a t i s t i c a l c h a r a c t e r i s t i c s was relaxed by incorporating a p r i o r d i s t r i b u t i o n on the unknown s t a t i s t i c s . Very recently Fisher's information c r i t e r i o n has been introduced i n the f i e l d of hydrologic network design (48). In the early 1920's Fisher defined the information content i n a sequence of observations as the r e c i p r o c a l of the variance of the estimates of the parameter of i n t e r e s t . Matalas (47) has proposed two basic approaches f o r designing the optimum gauging schemes i n a region, both u t i l i z i n g Fisher's information c r i t e r i o n . The f i r s t approach was based on the i d e n t i f i c a t i o n of those stations which were to be discontinued from a dense network due to budget curtailments. I t was based on the p r i n c i p l e of maximizing the t o t a l information content. The second approach was based on a marginal information concept. In t h i s case the stations i n 11. a region were discontinued i n such a manner that the decrease i n the value of Fisher's information was minimized. The information content of both s e r i a l l y c o r r e l a t e d and cross correlated time ser i e s hydrologic data, again based on Fisher's measure, was derived by Matalas and Langbein (48). I t was concluded that both s e r i a l and c r o s s - c o r r e l a t i o n tend to decrease the information content of the time s e r i e s data. An information t r a n s f e r c r i t e r i o n from gauged to ungauged points was discussed by W a l l i s and Matalas (72). The c h a r a c t e r i s t i c s of the data series were assumed to be Markovian. The model thus developed was an improvement over a previous model by Maddock (45) by r e l a x i n g the assumptions of l i n e a r i t y and stochastic independency through the a p p l i c a t i o n of a Monte Carlo technique. Maddock (45) and Karlinger (11) applied the above methodologies to the r e a l world s i t u a t i o n s using a mixed integer programming formulation along with a number of budgetary and information t r a n s f e r a b i l i t y c o n s t r a i n t s . The c r i t e r i o n of determining the expected information losses due to d i s c r e t i z a t i o n of a continuous stochastic process was discussed by Szollosi-Nagy (64) and Nielson (58). Nagy emphasized the information losses due to l i m i t e d sampling of hydrologic records i n time and space. He used Bayesian d e c i s i o n theory as the supporting mathematical t o o l . The p o s s i b i l i t y of extending Bayesian theory and i n t e r r e l a t i n g the expected information l o s s with the cost function f o r data c o l l e c t i o n was considered. 2.2 Limitations of the E x i s t i n g Methods The p r i n c i p a l l i m i t a t i o n s of the network design methods discussed above can be summarized as follows: 1. The approach based on the mean square error and i t s r e l a t i o n to i n t e r s t a t i o n distance i s a u s e f u l approach to the preliminary design of a network. However, i t cannot be used for the accurate determination of the network density since Equation (2.1) does not take into account the heterogeneity of the hydrologic v a r i a b l e s which ex i s t s i n mountainous regions. I t i s also d i f f i c u l t to decide upon the value of the required c o e f f i c i e n t of v a r i a t i o n i n Equation (2.1). Furthermore, t h i s approach i s not able to discriminate between primary and secondary stations i n a region. 2. In methods of i s o c o r r e l a t i o n and i n t e r - s t a t i o n distance c o r r e l a t i o n there i s no r u l e to provide the value of the r e l a t i v e e f f i c i e n c y and the c o r r e l a t i o n c o e f f i c i e n t for a very dense network. The term dense network i s i t s e l f a r e l a t i v e term which i s d i f f i c u l t to assess. Since the c o r r e l a t i o n c o e f f i c i e n t i s a function of time and space i t may vary from month to month, hence the stationary time series assumption may not be v a l i d . 3. The data synthesis and simulation approach to network design i s based on the assumption that the hydrologic parameters must be known. Hence the accuracy o f t h i s approach depends upon the extent of the p r i o r knowledge of the hydrologic parameters. During the network planning stage very l i t t l e p r i o r knowledge concerning these parameters are a v a i l a b l e and, as a consequence, the methodology i s not e f f e c t i v e . 4. The Bayesian d e c i s i o n approach accounts f o r the uncertainty i n the estimates o f the hydrologic parameters through the use of the j o i n t and c o n d i t i o n a l p r o b a b i l i t y d i s t r i b u t i o n of the parameters being estimated. Although i n a s i m p l i f i e d form i t i s a s t r a i g h t -forward approach, i t can only be used f o r s i n g l e time steps into the future to make the d e c i s i o n whether or not to c o l l e c t data. For multiple time step decisions, the mathematical complexity increases d r a s t i c a l l y due to the combinatorial nature of the problem. Hence the method i s too r e s t r i c t i v e f o r network design which needs more foresight than the methodology provides. 5. The concept of applying Fisher's information measure to the design of a hydrologic network i s based on optimizing the r e c i p r o c a l of the variance of the s t a t i s t i c a l parameters of the v a r i a b l e being estimated. One c r i t i c i s m of t h i s measure i s that Fisher's information content always has an i n f i n i t e upper bound regardless of the s p e c i f i c circumstances. In the context of hydrologic network design, Fisher's information r e l i e s upon l i n e a r i t y and normality assumptions. Each of the e x i s t i n g approaches to network design described above, address themselves to very s p e c i f i c problems and depend upon very r e s t r i c t i v e assumptions. A broader, more universal basis for the performance, evaluation, and design of s p a t i a l data c o l l e c t i o n networks i s needed. In the remaining chapters of t h i s thesis a methodology f o r the design of s p a t i a l data c o l l e c t i o n networks, which i s based on Shannon's information theory, i s proposed. As w i l l be shown, t h i s methodology does provide a u n i v e r s a l dimensionless network performance measure 14. which can also be extended to estimation on a p r a c t i c a l basis for the i design of permanent networks using data provided by dense temporary networks. 15. CHAPTER I I I SHANNON'S INFORMATION THEORY AND HYDROLOGIC NETWORK DESIGN 3.1 Hydrologic Network as a Communication System The concept of a hydrologic network as a communication system i s shown schematically i n Figure 3.1c. The hydrology of a region served by a network can be represented by a set of di s c r e t e v a r i a b l e s each of which describes the magnitude of hydrologic events occurring at one of a large number of point lo c a t i o n s dispersed throughout the region (Fig. 3.1a). This set of va r i a b l e s may be considered to form the hydrologic informa- t i o n or message source. The r o l e of the hydrologic network s t a t i o n s , which occupy only a small number of the point locations associated with the message source, i s to act c o l l e c t i v e l y as a transmitter of a l l of the hydrologic information. Thus the measurements at network s t a t i o n locations are considered to embody, i n some way, information on events at both gauged and ungauged lo c a t i o n s throughout the whole region. The magnitudes of the hydrologic events at the ungauged points are subsequently reconstituted using the network s t a t i o n measurements and some form of i n t e r p o l a t i v e (e.g. Theissen polygon method) or p r e d i c t i v e (e.g. S p a t i a l estimation) models. The combined network stations measurements and reconstituted values for ungauged locations represent the complete output signals which u l t i m a t e l y w i l l be a r e f l e c t i o n of the input message. The process of hydrologic data transmission from hydrologic information sources to the received signals i s analogous to encoding while the process of r e c o n s t i t u t i n g the data into an estimate of the o r i g i n a l message i s analogous to decoding. 16. The performance of the hydrologic network, i n the most general terms, i s r e f l e c t e d i n how much, on average, i t s output s i g n a l reduces uncertainty concerning the input message. C l e a r l y the number of permanent statio n s i s r e s t r i c t e d and we do not a n t i c i p a t e l o c a t i n g a s t a t i o n a t every point l o c a t i o n of i n t e r e s t i n the region. Therefore, i n p r a c t i c e , we are prepared to accept the information contained i n the output s i g n a l , c o n s i s t i n g of both measurements and estimates, which w i l l not eliminate a l l uncertainty concerning the regional hydrology. The output s i g n a l w i l l contain errors and these errors w i l l o r i g i n a t e i n both measurement and estimation. Regardless of how and where these errors a r i s e , the a b i l i t y of a network to convey information, and reduce uncertainty, can be assessed q u a n t i t a t i v e l y , as f o r any imperfect communication channel, by measuring the information transmitted, or mutual information, between input message and output s i g n a l . No t h e o r e t i c a l r e s t r i c t i o n w i l l be placed on the number of d i f f e r e n t types of events which might be considered at each hydrologic sampling point but, for s i m p l i c i t y , only one type of event, f o r example d a i l y point p r e c i p i t a t i o n , w i l l be considered i n t h i s study. 3.2 Entropy, Information and Uncertainty Consider a v a r i a b l e X which represents the magnitudes of the hydrologic events at some s i n g l e l o c a t i o n . The time ser i e s data for X w i l l be considered to be f i r s t d i s c r e t i z e d into m i n t e r v a l s . Assume that the p r o b a b i l i t y of occurrence of a hydrologic event i n the i t h i n t e r v a l i s denoted by P[x^]. N H(X) = -kX P[x.].log P[x.] (3.1) . , i i This average uncertainty i s also c a l l e d entropy. I t i s measured i n " b i t s " . The value of k i s unity i f the logarithm i s taken to the base two. I t i s a f i n i t e , non-negative quantity. I t i s used as a measure of uncertainty of the occurrence of the events. I t i s also used as a measure of disorder and r e f l e c t s the ignorance of the actual structure of the system. Entropy i s a us e f u l measure f o r ch a r a c t e r i z i n g the v a r i a b i l i t y i n the outcome of a random event. I t can also be used i n the form of error entropy to measure the p r e d i c t i v e performance of an estimation model i n a s i m i l a r fashion to the sum of the squares of the re s i d u a l e r r o r c r i t e r i o n . Due to i t s dimensionless character, the entropy concept i s more general than the variance approach (24). 3.3 A Measure of Information Transmitted The following i s intended to serve as a b r i e f introduction to c e r t a i n relevant aspects of information theory and considers only d i s c r e t e input output communication channels. A communication channel i s supplied with a s i n g l e random input X, which has p o s s i b l e d i s c r e t e values x^ : i = 1,..,N, and produces a random output Y with values y ; j = 1,...,M. P r o b a b i l i t y d i s t r i b u -tions f o r X and Y, namely P[x^] and Pty^], are known, together with t h e i r j o i n t p r o b a b i l i t i e s P[x^,y_.] . The information provided by y_. about x^ i s r e l a t e d to changing the p r o b a b i l i t y of x^ from the p r i o r value P[x^] to the p o s t e r i o r p r o b a b i l i t y P[x.jjy_.]. The amount of information T(x^;y.J supplied by output y_. about, the occurrence of the input event x^, as defined by Shannon (62), may be stated as: 18. T ( x ± ; y ) = log P t x . J y ] - log P[x ±] (3.2) The logarithmic base used i n t h i s d e f i n i t i o n w i l l determine the u n i t s of the information measure. If the base 2 i s used then the u n i t of informa-t i o n i s the " b i t " . One b i t i s equal to the information transmitted when the p o s t e r i o r p r o b a b i l i t y of an event i s increased by a f a c t o r of two following r e c e i p t of the output s i g n a l . I f base e i s used then the u n i t s are nats (natural units) and i f base 10 i s used then the units are decibels. Provided a consistent logarithmic base i s used, then the choice of u n i t s i s not c r i t i c a l i n the theory presented here. For convenience the base e was used for a l l the numerical r e s u l t s presented here. The amount of information transmitted under extreme conditions can e a s i l y be demonstrated. I f the channel output y i s completely independent of the input x^ then: P [ x i | y j ] = P[x ±] and the information transmitted i s : T(x±;y^) = log P[x i|y^] - log P[x ±] = log P t x ^ - log P[x i] = zero I f , on the other hand, output y. uniquely defines that the input \ 3 i s x. then: l P [ x . | y j ] = 1.0 and the information transmitted i s : 19. T(xi#-y_.) = log 1.0 - log P[x^] = -log Pfx^] Thus the quantity -log P[x^] i s the amount of information transmitted when a l l uncertainty concerning event x^ i s removed. A l t e r n a t i v e l y stated, i t i s the maximum amount of information which can be provided about x.. I t i s ref e r r e d to as the s e l f information of x.. Unlike i l Fisher's measure of information (see Chapter VII), which approaches i n f i n i t y as the parameter variance approaches zero, -log[P x^] provides a f i n i t e and predictable upper bound of information. As x^ i s only one of a number of possible input events then the average s e l f information for a l l input events X = x^ ; i = 1,...,N i s : -E P[x. ] log P[x.] = H(X) (3.3) i The quantity H(X) i s known as the entropy of X and i s the average uncertainty associated with the random input X. I t i s also a measure of the disp e r s i o n or v a r i a b i l i t y of X. The average information transmitted by the channel, per input output event over a l l input output events, T(X;Y) i s the sum of information transmitted by each p a i r of possible input and output events weighted by the p r o b a b i l i t y of t h e i r occurrence. T(X;Y) = Z Z P[x.,y.](log P[x.|y.] - l o g P[x.]) . . (3.4) i j 1 3 1 3 1 The quantity T(X,-Y) i s of c e n t r a l importance i n information theory and i s referred to as the average information transmission, or simply the mutual information, between X and Y. T(X;Y) can also be expressed more conveniently as a function of the entropies H(X) and H(Y) and the j o i n t entropy H(X,Y). 20. From Equation (3.4): P t x j y . ] T(X;Y) = Z £ P[x ,y ] log , , ± j F l X i J But, P[x ,y ] P[x. y.] = 3 i , J V P[y..] so that, P[x.,y.] T(X;Y) = Z Z P[x.,y ] log » - H P[x ±,y ] log P[x ±] - H P[x i,y ] log P[y ] i j i j + J X P[x.,y.] log P[x.y.] (3.5) ^ I D ID The entropy of a single v a r i a b l e was defined i n Equation (3.3) and, by extension, the j o i n t entropy of two v a r i a b l e s i s defined as: H(X,Y) = - Z E P [x . , y . ] log P [x . , y . ] (3.6) i j 1 3 1 3 Note that the j o i n t entropy H(X,Y) i s a combined measure of both the dispersion and the ass o c i a t i o n of the v a r i a b l e s X and Y. Thus, from Equation (3.6) i t follows that: T(X;Y) = H(X) + H (Y) - H(X,Y) (3.7) From Equation (3.4) i t follows also that: T(X;Y) = H(X) - H(X|Y) (3.8) where the c o n d i t i o n a l entropy i s defined as: H(X|Y) = - Z Z P[x ±,y ] l o g P[x ±|y ] (3.9) i j 21. I t can be seen from Equation (3.8) that transmitted information measures the reduction i n average uncertainty concerning X following r e c e i p t of the output s i g n a l Y. By expanding the c o n d i t i o n a l p r o b a b i l i t i e s to j o i n t and s i n g l e p r o b a b i l i t i e s , i t can be shown that the ordering of v a r i a b l e s does not a f f e c t the value of information transmission, i . e . T(X;Y) = T(Y;X) (3.10) Equation (3.4) does not, however, provide a measure of information transmission which conforms to the multiple v a r i a b l e input and output model. The theory has been extended to the evaluation of information transmitted by a communication channel involving v separate input channels and w separate output channels, where v i s not n e c e s s a r i l y equal to w. I f X^,X^,...,X^ represents the input v a r i a b l e s and Yl' Y2'"""' Yw r e P r e s e n t s the output v a r i a b l e s , then the information transmission i s given by: T(X,,X„,...,X ;Y.. ,Y,...,Y) I 2 v i z w = H(X.,X ,...,X ) + H(Y ,Y„,...Y) 1 2 V 1 2 W - H(X.,X,...,X ,Y 1 fY„,...,Y ) (3.11) 1 2 V 1 2 w where the j o i n t entropy terms f o r more than two v a r i a b l e s are t y p i c a l l y defined, by extension of Equation (3.6), as: H(X ,X ,...,X ) = -E E E P(x.,x.,...,x ) log P(x.,x.,...,x )..(3.12) ID q where x^,x..,..., and x^ represent d i s c r e t e outcomes of X^,X^, ... ,X and X_ r e s p e c t i v e l y . A l t e r n a t i v e l y the multivariate form of information transmission can be written i n i t s conditional form: T (X^, X2 t • • • i Y.^  , , . . . Y^) = H(X ,X ,...,X ) - H(X X„,...,X |Y ,Y ,...,Y ) (3.13) L Z V X Z V X z w Here again information transmitted can be seen to measure the reduction i n average uncertainty concerning the input v a r i a b l e X^  ,X2, .. . ,X^  following r e c e i p t of the output v a r i a b l e s Y^,Y^,...Y^. As f o r the s i n g l e input output case i t can be shown that: T(X, ,X, X ;Y..,...,Y) = T(Y,Y_,...,Y;X..,X,..., X ) 1 Z V J L W 1 Z w ± 2 V and that the ordering of v a r i a b l e s on e i t h e r side of the semi colon i s a r b i t r a r y . F i n a l l y , the following useful decomposition of m u l t i v a r i a t e information transmission i s provided by Gallager (22) . Let represent the array of v a r i a b l e s representing the input message, and l e t and represent two sub arrays of v a r i a b l e s which, combined, form the output s i g n a l . Then the information transmitted by U^, about i s given by: T(U i ;U 2,U 3) = T(U i ;U 2) + T(U X; U3|u ) (3.14) where the c o n d i t i o n a l information transmission i s defined by: P[u 1|u 2,u 3] T(U 1 ;U 2|U 3) = I I E log •• (3.15) U X U 2 U 3 1 1 l U 3 J where u j r u 2 , and u 3 represent outcomes of U^U^f and r e s p e c t i v e l y . 3.4 Information Transmitted by a Hydrologic Network Two cases w i l l be considered. The f i r s t case concerns the information which would be transmitted by measurements and estimates derived from an n s t a t i o n permanent network concerning events as measured by a temporary network with stations at m point locations i n the region. The permanent s t a t i o n v a r i a b l e set i s a subset of the m point v a r i a b l e set S. The second case concerns information transmitted by an n s t a t i o n permanent network concerning the true hydrology of the region represented by actual hydrologic events at a very large number k of locations throughout the region. The following notation i s adopted. The word ensemble i s used to describe a v a r i a b l e set but also implies the existence of p r o b a b i l i t i e s known or unknown, for a l l d i s c r e t e j o i n t outcomes of the v a r i a b l e s i n the set. m i s the number of point locations of a dense temporary network from which p r i o r data has been attained. n i s the number of stati o n s i n a permanent network and n<m. S i s the ensemble of m va r i a b l e s representing the measured hydrologic events. S? i s the ensemble of n va r i a b l e s representing the measured events at the i th combination of n locations . _ m! (m-n) !n! ¥ i s the ensemble of va r i a b l e s which describe true events at a very large number k of locations i n the region and k » m . 3.4.1 Transmission of information provided by a temporary network The measurements attained at the permanent s t a t i o n l o c a t i o n s , denoted by S? , w i l l be used to provide an estimate of events which have occurred at the remaining l o c a t i o n s . The estimator w i l l be i n the form of some (m-n) valued transformation F ( s n ) with no r e s t r i c t i o n 24. as to the mathematical form of F( ) other than i t should be an unbiased estimator [73] . Thus i n the communication channel representa-t i o n of the permanent network, the input message i s S and the output signal consists of and F(s") combined. The purpose of the network, i n informational terms, i s to reduce the average uncertainty concerning S following r e c e i p t of the output s i g n a l . This average reduction i n uncertainty concerning S i s measured by T(S; S^,F(S^)). From Equation (3.14) T(S; S n,F(S n) = T(S;S n) +T(S;F (S^ls?) (3.16) 1 1 l l ' l The mutual information T(S;S^) can be expanded using Equation (3.11) so that T(S;S") = H(S) + H(s") - H(S,s") (3.17) But d u p l i c a t i o n of v a r i a b l e s i n a j o i n t entropy term does not a f f e c t i t s value. As S1? represents a set of v a r i a b l e s which are a subset of l S then H(S,s") = H(S) and i t follows from Equation (3.17) that T(S;S n) = H(S n) (3.18) l l I f s and s^ represent t y p i c a l outcomes of S and then the condit i o n a l information transmission T(S;F(S.)|s n) i s defined by Equation (3.15) as P [ s | F ( s n ) , s n ] T(S;F(s n) |sn)=£ • - £ P[s,F(s n) |s n] log — . . (3.19) 1 1 s n n 1 1 P[s|s n] F(a) s i ' I But as s" w i l l always be a subset of s then, due to the redundancy i n the condition, P [ F ( s n ) l s n ] = P [ F ( s n ) | s n , s ] ( 3 . 2 0 ) 1 ' 1 l ' l M u l t i p l y i n g both sides of Equation ( 3 . 2 0 ) by P(s|s") y i e l d s P [ F ( s n ) | s n ] P [ s | s n ] = P [ F ( s n ) | s n , s ] P [ s | s n ] 1 ' X ' X 1 1 X = P[F(s"),s|s"] By d e f i n i t i o n so that r i , i P [ a b c] P t a l b c ] = T T b T c T P [ s , F ( s n ) | s n ] r I n. n. x x P[s F(s.)s.] = P[F(s.) s.J P[s|s^] iTo r n, = P[s S . ] X Thus, i n the log term of Equation ( 3 . 1 9 ) the numerator and denominator are always equal and the log term zero. Hence i t can be concluded that: T(S;F(S n) | s n ) = zero ( 3 . 2 1 ) x 1 x so that the c o n d i t i o n a l information transmission i s zero. Combining Eguations ( 3 . 1 6 ) , ( 3 . 1 8 ) , and ( 3 . 2 1 ) y i e l d s the r e s u l t T(S ; s" fF(S?)) = H ( s " ) ( 3 . 2 2 ) This r e s u l t i s s u r p r i s i n g as i t indicates that the estimation of events on non permanent st a t i o n l o c a t i o n s contributes no further reduction i n uncertainty concerning S than i s provided by the network output S? alone. I t should not be interpreted to mean that the estimation of events at non s t a t i o n l o c a t i o n s i s unnecessary or inappropriate but does confirm that estimation does not add further to the average amount of s t a t i s t i c a l data already conveyed by the permanent network s t a t i o n data. This r e s u l t i s s i g n i f i c a n t i n the context of designing hydrologic networks. 3.4.2 Transmission of hydrologic information throughout the region The true hydrology of a region can be represented accurately by an ensemble of v a r i a b l e s which describes the true magnitudes of point hydrologic events at a very large number k of point locations i n the region. Although e x i s t s i t i s not p r a c t i c a l l y possible to monitor events at t h i s very large number of l o c a t i o n s . As before, a temporary network provides measurement data at m l o c a t i o n s and a permanent network w i l l have stations at n of these l o c a t i o n s . The combined permanent s t a t i o n and estimation output w i l l be represented by . i . e . { = {S n, F, (S n)} (3.23a) where F^( ) i s a k-n valued transformation of S? and provides estimation of true hydrologic events at the k-n non s t a t i o n l o c a t i o n . Figure (3.2) shows how the s i t u a t i o n can be viewed as a cascaded channel communication system. Channel A output S provides l i m i t e d measurements on the regional hydrology while channel B u t i l i z e s S as i t s input message and provides an output s i g n a l ip. C l e a r l y the o v e r a l l performance of t h i s communication system i s r e f l e c t e d i n the average reduction i n uncertainty concerning following r e c e i p t of the output s i g n a l ^ and i s measured by the mutual information /\ F i r s t , consider the mutual information T(l^;S,ip). The order of the v a r i a b l e s following the semi colon does not a f f e c t the value of transmitted information, so that: T($,-S,Tj» = T($;I|),S) From Equation (3.14) i t follows that T ( J;S) + T($;1|;|S) = T($;lJ)) + T($;S |l(>) (3.23b) As was demonstrated i n the proof of Equation (3.21) when the information transmission by ensemble about i s co n d i t i o n a l upon a t h i r d ensemble and i s f u n c t i o n a l l y dependent upon or a subset of U^ , then the value of the transmitted information TdJ^;!^ (u^) i s zero. As shown i n Equation (3.23a), 4» i s f u n c t i o n a l l y dependent upon a subset of S, namely , so that i t can be proved, i n a s i m i l a r fashion to Equation (3.21) that T(ty;ty\S) = zero Hence, from Equation (3.23b) T(J;S) = T($;ljj) + T($;s|lp) (3.24) Although T(^;s|l^) cannot be quantified, because i s not d i r e c t l y observed, a s i m i l a r argument concerning the existence of a f u n c t i o n a l r e l a t i o n s h i p between S and can be made. S, which 28. represents measured hydrologic events, w i l l approximate to a subset of ijj by taking measurements of the true hydrologic events at the m locations of the temporary network. The approximation i s due e n t i r e l y to measurement errors which w i l l presumably be r e l a t i v e l y small and s t a t i s t i c a l l y independent. This suggests that T(^,-s|^) w i l l be very small i f S i s a reasonably f a i t h f u l representation of a subset of I J J . In any event, information transmission can never be negative, so that we can state c a t e g o r i c a l l y that: T($,-s|i|;) > zero Hence, from Equation (3.24) T($;S) > T($rlJ0 (3.25) /% Thus T(lp;S) i s an upper bound to the information transmitted by the network. Recalling the = {s",F^(s")} and the r e s u l t obtained i n Equation (3.25), then T ( $ ; W < T(S",F^(S");S) or Ttytty) < H(S^) (3.26) Where the equality holds when S i s a true subset of ijj. Equation (3.26) provides the important r e s u l t that the j o i n t entropy of the permanent s t a t i o n output data i s an upper bound on the reduction i n average uncertainty concerning the true basin hydrology. This i s true regardless of the form of estimation process used i n estimating the hydrology of the region outside the s t a t i o n l o c a t i o n s . 3.5 Hydrologic Network Design f o r Optimum Information Transmission The hydrologic network design problem i s defined here as the s e l e c t i o n of n permanent s t a t i o n s i t e s from m given l o c a t i o n s . I t i s assumed that a dense temporary network has provided concurrent data at a l l m locations over some representative period of time. I f the r o l e of the permanent network i s viewed as providing information which ultim a t e l y reduces uncertainty concerning events at the m l o c a t i o n s then the design c r i t e r i o n i s to maximize t h i s reduction i n uncertainty. This i s equivalent to maximizing the mutual information between events at the m locations and the measurements provided by the n s t a t i o n permanent network together with the derived estimates f o r the m-n l o c a t i o n s . Thus the objective f o r the network design problem stated above i s to s e l e c t the n s t a t i o n locations subset i from the m l o c a t i o n set to maximize the information transmission T(S;S7,F(S.)), i . e . MAXIMIZE , /f, n n , n . n I [T(S;S ,F(S )) ] 1 |n 1 1 which, i n Equation (3.22), was shown to be i d e n t i c a l to " " f f 1 1 * 5 [H(sn>] (3.27) i n l This optimization problem can be solved by evaluating the j o i n t entropies of a l l combinations of n hydrologic v a r i a b l e s and s e l e c t i n g the combination which y i e l d s the highest j o i n t entropy. The locations associated with the hydrologic v a r i a b l e s i n the optimal j o i n t entropy represent the optimal network s t a t i o n l o c a t i o n s . A l t e r n a t i v e l y , the r o l e of the permanent s t a t i o n network may be viewed as reducing uncertainty concerning hydrologic events throughout 30. the e n t i r e region which i s only sparsely sampled by the temporary net-work. In t h i s case the information transmission by the network output and the estimation derived from the output about the true hydrologic input cannot be evaluated. From Equation (3.26), the design c r i t e r i o n i s to maximize the upper bound of the reduction i n uncertainty. The objective then i s to s e l e c t the n s t a t i o n l o c a t i o n subset from the m l o c a t i o n set to maximize the upper bound of T(ip; S^,F^ (s")) . MAXIMIZE i I n upper bound [T(l^;S",F^(s") ) ] (3.28) which i n Equation (3.26), was shown to be i d e n t i c a l to M * ™ 2 * tH(S n)] i n I Thus, the two d i f f e r e n t views of the r o l e of the permanent network, described above, both lead to i d e n t i c a l objective functions. In the event that c e r t a i n temporary network s t a t i o n locations are not s u i t a b l e f o r permanent sta t i o n s , for a v a r i e t y of p r a c t i c a l reasons, then the search for the optimal l o c a t i o n set would be confined to f e a s i b l e permanent s t a t i o n l o c a t i o n s . 3.6 Implications of the Optimum Information Transmission I t should be noted that optimizing information transmission between \p and does not ensure optimal information transmission for any other derived s t a t i s t i c s or transformations of ty. The r e s u l t s presented here are true, however, f o r any case where S? i s a true subset of and no r e s t r i c t i o n i s made on what constitutes IJJ. 31. 3.7 Relative Measures of Network Performance and Size The terms "normed information rate" and " i n t e n s i f i c a t i o n r a t i o " , which are defined below, w i l l be extensively used i n the subsequent chapters to compare the r e l a t i v e performance of networks i n regions of d i f f e r e n t characters. The normed information rate i s defined as the r a t i o of the information transmission, T(S;S?) by a subset of s t a t i o n l o c a t i o n s S^ about a set of point locations S and the t o t a l entropy H(S;S^) which i s equal to H(S). Mathematically, i t can be expressed as T(S;S") T J S ; S J = N 1 H(S;Sj) H(SJ) H(S) where T (S;S n) i s the normed information r a t e . N l The term " i n t e n s i f i c a t i o n r a t i o " i s the r a t i o of the number of s t a t i o n locations n to the t o t a l number of point locations m i n the region. In subsequent chapters i t i s usually expressed as a percentage. 32. Ungauged Grid Points Hydrologic Network Stations FIG.3.1a-GRID POINT REPRESENTATION OF BASIN HYDROLOGY. Message Encode r Transmitter Receiver Decoder Out put Source Signals FIG.3.1b-CONVENTIONAL REPRESENTATION OF A COMMUNICATION SYSTEM (After Shannon 1949). Message Source Encoding -Transmission -Decoding — i Data h -Jp Base Actual Hydrologic Events at Grid Points Actual I Hydrologic' Events at Station Points Spatial Estimation Measured Events "at Station Points U Estimated Hydrologic Events at Non- station Points FIG.3.1c-HYDROLOGIC NETWORK AS A COMMUNICATION SYSTEM . 33. Message Source Channel S Channel A Measured B Set £={s?,ty(Sin)} ty(S?) |The Estimating Systemj FIG. 3 .2-CASCADED CHANNEL COMMUNI CATION SYSTEM 34. CHAPTER IV NETWORK ANALYSIS BY INFORMATION THEORY 4.1 A p p l i c a t i o n of the Optimum Information Transmission C r i t e r i o n Preliminary tests of the a p p l i c a b i l i t y of the mul t i v a r i a t e information theory to hydrologic network involved the simulation of hydrologic data at numerous point locations or g r i d points i n a hypothetical region as well as r e a l data from two e x i s t i n g networks. The expressions " g r i d point" and "point l o c a t i o n " are used i n t h i s study. The word " g r i d " i s used to imply that the point locations are dispersed throughout a region on a uniform g r i d or other form of systematic or r a t i o n a l dispersion. The following two types of simulation schemes were used. 4.1.1 C o r r e l a t i o n matrix approach A c o r r e l a t i o n matrix, which prescribes the c o r r e l a t i o n between hydrologic event magnitudes at a l l p a i r s of point l o c a t i o n s i n the basin model, was s p e c i f i e d . The mean value of the hydrologic events at each lo c a t i o n was also s p e c i f i e d - A computer package (UBC-S:NORMAL) which generates a random data sequence at each l o c a t i o n , while preserving the c o r r e l a t i o n and mean s p e c i f i e d , was then u t i l i z e d . In a l l cases the underline d i s t r i b u t i o n was normally d i s t r i b u t e d . 4.1.2 Mu l t i v a r i a t e synthetic approach In t h i s case the s t a t i s t i c a l mean, variance, the s e r i a l c o r r e l a -t i o n c o e f f i c i e n t within the generated data s e r i e s at each l o c a t i o n , and the cross c o r r e l a t i o n c o e f f i c i e n t between locations could be cont r o l l e d and preserved within the simulated sets of data. Dependent and independent primary locat i o n s were introduced i n the simulation scheme. 35. For a single independent primary l o c a t i o n , the simulation was performed on the basis of lagged c o r r e l a t i o n within the data at that l o c a t i o n . The simulation at the dependent primary l o c a t i o n s were based on the cross c o r r e l a t i o n with the independent primary s t a t i o n . The character of t h i s simulation scheme resembles that proposed by F i e r i n g (20). The scheme was formulated as follows: (a) The coordinates of a number of point locations i n a hypothetical region were defined. Also the mean and the standard deviation of "n" primary locat i o n s were assumed to be known. (b) Data was f i r s t simulated f o r the single independent primary l o c a t i o n with a f i r s t order Markovian model. The simulation scheme for t h i s s t a t i o n was: x i , k = T i ( x i , k - i - v + v \ - r V 1 - 7 ! ( 4 - x ) (k = 1,2,...,N) where, x, , i s the kth event; 1/k i s the s e r i a l c o r r e l a t i o n c o e f f i c i e n t ; i s the mean of the data; (t> , i s a random normal deviate; k-1 i s the standard deviation of the data; N i s the number of samples to be simulated. (c) The data generation scheme for the remaining " n - l " dependent primary locat i o n s was based on t h e i r cross c o r r e l a t i o n with the previously simulated independent primary l o c a t i o n . The formulation f o r t h i s generation was as follows: 36. = p i , j ( x i , k - V + ( 1 - p i . + * k ^ - P i . j • • • • ( 4 - 2 ) where, x. , i s the kth event of j t h s t a t i o n to be simulated; p i s the cross c o r r e l a t i o n between j t h dependent primary I'D lo c a t i o n and the independent primary location,-U_.,0_. are r e s p e c t i v e l y the mean and the standard deviation at the j t h sta t i o n ; <t>, i s the random normal deviate, k (d) A l l remaining l o c a t i o n s were treated as dependent va r i a b l e s , the data generation f o r these "m" points depended upon a multiple regression scheme with the "n" primary locations acting as independent v a r i a b l e s . The formulation for the generation was: n y. . = E 3 . •, x. , . + 0>.c!c! (4.3) *i,D I > = 1 I,T,' ^,,D D i i where, 3 . i s the regression c o e f f i c i e n t between i t h secondary and l , t i'th primary l o c a t i o n i n the region; y. . i s the j t h event of the i t h loca t i o n ; I'D c^ i s a factor c o n t r o l l i n g the randomness of i t h l o c a t i o n . I t v a r i e s from zero to one. U i ' a i are re s p e c t i v e l y the mean and standard deviation of the i t h l o c a t i o n and are defined as follows: , n n n n u. = I 3 . .,y., / I 6 . ; a2.. = E 3 2 • ,o 2., / E 3 2 •, ; and n " " '=1 ^  V I ^ = Y X ' i'=l  x ' % , -d. . , = e where d. . . i s the distance between i t h secondary l o c a t i o n and i'th primary l o c a t i o n . 37. 4.2 Analysis The proposed objectives of using two d i f f e r e n t simulation schemes were (1) to t e s t the methodology under extreme hydrologic conditions, and (2) to compare the r e s u l t s obtained by the methodology with the predetermined p o t e n t i a l s t a t i o n s . In order to achieve the f i r s t goal, the m u l t i v a r i a t e c o r r e l a t i o n approach was used to simulate data f o r highly homogeneous (highly correlated) region as well as the heterogeneou (poorly correlated) region. In order to achieve the second goal, the data was simulated using m u l t i v a r i a t e synthetic approach which was able to discriminate primary stations from secondary ones. 4.2.1 M u l t i v a r i a t e c o r r e l a t i o n matrix approach Two types of mu l t i v a r i a t e c o r r e l a t i o n matrices were defined for simulation purposes. In the f i r s t case a very high c o r r e l a t i o n between the point locations was s p e c i f i e d to r e f l e c t homogeneous conditions throughout the region and t h i s c o r r e l a t i o n matrix i s shown i n Table 4.1 In the second case a weakly cor r e l a t e d matrix was used to represent a heterogeneous region. This second c o r r e l a t i o n matrix i s shown i n Table 4.2a. In both cases, 300 sets of data were generated for each of the ten locations i n the hypothetical region. The maximum information transmission c r i t e r i o n , s p e c i f i e d by Equation (3.27), was applied to determine the optimum st a t i o n locations The data at each g r i d point was f i r s t d i s c r e t i z e d by assigning the generated values into eight equal i n t e r v a l s ranging from minimum to the maximum values. The r e l a t i v e frequency d i s t r i b u t i o n was then computed and assumed to define the d i s c r e t e univariate p r o b a b i l i t y d i s t r i b u t i o n . The mul t i v a r i a t e r e l a t i v e frequencies was also computed 38. i n a s i m i l a r fashion and assumed to define the d i s c r e t e j o i n t p r o b a b i l i t y d i s t r i b u t i o n s . The optimum s t a t i o n locations were then determined by maximizing the entropy H(s") for the single s t a t i o n , two stations, three stations, etc. The r e s u l t s are l i s t e d i n Tables 4.1b and 4.2b. While t h i s approach r a i s e s questions of adequacy of the sample, i n the context of an exploratory example the p r e c i s i o n of the estimated d i s t r i b u t i o n was not considered c r i t i c a l . In order to compare the r e s u l t s obtained for homogeneous (for highly correlated) and heterogeneous (for weakly correlated) regions, the r e l a t i v e i n t e n s i f i c a t i o n ( i . e . the r a t i o of the optimum number of stations to the t o t a l number of g r i d points) was p l o t t e d against the corresponding r e l a t i v e optimal information transmission ( i . e . the r a t i o of optimum information transmission to the upper bound of information transmission). The r e s u l t i n g graphs are shown i n Figures 4.1 to 4.3. I t was revealed that, i n the case of a homogeneous region, the information transmission was very high for the i n i t i a l s i n g l e s t a t i o n network and then the marginal information gain for a d d i t i o n a l stations decreased very r a p i d l y . For an optimal s i n g l e s t a t i o n network, the r e l a t i v e information transmitted was 59%. In the case of heterogeneous regions, the r e l a t i v e information gain for an optimal single s t a t i o n network was 18% i n d i c a t i n g that a s i n g l e s t a t i o n could reduce only 18% of the region uncertainty. At 50% i n t e n s i f i c a t i o n , the r e l a t i v e information transmission was approximately 87% f o r the homogeneous region and 79% for the heterogeneous region. 4.2.2 M u l t i v a r i a t e synthetic approach Using a mu l t i v a r i a t e synthetic simulation approach, the data was simulated f o r ten point locations i n a region. Using the coordinates s p e c i f i e d i n Figure 4.4, the points i d e n t i f i e d as #1 acted as the primary independent l o c a t i o n and #2 and #3 acted as primary dependent l o c a t i o n s . A constraint of zero cross c o r r e l a t i o n was f i r s t imposed on the three primary locations and, i n addition, the s e r i a l c o r r e l a t i o n for the primary independent l o c a t i o n was set at zero. The mean and the standard deviation of p r e c i p i t a t i o n at a l l the point locations were assumed to be the same with the numerical values as 10 cm and 3 cm r e s p e c t i v e l y . The remaining seven g r i d points were numbered as shown i n the Figure 4.4. 300 samples were generated at each g r i d point by using the m u l t i v a r i a t e synthetic approach as discussed previously i n Section 4.1.2. The r e s u l t -ing c o r r e l a t i o n matrix, which i s l i s t e d i n Table 4.3a, confirms that the points #1, #2, and #3 are almost completely independent of each other but have high c o r r e l a t i o n with the remaining g r i d point v a r i a bles i n the region as was intended. Optimum d i s c r e t e information transmission l o c a t i o n s were found for networks having from one to ten st a t i o n s . The optimum network s t a t i o n locations are l i s t e d i n Table 4.3b. I t was found that the optimum single s t a t i o n i n the region was #3 transmitting 1.837 nats of informa-t i o n . In the two and three s t a t i o n cases, the best s t a t i o n sets were #1, #3 and #1, #2, #3.repectively, with the corresponding information transmitted as 3.565 and 4.808 nats r e s p e c t i v e l y . The t o t a l uncertainty H(S) associated with the region i s 5.499 nats which indicates that three s t a t i o n sets resolved about 90% of the t o t a l uncertainty associated with the region. The marginal information gain per s t a t i o n was i n i t i a l l y 40. high and decreased r a p i d l y with a d d i t i o n a l network s t a t i o n s . The marginal information increment for the fourth a d d i t i o n a l s t a t i o n was quite low. 4.2.3 Lower Mainland Region of B.C. Two years of d a i l y r a i n f a l l data (October 1974 to September 1976) were c o l l e c t e d from an e x i s t i n g network c o n s i s t i n g of 25 raingauges with an approximate density of 65 square kilometers per s t a t i o n . The region selected was i n the Lower Mainland Region of B r i t i s h Columbia , these stations being dispersed throughout an area of approximately 1600 square kilometers. This i s a coastal region and includes both mountainous t e r r a i n and v a l l e y f l o o r . There i s a s i g n i f i c a n t v a r i a t i o n i n the d a i l y p r e c i p i t a t i o n within t h i s area. The s t a t i o n locations and t h e i r a l t i t u d e s above mean sea l e v e l are l i s t e d i n Table 4.4. Note that the s t a t i o n #5 ( i . e . Chilliwack R. Center Cr.) i s at the highest a l t i t u d e of 1600 f t while the s t a t i o n #24 ( i . e . Vancouver International Airport) i s at the lowest elevation of 16 f t . The study area was divided i n t o three subregions depending upon t h e i r l o c a t i o n s and the sparseness of the network. These d i v i s i o n s permit comparison of r e s u l t s obtained i n various subregions with d i f f e r e n t network d e n s i t i e s and d i f f e r e n t topographic v a r i a t i o n s . Sub-region #1 includes nine stations i d e n t i f i e d as stations #1, #2, #3, #5, #8, #9, #17, #18, and #19. These stations are sparsely d i s t r i b u t e d . Subregion #11 and #111 have r e l a t i v e l y dense networks and are homogeneous. Stations #4, #6, #7, #10, #20, and #21 constitute the six stations of subregion I I . Subregion I I I consists of 10 stations i d e n t i f i e d as #11, #12, #13, #14, #15, #16, #22, #23, #24, and #25. The t h i r d subregion i s the most homogeneous since the o v e r a l l entropy of i t s data was lowest. Using the same basis of d i s c r e t e entropy computation described previously f o r the simulated cases, the optimal information and s t a t i o n l o c a t i o n s were determined f or each of the subregions. The optimal s i n g l e s t a t i o n networks were s t a t i o n #9 i n subregion #1, s t a t i o n #10 i n sub-region #11 and s t a t i o n #11 i n subregion #111. The r e s u l t s are l i s t e d i n Tables 4.5a, 4.5b, and 4.5c. The normed information rate was computed for each subregion. The r e s u l t s l i s t e d i n Tables 4.5a to 4.5c revealed that the normed information rate of subregion I i s lower than f o r the subregions II and III which showed the r e l a t i v e heterogeneity of the subregion I compared to subregions II and I I I . 4.2.4 I n t e r i o r B r i t i s h Columbia Region A second region i n the i n t e r i o r of B.C. was selected to compare the information transmission performance of i t s network with the networks of the Lower Mainland Region discussed i n Section 4.2.3. Ten widely separated stations were selected and t h e i r locations are given i n Table 4.6. There i s a high v a r i a t i o n of r a i n f a l l among these ten stations which are dispersed i n an area of approximately 120,000 square kilometers with an average density of 12,000 square kilometers per s t a t i o n . Also the s t a t i o n #3 ( i . e . Hope) i s at the lowest a l t i t u d e of 152 f t while the s t a t i o n #1 ( i . e . B a r k e r v i l l e ) i s at the highest a l t i t u d e of 4180 f t . Two years of d a i l y p r e c i p i t a t i o n data was used f o r the a n a l y s i s . The maximum information transmission c r i t e r i o n as discussed i n Section 3.3, was applied to i d e n t i f y the optimal s t a t i o n l o c a t i o n s i n the region. By maximizing the j o i n t entropy i n Equation (3.27) of the various combinations of s t a t i o n sets, the optimum information transmission and the corresponding optimum s t a t i o n locations could be obtained (Fig. 4.6). The j o i n t entropies were computed as described in Section 4.2.1, and the optimum s t a t i o n sets l i s t e d i n Table 4.7. These r e s u l t s were then compared with the r e l a t i v e l y dense and homogeneous subregion I I I . A comparison i s shown i n Figure 4.7. In the single s t a t i o n case ( i . e . 10% r e l a t i v e i n t e n s i f i c a t i o n ) , the normed information rate for the Lower Mainland Region was approximately 42% while i n I n t e r i o r B.C. i t was only 22% f o r the equivalent i n t e n s i f i c a -t i o n . This indicates that, i n terms of information, a single s t a t i o n i n the Lower Mainland can resolve approximately 42% of the regional uncertain-ty while i n I n t e r i o r B.C. i t can resolve only 22% of the regional uncertainty. The marginal information gain per network s t a t i o n i n the I n t e r i o r B.C. i s higher than i n Lower Mainland for a l l s t a t i o n additions and t h i s r e f l e c t s sparse g r i d points and the heterogeneity i n the I n t e r i o r B.C. Region. 4.3 Further Discussion of Results Marginal information gain always decreases with the increase i n the number of s t a t i o n s . The decay i n the marginal information gain i s an i n d i c a t i o n of the sparseness of the network density. High decay rate i n marginal information shows r e l a t i v e l y dense or homogeneous network. Information transmission curves are monotonically increasing smooth curves which, when compared to each other, give an i n d i c a t i o n of the r e l a t i v e density of the network. The numerical values of the information transmission for a given s t a t i o n set also give the i n d i c a t i o n of the heterogeneity i n the time ser i e s data. Higher value indicates higher heterogeneity i n the data than the data with lower values. As revealed from the station selection, the results obtained are persistent. The optimum station selection for "n-1" station cases are usually retained i n the network of "n" optimum station sets. I t has been observed i n many simulated and real examples, a few of which are l i s t e d i n t h i s chapter. For a given normed information rate, the number of stations required i n a r e l a t i v e l y homogeneous network i s lower than i n the sparse network. 44. TABLE 4.1a. CORRELATION MATRIX FOR A LOW CORRELATED SIMULATED BASIN 1 2 3 4 5 6 7 8 9 10 1 1. .0002 -.0002 -.0002 -.0002 .0002 .0001 -.0001 .0003 -.0001 2 1. -.0003 -.0002 .0002 -.0002 .0001 .0001 .0000 -.0001 3 1. .0001 .0001 .0003 .0002 .0001 -.0004 .0001 4 1. -.0002 .0001 .0001 -.0001 -.0004 .0001 5 1. -.0001 .0001 -.0003 -.0003 .0001 6 1. .0001 -.0001 -.0001 .0001 7 1. -.0003 .0003 .0001 8 1. .0001 -.0003 9 1.' -.0002 10 1. TABLE 4.1b OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN A SIMULATED LOW CORRELATED BASIN No. of Stations to be Retained Opt. Infor-mation I n t e n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 1.179 10 0.181 9 2 2.315 20 0.366 2,9 3 3.395 30 0.521 2,4,9 4 4.396 40 0.674 2,4,8,9 5 5.179 50 0.794 2,4,7,8,9 6 5.779 60 0.886 2,4,5,7,8,9 7 6.166 70 0.946 2,3,4,5,7,8,9 8 6.376 80 0.978 1,2,3,4,5,7,8,9 9 6.488 90 0.995 1,2,3,4,5,7,8,9,10 10 6.519 100 1.000 A l l Stations TABLE 4.2a CORRELATION MATRIX FOR A SIMULATED HIGHLY CORRELATED BASIN i \ 1 2 3 4 5 6 7 8 9 10 1 1. .999 .998 .997 .996 .995 .994 .993 .992 .990 2 1. .999 .998 .997 .996 .995 .994 .993 .992 3 1. .999 .998 .997 .996 .995 .994 .993 4 1. .999 .998 .997 .996 .995 .994 5 1. .999 .998 .997 .996 .995 6 1. .999 .998 .997 .996 7 1. .999 .998 .997 8 1. .999 .998 9 1. .999 10 1. TABLE 4.2b OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN A SIMULATED HIGHLY CORRELATED BASIN No. of Stations to be Retained Opt. Infor-mation I n t e n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 1.693 10 0.586 7 2 2.186 20 0.756 1,10 3 2.411 30 0.835 1,6,10 4 2 .560 40 0.886 1,6,9,10 5 2.655 50 0.919 1,4.7,9,10 6 2.725 60 0.943 1,4,5,7,9,10 7 2.784 70 0.964 1,3,5,6,8,9,10 8 2.824 80 0.978 1,2,4,5,6,7,9,10 9 2 .860 90 0.990 Except 8 10 2.889 100 1.000 A l l Stations TABLE 4.3a CORRELATION MATRIX OF A TYPICAL SIMULATED BASIN USING A MULTIVARIATE SYNTHETIC APPROACH H 1 2 3 4 5 6 7 8 9 10 i 1. .001 -.048 .767 .765 .571 .645 .564 .407 .387 2 1. -.058 .501 .373 .696 .529 .403 .632 .361 3 1. .322 .453 . 361 .487 .663 .600 .790 4 1. .984 .960 .977 .921 .889 .818 5 1. .924 .978 .958 .890 .876 6 1. .976 .916 .958 .841 7 1. .974 .961 .905 8 1. .959 .973 9 1. .941 10 1. TABLE 4.3b OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN A TYPICAL SIMULATED BASIN No. of Stations to be Retained Opt. Infor-mation I n t e n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 1.837 10 0.330 3 2 3.565 20 0.648 1,3 3 4 .808 30 0.874 1,2,3 4 5.083 40 0.923 1,2,3,6 5 5.257 50 0.955 1,2,3,6,9 6 5.352 60 0.972 1,2,3,4,8,9 7 5.427 70 0.987 1,2,3,4,6,8,9 8 5.466 80 0.993 1,2,3,4,6,8,9,10 9 5.488 90 0.998 Except 5 10 5.499 100 1.000 A l l 47. TABLE 4.4 - DESCRIPTION OF THE PRECIPITATION GAUGES IN LOWER MAINLAND Station Number Station Name Latitude Deg Min Longitude Deg Min Elevation i n Feet 1 Abbotsford A. 49 01 122 22 198 2 Agassiz CDA 49 17 121 46 50 3 Aldergrove 49 17 122 29 450 4 Burnaby SFU 49 17 122 55 1200 5 Chilliwack R. Centre Cr 49 06 121 33 1600 6 Delta Pebble H i l l s 49 01 123 04 40 7 Haney UBC RF Admin 49 17 122 34 470 8 Langley Lo c h i e l 49 02 122 35 345 9 Mission West Abby 49 08 122 17 725 10 North Burnaby Union 49 18 122 59 190 11 North Vane. Capillano 49 20 123 06 400 12 N. Vane. Cleveland 49 22 123 06 500 13 N. Vane. Cloverley 49 19 123 03 260 14 N. Vane. Seymour Blvd. 49 19 123 01 28 15 N. Vane. Upper Lynn 49 21 123 02 725 16 N. Vane. Wharves 49 19 123 07 20 17 Rosedale 49 11 121 48 35 18 Sardis 49 05 121 55 500 19 Sumas Canal 49 07 122 07 21 20 Surrey Newton 49 06 122 50 250 21 Surrey Sunnyside 49 03 122 48 330 22 Vane. Cedar Cottage 49 17 123 07 286 23 Vane. Dunbar 49 14 123 11 200 24 Vane. Inter. A i r p o r t 49 11 123 10 16 25 Vane. UBC 49 16 123 15 305 48. TABLE 4.5a OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN LOWER MAINLAND REGION-SUBREGION I (STATIONS: #1, #2, #3, #5, #8, #9, #17, #18, #19) No. of Stations to be Retained Opt. Infor-mation In t e n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 0.866 11 0.310 9 2 1.486 22 0.520 1,9 3 1.902 33 0.670 1,3,9 4 2.268 • 44 0.800 1,2,3,9 5 2.500 55 0.880 1,2,3,9,18 6 2.648 67 0.930 1,2,3,5,9,18 7 2.715 78 0.960 1,2,3,5,8,9,18 8 2.795 89 0.980 1,2,3,5,8,9,17,18 9 2.841 100 1.00 A l l TABLE 4.5b OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN LOWER MAINLAND REGION-SUBREGION II (STATIONS: #4, #6, #7, #10, #20, #21) No. Of Stations to be Retained Opt. Infor-mation Inte n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 0.849 17 0.410 10 2 1.290 33 0.620 10,21 3 1.625 50 0.780 4,6,10 4 1.831 67 0.880 4,6,10,21 5 1.999 83 0.950 4,6,7,10,21 6 2.094 100 1.00 4,6,7,10,20,21 49. TABLE 4.5c OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN LOWER MAINLAND REGION-SUBREGION III (STATIONS: #11, #12, #13, #14, #15, #16, #22, #23, #24, #25) No. of Stations to be Retained Opt. Infor-mation In t e n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 0.883 10 0.420 11 2 1.294 20 0.620 11,12 3 1.562 30 0.750 11,12,24 4 1.765 40 0.850 11,12,22,24 5 1.872 50 0.900 11,12,22,23,24 6 1.953 60 0.940 11,12,15,22,23,24 7 2 .010 70 0.965 11,12,15,16,22,23,24 8 2.062 80 0.973 11,12,15,16,22,23,24,25 9 2.082 90 0.982 Except 14 10 2.086 100 1.00 A l l TABLE 4.6 - DESCRIPTION OF THE PRECIPITATION GAUGES IN INTERIOR B.C. REGION Station Number Station Name Latitude Deg Min Longitude Deg Min Elevation i n Feet 1 B a r k e r v i l l e 53 04 121 31 4180 2 Blue River 52 09 119 17 2243 3 Hope A. 49 23 121 26 152 4 Joe Rich Cr. 49 51 119 08 2870 5 Kamloop A. 50 43 120 26 1132 6 Kelowna A. 49 58 119 23 1368 7 Lytton 50 14 121 35 845 8 McCulloch 49 48 119 12 4100 9 Penticton A. 49 28 119 36 1121 10 Prince George 53 50 12 2 48 1870 11 Puntzi Mount. 52 10 124 12 4500 12 Quesnel A. 53 02 122 31 1787 13 Salmon Arm 50 42 119 14 1660 14 Vavenby 51 35 119 47 1465 15 Vernon V.A. 50 15 119 16 1383 16 W i l l . Lake A. 52 11 122 03 3088 51. TABLE 4.7 OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN INTERIOR B.C. REGION No. of Stations to be Retained Opt. Infor-mation In t e n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 0.777 10 0.222 8 2 1.405 20 0.401 1,8 3 2 .010 30 0.574 1,8,13 4 2.474 40 0.706 1,2,8,13 5 2 .748 50 0.784 1,2,5,8,13 6 2.958 60 0.844 1,2,5,8,13,16 7 3.169 70 0.904 1,2,3,5,8,10,13 8 3.302 . 80 0.943 1,2,3,5,8,10,13,16 9 3.416 90 0.975 1,2,3,4,5,8,10,13,16 10 3.503 100 0.975 1,2,3,4,5,8,10,13,15,16 FIG.4.1 - OPTIMUM INFORMATION TRANSMISSION AND OPTIMAL STATION SELECTION IN A POORLY CORRELATED SIMULATED REGION.(Simulation by Correlation Matrix). F I G . 4 . 2 - O P T I M U M I N F O R M A T I O N T R A N S M I S S I O N AND O P T I M U M S T A T I O N S E L E C T I O N IN A H IGHLY C O R R E L A T E D S I M U L A T E D REGION (S imula t ion by Corre la t ion Ma t r i x ) . F I G . 4 . 3 - C O M P A R I S O N OF I N F O R M A T I O N T R A N S M I S S I O N IN HIGHLY C O R R E L A T E D A N D P O O R L Y C O R R E L A T E D S I M U L A T E D R E G I O N . 55. FIG.4.4- COORDINATES OF VARIOUS POINT LOCATIONS USED IN MULTIVARIATE SYNTHETIC APPROACH. FIG. 4 . 5 - O P T I M U M INFORMATION TRANSMISSION AND OPTIMAL STATION S E L E C T I O N IN LOWER MAINLAND - S U B R E G I O N TH . On H (S) = 3.503 0.0 ,2,3,4,5,8,10,13,16) (1,2,3,5,8,10,13,16) (1,2,3,5,8,13,16) ,2,5,8,13,16) (1,2,5,8,13) 8 10 0 I 2 3 4 5 6 7 Number of Stations FIG.4.6-0PTIMUM INFORMATION TRANSMISSION AND OPTIMUM STATION SELECTION IN INTERIOR B.C. REGION. 01 FIG. 4 . 7 - C O M P A R I S O N OF OPTIMUM INFORMATION TRANSMISSION IN L O W E R M A I N L A N D AND INTERIOR B . C . R E G I O N . 59. CHAPTER V COMPUTATIONAL AIDS 5.1 Introduction The most d i r e c t approach for s e l e c t i n g "n" optimum stations from "m" point locations in a hydrologic basin i s to evaluate the information transmission f o r a l l ( m) possible combinations. This n approach becomes computationally i n f e a s i b l e for values of "m" and "n" greater than, for example, n=5, m=50. I t i s therefore e s s e n t i a l to investigate possible techniques f o r reducing the combinatorial search and thereby achieve acceptable computational demands. Some s e l e c t i o n techniques which have been proposed i n the l i t e r a t u r e and have some relevance to the combinatorial search problem which a r i s e s here, are summarized below: The p r o b a b i l i t y of error i s the most commonly used approach i n pattern recognition (54). I t i s defined as the f r a c t i o n of patterns that are m i s c l a s s i f i e d using a single property of the pattern. The f r a c t i o n of patterns that are m i s c l a s s i f i e d i s assumed equivalent to the f r a c t i o n of unresolved uncertainty. Hence, the p r o b a b i l i t y of error, i n context of hydrologic network design, i s the f r a c t i o n of the unresolved uncertainty by the s t a t i o n set about the basin. The s t a t i o n sets with high p r o b a b i l i t y of error are eliminated and those with the lowest p r o b a b i l i t y of error are selected. The search for the network stations i s then confined to the high ranking points. Average c o r r e l a t i o n c o e f f i c i e n t takes i n t o account the s p a t i a l corre-l a t i o n by ranking the g r i d points according to t h e i r average c o r r e l a t i o n c o e f f i c i e n t s . I n i t i a l l y a key single s t a t i o n i n the region i s selected 60. on the basis of the p r o b a b i l i t y of error approach described above. The second s t a t i o n selected i s the s t a t i o n which has the smallest c o r r e l a t i o n with the key s t a t i o n . The t h i r d s t a t i o n i s selected such that i t s average c o r r e l a t i o n with the f i r s t two selected stations i s smaller than that of a l l the remaining g r i d point l o c a t i o n s . The procedure i s repeated u n t i l the required number of stations are obtained (61). The sequential approach requires the t e s t i n g of each new s t a t i o n added. A single s t a t i o n i s f i r s t selected using the smallest p r o b a b i l i t y of error approach described e a r l i e r . The second s t a t i o n selected i s the s t a t i o n which i s the best discriminator, i n e f f e c t having the lowest c o r r e l a t i o n with the previously selected s t a t i o n . The t h i r d s t a t i o n i s selected on a s i m i l a r basis (71). Each of these approaches has i t s own l i m i t a t i o n s . For example, the p r o b a b i l i t y of erro r approach does not take into account the assoc i a t i o n among the point locations and hence the stations chosen may contain redundant s t a t i o n s . The average c o r r e l a t i o n c o e f f i c i e n t approach i s based on the assumption that the events at the two stations are normally d i s t r i b u t e d which may, or may not, be true i n r e a l hydrologic problems. The sequential analysis approach has been j u s t i f i e d only by using normality and independency assumptions (54). The computational aids which w i l l be proposed i n t h i s chapter are, however, not r e s t r i c t e d to the normality, l i n e a r i t y and independency assumptions. These proposed e l i m i n a t i o n techniques are based on Shannon's information measure which are summarized below: 5.2 Elim i n a t i o n Based on the Entropy of the Individual Station In t h i s case the single entropies of a l l the g r i d points using univariate p r o b a b i l i t y d i s t r i b u t i o n s are computed. The i n d i v i d u a l point locations are ranked on the basis of the entropies of t h e i r events. For example, the point l o c a t i o n with the highest entropy w i l l be ranked as the f i r s t and so on. The underlying basis of s e l e c t i o n i s that the entropy at a single l o c a t i o n i s a measure of the hydrologic information transmitting c a p a b i l i t y of a network c o n s i s t i n g of a single s t a t i o n l o c a t i o n at that i n d i v i d u a l point. The l o c a t i o n s which are the most heterogeneous i n nature are selected as the p o t e n t i a l s t a t i o n s . This approach i s e f f e c t i v e when the network i s not dense and heterogeneity, due to topographical and c l i m a t i c v a r i a t i o n s , i s a dominating f a c t o r . I t would not be able to recognize the redundancy when two high entropy locations were p e r f e c t l y correlated. 5.3 Elimination Based on The J o i n t Entropies of the Pair Point Locations The number of paired combinations which must be evaluated by t h i s method to determine the optimum s t a t i o n set i n a region of "m" g r i d points i s m(m-l)/2. With t h i s s e l e c t i o n method, i t i s necessary to evaluate the b i v a r i a t e p r o b a b i l i t y d i s t r i b u t i o n s of events at a l l of the s t a t i o n p a i r s , while i n the previous case only the univariate form of the p r o b a b i l i t y d i s t r i b u t i o n was needed. At the b i v a r i a t e l e v e l the information transmission between a l l possible p a i r s i s evaluated. The frequency of occurrence of i n d i v i d u a l points i n , for example, the top 50% of the highest information transmission p a i r s , i s determined. The most frequently occurring points w i l l be the optimum stations and the next to the most frequent w i l l be the second optimum and so on. 62. 5.4 Dynamic Elimination Based on the Station Persistence This technique i s based on a dynamic optimization c r i t e r i o n . The s e l e c t i o n of the number of stations, for example k at a time depends upon the optimum s e l e c t i o n of k-1 previously selected s t a t i o n s . This technique i s based on the entropy maximization p r i n c i p l e which, i n turn, i s r e l a t e d to the maximum information transmission c r i t e r i o n . Assume that there are m location s i n a region with the events denoted by {x^,X2,...,X^}. We are interested i n optimally s e l e c t i n g "n" s t a t i o n l o c a t i o n s . Using Equation (3.27), i t i s not computationally d i f f i c u l t to determine the optimum information transmission by a single s t a t i o n i n the region. Assume that S** = X. i d e n t i f i e s the events at 3 k the optimum single s t a t i o n . The rigorous approach to f i n d i n g the optimum two s t a t i o n network i s to compute the j o i n t entropies f o r a l l of the possible m(m-l)/2 p a i r s and se l e c t the p a i r which has the maximum entropy. In other words, T*(S;S 2*) = Max H(Xi,X;.) (5.1) for a l l i , j A simpler approach i s to consider a l l possible (m-1) l o c a t i o n p a i r s including S 1* = X and se l e c t i n g the second s t a t i o n on the basis iC of the maximum entropy p a i r s . Mathematically i t can be explained as follows: T*(S;X k,X ] l) = Max H(Xk,X_.) j?k where T*(S;X k,X^) i s the optimum information transmission by the optimum s t a t i o n set (x ,X^); S 2* i s the set of optimum two st a t i o n s . 63. The set S 2* = (X^,X^) w i l l be optimum only when the following conditions hold true: H(X ,X„) >H(X.,X.) for a l l X., X. i ^ j Although there i s no d i r e c t proof for the above conditions, i t was found to be true i n v i r t u a l l y a l l cases where numerical analysis was performed. 5.5 Comparison of the Elimination Techniques The three combinatorial search reduction techniques proposed were applied to both simulated and r e a l p r e c i p i t a t i o n data. The r e s u l t s obtained by applying each proposed elimination technique to the simulated data were compared with the optimum r e s u l t s already obtained by r i g o r o u s l y applying the entropy maximization p r i n c i p l e . The techniques were also applied to larger hydrologic networks. In the weakly correlated region, with the c o r r e l a t i o n matrix given i n Table (4.1a), the stations selected on the basis of the i n d i v i d u a l and b i v a r i a t e entropies coincided with those selected by rigorous method. The r e s u l t s are l i s t e d i n Table 5.1. S i g n i f i c a n t l y the optimum stations selected f o r the network of size n retained a l l stations i n the optimum n - l s t a t i o n network. Hence no discrepancy was found i n the st a t i o n s e l e c t i o n using a l l the three proposed elimination techniques and s t a t i o n s e l e c t i o n obtained by rigorous optimization. This i s a t t r i b u t a b l e to the high independency among the point l o c a t i o n s . Independency w i l l cause the j o i n t entropy of a group of stations to be approximately equal to the sum of the entropies o f events a t the i n d i v i d u a l s t a t i o n s . Hence under t h i s condition, these stations having i n d i v i d u a l l y the highest entropies would always appear on optimum s t a t i o n points. 64 . In the case of a highly c o r r e l a t e d region, defined by the c o r r e l a t i o n matrix of Table 4.2a, the elimination techniques based on i n d i v i d u a l entropy was not very e f f e c t i v e . The b i v a r i a t e entropy approach off e r e d improvement over the i n d i v i d u a l entropy approach but there was s t i l l some discrepancy with the true optimum s t a t i o n s e l e c t i o n . Referring to Table 5.2a, the i n d i v i d u a l entropy approach selected locations #7 and #9 for the two s t a t i o n case; locations #6, #7, #9 for a three s t a t i o n case and #1, #6, #7, and #9 for a four s t a t i o n case. These r e s u l t s ranked to 33/45, 108/120, and 85/210 res p e c t i v e l y , where 45, 120, and 210 are the numbers of possible combinations which must be evaluated i n order to obtain the true optimum values. However, using the b i v a r i a t e entropy elimination concept, near optimum s t a t i o n sets were obtained. The three s t a t i o n set ranked fourth; the four s t a t i o n case ranked t h i r d ; and the f i v e s t a t i o n set ranked f i f t e e n t h . F i n a l l y , the dynamic elimination technique was able to recognize a l l of the true optimum s t a t i o n sets. The c h a r a c t e r i s t i c persistence of the optimal station l o c a t i o n f o r one s i z e of network i n l a r g e r optimal networks i s demonstrated i n t h i s example. In a network s i t u a t i o n where st a t i o n locations are to be selected from ten g r i d points i n a region, the number of combinations to be evaluated using dynamic elimination technique i n s i n g l e , two, and three s t a t i o n s e l e c t i o n are r e s p e c t i v e l y , 10, 9, and 8 compared with 10, 45, and 120 for rigorous optimization. The r e s u l t s obtained from each elimination method are summarized i n Tables 5.2a and 5.2b. The performance of three elimination techniques was also compared when applied to r e a l hydrologic data i n the Lower Mainland and I n t e r i o r regions of B r i t i s h Columbia. The r e s u l t s are summarized i n Tables 5.3 65. and 5.4. The elimination technique based on i n d i v i d u a l entropy performed better for the I n t e r i o r B.C. Region than f o r the Lower Mainland region. The b i v a r i a t e entropy elimination outperformed the univariate entropy elimination but discrepancies with the optimum sets were noticed as can be seen i n Tables 5.3b. The r e s u l t s based on dynamic elimination were completely p e r s i s t e n t with the true optimum r e s u l t . These techniques were applied to a l l 25 e x i s t i n g s t a t i o n points i n the Lower Mainland region. The ten best stations were then selected using the f i r s t two approaches. Using the t o t a l enumeration approach of optimization as discussed i n Chapter I I I , the optimum s t a t i o n combinations within the selected stations were then i d e n t i f i e d and are l i s t e d i n Table 5.4c. Using the dynamic elimination technique, the optimum s t a t i o n combinations up to 10 s t a t i o n sets were determined. The optimum informa-t i o n transmission and the corresponding s t a t i o n sets are l i s t e d i n Table 5.4d. The r e s u l t s obtained i n both cases were compared. I t was concluded that up to f o u r - s t a t i o n combinations, the s t a t i o n selections i n a l l cases were s i m i l a r . The optimal s t a t i o n combinations for more than four stations d i f f e r e d s l i g h t l y from each other. The r e s u l t s confirmed that the information transmission by the stations selected using the dynamic elimination technique on the 25 stations was higher than f or the optimum stations obtained by the elimination techniques based on univariate and b i v a r i a t e entropies. I t indicates the superior performance of the dynamic elimination technique to other proposed techniques. In order to t e s t the v a l i d i t y of these techniques further, the d a i l y p r e c i p i t a t i o n data from an experimental dense network from Oklahoma was used. A b r i e f d e s c r i p t i o n of the watershed i s given below. 66. 5.6 Washita River Watershed, Chickasha, Oklahoma A temporary dense network spaced on a 3 mile square g r i d was i n s t a l l e d i n Oklahoma i n an 1130 square mile area. Approximately 175 raingauges were established. R a i n f a l l data (>0.01 inch) for the year 1962, 1963, and 1964 for approximately 400 days were a v a i l a b l e . The network area i s i n a region of moist to dry sub-humid climate. Normal annual p r e c i p i t a t i o n varies from 33 inches on the east to 28 inches on the west edge of the network. The d i s t r i b u t i o n of the p r e c i p i t a -t i o n i s bimodal with peaks occurring i n May and September. About 98% of the yearly p r e c i p i t a t i o n i s contributed by r a i n f a l l and the remaining 2% by snow. Flooding i n the area i s generally caused by thunderstorms. The locations of the various gauges are shown i n Figure 5.1 and t h e i r l a t i t u d e s , longitudes and the elevations are l i s t e d i n Table 5.5. The whole basin was divided into f i v e subregions as shown i n the Figure 5.1. The natural watershed boundaries were used to define the subregion boundaries. Since i t i s computationally i n f e a s i b l e to determine the optimal ten stations d i r e c t l y from 175 point locations using the t o t a l enumeration approach, analysis was performed on a subregion basis only. The ten best s t a t i o n network was selected for each subregion using a l l three elimination techniques. Univariate and b i v a r i a t e entropy techniques provided s i m i l a r r e s u l t s i n s e l e c t i n g the ten optimum stations but these r e s u l t s d i f f e r e d from those obtained by dynamic elimination approach applied to a l l points i n the subregion. The r e s u l t s f o r each subregion are l i s t e d i n Tables 5.6a to 5.10b. I t was concluded that i n those cases where a true optimal s o l u t i o n could be obtained by t o t a l enumeration of a l l possible s t a t i o n combinations, the dynamic elimination technique produced an equally optimal r e s u l t . In those cases where t o t a l enumeration was not possible, the dynamic elimination technique produced superior information transmission to that obtained by the univariate and bivariate elimination techniques TABLE 5.1 VALIDITY OF THE ELIMINATION TECHNIQUE BASED ON THE UNIVARIATE ENTROPY CONCEPT - WEAKLY CORRELATED SIMULATED REGION No, of Stations True Optimum Result Based Elimination on Technique Rank Optimum Station Inf. Trans. Optimum Station Inf. Trans. 1 9 1.179 9 1.179 I 2 2,9 2.315 2,9 2.315 I 3 2,4,9 3.395 2,4,9 3.395 I 4 2,4,8,9 4.396 2,4,8,9 4.396 I 5 2,4,7,8,9 5.178 2,4,7,8,9 5.178 I 69. TABLE 5.2a VALIDITY OF THE ELIMINATION TECHNIQUE BASED ON THE UNIVARIATE ENTROPY CONCEPT - HIGHLY CORRELATED SIMULATED REGION No. of Stations True Optimum Result Based Elimination on Technique Rank Optimum Station Inf. Trans. Optimum Statio n Inf. Trans. 1 7 1.693 7 1.693 1 2 1,10 2.186 7,9 1.945 33rd 3 1,6,10 2.411 6,7,9 2 .088 108th 4 1,6,9,10 2.560 1,6,7,9 2.396 85 th 5 1,4,7,9,10 2.655 1,6,7,8,9 2.452 165th TABLE 5.2b VALIDITY OF THE ELIMINATION TECHNIQUE BASED ON THE BIVARIATE ENTROPY CONCEPT - HIGHLY CORRELATED SIMULATED REGION No. of Stations True Optimum Result Based on Elimination Technique Rank Optimum Station * Inf. Trans. Optimum Station Inf. Trans. 2 1,10 2.186 1,10 2.186 I 3 1,6,10 2.411 1,9,10 2.382 IV 4 1,6,9,10 2.560 1,7,5,10 2.533 II I 5 1,4,7,9,10 2.655 1,2,7,9,10 2.613 15th TABLE 5.3a VALIDITY OF THE ELIMINATION TECHNIQUE BASED ON UNIVARIATE ENTROPY CONCEPT - INTERIOR B.C. REGION No. of Stations True Optimum Result Based Elimination on Technique Rank Optimum Station Inf. Trans. Optimum Station Inf. Trans. 1 6 0.777 67 0.777 I 2 2,6 1.473 2,6 1.473 I 3 2,6,8 2.010 2,6,8 2 .010 I 4 1,2,6,8 2.474 1,2,6,8 2.474 I 5 1,2,5,6,8 2.748 1,2,9,6,8 2.701 V TABLE 5.3b VALIDITY OF THE ELIMINATION TECHNIQUE BASED ON BIVARIATE ENTROPY CONCEPT - INTERIOR B.C. REGION No. of Stations True Optimum Result Based Elim i n a t i o n on Technique Rank Optimum Station Inf. Trans. Optimum Station Inf. Trans. 1 6 0.777 6 0.777 I 2 2,6 1.473 2,6 1.473 I 3 2,6,8 2.010 2,6,8 2 .010 I 4 1,2,6,8 2.474 1,2,6,8 2.474 I 5 1,2,5,6,8 2.748 1,2,5,6,8 2.748 I 71. TABLE 5.4a VALIDITY OF THE ELIMINATION TECHNIQUE BASED ON UNIVARIATE ENTROPY CONCEPT - LOWER MAINLAND SUBREGION III No. of True Optimum Result Based Elimination on Technique Rank Stations Optimum Station Inf. Trans. Optimum Statio n Inf. Trans. 1 7 0.883 7 0.883 I 2 1 ,7 1.506 5,7 1.360 7 th 3 1 ,2,7 1.930 5,6,7 1.707 41st 4 1 ,2,5,7 2.260 4,5,6,7 1.916 122nd 5 1 ,2,5,6,7 2.456 4,5,6,7,9 2.065 204 th TABLE 5.4b VALIDITY OF THE ELIMINATION TECHNIQUE BASED ON BIVARIATE ENTROPY CONCEPT - LOWER MAINLAND SUBREGION III No. of Stations True Optimum Result Based Elimination on Technique Rank Optimum Station Inf. Trans. Optimum Station Inf. Trans. 2 1,7 1.506 1,7 1.506 I 3 1,2,7 1.930 1,6,7 1.909 4 th 4 1,2,5,7 2.260 1,2,6,7 2.180 15th 5 1,2,5,6,7 2.456 1,2,5,6,7 2.456 I TABLE 5.4c OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN LOWER MAINLAND REGION. (ELIMINATION METHOD BASED ON SINGLE AND BIVARIATE ENTROPY CONCEPT: ALL 25 LOCATIONS) No. of Stations to be Retained Opt. Infor-mation Inte n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 0.883 04 0.255 11 2 1.506 08 0.452 11,1 3 1.931 12 0.579 11,1,3 4 2.261 16 0.668 11,1,3,9 5 2.556 20 0.766 11,1,3,9,4 6 2.710 24 0.813 11,1,3,9,4,5 7 2.802 28 0.846 11,1,3,9,4,10,12 8 2.839 32 0.852 11,1,3,9,4,10,12,5 9 2 .851 36 0.855 11,1,3,9,4,10,12,5,22 10 2 .863 40 0.870 11,1,3,9,4,10,12,5,22,21 TABLE 5.4d OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN LOWER MAINLAND REGION. (SELECTION BASED ON DYNAMIC ELIMINATION METHOD: ALL 25 LOCATIONS) No. O f Stations to be Retained Opt. Infor-mation Inte n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 0.883 04 0.255 11 2 1.506 08 0.452 11,1 3 1.931 12 0.579 11,1,3 4 2.296 16 0.690 11,1,3,2 5 2.586 20 0.765 11,1,3,2,9 6 2.740 24 0.836 11,1,3,2,9,5 7 2.889 28 0.878 11,1,3,2,9,5,10 8 2.971 32 0.891 11,1,3,2,9,5,10,12 9 3.027 36 0.908 11,1,3,2,9,5,10,12,21 10 3 .062 40 0.918 11,1,3,2,9,5,10,12,21,22 73. TABLE 5.5 RAINGAGE NETWORK STATION DATA, CHICKASHA, OKLAHOMA Latitude Gage No. Degrees Minutes Seconds 1 35 31 21 2 35 30 53 3 35 30 45 4 35 26 59 5 35 27 25 6 35 27 28 7 35 27 53 8 35 28 57 9 35 25 14 10 35 25 15 11 35 24 23 12 35 24 22 13 35 24 17 14 35 24 00 15 35 21 18 16 35 20 56 17 35 21 34 18 35 21 45 19 35 21 59 20 35 22 00 21 35 23 05 22 35 19 41 23 35 19 09 24 35 19 03 25 35 18 17 26 35 18 32 27 35 18 06 28 35 17 47 29 35 17 31 30 35 17 30 31 35 17 26 32 35 16 52 33 35 16 33 34 35 16 16 35 35 15 57 36 35 13 57 37 35 13 25 38 35 13 57 39 35 13 55 40 35 13 58 41 35 14 24 42 35 14 52 43 35 15 10 44 35 15 45 45 Longitude Elevation Degrees Minutes Seconds Feet 98 28 57 1540 98 32 43 1580 98 21 00 1610 98 18 10 1640 98 21 20 1580 98 24 44 1640 98 27 46 1650 98 30 04 1640 98 27 53 1640 98 24 53 1500 98 21 31 1440 98 18 15 1630 98 15 14 1660 98 12 08 1540 98 10 06 1540 98 12 52 1590 98 15 12 1580 98 19 07 1520 98 22 33 1460 98 25 47 1610 98 27 54 1650 98 25 46 1510 98 22 45 1990 98 19 25 1440 98 16 16 1400 98 13 17 1360 98 10 00 1440 98 06 54 1520 98 04 33 1520 98 01 19 1350 97 57 41 1350 97 55 06 1420 97 52 02 1340 97 48 47 1330 97 45 34 1300 97 46 03 1400 97 48 44 1320 97 52 03 1240 97 55 06 1150 97 58 22 1250 98 01 28 1240 98 05 13 1350 98 07 57 1370 98 11 24 1300 98 14 04 1460 74. TABLE 5.5 Raingage Network Station Data, Chickasha, Oklahoma (Continued) Latitude Gage No. Degrees Minutes Seconds 46 35 15 54 47 35 16 10 48 35 17 03 49 35 17 25 50 35 14 40 51 35 14 12 52 35 14 02 53 35 13 56 54 35 13 08 55 . do . . 56 35 12 38 57 35 12 13 58 35 12 14 59 35 12 13 60 35 11 19 61 35 11 08 62 35 10 41 63 35 10 30 64 35 10 35 65 36 07 28 66 35 07 44 67 35 08 05 68 35 08 10 69 35 08 56 70 35 08 43 71 35 08 49 72 35 08 55 73 35 09 23 74 35 09 21 75 35 09 58 76 35 10 39 77 35 10 36 78 35 10 30 79 35 10 46 80 35 07 49 81 35 07 41 82 35 07 02 83 35 07 01 84 . do . . 85 35 06 53 86 35 06 06 87 35 06 07 88 35 06 01 89 35 05 40 90 35 05 17 91 35 05 05 92 35 05 02 93 35 02 21 94 35 02 37 95 . do . . Longitude Elevation Degrees Minutes Seconds Feet 90 17 20 1360 98 20 31 1420 98 23 38 1540 98 26 27 1450 98 26 47 1450 98 23 39 1510 98 20 30 1430 98 17 49 1430 98 14 08 1490 98 10 38 1280 98 07 48 1280 98 05 02 1210 98 01 58 1210 97 58 32 1230 97 55 29 1140 97 52 53 1230 97 49 4'3 1200 97 46 04 1370 97 43 20 1280 97 40 27 1310 97 43 20 1360 97 46 35 1280 97 49 45 1220 97 52 52 1220 97 56 12 1140 97 59 18 1200 98 02 20 1140 98 05 28 1240 98 08 36 1260 96 11 48 1380 98 14 56 1220 98 18 05 1400 98 21 17 1440 98 24 26 1440 98 18 07 1380 98 15 18 1360 98 12 12 1180 98 09 01 1220 98 06 00 1190 98 02 31 1120 97 59 38 1130 97 56 08 1150 97 52 52 1220 97 49 13 1200 97 47 51 1250 97 44 26 1250 97 41 16 1310 97 41 18 1300 97 44 06 1250 97 46 48 1250 7 b . T7ABLE 5.5 Raingage Network Station Data, Chickasha, Oklahoma (Continued) Latitude Longitude Elevation Gage No. Degrees Minutes Seconds Degrees Minutes Seconds Feet 96 35 03 33 97 50 01 1150 97 35 02 47 97 53 33 1110 98 35 03 22 97 57 05 1090 99 35 03 55 98 00 19 1110 100 35 04 20 98 02 26 1140 101 35 04 32 98 06 45 1200 102 35 04 18 98 09 49 1180 103 35 04 46 98 12 52 1180 104 35 05 42 98 15 17 1190 105 35 02 37 98 16 04 1310 106 35 02 22 98 13 59 1220 107 35 02 17 98 09 50 1240 108 35 01 51 98 06 36 1350 109 35 01 45 98 03 27 1230 110 35 00 50 98 00 22 1240 111 35 00 45 97 57 09 1210 112 35 00 55 97 53 41 1050 113 35 00 04 97 50 34 1100 114 35 00 03 97 47 38 1150 115 34 59 44 97 44 29 1150 116 35 00 05 97 41 27 1160 117 34 57 05 97 42 22 1040 118 34 57 01 97 45 32 1100 119 34 56 49 97 47 36 1040 120 34 57 36 97 50 18 1060 121 34 57 31 97 53 55 1120 122 34 58 39 97 57 08 1170 123 34 58 16 98 00 48 1220 124 34 58 26 98 03 28 1280 125 34 59 08 98 07 39 1380 126 34 59 00 98 18 49 1340 127 34 59 46 98 14 01 1380 128 34 59 23 98 17 12 1340 129 34 59 59 98 20 51 1360 130 34 57 23 98 17 05 1410 131 34 56 53 98 13 59 1490 132 34 56 30 98 10 42 1400 133 34 56 31 98 07 30 1400 134 34 56 06 98 04 31 1260 135 34 55 39 98 01 10 1180 136 34 55 37 97 58 01 1200 137 34 54 57 97 55 02 1140 138 34 54 46 97 51 21 1170 139 34 54 48 97 48 01 1060 140 34 54 40 97 44 55 1040 141 34 51 55 97 45 31 1140 142 34 51 03 97 48 40 1000 76. TABLE 5.5 Raingage Network Station Data, Chickasha, Oklahoma (Continued) Latitude Longitude El e v a t i o n Gage No. Degrees Minutes Seconds Degrees Minutes Seconds Feet 143 34 52 34 97 51 49 1160 144 34 52 47 97 54 59 1260 145 34 53 03 97 58 28 1220 146 34 53 09 98 01 22 1180 147 34 53 01 98 04 30 1410 148 34 54 03 98 07 40 1400 149 34 54 05 98 11 20 1360 150 34 54 21 98 15 02 1400 151 34 54 48 98 17 40 1450 152 34 51 39 98 15 02 1360 153 34 51 29 98 11 54 1340 154 34 51 19 98 08 01 1320 155 34 50 26 98 02 10 1290 156 34 50 33 97 57 28 1310 157 34 50 11 97 54 48 1250 158 34 47 00 97 55 55 1340 159 34 47 35 97 59 03 1420 160 34 48 05 98 02 12 1350 161 34 47 51 98 04 59 1400 162 34 48 48 98 08 30 1310 163 34 49 03 98 11 42 1340 164 34 49 34 98 15 49 1310 165 34 46 58 98 08 45 1300 166 34 45 14 98 05 21 1280 167 34 45 16 98 02 10 1300 168 34 45 14 97 59 38 1310 169 34 54 16 97 52 53 1160 170 34 55 40 97 51 53 1160 171 34 56 33 97 51 14 1160 172 34 57 12 97 51 00 1100 173 35 02 45 97 54 38 1090 174 35 06 16 98 02 33 1120 175 35 08 47 98 07 42 1180 TABLE 5.6a OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN CHICKASHA WATERSHED, OKLAHOMA SUBREGION I (SELECTION BASED ON SINGLE AND BIVARIATE ENTROPY CONCEPT) No. of Stations to be Retained Opt. Infor-mation In t e n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 0.730 2.8 0.372 54 2 1.151 5.3 0.588 1,54 3 1.394 8.3 0.712 1,14,54 4 1.542 11.1 0.787 1,14,24,54 5 1.624 13.9 0.829 1,14,24,26,54 6 1.688 16.7 0.866 1,14,15,24,26,54 7 1.710 19.4 0.872 1,14,15,18,24,26,54 8 1.734 22.2 0.885 1,12,14,15,18,24,26,54 9 1.746 25 .0 0.891 1,12,14,15,18,24,26,45,54 10 1.755 27.7 0.896 1,12,13,14,15,18,24,26,45, 54 TABLE 5.6b OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN CHICKASHA WATERSHED, OKLAHOMA SUBREGION I (SELECTION BASED ON DYNAMIC ELIMINATION METHOD) No. of Stations to be Retained Opt. Infor-mation In t e n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 0.730 2.8 0.372 54 2 1.151 5.3 0.588 1,54 3 1.394 8.3 0.712 1,14,54 4 1.542 11.1 0.787 1,14,24,54 5 1.632 13.9 0.829 1,5,14,24,54 6 1.696 16.7 0.866 1,5,14,15,24,54 7 1.739 19.4 0.887 1,5,14,15,18,24,54 8 1.770 22.2 0.904 1,5,9,14,15,18,24,54 9 1.785 25.0 0.911 1,5,8,9,14,15,18,24,54 10 1.785 27.7 0.913 1,5,8,9,10,14,15,18,24,54 78. TABLE 5.7a OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN CHICKASHA WATERSHED, OKLAHOMA SUBREGION II (SELECTION BASED ON SINGLE AND BIVARIATE ENTROPY CONCEPT) No. of Stations to be Retained Opt. Infor-mation Inte n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 0.824 2.6 0.360 29 2 1.285 5.3 0.562 29,81 3 1.614 7.9 0.705 29,81,87 4 1.788 10.5 0.782 29,76,81,87 5 1.896 13.2 0.829 28,29,76,81,87 6 1.960 15.8 0.857 28,29,30,76,81,87 7 1.990 18.4 0.867 27,29,30,44,76,81,87 8 2 .023 21.1 0.884 27,28,29,30,44,76,81,87 9 2 .033 23 .7 0.889 27,28,29,30,44,76,80,81,87 10 2.033 26.3 0.889 27,28,29,30,44,54,76,80,81, 87 TABLE 5.7b OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN CHICKASHA WATERSHED, OKLAHOMA SUBREGION II (SELECTION BASED ON DYNAMIC ELIMINATION METHOD) No. of Stations to be Retained Opt. Infor-mation Inte n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 0.824 2.6 0.360 29 2 1.285 5.3 0.562 29,81 3 1.614 7.9 0.705 29,81,87 4 1.788 10.5 0.782 29,81,87,76 5 1.896 13.2 0.829 29,81,87,76,28 6 1.975 15.8 0.863 29,81,87,76,28,39 7 2.021 18.4 0.884 29,81,87,76,28,39,44 8 2.064 21.1 0.902 29,81,87,76,28,39,44,38 9 2.103 23.7 0.920 29,81,87,76,28,39,44,38,33 10 2.137 26.3 0.934 29,81,87,76,28,39,44,38,33, 42 TABLE 5.8a OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN CHICKASHA WATERSHED, OKLAHOMA SUBREGION III (SELECTION BASED ON SINGLE AND BIVARIATE ENTROPY CONCEPT) No. of Stations to be Opt. Infor-mation In t e n s i -f i c a t i o n Ratio ' Normed Inf. Rate Opt. Station Locations Retained 1 0.783 3.5 0.376 115 2 1.251 5.1 0.602 35,115 3 1.484 7.6 0.714 35,69,117 4 1.649 10.3 0.789 35,69,115,117 5 1.740 17.2 0.837 35,69,88,115,117 6 1.798 20.7 0.865 35,64,69,88,115,117 7 1.838 24.1 0.885 35,36,64,69,88,115,117 8 1.868 27.6 0.899 35,36,64,69,88,114,115,117 9 1.888 31.0 0.909 35,36,63,64,69,88,114,115, 117 10 1.891 34 .5 0.910 35,36,62,63,64,69,88,114, 115,117 TABLE 5.8b OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN CHICKASHA WATERSHED, OKLAHOMA SUBREGION III (SELECTION BASED ON DYNAMIC ELIMINATION METHOD) No. O f Stations to be Retained Opt. Infor-mation Inte n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 0.783 3.5 0.376 115 2 1.251 5.1 0.602 115,35 3 1.484 7.6 0.714 115,35,117 4 1.649 10.3 0.789 115,35,117,69 5 1.746 17.2 0.837 115,35,117,69,34 6 1.822 20.7 0.877 115,35,117,69,34,88 7 1.881 24.1 0.905 115,35,117,69,34,88,92 8 1.923 27.6 0.925 115,35,117,69,34,88,92,67 9 1.958 31.0 0.942 115,35,117,69,34,88,92,67, 93 10 1.984 34 .5 0.955 115,35,117,69,34,88,92,67, 93,63 80. TABLE 5.9a OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN CHICKASHA WATERSHED, OKLAHOMA SUBREGION IV (SELECTION BASED ON SINGLE AND BIVARIATE ENTROPY CONCEPT) No. of Stations to be Retained Opt. Infor-mation In t e n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 0.832 5.0 0.397 119 2 1.217 10.0 0.581 119,98 3 1.490 15.0 0.713 98,137,140 4 1.644 20.0 0.784 98,119,137,140 5 1.740 25.0 0.830 98,110,119,137,140 6 1.821 30.0 0.869 98,110,119,137,140,141 7 1.872 35.0 0.893 98,110,119,137,140,141,142 8 1.906 40.0 0.893 98,110,119,137,139,140,141 142 9 1.928 45.0 0.909 98,110,119,137,139,140,141 142,143 10 1.933 50.0 0.922 98,110,119,137,138,139,140 141,142,143 TABLE 5.9b OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN CHICKASHA WATERSHED, OKLAHOMA SUBREGION IV (SELECTION BASED ON DYNAMIC ELIMINATION METHOD) No. of Stations to be Opt. Infor-mation In t e n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations Retained 1 0.832 5.0 0.397 119 2 1.217 10.0 0.581 119,98 3 1.490 15.0 0.713 119,98,137 4 1.644 20.0 0.784 119,98,137,140 5 1.740 25.0 0.830 119,89,137,149,110 6 1.821 30.0 0.869 119,89,137,140,110,141 7 1.887 35.0 0.900 119,98,137,140,110,141,120 8 1.938 40.0 0.925 119,98,137,140,110,141,120 109 9 1.977 45.0 0.943 119,98,137,140,110,141,120 109,143 10 2.006 50.0 0.957 119,98,137,140,110,141,120 109,143,99 81. TABLE 5.10a OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN CHICKASHA WATERSHED, OKLAHOMA SUBREGION V (SELECTION BASED ON SINGLE AND BIVARIATE ENTROPY CONCEPT) No. of Stations to be Retained Opt. Infor-mation In t e n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 0.850 2.3 0.351 153 2 1.237 4.5 0.511 153,161 3 1.506 6.8 0.623 153,161,165 4 1.698 9.1 0.702 124,153,161,165 5 1.813 11.4 0.749 124,153,161,162,165 6 1.889 13 .0 0.781 124,153,157,161,162,165 7 1.921 15.9 0.794 124,153,157,160,161,162,165 8 1.944 18.2 0.804 124,153,157,160,161,162,165 167 9 1.952 20.9 0.807 124,153,154,157,160,161,162 165,167 10 1.952 22.7 0.807 124,153,154,155,157,160,161 162,165,167 TABLE 5.10b OPTIMUM INFORMATION TRANSMISSION AND THE OPTIMAL STATION LOCATIONS IN CHICKASHA WATERSHED, OKLAHOMA SUBREGION V (SELECTION BASED ON DYNAMIC ELIMINATION METHOD) No. of Stations to be Retained Opt. Infor-mation I n t e n s i -f i c a t i o n Ratio Normed Inf. Rate Opt. Station Locations 1 0.850 2.3 0.351 145 2 1.285 4.5 0.531 145,105 3 1.563 6.8 0.646 145,105,167 4 1.758 9.1 0.727 145,105,167,164 5 1.887 11.4 0.780 145,105,167,164,154 6 1.967 13.6 0.813 145,105,167,164,154,162 7 2.038 15.9 0.842 145,105,167,164,154,162,129 8 2.097 18.2 0.867 145,105,167,164,154,162,129, 131 9 2.149 20.5 0.888 145,105,167,164,154,162,129, 131,157 10 2.181 22.7 0.902 145,105,167,164,154,162,129, p.31,157,125 82. \ I • * \ ? 5 V I I ? \ I \2<-\ \ lit f I I I V « I , • ID SI — ^ -« w • -.TVTTUE > ( / / \ I ^ s. S I x i r v to .« / A*. f» •» i J00 fJic.fi J J jot f JV •07> ^ . C M C K A S M ^ J * * -ay / X  / * ~ Z s LEGEND S U B R E O I O H B O U N D A R I E S •v^v MTERSrCD MUNDMCS — wnrmur MM GMt m T K M tS ernes - TOWMS *ASH»TA HIVEB EKPERIMtNTA,. WATERSHED C H I C K A S H A . O K L A H O M A R A I N G A G E N E T W O R K FIG.5 .1- RAIN GAGE NETWORK,WASHITA RIVER EXPERIMENTAL WATERSHED, CHICKASHA,OKLAHOMA. 83. CHAPTER VI DERIVATIONS OF INFORMATION TRANSMISSION FROM MULTIVARIATE CONTINUOUS DISTRIBUTIONS The computation of the j o i n t entropy terms i n di s c r e t e form i s straightforward but, i n dealing with the large number of g r i d points, dimensional d i f f i c u l t i e s do a r i s e . Therefore, more a n a l y t i c a l methods for computing j o i n t entropies, which involve the use of f i t t e d m u l t i v a r i a t e continuous d i s t r i b u t i o n s , were explored. Single and b i v a r i a t e entropy derivations for the univariate and b i v a r i a t e normal, lognormal, gamma, exponential, and extreme value d i s t r i b u t i o n s were i n i t i a l l y investigated. This was then extended to mu l t i v a r i a t e normal and lognormal d i s t r i b u t i o n s to compute multivariate j o i n t entropies. 6.1 Univariate Entropy Derivation for Continuous D i s t r i b u t i o n s The entropy de r i v a t i o n for the normal d i s t r i b u t i o n has been explored previously and i s l i s t e d i n the l i t e r a t u r e (62) . Due to the close r e l a t i o n s h i p between the normal and lognormal d i s t r i b u t i o n s , the entropy for lognormal can be obtained by transformation. The entropy derivations for other d i s t r i b u t i o n s such as gamma, beta, and extreme value have not been previously evaluated and are presented i n t h i s chapter. The d e t a i l s of these derivations are l i s t e d i n Appendix A. The r e s u l t s are summarized i n Table 6.1. 84. TABLE 6 .1 UNIVARIATE ENTROPY FOR SOME COMMON CONTINUOUS DISTRIBUTIONS Name Pr o b a b i l i t y Density Function Expected Value,E(X) Variance Var(X) Entropy H(X) Normal f (x;y,G) N = — — e - ( x - y ) 2 a / 2 i 2a2 —00< x,y < 0 0 o < a 2 y a 2 0.5Jon(2TT0 2 )+0.5 Log-normal fLN te-'Vx' 0^ -(£nx-y ) 2 _ e : Y_ xa y /2TT 2 0 2 a =£n /(y 2+a 2 ) / y 2 y x x x y =£n / y 4 / ( y 2+a z) y X X X y +a e y y e 2 ( y y + a y ) • ( e 2 a Y - l ) hin ( 2 f T a y 2 ) + 0 . 5 + E(£nX) where E(*) i s the expected value Gamma f G(x;a,V) V - l -x/a -V x e a Hv) 0<x,a,v Va Va 2 £n T(V)+V+£na -(V-l) VJ(V) where ip(V) i s p s i function Beta f_(x ; a f B) 15 T(a+B) ^ d - x ) 3 " 1 T(a) T(B) U 0<x<l 0<a,B a aB inT(a)+inT($) - i n Ha+B) -(a -1) [i|.(a)Hj,«x+B)] -(B-l) [VJ(& -ip(a+B)l a+B (a+B+D (a+B)2 Extreme Value f (x) EXT -X -x -e = e e - 0 0 < x < 0 0 zero one E(e" x) where E i s the expected value Expon-e n t i a l (x;B) rEXP x B B B2 £nB+l 85. 6.2 Information Transmission at the B i v a r i a t e Level The information transmission by the random v a r i a b l e Y about X i s given by T(X;Y) = H(X) + H(Y) - H(X,Y) (6.1) where H(X) and H(Y) are the univariate entropies of X and Y r e s p e c t i v e l y . H(X,Y) i s the j o i n t b i v a r i a t e entropy for X and Y. Information transmission at t h i s b i v a r i a t e l e v e l involves computing univariate entropy f o r X and Y and t h e i r j o i n t entropy using the b i v a r i a t e p r o b a b i l i t y density function. This section demonstrates how b i v a r i a t e entropy and the corresponding b i v a r i a t e information transmission are computed from normal, lognormal, gamma, and extreme value d i s t r i b u t i o n s . 6.2.1 Normal d i s t r i b u t i o n The b i v a r i a t e normal p r o b a b i l i t y density function i s defined as: 1 Exp f (x,y;u ,y ,0 ,0 ) = zr^m N V V x y ^ a / 2 x y r X - y * ) 2 - 2 P ( X l ! V a. 2 ( l - p z ) { x y-u J _i (6.2) yx,ax are r e s p e c t i v e l y the mean and standard deviation of X y ,0 are re s p e c t i v e l y the mean and standard d e v i a t i o n of Y y y p i s the c o r r e l a t i o n c o e f f i c i e n t between X and Y. The marginal p r o b a b i l i t y density functions of X and Y are also normally d i s t r i b u t e d with parameters (yx,o*x) and (yy,ay) r e s p e c t i v e l y . 86. The entropies H(X) and H(Y) are computed using these marginal d i s t r i b u -tions and the j o i n t entropy H(X,Y) using the b i v a r i a t e density function of Equation (6.2) which have been derived i n the l i t e r a t u r e (62). The s i m p l i f i e d r e s u l t s are: Marginal entropy: H(X) = 0.5 £n(2ira x 2) + 0.5 Marginal entropy: H(Y) = 0.5 £n(2Tra y 2) + 0.5 2 -7 J o i n t entropy: H(X,Y) = 0.5 £n((2u) 2 |P |) + 1.0 where, |P| i s the determinant of the variance covariance matrix of X and Y which i s defined as follows: a 2 poo x x y poo a 2 x y y = a 2 a 2 (l-p 2) x y Hence H(X,Y) = 1.0 + 0.5£n(2TT)2 + 0.5£n a 2 + 0.5£n a 2 x y + 0.5£n(l-p2) (6.3) Substituting the values of H(X), H(Y), and H(X,Y) i n Equation (6.1), we get: T(X;Y) = -0.5 £n(l-p2) 6.2.2 Lognormal d i s t r i b u t i o n If the va r i a b l e s X and Y are lognormally d i s t r i b u t e d , then the transformed v a r i a b l e s U = &n(X) and V = &n(Y) are normally d i s t r i b u t e d . Using t h i s p r i n c i p l e , the information transmission between the two lognormally d i s t r i b u t e d v a r i a b l e s are defined as: T(X;Y) = -0.5 £n(l-p u v 2) where 87. P. i s the c o r r e l a t i o n c o e f f i c i e n t between the transformed uv variables U and V. In the f i e l d of applied science, the most frequently used d i s t r i b u t i o n i s the normal d i s t r i b u t i o n due to both i t s s i m p l i c i t y and the appropriateness of the additive property. But, due to the skewness which i s common i n hydrologic time series data, the a p p l i -c a b i l i t y of the normal d i s t r i b u t i o n i s doubtful and quite s e n s i t i v e to errors, p a r t i c u l a r l y i n the t a i l s of the d i s t r i b u t i o n . Hence, i t i s useful to investigate information transmission for gamma, extreme value, and the exponential d i s t r i b u t i o n s . 6.2.3 Gamma d i s t r i b u t i o n There are various forms of the b i v a r i a t e gamma d i s t r i b u t i o n (39,46) but, due to i t s t r a c t a b l e marginal d i s t r i b u t i o n , the following form i s adopted here: f„(x,y;a,p,q) = _(p+q) p - i -, x e ay (y-x) q - l (6.6) r(p) Hq) y >x > 0, a,p,q > 0 The marginal p r o b a b i l i t y density functions of X and Y are also gamma d i s t r i b u t e d which are as follows: f (x;p,a) p-1 - i x e ax (6.7) The entropies H(X) and H(Y) are computed using the marginal Equation (6.7) with the a i d of the entropy formulae i n Appendix A and are as follows: 88. H(X) = aE(X) + I n R p ) - p£na -(p-l)E(AnX) H(Y) = a E(Y) + An T(p+q)-(p+q)£na-(p+q-1)E(£nY) (6.8) The j o i n t entropy H(X,Y) i s computed from the b i v a r i a t e p r o b a b i l i t y density function of Equation (6.6) as follows: H(X,Y) = - f (x,y;a,p,q)£nf (x,y;a,p,q)dxdy . (6.9a) Taking natural logarithm of f ( ) and su b s t i t u t i n g i n Equation (6.1), G we get: H(X,Y) 0 0 f(p+q)£na + (p-l)£nx - ay 0 t +(q-l» l n(y-x) - £nr ( p) - ZnT(q)j ,y;a,p,q)dxdy •(p+q) Ana - (p-1) E(£nX) + a E(Y) •(q-1) E(£n(Y-x)) + £nT(p) + £ nT(q) (6.9b) Substituting the values of H(X), H(Y) and H(X,Y) i n Equation (6.1), we get, f o r the b i v a r i a t e gamma case: T(X;Y) = i n + a E(x) - p in a - (p+q-1) E An(Y) + (q-1) E(£n(Y-X)) (6.10) 6.2.4 Exponential d i s t r i b u t i o n Gumbel has developed three b i v a r i a t e exponential d i s t r i b u t i o n s whose marginals are standard exponentials (26,46). Among these, the following i s the most suited to entropy computation: f E X p ( x , y ) = EXP(-x-y){l+a(2e" X-l)(2e" y-l)} -1 < a < 1 (6.11) 89. with the marginals E X P X £ _ (6.12) W y ) = e _ Y The requirement of the above b i v a r i a t e p r o b a b i l i t y density function i s that the c o r r e l a t i o n c o e f f i c i e n t between X and Y should be i n the range of and The entropies H(X) and H(Y) are computed using the derived r e l a t i o n s h i p l i s t e d i n Table 6.1. For the standardized variables with mean unity and density functions of Equation (6.12), the derived entropies H(X) and H(Y) are as follows: H(X) = H(Y) = 1 (6.13) The j o i n t entropy H(X,Y) i s derived as follows: .00 »00 H(X,Y) = - f ( x , y H n f (x,y)dxdy (6.14a) J 0 J 0 EXP EXP Taking natural logarithm of f ( ) of Equation (6.11) and EXP su b s t i t u t i n g i n Equation (6.14a), we get: -CO .00 H(X,Y) = - {-x-y+£n[l+a(2e" X-l)(2e" Y-l)]} f E x p ( x , y ) d x d y ) Jo J o = E(X) + E(Y) - E £n[l+a(2e~ X-l)(2e" Y-l)] where E( X ) i s the expected value of ( X ) . But, E(X) = E(Y) = 1 Therefore: H(X,Y) = 2 - E(£n[l+a(2e" X-l) (2e" Y-l) ]) (6.14b) 90. Substituting the values of H(X), H(Y) and H(X,Y) i n Equation (6.1), we get the b i v a r i a t e information transmission f o r exponential d i s t r i b u t i o n as follows: T(X;Y) = E(£n[l+a(2e" X-l)(2e~ Y-l)]) (6.15) 6.2.5 Extreme value d i s t r i b u t i o n At the b i v a r i a t e l e v e l , Type A and Type B d i s t r i b u t i o n s are considered: Type A D i s t r i b u t i o n : The b i v a r i a t e p r o b a b i l i t y density function of the reduced va r i a b l e s with the assoc i a t i o n parameter '6' i s given below (10,11): -(x+y) r, r., 2x 2y. . x y.-2 2(x+y) . x y, -3 f . J x , y ; ) = e -' [1-9 (e +e *) (e +e*) +26e (e +e*) + 9 2 e 2 ( x + y ) ( e W r 4 ] -Exp [ - e ^ - e ^ + e ^ + e V 1 ] . .(6.16) with marginals as -x / i -x - e W x ) = e e (6.17a) -y -e~ Y f (y) = e y e EXT The entropies H(X) and H(Y) are computed with the a i d of entropy formulae l i s t e d i n Table 6.1 and are as follows: H ( X ) = E ( e " X ) ) (6.17b) H(Y) = E(e" Y) The j o i n t entropy H(X,Y) i s computed as follows: ,oo roo H(X,Y) = - f E X T(x,y;9) Anf E X T(x,y,-e)dxdy (6.18a) 91. Substituting the value of i n f r v „ ( ) i n Equation (6.18a) using EXT Equation (6.16) and si m p l i f y i n g , we get: H(X,Y) = E(e~ X) + E(e~ Y) + 6/E (e X+e Y) - E(Z) (6.18b) where n r-, n , 2x 2y, . x y. -2 - 2 (x+y) . x y.-3 A2 2 (x+y) . x y. -4-, Z = £n{l - 6(e +e 1) (e +ey) +20e * (e +e^) +0 e * (e +e^) } Information i n t h i s case i s : T(X;Y) = E(Z) + 6/E[ (e X+e Y) ] . . (6.19) Type B D i s t r i b u t i o n : The b i v a r i a t e p r o b a b i l i t y density function with a s s o c i a t i o n parameter 'm' i s as follows: -2+- 1/m f E X T(x,y;m) = e *'(e +e *) m r , . -mx -my. -. -m (x+y)^e-mx+e-myj im-l+(e +e ) s , , ( 6 . 2 0 ) 1/m „ . . r # -mx -my, , Exp[-(e +e *) ] The marginal p r o b a b i l i t y density functions i n t h i s case are also of the standard type as given f o r the type A d i s t r i b u t i o n . Information transmission, T(X;Y), i s s i m p l i f i e d to: 1/m T(XfY) = (2-k E ( e - m X + e ^ Y ) - E [ (m-1) + (e^+e"™*) ] m 1/m ( 6 . 2 1 ) „. -mx -my. _,. -x. . -y. + E(e +e J) -E(e ) - E(e *) 6.3 Information Transmission at the M u l t i v a r i a t e Level 6.3.1 Normal d i s t r i b u t i o n The m u l t i v a r i a t e normal density function i s defined as: f N U ; e x ' V = W~ J" (EXP - J s U - e ) ^ " 1 ^ ) ) . . (6.22) N X X ( 2 T T ) N / 2 ( | Z | x ) ^ X X 92. where x = E x ^ -1 (X,, X„,...,X ). An N-dimensional vector of variables. 1 2 N (x, , x„,...,x ). A vector of outcomes. 1 2 N Variance covariance matrix of va r i a b l e s (X l fX_,...X ), 1 2 N Determinant of E x Mean vector of (X.. ,X_, ... ,X ) 1 2 N Transpose of (•) Inverse of E__ The j o i n t entropy H(X^,X2,...,X^) i s : H(X^,X2,••.,X^) — f N(x:9 x,E x)£nf N(x:e x,E x)dx and s i m p l i f i e s to: H ' ^ 2 ' " ' " ' I £n(2ff) + Un(|E x|) + | (6.23) S i m i l a r l y , i f Y = (Y ,Y ,...,Y ) i s a vector of M v a r i a b l e s , 1 2 M with p r o b a b i l i t y density function f N ( y ; 6 y , £ y ) , then: ( Y r Y 2 Y M) = |Aln(27T) + ^n(|E y|) + f (6.24) The j o i n t entropy of (X ,X 2»...,X N) and ( Y ^ Y ^ ..., Y M) also s i m p l i f i e s to: H(X 1,X 2 X N,Y l fY 2 Y M) = ^ £ n ( 2 T T ) = U n | E | + ^ . .(6.25) where E = xy T xy I i s the covariance matrix (NxM) between the X and Y vectors, xy The information transmission by Y about X i s as follows: \l | |I | T(X;Y) = h An — 2t_ (6.26) 6.3 . 2 Lognormal d i s t r i b u t i o n Let X = (X,,X„, ,X ) be a vector of N va r i a b l e s with a 1 2 N multi v a r i a t e lognormal d i s t r i b u t i o n and l e t Z = ( Z ^ r Z ^ , . . . , Z ) be a transformed N-variable vector such that Z. = An X . . Then (Z.. , Z_ , . .., Z ) 3 J 1 2 N are normally d i s t r i b u t e d with mean vector 8 and variance covariance 1 z matrix £ . z Therefore: N H(X. ,x_, ...,X ) = ^  An(2TT) + f + W n |Z | + £ E(Z.) . . . . (6.27) 1 = 1 where E(Z.) i s the expected value of Z.. l l S i m i l a r l y the j o i n t entropy of log-normally d i s t r i b u t e d v a r i a b l e s ( w - - y i s M I(Y.,Y-,...,Y ) =^-An(2TT) + | . + i s£n|l | + E E (W.). . . . (6.28) 1 Z Fl Z £ W . v=l where E i s the variance-covariance matrix of the transformed w variables (W.,W_,...,WW), and W. = An(Y.) 1 2 M l . 1 E(ftL) i s the expected value of W^ . 94. The j o i n t entropy H(X n,X ,...,X ,Y ,Y ,...,Y ) i s as follows: J 1 2 N 1 2 M H(X 1,X 2,...,X N, Y X,Y 2,-..,Y M) N+M 2 Sln(2v) + ™ + U n ( | Z | ) (6.2 N M + Z E (Z.) + Z E ( W . ) i = l 1 i = l 1 where Z i s the variance covariance matrix of the transformed vector (Z n,Z ,...,Z ) and (W,,W ....,W„). Therefore the mul t i v a r i a t e 1 2 N 1 2. M information transmitted i n t h i s case i s : T(X l fX 2,...,X N? Y ,Y2,...,YM) = hln z • | z •7 w' Z (6.3 95. CHAPTER VII COMPARISON OF SHANNON'S AND FISHER'S INFORMATION MEASURES IN THEORY AND IN NETWORK DESIGN APPLICATIONS 7.1 Comparative Study In the case of Fisher's information theory, information increases with increased knowledge concerning a p a r t i c u l a r parameter that has to be estimated from a sample. By contrast, Shannon's information theory concerns the reduction i n uncertainty provided by a message a f t e r i t s transmission. Shannon's information has a d e f i n i t e upper bound which i s r e l a t e d to the channel capacity or the message uncertainty while the upper bound i n the case of Fisher's information i s not always f i n i t e . Fisher's measure i s confined to s t a t i s t i c a l inference but Shannon's concept has been recognized as being relevant i n physical as well as s t a t i s t i c a l contexts i n a number of d i s c i p l i n e s and i s also i n t e r p r e t a b l e i n stochastic processes. The derived b i v a r i a t e and multivariate forms of Fisher's and Shannon's information measures, i n the context of hydrologic network design, are compared i n the following sections. 7.1.1 B i v a r i a t e case Consider two hydrologic v a r i a b l e s , X. = (x. 1,x. ,...,x. ) 3 I f 1 312- 3,Nj+N2 and X. = (x. ,,x. x. „ ) with (Nn+N?) and N-, observations l l , 1 i , z x, N^ ± x res p e c t i v e l y . A f t e r c o l l e c t i n g N^ observations concurrently on Xj and X., the data c o l l e c t i o n scheme f o r the v a r i a b l e X. i s discontinued x x but i s extended to N2 additi o n a l measurements on X j . Based on the concurrent N-, observations, a l i n e a r regression, with X^  as dependent 96. v a r i a b l e and X_. as independent v a r i a b l e , i s c a r r i e d out and y i e l d s the l i n e a r model: x._ = X. + b(x. - X.) i,k 1 3,k j (7.1) Us ing the above equation, N 2 estimates of X^ based on the ad d i t i o n a l observations of X^ are obtained. Combining the observa-tions and a d d i t i o n a l estimates, the weighted estimates of the mean i s derived as follows: y i = ( N1 X i + N2 X i ) / ( N 1 + N 2 ) where, N l X = E x . ,/N ; X i k = 1 i,k 1 i N 1 + N 2 E k=N x. , /N„ i,k 2 1+1 The variance of u., denoted by O , i s as follows: 1 m 1 m N, 1 " ZTT-- {P • " ( 1 ~ p'.)/N,-3} N 1+N 2 "13 1^3 1 (7.2) where, a , i s the variance of the estimates of the mean f o r the V N i random sequence of X^ with observations. the c o r r e l a t i o n c o e f f i c i e n t between X. and X. based on concurrent observations. Fisher's information I f ( X . ; X J , which i s equal to the r a t i o of the variance of the mean for a purely random sequence and the variance of the mean for the sequence of i n t e r e s t , i s defined as follows: I f ( X i ; X j ) = 1/ = 1 N 2-N 1 + -r { ( i - p 2 . ) ( — - ) } N l 1 3 3 _ N 1 J for a l l i ^ j 1 . • • - (7" 3) The information transmission by X.. about X^ using Shannon's information concept, has been d e t a i l e d i n Chapter I II i n terms of single and j o i n t entropies. The equivalent form here i s : T(X.;X.) = H(X.) +H(X.) - H (X . , X . ) f or a l l i ^ j ^  1 3 1 3 1 J = H(X.) f o r i=j I (7.4) The information transmission, as derived by Matalas i n Equation (7.3), i s r e s t r i c t e d to the assumptions of l i n e a r i t y and normal-i t y . These assumptions may not be applicable i n hydrologic network design due to n o n l i n e a r i t y and skewness which commonly e x i s t s i n time serie s data. Shannon's information transmission c r i t e r i o n , however, i s not r e s t r i c t e d to the above assumptions. Optimum s t a t i o n sets can be obtained by both of these methodologies by d e f i n i n g the objective function and the co n s t r a i n t s . The objective for network design, applied here, i s to s e l e c t the optimum s t a t i o n locat i o n s for the reduced network so that the maximum possible informa-t i o n concerning the new ungauged locations i s transmitted. The objective f o r a network of "m" stations proposed by Matalas i s (47): m Z = Max E I^fX. ;X .) 6 (7.5) i , j = l ' 1 3 ^ where, Z i s c a l l e d the objective function 6. . i s a dec i s i o n v a r i a b l e which i s zero i f the s t a t i o n i=j i s 1 / 3 to be discontinued or, f o r i / j , information cannot be transferred from j to i . On the other hand, 6. . i s equal to one when the information i s t r a n s f e r r e d from j to i . 98. Using Shannon's information concept, as discussed i n Chapter I I I , a s i m i l a r objective function can be written as: m Z = Max Z T(X.;X.)6. (7.6a) i , j = l 1 3 ^ The following two constraints are common to both forms of network objectives described above: Information T r a n s f e r r a b i l i t y Constraint: This indicates that information from only one s t a t i o n j can be transferred to another s t a t i o n i . m Z 6. . = 1 (7.6b) j=l 1 , 3 Budgetary Constraint: If the maximum number of stations which can be retained are n, due to a budgetary constraint, then m Z 6. . = n (7.6c) 1=1 1 ' 1 7.1.2 M u l t i v a r i a t e case Fisher's information concept has been extended to the multiple regression case where estimation at a point l o c a t i o n i s based on data from a number of s t a t i o n locations (47,48). This modified form of Fisher's information equation i s s i m p l i f i e d , i n terms of a multiple c o r r e l a t i o n c o e f f i c i e n t , as follows: I,(X.;X.X. ,...,X ) = 1/ f l 3 k n n-(N -2) p 2 W - l " i „ ?) } 2 n-(N -2) (7.7) where, P. i s the multiple c o r r e l a t i o n c o e f f i c i e n t between the i t h 99. v a r i a b l e X^ and a s p e c i f i c set q of n independent v a r i a b l e s ( x j f x k , . . . , x n ) . To carry out the analysis to s e l e c t the optimum n s t a t i o n locations from a dense network of m locati o n s , the objective function i s defined as follows: m Z = Maximizing E I. (X. ;X . ,X, , . .. ,X ) (7.8) . , f i 1 k n 1=1 j , k , l , . ...,n Subject to I,(X.;X.,X. ,...,X )=0 i f p 2 < " . f l j k n i * n N^-2 S t r i c t l y speaking, hydrologic network design using Fisher's information measure i s r e s t r i c t e d to s e l e c t i n g a single s t a t i o n l o c a t i o n . As i s shown above, the theory has been extended for m u l t i - g r i d point l o c a t i o n s by f i r s t applying an independency assumption, and then by measuring the summed information f o r s t a t i o n sets. Shannon's information transmission i s applicable i n a true m u l t i -v a r i a t e form. I t takes the covariance structure into consideration both between the g r i d point l o c a t i o n s and s t a t i o n locations and also within the s t a t i o n set locations as well as g r i d point l o c a t i o n s . If (X WX_,...,X ) are var i a b l e s representing the hydrologic 1 2 m outcomes at the point locations set and (X^,X_., . .. ,X^) at s t a t i o n set and s t a t i o n set i s subset of point l o c a t i o n s , then from Equation (3.27), the objective function i s defined as: Max T(X,,X„,...,X ;X ,X ,...,X ) = 1 2 m l 3 n Max H(Xi#X.,...,X ) (7.9) i , j , ...,n 3 n 100. 7.2 Estimation Performance C r i t e r i a The performance of networks designed by Shannon's and Fisher's methodologies described i n Section 7.1 were compared on the basis of th e i r estimation accuracy. M u l t i p l e l i n e a r regression and Kalman F i l t e r estimation models were used for t h i s purpose. 7.2.1 Multiple l i n e a r regression The p o t e n t i a l stations were i d e n t i f i e d by both methodologies on the basis of an i n i t i a l set of observations at a l l of the gr i d point locations i n a basin. The values associated with the events at the optimal stations act as the independent v a r i a b l e s . The values associated with the events at common g r i d point locations not appearing i n the optimal networks s p e c i f i e d by ei t h e r method act as common dependent var i a b l e s . Regression c o e f f i c i e n t s were computed using the f i r s t observations. Estimated values were obtained for the dependent v a r i a b l e s using the subsequent observations of the independent variables and compared with the true values. The error variance was then determined for both methods. 7.2.2 Estimation by Kalman F i l t e r models A d e t a i l e d formulation and the parameter estimation for the Kalman F i l t e r model i s discussed i n Chapter VIII. Using optimum stations i d e n t i f i e d by each network design method as independent va r i a b l e s and the common g r i d point locations as the dependent variab l e s , the model parameters were estimated using the data values. The estimates were then obtained for the dependent v a r i a b l e s based on data values at the optimum stations using the Kalman F i l t e r parameter estimation algorithm described i n Chapter VIII. The error variance 101. was then determined i n each case and was used to compare the performance of the two methodologies. 7.3 Analysis Both simulated and r e a l p r e c i p i t a t i o n data were analyzed to study the r e l a t i v e performance of Fisher's and Shannon's information measures. 7.3.1 Simulated data The time seri e s data f o r eight point l o c a t i o n s were simulated using the multivariate c o r r e l a t i o n approach discussed i n Section 4.1.1. The input c o r r e l a t i o n matrix of the simulated data, together with the expected values at each point l o c a t i o n , are l i s t e d i n Table 7.1a. Out of 300 values simulated at each point l o c a t i o n , 150 values were used i n optimum s t a t i o n s e l e c t i o n by the methodologies based on Fisher's and on Shannon's information. The remaining 150 values were used to study the p r e d i c t i v e performance of the optimum selected stations at the common ungauged point l o c a t i o n s . Using the formulation i n Section 7.1, the optimum s t a t i o n sets and the corresponding information transmission were determined using Shannon's d i s c r e t e entropy concept at the b i v a r i a t e l e v e l s and are l i s t e d i n Table 7.1b. The best s i n g l e , two, and three s t a t i o n sets were (#5); (#4, and #5); and (#3, #4, and #5) r e s p e c t i v e l y as shown i n Table 7.1 c. S i m i l a r l y , the r e s u l t s l i s t e d i n Table 7.1 d, were obtained using Fisher's information c r i t e r i o n of Equation (7.4). The optimum stations were i d e n t i f i e d by maximizing the information content as summarized i n Equation (7.6). The optimum station(s) and t h e i r corresponding information content are l i s t e d i n Table 7.1 e. The 102. optimum single, two, and three s t a t i o n s e t s were found to be (#3); (#3, #6); and (#3, #6, and #8) r e s p e c t i v e l y . At the multivariate l e v e l of information transmission, Equation (7.11) for Shannon's methodology, and Equation (7.10) f o r the Fisher's information c r i t e r i o n , were applied. The optimum stations i n Shannon's case were (#4, and #5); and (#3, #4, and #5). Fisher's information c r i t e r i o n i d e n t i f i e d point l o c a t i o n (#3) as the optimum sing l e s t a t i o n , point locations (#4,#6) as the optimum two s t a t i o n set, and point locations (#3, #4, #6) as the optimum three s t a t i o n combinations. These r e s u l t s are l i s t e d i n Table 7.1 e. The parameters of the estimation models were computed from the i n i t i a l 150 simulated observations. Values at the common point locations not included i n the l i s t of optimal design by e i t h e r method (#1, #2, #7 and #8) were then estimated by the optimum s t a t i o n sets obtained by the d i f f e r e n t c r i t e r i a as discussed i n Section 7.2.1. The error variances thus obtained are l i s t e d i n Table 7.1f and 7.1g. 7.3.2 Daily p r e c i p i t a t i o n data from Lower Mainland Region In the second example d a i l y p r e c i p i t a t i o n data from the f i r s t eight stations from the Lower Mainland Region of the B.C., as l i s t e d i n Table 4.4, was used i n the analysis. The i n i t i a l 350 data values at each point l o c a t i o n were used for optimum st a t i o n s e l e c t i o n and the remaining 350 observations were reserved for t e s t i n g of the estimation performance of the stations selected by e i t h e r method. The analysis f o r t h i s example was performed using both d i s c r e t e p r o b a b i l i t i e s and continuous d i s t r i b u t i o n s . The f i r s t part of the analysis was c a r r i e d out i n a s i m i l a r fashion as i n simulated data case described i n Section 7.3.1. The r e s u l t s 103. obtained are l i s t e d i n Tables 7 .2 a,b,c,d, e,f, and g. In the second part of the ana l y s i s , the optimum stations were obtained using continuous d i s t r i b u t i o n s . The optimum s t a t i o n s e l e c t i o n was based on the p r i n c i p l e o u t l i n e d i n Section 7.1 but the entropy terms used i n Equation (7.4 and (7.9) were evaluated using the r e l a t i o n s h i p s derived i n Chapter VI. The r e s u l t s obtained are l i s t e d i n Tables 7.3a to 7.3 f. The estimation error variances, f o r the common point locations were evaluated and the r e s u l t s thus obtained are l i s t e d i n Tables 7.3 g, h and i . 7.4 Conclusions The following p r i n c i p a l conclusions are based only upon the simulated and r e a l data r e s u l t s described i n t h i s chapter. The following conclusions were drawn from the tabulated r e s u l t s i n the f i r s t example: 1. When estimating on the basis of single s t a t i o n data, estimation error variance using Shannon's b i v a r i a t e information c r i t e r i o n , was les s than that obtained with the comparable Fisher's c r i t e r i o n . 2. When estimating on the basis of data from two stati o n s , Shannon's b i v a r i a t e and multivariate information transmission c r i t e r i a i n most cases outperformed Fisher's c r i t e r i a . In the second example, using r e a l p r e c i p i t a t i o n data, the following conclusions were drawn: 1. In two and three s t a t i o n estimation, the stations i d e n t i f i e d by Shannon's mu l t i v a r i a t e information transmission, based on the disc r e t e entropy concept, were always the best estimator (Tables 7.2 h and 7.2 i) . 104. 2. No d i s t i n c t superiority for either Shannon's discrete bivariate or Fisher's bivariate methods were indicated. 105. TABLE 7.1a - CORRELATION MATRIX OF THE SIMULATED DATA (300 OBSERVATIONS AT 8 LOCATIONS) 1 \ 1 2 3 4 5 6 7 8 1 1.00 0.35 0.34 0.31 0.30 0.13 0.10 0.05 2 1.00 0.30 0.25 0.22 0.20 0.1.6 0.14 3 1.00 0.50 0.42 0.36 0.33 0.21 4 1.00 0.24 0.18 0.13 0.21 5 1.00 0.51 0.35 0.25 6 1.00 0.45 0.25 7 1.00 0.30 8 1.00 ean= 10.0 10.3 10.9 8.5 8.3 7.2 6.9 9.5 TABLE 7.1b - SHANNON'S INFORMATION MATRIX BY DISCRETE ENTROPY CONCEPT (BASED ON 150 OBSERVATIONS). X 1 2 3 4 5 6 7 8 1 1.7489 0 .1930 0 .1980 0 .1685 0 .2512 0.1802 0.1696 0.1173 2 1 .8215 0 .2573 0 .1658 0 .1595 0.1737 0.1701 0.1739 3 1 .8592 0 .2847 0 .2554 0.2386 0.2450 0.1567 4 1 .9249 0 .1844 0.1979 0.2247 0.2015 5 1 .8881 0.2945 0.2787 0.2242 6 1.7837 0.2865 0.2246 7 1.8256 0.2381 8 1.8150 TABLE 7.1c - OPTIMUM RETAINED STATIONS AND THE OPTIMUM INFORMATION TRANSMISSION Bi v a r i a t e M u l t i v a r i a t e S.N. Optimum Retained Stations Optimum Information Transmission Optimum Retained Stations Optimum Information Transmission 1 5 3.5360 2 4,5 5.3121 4,5 3.629 3 3,4,5 6.9718 4,5,8 4.525 106. TABLE 7.1d - FISHER'S INFORMATION MATRIX OF THE SIMULATED DATA (BASED ON 150 OBSERVATIONS AT 8 LOCATIONS) N. j 1 2 3 4 5 6 7 8 1 1.0000 0.6263 0.6188 0.5863 0.6195 0.5540 0.5642 0.5026 2 1.0000 0.6223 0.5621 0.5600 0.5492 0.5377 0.5265 3 1.0000 0.6689 0.6345 0.6316 0.6270 0.5627 4 1.0000 0.5771 0.5453 0.5436 0.5531 5 1.0000 0.6501 0.6239 0.5912 6 1.0000 0.6620 0.5904 7 1.0000 0.5867 8 1.0000 TABLE 7.1e - OPTIMUM RETAINED STATIONS AND THE OPTIMUM INFORMATION TRANSMISSION Bi v a r i a t e M ultivariate S.N. Optimum Optimum Optimum Optimum Retained Information Retained Information Stations Transmission Stations Transmission 1 3 5.3658 3 5.3658 2 3,6 5.8125 3,6 5.8525 3 3,6,8 6.2221 3,6,8 6.3100 107. TABLE 7.1 f - COMPARISON OF SHANNON AND FISHER METHODOLOGIES USING ESTIMATION ERROR METHOD (SINGLE STATION ESTIMATION) Grid Station Error Variance Methodologies Multiple Regression Kalman F i l t e r Estimation 1 5 0.885 0.885 Shannon (Bivariate) 1 3 0.919 0.920 Fisher (Bivariate) 2 5 0.835 0.828 Shannon (Bivariate) 2 3 0.907 0.910 Fisher (Bivariate) 7 5 1.048 1.063 Shannon (Bivariate) 7 3 0.976 0.981 Fisher (Bivariate) 8 5 0.887 0.884 Shannon (Bivariate) 8 3 0.906 0.900 Fisher (Bivariate) TABLE 7.1g - COMPARISON OF SHANNON AND FISHER METHODOLOGIES USING ESTIMATION ERROR METHOD (TWO-STATION ESTIMATION) Grid Station Error Variance Methodologies Multiple Regression Kalman F i l t e r Estimation 1 4,5 0.895 0.907 Shannon (Multivariate) 1 3,5 0.908 0.984 Shannon (Bivariate) 1 3,6 0.917 0.984 Fisher (Bivariate) 1 4,6 0.895 0.933 Fisher (Multivariate) 2 4,5 0.814 0.815 Shannon (Multivariate) 2 3,5 0.906 0.930 Shannon (Bivariate) 2 3,6 0.909 0.911 Fisher (Bivariate) 2 4,6 0.819 0.816 Fisher (Multivariate) 7 4,5 1.005 0.997 Shannon (Multivariate) 7 3,5 0.969 0.967 Shannon (Bivariate) 7 3,6 0.963 0.852 Fisher (Bivariate) 7 4,6 0.924 0.889 Fisher (Multivariate) 8 4,5 0.888 0.906 Shannon (Multivariate) 8 3,5 0.898 0.961 Shannon (Bivariate) 8 3,6 0.901 0.969 Fisher (Bivariate) 8 4,6 0.870 0.932 Fisher (Multivariate) 108. TABLE 7.2a - FISHER'S INFORMATION MATRIX (BASED ON REAL OBSERVATIONS OBTAINED FROM LOWER MAINLAND REGION: FIRST EIGHT STATIONS WITH 350 OBSERVATIONS) \ . j i >v 1 2 3 4 5 6 7 8 1 1.0000 0 .7625 0 .6133 0 .6344 0.6294 0.5911 0 .6456 0.6601 2 1 .0000 0 .6671 0 .7295 0.6503 0.6297 0 .7645 0.7286 3 1 .0000 0 .8376 0.7211 0.6790 0 .7813 0.7787 4 1 .0000 0.7571 0.7483 0 .8837 0.8558 5 1.0000 0.7128 0 .7137 0.7563 6 1.0000 0 .7175 0.7515 7 1 .0000 0.8183 8 1.0000 TABLE 7.2b - OPTIMUM RETAINED STATIONS AND THE OPTIMUM INFORMATION TRANSMISSION Bi v a r i a t e M u l t i v a r i a t e S.N. Optimum Retained Stations Optimum Information Transmission Optimum Retained Stations Optimum Information Transmission 1 4 6.4464 4 6.4464 2 1,4 6.8450 1,4 6.9286 3 2,4,6 7.0967 2,4,6 7.2059 109. TABLE 7.2c - SHANNON'S INFORMATION MATRIX (BASED ON REAL OBSERVATIONS OBTAINED FROM LOWER MAINLAND REGION: FIRST EIGHT STATIONS WITH 350 OBSERVATIONS) 1 \ ^ 1 2 3 4 5 6 7 8 1 0.7614 0.2310 0.1146 0.1163 0.1007 0.0198 0.1407 0.1621 2 0.6236 0.1340 0.1463 0.1168 0.094 8 0.1721 0.1840 3 0.7870 0.2440 0.2163 0.1964 0.2277 0.2884 4 0.6654 0.2104 0.2205 0.3296 0.2585 5 0.7004 0.2106 0.1577 0.2355 6 0.8131 0.2075 0.2296 7 0.6534 0.2584 8 0.6790 TABLE 7.2d - OPTIMUM RETAINED STATIONS AND THE OPTIMUM INFORMATION TRANSMISSION S.N. Bi v a r i a t e Multl! /ariate Optimum Retained Stations Optimum Information Transmission Optimum Retained Stations Optimum Information Transmission 1 2 3 8 1,3 1,3,6 2.2960 2.9522 3.5689 1,6 1,3,6 1.4830 1.9400 110. TABLE 7.2e - COMPARISON OF SHANNON AND FISHER METHODOLOGIES USING ESTIMATION ERROR METHOD (ONE-STATION ESTIMATION) Error Variance Grid Station Multiple Kalman Methodologies Regression F i l t e r Estimation 2 8 1450 1505 Shannon (Bivariate) 2 4 2085 2147 Fisher (Bivariate) 5 8 383 323 Shannon (Bivariate) 5 4 533 471 Fisher (Bivariate) TABLE 7.2 f - COMPARISON OF SHANNON AND FISHER METHODOLOGIES USING ESTIMATION ERROR METHOD (TWO-STATION ESTIMATION) Error Variance Grid Station M u l t i p l e Regression Kalman F i l t e r Estimation Methodologies 2 1,3 828 963 Shannon (Bivariate) 2 1,6 775 902 Shannon (Multivariate) 2 1,4 909 1000 Fisher (Bivar- & Multi.) 5 1,3 1041 925 Shannon (Bivariate) 5 1,6 422 404 Shannon (Multivariate) 5 1,4 527 491 Fisher (Bivar.&Multi.) TABLE 7.2g - COMPARISON OF SHANNON AND FISHER METHODOLOGIES USING ESTIMATION ERROR METHOD (THREE-STATION ESTIMATION) Error Variance Grid Station M u l t i p l e Regression Kalman F i l t e r Estimation Methodologies 5 1,3,6 436 348 Shannon (Bivar. &: Multi.) 5 2,4,6 328 383 Fisher (Bivar. & Multi.) 7 1,3,6 987 990 Shannon (Bivar. & Multi.) 7 2,4,6 849 893 Fisher (Bivar. & Multi.) 111. TABLE 7.3a - SHANNON'S INFORMATION MATRIX (BASED ON REAL OBSERVATIONS OBTAINED FROM LOWER MAINLAND REGION: FIRST EIGHT STATIONS WITH 350 OBSERVATIONS) (Bivariate Normal Dis t r i b u t i o n ) \. j i \ . 1 2 3 4 5 6 7 8 1 4.7827 0.3224 0.0742 0.0999 0.0935 0.0506 0.1147 0.1350 2 4.9855 0.1455 0.2531 0.1211 0.0938 0.3271 0.2513 3 4.9260 0.5261 0.2370 0.1638 0.3667 0.3604 4 4.9131 0.3104 0.2915 0.7028 0.5890 5 5.0620 0.2215 0.2232 0.3086 6 4.7395 0.2301 0.2983 7 4.9953 0.4660 8 4.7351 TABLE 7.3b - OPTIMUM RETAINED STATIONS AND THE OPTIMUM INFORMATION TRANSMISSION Biv a r i a t e M u l t i v a r i a t e S.N. Optimum Optimum Optimum Optimum Retained Information Retained Information Stations Transmission Stations Transmission 1 4 7 .6859 2 2,4 12.6408 2,5 9.9307 3 2,4,5 17.3924 2,3,5 14.5709 112. TABLE 7.3c - SHANNON'S INFORMATION MATRIX (BASED ON REAL OBSERVATIONS OBTAINED FROM LOWER MAINLAND REGION: FIRST EIGHT STATIONS WITH 350 OBSERVATIONS) (Bivariate Log-normal Di s t r i b u t i o n ) i ^ 1 2 3 4 5 6 7 8 1 4.5258 0 .0049 0 .0128 0 .0046 0 .0315 0.0025 0.0035 0.0003 2 4 .6760 0 .0002 0 .0054 0 .0017 0.0018 0.0010 0.0011 3 4 .6310 0 .0322 0 .0184 0.0066 0.0452 0.0598 4 4 .6294 0 .0058 0.0049 0.0038 0.0083 5 4 .6610 0.0057 0.0043 0.0001 6 4.4659 0.0001 0.0002 7 4.7252 0.0589 8 4 .3679 TABLE 7.3d - OPTIMUM RETAINED STATIONS AND THE OPTIMUM INFORMATION TRANSMISSION B i v a r i a t e M u l t i v a r i a t e S.N. Optimum Optimum Optimum Optimum Retained Information Retained Information Stations Transmission Stations Transmission 1 7 4.8419 2 5,7 9.5350 2,7 9.4045 3 2,5,7 14.2093 2,5,7 14.0620 113. TABLE 7.3e - SHANNON'S INFORMATION MATRIX (BASED ON REAL OBSERVATIONS OBTAINED FROM LOWER MAINLAND REGION: FIRST EIGHT STATIONS WITH 350 OBSERVATIONS) (Bivariate Gamma Dist r i b u t i o n ) 1 2 3 4 5 6 7 8 1 5.3869 0 .1002 1 .1036 0 .5504 0.0677 0 .0453 0.9274 0.0669 2 5 .2574 0 .4598 1 .0377 1.3736 0 .4047 0.5078 0.8609 3 5 .3562 1 .2241 0.1474 0 .1281 2.3235 0.4671 4 5 .3128 0.5168 0 .3039 1.5789 0.1291 5 5.1976 0 .4173 0.2431 1.7054 6 5 .5494 0.2556 0.4352 7 5.3358 0.2405 8 5.2014 TABLE 7.3 f - OPTIMUM RETAINED STATIONS AND THE OPTIMUM INFORMATION TRANSMISSION Optimum Optimum S.N. Retained Information Stations Transmission 1 7 11.4126 2 5,7 18.8598 3 5,6,7 23.9916 114. TABLE 7.3g - COMPARISON OF SHANNON AND FISHER METHODOLOGIES USING ESTIMATION ERROR METHOD (ONE-STATION ESTIMATION) Error Variance Grid Station Multiple Regression Kalman F i l t e r Estimation Methodologies 1 7 1216 1264 Gamma Bi v a r i a t e 1 7 1216 1264 Lognorm B i v a r i a t e 1 4 1115 1157 Normal B i v a r i a t e 1 4 1233 1294 Fisher B i v a r i a t e 2 7 1949 1996 Gamma B i v a r i a t e 2 7 1949 1996 Lognorm B i v a r i a t e 2 4 1478 1496 Normal B i v a r i a t e 2 4 2085 2167 Fisher B i v a r i a t e 3 7 1157 1180 Gamma Bi v a r i a t e 3 7 1157 1180 Lognorm B i v a r i a t e 3 4 957 1027 Normal B i v a r i a t e 3 4 891 909 Fisher B i v a r i a t e 6 7 1423 1462 Gamma Bi v a r i a t e 6 7 1423 1462 Lognorm B i v a r i a t e 6 4 933 989 Normal B i v a r i a t e 6 4 906 917 Fisher B i v a r i a t e 8 7 579 576 Gamma Bi v a r i a t e 8 7 579 576 Lognorm B i v a r i a t e 8 4 236 233 Normal B i v a r i a t e 8 4 523 512 Fisher B i v a r i a t e 115. TABLE 7.3h - COMPARISON OF SHANNON AND FISHER METHODOLOGIES USING ESTIMATION ERROR METHOD (TWO-STATION ESTIMATION) Grid Station Error Variance Methodologies Mult i p l e Regression Kalman F i l t e r Estimation 3 5,7 958 995 Gamma B i v a r i a t e 3 5,7 958 995 Lognorm B i v a r i a t e 3 2,7 1157 1304 Lognorm M u l t i v a r i a t e 3 2,5 1183 1300 Normal M u l t i v a r i a t e 3 2,4 897 963 Normal B i v a r i a t e 3 1,4 887 951 Fisher Bivar, &Multi. 6 5,7 971 856 Gamma Bi v a r i a t e 6 5,7 971 856 Lognorm B i v a r i a t e 6 2,7 1429 1558 Lognorm M u l t i v a r i a t e 6 2,5 1112 1159 Normal M u l t i v a r i a t e 6 2,4 923 1033 Normal B i v a r i a t e 6 1,4 941 971 Fish e r Bivar . & Multi 8 5,7 394 385 Gamma B i v a r i a t e 8 5,7 394 385 Lognorm B i v a r i a t e 8 2,7 540 574 Lognorm M u l t i v a r i a t e 8 2,5 406 405 Normal M u l t i v a r i a t e 8 2,4 460 454 Normal B i v a r i a t e 8 1,4 495 470 Fisher Bivar. & M u l t i . TABLE 7 . 3 i - COMPARISON OF SHANNON AND FISHER METHODOLOGIES USING ESTIMATION ERROR METHOD (THREE-STATION ESTIMATION) Error Variance Grid Station M u l t i p l e Regression Kalman F i l t e r Estimation Methodologies 1 5,6,7 819 731 Gamma Bi v a r i a t e 1 2,5,7 597 792 Lognorm B i v a r i a t e 1 2,5,7 597 792 Lognorm Mu l t i v a r i a t e 1 2,3,5 601 762 Normal M u l t i v a r i a t e 1 2,4,5 601 824 Normal M u l t i v a r i a t e 1 2,4,6 595 842 Fisher Bivar.& M u l t i . 8 5,6,7 331 346 Gamma Bi v a r i a t e 8 2,5,7 364 389 Lognorm B i v a r i a t e 8 2,5,7 364 389 Lognorm Mul t i v a r i a t e 8 2,3,5 407 396 Normal M u l t i v a r i a t e 8 2,4,5 401 394 Normal B i v a r i a t e 8 2,4,6 389 404 Fisher Bivar. & M u l t i . 116. CHAPTER VIII ESTIMATION AND INFORMATION TRANSMISSION 8.1 Introduction The estimation of hydrologic events at ungauged locations using information obtained from s t a t i o n locations often a r i s e s i n hydrology. In many cases, the hydrologic events are transformed from one form to another, f o r example, the transformation of r a i n f a l l into runoff using physical and conceptual mathematical models. Developing a complete and accurate model based on the physical concepts i s not always f e a s i b l e due to the l i m i t e d a v a i l a b l e information concerning various physical parameters. Conceptual models combining hydrologic properties with estimation technigues have been very popular i n the past two decades. The estimation of hydrologic events at various locations i n the basin on the basis of measurements at s t a t i o n locations i s m u l t i v a r i a t e i n nature and involves i n t e r a c t i o n s with v a r i a b l e s i n both time and space domains. The estimation of the hydrologic events, p a r t i c u l a r l y runoff, using r a i n f a l l data was c a r r i e d out using the concept of Weiner's theory (74) on the p r i n c i p l e of minimization of mean square error between the estimated events and the concurrent observed events (30). The r e s t r i c t i o n s imposed i n Wiener's theory were l a t e r improved (6,78). Natale and Todini (55,56) applied constrained parameter estimation techniques to runoff p r e d i c t i o n . In 1960, Kalman developed an optimal sequential estimation technique which i s usually r e f e r r e d to as the Kalman f i l t e r (37) . I t can be applied to mul t i v a r i a t e problems involving both s p a t i a l and 117. autocorrelation e f f e c t s . The Kalman f i l t e r model i s recursive and dynamic i n nature. Unlike other conventional estimation models, i t does not require that a l l of the past measurements be stored for the purpose of p r e d i c t i n g future hydrologic events. I t integrates state-space r e l a t i o n s h i p s and measurement-state r e l a t i o n s h i p s with the estimation equations. These r e l a t i o n s h i p s w i l l be defined l a t e r i n t h i s chapter. Using the Kalman f i l t e r model, a r e a l time model implementation for f l o o d p r e d i c t i o n using on-line measurement instruments i s also p o s s i b l e . The model i s also capable of including time v a r i a b l e parameters and d i f f e r e n t sources of e r r o r s . The p r i n c i p a l intent of t h i s chapter i s to show the r e l a t i o n s h i p s between Shannon's information transmission, estimation i n general, l e a s t squares regression, and Kalman f i l t e r model. 8.2 General Estimation and Information Transmission Consider two sets of vectors Z= (Z-, , Z~ , .. . ,Z ) and X=(X, ,X_, .. .,X ), J. 2 n L 2 m the information transmission by the set of v a r i a b l e s X about Z i s defined i n continuous form (22) as: T(Z;X) = T(X;Z) = p(Z,X) Jon(p(z|x)/p(Z) ) dZ dX . . (8.1) where, dZ=dZn ,dZ_,dZ_, ,dZ ; 1 2 3 n dX=dX 7,dX„,dX_,...,dX ; 1 2 3 m p(Z,X) i s the m u l t i v a r i a t e j o i n t p r o b a b i l i t y density function of the vectors Z and X. P(z|x) i s the conditional multivariate p r o b a b i l i t y density of Z given X. 118. As stated e a r l i e r , i n Equations (3.11) and (3.13), the information transmission can be defined as follows: T(Z;X) = H(Z) - H(Z|X) (8.2] = H(Z) + H(X') - H(Z,X) In l i n e a r regression, i t i s assumed that the estimated variable vector X i s independent of the estimation error vector X where X = X - X . With independency between random v a r i a b l e s , no information can be transmitted by one about the other. Thus: T(X;X) = 0 (8.3a) and also H(X) = H(x|x) (8.3b) I t i s evident that, p(X,X) = p(X-X, X) = p(X,X), S i m i l a r l y , p(x|x) = p(x-x|x) =p(x|x), hence, p(X,X) in p(x|x) dXdX = - p(X,X) in p(x|x) dXdX or H(X|X) = H(X|X) = H(X) (8.4) hence T(X;X) = H(X) - H(x|x) (8.5) = H(X) - H(X) Therefore, maximizing the information transmission between X and /\ — X, i s equivalent to maximizing H(X)-H(X). But H(X) i s constant for a given set of data, hence, 119. Max [ T ( X ; X ) ] = Min [H ( X ) ] (8.6) So that maximizing the information transmitted by estimates about the true values i s equivalent to minimizing the error entropy. Consider a case i n which the errors a r i s i n g from l i n e a r regression are not independent and t h e i r d i s t r i b u t i o n i s not defined i n advance. Also assume that: X ' i s the set of var i a b l e s with the estimated events from the true optimum estimator; and X ' i s the error vector a r i s i n g from the true optimum estimator with no r e s t r i c t i o n of error independency. Then T ( X ' ; X ) > 0 (8.7a) or H ( X ' ) - H ( X ' | x ' ) > 0 (8.7b) or -H ( X ' ) < - H ( X ' | x ' ) Adding H ( X ) on both sides of the in e q u a l i t y , we get, H(X) - H ( X ' ) < H ( X ) - H ( X ' | x ' ) but H ( X ' | x ' ) = H ( x | x ' ) , hence, H ( X ) - H ( X ' ) < H( X ) - H ( x | x ' ) (8.8) As defined e a r l i e r i n t h i s section that X and X are res p e c t i v e l y as the set of va r i a b l e s of the estimated and error events obtained by the estimator with r e s t r i c t i o n of error independency. Hence, H ( X ' ) < H ( X ) (8.9) Hence, from Equation (6.5) and (8.8) T ( X ; X ' ) > T ( X ; X ) (8.10) 120. where, T (X;X) i s the information transmission by the estimated vector about the observed vector when the error terms are assumed to be independent normally d i s t r i b u t e d . T (X;X') i s the information transmission by the estimated vector about the observed vector of the va r i a b l e s when the error terms are not independent. The above equation (8.10) reveals that the information transmitted by estimates obtained from an estimation model which assumes error independency i s l e s s than the information transmitted by a model which does not make t h i s assumption. 8.3 Parameter Estimation for a Multiple Linear Regression Model  Using the Maximum Information Transmission C r i t e r i o n The multiple l i n e a r regression equation i s as follows: m x . = I h i z„ . + x (8.11a) 3 1=1 £ ' 3 3 In vector form, i t can be written as: X = bZ + X (8.11b) where, X i s the dependent v a r i a b l e b i s an n-dimensional vector of regression c o e f f i c i e n t s . Z i s a set of n independent v a r i a b l e s X i s the vector of err o r terms. The regression parameters, using the l e a s t square technique, are given i n the following form: b = ( Z T Z ) _ 1 Z TX (8.12a) 121. S i m i l a r l y f or the i t h dependent v a r i a b l e , the vector of the regression c o e f f i c i e n t s b^ i s as follows: b. = ( Z T Z ) _ 1 Z TX (8.12b) 1 x The estimation parameters, as estimated by the l e a s t square technique, can also be estimated using Shannon's information theory. For independent error terms, the maximization of information transmission by the estimate about the true event, as proved i n Equation (8.6), i s equivalent to minimization of the error entropy. In the l e a s t squares technique, i n addition to independency, the normality assumption i s also taken into consideration. On the basis of the normality assumption, the entropy H(X) can be defined as follows: H(X) = 0.5 Jin [ (2TTe) m|p, |] (8.13) where, | P | i s the determinant of the error variance covariance matrix of si z e mxm and i s defined as follows: m |p, I = IT a 2 (8.14) k i = l X i where, ? R ~ ^ TT oxz = E [ ( x i - x ± ) ( x i - x ± ) ] N m = E (x. . - E b.. Z 0 , ) 2 N-l j = ]_ ID £ = 1 iA 13 1 N -=r- E (x. . • b . Z . ) 2 (8.15a) x ± N-l 13 l : O 2 Hence n N H(X) = 2. An 2TT + E An [-=7- E (x. . • b.Z.) 2] (8.15b) 2 i = l ^ j=l 1 3 1 3 122. In order to minimize H(X), Equation (8.15b) i s d i f f e r e n t i a t e d with respect to b^ and equated to zero. I t y i e l d s the r e s u l t : b. = ( Z T Z ) _ 1 Z TX. (8.16) l I Thus Equation (8.16) i s i d e n t i c a l to Equation (8.12b). Therefore, f o r uncorrelated Gaussian errors, parameter estimation using the l e a s t squares technique produces i d e n t i c a l parameters to those derived using the maximum information transmission c r i t e r i o n . Thus l i n e a r regression i s a sp e c i a l case of optimal information transmission by estimates about the true values. 8.4 Parameter Estimation for a Kalman F i l t e r Model As discussed i n Section 8.1, the Kalman f i l t e r model has both dynamic and recursive properties. The parameters of t h i s optimal f i l t e r i n g model have been estimated i n the past using an orthogonal p r o j e c t i o n method (37) , maximum l i k e l i h o o d method (38) and Bayesian approach (10). Very recently, the concept of maximizing information transmission has been used to estimation process (73). There are three possible uses of the Kalman f i l t e r . These are: parameter estimation based on the past observations, i . e . smoothing; parameter estimation based on the past and current information, i . e . f i l t e r i n g ; and future state estimation i . e . p r e d i c t i o n . A l l of these uses of the Kalman f i l t e r have p o t e n t i a l applications i n hydrologic modelling and estimation. Only f i l t e r i n g a p p l ications w i l l be discussed here. The state-space r e l a t i o n s h i p and the measurement-state r e l a t i o n -ships of the Kalman f i l t e r i s described by the following sets of equations: 123. = F X + G U t t t t (8.17) Z t+1 C + V (8.18) t+1 where X i s a n-dimensional state vector describing the state of the dynamic system at time t . Z^_ i s an m-dimensional measurement vector; U i s an r-dimensional system noise vector representing the system uncertainty; V^_+^  i s an m-dimensional measurement noise vector i n d i c a t i n g the uncertainty i n the measurement process; F i s an nxn state t r a n s i t i o n matrix i n t e r r e l a t i n g the state v a r i a b l e at time t and t+1; G^_ i s an nxr matrix d e f i n i n g the system error c h a r a c t e r i s t i c s ; ct + n ^ i s an Mxn matrix i n t e r r e l a t i n g the measurement and state vector. I t i s assumed that U"t, Vfc and X q are mutually independent, normally d i s t r i b u t e d , and with i d e n t i c a l l y zero means and covariance matrices as follows: Cov (ufc,ur) = Qfc 6(t,p) Cov (v t,v ) = R t 6(t,p) where, <5(t,p) = 1 i f t=p, otherwise = 0.0, and Cov(U.,V ) = 0.0 P E ( U J = 0.0 Var(X ) = w o o 124 . Here Q and W q are symmetric non-negative d e f i n i t e matrices while R i s p o s i t i v e d e f i n i t e , which means that no component of the observed signals can be measured exactly. Based on the r e l a t i o n s h i p s s p e c i f i e d by Equations (8.17) and (8.18), and the above assumptions, the optimum estimates X of the state v a r i a b l e s , based on the measurements or observations and the previous estimates of the state v a r i a b l e s X^, are as follows: Xt+1 = \ X t + St+1 Zt+1 (8.19) Where A^ and S^ +^ are unknown matrices. The objective here i s to determine the unknown matrices A^ and S ^ from the viewpoint of Shannon's information theory. Under the assumptions of uncorrelated processes with Gaussian d i s t r i b u t i o n , i t has been proved i n Section 8.2, that the maximization of information transmission about the state vector X^ by the estimate X i s equivalent to minimizing the entropy of the estimation error (68) . In mathematical form, M A X [T(X t;X t>] «- -»• M I N [H(X )] (8.20) V S t + l V S t + l For the Gaussian vector, X^, the m u l t i v a r i a t e entropy, H(X^) as shown i n Equation (8.13), i s as follows: H(X t) = W n [ (2TTe) m|P k|] (8.21) Hence, to minimize H(X ), |p | should be minimized. On the other hand, the determinant has the following property. +00 exp(-YTP Y)dY (8.22) 125. ~ T Hence to minimize H(X ) i s equivalent to minimizing Y P, Y for t k the n-dimensional vector Y (68). Tomita et a l (68) addressed themselves to the information transmission c h a r a c t e r i s t i c s of the Kalman f i l t e r . They did not recognize the ge n e r a l i t y which Equations (8.9) and (8.10) suggest and are v a l i d i n general estimation. Using the minimum error entropy concept of Equation (8.20), the matrices A^ and S^ +^ were estimated and are as follows: \ = F t " S t + l C t + l F t ( 8 - 2 3 ) s t + i = pt+ict+i R w ( 8 - 2 4 ) where pt + i = ( r J i + c J i <w_1 (8-25) rt + i = F t p t < + G t Q t G t < 8- 2 6 ) P t = E{X t X t T} (8.27) where * t - \ ~ \ and X^ i s the true state vector X^_ i s the estimated state vector X^_ i s the error state vector Adaptation of the Kalman f i l t e r are presented here i n the context of c e r t a i n elementary problems associated with runoff p r e d i c t i o n using data from an optimal p r e c i p i t a t i o n network. I t i s believed that these adaptations of the Kalman f i l t e r w i l l suggest further applications i n hydrologic modelling and estimation. 126. CHAPTER IX INFORMATION TRANSMISSION AND NETWORK COST 9.1 Introduction The cost of i n s t a l l i n g and operating the network, and the probable future benefits which w i l l a r i s e from the reduction i n the uncertainty associated with the input of the water resources systems, are the two main factors a f f e c t i n g the economic planning of hydrologic network design. I t i s easy to estimate the cost of the network but i t i s d i f f i c u l t to evaluate the probable benefits since these are associated with the actual future developments i n the region. The estimation error, which depends i n part upon instrumentation errors, has been used i n the past to evaluate the network output. The reduction of the estimation error, due to an increase i n the network density, has been translated into percentage cost savings but has been attempted only f o r s p e c i f i c types of water resources development (51,70). This chapter deals only with the optimization of hydrologic network performance as measured by information transmission, and i s based on the p r i n c i p l e of minimizing network cost f o r a s p e c i f i e d value of information transmission. An example i s presented using simulated data. The approach minimizes the t o t a l i n s t a l l a t i o n and operation costs of the network by considering a denser network of less precise and low cost instruments f o r higher accuracy high cost instruments with low density. The same optimum information transmission f o r a hydrologic basin can be attained by networks having d i f f e r e n t p r e c i s i o n s . The minimum cost net-work for a given l e v e l of information transmission i s obtained using an assumed cost accuracy r e l a t i o n s h i p for the instruments. 127. 9.2 Data Simulation Based on Instrument Accuracy Time series data (300 samples) for a hypothetical basin consisting of six point locations was simulated for the s p e c i f i e d m u l t i v a r i a t e c o r r e l a t i o n matrix l i s t e d i n Table 9.1. The set of var i a b l e s , denoted by S, representing true hydrologic events i n the basin are (X^tX^,...,Xg) where i s the v a r i a b l e representing the true hydrologic events at the i t h point l o c a t i o n . This synthesized basin data was then modified by introducing a random component which depended only upon the magnitude of a constant, "e " where £ determines the instrument p r e c i s i o n . When £ was unity, the error introduced i n the modified data sets was a maximum but when £^ was zero, no error was introduced. The mathematical model used f o r t h i s instrument simulations was: x'. . = x. . + <j>. 0". e, (9.1) where, d>. i s the random normal deviate; 3 x. . i s the j t h true events a t i t h point locations; x! . i s the j t h measurement from an instrument at point l o c a t i o n i when accuracy i s prescribed by E ; i s the standard deviation of the events at i t h point l o c a t i o n ; and i s a c o e f f i c i e n t specifying the instrument p r e c i s i o n ; S' = {x',X',. ..,X'} i s the modified set of variables representing 1 2 6 network s t a t i o n l o c a t i o n s with £, instrument error. k Modified time ser i e s data were generated for instruments at a l l locat i o n s f o r values of efc ranging from 0.1 to 0.5. 128. 9.3 Theoretical Development and Analysis Information transmission about the set S by the qth combination of n imperfect stations i s : T(S;S < n) = H(S) + H(s'n) - H(S,s'n) (9.2) q q q For the sake of computational convenience, f i t t e d m u l t i v a r i a t e normal d i s t r i b u t i o n s were used to determine the value of T(S;S n). q The d e r i v a t i o n of m u l t i v a r i a t e entropy for the multivariate normal d i s t r i b u t i o n was discussed i n Chapter VI. The variance covariance 'n 'n matrices for S;(S,S ) and S , which provided the parameters of the q q m u l t i v a r i a t e normal d i s t r i b u t i o n s , were obtained from the synthesized data. The assumed r e l a t i o n s h i p s between u n i t cost of instruments and instrument p r e c i s i o n prescribed by £ i s shown i n Figure 9.1. Optimum information transmission for networks having the l e a s t instrument error, prescribed by £ =0.1, was f i r s t computed using Equation (9.2). The same procedure was repeated for other networks having l e s s precise instruments. Optimum information transmission r e s u l t s for a l l cases and sizes of network are shown i n Figure (9.2). As Figure 9.2 indicates, the network with le s s precise instruments transmitted l e s s information about the true hydrologic set than the network with high p r e c i s i o n instruments. Figure 9.2 was used to determine the l e a s t number of stations i n optimal networks with £^ > 0.1 required to match the information transmitted by optimal one and two s t a t i o n networks for £ = 0 . 1 . k The associated costs of the equivalent networks were determined with the a i d of the u n i t cost curve (Figure 9.1) and are shown i n Figure 9.3. 129. The o v e r a l l minimum costs were then determined from the cost curves i n Figure 9.3 and compared with the cost of the network with £^=0.1. This i s summarized i n Figure 9.4. It i s concluded from t h i s study that within the l i m i t a t i o n s of the example, a l e s s accurate high density network can be more economical than a more precise l e s s dense network f o r the same information transmission. More s i g n i f i c a n t l y , i t demonstrates how information transmission can be applied to design the l e a s t cost network where a choice of instrument p r e c i s i o n e x i s t s . I t should be noted, however, that t h i s does not e n t i r e l y resolve the f u l l economic implications of the data error a r i s i n g from low p r e c i s i o n networks. 130. TABLE 9.1. CORRELATION MATRIX OF THE SIMULATED BASIN WITH MOST PRECISE INSTRUMENTS i ^ \ 1 2 3 4 5 6 1 1.00 0.79 0.78 0.77 0.76 0.75 2 1.00 0.79 0.78 0.77 0.76 3 1.00 0.79 0.77 0.75 4 1.00 0.79 0.77 5 1.00 0.79 6 1.00 131. 6 0 0 0 i 5 0 0 0 | c tt> E 3 k-2 4 0 0 0 1 c 3 0 0 0 1 Q. O o 2 0 0 0 a) E < I 0 0 0 1 o1 0.1 0.2 0.3 0.4 Instrument Error Coefficient, € K 0.5 FIG.9.1-COST PER INSTRUMENT AGAINST INSTRUMENT ERROR COEFFICIENT € K. 132 . FIG.9.2- INFORMATION TRANSMISSION BY VARIOUS OPTIMUM STATION SETS IN THE NETWORKS OF VARIOUS PRECISIONS. 133. =l4000i-c 6 I2000 | -3 C % 10000 J 8 000 o o> Z o» c 6000 4000 o £ 2000 o o (2) <? \ \ V (I) M3) \ s (6) (3) (3) 0 0.1 0.2 0.3 0.4 Instrument Error Coeff icient, € K 0.5 FIG.9.3-COST OF NETWORK WITH LESS PRECISE INSTRUMENTS EQUIVALENT NETWORK HAVING HIGHLY PRECISE INSTRUMENTS. 134. I2000r-IOOOOH c JC o J a> Z 80001— «= 60001-to o a 40001-20001— C o s t of ins ta l l ing Network h a v i n g H igh P r e c i s i o n I n s t r u m e n t O p t i m i z e d C o s t c u r v e for E q u i v a l e n t I n f o r m a t i o n T r a n s m i s s i o n by L e s s P r e c i s e N e t w o r k s 0 1.125 2.25 Optimal Information Transmission (Nats) FIG.9.4-COST COMPARISON OF NETWORKS. 135. CHAPTER X SUMMARY AND CONCLUSIONS The hydrologic data network acts as the p r i n c i p a l l i n k between the hydrologic phenomena occurring i n a basin with the design and operations of water resources p r o j e c t s . The s p a t i a l nature of a basin and i t s hydrologic data leads to problems of s p a t i a l a n a l y s i s . In t h i s Thesis, a t t e n t i o n i s i n i t i a l l y focussed on the problem of determining the optimum number of stations and t h e i r optimum locations i n a basin data c o l l e c t i o n network. The methodology developed i n t h i s t hesis f o r network design i s based upon information theory, a subject which was o r i g i n a l l y established i n the f i e l d of communications engineer-ing. For expository purposes, only p r e c i p i t a t i o n networks have been considered but the methodology developed i s capable of dealing with other types of networks including networks c o l l e c t i n g d i f f e r e n t types of data. The methodology provides a simple c r i t e r i o n which trea t s the network as an integrated system. Information theory provides a compre-hensive and dimensionless performance c r i t e r i o n f o r optimizing and c o l l e c t i o n of s p a t i a l data by networks. This c r i t e r i o n avoids the necessity of specifying any s p a t i a l estimation model. At the same time i t ensures a maximum upper bound of information transmission f o r s p a t i a l estimation i n the future. A p p l i c a t i o n of information theory to network design r a i s e s computational problems of both combinatorial and dimensional types. These have been reduced to a p r a c t i c a l l e v e l by introducing elimination techniques. In add i t i o n a d e r i v a t i o n of information transmission f o r f i t t e d m u l t ivariate continuous d i s t r i b u t i o n s , which may also further reduce 136 . computation, i s presented. The optimal information transmission c r i t e r i o n i s also applicable to the process of estimating events an non s t a t i o n l o c a t i o n s . For example, i t was shown that determining the optimal estimation parameters f o r l i n e a r regression i s simply a s p e c i a l case of optimal information transmission and also applies to the estimation of parameters i n Kalman f i l t e r estimation. The optimal performance bound of an estimation model can also be obtained without a c t u a l l y evaluating the parameters. In conclusion, information theory provides a u n i f y i n g t h e o r e t i c a l and p r a c t i c a l basis for designing s p a t i a l data c o l l e c t i o n systems and associated s p a t i a l estimation models. 137. BIBLIOGRAPHY 1- A l l e n , D.M., Hann, C.T., Linton, D., Street, J . , and Jordon, D., "Stochastic Simulation of Dail y R a i n f a l l " , Kentucky Water Resources I n s t i t u t e , Kentucky Research Report No. 82, 1975, 112p. 2. Bellman, R.E., Introduction to Matrix Analysis, McGraw H i l l , New York, 1960. 3. Benson, M.A., "Use of Multi p l e Regression Analysis i n the Design of a Stream Gaging Network - Practice i n the U.S.A.", Casebook on Hydrometerological Network Design Practice: WiO - No. 324, Geneva, Switzerland, 1972, pp. III-3.2-1 to 3.2-4. 4. Benson, M.A., " A l l o c a t i o n of Stream Gaging Stations within a Country", WMO/IASH Symposium on the Design of Hydrological Network, Quebec IASH Publication No. 67, 1965, pp. 222-228. 5. Bleasdale, A., "Raingage Networks Development and Design with the Special Reference to the United Kingdom, Proc. WMO/IASH No. 67, 1965, pp. 46-54. 6. Bode, H.W., and Shannon, C E . , "A S i m p l i f i e d Derivation o f Linear Least Square Smoothing and Pre d i c t i o n Theory", Proc. IRE, Vol. 38, 1950, pp. 417-425. 7. Bras, R.L., and Rodriquez-Iturbe, I., "Rainfall-Runoff as Sp a t i a l Stochastic Process Data C o l l e c t i o n and Synthesis", Tech. Rep. 196, Ralph M. Parsons Lab. f o r Water Resources and Hydrodyn., MIT Cambridge, Mass., 1975. 8. Bras, R.L., and Rodriquez-Iturbe, I., " R a i n f a l l Generation - A Non-stationary Time Varying Multi-dimensional Model", Water Resources Research, Vol. 12, No. 3, June 1976, pp. 450-456. 9. Bras, R.L., and Rodriquez-Iturbe, I., " R a i n f a l l Network Design for Runoff P r e d i c t i o n " , Water Resources Research, Vol. 12, No. 6, Dec. 1976, pp. 1197-1208. 10. Bucy, R.S., "Nonlinear F i l t e r i n g Theory", IEEE Trans, on Automatic Control, 10, 198, 1965. 11. Carrigan, P.H., and Golden, H.G., "Optimizing Information Transfer i n a Stream-gaging Network", Water Resources Investigation 30-75, U.S. Geological Survey, Sept. 1975, 25p. 12. Davis, D.R., K i s i e l , C.C., and Duckstein, L., "Optimum Design of Mountainous Raingage Networks Using Bayesian Decision Theory", Int. Symp. on D i s t r i b u t i o n of P r e c i p i t a t i o n i n Mountainous Area, Norway, 1972. 13. Dawdy, D.R., and Langbein, W.B., "Mapping Mean Areal P r e c i p i t a t i o n " , IASH Bulletin, No. 10, 1960, pp. 16-23. 138. 14. Dawdy, D.R., Kubik, H.E., and Close, E.R., "Value of Streamflow Data for Project Design - A P i l o t Study", Water Resources Research, Vol. 6, No. 4, 1970, pp. 1045-1050. 15. Desi, F., Czelani, R., and Rackoczi, F., "On Determining the Rational Density o f P r e c i p i t a t i o n Measuring Networks", Proc. WMO/IASH Symp. on the Design of Hydrological Network, Quebec, IASH Publication No. 67, 1965, pp. 127-129. 16. Drozdov, O.A., "A Method f o r Setting up a Network of Meteorological Station f o r a Level Region", Trudy GGO, No. 12(3), 1936. 17. Eagleson, P.S., "Optimum Density of R a i n f a l l Networks", Water Resources Research, Vol. 3, No. 4, 1967, pp. 1021-1033. 18. F i e r i n g , M.B., "An Optimization Scheme for Gaging", Water Resources Research, Vol. 1, No. 4, Fourth Quarter 1965, pp. 463-470. 19. F i e r i n g , M.B., "Use of C o r r e l a t i o n to Improve Estimates of the Mean and Variance", U.S. Geological Survey, Prof. Paper 434-C, 1965, 9pp. 20. F i e r i n g , M.B., "Multivariate Technique for Synthetic Hydrology", Journal of the Hydraulics Division, Proc. ASCE, Vol. 90, HY5, Sept. 1964, pp. 43-61. 21. Freund, J.E., "A B i v a r i a t e Extension of the Exponential D i s t r i b u t i o n " , American Statistical Association Journal, Vol. 56, Dec. 1961, pp. 971-977. 22. Gallager, R.G., Information Theory and Reliable Communication, Wiley, N.Y., 1968. 23. Ganguli, M.K., Rangarajan, R., and Panchang, G.M., "Accuracy o f R a i n f a l l Estimates — Data of Damodar Catchment", Irrigation and Power Journal, Vol. 8, pp. 278-284, 1951. 24. Garner, W.R., and M c G i l l , W.J., "The Relation Between Information and Variance Analyses", Psychometrica, Vol. 21, No. 3, Sept. 1956, pp. 219-230. 25. Grishanin, B.A., "Information Cost i n the Theory of Information Value", Engineering Cybernetics, No. 2, 1967, pp. 6-13. 26. Gumbel, E.J., " B i v a r i a t e Exponential D i s t r i b u t i o n s " , American Statistical Association Journal, Vol. 55, Dec. 1960, pp. 698-707. 27. Gumbel, E.J., " S t a t i s t i c s of Extremes", Columbia U n i v e r s i t y Press, 1958. 28. Hendrick, R.L., "Space V a r i a t i o n o f P r e c i p i t a t i o n and Implications f o r Raingage Network Design", Journal of Hydrology, Vol. 10, 1970, pp. 151-163. 139. 29. Hershfield, D.M., "On the Spacing of Raingages", Proc. WMO/IASH Symp. on the Design o f Hydrological Networks, Quebec, IASH Publication No. 67, 1965, pp. 72-79. 30. Hino, M., "Runoff-Forecasting by Linear P r e d i c t i v e F i l t e r " , Proc. ASCE, Journal of the Hydraulics Division, Vol. 96, HY3, March 1970, pp. 681-702. 31. Hoffmann, W., Objectives and Approaches i n Hydrologic Network Planning and Design", Hydrologic Network Design and Information Transfer, WMO Publication No. 433, 1974, pp. 9-14. 32. Horton, R.E., "The Accuracy of Areal R a i n f a l l Estimates", Monthly Weather Review, pp. 348-353, 1923. 33. Howard, R.L., "Information Value Theory", IEEE Trans. On Systems Science and Cybernetics, Vol. SSC-2, No. 1, Aug. 1966, pp. 22-26. 34. Hutchinson, P., "Estimation of R a i n f a l l i n Sparsely Gauged Areas", Bulletin of IASH, Vol. 14, pp. 101-119, 1969. 35. Huff, F.A., and Changnon, S.A., "Development and U t i l i z a t i o n of I l l i n o i s P r e c i p i t a t i o n Networks", Proc. WMO/IASH Symp. on the Design of Hydrological Network, Quebec, IASH Publication No. 67, 1965, pp. 97-126. 36. Jacobi, S., "Economic Optimum Record Length, Nordic Hydrology, Vol. 6, 1975, pp. 28-42. 37. Kalman, R.E., "A New Approach to Linear F i l t e r i n g and Pr e d i c t i o n Problems", Trans. ASME, Journal of Basic Engineering, Series D, Vol. 82, 1960, pp. 35-45. 38. Kalman, R.E., and Bucy, R.S., "New Results i n Linear F i l t e r i n g and P r e d i c t i o n Theory", Trans. ASME, Journal of Basic Engineering, Vol. 83, March 1961, pp. 995-1007. 39. Kibble, W.F., "A Two Variate Gamma Type D i s t r i b u t i o n " , Sankhya, Vol. 5, 1941, pp. 137-150. 40. Kohler, M.A., "Design of Hydrological Networks", WMO Tech. Note No. 25, 1965, 16pp. 41. Kolmogorov, A.N., "Interpolation and Extrapolation of Stationary Random Sequences", Bulletin Academy of Science USSR, Math Series, Vol. 5, 1941. 42. Kupriianov, W., "General P r i n c i p l e s of Hydrological Network Design", Hydrologic Network Design and Information Transfer, WMO Publication No. 433, 1974, pp. 151-165. 43. Langbein, W.B., "Streamgaging Networks", IASH Pub. No. 38, 1954, pp. 293-303. 140. 44. Langbein, W.B., "National Networks of Hydrological Data", Proc. WMO/ IASH Symp. on the Design of Hydrological Networks, IASH Pub. No. 67, 1965, pp.5-11. 45. Maddock, T., I l l , "An Optimum Reduction of Gauges to Meet Data Program Constraints", Bulletin Int. Assoc. Hydrologic Sciences, Vol. 19, No.3, 1977, pp.337-345. 46. Mardia, K.V., "Families of B i v a r i a t e D i s t r i b u t i o n s " , Griffins Statistical Monograph and Courses, G r i f f i n s , London, pp. 82-107. 47. Matalas, N.C., "Optimum Gaging Station Locations", IBM S c i e n t i f i c Computing Symposium on Water and A i r Resource Management, Yorktown Heights, N.Y., 1973, pp. 85-94. 48. Matalas, N.C. and Langbein, W.B., "Information Content of the Mean", Journal Geophysical Research, Vol. 67, No. 9, pp. 3441-3447. 49. Matalas, N.C, and G i l o r y , E.J. , "Some Comments on Regionalization i n Hydrologic Series", Water Resources Research, Vol. 4, No. 6, 1968, pp. 1361-1369. 50. Moss, M.E., "Expected Economic Record Length as a Basis f o r Hydrologic Network Design", Proc. of Int. Symposium on Water Resources Planning, Mexico C i t y , Dec. 1972. 51. Moss, M.E., "Optimum Operating Procedure f or a River Gaging Stat i o n Established to Provide Data For Design of a Water Supply Project", Water Resources Research, Vol. 6, No. 4, 1970, pp. 1051-1061. 52. Moss, M.E., and Karlinger, M.R., "Surface Water Network Design by Regression Analysis and Simulation", Water Resources Research, Vol. 10, No. 3, pp. 427-433. 53. Moss, M.E., and Dawdy, D.R., "The Worth of Data i n Hydrologic Design", Highway Research Record, No. 479, 1973. 54. Mucciardi, A.N. and Gose, E.E., "A Comparison of Seven Techniques f o r Choosing Subsets of Pattern Recognition Properties", IEEE Trans. Computers, Vol. C-20, No. 9, Sept. 1971. 55. Natale, L., and Todini, E., "A Stable Estimator f o r Linear Model-Theoretical Development and Monte Carlo Experiment", Water Resources Research, Vol. 12, No. 4, Aug. 1976, pp. 667-671. 56. Natale, L., and Todini, E., "A Stable Estimator f o r Linear Model-Real World Hydrologic A p p l i c a t i o n " , Water Resources Research, Vol. 12, No. 4, Aug. 1976, pp. 672-676. 57. Nick, A.D., " F i e l d Evaluation of Raingage Network besign P r i n c i p l e " , WMO/IASH Symp. on the Design of Hydrological Networks, Quebec, IASH Publication No. 67, 1965, pp. 82-93. 58. Nielson, M.D., "Loss of Information by D i s c r e t i z i n g Hydrologic Series", Hydrology Paper 54, Colorado State University, Fort C o l l i n s , Dec. 1972, 57p. 141. 59. Rodda, J . C , "Hydrologic Network Design - Needs, Problems and Approaches", Report No. 12, WMO/IHD Project, 1969. 60. Sage, A.P., and Melsa, J.L., Systems Identification, Academic Press, N.Y., 1971. 61. Sebestyen, G.S., and Edie, J., "Pattern Recognition Research", L i t t o n Systems Waltham, Mass., F i n a l Report, Contract AF 19(628) - 1604 AD 620 236, J u l y 1965. 62. Shannon, C.E., and Weaver, W., "The Mathematical Theory of Communication", The Un i v e r s i t y of I l l i n o i s Press, Urbana, I l l i n o i s , 1949. 63. Stratonovich, R.L., "On Information Cost", Engineering Cybernetics, No. 5, 1965, pp. 1-9. 64. Szollosi-Nagy, A., "Determination of Expected Information Losses Due to Sampling of Hydrological Records i n Time/Space Using Bayesian Decision Theory", Hydrologic Network Design and Information Transfer, WMO Publication No. 433, 1974, pp. 27-43. 65. Solomon, S.I., " M u l t i - r e g i o n a l i z a t i o n and Network Strategy", Casebook on Hydrometeorological Network Design P r a c t i c e : WMO -No. 324, Geneva, Switzerland, 1972, pp. III-3.3-1 to 3.3-11. 66. Solomon, S.I., Denouvilliez, J.P., Chart, E.J., Woolley, J.A., and Cadou, C , "The Use of Square Grid System f o r Computer Estimation of P r e c i p i t a t i o n , Temperature and Runoff", Water Resources Research, Vol. 4, No. 5, 1968, pp. 919-929. 67. S t o l , Ph.Th., "The Relative E f f i c i e n c y of the Density of Raingage Networks", Journal of Hydrology, Vol. 15, 1972, pp. 193-208. 68. Tomita, S., Omatu, S., and Soeda, T., "An A p p l i c a t i o n of Informa-t i o n Theory to F i l t e r i n g Problems", Information Science, Vol. 11, 1976, pp. 13-27. 69. Torranin, P., " A p p l i c a b i l i t y of Cannonical C o r r e l a t i o n i n Hydrology", Hydrology Paper 58, Colorado State University, Fort C o l l i n s , Nov. 1972, 30p. 70. Uryvaeu, V.A., "Basic P r i n c i p l e Governing the Design of a Hydrologi-c a l Network", Proc. WMO/IASH Symp. on the Design of Hydrological Networks, IASH Publication No. 67, 1965, pp. 199-206. 71. Wald, A., Sequential Analysis, New York: Wiley, 1947. 72. W a l l i s , J.R. and Matalas, N.C., "Information Transfer Via Regression i n Markovian Worlds", RC 4207, IBM Research Centre, Yorktown Heights, N.Y., Oct. 1972. 73. Weidemann, H.L., "Entropy Analysis of Parameter Estimation", Information and Control, Vol. 14, 1969, pp. 493-506. 142. 74. Wiener, N., The Extrapolation, Interpolation and Smoothing of Stationary Time Series, John Wiley and Sons Inc., N.Y., 1949. 75. Wilson, K.G., "Cost-Benefit Approach to Hydrometric Network Planning", Water Resources Research, Vol. 8, No. 5, pp. 1347-1353. 76. WMO Guide to Hydrological Practices, No. 168 - T.P. 82. 77. Yukimachi, T., "On Measure of Theoretical Structure of M u l t i v a r i a t e Information Among Random Variables", Information and Control, Vol. 24, 1974, pp. 279-291. 78. Zadeh, L.A. and Ragazzini, J.R., "An Extension of Wieners Theory of P r e d i c t i o n " , Journal of Applied Physics, Vol. 21, 1950, pp. 645-655. 143. APPENDIX A ENTROPY DERIVATIONS OF CONTINUOUS DISTRIBUTIONS Normal D i s t r i b u t i o n : The p r o b a b i l i t y density function of a normal d i s t r i b u t i o n f (x;U,a) can be defined as follows: N f M(x;U,a) N e -(x-y) 20 2 (A-l) where, y and 0 are re s p e c t i v e l y the mean and standard deviation of the random v a r i a b l e X. The entropy, H(X), can be computed as follows: H(X) = f (x;y,0) Un f (x;y,a)dx N N l -(x-y) e 0/2H 2a' -£na - hln 21T - (x-y) 2a2 dx = In 0 + 35£n(2lT) + h or H(X) = ^n(2TTea 2) ( A _ 2 ) Log-normal D i s t r i b u t i o n : The p r o b a b i l i t y density function of a log-normal d i s t r i b u t i o n f (x,-y,C ) i s given as follows: LN -(Jon x-y) . 2a2 144 . where y and O are re s p e c t i v e l y the scale and shape parameters. But, H(X) f (x ;y,a) An f (x,y,a)dx L N L N - (An x -y) . 2 2tT a XV^2TT -Ana-^An(2TT) -An(x) (Anx)-y) 2 2a' dx By s i m p l i f y i n g we get, H(x) = h An(2ir e cr) + y (A-4) Gamma D i s t r i b u t i o n : The p r o b a b i l i t y density function of a gamma d i s t r i b u t e d random va r i a b l e X i s defined as follows: £ , . x V _ 1 " a -V (A-5) f G(x;a,V) = w e x > 0, a,V > 0 where a i s scale parameter and V i s the shape parameter. T(V) i s a gamma function of V and i s defined as follows: r(V) = V - l -x , x e dx The entropy of the above density function i s defined as follows: H(X) = - f (x;a,V) An f (x;a,V) dx G G 145. = -(V-l) £n(-) f (x;a,V)dx + a G — f (x;a,v)dx a VJ + £n a f_(x;a, )dx + £n T(V) G f G(x;a,v)dx -(V-l) E(£n -) + - E ( X ) + £ n a + £n T(v) 3. ci (A-6) where E( * ) i s the expected value of ( • ) The expected values E(X) and E £n(—) can be computed as follows: cl E(X) = -X V - l 1  X V a T(V) x a dx T(V) (-) e a dx = a r(v+p r(v) S i m i l a r l y , E(£n-) a £n-a (*-) a V - l x_ a aT(v) dx Putting = y aHv) „ V - l -y x-ny y e ady T(V) 9v V - l -y y e dy T(V) 8v ^7 r ( V ) But ty{V) = 1 -l r(v) nv) dv 146. where ip(V) i s digamma or P s i function of V. Therefore, E(in -) = i>(v) Substituting the values of E(X) and E(£n —) i n Equation (A-6) we get: cl H(X), = -(V-l) ^(V) + n r ^ } + in (aT(v)) (A-7) Beta D i s t r i b u t i o n : The p r o b a b i l i t y density function of a random v a r i a b l e X, which i s beta d i s t r i b u t e d , i s as follows: A, , A, fB( x ' W - r a j r a j x ( 1 _ x ) (A-8) 0 < x < 1 V A 2 > ° where, A^ and A 2 are shape parameters. The entropy, H(X), can be c a l c u l a t e d as follows: H(X) = - f B ( x ; A 1 , A 2 ) in f (x;A 1,A 2)dx = - in T(A 1+A 2) r l H A J T(A 2) fB ( x ' \ ' \ ) d x " a i - l > " <X2-1) in x f B(x;A^,A 2)dx £n(l-x) f B ( x ; A 1 ( A 2 ) d x 147. ru ) nx > H ( x ) = Z n m +x ) " ( A i - i } E U n X ) " ( X 2 - i ) E ^ n ( 1 - X ) ) (A-9) But, E(JcnX) = i r g 1 + A 2 ) x 2 - 1  Q£ n x . r (A 1)r(A 2) x ( 1 ' x ) to r(x1+x2) a r(X1) nx2) d\± 1 A i - i x 2 - i x (1-x) dx 0 T ( X 1 + X 2 ) T ( X 1 ) T ( X 2 ) 3X, r(X 1 ) ru2) r < V X 2 ) which s i m p l i f i e s to: E(£nX) = ty(X ) - ^ ( X 1 + X 2 ) S i m i l a r l y i t can be proved that E U n (1-x)) = i p ( X 2 ) - T | » ( A 1 + X 2 ) Therefore, H(X) i n Equation (A-9) becomes: H(X) = Jin r ( V r ( X 2 } + (X ) (4»(X_+X_) - 1|J(X ) nx1+x2) 1 1 + ( X 2 _ 1 ) ( ^ ( X ^ ) - i | » ( X 2 ) ) (A-10) Exponential D i s t r i b u t i o n : P r o b a b i l i t y density function of v a r i a b l e X of exponentially d i s t r i b u t e d i s as follows: f E X P ( X ^ } i -x/y — e y (A-ll) 0 < x 148. where y i s the mean of the va r i a b l e .00 H(X) = - i e " X / y (in i - - ^ d x J _ M M r 1 £ny + E ( x ) = £ny + 1 (A-12) For standardized v a r i a b l e with mean one, the entropy w i l l be unity. Extreme Value D i s t r i b u t i o n : The extreme value p r o b a b i l i t y density function f o r the reduced v a r i a b l e X i s as follows: / \ -x -e f E X T ( x ) = e • e -x (A-13) The entropy, H(X), i s derived as follows: .oo H ( X ) = - £ E X T ( X ) £ n f E X T ( X ) d X = E(X) + E(e ) (A-14) For reduced variable,E(X) = 0 Hence, H(X) = E(e ) (A-15) 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0062559/manifest

Comment

Related Items