Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

An analysis of evaluative research : the case of primary health care Chauvin, James Brodie 1985

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1985_A6_7 C42.pdf [ 6.73MB ]
Metadata
JSON: 831-1.0096053.json
JSON-LD: 831-1.0096053-ld.json
RDF/XML (Pretty): 831-1.0096053-rdf.xml
RDF/JSON: 831-1.0096053-rdf.json
Turtle: 831-1.0096053-turtle.txt
N-Triples: 831-1.0096053-rdf-ntriples.txt
Original Record: 831-1.0096053-source.json
Full Text
831-1.0096053-fulltext.txt
Citation
831-1.0096053.ris

Full Text

AN A N A L Y S I S OF E V A L U A T I V E R E S E A R C H : THE CASE OF P R I M A R Y H E A L T H CARE by JAMES B R O D I E C H A U V I N B.A., Q u e e n ' s U n i v e r s i t y a t K i n g s t o n , 1 9 7 3 M.A., Q u e e n ' s U n i v e r s i t y a t K i n g s t o n , 1 9 7 9 A T H E S I S S U B M I T T E D I N P A R T I A L F U L F I L M E N T OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF S C I E N C E i n THE F A C U L T Y OF GRADUATE S T U D I E S . (DEPARTMENT OF H E A L T H CARE AND E P I D E M I O L O G Y HEALTH S E R V I C E S P L A N N I N G AND A D M I N I S T R A T I O N PROGRAM) We a c c e p t t h i s t h e s i s a s c o n f o r m i n g t o t h e r e q u i r e d s t a n d a r d THE U N I V E R S I T Y OF B R I T I S H COLUMBIA A p r i l 1 9 8 5 (c) J a m e s B r o d i e C h a u v i n , 1 9 8 5 In presenting t h i s thesis i n p a r t i a l f u l f i l m e n t of the requirements for an advanced degree at the University of B r i t i s h Columbia, I agree that the Library s h a l l make i t f r e e l y a v a i l a b l e for reference and study. I further agree that permission for extensive copying of t h i s thesis for s c h o l a r l y purposes may be granted by the head of my department or by his or her representatives. I t i s understood that copying or publication of t h i s thesis for f i n a n c i a l gain s h a l l not be allowed without my written permission. Department O f HEALTH CARE AND EPIDEMIOLOGY The University of B r i t i s h Columbia 1956 Main Mall Vancouver, Canada V6T 1Y3 Date MARCH 3, 1985 i i ABSTRACT The p r i m a r y h e a l t h c a r e (PHC) model i s b e i n g a c t i v e l y promoted as an e f f e c t i v e and l o w e r - c o s t a l t e r n a t i v e to c o n v e n t i o n a l h e a l t h c a r e d e l i v e r y s y s tems i n many d e v e l o p i n g c o u n t r i e s . D e s p i t e t h e f a c t t h a t o v e r 300 PHC p r o j e c t s of v a r y i n g s c a l e have been implemented and r e p o r t e d on t h r o u g h o u t t h e T h i r d W o r l d o v e r t h e p a s t two d e c a d e s , t h e r e a p p e a r s to be l i t t l e e v i d e n c e a v a i l a b l e to s u p p o r t t h e p o p u l a r h y p o t h e s i s t h a t t h e a v a i l a b i l i t y and u t i l i z a t i o n o f p r i m a r y h e a l t h c a r e s e r v i c e s n e c e s s a r i l y r e s u l t s i n s i g n i f i c a n t improvements i n h e a l t h . The o b j e c t i v e o f t h i s t h e s i s i s to i d e n t i f y a l t e r n a t i v e s t r a t e g i e s f o r e v a l u a t i n g PHC p r o j e c t s w h i c h w i l l e s t a b l i s h c r e d i b l e and u s e f u l r e s u l t s . The t h e s i s r e v i e w s t h e e v o l u t i o n o f b o t h t h e PHC model and e v a l u a t i v e r e s e a r c h m e t h o d o l o g i e s , and t h e n p r e s e n t s a c r i t i c a l a n a l y s i s of a s e t o f PHC p r o j e c t e v a l u a t i o n s . The aim o f t h i s e x e r c i s e i s to i d e n t i f y some o f the major f a c t o r s w h i c h have l i m i t e d t h e v a l i d i t y , u t i l i t y and s i g n i f i c a n c e o f t h e e v a l u a t i o n r e s u l t s . The t h e s i s s u g g e s t s t h a t l e s s r i g o r o u s e v a l u a t i v e r e s e a r c h d e s i g n s and e v a l u a t i v e t e c h n i q u e s w h i c h use a c o m b i n a t i o n of q u a n t i t a t i v e and q u a l i t a t i v e d a t a be used t o enhance the c r e d i b i l i t y and u t i l i t y o f e v a l u a t i o n r e s u l t s . i i i TABLE OF CONTENTS ABSTRACT i i L I S T OF TABLES v ACKNOWLEDGEMENT . v i INTRODUCTION 1 CHAPTER ONE THE PRIMARY HEALTH CARE MODEL I n t r o d u c t i o n 4 P r i m a r y H e a l t h C a r e D e f i n e d 4 The E v o l u t i o n o f the P r i m a r y H e a l t h C a r e Model 9 The P r e s e n t S t a t u s of t h e PHC Model 19 Summary 24 R e f e r e n c e s 26 CHAPTER TWO EVALUATIVE RESEARCH IN PRIMARY HEALTH CARE I n t r o d u c t i o n 31 E v a l u a t i v e R e s e a r c h D e f i n e d 32 The R a t i o n a l e f o r E v a l u a t i v e R e s e a r c h 34 E v a l u a t i v e R e s e a r c h i n P r i m a r y H e a l t h C a r e . . 36 G e n e r a l A n a l y s i s o f E v a l u a t i v e R e s e a r c h o f PHC Programs 37 D e t a i l e d A n a l y s i s o f F i v e P r i m a r y H e a l t h C a r e E v a l u a t i v e S t u d i e s 47 i . G e n e r a l D e s c r i p t i o n of the F i v e Case S t u d i e s 50 i i . K a v a r V i l l a g e H e a l t h Worker P r o j e c t ( I r a n ) 56 i i i . R u r a l H e a l t h D e velopment Program ( C o l o m b i a ) 62 i v . R u r a l H e a l t h and F a m i l y P l a n n i n g P r o j e c t ( H a i t i ) 65 v. V o l u n t e e r H e a l t h Workers ( T h a i l a n d ) . . 68 v i . R u r a l T e a c h e r s H e a l t h Program ( P a r a g u a y ) 71 A n a l y s i s o f t h e E v a l u a t i o n R e p o r t s 75 Summary 95 R e f e r e n c e s 97 i v CHAPTER THREE ALTERNATIVE APPROACHES FOR CONDUCTING FIELD EVALUATIONS I n t r o d u c t i o n 103 The L e v e l o f C e r t a i n t y R e q u i r e d i n E v a l u a t i v e R e s e a r c h . . 104 A l t e r n a t i v e A p p r o a c h e s i n E v a l u a t i v e R e s e a r c h i . E v a l u a t i v e R e s e a r c h D e s i g n s 110 i i . E v a l u a t i v e Methods and T e c h n i q u e s . . . 119 Summary 136 R e f e r e n c e s 138 CHAPTER FOUR CONCLUSION AND RECOMMENDATIONS 141 SELECTED BIBLIOGRAPHY 151 V L I S T OF TABLES 1. PHC CASE STUDY CHARACTERISTICS 2. CHARACTERISTICS OF EVALUATIONS v i ACKNOWLEDGEMENT I would l i k e t o t a k e t h i s o p p o r t u n i t y t o e x p r e s s my a p p r e c i a t i o n and t h a n k s t o my t h e s i s a d v i s o r s , i n p a r t i c u l a r D r. Nancy W a x i e r M o r r i s o n , f o r t h e i r a s s i s t a n c e , c o n s t r u c t i v e c r i t i c i s m , and encouragement i n c o m p l e t i n g t h i s t a s k . T h i s t h e s i s i s d e d i c a t e d t o my c h i l d r e n , C a r o l i n e and D a v i d . James B. C h a u v i n A p r i l 1985 1 INTRODUCTION Over t h e p a s t f i f t e e n y e a r s , t h e r e has been a s h i f t i n emphasis i n t h e n a t u r e o f h e a l t h c a r e d e l i v e r y systems i m p l e m e n t e d i n t h e m a j o r i t y o f d e v e l o p i n g c o u n t r i e s . T h i r d W o r l d g o v e r n m e n t s , many a t t h e u r g i n g o f f o r e i g n a i d donor n a t i o n s , a r e i n v e s t i n g h u n d r e d s o f m i l l i o n s o f d o l l a r s i n t o t h e p l a n n i n g , d e v e l o p m e n t , and i m p l e m e n t a t i o n o f p r i m a r y h e a l t h c a r e (PHC) s t r a t e g i e s . The p r i m a r y h e a l t h c a r e model was a p p r o v e d as an a l t e r n a t i v e h e a l t h c a r e d e l i v e r y s y s t e m t o c o n v e n t i o n a l a p p r o a c h e s , p a r t i c u l a r l y i n t h e d e v e l o p i n g c o u n t r i e s , by t h e G e n e r a l A s s e m b l y o f t h e W o r l d H e a l t h O r g a n i z a t i o n i n September 1978. The aim o f t h i s a p p r o a c h i s t o i n c r e a s e b o t h t h e a v a i l a b i l i t y and a c c e s s i b i l i t y o f h e a l t h c a r e r e s o u r c e s and s e r v i c e s w h i c h a r e a p p r o p r i a t e t o t h e needs o f , and w h i c h have a s i g n i f i c a n t i m p a c t on t h e h e a l t h o f , t h e t a r g e t p o p u l a t i o n . I t s s u c c e s s i s p r e d i c a t e d on t h e a c t i v e p a r t i c i p a t i o n o f t h e t a r g e t community i n t h e d e f i n i t i o n o f t h e i r h e a l t h p r o b l e m s , and i n t h e d e s i g n and i m p l e m e n t a t i o n o f p r a c t i c a l s o l u t i o n s . A d d i t i o n a l l y , t h e h e a l t h p a r a p r o f e s s i o n a l i s e x p e c t e d t o assume many o f t h e a c t i v i t i e s t r a d i t i o n a l l y c a r r i e d o u t by t h e h e a l t h p r o f e s s i o n a l . ( 1 ) To d a t e , o v e r 300 p r i m a r y h e a l t h c a r e p r o j e c t s have been r e p o r t e d on. O f t e n t h e s e c a s e s t u d i e s r e p o r t o n l y t h e most b a s i c i n f o r m a t i o n , s u c h as t h e number o f h e a l t h 2 p a r a p r o f e s s i o n a l s t r a i n e d and d e p l o y e d , and t h e number o f p e o p l e who u t i l i z e t h e s e r v i c e s o f f e r e d . R a r e l y i s t h e e f f e c t i v e n e s s o f t h i s a p p r o a c h d i s c u s s e d i n terms o f i t s i m p a c t on h e a l t h , much l e s s i t s r e l a t i v e c o s t e f f e c t i v e n e s s i n c o m p a r i s o n t o a l t e r n a t i v e and c o n v e n t i o n a l h e a l t h c a r e d e l i v e r y s y s t e m s . Where i m p a c t e v a l u a t i o n s have been c o n d u c t e d , t h e i r u t i l i t y t e n d s t o be l i m i t e d due t o weaknesses i n t h e e v a l u a t i v e r e s e a r c h d e s i g n and i n t h e p r o c e s s i n v o l v e d i n c o n d u c t i n g the e v a l u a t i o n . The p u r p o s e o f t h i s t h e s i s i s t o i n v e s t i g a t e v a r i o u s s t r a t e g i e s f o r c o n d u c t i n g e v a l u a t i o n s o f p r i m a r y h e a l t h c a r e programs i n f i e l d s e t t i n g s . T h i s w i l l be a c c o m p l i s h e d t h r o u g h an a n a l y s i s o f t h e e v a l u a t i o n r e p o r t s o f s e v e r a l d e v e l o p i n g c o u n t r y - b a s e d p r i m a r y h e a l t h c a r e p r o g r a m s . Based on t h e f i n d i n g s o f t h i s a n a l y s i s , s t r a t e g i e s d e s i g n e d t o i n c r e a s e the c r e d i b i l i t y and u t i l i t y o f an e v a l u a t i o n w i l l be p r o p o s e d . W h i l e t h e f o c u s o f t h i s r e s e a r c h i s on e v a l u a t i v e r e s e a r c h c o n d u c t e d i n t h e T h i r d W o r l d , t h e f i n d i n g s and recommendations a r e i n t e n d e d t o be a p p l i c a b l e g l o b a l l y . C h a p t e r One p r e s e n t s an o v e r v i e w o f t h e d e v e l o p m e n t o f t h e p r i m a r y h e a l t h c a r e s t r a t e g y and t h e j u s t i f i c a t i o n f o r i t s a d o p t i o n as an a l t e r n a t i v e a ndeor a d j u n c t t o e x i s t i n g h e a l t h c a r e d e l i v e r y s y s t e m s i n t h e d e v e l o p i n g c o u n t r i e s . C h a p t e r Two r e v i e w s t h e r o l e o f e v a l u a t i v e r e s e a r c h i n p r i m a r y h e a l t h c a r e , and c o n d u c t s an a n a l y s i s o f s e v e r a l p r i m a r y h e a l t h c a r e program e v a l u a t i o n s . The o b j e c t i v e o f t h i s a n a l y s i s i s t o i d e n t i f y 3 f a c t o r s w h i c h t e n d t o l i m i t t h e v a l i d i t y and u t i l i t y o f t h e r e s u l t s o f t h e s e p r o j e c t e v a l u a t i o n s . B a s e d on t h i s a n a l y s i s , C h a p t e r T h r e e p r e s e n t s a l t e r n a t i v e a p p r o a c h e s f o r c o n d u c t i n g f i e l d - b a s e d e v a l u a t i o n s . These a p p r o a c h e s a r e d e s i g n e d t o m i n i m i z e t h e e f f e c t o f t h e f a c t o r s i d e n t i f i e d i n t h e p r e v i o u s c h a p t e r , w h i l e a t t h e same t i m e e n s u r i n g t h a t t h e e v a l u a t i o n s a r e c r e d i b l e . The f i n a l c h a p t e r p r o v i d e s a summary o v e r v i e w o f t h e arguments and f i n d i n g s o f t h e t h e s i s . A p l a n o f a c t i o n f o r f u r t h e r s t u d y on t h i s t o p i c i s a l s o d i s c u s s e d . REFERENCES 1. U n i t e d N a t i o n s , W o r l d H e a l t h O r g a n i z a t i o n and UNICEF, R e p o r t o f t h e I n t e r n a t i o n a l C o n f e r e n c e on P r i m a r y H e a l t h C a r e . A l m a - A t a , USSR, (ICPHC/ALA/78.10), September 6-12, 1978. 4 CHAPTER ONE THE PRIMARY H E A L T H CARE MODEL I n t r o d u c t i o n O v e r t h e p a s t s e v e r a l y e a r s t h e r e h a s b e e n a s i g n i f i c a n t s h i f t i n t h e n a t u r e o f h e a l t h c a r e s e r v i c e d e l i v e r y s y s t e m s b e i n g i m p l e m e n t e d i n many d e v e l o p i n g c o u n t r i e s . T h i s i s i n r e s p o n s e t o t h e r e a l i z a t i o n t h a t c o n v e n t i o n a l h e a l t h c a r e p r o g r a m s h a v e n o t s u c c e e d e d i n i m p r o v i n g t h e h e a l t h s t a t u s o f t h e v a s t m a j o r i t y o f t h e i r p o p u l a t i o n s . T h e p r i m a r y h e a l t h c a r e m o d e l i s p e r c e i v e d a s a m o r e p r a c t i c a l a n d l e s s e x p e n s i v e a p p r o a c h . T h e p u r p o s e o f t h i s c h a p t e r i s t o e x a m i n e t h i s c o n c e p t . T h e c h a p t e r b e g i n s by d e f i n i n g p r i m a r y h e a l t h c a r e , t r a c e s i t s d e v e l o p m e n t a s a h e a l t h c a r e d e l i v e r y s t r a t e g y , d i s c u s s e s i t s c u r r e n t s t a t u s , a n d t h e means p r o p o s e d t o m e a s u r e i t s s u c c e s s . P r i m a r y H e a l t h C a r e D e f i n e d P r i m a r y h e a l t h c a r e ( a l s o k n o w n by t h e a c r o n y m P H C ) , a s d e f i n e d i n t h e A l m a - A t a C o n f e r e n c e J o i n t R e p o r t f o r t h e W o r l d H e a l t h O r g a n i z a t i o n (WHO) a n d t h e U n i t e d N a t i o n s C h i l d r e n ' s F u n d ( U N I C E F ) , i s e s s e n t i a l l y h e a l t h c a r e b a s e d o n p r a c t i c a l , s c i e n t i f i c a l l y s o u n d a n d s o c i a l l y a c c e p t a b l e m e t h o d s 5 and t e c h n o l o g y made u n i v e r s a l l y a c c e s s i b l e to i n d i v i d u a l s and f a m i l i e s i n the community t h r o u g h t h e i r f u l l p a r t i c i p a t i o n and a t a c o s t t h a t the community and c o u n t r y can a f f o r d , to m a i n t a i n a t e v e r y s t a g e o f t h e i r d e v e l o p m e n t i n the s p i r i t of s e l f - r e l i a n c e and s e l f - d e t e r m i n a t i o n . I t forms an i n t e g r a l p a r t b o t h of t h e c o u n t r y ' s h e a l t h s y s t e m of w h i c h i t i s the c e n t r a " ! f u n c t i o n and main f o c u s o f the o v e r a l l s o c i a l and economic d e v e l o p m e n t o f the community. I t i s the f i r s t l e v e l of c o n t a c t of i n d i v i d u a l s , the f a m i l y and the community w i t h the n a t i o n a l h e a l t h c a r e s y s t e m , b r i n g i n g h e a l t h c a r e as c l o s e as p o s s i b l e to where p e o p l e l i v e and work and c o n s t i t u t e s t h e f i r s t e l e m e n t of a c o n t i n u i n g h e a l t h c a r e p r o c e s s . (1) The aim o f t h i s a p p r o a c h i s to p r o v i d e the e n t i r e p o p u l a t i o n w i t h e s s e n t i a l h e a l t h c a r e . T h i s i s a c h i e v e d by i n c r e a s i n g b o t h the a v a i l a b i l i t y and a c c e s s i b i l i t y o f h e a l t h c a r e s e r v i c e s and r e s o u r c e s w h i c h a r e a p p r o p r i a t e to t h e needs and e x p e c t a t i o n s o f t h e i n t e n d e d r e c i p i e n t p o p u l a t i o n w i t h i n the bounds o f a v a i l a b l e r e s o u r c e s . From an o p e r a t i o n a l v i e w p o i n t , t h i s a p p r o a c h c o n s i s t s of the f o l l o w i n g b a s i c e1ement s: 1. a c t i v i t i e s p e r t a i n i n g to the p r i m a r y c a r e l e v e l o f the f o r m a l o r i n s t i t u t i o n a l i z e d h e a l t h s y s t e m ; 2. community a c t i o n s p e r t a i n i n g to t h e t r a d i t i o n a l or ' i n f o r m a l ' h e a l t h s y s t e m w h i c h a r e d i r e c t e d a t s o l v i n g l o c a l h e a l t h p r o b l e m s ; 3. the c o n c e p t o f u n i v e r s a l c o v e r a g e and a c c e s s i b i l i t y to a p p r o p r i a t e h e a l t h t e c h n o l o g i e s ; and, 6 4 . mechanisms w h i c h must be c r e a t e d and s u p p o r t e d f o r L i n k i n g t h e s e t h r e e s e t s of a c t i v i t i e s and i n c o r p o r a t i n g them i n t o an i n t e g r a t e d a p p r o a c h to s o c i o e c o n o m i c d e v e l o p m e n t . (2 ) The n a t u r e of the s e r v i c e s p r o v i d e d and t h e i r mode o f d e l i v e r y a r e n o t r e s t r i c t e d to any one s t r a t e g y . T h i s a p p r o a c h d e p a r t s f r o m the c o n v e n t i o n a l c u r a t i v e , h o s p i t a l - b a s e d and p h y s i c i a n - f o c u s e d model. I n t h e o r y , the PHC a p p r o a c h c o n s i s t s of m u l t i p l e s e r v i c e s , e n c o m p a s s i n g s u c h a c t i v i t i e s as h e a l t h e d u c a t i o n , p r o m o t i o n , r e h a b i l i t a t i o n and p r e v e n t i v e s e r v i c e s , and f a m i l y h e a l t h c a r e . T h e s e s e r v i c e s i n c l u d e i m m u n i z a t i o n c a m p a i g n s , c o n t r o l of endemic v e c t o r - b o r n e d i s e a s e s , and m o t h e r - c h i l d h e a l t h p r o g r a m s . P r i m a r y h e a l t h c a r e i s i n t e n d e d to be p a r t of t h e b r o a d e r d e v e l o p m e n t p r o c e s s . The s t r a t e g i e s f o r i m p r o v i n g the h e a l t h o f the community a r e n o t r e s t r i c t e d to h e a l t h a c t i v i t i e s . T h i s i n t e r s e c t o r a 1 a p p r o a c h i s ba s e d on the t e n e t t h a t improvements i n h e a l t h a r e n o t s i m p l y a c o n s e q u e n c e of m e d i c a l or h e a l t h c a r e i n t e r v e n t i o n s . They a r e a l s o i n f l u e n c e d by f a c t o r s s u c h as n u t r i t i o n , e d u c a t i o n , s a n i t a t i o n , w a t e r , and h o u s i n g . T h e s e i n t e r v e n t i o n s a r e t h o u g h t to have a s y n e r g i s t i c e f f e c t on h e a l t h , when such programs a r e u n d e r t a k e n s i m u l t a n e o u s l y to h e a l t h c a r e p r o g r a m s . I n e s s e n c e , the a p p r o a c h r e l i e s on t h e c o o r d i n a t i o n of a c t i v i t i e s w i t h i n a v a r i e t y o f s e c t o r s , w h i c h u l t i m a t e l y i n f l u e n c e the h e a l t h s t a t u s o f t h e community. 7 The u t i l i z a t i o n of i n d i g e n o u s h e a l t h p a r a p r o f e s s i o n a 1 s , who a r e e x p e c t e d to f u n c t i o n i n a v i l l a g e s e t t i n g , i s a n o t h e r e l e m e n t o f t h i s a p p r o a c h which s e t s i t a p a r t from c o n v e n t i o n a l a p p r o a c h e s . I f p o s s i b l e , t h e y a r e drawn from the v i l l a g e or r u r a l community t a r g e t e d to r e c e i v e p r i m a r y h e a l t h c a r e s e r v i c e s . As s u c h , t h o s e r e s p o n s i b l e f o r d e l i v e r i n g t h e s e s e r v i c e s have a p e r s o n a l s t a k e i n e n s u r i n g t h e i r c o r r e c t d e l i v e r y . B e c a u s e of the many s i d e d n a t u r e of the t a s k s i n v o l v e d , the p a r a p r o f e s s i o n a l ' s d u t i e s a r e n o t g e n e r a l l y c o n f i n e d to h e a l t h work. R a t h e r , they w i l l be r e l a t e d as much as p o s s i b l e to the many a s p e c t s of s o c i a l and community l i f e t h a t a f f e c t s an i n d i v i d u a l ' s we 1 1 - b e i n g . ( 3 ) The s u c c e s s o f t h i s a p p r o a c h a l s o r e l i e s on the a c t i v e p a r t i c i p a t i o n of community members i n the i d e n t i f i c a t i o n of l o c a l needs and the d e v e l o p m e n t and i m p l e m e n t a t i o n o f t h e means which r e s p o n d to them. The i n c l u s i o n of t h i s e l e m e n t i s based on the f i n d i n g s of the j o i n t UNICEF/WHO s t u d y o f a l t e r n a t i v e a p p r o a c h e s to m e e t i n g b a s i c h e a l t h c a r e n e e d s . I t n o t e d t h a t s u c c e s s f u l , s i m p l i f i e d h e a l t h c a r e programs were c h a r a c t e r i z e d by e x t e n s i v e community p a r t i c i p a t i o n . ( 4 ) As the R e p o r t to the I n t e r n a t i o n a l C o n f e r e n c e on P r i m a r y H e a l t h C a r e s t a t e s : Community p a r t i c i p a t i o n i s the p r o c e s s by w hich i n d i v i d u a l s and f a m i l i e s assume r e s p o n s i b i l i t y f o r t h e i r own h e a l t h and w e l f a r e and f o r t h o s e of the community, and d e v e l o p the c a p a c i t y to c o n t r i b u t e to t h e i r and the community's d e v e l o p m e n t . T h i s e n a b l e s them to become a g e n t s o f t h e i r own d e v e l o p m e n t i n s t e a d of p a s s i v e b e n e f i c i a r i e s o f d e v e l o p m e n t a i d . (5) 8 C o n t r o l of the p l a n n i n g and o p e r a t i o n of community h e a l t h c a r e s e r v i c e s a r e t h e r e f o r e t r a n s f e r e d from t h e m e d i c a l p r o f e s s i o n and p o l i t i c i a n s to community members. T h i s assumes t h a t i n d i v i d u a l s a r e c o g n i z a n t of the h e a l t h p r o b l e m s w h i c h a f f e c t them, as w e l l as the means to r e s o l v e them. I t a l s o assumes t h a t the community w i l l have the r e s o u r c e s a t i t s d i s p o s a l to d e s i g n and implement them. No one PHC s t r a t e g y i s a d v o c a t e d f o r a l l s i t u a t i o n s . L o c a l h e a l t h c o n d i t i o n s and e c o l o g i c a l f a c t o r s w i l l d e f i n e t h e most a p p r o p r i a t e r e s p o n s e . E a c h c a s e may r e q u i r e d i f f e r e n t l e v e l s and t y p e s of c u r a t i v e and p r e v e n t i v e m e a s u r e s , as w e l l as v a r y i n g modes o f s e r v i c e d e l i v e r y . C o n s e q u e n t l y , the n a t u r e and c o m b i n a t i o n of e l e m e n t s w i t h i n a p r i m a r y h e a l t h c a r e program tend to d i f f e r from p l a c e to p l a c e . T h i s f l e x i b i l i t y i s the s t r e n g t h o f t h e a p p r o a c h . B e s i d e s w e l l - d e f i n e d o p e r a t i o n a l s t r a t e g i e s , the p r i m a r y h e a l t h c a r e a p p r o a c h a l s o s t r e s s e s the need f o r c o n t i n u o u s e v a l u a t i o n of program p e r f o r m a n c e . The o b j e c t i v e of any e v a l u a t i v e e x e r c i s e s h o u l d be to e n s u r e t h a t the system i s f u n c t i o n i n g ' c o r r e c t l y ' . T h a t i s , t h a t the e l e m e n t s which c o n s t i t u t e a p a r t i c u l a r PHC s t r a t e g y a r e e f f e c t i v e i n i m p r o v i n g the h e a l t h of the r e c i p i e n t community. A c c o r d i n g to the p r i m a r y h e a l t h c a r e c o n c e p t , as d e f i n e d a t A l m a - A t a , e v a l u a t i o n i s a c o n t i n u o u s p r o c e s s , m e a s u r i n g the c a p a c i t y of the system to 9 p e r f o r m as i n t e n d e d . I t a l s o s e r v e s to i d e n t i f y weaknesses and s t r e n g t h s i n t h e o p e r a t i o n and m a i n t e n a n c e of the system which a f f e c t i t s e f f e c t i v e n e s s . Such f e e d b a c k i s e s s e n t i a l f o r p r o p e r management and p o l i c y d e c i s i o n s . As d e f i n e d , the p r i m a r y h e a l t h c a r e s t r a t e g y i s meant to s e r v e as a c o m p r e h e n s i v e and we 11-deve1oped means f o r d e l i v e r i n g a p p r o p r i a t e , l o w - c o s t h e a l t h c a r e s e r v i c e s , p a r t i c u l a r l y to u n d e r s e r v i c e d p o p u l a t i o n s . The E v o l u t i o n of the P r i m a r y H e a l t h C a r e Model C e r t a i n e l e m e n t s of the p r i m a r y h e a l t h c a r e a p p r o a c h a r e n o t r e c e n t d e v e l o p m e n t s . Community-based and s u p p o r t e d s e r v i c e s were o p e r a t i n g i n most d e v e l o p i n g c o u n t r i e s l o n g b e f o r e the e s t a b l i s h m e n t of the E u r o p e a n h e a l t h c a r e systems under the c o l o n i a l a d m i n i s t r a t i o n s . The p r e s e n t p o p u l a r l a b e l f o r such h e a l t h c a r e d e l i v e r y systems i s ' t r a d i t i o n a l m e d i c i n e ' . But i n d i g e n o u s m e d i c a l p a r a p r o f e s s i o n a 1 s were a l s o t r a i n e d and d e p l o y e d to s e r v e w i t h i n the c o l o n i a l s y s t e m s . F o r example, d u r i n g the 1 7 t h C e n t u r y i n J a m a i c a , p h y s i c i a n ' s a t t e n d a n t s who had s e r v e d a p p r e n t i c e s h i p s and had p a s s e d e x a m i n a t i o n s were a l l o w e d to s e l l h e r b a l p r e p a r a t i o n s and d r u g s , and c o u l d work as h e a l e r s i n t h e r u r a l a r e a s . F o l l o w i n g e m a n c i p a t i o n , f o r m a l t r a i n i n g f o r i n d i g e n o u s m e d i c a l d i s p e n s e r s was e s t a b 1 i s h e d . ( 6 ) T h i s s y s t e m was c o p i e d w i t h v a r i a t i o n s i n s e v e r a l A f r i c a n 10 c o l o n i e s . The t r a i n i n g and d e p l o y m e n t o f m e d i c a l a s s i s t a n t s was u n d e r t a k e n i n Uganda and o t h e r B r i t i s h c o l o n i e s by the C h u r c h M i s s i o n S o c i e t y . ( 7 ) I n S e n e g a l , the F r e n c h c o l o n i a l a d m i n i s t r a t i o n f o u n d e d the E c o l e A f r i c a i n e de M e d ecine i n 1918, where a u x i l i a r i e s were t r a i n e d f o r the ' S e r v i c e s de g r a n d e s endemies' .(8 ) S i m i l a r programs were d e v e l o p e d and implemented i n M a d a g a s c a r , N i g e r i a , Z a n z i b a r and F i j i . T h e se c a d r e s o f p a r a p r o f e s s i o n a 1 s were an e f f e c t i v e and l o w - c o s t means o f d e l i v e r i n g h e a l t h c a r e s e r v i c e s to t h e r u r a l p o p u l a t i o n s f o r two r e a s o n s . F i r s t , the i n d i g e n o u s p a r a p r o f e s s i o n a 1 s were e f f e c t i v e s i n c e t h e y c a r e d f o r t h e i r own p e o p l e i n c o n d i t i o n s t h e s e p e o p l e were a c c u s t o m e d t o . ( 9 ) S e c o n d l y , most E u r o p e a n d o c t o r s , w i t h the e x c e p t i o n o f a few d e d i c a t e d i n d i v i d u a l s , seldom l e f t the c i t i e s and towns i n the c o l o n i e s . The i n d i g e n o u s p a r a p r o f e s s i o n a 1 s t h e r e f o r e a c t e d as s u b s t i t u t e s f o r the E u r o p e a n d o c t o r s i n the r u r a l a r e a s . However, su c h programs u s u a l l y e x i s t e d o n l y i n the h i n t e r l a n d s u r r o u n d i n g the major c o l o n i a l towns. A d d i t i o n a l l y , t h e s e programs tended to be s p o r a d i c , and r e c e i v e d l i t t l e s u p p o r t from government. S t r a t e g i e s d e s i g n e d to promote the l e g i t i m i z a t i o n and i n t e g r a t i o n of t h e i n d i g e n o u s p a r a p r o f e s s i o n a 1 s i n t o the 'European' h e a l t h c a r e system were a c t i v e l y r e s i s t e d . C o n s e q u e n t l y , the i n d i g e n o u s p r a c t i t i o n e r s t ended to be a d j u n c t s to the E u r o p e a n d o c t o r s . As a r e s u l t , the u r b a n r i c h f o r whom the s y s t e m was r e a s o n a b l y w e l l s u i t e d 11 c o n t i n u e d to r e c e i v e a d e q u a t e t r e a t m e n t , whereas most of the r u r a l and low-income p e r i p h e r a l u r b a n p o p u l a t i o n s had l i t t l e or no a c c e s s to m e d i c a l s e r v i c e s . ( 1 0 ) Up to t h e end o f the Second World War, t h e d e v e l o p m e n t of E u r o p e a n - s t y 1 e d h e a l t h s e r v i c e s i n the r u r a l a r e a s of the ' c o l o n i e s ' r e s t e d w i t h the m i s s i o n a r i e s and p r i v a t e b u s i n e s s i n t e r e s t s . However, t h e s e t e n d e d to be q u i t e d i s p e r s e d , and were l i m i t e d i n t h e r a n g e o f s e r v i c e s o f f e r e d . The major emphasis was c u r a t i v e m e d i c i n e . T h e r e was a l s o l i t t l e i f any c o o r d i n a t i o n among the m e d i c a l m i s s i o n s , or between the m i s s i o n - r e l a t e d s e r v i c e s and g o v e r n m e n t - s u p p o r t e d h e a l t h c a r e s e r v i c e s . ( 1 1 ) As a r e s u l t , f o r m a l h e a l t h c a r e s e r v i c e s were o f t e n d u p l i c a t e d , and t h e r e was a l a c k of i n t e g r a t i o n i n t o the r e g i o n a l o r n a t i o n a l h e a l t h c a r e d e l i v e r y s y s t e m . W h i l e the i n d u s t r i a l and a g r i c u l t u r a l s e c t o r - r e l a t e d h e a l t h c a r e s e r v i c e s t e n d e d to i n c l u d e some p u b l i c h e a l t h a c t i v i t i e s i n a d d i t i o n to t h e i r c u r a t i v e s e r v i c e s , t h e i r o b j e c t i v e was q u i t e s t r a i g h t f o r w a r d : to e n s u r e a h e a l t h y and p r o d u c t i v e l a b o u r f o r c e . The r a t i o n a l e f o r the d e v e l o p m e n t o f f a c t o r y and p l a n t a t i o n - b a s e d h e a l t h s e r v i c e s i s s t a t e d i n the remarks o f a f o r m e r v i c e - p r e s i d e n t o f the U n i t e d F r u i t Company: The work t h a t has been done ( i n p r o v i d i n g h e a l t h s e r v i c e s ) was done f o r a v e r y p r a c t i c a l h a r d - h e a d e d r e a s o n - t h a t o f s e l f - i n t e r e s t . . . s i c k p e o p l e c a n n o t work . . . I t may have been an e n l i g h t e n e d s e l f - i n t e r e s t , b u t i t was l a r g e l y done b e c a u s e 12 ( A m e r i c a n b u s i n e s s i n t e r e s t s ) c o u l d n o t g e t o u t the o r e , o r r a i s e the bananas . . . u n l e s s t h e s e f u n d a m e n t a l s were t a k e n c a r e o f . (12) A l t h o u g h t h e s e h e a l t h programs s e r v e d the need of p r o v i d i n g a h e a l t h y and p r o d u c t i v e w o r k f o r c e , t h e i r e f f e c t was m i n i m a l on the h e a l t h and w e l l - b e i n g o f community members i n g e n e r a l . The range o f s e r v i c e s was u s u a l l y l i m i t e d to immediate h e a l t h needs of the w o r k f o r c e , i n p a r t i c u l a r emergency f i r s t a i d . H e a l t h and m e d i c a l s e r v i c e s f o r women, c h i l d r e n and t h e e l d e r l y were g e n e r a l l y n o n e x i s t e n t . They were c o n s i d e r e d to be n o n p r o d u c t i v e members of the g e n e r a l p o p u l a t i o n , and as su c h the f o r e i g n companies were n o t r e s p o n s i b l e f o r t h e i r g e n e r a l w e l f a r e . O n l y when i t was p e r c e i v e d t h a t l o c a l m o r b i d i t y and m o r t a l i t y r a t e s were e x t r e m e l y h i g h and t h r e a t e n i n g the c a p a c i t y o f women to be a r c h i l d r e n (and thus r e p l e n i s h t h e l a b o u r f o r c e ) were m o t h e r - c h i l d programs e s t a b 1 i s h e d . ( 1 3 ) The s o l u t i o n was t e c h n i c a l and b i o l o g i c a l r a t h e r t h a n s o c i a l , p o l i t i c a l o r eco n o m i c . S u c c e s s o f su c h programs was n o t measured i n terms o f changes i n the q u a l i t y of l i f e , b u t r a t h e r by q u a n t i t a t i v e i n c r e a s e s i n p r o d u c t i v i t y . ( l k ) With the e s t a b l i s h m e n t o f f o r m a l f o r e i g n a i d programs f o l l o w i n g the Second W o r l d War, h e a l t h was i d e n t i f i e d as one of the p r i n c i p a l s e c t o r s to be d e v e l o p e d . I l l - h e a l t h was p e r c e i v e d as a h i g h l y v i s i b l e p r o b l e m , one w h i c h c o u l d c a u s e s o c i a l u n r e s t . T h e r e was a l s o a g r o w i n g v o c a l demand f o r n a t i o n a l i n d e p e n d e n c e i n the T h i r d W o r l d . The f e a r o f 'Communism' and of 13 i t s p o t e n t i a l i n f l u e n c e on t h e n a t i o n a l i s t i c t e n d e n c i e s of some d e v e l o p i n g n a t i o n s l e n t i t s e l f to the r h e t o r i c of the p e r i o d . The o p e n i n g s t a t e m e n t by the Dean o f the H a r v a r d S c h o o l of P u b l i c H e a l t h a t t h e C o n f e r e n c e on H e a l t h P r o b l e m s of I n d u s t r i e s O p e r a t i n g i n T r o p i c a l C o u n t r i e s h e l d a t H a r v a r d i n 1950 r e f l e c t s t h i s s e n t i m e n t : P o w e r f u l Communist f o r c e s a r e a t work . . . t h r o u g h o u t the w o r l d , t a k i n g a d v a n t a g e o f s i c k and i m p o v e r i s h e d p e o p l e , e x p l o i t i n g t h e i r d i s c o n t e n t and h o p e l e s s n e s s to u ndermine t h e i r p o l i t i c a l b e l i e f s . H e a l t h i s one s a f e g u a r d a g a i n s t t h i s p r o p a g a n d a . H e a l t h i s n o t c h a r i t y , i t i s n o t m i s s i o n a r y work, i t i s n o t m e r e l y good b u s i n e s s - i t i s s h e e r s e l f - p r e s e r v a t i o n f o r us and f o r the way of l i f e w h i c h we r e g a r d as d e c e n t . T h r o u g h h e a l t h we can . . . p r o v e . . . to the w o r l d the wholesomeness and T i g h t n e s s o f Democracy. T h r o u g h h e a l t h we can d e f e a t t h e e v i l t h r e a t o f Communism. (15) The p o 1 i t i c i z a t i o n of h e a l t h as a d e v e l o p m e n t a l i n s t r u m e n t i n the T h i r d World was to have a major e f f e c t on the n a t u r e of h e a l t h c a r e s e r v i c e s w h i c h d e v e l o p e d t h e r e a f t e r . The s t r a t e g y w h i c h e v o l v e d f o r d e v e l o p i n g the h e a l t h c a r e s e r v i c e s i n f r a s t r u c t u r e i n d e v e l o p i n g c o u n t r i e s was p r e d i c a t e d on two a s s u m p t i o n s : 1. t h a t t h e b e s t and most advanced w e s t e r n p r e v e n t i v e and c u r a t i v e m e d i c a l p r a c t i c e s , and the i n s t i t u t i o n a l framework t h a t p r o v i d e s t h e s e s e r v i c e s , a r e u n i v e r s a l s t h a t work e q u a l l y w e l l i n a l l s o c i o - c u 1 t u r a 1 and economic s e t t i n g s ; and, 14 t h a t the p e o p l e i n i m m e d i a t e l y p e r c e i v e t h e i f t h e y g i v e up o l d med ones. (16) d e v e l o p i n g c o u n t r i e s w i l l a d v a n t a g e s a c c r u i n g to them i c a l p r a c t i c e s and a d o p t new The e t h n o c e n t r i c p e r c e p t i o n t h a t w e s t e r n c i v i l i z a t i o n was s u p e r i o r i n a l l ways to a l l o t h e r s o c i e t i e s and t h a t g i v e n the o p p o r t u n i t y , p e o p l e i n the ' l e s s f o r t u n a t e ' c o u n t r i e s would c l a m o u r to a d o p t w e s t e r n models o f h e a l t h c a r e and o t h e r s o c i a l s e r v i c e s d e l i v e r y had a major e f f e c t on the d i r e c t i o n o f d e v e l o p m e n t o f the s o c i a l s e r v i c e i n f r a s t r u c t u r e i n most o f the T h i r d W o r l d . I n the h e a l t h c a r e f i e l d f o r i n s t a n c e , w e s t e r n f o r e i g n a s s i s t a n c e programs f o c u s s e d on the c o n s t r u c t i o n of major m e d i c a l c e n t r e s , the i m p l e m e n t a t i o n of m a s s i v e p u b l i c h e a l t h p r o g r a m s , and a d r i v e t o o r i e n t m e d i c a l e d u c a t i o n i n the d e v e l o p i n g c o u n t r i e s to a d o p t and c o n f o r m to w e s t e r n s t a n d a r d s . ( 1 7 ) Emphasis on the e s t a b l i s h m e n t of c e n t r e s o f e x c e l l e n c e f o r t r a i n i n g and a d m i n i s t r a t i o n was l e g i t i m i z e d by w e s t e r n m e d i c a l a s s o c i a t i o n s , p r i v a t e f o u n d a t i o n s , and by the r e c e n t l y formed World H e a l t h O r g a n i z a t i o n . T hese g r o u p s p r o m u l g a t e d ' i n t e r n a t i o n a l ' s t a n d a r d s f o r c u r r i c u l a and f o r s t a f f i n g i n the h e a l t h f i e l d . The s t a n d a r d s were e x p e c t e d to e n s u r e t h a t m e d i c a l and h e a l t h c a r e s e r v i c e s e s t a b l i s h e d i n the d e v e l o p i n g c o u n t r i e s would meet the h e a l t h needs of each r e s p e c t i v e 15 n a t i o n , w h i l e accommodating the economic and t e c h n i c a l c o n s t r a i n t s on h e a l t h c a r e t h a t had l e d e a r l i e r to the use o f d r e s s e r s , d i s p e n s e r s and m e d i c a l a s s i s t a n t s . ( 1 8 ) The a i d programs a l s o s e r v e d to r e i n f o r c e t h e r o l e of c u r a t i v e m e d i c i n e , e f f e c t i v e l y r e s t r i c t i n g t h e scope and range of s e r v i c e s o f f e r e d . I n many p o s t - i n d e p e n d e n c e d e v e l o p i n g c o u n t r i e s , t h e i n d i g e n o u s w e s t e r n - t r a i n e d d o c t o r s who i n h e r i t e d the c o l o n i a l h e a l t h c a r e systems c h o s e to m a i n t a i n them p r i m a r i l y f o r p o l i t i c a l and s o c i a l r e a s o n s . Not o n l y d i d t h i s s e r v e to e n s u r e t h e i r dominance as a new e l i t e , b u t a l s o p r o v i d e d them the e x c u s e to r e m a i n i n t h e major u r b a n a r e a s , where the s u p p o r t i n g f a c i l i t i e s were l o c a t e d , " i n c l u d i n g good s c h o o l s f o r t h e i r c h i l d r e n and a l i f e s t y l e s u i t e d to the 'new c o l o n i a l s ' " . ( 1 9 ) A c c e s s i b i l i t y to f o r m a l h e a l t h s e r v i c e s f o r t h e r u r a l p o p u l a t i o n s was f u r t h e r l i m i t e d i n some c o u n t r i e s by the d i s m a n t l i n g o f the i n d i g e n o u s h e a l t h c a r e s e r v i c e s . A number o f newly i n d e p e n d e n t n a t i o n s n o t o n l y s o u g h t to a b o l i s h what were p e r c e i v e d to be s e c o n d - c l a s s s e r v i c e s e s t a b l i s h e d under the c o l o n i a l r e g i m e s , b u t some governments a l s o o u t l a w e d the p r a c t i c e o f t r a d i t i o n a l m e d i c i n e . ( 2 0 ) C o n s e q u e n t l y , the o p e r a t i o n and m a i n t e n a n c e o f a r u r a l h e a l t h c a r e s y s t e m c e a s e d i n many c o u n t r i e s . What s e r v i c e s c o n t i n u e d to e x i s t were p r o v i d e d e i t h e r c l a n d e s t i n e l y by t r a d i t i o n a l p r a c t i t i o n e r s , or were p r o v i d e d by m i s s i o n a r y and p r i v a t e n o n - g o v e r n m e n t a l 16 o r g a n i z a t i o n s and f o u n d a t i o n s . D u r i n g t h e 1960s and 1970s, b i l l i o n s of d o l l a r s of f o r e i g n a s s i s t a n c e were d i r e c t e d towards the d e v e l o p m e n t o f h e a l t h c a r e d e l i v e r y s y s tems i n the d e v e l o p i n g c o u n t r i e s . D e s p i t e the s c a l e o f t h e s e p r o g r a m s , the s t a n d a r d of l i v i n g , and i n p a r t i c u l a r the h e a l t h s t a t u s o f the r u r a l and p e r i - u r b a n p o p u l a t i o n s of t h e s e c o u n t r i e s d i d n o t show improvement.(21) The e x p e c t a t i o n t h a t t h e s e m e d i c a l c e n t r e s o f e x c e l l e n c e would p r o d u c e d o c t o r s to s e r v e f a r m e r s , p r o d u c t i o n w o r k e r s , and t h e i r f a m i l i e s was n o t becoming a r e a l i t y . ( 2 2 ) I n 1974, the World H e a l t h O r g a n i z a t i o n and the U n i t e d N a t i o n s C h i l d r e n ' s Fund (UNICEF) i s s u e d a j o i n t r e p o r t s t a t i n g t h a t " t h e s t r a t e g y so f a r a d o p t e d by many d e v e l o p i n g c o u n t r i e s has f a i l e d " . ( 2 3 ) T h i s d i s a p p o i n t i n g p e r f o r m a n c e was a t t r i b u t e d to a l a c k o f commitment on the p a r t of T h i r d World governments to improve the s t a n d a r d o f l i v i n g and t h e p r o v i s i o n o f s o c i a l s e r v i c e s to the r u r a l a r e a s as w e l l as to t h e i n t e r n a l l i m i t a t i o n s o f e x i s t i n g r u r a l h e a l t h c a r e s e r v i c e s . ( 2 4 ) I n g e n e r a l , the f o r m a l r u r a l h e a l t h c a r e d e l i v e r y s y s t e m was c o n s i d e r e d i n e f f e c t i v e and c o s t l y b e c a u s e i t was p o o r l y s u i t e d to the e x p e c t a t i o n s of the p o p u l a t i o n , and o f t e n i n a p p r o p r i a t e to t h e i r n e e d s . Though t h e s e h e a l t h c a r e s e r v i c e s d i d p r o v i d e a b r o a d r a n g e o f c u r a t i v e s e r v i c e s , a c t i v i t i e s s u c h as i m m u n i z a t i o n , h e a l t h e d u c a t i o n , and community improvement 17 programs a d d r e s s i n g t h e c a u s e s of i l l n e s s and d i s e a s e were l a c k i n g . The f i n a n c i a l b u r d e n a s s o c i a t e d w i t h the m a i n t e n a n c e of a h o s p i t a 1 / c 1 i n i c - b a s e d h e a l t h c a r e d e l i v e r y s y s t e m was enormous and l i m i t e d t h e e f f i c i e n c y o f the h e a l t h c a r e d e l i v e r y s y s t e m . Few T h i r d W o r l d governments p o s s e s s e d e i t h e r the f i n a n c i a l r e s o u r c e s r e q u i r e d to m a i n t a i n a sy s t e m w h i c h conformed to a c c e p t e d i n t e r n a t i o n a l s t a n d a r d s , o r the a d m i n i s t r a t i v e c a p a c i t y to o v e r s e e i t s e f f i c i e n t o p e r a t i o n . W h i l e f o r e i g n a s s i s t a n c e programs c o v e r e d the c a p i t a l c o s t s o f c o n s t r u c t i n g the showcase f a c i l i t i e s and t h e p u r c h a s e o f the a t t e n d a n t m e d i c a l h a r d w a r e , r e c u r r e n t o p e r a t i n g c o s t s were the r e s p o n s i b i l i t y o f the r e c i p i e n t government. I n some c a s e s , r e c u r r e n t o p e r a t i n g c o s t s a n n u a l l y a c c o u n t e d f o r up to 25% of the o r i g i n a l f a c i l i t y c o n s t r u c t i o n c o s t s . ( 2 5 ) One s t u d y on the f i n a n c i a l b u r d e n o f m a i n t a i n i n g c o n v e n t i o n a l m e d i c a l f a c i l i t i e s and s e r v i c e s i n d e v e l o p i n g c o u n t r i e s r e p o r t s t h a t the o p e r a t i n g c o s t s o f a s i n g l e t e a c h i n g h o s p i t a l c a n a b s o r b o n e - t h i r d o f a n a t i o n ' s t o t a l h e a l t h b u d g e t . ( 2 6 ) I n many d e v e l o p i n g c o u n t r i e s , o v e r o n e - h a l f of t h e n a t i o n a l h e a l t h b u d g e t s was s p e n t on m a i n t a i n i n g u r b a n - b a s e d c u r a t i v e h e a l t h c a r e s e r v i c e s , e ven though t h e u r b a n p o p u l a t i o n a c c o u n t e d f o r l e s s than o n e - f i f t h o f t h e t o t a l n a t i o n a l p o p u l a t i o n . ( 2 7 ) By the b e g i n n i n g of the 1970s, i t was e s t i m a t e d t h a t o v e r 407. of the d e v e l o p i n g w o r l d ' s p o p u l a t i o n was w i t h o u t a c c e s s to w e s t e r n 18 b i o m e d i c a l h e a l t h s e r v i c e s , and t h a t t h o s e l i v i n g i n l o c a l i t i e s o f l e s s t h a n 20,000 i n h a b i t a n t s had a c c e s s to o n l y m i n i m a l s e r v i c e s . ( 2 8 ) B e s i d e s b e i n g r e l a t i v e l y i n a c c e s s i b l e to the m a j o r i t y of i n h a b i t a n t s i n the d e v e l o p i n g c o u n t r i e s , the s e r v i c e s o f f e r e d by the c o n v e n t i o n a l m e d i c a l and h e a l t h c a r e system tended to be i n a p p r o p r i a t e to t h e i r i l l - h e a l t h c o n d i t i o n s . F o r i n s t a n c e , the most common d i s e a s e s i n the d e v e l o p i n g c o u n t r i e s d u r i n g t h i s p e r i o d were i n t e s t i n a l , p a r a s i t i c and i n f e c t i o u s d i a r r h e a l d i s o r d e r s . I n E g y p t , I r a n and V e n e z u a l a , f o r example, d u r i n g t h e 1960s the r e c o r d e d m o n t h l y i n c i d e n c e o f d i a r r h e a among p r e s choo 1 - aged c h i l d r e n was e s t i m a t e d to be between 407. and 507..(29) The d i s e a s e c a t e g o r y ' b a c i l l a r y d y s e n t r y and a m e o b i a s i s , e n t e r i t i s and o t h e r d i a r r h e a l d i s e a s e s ' was the l e a d i n g i d e n t i f i e d c a u s e o f d e a t h i n P a r a g u a y , G u a t e m a l a and E l S a l v a d o r i n the e a r l y 1970s. I n P a k i s t a n and B a n g l a d e s h f o r 1972, t h e c a t e g o r y ' a l l forms o f d y s e n t r y ' was the most f r e q u e n t l y n o t e d communicable d i s e a s e . D i a r r h e a l d i s e a s e s a l s o a c c o u n t e d f o r the m a j o r i t y of r e p o r t e d i n f a n t and c h i l d h o o d d e a t h s i n t h e A f r i c a r e g i o n f o r t h i s p e r i o d . ( 3 0 ) I t has been e s t i m a t e d t h t d i a r r h e a l d i s e a s e s have been the p r i m a r y c a u s e o f some 4,600,000 c h i l d h o o d d e a t h s p e r y e a r , r e s u l t i n g from about 600 m i l l i o n e p i s o d e s o f d i a r r h e a . (31 ) The s e c o n d and t h i r d major d i s e a s e g r o u p s i n t h e T h i r d W o r l d up to 1972 were a i r - b o r n e d i s e a s e s ( r e s p i r a t o r y a i l m e n t s ) and m a l n u t r i t i o n 19 r e s p e c t i v e l y . ( 3 2 ) T o g e t h e r , t h e s e t h r e e d i s e a s e c a t e g o r i e s a c c o u n t e d f o r t h e m a j o r i t y of r e p o r t e d d e a t h s i n the r u r a l and p e r i - u r b a n p o p u l a t i o n s i n the T h i r d W o r l d , and p a r t i c u l a r l y among c h i l d r e n below the age o f f i v e y e a r s . What becomes e v i d e n t upon even a c u r s o r y e x a m i n a t i o n o f t h e s e g l o b a l s t a t i s t i c s i s t h a t c o n v e n t i o n a l a p p r o a c h e s f o r the d e l i v e r y of h e a l t h s e r v i c e s i s i n a p p r o p r i a t e . The i n c i d e n c e o f the most p r e v a l e n t d i s e a s e s and h e a l t h s t a t u s i n g e n e r a l a r e c o n t i n g e n t upon f a c t o r s l a r g e l y u n r e l a t e d to the a c c e s s i b i l i t y of c u r a t i v e m e d i c a l s e r v i c e s , s u c h as i n a d e q u a t e n u t r i t i o n , c o n t a m i n a t e d w a t e r s u p p l i e s and s u b s t a n d a r d h o u s i n g . S t r a t e g i e s w h i c h f o c u s on i d e n t i f y i n g s o c i o - c u 1 t u r a 1, e n v i r o n m e n t a l , and economic d e t e r m i n a n t s of i l l - h e a l t h , and d e s i g n i n g h e a l t h c a r e programs w h i c h a d d r e s s t h e s e i s s u e s i s a more s u i t a b l e a p p r o a c h . I t was becoming q u i t e e v i d e n t as t h e 1970s p r o g r e s s e d t h a t c o n v e n t i o n a l m e d i c a l models were n o t p r o d u c i n g t h e e x p e c t e d i m p a c t s on h e a l t h . ( 3 3 ) The P r e s e n t S t a t u s o f t h e PHC Model Over the p a s t f i f t e e n y e a r s , governments have come to r e c o g n i z e t h a t c o n v e n t i o n a l h e a l t h c a r e d e l i v e r y systems a r e i n a d e q u a t e i n r e s p o n d i n g to t h e needs o f the m a j o r i t y of the p o p u l a t i o n i n the T h i r d W o r l d . F r u s t r a t i o n w i t h the ' t r i c k l e d o w n ' a p p r o a c h to d e v e l o p m e n t w h i c h was f a s h i o n a b l e i n 20 d e v e l o p m e n t t h e o r y and p o l i c y t h r o u g h o u t t h e l a t e 1950s and 1960s a p p e a r s to have i n f i l t r a t e d the h e a l t h f i e l d i n the e a r l y 1970s. R e c o g n i t i o n of the i n e q u a l i t y i n a c c e s s to h e a l t h c a r e f o r s i g n i f i c a n t p r o p o r t i o n s o f many d e v e l o p i n g c o u n t r y p o p u l a t i o n s , and t h e f a i l u r e and h i g h c o s t o f e x i s t i n g h e a l t h c a r e s y s t e m s , prompted i n t e r n a t i o n a l d o n o r s and T h i r d World governments to s e a r c h f o r a more p r a c t i c a l s t r a t e g y f o r d e l i v e r i n g h e a l t h c a r e . In 1971 the Wo r l d H e a l t h O r g a n i z a t i o n and UNICEF e s t a b l i s h e d the J o i n t Committee on A l t e r n a t i v e A p p r o a c h e s to M e e t i n g B a s i c H e a l t h Needs o f P o p u l a t i o n s i n D e v e l o p i n g C o u n t r i e s . I t i d e n t i f i e d s e v e r a l f a c t o r s as c o n t r i b u t i n g to the i n e f f e c t i v e n e s s of e x i s t i n g h e a l t h c a r e d e l i v e r y s y s t e m s . Among t h e s e were: the l a c k o f c l e a r n a t i o n a l h e a l t h p o l i c i e s and poor l i n k a g e of h e a l t h s y s tems w i t h o t h e r s e c t o r s of n a t i o n a l d e v e l o p m e n t ; the l a c k of c l e a r h e a l t h p r i o r i t i e s b a s e d on f a c t u a l d a t a ; t h e r e s i s t a n c e to change s o c i a l a s p e c t s o f h e a l t h p o l i c y ; the l a c k of a c t i v e community i n v o l v e m e n t i n p l a n n i n g and p r o v i d i n g h e a l t h c a r e ; and, the i n a p p r o p r i a t e t r a i n i n g o f h e a l t h p e r s o n n e l . The Committee a l s o c i t e d a l a c k o f r e a l i s t i c and p r o p e r h e a l t h p l a n n i n g as r e s p o n s i b l e f o r the i n e f f i c i e n c y o f e x i s t i n g h e a l t h c a r e d e l i v e r y s y s t e m s . Among the e l e m e n t s i d e n t i f i e d as c o n t r i b u t i n g to i n e f f e c t i v e h e a l t h c a r e d e l i v e r y were d e f i c i e n c i e s o f c o m m u n i c a t i o n and t r a n s p o r t a t i o n n e t w o r k s , l a c k o f b a s i c s a n i t a t i o n , i n s u f f i c i e n t use o f h e a l t h e d u c a t i o n , 21 and l a c k o f a d e q u a t e h e a l t h i n f o r m a t i o n . ( 3 4 ) The R e p o r t o f the Committee was p r e s e n t e d to the UNICEF/WHO J o i n t Committee on H e a l t h P o l i c y i n F e b r u a r y 1975, and s u b m i t t e d to t h e World H e a l t h Assembly i n May o f the same y e a r . B a s e d on an a n a l y s i s of s e v e r a l community-based h e a l t h c a r e d e l i v e r y s t r a t e g i e s , the Committee c o n c l u d e d t h a t : D e s p i t e t h e immense p r o b l e m s and the d a u n t i n g economic s i t u a t i o n , i t i s p o s s i b l e , u s i n g t h e r e s o u r c e s a v a i l a b l e , to meet c e r t a i n b a s i c h e a l t h needs of the p o p u l a t i o n s i n d e v e l o p i n g c o u n t r i e s , a c h i e v e b e t t e r h e a l t h c a r e c o v e r a g e , and improve l e v e l s o f h e a l t h . (35) T h i s c o u l d be a c h i e v e d , a c c o r d i n g to the f i n d i n g s o f the Committee, by a d o p t i n g a more h o l i s t i c a p p r o a c h . Such a s t r a t e g y r e q u i r e s a s h i f t i n emphasis to a c u r a t i v e - p r e v e n t i v e a p p r o a c h on a n a t i o n a l s c a l e , and an i n t e g r a t i o n o f h e a l t h s e r v i c e s w i t h a c t i v i t i e s i n o t h e r s e c t o r s . T h i s s t u d y was used to promote d i s c u s s i o n among h e a l t h e x p e r t s and to d e v e l o p b a c k g r o u n d m a t e r i a l s f o r the I n t e r n a t i o n a l C o n f e r e n c e on P r i m a r y H e a l t h C a r e , h e l d i n September, 1978, i n A l m a - A t a , USSR. The c e n t r a l f o c u s o f t h i s C o n f e r e n c e was the c o n c e r n f o r the s t a t u s o f h e a l t h c a r e and the s t a n d a r d o f l i v i n g i n the T h i r d W o r l d . R e p r e s e n t a t i v e s o f 134 governments and s e v e r a l UN a g e n c i e s and i n t e r n a t i o n a l and n a t i o n a l n o n - g o v e r n m e n t a l o r g a n i z a t i o n s a t t e n d e d . The C o n f e r e n c e s e r v e d to s t r e n g t h e n g e n e r a l s u p p o r t f o r a s t r a t e g y 22 to d e v e l o p a more s u i t a b l e and r e a l i s t i c a p p r o a c h to the d e l i v e r y o f h e a l t h c a r e s e r v i c e s w o r l d w i d e , p a r t i c u l a r l y i n the T h i r d W o r l d . The outcome of t h e C o n f e r e n c e was a unanimous agreement, l a b e l l e d the Al m a - A t a D e c l a r a t i o n , w h i c h a s s e r t s t h a t h e a l t h i s a u n i v e r s a l human r i g h t and t h a t governments s h o u l d p u r s u e p o l i c i e s to p r o v i d e a c c e s s i b l e , a f f o r d a b l e , and s o c i a l l y r e l e v a n t h e a l t h c a r e to a l l , w i t h a t a r g e t d a t e o f the y e a r 2000.(36) B e n n e t t (1979) has d i v i d e d the d e v e l o p i n g w o r l d i n t o f i v e camps w i t h r e s p e c t to t h e r e s p o n s e and a c c e p t a n c e o f the PHC a p p r o a c h as the b a s i s f o r n a t i o n a l h e a l t h p o l i c y : t h o s e c o u n t r i e s where p r i m a r y h e a l t h c a r e i s a n o r m a l p a r t of t h e i r program o f s o c i a l d e v e l o p m e n t ; t h o s e where i t i s a p a r t of the n a t i o n a l h e a l t h p l a n and has been r e c e n t l y added on an ad hoc b a s i s ; t h o s e where p r i m a r y h e a l t h c a r e i s s t i l l a t t h e p i l o t p r o j e c t s t a g e ; t h o s e where i t i s s a i d to be c o n t e m p l a t e d i n the n e x t few y e a r s ; and l a s t l y , t h o s e who have as y e t l i t t l e i n t e r e s t i n t h e c o n c e p t . ( 3 7 ) To d a t e , t h i r t y - n i n e c o u n t r i e s have s i g n e d r e g i o n a l c h a r t e r s p l e d g i n g t h e m s e l v e s to s t r i v e to a c h i e v e t h e g o a l ' H e a l t h f o r A l l by t h e Y e a r 2000'.(38) Most e f f o r t s to promote the p r i m a r y h e a l t h c a r e model have c o n c e n t r a t e d on p r o v i d i n g b a s i c s e r v i c e s a t the v i l l a g e l e v e l , u s i n g a l o c a l l a y community member t r a i n e d as a p a r a p r o f e s s i o n a 1 ( t h e ' v i l l a g e ' or 'community' h e a l t h w o r k e r ) . I n g e n e r a l , the v i l l a g e h e a l t h 23 w o r k e r s a r e r e c r u i t e d l o c a l l y , t r a i n e d a t a c e n t r a l i n s t i t u t e , and r e t u r n to t h e i r home v i l l a g e . The t r a i n i n g f o c u s e s on c e r t a i n c u r a t i v e and p r e v e n t i v e a c t i v i t i e s a p p r o p r i a t e to the needs o f the community he or she w i l l s e r v e . F o r example, 13,000 community h e a l t h w o r k e r s have been t r a i n e d and d e p l o y e d i n the f i e l d i n Burma w i t h i n f i v e y e a r s ; i n I n d i a , o v e r 180,000 community h e a l t h c a r e w o r k e r s have been t r a i n e d s i n c e 1977. S i m i l a r programs have been implemented i n I n d o n e s i a ( 1 5 0 , 0 0 0 ) , T h a i l a n d ( 8 5 , 0 0 0 ) , P e r u (10 ,000) and S w a z i l a n d (1 , 5 0 0 ) . ( 3 9 ) A r e c e n t s t u d y c o m m i s s i o n e d by the U n i t e d S t a t e s Agency f o r I n t e r n a t i o n a l D evelopment (USAID) i d e n t i f i e s twenty PHC p r o j e c t s f u n d e d by USAID w h i c h a r e n a t i o n a l i n s c o p e . ( 4 0 ) Many d e v e l o p i n g c o u n t r i e s a r e e x p e r i m e n t i n g w i t h the p r i m a r y h e a l t h c a r e model a t the p i l o t p r o j e c t s t a g e . These tend to be s m a l l - s c a l e and e x p e r i m e n t a l i n n a t u r e . ( 4 1 ) Most a r e s u p e r v i s e d o r a s s i s t e d by p r i v a t e n o n - g o v e r n m e n t a l and v o l u n t e e r o r g a n i z a t i o n s , r e l i g i o u s g r o u p s , p r i v a t e f o u n d a t i o n s , l o c a l p h y s i c i a n s and o t h e r i n d i v i d u a l s . T h e s e have l i t t l e r e l a t i o n s h i p to n a t i o n a l h e a l t h p o l i c y , and o f t e n s e r v e o n l y as l o c a l d e m o n s t r a t i o n p r o j e c t s to show the i m p a c t o f t h i s a p p r o a c h on t h e s t a n d a r d o f l i v i n g o f community members. Working w i t h l i m i t e d r e s o u r c e s , t h e y r e l y h e a v i l y on l o c a l l e a d e r s h i p , v o l u n t a r i s m , and i n d i g e n o u s r e s o u r c e s . ( 4 2 ) O n l y i n a few c o u n t r i e s , s u c h as C h i n a , T a n z a n i a , and Cuba, has t h e r e been a c o n c e r t e d e f f o r t to t r a n s f o r m the p r e v a i l i n g p h i l o s o p h y 24 of h e a l t h c a r e d e l i v e r y . T h e s e c o u n t r i e s p l a c e a h i g h p r i o r i t y on a c h i e v i n g a b r o a d d i s t r i b u t i o n of r u r a l h e a l t h s e r v i c e s t h r o u g h the dep l o y m e n t of a u x i l i a r y h e a l t h p e r s o n n e l and by d e c r e a s i n g dependence on u r b a n - b a s e d m e d i c a l f a c i l i t i e s and p e r s o n n e 1 . ( 4 3 ) I n v e s t m e n t s i n the d e v e l o p m e n t and i m p l e m e n t a t i o n of p r i m a r y h e a l t h c a r e p r o j e c t s has i n c r e a s e d s i g n i f i c a n t l y o v e r the p a s t f i v e y e a r s . ( 4 4 ) F o r example, the USAID s t u d y r e p o r t s t h a t t h e Agency a l l o c a t e d $84.6 m i l l i o n to p r i m a r y h e a l t h c a r e p r o j e c t s i n f i s c a l y e a r 1980, compared w i t h $27.5 m i l l i o n f o u r y e a r s p r e v i o u s . ( 4 5 ) F o r Canada, i t i s e s t i m a t e d t h a t a p p r o x i m a t e l y 47. o f t h a t n a t i o n ' s o v e r s e a s d e v e l o p m e n t a s s i s t a n c e b u d g e t i s a l l o c a t e d to s u p p o r t i n g the o p e r a t i o n of p r i m a r y h e a l t h c a r e p r o j e c t s i n the T h i r d W o r l d . ( 4 6 ) T h i s does n o t i n c l u d e f u n d s i n v e s t e d by C a n a d i a n n o n - g o v e r n m e n t a l o r g a n i z a t i o n s . W h i l e t h e r e i s no f i r m e s t i m a t e o f the v a l u e o f i n v e s t m e n t i n PHC p r o j e c t s i n the T h i r d W o r l d , t h e r e i s no doubt t h a t i t c o n s t i t u t e s a s i g n i f i c a n t p r o p o r t i o n o f h e a l t h - r e l a t e d a c t i v i t y program f u n d s . S ummary The p r i m a r y h e a l t h c a r e a p p r o a c h d e v e l o p e d l a r g e l y r e s p o n s e to a r e c o g n i t i o n t h a t c o n v e n t i o n a l a p p r o a c h e s i n to 25 d e l i v e r i n g h e a l t h c a r e were n o t b e n e f i t i n g l a r g e segments of the p o p u l a t i o n s i n d e v e l o p i n g c o u n t r i e s . T h i s s t r a t e g y i s i n c r e a s i n g l y b e i n g a d o p t e d as a means o f d e l i v e r i n g h e a l t h c a r e s e r v i c e s to u n d e r s e r v i c e d p o p u l a t i o n s i n many d e v e l o p i n g c o u n t r i e s . S i g n i f i c a n t l e v e l s of r e s o u r c e s a r e b e i n g i n v e s t e d i n the d e v e l o p m e n t and o p e r a t i o n o f PHC p r o j e c t s . T h i s s t r a t e g y i s a c t i v e l y promoted as a panacea to the l i m i t a t i o n s o f the c o n v e n t i o n a l h e a l t h c a r e d e l i v e r y systems w h i c h c h a r a c t e r i z e many of t h e s e c o u n t r i e s . But as t h e f o l l o w i n g c h a p t e r w i l l d e m o n s t r a t e , t h e r e i s l i t t l e e v i d e n c e to s u p p o r t the c l a i m s t h a t t h i s p a r t i c u l a r s t r a t e g y i s any more e f f e c t i v e t h an c o n v e n t i o n a l s t r a t e g i e s i n i m p r o v i n g h e a l t h s t a t u s . 26 REFERENCES 1. U n i t e d N a t i o n s , World H e a l t h O r g a n i z a t i o n and UNICEF, R e p o r t of the I n t e r n a t i o n a l C o n f e r e n c e on P r i m a r y H e a l t h C a r e . A l m a - A t a , USSR, (ICPHC/ALA/78.10), September 6-12, 1978. 2. U n i t e d N a t i o n s , W o r l d H e a l t h O r g a n i z a t i o n , R e p o r t of the  R e g i o n a l D i r e c t o r f o r the E u r o p e a n R e g i o n , I n t e r n a t i o n a l C o n f e r e n c e on P r i m a r y H e a l t h C a r e , A l m a - A t a , USSR, (ICPHC/ ALA/78.7), A p r i l 1, 1978, pp. 2-4; R . J . C o s t e l l a n o s , "Improved h e a l t h c o v e r a g e , c o o r d i n a t i o n of h e a l t h c a r e l e v e l s , and p r i m a r y h e a l t h c a r e " , Pan A m e r i c a n H e a l t h O r g a n i z a t i o n  B u l l e t i n , 1 1(3) (1977):202. 3. U n i t e d N a t i o n s , W o r l d H e a l t h O r g a n i z a t i o n , The t r a i n i n g  and u t i l i z a t i o n of a u x i l i a r y p e r s o n n e l f o r r u r a l h e a l t h teams  i n d e v e l o p i n g c o u n t r i e s (WHO T e c h n i c a l R e p o r t S e r i e s , no. 6 3 3 ) , 1979, p. 8. 4. V. D j u k a n o v i c and E.P. Mach, e d s . , A 1 1 e r n a t i v e  A p p r o a c h e s to M e e t i n g B a s i c H e a l t h Needs i n D e v e l o p i n g  C o u n t r i e s , (Geneva: W o r l d H e a l t h O r g a n i z a t i o n , 1975), pp. 98-99. 5. U n i t e d N a t i o n s , World H e a l t h O r g a n i z a t i o n and UNICEF, p . 50 . 6. G.M. F o s t e r , " M e d i c a l A n t h r o p o l o g y and I n t e r n a t i o n a l H e a l t h , " S o c. S c i . Med. 11(2) (197 7 ) : 5 3 2 . 7. W.D. F o s t e r , The C h u r c h M i s s i o n S o c i e t y and Modern  M e d i c i n e i n Uganda ( N e w h a v e n ( E n g . ) : N e w h a v e n P r e s s , 1 9 7 8) 8. A. D o r o z y n s k i , D o c t o r s and H e a l e r s ( Ottawa: I n t e r n a t i o n a l Development R e s e a r c h C e n t r e , 1975), p.35. 9. I b i d . 10. J . B r y a n t , H e a l t h and the D e v e l o p i n g World ( I t h a c a : C o r n e l l U n i v e r s i t y P r e s s , 1969); R. F r a n k e n b e r g and J . L e s s o n , "The S o c i o l o g y o f H e a l t h Dilemmas i n the P o s t - C o l o n i a 1 W o r l d : I n t e r m e d i a t e T e c h n o l o g y and M e d i c a l Care i n Zambia, Z a i r e and C h i n a , " i n S o c i o l o g y and Dev e l o p m e n t , e d s . E. DeKadt and G. W i l l i a m s ( L o n d o n : T a v i s t o c k P u b l i c a t i o n s , 1 974), pp. 255-278; M. K i n g , M e d i c a l C a r e i n D e v e l o p i n g C o u n t r i e s (London: O x f o r d U n i v e r i s t y P r e s s , 1966 ) . 27 11. B r y a n t , H e a l t h , p. 123; M. T u r s h e n , "The Impact of C o l o n i a l i s m on H e a l t h and H e a l t h S e r v i c e s i n T a n z a n i a , " I n t ' 1 J .  H e a l t h S e r v i c e s 7(1) ( 1 9 7 7 ) : 2 7 . 12. J . C . M c L i n t o c k , " R e s p o n s i b i l i t i e s of I n d u s t r y f o r H e a l t h of L o c a l P o p u l a t i o n s A b r o a d , " I n d u s t r y and T r o p i c a l H e a l t h V o l . 1 1 ( 1 9 5 4 ) , q u o t e d i n H. C l e a v e r , " M a l a r i a and P o l i t i c a l Economy o f P u b l i c H e a l t h , " I n t ' l J . H e a l t h S e r v i c e s 7(4) ( 19 7 7 ) : 5 6 6 . 13. T u r s h e n , " H e a l t h S e r v i c e s i n T a n z a n i a , " p. 2; M. T a u s s i g , " N u t r i t i o n , Development and F o r e i g n A i d : A Case Study of U . S . - D i r e c t e d H e a l t h C a r e i n a C o l o m b i a n P l a n t a t i o n Zone," I n t ' l J . H e a l t h S e r v i c e s 8 ( 1 ) ( 1 9 7 8 ) . 14. E . J . Brown, " P u b l i c H e a l t h i n I m p e r i a l i s m , " AJPH 66(9) (1976) : 897 . 15. J . S . Simmons, welcoming a d d r e s s to the C o n f e r e n c e on H e a l t h P r o b l e m s of I n d u s t r i e s O p e r a t i n g i n T r o p i c a l C o u n t r i e s , H a r v a r d S c h o o l o f P u b l i c H e a l t h , q u o t e d i n I n d u s t r y and T r o p i c a l  H e a l t h 1 ( 1 9 5 0 ) : 1 3 . 16. F o s t e r , " M e d i c a l A n t h r o p o l o g y , " p. 527. 17. G. Mo 1ina-Guzman, " T h i r d World E x p e r i e n c e s i n H e a l t h P l a n n i n g , " I n t ' l J . H e a l t h S e r v i c e s 9 ( 1 ) ( 1 9 7 9 ) : 1 4 0 . 18. W o r l d Bank, H e a l t h P r o b l e m s and P o l i c i e s i n the  D e v e l o p i n g C o u n t r i e s ( S t a f f W o r king P a p e r , no. 4 1 2 ) , ( W a s h i n g t o n : World Bank, A u g u s t 1980), p. 27. 19. P.A. Twumasi, " C o l o n i a l i s m and I n t e r n a t i o n a l H e a l t h : A S t u d y i n S o c i a l Changes i n Ghana," Soc. S c i . Med. 15B(2) (1981):149,150. 20. T u r s h e n , " H e a l t h S e r v i c e s i n T a n z a n i a , " p. 28. 21. Molina-Guzman, " H e a l t h P l a n n i n g , " p. 143; U n i t e d N a t i o n s , W o r l d H e a l t h O r g a n i z a t i o n , O f f i c i a l R e c o r d s of the  World H e a l t h O r g a n i z a t i o n ( 9 4 ) , q u o t e d i n The F i r s t R e g i o n a l  Seminar on R u r a l H e a l t h S e r v i c e s : F i n a l R e p o r t (WHO R e g i o n a l O f f i c e f o r the W e s t e r n P a c i f i c ) , J u l y 1963, p. 92. 22. D. M o r l e y , " P a e d i a t r i c P r i o r i t i e s i n E v o l v i n g Community Programs f o r D e v e l o p i n g C o u n t r i e s , " L a n c e t 7993 (1978):1013. 23. D j u k a n o v i c and Mach, A l t e r n a t i v e A p p r o a c h e s , p. 4. 28 24. I b i d . 25. J . S . M c K e n z i e , " P u t t i n g a P r i c e Tag on H e a l t h : A T e n - D o l l a r H e a l t h P l a n , " I n t ' l J . H e a l t h S e r v i c e s 16(1) (1974) :27. 26. D. M o r l e y , "The P a r t Time H e a l t h Worker i n the D e l i v e r y of H e a l t h C a r e , " J . Trop.Med. & H y g i e n e 81(11) (1978):214. 27. D j u k a n o v i c and Mach, A l t e r n a t i v e A p p r o a c h e s , p.18. 28. Pan A m e r i c a n H e a l t h O r g a n i z a t i o n , H e a l t h F o r A l l by the  Y e a r 2000 - S t r a t e g i e s , ( W a s h i n g t o n , D.C.: n.p., 1980), p. 75; U n i t e d N a t i o n s , W o r l d H e a l t h O r g a n i z a t i o n , F i n a l R e p o r t - F i r s t  R e g i o n a l Seminar on R u r a l H e a l t h S e r v i c e s (WHO R e g i o n a l O f f i c e f o r A f r i c a ) , J a n u a r y 1974, p. 14. 29. W.J. Van Z i j l , " S t u d i e s on D i a r r h e a l D i s e a s e s i n Seven C o u n t r i e s , " B u l l e t i n of the World H e a l t h O r g a n i z a t i o n 35 (1966) : 249-261 . 30. U n i t e d N a t i o n s , World H e a l t h O r g a n i z a t i o n , The F i f t h  R e p o r t on the World H e a l t h S i t u a t i o n 1969-1972 - P a r t I I Review  by C o u n t r y and T e r r i t o r y 1974. 31. Idem, Manual f o r the P l a n n i n g and E v a l u a t i o n of  D i a r r h e a l D i s e a s e C o n t r o l Programmes (WHO/CDD/SER/81.5), 1981. 32. Idem, The F i f t h R e p o r t . 33. Idem, " M e e t i n g B a s i c H e a l t h Needs i n D e v e l o p i n g C o u n t r i e s : A l t e r n a t i v e A p p r o a c h e s , " WHO C h r o n i c l e 29 (1975) : 168-187 . 34. A. B e n y o u s s e f and B. C h r i s t i a n , " H e a l t h Care i n D e v e l o p i n g C o u n t r i e s , " Soc. S c i . Med. 11, 1977, pp. 309-408; U n i t e d N a t i o n s , World H e a l t h O r g a n i z a t i o n , Towards a P h i l o s o p h y  of H e a l t h Work i n the A f r i c a n R e g i o n , ( A f r i c a n R e g i o n a l O f f i c e T e c h n i c a l P a p e r No. 1 ) , 1970, p. 15; The I n t e r n a t i o n a l Bank f o r R e c o n s t r u c t i o n and D evelopment, The A s s a u l t on World P o v e r t y :  P r o b l e m s of R u r a l Development, E d u c a t i o n , and H e a l t h ( B a l t i m o r e : The Johns H o p k i n s U n i v e r s i t y P r e s s , 1975), pp. 345-382. 35. U n i t e d N a t i o n s , World H e a l t h O r g a n i z a t i o n , M e e t i n g B a s i c  H e a l t h Needs p. 185. 29 36. R e s o l u t i o n s WHA 28.88 and WHA 29.19, a d o p t e d May 1975 and 1976 r e s p e c t i v e l y . F o r a c o m p l e t e s t a t e m e n t of the D e c l a r a t i o n , see A l ma-Ata 1978: P r i m a r y H e a l t h C a r e , R e p o r t of the I n t e r n a t i o n a l C o n f e r e n c e on P r i m a r y H e a l t h C a r e , A l m a - A t a , USSR, September 1978, (Geneva: W o r l d H e a l t h O r g a n i z a t i o n , 1978), pp. 2 - 6 . 37. F . J . B e n n e t t , " P r i m a r y H e a l t h C a r e and D e v e l o p i n g C o u n t r i e s , " Soc. S c i . Med. 13A ( 1979):506. 38. " P r i m a r y H e a l t h C a r e : A F i r s t A s s e s s m e n t , " P e o p l e 10(3) (1983) : 6 . 39 . I b i d . , pp. 8,9. 40. U.S. Agency f o r I n t e r n a t i o n a l Development (USAID), O f f i c e o f H e a l t h , q u o t e d i n M.B. P a r l a t o and M.N. F a v i n , P r i m a r y  H e a l t h C a r e P r o g r e s s and P r o b l e m s : An A n a l y s i s of 52  A I D - A s s i s t e d P r o j e c t s ( W a s h i n g t o n : A m e r i c a n P u b l i c H e a l t h A s s o c i a t i o n , A u g u s t 1 9 8 2 ) , pp. 24,26. 41. I b i d . ; " P r o g r e s s s i n c e A l m a - A t a : H e a l t h and E q u i t y , " P e o p l e 10(2) ( 1 9 8 3 ) : 3 - 5 . 42. P. Berman, " V i l l a g e H e a l t h W o r k ers: B a c k g r o u n d and I s s u e s f o r A n a l y s i s , " p a p e r f o r the R u r a l Development Committee, C e n t r e f o r I n t e r n a t i o n a l S t u d i e s , C o r n e l l U n i v e r s i t y , I t h a c a , N.Y., September 1978; R i f k i n , " I s s u e s and P e r s p e c t i v e s , " p. 129. 43. S. A k h t a r , H e a l t h C a r e i n the P e o p l e s R e p u b l i c of C h i n a ( O t t a w a : I n t e r n a t i o n a l Development R e s e a r c h C e n t r e ( I D R C ) , 1975); P. W i l e n s k i , The D e l i v e r y of H e a l t h S e r v i c e s i n the  P e o p l e ' s R e p u b l i c of C h i n a ( O t t a w a : IDRC, 1976); W.K. C h a g u l a and E. T a r i m o , " M e e t i n g B a s i c H e a l t h Needs i n T a n z a n i a , " i n H e a l t h by the P e o p l e , ed. K. N e w e l l (Geneva: W o r l d H e a l t h O r g a n i z a t i o n , 1975), pp. 145-168; V. D j u k a n o v i c e t a l . , " U n i t e d R e p u b l i c o f T a n z a n i a : An I n n o v a t i v e A p p r o a c h to the Development of H e a l t h S e r v i c e s " , i n D j u k a n o v i c and Mach, pp. 57-62; 0. G i s h , P l a n n i n g the H e a l t h S e c t o r : The T a n z a n i a n E x p e r i e n c e (London: Croom Helm, 1975 ) ; G.M. Van E t t e n , R u r a l H e a l t h i n T a n z a n i a ( A s s e n ( N e t h . ) : Van Gorcum & Co., 1976); R. D a n i e l s o n , "Cuban H e a l t h O r g a n i z a t i o n : H i s t o r y and D e v e l o p m e n t " (Ph.D. D i s s e r t a t i o n , U n i v e r s i t y o f M i c h i g a n , 1973); A.F.T. F e r n a n d e z , "The N a t i o n a l H e a l t h System i n Cuba," i n N e w e l l , pp. 13-29; V. N a v a r r o , " H e a l t h S e r v i c e s i n Cuba: An I n i t i a l A p p r a i s a l , " New  Eng. J . Med. 287(19) 19 7 2:954-959. 30 44. C h r i s t i a n M e d i c a l Commission, C o n t a c t , 75 ( 1 9 8 3 ) : 1 . 45. P a r l a t o and F a v i n , p. 13. 46. Canada, C a n a d i a n I n t e r n a t i o n a l Development Agency ( C I D A ) , " H e a l t h : CIDA's I n v o l v e m e n t " , A p r i l 1980 ( M i m e o g r a p h e d ) , p . 5 . 31 CHAPTER TWO EVALUATIVE RESEARCH IN PRIMARY HEALTH CARE I n t r o d u c t i o n D e c i s i o n s c o n c e r n i n g the f u t u r e d i r e c t i o n of h e a l t h c a r e d e l i v e r y s y s tems i n d e v e l o p i n g c o u n t r i e s r e q u i r e a c c u r a t e , v a l i d and r e l e v a n t i n f o r m a t i o n . H e a l t h s e r v i c e s r e s e a r c h , i n c l u d i n g t h e e v a l u a t i o n of e x i s t i n g p r i m a r y h e a l t h c a r e p rograms, i s t h e p r i m a r y means of o b t a i n i n g t h i s i n f o r m a t i o n . I n o r d e r to e n s u r e t h a t e v a l u a t i o n r e s u l t s meet t h e s e i n f o r m a t i o n c r i t e r i a , - c a r e must be e x e r c i s e d t h a t the e v a l u a t i v e r e s e a r c h d e s i g n s and methods employed a r e a p p r o p r i a t e to the needs and o b j e c t i v e of the e x e r c i s e , t a k i n g i n t o c o n s i d e r a t i o n t h e r e a l i t i e s of the f i e l d s e t t i n g i n which the program e x i s t s . The p u r p o s e o f t h i s c h a p t e r i s to i d e n t i f y the p r i n c i p l e w eaknesses i n h e r e n t i n e v a l u a t i v e r e s e a r c h on d e v e l o p i n g c o u n t r y - b a s e d p r i m a r y h e a l t h c a r e programs c o n d u c t e d to d a t e . The c h a p t e r b e g i n s by d e f i n i n g e v a l u a t i v e r e s e a r c h , i t s r a t i o n a l e , and the need f o r e v a l u a t i v e r e s e a r c h i n p r i m a r y h e a l t h c a r e . The c h a p t e r t h e n p r o v i d e s a g e n e r a l a n a l y s i s o f e v a l u a t i v e r e s e a r c h on p r i m a r y h e a l t h c a r e p r o j e c t s i n the d e v e l o p i n g c o u n t r i e s , p r i n c i p a l l y f u n d e d and e x e c u t e d by e x p a t r i a t e o r g a n i z a t i o n s and i n d i v i d u a l s . I n o r d e r to p r o v i d e a c o m p a r i s o n , f i v e p r i m a r y h e a l t h c a r e program e v a l u a t i o n s t u d i e s 32 c o n d u c t e d by i n d i g e n o u s r e s e a r c h e r s a r e t h e n p r e s e n t e d and a n a l y z e d . The c h a p t e r w i l l d e m o n s t r a t e t h a t the r e s u l t s o f e v a l u a t i v e r e s e a r c h on p r i m a r y h e a l t h c a r e p r o j e c t s i n d e v e l o p i n g c o u n t r i e s t e n d to be l i m i t e d i n t h e r v a l i d i t y and u t i l i t y due to s e v e r a l f a c t o r s , r e l a t e d p r i m a r i l y to e v a l u a t i v e r e s e a r c h d e s i g n f a u l t s , and weaknesses i n e v a l u a t i v e methods and t e c h n i q u e s . E v a l u a t i v e R e s e a r c h D e f i n e d The d e f i n i t i o n o f e v a l u a t i o n (and so, t o o , e v a l u a t i v e r e s e a r c h ) d i f f e r s among e x p e r t s i n the f i e l d . ' F o r some, e v a l u a t i o n i s the measure o f the d e g r e e o f s u c c e s s or the e f f e c t i v e n e s s of a program i n a c h i e v i n g i t s p r e d e t e r m i n e d o b j e c t i v e s . F o r example, e v a l u a t i o n has been d e f i n e d as an a t t e m p t " t o answer s t a t i s t i c a l l y what p r o p o r t i o n of the g o a l has been r e a c h e d and how much of t h i s can be c r e d i t e d to the p r o g r a m " . ( 1 ) L i k e w i s e , i t has been d e f i n e d as " t h e p r o c e s s of d e t e r m i n i n g the . . . amount of s u c c e s s i n a c h i e v i n g a p r e d e t e r m i n e d o b j e c t i v e " . ( 2 ) E v a l u a t i o n has a l s o been d e f i n e d i n b r o a d e r t e r m s , to i n c l u d e an a s s e s s m e n t o f program e f f i c i e n c y , a d e q u a c y , e f f o r t , p e r f o r m a n c e , and i m p a c t . ( 3 ) Program e f f i c i e n c y i s the c o s t i n r e s o u r c e s o f a t t a i n i n g a d e s i r e d o b j e c t i v e i n the s p e c i f i e d time p e r i o d w i t h t h i s p a r t i c u l a r program. Adequacy r e f e r s to the c a p a c i t y of a 33 program to meet t o t a l n e e d s . Program e f f o r t r e l a t e s to the t i m e , e n e r g y and r e s o u r c e s expended to c a r r y o u t program a c t i v i t i e s . The p e r f o r m a n c e o f a program i s an e x a m i n a t i o n o f i t s c o n s t i t u e n t components to d e t e r m i n e whether t h e s e a c t i v i t i e s o c c u r e d as p l a n n e d . An i d e n t i f i c a t i o n of the n a t u r e and e f f e c t of the c o n s e q u e n c e s ( b o t h the p o s i t i v e and n e g a t i v e s i d e - e f f e c t s ) o f the program c o n s t i t u t e s an e v a l u a t i o n of i m p a c t . P e r h a p s Weiss (1972) p r o v i d e s the most s u c c i n c t y e t c o m p r e h e n s i v e d e f i n i t i o n of e v a l u a t i o n : An e l a s t i c word t h a t s t r e t c h e s to c o v e r judgements of many k i n d s . . . ( i t ) i n v e s t i g a t e s the c o n s e q u e n c e s of dynamic programmes t h a t a t t e m p t to a l t e r key v a r i a b l e s i n p e o p l e s ' l i v e s . (4) The aim of the e x e r c i s e i s to p r o v i d e program p o l i c y makers, p l a n n e r s and managers w i t h v a l i d and r e l i a b l e d a t a to i mprove, e x t e n d , o r i n some way m o d i f y ( i n c l u d i n g the p o s s i b i l i t y o f d o i n g away w i t h ) the p r o g r a m . ( 5 ) E v a l u a t i v e r e s e a r c h , i n t u r n , i s s i m p l y the s e t of s y s t e m a t i c p r o c e d u r e s r e q u i r e d f o r c o l l e c t i n g and a n a l y s i n g d a t a w h i c h i n c r e a s e the p o s s i b i l i t y f o r ' p r o v i n g ' r a t h e r t h a n ' a s s e r t i n g ' the worth of some s o c i a l a c t i v i t y . ( 6 ) I n o r d e r to c o n d u c t e v a l u a t i v e r e s e a r c h , a s e t of t o o l s and methods have e v o l v e d , w h i c h i n t h e m s e l v e s have become a s u b j e c t of s t u d y and d e b a t e . 34 The R a t i o n a l e f o r E v a l u a t i v e R e s e a r c h The use o f e v a l u a t i v e r e s e a r c h to a s s e s s t h e p e r f o r m a n c e of s o c i a l programs i s a f a i r l y r e c e n t phenomenon. Much of the i n t e r e s t i n t h e f i e l d and most of the c u r r e n t e v a l u a t i o n t e c h n i q u e s and models were d e v e l o p e d i n the U n i t e d S t a t e s . S o c i a l w e l f a r e programs b o r n e o f P r e s i d e n t R o o s e v e l t ' s New D e a l and p o s t - W o r l d War I I U.S. government a d m i n i s t r a t i o n p o l i c i e s s p u r r e d the d e v e l o p m e n t o f models and t e c h n i q u e s f o r e v a l u a t i n g t h e i r p e r f o r m a n c e . ( 7 ) A marked growth i n b o t h c a s e s t u d i e s of s o c i a l w e l f a r e programs and the d e v e l o p m e n t and r e v i e w of e v a l u a t i o n m e t h o d o l o g i e s o c c u r r e d t h r o u g h o u t th e 1950s and e a r l y 1 9 6 0 s.(8) The growth o f t h e economy, and the number and c o m p l e x i t y o f s o c i a l w e l f a r e programs implemented i n the U n i t e d S t a t e s d u r i n g the 1960s, p a r t i c u l a r l y d u r i n g the Kennedy a d m i n i s t r a t i o n , was u n p r e c e d e n t e d . B i l l i o n s of d o l l a r s were i n v e s t e d i n economic d e v e l o p m e n t programs such as i n d u s t r i a l i n c e n t i v e p r o g r a m s , space e x p l o r a t i o n ( p a r t i c u l a r l y the d e v e l o p m e n t o f m i l i t a r y t e c h n o l o g i e s ) , and i n s o c i a l w e l f a r e programs s u c h as e d u c a t i o n a l programs f o r the u n d e r p r i v e 1 e g e d , u r b a n r e n e w a l , f o r e i g n a s s i s t a n c e a i d and e n v i r o n m e n t a l p r ograms. The u n d e r l y i n g a s s u m p t i o n o f t h e t h e s e programs was t h a t w i t h enough d o l l a r s and the r i g h t e f f o r t s , s u s t a i n e d 35 economic growth would occur and the s o c i a l i l l s of the country would be e l i m i n a t e d . But such a r a p i d and unprecedented s c a l e of investment of p u b l i c funds , e s p e c i a l l y i n t o s o c i a l we l fare programs, f o r which the r e t u r n on investment was u n c e r t a i n at b e s t , c o u l d not occur i n d e f i n i t e l y wi thout some c o n t r o l and a c c o u n t a b i 1 i t y . The t a k e - o f f i n e v a l u a t i v e r e s e a r c h o c c u r r e d i n response to s e v e r a l f a c t o r s : 1. a demand f o r g r e a t e r a c c o u n t a b i l i t y f o r funds i n v e s t e d i n pub 1 i c l y - f u n d e d programs; 2. an i n c r e a s i n g s c a r c i t y of r e s o u r c e s , both f i n a n c i a l and human, to m a i n t a i n such programs; 3. the development of new t e c h n o l o g i e s and approaches to s e r v i c e d e l i v e r y of s o c i a l programs which were c h a l l e n g i n g t r a d i t i o n a l approaches; 4. i n c r e a s i n g i n t e r e s t among s o c i a l s c i e n t i s t s i n the s o c i a l r e l e v a n c e of c o n d u c t i n g r e s e a r c h i n s o c i a l s e t t i n g s as opposed to s i m u l a t e d exper iments ; 5. a s h i f t i n employment o p p o r t u n i t i e s f o r academics as a r e s u l t of fund ing c o n s t r a i n t s , f o r c i n g s o c i a l s c i e n t i s t s to become e v a l u a t o r s i n search of r e s e a r c h fund s ; and , 6. a r e c o g n i t i o n that s o c i a l c o n d i t i o n s and economic w e l l - b e i n g were not n e c e s s a r i l y improv ing f o r the maj o r i t y . (9) 36 As the demand f o r program e v a l u a t i o n s i n c r e a s e d , so too d i d the need to l e g i t i m i z e the f i e l d of e v a l u a t i v e r e s e a r c h . T h i s p e r s p e c t i v e r e s u l t e d i n the p r o m o t i o n o f s c i e n t i f i c a 1 l y - b a s e d p a r a d i g m s . A c c o r d i n g to t h e o r y , such models s h o u l d p r o v i d e v a l i d and c o n c l u s i v e e v i d e n c e . To t h i s end, the e x p e r i m e n t a l p a r a d i g m s came to dominate e v a l u a t i v e r e s e a r c h . E v a l u a t i v e R e s e a r c h i n P r i m a r y H e a l t h C are S i g n i f i c a n t improvements i n the h e a l t h s t a t u s of the i n t e n d e d b e n e f i c i a r i e s c an o n l y o c c u r a f t e r p r i m a r y h e a l t h c a r e programs s u c c e e d i n c o n v i n c i n g the t a r g e t p o p u l a t i o n of the b e n e f i t s a c c r u i n g from the u t i l i z a t i o n of t h e s e s e r v i c e s , and from t h e i r a c t i v e p a r t i c i p a t i o n i n community-based and f o c u s e d a c t i v i t i e s , s u c h as e d u c a t i o n a l and s e l f - h e l p p r o g r a m s , which seek to m o d i f y e x i s t i n g knowledge, a t t i t u d e s and p r a c t i c e s w i t h r e s p e c t to h e a 1 t h - r e l a t e d b e h a v i o u r . The b u r d e n i s t h e r e f o r e on the p r i m a r y h e a l t h c a r e program to d e m o n s t r a t e i t s c a p a c i t y to overcome c o n s t r a i n t s w h i c h l i m i t the a b i l i t y of community members to p a r t i c i p a t e a c t i v e l y i n a l l a s p e c t s of program d e v e l o p m e n t and i m p l e m e n t a t i o n . A c c o r d i n g to the R e p o r t o f the I n t e r n a t i o n a l C o n f e r e n c e on P r i m a r y H e a l t h C a r e , e v a l u a t i o n i s e s s e n t i a l i n o r d e r to e n s u r e t h a t PHC programs f u n c t i o n c o r r e c t l y . P a r t i c i p a n t government r e p r e s e n t a t i v e s were e n c o u r a g e d to use e v a l u a t i o n r e s u l t s to 37 improve program e f f e c t i v e n e s s and e f f i c i e n c y . The C o n f e r e n c e r e p o r t a l s o s u g g e s t e d t h a t PHC programs be e v a l u a t e d w i t h r e s p e c t to f i v e c r i t e r i a : r e l e v a n c e , p r o g r e s s , e f f i c i e n c y , e f f e c t i v e n e s s and i m p a c t . ( 1 0 ) However, w h i l e p a y i n g l i p s e r v i c e to t h e i m p o r t a n c e o f e v a l u a t i o n ,as an e s s e n t i a l t o o l f o r p o l i c y f o r m u l a t i o n and program management, the R e p o r t p r o v i d e d no g u i d a n c e as to the 'how t o ' o f d e s i g n i n g or c a r r y i n g o u t a v a l i d and u s e f u l e v a l u a t i o n e x e r c i s e . G e n e r a l A n a l y s i s o f E v a l u a t i v e R e s e a r c h o f PHC Programs To d a t e , o v e r 300 d e v e l o p i n g c o u n t r y - b a s e d p r i m a r y h e a l t h c a r e p r o j e c t s have been r e p o r t e d on i n a v a r i e t y of pub 1 i c a t i o n s . ( 1 1 ) T h e s e range i n s c a l e from s m a l l , r e g i o n a l e f f o r t s f o c u s i n g on s p e c i f i c h e a 1 t h - r e l a t e d t a s k s to n a t i o n a l programs c o v e r i n g a wide r a n g e of h e a 1 t h - r e l a t e d a c t i v i t i e s . However, w h i l e most p r o v i d e some s t a t i s t i c s of p r o g r e s s and e f f o r t , few p r o v i d e d e t a i l s o f p r o j e c t i m p a c t . To d a t e , o n l y two s t u d i e s have been p u b l i s h e d which p r o v i d e an o v e r v i e w of the r e s u l t s o f d e v e l o p i n g c o u n t r y - b a s e d PHC p r o j e c t s . ( 1 2 ) The r e v i e w w h i c h f o l l o w s i s based on the f i n d i n g s of t h e s e r e p o r t s . The p u r p o s e o f t h i s o v e r v i e w i s to i d e n t i f y some of the major f a c t o r s w h i c h l i m i t the v a l i d i t y and u t i l i t y of e v a l u a t i v e s t u d i e s on p r i m a r y h e a l t h c a r e p r o j e c t s i n d e v e l o p i n g c o u n t r i e s . W h i l e by no means a d e f i n i t i v e l i s t , t h i s 38 a n a l y s i s a t t e m p t s to i d e n t i f y some of the i m p o r t a n t weaknesses i n e v a l u a t i v e r e s e a r c h d e s i g n and t e c h n i q u e s w h i c h l i m i t t h e i r v a l i d i t y and u t i l i t y . One f a c t o r i d e n t i f i e d as a d v e s e l y a f f e c t i n g the u s e f u l n e s s and v a l i d i t y PHC e v l u a t i o n s i s r e l a t e d to who d e f i n e s the g o a l s , o b j e c t i v e s , and the i n d i c a t o r s of p r o g r e s s f o r t h e s e e x e r c i s e s . T h e r e i s e v i d e n c e to s u g g e s t t h a t some e v a l u a t i o n s a r e c o m m i s s i o n e d i n o r d e r to a c c o u n t f o r f u n d s expended on i n t e r n a t i o n a l a s s i s t a n c e p r o j e c t s . ( 1 3 ) The g o a l s of the e x e r c i s e an the c r i t e r i a s e l e c t e d to measure a t t a i n m e n t of the g o a l s t e n d to be i n f l u e n c e d by the needs o f the f u n d i n g a g e n c i e s and o r g a n i z a t i o n s w i t h l i t t l e r e g a r d f o r l o c a l needs and e x p e c t a t i o n s . ( 1 4 ) I n e s s e n c e , the e v a l u a t i o n s t u d i e s become ends i n t h e m s e l v e s . I n s t e a d of d e t e r m i n i n g how p r o j e c t a c t i v i t i e s i mprove the h e a l t h of the r e c i p i e n t p o p u l a t i o n , they tend to r e p o r t t h e number o f f a c i l i t i e s c o n s t r u c t e d , and p e r s o n n e l t r a i n e d and d e p l o y e d . I t has been r e p o r t e d t h a t some h o s t c o u n t r y o f f i c i a l s d i s c o u n t the i m p o r t a n c e o f the e v a l u a t i o n s t u d i e s b e c a u s e t h e y b e l i e v e e v a l u a t i o n i s p r i m a r i l y the i n t e r e s t of t h e o u t s i d e f u n d i n g a g e n c y , and t h a t the e x e r c i s e i s e x c e s s i v e i f n o t i r r e l e v a n t to h e a l t h c a r e s e r v i c e s d e 1 i v e r y . ( 1 5 ) C o n s e q u e n t l y , the u t i l i t y of the e v a l u a t i v e s t u d i e s to the h o s t government or agency r e s p o n s i b l e f o r the a d m i n i s t r a t i o n and management of the p r i m a r y h e a l t h c a r e program i s e x t r e m e l y l i m i t e d . 39 A n o t h e r l i m i t i n g f a c t o r i s t h e l a c k of d e f i n e d and e x p l i c i t o b j e c t i v e s and g o a l s f o r the program. I t i s o f t e n o b s e r v e d t h a t l i t t l e i s known from the e v a l u a t i o n s o f many h e a l t h c a r e programs b e c a u s e the r i g h t q u e s t i o n s a r e f r e q u e n t l y n o t asked i n the f i r s t p l a c e . ( 1 6 ) I n the c a s e o f new programs b e i n g i n i t i a t e d , g e n e r a l , n o n s p e c i f i c o b j e c t i v e s t e n d to be s t a t e d . T h i s p r o b l e m i s i l l u s t r a t e d i n a r e v i e w o f e v a l u a t i o n s t u d i e s of 180 p r i m a r y h e a l t h c a r e p r o j e c t s i n d e v e l o p i n g c o u n t r i e s c o m m i s s i o n e d by the A m e r i c a n P u b l i c H e a l t h A s s o c i a t i o n ( 1 9 7 7 ) : [A] l a r g e p r o p o r t i o n of p r o j e c t s a p p e a r to be engaged i n a v a r i e t y of o b j e c t i v e s w h i c h a r e l a r g e l y i m p l i c i t and u n a r t i c u l a t e d . C o n s e q u e n t l y , a c h i e v e m e n t t e n d s to be measured, n o t i n terms of movement i n an i n t e n t i o n a l d i r e c t i o n towards a s t a t e d end, b u t r a t h e r i n terms o f e f f o r t and e n e r g y expended. (17) The r e v i e w a l s o f o u n d t h a t t h e e x t e n t o f change e x p e c t e d i n i n d i c a t o r s and the time p e r i o d o v e r w h i c h s u c h changes a r e e x p e c t e d to o c c u r a r e r a r e l y s p e c i f i e d . A n o t h e r f a c t o r w h i c h has been f o u n d to l i m i t the v a l u e o f e v a l u a t i o n s t u d i e s i s t h e l a c k o f i n d i c a t o r s w h i c h measure the e f f e c t o f the p r o j e c t a c t i v i t i e s on the h e a l t h and w e l l - b e i n g of the i n t e n d e d t a r g e t p o p u l a t i o n . The U n i t e d S t a t e s Agency f o r I n t e r n a t i o n a l D evelopment (USAID) s t u d y o f 52 U S A I D - s u p p o r t e d p r i m a r y h e a l t h c a r e p r o j e c t s f o u n d t h a t t h e e v a l u a t i o n documents r e v i e w e d c o n t a i n e d few d a t a t h a t i n d i c a t e the impact of the p r o j e c t a c t i v i t i e s on h e a l t h s t a t u s , and o n l y l i m i t e d i n f o r m a t i o n on o t h e r meaures o f p r o g r e s s . Such d a t a , i t was 40 f o u n d , a r e e x p e n s i v e , d i f f i c u l t , and t i m e - c o n s u m i n g to p r o d u c e . And as the time e l a p s e d between the i n i t i a t i o n of p r o j e c t a c t i v i t i e s and the c o n c l u s i o n o f the e v a l u a t i o n was r e l a t i v e l y s h o r t , i t was d i f f i c u l t to d e v i s e s h o r t - t e r m measures s e n s i t i v e enough to d e t e c t changes i n the h e a l t h s t a t u s of the p o p u l a t i o n . I t a l s o f o u n d t h a t b e c a u s e o f f a c t o r s n o t a c c o u n t e d f o r i n t h e s t u d y , measured changes c a n n o t a l w a y s be d e f i n i t e l y a t t r i b u t e d to p r o j e c t i n t e r v e n t i o n s . As a r e s u l t , changes i n s e r v i c e use o f t e n s e r v e as i n d i r e c t o r i n t e r m e d i a t e and sometimes s u r r o g a t e measures of i m p a c t . ( 1 8 ) As M i l i o (1977) s t a t e s : The most common and t r a d i t i o n a l l y p r e v a l e n t c r i t e r i a a r e the f a m i l i a r i n p u t - o u t p u t m e a s u r e s , s i m p l y q u a n t i t i e s o f p e r s o n n e l , p a t i e n t s , v i s i t s , a d m i s s i o n s , i m m u n i z a t i o n s , c o n t r a c e p t i v e s , e t c . Th e s e g r o s s q u a n t i t i e s , as w e l l as p r o c e s s m e a s u r e s , c a n n o t of c o u r s e i n d i c a t e h e a l t h outcomes. (19) In s h o r t , p r i m a r y h e a l t h c a r e p r o j e c t s do n o t tend to be e v a l u a t e d f o r t h e i r i m p a c t on the h e a l t h and w e l l - b e i n g of the t a r g e t p o p u l a t i o n s . A t b e s t , t h e i r i m p a c t i s measured under the p r e m i s e t h a t t h e a v a i l a b i l i t y o f h e a l t h p e r s o n n e l and s e r v i c e s , and the d e l i v e r y o f t h e s e s e r v i c e s w i l l n e c e s s a r i l y r e s u l t i n improved h e a l t h . W h i l e s u c h d a t a may be e a s i e r to c o l l e c t , they a r e n o t n e c e s s a r i l y s u i t a b l e or r e l i a b l e s u r r o g a t e s of i m p a c t . A n o t h e r l i m i t a t i o n of t h e s e e v a l u a t i o n s t u d i e s i s t h a t many have f o c u s e d on a s i n g l e a r e a o f p r i m a r y h e a l t h c a r e such as i m m u n i z a t i o n a c t i v i t i e s , n u t r i t i o n , f a m i l y p l a n n i n g or 41 s a n i t a t i o n . ( 2 0 ) B u t the p r i m a r y h e a l t h c a r e model i s an i n t e g r a t e d a p p r o a c h . As s u c h , e v a l u a t i o n s s h o u l d r e v i e w a l l a c t i v i t i e s c a r r i e d o u t under the program, and n o t a s s e s s s u c c e s s o r f a i l u r e on f r a g m e n t e d components. I n the same v e i n , i t i s e q u a l l y i m p o r t a n t to know how each o f the v a r i o u s PHC components has a f f e c t e d the h e a l t h s t a t u s o f the t a r g e t p o p u l a t i o n . B e c a u s e t h e s e PHC a c t i v i t i e s o f t e n t a k e p l a c e s i m u l t a n e o u s l y , and i n some programs a t the same time as c e r t a i n v e r t i c a l p rograms, i t i s r a r e l y p o s s i b l e to i d e n t i f y the e f f e c t s o f i n t r o d u c i n g p e r i p h e r a l , community-based s e r v i c e s . Few p r o j e c t e v a l u a t i o n s have been d e s i g n e d to p r o v i d e f e e d b a c k u s e f u l i n e v a l u a t i n g each PHC component s e p a r a t e l y . ( 2 1 ) A d d i t i o n a l l y , the i n f l u e n c e of changes i n o t h e r s e c t o r s on t h e h e a l t h and w e l l - b e i n g o f t h e s t u d y p o p u l a t i o n i s r a r e l y a c c o u n t e d f o r . E v a l u a t i o n s have a l s o been f o u n d to be i n a d e q u a t e due to the l a c k o f p e r s o n n e l t r a i n e d i n e v a l u a t i o n d e s i g n and t e c h n i q u e s , t h e l a c k o f r e l i a b l e s t a t i s t i c s and e s s e n t i a l r e c o r d s , and i n s u f f i c i e n t i n t e r e s t on the p a r t of h e a l t h s t a f f to c o n d u c t e v a l u a t i o n s . ( 2 2 ) V e r y o f t e n t h e e v a l u a t i o n s were c o n d u c t e d o v e r and above the n o r m a l w o r k l o a d of the i n d i v i d u a l s c o n c e r n e d . F o r example, many s t u d i e s u t i l i z e t h o s e p r o v i d i n g t h e s e r v i c e s to c o l l e c t d a t a . U n f o r t u n a t e l y , s e r v i c e p r o v i d e r s a r e o f t e n i l l - t r a i n e d i n d a t a c o l l e c t i o n t e c h n i q u e s , and ten d to have l i t t l e time o r i n t e r e s t i n t h i s t a s k . ( 2 3 ) The l a c k of 42 a d e q u a t e l y t r a i n e d p e r s o n n e l i s a l s o r e s p o n s i b l e f o r a s i g n i f i c a n t amount of u n r e l i a b l e d a t a b e i n g c o l l e c t e d . F o r example, the USAID s t u d y n o t e s t h a t s e v e r a l p r o j e c t s had to abandon p l a n s to c o n d u c t f o l l o w - u p f i e l d s u r v e y s b e c a u s e the b a s e l i n e d a t a were of poor q u a l i t y , or the d a t a c o l l e c t i o n t e c h n i q u e s were too c o m p l e x . ( 2 4 ) The l a c k o f p e r s o n n e l t r a i n e d i n d a t a c o l l e c t i o n t e c h n i q u e s a l s o a c c o u n t s f o r the r e s t r i c t e d range of d a t a c o l l e c t e d . S t a n d a r d s u r v e y s used i n p r i m a r y h e a l t h c a r e e v a l u a t i o n s f r e q u e n t l y t e n d to c o l l e c t s u p e r f i c i a l d a t a , b u t r a r e l y probe the b e l i e f s and a t t i t u d e s w h i c h a c c o u n t f o r p r a c t i c e s . ( 2 5 ) W h i l e p r o j e c t s c o l l e c t i n v e n t o r y d a t a of s t r u c t u r e s and p e r s o n n e l , the number o f p a t i e n t v i s i t s , or the u t i l i z a t i o n of s t a f f , f a c i l i t i e s or e q uipment, t h e y f a i l to i d e n t i f y and e x p l a i n changes i n b e h a v i o u r o f the s t u d y p o p u l a t i o n as a r e s u l t of the i n c r e a s e d a v a i l a b i l i t y and a c c e s s i b i l i t y of h e a l t h c a r e s e r v i c e s . The c o l l e c t i o n of s u c h i n f o r m a t i o n r e q u i r e s a s e n s i t i v e and c u l t u r a l l y a p p r o p r i a t e t e c h n i q u e . G i v e n t h e s h o r t time a l l o w a n c e s f o r most e v a l u a t i o n s , r e s t r i c t e d b u d g e t s , and l a c k of t r a i n e d manpower, su c h i n f o r m a t i o n i s r a r e l y c o l l e c t e d . Many p r o j e c t s e x p e r i e n c e d d i f f i c u l t i e s i n e s t a b l i s h i n g f u n c t i o n a l d a t a c o l l e c t i o n and a n a l y s i s s y s t e m s . Many of t h e s e systems were too c o m p l i c a t e d f o r t h e c a p a b i l i t i e s of t h o s e r e s p o n s i b l e f o r the d a t a c o l l e c t i o n and a n a l y s i s t a s k s . O f t e n , 43 m o r e d a t a t h a n was n e c e s s a r y was c o l l e c t e d , o v e r w h e l m i n g t h o s e r e s p o n s i b l e f o r d a t a a n a l y s i s . T h e U S A I D - s p o n s o r e d s t u d y c i t e s a p r o j e c t i n G u a t e m a l a w h e r e i t was r e p o r t e d t h a t t h e d a t a s y s t e m r e q u i r e d t h e h e a l t h p r o m o t e r t o c o m p l e t e m o r e t h a n 2 , 200 f i l e r e p o r t p a g e s p e r m o n t h . T h e p r o j e c t s u p e r v i s o r was t h e n e x p e c t e d t o s u m m a r i z e 20 r e p o r t s d u r i n g t h e same p e r i o d . ( 2 6 ) T h e A P H A - c o m m i s s i o n e d s t u d y f o u n d t h a t w h i l e l a r g e n u m b e r s o f p r o j e c t s r e p o r t c o l l e c t i n g a v a r i e t y o f d a t a , r e l a t i v e l y f e w u s e d t h e d a t a a s f e e d b a c k t o i m p r o v e t h e o p e r a t i o n o f t h e p r o g r a m . S u c h d a t a w e r e o f t e n u s e d i n s t e a d t o k e e p f u n d i n g o f f i c i a l s i n f o r m e d o f p r o j e c t a c t i v i t i e s . ( 2 7 ) T h e s e e v a l u a t i o n s a l s o t e n d e d t o c o l l e c t o n l y q u a n t i t a t i v e d a t a , o f t e n o v e r l o o k i n g p o t e n t i a l l y s i g n i f i c a n t q u a l i t a t i v e i n f o r m a t i o n . ' O b j e c t i v e ' s u r v e y d a t a b a s e d o n c o u n t i n g may n o t be a p p r o p r i a t e t o i d e n t i f y a n d e x p l a i n i d e n t i f i e d b e h a v i o u r p a t t e r n s . A s w e l l , some a c t i v i t i e s , s u c h a s e a t i n g a n d d e f e c a t i o n h a b i t s , a r e n o t a l w a y s o b s e r v a b l e o r q u a n t i f i a b 1 e . ( 2 8 ) T h e s c a l e o f t h e p r o j e c t a n d t h e p e o p l e i n v o l v e d i n i t s m a n a g e m e n t a n d a d m i n i s t r a t i o n a r e a l s o c r i t i c a l e l e m e n t s i n d e t e r m i n i n g t h e s u c c e s s o r f a i l u r e o f a p r o j e c t . S m a l l - s c a l e p r o j e c t s t e n d t o be m o r e m a n a g e a b l e , a n d o f t e n a r e p r o m o t e d by h i g h l y m o t i v a t e d l o c a l a c t i v i s t s who a r e a b l e t o e x t r a c t e x t r a o r d i n a r y e f f o r t s f r o m p r o j e c t s t a f f . H o w e v e r , s u c h f a c t o r s a r e o f t e n n o t a c c o u n t e d f o r . ( 2 9 ) T h e t i m e f r a m e o v e r w h i c h a n e v a l u a t i o n i s c o n d u c t e d , a n d 44 the p o i n t a t w h i c h the e v a l u a t o r becomes i n v o l v e d , can a l s o a f f e c t the v a l i d i t y and u t i l i t y of the r e s u l t s of the a n a l y s i s . The f i r s t few y e a r s of a p r i m a r y h e a l t h c a r e program a r e u s u a l l y s p e n t i n d e v e l o p i n g o p e r a t i o n a l c o h e s i o n , b u i l d i n g community c o o p e r a t i o n and t r u s t , or c o p i n g w i t h the a f t e r m a t h o f an u n e x p e c t e d s i t u a t i o n . ( 3 0 ) E v a l u a t o r s p a r a c h u t e d i n t o the p r o j e c t o f t e n a r e n o t c o n s c i o u s o f t h e s e p r o c e s s e s . They may c o n c l u d e t h a t the p r o j e c t has f a i l e d b e c a u s e i t d i d n o t a c h i e v e some p r e d e t e r m i n e d o b j e c t i v e o r t a r g e t by a p r e d e f i n e d p o i n t i n t i m e . As Hyman and W r i g h t (1971) s u g g e s t , c o n d u c t i n g an e v a l u a t i o n d u r i n g what they c a l l the ' f i r s t c y c l e ' o f a p r o j e c t can r e s u l t i n a m i s i n t e r p r e t a t i o n of e v e n t s , as the p r o j e c t i s n o t y e t f u n c t i o n i n g a t i t s maximum e f f i c i e n c y and e f f e c t i v e n e s s , and s t a f f may be o v e r l y e n t h u s i a s t i c i n i m p l e m e n t i n g the p r o j e c t . C o n s e q u e n t l y , the r e s u l t s of e v a l u a t i v e r e s e a r c h c o n d u c t e d d u r i n g t h i s p e r i o d a r e n o t n e c e s s a r i l y v a l i d o r m e a n i n g f u l f o r the e n t i r e p r o g r a m . ( 3 1 ) As G w a t k i n e t a l . (1980) p o i n t o u t , t h i s i s an i m p o r t a n t f a c t o r w h i c h i s o f t e n o v e r l o o k e d when e v a l u a t i o n s a r e c o n d u c t e d : The e x p e r i e n c e of t h e s e p r o j e c t s s u g g e s t s t h a t d e v e l o p i n g c o u n t r y v i l l a g e s more o f t e n t h a n n o t d i f f e r f rom one a n o t h e r i n ways t h a t a r e p o t e n t i a l l y i m p o r t a n t f o r i n f a n t and c h i l d m o r t a l i t y , b u t w h i c h a r e o f t e n n o t o b v i o u s to even the most a s t u t e o b s e r v e r s u n t i l t h e y ' v e a c t u a l l y been on the s c e n e f o r a y e a r , two y e a r s , or more. (32) 45 Two o t h e r f a c t o r s i d e n t i f i e d as l i m i t i n g the v a l i d i t y and u s e f u l n e s s of many p r i m a r y h e a l t h c a r e e v a l u a t i o n e f f o r t s have been t h e use o f s m a l l sample s i z e s and i n a d e q u a t e c o n t r o l a r e a s f o r c o m p a r i s o n w i t h the e x p e r i m e n t a l areas.(33) As one s t u d y r e p o r t s : Some o f t h e s e d i f f e r e n c e s (between s t u d y v i l l a g e s ) c an be p a r t l y c o n t r o l l e d f o r by s t a t i s t i c a l means. B u t the a p p l i c a t i o n o f s u c h c o n t r o l s r e q u i r e s c o n s i d e r a b l y l a r g e r samples t h a n has h i t h e r t o been the norm, and . even t h e n t h e i r use o n l y r a r e l y l e a d s to f u l l y m e a n i n g f u l p e r s u a s i v e r e s u l t s . (34) However, the use of complex e x p e r i m e n t - b a s e d r e s e a r c h d e s i g n s a r e e x p e n s i v e . A d d i t i o n a l l y , the manpower r e q u i r e d to d e s i g n and implement them a r e o f t e n n o t a v a i l a b l e i n d e v e l o p i n g c o u n t r i e s due to t h e academic q u a l i f i c a t i o n s r e q u i r e d . Such a p p r o a c h e s as o f t e n n o t p o l i t i c a l l y f e a s i b l e , e s p e c i a l l y when t h e r e a r e c o n t r o l a r e a s where s e r v i c e s a r e n o t to be i n t r o d u c e d c o n c u r r e n t l y w i t h t h e e x p e r i m e n t a l c o m m u n i t i e s . But even where q u a s i - e x p e r i m e n t a 1 d e s i g n s were u s e d , o t h e r f a c t o r s t e n d e d to a f f e c t t h e i r v a l i d i t y and u t i l i t y . I n some c a s e s , c e r t a i n e v a l u a t i o n a c t i v i t i e s were d e l a y e d o r c a n c e l l e d . M o d i f i c a t i o n s to the d e s i g n of many e v a l u a t i o n s a l s o o c c u r e d . T h e s e f a c t o r s o f t e n had s e r i o u s i m p l i c a t i o n s f o r the v a l i d i t y of the e x e r c i s e.(35) The f a c t t h a t the m a j o r i t y of t h e s e PHC p r o j e c t s were d e s i g n e d , i m p l e m e n t e d , e v a l u a t e d , and f i n a n c e d by e x t e r n a l 46 f u n d i n g a g e n c i e s can a l s o a f f e c t t h e i r i n h e r e n t v a l i d i t y and u s e f u l n e s s . As the USAID s t u d y p o i n t s o u t , c o n d i t i o n s imposed by e x t e r n a l f u n d i n g a g e n c i e s and the time t a k e n to d e s i g n and a p p r o v e a p r o p o s a l c an s e r i o u s l y a f f e c t b o t h the PHC program and i t s e v a l u a t i o n : [The] b u r e a c r a t i c p r o c e d u r e s a r e so numerous and t i m e - c o n s u m i n g t h a t t h e y , and n o t t h e s u b s t a n c e of the program, become the f o c i . . . b e c a u s e o f them . . . c e r t a i n h o s t - c o u n t r y r e a l i t i e s ( a r e ) g l o s s e d o v e r . (36) By the time the p r o j e c t i s a p p r o v e d , o b j e c t i v e s and t a r g e t s i n the o r i g i n a l p l a n n i n g document may be i r r e l e v a n t and i m p r a c t i c a l . L i n k e d to t h i s i s the t i m e t a b l e imposed by f u n d i n g a g e n c i e s . F o r b o t h USAID and CIDA, m u l t i y e a r p r o j e c t p l a n s a r e l i m i t e d to a maximum o f t h r e e y e a r s . ( 3 7 ) However, as the USAID r e p o r t p o i n t s o u t , the e x p e r i e n c e s of the PHC p r o j e c t s examined would s u g g e s t t h a t t h i s i s an i n s u f f i c i e n t time h o r i z o n . D e m o n s t r a t i n g h e a l t h i m p a c t may i n r e a l i t y r e q u i r e f i v e o r t e n y e a r s . A d d i t i o n a l l y , most p r o j e c t s a r e c h a r a c t e r i z e d by d e l a y s i n p r o j e c t a c t i v i t i e s , due to a v a r i e t y of r e a s o n s . The u n d e r e s t i m a t i o n o f a r e a l i s t i c t i m e f r a m e f o r the p r o j e c t (and i t s e v a l u a t i o n ) c an s i g n i f i c a n t l y d i s t o r t the i n t e r p r e t a t i o n of the r e s u l t s o f the e v a l u a t i o n e x e r c i s e . The f o r e g o i n g a n a l y s i s has i d e n t i f i e d s e v e r a l f a c t o r s which l i m i t t h e u t i l i t y and v a l i d i t y of d e v e l o p i n g c o u n t r y - b a s e d 47 e v a l u a t i v e r e s e a r c h on p r i m a r y h e a l t h c a r e . But the m a j o r i t y of t h e s e PHC p r o j e c t s were d e s i g n e d , c a r r i e d o ut and e v a l u a t e d by e x p a t r i a t e s . Some o f the pr o b l e m s d i s c u s s e d a r e d i r e c t l y r e l a t e d to t h i s s i t u a t i o n . T h e r e f o r e , i t would be of i n t e r e s t to examine a s e l e c t i o n o f PHC e v a l u a t i o n s d e v e l o p e d , i mplemented and e v a l u a t e d by i n d i g e n o u s r e s e a r c h e r s to see whether t h e y s h a r e the same p r o b l e m s . D e t a i l e d A n a l y s i s o f F i v e P r i m a r y H e a l t h C a r e E v a l u a t i v e S t u d i e s One of t h e f a c t o r s w h i c h has l i m i t e d the v a l i d i t y and u t i l i t y of e v a l u a t i v e r e s e a r c h s t u d i e s on p r i m a r y h e a l t h c a r e c o n d u c t e d i n the d e v e l o p i n g c o u n t r i e s i s the l a c k o f i n d i g e n o u s p e r s o n n e l a d e q u a t e l y t r a i n e d i n e v a l u a t i o n d e s i g n and t e c h n i q u e s . The de v e l o p m e n t of a c a p a c i t y to d e s i g n and implement r e a l i s t i c and p r a g m a t i c d e v e l o p m e n t programs r e q u i r e s the c o n c o m i t a n t d e v e l o p m e n t o f a c a p a c i t y to d e s i g n and implement program e v a l u a t i o n s . Such programs must be e v a l u a t e d i n terms o f t h e i r p o t e n t i a l c o n t r i b u t i o n to the development p r o c e s s . T h i s i s a p r e r e q u i s i t e f o r a c o u n t r y to f o r m u l a t e c o h e r e n t programs r e l a t e d to t h e i r d e v e l o p m e n t o b j e c t i v e s . The aim of d e v e l o p i n g c a d r e s of d e v e l o p i n g c o u n t r y r e s e a r c h e r s and f i e l d s t a f f t r a i n e d i n e v a l u a t i v e r e s e a r c h methods and t e c h n i q u e s a l s o s u p p o r t s one o f the e l e m e n t s of the r a t i o n a l e f o r f o r e i g n a s s i s t a n c e p r o g r a m s : to make the 48 d e v e l o p i n g c o u n t r i e s s e l f - r e l i a n t and to a l a r g e e x t e n t s e l f - s u f f i c i e n t , w i t h r e s p e c t to t h e i r c a p a b i l i t y to u t i l i z e i n d i g e n o u s p e r s o n n e l to d e s i g n , implement and e v a l u a t e n a t i o n a l s o c i a l s e r v i c e p rograms. Such a s t r a t e g y c o u l d s e r v e to r e d u c e the dependence on f o r e i g n e x p e r t s and c o n s u l t a n t s to d e s i g n and implement s o p h i s t i c a t e d , e x p e n s i v e and t i m e - c o n s u m i n g program e v a l u a t i o n s w h i c h may have l i m i t e d u t i l i t y to n a t i o n a l d e c i s i o n - m a k e r s . A d d i t i o n a l l y , l o c a l l y r e l e v a n t e v a l u a t i v e p r o c e d u r e s c o u l d be e s t a b l i s h e d , more s e n s i t i v e to l o c a l c o n d i t i o n s t h a n i m p o r t e d t e c h n i q u e s . The p u r p o s e o f t h e f o l l o w i n g a n a l y s i s i s to i d e n t i f y the f a c t o r s w h i c h l i m i t the u t i l i t y and v a l i d i t y o f e v a l u a t i v e r e s e a r c h on p r i m a r y h e a l t h c a r e c o n d u c t e d by i n d i g e n o u s r e s e a r c h e r s . T h i s a n a l y s i s does n o t p u r p o r t to p r o v e t h a t e v a l u a t i v e r e s e a r c h c o n d u c t e d by d e v e l o p i n g c o u n t r y r e s e a r c h e r s i s b e t t e r o r worse t h a n t h a t c o n d u c t e d by e x p a t r i a t e r e s e a r c h e r s . The aim i s to p r o b e , to i d e n t i f y what e l e m e n t s of e v a l u a t i v e ' r e s e a r c h a r e i n a d e q u a t e . The f i v e c a s e s t u d i e s t h a t f o l l o w were n o t s e l e c t e d a t random. They r e p r e s e n t c a s e s w h i c h a r e a d e q u a t e f o r the p r e s e n t s t u d y , i n terms of the n a t u r e , q u a n t i t y , and q u a l i t y of the i n f o r m a t i o n r e p o r t e d . The s e l e c t i o n c r i t e r i a f o r the e v a l u a t i o n s t u d i e s r e v i e w e d h e r e were t h a t the p r i m a r y h e a l t h c a r e p r o j e c t i n c l u d e d a w e l l d e t a i l e d e v a l u a t i v e r e s e a r c h component; t h a t t h e e v a l u a t i o n had been c o n d u c t e d by d e v e l o p i n g c o u n t r y 49 r e s e a r c h e r s ; and, t h a t the e v a l u a t i o n r e p o r t was a v a i l a b l e i n e i t h e r E n g l i s h , F r e n c h , or S p a n i s h . T h e s e c r i t e r i a s e v e r e l y l i m i t e d the number of p r o j e c t r e p o r t s a p p l i c a b l e to t h i s s t u d y . F i r s t , p r o j e c t r e p o r t s w hich p r o v i d e d e t a i l e d d e s c r i p t i o n s of p r o j e c t - r e l a t e d e v a l u a t i v e a c t i v i t i e s a r e few. Many s t u d i e s r e p o r t g e n e r a l f i n d i n g s , b u t f a i l to p r o v i d e i n f o r m a t i o n on the d e s i g n and t e c h n i q u e s used i n c a r r y i n g o u t t h e e v a l u a t i o n . Second, the number o f e v a l u a t i o n s c o n d u c t e d by d e v e l o p i n g c o u n t r y r e s e a r c h e r s , f o r w h i c h r e p o r t s a r e a v a i l a b l e , i s l i m i t e d . F o l l o w i n g a c a r e f u l r e v i e w of a dozen p o t e n t i a l l y a p p l i c a b l e p r i m a r y h e a l t h c a r e e v a l u a t i o n r e p o r t s , f i v e were c h o s e n . The p r o j e c t s s e l e c t e d f o r t h i s s t u d y do n o t r e p r e s e n t the t o t a l i t y o f p r i m a r y h e a l t h c a r e p r o j e c t s implemented i n d e v e l o p i n g c o u n t r i e s , o n l y a s u b s e t . However, the a u t h o r b e l i e v e s , b a s e d on h i s p e r s o n a l e x p e r i e n c e , t h a t the s h o r t c o m i n g s w i t h i n t h e c a s e s to be r e v i e w e d a r e r e p r e s e n t a t i v e of the m a j o r i t y o f PHC program e v a l u a t i o n s w h i c h a r e c o n d u c t e d i n d e v e l o p i n g c o u n t r i e s . Nor s h o u l d t h i s s t u d y be t a k e n as a c r i t i q u e of the e f f o r t s of the r e s e a r c h teams i n v o l v e d , n o r the r a t i o n a l e f o r or a c t i v i t i e s o f the p r o j e c t s . The o b j e c t i v e o f t h i s e x e r c i s e i s s i m p l y to r e v i e w and a n a l y s e the p r o j e c t e v a l u a t i o n s . As s u c h , t h i s r e v i e w c o n f i n e s i t s e l f to p r e s e n t i n g d a t a r e v e a l e d i n the p r o j e c t e v a l u a t i o n r e p o r t s . 50 i . G e n e r a l D e s c r i p t i o n of the F i v e Case S t u d i e s The f i v e p r i m a r y h e a l t h c a r e p r o j e c t s to be r e v i e w e d t o o k p l a c e i n r u r a l s e t t i n g s i n a v a r i e t y of T h i r d W orld c o u n t r i e s : t h r e e i n L a t i n A m e r i c a ; two i n the C a r i b b e a n ; and two i n A s i a . No p r i m a r y h e a l t h c a r e e v a l u a t i o n r e p o r t s from A f r i c a s a t i s f i e d the s e l e c t i o n c r i t e r i a . The p r o j e c t s took p l a c e between 1973 and 1980. The l o n g e s t p r o j e c t p e r i o d e v a l u a t e d spanned f i v e y e a r s , and the s h o r t e s t two y e a r s . A l l the p r i n c i p a l i n v e s t i g a t o r s f o r the p r o j e c t s and t h e i r e v a l u a t i o n s were i n d i g e n o u s a c a d e m i c s , c o n n e c t e d w i t h l o c a l S c h o o l s o f P u b l i c a n d / o r Community H e a l t h i n t h e i r r e s p e c t i v e c o u n t r i e s . The p r o j e c t s d i d n o t v a r y s i g n i f i c a n t l y w i t h r e s p e c t to the n a t u r e and r a n g e o f p r i m a r y h e a l t h c a r e s e r v i c e s o f f e r e d . I n g e n e r a l , t h e f i v e p r o j e c t s c o n c e n t r a t e on d e l i v e r i n g p r e v e n t i v e s e r v i c e s , and a l l p r o v i d e d b a s i c c u r a t i v e s e r v i c e s . Most o f f e r e d m o t h e r - c h i l d h e a l t h a c t i v i t i e s , m o n i t o r e d n u t r i t i o n a l s t a t u s o f c h i l d r e n , p r o v i d e d i m m u n i z a t i o n s , and promoted h e a l t h e d u c a t i o n a c t i v i t i e s . T h r e e o f the p r o j e c t s a l s o promoted f a m i l y p l a n n i n g , and f o u r of them i n c l u d e d w a t e r s u p p l y and s a n i t a t i o n improvement a c t i v i t i e s . F o u r o f the e v a l u a t i o n s s o u g h t to a s s e s s the i mpact of p r i m a r y h e a l t h c a r e a c t i v i t i e s on the h e a l t h s t a t u s and w e l l - b e i n g o f the s t u d y c o m m u n i t i e s . The o t h e r t h r e e were p r i m a r i l y c o n c e r n e d w i t h c o n d u c t i n g p r o c e s s and o u t p u t 51 e v a l u a t i o n s . Two of t h e s e r e v i e w e d the a t t i t u d e s o f community h e a l t h w o r k e r s (CHWs) i n c a r r y i n g out t h e i r a s s i g n e d t a s k s , and t h e a t t i t u d e s of community members towards the CHWs. The r e m a i n i n g s t u d y a n a l y z e d the o u t p u t and a c t i v i t i e s c a r r i e d o u t by p r i m a r y h e a l t h c a r e d i s p e n s a r i e s i n t h e s t u d y a r e a . Most o f the e v a l u a t i o n s were c o n c e r n e d w i t h s e r v i c e d e l i v e r y r a t h e r t h a n w i t h outcomes. The i n d i c a t o r s most commonly us e d were d a t a on changes i n h e a l t h c a r e f a c i l i t y and p e r s o n n e l u t i l i z a t i o n , a v a i l a b i l i t y , and a c c e s s i b i l i t y , and i n the k n owledge, a t t i t u d e s and p r a c t i c e s o f the s t u d y p o p u l a t i o n s w i t h r e s p e c t to h e a l t h - r e l a t e d b e h a v i o u r . None of the e v a l u a t i o n s u s e d a t r u e e x p e r i m e n t a l d e s i g n , a l t h o u g h two o r i g i n a l l y i n t e n d e d to do s o . The most f r e q u e n t l y used d e s i g n was a one group p r e t e s t - p o s t t e s t d e s i g n w h i c h compared c o n d i t i o n s i n the s t u d y v i l l a g e s b e f o r e and a f t e r the i n t r o d u c t i o n of the p r i m a r y h e a l t h c a r e a c t i v i t i e s . The p r o j e c t s and the r e s u l t s of t h e i r e v a l u a t i o n r e p o r t s a r e p r e s e n t e d s e p a r a t e l y . E a c h r e v i e w c o n s i s t s o f a b r i e f b a c k g r o u n d d e s c r i p t i o n of t h e p r o j e c t , i t s o b j e c t i v e s , and a d e t a i l e d a n a l y s i s o f the e v a l u a t i v e r e s e a r c h d e s i g n and t e c h n i q u e s . A summary o f p r o j e c t c h a r a c t e r i s t i c s i s p r o v i d e d i n T a b l e 1, and a summary of the c h a r a c t e r i s t i c s o f t h e i r r e s p e c t i v e e v a l u a t i o n s i n T a b l e 2. PHC P r o j e c t D u r a t i o n l . K a v a r V i l l a g e H e a l t h Worker P r o j ec t ( I r a n ) 1973- 1974 (24 mos.) (phase 1) 1974- 1978 (38 mos.) (phase 2) 2 . R u r a l H e a l t h Development Program ( C o l o m b i a ) 1974-1979 (60 mos.) 3 . R u r a l H e a l t h and F a m i l y P l a n n i n g P r o j e c t ( H a i t i ) 1975-1978 (36 mos.) TABLE 1 STUDY CHARACTERISTICS S e r v i c e s O f f e r e d T o t a l S t u d y Sample P o p u l a t i o n S i z e - b a s i c c u r a t i v e 10,000 1007. - r e f e r r a l to c l i n i c - f a m i l y p l a n n i n g - e n v i r o n m e n t a l h e a l t h e d u c a t i o n - b a s i c c u r a t i v e 10,600 507. - r e f e r r a l to c l i n i c - f a m i l y p l a n n i n g - n u t r i t i o n e d u c a t i o n - c h i l d i m m u n i z a t i o n - e n v i r o n m e n t a l h e a l t h promo t i o n - b a s i c c u r a t i v e 34,000 - p r e v e n t i v e - f a m i l y p l a n n i n g - c h i l d i m m u n i z a t i o n - n u t r i t i o n e d u c a t i o n / r e h a b i l i t a t i o n - m a t e r n a l / c h i l d h e a l t h 1007. (2 07. of women 15-44 y e a r s ) TABLE 1 ( c o n t i n u e d ) 4 . V o l u n t e e r 1 9 7 6 - 1 9 7 8 H e a l t h ( 3 0 mos.) Workers ( T h a i l a n d ) 5 . R u r a l T e a c h e r s 1 9 7 8 - 1 9 8 0 H e a l t h Program ( 3 2 mos.) (Pa r a g u a y ) b a s i c c u r a t i v e m a l a r i a t r e a t m e n t home b i r t h s n u t r i t i o n e d u c a t i o n p r e v e n t i o n of communicable d i s e a s e s 5 6 4 h o u s e h o l d s ( 4 / 9 s t u d y v i l l a g e s . 5 0 7 o sample f o r dem o g r a p h i c s u r v e y ) 3 2 5 h o u s e h o l d s ( 2 / 9 s t u d y v i l l a g e s f o r KAP and m o r b i d i t y d a t a ) b a s i c c u r a t i v e 2 3 , 9 5 8 1 0 0 7 , c h i l d i m m u n i z a t i o n r e f e r r a l to c l i n i c p r e v e n t i v e h e a l t h promo t i o n e n v i r o n m e n t a l h e a l t h p r o m o t i o n TABLE 2 CHARACTERISTICS OF EVALUATIONS P r o j e c t R e s e a r c h Outcome D a t a C o l l e c t i o n Pes i g n Measures T e c h n i q u e s l . K a v a r V i l l a g e H e a l t h Worker P r o j e c t ( I r a n ) -phase 1 -phase 2 one group p r e t e s t - p o s t t e s t n o n e q u i v a l e n t group p o s t t e s t • 7. change IMR • 7« change m a t e r n a l m o r t a l i t y •changes i n KAP f o r p e r s o n a l h y g i e n e & h e a l t h m a i n t e n a n c e • 7o change i n b i r t h r a t e and f a m i l y s i z e - d i r e c t s u r v e y s ( r e s e a r c h e r s and VHWs) - r e c o r d s y s tems (VHWs & d i s t r i c t h e a l t h c l i n i c ) 2 . R u r a l H e a l t h Development Program ( C o l o m b i a ) one group p r e t e s t - p o s t t e s t • a c c e p t a n c e o f VHws by community • % of p o p u l a t i o n c o v e r e d • f r e q u e n c y o f VHW a c t i v i t i e s f o r same c l i e n t •# a c t i v i t i e s c o n d u c t e d by VHW d i r e c t s u r v e y s (VHWs) r e c o r d systems (VHWs) monthly s u p e r -v i s o r y r e p o r t s ( r e s e a r c h team) 3 . R u r a l H e a l t h and F a m i l y P l a n n i n g P r o j e c t ( H a i t i ) s i n g 1 e t i m e - s e r i e s a n a l y s i s - 7. change IMR - 7. change i n f a n t m o r b i d i t y - % change a d u l t m o r b i d i t y - i n f a n t growth - 7» change b i r t h r a t e - d i r e c t s u r v e y s (VHWs & r e s e a r c h team ) - r e c o r d s y s tems (VHWs & d i s t r i c t h e a l t h c l i n i c ) TABLE 2 ( c o n t i n u e d ) 4 . V o l u n t e e r o n e - s h o t H e a l t h Workers c a s e s t u d y ( T h a i l a n d ) 5 . R u r a l T e a c h e r s one group H e a l t h Program p r e t e s t - p o s t t e s t ( P a r a g u a y ) of VHW v i l l a g e r s u t i l i z a t i o n a t t i t u d e of towards VHW use o f a l t e r n a t i v e m e d i c a l s e r v i c e s # of a c t i v i t i e s c o n d u c t e d by VHW • d i r e c t s u r v e y s (VHWs & r e s e a r c h team) • r e c o r d systems (VHWs) % change c h i l d m o r t a l ! t y 7.. change c h i l d m o r b i d i t y u t i l i z a t i o n of t e a c h e r s as VHWs a c c e p t a n c e of t e a c h e r as VHW by community change i n e n v i r o n m e n t a l c o n d i t i o n s d i r e c t s u r v e y s ( t e a c h e r s & r e s e a r c h team) r e c o r d systems ( t e a c h e r s & d i s t r i c t h e a l t h c l i n i c s ) 56 i i . K a v a r V i l l a g e H e a l t h Worker P r o j e c t ( I r a n ) Of t h e f i v e p r o j e c t e v a l u a t i o n s to be r e v i e w e d , t h i s i s the most e x t e n s i v e l y r e p o r t e d . ( 3 8 ) The p r o j e c t e x t e n d e d o v e r a f i v e y e a r p e r i o d (1973-1978) and was d e s i g n e d and implemented by p e r s o n n e l from the Department of Community H e a l t h , P a h l a v i U n i v e r s i t y , l o c a t e d i n S h i r a z , I r a n . The p r o j e c t s i t e was s i t u a t e d on t h e K a v a r a g r i c u l t u r a l p l a i n i n s o u t h e r n I r a n ( F a r s P r o v i n c e ) . The v i l l a g e h e a l t h w o r k e r s (VHWs) were e x p e c t e d to p r o v i d e b a s i c c u r a t i v e s e r v i c e s ( p r i m a r i l y f i r s t a i d ) , r e f e r r a l s to the r e g i o n a l h e a l t h s t a t i o n , f a m i l y p l a n n i n g i n f o r m a t i o n , and e n v i r o n m e n t a l improvement and h y g i e n e e d u c a t i o n . The u l t i m a t e o b j e c t i v e o f the p r o j e c t e v a l u a t i o n was to measure t h e i m p a c t of l o c a l l y s e l e c t e d and t r a i n e d v i l l a g e h e a l t h w o r k e r s . The p r o j e c t was d i v i d e d i n t o two p h a s e s . The o b j e c t i v e o f the f i r s t p h a s e , w h i c h e x t e n d e d o v e r a twenty f o u r month p e r i o d , was to " s t u d y the f e a s i b i l i t y of t r a i n i n g v i l l a g e r s who p o s s e s s b a s i c l i t e r a c y to p r o v i d e p r i m a r y m e d i c a l c a r e and p r e v e n t i v e h e a l t h c a r e i n r u r a l a r e a s " . D u r i n g the f i r s t s i x months of the p r o j e c t , s t u d y v i l l a g e s were s e l e c t e d , the t r a i n i n g c u r r i c u l a and m a t e r i a l s d e v e l o p e d , VHW t r a i n e e s s e l e c t e d , and the b a s e l i n e s u r v e y c o m p l e t e d . The t r a i n e d VHWs were t r a i n e d and d e p l o y e d to the f i e l d d u r i n g the s econd s i x 57 month p e r i o d . A t t h e end o f the twenty f o u r months, the f o l l o w - u p s u r v e y s were c a r r i e d o u t , d a t a a n a l y s e d , and an e v a l u a t i o n r e p o r t p r o d u c e d . The r e s e a r c h e r s i n t e n d e d o r i g i n a l l y to use a p r e t e s t - p o s t t e s t c o n t r o l group e v a l u a t i o n r e s e a r c h d e s i g n . I n f a c t , the a c t u a l d e s i g n used i n the f i r s t phase e v a l u a t i o n was a one-group p r e t e s t - p o s t t e s t . No e x p l a n a t i o n was p r o v i d e d as to why t h i s change o c c u r r e d . Of the f i f t y - f i v e v i l l a g e s o r i g i n a l l y s c h e d u l e d to be i n c l u d e d i n t h e s t u d y , f i f t e e n were d e s i g n a t e d as e x p e r i m e n t a l v i l l a g e s , and s i x as c o n t r o l s . No c r i t e r i a f o r s e l e c t i n g t h e s e p a r t i c u l a r v i l l a g e s were p r o v i d e d . B a s e l i n e s u r v e y s were c a r r i e d o u t to c o l l e c t d emographic d a t a , as w e l l as i n f o r m a t i o n on p r e v a i l i n g e n v i r o n m e n t a l c o n d i t i o n s ( w e l l s , l a t r i n e s , e t c . ) , and the i d e n t i f i c a t i o n of u s e r s o f f a m i l y p l a n n i n g methods i n a l l s t u d y v i l l a g e s . A b a s e l i n e k n owledge, a t t i t u d e s and p r a c t i c e (KAP) s u r v e y on h e a l t h - r e l a t e d b e h a v i o u r was c o n d u c t e d f o r 2 0 0 h o u s e h o l d s i n the s t u d y a r e a (a 1 0 7 o sample, randomly s e l e c t e d ) . No s u c h s t u d y was c o n d u c t e d f o r the c o n t r o l v i l l a g e s . The r e s e a r c h e r s r e p o r t e d t h a t "due to c u l t u r a l r e a s o n s , f e m a l e s may be u n d e r r e p o r t e d i n a c t u a l b a s e l i n e p o p u l a t i o n s u r v e y " . However, no i n d i c a t i o n was g i v e n as to e i t h e r the magnitude of the u n d e r r e p o r t i n g , n o r the e f f e c t i t had on the a n a l y s i s . F o l l o w - u p s u r v e y s were s c h e d u l e d to be c o n d u c t e d i n the e i g h t e e n t h month of the p r o j e c t p e r i o d , and r e p e a t e d a t the end 58 o f the p r o j e c t . The i n d i c a t o r s c h o s e n by the r e s e a r c h team to measure the s u c c e s s o f t h e program were the u t i l i z a t i o n of s e r v i c e s o f f e r e d by the VHW, changes i n s a n i t a t i o n and h y g i e n e b e h a v i o u r , and an i n c r e a s e i n the a c c e p t a n c e o f f a m i l y p l a n n i n g . A c c e s s i b i l i t y was n o t measured s i n c e the VHW c l i n i c was j u s t a s h o r t walk from the hou s e s i n t h e v i l l a g e s . The r e s e a r c h e r s assumed t h a t the u t i l i z a t i o n o f s e r v i c e s would s e r v e as an a d e q u a t e s u r r o g a t e measure f o r the a c c e p t a n c e of the VHW by the e x p e r i m e n t a l p o p u l a t i o n . The VHWs m a i n t a i n e d r e c o r d s on the number o f mo n t h l y p a t i e n t v i s i t s to the v i l l a g e c l i n i c they were a s s i g n e d t o , as w e l l as the age, sex and d i a g n o s i s o f each c a s e . They a l s o r e c o r d e d the t y p e and f r e q u e n c y of e n v i r o n m e n t a l improvement, community e d u c a t i o n , home and s c h o o l v i s i t s and f a m i l y p l a n n i n g a c t i v i t i e s . They were a l s o e x p e c t e d to p r e p a r e b i m o n t h l y r e p o r t s on v i t a l s t a t i s t i c s f o r t h e i r a s s i g n e d v i l l a g e s . The p r o j e c t e v a l u a t i o n r e p o r t e d t h a t 11,467 p a t i e n t s had c o n s u l t e d t h e VHWs d u r i n g t h e s t u d y p e r i o d , an a v e r a g e of 1.4 v i s i t s / p e r s o n / y e a r . I t was a l s o r e p o r t e d t h a t p e r s o n s aged 20-44 y e a r s t e n d e d to use the VHW s e r v i c e s l e s s t h a n t h o s e o f f e r e d a t t h e h i g h e r l e v e l H e a l t h C e n t e r . The VHWs a l s o tended to be c o n s u l t e d more f o r l o w e r - o r d e r m e d i c a l c o n d i t i o n s . The p r o j e c t e v a l u a t i o n r e p o r t n o t e d t h a t f e m a l e s used the v i l l a g e h e a l t h w o r k e r s ' s e r v i c e s i n s i g n i f i c a n t l y g r e a t e r p r o p o r t i o n 59 (p>.01) t h a n m a l e s , w i t h t h i s d i f f e r e n t i a l b e i n g most p r o n o u n c e d where f e m a l e VHWs o f f e r e d t r e a t m e n t s e r v i c e s . The Phase 1 e v a l u a t i o n s t u d y a l s o f o u n d t h a t the number of p a t i e n t v i s i t s to the d i s t r i c t h e a l t h c e n t r e d e c r e a s e d by 43.57. f o r the s t u d y v i l l a g e s o v e r t h e f i r s t s i x months of t h e s t u d y , b u t t h a t no s u c h d e c r e a s e o c c u r r e d i n u t i l i z a t i o n p a t t e r n s i n the c o n t r o l v i l l a g e s . D u r i n g the c o u r s e of the f i r s t phase of the p r o j e c t , f e m a l e u s e r s o f c o n t r a c e p t i v e s r e p o r t e d l y i n c r e a s e d by 2307.. B e f o r e the a r r i v a l o f VHWs, 77. o f women 15-44 y e a r s of age s t a t e d t h a t t h e y u s e d some form o f c o n t r a c e p t i v e ; a t t h e end of the Phase 1 s t u d y p e r i o d , 247. o f t h e same c o h o r t s t a t e d t h a t t h e y used some form of c o n t r a c e p t i o n . The t e r m i n a t i o n r a t e i n t h e s t u d y v i l l a g e s was c a l c u l a t e d as 25.6/1000 women 15-44 y e a r s of age u s i n g c o n t r a c e p t i v e s , and 30.2/1000 f o r the same c o h o r t i n the c o n t r o l v i l l a g e s . A c c o r d i n g to the s t u d y , t h i s p o i n t p a t t e r n d e m o n s t r a t e d t h a t t h e i n c r e a s e i n p i l l use i n VHW v i l l a g e s has n o t been among women who s i m p l y a c c e p t e d a p a c k e t o f p i l l s b e c a u s e t h e y were u r g e d to by the VHW, b u t i l l u s t r a t e d the e f f e c t i v e n e s s of the f a m i l y p l a n n i n g e d u c a t i o n component of the p r o j e c t . The r e s e a r c h e r s c o n d u c t e d a s u r v e y a t t h e end of the p r o j e c t p e r i o d to d e t e r m i n e v i l l a g e r s a t i s f a c t i o n w i t h the VHWs. The v a s t m a j o r i t y o f the r e s p o n d e n t s e x p r e s s e d s a t i s f a c t i o n , and t h a t most had had v e r b a l c o n t a c t w i t h the VHW 60 sometime d u r i n g the p r o j e c t p e r i o d . U n f o r t u n a t e l y , d a t a on s a n i t a t i o n and p r e v e n t i v e s e r v i c e s were c o l l e c t e d t oo i n f r e q u e n t l y to e n a b l e the r e s e a r c h e r s to e v a l u a t e the r e l a t i o n s h i p between VHW a c t i v i t i e s and s a n i t a r y and h y g i e n e b e h a v i o u r . Based on t h e r e s u l t s of the e v a l u a t i v e a n a l y s i s , the r e s e a r c h e r s c o n c l u d e d t h a t the "VHWs a r e h i g h l y p r o d u c t i v e and w e l l a c c e p t e d i n t h e v i l l a g e s they s e r v e " and t h a t " t h e VHW c o n c e p t i s n o t o n l y f e a s i b l e , b u t would p r o v i d e a s t r o n g f o u n d a t i o n f o r I r a n ' s d e v e l o p i n g h e a l t h d e l i v e r y s y s t e m " . The s e c o n d phase o f the p r o j e c t , w h i c h e x t e n d e d o v e r t h i r t y e i g h t months, s o u g h t to d e t e r m i n e " i f t h e r e was some i n d i c a t i o n t h a t t h e s e r v i c e s c o n d u c t e d by the VHWs were e f f e c t i n g a p o s i t i v e change on the h e a l t h s t a t u s of c o n s t i t u e n t p o p u l a t i o n s " . The r e s e a r c h team s u g g e s t e d t h a t a p o s i t i v e d i f f e r e n c e i n v i t a l s t a t i s t i c s s u c h as i n f a n t m o r t a l i t y r a t e s between t h e e x p e r i m e n t a l and c o n t r o l v i l l a g e s would i n d i c a t e t h a t t h e VHWs were e f f e c t i v e . T h i s phase was i n t e n d e d to be an e x t e n s i o n of t h e f i r s t phase e v a l u a t i v e e x e r c i s e . B a s e l i n e d a t a c o l l e c t e d i n Phase 1 was to be used f o r c o m p a r i s o n w i t h d a t a c o l l e c t e d a t the end o f Phase 2 . However, t h i s d i d n o t o c c u r . I n f a c t , the e v a l u a t i v e r e s e a r c h d e s i g n used to measure impact of the s e c o n d phase o f the p r o j e c t was a n o n e q u i v a 1 e n t group pos t t e s t . The s e c o n d phase e v a l u a t i o n r e p o r t n o t e d t h a t the v i l l a g e r s 61 c o n s t r u c t e d many l a t r i n e s d u r i n g the p r o j e c t p e r i o d . However, whether o r n o t t h i s was due to e n v i r o n m e n t a l c o n t r o l p r o m o t i o n a l a c t i v i t i e s o f the VHW was n o t e s t a b l i s h e d . Nor were t h e a c c e p t a n c e and u t i l i z a t i o n of f a m i l y p l a n n i n g methods e v a l u a t e d i n t h i s p h a s e . The e v a l u a t i o n a l s o r e p o r t e d t h a t the i n f a n t m o r t a l i t y r a t e f o r t h e e x p e r i m e n t a l v i l l a g e s (84/1000 i n f a n t s l e s s t h a n 12 months o l d ) was l e s s t h a n t h a t documented i n the c o n t r o l v i l l a g e s (138/1000 i n f a n t s l e s s t h a n 12 months o l d ) a t the end o f the p r o j e c t . However, i t i s h a r d to u n d e r s t a n d how t h e r e s e a r c h e r s c o u l d c o n c l u d e t h a t the v i l l a g e h e a l t h w o r k e r s were r e s p o n s i b l e f o r p o s i t i v e l y a f f e c t i n g the h e a l t h s t a t u s of the s t u d y p o p u l a t i o n . F i r s t , t h e e v a l u a t i v e r e s e a r c h d e s i g n used i n t h e s t u d y d i d n o t compare p r e t e s t and p o s t t e s t d a t a f o r t h e s t u d y and c o m p a r a t i v e p o p u l a t i o n s . B a s i n g r e s e a r c h r e s u l t s on p o s t t e s t d a t a between n o n e q u i v a l e n t g r o u p s i s a v e r y weak d e s i g n . S e cond, t h e d a t a w h i c h was c o l l e c t e d s i x months a f t e the s t a r t o f Phase Two would seem to c o n t r a d i c t the c o n c l u s i o n drawn by t h e r e s e a r c h team. T h e s e d a t a i n d i c a t e d t h a t t h e i n f a n t m o r t a l i t y r a t e s were 65/1000 and 128/1000 i n f a n t s r e s p e c t i v e l y . I n b o t h c a s e s , t h e i n f a n t m o r t a l i t y r a t e s a c t u a l l y i n c r e a s e d o v e r the s t u d y p e r i o d . Y e t the r e s e a r c h e r s f a i l e d to c o n d u c t a c o m p a r a t i v e a n a l y s i s of t h e s e d a t a . No e x p l a n a t i o n was p r o v i d e d f o r t h i s o v e r s i g h t . 62 i i i . R u r a l H e a l t h Development Program ( C o l o m b i a ) The g o a l o f t h i s p r o j e c t was to d e s i g n , implement and e v a l u a t e a r u r a l h e a l t h c a r e d e l i v e r y system d e s i g n e d to improve the h e a l t h s t a t u s of r u r a l c o m m u n i t i e s . ( 3 9 ) The h e a l t h c a r e d e l i v e r y program was one c o n s t i t u e n t e l e m e n t of a l a r g e r m u l t i d i s c i p l i n a r y d e v e l o p m e n t program c o n d u c t e d by the C e n t r o de I n v e s t i g a c i o n e s M u l ' t i d i s c i p l i n a r i a s en D e s a r o l l o (CIMDER), l o c a t e d i n C a l i , C o l o m b i a . A wide r a n g e of s e r v i c e s were o f f e r e d , i n c l u d i n g f a m i l y p l a n n i n g , n u t r i t i o n a l s u r v e i l l e n c e and e d u c a t i o n , and the p r o m o t i o n of community p a r t i c i p a t i o n i n h e a 1 t h - r e l a t e d a c t i v i t i e s . A l t h o u g h the p r o j e c t m o n i t o r e d n u t r i t i o n a l s t a t u s , t h e p r o j e c t d i d n o t i n c l u d e a s u p p l e m e n t a l f e e d i n g program. N e i t h e r d i d the p r o j e c t i n v o l v e the c o n s t r u c t i o n o f i m p roved w e l l s and l a t r i n e s , a l t h o u g h the community h e a l t h w o r k e r s promoted t h e i r c o n s t r u c t i o n and u t i l i z a t i o n . The p r o j e c t p e r i o d e x t e n d e d o v e r f i v e y e a r s ( 1 9 7 4 - 1 9 7 9 ) . The p r o j e c t a r e a was l o c a t e d i n Cauca s t a t e , and c o n t a i n e d a p o p u l a t i o n of a p p r o x i m a t e l y 10,600. The s t u d y a r e a was d i v i d e d i n t o s e v e n t e e n s u b a r e a s , w i t h a t r a i n e d h e a l t h p r o m o t e r a s s i g n e d to each s u b a r e a . The o b j e c t i v e s of the h e a l t h c a r e p r o j e c t were to make p r i m a r y h e a l t h c a r e a v a i l a b l e to 1007. of the e x p e r i m e n t a l p o p u l a t i o n , to improve a c c e s s i b i l i t y to p r i m a r y h e a l t h c a r e s e r v i c e s f o r a t l e a s t 807. of t h i s 63 p o p u l a t i o n , and to r e d u c e the magnitude of t h e i r h e a l t h prob1 ems. To e v a l u a t e the p e r f o r m a n c e and i m p a c t of the p r o j e c t , the r e s e a r c h e r s used a one group p r e t e s t - p o s t t e s t e v a l u a t i v e r e s e a r c h d e s i g n . B a s e l i n e d e m o graphic and h e a l t h d a t a were c o l l e c t e d i n a h o u s e h o l d s u r v e y of 1 ,200 f a m i l i e s (507. s a m p l e ) . The h e a l t h p r o m o t e r s a l s o c o l l e c t e d h e a l t h s t a t u s i n f o r m a t i o n m o n t hly on a l l i n d i v i d u a l s r e s i d i n g i n t h e i r a r e a of a s s i g n m e n t . T h i s i n f o r m a t i o n was used to p l a n t h e i r a c t i v i t i e s d u r i n g the p r o j e c t , and to s u p p l e m e n t the d a t a c o l l e c t e d i n the b a s e l i n e s u r v e y . I n f o r m a t i o n was a l s o c o l l e c t e d on community a c c e p t a n c e of and s a t i s f a c t i o n w i t h the p r o m o t e r s , and t h e i r p a r t i c i p a t i o n i n h e a 1 t h - r e l a t e d a c t i v i t i e s . The p r o m o t e r s were a l s o a s k e d t h e i r o p i n i o n s on the a t t i t u d e of the r e c i p i e n t p o p u l a t i o n to t h e i r a c t i v i t i e s , and on t h e i r w o r k l o a d . The h o u s e h o l d s u r v e y s were r e p e a t e d i n 1978 and 1979. I n g e n e r a l , i t a p p e a r s t h a t the p r o j e c t o b j e c t i v e s were a t t a i n e d a c c o r d i n g to the r e s u l t s o f the e v a l u a t i o n . F o r example, the p r o j e c t c l a i m e d to have a c h i e v e d an 847» l e v e l of c o v e r a g e , based on a r a t e of 2.52 c o n t a c t s p e r p e r s o n per y e a r a f t e r t w e l v e months, and a r a t e of 3.76 a f t e r twenty f o u r months. The r e s e a r c h e r s were u n a b l e to c a l c u l a t e the d e g r e e of d e c l i n e i n m a t e r n a l m o r t a l i t y as the benchmark r a t e had n o t been e s t a b l i s h e d a t the commencement of the p r o j e c t . However, the e v a l u a t i o n s t u d y r e p o r t e d t h a t the i n c i d e n c e of 64 c o m p l i c a t i o n s a t d e l i v e r y and d u r i n g the p o s t p a r t u m p e r i o d had d e c r e a s e d from 75/1000 b i r t h s to 67/1000 b i r t h s o v e r the f i n a l two y e a r s of the p r o j e c t . I n f a n t m o r t a l i t y r e p o r t e d l y d e c r e a s e d o v e r the p r o j e c t p e r i o d as w e l l , from 151/1000 i n f a n t s to 39/1000 i n f a n t s o v e r a t w e n t y - f o u r month p e r i o d . A s i g n i f i c a n t d e c r e a s e i n the i n c i d e n c e o f d i a r r h e a l - r e l a t e d d i s e a s e s was r e c o r d e d between 1977 and 1979 ( f r o m 295/1000 c h i l d r e n aged l e s s t h a n 14 y e a r s to 105/1000 c h i l d r e n ) . The e v a l u a t i o n a l s o n o t e d an i n c r e a s e i n the number o f l a t r i n e s c o n s t r u c t e d and t h a t the number of p e r s o n s d r a w i n g water from improved w e l l s i n c r e a s e d by 667. o v e r the s t u d y p e r i o d . Whether t h i s was a r e s u l t of the c o n s t r u c t i o n and u t i l i z a t i o n of improved w e l l s and l a t r i n e s was n o t e s t a b l i s h e d . The p r e v a l e n c e of i n c a p a c i t a t i n g d i s e a s e s i n c h i l d r e n , as measured by the number of m i s s e d s c h o o l d a y s , r e p o r t e d l y d e c r e a s e d from 52/1000 s c h o o l c h i l d r e n to 21/1000 c h i l d r e n o v e r the f i v e y e a r s t u d y p e r i o d . I m m u n i z a t i o n - c o n t r o l l a b l e d i s e a s e s a l s o were r e p o r t e d to have d e c r e a s e d o v e r the same p e r i o d , from 24.7 c a s e s per 1000 c h i l d r e n to 15.8/1000, f o l l o w i n g an i n t e n s i v e s t u d y a r e a - w i d e i m m u n i z a t i o n program. Improvements i n g e n e r a l n u t r i t i o n a l s t a t u s of c h i l d r e n were a l s o r e p o r t e d . I t was e s t i m a t e d t h a t t h e p e r c e n t a g e o f c h i l d r e n c l a s s i f i e d as b e i n g i n a normal s t a t e of n u t r i t i o n i n c r e a s e d by 407. o v e r the s t u d y p e r i o d . D e n t a l c a r e r e c o r d s were n o t a n a l y z e d . The 65 e v a l u a t i o n a l s o r e p o r t e d t h a t the m o r b i d i t y r a t e f o r a d u l t s d e c r e a s e d s i g n i f i c a n t l y o v e r the s t u d y p e r i o d . A d e c r e a s e i n the g e n e r a l m o r t a l i t y r a t e was a l s o n o t e d , from 107/1000 p e r s o n s to 75/1000 p e r s o n s . The r e s e a r c h e r s b e l i e v e d t h i s to be of s i g n i f i c a n c e , s i n c e the n a t i o n a l m o r t a l i t y r a t e f o r the same p e r i o d was 90/1000 p e r s o n s . The r e s e a r c h e r s c o n c l u d e d i n the e v a l u a t i o n r e p o r t t h a t s i n c e the s o c i o - e c o n o m i c c o n d i t i o n s of the s t u d y p o p u l a t i o n had n o t changed o v e r the s t u d y p e r i o d , y e t the l e v e l o f c o v e r a g e and a v a i l a b i l i t y of h e a l t h c a r e s e r v i c e s had, the d e c r e a s e s r e c o r d e d i n the m o r t a l i t y and m o r b i d i t y r a t e s were a d i r e c t c o n s e q u e n c e of the e f f e c t s of the h e a l t h program. i v . R u r a l H e a l t h and F a m i l y P l a n n i n g P r o j e c t ( H a i t i ) T h i s p r o j e c t , w h i c h was c o n d u c t e d by p e r s o n n e l from the P u b l i c H e a l t h and P o p u l a t i o n Department of the H a i t i a n M i n i s t r y of P u b l i c H e a l t h , was p a r t of an e x p e r i m e n t a l p r i m a r y h e a l t h c a r e program w h i c h had as i t s g o a l s the p r o v i s i o n of n u t r i t i o n and g e n e r a l community-based p r e v e n t i v e s e r v i c e s , the d e l i v e r y of f a m i l y p l a n n i n g , m a t e r n a 1 - c h i I d h e a l t h s e r v i c e s . Community h e a l t h w o r k e r s (CHWs) were t r a i n e d and d e p l o y e d to d e l i v e r t h e s e s e r v i c e s . The program a l s o i n c l u d e d the t r a i n i n g of l o c a l r e s e a r c h p e r s o n n e l to c o n d u c t a p r o c e s s and i m p a c t e v a l u a t i o n of the h e a l t h c a r e p r o j e c t . ( 4 0 ) The p r o j e c t was implemented i n 66 a r u r a l a r e a i n s o u t h w e s t H a i t i , w i t h a t o t a l s t u d y p o p u l a t i o n of a p p r o x i m a t e l y 34,000 p e o p l e . The C a n a d i a n I n t e r n a t i o n a l D evelopment Agency (CIDA) was f i n a n c i n g a major i n t e g r a t e d r u r a l d e v e l o p m e n t program i n the s t u d y a r e a , i n c o o p e r a t i o n w i t h s e v e r a l H a i t i a n government m i n i s t r i e s . The p r i m a r y h e a l t h c a r e p r o j e c t e x t e n d e d o v e r f i v e y e a r s ( 1 9 7 5 - 1 9 8 0 ) . The e v a l u a t i o n s t u d y c o v e r s the f i r s t t h r e e y e a r s of p r o j e c t a c t i v i t i e s . A c a s e - c o n t r o l s t u d y , u s i n g t h r e e e x p e r i m e n t a l s i t e s and one c o n t r o l , was o r i g i n a l l y p l a n n e d to e v a l u a t e the PHC p r o j e c t . However, a time s e r i e s a n a l y s i s w h i c h compares outcome d a t a from t h e t h r e e e x p e r i m e n t a l s i t e s was u l t i m a t e l y u s e d . No r e a s o n f o r t h i s change i n d e s i g n s t r u c t u r e was p r o v i d e d by the r e s e a r c h e r s . I n d i c a t o r s to measure the i m p a c t and p e r f o r m a n c e of the p r o j e c t were m o r t a l i t y , m o r b i d i t y and f e r t i l i t y r a t e s . B a s e l i n e d e m o g r a p h i c and v i t a l s t a t i s t i c s d a t a were c o l l e c t e d a t the t h r e e e x p e r i m e n t a l s i t e s f o r a l l h o u s e h o l d s . A sample of 750 women aged 15-44 y e a r s ( a p p r o x i m a t e l y a 207. sample of t h i s c o h o r t ) was s e l e c t e d to be i n t e r v i e w e d and f o l l o w e d c o n c e r n i n g b i r t h s and c o n t r a c e p t i v e p r a c t i c e s . An e p i d e m i o l o g i c a l s u r v e y on the p r e v a l e n c e o f i n t e s t i n a l h e l m i n t h s was c o n d u c t e d a t the s t a r t o f the p r o j e c t , as w e l l as a s o c i o - e c o n o m i c s u r v e y of the s t u d y p o p u l a t i o n . M o r b i d i t y and m o r t a l i t y d a t a and r e c o r d s of v a c c i n a t i o n s were c o l l e c t e d from r e c o r d s m a i n t a i n e d by the CHWs d u r i n g the c o u r s e o f the p r o j e c t . A f o l l o w - u p demographic 67 s u r v e y was c o n d u c t e d i n 1978. S p e c i a l l y t r a i n e d e n u m e r a t o r s were t r a i n e d to c o l l e c t the d emographic d a t a . The e v a l u a t i o n p r o v i d e d d e t a i l s on some p r o c e s s and outcome m e a s u r e s . F o r example, the e v a l u a t i o n r e p o r t e d t h a t o v e r 907„ of c h i l d r e n aged 0-4 y e a r s r e c e i v e d t h r e e doses o f DTP v a c c i n e , and o v e r 857„ o f c h i l d r e n aged 0-14 y e a r s r e c e i v e d TB i m m u n i z a t i o n s . The e v a l u a t i o n a l s o r e p o r t e d t h a t as a r e s u l t o f t h e campaign to e r a d i c a t e n e o - n a t a l t e t a n u s , the i n f a n t m o r t a l i t y r a t e a t one o f t h e s t u d y s i t e s d e c r e a s e d from 117/1000 i n f a n t s to 106/1000 i n f a n t s o v e r the f i r s t y e a r of the p r o j e c t . T h i s t r e n d c o n t i n u e d t h r o u g h the s econd y e a r , and by 1978, i t was r e p o r t e d to be as low as the i n f a n t m o r t a l i t y r a t e s i n the o t h e r two s t u d y s i t e s . The n u t r i t i o n a l e d u c a t i o n program f o r mothers o f f e r e d by the CHWs a l s o was r e p o r t e d as b e i n g v e r y s u c c e s s f u l when compared to the same program o f f e r e d a t l o c a l n u t r i t i o n r e h a b i l i t a t i o n c e n t r e s . T h i s c o n c l u s i o n was b ased on s t a t i s t i c s w h i c h i n d i c a t e d t h a t w h i l e the p e r c e n t a g e of c h i l d r e n who g a i n e d w e i g h t o v e r th e s t u d y p e r i o d was h i g h e r f o r t h o s e a t t e n d i n g the r e h a b i l i t a t i o n c e n t r e program, the w e i g h t g a i n of c h i l d r e n whose mothers p a r t i c i p a t e d i n the community-based program was h i g h e r , and e x t e n d e d to s i b l i n g s . The i m p a c t of the program i n a f f e c t i n g m o r t a l i t y r a t e s f o r major d i s o r d e r s i s u n c l e a r from the i n f o r m a t i o n p r o v i d e d i n the e v a l u a t i o n . F o r example, o v e r the t h r e e y e a r p e r i o d t h a t the 68 e v a l u a t i o n c o v e r s , the m o r t a l i t y r a t e f o r a d u l t s aged 15-49 y e a r s f o r d e a t h s r e l a t e d to TB r e m a i n e d unchanged, as d i d the i n f a n t m o r t a l i t y r a t e f o r d e a t h s due to g a s t r o e n t e r i t i s . I n f a c t , t h e l a t t e r i n c r e a s e d o v e r the s t u d y p e r i o d . The r e s e a r c h team c o n c l u d e d t h a t the p r o j e c t had been a s u c c e s s as i t d e m o n s t r a t e d t h a t the n u t r i t i o n a l a s s e s s m e n t methods used had a g r e a t e r i m p a c t on i n f a n t growth than the r e h a b i l i t a t i o n c e n t r e - b a s e d program; t h a t m o b i l e teams of h e a l t h a u x i l i a r i e s had i n c r e a s e d a c c e s s to h e a l t h c a r e s e r v i c e s f o r the s t u d y p o p u l a t i o n ; and, t h a t n e o n a t a l t e t a n u s had been v i r t u a l l y e r a d i c a t e d i n the s t u d y a r e a o v e r the p r o j e c t p e r i o d . v. V o l u n t e e r H e a l t h Workers ( T h a i l a n d ) T h i s t h i r t y month p r o j e c t (1976-1978) was c o n d u c t e d by s t a f f o f the F a c u l t y o f M e d i c i n e , Khon Kaen U n i v e r s i t y , l o c a t e d i n Khon Kaen, T h a i l a n d . The p r o j e c t took p l a c e i n a r u r a l a r e a o f n o r t h e a s t T h a i l a n d . The o b j e c t i v e s of the p r o j e c t were to t r a i n l o c a l v i l l a g e r s as p r i m a r y h e a l t h c a r e w o r k e r s , use them as c o n t a c t s w i t h t h e e x i s t i n g h e a l t h c a r e s y s t e m , and e v a l u a t e them i n t h e i r t a s k s . The v o l u n t e e r h e a l t h w o r k e r s (VHWs) were t r a i n e d i n s u c h t a s k s as a d v i s i n g v i l l a g e r s on n u t r i t i o n and p r e v e n t i o n o f communicable d i s e a s e s , and i n p r o v i d i n g f i r s t a i d , m a l a r i a t r e a t m e n t , and d e l i v e r i n g b a b i e s a t home.(41) The r e s e a r c h e r s had o r i g i n a l l y i n t e n d e d to use a one group 69 p r e t e s t - p o s t t e s t e v a l u a t i v e r e s e a r c h d e s i g n . However, t h i s a p p r o a c h had to be abandoned as the q u a l i t y of d a t a c o l l e c t e d by the VHWs was i n a d e q u a t e and n o t o f a q u a l i t y w h i c h a l l o w e d c o m p a r i s o n w i t h b a s e l i n e d a t a . As a r e s u l t , the e v a l u a t i o n used a o n e - s h o t c a s e s t u d y d e s i g n . Demographic d a t a and v i t a l s t a t i s t i c s were c o l l e c t e d from a sample of 282 h o u s e h o l d s . T h i s f i g u r e r e p r e s e n t e d 507<> of a l l h o u s e h o l d s i n f o u r s t u d y v i l l a g e s ( t h e s t u y was to i n c l u d e n i n e s t u d y v i l l a g e s i n i t i a l l y ) . The VHWs were a l s o e x p e c t e d to m a i n t a i n r e c o r d s of a c t i v i t i e s p e r f o r m e d , to be t u r n e d i n t o the r e s e a r c h e r s on a mo n t h l y b a s i s . The VHWs were supposed to c o n d u c t b a s e l i n e s u r v e y s i n the s t u d y v i l l a g e s . They were to c o l l e c t m o r t a l i t y and m o r b i d i t y , d e m o g r a p h i c , and s o c i o - e c o n o m i c d a t a f o r each s t u d y v i l l a g e . They were a l s o to c o n d u c t a s u r v e y of e n v i r o n m e n t a l c o n d i t i o n s , and the a v a i l a b i l i t y o f improved w e l l s and l a t r i n e s . However, i n the end, t h e r e s e a r c h e r s took a random sample of 325 f a m i l i e s i n two of the s t u d y v i l l a g e s ( o u t o f a t o t a l o f 9 s t u d y v i l l a g e s ) . T h ese h o u s e h o l d s were i n t e r v i e w e d to c o l l e c t i n f o r m a t i o n on knowledge, a t t i t u d e s and p r a c t i c e s w i t h r e s p e c t to s a n i t a r y b e h a v i o u r , and m o r b i d i t y d a t a on the p r e v a l e n c e o f p a r a s i t i c i n f e c t i o n s . I n f o r m a t i o n on mother and c h i l d h e a l t h was s o l i c i t e d f r o m 311 of t h e s e same h o u s e h o l d s , w h i l e an u n s p e c i f i e d number o f s t u d y h o u s h o l d s were a s k e d q u e s t i o n s p e r t a i n i n g to t h e i r knowledge, a t t i t u d e s and p r a c t i c e s about 7 0 f a m i l y p l a n n i n g . The r e s u l t s o f the e v a l u a t i o n i n d i c a t e t h a t l e s s t h a n 5 0 7 o of the h o u s e h o l d s i n t e r v i e w e d s t a t e d t h a t they had used the VHW's s e r v i c e s . F o r t h o s e who d i d u t i l i z e t he VHW, the y s t a t e d t h a t t h e y were g e n e r a l l y s a t i s f i e d w i t h the s e r v i c e s r e n d e r e d . A t the time o f the b a s e l i n e s u r v e y , 4 2 7 . o f the r e p o n d e n t s s t a t e d t h a t t h e y used s e l f - m e d i c a t i o n and d r u g s t o r e s e r v i c e s f o r t h e i r h e a l t h n e e d s . The f o l l o w - u p s u r v e y i n d i c a t e d t h a t t h i s l e v e l had d e c r e a s e d to 1 6 7 . of the r e s p o n d e n t s . However, t h e r e was no i n d i c a t i o n whether the same h o u s e h o l d s were i n t e r v i e w e d , n o r what a f f e c t e d the d e c i s i o n of v i l l a g e r s to use t h e s e methods. The r e s e a r c h e r s c l a i m e d t h a t the VHWs were r e s p o n s i b l e f o r i n c r e a s i n g the use of f a m i l y p l a n n i n g methods. W h i l e the number of a c c e p t o r s d i d i n c r e a s e , a c c o r d i n g to the d a t a c o l l e c t e d ( f r o m 2 , 0 6 1 to 2 , 6 2 0 p e o p l e ) , t h e r e i s no c o r r a b o r a t i v e d a t a to s u p p o r t t h i s c l a i m . F o r example, i t was n e v e r e s t a b l i s h e d whether t h i s t r e n d was symptomatic o f a s i m i l a r t r e n d t h r o u g h o u t T h a i l a n d . The m o n t h l y r e p o r t s s u b m i t t e d by the VHWs i n d i c a t e d t h a t t h e y most f r e q u e n t l y t r e a t e d c o l d s , h e a d a c h e s , d y s e n t r y , and b u r n s . However, the q u a l i t y and r e l i a b i l i t y o f t h e s e d a t a a r e q u e s t i o n a b l e , s i n c e r e p o r t p r e p a r a t i o n and s u b m i s s i o n was v e r y i n c o n s i s t e n t . F o r example, the i n i t i a l r e s p o n s e r a t e f o r s u b m i t t i n g a c t i v i t y r e c o r d s a t the s t a r t of the p r o j e c t p e r i o d 0 was • v e r y low. The r e s e a r c h e r s d i s c o v e r e d t h a t the VHWs d i d n o t 71 know how to c o m p l e t e the f o r m s . F o l l o w i n g a s h o r t t r a i n i n g c o u r s e i n t h e v i l l a g e s , t h e r e s p o n s e r a t e f o r s u b m i t t i n g the a c t i v i t y r e p o r t s d i d i n c r e a s e to 587. ( 6 7 / 1 0 5 ) . T h i s i n c r e a s e d d u r i n g the s u b s e q u e n t months, r e a c h i n g an 807. r e s p o n s e r a t e a t one p o i n t . B u t a f t e r one y e a r , the p r o p o r t i o n o f VHWs s u b m i t t i n g a c t i v i t y r e p o r t s had d e c l i n e d to below 507.. C o n s e q u e n t l y , the e v a l u a t i o n was i n c o n c l u s i v e as to the e f f e c t o f the VHWs on the h e a l t h s t a t u s of t h e s t u d y p o p u l a t i o n . The r e s e a r c h e r s , n e v e r t h e l e s s , c o n c l u d e d t h a t the VHW program was a s u c c e s s , a l t h o u g h " i t i s too e a r l y to c o n c l u d e on i t s t o t a l i m p a c t " . v i . R u r a l T e a c h e r s H e a l t h Program ( P a r a g u a y ) The aim o f t h i s t h i r t y two month s t u d y (1978-1980) was to improve t h e h e a l t h s t a t u s o f the r u r a l p o p u l a t i o n by i m p l e m e n t i n g a p r i m a r y h e a l t h c a r e program u t i l i z i n g t e a c h e r s t r a i n e d i n and a s s i g n e d s p e c i f i c h e a l t h d u t i e s . The p r o j e c t was d e s i g n e d , i mplemented and e v a l u a t e d by s e n i o r s t a f f of the M i n i s t r y o f P u b l i c H e a l t h and W e l f a r e i n P a r a g u a y . The p r o j e c t was b a s e d on two f u n d a m e n t a l a s s u m p t i o n s : t h a t t h e t e a c h e r , i n h i s c a p a c i t y as a community l e a d e r , has a s i g n i f i c a n t i n f l u e n c e on c h i l d r e n and p a r e n t s , and c o u l d t h e r e f o r e be used to m o b i l i z e p e o p l e to s o l v e t h e i r own p r o b l e m s ; and, t h a t the s c h o o l c h i l d r e n c o u l d s e r v e as a l i n k between the t e a c h e r and 72 t h e i r f a m i l i e s to promote improvements i n e n v i r o n m e n t a l c o n d i t i o n s and h e a l t h b e h a v i o u r i n the home. The t e a c h e r s were e x p e c t e d to p r o v i d e p r e v e n t i v e h e a l t h e d u c a t i o n i n the p r i m a r y s c h o o l s , make home v i s i t s to promote b e t t e r h e a l t h and e n v i r o n m e n t a l c o n d i t i o n s , to p r o v i d e b a s i c c u r a t i v e s e r v i c e s ( p r i m a r i l y f i r s t a i d ) , and to d i a g n o s e and r e f e r more s e r i o u s c a s e s to l o c a l h e a l t h c l i n i c s . ( 4 2 ) To e v a l u a t e the im p a c t and p e r f o r m a n c e o f the t e a c h e r s , the r e s e a r c h e r s u t i l i z e d a one group p r e t e s t - p o s t t e s t e v a l u a t i v e r e s e a r c h d e s i g n . The p r o j e c t would be e v a l u a t e d a c c o r d i n g to the f o l l o w i n g i n d i c a t o r s : changes i n key m o r t a l i t y and m o r b i d i t y d a t a ; change i n e n v i r o n m e n t a l c o n d i t i o n s ; u t i l i z a t i o n of the t e a c h e r s f o r h e a l t h p r o b l e m s ; and, a c c e p t a n c e o f the t e a c h e r s by the community. A b a s e l i n e d e m o graphic and m o r b i d i t y s u r v e y was c o n d u c t e d f o r a l l h o u s e h o l d s i n the s t u d y a r e a . A permanent r e c o r d s y s t e m was e s t a b l i s h e d , to be m a i n t a i n e d by the t e a c h e r s , to g a t h e r d e m o g r a p h i c , m o r b i d i t y and m o r t a l i t y d a t a d u r i n g t h e c o u r s e o f the p r o j e c t . An e n d - o f - p r o j e c t s u r v e y to a s s e s s a c c e p t a n c e of the program was a l s o c o n d u c t e d . The e v a l u a t i o n r e p o r t e d t h a t the r u r a l h e a l t h p r o m o t e r s were r e f e r r i n g c a s e s to the r e g i o n a l h e a l t h c l i n i c , and t h a t the r e f e r r e d c a s e s were f o l l o w i n g r e f e r r a l i n s t r u c t i o n s they had been g i v e n . However, i t was d i f f i c u l t to e s t a b l i s h the a c t u a l number o f c a s e s s i n c e some o f the p a t i e n t s r e f e r r e d d i d n o t go to the h e a l t h c e n t r e to wh i c h t h e y were r e f e r r e d . In 73 some c a s e s the h e a l t h c l i n i c d i d n o t r e t a i n or f i l e the r e f e r r a l c a r d , or though the p a t i e n t s d i d t r a v e l to the h e a l t h c l i n i c , t h e y r e c e i v e d no t r e a t m e n t . The e v a l u a t i o n a l s o n o t e d t h a t a c c e s s i b i l i t y to p r i m a r y h e a l t h c a r e p e r s o n n e l had i n c r e a s e d i n t h e s t u d y a r e a , and t h a t a m o d i f i c a t i o n i n demand p a t t e r n s f o r h e a l t h p e r s o n n e l had a l s o o c c u r r e d , w i t h 13.77, o f demand f o r p r i m a r y c o n t a c t s d i r e c t e d towards the r u r a l h e a l t h p r o m o t e r s by the end of the s t u d y p e r i o d . An i n c r e a s e i n the u t i l i z a t i o n of t r a i n e d midwives f o r d e l i v e r i e s were a l s o n o t e d o v e r the s t u d y p e r i o d , a l t h o u g h t h e r e was no e v i d e n c e p r e s e n t e d whether t h i s was a c o n t i n u a n c e of a p r e - s t u d y t r e n d . I m m u n i z a t i o n r a t e s of 607, were r e p o r t e d , w i t h 957, of t h o s e immunized r e p o r t i n g t h a t they had a t t e n d e d t a l k s on the s u b j e c t p r i o r to the campaign. W h i l e th e m o r b i d i t y p a t t e r n of the s t u d y p o p u l a t i o n a p p a r e n t l y changed o v e r the s t u d y p e r i o d , the r e s e a r c h f a i l e d to d e m o n s t r a t e a c a u s a l l i n k between the a v a i l a b i l i t y o f r u r a l h e a l t h p r o m o t e r s and t h i s t r e n d . However, the program a p p e a r s to have had a s i g n i f i c a n t i m p a c t on the i n s t a l l a t i o n of l a t r i n e s . T h e r e a l s o a p p e a r s to be some a s s o c i a t i o n between t h i s and t h e lower f r e q u e n c y o f g a s t r o e n t e r i c d i s o r d e r s . D u r i n g the s t u d y p e r i o d , the p e r c e n t a g e i n c r e a s e i n the number of l a t r i n e s c o n s t r u c t e d was 3757., as compared f o r an i n c r e a s e of 1467, f o r t h e e n t i r e h e a l t h d i s t r i c t o v e r the same p e r i o d . D u r i n g t h i s same p e r i o d , m o r b i d i t y due to g a s t r o e n t e r i c 74 d i s e a s e s d r o p p e d from 260/1000 to 170/1000 p e r s o n s . Improvements i n the s a n i t a r y c o n d i t i o n s of v i l l a g e homes were a l s o r e c o r d e d , as i l l u s t r a t e d by the f a c t t h a t the p r o p o r t i o n o f homes where g a r b a g e was b u r n e d r o s e from 537. to 687. o v e r the s t u d y p e r i o d . An i n c r e a s e i n h o u s e h o l d s u s i n g w a t e r drawn from the v i l l a g e w e l l s was a l s o n o t e d . As the s c h o o l c h i l d r e n had been t h e r e c i p i e n t s of a major h e a l t h e d u c a t i o n program, i t was assumed t h a t t h i s program was r e s p o n s i b l e f o r t h e c h a n g e s . However, h i s t o r i c a l d a t a w h i c h may have p r o v i d e d s u p p o r t was u n a v a i l a b l e to c o r r o b o r a t e t h i s c o n e l u s i o n . T h e r e was an i n d i c a t i o n t h a t the i n c r e a s e d r e s p o n s i b i l i t i e s g i v e n to the t e a c h e r s a f f e c t e d the t e a c h e r s ' c a p a c i t y to f u n c t i o n e f f e c t i v e l y . I n v o l v e m e n t i n the program was v o l u n t a r y and r e q u i r e d the t e a c h e r s to d o n a t e t h e i r f r e e time to h e a l t h a c t i v i t i e s . Over the c o u r s e o f the p r o j e c t , a s i g n i f i c a n t p r o p o r t i o n o f the t e a c h e r s i n i t i a l l y i n v o l v e d i n the p r o j e c t (207.) f o u n d t h i s u n a c c e p t a b l e and r e s i g n e d from i t . How t h i s a f f e c t e d the o v e r a l l i m p a c t of t h e program was n o t r e c o r d e d i n the e v a l u a t i o n s t u d y . A l t h o u g h i t was n o t a s p e c i f i c o b j e c t i v e o f the p r o j e c t , t h e e v a l u a t i o n i n d i c a t e d t h a t t h e c h i l d r e n a r e i m p o r t a n t change a g e n t s i n the c o m m u n i t i e s . I t was n o t e d t h a t they e x e r c i s e d a s i g n i f i c a n t e f f e c t on the d e c i s i o n s o f t h e i r p a r e n t s to seek p r e v e n t i v e h e a l t h c a r e and improve e n v i r o n m e n t a l and s a n i t a r y 75 c o n d i t i o n s . Based on the e v a l u a t i v e r e s e a r c h f i n d i n g s , the r e s e a r c h e r s c o n c l u d e d t h a t t h e r u r a l t e a c h e r s c o u l d be t r a i n e d to p r o v i d e c e r t a i n h e a 1 t h - r e l a t e d a c t i v i t i e s , and t h a t t h i s was an a c c e p t a b l e a r r a n g e m e n t to the c o m m u n i t i e s . However, the e v a l u a t i o n f a i l e d to p r o v i d e c o n c l u s i v e e v i d e n c e t h a t the r e t r a i n e d r u r a l t e a c h e r s were d i r e c t l y r e s p o n s i b l e f o r changes i n m o r t a l i t y and m o r b i d i t y r a t e s . A n a l y s i s o f the E v a l u a t i o n R e p o r t s The f o r e g o i n g p r o v i d e s a d e s c r i p t i v e o v e r v i e w of the o b j e c t i v e s , a c t i v i t i e s and i n d i c a t o r s used to e v a l u a t e the f i v e c a s e s t u d i e s s e l e c t e d f o r t h i s r e v i e w . I n summary, the e v a l u a t i o n s c l a i m e d t h a t the d e p l o y m e n t of community-based h e a l t h w o r k e r s a f f e c t e d p o s i t i v e l y the h e a l t h of the t a r g e t p o p u l a t i o n s . Y e t even a s u p e r f i c i a l a n a l y s i s o f t h e c a s e s t u d i e s seems to r e v e a l t h a t s u c h a c l a i m i s u n s u b s t a n t i a t e d . The r e m a i n d e r o f t h i s c h a p t e r a n a l y z e s t h e f i v e c a s e s t u d i e s i n o r d e r to i d e n t i f y t h o s e f a c t o r s which l i m i t the v a l i d i t y and u t i l i t y of the e v a l u a t i o n r e s u l t s . P e r h a p s the p r i m a r y l i m i t i n g f a c t o r r e l a t e s to the n a t u r e of the measures used to a s s e s s p r o j e c t e f f e c t i v e n e s s . I n d i c e s b ased on m o r t a l i t y ( s u c h as c r u d e and a g e - a d j u s t e d d e a t h r a t e s , i n f a n t m o r t a l i t y r a t e s , and t h e e x p e c t a t i o n of l i f e ) and on m o r b i d i t y ( p r e v a l e n c e and i n c i d e n c e o f major d i s e a s e 76 c a t e g o r i e s ) have t r a d i t i o n a l l y been used as measures of improvement i n h e a l t h s t a t u s . C o n t r o l of m o r t a l i t y and m o r b i d i t y have u s u a l l y been the paramount g o a l s o f h e a l t h p r o g r a m s . V a r i a t i o n s i n d e a t h and i l l n e s s r a t e s have been c o n s i d e r e d as d i r e c t measures of p r o g r e s s toward t h a t g o a l . ( 4 3 ) The s u p posed i m p o r t a n c e of t h e s e i n d i c e s as c r i t e r i a o f s u c c e s s f o r h e a l t h c a r e programs i s i l l u s t r a t e d i n t h e f o l l o w i n g s t a t e m e n t : The u l t i m a t e g o a l of a h e a l t h program i s to r e d u c e m o r b i d i t y and m o r t a l i t y i n the p o p u l a t i o n c o v e r e d . A l l d e s c r i p t i o n s a b o u t a c c e p t a b i l i t y , c o v e r a g e , c o s t s , or f e a s i b i l i t y , and q u a l i t y c o n t r o l p r o c e s s a r e m e a n i n g l e s s to h e a l t h o f f i c i a l s u n l e s s some c o n c r e t e outcome i n r e d u c e d i l l n e s s and d e a t h i s d e m o n s t r a t e d . (44) But w h i l e changes i n h e a l t h s t a t u s may be the b e s t i n d i c a t o r of i m p a c t , s u c h d a t a a r e e x p e n s i v e , d i f f i c u l t , and o f t e n t i m e - c o n s u m i n g to c o l l e c t , a n a l y z e and i n t e r p r e t . They o f t e n y i e l d ambiguous r e s u l t s . T h i s may be due i n p a r t to the f a c t t h a t t h e r e i s no f u l l y s a t i s f a c t o r y d e f i n i t i o n of h e a l t h t h a t p e r m i t s the use of c o m p r e h e n s i v e measurements, or the e s t a b l i s h m e n t o f c o n c l u s i v e r e l a t i o n s h i p s between h e a 1 t h - r e l a t e d i n t e r v e n t i o n s and change i n h e a l t h s t a t u s . The c o n c e p t o f h e a l t h as the 'absence of d i s e a s e ' i s u n s a t i s f a c t o r y f o r s e v e r a l r e a s o n s . F i r s t , h e a l t h i s more t h a n j u s t the a b s e n c e of d i s e a s e . C o n s i d e r a t i o n s o f consumer s a t i s f a c t i o n o r q u a l i t y of l i f e a r e 77 a l s o i m p o r t a n t c o n s i d e r a t i o n s . A d d i t i o n a l l y , the i n t e r r e l a t i o n s h i p between h e a l t h c a r e programs and h e a l t h s t a t u s i s complex. S t a t i s t i c a l i n d i c a t o r s s u c h as m o r b i d i t y and m o r t a l i t y may r e f l e c t changes t h a t have n o t d i r e c t l y r e s u l t e d from h e a l t h s e r v i c e d e l i v e r y p rograms, b u t r a t h e r from improvements i n o t h e r s e c t o r s . ( 4 5 ) Due to t h e s e s h o r t c o m i n g s , more a d e q u a t e and a p p r o p r i a t e measures to e v a l u a t e t h e i mpact and p e r f o r m a n c e of h e a l t h c a r e programs were p r o p o s e d . S econd, the d e f i n i t i o n o f h e a l t h i s a f u n c t i o n o f c u l t u r e . B o t h the c o n c e p t o f d i s e a s e and i t s c a u s a t i o n a r e c u l t u r e - b o u n d . What i s p e r c e i v e d as i l l - h e a l t h i n one community or segment o f the p o p u l a t i o n may n o t be s i m i l a r l y p e r c e i v e d i n a n o t h e r community or p o p u l a t i o n s u b g r o u p . T h i s may be due to d i f f e r e n c e s i n c u l t u r e . T h i r d , to measure whether, o r to what d e g r e e , a p a r t i c u l a r d i s e a s e or c o n d i t i o n i s p r e s e n t i n a p o p u l a t i o n i g n o r e s the m a c r o - l e v e l r e l a t i o n s h i p s between h e a l t h and i l l - h e a l t h . F o c u s i n g on one p a r t i c u l a r c o n d i t i o n may be i m p r a c t i c a l or u n r e a l i s t i c when o t h e r d i s e a s e s e x i s t w h i c h may have an e q u a l l y d e b i l i t a t i n g e f f e c t . As S c h a e f e r p o i n t s o u t , " t o be s a v e d from m a l a r i a so as to s u f f e r or d i e from m a l n u t r i t i o n o f f e r s a poor c h o i c e " . ( 4 6 ) The l e s s o n h e r e i s t h a t an e v a l u a t i o n , i f i t i n t e n d s to document the r e l a t i o n s h i p between i n t e r v e n t i o n s and h e a l t h s t a t u s , must be c o m p r e h e n s i v e . T h i s e n t a i l s a t h o r o u g h knowledge o f the i n t e r - r e l a t i o n s h i p s among i d e n i t i f i e d 78 c o n d i t i o n s o f i l l - h e a l t h , an t h e i r c u m u l a t i v e e f f e c t on h e a l t h s t a t u s . I t i s a l s o d i f f i c u l t to ' p r o v e ' t h a t h e a l t h i n t e r v e n t i o n s l e a d to improvements i n h e a l t h . I t i s n o t s i m p l y a m a t t e r of d o c u m e n t i n g the a v a i l a b i l i t y and u t i l i z a t i o n o f t e c h n i c a l i n t e r v e n t i o n s . I f t h e s e i n t e r v e n t i o n s a r e e x p e c t e d to be e f f e c t i v e , t h e n p r e v a i l i n g b e h a v i o u r p a t t e r n s must a l s o be a f f e c t e d . But to document the p r o c e s s of b e h a v i o u r a l change i s b o t h a t i m e - c o n s u m i n g and e x p e n s i v e e x e r c i s e . F o u r of the f i v e c a s e s t u d i e s c i t e d above used m o r t a l i t y and m o r b i d i t y d a t a to e v a l u a t e the i m p a c t of the PHC p r o j e c t s . Y e t the v a l i d i t y of t h e s e measures i s put i n t o doubt when examined c l o s e l y . F o r example, i n the I r a n p r o j e c t , the r e s e a r c h e r s c l a i m e d t h a t the e f f e c t i v e n e s s of t h e a u x i l i a r y h e a l t h w o r k e r s was d e m o n s t r a t e d . They based t h i s c l a i m on e v i d e n c e p r e s e n t e d to show t h a t the i n f a n t m o r t a l i t y r a t e i n the s t u d y v i l l a g e s was l e s s t h a n i n the c o n t r o l v i l l a g e s a t the end of the p r o j e c t (84/1000 i n f a n t s v e r s u s 138/1000 i n f a n t s r e s p e c t i v e l y ) . However, the a c t u a l t r e n d f o r i n f a n t m o r t a l i t y s t a t i s t i c s i n the s t u d y and c o n t r o l r e g i o n s was t h e f o l l o w i n g : IMR/1000 I n f a n t s S t u d y v i l l a g e s C o n t r o l V i l l a g e s 1972 36 134 1975 64 128 1978 84 138 (47) 79 I f one a c c e p t s t h i s d a t a as v a l i d , t h e n i t d e n o t e s a d i f f e r e n t i n t e r p r e t a t i o n from t h a t r e p o r t e d by the I r a n i a n r e s e a r c h e r s i n t h e i r e v a l u a t i o n r e p o r t . W h i l e the i n f a n t m o r t a l i t y r a t e i n the s t u d y v i l l a g e s was lower t h a n t h a t i n the c o n t r o l v i l l a g e s a t the end of t h e p r o j e c t p e r i o d , t h i s was n o t a r e c e n t phenomenon. I n f a c t , the i n f a n t m o r t a l i t y r a t e i n the s t u d y v i l l a g e s was h i s t o r i c a l l y i n f e r i o r to t h a t r e c o r d e d i n the c o n t r o l v i l l a g e s . Y e t no c o m p a r a t i v e a n a l y s i s o f t h i s t r e n d was r e p o r t e d i n the p r o j e c t e v a l u a t i o n . A s e c o n d major f l a w i n t h e e v i d e n c e p r e s e n t e d i n the p r o j e c t e v a l u a t i o n i s t h a t o v e r the s t u d y p e r i o d , t h e i n f a n t m o r t a l i t y r a t e r e c o r d e d i n t h e s t u d y v i l l a g e s i n c r e a s e d s i g n i f i c a n t l y . However, the i n f a n t m o r t a l i t y r a t e i n the c o n t r o l v i l l a g e s a c t u a l l y d e c r e a s e d o v e r the f i r s t t h r e e y e a r s of the p r o j e c t , b e f o r e i n c r e a s i n g towards the end. G i v e n t h a t t h e t o t a l p o p u l a t i o n f o r t h e s t u d y v i l l a g e s was a p p r o x i m a t e l y 10,000 p e o p l e , and a s s u m i n g t h a t the p r o p o r t i o n of i n f a n t s aged l e s s t h a n f i v e y e a r s of age c o n s t i t u t e d a p p r o x i m a t e l y 207, of t h i s p o p u l a t i o n , t h e n i t c o u l d be e s t i m a t e d t h a t t h e r e were a p p r o x i m a t e l y 2,000 i n f a n t s r e s i d e n t i n the s t u d y v i l l a g e s . Based on t h e r e c o r d e d i n f a n t m o r t a l i t y d a t a , the a c t u a l number of i n f a n t s to d i e i n 1972 and 1978 i n c r e a s e d from 52 to 168 ( a s s u m i n g a c o n s t a n t p o p u l a t i o n f o r t h i s age group o v e r the s t u d y p e r i o d ) . T h i s c o n s t i t u t e s a 2237. i n c r e a s e i n i n f a n t m o r t a l i t y . By c o m p a r i s o n , i n f a n t m o r t a l i t y i n c r e a s e d by 37. i n 80 the c o t r o l v i l l a g e s o v e r the sae p e r i o d . T h i s would seem to r e f u t e the r e s e a r c h e r s ' c l a i m t h a t the a u x i l i a r i e s were e f f e c t i v e i n i m p r o v i n g h e a l t h s t a t u s . U s i n g i n f a n t m o r t a l i t y as an i n d i c a t o r of community h e a l t h s t a t u s was n o t p r a c t i c a l o r v a l i d i n t h i s c a s e . A d d i t i o n a l l y , t h e r e s e a r c h e r s d i d n o t p r o v i d e s t a t i s t i c a l i n f o r m a t i o n f o r o t h e r i l l - h e a l t h c o n d i t i o n s i n the s t u d y and c o n t r o l v i l l a g e s . One i s l e f t w i t h the i m p r e s s i o n t h a t i n f a n t m o r t a l i t y r e p r e s e n t s a l l i l l - h e a l t h c o n d i t i o n s i n the s t u d y a r e a , an u n r e a l i s t i c a s s u m p t i o n a t b e s t . By r e s t r i c t i n g t h e range of c o n d i t i o n s to be measured, the r e s e a r c h e r s i n e f f e c t l i m i t the v a l i d i t y of t h e i r r e s e a r c h . The n a t u r e and means of u s i n g m o r b i d i t y and m o r t a l i t y d a t a a l s o l i m i t e d the c o n c l u s i o n s w h i c h c o u l d be drawn from the e v a l u a t i o n o f the C o l o m b i a n p r o j e c t . In t h i s c a s e , the c l a i m c o n c e r n i n g the s u c c e s s of the p r o j e c t was based on e v i d e n c e drawn from s e v e r a l d i f f e r e n t measures ( s e e T a b l e 2). However, w h i l e the m o r b i d i t y and m o r t a l i t y d a t a used would seem to i n d i c a t e t h a t improvements i n h e a l t h o c c u r e d f o r s p e c i f i c p o p u l a t i o n g r o u p s o v e r the s t u d y p e r i o d , i t i s d i f f i c u l t to b ase the i m p a c t o f the program on the e v i d e n c e p r e s e n t e d . F i r s t , as i n the I r a n p r o j e c t e v a l u a t i o n , h i s t o r i c a l m o r b i d i t y and m o r t a l i t y r a t e s were n o t e s t a b l i s h e d or r e p o r t e d . T h e r e f o r e , t h e m o r b i d i t y and m o r t a l i t y d a t a ' c i t e d i n the r e p o r t c o u l d n o t be s e e n i n p e r s p e c t i v e of h i s t o r i c a l t r e n d s o r 81 p a t t e r n s . P e r h a p s the d e c r e a s e i n the i n f a n t m o r t a l i t y r a t e c i t e d i n the p r o j e c t e v a l u a t i o n was s i m p l y an e x t e n s i o n o f a t r e n d , and n o t the r e s u l t o f the im p a c t o f p r o j e c t a c t i v i t i e s . S e c o n d , no d a t a f o r c o n t r o l v i l l a g e s were r e c o r d e d or a n a l y z e d . I t i s d i f f i c u l t to c l a i m s u c c e s s o f an i n t e r v e n t i o n w i t h o u t the p r e s e n c e of c o m p a r a t i v e d a t a f o r s t u d y p o p u l a i o n s w h i c h d i d n o t r e c e i v e the i n t e r v e n t i o n . W h i l e c e r t a i n p a t t e r n s o f i n f a n t m o r t a l i t y were r e c o r d e d i n the s t u d y v i l l a g e s , t h e r e s e a r c h e r s p r o v i d e d no d a t a w i t h w h i c h to d e t e r m i n e i f s u c h t r e n d s were u n i q u e to t h e a r e a . T h i r d , w h i l e the C o l o m b i a n r e s e a r c h e r s d i d use a w i d e r range o f d i s e a s e c a t e g o r i e s to c a l c u l a t e m o r b i d i t y d a t a t h a n t h e I r a n p r o j e c t d i d , t h e s e d a t a were r e p r e s e n t a t i v e o n l y of s p e c i f i c p o p u l a t i o n g r o u p s . They do n o t n e c e s s a r i l y r e f l e c t t h e i l l - h e a l t h c o n d i t i o n s w h i c h c h a r a c t e r i z e the m a j o r i t y o f the s t u d y p o p u l a t i o n . The r e s e a r c h e r s use a narrow band of i l l - h e a l t h c o n d i t i o n s to g e n e r a l i z e a b o u t community h e a l t h s t a t u s . C o n s e q u e n t l y , the v a l i d i t y and the u t i l i t y of the r e s e a r c h r e s u l t s a r e thrown i n t o d o u b t . I n t h e R u r a l H e a l t h and F a m i l y P l a n n i n g P r o j e c t ( H a i t i ) , t h e m o r b i d i t y and m o r t a l i t y d a t a r e c o r d e d were n e i t h e r c o m p r e h e n s i v e , n o r c o n s i s t e n t i n the f o r m a t i n w h i c h they were r e p o r t e d . F o r example, d a t a on the p r e v a l e n c e of t u b e r c u l o s i s i n the t h r e e s t u d y v i l l a g e s was r e p o r t e d as an a v e r a g e o v e r the s t u d y p e r i o d . The l a c k o f d i s a g g r e g a t e d s t a t i s t i c s does n o t a l l o w f o r c o m p a r i s o n among t h e s e v i l l a g e s , n o r a c r o s s t i m e . 82 A d d i t i o n a l l y , the d a t a r e c o r d e d r e l a t e d to i l l - h e a l t h c o n d i t i o n s f o r s p e c i f i c p o p u l a t i o n s u b g r o u p s . T h e r e were no c o m p a r a t i v e s t a t i s t i c s from p o p u l a t i o n s o u t s i d e the s t u d y a r e a , n o r was t h e r e an e x a m i n a t i o n of i l l - h e a l t h c o n d i t i o n s r e p r e s e n t a t i v e of the e n t i r e s t u d y p o p u l a t i o n . The s t a t i s t i c s r e p o r t e d i n the e v a l u a t i o n r e o r t do n o t p r o v i d e s u f f i c i e n t e v i d e n c e to s u g g e s t t h a t t h e PHC a p p r o a c h use i n t h i s p a r t i c u l a r p r o j e c t had a p o s i t i v e i m p a c t on the h e a l t h of the i n t e n d e d b e n e f i c i a r i e s . The R u r a l T e a c h e r s H e a l t h Program ( P a r a g u a y ) r e c o r d e d m o r b i d i t y d a t a f o r t h o s e d i s e a s e s most common to t h e s t u d y p o p u l a t i o n . T h e s e s t a t i s t i c s were c o l l e c t e d a t the s t a r t and a t t h e t e r m i n a t i o n o f the p r o j e c t . Over the s t u d y p e r i o d , the r e s e a r c h e r s f o u n d t h a t the p r e v a l e n c e o f t h e s e d i s e a s e s tended to d e c r e a s e i n the s t u d y c o m m u n i t i e s . However, as w i t h the p r e v i o u s l y c i t e d s t u d i e s , no c o m p a r a t i v e d a t a was c o l l e c t e d . C o n s e q u e n t l y , the r e s e a r c h e r s d i d n o t e s t a b l i s h whether the r e c o r d e d m o r b i d i t y t r e n d s and p a t t e r n s were u n i q u e to the s t u d y a r e a , or were c o n s i s t e n t w i t h m o r b i d i t y t r e n d s and p a t t e r n s e l s e w h e r e i n P a r a g u a y . A n o t h e r s h o r t c o m i n g w h i c h c h a r a c t e r i z e d t h e p r o j e c t e v a l u a t i o n s w h i c h used g e n e r a l m o r b i d i t y r a t e s f o r m e a s u r i n g i m p a c t i s t h a t t h e s e measures t e n d to subsume the e x i s t e n c e o f d i f f e r e n t i a l r a t e s f o r v a r i o u s p o p u l a t i o n g r o u p s and a c r o s s d i s e a s e s . Not a l l members of the s t u d y p o p u l a t i o n have the same 83 d i s e a s e o r i l l - h e a l t h c o n d i t i o n , a t the same l e v e l o f s e v e r i t y , o r a t the same p o i n t i n the d i s e a s e c y c l e . L i k e w i s e , t h o s e who a r e i l l may be the v i c t i m s o f m u l t i p l e i l l - h e a l t h c o n d i t i o n s s i m u l t a n e o u s l y , a l t h o u g h t h e y a r e d e m o n s t r a t i n g symptoms o f o n l y one d i s e a s e . T h e r e f o r e , i t i s d i f f i c u l t to d e t e r m i n e whether or n o t a p p r o a p r i a t e and a d e q u a t e h e a l t h c a r e s e r v i c e s a r e b i e n g p r o v i d e d by c o l l e c t i n g m o r b i d i t y d a t a a c r o s s an e n t i r e s t u d y p o p u l a t i o n a t a u n i q u e p o i n t i n t i m e . The i n d i c e s u s e d to measure p r o j e c t p e r f o r m a n c e o r o u t p u t u s u a l l y f o c u s on the a v a i l a b i l i t y o f h e a l t h s e r v i c e s and p e r s o n n e l , and on the n a t u r e and f r e q u e n c y of t a s k s p e r f o r m e d . But t h i s i n f o r m a t i o n p r o v i d e s l i t t l e i f any i n d i c a t i o n of the b e n e f i t s a c c r u i n g to the r e c i p i e n t p o p u l a t i o n . Even when b o t h i m p a c t and p e r f o r m a n c e i n d i c e s a r e u s e d , l i t t l e e f f o r t i s made to i d e n t i f y the n a t u r e o r the d e g r e e o f the r e l a t i o n s h i p between the c h a r a c t e r i s t i c s o f t h e h e a l t h c a r e d e l i v e r y system and h e a l t h . As one a u t h o r s t a t e s , "we do n o t know v e r y much a b o u t how or whether most h e a l t h p e r s o n n e l s e r v i c e p r o c e d u r e s a r e r e l a t e d to improvements i n the h e a l t h of i n d i v i d u a l s [ o r ] of p o p u l a t i o n s " . ( 4 8 ) One i s u s u a l l y l e f t to assume t h a t i f p o s i t i v e changes i n h e a l t h s t a t u s i n d i c e s o c c u r o v e r the p r o j e c t p e r i o d , t h a t the h e a l t h c a r e s e r v i c e s d e l i v e r e d were a p p r o p r i a t e , and the q u a l i t y o f c a r e a d e q u a t e . Y e t s u c h i s n o t n e c e s s a r i l y t h e c a s e . T h i s a n a l y s i s has d e m o n s t r a t e d t h a t the use o f m o r b i d i t y 84 and m o r t a l i t y d a t a i n the c a s e e v a l u a t i o n s p r e s e n t e d have n o t s u p p o r t e d c l a i m s t h a t the PHC a p p r o a c h e s used have had e i t h e r a p o s i t i v e or s i g n i f i c a n t e f f e c t on h e a l t h . F i r s t , the range o f i l l - h e a l t h c o n d i t i o n s c i t e d r e p r e s e n t o n l y a s u b s e t o f the c o n d i t i o n s w h i c h a f f e c t any community. S e c o n d , some o f the measures were r e c o r d e d o n l y a t one p o i n t i n t i m e . T h i r d , c o m p a r a t i v e i n f o r m a t i o n was n o t r e c o r d e d f o r p o p u l a t i o n s n o t r e c e i v i n g the i n t e r v e n t i o n s . S t a n d i n g on t h e i r own m e r i t , such d a t a p r o v i d e l i t t l e s u b s t a n t i v e i n f o r m a t i o n . A d d i t i o n a l l y , the l a c k o f a h i s t o r i c a l p e r s p e c t i v e o f m o r b i d i t y and m o r t a l i t y d a t a a l s o l i m i t s the v a l i d i t y and u t i l i t y o f the r e c o r d e d d a t a . A n o t h e r f a c t o r w h i c h l i m i t s the c r e d i b i l i t y of the m o r b i d i t y and m o r t a l i t y d a t a r e c o r d e d i n t h e s e e v a l u a t i o n s t u d i e s i s the time i n t e r v a l o v e r w h i c h t h e d a t a a r e compared, p a r t i c u l a r l y the time i n t e r v a l between the i n t r o d u c t i o n of the i n t e r v e n t i o n and t h e u l t i m a t e c o l l e c t i o n o f d a t a . T h e r e a p p e a r s to be an e x p e c t a t i o n t h a t changes i n h e a l t h s t a t u s w i l l o c c u r s h o r t l y a f t e r t h e i n t r o d u c t i o n o f h e a 1 t h - r e l a t e d i n t e r v e n t i o n s . Y e t b e f o r e one can e x p e c t h e a l t h to change, a t t i t u d e s and b e l i e f s w h i c h a f f e c t b e h a v i o u r must be changed. T h i s i s a l o n g and complex p r o c e s s . A l s o , a s s u m i n g t h a t p r o j e c t a c t i v i t i e s w i l l p r o c e e d as p l a n n e d i s u n r e a l i s t c . Most p r o j e c t s i n c u r d e l a y s , many due to u n e x p e c t e d e v e n t s . T y p i c a l l y , p r o j e c t s f a l l b e h i n d s c h e d u l e b e c a u s e o f d e l a y s i n c u r r e d i n the t r a n s f e r and a v a i l a b i l i t y o f f u n d s ; d e l a y s i n the t r a i n i n g and deployment o f 85 c o m m u n i t y - l e v e 1 h e a l t h w o r k e r s ; d e l a y s i n r e c e i v i n g equipment and s u p p l i e s ; and, d e l a y s i n p l a n n e d a c t i v i t i e s due to i n c l e m e n t weather c o n d i t i o n s , u n a v a i l a b i l i t y of t r a n s p o r t a t i o n , o r more s e r i o u s l y , p o l i t i c a l o r c i v i l u n r e s t i n t h e s t u d y a r e a . The c u m u l a t i v e e f f e c t of s u c h f a c t o r s can have a d e l i t e r i o u s e f f e c t on t h e e x p e c t a t i o n s o f p r o j e c t s t a f f w i t h r e s p e c t to the i m p a c t o f p r o j e c t a c t i v i t i e s on h e a l t h . The l o n g e s t p r o j e c t p e r i o d was f i v e y e a r s ( C o l o m b i a and I r a n ) . But the a c t u a l p e r i o d between the i n t r o d u c t i o n of the h e a l t h - r e l a t e d i n t e r v e n t i o n s and the p o i n t i n time when the f i n a l s u r v e y s were c a r r i e d o u t were much s h o r t e r . F o r example, the community h e a l t h w o r k e r s i n t h e H a i t i and I r a n i a n p r o j e c t s were n o t d e p l o y e d a t the s t a r t of t h e s e p r o j e c t s , n o r to a l l s t u d y v i l l a g e s s i m u l t a n e o u s l y . I n o t h e r c a s e s , equipment and s u p p l i e s were l a t e i n a r r i v i n g a t p r o j e c t s i t e s , t h e r e b y c a u s i n g s i g n i f i c a n t d e l a y s i n the s t a r t - u p o f p r o j e c t a c t i v i t i e s ( C o l o m b i a and P a r a g u a y ) . I n the P a r a g u a y p r o j e c t , i n c l e m e n t w e a t h e r and poor r o a d s were r e s p o n s i b l e f o r r e d u c i n g the number o f s u p e r v i s o r y and m o n i t o r i n g v i s i t s by the r e s e a r c h team to s e v e r a l s c h o o l s , p a r t i c u l a r l y t h o s e more i n a c c e s s i b l e l o c a t i o n s i n the s t u d y a r e a . Due to t h e i r a l r e a d y heavy w o r k l o a d , and the f a c t t h a t t h e i r h e a l t h a c t i v i t i e s were v o l u n t a r y and p e r f o r m e d i n t h e i r s p a r e t i m e , the t e a c h e r s i n v o l v e d i n the program r e q u i r e d c l o s e s u p e r v i s i o n and e n couragement. As the f i n a l e v a l u a t i o n r e p o r t p o i n t s o u t , the 86 s u p e r v i s i o n p r o v i d e d was i n a d e q u a t e to e n s u r e a c o n t i n u o u s and c o m p r e h e n s i v e program. T h i s c o u l d c o n c e i v a b l y a f f e c t the i n t e n s i t y o f t h e i r h e a l t h - r e l a t e d a c t i v i t i e s , and t h e r e f o r e t h e i r i m p a c t on the community. Had t h e p r o j e c t p e r i o d been l o n g e r , more s u p e r v i s i o n m i g h t have been p r o v i d e d . As i t was, the p r o j e c t p e r i o d was too s h o r t to d e m o n s t r a t e any s i g n i f i c a n t change i n h e a l t h . The e f f e c t i v e p r o j e c t time p e r i o d was s i g n i f i c a n t l y r e d u c e d i n t h e s e p r o j e c t s . The USAID r e p o r t s u g g e s t s t h a t f i v e to t e n y e a r s , r a t h e r t h a n t h r e e , be a d o p t e d as a more r e a l i s t i c t i m e f r a m e i n w h i c h to implement p r o j e c t a c t i v i t i e s s a t i s f a c t o r i l y , and i n w h i c h to measure h e a l t h i n d i c e s . ( 4 9 ) A n o t h e r f a c t o r w h i c h l i m i t e d the u s e f u l n e s s and v a l i d i t y of t h e f i v e c a s e s t u d y e v a l u a t i o n s was the e v a l u a t i v e r e s e a r c h d e s i g n u s e d . Two o f t h e f i v e c a s e s c i t e d ( I r a n and H a i t i ) o r i g i n a l l y i n t e n d e d to use a t r u e e x p e r i m e n t a l d e s i g n (a p r e t e s t - p o s t t e s t c o n t r o l group d e s i g n ) . However, f o r a v a r i e t y o f r e a s o n s , none of w h i c h were e x p l a i n e d i n t h e e v a l u a t i o n s , the a c t u a l r e s e a r c h d e s i g n used was n o n - e x p e r i m e n t a l . The r a t i o n a l e f o r u s i n g a t r u e e x p e r i m e n t a l d e s i g n i s to d e m o n s t r a t e t h a t t h e i n t e r v e n t i o n c a u s e d a s i g n i f i c a n t d i f f e r e n c e i n s e l e c t e d measures between the s t u d y and c o m p a r i s o n ( c o n t r o l ) p o p u l a t i o n s o v e r t i m e . Such p a r a d i g m s r e q u i r e two c o n d i t i o n s : t h a t s u b j e c t s be r andomly a l l o c a t e d to the t r e a t m e n t and c o n t r o l g r o u p s ; and, t h a t p r e t r e a t m e n t 87 measures be t a k e n i n b o t h p o p u l a t i o n s . The random a l l o c a t i o n o f s u b j e c t s to t r e a t m e n t and c o n t r o l g r o u p s c o n s t i t u t e s t h e c o r n e r s t o n e o f the e x p e r i m e n t a l p a r a d i g m . R a n d o m i z a t i o n i s a p o w e r f u l t o o l i n t h a t i t i n c r e a s e s t h e p r o b a b i l i t y t h a t t r e a t m e n t and c o n t r o l g r o u p s w i l l be s i m i l a r so t h a t a l t e r n a t i v e e x p l a n a t i o n s f o r program r e s u l t s may be r u l e d o u t . I n random s e l e c t i o n the e x p e r i m e n t a l group and c o n t r o l g r o u p s a r e b o t h drawn from the same p o p u l a t i o n . S e l e c t i o n of p e r s o n s f o r t h e two g r o u p s i s c a r r i e d o u t i n such a way t h a t e a c h p e r s o n , or segment of the p o p u l a t i o n ( d e p e n d e n t on the s e l e c t i o n c r i t e r i a ) has the same p r o b a b i l i t y o f b e i n g s e l e c t e d i n e i t h e r g r o u p . A s e c o n d f a c t o r r e q u i r e d f o r the c o r r e c t c o m p a r a t i v e a n a l y s i s of the t r e a t m e n t ( e x p e r i m e n t a l ) and n o n - t r e a t m e n t ( c o n t r o l ) g r o u p s i s t h e use of p r e t r e a t m e n t m e a s u r e s . These measures compare the e x p e r i m e n t a l and t h e c o n t r o l g r o u p s b e f o r e the e x p e r i m e n t and d e f i n e t h o s e v a r i a b l e s c o n s i d e r e d to be r e l e v a n t to the c r i t e r i o n t h a t w i l l be used to measure the change. The u t i l i z a t i o n o f measurement b e f o r e the e x p e r i m e n t s e r v e s two p u r p o s e s . F i r s t , i t p e r m i t s d e m o n s t r a t i o n t h a t the e x p e r i m e n t a l and c o n t r o l g r o u p s a r e e q u i v a l e n t to b e g i n w i t h , and s e c o n d , i t p r o v i d e s the r e f e r e n c e p o i n t f o r change. D e s p i t e t h e m e t h o d o l o g i c a l r i g o u r and the o b j e c t i v i t y o f f e r e d by t h e e x p e r i m e n t a l d e s i g n s , t h e i r s h o r t c o m i n g s , p a r t i c u l a r l y when a p p l i e d to a f i e l d s e t t i n g , have p l a c e d them 88 under i n c r e a s i n g s c r u t i n y . Many of the c r i t i c i s m s a r e r e l a t e d to the p r a c t i c a l p r o b l e m s w h i c h a r i s e when u s i n g them, p a r t i c u l a r l y i n f i e l d s e t t i n g s . R e s e a r c h e r s have f o u n d t h a t many o f the key r e q u i r e m e n t s c a n r a r e l y be met: the s u b j e c t s c a n n o t o f t e n be randomly a s s i g n e d to g r o u p s ; e q u i v a l e n c e o f the e x p e r i m e n t a l and c o n t r o l g r o u p s c a n n o t be g u a r a n t e e d ; i n t e r v e n t i o n and outcome commonly have o n l y tenuous t h e o r e t i c a l l i n k s ; many programs s i m u l t a n e o u s l y employ m u l t i p l e i n t e r v e n t i o n s ; and, key s o u r c e s o f v a r i a n c e c a n n o t be c o n t r o l l e d . I n t h e I r a n and H a i t i c a s e s , the e v a l u a t i v e d e s i g n c h o s e n was i n a p p r o p r i a t e s i n c e none o f the key r e q u i r e m e n t s c o u l d be met. C o n s e q u e n t l y , t h e e f f e c t i v e n e s s of t h e p r o p o s e d e v a l u a t i v e r e s e a r c h d e s i g n s was s e v e r e l y l i m i t e d . The t h r e e r e m a i n i n g c a s e s t u d i e s used n o n - e x p e r i m e n t a 1 r e s e a r c h d e s i g n s . W h i l e t h e s e a p p r o a c h e s a r e l e s s r i g o r o u s , and r e q u i r e fewer c o n t r o l l i n g a s s u m p t i o n s , t h e y a r e a l s o l a r g e l y i n e f f e c t i v e i n d e m o n s t r a t i n g c a u s a l i t y . F o r example, i n b o t h the C o l o m b i a n and P a r a g u a y a n c a s e s t u d i e s , the r e s e a r c h e r s used a one-group p r e t e s t - p o s t t e s t d e s i g n . However, w i t h o u t e x t e r n a l c o m p a r a t i v e p o p u l a t i o n s , the v a l u e o f t h e d a t a c o l l e c t e d i s s i g n i f i c a n t l y l i m i t e d . S i m p l y c o m p a r i n g b e f o r e - a f t e r measures w i t h i n one s t u d y p o p u l a t i o n w i t h o u t r e f e r e n c e to t r e n d s and p a t t e r n s i n c o m p a r a t i v e g r o u p s w h i c h d i d n o t r e c e i v e the i n t e r v e n t i o n i s n o t s u f f i c i e n t to c l a i m t h a t the i n t e r v e n t i o n s were e f f e c t i v e i n a f f e c t i n g h e a l t h . 89 A n o t h e r f a c t o r f o u n d to l i m i t the v a l i d i t y of the e v a l u a t i o n s t u d i e s p r e s e n t e d i n t h i s s t u d y i s the f a i l u r e on the p a r t of t h e r e s e a r c h e r s to a c c o u n t f o r the e x i s t e n c e and p o s s i b l e e f f e c t on h e a l t h s t a t u s o f o t h e r d e v e l o p m e n t p r o j e c t s w h i c h o c c u r e d s i m u l t a n e o u s l y w i t h the p r i m a r y h e a l t h c a r e p r o j e c t s . O t h e r d e v e l o p m e n t a c t i v i t i e s c o u l d have a p r o f o u n d e f f e c t on h e a l t h , a l t h o u g h t h e y a r e n o t h e a l t h s p e c i f i c . F o r example, the R u r a l H e a l t h and F a m i l y P l a n n i n g ( H a i t i ) p r o j e c t e v a l u a t i o n c l a i m e d t h a t t h e community-based n u t r i t i o n e d u c a t i o n program was l a r g e l y r e s p o n s i b l e f o r improvements i n the n u t r i t i o n a l s t a t u s o f c h i l d r e n i n the s t u d y a r e a . Y e t s i m u l t a n e o u s to the d e p l o y m e n t of the p r o j e c t community h e a l t h w o r k e r s , a major i n t e g r a t e d d e v e l o p m e n t program was b e i n g implemented i n the s t u d y a r e a . Changes i n the a v a i l a b i l i t y of l o c a l f o o d s t u f f s and b e t t e r m a r k e t i n g p r o c e d u r e s r e s u l t i n g from d e v e l o p m e n t program a c t i v i t i e s may have been more i n f l u e n t i a l i n a f f e c t i n g the n u t r i t i o n a l s t a t u s o f H a i t i a n c h i l d r e n t h a n the community-based n u t r i t i o n e d u c a t i o n program. However, t h i s p o s s i b i l i t y was n o t d i s c u s s e d i n t h e p r o j e c t e v a l u a t i o n . I n the C o l o m b i a n p r o j e c t , the h e a l t h - r e l a t e d a c t i v i t i e s were o n l y one component o f a w i d e r s p e c t r u m o f community d e v e l o p m e n t a c t i v i t i e s . A g a i n , t h e r e s e a r c h e r s d i d n o t examine the p o t e n t i a l e f f e c t o f the o t h e r p r o j e c t a c t i v i t i e s on the h e a l t h o f the s t u d y p o p u l a t i o n . T h e r e f o r e , c l a i m s t h a t the h e a l t h program was u n i q u e l y r e s p o n s i b l e f o r n o t e d changes i n 90 h e a l t h i n d i c e s a r e s p u r i o u s . The same can be s a i d a b o u t the r e s u l t s r e p o r t e d f o r the P a r a g u a y a n p r o j e c t , as no i n f o r m a t i o n was p r o v i d e d c o n c e r n i n g the e x i s t e n c e o f p a r a l l e l h e a l t h p r o m o t i o n campaigns i n t h e s t u d y a r e a . The l a c k o f a d e q u a t e and r e l i a b l e d a t a c o l l e c t i o n t e c h n i q u e s i s a n o t h e r f a c t o r w h i c h l i m i t s the v a l i d i t y of the e v a l u a t i o n e x e r c i s e s and the u t i l i t y o f the d a t a c o l l e c t e d . T h i s c r i t i c i s m r e l a t e s as w e l l to the i n s t r u m e n t s used to g a t h e r d a t a and t h e manner i n w h i c h they a r e a d m i n i s t e r e d . The r e s u l t s o b t a i n e d may n o t be v a l i d i f the q u e s t i o n n a i r e s a d m i n i s t e r e d b e f o r e and a f t e r t h e program were n o t i d e n t i c a l , o r i f i n t e r v i e w t e c h n i q u e s d i f f e r e d , i n c l u d i n g t h o s e r e s p o n s i b l e f o r c o n d u c t i n g the i n t e r v i e w s . W h i l e f o l l o w - u p s u r v e y s were c o n d u c t e d i n most p r o j e c t s , t h e r e was no i n d i c a t i o n t h a t t h e c o n t e n t o f the q u e s t i o n n a i r e s was the same as t h a t u s e d i n t h e b a s e l i n e s u r v e y . F o r example, i t was u n c l e a r i n t h e m a j o r i t y o f the s t u d i e s whether t h e same f a m i l i e s were i n t e r v i e w e d d u r i n g b o t h the b a s e l i n e and f o l l o w - u p s u r v e y s . No i n d i c a t i o n was p r o v i d e d as to the r a t e o f a b s e n t e e i s m among t h e p o p u l a t i o n d u r i n g t h e f o l l o w - u p s u r v e y s , a s i t u a t i o n w h i c h c o u l d s i g n i f i c a n t l y b i a s t h e v a l i d i t y and i n t e r p r e t a t i o n of the d a t a c o l l e c t e d . As w e l l , w h i l e some o f the s t u d i e s u t i l i z e d s p e c i a l l y t r a i n e d p e r s o n n e l to c o n d u c t i n t e r v i e w s , i t can o n l y be assumed t h a t t h e same p e o p l e c o n d u c t e d t h e f o l l o w - u p s u r v e y s i n o r d e r to save time and 91 expense r e l a t e d to t r a i n i n g . A c a s e i n p o i n t i s t h e i n f o r m a t i o n c o l l e c t e d f o r the V o l u n t e e r H e a l t h Worker ( T h a i l a n d ) p r o j e c t . The met h o d o l o g y used to c o l l e c t t h e d a t a s e v e r e l y l i m i t e d t h e u s e f u l n e s s and s i g n i f i c a n c e of the e n t i r e e v a l u a t i o n e x e r c i s e . The c o l l e c t i o n of b a s e l i n e d a t a on s t u d y v i l l a g e s was b a s e d upon the VHW's r e c a l l a b i l i t i e s . They were e x p e c t e d to r e c a l l s p e c i f i c c o n d i t i o n s i n the v i l l a g e s p r i o r to t h e i r t r a i n i n g , even though o v e r two y e a r s had l a p s e d i n many c a s e s . T h e r e was a l s o b i a s i n r e p o r t i n g h e a l t h s t a t i s t i c s due to the h i g h r a t e o f n o n - r e s p o n s e o f VHWs i n s u b m i t t i n g t h e i r a c t i v i t y r e p o r t s . I n t h e K a v a r s t u d y , the r e s e a r c h team i n t e n d e d to c o l l e c t and a n a l y s e d a t a r e l a t e d to improvements i n s a n i t a r y f a c i l i t i e s and changes i n s a n i t a t i o n b e h a v i o u r o v e r the c o u r s e o f the p r o j e c t . However, as the e v a l u a t i o n n o t e d , t h e s e d a t a were c o l l e c t e d t oo i n f r e q u e n t l y to be o f much u s e . T h i s i s a l s o the r e s u l t o f t h o s e e x p e c t e d to d e l i v e r h e a l t h c a r e s e r v i c e s a l s o b e i n g e x p e c t e d to c o l l e c t and c o m p i l e s t a t i s t i c a l d a t a . The g r e a t e r the amount o f d a t a to be c o l l e c t e d by f i e l d w o r k e r s , p a r t i c u l a r l y i f t h e y a r e the same p e o p l e d e l i v e r i n g the s e r v i c e s , the g r e a t e r t h e p r o b a b i l i t y o f d a t a e r r o r s . The same i s a l s o t r u e f o r q u e s t i o n n a i r e s o r forms w h i c h a r e complex and d i f f i c u l t t o a d m i n i s t e r and i n t e r p r e t . A c a s e i n p o i n t was the Pa r a g u a y p r o j e c t , where the r e s e a r c h e r s r e p o r t e d t h a t even t h o s e p e o p l e g i v e n p r i o r t r a i n i n g ( i n c l u d i n g the r u r a l 92 t e a c h e r s ) e x p e r i e n c e d d i f f i c u l t y i n u n d e r s t a n d i n g and u s i n g the d a t a s h e e t s c o r r e c t l y . D e s p i t e the t r a i n i n g s e s s i o n s where i t was e x p l a i n e d how the forms were to be c o m p l e t e d and d e s p i t e the emphasis p l a c e d on the i m p o r t a n c e o f t h forms i n the e v a l u a t i o n o f the p r o j e c t , the o n - g o i n g t r a i n i n g of t e a c h e r s and s u p e r v i s i o n o f p r o j e c t a c t i v i t i e s , a s i g n i f i c a n t number of t h e t e a c h e r s made v i r t u a l l y no use of them. Of t h o s e forms w h i c h were c o m p l e t e d and s u b m i t t e d to p r o j e c t a d m i n i s t r a t i o n , i t was r e p o r t e d t h a t a p p r o x i m a t e l y 407. were i n c o r r e c t l y or i n a d e q u a t e l y c o m p l e t e d . C o n s e q u e n t l y , the r e s e a r c h team was u n a b l e to measure changes i n some of t h e i n d i c e s w h i c h the program o r i g i n a l l y s o u g h t to examine. A n o t h e r s h o r t c o m i n g o f many of t h e s e e v a l u a t i o n s i s the l a c k o f v e r i f i c a t i o n o f the d a t a c o l l e c t e d . F o r example, w h i l e some of the p r o j e c t s used s p e c i a l l y t r a i n e d e n u m e r a t o r s to c o l l e c t d a t a , no d e t a i l s were p r o v i d e d o f the methods used to v e r i f y t h e v a l i d i t y of t h e d a t a c o l l e c t e d . An e x c e p t i o n was the C o l o m b i a p r o j e c t w h i c h u n d e r t o o k to e n s u r e a h i g h d e g r e e o f r e l i a b i l i t y and c o m p l e t e n e s s f o r the i n f o r m a t i o n c o l l e c t e d by the r u r a l h e a l t h p r o m o t e r s . I n a s e p a r a t e s u r v e y , the d a t a c o l l e c t e d i n the i n i t i a l s u r v e y s were c h e c k e d a g a i n s t the r e s p o n s e s o f a random sample of one t h i r d of a l l the i n d i v i d u a l s s e r v e d by them. A l l f a m i l y members were i n t e r v i e w e d and t h e p r e v a i l i n g s a n i t a r y c o n d i t i o n s o f t h e h o u s e h o l d compound r e c o r d e d . I n f o r m a t i o n on s i c k n e s s and i n c a p a c i t a t i n g 93 i l l n e s s was a l s o c o l l e c t e d o v e r a p e r i o d of two weeks, w h i l e m o r t a l i t y and p a r a s i t i c i n f e c t i o n r a t e s were c o l l e c t e d o v e r a p e r i o d of one y e a r . The e v a l u a t i o n f o u n d t h a t the i n f o r m a t i o n c o l l e c t e d by the r u r a l h e a l t h p r o m o t e r s was c o n s i s t e n t l y of a h i g h q u a l i t y , and d i f f e r e d l i t t l e f rom th e d a t a c o l l e c t e d u s i n g the c o m p a r a t i v e s u r v e y . The use o f q u a n t i t a t i v e d a t a , a l m o s t to the e x c l u s i o n of q u a l i t a t i v e d a t a , i s a n o t h e r f a c t o r w h i c h seems to be r e s p o n s i b l e f o r l i m i t i n g the s i g n i f i c a n c e and u t i l i t y of the c a s e s t u d y e v a l u a t i o n r e s u l t s . R e s e a r c h e r s t e n d to c o n c e n t r a t e on c o l l e c t i n g q u a n t i t a t i v e d a t a by o b j e c t i v e s u r v e y methods, to the n e g l e c t of o t h e r d a t a w h i c h may be more s a l i e n t to the p e r f o r m a n c e and e f f e c t of the i n n o v a t i v e a p p r o a c h b e i n g u s e d . They d i s r e g a r d s u c h d a t a b e c a u s e i t i s ' s u b j e c t i v e ' , ' a n e c d o t a l ' , o r ' i m p r e s s i o n i s t i c ' . ( 5 0 ) C o c h r a n (1980) a r g u e s t h a t q u a n t i f i c a t i o n t e n d s to add e l e m e n t s t h a t a r e unwanted and o v e r l o o k s dynamics t h a t a r e p a r t o f the p r o c e s s o f s o c i a l change and i n t e r a c t i o n . ( 5 1 ) She warns t h a t q u a n t i t a t i v e d a t a may have an a u r a o f a u t h e n t i c i t y even when they a r e i n a c c u r a t e o r f a l s e , e s p e c i a l l y i f t h e y come from a r e p u t a b l e s o u r c e . ( 5 2 ) F o r the most p a r t , t h e f i v e c a s e s t u d i e s r e l i e d on q u a n t i t a t i v e d a t a as e v i d e n c e of t h e i m p a c t of t h e i r r e s p e c t i v e p r o j e c t a c t i v i t i e s on h e a l t h . But the v a l i d i t y o f s u c h d a t a i s s i g n i f i c a n t l y d e c r e a s e d due to the a b s e n c e o f c o m p a r a t i v e and c o r r o b o r a t i n g d a t a , as has been p r e v i o u s l y d i s c u s s e d . 94 Q u a l i t a t i v e d a t a c o u l d have s e r v e d as c o r r o b o r a t i v e e v i d e n c e to s u p p o r t th e q u a n t i t a t i v e d a t a c o l l e c t e d i n t h e s e s t u d i e s . F o r example, i n the P a r a g u a y PHC program e v a l u a t i o n , i n t e r v i e w s on r e a c t i o n s to t h e program were c o n d u c t e d w i t h b o t h the t e a c h e r s and the v i l l a g e r s . W h i l e the q u a l i t a t i v e d a t a c o l l e c t e d i n the s t u d y d i d n o t c o n c l u s i v e l y l i n k the a c t i v i t i e s o f the t e a c h e r s w i t h a p p a r e n t changes i n the b e h a v i o u r of v i l l a g e r s and t h e i r h e a l t h s t a t u s , the d a t a c o l l e c t e d d u r i n g the i n t e r v i e w s l e n t s u p p o r t to t h e a s s o c i a t i o n s u g g e s t e d by the q u a n t i t a t i v e d a t a . The q u a l i t a t i v e i n f o r m a t i o n c o l l e c t e d s u g g e s t e d t h a t the s c h o o l c h i l d r e n assumed the r o l e o f community change a g e n t s , p l a y i n g an e x t r e m e l y i m p o r t a n t r o l e i n d i s s e m i n a t i n g i n f o r m a t i o n r e l a t i n g to h e a l t h a c t i v i t i e s . As a r e s u l t , the v a c c i n a t i o n campaign was v e r y s u c c e s s f u l and i n c l u d e d t h e i r p r e - s c h o o l aged s i b l i n g s as p a r t i c i p a n t s , an a c t i v i t y w h i c h had n o t been f o r e s e e n i n the o r i g i n a l p r o j e c t g o a l s . I t was a l s o r e p o r t e d t h a t due to t h e i n f l u e n c e o f the c h i l d r e n on c h a n g i n g b e h a v i o u r a l a t t i t u d e s w i t h i n the f a m i l y , a s i g n i f i c a n t number of h o u s e h o l d s d e c i d e d to m o d i f y t h e i r h y g i e n e and s a n i t a t i o n h a b i t s . Had s u c h i n f o r m a t i o n been c o l l e c t e d i n t h e o t h e r s t u d i e s , p e r h a p s the r e s u l t s drawn from the q u a n t i t a t i v e d a t a would have been of g r e a t e r s i g n i f i c a n c e ad u s e f u l n e s s. The f o r e g o i n g a n a l y s i s would seem to i n d i c a t e t h a t the f a c t o r s w h i c h a f f e c t t h e v a l i d i t y and u t i l i t y o f e v a l u a t i o n s of 95 p r i m a r y h e a l t h c a r e programs d e s i g n e d and c a r r i e d out by e x p a t r i a t e a g e n c i e s and o r g a n i z a t i o n s a l s o c h a r a c t e r i z e d t h o s e d e s i g n e d and implemented by i n d i g e n o u s r e s e a r c h e r s . The major p r o b l e m s i d e n t i f i e d i n t h i s s t u d y can be summarized as f o l l o w s : e v a l u a t i v e r e s e a r c h d e s i g n s w h i c h a r e e i t h e r too complex f o r the f i e l d s i t u a t i o n a t hand, o r a r e o v e r - s i m p l i f i e d s u c h t h a t t h e y a r e i n c a p a b l e o f d e m o n s t r a t i n g even a s t r o n g a s s o c i a t i o n between t h e i n t e r v e n t i o n and program p e r f o r m a n c e and i m p a c t ; an i n s u f f i c i e n t t i m e f r a m e o v e r w h i c h the i m p a c t on h e a l t h of h e a l t h - r e l a t e d i n t e r v e n t i o n s c a n be a d e q u a t e l y measured; the n a t u r e and q u a n t i t y of d a t a c o l l e c t e d ; t h e r e l i a n c e on m o r b i d i t y and m o r t a l i t y d a t a to d e m o n s t r a t e c a u s a l i t y ; t h e l a c k o f a g l o b a l p e r s p e c t i v e w i t h r e s p e c t to a c t i v i t i e s i n o t h e r economic and s o c i a l s e c t o r s w h i c h may a f f e c t h e a l t h ; and, the l a c k of q u a l i t a t i v e d a t a to s u b s t a n t i a t e t h e c l a i m s made u s i n g q u a n t i t a t i v e d a t a . T h e s e f a c t o r s seem to f a l l i n t o two major c a t e g o r i e s : d e s i g n d e f i c e n c i e s and weaknesses i n m e t h o d o l o g y and t e c h n i q u e . The t a s k now becomes t h a t o f i d e n t i f y i n g s t r a t e g i e s to overcome t h e s e l i m i t i n g f a c t o r s , so t h a t f i e l d - b a s e d e v a l u a t i o n s a r e c r e d i b l e . Summary The c h a p t e r has p r e s e n t e d an a n a l y s i s o f the major f a c t o r s w h i c h l i m i t t h e v a l i d i t y , and u l t i m a t e l y t h e u t i l i t y of 96 e v a l u a t i v e r e s e a r c h on p r i m a r y h e a l t h c a r e i n d e v e l o p i n g c o u n t r i e s . The a n a l y s i s has i l l u s t r a t e d t h a t t h e s e a r e n o t r e s t r i c t e d to e v a l u a t i v e r e s e a r c h d e s i g n f a u l t s . The i n d i c e s c h o s e n to measure program p e r f o r m a n c e and i m p a c t , and the t e c h n i q u e s u s e d to c o n d u c t t h e e v a l u a t i o n , p a r t i c u l a r l y the d a t a c o l l e c t i o n methods, a l s o have a s i g n i f i c a n t e f f e c t on the v a l i d i t y and r e l i a b i l i t y o f e v a l u a t i o n r e s u l t s . The i s s u e i s t h e r e f o r e n o t so much whether one p a r t i c u l a r r e s e a r c h d e s i g n i s b e t t e r t h a n a n o t h e r . R a t h e r , t h e aim of t h e e x e r c i s e i s to e n s u r e t h a t t h e c o n s t r u c t d e f i c i e n c i e s i n h e r e n t i n the d e s i g n and the t e c h n i q u e s c h o s e n a r e a d e q u a t e l y d e a l t w i t h . The n e x t c h a p t e r p r e s e n t s a number of s u g g e s t i o n s to s t r e n g t h e n the v a l i d i t y and u t i l i t y o f f i e l d - b a s e d e v a l u a t i o n s c o n d u c t e d i n d e v e l o p i n g c o u n t r i e s . 97 REFERENCES 1. B.G. G r e e n b e r g , " E v a l u a t i o n of S o c i a l P rograms," i n R e a d i n g s i n E v a l u a t i o n R e s e a r c h , ed. F.G. C aro (New Y o r k : R u s s e l l Sage F o u n d a t i o n , 1971), p. 160. 2. A m e r i c a n P u b l i c H e a l t h A s s o c i a t i o n , Committee on P u b l i c H e a l t h A d m i n i s t r a t i o n , " G l o s s a r y of A d m i n i s t r a t i v e Terms i n P u b l i c H e a l t h , " AJPH 50 ( 1 9 6 0 ) : 225 - 226 ; Committee on E v a l u a t i o n and S t a n d a r d s , " G l o s s a r y o f E v a l u a t i v e Terms i n P u b l i c H e a l t h , " AJPH 60 ( 1 9 7 0 ) : 1 5 4 6 . 3. M.D. Warren, "The p r o c e s s and methods o f e v a l u a t i n g p u b l i c h e a l t h programmes", i n Methods of E v a l u a t i n g P u b l i c  H e a l t h Programmes, p. 35 (Copenhagen: WHO R e g i o n a l O f f i c e f o r E u r o p e , 1 9 6 8 ) . 4. C.H. W e i s s , E v a l u a t i v e R e s e a r c h : Methods o f A s s e s s i n g  Program E f f e c t i v e n e s s , (Englewood C l i f f s : P r e n t i c e H a l l I n c . Methods o f S o c i a l S c i e n c e S e r i e s , 1 972), p r e f a c e p. 8, p. 1. 5. S.M. S h o r t e l l and W.C. R i c h a r d s o n , H e a l t h Program  E v a l u a t i o n , ( S t . L o u i s : The C.V. Mosby Company, 1978), p. 8. 6. E.A. Suchman, E v a l u a t i v e R e s e a r c h : P r i n c i p l e s and  P r a c t i c e i n P u b l i c S e r v i c e and S o c i a l A c t i o n Programs (New Y o r k : R u s s e l l Sage F o u n d a t i o n , 1 9 6 7 ) , pp. 60-63. 7. F.S. C h a p i n , E x p e r i m e n t a l D e s i g n s i n S o c i o l o g i c a l  R e s e a r c h (New Y o r k : H a r p e r and B r o t h e r s , 1 9 47). 8. V. G e t t i n g , " E v a l u a t i o n , " AJPH 47 (1957):400-413 ; E. H e r z o g , Some G u i d e l i n e s f o r E v a l u a t i o n R e s e a r c h ( W a s h i n g t o n , D.C.: U.S. D e partment o f E d u c a t i o n , H e a l t h and W e l f a r e , 1959); M. J o h a d a and E. B a r n i t z , "The N a t u r e o f E v a l u a t i o n , " I n t ' 1 .  Soc. S c i . J o u r n a l 7 ( 1 9 5 5 ) : 353-364; 0. K l i n e b e r g , "The P r o b l e m o f E v a l u a t i o n , " I n t ' l . Soc. S c i . J o u r n a l 7 ( 1 9 5 5 ) : 346 - 350 ; B.D. P a u l , " S o c i a l S c i e n c e i n P u b l i c H e a l t h , " AJPH 46 ( 1956):1390-1396. 9. A t t k i n s o n and B r o s k o w s k i , p. 8; E.W. F l a h e r t y and J.A. M o r e l l , " E v a l u a t i o n s : M a n i f e s t a t i o n s of a New F i e l d , " J . of  E v a l . and Program P l a n n i n g 1 ( 1 ) ( 1 9 7 8 ) : 3 ; J.W. Townsend, W.T. F a r r e l l and R.E. K l e i n , " S p e c i a l I s s u e s f o r the Measurement o f Program Impact i n D e v e l o p i n g C o u n t r i e s , " i n E v a l u a t i n g the  Impact o f N u t r i t i o n and H e a l t h Programs, eds . R.E. K l e i n e t a l . (New Y o r k : Plenum P r e s s , 1 9 7 9 ) , p. 102. 98 10. U n i t e d N a t i o n s , W o r l d H e a l t h O r g a n i z a t i o n , R e p o r t of the  I n t e r n a t i o n a l C o n f e r e n c e on P r i m a r y H e a l t h C a r e , A l m a - A t a , USSR, 6-12 September 1978 , (ICPHC/ALA/78.10 ) , 12 September 1978 , p. 12. 11. D e t a i l s o f s p e c i f i c PHC p r o j e c t s can be o b t a i n e d from t h e f o l l o w i n g documents: A m e r i c a n P u b l i c H e a l t h A s s o c i a t i o n , The  S t a t e of the A r t of D e l i v e r i n g Low C o s t H e a l t h S e r v i c e s i n  D e v e l o p i n g C o u n t r i e s : A Summary of 180 H e a l t h P r o j e c t s ( W a s h i n g t o n , D.C.: APHA, 1977) ; M.B. P a r l a t o and M.N. F a v i n , P r i m a r y H e a l t h C a r e : P r o g r e s s and P r o b l e m s . An A n a l y s i s of 52  A I D - a s s i s t e d P r o j e c t s ( W a s h i n g t o n , D.C.: APHA, 1982) ; N a t i o n a l C o u n c i l f o r I n t e r n a t i o n a l H e a l t h , I n t e r n a t i o n a l H e a l t h :  M e a s u r i n g P r o g r e s s - P r o c e e d i n g s 1980 I n t e r n a t i o n a l H e a l t h  C o n f e r e n c e ( W a s h i n g t o n , D.C.: NCIH, 1980); SALUS: Low C o s t R u r a l  H e a l t h Care and H e a l t h Manpower T r a i n i n g : an a n n o t a t e d  b i b l i o g r a p h y w i t h s p e c i a l emphasis on d e v e l o p i n g c o u n t r i e s ( O t t a w a : I n t e r n a t i o n a l Development R e s e a r c h C e n t r e (IDRC), 1975 ( v o l . D - 1 9 8 3 ( v o l . 1 3 ) ) . 12. A m e r i c a n P u b l i c H e a l t h A s s o c i a t i o n , The S t a t e of the A r t  of D e l i v e r i n g Low C o s t H e a l t h S e r v i c e s i n D e v e l o p i n g C o u n t r i e s : A Summary Study of 180 H e a l t h P r o j e c t s ( W a s h i n g t o n , D.C.: APHA, 1977) and M.B. P a r l a t o and M.N. F a v i n , P r i m a r y H e a l t h C a r e :  P r o g r e s s and P r o b l e m s . An A n a l y s i s of 52 A I D - A s s i s t e d P r o j e c t s ( W a s h i n g t o n , D.C.: APHA, 1 9 8 2 ) . 13. J.W. Townsend, W.T. F a r r e l l and R.E. K l e i n , " S p e c i a l I s s u e s f o r t h e Measurement o f Program Impact i n D e v e l o p i n g C o u n t r i e s , " i n E v a l u a t i n g the Impact of N u t r i t i o n and H e a l t h  P rograms, e d s . R.E. K l e i n e t a l . (New Y o r k : Plenum P r e s s , 1979), p. 99. Ik. A m e r i c a n P u b l i c H e a l t h A s s o c i a t i o n , D e l i v e r i n g H e a l t h  S e r v i c e s , p. 53; A. E l z i n g a , E v a l u a t i n g the E v a l u a t i o n Game: On  the M e t h o d o l o g y of P r o j e c t E v a l u a t i o n , w i t h S p e c i a l R e f e r e n c e to  Development C o o p e r a t i o n ( S t o c k h o l m : S w edish Agency f o r R e s e a r c h C o o p e r a t i o n w i t h D e v e l o p i n g C o u n t r i e s , 1 9 8 1 ) , p. 13; D.A. R o n d i n e l l i , " I n t e r n a t i o n a l R e q u i r e m e n t s f o r P r o j e c t P r e p a r a t i o n : A i d s o r O b s t a c l e s to Development P l a n n i n g ? , " Amer. Ins t .  P l a n n e r s J o u r n a l 13 ( 1 9 7 6 ) : 3 1 4 . 15. A m e r i c a n P u b l i c H e a l t h A s s o c i a t i o n , D e l i v e r i n g H e a l t h  S e r v i c e s , p. 53. 16. S.M. S h o r t e l l and W.C. R i c h a r d s o n , H e a l t h Program  E v a l u a t i o n ( S t . L o u i s : The C.V. Mosby Company, 19 7 8 ) , p. 16. 99 17. A m e r i c a n P u b l i c H e a l t h A s s o c i a t i o n , D e l i v e r i n g H e a l t h  S e r v i c e s , p. 53. 18. P a r l a t o and F a v i n , P r i m a r y H e a l t h C a r e , p. 79. 19. N. M i l i o , Community H e a l t h W o r k ers: E v a l u a t i o n of t h e i r  Impact on H e a l t h (New Y o r k : I n t e r n a t i o n a l M e d i c a l and R e s e a r c h F o u n d a t i o n , 1977), p. 13. 20. A m e r i c a n P u b l i c H e a l t h A s s o c i a t i o n , D e l i v e r i n g H e a l t h  S e r v i c e s , p. 14,21-45; P a r l a t o and F a v i n , P r i m a r y H e a l t h C a r e , p. 15,27. 21. I b i d . , p. 79. 22. R o n d i n e l l i , " I n t e r n a t i o n a l R e q u i r e m e n t s f o r P r o j e c t P r e p a r a t i o n , " p. 322; Townsend e t a l . , " S p e c i a l I s s u e s f o r Measurement," p. 118; U n i t e d N a t i o n s , World H e a l t h O r g a n i z a t i o n , F i r s t R e g i o n a l Seminar on R u r a l H e a l t h S e r v i c e s F i n a l R e p o r t (WHO R e g i o n a l O f f i c e f o r t h e W e s t e r n P a c i f i c , M a n i l a : n.p., J u l y 1963) , p . 37 . 23. A.K. Neumann e t a l . , " E d u c a t i o n and e v a l u a t i o n i n an i n t e g r a t e d MCH/FP p r o j e c t i n r u r a l Ghana: the D a n f a p r o j e c t , " I n t l . J n l . H l t h . Ed. 19(4) ( 1 9 7 6 ) : 2 3 5 ; U n i t e d N a t i o n s , W o r l d H e a l t h O r g a n i z a t i o n F i r s t R e g i o n a l Seminar on R u r a l H e a l t h  S e r v i c e s , p. 37; N.E. W i l l i a m s o n , "Outcome I n d i c a t o r s Used to E v a l u a t e t h e B o h o l P r o j e c t , " i n N a t i o n a l C o u n c i l f o r I n t e r n a t i o n a l H e a l t h , I n t e r n a t i o n a l H e a l t h : M e a s u r i n g P r o g r e s s -P r o c e e d i n g s 1980 I n t e r n a t i o n a l H e a l t h C o n f e r e n c e ( W a s h i n g t o n , D.C.: NCIH, 1980) , p. 32. 24. A m e r i c a n P u b l i c H e a l t h A s s o c i a t i o n , D e l i v e r i n g H e a l t h  S e r v i c e s , p. 51; P a r l a t o and F a v i n , P r i m a r y H e a l t h C a r e , p. 80. 25. O.L. D e n i s t o n e t a l . , " E v a l u a t i o n of Program E f f e c t i v e n e s s , " i n Program E v a l u a t i o n i n the H e a l t h F i e l d s , e d s . H.C. S c h u l b e r g e t a l . (New Y o r k : B e h a v i o u r a l P u b l i c a t i o n s , 1 969), p. 235; A.U. K i r k b y , The Development of a U s e r - C h o i c e  A p p r o a c h i n R u r a l Water S u p p l y , R u r a l Water S u p p l y and S a n i t a t i o n W o r k i n g P a p e r , no. 7 ( O t t a w a : IDRC, 1973), u n p a g i n a t e d ; S.D. M e s s i n g e t a l . , "A method of h e a l t h c u l t u r e r e s e a r c h i n an A f r i c a n c o u n t r y , " J n l . H l t h . & Hum. Beh. 7(4) ( 1 9 6 6 ) : 273 . 26. P a r l a t o and F a v i n , P r i m a r y H e a l t h C a r e , p. 81. 27. A m e r i c a n P u b l i c H e a l t h A s s o c i a t i o n , D e l i v e r i n g H e a l t h  S e r v i c e s , p. 51. 100 28. F . J . B e n n e t t , " E v a l u a t i o n i n H e a l t h E d u c a t i o n , " J n l .  T r o p . Med. & Hyg. , 66(1) (1963 ) : 25 - 28 ; B. R o b e r t s , " R e s e a r c h i n E d u c a t i o n A s p e c t s o f H e a l t h Programmes," I n t l . J n l . H l t h . Ed., 1 3 ( 1 ) , s u p p l e m e n t ( 1 9 7 0 ) : 8 . 29. G o l l o d a y and Koch-Weser, "New P o l i c i e s f o r R u r a l H e a l t h , " p. 172. 30. C.H. S i n n e t t e , " E v a l u a t i o n : A Measurement Dilemma," i n I n t e r n a t i o n a l H e a l t h : M e a s u r i n g P r o g r e s s - P r o c e e d i n g s 1980  I n t e r n a t i o n a l H e a l t h C o n f e r e n c e ( W a s h i n g t o n , D.C.: NCIH, 1 9 8 0 ) , p . 9 . 31. H.H. Hyman and C.R. W r i g h t , " E v a l u a t i n g S o c i a l A c t i o n Progams," i n Uses of S o c i o l o g y , e d s . L a z a r s f e l d e t a l . (New Y o r k : B a s i c B ooks, 1 9 6 7 ) , p. 764. 32. D.R. G w a t k i n e t a l . , "Can H e a l t h and N u t r i t i o n I n t e r v e n t i o n s Make a D i f f e r e n c e ? , " Soc. S c i . & Med. 14(C) ( 1 9 8 0 ) : 1 2 5 . 33. I b i d . , p.125; G.M.K. Kpedekpo, "The P l a n n i n g and D e s i g n o f S a m p l i n g S u r v e y s w i t h P a r t i c u l a r R e f e r e n c e to the E p i d e m i o l o g i c a l S u r v e y o f t h e D a n f a P r o j e c t i n Ghana," Ghana  Med. J . (December 1972):378; M e s s i n g e t a l . , "Method o f H e a l t h C u l t u r e R e s e a r c h , " p. 274. 34. G w a t k i n e t a l . , " H e a l t h and N u t r i t i o n I n t e r v e n t i o n s , " p. 125. 35. P a r l a t o and F a v i n , P r i m a r y H e a l t h C a r e , p. 80. 36. I b i d . , p. 87. 37. I b i d ; d i s c u s s i o n s w i t h CIDA p e r s o n n e l . 38. S. A m i d i , " E f f e c t i v e n e s s o f v i l l a g e h e a l t h w o r k e r s f o r p r i m a r y h e a l t h c a r e i n s o u t h e r n I r a n , " C o u r r i e r of the  I n t e r n a t i o n a l C h i l d r e n ' s C e n t r e ( P a r i s ) 27(2) (1977):109-112; H.A. Ronaghy, " K a v a r V i l l a g e H e a l t h Worker P r o j e c t , " J . T r o p .  Ped. and E n v i r . C h i l d H e a l t h 2 4 ( 1 ) , Monograph No. 52 (1978) : 13-60 ; B. Z e i g h a m i and E. Z e i g h a m i , " E v a l u a t i o n of I r a n i a n v i l l a g e w o r k e r s e f f i c a c y , " i n V i l l a g e H e a l t h W o rkers, e d s . H.A. Ronaghy, Y. Mousseau-Gershman and A. D o r o z y n s k i ( O t t a w a : IDRC, 1 9 7 6 ) , pp. 14-20; B. Z e i g h a m i e t a l . , " S t r e t c h i n g h e a l t h manpower: the r u r a l h e a l t h a u x i l i a r y , " Cdn. J . Pub. H e a l t h . 6 8 ( 5 ) ( 1 9 7 7 ) : 378 - 381. 101 39. 0. E c h e v e r r i and J . S a r a v i a , S i s t e m a de s e r v i c i o s de  s a l u d r u r a l como un componente b a s l c o de l o s p r o y e c t o s de  d e s a r r o l l o r u r a l en C o l o m b i a [ R u r a l h e a l t h s e r v i c e s s y s t e m as a b a s i c component o f r u r a l d e v e l o p m e n t p r o j e c t s i n C o l o m b i a ] ( C a l i , C o l o m b i a : U n i v e r s i d a d d e l V a l l e , C e n t r o de I n v e s t i g a c i o n e s M u 1 t i d i s c i p 1 i n a r i a s en D e s a r r o l l o R u r a l (CIMDER), 197 4 ) ; M. I n g l e s i a s and P. V i l l a f o r e , S i s t e m a r u r a l de  s e r v i c i o s de s a l u d : i n f o r m e p r e l i m i n a r s o b r e o r g a n i z a c i o n de l a  comunidad [ R u r a l h e a l t h s e r v i c e s s y s t e m : p r e l i m i n a r y r e p o r t on community o r g a n i z a t i o n ] ( C a l i , C o l o m b i a : U n i v e r s i d a d d e l V a l l e , C e n t r o de I n v e s t i g a c i o n e s M u 1 t i d i s c i p 1 i n a r i a s en D e s a r o l l o R u r a l (CIMDER), 197 7 ) ; J . S a r a v i a and 0. E c h e v e r r i , " P r o y e c t o d e l s i s t e m a r u r a l de s e r v i c i o s : E v a l u a c i o n d e l s i s t e m a r u r a l de s e r v i c i o s de s a l u d 1977-1979," [ R u r a l h e a l t h s e r v i c e s s y s t e m p r o j e c t : E v a l u a t i o n o f t h e r u r a l h e a l t h c a r e s y s t e m 1977-1979] ( C a l i , C o l o m b i a : U n i v e r s i d a d d e l V a l l e , C e n t r o de I n v e s t i g a c i o n e s M u 1 t i d i s c i p 1 i n a r i a s en D e s a r r o l l o R u r a l (CIMDER), 1 9 8 0 ) . 40. Gouvernement de H a i t i , D e p a r t e m e n t de l a S a n t e P u b l i q u e e t de l a P o p u l a t i o n , " R a p p o r t F i n a l du P r o j e t I n t e g r e de S a n t e e t de P o p u l a t i o n de P e t i t - G o a v e , " [ F i n a l r e p o r t of t h e i n t e g r a t e d h e a l t h and p o p u l a t i o n p r o j e c t a t P e t i t - G o a v e ] ( P o r t - a u - P r i n c e , H a i t i : D e p a r t e m e n t de l a S a n t e P u b l i q u e e t de l a P o p u l a t i o n , 1 9 7 9 ) . 41. K. T u n g s u b u t r a , " H e a l t h c a r e d e l i v e r y s y s t e m to r e a c h t h e p e o p l e , " A s p e c t s : the I n t e r n a t i o n a l J o u r n a l of V o l u n t e e r  S e r v i c e s ( W a s h i n g t o n ) 1 ( 4 ) ( 1 9 7 7 ) : 4 - 8 ; K. T u n g s u b u t r a , " V o l u n t e e r H e a l t h W o r k e r s : A R e s e a r c h P r o j e c t c a r r i e d o u t by the F a c u l t y o f M e d i c i n e , Khon Kaen U n i v e r s i t y , " (Khon Kaen, T h a i l a n d : Khon Kaen U n i v e r s i t y , 1 9 7 9 ) . 42. R.H. M a l l o q u i n e t a l . , I n f o r m e F i n a l y C o n c l u s i o n e s d e l  Programa E x t e n s i o n de l o s S e r v i c i o s S a l u d a t r a v e s de M a e s t r o s  R u r a l e s [ F i n a l r e p o r t and c o n c l u s i o n s c o n c e r n i n g t h e e x t e n s i o n o f h e a l t h s e r v i c e s t h r o u g h r u r a l s c h o o l t e a c h e r s ] ( A s u n c i o n , P a r a g u a y : M i n i s t e r i o de S a n i d a d y A s i s t e n c i a S o c i a l , 1 9 8 0 ) . 43. D.F. S u l l i v a n , C o n c e p t u a l P r o b l e m s i n D e v e l o p i n g an  Index of H e a l t h ( W a s h i n g t o n , D.C.: N a t i o n a l C e n t e r f o r H e a l t h S t a t i s t i c s , 1 9 6 6 ) , S e r i e s 2, no. 17, p. 1. 44. J . - P . H a b i c h t and t h e W o r k i n g Group on R u r a l M e d i c a l C a r e , " D e l i v e r y o f P r i m a r y C a r e by M e d i c a l A u x i l i a r i e s : T e c h n i q u e s o f Use and A n a l y s i s o f B e n e f i t s A c h i e v e d i n some R u r a l V i l l a g e s i n G u a t e m a l a , " i n M e d i c a l C are A u x i l i a r i e s , p. 34, P A H 0 / W H 0 , S c i e n t i f i c P u b l i c a t i o n , no. 278 ( W a s h i n g t o n , D.C.: PAH0, 1 9 7 3 ) . 102 45. J . - P . H a b i c h t and P.A. Berman, " P l a n n i n g P r i m a r y H e a l t h S e r v i c e s f r o m a Body C o u n t ? , " Soc. S c i . & Med. 14C (1980):134; L. Ramchandran, " D e v e l o p i n g I n d i c e s f o r t h e E v a l u a t i o n o f H e a l t h S e r v i c e s , " I n d i a n J . P u b l i c H e a l t h ( C a l c u t t a ) p. 32; U n i t e d N a t i o n s , W o r l d H e a l t h O r g a n i z a t i o n , H e a l t h A s p e c t s of P o p u l a t i o n  T r e n d s and P r o s p e c t s ( W o r l d H e a l t h S t a t i s t i c a l R e p o r t , v o l . 27, no. 5 ) , 1974, p. 201. 46. M. S c h a e f e r , E v a l u a t i o n / D e c i s i o n - M a k i n g i n H e a l t h  P l a n n i n g and A d m i n i s t r a t i o n (Durham, N.C.: U n i v e r s i t y o f N o r t h C a r o l i n a P r e s s , 1973), p. 102. 47. A m i d i , " V i l l a g e h e a l t h w o r k e r s , " p. 110; Ronaghy, "Kavar p r o j e c t , " p. 45; G w a t k i n e t a l . , " H e a l t h and N u t r i t i o n I n t e r v e n t i o n s , " p. 124. 48. N. M i l i o , Community h e a l t h w o r k e r s : e v a l u a t i o n of t h e i r i m p a c t on h e a l t h (New Y o r k : I n t e r n a t i o n a l M e d i c a l and R e s e a r c h F o u n d a t i o n , 1 9 7 7 ) , p. 14. 49. P a r l a t o and F a v i n , P r i m a r y H e a l t h C a r e , p. 88. 50. M. P a r l e t t and D. H a m i l t o n , E v a l u a t i o n as i l l u m i n a t i o n :  a new a p p r o a c h to the s t u d y of i n n o v a t o r y programs ( E d i n b u r g h : U n i v e r s i t y o f E d i n b u r g h , C e n t r e f o r R e s e a r c h i n the E d u c a t i o n a l S c i e n c e s , 1 9 7 2 ) , p. 7. 51. N. C o c h r a n , " S o c i e t y as emergent and more t h a n r a t i o n a l : An e s s a y on t h e i n a p p r o p r i a t e n e s s o f program e v a l u a t i o n , " P o l i c y  S c i e n c e s 12 ( 1 9 8 0 ) : 1 2 6 . 52. N. C o c h r a n , " C o g n i t i v e p r o c e s s e s , s o c i a l mores, and t h e a c c u m u l a t i o n o f d a t a : Program e v a l u a t i o n and t h e s t a t u s quo," E v a l u a t i o n Q u a r t e r l y 2 ( 1978):343-358. 103 CHAPTER THREE ALTERNATIVE APPROACHES FOR CONDUCTING FIELD EVALUATIONS I n t r o d u c t i o n B e f o r e f u r t h e r i n v e s t m e n t s a r e made by governments and f u n d i n g a g e n c i e s i n the d e v e l o p m e n t , i m p l e m e n t a t i o n and e x p a n s i o n o f p r i m a r y h e a l t h c a r e p r o g r a m s , v a l i d and u s e f u l i n f o r m a t i o n a b o u t program p e r f o r m a n c e and i m p a c t a r e r e q u i r e d . But as t h e t h e s i s has p o i n t e d o u t , the r e s u l t s of many, i f n o t most, o f t h e e v a l u a t i o n s c o n d u c t e d to d a t e a r e n o t c r e d i b l e . The p r e v i o u s c h a p t e r i d e n t i f i e d s e v e r a l f a c t o r s w h i c h have l i m i t e d the v a l i d i t y and u t i l i t y of t h e r e s u l t s o f e v a l u a t i v e r e s e a r c h on p r i m a r y h e a l t h c a r e i n d e v e l o p i n g c o u n t r i e s . W h i l e some a r e r e l a t e d to d e s i g n c o n s i d e r a t i o n s , most a r e a s s o c i a t e d w i t h t h e t e c h n i q u e s and methods used i n c o n d u c t i n g an e v a l u a t i o n . T h e r e f o r e , i f t h e q u a l i t y of e v a l u a t i o n s i s to be im p r o v e d , t h e n s t r a t e g i e s to a d d r e s s t h e s e s h o r t c o m i n g s must be f o u n d . The p u r p o s e o f t h i s c h a p t e r i s to i d e n t i f y a l t e r n a t i v e d e s i g n s and t e c h n i q u e s f o r e v a l u a t i n g p r i m a r y h e a l t h c a r e programs i n d e v e l o p i n g c o u n t r i e s w h i c h may be a p p r o p r i a t e i n a d d r e s s i n g t h e s e l i m i t a t i o n s . The f i r s t i s s u e to be d e a l t w i t h c o n c e r n s t h e need f o r d e m o n s t r a t i n g c a u s a l i t y . The c h a p t e r t h e r e f o r e b e g i n s w i t h a d i s c u s s i o n of t h e l e v e l o f c e r t a i n t y 104 r e q u i r e d f o r f i e l d e v a l u a t i o n s of s o c i a l s e r v i c e programs c o n d u c t e d i n d e v e l o p i n g c o u n t r i e s . I t t h e n p r e s e n t s some a l t e r n a t i v e s t r a t e g i e s and t e c h n i q u e s f o r c o n d u c t i n g v a l i d and u s e f u l e v a l u a t i o n s o f p r i m a r y h e a l t h c a r e p r o j e c t s i n f i e l d s e t t i n g s . A l t h o u g h the c a s e s t u d i e s r e v i e w e d r e f e r p a r t i c u l a r l y to PHC p r o j e c t s c a r r i e d o u t i n d e v e l o p i n g c o u n t r i e s , the s u g g e s t i o n s made a r e meant to a p p l y as w e l l to f i e l d - b a s e d s o c i a l s e r v i c e e v a l u a t i o n s i n g e n e r a l . The L e v e l o f C e r t a i n t y R e q u i r e d i n E v a l u a t i v e R e s e a r c h Many o f the e v a l u a t i o n s t u d i e s r e f e r r e d to i n the p r e v i o u s c h a p t e r a t t e m p t e d to e s t a b l i s h a c a u s a l l i n k between a s e t of s e r v i c e s p r o v i d e d by p r i m a r y h e a l t h c a r e w o r k e r s and improvements i n t h e h e a l t h s t a t u s o f the s t u d y community. The e v a l u a t i o n s s o u g h t to p r o d u c e c o n c l u s i v e e v i d e n c e a b o u t the r e l a t i o n s h i p between p r i m a r y h e a l t h c a r e a c t i v i t i e s and h e a l t h s t a t u s . An e v a l u a t i o n s t u d y w h i c h p r o v i d e s c o n c l u s i v e e v i d e n c e p r o v i d e s t h e s o u n d e s t b a s i s f o r a c t i o n , as i t " e s t a b l i s h e s t h a t an e v e n t , p r o c e s s , o r outcome [was] p o s s i b l e and p r o b a b l e , [and] t h a t i n d e e d i t d i d o c c u r " . ( 1 ) F o r S h o r t e l l and R i c h a r d s o n ( 1 9 7 8 ) , r i g o r o u s e x p e r i m e n t a l program e v a l u a t i o n s a r e n e c e s s a r y when b e l i e f s about c a u s e / e f f e c t r e l a t i o n s h i p s a r e c e r t a i n . I n s u c h c a s e s , r i g o r o u s e x p e r i m e n t a l e v a l u a t i o n s [ a r e ] i m p o r t a n t i n program e v a l u a t i o n r e s e a r c h . . . as a 105 d e f e n s e a g a i n s t c r i t i c s [ o f t h e p r o g r a m ] who w i l l a t t e m p t t o r e f u t e t h e s c i e n t i f i c e v i d e n c e b y c r i t i c i z i n g t h e s t u d y d e s i g n , m e t h o d o l o g y a n d a n a l y s i s . T h e s t r o n g e r t h e d e s i g n u s e d . . . t h e g r e a t e r t h e a m m u n i t i o n a v a i l a b l e . . . t o d e f e n d t h e p r o g r a m ' s o p e r a t i o n , a s s u m i n g , o f c o u r s e , t h a t t h e r e s u l t s a r e b a s i c a l l y f a v o r a b l e . ( 2 ) H o w e v e r , w h i l e r i g o r o u s , e x p e r i m e n t a l e v a l u a t i v e r e s e a r c h d e s i g n s may p r o v i d e t h e s t r o n g e s t f o r m o f e v i d e n c e , t h e y a r e t h e m o s t e x p e n s i v e , d i f f i c u l t a n d t i m e - c o n s u m i n g t o c a r r y o u t . H o w e v e r , a s t h e p r e v i o u s c h a p t e r s h a s p o i n t e d o u t , t h e r e a r e n u m e r o u s p r a c t i c a l p r o b l e m s r e l a t e d t o c o n d u c t i n g p r o g r a m e v a l u a t i o n s i n d e v e l o p i n g c o u n t r i e s . A s a r e s u l t , w h a t a r e o r i g i n a l l y i n t e n d e d t o b e s c i e n t i f i c , c o n e l u s i o n - o r i e n t e d s t u d i e s o f t e n f a i l e d t o p r o d u c e t h e e x p e c t e d r e s u l t s . I n many c a s e s , t h e y s i m p l y r e p o r t t h a t v i l l a g e h e a l t h w o r k e r s h a d b e e n t r a i n e d a n d d e p l o y e d , a n d i n some c a s e s , t h a t p r i m a r y h e a l t h c a r e s e r v i c e s w e r e u t i l i z e d . Y e t , t h e n e e d t o p r o v e , t o d e m o n s t r a t e c o n c l u s i v e l y t h a t a c a u s a l r e l a t i o n s h i p e x i s t s , may n o t b e n e c e s s a r y w h e n o n e i s c o n c e r n e d w i t h t h e p r o d u c t i o n o f i n f o r m a t i o n f o r a d m i n i s t r a t i v e d e c i s i o n - m a k i n g . S e v e r a l a u t h o r s h a v e d i s c u s s e d t h i s i s s u e a t l e n g t h . ( 3 ) H o u s e c o n c l u d e s t h a t e x p e c t i n g e v a l u a t i o n t o p r o v i d e c o n c l u s i v e a n d c o m p e l l i n g e v i d e n c e i s " h o p i n g f o r m o r e t h a n e v a l u a t i o n c a n d e l i v e r " . B u t i f [ e v a l u a t i o n ] c a n n o t p r o d u c e t h e n e c e s s a r y , i t c a n p r o v i d e t h e c r e d i b l e , t h e p l a u s i b l e , a n d t h e p r o b a b l e . T h e r e s u l t s a r e l e s s t h a n c e r t a i n b u t s t i l l may be u s e f u l . . . e v a l u a t i o n p e r s u a d e s r a t h e r t h a n 106 c o n v i n c e s . . i s c r e d i b l e r a t h e r t h a n c e r t a i n . . . Once t h e b u r d e n o f c e r t a i n t y i s l i f t e d , t he p o s s i b i l i t i e s f o r i n f o r m e d a c t i o n a r e i n c r e a s e d r a t h e r t h a n d e c r e a s e d . (4) W h i l e t h e r e i s a need to c o n s i d e r e s t a b l i s h i n g c o n c l u s i v e e v i d e n c e , one c a n n o t a f f o r d to be o v e r l y z e a l o u s i n a t t e m p t i n g to e n s u r e t h e h i g h e s t d e g r e e o f c o n f i d e n c e and c r e d i b i l i t y i n the e v a l u a t i o n r e s u l t s . As has been p r e v i o u s l y s t a t e d , too r i g o r o u s an e v a l u a t i o n framework, p a r t i c u l a r l y i n a n o n - s u p p o r t i v e e n v i r o n m e n t , w i l l most l i k e l y l e a d to e r r o n e o u s r e s u l t s . As S m i t h e x p l a i n s , t h e r e a r e many l e v e l s o f c e r t a i n t y , and t h a t i n g e n e r a l , the l e v e l o f c e r t a i n t y o f an e v a l u a t i o n s t u d y i s d e p e n d e n t on t h e f o l l o w i n g c r i t e r i a : 1. t h a t a l t e r n a t i v e c o n c l u s i o n s have been e l i m i n a t e d ; 2. t h a t s y s t e m a t i c b i a s o r e r r o r has been e l i m i n a t e d ; 3. t h a t c o n t e x t u a l f a c t o r s have been a p p r o p r i a t e l y a c c o u n t e d f o r ; 4. t h a t a s s u m p t i o n s made a r e a c c u r a t e ; 5. t h a t t h e most r e l e v a n t v a r i a b l e s and r e l a t i o n s h i p s have been examined; 6. t h a t a l l s a l i e n t a u d i e n c e p e r s p e c t i v e s have been c o n s i d e r e d ; and, 7. t h a t e v i d e n c e was f i r s t - h a n d and m a t e r i a l r a t h e r t h e n c i r c u m s t a n t i a l and h e a r s a y . (5) 107 I n s t e a d , d e m o n s t r a t i n g a s t r o n g a s s o c i a t i o n between s e v e r a l f u n d a m e n t a l c h a r a c t e r i s t i c s o f p r i m a r y h e a l h c a r e s e r v i c e s and p e r s o n n e l ( i . e . t h e i r a v a i l a b i l i t y , a c c e s s i b i l i t y , p e r f o r m a n c e , e t c . ) and changes i n q u a l i t y o f l i f e and h e a l t h i n d i c e s may be s u f f i c i e n t . By d e f i n i n g e v a l u a t i o n as an a d m i n i s t r a t i v e t o o l r a t h e r t h a n as a s c i e n c e , l e s s t h a n c o n c l u s i v e e v i d e n c e may be b o t h a p p r o p r i a t e and s u f f i c i e n t to s a t i s f y the i n f o r m a t i o n needs o f t h o s e f o r whom the e v a l u a t i v e r e s e a r c h i s u n d e r t a k e n . Where t h e r e a r e l i m i t e d r e s o u r c e s f o r e v a l u a t i v e r e s e a r c h , l e s s t h a n c o n c l u s i v e e v i d e n c e may be t h e o n l y p o s s i b i 1 i t y . ( 6 ) S m i t h (1981) p r o p o s e s t h a t where t r e a t m e n t e f f e c t s a r e n o t e x p e c t e d to be h a r m f u l , where the i n c r e m e n t a l b e n e f i t to i n d i v i d u a l s i s h i g h , where the p o t e n t i a l r i s k to p a r t i c i p a n t s i s low, and where the program i s d e v e l o p m e n t a l , l e s s c e r t a i n t y i s needed f o r e s t a b l i s h i n g r e l a t i o n s h i p s between the p r o j e c t i n t e r v e n t i o n s and d e m o n s t r a t e d i m p a c t . He d e f i n e s two o t h e r l e v e l s o f e v i d e n c e w h i c h may be a p p r o p r i a t e to the needs o f program a d m i n i s t r a t o r s of h e a l t h c a r e p r o g r a m s . The f i r s t of t h e s e i s what he c a l l s s u g g e s t i v e e v i d e n c e , w h e r e i n i t can be s t a t e d t h a t 'X [ t h e h y p o t h e s i s ] i s p o s s i b l y t r u e ' . As S m i t h e x p l a i n s , an e v a l u a t i o n s t u d y t h a t p r o v i d e s o n l y s u g g e s t i v e e v i d e n c e : m e r e l y e s t a b l i s h e s t h a t c e r t a i n e v e n t s , p r o c e s s e s , or outcomes a r e p l a u s i b l e . . . [ i t ] i s o f t e n t h e o n l y t y p e o f e v i d e n c e a v a i l a b l e i n f o r m a t i v e s t a g e s of programs . . . [and] i s the weakest form o f e v i d e n c e , the e a s i e s t to o b t a i n . . . (7) 108 The s e c o n d l e v e l o f c e r t a i n t y i n h e a l t h program e v a l u a t i v e r e s e a r c h p r o p o s e d by S m i t h i s one f o r w h i c h i t c a n be s t a t e d t h a t 'X i s p r o b a b l y t r u e ' , c a l l e d p r e p o n d e r a n t e v i d e n c e . An e v a l u a t i o n s t u d y w h i c h p r o v i d e s p r e p o n d e r a n t e v i d e n c e e s t a b l i s h e s t h a t . . . an e v e n t , p r o c e s s or outcome . more l i k e l y o c c u r r e d t h a n n o t . A p r e p o n d e r a n t c l a i m i n d i c a t e s t h a t t h e w e i g h t of the i n f o r m a t i o n c o l l e c t e d to d a t e s u p p o r t s t h e c l a i m as p r o b a b l y b e i n g t r u e . ( 8 ) Most ' i n f o r m a t i o n - b a s e d ' d e c i s i o n s a r e a c t u a l l y made on the b a s i s o f p r e p o n d e r a n t e v i d e n c e due to the l a c k o f a v a i l a b l e d a t a to s u p p o r t c o n c l u s i v e e v i d e n c e . T h i s l e v e l o f e v a l u a t i o n p r o v i d e s a s t r o n g e r c l a i m t h a n s u g g e s t i v e e v i d e n c e on which to base d e c i s i o n s . Where r e s o u r c e s a r e l i m i t e d , and the need f o r c o n c l u s i v e e v i d e n c e may n o t be r e q u i r e d o r a p p r o p r i a t e . I n s u c h c a s e s , the use o f e i t h e r a p r e p o n d e r a n t or s u g g e s t i v e l e v e l e v a l u a t i o n s t u d y i s a d v o c a t e d on p r a c t i c a l g r o u n d s . A l t h o u g h t h e s e f a c t o r s may s u g g e s t what l e v e l o f c e r t a i n t y i s most d e s i r a b l e , d e t e r m i n i n g what l e v e l o f c e r t a i n t y i s a t t a i n a b l e w i l l r e q u i r e b a l a n c i n g t h e s e f a c t o r s a g a i n s t the human and f i n a n c i a l r e s o u r c e s and t h e t i m e a v a i l a b l e as w e l l as the p r e v a i l i n g f i e l d c o n d i t i o n s . ( 9 ) U s i n g p r o f e s s i o n a l judgement i n b a l a n c i n g t h e s e c r i t e r i a w i l l e n a b l e t h e e v a l u a t o r to j u s t i f y w h ether a h i g h e r or l o w e r o r d e r o f p r o o f i s r e q u i r e d . 109 Of c r i t i c a l i m p o r t a n c e i s c h o o s i n g the a p p r o a c h most s u i t a b l e f o r t h e s i t u a t i o n and c o n t e x t i n w h i c h th e e v a l u a t i o n i s to be c o n d u c t e d . The method s h o u l d f i t the program and the p r o b l e m r a t h e r t h a n t h e o t h e r way a r o u n d . The r e s e a r c h a p p r o a c h g e n e r a l l y s h o u l d be dynamic and a d a p t a b l e , and c o n c e n t r a t e on p r o v i d i n g t h e most r e l e v a n t and c o m p r e h e n s i v e i n f o r m a t i o n . L e s s e x p e r i m e n t a l , open-ended a p p r o a c h e s a l l o w i n g f o r s e r e n d i p i t o u s f i n d i n g s , may be t h e most a p p r o p r i a t e . ( 1 0 ) A l t e r n a t i v e A p p r o a c h e s i n E v a l u a t i v e R e s e a r c h P r o v i n g c o n c l u s i v e l y t h a t a p a r t i c u l a r i n t e r v e n t i o n has a p a r t i c u l a r outcome i s a d i f f i c u l t t a s k even under th e b e s t of c i r c u m s t a n c e s . The p r o b l e m s r e l a t e d to c a r r y i n g o u t v a l i d and m e a n i n g f u l e v a l u a t i o n s i n f i e l d s e t t i n g s has prompted the s e a r c h f o r a l t e r n a t i v e a p p r o a c h e s . As the a n a l y s i s c a r r i e d out i n the p r e v i o u s c h a p t e r p o i n t s o u t , t h e r e a r e s e v e r a l f a c t o r s w h i c h t e n d to l i m i t the v a l i d i t y and v a l u e o f d e v e l o p i n g c o u n t r y - b a s e d p r i m a r y h e a l t h c a r e e v a l u a t i o n s . T h e s e c a n be g r o u ped i n t o two b r o a d c a t e g o r i e s : e v a l u a t i v e r e s e a r c h d e s i g n ; and, e v a l u a t i o n m e t h o d o l o g y and t e c h n i q u e s . The r e m a i n d e r o f t h i s c h a p t e r w i l l a d d r e s s each o f t h e s e and s u g g e s t a l t e r n a t i v e a p p r o a c h e s w h i c h a r e e x p e c t e d to a d d r e s s t h e s e l i m i t a t i o n s . 110 i . E v a l u a t i v e R e s e a r c h D e s i g n s The e v a l u a t i v e r e s e a r c h d e s i g n s used by t h e r e s e a r c h teams to e v a l u a t e the p r i m a r y h e a l t h c a r e p r o j e c t s c i t e d i n the p r e v i o u s c h a p t e r were i n s u f f i c i e n t to d e m o n s t r a t e any s i g n i f i c a n t r e l a t i o n s h i p between the p r o j e c t i n t e r v e n t i o n s and changes i n h e a l t h s t a t u s o f t h e r e c i p i e n t p o p u l a t i o n s . The f a c t o r s r e s p o n s i b l e f o r l i m i t i n g the v a l i d i t y o f t h e s e d e s i g n s were i d e n t i f i e d and r e v i e w e d i n t h e p r e v i o u s c h a p t e r . B a s i c a l l y , the r i g o r o u s e x p e r i m e n t a l d e s i g n s , b e s i d e s b e i n g c o s t l y and t i m e - c o n s u m i n g , r e q u i r e so many c o n t r o l l i n g a s s u m p t i o n s t h a t t h e program b e i n g s t u d i e d becomes f r a g m e n t e d . I n t h e end, the r e s e a r c h r e s u l t s a r e u n r e a l i s t i c and i r r e l e v a n t to p r o g r a m m a t i c d e c i s i o n - m a k i n g . ( 1 1 ) F o r H a l l ( 1 9 7 9 ) , t h e e x p e r i m e n t a l e v a l u a t i v e r e s e a r c h d e s i g n t e n d s to o v e r s i m p l i f y s o c i a l r e a l i t y by e m p l o y i n g s u r v e y r e s e a r c h methods w h i c h e x t r a c t i n f o r m a t i o n from i n d i v i d u a l s i n i s o l a t i o n o f one a n o t h e r , and t h e n a g g r e g a t e t h e s e d a t a i n t o a s i n g l e s e t o f f i g u r e s . A c c o r d i n g to H a l l , t h i s s e r v e s to r e d u c e t h e c o m p l e x i t y and r i c h n e s s of human e x p e r i e n c e . S o c i a l r e s p o n s e s to p r o b l e m s by g r o u p s o f p e o p l e a r e n o t n e c e s s a r i l y t h e same as t h e t o t a l o f i n d i v i d u a l r e s p o n s e s o f p e o p l e a c t i n g a l o n e . To f a c i l i t a t e t h e a n a l y s i s o f t h i s i n f o r m a t i o n , r e s p o n s e s a r e o f t e n q u a n t i f i e d . However, what o f t e n r e s u l t s i s I l l an i l l u s i o n o f a c c u r a c y , r e s u l t i n g i n o f t e n u n c l e a r and m y s t i f y i n g r e s e a r c h . ( 1 2 ) B o r u c h and Gomez (1979) a r g u e t h a t g i v e n the r e a l i t i e s w h i c h a f f e c t the i m p l e m e n t a t i o n and e v a l u a t i o n o f f i e l d - b a s e d p r o g r a m s , much o f the e f f o r t i n v o l v e d i n e n s u r i n g p r e c i s e e x p e r i m e n t a l r e s e a r c h and t e s t i n g i s i n a c t u a l i t y a f u t i l e e x e r c i s e . They c o n t e n d t h a t s t a t i s t i c a l t e s t s of l e s s t h a n p e r f e c t r e l i a b i l i t y and s e n s i t i v i t y , measures of l e s s t h an p e r f e c t a c c u r a c y , and programs w h i c h a r e l e s s t h a n p e r f e c t l y i m plemented combine to compound t h e p o s s i b i l i t y (and p r o b a b i l i t y ) t h a t t h e s e e x p e r i m e n t - b a s e d models w i l l n o t d e m o n s t r a t e s i g n i f i c a n t program e f f e c t s , even i f t h e y o c c u r . G i v e n t h e r a r i t y o f c o n d u c t i n g a s u c c e s s f u l t r u e e x p e r i m e n t , and t h a t q u a s i - e x p e r i m e n t s r e q u i r e a d d i t i o n a l s t a t i s t i c a l m a n i p u l a t i o n s , t h e n t h e program i m p a c t would have to be enormous to be n o t i c e d . ( 1 3 ) The n o n - e x p e r i m e n t a 1 a p p r o a c h used by some o f the e v a l u a t i o n s c i t e d i n the p r e v i o u s c h a p t e r , on t h e o t h e r hand, a r e s t r u c t u r a l l y weak. The l a c k of c o n t r o l o r c o m p a r a t i v e p o p u l a t i o n s , h i s t o r i c a l d a t a , o r i n v e s t i g a t i o n s i n t o p a r a l l e l program a c t i v i t i e s and t h e i r p o s s i b l e e f f e c t on the h e a l t h of the s t u d y p o p u l a t i o n c o n s t i t u t e s e r i o u s s h o r t c o m i n g s . W h i l e t h e y a r e e a s i e r to c o n d u c t , t h e i r r e s u l t s p r o v i d e i n s i g n i f i c a n t and i r r e l e v a n t d a t a . The e v i d e n c e p r e s e n t e d s u g g e s t s t h a t n e i t h e r the r i g o r o u s 112 and complex e x p e r i m e n t a l r e s e a r c h d e s i g n s , n o r t h e s i m p l i s t i c n o n - e x p e r i m e n t a l d e s i g n s a r e a p p r o p r i a t e f o r c o n d u c t i n g f i e l d - b a s e d e v a l u a t i o n s . What a r e r e q u i r e d a r e r e s e a r c h d e s i g n s w h i c h a l l o w r e s e a r c h e r s to d e m o n s t r a t e s t r o n g a s s o c i a t i o n s between the p r e s e n c e and u t i l i z a t i o n of p r i m a r y h e a l t h c a r e s e r v i c e s and p e r s o n n e l and changes i n h e a l t h s t a t u s , under o f t e n d i f f i c u l t and demanding f i e l d c o n d i t i o n s , u s i n g the human and c a p i t a l r e s o u r c e s a t t h e i r d i s p o s a l . Q u a s i - e x p e r i m e n t a l r e s e a r c h d e s i g n s p e r h a p s o f f e r an a p p r o p r i a t e and r e a l i s t i c s o l u t i o n . A l t h o u g h t h e term was n o t c o i n e d u n t i l l a t e r , S t o u f f e r (1950) and C a m p b e l l (1957) were f i r s t to i n t r o d u c e the c o n c e p t o f q u a s i - e x p e r i m e n t a t i o n . ( 1 4 ) The c l a s s i c s t u d y of q u a s i - e x p e r i m e n t a 1 m o d e l s , w h i c h e v o l v e d from the f i e l d of e d u c a t i o n a l r e s e a r c h , was p r o d u c e d by C a m p b e l l and S t a n l e y ( 1 9 8 0 ) . ( 1 5 ) Q u a s i - e x p e r i m e n t a t i o n i s a r e s e a r c h s t r a t e g y d e s i g n e d to i n v e s t i g a t e c a u s a l r e l a t i o n s h i p s i n s i t u a t i o n s i n w h i c h t r u e e x p e r i m e n t a l d e s i g n s a r e i n a p p r o p r i t e o r d i f f i c u l t t o a p p l y . As Cook (1983) s t a t e s , the s t r a t e g y i s to t e s t whether a p a r t i c u l a r p a t t e r n o f r e s u l t s i s o b t a i n e d d e s p i t e the o p e r a t i o n of p o t e n t i a l v a l i d i t y t h r e a t s . ( 1 6 ) As C a m p b e l l and S t a n l e y (1980) e x p l a i n , the q u a s i - e x p e r i m e n t a 1 d e s i g n s a r e a p p l i c a b l e i n s i t u a t i o n s where t h e r e s e a r c h e r wants to i n t r o d u c e s o m e t h i n g a k i n to a t r u e e x p e r i m e n t a l d e s i g n i n t o t h e s c h e d u l i n g of d a t a c o l l e c t i o n p r o c e d u r e s ( e g . , the when and to whom o f 113 measurement), even though f u l l c o n t r o l o v e r the s c h e d u l i n g o f e x p e r i m e n t a l s t i m u l i ( t h e a b i l i t y to r a n d o m i z e e x p o s u r e s ) w h i c h makes a t r u e e x p e r i m e n t p o s s i b l e , i s l a c k i n g . ( 1 7 ) The s t r e n g t h of the q u a s i - e x p e r i m e n t a 1 p a r a d i g m i s t h a t i t p e r m i t s p r o b e s o f c a u s a l c o n n e c t i o n s i n f i e l d s e t t i n g s when or where the t r u e e x p e r i m e n t a l d e s i g n s a r e n o t f e a s i b l e , s u i t a b l e , o r d e s i r a b l e . The q u a s i - e x p e r i m e n t a 1 p a r a d i g m u t i l i z e s a b e f o r e / a f t e r d e s i g n , i n w h i c h the s t a t u s o f t h e outcome, o r dependent v a r i a b l e , i s measured i n e i t h e r a s i n g l e g r o u p , or i n b o t h the s t u d y and c o m p a r a t i v e g r o u p , b o t h b e f o r e and a f t e r the i n t r o d u c t i o n o f t h e i n t e r v e n t i o n . T h e s e d e s i g n s a r e i n t e n d e d f o r use i n s e t t i n g s i n w h i c h t h e r e s e a r c h e r has p a r t i a l b u t n o t c o m p l e t e c o n t r o l o v e r t h e a s s i g n m e n t p r o c e s s . L i k e the c o n v e n t i o n a l e x p e r i m e n t a l p a r a d i g m , q u a s i - e x p e r i m e n t s have i n t e r v e n t i o n s , outcome mea s u r e s , and e x p e r i m e n t a l p o p u l a t i o n s . However, t h e y do n o t use random a s s i g n m e n t to c r e a t e the c o m p a r i s o n f r o m w h i c h change c a u s e d by t h e i n t e r v e n t i o n i s i n f e r r e d . R a t h e r , a s s i g n m e n t to the t r e a t m e n t g r o u p i s by s e I f - s e 1 e c t i o n o r a d m i n i s t r a t o r f i a t r a t h e r t h a n by c h a n c e . The c o m p a r i s o n depends upon p o s s i b l y n o n e q u i v a 1 e n t g r o u p s t h a t d i f f e r f r o m e a c h o t h e r i n s e v e r a l c h a r a c t e r i s t i c s o t h e r than the e f f e c t of some i n t e r v e n t i o n . The t a s k c o n f r o n t i n g t h o s e who i n t e r p r e t t h e r e s u l t s f r o m q u a s i - e x p e r i m e n t s i s b a s i c a l l y one of a s s e s s i n g the e x t e n t o f d i f f e r e n c e s and s e p a r a t i n g the e f f e c t s o f a t r e a t m e n t from t h o s e due to the i n i t i a l 114 n o n c o m p a r a b 1 i 1 t y between the a v e r a g e u n i t s i n each e x p e r i m e n t g r o u p ; o n l y t h e e f f e c t s of t h e i n t e r v e n t i o n a r e of i n t e r e s t . To a c h i e v e t h i s s e p a r a t i o n of e f f e c t s , t h e r e s e a r c h e r has to e x p l a i n the s p e c i f i c t h r e a t s to v a l i d c a u s a l i n f e r e n c e t h a t random a s s i g n m e n t r u l e s o u t and t h e n i n some way d e a l w i t h t h e s e t h r e a t s . The i r r e l e v a n t c a u s a l f a c t o r s must t h e r e f o r e ' be i d e n t i f i e d and e x p l a i n e d w i t h i n the c e t e r i s p a r i b u s o f random a s s i g n m e n t . ( 1 8 ) The g o a l o f t h e q u a s i - e x p e r i m e n t a l p a r a d i g m i s to i d e n t i f y t h e f a c t o r s t h a t a r e presumed to make a d i f f e r e n c e to s p e c i f i c outcomes, even i f t h e s e i n v o l v e o n l y a s m a l l p a r t of a l a r g e r s y s t e m . Q u a s i - e x p e r i m e n t a t i o n i s moot on t h e i s s u e of the m u l t i v a r i a t e c o m p l e x i t y r e q u i r e d f o r c o m p l e t e c a u s a l u n d e r s t a n d i n g , c o m p l e t e c a u s a l e x p l a n a t i o n , and t o t a l p r e d i c t i o n . I t s e e k s o n l y to d i s c o v e r t h o s e p a r t s o f the r e a l w o r l d i n w h i c h c a u s a l r e l a t i o n s h i p s e x i s t and i n w h i c h m a n i p u l a t i n g one t h i n g p r e d i c t a b l y c a u s e s a n o t h e r to change. The t e s t i n g o f c a u s a l l a w s , the g e n e r a t i o n o f c a u s a l u n d e r s t a n d i n g a r e n o t i t s s t r o n g p o i n t s . ( 1 9 ) T h e r e a r e f o u r b a s i c q u a s i - e x p e r i m e n t a 1 r e s e a r c h d e s i g n s w hich a r e a p p r o p r i a t e to the p r e s e n t d i s c u s s i o n : time s e r i e s ; m u l t i p l e t i m e s e r i e s ; n o n e q u i v a l e n t c o m p a r i s o n g r o u p ; and, r e c u r r e n t i n s t i t u t i o n a l c y c l e . ( 2 0 ) The t i m e s e r i e s d e s i g n i s p e r h a p s th e most u n s o p h i s t i c a t e d of the q u a s i - e x p e r i m e n t a 1 d e s i g n s . I t a l l o w s r e s e a r c h e r s to c a l c u l a t e m u l t i p l e measures 115 b o t h b e f o r e and a f t e r the i n t r o d u c t i o n of the i n t e r v e n t i o n . The p r i m a r y a d v a n t a g e o f t h i s p a r t i c u l a r d e s i g n o v e r t h e s i n g l e p r e t e s t - p o s t t e s t a r e i t s c o n t r o l of m a t u r a t i o n a l e f f e c t s and p a r t i a l c o n t r o l o f h i s t o r i c a l e f f e c t s (due to the p r e s e n c e o f m u l t i p l e measures p r i o r to the i n t r o d u c t i o n o f the i n t e r v e n t i o n ) . The t i m e s e r i e s d e s i g n i s most u s e f u l when e v a l u a t i n g programs w h i c h a r e n o t s u b j e c t to s e a s o n a l v a r i a t i o n s . I f s u c h i s the c a s e , t h e n the o b s e r v a t i o n s p e i o d s must be more f r e q u e n t . The m u l t i p l e t i m e s e r i e s d e s i g n i s s i m i l a r to the p r e v i o u s d e s i g n , e x c e p t f o r t h e a d d i t i o n o f a c o m p a r i s o n g r o u p . T h i s p o p u l a t i o n must be as s i m i l a r as p o s s i b l e to t h e s t u d y p o p u l a t i o n ( m a x i m i z a t i o n o f e q u i v a l e n c y ) . The g r e a t e r the s i m i l a r i t y between the t r e a t m e n t and the c o m p a r i s o n g r o u p , the g r e a t e r t h e d e g r e e of c o n f i d e n c e w h i c h c a n be p l a c e d i n the r e s e a r c h r e s u l t s . A ssuming t h a t the t r e a t m e n t and c o m p a r i s o n p o p u l a t i o n s a r e matched, t h e n p o s t t e s t d i f f e r e n c e s between t h e s e two p o p u l a t i o n s w h i c h a r e n o t due to the e f f e c t o f the pogram under s t u d y must be r e l a t e d to s e l e c t i o n d i f f e r e n c e s . M u l t i p l e o b s e r v a t i o n s a r e c o l l e c t e d f o r b o t h g r o u p s p r i o r and s u b s e q u e n t to the i n t r o d u c t i o n of t h e i n t e r v e n t i o n . The major a d v a n t a g e o f t h i s d e s i g n o v e r t h e s i n g l e t i m e s e r i e s i s t h a t i t a l l o w s f o r g r e a t e r c o n t r o l f o r h i s t o r i c a l e f f e c t s . The n o n e q u i v a l e n t c o m p a r i s o n group d e s i g n i s s i m i l a r to the e x p e r i m e n t a l p r e t e s t - p o s t t e s t c o n t r o l group d e s i g n . However, i n 116 t h i s c a s e , t h e s u b j e c t s a r e n o t randomly a l l o c a t e d to the t r e a t m e n t p o p u l a t i o n and the c o n t r o l g r o u p . I n s t e a d , a s e c o n d p o p u l a t i o n i s s e l e c t e d w h i c h i s as s i m i l a r to the t r e a t m e n t p o p u l a t i o n as p o s s i b l e . W h i l e c o m p a r a b l e to t h e m u l t i p l e time s e r i e s a n a l y s i s d e s i g n , the n o n e q u i v a 1 e n t c o m p a r i s o n group d e s i g n has o n l y one p r e t e s t and one p o s t t e s t o b s e r v a t i o n , i n s t e a d o f m u l t i p l e o b s e r v a t i o n s . The r e c u r r e n t i n s t i t u t i o n a l c y c l e d e s i g n i s a c o m p o s i t e a p p r o a c h w h i c h compares a o n e - s h o t c a s e s t u d y ( p o s t t e s t o b s e r v a t i o n ) f o r t h e t r e a t m e n t p o p u l a t i o n w i t h a s i n g l e p r e s t e s t - p o s t t e s t d e s i g n f o r the c o m p a r i s o n p o p u l a t i o n . I t a l l o w s t h e r e s e a r c h e r s to compare the c h a r a t e r i s t i c s o f a p o p u l a t i o n w h i c h has been i n v o l v e d i n t h e i n n o v a t i v e program f o r some time w i t h t h o s e o f a p o p u l a t i o n w h i c h i s a b o u t to be e x p o s e d to the same program. S i n g u l a r i l y , the i n d i v i d u a l components o f t h i s d e s i g n a r e s u b j e c t to s e v e r a l t h r e a t s to v a l i d i t y . But when combined, t h i s d e s i g n c o n t r o l s f o r a number o f t h r e a t s to v a l i d i t y , p a r t i c u l a r l y h i s t o r i c a l , t e s t i n g , i n s t r u m e n t a t i o n and a t t r i t i o n e f f e c t s . However, m a t u r a t i o n and r e g r e s s i o n e f f e c t s r e m a i n as p o t e n t i a l a l t e r n a t i v e e x p l a n a t i o n s f o r changes i n p o p u l a t i o n c h a r a c t e r i s t i c s . The e v a l u a t i v e r e s e a r c h d e s i g n s used i n the f i v e p r i m a r y h e a l t h c a r e p r o j e c t e v a l u a t i o n c a s e s t u d i e s c i t e d i n the p r e v i o u s c h a p t e r c o u l d have been a d a p t e d to a q u a s i - e x p e r i m e n t a l d e s i g n f o r m a t . F o r example, the o r i g i n a l 117 i n t e n t of the r e s e a r c h e r s i n the I r a n and H a i t i c a s e s t u d i e s was to use an e x p e r i m e n t a l c a s e - c o n t r o l d e s i g n . But due to s e v e r a l f a c t o r s n o t i d e n t i f i e d i n the p r o j e c t documents, t h e s e d e s i g n s were abandoned. However, even t h e m o d i f i e d d e s i g n s were f a u l t y , s i n c e the t r e a t m e n t and c o m p a r i s o n p o p u l a t i o n s were n o t matched. C o n s e q u e n t l y , - d i f f e r e n c e s i n p o s t t e s t measures between t h e s e two p o p u l a t i o n s c an n o t be a t t r i b u t e d to the e f f e c t s of the p r o j e c t , as too many o t h e r f a c t o r s w h i c h c o u l d c o n c e i v a b l y a f f e c t program i m p a c t were u n a c c o u n t e d f o r . Had t h e r e s e a r c h e r s matched t h e s t u d y and c o m p a r i s o n p o p u l a t i o n s , a n o n e q u i v a l e n t c o m p a r i s o n group d e s i g n would have r e s u l t e d ( a s s u m i n g , of c o u r s e , t h a t p r e t e s t and p o s t t e s t o b s e r v a t i o n s were made f o r b o t h p o p u l a t i o n g r o u p s ) . The a d d i t i o n of m u l t i p l e p r e t e s t and p o s t t e s t o b s e r v a t i o n p o i n t s i n t h e o t h e r c a s e s t u d y c o u l d have t r a n s f o r m e d the e v a l u a t i v e r e s e a r c h d e s i g n s used from s i n g l e p r e t e s t - p o s t t e s t d e s i g n s to s i n g l e t i m e s e r i e s d e s i g n s . As was p o i n t e d o u t i n the C o l o m b i a and P a r a g u a y c a s e s t u d i e s , the a b s e n c e o f h i s t o r i c a l d a t a f o r the s e l e c t e d measures l i m i t e d the r e l e v a n c e of the s t u d y f i n d i n g s . The a d d i t i o n o f c o m p a r i s o n g r o u p s would have r e s u l t e d i n a m u l t i p l e time s e r i e s d e s i g n . I f the c o m p a r i s o n group was matched w i t h the t r e a t m e n t p o p u l a t i o n , t h e n a n o n e q u i v a l e n t c o m p a r i s o n group d e s i g n would have r e s u l t e d . I n any c a s e , the use o f a d d i t i o n a l o b s e r v a t i o n p o i n t s a n d / o r matched c o m p a r i s o n g r o u p s would have s e r v e d to 118 s t r e n g t h e n t h e e v a l u a t i v e r e s e a r c h d e s i g n s of t h e s e p a r t i c u l a r s t u d i e s . The e x t e n s i o n o f the t i m e i n t e r v a l o v e r w h i c h the o b s e r v a t i o n s a r e t a k e n i s a l s o an i m p o r t a n t c o n s i d e r a t i o n . As p o i n t e d o'ut i n t h e p r e c e e d i n g a n a l y s i s , the p r o j e c t d u r a t i o n s t e n d e d to be o f i n s u f f i c i e n t l e n g t h to a d e q u a t e l y d e m o n s t r a t e a s i g n i f i c a n t o r p o s i t i v e l i n k between the p r i m a r y h e a l t h c a r e i n t e r v e n t i o n s and changes i n h e a l t h i n t h e s t u d y community. Change i n h e a l t h i s a complex and o f t e n l e n g t h y p r o c e s s . Nor i s i t a l i n e a r p r o c e s s , w h e r e i n changes i n h e a l t h s t a t u s a r e a d i r e c t f u n c t i o n o f t h e a v a i l a b i l i t y o f p r e v e n t i v e and c u r a t i v e h e a 1 t h - r e l a t e d a c t i v i t i e s . I f a r e a l i s t i c t i m e t a b l e f o r a s s e s s i n g changes i n h e a l t h i s to be a d o p t e d , t h e n a more p r a c t i c a l and r e a l i s t i c p r o j e c t p e r i o d t h a n n o r m a l l y used i s r e q u i r e d . B u t a l o n g e r p e r i o d between the p o i n t i n time when th e i n t e r v e n t i o n s a r e i n t r o d u c e d and t h e p o i n t i n time when the f i n a l p r o j e c t s u r v e y s a r e made w i l l n e c e s s a r i l y i n c r e a s e the number of o b s e v a t i o n p o i n t s and t h e t a s k s of t h o s e i n v o l v e d i n d a t a c o l l e c t i o n and a n a l y s i s . The c o s t s a s s o c i a t e d w i t h the p r o j e c t e v a l u a t i o n w i l l a l s o i n c r e a s e . However, by i n c r e a s i n g the t i m e f r a m e f o r the e v a l u a t i o n , the v a l i d i t y and s i g n i f i c a n c e o f t h e r e s e a r c h d e s i g n w i l l be e n h a n c e d . But even a 'good' d e s i g n can g u a r a n t e e the v a l i d i t y , s i g n i f i c a n c e , and r e l e v a n c e of r e s e a r c h r e s u l t s . As the n e x t s e c t i o n o f t h i s c h a p t e r w i l l d e m o n s t r a t e , the methods and 119 t e c h n i q u e s used to c o l l e c t d a t a and the n a t u r e o f the d a t a can must a l s o be a d d r e s s e d . i i . E v a l u a t i o n Methods and T e c h n i q u e s B e s i d e s e n s u r i n g t h a t t h e e v a l u a t i v e r e s e a r c h d e s i g n s a r e a p p r o p r i a t e f o r d e m o n s t r a t i n g program i m p a c t , and c o n d u c i v e to t h e r e a l i t i e s of the p r e v a i l i n g f i e l d c o n d i t i o n s , the r e s e a r c h e r s must a l s o e n s u r e t h a t t h e methods and t e c h n i q u e s u s e d , p a r t i c u l a r l y w i t h r e s p e c t to the n a t u r e and means of c o l l e c t i n g d a t a , a r e a l s o a p p r o p r i a t e . The aim s h o u l d be to e n s u r e t h a t t h e methods and t e c h n i q u e s u s e d s e r v e to s t r e n g t h e n the c a p a c i t y o f t h e e v a l u a t i v e r e s e a r c h d e s i g n to d e m o n s t r a t e program i m p a c t . The n a t u r e and number of measures to be used i n the e v a l u a t i o n a r e an i m p o r t a n t c o n s i d e r a t i o n . The s e l e c t e d c a s e s t u d i e s d i d n o t e s t a b l i s h any l i n k a g e between the a c t i v i t i e s of the c o m m u n i t y - l e v e 1 h e a l t h w o r k e r s and the measures s e l e c t e d . F o r example, w h i l e i n f a n t m o r t a l i t y may be c o n s i d e r e d a s e n s i t i v e i n d i c a t o r , i t i s a f f e c t e d by more t h a n j u s t the a c t i o n s o f t h e community h e a l t h worker o r the p r i m a r y h e a l t h c a r e p r o j e c t a c t i v i t i e s . A l t h o u g h a PHC program may i n c l u d e an i m m u n i z a t i o n campaign to combat c e r t a i n d i s e a s e s , o r n u t r i t i o n e d u c a t i o n c l a s s e s , u n s a n i t a r y e n v i r o n m e n t a l c o n d i t i o n s and/or the l a c k o f a d e q u a t e n u t r i t i o u s f o o d may have a g r e a t e r and 120 o p p o s t i v e e f f e c t on i n f a n t m o r t a l i t y . A d d i t i o n a l l y , h e a l t h c a n n o t be p e r c e i v e d as a s i n g u l a r a c t i v i t y or s e c t o r . H e a l t h i s an i n t r i n s i c component o f the d e v e l o p m e n t p r o c e s s , t h e u l t i m a t e o b j e c t i v e o f w h i c h i s the e r a d i c a t i o n o f p o v e r t y . The n a t u r e o f the i n t e r r e l a t i o n s h i p among the v a r i o u s m a n i f e s t a t i o n s o f p o v e r t y i s such t h a t one f a c t o r i n f l u e n c e s a l l o t h e r s i n b o t h p o s i t i v e and n e g a t i v e s y n e r g i s t i c e f f e c t s . ( 2 1 ) As H e n d r a t t a (1979) s t a t e s : The i m p l i c a t i o n s on t h e d e v e l o p m e n t of a h e a l t h program a r e c l e a r . H e a l t h i s j u s t one e l e m e n t i n the t o t a l d e v e l o p m e n t p i c t u r e ; a c c o r d i n g l y , h e a l t h p r o b l e m s c a n n o t be t a c k l e d i n i s o l a t i o n from the o t h e r e l e m e n t s . I n o t h e r words, t h e a t t a c k on t h e p r o b l e m s of h e a l t h s h o u l d be mounted as an i n t e g r a l p a r t of a b r o a d f r o n t a l a t t a c k d i r e c t e d a g a i n s t the m u l t i f a r i o u s forms o f p o v e r t y . T h e r e can be no hope f o r a s i g n i f i c a n t and l a s t i n g improvement i n h e a l t h i n the a b s e n c e o f a s i m u l t a n e o u s improvement i n a l l o t h e r s e c t o r s . (22) The l a c k o f c o n s i d e r a t i o n o f d e v e l o p m e n t a c t i v i t i e s i n o t h e r s e c t o r s w h i c h p a r a l l e l t h o s e r e p o r t e d i n the c a s e s t u d i e s a l s o s e r v e to l i m i t t h e r e l e v a n c e o f the e v a l u a t i o n r e s u l t s . F o r example, i n t h e C o l o m b i a n c a s e , the h e a l t h a c t i v i t i e s were one component o f a l a r g e r community d e v e l o p m e n t p r o j e c t . Y e t no e f f o r t was made to examine the p o t e n t i a l e f f e c t o f d e v e l o p m e n t a c t i v i t i e s i n o t h e r economic s e c t o r s w i t h changes i n h e a l t h . T h e r e a r e a l s o f a c t o r s i n t e r n a l to the p r o c e s s of d e l i v e r i n g h e a l t h c a r e s e r v i c e s w h i c h may a f f e c t outcome, but w h i c h a r e n o t n e c e s s a r i l y i d e n t i f i e d or a c c o u n t e d f o r d u r i n g 121 the c o u r s e of t h e e v a l u a t i o n . T h e s e i n c l u d e consumer a t t i t u d e s to p r o j e c t a c t i v i t i e s and p e r s o n n e l , or h e a 1 t h - s p e c i f i c b e h a v i o u r s w h i c h a f f e c t p r a c t i c e s , and u l t i m a t e l y , h e a l t h . T h i s b r o a d e r frame o f r e f e r e n c e i s n e c e s s a r y i n o r d e r to a c c o u n t and c o n t r o l f o r f a c t o r s w h i c h c o u l d a f f e c t the r e l a t i o n s h i p between the p r o j e c t ' s d e p e n d e n t and i n d e p e n d e n t v a r i a b l e s . However, s i m p l y i n c r e a s i n g t h e number o f .measures w i l l n o t n e c e s s a r i l y r e d r e s s t h e i s s u e , as t h i s may o n l y s e r v e to i n c r e a s e the c o m p l e x i t y and the c o s t s a s s o c i a t e d w i t h t h e e v a l u a t i o n . C are must be e x e r c i s e d to e n s u r e t h a t the most r e l e v a n t and s e n s i t i v e measures a r e s e l e c t e d . Such an a p p r o a c h r e q u i r e s a r e d e f i n i t i o n o f the f o c u s o f t h e e x e r c i s e . One method w h i c h c o u l d be used to h i g h l i g h t the e x i s t e n c e of t h e s e f a c t o r s , as w e l l as the d i r e c t i o n and d e g r e e o f t h e i r r e l a t i o n s h i p to changes i n h e a l t h s t a t u s , i s the c a s e s t u d y . The c a s e s t u d y a p p r o a c h c o n c e n t r a t e s on the program p r o c e s s e s and on how p e o p l e v i e w the program. T h i s a p p r o a c h p r o v i d e s an u n d e r s t a n d i n g o f the p e r c e p t i o n s o f t h o s e who a r e f a m i l i a r w i t h t h e program. I t i s n a t u r a l i s t i c i n v e s t i g a t i o n . The aim i s to e s t a b l i s h c r e d i b i l i t y and c o n f i d e n c e i n the r e p o r t e d outcomes. S t a k e c a l l s c a s e s t u d i e s a r e s p o n s i v e a p p r o a c h to e v a l u a t i o n , w h i c h e l i c i t s o p i n i o n , f i n d i n g o u t what i s o f v a l u e to the a u d i e n c e . ( 2 3 ) Case s t u d i e s p r o v i d e r i c h and p e r s u a s i v e i n f o r m a t i o n t h a t i s n o t a v a i l a b l e from o t h e r a p p r o a c h e s . A s t r e n g t h o f t h i s a p p r o a c h i s t h a t i t a l l o w s the r e p r e s e n t a t i o n 122 o f d i v e r s e p o i n t s o f v i e w , and can a c c o u n t f o r f a c t o r s which a f f e c t outcome, b u t w h i c h would n o t n e c e s s a r i l y be a c c o u n t e d f o r i n t h e e v a l u a t i o n . An e q u a l l y i m p o r t a n t e l e m e n t of t h i s a p p r o a c h i s t h a t i t p r o v i d e s an o p p o r t u n i t y f o r s e r v i c e p r o v i d e r s and community members who use the s e r v i c e s to become d i r e c t l y i n v o l v e d i n an a s s e s s m e n t of the p e r f o r m a n c e and a c c e p t a b i l i t y o f the p r i m a r y h e a l t h c a r e s e r v i c e s , an i m p o r t a n t c o n s i d e r a t i o n i f one of he aims of the program i s to i m p l i c a t e the community i n the d e v e l o p m e n t and o p e r a t i o n o f t h e s e s e r v i c e s . The e v a l u a t o r makes no a t t e m p t to m a n i p u l a t e , c o n t r o l , o r e l i m i n a t e s i t u a t i o n a l v a r i a b l e s . The e x i s t i n g and complex s o c i o - e c o n o m i c , c u l t u r a l and e c o l o g i c a l e n v i r o n m e n t i s t a k e n as a g i v e n . The c a s e s t u d y a p p r o a c h i s b a s e d on the f o l l o w i n g a s s u m p t i o n s : the i m p o r t a n c e o f u n d e r s t a n d i n g p e o p l e and programs i n c o n t e x t ; a commitment to s t u d y n a t u r a l l y o c c u r i n g phenomena w i t h o u t i n t r o d u c i n g e x t e r n a l c o n t r o l s or m a n i p u l a t i o n ; and the a s s u m p t i o n t h a t u n d e r s t a n d i n g emerges most m e a n i n g f u l l y from an i n d u c t i v e a n a l y s i s o f open-ended, d e t a i l e d , d e s c r i p t i v e , and a t t i m e s a n e c d o t a l d a t a g a t h e r e d t h r o u g h d i r e c t c o n t a c t w i t h the program and i t s p a r t i c i p a n t s . ( 2 4 ) As House s t a t e s , the c a s e s t u d y a p p r o a c h " a l l o w s one to come to g r i p s w i t h the i n n e r w o r k i n g s o f t h e program".(25 ) I t i s a s t r a t e g y w h i c h a d v o c a t e s ' p r o g r e s s i v e f o c u s s i n g " . The t a s k of t h e e v a l u a t o r i s to p r o v i d e a c o m p r e h e n s i v e u n d e r s t a n d i n g o f t h e complex r e a l i t i e s 123 s u r r o u n d i n g , an p o s s i b l y a f f e c t i n g , b o t h the p r o c e s s of i m p l e m e n t a t i o n and t h e i m p a c t o f the program. The e v a l u a t o r t r i e s to s h a r p e n d i s c u s s i o n , d i s e n t a n g l e c o m p l e x i t i e s , and i s o l a t e the s i g n i f i c a n t from the t r i v i a l . ( 2 6 ) As F o s t e r (1969) s t a t e s , a more h o l i s t i c a p p r o a c h to e v a l u a t i o n i s more than s e e k i n g and e x p l a i n i n g u n i v a r i a t e c a u s e s and e f f e c t s . The use o f c a s e s t u d i e s i n e v a l u a t i o n i s an i t e r a t i v e p r o c e s s , one w h i c h r e q u i r e s an u n u s u a l l y b r o a d and f l e x i b l e f i e l d r e s e a r c h m e t h o d o l o g y b a s e d on a h o l i s t i c v i e w of s o c i e t y and c u l t u r e . . . Where the t e c h n i c a l , s o c i a l , c u l t u r a l , e c o n o m i c , p s y c h o l o g i c a l , and o t h e r p e r t i n e n t f a c t o r s a r e . . . n o t r e c o g n i z e d i n a d v a n c e , t h i s e x p l o r a t o r y q u a l i t y i s e n o r m o u s l y a d v a n t a g e o u s . I t v a s t l y i n c r e a s e s t h e i n v e s t i g a t o r ' s c h a n c e s o f h i t t i n g upon the c r i t i c a l e l e m e n t i n any s i t u a t i o n . (27) The i n t e g r a t i o n o f q u a l i t a t i v e and q u a n t i t a t i v e methods i n e v a l u a t i o n can p r o v i d e a more c o m p r e h e n s i v e a n a l y s i s o f the r e l a t i o n s h i p between program a c t i v i t i e s and t h e i r i m p a c t . But as P a t t o n (1980) p o i n t s o u t , t h e r e i s a need to c a r e f u l l y b a l a n c e th e p r o p o r t i o n of q u a n t i t a t i v e and q u a l i t a t i v e d a t a i n e v a l u a t i n g p r o g r a m s . As he s t a t e s : I n many ways the r e a l t r a d e - o f f between q u a n t i t a t i v e methods and q u a l i t a t i v e methods i s a t r a d e - o f f between b r e a d t h and d e p t h . Q u a l i t a t i v e methods p e r m i t th e e v a l u a t o r to s t u d y s e l e c t e d i s s u e s i n d e p t h and d e t a i l ; t h e f a c t t h a t d a t a c o l l e c t i o n i s n o t c o n s t r a i n e d by p r e d e t e r m i n e d c a t e g o r i e s of a n a l y s i s c o n t r i b u t e s to the d e p t h and d e t a i l o f q u a n t i t a t i v e d a t a . Q u a n t i t a t i v e methods, on the o t h e r hand, r e q u i r e the use o f s t a n d a r d i z e d s t i m u l u s so t h a t a l l e x p e r i e n c e s of p e o p l e a r e l i m i t e d to c e r t a i n r e s p o n s e 124 c a t e g o r i e s . The a d v a n t a g e o f the q u a n t i t a t i v e a p p r o a c h i s t h a t i t i s p o s s i b l e to measure the r e a c t i o n s o f many s u b j e c t s i n a l i m i t e d s e t o f q u e s t i o n s , thus f a c i l i t a t i n g c o m p a r i s o n and s t a t i s t i c a l a g g r e g a t i o n o f the d a t a . By c o n t r a s t , q u a l i t a t i v e methods t y p i c a l l y p r o d u c e a w e a l t h o f d e t a i l e d d a t a a b o u t a s m a l l e r number of p e o p l e and c a s e s . (28) The b a l a n c e between the ' r i g h t amount' and ' r i g h t t y p e ' of q u a n t i t a t i v e and q u a l i t a t i v e methods i s an i m p o r t a n t i s s u e . As P a t t o n (1980) p o i n t s o u t , i t i s q u a l i t a t i v e d a t a t h a t a r e most o f t e n made to b e a r t h e l a r g e r b u r d e n o f p r o o f . Q u a l i t a t i v e d a t a t e n d to be r e j e c t e d i f t h e y do n o t s u p p o r t the q u a n t i t a t i v e d a t a . ( 2 9 ) The s u c c e s s f u l i n t e g r a t i o n of q u a l i t a t i v e and q u a n t i t a t i v e methods c a n be a d i f f i c u l t and complex t a s k . However, t h e r i c h n e s s o f d e t a i l w i t h r e s p e c t to e x p l a n a t i o n s of 'how' and 'why' program o u t p u t s and p r o c e s s e s a f f e c t program outcome w h i c h r e s u l t f r o m a c o m b i n a t i o n o f t h e s e two m e t h o d o l o g i e s may be w o r t h t h e p r i c e . The use o f c a s e s t u d i e s as s u p p l e m e n t s to the q u a n t i t a t i v e d a t a c o l l e c t e d may have enhanced the c r e d i b i l i t y o f the r e s u l t s o f the f i v e c a s e s t u d y e v a l u a t i o n s . W h i l e some o f the c a s e s t u d i e s c i t e d i n the p r e v i o u s c h a p t e r d i d c o l l e c t and a n a l y s e some q u a l i t a t i v e d a t a , i t t e n d e d to be r e l a t e d to s u r v e y s of the q u a l i t y o f p r i m a r y h e a l t h J c a r e s e r v i c e s p r o v i d e d and p e r s o n n e l , b a s e d on the p e r c e p t i o n of u s e r s . Such s u r v e y s t e n d e d to be c 1 o s e d - e n d e d , and r a r e l y a l l o w e d f o r an expanded e x p l a n a t i o n . The s i g n i f i c a n c e o f the i n f o r m a t i o n c o l l e c t e d i s a l s o q u e s t i o n a b l e . E x p e c t i n g a n y t h i n g o t h e r t h a n a p o s i t i v e 125 answer i n r e p l y to a q u e s t i o n as to whether v i l l a g e r s a r e s a t i s f i e d t h a t PHC p e r s o n n e l a r e a v a i l a b l e i n a s i t u a t i o n where no h e a l t h p e r s o n n e l p r e v i o u s l y e x i s t e d would be n a i v e . T h i s a l s o a p p l i e s to t h e r e s p o n s e s g i v e n i n r e p l y to q u e s t i o n s a b o u t whether t h e community-based p e r s o n n e l s h o u l d r e m a i n . G i v e n t h a t t h e i r p r i m a r y p u r p o s e a p p e a r s to have been to p r o v i d e c u r a t i v e emergency c a r e , t h e i r a b s e n c e would mean h a v i n g to walk l o n g d i s t a n c e s to h e a l t h c l i n i c s to o b t a i n t r e a t m e n t . Such d a t a added r e l a t i v e l y l i t t l e to the s i g n i f i c a n c e and u t i l i t y of the q u a n t i t a t i v e d a t a w h i c h was c o l l e c t e d . The a n a l y s i s o f d a t a w h i c h examined t h e r e l a t i o n s h i p between t h e r a n g e of community d e v e l o p m e n t a c t i v i t i e s b e i n g implemented i n t h e s t u d y a r e a s and the e f f e c t ( b o t h r e a l and p e r c e i v e d ) o f t h e s e a c t i v i t i e s on the h e a l t h and w e l f a r e of the i n h a b i t a n t s o f t h e s t u d y c o m m u n i t i e s would have c o n s t i t u t e d a v a l u a b l e a d d i t i o n to the q u a n t i t a t i v e d a t a c o l l e c t e d . F o r example, i n t h e C o l o m b i a n and H a i t i a n c a s e s , r e g i o n a l community d e v e l o p m e n t programs were b e i n g c a r r i e d o u t i n the s t u d y a r e a s s i m u l t a n e o u s l y to the p r i m a r y h e a l t h c a r e f i e l d a c t i v i t i e s . Y e t n e i t h e r e v a l u a t i o n examined t h e r e l a t i o n s h i p between t h e s e a c t i v i t i e s , n o r t h e i r s i n g u l a r or c u m u l a t i v e e f f e c t on the w e l l - b e i n g of the PHC s t u d y p o p u l a t i o n . I n the T h a i p r o j e c t , c a s e s t u d i e s o f some o f the s t u d y v i l l a g e s may have p r o v i d e d a more c o m p r e h e n s i v e p e r s p e c t i v e a b o u t h i s t o r i c a l e conomic, s o c i a l , c u l t u r a l and p h y s i c a l e n v i r o n m e n t a l f a c t o r s w h i c h 126 a f f e c t t h e h e a l t h o f the i n h a b i t a n t s . A l i n k a g e between t h e s e f a c t o r s and t h e a c t i v i t i e s o f t h e v i 1 l a g e - l e v e 1 h e a l t h w o r k e r s c o u l d have e s t a b l i s h e d w h e t h e r , and how t h e i r h e a 1 t h - r e l a t e d a c t i v i t i e s were r e s p o n d i n g to the m a n i f e s t a t i o n s of t h e s e f a c t o r s . The same h o l d s t r u e f o r the P a r a g u a y a n and I r a n i a n c a s e s . T h e r e a r e a m y r i a d o f s u r v e y t e c h n i q u e s f o r c o l l e c t i n g b o t h q u a n t i t a t i v e and q u a l i t a t i v e d a t a . D i r e c t i n t e r v i e w t e c h n i q u e s ( q u e s t i o n n a i r e s ) , c o u n t i n g s u r v e y s , and the e x a m i n a t i o n o f e s t a b l i s h e d r e c o r d systems ( p r e c o 1 l e c t e d d a t a ) a r e p r o b a b l y the two most e x t e n s i v e l y u s e d t e c h n i q u e s . As p r e v i o u s l y d i s c u s s e d , t h e f i v e c a s e s t u d i e s c i t e d i n the p r e v i o u s c h a p t e r t e n d e d to c o l l e c t q u a n t i t a t i v e d a t a t h r o u g h t h e s e t r a d i t i o n a l a p p r o a c h e s . D i r e c t i n t e r v i e w t e c h n i q u e s were us e d e x t e n s i v e l y f o r c o l l e c t i n g d e m o g r a p h i c d a t a and i n f o r m a t i o n on the p r e s e n c e o f l a t r i n e s , i m p r oved w e l l s , and g e n e r a l e n v i r o n m e n t a l c o n d i t i o n s i n the s t u d y a r e a s . D a t a drawn from c l i n i c r e c o r d s were us e d to c a l c u l a t e r a t e s o f d i s e a s e and to v e r i f y the n a t u r e and f r e q u e n c y o f r e f e r r a l s made by the community h e a l t h w o r k e r s . The o b j e c t i v e of t h e s e s u r v e y s was to c o l l e c t i n f o r m a t i o n to c a l c u l a t e r a t e s and f r e q u e n c i e s . But t h e c o l l e c t i o n o f q u a l i t a t i v e d a t a may r e q u i r e a somewhat d i f f e r e n t a p p r o a c h . T h e r e a r e i n s t a n c e s where the c o l l e c t i o n o f d a t a on c e r t a i n a c t i v i t i e s may be c u l t u r a l l y u n a c c e p t a b l e . I n s u c h c a s e s , i t would be e x p e c t e d t h a t the 127 e v a l u a t i o n would n o t c o l l e c t s u c h d a t a on e t h i c a l g r o u n d s . However, an e x p l a n a t i o n s h o u l d be made i n the f i n a l r e p o r t to t h i s e f f e c t . F o r o t h e r a c t i v i t i e s , the d i r e c t i n t e r v i e w t e c h n i q u e s may be t o t a l l y i n a p p r o p r i a t e . Examples i n c l u d e e a t i n g and p e r s o n a l h y g i e n e b e h a v i o u r . I n s u c h c a s e s , an a l t e r n a t i v e a p p r o a c h i s p r e s c r i b e d . P a r t i c i p a n t o b s e r v a t i o n i s u s e f u l i n c a s e s t u d i e s where one i s i n t e r e s t e d i n d e t e r m i n i n g t h e a t t i t u d e s and v i e w s of p a r t i c u l a r i n d i v i d u a l s w i t h i n a p o p u l a t i o n , b u t d i r e c t i n t e r v i e w t e c h n i q u e s may n o t be p r a c t i c a l o r a p p r o p r i a t e . The a s s u m p t i o n b e h i n d t h i s r e s e a r c h method i s t h a t i n o r d e r to u n d e r s t a n d t h e p e o p l e under s t u d y , one must g e t as c l o s e as p o s s i b l e to them i n t h e i r n a t u r a l s e t t i n g . O n l y by e x t e n s i v e p a r t i c i p a t i o n w i t h the group under s t u d y can one u n d e r s t a n d how t h e y v i e w t h e m s e l v e s , the program b e i n g e v a l u a t e d , and t h e i r i n t e r r e l a t i o n s h i p s and a t t i t u d e s towards the program. P a r t i c i p a n t o b s e r v a t i o n i n c l u d e s a v a r i e t y o f d a t a c o l l e c t i o n methods s u c h as i n d i r e c t o b s e r v a t i o n , i n f o r m a l i n t e r v i e w i n g , f o r m a l i n t e r v i e w i n g and document a n a l y s i s . Good f i e l d work uses s e v e r a l o f t h e s e t e c h n i q u e s . ( 3 0 ) F o r t h e p u r p o s e s o f e v a l u a t i v e r e s e a r c h on p r i m a r y h e a l t h c a r e p r o g r a m s , p a r t i c i p a n t o b s e r v a t i o n may be u s e f u l f o r d e t e r m i n i n g t h e f o l l o w i n g : 128 1. How the program a c t u a l l y o p e r a t e s a t the l o c a l l e v e l , p a r t i c u l a r l y w i t h r e s p e c t to how l o c a l e n v i r o n m e n t a l f a c t o r s a f f e c t e d i t s o p e r a t i o n ; 2. How the program has been i m p l e m e n t e d . A d e t a i l e d d e s c r i p t i o n of program c h a r a c t e r i s t i c s s u c h as i t s s t r u c t u r a l o r g a n i z a t i o n , a c t i v i t i e s , and the v i e w s o f h e a l t h c a r e p r o v i d e r s and b o t h u s e r s and n o n - u s e r s of s e r v i c e s ; 3. How i n d i v i d u a l s f e e l a b o u t the program. As opposed to l i s t i n g s a t i s f a c t i o n a c c o r d i n g to an i n t e r v a l or o r d i n a l s c a l e , d e t a i l e d a n a l y s e s c a n be p r o v i d e d as to consumer s a t i s f a c t i o n w i t h th e s e r v i c e s p r o v i d e d , f o r i n s t a n c e ; and, 4. How i n d i v i d u a l s have been changed by the program. P a r t i c i p a n t o b s e r v a t i o n c a n p r o v i d e a d e s c r i p t i v e a c c o u n t o f how s e r v i c e u s e r ' s h e a l t h s t a t u s was p r i o r to e n t e r i n g the h e a l t h c a r e d e l i v e r y s y s t e m , how p a r t i c u l a r s e r v i c e s a f f e c t e d them, and how t h e i r l i v e s changed f o l l o w i n g t r e a t m e n t . ( 3 1 ) P a r t i c i p a n t o b s e r v a t i o n o f f e r s a means of v i e w i n g the dynamic i n t e r r e l a t i o n s h i p o f p e o p l e and t h e i r e n v i r o n m e n t . I t p r o v i d e s t h e o p p o r t u n i t y to t h e r e s e a r c h e r to v i e w f i r s t - h a n d t h e s t r u c t u r e s and p r o c e s s e s w h i c h a f f e c t , and w h i c h a r e 129 a f f e c t e d by, the program i n t e r v e n t i o n s . None o f t h e f i v e c a s e s t u d i e s used p a r t i c i p a n t o b s e r v a t i o n as a means of c o l l e c t i n g v i 1 l a g e - l e v e 1 d a t a . T h i s t e c h n i q u e would have been u s e f u l to v e r i f y b e h a v i o u r and to i d e n t i f y n o n - u s e r s of p r i m a r y h e a l t h c a r e s e r v i c e s . F o r example, a l t h o u g h the P a r a g u a y and C o l o m b i a n e v a l u a t i o n teams c o l l e c t e d i n f o r m a t i o n on the number of l a t r i n e s c o n s t r u c t e d i n t h e s t u d y a r e a s o v e r the p r o j e c t p e r i o d , t h e r e a r e no d a t a to i n d i c a t e t h a t t h e y were b e i n g u s e d . The s u c c e s s of the PHC programs s h o u l d be measured r e l a t i v e to the use o f s u c h f a c i l i t i e s , n o t s i m p l y w i t h r e s p e c t to t h e i r b e i n g c o n s t r u c t e d . B u t t r a d i t i o n a l d a t a c o l l e c t i o n t e c h n i q u e s , s u c h as t h o s e r e f e r r e d to above, p l a c e t h e r e s e a r c h team as o b s e r v e r s . Those b e i n g e v a l u a t e d p l a y a p a s s i v e r o l e t h r o u g h o u t the e x e r c i s e . The i n f o r m a t i o n needs o f t h e e v a l u a t i o n a r e d e f i n e d e x t e r n a l to the s t u d y a r e a . F e u e r s t e i n (1980) r e f e r s to t h i s as t h e ' s t u d y of s p e c i m e n s ' a p p r o a c h , whereby program p a r t i c i p a n t s a r e e x p e c t e d to p l a y o n l y a m i n i m a l r o l e i n t h e e v a l u a t o r y p r o c e s s . T h i s a p p r o a c h c o n s i s t s e s s e n t i a l l y o f a b r i e f e x p l a n a t i o n of t h e s t u d y ' s o b j e c t i v e s , w h e r e i n the p a r t i c i p a n t s a g r e e to be c o u n t e d , examined and even q u e s t i o n e d . T h e s e r e s u l t s a r e t h e n removed from the s t u d y a r e a f o r a n a l y s i s . T h e r e i s no f e e d b a c k of the f i n d i n g s to t h e p a r t i c i p a n t s - n e i t h e r do t h e y e x p e c t i t . The u t i l i z e r s o f the s e r v i c e s a r e d i v o r c e d from the e v a l u a t i o n r e s u l t s . The p r o d u c t of the e x e r c i s e i s a 130 c o n v e n t i o n a l e v a l u a t o r - f o c u s e d e v a l u a t i o n . ( 3 2 ) T h i s was the a p p r o a c h used i n t h e f i v e c a s e s t u d i e s . H a l l (1979) c o n t e n d s t h a t c o n v e n t i o n a l d a t a c o l l e c t i o n methods t e n d to d i v o r c e e v a l u a t o r s f r o m the r e a l i t i e s o f the s i t u a t i o n t h e y a r e meant to i n v e s t i g a t e . Q u e s t i o n n a i r e s or i n t e r v i e w s d e s i g n e d i n an o f f i c e o f a u n i v e r s i t y a r e by n a t u r e o n e - s i d e d . The s t u d y p o p u l a t i o n i s r e l e g a t e d to b e i n g a s o u r c e of i n f o r m a t i o n . P e o p l e a r e r e g a r d e d as h a v i n g b i t s o f i s o l a t e d k n o w l e d ge. However, t h e y a r e n e v e r e x p e c t e d o r a p p a r e n t l y assumed a b l e to a n a l y z e a g i v e n s o c i a l r e a l i t y . A t t h e extreme, r e s e a r c h e r s t a k e up p e o p l e ' s time w i t h b a d l y f o r m u l a t e d q u e s t i o n s , and make i n t e r p r e t a t i o n s b a s e d on l i t t l e e x p e r i e n c e i n t h e a r e a . H a l l a l s o p r o p o s e s t h a t r i g o r o u s , h i g h l y t e c h n i c a l , and s c i e n t i f i c a l l y ' p u r e ' r e s e a r c h c r e a t e s an i l l u s i on t h a t o n l y the u n i v e r s i t y - t r a i n e d a r e c a p a b l e o f c o l l e c t i n g d a t a . The a b i l i t i e s o f p e o p l e to i n v e s t i g a t e t h e i r own o b j e c t i v e r e a l i t i e s a r e n o t s t i m u l a t e d , and the p o o l of human c r e a t i v i t y i s k e p t w i t h i n n a r r o w c o n f i n e s . As he s t a t e s : T h o s e most f a m i l i a r w i t h t h e pr o b l e m s and t h o s e whose d a i l y e x i s t e n c e i s a f f e c t e d by poor h e a l t h , poor n u t r i t i o n , and low l e v e l s of p r o d u c t i o n a r e e f f e c t i v e l y t a k e n o u t o f t h e a c t i v e p r o c e s s o f making the changes w h i c h m i g h t l e a d to im p r o v e m e n t s . C o n t r o l i s l e f t to t h o s e who, by d e f i n i t i o n and l e v e l s of t r a i n i n g , a r e o u t s i d e t h e e x p e r i e n c e s w i t h i n w h i c h change i s s o u g h t . (33) 131 H a l l a l s o c r i t i c i z e s the use of e x p e r i m e n t a l d e s i g n s f o r f a i l i n g to p r o v i d e e a s y l i n k s to p o s s i b l e s u b s e q u e n t a c t i o n . A l t h o u g h s h o r t c o m i n g s i n s t a t e d program o b j e c t i v e s and i n program a c t i v i t i e s a r e o f t e n c i t e d , s t r a t e g i e s d e s i g n e d to overcome t h e s e l i m i t a t i o n s and improve program i m p a c t a r e r a r e l y i d e n t i f i e d . Y e t the p u r p o s e o f r e s e a r c h i s to promote a c t i o n . I t i s e x p e c t e d t h a t when changes a r e made, the p e o p l e i n the community or the p a r t i c i p a n t s i n the program w i l l p a r t i c i p a t e more a c t i v e l y , more e f f i c i e n t l y , o r w i l l g a i n i n c r e a s e d b e n e f i t s o v e r what had e x i s t e d b e f o r e . However, t h i s o f t e n i s n o t t h e c a s e . I n a d d i t i o n to r e s u l t i n g i n poor s o u r c e s o f i n f o r m a t i o n , s u c h r e s e a r c h methods t e n d to a l i e n a t e the r e s p o n d e n t s , or a t b e s t t r e a t them as s o u r c e s o f p r i m i t i v e i n f o r m a t i o n . C o n s e q u e n t l y , t h e r e i s l i t t l e l i k e l i h o o d of c r e a t i n g t h e a c t i v e and s u p p o r t i v e e n v i r o n m e n t e s s e n t i a l f o r c h a n g e . ( 3 4 ) Such t e c h n i q u e s assume t h a t i n d i v i d u a l s do n o t p l a c e v a l u e on the i n t e r v e n t i o n , t h a t t h e y r e a c t o b j e c t i v e l y to i t s i n t r o d u c t i o n . Y e t , as p r e v i o u s l y d i s c u s s e d , t h e v a l u e o f q u a l i t a t i v e d a t a l i e s i n i t s a b i l i t y to i d e n t i f y and e x p l a i n i n t e r r e l a t i o n s h i p s o v e r l o o k e d or i g n o r e d by q u a n t i t a t i v e d a t a c o l l e c t i o n and a n a l y s i s t e c h n i q u e s . An a l t e r n a t i v e a p p r o a c h to c o l l e c t i n g and a n a l y z i n g s u c h d a t a i s one e l e m e n t of the p a r t i c i p a t o r y r e s e a r c h c o n c e p t . 132 P a r t i c i p a t o r y r e s e a r c h i s d e f i n e d as a method o f s o c i a l i n v e s t i g a t i o n i n v o l v i n g the f u l l p a r t i c i p a t i o n o f t h e s t u d y community.(35) The aim o f t h i s a p p r o a c h i s to c r e a t e an a w a reness on the p a r t o f t h e community o f t h e i r own r e s o u r c e s , and m o b i l i z e them to t a k e r e s p o n s i b i l i t y f o r t h e i r s i t u a t i o n . The e v a l u a t o r becomes a p a r t i c i p a n t and l e a r n e r i n the e v a l u a t o r y p r o c e s s . The community t a k e s a f u l l and a c t i v e r o l e i n the e v a l u a t i o n p r o c e s s , and have p r i o r i t y i n d e c i s i o n making w i t h r e s p e c t to the d e s i g n and i m p l e m e n t a t i o n o f t h e e v a l u a t i v e r e s e a r c h e x e r c i s e and i n t h e d i s s e m i n a t i o n of r e s e a r c h f i n d i n g s . B e c a u s e members of t h e s t u d y p o p u l a t i o n have been i n t i m a t e l y i n v o l v e d i n a l l a s p e c t s of the program and the e v a l u a t i o n p r o c e s s , t h e y may t h e n r e q u i r e o n l y m i n i m a l h e l p i n i n i t i a t i n g and c a r r y i n g o u t f u t u r e e v a l u a t i o n s t u d i e s . I t i s t h i s l a t t e r a p p r o a c h w h i c h i s c r i t i c a l to the s u c c e s s and r e l e v a n c e o f the c a s e s t u d y s t r a t e g y . The s t r e n g t h o f p a r t i c i p a t o r y r e s e a r c h i s t h a t i t r e c o g n i z e s t h a t t h e e v a l u a t o r and s u b j e c t s i n h a b i t d i f f e r e n t r e a l i t i e s and t h a t i t i s presumptuous f o r t h e i n v e s t i g a t o r a l o n e to d e t e r m i n e what i s to be i n v e s t i g a t e d . P a r t i c i p a t o r y e v a l u a t i o n d e s c r i b e s and a n a l y z e s the p r o c e s s by w h i c h h e a l t h c a r e i s a d m i n i s t e r e d and t h e p e r s o n a l meanings a t t r i b u t e d to t h i s p r o c e s s by t h e s e r v i c e u s e r s . I n t h i s r e g a r d , i t c h a l l e n g e s the c o n v e n t i o n a l s y s t e r n s - o r i e n t e d a p p r o a c h e s of i n p u t - o u t p u t a n a l y s i s and q u a n t i f i e d r e s u l t s . M o r e o v e r , the 133 p a r t i c i p a t o r y e v a l u a t i o n a p p r o a c h a t t e m p t s to c o n n e c t t h e i n d i v i d u a l c a s e to t h e l a r g e r economic and s o c i a l c o n t e x t i n w h i c h t h e program e x i s t s . ( 3 6 ) P r o p o n e n t s of the p a r t i c i p a t o r y a p p r o a c h a g r e e t h a t t h e r e i s no d e f i n i t i v e m e t h o d o l o g y f o r c o n d u c t i n g an e v a l u a t i v e r e s e a r c h s t u d y o f t h i s n a t u r e . The b a s i c e l e m e n t of t h i s a p p r o a c h i s o b s e r v a t i o n and d e s c r i p t i o n . Q u a l i t a t i v e r a t h e r t h a n q u a n t i t a t i v e i n f o r m a t i o n i s c o l l e c t e d and a n a l y z e d to a g r e a t e r e x t e n t . What ensues as a r e s u l t o f t h e a c t i v e p a r t i c i p a t i o n o f t h e community i n t h e r e s e a r c h p r o c e s s i s a more a c c u r a t e and a u t h e n t i c a n a l y s i s of the program and i t s f u n c t i o n a l c a p a b i l i t i e s , s t r e n g t h s and weaknesses w i t h i n the c o n t e x t o f t h e p r e v a i l i n g s o c i a l and economic r e a l i t y . R e f e r r i n g back to S m i t h ' s d i s c u s s i o n o f t h e l e v e l o f c e r t a i n t y r e q u i r e d t o e n s u r e t h a t t h e r e s e a r c h r e s u l t s a r e u t i l i z e d , the p a r t i c i p a t o r y r e s e a r c h a p p r o a c h p r o v i d e s more t h a n s u g g e s t i v e o r p r e p o n d e r a n t e v i d e n c e . The f i r s t - h a n d e x p e r i e n c e s of i n d i v i d u a l s , combined i n t o t h e whole o f the community, p r o v i d e a r e a l and h o l i s t i c a n a l y s i s o f the p e r f o r m a n c e and i m p a c t of program a c t i v i t i e s , e q u a l i n v a l i d i t y , s e n s i t i v i t y and s i g n i f i c a n c e to q u a n t i t a t i v e d a t a . The a c t i v e p a r t i c i p a t i o n o f f i e l d w o r k e r s and the community i n r e s e a r c h on p r i m a r y h e a l t h c a r e i s a d v o c a t e d by the World H e a l t h O r g a n i z a t i o n . ( 3 7 ) I n f a c t , most o f t h e f i v e c a s e s t u d i e s u s e d v i 1 l a g e - l e v e 1 h e a l t h w o r k e r s to g a t h e r 134 i n f o r m a t i o n . W h i l e the d a t a c o l l e c t e d i n some i n s t a n c e s was v a l i d and r e l e v a n t , i n o t h e r c a s e s i t was n o t . As was p o i n t e d o u t i n the p r e v i o u s c h a p t e r , the f a c t o r s r e s p o n s i b l e f o r l i m i t i n g t h e v a l i d i t y and r e l e v e n c e o f the d a t a c o l l e c t e d by t h e community h e a l t h w o r k e r s were r e l a t e d p r i m a r i l y to i n a d e q u a t e t r a i n i n g and s u p e r v i s i o n , and an e x p e c t a t i o n t h a t t h e y would be c a p a b l e of c o m p l e t i n g l a r g e numbers o f forms i n a d d i t i o n to a l l the o t h e r t a s k s t h e y were e x p e c t e d to p e r f o r m . However, t h e s e t e n d to be l o g i s t i c a l p r o b l e m s , l a r g e l y u n r e l a t e d to the a b i l i t y o f t h e community h e a l t h w o r k e r , o r o t h e r community members f o r t h a t m a t t e r , to c o l l e c t d a t a f o r the program. The p r o b l e m i s to e n s u r e t h a t the t e c h n i q u e s used to c o l l e c t t h e d a t a a r e w i t h i n t h e a b i l i t i e s o f t h o s e engaged to c a r r y o u t t h i s t a s k . T h i s e n t a i l s d e s i g n i n g s u r v e y s and o t h e r d a t a c o l l e c t i o n t e c h n i q u e s w h i c h a r e u n d e r s t o o d and r e l e v a n t to t h o s e who w i l l be c o m p l e t i n g them. A d d i t i o n a l l y , a r e a l i s t i c and p r a c t i c a l i n t e r v i e w / o b s e r v a t i o n s c h e d u l e s h o u l d be d e s i g n e d , t a k i n g i n t o c o n s i d e r a t i o n t h e d a i l y w o r k l o a d o f t h o s e r e s p o n s i b l e f o r c o l l e c t i n g the d a t a . B u t t h e y a r e n o t c o n f i n e d to c o l l e c t i n g the d a t a . They a l s o c o l l a b o r a t e i n t h e i r c o m p i l a t i o n and a n a l y s i s . One s t u d y f o u n d t h a t a v i t a l p a r t of the e v a l u a t o r y p r o c e s s was t h e d i s c u s s i o n o f the f i n d i n g s o f the d a t a c o l l e c t i o n e x e r c i s e among t h e v i l l a g e h e a l t h w o r k e r s and the v i l l a g e r s r e s p o n s i b l e f o r c o l l e c t i n g t h e d a t a . The p o t e n t i a l f e e d b a c k from s u c h a d i s c u s s i o n has i m p o r t a n t 135 i m p l i c a t i o n s f o r the f u t u r e s u c c e s s and impact of the program. One of the community h e a l t h w o r k e r s i n v o l v e d i n the p r o j e c t o b s e r v e d t h a t We d i d n o t want an e v a l u a t i o n t h a t we would n o t u n d e r s t a n d and t h a t would n o t have h e l p e d us to u n d e r s t a n d our p r o b l e m s - l i k e j u s t a n s w e r i n g q u e s t i o n n a i r e s . Our e v a l u a t i o n i s v e r y i m p o r t a n t b e c a u s e i n a l l the y e a r s we have worked we have n e v e r so c l e a r l y s een the v a l u e of our work. (38) A l l o w i n g the community to become i n v o l v e d i n the d a t a c o l l e c t i o n , a n a l y s i s and d i s c u s s i o n a c t i v i t i e s adds a n o t h e r d i m e n s i o n to the e v a l u a t i o n , n o t o n l y i n l o g i s t i c t erms, but more i m p o r t a n t l y , w i t h r e s p e c t to the c o m p r e h e n s i v e n e s s of the f i n d i n g s o f the e n t i r e e x e r c i s e . Q u a l i t a t i v e i n f o r m a t i o n on a t t i t u d e s , p e r c e p t i o n s , r e l a t i o n s h i p s , f e a r s , m o t i v a t i o n s , c o m m u n i c a t i o n b a r r i e r s , r e a s o n s f o r p r i o r i t i e s i n c a r r y i n g o u t program a c t i v i t i e s a r e c r i t i c a l i n u n d e r s t a n d i n g the f a c t o r s w hich a f f e c t the p e r f o r m a n c e and impact o f the p r o g r a m . ( 3 9 ) The a c t i v e p a r t i c i p a t i o n of community members and f i e l d s t a f f i n the e v a l u a t i o n can p r o v i d e d e t a i l e d i n f o r m a t i o n as to 'why' t h e y r e a c t to the a v a i l a b i l i t y of program s e r v i c e s . T hrough t h i s a p p r o a c h , the e v a l u a t o r ( s ) a c t i v e l y seek to u n d e r s t a n d u s e r and p r o v i d e r c o n c e r n s . I n t h i s way, program g o a l s and a c t i v i t i e s can be m o d i f i e d to a d a p t to c h a n g i n g a t t i t u d e s and c o n t e x t i n which the program e x i s t s , t h e r e b y making the program more r e l e v a n t and s i g n i f i c a n t to the needs o f the i n t e n d e d b e n e f i c i a r y p o p u l a t i o n . 136 S ummary The r e s u l t s o f an e v a l u a t i o n a r e l o c a t i o n - s p e c i f i c , and c a n n o t n e c e s s a r i l y be g e n e r a l i z e d . A t t e m p t i n g to a p p l y any one e v a l u a t i v e r e s e a r c h d e s i g n o r method u n i v e r s a l l y i s i m p r a c t i c a l . Such an a p p r o a c h w i l l f a i l to i d e n t i f y i m p o r t a n t l o c a l f a c t o r s w h i c h a f f e c t the p e r f o r m a n c e and impact of the program. C o n s e q u e n t l y , the v a l i d i t y and r e l e v a n c e of the r e s u l t s of the e v a l u a t i o n w i l l be q u e s t i o n a b l e . What i s r e q u i r e d i s a more f l e x i b l e a p p r o a c h to c o n d u c t i n g e v a l u a t i o n s i n f i e l d s e t t i n g s . As t h i s c h a p t e r has shown, q u a s i - e x p e r i m e n t a 1 d e s i g n s o f f e r a p r a c t i c a l means of o v e r c o m i n g many of the f a c t o r s w h i c h l i m i t the v a l i d i t y of the more e x p e r i m e n t a l e v a l u a t i v e r e s e a r c h d e s i g n s i n a f i e l d s e t t i n g . A d d i t i o n a l l y , the use o f measures o t h e r t h a n m o r b i d i t y and m o r t a l i t y may p r o v i d e the r e s e a r c h e r w i t h a more c r e d i b l e b a s i s upon w h i c h to base r e s e a r c h f i n d i n g s . The use o f r e s e a r c h s u r v e y t e c h n i q u e s , s u c h as p a r t i c i p a n t o b s e r v a t i o n , as a l t e r n a t i v e s a n d / o r a d j u n c t s to the more c o n v e n t i o n a l a p p r o a c h e s s h o u l d a l l o w r e s e a r c h e r s to c o l l e c t a w i d e r range of d a t a . The use o f q u a l i t a t i v e d a t a as an a d j u n c t to q u a n t i t a t i v e d a t a s h o u l d a l s o s e r v e to s t r e n g t h e n the v a l i d i t y and r e l e v e n c e of the e v a l u a t i o n r e s u l t s . L a s t l y , c o n s i d e r a t i o n s h o u l d be 137 g iven to seek a c t i v e community and s e r v i c e p r o v i d e r p a r t i c i p a t i o n i n the e v a l u a t i o n e x e r c i s e . In t h i s way, important i n f o r m a t i o n which might o therwise have been o v e r l o o k e d , p a r t i c u l a r l y wi th r e s p e c t to t h e i r p e r c e p t i o n s of the h e a l t h s e r v i c e s and i t s p o t e n t i a l f o r b e n e f i t i n g the community. What the chapter proposes i s a more h o l i s t i c approach to e v a l u a t i v e r e s e a r c h , one which should serve to i n c r e a s e the c r e d i b i l i t y of the e v a l u a t i o n e x e r c i s e , but which w i l l a l s o be s e n s i t i v e to those f a c t o r s which tend to a d v e r s e l y a f f e c t the c a p a c i t y to conduct f i e l d - b a s e d r e s e a r c h . 138 REFERENCES 1. N.L. S m i t h , "The C e r t a i n t y o f Judgements i n H e a l t h E v a l u a t i o n s , " E v a l u a t i o n and Program P l a n n i n g 4(3 & 4) ( 1 9 8 1 ) : 274 . 2. S.M. S h o r t e l l and W.C. R i c h a r d s o n , H e a l t h Program  E v a l u a t i o n ( S t . L o u i s : The C.V. Mosby Company, 19 7 8 ) , pp. 124, 131 . 3. M.L. D u l a n s e y , e d . , A p p r o a c h e s to A p p r o p r i a t e  E v a l u a t i o n - A R e p o r t on a S e r i e s of Workshops on E v a l u a t i o n (New Y o r k : A m e r i c a n C o u n c i l o f V o l u n t a r y A g e n c i e s f o r F o r e i g n S e r v i c e I n c . , 1 9 7 8 ) ; M.-T. F e u e r s t e i n , " E v a l u a t i o n - By the P e o p l e , " I n t e r n a t i o n a l N u r s i n g Review 25(5) ( 1 9 7 8 ) : 146-153; B.L. H a l l , " P a r t i c i p a t o r y R e s e a r c h : E x p a n d i n g the Base o f A n a l y s i s , " I n t e r n a t i o n a l Development R e v i e w / F o c u s ( 1977): 23-28 ; Idem, " B r e a k i n g the Monopoly on Knowledge: R e s e a r c h Methods, P a r t i c i p a t i o n and D e v e l o p m e n t , " i n A d u l t L e a r n i n g : A D e s i g n f o r  A c t i o n , e d s . B.L. H a l l and J.R. K i d d ( U n i t e d Kingdom: Pergamon P r e s s , 1 9 7 8 ) ; Y. Kassam and K. M u s t a f a , e d s . , P a r t i c i p a t o r y  R e s e a r c h : an E m e r g i n g A l t e r n a t i v e M e t h o d o l o g y i n S o c i a l S c i e n c e  R e s e a r c h (New D e l h i : S o c i e t y f o r P a r t i c i p a t o r y R e s e a r c h i n A s i a , 1 9 8 2 ) ; M.L. Swantz , " R e s e a r c h as an E d u c a t i o n a l T o o l f o r D e v e l o p m e n t , " C o n v e r g e n c e 8 ( 2 ) ( 1975):44-53. 4. E.R. House, E v a l u a t i n g w i t h V a l i d i t y ( B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1980) , pp. 72 , 73 . 5. S m i t h , " C e r t a i n t y o f J u dgements," p. 274. 6 . I b i d . , pp. 273-275. 7. I b i d . , p. 274. 8. I b i d . 9., T. C o n n o l l y and A.L. P o r t e r , "A u s e r - f o c u s e d model f o r t h e u t i l i z a t i o n o f e v a l u a t i o n , " E v a l u a t i o n and Program P l a n n i n g 3 ( 2 ) ( 1 9 8 0 ) : 2 7 7 . 10. B.T. P e r r i n , "Use o f E v a l u a t i o n : D e c i s i o n - M a k i n g , o r ...," i n C a n a d i a n E v a l u a t i o n S o c i e t y , P r o c e e d i n g s of the 2nd  A n n u a l C o n f e r e n c e ( T o r o n t o : n.p., 1 9 8 1 ) , pp. 172-174. 11. H.C. S c h u l b e r g and F. B a k e r , " E v a l u a t i n g H e a l t h Programs: A r t a n d / o r s c i e n c e ? , " i n Program E v a l u a t i o n i n the  H e a l t h F i e l d s , V o l . 2, e d s . H.C. S c h u l b e r g and F. B a k e r (New Y o r k : Human S c i e n c e s P r e s s , 1 9 7 9 ) , p. 12. 139 12. B. H a l l , " P a r t i c i p a t o r y R e s e a r c h : B r e a k i n g the Academic Monopoly," p. 51 (Mimeographed.) 13. R. B o r u c h and H. Gomez, "Power t h e o r y i n s o c i a l program e v a l u a t i o n s , " i n P r o c e e d i n g s o f the E v a l u a t i o n R e s e a r c h S o c i e t y e d s . L. D a t t a and R.A. P e r l o f f ( B e v e r l y H i l l s : Sage, 1 9 7 9 ) . 14. S.A. S t o u f f e r , "Some O b s e r v a t i o n s on Study D e s i g n , " Am.  J . S o c i o l . 55 (1950):355-361; D.T. C a m p b e l l , " F a c t o r s R e l e v a n t to the V a l i d i t y o f E x p e r i m e n t s i n S o c i a l S e t t i n g s , " P s y c h o 1.  B u l l . 54 ( 1957): 297-312 , c i t e d by T.D. Cook and D.T. C a m p b e l l , Q u a s i - E x p e r i m e n t a t i o n : D e s i g n and A n a l y s i s I s s u e s f o r F i e l d  S e t t i n g s ( B o s t o n : Houghton M i f f l i n Company, 19 7 9 ) , p. 6. 15. D.T. C a m p b e l l and J . C . S t a n l e y , E x p e r i m e n t a l and Q u a s i - E x p e r i m e n t a 1 D e s i g n s f o r R e s e a r c h ( C h i c a g o : Rand M c N a l l y C o l l e g e P u b l i s h i n g Company, 1 9 6 3 ) , pp. 34-63. 16. T.D. Cook, " Q u a s i - E x p e r i m e n t a t i o n : I t s O n t o l o g y , E p i s t e m o l o g y , and M e t h o d o l o g y , " i n Beyond Method: S t r a t e g i e s f o r  S o c i a l R e s e a r c h , ed. G. Morgan ( B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1983), p. 84. 17. C a m p b e l l and S t a n l e y , D e s i g n s f o r R e s e a r c h , p. 34. 18. Cook and C a m p b e l l , Q u a s i - E x p e r i m e n t a t i o n , p. 6. 19. Cook, " Q u a s i - E x p e r i m e n t a t i o n , " pp. 79,80. 20. C a m p b e l l and S t a n l e y , D e s i g n s f o r R e s e a r c h , pp. 37-42, 47-50, 55-64. 21. F . L . G o l l a d a y and C.K. Koch-Weser, "The New P o l i c i e s f o r R u r a l H e a l t h : I n s t i t u t i o n a l , s o c i a l and f i n a n c i a l c h a l l e n g e s to l a r g e - s c a l e i m p l e m e n t a t i o n , " P r o c . R. Soc. Lond. v o l . B, no. 199 ( 1 9 7 7 ) : 1 7 0 ; L. H e n d r a t t a , " H e a l t h and D e v e l o p m e n t : Review of P o l i c y O p t i o n s a t t h e G r a s s R o o t s L e v e l , " i n The F u t u r e of  Academic Community M e d i c i n e i n D e v e l o p i n g C o u n t r i e s , ed. W. Lathem (New Y o r k : P r a e g e r P u b l i s h e r s , 1 9 7 9 ) , p. 59; U n i t e d N a t i o n s , C o n f e r e n c e on Human S e t t l e m e n t s , "Community Development A p p r o a c h e s f o r t h e Improvement o f H e a l t h , " from H e a1th  I m p l i c a t i o n s i n the Development of the P h y s i c a l and S o c i a l  E n v i r o n m e n t s of Human S e t t l e m e n t s ( V a n c o u v e r : n.p., 1 9 7 6 ) , p. 1. 22. H e n d r a t t a , " H e a l t h and D e v e l o p m e n t , " p. 59. 23. R.E. S t a k e , E v a l u a t i n g the a r t s i n e d u c a t i o n : A  r e s p o n s i v e a p p r o a c h (Columbus, O h i o : C h a r l e s E. M e r r i l l , 1 9 7 5 ) , p. 14. 140 24. M.Q. P a t t o n , Q u a l i t a t i v e E v a l u a t i o n Methods ( B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1 9 8 0 ) , p. 55. 25. House, E v a l u a t i n g , pp. 244-246. 26. M. P a r l e t t and D. H a m i l t o n , E v a l u a t i o n as I l l u m i n a t i o n :  A New A p p r o a c h to the Study of I n n o v a t i v e Programs ( E d i n b u r g h : U n i v e r s i t y o f E d i n b u r g h , C e n t r e f o r R e s e a r c h i n the E d u c a t i o n a l S c i e n c e s , 1 9 7 2 ) , p. 30. 27. G. F o s t e r , A p p l i e d A n t h r o p o l o g y ( B o s t o n : L i t t l e , Brown, 1969) , pp . 57 , 58 . 28. P a t t o n , Q u a l i t a t i v e , pp. 97,98. 29. Idem, "Making methods c h o i c e s , " E v a l u a t i o n and Program  P l a n n i n g 3 ( 1 9 8 1 ) : 2 2 7 . 30. D.E. K u r z , "The u s e o f p a r t i c i p a n t o b s e r v a t i o n i n e v a l u a t i o n r e s e a r c h , " E v a l u a t i o n and Program P l a n n i n g 6 (1983) : 93 . 31. T h i s d e s c r i p t i o n i s b a s e d on a s i m i l a r p r e s e n t a t i o n p r o v i d e d i n K u r z , " P a r t i c i p a n t o b s e r v a t i o n , " pp. 94,95. 32. M.-T. F e u e r s t e i n , " P a r t i c i p a t o r y e v a l u a t i o n - an a p p r o p r i a t e t e c h n o l o g y f o r community h e a l t h programmes," C o n t a c t 55 ( F e b r u a r y 1 9 8 0 ):3. 33. H a l l , " P a r t i c i p a t o r y R e s e a r c h , " p. 51. 34 . I b i d . , p. 52. 35. I b i d . , p. 54. 36. I b i d . , pp. 51,52; B. H a l l , A. E t h e r i n g t o n , and T. J a c k s o n , " E v a l u a t i o n , P a r t i c i p a t i o n and Community H e a l t h : C r i t i q u e and L e s s o n s , " T o r o n t o : I n t e r n a t i o n a l C o u n c i l f o r A d u l t E d u c a t i o n , 1979. (Mimeographed.) 37. S. C o l e - K i n g , A p p r o a c h e s to the E v a l u a t i o n of M a t e r n a l  and C h i l d H e a l t h C are i n the C o n t e x t of P r i m a r y H e a l t h C a r e  Programmes (Geneva: W o r l d H e a l t h O r g a n i z a t i o n , 19 7 9) , p . 5 0 . 38. M.-T. F e u e r s t e i n , "The e d u c a t i v e a p p r o a c h i n e v a l u a t i o n : an a p p r o p r i a t e t e c h n o l o g y f o r a r u r a l h e a l t h programme," J .  H e a l t h Ed., 2 1 ( 1 ) 1978:64. 39. Idem, " P a r t i c i p a t o r y e v a l u a t i o n , " p. 5. 141 CHAPTER"FOUR CONCLUSION AND RECOMMENDATIONS The p r i m a r y h e a l t h c a r e a p p r o a c h i s p r o f f e r e d as a more e f f e c t i v e and l o w e r - c o s t a l t e r n a t i v e to c o n v e n t i o n a l h e a l t h c a r e d e l i v e r y a p p r o a c h e s f o r d e v e l o p i n g c o u n t r i e s . Over the p a s t two d e c a d e s , m i l l i o n s o f d o l l a r s have been i n v e s t e d i n the de v e l o p m e n t and i m p l e m e n t a t i o n o f p r i m a r y h e a l t h c a r e p r o j e c t s t h r o u g h o u t the T h i r d W o r l d . The common a s s u m p t i o n of t h e s e p r o j e c t s i s t h a t the a v a i l a b i l i t y and u t i l i z a t i o n of the p r i m a r y h e a l t h c a r e s e r v i c e s and p e r s o n n e l w i l l n e c e s s a r i l y r e s u l t i n a s i g n i f i c a n t improvement i n t h e h e a l t h s t a t u s of the r e c i p i e n t p o p u l a t i o n . However, to d a t e t h e r e i s l i t t l e e v i d e n c e to s u p p o r t t h i s h y p o t h e s i s . F o r the most p a r t , few p r o j e c t s have been e v a l u a t e d . Of t h o s e w h i c h have been, the v a l i d i t y of the r e s u l t s of the e v a l u a t i o n i s d e b a t a b l e . Y e t d e s p i t e t h i s l a c k of s u b s t a n t i v e e v i d e n c e o f a s i g n i f i c a n t i m p a c t on h e a l t h , donor and h o s t n a t i o n a l governments and f o r e i g n a i d o r g a n i z a t i o n s c o n t i n u e to i n v e s t h e a v i l y i n t h i s s t r a t e g y . T h i s t h e s i s has i d e n t i f i e d s e v e r a l f a c t o r s which tend to l i m i t the c r e d i b i l i t y , v a l i d i t y and u t i l i t y of e v a l u a t i o n s o f p r i m a r y h e a l t h c a r e p r o j e c t s i n the T h i r d W o r l d . These f a c t o r s can be gro u p e d i n t o two c a t e g o r i e s : e v a l u a t i v e r e s e a r c h d e s i g n s ; and e v a l u a t o r y methods and t e c h n i q u e s . 142 In some c a s e s , the e v a l u a t i v e r e s e a r c h d e s i g n s a r e s i m p l y too complex and r i g o r o u s f o r f i e l d s i t u a t i o n s and f o r the c a p a b i l i t i e s of t h o s e r e s p o n s i b l e f o r c a r r y i n g o u t t h e e v a l u a t i o n s . Many of the c r i t i c i s m s l e v e l l e d a t the t r u e e x p e r i m e n t a l r e s e a r c h d e s i g n s r e l a t e to the p r a c t i c a l p r o b l e m s w h i c h a r i s e when a p p l y i n g them to a f i e l d s e t t i n g . Many of the f u n d a m e n t a l p r e r e q u i s i t e s c an r a r e l y be met: the s u b j e c t s c a n n o t be randomly a s s i g n e d to s t u d y and c o n t r o l g r o u p s ; e q u i v a l e n c e o f the e x p e r i m e n t a l and c o n t r o l g r o u p s c a n n o t be g u a r a n t e e d ; i n t e r v e n t i o n and outcome commonly have o n l y tenuous l i n k s ; many programs s i m u l t a n e o u s l y employ m u l i t p l e i n t e r v e n t i o n s ; and l a s t l y , key s o u r c e s of v a r i a n c e c a n n o t be c o n t r o l l e d . A d d i t i o n a l l y , the expense and manpower q u a l i f i c a t i o n r e q u i r e d to implement we 1 1 - m o n i t o r e d and t i g h t l y s u p e r v i s e d e x p e r i m e n t - b a s e d e v a l u a t i o n s a r e o f t e n beyond the c a p a b i l i t i e s of the v a s t m a j o r i t y of t h o s e i n v o l v e d i n i m p l e m e n t i n g p r i m a r y h e a l t h c a r e p r o j e c t s . Such d e s i g n s can a l s o be p o l i t i c a l l y u n a c c e p t a b l e , e s p e c i a l l y when c o n t r o l a r e a s where s e r v i c e s a r e n o t to be i n t r o d u c e d a r e c o n t i g u o u s or n e a r b y the e x p e r i m e n t a l c o m m u n i t i e s . In o t h e r c a s e s , the d e s i g n s a r e too weak. The ab s e n c e o f h i s t o r i c a l d a t a and c o m p a r a t i v e p o p u l a t i o n s l i m i t s the a b i l i t y of e v a l u a t i o n r e p o r t s to c l a i m t h a t t h e i n t e r v e n t i o n has had any s i g n i f i c a n t p o s i t i v e e f f e c t on h e a l t h i n the s t u d y p o p u l a t i o n . I n o r d e r to enhance the c r e d i b i l i t y of the 143 e v a l u a t i v e r e s e a r c h d e s i g n , a d d i t i o n a l o b s e r v a t i o n p o i n t s and c o m p a r a t i v e p o p u l a t i o n s s h o u l d be c o n s i d e r e d . F o r the most p a r t , t hough, the l i m i t i n g f a c t o r s were r e l a t e d to p r a c t i c a l m e t h o d o l o g i c a l i s s u e s , such as d a t a c o l l e c t i o n i n s t r u m e n t s and t e c h n i q u e s . I n many c a s e s , q u a n t i t a t i v e d a t a were of an i n f e r i o r q u a l i t y due to a l a c k of t r a i n i n g o f p e r s o n n e l . I n o t h e r c a s e s , the t y p e of d a t a c o l l e c t e d b e f o r e and a f t e r the i n t e r v e n t i o n v a r i e d , t h e r e b y making c o m p a r a t i v e a n a l y s e s i m p o s s i b l e . The amount of d a t a c o l l e c t e d was a l s o f o u n d to be a c o n t r i b u t i n g f a c t o r . I n some c a s e s , t h o s e r e s p o n s i b l e f o r d e l i v e r i n g h e a l t h c a r e s e r v i c e s were a l s o e x p e c t e d to c o m p l e t e l o n g and complex q u e s t i o n n a i r e s and r e p o r t s . R e l a t e d to t h i s was the f o r m a t and e x i g e n c i e s o f the q u e s t i o n n a i r e or d a t a c o l l e c t i o n i n s t r u m e n t . The more numerous, complex, or d i f f i c u l t to a d m i n i s t e r and i n t e r p r e t , the h i g h e r the p r o b a b i l i t y of e r r o r s i n d a t a r e p o r t i n g and c omp i l a t i o n . An i m p o r t a n t e l e m e n t w h i c h c o n t r i b u t e d to l i m i t i n g the v a l i d i t y and u t i l i t y of the e v a l u a t i o n s t u d i e s i s the type of i n d i c e s used to measure program p e r f o r m n a c e and i m p a c t . In many c a s e s , the i n d i c e s c h o s e n were u n s u i t a b l e f o r the p u r p o s e o f m e a s u r i n g program i m p a c t . Much of t h e d a t a r e l a t e d to the q u a n t i t y and n a t u r e of s e r v i c e s p r o v i d e d , the number o f h e a l t h p e r s o n n e l t r a i n e d and d e p l o y e d , and i n some c a s e s , consumer p e r c e p t i o n s about the s e r v i c e s and and t h o s e d e l i v e r i n g the 144 h e a l t h c a r e s e r v i c e s . W h i l e such i n d i c e s p r o v i d e u s e f u l i n f o r m a t i o n c o n c e r n i n g program o u t p u t , they p r o v i d e no i n d i c a t i o n of how the s e r v i c e s or p e r s o n n e l a f f e c t e d the h e a l t h s t a t u s o f the r e c i p i e n t community. A n o t h e r c o n c e r n i s the r e l i a n c e on m o r b i d i t y and m o r t a l i t y d a t a as outcome measures. O f t e n , the d a t a c o l l e c t e d p e r t a i n s to a subgroup of the s t u d y p o p u l a t i o n . Y e t g e n e r a l i z a t i o n s c o n c e r n i n g the s u c c e s s of the p r o j e c t ' s PHC a p p r o a c h a r e made f o r t h e e n t i r e p o p u l a t i o n , b a s e d on t h i s n a r r o w l y d e f i n e d measure. A d d i t i o n a l l y , h e a l t h i s n o t a f f e c t e d s o l e l y t h r o u g h h e a 1 t h - r e l a t e d a c t i v i t i e s . By r e s t r i c t i n g the scope o f measures u s e d , and the range o f a c t i v i t i e s w h ich may a f f e c t h e a l t h , the e v a l u a t i o n becomes l i m i t e d i n i t s a p p r o x i m a t i o n o f r e a l i n t e r r e l a t i o n s h i p s . A n o t h e r f a c t o r f o u n d to i n f l u e n c e the c r e d i b i l i t y and u t i l i t y of the e v a l u a t i o n s i s the dependence on q u a n t i t a t i v e d a t a , a l m o s t to the e x c l u s i o n of q u a l i t a t i v e d a t a . P a r t of t h i s r e l u c t a n c e to c o l l e c t and a n a l y z e q u a l i t a t i v e d a t a may be due to the e x t r a expense and manpower r e q u i r e m e n t s i n v o l v e d , as w e l l as a p e r c e p t i o n t h a t such d a t a a r e g e n e r a l l y of an i n f e r i o r q u a l i t y to q u a n t i t a t i v e d a t a . W h i l e q u a l i t a t i v e d a t a may p r o v i d e some c o r r a b o r a t i v e s u p p o r t to the f i n d i n g s of q u a n t i t a t i v e d a t a , i t i s w i d e l y h e l d t h a t the use o f q u a l i t a t i v e d a t a a l o n e i s u n s c i e n t i f i c , and t h e r e f o r e i n s u f f i c i e n t . However, q u a l i t a t i v e d a t a c an p r o v i d e u n i q u e 145 i n s i g h t s i n t o the r e a s o n s why p e o p l e choose to ( o r choose n o t to) use p r i m a r y h e a l t h c a r e s e r v i c e s . They can a l s o p r o v i d e r e s e a r c h e r s w i t h i m p o r t a n t i n f o r m a t i o n on the n a t u r e and r e a s o n s f o r u n a n t i c i p a t e d s i d e e f f e c t s of the program, and can a s s i s t i n i d e n t i f y i n g f a c t o r s w hich impede the e f f e c t i v e n e s s and e f f i c e n c y of the program. P e r h a p s one of t h e more i m p o r t a n t l i m i t i n g f a c t o r s i s the p r e o c c u p a t i o n w i t h d e m o n s t r a t i n g c o n c l u s i v e e v i d e n c e of a l i n k between the a v a i l a b i 1 i t y / u t i 1 i z a t i o n of p r i m a r y h e a l t h c a r e s e r v i c e s and improvements i n the h e a l t h s t a t u s of the r e c i p i e n t community. I n t h a t e v a l u a t i o n s a r e used p r i m a r i l y f o r d e c i s i o n - m a k i n g p u r p o s e s , the p r e s e n t s t u d y s u g g e s t s t h a t l e s s t h a n c o n c l u s i v e e v i d e n c e may be a p p r o p r i a t e i n such c a s e s . T h i s i s e s p e c i a l l y t r u e when one c o n s i d e r s t h a t to d e m o n s t r a t e an improvement i n h e a l t h r e q u i r e s a l o n g time p e r i o d and b e f o r e / a f t e r d i f f e r e n t i a l s of s i g n i f i c a n t m a g n i t u d e . However, program managers r e q u i r e i n f o r m a t i v e f e e d b a c k q u i c k l y . What may be r e q u i r e d , and may be a more a p p r o p r i a t e a p p r o a c h g i v e n t h e s e r e q u i r e m e n t s , i s a t r a d e - o f f between the need f o r a h i g h d e g r e e of p r o o f of program e f f e c t i v e n e s s and the need f o r a l o n g e v a l u a t i v e r e s e a r c h time p e r i o d . D e m o n s t r a t i n g a s t r o n g a s s o c i a t i o n between the dependent and i n d e p e n d e n t v a r i a b l e s may be s u f f i c i e n t i f an e q u a l l y i m p o r t a n t c o n s i d e r a t i o n i s to d e c r e a s e the time i n t e r v a l between the commencement of p r o j e c t a c t i v i t i e s and the p o i n t i n time when the e v a l u a t i o n of i t s 146 imp a c t and p e r f o r m a n c e t a k e s p l a c e . Based on t h i s a n a l y s i s , a l t e r n a t i v e a p p r o a c h e s f o r e v a l u a t i n g p r i m a r y h e a l t h c a r e programs which a d d r e s s e s many of the i d e n t i f i e d l i m i t i n g f a c t o r a r e p r o p o s e d . The t h e s i s s u g g e s t s the use of q u a s i - e x p e r i m e n t a 1 e v a l u a t i v e r e s e a r c h d e s i g n s may s e r v e to s t r e n g t h e n t h e c a p a c i t y of the e v a l u a t i o n to d e m o n s t r a t e a s t r o n g r e l a t i o n s h i p between the a v a i l a b i l i t y and u t i l i z a t i o n o f p r i m a r y h e a l t h c a r e s e r v i c e s and a change i n h e a l t h o f the s t u d y p o p u l a t i o n . W h i l e t h e s e d e s i g n s do r e q u i r e c a r e f u l m o n i t o r i n g , t h e y a r e more a d a p t a b l e to f i e l d s e t t i n g s . They a r e n o t as complex as the t r u e e x p e r i m e n t a l d e s i g n s . Nor do t h e i r u n d e r l y i n g a s s u m p t i o n s r e s t r i c t t h e i r f l e x i b i l i t y . A r e v i e w of the e v a l u a t i v e r e s e a r c h d e s i g n s used i n the c a s e s t u d i e s examined i l l u s t r a t e d t h a t s l i g h t m o d i f i c a t i o n s to t h e s e d e s i g n s would have r e s u l t e d i n a c c e p t a b l e q u a s i - e x p e r i m e n t a 1 d e s i g n s . As a means of i n c r e a s i n g the c r e d i b i l i t y and u t i l i t y of the r e s e a r c h r e s u l t s , i t i s p r o p o s e d t h a t q u a l i t a t i v e d a t a be used as an a d j u n c t to q u a n t i t a t i v e d a t a . To t h i s end, a c a s e s t u d y a p p r o a c h i s recommended. By e x a m i n i n g p a r t i c u l a r p o p u l a t i o n g r o u p s o r PHC a c t i v i t i e s , r e s e a r c h e r s may be a b l e to i d e n i t f y f a c t o r s w h i c h e i t h e r i n c r e a s e or d e c r e a s e the c a p a c i t y of the PHC a p p r o a c h to a f f e c t h e a l t h . Q u a l i t a t i v e i n f o r m a t i o n about v a r i o u s a s p e c t s of the program from s e r v i c e p r o v i d e r s , consumers and non-consumers can i d e n t i f y b o t h the s t r o n g and 147 weak p o i n t s o f the program. Such i n f o r m a t i o n i s n o t r e s t r i c t e d to q u e s t i o n s c o n c e r n i n g the a v a i l a b i l i t y and u t i l i z a t i o n of s e r v i c e s and h e a l t h c a r e w o r k e r s . R a t h e r , s u c h d a t a can p r o v i d e i m p o r t a n t i n s i g h t i n t o the p e r c e p t i o n s , a t t i t u d e s and b e h a v i o u r of t h e s e g r o u p s . The r e s e a r c h e r s , t h e r e f o r e , n o t o n l y come to know the 'when', 'where' and 'how much' of the program, but a l s o the 'why'. C o n v e n t i o n a l d a t a c o l l e c t i o n t e c h n i q u e s a l s o s e r v e to r e s t r i c t the v a l u e of the d a t a . S t r u c t u r e d d i r e c t q u e s t i o n n a i r e s w h i c h a r e u s u a l l y c o n s t r u c t e d e x t e r n a l l y to the s t u d y a r e a and p o p u l a t i o n may o v e r l o o k i m p o r t a n t v a r i a b l e s w h ich a f f e c t program p e r f o r m a n c e and p o t e n t i a l i m p a c t . A l t e r n a t i v e d a t a c o l l e c t i o n methods, such as p a r t i c i p a n t o b s e r v a t i o n , a r e s u g g e s t e d as a means of i n c r e a s i n g the i m p o r t a n c e of the d a t a to be c o l l e c t e d . A n o t h e r a p p r o a c h w h i c h c o u l d s t r e n g t h e n the p o t e n t i a l f o r program s u c c e s s i s the a c t i v e i n v o l v e m e n t of s e r v i c e p r o v i d e r s and community members i n the e v a l u a t i o n p r o c e s s . A l l too o f t e n , p r o v i d e r s and b e n e f i c i a r i e s a r e p e r c e i v e d as s o u r c e s of i n f o r m a t i o n , r a t h e r t h a n e l e m e n t s c r i t i c a l to the program's s u c c e s s . R e s e a r c h e r s , e i t h e r k n o w i n g l y or u n k n o w i n g l y , tend to r e l e g a t e t h e i r c o n c e r n s to a n e g l i g i b l e p o s i t i o n . They a r e n o t vie w e d as c o l l a b o r a t o r s i n the p r o j e c t , b u t as s u b j e c t s . C o n s e q u e n t l y , they o f t e n do n o t b e l i e v e t h a t they have any p e r s o n a l s t a k e i n the s u c c e s s of the program. F o r the 148 community, t h i s can o n l y s e r v e to a f f e c t t h e i r a t t i t u d e s and p e r c e p t i o n s o f the program, and u l t i m a t e l y t h e i r b e h a v i o u r and u t i l i z a t i o n of the s e r v i c e s . H e a l t h w o r k e r s view t h e i r r o l e as a j o b , b e l i e v i n g t h a t t h e y have no i n p u t to c o n t r i b u t e c o n c e r n i n g the f u t u r e d e v e l o p m e n t of the program. T h i s i s u n f o r t u n a t e , s i n c e they c an u s u a l l y i d e n t i f y f a c t o r s r e s p o n s i b l e f o r l i m i t i n g the e f f e c t i v e n e s s and e f f i c i e n c y of a program. By i n t e g r a t i n g t h e s e g r o u p s i n t o the e v a l u a t i o n , t h e y w i l l be p r o v i d e d w i t h f e e d b a c k . T h i s s h o u l d i n c r e a s e t h e i r u n d e r s t a n d i n g o f the n a t u r e and dynamics of t h e i r p r o b l e m s , and i n some c a s e s , the means to r e s o l v e them, o r a t l e a s t to m i n i m i z e t h e i r e f f e c t s . Based on the f i n d i n g s of t h i s s t u d y , the f o l l o w i n g recommendations a r e p r e s e n t e d . They a r e d e s i g n e d to e n s u r e t h a t e v a l u a t i v e r e s e a r c h f o r p r i m a r y h e a l t h c a r e p r o j e c t s , and f o r s o c i a l d e v e l o p m e n t programs i n g e n e r a l , i n d e v e l o p i n g c o u n t r i e s i s c r e d i b l e and u s e f u l . F i r s t , e v a l u a t i v e r e s e a r c h i n the T h i r d W o rld must be a c t i o n - o r i e n t e d . I n o t h e r words, i t must be a p r o c e s s f o r c l a r i f y i n g and r e s p o n d i n g to a c t u a l p r o b l e m s o f the r u r a l and p e r i - u r b a n p o o r . T h i s c an be a c h i e v e d by e n s u r i n g t h a t i t i s a bottom-up a p p r o a c h , a c t i v e l y i n c o r p o r a t i n g t h e e n t i r e program community i n t h e e v a l u a t i o n p r o c e s s . I n t h i s way, t h e p o t e n t i a l of s u c c e s s f o r t h e program w i l l be i n c r e a s e d , s i n c e community members w i l l be i n v o l v e d i n the de v e l o p m e n t o f f u t u r e program d e c i s i o n s . 149 Second, the d e s i g n and m e t h o d o l o g i e s used must be a p p r o p r i a t e to the s i t u a t i o n a t hand. T h i s p e r t a i n s n o t o n l y to p r e v a i l i n g e n v i r o n m e n t a l f a c t o r s , but a l s o the needs o f the a u d i e n c e s f o r the e v a l u a t i v e r e s u l t s . D evelopment p r o j e c t s , i n c l u d i n g p r i m a r y h e a l t h c a r e programs, do n o t e x i s t i n a vacuum. They a f f e c t , and a r e a f f e c t e d by, a wide range of e c o l o g i c a l and s o c i o - c u l t u r a l f a c t o r s . R e s e a r c h d e s i g n s s h o u l d be f l e x i b l e to a l l o w r e s e a r c h e r s t h e o p p o r t u n i t y to use a v a r i e t y o f m e t h o d o l o g i e s which may a s s i s t them to i d e n t i f y the n a t u r e o f t h e s e f a c t o r s and to measure t h e i r i m p a c t on h e a l t h and w e l l - b e i n g . As was s t a t e d p r e v i o u s l y , no one m e t h o d o l o g y i s a p p r o p r i a t e f o r e v e r y s i t u a t i o n ; b u t n e i t h e r does e v e r y s i t u a t i o n r e q u i r e the use o f e v e r y m e t h o d o l o g y a v a i l a b l e . T h i r d , i t i s e v i d e n t t h a t t h e r e i s a l a c k o f d e v e l o p i n g c o u n t r y p e o p l e t r a i n e d i n e v a l u a t i v e r e s e a r c h methods. Few have the s k i l l s to d e s i g n or manage an e v a l u a t i o n , or to t r a i n o t h e r s i n i n t e r v i e w t e c h n i q u e s . What i s r e q u i r e d a r e s h o r t - t e r m , but i n t e n s i v e , t r a i n i n g programs i n a v a r i e t y of e v a l u a t i v e r e s e a r c h a p p r o a c h e s . By b u i l d i n g t h e c a p a c i t y of T h i r d W o r l d r e s e a r c h e r s to c o n d u c t e v a l u a t i o n s o f s o c i a l s e r v i c e p r o g r a m s , the e f f e c t of c u l t u r a l i n s e n s i t i v i t y w hich o f t e n c h a r a c t e r i z e s e v a l u a t i o n s c o n d u c t e d by e x p a t r i a t e r e s e a r c h e r s s h o u l d be s i g n i f i c a n t l y r e d u c e d . F i n a l l y , more f u n d s s h o u l d be made a v a i l a b l e by donor a g e n c i e s to f i e l d - t e s t a l t e r n a t i v e e v a l u a t i v e r e s e a r c h 150 m e t h o d o l o g i e s , s u c h as t h a t p r o p o s e d i n t h i s s t u d y . Donor a g e n c i e s must become more aware o f the need to e v a l u a t e h e a l t h and s o c i a l programs i n terms o f t h e i r r e a l e f f e c t on the i n t e n d e d r e c i p i e n t p o p u l a t i o n . S i m p l y p a r a c h u t i n g c o n v e n t i o n a l e v a l u a t i v e r e s e a r c h a p p r o a c h e s i n t o d e v e l o p i n g c o u n t r i e s does n o t g u a r a n t e e u s e f u l or v a l i d e v a l u a t i o n r e s u l t s . E v a l u a t i o n s s h o u l d be more t h a n s i m p l e a c c o u n t i n g e x e r c i s e s . The g o a l of d e v e l o p m e n t i s to improve the q u a l i t y of l i f e of p e o p l e . T h e r e f o r e , e v a l u a t i o n s must be p e o p l e - f o c u s e d . M e a s u r i n g changes i n i m p e r s o n a l i n d i c e s w i t h o u t a c o n s i d e r a t i o n and u n d e r s t a n d i n g o f t h e r e l a t i o n s h i p between the program and the p e o p l e i t i s i n t e n d e d to b e n e f i t s e v e r e l y l i m i t s the s i g n i f i c a n c e and v a l i d i t y o f an e v a l u a t i o n . I n the end, the e v a l u a t i o n becomes an a c ademic e x e r c i s e i n s t e a d o f a p r a c t i c a l , p r o b l e m - s o l v i n g e x e r c i s e . A d d i t i o n a l r e s e a r c h i s r e q u i r e d to i d e n t i f y o t h e r a l t e r n a t i v e a p p r o a c h e s w h i c h may be a p p r o p r i a t e f o r c o n d u c t i n g f i e l d - b a s e d e v a l u a t i v e r e s e a r c h i n d e v e l o p i n g c o u n t r i e s to p r o d u c e v a l i d and m e a n i n g f u l i n f o r m a t i o n . 1 5 1 S E L E C T E D B I B L I O G R A P H Y A b e l - S m i t h , B., a n d L e i s e r s o n , A. P o v e r t y , D e v e l o p m e n t a n d H e a l t h P o l i c y . WHO P u b l i c H e a l t h P a p e r s , n o . 6 9 . G e n e v a : W o r l d H e a l t h O r g a n i z a t i o n , 1 9 7 8 . A b t , C.C., e d . T h e E v a l u a t i o n o f S o c i a l P r o g r a m s . B e v e r l y H i l l s : S a g e P u b l i c a t i o n s , 1 9 7 7 . A b e r t , J . G . , a n d K a m r a s s , M., e d s . S o c i a l E x p e r i m e n t s a n d S o c i a l P r o g r a m E v a l u a t i o n , B o s t o n : B a l l i n g e r P u b l i s h i n g C o . , 1 9 7 4 . A k e r e l e , 0.; T a b i b z a d e h , I . ; a n d M c G i l v r a y , J . "A new r o l e f o r m e d i c a l m i s s i o n a r i e s i n A f r i c a . " WHO C h r o n i c l e s 30 ( 1 9 7 6 ) : 1 7 5 - 1 8 0 . A k h t a r , S. H e a l t h C a r e i n t h e P e o p l e s R e p u b l i c o f C h i n a O t t a w a : I n t e r n a t i o n a l D e v e l o p m e n t R e s e a r c h C e n t r e , 1 9 7 5 . A l k i n , M.C. " U t i l i z a t i o n o f e v a l u a t i o n f i n d i n g s . " I n n o t e c h J o u r n a l 7 ( 1 ) ( J a n u a r y - J u n e 19 8 3 ) : 35-.4 2 . A l k i n , M.C.; D a i l l a k , R.; a n d W h i t e , P. U s i n g E v a l u a t i o n s . B e v e r l y H i l l s : S a g e P u b l i c a t i o n s , 1 9 7 9 . A l w i n , D.F., a n d S u l l i v a n , M . J . " I s s u e s o f d e s i g n a n d a n a l y s i s i n e v a l u a t i o n r e s e a r c h . " S o c i o l o g i c a l M e t h o d s a n d R e s e a r c h 4 ( 1 9 7 5 ) : 7 7 - 1 0 0 . A m e r i c a n P u b l i c H e a l t h A s s o c i a t i o n . T h e S t a t e o f t h e A r t o f D e l i v e r i n g Low C o s t H e a l t h S e r v i c e s i n D e v e l o p i n g C o u n t r i e s : A Summary o f 180 H e a l t h P r o j e c t s . W a s h i n g t o n , D.C.: A m e r i c a n P u b l i c H e a l t h A s s o c i a t i o n , 1 9 7 7 . A m i d i , S. " E f f e c t i v e n e s s o f v i l l a g e h e a l t h w o r k e r s f o r p r i m a r y h e a l t h c a r e i n s o u t h e r n I r a n . " C o u r r i e r o f t h e  I n t e r n a t i o n a l C h i l d r e n ' s C e n t r e ( P a r i s ) 2 7 ( 2 ) ( 1 9 7 7 ) : 1 0 9 - 1 1 2 . A n d r e a n o , R . L . ; C o l e - K i n g , S.; a n d K a t z , F. A s s i g n m e n t R e p o r t :  E v a l u a t i o n o f P r i m a r y H e a l t h C a r e P r o j e c t s i n I r a n . W o r l d H e a l t h O r g a n i z a t i o n , 1 9 7 6 . A 1 e x a n d r i a , E g y p t : 152 A t t k i n s o n , C.C., and B r o s k o w s k i , A. " E v a l u a t i o n and the em e r g i n g human s e r v i c e c o n c e p t . " I n E v a l u a t i o n of Human  S e r v i c e s Programs, pp. 3-26. E d i t e d by C.C. A t t k i n s o n and W.A. H a r g r e a v e s . New Y o r k : Academic P r e s s , 1978. A t t k i s s o n , C.C.; H a r g r e a v e s , W.A.; H o r o w i t z , M.J.; and S o r e n s o n , J . E . " E v a l u a t i o n : c u r r e n t s t r e n g t h s and f u t u r e d i r e c t i o n s . " I n E v a l u a t i o n of Human S e r v i c e s P r o g r a m s , pp. 465-477. E d i t e d by C.C. A t t k i n s s o n and W.A. H a r g r e a v e s . New Y o r k : A c a d e m i c P r e s s , 1978. A u j o u l a t , L.-P. "The use o f community methods." I n t e r n a t i o n a l  J o u r n a l of H e a l t h E d u c a t i o n 7 ( 4 ) ( 1 9 6 4 ) . B a n e r j i , D. " H e a l t h B e h a v i o u r o f R u r a l P o p u l a t i o n s : Impact of R u r a l H e a l t h S e r v i c e s . " Economic and P o l i t i c a l Weekly  (Bombay) 8(51) ( 1973):2261-2268 . B e i g e l , A., and L e v e n s o n , A . I . "Program E v a l u a t i o n on a s h o e s t r i n g b u d g e t . " I n E v a l u a t i o n of Human S e r v i c e s  P r o grams, pp. 97-124. E d i t e d by C.C. A t t k i n s s o n and W.A. H a r g r e a v e s . New Y o r k : Academic P r e s s , 1978. B e l o n c l e , G., and F o u r n i e r , G. S a n t e e t Developpement en m i l i e u  r u r a l a f r i c a i n [ H e a l t h and d e v e l o p m e n t i n an A f r i c a n r u r a l e n v i r o n m e n t ] . P a r i s : L e s E d i t i o n s O u v r i e r e s , 1975. B e n n e t t , C.A. and Lu m s d a i n e , A.A. E v a l u a t i o n and E x p e r i m e n t . New Y o r k : Academic P r e s s , 1975. B e n n e t t , F . J . " P r i m a r y h e a l t h c a r e and d e v e l o p i n g c o u n t r i e s . " S o c i a l S c i e n c e & M e d i c i n e 13A (1979):505-514. B e n y o u s s e f , A., and C h r i s t i a n , B. " H e a l t h c a r e i n d e v e l o p i n g c o u n t r i e s . " S o c i a l S c i e n c e & M e d i c i n e 11 ( 1 9 7 7 ) : 309-408. Berman, P. " V i l l a g e H e a l t h W o r k e r s : B a c k g r o u n d and I s s u e s f o r A n a l y s i s . " P a p e r f o r t h e R u r a l Development Committee, C e n t r e f o r I n t e r n a t i o n a l S t u d i e s , C o r n e l l U n i v e r s i t y , I t h a c a , N.Y., September 1978. B e r n s t e i n , I.N., ed. V a l i d i t y I s s u e s i n E v a l u a t i v e R e s e a r c h . B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1976. B i c e , T.W. "Comments on H e a l t h I n d i c a t o r s : M e t h o d o l o g i c a l P e r s p e c t i v e s . " I n t e r n a t i o n a l J o u r n a l of H e a l t h S e r v i c e s 6 ( 3 ) ( 1 9 7 6 ) : 509-519 . 153 Bogdan, R., and T a y l o r , S. I n t r o d u c t i o n to Q u a l i t a t i v e Methods. New Y o r k : W i l e y , 1975. Bogdan, R., and K s a n d e r , M. " P o l i c y D a t a as a S o c i a l P r o c e s s : A Q u a l i t a t i v e A p p r o a c h to Q u a n t i t a t i v e D a t a . " Human  O r g a n i z a t i o n 39(4) ( 1 9 8 0 ) : 302 - 309 . B o r g a t t a , E . F . " R e s e a r c h p r o b l e m s i n e v a l u a t i o n of h e a l t h s e r v i c e d e m o n s t r a t i o n s . " M i l l b a n k M e m o r i a l Fund Q u a r t e r l y v o l . 44, no. 4, p a r t 2 ( O c t o b e r 1966): 182-201 . B o r u c h , R., and Gomez, H. "Power t h e o r y i n s o c i a l program e v a l u a t i o n s . " I n P r o c e e d i n g s of the E v a l u a t i o n R e s e a r c h  S o c i e t y , n.p. E d i t e d by L. D a t t a and R.A. P e r l o f f . B e v e r l y H i l l s : Sage, 1979. B o s s e r t , T. " H e a l t h P o l i c i e s i n A f r i c a and L.A.: A d o p t i n g th e P r i m a r y C a r e A p p r o a c h . " S o c i a l S c i e n c e and M e d i c i n e 1 3 c ( 2 ) (1979) : 65-68 . Brown, E . J . " P u b l i c h e a l t h i n i m p e r i a l i s m . " A m e r i c a n J o u r n a l of  P u b l i c H e a l t h 66(9) ( 1 9 7 6 ) : . B r y a n t , J . H e a l t h and the D e v e l o p i n g W o r l d . I t h a c a : C o r n e l l U n i v e r s i t y P r e s s , 1969. C a m p b e l l , D.T. " F a c t o r s R e l e v a n t to the V a l i d i t y of E x p e r i m e n t s i n S o c i a l S e t t i n g s . " P s y c h o l o g i c a l B u l l e t i n 54 ( 1 9 5 7 ) : 297-312 . "Reforms as E x p e r i m e n t s . " A m e r i c a n P s y c h o l o g i s t 24 ( 1 969):409-429. C a m p b e l l , D.T., and S t a n l e y , J . C . E x p e r i m e n t a l and Q u a s i - E x p e r i m e n t a l D e s i g n s f o r R e s e a r c h . C h i c a g o : Rand M c N a l l y , 1966; r e p r i n t e d . , C h i c a g o : Rand M c N a l l y C o l l e g e P u b l i s h i n g Company, 1980. Canada, C a n a d i a n I n t e r n a t i o n a l D e velopment Agency ( C I D A ) , E v a l u a t i o n D i v i s i o n . P r o j e c t E v a l u a t i o n : P e r s p e c t i v e and  M e t h o d o l o g y Ottawa: C a n a d i a n I n t e r n a t i o n a l D e v e l o p m e n t Agency, O c t o b e r 1977. Canada, C a n a d i a n I n t e r n a t i o n a l Development Agency ( C I D A ) . " H e a l t h : CIDA's I n v o l v e m e n t . " Ottawa, A p r i l 1980. (Mimeographed.) 154 C a p o r a s o , J.A. and Roos, L . L . Q u a s i - E x p e r i m e n t a l A p p r o a c h e s :  T e s t i n g T h e o r y and E v a l u a t i n g P o l i c y . E v a n s t o n ( 1 1 1 . ) : N o r t h w e s t e r n U n i v e r s i t y P r e s s , 1973. C a s l e y , D . J . , and L u r y , D.A. D a t a C o l l e c t i o n i n D e v e l o p i n g  C o u n t r i e s . O x f o r d : C l a r e n d o n P r e s s , 1981. C h a g u l a , W.K., and T a r i m o , E. " M e e t i n g B a s i c H e a l t h Needs i n T a n z a n i a . " I n H e a l t h by the P e o p l e , pp. 145-168. E d i t e d by K. N e w e l l . Geneva: W o r l d H e a l t h O r g a n i z a t i o n , 1975. C h a p i n , F.S. E x p e r i m e n t a l D e s i g n s i n S o c i o l o g i c a l R e s e a r c h . New Y o r k : H a r p e r and B r o t h e r s , 1947. C o c h r a n , N. " C o g n i t i v e p r o c e s s e s , s o c i a l mores, and the a c c u m u l a t i o n o f d a t a : Program e v a l u a t i o n and t h e s t a t u s quo." E v a l u a t i o n Q u a r t e r l y 2 (1978):343-358. . " S o c i e t y as emergent and more t h a n r a t i o n a l : an e s s a y on t h e i n a p p r o p r i a t e n e s s o f program e v a l u a t i o n . " P o l i c y  S c i e n c e s 12 (1980):113-129. C o c h r a n , W.G., and Cox, G.M. E x p e r i m e n t a l D e s i g n s . New Y o r k : J o h n W i l e y , 1957. C o l e - K i n g , S. A p p r o a c h e s to the E v a l u a t i o n of M a t e r n a l and  C h i l d H e a l t h C a r e i n the C o n t e x t of P r i m a r y H e a l t h C a r e  Programmes. Geneva: W o r l d H e a l t h O r g a n i z a t i o n , 1979. Coleman, J . P o l i c y R e s e a r c h i n S o c i a l S c i e n c e s . M o r r i s t o w n , N . J . : G e n e r a l L e a r n i n g P r e s s , 1972. Committee on E v a l u a t i o n and S t a n d a r d s . " G l o s s a r y of E v a l u a t i v e Terms i n P u b l i c H e a l t h . " A m e r i c a n J o u r n a l of P u b l i c H e a l t h 60 ( 1 9 7 0 ) : 1546-1552 . C o n n o l l y , T., and P o r t e r , A.L. "A u s e r - f o c u s e d model f o r t h e u t i l i z a t i o n o f e v a l u a t i o n . " E v a l u a t i o n and Program P l a n n i n g 3 ( 2 ) (1980):131-140 . Cook, T.D. " Q u a s i - E x p e r i m e n t a t i o n : I t s O n t o l o g y , E p i s t e m o l o g y , and M e t h o d o l o g y . " I n Beyond Method: S t r a t e g i e s f o r S o c i a l  R e s e a r c h , pp.74-94. E d i t e d by G. Morgan. B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1983. Cook, T.D., and C a m p b e l l , D.T. Q u a s i - E x p e r i m e n t a t i o n : D e s i g n  and A n a l y s i s I s s u e s f o r F i e l d S e t t i n g s . B o s t o n : Houghton M i f f l i n Company, 1979. 155 Cook, T.D., and McAnany, E.G. "Rec e n t U n i t e d S t a t e s E x p e r i e n c e s i n E v a l u a t i o n R e s e a r c h w i t h I m p l i c a t i o n s f o r L a t i n A m e r i c a . " I n E v a l u a t i n g the Impact of N u t r i t i o n and H e a l t h  P rograms, pp. 39-75. E d i t e d by R.E. K l e i n e t a l . New Y o r k : Plenum P r e s s , 1979. Cook, T.D. and R e i c h a r d t , C.S., e d s . Q u a l i t a t i v e and Q u a n t i t a t i v e Methods i n E v a l u a t i o n R e s e a r c h . B e v e r l y H i l l s ; Sage P u b l i c a t i o n s , 1979. C o s t e l l a n o s , R . J . "Improved h e a l t h c o v e r a g e , c o o r d i n a t i o n of h e a l t h c a r e l e v e l s , and p r i m a r y h e a l t h c a r e . " Pan A m e r i c a n  H e a l t h O r g a n i z a t i o n B u l l e t i n 11(3) ( 1977): 195-205. C r o n b a c h , L . J . ; Ambron, S.R.; D o r n b u s c h , S.M.; H e s s , R.D.; H o r n i k , R.C.; P h i l l i p s , D.C.; W a l k e r , D.F.; and W e i n e r , S.S. Toward Reform o f Program E v a l u a t i o n . San F r a n s i s c o : J o s s e y - B a s s , 1980. d e K a l t , E . , and W i l l i a m s , G., e d s . S o c i o l o g y and D e v elopment. Lond on: T a v i s t o c k P u b l i c a t i o n s , 1974. D a n i e l s o n , R. "Cuban H e a l t h O r g a n i z a t i o n : H i s t o r y and D e v e l o p m e n t . " Ph.D. d i s s e r t a t i o n , U n i v e r s i t y o f M i c h i g a n , 1973 . D e n i s t o n , O.L. "Whether E v a l u a t i o n - Whether U t i l i z a t i o n . " E v a l u a t i o n and Program P l a n n i n g 3 ( 1 9 8 0 ) : 9 1 - 9 4 . D e n i s t o n , O.L., and R o s e n s t o c k , I.M. " E v a l u a t i n g H e a l t h P r o g r a m s . " P u b l i c H e a l t h R e p o r t s 85 (1970 ): 835-840. D e n i s t o n , O.L., and R o s e n s t o c k , I.M. "The V a l i d i t y of N o n e x p e r i m e n t a l D e s i g n s f o r E v a l u a t i n g H e a l t h S e r v i c e s . " H e a l t h S e r v i c e s R e p o r t s 88 ( 2 ) ( F e b r u a r y 1 9 73): 153-164. D e n i s t o n , O.L.; R o s e n s t o c k , I.M.; and G e t t i n g , M.D. " E v a l u a t i o n o f Program E f f e c t i v e n e s s . " P u b l i c H e a l t h R e p o r t s 83(4) ( 1 9 6 8 ) : 3 2 3 - 3 3 5 . D j u k a n o v i c , V., and Mach, E.P., e d s . A l t e r n a t i v e A p p r o a c h e s to  M e e t i n g B a s i c H e a l t h Needs i n D e v e l o p i n g C o u n t r i e s . Geneva: W o r l d H e a l t h O r g a n i z a t i o n , 1975. D o r o z y n s k i , A. D o c t o r s and H e a l e r s . Ottawa: I n t e r n a t i o n a l D e v e l o p m e n t R e s e a r c h C e n t r e ( I D R C ) , 1975. D r o b y n , A. " L a t i n A m e r i c a n E x p e r i e n c e R e l a t e d to the S o l u t i o n o f R u r a l H e a l t h P r o b l e m s i n t h e U.S." A m e r i c a n J o u r n a l P u b l i c H e a l t h 63 ( 1 9 7 3 ) : 66-70. 156 D u l a n s e y , M.L., ed. A p p r o a c h e s to A p p r o p r i a t e E v a l u a t i o n - A R e p o r t on a S e r i e s of Workshops on E v a l u a t i o n . New Y o r k : A m e r i c a n C o u n c i l o f V o l u n t a r y A g e n c i e s f o r F o r e i g n S e r v i c e I n c . , 1978. D u n l o p , D.D. Toward a H e a l t h P r o j e c t E v a l u a t i o n Framework. USAID E v a l u a t i o n S p e c i a l S t u d y , no. 8. W a s h i n g t o n , D.C.: U n i t e d S t a t e s Agency f o r I n t e r n a t i o n a l D e velopment (USAID), June 1982. E c h e v e r r i , 0., and S a r a v i a , J . S i s t e m a de s e r v i c i o s de s a l u d  r u r a l como un componente b a s i c o de l o s p r o y e c t o s de  d e s a r r o l l o r u r a l en C o l o m b i a [ R u r a l h e a l t h s e r v i c e s s y s t e m as a b a s i c component of r u r a l d e v e l o p m e n t p r o j e c t s i n C o l o m b i a ] . C a l i , C o l o m b i a : U n i v e r s i d a d d e l V a l l e , C e n t r o de I n v e s t i g a c i o n e s M u 1 t i d i s c i p 1 i n a r i a s en D e s a r r o l l o R u r a l (CIMDER), 1974. Edwards, W.; G u t t e n t a g , M.; and S n a p p e r , K. "A D e c i s i o n -T h e o r e t i c A p p r o a c h to E v a l u a t i o n R e s e a r c h . " I n Handbook of  E v a l u a t i o n R e s e a r c h , v o l . 1, n.p. E d i t e d by E.L. S t r e u n i n g and M. G u t t e n t a g . B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1975. E l z i n g a , A. E v a l u a t i n g the E v a l u a t i o n Game: On the M e t h o d o l o g y  of P r o j e c t E v a l u a t i o n , w i t h S p e c i a l R e f e r e n c e to  D evelopment C o o p e r a t i o n . S t o c k h o l m : S w e d i s h Agency f o r R e s e a r c h C o o p e r a t i o n w i t h D e v e l o p i n g C o u n t r i e s , 1981. E p s t e i n , L . , and T r i p o d i , T. R e s e a r c h T e c h n i q u e s f o r Program  P l a n n i n g , M o n i t o r i n g , and E v a l u a t i o n . New Y o r k : C o l u m b i a U n i v e r s i t y P r e s s , 1977. E s p i n g , H. " P l a n n i n g , i m p l e m e n t a t i o n and e v a l u a t i o n o f s o c i a l programmes." I n U r b a n d e v e l o p m e n t and Urban Renewal, I n t e r n a t i o n a l Forum S e r i e s . An e d i t e d s e l e c t i o n o f p a p e r s p r e s e n t e d a t t h e 5 t h W o r l d C o n g r e s s o f E n g i n e e r s and A r c h i t e c t s i n I s r e a l , c onvened by t h e I n t e r n a t i o n a l T e c h n i c a l C o o p e r a t i o n C e n t r e ( I T C C ) and t h e A s s o c i a t i o n o f E n g i n e e r s and A r c h i t e c t s i n I s r e a l i n December 1979, pp. 281-290. E d i t e d by D. Soen. London: George Godwin L i m i t e d , 1981 . F e n d a l l , N.R.E. A u x i l i a r i e s i n H e a l t h C a r e . B a l t i m o r e : J o h n s H o p k i n s P r e s s , 1972. . " P r i m a r y M e d i c a l C a r e i n D e v e l o p i n g C o u n t r i e s . " I n t e r n a t i o n J o u r n a l of H e a l t h S e r v i c e s 2 (2) ( 1 9 7 2 ) : 297 - 315 . 157 " A u x i l i a r i e s i n t h e H e a l t h Team." I n M e d i c a l C a r e A u x i l i a r i e s . PAHO/WHO S c i e n t i f i c P u b l i c a t i o n No. 278. W a s h i n g t o n , D.C.: Pan A m e r i c a n H e a l t h O r g a n i z a t i o n (PAHO), 1973, pp. 5-8. F e r n a n d e z , A.F.T. "The N a t i o n a l H e a l t h System o f Cuba." I n H e a l t h by the P e o p l e , pp.13-29. E d i t e d by K. N e w e l l . Geneva: W o r l d H e a l t h O r g a n i z a t i o n , 1975. F e u e r s t e i n , M.-T. " E v a l u a t i o n - By the P e o p l e . " I n t e r n a t i o n a l  N u r s i n g Review 25(5) ( 1 9 7 8 ) : 146-153. "The e d u c a t i v e a p p r o a c h i n e v a l u a t i o n : an a p p r o p r i a t e t e c h n o l o g y f o r a r u r a l h e a l t h programme." I n t e r n a t i o n a l  J o u r n a l o f H e a l t h E d u c a t i o n 21(1) ( 1 9 7 8 ) : 56-64. ' P a r t i c i p a t o r y e v a l u a t i o n - an a p p r o p r i a t e t e c h n o l o g y f o r community h e a l t h programmes." C o n t a c t 55 ( F e b r u a r y 1980):1-8 . F l a h e r t y , E.W., and M o r e l l , J.A. " E v a l u a t i o n s : M a n i f e s t a t i o n s o f a New F i e l d . " E v a l u a t i o n and Program P l a n n i n g 1 ( 1 ) ( 1 9 7 8 ) : 1 - 1 0 . II F o s t e r , G.M. " M e d i c a l A n t h r o p o l o g y and I n t e r n a t i o n a l H e a l t h S o c i a l S c i e n c e & M e d i c i n e 11 (2 ) ( 1 977): 527-534. F o s t e r , W.D. The C h u r c h M i s s i o n S o c i e t y and Modern M e d i c i n e i n  Uganda. (Newhaven ( E n g . ) : Newhaven P r e s s , 1978. F o u r n i e r , G., and D j e r m a k o y e , I.A. " V i l l a g e Teams i n N i g e r ( M a r a d i D e p a r t m e n t ) . " I n H e a l t h by the P e o p l e , pp. 128-144, E d i t e d by K. N e w e l l . Geneva: W o r l d H e a l t h O r g a n i z a t i o n , 1975 . F r a n k e n b e r g , R., and L e s s o n , J . "The S o c i o l o g y o f H e a l t h Dilemmas i n the P o s t - C o l o n i a l W o r l d : I n t e r m e d i a t e T e c h n o l o g y and M e d i c a l C a r e i n Zambia, Z a i r e and C h i n a . " I n S o c i o l o g y and D e v e l o p m e n t , pp. 255-278. E d i t e d by E. DeKadt and G. W i l l i a m s . London, T a v i s t o c k P u b l i c a t i o n s , 1974. Freeman, H.E., and Soloman, M.A. "The n e x t decade i n e v a l u a t i o n r e s e a r c h . " E v a l u a t i o n and Program P l a n n i n g 2 ( 4 ) ( 1 9 7 9 ) : 255-262. Freeman, H.; R o s s i , P.; and W r i g h t , S. E v a l u a t i n g S o c i a l P r o j e c t s i n D e v e l o p i n g C o u n t i r e s . P a r i s : O r g a n i z a t i o n f o r E c onomic C o o p e r a t i o n and D e v e l o p m e n t , 1979. 158 G e t t i n g , V. " E v a l u a t i o n . " A m e r i c a n J o u r n a l of P u b l i c H e a l t h 47 (1957) :400-413. G i s h , 0. P l a n n i n g the H e a l t h S e c t o r : The T a n z a n i a n E x p e r i e n c e . London: Croom Helm, 1975. G l a s s , G.V., ed. E v a l u a t i o n S t u d i e s A n n u a l Review. V o l . 1. B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1976. G l a s s , G.V.; W i l s o n , V.L.; and Gottman, J.M. D e s i g n and A n a l y s i s o f T i m e - s e r i e s E x p e r i m e n t s . B o u l d e r ( C o l o . ) : C o l o r a d o A s s o c i a t e d U n i v e r s i t y P r e s s , 1975. G o l l a d a y , F . L . , and Koch-Weser, C.K. "The New P o l i c i e s f o r R u r a l H e a l t h : I n s t i t u t i o n a l , s o c i a l and f i n a n c i a l c h a l l e n g e s t o l a r g e - s c a l e i m p l e m e n t a t i o n . " P r o c e e d i n g s of  the R o y a l S o c i e t y of London V o l . B, no. 199 ( 1977 ) : 169-178 . Gouvernement d ' H a i t i , D e p a r t e m e n t de l a S a n t e P u b l i q u e e t de l a P o p u l a t i o n . " R a p p o r t F i n a l du P r o j e t I n t e g r e de S a n t e e t de P o p u l a t i o n de P e t i t - G o a v e " [ F i n a l r e p o r t o f t h e i n t e g r a t e d h e a l t h and p o p u l a t i o n p r o j e c t a t P e t i t - G o a v e ] , P o r t - a u - P r i n c e , H a i t i : D e p a r t e m e n t de l a S a n t e P u b l i q u e e t de l a P o p u l a t i o n , 1979. G r e e n b e r g , B.G. " E v a l u a t i o n o f S o c i a l P r o g r a m s . " I n R e a d i n g s i n  E v a l u a t i o n R e s e a r c h , pp. 155-175. E d i t e d by F.G. C a r o . New Y o r k : R u s s e l l Sage F o u n d a t i o n , 1971. G r e e n b e r g , E.M. " E v a l u a t i n g t h e E f f e c t i v e n e s s o f M e n t a l H e a l t h S e r v i c e s . " M i l l b a n k M e m o r i a l Fund Q u a r t e r l y 44 ( 1 9 6 6 ) : 1-402. Guba, E.G. Toward a m e t h o d o l o g y o f n a t u r a l i s t i c i n q u i r y i n e d u c a t i o n a l e v a l u a t i o n . Los A n g e l e s : UCLA C e n t e r f o r the S t u d y o f E d u c a t i o n , 1978. G u t t e n t a g , M. " S u b j e c t i v i t y and i t s Use i n E v a l u a t i o n R e s e a r c h . " E v a l u a t i o n 1 (2) ( 1 9 7 3 ) : 60-65 . G u t t e n t a g , M., and S t r e u n i n g , E . L . , e d s . Handbook of E v a l u a t i o n  R e s e a r c h . V o l 2. B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1975. G w a t k i n , D.R.; W i l c o x , J.R.; and Wray, J.D. "The p o l i c y i m p l i c a t i o n s o f f i e l d e x p e r i m e n t s i n p r i m a r y h e a l t h and n u t r i t i o n c a r e . S o c i a l S c i e n c e & M e d i c i n e 14(C) ( 1 9 8 0 ) : 121-128. 159 G w a t k i n , D.R.; W i l c o x , J.R.; and Wray, J.D. Can H e a l t h and  N u t r i t i o n I n t e r v e n t i o n Make a D i f f e r e n c e ? W a s h i n g t o n : O v e r s e a s Development C o u n c i l , F e b r u a r y 1980. H a b i c h t , J . - P . " H e a l t h f o r a l l by the y e a r 2000." A m e r i c a n  J o u r n a l o f P u b l i c H e a l t h 71(5) ( 1 981):459-461. H a b i c h t , J . - P . , and Berman, P.A. " P l a n n i n g P r i m a r y H e a l t h S e r v i c e s f r o m a Body C o u n t ? . " S o c i a l S c i e n c e & M e d i c i n e 14C ( 1 980):129-136. H a b i c h t , J . - P . , and B u t z , W.P. "Measurement o f H e a l t h and N u t r i t i o n E f f e c t s o f L a r g e S c a l e I n t e r v e n t i o n P r o j e c t s . " I n E v a l u a t i n g the Impact of N u t r i t i o n and H e a l t h P rograms, pp. 133-169. E d i t e d by R.E. K l e i n e t a l . New Y o r k : Plenum P r e s s , 1979. H a b i c h t , J . - P . , and t h e W o r k i n g Group on R u r a l M e d i c a l C a r e . " D e l i v e r y o f p r i m a r y c a r e by m e d i c a l a u x i l i a r i e s : T e c h n i q u e s o f u s e and a n a l y s i s o f b e n e f i t s a c h i e v e d i n some r u r a l v i l l a g e s i n G u a t e m a l a . " I n M e d i c a l C a r e A u x i l i a r i e s . PAHO/WHO S c i e n t i f i c P u b l i c a t i o n no. 278. W a s h i n g t o n , D.C.: Pan A m e r i c a n H e a l t h O r g a n i z a t i o n , 1973, pp.24-37. H a l l , B. " P a r t i c i p a t o r y R e s e a r c h : B r e a k i n g t h e Academic Monopoly." T o r o n t o , n.d. (Mimeographed.) . " P a r t i c i p a t o r y R e s e a r c h : E x p a n d i n g th e Base o f A n a l y s i s . " I n t e r n a t i o n a l Development R e v i e w / F o c u s ( 1 9 7 7 ) : 2 3 - 2 8 . " B r e a k i n g t h e Monopoly on Knowledge: R e s e a r c h Methods, P a r t i c i p a t i o n and D e v e l o p m e n t . " I n A d u l t L e a r n i n g :  A D e s i g n f o r A c t i o n , pp. 156-159. E d i t e d by B.L. H a l l and J.R. K i d d . U n i t e d Kingdom: Pergamon P r e s s , 1978. H a l l , B.; E t h e r i n g t o n , A.; and J a c k s o n , T. " E v a l u a t i o n , P a r t i c i p a t i o n and Community H e a l t h : C r i t i q u e and L e s s o n s . " T o r o n t o : I n t e r n a t i o n a l C o u n c i l f o r A d u l t E d u c a t i o n , 1979. (Mimeographed.) Hayes, S. M e a s u r i n g the r e s u l t s o f d e v e l o p m e n t p r o j e c t s . P a r i s : O r g a n i z a t i o n f o r Economic C o o p e r a t i o n and D e v e l o p m e n t , 1959 . H e i l m a n , J.G. " P a r a d i g m a t i c C h o i c e s i n E v a l u a t i o n M e t h o d o l o g y . " E v a l u a t i o n Review 4 ( 5 ) ( 1 9 8 0 ) : 693-712. 160 H e n d r a t t a , L. " H e a l t h and D e v e l o p m e n t : Review o f P o l i c y O p t i o n s a t t h e G r a s s R o o t s L e v e l . " I n The F u t u r e of Academic  Community M e d i c i n e i n D e v e l o p i n g C o u n t r i e s , pp. 58-68. E d i t e d by W. Lathem. New Y o r k : P r a e g e r P u b l i s h e r s , 1979. H e r s e y , J . C . " D i r t y r e s e a r c h i n r e a l p l a c e s : A p r a c t i t i o n e r ' s g u i d e to program e v a l u a t i o n i n t h e human s e r v i c e s . " E v a l u a t i o n and Program P l a n n i n g 2(2) (1979 ) : 153-157 . H e r z o g , E. Some G u i d e l i n e s f o r E v a l u a t i o n R e s e a r c h . W a s h i n g t o n , D.C.: U.S. D e p a r t m e n t o f E d u c a t i o n , H e a l t h and W e l f a r e , 1959 . Holmes, A.E. " E v a l u a t i o n i n H e a l t h E d u c a t i o n . " J o u r n a l o f  T r o p i c a l M e d i c i n e & H y g i e n e 66(1) ( 1 9 6 3 ) : 23-24. H o o l e , F.W. E v a l u a t i o n R e s e a r c h and Development A c t i v i t i e s . Sage L i b r a r y o f S o c i a l R e s e a r c h , v o l . 68. B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1978. House, E.R. E v a l u a t i n g w i t h V a l i d i t y . B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1980. Hughes, J . P . , ed. H e a l t h C a r e i n Remote A r e a s . O a k l a n d ( C a l i f . ) : K a i s e r F o u n d a t i o n I n t e r n a t i o n a l , 1972. Hyman, H.H., and W r i g h t , C.R. " E v a l u a t i n g S o c i a l A c t i o n P r o g r a m s . " I n R e a d i n g s i n E v a l u a t i o n R e s e a r c h , pp. 18 5-220. E d i t e d by F.G. C a r o . New Y o r k : R u s s e l l Sage F o u n d a t i o n , 1971 . Imboden, N. A Management A p p r o a c h to P r o j e c t A p p r a i s a l and  E v a l u a t i o n , w i t h S p e c i a l R e f e r e n c e to N o n - D i r e c t l y  P r o d u c t i v e P r o j e c t s . P a r i s : O r g a n i z a t i o n f o r Economic C o o p e r a t i o n and D e v e l o p m e n t , 1978. I n g l e s i a s , M., and V i l l a f o r e , P. S i s t e m a r u r a l de s e r v i c i o s de  s a l u d : i n f o r m e p r e l i m i n a r s o b r e o r g a n i z a c i o n de l a  comunidad [ R u r a l h e a l t h s e r v i c e s s y s t e m : p r e l i m i n a r y r e p o r t on community o r g a n i z a t i o n ] , C a l i , C o l o m b i a : U n i v e r s i d a d d e l V a l l e , C e n t r o de I n v e s t i g a c i o n e s M u 1 t i d i s c i p 1 i n a r i a s en D e s a r o l l o R u r a l (CIMDER), 1977. I n t e r n a t i o n a l Bank f o r R e c o n s t r u c t i o n and D e v e l o p m e n t . The A s s a u l t on World P o v e r t y : P r o b l e m s o f R u r a l D e v e l o p m e n t ,  E d u c a t i o n , and H e a l t h . B a l t i m o r e : The J o h n s H o p k i n s U n i v e r s i t y P r e s s , 1975. 161 I n t e r n a t i o n a l D evelopment R e s e a r c h C e n t r e . SALUS: Low C o s t  R u r a l H e a l t h C a r e and H e a l t h Manpower T r a i n i n g : an  a n n o t a t e d b i b l i o g r a p h y w i t h s p e c i a l emphasis on d e v e l o p i n g  c o u n t r i e s . Ottawa: I n t e r n a t i o n a l D evelopment R e s e a r c h C e n t r e ( I D R C ) , 1975 ( v o l . 1)-1983 ( v o l . 1 3 ) . J o h a d a , M., and B a r n i t z , E. "The N a t u r e o f E v a l u a t i o n . " I n t e r n a t i o n a l S o c i a l S c i e n c e J o u r n a l 7 (1955):353-364. J o s h i , C.B.; B h a t i a , A.K.; S a p r u , R.; and L a z a r e n k o , I.A. "A s t u d y on m u l t i p u r p o s e h e a l t h w o r k e r s scheme i n f i v e f i e l d p r a c t i c e d e m o n s t r a t i o n a r e a s o f M a h a r a s h t r a . " H e a l t h and  P o p u l a t i o n - P e r s p e c t i v e s & I s s u e s 1(2) (1978):123-143. Kassam, Y., and M u s t a f a , K., e d s . P a r t i c i p a t o r y R e s e a r c h : an  E m e r g i n g A l t e r n a t i v e M e t h o d o l o g y i n S o c i a l S c i e n c e  R e s e a r c h . New D e l h i : S o c i e t y f o r P a r t i c i p a t o r y R e s e a r c h i n A s i a , 1982. K i n g , M. M e d i c a l C a r e i n D e v e l o p i n g C o u n t r i e s . London: O x f o r d U n i v e r i s t y P r e s s , 1966. Kingma, S. " E d i t o r i a l N o t e . " C o n t a c t 75 ( 1 9 8 3 ) : l - 2 . K i r k b y , A.U. The Development of a U s e r - C h o i c e A p p r o a c h i n R u r a l  Water S u p p l y . R u r a l Water S u p p l y and S a n i t a t i o n W o r k i n g P a p e r No. 7. Ottawa: I n t e r n a t i o n a l D e velopment R e s e a r c h C e n t r e ( I D R C ) , 1973. K l e i n , R.E.; Read, M.S.; R i e k e n , H.W.; Brown, J.A.; P r a d i l l a , A.; and Daza, C.H. E v a l u a t i n g the Impact of N u t r i t i o n and  H e a l t h P r o g r a m s . New Y o r k : Plenum P r e s s , 1979. K l i n e b e r g , 0. "The P r o b l e m o f E v a l u a t i o n . " I n t e r n a t i o n a l S o c i a l  S c i e n c e J o u r n a l 7 ( 1 9 5 5 ) : 346-350. Kpedekpo, G.M.K. "The P l a n n i n g and D e s i g n o f S a m p l i n g S u r v e y s w i t h P a r t i c u l a r R e f e r e n c e to t h e E p i d e m i o l o g i c a l S u r v e y o f the D a n f a P r o j e c t i n Ghana." Ghana M e d i c a l J o u r n a l (December 1972 ) : 377 - 382 . K u r z , D.E. "The use o f p a r t i c i p a n t o b s e r v a t i o n i n e v a l u a t i o n r e s e a r c h . " E v a l u a t i o n and Program P l a n n i n g 6 ( 2 ) ( 1 9 8 3 ) : 93-102 . Lathem, W., ed. The F u t u r e of Academic Community M e d i c i n e i n  D e v e l o p i n g C o u n t r i e s . New Y o r k : P r a e g e r P u b l i s h e r s , 1979. 162 L i p t o n , M. and Moore, M.P. The M e t h o d o l o g y of V i l l a g e S t u d i e s  i n LDCs. B r i g h t o n ( E n g . ) : U n i v e r s i t y of S u s s e x , I n s t i t u t e o f D e velopment S t u d i e s , D i s c u s s i o n P a p e r #10, 1976. L i t c h f i e l d , N.; K e t t l e , P.; and W h i t b r e a d , M. E v a l u a t i o n i n the  P l a n n i n g P r o c e s s . London: Pergamon P r e s s , 1965. L u m s d a i n e , A.A., and B e n n e t t , C.A. " A s s e s s i n g A l t e r n a t i v e c o n c e p t i o n s o f e v a l u a t i o n . " I n E v a l u a t i o n and E x p e r i m e n t , pp. 525-553. E d i t e d by C.A. B e n n e t t and A.A. L u m s d a i n e . New Y o r k : A c a d e m i c P r e s s I n c . , 1975. M c K e n z i e , J . S . " P u t t i n g a P r i c e Tag on H e a l t h : A T e n - D o l l a r H e a l t h P l a n . " I n t e r n a t i o n a l J o u r n a l of H e a l t h S e r v i c e s 16(1) ( 1 9 7 4 ) : 2 7 - 2 8 . M a h l e r , H . " H e a l t h f o r a l l by t h e y e a r 2000." WHO C h r o n i c l e 29 (1975) :457-461 . M a l l o q u i n , R.H.; R o l o n , P.A.; and K u n z l e , C. I n f o r m e F i n a l y C o n c l u s i o n e s d e l Programa E x t e n s i o n de l o s S e r v i c i o s S a l u d  a t r a v e s de M a e s t r o s R u r a l e s . [ F i n a l r e p o r t and c o n c l u s i o n s c o n c e r n i n g t h e e x t e n s i o n o f h.ealth s e r v i c e s t h r o u g h r u r a l s c h o o l t e a c h e r s ] . A s u n c i o n , P a r a g u a y : M i n i s t e r i o de S a n i d a d y A s i s t e n c i a S o c i a l , 1980. Mathur, Y.C. and Kumar, A. " R o l e o f V i l l a g e H e a l t h Worker i n H e a l t h P r o b l e m s o f D e v e l o p i n g C o u n t r i e s . " T r o p i c a l  P e d i a t r i c s and E n v i r o n m e n t a l C h i l d H e a l t h 24(3) ( 1 9 7 8 ) : 133-135 . M e s s i n g , S . D . ; P r i n c e , J . S . ; and Yohannes, T. "Method o f h e a l t h c u l t u r e r e s e a r c h i n an A f r i c a n c o u n t r y . " J o u r n a l of H e a l t h  & Human B e h a v i o u r 6 ( W i n t e r 1965):261-263. M i l i o , N. The Care of H e a l t h i n C ommunities: A c c e s s f o r  O u t c a s t s . New Y o r k : M a c M i l l a n , 1975. Community H e a l t h W o r k e r s : E v a l u a t i o n of t h e i r Impact on H e a l t h . New Y o r k : I n t e r n a t i o n a l M e d i c a l and R e s e a r c h F o u n d a t i o n , 1977, pp. 11-25. Mo 1ina-Guzman, G. " T h i r d W o r l d E x p e r i e n c e s i n H e a l t h P l a n n i n g . " I n t e r n a t i o n a l J o u r n a l of H e a l t h S e r v i c e s 9 ( 1 ) ( 1 9 7 9 ) : 139-150. Morgan, G., ed. Beyond Method: S t r a t e g i e s f o r S o c i a l R e s e a r c h . B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1983. 163 M o r l e y , D. " P a e d i a t r i c P r i o r i t i e s i n E v o l v i n g Community Programs f o r D e v e l o p i n g C o u n t r i e s . " L a n c e t 7993 ( 1 9 7 8 ) : 1013 . "The P a r t Time H e a l t h Worker i n t h e D e l i v e r y o f H e a l t h C a r e . " J o u r n a l of T r o p i c a l M e d i c i n e & H y g i e n e 81(11) (1978) : 212-220 . N a t i o n a l C o u n c i l f o r I n t e r n a t i o n a l H e a l t h . S e l e c t e d P a p e r s from  I n t e r n a t i o n a l H e a l t h C o n f e r e n c e - W a s h i n g t o n , A p r i l 25-27,  1973. C h i c a g o : A m e r i c a n M e d i c a l A s s o c i a t i o n , 1973. ._ I n t e r n a t i o n a l H e a l t h : M e a s u r i n g P r o g r e s s -P r o c e e d i n g s 1980 I n t e r n a t i o n a l H e a l t h C o n f e r e n c e . W a s h i n g t o n , D.C.: NCIH, 1980. N a v a r r o , V. " H e a l t h S e r v i c e s i n Cuba: An I n i t i a l A p p r a i s a l . " New Engand J o u r n a l of M e d i c i n e 287(19) 19 7 2:954-959. Neumann, A.K.; Ward, W.6.; Pappoe, M.E.; and Boyd, D.L. " E d u c a t i o n and e v a l u a t i o n i n an i n t e g r a t e d MCH/FP p r o j e c t i n r u r a l Ghana: the D a n f a p r o j e c t . " I n t e r n a t i o n a l J o u r n a l  of H e a l t h E d u c a t i o n 19(4) ( 1 9 7 6 ) : 233-244. N e w e l l , K., ed. H e a l t h by the P e o p l e . Geneva: World H e a l t h O r g a n i z a t i o n , 1975. O ' B a r r , W i l l i a m 0.; S p a i n , D a v i d H.; and T e s s l e r , Mark A. S u r v e y R e s e a r c h i n A f r i c a : I t s A p p l i c a t i o n s and L i m i t s . E v a n s t o n : N o r t h w e s t e r n U n i v e r s i t y P r e s s , 1973. Pan A m e r i c a n H e a l t h O r g a n i z a t i o n . H e a l t h F o r A l l by the Y e a r  2000 - S t r a t e g i e s . W a s h i n g t o n , D.C.: n.p., 1980. P a r l a t o , M.B., and F a v i n , M.N. P r i m a r y H e a l t h C a r e : P r o g r e s s  and P r o b l e m s . An A n a l y s i s of 52 A I D - a s s i s t e d P r o j e c t s . W a s h i n g t o n , D.C.: A m e r i c a n P u b l i c H e a l t h A s s o c i a t i o n (APHA), 1982. P a r l e t t , M., and H a m i l t o n , D. E v a l u a t i o n as I l l u m i n a t i o n : A New  A p p r o a c h to the Study of I n n o v a t i n g P r o g r a m s . E d i n b u r g h : U n i v e r s i t y o f E d i n b u r g h , C e n t r e f o r R e s e a r c h i n the E d u c a t i o n a l S c i e n c e s , 1972. P a t t o n , M.Q. U t i l i z a t i o n - F o c u s e d E v a l u a t i o n . B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1978. Q u a l i t a t i v e E v a l u a t i o n Methods. B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1980. 164 "Making methods c h o i c e s . " E v a l u a t i o n and Program P l a n n i n g 3 ( 4 ) ( 1 9 8 0 ) : 219 - 228. P e r r i n , B.T. "Use o f E v a l u a t i o n : D e c i s i o n - M a k i n g , or ...." I n C a n a d i a n E v a l u a t i o n S o c i e t y , P r o c e e d i n g s of the 2nd A n n u a l  C o n f e r e n c e . T o r o n t o : n.p., 1981, pp. 172-174. " P r i m a r y H e a l t h C a r e : A F i r s t A s s e s s m e n t . " P e o p l e 1 0 ( 3 ) ( 1 9 8 3 ) : 6 - 9 . " P r o g r e s s s i n c e Alma A t a : H e a l t h and E q u i t y . " P e o p l e 10(2) ( 1 9 8 3 ) : 3-5 . Ramchandran, L. " D e v e l o p i n g I n d e c e s f o r t h e E v a l u a t i o n o f H e a l t h S e r v i c e s . " I n d i a n J . P u b l i c H e a l t h ( C a l c u t t a ) 11(1) ( 1 9 6 7 ) : 3 2 - 3 4 . Rao, P.S., and R i c h a r d , J . " M e a s u r i n g Community R e s p o n s e s to H e a l t h C e n t r e Programmes." I n d i a n J o u r n a l o f M e d i c a l  R e s e a r c h 58 ( 1 9 7 0 ) : 938-946 . Rao, P.S.; B e n j a m i n , V.; and R i c h a r d , J . "Methods o f e v a l u a t i n g h e a l t h c e n t r e s . " B r i t i s h J o u r n a l of P r e v e n t i v e and S o c i a l  M e d i c i n e (London) 26 (1972):46 - 52. R e i c h a r d t , C.S., and Cook, T.D. "Beyond q u a l i t a t i v e v e r s u s q u a n t i t a t i v e methods." I n Q u a l i t a t i v e and q u a n t i t a t i v e  methods i n e v a l u a t i o n r e s e a r c h , pp. 7-32. E d i t e d by T.D. Cook and C.S. R e i c h a r d t . B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1979 . R i f k i n , S.B., ed. Community H e a l t h i n A s i a . S i n g a p o r e : C h r i s t i a n C o n f e r e n c e o f A s i a Committee on H e a l t h C o n c e r n s , June 1977. . " P l a n n e r s ' a p p r o a c h e s to community p a r t i c i p a t i o n i n h e a l t h programmes: T h e o r y and r e a l i t y . " C o n t a c t 75 ( O c t o b e r 1983): 3-16 . R o b e r t s , B. " R e s e a r c h i n E d u c a t i o n A s p e c t s o f H e a l t h Programmes." I n t e r n a t i o n a l J o u r n a l o f H e a l t h E d u c a t i o n 1 3 ( 1 ) , s u p p l e m e n t ( 1 9 7 0 ) : 33p. Roemer, M.I. H e a l t h Care System i n World P e r s p e c t i v e . Ann A r b o r ( M i c h . ) : H e a l t h A d m i n i s t r a t i o n P r e s s , 1976. Ronaghy, H.A. " K a v a r V i l l a g e H e a l t h Worker P r o j e c t . " J o u r n a l of  T r o p i c a l P e d i a t r i c s and E n v i r o n m e n t a l C h i l d H e a l t h 2 4 ( 1 ) , Monograph No. 52 ( 1 9 7 8 ) : 13-60 . 165 R o n d i n e l l i , D.A. " I n t e r n a t i o n a l R e q u i r e m e n t s f o r P r o j e c t P r e p a r a t i o n : A i d s o r O b s t a c l e s to Development P l a n n i n g ? . " A m e r i c a n I n s t i t u t e of P l a n n e r s J o u r n a l 13 ( 1 9 7 6 ) : 313-326 . Roos, N.P. " E v a l u a t i o n , q u a s i - e v a l u a t i o n , and p u b l i c p o l i c y : o b s e r v a t i o n s by a s h o r t - t e r m b u r e a u c r a t . " I n Q u a s i - e x p e r i m e n t s : t e s t i n g t h e o r y and e v a l u a t i n g p o l i c y , pp.281-304. E d i t e d by J . C a p o r a s o and L . L . Roos, J r . C h i c a g o : N o r t h w e s t e r n U n i v e r s i t y P r e s s , 1973. R o s s i , P.H., and W i l l i a m s , W., e d s . E v a l u a t i n g S o c i a l Programs:  T h e o r y , P r a c t i c e s and P o l i t i c s . New Y o r k : Seminar P r e s s , 1972 . R o s s i , P.H.; Freeman, H.E.; and W r i g h t , S.R. E v a l u a t i o n : A  S y s t e m a t i c A p p r o a c h . B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1979 . S a r a v i a , J . , and E c h e v e r r i , 0. " P r o y e c t o d e l s i s t e m a r u r a l de s e r v i c i o s : E v a l u a c i o n d e l s i s t e m a r u r a l de s e r v i c i o s de s a l u d 1977-1979." [ R u r a l h e a l t h s e r v i c e s s y s t e m p r o j e c t : E v a l u a t i o n o f t h e r u r a l h e a l t h c a r e s y s t e m 1977-1979], C a l i , C o l o m b i a : U n i v e r s i d a d d e l V a l l e , C e n t r o de I n v e s t i g a c i o n e s M u 1 t i d i s c i p 1 i n a r i a s en D e s a r r o l l o R u r a l (CIMDER), 1980. S c h u l b e r g , H . C , and B a k e r , F. "Program E v a l u a t i o n M o d e ls and the I m p l e m e n t a t i o n o f R e s e a r c h F i n d i n g s . " A m e r i c a n J o u r n a l  of P u b l i c H e a l t h 5 8(7) ( 1 9 6 8 ) : 1248-1255 . . , e d s . Program E v a l u a t i o n i n the H e a l t h F i e l d s . V o l . 2. New Y o r k : Human S c i e n c e s P r e s s , 1979. S c h u l b e r g , H . C ; S h e l d o n , A.; and B a k e r , F., e d s . Program  E v a l u a t i o n i n the H e a l t h F i e l d s . New Y o r k : B e h a v i o u r a l P r e s s , 1969. Schwab, G. E v a l u a t i o n Handbook. 2nd ed. W a s h i n g t o n , D . C : U n i t e d S t a t e s Agency f o r I n t e r n a t i o n a l Development (USAID), O f f i c e o f Program E v a l u a t i o n , 1972. S c r i v e n , M. " P r o s e and Cons a b o u t G o a l - F r e e E v a l u a t i o n . " E v a l u a t i o n Comment 3 ( 4 ) (December 1 9 7 2 ) : l - 4 . S c r i v e n , M. " E v a l u a t i o n P e r s p e c t i v e s and P r o c e d u r e s . " i n E v a l u a t i o n i n E d u c a t i o n : C u r r e n t A p p l i c a t i o n s , pp.1-93. E d i t e d by W.J. Popham. B e r k e l e y : M cCutchan, 1974. 166 S h a r p s t o n , M.J. "Uneven G e o g r a p h i c a l D i s t r i b u t i o n of M e d i c a l C a r e : A G h a n i a n Case S t u d y . " J o u r n a l of Development S t u d i e s 8 ( 2 ) (1972):205-216. S h o r t e l l , S.M., and R i c h a r d s o n , W.C. H e a l t h Program E v a l u a t i o n . S t . L o u i s : The C.V. Mosby Company, 1978. S i n n e t t e , C.H. " E v a l u a t i o n : A Measurement Dilemma," I n I n t e r n a t i o n a l H e a l t h : M e a s u r i n g P r o g r e s s - P r o c e e d i n g s 1980  I n t e r n a t i o n a l H e a l t h C o n f e r e n c e , pp. 8-11. W a s h i n g t o n , D.C.: N a t i o n a l C o u n c i l f o r I n t e r n a t i o n a l H e a l t h (NCIH), 1980. S m i t h , N.L. "The F e a s i b i l i t y and D e s i r a b i l i t y o f E x p e r i m e n t a l Methods i n E v a l u a t i o n . " E v a l u a t i o n and Program P l a n n i n g 3 ( 1 980):251-256. "The C e r t a i n t y o f Judgements i n H e a l t h E v a l u a t i o n s . " E v a l u a t i o n and Program P l a n n i n g 4(3 & 4) (1981) : 273 - 278 . . " D e v e l o p i n g E v a l u a t i o n Methods." I n M e t a p h o r s f o r E v a l u a t i o n : S o u r c e s of New Methods, pp. 17-49. E d i t e d by N i c k L. S m i t h . New P e r s p e c t i v e s i n E v a l u a t i o n , v o l . 1. B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1981. . M e t a p h o r s f o r E v a l u a t i o n : S o u r c e s of New Methods. New P e r s p e c t i v e s i n E v a l u a t i o n , v o l . 1. B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1981. S t a k e , R.E. E v a l u a t i n g the a r t s i n e d u c a t i o n : A r e s p o n s i v e  a p p r o a c h . Columbus, O h i o : C h a r l e s E. M e r r i l l , 1975. . "The Case S t u d y Method i n S o c i a l I n q u i r y . " E d u c a t i o n a l R e s e a r c h e r 7 ( F e b r u a r y 1978):5-8. "The Case S t u d y Method i n S o c i a l E n q u i r y . " The O c c a s i o n a l R e s e a r c h e r A m e r i c a n E d u c a t i o n R e s e a r c h A s s o c i a t i o n , F e b r u a r y 1978. S t i f f m a n , L.H. Some H e a l t h S e c t o r Methods f o r D e v e l o p i n g  N a t i o n s . W a s h i n g t o n , D.C.: U n i t e d S t a t e s Agency f o r I n t e r n a t i o n a l D evelopment (USAID), 1974. S t r e u n i n g , E . L . , and G u t t e n t a g , M. Handbook of E v a l u a t i o n R e s e a r c h . V o l . 1. B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1975. Suchman, E.A. E v a l u a t i v e R e s e a r c h : P r i n c i p l e s and P r a c t i c e i n  P u b l i c S e r v i c e and S o c i a l A c t i o n P r o g r ams. New Y o r k : R u s s e l l Sage F o u n d a t i o n , 1967. 167 S u l l i v a n , D.F. C o n c e p t u a l P r o b l e m s i n D e v e l o p i n g an Index o f  H e a 1 t h, S e r i e s 2, no. 17, p. 1-18. W a s h i n g t o n , D.C.: N a t i o n a l C e n t e r f o r H e a l t h S t a t i s t i c s , 1966. S u s s e r , M. C a u s a l T h i n k i n g i n the H e a l t h S c i e n c e s . O x f o r d : O x f o r d U n i v e r s i t y P r e s s , 1973. Swantz, M.L. " R e s e a r c h as an E d u c a t i o n a l T o o l f o r D e v e l o p m e n t . " C o n v e r g e n c e 8 ( 2 ) ( 1975):44-53. T a u s s i g , M. " N u t r i t i o n , D e v elopment and F o r e i g n A i d : A Case S t u d y o f U . S . - D i r e c t e d H e a l t h C a r e i n a C o l o m b i a n P l a n t a t i o n Zone." I n t e r n a t i o n a l J o u r n a l of H e a l t h S e r v i c e s 8 ( 1 ) ( 1 9 7 8 ) : 101-121 . T e n d l e r , J . T u r n i n g p r i v a t e v o l u n t e e r o r g a n i z a t i o n s i n t o  d e v e l o p m e n t a g e n c i e s : q u e s t i o n s f o r e v a l u a t i o n . USAID Program E v a l u a t i o n D i s c u s s i o n P a p e r No. 12. W a s h i n g t o n , D.C.: U n i t e d S t a t e s Agency f o r I n t e r n a t i o n a l Development (USAID), 1982. T e n t o r i , F.V. " P r i m a r y h e a l t h c a r e : a s t r a t e g y f o r e x t e n d i n g c o v e r a g e to t h e u n s e r v e d p o p u l a t i o n . " B u l l e t i n of the Pan  A m e r i c a n H e a l t h O r g a n i z a t i o n 15(4) (1981):354-360. Townsend, J.W.; F a r r e l l , W.T.; and K l e i n , R.E. " S p e c i a l I s s u e s f o r t h e Measurement o f Program Impact i n D e v e l o p i n g C o u n t r i e s . " I n E v a l u a t i n g the Impact of N u t r i t i o n and  H e a l t h P r ograms, pp. 99-123. E d i t e d by R.E. K l e i n e t a l . New Y o r k : Plenum P r e s s , 1979. T u n g s u b u t r a , K. " H e a l t h c a r e d e l i v e r y s y s t e m to r e a c h t h e p e o p l e . " A s p e c t s : the I n t e r n a t i o n a l J o u r n a l of V o l u n t e e r  S e r v i c e s ( W a s h i n g t o n ) 1 ( 4 ) ( 1 9 7 7 ) : 4 - 8 . T u n g s u b u t r a , K. " V o l u n t e e r H e a l t h W o r k e r s : A R e s e a r c h P r o j e c t c a r r i e d o u t by t h e F a c u l t y o f M e d i c i n e , Khon Kaen U n i v e r s i t y , " Khon Kaen, T h a i l a n d : Khon Kaen U n i v e r s i t y , 1979 . T u r s h e n , M., "The Impact of C o l o n i a l i s m on H e a l t h and H e a l t h S e r v i c e s i n T a n z a n i a . " I n t e r n a t i o n a l J o u r n a l of H e a l t h S e r v i c e s 7 ( 1 ) ( 1 9 7 7 ) : 7 - 3 5 . Twumasi, P.A. " C o l o n i a l i s m and I n t e r n a t i o n a l H e a l t h : A S t u d y i n S o c i a l Changes i n Ghana." S o c i a l S c i e n c e & M e d i c i n e 15B(2) ( 1 9 8 1 ) : 147-151 . 168 U n i t e d N a t i o n s , C o n f e r e n c e on Human S e t t l e m e n t s . "Community-Development A p p r o a c h e s f o r the Improvement o f H e a l t h . " I n H e a l t h I m p l i c a t i o n s i n the Development of the P h y s i c a l and  S o c i a l E n v i r o n m e n t s of Human S e t t l e m e n t s . V a n c o u v e r : n.p., 1976 . U n i t e d N a t i o n s , U n i t e d N a t i o n s E d u c a t i o n a l , S c i e n t i f i c and C u l t u r a l O r g a n i z a t i o n . E v a l u a t i n g S o c i a l A c t i o n P r o j e c t s :  P r i n c i p l e s , M e t h o d o l o g i c a l A s p e c t s and S e l e c t e d Examples. S o c i o - e c o n o m i c S t u d i e s No. 1. P a r i s : UNESCO, 1980. U n i t e d N a t i o n s , W o r l d H e a l t h O r g a n i z a t i o n . F i r s t R e g i o n a l Seminar on R u r a l H e a l t h S e r v i c e s F i n a l R e p o r t . M a n i l a : WHO R e g i o n a l O f f i c e f o r the W e s t e r n P a c i f i c , J u l y 1963. P l a n n i n g and E v a l u a t i o n of H e a l t h E d u c a t i o n S e r v i c e s . WHO T e c h n i c a l R e p o r t S e r i e s , no. 409, 1969. ._ Towards a P h i l o s o p h y of H e a l t h Work i n the A f r i c a n R e g i o n . A f r i c a n R e g i o n a l O f f i c e T e c h n i c a l P a p e r No. 1. B r a z z a v i l l e : n.p., 1970. F i n a l R e p o r t - F i r s t R e g i o n a l Seminar on R u r a l H e a l t h S e r v i c e s . B r a z z a v i l l e : WHO R e g i o n a l O f f i c e f o r A f r i c a , J a n u a r y 1974. ._ H e a l t h A s p e c t s of P o p u l a t i o n T r e n d s and P r o s p e c t s . W o r l d H e a l t h S t a t i s t i c a l R e p o r t , v o l . 27, no. 5, 1974. ._ The F i f t h R e p o r t on the World H e a l t h S i t u a t i o n 1969-1972 - P a r t I I Review by C o u n t r y and T e r r i t o r y . Geneva: W o r l d H e a l t h O r g a n i z a t i o n , 1974. ._ " P r i m a r y H e a l t h C a r e i n t h e WHO R e g i o n s . " WHO C h r o n i c l e 32 (1978):431-438. ._ R e p o r t of the R e g i o n a l D i r e c t o r f o r the E u r o p e a n R e g i o n . I n t e r n a t i o n a l C o n f e r e n c e on P r i m a r y H e a l t h C a r e , A l m a - A t a , USSR, (ICPHC/ALA/78.7), A p r i l 1, 1978. _^ . The t r a i n i n g and u t i l i z a t i o n of a u x i l i a r y p e r s o n n e l f o r r u r a l h e a l t h teams i n d e v e l o p i n g c o u n t r i e s . WHO T e c h n i c a l R e p o r t S e r i e s , no. 633, 1979. . Manual f o r the P l a n n i n g and E v a l u a t i o n of D i a r r h e a l D i s e a s e C o n t r o l Programmes (WHO/CDD/SER/81.5), 1981. 169 U n i t e d N a t i o n s , World H e a l t h O r g a n i z a t i o n and UNICEF, R e p o r t of  the I n t e r n a t i o n a l C o n f e r e n c e on P r i m a r y H e a l t h C a r e , A l m a - A t a , USSR, (ICPHC/ALA/78.10), September 6-12, 1978. E t t e n , G.M. R u r a l H e a l t h i n T a n z a n i a . ( A s s e n ( N e t h . ) : Van Gorcum & Co. , 1976 . Maanen, J . , ed. Q u a l i t a t i v e M e t h o d o l o g y . B e v e r l y H i l l s : Sage P u b l i c a t i o n s , 1983. Z i j l , W.J. " S t u d i e s on D i a r r h e a l D i s e a s e s i n Seven C o u n t r i e s . " B u l l e t i n o f the World H e a l t h O r g a n i z a t i o n 35 ( 1 9 6 6 ) : 249-261 . V a r g a s , R. " H e a l t h p o l i c i e s i n L a t i n A m e r i c a . " S o c i a l S c i e n c e &  M e d i c i n e 13C(2) ( 1 9 7 9 ) : 69 - 73. Vaughan, J . P . " B a r e f o o t o r p r o f e s s i o n a l ? : Community H e a l t h Workers i n t h e T h i r d W o r l d . " J o u r n a l of T r o p i c a l M e d i c i n e  and H y g i e n e 83 ( 1 9 8 0 ) : 3 - 1 0 . W a l s h , J . , and Warren, K. " S e l e c t i v e p r i m a r y h e a l t h c a r e : an i n t e r i m s t r a t e g y f o r d i s ease c o n t r o l i n d e v e l o p i n g c o u n t r i e s . " New E n g l a n d Jou r n a l o f M e d i c i n e 301(18) (November 1979): 967-974 . Webb, E . ; C a m p b e l l , D.; S c h w a r t z , R.; and S e c h r e s t , L., e d s . U n o b t r u s i v e M e a s u r e s : N o n r e a c t i v e R e s e a r c h i n the S o c i a l  Sc i e n c e s . C h i c a g o : Rand M c N a l l y , 1966. W e i s s , C.H. E v a l u a t i o n R e s e a r c h , Methods o f A s s e s s i n g Program  E f f e c t i v e n e s s . Methods o f S o c i a l S c i e n c e S e r i e s . Englewood C l i f f s , N . J . : P r e n t i c e H a l l I n c . , 1972. ._ E v a l u a t i o n A c t i o n Programmes. B o s t o n : A l l y n & Bacon, 1972 . ._ " A l t e r n a t i v e models o f program e v a l u a t i o n . " S o c i a l Work 19 (1974):675-682. Wholey, J . S . E v a l u a t i o n : P r o m i s e and P e r f o r m a n c e . W a s h i n g t o n , D.C.: The Ur b a n I n s t i t u t e , 1976. W i d s t r a n d , C. "The E v a l u a t i o n o f R u r a l D e velopment P r o j e c t s . " C u l t u r e s e t De v e l o p p e m e n t s 6 ( 1 ) ( 1 9 7 4 ) : 109-133. W i l e n s k i , P. The D e l i v e r y of H e a l t h S e r v i c e s i n the P e o p l e ' s  R e p u b l i c of C h i n a . O t t a w a : I n t e r n a t i o n a l D e velopment R e s e a r c h C e n t r e ( I D R C ) , 1976. Van Van Van 170 W i l l i a m s , W. " I m p l e m e n t a t i o n and A n a l y s i s and A s s e s s m e n t . " P o l i c y A n a l y s i s 1 (3) ( 1 9 7 5 ) : 531-566. W i l l i a m s o n , N.E. "Outcome I n d i c a t o r s Used to E v a l u a t e the B o h o l P r o j e c t . " I n N a t i o n a l C o u n c i l f o r I n t e r n a t i o n a l H e a l t h , I n t e r n a t i o n a l H e a l t h ; M e a s u r i n g P r o g r e s s - P r o c e e d i n g s 1980  I n t e r n a t i o n a l H e a l t h C o n f e r e n c e , pp. 30-33. W a s h i n g t o n , D.C.: N a t i o n a l C o u n c i l f o r I n t e r n a t i o n a l H e a l t h (NCIH), 1980. W o r l d Bank. H e a l t h P r o b l e m s and P o l i c i e s i n the D e v e l o p i n g C o u n t r i e s . S t a f f W o r k i n g P a p e r , no. 412. W a s h i n g t o n : World Bank, A u g u s t 1980. W o rld C o u n c i l o f C h u r c h e s , C h r i s t i a n M e d i c a l C o mmission. "The P r o m o t i o n o f N a t i o n a l H e a l t h S e r v i c e s . " C o n t a c t 25 ( F e b r u a r y 1975):2-7. " R u r a l B a s i c H e a l t h S e r v i c e s : The L a r d i n Gabas Way." C o n t a c t 41 ( O c t o b e r 1 9 7 7 ) : l - 7 W r i g h t , C.R. " E v a l u a t i o n R e s e a r c h . " I n I n t e r n a t i o n a 1 E n c y c l o p e d i a of the S o c i a l S c i e n c e s , pp. 19 7-202. U.S.A.: The M a c M i l l a n Co. & The F r e e P r e s s , 1968. Z a l d , M.N., ed. O r g a n i z i n g f o r Community W e l f a r e . C h i c a g o : Q u a d r a n g l e Books, 1967. Z e i g h a m i , B., and Z e i g h a m i , E. " E v a l u a t i o n o f I r a n i a n v i l l a g e w o r k e r s e f f i c a c y . " I n V i l l a g e H e a l t h W o r k e r s , pp. 14-20. E d i t e d by H.A. Ronaghy, Y. Mousseau-Gershman and A. D o r o z y n s k i . Ottawa: I n t e r n a t i o n a l D e velopment R e s e a r c h C e n t r e ( I D R C ) , 1976. Z e i g h a m i , B.; Z e i g h a m i , E.; and Ronaghy, H. " S t r e t c h i n g h e a l t h manpower: t h e r u r a l h e a l t h a u x i l i a r y . " C a n a d i a n J o u r n a l o f P u b l i c H e a l t h 68(5) (1977 ): 378-381. 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0096053/manifest

Comment

Related Items