Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Performance evaluation of a program in pharmacy continuing education Fielding, David Wilson 1977

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-UBC_1977_A2 F54.pdf [ 9.77MB ]
Metadata
JSON: 831-1.0055955.json
JSON-LD: 831-1.0055955-ld.json
RDF/XML (Pretty): 831-1.0055955-rdf.xml
RDF/JSON: 831-1.0055955-rdf.json
Turtle: 831-1.0055955-turtle.txt
N-Triples: 831-1.0055955-rdf-ntriples.txt
Original Record: 831-1.0055955-source.json
Full Text
831-1.0055955-fulltext.txt
Citation
831-1.0055955.ris

Full Text

PERFORMANCE EVALUATION OF A PROGRAM IN PHARMACY CONTINUING EDUCATION by DAVID WILSON FIELDING B.Sc. (Pharm.) Dalhousie U n i v e r s i t y , 1970 M.Sc, Dalhousie U n i v e r s i t y , 1973 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF EDUCATION i n THE FACULTY OF GRADUATE STUDIES (Dept. of Adult Education) We accept t h i s t h e s i s as conforming THE UNIVERSITY OF BRITISH COLUMBIA J u l y 1977 David Wilson F i e l d i n g , 1977 to the required standard In present ing th i s thes i s in p a r t i a l fu l f i lment of the requirements for an advanced degree at the Un iver s i t y of B r i t i s h Columbia, I agree that the L ib ra ry sha l l make it f r ee l y ava i l ab le for reference and study. I f u r ther agree that permission for extensive copying of th is thes i s for s cho la r l y purposes may be granted by the Head of my Department or by h i s representat ives . It is understood that copying or pub l i ca t i on of th i s thes i s fo r f i n anc i a l gain sha l l not be allowed without my wr i t ten permiss ion. n .' . r Adult Education Department or ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ The Un iver s i ty of B r i t i s h Columbia 2 0 7 5 W e s b r o o k P l a c e V a n c o u v e r , C a n a d a V 6 T 1 W 5 August 3 0 , 1977 i i ABSTRACT The focus of t h i s i n v e s t i g a t i o n was on the e v a l u a t i o n of a pharmacy c o n t i n u i n g education program and the development and assess-ment of w r i t t e n s imulations as an e v a l u a t i v e t o o l . A program designed as a re g u l a r course o f f e r i n g by the F a c u l t y and D i v i s i o n of Continuing Education i n the Health Sciences to improve the primary care c o n s u l t i n g s k i l l s of pharmacists, was evaluated i n terms of improvement i n the p a r t i c i p a n t ' s r e a l - l i f e performance, gain i n f a c t u a l knowledge and s u b j e c t i v e r e a c t i o n to the educational a c t i v i t y . In order to measure the q u a l i t y of primary care c o n s u l t a t i o n , four "in-store-assessment" problems were developed which, i n the opinion of a panel of pharmacy p r a c t i t i o n e r s , occur d a i l y i n community pharmacy p r a c t i c e . These problems involved a consumer requesting a s s i s t a n c e f o r e i t h e r a " c o l d " or a "pain" complaint. Each request was accompanied by a l i s t of p o s s i b l e pharmacist responses which the panel members rated as d e s i r a b l e or undesirable behaviours. Four subsets of these behaviours: "data g a t h e r i n g " , " i n a p p r o p r i a t e recommendations", "appropriate recommendations" and "drug-use-counselling" were s e l e c t e d as performance c r i t e r i a f o r each problem. The e f f e c t of the educational a c t i v i t y on the primary care con-s u l t i n g behaviour of program p a r t i c i p a n t s (N=34) a t t h e i r r e g u l a r place of p r a c t i c e , was examined by the unobtrusive a d m i n i s t r a t i o n of the problems. Assessments were conducted both before and a f t e r the program. A "non-equivalent" c o n t r o l group (N=39) was assessed during the same time periods. The problems were presented i n a standardized manner by s p e c i a l l y t r a i n e d observers who recorded the pharmacists' behaviours on i i i prepared performance c h e c k l i s t s . Performance scores on each of the behavioural subsets and a t o t a l score were c a l c u l a t e d to represent the q u a l i t y of advice given by the pharmacists. The impact of the program on pharmaceutical s e r v i c e s was determined by c a l c u l a t i n g gain scores. There was a s i g n i f i c a n t improvement i n the o v e r a l l performance score f o r the pharmacists who attended the educational program. There were s i g n i f i c a n t gains i n the performance of course p a r t i c i p a n t s f o r "data gathering" and "appropriate recommendations" behaviours. There were no s i g n i f i c a n t changes, i n any of the performance areas, f o r the "non-equivalent" c o n t r o l group. Tests f o r f a c t u a l knowledge were developed f o r the content areas of " c o l d " and "pain". These were administered to the course p a r t i c i p a n t s before and a f t e r i n s t r u c t i o n on these t o p i c s . There were s i g n i f i c a n t gains i n the r e c a l l of the information by the pharmacists. There were no s i g n i f i c a n t r e l a t i o n s h i p s between an i n d i v i d u a l ' s score on these t e s t s and performance scores on the requests f o r primary care. A magnitude estimation s c a l i n g technique was used by the p a r t i c i p a n t s to evaluate the program on several dimensions. In t h e i r opinion t h i s program was s u p e r i o r i n a l l aspects to the average con-t i n u i n g education course attended i n the past. Four w r i t t e n s i m u l a t i o n s were developed with the a i d of the panel of pharmacy p r a c t i t i o n e r s . Each was constructed with respect to a s p e c i f i c request f o r primary care a s s i s t a n c e . The content of the s i m u l a t i o n s approximated the content of the four problems used to assess the pharmacists' behaviours at t h e i r place of employment. The i v pharmacists e n r o l l e d i n the program completed the simu l a t i o n s before and a f t e r i n s t r u c t i o n on " c o l d " and "pain". The performance scores on the four s i m u l a t i o n s were compared with the performance scores on the corresponding problems used during the "in-store-assessment". A l l four s i m u l a t i o n s had a p o s i t i v e c o r r e l a t i o n w i t h the pharmacist's r e a l - l i f e performance. Two of the simu l a t i o n s had s i g n i f i c a n t c o r r e l a t i o n c o e f f i c i e n t s . Selected common behaviours were compared between the . w r i t t e n s i m u l a t i o n s and the "in-store-assessment" problems. A con-s i s t e n c y score was c a l c u l a t e d representing the agreement among performance on the items. For the pre-and post-measurements the r e s p e c t i v e mean consistency scores were 66.54 and 65.71 percent. There were no s i g n i f i c a n t r e l a t i o n s h i p s between scores on the t e s t s of f a c t u a l knowledge and performance scores on the w r i t t e n simula-t i o n s . The r e s u l t s suggest that t h i s c o n t i n u i n g education program was a success. There were improvements i n r e a l - l i f e performance, s i g n i f i c a n t gains i n f a c t u a l knowledge and f e e l i n g s of s a t i s f a c t i o n with the program on the part of the program p a r t i c i p a n t s . As w e l l , the f i n d i n g s i n t h i s study i n d i c a t e d that w r i t t e n simulations hold promise as an o b j e c t i v e e v a l u a t i o n t o o l f o r c o n t i n u i n g pharmacy education and were capable of p r e d i c t i n g r e a l - l i f e behaviour. TABLE OF CONTENTS ABSTRACT LIST OF TABLES LIST OF FIGURES ACKNOWLEDGEMENTS CHAPTER I. THE PURPOSE Int r o d u c t i o n Purposes Importance of the Study Overview of Report References - Chapter One I I . LITERATURE REVIEW The Pharmacist as a Primary Care Consultant Continuing Pharmacy Education Mandatory Continuing Education Research P a r t i c i p a t i o n A t t i tude Content Processes of Continuing Pharmacy Educati . E v a l u a t i o n A d u l t Education Pharmacy Continuing Education Simulations An Overview Writ t e n Simulations R e l i a b i l i t y and V a l i d i t y P r e d i c t i v e V a l i d i t y Summary References - Chapter Two I I I . INSTRUMENT DEVELOPMENT In-Store-Assessment Problems Development Panel of Content Experts Procedure f o r Rating the ISAPs v i CHAPTER Page Judges' Ratings of the ISAPs 42 C h e c k l i s t s 44 Dual Function of the ISAPs 44 Scoring System Assigned to the ISAPs 47 Observers 48 R e l i a b i l i t y of the Observers 49 Written Simulations 52 Development 52 Procedure f o r Rating the Simulations 53 V a l i d i t y of the Simulations 55 Tests f o r Factual Knowledge 62 P a r t i c i p a n t s ' Evaluation of the Program 62 References - Chapter Three 64 IV. METHODOLOGY 65 Design of the Study 65 Subjects 66 Demographic Data on Subjects 67 The Continuing Education Course 67 The Program Emphasis 69 Instruments ' 70 Instrument Assignment 70 Observers 71 Procedure f o r Data Gathering 71 Data A n a l y s i s 75 References - Chapter Four 77 V. RESULTS AND DISCUSSION 78 Demographic Data f o r Controls and Experimental Groups 78 Improvement i n R e a l - L i f e Performance 80 S e t t i n g and S i t u a t i o n a l Factors 93 P r e d i c t i v e V a l i d i t y of Written Simulations 97 Factual Knowledge Tests 102 P a r t i c i p a n t s ' E v a luation of the Program 105 References - Chapter Five 113 VI. SUMMARY AND CONCLUSIONS 114 Summary ] 1 4 Conclusions V I I . APPENDICES 124 Appendix A - M a t e r i a l s R e l a t i n g to Development and use of the In-Store-Assessment Problems (ISAPs) 125 v i i CHAPTER Page Appendix B - Panel of Content Experts 159 Appendix C - M a t e r i a l s R e l a t i n g to the Written Simulations 161 Appendix D - Pre- and Post-Tests f o r Factual Knowledge 198 Appendix E - Evaluation Form 209 v i i i LIST OF TABLES Table Page I Rating Categories A p p l i e d To Problem S o l v i n g A c t i v i t i e s To E s t a b l i s h Performance C r i t e r i a . 41 II Judges' Ratings Of The I n d i v i d u a l Behaviours Suggested For ISAP One. 130 I I I Judges' Ratings Of the I n d i v i d u a l Behaviours Suggested For ISAP Two. 132 IV Judges' Ratings Of The I n d i v i d u a l Behaviours Suggested For ISAP Three. 134 V Judges' Ratings Of The I n d i v i d u a l Behaviours Suggested For ISAP Four. 1 3 6 VI The D i s t r i b u t i o n Of The Four Subsets Of Behaviours In The In-Store-Assessment Problems. 43 VII Means And Standard Deviations (S.D.) Of Judges' Ratings Of The Behaviours In The ISAPs. 45 V I I I I n t e r c o r r e l a t i o n s Of Judges' Ratings Of The Behaviours In ISAPs One To Four. 1 3 8 IX I n t e r c o r r e l a t i o n s Of Judges' Ratings Of The Behaviours In A l l Four ISAPs Combined. 47 X Matching Of Written Simulations To The In-Store-Assessment 53 Problems (ISAPs). XI The Manner In Which Scoring Weights Were Assigned To The Options In The S i m u l a t i o n s . 54 XII The Means And Standard Deviations (S.D.) Of The Judges' Ratings Of The Options In The Various S i m u l a t i o n s . 56 XII I I n t e r c o r r e l a t i o n s Of Judges Ratings Of The Behaviours In Simulations One To Four. 58 XIV I n t e r c o r r e l a t i o n s Of the Ten Judges' Ratings Of The 267 Options In Simulations One To Four. 1 9 7 XV C r i t e r i o n Group V a l i d i t y Data For Pharmacy Writ t e n v. .'. . : 60 Si m u l a t i o n s . i x Table Page XVI t - P r o b a b i l i t i e s For S i g n i f i c a n t D i f f e r e n c e In Performance Of C r i t e r i o n Groups On Written Simulations. 61 XVII Accuracies Of Eight Observers For Each Of Four Videotapes Used In The T r a i n i n g Procedure. 50 XVIII Inter-Observer R e l i a b i l i t y On A l l 103 Items Of The Four Videotapes. 51 XIX O u t l i n e Of The Content Of Program By Session. 68 XX Demographic Comparisons For Control And Experimental Subjects. 79 XXI Comparisons Of The Component Scores For The Two Cold In-Store-Assessment Problems For The Experimental And Control Groups. 81 XXII Comparisons Of The Component Scores For The Two Pain In-Store-Assessment Problems For The Experimental And Control Groups. 83 XXIII Gain For The Post-In-Store-Assessment Scores For The Experimental and Control Groups. 85 XXIV C o r r e l a t i o n M a t r i x Representing R e l a t i o n s h i p s Between The Total Score, I t s Components, And The S i t u a t i o n a l Factors For The Pre-In-Store-Assessments For The Experimental Group. 155 XXV C o r r e l a t i o n M a t r i x Representing R e l a t i o n s h i p s Between The Total Score, I t s Components, And the S i t u a t i o n a l Factors For The Post-In-Store-Assessments For The Experimental Group. 156 XXVI C o r r e l a t i o n M a t r i x Representing R e l a t i o n s h i p s Between The Total Score, I t s Components, And The S i t u a t i o n a l Factors For The Pre-In-Store-Assessments For The Control Group. 157 XXVII C o r r e l a t i o n M a trix Representing R e l a t i o n s h i p s Between The Total Score, I t s Components, And The S i t u a t i o n a l Factors For The Post-In-Store-Assessments For The Control Group. 158 XXVIII C o r r e l a t i o n s Among Components And With Total Scores Of The In-Store-Assessments For The Experimental And Control Groups. 90 XXIX C o r r e l a t i o n Among The S i t u a t i o n a l Factors Present During The In-Store-Assessments And The Total Performance Score. 94 X Table Page XXX P r e d i c t i v e Powers Of The Four Simulations For Behaviours On The Four ISAPs. 99 XXXI Gains In Factual Knowledge As Represented By Scores On The Pre-And Post-Tests And The R e l a t i o n s h i p s Between These Scores And R e a l - L i f e Performance. 103 XXXII R e l a t i o n s h i p Between Scores On Tests For Factual Knowledge Reca l l And Performance On Written Simulations. 105 XXXIII Magnitude Estimations By Experimental Subjects Comparing Dimensions Of This Program With The Average Continuing Education Program In The Past. 107 XXXIV The I n t e r c o r r e l a t i o n s Of The P a r t i c i p a n t s ' Ratings of the Program Dimensions, Year Of Graduation And Attendance At Previous Courses. 213 XXXV Magnitude Estimations By Experimental Subjects Comparing The Amount Of Learning A t t r i b u t a b l e To Program A c t i v i t i e s With That A t t r i b u t a b l e To Standard 30 Minute Lectures. 110 x i LIST OF FIGURES Figure Page 1. Experimental Design 55 2. Schematic Representation Of Data C o l l e c t i o n 72 3. Simulation One - P o s s i b l e Paths To Problem S o l u t i o n 193 4. Simulation Two - P o s s i b l e Paths To Problem S o l u t i o n 194 5. Si m u l a t i o n Three - P o s s i b l e Paths To Problem S o l u t i o n 195 6. Simulation Four - P o s s i b l e Paths To Problem S o l u t i o n 196 7. Gains f o r the Experimental and Control Groups on the ISAPs 1 Components and Total Scores 86 x i i ACKNOWLEDGEMENTS The author would l i k e to express deepest thanks to the mem-bers of his d i s sertat ion committee: Dr. Coolie Verner, Dr. John Co l -l i n s , Dr. Raymond Jang, Dr. Gordon Page and Dr. Bernard Riedel for t he i r continuous guidance and assistance. The assistance of Dr. Roger Boshier for his contr ibution to the i n i t i a l development of the research project i s acknowledged. In add i t ion, I am grateful to Dr. Dale Rusnell and Mr. Nicholas Rubidge for the i r assistance with some of the computer programming and for the guidance of Mr. Adrian Blunt in the preparation of the magnitude estimation evaluation form. The author would l i k e to express sincere thanks to the panel-i s t s who ass isted in the development of the wr itten simulations and the in-store-assessment problems: Dr. Dennis Andrews, Ms. Loree Eldr idge, Mr. Peter Hutt, Mr. Ronald Ingraham, Mr. Munroe MacKenzie, Mr. Nicholas Otten, Ms. Karen Pylatuk, Mr. Ian Sands, Ms. Sharon Tudor and Ms. Louanne Twaites. The assistance of Mr. Alexander McGechaen, Dr. Raymond Jang, Ms. Jennifer Boshier, Ms. Louanne Twaites and Ms. Helen F ie ld ing in the preparation of the four videotapes i s g ra te fu l l y acknowledged. Deepest thanks to Ms. Joan B r i t t , Ms. Ruth Burstahler, Ms. Ann Scott, Mr. Wayne Soucy, Dr. Mi tchel l Samek, Ms. Gail Riddel 1, Ms. Karen Unrah and Ms. Anna Welbourne for ass i s t ing with the co l l ec t i on of the in-store-assessment data. The f inanc ia l support for th i s study v/as provided by the Mr. and Mrs. P. A. Woodward Foundation. In add i t ion, the author would l i k e to thank the W. K. Kellogg Foundation for i t s f inanc ia l support and i t s Univers ity of B r i t i s h Columbia project o f f i c e r , Mr. Robert Gobert, for his assistance during th i s research study and the author 's graduate program. The support of the College of Pharmacists of B r i t i s h Columbia i s g ra te fu l l y acknowledged. The assistance of Ms. Margaret McLeod, Ms. Dolores Vader in typing the drafts and Ms. J u l i e t Hearty for typing the f i na l manuscript i s pa r t i cu l a r l y appreciated. Deepest thanks to my wi fe, Helen, for her constant support, encouragement and assistance with th i s project. -1-CHAPTER ONE PURPOSE OF THE STUDY Int r o d u c t i o n As a l l things change with time, so too does the p r o f e s s i o n of pharmacy. At one time the apothecary was seen as the mixer of s e c r e t formulae c o n t a i n i n g e x o t i c i n g r e d i e n t s . Later i n h i s t o r y , the emphasis was on the pharmacist's s k i l l i n manufacturing p i l l s , t a b l e t s , syrups and t i n c t u r e s . In more recent times, the pharmacist's r o l e changed to being p r i n c i p a l l y a dispenser of medications, with a strong emphasis on drug-use-control. Today, as never before, the pharmacist i s regarded as an important source of information on p r e s c r i p t i o n and n o n - p r e s c r i p t i o n drugs. I t i s suggested t h a t the charge of the modern day pharmacist i s to be a c o n s u l t a n t to other members of the "health team" and to the general p u b l i c on the use of drugs. There i s i n c r e a s i n g evidence that the community pharmacist i s being asked to perform more f r e q u e n t l y as a primary care c o n s u l t a n t . In t h i s r o l e , the pharmacist i s the f i r s t to be approached about health problems. Studies have found however, t h a t many community pharmacists are not adequately performing t h e i r r o l e of a d v i s o r to the consumer on the use of n o n - p r e s c r i p t i o n drugs (1,2,5). These r e p o r t s a l s o suggest t h a t t h i s performance d e f i c i t i s due, i n p a r t , to a lack of knowledge. There appears, t h e r e f o r e , to be a need f o r improving the pharmacist's a b i l i t y to act as an a d v i s o r to the p a t i e n t . Once a bona f i d e need has been i d e n t i f i e d , the t r a d i t i o n a l method of c l o s i n g - 2 -the gap between performance and d e s i r e d performance i s to design an educational program. Inherent i n every educational program i s the r e s p o n s i b i l i t y to evaluate i t s outcomes. The a v a i l a b l e means f o r e v a l u a t i o n can be placed on a continuum, ranging from the s u b j e c t i v e measures a t one end to the o b j e c t i v e measure at the opposite end. The s u b j e c t i v e e v a l u a t i o n instruments are u s u a l l y e a s i e r to design and administer but the data they y i e l d are of questionable value. The o b j e c t i v e e v a l u a t i o n instruments r e q u i r e c o n s i d e r a b l y more time, money and energy to design and v a l i d a t e , but t h e i r r e s u l t s are su p e r i o r i n documenting l e a r n i n g . The e v a l u a t i o n procedure a t the extreme end of the o b j e c t i v e pole, unobtrusive observation of r e a l - l i f e performance, i s at present the only r e l i a b l e i n d i c a t i o n of how much l e a r n i n g has been t r a n s f e r r e d i n t o a c t i o n i n the r e a l world. T r a d i t i o n a l l y , c o n t i n u i n g education i n pharmacy has been evaluated with s u b j e c t i v e measures. U s u a l l y these take the form of the "happiness index". About the only form of o b j e c t i v e e v a l u a t i o n that has been occuring c o n s i s t e n t l y i s the use of pre-and p o s t - t e s t s f o r f a c t u a l knowledge r e c a l l . This appears to be the l i m i t of o b j e c t i v e e v a l u a t i o n f o r program e f f e c t i v e n e s s . For a number of reasons, the most important forms of e v a l u a t i o n , i n v o l v i n g the more d i r e c t measures of behaviour change and determining the impact of i n d i v i d u a l programs on the q u a l i t y of pharmaceutical s e r v i c e s , have been v i r t u a l l y ignored. To date there has been only one reported study which looked a t r e a l -l i f e , on-the-job performance of pharmacists a f t e r p a r t i c i p a t i o n i n a -3-t o n t i n u i n g education program ( 4 ) . R e a l i z i n g that unobtrusive, on-the-job e v a l u a t i o n i s expensive to design and operate and d i f f i c u l t to arrange, many of the other health professions are examining other o b j e c t i v e e v a l u a t i o n instruments which purport to approximate r e a l - l i f e behaviour. One such technique, which i s r e c e i v i n g considerable a t t e n t i o n as an e v a l u a t i o n t o o l , i s w r i t t e n s i m u l a t i o n . This technique c a l l s f o r an i n d i v i d u a l to apply knowledge gained i n an educational program i n the s o l u t i o n of a simulated problem. Written s i m u l a t i o n s have not been used i n pharmacy to t h i s p o i n t . However, t h e i r a p p l i c a t i o n to assess the performance of a pharmacist a d v i s i n g a consumer on the use of n o n - p r e s c r i p t i o n medications i n response to a primary care request appears appropriate. The obvious question i s , "Do w r i t t e n sumulation scores p r e d i c t r e a l - l i f e performances?" Studies have compared an i n d i v i d u a l ' s performance on a w r i t t e n s i m u l a t i o n to h i s or her assessment of a known simulated p a t i e n t . How-ever, there has been only one study comparing performance on a w r i t t e n s i m u l a t i o n to performance i n an actual p r a c t i c e s i t u a t i o n where the sub-j e c t was unaware that he or she was being assessed ( 3 ) . Purposes There are three general purposes f o r t h i s study. The f i r s t i s to develop a method to o b j e c t i v e l y evaluate a c o n t i n u i n g education program i n terms of i t s impact on the pharmacy s e r v i c e s subsequently provided by the course p a r t i c i p a n t s . To achieve t h i s purpose, the f o l l o w i n g three o b j e c t i v e s were s e t : -4-1. To develop four r e a l i s t i c primary care requests, i n the area of n o n - p r e s c r i p t i o n medications, to be used as in-store-assessment problems (ISAPs). 2. To devise and v a l i d a t e a comprehensive l i s t of observable pharmacist behaviours, i n response to the requests, which w i l l enable an e v a l u a t i o n of pharmacists' primary care c o n s u l t i n g s k i l l s . 3. To assess the improvement i n the q u a l i t y of primary care c o n s u l t i n g by those pharmacists who p a r t i c i p a t e d i n the c o n t i n u i n g education program. The second purpose i s to develop methodology to evaluate and subsequently to evaluate the c o n t i n u i n g education program i n terms of more t r a d i t i o n a l e v a l u a t i o n data and to i n v e s t i g a t e r e l a t i o n s h i p s between these data and the course's impact on the pharmacy s e r v i c e s provided by course p a r t i c i p a n t s . To achieve t h i s purpose, three a d d i t i o n a l o b j e c t i v e s were s e t : 4. To measure the gain i n f a c t u a l knowledge as a r e s u l t of p a r t i c i p a t i o n i n the program. 5. To determine the r e l a t i o n s h i p s between scores on a f a c t u a l knowledge t e s t and r e a l - l i f e performance. 6. To o b t a i n the p a r t i c i p a n t ' s s u b j e c t i v e e v a l u a t i o n of the program. The t h i r d purpose i s to study the v a l i d i t y of w r i t t e n s i m u l a t i o n s as a technique f o r e v a l u a t i n g the e f f e c t i v e n e s s of c o n t i n u i n g pharmacy education programs. To achieve t h i s purpose, two o b j e c t i v e s were s e t : 7. To develop and v a l i d a t e four w r i t t e n s i m u l a t i o n s which were s u i t a b l e f o r e v a l u a t i n g a pharmacist's performance i n the area of primary care c o n s u l t i n g . 8. To assess the p r e d i c t i v e powers of the w r i t t e n s i m u l a t i o n s by comparing the pharmacist's r e a l - l i f e performance with t h a t of h i s performance on the s i m u l a t i o n s . - 5 -Importance of the Study Pharmacists are i n c r e a s i n g l y being asked to serve as an information source to the consumer on the use of n o n - p r e s c r i p t i o n drugs. Studies i n d i c a t e , however, that the l e v e l of performance i n t h i s area i s l e s s than d e s i r a b l e . Continuing education programs have been suggested as one means of improving the performance i n t h i s important area of pharmaceutical s e r v i c e s . T r a d i t i o n a l l y , such c o n t i n u i n g education programs i n pharmacy would be evaluated by measuring gains i n f a c t u a l knowledge and/or the s a t i s f a c t i o n of the course p a r t i c i p a n t s . Rarely has a pharmacy program been evaluated i n terms of the behavioural changes of p a r t i c i p a n t s . Moreover, l i t t l e research has been conducted on adapting or developing new e v a l u a t i o n techniques. This study of a c o n t i n u i n g education course i s s i g n i f i c a n t because i t i s one of the f i r s t e v a l u a t i o n s i n the health sciences to u n o b t r u s i v e l y evaluate change i n the ac t u a l p r a c t i c e performance of the p a r t i c i p a n t s . I t i s a l s o the f i r s t time that w r i t t e n s i m u l a t i o n s have been developed f o r pharmacy p r a c t i c e . I t i s one of only a few s t u d i e s which attempts to assess the p r e d i c t i v e v a l i d i t y of w r i t t e n s i m u l a t i o n s by comparing an i n d i v i d u a l ' s performance on a w r i t t e n s i m u l a t i o n to h i s or her r e a l - l i f e performance. Overview of t h i s Report The remaining chapters of t h i s document examine the r e l e v a n t l i t e r a t u r e ; discuss the development of the instruments used i n t h i s i n v e s t i g a t i o n , and present r e s u l t s documenting t h e i r r e l i a b i l i t y and - 6 -v a l i d i t y ; d e scribe the experimental design employed f o r e v a l u a t i n g the educational program and assessing the p r e d i c t i v e v a l i d i t y of the w r i t t e n s i m u l a t i o n s ; present and discuss the r e s u l t s of the e v a l u a t i o n of the program and the assessment of p r e d i c t i v e powers of the s i m u l a t i o n s . The f i n a l chapter summarizes the study and i t s c o n c l u s i o n s . -7-References Chapter One 1. Jang, R., "Evaluation of the Q u a l i t y of Drug-Related Services Provided by Community Pharmacists i n a M e t r o p o l i t a n Area." Ph.D. D i s s e r t a t i o n , Ohio State U n i v e r s i t y , 1971. 2. Knapp, D.A.; H.H. Wolf; D.E. Knapp; and T.A. Rudy, "The Pharmacist as a Drug Advisor." Journal of American Pharma- c e u t i c a l A s s o c i a t i o n NS9 (October, 1969): 502. 3. P a l v a , I.P. and V. Korhonen, " V a l i d i t y and Use of Written Simulation Tests of C l i n i c a l Performance." Journal of  Medical Education 51 (August, 1976): 657-661. 4. Watkins, R.L.; J.G. Norwood and F.L. M e i s t e r , "Improving the Q u a l i t y of the Pharmacist as a Drug Advisor to P a t i e n t s and P h y s i c i a n s Through Continuing Education." American Journal  of Pharmaceutical Education 40 (February, 1976): 34-39. 5. Wertheimer, A.I.; E. Shefter and R.M. Cooper, "More on the Pharmacist As ; a Drug Consultant. Three Case Stu d i e s . " Drug I n t e l l i g e n c e and C l i n i c a l Pharmacy 7 (February, 1973): 58-61. -8-CHAPTER TWO LITERATURE REVIEW This chapter w i l l present a review of the l i t e r a tu re pertaining to the pharmacist as a primary care consultant, continuing pharmacy educa-t i o n , evaluation and simulations, as these topics re late to th i s study. The Pharmacist as a Primary Care Consultant Community pharmacists are being asked to assume a role as a primary care consultant (10,11,18,28). There are a number of factors which have lead to th i s recent challenge. An information source i s needed between the consumer and the drug product - pa r t i cu l a r l y in re la t ion to the use of non-prescription medications. I t i s generally reported that adverse reactions to drugs (both prescr ipt ion and non-prescr ipt ion) account for three to f i ve percent of the hospital ad-missions in the United States (18). In the United Kingdom i t has been estimated that less than one-third of a l l symptoms of i l l health are referred to a physician (72). Because self-medication i s a part of current behaviour, non-prescription drug products account for one-th i rd of the tota l expenditures on medicaments and are used twice as frequently as prescr ipt ion drugs (72). In countries o f fer ing state paid medical benef i t s , physicians are often requested to treat con-d i t ions which are of short duration and s e l f - l i m i t i n g in nature (72). What i s needed in many such cases i s a non-prescription medication which w i l l give symptomatic r e l i e f . The pharmacist should be able to -9-provide the necessary advice in the se lect ion of such products. The profession of pharmacy i s advocating that i t s members accept re spons ib i l i t y for advising the consumer on the use of non-prescr ipt ion medications (17,22,33,61). In the United States, The American Pharmaceutical Association has been conducting national campaigns of publ ic education emphasizing the pharmacist as a source of information on non-prescription drugs. The theme of the 1975 campaign was "Over the Counter Inte l l igence" (59). The Assoc iat ion ' s theme for 1976 was expanded to include a l l medicinals and used the slogan "Be Wise With Medicines, Ask Your Pharmacist How" (11). Although advising the consumer on the use of non-prescription medication i s not a new ro l e , recognition as a consultant on primary care problems i s recent. This does not suggest that the advice of a pharmacist take the place of the treatment by a physician but there are some minor problems that can be dealt with e f f i c i e n t l y and e f f e c t -i ve ly by a pharmacist. However, i t i s c ruc ia l that the pharmacist have the a b i l i t y to d i f f e ren t i a te between those problems requir ing physician attention and those s e l f - l i m i t i n g problems in which a non-prescr ipt ion product to re l ieve the symptoms may be warranted. The role of the pharmacist as a primary care consultant i s "part of the tota l health care system" (72, p.173, see also Bass (10), L i s t e r (47)). Studies conducted to evaluate the qua l i ty of advice given by pharmacists in response to primary care requests indicate a low leve l of performance (40,42,73). In a surpr is ing number of cases, products -10-were recommended without the pharmacist asking any questions and th i s often resulted in the consumer purchasing a product which could have potent ia l l y harmful e f fec t s . Investigators were shocked at the casual manner in which requests were treated and questioned whether some re-commendations were not so le ly p r o f i t motivated. In a survey in the United Kingdom, 183 pharmacists were asked to give advice on a series of mild symptoms of short duration, including headache, sore throat, constipation and indigest ion. The investigators had some 70 d i f fe rent products recommended for the four complaints. An advisory panel f e l t that most were reasonable but did comment that, in t he i r opinion, about one in ten recommendations were i ne f fec t i ve . The responses of the pharmacists to the request ranged from no questions to many questions and from "here take t h i s " to "see a physician" (29,74). It has been suggested that pharmacy i s a marginal profession with professional and business goals often in c o n f l i c t (36,50). I f pharmacists could be categorized as business oriented and profess ional ly or iented, then perhaps there would be an explanation for the poor per-formance of some pharmacists as advisors to pat ients. However, studies were unable to substantiate th i s hypothesis (45,46). There has been l i t t l e research determining the extent to which pharmacists act as primary care consultants. One study in London, Ontario asked f i ve pharmacies to keep a record of primary care requests over a two day period (10). I t was found that three of the pharmacies provided 20 or more primary care services per day. S ixty percent of the requests could be c l a s s i f i e d upper-respiratory t rac t problems, -11-stomach and bowel problems, pain and vitamin enquir ies. In response to the request the pharmacists could (a) recommend a non-prescription pro-duct, (b) refer the indiv idual to a physic ian, or (c) give other advice, reassurance, or information which did not involve recommending a non-prescr ipt ion product. The advice given by pharmacists surveyed was as fo l lows: 80 percent recommended a product, 12 percent suggested seeing a physic ian, and 8 percent gave other advice. When the pharmacist responds to a primary care request the fol lowing approach has been suggested: take a b r i e f h i s tory, make an i n i t i a l assessment and suggest a course of action (10). There i s a role for the pharmacist in handling common complaints such as stomach and bowel problems, pain and upper-respiratory t rac t problems but there are doubts that the pharmacist i s trained s u f f i c i e n t l y to per-form th i s function. Pharmacy facu l t i e s have started jus t recently to include courses on applied therapeutics and non-prescription medications in t he i r programs. There are, therefore, implications for continuing education. Continuing Pharmacy Education Like a l l other professions, Pharmacy i s experiencing a knowledge explosion and increased demands on i t s members. The pharmacist i s being asked to provide better services and to assume new roles. There i s an awareness oh the part of the members of the pharmacy profession that continued learning i s a necessity. In one bibliography on continuing education for the years 1960-1975 there -12-were 82 c i ta t ions expounding the need for continuing education (26). The profession has responded. There are numerous continuing education programs yearly in every j u r i s d i c t i o n in North America. These are offered by facu l t ie s of pharmacy, e i ther alone, or in conjunction with some professional body. Mandatory Continuing Education The predominant theme of continuing pharmacy education of the l a te 60's and early 70's was whether or not i t should be mandatory in order to reta in l icensure status. Ten states (3) in the United States l eg i s l a ted mandatory continuing education during th i s time with l i t t l e regard for the standards of programs being offered, the legal implications of making i t mandatory, the a b i l i t y to supply programs in the mandatory states and the administrative headaches that i t would create. During 1972-1974 the l i t e r a t u r e re f lec t s some soul searching on the part of pharmacy educators concerning the appropriateness of mandatory continuing education. They f e l t that i f the goal i s to assure the continued competency of the members of the profession then perhaps mandatory par t i c ipat ion in education i s not the best means to that end. The professional bodies, as well as the educators, raised questions and th i s had a d i v i s i ve e f fec t . This i s demonstrated in the fol lowing ob-servation: [there appears] to be a s p l i t developing in the profession of pharmacy. NABP has no intent ion of backing down from i t s insistence that continuing education for pharmacy be mandatory, and that there be no "moratorium" on the enact-ment of state laws requir ing evidence of continuing educa--13-t i o n as a c o n d i t i o n of r e l i c e n s u r e . A moratorium has been c a l l e d f o r by the APhA and AACP, which c l a i m there i s as yet no evidence that mandatory, rat h e r than voluntary c o n t i n u i n g education i s needed to assure the continu i n g p r o f e s s i o n a l competence of phar-macists (4, p.26). In 1973, a task f o r c e was formed by the APhA and AACP to i n v e s t i g a t e and report on continu i n g competence. The p r e l i m i n a r y report of the committee was d e l i v e r e d i n 1974 and the f i n a l report i n 1975 (1,2). The f o l l o w i n g i s an important excerpt from the report which was c a r e f u l not to endorse mandatory contin u i n g education. Each i n d i v i d u a l p r a c t i t i o n e r could be ob l i g e d to comply with standards of competence, developed n a t i o n a l l y by the p r o f e s s i o n , and accepted, supported and enforced by each s t a t e through requirements f o r r e l i c e n s u r e (2, p. 433). The task f o r c e f u r t h e r proposed ...the c r e a t i o n of a Pharmacy P r a c t i c e Standards Com-mission whose primary r e s p o n s i b i l i t y would be to i d e n t i f y and recommend n a t i o n a l standards of compe-tence t h a t i n d i v i d u a l pharmacists would be required to meet. This commission should a l s o devise and i d e n t i f y the means by which conti n u i n g competence i n pharmacy may be determined (2, p.435). In the r e p o r t , there i s the underlying b e l i e f t h a t , u n t i l such time as the standards of competency are s e t , and the means to assess each pharmacist are devised, p a r t i c i p a t i o n i n continu i n g ed-ucation should be encouraged r a t h e r than required. In Canada, the approach to mandatory contin u i n g education has followed t h a t of the United States. Some provinces, A l b e r t a and Saskatchewan, have l e g i s l a t e d mandatory p a r t i c i p a t i o n i n continu i n g education. Some provinces, Quebec and Ontari o , provide c o n t i n u i n g education m a t e r i a l s which go to every pharmacist i n the province, but -14-do not r e q u i r e proof of completion. In B r i t i s h Columbia, the pr o f e s s i o n a l a s s o c i a t i o n i s c u r r e n t l y t r y i n g to i d e n t i f y the competencies a pharmacist should possess, i n order to a s s i s t i n determining c o n t i n u i n g education needs. C l e a r l y , l e g i s l a t i n g mandatory c o n t i n u i n g education i s pl a c i n g "the c a r t before the horse". There are some fundamental questions about the e f f e c t of contin u i n g education programs on the q u a l i t y of pharmaceutical s e r v i c e s which should f i r s t be answered. What i s needed are concentrated research e f f o r t s to develop b e t t e r e v a l u a t i v e instruments and apply them i n assessing program impact. Research In a review of the l i t e r a t u r e f o r the period 1960-1970 Nakamoto and Verner made the f o l l o w i n g comments: . . . The U n i v e r s i t y schools of Pharmacy have done v i r t u a l l y no research to provide a basis f o r programs and such t h a t has been done suggests t h a t c o n t i n u i n g education has made but a minimal impact upon the p r a c t i c e of pharmacy because the knowledge e s s e n t i a l to f u n c t i o n a l program planning has not been acquired through research. As the r o l e of the pharmacist changes, so must the form, content and dur a t i o n of education i n pharmacy. In the l i t e r a t u r e reviewed there was no evidence to i n d i c a t e an awareness of t h i s i n the schools. Of the several health p r o f e s s i o n s , pharmacy i s the most backward with respect to co n t i n u i n g education. The p r o f e s s i o n must be aroused to the need f o r i t to ensure the s u r v i v a l of pharmacy as a p r o f e s s i o n (56, p.p. 33-34). In t h e i r review of the l i t e r a t u r e f o r that ten year p e r i o d , Nakamoto and Verner claimed to have found "only 33 references r e l a t e d to c o n t i n u i n g education of which only seven were of a research nature" (56, p. 7). -15-The vast m a j o r i t y of research i n co n t i n u i n g pharmacy education has been of the d e s c r i p t i v e , e x h o r t a t i v e v a r i e t y . The predominant method of i n v e s t i g a t i o n has been the survey. There has been very l i t t l e experimental research i n which v a r i a b l e s are manipulated or hypotheses are teste d (56). For example, the research which i s a v a i l a b l e describes the reasons f o r p a r t i c i p a t i n g (or reasons f o r not p a r t i c i p a t i n g ) , the content and the d i f f e r e n c e processes used by the i n s t r u c t o r s and program planners. P a r t i c i p a t i o n i n Continuing Pharmacy Education A 1963 survey conducted i n M i s s i s s i p p i , i n which 900 question-naires were mailed out and from which there were 111 usable r e t u r n s , i n d i c a t e d t h a t 103 pharmacists expressed an i n t e r e s t or d e s i r e to p a r t i c i p a t e (35). Of these 103 pharmacists only 13.6 percent had a c t u a l l y p a r t i c i p a t e d i n previous programs. In a s i m i l a r survey conducted w i t h a random sample of 300 Indiana pharmacists, i t was found t h a t the c o n t i n u i n g education programs were reaching only a small proportion of the s t a t e ' s pharmacists (62). As w e l l , i t appeared i n t h i s study that the same pharmacists were being reached over and over again. Jobe (41) conducted a survey of a l l the c o l l e g e s of pharmacy holding membership i n AACP i n the United S t a t e s . A p a r t i c i p a t i o n index was used to measure the success i n promoting the programs. The p a r t i c i p a t i o n index was simply the number of p a r t i c i p a n t s d i v i d e d by the number promoted and expressed as a percentage. The f i n d i n g s i n d i c a t e d that when programs were sponsored j o i n t l y by the c o l l e g e of -16-pharmacy and the p r o f e s s i o n a l a s s o c i a t i o n s , the p a r t i c i p a t i o n i n d i c e s increased. As part of t h e i r annual l i c e n s e renewal i n 1973, a l l phar-macists i n Wisconsin were required to complete a questionnaire on par-t i c i p a t i o n i n c o n t i n u i n g education f o r the years 1969-1973. The r e s u l t s of the survey and o f f i c i a l r e g i s t r a t i o n data f o r that time period were computer tabulated and subjected to Chi square a n a l y s i s and reported by Arndt, Demuth and Weinswig (5). I t was found that 43.8 percent of 1,208 pharmacists p a r t i c i p a t e d i n at l e a s t one program f o r the period 1969-1973. The most frequent attendants a t the programs were age 30-49, were a c t i v e members of t h e i r p r o f e s s i o n a l a s s o c i a t i o n , were f u l l - t i m e pharmacists and were more l i k e l y to be h o s p i t a l pharmacists. Bernardi (13) conducted a survey of Connecticut pharmacists. His sample c o n s i s t e d of 168 pharmacists, 108 chosen at random from a s t a t e membership l i s t and 62 known p a r t i c i p a n t s i n conti n u i n g education. These pharmacists were c l a s s i f i e d as p a r t i c i p a n t s or n o n - p a r t i c i p a n t s depending on t h e i r enrollment i n conti n u i n g education programs over the l a s t three year p e r i o d . He found that e i g h t out of ten pharmacists spent four hours per week reading j o u r n a l s or were engaged i n some other form of c o n t i n u i n g education. Bernardi i n d i c a t e d t h a t three times as many p a r t i c i p a n t s as n o n - p a r t i c i p a n t s were i n favour of mandatory c o n t i n u i n g education. Another aspect of the study published elsewhere (14), measured pharmacists' a t t i t u d e s toward the concept of conti n u i n g education. He found that those who p a r t i c i p a t e d i n conti n u i n g education -17-had a more favourable a t t i t u d e toward c o n t i n u i n g education. A t t i t u d e Rouege, K i r k and Weinswig (63) conducted a study to measure the educational a t t i t u d e of p a r t i c i p a n t s and n o n - p a r t i c i p a n t s . They determined the d i f f e r e n c e i n a t t i t u d e was r e l a t e d to the degree of p a r t i c i p a t i o n i n a d u l t education a c t i v i t i e s . A s i g n i f i c a n t d i f f e r e n c e (p<.01) was found between the composite a t t i t u d e score when comparing p a r t i c i p a n t s and n o n - p a r t i c i p a n t s . A s i g n i f i c a n t c o r r e l a t i o n (p<.01) was found between a t t i t u d e toward c o n t i n u i n g education and p a r t i c i p a t i o n i n an organized l e a r n i n g a c t i v i t y . Kotzan and Jowdry (43) developed a questionnaire to measure the a t t i t u d e s of 95 Georgia pharmacists on the l a s t evening of a c o n t i n u i n g education course. The a t t i t u d e s measured were: general a t t i t u d e toward an educational experience, relevancy of program m a t e r i a l , a b i l i t y of program l e c t u r e r s and the management of the program. The only s i g -n i f i c a n t f i n d i n g was that i n s t i t u t i o n a l pharmacists - mostly h o s p i t a l pharmacists - had a more general negative f e e l i n g toward t h i s a d u l t education program. The p o s s i b l e explanation f o r t h i s f i n d i n g was that because the h o s p i t a l pharmacists work i n a t e a c h i n g - l e a r n i n g environment, they have a g r e a t e r access to j o u r n a l s and t e x t s and a l s o on the average attend more a d u l t education programs than most r e t a i l pharmacists, and may, t h e r e f o r e , f i n d the m a t e r i a l redundant (see a l s o Carl i n , ( 2 0 ) ) . Content In the surveys conducted to determine which t o p i c s were most pr e f e r r e d by pharmacists, pharmacology and/or advances i n pharmacy -18-u s u a l l y head the l i s t (24,35,39,62). Business management courses always appear high on the l i s t (35,62). This i s probably due to pharmacy being unique among most health professions i n that the (36) predominant branch of pharmacy has both a p r o f e s s i o n a l s e r v i c e com-ponent as w e l l as a business component. I n t e r p r o f e s s i o n a l r e l a t i o n -ships and new l e g i s l a t i o n are other t o p i c s requested (62,75). Processes of Continuing Pharmacy Education The concern over sparse attendance was evident e a r l y i n the h i s t o r y of c o n t i n u i n g education programs i n pharmacy. Strommen voiced the n e c e s s i t y f o r promotion, salesmanship and even showmanship to "coerce" pharmacists i n t o attending the programs (66). Along with the c r i e s f o r increased promotion, came the pleas f o r new and i n n o v a t i v e methods and techniques to d e l i v e r the message. Techniques were needed which would enable the pharmacist to study at home or at l e a s t not have to d r i v e f o r two hours i n order to attend. Most of the new ways of d e l i v e r i n g the continuing education s t i l l used some v a r i a t i o n of the l e c t u r e as the technique. This may be explained by the lack of f a m i l i a r i t y of the i n s t r u c t o r s and pharmacists with other techniques. A l s o , pharmacy conti n u i n g education programs have been used c h i e f l y to disseminate i n f o r m a t i o n . In the 1963 M i s s i s -s i p p i survey (35), 64.1 percent of the pharmacists responding p r e f e r r e d t h i s method of p r e s e n t a t i o n . One of the f i r s t attempts to use t e l e v i s i o n was by the P h i l a d e l p h i a College of Pharmacy and Science when they programmed 30 minutes of educational a c t i v i t y on a l o c a l commercial s t a t i o n (44). In -19-Minnesota, c l o s e d c i r c u i t t e l e v i s i o n was chosen i n preference to using the p u b l i c channels. This attempt to increase p a r t i c i p a t i o n i n v o l v e d a 60 minute presentation using videotapes and a telephone hookup to answer questions on l i v e camera (38). Hodapp and Kanun (37) compared student l e a r n i n g using pro-grammed i n s t r u c t i o n and closed c i r c u i t t e l e v i s i o n at the U n i v e r s i t y of Minnesota. They mailed a l e t t e r to 2,300 pharmacists asking t h e i r co-operation i n the study; 958 agreed. From t h i s l i s t a sample was randomly s e l e c t e d to p a r t i c i p a t e i n the programmed i n s t r u c t i o n . A p r e - t e s t was then mailed. A follow-up t e s t was administered three months l a t e r . The t e l e v i s i o n group completed a p r e - t e s t , a p o s t - t e s t and a follow-up t e s t . As w e l l , the p o s t - t e s t was mailed to a group who d i d not take part i n e i t h e r educational experience. When the r e s u l t s were compared both ex-perimental forms seemed to produce an equal gain i n knowledge compared to the c o n t r o l group. The a p p l i c a t i o n of the tele-conference method to continu i n g pharmacy education has been described by B l o c k s t e i n and Durant (16). A pre-taped l e c t u r e was played over a telephone system and c a r r i e d to various centres where i t was broadcast by a loudspeaker. A f t e r the l e c t u r e , a panel would d i s c u s s , l i v e , the important points and answer questions from the p a r t i c i p a n t s along the system. In pharmacy continuing education the correspondence courses from the St. Louis College of Pharmacy have gained widespread fame. When s t a r t e d i n 1964, there were 50 p a r t i c i p a t i n g pharmacists from the St. Louis area. In 1969 the course had 6,000 students from 14 co u n t r i e s around the world (9). In an e v a l u a t i o n of the program, Barnes (8) re--20-ceived an 85 percent return on 500 questionnaires mailed to p a r t i c i p a n t s . In a f u r t h e r e v a l u a t i o n (9).the correspondence courses were rated b e t t e r than other forms of contin u i n g education by 87 percent of those who com-ple t e d the course and 47 percent of those who d i d not complete the course. Audio c a s s e t t e tapes have become a popular means of d e l i v e r -ing c o n t i n u i n g education i n pharmacy. The U n i v e r s i t y of Wisconsin has perhaps become the most famous i n s t i t u t i o n i nvolved i n the preparation of such programs. Most c a s s e t t e tapes are approximately one hour long, deal with a s i n g l e concept and are accompanied by an o u t l i n e of the l e c t u r e . The package u s u a l l y includes a short quiz f o r a s e l t - t e s t (6, 34). An e v a l u a t i o n of an audio c a s s e t t e tape l e c t u r e course at the U n i v e r s i t y of Wisconsin was conducted by Blank, K i r k and Weinswig (15). The e v a l u a t i o n included pharmacists from F l o r i d a , where conti n u i n g educa-t i o n i s mandatory, and pharmacists from Michigan, where conti n u i n g educa-t i o n i s vol u n t a r y . The data were based on 69 i n d i v i d u a l s from Michigan and 67 from F l o r i d a . The program c o n s i s t e d of e i g h t tapes, each one hour i n l e n g t h , which were mailed to the pharmacists a f t e r a p r e - t e s t was administered. Two months l a t e r a p o s t - t e s t plus an e v a l u a t i o n form was mailed to the pharmacists i n the two s t a t e s . The i n v e s t i g a t o r s found no s i g n i f i c a n t d i f f e r e n c e between the pharmacists' scores on the p o s t - t e s t or p r e - t e s t when comparing the two s t a t e s . Thus, the pharmacists i n both s t a t e s were at the same l e v e l both before and a f t e r the course with respect to the program content. There was a s i g n i f i c a n t gain i n post-t e s t over p r e - t e s t f o r both groups. Hence, p r a c t i s i n g i n a s t a t e where -21-continuing education i s compulsory d i d not mean, i n t h i s i n s t a n c e , t h a t the pharmacists w i l l perform b e t t e r . Pharmacists w i t h l e s s than ten years' experience and h o s p i t a l pharmacists scored s i g n i f i c a n t l y higher on the p r e - t e s t . Pharmacists who had been p r a c t i s i n g l e s s than ten years scored s i g n i f i c a n t l y higher on the p o s t - t e s t . In an e v a l u a t i o n of the program, the l e a s t l i k e d aspects were.the i n a b i l i t y to ask questions and the d i f f i c u l t y i n f i n d i n g a s p e c i f i c spot on the tapes when searching f o r a p a r t i c u l a r p o i n t . Audiotapes, t h e r e f o r e , appear to be an e f f e c t i v e means of d e l i v e r i n g c o n t i n u i n g education f o r c e r t a i n groups of pharmacists (see a l s o DeMuth, K i r k and Weinswig ( 2 7 ) ) . E v a luation A d u l t Education Evaluation i s an e s s e n t i a l part of a d u l t education. Unless an ev a l u a t i o n i s conducted i n the context of the program o b j e c t i v e s , the program i s incomplete. U n t i l r e c e n t l y , a d u l t education was thought to be i n h e r e n t l y good and, t h e r e f o r e , i t was unnecessary to evaluate i t s e f f e c t s (19). I t has been only i n the l a s t quarter century that a d u l t educators have been asking questions about the attainment of t h e i r goals. They were no longer prepared to accept E s s e r t ' s a s s e r t i o n t h a t , "to a major extent a d u l t education stands on i t s own merits . . . I t must add s i g n i f i c a n c e to the l i f e of an a d u l t momentarily or permanently or i t does not continue." (30, p. 161) The attendance record was no longer the sol e means f o r e v a l u a t i o n . The U.S. Adult Education A s s o c i a t i o n ' s Committee on Evaluation suggested that e v a l u a t i o n of a d u l t education programs should be grounded i n the f o l l o w i n g conceptual -22-framework: The purposes of education are growth and change - change i n behaviour of i n d i v i d u a l s and groups. People behave d i f f e r e n t l y as the r e s u l t of education. The primary purpose of e v a l u a t i o n i n education i s to f i n d out how much growth and change have taken place as the r e s u l t of educational experiences. One evaluates a t o t a l program or major parts of i t to f i n d out how much progress has been made towards program o b j e c t i v e s (23, p. 7 ) . Adult educators (19,67) i n c r e a s i n g l y b e l i e v e t h a t the under-l y i n g assumption about e v a l u a t i o n i s th a t i t must measure a change i n behaviour. Thiede has stat e d the f o l l o w i n g reasons f o r e v a l u a t i o n : "1.) guiding growth and development; 2.) improving programs; 3.) defending programs; 4.) f a c i l i t a t i n g and encouraging s t a f f growth and p s y c h o l o g i c a l s e c u r i t y " (67, p. 192). Numerous authors (19,31,67, 69,70) agree t h a t e v a l u a t i o n s t a r t s w i t h some c l e a r cut statements of what i t i s hoped the program w i l l accomplish. These should be stat e d i n s p e c i f i c terms and a r t i c u l a t e d i n the context of measurable behaviours (70). Next i t i s necessary to s t a t e what i s acceptable evidence of the accomplishment of these goals or o b j e c t i v e s . Then f o l l o w s a d e c i s i o n regarding the procedure f o r c o l l e c t i n g the evidence. In the f i n a l step, the evidence i s c o l l e c t e d , summarized and used to make some judgements about the programs i n terms of i t s o r i g i n a l o b j e c t i v e s . Verner has stat e d (69,70) that there i s a need to apply the rigorous procedures of s o c i a l science research to e v a l u a t i o n i n a d u l t education. This should be done to develop and t e s t new instruments which would have wide usage i n the f i e l d . He emphasizes the need f o r accurate, well-reasoned e v a l u a t i o n t o o l s , f o r " i t i s b e t t e r not to -23-evaluate a t a l l than to do unwisely or i n e p t l y " (70, p. 95). The need f o r research i n the development of e v a l u a t i o n t o o l s i s a l s o expressed by Thiede (67). He b e l i e v e s that s e l f - e v a l u a t i o n t o o l s need development and experimental work to al l o w l a t e r i n c o r p o r a t i o n i n t o a d u l t education i n ways t h a t f a c i l i t a t e the l e a r n i n g process. Frederiksen (31) has discussed seven mechanisms f o r e v a l u a t i n g the outcomes of i n s t r u c t i o n . They are: s o l i c i t i n g o p i n i o n s , a d m i n i s t e r i n g a t t i t u d e s c a l e s , measuring knowledge, e l i c i t i n g r e l a t e d behaviour, e l i c i t i n g "what I would do" behaviour, e l i c i t i n g l i f e - l i k e behaviour and observing r e a l - l i f e behaviour. Each has advantages and disadvantages as well as appropriate uses. S o l i c i t i n g opinions about the educational a c t i v i t y w i l l not give an i n d i c a t i o n of a behavioural change, un l e s s , of course, one was t r y i n g to change o p i n i o n s . A t t i t u d e s c a l e s give very l i t t l e i n d i c a t i o n of a change i n behaviours. Frederiksen (31) discounts t h i s means of e v a l u a t i o n by suggesting t h a t there i s l i t t l e evidence which shows the scores on a t t i t u d e s c a l e s c o r r e l a t e with a c t u a l behaviours. I t i s appropriate to measure knowledge when the simple pos-session of information i s the o b j e c t i v e of i n s t r u c t i o n . However, i n most cases the o b j e c t i v e i s the a p p l i c a t i o n of th a t knowledge to a pressing problem. Therefore, i t i s the behaviour of applying the knowledge which i s the true measure of success of education and not the accumulation of inf o r m a t i o n . In some cases i t i s d i f f i c u l t to measure a c c u r a t e l y a p a r t i c u l a r -24-behaviour. In these instances a r e l a t e d behaviour could be measured. "But s i n c e the r e l a t i o n s h i p of the r e l a t e d behaviour to the u l t i m a t e c r i t e r i o n must be i n f e r r e d on the basis of a l o g i c a l r e l a t i o n s h i p , the v a l i d i t y of the c r i t e r i o n can not be taken f o r granted" (31). E l i c i t i n g "what I would do" behaviour may r e s u l t i n a response to please the examiner. I t may not be what the l e a r n e r would do i n r e a l l i f e . F rederiksen sees advantages i n e l i c i t i n g l i f e - l i k e behaviour r a t h e r than observing r e a l - l i f e behaviour: The r e a l - l i f e behaviour i s the c l o s e s t to the u l t i m a t e o b j e c t i v e s of i n s t r u c t i o n , but observing behaviour i n r e a l -l i f e i s r a r e l y a good technique f o r e v a l u a t i n g because of l a c k of c o n t r o l of the t e s t s i t u a t i o n . The method of e l i c i t i n g l i f e - l i k e behaviour i n s i t u a t i o n s t h a t simulate r e a l - l i f e i s recommended f o r f i r s t c o n s i d e r a t i o n as a measurement technique (31, p.345). Pharmacy Continuing Education In pharmacy, the f o l l o w i n g techniques have been used f o r e v a l u a t i n g programs and i n s t r u c t i o n . P a r t i c i p a t i o n as measured by enrollment f i g u r e s i s a c r i t e r i o n o f t e n used (5,13,35,41,60). The "happiness index" or s o l i c i t i n g l e a r n e r s ' opinions i s probably the most f r e q u e n t l y used e v a l u a t i o n technique (8,9). A t t i t u d e s c a l e s have become popular i n recent years (14,43,63). The p r e - t e s t / p o s t - t e s t design has o f t e n been used to measure knowledge gained (15,37). Most r e c e n t l y , there has been one study which u n o b t r u s i v e l y measured the behaviour of a group of pharmacists who completed a programmed i n s t r u c t i o n a l package on drugs used i n the treatment of u r i n a r y i n f e c t i o n (71). The i n v e s t i g a t o r s observed the pharmacists post-course, -25-r e a l - l i f e performance and compared i t with t h a t of a c o n t r o l group. They found t h a t there was a s i g n i f i c a n t d i f f e r e n c e i n the r e c a l l of f a c t u a l knowledge and i n the r e a l - l i f e performance but not i n a t t i t u d e between the two groups. In the health p r o f e s s i o n s , c o n t i n u i n g p r o f e s s i o n a l education has been seen as a panacea. I t i s f r e q u e n t l y viewed as the means f o r maintaining competence. However, Neylan (57), i n a review of the l i t e r a t u r e on maintaining competence, f o r the years 1970-1973, has suggested that t h i s f a i t h i s not supported by research. Long, has s a i d t h a t "an examination of the p u b l i c health l i t e r a t u r e r e v e a l s l i t t l e evidence of e f f o r t s to measure the actual accomplishments of c o n t i n u i n g education courses". According to Long, one of the under-l y i n g assumptions regarding c o n t i n u i n g education i s "new knowledge, new points of view, new o r i e n t a t i o n s acquired i n courses w i l l be r e t a i n e d and t r a n s l a t e d i n t o improved a t t i t u d e s and behaviours which w i l l , i n t u r n , e n r i c h p u b l i c health programs".(48, p. 968). Two s t u d i e s conducted i n medicine (51,76) i n d i c a t e t h a t c o n t i n u i n g education programs had no s i g n i f i c a n t improvement on the d e l i v e r y of health care i n t h a t p r o f e s s i o n . The other health p r o f e s s i o n s , i n c l u d i n g pharmacy, have done l i t t l e to attempt t h i s form of e v a l u a t i o n . Simulations An Overview L i k e so many other innovations i n education, s i m u l a t i o n technology was f i r s t developed by the m i l i t a r y (21,25,32). Gagne (32) -26-i n d i c a t e d t h a t s i m u l a t i o n s i n the m i l i t a r y are of value i n t r a i n i n g , assessment of p r o f i c i e n c y , and development of an o p e r a t i o n a l d o c t r i n e . On the issue of p r o f i c i e n c y measurement, Gagne f e l t t h a t "simulators f r e q u e n t l y provide the most convenient, r e a l i s t i c and o b j e c t i v e method a v a i l a b l e f o r the assessment of performance" (32, p. 237). Business and i n d u s t r y were the next to use s i m u l a t i o n s f o r the t r a i n i n g and the e v a l u a t i o n of d e c i s i o n making (25). Simulations were l a t e r a p p l i e d to educational processes such as i n s t r u c t i o n , e v a l u a t i o n , research and educational development (25,68). Assessment and i n s t r u c t i o n have seen the widest use of s i m u l a t i o n technology (54). McGuire e x p l a i n s s i m u l a t i o n s i n the f o l l o w i n g manner: Reduced to i t s essence, s i m u l a t i o n c o n s i s t s merely i n p l a c i n g an i n d i v i d u a l i n some aspect of r e a l i t y , and designing around th a t s e t t i n g a problem which r e q u i r e s the l e a r n e r ' s a c t i v e p a r t i c i p a t i o n i n i n i t i a t i n g and c a r r y i n g through a sequence of i n q u i r i e s , ..decisions and a c t i o n s . The s i t u a t i o n must be arranged so t h a t each of the l e a r n e r ' s a c t i v i t i e s t r i g g e r s appropriate feedback which he can u t i l i z e i n subsequent d e c i s i o n s about pending a c t i o n s , d e c i s i o n s which may i n turn modify the problem (54, p. 19). Simulation technology i s c u r r e n t l y being a p p l i e d to a number of educational endeavours. Business and i n d u s t r y have developed numerous simu l a t i o n s i n the area of management t r a i n i n g , business a d m i n i s t r a t i o n and labour r e l a t i o n s . A number of these s i m u l a t i o n s have been reproduced and are a v a i l a b l e commercially from p r i v a t e t r a i n i n g f i r m s . The medical p r o f e s s i o n has used si m u l a t i o n s f o r i n s t r u c t i o n i n the area of i n t e r p e r s o n a l s k i l l s , d i a g n o s t i c s k i l l s and problem-solving (25,52,53,54,64). As w e l l , the medical p r o f e s s i o n i s examining sim u l a t i o n s as a means of e v a l u a t i n g l e a r n i n g , assessing i n s t r u c t i o n -27-and measuring competency to p r a c t i c e (52,64). There are a number of d i f f e r e n t types of si m u l a t i o n s being used. These range from paper and p e n c i l , w r i t t e n s i m u l a t i o n s to computer aided s i m u l a t i o n s and l i v e s i m u l a t i o n s using s p e c i a l l y t r a i n e d p a t i e n t s . Each r e q u i r e the l e a r n e r or examinee to gather information r e l a t e d to some problem and to take a course of a c t i o n which r e s u l t s i n immediate feedback on the consequences of such a c t i o n . McGuire (53,54) discussed the f o l l o w i n g advantages of si m u l a t i o n s : "perceived relevance, predetermination and p r e s e l e c t i o n of task, s t a n d a r d i z a t i o n of the task, improved sampling of performance, improved r a t i n g of performance, increased r e s p o n s i b i l i t y and r e a l i s t i c feedback i n a p r a c t i c a l time frame, compression of r e a l time and increased l e a r n i n g . " The major l i m i t a t i o n s of s i m u l a t i o n s have been o u t l i n e d by McGuire (53,54) and Demers (25). They are only an i n d i c a t i o n of how an i n d i v i d u a l i s capable of behaving. They only approximate r e a l i t y . They do not e x a c t l y d u p l i c a t e r e a l i t y . They are, i n f a c t , simpler than r e a l i t y . "The more complex the task we are t r y i n g to teach or evaluate the more nearly the s i m u l a t i o n has to resemble r e a l i t y " (25, p. 47). The a d u l t education l i t e r a t u r e contains few references r e l a t e d to the use of s i m u l a t i o n technology. Mackenzie commented that t h i s was unfortunate and added t h a t , i n h i s o p i n i o n , s i m u l a t i o n games "hold f o r t h promise f o r the enhancement of a d u l t l e a r n i n g " (49, p. 293). Mackenzie f e l t t h a t s i m u l a t i o n games had a number of a t t r i b u t e s which -28-make them useful to a d u l t education. They are based on s i g n i f i c a n t p r i n c i p l e s of l e a r n i n g . They are mo t i v a t i n g . They provide the l e a r n e r with a chance to a c t i v e l y p a r t i c i p a t e i n group s i t u a t i o n s which encourage i n t e r a c t i o n of the l e a r n e r s . They provide o p p o r t u n i t i e s f o r important l e a r n i n g outcomes such as decision-making and problem-solving. Mackenzie summarized h i s view i n the f o l l o w i n g manner: " s i m u l a t i o n games technology w i l l not r e v o l u t i o n i z e a d u l t education, but t h i s technology can go a long way i n improving the q u a l i t y of a d u l t l e a r n i n g " (49, p. 74). Although two examples of the use of s i m u l a t i o n technology f o r i n s t r u c t i o n i n a d u l t education are reported by N i c e l y and K n o l l e (58) and Barkley and Dickinson ( 7 ) , i t has not as y e t been used e x t e n s i v e l y to evaluate i n s t r u c t i o n . One p r o f e s s i o n which has concentrated on the use of s i m u l a t i o n s to measure i n s t r u c t i o n a l outcomes i s medicine. A p a r t i c u l a r type of s i m u l a t i o n which has been ga i n i n g p o p u l a r i t y i n con t i n u i n g medical education i s the w r i t t e n s i m u l a t i o n or p a t i e n t management problems (PMP's). One reason f o r t h e i r p o p u l a r i t y i s th a t they are r e l a t i v e l y inexpensive. They are now being used widely i n medicine and experimentally i n other health professions such as nursin g , d i e t e t i c s and occupational therapy (65). They have not as y e t been used i n pharmacy (55). Writt e n Simulations Written s i m u l a t i o n s are reported to be of value i n assessing c r i t i c a l t h i n k i n g and problem s o l v i n g , however, there have been no stu d i e s to t e s t the v a l i d i t y of t h i s a s s e r t i o n . Sedlacek and Nattress -29-q u a l i f y t h e i r enthusiasm f o r w r i t t e n s i m u l a t i o n s by s t a t i n g t h a t "the greate s t d i f f i c u l t y i n attempting to use the r e s u l t s of PMP's i n ev a l u a t i o n i s th a t there i s no strong evidence of a d i r e c t r e l a t i o n -ship between performance on an e x e r c i s e and what a p h y s i c i a n may do i n p r a c t i c e " (64, p. 263). They l a t e r add th a t " u l t i m a t e l y PMP's must be v a l i d a t e d a g a i n s t an external c r i t e r i o n of p h y s i c i a n performance" (p. 266). The issue of g e n e r a l i z i n g r e s u l t s obtained from w r i t t e n s i m u l a t i o n s to the r e a l world i s a t present a clouded one. E a r l y s t u d i e s were somewhat encouraging according to McGuire (53), but l a t e r s t u d i e s are causing i n v e s t i g a t o r s to question the g e n e r a l i z -a b i l i t y of problem s o l v i n g s k i l l s demonstrated on w r i t t e n s i m u l a t i o n s . R e l i a b i l i t y and V a l i d i t y The r e l i a b i l i t y and v a l i d i t y of w r i t t e n s i m u l a t i o n s are d i f f i c u l t to determine because normal psychometric procedures do not always apply (65). The f o l l o w i n g summary was presented by Shannon. R e l i a b i l i t y : (a) i n t e r n a l consistency (compare parts or components): PMP's have i n t e r r e l a t e d d i f f e r e n t l y weighted p a r t s , hard to f i n d comparable parts (b) s t a b i l i t y ( t e s t - r e t e s t ) : hard to retake a PMP, too much l e a r n i n g takes p l a c e , plus the feedback answers are un-covered (c) combination of (a) and (b) ( a l t e r n a t e form): What i s an a l t e r n a t e form of a PMP? (65, p. 71). Shannon has s a i d t h a t the U n i v e r s i t y of I l l i n o i s School of Medicine has s t r e s s e d g e n e r a l i z a b i l i t y and i n t e r r a t e r r e l i a b i l i t y . -30-G e n e r a l i z a b l e t o : 1. s i m i l a r cases 2. a d i s c i p l i n e 3. o v e r a l l c l i n i c a l competence I n t e r r a t e r r e l i a b i l i t y : Do other M.D.'s Feel the weights are appropriate? (see Sedlacek and Nattress ( 6 5 ) ) . V a l i d i t y : (a) face v a l i d i t y : to date the major source of PMP v a l i d i t y . Experts ( c l i n i c a l f a c u l t y ) declare t h i s i s the process that one goes through i n case management: students have s a i d t h a t "these are more l i k e r e a l cases", e t c . (b) c o n s t r u c t v a l i d i t y : do second year students perform b e t t e r on PMP's than non-medical students and worse than r e s i d e n t s ? (c) concurrent v a l i d i t y : p a r a l l e l c r i t e r i a are u n r e l i a b l e or l a c k i n g , so l i t t l e c o n c l u s i v e r e s u l t s to date (65, p. 71). P r e d i c t i v e V a l i d i t y By f a r one of the most important questions i s the p r e d i c t i v e v a l i d i t y of w r i t t e n s i m u l a t i o n s . Does an i n d i v i d u a l solve a problem i n a w r i t t e n s i m u l a t i o n format i n much the same way as he or she solves that problem i n the r e a l world? Since an e v a l u a t i o n i n the r e a l world i s d i f f i c u l t and often impossible, s i m u l a t i o n s w i t h s i g n i f i c a n t p r e d i c t i v e v a l i d i t y would enable an e s t i m a t i o n of an i n d i v i d u a l ' s r e a l -l i f e performance. This would be among the strongest e v a l u a t i o n techniques c u r r e n t l y a v a i l a b l e to educators. I t i s d i f f i c u l t to a s c e r t a i n the p r e d i c t i v e v a l i d i t y of w r i t t e n s i m u l a t i o n s . In previous work, the performance on w r i t t e n s i m u l a t i o n s was compared to other simulated s i t u a t i o n s such as a simulated p a t i e n t i n a p h y s i c i a n ' s o f f i c e . In some i n s t a n c e s , i t was compared to a c h a r t review or a physician's record keeping behaviour. These were not r e a l - l i f e behaviours. More r e c e n t l y p h y s i c i a n s ' -31-performance on a w r i t t e n s i m u l a t i o n about drug induced agr a n u l o c y t o s i s was compared to t h e i r r e a l - l i f e p r a c t i c e (60). In both s i t u a t i o n s , the m o r t a l i t y r a t e from drug induced agranulocytosis was the same, ten percent. "This r e s u l t could be i n t e r p r e t e d as implying that per-formance i n the s i m u l a t i o n t e s t may be p r e d i c t i v e of c l i n i c a l p e r f o r -mance, at l e a s t when d e a l i n g with a g r a n u l o c y t o s i s " (60, p. 660). Summary Some of the l i t e r a t u r e reviewed f o r t h i s chapter i n d i c a t e d pharmacists were not performing adequately t h e i r r o l e as advisors to the p u b l i c on the use of n o n - p r e s c r i p t i o n medications. As w e l l , other authors saw an expanding r o l e f o r the pharmacist as a primary care c o n s u l t a n t . Continuing education, i t has been suggested, w i l l a s s i s t pharmacists to improve performance i n such areas and to cope with change. Furthermore, there was evidence i n the a r t i c l e s reviewed t h a t more j u r i s d i c t i o n s ^ are r e q u i r i n g mandatory p a r t i c i p a t i o n i n c o n t i n u i n g pharmacy education a c t i v i t i e s . Although, there has been a change i n both the q u a n t i t y and q u a l i t y of the research a c t i v i t i e s s i n c e the report of Nakomoto and Verner (56), much remains to be done to de-termine the e f f e c t i v e n e s s of c o n t i n u i n g education on the p r a c t i c e of the p a r t i c i p a n t s . The l i t e r a t u r e r e f l e c t s a need i n a l l the health professions as an e v a l u a t i v e technique, have not been used to t h i s p o i n t i n pharmacy. There was a need f o r the e x p l o r a t i o n of t h e i r For example, si n c e 1975 three Canadian Provinces ( A l b e r t a , Saskatchewan and Manitoba) have r e q u i r e d , or intend to r e q u i r e , man-datory p a r t i c i p a t i o n by pharmacists i n co n t i n u i n g education programs. See Canadian Pharmaceutical Journal reports of annual meetings 1975-1977. -32 -a p p l i c a t i o n to pharmacy and f u r t h e r i n v e s t i g a t i o n s of t h e i r e v a l u a t i v e powers. -33-References Chapter Two 1. "AACP/A.Ph.A. P r e l i m i n a r y Report of Task Force on Continuing Competence of Pharmacists." Journal of American Pharmaceutical  A s s o c i a t i o n NS14 (1974): 340. 2. "AACP/A.Ph.A. Task Force on Continuing Competence of Pharmacists." Journal of American Pharmaceutical A s s o c i a t i o n NS15 (August, 1975): 432-436 and 457. 3. American Druggist 169 (March 15, 1974): 16 4. American Druggist 169 (June 1, 1974): 11,26. 5. Arndt, J.R.; J.E. Demuth and M.H. Weinswig, " P a r t i c i p a t i o n of Wisconsin Pharmacists i n Continuing Education, 1969-1973: A Pr e l i m i n a r y Report." American Journal of Pharmaceutical Educa- t i o n 39 (August, 1975): 264-266. 6. Arndt, J.R. and M.H. Weinswig, "Classrooms Without Walls: Con-t i n u i n g Education Course f o r Pharmacists V ia Cassette Tape Re-cordings." American Journal of Pharmacy Education 37 (1973): 61-64. 7. Bar k l e y , W.D. and J.G. Di c k i n s o n , "The Design and Evaluation of a Rural Land Use Simulation Game." Adult Education 24 (1974): 280-282. 8. Barnes, B.A., " P r a c t i c a l Approach to Continuing Education - Cor-respondence." Journal of American Pharmaceutical A s s o c i a t i o n NS7 (1967): 18-22. 9. Barnes, B.A., "Continuing Education by Correspondence." American Journal of Pharmaceutical Education 33 (1969): 826-831. 10. Bass, M., "The Pharmacist as a Provider of Primary Care." Cana- dian Medical A s s o c i a t i o n Journal 112 (January 11, 1975): 60-64. 11. "Be Wise With Medicines, Ask Your Pharmacist How." Journal of  American Pharmaceutical A s s o c i a t i o n NS15 (August, 1975): 428-429. 12. Beauland, W.A., "S e l f - M e d i c a t i o n i n Whose I n t e r e s t ? A Pharmacist's View." Royal S o c i e t y of Health Journal 94 (August, 1974): 169-172. 13. B e r n a r d i , V.W., "Continuing Education and the Pharmacist: An Analy-s i s Part 1." Apothecary 86 (1974): 5-8, 25-27. -34-14. B e r n a r d i , V.W., " A n a l y s i s of Pharmacist P a r t i c i p a t i o n i n and A t t i t u d e s Toward Continuing Education." American Journal of  Pharmaceutical Education 39 (August, 1 9 7 5 J 1 2 6 7-271. 15. Blank, J.W.; K.W. K i r k and M.H. Weinswig, "Evaluation of an Audio Cassette Tape Lecture Course." American Journal of  Pharmaceutical Education 39 (August, 1975): 275-278. 16. B l o c k s t e i n , W.L. and W.J. Durant, "Continuing Education and the Tele-Conference Technique." American Journal of Pharmaceutical  Education 32 (1968): 619-624. 17. B i c k e t t , J.W., "Autotherapy - The Future i s Now." Journal of  American Pharmaceutical A s s o c i a t i o n NS12 (November, 1972): 560-562, 564., 18. Bowles, J r . , G.C., "Drugs Sold i n the Community can be Source of Adverse Reactions i n the H o s p i t a l . " Modern Hos p i t a l 119 (December, 1972): 108. 19. Brunner, E. des. et a l _ . , "Problems of Evaluation Research", i n An Overview of Adult Education Research. Chicago: Adult Ed-ucation A s s o c i a t i o n of the U.S.A., 1959. 20. C a l v i n , H.S., "Continuing Education Requirements f o r I n s t i t u -t i o n a l Pharmacy P r a c t i c e . " American Journal H o s p i t a l Pharmacy 24 (1967): 188-191. 21. C a s s e l , R.N., " I n s t r u c t i o n a l Gaming and S i m u l a t i o n . " Contemporary  Education 45 (1974): 100-105 22. Close, CM. and M.D. Danion, "The Value of the Use of the Family Medication Record." Journal of American Pharmaceutical A s s o c i a - t i o n NS13 ( J u l y , 19737": 342-343 23. Committee on E v a l u a t i o n , Adult Education A s s o c i a t i o n , Program Evaluation i n Adult Education, Washington, D.C. (1952) c i t e d by Brunner e_t a l _ . , oj_ c i t . 24. "Continuing Education: Voluntary or Compulsory?" Pharmacy Times (May, 1970): 38-43. 25. Demers, J.L., "Simulations i n C l i n i c a l E v a l u a t i o n . " C I i n i c a l  E v a l u a t i o n : A l t e r n a t i v e For Health Related Educators. Edited by D.M. Irby and J.K. Morgan, A P u b l i c a t i o n of the A l l i e d Health I n s t r u c t i o n a l Personnel, G a i n s v i l l e , F l o r i d a (1974): 47-56. 26. DeMuth, J.E., "Bibliography of Continuing Education, 1960-1975." Xerox Copy, Madison, Wisconsin. -35-27. DeMuth, J.E.; K.W. K i r k and M.H. Weinswig, "Continuing Education For Non-Practising P r o f e s s i o n a l s : A Case Study of a Program For Pharmacists." Adult Education 26 (1976): 157-166. 28. E d i t o r i a l : "Ask Your F r i e n d l y Pharmacist." B r i t i s h Medical  Journal (March 15, 1975): 594. 29. Edwards, J . ; J . A t t a n ; A. Wilson; B. White and A. Herxheimer, "General P r a c t i c e Pharmacists: How Good i s the Advice They Give P a t i e n t s ? " The Pharmaceutical Journal (June 28, 1975): 568-570. 30. E s s e r t , P., C r e a t i v e Leadership i n Adult Education New York: P r e n t i c e - H a l l , Inc., 1951. 31. F r e d e r i k s e n , N., " P r o f i c i e n c y Tests f o r T r a i n i n g E v a l u a t i o n " i n T r a i n i n g Research and Education. R. G l a s e r , E d i t o r , Uni-v e r s i t y of P i t t s b u r g Press, 1962, 323-346. 32. Gagne, R.M., "Simulators" i n T r a i n i n g Research and Education. R. G l a s e r , E d i t o r , U n i v e r s i t y of P i t t s b u r g Press, 1962, 223-246. 33. G e r v a s i , P., "The Community Pharmacist: Health Se n t i n e l on the Home Front." American Journal of P u b l i c Health 64 (August, 1974): 811-812. 34. G o l d s t e i n , F.J., "Continuing Education v i a Cassette Tape." American Journal of Pharmacy 144 (1972): 105-109. 35. Hartman, C.W. and C.E. Watkins, "Opinions on Continuing Educa-t i o n " American Journal of Pharmaceutical Education 27 (1963): 426-431. 36. Harvey, E., "Some I m p l i c a t i o n s of Value D i f f e r e n t i a t i o n i n i n Pharmacy." The Canadian Review of Sociology and Anthro- pology 3 (1966): 23-27. 37. Hodapp, W.J. and C. Kanun, "Measurement of R e l a t i v e E f f e c t i v e n e s s of Programmed I n s t r u c t i o n and Closed C i r c u i t T e l e v i s i o n as Adult Education Tools." American Journal of Pharmaceutical Education 34 (1970): 241-248. 38. Hodapp, W.J. and A.W. Walker, "An Experiment i n Continuing Ed-ucation U t i l i z i n g Closed C i r c u i t T e l e v i s i o n . " American Journal  of Pharmaceutical Education 32 (1968): 625-633. 39. Huyck, C.L., "Topics P r a c t i c i n g Pharmacists P r e f e r on Refresher Programs." American Journal of Pharmaceutical Education 24 (1960): 185-190. -36-40. Jang, R., "Evaluation of the Q u a l i t y of Drug-Related Services Provided by Community Pharmacists i n a Metropolitan Area." Ph.D. D i s s e r t a t i o n , Ohio State U n i v e r s i t y , 1971. 41. Jobe, B., "Growth and Development of Continuing Education Of-f e r i n g s by Schools of Pharmacy." American Journal of Pharma- c e u t i c a l Education 32 (1968): 610-618. 42. Knapp, D.A.; H.H. Wolf; D.E. Knapp and T.A. Rudy, "The Pharma-c i s t as a Drug Advisor." Journal of American Pharmaceutical  A s s o c i a t i o n NS9 (1969): 502. 43. Kotzan, J.A. and A.W. Jowdry, " D i f f e r e n t i a l A t t i t u d e s Toward an Adult Education Program f o r Pharmacists." Adult Education 21 (1970): 20-28. 44. Krammer, J.E. and L.F. T i c e , "P.CP. and S. and The U n i v e r s i t y of the A i r . " American Journal of Pharmaceutical Education 27 (1963): 454. 45. L i n n , L.S., "Indicated Versus Actual Behaviour: The Pharmacist as a Health Advisor." S o c i a l Science and Medicine 7 (1973): 191-197. 46. L i n n , L.S. and M.S. Davis, "Factors Associated with Actual and P r e f e r r e d A c t i v i t i e s of Pharmacists." Journal of American  Pharmaceutical A s s o c i a t i o n NS11 (1971): 545-548. 47. L i s t e r , J . , "An Ancient R i v a l r y . " New England Journal of Medicine 292 (May, 1975): 1173. 48. Long, L.D., "The Evaluation of Continuing Education E f f o r t s . " American Journal of P u b l i c Health 59 (1969): 967-973. 49. MacKenzie, L., "Simulation Games and Adult Education." Adult  Leadership 22 (1974): 293-295. 50. McCormick, T., "The Druggists' Dilemma: Problems of a Marginal Occupation." American Journal of Sociology 61 (1956): 308-315. 51. McGuire, C.H.; R. H u r t l e y ; D. Babbott and J . Butterworth, "Auscultatory S k i l l : Gain and Retention a f t e r Intensive I n s t r u c t i o n . " Journal of Medical Education 39 (1964): 120-131. 52. McGuire, C H . and D. Babbott, "Simulation Technique i n the Measurement of Problem S o l v i n g S k i l l s . " Journal of Educa- t i o n a l Measurement 4 ( S p r i n g , 1967): 1-10. 53. McGuire, C.H., "Simulation Techniques i n the Teaching and Test-ing of Problem S o l v i n g S k i l l s " , a v a i l a b l e from Ohio State Uni-v e r s i t y Centre f o r Science and Mathematics, 244 Arps H a l l , Columbus, Ohio, 43210. ERIC ED#091152, 1973. -37-54. McGuire, C.H. and F.H. Wezeman, "Simulation i n I n s t r u c t i o n and Evaluation i n Medicine" i n Educational S t r a t e g i e s f o r the Health  P r o f e s s i o n s . Edited by G.E. M i l l e r and T. Fulop, P u b l i c Health Papers #61, World Health O r g a n i z a t i o n , Geneva, 1974, 18-34. 55. McGuire, C.H., Interviewed by D.W. F i e l d i n g , A p r i l 10, 1975 at The U n i v e r s i t y of B r i t i s h Columbia, Vancouver, B.C. 56. Nakamoto, J . and C. Verner, Continuing Education i n Pharmacy: A  Review of North American L i t e r a t u r e 1960-1970. (W.K. Kellogg P r o j e c t Report #6), Vancouver: Adult Education Research Centre and D i v i s i o n of Continuing Education i n the Health Sciences, The U n i v e r s i t y of B r i t i s h Columbia, 1972. 57. Neylan, M., L i t e r a t u r e Review: Maintaining the Competence of  Health P r o f e s s i o n a l s , 1970-1973. The U n i v e r s i t y of B r i t i s h Columbia Health Sciences Centre, Continuing Education i n the Health Sciences, August, 1974. 58. N i c e l y , R.F. and K n o l l e , L.M. "Simulation i n Adult Education." Journal of Business Education 49 (1974): 323-325. 59. "Over the Counter I n t e l l i g e n c e , Your Pharmacist Has I t . " Journal  of American Pharmaceutical A s s o c i a t i o n NS14 (March, 1974): 116-T2_ : 60. P a l v a , I.P. and V. Korhonen, " V a l i d i t y and Use of Written Simula-t i o n Tests of C l i n i c a l Performance." Journal of Medical Educa- t i o n 51 (August, 1976): 657-661. 61. "Report of the Task Force on Roles of the P r a c t i t i o n e r of Pharmacy and the Subprofessional i n Pharmacy." Journal of American Pharma- c e u t i c a l A s s o c i a t i o n NS9 (August, 1969): 416-423. 62. Rodowskas, C.A. and R.V. Evanson, "Continuing Education i n Pharma-cy: M o t i v a t i o n and Content." American Journal of Pharmaceutical  Education 28 (1964): 393-403. 63. Rouege, A.M.; K.W. K i r k and M.H. Weinswig, "Continuing Education and the Pharmacist: An A n a l y s i s , Part 2." The Apothecary 86 (February, 1972): 14-16, 30-32. 64. Sedlacek, W.E. and L.W. N a t t r e s s , "A Technique For Determination of the V a l i d i t y of P a t i e n t Management Problems." Journal of  Medical Education 47 (1972): 263-266. 65. Shannon, F.N., "The P a t i e n t Management Problem" i n C l i n i c a l Evalua- t i o n : A l t e r n a t i v e s f o r Health Related Educators, o p , c i t . , 57-72. 66. Strommen, R.S. "Education by Coercion. E f f e c t i v e Methods of In-c r e a s i n g Attendance at Adult Education Programs through Increased Services and Expanded Promotional E f f o r t . " American Journal of  Pharmaceutical Education 24 (1960): 180-184. -38-67. Thiede, E., "Evaluation and Adult Education" i n Adult Education  Outlines of an Emerging F i e l d of U n i v e r s i t y Study. Edited by G. Jensen, A.A. L i v e r i g h t , and W. Hallenback, Adult Education A s s o c i a t i o n of the U.S.A., 1964. 68. Twelker, P.A., "Simulations: An Overview." ERIC ED #025450, 1968. 69. Verner, C , "Some Considerations of Ev a l u a t i o n " i n Adult Educa- t i o n Extension Papers. Department of Extension, U n i v e r s i t y of A l b e r t a , Edmonton, A l b e r t a , 1960. 70. Verner, C. and A. Booth, Adult Education. The Center For Appl i e d Research i n Education, Inc., New York, 1964, 91-105. 71. Watkins, R.L.; G.J. Norwood and F.L. M e i s t e r , "Improving the Q u a l i t y o f the Pharmacist as a Drug Advisor to P a t i e n t s and Physicians through Continuing Education." American Journal  of Pharmaceutical Education 40 (February, 1976): 34-39. 72. W e l l , J.P., " S e l f - M e d i c a t i o n i n Whose I n t e r e s t ? View of Industry." Royal S o c i e t y of Health Journal 94 (August, 1974) : 169-172. 73. Wertheimer, A.I.; E. Shefter and R.M. Cooper, "More on The Pharmacist As a Drug Consultant: Three Case Studies." Drug I n t e l l i g e n c e and C l i n i c a l Pharmacy 7 (February, 1973): 58-61. 74. Which? (March, 1975): 81; c i t e d by John L i s t e r i n , "An Ancient R i v a l r y . " New England Journal of Medicine 292 (May, 1975) : 1173. 75. Wilk, D., "The Need f o r I n t e r d i s c i p l i n a r y Continuing Education." Proceeding 12th Annual Meeting: Section on Teachers of Continu- ing Education (AACP). (1974): 133-140. 76. Williamson, J . ; M. Alexander and G. M i l l e r , "Continuing Education and P a t i e n t Care Research: P h y s i c i a n Response to Screening Test R e s u l t s . " Journal of American Medical A s s o c i a t i o n 201 (1967): 938-942. -39-CHAPTER THREE INSTRUMENT DEVELOPMENT The purpose of the study was to evaluate a co n t i n u i n g pharmacy education program by measuring p a r t i c i p a n t r e a l - l i f e behaviour change, gain i n f a c t u a l knowledge and s a t i s f a c t i o n . As w e l l , the p r e d i c t i v e powers of w r i t t e n s i m u l a t i o n s were i n v e s t i g a t e d . To accomplish the s p e c i f i c o b j e c t i v e s a s s o c i a t e d with these purposes, a number of measurement instruments were developed. This chapter describes t h e i r development. In-Store-Assessment Problems Development Four in-store-assessment-problems (ISAPs) were developed to enable a d i s g u i s e d e v a l u a t i o n of pharmacists as primary care c o n s u l t a n t s , at t h e i r place of employment. The ISAPs were developed around consumer requests f o r a s s i s t a n c e on a health matter. Two of the problems concerned the request, "What i s good f o r a c o l d ? " and two concerned the request "What i s the strongest pain r e l i e v e r I may purchase without a p r e s c r i p t i o n ? " These two areas were chosen because a s s i s t a n c e on " c o l d " and "pain" problems are among the most frequent primary care requests received by a pharmacist. Around each, a scenario was constructed d e s c r i b i n g the person with the problem, the symptoms, the i n d i v i d u a l ' s use of medications, the d u r a t i o n , and a l l e r g i e s . One " c o l d " problem and one "pain" problem were of a l e s s s erious nature and, t h e r e f o r e , i t was appropriate f o r the pharmacist to recommend a product. The other two -40 -problems were of a serious nature and warranted p h y s i c i a n c o n s u l t a t i o n . The problems always involved an i n d i v i d u a l asking on behalf of a f r i e n d or r e l a t i v e . The four problems are d e t a i l e d i n Appendix A. Accompanying each ISAP, a l i s t of p l a u s i b l e pharmacist responses to the problem was developed. The l i s t attempted to be comprehensive and to in c l u d e appropriate and in a p p r o p r i a t e responses. Panel of Content Experts The four ISAPs and t h e i r corresponding l i s t of behaviours were submitted to a ten-member panel of content experts f o r v a l i d a t i o n . These i n d i v i d u a l s are described i n Appendix B. Three i n d i v i d u a l s were members of the D i v i s i o n of C l i n i c a l Pharmacy, Fa c u l t y of Pharmaceutical Sciences at U.B.C., who had experience i n community pharmacy. The other seven were pharmacy p r a c t i t i o n e r s who had varying amounts of community pharmacy experience. Procedure f o r Rating the ISAPs In the f i r s t meeting, the p a n e l i s t s were asked to r a t e the c l i n i c a l importance, r e a l i s m and p l a u s i b i l i t y of each problem and they were encouraged to suggest improvements. As w e l l , they were asked to examine the l i s t of behaviours f o r each ISAP and to suggest a d d i t i o n s and d e l e t i o n s . They were then given the ISAPs and behaviours to take home. Over the next two weeks, they rated each behaviour as something a pharmacist: must do, should do, could do, should not do or must not do i n accord with the c r i t e r i a o u t l i n e d i n Table 1. -41-TABLE I RATING CATEGORIES APPLIED TO PROBLEM SOLVING ACTIVITIES TO ESTABLISH PERFORMANCE CRITERIA* MUST DO Should Do The a c t i s c r u c i a l to safe and e f f e c t i v e p a t i e n t care. Without exception i t must be performed by the pharmacist. The act f u r t h e r c o n t r i b u t e s to safe and e f f e c t i v e p a t i e n t care. I t may be omitted only under cer-t a i n circumstances, e.g., the p a t i e n t already knows, or an emergency occurs i n the s t o r e . NOTE: workload pressure i s NOT s u f f i c i e n t j u s t i -f i c a t i o n f o r omission. Could Do Should NOT Do The act f u r t h e r c o n t r i b u t e s to safe and e f f e c t i v e p a t i e n t care. In gen e r a l , however, the pharmacist need not perform i t . The act i s p o t e n t i a l l y bad f o r safe and e f f e c t i v e p a t i e n t care. I t may mislead the p a t i e n t causing excessive delay i n treatment and an exacerbation of the i l l n e s s . Must NOT Do The act i s c l e a r l y detrimental to safe and e f f e c t i v e p a t i e n t care. I t has dangerous consequences f o r the p a t i e n t . *Modified Form: R. Jang. "Evaluation of the Q u a l i t y of Drug-Related Services Provided by Community Pharmacists i n a Metro-p o l i t a n Area", Unpublished Ph.D. D i s s e r t a t i o n , Ohio State U n i v e r s i t y , 1971, p. 34. A f t e r two weeks, a second meeting was held to discuss the ISAPs. Any new, r e l e v a n t behaviours suggested by a panel member were rated by the other panel members. The importance of each a c t i v i t y was determined by weighting each category.thus: must do = 5; should do = 4; could do = 3; should not do = 2; and must not do = 1. The mean, standard d e v i a t i o n , range and -42-i n t e r q u a r t i l e range (Q-value) were c a l c u l a t e d f o r each behaviour and are reported i n Tables II to V i n Appendix A. Judges' Ratings of the ISAPs The judges found the four ISAPs were f e a s i b l e and d i d represent s i t u a t i o n s which can occur d a i l y i n community pharmacies. As w e l l , they found the l i s t s of pharmacist behaviours developed f o r each problem were adequate and r e a l i s t i c representations of p o s s i b l e responses to the p a r t i c u l a r primary care requests. The face v a l i d i t y of the ISAPs was thereby e s t a b l i s h e d . The judges' r a t i n g s of the i n d i v i d u a l behaviours i n the four ISAPs may be summarized i n the f o l l o w i n g manner. For the 25 behaviours i n ISAP one - c o l d appropriate - a l l ten judges were unanimous i n t h e i r r a t i n g s of three behaviours, and i n a d d i t i o n , at l e a s t s i x judges agreed on another 15 items. The gre a t e s t range f o r the judges' r a t i n g s , on any item, i n t h i s ISAP was two. ISAP two - c o l d i n a p p r o p r i a t e - had 27 be-haviours, the ten judges were unanimous i n t h e i r r a t i n g s of two behav-i o u r s , and at l e a s t s i x of the judges agreed on another 15. The great-e s t range f o r the judges' r a t i n g s , on any item, i n t h i s ISAP was t h r e e , occuring on only one item. There were 25 items i n ISAP three - pain appropriate - three of which received unanimous r a t i n g s and 14 re-ceived the same r a t i n g s by at l e a s t s i x of the judges. The greatest range of the judges' r a t i n g s , on any item, i n t h i s ISAP was three, which occurred on two items. The ten judges were unanimous on two of the 24 items i n ISAP fo u r - c o l d i n a p p r o p r i a t e - and the m a j o r i t y agreed on another 15 items. The gre a t e s t range f o r the judges' r a t i n g s , on any -43-item, was two. (See Tables II to V i n Appendix A). These r e s u l t s i n d i c a t e that there was s u b s t a n t i a l agreement on what a pharmacist should or should not do i n response to the primary care requests made i n the ISAPs. The u n i f o r m i t y of the judges' r a t i n g s was a l s o studied i n terms of the mean and standard d e v i a t i o n o f the r a t i n g s of each judge on each problem. For the four ISAPs developed f o r t h i s study, the judges' assessments range from 3.72 to 4.31, from 3.14 to 4.19, from 3.33 to 3.72 and from 32.5 to 3.72 f o r problems one to four r e s p e c t i v e l y . A summary of these f i n d i n g s are presented i n Table VI. TABLE VI MEANS AND STANDARD DEVIATIONS (S.D.) OF JUDGES' WEIGHTINGS* OF THE BEHAVIOURS IN THE ISAPs Cold Cold Pain Pain Appropriate Inappropriate Appropriate Inappropriate  Judge Mean S.D. Mean S.D. Mean S.D. Mean S.D. 1. 4.04 1.40 3.25 1 .60 3.63 1 .56 3.72 1.55 2. 3.81 1.29 3.50 1 .45 3.35 1.46 3.50 1.43 3. 4.09 1.26 3.52 1 .33 3.58 1.54 3.55 1.60 4. 4.31 1.12 4.19 1 .17 3.72 1.38 3.68 1.49 5. 3.90 1.07 3.47 1 .61 3.60 1.39 3.25 1.58 6. 4.00 1.23 3.40 1 .59 3.47 1.56 3.68 1.61 7. 3.85 1.18 3.20 1 .60 3.50 1.50 3.50 1.66 8. 3.96 1.46 3.42 1 .65 3.50 1.59 3.68 1.67 9. 3.81 1.22 3.64 1 .07 3.33 1.39 3.43 1.40 10. 3.72 1.32 3.14 1 .23 3.46 1.29 3.36 1.40 *5 = 1 = Must do, Must Not 4 = Should Do, Do: 3 = Could Do, 2 = Should Not Do, and -44-Evidence i n d i c a t i n g the extent of agreement among the judges on the weights to be assigned to the behaviours i n the ISAPs i s presented by the c o r r e l a t i o n c o e f f i c i e n t s i n Table V I I . The mean c o r r e l a t i o n c o e f f i c i e n t s among the judges' r a t i n g s f o r the behaviours i n ISAP one -co l d appropriate - was 0.83. The other mean c o r r e l a t i o n c o e f f i c i e n t s f o r the judges' r a t i n g s were 0.74 f o r c o l d i n a p p r o r p i a t e , 0.86 f o r pain appropriate and 0.84 f o r pain i n a p p r o p r i a t e . The mean c o r r e l a t i o n r e p resenting the extent of agreement among the ten judges over a l l items i n a l l four problems was 0.81 (Table V I I I , Appendix A). The r e s u l t s i n Tables VI and VII provide c o r r o b o r a t i o n t h a t the judges had reasonably agreed upon the d e s i r a b i l i t y of the behaviours suggested f o r each ISAP. Therefore, i t was concluded that the ISAPs and t h e i r l i s t of behaviours could be used to provide v a l i d assessments of a pharmacist's primary care c o n s u l t i n g behaviour. C h e c k l i s t s The behaviours a s s o c i a t e d with the ISAPs were used to c o n s t r u c t a performance c h e c k l i s t f o r each. These c h e c k l i s t s were used l a t e r to record the pharmacist's responses to the primary care request (see Appendix A). Dual Function of the ISAPs The ISAPs served two purposes: to evaluate the improvement i n performance of the program r e g i s t r a n t s , and to t e s t the p r e d i c t i v e v a l i d i t y of the four s i m u l a t i o n s . The si m u l a t i o n s completed between the times of pre- and post-in-store-assessment, were v a l i d a t e d a g a i n s t an i n d i v i d u a l ' s performance on the ISAPs. Because the same s i t u a t i o n s could -45-TABLE VII INTERCORRELATIONS OF JUDGES' RATINGS OF THE BEHAVIOURS IN ISAPs ONE TO FOUR jdqe In-Store-2 3 4 ! 6 7 8 9 10 Assessment Problem* 1 One .7941 .9112 .9266 8805 .8844 .8537 .9100 .8713 .8363 Two .7560 .7623 .6481 8711 .7933 .8851 .8829 .6626 .7434 Three .8352 .9008 .8548 .9519 .8676 .8932 .8990 .8947 .8608 Four .7478 .9485 .7848 7840 .7824 .8548 .9574 .8936 .8173 2 One .7344 .6922 8031 .7689 .7654 .7488 .7910 .6955 Two .6271 .6792 .9137 .7735 .7478 .8753 .7481 .6487 Three .8259 .7703 9016 .8585 .8262 .8881 .8810 .7831 Four .7171 .7649 .6459 .8912 .8298 .7812 .8259 .8857 3 One .8429 8619 .8164 .8446 .8743 .8107 .7855 Two .4144 6654 .8711 .6906 .6321 .7150 .5799 Three .7751 9367 . .8138 .9409 .9233 .8254 .8945 Four .7301 .9678 .7399 .8343 .9438 .8857 .8190 4 One 8201 .8903 .8318 .9207 .8389 .8624 Two 8557 .5144 .5815 .8023 .7640 .7410 Three 8018 .8895 .8907 .7533 .9138 .8123 Four 7567 .8648 .9545 .7967 .8891 .8792 5 One .8214 .8187 .8593 .9049 .7701 Two .7308 .8299 .9248 .8040 .8249 Three .8779 .8805 .9197 .8670 .8811 Four .7005 .8198 .7896 .8503 .8920 6 One .8965 .8808 .8167 .8790 Two .7133 .7785 .6213 .5955 Three .8333 .7829 .9129 .8098 Four .8642 .8067 .8441 .7914 7 One .8494 .7711 .7726 Two .9074 .6260 .7578 Three .8827 .8856 .9016 Four .8859 .9447 .9258 8 One *ISAP One = Cold Appropr iate .8753 .8342 Two ISAP Two = Cold Inappropriate .7320 .7341 Three ISAP Three = Pain Appropr iate .7871 .8725 Four ISAP Four = Pain Inappropriate .8986 .8047 9 One .8276 Two .7075 Three .8725 Four .8885 10 One Two Three Four -46-not be used i n a pre- and p o s t - s i m u l a t i o n - too much l e a r n i n g occurs i n completing the f i r s t s i m u l a t i o n - d i f f e r e n t ISAPs had to be used as the pre- and post-assessments. This f a c t created d i f f i c u l t i e s i n making inferences about the e f f e c t of the educational program on r e a l - l i f e performance. Thus, four subsets of behaviours were s e l e c t e d which were common to both the pre- and post-ISAPs. These behaviours were used as the c r i t e r i a f o r e v a l u a t i n g the pharmacist's performance and are i n d i c a t e d by a s t e r i s k s i n Tables II to V i n Appendix A. The r a t i o n a l e f o r these subsets was to ensure maximum p a r a l l e l i s m betweeen the pre- and post-course e v a l u a t i o n s . The four behaviour c a t e g o r i e s employed were "data g a t h e r i n g " , " i n a p p r o p r i a t e recommendations", "appropriate recommendations" and "drug-use-counselling". "Data gathering" c o n s i s t e d of those questions the panel of judges f e l t should be asked about the problem before recommending a course of a c t i o n . The s e c t i o n s "appropriate" and " i n a p p r o p r i a t e recommendations" cont a i n those pharmacist behaviours which, i n the opinion of the judges were e i t h e r acceptable or unaccept-able s o l u t i o n s to the problems.^ "Drug-use-counselling" contained those recommendations which the panel f e l t should accompany any n o n - p r e s c r i p t i o n medication suggested. The d i s t r i b u t i o n of behaviours i n these subsets, over the four problems, i s o u t l i n e d i n Table IX. From the data i n Table IX, one can see that the two " c o l d " problems and the two "pain" problems are approximately p a r a l l e l i n terms of both the number and nature of items w i t h i n each subset. See page 119. -47-TABLE IX THE DISTRIBUTION OF THE FOUR SUBSETS OF BEHAVIOURS IN THE IN-STORE-ASSESSMENT PROBLEMS IN-STORE- NUMBER OF BEHAVIOURS IN EACH CATEGORY ASSESSMENT DATA INAPPROPRIATE APPROPRIATE DRUG-USE-PROBLEMS GATHERING RECOMMENDATIONS RECOMMENDATIONS COUNSELLING 1. COLD APPROPRIATE 8 a 7 b 2 C 4 d 2. COLD INAPPROPRIATE 8 7 2 4 3. PAIN APPROPRIATE 7 7 2 4 4. PAIN INAPPROPRIATE 7 7 2 4 a. The "data gathering" behaviours are ident ica l within the cold and with in the pain problems. b. The majority of " inappropriate recommendations" are i d e n t i c a l , some however are problem dependent. c. "Appropriate recommendations" are problem dependent and, therefore, not the same in any problem. d. "Dr-ug-use-counselling" behaviours are the same in a l l four problems. Scoring System Assigned to the ISAPs Agreements among the judges about appropriate weights for each item, were used to sort the items into "des i rab le " and "undesirable" act ions. A r b i t r a r i l y , those with a mean rat ing greater than 3.20 were c l a s s i f i e d as desirable actions and those with a mean rat ing of less than 2.80 were c l a s s i f i e d as undesirable act ions. Log ica l ly a l l items in the "data gathering", "appropriate recommendations" and "drug-use-counsell ing" subsets were desirable a c t i v i t i e s and a l l items in the " inappropriate recommendations" subset were undesirable a c t i v i t i e s . Af ter sort ing the items, a standardized scoring system was used for a l l four ISAPs. A value of 1 was assigned to a l l behaviours in the -48-"data gathering" and "drug counsel l ing" sections. A value of -1 was given to a l l behaviours in the " inappropriate recommendations" sect ion. For the "appropriate recommendations" sect ion, the fol lowing values were assigned to the behaviours, a 2 to the f i r s t item and a 1 to the second item. The f i r s t item represented the ideal recommendation and the second item represented an acceptable a l ternat ive given the symptoms, drugs being used, etc. The overal l performance scores were obtained by summing the values assigned to the behaviours performed. The maximum score for cold problems was 14 and for the pain problems was 13. Observers Eight graduate students in adult education were trained as the observers who were to present the ISAPs. Of the eight ind iv idua l s , s ix were female and two were male and the i r ages ranged from 31 to 53, with a mean age of 38. Each observer was given approximately four hours of t ra in ing . This consisted of ro le playing in which they worked in pairs and took the ro le of the consumer and then the ro le of the pharmacist for each ISAP s i tua t i on . This was repeated un t i l they f e l t comfortable with the i r ro le s . Each indiv idual had to play s a t i s f a c t o r i l y a l l four s i tuat ions fo r the invest igator. The observers were then assigned the name of a community pharmacist in Vancouver in.which to p i l o t - t e s t one of the s i tuat ions . At a subsequent meeting any problems encountered during the p i l o t - t e s t were discussed. The observers watched four videotapes of simulated consumer-pharmacist interact ion concerning a primary care request. These -49-videotapes were developed around s i tuat ions ident ica l to the four ISAPs. After viewing each videotape, the trainees completed the corresponding performance check l i s t for that ISAP. In add i t ion, for each videotape, the trainees were asked related questions such as "How many customers did you see?" or "How many indiv iduals were in the dispensary?" Each i nd i v i dua l ' s check l i s t and answers to the aux i l i a r y questions were compared with what actua l ly occurred in the videotape and the i n t e r -rater r e l i a b i l i t y for each s i tuat ion was ca lcu lated. R e l i a b i l i t y of the Observers The mean percentage agreement among the eight observers to what actua l ly occurred on each of the four videotapes was 85.50, 74.57, 66.71 and 74.59 and for the 103 items in a l l four videotapes, the mean agree-ment was 75.51 percent (Table XVII). Approximately 75 percent agreement among the observers i s somewhat low, however, the invest igator was confident that these percentages would have been considerably higher had there been better qua l i ty audio on the videotapes. Therefore, these f indings were accepted as evidence that the observers were accurate and dependable in the i r observations. Of the eight observers, four were used in both the pre- and post-course assessment. Two observers who co l lected pre-course data were unavailable for post-course data c o l l e c t i o n . Two observers were trained to take the i r places. For the s ix observers who co l lected the pre-course data there was a mean of 77.49 percent agreement with the four videotapes and for the s ix observers who co l lected the post-course data there was a mean of 75.06 percent agreement. There i s no TABLE XVII ACCURACIES OF EIGHT OBSERVERS FOR EACH OF FOUR VIDEOTAPES USED IN THE TRAINING PROCEDURE OBSERVER ALL FOUR , VIDEOTAPES (103)' COLD APPROPRIATE VIDEOTAPE (25) COLD INAPPROPRIATE VIDEOTAPE (27) PAIN APPROPRIATE VIDEOTAPE (25) PAIN INAPPROPRIATE VIDEOTAPE (26) 1 78.53 100.00 79.66 70.01 67.70 2 76,93 87.06 75.00 71.40 73.19 3 76.02 73.85 54.77 76.28 100.00 4 79.82 61.24 88.83 73.60 83.75 5* 78.12 100.00 79.72 76.13 60.31 6* 75.57 100.00 78.11 50.71 79.06' 7** 70.00 88.85 68.47 54.24 69.53 3** 69.11 73.85 72.06 61.38 63.25 Mean 75.51 85.60 74.57 66.71 74.59 .1 -* _ * * _ Number of items considered Pre-course data co l l ec t i on only Post-course data co l l ec t i on only TABLE XVIII INTER-OBSERVER RELIABILITY ON ALL 103 ITEMS OF THE FOUR VIDEOTAPES OBSERVER 1 2 3 4 5 6 7 . 8 1 .8141 .6690 .7445 .7401 .8055 .6679 .7983 2 .6131 .6836 .7372 .7567 .7543 .8225 3 .7325 .6565 .6133 .6012 .6654 4 .7928 .7220 .5828 .6797 5 .7502 .7404 .7277 6 .7263 .6805 7 .7182 8 Mean cor re lat ion coe f f i c i en t = 0.7119 -52-s i gn i f i can t difference between the two means. I t was concluded that each group of observers would be equally accurate in the i r observations. The i n te r - r a te r r e l i a b i l i t y was calculated for the eight observers over the ent i re 103 questions.asked about the videotapes (Table XVII). The mean corre lat ion coe f f i c i en t was 0.7110, and was interpreted as acceptable inter-observer r e l i a b i l i t y . Written Simulations Development One of the objectives of th i s study was to develop wr itten simulations which would be suitable for evaluating a pharmacist's performance in the area of primary care consult ing. Four simulations matching the four ISAPs were prepared by the invest igator. Two dealt with consumers asking for assistance on a " co ld " problem and two dealt with a "pain" problem. The simulations were constructed so that the symptoms, drug complications and recommended solutions approximated, as c lose ly as poss ible, the ISAPs described previously. This matching of the simulations and the ISAPs i s outl ined in Table X. The contents of the wr itten simulations and ISAPs were matched to enable a test of the predict ive v a l i d i t y of the simulations. -53 -TABLE X MATCHING OF WRITTEN SIMULATIONS TO THE IN-STORE-ASSESSMENTS (ISAPs) CONTENT WRITTEN SIMULATION IN-STORE-ASSESSMENT PROBLEM PHYSICIAN CONSULTATION 1. "COLD" ONE ONE NOT WARRANTED TWO TWO WARRANTED 2. "PAIN" THREE THREE NOT WARRANTED FOUR FOUR WARRANTED The si m u l a t i o n s used the l a t e n t image technique which has been described by McGuire ( 4 ) . With the l a t e n t image format, the response to the questions asked and the consequences of ac t i o n s taken are u s u a l l y on a separate sheet of paper and are i n v i s i b l e . They are developed with the use of a s p e c i a l pen. The information gained from t a k i n g a s p e c i f i c a c t i o n and developing the l a t e n t image response enables the i n d i v i d u a l to proceed with s o l v i n g the problem. The general p r i n c i p l e s and procedures described i n a manual f o r the preparation of w r i t t e n s i m u l a t i o n s were followed ( 3 ) . The simu l a t i o n s employed a branching format which allowed an i n d i v i d u a l to f o l l o w one of several routes to a s o l u t i o n . Procedure f o r Rating the Simulations Written s i m u l a t i o n s l i s t i n g the options a v a i l a b l e to the pharma-c i s t and a l l of the p o s s i b l e responses were submitted to the panel of judges. The p a n e l i s t s were asked to e d i t the si m u l a t i o n s f o r correctness -54-of content and to suggest ways of improving t h e i r r e a l i s m . They were asked to r a t e each option according to the c r i t e r i a employed f o r the ISAPs o u t l i n e d i n Table I. At a second meeting, any problems encountered i n t h i s process were discussed and weights were assigned to any new options suggested. The same values used f o r weighting the assigned c a t e g o r i e s i n ISAPs were used with the si m u l a t i o n s (1 through 5 values f o r Must not Do -Must Do). The mean, standard d e v i a t i o n , range, i n t e r q u a r t i l e range (Q value) were c a l c u l a t e d . P a n e l i s t s were contacted by phone to r e s o l v e any ambiguities about an option with a Q value exceeding 1.50. Pearsonian c o r r e l a t i o n s were generated to represent the extent of agree-ment among p a n e l i s t s . The mean of the judges' r a t i n g was used to assign a s c o r i n g weight to each option i n a s i m u l a t i o n (Table X I ) . TABLE XI THE MANNER IN WHICH SCORING WEIGHTS WERE ASSIGNED TO THE OPTIONS IN THE SIMULATIONS MEAN RATING 1.00 - 1.50 SCORING WEIGHT ASSIGNED -2 1.51 2.50 -1 2.51 3.50 0 3.51 4.50 4.51 5.00 2 An optimal route through each s i m u l a t i o n was devised i n c o r p o r -a t i n g suggestions made by the panel o f judges. The maximum score -55-obtainable by fol lowing the optimal route was calculated for scoring purposes. Samples of the four wr itten simulations, including a l l the responses contained in the latent images, and outl ines of the possible paths through each problem are shown in Appendix C. Va l i d i t y of the Simulations The means and standard deviations of the judges' ratings for the four simulations are contained in Table XII. As w e l l , th i s table con-tains the means and standard deviations of the judges' ratings of the tota l 267 options in a l l four simulations. A majority (at least s i x out of ten) of the judges agreed on the ratings of 127 of the 267 items (47.56 percent). In add i t ion, a l l ten judges agreed on the ratings of 15 (5.61 percent) of the items. In over 50 percent of the items at least 6 of the judges assigned the same weight. -56-TABLE XII MEANS AND STANDARD DEVIATIONS (S.D.) OF THE JUDGES' WEIGHTINGS* OF THE OPTIONS IN THE VARIOUS SIMULATIONS A l l Cold Cold Pain Pain Simulations Appropriate Inappropriate Appropriate Inappropriate Combined  Judge Mean S.D. Mean S.D. Mean S.D. Mean S.D. Mean S.D. 1. 3.29 1.61 3.19 1.59 3.25 1.46 3.02 1.53 3.19 1.54 2. 3.31 1.25 3.39 1.18 3.13 1.16 3.02 1.28 3.23 1.22 3. 2.84 1.38 2.85 1.36 3.32 1.27 3.38 1.30 3.06 1.35 4. 3.06 1.51 3,33 1.02 2.95 1.47 3.18 1.36 3.13 1.36 5. 3.05 1.13 3.36 1.01 3.13 1.23 3.09 1.26 3.15 1.15 6. 2.96 1.60 3.30 1.48 3.21 1.33 3.15 1.46 3.14 3.48 7. 3.00 1.48 3.21 1.44 2.95 1.50 3.16 1.49 3.07 1.47 8. 3.12 1.48 3.23 1.20 2.95 1.34 2.88 1.40 3.05 1.36 9. 2.78 1.26 3.24 0.80 2.93 1.23 3.07 0.98 2.99 1.10 10. 2.89 1.63 3.15 1.72 3.03 1.74 2.79 1.83 2.97 1.73 *5 = Must Do, 4 = Should Do, 3 = Could Do, 2 = Should Not Do, 1 = Must Not Do. Sedlacek and Nattress (5) f e l t t hat an estimate of the v a l i d i t y of w r i t t e n s i m u l a t i o n s could be obtained by c a l c u l a t i n g i n t e r - j u d g e agreement. In a d d i t i o n , they suggested that i f there was a high degree of agreement among the judges then weights could be assigned to the response options based on the mean judgements of the experts f o r each o p t i o n . I f such a sc o r i n g system was used, a t o t a l score f o r each i n d i v i d u a l completing the s i m u l a t i o n could be c a l c u l a t e d . The i n t e r - j u d g e agreement on the weightings of options f o r the r e s p e c t i v e s i m u l a t i o n s are di s p l a y e d i n Table X I I I . The average c o r r e l a -t i o n c o e f f i c i e n t s 0.78, 0.68, 0.73, and 0.74 c a l c u l a t e d f o r the four -57-sim u l a t i o n s represent a high l e v e l of i n t e r - j u d g e of agreement. The average c o r r e l a t i o n c o e f f i c i e n t f o r the judges' r a t i n g s of the e n t i r e 267 options i n a l l f o u r s i m u l a t i o n s was 0.73 (Table XIV Appendix C). -58-TABLE XIII INTERCORRELATIONS OF JUDGES RATINGS OF THE BEHAVIOURS IN SIMULATIONS ONE TO FOUR Judge 1 2 3 4 5 6 7 8 9 10 S imu l a t i on * 1 One Two Three Four .7634 .7892 .7342 .8701 ,8083 .8843 ,8245 .8387 .8549 .7193 .7438 .8202 .7995 .7328 .7485 .7977 .8224 .7818 .6295 .6683 .8899 .7981 .7965 .8555 .7276 .5228 .7745 .7446 .7949 .6299 .6470 .6906 .8008 .8276 .7147 .8054 2 One Two Three Four .7158 .7528 .6941 .7895 .7183 .6850 .7735 .8195 .7694 .7710 .8108 .8123 .7889 .7495 .5649 .6383 .7662 .6822 .7656 .8756 .6899 .5002 .6876 .7159 .6991 .6873 .7179 .7285 .8195 .8032 .7790 .7750 3 One Two Three Four .8679 .6785 .7152 .7140 .8202 .6667 .7199 .7722 .7755 .7602 .5939 .6984 .8144 .7261 .7468 .7804 .6421 .5898 .6732 .7231 .8464 .5897 .6227 .7423 .7276 .7549 ,7138 .7287 4 One Two Three Four .8086 .8096 .8417 .7714 .8018 .6273 .5622 .6822 .8172 .7251 .9463 .8407 .6644 .5239 .7718 .7243 .8474 .6080 .7915 .6855 .7888 .6469 .7423 .6875 5 One Two Three Four .7681 .5971 .5335 .6073 .8280 .6865 .8777 .7954 .7656 .6187 .7672 .6765 .7997 .6802 .8579 .6618 .8267 .6935 .7831 .6678 6 One Two Three Four .7995 .7175 .6019 .7113 .6648 .5843 .7470 .7297 .8109 .6187 .6296 .7653 .8585 .8624 .6022 .7196 7 One Two Three Four .7064 .5735 .8392 .6649 .7842 .5478 .8204 .7041 .7896 .7401 .7583 .7877 8 9 One Two Three Four One Two Three Four *S imulat ion S imulat ion S imulat ion S imulat ion One Two Three Four = Cold Appropr iate = Cold Inappropriate = Pain Appropr iate = Pain Inappropriate .7065 .5626 .8136 .7283 .7478 .6407 .7020 .6501 .8205 .5626 .6816 .6661 10 One Two Three Four -59-From the data presented i n these t a b l e s , i t was f e l t t h a t there was an acceptable l e v e l of agreement among the ten judges. T h i s , according to Sedaleck and Nattress ( 5 ) , provides an estimate of the v a l i d i t y of the four s i m u l a t i o n s and allowed weights to be assigned to the options w i t h i n the s i m u l a t i o n s to enable f u r t h e r v a l i d i t y t e s t i n g . More d i r e c t evidence of the v a l i d i t y of the s i m u l a t i o n s i s presented i n the form of c r i t e r i o n group v a l i d i t y data (Table XV). Each problem was administered to four groups of i n d i v i d u a l s who had varying degrees of knowledge about the use of n o n - p r e s c r i p t i o n drugs. The groups were: f i r s t year psychology students (N=118), f i r s t year pharmacy students (N=107), f i n a l year pharmacy students (N=93) and p r a c t i s i n g pharmacists (N=87). I f the problems have c r i t e r i o n group v a l i d i t y , the f i r s t year psychology students should score low and the f i n a l year pharmacy students and/or the p r a c t i s i n g pharmacists should score high. In a l l four problems, the trend was, as one would expect, psychology students scored the lowest and the f o u r t h year students or the p r a c t i s i n g pharmacists scored the highest. In only one problem, number three, d i d the f i r s t year pharmacy students score higher than 2 the f o u r t h year pharmacy students. The data i n Table XV. i n d i c a t e s t h a t the scores derived from the s i m u l a t i o n s do vary as expected with d i f f e r e n t l e v e l s of e x p e r t i s e as a pharmacist. Table XVI presents the t - p r o b a b i l i t i e s f o r s i g n i f i c a n t d i f f e r e n c e s ^This was an a r t i f a c t due to an e r r o r i n the d i r e c t i o n s w i t h i n problem three which prevented an i n d i v i d u a l from reaching the optimal s o l u t i o n . As the f o u r t h year students were the f i r s t t e s t group to complete the s i m u l a t i o n s t h e i r scores would be low on t h i s problem. I t was c o r r e c t e d f o r subsequent groups. T A B L E XV C R I T E R I O N GROUP V A L I D I T Y DATA FOR W R I T T E N S I M U L A T I O N P R O B L E M S S i m u l a t i o n O n e 3 MPS = 4 2 . 0 e S i m u l a t i o n MPS = 3 6 . T w o b , 0 S i m u l a t i o n M P S = 3 4 . T h r e e c • 0 S i m u l a t i o n F o u r d MPS = 2 7 . 0 G r o u p s M e a n S t a n d a r d D e v i a t i o n M e a n S t a n d a r d D e v i a t i o n M e a n S t a n d a r d D e v i a t i o n M e a n S t a n d a r d D e v i a t i o n P s y c h o l o g y 1 0 0 S t u d e n t s 1 3 . 3 5 ( 2 9 ) f 6 . 2 2 7 . 0 7 ( 3 0 ) 1 0 . . 6 8 6 . 9 7 ( 3 0 ) 7 , . 6 8 1 5 . 5 5 ( 2 9 ) 6 . 3 8 F i r s t Y e a r P h a r m a c y S t u d e n t s 1 7 . 9 0 ( 2 9 ) 9 . 6 9 1 2 . 4 7 ( 3 2 ) 11 . 9 3 1 5 . 3 6 ( 2 8 ) 7 . . 8 8 1 5 . 6 1 ( 2 8 ) 7 . 3 7 F o u r t h Y e a r P h a r m a c y S t u d e n t s 2 2 . 8 6 ( 2 2 ) 9 . 4 0 1 6 . 9 6 ( 2 4 ) 1 0 . 6 2 1 2 . 8 5 ( 2 6 ) 9 . 6 0 1 9 . 5 2 ( 2 1 ) 3 . 1 1 G r a d u a t e P h a r m a c i s t s S 2 3 . 4 5 ( 1 2 ) ( 1 0 ) 8 . 8 5 2 3 . 0 7 ( 1 4 ) ( 1 0 ) 7. . 6 5 1 9 . 3 4 ( 1 0 ) ( 1 1 ) 7 . 5 6 1 8 . 3 4 ( 1 2 ) ( 8 ) 4 . 8 3 a C o l d a p p r o p r i a t e , i . e . , a p p r o p r i a t e t o r e c o m m e n d a n o n - p r e s c r i p t i o n p r o d u c t b C o l d i n a p p r o p r i a t e , i . e . , i n a p p r o p r i a t e t o r e c o m m e n d o n l y a n o n - p r e s c r i p t i o n p r o d u c t c P a i n a p p r o p r i a t e , i . e . , a p p r o p r i a t e t o r e c o m m e n d a n o n - p r e s c r i p t i o n p r o d u c t d P a i n i n a p p r o p r i a t e , i . e . i n a p p r o p r i a t e t o r e c o m m e n d o n l y a n o n - p r e s c r i p t i o n p r o d u c t . e M a x i m u m p o s s i b l e s c o r e f N u m b e r s i n p a r e n t h e s i s i n d i c a t e s i n d i v i d u a l s c o m p l e t i n g p r o b l e m s g M e a n o f t w o g r o u p s o f p r a c t i s i n g p h a r m a c i s t s -61-i n the performances of the four groups on the w r i t t e n s i m u l a t i o n s . The scores of the psychology students are s i g n i f i c a n t l y lower than the f o u r t h year pharmacy students on a l l four problems . As w e l l , t h e i r scores were s i g n i f i c a n t l y lower on three of the four problems when compared to those of the graduate pharmacists. There are s i g n i f i c a n t d i f f e r e n c e s between the performances of the f i r s t year pharmacy students and those with more education and experience i n pharmacy. The data presented i n Tables XV and XVI give a d d i t i o n a l support to the v a l i d i t y of the four s i m u l a t i o n s . TABLE XVI t-PROBABILITIES FOR SIGNIFICANT DIFFERENCE IN PERFORMANCE OF CRITERION GROUPS ON WRITTEN SIMULATIONS F i r s t Year Fourth Year Graduate Group Pharmacy Students Pharmacy Students Pharmacists Simulations Psychology One 0.055 0.000* 0.000* 100 s t u - Two 0.063 0.001* 0.000* dents Three 0.000* 0.012* 0.000* Four 0.926 0.006* 0.117 F i r s t Year One 0.056 0.036* pharmacy Two 0.146 0.000* students Three 0.305 0.067 Four 0.015* 0.141 Fourth Year One 0.802 pharmacy Two 0.018* students Three 0.015* Four 0.295 * S i g n i f i c a n t at 0.05 l e v e l . .1. S i mulation one = c o l d a p p r o p r i a t e , two = c o l d i n a p p r o p r i a t e , three pain appropriate and four = pain i n a p p r o p r i a t e ; -62-Tests For Factual Knowledge To measure the gain i n f a c t u a l knowledge by the i n d i v i d u a l s who p a r t i c i p a t e d i n the continu i n g pharmacy education programs, t e s t s were prepared. A l l the i n s t r u c t o r s involved i n the program were asked to prepare i n s t r u c t i o n a l o b j e c t i v e s (2) before designing t h e i r presenta-t i o n s . The i n s t r u c t o r s f o r " c o l d " and "pain" were to choose those o b j e c t i v e s . t h e y f e l t most important and prepare t e s t s f o r f a c t u a l knowledge on the content. The t e s t s c o n s i s t e d mostly of m u l t i p l e choice questions but some included true or f a l s e questions. Two forms of each t e s t were developed, a pre-and a p o s t - t e s t . Both t e s t s contained the same questions but i n d i f f e r e n t sequences, and with d i f f e r e n t orderings of a l t e r n a t i v e s . These t e s t s were pre-tested on pharmacists e n r o l l e d i n s i m i l a r c o n t i n u i n g education programs i n other l o c a l i t i e s i n the province. These r e s u l t s were analyzed to i d e n t i f y the questions with the best d i s c r i m i n a t i n g powers. Those items which were found to be the most "l e a r n a b l e " and to have the highest p o i n t b i - s e r i a l c o r r e l a t i o n with t e s t content were r e t a i n e d f o r the f i n a l v e r s i o n of the t e s t s (see Appendix D). P a r t i c i p a n t s ' E v a luation of the Program A magnitude es t i m a t i o n s c a l i n g technique was used to a s c e r t a i n the p a r t i c i p a n t s ' r e a c t i o n to the course (1). P a r t i c i p a n t s were asked to compare aspects of t h i s course with ones attended i n the past. As w e l l , they were asked to estimate the amount of l e a r n i n g a t t r i b u t a b l e to . components of t h i s course i n comparison to that a t t r i b u t a b l e to a -63-standard 30 minute l e c t u r e . This e v a l u a t i o n form i s shown i n Appendix E. -64-References Chapter Three 1. B l u n t , A., "The Construction of a Magnitude Estimation Scale o f Ad u l t Learning", paper presented a t the 1976 A.E.A., Adul t Educa-t i o n Research Conference, Toronto, Canada, 7-9 A p r i l 1976. 2. Mager, R.F., Preparing I n s t r u c t i o n a l O b j e c t i v e s . Belmont, C a l i f o r n i a : Fearon P u b l i s h e r s , 1962. 3. Manual f o r Northern I l l i n o i s Workshop: Construction of Written  S i m u l a t i o n s . Research and Eva l u a t i o n S e c t i o n , Center f o r Educa-t i o n a l Development, U n i v e r s i t y o f I l l i n o i s College of Medicine, Chicago, I l l i n o i s , 1973. 4. McGuire, C.H. and F.H. Wezeman, "Simulation i n I n s t r u c t i o n and Evaluation i n Medicine" i n Educational S t r a t e g i e s f o r the Health  P r o f e s s i o n s . Edited by G.E. M i l l e r and T. Fulop, P u b l i c Health Papers #61, World Health O r g a n i z a t i o n , Geneva, 1974, 18-34. 5. Sedlacek, W.E. and L.W. N a t t r e s s , "A Technique f o r Determination of the V a l i d i t y of P a t i e n t Management Problems." Journal of  Medical Education 47 (1972): 263-266. -65-CHAPTER FOUR METHODOLOGY The two p r i n c i p a l research questions addressed i n t h i s study were: (a) "Did a continu i n g education course change the behaviours of the i n d i v i d u a l s e n r o l l e d ? " and (b) "Did the w r i t t e n s i m u l a t i o n s developed f o r t h i s i n v e s t i g a t i o n p r e d i c t r e a l - l i f e behaviour?" In a d d i t i o n , the f o l l o w i n g secondary questions were i n v e s t i g a t e d : 1.) "Was there a r e l a t i o n s h i p between the scores on a f a c t u a l knowledge t e s t and performance on w r i t t e n s i m u l a t i o n s ? " 2.) "Was there a r e l a t i o n s h i p between the scores on a f a c t u a l knowledge t e s t and performance on the in-store-assessments?" 3.) "What were the rea c t i o n s of the p a r t i c i p a n t s to the educational program?" This chapter describes the methodology employed i n t h i s study's approach to these questions. Design of the Study To answer the p r i n c i p a l questions, a m o d i f i c a t i o n of the qua s i -experimental research design known as the "non-equivalent c o n t r o l group" was adopted (1). In t h i s design there i s an experimental group and a co n t r o l group, but "the c o n t r o l group and the experimental group do not have pre-experimental sampling equivalence" ( l , p. 47). This design i s represented s c h e m a t i c a l l y i n Figure 1. To answer the f i r s t q u e s t i o n , the educational program was the independent v a r i a b l e and the primary care c o n s u l t i n g behaviour was the dependent v a r i a b l e . This design enabled a measurement of the i n f l u e n c e -66-of p a r t i c i p a t i o n i n the c o n t i n u i n g education program on the primary care c o n s u l t i n g behaviour of the pharmacist p a r t i c i p a n t s . To answer the second p r i n c i p a l q u e s t i o n , the r e l a t i o n s h i p between the performance on the matching ISAP and the w r i t t e n s i m u l a t i o n was i n v e s t i g a t e d . F i g . 1. Experimental Design o1 o2 x o3 o4 Where: X i s the i n s t r u c t i o n 0"i i s the pre-course in-store-assessment of the experimental group 02 i s the p r e - i n s t r u c t i o n w r i t t e n s i m u l a t i o n 03 i s the p o s t - i n s t r u c t i o n w r i t t e n s i m u l a t i o n 04 i s the post-course in-store-assessment of the experimental group 05 i s the pre-course in-store-assessment of the c o n t r o l group O5 i s the post-course in-store-assessment of the c o n t r o l group Data c o l l e c t e d during the educational program enabled a f u r t h e r i n v e s t i g a t i o n of the secondary questions o u t l i n e d i n the i n t r o d u c t i o n to t h i s chapter. Subjects The experimental subjects (N=34) were among the p a r t i c i p a n t s i n the c o n t i n u i n g education course held i n V i c t o r i a and Duncan, B.C. as part of the r e g u l a r c o n t i n u i n g education a c t i v i t i e s of the F a c u l t y of Pharmaceutical Sciences, U.B.C. A non-equivalent c o n t r o l group (N=40) was randomly s e l e c t e d -67-from the r e g i s t r a t i o n l i s t of the College of Pharmacists of B r i t i s h Columbia. Only pharmacists on record a t the College as working i n Vancouver, B.C. were incl u d e d . Pharmacists working i n the Chinatown area of the c i t y were excluded. As w e l l , any pharmacist who e i t h e r attended t h i s c o n t i n u i n g education program when o f f e r e d the previous f a l l or who worked i n a s t o r e with someone who p r e v i o u s l y attended the course, was excluded. They were el i m i n a t e d because they would have access to much of the course m a t e r i a l which could p o s s i b l y i n f l u e n c e t h e i r behaviour between the pre-course and post-course assessment. Of the 40 c o n t r o l subjects one was dropped from the study because she r e f e r r e d the observer to another pharmacist. Demographic Data on Subjects Demographic data were c o l l e c t e d and compared to determine equiva-lency between the groups of s u b j e c t s . The data were obtained from the f i l e s of the B r i t i s h Columbia College of Pharmacists and included age, sex, b i r t h d a t e , employee p o s i t i o n i n the pharmacy, degree or diploma s t a t u s , where and when obtained, a d d i t i o n a l u n i v e r s i t y q u a l i f i c a t i o n s , a d d i t i o n a l q u a l i f i c a t i o n s other than u n i v e r s i t y , and Pharmacy Examining Board of Canada s t a t u s . In a d d i t i o n , whether an i n d i v i d u a l worked i n an independent or chain s t o r e and h i s or her membership i n pharmacy o r g a n i z a t i o n s were recorded. Continuing Education Program The c o n t i n u i n g education course was an evening l e c t u r e s e r i e s o f f e r e d i n Duncan, B.C. on Tuesday and on Wednesday i n V i c t o r i a . The same i n s t r u c t o r s gave both s e s s i o n s . The program c o n s i s t e d of two-hour -68-s e s s i o n s , one night per week, f o r e i g h t consecutive weeks. An overview of the program content i s presented i n Table XIX. TABLE XIX OUTLINE OF THE CONTENT OF PROGRAM BY SESSION Session Topic One A. Schedule A, Part I I I Drugs (N o n - p r e s c r i p t i o n drugs) B. A d v i s i n g P a t i e n t s - Ge t t i n g the f a c t s Two A. Ophthalmics, O t i c s B. Contact Lens S o l u t i o n s Three A. Antacids B. Dental Products Four A. Laxatives B. A n t i d i a r r h e a l Preparations Five A. Cough Medications S i x A. Cold Medications Seven A. Analgesics B. Sleep Aids Eight A. Liniments, Hemorrhoidal Preparations B. Sunscreen, Burn Products Part A of the f i r s t evening o u t l i n e d new l e g i s l a t i v e changes to the B.C. Pharmacy Act that increased the pharmacist's r e s p o n s i b i l i t y f o r c o n t r o l l i n g p u b l i c access to.;some n o n - p r e s c r i p t i o n medications. P a r t B, that evening, d e a l t s p e c i f i c a l l y with communication s k i l l s , such as e f f e c t i v e l i s t e n i n g and questioning s t r a t e g i e s . Evenings Two through. Eight i n v o l v e d d i s c u s s i o n s of common problems i n s e l e c t e d t h e r a p e u t i c areas which are oft e n brought to pharmacists f o r advice. Included i n each se s s i o n was a d i s c u s s i o n on the e t i o l o g y of common ai l m e n t s , the most -69-prevalent symptoms and the key questions to help d i f f e r e n t i a t e the seriousness of the complaints. These points were emphasized by the i n s t r u c t o r and by video-tapes of simulated consumer-pharmacist en-counters. The Program Emphasis The i n s t r u c t o r s were a l s o asked to emphasize the f o l l o w i n g l i s t o f recommended behaviours. In response to a primary care request the pharmacist w i l l : (1.) respond to the request himself or d i r e c t the consumer to another t r a i n e d health p r o f e s s i o n a l , (2.) obtain a b r i e f h i s t o r y before proceeding on any course of a c t i o n , (3.) i n c l u d e i n that h i s t o r y questions about the symptoms, (4.) ask about a l l e r g i e s to drugs, e t c . , (5.) ask about concurrent use of p r e s c r i p t i o n drugs, (6.) ask about concurrent use of n o n - p r e s c r i p t i o n drugs, (7.) recommend a product which i s compatible with f a c t s a s c e r t a i n e d i n steps three to s i x , (8.) when warranted recommend t h a t the i n d i v i d u a l seek p h y s i c i a n advice and not recommend a n o n - p r e s c r i p t i o n medication, (9.) provide precautions about r e l e v a n t side e f f e c t s of any medications recommended, (10.) suggest proper method f o r use (dosage, i n s t i l l a t i o n of drops, sprays, e t c . ) , -70-(11.) suggest the l i m i t a t i o n s of s e l f - m e d i c a t i o n ( t h a t i s , how long to take and when to see a p h y s i c i a n ) . Instruments The w r i t t e n simulations and the ISAPs discussed p r e v i o u s l y were employed to measure the above behaviours when a pharmacist responded to a request f o r a s s i s t a n c e on a " c o l d " problem and a "pain" problem. Tests f o r f a c t u a l knowledge were a l s o developed f o r the sessions on "pain" and "cold 1. 1. Instrument Assignment The ISAPs were assigned to experimental and c o n t r o l subjects i n the f o l l o w i n g manner: program p a r t i c i p a n t s were asked to p r e - r e g i s t e r ; and the f i r s t r e g i s t r a n t was assigned to problem one, the second to problem two, the t h i r d to problem three, the f o u r t h to problem f o u r , the f i f t h to problem one and so on. As the c o n t r o l group was s e l e c t e d , they were assigned problems according to the same system. However, there were some i n s t a n c e s , both i n the c o n t r o l , and the experimental groups, where two pharmacists worked a t the same pharmacy. In t h i s case, one would be given a " c o l d " problem and the other a "pain" problem to insure the unobtrusive nature of t h i s phase of the data c o l l e c t i o n . Every evening during the co n t i n u i n g education program, the p a r t i c i p a n t s received a personal r e g i s t r a t i o n k i t . Each night t h i s contained a p r e - t e s t f o r f a c t u a l knowledge and any l e c t u r e o u t l i n e s or other m a t e r i a l s f o r th a t evening's i n s t r u c t i o n . In order to assure t h a t i n d i v i d u a l s r e c e i v e d w r i t t e n s i m u l a t i o n s which corresponded to the pre-course ISAP on " c o l d " or "pain", matching si m u l a t i o n s were placed -71-i n the r e g i s t r a t i o n k i t s . The post-course w r i t t e n s i m u l a t i o n was mailed to each i n d i v i d u a l and matched the post-course ISAP used to assess the r e g i s t r a n t . Only the experimental subjects completed w r i t t e n s i m u l a t i o n s . Observers Eight graduate students were employed as the observers f o r the in-store-assessments. They were not t o l d which pharmacists comprised the c o n t r o l and experimental groups and the same observer was never u t i l i z e d to do both the pre-and post-course assessment of any one pharmacist. The d i r e c t i o n s given the observers f o r the in-store-assessments are contained i n Appendix A. Procedure f o r Data Gathering The data c o l l e c t e d f o r both the experimental and c o n t r o l groups included pre-and post-course in-store-assessments and demographic data r e l a t i n g to an i n d i v i d u a l ' s career i n pharmacy. For the experimental group o n l y , the f o l l o w i n g a d d i t i o n a l data were c o l l e c t e d : pre-and post-i n s t r u c t i o n t e s t s f o r f a c t u a l knowledge, pre-and p o s t - i n s t r u c t i o n per-formance on w r i t t e n s i m u l a t i o n s and a s u b j e c t i v e e v a l u a t i o n of the ed-ucation program. An o u t l i n e of the sequence of data c o l l e c t i o n i s pre-sented i n Figure 2. The pre-course in-store-assessments.were performed on the experimental group as the i n d i v i d u a l s p r e - r e g i s t e r e d . This group was assessed between February 27th and March 17th, 1976. Of the 34 experimental s u b j e c t s , 28 p r e - r e g i s t e r e d and were assessed before being exposed to any i n s t r u c t i o n . Four i n d i v i d u a l s , who d i d not p r e - r e g i s t e r , attended the f i r s t evening and were assessed between the f i r s t and -72-Figure 2. Schematic Representation of Data C o l l e c t i o n B P PRE-COURSE p Q IN-STORE-ASSESSMENT 0 G R R E A M Session One Pre-Test I n s t r u c t i o n Post-Test Two Pre-Test I n s t r u c t i o n Post-Test Three Pre-Test I n s t r u c t i o n Post-Test P Four : Pre-Test R I n s t r u c t i o n 0 Post-Test Q R Five Pre-Test ^ I n s t r u c t i o n •YI Post-Test P r a c t i c e S imulation S i x Pre-Test Simulation I n s t r u c t i o n Post-Test Seven Pre-Test Simulation I n s t r u c t i o n Post-Test E i g h t Pre-Test I n s t r u c t i o n Post-Test Evaluation P P 0 R POST-COURSE S 0 SIMULATIONS T G R A M POST-COURSE IN-STORE-ASSESSMENT -73-second week. Two i n d i v i d u a l s , who d i d not p r e - r e g i s t e r , attended the second evening and were assessed between the second and t h i r d week. The scores of these s i x i n d i v i d u a l s were we l l w i t h i n the range of the other p a r t i c i p a n t s ' scores and so were included i n the f i n a l sample. The c o n t r o l group was assessed between February 25th and March 19th, 1976. A l l assessments were performed i n an unobtrusive manner, with the t r a i n e d observers posing as r e g u l a r customers. In these assessments, the observers were i n s t r u c t e d to make sure the assigned pharmacist was on duty and to v e r i f y the i d e n t i t y by having one of the c l e r k s i d e n t i f y him or her before a d m i n i s t e r i n g the problem. A l l the s i t u a t i o n s con-cerned a f r i e n d or r e l a t i v e and the observers were to act concerned but not worried. They were to answer any of the pharmacist's questions with the information provided with each problem. They were not to volunteer any i n f o r m a t i o n . They were to buy the recommended product and leave the store and complete the performance c h e c k l i s t (Appendix A). The pharma-c i s t ' s performance was scored a t a l a t e r date by the i n v e s t i g a t o r . In a d d i t i o n to the performance c h e c k l i s t , s i t u a t i o n a l data^" and a 2 phy s i c a l d e s c r i p t i o n of the pharmacist were recorded by the observer i n c l u d i n g : sex, height, weight, age, h a i r c o l o u r , b u i l d , e t c . ; length of time spent with the pharmacist; whether he/she appeared busy; the number of pharmacists on duty, number of c l e r k s on duty; number of customers a t The s i t u a t i o n a l data were recorded to r u l e out environmental f a c t o r s which might account f o r a pharmacist's performance. 2 This enabled the i n v e s t i g a t o r to double check the i d e n t i t y of the pharmacist assessed. -74-the p r e s c r i p t i o n counter, i n the whole s t o r e , at cash r e g i s t e r one and at cash r e g i s t e r two. They were a l s o to complete two semantic d i f f e r -e n t i a l s c a l e s d e s c r i b i n g the pharmacist's treatment of them as a person and of t h e i r n o n - p r e s c r i p t i o n medication request. (See Appendix A f o r copy of t h i s record sheet.) During the educational program i t s e l f , a d d i t i o n a l data were gathered on the experimental group. Each evening of the c l a s s the pharmacist completed the pre-and p o s t - t e s t s f o r f a c t u a l knowledge. These were s e l f - g r a d e d by the pharmacist and handed i n at the end of the evening. In the i n t e r v e n i n g week they were checked to see t h a t they had been marked properly and analyzed to produce a d i s t r i b u t i o n of scores and a mean and standard d e v i a t i o n f o r each set of t e s t s . To acquaint p a r t i c i p a n t s with w r i t t e n s i m u l a t i o n s and the l a t e n t image format, on the f i f t h evening, as the l a s t task, they were given a p r a c t i c e s i m u l a t i o n . On the subsequent two evenings, they completed si m u l a t i o n s on " c o l d " and "pain" which were d i s t r i b u t e d i n such a manner ( v i a the r e g i s t r a t i o n k i t s ) that the s i m u l a t i o n received matched the pre-course ISAP. As the l a s t a c t i v i t y of the course (evening e i g h t ) the p a r t i c i p a n t s completed a course e v a l u a t i o n form (see Appendix E). On May 19th, 1976, two post-course s i m u l a t i o n s were mailed to each pharmacist r e g i s t e r e d i n the course, one of which matched the post -course ISAP. This m a i l i n g produced a s i x t y percent r e t u r n . A second m a i l i n g was conducted on May 31, 1976. This was followed i n two weeks by a telephone c a l l to a l l non-respondents. A t o t a l of 32 out of a p o s s i b l e 34 returns were obtained. Two of the returned s i m u l a t i o n s -75-were unusable because the i n d i v i d u a l s developed more than the appropriate number of l a t e n t images. The post-course i n store assessments f o r the experimental group 3 were performed between June 7th and June 25th, 1976. The c o n t r o l group 4 was assessed between May 31st and June 29th, 1976/ For both the pre-and post-in-store^-assessments, the t o t a l t e s t score, i t s four component sub scores, and the s i t u a t i o n a l data were analyzed f o r s i g n i f i c a n t gains and r e l a t i o n s h i p s . Data A n a l y s i s The data were analyzed on the IBM 360 computer a t the U n i v e r s i t y of B r i t i s h Columbia using programs appropriate a t each stage. The demographic data were analyzed by preparing b i v a r i a t e frequency t a b l e s and c a l c u l a t i n g Pearson's Chi square as a t e s t of s i g n i f i c a n c e f o r the d i s t r i b u t i o n of cases w i t h i n the c e l l s of the t a b l e s . This permitted an examination of the equivalency of the c o n t r o l and experimental groups. The mean and standard d e v i a t i o n s o f the m u l t i p l e choice t e s t scores were c a l c u l a t e d . In a d d i t i o n , t - t e s t s f o r paired comparisons were performed to determine the s i g n i f i c a n c e of the gain between the pre-and p o s t - t e s t scores. The mean and standard d e v i a t i o n s were c a l c u l a t e d f o r the pharmacists' t o t a l performance scores on each s i m u l a t i o n . The one exception was on J u l y 20th, 1976 f o r an i n d i v i d u a l who was on vacation during the p e r i o d . A "Four i n d i v i d u a l s , who were on v a c a t i o n , were assessed between J u l y 6th to 8 t h , 1976. -76-The s i g n i f i c a n c e of the gain between the pre-and post-course i n -store-assessments were determined by performing the t - t e s t f o r paired comparisons. The extent of a s s o c i a t i o n between (a) the s i t u a t i o n a l f a c t o r s present during the in-store-assessment with an i n d i v i d u a l ' s ISAP per-formance score, (b) the scores f o r f a c t u a l knowledge with performance on the ISAPs, and (c) the scores f o r f a c t u a l knowledge with performance on w r i t t e n s i m u l a t i o n s were determined by c a l c u l a t i n g Pearsonian c o r r e l a t i o n s . To i d e n t i f y the r e l a t i o n s h i p between performance on the ISAP and performance on the w r i t t e n s i m u l a t i o n s , Pearsonian c o r r e l a t i o n s were c a l c u l a t e d f o r the t o t a l performance scores. In a d d i t i o n , a consistency score representing the agreement between what an i n d i v i d u a l d i d or d i d not do i n r e a l - l i f e , with what he or she d i d or d i d not do on the w r i t t e n s i m u l a t i o n was c a l c u l a t e d . The score was determined by comparing an i n d i v i d u a l ' s performance on the items i n the ISAP w i t h the corresponding items i n the s i m u l a t i o n s . I f an i n d i v i d u a l performed an a c t i v i t y on a measurement, i t was recorded as a one. I f an i n d i v i d u a l d i d not perform an a c t i v i t y on a measurement, i t was recorded as a zero. A percentage was then c a l c u l a t e d by comparing the agreement between the ones and zeros on the corresponding items. -77-References Chapter Four 1. Campbell, D.T. and J.C. S t a n l e y , Experimental and Quasi-Experimental  Designs f o r Research, Chicago: Rand McNally College P u b l i s h i n g Company, 1963. -78-CHAPTER FIVE RESULTS AND DISCUSSIONS I n i t i a l l y , t h i s chapter presents data with respect to the equivalency of the two research groups. The remainder of the chapter i s devoted to an a n a l y s i s and d i s c u s s i o n of the data to answer the research questions st a t e d i n the previous chapter. Demographic Data f o r Controls and Experimental Groups To e s t a b l i s h the degree of "equivalency" between the c o n t r o l group and experimental group, demographic data were recorded on the i n d i v i d u a l s i n each group. (See Table XX.) -79-TABLE XX DEMOGRAPHIC COMPARISONS FOR CONTROL AND EXPERIMENTAL SUBJECTS VARIABLE PEARSON'S CHI-SQUARE CHI PROBABILITY Country of b i r t h 0.46 0.50 Province o f b i r t h 1.99 0.58 Sex 1.39 0.24 M a r i t a l Status 0.83 0.67 Employed i n more than one Pharmacy 1.79 0.41 Diploma 5.74* 0.02 Province where diploma obtained 7.80* 0.01 Degree 5.74* 0.02 Province where degree obtained 0.02 0.85 A d d i t i o n a l degrees 0.02 0.85 A d d i t i o n a l diplomas 0.67 0.42 A d d i t i o n a l q u a l i f i c a t i o n s 0.00 0.95 Pharmacy Examining Board of Canada Status 7.09* 0.03 Membership i n : B.C. Pharmaceutical S o c i e t y 2.42 0.12 Canadian S o c i e t y of Hosp i t a l Pharmacists 0.00 0.95 R e g i s t r a t i o n i n other provinces 1.46 0.22 Employment Status 3.17 0.38 Hours o f work per week 3.30 0.35 Type of Store 1 .07 0.59 * S i g n i f i c a n t The c o n t r o l group had s i g n i f i c a n t l y more people who recei v e d t h e i r l i c e n s e to p r a c t i c e pharmacy a f t e r completing a diploma course. More of these i n d i v i d u a l s received t h e i r diplomas from a province other than -80-B r i t i s h Columbia. There were s i g n i f i c a n t l y more i n d i v i d u a l s i n the experimental group who had received u n i v e r s i t y degrees i n pharmacy. As w e l l , a s i g n i f i c a n t l y l a r g e r number of pharmacists i n the experimental group have received t h e i r Pharmacy Examining Board of Canada C e r t i f i c a t e . Although the two groups were not randomly s e l e c t e d from the same po p u l a t i o n , the data i n Table XX i n d i c a t e d t h a t they were e q u i v a l e n t on a number of demographic v a r i a b l e s . For those v a r i a b l e s f o r which there were s i g n i f i c a n t d i f f e r e n c e s , none should a f f e c t an i n d i v i d u a l ' s a b i l i t y to l e a r n and th e r e f o r e could not be used to e x p l a i n away any gain i n e i t h e r of the group's performances on the in-store-assessments. In a d d i t i o n , t - t e s t s f o r s i g n i f i c a n t d i f f e r e n c e s between the two groups were performed on the year of b i r t h , the year of graduation wit h a diploma and the year of graduation w i t h a degree. There were no d i f f e r e n c e s s i g n i f i c a n t a t the 0.05 l e v e l of s i g n i f i c a n c e . Improvement i n R e a l - L i f e Performance In t h i s study four primary care requests were used as the pre-in-store-assessment t e s t s and the same four primary care requests were used as post-in-store-assessment t e s t s . Although an i n d i v i d u a l d i d not rec e i v e the same problem as both the pre- and p o s t - t e s t , he or she d i d re c e i v e the i d e n t i c a l problem type, i . e . , two c o l d problems or two pain problems. The maximum score one could o b t a i n with the two c o l d problems or.with the two pain problems i s the same. To t e s t f o r the equivalence of the two d i f f e r e n t ISAPs which d e a l t w i t h c o l d , the components' scores on the in-store-assessments were compared across the "cold"problems (Table XXI). The components of the - 8 1 -TABLE XXI COMPARISON OF THE COMPONENT SCORES FOR THE TWO COLD IN-STORE-ASSESSMENT PROBLEMS FOR THE EXPERIMENTAL AND CONTROL GROUPS EXPERIMENTAL GROUP PRE-IN-STORE-ASSESSMENT MEAN PERFORMANCE SCORES COMPONENTS APPROPRIATE INAPPROPRIATE t-VALUE D.F. t-PROB 1 F-PROB 2 1. DATA GATHERING 2 .25 2 .86 -0.93 20 0 .37 0.90 2. INAPPROPRIATE RECOMMENDATIONS -1 .00 -0, .86 ** 3. APPROPRIATE RECOMMENDATIONS 0. .00 0 .36 ** 4. DRUG-USE-COUNSELLING 0 .88 2 .15 -2.61 19 0, .02* 0.08 5. TOTAL SCORE 2 .12 4. .35 -1.96 20 0. .06 0.22 CONTROL GROUP PRE-IN-STORE-ASSESSMENT COMPONENTS 1 . DATA GATHERING 2, .23 1. .82 0.68 22 0. ,51 0.78 2. INAPPROPRIATE RECOMMENDATIONS -1 .23 -1. .00 ** 3. APPROPRIATE RECOMMENDATIONS 0, .16 0. .00 ** 4. DRUG-USE-COUNSELLING 0, .62 0. .82 -0.54 22 0. .60 0.26 5. TOTAL SCORE 1 . .77 1 , .55 0.24 22 0. .80 0.83 EXPERIMENTAL GROUP POST-IN-STORE-ASSESSMENT MEAN PERFORMANCE SCORES COLD COLD COMPONENTS INAPPROPRIATE APPROPRIATE t-VALUE D.F. t-PROB F-PROB 1. DATA GATHERING 3, .60 3. .14 0.61 20 0. .55 0.26 2. INAPPROPRIATE RECOMMENDATIONS -0. .75 -0. .85 -0.47 20 0, .65 0.73 3. APPROPRIATE RECOMMENDATIONS 0, .63 0. .71 -0.19 20 0. .83 0.39 4. DRUG-USE-COUNSELLING 1, .75 2, .43 -1 .02 20 0. .32 0.55 5. TOTAL SCORE 5. .25 5. .42 -0.11 20 0. .88 0.76 CONTROL GROUP POST-IN-STORE-ASSESSMENT COMPONENTS 1 . DATA GATHERING 1 , .69 2 .36 -1.01 22 0, .32 0.94 2. INAPPROPRIATE RECOMMENDATIONS -1, .00 -1. .09 -0.47 22 0 .65 0.94 3. APPROPRIATE RECOMMENDATIONS 0. .15 0. .09 0.33 22 0. .74 0.06 4. DRUG-USE-COUNSELLING 0. .46 1 , .09 -1 .18 22 0 .25 0.94 5. TOTAL SCORE 1 , .31 2. .36 -1 .10 22 0 .29 0.70 1. t - P r o b a b i 1 i t y . A t - p r o b a b i l i t y o f 0.05 o r l e s s r e p r e s e n t s a s i g n i f i c a n t d i f f e r e n c e . 2. F - P r o b a b i l i t y . A F - p r o b a b i l i t y o f 0.05 or l e s s i n d i c a t e s t h a t the sample v a r i a n c e s a r e s i g n i f i c a n t l y d i f f e r e n t and t h e r e f o r e come from d i f f e r e n t p o p u l a t i o n s . ** S i g n i f i c a n t d i f f e r e n c e s were n o t computed i n t h o s e i n s t a n c e s where p h a r m a c i s t s ' b e h a v i o u r s were so s i m i l a r as to produce no v a r i a n c e . * S i g n i f i c a n t d i f f e r e n c e . -82-in-store-assessments are "data gathering", " i n a p p r o p r i a t e recommendations", "appropriate recommendations", and "drug-use-counselling" which, when added together give an " o v e r a l l performance score". Only the "drug-use-c o u n s e l l i n g " scores f o r the experimental group i n the " c o l d " problems as pr e - t e s t s was s i g n i f i c a n t l y d i f f e r e n t at the 0.05 l e v e l of s i g n i f i c a n c e . For the c o n t r o l group, there were no s i g n i f i c a n t d i f f e r e n c e s i n the components of e i t h e r the pre- or p o s t - t e s t s across problems. These f i n d i n g s would suggest that the two " c o l d " ISAPs are equ i v a l e n t i n r e l a t i o n to t h e i r assessment of the l e v e l of a pharmacist's performance. Li k e w i s e , the components f o r "pain" problems were compared (Table X X I I ) . There were no s i g n i f i c a n t d i f f e r e n c e s , a t the 0.05 l e v e l , i n the components when used e i t h e r as p r e - t e s t or p o s t - t e s t . This was the case f o r both experimental and c o n t r o l groups and supported the p a r a l l e l nature of the "pain" ISAPs. For the purpose o f assessing the impact of the course on the performance of a pharmacist, the evidence c i t e d above suggests that i t was l e g i t i m a t e to c o l l a p s e the subjects i n t o two groups - those who received " c o l d " problems and those who received "pain" problems. There were 22 experimental subjects and 24 c o n t r o l subjects who received " c o l d " problems as t h e i r pre- and p o s t - t e s t s . There were 12 experimental sub-j e c t s and 15 c o n t r o l subjects who received "pain" problems as t h e i r pre-and p o s t - t e s t s . Furthermore, a l l 34 experimental subjects were t r e a t e d as one group and a l l 39 c o n t r o l subjects as another. The r a t i o n a l e f o r t h i s grouping was as f o l l o w s . The number of i n d i v i d u a l s i n each group who -83-TABLE XXII COMPARISON OF THE COMPONENT SCORES FOR THE TWO PAIN IN-STORE-ASSESSMENT PROBLEMS FOR THE EXPERIMENTAL AND CONTROL GROUPS EXPERIMENTAL GROUP PRE-IN-STORE-ASSESSMENT MEAN PERFORMANCE SCORES PAIN PAIN i COMPONENTS APPROPRIATE INAPPROPRIATE t-VALUE D.F. t-PROB 1 F-PROB' 1. DATA GATHERING 0. .99 1. 00 0.00 10 0.95 0.24 2. INAPPROPRIATE RECOMMENDATIONS -0. .75 -1. 50 -1.82 10 0.10 0.09 3. APPROPRIATE RECOMMENDATIONS 0. .25 0.25 0.00 10 0.95 0.79 4. DRUG-USE-COUNSELLING 0, .99 0. ,25 1.81 10 0.26 0.19 5. TOTAL SCORE 1 , .50 0. ,00 0.74 10 0.48 0.47 CONTROL GROUP PRE-IN-STORE-ASSESSMENT COMPONENTS 1. DATA GATHERING 0 .00 0. .20 ** 2. INAPPROPRIATE RECOMMENDATIONS -1 .00 -1.00 ** 3. APPROPRIATE RECOMMENDATIONS 0 .10 0.00 ** 4. DRUG-USE-COUNSELLING 0 .00 0. .60 ** 5. TOTAL SCORE -0 .90 -0. .20 -1 .183 4 0.30 0.00 EXPERIMENTAL GROUP POST-IN-STORE-•ASSESSMENT MEAN PERFORMANCES SCORES PAIN PAIN COMPONENTS INAPPROPRIATE APPROPRIATE t-VALUE D.F. t-PROB F-PROB 1. DATA GATHERING 1 .88 1. .92 -0.46 10 0.66 0.79 2. INAPPROPRIATE RECOMMENDATIONS -1 .25 -0. .75 0.90 10 0.39 0.78 3. APPROPRIATE RECOMMENDATIONS 0 .50 0. .50 0.00 10 0.95 0.71 4. DRUG-USE-COUNSELLING 0 .75 1, .75 1 .28 10 0.23 0.58 5. TOTAL SCORE 1 .87 4. .00 -0.83 10 0.43 0.49 CONTROL GROUP POST-IN-STORE-ASSESSMENT COMPONENTS 1. DATA GATHERING 0 .20 0 .00 ** 2. INAPPROPRIATE RECOMMENDATIONS -1 .00 -1 .00 ** 3. APPROPRIATE RECOMMENDATIONS 0 .10 0 .00 ** 4. DRUG-USE-COUNSELLING 0 .00 0 .20 ** 5. TOTAL SCORE -0.70 -0 .80 0.22 13 0.81 0.16 1. t - P r o b a b i l i t y . A t - p r o b a b i 1 i t y o f 0.05 o r l e s s r e p r e s e n t s a s i g n i f i c a n t d i f f e r e n c e . 2. F - P r o b a b i l i t y . A F - p r o b a b i l i t y o f 0.05 o r l e s s i n d i c a t e s t h a t the sample v a r i a n c e s a r e s i g n i f i c a n t l y d i f f e r e n t and t h e r e f o r e come from d i f f e r e n t p o p u l a t i o n s . ** S i g n i f i c a n t d i f f e r e n c e s were n o t computed f o r t h o s e i n s t a n c e s where p h a r m a c i s t s ' b e h a v i o u r s were so s i m i l a r as to produce no v a r i a n c e . -84-received pain problems and the number who recei v e d c o l d problems r e-mained the same from pre- to p o s t - t e s t . The maximum p o s s i b l e score f o r both c o l d problems was 14. The maximum p o s s i b l e score f o r the pain problems was 13. The o v e r a l l maximum score, f o r each group, r e-mains the same from p r e - t e s t to p o s t - t e s t . Table XXIII presents a summary of the performance o f the pharmacists on the pre- and post-in-store-assessments. For the ex-perimental group, there were s t a t i s t i c a l l y s i g n i f i c a n t g a i n s, a t the .05 l e v e l or b e t t e r , f o r "data gathering" behaviours, "appropriate r e -commendations" and the t o t a l " o v e r a l l performance." There was, how-ever, no s i g n i f i c a n t decrease i n the number of "i n a p p r o p r i a t e recom-mendations" nor a s i g n i f i c a n t improvement i n the "drug-use-counseling" behaviours of the pharmacists. For the co n t r o l group there was no s i g n i f i c a n t improvement i n any of the components or the t o t a l score. Figure 7 presents a v i s u a l r e p r e s e n t a t i o n of these r e s u l t s . These f i n d i n g s i n d i c a t e that the educational program d i d have a b e n e f i c i a l e f f e c t on the pharmacists who were i n attendance. As r e f l e c t e d by the performances on the in-store-assessment problems, there were s i g n i f i c a n t improvements i n the q u a l i t y of the primary care c o n s u l t i n g s e r v i c e s provided by the experimental group. The data i n Table XXIII i n d i c a t e d that the l e v e l of per-formance by pharmacists i n t h i s area i s low. Even a f t e r the educa-t i o n a l program, the mean t o t a l performance score was only 4.38 out of a maximum t o t a l performance score of 13 and 14. The reason f o r such poor performance i s a question worthy of fu t u r e i n v e s t i g a t i o n . -85-TABLE XXIII GAIN FOR THE POST-IN-STORE-ASSESSMENT SCORES FOR THE EXPERIMENTAL AND CONTROL GROUPS EXPERIMENTAL Pre-Test Post-Test Component Mean S.D. Mean S.D. ference t-Value D.F. t-Prob Data Gathering 2 .06 1 .63 2 .88 1 .96 0 .82 2 .49 33 0.02* Inappropriate Recommendations -0 .94 0 .55 -0 .91 0 .67 -0 .03 -0 .21 33 0.82 Appropriate Recommendations 0 .24 0 .55 0 .59 0 .89 0 .35 2 .10 33 0.04* Drug-Use-Counsel l i n g 1 .33 1 .24 1 .79 1 .51 0 .46 1 .69 33 0.10 Total Score 2 .65. 2 .13 4 .38 3 .99 1 .74 2 .44 33 0.01* CONTROL Component Pre-Mean •Test S.D. Post-Mean •Test S.D. D i f -ference t--Value D!F! t--Prob Data Gathering 1.28 1.05 1.28 1.59 0.00 0 .00 38 0 .95 Inappropriate Recommendations -1.08 0.35 -1.03 0.36 -0.05 -0 .57 38 0 .58 Appropriate Recommendations 0.08 0.35 0.10 0.38 0.03 0 .30 38 0 .76 Drug-Use-Counsel1ing 0.51 0.82 0.46 0.94 -0.05 -0 .36 38 0 .72 Total Score 0.77 2.13 0.82 2.28 0.05 0 .14 38 0 .82 1 = t . P r o b a b i l i t y . A t - P r o b a b i l i t y of 0.05 or l e s s represents a s i g n i f i c a n t g a i n . S i g n i f i c a n t d i f f e r e n c e -86-Fig. 7 Gains for the Experimental and Control Groups on the ISAPs' Component and Total Scores. KEY Experimental — — — — Control D.G.= Data Gathering I.R.= Inappropriate Recommendations A.R.= Appropriate Recommendations D.U.C.= Drug-Use-Counselling T.S.= Total Score ISAP= In-Store-Assessment Problem -87-The one f i n d i n g that remains unexplained i s the l a c k of a s i g n i f i c a n t decrease i n the number of post-course " i n a p p r o p r i a t e recommendations" and the presence of a s i g n i f i c a n t increase i n the number of post-course "appropriate recommendations". However, si n c e these two a c t i v i t i e s were not mutually e x c l u s i v e , i t was p o s s i b l e f o r i n d i v i d u a l s to make both " i n a p p r o p r i a t e " and "appropriate recommenda-t i o n s " . This occurred i n those instances where an i n d i v i d u a l recommended more than one course of a c t i o n , one of which being "appropriate" and the other " i n a p p r o p r i a t e " . There were four such cases on the post-in-store-assessments of the experimental group. There were no such instances on the pre-in-store-assessments f o r the experimental nor i n e i t h e r the pre-or post-assessment of the c o n t r o l group. For the experimental group, there were 28 " i n a p p r o p r i a t e recommendations" on the pre and 25 on the post-in-store-assessments. As w e l l , there were s i x "appropriate recommendations" on the pre-and 13 on the post-in-store-assessments. The s i g n i f i c a n t increase i n the post-course "appropriate recommendations" could be a t t r i b u t e d to changes i n two areas. A decrease i n the number of " i n a p p r o p r i a t e recommendations", which by i t s e l f was i n s i g n i f i c a n t , and a change w i t h i n the "appropriate recommendation" s e c t i o n s . I t was p o s s i b l e f o r i n d i v i d u a l s to improve t h e i r scores by g i v i n g the "acceptable" a l t e r n a t i v e on the p r e - t e s t and "the most appropriate" recommendation on the p o s t - t e s t . Therefore i t would appear that most of the increase i n t h i s component came from an improvement of the performance w i t h i n the category. Those who were -88-already g i v i n g "good" advice gave " b e t t e r " advice. From the data already presented i n Table X X I I I , i t i s obvious that the one aspect which improved the most as the r e s u l t of the c o n t i n u i n g education program was the "data gathering" behaviours. Pharmacists asked more questions about observers' primary care requests a f t e r the educational program than they d i d before. To examine the in-store-assessment f i n d i n g s i n d e t a i l , Tables XXIV to XXVII, i n Appendix A, present the r e l a t i o n s h i p s between the performance on the in-store-assessments and the various s i t u a t i o n a l f a c t o r s recorded by the observers during t h e i r v i s i t s to the pharmacies. Table XXIV contains the data f o r the pre-assessment and Table XXV con-t a i n s the data f o r the post-assessment f o r the experimental group. Tables XXVI and XXVII cont a i n the data f o r the pre- and post-assessments, r e s p e c t i v e l y , f o r the c o n t r o l group. . The pharmacist's performance as a primary care c o n s u l t a n t was evaluated by c a l c u l a t i n g a t o t a l performance score over the f o l l o w i n g four subsets of behaviours: "data g a t h e r i n g " , " i n a p p r o p r i a t e recommend-a t i o n s " , "appropriate recommendations", and "drug-use-counselling". "Data g a t h e r i n g " , "appropriate recommendations" and "drug-use-c o u n s e l l i n g " c o n t r i b u t e p o s i t i v e l y and " i n a p p r o p r i a t e recommendations" c o n t r i b u t e s n e g a t i v e l y to the t o t a l performance score. I f these behaviours are r e l a t e d and are r e p r e s e n t a t i v e of the steps involved when a pharmacist responds to a primary care request, then they should c o r r e l a t e s i g n i f i c a n t l y w i t h one another. "Data g a t h e r i n g " , "appropriate recommendations" and "drug-use-counselling" should c o r r e l a t e -89-p o s i t i v e l y with one another and the t o t a l score. "Inappropriate recommendations" should c o r r e l a t e n e g a t i v e l y with the other subsets and with the t o t a l score. However, c o r r e l a t i o n c o e f f i c i e n t s between the components and the t o t a l performance score are i n f l a t e d s i n c e scores of the various subsets are summed to give the t o t a l performance score (see Table XXVIII). -90-TABLE XXVIII CORRELATIONS AMONG COMPONENTS AND WITH TOTAL SCORES OF THE IN-STORE-ASSESSMENTS FOR THE EXPERIMENTAL AND CONTROL GROUPS DATA EXPERIMENTAL PRE-ASSESSMENT -0.57 GATHERING EXPERIMENTAL POST-ASSESSMENT -0.47 CONTROL PRE-ASSESSMENT -0.20 CONTROL POST-ASSESSMENT -0.06 INAPPROPRIATE EXPERIMENTAL PRE-ASSESSMENT RECOMMENDATIONS EXPERIMENTAL POST-ASSESSMENT CONTROL PRE-ASSESSMENT CONTROL POST-ASSESSMENT APPROPRIATE EXPERIMENTAL PRE-ASSESSMENT RECOMMENDATIONS EXPERIMENTAL POST-ASSESSMENT CONTROL PRE-ASSESSMENT CONTROL POST-ASSESSMENT DRUG-USE- EXPERIMENTAL PRE-ASSESSMENT COUNSELLING EXPERIMENTAL POST-ASSESSMENT CONTROL PRE-ASSESSMENT CONTROL POST-ASSESSMENT TOTAL . EXPERIMENTAL PRE-ASSESSMENT SCORE EXPERIMENTAL POST-ASSESSMENT CONTROL PRE-ASSESSMENT CONTROL POST-ASSESSMENT DATA INAPPROPRIATE APPROPRIATE DRUG-USE- TOTAL GATHERING RECOMMENDATIONS RECOMMENDATIONS COUNSELLING SCORE 1 0.45 0_ •JLL 0.89 0.37 0_ d £ 0.86 0.01 0. 0.93 0.49 0 .30 0.88 •0.75 -0 .32 -0.70 •0.68 -0 .26 -0.70 •0.47 0 .22 -0.01 •0.43 0. .21 -0.30 0. .31 0.65 0. .14 0.62 -0. .13 0.20 0. .00 0.65 0.77 0.73 0.80 0.54 1. F o r the e x p e r i m e n t a l group, w i t h 32 degrees o f freedom, c o e f f i c i e n t s o f 0.34 and 0.44 r e s p e c t i v e l y a r e s i g n i f i c a n t a t 0.05 and 0.01 l e v e l s o f s i g n i f i c a n c e . In t h e c o n t r o l group, w i t h 37 .degrees o f freedom, c o e f f i c i e n t s o f 0.32 and 0.41 r e s p e c t i v e l y a r e s i g n i f i c a n t a t 0.05 and 0.01 l e v e l s o f s i g n i f i c a n c e . -91-The "data gathering" subset had s i g n i f i c a n t ^ negative c o r r e l a t i o n w i t h both the pre-(-0.57) and the post-(-0.47) " i n a p p r o p r i a t e recommenda-t i o n s " subset f o r the experimental group. Those pharmacists i n the experimental group who spent more time asking questions about the problem which p r e c i p i t a t e d the primary care request made fewer " i n a p p r o p r i a t e recommendations". There were no s i g n i f i c a n t c o r r e l a t i o n s between these subsets on e i t h e r the pre-or post-assessment f o r the pharmacists i n the co n t r o l group. There were s i g n i f i c a n t p o s i t i v e c o r r e l a t i o n s between the pharmacists' performances on the subsets of "data gathering" and "appropriate recommendations" on the pre-(0.45) and post-(0.37) assess-ments f o r the experimental group. There was a l s o a s i g n i f i c a n t p o s i t i v e c o r r e l a t i o n (0.49) f o r these two subsets of behaviours on the post-assessment f o r the pharmacists i n the c o n t r o l group. Those pharmacists who gathered the most data about the complaint were more l i k e l y to make an "appropriate recommendation". "Data gathering" performance was s i g n i f i c a n t l y c o r r e l a t e d with "drug use c o u n s e l l i n g " behaviours i n the pre-(0.51) assessments and post-(0.49) assessments f o r the pharmacists i n the experimental group. A s i g n i f i c a n t c o r r e l a t i o n (0.69) f o r performance on these two subsets was a l s o found f o r the pre-assessments of the c o n t r o l group pharmacists. These f i n d i n g s were i n t e r p r e t e d to mean those pharmacists who spend more For the experimental group, with 32 degrees of freedom, c o e f f i c -i e n t s of 0.34 and 0.44 r e s p e c t i v e l y are s i g n i f i c a n t a t 0.05 and 0.01 l e v e l s of s i g n i f i c a n c e . For the c o n t r o l group, w i t h 37 degrees of freedom, c o e f f i c i e n t s of 0.32 and 0.4T r e s p e c t i v e l y are s i g n i f i c a n t at 0.05 and 0.01 l e v e l s o f s i g n i f i c a n c e . -92-time questioning the observers about the problem were a l s o the ones who spend more time i n s t r u c t i n g about the use of a product they recommend. Since a pharmacist's recommendations were "appropriate" or " i n a p p r o p r i a t e " , the pharmacist's performances i n these two subsets should be n e g a t i v e l y c o r r e l a t e d . The f i n d i n g s support t h i s statement. These two subsets had c o r r e l a t i o n c o e f f i c i e n t s of -0.75 on the pre-assessments and -0.68 on the post-assessments f o r the experimental group. The corresponding c o e f f i c i e n t s f o r the pharmacists i n the c o n t r o l group were -0.42 and -0.43. There were no s i g n i f i c a n t c o r r e l a t i o n c o e f f i c i e n t s between the performance scores on the subsets " i n a p p r o p r i a t e recommendations" and "drug-use-counselling". One might expect a s i g n i f i c a n t negative c o r r e l a t i o n between the two. However, i t i s p o s s i b l e to recommend an i n a p p r o p r i a t e product f o r a complaint and s t i l l give good i n s t r u c t i o n s on how to use the product. There were no s i g n i f i c a n t c o r r e l a t i o n c o e f f i c i e n t s between performance on the subset "appropriate recommendations" and performance on the subset "drug-use-counselling" f o r e i t h e r group on e i t h e r assess-ment. This f i n d i n g may be explained by the f a c t that some of the "appropriate recommendations" to the primary care requests used i n t h i s study d i d not i n v o l v e recommending a drug product. Therefore, the pharmacist would not acquire points f o r informing the observers, when and how to use the product. A l s o , some pharmacists may have f e l t t hat i t was unnecessary to spend time c o u n s e l l i n g the consumer on how to use the n o n - p r e s c r i p t i o n product s i n c e a l l products are accompanied by -93-w r i t t e n i n s t r u c t i o n s . S e t t i n g and S i t u a t i o n a l Factors The observers were required to record several s i t u a t i o n a l f a c t o r s when they were conducting the in-store-assessments. The r e l a t i o n s h i p s among these f a c t o r s and the behavioural components are contained i n Table XXIV to XXVII i n Appendix A. Table XXIX presents the c o r r e l a t i o n s between the s i t u a t i o n a l f a c t o r s and the t o t a l performance score. TABLE XXIX CORRELATIONS AMONG THE SITUATIONAL FACTORS PRESENT DURING THE IN-STORE-ASSESSMENTS AND THE TOTAL PERFORMANCE SCORES CORRELATIONS WITH TOTAL PERFORMANCE SCORES EXPERIMENTAL EXPERIMENTAL CONTROL CONTROL SITUATIONAL FACTORS PRE-ASSESSMENT POST-ASSESSMENT PRE-ASSESSMENT POST-ASSESSMENT AGE -0.19 0.26 0.10 -0.01 TIME TO BE GREETED BY PHARMACIST 0.06 -0.49 1 0.21 -0.09 TIME SPENT WITH THE PHARMACIST 0.66 0.60 0.17 0.13 BUSY -0.38 0.02 -0.04 -0.19 NUMBER OF PHARMACISTS WORKING -0.03 -0.11 -0.21 -0.13 NUMBER OF CUSTOMERS AT THE PRESCRIPTION COUNTER 0.14 -0.14 -0.13 -0.15 NUMBER OF CUSTOMERS IN WHOLE STORE 0.10 -0.23 0.05 -0.11 NUMBER OF CUSTOMERS AT CASH REGISTER ONE 0.08 -0.31 0.11 0.01 NUMBER OF CUSTOMERS AT CASH REGISTER TWO 0.00 -0.04 0.31 0.26 NUMBER OF CLERKS 0.11 -0.24 0.06 -0.18 PHARMACIST'S TREATMENT OF OBSERVER** -0.80 -0.51 -0.40 -0.43 PHARMACIST'S TREATMENT OF THE OBSERVER'S REQUEST** -0.67 -0.39 -0.05 -0.30 ** Low scores would i n d i c a t e that the observer f e l t that the pharmacist was i n t e r e s t e d i n him as a person and hi s problem. 1. For the experimental group, with 29 degrees of freedom, c o e f f i c i e n t s of 0.36 and 0.46 r e s p e c t i v e l y are s i g n i f i c a n t a t 0.05 and 0.01 l e v e l s of s i g n i f i c a n c e . For the c o n t r o l group, with 25 degrees of freedom, c o e f f i c i e n t s of 0.38 and 0.49 r e s p e c t i v e l y are s i g n i f i c a n t a t 0.05 and 0.01 l e v e l s of s i g n i f i c a n c e . -95-The only instance when the age of the pharmacists had a s i g n i f i c a n t c o r r e l a t i o n c o e f f i c i e n t was i n the comparisons f o r the post-assessments f o r the experimental groups. There were s i g n i f i c a n t c o e f f i c i e n t s between age and "appropriate recommendations" (0.41) and between age and "drug-use-counselling" (0.86). Older pharmacists, i n the experimental group, t h e r e f o r e , gave more "appropriate recommendations" and more advice on the use of the drug product a f t e r the educational program than d i d the younger pharmacists. The time which elapsed between when the observer entered the st o r e and when the request was a c t u a l l y presented to the pharmacist would be one i n d i c a t i o n o f how busy t h a t p a r t i c u l a r s t o r e was on th a t occasion. In the post-course assessments f o r the experimental group, there were s i g n i f i c a n t c o r r e l a t i o n c o e f f i c i e n t s between time elapsed and the scores f o r " i n a p p r o p r i a t e recommendations" (0.53), "drug-use-c o u n s e l l i n g " (0.49) and the t o t a l performance score (-0.49). The longer the observer had to w a i t , i n t h i s i n s t a n c e , the more l i k e l y the pharmacist would give an " i n a p p r o p r i a t e recommendation" and the l e s s l i k e l y the pharmacist would counsel about the use of the n o n - p r e s c r i p t i o n product. For both pre-and post-assessments f o r the experimental group, the length of time the observers spent with the pharmacists was an i n f l u e n c e on the q u a l i t y of advice given by the pharmacist. The amount of time spent with the pharmacist d i s c u s s i n g the request was s i g n i f i c a n t l y c o r r e l a t e d , i n both these assessments, w i t h the amount of "data gathered", whether the recommendations were appropriate or i n a p p r o p r i a t e , -96-the amount of information given about the use of the n o n - p r e s c r i p t i o n product and the o v e r a l l performance score. While time was an important f a c t o r i n the performance of the pharmacists i n the experimental group i t was not an important f a c t o r i n the performance of the pharmacists i n the c o n t r o l group. However, the o v e r a l l performance of the c o n t r o l group was i n f e r i o r to t h a t of the experimental group. The observers were asked to record whether or not the pharmacist "appeared" busy i n the dispensary as they were to present t h e i r request. In only one i n s t a n c e , the pre-assessments of the pharmacists i n the experimental group, d i d the appearance of being busy s i g n i f i c a n t l y c o r r e l a t e (-0.38) with the o v e r a l l performance scores of the pharmacists. In the other assessment s i t u a t i o n s , t h i s was not a s i g n i f i c a n t f a c t o r i n the performance of the pharmacists. For n e i t h e r experimental nor c o n t r o l group on the pre- or post-assessments d i d the q u a l i t y of pharmaceutical s e r v i c e s s i g n i f i c a n t l y cor-r e l a t e with any of the f o l l o w i n g : number of customers at the p r e s c r i p t i o n counter, number of customers i n the whole s t o r e , number of pharmacists i n the dispensary or c l e r k s on duty. A l l of these would be i n d i c a t o r s of how busy the pharmacy was a t the time the assessment was conducted. The observers were required to complete two semantic d i f f e r e n t i a l s c a l e s as part of the s i t u a t i o n a l f a c t o r s recorded (see Appendix A). One contained f i v e v a r i a b l e s , d e a l i n g with the pharmacist's treatment of the observer. The second contained s i x v a r i a b l e s d e a l i n g w i t h the pharma-c i s t s treatment of the n o n - p r e s c r i p t i o n medication request. A low score would i n d i c a t e t h a t , f o r the f i r s t s c a l e , the observer f e l t the -97-pharmacist was concerned about the person, and f o r the second s c a l e , the observer f e l t the pharmacist was c o n f i d e n t i n handling the request. Scores f o r each s c a l e were c o r r e l a t e d with the other s i t u a t i o n f a c t o r s and performance scores. The treatment of the observer by the pharmacist, as recorded by the semantic d i f f e r e n t i a l s c a l e , was s i g n i f i c a n t l y c o r r e l a t e d to the o v e r a l l performance of both groups on each assessment. With one exception, the pre-assessment f o r the pharmacists i n the experimental group, i t was not s i g n i f i c a n t l y c o r r e l a t e d with the number of "appropriate recommendations". The observers' r a t i n g s of the pharmacists' treatment of t h e i r n o n - p r e s c r i p t i o n request was s i g n i f i c a n t l y c o r r e a l t e d w i t h the o v e r a l l performance scores f o r both pre-and post-assessments f o r the pharmacists i n the experimental group. For n e i t h e r the pre-nor the post-assessments of the pharmacists i n the experimental or c o n t r o l group was there a s i g n i f i c a n t c o r r e l a t i o n between t h i s r a t i n g and the number of "appropriate recommendations". These f i n d i n g s from the observers' r a t i n g s of the pharmacists i n t h i s study, i n d i c a t e that although the pharmacist may show concern f o r a problem and be c o n f i d e n t about the s o l u t i o n suggested, the recommendations may not always be appropriate. P r e d i c t i v e V a l i d i t y of the Written Simulation The four w r i t t e n s i m u l a t i o n s developed f o r t h i s i n v e s t i g a t i o n were used on two occasions. They were completed by a group of pharmacists before r e c e i v i n g i n s t r u c t i o n on " c o l d " or "pain" i n the c o n t i n u i n g -98-education program and they were a l s o completed by a second group of pharmacists a f t e r the educational program. In both instances the w r i t t e n s i m u l a t i o n matched the ISAP used to assess the pharmacist. The pre-d i c t i v e powers of the s i m u l a t i o n s were then stu d i e d by c o r r e l a t i n g the i n d i v i d u a l ' s performance score on the simulated problem with h i s or her r e a l - l i f e behaviour i n response to the same primary care requests. This r e s u l t s i n e i g h t d i s t i n c t c o r r e l a t i o n c o e f f i c i e n t s (Table XXX). A l l the e i g h t c o r r e l a t i o n c o e f f i c i e n t s were p o s i t i v e . However, of the e i g h t c o r r e l a t i o n s , only two are s i g n i f i c a n t at the 0.05 l e v e l of s i g -n i f i c a n c e . Written s i m u l a t i o n "Cold Appropriate" compared with t h a t ISAP, as p o s t - t e s t s , and w r i t t e n s i m u l a t i o n "Pain Appropriate" compared with t h a t ISAP, as p o s t - t e s t s , were s i g n i f i c a n t l y c o r r e l a t e d . The c o r r e l a t i o n c o e f f i c i e n t s were 0.54, w i t h 12 degrees of freedom, and 0.98 w i t h three degrees of freedom. I t would appear that these two s i m u l a t i o n s do p r e d i c t an i n d i v i d u a l ' s r e a l - l i f e performance. However, the evidence i s weak. TABLE XXX VALIDITY COEFFICIENTS FOR THE FOUR WRITTEN SIMULATIONS JUDGED AGAINST THE FOUR IN-STORE-ASSESSMENTS WRITTEN SIMULATION COLD APPROPRIATE COLD INAPPROPRIATE PAIN APPROPRIATE PAIN INAPPROPRIATE PRE-COURSE VALIDATION -, ? C r i t i c a l : r. d . f / V a l u e ' 0.62 0.44 0.48 0.39 6 12 6 3 0.71 0.53 0.71 0.88 POST-COURSE VALIDATION C r i t i c a l d.f. Value 0.54* 12 0.01 0.98* 0.36 0.53 0.71 0.88 0.71 COMBINED VALIDATION C r i t i c a l d.f. Value 0.56* 17 0.37 17 0.68* 10 0.26 10 0.46 0.46 0.58 0.58 1. C o r r e l a t i o n c o e f f i c i e n t . 2. Degrees of freedom. * C o r r e l a t i o n i s greater than the c r i t i c a l value. -100-The previous c o r r e l a t i o n s were separated i n t o those r e l a t e d to pre-course and post-course v a l i d a t i o n r e s u l t i n g i n e i g h t d i s t i n c t groups. I f a s i m u l a t i o n has p r e d i c t i v e powers, then i t should p r e d i c t an i n d i v i d u a l ' s r e a l - l i f e performance regardless of when i t i s used. I t i s , t h e r e f o r e , l e g i t i m a t e to combine the pre- and post-course groups f o r each s i m u l a t i o n . This r e s u l t s i n four d i s t i n c t c a t e g o r i e s [Table XXX). In so doing, performance on s i m u l a t i o n "Cold Appropriate" compared with the i n d i v i d u a l ' s r e a l - l i f e performance was 0.56, which was s i g n i f i c a n t . For the s i m u l a t i o n "Cold Inappropriate" the c o r r e l a t i o n c o e f f i c i e n t was 0.37 which was not s i g n i f i c a n t . Likewise f o r s i m u l a t i o n s "Pain Appropriate" and "Pain Inappropriate", the c o e f f i c i e n t s were 0.68 and 0.26 r e s p e c t i v e l y . The c o e f f i c i e n t of s i m u l a t i o n "Pain Appropriate" was s i g n i f i c a n t and the c o e f f i c i e n t f o r s i m u l a t i o n "Pain Inappropriate" was not s i g n i f i c a n t . Here again the evidence, although not overwhelming, was somewhat encouraging. Written s i m u l a t i o n "Cold Appropriate" r e t a i n e d i t s s i g n i f i c a n t p r e d i c t i v e powers. The p r e d i c t i v e powers of s i m u l a t i o n "Cold Inappropriate" improved o v e r a l l . Combining the pre- and post-course s i m u l a t i o n s of "Pain Appropriate" r e s u l t e d i n a lower but s t i l l s i g -n i f i c a n t c o e f f i c i e n t , t h i s f i n d i n g i s probably more r e a l i s t i c s i n c e the sample s i z e increased. With the combined v a l i d a t i o n procedure, the p r e d i c t i v e powers of w r i t t e n s i m u l a t i o n "Pain Inappropriate" were not s i g n i f i c a n t . Although the evidence i s only m i l d l y s u p p o r t i v e , i t would appear th a t some of the w r i t t e n s i m u l a t i o n s developed f o r t h i s study do p r e d i c t r e a l - l i f e performance. Perhaps the s i n g l e l a r g e s t f a c t o r hampering a -101-more c o n c l u s i v e statement as a r e s u l t of t h i s study was the small sample s i z e . Future study of the p r e d i c t i v e powers of the w r i t t e n s i m u l a t i o n s w i l l demand l a r g e r sample s i z e s . The c o r r e l a t i o n c o e f f i c i e n t s discussed so f a r represented the agreement between a pharmacist's score on the w r i t t e n s i m u l a t i o n to hi s or her score on the in-store-assessment. These c o e f f i c i e n t s con-si d e r e d t o t a l scores only and not i n d i v i d u a l items. In a d d i t i o n to these c o r r e l a t i o n c o e f f i c i e n t s a consistency score was c a l c u l a t e d comparing i n d i v i d u a l items which were common to both the ISAPs and the w r i t t e n simulations.. There were 17 common behaviours between the w r i t t e n s i m u l a t i o n and the ISAP "Cold Appropriate". Likewise there were 17, 15 and 18 common behaviours between the three remaining ISAPs and s i m u l a t i o n s s e l e c t e d because they were i d e n t i c a l on both instruments. For those " i d e n t i c a l items", which behaviours d i d a pharmacist do or not do on the w r i t t e n s i m u l a t i o n s , how many d i d he or she do or not do during the in-store-assessment? The agreement on items between ISAP and s i m u l a t i o n was expressed as a percentage. The mean agreement on the pre-assessment was 66.54 percent. The mean agreement f o r the post-assessment was 65.71 percent. I f the w r i t t e n s i m u l a t i o n s and the in-store-assessments contained a l l the p l a u s i b l e pharmacist behaviours i n response to a primary care request, these consistency scores present f u r t h e r evidence of the p r e d i c t i v e powers of w r i t t e n s i m u l a t i o n s . Pharmacists performed b e t t e r on the si m u l a t i o n s than they d i d on the ISAPs, both i n o v e r a l l performance scores and r e l a t i v e to the -102-"optimal l e v e l of performance." On the s i m u l a t i o n s they asked more questions about the symptoms and were more l i k e l y to ask about drug use. I t would appear t h a t presenting a l i s t of a l t e r n a t i v e s has a cueing e f f e c t (1). In a d d i t i o n , performance on those si m u l a t i o n s i n v o l v i n g a "minor" complaint was a b e t t e r i n d i c a t o r of r e a l - l i f e behaviour. For the problems c o l d and pain i n a p p r o p r i a t e the panel of experts s t r o n g l y f e l t t h a t p h y s i c i a n c o n s u l t a t i o n was warranted. More pharmacists included t h i s i n t h e i r recommendations on the s i m u l a t i o n s than on the in-store-assessments. Again, t h i s may be due to the cueing e f f e c t . Factual Knowledge Tests One of the t r a d i t i o n a l ways of e v a l u a t i n g the success of a pharmacy contin u i n g education program has been to t e s t f o r f a c t u a l knowledge r e c a l l . In the c o n t i n u i n g education program being evaluated i n t h i s study, f a c t u a l knowledge t e s t s were administered every evening. The mean and standard d e v i a t i o n s f o r the t e s t s con-cerning " c o l d " and "pain" subject matter are presented i n Table XXXI. -103-TABLE XXXI GAINS IN FACTUAL KNOWLEDGE AS REPRESENTED BY SCORES ON PRE- AND POST-TESTS! AND THE RELATIONSHIPS BETWEEN THESE SCORES AND REAL-LIFE PERFORMANCE TEST CONTENT MPS 2 PRE-MEAN •TEST r 3 POST-TEST MEAN r t-VALUE D.F. t-PROB 4 COLD 15 8.44 -0.14 12.00 -0.29 5.09 16 0.001* PAIN 9 4.64 -0.26 8.27 0.34 10.77 10 0.001* 1 = M u l t i p l e - c h o i c e and Tr u e - f a l s e Questions 2 = Maximum p o s s i b l e score 3 = C o r r e l a t i o n c o e f f i c i e n t between knowledge t e s t score and performance on the ISAPs 4 = A t - p r o b a b i l i t y of l e s s than 0.05 represents a s i g n i f i c a n t gain In the context o f gains i n f a c t u a l knowledge r e c a l l , the contin u i n g education program was a success, s i n c e there were s i g n i f i c a n t gains on both pain and c o l d p o s t - t e s t s . In a d d i t i o n , i t has been p r e v i o u s l y shown t h a t there were s i g n i f i c a n t gains i n the post program assessments of r e a l - l i f e performance. On two e v a l u a t i o n instruments, then, the course p a r t i c i p a n t s had s t a t i s t i c a l l y s i g n i f i c a n t improved performance, t h e r e f o r e the educational program had an impact. There was, however, no s t a t i s t i c a l l y s i g n i f i c a n t c o r r e l a t i o n (0.14) between the two gain scores. From the data i n Table XXXI, i t i s obvious t h a t there i s no strong r e l a t i o n s h i p between the performance on a t e s t f o r r e c a l l of information and r e a l - l i f e performance. Factual knowledge t e s t s i n t h i s study were not accurate p r e d i c t o r s o f r e a l - l i f e performance. The f a c t u a l knowledge t e s t s developed f o r t h i s i n v e s t i g a t i o n , purport to measure r e c a l l of s p e c i f i c i n f o r m a t i o n . And the w r i t t e n -104-s i m u l a t i o n s purport to measure the a p p l i c a t i o n and i n t e g r a t i o n of t h i s information to the s o l u t i o n of a problem. Knowledge i s a necessary p r e r e q u i s i t e f o r problem s o l v i n g . However, knowledge scores and probl s o l v i n g scores are found to be unrelated (Table XXXII). Therefore, e i t h e r the knowledge i s not that required to solve the problems i n q u e s t i o n , or knowledge alone i s not s u f f i c i e n t - problem s o l v i n g i s a c o g n i t i v e s k i l l and as a s k i l l , i t i s developed through p r a c t i c e and feedback on l e v e l of performance. -105-TABLE XXXII RELATIONSHIP BETWEEN SCORES ON TESTS FOR FACTUAL KNOWLEDGE RECALL AND PERFORMANCE ON WRITTEN SIMULATIONS Simulation One (Cold Appropriate) Performance Factual Knowledge Pre-Test r 1 = -0.11 d . f . 2 6 "Cold" P o s t - T e s t 3 r = 0.18 d.f. 12 Simu l a t i o n Two (Cold Inappropriate) Performance Factual Knowledge Pre-Test r = -0.02 d.f. 12 "Cold" Post-Test r = -0.28 d.f. 6 Simulation Three (Pain Appropriate) Performance Factual Knowledge Pre-Test r = -0.59 d.f. 6 "Pain" Post-Test^ r = 0.25 d.f. 3 Simu l a t i o n Four (Pain Inappropriate) Performance Factual Knowledge Pre-Test r = -0.60 d.f. 3 "Pain" Post-Test r = 0.26 d.f. 6 1 = C o r r e l a t i o n c o e f f i c i e n t between pre- or p o s t - t e s t score and performance on pre- or p o s t - w r i t t e n s i m u l a t i o n . 2 = Degrees of freedom. 3 = I n d i v i d u a l s who received w r i t t e n s i m u l a t i o n one as p r e - t e s t received w r i t t e n s i m u l a t i o n two as p o s t - t e s t and v i c e versa. 4 = I n d i v i d u a l s who received w r i t t e n s i m u l a t i o n three as p r e - t e s t received w r i t t e n s i m u l a t i o n four as p o s t - t e s t and v i c e versa. P a r t i c i p a n t s ' E v aluation of the Program One of the o b j e c t i v e s of t h i s i n v e s t i g a t i o n was to obt a i n the p a r t i c i p a n t s ' s u b j e c t i v e e v a l u a t i o n o f the course. Was there a r e l a t i o n s h i p between " l i k i n g " the course and performance on the t e s t s f o r f a c t u a l knowledge and r e a l - l i f e performance? To obt a i n t h e i r r e a c t i o n to the course, p a r t i c i p a n t s were asked to complete an e v a l u a t i o n form on the l a s t evening of the -106-program. They were asked to use a magnitude estimation s c a l e to r a t e various components of the program. A l i n e of a standardized length represented the average c o n t i n u i n g education program, and they were then asked to draw a l i n e r a t i n g the present course on a number of dimensions. T h e i r l i n e s were l a t e r measured and expressed as a r a t i o of the standard l i n e . Table XXXIII presents a r i t h m e t i c mean, geometric mean (2) and the median values f o r the p a r t i c i p a n t s ' r a t i n g s . A r a t i o of one would i n d i c a t e that they f e l t t h i s course was e q u i v a l e n t to the average c o n t i n u i n g education course. A r a t i o of l e s s than one would mean they rated t h i s course i n f e r i o r to the average c o n t i n u i n g education course on that dimension. A r a t i o g reater than one would i n d i c a t e t h a t they rated t h i s course s u p e r i o r on that dimension. From the data contained i n Table XXXIII, i t i s obvious that the experimental subjects rated t h i s program equal to or s u p e r i o r to the average course attended i n the past on a l l dimensions. Those areas f o r p o s s i b l e im-provement are: 1. more emphasis on the signs and symptoms of the c o n d i t i o n being d i s c u s s e d , 2. the length of the l e c t u r e s could be changed, 3. s e l e c t i o n o f speakers who are more knowledgeable and have a b e t t e r s t y l e of p r e s e n t a t i o n and 4. provide more o p p o r t u n i t i e s f o r d i s c u s s i o n . With respect to number two and t h r e e , some p a r t i c i p a n t s f e l t t hat the l e c t u r e s tended to be "a b i t " too long and that the younger speakers lacked the p r a c t i c a l pharmacy experience and t h i s a f f e c t e d t h e i r c r e d i b i l i t y . -107-TABLE XXXIII MAGNITUDE ESTIMATIONS BY EXPERIMENTAL SUBJECTS COMPARING DIMENSIONS OF THIS PROGRAM WITH THE AVERAGE CONTINUING EDUCATION PROGRAM IN THE PAST Geometric A r i t h m e t i c Dimension Mean^ Mean Median 1. Usefulness of material learned. 1.66 1.78 1.84 2. Use of m a t e r i a l f o r a d v i s i n g p a t i e n t s . 1.63 1.80 1.86 2 3. M a t e r i a l too elementary. 0.70 0.79 0.93 4. Emphasis on drug products. 1.40 1.54 1.57 5. Emphasis on signs and symptoms of r e l e v a n t diseases. 1.24 1.36 1.02 6. Length of l e c t u r e s . 1.24 1.32 1.03 7. Speakers were knowledgeable and presented m a t e r i a l w e l l . 1.16 1.31 1.00 8. Usefulness of handouts. 1.45 1.60 1.50 9. Opportunity f o r d i s c u s s i o n . 1.35 1.45 1.14 10. Usefulness of p r e - / p o s t - t e s t s . 1.55 1.75 1.81 11. E f f i c i e n t l e a r n i n g experiences. 1.70 1.82 1.59 12. P r e f e r a b l e l e a r n i n g experience. 1.48 1.69 1.52 1. A mean of 1.0 would i n d i c a t e the experimental subjects f e l t t h i s program equal l e d the average cont i n u i n g education program on t h a t dimension. 2. This item was the reverse of a l l other items. A l i n e l a r g e r than the standard l i n e would i n d i c a t e the material was not too elemen-t a r y . Although i t was e x p l a i n e d , i t may have been misunderstood by some of the p a r t i c i p a n t s . The pharmacists' r a t i n g s of the dimensions were c o r r e l a t e d with one another. As w e l l , the year the pharmacist graduated from -108-l i n i v e r s i t y and the number of c o n t i n u i n g education programs attended the previous three years were included i n t h i s c o r r e l a t i o n matrix. S i g n i f i c a n t c o r r e l a t i o n c o e f f i c i e n t s , at the 0.05 l e v e l of s i g n i f i -cance revealed the f o l l o w i n g i n f o r m a t i o n . (See Table XXXIV, Appendix E) The o l d e r pharmacists found the m a t e r i a l more useful i n a d v i s i n g p a t i e n t s and were more l i k e l y to agree that the handouts were u s e f u l . Many of the o l d e r pharmacists would have an e s t a b l i s h e d approach to a d v i s i n g p a t i e n t s . However, previous f i n d i n g s reported i n t h i s study suggested that i t was t h i s group of pharmacists who changed the most from the pre to post course in-store-assessments. The only time that age was s i g n i f i c a n t l y c o r r e l a t e d with performance on the i n -store-assessments was during the post program assessments. The o l d e r pharmacists had higher o v e r a l l performance scores and had more appro-p r i a t e recommendations. Those i n d i v i d u a l s who rated the m a t e r i a l useful a l s o rated a number of other dimensions of the program h i g h l y . The exceptions were th a t they d i d not t h i n k the m a t e r i a l was too elementary, they d i d not f e e l there was enough opportunity f o r d i s c u s s i o n , they d i d not l i k e the length of the l e c t u r e s and they were l e s s l i k e l y to r a t e the p r e - / p o s t - t e s t s as u s e f u l . Those i n d i v i d u a l s who f e l t the information was useful i n a d v i s i n g p a t i e n t s agreed there was the proper emphasis on the products, there was the proper emphasis on the disease c o n d i t i o n s being d i s -cussed, the speakers were knowledgeable and presented the m a t e r i a l w e l l -109-and the handouts were useful . Also they f e l t i t was an e f f i c i e n t and preferable learning experience. Pharmacists who f e l t th i s program placed the proper amount of emphasis on the drug products also agreed on a number of other items. They f e l t there had been the proper emphasis on the signs and symptoms of the conditions being discussed, the handouts and pre-and post-tests were useful and they rated the program as an e f f i c i e n t and preferable learning a c t i v i t y . Individuals who l i ked the emphasis on the signs and symptoms of the disease also thought the handouts were useful and that the speakers were knowledgeable. In add i t ion, they f e l t the program was a more e f f i c i e n t and preferable learning experience. There were s i gn i f i can t corre lat ion coef f i c ient s between the ratings of ind iv iduals who f e l t the lectures were the proper length and the usefulness of the handouts and the knowledge level of the speakers. Pharmacists who rated the speakers highly also rated the handouts useful . They also f e l t the program was an e f f i c i e n t and preferable learning experience. Individuals who found the handouts useful f e l t the program was both an e f f i c i e n t and preferable learning experience. Pharmacists who f e l t th i s program was more e f f i c i e n t than the average past program also f e l t i t was a preferable program. There were few s i gn i f i can t corre lat ion coef f i c ient s between the items "length of lectures " and "opportunity for d iscuss ion." This indicates considerable disagreement on the magnitude of the value -no-assigned to these two items and suggests in future for th i s program these character i s t i c s should be a l tered. The evaluation form also asked the pharmacists to use the magnitude estimation technique to compare the learning at t r ibutab le to various program a c t i v i t i e s with the average learning in a t h i r t y minute lecture. The arithmetic mean, geometric mean and medican values for these comparisons are contained in Table XXXV. The phar-macists at t r ibuted less learning to the simulations, videotapes, and buzz groups. These were innovations and some of these results may be explained by the pharmacists fee l ing uncomfortable with these new i n -st ruct ional techniques and devices. They att r ibuted an equal amount or more learning to the lec ture , pre-and post - tests , large group d i s -cussions and question periods. (A l l modes of learning to which they were accustomed.) TABLE XXXV MAGNITUDE ESTIMATIONS BY EXPERIMENTAL SUBJECTS COMPARING THE AMOUNT OF LEARNING ATTRIBUTABLE TO PROGRAM ACTIVITIES WITH THAT ATTRIBUTABLE TO STANDARD 30 MINUTE LECTURE Geometric Arithmetic A c t i v i t y Mean Mean Median Simulations 0.72 0.87 0.91 Videotapes 0.60 0.77 0.62 Pre-/Post-Tests 1.10 1.24 1.05 Lectures 1.30 1.45 1.12 Buzz Groups 0.71 0.84 0.74 Large Group Discussions 1.00 1.13 1.00 Question and Answer Periods 1.04 1.13 1.10 - I l l -There were s i gn i f i can t cor re lat ions , at the 0.05 level of s i gn i f i cance, in the fol lowing areas: pos i t ive re lat ions between the ratings of learning via simulations and the video tapes. Those indiv iduals who attr ibuted more learning to the pre-and post-tests also f e l t more learning occured during the lectures , the large group discussions and the question and answer periods. Part ic ipants who attr ibuted a large amount of learning to the group discussions also f e l t the question and answer periods made s i g n i f i -cant contributions to the i r learning. Pharmacists who f e l t the material was usefu l , had the proper emphasis on signs and symptoms of relevant diseases, had actua l ly used the material in advising patients and f e l t the speakers knowledgeable tended to a t t r ibute more learning to the lectures. Those part ic ipants who f e l t there was the proper emphasis on the drug products also a t t r i -buted s i gn i f i c an t amounts of learning to the pre and post tes t s . Those who rated the speakers highly f e l t they had learned more from the l e c -tures. Individuals who were s a t i s f i ed with the pre-and post-tests also f e l t they learned from them. Pharmacists who rated th i s program a more e f f i c i e n t learning experience attr ibuted more learning to the lectures. Part ic ipants rat ing th i s program as preferable att r ibuted larger amounts of learning to the s imulations, the video tapes, the pre and post tests and the lectures. The only s i gn i f i c an t corre lat ion coe f f i c ient s between the number of programs attended in the three previous years and any of the dimensions in Table XXXIII or a c t i v i t i e s in Table XXXV were with the length of the lectures and the amount of learning at t r ibutab le to -112-buzz groups. Those ind iv iduals who had attended more programs were d i s s a t i s f i e d with the length of the lectures and attr ibuted less learning to the buzz group a c t i v i t y . From the subjective evaluations of the 23 pharmacists in the experimental group who completed the form, the program was a success. They rated i t equal to or superior to the average con-t inuing education program attended in the past on a l l the twelve dimensions on the evaluation form. The majority f e l t i t had been an e f f i c i e n t and a preferable learning experience in comparison with the average continuing education program in the past. The program was a success in the eyes of the part ic ipants . This was supported with the objective evidence of gains in factual knowledge and improved r e a l - l i f e performance. There were, however, no s t a t i s t i c a l l y s i gn i f i can t r e l a t i o n -ships between the pa r t i c ipant s ' e f f i c i ency or preference ratings of the program and gains in the tota l performance score or gains in the factual knowledge r e c a l l . -113-References Chapter Five Goran, M.^., J.W. Williamson and J.S. Gonnella, " V a l i d i t y of P a t i e n t Management Problems" Journal of Medical Education 48 (February, 1973): 171-177. Stevens, S.S., "A M e t r i c f o r the S o c i a l Consensus" Science 151 (February 4, 1966): 530-541. -114-CHAPTER SIX SUMMARY AND CONCLUSIONS The f o l l o w i n g i s a summary of the a c t i v i t i e s conducted during and the conclusions formulated as a r e s u l t of t h i s i n v e s t i g a t i o n . Summary This study evaluated a contin u i n g education program designed to improve a group of community pharmacists' performance as primary care c o n s u l t a n t s . A second, and e q u a l l y important, purpose of t h i s i n v e s t i g a -t i o n was the development of pharmacy w r i t t e n s imulations of primary care requests and the e x p l o r a t i o n of t h e i r powers to p r e d i c t an i n d i v i d u a l ' s performance i n the r e a l world. The educational program was evaluated using three modes of measurements: unobtrusive observation of change i n the r e a l - l i f e performance of the pharmacists as primary care c o n s u l t a n t s , gain i n f a c t u a l knowledge as the r e s u l t of i n s t r u c t i o n during the program, and p a r t i c i p a n t s ' s u b j e c t i v e r e a c t i o n to the program contents and processes. To conduct these measurements, several e v a l u a t i o n instruments were developed. F i r s t , a panel of content experts a s s i s t e d i n the con-s t r u c t i o n of four "in-store-assessment problems". The purpose of these problems was to evaluate pharmacists, i n t h e i r place of employment, as primary care consultants on " c o l d " and "pain" requests. The accompanying performance c h e c k l i s t permitted an e v a l u a t i o n of the pharmacist i n the f o l l o w i n g areas: "data g a t h e r i n g " , "appropriateness" of recommended ac t i o n s and "drug-use-counselling". Second, knowledge t e s t s were -115-developed i n the areas of " c o l d " and "pain" n o n - p r e s c r i p t i o n medication requests. These were administered before and a f t e r i n s t r u c t i o n had occurred on the above therapeutic c a t e g o r i e s . T h i r d , an instrument was developed which used a magnitude e s t i m a t i o n format and asked the p a r t i c i p a n t s to compare the present course to the "average" course they had attended i n the past. Before and a f t e r the educational program, t r a i n e d observers were used to administer the ISAPs to determine i f the primary care c o n s u l t i n g s k i l l s of the r e g i s t r a n t s had improved. A ."non-equivalent" c o n t r o l group was a l s o assessed a t these times. This group c o n t r o l l e d f o r other educational a c t i v i t i e s which may have i n f l u e n c e d the primary care c o n s u l t i n g behaviour of a l l the pharmacists i n the province. The . knowledge t e s t s measured increased information r e c a l l 1 as a r e s u l t of i n s t r u c t i o n . The r e l a t i o n s h i p between scores on these t e s t s and performance i n the r e a l world was s t u d i e d . In order to obtain the pharmacists' r e a c t i o n to the "value" or "worth" of the program, a magnitude e s t i m a t i o n e v a l u a t i o n form was used on the l a s t evening. The panel of content experts a l s o a s s i s t e d i n the development of four w r i t t e n s i m u l a t i o n s . The content of the s i m u l a t i o n s p a r a l l e l e d the four ISAPs. The simulations were completed by the pharmacists e n r o l l e d i n the program before and a f t e r r e c e i v i n g i n s t r u c t i o n on " c o l d " and "pain" primary care requests. These were assigned i n each i n s t a n c e , so t h a t the content of the s i m u l a t i o n matched the content of the pharmacist's pre-and post-"in-store-assessments". The performance of the of the pharmacists on the w r i t t e n s i m u l a t i o n s was studied to determine i f -116-i t p r e d i c t e d an i n d i v i d u a l ' s r e a l - l i f e performance. Conclusions The conclusions formulated as a r e s u l t of t h i s i n v e s t i g a t i o n are discussed i n terms of the e i g h t o r i g i n a l o b j e c t i v e s o u t l i n e d i n Chapter One. Objective 1: To develop four r e a l i s t i c primary care requests, i n  the area of n o n - p r e s c r i p t i o n medications, to be used as in-store-assessment  problems (ISAPs). A panel of content experts agreed t h a t the f o u r requests were r e a l i s t i c and p l a u s i b l e and d i d i n f a c t represent s i t u a t i o n s which occur d a i l y i n community pharmacies. I t was concluded from t h i s evidence, that the primary care requests used as the ISAPs were v a l i d . O b j ective 2: To devise and v a l i d a t e a comprehensive l i s t of observ- able pharmacist behaviours, i n response to the primary care requests, which  w i l l enable an e v a l u a t i o n of pharmacists' primary care c o n s u l t i n g s k i l l s . The panel of content experts, agreed that the l i s t s of p o s s i b l e pharmacist behaviours accompanying the requests were comprehensive i n nature. In a d d i t i o n , there were high;.i n t e r - r a t e r c o r r e l a t i o n c o e f f i c i e n t s f o r the p a n e l i s t s ' r a t i n g s of the appropriateness of the behaviours i n a l l f our problems. I t was concluded t h a t the l i s t s of pharmacist behaviours could be used to assess an i n d i v i d u a l ' s response to the primary care requests and that the l e v e l of agreement on the weightings of the items allowed f o r separating the behaviours i n each l i s t i n t o "appropriate" and " i n a p p r o p r i a t e " responses to the requests. Objective 3: To assess the improvement i n the q u a l i t y of primary  care c o n s u l t i n g of those pharmacists who p a r t i c i p a t e d i n the c o n t i n u i n g -117-education program. There were s i g n i f i c a n t gains i n the o v e r a l l performance o f the course p a r t i c i p a n t s as measured by the ISAPs. This improvement i n performance was a t t r i b u t a b l e to s i g n i f i c a n t increases i n "data gathering" and the number of "appropriate recommendations". There were no s i g -n i f i c a n t changes i n the number of " i n a p p r o p r i a t e recommendations" or i n the "drug-use-counselling" behaviour. There were no s i g n i f i c a n t improvements i n any of these areas f o r the "non-equivalent c o n t r o l " group. I t was concluded that the noted improvements i n the course p a r t i c i p a n t s ' performance were the r e s u l t of p a r t i c i p a t i o n i n the program. Obj e c t i v e 4: To measure the gains i n f a c t u a l knowledge as a  r e s u l t of p a r t i c i p a t i o n i n the program. There were s i g n i f i c a n t gains i n the p o s t - t e s t scores of the course p a r t i c i p a n t s . I t was concluded that the program improved the p a r t i c i p a n t s ' f a c t u a l knowledge r e c a l l on " c o l d " and "pain" non-p r e s c r i p t i o n medications. Objective 5: To determine the r e l a t i o n s h i p between scores on a  t e s t f o r f a c t u a l knowledge and r e a l - l i f e performance. There was no s t a t i s t i c a l l y s i g n i f i c a n t r e l a t i o n s h i p between scores on a t e s t f o r f a c t u a l knowledge and performance on the " i n - s t o r e -assessments". In t h i s program, the p a r t i c i p a n t s who recorded the highest marks on the knowledge t e s t were not n e c e s s a r i l y the i n d i v i d u a l s who performed the best on the j o b . Ob j e c t i v e 6: To obt a i n the p a r t i c i p a n t s s u b j e c t i v e e v a l u a t i o n  of the program. -118-On twelve dimensions, the p a r t i c i p a n t s rated t h i s program equal to or s u p e r i o r to the "average" c o n t i n u i n g education program they had attended i n the past. I t was concluded that the program was a success i n terms of p a r t i c i p a n t s a t i s f a c t i o n . Objective 7: To develop and v a l i d a t e four w r i t t e n simula- t i o n s which were s u i t a b l e f o r e v a l u a t i n g a pharmacist's performance  i n the area of primary care c o n s u l t i n g . A panel of content experts agreed t h a t the four s i m u l a t i o n s were r e a l i s t i c and the primary care problems they contained were p l a u s i b l e . As w e l l , there were s i g n i f i c a n t i n t e r - r a t e r c o r r e l a t i o n c o e f f i c i e n t s f o r the weights to be assigned to each item w i t h i n the s i m u l a t i o n s . In a d d i t i o n , the performance scores of c r i t e r i o n groups, possessing varying amounts of e x p e r t i s e i n the sim u l a t i o n s subject matter, increased w i t h the amount of experience. On the basis of these data i t was concluded t h a t there was s u b s t a n t i a l evidence sup-p o r t i n g the v a l i d i t y of the four w r i t t e n s i m u l a t i o n s developed to assess primary care c o n s u l t i n g behaviour. Objective 8: To assess the p r e d i c t i v e powers of the w r i t t e n  s i m u l a t i o n s by comparing the pharmacist's r e a l - l i f e performance with  t h a t of h i s performance on the s i m u l a t i o n s . A l l four w r i t t e n s i m u l a t i o n s had p o s i t i v e c o r r e l a t i o n c o e f f i c i e n t s with the pharmacists' r e a l - l i f e performance. Two of the f o u r had s t a t i s t i c a l l y s i g n i f i c a n t but weak c o r r e l a t i o n c o e f f i c i e n t s with i n d i v i d u a l s ' r e a l - l i f e performance. As w e l l , there were high consistency scores representing agreement of i n d i v i d u a l behaviours -119-between the two measurement techniques. I t was concluded t h a t two of the simu l a t i o n s developed f o r t h i s study d i d p r e d i c t r e a l - l i f e performance and a l s o t h a t a l l four hold promise of having s i g n i f i c a n t p r e d i c t i v e powers. Two of the most important c o n t r i b u t i o n s of t h i s i n v e s t i g a t i o n are the approach to the e v a l u a t i o n of the e f f e c t of a co n t i n u i n g education program and the development and v a l i d a t i o n of w r i t t e n s i m u l a t i o n s f o r pharmacy p r a c t i c e . Further developmental work i s encouraged i n each area. The f o l l o w i n g l i m i t a t i o n s to t h i s study should be considered when designing f u t u r e s t u d i e s . 1. The "col d appropriate" and the "pain a p p r o p r i a t e " simulations and ISAPs were developed to have p o t e n t i a l disease-drug c o n t r a i n d i c a t i o n s . To avoid the p o s s i b i l i t y of a pharmacist r e c o g n i z i n g a connection between the two measurements, the ISAPs and si m u l a t i o n s were constructed to have d i f f e r e n t but e q u a l l y s i g n i f i c a n t disease-drug c o n t r a i n d i c a t i o n s . Although the m a j o r i t y of p a n e l i s t s rated a l l these disease-drug combinations as s i t u a t i o n s to be avoided, c u r r e n t l i t e r a t u r e (1 ,2) would not support a l l of them as being c l i n i c a l l y s i g n i f i c a n t . Therefore, some pharmacists were penalized f o r recommending products which were rated as incompatiable with an e x i s t i n g disease e n t i t y even though the accuracy of the r a t i n g s i s i n dispute. This would be r e f l e c t e d i n lower performance scores. In a d d i t i o n , the disease-drug c o n t r a i n d i c a t i o n s i n some s i t u a t i o n s may have been more c l i n i c a l l y s i g n i f i c a n t and, t h e r e f o r e , the ISAPs and the simulations were not p a r a l l e l i n t h i s regard. Future s t u d i e s which incorporate disease-drug c o n t r a i n d i c a t i o n s should ensure that the c o n t r a i n d i c a t i o n s are supported by -120-c l i n i c a l data as well as pane l i s t s ' judgements and are ident ica l in the simulations and ISAPs. 2. The ISAPs were scored by the author and th i s i s recognized as possibly introducing a source of bias into the scores assigned. Future studies should use a " b l i n d " scoring procedure and have each performance check l i s t scored by several ind iv idua l s . 3. The observers who co l lected the in-store-assessment data recorded the resu lts of the i r interact ion with the pharmacist on the performance check l i s t . In retrospect, i t would have been useful to have them also make a second record on an audio cassette tape. This would have enabled a check of the material recorded by the observer. Such a procedure was followed by Jang (3 ) . 4. Experience gained from th is invest igat ion would indicate that establ i sh ing performance c r i t e r i a i s a d i f f i c u l t task. The author acknowl-edges that the f i n a l c r i t e r i a used are a r e f l e c t i on of th i s par t i cu la r panel. A second panel may have derived a s l i g h t l y d i f fe rent set of c r i t e r i a . A more thorough pre-test ing of the performance c r i t e r i a i s recomemnded for future studies, but studies based on mult ip le c r i t e r i a probably have the best overal l chances for success. Such mult ip le measures may include "expert" panels, user panels, pat ient/c l ient panels and findings in research journals. In summary, the educational program studied did improve the r e a l -l i f e performance of the pharmacists enro l led, did improve the i r factual knowledge r e ca l l and was evaluated very highly by the part ic ipants . Two of the simulations developed for th i s study did predict behaviour on the job. These conclusions re la te only to th i s invest igat ion. The small -121-sample s i z e and the absence of random assignment of subjects to c o n t r o l and experimental groups prevent g e n e r a l i z i n g beyond the two groups i n t h i s study. Further research should be conducted with l a r g e r sample s i z e s and more r i g i d l y c o n t r o l l e d experimental c o n d i t i o n s . On the basis of the f i n d i n g s i n t h i s study, there are a number of i m p l i c a t i o n s f o r pharmacy p r a c t i c e , c o n t i n u i n g pharmacy education and the a p p l i c a t i o n of w r i t t e n s i m u l a t i o n s i n pharmacy. Many would advocate an expanded r o l e f o r the pharmacist as a provider of primary care. However, i n t h i s study, there was a considerable range i n the performance of both experimental groups when responding to the primary care requests. The o v e r a l l q u a l i t y of health care i n t h i s area, as defined by the panel of content experts, was q u i t e low. In a d v i s i n g consumers, pharmacists asked few questions to determine the nature of the complaint and seldom gave advice on how to use the product recommended. Further research i s needed to e s t a b l i s h the cause of t h i s d e f i c i t i n performance. Is i t due to a genuine lack of knowledge or a t t r i b u t a b l e to a r o l e c o n f l i c t ? One question which i s often asked by a d u l t educators i n the health professions i s "Are our c o n t i n u i n g education programs r e a l l y making any d i f f e r e n c e s i n the q u a l i t y of care d e l i v e r e d to the p u b l i c ? " This study supports a weak a f f i r m a t i v e response to t h a t very important question. More st u d i e s are needed which measure the impact of programs on the actual p r a c t i c e of p r o f e s s i o n a l s . In a d d i t i o n , s t u d i e s are needed to determine the components of c o n t i n u i n g education a c t i v i t i e s which maximize the t r a n s f e r of l e a r n i n g to the everyday p r a c t i c e of -122-p r o f e s s i o n a l s . Continuing pharmacy education programs g e n e r a l l y use a pre- and p o s t - t e s t of f a c t u a l knowledge to evaluate the success of i n s t r u c t i o n . This i n v e s t i g a t i o n could not document any s t a t i s t i c a l l y s i g n i f i c a n t r e -l a t i o n s h i p between f a c t u a l knowledge scores and performance on the job. Therefore, c o n t i n u i n g educators should e x p l o i t every opportunity to i n v e s t i g a t e new e v a l u a t i v e instruments which may be b e t t e r p r e d i c t o r s of r e a l - l i f e performance. The w r i t t e n s i m u l a t i o n s developed f o r t h i s study provided evidence t h a t the technique i s a p p l i c a b l e to the e v a l u a t i o n of c o n t i n u i n g pharmacy education programs and can p r e d i c t r e a l - l i f e performance. Future work should continue to develop s i m u l a t i o n s i n t h i s and other areas of pharmacy p r a c t i c e and i n v e s t i g a t e t h e i r p r e d i c t i v e v a l i d i t y . I f a s u f f i c i e n t number of v a l i d w r i t t e n s i m u l a t i o n s can be con s t r u c t e d , they may be a useful device i n the process of competency determination. As w e l l , the e f f e c t i v e n e s s o f w r i t t e n simulations as an i n s t r u c t i o n a l a i d i n pharmacy education warrants study. -123-References Chapter S i x 1. Evaluations of Drug I n t e r a c t i o n s (Washington, D.C; American Pharmaceutical A s s o c i a t i o n , Second E d i t i o n , 1976) pp. 381-387, 422-429, 447-448. 2. Hansten, P.D., Drug I n t e r a c t i o n s ( P h i l a d e l p h i a : Lea and Febiger, Third E d i t i o n , 1975) pp. 59, 63, 264-266, 270, 319. 3. Jang, R., "Eva l u a t i o n of the Q u a l i t y of Drug-Related Services Provided by Community Pharmacists i n a M e t r o p o l i t a n Area." Ph.D. D i s s e r t a t i o n , Ohio State U n i v e r s i t y , 1971. -124-APPENDICES -125-APPENDIX A: MATERIALS RELATING TO THE DEVELOPMENT AND USE OF THE IN-STORE-ASSESSMENT PROBLEMS (ISAP'S). Page 1. In-Store-Assessment Problem One 2. In-Store-Assessment Problem Two 3. In-Store-Assessment Problem Three 4. In-Store-Assessment Problem Four 5. Table I I . Judges' Ratings of the Suggested Behaviours f o r Problem One 6. Table I I I . Judges' Ratings of the Suggested Behaviours f o r Problem Two 7. Table IV. Judges' Ratings of the Suggested Behaviours f o r Problem Three 8. Table V: Judges' Ratings of the Suggested Behaviours f o r Problem Four 9. Table V I I I . I n t e r c o r r e l a t i o n s of Judges' Ratings of the Behaviours i n a l l four ISAPs combined. 10. Performance C h e c k l i s t Problem One 11 Two 12 Three 13 Four 14. Performance C r i t e r i a Used In Ev a l u a t i n g A Pharmacist's Response to ISAP One 15. Performance C r i t e r i a Used i n Evaluating A Pharmacist's Response to ISAP Two 16. Performance C r i t e r i a Used i n Evaluating A Pharmacist's Response to ISAP Three 17. Performance C r i t e r i a Used i n Evaluating A Pharmacist's Response to ISAP Four 18. D i r e c t i o n s For The Observers 19. Observation Procedure 20. V i s i t Record Sheet 21. Table XXIV. C o r r e l a t i o n M atrix Representing R e l a t i o n s h i p s Between the Total Score, I t s Components, and the S i t u a t i o n a l Factors f o r the Pre-In-Store-Assessments f o r the Experimental Group 22. Table XXV. C o r r e l a t i o n M a t r i x Representing R e l a t i o n s h i p s Between the Total Score, i t s Components, and the S i t u a t i o n a l Factors f o r the Post-In-Store-Assessment f o r the Experimental Group 23. Table XXVI. C o r r e l a t i o n M a t r i x Representing R e l a t i o n s h i p s Between the Total Score, i t s components, and the S i t u a t i o n a l Factors f o r the Pre-In-Store-Assessment f o r the Control Group 24. Table XXVII. C o r r e l a t i o n M a t r i x Representing R e l a t i o n s h i p s Between the Total Score, i t s Components, and the S i t u a t i o n a l Factors f o r the Post-In-Store-Assessments f o r the Control Group -126-PROBLEM ONE For the past month, some f r i e n d s , a married couple have been v i s i t i n g you. I t has been p a r t l y pleasure and p a r t l y business as the husband has been combining some company business with t h i s v a c a t i o n . Your f r i e n d s are s t a y i n g w i t h you and you have been t a k i n g them on short t r i p s to other parts of the province. The husband, who i s i n h i s e a r l y 50's, has developed a sore t h r o a t w i t h i n the l a s t twenty-four hours. He says that he had t r o u b l e s l e e p i n g l a s t n i g h t due to h i s s t u f f e d nose d r i p p i n g down the back of his t h r o a t . His nose i s congested and he says t h a t he i s having t r o u b l e breathing through i t . He has a s l i g h t cough but there i s no congestion i n h i s chest. He does not appear to have a f e v e r . His ap p e t i t e i s good. He i s convinced that i t i s j u s t a c o l d and f e e l s i t does not warrant p h y s i c i a n treatment. Otherwise, he seems to be i n good health and good s p i r i t s . He i s a d i a b e t i c and he takes a white p i l l each morning with break-f a s t , you do not know which type. He has remarked t h a t h i s p h y s i c i a n only l a s t month s a i d t h a t he was doing f i n e and v/as i n good h e a l t h . You have been i n the area and decided that you would stop i n at the drugstore to get something f o r h i s c o l d . SITUATION: Find the cough and c o l d s e c t i o n , examine the products f o r some time,two or three minutes. Wait your chance and present the problem to the pharmacist. Ask the pharmacist: "What i s good f o r a c o l d ? " Answer the pharmacist's questions from the background m a t e r i a l provided. C a r e f u l l y observe the pharmacist's responses f o r recording l a t e r . Agree with h i s suggestions f o r a n o n - p r e s c r i p t i o n product and purchase a "reasonable" s i z e package. Thank the pharmacist f o r his/ h e r a s s i s t a n c e and leave. - 1 2 7 -PROBLEM TWO Your brother, v/ho i s t h i r t y - f i v e , has been v i s i t i n g you f o r almost three weeks. This i s h i s f i r s t t r i p to B.C. and you have been ta k i n g him on short t r i p s to show him some of the province. He i s due to leave i n two or three days. He has not " f e l t w e l l " f o r four or f i v e days. He has f e l t worse i n the l a s t day or so. You have not taken h i s temperature but t h i s morning he was hot to touch. He says that he has sore muscles and f e e l s t i r e d . Last evening he had the " c h i l l s " . His th r o a t i s sore and h i s nose i s congested. He f e e l s congested i n the chest and has a v i o l e n t cough and when coughing brings up some congestion. He has l i t t l e a p p e t i t e . He would l i k e something to r e l i e v e the cough, the plugged nose and the sore muscles. Since he i s l e a v i n g i n a few days, he says t h a t he w i l l see h i s p h y s i c i a n when he returns home i f the c o n d i t i o n i s no b e t t e r . He does not take any p r e s c r i p t i o n drugs or n o n - p r e s c r i p t i o n drugs. To the best of your knowledge he has no a l l e r g i e s . You are concerned and decide to ask f o r something to r e l i e v e the symptoms the next time you are near a drugstore. SITUATION: Enter the st o r e and f i n d the "cough and c o l d " s e c t i o n . Examine the products f o r two or three minutes then wait f o r your chance and present the problem to the pharmacist. Ask the pharmacist: "What i s good f o r a co l d ? " Answer the pharmacist's questions from the background m a t e r i a l provided. C a r e f u l l y observe the pharmacist's behaviour and responses f o r recording l a t e r . Agree with h i s suggestions f o r n o n - p r e s c r i p t i o n products and purchase a "reasonable" s i z e package. Thank the pharmacist f o r h i s / h e r a s s i s t a n c e and leave. -128-PROBLEM THREE Some good f r i e n d s , a couple, have been v i s i t i n g you f o r about three weeks. They have been s t a y i n g a t your home when they have not been taki n g short t r i p s around the province. You o c c a s i o n a l l y have accompanied them on these t r i p s , p a r t i c u l a r l y on week-ends and your days o f f . They have enjoyed t h e i r stay and cannot b e l i e v e t h a t the time has passed so q u i c k l y and they w i l l be l e a v i n g i n a few days. The w i f e , who i s i n her l a t e f o r t i e s , says that she has had a headache a l l day and although she f e e l s i t does not warrant a physician's treatment she would l i k e to take something f o r the pain. You do not have any pain r e l i e v e r s a t home. You are i n the area of t h i s drugstore so you decide to purchase some pain r e l i e v e r s f o r her. To the best of your knowledge your f r i e n d i s i n good health. She does not have a h i s t o r y of migraine headaches. She i s a d i a b e t i c and has been t a k i n g white t a b l e t s each morning with her breakf a s t f o r several years. She i s a l s o on a r e s t r i c t e d d i e t . She has remarked that her doctor s a i d only l a s t month t h a t she was i n "good h e a l t h " . To the best of your knowledge she has no a l l e r g i e s and she takes no n o n - p r e s c r i p t i o n drugs. SITUATION: Go to the drugstore and f i n d the s e c t i o n c o n t a i n i n g the pain r e l i e v e r s ( A s p r i n R , AnacinR, e t cetera.) Examine the products f o r a period of time (two or three minutes). Wait f o r your opportunity to present the problem to the pharmacist. Ask the pharmacist: "What i s the strongest pain r e l i e v e r I may buy without a p r e s c r i p t i o n ? " Answer the pharmacist's questions from the background i n f o r -mation provided. C a r e f u l l y observe the pharmacist's behaviours and responses f o r recording l a t e r . Agree with his suggestions f o r a non-p r e s c r i p t i o n product and purchase a "reasonable" s i z e package. Thank the pharmacist f o r his/ h e r a s s i s t a n c e and leave. -129-PROBLEM FOUR Your s i s t e r and brother-in-law are v i s i t i n g you f o r three weeks. The i r stay w i t h you i s almost over. Your s i s t e r has been complaining of pains i n her w r i s t s and f i n g e r s . She says t h a t i t i s more annoying than p a i n f u l and i t i s l i k e a " t i g h t n e s s " or " s t i f f n e s s " i n hands and w r i s t s . I t i s worse f i r s t t h i n g i n the morning. She says t h a t i t has been coming and going f o r "several months". Her husband says that she has been pro-c r a s t i n a t i n g about going to a doctor, much to h i s d i s a p p r o v a l . She has been t a k i n g two or three a s p r i n ^ t a b l e t s a day f o r the pain but she f e e l s t h a t they are not r e a l l y much good. In a d d i t i o n to the " s t i f f n e s s " she has mentioned t h a t she f e e l s as i f she i s t i r i n g f a s t e r than u s u a l . She i s t h i r t y - s i x years o l d . You decide t h a t s i n c e you are i n the drugstore l o o k i n g f o r some-th i n g e l s e that you w i l l i n q u i r e about a stronger pain r e l i e v e r . To the best of your knowledge your s i s t e r takes no p r e s c r i p t i o n drugs and no no n - p r e s c r i p t i o n medications except a s p r i n R . She has no a l l e r g i e s . She l a s t saw a p h y s i c i a n about a year ago concerning a c o l d and t h i s problem was not discussed a t th a t time. SITUATION: Go to the st o r e and f i n d the s e c t i o n c o n t a i n i n g the pain r e l i e v e r s . Examine the products f o r two or three minutes. Then, when your opportunity to present the problem comes up, speak to the pharmacist on duty. Ask: "What i s the strongest pain r e l i e v e r I may purchase without a p r e s c r i p t i o n ? " Answer the pharmacist's questions from the background information provided. C a r e f u l l y observe the pharmacist's responses and behaviours f o r recording l a t e r . Agree with h i s suggestions and purchase a "reasonable" s i z e package of a n o n - p r e s c r i p t i o n pain r e l i e v e r i f he recommends one. Thank the pharmacist f o r his/ h e r a s s i s t a n c e and leave. -130-TABLE I I JUDGES' RATINGS OF THE INDIVIDUAL BEHAVIOURS SUGGESTED FOR ISAP ONE In response to the primary care request "What i s good f o r a c o l d ? " should a pharmacist: Q3-Q1 BEHAVIOURS Mean S.D. Range 2 * 1. Ask who i t was f o r . 4 .90 .3162 1 .00 .00 * 2. Inquire about the symptoms. 5 .00 .0000 .00 .00 * 3. Ask about f e v e r . 4 .40 .6992 2 .00 .50 * 4. Ask f o r a d e s c r i p t i o n of any cough. 4 .40 .6992 2 .00 .50 * 5. Ask about the dur a t i o n of the symptoms. 4 .50 .5270 1 .00 .50 * 6. Ask about a l l e r g i e s . 4 .20 .7888 2 .00 .50 * 7. Ask about concurrent use of p r e s c r i p t i o n drugs. 4 .70 .4830 1 .00 .50 * 8. Ask about concurrent use of no n - p r e s c r i p t i o n drugs. 4 .50 .7071 2 .00 .50 * 9. D i r e c t you to a c l e r k f o r a s s i s t a n c e . 1 .09 .3162 1 .00 .00 10. D i r e c t you to a pharmacist f o r a s s i s t a n c e . 3 .00 .0000 .00 .00 *11. Without asking about the symptoms d i r e c t you to the "cough and c o l d " s e c t i o n out f r o n t . 1 .29 .4839 1 .00 .50 *12. Without asking about the symptoms, recommend a product. 1 .50 .7071 2 .00 .50 *13. Without asking about the symptoms, d i r e c t you to a ph y s i c i a n f o r a s s i s t a n c e . 1 .90 .3162 1 .00 .50 *14. A f t e r i n q u i r i n g about the symptoms d i r e c t you to a p h y s i c i a n . 2 .50 .5270 1 .00 .50 -131-TABLE I I . cont. BEHAVIOURS Mean S.D. Range 2 *15. Suggest the c o n d i t i o n i s not serious enough to warrant treatment with a non-p r e s c r i p t i o n product. 2.00 .0000 ..00 .00 *16. Recommend a product c o n t a i n i n g a sympathomimetic amine. 2.20 .9189 2.00 1.00 **17. A f t e r i n q u i r i n g about the symp-toms, recommended a product. 3.90 .5676 2.00 .00 **18. Recommend a t o p i c a l deconges-tant or an o r a l product con-t a i n i n g no sympathomimetic amines. 4.09 .5676 2.00 .00 **19. Recommended a sugarless product a f t e r i n q u i r i n g about symptoms. 4.50 .7071 2.00 .50 **20. Suggest seeing a p h y s i c i a n i f symptoms p e r s i s t . 4.50 .5270 1.00 .00 *21. I n d i c a t e how to use the pro-duct. 4.59 .5163 1.00 .50 *22. I n d i c a t e when to take the product. 4.59 .5163 1.00 .50 *23. I n d i c a t e how long to take product. 4.59 .5163 1.00 .50 *24. E x p l a i n why i t i s appropriate or i n a p p r o p r i a t e to use a p a r t i c u l a r product given the symptoms and other f a c t o r s ( p r e s c r i p t i o n drugs taken, e t c . ) . 4.09 .5676 2.00 .00 25. Come out from behind the d i s -pensary counter to discus s your problem or to recommend a product. 3.70 .6324 2.00 .50 *Used as a f i n a l c r i t e r i o n i n the e v a l u a t i o n of the pharmacists. **These four behaviours were combined i n t o the f o l l o w i n g two c r i t e r i a : i . A f t e r i n q u i r i n g about the symptoms, recommended a t o p i c a l deconges-t a n t or an o r a l sugarless product which does not conta i n a sympatho-mimetic amine and suggest c o n s u l t i n g a p h y s i c i a n i f symptoms p e r s i s t , i i . A f t e r i n q u i r i n g about the symptoms, recommend a t o p i c a l decongestant or an o r a l sugarless product c o n t a i n i n g no sympathomimetic amine. -132-TABLE I I I JUDGES' RATINGS OF THE INDIVIDUAL BEHAVIOURS SUGGESTED FOR ISAP. TWO In response to the primary care request "What i s good f o r a c o l d ? " should a pharmacist: _J93b91 BEHAVIOURS Mean S.D. Range 2 *1 *2 *3 *4 *6 *7 10 *11 Ask who i t was f o r . 4.90 .3162 1.00 .00 Ask f o r a d e s c r i p t i o n of the ailment. 5.00 .0000 .00 .00 Ask about a f e v e r . 4.90 .3162 1.00 .00 Ask f o r a d e s c r i p t i o n of any cough. 4.90 .3162 1.00 .00 Ask about the dur a t i o n of the symptoms. 4.79 .4216 1.00 .00 Ask about a l l e r g i e s . 4.40 .8432 2.00 .50 Ask about the use of p r e s c r i p -t i o n drugs. 4.79 .4216 1.00 .00 Ask about the use of non-p r e s c r i p t i o n drugs. 4.50 .6992 2.00 .00 D i r e c t you to a c l e r k f o r a s s i s t a n c e . 1.00 .0000 .00 .00 Ask another pharmacist to help. 3.50 .5270 1.00 .50 Without asking about the symp-toms d i r e c t you to the "cough and c o l d " s e c t i o n out f r o n t . 1.20 .4216 1.00 .00 r12. Without asking about the symp-toms recommend a product. 1.44 .7264 2.00 .50 r13. Without asking about symptoms, suggest t h a t you see a p h y s i c i a n . 1.79 .4216 1.00 .00 e14. Suggest the c o n d i t i o n i s not ser i o u s and, t h e r e f o r e , does not warrant treatment with a n o n - p r e s c r i p t i o n product. 1.50 .7071 2.00 .50 r15. Without i n q u i r i n g about the symptoms, recommend a product to r e l i e v e the symptoms and suggest you see a p h y s i c i a n . 1.70 .8232 2.00 .50 -133-TABLE I I I Cont. Q3-Q1 BEHAVIOURS Mean S.D. Range 2 **16. Recommend a product a f t e r i n -q u i r i n g about the symptoms. 3.79 1.0327 3.00 1.00 **17. A f t e r i n q u i r i n g about the symp-toms, recommend seeing a p h y s i c i a n . **18. Inquire about the symptoms, rec-omend a product and suggest seeing a p h y s i c i a n . 19. A f t e r i n q u i r i n g about symptoms, recommend a cough product. 20. A f t e r i n q u i r i n g about the symp-toms recommend a t o p i c a l de-congestant or an o r a l c o l d product. *21. I n d i c a t e how to use the product. *22. Indicate when to use the pro-duct. *23. I n d i c a t e how long to take product. *24. E x p l a i n why i t i s appropriate or i n a p p r o p r i a t e to use a p a r t i c u l a r product given the symptoms d e s c r i b e d , pre-s c r i p t i o n drugs being used et c . 4.20 .6324 2.00 .50 .. 25. Come out from behind the d i s -pensary counter when d i s -cussing your problem. 3.40 .6992 2.00 .50 26: Caution about side e f f e c t s . 4.44 .5270 1.00 .50 *27. A f t e r i n q u i r i n g about the symp-toms, recommending a product only. 2.00 .0000 .00 .00 *Used as a f i n a l c r i t e r i o n i n the e v a l u a t i o n of the pharmacists. **These three behaviours were comgined i n t o the f o l l o w i n g two c r i t e r i a : i . A f t e r i n q u i r i n g about the symptoms, recommend a product, suggest c o n s u l t i n g a p h y s i c i a n , i i . A f t e r i n q u i r i n g about the symptoms suggest c o n s u l t i n g a p h y s i c i a n , 4.09 .8755 2.00 1.00 4.29 .8232 2.00 .50 3.59 .6992 2.00 .50 3.50 .7071 2.00 .50 4.50 .5270 1.00 .50 4.50 .5270 1.00 .50 4.50 .5270 1.00 .50 -134-TABLE IV JUDGES' RATINGS OF THE INDIVIDUAL BEHAVIOURS SUGGESTED FOR ISAP THREE In response to the primary care request "What i s the strongest pain r e l i e v e r I may purchase without a pre-s c r i p t i o n ? " should a pharmacist: Q3-Q1 BEHAVIOURS Mean S.D. Range 2 * 1. Ask who i t was f o r . 5 .00 .0000 .00 .00 * 2. Ask what the product i s to be used f o r . 4 .79 .4216 1 .00 .00 * 3. Ask f o r a d e s c r i p t i o n of the pain. 4 .79 .4216 1 .00 .00 * 4. Ask about the dur a t i o n of the pain:. 4 .90 .3162 1 .00 .00 * 5. Ask about a l l e r g i e s . 4 .40 .8432 2 .00 .50 * 6. Ask about concurrent use of p r e s c r i p t i o n drugs. 4 .70 .6749 2 .00 .00 * 7: Ask about concurrent use of n o n - p r e s c r i p t i o n drugs. 4 .70 .6749 2 .00 .00 * 8. Ask a c l e r k to help. 1 .20 .4216 1 .00 .00 9. Ask another pharmacist to help. 3 .00 .0000 .00 .00 *10. Without asking about the symp-toms, d i r e c t you to the ap-p r o p r i a t e s e c t i o n out f r o n t . 1 .20 .4216 1 .00 .00 *11. Without asking about the symp-toms, suggest a product. 1 .40 .5639 1 .00 .50 *12. Without asking about the symp-toms, suggest c o n s u l t i n g a p h y s i c i a n . 1 .70 .4830 1 .00 .50 *13. Suggest c o n d i t i o n not serious and, t h e r e f o r e , does not warrant treatment with a non-p r e s c r i p t i o n product. 1 .70 .4830 1 .00 .50 *14. Suggest c o n s u l t i n g a p h y s i c i a n a f t e r t a k i n g a h i s t o r y . 2 .79 .9189 3 .00 .50 *15. Suggest a n o n - p r e s c r i p t i o n product c o n t a i n i n g ASA. 2 .09 .9944 3 .00 1.00 -135-TABLE IV Cont. BEHAVIOURS Mean S.D. Range 2 16. Recommend a product c o n t a i n i n g codeine. 3.00 .0000 .00 .00 **17. Suggest a n o n - p r e s c r i p t i o n pro-duct a f t e r i n q u i r i n g about the symptoms. 4.20 .6324 2.00 .50 **18. Suggest a n o n - p r e s c r i p t i o n pro-duct which does not conta i n ASA. 3.90 .7378 2.00 .50 **19. Suggest seeing a p h y s i c i a n i f c o n d i t i o n p e r s i s t s . 4.79 .4216 1.00 .00 *20. I n d i c a t e how long to use the product. 4.59 .5163 1.00 .50 *21. I n d i c a t e when to take the product. 4.50 .5270 1.00 .50 *22. I n d i c a t e how to take the pro-duct. 4.50 .5270 1.00 .50 *23. E x p l a i n why a p a r t i c u l a r pro-duct was appropriate or i n -appropriate to use i n t h i s p a r t i c u l a r case. 4.29 .6749 2.00 .50 24. Come out from behind the coun-t e r to d i s c u s s your problem or to discus s a product. 3.70 .6749 2.00 .50 25. Ask about concurrent disease s t a t e s . 4.40 .6992 2.00 .50 *Used as a f i n a l c r i t e r i o n i n the e v a l u a t i o n of the pharmacists. **These three behaviours were combined i n t o the 2 c r i t e r i a : i . A f t e r i n q u i r i n g about the symptoms, recommend a product not contain-ing ASA and suggest c o n s u l t i n g , a p h y s i c i a n i f the symptoms p e r s i s t , i i . A f t e r i n q u i r i n g about the symptoms, recommend a product not contain-ing ASA. -136-TABLE V. JUDGES' RATINGS OF THE INDIVIDUAL BEHAVIOURS SUGGESTED FOR ISAP FOUR In response to the primary care request "What i s the strongest pain r e l i e v e r I may purchase without a pre-s c r i p t i o n ? " should the pharmacist: BEHAVIOURS Mean S.D. Range 2 * 1. Ask who i t i s f o r * 2. Ask what the product i s needed f o r . * 3. Ask f o r a d e s c r i p t i o n of the ailment. * 4. Ask about the d u r a t i o n of the pain. * 5. Ask about a l l e r g i e s . * 6. Ask about the concurrent use of p r e s c r i p t i o n drugs. * 7. Ask about the concurrent use of no n - p r e s c r i p t i o n drugs. * 8. Ask a c l e r k to help. 9. Ask another pharmacist to help. *10. D i r e c t you to the appropriate s e c t i o n out f r o n t . *11. Recommend a product without i n -q u i r i n g about the symptoms. *12. Suggest c o n s u l t i n g a p h y s i c i a n without i n q u i r i n g about the symptoms. *13. Suggest t h a t the c o n d i t i o n i s not s e r i o u s enough to warrant treatment with a non-prescrip-t i o n medication. *14. Recommend a product not contain-ing ASA. *15. Recommend a product c o n t a i n i n g codeine. 4.90 .3162 1.00 .00 4.90 .3162 1.00 .00 5.00 .0000 .00 .00 4.90 .3162 1.00 .00 4.29 .9486 2.00 1.00 4.70 .6749 2.00 .00 4.70 .6749 2.00 .00 1.09 .3162 1.00 .00 3.00 .0000 .00 .00 1.29 .4839 1.00 .50 1.29 .4830 1.00 .50 1.59 .5163 1.00 .50 1.20 .4216 1.00 .00 1.90 .7378 2.00 .50 2.40 .6992 1.00 .50 -137-TABLE V Cont. Q3-Q1 BEHAVIOURS Mean S.D. Range ~ 2 ~ 3.77 .8333 2.00 .50 4.09 .7378 2.00 .50 4.29 .6749 2.00 .00 4.29 .6749 2.00 .00 4.59 .5163 . 1.00 .50 4.59 .5163 1.00 .50 4.59 .5163 1.00 .50 4.40 .5163 1.00 .50 3.79 .6324 1.00 .50 **16. A f t e r i n q u i r i n g about the symp-toms, recommend a product. **17. Recommend a product c o n t a i n i n g ASA. **18. Recommend c o n s u l t i n g a p h y s i c i a n a f t e r i n q u i r i n g about the symp-toms . **19. Recommend i n c r e a s i n g the dose of ASA and c o n s u l t i n g a p h y s i c i a n about the c o n d i t i o n . *20. Indicate how long to use pro-duct. *21. Indicate when to take product. *22. Indicate how to take product. *23. Explain why i t i s appropriate or i n a p p r o p r i a t e to use a par-t i c u l a r product given the symp-toms, p r e s c r i p t i o n drugs used, et c . 24. Come from behind the dispensary counter to discuss your problem or to discuss a p a r t i c u l a r pro-duct. *Used as a f i n a l c r i t e r i o n i n the e v a l u a t i o n of the pharmacists. **These fo u r behaviours were combined i n t o the f o l l o w i n g two c r i t e r i a : i . A f t e r i n q u i r i n g about the symptoms, suggest i n c r e a s i n g the dose of ASA or recommending a product c o n t a i n i n g ASA (but not codeine) and suggest seeing a p h y s i c i a n , i i . A f t e r i n q u i r i n g .about the symptoms, suggest c o n s u l t i n g a p h y s i c i a n . TABLE VI I I INTERCORRELATIONS OF JUDGES'RATINGS OF THE BEHAVIOURS IN ALL FOUR ISAP's COMBINED Judge 1 1 3 4 5. 6 7 8 9 10 1 .7716 .8771 .7716 .8565 .8272 .8725 .9089 .8054 .8124 2 .7252 .7244 .8032 .8258 .7908 .8300 .8067 .7278 3 .6959 .8612 .8014 .8223 .8245 .8138 .7724 4 .7986 .7747 .8119 .7947 .8602 .8106 5 .7648 .8353 .8643 .8344 .8323 6 .8199 .8073 .7912 ,7596 7 .88 7 .7967 .8428 8 .8151 .8067 9 .7959 10 Mean c o r r e l a t i o n = 0.8099 -139-PERFORMANCE CHECKLIST PROBLEM 1 In response to the qu e s t i o n , "what i s good f o r a c o l d ? " Did the Pharmacist: ( I n d i c a t e YES or NO) 1. Ask who i t was f o r 2. Inquire about the symptoms 3. Ask about fever 4. Ask f o r a d e s c r i p t i o n of any cough 5. Ask about the d u r a t i o n of the symptoms 6. Ask about a l l e r g i e s 7. Ask about concurrent use of p r e s c r i p t i o n drugs 8. Ask about concurrent use of n o n - p r e s c r i p t i o n drugs 9. D i r e c t you to a c l e r k f o r a s s i s t a n c e 10. Without asking about the symptoms, d i r e c t you to the "cough and c o l d " s e c t i o n out f r o n t . 11. Without asking about symptoms e t c . , recommend a product 12. Without asking about the symptoms e t c . , d i r e c t you to a phy-s i c i a n f o r assistance. .: 13. A f t e r i n q u i r i n g about the symptoms e t c . , d i r e c t you to a p h y s i c i a n 14. Suggest t h a t the c o n d i t i o n i s not serious enough to warrant treatment with a n o n - p r e s c r i p t i o n product 15. Recommend a product c o n t a i n i n g a sympathomimetic amine 16. A f t e r i n q u i r i n g about the symptoms e t c . , recommend a product 17. Recommend a t o p i c a l decongestant or an o r a l product c o n t a i n i n g no sympathomimetic amines 18. Recommend a sugarless cough product a f t e r i n q u i r i n g about symptoms 19. Suggest seeing a p h y s i c i a n i f the symptoms p e r s i s t 20. I n d i c a t e how to use the product 21. I n d i c a t e when to take the product 22. I n d i c a t e how long to take the product 23. E x p l a i n why i t i s appropriate or i n a p p r o p r i a t e to use a par-t i c u l a r product given the symptoms and other f a c t o r s (pre-s c r i p t i o n , drugs taken etc.) 24. Come out from behind the dispensary counter to discuss your problem PROBLEM 1 contd. Name of product recommended P r i c e of product recommended Comments -141 -PERFORMANCE CHECKLIST  PROBLEM 2 In response to the qu e s t i o n , "What i s good f o r a co l d ? " Did the Pharmacist ( I n d i c a t e YES or NO) 1. Ask who i t was f o r 2. Ask f o r a d e s c r i p t i o n of the ailment 3. Ask about a f e v e r 4. Ask f o r a d e s c r i p t i o n of any cough 5. Ask about the dur a t i o n of the symptoms 6. Ask about a l l e r g i e s 7. . Ask about concurrent use of p r e s c r i p t i o n drugs 8. Ask about concurrent use of n o n - p r e s c r i p t i o n drugs 9. D i r e c t you to a c l e r k f o r a s s i s t a n c e 10. Without asking about the symptoms d i r e c t you to the "cough and c o l d " s e c t i o n out f r o n t 11. Without asking about the symptoms recommend a product 12. Without asking about the symptoms suggest t h a t you see a ph y s i c i a n 13. Suggest t h a t the c o n d i t i o n i s not serious and, t h e r e f o r e , does not warrant treatment with a n o n - p r e s c r i p t i o n product 14. Without i n q u i r i n g about the symptoms, recommend a product to r e l i e v e the symptoms and suggest you see a p h y s i c i a n 15. Recommend a product a f t e r i n q u i r i n g about the symptoms 16. A f t e r i n q u i r i n g about the symptoms recommend seeing a p h y s i c i a n 17. Inquire about the symptoms, recommend a product and suggest seeing a p h y s i c i a n 18. Recommend a cough product a f t e r i n q u i r i n g about symptoms 19. Recommend a t o p i c a l decongestant or an o r a l c o l d product a f t e r i n q u i r i n g about symptoms 20. I n d i c a t e how to use the product 21. I n d i c a t e when to use the product 22. I n d i c a t e how long to use the product 23. E x p l a i n why i t i s appropriate or i n a p p r o p r i a t e to use a par-t i c u l a r product given the symptoms de s c r i b e d , p r e s c r i p t i o n drugs being used e t c . 24. Come out from behind the dispensary counter when d i s c u s s i n g your problem -142-PROBLEM 2.contd. 25. Caution about s i d e e f f e c t s Name of product recommended P r i c e of product recommended Comments -143-PERFORMANCE CHECKLIST PROBLEM 3 In response to the qu e s t i o n , "What i s the strongest pain r e l i e v e r I may purchase without a p r e s c r i p t i o n ? " Did the pharmacist: ( I n d i c a t e YES or NO) 1. Ask who i t was f o r 2. Ask what the product i s to be used f o r 3. Ask f o r a d e s c r i p t i o n of the pain 4. Ask about the dur a t i o n of the pain (How long has the person had the headache?) 5. Ask about a l l e r g i e s 6. Ask about concurrent use of p r e s c r i p t i o n drugs 7. Ask about concurrent use of n o n - p r e s c r i p t i o n drugs 8. Ask a c l e r k to help 9. Without asking about the symptoms d i r e c t you to the appropriate s e c t i o n out f r o n t 10. Without asking about the symptoms, suggest a product 11. Without asking about the symptoms, suggest c o n s u l t i n g a ph y s i c i a n 12. Suggest the c o n d i t i o n i s not ser i o u s and, t h e r e f o r e , does not warrant treatment with a n o n - p r e s c r i p t i o n product 13. Suggest c o n s u l t i n g a p h y s i c i a n a f t e r t a k i n g a h i s t o r y 14. Suggest a n o n - p r e s c r i p t i o n product c o n t a i n i n g ASA 15. Suggest a n o n - p r e s c r i p t i o n product a f t e r i n q u i r i n g about symptoms 16. Suggest a n o n - p r e s c r i p t i o n product which does not contain ASA 17. Suggest seeing a p h y s i c i a n i f c o n d i t i o n p e r s i s t s 18. I n d i c a t e how long to use the product 19. I n d i c a t e when to take the product 20. I n d i c a t e how to take the product 21. E x p l a i n why a p a r t i c u l a r product was appropriate or inap-p r o p r i a t e to use i n t h i s p a r t i c u l a r case 22. Come out from behind the counter to discuss your problem or to discu s s a product 23. Ask about concurrent disease s t a t e s PROBLEM 3 contd. Name of product recommended P r i c e of product recommended Comments -145-PERFORMANCE CHECKLIST  PROBLEM 4 In response to the ques t i o n , "What i s the strongest pain r e l i e v e r I may purchase without a p r e s c r i p t i o n ? " Did the Pharmacist: ( I n d i c a t e YES or No) 1. Ask who i t i s f o r 2. Ask what the product i s needed f o r 3. Ask f o r a d e s c r i p t i o n of the ailment 4. Ask about the duration of the pain 5. Ask about a l l e r g i e s 6. Ask about concurrent use of p r e s c r i p t i o n drugs 7. Ask about concurrent use of n o n - p r e s c r i p t i o n drugs 8. Ask a c l e r k to help 9. D i r e c t you to the appropriate s e c t i o n out f r o n t 10. Recommend a product without i n q u i r i n g about the symptoms 11. Suggest c o n s u l t i n g a p h y s i c i a n without i n q u i r i n g about the symptoms 12. Suggest t h a t the c o n d i t i o n i s not serious enough to warrant treatment with a n o n - p r e s c r i p t i o n medication 13. Recommend a product not c o n t a i n i n g ASA 14. Recommend a product c o n t a i n i n g codeine 15. A f t e r i n q u i r i n g about the symptoms, recommending a product 16. Recommend a product c o n t a i n i n g ASA 17. Recommend c o n s u l t i n g a p h y s i c i a n a f t e r i n q u i r i n g about the symptoms 18. Recommend i n c r e a s i n g the dose of ASA and c o n s u l t i n g a p h y s i c i a n about the c o n d i t i o n 19. I n d i c a t e how long to use the product 20. I n d i c a t e when to take the product 21. I n d i c a t e how to take the product 22. E x p l a i n why i t i s appropriate or i n a p p r o p r i a t e to use a par-' t i c u l a r product given the symptoms, p r e s c r i p t i o n drugs used e t c . 23. Come out from behind the dispensary counter to discuss your problem or to discuss a p a r t i c u l a r product Name of product recommended P r i c e of product recommended Comments -146-PERFORMANCE CRITERIA USED IN EVALUATING A PHARMACIST'S RESPONSE TO ISAP ONE (COLD APPROPRIATE): "What i s good f o r a c o l d ? " DATA GATHERING 1. Ask who i t was f o r 2. Inquire about the symptoms 3. Ask about f e v e r 4. Ask f o r a d e s c r i p t i o n of any cough 5. Ask about the d u r a t i o n of the symptoms 6. Ask about a l l e r g i e s 7. Ask about concurrent use of p r e s c r i p t i o n drugs 8. Ask about concurrent use of n o n - p r e s c r i p t i o n drugs INAPPROPRIATE RECOMMENDATIONS 9. D i r e c t you to a c l e r k f o r a s s i s t a n c e 10. Without asking about the symptoms, d i r e c t you to the "cough and c o l d " s e c t i o n out f r o n t 11. Without asking about symptoms e t c . , recommend a product 12. Without asking about the symptoms e t c . , d i r e c t you to a p h y s i c i a n f o r a s s i s t a n c e 13. A f t e r i n q u i r i n g about the symptoms e t c . , d i r e c t you to a p h y s i c i a n 14. Suggest t h a t the c o n d i t i o n i s not serious enough to warrant treatment with a n o n - p r e s c r i p t i o n product 15. Recommend a product co n t a i n i n g a sympathomimetic amine APPROPRIATE RECOMMENDATIONS 16. A f t e r i n q u i r i n g about the symptoms, recommend a t o p i c a l decongestant or an o r a l sugarless product which does not contain a sympathomimetic amine and suggest c o n s u l t i n g a p h y s i c i a n i f symptoms p e r s i s t 17. A f t e r i n q u i r i n g about the symptoms, recommend a t o p i c a l decongestant or an o r a l sugarless product c o n t a i n i n g no sympathomimetic amines DRUG USE COUNSELLING 18. In d i c a t e how to use the product 19. I n d i c a t e when to take the product 20. I n d i c a t e how long to take the product 21. E x p l a i n why i t i s appropriate or i n a p p r o p r i a t e to use a p a r t i c u l a r product given the symptoms and other f a c t o r s ( p r e s c r i p t i o n drugs taken etc.) -147-PERFORMANGE CRITERIA USED IN EVALUATING A PHARMACIST'S RESPONSE TO ISAP TWO (COLD INAPPROPRIATE): "What i s good f o r a c o l d ? " DATA GATHERING 1. Ask who i t was f o r 2. Ask f o r a d e s c r i p t i o n of the ailment 3. Ask about a f e v e r 4. Ask f o r a d e s c r i p t i o n o f any cough 5. Ask about the duration of the symptoms 6. Ask about a l l e r g i e s 7. Ask about concurrent use of p r e s c r i p t i o n drugs 8. Ask about concurrent use of n o n - p r e s c r i p t i o n drugs INAPPROPRIATE RECOMMENDATIONS 9. D i r e c t you to a c l e r k f o r a s s i s t a n c e 10. Without asking about the symptoms d i r e c t you to the "cough" and " c o l d " s e c t i o n out f r o n t 11. Without asking about the symptoms recommend a product 12. Without asking about the symptoms suggest that you see a.physician 13. Suggest that the c o n d i t i o n i s not serious and, t h e r e f o r e , does not warrant treatment with a n o n - p r e s c r i p t i o n product. 14. Without i n q u i r i n g about the symptoms, recommend a product to r e l i e v e the symptoms and suggest you see a p h y s i c i a n 15. A f t e r i n q u i r i n g about the symptoms, recommending a product only APPROPRIATE RECOMMENDATIONS 16. A f t e r i n q u i r i n g about the symptoms, recommend a product, suggest c o n s u l t i n g a p h y s i c i a n 17. A f t e r i n q u i r i n g about the symptoms, suggest c o n s u l t i n g a p h y s i c i a n DRUG USE COUNSELLING 18. I n d i c a t e how to use the product 19. I n d i c a t e when to use the product 20. I n d i c a t e how long to use the product 21. E x p l a i n why i t i s appropriate or i n a p p r o p r i a t e to use a p a r t i c u l a r product -148-PERFORMANCE CRITERIA USED IN EVALUATING A PHARMACIST'S RESPONSE TO ISAP THREE (PAIN APPROPRIATE): "What i s the strongest pain r e l i e v e r I may purchase without a p r e s c r i p t i o n ? " DATA GATHERING 1. Ask who i t was f o r 2. Ask what the product i s to be used f o r 3. Ask f o r a d e s c r i p t i o n of the pain 4. Ask about the duration of the pain (how long has the person had the headache) 5. Ask about a l l e r g i e s 6. Ask about concurrent use of p r e s c r i p t i o n drugs 7. Ask about concurrent use of n o n - p r e s c r i p t i o n drugs INAPPROPRIATE RECOMMENDATIONS 8. Ask a c l e r k to 9. Without asking s e c t i o n 10. Without 11. Without 12. Suggest warrant 13. Suggest 14. Suggest help about the symptoms d i r e c t you to the appropriate out f r o n t asking about the asking about the the c o n d i t i o n i s treatment with a symptoms, suggest a product symptoms, suggest c o n s u l t i n g a p h y s i c i a n not s e r i o u s and, t h e r e f o r e , does not no n - p r e s c r i p t i o n product c o n s u l t i n g a p h y s i c i a n a f t e r t a k i n g a h i s t o r y a n o n - p r e s c r i p t i o n product c o n t a i n i n g ASA APPROPRIATE RECOMMENDATIONS 15. A f t e r i n q u i r i n g about the symptoms, recommend a product not con-t a i n i n g ASA and suggest c o n s u l t i n g a p h y s i c i a n i f the symptoms p e r s i s t 16. A f t e r i n q u i r i n g about the symptoms, recommend a product not con-t a i n i n g ASA DRUG USE COUNSELLING 17. I n d i c a t e how long to use the product 18. I n d i c a t e when to take the product 19. I n d i c a t e how to take the product 20. Exp l a i n why a p a r t i c u l a r product was appropriate or i n a p p r o p r i a t e to use i n t h i s p a r t i c u l a r case -149-PERFORMANCE CRITERIA USED IN EVALUATING A PHARMACIST'S RESPONSE TO ISAP FOUR (PAIN INAPPROPRIATE): "What i s the strongest pain r e l i e v e r I may purchase without a p r e s c r i p t i o n ? " DATA GATHERING 1. Ask who i t i s f o r 2. Ask what the product i s needed f o r 3. Ask f o r a d e s c r i p t i o n of the ailment 4. Ask about the duration of the pain 5. Ask about a l l e r g i e s 6. Ask about concurrent use of p r e s c r i p t i o n drugs 7. Ask about concurrent use of n o n - p r e s c r i p t i o n drugs INAPPROPRIATE RECOMMENDATIONS 8. Ask a c l e r k to help 9. D i r e c t you to the appropriate s e c t i o n out f r o n t 10. Recommend a product without i n q u i r i n g about the symptoms 11. Suggest c o n s u l t i n g a p h y s i c i a n without i n q u i r i n g about the symptoms 12. Suggest that the c o n d i t i o n i s not serious enough to warrant t r e a t -ment with a n o n - p r e s c r i p t i o n medication 13. Recommend a product not c o n t a i n i n g ASA 14. Recommend a product c o n t a i n i n g codeine APPROPRIATE RECOMMENDATIONS 15. A f t e r i n q u i r i n g about the symptoms, suggest i n c r e a s i n g the dose of ASA or recommending a product c o n t a i n i n g ASA (but not codeine) and suggest seeing a p h y s i c i a n 16. A f t e r i n q u i r i n g about the symptoms, suggest c o n s u l t i n g a p h y s i c i a n DRUG USE COUNSELLING 17. Indicate how long to use the product 18. I n d i c a t e when t o take the product 19. Indicate how to take the product 20. E x p l a i n why i t i s appropriate or ina p p r o p r i a t e to use a p a r t i c u l a r product given the symptoms, p r e s c r i p t i o n drugs used e t c . -150-DIRECTIONS FOR OBSERVERS General Procedure Present the s i t u a t i o n to each of the pharmacists assigned to you. I t i s most important t h a t you s t r i v e to make your presentations identical.', f o r each pharmacist. Memorize the background information and the wording of the b a i s c questions and rehearse the f u l l p r e s e n t a t i o n before your f i r s t v i s i t so you can concentrate on observing the pharmacist's response. For each v i s i t , f o l l o w the standard f i e l d observation procedure (see attached s h e e t ) . Be accurate i n your c l o c k readings when timing the s i t u a t i o n a l f a c t o r s . Be sure to complete a l l parts of the V i s i t Record Sheet. Place completed V i s i t Record Sheets, purchased items and a l l r e c e i p t s i n the manila envelope f o r each pharmacist. Dress and Manner During V i s i t s To prevent unusual r e a c t i o n s to your dress you are requested to wear c l o t h i n g which i s n e i t h e r conspicuously poor nor conspicuously glamourous. P r a c t i c e a p o l i t e , i n t e r e s t e d manner f o r your p r e s e n t a t i o n to the pharmacist. Accept h i s statements agreeably and comply with the pharmacist's recommendations. I f a disagreeable s i t u a t i o n a r i s e s r e a c t with s u r p r i s e d s i l e n c e without comment, and leave. Verbal exchanges may lead to statements which would j e o p a r d i z e the e n t i r e study. I f your a u t h e n t i c i t y as a customer i s challenged leave the pharmacy and be sure to r e p o r t such challenges on your observation sheet. I d e n t i f y i n g the Pharmacist I t i s c r i t i c a l to the study t h a t you present the s i t u a t i o n s only to the pharmacists assigned to you. Assignments are based upon the best information a v a i l a b l e , but s i n c e pharmacists do change jobs you w i l l need to check the l o c a t i o n of the pharmacist before your f i r s t v i s i t . You w i l l be s u p p l i e d w i t h the name, approximate age, p h y s i c a l d e s c r i p t i o n , l a s t known working address and telephone number of each pharmacist. J u s t before your v i s i t , c a l l the pharmacy. I f a c l e r k answers, ask f o r the pharmacist by name. Attempt to f i n d out when the pharmacist i s working, or i f the pharmacist i s working elsewhere, where -151-his new p o s i t i o n i s . I f asked why, say he helped you before and you have another problem. I f the assigned pharmacist answers the telephone, say your neigh-bour t o l d you he/she could help you and from the attached l i s t of s p e c i a l q u e s tions, ask one. A l s o , ask i f he/she w i l l be i n the st o r e so you w i l l know who to t a l k to when you go to buy the item. When you enter the store ask the c l e r k i f the assigned pharmacist i s i n . Try to get her to p o i n t him or her out to.you. Look f o r a man or a woman about the r i g h t age, wearing some type of white j a c k e t . A l s o look f o r an i d e n t i f y i n g name tag or c e r t i f i c a t e . I f there i s no c l e r k , use a l l the v i s i b l e clues you can and c a r e f u l l y note the pharmacist's d e s c r i p t i o n f o r the record sheet. I f you are not sure you have the r i g h t pharmacist do NOT attempt the p r e s e n t a t i o n . I f you are unsuccessful i n l o c a t i n g and/or i d e n t i f y i n g the phar-macist, r e p o r t t h i s f a c t to the c o - o r d i n a t o r . Meanwhile, proceed with the next pharmacist on the l i s t . -152-OBSERVATION PROCEDURE Before V i s i t : 1. Review the t e s t s i t u a t i o n 2. Locate the pharmacist - c a l l up to see i f on duty - and f o r how long During V i s i t : 1. C a r e f u l l y observe s i t u a t i o n a l f a c t o r s 2. Present problem to pharmacist 3. C a r e f u l l y observe pharmacist's response A f t e r V i s i t : 1. Immediately r e t u r n to c a r , out of s i g h t of pharmacy 2. F i l l i n performance c h e c k l i s t and v i s i t record sheet 3. Store V i s i t Record Sheet, purchased items, and r e c e i p t s i n the pharmacist's manila envelope 4. Keep a l l data i n a safe place. Use these questions, i f necessary, as reasons f o r asking f o r the pharmacist by name. Say a neighbour r e f e r r e d you to him. 1. That you w i l l be t r a v e l l i n g to the t r o p i c s ; F i j i , New Zealand, A u s t r a l i a , Kenya and Tanzania. A f r i e n d has t o l d you that you should take a n t i - m a l a r i a p i l l s - Chloraquine. Are they a v a i l a b l e without a p r e s c r i p t i o n ? I f not, are there any a n t i - m a l a r i a , p i l l s a v a i l a b l e without a p r e s c r i p t i o n ? 2. You are t r a v e l l i n g to Mexico. On your l a s t t r i p to Mexico you bought some Entero Vioform i n case of d i a r r e h i a . You want to buy some more but you have been t o l d by a f r i e n d t h a t they now r e q u i r e a p r e s c r i p t i o n . Is i t true? -153= V i s i t Record Sheet Observer Date of V i s i t S i t u a t i o n No. Pharmacist's Name A. D e s c r i p t i o n of the Pharmacist W B 0 Ht Wt Age Male Female B u i l d : S l i g h t Medium Large Hai r : Bald Normal Long Sideburns Color Mustache: Yes No Beard: Yes No B. S i t u a t i o n a l Factors (Record time to the nearest minute) Time i n st o r e door: Time greeted by pharmacist: Time f i n i s h e d w i t h pharmacist: Time out st o r e door: What was the pharmacist doing when you approached the p r e s c r i p t i o n counter? ( F i l l i n g p r e s c r i p t i o n s , s t o c k i n g s h e l v e s , etc.) Did he appear to have many p r e s c r i p t i o n s w a i t i n g ? Yes No How many pharmacists were on duty during your v i s i t ? How many other patrons were at the p r e s c r i p t i o n counter? How many customers were i n the whole store? How many customers were w a i t i n g at cash r e g i s t e r 1? cash r e g i s t e r 2? How many c l e r k s were on duty? -154-C. Treatment Scale Rate the pharmacist's treatment of you and your request by c i r c l i n g the number of each of the f o l l o w i n g s c a l e s which best matches her/ h i s behaviour during t h i s v i s i t . Be sure to r a t e every s c a l e . Treatment of v = very Q = q u i t e S=. S l i g h t N = n e i t h e r v a S N S a V You as a f r i e n d l y 1 2 3 4 5 6 7 h o s t i l e Person p o l i t e 1 2 3 4 5 6 7 rude f o r 1 2 3 4 5 6 7 ag a i n s t encouraging 1 2 3 4 5 6 7 discouraging accepting 1 2 3 4 5 6 7 questioning Over-the l e i s u r e l y 1 2 3 4 5 6 7 h u r r i e d counter prompt 1 2 3 4 5 6 7 slow drug c o n f i d e n t 1 2 3 4 5 6 7 unsure i n t e r e s t e d 1 2 3 4 5 6 7 d i s i n t e r e s t e d p o s i t i v e 1 2 3 4 5 6 7 negative wi11i ng 1 2 3 4 5 6 7 r e l u c t a n t TABLE XXIV CORRELATION MATRIX REPRESENTING THE RELATIONSHIPS BETWEEN THE TOTAL SCORE, ITS FOUR COMPONENTS AND THE SITUATIONAL FACTORS IN THE PRE-IN-STORE-ASSESSMENT FOR THE EXPERIMENTAL GROUP V a r i -able 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1 1.00 2 -0.57 1.00 . 3 0.45 -0.75 1.00 C o r r e l a t i o n c o e f f i c i e n t s of 0.36 are ZL n m n n i nn s i g n i f i c a n t a t the 0.05 l e v e l with 29 f IKOJ. -v.it U . J I i.uu degrees of freedom. C o r r e l a t i o n coef-5 0.89 -0.70 0.65 0.77 1.00 f i c i e n t s of 0.46 are s i g n i f i c a n t a t the 6. -0.11 0.33-0.14-0.14-0.19 1.00 0.01 l e v e l with 29 degrees of freedom. 7 -0.02 -0.02 -0.13 0.26 0.06 -0.23 1.00 8 0.59 -0.47 0.61 0.44 0.66 -0.21 0.04 1.00 9 -0.32 0.21 -0.19 -0.34 -0.38 0.05-0.14-0.22 1.00 10 -0.01 0.08-0.13 0.00-0.03 0.09-0.21-0.08 0.04 1.00 11 0.01 -0.14 0.10 0.23 0.14 0.01 0.34 0.42 -0.22 0.20 1.00 12 0.10-0.16 0.07-0.03 0.10 0.11 0.23 0.25-0.12 0.30 0.64 1.00 13 0.08-0.22 0.10-0.11 0.08 0.14-0.28 0.02-0.24 0.05 0.03 0.50 1.00 14 0.17-0.24 0.15 -0.40 0.00 0.00 -0.20 -0.24 -0.38 0.25-0.10 0.36 0.51 1.00 15 0.06-0.15 0.06 0.09 0.11 -0.49 0.38 0.15 -0.48 0.05 0.53 0.63 0.29 0.38 1.00 16 -0.71 0.45 -0.45 -0.71 -0.80 -0.01 -0.15 -0.48 0.29 0.00-0.06-0.06 0.08 0.32-0.00 1.00 17 -0.57 0.38 -0.35 -0.60 -0.67 0.08 0.11 -0.37 0.41 -0.19 0.04-0.06-0.15-0.02-0.00 0.67 1.00 l=Data g a t h e r i n g , 2=Inappropriate recommendations, 3=Appropriate recommendations, 4=Drug-use-counselling, 5=Total Score, 6=Age, 7=Time to be greeted by pharmacist, 8=Time spent w i t h pharmacist, 9=Busy, 10=Number of pharmacists working i n the dispensary, ll=Number of consumers at the p r e s c r i p t i o n counter, 12=Number of con-sumers i n the whole s t o r e , 13=Number of consumers a t cash r e g i s t e r one, 14=Number of consumers a t cash r e g i s -t e r two, 15=Number of c l e r k s , 16=Pharmacist's treatment of the consumer, 17=Pharmacist's treatment of the consumer's request f o r a non- p r e s c r i p t i o n product. TABLE XXV CORRELATION MATRIX REPRESENTING THE RELATIONSHIPS BETWEEN THE TOTAL SCORE, ITS FOUR COMPONENTS AND THE SITUATIONAL FACTORS IN THE POST-IN-STORE-ASSESSMENT OF THE EXPERIMENTAL GROUP V a r i -able 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1 1.00 2 -0.47 1.00 . , . . ,,. . . . . 0 , C o r r e l a t i o n c o e f f i c i e n t s of 0.36 are s i g -3 0.37 -0.68 1.00 n i f i c a n t a t the 0.05 l e v e l with 29 degrees 4 0.49-0.26 0.14 1.00 of freedom. C o r r e l a t i o n c o e f f i c i e n t s of 0.46 are s i g n i f i c a n t at the 0.01 l e v e l 5 0.86 -0.70 0.62 0.73 1.00 with 29 degrees of freedom. 6 0.11 -0.34 0.41 0.86 0.26 1.00 7 -0.29 0.53 -0.20 -0.49 -0.49 0.13 1.00 8 0.53 -0.42 0.41 0.40 0.60 0.27 -0.36 1.00 9 0.06 0.19-0.14 0.17 0.02 -0.15 -0.07 -0.23 '1.00 '10 0.01 0.23 -0.16 -0.07 -0.11 0.05 0.23 0.15 -0.39 1.00 11 0.03 0.02-0.06-0.32-0.14-0.08 0.21 -0.05 -0.39 0.17 1.00 12 -0.13 0.15-0.03-0.31-0.23-0.07 0.48 -0.15 -0.34 0.07 0.55 1.00 13 -0.18 0.21-0.28-0.28-0.31-0.18 0.37 -0.28 -0.37 -0.03 0.52 0.74 1.00 14 -0.13-0.05 0.02 0.03=0.04-0.19.-0.05 0.20-0.32-0.01 0.24 0.60 0.64 1.00 15 -0.05 0.28-0.09-0.34-0.24-0.07 0.60 -0.20 -0.30 0.16 0.60 0.75 0.57 0.30 1.00 16 -0.49 0.42 -0.00 -0.47 -0.51 -0.06 0.36 -0.54 0.09 0.17 -0.07 0.18 -0.01 -0.11 0.22 1.00; 17 -0.43 0.19 0.00 -0.37 -0.39 -0.14 0.09 -0.50 0.19 0.04 0.05-0.02-0.00 0.05 0.25 0.66 1.00 l=Data g a t h e r i n g , 2=Inappropriate recommendations, 3=Appropriate recommendations, 4=Drug-use-counselling, 5=Total Score, 6=Age, 7=Time to be greeted by pharmacist, 8=Time spent w i t h pharmacist, 9=Busy, 10=Number of pharmacists working i n the dispensary, ll=Number of consumers a t the p r e s c r i p t i o n counter, 12=Number of consumers i n the whole s t o r e , 13=Number of consumers at cash r e g i s t e r one, 14=Number of consumers a t cash r e g i s t e r two, 15=Number of c l e r k s , 16=Pharmacist's treatment of the consumer, 17=Pharmacist's treatment of the consumer's request f o r a non-prescription product. TABLE XXVI CORRELATION MATRIX REPRESENTING THE RELATIONSHIPS BETWEEN THE TOTAL SCORE, ITS COMPONENTS AND THE SITUATIONAL FACTORS FOR THE PRE-IN-STORE-ASSESSMENT FOR THE CONTROL GROUP V a r i -able 1 2 3 4 5 1 1.00 2 0.20 1.00 3 0.01 -0.47 1.00 4 0.69 0.22 -0.13 1.00 5 0.93 -0.01 0.20 0.80 1.00 6 0.97 -0.27 0.13 -0.02 0.10 7 0.11 0.00 -0.00 0.37 0.21 8 0.18 0.13 -0.04 0.18 0.17 9 -0.18 -0.37 0.28 -0.15 -0.04 10 -0.25 -0.25 0.11 -0.21 -0.21 11 -0.08 0.07 0.17 -0.22 -0.13 12 0.01 -0.01 0.10 0.09 0.05 13 0.17 0.12 0.25 -0.02 0.11 14 0.19 -0.12 0.00 0.51 0.31 15 0.05 -0.14 0.17 0.01 0.06 16 -0.42 -0.02 -0.20 -0.21 -0.40 17 -0.10 -0.21 -0.01 -0.05 -0.05 10 11 12 13 14 15 16 17 C o r r e l a t i o n c o e f f i c i e n t s of 0.38 are s i g -n i f i c a n t a t the 0.05 l e v e l with 25 deg-rees of freedom. C o r r e l a t i o n c o e f f i c i e n t s of 0.49 are s i g n i f i c a n t at the 0.01 l e v e l w i t h 25 degrees of freedom. 1.00 ;0.23 1.00 0.27 0.12 1, .00 0.27 -0.42 -0. .00 1.00 0.14 -0.07 -0, .19 T0.29 1.00 -0.27 0.09 -o; .16 -0.23 0.25 1.00 -0.19 0.31 -0, .13 -0.47 0.25 0.48 1.00 -0.16 -0.02 0. .05 -0.14 -0.02 0.46 0.58 1 .00 -0.18 0.28 0, .02 -0.33 -0.10 0.32 0.95 0 .79 1 .00 -0.34 0.15 -0. .26 -0.11 0.21 0.38 0.49 0 .45 0 .37 1 .00 0.05 -0.02 -0. .01 0.19 0.03 0.20 0.00 -0 .05 -0 .03 0 .10 0.18 0.07 0. .10 0.44 -0.11 0.07 -0.20 -0 .80 -0 .30 0 .02 1.00 0.42 1.00 l=Data g a t h e r i n g , 2=Inappropriate recommendations, 3=Appropriate recommendations, 4=Drug-use-counselling, 5=Total Score, 6=Age, 7=Time to be. greeted, by pharmacist, 8=Time spent with pharmacist, 9=Busy, 10=Number of pharmacists working i n the dispensary, ll=Number of consumers at the p r e s c r i p t i o n counter, 12=Number of consumers i n the whole s t o r e , 13=Number of consumers at cash r e g i s t e r one, 14=Number of consumers at cash r e g i s t e r two, 15=Number of c l e r k s , 16=Pharmacist's treatment of the consumer, 17=Pharmacist's treatment of the consumer's request f o r a non- p r e s c r i p t i o n product. TABLE XXVII CORRELATION MATRIX REPRESENTING THE RELATIONSHIPS BETWEEN THE TOTAL SCORE, ITS COMPONENTS AND THE SITUATIONAL FACTORS FOR THE POST-IN-STORE-ASSESSMENT FOR THE CONTROL GROUP V a r i -albe 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1 1.00 2 -0.06 1.00 C o r r e l a t i o n c o e f f i c i e n t s of 0.38 are s i g -3 0 49 -0 43 1 00 n i f i c a n t a t the 0.05 l e v e l w i t h 25 degrees — — ~ — — ' of freedom. C o r r e l a t i o n c o e f f i c i e n t s of 4 0.30 0.21 0.00 1.00 0.49 are s i g n i f i c a n t a t the 0.01 l e v e l 5 0,88 -0.30 0^5 0^54 1.00 with 25 degrees of freedom. 6 0.05 0.29 0.04 0.03 -0.01 1.00 7 -0.07 -0.06 -0.09 -0.08 -0.09 -0.54 1.00 8 0.10 0.17 0.02 0.30 0.13 -0.01 0.10 1.00 9 -0.25 -0.12'-0.10 -0.01 -0.19 0.23 -0.25 -0.17 1.00 10 -0.10 0.13 -0.11 -0.05 -0.13 -0.03 0.05 -0.09 -0.09 1.00 11 -0.09 0.07 -0.04 -0.18 -0.15 -0.14 0.36 -0.05 -0.54 0.25 1.00 12 -0.24 0.00 -0.17 0.21 -0.11 -0.40 0.36 0.09 -0.33 0.37 0.52 1.00 13 -0.20 -0.15 0.02 0.24 0.01 -0.27 0.27 0.06 0.02 0.39 0.38 0.76 1.00 14 0.06 -0.12 0.08 0.38 0.26 -0.04 0.36 -0.05 0.38 0.33 0.21 0.59 0.88 1.00 15 -0.09 0.29-0.17-0.03-0.18-0.26 0.32-0.18-0.27 0.55 0.34 0.59 0.33 0.17 1.00 16 -0.31 0.05 -0.16 -0.46 -0.43 0.12-0.08-0.25 0.08 0.25 0.17 0.07 0.03-0.20 0.14 1.00 17 -0.19 0.16-0.24-0.19-0.30 0.30-0.11-0,11 0.27 0.23 0.05-0.08-0.06-0.06 0.01 0.72 1.00 l=Data g a t h e r i n g , 2=Inappropriate recommendations, 3=Appropriate recommendations, 4=Drug-use-counsel!ing, 5=Total Score, 6=Age, 7=Time to be greeted by pharmacist, 8=Time spent w i t h pharmacist, 9=Busy, 10=Number of pharmacists working i n the dispensary, ll=Number of consumers at the p r e s c r i p t i o n counter, 12=Number of consumers i n the whole s t o r e , 13=Number of consumers a t cash r e g i s t e r one, 14=Number of consumers a t cash r e g i s t e r two, 15=Number of c l e r k s , 16=Pharmacist's treatment of the consumer, 17=Pharmacist's treatment of consumer's request f o r a no n - p r e s c r i p t i o n product. -159-APPENDIX B: PANEL OF CONTENT EXPERTS -160-Panel members were drawn from the f a c u l t y of Pharmaceutical Sciences at the U n i v e r s i t y of B r i t i s h Columbia and from community pharmacy p r a c t i t i o n e r s i n Vancouver, B r i t i s h Columbia. P a n e l i s t s Dennis Andrews,Pharm D. A s s i s t a n t P r o f e s s o r , D i v i s i o n of C l i n i c a l Pharmacy. Loree E l d r i d g e , B.Sc. (Pharm.) C l i n i c a l I n s t r u c t o r , Community pharmacist with three years experience. Peter Hutt, B . S c . (Pharm.) C l i n i c a l I n s t r u c t o r , Community pharmacist with two years experience. Ronald Ingraham, B.Sc. (Pharm.) C l i n i c a l I n s t r u c t o r , Community pharmacist w i t h 12 years experience. Munroe MacKenzie B.S.P. C l i n i c a l I n s t r u c t o r , Community pharmacist w i t h 24 years experience. Nicholas Otten, B.Sc. (Pharm.) Pharmacist, Family P r a c t i c e U n i t , U.B.C. Karen Pylatuk, B.Sc. (Pharm.), M.Sc. Le c t u r e r , D i v i s i o n of C l i n i c a l Pharmacy, ( s p e c i a l i t y drug i n f o r m a t i o n ) . Ian Sands, B.Sc. (Pharm.) M.Sc. C l i n i c a l I n s t r u c t o r , Community pharmacist wit h three years experience. Sharon Tudor, B.Sc. (Pharm.) Chairman of the D i v i s i o n of C l i n i c a l Pharmacy, p r e v i o u s l y 11 years experience i n community pharmacy. Louanne Twaites, B.S.P. Health Science Centre H o s p i t a l , U.B.C. C l i n i c a l I n s t r u c t o r , p r e v i o u s l y 17 years experience i n community pharmacy. -161-APPENDIX C: MATERIALS RELATING TO THE WRITTEN SIMULATIONS Page 1. P r a c t i c e S i m u l a t i o n : E n t i t l e d The Mystery. (The mat e r i a l contained on the r i g h t hand side would normally be i n v i s i b l e ) 2. Simulation One 3. Latent Image Responses to Simulation One 4. Simulation Two 5. Latent Image Responses to Simulation Two 6. Simulation Three 7. Latent Image Responses to Simulation Three 8. Simulation Four 9. Latent Image Responses to Simulation Four 10. Figure 3. Simulation One. P o s s i b l e Paths to Problem S o l u t i o n 11. Figure 4. Simulation Two. P o s s i b l e Paths to Problem S o l u t i o n 12. Figure 5. Simulation Three. P o s s i b l e Paths to Problem S o l u t i o n 13. Figure 6. Simulation Four. P o s s i b l e Paths to Problem S o l u t i o n 14. Table XIV. I n t e r c o r r e l a t i o n s . of the Ten Judges' Ratings of ..  the 267 Options i n Simulations One to Four. -162-A MYSTERY As a young d e t e c t i v e assigned to the homicide d i v i s i o n , you are c a l l e d to i n v e s t i g a t e a death. You a r r i v e at a home f u l l of d i s t r a u g h t people: a man, h i s w i f e , t h e i r son and daughter, the b u t l e r and the maid. Grandma has been found dead i n bed. On a night t a b l e beside her i s a g l a s s , n early empty, c o n t a i n i n g some brownish l i q u i d . The f a m i l y asks how soon they may d i s c u s s the w i l l w i t h Grandma's lawyer. You begin i n v e s t i g a t i o n . NOW CONTINUE WITH SECTION A. SECTION A: Data Gathering In gathering data about the case, you would ( S e l e c t AS MANY as you consider p e r t i n e n t at t h i s stage of the i n v e s t i g a t i o n ) ; 1. Ask what Grandma ate f o r supper 1. the n i g h t before. 2. Ask who i n h e r i t s what. 2. 3. Inquire about Grandma's recent 3. s t a t e of h e a l t h . 4. Check f o r f i n g e r p r i n t s on the 4. g l a s s . 5. Question everyone about h i s 5. whereabouts yesterday. 6. Perform s u p e r f i c i a l exami- 6. nation of the body. 7. Try to i d e n t i f y the contents 7. of the glass on the n i g h t t a b l e . 8. A s c e r t a i n that Grandma i s dead. 8. Tea and t o a s t . Her son says the f a m i l y cat got i t a l l . Son claims she was healthy and hearty. F i n g e r p r i n t s examined appear to match Grandma's. Appropriate a l i b i s are provided by a l l those present. No marks or b r u i s e s are seen on the body. There i s a s l i g h t smile on the l i p s . A l i q u i d s m e l l i n g of almonds and a l c o h o l . The m i r r o r t e s t shows no vapour inf o r m a t i o n . No heart sounds or pulse. A l o c a l doctor pro-nounces her dead. SECTION A CONTINUED ON NEXT PAGE. -163-You would NOW (Choose ONLY ONE): 9. A r r e s t someone f o r murder. 10. Try to gather more infor m a t i o n . 11. Report to your sup e r i o r s t h a t Grandma i s probably a s u i c i d e . 12. Report to your s u p e r i o r s t h a t Grandma died a natural death. 13. Try to r e v i v e Grandma. 9. Continue with Section B. 10. Turn to Section C. 11. You are asked to provide e v i -dence. You are unable to do so and must make another choice from t h i s s e c t i o n . 12. You are asked to provide evidence. You are unable to do so and must make another choice from t h i s s e c t i o n . 13. You are unable to do so and must make another choice from t h i s s e c t i o n SECTION B: Management At t h i s time you would a r r e s t ( S e l e c t AS MANY as you consider suspect): 14. The son, 15. The daughter-in-law. 16. The grandson. 17. The granddaughter. 14. You are rebuked by your super-i o r s s i n c e you have not estab-l i s h e d cause of death, nor have you evidence of oppor-t u n i t y or motive f o r the accused. 15. You are rebuked by your super-i o r s s i n c e you have not estab-l i s h e d cause of death, nor have you evidence of opportunity or motive f o r the accused. 16. You are rebuked by your super-i o r s s i n c e you have not estab-l i s h e d cause of death, nor have you evidence of opportunity or motive f o r the accused. 17. You are rebuked by your super-i o r s s i n c e you have not estab-l i s h e d cause of death, nor have you evidence of opportunity or motive f o r the accused. CONTINUED NEXT PAGE -164-18. The maid. 18. You are rebuked by your super-i o r s s i n c e you have not estab-l i s h e d cause of death, nor have you evidence of oppor-t u n i t y or motive f o r the accused. 19. The b u t l e r . 19. You are rebuked by your super-i o r s s i n c e you have not estab-l i s h e d cause of death, nor have you evidence of oppor-t u n i t y or motive f o r the accused. UNLESS OTHERWISE DIRECTED IN THE RESPONSE COLUMN, DEVELOP RESPONSE #39 AS SOON AS YOU HAVE COMPLETED THIS SECTION SECTION C: Data Gathering In gathering more i n f o r m a t i o n , you would ( S e l e c t AS MANY as you consider p e r t i n e n t at t h i s stage of the i n v e s t i g a t i o n ) : 20. V e r i f y what Grandma ate the 20. night before. 21. V e r i f y w i l l with lawyer. 21. 22. Contact Grandma's p h y s i c i a n f o r 22. f u r t h e r information about her recent s t a t e of h e a l t h . 23. V e r i f y f i n g e r p r i n t on the g l a s s . 23. 24. Check everyone's a l i b i f o r 24. yesterday. 25. Obtain p h y s i c i a n ' s r e p o r t on 25. time and cause of death. 26. V e r i f y contents of glass on the 26. n i g h t t a b l e . Examination of stomach revealed what looks l i k e tea and t o a s t . Grandma's lawyer says the cat got i t a l 1 . Grandma was a t i p p l e r , had a very bad heart c o n d i t i o n and was d i a b e t i c . He c o n s t a n t l y had to refuse her demands f o r n a r c o t i c cough mixutre. Grandma's f i n g e r p r i n t v e r i f i e d . A l l a l i b i s are v e r i f i e d by r e l i a b l e witnesses. Grandma has been dead f o r s i x hours. No evidence of trauma. Contents sent to the l a b . No cyanide found. L i q u i d con-t a i n s sugar, water, peach f l a v o u r i n g , vitamins and small amounts of s t r y c h n i n e . At t h i s time you would (Choose ONLY ONE): 27. A r r e s t someone i f you have not already done so. 27. Turn to S e c t i o n B. CONTINUED NEXT PAGE -165-28. Seek a d d i t i o n a l information. 29. Report to your sup e r i o r s t h a t Grandma was a s u i c i d e . 30. Report to your s u p e r i o r s t h a t Grandma died of natural causes, 28. Continue with S e c t i o n D. 29 Your s u p e r i o r s ask f o r f u r t h e r evidence; you are unable to provide i t and must make another choice from t h i s sec-t i o n . 30. Your sup e r i o r s ask f o r f u r t h e r evidence; you are unable to provide i t and must make another choice from t h i s sec-t i o n . SECTION D: Data Gathering In seeking f u r t h e r i n f o r m a t i o n , you would ( S e l e c t AS MANY as you consider p e r t i n e n t a t t h i s stage of the i n v e s t i g a t i o n ) : 31. Order chemical a n a l y s i s of 31 stomach contents. 32. Order post mortem examination of 32. the body. 33. Order chemical a n a l y s i s of blood f o r s t r y c h n i n e . 33. 34. Order chemical a n a l y s i s of 34. ma t e r i a l i n glass on nig h t t a b l e . A n a l y s i s reveals sucrose, water, s t a r c h , glucose, charcoal and tannic a c i d (tea and dry t o a s t ) . P a t h o l o g i s t t e l l s you h i s examination reveals t h a t she had a heart a t t a c k . Trace found. The chemical composition i s c o n s i s t e n t w i t h t h a t of a harmless g e r i a t r i c t o n i c . At t h i s time you would re p o r t to your sup e r i o r s t h a t (Choose ONLY ONE) 35. Grandma was murdered. 35, 36. Grandma was s u i c i d e . 36. 37. Grandma died of natural causes. 37. 38. Data regarding cause of death 38. are s t i l l i n c o n c l u s i v e . Report returned w i t h some choice e p i t h e t s . Develop response 39. Report returned with some choice e p i t h e t s . Develop response 39. N a t u r a l l y . END OF PROBLEM. Superiors use some choice e p i t h e t s i n suggesting you f i n d out what an extensive myocardial i n f a r c t i o n i s . Develop response 39. CONTINUED NEXT PAGE An autopsy examination r e v e a l s an acute myocardial i n f a r c t i o n of e n t i r e a n t e r i o r l e f t vent-t i c l e (heart a t t a c k ) . Your super i o r s t e l l you th a t means Grandma died of natural causes. They recommend your prompt demotion. END OF PROBLEM. -167-PROBLEM 1  OPENING SCENE You are a community pharmacist working i n a medium s i z e st o r e of a l a r g e n a t i o n a l c h a i n . The dispensary i n t h i s s t o r e i s s l i g h t l y elevated and l o c a t e d a t the back of a b r i g h t and w e l l l a i d out main f l o o r . The dispensary i s of adequate s i z e and i s w e l l stocked w i t h references and a l l the necessary a u x i l i a r y equipment. On t h i s p a r t i c u l a r day you are the only pharmacist i n the dispen-sary and there are two employees working "out f r o n t " . I t i s the noon hour and you are f a i r l y busy. There are a number of phoned-in p r e s c r i p t i o n s y e t to be done and there are two i n d i v i d u a l s w a i t i n g f o r t h e i r p r e s c r i p -t i o n s which you are p r e s e n t l y working on. A middle-aged woman enters the s t o r e and heads f o r the dispensary, she stops i n f r o n t of you and asks the f o l l o w i n g q u e s t i o n , "What have you got f o r a c o l d ? " SECTION A Considering the circumstances j u s t described you would NOW (Choose ONLY ONE).:. 1. Ask one of the "out f r o n t " em-ployees to help her. 5. Since a l l n o n - p r e s c r i p t i o n medi-c a t i o n s designed to r e l i e v e the symptoms of the common co l d are the same, recommend one t h a t you have found b e n e f i c i a l i n the past. 6. Recommend th a t she contact her p h y s i c i a n . SECTION B At t h i s time you would (Choose ONLY ONE): 8. Recommend a product that you have been recommending fori years. Obtain a d e s c r i p t i o n of the ailment. Recommend th a t she see her f a m i l y p h y s i c i a n . Suggest that she w a i t a few days to see i f the " c o l d " gets b e t t e r , i f not she should con-t a c t her f a m i l y p h y s i c i a n . 2. Recommend th a t she s e l e c t a product from the "Cough and Cold" 9. s e c t i o n . 3. Recommend a product which she 10. would recognize because i t i s a d v e r t i s e d on n a t i o n a l t e l e v i s i o n . ^ 4. Ask her to be seated and you w i l l be w i t h her very s h o r t l y . 7. Drop what you are doing and walk with her to the "Cough and Cold" s e c t i o n . -168-PROBLEM 1. SECTION C You would be i n t e r e s t e d i n asking about which of the f o l l o w i n g ( S e l e c t AS MANY as you.consider ESPECIALLY p e r t i n e n t ) : 12. Who has the c o l d . 13. I n d i v i d u a l ' s age. 14. Fever. 15. D e s c r i p t i o n of any cough 16. Nasal discharge. 17. Nasal congestion. 18. Post nasal d r i p . 19. Sore t h r o a t . 20. Muscle aches and pains. 21. Headache. 22. Swollen glands or sore neck. 23. Nausea and vomiting. 24. Ear ache. 25. Night sweats. 26. Duration of the symptoms. 27. Shortness of breath. 28. Trouble s l e e p i n g . 29. Bowel movements. 30. A l l e r g i e s . 31. I f there are any small c h i l d r e n around house. 32. A p p e t i t e . 33. Does she sleep w i t h a window open. 34. Are there other members of the f a m i l y a f f l i c t e d w i t h a " c o l d " . 35. Does she smoke c i g a r e t t e s . 36. Is she experiencing any i n d i g e s -t i o n . At t h i s time you would (Choose ONLY ONE): 37. Recommend some n o n - p r e s c r i p t i o n medication to a l l e v i a t e the symptoms de s c r i b e d . 38. Inquire about other ailments the consumer may have and medi-c a t i o n s she may be t a k i n g . 39. Recommend th a t she see her f a m i l y p h y s i c i a n s i n c e the symptoms described i n d i c a t e a serious upper r e s p i r a t o r y t r a c t i n f e c t i o n . 40. Recommend that because the woman e x h i b i t s symptoms of a common c o l d she should take ASA, d r i n k p l e n t y of f l u i d s , and r e s t f o r the next two to three days. -169-PROBLEM 1. SECTION D You would be i n t e r e s t e d i n which of the f o l l o w i n g information ( S e l e c t AS MANY as you consider ESPECIALLY p e r t i n e n t ) : 41. Is she being t r e a t e d by a ph y s i -. • c i a n f o r any other ailment? 42. Is she t a k i n g any p r e s c r i p t i o n medications? 43. When was the l a s t time she saw her f a m i l y p h y s i c i a n . 44. Is she t a k i n g any non-p r e s c r i p t i o n medication? 45. How long has she been t a k i n g her medications? GO DIRECTLY TO SECTION F. SECTION E At t h i s time you would (Choose ONLY ONE): 46. Recommend an a l t e r n a t e product, 47. Ask consumer to describe the symptoms. 48. Suggest that she see her f a m i l y p h y s i c i a n . 49. Suggest that " c o l d " i s not serious enough to warrant t r e a t -ment at t h i s time and perhaps she should wait and see what develops. SECTION F At t h i s time you would recommend th a t (Choose ONLY ONE): 50. Since the symptoms described i n d i c a t e an upper r e s p i r a t o r y t r a c t i n f e c t i o n of a serious nature she should see her f a m i l y p h y s i c i a n . 51. The consumer has the symptoms of a common c o l d and should take ASA and dr i n k plenty of f l u i d s and r e s t f o r the next two to three days. 52. Some n o n - p r e s c r i p t i o n medication to a l l e v i a t e the symptoms described i s warranted. SECTION G At t h i s time you would ( S e l e c t AS MANY as you f e e l p e r t i n e n t ) : 54. Inform the consumer of a l l p o s s i b l e side e f f e c t s and adverse a c t i o n s of her non-p r e s c r i p t i o n medication. 55. Inform consumer about the proper dosage and the i n s t r u c -t i o n s on how.to take the medication. 56. Suggest t h a t i f the cough p e r s i s t s , i f a fever develops, or i f any of the symptoms get worse, she should contact her p h y s i c i a n . COMPLETION OF SECTION G MARKS THE END OF THIS PROBLEM. -170-PROBLEM 1 SECTION H At t h i s time you would recommend (Choose ONLY ONE): 57. B e n y l i n r w i t h codeine f o r the cough and T r i a m i n i c i n r f o r the nasal congestion. 58. An o r a l product c o n t a i n i n g a sympathomimetic and an a n t i h i s -tamine. 59. Ornade DMr l i q u i d . 60. Throat lozenge. 61. A t o p i c a l nasal decongestant such as 0 t r i v i n r . 62. Dimetapp r E l i x i r . 63. Suggest t h a t she use a v a p o r i z -er i n the evening. 64. Contac C r. 65. C o r i c i d i n r c o l d t a b l e t s . 66. Promatusssin r Expectorant. 67. R o b i t u s s i n r . 68. B e n y l i n D i e t e t i c r . 69. Buckely's Sugar F r e e r . SECTION I-:-At t h i s time you would recommend (Choose ONLY ONE): 70. B e n y l i n DMr. 71. D r i s t a n Tablets'". 72. Ornade DMr l i q u i d . 73. Contac C r. 74. H a l l ' s E u c a l y p t u s r . 75. C o r i c i d i n r Cold T a b l e t s . 76. R o b i t u s s i n r . SECTION I (Cont.) 77. P r o m a t u s s i n r Expectorant. 78. Dimetapp r E l i x i r . 79. Dristan 1" nasal mist. SECTION J At t h i s time you would (Choose ONLY ONE): 80. Ask consumer to describe symptoms. 81. Recommend a product. 82. Suggest t h a t she see her fa m i l y p h y s i c i a n . -171-LATENT IMAGE RESPONSES PROBLEM 1 1. Consumer says she wants your a s s i s t a n c e . Go to Section B. 2. Consumer spends f i v e minutes searching, waits u n t i l the two w a i t i n g customers leave with t h e i r p r e s c r i p t i o n s and then asks you to suggest a product f o r her c o l d . Go to Section B. .3. Go to Section I. 4. Consumer waits her turn and says, "What would you recom-mend f o r a c o l d ? " Go to Section B. 5. Go to Sectio n I. 6. Consumer f e e l s the c o l d does not warrant p h y s i c i a n therapy and asks you to recommend a product. Go to Section I. 7. This upsets one of the con-sumers who has been w a i t i n g ..for ten minutes f o r you to f i l l h i s p r e s c r i p t i o n , he expresses h i s d i s p l e a s u r e to you and a f t e r the d i s c u s s i o n that f o l l o w s the woman says she w i l l w a i t . Go to Section B. 8. Go to Sectio n I. 9. Go to Sectio n C. 10. Consumer f e e l s the " c o l d " does not warrant p h y s i c i a n therapy, and wants you to recommend product. Make another choice i n t h i s sec-t i o n . 11. Returns the next day complaining t h a t " c o l d " has not improved. Make another choice i n t h i s s e c t i o n . 12. I do. 13. Consumer i s obviously a g i t a t e d by the question and w i l l not t e l l you her age. 14. No. 15. Dry, hacking and non-productive. 16. Yes. 17. Yes, d i f f i c u l t y i n breathing through nose. 18. A s l i g h t post nasal d r i p . 19. Scratchy t h r o a t but no problem swallowing. 20. No. 21. No. 22. Neck not sore, does not know i f glands are swollen. 23. No. 24. No. 25. No. 26. Noticed f i r s t signs of " c o l d " 36 hours ago. 27. No. PROBLEM 1 -172-28. Trouble breathing through nose, kept awake longer than usual l a s t n i g h t . 29. Regular. 30. No. 31. Yes, but does not see tha t has to do with a what c o l d . 32. Good. 33. Yes. 34. No. 35. No. 36. No. 37. Go to Section H. 38. Go to Sectio n D. 39. Consumer returns that evening l o o k i n g f o r a t o p i c a l decon-gestant which her ph y s i c i a n recommended f o r her nasal congestion due to the " c o l d " . END OF PROBLEM. 40. Consumer requests t h a t you suggest something f o r the symptoms. Go to Section H. 41. Yes, high blood pressure and diabetes. 42. Yes, 0 r i n a s e r , Aldometr and N o r i n l y l r . 43. Four weeks ago f o r renewal of the N o r i n l y l r . 44. Yes, vitamins o c c a s i o n a l l y . 45. Two years f o r 0 r i n a s e r and Aldomet r, 18 months f o r the Norinyl l r . 46. Consumer upset t h a t you would keep recommending products without asking about the symp-toms. Go to Section J . 47. Go to Sectio n C. 48. Since you have not made any i n -q u i r i e s about the symptoms of the ailment, consumer does not see how you could p o s s i b l y as-c e r t a i n that the " c o l d " war-rants p h y s i c i a n treatment. Wants you to handle her problem. Go to Sectio n J . 49. Consumer returns next day, says " c o l d " i s not improving. Go to Section J . 50. Consumer returns that evening l o o k i n g f o r a t o p i c a l decongest-ant which her p h y s i c i a n recommended f o r her nasal congestion due to the " c o l d " . END OF PROBLEM. 51. Consumer requests t h a t you recom-mend something f o r the symptoms. I f you now wish to recommend a non-p r e s c r i p t i o n medication i n a d d i t i o n to ASA, go to Sectio n H. Otherwise, develop response 53. 52. Go to Section H. 53. The consumer expresses extreme d i s p l e a s u r e a t your reluctance to help her by suggesting something f o r the symptoms, and i n d i c a t e s she s h a l l f i n d a more cooperative pharmacist. END OF PROBLEM. 54. Consumer i s alarmed and leaves s t o r e without product. END OF PROBLEM; -173-PROBLEM 1 55. Consumer thanks you f o r your help and advice and leaves the s t o r e w i t h the medication you have recommended. 56. Consumer returns i n f i v e days to get her p r e s c r i p t i o n f o r Aldomet r r e f i l l e d and a t th a t time she informs you th a t she i s f e e l i n g much b e t t e r , her nasal congestion i s gone and she has no post nasal d r i p , cough or sore t h r o a t . 57. The next day the consumer develops a headache, a gene-r a l f e e l i n g of malaise. She goes to her p h y s i c i a n who claims i t i s a d i r e c t r e s u l t of the n o n - p r e s c r i p t i o n medi-c a t i o n you recommend. END OF PROBLEM. 58. The next day the consumer develops a headache and a general f e e l i n g of malaise. She goes to her f a m i l y phy-s i c i a n who a t t r i b u t e s t h i s to the n o n - p r e s c r i p t i o n medica-t i o n you recommended. END OF PROBLEM. 59. The next day the consumer develops a headache and a general g e e l i n g of malaise. xShe goes to her f a m i l y phy-s i c i a n who a t t r i b u t e s t h i s to the n o n - p r e s c r i p t i o n medica-t i o n you recommended. END OF PROBLEM. . 60. The consumer would a l s o l i k e something f o r her nasal con-g e s t i o n . I f you wish to recommend a second product, make another choice from t h i s s e c t i o n , i f not go to Section G. 61. Before consumer leaves she asks i f there i s anything she should know about t h i s product. Go to Section G. 62. .The next day the consumer no t i c e s an increased l e v e l of glucose i n her u r i n e . Phones her p h y s i c i a n who says i t may be due to the n o n - p r e s c r i p t i o n medication you have recommended. END OF PROBLEM. 63. Consumer f e e l s she needs some-th i n g f o r daytime r e l i e f of her congestion, and wants you to recommend something e l s e . Make another choice from t h i s s e c t i o n . 64. The next day the consumer develops a headache and a general f e e l i n g of malaise. She goes to her f a m i l y p h y s i c i a n who a t t r i b u t e s t h i s to the non-p r e s c r i p t i o n medication you have recommended. END OF PROBLEM. 65. The next day the consumer develops a headache and a general f e e l i n g of malaise. She goes to her f a m i l y p h y s i c i a n who a t t r i b u t e s t h i s to the non-p r e s c r i p t i o n medication recom-mended,.' END PROBLEM. 66. The next day the consumer not i c e s an increased l e v e l of glucose i n her u r i n e . She phones her ph y s i c i a n who says i t may be due to the n o n - p r e s c r i p t i o n medi-c a t i o n you recommended. END OF PROBLEM. -174-PROBLEM 1 67. The. next day the consumer no t i c e s an increased l e v e l of glucose i n her u r i n e . She phones her p h y s i c i a n who says i t may be due to the no n - p r e s c r i p t i o n medication you recommended. END OF PROBLEM. 73. Consumer f i n d s the product on the s h e l f and reads the d i r e c -t i o n s . Since you have f i n i s h e d w i t h your previous customers she returns to you and says t h a t product i s u n s u i t a b l e . She asks you to recommend something e l s e . Go to Sectio n E. 68. Go to Section G. 69. Go to Sect i o n G. 70. Consumer goes and f i n d s the product on the s h e l f , she reads the d i r e c t i o n s . Since you have f i n i s h e d .with;your previous ..custom-ers she returns to you and says t h a t product i s un-s u i t a b l e . She asks you to recommend something e l s e . Go to Sectio n E. 71. Consumer f i n d s the product on the s h e l f and reads the d i r e c t i o n s . Since you have f i n i s h e d w i t h your previous customers she returns to you and says t h a t product i s u n s u i t a b l e . She asks you to recommend something e l s e . Go to Sectio n E. 72. Consumer f i n d s the product on the s h e l f and reads the d i r e c t i o n s . Since you have f i n i s h e d w i t h your previous customers she returns to you and says t h a t product i s u n s u i t a b l e . She asks you to recommend something e l s e . Go to Section E. 74,' Consumer f i n d s the product on the s h e l f and reads the d i r e c -t i o n s . Since you have f i n i s h e d with your previous customers she returns to you and says t h a t product i s u n s u i t a b l e . She asks you to recommend something e l s e . Go to Se c t i o n E. 75. Consumer f i n d s the product on the s h e l f and reads the d i r e c -t i o n s . Since you have f i n i s h e d w i t h your previous customers she returns to you and says that product i s u n s u i t a b l e . She asks you to recommend something e l s e . Go to Se c t i o n E. 76. Consumer f i n d s the product on the s h e l f and reads the d i r e c -t i o n s . Since you have f i n i s h e d with your previous customers she returns to you and says t h a t pro-duct i s u n s u i t a b l e . She asks you to recommend something e l s e . Go to Sect i o n E. 77. Consumer f i n d s the product on the s h e l f and reads the d i r e c t i o n s . Since you have f i n i s h e d with your previous customers she returns to you and says t h a t product i s un-s u i t a b l e . She asks you to recom-mend something e l s e . Go to Sec-t i o n E. PROBLEM 1 -175-78. Consumer f i n d s the product on the s h e l f and reads the d i r e c t i o n s . Since you have f i n i s h e d w i t h your previous customers she returns to you and says that product i s un s u i t a b l e . She asks you to recommend something e l s e . Go to Sectio n E. 79. Consumer f i n d s the product on the s h e l f and reads the . d i r e c t i o n s . Since you have f i n i s h e d with your previous customers she returns to you and says that product i s un-s u i t a b l e . She asks you t o recommend something e l s e . Go to Sec t i o n E. 80. Go to Sec t i o n C. 81. Go to Sec t i o n H. 82. Consumer f e e l s t h a t " c o l d " does not warrant p h y s i c i a n treatment, wants you to recommend a product. Make another choice i n t h i s s e c t i o n . -176-PROBLEM 2 OPENING SCENE You are a community pharmacist working i n a medium s i z e s t o r e of a l a r g e n a t i o n a l c h a i n . The dispensary i n t h i s s t o r e i s s l i g h t l y elevated and located a t the back of a b r i g h t and w e l l l a i d out main f l o o r . The dispensary i s of adequate s i z e and i s well stocked with references and a l l the necessary a u x i l i a r y equipment. On t h i s p a r t i c u l a r Monday you are the only pharmacist i n the d i s -pensary and there are two employees working "out f r o n t " . I t i s 10 a.m. and you are f a i r l y busy. There are a number of phoned-in p r e s c r i p t i o n s y e t to be done and there are two i n d i v i d u a l s w a i t i n g f o r t h e i r p r e s c r i p -t i o n s which you are p r e s e n t l y working on. A young man enters the store and heads f o r the dispensary, stops i n f r o n t of you and asks the f o l l o w i n g question, "What have you got f o r a co l d ? " SECTION A SECTION A contd. Considering the circumstances j u s t described you would NOW (Choose ONLY ONE): .1. Ask one of the "out f r o n t " employees to help him. 2. Recommend tha t he s e l e c t a product from the "Cough and Cold" s e c t i o n . 3. Recommend a product which he would recognize be-cause i t i s a d v e r t i s e d on nat i o n a l t e l e v i s i o n . 4. Ask him to be seated and you w i l l be with him s h o r t l y . 5. Since a l l non-prescrip-t i o n medications designed to r e l i e v e the symptoms of the common c o l d are the .: .. same, recommend one t h a t you have found b e n e f i c i a l i n the past. 6. Recommend that he contact hi s f a m i l y p h y s i c i a n . 7. Drop what you are doing and walk with him to the "Cough and Cold s e c t i o n . ' -SECTION B 8. Recommend a product t h a t you have been recommending f o r years. 9. Obtain a d e s c r i p t i o n of the ailment. 10. Recommend tha t he see h i s f a m i l y p h y s i c i a n . 11. Suggest t h a t he wait a few days to see i f the " c o l d " gets b e t t e r , i f not he should contact h i s f a m i l y p h y s i c i a n . SECTION C You would be i n t e r e s t e d i n asking about which of the f o l l o w i n g ( S e l e c t AS MANY as you consider ESPECIALLY p e r t i n e n t ) : 12. Who has the c o l d . 13. I n d i v i d u a l ' s age. 14. Fever. SECTION C CONTINUED ON NEXT PAGE -17 PROBLEM 2 SECTION C contd. 15. Any cough. 16. D e s c r i p t i o n of any cough. 17. Nasal discharge. 18. Nasal congestion. 19. Post nasal d r i p . 20. Sore t h r o a t . 21. Muscle aches and pains. 22. Headache. 23. Swollen glands or sore neck. 24. Nausea and vomiting. 25. Ear ache. 26. Night sweats. 27. Duration of the symptoms. 28. Shortness of breath. 29: Trouble s l e e p i n g . 30. Bowel movements. 31. A l l e r g i e s . 32. I f there are any small c h i l d -ren around house. 33. A p p e t i t e . 34. Does he sleep w i t h a window open. 35. Are there other members of the f a m i l y a f f l i c t e d with a ".cold". 36. Does he smoke c i g a r e t t e s . 37. Is he experiencing any i n -d i g e s t i o n . At t h i s time you would (Choose ONLY ONE): 38. Recommend some non-pres-c r i p t i o n medication to a l l e v i a t e the symptoms described. SECTION C contd. 39. Inquire about other ailments he may have and medications he may be t a k i n g . 40. Recommend th a t he see h i s f a m i l y p h y s i c i a n s i n c e the symptoms de-s c r i b e d i n d i c a t e s a s e r i o u s upper r e s p i r a t o r y t r a c t i n f e c t i o n . 41. Recommend that because he e x h i b i t s symptoms of a common c o l d he should take ASA, d r i n k plenty of f l u i d s , and r e s t f o r the next f i v e to seven days. 42. Recommend he go to " o u t - p a t i e n t s " . SECTION D You would be i n t e r e s t e d i n which of the f o l l o w i n g information ( S e l e c t AS MANY as you consider ESPECIALLY p e r t i n e n t ) : 43. Is he being t r e a t e d by a p h y s i c i a n f o r any other ailment. 44. Is he t a k i n g any p r e s c r i p t i o n medi-c a t i o n s . 45. When was the l a s t time he saw a p h y s i c i a n . 46. Is he takin g any n o n - p r e s c r i p t i o n medication. GO DIRECTLY TO SECTION E. SECTION E At t h i s time you would recommend th a t (Choose ONLY ONE): 47. S i n c e , the symptoms described i n -d i c a t e an upper r e s p i r a t o r y t r a c t i n f e c t i o n of a serious nature he should see h i s f a m i l y p h y s i c i a n . 48. The consumer has the symptoms of a common c o l d and should take ASA and dri n k plenty of f l u i d s and r e s t f o r next two to three days. SECTION E CONTINUED ON NEXT PAGE -178 PROBLEM 2 SECTION E contd. 49. Some n o n - p r e s c r i p t i o n medication to a l l e v i a t e the symptoms de-sc r i b e d i s warranted. 51. Suggest he go to " o u t - p a t i e n t s " . 52. Emphasize the importance of seeing a p h y s i c i a n . I f p o s s i b l e , suggest the name of a phy s i c i a n he could contact or arrange f o r hirn^to contact a p h y s i c i a n . 53. Recommend ASA and a non-prescrip-t i o n product f o r the symptoms and suggest he see a p h y s i c i a n . SECTION F At t h i s time you would recommend (Choose ONLY ONE): •r 54. B e n y l i n w i t h codeine f o r the cough and T r i a m i n i c i n r f o r the nasal congestion. 55. An o r a l product c o n t a i n i n g a sympa-thomimetic and an a n t i h i s t a m i n e . 56. Orande DMr l i q u i d . 57. Throat lozenge. 58. A t o p i c a l nasal decongestant such as O t r i v i n r . 59. Dimetapp r E l i x i r . 60. Suggest t h a t he use a vapourizer i n the evening. 61. Contac C r. 62. C o r i c i d i n r c o l d t a b l e t s . 63. Promatussin r Expectorant. 64. Robi,tussin r. SECTION G At t h i s time you would (Choose ONLY ONE): 65. Ask consumer to describe symptoms. 66. Recommend a product. 67. Suggest that he go to "out-p a t i e n t s " . -179-LATENT IMAGE RESPONSES PROBLEM 2 1. Consumer says he wants your a s s i s t a n c e . Go to Section B. 2. Consumer spends f i v e minutes searching, waits u n t i l the two w a i t i n g customers leave w i t h t h e i r p r e s c r i p t i o n s and then asks you to suggest a product f o r a c o l d . Go to Section B. 3. Go to Sectio n F. 4. Consumer, waits h i s turn and says, "What would you recom-mend f o r a co l d ? " Go to Section B. 5. Go to Section F. 6. Consumer does not have a f a m i l y p h y s i c i a n i n t h i s c i t y . Go to Section G. 7. This upsets one of the consu-mers who has been w a i t i n g f o r ten minutes f o r you to f i l l h i s p r e s c r i p t i o n , he expresses h i s di s p l e a s u r e to you and, a f t e r the d i s c u s s i o n that f o l l o w s , the young man says he w i l l w ait. Go to Secti o n B. 8. Go to Sectio n F. 9. Go to Section C. 10. Wants you to help. Make another choice i n t h i s s e c t i o n . 11. Returns the next day complain-ing t h a t h i s " c o l d " has not im-proved. \.;Make another!.choice i n t h i s s e c t i o n . Have not taken i t , but has f e l t "hot" f o r two days. Yes. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. I do. 19. Quite f i e r c e a t times and b r i n g -up congestion. Yes. Yes, d i f f i c u l t y i n breathing through nose. A s l i g h t post nasal d r i p . Yes, hard to swallow. Feels t i r e d a l l over, no energy. No. Neck not sore, does not know i f glands are swollen. 24. 25. 26. 27. 28. 29. No. No. Did have the " c h i l l s " l a s t n i g h t. Noticed f i r s t signs of " c o l d " 5 or 6 days ago. No. Trouble breathing through nose, kept awake longer than usual l a s t n i g h t . 30. Regular. PROBLEM 2 -180-31. No. 32. Not l i v i n g a t home, he i s h i t c h - h i k i n g across Canada. 33. Good. 34. S l e p t outdoors i n a te n t f o r l a s t two weeks. 35. No. 36. Yes.. 37. No.. 38. Go to Secti o n F. 39. Go to Section D. 40. Has no doctor i n t h i s c i t y . Make another choice from t h i s s e c t i o n . 41. Consumer requests t h a t you suggest something f o r h i s symptoms. Go .to Section F. 42. Consumer thanks you f o r your advice and leaves the s t o r e . END OF PROBLEM. 43. No. 44. No. 45. S i x months ago. 46. No. 47. Says he does not know a phy-s i c i a n i n t h i s town. Make another s e l e c t i o n from t h i s s e c t i o n . 48. Consumer requests t h a t you r e c -ommend something f o r h i s symp-toms. I f you NOW wish to r e c -ommend, a n o n - p r e s c r i p t i o n medication i n a d d i t i o n to ASA, go to Section F. Otherwise, develop response 50. 49. Go to Secti o n F. 50. The consumer expresses extreme d i s p l e a s u r e a t your reluctance to help him by suggesting some-th i n g f o r h i s symptoms and i n -d i c a t e s he s h a l l , f i n d a more cooperative pharmacist. END OF PROBLEM. 51. Consumer thanks you f o r your advice and leaves the s t o r e . Re-turns i n 3 hours with p r e s c r i p -t i o n s f o r P e n b r i t i n r 500 mg. and R o b i t u s s i n r . END OF PROBLEM. 52. Consumer thanks you f o r your help and leaves the s t o r e . Returns i n 3 hours with p r e s c r i p t i o n s f o r P e n b r i t i n r 500 mg. and Robitus-s i n r . END OF PROBLEM. 53. Go to Section F. 54. He purchases product and leaves. The next day the young man r e -turns w i t h a p r e s c r i p t i o n f o r P e n b r i t i n r 500 mg. and Robitus-s i n r . END OF PROBLEM. 55. He purchases product and leaves. The next day the young man r e -turns w i t h a p r e s c r i p t i o n f o r P e n b r i t i n r 500 mg. and R o b i t u s s i n r . END OF PROBLEM. 56. He purchases product and leaves. The next day the young man returns with a p r e s c r i p t i o n f o r P e n b r i t i n r 500 mg. and R o b i t u s s i n r . END OF PROBLEM. PROBLEM 2 -181-57. He purchases product and leaves. The next day the young man returns with a p r e s c r i p t i o n f o r P e n b r i t i n r 500 mg. and Robitussin 1". END OF PROBLEM. 58. He purchases product and leaves. The next day the young man returns with a pre-s c r i p t i o n f o r P e n b r i t i n r 500 mg. and R o b i t u s s i n r . END OF PROBLEM. 64. He purchases product and leaves. The next day the young man returns w i t h a p r e s c r i p t i o n f o r P e n b r i t i n r 500 mg. END OF PROBLEM. 65. Go to Sectio n C. 66. Go to Section F. 67. He i s not too w i l l i n g to take t h i s advice. Make another s e l e c t i o n from t h i s s e c t i o n . 59. He purchases product and leaves. The next day the young man re-turns with a p r e s c r i p t i o n f o r P e n b r i t i n r 500 mg. and Robi-t u s s i n r . END OF PROBLEM. 60. Since he i s " t e n t i n g " he can-not use a vapourizer. Could you suggest something e l s e ? Make another choice from t h i s s e c t i o n . 61. He purchases product and leaves. The next day the young man r e -turns w i t h a p r e s c r i p t i o n f o r P e n b r i t i n r 500 mg. and Robitus-s i n r . END OF PROBLEM. 62. He purchases product and leaves. The next day the young man r e -turns with a p r e s c r i p t i o n f o r P e n b r i t i n r 500 mg. and Robitus-s i n r . END OF PROBLEM. 63. He purchases product and leaves. The next day the young man r e -turns w i t h a p r e s c r i p t i o n f o r P e n b r i t i n r 500 mg. and Robitus-s i n r . END OF PROBLEM. -182-PROBLEM 3 OPENING SCENE You are a community pharmacist employed at MacLean's pharmacy. The sto r e i s l o c a t e d i n a middle c l a s s neighbourhood and i t s owner, Mr. MacLean, has been serving t h i s area f o r 23 years. Mr. MacLean i s t h i n k i n g of r e t i r i n g and would l i k e ;you to buy the business. He has b u i l t up a good r e l a t i o n s h i p w i t h h i s customers and he i s o f t e n c a l l e d upon f o r h i s advice on health matters. You both f a i t h f u l l y maintain a p a t i e n t record system and Mr. MacLean i s very proud of the s e r v i c e t h a t he provides h i s customers. The st o r e i s open from 9a.m. to 9p.m. Monday to Friday and 9a.m. to 6p.m. Saturday. I t i s Wednesday ni g h t and you and one out f r o n t c l e r k are working. I t i s a "slow" n i g h t and at present you are caught up on your p r e s c r i p t i o n orders. There are three consumers browsing i n various areas of the s t o r e . One of the consumers, a young man, asks to speak to Mr. MacLean. You e x p l a i n that i t i s Mr. MacLean's ni g h t o f f and you o f f e r your a s s i s t a n c e . The young man then asks, "What i s the strongest pain r e l i e v e r I may purchase without a p r e s c r i p t i o n ? " SECTION A Considering the circumstances j u s t described you would NOW (choose ONLY ONE): 1. Recommend a strong non-p r e s c r i p t i o n pain r e l i e v e r . 2. D i r e c t him to the appropriate s e c t i o n out f r o n t . 3. Suggest he see a p h y s i c i a n . 4. Suggest, t h a t he go to "out-p a t i e n t s " . 5. Ask him f o r more in f o r m a t i o n . 6. Ask the out f r o n t c l e r k to help him. 7. Suggest t h a t i f the pain does not disappear i n a few days to see a p h y s i c i a n but a t present i t does not warrant treatment with a non-prescrip-t i o n product. SECTION B You would be i n t e r e s t e d i n which of the f o l l o w i n g i n f o r m a t i o n . (Choose as MANY as you consider ESPECIALLY p e r t i n e n t ) : 8. Is the i n d i v i d u a l being t r e a t e d by a p h y s i c i a n . 9. When was the l a s t time he saw a p h y s i c i a n . 10. Does he take any p r e s c r i p t i o n drugs. 11. Does he take any n o n - p r e s c r i p t i o n drugs. PROCEED DIRECTLY TO SECTION C -183-PROBLEM 3 SECTION C At t h i s time you would (Choose ONLY ONE): 12. Recommend t h a t he take a non-p r e s c r i p t i o n pain r e l i e v e r . 13. Recommend th a t he go to "out-p a t i e n t s " i f he i s unable to reach h i s p h y s i c i a n . 14. Suggest that i n your opinion there i s r e a l l y no need to take a pain r e l i e v e r . 15. Look up Fred A r c h i b a l d i n the P a t i e n t Record System. 16. Recommend a n o n - p r e s c r i p t i o n product and suggest c o n s u l t i n g a p h y s i c i a n i f not r e l i e v e d . UNLESS OTHERWISE DIRECTED, GO TO SECTION G. SECTION D At t h i s time you would recommend (Choose ONLY ONE): 17. A s p r i n r . . 18. Aspergum r. 19. TylenoT r. 20. B u f f e r i n g 21. Bromo S e l t z e r r . 22. Enthrophen r. 23. F r o s s t 217 r. 24. F r o s s t 222 r. 25. A n a c i n w i t h Codeine r. 26. Tempra r. 27. E x c e d r i n r . 28. Tylenol # l r . SECTION E You would at t h i s time (Choose ONLY ONE): 29. Recommend a n o n - p r e s c r i p t i o n product. 30. Recommend th a t he see a p h y s i c i a n . 31. Suggest t h a t he go to "out-p a t i e n t s " . 32. Ask f o r more in f o r m a t i o n . 33. Suggest t h a t i n your opinion there i s no need to take a no n - p r e s c r i p t i o n medication. 34. Recommend a n o n - p r e s c r i p t i o n product and suggest c o n s u l t i n g a p h y s i c i a n i f not r e l i e v e d . SECTION F You would be i n t e r e s t e d i n asking about which of the f o l l o w i n g (Se-l e c t as MANY as you consider ESPECIALLY important): 35. Who i s i t f o r . 36. What type of pain. 37. Duration. 38. L o c a t i o n . 39. Age. 40. Fever. 41. Nausea and vomiting. 42. Sore t h r o a t . 43. Muscle aches and pains. 44. Does he smoke. 45. A l l e r g i e s . SECTION F CONTINUED ON NEXT PAGE -184-PROBLEM 3 SECTION F (Cont.) 46. General a l l around h e a l t h . 47. How often does he get these pains. 48. What does he u s u a l l y take f o r pain. 49. How i s his eyesight. At t h i s time you would (Choose ONLY ONE): 50. Recommend a n o n - p r e s c r i p t i o n medication to a l l e v i a t e the pain. 51. I n q u i r e about other ailments he may be taking medications f o r . 52. Recommend th a t he see a p h y s i c i a n . 53. Suggest t h a t the symptoms described do not warrant therapy with a non-prescrip-t i o n medication and t h e r e f o r e , you w i l l not recommend a product. 54. Suggest t h a t i f he cannot reach a p h y s i c i a n t h a t he should go the " o u t - p a t i e n t s " department of the nearest h o s p i t a l . 55. Recommend a n o n - p r e s c r i p t i o n medication and suggest con-s u l t i n g a p h y s i c i a n i f not r e l i e v e d . SECTION G At t h i s time you would (Choose ONLY ONE): 56. Recommend a n o n - p r e s c r i p t i o n medication. 57. Suggest t h a t the i n d i v i d u a l see a p h y s i c i a n . 58. Recommend that no non-p r e s c r i p t i o n medication be taken. 59. Suggest t h a t the i n d i v i d u a l go to the " o u t - p a t i e n t s " department of nearest h o s p i t a l . 60. Recommend a n o n - p r e s c r i p t i o n medication and suggest con-s u l t i n g a p h y s i c i a n i f not r e l i e v e d . -185-LATENT IMAGE RESPONSES PROBLEM 3 1. Go to Sectio n D. 12. Go to Section D. 2. He says t h a t he has searched there and i s confused as to which i s the strongest.. He wants you to recommend some-t h i n g . Go to Sectio n E. .3.. At t h i s hour of the night he wants you to recommend some-t h i n g . Go to Section E. 4. He f e e l s t h i s i s a b i t dras-t i c and wants you to recom- . mend something. Go to Sec-t i o n E. 5. Go to Sectio n F. 6. This upsets him and he wants you to a s s i s t him. Go to Section E. 7. He says t h a t he would l i k e you to recommend something t o n i g h t . Go to Section E. 8. Yes, but does not know f o r what c o n d i t i o n . 9. Do not know e x a c t l y but he does see one o c c a s i o n a l l y . 10. Yes, some w h i t e . p i l l s but does not know what kind. However, h i s f a t h e r - i n -law does get h i s p r e s c r i p -t i o n s f i l l e d a t MacLean's, w i l l t h a t help? 11. Yes, he b e l i e v e s s t r o n g l y i n vitamins and takes some d a i l y . 13. Customer leaves s t o r e . END OF PROBLEM. 14. Customer leaves s t o r e . END OF PROBLEM. 15. He has been r e c e i v i n g Z y l o p r i m r since 1969. He does not take any other medication. No a l l e r g i e s . 16. Go to Sectio n D. 17. He purchases product and leaves s t o r e . END OF PROBLEM. 18. He purchases product and leaves the s t o r e . END OF PROBLEM. 19. He purchases product and leaves the s t o r e . END OF PROBLEM. 20. He purchases product and leaves the s t o r e . END OF PROBLEM. 21. He purchases product and leaves the s t o r e . END OF PROBLEM. 22. He purchases product and leaves the s t o r e . END OF PROBLEM. 23. He purchases product and leaves the s t o r e . END OF PROBLEM. 24. He purchases product and leaves the s t o r e . END OF PROBLEM. 25. He purchases product and leaves the s t o r e . END OF PROBLEM. 26. He purchases product and leaves the s t o r e . END OF PROBLEM. Page 187 does not e x i s t . -186-PROBLEM 3 27. He purchases product and leaves 47. Says t h a t he gets headaches the s t o r e . END OF PROBLEM. o c c a s i o n a l l y but t h i s one has l a s t e d a l l day and he needs to 28. He purchases product and leaves take something. the s t o r e . END OF PROBLEM. 48. Nothing. 29. Go to Section D. 49. Good. 30. Consumer f e e l s t h a t he would be unable to get i n touch with one 50. Go to Section D. t h i s l a t e a t n i g h t . Make another choice from t h i s S e c t i o n . 51. Go to Section B. 31. Consumer leaves the s t o r e . END 52. He leaves the s t o r e . END OF OF PROBLEM. PROBLEM. 32. Go to Section F. 53. He leaves the s t o r e . END OF PROBLEM. 33. Consumer leaves the s t o r e . END OF PROBLEM. 54. He leaves the s t o r e . END OF PROBLEM. 34. Go to Sectio n D. 55. Go to Sectio n D. 35. His f a t h e r - i n - l a w , Fred A r c h i b a l d . 56. Go to Sectio n D. 36. Throbbing headache. 57. Consumer thanks you and leaves s t o r e . END OF PROBLEM. 37. A l l day. 58. Consumer thanks you and leaves 38. Mostly i n . t h e forehead. s t o r e . END OF PROBLEM. 39. Let's see, he has j u s t r e t i r e d 59. Consumer thanks you and leaves so maybe 65. s t o r e . END OF PROBLEM. 40. No. 60. Go to Sectio n D. 41. No. 42. No. 43. No. 44. No. 45. None, th a t I am aware of. 46. He i s h e a l t h i e r than I am. LEAF 187 OMITTED IN PAGE NUMBERING. -188-PROBLEM 4  OPENING SCENE You are a community pharmacist employed a t Mid-City Drugs. The st o r e i s lo c a t e d i n an area composed of mostly working c l a s s , recent im-migrant f a m i l i e s . This p a r t i c u l a r s t o r e has been operated by the same pharmacist f o r ne a r l y t h i r t y years. You are a recent graduate and you are t r y i n g to upgrade the q u a l i t y of pharmaceutical s e r v i c e s provided by t h i s s t o r e . You have added s u b s t a n t i a l l y to the dispensary l i b r a r y and you have persuaded the owner to purchase a P a t i e n t Record System. You both f a i t h f u l l y keep the records up to date although your boss i s skep-t i c a l of t h e i r usefulness. The system has received a mixed r e a c t i o n from c l i e n t e l e . The store i s open 9 a.m. to 9 p.m., Monday to Saturday. I t i s 8 p.m. on a Saturday evening and you are the only pharmacist working i n the s t o r e . There i s one "out f r o n t " c l e r k working the cash and dust i n g shelves. You have f i n i s h e d a l l your p r e s c r i p t i o n orders. Mrs. M i k u l i k , a woman i n her mid t h i r t i e s , whom you recognize as a patron, approaches you. "Could you recommend something f o r pain?", she asks. SECTION A Considering the circumstances j u s t d e s c r i b e d , you would now (Choose ONLY ONE): 1. Recommend a n o n - p r e s c r i p t i o n pain r e l i e v e r . 2. Suggest t h a t she see a p h y s i c i a n . 3. Ask the "out f r o n t " c l e r k to help. 4. Suggest t h a t she choose.from the products i n the appropriate s e c t i o n of the s t o r e . 5. Suggest t h a t she go to "out-p a t i e n t s " . 6. Ask f o r more i n f o r m a t i o n . 7. Suggest t h a t i t i s not wise to self-medicate f o r pain. I f the pain does not disappear i n a few days then she should con-s u l t with her p h y s i c i a n . SECTION B You would be i n t e r e s t e d i n which of the f o l l o w i n g i n f o r m a t i o n . (Choose AS MANY as you consider ESPECIALLY PERTINENT): 8. When was the l a s t time she saw a p h y s i c i a n . 9. Is she c u r r e n t l y being t r e a t e d by a p h y s i c i a n f o r any c o n d i t i o n . 10. Is she taki n g any p r e s c r i p t i o n medication. 11. Does she take any n o n - p r e s c r i p t i o n medication. PROCEED DIRECTLY TO SECTION D. SECTION C At t h i s time you would NOW (Choose only ONE): 12. Suggest t h a t she go to "out-p a t i e n t s " . 13. Recommend a product. -189-PROBLEM 4 SECTION C (cont.) 14. Ask f o r inf o r m a t i o n . 15. Suggest t h a t she not s e l f -medicate and i f .the pain per-s i s t s , she should see a p h y s i c i a n . 16. Suggest she choose a product from the appropriate s e c t i o n out f r o n t . SECTION D At t h i s time you would NOW (Choose ONLY ONE): 17. Recommend a n o n - p r e s c r i p t i o n product. 18. Suggest t h a t she should con-tin u e t a k i n g A s p r i n r and see a p h y s i c i a n next week. 19. Suggest t h a t she go to "out-p a t i e n t s " . 20. Suggest t h a t she see a phy-s i c i a n soon about the symptoms. 21. E x p l a i n that symptoms are not serious enough to war-rant s e l f - m e d i c a t i o n w i t h a no n - p r e s c r i p t i o n product. 22. Look up Mrs. M i k u l i k i n the p a t i e n t record system. UNLESS OTHERWISE DIRECTED GO DIRECTLY TO SECTION E. SECTION E At t h i s time you would (Choose ONLY ONE): 23. Suggest t h a t she make an ap-pointment to see a p h y s i c i a n about the pain. 24. Recommend a n o n - p r e s c r i p t i o n pain r e l i e v e r . SECTION E (cont.) 25. Suggest t h a t she go to "out-p a t i e n t s " . 26. Suggest t h a t the symptoms de-s c r i b e d are not serious enough to warrant s e l f - m e d i e a t i n g . 27. Suggest doubling the dose of the A s p r i n r and c o n s u l t i n g her ph y s i c i a n next week. SECTION F. At t h i s time you would (Choose ONLY ONE): 28. Suggest t h a t she see a p h y s i c i a n . 29. Ask f o r more in f o r m a t i o n . 30. Recommend a n o n - p r e s c r i p t i o n product. 31. Suggest t h a t she should not s e l f -medicate, and i f the pain i s no b e t t e r on Monday, she should see a p h y s i c i a n . SECTION G At t h i s time you would now recommend (Choose ONLY ONE): 32. Entrophen r. 33. A s p r i n r . 34. F r o s s t 222 r. 35. Excedrin . 36. Alka S e l t z e r r . 37. A n a c i n r . 38. Anacin with Codeine r. 39. C-2's r. 40. F r o s s t 217 S t r o n g r . 41. Aspergum r. 42. E c o t r i n r . 43. Tempra r. 44. T y l e n o l r . 45. Tylenol # l r . PROBLEM 4 -190-SECTION H You would be i n t e r e s t e d i n asking about which of the f o l l o w i n g ( S e l e c t AS MANY as you consider ESPECIALLY IMPORTANT): 46. Who i s i t f o r . 47. D e s c r i p t i o n of pain. 48. Duration. 49. L o c a t i o n . 50. Age. 51. Fever. 52. Nausea and vomiting. 53. Muscle aches and pains. 54. Does she smoke. 55. A l l e r g i e s . 56. Aside from the p a i n , how do you f e e l . 57: How often do you get these pains. PROCEED DIRECTLY TO SECTION B. -191-LATENT IMAGE RESPONSES PROBLEM 4 .1. Go to Section G. 2. At t h i s hour of nig h t she f e e l s she would be unable to reach her p h y s i c i a n . Go to Section C. ,3. Mrs. M i k u l i k would l i k e you to a s s i s t her. Go to Section C. .4. "She has looked a t those pro-ducts and could you recommend a strong one." Go to Section C. 5. "Is th a t r e a l l y necessary? Could you not recommend some-th i n g ? " Go to Sectio n F. 6. Go to Section H. 7. "Could you not recommend some-thin g to r e l i e v e the pain over the week-end?" Go to Sectio n C. 8. S i x months ago f o r renewal of her b i r t h c o n t r o l p r e s c r i p -t i o n . 9. No. 10. Only her b i r t h c o n t r o l p i l l s . 11. She has been taki n g 3 or 4 A s p r i n r t a b l e t s / d a y f o r the pain f o r about 10 days. They do not appear to be helping very much. She o c c a s i o n a l l y takes E x l a x r p i l l s . 12. She f e e l s t h i s i s a b i t dras-t i c . Make another s e l e c t i o n from t h i s s e c t i o n . 14. Go to Sectio n H. 15. She says the pain i s r e a l l y q u i t e annoying. Can't you suggest something. Make anoth-er choice from t h i s s e c t i o n . 16. Has searched and i s confused. Could you recommend something. Go to Se c t i o n F. 17. Go to Sectio n G. 18. She thanks you and leaves the s t o r e . END OF PROBLEM. 19. She thanks you and leaves the s t o r e . END OF PROBLEM. 20. She thanks you and leaves the s t o r e . END OF PROBLEM. 21. She leaves the s t o r e . END OF PROBLEM. 22. She has been t a k i n g Ortho l / 8 0 r s i n c e Dec. 1973. No other p r e s c r i p t i o n s noted, no a l l e r -g i e s . 23. She thanks you f o r your advice and leaves. END OF PROBLEM. 24. Go to Sectio n G. 25. She thanks you f o r your advice and leaves the s t o r e . END OF PROBLEM. 26. She thanks you f o r your advice and leaves. END OF PROBLEM. 27. She thanks you f o r your advice and leaves. END OF PROBLEM. 13. Go to Section G. -192-PROBLEM 4 28. On Saturday n i g h t she f e e l s she would be unable to reach him, could you not recommend something. Make another s e l e c t i o n from t h i s s e c t i o n . 29, Go to Section H. 30. Go to Se c t i o n G. 31. She leaves the s t o r e . END OF PROBLEM. 32. She purchases product and leaves. END OF PROBLEM. 33. She purchases product and leaves. END OF PROBLEM. 34. She purchases product and leaves. END OF PROBLEM. 35. She purchases product and . leaves. END OF PROBLEM. 36. She purchases product and leaves. END OF PROBLEM. 37. She purchases product and leaves. END OF PROBLEM. 38. She purchases product and leaves. END OF PROBLEM. 39. She purchases product and leaves. END OF PROBLEM. 40. She purchases product and leaves. END OF PROBLEM. 41. She purchases product and leaves. END OF PROBLEM. 42. She purchases product and leaves. END OF PROBLEM. 43. She purchases product and leaves. END OF PROBLEM. 44. She pruchases product and leaves. END OF PROBLEM. 45. She purchases product and leaves. END OF PROBLEM. 46. H e r s e l f . 47. P e r s i s t e n t . 48. Has had these pains o f f and on f o r some months. 49. Wrists and j o i n t s of hands. 50. 37. 51. No. 52. No. 53. Feels q u i t e s t i f f f o r the f i r s t l i t t l e w h i l e i n the morning. 54. Yes. 55. None. 56. A l r i g h t I guess, I f i n d t h a t I get t i r e d i n the mid afternoon and must r e s t . 57. L a t e l y i t seems the pain i s always there. - 1 9 3 -Fig. 3 SIMULATION ONE Possible Paths to Problem Solution End of problem End of problem -<--A 1 E OPTIMAL PATH ALTERNATE PATH End of problem G End of problem -194-ig. 4 SIMULATION TWO Possible Paths to Problem Solutio B 1 End of problem » * End of problem r End of problem OPTIMAL PATH ALTERNATE PATH -195-Fig. 5. SIMULATION THREE Possible Paths to Problem Solution End of problem End of problem End of problem End of problem OPTIMAL PATH ALTERNATE PATH -196-6. SIMULATION FOUR Possible Paths to Problem Solution • End of problem • • End of problem OPTIMAL PATH ALTERNATE PATH End of problem • • End of problem TABLE XIV INTERCORRELATIONS OF THE TEN JUDGES' RATINGS OF THE 267 OPTIONS IN SIMULATIONS ONE TO FOUR Judge 1 2. 3 4 5 6 7 . 8 9 1 0 1 .7859 .8305 .7754 .7632 .7357 .8206 .6907 .6788 .7879 2 .7000 .7381 .7841 .6982 .7646 .6566 .6855 .7930 3 .7293 .7238 .7063 .7518 .6380 .6910 .7112 4 .8037 .6820 .8309 .6783 .7665 .7125 5 .6401 .7983 .7073 .7560 .7748 6 .7178 .6754. .7159 .7742 7 .6924 .7236 .7657 8 .7061 .6810 9 .7008 10 Mean c o r r e l a t i o n = 0.7313 -198-APPENDIX D: PRE AND POST TESTS FOR FACTUAL KNOWLEDGE Page 1. Cold Medication Pre-Test 2. Cold Medication Post-Test 3. Analgesics Pre-Test 4. Analgesics Post-Test -199-LINK A d v i s i n g P a t i e n t s on Non P r e s c r i p t i o n Medications Cold Medications PRE TEST True/False: C i r c l e e i t h e r T or F to i n d i c a t e the answer t h a t you b e l i e v e most app r o p r i a t e . 1. A p a t i e n t on a regimen of p r o p y l t h i o u r a c i l can s a f e l y take E l t o r capsules to r e l i e v e h i s c o l d symptoms. T F 2. Fever i s a symptom that u s u a l l y accompanies the common c o l d . T F 3. A l l e r g i c r h i n i t i s and r e s p i r a t o r y i n f e c t i o n s due to Strep, pyogenes may produce symptoms resembling a common c o l d . T F 4. Topical nasal decongestants take longer to be absorbed i n t o the systemic c i r c u l a t i o n and th e r e f o r e have a slower response than systemic nasal decongestants. T F 5. Rebound congestion i s one p o s s i b l e adverse e f f e c t of systemic nasal decongestants. T F 6. Non-pre s c r i p t i o n preparations intended f o r t o p i c a l a p p l i c a t i o n c o n t a i n i n g naphazoline should not be used i n c h i l d r e n and i n f a n t s under 6 years o l d . T F C i r c l e the ONE BEST answer. 1. Which of the f o l l o w i n g i s NOT a c h a r a c t e r i s t i c of the common cold? (a) The incidence i s higher i n males than females. (b) The incidence i s higher i n c h i l d r e n than a d u l t s . (c) The incidence i s higher i n females than males. (d) The incidence i s higher i n winter than summer. (e) The " c o l d " accounts f o r approximately 40% of the l o s t time from work. 2. Which of the f o l l o w i n g i s u s u a l l y NOT a symptom of the common c o l d i n a d u l t s ? (a) nasal o b s t r u c t i o n (b) nasal discharge (c) f e v e r (d) sore t h r o a t (e) sneezing 3. Which of the f o l l o w i n g i s NOT a general measure to recommend each time to r e l i e v e the symptoms of the common cold? (a) bed r e s t (b) h u m i d i f i c a t i o n (c) increase f l u i d i n t a k e (d) A.S.A. or Acetaminophen (e) Antihistamines and/or sympathomimetic amines -200-4. The r o l e of sympathomimetics i n the treatment of the symptoms of a c o l d i s : (a) to s t i m u l a t e the adrenergic receptors to cause bronchocon-s t r i c t i o n to ease breathing (b) to increase the flow of blood to the nasal mucosa (c) to d i l a t e the blood v e s s e l s i n the nasal mucosa (d) to s t i m u l a t e the adrenergic receptors of the v a s c u l a r smooth muscle and thus reduce the blood flow to the nasal mucosa (e) a l l of the above 5. Which of the f o l l o w i n g nasal decongestants may be administered t o p i c a l l y and s y s t e m i c a l l y : (a) oxymetazoline (b) phenylephrine (c) propylephrine (d) xylometazoline (e) pseudoephedrine 6. A consumer has asked f o r a s s i s t a n c e i n s e l e c t i n g a n o n - p r e s c r i p t i o n c o l d product. He has the f o l l o w i n g symptoms; sore t h r o a t , post nasal d r i p , nasal congestion, a f e v e r and sore muscles. He has a productive cough producing greenish phlegm. He says the symptoms were f i r s t n o t i c ed about seven days ago. The i n d i v i d u a l i s not being t r e a t e d by a p h y s i c i a n , takes_;no p r e s c r i p t i o n nor n o n - p r e s c r i p t i o n drugs and has no a l l e r g i e s . Which of the f o l l o w i n g would be the best advice to t h i s customer? (a) recommend an o r a l c o l d product (b) recommend a t o p i c a l nasal decongestant (c) suggest t h a t he see a p h y s i c i a n as soon as p o s s i b l e (d) recommend n o n - p r e s c r i p t i o n products to r e l i e v e the cough and congestion (e) both c and d 7. A consumer has asked f o r a s s i s t a n c e i n s e l e c t i n g a n o n - p r e s c r i p t i o n c o l d product. She has the f o l l o w i n g symptoms; sore t h r o a t , nasal congestion, post nasal d r i p and a s l i g h t cough which i s non-produc-t i v e i n nature. She has no f e v e r . The symptoms have developed over the l a s t two days. She i s a d i a b e t i c and has been t a k i n g Diabinese^ f o r several years. She says she takes the occasional l a x a t i v e . She has no a l l e r g i e s . Which of the f o l l o w i n g would be the best advice f o r t h i s i n d i v i d u a l ? (a) recommend an o r a l c o l d product (b) recommend a t o p i c a l nasal decongestant (c) suggest t h a t she see a p h y s i c i a n as soon as p o s s i b l e (d) recommend an o r a l c o l d product f o r the congestion and an a n t i t u s s i v e cough preparation (e) both c and d 8. A consumer asks you f o r a s s i s t a n c e concerning what he b e l i e v e s to be a " c o l d " . He says he i s bothered by nasal congestion, a m i l d sore t h r o a t and a post nasal d r i p which i s keeping him awake at n i g h t . The symptoms developed i n the l a s t 24 hours. He says he has no f e v e r , no cough and g e n e r a l l y f e e l s good. He takes no p r e s c r i p t i o n medication. -201-He has no drug a l l e r g i e s . Which of the f o l l o w i n g would be the best advice f o r t h i s i n d i v i d u a l ? (a) recommend an o r a l c o l d product c o n t a i n i n g a nasal decongestant (b) recommend a t o p i c a l nasal decongestant (c) recommend he see a p h y s i c i a n as soon as p o s s i b l e (d) a l l of the above (e) e i t h e r a or b 9. Mrs. Smith wants your a s s i s t a n c e . She says she i s f e e l i n g " t e r r i b l e " and a t t r i b u t e s i t to a " c o l d " . She says her nose i s plugged and congested. She a l s o complains of a scratchy t h r o a t , a post nasal d r i p . She has a s l i g h t cough. She does not have a f e v e r . She has taken medicine f o r "high blood pressure" f o r f i v e y ears. She says she takes no n o n - p r e s c r i p t i o n medications and has no drug a l l e r g i e s . Which of the f o l l o w i n g would be the best advice: (a) recommend an o r a l c o l d product c o n t a i n i n g a nasal decon-gestant (b) recommend a t o p i c a l nasal decongestant (c) recommend a expectorant cough product (d) recommend that she see a p h y s i c i a n as soon as p o s s i b l e (e) both c and d -202-LINK Ad v i s i n g P a t i e n t s On Non P r e s c r i p t i o n Medications Cold Medications POST TEST' True/False: C i r c l e e i t h e r T or F to i n d i c a t e the answer th a t you be l i e v e i s most app r o p r i a t e . 1. Topical nasal decongestants take longer to be absorbed i n t o the systemic c i r c u l a t i o n and there f o r e have a slower response than systemic nasal decongestants. T F 2. N o n - p r e s c r i p t i o n preparations intended f o r t o p i c a l a p p l i c a t i o n c o n t a i n i n g naphazoline should not be used i n c h i l d r e n and i n f a n t s under 6 years o l d . T F 3. A l l e r g i c r h i n i t i s and r e s p i r a t o r y i n f e c t i o n s due to Strep, pyogenes may produce symptoms resembling a common c o l d . T F 4. A p a t i e n t on a regimen of p r o p y l t h i o u r a c i l can s a f e l y take El t o r capsules to r e l i e v e h i s c o l d symptoms. T F 5. Rebound congestion i s one p o s s i b l e adverse e f f e c t of systemic nasal decongestants. T F 6. Fever i s a symptom that u s u a l l y accompanies the common c o l d . T F C i r c l e the ONE BEST answer. 1. Which of the f o l l o w i n g i s NOT a general measure to recommend each time to r e l i e v e the symptoms of the common cold? (a) h u m i d i f i c a t i o n (b) Antihistamines and/or sympathomimetic amines (c) bed r e s t (d) A.S.A. or Acetaminophen (e) increase f l u i d i n t a k e 2. A consumer has asked f o r a s s i s t a n c e i n s e l e c t i n g a n o n - p r e s c r i p t i o n c o l d product. He has the f o l l o w i n g symptoms; sore t h r o a t , post nasal d r i p , nasal congestion, a f e v e r and sore muscles. He has a productive cough producing greenish phlegm. He says the symptoms were f i r s t n o t i c e d about seven days ago. The i n d i v i d u a l i s not being t r e a t e d by a p h y s i c i a n , takes no p r e s c r i p t i o n nor non-p r e s c r i p t i o n drugs and has no a l l e r g i e s . Which of the f o l l o w i n g would be the best advice to t h i s customer? (a) recommend n o n - p r e s c r i p t i o n products to r e l i e v e the cough and congestion (b) suggest that he see a ph y s i c i a n as soon as p o s s i b l e (c) recommend an o r a l c o l d product (d) recommend a t o p i c a l nasal decongestant (e) both a and b 3. A consumer asks you f o r a s s i s t a n c e concerning what he be l i e v e s to be a " c o l d " . He says he i s bothered by nasal congestion, a m i l d sore -203-t h r o a t and a post nasal d r i p which i s keeping him awake at n i g h t . The symptoms developed i n the l a s t 24 hours. He says he has no f e v e r , no cough and g e n e r a l l y f e e l s good. He takes no p r e s c r i p t i o n medication nor n o n - p r e s c r i p t i o n medication. He has no drug a l l e r g i e s . Which of the f o l l o w i n g would be the best advice f o r t h i s i n d i v i d u a l ? (a) recommend he see a p h y s i c i a n as soon as p o s s i b l e (b) recommend an o r a l c o l d product c o n t a i n i n g a nasal decongestant (c) recommend a t o p i c a l nasal decongestant (d) e i t h e r b'or c (e) a l l of the above 4. Which of the f o l l o w i n g nasal decongestants may be administered t o p i c a l l y and s y s t e m i c a l l y : (a) propylephrine (b) pseudoephedrine (c) oxymetazoline (d) phenylephrine (e) xylometazoline 5. The r o l e of sympathomimetics i n the treatment of the symptoms of a c o l d i s : (a) to s t i m u l a t e the adrenergic receptors of the vas c u l a r smooth muscle and thus reduce the blood flow to the nasal mucosa (b) to d i l a t e the blood v e s s e l s i n the nasal mucosa (c) to increase the flow of blood to the nasal mucosa (d) to s t i m u l a t e the adrenergic receptors to cause bronchocon-s t r i c t i o n to ease breathing (e) a l l of the above 6. Which of the f o l l o w i n g i s u s u a l l y NOT a symptom of the common c o l d i n adul t s ? (a) f e v e r (b) sneezing (c) sore t h r o a t (d) nasal discharge (e) nasal o b s t r u c t i o n 7. Which of the f o l l o w i n g i s NOT a c h a r a c t e r i s t i c of the common cold? (a) the " c o l d " accounts f o r approximately 40% of the l o s t time from work (b) the incidence i s higher i n males than females (c) the incidence i s higher i n females than males (d) the incidence i s higher i n wi n t e r than summer (e) the incidence i s higher i n c h i l d r e n than a d u l t s 8. A consumer has asked f o r a s s i s t a n c e i n s e l e c t i n g a n o n - p r e s c r i p t i o n c o l d product. She has the f o l l o w i n g symptoms; sore t h r o a t , nasal congestion, post nasal d r i p and a s l i g h t cough which i s non-productive i n nature. She i s a d i a b e t i c and has been t a k i n g Diabinese^ f o r several years. She says she takes the occasional l a x a t i v e . She has no a l l e r g i e s . Which of the f o l l o w i n g would be the best advice f o r t h i s i n d i v i d u a l ? (a) recommend an o r a l c o l d product f o r the congestion and an a n t i t u s s i v e cough preparation (b) recommend an o r a l c o l d product -204-(c) suggest t h a t she see a p h y s i c i a n as soon as p o s s i b l e (d) recommend a t o p i c a l nasal decongestant (e) both c and d 9. Mrs. Smith wants your a s s i s t a n c e . She says she i s f e e l i n g " t e r r i b l e " and a t t r i b u t e s i t to a " c o l d " . She says her nose i s plugged and congested. She a s l o complains of a scratchy t h r o a t , a post nasal d r i p . She has a s l i g h t cough. She does not have a f e v e r . She has taken medicine f o r "high blood pressure" f o r f i v e years. She says she take no n o n - p r e s c r i p t i o n medications and has no drug a l l e r g i e s . Which of the f o l l o w i n g would be the best advice: (a) recommend a expectorant cough product (b) recommend an o r a l c o l d product c o n t a i n i n g a nasal decongestant (c) recommend a t o p i c a l nasal decongestant (d) recommend that she see a p h y s i c i a n as soon as p o s s i b l e (e) both c and d -205-L INK A d v i s i n g P a t i e n t s on Non P r e s c r i p t i o n Medications Analgesics PRE TEST C i r c l e the ONE BEST answer. A. A s p i r i n i n t e r a c t s with numerous p r e s c r i p t i o n drugs, sometimes with serious c o m p l i c a t i o n s . Assume that you have j u s t dispensed the f o l l o w i n g p r e s c r i p t i o n s . Which p a t i e n t should be warned about the i n t e r a c t i o n and advised to avoid concurrent i n g e s t i o n of a s p i r i n : The p a t i e n t on: (a) Diazepam (Valium) (b) Pentids (c) Coumadin (d) H y d r o d i u r i l (e) Ismelin The p a t i e n t on: (a) Tuinal (b) Inderal (c) G a n t r i s i n (d) Diabinese (e) Lanoxin The p a t i e n t on: (a) Achromycin (b) Dyazide (c) Thyroid (d) N i t r o g l y c e r i n (e) B u t a z o l i d i n B. Acetaminophen i s a useful a l t e r n a t i v e to a s p i r i n i n p a t i e n t s who are a l l e r g i c to the l a t t e r . However, acetaminophen cannot s u b s t i t u t e f o r a s p i r i n i n a l l i n s t a n c e s . Which of the f o l l o w i n g p a t i e n t s would not b e n e f i t from a switch to acetaminophen: The p a t i e n t needing r e l i e f from: (a) f e v e r (b) headaches (c) muscle pain (d) inflammation of the j o i n t s (e) b and c above C. The b i p h a s i c a c t i o n of the s a l i c y l a t e s i n u r i c a c i d clearance i s a dose-related phenomenon. Which of the f o l l o w i n g doses of a s p i r i n would b r i n g out the symptoms of a b o r d e r l i n e gout p a t i e n t ? (a) l e s s than 2.4 gms per day (b) 3 to 6 gms per day (c) 9 to 10 gms per day (d) more than 10 gms per day (e) a and d above -206-D. In t a l k i n g to a p a t i e n t you l e a r n t h a t he has a h i s t o r y of severe peptic u l c e r disease. He i s not ta k i n g any p r e s c r i p t i o n or non-p r e s c r i p t i o n drugs c u r r e n t l y . 1. He asks f o r your advice on an e f f e c t i v e p a i n - r e l i e v e r which of the f o l l o w i n g would you suggest? (a) F r o s s t ' s 222's (b) E c o t r i n (c) Tylenol (d) Entrophen (e) Alka S e l t z e r 2. I f the u l c e r p a t i e n t a l s o s u f f e r e d from angina and you knew he was takin g n i t r o g l y c e r i n to r e l i e v e i t which of the f o l l o w i n g products would help r e l i e v e h i s headache: (a) Sinutabs (b) Campain (c) F r o s s t ' s 222's (d) a l l of the above (e) none of the above E. A middle-aged female with a h i s t o r y of chronic rhinorrhea has been takin g a s p i r i n f o r menstrual cramps. She reports severe shortness of breath and wonders why. What would you t e l l her? (a) her asthma i s r e l a t e d to the rhinorrhea (b) her asthma i s due to an a s p i r i n a l l e r g y (c) her asthma i s a r e s u l t of the rhinorrhea and menstruation (d) her asthma i s unrelated to any of the f a c t o r s named F. A p a t i e n t s t a b i l i z e d on Benemid asks i f there would be any harm i n the occasional dose of a s p i r i n to r e l i e v e h i s pains. What would you t e l l him? (a) there i s no reason why he shouldn't take a s p i r i n (b) the a s p i r i n w i l l a f f e c t h i s Benemid regimen but only s l i g h t l y , so i t can be taken (c) he should n o t i f y h i s p h y s i c i a n so that the dose of Benemid can be adjusted (d) the combination of a s p i r i n and Benemid should be avoided a t a l l c osts -207-LINK _ A d v i s i n g P a t i e n t s on Non P r e s c r i p t i o n Medications Analgesics POST. TESTS C i r c l e the ONE BEST answer. A. The Diphasic a c t i o n of the s a l i c y l a t e s i n u r i c a c i d clearance i s a dose-related phenomenon. Which of the f o l l o w i n g doses of a s p i r i n would b r i n g out the symptoms of a b o r d e r l i n e gout p a t i e n t ? (a) more than 10 gms per day (b) l e s s than 2.4 gms per day (c) 3 to 6 gms per day (d) 9 to 10 gms per day (e) a and d above B. Acetaminophen i s a useful a l t e r n a t i v e to a s p i r i n i n p a t i e n t s who are a l l e r g i c to the l a t t e r . However, acetaminophen cannot s u b s t i t u t e f o r a s p i r i n i n a l l i n s t a n c e s . Which of the f o l l o w i n g p a t i e n t s would not b e n e f i t from a switch to acetaminophen: The p a t i e n t needing r e l i e f from: (a) inflammation of the j o i n t s (b) f e v e r (c) headaches (d) muscle pain (e) b and c above C. A p a t i e n t s t a b i l i z e d on Benemid asks i f there would be any harm i n the occasional dose of a s p i r i n to r e l i e v e h i s pains. What would you t e l l him? (a) the combination of a s p i r i n and Benemid should be avoided at a l l c o s t s (b) there i s no reason why he shouldn't take a s p i r i n (c) he should n o t i f y h i s p h y s i c i a n so t h a t the dose of Benemid can be adjusted (d) the a s p i r i n w i l l a f f e c t h i s Benemid regimen but only s l i g h t l y , so i t can be taken D. In t a l k i n g to a p a t i e n t you l e a r n t h a t he has a h i s t o r y of severe p e p t i c u l c e r d i s e a s e . He i s not t a k i n g any p r e s c r i p t i o n or non-p r e s c r i p t i o n drugs c u r r e n t l y . 1. He asks f o r your advice on an e f f e c t i v e p a i n - r e l i e v e r which of the f o l l o w i n g would you suggest? (a) F r o s s t ' s 222's (b) E c o t r i n (c) Tylenol (d) Alka S e l t z e r (e) Entrophen -208-1. I f the u l c e r p a t i e n t a l s o s u f f e r e d from angina and you knew he was ta k i n g n i t r o g l y c e r i n to r e l i e v e i t which of the f o l l o w i n g products would help r e l i e v e h i s headache: (a) F r o s s t ' s 222's (b) Campain (c) Sinutabs (d) a l l of the above (e) none of the above A s p i r i n i n t e r a c t s w i t h numerous p r e s c r i p t i o n drugs, sometimes with serious c o m p l i c a t i o n s . Assume th a t you have j u s t dispensed the f o l l o w i n g p r e s c r i p t i o n s . Which p a t i e n t should be warned about the i n t e r a c t i o n and advised to avoid concurrent i n g e s t i o n of a s p i r i n : 1. The p a t i e n t on: (a) N i t r o g l y c e r i n (b) B u t a z o l i d i n (c) Dyazide (d) Achromycin (e) Thyroid 2. The p a t i e n t on: (a) Diazepam (Valium) (b) H y d r o d i u r i l (c) Ismelin (d) Pentids (e) Coumadin 3. The p a t i e n t on: (a) Diabinese (b) Lanoxin (c) Inderal (d) Tuinal (e) G a n t r i s i n A middle-aged female with a h i s t o r y of chronic rhinorrehea has been ta k i n g a s p i r i n f o r menstrual cramps. She reports severe shortness of breath and wonders why. What would you t e l l her? (a) her asthma i s unrelated to any of the f a c t o r s named (b) her asthma i s r e l a t e d to the rhinorrhea (c) her asthma i s a r e s u l t of the rhinorrhea and menstruation (d) her asthma i s due to an a s p i r i n a l l e r g y -209-APPENDIX E: EVALUATION FORM Page 1. Continuing Education Program Evaluation Form. 2. Table XXXIV. The I n t e r c o r r e l a t i o n s of the P a r t i c i p a n t s ' Ratings of the Program Dimensions, Year of Graduation and Attendance at Previous Courses. -210-F a c u l t y of Pharmaceutical Sciences U n i v e r s i t y o f B r i t i s h Columbia LINK CONTINUING EDUCATION COURSE EVALUATION ADVISING PATIENTS ON THE USE OF NON PRESCRIPTION DRUGS To help us present programs t h a t meet your needs please complete form as d i r e c t e d . YEAR OF GRADUATION OR FIRST LICENSURE This i s your opportunity to evaluate c e r t a i n aspects of t h i s course. We would l i k e you to compare t h i s c o n t i n u i n g education program with the average conti n u i n g education course you have attended i n the past. (The comparison i s to be made by drawing l i n e s to i n d i c a t e the extent of any d i f f e r e n c e . I f i n your opinion the present course i s only one h a l f as good as the average c o n t i n u i n g pharmaceutical education course then draw a l i n e one h a l f as long. I f i n your opinion t h i s course i s two-and-one-half times b e t t e r than the average c o n t i n u i n g pharmaceutical education ocurse, draw a l i n e two-and-one-half times as long as the standard l i n e . ) I. I f the average conti n u i n g education course can be represented by a j l i n e of t h i s length f o r each of the f o l l o w i n g items: could you now i n d i c a t e how s a t i s f i e d you are with the present program on the f o l l o w i n g dimensions? Example. Opportunities f o r t a l k i n g to members of opposite sex. Average C E . course. t h i s course.  1. Usefulness of m a t e r i a l learned. Average C E . course. H jThis C E . course. 2. Use of m a t e r i a l f o r a d v i s i n g p a t i e n t s . Average C E . course. This C E . course. 3. M a t e r i a l too elementary. Average C E . Course. This C E . course. -211-4. Emphasis on drug products. Average C E . course. This C E . course. 5. Emphasis on signs and symptoms of r e l e v a n t diseases. Average C E . course. This C E . course. I 6. Length of l e c t u r e s . Average C E . course. I „ . This C E . course. I 7. Speakers were knowledgeable and presented m a t e r i a l w e l l . Average C E . course. I This course. 8. Usefulness of handouts. Average C E . course. I This C E . course. I 9. Opportunity f o r d i s c u s s i o n . Average C E . course. h i s C E . course. I TO. Usefulness of pre/post t e s t s . Average C E . course. I • jThis course. .11. E f f i c i e n t l e a r n i n g experience. (Was i t worth your time and money I f o r what you learned?) Average C E . course. .This C E . course. -212-12. P r e f e r a b l e l e a r n i n g experience. (Future courses should be presented l i k e t h i s one) Average C.E. course. This C.E. course. I I I . I f the length of the l i n e below represents the amount of l e a r n i n g that can be a t t r i b u t e d to the average 30 minute l e c t u r e i n a continuing education course, average l e a r n i n g i n a 30 minute l e c t u r e : Would you please i n d i c a t e the amount of l e a r n i n g a t t r i b u t a b l e to the f o l l o w i n g a c t i v i t i e s i n t h i s program? Example. Coffee breaks. Average l e c t u r e . I Coffee breaks i n t h i s course 1. Si m u l a t i o n s . 2. Videotapes. 3. Pre/post t e s t s . 4. The l e c t u r e s . 5. Buzz groups. 6. Large group d i s c u s s i o n w i t h speakers. 7. The question and answer periods. I I I . HOW MANY C.E. COURSES HAVE YOU ATTENDED IN THE PREVIOUS 3 YEARS? -213-TABLE XXXIV THE INTERCORRELATIONS OF THE PARTICIPANTS' RATINGS OF THE PROGRAM DIMENSIONS, YEAR OF GRADUATION AND ATTENDANCE AT PREVIOUS COURSES VARIABLE 1 2 3 4 5 6 7 •8 9 10 11 12 13 14 1 -0.27 -0.42* 0.02 -0.39 -0.37 -0.17 -0.39 -0.56 -0.20 -0.41 ' -0.28 -0.39 -0.30 2 0.76 0.15 0.49 0.72 0.18 0.68 0.70 0.02 0.30 0.66 0.44 -0.07 3 0.07 0.58 0.73 0.16 0.67 0.76 0.05 0.21 0.61 0.50 0.15 4 -0.24 0.00 -0.21 0.05 0.25 -0.15 -0.31 -0.20 -0.05 -0.06 5 0.45 -0.20 0.25 0.53 0.06 0.47 0.50 0.47 0.15 6 0.09 0.59 0.59 0.16 0.15 0.45 0.52 0.24 7 0.60 0.42 0.28 0.17 0.30 0.03 -0.50 0 0.75 0.22 0.24 0.70 0.53 -0.29 9 0.24 0.14 0.54 0.45 -0.03 10 0.20 0.26 0.41 -0.06 11 0.33 0.13 -0.16 12 0.54 -0.17 13 0.21 14 VARIABLES: 1 = YEAR OF GRADUATION, 2 = USEFULNESS OF MATERIAL, 3 = USE OF MATERIAL FOR ADVISING PATIENTS, 4 = MATERIAL TOO ELEMENTARY, 5 = PROPER EMPHASIS ON DRUG PRODUCTS, 6 = PROPER EMPHASIS OH THE SIGNS AND SYMPTOMS OF THE RELEVANT DISEASES, 7 = LENGTH OF LECTURES, 8 = SPEAKERS WERE KNOWLEDGE-ABLE AND PRESENTED MATERIAL WELL, 9 = USEFULNESS OF HANDOUTS, 10 = OPPORTUNITY FOR DISCUSSION, 11 = USEFULNESS OF PRE/POST TESTS, 12 = EFFICIENT LEARNING EXPERIENCE, 13 = PREFERABLE LEARNING EXPERIENCE, 14 = NUMBER OF PREVIOUS C.E. COURSES ATTENDED. * THOSE COEFFICIENTS UNDERLINED ARE SIGNIFICANT AT THE 0.05 LEVEL OF SIGNIFICANCE. 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0055955/manifest

Comment

Related Items