UBC Faculty Research and Publications

Development of an instrument to assess the impact of an enhanced experiential model on pharmacy students'… Kassam, Rosemin; Poole, Gary; Collins, John B Apr 8, 2008

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-12909_2007_Article_227.pdf [ 545.16kB ]
JSON: 52383-1.0223538.json
JSON-LD: 52383-1.0223538-ld.json
RDF/XML (Pretty): 52383-1.0223538-rdf.xml
RDF/JSON: 52383-1.0223538-rdf.json
Turtle: 52383-1.0223538-turtle.txt
N-Triples: 52383-1.0223538-rdf-ntriples.txt
Original Record: 52383-1.0223538-source.json
Full Text

Full Text

ralssBioMed CentBMC Medical EducationOpen AcceResearch articleDevelopment of an instrument to assess the impact of an enhanced experiential model on pharmacy students' learning opportunities, skills and attitudes: A retrospective comparative-experimentalist studyRosemin Kassam*†1, Gary Poole†2 and John B Collins†3Address: 1Faculty of Pharmaceutical Sciences, The University of British Columbia, Vancouver, British Columbia, Canada, 2Centre for Teaching and Academic Growth, The University of British Columbia, Vancouver, British Columbia, Canada and 3Department of Educational Studies, The University of British Columbia, Vancouver, British Columbia, CanadaEmail: Rosemin Kassam* - rokassam@interchange.ubc.ca; Gary Poole - gpoole@exchange.ubc.ca; John B Collins - john.collins@ubc.ca* Corresponding author    †Equal contributorsAbstractBackground: Pharmacy schools across North America have been charged to ensure theirstudents are adequately skilled in the principles and practices of pharmaceutical care. Despite thismandate, a large percentage of students experience insufficient opportunities to practice theactivities, tasks and processes essential to pharmaceutical care. The objective of this retrospectivestudy of pharmacy students was to: (1) as "proof of concept", test the overall educational impactof an enhanced advanced pharmacy practice experiential (APPE) model on student competencies;(2) develop an instrument to measure students' and preceptors' experiences; and (3) assess thepsychometric properties of the instrument.Methods: A comparative-experimental design, using student and preceptor surveys, was used toevaluate the impact of the enhanced community-based APPE over the traditional APPE model. Thestudy was grounded in a 5-stage learning model: (1) an enhanced learning climate leads to (2) betterutilization of learning opportunities, including (3) more frequent student/patient consultation, thento (4) improved skills acquisition, thence to (5) more favorable attitudes toward pharmaceuticalcare practice. The intervention included a one-day preceptor workshop, a comprehensive on-sitestudent orientation and extending the experience from two four-week experiences in differentpharmacies to one eight-week in one pharmacy.Results: The 35 student and 38 preceptor survey results favored the enhanced model; withstudents conducting many more patient consultations and reporting greater skills improvement. Inaddition, the student self-assessment suggested changes in attitudes favoring pharmaceutical careprinciples. Psychometric testing showed the instrument to be sensitive, valid and reliable inascertaining differences between the enhanced and traditional arms.Conclusion: The enhanced experiential model positively affects learning opportunities andcompetency acquisition, as measured by a new instrument showing sound psychometricproperties.Published: 8 April 2008BMC Medical Education 2008, 8:17 doi:10.1186/1472-6920-8-17Received: 19 September 2007Accepted: 8 April 2008This article is available from: http://www.biomedcentral.com/1472-6920/8/17© 2008 Kassam et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Page 1 of 13(page number not for citation purposes)BMC Medical Education 2008, 8:17 http://www.biomedcentral.com/1472-6920/8/17BackgroundPreventable drug-related problems due to inappropriateprescribing, under-prescribing, and inappropriate medi-cation use contribute significantly to both the economicand human costs of health care [1-4]. Several studies havedemonstrated that pharmacists have a vital role to play indrug therapy management, thus enhancing overall patienthealth outcomes [5-10]. In response to economic andsocietal needs, the pharmacy profession throughoutNorth America has adopted pharmaceutical care as its newpractice mandate. Pharmaceutical care is defined as a phi-losophy of practice wherein the "pharmacist cooperateswith patients and other professionals in designing, imple-menting and monitoring therapeutic plans that will pro-duce specific therapeutic outcomes" [1]. Pharmaceuticalcare takes the profession beyond simple drug distributingand counseling responsibilities to a broader mandate ofpatient-centered care to maximize the positive outcomesof patients' drug therapies. Consequently, the CanadianCouncil for Accreditation of Pharmacy Programs hascharged pharmacy schools across Canada to ensure thatstudents are adequately skilled in the principles and prac-tices of pharmaceutical care [11].In 1999, the structured practice education program(SPEP) at the Faculty of Pharmaceutical Sciences (the Fac-ulty), University of British Columbia (UBC) – Vancouver,Canada, used an iterative process involving faculty mem-bers and UBC pharmacy preceptors to introduce a newcommunity-based advanced pharmacy practice experi-ence (APPE) to meet accreditation standards. The newAPPE detailed 13 content domains that emphasized thespecific knowledge, skills and attitudes required to prac-tice pharmaceutical care; these domains can be summa-rized under four general themes: (1) developing caredelivery strategies that promote discussing, acquiring andassessing relevant patient, drug and disease; (2) develop-ing care plans in collaboration with patients and otherhealth professionals to prevent or resolve drug-relatedproblems; (3) assuming responsibility for managing drug-related problems by monitoring patients' progress andensuring follow-up care; and (4) promoting healththrough disease prevention and wellness clinics and sem-inars. Table 1 summarizes the range of specific patientcare activities students are required to engage in for thisAPPE. The development process of this new program hasbeen discussed in detail in a previous publication [12].As part of the SPEP office's continuous quality improve-ment initiative, a detailed evaluation of the newly imple-mented community-based APPE was conducted in 2000[13,14]. The findings from the evaluation suggested that –while students had ample opportunity to engage in learn-ing activities that the pharmacy staff regularly encoun-Table 1: Community-based advanced pharmacy practice experience activitiesPC Competency Domains Activity Description1. Asking about patient expectations 1. Assess patients with new prescriptions and develop care plans to resolve/prevent drug-related problems2. Collecting relevant information 2. Assess patients with refill prescriptions and develop care plans to resolve/prevent drug-related problems3. Integrating patient information 3. Present and discuss 1 prescription AND 1 non-prescription drug class with preceptor4. Evaluating different treatment options 4. Provide pharmaceutical care patients requesting non-prescription products, develop care plan for all interventions5. Documenting patient info: continuity of care 5. Provide follow-up to patients encountered in activities # 1, 2, 4 and 9, document follow-up care6. Prioritizing drug related problems 6. Provide drug information to patients, preceptors and other health care providers, document all recommendations7. Determining patient experiences: effectiveness or undesirable effects of current medications7. Shadow another health care professional for 1/2 to 1 day, complete the reflection page8. Determining whether patients were managing and adhering to their medication regimes8. Discuss pharmacy practice issues related to pharmaceutical care (barriers and opportunities9. Establishing monitoring parameters with patients9. Provide comprehensive pharmaceutical care by assessing all drug-related needs of your patient, identify drug-related problems and develop care plans to resolve/prevent drug-related problems10. Following-up patients by phone or in-person10. Initiate and complete a patient care project, submit a summary of your project11. Developing professional relationships: with other health care providers, physicians12. Participating in clinics, seminars, projects or presentations13. Providing basic and comprehensive Page 2 of 13(page number not for citation purposes)pharmaceutical careBMC Medical Education 2008, 8:17 http://www.biomedcentral.com/1472-6920/8/17tered in their practices, such as assessment of new andrefill prescriptions and over-the-counter requests, theyhad significantly fewer opportunities to engage in activi-ties related to comprehensive pharmaceutical care thatincluded developing and maintaining relationships withpatients, assuming responsibility for the management ofdrug-related issues, and evaluating the patients' drug ther-apy through follow-up care. Student learning was furtherhampered by limited opportunities to engage in a varietyof non-direct pharmaceutical care activities, such as inter-professional collaborations and health promotion/dis-ease prevention clinics, seminars and presentations.Interestingly, results from a recent survey found that alarge percentage of students from the United States alsoexperienced insufficient opportunities to practice theactivities, tasks and processes essential to pharmaceuticalcare; suggesting this to be a widespread concern [15].To enhance UBC students' learning experience duringtheir community-based APPE, the SPEP faculty undertooka project to explore structures and processes that couldhelp promote better congruence with the principles ofpharmaceutical care and the patient care delivery processthe students engaged in. The structures and processes wereintroduced in a stepwise fashion, supported by evidencegathered at different stages of the project. This articlereports on a pilot study focusing on three goals: (1) as"proof of concept," to test the overall educational impactof an enhanced APPE model on student competencies; (2)to develop an instrument to measure how both studentsand preceptors experienced the enhanced community-based model; and (3) to assess the reliability, validity anddiscriminating power of the instrument to detect differ-ences between the traditional and the enhanced APPEmodels. The primary hypothesis was that the competencyscores for the enhanced model (treatment arm) wouldimprove more than would scores for the traditionalmodel (control arms), and that such improvementswould be evident for both students and their preceptors.MethodsDesignA retrospective comparative-experimentalist design wasused to evaluate the impact of the enhanced community-based APPE model on students' learning and performance[16]. The overall logic of the study was grounded in a 5-stage learning model: (1) An enhanced learning climate(trained preceptors, explicit SPEP guidelines, studenttraining and support) should lead to (2) better utilizationof APPE learning opportunities, including (3) more fre-quent student/patient consultation, then to (4) improvedskills acquisition, thence to (5) more favorable attitudestoward pharmaceutical care practice. The study was con-placement sites for APPE. Ethics approval had beenobtained through UBC's Office of Research Services andconsent was obtained from all students and preceptors toallow dissemination and publication of the study's find-ings.ParticipantsThis study was grounded in a unique partnership devel-oped between the University's Pharmacy school and anationally operating pharmacy chain. As part of this part-nership, the national chain had agreed to two importantaspects: (1) to provide release time for their preceptors toattend a full-day's preceptor training workshop at com-pany expense; and (2) to support the costs associated withthe delivery of the training program and the evaluation ofthe APPE post-training. A student researcher was hired tomanage data collection and collation, and an instrumentdeveloper/statistician was hired to assist with surveyinstrument development, data analysis and interpreta-tion.The aim of the study was to recruit seven communitypharmacies into the study's treatment arm (enhancedrotation) and a similar number into each of the two con-trol arms (traditional rotation). To extend the generaliza-bility of the study results, one of the control armscomprised pharmacies from the same national partneringchain while the second control arm consisted of an assort-ment of community pharmacies from other chains andindependent pharmacies. Purposeful selection was usedto recruit the pharmacies, ensuring representation fromboth urban and rural settings; all sites had a long historyof serving as placement sites for UBC. The communitypharmacies in the two control arms were matched to theenhanced pharmacies in terms of geographic location andcommunity size, and all had previously served as place-ment sites with UBC. All recruited pharmacies were askedto designate one pharmacist to serve as the primary pre-ceptor, whose responsibility would be to set the clerkshipexpectations with the student, facilitate learning opportu-nities, provide on-going feedback and conduct the mid-point and final evaluation of the student. In addition,other pharmacists were also encouraged to participate inthe student's learning if they were interested in doing so;these pharmacists served as secondary preceptors andworked within the framework established by the primarypreceptor. Student participation in the enhanced arm wasvoluntary because its structure was new to the curriculum,would require additional student commitment beyondthe traditional rotation, and its impact had not yet beenassessed. The enhanced APPE was advertised to all fourth-year (senior year) students through e-mail and class pres-entations, and interested students were recruited on aPage 3 of 13(page number not for citation purposes)ducted between September 2001 and May 2002 in com-munity pharmacies with continuing histories asfirst-come basis. Each placement site (enhanced and con-BMC Medical Education 2008, 8:17 http://www.biomedcentral.com/1472-6920/8/17trol) accepted a maximum of two students at differenttimes within the academic year.InterventionDuring their 8-week community-based APPE, students inboth the enhanced and control sites were expected tomeet similar learning objectives, participate in the samelearning activities, and were held to the same learning andperformance criteria and evaluation standards.Students and preceptors in both control arms received theSPEP manual outlining the APPE's learning activities,expectations, policies, evaluation processes and forms,patient care tools and conduct standards. The APPE dura-tion for the control arm students comprised of two 4-weekcycles at two different pharmacy locations. The controlstudents participated in a mandatory 3-hour face-to-faceorientation session and an on-line quiz reinforcing theAPPE expectations, but no additional intervention wasprovided to the control preceptors other than the routineverbal and written communications that occurs betweenthe SPEP office and the preceptors.Primary preceptors and students in the enhanced armwere provided with a few additional interventions beyondthose offered to the control arm. The intervention con-sisted of separately structured experiences for both stu-dents and preceptors: (1) a one-day preceptor educationworkshop to discuss the proposed pharmaceutical carepractice model, to review the course syllabus, to clarify thelearning tasks expected of students, and to work throughan ice-breaker exercise to be conducted with their stu-dents; (2) a five-day student orientation at the pharmacysite prior to the start of the practice education experienceto allow pharmacists to assess their students' baselinecompetencies with various distribution activities andselected pharmaceutical care activities outlined in Table 1(specifically activities: # 1, 2, 4, 6 and 9); and (3) extend-ing the experience from the traditional two 4-week rota-tions completed at two different pharmacies to one 8-week experience completed at the same enhanced armpharmacy.Instrument DevelopmentA 70-item, 5-point Likert-type survey was developedaround the pharmaceutical care activities and competen-cies students were expected to master. The survey layoutrecapitulated the 5-stage learning model: [1] learning cli-mate and preceptor support (9 questions), [2] learningopportunities (13 questions), [3] patient consultationestimates (one question), [4] skills improvement (17questions), and [5] attitude enhancement (29 questions).In approximate terms, the survey was structured aroundeach of the competency domains was assessed three times;once as a learning opportunity issue, secondly as a skillimprovement topic, and third as an attitude-related mat-ter. A final question for preceptors alone asked whetherthe experience of precepting students had expanded theirown grasp of pharmaceutical care principles and practice.Instrument development activities focused on verifyingthat questionnaire items putatively assigned to preceptorsupports scales, learning opportunities, skills improve-ment, and attitudes all conformed to generally acceptedscale development criteria, in that they: (1) distinguishedeffectively between enhanced and control settings; (2)demonstrated alpha reliabilities equal to or exceeding therule-of-thumb threshold of α = 0.70; (3) converged relia-bly on their respective specific themes; (4) did not "spillacross" to other themes; and (5) gathered informationunbiased with respect to source (preceptor or student).Scales and items passing all these tests would be retainedfor further refinement in post-pilot refinements andfuture iterations of the enhanced APPE model [17,18].Data collectionBoth the students and their primary preceptors completedretrospective surveys shortly following the completion ofthe APPE. As well, both groups were assured that surveyresults would not affect the students' final grades. The sur-vey instructions requested the preceptors in the enhancedgroup and students from both groups to reflect back to thebaseline day of their 8-week APPE. The control preceptorsassessed the students over the second 4-week experience.While both the students and preceptors were asked aboutvirtually identical topics, the question phrasing was keyedto the appropriate audience, for example: student itemswere phrased in terms of "There were opportunities to ...","I believe that..." whereas preceptor items were phrased interms of "The rotation provided the student with...", or "Imet with the student to...". Each survey packet consistedof seven generic Scantron sheets (#70921) overprintedwith instructions, question stems and 5-point responsecategories; 'Never' to 'Always', 'No Improvement' to 'Sig-nificant Improvement' or 'Strongly Disagree' to 'StronglyAgree'. Surveys were distributed to students and precep-tors by mail, and returned either by mail or in-person.AnalysisScantron survey results were verified, summarized andanalyzed using SPSS. Results of the study's outcomes pro-gressed though a series of increasingly stringent analyses;overall summaries (SPSS Frequencies and Descriptives),differences between enhanced and control arms (t-tests,ANOVA), differences between preceptor and studentassessments (t-tests, ANOVA) and possible interactionsPage 4 of 13(page number not for citation purposes)the 13 pharmaceutical care competency domains empha-sized in the community-based APPE (Table 1). In general,(ANOVA), reliability and validity estimates of compe-tency domains and study rationale measures (Reliability),BMC Medical Education 2008, 8:17 http://www.biomedcentral.com/1472-6920/8/17and tests of the various scales' abilities to discriminatebetween enhanced and control conditions (Discrimi-nant). For all tests, the a priori level of significance was setat p ≤ 0.05.Since the overall objective of the pilot study was to test theimpact on student learning of the enhanced APPE, wereport those overall results first, followed by moredetailed analyses of the learning model components(learning climate and preceptor support → learningopportunities + patient consultations → skills improve-ment → attitude enhancement). Next we report differ-ences across the 13 domains of pharmaceutical care andthe differential impact of learning in the enhanced andcontrol settings. Finally, we report the psychometric prop-erties of the scales we developed to examine these variousaspects of program enhancement and student perform-ance with a focus on scale refining and streamlining. Weconclude with a number of 'lessons learned' about furtherimprovements to the APPE, easier data collection proce-dures and expanded scope of future enhanced APPE.ResultsAt the end of the APPE, there were 13 students who com-pleted the enhanced APPE (treatment arm) and 28 stu-dents in the two control APPEs (14 in each arm), onestudent from the enhanced arm dropped out prior to thestart of the APPE. All sites had one designated primarypreceptor, two sites in each of the three arms had a sec-ondary preceptor, one site in the treatment arm had twosecondary preceptors. A total of 74 surveys were returnedat the end of the study: 38 from preceptors and 36 fromstudents. From the preceptor group, 13 (100%) surveyswere returned from the enhanced arm and 11 (78.6%)from the control arm representing the national partneringchain and 14 (100%) from the control arm representingan assortment of pharmacies. From the student group, 13(100%) surveys were returned from the enhanced arm,nine (64.3%) from the national chain control arm and 14(100%) from the control arm representing an assortmentof pharmacies. One student from the control arm left sub-stantial portions of the survey unanswered and was elim-inated from further analysis, for a total of 35 useablestudent surveys.Overall ResultsTable 2 demonstrates that, overall, student and preceptoroutcomes favored the enhanced APPE structure over thestructure in the control arms. From the students' own per-spectives and for all 70 measures combined, studentscores in the enhanced APPE outperformed those in thecontrol arm by about 0.37 points out of a possible five:(4.16 vs. 3.79; t = 2.33, p < 0.026). Parallel information70 measures combined, preceptors' assessments showed asignificant 0.39 point advantage for the enhanced APPE,(4.07 vs. 3.68; t = 2.67, p < 0.011) slightly exceeding stu-dents' own reports. From students' and preceptors' com-bined perspectives, the enhanced APPE showed similarstatistical advantages; the enhanced APPE outperformedthe experience in the control arm by 0.38 points (4.11 vs.3.73; t = 3.59, df = 71, p < 0.001) and showed an overall18% gain in performance (η2 = 0.179).The 5-Stage Learning ModelTable 2 also summarizes results for each of the five stagesof the learning model: learning climate and supports,learning opportunities, patient consultations, skillsimprovements, and pharmaceutical care-related attitudes.The results always favored the enhanced APPE model,with two-thirds of the comparisons being statistically sig-nificant. Since ONEWAY Tukey range tests had shownthere were no statistical differences between the two con-trol groups for either students or preceptors, the two con-trol arms were collapsed into a single control group forfurther analyses.Students reported statistically significant benefits of theenhanced APPE in terms of the number of comprehensiveconsultations it afforded (16.65 vs. 5.30; t = 7.29, p <0.000), skills improvements noted (4.21 vs. 3.50; t = 3.05,p < 0.004) and attitudes favoring pharmaceutical careprinciples (4.39 vs. 4.03; t = 2.41, p < 0.021). Preceptorsin the enhanced model reported noted benefits of theenhanced APPE, the estimated numbers of comprehen-sive consultations conducted by students (17.50 vs. 8.74;t = 5.65, p < 0.000), and skills improvement observed(4.12 vs. 3.22; t = 3.81, p < 0.004). Although more precep-tors in the enhanced model agreed that their SPEP partic-ipation "provided [more] opportunity to practice andimprove my pharmaceutical care skills" than did precep-tors in the control arms, this was not statistically signifi-cant.It is also noteworthy that a two-way ANOVA showed gen-erally no differences between students' self-reports andthose of their preceptors, except for the finding that pre-ceptors in both the control and enhanced arms signifi-cantly (F = 5.65, p < 0.020) over-estimated the number ofpatient consultations (11.7 vs. 9.5) provided by their stu-dents (and verified by the SPEP faculty by reviewing doc-umentation submitted by the students in their portfolio).Interestingly, the overestimates were greater in the controlsettings than in the enhanced APPE, perhaps suggestingthat preceptors' oversight of student experience is lessdetailed in controlled settings. As well, there were no treat-ment-by-group interactions indicating that both studentsPage 5 of 13(page number not for citation purposes)from preceptors about their students' overall performancegenerally corroborated students' own assessments. For alland preceptors interpreted and responded to the ques-tions in essentially the same ways.BMC Medical Education 2008, 8:17 http://www.biomedcentral.com/1472-6920/8/17Pharmaceutical Care Competency DomainsTable 3 shows that some competencies were much betterlearned than others – irrespective of study arm. For ease ofinterpretation, the central columns of the table showscompetency means converted to percentages (mean/5) foroverall results as well as for enhanced/control differences,thus showing the fraction of what was reported to havebeen learned as differential percentages of the maximumthat could be reported. Examining all groups combined(enhanced, control, students, and preceptors), the tablefrom lows of about 40% to highs of near 90%. For exam-ple, topics such as collecting relevant patient information(4.38 out of 5) or determining whether patients were manag-ing their medication regimes (4.17) were readily graspedwhile participating in clinics, seminars, projects or presenta-tions (1.99) or developing professional relationships with otherhealth-care providers (3.47) were much less well inculcated.Finally, the bottom row of Table 3 summarizes the overallstudent grasp of these 13 competencies and suggests thatabout 77% of what could have been reported wasTable 2: Student and preceptor results for the 5-component learning modelPerspective Group Mean t-test SignificanceLearning Climate and Preceptor Supports (number (n) of items = 9; score range: 2.00 – 5.00)Students' Control APPE 3.87 1.21 nsEnhanced APPE 4.21Preceptors' Control APPE 3.73 1.89 nsEnhanced APPE 4.07Learning Opportunities (n items = 13; score range: 2.46 – 4.62)Students' Control APPE 3.76 0.34 nsEnhanced APPE 3.82Preceptors' Control APPE 3.50 1.56 nsEnhanced APPE 3.79Estimated Number of Comprehensive Consultations (n items = 1; score range: 1.00 – 17.00)Students' Control t APPE 5.30 7.29 p < 0.000Enhanced APPE 16.65Preceptors' Control APPE 8.74 5.65 p < 0.000Enhanced APPE 17.50Skills Improvement (n items = 17; score range: 1.94 – 5.00)Students' Control APPE 3.50 3.05 p < 0.004Enhanced APPE 4.21Preceptors' Control APPE 3.22 3.81 p < 0.001Enhanced APPE 4.12Attitudes (n items = 29; score range: 2.90 – 5.00)Students' Control APPE 4.03 2.41 p < 0.021Enhanced APPE 4.39Preceptors' Control APPE 4.16 0.86 nsEnhanced APPE 4.31Overall Scale (n items = 70; score range: 2.65 – 4.89)Students' Control APPE 3.78 2.33 p < 0.026Enhanced APPE 4.16Preceptors' Control APPE 3.68 2.67 p < 0.011Enhanced APPE 4.07Page 6 of 13(page number not for citation purposes)shows that content domains differ considerably in theirrespective levels of learning achievement (M%); rangingreported. Moreover, it shows that the enhanced rotationsenjoyed about a 7% advantage over the traditional modelBMC Medical Education 2008, 8:17 http://www.biomedcentral.com/1472-6920/8/17(F = 11.40, p < 0.001) in bringing students nearer to read-iness for entry-to-practice.When comparing the students' experiences between theenhanced and the control arms for all 13 of the compe-tency domains, the outcomes consistently favoured theenhanced model; with six of the 13 competencies beingsignificantly improved. These included competencies:"Integrating patient information," "Documenting patientinformation," "Prioritizing drug-related problems,""Establishing monitoring parameters with patients," "Patient follow-up by phone or in-person," and "Providingbasic and comprehensive pharmaceutical care." Addition-ally, the overall index of learning in all 13 pharmaceuticalcare competency domains was significantly better for stu-dents in the enhanced arm (4.13 vs. 3.81; t = 2.15, p <0.038) than the control arm – an improvement of 6.25%.The preceptor pattern was slightly different; they reportedsignificant improvement in seven of the 13 competencies:"Asking about patient expectations," "Collecting relevantinformation," "Integrating patient information," "Docu-menting patient information," "Establishing monitoringparameters," "Patient follow-up by phone or in-person,""Providing basic and comprehensive pharmaceuticalcare," as well as in the overall index of competencydomains (4.04 vs. 3.66; t = 2.61, p < 0.013).Table 3: Competency domains for learning pharmaceutical care principles and delivery:Overall, enhanced (treatment) and traditional (control) scores for students and preceptors combined.PC Competency Domains Overall Mean Overall Mean % Enhanced Mean % Control Mean % SignificanceaAsking about patient expectations 3.84 76.9 82.1 74.0 0.005Collecting relevant information 4.38 87.6 91.3 85.5 0.036Integrating patient information 4.08 81.6 89.0 77.6 0.000Evaluating different treatment options 4.18 82.5 87.4 79.7 0.010Documenting patient info: continuity of care 3.91 78.3 85.4 74.3 0.001Prioritizing drug related problems 3.97 79.4 84.6 76.5 0.002Determining patient experiences: effectiveness or undesirable effects of current medications4.23 84.6 88.5 82.4 0.023Determining whether patients were managing and adhering to their medication regimes4.17 83.4 87.2 81.3 0.022Establishing monitoring parameters with patients3.95 78.9 86.2 74.9 0.000Following-up patients by phone or in-person 3.90 77.9 85.8 73.6 0.000Developing professional relationships: with other health care providers, physicians3.47 69.5 69.2 69.6 nsParticipating in clinics, seminars, projects or presentations1.99 39.9 37.6 41.1 nsProviding basic and comprehensive pharmaceutical care4.14 82.7 88.0 79.8 0.001Overall Index of Content Domain Learning 3.86 77.2 81.7 74.6 0.001a Indicates significance of the differences observed between the enhanced (treatment) versus traditional (control)Table 4: Alpha-reliability coefficients, convergent and discriminant estimates for original study scales.Learning Model: Original Scales Mean* Standard Deviation Number of Items αa rb Scale Biasc Scale SensitivitydLearning Climate & Preceptor Support3.92 0.69 9 0.884 0.549 0.110 0.305Learning Opportunities 3.69 0.55 17 0.884 0.574 0.162 0.180Patient Consultations (estimated number)10.67 6.66 1 0.475e 0.475 -0.0168 0.520Skills Improvement 3.68 0.73 17 0.938 0.476 0.118 0.510Attitude Enhancement 4.19 0.46 29 0.939 0.479 -0.052 0.261Overall Study Model Scale (original) 3.87 0.47 70 0.959 0.816 0.121 0.410αa – Cronbach's alpha-reliability, and the index of convergent validity; how well do items converge on each scale's central concept.rb – Highest correlation with any other scale. Low correlations mean the scale does not duplicate information in other scales.c – Pearson r with preceptor vs. student. Non-significant r's indicate absence of bias.Page 7 of 13(page number not for citation purposes)d – Pearson r (or eta) with study arm. Positive correlations indicate sensitivity to predicting enhanced clerkship.e – Since the patient consultation estimate was a single variable measure, its α-reliability estimate is its SMR with the other four indices.BMC Medical Education 2008, 8:17 http://www.biomedcentral.com/1472-6920/8/17Instrument ValidationTables 4 and 5 summarize the broad array of analyses con-ducted to validate and test the survey instrument. Theaccepted criteria of face validity, reliability, discriminantvalidity, construct validity, absence of bias, and sensitivitywere all used as benchmarks to determine whether thesurvey's items, scales, and psychometric propertiesachieved acceptable levels.Face validityFace validity tests whether the individual items in anyinstrument "make sense" and whether knowledgeableexperts concur that they are "important things to beasked". The instrument's 70 items were reviewed and ver-ified by several cycles of item generation, pre-testing andrefinement by this report's first two authors. The guidingquestion was always "Is this what we want to know aboutstudent response to the two versions of the APPE, and isthis the way to ask it?" Subsequently, five faculty and fiverandomly selected non-study pharmacy students reviewedthe survey items and provided helpful comments on read-ability and clarity.ReliabilityReliability analysis examines whether the various itemsthought to constitute a scale are well-aligned, consistentamong themselves, and generally corroborate each other.Contrary items can be refined, rephrased, or eliminated,although overall reliability measures depend on thenumber of items in the scale. The most common index ofscale reliability is Cronbach's alpha (α) and scale develop-ers strive for reliability measures of 0.70 or greater. Table4's fourth column (αa) shows that four of the five learningmodel scales (preceptor support (0.88), learning opportu-nities (0.88), skills improvement (0.94) and attitudeenhancement (0.94) far exceed the guideline, indicatingthat the individual items within each scale are good oper-validity of the scales' conceptual foundations. Sincepatient consultations is a single measure count, the con-cept of scale reliability doesn't apply in the same way,hence the table entry is its squared multiple correlation(0.48) with the four remaining scales. In short, the variousscales comprising the 5-step learning model are well oper-ationalized by the items constituting them.For the 13 pharmaceutical care competency domains,their reliabilities are acceptable but not quite so strongbecause the number of items in most scales is often onlythree or four. For example: "following-up ...(0.76),""developing professional relationships ...(0.77)," "partic-ipating in clinics ...(0.78)," and "providing basic andcomprehensive pharmaceutical care ...(0.84)" are welloperationalized. In contrast, " asking about patient expec-tations ...(0.27)," "prioritizing about drug-related prob-lems ...(0.33)" and "determining if patients aremanaging/adhering...(0.46)" are less well-anchored. Nev-ertheless, the reliability of the overall index calculatedacross all 13 domains is 0.92, strong by any measure. Thusthe scales derived from the 70-item survey generallyexhibit reliabilities that are "acceptable" to "strong."Discriminant validityDiscriminant validity analysis tests whether the items inany one scale are all more closely correlated among them-selves than with any different scale, in short, do any of theitems "spill over" into the domain of a different scale?Table 4, column r(b) show that the highest correlationsbetween any one scale in the learning model and anyother scale are always lower than their respective alphareliabilities indicating that each scale converges on its cen-tral concept and discriminates it from the other scales[18].Construct ValidityTable 5: Psychometric properties of revised learning model scales.Learning Model: Refined Scales Mean Standard Deviation Number of Items αa rb Scale Biasc Scale SensitivitydPreceptor Support 3.94 0.69 6 0.847 0.510 0.104 0.336Learning Opportunities 4.00 0.62 6 0.837 0.510 0.214 0.304Patient Consultations (estimated number) 10.67 6.66 1 0.475 0.580 -0.182 0.715Skills Improvement 3.65 0.86 6 0.873 0.520 0.093 0.622Attitude Enhancement 4.03 0.65 6 0.837 0.520 -0.157 0.393Overall Study Model Scale (refinedf) 3.89 0.56 25 0.920 0.796 0.057 0.574Overall Study Model Scale (original)f 3.87 0.47 70 0.959 0.816 0.121 0.410αa – Cronbach's alpha-reliability, and the index of convergent validity; how well do items converge on each scale's central concept.rb – Highest correlation with any other scale. Low correlations mean the scale does not duplicate information in other scales.c – Pearson r with preceptor vs. student. Non-significant r's indicate absence of bias.d – Pearson r (or eta) with study arm. Positive correlations indicate sensitivity to predicting enhanced clerkship.e – Since the patient consultation estimate was a single variable measure, its α-reliability estimate is its SMR with the other four indices.f – Both Refined and Original scales are listed to document that streamlining resulted in no appreciable loss of scale power and efficiency.Page 8 of 13(page number not for citation purposes)ational measures of the scale's conceptual content whilesimultaneously lending strong support to the constructConstruct validity is a 'concept in reverse' [19]. Ratherthan querying how well each scale (or construct) is oper-BMC Medical Education 2008, 8:17 http://www.biomedcentral.com/1472-6920/8/17ationalized, it tests the proposition that "given the opera-tional evidence, how much credibility can be (backwards)ascribed to the construct (or scale). If the operational evi-dence is strong, then there are more solid grounds toaccept the notion that "there is such as thing as learningclimate, or learning opportunity, or skills improvement orattitude enhancement". In this study, the learning climatetotal (of 9 items), learning opportunities total (of 17items), patient consultation estimate, skills improve-ments total (of 17 items), attitude enhancement total (of29 items), and the combined study model total (of 70items) all have strong construct validities as evidenced bytheir high corrected item-total correlations (not shown)and their high α-reliabilities (0.85 and higher). Collec-tively, these reliability measures and item-total correla-tions confirm that such concepts as learning climate,learning opportunities, skills improvement and attitudeenhancement do exist and can be validly inferred from theresponse patterns of pharmacy students and their precep-tors.Scale biasTests for scale bias seek to verify that both individualitems and the scales constructed from them are unbiasedin terms of the respondent who happens to be completingthe survey. In this study, it meant testing that both stu-dents and preceptors understood and interpreted theitems in essentially the same way. Study results would beweakened if it could be shown that any of the learningmodel scales favoured preceptors over students or vice-versa. The 'scale bias' column in Table 5 shows that corre-lations between scale totals and the persons completingthe surveys (students vs. preceptors) are small and non-significant, hence threats of study outcomes being spuri-ous results of student/preceptor differences can be ruledout. Further evidence that preceptors and students saw theitems and scales similarly was obtained by correlating stu-dent/preceptor pairs' responses across the various practicelocations. In general, preceptor/student pairs concurabout the learning aspects of the APPE: the correlationbetween the pairs for combined feature of the overalllearning model was 0.472. The overall correlations acrossall components of the model are fractionally higher forthe enhanced pairs (0.495) than for the controls pairs(0.460), thus confirming that there was general, if loose,agreement between preceptors and students about whenLearning Opportunities were (or were not) available,when clinical instructors had (or had not) facilitated insuch as way as to enable students to meet their overallSPEP objectives, when Skills Improvements had (or hadnot) occurred, and when students' Attitudes and Attribu-tions of Importance favoured (or did not favour) pharma-ceutical care focused practice.Scale SensitivityTo be helpful in testing for differences between enhancedand control APPE, both survey items and the scalesderived from them must be sensitive to any differencesthat do exist. The rightmost column in Table 4 shows thatthe estimate of numbers of patient consultations per-formed was the most sensitive discriminator between peo-ple in enhanced APPE vs. those in control settings. Skillsimprovement, learning climate, attitude enhancementand learning opportunities showed decreasing sensitivityto enhanced/control differences. Some 19 of the 70 weresignificant and sensitive indicators of enhanced vs. con-trol differences: 11 from Skills Improvement, five fromAttitude Enhancement, two from Learning Climate andNumber of patient consultations. Disregarding for themoment the scales into which the 70 items were aggre-gated, in descending order of sensitivity, the 19 best dis-criminating items were: number of patient consultations,patient follow-up, integrating information from patientswith multiple problems, providing comprehensive phar-maceutical care to patients with few problems, integratingpatient information to identify drug-related problems,prioritizing drug-related problems, determining patientexperiences regarding drug effectiveness and undesirableeffects, evaluating different treatment options usingguidelines, disaffirming that follow-up is too time-con-suming, discussing monitoring parameters with patients,asking patients about their drug therapy expectations,determining whether patients were managing their medi-cations, carrying out independent tasks without unduereliance on the pharmacist, preceptors encouraging stu-dents to provide patient follow-up, believing that provid-ing pharmaceutical care gave students a betterunderstanding of patients drug-related needs, acknowl-edging that the SPEP rotation was a key factor in stimulat-ing pharmaceutical care, feeling comfortable with theprocess of providing pharmaceutical care, agreeing thatdiscussing monitoring parameters with patients is impor-tant, and disagreeing that providing patient follow-up isbeyond a pharmacist's responsibility.Learning Model Scale Inter-CorrelationCorrelations among the scales for learning climate, learn-ing opportunities, skills improvement, attitude enhance-ment, and estimated numbers of patient consultationsaveraged 0.43, which together with the high intra-scalealpha reliability suggests that each of these five indicestaps a different repertory of student performances that areinter-related but uniquely different from each other. Inter-estingly, it made a less-than-expected-difference whetherthese intercorrelations were tested for the enhanced (r =0.44) or the control respondents (r = 0.33). Similarly,there was only slightly greater coherence among precep-Page 9 of 13(page number not for citation purposes)tors' assessments (r = 0.53) than among students' (r =0.37). The general conclusion is the same in all instancesBMC Medical Education 2008, 8:17 http://www.biomedcentral.com/1472-6920/8/17– the five measures overlap to yield a reasonably general-ized index of APPE training effectiveness which is signifi-cantly higher for the enhanced cohort than theirclassmates in either control setting.Instrument RefinementRecognizing that completing a 70-item instrument wastime-consuming and tedious for both students and pre-ceptors, a subsequent phase of analysis tested whether amore streamlined instrument could be devised, and onewhich maximized the scales' differential abilities to dis-criminate between enhanced APPE and their traditionalcounterparts. Table 5 reports the item parameters forrefined learning model scales when only the best-discrim-inating six items in each scale were included. Four, fiveand seven item versions were also tested but scale reliabil-ities, interscale differentiation, and scale sensitivity wereall optimized with six items per scale. Selecting only thosespecific items from each learning model scale which mostclearly distinguished between enhanced and control armsresulted in improved precision, such that all five refinedscales discriminated significantly between the study'streatment arms whereas only three of the study's originalscales were significant discriminators. The most commonthemes among items in the streamlined scales included:"patient follow-up," "documenting patient information,""providing pharmaceutical care-based care," "prioritizingdrug-related problems," and "determining whetherpatients were managing their medications or had newquestions or concerns," all of which reflect generallyhigher-level skills of pharmaceutical care practice ratherthan the more elementary learning tasks such as askingabout patient expectations, collecting relevant informa-tion, etc. Streamlining the scales from 70 down to 25items demonstrated the multiple advantages of brevity,greater precision, greater sensitivity to treatment improve-ments, all while maintaining absence of bias.DiscussionIn general, the present study's results demonstrated thatboth students' and preceptors' outcomes favored theenhanced APPE in achieving the overall goals, although inslightly different fashion. Considering the combined stu-dent and preceptor outcomes on the full 70-item survey,the study's results suggested that the enhanced APPEresulted in about three times as many patient consulta-tions as did the control experience, together with greaterskills improvement, more favourable pharmaceutical carefocused attitudes and an improved overall programimpact of about 18%. Slight differences resulted when stu-dent and preceptor data was analyzed separately. Forexample, only the preceptor results suggested that stu-dents in the enhanced APPE were provided with a supe-enhanced APPE students experienced significantly greaterattitude enhancement compare to their counterparts.Interestingly, the survey did not detect any difference inlearning opportunity from the students' or preceptors'perspectives. When considering the 13 pharmaceuticalcare competency domains around which the APPE wasstructured from combined student and preceptor perspec-tives, all competencies favored the enhanced experience;and significantly so for all but two of the competencies(developing professional relationships with other healthproviders, physicians and participating in clinics, semi-nars, projects or presentations).Past research has shown that curricular innovation ismuch more likely to be successful when instructors areprovided with adequate opportunities to become familiarwith the innovations and their rationale [20,21]. Thisapplies equally well to preceptors asked to supportchanges to clinical placements such as those implementedin the enhanced APPE. Thus, while student accountabili-ties are spelled out in considerable detail in the SPEPmanual given to students and preceptors in both controland enhanced APPE, preceptors in the enhanced APPEreceived additional education about how to foregroundthese expectations. This was accomplished by presentingthe preceptors with suggestions on how to effectively setstudent expectations and facilitate student engagement inthe available learning opportunities, as well as a discus-sion on each of the activities in which the students wereasked to partake. Similarly, students were provided with a5-day training that consisted of the Faculty discussing theAPPE activities and expectations in detail with the stu-dents, giving the students the opportunity to spend timeat their respective APPE sites prior to the start of theirexperience to familiarize themselves with their commu-nity pharmacy's processes and structures, initiate relation-ship and trust building with their preceptor and pharmacystaff, and allow their preceptor to assess the students' areasof strengths and weaknesses. Thus, the study instrumenta-tion was built around procedures and accountability sys-tems available equally to students and preceptors in bothcontrol and enhanced APPE. The non-difference in learn-ing opportunity outcomes, but the significant differencein skills and attitudes indicates that the mere presence ofopportunities is insufficient, but rather being expected toput them into practice with preceptors' guidance makesthe enhanced model more successful than the traditionalmodel as confirmed by the rich literature on situatedlearning in "communities of practice" [22]. In short, "welearn what we pay attention to," but when that attentionis guided by explicit accountabilities, regular monitoring,and preceptor follow-up, the learning is more meaningfuland better integrated [21-24].Page 10 of 13(page number not for citation purposes)rior learning climate compared to students in the controlAPPE; and only the student results suggested that theBMC Medical Education 2008, 8:17 http://www.biomedcentral.com/1472-6920/8/17Pharmaceutical Care Competency DomainsIn any learning venue, classroom or experiential, there isalways the likelihood that some domains are apt to be bet-ter mastered than others; and there is instructionally diag-nostic value in knowing which domains are successfullylearned and which less so, and whether membership inthe enhanced APPE cohort predicts learning gains in somedomains more than others. Irrespective of the study arm,combined results from both students and preceptorsshowed that some of the pharmaceutical care competen-cies were better learned than others. For example, collect-ing relevant patient information, questioning patients todetermine their experiences with drug effectiveness or undesir-able effects, determining whether they were adhering to theirmedication regimens were all reasonably well achieved(mid 80%). In contrast, participating in clinics, projects,presentations or seminars was woefully underachieved(40%) irrespective of study arm. Between those twoextremes, evidence shows clearly greater content masteryby students in the enhanced APPE; notably in patient fol-low-up, integrating patient information, documenting for con-tinuity of care, and establishing monitoring parameters. Fortwo domains (developing professional relationships with phy-sicians and other health care providers and participating inclinics, projects, presentations or seminars), the controlcohort fractionally outperformed the enhanced cohortbut not significantly. In the present study, the finding thatsome competencies were better learned than others allowscurriculum planners to focus on areas where more learn-ing opportunities are required.Instrument RefinementFrom the outset, it was evident that a 70-item survey wastoo long and oversampled the relevant content domains.Issues of frustration, inattention, and non-compliance areroutine consequences of overlong surveys [25]. Neverthe-less at the outset, it was unclear which items might proveto be the best indicators of the study's objectives, hence aninitial oversampling was intentional for this proof-of-con-cept study. While there are many justifiable strategies forsurvey streamlining such as (1) item winnowing on thebasis of face validity, (2) selecting only the half-dozenmost reliable items in each scale, (3) scale reduction viafactor analysis, or (4) multi-trait, multi-method matrixprocedures, the overall objective in this study was to max-imize the discrimination between enhanced and controlAPPE and to test for survey items which best accom-plished that. Consequently, we examined the items withineach learning model scale that demonstrated the highestenhanced versus control differentiation. In an effort torebalance scale "lopsidedness" where some scales had asfew as nine items while others had as many as 29, weexamined results for four, five, six, and seven-item solu-which maximized the scales' sensitivities while retaininghigh alpha-reliabilities and holding scale bias to a mini-mum. Scale sensitivity (ability to discriminate betweenenhanced and control APPE) increased from 0.41 to 0.57through judicious item selection. This refined combina-tion of scale items allows accurate discrimination betweenenhanced settings and the two different control groups of89.0% and distinguishes the enhanced from the com-bined control participants with an accuracy of 96.2%. Aswell, all five of the refined learning model scales becomesignificantly sensitive to enhanced/control differenceswhereas in the original scales, only some were.As before, the number of patient consultations is by farthe best predictor of the enhanced APPE arm member-ship. In order of descending sensitivity within learningmodel scales, Table 6 shows these refined items with sig-nificance of Learning Model Scale sensitivity in brackets.In their refined forms, all five of the learning model scalesbecame significant discriminators between enhanced andcontrol APPE, and all but one was unbiased in terms ofpreceptor versus student differences. Only the estimatednumber of patient consultations again showed consistentoverestimating by preceptors (p < 0.007) irrespective ofenhanced vs. control membership. As Table 6 also shows,certain pharmaceutical care competencies such as docu-menting, providing follow-up, prioritizing drug-relatedproblems, monitoring, etc., appear repeatedly – irrespec-tive of whether they were phrased as Learning Opportuni-ties, Skills Improvements, or Attitude-related. These arethe pharmaceutical care practice aspects of patient consul-tations, a good indicator that direct interaction withpatients is a most powerful learning tool.LimitationsThe study's interpretation and its generalization to othersettings need to be viewed with certain cautions. The par-ticipant sample was both small and select, as befits a pilotstudy. Preceptors and students alike were volunteers andsuch self-selection may result in a Hawthorne-like upwarddrift in both interest and performance. Whether moremainstream participants (both preceptors and students)would demonstrate similarly favourable improvementsbetween enhanced vs. control reports remains to betested. The enhanced APPE intervention consisted of twocomponents: preceptor education and student prepara-tion prior to the start of the clerkship. One could reason-ably question how much of the improved learning thatwas observed could have been the result of increased stu-dent preparation compared to preceptor training. The cur-rent study design and participant numbers do not allow adirect analysis of their reciprocal effects, and future inves-tigation is planned. The study was retrospective in its con-Page 11 of 13(page number not for citation purposes)tions for each of the 5-stage learning model scales. Table5 shows the results for a six-item version (for each stage)ceptualization whereas pre/post designs coupled withenhanced/control conditions offer more explanatoryBMC Medical Education 2008, 8:17 http://www.biomedcentral.com/1472-6920/8/17power. The study's 70-item Scantron questionnaire pre-sented some disadvantages. Questions and theirresponses were overprinted on about every third line ofthe Scantron bubbles resulting in alignment problemsboth in terms of printing and ease of reading. Future stud-ies will ensure that respondents have the sole task of inter-preting the questions and generating accurate responseswhile the researchers will accept the task of assigning scalenumbers to conceptual categories even though it results inmore laborious data entry. Also, 70 questions areundoubtedly too many resulting in respondent fatigueand loss of interest. Judicious combinations of reliabilityand discriminant analyses with larger and more represent-ative samples will allow even further streamlining suchthat future questionnaires are shorter, more targeted andmore focused, probably in the 20-to-30 item (and 6 or 7domain) range of well-validated clinical learning instru-ments such as Stanford University's SFDP-26 or ClevelandClinic's Teaching Effectiveness Instrument measures[26,27]. Finally, the learning of competencies wasassessed, in part, by student self-report. It could be arguedthat it is in a student's best interest to report that learninghas taken place, and that this motivation might be greaterafter a 5-day intervention. It was reassuring, therefore, tocally different. It can also be argued that student self-report and, more generally, the opportunities for studentsto reflect on their own learning, are vital components ofprograms such as the ones presented here.ConclusionIn summary, the proof-of-concept examination showedthat the enhanced arm generated higher overall resultsthan did the control arms, with specifically higher scoresin student views about patient consultations, skillsimprovement and attitudes toward pharmaceutical carepractice. Similarly, preceptors reported increased precep-tor supports, numbers of patient consultations, studentskills improvement, and enhanced student attitudes.These study outcomes helped the SPEP faculty to securefunding from several additional community pharmacychains within British Columbia to further develop, evalu-ate and expand the enhanced APPE to other preceptorsand students in future years. Concurrent with develop-ment of the enhanced APPE's rationale and operationalrealities, an assessment instrument was developed,reviewed and refined which was closely keyed to therequirements and expectations of SPEP objectives. Also,the psychometric properties of the assessment survey wereTable 6: Refined scale composition with the six best-discriminating items per scale.Number of patient Consulted: (F = 41.35, p < 0.000)*. • Estimated number of consultations.Learning Climate: (F = 3.88, p < 0.026). • Preceptor encouraged patient follow-up.• SPEP was significant in stimulating PC delivery to patients.• Preceptor met with me on a regular basis to talk about patient issues.• Clerkship gave me opportunities to practice and improve my PC skills.• Preceptor identified activities for me to meet PC learning objectives.• Preceptor encouraged me to provide evidence and justify my recommendations.Learning Opportunities: (F = 3.23, p < 0.046). • Document patient information in forms usable by other pharmacists.• Provide follow-up (phone or in-person) using the monitoring plan.• Discuss monitoring parameters with patients regarding their therapies.• Determine whether patients are managing their medications or have questions.• Prioritize drug-related problems so that most relevant problems get addressed first.• Question patients to determine their experiences regarding drug effectiveness.Skills Improvement: (F = 20.69, p < 0.000). • Provide follow-up (phone or in-person) using the monitoring plan.• Integrate information for patients with multiple problems to identify drug-related problems.• Provide comprehensive PC to patients with fewer problems (4 or less).• Integrate patient information to identify actual/potential drug-related problems.• Prioritize drug-related problems so most relevant problems get addressed first.• Document patient information in forms to be used by other pharmacists/practitioners.Attitude Enhancement: (F = 6.86, p < 0.002). • Disagrees: Follow-up is too time-consuming.• Providing PC gives me a better understanding of patients' drug-related needs.• Agrees: I am comfortable with the process of providing PC.• Discussing monitoring parameters with patients is important.• Disagrees: Follow-up is too expensive.• Documenting care plans for continuity of care is important.* F and p values indicate the significance with which each scale discriminated between enhanced and control experiences.Page 12 of 13(page number not for citation purposes)discover that comparisons of the students' perceptions oftheir learning to those of the preceptors were not statisti-examined during this first iteration of the enhanced SPEProtation and shown to be generally supportive of the pro-BMC Medical Education 2008, 8:17 http://www.biomedcentral.com/1472-6920/8/17gram's conceptual underpinnings – both in terms oflearning process and of pharmaceutical care contentdomains. Finally, scale refinement analyses pointed to theneed for briefer and more precise scales of clerkship learn-ing and experience.AbbreviationsThe following abbreviations were used but not defined inthe text: SPSS, Statistical Package for the Social Sciences,ANOVA, Analysis of VarianceCompeting interestsThe author(s) declare that they have no competing inter-ests.Authors' contributionsRK conceived the study; participated in the design andimplementation of the study and survey; participated inthe acquisition of the data and interpretation of the data;and drafted and revised the manuscript. GP participatedin the conception and design of the study and survey, par-ticipated in interpretation of the data and revision of themanuscript. JBC conducted data analysis and participatedin the interpretation of the data and drafting and revisingof the manuscript. All authors read and approved the finalmanuscript.AcknowledgementsShoppers Drug Mart, British Columbia, Canada provided funding for the preceptor education workshop and evaluation of the pilot study. Professor John B Collins provided statistical support. We thank all participants in the study.References1. Hepler CD, Strand LM: Opportunities and responsibilities inpharmaceutical care.  AJHP 1989, 47:533-543.2. Cipolle RJ, Strand LM, Morley PC: A new professional practice.  InPharmaceutical Care Practice Edited by: Zollo S, Navrozv M. New York:McGraw Hill; 1998:1-35. 3. Zhan C, Arispe I, Kelley E, Ding T, Burt CW, Shinogle J, Stryer D:Ambulatory care visits for treating adverse drug effects inthe United States, 1995–2001.  Jt Comm J Qual Saf 2005,31(7):372-378.4. Budnitz DS, Pollock DA, Weidenbach KN, Mendelsohn AB,Schroeder TJ, Annest JL: National Surveillance of emergencydepartment visits for outpatient adverse drug events.  JAMA2006, 296(15):1858-1866.5. Kassam R, Farris KB, Burback L, Volume CI, Cox CE, Cave A: Phar-maceutical care research and education project (PREP):pharmacists' interventions.  JAPhA 2001, 41:401-410.6. Currie JD, Chrischilles EA, Kuehl AK, Buser RA: Effect of a trainingprogram on community pharmacists' detection of and inter-ventions in drug-related problems.  JAPhA 1997:182-191.7. Buurma H, De Smet PAGM, Leufkens HGM, Egberts A: Evaluationof the clinical value of pharmacists' modifications of prescrip-tion errors.  Br J Clin Pharmacol 2004, 58(5):503-511.8. Vinks THAM, De Koning FHP, De Lange TM, Egberts TCG: Identifi-cation of potential drug-related problems in the elderly: therole of the community pharmacist.  PWS 2006, 28:33-38.9. Etemad LR, Hay JW: Cost-effectiveness analysis of pharmaceu-tical care in medicare drug benefit program.  Value Health10. Schumock GT, Butler MG, Meek PD, Vermeulen LC, Arondekar BV,Bauman JL: Evidence of the economic benefit of clinical phar-macy services: 1996–2000.  Pharmacotherapy 2003, 1:113-132.11. Association of Faculties of Pharmacy of Canada (AFPC): Develop-ment of levels and ranges of educational outcomes expectedof baccalaureate graduates.   [http://www.afpc.info/downloads/1/Educational_Outcomes_1999.pdf].12. Kassam R, Volume-Smith C: Focus group methodology todevelop a community pharmaceutical care clerkship pro-gram.  Am J Pharm Educ 2003, 67(3article 88 [http://www.ajpe.org/default.asp].13. Kassam R: Students' and preceptors' experiences and percep-tions of a newly developed community pharmacy pharma-ceutical care clerkship.  Pharm Educ 2006, 6(3):1-10.14. Kassam R: Evaluation of pharmaceutical care opportunitieswithin an advanced pharmacy practice experience.  Am JPharm Educ 2006, 6770(3article 49 [http://www.ajpe.org/default.asp].15. Zaremski DG, Boyer G, Vlasses PH: A survey of advanced com-munity pharmacy practice experiences in the final year ofthe PharmD curriculum at the US colleges and schools ofpharmacy.  Am J Pharm Educ 2005, 69(1article 2 [http://www.ajpe.org/default.asp].16. Ovretveit J: Evaluating Health Interventions Philadelphia: Open Univer-sity Press; 1998. 17. Nunnally JC, Bernstein I: Psychometric Theory New York: McGraw-Hill;1994. 18. Campbell DT, Fiske DW: Convergent and discriminant valida-tion by the multi matrix.  Psychological Bulletin 1959, 56:81-105.19. Messick S: Validity.  In Educational measurement Edited by: Linn R.New York: American Council on Education and Macmillan PublishingCompany; 1989:13-103. 20. Association of American Medical Colleges (AAMC): Educating medicalstudents: assessing change in medical education: the road to implementa-tion (ACME-TRI) Washington, DC: Association of American MedicalColleges; 1992. 21. Sheets KJ, Quirk ME, Davis AK: The family medicine curriculumresource project: implications for faculty development.  FamMed 2007, 39(1):50-52.22. Lave J, Wenger E: Situated learning: Legitimate peripheral par-ticipation.  New York: Cambridge University Press; 1991. 23. Swanson AG, Anderson MB: Educating medical students: assess-ing change in medical education – the road to implementa-tion.  Acad Med 1993, 68(6):S1-S46.24. Muller S: Physicians for the twenty-first century: report of theproject panel on the general professional education of thephysician and college preparation for medicine.  J Med Educ1984, 59(11):1-208.25. Dillman DA: Mail and telephone Surveys, the Total DesignMethod.  New York: John Wiley and Sons; 1978. 26. Williams BC, Litzelman DK, Babbott SF, Lubitz RM, Hofer TP: Vali-dation of a global measure of faculty's clinical teaching per-formance.  Academic Medicine 2002, 77(2):177-180.27. Copeland HL, Hewson MG: Developing and testing an instru-ment to measure the effectiveness of clinical teaching in anacademic medical center.  Academic Medicine 2000,75(2):161-166.Pre-publication historyThe pre-publication history for this paper can be accessedhere:http://www.biomedcentral.com/1472-6920/8/17/prepubPage 13 of 13(page number not for citation purposes)2003, 6:425-435.


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items