UBC Faculty Research and Publications

Pilot-testing an adverse drug event reporting form prior to its implementation in an electronic health… Chruscicki, Adam; Badke, Katherin; Peddie, David; Small, Serena; Balka, Ellen; Hohl, Corinne M Oct 11, 2016

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-40064_2016_Article_3382.pdf [ 838.75kB ]
JSON: 52383-1.0357967.json
JSON-LD: 52383-1.0357967-ld.json
RDF/XML (Pretty): 52383-1.0357967-rdf.xml
RDF/JSON: 52383-1.0357967-rdf.json
Turtle: 52383-1.0357967-turtle.txt
N-Triples: 52383-1.0357967-rdf-ntriples.txt
Original Record: 52383-1.0357967-source.json
Full Text

Full Text

Chruscicki et al. SpringerPlus  (2016) 5:1764 DOI 10.1186/s40064-016-3382-zRESEARCHPilot-testing an adverse drug event reporting form prior to its implementation  in an electronic health recordAdam Chruscicki1, Katherin Badke2, David Peddie3, Serena Small3, Ellen Balka3,4 and Corinne M. Hohl2,4*Abstract Background: Adverse drug events (ADEs), harmful unintended consequences of medication use, are a leading cause of hospital admissions, yet are rarely documented in a structured format between care providers. We describe pilot-testing structured ADE documentation fields prior to integration into an electronic medical record (EMR).Methods: We completed a qualitative study at two Canadian hospitals. Using data derived from a systematic review of the literature, we developed screen mock-ups for an ADE reporting platform, iteratively revised in participatory workshops with diverse end-user groups. We designed a paper-based form reflecting the data elements contained in the mock-ups. We distributed them to a convenience sample of clinical pharmacists, and completed ethnographic workplace observations while the forms were used. We reviewed completed forms, collected feedback from pharma-cists using semi-structured interviews, and coded the data in NVivo for themes related to the ADE form.Results: We completed 25 h of clinical observations, and 24 ADEs were documented. Pharmacists perceived the form as simple and clear, with sufficient detail to capture ADEs. They identified fields for omission, and others requir-ing more detail. Pharmacists encountered barriers to documenting ADEs including uncertainty about what consti-tuted a reportable ADE, inability to complete patient follow-up, the need for inter-professional communication to rule out alternative diagnoses, and concern about creating a permanent record.Conclusion: Paper-based pilot-testing allowed planning for important modifications in an ADE documentation form prior to implementation in an EMR. While paper-based piloting is rarely reported prior to EMR implementations, it can inform design and enhance functionality. Piloting with other groups of care providers and in different healthcare set-tings will likely lead to further revisions prior to broader implementations.Keywords: Adverse drug events, Pilot-testing, Electronic medical records, Reporting© 2016 The Author(s). This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.BackgroundAdverse drug events (ADEs) are harmful and unintended consequences of medications that account for 1.8 mil-lion emergency department visits in Canada each year, and are a leading cause of unplanned admissions (Hohl et al. 2001; Zed et al. 2008; Budnitz et al. 2011). Despite the significant burden that ADEs pose on patients and the healthcare system, they are often not documented by clinicians, nor effectively communicated between health professionals or across healthcare settings (Hohl et  al. 2005, 2010), contributing to unintentional re-exposures of culprit drugs and repeat ADEs (Zhang et al. 2007). Van der Linden et al. (2006) estimate that 27 % of medications withdrawn during hospitalization due to an ADE are re-prescribed within only 6  months, indicating an urgent need to develop electronic systems that can help clinical care providers prevent repeat unintentional exposures to harmful drugs. In a recent systematic review, under-reporting of ADEs by healthcare providers was identi-fied as the main reasons why the effectiveness of current electronic systems to prevent unintentional re-expo-sures is limited (Van der Linden et  al. 2013). Improved, Open Access*Correspondence:  chohl@mail.ubc.ca 2 Department of Emergency Medicine, University of British Columbia, 855 West 12th Avenue, Vancouver, BC V5Z 1M9, CanadaFull list of author information is available at the end of the articlePage 2 of 9Chruscicki et al. SpringerPlus  (2016) 5:1764 structured electronic documentation of ADEs may facili-tate the creation of patient-specific report that can be used to generate medication-level or medication class-level alerts to prevent unintentional re-exposures (Van der Linden et al. 2015).While many electronic medical records (EMRs) pro-vide dedicated space allowing care providers to record ADEs, most are focused on allergy information and do not provide the option for structured reporting (Van der Linden et al. 2013). A systematic review of ADE reporting systems that are external to EMRs (e.g., Health Canada’s Medwatch program) found wide variation in the variety and type of ADE data collected (submitted). None of the systems reported pilot-testing electronic fields prior to their implementation to ensure user-friendliness, suc-cinctness, relevance and correct interpretation of fields by care providers. In participatory workshops completed by our group reflecting the views of over 120 care provid-ers in varied clinical settings, the length of ADE reporting forms, the time required to complete them, duplication of information requested, and lack of relevance to clini-cal care were barriers to ADE documentation (submitted and unpublished data). Creating structured, succinct and clear ADE data input fields in electronic record systems that can be leveraged to create patient-specific safety alerts to avoid unintentional re-prescribing was per-ceived as an incentive to report.The design of electronic information systems in health-care is complex, as systems must be user-friendly, meet the needs of multiple end-user groups, require approval from a broad range of stakeholders, and be adaptable to multiple environments (Kushniruk 2002). New systems implemented without pilot-testing and refining often fall short of anticipated goals due to design failures, or systems’ architecture constraints that could have been identified and addressed prior to their final build. This is particularly evident in healthcare where the rapidly changing user needs, incomplete information and shift-ing goals can derail even the most meticulously planned system implementation (Kushniruk 2002). Pilot studies provide an opportunity to evaluate new concepts at inter-mediate stages of design (Tejilingen et al. 2001). However, piloting electronic data forms is time-consuming and costly, as reprogramming is required to introduce refine-ments. Instead, pilot-testing paper-based forms may yield more advanced designs at lower cost and save the time and costs required to reprogram interfaces (Grady 2000). Paper-based, iterative design has been widely adopted by software developers, yet few examples of this approach in healthcare systems design exist (Anderson et al. 2001; Girsedale et al. 1997). Our objective is to describe paper-based piloting of a new electronic ADE documentation platform that aims to enhance communication between providers, and the design insights that resulted from this process.MethodsDesign and settingThis qualitative study was part of a larger research pro-gram, which has as its goal the design and implementa-tion of an electronic ADE reporting system within an EMR at thirteen hospitals (Peddie et al. 2016). This study was conducted in the emergency departments and hos-pital wards of Vancouver General Hospital (VGH), a ter-tiary care referral, and Lions Gate Hospital (LGH), an urban community hospital in British Columbia, Canada, between June and August, 2015.Compliance with ethical standardsThe University of British Columbia Clinical Research Ethics Board approved the study protocol (H13-02316-009). We obtained verbal informed consent from all participants. This study was funded by the Canadian Institutes of Health Research (Grant No. 2935460). None of the authors have any conflicts of interest to declare.Study participantsWe enrolled a convenience sample of key informants, all of whom were clinical pharmacists working in settings with a high ADE prevalence. We recruited participants by sending email invitations to all clinical pharmacists working at both institutions through the Department of Pharmaceutical Sciences, and encouraged volunteers to recruit colleagues by word-of-mouth. The only inclusion criterion was that participants actively practice clinical pharmacy at a participating hospital.Design of the paper‑based ADE formWe previously identified a minimum required dataset for ADE reporting by conducting a systematic review of the literature (submitted), and eight participatory work-shops with over 120 physician and pharmacist end users working in inpatient and outpatient settings across our healthcare region. The participants identified which data fields were relevant, and proposed a sequence in which fields should be presented to end-users to enhance func-tionality. We used this information to draft a paper-based version of the electronic reporting form (Fig.  1), and organized individual ADE fields into boxes labeled A to I to help with the progression of the form (Table 1). The electronic version will use auto-populating fields that contain medication dispensing information from drug plan data which we could not represent in the paper-based form. Page 3 of 9Chruscicki et al. SpringerPlus  (2016) 5:1764 Adverse Drug Event Reporting FormA. Select Suspect Drugs****In the electronic version, this section will self-populate a checklist with the current list of drugs from Pharmanet, BPMH and active inpatient medications. You will also have an option of manually entering the medication. After you select the culprit medications from this list, in the electronic version they would populate the LIGHT GREY fields in this form. In the paper form please fill them out manually.For the purpose of this form, please list names of suspect drugs onlyB1. Suspect Drug 1Drug/Product name Dose taken/received Route of administration OralSCIVTopicalIMFrequency taken/receivedIndication for drug Date of last dispense_________________Irrelevant Unknown> 1 yearOther dosing informationC. What type of Adverse Drug Event do you suspect?Adverse Drug Event type (select all that apply):Adverse Drug ReactionAllergyIncorrect/Wrong DrugSubtherapeutic dosesSupratherapuetic dosesTreatment failureDrug withdrawalDrug interactionNon-adherenceOtherDescribe the drug interaction:E. Treatment recommended or administered Suspect Drug1 and doseDiscontinue medicationChange dose to________________No changeSuspect Drug 2 and doseDiscontinue medicationChange dose to________________No changeD. Are there symptoms, diagnoses, or laboratory tests that you suspect are a manifestation of the Adverse Drug Event?Symptom caused or exacerbated by the Adverse Drug EventDiagnosis caused or exacerbated by the Adverse Drug EventRelevant laboratory data (include dates)Additional commentsB2. Suspect Drug 2Drug/Product name Dose taken/received Route of administration OralSCIVTopicalIMFrequencytaken/receivedIndication for drug Date of last dispense_________________Irrelevant Unknown> 1 yearOther dosing informationG. Causality/OutcomeWhat happened to the patient’s symptoms after dechallenge/treatmentRequires Follow-Up_________________________________Complete resolutionWorsenedImproved without complete resolutionNo changeIndicate your level of certainty that the adverse event was caused by the suspect drug(s):PossibleProbable DefiniteOutcome caused by Adverse Drug EventDeath resultingPermanent Disability _________________________________Exacerbated pre-existing condition_________________________________Congenital anomalyHospitalizationEmergency Department VisitUnknownOther (specify)_________________________________Additional commentsF. Add new medicationsSpecify new medication 1 Dose Route Frequency Start dateOther treatments/ Additional commentsSpecify new medication 2 Dose Route Frequency Start dateOther treatments/ Additional comments H. Report submission – note this is for STUDY PURPOSES ONLY, no actual report will be submitted based on this format at this timeReport is incomplete (will remain in inpatient system only)Report is complete (will be recorded by Health Canada)Communicate to outpatient provided in PharmanetI. Follow-up itemsFig. 1 Paper-version of the electronic ADE reporting formPage 4 of 9Chruscicki et al. SpringerPlus  (2016) 5:1764 Data collectionWe used lightweight ethnography, a social science approach allowing investigators to collect specific and relevant information efficiently while accepting the impossibility of a complete understanding of a work set-ting (Randall et  al. 2007). Lightweight ethnography can provide guidelines for technology design as it is neither time nor resource intensive, and is well suited for pilot studies. One research assistant (AC) shadowed clini-cal pharmacists during 2–4  h data collection shifts. At the beginning of each observation shift, the research assistant presented pharmacists with the ADE form and explained the form functionality, as well as the scope of ADEs that it is aimed to capture. Pharmacists were asked to complete the form when an ADE was encountered. In addition to collecting ethnographic information, the research assistant recorded how long it took users to complete the form, from the moment they started filling it out to when it was ready to be submitted. The research assistant also recorded the types of information sources accessed and the number of internal and external inter-ruptions to pharmacists’ work while completing the form. Internal interruptions were defined as instances where the user had to access another information source in order to complete the form. External interruptions were defined as instances where another person inter-rupted the user completing the ADE form. The research assistant recorded pharmacists’ comments and impres-sions during work, and subsequently completed semi-structured interviews with pharmacists about the diagnostic process, users’ perceptions of individual fields and the form as a whole, and challenges in documenting ADEs. We collected all completed ADE reporting forms after each shift.Data analysisWe calculated the proportion of completed data fields for individual ADE reporting forms by taking the num-ber of data fields completed by the user and dividing this number by the twenty-seven data collection fields con-tained on the ADE form. An individual data collection field was defined as one unique question-response pair. A data field was marked as filled, if the user provided either a written response or checked off a tick-box. The form was divided into boxes labeled A to I. Boxes B, “Suspect drug”, and F, “Add new medication”, contain duplicate spaces to accommodate users entering more than one drug at a time. If only one culprit drug was suspected, the duplicate space was left out of the average data field completion rate calculations. If two culprit drugs were suspected, the duplicate space was included in calcula-tions. To calculate the average data field completion rates across all users, we averaged the individual completion rates from all the collected forms. The average and stand-ard deviation for the number of interruptions and time to complete the form were used to calculate the 95 % con-fidence intervals. We transcribed field notes and coded data using NVivo 10, a qualitative data analysis software that can be used to interrogate and analyze unstructured qualitative data. We combined inductive reasoning (mov-ing from particular to general) and constant comparison (generalizing concepts and categories) to code the data Table 1 Data collection fields used in the paper version of the ADE reporting formConcept Data field Field descriptionName and dose of the culprit drug(s) A. Select suspect drug(s) Enter a list of suspected drugsB. Suspect drug(s) Enter the name(s), dose, route of administration, frequency, indication for, date of last dispense, and other relevant information about the suspect drugEffect(s) of the ADE on the patient C. What type of ADE do you suspect? Select the type of ADE that occurred, if it was a drug-drug interaction, describe it in more detailD. Are there symptoms, diagnoses, or laboratory tests that you suspect are an ADE manifestation?Describe the symptoms and/or diagnosis caused or exacerbated by the ADE. Include relevant laboratory data and additional commentsTreatment received E. Treatment recommended or administered Describe the treatment for the ADEF. Add new medications List any newly recommended medicationsOutcome G. Causality and outcome Describe what happened to the patient’s symptoms after treatment, the outcome of the ADE, and indi-cate the level of certainty that the event was caused by the suspect drug. List additional commentsH. Report submission Indicate whether the report should remain in the inpatient record, be submitted to Health Canada or communicated to the drug planI. Follow-up items List additional commentsPage 5 of 9Chruscicki et al. SpringerPlus  (2016) 5:1764 for themes related to the ADE form, challenges in diag-nosing and reporting ADEs, and workflow.ResultsADE reportingWe observed six clinical pharmacists for 25 h across 11 data collection shifts at VGH and LGH. During this time, pharmacists completed 24 ADE forms. The clinical phar-macists perceived the paper-based ADE form as an effi-cient, user-friendly, and intuitive way to record ADEs. Users preferred the form on a single page, and preferred checkboxes over drop-down menus and free text. They also felt that some data entry fields could be omitted (Table  2). For example, pharmacists indicated that sec-tion B, “Suspect Drug”, should be better at capturing the order in which medications were prescribed. Pharmacists also felt that subsection of “Suspect Drug”—“date of last dispense” lacked utility and could be removed. Users felt that instructions for some fields needed further clarifica-tion or simplification. For example, section F, “Add new medications”, required clarification. Pharmacists were unsure about whether to list medications used to treat an ADE, a drug that was prescribed to replace a culprit medication that was discontinued, or both. They also thought that section H, “Report submission” was confus-ing, as it contained multiple reporting options (Table 2).During piloting, we observed that pharmacists inter-preted the word “reporting” in our form to imply that its purpose was to generate an ADE report for Health Can-ada, something they felt was outside of the scope of clini-cal care provision.Barriers to reportingPharmacists were most likely to report more severe or rare ADEs, and adverse drug reactions were seen as the most “reportable” events. Pharmacists were gener-ally hesitant to report ADEs when they were concerned about creating a permanent record without the ability to update, modify or delete it (e.g., if an alternative diagno-sis became obvious at a later point in time), even though we emphasized that this would be possible in an elec-tronic version. Pharmacists also thought that some ADE diagnosis might become irrelevant with time (e.g., a per-son with orthostatic hypotension caused by a high drug dose, who becomes more hypertensive with time and requires higher doses of said hypertensive medication). This indicated the need to for pharmacists to be able to re-access and modify electronic reports. Although phar-macists were familiar with the scope of ADEs intended to be captured by this form, some ADEs were challenging to recognize and diagnose. In these instances pharma-cists remained uncertain about which events warranted Table 2 Comments and proposed resolutionsNA not applicable, MD physicianData field Comments Proposed solutionA. Select suspect drug(s) None NAB. Suspect drug(s) “Date of last dispense” is irrelevantDifficult to capture order of prescribingDifficult to enter complex dosage regimesRemove the “date of last dispense” fieldIncrease the amount of free-text entryRemove one data entry box for drugs (to autopopulate in electronic form)C. What type of ADE do you suspect? Checkboxes preferred over drop-down menusProvide the option to describe “other”Use check boxes instead of drop-down menusModify the free-text optionD. Are there symptoms, or laboratory tests that you suspect are an ADE manifestation?Provide space to list vital signs Add option to add vitals in the “laboratory data” sectionE. Treatment recommended  or administeredNeed to be able to input start and stop time of the changesAdd a “start” and “stop” date data inputF. Add new medications Name of field is confusingUnsure about which medications to listChange the name of the box to clarify the instruc-tionsG. Causality and outcome Pharmacists often don’t know the patient’s outcomePharmacists would like to pass the form to another care provider for completion (e.g., GP)Provide option for other care provider(s) to complete symptom resolution and outcome reporting.Requires linkage to MD electronic data entryH. Report submission Improve clarity of instructions for inpatient reportingPharmacists are hesitant to report without a definite diagnosis, especially if their identification is attached to the reportSimplify reporting optionsAdd option to remove or modify existing report(s)Educate pharmacists that the form is primarily to improve documentation and communication between care providers, rather than to reportChange name of form to “documentation and communication” to clarify intentI. Follow-up items Field is unnecessary Remove free-text boxesPage 6 of 9Chruscicki et al. SpringerPlus  (2016) 5:1764 documentation, indicating the need for education and guidance around this during the implementation phase of the electronic fields. They were also hesitant to report events in which they had not witnessed the patient’s outcome, as the outcome often impacted their causality assessment. Finally, they were reluctant to report sus-pected events. Most pharmacists initiated ADE docu-mentation independently, without discussing the case with other care providers, even though an ADE diag-nosis requires ruling out alternative diagnoses by the treating physician (e.g., ruling out of urinary tract infec-tion prior to ascribing a diagnosis of delirium to a drug). Of completed forms, 15 of 22 (68  %) listed as outcome “requires follow-up,” highlighting the importance of ena-bling communication by allowing multiple care providers to access an electronic ADE report in both the inpatient and outpatient settings. This could enable communica-tion whenever follow-up is provided in a different health-care setting (e.g., after hospital discharge). Pharmacists were concerned that incorrectly reported suspect ADEs could conceivably hinder a patient’s future access to indi-cated medications. Thus, in an electronic design, report-ers must be provided with the option of communicating with other care providers about new ADEs, as well as any changes to the patient status, as means of overcoming concerns about withholding what might be appropriate medications.The average data field completion across all ADE forms was 50 % (95 % CI 47–53 %, n = 24). The least completed sections were D “are there symptoms, diagnoses, or labo-ratory tests that you suspect are an ADE manifestation?” of which, on average, only 41 % (95 % CI 35–47 %, n = 24) were complete. The low completion rate within section D was primarily due to the pharmacist deciding that only one field was necessary to express the problem caused by the ADE—generally either “symptom” or “diagnosis” (e.g. a cough caused by Ramipril did not require diagnosis and lab data, “GI bleed” was sufficient to describe an ADE to Rivaroxaban). Section F “add new medication” was only completed 19 % (95 % CI 6–32 %) of the time. Five out of six pharmacists generally completed one of four data col-lection fields in section D (symptoms, diagnosis, lab data, or additional comments), as not all fields were relevant to each ADE (e.g., lab values are only relevant when ADE has biochemically measurable outcomes). Pharmacists would usually fill out either the “symptom” or “diagnosis” of the ADE, but usually not both as this information was perceived as somewhat redundant. Our findings high-light the fact that form certain ADEs the report can be complete, even though users may not fill out all the data fields.The most commonly reported ADEs were catego-rized as adverse drug reactions (16/24; 67 %) followed by drug–drug interactions (3/24; 13  %) and allergies (3/24; 13  %). The most common choice of treatment was dis-continuation of the drug (15/24; 63 %). Pharmacists were reluctant to report any information about symptom res-olution, and most commonly selected: “patient requires follow-up” (15/22; 68 %), as they were often not privy to this information at the time of reporting because symp-tom resolution would often occur only after discharge necessitating that another healthcare provider follow-up and complete the form. Follow-up was deemed neces-sary to report on alternative diagnoses that could rule out an ADE, and to update incomplete information (e.g., lab results).The average time required to complete one form was 5.0 min (95 % CI 4.4–5.6 min, n = 12). The average num-ber of internal interruptions was between 2 and 7, with a median number of 3 interruptions per form. Most interruptions occurred in order to access the printed patient’s outpatient medication dispensing record. Once implemented electronically, this step will be facilitated by prepopulating the ADE reporting form with a list of the patient’s dispensed medications, allowing pharmacists to tick the suspect drug(s) for the ADE.DiscussionOur objective was to pilot-test a paper-based version of a newly designed ADE reporting form in three clinical settings prior to integrating it into an EMR. Our work highlights the utility of pilot-testing health technology interventions by intended end-users within clinical set-tings in order to maximize user-friendliness, utility and relevance, even in  situations in which end-users were involved in earlier design stages. While there are dif-ferences between electronic and paper data collection forms, the two approaches can produce synonymous results (Boyer et al. 2002; Huang 2006). Although not all functionalities of an electronic form can be mimicked by a paper-based form, crucial design elements required for a successful electronic implementation became apparent to end-users in paper-based testing and will influence our future electronic build. Our fieldwork helped end-users and researchers anticipate how the ADE form’s function-ality could be improved to assist clinicians in communi-cating relevant ADE information between care providers on different wards and across healthcare sectors, and as handover tools. This enabled us to anticipate the need for electronic linkages between different components of the EMR being implemented, ideally including a bidirec-tional link with drug plan data.One of our concerns at the outset was that the form would be too lengthy and require too much time to com-plete, distracting its users from other work duties. Sur-prisingly, our fieldwork did not confirm this, as most Page 7 of 9Chruscicki et al. SpringerPlus  (2016) 5:1764 users completed the form within 5  min and generally approved of its length and level of detail. Although the paper-based version did not allow us to display future functionalities (e.g., pop-up windows, ability to revise ADE reports in the future), end-users were able to iden-tify preferences when different design options were pro-posed. An important caveat is that additional features added in an electronic build may contribute to increased functionality, but may also add complexity and require more time, necessitating further refinements.ADEs are vastly underreported using current ADE reporting systems (Hohl et  al. 2013; Wiktorowicz et  al. 2010). Our fieldwork identified important avenues for improving reporting that may be addressed in a future electronic ADE documentation and communication form that is integrated into an EMR. These include address-ing uncertainty about which ADE types should be docu-mented (possibly through pop-up instructions), allowing providers to document uncertainty in the ADE diag-nosis, enabling reports to be removed or modified after follow-up, providing space for alternative diagnoses, and enabling inter-professional communication across hand-overs and between inpatient and outpatient settings (pos-sibly via patient-specific safety alerts). In our study, the majority of reported ADEs were adverse drug reactions. Other kinds of ADEs, such as non-adherence, sub- or supra-therapeutic doses were seen as more complicated, as the implications of reporting were less clear. We used an extended definition of an ADE, which included non-compliance and improper dosing regiments. While, all these events fall under the scope of medication-related problems (MRPs), our form purposely avoided this term to increase signal to noise ratio, and prevent reporting multiple non-clinically significant events per patient, as our overarching goal was to prevent recurrence of seri-ous ADEs while avoiding alert fatigue and rendering documentation feasible. In previous workshops we held with end-users in advance of pilot-testing, pharmacists’ insisted on retaining the option to record non-adverse drug reaction ADEs (unpublished data). This conundrum might be addressed by educating users about the various kinds of ADEs encountered and need for communication across providers, and supporting a common approach to preventing future ADEs.This study confirms previous observational work by our group that suggests that ADE diagnosis is a com-plex and multi-step process (unpublished data). If ADE reporting is to succeed, electronic forms that are created for this process must reflect this complexity, and enable reporting as a multi-step process. Multiple care providers including those who provide insight into alternative diag-noses for suspect events or provide follow-up of patient outcomes must be able to access and update information. The immediate implication for the design of electronic reporting systems is that they must enable communica-tion between providers and across healthcare sectors. While we piloted the form with clinical pharmacists, doctors and nurses in hospital and community settings are likely to utilize the form as well. Thus, we anticipate further piloting and design adjustments as the form is implemented in other healthcare environments and for other provider groups.During our fieldwork, we referred to the ADE form as a reporting tool. However “reporting” was very spe-cifically associated with the communication of a subset of events to Health Canada through MedEffect form, as opposed to their documentation within an electronic record. An implication of “reporting” was an assumed permanency of the record that would be created, as none of the currently available reporting mechanisms allow for updates or modification after a report has been generated. As the overarching objective of our project is to develop a documentation tool that sup-ports communication between care providers (rather than communicate events to external agencies), we changed the name of our form to “Adverse Drug Event Communication and Documentation Form” to high-light its intended purpose. We hope that our findings highlight the need for a culture shift around ADE com-munication, from an approach that serves to generate health data for external agencies, implied by “report-ing”, to a patient-safety oriented approach that focuses on communication and documentation for prevention of repeat events.Low completion rates can indicate problems with availability of information needed to complete a section of the form, or content problems with the section itself. Among the sections with the highest non-completion rates were those for the ADE symptom and diagnosis. ADEs are notoriously difficult to diagnose, and our prior observational work and workshops with stakeholders suggested that providing a record which allowed a subse-quent care provider to re-trace evidence upon which an ADE diagnosis was based as an important aspect of ADE documentation. Pharmacists often listed presumed ADE symptoms; however clustering them into a diagnosis can be challenging, or require communication with physi-cians to rule out alternative diagnoses or await the results of confirmatory testing, leading them to skip this field. This finding may also in part explain some of the uncer-tainty expressed by pharmacists about reporting more complex, or less traditional ADEs.Pharmacists were often unclear about which treat-ment recommendations to list within the ADE doc-umentation form (e.g., whether to document the medication used to treat the ADE, or a medication used Page 8 of 9Chruscicki et al. SpringerPlus  (2016) 5:1764 to replace the culprit drug). As a result users would often leave this field empty. We were unable to capture the full functionality of our electronic form—which will enable the pharmacist to recommend changes to a patient’s medication regiment using the EMR to a physi-cian who can approve of them or alter them. While the electronic build may contain sufficient contextual infor-mation to address the ambiguities which existed in the paper based version of the form, this is likely to require electronic piloting.Our findings demonstrate the value of completing pilot studies of electronic health information technology implementations with paper-based forms. While these cannot mimic the full functionality of an electronic inter-face, they provided vital feedback for subsequent design and pre-implementation user education that we might have otherwise overlooked, including questions regard-ing systems architecture.LimitationsLightweight ethnography, although time and resource efficient, carries a risk of only skimming the surface and providing only partial explanations. By only briefly engag-ing in the work environment, the observers risk missing less common routines and events that could otherwise have been captured. We relied on volunteers as the sub-jects of our observations, and therefore used a limited number of participants. This study is limited by the sole inclusion of clinical pharmacists, who were identified in our healthcare settings as the most likely care providers to encounter and document ADEs. It is possible that we might have uncovered other aspects of the ADE form requiring modifications had we been able to recruit more participants from other clinical backgrounds or settings, and anticipate further design adjustments as the use of the form is expanded. Also, our findings are susceptible to the Hawthorne effect, which occurs when participants are aware of the study objectives, potentially influencing their behavior. Finally, paper-based field evaluation of software designs has limitations, and hence findings must be evaluated in relation to data collected through other means.ConclusionWe piloted a paper-based version of an ADE documen-tation form prior to its electronic build. As a result, we were able to modify its design, and envision unique requirements for the system’s architecture as well as educational needs prior to system implementation. As a result of our pilot study, we will be able to address these issues to enhance functionality prior to an electronic build.AbbreviationsADE: adverse drug event; EMR: electronic medical record; VGH: Vancouver General Hospital; LGH: Lion’s Gate Hospital.Authors’ contributionsAC and KB led and coordinated data acquisition and the development and writing of the manuscript. DP, SS, EB, and CH participated throughout the anal-ysis of the obtained data, interpretation, and writing of the manuscript and contributed intellectual content and feedback on drafts of the manuscript. All authors read and approved the final manuscript.Author details1 Faculty of Medicine, Queen’s University, 15 Arch Street, Kingston, ON K7L 3N8, Canada. 2 Department of Emergency Medicine, University of British Columbia, 855 West 12th Avenue, Vancouver, BC V5Z 1M9, Canada. 3 School of Communication, Simon Fraser University, 8888 University Drive, Burnaby, BC V5A 1A6, Canada. 4 Centre for Clinical Epidemiology and Evaluation, Vancouver Coastal Health Research Institute, 828 West 10th Ave, Vancouver, BC V5Z 1M9, Canada. AcknowledgementsThis research was sponsored by the Canadian Institutes of Health Research, Partnership for Health System Improvement Grant (No. 293546), The Michael Smith Foundation for Health Research (No. PJ HSP 00002), Vancouver Coastal Health, the BC Patient Safety and Quality Council, and Health Canada.Competing interestsThe authors declare that they have no competing interests.Received: 1 June 2016   Accepted: 25 September 2016ReferencesAnderson J, Fleek F, Garrity K, Drake F (2001) Integrating usability techniques into software development. IEEE Softw 18:46–53Boyer KK, Olson JR, Calantone RJ, Jackson EC (2002) Print versus electronic surveys: a comparison of two data collection methodologies. J Oper Manag 20:357–373Budnitz D, Lovegrove M, Shehab N, Richards C (2011) Emergency hospitali-zations for adverse drug events in older Americans. New Engl J Med 365:2002–2012Girsedale S, Graves M, Grunsteidl A (1997) Designing a graphical user interface for healthcare workers in rural India. Des Brief 97:471–478Grady HM (2000) Web site design: a case study in usability testing using paper prototypes. IEEE [Internet]. IEEE, pp 39–45. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=887259Hohl CM, Dankoff J, Colacone A, Afilalo M (2001) Polypharmacy, adverse drug-related events, and potential adverse drug interactions in elderly patients presenting to an emergency department. Ann Emerg Med 38:666–671Hohl CM, Robitaille C, Lord V, Dankoff J, Colacone A, Pham L et al (2005) Emergency physician recognition of adverse drug-related events in elder patients presenting to an emergency department. Acad Emerg Med 12:197–205Hohl CM, Zed PJ, Brubacher JR, Abu-Laban RB, Loewen PS, Purssell R (2010) Do emergency physicians attribute drug-related emergency department visits to medication-related problems? Ann Emerg Med 55:493–502Hohl CM, Kuramoto L, Yu E, Rogula B, Stausberg J, Sobolev B (2013) Evaluating adverse drug event reporting in administrative data from emergency departments: a validation study. BMC Health Serv Res 13:473Huang HM (2006) Do print and Web surveys provide the same results? Com-put Hum Behav 22:334–350Kushniruk A (2002) Evaluation in the design of health information systems: application of approaches emerging from usability engineering. Comput Biol Med 32:141–149Peddie D, Small SS, Badke K, Wickham ME, Bailey C, Chruscicki A, Ackerley C, Balka E, Hohl CM (2016) Designing an adverse drug event reporting system to prevent unintentional reexposures to harmful drugs: study Page 9 of 9Chruscicki et al. SpringerPlus  (2016) 5:1764 protocol for a multiple methods design. JMIR Res Protoc 5(3):e169. doi:10.2196/resprot.5967 (PMID: 27538362)Randall D, Rouncefield M, Harper R (2007) Fieldwork for design: theory and practice. Springer, LondonTejilingen E, Rennie AM, Hundley V, Graham W (2001) The importance of con-ducting and reporting pilot studies: the example of the Scottish Births Survey. Methodol Issues Nurs Res 34:289–295Van der Linden C, Kerskes MCH, Bijl AMH, Maas HAAM, Egberts ACG, Jansen PAF (2006) Represcription after adverse drug reaction in the elderly: a descriptive study. Arch Intern Med 166:1666–1667Van der Linden CMJ, Jansen PF, Grouls RJE, Van Marum RJ, Verberne MJW, Aussems LM et al (2013) Systems that prevent unwanted represcription of drugs withdrawn because of adverse drug events: a systematic review. Ther Adv drug Saf 4:73–90Van der Linden C, Jansen P, Van Geerenstein E, Van Marum R, Grouls R, Egberts T et al (2015) Reasons for discontinuation of medication during hospitalization and documentation thereof: a descriptive study of 400 geriatric and internal medicine patients. Arch Intern Med 170:1085–1087Wiktorowicz M, Lexchin J, Moscou K, Silversides A, Eggertson L (2010) Keeping an eye on prescription drugs, keeping Canadians safe. Active monitoring systems for drug safety and effectiveness in Canada and internationally. www.healthcouncilcanada.ca. Accessed 1 June 2016Zed P, Abu-Laban R, Balen R, Loewen P, Hohl C, Brubacher J et al (2008) Incidence, severity and preventability of medication-related visits to the emergency department: a prospective study. Can Med Assoc J 178:1563–1569Zhang M, Holman CDJ, Preen DB, Brameld K (2007) Repeat adverse drug reactions causing hospitalization in older Australians: a population-based longitudinal study 1980–2003. Br J Clin Pharmacol 63:163–170


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items