UBC Faculty Research and Publications

Evaluating adverse drug event reporting in administrative data from emergency departments: a validation… Hohl, Corinne M; Kuramoto, Lisa; Yu, Eugenia; Rogula, Basia; Stausberg, Jürgen; Sobolev, Boris Nov 12, 2013

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-12913_2012_Article_2907.pdf [ 335.62kB ]
JSON: 52383-1.0221349.json
JSON-LD: 52383-1.0221349-ld.json
RDF/XML (Pretty): 52383-1.0221349-rdf.xml
RDF/JSON: 52383-1.0221349-rdf.json
Turtle: 52383-1.0221349-turtle.txt
N-Triples: 52383-1.0221349-rdf-ntriples.txt
Original Record: 52383-1.0221349-source.json
Full Text

Full Text

RESEARCH ARTICLE Open AccessEvaluating adverse drug event reporting inadministrative data from emergencydepartments: a validation studyCorinne M Hohl1,2,3*, Lisa Kuramoto3, Eugenia Yu4, Basia Rogula3, Jürgen Stausberg5 and Boris Sobolev3,6AbstractBackground: Adverse drug events are a frequent cause of emergency department presentations. Administrativedata could be used to identify patients presenting with adverse drug events for post-market surveillance, and toconduct research in patient safety and in drug safety and effectiveness. However, such data sources have not beenevaluated for their completeness with regard to adverse drug event reporting. Our objective was to determinethe proportion of adverse drug events to outpatient medications diagnosed at the point-of-care in emergencydepartments that were documented in administrative data.Methods: We linked the records of patients enrolled in a prospective observational cohort study on adverse drugevents conducted in two Canadian tertiary care emergency departments to their administrative data. We comparedthe number of adverse drug events diagnosed and recorded at the point-of-care in the prospective study with thenumber of adverse drug events recorded in the administrative data.Results: Among 1574 emergency department visits, 221 were identified as adverse drug event-related in theprospective database. We found 15 adverse drug events documented in administrative records with ICD-10 codesclearly indicating an adverse drug event, indicating a sensitivity of 6.8% (95% CI 4.0–11.2%) of this code set. Whenthe ICD-10 code categories were broadened to include codes indicating a very likely, likely or possible adverseevent to a medication, 62 of 221 events were identifiable in administrative data, corresponding to a sensitivity of28.1% (95% CI 22.3-34.6%).Conclusions: Adverse drug events to outpatient medications were underreported in emergency departmentadministrative data compared to the number of adverse drug events diagnosed and recorded at the point-of-care.Keywords: Adverse drug event, Adverse drug reaction, Administrative data, Emergency department, Validation,Post-market surveillance, Drug safetyBackgroundOutpatient medication use is common, but may conferhealth risks that compromise its therapeutic benefits [1-3].Health risks associated with medications have been shownto vary substantially in clinical practice from those observedin published randomized controlled trials [4,5]. Capturingcomplete data on adverse drug events observed in clinicalpractice is important for post-market surveillance, drugregulatory activities, and research in drug safety and effect-iveness and patient safety [6-8].Emergency Departments play a pivotal role in NorthAmerican healthcare systems [9]. They serve as acutediagnostic and treatment centers for ambulatory patientswith unexpected and serious medical problems, as a safetynet for the underserved and uninsured, and are an access-ible portal of entry into acute care hospitals for sick pa-tients. A growing proportion of urgent outpatient visitsoccur in emergency departments, and the majority of hos-pital admissions in the United States occur through emer-gency departments [9]. Ambulatory patients sufferingfrom serious adverse drug events, the unexpected and* Correspondence: chohl@mail.ubc.ca1Department of Emergency Medicine, University of British Columbia, 855West 12th Avenue, Vancouver, BC V5Z 1 M9, Canada2Department of Emergency Medicine, Vancouver General Hospital, 855 West12th Avenue, Vancouver, BC V5Z 1 M9, CanadaFull list of author information is available at the end of the article© 2013 Hohl et al.; licensee BioMed Central Ltd. This is an open access article distributed under the terms of the CreativeCommons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, andreproduction in any medium, provided the original work is properly cited.Hohl et al. BMC Health Services Research 2013, 13:473http://www.biomedcentral.com/1472-6963/13/473unintended complications of medication use, commonlyseek care in emergency departments [10-15]. Therefore,emergency department administrative data may offerunique opportunities to track population-level data onclinically significant adverse drug events to outpatientmedications for the purposes of surveillance and research[16-20].Administrative databases are readily available, inex-pensive and can provide population-level health data onimportant health outcomes [8]. However, emergency de-partment administrative data have not been evaluatedfor their completeness in adverse drug event reporting.Our objective was to determine the proportion of ad-verse drug events identified at the point-of care in twoemergency departments by pharmacists and physiciansthat were recorded in administrative data.MethodsStudy designWe linked data from a prospective observational cohortstudy to emergency department administrative data [21].We obtained data on patient demographics, emergencydepartment visits, and adverse drug events to outpatientmedications from the prospective study. We used eachpatient’s unique identifier and emergency departmentvisit date to link the prospective study database with ad-ministrative databases to look for records of adversedrug events within the administrative data. The adminis-trative databases used ICD-10 diagnostic codes, and onealso recorded chief complaint codes. The University ofBritish Columbia Clinical Research Ethics Board (H10-01632) reviewed and approved the study protocol, andwaived the need for informed consent.SettingWe used data collected as part of a prospective observa-tional study that was conducted in two tertiary careemergency departments in Canada with a combinedcensus of 145,000 patient-visits per year: VancouverGeneral Hospital (VGH) and St. Paul's Hospital (SPH).Study cohortOur study cohort consisted of patients enrolled into theprospective study. Patients were eligible for enrolment ifthey presented to the VGH or SPH emergency depart-ments between July 1, 2008 and January 24, 2009.Research assistants enrolled patients using a previouslydescribed systematic patient selection algorithm to gen-erate a representative sample. We included all English-speaking patients who were 19 years of age or older andreported using at least one prescription or over-the-counter medication within two weeks of presentation. Inthe prospective study, we excluded patients if they wereagitated, presented with intentional self-poisoning, hadpreviously been enrolled, presented for a scheduled re-visit, had been transferred from another hospital, or leftagainst medical advice or prior to being seen by thephysician or pharmacist. When we linked the study data-bases we subsequently excluded the records of patientswith multiple visits on the same day because they re-sulted in unresolved linkages.Identification of adverse drug events at the point-of-careOne of three residency-trained clinical pharmacists whowere research assistants in the prospective study and thetreating emergency physician assessed each patient foradverse drug events in the emergency department in amanner that was independent and blinded to each other’sassessments using a pre-defined algorithm (Additional file1): First, the pharmacists evaluated whether or not the pa-tient’s visit was due to an adverse drug event usingthree adapted causality algorithms [22-24]. The causal-ity algorithms were used to standardize the pharma-cists’ assessments, and to ensure that the exacerbationof underlying disease was considered as a cause for anyevents deemed potentially due to an adverse drugevent. Inter-rater reliability of the pharmacists’ assess-ment using the causality algorithms was 0.75. After thepharmacy assessment was complete, we interviewedthe treating emergency physician using a standardizedquestionnaire to determine the patient’s working diagno-sis. When the physician and pharmacist determinations ofa patient’s adverse drug event status were concordant (i.e.,ADE/ADE or No ADE/No ADE), this was considered thecriterion standard. If there was any disagreement betweenratings (e.g., ADE/No ADE) or uncertainty (e.g., ADE/Un-certain), an independent committee consisting of a clinicalpharmacist and a medical toxicologist (who was also aphysician), both of whom were otherwise not involved inthe study, adjudicated the cases (See Algorithm for adjudi-cation committee, Additional file 1). If the case definitionof adverse drug events was met, we categorized the eventsaccording to the Hepler & Strand taxonomy (Additionalfile 2) [25].Identification of adverse drug events in theadministrative dataWe used administrative data that the Vancouver CoastalHealth Authority routinely submits to the Canadian In-stitute for Health Information (CIHI) National Ambula-tory Care Reporting System (NACRS). This databasecontains data on emergency department visits and in-cludes the patient’s diagnosis at the time of discharge inaddition to secondary diagnosis fields. We identifiedadverse drug events in the administrative data using alist of 650 ICD-10 codes generated through a review ofthe literature [26] and from Stausberg et al., [27,28]and by searching for relevant chief complaint codesHohl et al. BMC Health Services Research 2013, 13:473 Page 2 of 11http://www.biomedcentral.com/1472-6963/13/473(Additional file 3). Our ICD-10 code set includedcodes that may indicate a manifestation of an adversedrug event (e.g., K25 Gastric ulcer) or an external cause(e.g., Y40-Y59 Drugs, medicaments and biological sub-stances causing adverse effects in therapeutic use). Tosearch for categories of adverse drug events not classifiedas adverse drug reactions, we also included Y66 (non-ad-ministration of surgical and medical care), T36-50 (poi-soning by drugs, including overdose and wrong substancegiven or taken in error), T88.7-T88.9 (unspecified adverseevents of a drug, and unspecified complications of medicalcare), Y57.9 (complications of medical and surgical care:drug or medicament, unspecified), Y69 (unspecified mis-adventure during surgical and medical care), and Y88.0(sequelae of adverse effects caused by drugs in therapeuticuse). The likelihood of an ICD-10 code representing anadverse drug event was based on the ICD-10 code descrip-tion and on clinical reasoning (Table 1).DefinitionsAdverse drug events were defined as “untoward and un-intended symptoms, signs or abnormal laboratory valuesarising from the appropriate or inappropriate use ofprescription or over-the counter medications” [30-32].All cases deemed adverse drug events had to be associ-ated with an emergency department visit. Adverse drugevents were distinguished from drug-related problemsby the presence of untoward and unintended symptoms,signs or abnormal laboratory values. Once identified, ad-verse drug events were classified into mutually exclusivecategories according to their etiology (Additional file 2)[25]. Those due to drug exposure were classified as: (1)adverse drug reactions, defined as noxious and/or unin-tended responses to medication which occurred despiteappropriate drug dosage for prophylaxis, diagnosis ortherapy of the indicating medical condition; [32] (2) druginteraction, (3) drug use without indication, or (4)supratherapeutic/high dose. Adverse events related to al-cohol abuse and illicit drugs were not considered ad-verse drug events. Adverse drug events due to lack ofexposure to a drug were classified as: (6) subtherapeutic/low dose, (7) need to add drug/untreated indication, (8)wrong drug, or (9) noncompliance/failure to receivedrug [11,30,31,33] Cases in which drugs were never con-sidered for use in the first place were not considered ad-verse drug events. Adverse drug events were categorizedinto chief-complaint-related events versus those foundincidentally. The former were deemed to result in thepatient’s presenting complaint (i.e., a non steroidal anti-inflammatory drug leading to a gastrointestinal bleed, inwhich the patient’s complained of “vomiting blood”).The latter, were not deemed to result in the patient’spresenting complaint (i.e., hydrochlorothiazide leadingto an abrupt and significant decline in serum sodium ina patient presenting with a skin infection). The emer-gency physician assigned an adverse drug event severitycategory at the point-of-care: A severe event causeddeath or required admission, a moderate event requireda change in medical management, and a mild event re-quired no change in therapy [22-24]. The categorizationsof adverse drug events by chief-complaint and severitywere independent.Table 1 Examples of ICD-10 codes indicating adverse drug events within each code categoryCodeCategoryDefinition Numberof codesin categoryExamplesCode Code description*A1 The ICD-10 code description includes the phrase“induced by medication/drug.”114 J70.2 Acute drug-induced interstitial lung disordersA2 The ICD-10 code description includes the phrase“induced by medication or other causes.”81 142.7 Cardiomyopathy due to drugs and other external agentsT88.7 Unspecified adverse event of drug or medicamentB1 The ICD-10 code dictionary includes the phrase “poison-ing by medication.”134 T36 Poisoning by systemic antibioticsB2 The ICD-10 code dictionary includes the phrase “poison-ing by or harmful use of medication or other causes.”19 X44 Accidental poisoning by, and exposure to, other andunspecified drugs, medicaments and biologicalsubstancesC§ Adverse drug event deemed to be very likely althoughthe ICD-10 code description does not refer to a drug.32 L51.2 Toxic epidermal necrolysisD§ Adverse drug event deemed to be likely although theICD-10 code description does not refer to a drug.86 N17 Acute renal failure with tubular necrosisE§ Adverse drug event deemed to be possible althoughthe ICD-10 code dictionary does not refer to a drug.85 K25 Gastric ulcerY66 Non administration of surgical and medical care*The code descriptions are from the ICD-10 coding dictionary [29].§The likelihood of the ICD-10 code indicating an adverse drug event was adapted from Stausberg et al., [27,28] and where missing, was determined by twoinvestigators who arrived at their ratings independently, and subsequently obtained consensus through discussion (CH and JS).Hohl et al. BMC Health Services Research 2013, 13:473 Page 3 of 11http://www.biomedcentral.com/1472-6963/13/473AnalysisWe used descriptive statistics to summarize the base-line characteristics of the patient population. We esti-mated the proportion of patients with at least oneadverse drug event code recorded in the administrativehospital or emergency department database amongthose who were diagnosed with one or more adversedrug events at the point-of-care. This was estimated asthe number of patients with recorded adverse drugevent codes in categories A1, A2, B1 or B2 in the ad-ministrative database, divided by the number of pa-tients with adverse drug events diagnosed at the point-of-care in the prospective database multiplied by 100.We estimated the proportion of false positives by divid-ing the number of patients with at least one adversedrug event code recorded in the administrative data-base and no adverse drug events identified at the point-of-care by the number of all patients without adversedrug events identified at the point-of-care, multipliedby 100. As the likelihood of an ICD-10 code identifyinga true adverse drug event varied, we expanded the nu-merator to include codes in categories C, D, or E.ResultsAdverse drug events identified at the point-of-careAmong 2289 patients who were approached for enrolment,1591 met the prospective study’s inclusion and exclusioncriteria (Figure 1) [21]. Among these, 17 were excluded be-cause they had multiple visits on the same day in the ad-ministrative data leading to unresolved linkages.The place of presentation was distributed unevenly be-tween the two emergency departments: 1152 (73.2%) pa-tients presented to VGH, and 422 (26.8%) to SPH. Theaverage age was 51.4 years, 51.1% were female, and the me-dian number of prescribed medications was two (Table 2).Among these patients, 221 (14%, 95% confidence interval12.4–15.8%) were diagnosed with 237 adverse drug eventsat the point–of–care (Table 3). One hundred and forty sixpatients (146/221; 66.1% 95% CI 59.6–72.0%) had one ad-verse drug event related to their chief–complaint, 60 (60/221; 27.1 95% CI 21.7–33.4%) had one incidentally foundevent, and 15 (15/221; 6.8% 95% CI 4.2–10.9%) had morethan one adverse drug event. The most common categoriesof adverse drug events resulted from adverse drug reac-tions (74/237; 33.5%; 95% CI 27.6–40.0%) (Table 3). Othercategories of adverse drug events were due to noncompli-ance/failure to receive a drug (62/237; 26.2%; 95% CI 21.0-32.1%), need to add a drug/untreated indication (35/237;14.8%; 95% CI 10.8-19.9%), subtherapeutic/low dose (27/237; 11.4%; 95% CI 8.0-16.1%), supratherapeutic/high dose(20/237; 8.4%; 5.5-12.7 95% CI), wrong drug (15/237;6.3%; 95% CI 3.9-10.2%) and drug use without indication(4/237; 1.7%; 95% CI 0.7-4.2%). No adverse drug eventswere attributed to drug interactions.Most adverse drug events (226/237; 95.4%; 95% CI91.9.0–97.4%) were deemed at least moderate in severity,requiring a change in medical management, consultation,hospital admission, or were life-threatening. The mostcommonly implicated medications were acetaminophenwith codeine (15 events), warfarin (15 events) and pheny-toin (9 events).•••••••Figure 1 Patient flow.Hohl et al. BMC Health Services Research 2013, 13:473 Page 4 of 11http://www.biomedcentral.com/1472-6963/13/473Table 2 Characteristics of 1,574 patients presenting to the emergency departmentCharacteristic All patients(n = 1574)With chief complaintrelated ADEWith incidentallyfound ADE onlyWithout ADEs(n = 158) (n = 63) (n = 1,353)Age, yrs, mean (SD) 51.4 (20.3) 50.8 (20.8) 63.5 (19.7) 50.9 (20.1)Female 805 (51.1%) 76 (48.1%) 26 (41.3%) 703 (52.0%)Arrived from, no. (%)†Home 1404 (89.2%) 131 (82.9%) 56 (88.9%) 1217 (89.9%)Homeless/Shelter 64 (4.1%) 14 (8.9%) 2 (3.2%) 48 (3.5%)Nursing home 65 (4.1%) 9 (5.7%) 3 (4.8%) 53 (3.9%)Other 39 (2.5%) 4 (2.5%) 2 (3.2%) 33 (2.4%)Canadian triage acuity score, no. (%)1 6 (0.4%) 2 (1.3%) 0 (0.0%) 4 (0.3%)2 225 (14.3%) 20 (12.7%) 10 (15.9%) 195 (14.4%)3 693 (44.0%) 77 (48.7%) 36 (57.1%) 580 (42.9%)4 579 (36.8%) 50 (31.6%) 14 (22.2%) 515 (38.1%)5 71 (4.5%) 9 (5.7%) 3 (4.8%) 59 (4.4%)Most common chief complaints, no. (%)Abdominal pain 157 (10.0%) 4 (2.5%) 9 (14.3%) 144 (10.6%)Chest pain 115 (7.3%) 4 (2.5%) 4 (6.4%) 107 (7.9%)Shortness of breath 96 (6.1%) 14 (8.9%) 5 (7.9%) 77 (5.7%)Lower extremity pain 79 (5.0%) 4 (2.5%) 2 (3.2%) 73 (5.4%)Back pain 64 (4.1%) 4 (2.5%) 0 (0.0%) 60 (4.4%)No. comorbid conditions, mean (SD) 2.1 (2.0) 2.7 (2.2) 3.3 (1.8) 1.9 (2.0)Most prevalent comorbid conditions, no. (%)Hypertension 382 (24.3%) 43 (27.2%) 36 (57.1%) 303 (22.4%)Mood disorder 201 (12.8%) 35 (22.2%) 7 (11.1%) 159 (11.8%)Dyslipidemia 125 (7.9%) 15 (9.5%) 5 (7.9%) 105 (7.8%)Asthma 120 (7.6%) 20 (12.7%) 6 (9.5%) 94 (6.9%)Diabetes Mellitus 119 (7.6%) 22 (13.9%) 9 (14.3%) 88 (6.5%)No. prescribed medications, median (IQR) 2 (1,5) 4 (2,7) 4 (2,8) 2 (1,5)Most commonly prescribed outpatient medications, no. (%)Acetaminophen with codeine 195 (12.4%) 29 (18.4%) 8 (12.7%) 158 (11.7%)Ramipril 134 (8.5%) 12 (7.6%) 7 (11.1%) 115 (8.5%)Salbutamol 123 (7.8%) 16 (10.1%) 8 (12.7%) 99 (7.3%)Rabeprazole 117 (7.4%) 12 (7.6%) 10 (15.9%) 95 (7.0%)Lorazepam 106 (6.7%) 7 (4.4%) 6 (9.5%) 93 (6.9%)Disposition from the ED, no. (%)‡Admitted 283 (18.0%) 40 (25.3%) 20 (31.7%) 223 (16.5%)Home 1279 (81.3%) 113 (71.5%) 42 (66.7%) 1124 (83.1%)Deceased 3 (0.2%) 0 (0.0%) 0 (0.0%) 3 (0.2%)Other 7 (0.4%) 5 (3.2%) 1 (1.6%) 1 (0.1%)Abbreviations: ADE = adverse drug event, SD = standard deviation, IQR = interquartile range,.†There were two patients who arrived from an unknown location.‡There were two patients with unknown disposition from the ED.Hohl et al. BMC Health Services Research 2013, 13:473 Page 5 of 11http://www.biomedcentral.com/1472-6963/13/473Proportion of adverse drug events reported in theadministrative databases with codes clearly linking theevent to a culprit medicationWe found ICD-10 codes that clearly identified an ad-verse drug event (categories A1, A2, B1 and B2) in 15of 221 records of patients diagnosed with one or moreadverse drug events at the point–of–care (Table 4),corresponding to a sensitivity of 6.8% (95% CI 4.0–11.2%). This code set identified 18 of 1353 records asfalse positive (1.3%; 95% CI 0.8-2.1%), corresponding toa specificity of 98.7% (95% CI 97.9-99.2%). The positivepredictive value of the code set was 45.5% (95% CI 28.1–63.7%), and its negative predictive value 86.6% (95% CI84.8 – 88.3%).Two of 59 adverse drug reactions (3.4%; 95% CI 0.4–11.7%) and 3 of 22 severe adverse drug events (13.6%;95% CI 2.9–34.9%) were identified as medication-related in the administrative data with code categoriesA1, A2, B1 or B2, among patients with only one event.Among patients admitted to hospital from the emer-gency department, 18.2% of adverse drug events wereidentified with an ICD-10 code clearly linking the eventto medication use (Table 5). We found ICD-10 codesthat clearly linked a culprit medication to an adversedrug event in 14 of 158 (8.9%, 95% CI 5.1–14.7%) re-cords of patients presenting with a chief complaint-related adverse drug event.Adverse drug events identified in the administrative datawith codes indicating a very likely, likely or possiblerelationship to a medicationWhen the ICD-10 code categories were broadened to in-clude codes that very likely, likely or possibly indicatedan adverse drug event, we were able to identify 62 codesfor adverse drug events in the 221 records of patients di-agnosed with one or more adverse drug events at thepoint–of–care (Table 4). This corresponded to a sensi-tivity of 28.1% (95% CI 22.3–34.6%) for the broader codeset. The positive predictive value of the code set was27.2% (95% CI 21.5 – 33.5%), and its negative predictivevalue 88.2% (95% CI 86.3 – 89.9%).Using the broader code set we identified 23 of 59 ad-verse drug reactions (39.0%; 95% CI 26.6–52.6%), andseven of 22 severe adverse drug events (31.8%; 95% CI13.9–54.9%) among patients with one event only.Among admitted patients, 54.6% of adverse drug eventswere identified with an ICD-10 code indicating a pos-sible, likely, or very likely adverse drug event, or clearlylinking the event to medication use (Table 5). Thebroader code categories identified 45 of 158 (28.5%,21.7%–36.3%) adverse drug events related to the chief-complaint. This code set incorrectly identified 166 of1353 records as false positive (12.3%; 95% CI 10.6-14.1%), corresponding to a specificity of 87.7% (95% CI85.9-89.4%).Table 3 Characteristics of 237 adverse drug eventsidentified at the point-of-care in 1574 emergency depart-ment patientsCharacteristic No. (%) of eventsChief complaint-relatedIncidentallyfound(n = 158) (n = 79)TypeAdverse drug reaction 48 (30.4) 26 (32.9)Drug interactions 0 (0.0) 0 (0.0)Drug use without indication 3 (1.9) 1 (1.3)Supratherapeutic/high dose 12 (7.6) 8 (10.1)Subtherapeutic/low dose 13 (8.2) 14 (17.7)Need to add drug/untreatedindication25 (15.8) 10 (12.7)Noncompliance/failure toreceive drug45 (28.5) 17 (21.5)Wrong drug 12 (7.6) 3 (3.8)SeveritySevere 21 (13.1) 3 (3.8)Moderate 131 (82.9) 71(89.9)Mild 6 (3.8) 5 (6.3)PreventabilityPreventable 118 (74.7) 60 (75.9)Non-preventable 40 (25.3) 19 (24.1)Table 4 Patients with records in the administrative data indicating an adverse drug event among patients withadverse drug events identified at the point-of-care in the emergency department, by category of diagnostic ICD-10code and type of adverse drug eventICD-10 code category* Any ADE Chief complaint-related Incidentally found(n = 221) (n = 158)* (n = 63)**Induced or related to a medication (%; 95% CI) 15 (6.8; 4.0–11.2) 14 (8.9; 5.1–14.7) 1 (1.6; 0.1–9.7)Induced or related to a medication,very likely,likely or possible ADE (%; 95% CI)62 (28.1; 22.3–34.6) 45 (28.5; 21.7 – 36.3) 17 (27.0; 16.9–39.9)Abbreviations: ICD = international classification of disease; ADE = adverse drug event, CI = confidence interval.* “Chief complaint-related” refers to patients with at least one chief complaint-related ADE.** “Incidentally found” refers to patients with incidentally found ADEs only.Hohl et al. BMC Health Services Research 2013, 13:473 Page 6 of 11http://www.biomedcentral.com/1472-6963/13/473The most common culprit medications of all identifiedadverse drug events (categories A1–E) were olanzapine(five events), and warfarin, phenytoin, vancomycin, gly-buride, clopidogrel and aspirin (two events each).DiscussionOur objective was to determine the proportion of ad-verse drug events to outpatient medications resulting inemergency department visits that were reported in ad-ministrative data. We found adverse drug events to beunderreported in the administrative data of two largeCanadian university hospitals, despite using an extensivelist of ICD-10 codes. Even when a broad set of ICD-10codes was used to include diagnoses indicating a verylikely, likely or possible relationship with a medication,we were only able to identify 28% of adverse drugevents, and 39% of adverse drug reactions. Among pa-tients who required hospital admission, we were able toidentify 55% of adverse drug events.Prescribing medication is the most common medicalintervention performed by physicians. Yet, many prescrib-ing decisions are informed by incomplete or conflictingevidence, or by the results of randomized trials thatmay not be transferrable to clinical practice [7,34,35]. Inaddition, off-label use of medications and varying compli-ance behavior of patients contribute to suboptimal treat-ment outcomes, leading to a growing interest in developingimproved methods to capture adverse drug event data fromthe real-world to generate more robust estimates about thecomparative safety and effectiveness of medications, and todevelop interventions to improve patient care [6-8,17,18].In North America, emergency departments offer themajority of healthcare delivered for acute and unexpectedmedical conditions, including adverse drug events [9].Therefore, emergency department administrative datamay offer unique opportunities to capture data on clinic-ally significant adverse drug events that result from out-patient medication use [9,11-13]. However, before suchdata are considered for this purpose, they should be evalu-ated for their completeness.Our study is the first in the peer-reviewed literature tocompare adverse drug event reports in administrative re-cords of emergency department patients with adversedrug events diagnosed at the point-of-care. Prior studieshave attempted to validate adverse drug event codes inadministrative data by comparing adverse drug event re-ports in administrative data to events identified by chartreview, using electronic trigger methods or between ad-ministrative databases [36-41]. Our study differs fromthese studies in the premise that all clinically significantadverse drug events are recorded in the medical recordor identifiable using trigger methods. Indeed, two priorstudies support the assumption that 40% of adverse drugevents may not be documented in emergency depart-ment records [33,42]. Therefore, in order to understandthe sensitivity of administrative data and the ICD-10code set, we derived our criterion standard at the point-of care using a pre-defined algorithm that included as-sessment by a pharmacist and a physician. We believethat this led to more precise estimates of adverse drugevents. We disclosed all adverse drug events suspectedin the emergency department to treating physicians (re-quired by Ethics to ensure optimal patient care) prior tocoding, thus optimizing the chances of their documenta-tion in the medical chart.A few studies have examined the sensitivity of adversedrug event codes within the ICD-9 coding system for eventsthat occurred as a result of inpatient medications [38,39].Hougland et al. found that their code set detected moreevents than the hospital’s computerized adverse drug eventsurveillance system, and estimated that 55% of adverse drugevents causing hospitalization, and 10% of adverse drugevents occurring during the course of hospitalization wereidentified when compared to medical record review [38].Leonard et al. found that the sensitivity of their ICD-9 codeset varied substantially by the type of adverse drug reactionthey searched for, and estimated that the sensitivity of thecodes for digoxin and phenytoin related events may be 84%and 86.7% respectively [39]. However, the authors deter-mined the criterion standard retrospectively by chart reviewin only 19-40% of records, all of which had been were pre-screened using an ICD-9 code set that included the adversedrug reaction codes [39]. This may have falsely elevated thesensitivity of their code set, because the determination ofthe criterion standard was not independent of the code setthey used to identify events.Table 5 Patients with records in the administrative data indicating an adverse drug event among patients withadverse drug events identified at the point-of-care in the emergency department, by category of diagnostic code andadmission status*ICD-10 code category Admitted Discharged(n = 66) (n = 146)Induced or related to a medication (%; 95% CI) 12 (18.2; 9.8–29.6) 3 (2.1; 0.4–5.9)Induced or related to a medication, very likely, likely or possible ADE (%; 95% CI) 36 (54.6; 41.8–66.9) 26 (17.8; 12.0–25.0)ADE = adverse drug event; CI = confidence interval.*Nine patients did not have admission status reported in the administrative data.Hohl et al. BMC Health Services Research 2013, 13:473 Page 7 of 11http://www.biomedcentral.com/1472-6963/13/473Only one previous study has evaluated the sensitivityof emergency department data coded in ICD-10 for ad-verse drug reactions [41]. Wu et al. used CIHI data tocompare the emergency department discharge diagnosiswith the admitting diagnosis among patients who wereadmitted to the hospital through emergency depart-ments. The authors’ premise was that in patients admit-ted to hospital through the emergency department for adiagnosis of an adverse drug reaction, the patient’s emer-gency department discharge diagnosis and hospital ad-mitting diagnosis should be the same, if adverse drugreactions are appropriately identified, recorded andcoded. Using an ICD-10 code set containing 245 codesincluding the external cause codes Y40-59, Wu et al.found that 15% of emergency department visits for adversedrug reactions leading to hospital admission were codedwith the corresponding admitting diagnosis in CIHI. Incomparison, in our study including all emergency depart-ment patients (not just those admitted to hospital), wewere only able to identify 3.4% of adverse drug reactionsin the administrative data using our “narrower” code setcontaining ICD-10 codes categorized as A1, A2, B1 andB2. We believe that this large difference in our estimateof the degree of underreporting may be due to Wu et al.’scomparison of adverse drug reactions coded within oneset of administrative data (NACRS) to another (the Dis-charge Abstract Database), as opposed to our compari-son with a prospective standard. This indicates thatadverse drug event reporting may be overestimatedwhen reporting is evaluated by comparing between twoadministrative databases.The strengths of our study include a rigorous assess-ment of adverse drug events at the point-of-care beforeany administrative coding occurred. Both a clinicalpharmacist and a treating emergency physician assessedall patients in our cohort. The clinical pharmacists in ourstudy evaluated patients independently from physicians,and took their own medical histories, contributing to theaccuracy of the available medication information ratherthan relying on retrospective chart review. Pharmacistsdocumented any suspected adverse drug events in the pa-tients’ records and informed physicians of all potentiallymissed cases. All cases in which the pharmacists’ and phy-sicians’ assessments of adverse drug events were discord-ant or uncertain were reviewed and adjudicated by anindependent committee consisting of a clinical pharmacistand a medical toxicologist. Another strength of our studyincludes having conducted a literature review to identifyadverse drug event codes in the ICD-10 coding system, re-ducing the possibility that we underestimated the capacityof the ICD-10 coding system to identify adverse drugevents by using too narrow of a code set [26].The operational definition of adverse drug events re-mains problematic, as several interpretations of its mostcommon definition “harm caused by the use of a drug”exist [30,31]. We approached our case definition of ad-verse drug events from the health services research per-spective, in which the utilization of the emergencydepartment leading to bed occupancy and incurring costwas the primary end point. Thus, all our cases were as-sociated with an emergency department visit, and we didnot capture any “harm” or injury” from illnesses not as-sociated with an emergency department visit. Despitethis, not all of the events captured in our study will beof interest from a pharmacovigilance or regulatory bodyperspective. From the latter perspective, adverse drug re-actions, a subset of adverse drug events, are most rele-vant. Examples of events falling into our case definitionthat may not be relevant from a pharmacovigilance orregulatory body perspective were the following: the needto add a drug/untreated indication (e.g., lack of anticoa-gulation therapy leading to stroke in a patient with apreviously established diagnosis of atrial fibrillation, ahigh CHADS2 score and previous documentation of theneed for anticoagulation), too high or too low dose (e.g.,a reduction in furosemide dose leading to pulmonaryedema in a patient with previously controlled congestiveheart failure and no alternate explanation), noncompli-ance/failure to receive a drug (e.g., noncompliancewith insulin leading to diabetic ketoacidosis) or wrongdrug (e.g., a patient with type II diabetes mellitus withrecurrent episodes of hypoglycemia on glyburide). Wedeemed the inclusion of these types of events importantfrom a health services research perspective, as thesetypes of events have previously been associated with in-creased health services utilization and cost, [43] andmany were classified as preventable [21]. From a patient,clinician and system perspective, the development ofmethods to identify and monitor these types of events isdesirable to generate a factual basis for generating hy-potheses about their prevention and to inform healthpolicies to reduce their occurrence. These may includespecific actions related to prescribing, administering ormonitoring of high-risk medications, or actions targetingspecific patient groups. Data on the extent of occurrenceand associated burden of events can be used to prioritizeactions in a resource-constrained environment to targetcommonly occurring preventable and costly events.For example, through a recently implemented adversedrug event screening program in the Vancouver CostalHealth Authority, through which detailed regional ad-verse drug event data are collected, our group identifiedthat a large proportion of emergency department visitscan be attributed to supratherapeutic/high warfarin dosewithout any associated bleeding. Identifying the etiologiccause of these visits is informing the development ofspecific preventative policies within the Health Author-ity, as well as the discourse between primary care andHohl et al. BMC Health Services Research 2013, 13:473 Page 8 of 11http://www.biomedcentral.com/1472-6963/13/473acute care in terms of the etiology of outpatient adversedrug events and measures for prevention.In order to ensure that we did not apply too broad ofa case definition of adverse drug events, we put mecha-nisms in place to ensure that events that could be ex-plained by the exacerbation of the patient’s underlyingdisease or by alternate diagnoses were excluded. Thesemechanisms included capturing the physician’s workingdiagnosis, mandating the use of causality algorithmsand using an independent adjudication committee. Wedid not consider the failure to use drug in the first place asan adverse drug event, unless the drug had clearly beendocumented as being indicated in the patient’s medicalrecord. We put these safeguards in place, as adopting toobroad of a case definition and overcalling cases as adversedrug events that might not be, may risk promoting in-appropriate use of non-medicinal therapies, and may notserve to promote prudent and rational medication use.Our study is not without limitations. First, we consid-ered all adverse drug events that had been reported ineither primary or secondary diagnostic codes, becausesome patients presented to the emergency departmentwith more than one adverse drug event, one of whichmay have been coded under a secondary diagnosis field.Also, some patients may have been diagnosed with morethan one diagnosis in the emergency department, one ofwhich was deemed the primary reason for presentationor admission. Therefore, we did not exclude adversedrug event codes that were coded in secondary diagnosisfields in order to avoid underestimating the sensitivity ofthe administrative dataset and the ICD-10 code set.However, this means that we may have picked up ad-verse drug events that resulted from in-hospital treat-ment rather than from outpatient medications. Thiswould have resulted in an overestimation of the sensitiv-ity of the ICD-10 codes. Second, because of the cost andlabor involved in establishing a prospective standard foradverse drug events, our sample size is limited. Thus,our study should be regarded as preliminary. Third, ourresults reflect two Canadian institutions and may not begeneralizable to other institutions. Fourth, we expandedour code set to include possible adverse drug events(code categories C, D and E), to allow for better ICD-10data capture which resulted in a greater proportion offalse positives. Fifth, our results may have been influ-enced by the existing variation in the use of the termin-ology surrounding adverse drug events [31]. It ispossible that physicians were less likely to record (andcoders less likely to code) events that they personally feltshould not be considered drug-related, even though thepresentation met our outcome definition. Finally, wewish to clarify why the number of events listed in thisstudy differs from its parent study [21]: The prospectivedata used for this study was derived from a prospectiveobservational clinical decision rule derivation study inwhich we collected data on all outcomes (n = 221). Thepurpose of the parent study was to derive clinical deci-sion rules to aid health care workers at the point-of-careto identify patients with a broad range of adverse drugevents. Yet, this was not possible, likely due to the het-erogeneity of the events. Therefore, as stated a priori inthe protocol of our parent study, we proceeded to deriveclinical decision rules for two narrower categorizationsof adverse drug events. Thus, the clinical decision rulederivation study represents a subset of the events ana-lyzed in the present study.ConclusionWe found adverse events to outpatient medications result-ing in emergency department visits to be underreported inexisting administrative data of two large Canadian tertiarycare hospitals. The performance characteristics of the codesets examined, in terms of their sensitivity, specificity, posi-tive and negative predictive values, indicate that adminis-trative data alone may not be appropriate as a stand-alonemeans of identifying adverse drug events in these data.This study may serve as a point of reference for futurework in this area, considering the paucity of the literatureon evaluating the ICD-10 system and emergency depart-ment administrative data for adverse drug event reports.Future research on differential reporting of outpatient ad-verse drug events in administrative data would be useful togain a better understanding of those events for which ad-ministrative data may have a greater sensitivity [38]. Finally,despite the cost and labor involved in establishing adversedrug event surveillance systems using prospective data, ac-tive case finding methods may result in more completeand accurate estimates compared with administrative data.Additional filesAdditional file 1: ADE Evaluation Algorithm at the Point-of-Care.In the ED each patient was evaluated by a clinical pharmacist and thetreating emergency physician independently, and blinded to each other’sevaluations. The ratings were combined while the patient was still in theED. If there was any disagreement about the rating (i.e., yes/no, yes/uncertain,no/uncertain, etc.), or if either or both of the evaluations were uncertain, thecase proceeded to independent adjudication by a committee consisting of apharmacist and physician not in any other way involved in the study.Additional file 2: Adverse drug events were diagnosed only inpatients presenting to the emergency department with untowardand unintended symptoms, signs or abnormal laboratory valuesthat arose from appropriate or inappropriate medication use. Eventsmeeting this case definition were categorized according to the taxonomyof drug-related problems [25]. For all events that were categorized asdue to “Need to Add Drug/Untreated Indication” or “Failure to Receive aDrug/Noncompliance” a pre-existing diagnosis had to have been docu-mented prior to the emergency department visit. Failure to use a drug inthe first place was not considered an adverse drug event. Asymptomaticdrug-related problems were not captured. Adverse drug reactions weredefined according to the World Health Organization [31].Additional file 3: See code set provided in an Excel spreadsheet.Hohl et al. BMC Health Services Research 2013, 13:473 Page 9 of 11http://www.biomedcentral.com/1472-6963/13/473Competing interestsNone of the authors have identified competing financial or non-financial interests.Authors’ contributionsCH, BS and JS have made substantial contributions to the conception anddesign of the study, CH and EY aquired the data, LK and BR analyzed thedata, and all authors interpreted the results of data analysis. CH, LK, BSand JS were involved in drafting and critically reviewing the manuscript.All authors have approved the manuscript in its current form.Role of the funding sourceThis study was supported by grants from the Canadian Patient SafetyInstitute, the Michael Smith Foundation for Health Research and theVancouver Coastal Health Authority. None of the sponsors had any role instudy design, data collection or processing, analysis or preparation of themanuscript. All authors had access to the study data and agreed to submitthe manuscript in its current form. Dr Hohl is supported by a NewInvestigator Award from the Canadian Institutes of Health Research.Author details1Department of Emergency Medicine, University of British Columbia, 855West 12th Avenue, Vancouver, BC V5Z 1 M9, Canada. 2Department ofEmergency Medicine, Vancouver General Hospital, 855 West 12th Avenue,Vancouver, BC V5Z 1 M9, Canada. 3Centre for Clinical Epidemiology &Evaluation, Vancouver Coastal Health Research Institute, 828 West 10th,Vancouver, BC V5Z 1 M9, Canada. 4Department of Statistics, University ofBritish Columbia, 900 West 10th Ave, Vancouver, BC V5Z 1 M9, Canada.5InstitutfürMedizinischeInformationsverarbeitung, Biometrie undEpidemiologie, Ludwig-Maximilians-UniversitätMünchen, München, Germany.6School of Population and Public Health, University of British Columbia,Vancouver, BC, Canada.Received: 23 October 2012 Accepted: 4 November 2013Published: 12 November 2013References1. Bates D: Drugs and adverse drug reactions. How worried should We Be?JAMA 1998, 279(15):1216–1217.2. Patterns of medication use in the United States 2004: a report from the slonesurvey. http://www.bu.edu/slone/files/2012/11/SloneSurveyReport2006.pdf.3. Gurwitz JH, Field TS, Harrold LR, Rothschild J, Debellis K, Seger AC, Cadoret C,Fish LS, Garber L, Kelleher M, et al: Incidence and preventability of adversedrug events among older persons in the ambulatory setting. JAMA 2003,289(9):1107–1016.4. Sorensen SV, Dewilde S, Singer DE, Goldhaber SZ, Monz BU, Plumb JM:Cost-effectiveness of warfarin: trial versus “real-world” stroke preventionin atrial fibrillation. Am Heart J 2009, 157:1064–1073.5. T-Pv S, Leufkens HG, Zhang B, Smeeth L: A comparison of costeffectiveness using data from randomized trials or actual clinical practice:selective Cox-2 inhibitors as an example. PLoS Med 2009, 6(12):e1000194.6. Wiktorowicz M, Lexchin J, Paterson M, Mintzes B, Metge C, Light D, MorganS, Holbrook A, Tamblyn R, Zaki E, et al: Research networks involved inpost-market pharmacosurveillance in the United States, United Kingdom,France, New Zealand, Australia, Norway and European Union: Lessonsfor Canada. In Edited by the Canadian Institute for Patient Safety.Edmonton, Canada: The Canadian Institute for Patient Safety; 2008.7. Developing a Protocol for Observational Comparative EffectivenessResearch (OCER): A User’s Guide (XXX under Contract No. XXX HHSA.).http://www.effectivehealthcare.ahrq.gov/ehc/products/440/1067/AHRQ_CER_Protocol_User_s_Guide_DRAFT-COPY_AllChapters.pdf.8. Suissa S, Garbe E: Primer: administrative health databases in observationalstudies of drug effects—advantages and disadvantages. Nat Clin PractRheum 2007, 3(12):725–732.9. Schuur JD, Venkatesh AK: The growing role of emergency departments inhospital admissions. N Engl J Med 2012, 367:391–393.10. Hafner JW Jr, Belknap SM, Squillante MD, Bucheit KA: Adverse drug eventsin emergency department patients. Ann Emerg Med 2002, 39(3):258–267.11. Hohl CM, Dankoff J, Colacone A, Afilalo M: Polypharmacy, adverse drug-related events, and potential adverse drug interactions in elderly pa-tients presenting to an emergency department. Ann Emerg Med 2001,38(6):666–671.12. Zed PJ, Abu-Laban RB, Balen RM, Loewen PS, Hohl CM, Brubacher JR, Wilbur K,Wiens MO, Samoy LJ, Lacaria K, Pursell RA: Incidence, severity and preventabi-lity of medication-related visits to the emergency department: a prospectivestudy. Can Med Ass J 2008, 178(12):1563–1569.13. Budnitz DS, Lovegrove MC, Shehab N, Richards CL: Emergency Hospitalizationsfor Adverse Drug Events in Older Americans. N Engl J Med 2011, 365:2002–2012.14. WHO: Nternational drug monitoring: the role of national centres. Report of aWHO meeting. Geneva: World Health Organization; 1972:1–44.15. Zed PJ: Drug-related visits to the emergency department. J Pharm Pract2005, 18:329–335.16. Budnitz DS, Pollock DA, Weidenbach KN, Mendelsohn AB, Schroeder TJ,Annest JL: National surveillance of emergency department visits foroutpatient adverse drug events. JAMA 2006, 296(15):1858–1866.17. Carleton B, Foerster V, Warren L, Smith A: Overview of novel drug plan anddrug regulatory pharmacosurveillance initiatives in the United States,United Kingdom, and select other jurisdictions: A background paperprepared for the Working Conference on Strengthening the Evaluation ofReal World Drug Safety and Effectiveness. September 13-15, 2005.18. Keeping an eye onprescription drugs, keeping Canadians safe: activemonitoring systems for drug safety and effectiveness in Canada andinternationally. http://publications.gc.ca/collections/collection_2011/ccs-hcc/H174-21-2010-eng.pdf.19. Budnitz DS, Pollock DA, Mendelsohn AB, Weidenbach KN, McDonald AK,Annest JL: Emergency department visits for outpatient adverse drugevents: demonstration for a national surveillance system. Ann Emerg Med2005, 45(2):197–206.20. Jhung MA, Budnitz DS, Mendelsohn AB, Weidenbach KN, Nelson TD, PollockDA: Evaluation and Overview of the National Electronic InjurySurveillance System-Cooperative Adverse Drug Event Surveillance Pro-ject (NEISS-CADES). Med Care 2007, 45(10):S96–S102.21. Hohl CM, Brubacher J, Hunte G, Wiens MO, Abu-Laban R, Singer J, Sheps S:Clinical Decision Rules to Improve the Detection of Adverse Drug Eventsin Emergency Department Patients. Acad Emerg Med 2012, 19(6):640–649.22. Forster A, Asmis T, Clark H, Al Saied G, Code C, Caughey S, Baker K,Watters J, Worthington J, van Walraven C: Ottawa Hospital PatientSafety Study: incidence and timing of adverse events in patientsadmitted to a Canadian teaching hospital. Can Med Ass J 2004,170(8):1235–1240.23. Naranjo CA, Busto U, Sellers EM, Sandor P, Ruiz I, Roberts EA, Janecek E,Domecq C, Greenblatt DJ: A method for estimating the probability ofadverse drug reactions. Clin Pharmacol Ther 1981, 30(2):239–245.24. World Health Organization: Letter MIO/372/2(A). Geneva, Switzerland: WHO; 1991.25. Hepler C, Strand L: Opportunities and responsibilities in pharmaceuticalcare. Am J Hosp Pharm 1990, 47:533–543.26. Karpov A, Stausberg J, Doyle-Waters M, Hohl CM: Adverse drug eventcodes in ICD-10: a systematic review of the literature. In press.27. Stausberg J, Hasford J: Identification of adverse drug events: the use ofICD-10 coded diagnoses in routine hospital data. DtschArztebl Int 2010,107(3):23–29.28. Stausberg J, Hasford J: Drug-related admissions and hospital-acquired ad-verse drug events in Germany: a longitudinal analysis from 2003 to 2007of ICD-10-coded routine data. BMC Health Serv Res 2011, 11:134.29. International statistical classification of diseases and related health problems10th revision. http://apps.who.int/classifications/icd10/browse/2010/en.30. Edwards IR, Aronson JK: Adverse drug reactions: definitions, diagnosis,and management. Lancet 2000, 356(9237):1255–1259.31. Nebeker J, Barach P, Samore M: Clarifying adverse drug events: a clinician’sguide to terminology, documentation, and reporting. Ann Intern Med2004, 140:795–801.32. WHO: International drug monitoring: the role of the hospital. Report of aWHO meeting. WHO Technical Report Series 1966, 425:5–24.33. Hohl CM, Robitaille C, Lord V, Dankoff J, Colacone A, Pham L, Berard A,Pepin J, Afilalo M: Emergency physician recognition of adversedrug-related events in elder patients presenting to an emergencydepartment. Acad Emerg Med 2005, 12(3):197–205.34. Glasgow RE, Magid DJ, Beck A, Ritzwoller D, Estabrooks PA: Practical clinicaltrials for translating research to practice design and measurementrecommendations. Med Care 2005, 43:551–557.35. Tunis SR, Stryer DB, Clancy CM: Practical clinical trials. Increasing the valueof clinical research for decision making in clinical and health policy.JAMA 2003, 290(12):1624–1632.Hohl et al. BMC Health Services Research 2013, 13:473 Page 10 of 11http://www.biomedcentral.com/1472-6963/13/47336. Verelst S, Jacques J, Heede KV, Gillet P, Kolh P, Vleugels A, Sermeus W:Validation of hospital administrative dataset for adverse event screening.Qual Saf Health Care 2010, 19:e25.37. Hougland P, Nebeker J, Pickard S, Tuinen MV, Masheter C, Elder S, WilliamsS, Xu W: Using ICD-9-CM codes in hospital claims data to detect adverseevents in patient safety surveillance, Volume 1. Agency for HealthcareResearch and Quality: Rockville (MD); 2008.38. Hougland P, Xu W, Pickard S, Masheter C, Williams SD: Performance ofinternational classification of diseases, 9th revision, clinical modificationcodes as an adverse drug event surveillance system. Med Care 2006,44:629–636.39. Leonard CE, Haynes K, Localio R, Hennessy S, Tjia J, Cohen A, Kimmel SE,Feldman HI, Metlay JP: Diagnostic E-codes for commonly used, narrowtherapeutic index medications poorly predict adverse drug events.J Clin Epidemiolog 2008, 61:561–571.40. Drösler S, Romano P, Wei L: Health Care Quality Indicators Project:Patient Safety Indicators Report 2009. In Organization for EconomicCo-operation and Development (OECD) Health Working Papers No 47.Edited by Head of Publications Service, OECD. Paris, France: Head ofPublications Service, OECD; 2009:1–47.41. Wu C: Adverse Drug Reactions in the Emergency Department Population inOntario: Analysis of National Ambulatory Care Reporting System and DischargeAbstract Database 2003–2007. Toronto: University of Toronto; 2009.42. Hohl CM, Zed PJ, Brubacher JR, Abu-Laban RB, Loewen PS, Purssell R:Do emergency physicians attribute drug-related emergencydepartment visits to medication-related problems? Ann EmergMed 2010, 55(6):493–502.43. Hohl CM, Nosyk B, Zed P, Kuramoto L, Sobolev B, Brubacher J, Abu-Laban R,Loewen PS, Sheps S: Outcomes of emergency department patients pre-senting with adverse drug events. Ann Emerg Med 2011, 58(3):270–279.doi:10.1186/1472-6963-13-473Cite this article as: Hohl et al.: Evaluating adverse drug event reportingin administrative data from emergency departments: a validation study.BMC Health Services Research 2013 13:473.Submit your next manuscript to BioMed Centraland take full advantage of: • Convenient online submission• Thorough peer review• No space constraints or color figure charges• Immediate publication on acceptance• Inclusion in PubMed, CAS, Scopus and Google Scholar• Research which is freely available for redistributionSubmit your manuscript at www.biomedcentral.com/submitHohl et al. BMC Health Services Research 2013, 13:473 Page 11 of 11http://www.biomedcentral.com/1472-6963/13/473


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items