Methods for evaluating adverse drug event preventability in emergency department patients

UBC Faculty Research and Publications

Methods for evaluating adverse drug event preventability in emergency department patients Woo, Stephanie A.; Cragg, Amber; Wickham, Maeve E.; Peddie, David; Balka, Ellen; Scheuermeyer, Frank X.; Villanyi, Diane; Hohl, Corinne M.

Abstract

Background: There is a high degree of variability in assessing the preventability of adverse drug events, limiting the ability to compare rates of preventable adverse drug events across different studies. We compared three methods for determining preventability of adverse drug events in emergency department patients and explored their strengths and weaknesses. Methods: This mixed-methods study enrolled emergency department patients diagnosed with at least one adverse drug event from three prior prospective studies. A clinical pharmacist and physician reviewed the medical and research records of all patients, and independently rated each event’s preventability using a “best practice-based” approach, an “error-based” approach, and an “algorithm-based” approach. Raters discussed discordant ratings until reaching consensus. We assessed the inter-rater agreement between clinicians using the same assessment method, and between different assessment methods using Cohen’s kappa with 95% confidence intervals (95% CI). Qualitative researchers observed discussions, took field notes, and reviewed free text comments made by clinicians in a “comment” box in the data collection form. We developed a coding structure and iteratively analyzed qualitative data for emerging themes regarding the application of each preventability assessment method using NVivo. Results: Among 1356 adverse drug events, a best practice-based approach rated 64.1% (95% CI: 61.5–66.6%) of events as preventable, an error-based approach rated 64.3% (95% CI: 61.8–66.9%) of events as preventable, and an algorithm-based approach rated 68.8% (95% CI: 66.1–71.1%) of events as preventable. When applying the same method, the inter-rater agreement between clinicians was 0.53 (95% CI: 0.48–0.59), 0.55 (95%CI: 0.50–0.60) and 0.55 (95% CI: 0.49–0.55) for the best practice-, error-, and algorithm-based approaches, respectively. The inter-rater agreement between different assessment methods using consensus ratings for each ranged between 0.88 (95% CI 0.85–0.91) and 0.99 (95% CI 0.98–1.00). Compared to a best practice-based assessment, clinicians believed the algorithm-based assessment was too rigid. It did not account for the complexities of and variations in clinical practice, and frequently was too definitive when assigning preventability ratings. Conclusion: There was good agreement between all three methods of determining the preventability of adverse drug events. However, clinicians found the algorithmic approach constraining, and preferred a best practice-based assessment method.

Item Metadata

Title	Methods for evaluating adverse drug event preventability in emergency department patients
Creator	Woo, Stephanie A.; Cragg, Amber; Wickham, Maeve E.; Peddie, David; Balka, Ellen; Scheuermeyer, Frank X.; Villanyi, Diane; Hohl, Corinne M.
Contributor	Vancouver Coastal Health Authority. Research Institute; Vancouver Coastal Health Authority. Centre for Clinical Epidemiology and Evaluation
Publisher	BioMed Central
Date Issued	2018-12-04
Description	Background: There is a high degree of variability in assessing the preventability of adverse drug events, limiting the ability to compare rates of preventable adverse drug events across different studies. We compared three methods for determining preventability of adverse drug events in emergency department patients and explored their strengths and weaknesses. Methods: This mixed-methods study enrolled emergency department patients diagnosed with at least one adverse drug event from three prior prospective studies. A clinical pharmacist and physician reviewed the medical and research records of all patients, and independently rated each event’s preventability using a “best practice-based” approach, an “error-based” approach, and an “algorithm-based” approach. Raters discussed discordant ratings until reaching consensus. We assessed the inter-rater agreement between clinicians using the same assessment method, and between different assessment methods using Cohen’s kappa with 95% confidence intervals (95% CI). Qualitative researchers observed discussions, took field notes, and reviewed free text comments made by clinicians in a “comment” box in the data collection form. We developed a coding structure and iteratively analyzed qualitative data for emerging themes regarding the application of each preventability assessment method using NVivo. Results: Among 1356 adverse drug events, a best practice-based approach rated 64.1% (95% CI: 61.5–66.6%) of events as preventable, an error-based approach rated 64.3% (95% CI: 61.8–66.9%) of events as preventable, and an algorithm-based approach rated 68.8% (95% CI: 66.1–71.1%) of events as preventable. When applying the same method, the inter-rater agreement between clinicians was 0.53 (95% CI: 0.48–0.59), 0.55 (95%CI: 0.50–0.60) and 0.55 (95% CI: 0.49–0.55) for the best practice-, error-, and algorithm-based approaches, respectively. The inter-rater agreement between different assessment methods using consensus ratings for each ranged between 0.88 (95% CI 0.85–0.91) and 0.99 (95% CI 0.98–1.00). Compared to a best practice-based assessment, clinicians believed the algorithm-based assessment was too rigid. It did not account for the complexities of and variations in clinical practice, and frequently was too definitive when assigning preventability ratings. Conclusion: There was good agreement between all three methods of determining the preventability of adverse drug events. However, clinicians found the algorithmic approach constraining, and preferred a best practice-based assessment method.
Subject	Adverse drug event; Adverse drug reaction; Preventability; Preventable; Explicit review; Implicit review
Genre	Article
Type	Text
Language	eng
Date Available	2018-12-05
Provider	Vancouver : University of British Columbia Library
Rights	Attribution 4.0 International (CC BY 4.0)
DOI	10.14288/1.0375384
URI	http://hdl.handle.net/2429/67973
Affiliation	Medicine, Faculty of; Non UBC; Emergency Medicine, Department of; Population and Public Health (SPPH), School of; Other UBC
Citation	BMC Medical Research Methodology. 2018 Dec 04;18(1):160
Publisher DOI	10.1186/s12874-018-0617-4
Peer Review Status	Reviewed
Scholarly Level	Faculty
Copyright Holder	The Author(s).
Rights URI	http://creativecommons.org/licenses/by/4.0/
Aggregated Source Repository	DSpace

Open Collections

UBC Faculty Research and Publications