Applied Science, Faculty of
Nursing, School of
DSpace
UBCV
Recsky, Chantelle
Doctor of Philosophy - PhD
University of British Columbia
BACKGROUND: Healthcare organizations are investing in health information technology (HIT) and putting greater emphasis on care in primary and community care settings. While intended to support the delivery of care, HIT can also bring about unanticipated consequences including technology-mediated adverse events (TMAE) (i.e., unintended harm, or potential harm, that involved a HIT-based system). For example, information required for coordinating and delivering client care may be stored in two or more electronic record systems, which creates the potential for missing information, breakdowns in the continuity of care, and may even cause harm to clients. METHODS & RESULTS: A multi-study research project was conducted at a large organization in Western Canada, consisting of: 1) A scoping review of the literature on TMAE and harm in primary and community care settings, which revealed that the current body of evidence is under-developed; 2) A retrospective descriptive study of TMAE reported in primary and community care settings, which found TMAE are diverse and warrant consideration from a sociotechnical perspective; 3) An integrated knowledge translation (IKT) approach for co-creating and applying new knowledge to address TMAE, from which a process for safety huddles and rapid sociotechnical analysis of TMAE was co-produced; and 4) A qualitative descriptive study about how clinical informatics specialists perceived their experiences in managing TMAE, which determined that research using an IKT approach is beneficial to study TMAE, and safety huddles and rapid sociotechnical analysis of TMAE may help to improve care. CONCLUSION: Research in the area of HIT safety in primary and community care settings is under-developed, yet healthcare services are continuing to place greater emphasis on both technology and non-acute services. This research suggests that harm related to HIT can occur in these settings, demonstrated the sociotechnical nature of harm in this context, and found safety huddles and rapid sociotechnical analysis of TMAE to be a feasible method to address TMAE. Further research, incorporating multiple disciplinary perspectives and action-oriented strategies, is needed to gain insight into the impact of TMAE in primary and community care settings and take action to ensure HIT safety going forward.
CHARACTERIZATION AND MANAGEMENT OF TECHNOLOGY-MEDIATED ADVERSE EVENTS IN PRIMARY AND COMMUNITY CARE by Chantelle Recsky B.N., The University of Calgary, 2005 M.S.N, The University of British Columbia – Okanagan, 2009 A DISSERTATION SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Nursing) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) December 2021 © Chantelle Recsky, 2021 ii The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, the dissertation entitled: Characterization and Management of Technology-Mediated Adverse Events in Primary and Community Care submitted by Chantelle Recsky in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Nursing Examining Committee: Leanne M. Currie, Associate Professor, School of Nursing, UBC Supervisor Maura MacPhee, Professor, School of Nursing, UBC Supervisory Committee Member Kathy L. Rush, Professor, School of Nursing, UBC Supervisory Committee Member Dr. Heather L. O’Brien, Associate Professor, iSchool, UBC University Examiner Dr. Jason Sutherland, Professor, School of Population and Public Health, UBC University Examiner iii Abstract BACKGROUND: Healthcare organizations are investing in health information technology (HIT) and putting greater emphasis on care in primary and community care settings. While intended to support the delivery of care, HIT can also bring about unanticipated consequences including technology-mediated adverse events (TMAE) (i.e., unintended harm, or potential harm, that involved a HIT-based system). For example, information required for coordinating and delivering client care may be stored in two or more electronic record systems, which creates the potential for missing information, breakdowns in the continuity of care, and may even cause harm to clients. METHODS & RESULTS: A multi-study research project was conducted at a large organization in Western Canada, consisting of: 1) A scoping review of the literature on TMAE and harm in primary and community care settings, which revealed that the current body of evidence is under-developed; 2) A retrospective descriptive study of TMAE reported in primary and community care settings, which found TMAE are diverse and warrant consideration from a sociotechnical perspective; 3) An integrated knowledge translation (IKT) approach for co-creating and applying new knowledge to address TMAE, from which a process for safety huddles and rapid sociotechnical analysis of TMAE was co-produced; and 4) A qualitative descriptive study about how clinical informatics specialists perceived their experiences in managing TMAE, which determined that research using an IKT approach is beneficial to study TMAE, and safety huddles and rapid sociotechnical analysis of TMAE may help to improve care. CONCLUSION: Research in the area of HIT safety in primary and community care settings is under-developed, yet healthcare services are continuing to place greater emphasis on both iv technology and non-acute services. This research suggests that harm related to HIT can occur in these settings, demonstrated the sociotechnical nature of harm in this context, and found safety huddles and rapid sociotechnical analysis of TMAE to be a feasible method to address TMAE. Further research, incorporating multiple disciplinary perspectives and action-oriented strategies, is needed to gain insight into the impact of TMAE in primary and community care settings and take action to ensure HIT safety going forward. v Lay Summary Healthcare organizations are currently working to enhance primary and community care services. Health information technologies are essential to providing care in these areas, yet these technologies can also be cause for concern. For example, information required for coordinating and delivering client care may be stored in two or more electronic record systems, which creates the potential for missing information, breakdowns in the continuity of care, and may even cause harm to clients. To address this issue, a multi-study research project was conducted in partnership with a health information technology team at a large healthcare organization in Western Canada. Safety concerns involving technology were analyzed to learn more about how technology may contribute to harm, and a new process for rapidly handling safety events was co-created with the team. As a result, the team developed their ability to learn from safety concerns and use what they learn to improve care. vi Preface This dissertation consists of four studies and was performed using IKT techniques which included regular consultation with knowledge users. Human subjects ethics approval was obtained from the UBC Research Ethics Board [H18-02677] and operational approval was obtained from the Vancouver Coastal Health Research Institute [V18-02677]. The research plan was based on the candidate’s areas of interest (patient safety and health information technology) and expertise, as well as the needs of the organization. Organizational leaders from a regional health authority: Allison Muniak (AM), Megan Stowe (MS), and Lorraine Blackburn (LB), identified the target area of primary and community care and the doctoral candidate, Chantelle Recsky (CR), Dr. Leanne Currie (LC), Dr. Maura MacPhee (MM), and Dr. Kathy Rush (KR) determined the research methods. The scoping review (Study 1) was conceived of, carried out and written up by CR, with consultation from LC, AM, MS, and LB with the assistance of subject matter librarians from UBC, Katherine Miller, and VCH, Yvette Isparalexi. Study 2 was conceived of, carried out and written up by CR with MS participating in the data analysis and with consultation from LC. Preliminary data from Study 2 were presented as a poster at the American Medical Informatics Association Annual Symposium (Recsky et al., 2019). The IKT activities (Study 3) and qualitative study (Study 4) were led by CR in consultation with LC. This dissertation document was written by CR, in consultation with LC, with review and revision from the doctoral committee members MM and KR. vii Table of Contents Abstract ......................................................................................................................................... iii Lay Summary .................................................................................................................................v Preface ........................................................................................................................................... vi Table of Contents ........................................................................................................................ vii List of Tables ................................................................................................................................xv List of Figures ............................................................................................................................. xvi Acknowledgements ................................................................................................................... xvii Chapter 1: Introduction ................................................................................................................1 Introduction ..................................................................................................................... 1 Patient Safety .................................................................................................................. 2 Unintended Consequences of Health Information Technology ...................................... 3 Technology Mediated Adverse Events ........................................................................... 5 Characterizing Technology Mediated Adverse Events ................................................... 6 The Context of Primary and Community Care ............................................................... 6 Patient Safety in Primary and Community Care Settings ............................................... 7 Significance ..................................................................................................................... 9 Relevance to Nursing ...................................................................................................... 9 Aim and Research Questions ........................................................................................ 11 Overall Study Context ................................................................................................... 12 Integrated Knowledge Translation ................................................................................ 13 Overview of Dissertation .............................................................................................. 14 viii Chapter 2: Literature Review .....................................................................................................15 Introduction ................................................................................................................... 15 Health Information Technology and Patient Safety ...................................................... 15 Incidence and Antecedents of HIT Safety Events ........................................................ 16 Sociotechnical Systems Perspectives to Address HIT Safety ....................................... 18 A Framework for Diagnosing Technology-Induced Errors in Healthcare ........... 19 Classification of Problems Reported in Computer-related Safety Incidents ........ 20 Systems Engineering Initiative for Patient Safety Model ..................................... 21 Sociotechnical Model ............................................................................................ 21 Additional Models ................................................................................................ 24 Discussion of Sociotechnical Perspectives ........................................................... 25 Summary of the Literature Review ............................................................................... 27 Chapter 3: Study 1: Technology-mediated Adverse Events (TMAE) in Primary and Community Care Settings: A Scoping Review ..........................................................................28 Introduction ................................................................................................................... 28 Background ........................................................................................................... 28 Methods......................................................................................................................... 29 Research Questions ............................................................................................... 29 Identifying Relevant Studies ................................................................................. 30 Selecting Studies ................................................................................................... 31 Search Results and Selection of Literature ........................................................... 34 Charting the Data .................................................................................................. 37 Consultation with Stakeholders .................................................................................... 37 ix Results ........................................................................................................................... 38 Summary of Identified Literature ......................................................................... 41 Key Findings ......................................................................................................... 46 Types of Technology-related Errors ................................................................. 46 Risk Mitigation Strategies ................................................................................. 48 Consultation with Stakeholders .................................................................................... 50 Additional Organizational Challenges Identified by Stakeholders ....................... 51 Organizational Priorities Identified by Stakeholders ............................................ 53 Discussion ..................................................................................................................... 54 Key Findings ......................................................................................................... 54 Implications ........................................................................................................... 56 Directions for Future Research ............................................................................. 58 Limitations ............................................................................................................ 59 Conclusion .................................................................................................................... 60 Chapter 4: Study 2: Characterization of Technology-Mediated Adverse Event Reports in Primary and Community Care ...................................................................................................61 Introduction ................................................................................................................... 61 Limited Evidence Describing TMAE in Primary and Community Care Settings ........ 61 Safety Event Reporting ......................................................................................... 62 Methods......................................................................................................................... 63 Research Questions ............................................................................................... 64 Setting ................................................................................................................... 64 Local Safety Event Reporting ............................................................................... 65 x Inclusion and Exclusion Criteria ........................................................................... 66 Data Extraction and Analysis ................................................................................ 67 Sociotechnical Model ........................................................................................ 68 Sociotechnical Analysis .................................................................................... 68 Interrater Agreement and Interrater Reliability ................................................ 70 Results ........................................................................................................................... 71 Events by Setting, Type, and Degree of Harm ..................................................... 71 Events by Sociotechnical Dimensions .................................................................. 74 Discussion ..................................................................................................................... 80 Key Findings ......................................................................................................... 80 Capturing HIT-related Safety Issues ..................................................................... 81 Limitations of Voluntary Reporting Systems ....................................................... 82 Events in Primary and Community Care Settings ................................................ 84 Sociotechnical Dimensions ................................................................................... 84 Achieving Consensus in Coding Sociotechnical Dimensions .............................. 85 Risk of Harm in Primary and Community Care Settings ..................................... 86 Impact of Integrated Knowledge Translation ....................................................... 87 Directions for Future Research ............................................................................. 87 Limitations ............................................................................................................ 88 Conclusion .................................................................................................................... 90 Chapter 5: Study 3: Addressing TMAE with an IKT Approach ............................................91 Introduction ................................................................................................................... 91 Traditional Knowledge Translation .............................................................................. 91 xi Integrated Knowledge Translation ................................................................................ 92 IKT Competencies ........................................................................................................ 93 Embedded Researcher in Current Project ..................................................................... 94 Management of TMAE in Healthcare Organizations ................................................... 94 Gap in Evidence for IKT about TMAE ................................................................ 95 Research Project: Understanding and Mitigating TMAE ............................................. 95 Knowledge Users .................................................................................................. 96 The Role of the Embedded Researcher ................................................................. 97 Research Question ................................................................................................ 98 IKT Approach to Address Technology Mediated Adverse Events .............................. 98 Guiding Perspectives .......................................................................................... 101 Integrated Knowledge Translation Activities ..................................................... 103 Table 5.2 Integrated Knowledge Translation Overview ................................................. 106 CCI Team Process for Addressing TMAE ......................................................... 106 Analyses of TMAE Reports ............................................................................ 106 Safety Huddles for Learning from TMAE Analyses ...................................... 108 Sustainability................................................................................................... 110 Discussion ................................................................................................................... 110 Team Process for Addressing TMAE ................................................................. 110 IKT Approach to Address TMAE ....................................................................... 112 Circumstances of Collaboration ...................................................................... 112 Stance of Intentionality ................................................................................... 113 Bidirectional Knowledge Transfer and Andragogy ........................................ 114 xii Using Evidence to Inform Practice and Generate Knowledge ........................... 115 Limitations .......................................................................................................... 116 Directions for Future Research ........................................................................... 117 Conclusion .................................................................................................................. 118 Chapter 6: Study 4: “Workflow or Chaos or Whatever”: Team Perspectives about TMAE......................................................................................................................................................119 Introduction ................................................................................................................. 119 Methods....................................................................................................................... 119 Research Question .............................................................................................. 120 Sampling, Sample, and Recruitment ................................................................... 120 Data Collection ................................................................................................... 121 Data Analysis ...................................................................................................... 122 Rigour ................................................................................................................. 123 Results ......................................................................................................................... 124 Barriers ................................................................................................................ 125 Enablers ............................................................................................................... 129 Outcomes ............................................................................................................ 132 Discussion ................................................................................................................... 140 Key Findings ....................................................................................................... 140 Barriers to Uptake of HIT Safety ........................................................................ 140 Supportive Learning Environment and Openness to New Ideas ........................ 142 Effective Knowledge Mobilisation ..................................................................... 144 Limitations .......................................................................................................... 147 xiii Directions for Future Research ........................................................................... 148 Researcher’s Personal Perspectives on Conducting IKT ............................................ 149 Conclusion .................................................................................................................. 150 Chapter 7: Overall Discussion and Conclusion ......................................................................151 Introduction ................................................................................................................. 151 Themes ........................................................................................................................ 151 Emerging Evidence of TMAE in Primary and Community Care ....................... 151 Enumerating and Characterising TMAE ............................................................ 152 Characterisation of Harm .................................................................................... 153 A Complex and Constantly Evolving System .................................................... 154 Characterizing and Managing TMAE ................................................................. 155 IKT Approach to Characterizing and Managing TMAE .................................... 155 Recommendations ....................................................................................................... 156 Recommendations for Clinical Informatics ........................................................ 157 Recommendations for Nursing ........................................................................... 158 Recommendations for Patient Safety .................................................................. 160 Limitations .................................................................................................................. 162 Future Research Directions ......................................................................................... 163 Conclusion .................................................................................................................. 164 Bibliography ...............................................................................................................................165 Appendices ..................................................................................................................................204 Appendix A Scoping Review Search Terms, Databases, and Yields ..................................... 204 Appendix B Interrater Agreement Calculations ..................................................................... 206 xiv Appendix C Combinations of Sociotechnical Dimensions Assigned ..................................... 207 Appendix D Detailed Description of IKT Activities .............................................................. 208 Appendix E PSLS User Guide ................................................................................................ 211 Appendix F Safety Huddle Guiding Principles and Agenda .................................................. 217 Appendix G Focus Group Script and Questions ..................................................................... 219 xv List of Tables Table 2.1 Terms and Definitions Related to HIT Safety .............................................................. 15 Table 2.2 Sociotechnical Model Dimensions ............................................................................... 22 Table 3.1 Initial Screening Reasons for Exclusion ....................................................................... 35 Table 3.2 Context Summary of Articles Included in Scoping Review ......................................... 39 Table 3.3 Content Summary of Articles Included in Scoping Review ......................................... 40 Table 3.4 Types of Technology-related Errors Identified in Literature ........................................ 48 Table 3.5 Risk Mitigation Strategies ............................................................................................ 50 Table 4.1 WHO Categories for PSLS Events ............................................................................... 65 Table 4.2 Sociotechnical Model Dimensions ............................................................................... 68 Table 4.3 Heuristics for Sociotechnical Analysis ......................................................................... 69 Table 4.4 Interrater Agreement on Sociotechnical Dimensions ................................................... 71 Table 4.5 Examples of Safety Events by Sociotechnical Dimension Featured ............................ 76 Table 5.1 Guiding Perspectives .................................................................................................. 101 Table 5.2 Integrated Knowledge Translation Overview ............................................................. 106 Table 6.1 Operational Definitions of Analytic Categories ......................................................... 123 Table 6.2 Categories and Sub-Categories of Findings ................................................................ 125 xvi List of Figures Figure 2.1 Sociotechnical Model for Studying HIT ..................................................................... 23 Figure 3.1 Scoping Review Search Areas ..................................................................................... 32 Figure 3.2 PRISMA Diagram of Scoping Review Search ............................................................ 36 Figure 3.3 Year of Publication of Included Articles ..................................................................... 42 Figure 3.4 Care Setting of Included Articles ................................................................................ 42 Figure 3.5 Sources of Data of Included Articles ........................................................................... 44 Figure 4.1 Event Type Across Care Settings ................................................................................ 72 Figure 4.2 Reported Level of Harm Across Care Settings ........................................................... 73 Figure 4.3 Reported Level of Harm Across Event Types ............................................................. 73 Figure 4.4 Sociotechnical Dimensions Identified Among Events ................................................ 75 Figure 4.5 Most Common Sociotechnical Dimensions (Individual and Combinations) .............. 77 Figure 4.6 Number of Events by Technical, Social, and Sociotechnical Dimensions ................. 78 Figure 5.1 Knowledge Users ......................................................................................................... 96 Figure 5.2 Conceptual Model: IKT Approach to Address TMAE ............................................. 100 Figure 5.3 Integrated Knowledge Translation Activities Timeline ............................................ 105 Figure 5.4 CCI Team Process for Addressing TMAE ................................................................ 109 xvii Acknowledgements With gratitude, humility, and respect, I acknowledge that this research was conducted on the unceded lands of Indigenous peoples. I would like to express my sincere gratitude to my supervisor, Dr. Leanne M. Currie and committee members, Dr. Maura MacPhee and Dr. Kathy Rush, who provided invaluable guidance, expertise, and support. Leanne, thank you for seeing what I was capable of and what I wanted to achieve, long before I was able to realize it myself. Your patience and commitment to my learning has been exceptional, and I could not have hoped for a better guide on my side. Thank you Maura, for kindly sharing your expertise, and for your efforts to involve me in opportunities to enhance my learning. Thank you Kathy, for your enthusiasm, guidance, and availability. It was particularly special to have your support in pursuing my graduate studies once again. I am extremely grateful for the funding provided to me by Canadian Institutes of Health Research (CIHR) and the University of British Columbia to support me during my studies. I was also fortunate to benefit from the financial and mentorship support provided by the Canadian Nurses Foundation Dr. Kathryn J Hannah Nursing Informatics scholarship and the American Medical Informatics Association LEADS scholarship. I would also like to acknowledge my family, friends and colleagues for their ongoing support and encouragement. Bryce and Edwin, thank you does not begin to express my gratitude. Your patience and unwavering support throughout this process has been invaluable. Melissa, Lillian, xviii Bubli, and Jae-Yung, every student should be so lucky to have such a supportive group of peers with them along their journey. Lastly, I would like to acknowledge the commitment and partnership from Vancouver Coastal Health. I am grateful for the many people who shared their time, resources, knowledge and personal experiences that made this project possible and meaningful. Thank you to Lorraine Blackburn, Megan Stowe, Allison Muniak, Michael Li, and Jeb Dykema, for your collaboration and inspiring leadership. And a very special thank you to the members of the Community Clinical Informatics team. The kindness, enthusiasm, and dedication of each person in this team is truly remarkable. 1 Chapter 1: Introduction Introduction Healthcare delivery is increasingly dependent on technology and healthcare organizations are heavily investing in technological infrastructure (British Columbia Ministry of Health, 2018; Canada Health Infoway, 2017, 2018). Health information technologies (HIT) such as electronic health records (EHR), computerized provider order entry (CPOE), and mobile devices play an ever-increasing role in clinical practice, and it is largely thought that these technologies promote safe care and contribute to better patient outcomes (Canada Health Infoway, 2018; King, Patel, Jamoom, & Furukawa, 2014). However, a growing body of research has also identified the potential for HIT to contribute to adverse events (Borycki & Kushniruk, 2010; Coiera, 2015; Koppel et al., 2005; Magrabi, Ong, Runciman, & Coiera, 2010). In this dissertation the term HIT refers to software used in the healthcare system for the purpose of health information management. Further, the term technology-mediated adverse event (TMAE) is used to refer to unintended harm, or potential harm, that involved a HIT-based system. One example of a TMAE is when a patient received 39 antibiotic pills when they should have received only one pill (Wachter, 2015a). This event occurred because of a series of computer and human failures including, a confusing computer interface design, an automated 'robot' system in the pharmacy that performed a 'double check', a hidden curriculum for prescribing doctors to 'ignore all computer alerts', and a novice nurse who trusted the computers’ recommendations because ‘the computer had been right’ in the past (Wachter, 2015a). This specific example demonstrates that technology can lead to unintended consequences, and there is increasing recognition that events such as this are happening alongside the increasing uptake of 2 technologies in healthcare (Borycki & Kushniruk, 2010; Coiera, 2015; Koppel et al., 2005; Magrabi, Ong, Runciman, & Coiera, 2010). On the horizon, are more cutting-edge technologies such as smart devices that can track and transmit clinical data remotely, artificial intelligence designed to synthesize data and support diagnostic decision-making with comparable accuracy to an experienced clinician, and robots that can perform clinical tasks such as patient transfers and provide social support (Krittanawong, 2018; Manogaran et al., 2017). It is exciting to think of the potential benefits that these technologies may bring. However, as technologies in healthcare continue to advance, it is imperative to monitor for unintended consequences, particularly the potential for harm to patients, that may inadvertently occur (Coiera, Aarts, & Kulikowski, 2012). Patient Safety Patient safety has long been fundamental to healthcare, with the notion of ‘first do no harm’ rooted deeply in the history of healthcare (Kohn, Corrigan, & Donaldson, 2000). In simple terms, safety is the avoidance of harm that could result from providing care (British Columbia Patient Safety & Quality Council, 2017). Healthcare delivery focuses on achieving the best possible outcomes for patients. However, the emphasis on the avoidance of harm became more explicit near the turn of the last century when the report To Err is Human estimated that 44,000 – 98,000 Americans die in hospitals each year as a result of preventable medical errors (Kohn et al., 2000). In Canada, research in 2000 to estimate the prevalence of harm echoed these alarming findings, positing that 7.5% of hospital admissions are associated with an adverse event, of which 9,250 to 23,750 deaths from adverse events could have been prevented (Baker et al., 2004). Patient safety has evolved into a global movement embedded within the provision of 3 healthcare. According to the World Health Organization, patient safety is defined as, “the reduction of risk of unnecessary harm associated with health care to an acceptable minimum” (World Health Organization, 2009, p.15). Patient safety continues to be an issue across high income nations, where recent incidence rates of patient harm are estimated to be between 6% and 17% among hospitalised patients (Canadian Patient Safety Institute (CPSI), 2017), and medical error has been estimated to be the third leading cause of death in the United States (Makary & Daniel, 2016). These estimations have prompted an emphasis on patient safety in healthcare with calls to increase recognition, reporting, and analysis of safety events, and to take measures to mitigate future harms to patients (Mitchell, Schuster, Smith, Pronovost, & Wu, 2016; Serou et al., 2021; Vincent & Amalberti, 2016). Efforts to improve patient safety have shifted from focusing on assigning blame to one person, toward systems thinking including a ‘just culture’, which purports a balance of accountability and learning, recognizing the contextual and complex nature of safety issues (Boysen, 2013; Dekker, 2009). While progress has been made in advancing patient safety in some areas, the aim of preventing harm in healthcare has proven challenging (Bates & Singh, 2018; Panesar et al., 2016). Bates and Singh noted that two decades after the To Err is Human report, three main areas had not yet been addressed: outpatient/ambulatory care, technology-related errors, and diagnostic error (Bates & Singh, 2018). This dissertation addresses the first two areas. Unintended Consequences of Health Information Technology In recent years, technology use in healthcare has become widespread, with over half of clinicians reporting EHR use in Canada (Chang & Gupta, 2015; Gheorghiu & Hagens, 2016). As 4 technologies intended to improve care continue to be implemented, careful attention must be paid to the impact that they have on the quality of care (Bullard, Emond, Graham, Ho, & Holroyd, 2007; Granja, Janssen, & Johansen, 2018). There is some evidence that HIT has had a positive effect on patient safety outcomes, such as reducing some types of medication errors (Ammenwerth, Schnell-Inderst, Machan, & Siebert, 2008; Brenner et al., 2016; Chaudhry et al., 2006; Feldman, Buchalter, & Hayes, 2018) and improving guideline adherence in specific settings (Bright et al., 2012). However, there is growing research demonstrating that HIT can also bring about unanticipated negative consequences with regard to patient safety (Borycki, 2013; Kim, Coiera, & Magrabi, 2017; Makeham, Magrabi, Hibbert, & Hardie, 2017; Meeks, Smith, et al., 2014; Zheng, Abraham, Novak, Reynolds, & Gettinger, 2016). Some recommendations on how to address unanticipated negative consequences are beginning to emerge (Karsh, Weinger, Abbott, & Wears, 2010; Makeham et al., 2017; Sittig & Singh, 2015). Internationally, many countries are developing HIT safety initiatives, however establishing a robust program of HIT oversight has proven to be challenging (Kushniruk, Bates, Bainbridge, Househ, & Borycki, 2013; Magrabi et al., 2013; Makeham et al., 2017). One issue is that these efforts are largely focused on medical devices, and the majority of common HIT systems such as EHRs and CPOE remain unregulated (Magrabi et al., 2013). Guidelines for EHR safety were developed in the United States, yet organizational uptake has been lagging (Sittig, Salimi, et al., 2018). In a 2018 survey in Canada, researchers identified that seven out of 18 organizations surveyed had no program addressing HIT safety, and of those that had a HIT safety program, almost all were at an early stage of development (Digital Health Canada, 2018). Thus, despite the rapid uptake of technology in healthcare and growing interest in understanding HIT safety, there appears to be a lag in healthcare organizations adopting methods to ensure HIT 5 safety (Bellwood, Borycki, & Kushniruk, 2015). Technology Mediated Adverse Events For the purpose of this research, the term TMAE is used. To introduce the meaning of this term, it can be broken down into its component concepts. The term technology1 refers to “the application of scientific knowledge for practical purposes, especially in industry” (Oxford Living Dictionaries, 2019). Mediate is a verb that means to act as an intermediary (Merriam-Webster, 2019), or more simply, ‘to bring about’ (Oxford Living Dictionaries, 2019). An adverse event is “an injury resulting from a medical intervention, or in other words, it is not due to the underlying condition of the patient” (Kohn et al., 2000, p. 4). From these individual conceptual meanings, the term TMAE refers to when HIT has played a role in the process that led to a negative outcome concerning patient safety (including potential for a negative outcome, also known as a ‘near miss’ adverse event). Thus, for the purpose of this dissertation the term TMAE refers to unintended harm, or potential harm, that involved a HIT-based system. The term TMAE falls within a collection of terminologies related to HIT safety (e.g., e-iatrogenesis), the nuances of which will be described further in Chapter 2. 1 In some instances, medical devices such as electronic pumps or diagnostic machinery may also be included in the definition of ‘technology’. However, technologies that meet the national definition of medical devices are required to meet national safety certification requirements in Canada (Health Canada, n.d.), and are beyond the scope of this dissertation. For the purpose of this dissertation a narrower term, HIT, is used. HIT are typically technologies that are software-based, such as EHRs or CPOE. In Canada, software is not considered a medical device unless it replaces the activities of a clinician (Health Canada, 2019). EHRs and CPOE provide support for clinical practice, but do not replace clinical decision-making. Thus, in Canada EHRs and CPOE are not considered medical devices (Health Canada, 2019). 6 Characterizing Technology Mediated Adverse Events To date, the research concerning TMAE has largely focused on acute care settings (Bates & Singh, 2018), with an increased interest in other settings emerging more recently (Cooper et al., 2018; Magrabi et al., 2016; Zheng et al., 2016). Eighteen studies published between 2005 and 2017 identified harm or potential for harm as a result of HIT problems in acute care, and seven of the studies identified that problems with HIT were associated with patient deaths (Kim et al., 2017). The most common technical problems were functionality issues, system configuration, and inaccessible software; other contributing factors included staffing or training issues, poor integration with clinical workflow, and increased cognitive load (Kim et al., 2017). There has been minimal investigation into TMAE in the context of primary or community care settings. Only one article was identified that focused specifically on the types of patient safety problems associated with IT in primary care (Magrabi et al., 2016), and none were identified that focused specifically on community care. The Context of Primary and Community Care Definitions of primary and community care are varied in both the literature and in practice, and in some instances, community care is even thought to be included within the definition of primary care (Panesar et al., 2016). For the purpose of this dissertation research, definitions from the local context were used as follows: primary care is described as the first point of contact with the health system, and concerns helping citizens to stay healthy, get better, manage chronic conditions, and die with dignity (British Columbia Ministry of Health, n.d.-a). Community care includes services provided in citizens’ home environments on a short-term or 7 long-term basis to promote healthy, independent living, and circumvent hospital admission (British Columbia Ministry of Health, n.d.-b). In recent years, there has been a shift in provision of care from hospitals to ambulatory/outpatient care, including primary and community care (Carpenter et al., 2017; Chandrashekar, Moodley, & Jain, 2019; Gerhant & Ankit, 2020; Parthasarathy & Kumar, 2020). In general, this change is related to: strategies to reduce hospital length of stay; advances in technologies, procedures and anesthesia that have allowed for minimally invasive procedures which reduce hospitalizations and lengths of stay; patients’ preferences for outpatient-based services when available; and growth of private industry to provide care that was historically hospital-based (Gerhant & Ankit, 2020; Parthasarathy & Kumar, 2020). Given the high use of ambulatory and primary care services, and projections for increasing use of these sectors, attention to primary and community care settings is crucial. Patient Safety in Primary and Community Care Settings The body of evidence concerning patient safety has been largely focused on hospital care, with minimal attention to primary and community care settings (American College of Physicians, 2017; Daker-White et al., 2015; Gandhi & Lee, 2010; Panesar et al., 2016; The Health Foundation, 2011). Despite the volume of data available in primary care, there is very little that routinely and robustly captures the incidence of harm (Carson-Stevens et al., 2016; Tsang, Majeed, & Aylin, 2012). Studies estimate that 1-2% of primary care consultations may include an adverse event, and these events are most commonly related to medications and communication (The Health Foundation, 2011). It is also estimated that of the reported safety incidents in primary care, 4% may be associated with severe harm, and most commonly, harm is 8 associated with diagnostic or prescribing incidents (Panesar et al., 2016). Historically, the notion of patient safety in primary care settings was attributed to an individual clinician’s competency, with a recent emphasis on greater integration of knowledge of complexity and systems thinking (Bowie, McKay, McNab, & De Wet, 2016). In homecare settings, the body of evidence concerning patient safety is also small but growing (Carpenter et al., 2017; Masotti, McColl, & Green, 2010). Internationally, the incidence rate of adverse events in homecare is estimated to be from 3.5 to 15.1% (Masotti et al., 2010). In Canada, the rate of adverse events in homecare settings is estimated to be 13%, with 7.5% of these events resulting in death (Canadian Patient Safety Institute, 2013). It is also estimated that the impact of harm in homecare settings will cost the Canadian healthcare system nearly $1.5 billion annually (Canadian Patient Safety Institute & Risk Analytica, 2017). Patient safety issues in the home setting are wide-ranging and multifaceted; some examples include fragmentation of care, household hazards, and inadequate communication among patients, caregivers, and health professionals (Carpenter et al., 2017). Further, a wide variety of risks associated with patient safety are unique to homecare, including the home environment itself, communication and education concerning patients and caregivers, and the challenge of balancing risk with patient autonomy (Chandrashekar et al., 2019). As demand for healthcare services continues to grow in primary and community care settings (Carpenter et al., 2017; Chandrashekar et al., 2019; Gerhant & Ankit, 2020; Parthasarathy & Kumar, 2020), more attention and research is needed to better understand the risks to patient safety (Carpenter et al., 2017; Gandhi, Feeley, & Schummers, 2020; Gandhi & Lee, 2010; Ricciardi & Shofer, 2018). 9 Significance Research concerning HIT safety has evolved out of the patient safety movement and many studies have drawn on adverse event reporting to determine the incidence and further investigate events. The collective of studies to date has produced a sound body of evidence to establish this as a field of research. However, leading experts have expressed a need for researchers to continue to build on this knowledge. There are calls for further studies to measure the frequency and magnitude of the negative effects of technology (Kim et al., 2017), to deploy and evaluate strategies to prevent harm related to technology (Zheng et al., 2016), as well as to take up more innovative approaches to better understand the complex interactions among the sociotechnical elements of the healthcare system (Carayon et al., 2011; Kuziemsky, Nøhr, Aarts, Jaspers, & Beuscart-Zephir, 2013; Singh & Sittig, 2020; Turner, Kushniruk, & Nohr, 2017). Further to this, there is an urgent need for investigation in non-acute healthcare settings. To date, there has been minimal research on TMAE in primary and community care (Magrabi et al., 2016). Given the increased shift from hospital-based care to primary and community care, and given the specific types of safety issues in this context, an exploration into TMAE in primary and community care is warranted (Brenner et al., 2016). Relevance to Nursing The perspectives of nurses are essential to developing a robust understanding of the impact of HIT. The work processes of nurses are increasingly augmented by technologies. For example, a Canadian survey found 76% of clinical nurses are using secure messaging for interprofessional communication, 74% are using electronic records, and 43% are using CPOE systems in their practice (Canada Health Infoway, 2017). This same survey also found 1 in 10 10 nurses reported the quality of care had decreased since electronic records were implemented (Canada Health Infoway, 2017). While, technology is becoming increasingly embedded in nursing practice, little is known about the impact of HIT on nurses’ work and patient outcomes (Dowding et al., 2013). To date, research related to HIT safety has largely been taken up from medical and engineering perspectives. Nurse researchers have expressed the importance of considering nurses’ perceptions and usage of technology to delivering safe patient care (Piscotty, Kalisch, & Gracey-Thomas, 2015; Topaz et al., 2016). Learning about how technologies are employed in the delivery of healthcare from a nursing perspective, and educating nurses about the potential for TMAE, is an essential step towards mitigating patient safety risks related to technology (Borycki et al., 2017; Cronenwett et al., 2007; Dowding et al., 2013; Powell-Cope et al., 2008; Staggers et al., 2001; Stevenson & Nilsson, 2011). As the largest professional body in healthcare delivery in Canada, (Canadian Institute for Health Information, 2017), nurses are positioned throughout the healthcare system, from leadership roles to direct patient care, and have both the opportunity and professional commitment to ensure safe care for patients (Benner, Sutphen, Day, & Leonard, 2009; Cronenwett et al., 2007; Hughes, 2008). Often, nurses are working directly with clients and patients, located at what is considered the ‘sharp end’ of patient care, inheriting the systemic failures and errors as the last bastion before harm is ultimately incurred (Hughes, 2008; Reason, 1995). Having awareness of the potential threats to patient safety that are associated with HIT is a crucial additional skill in the repertoire of skills that professional nurses possess. When technology does not function as intended, nurses are well positioned to recognize and correct threats to patient safety (Bakken et al., 2004; Booth, 2016; Ricciardi & Shofer, 2018; 11 Strudwick, Nagle, Kassam, Pahwa, & Sequeira, 2019; Strudwick, Wiljer, & Fiona, 2020; Van De Castle et al., 2004). While this dissertation research does not specifically focus on nursing, the studies were carried out by a nurse, and considerations of the impact of the findings for the profession of nursing will be threaded through this dissertation. Aim and Research Questions The research in this dissertation aimed to understand TMAE in primary and community care. The dissertation is composed of four research studies including: 1) A scoping review of the literature on TMAE and harm in primary and community care settings; 2) A retrospective descriptive study of TMAE reported in primary and community care settings; 3) A description of an IKT approach to co-creating and applying new knowledge to address TMAE; and 4) A qualitative descriptive study about how clinical informatics specialists perceived their experiences in managing TMAE. The following research questions were addressed: • Study 1: What is known about TMAE in primary and community care settings? • Study 2: What are the characteristics of TMAE occurring in primary and community care settings? What sociotechnical dimensions are related to TMAE occurring in primary and community care settings? • Study 3: How might researchers and knowledge users collaborate to co-create effective practices to address TMAE? • Study 4: What are the knowledge users’ experiences of managing TMAE in primary and community care? 12 Overall Study Context The study was carried out in a large healthcare organization in Western Canada called Vancouver Coastal Health (VCH). VCH is responsible for delivering health services to meet the needs of the population within its respective geographic region in British Columbia. The organization provides acute, primary and community care for over 1.25 million people, and includes both densely populated urban areas, as well as rural and remote communities (Vancouver Coastal Health, 2021). At the inception of this study, expanding, enhancing, and improving access to primary and community care was a provincial priority for health service delivery in the region (British Columbia Ministry of Health, 2018). Further, improvement of health information management was identified as an essential strategy to address this priority (British Columbia Ministry of Health, 2018). In working to meet these system mandates, concerns regarding safety in the context of rapidly evolving and complex non-acute care settings formed the basis for the research questions. This research was supported by the Canadian Institutes of Health Research (CIHR) Health Systems Impact Fellowship (HSIF) program, which is designed to promote research capacity within healthcare organizations and enable evidence informed improvements and system transformation (McMahon & Tamblyn, 2019). Thus, the research plan was focused on the area of clinical informatics in the primary and community care settings, and the doctoral candidate was embedded within a Community Clinical Informatics team to lead this work. 13 Integrated Knowledge Translation In addition to generating new knowledge in the field, this research aimed to assist local organizational leaders to gain greater insight into the complex problem of TMAE in the context of primary and community care settings. Drawing on data generated within the healthcare system and incorporating knowledge and input from a wide variety of health professionals in these care delivery areas, an Integrated Knowledge Translation (IKT) approach to conducting research was used. IKT2 has been described as an “active collaboration between researchers and knowledge users throughout the research process” (Kothari & Wathen, 2013, p. 187). Through IKT processes, researchers and knowledge users actively collaborate in research to address questions that are of concern to knowledge users, and generate new knowledge that is actionable within their local context (Bowen & Graham, 2013). This approach is well-suited to research concerning health services delivery, where problems are often complex, because it leverages the expertise of both researchers and knowledge users in an ongoing and dynamic fashion (Gagliardi, Berta, Kothari, Boyko, & Urquhart, 2016). The methods to address the research questions in this dissertation were carried out within the context of a local healthcare organization. Knowledge users included executive leadership as well as a clinical informatics team for primary and community care, with most knowledge users holding a clinical degree in a health profession including nursing, occupational therapy, physical therapy, and counselling (no physicians or nurse practitioners participated). The data collected and analyzed were operational data and analysis proceeded in an iterative fashion, inviting input 2 While IKT has many similarities to implementation science, stakeholder consultation, and patient-oriented research, a full discussion of how IKT is similar to or different from other activities is beyond the scope of this dissertation. However, the activities of the embedded researcher co-producing evidence in this IKT study may inform other implementation science-type activities. 14 from organizational leaders and clinicians, and seeking answers to their most pressing questions regarding the delivery of health services. In addition to producing generalizable insights into TMAE in the context of primary and community care, this research approach was action-oriented and actively involved knowledge users in a variety of ways. As described in Study 1 (Chapter 3), organizational leaders participated in the consultation aspect of the scoping review, sharing their experiential expertise related to TMAE in primary and community care settings. Additionally, as described in Study 2 (Chapter 4), one of the leaders participated in the analysis of the TMAE event reports. Chapter 5 (Study 3) details how a team of knowledge users was actively involved in the co-creation of a multifaceted strategy to address TMAE in primary and community care settings. Finally, knowledge users participated as informants in the qualitative study described in Study 4 (Chapter 6). Overview of Dissertation This dissertation presents an overview of the literature on TMAE, followed by four chapters, presenting the four studies: 1) A scoping review on TMAE in primary and community care; 2) A retrospective descriptive study characterizing reported TMAE; 3) A description of an IKT approach to co-creating and applying new knowledge to address TMAE; and 4) A qualitative descriptive study about how clinical informatics specialists perceived their experiences in developing a process for managing TMAE. Each of these four studies includes a description of the methods, results, and a discussion of the findings. The final chapter includes a summary of the four studies and an overall discussion of the findings, as well as implications and recommendations for practice and future research concerning TMAE in primary and community care. 15 Chapter 2: Literature Review Introduction In this chapter, an overview of the literature concerning HIT safety is presented. The chapter includes a brief overview of the evolution of terms related to HIT safety, followed by current literature about incidence and types of TMAE, and closing with models that have been proposed to consider TMAE. Only one article was identified that presented an inventory of the types of patient safety problems associated with IT in primary care (Magrabi et al., 2016). Therefore, this dissertation research began with a scoping review to explore the literature concerning the types and severity of harm related to TMAE in primary and community care. The scoping review is presented in Chapter 3. Health Information Technology and Patient Safety Several terms have been used in reference to HIT Safety. Table 2.1 below shows the progression of the most common terms including Unintended Consequences, e-iatrogenesis, Technology Induced Error, EHR-related Safety Concern and TMAE. Table 2.1 Terms and Definitions Related to HIT Safety Term (date) Definition Unintended Consequences (2006) “….events that are neither anticipated [n]or the specific goals of the associated CPOE project” (Campbell, Sittig, Ash, Guappone, & Dykstra, 2006, p.548) e-iatrogenesis (2007) “….patient harm caused at least in part by the application of health information technology” (Weiner, Kfuri, Chan, & Fowles, 2007, p.367) Technology Induced Error (2013) “Medical errors that arise from the: design and development of technology; implementation and customization of technology; and interactions between the operation of a new technology and the new work processes that arise from the technology’s use” (Borycki, 2013, p.70) EHR-related Safety Concern (2017) “Patient safety events that reached the patient (regardless of whether harm occurred) as well as near misses and unsafe conditions” related to EHRs (Sittig et al., 2017, p. 200). TMAE (2016) Unintended harm, or potential harm, that involved a HIT-based system (Recsky & Currie, 2016). 16 There is some overlap in the meanings of the different terms, however the subtle differences may reflect different characterizations of technology, and/or inferences of causality, and/or outcomes in terms of patient safety. Experts in the field may refer to HIT in a general sense (i.e., unintended consequences, e-iatrogenesis), or delineate among different types of technology such as electronic health records (EHR) or computerized provider order entry (CPOE) (i.e., EHR-related safety concern). The term unintended consequences can be used to capture any unexpected result from HIT, including positive outcomes, and the term technology-induced error focuses on slips and mistakes induced by technology (Borycki, 2013). As described in Chapter 1, the term TMAE will be used in this dissertation. Similar to e-iatrogenesis, TMAE refers to unintended harm, or potential harm, that involved a HIT-based system. The term TMAE is intended to emphasize both the role of technology and the clinical implications, and to align with the established taxonomies concerning patient safety reporting where the term ‘adverse event’ is used to describe events that were unexpected and undesired (Cochrane et al., 2009; World Health Organization, 2009). Additionally, it is important to note that the term ‘mediate’ does not necessarily infer causality, and harm may be associated with HIT, regardless of any errors, misuse, or malfunction. Incidence and Antecedents of HIT Safety Events One of the earliest studies to document the potential for increased risk in relation to computerization was published in 2005, where 22 new potential sources of medication errors were identified in relation to a commonly used CPOE. The authors categorized these errors into two types: 1) information errors/fragmentation and 2) human-machine interface flaws (Koppel et al., 2005). Research into the impact of technology on the quality and safety of care has 17 progressed since this initial work, however the frequency, scale, severity, and remediation of these issues has yet to be adequately captured (Kim et al., 2017; Zheng et al., 2016). In reviewing the literature on the antecedents to safe use of HIT published between 2000 and 2014, Salahuddin and Ismail (2015) identified both social and technical aspects that affect safety, including, “competency, system quality, information quality, service quality, task-related stressor, training, organisation[al] resources, teamwork, physical layout, and noise” (p.889). A systematic review of 34 studies related to technology problems and their effects on care delivery published between 2004 and 2015 identified that 53% (n=18) of studies reported outcomes of actual or potential patient harm, including seven studies (21%) that reported patient deaths (Kim et al., 2017). Ten types of technical problems were identified across the studies, the most common being: functionality issues, including user interface and task fit (76%); system configuration, including decision support rules (53%); and software not accessible (35%) (Kim et al., 2017). Five contributing factors were also identified, including: staffing/training (56%), integration with clinical workflow (44%), cognitive load (38%), information governance (29%), and failure to carry out duty (24%) (Kim et al., 2017). Surveying the literature from 2014-2015, Zheng, et al. (2016) noted an expanding perspective on the unintended consequences of HIT with studies focusing on new types of HIT (i.e., electronic messaging), conducted beyond conventional acute settings, and producing new dimensions of unintended consequences (i.e., financial impact, staff attrition). They posit that ongoing research requires mixed method approaches to gain deeper insights into the existing quantitative evidence, and an increased focus on prevention and remediation of such issues (Zheng et al., 2016). 18 While the body of evidence in this area is continuing to develop, the prevalence of HIT safety events is difficult to assess, with some studies estimating the rate of safety events related to technology to be .01-.02% (Magrabi, Ong, Runciman, & Coiera, 2012; Magrabi et al., 2010), while others have found over 17% of medication incidents involved technology (Samaranayake, Cheung, Chui, & Cheung, 2012). In a retrospective analysis of daily safety huddle discussions in a hospital setting, Menon et al. (2017) found that 7% of reported safety concerns were EHR-related. All of the above-mentioned studies and reviews focused on the acute care setting. In reviewing the literature concerning TMAE in primary and community care, only one study was identified that focused exclusively on capturing HIT safety issues in a non-acute setting. In their study of 87 general practitioners in Australia over 1.5 years (May 2012 to November 2013) Magrabi and colleagues found 90 incidents of harm or potential harm, and identified 170 technology-related factors that contributed to these events (Magrabi et al., 2016). Thus, the notion of HIT safety has been established as an issue within acute settings, albeit the nature of HIT safety issues requires further study. Outside of acute care, the body of literature concerning HIT safety is less developed, and more research to identify, understand, and mitigate TMAE is warranted. Sociotechnical Systems Perspectives to Address HIT Safety A variety of theoretical models and frameworks have been created to characterize patient safety implications of HIT. Many of the theoretical approaches to HIT safety are informed by a sociotechnical approach. From a sociotechnical perspective, the relationship between people and technology is viewed as interdependent, complex, and contextual (Berg, 1999; Carayon, 2006; Coiera, 2007; Eason, 2014; Klein, 2014). In the context of HIT, a sociotechnical approach is said 19 to be a user-oriented perspective, emphasizing a thorough examination of people’s work practices surrounding technology and their interactions with the machines they use (Berg, 1999; Berg, Aarts, & Van Der Lei, 2003). Increasingly, sociotechnical theory is used as a perspective through which to view safety in relation to technology in healthcare to gain insight into the harm to patients that may occur. Some of the more commonly known models relating to HIT and safety that draw on a sociotechnical approach will now be discussed. A Framework for Diagnosing Technology-Induced Errors in Healthcare Building on James Reason’s seminal patient safety ‘Swiss Cheese’ model3 (Reason, 2000), Borycki and colleagues (2009) identify a continuum of layers that can contribute to errors involving technology, ranging from ‘blunt end’ causes such as organizational policy and design problems, to ‘sharp end’ errors made by the individual at the point-of-care (Borycki, Kushniruk, Keay, & Kuo, 2009). This framework aims to illustrate the layers of both human and technical components that ultimately contribute to technology-induced errors, and to highlight the numerous organizational structures/barriers that may contribute to or mitigate technology-induced errors (Borycki, Kushniruk, Keay, & Kuo, 2009). In assessing the impact of HIT, Borycki and colleagues have also articulated a integrative cognitive-socio-technical approach for studying technology-induced errors, identifying the most appropriate study methods along the continuum from ‘blunt end’ to ‘sharp end’ causes (Borycki & Kushniruk, 2010). In this model, Borycki and Kushniruk (2010) posit naturalistic studies that examine HIT in the ‘real-world’ context can identify problems in the organizational and social structures surrounding HIT; 3 James Reason’s model for patient safety is based on a Swiss Cheese metaphor that suggests that layered slices of cheese represent barriers in a system that are intended to prevent errors. Holes in the layers of cheese represent opportunities for error to permeate the barrier. When the holes of each layer of cheese align, errors can pass through, representing a trajectory of failures through the barriers, which ultimately lead to an adverse event (Reason et al., 2006). 20 narrowing the focus toward ‘sharp end’ errors, simulation studies can focus on the individual user interacting with HIT (Borycki & Kushniruk, 2010). Classification of Problems Reported in Computer-related Safety Incidents Another approach to investigating HIT safety is a classification framework that separates patient safety incidents into machine-related or human-computer interaction problems, and further breaks down the analysis to identify all relevant information issues, computer problems, and human errors (Coiera, 2015; Magrabi et al., 2010). In identifying multiple categories that relate to a particular event, this approach also aims to highlight the complexity of factors that contribute to adverse events. The classification framework was generated from a research study on computer-related patient safety incidents, where 99 incidents were analyzed and 55% of the problems were determined to be machine-related, and the remaining 45% were problems in human-computer interaction, and these were further analyzed according to sub-categories within these two groups (Magrabi et al., 2010). In a subsequent study of 850 HIT-related events, Magrabi et al. (2015) found that 92% of events were technical problems, while 8% were attributed solely to human factors. This classification framework has been taken up by other researchers to categorize HIT safety incidents. Warm & Edwards (2012) analyzed 149 incidents, across healthcare settings in Wales and found the majority were ‘machine-related’; they recommended that the framework be expanded to include hardware malfunction and machine retrieval errors (Warm & Edwards, 2012). Palojoki et al. (2017), applied the classification to categorize 2,379 EHR-related safety incidents and found that the majority of events (73%) involved problems related to human-computer interaction. 21 Systems Engineering Initiative for Patient Safety Model The Systems Engineering Initiative for Patient Safety (SEIPS) model emerged from the field of human factors and emphasizes the interactions among five sociotechnical work system elements: people, tasks, tools and technology, physical environment, and organizational conditions in addressing patient safety and quality of care concerns (Carayon, 2012; Carayon et al., 2006). The SEIPS model incorporates the Structure-Process-Outcome model of healthcare quality (Donabedian, 1966), depicting that the sociotechnical system of work (structure), produces work processes, which in turn shape outcomes (Carayon et al., 2006; Holden et al., 2013). A revised model adds the concepts of configuration, engagement, and adaptation to better convey the complex nature of healthcare systems (Holden et al., 2013). The SEIPS model provides a framework for understanding the complex interactions within a system that can pose risks to patient safety (Carayon et al., 2011). In a systematic review of the literature on antecedents to safe use of HIT, Salahuddin and Ismail (2015) applied the SEIPS model, finding that all five elements of the sociotechnical work system have been shown to be associated with HIT safety. Sociotechnical Model Another approach developed to address HIT safety is the “Sociotechnical model for studying health information technology in complex adaptive healthcare systems” developed by Sittig and Singh (2010, 2017). This model is intended for use across a variety of design and development, implementation, use, and evaluation applications, including patient safety-related investigations. This model includes eight technical and non-technical dimensions related to HIT 22 and focuses on their interactions within the context of a complex adaptive system4 (Sittig & Singh, 2010, 2017). The eight dimensions include: Hardware and Software; Clinical Content; User Interface; Personnel; Workflow and Communication; Internal Organizational Policies; External Rules and Regulations; and Measurement and Monitoring (Sittig & Singh, 2010, 2017). These eight interdependent and inter-related dimensions are described in more detail in Table 2.2 below. Additionally, Figure 2.1 provides a visual of the model to illustrate the interactivity among the dimensions of the model that Sittig and Singh posit. Table 2.2 Sociotechnical Model Dimensions Dimension Description 1 Hardware & Software Purely technical, the physical devices and software. 2 Clinical Content All textual data stored within the system. 3 User Interface Aspects of the computer that the user can touch, see, and hear. 4 Personnel Humans involved in the design, development, implementation, and use of HIT, purely social. 5 Workflow & Communication Processes to deliver care effectively, people need to work together. 6 Internal Organizational Policies, Procedures, & Culture Affects all dimensions in the model, influence of leadership and resources. Aligns with external regulation. 7 External Rules & Regulations Forces that facilitate or constrain HIT in the clinical setting. 8 Measurement & Monitoring Assessing the effects of HIT on an ongoing basis. (Singh & Sittig, 2020; Sittig & Singh, 2010, 2017) 4 A complex adaptive system (CAS) is defined as “a collection of individual agents with freedom to act in ways that are not always totally predictable, and whose actions are interconnected so that one agent’s actions changes the context for other agents” (Plsek & Greenhalgh, 2001, p.625). 23 Figure 2.1 Sociotechnical Model for Studying HIT Reproduced from BMJ Quality and Safety, Sittig and Singh, 19, i68-i74, 2020 with permission from BMJ Publishing Group Ltd. The Sittig & Singh Sociotechnical Model is both comprehensive and dynamic; it breaks down technology into its component parts, including hardware, software, content, and interface and situates it within the complexities of a clinical context (Sittig & Singh, 2010). It can be employed for both aggregated analyses, coding factors related to TMAE according to the eight dimensions, and can also capture the interactions among these dimensions in an investigative analysis (Makeham et al., 2017). The model was applied by Meeks et al. (2014) to analyze 100 closed safety incident investigations concerning EHRs, and the findings indicated that most 24 often, incidents were a result of a complex interaction among two or more dimensions of the model (Meeks et al., 2014). Additional Models Beyond the small collection of sociotechnical perspectives on HIT safety described above, a number of additional models and approaches have been generated. An exhaustive list is beyond the scope of this discussion, however additional contributions to the field will be briefly described. Pioneering efforts to understand the unintended consequences of technology, Ash, Berg, and Coiera (2004) classified errors related to information systems into two categories: 1) errors in entering and retrieving information in the system, and 2) errors in the communication and coordination processes that the system is intended to support. The authors emphasised that the category of ‘communication errors’ are frequently latent (hidden) errors which makes it even more important that there is ongoing attention to these issues. Campbell et al. (2006), developed a categorization scheme to capture the unintended adverse consequences related to CPOE, establishing nine categories to illustrate the negative impact of introducing a new CPOE system: 1) more/new work for clinicians, 2) workflow issues, 3) never-ending system demands, 4) paper persistence, 5) changes in communication patterns and practices, 6) emotions, 7) new kinds of errors, 8) changes in power structure5, and 9) overdependence on technology. The Interactive Sociotechnical Analysis put forward by Harrison, Koppel, and Bar-Lev (2007), focuses on five types of interactions among the sociotechnical components of an HIT-enabled system, stressing 5 Changes in power structure refers to prioritization of HIT system configurations over clinical workflows and clinicians’ practices. For example, HIT system configurations may require specific permissions for some functionalities, which may concern issues of professional autonomy and scope of practice, (e.g., order entry options for a physician compared to a nurse practitioner). 25 the critical aspects for a successful HIT implementations. Booth et al. (2015) discusses Actor-Network theory as a sociotechnical lens through which nurse researchers can study technology in nursing practice. Finally, Palojoki et al. (2017) developed a survey instrument for users to assess the safety of an EHR from a sociotechnical perspective to proactively identify and address safety risks. While the various sociotechnical models and approaches to HIT safety presented here is not an exhaustive list, it provides a sense of the development, perspectives, and applications to gain insight into the potential safety risks that HIT may introduce. Discussion of Sociotechnical Perspectives Sociotechnical perspectives emphasize the interactivity among social/human and technical components within a system. Based on their unique features, the sociotechnical models summarized above each exhibit differences in their strengths and suitability to different applications. The framework developed by Borycki and colleagues (2009) is well suited to an in-depth analysis of a specific incident because its semblance to ‘swiss cheese’ can illustrate the trajectory of an event and how the components of the framework contributed to the outcome. Further to this, the framework can highlight the positioning of nursing along the trajectory of events, with nursing leadership contributing to the organizational context and decision-making at the ‘blunt end’ and highlights the proximity of nurses to the ‘sharp end’ (Hughes, 2008), where preventing harm from reaching the patient falls to the vigilance of the nurse delivering care. On the other hand, the classification framework by Magrabi et al. (2010, 2015) is well-suited to aggregate analyses, identifying all of the factors related to a collection of events. Having applied this classification in two different countries, aggregate analyses using this model highlight the influence of national context on HIT safety (Magrabi et al., 2010, 2015). At the 26 same time, the bifurcation of this classification as to whether incidents were primarily related to technical problems or human factors may potentially oversimplify the complex, sociotechnical nature of the event. The SEIPS model incorporates an understanding of the complexity of healthcare, and stresses that the organizational context within which HIT is operating must be considered in relation to understanding safety (Carayon, 2012). Rooted in human factors, the SEIPS framework of Structure-Process-Outcome, accounts for the sociotechnical components of a work system, links directly to the outcomes of care, and inherently lends itself to improving quality of care. The Sittig & Singh Sociotechnical Model brings a valuable perspective to research and investigations concerning TMAE because it takes the entire context of care into account, and emphasizes the interrelatedness among its eight sociotechnical components. The Sociotechnical Model is explicit in recognizing the context of healthcare to be a complex adaptive system (Sittig & Singh, 2010), and, as such, may be helpful to untangle interrelated relationships. Further, the model uses accessible language with a clear visual, and is relatively easy for people with minimal previous knowledge to quickly gain a functional understanding of the sociotechnical nature of HIT safety. The versatility and accessibility of this model made it ideal to support this research into TMAE, particularly given the largely unexplored context of primary and community care and the IKT approach. The details of how the Sittig & Singh Sociotechnical Model informed this research project are presented in Chapters 4 and 5. 27 Summary of the Literature Review While the field related to HIT safety is rapidly developing, research into TMAE in primary and community care settings lags behind. The body of evidence of safety concerns related to HIT in non-acute settings is beginning to develop, and the literature to date concerning TMAE in primary and community care is explored in the scoping review presented in the chapter that follows. More research is needed to identify, analyze, and mitigate against risks to patient safety that inadvertently accompany the advancements of HIT in primary and community care settings. To that end, four studies relating to the characterization and management of TMAE in primary and community care are presented in this dissertation. 28 Chapter 3: Study 1: Technology-mediated Adverse Events (TMAE) in Primary and Community Care Settings: A Scoping Review Introduction In this chapter the methods and results of the scoping review are presented. The purpose of this scoping review was to assess the current body of knowledge related to technology-mediated adverse events (TMAE), patient safety, and harm in primary and community care settings, and to support and inform the ongoing research and knowledge translation activities that were undertaken as part of this dissertation. In carrying out the review, the conceptualization of TMAE was clarified, particularly with regard to primary and community care settings, and gaps in the evidence and potential questions for further research were identified. Additionally, information relevant to supporting the ongoing planning and delivery of services for the partner organization was gathered and shared with key stakeholders as part of the scoping review consultation and integrated knowledge translation (IKT) activities. Background A scoping review is a systematic and panoramic approach to exploring a body of literature. It is a method to map the relevant research and information to illustrate the breadth and depth of current knowledge in a particular field (Arksey & O’Malley, 2005; Levac, Colquhoun, & O’Brien, 2010; Peters et al., 2020; The Joanna Briggs Institute, 2015). Taking a broad perspective, scoping reviews can include many different types of research and non-research materials (Davis, Drey, & Gould, 2009), and are a useful approach to identify opportunities for further research, as well as to provide policy makers, practitioners, and healthcare leaders with a 29 summary of literature to inform their work (Arksey & O’Malley, 2005). A scoping review can also be particularly useful when there is limited literature available on a topic, and to bring together data from disparate and heterogeneous sources (The Joanna Briggs Institute, 2015). The purpose of this scoping review was to map the current body of knowledge on patient safety concerns related to technology in community and primary care settings, as well as to share the findings with organizational partners and obtain their perspectives, in alignment with the IKT approach of this dissertation work. Methods The methodological framework proposed by Arksey and O’Malley (2005), and further developed by Levac, et al. (2010), was followed in conducting this scoping review. It consisted of the following six stages: 1) identifying the research question; 2) identifying relevant studies; 3) selecting studies; 4) charting the data; and 5) collating; summarizing, and reporting the results; as well as 6) consulting with healthcare leaders to inform and validate findings. Research Questions This scoping review was guided by the overarching question: • What is known about TMAE in primary and community care settings? Four additional sub-questions were explored: • What approaches have been taken up to understand TMAE in primary and community care settings? • How is harm defined and measured in TMAE in primary and community care settings? 30 • What is important for organizational leaders to know about measuring and mitigating TMAE? • What gaps in this knowledge base remain to be addressed by further research efforts? Finally, this scoping review also examined existing definitions and conceptual meanings of key terms related to technology, patient safety, and non-acute care settings. Identifying Relevant Studies The search process for a scoping review is iterative, whereby researchers engage reflexively, and emphasize the relevance and contributions of evidence, rather than rigidly adhering to a methodological protocol (Arksey & O’Malley, 2005; Davis et al., 2009). The search strategy for this scoping review included searching the following databases: CINAHL, PubMed, MEDLINE, PSYCinfo, EMBASE, Web of Science, and Engineering Village. The search also included an open internet search, with a review of the first ten pages (i.e., 100 results each) from two internet search engines (i.e., Google and Google Scholar). Additionally, iterative hand-searching of select journals, websites, and reference lists to identify as many relevant sources as possible was employed (Arksey & O’Malley, 2005; The Joanna Briggs Institute, 2015). This search strategy was supported by an experienced subject matter librarian who aided in identifying and refining search terms in accordance with the search capabilities of each unique database. Beginning with an exploratory search, single and combined search terms were used initially, including: patient safety, adverse events, technology, informatics, technology-mediated adverse event, technology-induced error, community, primary care, and nursing. In this process, additional related terms were identified and noted to inform the ongoing search. All types of 31 literature were considered, including research studies, review articles, policy papers, conference proceedings, opinion articles, and white papers. Because of the nascent body of literature identified to date, no date limit was applied to the database searches. The open internet search using Google Scholar was limited to articles published since 2010 and sorted by relevance to narrow the results. The initial goal was to identify articles reporting research on TMAE, including the classification of HIT contributing factors and patient safety outcomes in primary and community care settings. Only one article was found to fit this criteria (Magrabi et al., 2016). Therefore, the focus was broadened to articles with content related to TMAE in primary and community care settings, such as safety concerns or specific types of errors that relate to the use of technology and have potential patient safety implications including harm. The final search of the databases was carried out in May 2020. The databases searched and the respective search terms used are captured in Appendix A. Selecting Studies To be included in the scoping review, articles had to: 1) focus on services provided by healthcare professionals in primary or community care settings; 2) include content relevant to patient safety or harm related to health information technology (HIT) (e.g., occurrences, rates, definitions, or outcomes); and 3) be written in English. The three areas shown in Figure 3.1 below, formed the basis for inclusion and exclusion. First, the context of primary and community care settings included any non-acute setting that was episodic in nature, including care delivered in ambulatory and outpatient settings. Second, the concept of patient safety focused on actual or potential harm to patients or clients, and included terms such as adverse event and near miss. The 32 third concept, HIT, was specific to information technologies, such as electronic health records (EHRs). Figure 3.1 Scoping Review Search Areas All articles where the title and abstract indicated components of HIT and patient safety in the context of a non-acute setting were included for screening. If it was not clear if all of these criteria were met, the article was included for a full text screening to determine its relevance and appropriateness for inclusion. For example, an article on the outcomes of new HIT in a community setting where it was not clear if the outcomes included safety issues was included for full text review. Inclusion and exclusion criteria were devised, and revised, based on increasing familiarity with the literature (Arksey & O’Malley, 2005). There were some challenges in delineating the scope for inclusion based on the heterogeneity of terms across databases and literature. For 33 example, different terms were used to refer to HIT such as electronic health record (EHR), electronic medical record (EMR), or computerized provider order entry (CPOE), yet the exact purpose and functionality of these technologies were not always described. Another example is the term ‘outpatient’, which in MEDLINE is defined as an in-hospital or free-standing facility, whereas in CINAHL it is exclusively in-hospital, and in both cases it was challenging to discern whether the HIT in use was designed for episodic care, as defined by the inclusion criteria for this scoping review. Additionally, in order to be included, the outcome needed to be specifically related to patient safety or harm. For example, a study that measured ‘quality of care indicators’ such as whether or not appropriate medications were prescribed for a patient with a chronic condition was not included because absence of treatment does not necessarily convey harm was, or could have been, incurred by the patient (i.e., if a subsequent outcome such as harm was not measured, the article was not included, but if harm or potential harm was measured, the article was included). Further to this, articles concerning prescription errors, where the prescription did not contain all of the required information to be dispensed properly were excluded unless these errors were described explicitly in relation to the use of HIT. Articles concerning workflows or clinical processes, such as studies that focused on actual or potential harm related to a delayed or absent response to abnormal laboratory results were included if HIT was identified as a key factor in the analysis, as opposed to clinician variation or other organizational factors not related to technology. Finally, articles that reported on both acute and non-acute settings were included only if the findings or insights related to non-acute settings were discernable from that of acute settings. 34 Articles were excluded if they referred to long-term care or residential care settings, were not specifically focused on adverse events, or discussed other health-related technologies such as medical devices, robotics, virtual care, or social media. With regard to medical devices, this again is an area where the terminologies in this field are not distinct. Medical devices are sometimes labelled as HIT (Kang, Wang, Yao, Zhou, & Gong, 2019), yet healthcare software may sometimes be categorized as a medical device (Health Canada, 2019) (see footnote about medical devices in Chapter 1). For this scoping review, articles concerning medical devices were excluded unless there was a significant software component that was used or monitored by healthcare providers and could have contributed to TMAE. Finally, patient-facing technologies such as patient portals or mobile health apps were also excluded in order to maintain a manageable focus for the scoping review and align with the priorities of the organizational partners involved in this IKT project. Search Results and Selection of Literature The search generated a total 3,134 articles, of which the titles and abstracts were reviewed and screened for inclusion (by CR, in consultation with LC), and duplicates were excluded. Based on the reasons for exclusion listed below (Table 3.1), a total of 3,094 articles were excluded during the initial screening, resulting in 40 articles for full text review. 35 Table 3.1 Initial Screening Reasons for Exclusion Patient Safety/Surveillance • Adverse events with no linkage to technology • Use of EMR to extract information on safety - Surveillance, screening, trigger tools, detection of adverse events or processes and systems for reporting adverse events or patient safety concerns Use/Workflow • Clinical Processes where the functioning of HIT was not a central issue (e.g., medication reconciliation) • Effects of technology on productivity, cost effectiveness, access, confidentiality/privacy, where patient safety was not also addressed • Implementation/Adoption of technology or patient/provider satisfaction, unless patient safety explicitly addressed • Usability, where patient safety was not also addressed Context • Concerning transitions to/from hospital unless HIT issue explicit • Dental devices/dentistry settings • Specific disease trajectories (e.g., diabetes), unless there was evidence about the TMAE that pertained to non-acute settings Technology • Commercial documents promoting products • Computer/video games, virtual reality, robotics, or internet of things/smart monitoring, or big data, unless patient safety concerns were addressed • Only discussing benefits of technology, no concerns regarding patient safety Types of Papers • Research protocols or descriptions of planned or proposed studies, research in progress, not reporting any outcomes or conclusions Thirty of the articles were further excluded based on full text review, and the remaining ten articles were included in the scoping review. Additionally, hand-searching via the reference lists of these ten articles, as well as using the ‘related articles’ and ‘cited by’ functions within Google Scholar for these ten articles surfaced an additional seven articles. This search concluded on May 22, 2020 and a total of 17 articles were included in this scoping review. The entire search process is captured in a PRISMA diagram in Figure 3.2 below, including the reasons for exclusion following full text review. 36 Figure 3.2 PRISMA Diagram of Scoping Review Search 37 Charting the Data Data were extracted from the included literature and charted in a data-charting form as recommended by Arksey and O’Malley (2005). The data-charting form was designed to capture the general characteristics of each included article, such as the year of publication, geographical location, type of study and data collected, and sample size, as well as the stated purpose and main conclusions. The type of HIT and care setting as described in each article was also noted, as well as the conceptualization of patient safety, specifically noting whether the focus was on actual harm incurred, the potential for harm, or both. Data were also gathered to note if the study applied any framework, or was related to nursing practice in any manner. Using descriptive analysis (Arksey & O’Malley, 2005; Peters et al., 2020), emerging themes that corresponded to the guiding questions for the scoping review were coded using NVivo 12 software (QSR International) and categorized. Specifically, errors related to technology that resulted in harm, or the potential for harm, in each article were captured, and information that was deemed insightful for a health service organization to aide in mitigating against TMAE was coded and themed. Each item was analyzed in an iterative fashion, over the course of several readings, and in consultation with an academic supervisor (LC). Consultation with Stakeholders In alignment with the IKT approach to this study, the consultation element of a scoping review was employed to share insights from the reviewed literature and draw on the experiential expertise of key stakeholders (Arksey & O’Malley, 2005; Levac et al., 2010; The Joanna Briggs Institute, 2015). This additional phase is thought to enrich the findings and add to the methodological rigor of a scoping review (Arksey & O’Malley, 2005; Levac et al., 2010). 38 Preliminary findings were shared in consultation interviews with three stakeholders who were in executive leadership positions at the local health authority. A summary of the findings from the literature was shared electronically for the leaders to review, and a follow up interview was conducted with each leader individually to summarize the findings and collect their input. Ethical approval was obtained for the consultation process. Each informant provided informed consent and the interviews were video and audio recorded. Key themes from each conversation were summarized and sent to each respective informant for their review and approval. Results This scoping review required an expansive list of search terms, and resulted in a small collection of heterogenous literature, with a wide range of practice settings, methods, and terminologies. The final collection of papers is summarized in tabular form below in chronological order, with the contextual elements captured in Table 3.2, and the purpose and key finding for each study in Table 3.3. The general characteristics of the data are described in Section 3.4.1. 39 Table 3.2 Context Summary of Articles Included in Scoping Review # Author, (Year), Place Study Method Context Technology Patient Safety Data Source Sample Size 1 Singh et al., (2010), US Quantitative Outpatient EHR/EMR Potential Harm Medical Record 78,158 2 Fernandopulle & Patel, (2010), US Qualitative Primary Care EHR/EMR Potential Harm Case Study 1 3 Yackel & Embi, (2010), US Classification Ambulatory EHR/EMR Potential Harm Incident Reports and System Monitoring Not reported 4 Palchuk et al., (2010), US Classification Ambulatory e-prescribing Potential Harm Prescriptions 2,914 5 Nanji et al., (2011), US Classification Outpatient e-prescribing Potential Harm Prescriptions 3,850 6 Abramson et al., (2011), US Quantitative Ambulatory EHR/EMR Potential Harm Prescriptions 646/ 736/ 715 7 Bazhenova & DeMoor, (2012), US Classification Outpatient CPOE Potential Harm Incident Reports or User Reports Not reported 8 Odukoya & Chui, (2013a), US Qualitative Pharmacy e-prescribing Potential Harm Interview and Observation 30 9 Odukoya & Chui, (2013b), US Review Primary Care e-prescribing Potential Harm Publications N/A 10 Aita et al., (2013), Italy Classification Outpatient e-prescribing Potential Harm Prescriptions 835 11 Slight et al., (2013), UK Qualitative General Practice EHR/EMR Potential Harm Interview & Focus Group 34/46 12 Carling, Kirkehei, Dalsbo, & Paulsen, (2013), US Review Ambulatory General HIT Potential Harm Publications 39 13 Schiff, et al., (2015), US Multimethod Outpatient (extracted) CPOE Potential Harm Multiple Sources 16,790/ 2,522 14 Magrabi et al., (2016) Australia Classification General Practice General HIT Actual and Potential Harm Incident Reports or User Reports 90 15 Lourenco, Bursua, & Groo, (2016), US Qualitative Pharmacy e-prescribing Actual and Potential Harm Case Study 5 16 Ai, Wong, Amato, & Wright, (2018), US Classification Outpatient e-prescribing Potential Harm Medical Record 10,000/1,157 17 Graber et al., (2019), US Mixed Methods Ambulatory (extracted) EHR/EMR Harm incurred Malpractice Claims 146 Note: US – United States, UK – United Kingdom, EHR – Electronic Health Record, EMR – Electronic Medical Record 40 Table 3.3 Content Summary of Articles Included in Scoping Review # Author, (Year), Place Purpose or Objective Notable Findings 1 Singh et al., (2010), US To examine whether alerts of abnormal laboratory results, in an integrated EMR, resulted in timely follow-up. 6.8% of alerts lacked follow-up at 30 days; follow-up is not fail-safe even when providers are alerted by the EMR. 2 Fernandopulle & Patel, (2010), US To report experiences using a commercial EHR in a small, innovative practice. Technology did not support the multidisciplinary team approach to care that is a hallmark of the medical home. 3 Yackel & Embi, (2010), US To report experience with test result management. Four error categories: interface and results routing logic, provider record issues, EHR settings, system maintenance. 4 Palchuk et al., (2010), US To quantify discrepancies between structured and free text fields of electronic prescriptions and analyze potential adverse events. 16.1% of prescriptions were found to have at least one discrepancy, 16.8% of those were potentially severe enough to lead to a hospitalization. 5 Nanji et al., (2011), US To characterize the errors associated with computer-generated prescriptions in the community setting. About one in 10 computer-generated prescriptions included at least one error, of which a third had potential for harm. 6 Abramson et al., (2011), US To assess the effect of transitioning to a new EHR system on prescribing errors. Overall prescribing error rate was lowest at baseline and highest 12 weeks post-implementation. 7 Bazhenova & DeMoor, (2012), US To assess the effect of CPOE on medication errors in an outpatient chemotherapy center. 35% of medication errors were facilitated by CPOE; related to implementation learning curve and CPOE complexity. 8 Odukoya & Chui, (2013a), US To characterize the safety hazards related to e-prescribing in community pharmacies. Identified three themes that increase potential for error: increased cognitive burden, interruptions, communication issues. 9 Odukoya & Chui, (2013b), US To provide a focused review of patient safety issues related to using e-prescribing systems in primary care and pharmacies. In comparison with hospital settings, a relatively small number of studies have evaluated e-prescribing in primary care. 10 Aita et al., (2013), Italy To evaluate outpatient chemotherapy prescribing errors where electronic prescribing is used. 20% error rate; most errors due to mistakes in CPOE system configuration or misuse of copy and paste function. 11 Slight et al., (2013), UK To examine the causes of prescribing and monitoring errors in general practice and provide recommendations. Selecting the wrong item from an electronic pick-list, unnecessary and/or inappropriate alerts, computer speed identified as possible error-producing conditions. Note: US – United States, UK – United Kingdom, EHR – Electronic Health Record, EMR – Electronic Medical Record, CPOE – Computerized Provider Order Entry 41 Table 3.3 Content Summary of Articles Included in Scoping Review (continued) # Author, (Year), Place Purpose or Objective Notable Findings 12 Carling, Kirkehei, Dalsbo, & Paulsen, (2013), US To find evidence that electronic applications for medication management in ambulatory care might harm patients. Little evidence of new risks to patient safety, more research needed on the drawbacks and negative outcomes that implementation might introduce. 13 Schiff, et al., (2015), US To investigate ways to better identify, understand, and prevent electronic ordering errors in the future. The absence of any efficient system for systematically gathering, analyzing, and sharing data on CPOE risks and medication errors. 14 Magrabi et al., (2016) Australia To categorize problems with information technology, which affect patient safety in general practice. Broad categories of problems are similar to those reported in hospitals, routine updates to software particularly problematic in general practice. 15 Lourenco, Bursua, & Groo, (2016), US To report a series of cases in which e-prescribing contributed to medication errors. Improved methods of electronic communication between the clinic and the pharmacy are needed. 16 Ai, Wong, Amato, & Wright, (2018), US To examine information intended for pharmacists in a field not sent to pharmacists and assess the potential harm. More than one-third of the prescriptions contained comments that had potential for harm if not communicated. 17 Graber et al., (2019), (2019), US To obtain additional information on HIT–related problems from malpractice suits and claims. Problems relating to hybrid record systems were the leading contributing factor. Note: US – United States, CPOE – Computerized Provider Order Entry Summary of Identified Literature Most of the included studies were written by researchers from the United States (n=14), with three studies from other countries, namely Australia, Italy, and the United Kingdom. Even though date limits were not applied except for Google Scholar searches, all of the included literature has been published since 2010, with the distribution displayed in Figure 3.3 below. 42 Figure 3.3 Year of Publication of Included Articles The care settings for each article were spread across various primary and community care areas, including one study that pertained to both pharmacy and primary care, as illustrated in Figure 3.4 below. Two articles, Schiff et al. (2015) and Graber et al. (2019), focused on both acute and community care settings, however the authors separated out details specific to community-based settings that could be extracted to inform this scoping review. Figure 3.4 Care Setting of Included Articles 01234562010 2011 2012 2013 2014 2015 2016 2017 2018 20190 1 2 3 4 5 6Community PharmacyGeneral (non-acute extracted)Primary Care/General PracticeAmbulatoryOutpatient 43 There were a variety of different research methods used among the included articles, with four studies using qualitative methods such as case studies or interviews to examine TMAE (Fernandopulle & Patel, 2010; Lourenco et al., 2016; Odukoya & Chui, 2013b; Slight, et al., 2013), two using quantitative methods to capture the rate of errors or safety concerns (Abramson et al., 2011; Singh et al., 2010) and seven articles deductively classifying adverse events either by type of patient safety concern or HIT error (Ai et al., 2018; Aita et al., 2013; Bazhenova & DeMoor, 2012; Magrabi et al., 2016; Nanji et al., 2011; Palchuk et al., 2010; Yackel & Embi, 2010). Of the remaining four articles, one used both qualitative and quantitative methods to analyze cases of malpractice involving TMAE (Graber et al., 2019), another was a report that drew on multiple methods to capture TMAE from a number of different sites (Schiff, et al., 2015), and finally two articles were reviews of literature related to TMAE concerning a non-acute setting (Carling et al., 2013; Odukoya & Chui, 2013a). With regard to health disciplines, six articles used general terms like clinician or interdisciplinary team, and 11 articles focused on the medical provider or prescriber, most often a physician. Five articles also included a focus on pharmacists. Only two articles made mention of nursing as part of the interdisciplinary team, or as a nurse prescriber. The 17 included articles could also be grouped into two categories, according to their focus on either HIT or patient safety. Some of the articles (n=10) emphasized the errors and problems with HIT and then secondarily, connected these issues to the patient safety concerns that resulted. The second group of articles (n=7) focussed on the patient safety outcomes or harm that were, or could be incurred, and then linked these outcomes back to the HIT that was in place. 44 There were a considerable variety of data sources, as shown in Figure 3.5 below. Four studies specified that they analyzed prescriptions to identify errors and potential for harm. Three stated their source of data was medical records, and within this, authors often identified a specific aspect of the medical record such as lab notifications. Three studies used incident reports. One article was a multi-site study conducted over a two-year period that drew on multiple sources, including safety meeting minutes, user logs, EHR audits, in addition to the aforementioned data sources. Finally, one other notable source of data was malpractice claims (Graber et al., 2019). Figure 3.5 Sources of Data of Included Articles There was considerable heterogeneity across the studies with regard to definitions and meanings of the terms related to HIT and patient safety. For example, Abramson (2011) studied error rates during a transition to a newer EHR system with more advanced clinical decision support, and yet the description of the functionality of this system aligned with what other 0 1 2 3 4System MonitoringMultiple SourcesMalpractice ClaimsPublications (review)Observation or InterviewCase StudyMedical RecordIncident or User ReportsPrescriptions 45 authors referred to as e-prescribing. The use of key terms for the patient safety consequences captured in each article was also heterogeneous. For example, only one study focused exclusively on instances where actual harm occurred drawing on data from malpractice claims (Graber et al., 2019). Over half (n=9) of the studies either assessed errors, hazards, issues, or challenges that had the potential to cause harm. The other eight studies focused on both potential and actual instances of harm, yet most did not have data on actual patient outcomes, rather the researchers inferred the degree of harm based on the description of an error. Within this collection of articles, there were six studies that classified events by various criteria such as type of error or harm incurred. For example, in the study by Nanji et al. (2011), computer-related medication errors were categorized by both drug type and severity of the clinical outcome. Similarly, Aita et al. (2013) classified prescribing errors by type, potential and actual severity, as well as preventability. There were also seven articles that described or employed some type of pre-existing framework to analyze TMAE. For example, Graber et al. (2019) used a proprietary sociotechnical framework which was refined over the course of their study. In their literature review, Odukoya and Chui (2013a) applied a sociotechnical framework from the field of human factors engineering to analyze safety hazards related to e-prescribing, and reviewed the merits of this model and the human factors engineering needed to improve HIT safety. Both Magrabi et al. (2016) and Schiff et al. (2015) categorized an HIT-related safety event according to whether it was: 1) a new issue unique to technology; 2) an issue that also had historically been a problem using paper records; or 3) a known issue made more likely or more impactful because of technology. Magrabi et al. (2016) also conducted further analysis on safety incidents, classifying them primarily as human factors or technical problems and identifying factors associated with both categories. Finally, Schiff et al. (2015) generated a new assessment 46 framework from their analysis, which included two components to guide the analysis of a computer related issue: 1) what happened to the patient? and 2) what happened within the CPOE system? Key Findings From the included articles, a list of potential safety concerns related to HIT was extracted, as displayed in Table 3.5, and key content concerning mitigating the risks of TMAE in primary and community care settings were identified and captured in Table 3.6. Types of Technology-related Errors Twelve different issues or errors related to HIT that led to safety concerns were identified within the literature. These are summarized in Table 3.4 below according to how the authors characterized them, and in order of the frequency with which they were identified. The most frequently mentioned HIT error, identified in eight articles, was a mismatch between what the HIT was capable of and what clinician users needed the HIT to do. For example, in Fernandopulle and Patel (2010), the software was not configured to support the interdisciplinary care delivery model because the documentation template was designed for physicians and did not suit the needs of nonphysician staff. Also, a frequently identified issue mentioned in eight studies, was missing, delayed, or incorrect data, which most often referred to erroneous prescriptions from electronic systems potentially leading to medication errors, or delayed or missing critical lab values that may result in diagnostic error. Malfunctioning of HIT software itself was also identified in seven articles, where issues such as glitches or unexpected down time led to missteps in care. Further to this, six articles 47 identified the usability of software as an issue, where poor design made it more challenging for clinicians to use the software as intended. For example, a provider may have entered special instructions into an e-prescribing system, however the software was not designed to relay that information to a pharmacist. In Ai et al. (2018), one third of e-prescriptions identified the potential for harm because important information was not communicated. Similarly, six articles defined some errors as user error, where the clinician did not use the software correctly. For example in Aita et al. (2013), prescribing errors were identified where the clinician failed to adjust dosing in the CPOE as clinically indicated. Six articles also discussed software configurations and a lack of interoperability as reasons for errors, in which users were required to access and manually search sources for client information that led to important information being overlooked. Related to this, hybrid systems, where both electronic and paper records were being used, also increased the risk of missing important information. Additionally, the transition period to a new or updated system could lead to errors until users become proficient at using the new system. Transitions of personnel could also lead to issues, where system access and functionality lags behind clinical staffing changes (Graber et al., 2019; Yackel & Embi, 2010). Finally, there were a number of specific issues related to how HIT is used including clinicians not paying attention to alert notifications, misuse of short cuts such as copy and paste function, and trusting the information in the system without clinically validating it. 48 Table 3.4 Types of Technology-related Errors Identified in Literature Type of Technology-related Error Article in which Error Identified Technology limitations or incompatibilities with workflow Bazhenova & DeMoor, (2012); Fernandopulle & Patel, (2010); Lourenco et al., (2016); Magrabi et al., (2016); Odukoya & Chui, (2013a); Odukoya & Chui, (2013b); Schiff et al., (2015); Singh et al., (2010) Missing, delayed, or incorrect data Abramson et al., (2011); Ai et al., (2018); Aita et al., (2013); Carling et al., (2013); Graber et al., (2019); Nanji et al., (2011); Odukoya & Chui, (2013b); Palchuk et al., (2010) Faulty programming or unreliable software Carling et al., (2013); Fernandopulle & Patel, (2010); Lourenco et al., (2016); Magrabi et al., (2016); Nanji et al., (2011); Schiff et al., (2015); Yackel & Embi, (2010) Poor user interface Carling et al., (2013); Lourenco et al., (2016); Magrabi et al., (2016); Nanji et al., (2011); Schiff et al., (2015); Yackel & Embi, (2010) User error Bazhenova & DeMoor, (2012); Lourenco et al., (2016) ; Nanji et al., (2011); Odukoya & Chui, (2013a); Slight et al., (2013); Yackel & Embi, (2010) Poor configurations, lack of interoperability, routing problems Ai et al., (2018); Aita et al., (2013); Fernandopulle & Patel, 2010); Odukoya & Chui, (2013b); Schiff et al., (2015); Yackel & Embi, (2010) Software updates or system transitions Abramson et al., (2011); Bazhenova & DeMoor, (2012); Graber et al., (2019); Magrabi et al., (2016) Alert inconsistency or fatigue Fernandopulle & Patel, (2010); Schiff et al., (2015); Singh et al., (2010); Slight et al., (2013) Over-reliance on HIT Graber et al., (2019); Lourenco et al., (2016); Singh et al., (2010) Hybrid systems Graber et al., (2019); Schiff et al., (2015) Personnel challenges Graber et al., (2019); Yackel & Embi, (2010) Copy/Paste, workarounds, pre-population Graber et al., (2019); Schiff et al., (2015) Risk Mitigation Strategies In addition to the different types of HIT errors, there were a number of insights into the risks and potential mitigation strategies related to TMAE. Using descriptive analysis, key findings within each article were highlighted and then iteratively grouped and categorized into themes to address the question of what is important for organizational leaders to know about managing and mitigating TMAE. These themes are listed below in Table 3.5 in order of the frequency with which they were identified. 49 In this collection of literature, medication errors were often measured to assess harm or the potential for harm related to HIT, which highlights the need for careful attention to the systems and processes related to prescribing in mitigating TMAE. In over half of the included items, authors also articulated the need to learn more and called for ongoing reporting and analyzing of HIT-related safety concerns. For example, in planning for HIT implementation or changes, authors highlighted that it is important for organizations to carefully analyze their clinical needs and workflows as well as the capabilities and designs of potential HIT systems. They further recommended that during the process of implementing a new system, leaders should also be circumspect with rigorous testing and thorough education for users, not only on operating HIT but also on the risks for errors and potential TMAEs that can occur. In looking comparatively at acute care, four articles identified that non-acute settings shared many similar HIT related concerns, although Graber et al. (2019) suggest that the severity of harm in the community/primary care setting that resulted was less than that of acute care settings. The authors posited that this was possibly because of the fact that ambulatory patients have lower acuity medical problems or that errors in the ambulatory setting are more easily detected and mitigated (Graber et al., 2019). One article, (Magrabi et al., 2016) identified HIT issues that are particularly problematic in a general practice setting, noting migrating records and software updates as increasing risk of errors. Two articles (Aita et al., 2013; Magrabi et al., 2016) highlighted the risk of large-scale adverse events related to technology, where a single technical malfunction could potentially affect the entire collection of patient records, and subsequently patients. Finally, three articles mentioned the impact of HIT on the users, pointing out the time and attention clinicians must invest in using HIT, particularly when it is difficult to 50 use, or malfunctioning, and relating this to the potential impact on their capabilities to deliver care (Magrabi et al., 2016; Nanji et al., 2011; Slight, Howard, et al., 2013). Table 3.5 Risk Mitigation Strategies Focus of Risk Mitigation Strategy Articles in which Strategy Identified Role of pharmacy in community Abramson et al., (2011); Ai et al., (2018); Aita et al., (2013); Lourenco et al., (2016); Nanji et al., (2011); Odukoya & Chui, (2013a); Odukoya & Chui, (2013b); Palchuk et al., (2010); Schiff et al., (2015) Ongoing monitoring/analyzing of TMAE Abramson et al., (2011); Aita et al., (2013); Bazhenova & DeMoor, (2012); Carling et al., (2013); Graber et al., (2019); Magrabi et al., (2016); Palchuk et al., (2010); Schiff et al., (2015); Singh et al., (2010) Planning for HIT Aita et al., (2013); Fernandopulle & Patel, (2010); Nanji et al., (2011); Odukoya & Chui, (2013a); Odukoya & Chui, (2013b); Schiff et al., (2015); Singh et al., (2010); Yackel & Embi, (2010) Similarities to acute care Graber et al., (2019); Magrabi et al., (2016); Nanji et al., (2011); Schiff et al., (2015) Educate users Carling et al., (2013); Lourenco et al., (2016); Odukoya & Chui, (2013a); (Slight et al., (2013) Underreporting of HIT safety issues Carling et al., (2013); Lourenco et al., (2016); (Yackel & Embi, (2010) HIT as a burden to clinicians Magrabi et al., (2016); Nanji et al., (2011); (Slight et al., (2013) Distributed impact of HIT issues Aita et al., (2013); Magrabi et al., (2016) Consultation with Stakeholders In the consultation interviews, the stakeholders compared and contrasted findings from the literature with their own organizational knowledge and experience. All three organizational leaders expressed that they were aware that similar technology-related issues have occurred within their organizations to some degree, and were also in agreement with the themes extracted from the scoping review. Building upon the themes from the scoping review, each of the organizational leaders shared insights from their experiences in their respective roles in relation to technology and patient safety in primary and community care. These conversations surfaced 51 additional risks and issues related to technology, as well as how healthcare organizations can respond to TMAE and promote safety in primary and community care. They also shared topic areas that warrant further research. Comments from these consultations are presented generally, without attributing comments to the specific consultation participants. Two areas were identified beyond the scoping review: additional organizational challenges and organizational priorities. Additional Organizational Challenges Identified by Stakeholders Beyond the themes from the scoping review literature, additional topic areas related to HIT safety that surfaced during the consultation interviews include the risks related to medical devices and virtual care. Medical devices, which are outside of the organization’s conventional HIT infrastructure, are being increasingly used in non-acute settings, and may convey information related to client status or treatment. Examples of such devices are home telemonitoring equipment, where clients conduct their own assessments and send the information to their care provider, and infusion pumps, which deliver medications monitored remotely by the care provider. In these examples, devices carry and relay information pertaining to the client’s status or care, and the opportunity for error lies in the assumption that this communication is correct and occurs in a closed loop with the care providers. Regarding virtual care, the Covid-19 pandemic, which occurred as this dissertation work was being carried out, forced a rapid deployment of virtual services and health leaders expressed concern about the unintended consequences that may surface. In relation to the potential for harm, they specifically voiced concerns about how it was difficult to determine which encounters were most appropriate to be carried out via virtual appointment, the potential lack of accessibility of virtual care for some 52 patient groups, and challenges in ensuring continuity of client information when using virtual care. In general, documentation and continuity of information in primary and community care were of great concern to the organizational leaders. Stakeholders reported that current HIT infrastructure is a composite of a number of different software systems with little interoperability, and safety concerns may arise where clients access multiple services, and their clinical information is siloed and disjointed amongst their providers. Related to this, the HIT infrastructure was also identified as practice support, where documentation assists clinicians in adhering to evidence-based organizational policies. Yet, if the HIT proves onerous for clinicians, they may not use it as intended and miss important opportunities to adhere to evidence-based guidelines. The leaders identified additional challenges in relation to promoting patient safety and technological advancements within the organization. From an organizational perspective, it is challenging to balance these two priorities, given that technologies advance quite rapidly, yet for a large, complex health organization, large scale changes are slow and cost prohibitive. This results in decision-making related to HIT and safety often being in the context of an interim state, with ongoing HIT upgrades, implementations, and innovations constantly changing the technological landscape, and potentially obscuring the unintended consequences within the complexity of this constantly evolving system. Further to this, models of delivering care in primary and community care settings are also changing, with efforts to provide higher acuity care outside of hospitals, using interdisciplinary care teams, adding additional complexity to consider in ensuring HIT is safe and effective. A final challenge shared in these consultations was that the records related to adverse events in the community are not afforded the same 53 protection from legal proceedings as the corresponding quality assurance documentation in acute care (i.e., where patients admitted to a hospital are the legal responsibility of the hospital). This is largely due to the number of patient interactions with providers and other organizations that are not part of the healthcare organization (i.e., primary care providers are largely independent businesses), which potentially limits quality and safety reporting and improvement efforts. Organizational Priorities Identified by Stakeholders The consultation discussions with leaders also surfaced a number of key priorities for ongoing service delivery, as well as future research. Understanding patient safety and identifying harm beyond medication errors in non-acute settings is essential to furthering our knowledge of TMAE. For example, the trajectories of care for many clients weave between and among acute and non-acute settings, and threats to the fidelity and continuity of information have the potential to negatively impact client care. In order to continue to identify and mitigate against TMAE, leaders identified some requisite assets within their organization. First, an organizational mindset of continuous improvement, with robust data collection on both HIT use and patient outcomes is necessary to inform decision-making around technology. Further to this, promoting safe HIT requires the support of clinical informaticians who are uniquely situated within the organization to recognize and address TMAE given their expertise in both the HIT systems in use, and the clinical services they are intended to support. Additionally, given the episodic nature of care in primary and community care, leaders also considered the potential role of clients and families in helping to recognize and report information errors or incidents of harm related to their care. 54 Discussion This scoping review aimed to uncover what is known about TMAE, patient safety, and harm in primary and community care settings. While few studies to date have focused on the inadvertent harms incurred by patients related to HIT in non-acute settings, there are a number of technology errors and issues identified in the literature that can in fact, lead to negative consequences. Additionally, the literature in this area contains some important themes that can inform the work of health service organizations to promote patient safety in relation to technology in non-acute settings and guide future research. Further to this, drawing on the experience and expertise of organizational leaders, the scoping review consultation surfaced some important insights to inform future work, both in practice and in future research. Key Findings In this scoping review, the body of literature concerning patient safety and technology in primary and community care settings was mapped. The emerging body of literature on TMAE in these settings is heterogenous in nature, with a wide variety of studies exploring different aspects of this field with varying terminologies and frameworks. This scoping review aimed to examine the definitions and conceptual meanings in the literature, and found some key concepts, such as technology and patient safety, lacking consistent definitions. Specifically, this scoping review was focused on how harm is defined and measured in primary and community care settings, and found that in relation to HIT, the actual harm incurred is not always assessed, rather errors or issues may be proxy measures for potentially adverse outcomes for patients. A small number of analytic frameworks have been used to study TMAE or HIT errors. These frameworks 55 commonly employed a sociotechnical perspective, recognizing both human and machine, but differences remained with regard to if and how, the consequences for the patient were assessed. Ultimately, the goal of this scoping review was to assess what is known about TMAE in primary and community care settings. The HIT-related errors identified from this collection of studies were largely similar to those of acute care (Kim et al., 2017), with the most predominant types of technology-related errors identified being: 1) technology limitations or incompatibilities with workflow; 2) missing, delayed or incorrect data; and 3) faulty programming or unreliable software. Additionally, key themes related to the risks and potential mitigation strategies for TMAE in non-acute settings were identified in this scoping review, including 1) the predominance of medication-related safety concerns; 2) the importance of recognizing, reporting and learning from TMAE; as well as 3) the need for careful HIT planning, and ongoing support for clinicians as users of HIT. Medication-related issues may be predominant because medications are a main intervention in primary and community care, and data related to prescriptions can be extracted from pharmacy systems (i.e., medication data are readily available). In analyzing and attempting to mitigate against TMAE, these efforts need to take the complexity of primary and community care into account, recognizing the potential for an error to parlay into a larger, more widespread problem. The consultation aspect of this scoping review validated the findings from the literature, and also surfaced some additional areas for concern related to TMAE. Organizational leaders articulated that the current disparate technological infrastructure in the primary and community care settings in their context poses risks in relation to TMAE. Additionally, the constant churn of both organizational and technological change further compounds this challenge, making it difficult to articulate a definitive understanding of TMAE. In response to these identified 56 challenges, leaders identified opportunities and resources within the organization to begin to mitigate against TMAE, namely dedicated clinical informaticians with expertise in both clinical practice and HIT, and a data-driven approach to recognize and respond to TMAE in an effort towards continuous improvement. These findings are echoed in the literature related to technology and safety, where organizations are called upon to ensure adequate resources and continuous training for safe use of HIT (Salahuddin & Ismail, 2015). There is also emphasis on the importance of capturing TMAE in a standardized manner with collective oversight to share learnings towards promoting safe use of HIT (Kushniruk et al., 2013; Singh, Classen, & Sittig, 2011). Additionally, the challenges organizations face because of the constantly evolving state of both healthcare and technology are also articulated (Bates & Singh, 2018; Sittig et al., 2020). Recognizing these challenges, best practices are beginning to emerge to help guide organizations to use their HIT tools in a safe and effective manner (Petersen et al., 2021). Further to this, new methods to optimize the analysis and learning from safety reports by using machine learning are being developed (Fong et al., 2018). Implications While the collection of literature in this scoping review was small, it was also diverse, covering a wide variety of non-acute care settings and issues related to TMAE, and surfacing a number of important considerations for health service organizations. First, in delivering services in primary and community care, findings from this scoping review indicate the importance of exercising vigilance with regard to technology and safety by monitoring, reporting, and systematically analyzing TMAE to inform ongoing mitigation efforts. In looking at technology- 57 related safety concerns, having a standard taxonomy or analytic framework can assist with building the organizational knowledge of TMAE and support data-driven decision-making to promote safety as technological advancements continue (Singh & Sittig, 2020; Sittig et al., 2020). Building upon this, as the delivery of healthcare services continues to evolve in new and innovative ways, organizations need to approach patient safety proactively, and consider what harm means on an ongoing basis. While medication errors are commonly recognized as adverse events, there is more to learn about additional instances where clients may be negatively affected in the process of receiving healthcare services, particularly outside of acute care (Bates & Singh, 2018). Additionally, while the severity of harm is thought to be lower outside of acute care (Graber et al., 2019), it is important to consider the potential for harm in non-acute settings. That is, with more episodic events and greater patient autonomy, it may be that harm in primary and community care settings is not recognized or reported in the same manner as in hospital settings (Canadian Patient Safety Institute, 2013; Cooper et al., 2018; Nora & Beghetto, 2020). To support efforts to learn and improve, organizations may also consider opportunities for patients and families to play a role in recognizing and reporting safety events as they experience them (Borycki, Househ, Kushniruk, Nohr, & Takeda, 2012). Primary and community care settings are varied in their structures and operations, yet the patient’s journey weaves consistently through all of them and, as such, standardization, ongoing monitoring of safety issues, and including the patient voice may offer valuable insights to mitigate against harm (Giardina et al., 2018a). 58 Directions for Future Research A wide range of safety concerns related to technology were identified within the literature, yet the degree to which we understand these concerns and their implications warrants further exploration. Current research in this area is disparate, and requires standardization of the related terms and concepts. Future research should aim to define and measure harm in primary and community care settings, and specify if, and how, technology was involved in a consistent manner. More research is also needed to understand TMAE outside of acute care. Further to this, because of the more geographically and temporally dispersed nature of care in primary and community care settings, there is the potential for technology-related errors to remain latent, and future research should also investigate how TMAE occur, from a sociotechnical perspective (Singh & Sittig, 2020). The studies in this scoping review included a variety of approaches to researching HIT-related safety concerns in the context of primary and community care. While greater standardization in the field is required, in order to capture the real-world implications of TMAE, future research needs to carefully balance rigour with a pragmatic approach. It is critical to ensure HIT fits well within its organizational context, and further investigation ought to be from the perspective of the “messy and complex reality in which HIT is situated” (Kuziemsky, 2015, p. 156). Action-oriented, embedded research approaches may be well-suited to conducting research within a complex adaptive system such as healthcare, where change is constant (Plsek & Greenhalgh, 2001). Additionally, research methods to investigate TMAE should also include analysis of both HIT function and patient outcomes in order to understand the implications of technology in relation to safety. Systematically classifying the risks to patient safety, alongside the HIT problems is recommended to enhance measurement in this area and prioritize the most 59 significant threats (Kim et al., 2017). Finally, future research should also focus on the role of nurses within the context of TMAE. Realizing the benefits and risks of HIT is incumbent upon nurses and nurse leaders as they are ideally positioned to identify, report, and mitigate against TMAE within the healthcare system (Bakken et al., 2004; Booth, 2016; Strudwick et al., 2019, 2020; Van De Castle et al., 2004), Limitations Given the paucity of literature related to TMAE in non-acute settings, there was a relatively small number of eligible studies. While a very thorough search was conducted, it is not presumed to be exhaustive. Selection of studies for inclusion, was done in consultation with an academic supervisor, and may not have been as rigorous as a two-person, independent review process. Also, no studies were excluded from the review due to reasons related to study type or quality, as critical appraisal is not standard practice in a scoping review (Arksey & O’Malley, 2005; Peters et al., 2020). A further limitation is the heterogeneous collection of literature, with inconsistent terminologies and definitions of terms, that made it challenging to identify commonalities and themes across articles. Additionally, in extracting the themes and TMAE-related concerns, the frequency or emphasis on each within and among the articles was not accounted for. A final limitation is that the literature search concluded in May of 2020, and because HIT is a rapidly evolving field, the findings may not pertain to the breadth of safety concerns on an ongoing basis, particularly with regard to the changes brought about by the Covid-19 pandemic response. 60 Conclusion This chapter presented a scoping review related to TMAE in primary and community care. The literature was searched for items concerning technology, patient safety, and non-acute healthcare settings. Seventeen articles were included and mapped, with themes related to technology errors and insights for health organizations highlighted. Based on this collection of literature, there is a need for more research, particularly to explore harm in relation to HIT. This scoping review also included a consultation exercise with organizational stakeholders to both disseminate the findings from the literature as a knowledge translation exercise, and gain additional insights into this topic area drawing on their expertise and experience. There is opportunity for further exploration in this domain, with the consultation informants identifying a variety of issues beyond what was identified in the literature. 61 Chapter 4: Study 2: Characterization of Technology-Mediated Adverse Event Reports in Primary and Community Care Introduction In this chapter, a retrospective descriptive study focused on TMAE in primary and community care settings is presented. The purpose of this study was to examine the characteristics of TMAE that were reported in a primary and community care setting over two years. Voluntarily reported safety events were analyzed using a sociotechnical framework to examine the health information technology (HIT)-related dimensions of each event. As part of the Integrated Knowledge Translation (IKT) activities incorporated into this dissertation, the analysis also included participation from an organizational executive leader, and findings were disseminated locally within the organization. Limited Evidence Describing TMAE in Primary and Community Care Settings As identified in the scoping review presented in Chapter 3, there is a need to further explore the field of TMAE in primary and community care settings. Healthcare organizations have expanded services and are leveraging new technologies in non-acute settings (British Columbia Ministry of Health, 2020; Canada Health Infoway, 2018), thus, the potential safety implications related to technologies in non-acute settings need to be distinctly understood. Research on the negative unintended consequences of technology in healthcare has largely been based on acute care/inpatient/hospital settings with minimal research outside of these settings (Abraham et al., 2020; Brenner et al., 2016; Makeham et al., 2017). Yet, there is emerging evidence that patient safety events associated with HIT are occurring in primary and community 62 care settings (Graber et al., 2019; Magrabi et al., 2016; Powell, Sittig, Chrouser, & Singh, 2020). Several different sources of errors may contribute to TMAE in non-acute settings such as missing or incorrect data (Ai et al., 2018; Nanji et al., 2011), unreliable software (Lourenco et al., 2016; Magrabi et al., 2016), and poor usability (Odukoya & Chui, 2013b; Slight, Howard, et al., 2013). Opportunities for improving care that can come from recognizing and reporting TMAE have also been identified (Borycki, 2013; Singh & Sittig, 2020). This study aimed to leverage data from a web-based voluntary safety reporting system to examine the characteristics of reported TMAE that have occurred in primary and community care in one large healthcare organization. Safety Event Reporting One approach to understanding the potential harms related to technology in the context of care delivery is through safety event reporting. Monitoring, reporting, and learning from incidents where patient safety was, or could have been, compromised can inform system improvements to reduce the likelihood of similar events occurring in the future (Kohn et al., 2000). Safety event reporting data has proven to be a valuable source of information to support learning and improvement (Flemons & Mcrae, 2012; Mitchell et al., 2016). For TMAE specifically, voluntarily reported safety events, can provide insight into the emerging risks that are associated with HIT in healthcare settings (Fong et al., 2018; Magrabi et al., 2015; Martin et al., 2019; Palojoki, Mäkelä, et al., 2017; Warm & Edwards, 2012). Healthcare organizations around the world have developed reporting infrastructures to formally measure, analyze, and improve patient safety (Howell et al., 2017; Runciman, 2002). However, reporting and analyzing HIT-related safety events is not yet a widespread practice within healthcare (Bellwood et al., 63 2015; Digital Health Canada, 2018) and developing real-time methods for HIT safety surveillance is an ongoing challenge (Sittig et al., 2020). The British Columbia (BC) Patient Safety and Learning System (PSLS) is web-based voluntary reporting system operating across BC (Cochrane et al., 2009). In 2016, drawing on research by Magrabi et al. (2010), PSLS leadership added an optional screening question to the reporting interface to capture if technology was involved in the safety event. The specific configuration of this question was determined in consultation with a number of local experts and used the Magrabi et al. (2010) study as a guide. Thus, the purpose of this study was to examine safety events in which a computer system was described as having been involved in the safety event. To date, no studies were found that had implemented evaluation of reported adverse events that included a screening question such as this. While it is recognised that voluntary reporting systems very likely only represent a small portion of safety concerns (Macrae, 2015; Noble & Pronovost, 2010), it was thought that an examination of these data would be useful to understand patterns and characteristics of reported events. Methods The descriptive study presented in this chapter is a quantitative, retrospective analysis of voluntarily reported safety events from the PSLS database. Events from primary and community care settings that indicated computer involvement reported for a two-year period between 2016 and 2018 were extracted and analyzed. 64 Research Questions The following research questions were addressed: 1. What are the characteristics of TMAE reported in primary and community care settings? 2. What sociotechnical dimensions are related to TMAE reported in primary and community care settings? To address the first research question, the number of reported TMAE in primary and community care settings, type of safety concerns, and severity of harm that resulted from the TMAE were identified in a descriptive analysis. To address the second question, the data were analyzed using a sociotechnical model to identify the most prominently featured sociotechnical dimensions contributing to each safety event. The research methods that were employed for this study included aspects of IKT, incorporating organizational partner participation and with the goal to promote direct translation of findings into practice. Setting For the purposes of this study, the scope of primary and community care was defined by the service delivery model within the partnering organization, which includes all services delivered outside of acute care settings, (i.e., home care, public health, and mental health and substance use services), with the exception of residential care. Residential care is included in the local operational definition of community care; however, it was not included in this study because the service delivery models and current documentation systems in residential facilities were more similar to that of acute care facilities, particularly because patients are receiving full-time rather than episodic care. At the time of the study, HIT systems were widely implemented, 65 yet records remained hybrid to some degree, with system maturity at stage 2 or 3, according to the O-ENRAM stages (HIMMS, n.d.). There were two main commercial HIT systems that were broadly used across primary and community care, as well as a variety of ancillary systems in use at specific locations. Local Safety Event Reporting The PSLS is a web-based tool that is linked to a secure database that houses voluntarily reported safety incident information (Cochrane et al., 2009). Healthcare providers are encouraged to report instances of patient harm or circumstances with the potential to cause harm using the PSLS electronic reporting system. Clinicians who make a report are guided to identify the type of incident and degree of harm that resulted from the event. These categories, based on the World Health Organization’s (WHO) International Classification for Patient Safety (World Health Organization, 2009), are presented in Table 4.1. Table 4.1 WHO Categories for PSLS Events Term Definition PSLS Categories Incident Type A descriptive term for a category made up of incidents of a common nature, grouped because of shared, agreed features • Clinical Administration • Clinical Process/Procedure • Documentation • Healthcare Associated Infection • Medication/IV Fluids/Biological • Blood/Blood Products • Nutrition • Oxygen/Gas/Vapour • Medical Device/Equipment • Behaviour • Patient Accidents • Infrastructure/Building/Fixtures • Resources/Organizational Management 66 Table 4.1 WHO Categories for PSLS Events (continued) Term Definition PSLS Categories Degree of Harm The severity and duration of harm, and any treatment implications, that result from an incident • None – patient outcome is not symptomatic or no symptoms detected and no treatment is required. • Mild – patient outcome is symptomatic, symptoms are mild, loss of function or harm is minimal or intermediate but short term, and no or minimal intervention • Moderate – patient outcome is symptomatic, requiring intervention • Severe – patient outcome is symptomatic, requiring life-saving intervention or major surgical/medical intervention, shortening life expectancy or causing major permanent or long-term harm or loss of function • Death – on balance of probabilities, death was caused or brought forward in the short term by the incident. In 2016, an optional question was added to the reporting interface to capture if technology was involved in the event: “Was the computer system involved in this event?” (Magrabi et al., 2010). The PSLS event reports that have been classified as having ‘computer system involvement’ include a categorization of the incident type and the degree of harm incurred, and a free text description of what happened and how the computer was involved. An example of a TMAE that was reported to the PSLS system is an incident where the EHR allergy alert system did not function properly, resulting in a patient being administered a medication that was a known allergen followed by the patient experiencing an allergic reaction requiring urgent medical attention. Inclusion and Exclusion Criteria All reports were screened for duplicates and appropriateness. Two reports that seemed similar, (i.e., could have been the same incident), were considered to be separate incidents for the purposes of this analysis because the narrative text was not identical. During initial screening, one report was excluded because it was determined that the computer did not contribute to an 67 error or harmful outcome; instead, the reporter claimed that the computer system was used to obtain data relevant to the event. Data Extraction and Analysis Ethical approval was obtained for this study, including special permission to access the PSLS database directly based on the premise of the organizational partnership and embedded researcher working in an IKT capacity. Data included all reports from the PSLS event reporting systems where “yes” was indicated in response to the question noted above from primary and community care settings from November 1, 2016 - November 1, 2018. The data were manually extracted by the organizational database custodian, screened and de-identified by a privacy associate, and then provided to the research team. The events included the following information: de-identified incident unique ID; date; location; type of care setting in which the incident occurred; type of incident (see Table 4.1); degree of harm (see Table 4.1); and a free text description of the incident by the reporting clinician including how the computer system contributed to the event. Some reports also contained additional details added by a local supervisor after reviewing the event report. To address Research Question 1, the PSLS data were analyzed and summarized using descriptive statistics to capture characteristics of the TMAE based on the information provided by the reporting clinician. To address Research Question 2, the free text descriptions of what happened in each event and how the computer was involved were analyzed using the sociotechnical model developed by Sittig and Singh (2010, 2017) to identify the most prominent sociotechnical features related to each event. 68 Sociotechnical Model As described earlier, the Sittig & Singh Sociotechnical Model is a framework that breaks down technology into its component parts, including hardware, software, content, and interface and situates it in relation to the user, and within the complexities of a clinical context (Sittig & Singh, 2010). The eight dimensions are listed and described briefly in Table 4.2. Table 4.2 Sociotechnical Model Dimensions Dimension Description 1 Hardware & Software Purely technical, the physical devices and software. 2 Clinical Content All textual data stored within the system. 3 User Interface Aspects of the computer that the user can touch, see, and hear. 4 Personnel Humans involved in the design, development, implementation, and use of HIT, purely social. 5 Workflow & Communication Processes to deliver care effectively, people need to work together. 6 Internal Organizational Policies, Procedures, & Culture Affects all dimensions in the model, influence of leadership and resources. Aligns with external regulation. 7 External Rules & Regulations Forces that facilitate or constrain HIT in the clinical setting. 8 Measurement & Monitoring Assessing the effects of HIT on an ongoing basis. (Singh & Sittig, 2020; Sittig & Singh, 2010, 2017) Sociotechnical Analysis Two research team members, the embedded researcher and an organizational executive leader from the partnering health organization, conducted the sociotechnical analysis. The first step in preparing to apply the sociotechnical model to code the PSLS events was for the coders to familiarize themselves with the model by reviewing literature concerning its development, refinement, and application (Meeks, et al., 2014; Sittig & Singh, 2010, 2017). Both researchers independently reviewed and then discussed this literature to align their understanding of the descriptions of each dimension of the model. Following this, the researchers reviewed a training 69 set of ten reported PSLS events (from December 2019) together and discussed how they would apply the model to code each event. After discussing each of the ten training events, agreeing on the sociotechnical dimension codes to apply and the rationale for these choices, the researchers were prepared to code independently. Coding was completed in cycles, initially in batches of 20 events, with checkpoints in between to assess level of agreement. These checkpoints provided an opportunity for the coders to learn more about the others’ perspectives on assessing the events and understanding the dimensions of the sociotechnical model. As coding proceeded, a set of heuristics was developed to aide in applying the codes for the sociotechnical dimensions in a consistent manner, see Table 4.3. Table 4.3 Heuristics for Sociotechnical Analysis Sociotechnical Dimension Coding Heuristic 1 Hardware & Software Concerns if the software or hardware did not work as intended 2 Clinical Content Concerns if the HIT system did not include the information that was needed or content was captured in a way that was inconsistent with user’s needs. 3 User Interface Concerns if the information was available in the HIT system, but the user missed it 4 Personnel Includes people involved in design, development, and use; did not apply if person caught error (near miss) 5 Workflow & Communication not created* 6 Internal Organizational Policies, Procedures, & Culture Concerns policies, procedures, and culture within the organization 7 External Rules & Regulations Concerns issues connected to outside organizations, beyond organizational policies, procedures, and culture 8 Measurement & Monitoring not created* General Reminder Only assess information available in report, avoid filling in the gaps in the story * Note: A heuristic was not created because the label was considered self-explanatory 70 Interrater Agreement and Interrater Reliability To assess interrater reliability, both interrater agreement (percent agreement) and Cohen’s kappa (!) were calculated after each cycle, and discrepancies were discussed to reach consensus. Interrater agreement was calculated according to the formula: #$%&'( *+ ,-(''%'./0#$%&'( *+ ,-(''%'./0 + #$%&'( *+ 230,-(''%'./0 Because each event could theoretically have up to eight codes assigned, the percent agreement of the number of individual codes assigned to each event was also calculated. (See Appendix B for a detailed description of interrater agreement calculations). Calculating the percent of interrater agreement does not account for agreement by chance (Polit & Beck, 2020), therefore, Cohen’s kappa (!) was also calculated. The minimal acceptance for Cohen’s kappa is .6, and .75 is preferred (Polit & Beck, 2020). While interrater agreement and Cohen’s kappa between the two independent coders was assessed, the goal was ultimately to reach consensus on the sociotechnical dimensions coded in each event to learn more about what may have facilitated the safety events and map patterns within the data. The agreement statistics for each round of coding are captured in Table 4.4 below. 71 Table 4.4 Interrater Agreement on Sociotechnical Dimensions Coding Round n Interrater agreement Interrater agreement by item Cohen;s kappa ! p-value Consensus after discussion Training 10* - - - - 100% 1 18 33% 58% .285 <.001 100% 2 21 58% 66% .578 <.001 100% 3 19 42% 55% .361 <.001 100% 4 46 50% 66% .362 <.001 100% Overall 104 45% 57% .405 <.001 100% Note: *training events not included in data set Finally, the various combinations of dimensions were also grouped into broader categories: purely technical, purely social, or elements of both social and technical. Technical dimensions were considered to be the first three dimensions: 1) Hardware and Software Computing Infrastructure; 2) Clinical Content; and 3) User Interface, and the remaining five dimensions were considered social. Results A total of 105 reports were extracted by the PSLS data custodian and screened for duplicates and appropriateness. As described above, one report was deemed inappropriate, and therefore the remaining 104 PSLS reports were included for analysis. Events by Setting, Type, and Degree of Harm Across the different types of settings, events from Public Health had the highest number of reports (n=45), while home health had the least (n=7). The most common type of event reported was Medication/IV fluid/biological (n=68). The types of events across the different care settings are depicted in Figure 4.1. 72 Figure 4.1 Event Type Across Care Settings With regard to the degree of harm incurred, as defined in Table 4.1 (2009), 85 (82%) of the reports indicated that the event resulted in no harm, 13 (13%) resulted in minor harm, five (5%) resulted in moderate harm, one (1%) report indicated severe harm, and no reports (0%) indicated death. The levels of harm reported across the five categories of primary and community care settings are depicted in Figure 4.2, harm across the different types of events in Figure 4.3, and narrative descriptions of events to illustrate the different degrees of harm reported follows. Home Health Primary Care Mental Health Pharmacy Public HealthUnsafe behaviour 0 0 1 0 0Medication/IV fluid/biological 1 4 13 21 29Laboratory 0 4 0 0 2Equipment/product/medical device 0 0 1 0 0Documentation 0 0 0 0 8Clinical process/procedure 4 3 2 0 4Clinical administration 2 2 1 0 2051015202530354045 73 Figure 4.2 Reported Level of Harm Across Care Settings Figure 4.3 Reported Level of Harm Across Event Types Home Health Primary Care Mental Health Pharmacy Public Health4 - Severe harm 0 0 1 0 03 - Moderate harm 2 2 1 0 02 - Minor harm 2 4 2 2 31 - No harm 3 7 14 19 42051015202530354045Equipment/product/medical deviceUnsafebehaviour LaboratoryClinicaladministration DocumentationClinicalprocess/procedureMedication/IVfluid/biological4 - Severe harm 0 0 0 1 0 0 03 - Moderate harm 0 0 1 2 0 1 12 - Minor harm 0 1 1 2 0 4 51 - No harm 1 0 4 2 8 8 62010203040506070 74 Most events were reported as involving no harm. No harm events were often related to immunizations, including vaccines administered off schedule, without proper consent, to the wrong person, past expiration date, and/or were not properly documented. An example of an event reported with minor harm was when a client only received a partial dose of their prescribed medication, which was reportedly due to the clinician having to enter the dose into two separate HIT systems and making a data entry error. An example of an event that incurred moderate harm was an incorrect referral to a social worker rather than a registered nurse that led to incomplete contact tracing and follow up care concerning a communicable disease. Finally, the single event where severe harm was reported involved a client who had been redirected from a community site to seek immediate attention at an emergency department after critical lab values were noted in their electronic record. The lab values were not relayed to, nor accessed by, the emergency department and the client was immediately released without treatment. There were no further details regarding the client’s outcome. While it is unclear specifically why the report indicated severe harm, they reported that the computer system contributed to the event because the client’s information that was generated in a computer system in the community setting was not accessible in the computer system in the acute setting. Events by Sociotechnical Dimensions From a sociotechnical perspective, a total of 182 sociotechnical dimensions were identified within the 104 PSLS events. The most commonly occurring dimension was Personnel (n= 48), followed by Workflow & Communication (n=37), and then Content (n=30). The dimension of Measurement & Monitoring was only identified in one instance. The total number of times each sociotechnical dimension code was assigned for the entire data set is depicted in 75 Figure 4.4, and examples to illustrate reports featuring each of the eight sociotechnical dimensions are presented in Table 4.5 below. (For a full list of all combinations of sociotechnical dimensions assigned, see Appendix C.) Figure 4.4 Sociotechnical Dimensions Identified Among Events 2130214837168105101520253035404550Hardware &SoftwareContent User Interface Personnel Workflow &CommunicationOrganizationalPoliciesExternalRegulationsMearurement &Monitoring 76 Table 4.5 Examples of Safety Events by Sociotechnical Dimension Featured Dimension Example Safety Event 1 Hardware & Software Client database was slow to load, leading to a client who had been barred from accessing a safe injection site being granted access because the clinician could not view the client’s history in the database. 2 Clinical Content Limited options for referral reason in EMR, leading to missed care because assigned care provider was anticipating wound care and not qualified to administer chemo as client required. 3 User Interface Alert fatigue, leading to an alert to follow up with police remaining active for 16 encounters before it was addressed. 4 Personnel User documented on the wrong client’s chart, leading to a client receiving a prescription with another client’s name, birthdate, etc. 5 Workflow & Communication Two different clinical services using the same HIT system with ambiguity over who was responsible for certain lab results, leading to the most responsible care provider remaining unaware of significant results. 6 Internal Organizational Policies, Procedures, & Culture Client information was not successfully transferred from one service to another, leading to missed care such as central line care or breastfeeding support. 7 External Rules & Regulations Care provider did not have access to pertinent client information from provincial medication database, leading to a client deemed high risk for overdose having to go without opioid agonist therapy for several days. 8 Measurement & Monitoring HIT system downtime prevented trigger to resume community services following a client’s discharge from acute care, leading to decline in their clinical status. Safety events were assigned between one and four sociotechnical dimensions. The majority of the events had one (n=43) or two (n=45) dimensions assigned to them. The most commonly assigned code was Personnel, followed by Personnel AND Clinical Content together. The most common combinations of sociotechnical dimensions that were coded are presented in Figure 4.5. 77 Figure 4.5 Most Common Sociotechnical Dimensions (Individual and Combinations) Grouping the events into three categories, technical, social, and sociotechnical, visualized in Figure 4.6 below, showed that nearly half of the events (n=50) were sociotechnical in nature, approximately a third pertained to social dimensions, and the remainder (n=20) were coded as featuring problems related to only technical dimensions. 0 2 4 6 8 10 12 14User Interface AND PersonnelWorkflow & Communicationn AND PersonnelWorkflow & CommunicationClinical ContentClinical Content AND PersonnelHardware & SoftwarePersonnel 78 Figure 4.6 Number of Events by Technical, Social, and Sociotechnical Dimensions To illustrate the complex, sociotechnical nature of a computer-related PSLS event, and the implications for patient care, two further examples are described here. The sociotechnical dimensions that were identified in the analysis will also be presented to illustrate how the sociotechnical model was applied. First, one report presented a scenario where approximately three months of lab values ordered by one clinician were returned to a primary care site by mail in hardcopy, rather than sent electronically, because the lab software failed to recognize the appropriate clinician who should have received the results. These lab values included at least one result that warranted immediate follow up. In this event, the sociotechnical dimensions were assigned as follows: • Hardware & Software, because the software failed to recognize the appropriate care provider to whom results should have been returned. 05101520253035404550Technical Social Sociotechnical 79 • Workflow & Communication, because there was a breakdown in the communication between two HIT systems. • External Rules, Regulations, & Pressures, because the delay in returning these mismatched lab values was due in part, to the policy of the lab organization, which is governed at a provincial level. A second example was an instance where a client with gender dysphoria in a suicidal state reported suffering additional harm due to being repeatedly misgendered. At the site where they were seeking care, client information was housed in two different HIT systems, and the reporter indicated the clinician involved may not have had access to the system that contained the client’s full history. Additionally, it was noted that this client’s record included a mix of pronouns in both HIT systems. The sociotechnical dimensions assigned to this event were as follows: • Clinical Content, because information on the client’s preferred name, pronouns, and gender was not consistent throughout the HIT systems. • Workflow & Communication, because pertinent clinical information was in two different HIT systems. • External Rules, Regulations, & Pressures, because the policy concerning gender designation on personal identification is at a provincial level. These two examples illustrate the diversity of TMAE that may surface in non-acute settings, including the implications for patient care across different settings, as well as the multiple and varied layers that warrant consideration from a sociotechnical perspective. 80 Discussion This study produced some interesting findings related to TMAE in primary and community care. The implications of these findings for healthcare organizations and future research, as well as the limitations of this study, are discussed below. Key Findings The aim of this study was to examine the characteristics of TMAE in primary and community care settings. This study confirms that TMAE are occurring in primary and community care settings. In a two-year period, 104 TMAE were reported from a large healthcare organization. The most commonly reported TMAE were in the public health setting while the home health setting had the fewest. Across all settings, medication errors were the most common type of TMAE reported, which is consistent with the evidence in the scoping review presented in the previous chapter. The vast majority of reported events indicated no harm was actually incurred, and no deaths were reported related to these TMAE. Additionally, this study also aimed to identify the most prominent sociotechnical dimensions related to TMAE in primary and community care settings. From a sociotechnical perspective, the dimensions of Personnel and Workflow & Communication were most prominently identified as contributing to TMAE, demonstrating the importance of considering the users and contexts surrounding technology in relation to patient safety (Ray et al., 2019). Additionally, by dividing the events into three categories: technical, social, or sociotechnical, it was shown that nearly half of the events were sociotechnical, with interdependent elements of both technology, and people and context, identified. 81 Capturing HIT-related Safety Issues This study makes an original and meaningful contribution to the field of technology and patient safety. Only a small number of previous studies have applied the Sittig and Singh model to analyze HIT-related safety issues, and these studies have divergent findings. Meeks et al. (2014) analysed 100 internal investigation reports from a large integrated health system that included acute, primary, and community care, and reported that the most commonly associated dimensions were Hardware & Software, Clinical Content, and Workflow & Communication (Meeks, et al., 2014). Unfortunately, the researchers did not disaggregate the data by setting, so the impact specifically on primary and community care is not clear. A second study examined 120 HIT-related sentinel events reported to the Joint Commission from accredited organizations in the US. These are voluntary reports where any of the more than 6000 hospitals in the US can provide a report for an event that resulted in a death. In that study, they found Clinical Content, Workflow & Communication, and User Interface as the most commonly associated sociotechnical dimensions (Castro, Buczkowski, & Hafner, 2016). The top three most commonly identified dimensions in these two studies included both technical and social dimensions, which contrasts with a more recent study (Kang et al., 2019) that found the most commonly identified dimensions were technical dimensions. Kang et al. (2019) applied the sociotechnical model to analyze 268 HIT-related events reported to the Federal Drug Administration (FDA) Manufacturer and User Facility Device Experience (MAUDE) database, and found that these events were most commonly related to the dimensions of Hardware & Software, Clinical Content, and User Interface. The variation between these studies may potentially reflect the differences in the databases from which the data were extracted. The MAUDE database is specifically designed for reports related to devices and 82 therefore may contain more information related to technical dimensions, whereas event reporting and incident investigations may collect more contextual and clinical details. This highlights that differences in the way information is collected and the perspectives of the people reporting may influence what insights are gleaned from reported events. Finally, in retrospectively analyzing safety huddle discussions about EHR concerns, Menon et al. (2017) applied the sociotechnical model to 245 identified concerns from a hospital setting, and found that three dimensions: Hardware & Software, Clinical Content, and Personnel, accounted for nearly three quarters of the safety issues identified. Interestingly, none of the studies listed above that used the Sittig & Singh model reported the combinations of dimensions for items that had more than one dimension assigned. Yet, in the current study Personnel had the largest proportion of individual events, followed by Personnel combined with Clinical Content, highlighting the interrelatedness of the dimensions of the model (Sittig & Singh, 2010, 2017). Thus, the current study demonstrated that the original premise of the Sittig and Singh model (i.e., interrelatedness of dimensions) is present in TMAE. Future activities that use this model should explore the interrelatedness of these dimensions, as there may be relevant patterns that could be more informative than the individual dimensions alone. Limitations of Voluntary Reporting Systems Limitations of voluntary reporting systems such as underreporting, subjectivity, misinterpretation of reports, and a lack of impact or follow up, have been well documented in the literature (Flemons & Mcrae, 2012; Mitchell et al., 2016; Noble & Pronovost, 2010; Shojania, 2008). These limitations are echoed in relation to voluntary reporting of TMAE as well. The details of reported events are subjective, and clinicians may not recognize or be able to fully 83 articulate the role of technology in the event, therefore TMAE may be underreported (Borycki, 2013; Howe, Hettinger, & Ratwani, 2019). For example, some of the contributing factors specific to HIT, such as increased cognitive load or poor user interface, may not be reported. Despite these challenges, this study demonstrates that there is some potential benefit from including technology/computer safety in voluntary reporting systems. Facilitating direct communication from point-of-care clinicians, the PSLS system can promote a rapid response to patient safety risks and help prevent future incidents from occurring (Cochrane et al., 2009). Technologies and their applications in care delivery will continue to evolve, and it is critical that healthcare organizations continue to monitor for unintended consequences and mitigate threats to patient safety. To the author’s knowledge, this is the first study to use a screening filter that has asked the reporter to identify if a computer was involved in the event. Modifying the reporting system to include direct questioning regarding technology can help to glean more information about TMAE that have occurred and help prioritize improvements to HIT. The screening question embedded within the PSLS interface was a helpful source of information for understanding more specifically how technology was a factor. This is a novel practice in safety reporting. Future work should consider the exact manner in which HIT-related questioning is constructed. The TMAE in this study were identified by the reporter’s affirmative response to the question, “Was the computer system involved in the event?”. While this question was designed based on local consultation by the PSLS team, there was no prior research to guide this decision. Therefore, it is possible that alternative phrasing, perhaps considering terms such as ‘technology’ versus ‘computer’, or ‘contribute to’ versus ‘involved’, may elicit different information from reporting clinicians. Further, one report was excluded for analysis because the reporter did not understand 84 the reason for the question, therefore it is likely that reporters who will be using this screening question would benefit from education as well. Events in Primary and Community Care Settings For primary and community care settings specifically, there is one recent study that has employed the Sittig and Singh sociotechnical model to classify TMAE. Powell et al. (2020) analyzed 214 reports of root cause analyses related to HIT in an outpatient setting to examine the HIT-related factors associated with diagnostic delay, and determined that the predominant sociotechnical dimensions were Personnel, and Workflow & Communication. This is similar to studies focused on acute care settings (Castro et al., 2016; Meeks, et al., 2014), and is also consistent with the findings of the present study. The similarity of these findings suggests the most significant risks that HIT inadvertently may pose to patient safety has more to do with the users and the context of use rather than the technology itself. Sociotechnical Dimensions It is also important to examine the sociotechnical model and its application in relation to these findings. The dimensions of Personnel, and Workflow & Communication, both have broad definitions. In the literature concerning the sociotechnical model, Personnel (also referred to as ‘people’) is defined as “the humans involved in all aspects of the design development, implementation, and use of HIT” (Sittig & Singh, 2010, p.i70) or “everyone who interacts in some way with technology, including developers, users, IT personnel, patients, and informaticians” (Singh & Sittig, 2020, p.S93). Workflow & Communication refers to “significant two way communication” and a mismatch between a clinical information system and 85 the clinical workflow (Sittig & Singh, 2010, p.i71) or “processes to ensure that patient care is carried out effectively” (Singh & Sittig, 2020, p.S93). Given these broad definitions and the fact that safety events typically relate to patients, clinicians, and care-related activities in some way, it is to be expected that the ‘people’ dimensions would be prominent in analyzing an aggregate of TMAE. A further observation from applying the sociotechnical model to voluntarily reported safety events is that the dimensions referring to the overarching contextual elements such as Internal Organizational Policies, Procedures, & Culture; External Rules, Regulations, & Pressures; and Measurement & Monitoring were least frequently identified as dimensions related to the TMAE in this study. This may potentially be related to the limited information available in the PSLS report, and the perspectives of the reporting clinician, who may not be aware of some of the governance or maintenance practices that could have influenced the situation (Sujan, 2015). Achieving Consensus in Coding Sociotechnical Dimensions Reflecting on using the sociotechnical model in this study, it was challenging to reach interrater agreement while independently assigning dimensions to the PSLS events, yet a discussion of the events quickly led to consensus. The difficulties in reaching agreement independently may have been attributable to the fact that the collection of events was very diverse, the details reported were often limited, and there was no systematic education for reporting TMAE using the PSLS. Clinical Content was the most frequently deliberated dimension. Sittig and Singh (2010) acknowledge the challenge in separating out Clinical Content, from the other technical aspects of Hardware & Software, and User Interface. For 86 example, in one event the format of date in the EMR contributed to an error in vaccine scheduling. The heuristics developed during the data analysis of this study helped to discern among these three dimensions for coding purposes. However, even with guidance from the heuristics, there continued to be discrepancies with regard to the technical dimensions in independently coding the events. While reaching consensus independently was challenging, the sociotechnical model proved to be a valuable and pragmatic tool to optimize the organizational learning from TMAE. The model helped to ensure events were reviewed systematically to identify all aspects related to each event and extract information about the contributing factors that might otherwise remain latent. A larger volume of reports may have helped to provide more precision to the coding process. Future research should continue to explore reliability of this coding scheme. Risk of Harm in Primary and Community Care Settings There are some important implications to consider regarding the context of primary and community care settings. There are few published accounts of applying the sociotechnical model to investigate TMAE, particularly outside of acute care. While the level of risk for harm may potentially be lower in ambulatory settings (Graber et al., 2019), more research is warranted to examine the prevalence and consequences of TMAE in non-acute settings. Given the more episodic nature of care and greater autonomy of patients, the instances of harm may be more obscure. Less overt instances of harm such as a delayed follow-up to an abnormal lab result could in fact, result in a missed or delayed diagnosis, failure or absence of treatment, and ultimately, a severe degree of harm (Maillet et al., 2018; Powell et al., 2020). Further to this, future work in this area needs to acknowledge the patient’s agency and perceptions of safety, 87 including the social and psychological impacts of care, in addition to physical harm, in recognizing and addressing safety concerns (Charles Vincent, Carthey, Macrae, & Amalberti, 2017). Impact of Integrated Knowledge Translation Healthcare organizations can work to mitigate against TMAE in primary and community care settings by leveraging voluntary safety reporting systems to learn from past events and identify opportunities for improvement. This study was conducted as part of an academic-practice partnership and provided an opportunity to engage healthcare decision-makers and knowledge users to support rapid application of the findings and organizational learning (Gagliardi et al., 2016; Nguyen et al., 2020). As part of the IKT approach embedded within the study methods, the analysis of this study was carried out in collaboration with an organizational executive leader. Applying the sociotechnical model to the data set provided an additional opportunity for an immersive learning experience related to TMAE and contributed to organizational decision-making. The work in this study was leveraged towards the establishment of a process for the Community Clinical Informatics department to receive future TMAE reported through PSLS, and conduct ongoing analyses of PSLS events using the sociotechnical model (presented in the next chapter). Directions for Future Research The body of literature related to TMAE is relatively young, yet is developing rapidly. Leading researchers have expressed a need for researchers to continue to develop the body of knowledge related to technology and patient safety (Borycki, 2013; Sittig et al., 2020). There are 88 calls for further studies to measure the frequency and magnitude of negative effects of technology (Kim et al., 2017), as well as to take up more innovative approaches to better understand the complex interactions among the sociotechnical elements of the healthcare system (Sittig et al., 2020). In the continued pursuit of knowledge on TMAE, there is an urgent need for further investigation in non-acute healthcare settings. Little is known about the incidence, antecedents, and consequences of TMAE in primary and community care settings. This is particularly important given the recent impetus to expand services in these areas, and the ongoing emergence of technologies in primary and community care settings. More investigations are needed to examine the incidence of TMAE in these settings. Further to this, in-depth studies into TMAE resulting in significant harm may also provide valuable insights into the potential risks and system vulnerabilities that may foment TMAE. The sociotechnical model helps to elucidate the complexity of TMAE and future studies could evaluate the integration of this model into safety reporting processes. Finally, future research in this area could also benefit from using applied methods and partnering with healthcare organizations to investigate strategies to mitigate against TMAE, while advancing the science of patient safety and HIT. Limitations There are a number of limitations to this study. First, there are limitations related to the use of voluntarily reported safety data, including incompleteness, subjectivity, and underreporting (Macrae, 2015; Noble & Pronovost, 2010). Additionally, given this was a secondary use of data, there were some challenges because the data were originally collected for a different purpose (Jacobson, Hamilton, & Galloway, 1993; Nicoll & Beyea, 1999). In this case, 89 the analysis may have been limited because the PSLS events that were reported concerned a variety of different HIT systems, and the specific HIT system involved was not specified, or details related to how HIT played a role in the event were limited. Further to this, the assessment of the degree of harm that resulted in each event was limited to that which was initially assigned by the reporting clinician and was difficult to verify based on the limited information available in the reports, even though there is a standard for reporting as shown in Table 4.1. Also, because it is likely that not all events that occurred were reported (Noble & Pronovost, 2010), it must be noted that the findings from this study does not represent the actual incidence of TMAE in primary and community care settings. There are also limitations related to the methods employed in this study. In the independent review of events, the level of agreement in assigning codes based on the sociotechnical dimensions was not strong. There was some degree of subjectivity to this process, where the perspectives of the reviewers were potentially influenced by their previous experiences and expertise in both clinical practice and informatics, as was evident in discussions when reviewing discrepancies. Yet in discussing the events, the two reviewers were quickly able to reach consensus. This begs the questions of both the purpose and the best approach to analyzing cases using this sociotechnical model. Methods where the aim is to establish consensus rather than assess independent agreement may be better suited to the applied, pragmatic nature of this work. Finally, the sample size of this study was small, and therefore may not have been representative of all types of TMAE, nor necessarily generalizable to other settings. 90 Conclusion This study used descriptive analysis to capture the type of events, and harm related to computer-related safety events reported from primary and community care settings via the PSLS reporting system. Two years of voluntarily reported safety event data from primary and community care settings (n=104) were analyzed and it was found that most events did not result in harm. Reports were most commonly generated from public health settings, and medication-related events were the most common type of adverse event reported. The sociotechnical analysis identified a total of 182 dimensions that were applicable, and the specific dimensions of Personnel and Workflow & Communication were most commonly recognized in relation to these reported events. Most events featured both technical and social dimensions, suggesting that the nature of these events is multifaceted, and that technology, users, and context are all important to consider in analyzing and addressing TMAE. The findings of this study provide an important contribution to advancing knowledge about the significance of TMAE within the contexts of primary and community care. 91 Chapter 5: Study 3: Addressing TMAE with an IKT Approach Introduction In this chapter, an integrated knowledge translation (IKT) approach related to the overall research project about technology-mediated adverse events (TMAE) in primary and community care is presented. The research project was part of a national cohort of IKT projects, supported by the Canadian Institutes of Health Research (CIHR) Health Systems Impact Fellowship (HSIF) program. The IKT approach involved an embedded researcher with expertise in patient safety, clinical education, and informatics, who was conducting the studies presented in this dissertation. The goal of the IKT approach was to build capacity within the Community Clinical Informatics (CCI) team at a large healthcare organization in Western Canada to incorporate patient safety into their work and to recognize and respond to TMAE within the organization. The structure of this chapter is as follows: First, a brief description of IKT, and a summary of how TMAE analysis has been used in organizations will be presented. This is followed by a description of the approach to IKT that was taken up in this research project, including guiding principles, knowledge translation activities, and a co-created process to address reports of TMAE. Traditional Knowledge Translation One of the main challenges in research is a lag between discovery of evidence, and its integration into practice. This is known as the ‘research-practice gap’, and it has been shown that it can take decades (i.e., between 17 and 54 years) between synthesis of evidence and integration into healthcare policy (Hanney et al., 2015; Morris, Wooding, & Grant, 2011). The research-practice gap is an immense international issue, and a variety of strategies have been put forward 92 to accelerate the uptake of evidence into practice across different countries (Nguyen et al., 2020; Robinson et al., 2020), including the HSIF program through CIHR (McMahon & Tamblyn, 2019). One of the contributors to the research-practice gap is the traditional approach to knowledge dissemination in which evidence is published in an academic journal at the completion of a study. In recent years, efforts have been made to shorten the research-practice gap through knowledge translation (KT)6, which is defined as “the synthesis, dissemination, exchange, and ethically sound application of knowledge to improve health, provide more effective health services and products, and strengthen the health care system” (Straus, Tetroe, & Graham, 2013, p. 3). KT is designed to move the dissemination process to the point-of-care by focusing the KT on a specific clinical problem, identifying relevant evidence-based literature, creating a process to change the previous practice, and monitoring (in an iterative manner) to assess if the KT process has improved healthcare outcomes. Integrated Knowledge Translation While KT has had some success, the research-practice gap persists and other strategies to reduce the gap have been devised. One of the frequently used strategies to reduce the gap in Canada is IKT (McMahon & Tamblyn, 2019; Nguyen et al., 2020). IKT refers to collaboration between researchers and knowledge users in a mutually beneficial research project. In IKT, knowledge users are active partners in conceptualising a research project, generating knowledge, and implementing research findings, in collaboration with researchers (Bowen & Graham, 2013; 6 KT is the term used in Canada. In other regions, terms such as ‘implementation science’, ‘research utilization’, dissemination and diffusion’, ‘resarch use’, or ‘knowledge transfer and uptake’ might be used (Straus, Tetroe, & Graham, 2009). 93 Gagliardi et al., 2016; Kothari, McCutcheon, & Graham, 2017). Focusing on both generating knowledge and increasing the impact of that knowledge, IKT is used to enhance research relevance, while simultaneously improving organizational operations (Gagliardi et al., 2016; Nguyen et al., 2020). While IKT is an increasingly popular approach in health services research, there is no singular defined method, and details about how decision-makers and other knowledge users have been involved throughout the research trajectory are not explicit in most IKT studies (Gagliardi et al., 2016; Gagliardi, Kothari, & Graham, 2017). While consensus on IKT methods in the health services context is yet to be established, there is growing evidence that IKT with an embedded researcher can help accelerate the application of research into practice. IKT Competencies Traditional health services and policy research competencies include: 1) Analysis and evaluation of health-related policies and programs; 2) Analysis of data, evidence, and critical thinking; 3) Understanding health systems and the policy making process; and 4) Knowledge translation, communication and brokerage (Bornstein et al., 2018). A recent evaluation of the inaugural cohort of the CIHR HSIF program, which provided a trainee researcher access to a health service organization, was shown to support researchers to develop the competencies that are needed to maximize the impact of research (McMahon, Brown, Bornstein, & Tamblyn, 2019). In addition to traditional health services and policy research competencies listed above, the group have identified “Enriched Core Competencies” for PhD graduates to be prepared for diverse career paths and to contribute to health system improvement, and place emphasis on ‘professional’ competencies, including: 1) Leadership, mentorship, and collaboration; 2) Change management and implementation; 3) Interdisciplinary work; 4) Project management; 5) Dialogue 94 and negotiation; and 6) Networking (Bornstein et al., 2018). From the perspective of participating researchers, IKT as supported by the HISF program “prioritizes impact and accelerates the process” (Sim et al., 2019, p. 327). IKT is emerging as a useful approach for researchers and knowledge users to partner and co-produce knowledge, accelerating the uptake and impact of research. Embedded Researcher in Current Project The structure of the HSIF program included a formal partnership between the researcher and the partnering health service organization. The application to the program required a research plan that was co-created by the research trainee, academic mentor, and health service organization mentor. Additionally, the program included a funding agreement, with the health service organization sharing the cost with CIHR. The program required the embedded researcher to spend 60-70% of their time, working on site, with the partnering organization providing work space and resources akin to what an employee would require. Building on the foundation of the HSIF program, this chapter describes the further adaptation and refinement of an IKT approach, specifically to address TMAE in primary and community care settings within the partnering organization. Management of TMAE in Healthcare Organizations As described earlier, in this dissertation TMAE refers to unintended harm, or potential harm, that involved a HIT-based system. Theories are emerging to address HIT safety within healthcare organizations (Borycki et al., 2016; Kushniruk et al., 2013; Sittig & Singh, 2017), yet there is still much to be learned about how TMAE are identified, analyzed, and remediated or 95 mitigated. While theoretical knowledge in this area is growing, integrating this work within health organizations as a formal infrastructure to improve HIT safety has yet to be realized (Kushniruk et al., 2013; Makeham et al., 2017; Sittig, Belmont, & Singh, 2018). As described in Chapter 2, TMAE have been a known problem in healthcare for over 15 years (Ash et al., 2004; Harrison et al., 2007; Koppel et al., 2005; Wachter, 2006), and as HIT continues to advance, new unintended adverse consequences also emerge (Sittig, Wright, Ash, & Singh, 2016). As technologies that are intended to improve care continue to be implemented, careful attention to the impact that they have on the quality of care is crucial (Bullard et al., 2007; Granja et al., 2018). Gap in Evidence for IKT about TMAE To date, no evidence could be identified that has examined the use of IKT to address the issue of TMAE. An IKT approach was selected because it supports the co-creation of research to address a specific contextual problem, focusing on generating new knowledge, and immediately applying that knowledge to achieve an impact on practice (Gagliardi & Dobrow, 2016). In this study, IKT offers the potential to not only better understand TMAE, but to develop practices to mitigate against TMAE, and to potentially reduce the research-practice gap. Research Project: Understanding and Mitigating TMAE The entire research project, including the research studies described in previous chapters, took place at a large healthcare organization in Western Canada. The research project was conceptualized in collaboration with organizational executive leaders for Community Clinical Informatics (CCI) and Quality and Patient Safety as part of an initial application for research 96 funding. The research project was designed to be collaborative and action-oriented to inform safe HIT practices in primary and community care settings. The collaborative, action-oriented approach both supported the work of the organization and enabled the generation of new knowledge that was anchored in the realities of clinical informatics practice. Knowledge Users Knowledge users are individuals, groups, or organizations that are positioned to use the research to inform their decisions (Canadian institute for Health Research, n.d.; Nguyen et al., 2020). Three knowledge user groups were involved in the research project: 1) organizational executive leadership; 2) a CCI team leader; and 3) members of the CCI team. Figure 5.1 below depicts the organizational structure of the knowledge users in relation to the research project. All knowledge users demonstrated a commitment to the project, with an openness to learning and collaborating. Figure 5.1 Knowledge Users 97 The organizational executive leadership knowledge users included two executive leaders of the CCI department, and one executive leader of the Quality and Patient Safety department. These three executive leaders were included as part of the research team. These leaders were highly committed and shared governance in the planning and oversight of the research project. The leader for the CCI team was a core knowledge user who facilitated entrée of the embedded researcher to the CCI team. The CCI team is a multidisciplinary team of approximately 15-20 employees that was established in late 2018. It was coincidental that the CCI team was created at the beginning of the research period. The CCI team is responsible for the clinical integration and operation of all HIT systems in primary and community care settings. Each team member has specific responsibilities in their portfolio. For example, the CCI educators are responsible for supporting clinical staff in using the clinical software systems. The CCI specialists are responsible for working with the HIT software development team to communicate changes that might be required to the HIT system, for example, practice policy changes. While the CCI team was the main focus for the IKT activities described in this chapter, the research project was continuously driven by collaboration among the research team (See Figure 5.1). The Role of the Embedded Researcher Alongside conducting the studies within the research project, the embedded researcher was integrated within the CCI team as a knowledge translator, supporting the synthesis, exchange, and application of knowledge to support their work (Canadian institute for Health Research, n.d.). The embedded researcher was onsite from October 2018 to March 2020 for four days a week, participating in the daily activities of the CCI team. The embedded researcher 98 supported team members to learn about patient safety, HIT safety, and apply their growing knowledge of TMAE to their work. Specific characteristics that the embedded researcher brought to this role included expertise in the following: clinical nursing, patient safety, educating point-of-care healthcare providers in patient safety, and knowledge of adult learning theory in the context of professional development of clinicians, as well as training in the IKT approach described above as part of the CIHR HSIF fellowship. Research Question The IKT approach to this research project was guided by the following question: How might researchers and knowledge users collaborate to co-create effective practices to address TMAE? The IKT approach used in this study was multifaceted, supported by a collection of guiding principles and theories, and comprised of a number of different activities aimed at supporting the movement of knowledge into action. The next section will provide an overview of the IKT approach, with a model to illustrate how the guiding perspectives were applied in conjunction with the knowledge translation strategies. IKT Approach to Address Technology Mediated Adverse Events A research project using IKT requires a carefully thought-out approach. Figure 5.2 below presents the model of the IKT approach that was co-created in this research project, illustrating the essential elements that contributed to the successful execution of the research plan. First, there were several infrastructure components required for the initiative to proceed, including dedicated funding, supportive leadership, engaged knowledge users, and co-location of an embedded researcher. Second, a number of guiding perspectives relevant to the field were placed 99 at the forefront, guiding the IKT. Third, the activities involved to support knowledge users in learning and applying their knowledge were multifaceted, responsive to the needs of the knowledge users, and relevant to the context of their work. The approach to IKT in this study included a wide variety of activities spanning over two years. The foundational infrastructure, supporting theoretical perspectives, and knowledge translation activities combine to produce an outcome that is unique to collaboration between researchers and knowledge users, and by design, will continue to support the knowledge users in accordance with their context. In this instance, the embedded researcher and the CCI team co-created a process to address TMAE through using sociotechnical analysis and safety huddle methods. 100 Figure 5.2 Conceptual Model: IKT Approach to Address TMAE 101 Guiding Perspectives A number of guiding perspectives informed the approach to IKT in this research project, including: Clinical Informatics, Sociotechnical Theory, Patient Safety, Improvement Science, Andragogy, and Front-Line Ownership. A brief description of each of these areas, and how they supported the IKT approach follows. Table 5.1 Guiding Perspectives Guiding Perspective Description Clinical Informatics • An applied, interdisciplinary field focusing on the use of HIT in healthcare. • Sits at the intersection of three domains: 1) Clinical Care; 2) Information and Communication Technology; and 3) The Health System (Cresswell, Bates, Wright, & Sheikh, 2017). • Clinical informaticians are well-positioned to analyze, design, implement, and evaluate HIT in the context of care delivery (Gardner et al., 2009). • Focused on understanding how HIT was operating in the primary and community care clinical settings, specifically learning from actual or potential threats to patient safety. Socio-technical theory • Regards technology and the people using technology as intertwined; they must be considered in relation to one another (Sittig & Singh, 2010, 2017). • Human factors (or human factors engineering) can be applied to understand the complex interplay between people and technology within the context of their environment. o Human factors is “a discipline dedicated to applying what we know about people’s strengths and limitations to the technologies, processes, and environments we live and work within to try to make these systems safer, more intuitive, and robust” (Cassano-Piché, Trbovich, Griffin, Ling Lin, & Easty, 2012, p.11). • Takes a comprehensive view of TMAE, considering both the problems and potential solutions within the context of the complex realities of practice where people are using HIT. 102 Table 5.1 Guiding Perspectives (Continued) Guiding Perspective Description Patient Safety Systems Thinking • Error is viewed as a product of faulty systems, as opposed to the fault of an individual person (Leape, 2021; Reason, 2000). • To prevent errors, the system must be addressed, not the individual (Leape, 2021; Reason, 2000). • Swiss Cheese Model as an explanatory metaphor for how a series of errors within a system may compound to produce an adverse event (Reason, Hollnagel, & Paries, 2006). o Additive perspective of risk informed the focus on identifying and responding to errors related to TMAE to prevent further harm. High Reliability Organization • An organization that is able to consistently achieve its intended outcomes, despite fluctuations and unpredictability in circumstances (Frankel, Haraden, Federico, & Lenoci-Edwards, 2017; Weick, Sutcliffe, & Obstfeld, 2008). • Collective mindfulness: a collection of processes to effectively identify and mitigate against errors: o Pre-occupation with failure, the continual looking for and reporting of hazards; o Reluctance to simplify, not accepting the obvious explanation for a failure; o Sensitivity to operations, paying attention to issues at the frontline; o Commitment to resilience, the ability to detect errors, react, and recover; o Deference to expertise, the flattening of the hierarchy so that the most qualified person is in charge, regardless of seniority (Leape, 2021; Weick et al., 2008). • Technology is viewed “less as an intractable technological imperative and more as a controllable option” (Weick et al., 2008, p.51). Culture of Safety • A shared commitment to achieving zero harm and a dedication to continually improving and innovating to promote safety (Leape, 2021). • Includes: o Informed culture: there are data available about safety incidents. o Reporting culture: people feel safe to report safety concerns and that reporting contributes to promoting safety (Leape, 2021; Reason, 2016). o Just culture: people are accountable for their actions; recognizes the role that the system can play when harm occurs, carefully balancing accountability with systems thinking, learning, and improving (Dekker, 2009; Leape, 2021). o Flexible culture: reconfigures as needed to address safety concerns, such as flattening a hierarchical structure. o Learning culture: an assumption that human nature is good and people are motivated to contribute to safety promotion (Reason, 2016). Psychological safety • The tacit belief that a work team is safe for interpersonal risk taking because there is a sense of trust and mutual respect among team members, and people feel comfortable being themselves (Edmondson, 1999). 103 Table 5.1 Guiding Perspectives (Continued) Guiding Perspective Description Organizational Improvement • The central law of improvement is “Every system is perfectly designed to deliver the results it produces” (Langley et al., 2009, p. 79). • The ability to make improvements comes from a combining of subject matter knowledge with profound knowledge: o Appreciation for a system; o Understanding variation; o Building knowledge; and o Human side of change (Langley et al., 2009). • Improving a system requires making changes within a system and this is done in an iterative fashion, focusing on learning, cycling through inductive and deductive thinking to study the impact of a change on a system. • Recognizing and remediating TMAE aims to improve care by making it safer. Front-line Ownership • Sustainability depends on the learners taking ownership of new knowledge and practices. • Emphasizes expertise and experience of the CCI team, and supporting team members to apply their learning and adopt new practices in a way that makes sense and is valuable from their perspective (Gardam, Gitterman, Rykert, Vicencio, & Bailey, 2017). • The role of the embedded researcher is to co-create learning experiences, and allow for flexibility in adoption of new practices (Plsek, 2017). Integrated Knowledge Translation Activities While there is emerging evidence that IKT can have a positive impact on addressing the research-practice gap (McMahon & Tamblyn, 2019; Sim et al., 2019), specific details as to how to conduct a study using an IKT approach are not well established (Gagliardi et al., 2016). The enablers, barriers, and outcomes from a scoping review of IKT studies summarized by Gagliardi and colleagues (2016), provided an initial framework to inform planning the approaches to IKT. In general, the role of the embedded researcher was to both conduct rigorous research, and be a ‘facilitator of learning’ (Knowles, Holton, & Swanson, 2012; Rogers, 1969), to support the translation and application of knowledge over the project timespan. This project incorporated multiple activities into a purposeful approach to IKT (Wensing, Bosch, & Grol, 2013). Figure 5.3 below illustrates the IKT activities that were carried out during the project. Each activity 104 involved multiple knowledge users, including executive leaders, team leaders, and members of the CCI team. The IKT activities were not as distinct or linear as the figure may depict, instead, the figure is intended to provide a visual summary of the activities over the course of the research project period. An overview of each of the IKT activities, including the involvement of the various knowledge users, is presented in Table 5.2 below. (For a detailed description of the IKT activities, see Appendix D). 105 Figure 5.3 Integrated Knowledge Translation Activities Timeline *Funding application April 2018, approved July 2018 - - - - The dotted lines around the rectangles indicates activities are not discrete nor mutually exclusive, rather overlapping and interlaced. Project Design*AssessmentRelationship BuildingOrganizational MeetingsAnalyses of TMAE reportsAd hoc supportSafety HuddlesFormal Learning SessionSustainmentDisseminationProject DesignOctober 2018 December 2020Formal Learning SessionCovid-19 Pandemic start 106 Table 5.2 Integrated Knowledge Translation Overview Activity Overview Participants Research project design Grant writing meetings, and revision meetings post-covid • Organizational executive leadership Organizational Meetings Regularly scheduled meetings to review organizational needs and research progress • Organizational executive leadership • CCI team leader • CCI team members Relationship Building Developing familiarity with knowledge users and the organization • Organizational executive leadership • CCI team leader • CCI team members Knowledge and Context Assessment Assessing opportunities to apply knowledge • Organizational executive leadership • CCI team leader • CCI team members • Clinical and support staff members Ad Hoc Support In the moment coaching, consultation, and teaching • Organizational executive leadership • CCI team leader • CCI team members Formal Learning Sessions Lectures and webinars, including case study activity • CCI team leader • CCI team members Note: the embedded researcher was involved in all activities CCI Team Process for Addressing TMAE The team process for addressing TMAE was developed through an iterative series of co-development activities. The process to address TMAE included a rapid sociotechnical analysis (Sittig & Singh, 2010, 2017) and a safety huddle, including a feedback mechanism to distribute learning from each event across the CCI team as well as back to the clinical setting, and a review of events in aggregate to inform broader decision-making. Details of this process are presented below and displayed in Figure 5.4. Analyses of TMAE Reports Leveraging the existing voluntary safety reporting system within the organization (the Patient Safety Learning System [PSLS]), event reports with ‘a computer involved’ were 107 extracted. Analysing these reports provided an experiential learning opportunity for knowledge users in applying the sociotechnical framework proposed by Sittig & Singh (2010, 2017). As described in Chapter 4 (section 4.3), reports from 2016-2018 (n=104) were extracted and coded using the sociotechnical framework by the embedded researcher and the executive leader for Clinical Informatics. Following that, 42 reports from 2018-2019 were extracted and used as practice events for CCI team members to gain experience in applying the sociotechnical framework and using the iterative process to achieve consensus described in Chapter 4. The embedded researcher then worked with the CCI team to iteratively co-create a process for receiving event reports in real-time starting in October 2019. Using this co-created process, the CCI team reviewed 43 reports of computer-related events from primary and community care settings that were reported in 2019-2020. At the initiation of these analyses, knowledge users expressed that they found the PSLS reporting system onerous to navigate. The embedded researcher put additional supports in place to help address this, including arranging a training session with the reporting system administrator and working with team members to develop a customized user guide (see Appendix E) to navigate the PSLS system to complete their analyses. The event review process that was ultimately developed included an internal workflow to analyze each individual event using the sociotechnical model, and a cumulative review of all events over time to monitor for patterns and inform the ongoing work of the CCI team. The process also included communicating with the person who reported the event and/or the local leader to share findings from an informatics perspective. See Figure 5.3 which depicts the overall process for addressing TMAE. 108 Safety Huddles for Learning from TMAE Analyses A safety huddle process was co-created with the CCI team to integrate the process of learning from the analyses of reported TMAE. Safety huddles are rapid and informal meetings designed to proactively identify and address safety concerns (Franklin et al., 2020; Glymph et al., 2015; Goldenhar, Brady, Sutcliffe, & Muething, 2013). There is some evidence of safety huddles as an effective strategy to promote HIT safety in a hospital setting (Menon et al., 2017). No reports of safety huddles in primary and community care were identified. However, given that safety huddles offer a systematic and efficient method to promote communication and psychological safety, increase awareness and accountability, and facilitate teamwork (Glymph et al., 2015; Goldenhar et al., 2013; Leape, 2021), it was assessed to be a potentially useful IKT activity for the CCI team to incorporate. The safety huddle activity was developed in an iterative fashion and modified to best fit the needs of the CCI team. Feedback on the process was continuously sought from CCI team members as they participated, and the structure of the safety huddles evolved to maximize efficiency and effectiveness. Initially, huddles were 30 minutes in duration and scheduled weekly, but after several weeks the frequency was reduced to bi-weekly to match the pace of incoming reports. At the beginning, the safety huddles were led by the embedded researcher, who guided participants in sharing and discussing the results from recent analyses of TMAE, linking the findings to practice, and reviewing the cumulative data. As the safety huddles progressed, the embedded researcher developed a set of guiding principles and an agenda to structure the discussion (see Appendix F), and CCI team members gradually assumed the role of leading the safety huddles. Safety huddles were initially designed to be in-person, however at the onset of the Covid-19 pandemic in March 2020, a virtual format was adopted. The guiding 109 principles and agenda template for the safety huddle were posted in the online meeting space by the program coordinator or designate at the beginning of each huddle. All team members were invited to attend as their schedule permitted, and anyone participating in the huddle could volunteer to guide the huddle discussion by following the five steps in the agenda. Figure 5.4 CCI Team Process for Addressing TMAE 110 Sustainability In concluding the project, there was a purposeful effort to promote the sustainability of the HIT safety initiative by focusing on standardizing, automating, and embedding the TMAE analysis and safety huddle activities within the routine work of the CCI team (Institute for Safe Medication Practices, 1999). The IKT activities were designed to be simple, and minimize time and resources required, while maximizing the team’s perceived value for time and effort spent (Kaminski, 2012). Additionally, the established workflow was designed to distribute the ownership across team members rather than be dependent on any one team member, role or expert (Maher, Gustafson, & Evans, 2010), as well as to promote a continuous experiential shared learning experience (Kolb, 1984). Discussion This chapter presented an IKT approach to addressing TMAE and leveraging an embedded researcher. This approach was guided by a variety of theoretical perspectives and principles, and included multiple IKT activities, tailored to the needs of the knowledge users. The IKT approach resulted in the co-creation and integration of a new process for analyzing and mitigating against TMAE. Additionally, this approach highlighted some of the required components for success in undertaking an IKT approach, and demonstrated how to rapidly apply research to inform practice. Team Process for Addressing TMAE The team process for addressing TMAE that was co-developed with the knowledge users in this study (Figure 5.4) was integrated into the ongoing work of the CCI team. Taking a 111 systematic approach and applying a sociotechnical framework was useful in analyzing TMAE, identifying multiple contributing factors, and highlighting new opportunities for improvement (Sittig & Singh, 2010, 2017). Safety huddles are recommended as a proactive approach to identifying, understanding, and addressing safety concerns related to HIT (Menon et al., 2017). There is value in informal learning processes around safety, to enable staff to raise concerns and actively contribute to improvement (Sujan, 2015). The practise of safety huddles spread the insights gleaned from TMAE analyses to the entire CCI team, supporting both immediate learning from each TMAE and an aggregation of knowledge based on patterns observed in the data over time. The co-creation of the team process to address TMAE was aligned with experiential learning theory, reflecting and abstracting knowledge from concrete experience, and applying that knowledge in a continuous cycle (Knowles et al., 2012; Kolb, 1984). Additionally, the CCI team’s process for addressing TMAE incorporated patient safety concepts such as human factors, just culture, and event reporting (Bartman, Bertoni, Merandi, Brady, & Bode, 2019; Leape, 2021). This process aligns with principles of high-reliability organizations, which have proven to support optimal care delivery (Oster & Deakins, 2018). The integration of individual and aggregate analyses over time allows examination of systems issues and supports more meaningful action plans in response (Charles Vincent et al., 2017). A notable aspect of the process to address TMAE was communication back to clinicians. In a study by Williams (2019), 13.7% of reported EHR-related safety events led to changes to improve the usability of the EHR. While the rates of change to the EHR was not measured in the current study, health organizations may benefit from adopting similar practices to promote HIT safety. Taking an IKT approach to implementing these new practices could promote effective implementation and sustainability by 112 tailoring the approach to the context and integrating new practices into existing work. Further, tracking any resulting transformation to EHR systems could be a straightforward way to measure impact. IKT Approach to Address TMAE In order to optimize for successful IKT, an overarching approach was created (Figure 5.2). The following sections highlight three aspects of the design and implementation of this model, including: the circumstances of collaboration, a stance of intentionality, and bidirectional knowledge transfer and andragogy. Circumstances of Collaboration In this instance, IKT proved to be well suited for examining the implications of technology in the delivery of healthcare, particularly for the monitoring of unintended consequences such as TMAE. There were many different factors that contributed to the execution of the IKT approach in this study. The circumstances around this initiative were highly conducive to a collaborative approach to research and applied learning, including having time and resources dedicated to the project (in the form of a funded, embedded, and co-located researcher), a well-developed research plan that was aligned with organizational priorities, and commitment and advocacy from knowledge users at all levels of the organization (Gagliardi & Dobrow, 2016). The coincidental timing of this project with the newly forming team may have also primed individuals’ readiness for change and willingness to engage (Choi & Ruona, 2011). Additionally, the physical co-location of the embedded researcher with the knowledge users 113 proved to be an invaluable aspect for supporting the development of relationships and facilitating IKT activities, including spontaneous opportunities for shared learning. Stance of Intentionality A research project using an IKT approach is also dependent upon a strong and consistent partnership throughout the entire process, from design to dissemination (Nguyen et al., 2020). In addition to the traditional academic knowledge and skills needed to conduct research, researchers operating in the context of an IKT approach require enhanced professional competencies such as leadership, change management, collaboration, negotiation, and project management to effectively contribute to health system improvements and make an impact with their research (Bornstein et al., 2018; McMahon et al., 2019). Decision makers and staff of the partnering organizations, may not be familiar with IKT or research methods in general, including the potential benefits and resource implications (Gagliardi & Dobrow, 2016). An essential component in a successful IKT initiative is taking the time at the outset to develop trust, mutual respect, and a clear understanding of the knowledge users’ needs, as well as their capacity and capabilities to engage (Lawrence, Bishop, & Curran, 2019). At the same time, both researchers and knowledge users need to be flexible and willing to adapt as the needs of the organization change over time, continuously communicating and negotiating how to best achieve their shared goals. Communication must be two-way with the embedded researcher remaining open and attentive to the needs of the organization and conveying back a clear, consistent message across all knowledge user groups (Gagliardi & Dobrow, 2016). Collaborations can be challenging, and for researchers leading an IKT approach, certain stances such as being flexible, comfortable in 114 the field, tolerant of less structure while maintaining rigorous standards, and being tenacious and creative, can all contribute to success in these endeavors (Rycroft-Malone et al., 2016). Bidirectional Knowledge Transfer and Andragogy There are a variety of methods and approaches that IKT can incorporate. While there are no established associations between IKT activities and favourable outcomes (Gagliardi et al., 2016), emerging evidence suggests that IKT may be a beneficial approach to research because it builds relationships and facilitates a bidirectional knowledge transfer between research and practice (McMahon et al., 2019; Sim et al., 2019). Based on this experience, it seems that it is important to not only consider what methods are used, but how these IKT methods are applied. In this instance of IKT, multiple, purposefully designed activities were delivered over time to facilitate learning. Taking an andragogical approach, where learning is viewed as a process, IKT activities in this project were designed to continuously build on learners’ existing knowledge, and incorporate multiple approaches in response to the inherent complexity of the system (Kolb, 1984; Plsek & Greenhalgh, 2001). Further to this, IKT activities also needed to go beyond what is conventionally thought to be education, and instead prioritize the applicability to the real-world challenges that knowledge users are facing, and integrate new knowledge into existing practices at a systems level (Institute for Safe Medication Practices, 2020; Soong & Shojania, 2020). The success of an IKT approach is also influenced by the competencies of the embedded researcher themselves. There are robust collections of competencies for supporting knowledge translation (Bornstein et al., 2018; Mallidou et al., 2018). However, seemingly absent from the existing literature were two important factors deemed to be impactful in this study: knowledge of 115 adult learning theory and facilitating front-line ownership of the work. From the perspective of the embedded researcher, these two additional factors were crucial to enable the IKT approach in this research project. Having experience and expertise in adult education theory proved to be beneficial in designing and delivering IKT activities. Additionally, the guiding perspectives (patient safety, sociotechnical systems theory) that informed the project served an important role in supporting IKT and making knowledge more relevant to knowledge users. The combination of education theory and facilitating front-line ownership worked to build the capabilities of knowledge users within the organization to address TMAE in their work. Overall, an IKT approach has the potential to create a ‘win-win’ situation for researchers and knowledge users, and a way to achieve this is through a thoughtfully designed approach. Embedded researchers need to carefully balance rigour and real-world applicability, separating out what is contextual from what is generalizable, and strive to maximize learning to contribute to the body of knowledge in the field and assist the organization at the same time (Kothari & Wathen, 2013). Using Evidence to Inform Practice and Generate Knowledge An IKT approach has the potential to reduce the research-practice gap. Some of the first TMAE were reported in acute care in 2005 (Koppel et al., 2005), and studies of TMAE in the context of primary and community care have been more recent (Magrabi et al., 2016). While healthcare research-practice timelines have sometimes taken decades (Hanney et al., 2015; Morris et al., 2011), uptake of TMAE mitigation strategies in this clinical informatics team took 116 place in 20207. Williams (2019) also demonstrated a rapid integration of learning from safety reporting; in analyzing 242 EHR-related safety events reported over one year, 30 of these events led to specific system changes to improve usability. While this study does not provide definitive results that there was an impact, the potential is there to reduce the research-practice gap using an IKT approach. The IKT methodology employed in this study generated a novel approach to analysing TMAE. To the author’s knowledge, this is the first study that has combined rapidly responding to event reporting with a sociotechnical analysis to mobilize clinical informaticians to address TMAE. This solution was generated in tandem with the embedded researcher and the CCI team to whom the issue is relevant. Additionally, an IKT approach supports collaboration between researchers and knowledge users to also contribute to the body of knowledge in a field. Immediately following the conclusion of the project in February 2021, a small group of knowledge users worked with the embedded researcher to present their review process and recent analyses of TMAE at a regional patient safety conference. Going forward, the findings from all studies conducted in this IKT partnership will be disseminated and knowledge users will be invited as co-authors or co-presenters for all dissemination activities. Limitations There are a number of limitations to this IKT project. It is challenging to convey any generalizable insights into effective IKT because it requires a constant balancing of the rigorous processes required of research with the needs and values of the organizational partner (Kothari & 7 Integrated knowledge translation activities were briefly paused upon the onset of the Covid-19 pandemic in 2020, however the project was well established at that point and able to be continued and integrated into the ongoing work of the team as it shifted in response to the pandemic. 117 Wathen, 2013; Rycroft-Malone et al., 2016). Hence, while IKT collaborations have become increasingly common in conducting research in healthcare, the specific strategies that achieve beneficial outcomes remain largely unknown (Gagliardi et al., 2016). The various approaches taken up in this instance may produce a different outcome when applied in an alternative setting. An additional limitation is that the Covid-19 pandemic forced the IKT activities to be adapted to an online format, and the impact of these changes is unknown. Furthermore, there was no longitudinal evaluation to assess if the practices initiated during the IKT period continued after the project ended, and what factors may relate to long-term sustainability following this IKT initiative. Directions for Future Research IKT has the potential to make a valuable contribution to the ongoing development of knowledge in clinical informatics. The field of clinical informatics is inherently an applied field with HIT knowledge integrated into the provision of healthcare services (Shortliffe, Detmer, & Munger, 2016). This is particularly true for the study of TMAE, where the events in clinical practice are the phenomena of interest. The field of HIT is advancing quickly and, concurrently, new unintended consequences may continue to surface. Bringing research endeavours into practice and having knowledge users shape and participate in research helps to expedite the process of generating and applying evidence. Future research using an IKT approach in this area could continue to offer mutual benefits to both researchers and knowledge users (Gagliardi et al., 2017), simultaneously advancing the field of clinical informatics while concurrently promoting HIT safety. Additionally, future research should evaluate the sustainability and impact of an IKT 118 approach such as this over the long term to further understand if, and how, there may be a lasting effect and what, if any, impact has been achieved. Conclusion This chapter presented an IKT approach to a research project concerning TMAE. The IKT approach was led by an embedded researcher and included multiple activities designed to integrate knowledge of HIT safety into the practices of a clinical informatics team. The goal of the IKT approach was to identify and develop methods that would build capacity within the team to incorporate patient safety into their work and to help them recognize and respond to TMAE within the organization. The detailed description of this IKT approach makes an important contribution, illustrating some of the key considerations for conducting an IKT study, as well as demonstrating IKT as a viable approach to addressing TMAE in practice. 119 Chapter 6: Study 4: “Workflow or Chaos or Whatever”: Team Perspectives about TMAE Introduction The IKT approach described in the previous chapter was tailored to the context of a clinical informatics team in primary and community care. Research has identified sociotechnical analysis and safety huddles as a potential strategy to promote EHR safety (Menon et al., 2017). However, no literature was identified that described clinical informatics specialists’ experiences with an IKT approach to identify a process to manage TMAE. Therefore, a qualitative descriptive study using focus groups (Jayasekara, 2012; Sandelowski, 2000) was conducted to understand the Community Clinical Informatics (CCI) team’s experiences of IKT activities related to TMAE. The findings convey some of the barriers, enablers, and outcomes related to IKT, specifically for addressing TMAE in primary and community care settings. Methods The study used a qualitative descriptive design to examine knowledge users’ experiences of IKT from the perspectives of the CCI team members. Focusing on the knowledge users’ accounts of participating in IKT, qualitative description was appropriate for capturing the meanings and interpretations that the informants ascribed to their experiences (Sandelowski, 2000, 2010). Aligned with a constructivist paradigm, which posits that human knowledge is subjective and socially constructed, the study design aimed to capture both the participants’ perspectives, and acknowledge the subjectivity and involvement of the researcher in constructing the data (Polit & Beck, 2020). In this study, having been embedded within the team for an extended period, the researcher was able to glean an in-depth understanding of the context 120 surrounding the participants. Data were collected and analyzed considering reflexivity throughout the process, attending to the unique positioning of the researcher, the circumstances surrounding the study, and the potential influences over knowledge construction (Polit & Beck, 2020). Research Question The study addressed the following research question: What are the knowledge users’ experiences of managing TMAE in primary and community care? Sampling, Sample, and Recruitment Purposive sampling was employed. This approach is suitable when researchers seek to understand the views of a specific group of people (Polit & Beck, 2020). The target sample consisted of members of the CCI team who had participated in IKT activities over the prior 12-24 months. In September 2020, all members of the clinical informatics team (N=16) were invited via email to participate in an online focus group to discuss their experiences in engaging with HIT safety. This invitation contained details about the purpose of the research and asked for their voluntary participation to share their insights and experience regarding patient safety and TMAE related to their clinical informatics role. The consent form was attached to the invitation email. Informants were asked to select a preferred time slot from a list of potential times. Those who volunteered were arranged into groups according to their availability. It was incidental that informants in leadership positions within the CCI team were grouped into one session together. Informants were given the details about their participation, including a request for their signed consent forms to be submitted in advance of participating in the focus group to ensure informed 121 consent was obtained prior to any data collection. Prior to conducting this study, an additional ethics approval was obtained from UBC Research Ethics Board and Vancouver Coastal Health Research Institute. Data Collection Due to the Covid-19 pandemic, all focus groups were held online via Zoom (Zoom Video Communications, Inc.). Prior to the focus group session, informants provided their personal and work demographic information via an online survey. Given the concurrent demands upon the team related to Covid-19, informants were not provided any interview questions in advance in an effort to minimize the demand on their time. At the beginning of each focus group, informants were reminded that they would be audio and video recorded, and expectations of informants were reviewed, including their option to withdraw their consent at any time, but with the proviso that even if they left the group, their data would be retained up to their point of leaving. No participants left any of the focus groups. Focus groups were facilitated by the embedded researcher using a semi-structured interview guide (see Appendix G). The questions in this framework were ordered to begin with broader topics and more general ideas, and then were narrowed down to more specific questions about the processes within the team and the informants’ individual actions as they related to patient safety and clinical informatics. The embedded researcher manually transcribed discussions verbatim, and provided informants with transcription to review and revise or withdraw any of their comments. All participants approved the transcripts and no revisions were made. 122 Data Analysis At the conclusion of each focus group, the researcher noted initial ideas related to patterns and considered potential codes for analysis in field notes. After the focus group discussions were transcribed, transcripts were imported into NVivo 12 software (QSR International). A thematic analysis approach was applied to analyze the data (Braun & Clarke, 2006). While descriptive qualitative methods typically employ content analysis methods, the analysis was not concerned with enumerating themes, but rather “balancing analytic narrative and illustrative extracts” (Braun & Clarke, 2006, p.96) and therefore, a thematic approach to analysis was selected (Sandelowski, 2000; Vaismoradi, Turunen, & Bondas, 2013). Thematic analysis is well-suited to addressing broad research questions and provides a flexible approach to remain ‘data-near’ (Sandelowski, 2010), searching across the data for patterns and allowing for both inductive and deductive approaches to the analysis. After becoming familiar with the data, a simple framework of three categories: barriers, enablers, and outcomes, based on Gagliardi et al.’s (2016) review of IKT approaches was used to provide initial structure to the analysis. The operational definitions for each of the three categories are presented below in Table 6.1. After the deductive coding to identify barriers, enablers, and outcomes, data were re-examined to inductively generate subthemes within these three overarching categories. Subcategories were developed and refined over several iterations, in consultation with an academic supervisor, Dr. Leanne Currie, in an effort to support descriptive and interpretive validity of the findings (Sandelowski, 2000). 123 Table 6.1 Operational Definitions of Analytic Categories Category Operational Definition Barrier Activities or conditions that may have impeded knowledge translation Enabler Activities or experiences that facilitated or promoted knowledge translation Outcome A product or result from engaging in IKT activities Rigour As described above, several processes were carried out to enhance rigour (Polit & Beck, 2020). Field notes were documented immediately following the focus groups so that the ideas would be fresh in the mind of the researcher. The embedded researcher carried out all transcriptions which ensured that nuances of the focus group discussions could be captured, and which aided in interpretation in order to identify ideas that were most important or most challenging to the participants. Reflexivity was used throughout the analysis process, and supported by the prolonged engagement that the embedded researcher had with the CCI team and the larger organisation (Polit & Beck, 2020). This deep knowledge allowed the researcher to understand the context of the statements and thereby make linkages across the concepts that arose during inductive coding. The researcher also acknowledged the circumstances surrounding the study including that it largely took place during a pandemic, and that the team was newly created. Other potential influences over knowledge construction include that the project was vetted by executive leadership and thus may have influenced the participants’ responses. However, the senior leadership were not invited to participate in the focus groups in order to remove this potential influence over responses. Finally, weekly meetings with the doctoral supervisor provided a team check, such that the researcher explained what the codes and themes meant, and through discussion was able to clearly articulate the meaning of the concepts and 124 themes. Themes from the preliminary analysis were also shared with the CCI team as a member check to confirm interpretation (Polit & Beck, 2020). Results Three one-hour focus groups were held in October and November 2020. A total of ten team members participated. Of the ten informants, half were Clinical Informatics Educators, and the other half held other roles on the team such as team leader or project manager. While the CCI team had only existed for two years, many team members had been in their roles, but on different teams, for several years. Five of the informants had been in their current role for less than two years, four had been in their current role for two to five years, and one had been in their current role for more than ten years. Nine of the informants had a clinical background, including six nurses, one physiotherapist, one occupational therapist, and one social worker. The team member who did not have a clinical background had been working in the healthcare sector for more than ten years. The informants’ previous work experience in their respective clinical roles, before taking on an informatics role, ranged from 0-24 years, with an average of 9.7 years. Reflecting on their participation in IKT activities over the course of the project, informants shared a number of barriers and enablers related to their experiences of learning about and developing strategies to address TMAE. They also described outcomes, such as new learning and capabilities related to having participated as knowledge users in the IKT activities. Table 6.2 provides an overview of the barriers, enablers, and outcomes that informants identified, and a description of each item, including excerpts from the data, follows. Details about the person sharing the included quotes are limited to preserve the anonymity of the informants. 125 Table 6.2 Categories and Sub-Categories of Findings Category Sub-Categories Barriers • Limited prior knowledge of HIT safety • Previous assumptions and biases • Competing priorities and organizational barriers • Difficulty with reporting system and processes • Limited number of reports for learning Enablers • Participating in learning sessions • Gaining experience analyzing TMAE • Participating in safety huddles • Role modelling and leadership from embedded researcher Outcomes • Increased ownership and interest in HIT safety • Developed a sociotechnical systems perspective • Thinking differently about TMAE • Developed ability to identify and analyze TMAE • Applied learning to mitigate against TMAE • Enhanced communication within the team • Informed future work and strategic planning • Promoted a culture of safety • Increased empathy for user perspectives Barriers The informants identified several barriers to understanding and applying methods to address TMAE including baseline lack of knowledge, baseline biases, organizational pressures, and challenges related to event reporting. Limited prior knowledge of HIT safety. One challenge that informants commonly identified was having little or no knowledge related to TMAE or patient safety at the outset of the project. As one informant reported: “My experience with technology safety was pretty limited prior to all of this… I didn’t really know much at all.” More specifically, informants shared that they had a limited understanding of the impact HIT systems could have at the point-of-care. For example, one person who was one of the most experienced in both clinical and informatics roles stated, “I had no idea what was going on 126 for people at the front-line level with whatever they consider a computer incident.” This limited initial understanding elucidates the learning needs within the team that needed to be addressed to begin to incorporate HIT safety practices into their work. Previous assumptions and biases. Informants also shared different assumptions and biases about safety and TMAE that they had prior to participating in this IKT initiative. Some were unaware that HIT could introduce risks to safety, as one informant stated, “Technology was always supposed to be something that would make things safer, right? You streamline some processes, you make some things maybe, more automatic, take out some of the human element – that should be safer.” Others expressed they were initially sceptical as to whether this topic was relevant to the work of the CCI team. For example, one informant who had been with the CCI team for two years stated: “I felt like it wasn’t really my role or responsibility, to be identifying them [TMAE].” Another informant shared that they had previously assumed another department of “internal auditors” was responsible for addressing problems related to HIT, and now they saw the value in having a clinical perspective, “focusing on solutions, rather than, oh it’s just a number”. Informants also described their initial attitudes toward TMAE as “narrow”, “judgey” or “blaming”, articulating that they previously dismissed TMAE as ‘user error.’ One educator described this shift away from a perspective of blame as, “not just dismissing it as ok, this one person screwed up this one thing, but more than likely, that person isn’t the only one who’s maybe made that same mistake”. Another educator explained how their perspectives on TMAE moved towards curiosity over the course of the project: 127 I came from a narrow point of view, you [sic: the end-user] did something wrong, the [HIT] system didn’t work for you, what was that about? And now it’s more about, more curious – what else happened? How could we support you? What would’ve filled the gaps to make this less of an incident? Competing priorities and organizational barriers. Beyond establishing foundational knowledge, the capacity and capabilities to address TMAE within the CCI team were challenged by competing priorities and organizational barriers. One informant who had worked within the organization for four years perceived this as a systems issue: The whole organization, I guess, gets caught up in all these new initiatives, all these new projects, all these things you know make things better you know, but learning from the past and all these safety incidents… I feel like they get brushed aside. Another team member who had been with the organization for 15 years, and with the CCI team for two years, described recognizing that there was a need for a focus on HIT safety, but struggling with knowing how to proceed, “It just seemed too big to wrap our hands around, without some support.” Difficulty with reporting system and processes. A further challenge that informants described was related to the internal safety reporting system (i.e., the PSLS system) and processes, including concerns that it is onerous to use and therefore is underutilized. One person who had extensive clinical experience prior to joining the 128 CCI team, empathized with a clinician’s perspective, questioning the usefulness of the PSLS reporting system, “Coming from the mindset of a clinician, you’re busy you know, am I going to take the time to do a [report], what’s the value there?”. On a broader scale, other informants recognized some knowledge gaps related to using the reporting system. As one person commented, “There’s still a lot to do just in developing the reporting culture around technology needs in community.” More specifically, in relation to TMAE, an informant noted how a lack of understanding by point-of-care staff of how to use the voluntary reporting system for ‘computer’ issues affects the CCI team’s ability to respond and effectively address TMAE, “People don’t really understand what [an HIT] system-related error really is, and so a lot get put into that [PSLS] system that may not be appropriate for our eyes.” Limited number of reports for learning. Informants were also concerned that underutilization of the PSLS reporting system meant that issues reported in the TMAE reports were just the tip of the iceberg, “There’s so many safety issues that we don’t know about, things that are actually happening that aren’t being reported.” The challenges with the reporting system impeded their ability to address HIT issues because fewer reports meant that they had fewer opportunities for learning through analyzing these TMAE. Another informant made an analogy to the concurrent Covid-19 pandemic: It’s just like the COVID out there right now, there’s probably more cases than there actually are, we just don’t know about them…what we don’t know, is how to actually properly capture that all and encourage people to come forward when they have an issue. 129 Enablers Informants reported some key aspects of the IKT activities as enablers to learning about TMAE and mobilizing that learning in practice including, participating in learning sessions and safety huddles, the hands-on experience of analyzing TMAE, and their observations of the embedded researcher as a role-model. Participating in learning sessions. Team members identified the learning sessions as supportive of their learning about TMAE, describing the lecture components as having “collapsed all the salient points into a quick, easy to understand, salient presentation.” The readings and resource components of the learning sessions were identified as helpful as well: I don’t know that there’s often opportunity to bring in scientific literature into our day-to-day jobs so I think that was a great opportunity, to hear and to see what’s happening in the academic realm and consider its application to practice. Informants highlighted the case study from the learning sessions as a useful and effective tool to begin to see the complexity and appreciate a sociotechnical systems perspective in looking at TMAE. As one clinical informatics educator who had been with the organization for 3.5 years stated: That article [case study] was very, very engaging and very interesting and you could see how like, just to see the break down like that… and it again, made me aware of all these little things that, you know, can go wrong or have to go wrong to lead to something like this, and how the [HIT] system played into it at each sort of step. 130 Gaining experience analyzing TMAE. Informants also shared that the experience of analyzing TMAE was valuable in gaining insight into how to apply what is gleaned from the analysis to make improvements. As one informant who had only been on the CCI team for one year explained, “Actually receiving the [PSLS reports] and actually doing the investigative work teaches you a lot about the [HIT] systems… I enjoyed doing that and then thinking about how it could be better in the long run.” They went on to explain how this experience provided an opportunity to identify what sociotechnical dimensions may contribute to TMAE in the clinical areas that they worked with, “You’re not just thinking about the actual documentation system, but you’re thinking about all the systems around that, like whether it be a workflow or chaos or whatever it is that contributed to that scenario.” Participating in safety huddles. Informants also explained that the experience of sharing the TMAE analyses with other members of the team and participating in the safety huddles was supportive to their learning around HIT safety in that this activity provided a peer learning experience. As one of the educators stated: I actually found it [safety huddles] really informative, especially having other people there that use other [HIT] systems, and again, it’s someone else’s perspective and how they’re reading the situation and what I can learn from that other person that I’m working with. 131 Another team member expressed an appreciation for safety huddles as a new venue for communication among the team members, and found them worth the time and effort for the team: I found that making space for us, like dedicated time and focus, to talk about these concerns that we have, or the patient safety events that have occurred… I’d never had that experience before and I found it was so helpful talking as a group about what we found or what those problems that were being reported were about…I found that it was easy to actually make the time and spend the effort to do that. You know we are all busy, but I think in the long run it’s all going to do us well as an organization, and as a team, to continue that. Role modelling and leadership from embedded researcher. Informants also identified the role of the embedded researcher was supportive to their learning and facilitated change within the team’s practices. One informant who had been with the team for two years described the key function of the embedded researcher as initiating a focus on HIT safety, “I think we needed someone to come in and really help to set the tone and set that framework… and so it’s been learning. I think the last year has been learning across the board.” Building on this, there was also recognition of the embedded researcher as a champion for HIT safety, as a different informant explained, “It’s helped really mobilize the team in that direction and create more of a team sort of focus on working with these teams committed to safety-related issues.” One educator from the team described the embedded researcher as a role model for how to approach TMAE, having “instilled curiosity” in the team, noting “we can apply that as well”. Finally, there was also a recognition of how to integrate an emphasis on HIT safety into the work 132 of the CCI team, as one long-standing member of the team noted, “what I didn’t realize before is really how well this conversation fits within an organizational structure and within a team structure.” Outcomes A variety of different outcomes from the IKT activities were surfaced in the focus group discussions, including increased ownership, learning, and responsiveness in relation to TMAE. Increased ownership and interest in HIT safety. Several informants shared an increased sense of personal interest in the topic of HIT safety. As one team member explained, “It’s given me an appreciation and actual interest in safety and how that pertains to design of [HIT] systems and how we interact with the [HIT] systems.” Further to this, one informant with eight years at the organization, described having an increased sense of both personal and team ownership in relation to HIT safety, “Not only is it my role and responsibility, but we’re really well positioned to identify and sort of bridge between practice and workflows.” Informants recognized the role their team plays within the organization in supporting the delivery of care, as one person with extensive clinical and informatics experience noted, “Having a good understanding of why we exist as informatics… it’s not just for the users, although that’s important, it’s also for the patients and reducing risk…so it all ties together… that kind of holistic view.” 133 Developed a sociotechnical systems perspective. Informants shared new insights into their work, based on learning about sociotechnical systems theory as it applies to informatics. They particularly had an increased appreciation for the relationships among the technology, the users, and the context in which these are situated. As one educator with 24 years’ experience explained: I sort of think that technology doesn’t take into account the human being. It’s just, technology is a set of algorithms, it’s a set of stuff that’s written by a developer who tries to take in all the considerations possible. But you can’t take in all the considerations of a human being, and how a human being will respond to certain situations, or certain pieces of technology. This was echoed by another educator with 3.5 years’ experience, who stated: Our technology is only as good as how people understand it, and so the education piece around it and you know, understanding the workflow and how to actually apply it and use our [HIT] systems is so important… because the [HIT] system could be working as designed, but if people don’t know how to use it properly, then it just leads to a lot of problems. Related to this, the notion of human factors as it applies to safe HIT use was also a new concept for many team members, as one informant explained, “I never knew this existed, human factors –and now I am seeing that there is a whole theory behind it, there’s a lot to learn about, there’s best practices in design, there’s all these things that I had no idea even existed.” 134 Thinking differently about TMAE. Building on their knowledge of the sociotechnical perspective, informants demonstrated an increased awareness of the factors and circumstances that may increase the risks for TMAE. As one educator with one year of experience in the CCI team highlighted, before participating in the IKT activities, they “had just kind of considered the obvious errors, like with a malfunction or with a bug or something like that”. However, within the focus group discussion, this educator went on to articulate that their awareness of safety risks had grown beyond the technical aspects of HIT, noting “Technology’s not infallible. There’s so many factors and it’s quite complex, and it’s given me kind of, an appreciation for…the whole topic, and it makes me think about problems in a different way.” Similarly, another informant, also an educator with one year on the CCI team, described how their view of risks for TMAE had expanded beyond just focusing on the HIT user, “It’s not simply…one person did something wrong. There’s so many different things – there’s the workflow, the human factor, was it the actual user interface – all those different subcategories.” Developed ability to identify and analyze TMAE. The focus group discussions also included examples of how informants had applied what they learned to their work, and developed the ability to identify and analyse TMAE within the organization. First, amongst the informants, there was a greater recognition of TMAE, or learning “just what is a safety event” as one of the more experienced members of the team put it. Further to this, several people described adopting the practice of systematically analyzing TMAE. For example, one person explained how they learned to “think of [TMAE] in those different dimensions that we learnt about with the sociotechnical model, just being able to, like, 135 think of it in a framework like that, in sort of a structured way, to help break it down.” Another informant shared how taking a systematic approach had changed their thinking with regard to TMAE, “It does help us see sort of where perhaps a gap was, with a reported [TMAE]. So before we just knew there was an issue, but… I wasn’t looking at it as all these different sort of levels.” Similarly, another informant explained their approach to analyzing TMAE, “What I’ve learned is how to break it up. Was it the [HIT] system? … Was it the workflow? The process?” Applied learning to mitigate against TMAE. Going a step further, informants also articulated how they applied what they had learned to be proactive in their work and try to prevent TMAE from occurring in the future. One educator explained how they have observed patterns over time from the analyses of TMAE and this had increased their awareness of the potential for future risks: I’m actually already starting to see some patterns, and starting to think about the complexities of some of these [HIT] systems. Just this one little tiny little move can make a big change, can put someone at risk, and it shouldn’t be that easy to put a client at risk. Another educator described having an increased awareness of patterns as well, and how this informed their thinking around mitigating future issues, “I think being aware of the patterns, and being aware of the common sort of issues is really helpful in terms of thinking about future solutions and making sure that those problems can’t be easily replicated.” Further to this, informants also described how they had been able to communicate concerns about HIT safety in the context of their work. As one of the more experienced team members explained, “I didn’t always have the language to describe why something in the [HIT] system was a problem, but 136 that information with systems thinking really helped me frame those conversations.” Another informant described how the CCI team has begun to take a more proactive approach to safety, “I think it’s also changed the conversation around, just when we’re discussing [HIT] system changes or potential projects that we may undertake, is just the safety risks factors. Having more general dialogue around that… just being more proactive.” Enhanced communication within the team. Within the focus group discussions, there was also mention of how sharing their experiences addressing TMAE worked to enhance the communication within the CCI team, and further expanded their awareness of TMAE or potential safety concerns. Within the team, there were separate groups that focused on different HIT systems and different clinical areas in primary and community care, which could sometimes create siloed communication. Team members expressed that participating in the IKT activities opened up new communication channels and supported an exchange of learning across the different teams. One educator, whose work was focused on one particular clinical area shared: It was great awareness to hear what was happening in other places… kind of raising that awareness so that if we see something similar in our clinical area, it just kind of, alerts you to look out for things that you maybe would’ve never considered… and all of a sudden your level of awareness is there.” Another informant highlighted how increasing the communication across the small teams within the CCI team helps to provide better support to the clinicians or HIT users: 137 Clinicians are interacting with many systems, and many applications, and many types of technologies, that in fact, we need those opportunities to speak with our colleagues who lead or support other [HIT] programs so that we really get a sense of what those safety events meant. Informed future work and strategic planning. On a broader scale, informants also shared that the IKT activities informed their perspectives on the role of the CCI team within the organization. As one of the more senior team members asserted: I think that our team is perfectly positioned to handle, be handlers of technology related reports, and make sure we close the loop. I think it needs to be part of our work and just have it as a regular ongoing piece of work that we do and…service that we provide to the organization. Additionally, informants shared insights into how their knowledge of HIT safety relates to organizational decision-making and strategic planning. One informant in an educator role considered how the CCI team can contribute to decisions about HIT: I think … whether it be just the organization, or operationally within a clinic, and they make a request for a change in a [HIT] system, or it’s a bigger change, like made at a higher level, even at the [executive] level, maybe they’re not making the best decision because we’re not providing them with the best information about our clinical [HIT] systems. 138 Another educator contemplated how ongoing learnings from TMAE might, and might not, be applicable to future changes to HIT within the organization: “If we go to a new [HIT] system, I worry about how we’ve learned how to kind of look for and educate and provide more support for certain aspects from the incidents that we’ve already received. And then we move to a new [HIT] system, which we think may solve a lot of these problems, but we’re going to be creating almost, new problems, and how to sort of come in from the front … taking these learnings with us.” Promoted a culture of safety. From a patient safety perspective, the focus group discussions also surfaced some interesting insights. There was an increased emphasis on a culture of safety and leveraging what they learned from analyzing TMAE to make improvements. As one informant explained: I think as a team member in clinical informatics, instead of focusing on the mistakes… thinking of the next time. What can be done better? How can it be resolved much better? It’s always the next time, it’s always learning, and it’s always a different situation. But it’s probably easier if you have the open mind to actually really understand…it’s just having that understanding that it’s not always the person at fault, it’s not always the system at fault, it could be a combination of everything. And again, what do we do next? How do we learn from this? 139 Increased empathy for user perspectives Empathy for clinicians and HIT users was a commonly expressed sentiment in the focus group discussions. Team members reflected on their previous experiences as clinical care providers and described making a greater effort to take this into account when considering the functions and dysfunctions of HIT in a clinical context. As one educator commented: When we get these reports now, I’m trying to think like the clinician. I’m not working as a clinician anymore, but I am trying to put myself in their shoes – what are all these other surrounding factors, what led them to report about this? Pairing this empathy with their knowledge of TMAE, another informant demonstrated new insight into the users’ experiences with HIT: And so they think it’s one way and then an error happens because they misread something or they didn’t know to check somewhere, and… I just felt like I never clued into how much the design can really impact that front end user experience. Further to this, another informant contextualized the challenges clinicians and HIT users may face in using HIT: I’m just thinking about really, the environment that people are working in, and the complexity of that… the environment might be…very chaotic, and then we’re asking them to do something very complex in the [HIT] system. I think that is a safety concern… I think that’s where it would be very easy to have errors, obviously. 140 Discussion This chapter presented knowledge users’ experiences of IKT activities related to TMAE in primary and community care. The findings, implications, and limitations of this study are discussed below. Key Findings In this study, knowledge users reported that participating in the IKT activities had an impact on their work in relation to TMAE. Informants in the focus groups shared a variety of sentiments that illustrated the barriers, enablers, and outcomes of the IKT activities. The key findings in each of these areas will be summarized and reflected upon in relation to findings and insights from the literature. Barriers to Uptake of HIT Safety From the perspective of the knowledge users, some of the barriers that proved challenging for IKT included the lack of knowledge about HIT safety and previous assumptions they carried at the outset of the project. For patient safety in general, a lack of knowledge about incident reporting and assumptions about the value or repercussions are known barriers to reporting (Sujan, 2015). Having spent time at the beginning of the project building relationships and assessing needs within the team, these gaps in knowledge were identified by the embedded researcher as well. Recognizing this limited understanding of HIT safety, IKT activities were focused on fundamental topics in HIT safety and contextualizing learning within their existing work. Another barrier identified through the focus groups was competing priorities and organizational barriers. This challenge is echoed in the literature on IKT (Bowen & Graham, 141 2013; Gagliardi et al., 2016). While this may be a perennial challenge in healthcare, the embedded researcher addressed this barrier from the outset of the project by strategically aligning and incorporating IKT activities into the current priorities of the knowledge users as much as possible. This was a foundational step in the IKT process, as Rycroft-Malone et al. explain, “mutual learning and better appreciation about each other’s perspectives and contributions may lead to the potential of both better processes and outcomes through the generation of more relevant, applicable knowledge” (Rycroft-Malone et al., 2016, p.222). The focus group discussions also surfaced challenges with the existing safety reporting systems, processes, and underutilization of the PSLS system as a barrier to taking up TMAE activities. Standard practice for PSLS in general was for events with little or no harm to be reported and addressed at a local level, while more severe events are immediately escalated to organizational leaders, and the aggregate data over time is reported and used to inform operational decision-making. Knowledge users were concerned that clinicians needed to more explicitly understand the importance of reporting, and stressed the value of ‘closing the loop’ to ensure the person who reported the event is aware of the implications their report had from an informatics perspective. The costs and benefits of safety reporting are debated in the literature. Insufficient action following a safety report is thought to negatively affect clinicians’ commitment to the reporting process (Mitchell et al., 2016). Macrae (2016) argues “we collect too much and do too little” (p.4), explaining that while the technical infrastructure for safety reporting has been established in many healthcare organizations, the requisite processes of investigation and improvement have been overshadowed. While other research suggests, rather than reporting, clinicians are more prone to apply quick fixes or work-arounds to system failures, rather than reporting issues to trigger more in-depth analysis and sustainable solutions (Hewitt & 142 Chreim, 2015; Paparella, 2018; Tucker & Edmondson, 2003). In any case, it seems that safety reporting is yet to achieve its full potential (Mitchell et al., 2016), and for TMAE in particular, the latency of safety issues related to technology may pose further challenges to effective reporting, with near misses and errors being dismissed or going undetected (Borycki, 2013). Beyond this, there are some additional barriers related to IKT identified by Gagliardi et al. (2016) that were not mentioned in these focus group discussions. A number of these were not applicable in the current context, such as willingness to take part in IKT activities, geographical distance, group/personal dynamics, and values and attitudes toward IKT, as the knowledge users from the CCI team involved in this study were largely keen to participate. Knowledge users in executive positions were also highly collaborative and generous with their time. Also, because of the research funding supporting this work, dedicated resources and funding were not perceived to be a barrier in this study (Gagliardi et al., 2016; Gagliardi & Dobrow, 2016). Supportive Learning Environment and Openness to New Ideas Enablers to IKT as articulated by the knowledge users in this study focussed primarily on the participatory and experiential activities. Facilitating informal, locally owned processes for learning around safety has been shown to enable staff to raise concerns and actively contribute to improvement (Sujan, 2015). While there is a growing collection of research studies that have applied the Sittig & Singh Sociotechnical model in analyzing safety concerns (Castro et al., 2016; Kang, Wang, Zhou, Miao, & Gong, 2017; Meeks et al., 2014; Menon et al., 2017), no literature that described incorporating the framework into experiential learning activities or embedding it into clinical informatics work processes in real-time was identified. Safety huddles specifically are thought to be transformational in shifting attitudes and practices related to safety, 143 providing a “reliable framework for interdisciplinary communication and action” (Brass, Olney, Glimp, Lemaire, & Kingston, 2018), however the evidence to support this is largely anecdotal (Franklin et al., 2020). Menon et al. (2017) used safety huddles to address EHR safety concerns in a hospital setting, and while the format differed from this study, the huddles promoted a culture of safety, providing a venue for open communication about safety concerns, and facilitating learning and improvement, which was also found in the current study. Additionally, informants also noted the role the embedded researcher played on the team to ‘set the tone’, ‘mobilize the team’, and ‘instil curiosity’ supported their learning in the project. Some key aspects of the IKT approach may have supported the efficacy of the embedded researcher (described in section 5.4.2 in the previous chapter). In working to establish a culture of safety, the embedded researcher initially participated in all TMAE analyses to highlight opportunities for learning and improving, as opposed to accountability or punitive action, and ensured that feedback from CCI team analyses were shared both with the team internally, as well as to external parties as appropriate (Health Quality & Safety Commission New Zealand, 2016). For an embedded researcher, fostering daily working relationships is thought to support engagement and build capacity in the team, applying learning into existing processes and promoting sustainable practices (Vindrola-Padros, Pape, Utley, & Fulop, 2017). A concerted effort was made in the initial stages of the project to develop a strong, collaborative partnership with knowledge users at all levels of the organization (Gagliardi et al., 2017; Rycroft-Malone et al., 2016). At the conclusion of the project, efforts were made to promote the sustainability of the project by integrating new practices into the team workflows, and stepping back to allow knowledge users’ to carry out new practices with minimal support from the embedded researcher (Maher et al., 2010) 144 The focus groups captured several aspects to indicate that the CCI team exhibited the knowledge, skills and attitudes of an effective team (Baker et al., 2006), which in turn, support the notion of a high-reliability team (Wilson, Burke, Priest, & Salas, 2005). For example, the participants found the safety huddles a ‘good use of their time’. Further to this, the context surrounding this research project (as described in Chapter 5) created a ‘fertile environment for IKT’ as the CIHR HSIF program supporting this initiative provided dedicated resources, endorsement and collaboration from leaders, and multiple opportunities for interactions between researchers and knowledge users (Gagliardi & Dobrow, 2016; Rycroft-Malone et al., 2016). The focus group participants recognised the value of having ‘academic literature brought into their realm’. Effective Knowledge Mobilisation The findings from this study include a number of positive outcomes in terms of moving knowledge into action. The IKT approach in this resesearch seemed to support a “group-level identity transformation”, incorporating different professional perspectives, adding value, and acting as a lever for system-wide, evidence-informed sustainable change (Kothari & Wathen, 2013). Informants described being more knowledgeable and engaged in HIT safety issues in their work. They developed their knowledge base in clinical informatics, with an increased recognition of some of the perennial problems related to HIT (Abbott & Weinger, 2020), and the organizational and social issues that surround the use of HIT in healthcare settings (Kuziemsky, 2015; Kuziemsky et al., 2013). Informants also expressed greater ownership with regard to safety, identifying the CCI team as ‘perfectly positioned’ to be identifying and mitigating against TMAE. This is echoed in the literature with EHR safety ascribed to be a shared responsibility 145 among key stakeholders among EHR developers, healthcare organizations and users, and government regulators (Sittig, Belmont, et al., 2018). Clinical informatics, with its emphasis on bridging technical and clinical perspectives can play a central role in facilitating efforts to improve safety (Cresswell et al., 2017; Gardner et al., 2009). The co-created process for addressing TMAE produced immediate and ongoing insights related to HIT safety that the CCI team could leverage to inform operational decision-making within the organization. While no formal analysis of changes to practice were measured, the analysis and learning from a reported TMAE may have prompted a change in practice, or in the HIT system. For example, one participant indicated that they ‘make sure to close the loop’ by interacting with the end-user who is using the system that had a reported TMAE. Similarly, by leveraging voluntary safety reporting for quality assurance, Williams (2019), identified 242 EHR-related safety events analyzed by nurse informaticians, 30 of which led to specific system changes to improve usability. This study, along with the current study, show that a clinically-focused informatics team is well-positioned to take on this work of ‘closing the loop.’ In this study, informants expressed an appreciation for the structure that the Sittig & Singh analytic framework brought to conducting an analysis and that they learned to look for patterns in reviewing reported TMAE. The process to address TMAE that the team developed was an adaptive approach, based on experiential cycles of learning, from which they gradually developed new insights and expanded their collective expertise on HIT safety (Plsek & Greenhalgh, 2001). Demonstrating a thoughtful approach to safety, the findings from this study indicate that informants’ perspectives of TMAE moved beyond a reactive ‘find and fix’ approach, embracing complexity to learn more about how to support, augment, and encourage ways for things to go right, as recommended by Braithwaite et al. (2015). For example, 146 partcipants mentioned planning and designing for future HIT systems with the lessons from previous safety reports in mind. Findings from this study also indicate enhanced communication, and helped team members develop a more empathetic approach in supporting clinicians using HIT. Safety huddles specifically, were thought to have improved communication within the team, as well as inform their perspectives in all aspects of their work, including planning for the future HIT-related needs of the organization (Goldenhar et al., 2013; Menon et al., 2017). The shifting attitudes from ‘blamey’ to ‘curiosity’ about the context of the TMAE reflect an increasing incorporation of the principles of high reliability including: being sensitive to operations, preoccupied with failure, deferring to expertise, reluctant to simplify, and committed to resilience (Oster & Deakins, 2018), all in an effort to continually improve. While incident reporting is not without its limitations (Shojania & Marang-van de Mheen, 2020), reporting can effectively contribute to participatory learning, improve practice, and promote safer care (de Kam et al., 2020). According to (Gagliardi et al., 2016) the reported outcomes of an IKT approach in general, are a mixture of positive and sub-optimal outcomes, largely focused on the process of IKT, as opposed to the generation of new evidence. For example, outcomes of IKT identified in the scoping review by Gagliardi et al. (2016) include building capacity related to conducting research and taking an IKT approach, and an enhanced appreciation for research and collaboration. While the informants articulated an appreciation for ‘scientific literature’ they did not mention individual capacity for conducting research. Less favourable outcomes identified in the review by Gagliardi include failures in: engaging stakeholders, collaborating, producing research or influencing decision-making (Gagliardi et al., 2016). This study did not inquire about 147 the knowledge users’ perceptions of conducting research and there were no findings related to this, rather the informants’ comments were largely focused on the outcomes related to conducting their work in clinical informatics. Again, this is likely because of the specific group of knowledge users in this study, as the organizational leaders who were directly involved in the co-design and oversight of the entire research project were not included in the focus groups. It is also possible that, despite being embedded as part of the team, the temporary nature of the funding for the embedded researcher perpetuated the idea that research is separate from the team’s work. Limitations A main limitation of this study is that the embedded researcher functioned as both the lead of the initiative as well as the interviewer for this project. It is possible that informants’ responses were influenced by social desirability (Polit & Beck, 2020). Yet, the participants’ responses also showed vulnerability. For example, several respondents indicated that they had been ‘judgey’ about end-users’ errors in the past. This level of candor suggests that the social response bias may have been minimal, and possibly was overcome by prolonged engagement given the long duration that the embedded researcher participated with the team. This study also only assessed the experiences of knowledge users in retrospect. It does not account for the impact that the IKT approaches may have had on the knowledge users in the same way that a pre-post or longitudinal design may have. Tracking the knowledge user’s experiences over the course of the project may have offered a more precise account of the impact of the various approaches to IKT and the progression of the partnerships within the research (Gagliardi et al., 2016). Further to this, knowledge users’ accounts of the impact of the IKT approaches may have 148 been influenced by having the embedded researcher conduct the focus groups where they shared their experiences as participants in the project (Jayasekara, 2012). Finally, a potential limitation of this study was the sample for the study did not include representation of the entire CCI team; most notably, there were no participants from public health informatics because they were not available due to the demands of the Covid-19 pandemic. Additionally, the focus groups did not include any knowledge users in executive leadership roles and therefore, the impact of the IKT approaches on the entire group of knowledge users is not fully known. Directions for Future Research IKT is an increasingly popular approach to conducting research, however further investigation is needed to determine the most effective processes and conditions for conducting IKT, and the impact it has on practice (Gagliardi et al., 2017). Reflecting on the study presented in this chapter, there are several directions to pursue with ongoing research. First, this study was limited to the experience of one group of knowledge users, and did not examine the experience of the executive leaders who were also partners in the research project. Future research should examine the similarities and differences among different knowledge user groups across an organization, to better understand how different users experience various aspects of IKT. Additionally, the impact of IKT over time was not evaluated in this study. Further research is needed to examine the long-term impact of research using an IKT approach. More specifically, future research should focus on assessing the mechanisms by which impact is achieved in an effort to articulate an optimal, replicable approach to conducting IKT. Finally, in this study, the knowledge users and embedded researcher co-created a process for addressing TMAE. While 149 this process was deemed valuable and integrated into the knowledge users’ work, further research is needed to validate the effectiveness of this process as a method to analyzing and mitigating against TMAE. Researcher’s Personal Perspectives on Conducting IKT The arrangements afforded by the HSIF fellowship to conduct dissertation research from an embedded position within an organization using an IKT approach offered a unique opportunity. Being positioned within a team and building relationships with knowledge users provided an enriched experience from which meaningful insights into the challenges health organizations face, and the ways in which research can help address these challenges, were gleaned. Pragmatically, having an internal position facilitated direct access to many different stakeholders, including experts inside and outside of the organization, executive leaders, clinicians, and clinical staff from a variety of settings, which provided significant insights to support the research. At the same time, also having an outsider perspective as a researcher allowed for exploration, curiosity, and learning from each of these stakeholders. There were also challenges associated with conducting these studies as an embedded researcher, mainly to do with the novelty of having a researcher within the organization. The role was ambiguous at times, with policies or guidance sometimes lacking clear direction for someone in a hybrid internal/external role (e.g., permissions to access to data differ for researchers and employees). Additionally, balancing the objectives of research with providing organizational supports required both agility and constant communication among all key stakeholders to adapt as the context changes and needs of the organization shift. 150 Overall, conducting this research was a very rewarding experience. It provided an amazing opportunity to further develop my academic skills and key professional competencies to maximize the impact of research. Working with health care leaders to identify areas for improvement and apply my academic skills and expertise to address these challenges was gratifying, and perhaps most importantly, being able to work with knowledge users to test and refine evidence-informed approaches reinforced the importance of research in supporting the health care system. Conclusion This chapter presented the qualitative descriptive study which assessed participants’ experiences with creating an approach to address TMAE using an IKT methodology. Based on the experiences of the knowledge users, the findings of this study indicate that there are barriers and enablers to conducting IKT research, and that it can be an effective approach to both generating new knowledge and facilitating the application of that knowledge. Further, the IKT focus on HIT and patient safety resulted in increased knowledge and some evidence of group level identity transformation related to management of HIT safety. 151 Chapter 7: Overall Discussion and Conclusion Introduction This dissertation presented multiple studies focused on TMAE in primary and community care settings, including: 1) a scoping review on TMAE in primary and community care; 2) a descriptive study characterizing reported TMAE; 3) a detailed description of an IKT approach to HIT-safety; and 4) a qualitative study about how clinical informatics specialists perceived their experiences in developing a process for managing TMAE. The findings of this study provide an important contribution to advancing knowledge about the significance of TMAE within the context of primary and community care. In this chapter, the impact of this work, and implications for the fields of clinical informatics, nursing, and patient safety are shared. Finally, the strengths and limitations of the overall research project, and suggestions for further research to explore are discussed. Themes Emerging Evidence of TMAE in Primary and Community Care Studies 1 and 2 in this dissertation were aimed at characterising TMAE. The scoping review found that the current body of evidence is underdeveloped, heterogenous, and lacking consistent definitions of related concepts and terms. In Study 2, a retrospective analysis of TMAE, it was found that TMAE exist, and inadvertent harm or potential harm can accompany TMAE. Further, a sociotechnical systems model was useful to characterise the complex, sociotechnical nature of these events. 152 Enumerating and Characterising TMAE At the time of the scoping review, only one article was identified that characterised TMAE in primary and acute care. Leading experts in the field are continuing to affirm that HIT-related safety concerns are a persistent problem (Abbott & Weinger, 2020; Sittig et al., 2020), and the focus on patient safety is beginning to expand beyond acute care settings, taking the entire care continuum into consideration (ECRI, 2021; National Patient Safety Foundation, 2015). For example, a recent study that analyzed ten years of incident reporting in England and Wales identified 2,627 safety incidents related to HIT, 82% of which were reported from hospital settings, and yet details concerning non-acute events were not explicated (Martin et al., 2019). It is not clear why the data from acute and non-acute settings are not systematically separated. It is possible that the definitions are unclear. Indeed, in the scoping review (Chapter 3), even the databases had different definitions for outpatient and inpatient. Alternatively, the lagging maturity of the HIT or safety reporting systems in non-acute settings, may also affect how events can be analyzed according to setting. A systematic approach to enumerating and characterising TMAE in the context of primary and community care may help to elucidate the nuances of issues in this setting such that tailored mitigation strategies can be devised. In study 2, TMAE were first enumerated, and then characterised using the sociotechnical model proposed by Sittig and Singh (2010, 2017). The descriptive analysis of voluntarily reported safety events indicated that most TMAE featured both technical and social dimensions, suggesting that the nature of these events is multifaceted, and that technology, users, and context are all important to consider in analyzing and addressing TMAE. Understanding TMAE in the context of primary and community care is challenging because of the distributed nature of care across time and space (Aita et al., 2013; Magrabi et al., 2016). Plsek and Greenhalgh (2001) 153 suggest, “To cope with escalating complexity in health care we must abandon linear models, accept unpredictability, respect (and utilise) autonomy and creativity, and respond flexibly to emerging patterns and opportunities” (p.628). The Sittig and Singh sociotechnical model was useful to capture the complexity of these events, acknowledging the non-linearity and interrelatedness among components (2010, 2017). Most of the TMAE were both social and technical and many of the TMAE had multiple social and technical factors that contributed to the event. Sittig and Singh (2010, 2017) suggest that the sociotechnical system is a complex adaptive system. While it was beyond the scope of this dissertation to examine these data as a complex adaptive system, it would be useful to articulate a more robust description of how Complex Adaptive Systems theory might be applied in the context of TMAE. Characterisation of Harm Due to limited research in enumerating TMAE in primary and community care, the focus of the scoping review was expanded to identify studies that described actual or potential harm. While recent research concerning HIT in primary care settings have focused on diagnostic delay or error (Campione, Mardon, & Mcdonald, 2018; Powell et al., 2020), they have yet to fully articulate harm. Graber (2019) suggests that there is less harm in primary and community care, yet there is little evidence that can verify that claim. In study 2, some evidence was provided to characterize actual harm. This is a way forward to obtain data that would support or refute the findings by Graber about lower rates of harm in primary and community care. It is possible that harm is less in acute care because of the safety net of professionals in attendance, but in acute care, patients are at higher levels of acuity making them more vulnerable. Yet, with the shift to more episodic, and more complex care, it might be that the levels of acuity for primary and 154 community care have also increased. When thinking about harm, in acute care a TMAE might be noticed quickly due to 24/7 monitoring, whereas in primary and community care, it is possible that TMAE might be missed due to large gaps in time between episodes of care. Further, it is possible that a TMAE might reach a wider group of patients in primary and community care due to the wide population reached. A Complex and Constantly Evolving System The consultation with organizational leaders for the scoping review surfaced concerns about the current disparate technological infrastructure in primary and community care settings, and the perpetual progression of organizational and technological change. The leaders articulated challenges with decision making due to the HIT context being in a perpetual ‘interim state’, potentially obscuring some of the threats to patient safety. This concern is echoed in the literature. Coiera et al. (2012) predicted that the evolution of HIT would initially be accompanied by a substantial increase in harm associated with HIT. Given the complexity of healthcare and the rapid pace of HIT implementations, strategies to mitigate against harm will need to keep pace. The process identified in this research has the potential to raise awareness and situate ownership of this issue in the clinical informatics team. While other organizations may have different infrastructures, it is possible to find a place of ownership so that safety can be attended to. Further, it may be that a shift from managing harm to a high reliability mindset where curiosity abounds, is how organizations can manage the perpetual churn of HIT and organizational change. 155 Characterizing and Managing TMAE The third study in this dissertation presented an approach to characterizing and managing TMAE that was generated through the IKT approach that informed this research project. This is an early exemplar of the potential value in leveraging safety event reporting and mobilizing clinical informaticians to address TMAE. Real-time methods for monitoring HIT performance and safety has been identified as a prominent challenge healthcare organizations are facing today (Sittig et al., 2020). The co-created process to address events (Study 3) included a rapid sociotechnical analysis (Sittig & Singh, 2010, 2017), a feedback mechanism to distribute learning from each event across the informatics team as well as back to the clinical setting, and a safety huddle process for reviewing events in aggregate to inform broader decision-making. Reviewing sentinel events can support social and participative learning, and contribute to increased and better quality reporting and improved care (de Kam et al., 2020). At the same time, de Kam et al. (2020) emphasize that incident reporting systems need to be dynamic to support learning as healthcare providers become more experienced and knowledgable. It is possible that the rapid sociotechnical analysis of TMAE was successful in this study because rather than imposing rigid processes, the approach was co-created with the knowledge users (Gagliardi & Dobrow, 2016). Thus, the process for addressing TMAE was generated in partnership with knowledge users to ensure the process fit with the current knowledge and priorities of the knowledge users. IKT Approach to Characterizing and Managing TMAE The IKT approach in this research project proved to be useful in studying TMAE. Barriers, enablers, and outcomes of the IKT activities were identified and, ultimately, knowledge 156 users reported that participating in the IKT activities had an impact on their work in relation to TMAE. Informants reported increased knowledge and ownership in relation to HIT safety, and applied their new learning to analyze and mitigate against TMAE. While the growing body of literature concerning IKT elucidates some of the essential aspects for success, best practices to support system-level impacts are not well established (Gagliardi et al., 2017). In the current study, having knowledge of adult learning theory and facilitating front-line ownership of the work proved to be two novel embedded researcher factors that directly supported the immediate impact for the knowledge users (Plsek, 2017; Stewart et al., 2015). Overall, the alignment of the IKT methodology with the context of TMAE research can both generate knowledge, and apply knowledge simultaneously, to keep up with the pace of change in service delivery. The rapid sociotechnical analysis of TMAE that was co-created in this IKT partnership (Figure 5.2) provided a framework for understanding TMAE, and directed the immediate application of that knowledge to advance safety. Further research is needed to refine and validate this model. Given that the outcomes of the model are a product of the context and collaborative partnership, other strategies or processes to address TMAE could be generated in future studies. It is also possible that this model might be transferrable to other (non-informatics) contexts. The key feature of having an embedded researcher who was supernumery to the team could direct organizations to allocate resources for similar organizational projects. Recommendations Based on the findings from the four studies presented in this dissertation, there are a number of insights to consider in order to promote safe use of HIT in primary and community 157 care settings. Recommendations for three key areas in healthcare: clinical informatics, nursing, and patient safety are discussed below. Recommendations for Clinical Informatics From a clinical informatics perspective, it is important to pay careful attention to unintended consequences from procurement of new technologies to beyond implementation. For the context of primary and community care specifically, the dispersion of care over time and across care settings means that clinical information is critical to supporting the continuity and comprehensiveness of care. As Black et al. (2011) assert, “clinical informatics should be no less concerned with safety and efficacy than the pharmaceutical industry” (p.12). Building on their seminal list of fallacies associated with HIT, Abbott and Weiner (2020) identified that the common misconceptions and overestimations of HIT they first identified in 2010 have persisted (Karsh et al., 2010). Notable fallacies they identified include the assumptions that HIT is risk free, that it meets the needs of all users, will be used as intended, and that the responsibility for safety belongs to ‘someone else’ (Abbott & Weinger, 2020). Clinical informatics can role model accountability for safety. Sitting at the intersection of clinical practice, information technology, and health service delivery, clinical informatics is well positioned to synthesize the perspectives on HIT safety from different key stakeholders. Ash et al. (2020) identified three major foci for health care organizations to promote safe use of EHRs: 1) decision making activities; 2) organizational learning activities; and 3) user-related activities, and while health organizations may be structured differently, clinical informatics teams can play a key role in all of these activities. By building safety into all work processes concerning planning, implementing, operating, and evaluating technologies, clinical informatics teams can 158 advocate for technical solutions that are well suited to the complexities of clinical practice, and support clinicians to understand the utility and limitations of the technologies at their disposal. To promote HIT safety, a clinical informatics perspective needs to take the complexities of the healthcare system into account. Using an analytic framework can assist with conducting a thorough analysis of the entire sociotechnical system (Kim et al., 2017; Sittig & Singh, 2017). In working to mitigate against TMAE, it is important to learn from instances of harm or potential harm, by “constantly questioning, rather than accepting and remaining committed to, current practices” (Tucker & Edmondson, 2003, p.68). To effectively address problems and promote safety, a clinical informatics perspective also needs to account for the needs of the user and the clinical context, and respond with approaches that incorporate human factors (Cassano-Piché et al., 2012), and principles of design (Norman, 2016). Going forward, the perspective of clinical informatics is key in building the organizational knowledge of TMAE and supporting data-driven decision-making to promote HIT safety in an effort towards continuous improvement. Recommendations for Nursing Given the rapid technological advancement and ongoing vision for HIT to be integrated in the provision of healthcare services, nursing is essential for promoting HIT safety. Based on nurses’ experience and perceptions of working alongside technology, nurses can offer valuable insights into the functionalities and advantages, as well as the risks that these technologies may pose. Committed to the delivery of technologically-enabled, compassionate care (Strudwick et al., 2020), nurses sit at the crux of HIT safety. Edmondson (2004) suggests that “health care’s emphasis on individual vigilance encourages nurses and other healthcare professionals to take personal responsibility to solve problems as they arise” (p.ii7). Further, Benner et al. (2009) 159 emphasize that “nurses maintain patient safety while managing multiple intrusive technologies where the margin of error is extremely narrow, and they do so in increasingly complex, hazardous work environments” (p.1). Thus, nurses are positioned at the sharp end of patient safety as users of HIT in the delivery of care, (Borycki, Kushniruk, Keay, & Kuo, 2009; Hughes, 2008), and as such, nurses may resign themselves to the difficulties of using HIT, simply accepting the role and responsibility of being intermediaries between HIT-related threats and delivery of safe care (Staggers, Elias, Makar, & Alexander, 2018). The unfortunate consequence of this is that nurses can then become the ‘second victim’ of TMAE, from the agony and grief they experience when their commitment to safe care is up-ended because they have failed to protect the patient from the series of systemic failures that preceded an error (Seys et al., 2013; Wu, 2000). For a more proactive stance against TMAE, the voice of nurses needs to be enhanced (Staggers et al., 2018). There is a need for nurses to recognize and articulate safety concerns related to HIT, and advocate for HIT systems that are designed and implemented with considerations for safety at the forefront. While usability is increasingly recognized as an important aspect of using HIT (Kaipio, Kuusisto, Hyppönen, Heponiemi, & Lääveri, 2020), the needs of nurses as users are not adequately addressed in current HIT systems, and the potential implications of this for patient outcomes warrants further attention (Topaz et al., 2016). Nurses at all levels can contribute to the advancement of HIT safety, to inform planning, implementing, and evaluating HIT. As end-users, their contribution to the design of HIT can help ensure that functionality and software requirements for nurses are addressed, with the potential to fully realise the benefits of HIT (Walker & Clendon, 2016). Nursing leaders who are knowledgeable about the risks and limitations of technology, and can advocate for technological advancements 160 that thoughtfully incorporate and support quality nursing care are essential (Booth et al., 2019; Risling & Low, 2019). Further to this, nurses with a specialization in informatics are a much needed resource to gain an in-depth understanding of HIT safety from a nursing perspective and provide leadership in mitigating against harm (Bakken & Hripcsak, 2004; Borycki et al., 2017). Finally, education programs for nurses can help to prepare new nurses with the necessary knowledge and skills to leverage the advancing technologies for delivering safe care (Risling, 2017). At all levels of practice, across all practice settings, developing professional insight into HIT safety and an in-depth understanding of TMAE can position nurses to mitigate harm to the patients and clients they care for. Recommendations for Patient Safety The patient safety movement has grown significantly in the past two decades, with the establishment of a number of strategies to address safety concerns and improve care, including training programs, incident reporting and investigation, and ‘just’ culture (Bartman et al., 2019). Despite these developments, patient safety problems have persisted (Bates & Singh, 2018; Singh, Sittig, & Gandhi, 2021). Reporting safety incidents has become a wider spread practice, yet it has yet to achieve a marked improvement in safety, with quantification of harm eclipsing mitigation (Macrae, 2015; Shojania & Marang-van de Mheen, 2020). Howell et al. (2017) cautions not to use safety reporting as an epidemiological tool to measure harm, but rather for identifying organizational needs related to safety. In their position piece calling for a public health response to harm in healthcare, Gandhi and colleagues (2020) articulate that “meaningful advancement in patient safety requires a shift from reactive, piecemeal interventions occurring at individual organizations to a coordinated system-wide effort geared at providing safe care 161 delivery across all aspects of care” (p.2). This is in line with the Institute for Healthcare Improvement (2021) which has proposed a shift toward ‘Whole System Quality’, with “integrated quality planning, quality control, and quality improvement activities that inform an organization-wide, interlinked, and customer-centric strategic approach to quality” (p. 8). Aligning with these shifting perspectives on safety, efforts to address TMAE may benefit from a more expansive understanding of ‘harm’, with safety considered to be “more than the absence of physical harm; it is also the pursuit of dignity and equity” (Feeley, 2017, p.4). For primary and community care settings specifically, patient safety needs further research and attention (Carpenter et al., 2017; Lawati, Dennis, Short, & Abdulhadi, 2018; Tsang et al., 2012; Verbakel et al., 2016). Some of the more subtle ways patients may be inadvertently harmed in the course of receiving health services are through delayed or incorrect diagnosis, or issues categorized under ‘patient experience’. For example, in Study 2, one TMAE focused on misnaming a non-CIS-gendered person resulting in additional psychological harm to an already vulnerable person. To that end, given the dispersion of care and autonomy of patients in primary and community care settings, recognizing safety concerns and analyzing the related trajectory of events may be enhanced by incorporating input from patients (King et al., 2010; Vincent et al., 2017; Zhu et al., 2011). Research has shown that patients can make valuable contributions to learning from diagnostic errors (Giardina et al., 2018b), and can potentially mitigate harm by identifying errors in their own records (Freise et al., 2021). Regarding TMAE in primary and community care, a broader perspective of patient safety, monitoring for things “that should not have happened and that you don’t want to happen again” may yield greater insight into how to mitigate against harm (Phillips, Dovey, Graham, Elder, & Hickner, 2006, p.141). 162 Limitations In addition to the limitations identified previously for each study, there are other limitations to consider in general. First, it should be noted that much of this research took place at the same time as the Covid-19 pandemic. The operations of healthcare shifted dramatically in response to the pandemic, including the focus and activities of the knowledge users who were partners in this research. While it is difficult to determine how this may have affected the research project, it is important to acknowledge that there were extenuating circumstances surrounding the execution of this research. A second overall limitation to note is related to the limitations of investigating HIT errors or safety events, given that it is well recognised that many errors, or adverse events go undetected or unreported. Outside of reporting, many safety concerns related to HIT remain unknown and unknowable (Koppel, 2016). Caution should be taken in interpreting findings related to errors or safety events as exhaustive data that can be treated as epidemiological representations of actual problems; what is recognized and reported is not necessarily representative of the actual problems related to HIT that are occurring in clinical practice. Finally, a third general limitation concerning the overall research project is the contextually, tailored approach to IKT that was applied in this study may be difficult to generalize. Specifically, the generalizability of findings is limited in that the design and execution of the research was a collaborative effort with the knowledge users from the partnering organization, and the knowledge gleaned is a product of this partnership. 163 Future Research Directions There is more to learn about HIT safety in primary and community care settings, particularly from TMAE. The clinical context is an important distinction to make as primary and community care is geographically and temporally dispersed. Compared to acute settings, care is typically more episodic and patients have greater autonomy, and as such, HIT systems are often disparate with client information segmented amongst them. Because of this, it is possible that harm in primary and community care settings is not recognized or reported in the same manner as in hospital settings. More research is needed to capture the incidence of TMAE in primary and community care settings. Ongoing research could also help develop a consensus around the best definitions and measures concerning HIT safety in primary and community care, and establish best practices for reporting how HIT was involved. Methods to investigate TMAE should also include analysis of both HIT function and patient outcomes in order to understand the implications of technology in relation to safety. Systematically classifying the risks to patient safety, alongside the HIT problems is recommended to enhance measurement in this area and prioritize the most significant threats (Kim et al., 2017). Future research would also benefit from multidisciplinary perspectives, including the role of nurses, within the context of TMAE. Nurses are ideally positioned to identify, report, and mitigate against TMAE, and are instrumental to achieving the potential benefits of HIT in delivering quality care (Bakken et al., 2004; Booth, 2016; Strudwick et al., 2019, 2020; Van De Castle et al., 2004). Action-oriented, embedded research approaches are also well-suited to conducting further research, accounting for the complexity within primary and community care settings, to generate a clinically-focused approach for preventing harm, and keeping pace with the continued advancement of technologies in healthcare. While the IKT approach was valuable 164 in this research project, further research is also needed to establish best practices, validate measures of success, and ultimately articulate how IKT can be used to support system-level impacts (Gagliardi et al., 2017) Conclusion It is important to consider the implications of HIT in relation to safety in primary and community care settings. Research in this area is under-developed, yet healthcare services are continuing to place greater emphasis on both technology and non-acute services. This research confirmed that harm related to HIT can occur in these settings, demonstrated the sociotechnical nature of harm in this context, and put forward a process to address TMAE. This dissertation research focused on a rapidly evolving area of care, using an IKT approach to investigation, with an embedded researcher working closely alongside organizational leaders and clinical informaticians to investigate and address TMAE. Further research, incorporating multiple disciplinary perspectives and action-oriented strategies, is needed to gain insight into the impact of TMAE in primary and community care settings and promote HIT safety going forward. 165 Bibliography Abbott, P. A., & Weinger, M. B. (2020). Health information technology: Fallacies and Sober realities – Redux A homage to Bentzi Karsh and Robert Wears. Applied Ergonomics, 82, 1-10. https://doi.org/10.1016/j.apergo.2019.102973 Abraham, J., Kitsiou, S., Meng, A., Burton, S., Vatani, H., & Kannampallil, T. (2020). Effects of CPOE-based medication ordering on outcomes: An overview of systematic reviews. BMJ Quality & Safety, 29(10), 1-2. https://doi.org/10.1136/bmjqs-2019-010436 Abramson, E. L., Malhotra, S., Fischer, K., Edwards, A., Pfoh, E. R., Osorio, S. N., Cheriff, A., & Kaushal, R. (2011). Transitioning between electronic health records: Effects on ambulatory prescribing safety. Journal of General Internal Medicine, 26(8), 868-874. https://doi.org/10.1007/s11606-011-1703-z Ai, A., Wong, A., Amato, M., & Wright, A. (2018). Communication failure: Analysis of prescribers’ use of an internal free-text field on electronic prescriptions. Journal of the American Medical Informatics Association, 25(6), 709–714. https://doi.org/10.1093/jamia/ocy003 Aita, M., Belvedere, O., De Carlo, E., Deroma, L., De Pauli, F., Gurrieri, L., Denaro, A., Zanier, L., & Fasola, G. (2013). Chemotherapy prescribing errors: An observational study on the role of information technology and computerized physician order entry systems. BMC Health Services Research, 13(1), 522-522. https://doi.org/10.1186/1472-6963-13-522 American College of Physicians. Patient Safety in the Office-Based Practice Setting. Philadelphia: American College of Physicians; 2017: Position Paper. https://www.acponline.org/acp_policy/policies/patient_safety_in_the_office_based_practice_setting_2017.pdf 166 Ammenwerth, E., Schnell-Inderst, P., Machan, C., & Siebert, U. (2008). The effect of electronic prescribing on medication errors and adverse drug events: A systematic review. Journal of the American Medical Informatics Association, 15(5), 585-600. https://doi.org/10.1197/jamia.M2667 Arksey, H., & O'Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19-32. https://doi.org/10.1080/1364557032000119616 Ash, J. S., Berg, M., & Coiera, E. (2004;2003;). Some unintended consequences of information technology in health care: The nature of patient care information system-related errors. Journal of the American Medical Informatics Association, 11(2), 104-112. https://doi.org/10.1197/jamia.M1471 Ash, J. S., Singh, H., Wright, A., Chase, D., & Sittig, D. F. (2020). Essential activities for electronic health record safety: A qualitative study. Health Informatics Journal, 26(4), 3140-3151. https://doi.org/10.1177/1460458219833109 Baker, D. P., Day, R., & Salas, E. (2006). Teamwork as an essential component of high-reliability organizations. Health Services Research, 41(4p2), 1576-1598. https://doi.org/10.1111/j.1475-6773.2006.00566.x Baker, G. R., Norton, P. G., Flintoft, V., Blais, R., Brown, A., Cox, J., Etchells, E., Ghali, W. A., Hebert, P., Majumdar, S. R., O'Beirne, M., Palacios-Derflingher, L., Reid, R. J., Sheps, S., & Tamblyn, R. (2004). The canadian adverse events study: The incidence of adverse events among hospital patients in canada. Canadian Medical Association Journal, 170(11), 1678-1686. https://doi.org/10.1503/cmaj.1040498 Bakken, S., & Hripcsak, G. (2004). An informatics infrastructure for patient safety and evidence- 167 based practice in home healthcare. Journal for Healthcare Quality, 26(3), 24-30. Bakken, S., Cook, S. S., Curtis, L., Desjardins, K., Hyun, S., Jenkins, M., John, R., Klein, W. T., Paguntalan, J., Roberts, W. D., & Soupios, M. (2004). Promoting patient safety through informatics-based nursing education. International Journal of Medical Informatics (Shannon, Ireland), 73(7), 581-589. https://doi.org/10.1016/j.ijmedinf.2004.04.008 Bartman, T., Bertoni, C. B., Merandi, J., Brady, M., & Bode, R. S. (2019). Patient safety: What is working and why? Current Treatment Options in Pediatrics, 5(2), 131-144. https://doi.org/10.1007/s40746-019-00156-5 Bates, D. W., & Singh, H. (2018). Two decades since to err is human: An assessment of progress and emerging priorities in patient safety. Health Affairs, 37(11), 1736-1743. https://doi.org/10.1377/hlthaff.2018.0738 Bates, D. W., Teich, J. M., Lee, J., Seger, D., Kuperman, G. J., Ma'Luf, N., Boyle, D., & Leape, L. (1999). The impact of computerized physician order entry on medication error prevention. Journal of the American Medical Informatics Association : JAMIA, 6(4), 313-321. https://doi.org/10.1136/jamia.1999.00660313 Bazhenova, L., & DeMoor, P. A. (2012). Impact of computerized provider order entry on medication errors in outpatient chemotherapy administration. Journal of Clinical Oncology, 30(34_suppl), 295-295. https://doi.org/10.1200/jco.2012.30.34_suppl.295 BC Patient Safety & Quality Council. (2017). BC Health Quality Matrix Handbook. https://bcpsqc.ca/wp-content/uploads/2020/02/BC-Health-Quality-Matrix-March-2020.pdf Bellwood, P., Borycki, E. M., & Kushniruk, A. W. (2015). Awareness of technology-induced errors and processes for identifying and preventing such errors. Studies in Health Technology and Informatics, 208, 61-65. https://doi.org/10.3233/978-1-61499-488-6-61 168 Benner, P., Sutphen, M., Leonard, V., & Day, L. (2009). Educating nurses: A call for radical transformation (Vol. 15). John Wiley & Sons. Berg, M. (1999). Patient care information systems and health care work: a sociotechnical approach. International Journal of Medical Informatics, 55(2), 87–101. https://doi.org/10.1016/S1386-5056(99)00011-8 Berg, M., Aarts, J., & van der Lei, J. (2003). ICT in health care: Sociotechnical approaches. Methods of Information in Medicine, 42(4), 297-301. Black, A. D., Car, J., Pagliari, C., Anandan, C., Cresswell, K., Bokun, T., McKinstry, B., Procter, R., Majeed, A., & Sheikh, A. (2011). The impact of eHealth on the quality and safety of health care: A systematic overview. PLoS Medicine, 8(1), e1000387-e1000387. https://doi.org/10.1371/journal.pmed.1000387 Booth, R. G. (2016). Informatics and nursing in a post-nursing informatics world: Future directions for nurses in an automated, artificially-intelligent, social-networked healthcare environment. Nursing Leadership (Toronto, Ont.), 28(4), 61-69. https://doi.org/10.12927/cjnl.2016.24563 Booth, R. G., Andrusyszyn, M., Iwasiw, C., Donelle, L., & Compeau, D. (2016). Actor-network theory as a sociotechnical lens to explore the relationship of nurses and technology in practice: Methodological considerations for nursing research. Nursing Inquiry, 23(2), 109-120. https://doi.org/10.1111/nin.12118 Booth, R. G., Strudwick, G., McMurray, J., Morse, A., Chan, R., & Zhang, T. (2019). The best way to predict the future is to [co]create it: A technology primer for healthcare leaders. In Leadership for Evidence-Based Innovation in Nursing and Health Professions (pp. 261– 169 283). Bornstein, S., Heritage, M., Chudak, A., Tamblyn, R., McMahon, M., & Brown, A. D. (2018). Development of enriched core competencies for health services and policy research. Health Services Research, 53(5), 4004-4023. https://doi.org/10.1111/1475-6773.12847 Borycki, E. M., & Kushniruk, A. W. (2010). Towards an integrative cognitive-socio-technical approach in health informatics: Analyzing technology-induced error involving health information systems to improve patient safety. The Open Medical Informatics Journal, 4(1), 181-187. https://doi.org/10.2174/1874431101004010181 Borycki, E. M. (2013). Trends in health information technology safety: From technology-induced errors to current approaches for ensuring technology safety. Healthcare Informatics Research, 19(2), 69-78. https://doi.org/10.4258/hir.2013.19.2.69 Borycki, E. M., Cummings, E., Kushniruk, A. W., & Saranto, K. (2017). Integrating health information technology safety into nursing informatics competencies. Studies in Health Technology and Informatics, 232, 222-228. https://doi.org/10.3233/978-1-61499-738-2-222 Borycki, E. M., Househ, F. M. S., Kushniruk, A. W., Nohr, C., & Takeda, H. (2012). Empowering patients: Making health information and systems safer for patients and the public: Contribution of the IMIA health informatics for patient safety working group. Yearbook of Medical Informatics, 21(1), 56-64. https://doi.org/10.1055/s-0038-1639431 Borycki, E. M., Kushniruk, A. W., Keay, L., & Kuo, A. (2009). A framework for diagnosing and identifying where technology-induced errors come from. Studies in Health Technology and Informatics, 148, 181-187. https://doi.org/10.3233/978-1-60750-043-8-181 Borycki, E. M., Senthriajah, Y., Kushniruk, A. W., Palojoki, S., Saranto, K., & Takeda, H. 170 (2016). Reducing technology-induced errors: Organizational and health systems approaches. Studies in Health Technology and Informatics, 225, 741–743. https://doi.org/10.3233/978-1-61499-658-3-741 Bowen, S., & Graham, I. D. (2013). Integrated Knowledge translation. In S. Straus., J. Tetroe., & I. D.Graham, (Eds.), Knowledge Translation in Health Care: Moving from evidence to practice, 2nd Edition (pp.14-24). Bowie, P., McKay, J., McNab, D., & de Wet, C. (2016). The past, present and future of patient safety education and research in primary care. Education for Primary Care, 27(1), 3–9. https://doi.org/10.1080/14739879.2015.1132672 Boysen, P. G. (2013). Just culture: A foundation for balanced accountability and patient safety. Ochsner Journal, 13(3), 400–406. Braithwaite, J., Wears, R. L., & Hollnagel, E. (2015). Resilient health care: turning patient safety on its head. International Journal for Quality in Health Care, 27(5), 418–420. https://doi.org/10.1093/intqhc/mzv063 Brass, S. D., Olney, G., Glimp, R., Lemaire, A., & Kingston, M. (2018). Using the patient safety huddle as a tool for high reliability. Joint Commission Journal on Quality and Patient Safety, 44(4), 219-226. https://doi.org/10.1016/j.jcjq.2017.10.004 Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa Brenner, S. K., Kaushal, R., Grinspan, Z., Joyce, C., Kim, I., Allard, R. J., Delgado, D., & Abramson, E. L. (2016). Effects of health information technology on patient outcomes: A systematic review. Journal of the American Medical Informatics Association, 23(5), 1016–1036. https://doi.org/10.1093/jamia/ocv138 171 Bright, T. J., Wong, A., Dhurjati, R., Bristow, E., Bastian, L., Coeytaux, R. R., Samsa, G., Hasselblad, V., Williams, J. W., Musty, M. D., Wing, L., Kendrick, A. S., Sanders, G. D., & Lobach, D. (2012). Effect of clinical decision-support systems: A systematic review. In Annals of Internal Medicine 157(1), pp. 29–43). https://doi.org/10.7326/0003-4819-157-1-201207030-00450 British Columbia Ministry of Health. (n.d.-a). Family Doctors and Other Primary Care Providers. Retrieved March 20, 2019, from https://www2.gov.bc.ca/gov/content/family-social-supports/seniors/health-safety/health-care-programs-and-services/family-doctors-and-other-primary-care-providers British Columbia Ministry of Health. (n.d.-b). Home and Community Care. Retrieved March 20, 2019, from https://www2.gov.bc.ca/gov/content/health/accessing-health-care/home-community-care British Columbia Ministry of Health. (2018). 2018/19-2020/2021 Service Plan. https://www.bcbudget.gov.bc.ca/2018/sp/pdf/ministry/hlth.pdf British Columbia Ministry of Health. (2020). 2020/21 – 2022/23 Service Plan. Retrieved from https://www.bcbudget.gov.bc.ca/2020/sp/pdf/ministry/fin.pdf Bullard, M. J., Emond, S. D., Graham, T. A. D., Ho, K., & Holroyd, B. R. (2007). Informatics and knowledge translation. Academic Emergency Medicine, 14(11), 996-1002. https://doi.org/10.1197/j.aem.2007.06.032 Campbell, E. M., Sittig, D. F., Ash, J. S., Guappone, K. P., & Dykstra, R. H. (2006). Types of unintended consequences related to computerized provider order entry.Journal of the American Medical Informatics Association : JAMIA, 13(5), 547-556. https://doi.org/10.1197/jamia.M2042 172 Campione, J. R., Mardon, R. E., & McDonald, K. M. (2019). Patient safety culture, health information technology implementation, and medical office problems that could lead to diagnostic error. Journal of Patient Safety, 15(4), 267-273. https://doi.org/10.1097/PTS.0000000000000531 Canada Health Infoway. (2017). 2017 National Survey of Canadian Nurses: Use of Digital Health Technology in Practice Final Executive Report. https://www.infoway-inforoute.ca/en/component/edocman/3320-2017-national-survey-of-canadian-nurses-use-of-digital-health-technology-in-practice/view-document?Itemid=0 Canada Health Infoway. (2018). 2018 Canadian Physician Survey Physicians’ Use of Digital Health and Information Technologies in Practice Background and Methodology. https://www.infoway-inforoute.ca/en/component/edocman/3643-2018-canadian-physician-survey/view-document?Itemid=0 Canadian Institute for Health Information. (2017). Regulated Nurses, 2016. https://www.cihi.ca/sites/default/files/document/regulated-nurses-2016-report-en-web.pdf Canadian Institute for Health Research. (n.d.). Knowledge translation at CIHR. Retrieved July 10, 2021, from https://cihr-irsc.gc.ca/e/29418.html#2 Canadian Patient Safety Institute. (2013). Safety at Home: A Pan-Canadian Home Care Safety Study. https://www.patientsafetyinstitute.ca/en/toolsResources/Research/commissionedResearch/SafetyatHome/Documents/Safety%20At%20Home%20Care.pdf#search=patient%20safety%20at%20home Canadian Patient Safety Institute, & Risk Analytica. (2017). The case for investing in patient safety. http://www.patientsafetyinstitute.ca/en/About/Documents/The Case for Investing in 173 Patient Safety.pdf Carayon, P. (2006). Human factors of complex sociotechnical systems. Applied Ergonomics, 37(4), 525-535. https://doi.org/10.1016/j.apergo.2006.04.011 Carayon, P. (2012). Sociotechnical systems approach to healthcare quality and patient safety. Work (Reading, Mass.), 41 Suppl 1(1), 3850-3854. https://doi.org/10.3233/WOR-2012-0091-3850 Carayon, P., Bass, E. J., Bellandi, T., Gurses, A. P., Hallbeck, M. S., & Mollo, V. (2011). Sociotechnical systems analysis in health care: A research agenda. IIE Transactions on Healthcare Systems Engineering, 1(3), 145-160. https://doi.org/10.1080/19488300.2011.619158 Carayon, P., Schoofs Hundt, A., Karsh, B., Gurses, A. P., Alvarado, C. J., Smith, M., & Flatley Brennan, P. (2006). Work system design for patient safety: The SEIPS model.Quality & Safety in Health Care, 15(suppl 1), i50-i58. https://doi.org/10.1136/qshc.2005.015842 Carling, C. L. L., Kirkehei, I., Dalsbø, T. K., & Paulsen, E. (2013). Risks to patient safety associated with implementation of electronic applications for medication management in ambulatory care--a systematic review. BMC Medical Informatics and Decision Making, 13(1), 133-133. https://doi.org/10.1186/1472-6947-13-133 Carpenter, D., Famolaro, T., Hassell, S., Betsy Kaeberle, M., Shannon Reefer, B., Cynthia Robins, M., & Siegel, S. (2017). Patient Safety in the Home. https://homecarenh.org/wp-content/uploads/2018/07/PatientSafetyInTheHome_2017.pdf Carson-Stevens, A., Hibbert, P., Williams, H., Evans, H. P., Cooper, A., Rees, P., … Edwards, A. (2016). Characterising the nature of primary care patient safety incident reports in the England and Wales National Reporting and Learning System: a mixed-methods agenda- 174 setting study for general practice. Health Services and Delivery Research, 4(27), 1–76. https://doi.org/10.3310/hsdr04270 Cassano-Piché, A., Trbovich, P., Griffin, M., Ling Lin, Y., & Easty, T. (2012). Human Factors for Health Technology Safety: Evaluating & Improving the use of Health Technology in the Real World. 268. http://cedglobal.org/wp-content/uploads/2016/11/CED-HF-Health-Technology-Safety.pdf Castro, G., Buczkowski, L., & Hafner, J. M. (2016). The contribution of sociotechnical factors to health information technology-related sentinel events. Joint Commission Journal on Quality and Patient Safety, 42(2), 70–76. https://doi.org/10.1016/s1553-7250(16)42008-8 Chandrashekar, P., Moodley, S., & Jain, S. H. (2019). 5 Obstacles to Home-Based Health Care, and How to Overcome Them. Harvard Business Review. https://hbr.org/2019/10/5-obstacles-to-home-based-health-care-and-how-to-overcome-them Chang, F., & Gupta, N. (2015). Progress in electronic medical record adoption in Canada. Canadian Family Physician, 61(12), 1076–1084. Chaudhry, B., Wang, J., Wu, S., Maglione, M., Mojica, W., Roth, E., … Shekelle, P. G. (2006). Systematic review: Impact of health information technology on quality, efficiency, and costs of medical care. Annals of Internal Medicine, 144(10), 742–752. https://doi.org/10.7326/0003-4819-144-10-200605160-00125 Choi, M., & Ruona, W. E. A. (2011). Individual readiness for organizational change and its implications for human resource and organization development. Human Resource Development Review, 10(1), 46–73. https://doi.org/10.1177/1534484310384957 Cochrane, D., Taylor, A., Miller, G., Valoria, Matsui, I., Bharadwaj, M., & Devine, P. (2009). Establishing a provincial patient safety and learning system: Pilot project results and lessons 175 learned. Healthcare Quarterly, 12(SP2), 147-153. https://doi.org/10.12927/hcq.2009.20717 Coiera, E. (2007). Putting the technical back into socio-technical systems research. International Journal of Medical Informatics, 76(SUPPL. 1), 98–103. https://doi.org/10.1016/j.ijmedinf.2006.05.026 Coiera, E. (2015). Guide to Health Informatics (3rd ed.). Boca Raton, FL: CRC Press. Coiera, E., Aarts, J., & Kulikowski, C. (2012;2011;). The dangerous decade. Journal of the American Medical Informatics Association, 19(1), 2-5. https://doi.org/10.1136/amiajnl-2011-000674 Cooper, J., Williams, H., Hibbert, P., Edwards, A., Butt, A., Wood, F., Parry, G., Smith, P., Sheikh, A., Donaldson, L., & Carson-Stevens, A. (2018). Classification of patient-safety incidents in primary care. Bulletin of the World Health Organization, 96(7), 498-505 https://doi.org/10.2471/BLT.17.199802 Cresswell, K. M., Bates, D. W., Wright, A., & Sheikh, A. (2017). An overview of clinical informatics. In Key Advances in Clinical Informatics: Transforming Health Care through Health Information Technology (pp. 3–11). https://doi.org/10.1016/B978-0-12-809523-2.00001-7 Cronenwett, L., Sherwood, G., Barnsteiner, J., Disch, J., & Johnson, J. (2007). Quality and safety education for nurses. Nursing Outlook, 55(3), 122–131. https://doi.org/10.1016/j.outlook.2007.02.006 Daker-White, G., Hays, R., McSharry, J., Giles, S., Cheraghi-Sohi, S., Rhodes, P., & Sanders, C. (2015). Blame the patient, blame the doctor or blame the system? A meta-synthesis of qualitative studies of patient safety in primary care. PLoS ONE, 10(8). https://doi.org/http://dx.doi.org/10.1371/journal.pone.0128329 176 Davis, K., Drey, N., & Gould, D. (2009). What are scoping studies? A review of the nursing literature. International Journal of Nursing Studies, 46(10), 1386-1400. https://doi.org/10.1016/j.ijnurstu.2009.02.010 de Kam, D., Kok, J., Grit, K., Leistikow, I., Vlemminx, M., & Bal, R. (2020). How incident reporting systems can stimulate social and participative learning: A mixed-methods study. Health Policy, 124(8), 834–841. https://doi.org/10.1016/j.healthpol.2020.05.018 Dekker, S. W. A. (2009). Just culture: Who gets to draw the line? Cognition, Technology and Work, 11(3), 177–185. https://doi.org/10.1007/s10111-008-0110-7 Digital Health Canada. (2018). e-Safety Update 2017-2018. Retreived from https://www.e-healthconference.com/resources-2/e-health-virtual-library/ Donabedian, A. (1966). Evaluating the Quality of Medical Care. The Milbank Memorial Fund Quarterly, 44(3-Part 2), 166–206. Dowding, D. W., Currie, L. M., Borycki, E., Clamp, S., Favela, J., Fitzpatrick, G., Gardner, P., Hamer, S., Hardiker, N., Johnson, O., Lawton, R., O'Brien, A., Randell, R., Westbrook, J., Whitewood-Moores, Z., & Dykes, P. C. (2013). International priorities for research in nursing informatics for patient care. Studies in Health Technology and Informatics, 192, 372-376. https://doi.org/10.3233/978-1-61499-289-9-372 Eason, K. (2014). Afterword: The past, present and future of sociotechnical systems theory. Applied Ergonomics, 45(2 Part A), 213–220. https://doi.org/10.1016/j.apergo.2013.09.017 ECRI. (2021). Top 10 Patient Safety Concerns 2021. Retrieved March 21, 2020, from https://www.ecri.org/top-10-patient-safety-concerns-2021 Edmondson, A. (1999). Psychological safety and learning behavior in work teams.Administrative Science Quarterly, 44(2), 350-383. https://doi.org/10.2307/2666999 177 Edmondson, A. C. (2011). Strategies of learning from failure. Harvard Business Review, 89(4), 48-137. Edmondson, A. C. (2004). Learning from failure in health care: frequent opportunities, pervasive barriers. BMJ Quality & Safety, 13(suppl 2), ii3-ii9. https://doi.org/10.1136/qshc.2003.009597 Feeley, D. (2017). Foreward. In A. Frankel, C. Haraden, F. Federico, & J. Lenoci-Edwards. A framework for safe, reliable, and effective care. Cambridge. Feldman, S. S., Buchalter, S., & Hayes, L. W. (2018). Health information technology in healthcare quality and patient safety: Literature review. JMIR Medical Informatics, 20(6), 1–14. https://doi.org/10.2196/10264 Fernandopulle, R., & Patel, N. (2010). How the electronic health record did not measure up to the demands of our medical home practice. Health Affairs (Project Hope), 29(4), 622–628. https://dx.doi.org/10.1377/hlthaff.2010.0065 Flemons, W. W., & McRae, G. (2012). Reporting, learning and the culture of safety. Healthcare Quarterly, 15, 12-17. https://doi.org/10.12927/hcq.2012.22847 Fong, A., Adams, K. T., Gaunt, M. J., Howe, J. L., Kellogg, K. M., & Ratwani, R. M. (2018). Identifying health information technology related safety event reports from patient safety event report databases. Journal of Biomedical Informatics, 86, 135–142. https://doi.org/10.1016/j.jbi.2018.09.007 Franklin, B. J., Gandhi, T. K., Bates, D. W., Huancahuari, N., Morris, C. A., Pearson, M., … Goralnick, E. (2020). Impact of multidisciplinary team huddles on patient safety: A systematic review and proposed taxonomy. BMJ Quality and Safety, Vol. 29, 844–853. https://doi.org/10.1136/bmjqs-2019-009911 178 Freise, L., Neves, A. L., Flott, K., Harrison, P., Kelly, J., Darzi, A., & Mayer, E. K. (2021). Assessment of patients’ ability to review electronic health record information to identify potential errors: Cross-sectional web-based survey. JMIR Formative Research, 5(2). https://doi.org/10.2196/19074 Gagliardi, A. R., Berta, W., Kothari, A., Boyko, J., & Urquhart, R. (2016). Integrated knowledge translation (IKT) in health care: A scoping review. Implementation Science : IS, 11(1), 38-38. https://doi.org/10.1186/s13012-016-0399-1 Gagliardi, A. R., & Dobrow, M. J. (2016). Identifying the conditions needed for integrated knowledge translation (IKT) in health care organizations: Qualitative interviews with researchers and research users. BMC Health Services Research, 16(1). https://doi.org/10.1186/s12913-016-1533-0 Gagliardi, A. R., Kothari, A., & Graham, I. D. (2017). Research agenda for integrated knowledge translation (IKT) in healthcare: What we know and do not yet know. Journal of Epidemiology and Community Health, 71(2), 105–106. https://doi.org/10.1136/jech-2016-207743 Gandhi, T. K., Feeley, D., & Schummers, D. (2020). Zero Harm in Health Care. NEJM Catalyst, 1(2). https://doi.org/10.1056/CAT.19.1137 Gandhi, T. K., & Lee, T. H. (2010). Patient safety beyond the hospital. The New England Journal of Medicine, 363(11), 1001-1003. https://doi.org/10.1056/NEJMp1003294 Gardam, M., Gitterman, L., Rykert, L., Vicencio, E., & Bailey, E. (2017). Five years of experience using front-line ownership to improve healthcare quality and safety. Healthcare Papers, 17(1), 8–23. https://doi.org/10.12927/hcpap.2017.25339 Gardner, R. M., Overhage, J. M., Steen, E. B., Munger, B. S., Holmes, J. H., Williamson, J. J., & 179 Detmer, D. E. (2009). Core Content for the Subspecialty of Clinical Informatics. Journal of the American Medical Informatics Association, 16(2), 153–157. https://doi.org/10.1197/jamia.M3045 Gerhant, W., & Ankit, A. (2020). Outpatient and virtual health care trends. Deloitte Insights. Retrieved June 28, 2021, from https://www2.deloitte.com/us/en/insights/industry/health-care/outpatient-virtual-health-care-trends.html Gheorghiu, B., & Hagens, S. (2016). Measuring interoperable EHR adoption and maturity: A Canadian example. BMC Medical Informatics and Decision Making, 16(1), 1–7. https://doi.org/10.1186/s12911-016-0247-x Giardina, T. D., Haskell, H., Menon, S., Hallisy, J., Southwick, F. S., Sarkar, U., Royse, K. E., & Singh, H. (2018). Learning from patients' experiences related to diagnostic errors is essential for progress in patient safety. Health Affairs Web Exclusive, 37(11), 1821-1827. https://doi.org/10.1377/hlthaff.2018.0698 Glymph, D. C., Olenick, M., Barbera, S., Brown, E. L., Prestianni, L., & Miller, C. (2015). Healthcare utilizing deliberate discussion linking events (HUDDLE): A systematic review. AANA Journal, 83(3), 183–188. Goldenhar, L. M., Brady, P. W., Sutcliffe, K. M., & Muething, S. E. (2013). Huddling for high reliability and situation awareness. BMJ Quality and Safety, 22(11), 899–906. https://doi.org/10.1136/bmjqs-2012-001467 Graber, M. L., Siegal, D., Riah, H., Johnston, D., & Kenyon, K. (2019). Electronic health record-related events in medical malpractice claims. Journal of Patient Safety, 15(2), 77–85. https://doi.org/https://dx.doi.org/10.1097/PTS.0000000000000240 Granja, C., Janssen, W., & Johansen, M. A. (2018). Factors determining the success and failure 180 of eHealth interventions: Systematic review of the literature. Journal of Medical Internet Research, 20(5), e10235-e10235. https://doi.org/10.2196/10235 Hanney, S. R., Castle-Clarke, S., Grant, J., Guthrie, S., Henshall, C., Mestre-Ferrandiz, J., Pistollato, M., Pollitt, A., Sussex, J., & Wooding, S. (2015). How long does biomedical research take? studying the time taken between biomedical and health research and its translation into products, policy, and practice. Health Research Policy and Systems, 13(1), 1-1. https://doi.org/10.1186/1478-4505-13-1 Harrison, M. I., Koppel, R., & Bar-Lev, S. (2007). Unintended consequences of information technologies in health care - An interactive sociotechnical analysis. Journal of the American Medical Informatics Association, 14(5), 542–549. https://doi.org/10.1197/jamia.M2384 Health Canada. (n.d.). Safe Medical Devices in Canada. Retrieved June 22, 2021, from https://www.canada.ca/en/health-canada/services/drugs-health-products/medical-devices/activities/fact-sheets/safe-medical-devices-fact-sheet.html Health Canada. (2019). Softward as Medical Device (SaMD): Definition and Classificaition. https://www.canada.ca/en/health-canada/services/drugs-health-products/medical-devices/application-information/guidance-documents/software-medical-device-guidance-document.html Health Quality & Safety Commission New Zealand. (2016). Patient safety reporting systems:A literature review of international practice. https://www.hqsc.govt.nz/assets/Reportable-Events/Publications/Patient-safety-reporting-systems-literature-review-Nov-2016.pdf Hewitt, T. A., & Chreim, S. (2015). Fix and forget or fix and report: A qualitative study of tensions at the front line of incident reporting. BMJ Quality and Safety, 24(5), 303–310. https://doi.org/10.1136/bmjqs-2014-003279 181 HIMMS (The Health Information and Management Systems Society). (n.d.) Outpatient-Electronic Medical Record Adoption Model. Retrieved December 16, 2021, from https://www.himss.org/what-we-do-solutions/digital-health-transformation/maturity-models/outpatient-electronic-medical-record-adoption-model-o-emram Holden, R. J., Carayon, P., Gurses, A. P., Hoonakker, P., Hundt, A. S., Ozok, A. A., & Rivera-Rodriguez, A. J. (2013). SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients. Ergonomics, 56(11), 1669–1686. https://doi.org/10.1080/00140139.2013.838643 Howe, J. L., Hettinger, A. Z., & Ratwani, R. M. (2019, July 1). Using patient safety-event report data to assess health-IT safety: benefits and challenges. The Lancet Digital Health, Vol. 1, pp. e104–e105. https://doi.org/10.1016/S2589-7500(19)30060-3 Howell, A.-M. M., Burns, E. M., Hull, L., Mayer, E., Sevdalis, N., & Darzi, A. (2017). International recommendations for national patient safety incident reporting systems: an expert Delphi consensus-building process. BMJ Quality and Safety, 26(2), 150–163. https://doi.org/10.1136/bmjqs-2015-004456 Hughes, R. G. (2008). Nurses at the “Sharp End” of Patient Care. In R. G. Hughes (Ed.), Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Agency for Healthcare Research and Quality (US). http://www.ncbi.nlm.nih.gov/pubmed/21328771 Hutchinson, A. M., & Estabrooks, C. A. (2013). Educational Theories. In I. D. Straus, S., Tetroe, J., & Graham (Ed.), Knowledge Translation in Health Care (pp. 298–307). Institute for Healthcare Improvement. (2021). Whole System Quality. http://www.ihi.org/resources/Pages/IHIWhitePapers/whole-system-quality.aspx Institute for Safe Medication Practices. (1999). Medication Error Prevention “Toolbox.” 182 https://www.ismp.org/resources/medication-error-prevention-toolbox Institute for Safe Medication Practices. (2020). Education is “predictably disappointing” and should never be relied upon alone to improve safety. Institute for Safe Medication Practices, 1–5. https://www.ismp.org/resources/education-predictably-disappointing-and-should-never-be-relied-upon-alone-improve-safety Jacobson, A. F., Hamilton, P., & Galloway, J. (1993). Obtaining and evaluating data sets for secondary analysis in nursing research. Western Journal of Nursing Research, 15(4), 483–494. https://doi.org/10.1177/019394599301500407 Jayasekara, R. S. (2012). Focus groups in nursing research: Methodological perspectives. Nursing Outlook, 60(6), 411–416. https://doi.org/10.1016/j.outlook.2012.02.001 Kaipio, J., Kuusisto, A., Hyppönen, H., Heponiemi, T., & Lääveri, T. (2020). Physicians’ and nurses’ experiences on EHR usability: Comparison between the professional groups by employment sector and system brand. International Journal of Medical Informatics, 134, 104018. https://doi.org/10.1016/j.ijmedinf.2019.104018 Kaminski, J. (2012). Diffusion of Innovation Theory. Canadian Journal of Nursing Informatics, 6(2), 1–7. Kang, H., Wang, F., Zhou, S., Miao, Q., & Gong, Y. (2017). Identifying and synchronizing health information technology (HIT) events from FDA medical device reports. Studies In Health Technology And Informatics, 245, 1048–1052. Kang, H., Wang, J., Yao, B., Zhou, S., & Gong, Y. (2019). Toward safer health care: a review strategy of FDA medical device adverse event database to identify and categorize health information technology related events. JAMIA Open, 2(1), 179–186. https://doi.org/10.1093/jamiaopen/ooy042 183 Karsh, B.-T. T., Weinger, M. B., Abbott, P. A., & Wears, R. L. (2010). Health information technology: Fallacies and sober realities. Journal of the American Medical Informatics Association, 17(6), 617–623. https://doi.org/10.1136/jamia.2010.005637 Kim, M. O., Coiera, E., & Magrabi, F. (2017). Problems with health information technology and their effects on care delivery and patient outcomes: A systematic review. Journal of the American Medical Informatics Association, 24(2), 246–260. https://doi.org/10.1093/jamia/ocw154 King, A., Daniels, J., Lim, J., Cochrane, D. D., Taylor, A., & Ansermino, J. M. (2010). Time to listen: A review of methods to solicit patient reports of adverse events. Quality and Safety in Health Care, 19(2), 148–157. https://doi.org/10.1136/qshc.2008.030114 King, J., Patel, V., Jamoom, E. W., & Furukawa, M. F. (2014;2013;). Clinical benefits of electronic health record use: National findings. Health Services Research, 49(1pt2), 392-404. https://doi.org/10.1111/1475-6773.12135 Klein, L. (2014). What do we actually mean by “sociotechnical”? On values, boundaries and the problems of language. Applied Ergonomics, 45(2), 137–142. https://doi.org/10.1016/j.apergo.2013.03.027 Knowles, M. S., Holton, E. ., & Swanson, R. A. (2012). The adult learner: The definitive classic in adult education and human resource development (7th ed.). Routledge. Kohn, L. T., Corrigan, J., & Donaldson, M. S. (2000). To err is human: Building a safer health system. National Academy Press. Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Prentice-Hall. Koppel, R. (2016). The health information technology safety framework: Building great 184 structures on vast voids. BMJ Quality and Safety, 25(4), 218–220. https://doi.org/10.1136/bmjqs-2015-004746 Koppel, R., Metlay, J. P., Cohen, A., Abaluck, B., Localio, A. R., Kimmel, S. E., & Strom, B. L. (2005). Role of computerized physician order entry systems in facilitating medication errors. Journal of American Medical Association, 293(10), 1197–1203. https://doi.org/10.1001/jama.293.10.1197 Kothari, A., McCutcheon, C., & Graham, I. D. (2017). Defining integrated knowledge translation and moving forward: A response to recent commentaries. International Journal of Health Policy and Management, 6(5), 299–300. https://doi.org/10.15171/ijhpm.2017.15 Kothari, A., & Wathen, C. N. (2013). A critical second look at integrated knowledge translation. Health Policy, 109(2), 187–191. https://doi.org/10.1016/j.healthpol.2012.11.004 Krittanawong, C. (2018). The rise of artificial intelligence and the uncertain future for physicians. European Journal of Internal Medicine, 48, e13–e14. https://doi.org/10.1016/J.EJIM.2017.06.017 Kushniruk, A. W., Bates, D. W., Bainbridge, M., Househ, M. S., & Borycki, E. M. (2013). National efforts to improve health information system safety in Canada, the United States of America and England. International Journal of Medical Informatics, 82(5), e149–e160. https://doi.org/10.1016/j.ijmedinf.2012.12.006 Kuziemsky, C. E. (2015). Review of social and organizational issues in health information technology. Healthcare Informatics Research, 21(3), 152–160. https://doi.org/10.4258/hir.2015.21.3.152 Kuziemsky, C. E., Nøhr, C., Aarts, J., Jaspers, M., & Beuscart-Zephir, M. C. (2013). Context sensitive health informatics: concepts, methods and tools. In M.-C. Beuscart-Zéphir, M. 185 Jaspers, C. E. Kuziemsky, C. Nohr, & J. Aarts (Eds.), Context Sensitive Health Informatics: Human and Sociotechnical Approaches (Vol. 194, pp. 1–7). https://doi.org/10.3233/978-1-61499-293-6-1 Langley, G. J., & O'Reilly for Higher Education. (2009). The improvement guide: A practical approach to enhancing organizational performance. Jossey-Bass. Lawati, M. H. Al, Dennis, S., Short, S. D., & Abdulhadi, N. N. (2018). Patient safety and safety culture in primary health care: A systematic review. BMC Family Practice, 19(1). https://doi.org/10.1186/s12875-018-0793-7 Lawrence, L. M., Bishop, A., & Curran, J. (2019). Integrated knowledge translation with public health policy makers : A scoping review. Healthcare Policy, 14(3), 55–77. https://doi.org/10.12927/hcpol.2019.25792 Leape, L. L., (2021). Making healthcare safe: The story of the patient safety movement. Springer International Publishing. https://doi.org/10.1007/978-3-030-71123-8 Levac, D., Colquhoun, H., & O’Brien, K. K. (2010). Scoping studies: Advancing the methodology. Implementation Science, 5(1), 1-9. https://doi.org/10.1186/1748-5908-5-69 Lourenco, L. M., Bursua, A., & Groo, V. L. (2016). Automatic errors: A case series on the errors inherent in electronic prescribing. Journal of General Internal Medicine, 31(7), 808–811. https://doi.org/10.1007/s11606-016-3606-5 Macrae, C. (2016). The problem with incident reporting. BMJ Quality & Safety, 25(2), 71-75. https://doi.org/10.1136/bmjqs-2015-004732 Magrabi, F., Aarts, J., Nohr, C., Baker, M., Harrison, S., Pelayo, S., Talmon, J., Sittig, D. F., & Coiera, E. (2012;2013;). A comparative review of patient safety initiatives for national health information technology. International Journal of Medical Informatics, 82(5), e139- 186 e148. https://doi.org/10.1016/j.ijmedinf.2012.11.014 Magrabi, F., Baker, M., Sinha, I., Ong, M., Harrison, S., Kidd, M. R., Runciman, W. B., & Coiera, E. (2015). Clinical safety of england's national programme for IT: A retrospective analysis of all reported safety events 2005 to 2011. International Journal of Medical Informatics, 84(3), 198-206. https://doi.org/10.1016/j.ijmedinf.2014.12.003 Magrabi, F., Liaw, S. T., Arachi, D., Runciman, W., Coiera, E., & Kidd, M. R. (2016). Identifying patient safety problems associated with information technology in general practice: An analysis of incident reports. BMJ Quality & Safety, 25(11), 870-880. https://doi.org/10.1136/bmjqs-2015-004323 Magrabi, F., Ong, M., Runciman, W., & Coiera, E. (2012). Using FDA reports to inform a classification for health information technology safety problems. Journal of the American Medical Informatics Association, 19(1), 45-53. https://doi.org/10.1136/amiajnl-2011-000369 Magrabi, F., Ong, M. S., Runciman, W., & Coiera, E. (2010). An analysis of computer-related patient safety incidents to inform the development of a classification. Journal of the American Medical Informatics Association, 17(6), 663–670. https://doi.org/10.1136/jamia.2009.002444 Maher, L., Gustafson, D., & Evans, A. (2010). Sustainability model and guide. NHS Institute for Innovation and Improvement. https://www.england.nhs.uk/improvement-hub/wp-content/uploads/sites/44/2017/11/NHS-Sustainability-Model-2010.pdf Maillet, É., Paré, G., Currie, L. M., Raymond, L., Ortiz de Guinea, A., Trudel, M.-C., & Marsan, J. (2018). Laboratory testing in primary care: A systematic review of health IT impacts. International Journal Of Medical Informatics, 116, 52–69. 187 https://doi.org/http://dx.doi.org/10.1016/j.ijmedinf.2018.05.009 Makary, M. A., & Daniel, M. (2016). Medical error—the third leading cause of death in the US. BMJ (Online), 353, i2139-i2139. https://doi.org/10.1136/bmj.i2139 Makeham, M., Magrabi, F., Hibbert, P., & Hardie, R.-A. (2017). Literature review and environmental scan on approaches to the review and investigation of Health IT-related patient safety incidents. https://www.safetyandquality.gov.au/sites/default/files/migrated/Literature-review-and-environmental-scan-on-approaches-to-review-and-investigation-of-Health-IT-related-patient-safety_Mar-2017.pdf Mallidou, A. A., Atherton, P., Chan, L., Frisch, N., Glegg, S., & Scarrow, G. (2018). Core knowledge translation competencies: A scoping review. BMC Health Services Research, 18(1). https://doi.org/10.1186/s12913-018-3314-4 Manogaran, G., Varatharajan, R., Lopez, D., Kumar, P. M., Sundarasekar, R., & Thota, C. (2017). A new architecture of Internet of Things and big data ecosystem for secured smart healthcare monitoring and alerting system. Future Generation Computer Systems. https://doi.org/10.1016/j.future.2017.10.045 Martin, G., Ghafur, S., Cingolani, I., Symons, J., King, D., Arora, S., & Darzi, A. (2019). The effects and preventability of 2627 patient safety incidents related to health information technology failures: a retrospective analysis of 10 years of incident reporting in England and Wales. The Lancet Digital Health, 1(3), e127–e135. https://doi.org/10.1016/S2589-7500(19)30057-3 Masotti, P., McColl, M. A., & Green, M. (2010). Adverse events experienced by homecare patients: A scoping review of the literature. International Journal for Quality in Health 188 Care. https://doi.org/10.1093/intqhc/mzq003 McMahon, M., Brown, A. D., Bornstein, S., & Tamblyn, R. (2019). Developing competencies for health system impact: Early lessons learned from the health system impact fellows. Healthcare Policy, 15, 61–72. https://doi.org/10.12927/hcpol.2019.25979 McMahon, M., & Tamblyn, R. (2019). The health system impact fellowship: Perspectives from the program leads comment on "CIHR health system impact fellows: Reflections on 'driving change' within the health system". International Journal of Health Policy and Management, 8(10), 623-626. https://doi.org/10.15171/ijhpm.2019.59 Meeks, D. W., Smith, M. W., Taylor, L., Sittig, D. F., Scott, J. M., & Singh, H. (2014). An analysis of electronic health record-related patient safety concerns. Journal of the American Medical Informatics Association, 21(6), 1053–1059. https://doi.org/10.1136/amiajnl-2013-002578 Menon, S., Singh, H., Giardina, T. D., Rayburn, W. L., Davis, B. P., Russo, E. M., & Sittig, D. F. (2017). Safety huddles to proactively identify and address electronic health record safety. Journal of the American Medical Informatics Association, 24(2), 261–267. https://doi.org/10.1093/jamia/ocw153 Merriam-Webster. (2019). Mediate. Retrieved from https://www.merriam-webster.com/dictionary/mediate Mitchell, I., Schuster, A., Smith, K., Pronovost, P., & Wu, A. (2016). Patient safety incident reporting: A qualitative study of thoughts and perceptions of experts 15 years after “To Err is Human.” BMJ Quality and Safety, 25(2), 92–99. https://doi.org/10.1136/bmjqs-2015-004405 Morris, Z. S., Wooding, S., & Grant, J. (2011). The answer is 17 years, what is the question: 189 Understanding time lags in translational research. Journal of the Royal Society of Medicine, 104(12), 510–520. https://doi.org/10.1258/jrsm.2011.110180 Nanji, K. C., Rothschild, J. M., Salzberg, C., Keohane, C. A., Zigmont, K., Devita, J., Gandhi, T. K., Dalal, A. K., Bates, D. W., & Poon, E. G. (2011). Errors associated with outpatient computerized prescribing systems. Journal of the American Medical Informatics Association, 18(6), 767-773. https://doi.org/10.1136/amiajnl-2011-000205 National Patient Safety Foundation. (2015). Free from Harm : Accelerating Patient Safety Improvement Fifteen Years after To Err Is Human. http://www.ihi.org/resources/Pages/Publications/Free-from-Harm-Accelerating-Patient-Safety-Improvement.aspx Nguyen, T., Graham, I. D., Mrklas, K. J., Bowen, S., Cargo, M., Estabrooks, C. A., Kothari, A., Lavis, J., Macaulay, A. C., MacLeod, M., Phipps, D., Ramsden, V. R., Renfrew, M. J., Salsberg, J., & Wallerstein, N. (2020). How does integrated knowledge translation (IKT) compare to other collaborative research approaches to generating and translating knowledge? learning from experts in the field. Health Research Policy and Systems, 18(1), 35-35. https://doi.org/10.1186/s12961-020-0539-6 Nicoll, L. H., & Beyea, S. C. (1999). Using secondary data analysis for nursing research. AORN Journal, 69(2), 428–433. https://doi.org/10.1016/S0001-2092(06)62504-0 Noble, D. J., & Pronovost, P. J. (2010). Underreporting of patient safety incidents reduces health care's ability to quantify and accurately measure harm reduction.Journal of Patient Safety, 6(4), 247-250. https://doi.org/10.1097/PTS.0b013e3181fd1697 Nora, C. R. D., & Beghetto, M. G. (2020). Patient safety challenges in primary health care: A scoping review. Revista Brasileira De Enfermagem, 73(5), e20190209- 190 11. https://doi.org/10.1590/0034-7167-2019-0209 Norman, D. (2013). The design of everyday things: Revised and expanded edition. Basic books. Odukoya, O. K., & Chui, M. A. (2013a). E-prescribing: a focused review and new approach to addressing safety in pharmacies and primary care. Research in Social & Administrative Pharmacy, 9(6), 996–1003. https://doi.org/https://dx.doi.org/10.1016/j.sapharm.2012.09.004 Odukoya, O. K., & Chui, M. A. (2013b). E-Prescribing: Characterisation of patient safety hazards in community pharmacies using a sociotechnical systems approach. BMJ Quality and Safety, 22(10), 816–825. https://doi.org/10.1136/bmjqs-2013-001834 Oster, C. A., & Deakins, S. (2018). Practical application of high-reliability principles in healthcare to optimize quality and safety outcomes. Journal of Nursing Administration, 48(1), 50–55. https://doi.org/10.1097/NNA.0000000000000570 Oxford Living Dictionaries. (2019). Technology, Mediate. Retrieved from https://en.oxforddictionaries.com/definition/technology Palchuk, M. B., Fang, E. A., Cygielnik, J. M., Labreche, M., Shubina, M., Ramelson, H. Z., Hamann, C., Broverman, C., Einbinder, J. S., & Turchin, A. (2010). An unintended consequence of electronic prescriptions: Prevalence and impact of internal discrepancies. Journal of the American Medical Informatics Association, 17(4), 472-476. https://doi.org/10.1136/jamia.2010.003335 Palojoki, S., Mäkelä, M., Lehtonen, L., & Saranto, K. (2017). An analysis of electronic health record-related patient safety incidents. Health Informatics Journal, 23(2), 134–145. https://doi.org/10.1177/1460458216631072 Palojoki, S., Pajunen, T., Lehtonen, L., & Saranto, K. (2017). FIN-TIERA: A tool for assessing 191 technology induced errors. Methods of Information in Medicine, 56(1), 1–12. https://doi.org/10.3414/ME16-01-0097 Panesar, S. S., deSilva, D., Carson-Stevens, A., Cresswell, K. M., Salvilla, S. A., Slight, S. P., Javad, S., Netuveli, G., Larizgoitia, I., Donaldson, L. J., Bates, D. W., & Sheikh, A. (2016;2015;). How safe is primary care? A systematic review. BMJ Quality & Safety, 25(7), 544-553. https://doi.org/10.1136/bmjqs-2015-004178 Paparella, S. F. (2018). First- and second-order problem solving: When rework and workarounds become an opportunity for improving safety. Journal of Emergency Nursing, 44(6), 652-654. https://doi.org/10.1016/j.jen.2018.07.008 Parthasarathy, R., & Kumar, P. (2020). Walking out of the hospital: The continued rise of ambulatory care and how to take advantage of it. https://www.mckinsey.com/~/media/mckinsey/industries/healthcare systems and services/our insights/walking out of the hospital/walking-out-of-the-hospital.pdf?shouldIndex=false Peters, M. D. J., Marnie, C., Tricco, A. C., Pollock, D., Munn, Z., Alexander, L., McInerney, P., Godfrey, C. M., & Khalil, H. (2020). Updated methodological guidance for the conduct of scoping reviews. JBI Evidence Synthesis, 18(10), 2119-2126. https://doi.org/10.11124/JBIES-20-00167 Petersen, C., Smith, J., Freimuth, R. R., Goodman, K. W., Jackson, G. P., Kannry, J., Liu, H., Madhavan, S., Sittig, D. F., & Wright, A. (2021). Recommendations for the safe, effective use of adaptive CDS in the US healthcare system: An AMIA position paper. Journal of the American Medical Informatics Association, 28(4), 677-684. https://doi.org/10.1093/jamia/ocaa319 192 Phillips, R. L., Dovey, S. M., Graham, D., Elder, N. C., & Hickner, J. M. (2006). Learning from different lenses: Reports of medical errors in primary care by clinicians, staff, and patients: A project of the american academy of family physicians national research network. Journal of Patient Safety, 2(3), 140-146. https://doi.org/10.1097/01.jps.0000235385.93406.d4 Piscotty, R. J., Kalisch, B., & Gracey-Thomas, A. (2015). Impact of healthcare information technology on nursing practice. Journal of Nursing Scholarship, 47(4), 287–293. https://doi.org/10.1111/jnu.12138 Plsek, P. (2017). Understanding the system you are in is key to improving it. Healthcare Papers, 17(1), 38–43. https://doi.org/10.12927/hcpap.2017.25335 Plsek, P. E, & Greenhalgh, T. (2001). Complexity science: The challenge of complexity in health care. BMJ, 323(7313), 625–628. https://doi.org/10.1136/bmj.323.7313.625 Plsek, Paul E, & Greenhalgh, T. (2001). Complexity science: The challenge of complexity in health care. British Medical Journal, 323(7313), 625–628. https://doi.org/10.1136/bmj.323.7313.625 Polit, D. F., & Beck, C. T. (2020). Nursing research: Generating and assessing evidence for nursing practice (11th Editi). Lippincott Williams & Wilkins. Powell-Cope, G., Nelson, A. L., & Patterson, E. S. (2008). Patient Care Technology and Safety. In R. G. Hughes (Ed.), Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Agency for Healthcare Research and Quality (US). Powell, L., Sittig, D. F., Chrouser, K., & Singh, H. (2020). Assessment of Health Information Technology-Related Outpatient Diagnostic Delays in the US Veterans Affairs Health Care System: A Qualitative Study of Aggregated Root Cause Analysis Data. JAMA Network Open, 3(6). https://doi.org/10.1001/jamanetworkopen.2020.6752 193 Ray, J. M., Ratwani, R. M., Sinsky, C. A., Frankel, R. M., Friedberg, M. W., Powsner, S. M., & Rosenthal, D. I. (2019). Six habits of highly successful health information technology: Powerful strategies for design and implementation. Journal of the American Medical Informatics Association, 26(10), 1109–1114. https://doi.org/10.1093/jamia/ocz098 Reason, J. (1995). Understanding adverse events: human factors. Quality and Safety in Health Care, 4(2), 80–89. https://doi.org/10.1136/qshc.4.2.80 Reason, J. (2000). Human error: Models and management. British Medical Journal, 320(7237), 768–770. https://doi.org/10.1136/bmj.320.7237.768 Reason, J. (2016). Managing the risks of organizational accidents. Taylor and Francis. https://doi.org/10.4324/9781315543543 Reason, J., Hollnagel, E., & Paries, J. (2006). Revisiting the “Swiss Cheese” Model of Accidents. European Organisation for the Safety Os Air Navigation, 12(13), 93–129. https://doi.org/10.1016/j.pec.2009.03.017 Recsky, C., Blackburn, L., Muniak, A., Rush, K. L., MacPhee, M., & & Currie, L. M. (2019, November 16-20). Technology-Mediated Adverse Events in Community and Primary Care. AMIA Annual Symposium, Washington DC, United States. https://amia.org/education-events/amia-2019-annual-symposium Recsky, C., & Currie, L. M. (2016, July 20-22). Technology-Mediated Adverse Events Taxonomy. Summer Institute for Nursing Informatics, Baltimore, MD, United States. https://www.nursing.umaryland.edu/academics/pe/events/sini/sini-2016/ Ricciardi, R., & Shofer, M. (2018). Moving patient safety into ambulatory settings and beyond. Journal of Nursing Care Quality, 33(3), 195-199. https://doi.org/10.1097/NCQ.0000000000000329 194 Risling, T. (2017). Educating the nurses of 2025: Technology trends of the next decade. Nurse Education in Practice, 22, 89–92. https://doi.org/10.1016/j.nepr.2016.12.007 Risling, T. L., & Low, C. (2019). Advocating for safe, quality and just care: What nursing leaders need to know about artificial intelligence in healthcare delivery. Nursing Leadership, 32(2), 31–45. https://doi.org/10.12927/cjnl.2019.25963 Robinson, T., Bailey, C., Morris, H., Burns, P., Melder, A., Croft, C., Spyridonidis, D., Bismantara, H., Skouteris, H., & Teede, H. (2020). Bridging the research–practice gap in healthcare: A rapid review of research translation centres in England and Australia. Health Research Policy and Systems, 18(1), 1-117. https://doi.org/10.1186/s12961-020-00621-w Rogers, C. R. (1969). Freedom to learn: A view of what education might become. C.E. Merrill Pub. Co. Runciman, W. B. (2002). Lessons from the Australian Patient Safety Foundation: Setting up a national patient safety surveillance system - Is this the right model? Quality and Safety in Health Care, 11(3), 246–251. https://doi.org/10.1136/qhc.11.3.246 Rycroft-Malone, J., Burton, C. R., Bucknall, T., Graham, I. D., Hutchinson, A. M., & Stacey, D. (2016). Collaboration and co-production of knowledge in healthcare: Opportunities and challenges editorial. Kerman University of Medical Sciences, 5(4), 221–223. https://doi.org/10.15171/ijhpm.2016.08 Salahuddin, L., & Ismail, Z. (2015). Classification of antecedents towards safety use of health information technology: A systematic review. International Journal of Medical Informatics, 84(11), 877–891. https://doi.org/10.1016/j.ijmedinf.2015.07.004 Samaranayake, N. R., Cheung, S. T. D., Chui, W. C. M., & Cheung, B. M. Y. (2012). Technology-related medication errors in a tertiary hospital: A 5-year analysis of reported 195 medication incidents. International Journal of Medical Informatics, 81(12), 828–833. https://doi.org/10.1016/j.ijmedinf.2012.09.002 Sandelowski, M. (2000). Focus on Research Methods Whatever Happened to Qualitative Description? Research in Nursing & Health, 23. https://doi.org/10.1002/1098-240X Sandelowski, M. (2010). What’s in a name? Qualitative description revisited. Research in Nursing & Health, 33(1), 77–84. https://doi.org/10.1002/NUR.20362 Schiff, G. D., Amato, M. G., Eguale, T., Boehne, J. J., Wright, A., Koppel, R., Rashidee, A. H., Elson, R. B., Whitney, D. L., Thach, T., Bates, D. W., & Seger, A. C. (2015). Computerised physician order entry-related medication errors: Analysis of reported errors and vulnerability testing of current systems. BMJ Quality & Safety, 24(4), 264-271. https://doi.org/10.1136/bmjqs-2014-003555 Schiff, G., Wright, A., Bates, D. W., Salazar, A., Amato, M. G., Slight, S. P., Sequist, T.D., Loudin, B., Smith, D., Adelman, J., Lambert, B., Galanter, W., Koppel, R., McGreevey, J.D., Taylor, K., Chan, I., & Brennan, C. (2015). Computerized prescriber order entry medication safety (CPOEMS): Uncovering and learning from issues and errors. United States Food and Drug Administration. https://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/downloads/1_CPOE_for_Medication_Orders.pdf Serou, N., Sahota, L. M., Husband, A. K., Forrest, S. P., Slight, R. D., & Slight, S. P. (2021). Learning from safety incidents in high-reliability organizations: A systematic review of learning tools that could be adapted and used in healthcare. International Journal for Quality in Health Care, 33(1). https://doi.org/10.1093/intqhc/mzab046 Seys, D., Wu, A. W., Gerven, E. V., Vleugels, A., Euwema, M., Panella, M., Scott, S. D., 196 Conway, J., Sermeus, W., & Vanhaecht, K. (2013). Health care professionals as second victims after adverse events: A systematic review. Evaluation and the Health Professions, 36, 135–162. https://doi.org/10.1177/0163278712458918 Shojania, K. G. (2008). The frustrating case of incident-reporting systems. Quality & Safety in Health Care, 17(6), 400-402. https://doi.org/10.1136/qshc.2008.029496 Shojania, K. G., & Marang-van de Mheen, Perla J. (2020). Identifying adverse events: Reflections on an imperfect gold standard after 20 years of patient safety research.BMJ Quality & Safety, 29(4), 265-270. https://doi.org/10.1136/bmjqs-2019-009731 Shortliffe, E. H., Detmer, D. E., & Munger, B. S.Clinical informatics: Emergence of a new profession. (pp. 3-21). Springer International Publishing. https://doi.org/10.1007/978-3-319-22753-5_1 Sim, S. M., Lai, J., Aubrecht, K., Cheng, I., Embrett, M., Ghandour, E. K., Highet, M., Liu, R., Casteli, C. P., Saari, M., Ouédraogo, S., & Williams-Roberts, H. (2019). CIHR health system impact fellows: Reflections on "driving change" within the health system. International Journal of Health Policy and Management, 8(6), 325-328. https://doi.org/10.15171/ijhpm.2018.124 Singh, H., Classen, D. C., & Sittig, D. F. (2011). Creating an oversight infrastructure for electronic health record-related patient safety hazards. Journal of Patient Safety, 7(4), 169–174. https://doi.org/10.1097/PTS.0b013e31823d8df0 Singh, H., & Sittig, D. F. (2020). A sociotechnical framework for safety-related electronic health record research reporting: The SAFER reporting framework. Annals of Internal Medicine, 172(11_Supplement), S92-S100. https://doi.org/10.7326/M19-0879 Singh, H., Sittig, D. F., & Gandhi, T. K. (2021). Fighting a common enemy: A catalyst to close 197 intractable safety gaps. BMJ Quality and Safety, 30(2), 141–145. https://doi.org/10.1136/bmjqs-2020-011390 Singh, H., Thomas, E. J., Sittig, D. F., Wilson, L., Espadas, D., Khan, M. M., & Petersen, L. A. (2010). Notification of abnormal lab test results in an electronic medical record: Do any safety concerns remain? American Journal of Medicine, 123(3), 238–244. https://doi.org/http://dx.doi.org/10.1016/j.amjmed.2009.07.027 Sittig, D. F., Belmont, E., & Singh, H. (2018). Improving the safety of health information technology requires shared responsibility: It is time we all step up. Healthcare, 6(1), 7–12. https://doi.org/10.1016/j.hjdsi.2017.06.004 Sittig, D. F., Salimi, M., Aiyagari, R., Banas, C., Clay, B., Gibson, K. A., Goel, A., Hines, R., Longhurst, C. A., Mishra, V., Sirajuddin, A. M., Satterly, T., & Singh, H. (2018). Adherence to recommended electronic health record safety practices across eight health care organizations. Journal of the American Medical Informatics Association, 25(7), 913-918. https://doi.org/10.1093/jamia/ocy033 Sittig, D. F., & Singh, H. (2010). A new sociotechnical model for studying health information technology in complex adaptive healthcare systems. Quality and Safety in Health Care, 19 Suppl 3(Suppl 3), i68–i74. https://doi.org/10.1136/qshc.2010.042085 Sittig, D. F., & Singh, H. (2015). In Sittig D. F., Singh H.(Eds.), SAFER electronic health records: Safety assurance factors for EHR resilience. Apple Academic Press. https://doi.org/10.1201/b18371 Sittig, D. F., & Singh, H. (2017). A sociotechnical approach to electronic health record related safety. In Key Advances in Clinical Informatics (pp. 197–216). https://doi.org/10.1016/B978-0-12-809523-2.00014-5 198 Sittig, D. F., Singh, H., (2017). A Sociotechnical Approach to Electronic Health Record Related Safety. In A. Sheikh, D. W., Bates, A. Wright, & K. Cresswell (Eds.), Key Advances in Clinical Informatics: Transforming Health Care Through Health Information Technology. Academic Press. Sittig, D. F., Wright, A., Ash, J., & Singh, H. (2016). New unintended adverse consequences of electronic health records. IMIA Yearbook, 7–12. https://doi.org/10.15265/IY-2016-023 Sittig, D. F., Wright, A., Coiera, E., Magrabi, F., Ratwani, R., Bates, D. W., & Singh, H. (2020). Current challenges in health information technology–related patient safety. Health Informatics Journal, 26(1), 181–189. https://doi.org/10.1177/1460458218814893 Slight, S. P., Howard, R., Ghaleb, M., Barber, N., Franklin, B. D., & Avery, A. J. (2013). The causes of prescribing errors in English general practices: A qualitative study. British Journal of General Practice, 63(615), e713-e720. https://doi.org/10.3399/bjgp13X673739 Slight, S. P., Seger, D. L., Nanji, K. C., Cho, I., Maniam, N., Dykes, P. C., & Bates, D. W. (2013). Are we heeding the warning signs? Examining providers’ overrides of computerized drug-drug interaction alerts in primary care. PLoS ONE, 8(12), e85071. https://doi.org/10.1371/journal.pone.0085071 Soong, C., & Shojania, K. G. (2020). Education as a low-value improvement intervention: often necessary but rarely sufficient. BMJ Qual Saf, 29, 353–357. https://doi.org/10.1136/bmjqs-2019-010411 Staggers, N., Elias, B. L., Makar, E., & Alexander, G. L. (2018). The imperative of solving nurses’ usability problems with health information technology. Journal of Nursing Administration, 48(4), 191–196. https://doi.org/10.1097/NNA.0000000000000598 Staggers, N., Gassert, C. A., & Curran, C. (2001). Informatics competencies for nurses at four 199 levels of practice. Journal of Nursing Education, 40(7), 303–308. https://doi.org/10.3928/0148-4834-20011001-05 Stevenson, J. E., & Nilsson, G. (2012). Nurses’ perceptions of an electronic patient record from a patient safety perspective: A qualitative study. Journal of Advanced Nursing, 68(3), 667–676. https://doi.org/10.1111/j.1365-2648.2011.05786.x Stewart, G. L., Manges, K. A., & Ward, M. M. (2015). Empowering sustained patient safety: The benefits of combining top-down and bottom-up approaches. Journal of Nursing Care Quality, 30(3), 240–246. https://doi.org/10.1097/NCQ.0000000000000103 Straus, S., Tetroe, J., & Graham, I. D. (Eds.). (2013). Knowledge translation in health care: moving from evidence to practice. John Wiley & Sons. Straus, S. E., Tetroe, J., & Graham, I. (2009). Defining knowledge translation. CMAJ, 181(3–4), 165–168. https://doi.org/10.1503/CMAJ.081229 Strudwick, G., Nagle, L., Kassam, I., Pahwa, M., & Sequeira, L. (2019). Informatics Competencies for Nurse Leaders: A Scoping Review. Journal of Nursing Administration, 49(6), 323–330. https://doi.org/10.1097/NNA.0000000000000760 Strudwick, G., Wiljer, D., Inglis, F. (2020). Nursing and Compassionate Care in a Technological World: A Discussion Paper. https://www.ams-inc.on.ca/wp-content/uploads/2020/02/Nursing-and-Compassionate-Care.pdf Sujan, M. (2015). An organisation without a memory: A qualitative study of hospital staff perceptions on reporting and organisational learning for patient safety. Reliability Engineering and System Safety, 144, 45–52. https://doi.org/10.1016/j.ress.2015.07.011 The Health Foundation. (2011). Evidence scan: Levels of harm in primary care. http://www.health.org.uk/publications/levels-of-harm-in-primary-care 200 The Joanna Briggs Institute. (2015). Joanna Briggs Institute Reviewers’ Manual. https://doi.org/10.1017/CBO9781107415324.004 Topaz, M., Ronquillo, C., Peltonen, L., Pruinelli, L., Sarmiento, R. F., Badger, M. K., Ali, S., Lewis, A., Georgsson, M., Jeon, E., Tayaben, J. L., Kuo, C., Islam, T., Sommer, J., Jung, H., Eler, G. J., Alhuwail, D., & Lee, Y. (2016). Nurse informaticians report low satisfaction and multi-level concerns with electronic health records: Results from an international survey. AMIA . Annual Symposium Proceedings, 2016, 2016-2025. Tsang, C., Majeed, A., & Aylin, P. (2012). Routinely recorded patient safety events in primary care: A literature review. Family Practice, 29(1), 8–15. https://doi.org/http://dx.doi.org/10.1093/fampra/cmr050 Tucker, A., & Edmondson, A. (2003). Why hospitals don’t learn from failures: Organizational and psychological dynamics that inhibit system change. California Management Review, 45(2), 55–72. Turner, P., Kushniruk, A., & Nohr, C. (2017). Are we there yet? Human factors knowledge and health information technology - The challenges of implementation and impact. Yearbook of Medical Informatics, 26(1), 84–91. https://doi.org/dx.doi.org/10.15265/IY-2017-014 Vaismoradi, M., Turunen, H., & Bondas, T. (2013). Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nursing & Health Sciences, 15(3), 398-405. https://doi.org/10.1111/nhs.12048 Van De Castle, B., Kim, J., Pedreira, M. L. G., Paiva, A., Goossen, W., & Bates, D. W. (2004). Information technology and patient safety in nursing practice: An international perspective. International Journal of Medical Informatics, 73(7–8), 607–614. https://doi.org/10.1016/j.ijmedinf.2004.04.007 201 Vancouver Coastal Health. (2010). Who we Serve. Retrieved December 1, 2021 from: http://www.vch.ca/about-us/who-we-serve Verbakel, N. J., Langelaan, M., Verheij, T. J. M., Wagner, C., & Zwart, D. L. M. (2016). Improving patient safety culture in primary care: A systematic review. Journal of Patient Safety, 12(3), 152-158. https://doi.org/10.1097/PTS.0000000000000075 Vincent, C, & Amalberti, R. (2016). Safer healthcare. Springer International Publishing. Vincent, C., Carthey, J., Macrae, C., & Amalberti, R. (2017). Safety analysis over time: Seven major changes to adverse event investigation. Implementation Science, 12(1), 1–10. https://doi.org/10.1186/s13012-017-0695-4 Vindrola-Padros, C., Pape, T., Utley, M., & Fulop, N. J. (2017). The role of embedded research in quality improvement: A narrative review. BMJ Quality & Safety, 26(1), 70–80. https://doi.org/10.1136/BMJQS-2015-004877 Wachter, R. M. (2006). Expected and unanticipated consequences of the quality and information technology revolutions. Journal of the American Medical Association, 295(23), 2780–2783. https://doi.org/10.1001/jama.295.23.2780 Wachter, R. M. (2015a). The digital doctor: Hope, hype and harm at the dawn of medicine’s computer age. McGraw-Hill. Wachter, R. M. (2015b). The Overdose: Harm in a Wired Hospital. Medium. https://medium.com/backchannel/how-technology-led-a-hospital-to-give-a-patient-38-times-his-dosage-ded7b3688558 Walker, L., & Clendon, J. (2016). The case for end-user involvement in design of health technologies. Journal of Telemedicine and Telecare, 22(8), 443–446. https://doi.org/10.1177/1357633X16670479 202 Ward, V., Smith, S., House, A., & Hamer, S. (2012). Exploring knowledge exchange: A useful framework for practice and policy. Social Science and Medicine, 74(3), 297–304. https://doi.org/10.1016/j.socscimed.2011.09.021 Warm, D., & Edwards, P. (2012). Classifying health information technology patient safety related incidents – an approach used in Wales. Applied Clinical Informatics, 3(2), 248-257. https://doi.org/10.4338/ACI-2012-03-RA-0010 Weick, K., Sutcliffe, K., & Obstfeld, D. (2008). Organizing for high reliability: Processes of collective mindfulness. In R.S. Sutton and B.M. Staw (Ed.), Crisis Management (Vol. 3, pp. 31–66). Weiner, J. P., Kfuri, T., Chan, K., & Fowles, J. B. (2007). “e-Iatrogenesis”: The most critical unintended consequence of CPOE and other HIT. Journal of the American Medical Informatics Association, 14(3), 387–388. https://doi.org/10.1197/jamia.M2338 Wensing, M., Bosch, M., & Grol, R. (2013). Selecting KT interventions knowledge translation interventions In S. Straus., J. Tetroe., & I. D.Graham, (Eds.), Knowledge Translation in Health Care: Moving from evidence to practice, 2nd Edition (pp.150-162). Williams, A. (2019). Nursing informaticians address patient safety to improve usability of health information technologies. Studies in Health Technology and Informatics, 257, 501–507. https://doi.org/10.3233/978-1-61499-951-5-501 Wilson, K. A., Burke, C. S., Priest, H. A., & Salas, E. (2005). Promoting health care safety through training high reliability teams. Quality and Safety in Health Care, Vol. 14, pp. 303–309. https://doi.org/10.1136/qshc.2004.010090 World Health Organization. (2009). Conceptual Framework for the International Classification for Patient Safety: Version 1.1 (Vol. 1). https://doi.org/WHO/IER/PSP/2010.2 203 Wu, A. W. (2000). Medical error: The second victim. BMJ, 172, 358–359. https://doi.org/10.12968/hmed.2012.73.Sup10.C146 Yackel, T. R., & Embi, P. J. (2010). Unintended errors with EHR-based result management: A case series. Journal of the American Medical Informatics Association, 17(1), 104–107. https://doi.org/https://dx.doi.org/10.1197/jamia.M3294 Zheng, K., Abraham, J., Novak, L. L., Reynolds, T. L., & Gettinger, A. (2016). A survey of the literature on unintended consequences associated with health information technology: 2014–2015. IMIA Yearbook. https://doi.org/10.15265/IY-2016-036 Zhu, J., Stuver, S. O., Epstein, A. M., Schneider, E. C., Weissman, J. S., & Weingart, S. N. (2011). Can we rely on patients’ reports of adverse events? Medical Care, 49(10), 948–955. https://doi.org/10.1097/MLR.0b013e31822047a8 204 Appendices Appendix A Scoping Review Search Terms, Databases, and Yields Source n Strategy Medline 1061 ((ambulatory care/OR community health services/OR community health nursing/OR home care services/OR primary health care/OR general practice/OR public health/OR outpatients/) OR (“ambulatory care” or outpatient or “community health” or “primary health” or public health”).ti.ab) AND ((public health informatics/OR informatics/OR nursing informatics/OR medical informatics/OR information systems/OR information technology/OR biomedical technology/OR medical records systems, computerized/OR medical order entry systems/OR medical records/OR electronic health records/OR health information exchange/OR nursing records/) OR (informatics or “information system*” or “health information technology” or “electronic health record*” or “EHR” or “electronic medical record*” or “EMR” or “medical record*” or “nursing record*”).ti.ab) AND ((patient harm/OR patient safety/OR medical errors/OR medication errors/OR near miss, healthcare/OR iatrogenic disease/ OR (“patient safety” or harm or “unintended consequence*” or “technology induced error*” or “iatrogen*” or “adverse event*”)ti.ab OR ((medical or medication* or drug*)adj1 error*).ti.ab) EMBASE 938 ((ambulatory care/OR general practice/OR community care/ OR home care/OR public health/OR outpatient care) OR (“ambulatory care” or outpatient or “community health” or “primary health” or “public health”).ti) AND ((information science/OR bioinformatics/OR medical informatics/OR nursing informatics/ OR information system/OR computerized provider order entry/OR decision support system/OR medical information system/OR nursing information system/OR information technology/ medical record/OR electronic health record/OR electronic medical record/) OR (informatics or "information system*" or "health information technology" or "electronic health record*" or "ehr" or "electronic medical record*" or "emr" or "medical record*" or "nursing record*”).ti) AND ((patient safety/OR medical error/OR diagnostic error/OR medication error/OR “near miss (health care)”/OR iatrogenic disease/OR adverse event/OR (“patient safety” or harm or “unintended consequence” or “technology induced error” or “iatrogen*” or “adverse event*”).ti OR (medical or medication* or drug*) adj1 error*).ti) CINAHL 503 (MH “ambulatory care”) OR (MH “ambulatory care nursing”) OR (MH “community health services”) OR (MH “community health nursing+”) OR (community health centers+”) OR (MH “home health care+”) OR (MH “immunization programs”) OR (MH “primary health care) OR (MH “family practice”) or (TI(“ambulatory care” or “community health” or “primary care” or “public health”)) 205 AND (MM “information science”) OR (TI(informatics or “information system” or “health information technology” or “electronic health record*” or “EHR” or “electronic medical record*”or “EMR” or “medical record*” or “nursing record*”)) AND (MH “patient safety) OR (MH “adverse health care event+”) OR (MH “incident reports”) OR (MH iatrogenic disease”) OR (TI(“patient safety” or harm or “unintended consequence” or “technology induced error” or “iatrogen*” or “adverse event*”)) PsycINFO 64 (DE “outpatient treatment” or DE “community heath” OR DE “community services” OR DE “primary health care” OR DE “public health” or TI (“outpatient” or “community” or “primary care” or “public health”) OR AB (“outpatient” or “community” or “primary care” or “public health”)) AND (DE “information and communication technology” OR DE “information systems” OR DE “information science” OR DE “health information” OR DE “cognitive processes” OR DE “health information technology” OR DE “electronic health records” OR TI (“informatics” or “information system” or “health information technology” or “electronic health record*” or “electronic medical record*”) OR AB (“informatics” or “information system” or “health information technology” or “electronic health record*” or “electronic medical record*”)) AND (DE “patient safety” OR TI (“patient safety” or “adverse event*” or “unintended consequence” OR “technology induced error*” OR “iatrogen*”) OR AB (“patient safety” or “adverse event*” or “unintended consequence” OR “technology induced error*” OR “iatrogen*”)) Web of Science 7 TI=(ambulatory OR “community care” OR “primary care” OR “home care” OR “general practice” OR “public health”) AND TI=(informatics or “healt information” OR “electronic health record” OR “EHR” OR “electronic medical record” OR “EMR” OR “computerize*”) AND TI=(“patient safety” OR “adverse event*” OR error OR iatrogen* OR “unintended consequence”) Engineering Village 361 (((("ambulatory care" or "primary care" or "general practice" or "home care" or "public health") WN KY) AND (("patient safety" or "human computer interaction" or "unintended consequence" or "adverse event" or "safety engineering" or "human factors" or ergonomics) WN KY)) AND ((informatics or "medical informatics" or computer*) WN KY)) Google Scholar 100 technology AND "patient safety" AND (community OR "primary care") Google 100 technology AND ("patient safety" OR "adverse event") AND (community OR "primary care") Hand search 7 TOTAL 3134 206 Appendix B Interrater Agreement Calculations Interrater agreement (percent agreement) was calculated in two different ways: 1. Total Agreement: Did the entire coding assigned by each coder match? For example: Report # Coder 1 Coder 2 Agreement 1 4 4 100% 2 1,4 1,4 100% 3 2,5 1,5 0% 2. Agreement by individual Code: What proportion of the individual codes assigned to each report by each coder matched? For example: Report # Coder 1 Coder 2 Agreement 1 4 4 100% 2 1,4 1,4 100% 3 2,5 1,5 50% The table below includes example sociotechnical codes and both calculations for interrater agreement to illustrate the difference. Report # Coder 1 Coder 2 Total Agreement Agreement by Individual Code 1 4 4 100% 100% 2 1,4 1,4 100% 100% 3 2,5 1,5 0% 50% 4 2,4,6 2,4,6 100% 100% 5 1,2,4 1,2,3 0% 67% Percent Agreement 60% 83.4% 207 Appendix C Combinations of Sociotechnical Dimensions Assigned Sociotechnical Dimensions # of times assigned Personnel 13 Clinical Content AND Personnel 9 Hardware & Software 9 Clinical Content 8 Workflow & Communication 7 Personnel AND Workflow & Communication 6 User Interface AND Personnel 6 User Interface AND Personnel AND Workflow & Communication 5 Clinical Content AND Workflow & Communication 5 User Interface AND Workflow & Communication 3 Hardware & Software AND Internal Policies 3 Hardware & Software AND Personnel and Workflow & Communication 2 Hardware & Software AND Workflow & Communication 2 Workflow & Communication AND Internal Policies 2 Clinical Content AND User Interface and Personnel 2 Clinical Content AND External Regulations 2 User Interface AND Internal Policies 2 User Interface 2 Internal Policies 2 External Regulations 2 Clinical Content AND Workflow & Communication AND Internal Policies 1 Hardware & Software AND Workflow & Communication AND Internal Policies 1 Hardware & Software AND Workflow & Communication AND External Regulations 1 Workflow & Communication AND Internal Policies AND External Regulation 1 Workflow & Communication AND External Regulation 1 Clinical Content AND Personnel AND Internal Policies AND External Regulations 1 Clinical Content AND Personnel AND Internal Policies 1 Hardware & Software AND Personnel AND Internal Policies 1 Hardware & Software AND Personnel 1 Personnel AND Internal Policies 1 Clinical Content AND User Interface 1 Hardware & Software AND Measurement & Monitoring 1 Total 104 DRAFT 208 Appendix D Detailed Description of IKT Activities Activity Detailed Description Research Project Design Over a series of meetings, the research team discussed the needs of the organization and the areas of expertise of the researcher to design a research project that included an IKT approach. Upon award of funding, the proposed research plan was reviewed to verify it continued to align with both organizational and research priorities. The research project was also refined with additional details related to timelines and logistics. The research project design was then reviewed with the leader of the CCI team, as well as the CCI team members to gather their input and to make them aware of all of the research studies and the IKT approach, including knowledge support available from the embedded researcher working within the team. The original research plan had two main aims: 1) understand the incidence of TMAE; and 2) develop methods to mitigate TMAE. The original methods for aim 1 were carried out as planned (see Chapter 4), but the methods for aim 2 changed from the plan to develop and test an investigative approach to TMAE to the approach described here. This change was in line with the expected need to co-create a process with the relevant stakeholders. Upon the onset of the Covid-19 pandemic, the research team reviewed the research project and adjusted some of the activities in accordance with the public health measures and shifting organizational priorities and needs. The IKT activities that resulted in response to the Covid-19 pandemic are described in Table 5.2. Organizational Meetings The embedded researcher met with various knowledge users on a weekly, monthly, and bi-monthly basis depending on knowledge users’ role in the organization. For the knowledge users who were executive leaders and part of the research team, monthly or quarterly individual meetings were scheduled to provide updates on the progress of the research and application of knowledge to support organizational efforts and obtain input into the project on a continuing basis (Gagliardi et al., 2016). Meetings with the CCI team leader were at least monthly to review the needs of the CCI team and provide ongoing progress reports of the research studies. Throughout the project the embedded researcher attended weekly meetings with the entire CCI team to iteratively assess the knowledge level(s) and contextual factors relevant to the design of IKT activities, report on the progress of the research studies, and participate in discussions, with a particular focus on patient safety and TMAE. Finally, the embedded researcher attended quarterly executive strategic planning meetings concerning HIT for the organization’s primary and community care services to observe the discussions. DRAFT 209 Activity Detailed Description Relationship Building Upon joining the CCI team, the embedded researcher focused on becoming familiar with the team structure, the various roles and functions, and the people that made up the team. Activities during this period included attending weekly status meetings to learn more about their work, and spending time with individual team members to learn more about their backgrounds, experiences, and day to day responsibilities within their respective roles. Beyond this team, the embedded researcher also met with people from other areas of the organization who were connected to the work of the CCI team such as leaders and staff from clinical areas across primary and community care as well as organizational leaders for risk management, quality and safety, professional practice, and information management. Being in a relatively fluid position within the organization also enabled the embedded researcher to move among various clinical and administrative areas and facilitate new connections among knowledge users. Generating opportunities for people working in different areas to learn from each other or work together supported an exchange of knowledge that could provide mutual benefits to their respective areas (Gagliardi et al., 2016). For example, when safety concerns related to gaps in information following discharge from hospital were identified, the embedded researcher brought point-of-care clinicians together with clinical informaticians from acute and community settings to assess and resolve the issue. Knowledge and Context Assessment At the beginning of the overall research project, the embedded researcher began to assess the current state within the CCI team and the broader organization regarding HIT safety. As described earlier, the CCI team was a newly formed team at the outset of the project. However, a number of team members had worked together on previous projects and the team was assessed to be cohesive and enthusiastic with a strong sense of dedication to quality in their work. Knowledge users in executive positions were highly collaborative and expressed commitment to both the research and IKT activities in the study. Recognizing the complex nature of the organization, the embedded researcher focused on learning about the work of the CCI team, and the multifaceted interactions between the team and other areas of the organization (Sim et al., 2019; Sittig & Singh, 2010). Discussions with knowledge users at all levels of the organization, with a focus on problems that surfaced concerning HIT safety, provided insight into their existing level of knowledge related to TMAE, as well as approaches that might be best suited to address these needs (Hutchinson & Estabrooks, 2013; Ward, Smith, House, & Hamer, 2012). Drawing on systems thinking, patient safety, and andragogical and constructivist learning theories, the embedded researcher looked to understand the context, including: understanding the complexity of the environment, assessing problems where new knowledge might be relevant, and identifying opportunities for experiential learning (Knowles et al., 2012; Kolb, 1984). DRAFT 210 Activity Detailed Description Ad Hoc Support Being physically present in the work setting afforded additional opportunities for the embedded researcher to interact with knowledge users to support knowledge translation (Gagliardi et al., 2016). For example, team members might ask for input or feedback on their work in the moment, such as reviewing a proposed system change and considering potential threats to patient safety. At other times, the embedded researcher was invited to a meeting on a specific topic that the CCI team was managing (e.g., HIT workflow and communication challenges related to using a hybrid of electronic and paper documentation systems) to offer an analysis or provide input. The various opportunities to consult, coach, and connect were all pragmatic opportunities to support translation of knowledge related to HIT safety into practice (Wensing et al., 2013). These ad hoc connections provided opportunities for the embedded researcher to integrate learning into conversations with knowledge users, conveying relevant content, and supporting the application of new knowledge immediately into practice. Further to this, the embedded researcher was able to role model a stance of psychological safety and systems thinking inherent (Bates et al., 1999; A. Edmondson, 1999; Reason, 2000). The embedded researcher ‘walked the talk’, in these ad hoc interactions, demonstrating that “avoiding consequential failures means rapidly identifying and correcting small failures” (Edmondson, 2011, para 12). Formal Learning Sessions The embedded researcher designed and delivered tailored formal learning sessions to the CCI team. The first formal learning session was provided in Spring 2019 as part of an existing lunch and learn lecture series. The content for this session was determined in collaboration with the team leader, and based on the embedded researcher’s assessment of the team’s knowledge to date. The learning session included an introduction to patient safety, sociotechnical theory, and examples of TMAE from the literature. This lecture was delivered in person and simulcast to team members who were working offsite. Approximately one year later (delayed due to Covid-19 pandemic), a follow-up series of learning sessions was designed and delivered (Wensing et al., 2013). This series was tailored to the knowledge users’ learning needs in relation to recent HIT safety issues and TMAE that had been reported to the team. The objective of the learning series was to support team members to be able to articulate risk factors and mitigation strategies related to TMAE. The learning series was delivered in three parts: 1) a webinar focused on patient safety, systems thinking, sociotechnical theory, and human factors 2) an independent reading of a TMAE case study (Wachter, 2015b) with reflective questions, and 3) a webinar focused on discussing and analyzing the TMAE case study and applying the concepts presented in the first webinar. Each webinar was offered twice to allow flexibility for team members to attend according to their availability. Following each webinar, a recording of the webinar was posted on the team’s internal online portal. Additional resources such as related academic articles, and online videos and podcasts, were also provided for participants’ interest. These resources were emailed to all members of the CCI team and posted on the team portal. DRAFT 211 Appendix E PSLS User Guide VCH Clinical Informatics Community PSLS Event Review Guide *Please be mindful of the sensitivity of information in these events* Is this event appropriate for CI follow up? □ NO 1. Please complete PSLS Clinical Informatics follow-up section in PSLS system: □Name and Reference tab: □ Change Current Status to being reviewed if needed □ Clinical Informatics Handler □CST - Clinical Informatics follow up tab: □ Computer system contributed – field updated? 2. Provide feedback to handler/reporter. □ YES 1. Review event in a short huddle with 3 or more team members, within 1 week of date reported. 2. Record relevant information into PSLS (communicate with handler/reporter to get more information as needed) □Name and Reference tab: □ Change Current Status to being reviewed if needed □ Clinical Informatics Handler □Review all pages with details (orange text) □CST - Clinical Informatics follow up tab: □ Computer system contributed – field updated? □ Date feedback given to reporter □ Date Clinical Informatics Handler completed review □Follow-up tab: □ Notes about review activities: type in brief summary of CI activities 3. Input data in the Analysis and Tracking excel sheet on sharepoint, and review analysis to date (optional Sociotechnical Analysis Worksheet below). 4. Provide feedback to handler/reporter (optional email template). See screenshots of PSLS and sharepoint below for help if needed. DRAFT 212 DRAFT 213 DRAFT 214 DRAFT 215 PSLS Sociotechnical Analysis Worksheet PSLS Dimension Brief Description Notes Hardware and Software Purely technical; the physical devices and software. Content All textual data stored within the HIT system. User Interface Aspects of the computer that the user can touch, see, and hear. Personnel Humans involved in the design, development, implementation, and use of HIT; purely social. Workflow and Communication Processes to deliver care effectively; people need to work together. Organizational Policies, Procedures, and Culture Influence of leadership and resources; affects all dimensions in the model and aligns with external regulation. External Rules and Regulations Forces that facilitate or constrain HIT in the clinical setting. Measurement and Monitoring Assessing the effects of HIT on an ongoing basis. *Additional resources on Sociotechnical Analysis: EHR Safety and Sociotechnical Analyses A New Socio-technical Model for Studying Health Information Technology in Complex Adaptive Healthcare Systems Additional resource on PSLS: BCPSLS Data Principles DRAFT 216 Closing the Loop email template To be sent to local handler and/or reporter after Clinical Informatics Community analysis complete. *Feel free to revise/edit as needed* Hello _______, I am emailing to let you know that the VCH Community Clinical Informatics team has followed up on the PSLS report recently submitted from your team where it was indicated that a computer system was involved in the event. Alongside the activities of the local a PSLS handler, our team has begun internally tracking events related to computers to ensure optimal functioning of our IT systems and to promote quality of care. We have analyzed this report to assess the role of technology in the event and will continue to monitor our systems as a result. INSERT: ADDITIONAL DETAILS ABOUT FOLLOW UP ON THIS EVENT. We appreciate people taking the time to report events! If you have any questions, please feel free to contact me. DRAFT 217 Appendix F Safety Huddle Guiding Principles and Agenda Guiding Principles • Keep it brisk. • The purpose is to LEARN and IMPROVE. • It’s a team sport. • Acknowledge complexity. And keep it simple. • Always close the loop. • Take a step back now and then. • Pursue perfection. DRAFT 218 DRAFT 219 Appendix G Focus Group Script and Questions Introduction (5 min) Good morning/afternoon. Thank you for taking the time to join me today. As indicated in the consent form, the purpose of this discussion is to help me understand your thoughts on how technology may impact patient safety, and how you came to adopt these ways of thinking. I am conducting 2 focus groups within the Clinical Informatics team with up to 10 people. Before we get into our discussion, let me make a few requests: • First, as outlined in the consent form you should know that I am audio and video recording the session so that we can refer back to the discussion when I write my report. If anyone is uncomfortable with participating or being recorded please say so and, of course, you are free to leave. • I am going to go through a series of five or six questions and we will have about 8 minutes for each question, with about 10 minutes at the end for wrap up. Please speak up and try to have just one person speak at a time to assure that everyone gets a turn, and to allow me to transcribe easier. • Finally, we’re here to share thoughts and experiences and gather the various perspectives on this topic so please say exactly what you think. Don’t worry about what I think or what other members of the group think. The goal is not necessarily to agree, but rather to capture everyone’s unique perspectives. • Since the group is familiar with one another I’m not going to take the time to introduce ourselves. • Any questions or comments before we begin? Our session ends at noon so let’s begin. Questioning Framework (45 min) 1. Before we started this work – what were your thoughts about technology safety? How has this changed, if at all? 2. With regard to technology safety, what do you know now that you didn’t know before – on a personal level? On an organizational level? 3. If at all, how has your awareness of technology safety impacted how you will do your work in the future? 4. Describe your experiences and thoughts about reviewing PSLS events, participating in safety huddles, and/or learning sessions? • What are some actual or potential safety concerns related to technology that you think are important to watch out for? 5. What do you think are the most important knowledge, skills, and attributes needed to promote safety in clinical informatics? • For example, what might be included in a course? Or onsite education? Additional question if time permits: What’s the most important thing about this work from your perspective? Wrap up (10 min) DRAFT 220 Thank you for sharing all your thoughts in this discussion. Does anyone have any final thoughts or additional comments they didn’t get a chance to share? Or something to add that our conversation didn’t cover? If something comes to mind later on, feel free to email me. I will follow up with each of you in the next few weeks to share a transcription of our conversation for you to review and you will have to opportunity to retract any of your statements. Any questions or comments? Thank you. Conversational Probes: • Is there anything else? • What I heard you say was… • Tell me more… • Would you say more… • I don’t quite understand… • Please describe what you mean… • Can you explain what you mean? • Would you explain further? • Does anyone have an example of that? • Would you give me an example of what you mean? • Is this anyone else’s experience? • Does anyone have a similar (or different) perspective? • I don’t understand. • You all seem to understand what he is saying, but I’m still confused. Can anyone help me? • Tell me what it is like. • You looked puzzled, why?
Thesis/Dissertation
eng
Nursing
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Currie, Leanne M.
Characterization and management of technology-mediated adverse events in primary and community care
Text