UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Nurses' use of iPads for clinical documentation : a usability study Ho, Wai Yu 2013

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2014_spring_ho_waiyu.pdf [ 1.09MB ]
Metadata
JSON: 24-1.0165730.json
JSON-LD: 24-1.0165730-ld.json
RDF/XML (Pretty): 24-1.0165730-rdf.xml
RDF/JSON: 24-1.0165730-rdf.json
Turtle: 24-1.0165730-turtle.txt
N-Triples: 24-1.0165730-rdf-ntriples.txt
Original Record: 24-1.0165730-source.json
Full Text
24-1.0165730-fulltext.txt
Citation
24-1.0165730.ris

Full Text

 NURSES? USE OF IPADS FOR CLINICAL DOCUMENTATION:  A USABILITY STUDY  by Wai Yu Ho  B.S.N., McGill University, 2007  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  MASTER OF SCIENCE in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Nursing)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)  December 2013  ? Wai Yu Ho, 2013 ii  Abstract Background/Purpose: Handheld devices are becoming increasingly prevalent in healthcare settings; however, few studies have examined the use of handheld devices for clinical documentation in the acute care setting. The purpose of this study was to investigate how nurses use the iPad for clinical documentation in the acute care setting. Methods: End user usability testing with think-aloud protocol was performed in a laboratory environment. Nurses who worked on an orthopedic surgery unit that use electronic clinical documentation were given a fictitious patient scenario with seven clinical documentation tasks that maximized the use of different functions of the iPad interface. A coding scheme was developed based on user and system behaviours defined by Dawson and Kushniruk. Results: Nine nurses participated in the usability testing. Nurses were in the nursing profession for an average of 13.2 years and in the study setting for an average of 11.5 years. All nurses reported that they had a desktop computer for personal use, and some nurses also owned an iPad (50%) and Smartphone (62.5%). Coded events were categorized into problems related to navigation, screen size, data entry, and login/logout. Navigational problems were mainly characterized by extra taps, mistaps, and the switch between tapping with finger and stylus. Nurses also had problems with the screen size when reading the font and icons, and selecting objects. Data entry problems were related to using a small external keyboard, and the mismatch with how data is saved between the iPad and the desktop computer terminals. Login/logout problems were significant barriers to clinical documentation as the process for both were time-consuming and required multiple steps. Conclusion: Improvements in clinical practice and the organization are warranted to enhance the use of iPads in the acute care setting, including education on the unit, and structural and technical supports from the organization. Organization supports include improving organizational iii  infrastructure, incorporating stakeholder engagement and testing into the planning and implementation processes, and evaluating the compatibility of the operating systems employed by the handheld device and the organization. Further research in this area is warranted to better inform practice and future decision-making. iv  Preface This study is part of a three phase study designed by Dr. Leanne Currie. I performed the usability study, collected and analyzed the research data, and wrote the entire thesis with guidance from my supervisor, Dr. Leanne Currie, and committee members, Dr. Victoria Bungay and Beatrice Da Silva. Ethics approval was received from the UBC Behavioural Research Ethics Board, Certificate Number H11-02026.  The recruitment for this study was carried out by Shelley Dalzell, who is the Patient Service Manager, and Anita Collier, who is the Clinical Educator, in the study setting.  v  Table of Contents Abstract .......................................................................................................................................... ii Preface ........................................................................................................................................... iv Table of Contents ...........................................................................................................................v List of Tables ................................................................................................................................ ix List of Figures .................................................................................................................................x List of Abbreviations ................................................................................................................... xi Acknowledgements ..................................................................................................................... xii Dedication ................................................................................................................................... xiii Chapter 1: Introduction ................................................................................................................1 1.1 Statement of Purpose ...................................................................................................... 4 1.2 Research Question .......................................................................................................... 4 1.3 Theoretical Framework ................................................................................................... 4 1.4 Summary ......................................................................................................................... 7 Chapter 2: Literature Review .......................................................................................................8 2.1 Search Strategies ............................................................................................................. 8 2.2 Handheld Devices in Nursing Clinical Practice .............................................................. 9 2.2.1 Nurses? Perceptions of Using Handheld Devices ..................................................... 15 2.3 Barriers and Challenges with Using Handheld Devices in Nursing Care .................... 17 2.3.1 Technology, Physicality and Usability Barriers to Handheld Device Use ............... 17 2.3.2 Individual Level Barriers to Handheld Device Use .................................................. 18 2.3.3 Organizational Level Barriers to Handheld Device Use ........................................... 20 2.4 Summary ....................................................................................................................... 22 vi  Chapter 3: Methods .....................................................................................................................23 3.1 Research Design............................................................................................................ 23 3.1.1 Usability Testing ....................................................................................................... 23 3.2 Sampling Design ........................................................................................................... 24 3.2.1 Sample Population .................................................................................................... 25 3.2.2 Inclusion and Exclusion Criteria ............................................................................... 25 3.2.3 Recruitment and Ethics ............................................................................................. 25 3.3 Data Collection ............................................................................................................. 26 3.3.1 Setting ....................................................................................................................... 26 3.3.2 Materials ................................................................................................................... 27 3.3.3 Education .................................................................................................................. 27 3.3.4 Demographics Survey ............................................................................................... 27 3.3.5 Usability Testing with Think-Aloud Method ........................................................... 28 3.3.5.1 Field Notes ........................................................................................................ 28 3.3.5.2 Morae Recorder ................................................................................................ 29 3.3.5.3 Testing Environment Set-up ............................................................................. 29 3.4 Data Analysis ................................................................................................................ 30 3.4.1 Demographics Surveys ............................................................................................. 30 3.4.2 Usability Testing Analysis ........................................................................................ 30 3.4.2.1 Coding Scheme ................................................................................................. 30 3.4.2.2 Morae Manager ................................................................................................. 32 3.4.2.3 Qualitative Data ................................................................................................ 33 3.5 Limitations of Usability Testing ................................................................................... 33 vii  Chapter 4: Results........................................................................................................................35 4.1 Results from Demographics Survey ............................................................................. 35 4.2 Results from Usability Testing ..................................................................................... 36 4.2.1 Time on Tasks ........................................................................................................... 37 4.2.2 Navigational Problems .............................................................................................. 38 4.2.2.1 Extra Taps ......................................................................................................... 39 4.2.2.2 Mistaps .............................................................................................................. 40 4.2.2.3 Stylus Versus Finger ......................................................................................... 40 4.2.2.4 Scrolling Problems ............................................................................................ 41 4.2.2.5 Screen Dragging................................................................................................ 42 4.2.3 Screen Size ................................................................................................................ 43 4.2.4 Data Entry ................................................................................................................. 45 4.2.5 Login/Logout Problems ............................................................................................ 48 4.3 Summary ....................................................................................................................... 52 Chapter 5: Discussion ..................................................................................................................53 5.1 Navigational Problems .................................................................................................. 54 5.2 Screen Size .................................................................................................................... 57 5.3 Data Entry ..................................................................................................................... 58 5.4 Login and Logout Problems.......................................................................................... 60 5.5 Recommendations for Clinical Practice........................................................................ 61 5.6 Recommendations for Organizations ............................................................................ 62 5.7 Study Limitations .......................................................................................................... 66 5.8 Recommendations for Future Research ........................................................................ 67 viii  5.9 Conclusion .................................................................................................................... 67 References .....................................................................................................................................69 Appendices ....................................................................................................................................77 Appendix A ............................................................................................................................... 77 Appendix B ............................................................................................................................... 79 ix  List of Tables Table 3.1 Coding Scheme ............................................................................................................. 31 Table 4.1 Description of Nurses? Characteristics ......................................................................... 36 Table 4.2 Difference Years in Nursing by Smartphone Use ........................................................ 36 Table 4.3 Time on Task ................................................................................................................ 37 Table 4.4 Frequencies of Marker Types by Tasks for Navigational Problems ............................ 38 Table 4.5 Frequencies of Marker Types by Tasks for Enlarging and Minimizing ....................... 43 Table 4.6 Frequencies of Marker Types by Tasks for Data Entry ................................................ 45  x  List of Figures Figure 1.1 Input-Transformation-Output Model for Health Provider Performance (Adapted from Karsh et al., 2006) ........................................................................................................................... 6 Figure 3.1 Set-up of Usability Testing Environment ................................................................... 29 Figure 4.1 Scrollbar in Respiratory Screen .................................................................................. 41 Figure 4.2 Arrow Key in Comment Box ...................................................................................... 42 Figure 4.3 Data Fields on Vital Signs Documentation Screen ..................................................... 44 Figure 4.4 F12 Function ............................................................................................................... 46 Figure 4.5 Five Screens for Login ................................................................................................ 48  xi  List of Abbreviations CDSS: Clinical decision support system CPOE: Computerized physician order entry DVT/VTE: Deep vein thrombosis/venous thromboembolism  IVRN: Intravenous registered nurse PCIS: Patient care information system PDA: Personal digital device WIMP: Window, icon, menu, and pointing device  xii  Acknowledgements I would like to thank the following people who have made this thesis possible.  My deepest and sincere gratitude goes to my mother for her abiding love and support, and for easing off my load in the months of writing when I needed it most. I could not have accomplished this without her unconditional love and patience.    I would like to extend my most sincere appreciation and thanks to Dr. Leanne Currie, my supervisor, for her unceasing encouragement, patience, and guidance during the program. Her enlightening wisdom and expertise have been invaluable not only in my thesis, but also in my personal growth by inspiring me to question more deeply, to broaden my perspectives, and to be a positive influence in our profession.  I would also like to thank Dr. Victoria Bungay and Beatrice Da Silva, my thesis committee members, for the challenging questions and stimulating discussions during our meetings. Their expertise, encouragement, and guidance have undoubtedly strengthened my learning.  I offer my enduring gratitude to my ?sisters,? Sheryl, Jeanne, and Christine, for keeping me sane in the past couple of years. I am thankful for the prayers, love, and support that they have provided. Lastly, special thanks to Karen, Keilee, Mila, and Andrea for the camaraderie during the program and the beautiful friendships that have blossomed from this whole experience.  xiii  Dedication To the One whose love and provision will always sustain and strengthen me, I dedicate this thesis to my God because with Him all things are possible. 1  Chapter 1: Introduction Nurses are increasingly using mobile technology in healthcare settings. In the past two decades, improvements in wireless connections, computer processing speed, and internal memory have facilitated mobile technology adoption in everyday life and also in healthcare settings. There are several names used for handheld devices in healthcare including ?mobile devices?, ?personal digital assistants (PDAs)?, ?tablet computers? and ?handheld devices?. For the purposes of this thesis, the term ?handheld device? will be used unless otherwise described.  Portability is one of the major catalysts for adopting handheld devices in health care settings. The small size and light weight of handheld devices allow ease of mobility and point of care usage (Carlson et al., 2010; Garrett & Klein, 2008; Johansson, Petersson, & Nilsson, 2011). Point-of-care refers to the delivery of health care products or services from clinicians to clients at the time of care; for example, point-of-care can be at the bedside where a nurse and patient interact.  Nurses can also use handheld devices that are connected via wireless networks for information access and retrieval, clinical decision support, online resources, and drug and medical references at the point of care (Bates & Gawande, 2003). Many clinicians are optimistic that handheld devices with wireless access can have positive clinical impacts by increasing efficiency and patient safety, improving evidence-based practice, and providing timely access to information and clinical decision support (Bates & Gawande, 2003; Di Pietro, Nguyen, & Doran, 2012; Divall, Camosso-Stefinovic, & Baker, 2013; Doran, 2009; Honeybourne, Sutton, & Ward, 2006; Stroud, Smith, & Erkel, 2009).  Health information technology is defined as ?the application of information processing involving both computer hardware and software that deals with the storage, retrieval, sharing, and use of health care information, data, and knowledge for communication and decision 2  making? (Office of the National Coordinator for Health Information Technology, n.d.). Bates and Gawande (2003) assert that information technology can reduce the rate of errors and adverse events. Some strategies for error prevention include improving communication and access to information, assisting decision support, and responding quickly to and tracking of adverse events. For example, handheld devices can improve communication through electronic-prescribing, and access to information by retrieving up-to-date clinical documentation (Garrett & Klein, 2008; Stroud et al., 2009). Depending on its functions, a handheld device may enhance efficiency in clinical practice. With increasing workload and higher patient acuity, nurses need to prioritize their tasks and plan their day efficiently in order to maximize their time. Handheld devices, such as PDAs or tablets, have the potential to improve efficiency and save clinicians? time by providing quick access to evidence-based resources or drug references, laboratory results, and clinical documentation (Di Pietro et al., 2012). Proponents who favour the adoption of handheld devices assert that time saved can be allocated back into patient care and other clinical work (Johansson et al., 2011).  Handheld devices may also have cost-saving effects by enabling prompt clinical documentation. Though not empirically studied, timely clinical documentation can possibly reduce cost by eliminating time wasted on finding missing information or re-collecting information (Carlson et al., 2010).  Despite the enthusiasm, ubiquitous adoption of handheld devices in health care is complicated and can face many barriers. Though the prevalence of using handheld devices for clinical documentation is growing, the evidence to support adoption remains inconclusive (Bosma, Balen, Davidson, & Jewesson, 2003; Carlson et al., 2010; Lee, 2006; Wager et al., 2010). Although there may be potential benefits of using handheld devices, there may be 3  unintended consequences of increasing adverse events if the product was not designed and fitted to the user (Kushniruk, Triola, Borycki, Stein, & Kannry, 2005). For example, use of a handheld device itself may be part of a ?technology-induced? medical error. This threat to patient safety may arise when the technology is poorly designed, does not fit the clinician?s work or is unreliable. The consequences of poorly designed technology can jeopardize patient safety by causing workflow disruptions, suboptimal care, adverse events, and organizational inefficiencies (Patel & Kaufman, 2006). Therefore, handheld devices need to be well-suited to the clinician?s work in order to enact the promise of improving patient safety.   The discipline of human factors engineering examines the overall system, which is composed of individuals, groups, organizations and environmental factors that affect human performance. Human factors engineering is concerned with ?design and engineering of human?machine systems for the purpose of enhancing human performance? (Dempsey, Wogalter, & Hancock, 2000, p. 6). Rather than solely focusing on the clinicians, human factors engineering goes beyond the individual and examines the system as a whole to improve the process of care (Karsh, Holden, Alper, & Or, 2006). In other words, in order to provide safe care, we need to enhance human performance and eliminate hazards by focusing on redesigning the system, rather than focusing only on individual factors. One way of doing this is by involving end users (e.g., clinicians) in the design of health information technology through usability evaluations.  Usability testing is a cost-effective way of evaluating a hardware or software product by involving end users. Usability is the ?extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use? (International Organization for Standardization, 1998). In other words, the product is designed for a group of target users, who should find the product useful, easy and enjoyable to 4  use. Some studies have examined the use of PDAs and/or tablets in clinical settings. However, the study interventions with using handheld devices were dissimilar among the studies. For example, studies have compared PDAs and/or tablet device use for clinical documentation, patient assessment modules, information retrieval and other tasks. In addition, usability studies mostly assessed the design of software application or clinical resource, while not typically focused on the usability of the device itself. Research describing the effects of using handheld devices, such as the iPad, for clinical documentation is limited and inconclusive. 1.1 Statement of Purpose To date, there is limited research about usability of handheld devices in healthcare and no reports on iPad usability; therefore, the purpose of the study was to examine how nurses used an iPad for clinical documentation. 1.2 Research Question The main question for this study was:  ? How do nurses use the iPad for clinical documentation in the acute care setting? 1.3 Theoretical Framework For the purpose of this study, a theoretical framework grounded in human factors engineering was used to underpin and guide the study concepts. The human factors engineering paradigm described by Karsh and colleagues (2006) is a useful framework for this study because, unlike other models that focus solely on error prevention from an individual perspective, this model aims to take a comprehensive approach in examining multiple system factors that influence performance. Although patient safety and quality of care are important markers for organizations, solely looking at these outcomes is insufficient to inform change. In other words, 5  focusing on errors requires that an error happens first before any patient safety initiatives can begin (Karsh et al., 2006).  Karsh et al. (2006) propose that different levels of work systems influence health care providers? performance. Moving from micro to macro systems, the different levels include ?patient and provider factors,? ?work system/unit factors,? ?organization factor,? and ?external environment factors? (Karsh et al., 2006). Each level is depicted as a piece that fits into a larger system, which illustrates how different levels of the system interact with each other. These multiple layers of work systems collectively feed into the transformation processes, which then produce performance outputs. Figure 1.1 shows Karsh et al.?s model for health provider performance.  Transformation processes are the acts of converting inputs into outputs. Such acts can be translated into physical performances, cognitive or mental processes, or social and behavioural activities (Karsh et al., 2006). The output of the transformation processes include things like changing a patient?s location, increasing or decreasing stress or fatigue, or improving team dynamics. The final goal of these cumulative outputs are patient safety, employee safety and quality of care (Karsh et al., 2006). In their model, performance outputs subsequently loop back to performance inputs. If performance outputs fail to meet standards and objectives for safety, efficiency, and/or quality, the system will result in further changes or redesign (Karsh et al., 2006).  6  Figure 1.1 Input-Transformation-Output Model for Health Provider Performance (Adapted from Karsh et al., 2006)  For the purpose of this study, the first three levels of performance input, that is, ?patient and provider factors,? ?work system/unit factors,? and ?organization factors? were the main focus for interpretation. By conducting a usability evaluation of the iPads for clinical documentation, the results may help reveal individual-, unit-, and organizational-level factors that influence documentation behaviours in the context of handheld devices. The immediate outcome of interest was ease of performing clinical documentation on the iPad. This outcome aligns with the overarching goals of the human factors engineering paradigm, which are patient safety, employee safety and quality of care. The results of this study were aimed as a feedback 7  mechanism to determine whether the iPad would be a useful device for nursing documentation and to guide future decisions and actions in the larger system.  1.4 Summary Handheld devices are increasingly being used in healthcare; however, limited research has been conducted to evaluate the nurse?s actual use of the device. This study explored nurses? use of an iPad handheld device in the context of clinical documentation.   8  Chapter 2: Literature Review Handheld computing devices are becoming increasingly prevalent in health care settings. In the 1990s, John Scully, the former CEO of Apple Computers, coined the term ?personal digital assistant? (PDA) and predicted that PDAs will be used to hold telephone numbers, store notes, and exchange messages wirelessly (The Seattle Times Company, 1992). Not only have those predictions been fulfilled, PDAs have now evolved into more versatile and powerful forms of handheld computing primarily due to the increase in internal memory and processing speed in smaller processing chips. In addition, handheld devices, such as smart-phones and tablets, have more functions and capabilities than their predecessors. In particular, tablets are conveniently portable and have wireless capabilities that allow sending and retrieving data anywhere with a network connection. Tablets also have wider, multi-touch screens that enable e-reading, and can readily support viewing and browsing information with comfort (Duffy, 2012).  Despite the versatility and portability of handheld devices in everyday computing, little is known about their fit in supporting nurses? work in healthcare. This chapter reviews the current literature on handheld devices in acute care settings. 2.1 Search Strategies  A search on CINAHL, Academic Search Complete, Medline, PsycInfo, and Compendex Engineering Village databases was done using the following search terms: ?personal digital assistant?, ?PDA?, ?handheld computers (computers, handheld)?, ?mobile (computing) device?, ?pocket computer?, ?nursing?, ?nurse-patient relations?, ?usability?, and ?usability testing?. Additional articles were found by combing through the reference lists of identified articles.    9  2.2 Handheld Devices in Nursing Clinical Practice  The use of handheld devices is becoming more common in health care. The decreased size and portability of handheld devices open more avenues for use in the hospital setting. With advances in processing speed, internal memory, and wireless connection, handheld devices can be more versatile than traditional desktops and laptop computers. Handheld devices come in a variety of models, operating systems, and designs. Most devices described in the studies included in this review were dissimilar likely due to the rapid technological advances in the past decade. Most studies examined the use of personal digital assistants (PDAs), while only a few looked at tablet use. Depending on the type of work it needs to support, a device may be chosen for the functionalities and features that it supports. A few studies examined the use of handheld devices, such as PDAs or tablets, as documentation tools. None of the studies examined iPad use in the context of nurses? work. Two studies examined the use of tablets to document vital signs in the acute care setting (Carlson et al., 2010; Wager et al., 2010). In a study comparing stationary computers, workstation-on-wheels (WOW), and tablets (n=3,212 total number of entries), nurses only opted to use tablets 2% of the time (n=59), whereas stationary computers were used 45% (n=1,458) and WOWs were used 53% of the time (n=1,689) (Carlson et al., 2010). The chief complaint for the lack of use of tablets was due to issues with poor connectivity, heaviness of the device, difficulty using the stylus and handwritten letters, and problems with system lagging and freezing (Carlson et al. 2010). However, in another study, the use of a tablet PC improved the timeliness and accuracy of point-of-care vital signs documentation (Wager et al, 2010). The authors examined vital signs documentation on paper charts, workstation-on-wheels, and tablets. In their study, a tablet was attached to a Dinamap (portable electronic blood pressure machine) pole and the average time to 10  document vital signs was 0:35 seconds, as compared to 1:25 and 9:15 minutes for paper and WOWs, respectively. The percentage of documentation error was also reduced from 16.8% and 15.2% on paper on WOWs, respectively, to 5.6% on tablet PC (Wager et al., 2010).  Dykes and colleague (2007) conducted a randomized cross-over study with registered nurses comparing the use of paper, PDA, and tablet for completing patient assessment modules to support problem identification, care planning, and tracking nurse sensitive outcomes. They randomized nurses into two groups: one group using paper (n=39), and one group using a device (n=40). Nurses from the device group were further randomized to use either a PDA (n=20) or a tablet (n=20). After 6 weeks, the groups were switched. At both phases of the study, nurses who were randomized to the device were permitted to use paper if they preferred. During the study, 1270 patients were admitted. Nurses were expected to use paper documentation for 50% of the patients, however they used paper to document on 68.3% (n=867) of these patients. Of the 635 patients who were expected to have documentation via a device, 36% had documentation by paper and 64% had documentation by a device. Of this 407 patients (64%), 226 had documentation by tablets (55.6%), and 181 had documentation via PDA (44%). Therefore, the actual use of PDAs and tablets were less than expected. The assessment modules were also analyzed for completeness of patient assessment documentations. They found that nurses who used PDAs or tablets were more likely to complete documentation for fall prevention, pressure ulcer prevention, pain management, aspiration prevention, malnutrition prevention, and DVT/VTE prevention (p<.001), and suicide prevention assessment (p<.05) (Dykes et al., 2007). A seven-point Likert scale was employed to assess nurses? satisfaction from a web survey. Using the Kruskal-Wallis test, the authors found that nurses who used electronic devices were generally more satisfied than nurses (n=79) who used the paper method (mean rank- PDA: 33.19, tablet: 11  33.08, paper: 32.74; p<.008). However, nurses found the login process for the devices was time-consuming and nurses were dissatisfied with the login process. While nurses reported being more satisfied with using electronic devices overall, the frequency in which paper modules were used more often than electronic modules suggests that more work is required to improve integration of the devices into nurses? workflow (Dykes et al., 2007).   Dawson and Kushniruk (2009) studied the usability of a tablet interface for a computerized provider order entry (CPOE) system. They identified and measured user and system behaviours that would affect the efficiency and accuracy of navigation and data entry in the CPOE system. In their usability evaluation, they coded a total of 1429 problematic user and system behaviours that included: extra taps, mistaps, wrong data in field, misrecognition, and preferred action not clear, to name a few. These errors were further categorized into ?slips,? ?mistakes detected by the application,? and ?mistakes undetected?. They found that 89.9% (125/139) of all errors were detected by the user and the application. They also found that 75% (1069/1429) of all problems were related to user navigation and data entry behaviours. While these behaviours were not associated with errors in their study, the behaviours impacted user?s efficiency in completing their tasks, which resulted in wasted time and effort (Dawson & Kushniruk, 2009). Di Pietro et al. (2008) conducted a usability study with acute care and home care nurses to identify the requirements needed for a clinical decision support tool at the point of care. Eight nurses from acute care and eight nurses from home care settings participated in the pilot test. The researchers conducted field testing that required nurses to complete three to five patient assessments; nurses in the acute care setting completed a total of twenty-three assessments, and nurses in home care completed a total of twenty-eight assessments. Nurses were asked to 12  complete a patient demographics form and a wireless connectivity issues form. Face-to-face and phone interviews were also conducted to assess the nurses? experiences with the devices. In terms of the physicality of the device, nurses enjoyed the size of the PDA because it was portable and fit in their hand easily. However, nurses requested increased font size, increased screen display brightness, and settings adjustments to prevent the display from shutting off during nurses? consultations. Wireless connectivity was an issue during field testing as one nurse reported experiencing weak wireless signal in a patient room that required her to relocate in order to capture a stronger signal. Similar to Carlson et al. (2010) and Dykes et al. (2007), nurses in this study also reported a time-consuming login process from turning on the device to beginning the assessment tool.  In another study, Di Pietro et al. (2012) examined eleven nurses using either a PDA or tablet to interact with clinical decision support systems (CDSS) to identify usability problems that might prevent their use at the point-of-care. In this laboratory study, eleven nurses were required to complete three tasks by navigating to three different practice guideline resources. During the study, there were thirty-nine occurrences of navigational problems within the decision support tool, which included accessing the wrong page, unsure how to proceed, and unsure what to do within a particular resource. There were a total of twenty-five occurrences with nurses experiencing the following problems: scrolling, highlighting text, tapping on text or links, and typing errors (Di Pietro et al., 2012). Nurses also experienced delayed responses caused by problems with the wireless network, a problem with the device, or both. The authors recommended that the organization needs to provide continuous information technology education and support for staff to better equip them with skills to access evidence efficiently and effectively (Di Pietro et al., 2012). They further asserted that familiarity and practice might 13  enhance nurses? confidence in using technology; tools that are considered too complex will not be used by nurses and will become wasted resources.  Doran et al. (2010) conducted a longitudinal study that examined the use of handheld devices, including PDAs or tablets, to improve information access, and explored the outcomes related to handheld devices use. The study was conducted across twenty-nine acute care (n=174 nurses), long-term care (n=121), home care (n=175), and primary care settings (n=18), with a total sample of 488 nurses. Responses on the frequency of PDA and tablet use were obtained through a questionnaire (n=182). Over 44.5% of the nurses used a PDA or tablet at least once every few days, with primary care (75%) and long-term care (54%) nurses using the devices most often; however, there were only 4 (22%) and 41(34%) respondents in the primary care and long-term care group, respectively. The authors found that PDAs and tablets augmented nurses? research awareness and values, and communication of research over the 12-month course of the study. About half of the respondents (55%) reported ?never? or ?almost never? using a PDA or tablet during the study. The reasons for non-use include technical problems; no or poor network, no or low battery, too slow, inconvenient or no need; preferred desktop, need keyboard, device too heavy, time restrictions; insufficient time, heavy workload, education needs, computer shy, and device unavailable.  Honeybourne et al. (2006) conducted a descriptive study by administering pre- and post-intervention questionnaires in two phases. The study assessed how often clinicians used clinical resources on the PDA to inform clinical decision-making, and what type of resources clinicians found most useful. The intervention consisted of supplying fourteen participants with a PDA and loading medical resources and applications for clinical use. At the end of phase one, four participants reported that the PDA helped answer clinical questions immediately at the point of 14  care; at the end of phase two, there was an increase in the number of participants (n=11) who found PDAs helpful in answering questions when questions arose. In phase two, clinicians who used PDAs for looking up drug information and viewing practice guidelines increased from baseline (n=5 and n=5, respectively) to end-of-phase two (n=7 and n=6, respectively). However, there seem to be no other significant increase in nurses performing other tasks while using PDAs in either phase one or phase two (e.g., reading health journals or health textbooks, using medical calculators, writing prescriptions). The results in the study were difficult to compare as the number of participants in the two groups for phase one and phase two were different (phase one baseline n=14, and end of phase one n=9; phase two baseline n=14, end of phase two n=12). The sample was a mixed group of nurses, consultants, doctors, a pharmacist, and clinical librarians which might have impacted the frequency and reason for using the PDA for each task. For example, clinical librarians would likely not perform tasks such as using medical calculators, writing prescriptions, and accessing patient data. Intra- and intergroup comparisons were therefore difficult due to attrition, small sample sizes, and mixed occupations.  Lastly, Bosma and colleagues (2003) assessed the use of PDAs by intravenous resource nurses (IVRN) to manage and collect data for patient consultations requiring intravenous therapy at the point of care. Nurses collected data on their consultations and transferred the data onto a database. The researchers examined data from the consultation service and did not compare the use of the PDA with any other group. There was no data on how often the device was used; however, there was a six-week period when manual and PDA-based collection of intravenous consultation data were employed concurrently and IVRNs used this time to become familiarized with the PDA device. At the end of the six weeks, the manual form of data collection was abandoned and IVRNs used PDAs exclusively. During the study, the authors reported equipment 15  failure on a number of occasions. For example, the software on the PDA frequently froze, which required multiple soft and hard resets. They also commented about the PDAs? slow responsiveness when large amount of data were entered, suggesting inadequate and insufficient memory and processing speed when carrying out extensive clinical documentation. Nurses were also losing the PDA stylus frequently and requiring a supply of replacements, in addition there was one instance of dropping the PDA on the floor, which damaged the device. The authors suggested that maintenance of these devices required diligence on the part of the nurse. They also had to monitor data overload on the PDA and required timely transfer of information onto the database to preserve optimal performance.   While handheld technology is becoming more prevalent in healthcare settings, the current research provides a mixed review of their effectiveness and efficiency. If the device is unreliable, it may be ill-suited for nurses? work regardless of how portable the device may be (Carlson et al., 2010) and nurses will opt for more dependable avenues to carry out their work. 2.2.1 Nurses? Perceptions of Using Handheld Devices In a study exploring advanced practice nurses? perception of wireless PDA technologies through questionnaires and focus groups, nurses reported some desirable functions and features that could enhance their work, such as freedom and mobility, supporting advanced practice, clinical data access, clinical decision support, and drug references and electronic-prescribing (Garrett & Klein, 2008). For example, the advantages of electronic-prescribing can potentially save time and reduce error by eliminating the need to interpret ambiguous and illegible hand-writing. Immediate access to clinical documentation, and reference materials while at off-site locations allowed nurses to respond to changes in patient?s condition and support clinical practice (Garrett & Klein, 2008). Nurses also perceived that PDAs can improve access to 16  information through clinical decision support tools that can augment specialist knowledge and protocols. The use of clinical decision support systems (CDSS) can help present intelligently-filtered knowledge and patient-specific information to health care providers at the right place at the right time (Berner, 2009). Similarly, nurses reported that handheld devices can support evidence-based practice, and allow immediate information access at the point of care (Garrett & Klein, 2008). Nurses also found electronic versions of medical and drug references were easier to access and use for search queries than paper-based references (Garrett & Klein, 2008). In a survey that examined the prevalence and patterns of PDA use among nurse practitioners, Stroud and colleagues (2009) found that nurse practitioners perceived PDAs could improve clinical decision-making, patient safety, and productivity in practice. Drug references were the most frequently installed application and were ranked as being the most useful, followed by medical text and reference books. Ninety-eight percent of all respondents (121/124) contended that PDAs would be a helpful tool in clinical practice (Stroud et al., 2009). Eighty-nine percent of PDA users believed that PDA use would also enhance safety of their care by preventing errors. Finally, many respondents believed the use of PDAs increased their productivity by enabling quick searches, sharing information with patients, educating patients during clinical encounter, and supporting decision-making (Stroud et al., 2009). Other opinion articles also suggested that handheld devices have potential benefits in health care. Handheld devices have the potential to bring information and resources to the point-of-care, and support clinical decision-making, facilitate communication, collaborative practices, and education in the work place (Doran, 2009). Similarly, Thompson (2005) proposed that handheld devices could make nursing practice more efficient, safer, and better quality; by 17  providing ease of access to information, this can help reduce the chances of error, provide broader knowledge base, and enable information for teaching patient (Thompson, 2005).  2.3 Barriers and Challenges with Using Handheld Devices in Nursing Care The process of incorporating handheld devices into clinical settings can be arduous, and is often met with different levels of barriers. Barriers can happen on a technology/device level, an individual level, and/or an organizational level. Problems such as usability violations, physicality of the device (e.g., weight), and organizational infrastructures (e.g., wireless connection) need to be carefully considered before implementing handheld devices (Carlson et al., 2010; Doran et al., 2010; Honeybourne et al., 2006; Lu, Xiao, Sears, & Jacko, 2005).  2.3.1 Technology, Physicality and Usability Barriers to Handheld Device Use  Barriers to device use can occur with problems such as technology issues, physicality of the device, and usability violations. Technology of a device refers to the operating system, processing speed, and memory of the device. For example, in one study, when a PDA was reaching the full internal memory capacity, the performance of the device deteriorated until some data was removed (Bosma et al., 2003). Devices nowadays typically have more memory and faster processing speed than previously; therefore, problems with internal memory are less common. However, file sizes are now generally larger, thus device memory capacity may still be a concern.  The physicality of the device refers to the physical attributes of the handheld computer, for example, the weight and size of the device. Carlson et al. (2010) found mixed reviews about the weight of the tablets. Some nurses found tablets too heavy to carry with them all day, while others lauded their portability and light weight. Doran et al. (2010) also had similar findings.  18  The usability of the device refers to the interaction between the user and the user interface of the device. Some barriers that hinder device use can be the screen size being too small (Garrett & Klein, 2008). The screen size of a PDA can impact the readability of larger volumes of data and create discomfort with using the device for long periods of time. Particularly, reading PDFs or long documents such as health journals can be very difficult. For that reason, some nurses in another study requested the font size on the PDA to be adjusted in order to enhance readability (Di Pietro et al., 2008).  2.3.2 Individual Level Barriers to Handheld Device Use Barriers at the individual level can include cognitive abilities, effects of age, personal perception and attitude about technology, and computer literacy (Arning & Ziefle, 2009). In a book by Barnum (2011), the author noted that generational differences among users are important factors to consider in usability evaluations. For example, as part of the normal aging process, older users have difficulty reading small font and closely spaced text, as well as remembering previous moves and content locations (Arning & Ziefle, 2009; Barnum, 2011; Garrett & Klein, 2008). In their study, Arning and Ziefle (2009) examined the interrelationship of user characteristics, and effectiveness and efficiency of task performance between younger and older subjects who were engaged in using a device for PDA menu navigation performance. They found that older subjects had significantly lower spatial abilities and verbal memory than younger participants, which weakened older users? performance and efficiency in completing proposed tasks. In addition, younger subjects required significantly less time, less keystrokes, and navigational steps to complete all required tasks than older subjects.  Individual perception and attitude about technology can be a factor in the success of adopting health information technology in the health care field. Individual perceptions can 19  outweigh the actual usefulness and ease of use of a technology (Nielsen Norman Group, 2012). Resistance or repellence against a product can be dependent on the users? first encounter with the product, and whether or not the product meets users? expectations (Barnum, 2011). Another individual characteristic that relates to ease of use is computer self-efficacy. Computer self-efficacy is the individual?s own confidence level in using technical devices (Arning & Ziefle, 2009). Lee et al. (2011) found that computer self-efficacy was related to ease of use; a user?s perception of their own computer self-efficacy can influence their performance and attitude towards technology, and lack of comfort with computer technology can hinder nurses from seeking information and charting electronically (Doran, 2009). An individual?s level of computer literacy and informatics literacy can highly influence the adoption of handheld devices as well. Rosenfeld et al. (2002) suggest that computer literacy differs from informatics literacy in that the former refers to skills in hardware and software applications, equipment, and PDA use, while the latter refers to the ability to obtain, use and critically analyze information retrieved by technological tools. Informatics literacy sometimes incorporates computer literacy because users require a certain level of knowledge to operate and obtain information from technology. Despite growing technology use in health care, informatics literacy and education have not been equal across nursing staff. For most nurses in the workforce, computer and informatics competence were not requisites at the time of hire. In a survey that assessed the level of integration of computer literacy competencies in baccalaureate nursing education programs, McNeil and colleague (2006) identified gaps in computer and informatics knowledge in nursing education. Respondents, who were deans, directors or faculty members, added that there were insufficient time and personnel to teach informatics in the programs, suggesting that even younger generation of nurses may be ill-prepared for the 20  technology demands in the workplace. In a self-assessment survey on proficiency with work-related computer skills administered to nurses and nurse support staff, more than 28% scored themselves as having poor to fair on all computer-related skills (Wilbright et al., 2006). In addition, those who were over age 50 and had graduated before 1984 were more likely to report lower proficiency. 2.3.3 Organizational Level Barriers to Handheld Device Use Organizational factors can also influence use of handheld devices in clinical settings. The high cost of PDAs/tablets and accompanying software, and fast turnover of technology life cycle are not favourable investments for healthcare organizations (Garrett & Klein, 2008; Stroud et al., 2009). On the same note, many healthcare organizations may not have the financial means of providing handheld devices to every unit or every nurse. Organizations must plan to have backfill staff and overstaff during planning and implementation phases in order to optimize integration of new innovations (Coplan & Masuda, 2011). Thus, the complexity, the cost and the human resources plan for implementing any innovative technology needs to be carefully considered. The initial phase of integrating an innovative technology will slow down existing processes, which can impact bed flow, and efficiency of nurses? work while they learn to incorporate new technology into their routine (Coplan & Masuda, 2011).  As mentioned above, poor computer self-efficacy will impact the user?s perception and actual use of the device. There must be adequate education on how to use the technology (Garrett & Klein, 2008). Practice and a time for ?play? can also be beneficial for users (Di Pietro et al., 2012). Organizations need to consider training modules that cater to different levels of informatics literacy and computer skills. In addition, supplemental education post-21  implementation can also educate advanced users and enhance users? experiences (Agno & Guo, 2013).  Organizational infrastructure needs to be in place to support mobile device use. As described above, wireless connection to the network was an issue in two of the studies; nurses indicated that due to poor network connection, the tablets did not work in all places (Carlson et al., 2010; Di Pietro et al., 2008). In addition, three studies mentioned problems regarding the login process (Carlson et al., 2010; Di Pietro et al., 2008; Dykes et al., 2007). Nurses reported having difficulty logging in unless the device was in the docking station (Carlson et al., 2010). Dykes and colleagues (2007) noted a two part login process whereby nurses must first log onto the device, and then onto the organization?s network. Nurses found this time-consuming and were dissatisfied with the lengthy process; however, the article did not specify the time required to log in. Similarly, Di Pietro and colleagues (2008) found that the time required to turn on the device, to login, and to begin the assessment tool was also too long.  Lastly, nosocomial, or hospital-acquired, infections are potential organizational barrier. Nosocomial infections are associated with higher mortality, morbidity, and longer length of stay in the hospital. Health care provider and hospital equipment may be vectors for transmission of infections. Some studies have shown that technology such as personal digital assistants, mobile devices, pagers, and keyboards can be reservoirs for noxious microorganisms (Braddy & Blair, 2005; Byrne, 2013; Hassoun, Vellozzi, & Smith, 2004; Singh, Kaur, Gardner, & Treen, 2002). Some of those studies also found antibiotic resistant microbial, such as methicillin-resistant Staphylococcus aureus and vancomycin-resistant Enterococcus, on the devices that health care providers carried (Braddy & Blair, 2005; Hassoun et al., 2004). Furthermore, reports suggest that health care providers have either never or only rarely cleaned their devices before (Hassoun et 22  al., 2004); however, it is unclear if disinfectants, such as wipes and alcohol, will cause damage to the device. The lack of clear practices and guidelines for effective cleaning can cause negative unintended consequences, such as spreading microorganisms to vulnerable patients, and other health care workers. 2.4 Summary Handheld devices are increasingly being integrated into different aspects of nurses? work. The overall goal of handheld device use is to improve patient safety and quality of care. There are potential benefits and barriers that influence their adoption; handheld devices have the potential to improve information access, retrieval, and use, support timeliness and completeness of clinical documentation, and accompany nurses to the point-of-care but barriers are present in different levels of the system. Strategies for implementation must consider the work system as a whole in order for successful adoption of handheld devices.  23  Chapter 3: Methods This chapter provides a brief overview of usability testing, followed by a detailed description of the research design, sampling population, sample selection criteria, and recruitment methods. Data collection methods and analytic plan, ethical consideration, and limitations are also discussed in this chapter.  3.1 Research Design This study employed usability testing methods to examine how nurses used the features and functionality of the iPad while documenting in a patient care information system (PCIS). A think-aloud protocol was used to capture the users? experience with the iPad. The think-aloud method allows recording of the users? cognitive processes, including their decision making, problem-solving, and reasoning processes (Polit & Beck, 2012). Lastly, a short demographics survey was completed with each participant. This research was part of a larger study that examined how nurses used iPads for clinical documentation in the acute care setting. Data from key informant and focus group interviews, and satisfaction survey were collected as part of the larger study to better understand nurses? experiences with using the iPad in their work. Data from the larger study are not reported in this thesis. 3.1.1 Usability Testing Usability refers to the manner in which end users utilize a product to achieve a specified goal with effectiveness, efficiency and satisfaction (International Organization for Standardization, 1998). Effectiveness and efficiency refer to how accurately and quickly a user can achieve his/her goal when using the product (Barnum, 2011). Satisfaction solely derives from the user?s perception of how pleasant is the design, which can sometimes outweigh 24  effectiveness and efficiency despite the actual quality of the product (Nielsen Norman Group, 2012).  Usability testing can help researchers obtain quantifiable results to assess the usefulness, ease of use, and satisfaction of a product. Usability testing is a process that uses subjects, who represent the end-users of the product, to evaluate whether or not a product meets specific usability criteria (Rubin & Chisnell, 2008). The results of a usability study, then, can help developers or the organization perform product or system redesign that will improve the user?s experience with the product, and consequently, enhance the user?s ability to accomplish their specified goal(s). Typically, a usability test of 10 participants can reveal up to 95% of errors (Faulkner, 2003). This scientific method is a cost-effective way to gather information on design problems or usability violations, that can help redesign and optimize its successful integration and utilization in the work setting (Kushniruk & Borycki, 2011).  3.2 Sampling Design The study took place in a forty-six bed acute surgical unit that incorporated seven different surgical specialties at a teaching hospital in British Columbia. Patients from this unit have undergone elective surgeries with an estimated length of stay of less than five days. The nursing unit was pre-determined because the unit had a unique characteristic of having a hybrid system for clinical documentation since 2008. A hybrid system is one in which both paper and electronic clinical documentation takes place. In this unit, the electronic system is referred to as the patient care information system (PCIS). The unit was also partially funded for the iPad study by the local health authority because no other units were using PCIS for electronic clinical documentation at the current time and, therefore, was considered best suited for this study. As 25  such, purposive sampling was used to select participants that would best resemble a broader group of interest (Fossey, Harvey, McDermott, & Davidson, 2002; Polit & Beck, 2012). 3.2.1 Sample Population The sample was taken from an acute surgical nursing unit in a tertiary care facility in British Columbia. Ten registered nurses were recruited for this study. 3.2.2 Inclusion and Exclusion Criteria To be included in the study, participants must have: 1) had a valid license to practice as a registered nurse in British Columbia; 2) been willing to be interviewed; 3) been articulate in expressing their experiences; 4) understood the purpose of the study and been able to commit time to participate; and 5) provided written consent.  Nurses were excluded from the study if they 1) were unable to commit time for the usability testing; 2) chose not to participate in the study; and 3) chose to withdraw at any point of the study.  3.2.3 Recruitment and Ethics The recruitment process took place between February and December in 2012. An email announcement was initially sent to the nurses by the patient service manager, followed by face-to-face recruitment by the unit?s clinical educator to increase participation rate. Posters were also displayed in the medication room and nurses? lounge. Participants were given at least twenty-four hours to decide whether they want to participate in the study. A total of sixteen nurses were initially recruited into the larger study. However, six dropped out due to reasons including refusal to participate in usability testing, scheduling issues, and not having a PCIS icon on their desktop.  26  Written informed consent was obtained prior to usability testing. Nurses were made aware that they could withdraw at any point during the study and that their participation would not impact their performance evaluation. A copy of the consent form was also given to the nurses to keep. Each participant was given a $50 drugstore gift certificate at completion of the usability testing in recognition for participating in the study.  3.3 Data Collection For the purpose of this descriptive study, two data collection methods were employed: 1) usability testing with think-aloud method, and 2) demographics survey. 3.3.1 Setting The unit had been using a hybrid system, electronic and paper, for clinical documentation since 2008. Nurses charted electronically for patients who were not part of a clinical pathway. A clinical pathway is a pre-printed patient care documentation tool that is used for specific types of surgeries on this unit. If patients were not on a clinical pathway, nurses would complete their full patient assessment and documentation on PCIS. In addition, nurses could access laboratory results and perform order entries in the system. In this study, nurses used the iPads solely for clinical documentation and could not, for example, search online resources or drug information due to organizational restrictions with privacy and security that were decided by the organization. Nurses were required to log into the Citrix Server through the Citrix Receiver application, which was downloaded onto each iPad to allow remote access to the hospital computer system. Due the organizational infrastructure, the iPads used a workstation-on-wheel as a proxy to connect to the wireless network system. Workstation-on-wheels (WOWs) are computers or laptops that are mounted to a cart with wheels which enable health care professionals to 27  manoeuvre the computers to the bedside. WOWs can facilitate point-of-care documentation or information retrieval, and can act as a storage space or provide additional surface area. When an iPad was using a workstation-on-wheel as a proxy, that workstation could not be used by other nurses because another nurse signing in on that computer would disconnect the iPad user. As a result, the workstation-on-wheels that corresponded with the iPads were locked in a closet to prevent use by other healthcare professionals.  3.3.2 Materials The unit obtained three 2nd-generation iPads from the organization for the study. The iPad was encased in a Bluetooth keyboard binder. A soft, rounded tip stylus was attached to the side of each iPad using Velcro.  3.3.3 Education  From January to February of 2013, the clinical educator of the unit taught nurses on how to login to the iPad. One-to-one education was provided to nurses during work hours. The education incorporated demonstration and return-demonstration on how to login, and some explorations of functions and features of the iPad. Nurses were offered reimbursement to come in on days off for education; however, all participants declined the offer, opting to be educated during working hours.  3.3.4 Demographics Survey A demographics survey was collected to obtain baseline information about the participants? age, education, years of practice, and personal computing devices use. (See Appendix B for demographics survey.) 28  3.3.5 Usability Testing with Think-Aloud Method Ten nurses were invited to participate in the usability testing. The researcher conducted each testing session with one nurse at a time. Nurses were given a fictitious patient scenario with a set of clinical documentation tasks, which they used the iPad to complete. The patient scenario was modified three times to improve clarity and eliminate redundancy. (See Appendix A for patient scenario with marked tasks.)  Nurses were instructed to enter the fictitious patient?s clinical information into the PCIS training environment while verbalizing their thoughts. Data were collected as nurses worked through the patient scenario by video-recording the iPad screen and audio-recording the nurses? verbalizations using a usability testing software application (Morae Recorder). The nurses themselves were not recorded in this process; only the computer screen was captured.   There was a four to six week lag from when the participants last used the iPads to the time of testing due to scheduling differences and timing of education. The iPads were used on the unit between January 30 and February 28 of 2013. The usability tests were carried out between March and April of 2013. 3.3.5.1 Field Notes At the end of each usability session, the researcher recorded field notes that described the factors that may have influenced the study, personal emotions/assumptions, non-verbal communications, characteristics of the nurses, and the progress during the study. Field notes allow the researcher to document a narrative account of what is happening during the study and can serve as data for analysis (Polit & Beck, 2012).  29  3.3.5.2 Morae Recorder  Recording software called Morae Recorder was used to capture audio- and video-recordings of the participants. The video-recordings were captured by Microsoft LifeCam VX-3000 webcam that was connected via a USB-port to a laptop. The webcam was mounted on a makeshift stand that was the appropriate height to record audio and visual data of the participants? interactions with the iPad. The recordings directly fed into the Morae Recorder software, which was saved and later analyzed.  3.3.5.3 Testing Environment Set-up The usability sessions were conducted in a staff computer room on the nursing unit. Due to the set-up of the room, the researcher was closely located to the right-hand side of the participant about twelve inches away. The iPad was placed on a long table in the closed position. The webcam, which was connected to a laptop computer running the Morae Recorder, was focused on the iPad. The laptop screen was tilted away from the participant. Instruction was given to not elicit interaction with researcher if possible and to attempt the fictitious patient scenario independently.  Figure 3.1 Set-up of Usability Testing Environment  30  3.4 Data Analysis This section outlines the data analysis methods. Each recording was reviewed a minimum of two times. Ambiguities in findings and coding were reviewed and discussed with the researcher?s supervisor (Dr. Leanne Currie) to arrive at a consensus.  3.4.1 Demographics Surveys Demographic surveys were entered into an Excel spreadsheet and then into SPSS version 21, and descriptive statistics were performed.  3.4.2 Usability Testing Analysis The usability testing analysis was completed using a software application called Morae Manager. A coding scheme was created to analyze the data. Data from the codes, audio-recordings, observations, and field notes were used for the analysis.   3.4.2.1 Coding Scheme A top-down approach, which refers to beginning the analysis with categories or codes, was used to log findings using pre-determined categories (Barnum, 2011). Dawson and Kushniruk (2009) had created categories for coded events in their study on computerized physician order entry using a tablet device. These categories were adopted and modified to fit the current study. After completing all usability session, the coding scheme was created through an iterative approach using Dawson and Kushniruk?s template as a starting point, and as new events arose in the initial analysis of the recordings, the coding scheme was modified. In addition, codes that highlighted the multi-touch screen functions of the iPad and problems specific to this study (e.g., login problems, pop-up menu) were also incorporated in the coding scheme (Apple, 2011). The coding scheme below identifies the seventeen markers used in the study and its definition. 31   Additional markers, including login problems, interaction with researcher, preferred action not clear, and quote/comment/observation were used to expand and supplement the understanding of problems in navigation, screen size, and data entry. Table 3.1 Coding Scheme Code Description Positive/Neutral/ Negative Navigational Problems   Tapping with finger? Tapping screen using finger Neutral Tapping with stylus? Tapping screen using stylus Neutral Mistaps* Tapping the intended target causing an unintended action/selection Negative Pop-up menu (Right Click) Tapping elicits unintentional pop-up menu Negative Extra taps* ? No response Participant?s first tap on the intended target resulted in NO response/changes, which further results in participant tapping at least once more Negative Extra taps* ? Processing/Loading Participant?s first tap on the intended target resulted in a response, but no obvious feedback to the user resulting in another tap Negative Drag screen? Dragging screen to visualize another area of the window. Neutral Scrolling problems* Problems related to scrolling Negative Screen size   Zoom in? Enlarge screen Neutral Zoom out? Minimize screen Neutral Data Entry   On-screen keyboard Using on-screen keyboard Neutral Keyboard shortcut Use of external keyboard shortcuts. Positive/Neutral Self-correction* Correcting own mistakes independently Positive Additional Markers   Login Problems Problems related to the login process Negative Interaction with researcher*  When participants have questions and/or require troubleshooting, prompting, or help Negative Preferred action not clear* Participant unaware of next step or how to proceed Negative Quote/Comment/Observation Verbatim quotes from participants, comments and observations by researcher Neutral * This code was adopted from Dawson and Kushniruk?s (2009) ? This code was adopted from the multi-touch screen function of the iPad (Apple, 2011).  32   Each code was categorized into navigational problem, screen size, data entry and additional markers, and a description is provided. The codes were given a weight of positive, neutral, or negative. Positive codes were behaviours or events that might help or support users during clinical documentation. Neutral codes were events that were neither positive nor negative, but depending on how the event presented during the task, the code was interpreted as either positive or negative. Negative codes were events or behaviours that hindered or impeded the user during clinical documentation. The coding scheme reflected provider and work system factor performance inputs from Karsh et al.?s (2006) input-transformation-output model. User behaviours in navigation, readability, and data entry are performance inputs. These behaviours can also be influenced by provider factors such as individual skills, training, education, and/or age, to name a few. Login problems are performance inputs related to work system factors. Events such as interaction with the researcher, preferred action not clear, and quote/observation/comment and field observations can reveal either provider or work system problems that influence performance, and therefore need to be interpreted within the context of the situation they originated. The work system, or the organization, provides tools such as technological devices, clinical documentation system, and wireless connection infrastructure. The organization is in charge of providing systems that support clinicians? work. These performance inputs contribute to performance outputs that cause changes in the individual, organization, and/or other parts of the system.  3.4.2.2 Morae Manager The Morae Manager software was employed for data analysis. The software enabled playback of the audio and video recordings taken during usability testing. The researcher logged different markers or codes to the videos at specific timestamps. The Morae Manager also enabled 33  exporting markers into Excel, which permitted the researcher to tabulate the frequency of each code, sort and search codes, and produce graphical charts.  The fictitious patient scenario was divided into seven tasks in Morae Manager: 1) login, 2) vital signs, 3) routine assessment, 4) tubes and drains, 5) behaviour/mood, 6) order entry, and 7) logout. The length of time it took the participants to complete the scenario and each task were also noted. 3.4.2.3 Qualitative Data Qualitative content analysis was used to analyze the codes, audio-recordings, observations, and field notes. Qualitative content analysis is a method that deconstructs data into smaller categorical units, or codes (Polit & Beck, 2012). The coded material was subsequently grouped into shared concepts or themes and the frequency of how often the event occurred was counted. Observations and field notes were examined and extracted to support further understanding of the context. Lastly, noteworthy quotes were transcribed, and reviewed in the context of what the participant was doing at the time. Qualitative data were used to provide context for the quantitative results. 3.5 Limitations of Usability Testing Usability testing has several limitations. Because usability testing is often carried out in a controlled setting or laboratory, the artificial environment does not adequately represent real world situations (Barnum, 2011; Rubin & Chisnell, 2008). For this reason, there are few to no interruptions or distractions during the usability testing that generally exist in real world situations. The participant has time and focus to complete each task. In addition, participants may not accurately represent the target population (Rubin & Chisnell, 2008). Furthermore, the results of the usability testing cannot always guarantee the product?s success as there are still 34  other real world factors that can influence its uptake. As a result, usability testing should be used in conjunction with other testing methods such as ethnographic observations, satisfaction surveys, and focus groups and/or key informant interviews to obtain a better understanding of how health information technology can impact nurses? work. Data analysis limitations might also exist because pre-determined codes were used to log findings in the audio and video recordings. Despite the codes being adapted and modified through an iterative process in the beginning, having pre-determined codes might run the risk of missing important findings that may not fit into categories. In addition, misidentification, or wrong labeling, can occur if there were ambiguities to the phenomenon (Barnum, 2011).  35  Chapter 4: Results This chapter will present the results of the demographics survey and usability study.  4.1 Results from Demographics Survey This study consisted of ten participants. One of the usability recordings was omitted due to an audio issue, resulting in nine recordings. Eight of the nine participants completed the demographics survey. Table 4.1 shows nurses? characteristics in the study. Half of the participants were between the ages of 41 and 50 (50%). Six of the participants obtained a Baccalaureate degree (75%), while two nurses had a diploma in nursing (25%). The majority of participants in this study were female (87.5%) and one male (12.5). The average number of years the nurses were in the profession was 13.2 (median= 10.5; min= 1, max= 31). They also worked in the study setting for an average of 11.5 years (median= 6.75; min= 1, max= 31).  All participants owned a desktop computer. More than half of the participants had a Smartphone for personal use (62.5%); four participants owned a tablet (50%); and six participants owned a laptop (75%). All participants used one or more personal computing device at least once a day, most of whom use a device several times a day (75%). Those participants who owned a tablet reported in the larger study that the tablets were iPads. Table 4.2 shows the difference of years in nursing by Smartphone use. There was 9.3 years difference in years of nursing with nurses who used Smartphones and nurses who do not use Smartphones. Nurses with less nursing experience more commonly used Smartphones compared to nurses with more experience.    36  Table 4.1 Description of Nurses? Characteristics   Table 4.2 Difference Years in Nursing by Smartphone Use  Mean Years in Nursing (Median; Min, Max) Smartphone Users (n=5) 19.0 (8.75; 1, 27) Non-Smartphone User (n=3) 9.7 (13; 5, 31) Mean Difference 9.3   4.2 Results from Usability Testing The audio- and video-recordings were reviewed and tagged using the coding scheme outlined in Chapter 3. There were a total of 1675 user behaviours and events tagged, of which 1148 were used as quantitative data: tapping with stylus, tapping with finger, zoom in, zoom out,  Frequency (%) (n=8) Age Group  21-30 1 (12.5) 31-40 1 (12.5) 41-50 4 (50) 51-60 1 (12.5) Prefer not to say 1 (12.5) Education  Diploma 2 (25) Baccalaureate degree  6 (75) Gender  Female 7 (87.5) Male 1 (12.5) Personal Computing Device Use, Yes (%)  Smartphone 5 (62.5) Tablet (iPad) 4 (50) Desktop 8 (100) Laptop  6 (75) Frequency of Use  Once per day 1 (12.5) Several Times per day 6 (75) Other- Many 1 (12.5) 37  mistaps, pop-up menu, extra taps with no response, extra taps during processing or loading, dragging screen, scrolling problems, on-screen keyboard use, self-correction, and keyboard shortcut use. The remainder 527 events were qualitative data from observations, quotes, and comments, interactions with researcher, preferred action not clear, and login problems. The qualitative data were integrated into the results section to further the understanding of the user behaviour and events. 4.2.1 Time on Tasks The fictitious patient scenario consisted of seven distinct tasks (see Appendix A). Each usability session was between thirty to sixty minutes long. The average time to complete all tasks was 35:24 minutes. Table 4.3 shows the average time on each task. The longest task in the scenario was routine assessment, which took an average of 9:20 minutes. For this task, users were required to document three different assessments: respiratory, gastro-intestinal, and skin. Because these three tabs were under the same heading of ?Routine Assessment/Intervention?, they were grouped into one task. Although this task would have generally required more time to complete, this portion of the patient scenario also acquired the second highest number of user behaviour problems (n=332). (See Table 4.4)  Table 4.3 Time on Task Task Mean (min:sec) Login 6:05 Vital Signs 5:16 Routine Assessment 9:20 Tubes/Drains 2:25 Behaviour/Mood 3:12 Order Entry 4:47 Logout 4:19 Average time for all tasks 35:24  (Median= 30.6, Min=18.1, Max= 53.6)  38  The second longest task was the login process, which took an average of six minutes. The timing of the login process first began with unlocking the iPad, signing into the Citrix server, selecting the computer identification, logging into Windows desktop, and finally signing into the PCIS training environment. Extra taps with no response was the highest user behaviour marker, followed by tapping with stylus and finger in this task. Login problems will be discussed further in this chapter.  The third most time-consuming task was the process of entering vital signs, which took an average of about five minutes. All nurses were familiar with the process of entering vital signs, however, this task acquired the highest number of user behaviour problems (n=390). The majority of the behaviours were related to extra tapping, switching between a stylus and finger, and zooming in and out of the screens.  4.2.2 Navigational Problems  Navigational problems consisted of user behaviours that include tapping with stylus, tapping with finger, mistaps, tapping that elicits the pop- up menu (or right click function), extra taps with no response, extra taps while system was loading, screen dragging and scrolling problems. Table 4.4 Frequencies of Marker Types by Tasks for Navigational Problems User Behaviours Login Vital Signs Routine Assessment Tubes/ Drains Behaviour/ Mood Order Entry Logout Total Tapping stylus 11 38 15 5 4 5 3 81 Tapping finger 13 42 16 6 8 5 4 94 Mistaps 3 32 31 9 5 12 2 94 Pop-up menu 8 19 12 1 1 2 1 44 Extra Taps- No response 50 170 172 55 29 60 17 553 Extra Taps- Processing/Loading 4 2 1 0 2 0 0 9 Drag screen  0 27 24 4 4 8 2 69 39  User Behaviours Login Vital Signs Routine Assessment Tubes/ Drains Behaviour/ Mood Order Entry Logout Total Scrolling problems 0 0 17 0 8 5 0 30 Total 89 330 288 80 61 97 29 974  4.2.2.1 Extra Taps As shown in Table 4.4, extra taps (n=533) that elicited no responses accrued the highest frequency among all tasks and all user behaviour incidents. As described in the coding scheme, extra taps without a response refers to a user?s attempt to tap on an intended target but the attempt results in no response which triggers additional taps. Typically, these cases happened when users had to tap on data entry fields, drop-down menus, checkboxes, and other objects that require precision. Oftentimes, users missed the intended target and switched to another method of tapping. If a user began the task with tapping with the finger and encountered difficulty selecting the objects, he/she would switch to using a stylus. This was a common back and forth occurrence, as evidenced by the high number of tapping with stylus (n=81) and tapping with finger (n=94). On some occasions after repeated unsuccessful taps, participants had a tendency to press on the screen harder. One nurse pressed the screen hard enough to dismount the iPad from the keyboard. This happened for both finger tapping and stylus tapping.  Another incident that occurred was when a user tried to add a column on PCIS and tapped on the ?Add? button multiple times without success in proceeding to the next screen. This might be related to system behaviour problems that may not be related to the device; however, it is unclear whether the interface of the iPad might have contributed to changes on PCIS.  Extra taps during system processing and loading only accounted for nine user behaviour incidents. This happened when users could not tell the status of the system while it was loading, and thus tapped the target again.  40  4.2.2.2 Mistaps Mistaps were also common during usability testing (n=94). A mistap was characterized by tapping on the intended target, but instead of successfully executing the selection, a different response occurs. The difference between extra taps and mistaps is that extra taps have no response, whereas mistaps produce an incorrect response. For example, mistaps frequently occurred when selectable objects were close together or when the pointer (stylus or finger) missed the object. In addition, almost half of all mistaps resulted in unintentionally accessing the pop-up menu (the right-click menu) (n=44). 4.2.2.3 Stylus Versus Finger Tapping with a stylus and tapping with finger accounted for 81 and 94 user behaviour markers, respectively. Few users predominantly used one form over the other. Most participants used the stylus or their finger interchangeably without being triggered by a problem to switch. However, there were a number of instances when users encountered problems, such as extra taps and mistaps, which caused users to switch between stylus and finger tapping. Users switched between the two methods because the original method failed to precisely select the intended target. Because the target was small, users had trouble with precision and accuracy with both the finger and the stylus. Users were also observed to use their fingernails to be more precise. The transition between finger and stylus could sometimes be rapid because users were unable to produce a successful response. In addition, when users encountered a data entry field where they needed to use the keyboard, they would select the next data field by tapping with their finger while still holding the stylus in the pointer hand. If the data entry had many letters to enter or users needed to navigate through multiple fields, they sometimes abandoned the stylus completely and used only their finger to tap.  41  4.2.2.4 Scrolling Problems Scrolling problems accounted for a small number of user behaviour problems (n=30), pre-dominantly concentrated in routine assessment, and behaviour and mood. Scrolling could occur in other parts of PCIS system, but the usability test patient scenario did not include those areas.  Figure 4.1 Scrollbar in Respiratory Screen  Under routine assessment, participants had to scroll through a table that spanned the entire width of the screen with a scroll bar at the edge of the right margin. Because the iPad was running the desktop version of Windows, users needed to use the scroll bar to navigate up and down the table. Difficulties occurred with using their finger or stylus to grasp and hold the scrollbar while scrolling and with tapping on the up and down arrow keys of the scrollbar. Participants were able to grasp and hold the scrollbar to scroll; however, if users lifted their finger slightly or deviated too far from a straight up and down movement, the hold was released and users had to try again. The up and down arrow keys were also small which resulted in users missing the target and produced subsequent extra taps and mistaps. In addition, participants, who were familiar with a tablet or Smartphone, also attempted to scroll by swiping the screen 42  upwards or downwards, however, this motion selected the contents within the table rather than scroll, which is inconsistent with general iPad functions. Figure 4.2 Arrow Key in Comment Box  In the patient behaviour and mood documentation task, participants were required to enter free text into a comment box. After documenting the text, participants would scroll up to check their work. Because the comment box was small, the distance between the up and down arrow keys was close together inside the comment box. Participants had difficulty tapping on the arrow keys precisely. One of the participants stated, ?It?s hard if you want to get in there to check your spelling. I don?t know how you can sort of scroll.? Some users left the cursor in the text, and used the external keyboard arrows to maneuver their position in the comment box. This latter approach gave the user more control. 4.2.2.5 Screen Dragging Participants were also observed to drag the screen a total of 69 times. Dragging the screen was typically associated with enlarging the screen (n=68); after zooming in, users could not 43  visualize another part of the screen; therefore, users had to drag the screen to see a different area. In addition, enlargement of the screen would also hide the patient identifier. During scenario the participants were asked to document data for one patient only, therefore, participants did not check to see whether they were on the correct patient when they zoomed in.  4.2.3 Screen Size Table 4.5 Frequencies of Marker Types by Tasks for Enlarging and Minimizing  Another usability problem was with the size of the font and icons, and the size of the screen on the iPad. Zooming in and zooming out accounted for 68 and 74 of the user behaviours marked, respectively. The above sections alluded to the small target objects that required precision or enlargement of the screen in order to produce a successful action. The majority of these behaviours were observed when documenting vital signs and routine assessments. The vital signs screen had eighteen data fields, eleven of which were drop-down lists and the remainder seven were text fields. Routine assessment tasks had multiple screens. The initial screen for respiratory status also had at least four drop down lists for documentation. In general, all data fields were closely grouped together. After selecting the first item, the system occasionally triggered the next drop-down list to open; however, this was not consistent for all cases. Despite the navigational problems mentioned above, nurses did not use the enlarging and minimizing functions significantly to facilitate free-text documentation efforts as evidenced by the low User Behaviours Login Vital Signs Routine Assessment Tubes/ Drains Behaviour/ Mood Order Entry Logout Total Zoom in 2 26 18 11 5 6 0 68 Zoom out 1 22 18 11 8 11 2 73 Total 3 48 36 22 13 17 2 141 44  frequency of zooming in for behaviour and mood task. Only three nurses use the zoom in function in that task.  Figure 4.3 Data Fields on Vital Signs Documentation Screen  During the usability session, participants also commented on the display being small. For the routine assessment task, one participant stated, ?I?m trying to get to Peripheral 1, but because it?s so small, I keep hitting the other one.? This statement embodied the issue of small targets resulting in the likelihood of mistapping or extra tapping. Another participant also mentioned that she was not ?comfortable? with having to enlarge and minimize the screen multiple times, and that given the option, she would prefer to use a stationary computer. Finally, there was an instance when the user zoomed in to enter his/her username and password at the Windows Desktop screen, and after successfully logging in, the screen appeared blank with no desktop icons visible. The participant was unaware that the screen was still zoomed in and continued to wait eighty seconds for the screen to ?load? and stated the system ?takes a long time?. The participant was confused as to how to proceed. As a result, the 45  researcher helped participant return to the original size of the desktop screen by zooming out. The participant was able to continue the scenario thereafter.  4.2.4 Data Entry Table 4.6 Frequencies of Marker Types by Tasks for Data Entry User Behaviours Login Vital Signs Routine Assessment Tubes/ Drains Behaviour/ Mood Order Entry Logout Total On-screen keyboard 0 3 0 0 0 0 3 6 Self-correction 0 5 8 3 2 4 0 22 Keyboard shortcut 0 4 0 1 0 0 0 5 Total 0 12 8 4 2 4 3 33  The most predominant data entry method was the Bluetooth-connected external keyboard. As soon as the testing began, participants used the external keyboard to enter all text-based data fields. One participant mentioned that the keyboard was too small and impeded proper typing. Similarly, another participant expressed his/her annoyance with typing on the external keyboard because of the small size. Keyboard shortcuts were also infrequently used (n=5). The ?Tab? key was used to continue onto the next data entry field; however, not all participants used keyboard shortcuts. Most of the users tapped on the screen to move onto the next data field. Self-correction accounted for 22 user behaviour incidents. This typically happened during free-text entries when users recognized a mistake or when users recognized that they were documenting in the wrong area in PCIS, which users would then correct their previous attempts. Self-corrections were more related to the use of PCIS rather than the iPad interface causing documentation errors.  Another problem with data entry was related to saving free-text data on the iPad. Saving data on PCIS while using the iPad was different compared to a computer with a full-size keyboard. On a computer with a full-size keyboard, the nurses used the F12 button to save free-46  text documentation on PCIS; that is, nurses pressed the F12 key on the keyboard which would save the data and close the documentation screen signifying that their information had been saved to the database. When documenting on the iPad, the on-screen keyboard or the external keyboard did not have an F12 key. In order to save, nurses had to tap on a tab named ?File? inside the comment box and select ?Save to database? from the dropdown menu. After doing so, the nurse had to manually leave the documentation screen. In the usability testing, after nurses had finished entering their free-text note, six out of nine nurses could not remember how to the save data and needed help to proceed. Most nurses were accustomed to using the F12 function, therefore had problems recalling how to save the data without the F12 key.  Figure 4.4 F12 Function   Participants did not use the on-screen keyboard for data entry, but the on-screen keyboard was unintentionally activated six times during the study. This occurrence, though infrequent, often caused delays and confusion during documentation. One of the ways the on-screen keyboard could appear was by pressing a shortcut on the external keyboard. When this happened, 47  the user could choose to type with the on-screen keyboard or the external keyboard. This scenario occurred with one participant, in which she had unintentionally activated the on-screen keyboard but was still able to type using the external keyboard.  Another way of triggering the on-screen keyboard was through the drop-down menu at the top of the screen. If a user selected the on-screen keyboard by this method, the user could not use the external keyboard. The on-screen keyboard could also be minimized to the bottom of the screen, which might not be obvious to the user. One user unintentionally accessed the on-screen keyboard by this method. The selection was not immediately apparent to the user while he/she continued to type with the external keyboard and no text was showing up. Because the on-screen keyboard was minimized to the bottom of the screen, the user de-selected and re-selected the on-screen keyboard again without noticing the difference. The user was unable to use the external keyboard until he/she had successfully turned off the on-screen keyboard.  When nurses were unable to solve the problem of unintentionally activating of the on-screen keyboard or the on-screen pointer, they interacted with the researcher to ask for help. In one usability testing session, the user had unknowingly selected the on-screen pointer and on-screen keyboard from the iPad drop-down navigational tool bar. When the on-screen pointer was selected, the user had to drag the pointer to the item they wanted to select, rather than tapping on the item. When the on-screen keyboard is selected from the navigational toolbar, the external keyboard cannot be used. When these two settings were selected, the user experienced difficulties with selecting items and entering text. The user, along with the researcher, took approximately ten minutes of trial and error to rectify the problem. After the problems had been identified and fixed, the user was able to proceed smoothly with the patient scenario. 48  One participant demonstrated the activation of the on-screen keyboard from the drop-down menu to the researcher. The participant stated, ?? if I'm using this [on-screen] keyboard... if you have this [iPad binder with external keyboard] on it, it doesn't work. So you get stuck in different screen more." This participant preferred to use the on-screen keyboard on the iPad in the clinical setting, and realized that only one keyboard can be used at a time.  4.2.5 Login/Logout Problems There were a total of forty-three incidents of login problems in the usability study, which were broken down into problems related to signing in (n=29), system problems (n=8), unable to recall the login process (n=4), and loss of connection with Bluetooth keyboard (n=2).  As mentioned above, the login process consisted of multiple steps, beginning with unlocking the iPad, signing into the Citrix server, selecting the computer identification, logging into Windows desktop, and finally signing into the PCIS training environment. Figure 4.5 shows screenshots of the login steps. Figure 4.5 Five Screens for Login Step 1: Login Screen   Step 2: Citrix Server Login Screen    49  Figure 4.5 Five Screens for Login (con?t) Step 3: Computer ID Screen  Step 4: Desktop Login  Step 5: PCIS Login Screen    Twenty-nine problems with account sign-in were related to participants being unable to recall which username and password to use for Citrix server and Windows desktop login, and also what computer identification number to enter. Users, who had recently changed their passwords, also had difficulty recalling their new passwords. In addition, participants sometimes used the wrong username, such as using their PCIS? login, to log into Windows desktop. Many users had to be reminded that their password for Citrix server and Windows desktop was the one they used to log into their email accounts. The computer identification numbers were used to connect the iPads with the corresponding mobile workstation computers that acted as a proxy for remote access. The computer identification number was taped on the border of the iPad screen. 50  When the users reached that step, they often could not recall what to enter for the computer identification number. There was one incident after the user had entered the computer identification number, the user waited fifty seconds for the system to load with no feedback. The researcher prompted the participant to cancel the process and try to login again. There were eight occurrences of problems that might have been caused by Citrix server system, network connection, or the iPad device. The root cause for these problems could not be identified. In two instances, the participants were attempting to sign into Citrix server, but the Citrix application closed without warning and without any user input. The participant was able to reopen the application and try again. Two other instances involved users being unable to sign into Citrix server. After multiple attempts, one participant was locked out from her account. The participant had to call the organization?s help desk to reset her account. Another participant had the same problem, but the unit?s clinical educator was able to sign in for the participant. The reason why these two participants? accounts did not work is unknown because their accounts were accessible prior to testing.  There were four incidents of users being unable to recall the login process. After unlocking the iPad, one user hovered the pointer over the Citrix Receiver application and the Safari application. She could not remember which one she needed to use in order to login. Similarly, another participant opened the Safari application and could not recall how to login. One participant simply stated she could not remember how to log in once she entered the Citrix Server application. In another instance, one user had successfully entered their username and password, and was aware that he/she needed to tap on an icon to proceed. In order to establish the remote desktop connection, the user needed to correctly enter username and password into the Citrix server, and then tap on the icon in the center of the screen in order to continue. The 51  participant could not recall the next step in order to proceed. He/She mistakenly tapped another button on the screen as a trial and error effort to continue. In all the above cases, the researcher prompted the users on how to proceed.  Two occurrences of login problems were related to connection. In one instance after the participant had unlocked the iPad, there was no wireless connection. The participant was unsure how to re-establish wireless connection under iPad settings. The user was shown how to reconnect to the organization?s network. Another instance was related to the Bluetooth external keyboard. The external keyboard was not functioning, and user was unsure how to reconnect the keyboard with the iPad. The user was shown how to turn on the Bluetooth on the external keyboard.  The logout process also created problems for users in the usability study. There were four steps to logging out that included first signing out of PCIS training, then Windows desktop, Citrix server, and finally closing the iPad (optional). There were sixteen instances in which users asked for help during the logout process. Six out of nine participants needed help. Two participants thought that they could log out by pressing the Home button on the iPad. In addition, they both thought that by swiping to ?Spotlight search? on the iPad was a means to finalize the logout process. Spotlight is a search tool that can be accessed by swiping to the right or by clicking the Home button when on the Home screen, and Spotlight is characterized by a dimmer screen and a search bar. Two participants thought they had signed off when they went to Spotlight search screen, likely under the impression that the dimmer screen signified logging off. One of the two participants believed the power button would log the user out of the system. After being guided on how to logout, one user stated, ?I should have written [the logout process] down and I [would] remember.? Four participants could not remember how to log off the Citrix Server 52  after signing out of PCIS training and Windows desktop. Two participants verbally stated, ?I can?t remember.? Once users had reached the Citrix application screen, participants must tap the ?Log Off? button to completely log off. The two participants had trouble remembering how to logout or thought they had successfully logged off the system when in fact they had not.  Lastly, one participant logged out of PCIS but skipped the logout for Windows desktop and went straight to the Citrix Server screen, which is possible from selecting log off from the drop-down navigational tool bar that the top of the screen). When she arrived at the Citrix Server screen, she knew how to sign out. However, the researcher prompted the user back to Windows desktop to sign out of her account. At this time, the taskbar disappeared and user could not access the Start button. The participant and the researcher tried to troubleshoot the problem. After approximately eight minutes of unsuccessful attempts to log out of Windows desktop, the clinical educator was notified to log off the user from the main computer.  4.3 Summary In this usability study, there were a number of usability violations that prevented users from efficiently completing clinical documentation. Some of these problems were related to the device and its compatibility with the organization?s information system (PCIS). These problems were related to navigation, data entry, screen size, and login/logout process.    53  Chapter 5: Discussion In this last chapter, the results of the study will be discussed in relation to current the literature, as well as the strengths and limitations of the study, and recommendations for future practice and research.  This study aimed to examine the features and functionalities of the iPad while nurses documented on the patient care information system (PCIS). Usability testing and the think-aloud protocol and a short demographic survey were used in this investigation. Nurses on the study unit have been using a hybrid system (i.e., paper and electronic) for clinical documentation since 2008. The sample was comprised of eight nurses, seven females and one male. The average number of years in the profession was 13.2 and the average number of years in the study setting was 11.5. All nurses reported that they had a desktop computer for personal use, and some nurses also owned a tablet (50%) and Smartphone (62.5%).  In the study, usability violations were categorized into four main groups: navigation problems, screen size, data entry, and login/logout problems. Extra taps, mistaps, scrolling problems, use of stylus and/or fingers, and dragging screen were common navigational problems. Screen size problems were often characterized by user zooming in and zooming out of the screens. Data entry was most problematic when the on-screen keyboard was unintentionally selected in addition to the external keyboard being too small. Login and logout problems were related to troubles with signing into own account (e.g., passwords), problems with the Citrix system, and inability to recall the login or logout process.  Relating back to Karsh et al.?s (2006) model, this study focused on the provider and work system factors as the performance inputs. Personal attributes such as computer skills, knowledge, education, and experience can contribute to nurses? interaction with and perception of a new 54  technology innovation. Work system factors include time and task demands, technology functions and features, as well as noise or distractions. These performance inputs undergo the transformation process, which in this study focused on cognitive performance such as memory, attention, problem-solving, analyzing, and learning. The resulting immediate performance output is a change in mental state such as fatigue, knowledge, and stress, and also a change of state in technology, such as timeliness of clinical documentation and ease of use of device. Karsh et al. (2006) propose the ultimate performance outputs are patient safety, employee safety, and quality of care. If these outputs are not met, the system will have to undergo redesign in order to remedy and/or create ways to reach these goals. The usability testing in this study was informative in revealing many areas that would benefit from thoughtful redesign.  5.1 Navigational Problems In the usability study, participants were observed to tap on the screen using the stylus or their fingers. The use of the stylus or finger is a neutral behaviour and is not inherently positive or negative; however, the repeated switching between finger and stylus was a source of frustration for nurses. Users frequently switched between tapping with the stylus and the finger because the method they began with produced problems such as extra taps, mistaps, and scrolling problems. Although documentation errors did not always occur with extra taps and mistaps, these problems caused wasted time and effort. This was consistent with the study by Dawson and Kushniruk (2009) in which the users of a CPOE system via a tablet required multiple extra taps to complete their work. Scrolling was also a problem during clinical documentation in the current study. The gesture for scrolling that would typically work on an iPad was ineffective when the iPad was running the desktop version of Windows. When an iPad is running Apple?s operating system, 55  users can flick the page up or down from anywhere within the window. In the study scenario in which the iPad was running Microsoft Windows, users had trouble scrolling up and down the screen because they needed to use the scrollbar in order to navigate the length of the table. Not only were the up and down arrow keys difficult to tap on precisely due to their small size, users also had trouble maintaining a hold on the scrollbar.  Microsoft Windows uses a graphical user interface (GUI) that allows users to interact with the system through graphical/visual elements. The most basic components of graphical user interface are window, icon, menu, and pointing device (WIMP) (Lawrence & Verzani, 2012). For example, a picture of a folder is a graphical representation that allows the user to explore the contents of the folder if clicked. A mouse, or pointer, allows for precise and rapid movements on the screen; therefore, small graphical objects can be easily selected using a mouse with this type of user interface (Wigdor & Wixon, 2011). However, running a WIMP graphical user interface is not compatible with a device that is optimized for touch-based interface since the WIMP interface constricts the natural functions of a handheld device.  A WIMP interface allows only one pointer input at any given time, whereas a touch-based interface can support interaction with more than one finger on the screen (in addition to other inputs such as tilting or turning the device). In the current study, running a WIMP graphical user interface on a small touch screen created problems such as decreasing the size of icons/objects or texts, and altering the features innate to the two systems (e.g., saving documentation or scrolling), which creates a mismatch between desktop and iPad use for the nurses. These changes can create discrepancies in nurses? workflow, accuracy and efficiency of clinical documentation, or timeliness of the documentation, which may ultimately affect patient safety.  56  Whether the users are utilizing the stylus or their finger, the mode that they choose to interact with the touch-screen can have significant impact on how users carry out their work. In a study that examined the performance in touch selections, tapping with finger was found to be faster but often less accurate than stylus or mouse, particularly with small objects (Cockburn, Ahlstr?m, & Gutwin, 2012). For dragging tasks, the study showed that finger input was slower than mouse/stylus input possibly because of higher surface friction (Cockburn et al., 2012). Wigdor and Wixon (2011) described a user input problem called the ?fat finger? problem. The ?fat finger? problem is a combination of two problems, the first being that a large area of the user?s finger comes into contact with the touch screen, and the second being that the user?s finger occludes where the user is tapping exactly. Many touch screen platforms cannot detect the finger input until the finger is in contact with the screen, therefore the specific area where the user needs to touch can be occluded by their finger (Wigdor & Wixon, 2011). The authors recommended that systems need to be designed such that users will not encounter the fat finger problem thereby increasing their success in hitting a target precisely. This could be achieved by increasing the physical size of the targets, making the on-screen graphic smaller than the actual area that will trigger a response, or by providing different techniques to allow selection of a smaller object (e.g., slide off, release) (Wigdor & Wixon, 2011). Furthermore, not only can larger buttons and targets be easy to hit, distracted mobile users can see the target more easily without having to focus on the screen for longer (Clark, 2012). The distance between targets also contribute to the accuracy of each tap; that is, the closer the buttons are together, the larger the buttons should be (Clark, 2012). Therefore, the success of executing an action on the touch screen is dependent on multiple factors such as target size, spacing, stylus dimension, and amount of friction on screen. If the iPad is going to be used in the clinical setting described in 57  this study, these factors have to be taken into consideration when choosing a device and designing the information system.  5.2 Screen Size  Screen size on the iPad device was a problem for some users in the usability study. Users struggled with the small target sizes but also commented on the small display of the contents. Users were observed to crane their neck and focus intensely on the device during the usability sessions. In some studies, users found the PDA screen size to be small and was difficult to read large volumes of data (Carlson et al., 2010; Di Pietro et al., 2008; Doran et al., 2010; Fischer, Stewart, Mehta, & Wax, 2003; Garrett & Klein, 2008). One study that examined tablet devices found users were more satisfied with the amount of information displayed on the tablet screen (Doran et al., 2010). However, many studies focused on the viewing of clinical information resources on smaller handheld devices such as PDAs (e.g., reference materials, practice guidelines) rather than on data entry or documentation purposes. The function that allows users to enlarge and minimize the screen during charting may be cumbersome to nurses when they want to chart quickly and also when there are multiple screens to go through. Using the zooming function on the iPad sometimes required users to drag the screen to focus on the area they want to see. Although dragging the screen is not an issue in and of itself, the action interrupts users from smoothly transitioning from one part of the screen to another. One aspect of concern in the current study is that the patient identifier was lost when the screen was enlarged; users would have to drag the screen to visualize the patient identifier or minimize the screen. In one study that examined a computerized physician order entry system, the authors reported problems with users selecting the wrong patient because names were close together, font was small, and the patient names were not on all screens (Koppel, Metlay, Cohen, 58  & Abaluck, 2005). Although this is not a device-related problem, the font and icons on the screen has significant impact to patient safety and speaks to the compatibility of the device with the software system.   Some studies suggested that the age of the user was a factor in the ability to read from small displays (Garrett & Klein, 2008; Wager et al., 2010). Darroch and colleagues (2005) studied the effect of age and font size on reading on a PDA. Older participants (61 to 78 years old) found font sizes at 2 and 4 points were uncomfortable or impossible to read. They found no statistical significance in reading performance with font sizes above 6 point; however, participants preferred font sizes in the range of 8 to 12 point range (Darroch et al., 2005). Another study examined the effects of age-related changes in vision and found that older subjects (mean age 68.3 years) who had no eye disease or deficits read as fast as young subjects (mean age 21.6 years) for certain font sizes (Akutsu, Legge, Ross, & Schuebel, 1991). Older subjects who had mild eye defects showed deficits when reading passages with very small or very large fonts and their reading speed declined to about 70% of young adult speeds (Akutsu et al., 1991). The authors suggested that such decline may be related to age from losses in visual contrast sensitivity. These factors should be considered when designing systems for nurses to use at the point of care. 5.3 Data Entry Data entries were predominantly accomplished by the Bluetooth-connected external keyboard, stylus, or finger depending on the type of data required. Item selection, checkbox, and drop-down entries were completed by stylus or finger, while free text entries were input via the external keyboard. Two users in the study commented on the small size of the external keyboard, thereby hindering their typing. Some users were seen typing with two to four fingers but it is 59  unclear whether this was due to the keyboard being too small or the users? normal keyboarding skill. Keyboard shortcuts were also not commonly used during the usability sessions. Similarly, Doran and colleagues (2010) reported that some barriers that hindered nurses? use of the tablet or PDA device were because nurses preferred the desktop and needed a keyboard. Likewise, nurses in another study reported that the use of a selectable, drop-down menu was preferred over free text entries (Garrett & Klein, 2008). This finding may suggest that nurses prefer a larger screen or a full size keyboard for documenting longer narrative entries. Furthermore, the on-screen keyboard was typically not used for data entry. In the event that the on-screen keyboard appeared, it was triggered unintentionally and caused delays in documentation. Although this happened infrequently in the study (n=6), one of the occurrences caused a significant delay (ten minutes) before the user could return back to the task.  Previous sections have alluded to the difficulties of tapping with finger and stylus during clinical documentation. In a study that examined a handheld prescription writing application for physicians, the authors revealed that physicians encountered ergonomic problems with properly using the handheld stylus (Kushniruk et al., 2005). In terms of the selection of a stylus, Ren and Zhou (2011) conducted a study that examined pen width, pen-tip width, and pen length for pointing, steering, and handwriting for different user age groups (i.e., children, young adults, and older adults).  The study results revealed that the dimension of the pen did not affect performance significantly; however, the dimensions did affect user preferences in that each user age group preferred different types of pens. However, this study did not examine users in the 24 to 59 age group, which is the common age group of nurses. The stylus type can undoubtedly influence data entry; however little to no research has specifically studied the type of stylus that is best suited for clinical documentation.  60  There was also a mismatch of how nurses typically saved data on the desktop computer. Nurses typically used the F12 function key on the full keyboard but that key does not exist on the on-screen keyboard or external keyboard. This mismatch caused confusion in that nurses needed to find a new way of saving data. Users should not have to wonder or recall what the differences are when using different devices; this is a violation of one of Nielsen?s heuristics of consistency and standards (Nielsen Norman Group, 2013). When users use the same software application on different devices, there should be no difference in how they go about their work. Though nurses were educated on the new procedure of saving the data, nurses did not trust the new method and often went back to verify if the information had been saved. If the same clinical documentation system were to be used, the basic fundamental functions should remain the same to avoid confusion.  5.4 Login and Logout Problems In the current study, in order for clinicians to login to the PCIS environment, users must successfully manoeuvre through five login screens; that is, unlocking the iPad, entering into Citrix Server, identifying the computer, signing into Windows desktop, and lastly entering the PCIS environment. Nurses had difficulty remembering the login process due to multiple usernames and passwords. Although this is not a device-related usability problem, this issue caused significant delay in clinical documentation. Some studies also reported similar problems with multiple login steps. Dykes et al. (2007) described a two-part login process whereby nurses in their study needed to sign on to the device first and then on to the hospital network, which nurses found to be a time-consuming procedure. Di Pietro and colleagues (2008) also identified problems with the amount of time required to sign onto the device and login. Nurses in another 61  study experienced technical difficulties such as maintaining wireless connection in some parts of the unit and logging in unless it was at the docking station (Carlson et al., 2010).  A five part login process is not conducive to nurses? busy workload. The average time from logging in to entering the first set of vital signs would take an average of eleven minutes. If login problems were compounded with wireless connection difficulties and distractions/interruptions, the use of this type of innovation would not fit into nurses? workflow.  Similarly, logout was also a problem as participants did not know how to completely logout on their own. If participants did not completely log out, the workstation-on-wheel that was used as a proxy for remote access would also lock out, preventing subsequent users from logging in. This incident would require nurses to locate the corresponding workstation and reboot the computer which could cause delays in nurses? activities.  5.5 Recommendations for Clinical Practice If a new device was introduced into a clinical setting, staff should have sufficient education on how to use the functions and features of the device, initial set up or login, and how to use the clinical software application on the new platform. Although the clinical documentation system was the same on the desktop and on the device in this study, some of the functions were different, such as the F12 function and scrolling (i.e., cannot scroll like normal iPad). Nurses should be given time to play and explore the device outside of work time to increase familiarity and practice. Nurses might also benefit from an education session on the basic functions, features, and gestures of the iPad in addition to using Microsoft Windows on the device. Education sessions should be offered outside of work hours when possible to minimize distraction and stress, or with workload assistance and backfill staff to optimize learning and adaptation of new technology.  62  Through verbal and non-verbal communications during the usability study, nurses were noted to be stressed about leaving their patients therefore might have resulted in an attempt to complete the usability scenario as quickly as possible. Future usability testing should also be done outside of work hours if possible or have workload assistance to minimize distraction and stress. 5.6 Recommendations for Organizations This study revealed invaluable knowledge about the organization?s readiness for implementing handheld devices. As mentioned in Chapter 1, handheld devices have the potential to produce ?technology-induced? medical errors, which are errors that are introduced or perpetuated by technology and can have detrimental effects to patient safety (Borycki et al., 2009; Kushniruk, Triola, Borycki, Stein, & Kannry, 2005). Borycki and colleagues (2009) described the complexity of technology-induced errors by emphasizing that although adverse events occur at the provider level, organizational and environmental factors inevitably contribute to these events. The organization must provide structural and technical support in order to facilitate the implementation of handheld devices into the clinical setting. In the current study, the login process was time-consuming and consisted of too many steps. Most nurses had trouble remembering all the actions required to login. User interface with good design indicates that users should be able to recognize rather than recall information from one part of a dialogue to another in order to minimize users? memory load (Nielsen Norman Group, 2013). If handheld devices are to be implemented in the acute care setting, it is imperative that the login procedure becomes simpler and quicker so to not create additional obstacles in nurses? workflow.  63  The clinical documentation system should be assessed for its compatibility with handheld devices. A handheld device that was designed for Microsoft Windows (consistent with the clinical documentation operating system) would likely be better suited since some of the functions and features native to the device could not be probably executed on a different operating system. In addition, patient care software application should ideally have a mobile application component that fits the screen and enables ease of use for clinical documentation. Fitting a desktop screen onto touch screen makes the items and contents difficult to select and read. Mobile applications can be designed to display important or most frequently used tabs to enhance ease of use and efficiency for the end user. Ultimately, when choosing to implement an innovative technology, the hardware device must fit the software application. The healthcare organization needs to have processes in place that will assess and consider the multifaceted factors influencing the implementation of an innovative technology, such as a mobile device. There are ten information technology knowledge areas that needs to be addressed when choosing and implementing information technology solutions: user requirements, infrastructure, conversion, software configuration, workflow, security, interface, testing, cutover, and support (Coplan & Masuda, 2011). User requirements management identifies and organizes the users? needs in order to evaluate options and acquire a solution that will fulfill the objectives and needs of the users and the organization (Coplan & Masuda, 2011). Stakeholder engagement, including clinicians from all disciplines, is imperative in identifying and defining users? needs, and can highly influence the success rate during implementation. In addition, preparing a high-level user requirements plan enables organizational leaders to make an informed decision on whether procuring and implementing a new technology is feasible given the current economic and financial climate of the organization. Healthcare organizations often have to achieve more 64  with less given multiple constraints with time, cost, and resource allocation, therefore, information technology management has to be strategically incorporated into the organization?s plans with consideration to the future state of the users, organization, and patients needs.  This study has revealed the challenges related to the organization?s infrastructure in supporting a stable wireless network connection. Infrastructure management is important when considering an information technology solution. Infrastructure management outlines the physical components of the system, such as the hardware, system capacity and performance requirements, physical and environmental barriers, and changes to the existing facilities (Coplan & Masuda, 2011). The hardware characteristics (e.g., weight, portability, screen size), environmental factors (e.g., wireless network connection in the facility, space to put down the device for charting) and additional factors have to be identified and managed appropriately. During the usability study, the wireless connection was stable and was not an issue. However, nurses had reported that wireless connection was one of the biggest barriers in utilizing the iPads at the bedside. The iPads were unable to connect directly to the hospital?s wireless network, and as a result, the iPads used a workstation-on-wheel as a proxy in order to establish wireless connection. Changes will have to occur in order to accommodate or make room for the new technology, but these decisions cannot be taken lightly without global consideration of other aspects in the project.  This study also highlighted the importance of testing of the device prior to purchasing additional devices or implementing them elsewhere in the organization. There are multiple tests that need to be performed in order to inspect the device (or software) for its readiness for use. Testing has to be incorporated into the planning process, not only as a one-time test, but as an iterative process with regression testing in the project life cycle. After testing is completed with problems fixed, additional testing has to be carried out to ensure the defects are fixed and that the 65  resolution did not create new problems elsewhere in the system. Furthermore, the organization should incorporate the cost of backfilling staff or providing workload assistance into the budget. Backfilling staff or providing workload assistance can minimize nurses? stress while they participate in the testing and also maintain safe patient care with the appropriate number of staff on the unit.  This study also emphasized the significance of stakeholder engagement in change management. Prior to purchasing iPads for the study, nurses on the unit were asked what they thought they need to improve their practice and effectiveness at work. The feedback from the nurses was to try a handheld device for point-of-care use. Leaders should engage and involve stakeholder and end-users in problem-solving as a team, and share the vision of ongoing unit or organizational changes to enhance employee acceptance to change and increase success of the change (Registered Nurses? Association of Ontario, 2013). As nurses are bombarded with constant changes and may feel change fatigue, encouraging and supporting them to be part of the decision-making process can increase buy-in, allow the changes to happen, or feel ownership in participating in making changes in the organization, which in turn can create a more positive work environment (Kramer, Schmalenberg, Maguire, & Brewer, 2009; Registered Nurses? Association of Ontario, 2013; Reineck, 2007). An important piece of change management is also staff education. An education plan that describes the target audience, current levels of informatics literacy, training curriculum, modality, schedule, feedback, logistics, standards, deadline, and resources are required to ensure all staff are successfully trained in the most effective way possible (Coplan & Masuda, 2011). In the same vein, organizational leaders should also be educated on the breadth and depth of implementing an information system or technology. Leaders need to be aware of the time and resources required to achieve successful outcomes not 66  only after the end of the project, but also resources for sustainability and operational needs for continual support.  5.7 Study Limitations  This research was focused on the usability evaluation of an iPad for clinical documentation in the acute care setting. A limitation to usability studies is that they are often undertaken in a controlled, laboratory setting. Due to the nature of the laboratory testing, issues such as portability, wireless connectivity, preferred screen orientation, and workflow could not be studied. Factors that influence the interaction between the users and handheld device including hand manipulation, mobility, contextual awareness, task hierarchy, and visual attention were also not examined in this study (Duh, Tan, & Chen, 2006). Duh and colleagues (2006) found that field testing was able to reveal significantly more usability problems than laboratory testing (Duh et al., 2006). Waterson, Landay, and Matthews (2002) conducted a usability study that allowed users to be in motion and data were collected through a remote web-based tool. The study revealed informative content-related usability problems but device-related problems were harder to capture through this remote testing method (Waterson et al., 2002). Nurses in this usability study took an average of 35 minutes to complete one routine assessment in an undisturbed laboratory setting. The role of distraction, cognitive load, and other factors that would typically play a role in the real world setting could not be explored through usability testing.  Participants who volunteered to be included in the study also might already have preconceived notions and perceptions of using iPads. The sample might not accurately represent the target population. In addition, the layout of the room might have been a limitation as it forced the researcher and participants to be in close proximity. The participants were able to interact 67  with the researcher easily, even though directions were given to attempt the fictitious patient scenario independently. Another limitation is the use of a fictitious user scenario that may not have accurately represented nurses? way of clinical documentation. The fictitious patient scenario may have forced nurses into unfamiliar tasks that are not normal for them. This was evidenced by the myriad of ways each nurse charted for the same task. There were multiple ways to achieve their goals and different places that nurses could enter the information.  A final limitation is that there was a prolonged period of time between their last use and the usability study that may have influenced the users? recall during the login process. Participants might have difficulty in remembering the complex new login procedure after an extended period of time of non-use.  5.8 Recommendations for Future Research This study explored how nurses? used an iPad for clinical documentation in a laboratory setting. Further research is required to focus on different types of handheld devices and their use in the clinical setting through ethnographic observations, field testing, and usability testing. Usability testing of clinical software on tablet devices should also be undertaken to ensure compatibility of software applications and hardware devices.  5.9 Conclusion Handheld devices are becoming commonplace in health care settings. Clinicians are increasingly using handheld devices for clinical use; however, little is known about their efficacy for clinical documentation in hospital setting. The current literature provides an optimistic, yet inconclusive, perspective on the use of health information technology in the hospital setting. This study found that several problems were encountered by nurses using an iPad for documentation 68  in a Windows-based clinical information system. Technological innovations have the potential to enhance patient safety and increase efficiency and effective, but they might also introduce new types of errors. Further usability testing research in this topic area, as well as ethnographic observations and field testing, will help inform practice and decision-making.   69  References Agno, C. F., & Guo, K. L. (2013). Electronic health systems: challenges faced by hospital-based providers. The Health Care Manager, 32(3), 246-252. Akutsu, H., Legge, G. E., Ross, J. A., & Schuebel, K. J. (1991). Psychophysics of Reading - X. Effects of Age-Related Changes in Vision. Journal of Gerontology, 46(6), 325-331. Apple. (2011). iPad User Guide. Retrieved December 7, 2013, from http://manuals.info.apple.com/MANUALS/1000/MA1569/en_US/ipad_2_user_guide.pdf Arning, K., & Ziefle, M. (2009). Effects of age, cognitive, and personal factors on PDA menu navigation performance. Behaviour & Information Technology, 28(3), 251?268. Barnum, C. (2011). Usability Testing Essentials: Ready, Set...Test! Morgan Kaufmann Publishers. Bates, D. W., & Gawande, A. A. (2003). Improving safety with information technology. New England Journal of Medicine, 348(25), 2526?2534. Berner, E. S. (2009). Clinical Decision Support Systems: State of the Art. Rockville, MD: Agency for Healthcare Research and Quality. Bosma, L., Balen, R., Davidson, E., & Jewesson, P. (2003). Point of care use of a personal digital assistant for patient consultation management: experience of an intravenous resource nurse team in a major Canadian teaching hospital. Computers, Informatics, Nursing, 21(4), 179?185. Braddy, C. M., & Blair, J. E. (2005). Colonization of personal digital assistants used in a health care setting. American Journal of Infection Control, 33(4), 230?232. Byrne, M. D. (2013). Mobile Devices in the Perianesthesia Environment. Journal of PeriAnesthesia Nursing, 28(1), 26?30. doi:10.1016/j.jopan.2012.11.006 70  Carlson, E., Catrambone, C., Oder, K., Nauseda, S., Fogg, L., Garcia, B., Brown, F. M., Jr., Johnson, M. E., Johnson, T. J., & Llewellyn, J. (2010). Point-of-care technology supports bedside documentation. Journal of Nursing Administration, 40(9), 360?365. doi:http://dx.doi.org/10.1097/NNA.0b013e3181ee4248 Clark, J. (2012, February 1). Designing for touch- Mobile- Creative Bloq [Web log message]. Retrieved December 7, 2013, from http://www.creativebloq.com/design/designing-touch-2123037 Cockburn, A., Ahlstr?m, D., & Gutwin, C. (2012). Understanding performance in touch selections: Tap, drag and radial pointing drag with finger, stylus and mouse. International Journal of Human-Computer Studies, 70(3), 218?233. doi:10.1016/j.ijhcs.2011.11.002 Coplan, S., & Masuda, D. (2011). Project Management for Healthcare Information Technology. McGraw-Hill. Darroch, I., Goodman, J., Brewster, S. A., & Gray, P. D. G. (2005). The effect of age and font size on reading text on handheld computers. Human-Computer Interaction- INTERACT 2005, 3585, 253-266. doi: 10.1007/11555261_23 Dawson, J., & Kushniruk, A. (2009). Improving the efficiency and accuracy of a tablet PC interface for computerized provider order entry through usability evaluation and provision of data entry strategies. Studies in Health Technology and Informatics, 143, 447?452. Dempsey, P. G., Wogalter, M. S., & Hancock, P. A. (2000). What?s in a name? Using terms from definitions to examine the fundamental foundation of human factors and ergonomics science. Theoretical Issues in Ergonomics Science, 1(1), 3?10. doi:10.1080/146392200308426 71  Di Pietro, T. L., Nguyen, H., & Doran, D. M. (2012). Usability evaluation: Results from ?Evaluation of mobile information technology to improve nurses? access to and use of research evidence.?, Computers Informatics Nursing, 30(8), 440?448. Di Pietro, T., Coburn, G., Dharamshi, N., Doran, D., Mylopoulos, J., Kushniruk, A., Nagle, L., Sidani, S., Tourangeau, A., Laurie-Shaw, B., Lefebre, N., Reid-Haughian, C., Carryer, J., & McArthur, G. (2008). What nurses want: Diffusion of an innovation. Journal of Nursing Care Quality, 23(2), 140?146. doi:10.1097/01.NCQ.0000313763.87019.0a Divall, P., Camosso-Stefinovic, J., & Baker, R. (2013). The use of personal digital assistants in clinical decision making by health care professionals: A systematic review. Health Informatics Journal, 19(1), 16?28. doi:10.1177/1460458212446761 Doran, D. (2009). The emerging role of PDAs in information use and clinical decision making. Evidence Based Nursing, 12(2), 35-38. doi:10.1136/ebn.12.2.35 Doran, D, Haynes, R. B., Kushniruk, A., Straus, S., Grimshaw, J., Hall, L. M., Dubrowski, A., Di Pietro, T., Newman, K., Almost, J., Nguyen, H., Carryer, J., & Jedras, D. (2010). Supporting evidence-based practice for nurses through information technologies. Worldviews on Evidence-Based Nursing, 7(1), 4?15. doi:10.1111/j.1741-6787.2009.00179.x Duh, H., Tan, G., & Chen, V. (2006). Usability evaluation for mobile device. In MobileHCI '06: Proceedings of the 8th conference on human-computer interaction with mobile devices and services (pp. 181?186). ACM. doi:10.1145/1152215.1152254 Dykes, P. C., Carroll, D. L., Benoit, A., & Coakley, A. (2007). A randomized trial of standardized nursing patient assessment using wireless devices. AMIA Annual Symposium Proceedings, 2007, 206?210. 72  Faulkner, L. (2003). Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, & Computers, 35(3), 379?383. doi:10.3758/BF03195514 Fischer, S., Stewart, T. E., Mehta, S., & Wax, R. (2003). Handheld computing in medicine. Journal of the American Medical Informatics Association, 10(2), 139?149. doi:10.1197/jamia.M1180 Fossey, E., Harvey, C., McDermott, F., & Davidson, L. (2002). Understanding and evaluating qualitative research. Australian and New Zealand Journal of Psychiatry, 36(6), 717?732. doi:10.1046/j.1440-1614.2002.01100.x Garrett, B., & Klein, G. (2008). Value of wireless personal digital assistants for practice: perceptions of advanced practice nurses. Journal of Clinical Nursing, 17(16), 2146?2154. doi:10.1111/j.1365-2702.2008.02351.x Hassoun, A., Vellozzi, E. M., & Smith, M. A. (2004). Colonization of personal digital assistants carried by healthcare professionals. Infection Control and Hospital Epidemiology: The Official Journal of the Society of Hospital Epidemiologists of America, 25(11), 1000?1001. Honeybourne, C., Sutton, S., & Ward, L. (2006). Knowledge in the Palm of your hands: PDAs in the clinical setting. Health Information & Libraries Journal, 23(1), 51?59. doi:10.1111/j.1471-1842.2006.00621.x International Organization for Standardization. (1998). ISO 9241-11:1998 (en) Ergonomic requirements for office work with visual display terminals (VDTs)- Part 11: Guidance on usability. Retrieved August 27, 2013, from https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-1:v1:en 73  Johansson, P., Petersson, G., & Nilsson, G. (2011). Experience of using a personal digital assistant in nursing practice - a single case study. Journal of Nursing Management, 19(7), 855?862. doi:10.1111/j.1365-2834.2011.01261.x Karsh, B.-T., Holden, R. J., Alper, S. J., & Or, C. K. L. (2006). A human factors engineering paradigm for patient safety: designing to support the performance of the healthcare professional. Quality & Safety in Health Care, 15(Suppl I), i59-i65. doi: 10.1136/qshc.2005.015974 Koppel, R., Metlay, J. P., Cohen, A., & Abaluck, B. (2005). Role of computerized physician order entry systems in facilitating medication errors. Journal of the American Medical Association, 293(10), 1197-1203. Kramer, M., Schmalenberg, C., Maguire, P., & Brewer, B. B. (2009). Walk the talk: Promoting control of nursing practice and a patient-centered culture. Critical Care Nurse, 29(3), 77-93. doi: 10.4037/ccn2009586 Kushniruk, A., & Borycki, E. (2011). Exploring the relationship between usability and technology-induced error: Unraveling a complex interaction. Studies in Health Technology and Informatics., 166, 48?56. doi:10.3233/978-1-60750-740-6-48 Kushniruk, A. W., Triola, M. M., Borycki, E. M., Stein, B., & Kannry, J. L. (2005). Technology induced error and usability: The relationship between usability problems and prescription errors when using a handheld application. International Journal of Medical Informatics, 74(7), 519?526. doi:10.1016/j.ijmedinf.2005.01.003 Lawrence, M., & Verzani, J. (2012). Programming Graphical User Interfaces in R. CRC Press. 74  Lee, T.-T. (2006). Adopting a personal digital assistant system: application of Lewin?s change theory. Journal of Advanced Nursing, 55(4), 487?496. doi:10.1111/j.1365-2648.2006.03935.x Lee, Y.-H., Hsieh, Y.-C., & Ma, C.-Y. (2011). A model of organizational employees? e-learning systems acceptance. Knowledge-Based Systems, 24(3), 355?366. doi:10.1016/j.knosys.2010.09.005 Lu, Y.-C., Xiao, Y., Sears, A., & Jacko, J. A. (2005). A review and a framework of handheld computer adoption in healthcare. International Journal of Medical Informatics, 74(5), 409-422. doi:10.1016/j.ijmedinf.2005.03.001 McNeil, B. J., Elfrink, V., Beyea, S. C., Pierce, S. T., & Bickford, C. J. (2006). Computer literacy study: Report of qualitative findings. Journal of Professional Nursing: Official Journal of the American Association of Colleges of Nursing, 22(1), 52?59. Nielsen Norman Group. (2012). Usability 101: Introduction to Usability. Retrieved August 27, 2013, from http://www.nngroup.com/articles/usability-101-introduction-to-usability/ Nielsen Norman Group. (2013). 10 Heuristics for User Interface Design: Article by Jakob Nielsen. Retrieved November 4, 2013, from http://www.nngroup.com/articles/ten-usability-heuristics/ Office of the National Coordinator for Health Information Technology. (n.d.). Glossary. Retrieved October 28, 2013, from http://www.healthit.gov/policy-researchers-implementers/glossary Polit, D. F., & Beck, C. T. (2012). Nursing Research: Generating and Assessing Evidence for Nursing Practice (9th ed.). Philadelphia: Wolters Kluwer Health/Lippincott Williams & Wilkins. 75  Registered Nurses? Association of Ontario. (2013). Developing and Sustaining Nursing Leadership Best Practice Guideline. Retrieved December 7, 2013, from http://rnao.ca/sites/rnao-ca/files/LeadershipBPG_Booklet_Web_1.pdf Reineck, C. (2007). Models of Change. Journal of Nursing Administration, 37(9), 388-391. Ren, X., & Zhou, X. (2011). An investigation of the usability of the stylus pen for various age groups on personal digital assistants. Behaviour & Information Technology, 30(6), 709?726. doi:10.1080/01449290903205437 Rubin, J., & Chisnell, D. (2008). Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests (2nd ed.). John Wiley & Sons. Singh, D., Kaur, H., Gardner, W. G., & Treen, L. B. (2002). Bacterial Contamination of Hospital Pagers. Infection Control and Hospital Epidemiology, 23(5), 274?276. doi:10.1086/502048 Stroud, S. D., Smith, C. A., & Erkel, E. A. (2009). Personal digital assistant use by nurse practitioners: A descriptive study. Journal of the American Academy of Nurse Practitioners, 21(1), 31?38. doi:10.1111/j.1745-7599.2008.00368.x The Seattle Times Company. (1992, May 29). Let Newton Assist You, Says Apple. Retrieved from http://community.seattletimes.nwsource.com/archive/?date=19920529&slug=1494354 Thompson, B. W. (2005). The transforming effect of handheld computers on nursing practice. Nursing Administration Quarterly, 29(4), 308-314. Wager, K., Schaffner, M., Foulois, B., Swanson Kazley, A., Parker, C., & Walo, H. (2010). Comparison of the quality and timeliness of vital signs data using three different data-entry devices. Computers, Informatics, Nursing, 28(4), 205?212. doi:http://dx.doi.org/10.1097/NCN.0b013e3181e1df19 76  Waterson, S., Landay, J., & Matthews, T. (2002). In the lab and out in the wild. In CHI ?02: Extended abstracts on human factors in computing systems (pp. 796?797). ACM. doi:10.1145/506443.506602 Wigdor, D., & Wixon, D. (2011). Brave NUI World: Designing Natural User Interfaces for Touch and Gesture. Morgan Kaufmann Publishers. Wilbright, W. A., Haun, D. E., Romano, T., Krutzfeldt, T., Fontenot, C. E., & Nolan, T. E. (2006). Computer use in an urban university hospital: Technology ahead of literacy. Computers, Informatics, Nursing, 24(1), 37?43.   77  Appendices Appendix A   Usability Testing Patient Scenario Instructions: Use your iPad device to enter information based on the following scenario. Only document the items listed. You do not have to document information that is not mentioned in the scenario (i.e., the other items are within normal limits, but we do not need them documented). TASK 1 Step 1: Login to your device. Step 2: Login to PCIS Training application. Username: T5050 Password: Training  Step 3: Read the following scenario and enter the information described. TASK 2 Scenario: A 62 year old woman, Practice, Ubchm, is being transferred to your care from PACU. She arrives post-op at 0830. She had a total abdominal hysterectomy and you just helped transfer her from the stretcher to the bed. ? Enter the following:   ? BP: 104/60 on the right arm, lying down with a Dinamap ? Pulse: 62 regular, manually on radial pulse ? Respiration rate (RR): 12/min, shallow ? O2 saturation: 98% on 3L/min via nasal prongs continuous  ? Temperature: 36.6C orally TASK 3 ? In the ?Routine Assessment/Intervention,? section, document the following:  ? Basic Respiratory Assessment tab:  o Breathing easily and rhythm regular o Symmetrical lung expansion, shallow breaths o Right anterior lower lobe has diminished air entry and fine crackles upon auscultation ? Skin tab:   o You noticed a rash on her right lower quadrant of her abdomen from the pre-operative skin preparation. ? GI tab:  o Patient has nausea. You hang Gravol 50mg IV through a 50cc mini bag 20 minutes after admission.  TASK 4 ? Enter Tubes/Drains: ? She has a 14Fr Foley in situ, draining into a collection bag. ? You empty her urine for 120cc on arrival TASK 5  ? Document her behaviour/mood: ?Ms. Practice becomes very anxious about having a hysterectomy and her cancer diagnoses. Emotional support was provided and time spent to answer questions and provided education.? 78  TASK 6 Ms. Practice?s BP increases and becomes tachycardic. You page the resident named Dr. Freeman and he responds at 0900. You receive stat orders. ? Enter a stat ECG and CBC + Diff TASK 7 ? Sign off PCIS and the device   79  Appendix B   Demographic Questions 1. How many years have you been a nurse?  ________ 2. How many years have you been a nurse in this setting/unit?  ________ 3. What degrees do you have? (select all that are true) ? Diploma ? BSN ? MS-NP ? MSN ? PhD ? Post-masters certificate ? Other Certifications/Relevant education _______________ 4. What is your age group? ? 21-30    ? 31-40 ? 41-50 ? 51-60 ? 61-70 ? >=71 ? Prefer to not say 5. Gender:  ? Female ? Male 6. Personal Computing Devices Used (check all that apply): ? Smartphone (e.g. iPhone, Blackberry, Galaxy) ? Tablet (e.g. iPad, Playbook) ? Desktop computer ? Laptop computer ? Other ___________________ 7. Frequency of computer use (select one): ? Several times/ day ? Once/ day ? Several times/ week ? Several times/ month ? Never ? Other ___________________   

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0165730/manifest

Comment

Related Items