UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Learning through exploration : how children, adults, and older adults interact with a new feature-rich… Mahmud, Shareen 2019

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata


24-ubc_2020_may_mahmud_shareen.pdf [ 4.06MB ]
JSON: 24-1.0384824.json
JSON-LD: 24-1.0384824-ld.json
RDF/XML (Pretty): 24-1.0384824-rdf.xml
RDF/JSON: 24-1.0384824-rdf.json
Turtle: 24-1.0384824-turtle.txt
N-Triples: 24-1.0384824-rdf-ntriples.txt
Original Record: 24-1.0384824-source.json
Full Text

Full Text

/Learning Through Exploration: HowChildren, Adults, and Older Adults Interactwith a New Feature-Rich ApplicationbyShareen MahmudBSc., North South University, 2016A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFMASTER OF SCIENCEinThe Faculty of Graduate and Postdoctoral Studies(Computer Science)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)October 2019c© Shareen Mahmud, 2019The following individuals certify that they have read, and recommend tothe Faculty of Graduate and Postdoctoral Studies for acceptance, the thesisentitled:Learning Through Exploration: How Children, Adults, and Older AdultsInteract with a New Feature-Rich Applicationsubmitted by Shareen Mahmud in partial fulllment of the requirementsfor the degree of Master of Science in Computer ScienceExamining Committee:Joanna McGrenere, Computer ScienceSupervisorReid Holmes, Computer ScienceSupervisory Committee MemberiiAbstractFeature-rich applications such as word processors and spreadsheets are notonly being used by working adults but increasingly also used by children andolder adults in highly varied contexts. Learning these applications is chal-lenging as they offer hundreds of commands throughout the interface. Weinvestigate how newcomers from different age groups explore the user inter-face of a feature-rich application to determine, locate and use relevant fea-tures. We conducted an in-lab observational study with 10 children (10-12),10 adults (20-35) and 10 older adults (60-75) who were all first-time users ofMicrosoft OneNote. Our results illustrate key exploration differences acrossage groups, including that children were careful and performed as efficientlyas the adults, although they struggled to locate contextual menus, whereasolder adults spent a longer time and repeated sequences of failed selections.Further, older adults’ exploration style was negatively influenced by theirpast knowledge of similar applications. We discuss design interventions forHCI to better accommodate the age-related differences in exploration styleswhen users interact with a feature-rich application for the first time.iiiLay SummaryFeature-rich applications (e.g., Microsoft Word) are complex software thatprovide users with hundreds of commands organized under different menus.Users often learn these applications by exploring the features displayed onthe user interface. However, this can be challenging as users must determine,locate, and make use of the relevant features for their tasks. Today, theseapplications are being used by children, adults, and older adults with differentpast experiences with technology. We observed how 10 children (10-12), 10adults (20-35), and 10 older adults (60-75) explored a feature-rich application,Microsoft OneNote, for the first time. We found that children performed aswell as the adults but struggled to locate the right-click menu, whereas olderadults spent a longer time and selected irrelevant features multiple times.Moreover, older adults’ interface exploration was negatively influenced bytheir past software knowledge. We discuss design ideas to accommodate theage-related differences for software newcomers.ivPrefaceThe study described in this thesis was conducted with the approval of theUBC Behavioural Research Ethics Board (certificate number H18-01880).Parts of this thesis appear in a conference paper manuscript that is cur-rently under submission, where I am the first author. I designed and con-ducted the user study, analyzed the data, and identified the results. The ini-tial draft of this paper was written by me which was later revised. Dr. JoannaMcGrenere provided supervisory assistance for this research and helped framethe research questions and the study procedure, as well as edited the fullmanuscript. Postdoctoral Fellow Jessalyn Alvina created the timeline datavisualization and significantly contributed to writing the data analysis andresults sections. Drs. Parmit Chilana and Andrea Bunt provided valuablefeedback throughout this research.vTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . viList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . x1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.1 Users’ Approaches to Learning New Applications . . . . . . . 42.2 Age-Related Differences in Exploratory Learning . . . . . . . . 52.3 Interface Guidelines to Support Age Differences . . . . . . . . 73 User Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83.1 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.2 Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9vi3.3 Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.4 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.1 Video Coding and Event Generation . . . . . . . . . . . . . . 134.2 Timeline Visualization . . . . . . . . . . . . . . . . . . . . . . 164.3 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.4 Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195.1 Participants’ Expertise and Overall Performance . . . . . . . . 205.1.1 Adults have the most experience with different appli-cations, children’s and older adults’ current softwareuse is similar . . . . . . . . . . . . . . . . . . . . . . . 205.1.2 Children are almost as fast as adults, older adults areslower . . . . . . . . . . . . . . . . . . . . . . . . . . . 225.2 Interface Exploration Styles . . . . . . . . . . . . . . . . . . . 255.2.1 Children and older adults face different challenges inlocating relevant features . . . . . . . . . . . . . . . . . 255.2.2 Older adults struggle more than children and adults todetermine the relevant sequence of selections . . . . . . 275.2.3 Adults and older adults rely on past software experi-ence, children rely on real-life experience . . . . . . . . 315.2.4 Tooltips can be helpful but are infrequently used bythe three age groups . . . . . . . . . . . . . . . . . . . 325.3 Facing Breakdowns . . . . . . . . . . . . . . . . . . . . . . . . 335.3.1 Children are quick at recovering from breakdowns, olderadults take time . . . . . . . . . . . . . . . . . . . . . . 335.3.2 Children rely on reading when facing a breakdown,older adults try out random options . . . . . . . . . . . 345.4 Overall Feeling and Help-Seeking Approach . . . . . . . . . . 36vii5.4.1 Children and adults seem fairly content with self-exploration,older adults feel disappointment . . . . . . . . . . . . . 365.4.2 Adults prefer Google search, children and older adultsprefer instructions from people or the built-in help . . . 376 Discussion and Implications for Design . . . . . . . . . . . . . 397 Threats to Validity . . . . . . . . . . . . . . . . . . . . . . . . . 437.1 Internal Validity . . . . . . . . . . . . . . . . . . . . . . . . . . 437.2 Construct Validity . . . . . . . . . . . . . . . . . . . . . . . . 437.3 External Validity . . . . . . . . . . . . . . . . . . . . . . . . . 448 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . 45Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56Call for Participation . . . . . . . . . . . . . . . . . . . . . . . . . . 56Consent Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58Conceptual Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 65Task List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68Expertise Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . 70Self-Efficacy Questionnaire . . . . . . . . . . . . . . . . . . . . . . . 76viiiList of Figures3.1 The interface of Microsoft OneNote . . . . . . . . . . . . . . . 114.1 Final list of events tagged . . . . . . . . . . . . . . . . . . . . 154.2 Snippet of a participant’s video coding file . . . . . . . . . . . 164.3 Example of three timelines, one from each age group . . . . . 175.1 Expertise characteristics of the three age groups . . . . . . . . 215.2 Average of the total duration per age group . . . . . . . . . . 235.3 Median number of selections by age group . . . . . . . . . . . 295.4 Example of the paths taken by O7 and O4 during a Sciencesub-task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30ixAcknowledgementsFirst and foremost, I would like to thank my supervisor, Joanna McGrenerefor believing in me, taking me on as her student, and giving my life a meaning-ful direction. I am grateful for her endless guidance and support throughoutmy journey at UBC. She has always been only a message away and evenon days when she has been swamped with work, she has made her studentsfeel like a priority. Joanna’s dedication and enthusiasm towards her workcontinue to inspire me. I simply could not have asked for a better supervisorwho truly cares about her students’ academic growth and well-being. I thankher for making my graduate school experience exciting and memorable.Next, I would like to thank Jessalyn Alvina for collaborating with meon this project and acting unofficially as co-supervisor. She has been verypatient with me, helping me overcome hurdles along the way and keeping myspirits up. Together, Joanna and Sally have nurtured me into a researcherand their kindness is contagious.I would also like to thank my Software Engineering professor, Reid Holmes,for graciously accepting to be my second reader. I am grateful to all the mem-bers of the e-DAPT and SPG research groups, particularly Parmit Chilanaand Andrea Bunt, for their insights that have improved this project in manyways. I am thankful to NSERC for funding this project through the StrategicPartnership Grant (STPGP 506797 - 1).Finally, I would like to thank all the wonderful friends that I made inVancouver who made this unknown city feel like home. I am forever indebtedxto my parents for their unconditional love and my best friend, ‘Millionaire’,for crossing oceans for me, quite literally.xiChapter 1IntroductionIn this chapter, we provide an overview of the problem space related tofeature-rich applications and exploratory learning. We also state the contri-butions of this work and outline how the rest of the work is organized intodifferent chapters.1.1 Problem DefinitionModern feature-rich applications such as word processors, spreadsheets, and3D modeling packages offer hundreds of commands organized under variousmenus, toolbars, and navigation structures. These applications are powerfuland highly flexible, but they can be overwhelming and difficult to learn [1,2, 3]. One common way for users to learn a new application is to explore thefunctionality displayed on the user interface [4, 5]. However, exploring theinterface of a feature-rich application can be challenging because users mustdetermine which features are needed to accomplish their tasks, understandhow individual features work (in isolation and together), and locate relevantfeatures in the interface [6, 7].The exploratory learnability problem can be particularly acute for new-comers which often include a diverse group of users such as children and older1adults, with varying past software experiences. While prior work has shownthat people often carry out self-directed exploration to learn new software[8, 9], the ways in which different user groups engage in interface explorationhas not been sufficiently investigated in the context of feature-rich applica-tions.Children are increasingly using productivity and learning applicationsin digital classrooms [10, 11]. With greater flexibility in retirement age,older adults (65+) are working longer and learning to use new applicationsfor knowledge work [12, 13, 14]. As young children to older adults nowuse feature-rich applications for education, professional, and entertainmentpurposes, it becomes critical that we understand the challenges new usersfrom different age groups face during self-directed exploration.Previous work on older adults has focused on the challenges that theyface while using new information technologies, for example, being fearfulto explore new applications, having lower confidence levels, and being morenegatively affected by errors than adults [15, 16, 17]. Children, on the otherhand, can be more eager to explore than adults [18, 19]. However, little isknown about the exploration styles of adults, children, and older adults dur-ing graphical user-interface exploration and the effectiveness of those stylesis even less understood. With this issue in mind, our core research questionsare: What are the age-related differences in users’ exploration styles whenusing a feature-rich application for the first time? In particular, how dodifferent user groups determine, locate, and use relevant features within theapplication? How do they deal with performance breakdowns? Characteriz-ing these differences in exploration styles of different age groups could helpdesigners make more inclusive design choices.We conducted a structured observational study with 30 newcomers to afeature-rich application, Microsoft OneNote: 10 children (10-12), 10 adults(20-35), and 10 older adults (60-75). Our goals were to identify and char-acterize exploration styles of the different age groups. We captured both2quantitative data (e.g., logged interactions) and qualitative data (e.g., retro-spective assessments of what made exploration easy or difficult) to provide aholistic characterization of exploration behaviour. Based on both our quan-titative and qualitative findings, we propose design implications for applica-tions seeking to foster efficient exploration for the different age groups.1.2 ContributionsOur work contributes the following:1. The first study, to the best of our knowledge, that simultaneously inves-tigates three age cohorts – children, adults, and older adults’ interfaceexploration styles2. Identification of the challenges that each age group faces when exploringthe interface of a feature-rich application to accomplish a goal3. Identification of the different strategies that each age group uses to dealwith hurdles during interface exploration4. Design implications to support efficient interface exploration for thedifferent age groups.1.3 OverviewIn Chapter 2 we summarize previous work relevant to this thesis. Chapter3 describes our user study. Chapter 4 explains our data analysis techniquesand Chapter 5 outlines our results. In Chapter 6 we discuss our insights andthe implications for design. Chapter 7 acknowledges the threats to validityand Chapter 8 concludes this thesis with directions for future work.3Chapter 2Related WorkIn this chapter, we summarize prior work in understanding how users learnnew applications in general, and the age-related differences when users par-ticularly learn through exploration. We also discuss the design guidelinesthat are available to support different age groups. In each of these sections,we identify the gaps in existing literature that we aim to address with thiswork.2.1 Users’ Approaches to Learning New Ap-plicationsPrior work has investigated the ways in which people learn to use a newapplication. Studies from back in the 1980s and 1990s to the late 2000s haveshown that people often have difficulties learning a new application due todifferent past experiences with technology [9, 4, 5]. Despite the availabilityof online learning resources, built-in help, and manuals, people often tend tobe reluctant to read and prefer to learn via self-directed exploration [8, 20, 9].Although learning an application through interface exploration is a preferredstrategy for many, users make more errors in the early stages of learning that4can cause them to feel confused and frustrated [4]. We were motivated toexplore software learnability of feature-rich applications across different usergroups, and this body of prior work helped us to focus on exploration as themethod of learning.2.2 Age-Related Differences in ExploratoryLearningThere has been work looking at various user groups’ approaches to learningnew applications. In case of adults, multiple studies in the past have shownhow adults learn via exploration [21, 22]. For example, Carol-Ina Trudelswork in the 90s focused on adults’ exploration of a computer-simulated dig-ital watch and showed that poor learners had “negative exploration strate-gies” where they repeated moves that had no effect, did not pay attention tofeedback, and inaccurately assessed what had been learned so far [23]. Simi-larly, another study by Novick et al. [5] investigated the usefulness of trial &error exploration with Microsoft Publisher, and found that adults relied onan interface’s signifiers, which sometimes took them in the wrong direction.When it comes to children, fewer studies have identified their explorationbehaviors with a new application. In the context of using a Tablet, Couse& Chen found that children between the ages of 3 to 6 were able to drawusing a limited version of Microsoft Word when guided by adults. They werepersistent during exploration even when they encountered errors [24]. Morerecent work on children’s use of a feature-rich 3D modeling application hasprovided insights on the barriers that they face when using help resourcesand found that children had difficulties locating the relevant UI elements andstruggled to formulate help queries [25].Significant work has also investigated how older adults learn new appli-cations [13, 26, 27]. Some has explicitly made design recommendations toimprove learning resources and reduce cognitive load [28, 29, 30]. Leung et5al. examined how older adults learned to use a smartphone and found thatparticipants switched between trial & error and reading the manual, and alsobenefited from a task list [16]. Other studies have focused on helping olderadults to use social networking [31, 32], healthcare [33, 34] and smart home[35, 36] applications. The above prior work has looked at different age groupsapproach to learning new applications in separate studies, and in differentcontexts, making it difficult to make direct comparisons.Some studies have considered more than one age cohort when learningsimpler applications, such as Chin & Tat-Fu [15] who pointed out that whenlearning from a link recommendation system, older adults took longer toclick on a link by first deciding if it was relevant or not, whereas adultswere more likely to click and see it. This could be because older adults havebeen shown to have greater computer anxiety than adults [28]. In addition,OBrien et al., found that older adults had more difficulties attributed toinsufficient prior knowledge than adults when learning to use technology intheir everyday lives [37]. The closest work to ours is a study by Chimboet. al [19] that looked at how children and adults explored a simple gaminginterface and found that adults would not make a move that they were unsureabout whereas children were open to trying out different actions to get aheadin the game. Although these studies look at more than one age cohort, noneof them investigate three age cohorts simultaneously and provide insights ontheir varying needs. Furthermore, many of these studies allowed users to seekexternal help (e.g., through manuals, online resources, expert advice) whereasin our study, we specifically isolate age-relate differences when restricted tointerface exploration.62.3 Interface Guidelines to Support Age Dif-ferencesTo help designers build technology for children, Chiasson & Gutwin [38]presented a catalog of design principles by gathering information from pastresearch in HCI, education, and psychology. The design guidelines aimedto support children’s cognitive, physical, social and emotional development.More recently, Soni et. al. [39] found conflicting guidelines in the literaturesuch that they do not equally benefit children in different age categories.Hence, although having guidelines specifically oriented towards children arevaluable, the age ranges in previous work vary considerably and more workis needed to test these guidelines across various age categories.When it comes to designing interfaces for older adults, several studies inthe HCI community [40, 41, 42] have proposed design recommendations toaddress older adults’ physical and cognitive needs as well as their varyingexperiences with technology. Darejeh & Singh [43] conducted a systematicliterature review to extract design principles for older adults. They foundthat novice elderly users benefited from a reduced feature set, descriptivetext, appropriate graphical representation of concepts, and system feedback.In addition, they discussed some similarities in user interface design needsfor older adults, young children and people with mental disorders, and calledfor future work to apply these design principles in practice.Lastly, Neilson, Molich, and Shneiderman [44, 45] have proposed universaluser interface design guidelines in the 1990s that have continued to be revisedover the years [46]. These guidelines highlight the importance of informingusers about the system state and helping them recover from errors. They arenot age-specific but assume that they would be useful for different groups ofusers. In our work, we discuss some of our design recommendations in thecontext of the guidelines above and investigate the ways in which they couldbenefit software newcomers of particular age groups.7Chapter 3User StudyTo address the gap in existing literature and understand how children, adults,and older adults explore the interface of a feature-rich application for the firsttime, we conducted an in-lab observational study. Our goal was not to testany hypothesis, but rather to observe how the participants determine, locateand use relevant features within the interface. We also sought to understandthe strategies that they use to deal with any hurdles during exploration.Lastly, we were also interested in knowing how they felt after exploring theinterface and what their help-seeking preferences would be if not restrictedto interface exploration. Using a mixed-method approach, we collected andanalyzed both quantitative and qualitative data. Our choice of feature-richapplication for the study was Microsoft OneNote. We chose OneNote forthree main reasons: (1) it is a note-taking application that is being used inwork settings, as well as by children in schools [11]; (2) released in 2012, itis relatively newer than other feature-rich productivity applications such asMicrosoft Word and therefore, we hoped to find first-time users of OneNotefrom all three age cohorts; (3) it requires much less domain knowledge thanfeature-rich applications such as Photoshop and the conceptual model is likelyto be easily understood regardless of any interface issues.83.1 ParticipantsWe recruited 10 children (10-12), 10 adults (20-35), and 10 older adults (60-75) with a gender balance. We selected these age ranges in order to minimizethe overlap in the physical, technical, and cognitive abilities of participants.The study was advertised in a local school, university, community centers,and newspapers. All of our 30 participants were first-time users of MicrosoftOneNote, comfortable with using a computer, and free of any motor impair-ments. Each received $20 for their participation.3.2 ApparatusA Microsoft Surface laptop with Windows 10 and Microsoft OneNote applica-tion installed was used for the study. The Tobii Eye Tracker 4C was attachedto the laptop to capture participants’ gaze data. Tobii Ghost was used tocreate a gaze overlay (in the form of a blue bubble) which was recorded by theOBS Studio software as participants’ interacted with OneNote throughoutthe study.3.3 TasksTo observe participants’ exploration styles when using OneNote for the firsttime, we gave them a set of tasks to accomplish. They were asked to imaginethat they were taking two online classes, Art and Science, for which theyhad to maintain a notebook. There were four main tasks: creating a newnotebook, adding notes related to the Art class, adding notes related to theScience class, and sharing the notebook with a friend (Appendix A.4). The‘Create Notebook’ and ‘Share Notebook’ tasks could ideally be completed inone step (‘Minimum Selections’ in Figure 5.3) whereas ‘Art Task’ and ‘Sci-ence Task’ had more complexity. The Art and Science tasks had 7 and 59sub-tasks respectively, consisting of, for example, drawing a flower, findinginformation about polar bears within OneNote, and locking the page with apassword. Figure 3.1(c) shows an example of the final result for the Sciencetask. Creating the notebook was always the first task, and sharing the note-book was always the last task so as to provide participants with a realisticflow. We counterbalanced the Art and Science tasks across participants. Weiteratively refined the Art and Science tasks and ran 6 pilot participants. Thepilot study helped us ensure that the tasks were comprehensible for all agegroups. For example, initially we had a Math task which required partici-pants to add an equation and generate its graph using OneNote. Our first fewpilot participants, all adults, seemed to struggle with it. Hence, we changedthe task as it might have been difficult for children and older adults as well.In addition, we ensured that the tasks included features under different menuareas, and could be completed within an hour.3.4 ProcedureAt the beginning of the study, the participants signed a consent form (Ap-pendix A.2) and filled out an expertise questionnaire followed by a self-efficacy questionnaire [47]. The expertise questionnaire collected their de-mographic information and captured their level of computer and applicationexpertise (Appendix A.5). The self-efficacy questionnaire asked the partici-pants to rate their confidence level of using a new note-taking application un-der a variety of conditions (Appendix A.6). The conditions included havingsomeone to help them get started, having used a similar application before,and being able to use the built-in help. After participants had answered thequestionnaires, the eye tracker was set up. We incorporated the eye trackerafter running the pilot participants where we realized how capturing gazedata could provide us with additional insights on participants’ explorationbehaviour.10Figure 3.1: The interface of Microsoft OneNote. (a) and (b) show an exampleof a child participant C2’s eye gaze (the blue bubble) traversing the featuresunder the ‘Insert’ menu, moving from ‘Pictures’ (a) to ‘Meeting Details’ (b)and finally to ‘Researcher’ (c) while the mouse cursor stays on ‘Insert’. Thefinal result of the Science task is shown in (c).11Next, we introduced participants to the conceptual model of OneNote(Appendix A.3), explaining that it is similar to a physical notebook, whereone can create a notebook and add several sections and pages within thatnotebook. Then, we gave them the list of tasks to carry out. We encouragedthe participants to explore the interface in whatever way they preferred butalso told them that they were not to use external help resources. Since wehad different age groups taking part in the study, we motivated them byacknowledging that the tasks were intended to be slightly difficult and thatthey would be given a hint if they were considerably stuck. We gave theparticipants a hint after 3 minutes of being stuck with a sub-task and theoption to move on to the next sub-task after 5 minutes of not making anyprogress. Each participant had one hour to complete the four main tasks andwas requested to think-aloud during the session. As participants worked onthe tasks, the laptop screen was recorded along with the audio and the gazemovement.Once the tasks were completed, for comparison purposes, participantsfilled out the same self-efficacy questionnaire that they answered at the begin-ning of the study, only this time they were asked to reflect on their experienceafter using OneNote. The sessions concluded with a semi-structured inter-view for 10 minutes where participants provided insights on their experienceof using OneNote for the first time and their exploration styles.12Chapter 4Data AnalysisIn this chapter, we discuss the procedure that we followed to code partici-pants’ screen recordings and how we used a combination of data visualizationand statistics to gather insights. We also outline how we analyzed the inter-view and questionnaire responses.4.1 Video Coding and Event GenerationWe started our data analysis by coding the observational screen recordingsof the participants’ interaction with OneNote. The goal of the video codingwas to generate and tag a set of events related to participants’ explorationactivities. Three members of the research team created and iterated overa codebook, with frequent references to the source videos. The codebookincluded events such as selecting a menu or a feature, performing an actionsuch as typing or drawing, repeating irrelevant selections, etc. We tookinspiration from Trudels [23] coding scheme of classifying exploration eventswhen users’ interacted with a digital watch and modified the event typesbased on the participants’ interaction with OneNote. Figure 4.1 shows ourcodebook with the list of events tagged.We followed the codebook to code the videos using the Boris software13[48]. Each video had over 250 occurrences of the logged events in total.For most of these events in the codebook, we only analyzed their frequencyof occurrence. For five of them that also had a time duration (includingskimming and off-task actions), we analyzed the duration of each occurrenceof the event. We then performed statistical analysis on the data files usingthe REML procedure with SAS JMP, followed by Tukey HSD tests for post-hoc comparisons to understand if the number of occurrences and duration ofthe tagged events were different across the three age groups throughout theirinteraction with OneNote.Figure 4.2 shows a snippet of the data file from a participant’s codedvideo and illustrates how the events from the codebook were logged in thedata files. For example, when the user selects the ‘View’ menu, the eventis tagged as a selection in the ‘Event Tag’ column, and the ‘Event Type’ iscategorized as an off-task selection. In addition, the amount of time that theparticipant takes to make this selection is noted under the ‘Event Duration’column and the ‘Additional Tag’ column is used to provide any further detailson the nature of event such as a non-unique selection.14Figure 4.1: Final list of events tagged. We counted the occurrence of eachevent. An event with (*) also has a duration.15Figure 4.2: Snippet of a participant’s video coding file. Event duration is inseconds.4.2 Timeline VisualizationTo better understand how often each of the tagged events occurred duringparticipants’ interaction with OneNote and the duration of each event, wecreated a timeline visualization using the video coding files. The timelinecategorized the events by the main tasks which helped us find contrastingpatterns when we compared children, adults’, older adults’ timelines, andtheir performance on each task, with one another (Figure 4.3). Using thetimeline, we were able to further identify interesting activities such as howa successful selection could be followed by an incorrectly performed action(off-task action) or how participants could repeat the same sequence of off-task selections (retries) for the same task. We ran another set of statisticalanalysis on the data files, using the same REML procedure as previouslydescribed, to focus on some of these events of interest per task.16Figure 4.3: Example of three timelines, one from each age group: A1 (top), C3 (middle), and O3 (bottom),where Ax, Cx, and Ox refer to participant IDs. The timelines show the occurrence and duration of theevents for the four main tasks. They show, for example, that O3 has longer duration of skim events (greybars) than A1 and C3.174.3 InterviewWe analyzed the interview transcripts in multiple sessions, where membersof the team discussed key themes that could further explain the behaviorsindicated by the logged events. The recurring themes highlighted in children,adults, and older adults responses were related to their overall experience ofusing OneNote, the specific challenges that they faced and their strategiesfor solving those challenges during exploration.4.4 QuestionnaireWe concluded our data analysis by looking at the responses from the threequestionnaires filled out by each participant related to their computer andapplication expertise as well as their before and after self-efficacy ratings.We performed the Kruskal-Wallis H test on the expertise questionnaire alongwith post-hoc comparisons and a combination of the Wilcoxon signed-ranktest and the Kruskal-Wallis H test on the self-efficacy questionnaires.18Chapter 5ResultsBefore moving on to our primary results, we describe our participant’s ex-pertise as analyzed from the demographic data. We then present our mainresults by first providing an overview of children’s, adults’, and older adults’performance of using OneNote for the first time. Next, we look at their in-terface exploration and unpack the differences in: (1) how the participantsin different age groups determine, locate and select relevant features withinthe application and (2) how they deal with performance breakdowns andevaluate the usefulness of their performed actions. Finally, we touch on theiroverall feeling after using OneNote and their general help-seeking preferences.In addition to the different age groups, we considered gender as an indepen-dent variable but found no significant difference in our results (recall thatwe recruited an equal number of male and female participants in each agecategory). Hence, to simplify, we focus on age-related differences.195.1 Participants’ Expertise and Overall Per-formanceIn this section, we describe the differences in our participants’ expertise withvarious devices and applications, and then we give an overview of their overallperformance times when using Microsoft OneNote for the first time.5.1.1 Adults have the most experience with differentapplications, children’s and older adults’ currentsoftware use is similarWe ran the Kruskal-Wallis H test on the expertise questionnaire data (Figure5.1). Participants could rate their years of experience per device from ‘Lessthat 1 year’ to ‘11 years and above’ and their frequency of use per applicationfrom ‘Never’ to ‘Daily.’ We found that there was a significant difference inmeans between the age groups when it came to their number of years of expe-rience with desktops (H = 14.561, p < .001), laptops (H = 10.292, p < .006),and smartphones (H = 8.810, p < .012). A post-hoc test to check pairwisecomparisons showed that children had significantly fewer years of desktop ex-perience than adults (p = .004) and older adults (p = .002) whereas adults’and older adults’ years of desktop experience was not significantly different.However, when it came to the years of laptop and smartphone experience,adults had significantly more experience than both children and older adults(p = .011 and p = .024 for laptops, p = .048 and p = .019 for smartphones)whereas children’s and older adults’ years of laptop and smartphone experi-ence was not different.20Figure 5.1: Expertise characteristics of the three age groups (median). Acategory with (*) indicates signicant difference between age groups.Next, we looked at participants’ frequency of use for different categoriesof applications over the 6 months prior to taking part in the study. Wefound a significant difference in their current frequency of use for word pro-cessing (H = 10.191, p < .006), spreadsheet (H = 14.280, p < .001), email(H = 18.347, p < .000) and presentation (H = 6.975, p < .031) applications.Pairwise comparisons revealed that adults used word processing, spreadsheetsand presentation applications significantly more often than both children andolder adults (p = .02 and p = .015 for word processors, p = .001 and p = .044for spreadsheets, p = .001 and p = .002 for presentation) whereas children’sand older adults’ current usage of these applications was not significantlydifferent. Older adults only used email applications significantly more oftenthan children (p = .001). Also note that 7 of our older adults were stillworking (4 part-time, 3 full-time) and 3 were retired.To summarize, our adults had the most experience with technology, andalthough our older adults had more years of desktop experience than children,their current usage of computer applications was not very different fromchildren.215.1.2 Children are almost as fast as adults, older adultsare slowerThe overall task completion times were not significantly different betweenchildren and adults, whereas older adults took a significantly longer timethan the other two age groups (main effect: F(2,27) = 22.1201, p < .001 andpairwise comparisons: p = .001 for older adults - adults, p = .001 for olderadults - children). Children and adults completed the four main tasks withall the sub-tasks whereas 4 out of the 10 older adults gave up on at least 1sub-task. On average, the total time to complete all four tasks (excludingthe ‘Read Instruction’ and ‘Idle’ times) was 14.88 minutes for adults, 18.29minutes for children, and 32.24 minutes for older adults.22Figure 5.2: Average of the total duration per age group categorized by the five tagged events that had aduration23The aggregated logged data clarifies which activities required the mosttime (Figure 5.2). It shows that children spent a bit longer than adultsskimming the interface (1.5x more) and performing successful actions (1.6xmore) such as drawing a flower. Surprisingly, they spent less time performingoff-task actions that were not sufficient to succeed in the task, such as tryingto insert a file from the computer instead of adding an online picture (1.4xless).Older adults were the slowest overall, taking 2.2x longer than adults and1.8x longer than children. This is partly expected as prior research has shownthat, in general, older adults take more time to complete movement tasks andtake more pauses [40, 41]. Although Figure 5.2 shows that older adults tooka longer time than children and adults in many of the categories, the biggestcontributors were the time they spent performing off-task actions (2.4x slowerthan adults and 3.6x slower than children) and skimming the interface (3.1xslower than adults and 2.1x slower than children).To better understand the skimming and off-task action activities, we con-sidered both the number of occurrences and the duration per event. We founda significant main effect of the occurrence of skimming and off-task actionevents (F(2,27) = 22.0034, p < 0.0001 for skimming and F(2,27) = 5.0963, p <0.05 for off-task actions). Pairwise comparisons showed that older adultsskimmed the interface and performed off-task actions significantly more of-ten than both children (p = .004 for skimming, p = .006 for off-task actions)and adults (p = .001 for skimming, p = .008 for off-task actions). On aver-age, the older adults skimmed 53.1 times and performed off-task actions 17times during the whole study, while adults and children only skimmed 28.2and 31.2 times respectively, and both performed off-task actions 9.2 times.In the next section, we elaborate on how children’s and (especially) theolder adults’ interface exploration styles resulted in their longer task comple-tion times.245.2 Interface Exploration StylesIn this section, we describe the differences in the interface exploration stylesof the three age groups in terms of how they determine, locate, and makeuse of relevant features for the tasks. We also discuss the effect of knowledgetransfer on participants’ performance.5.2.1 Children and older adults face different challengesin locating relevant featuresAdults were significantly quicker at making successful selections than olderadults (main effect: F(2,27) = 5.0797, p < 0.01 and pairwise comparisons:p = .018), whereas children were in between. The average of the medianduration per episode of successful selections was 4.2, 4.9, and 6.5 seconds foradults, children, and older adults respectively.Older adults particularly struggled to locate the relevant features becausethey did not sufficiently investigate the features in different menus. If weconsider the number of non-unique selections that were made by the threeage groups, where they selected an option that they had tried before, olderadults made significantly more non-unique selections than the other two agegroups (main effect: F(2,27) = 3.6534, p = 0.04 and pairwise comparisons:p = .002 for older adults - children, p = .015 for older adults - adults).The median percentages of non-unique selections were 55.5% for older adultscompared to 43.9% and 47.5% for adults and children respectively. This isperhaps because older adults had the tendency to only select the sub-menuwhere they expected the feature to be, and dismissed other potential menus.For example, O8 struggled to find the ‘Share’ icon that was located at thetop-right corner menu of the application. When asked, O8 said “I thoughtthose things won’t add value. When you write on paper you write from left toright. The headings are always on the left. I assumed the important thingsto be on the left side.” O6 expressed a similar thought regarding where he25expected certain features to be located: “Why are there these things [createnotebook/section] at the bottom? I would expect them to be at the top, maybeunder View.”Children faced different challenges when locating the relevant features.Even though their task completion times were similar to that of adults, theysometimes felt overwhelmed by the number of different menus and features.For example, C8 said “It was kind of complicated. Just like they need to makeit more simple in a way. It’s confusing why some options are down here [asub-menu] and some up there [top left corner menu].” The majority of chil-dren were looking at the options at the top menu but they sometimes didnot initially realize that there were more options outside ‘Home’. For exam-ple, C10 said “I didn’t know that if you pressed it [Insert] it would give moreoptions inside it. Once I looked outside Home it became obvious” In addi-tion, children particularly had difficulties locating features in the right-click(contextual) menu. The total number of right-click events was significantlylower for children (median 3 times during the whole study), than adults (10times) and older adults (8.5 times), (main effect: F(2,27) = 6.4952, p = 0.0053and pairwise comparisons: p = .01 and p = .01) . Only 3 out of 10 childrenreported that they knew that they could right-click for more options. C8 said“I actually don’t use the mouse usually so I didn’t know about right-clicking.”In addition, C10 mentioned how she discovered the right-click menu acciden-tally ‘I meant to click on it but I pressed both [right and left click] and it[right click menu] showed up.”Adults, being more frequent users of similar software applications, mademore efficient use of the different menus resulting in their quicker successfulselection times. Contrary to children and older adults, the majority of theadults found the features easy to locate. For example, A4 said “I think they[the features] were relatively simple to discover. There were obvious featureslike the Draw menu. You probably saw me right-clicking several times. If itdidn’t have what I was looking for then I would look somewhere else.” Similar26to A4, A6 also mentioned how he would just explore another sub-menu if hecould not find the feature where he expected it to be. “I would firstly go tothe menu that looks most relevant and scan from left to right. If this is not theright one, go to the next toolbar.” Therefore, overall, adults often found therelevant features because they were comfortable exploring and knowledgeableabout different menus, including the right-click menu.5.2.2 Older adults struggle more than children andadults to determine the relevant sequence of se-lectionsOlder adults missed selecting a significantly higher number of correct featuresthan children and adults, despite being on the right sub-menu (main effect:F(2,27) = 3.5099, p = 0.04 and pairwise comparisons: p = .004 for olderadults - children, p = .012 for older adults - adults). The median number ofmissed features was 15.5 for older adults, 9 for children and 7.5 for adults.This indicates that even though older adults were sometimes on the righttrack, they got confused about the features that were relevant to completethe task. For example, during the Science task, O6 had already determinedthe relevant sequence of selections and opened the ‘Insert’ menu but missedto click on the ‘Researcher’ feature which was the second last option fromthe right. In the semi-structured interview, he explained that he thoughtthat the feature would be something like a ‘magnifying glass symbol’ underthe ‘Insert’ menu. Similarly, there was O10 who had 32 instances of missedfeatures, the highest among all the participants. When asked about why shemissed some of the features she said “Maybe I am too used to looking on myleft-hand side than look at my right-hand side to search for the commands.If I got used to it I think I would try to look at both sides.”Furthermore, some older adults tried to determine the selection sequenceby searching for words similar to the task instructions on the interface. For27example, O2 mentioned “I didn’t have any plan on how to approach it. Iwould just rely on the instruction page and then go to see if I can find asimilar term. It all depends on the information available here [the task list]and here [the interface].” This approach did not always work for them asthe task list would not have keywords that could directly be mapped to thefeature set.Aside from missing the relevant features, older adults also performedoff-task selections more often than adults and children (Figure 5.3), a sta-tistically significant difference for the Science task (main effect: F(2,27) =4.8991, p = .002 and pairwise comparisons: p = .001 for older adults - adults,p = .002 for older adults - children). This could be because the sub-tasksunder Science were more challenging as the participants had to make theconnection between the feature ‘Researcher’ under ‘Insert’ menu to ‘insert-ing information about polar bears’ and the ‘Password Protect’ feature undera right-click menu to ‘locking the Science section with a password’.When we further looked at the nature of the off-task selections made bythe older adults in the Science task, we found that older adults had a signif-icantly higher number of cycles, than the other two age groups (main effect:F(2,27) = 5.9271, p < .001 and pairwise comparison: p = .01 for older adults- children, p = .04 for older adults - adults). 7 out of the 10 older adultshad at least one cycle where they repeated the same sequence of off-taskselections or a retry of the same feature, both of which sometimes lead themto perform off-task actions. Figure 5.4 shows the deviation from the shortestpath to complete a sub-task under Science by O7 and O4. O7 eventually didcomplete the sub-task whereas O7 gave up in the end without finding the‘Researcher’ feature. The struggle to determine the relevant sequence selec-tion sometimes caused frustration among older adults where they required ahint to proceed in the right direction.28Figure 5.3: Median number of selections by age group for each of the four tasks as well as the minimumnumber of selections required for each. Older adults performed more off-task selections during ‘ScienceTask.’29Figure 5.4: Example of the paths taken by O7 and O4 during a Science sub-task. Each of them deviatedfrom the shortest path and had off-task selection cycles and retries. O7 eventually got on the right pathand completed the sub-task whereas O4 gave up.305.2.3 Adults and older adults rely on past softwareexperience, children rely on real-life experienceThe adult participants reported that they often guessed the relevant featuresbased on their past experience with similar software, e.g., Microsoft Office, todetermine the possible menus and features. For example, talking about herdiscovery of relevant features, A8 said “It’s pretty similar to Word. Thereare more different tools to explore here. I don’t know if Word has the optionto share notes but it’s pretty cool. I did a lot of inference from my priorknowledge to guess and check.” This suggests positive knowledge transfer.Although rarely occurring, we also observed a negative knowledge transferfor adults, when a participant made a wrong assumption based on theirknowledge of previous software. For example, A2 had gone to the ‘Home’menu to create a new notebook mentioning that he thought he should goto the menu similar to the ‘File’ menu of Microsoft Word to create a newfile. This may potentially hinder efficient exploration if the user incorrectlynarrows down the scope of exploration.Similar to the adults, the older adults were also affected by their past ex-perience of using similar applications but for them, negative transfer learningseemed to dominate. For example, when O9 had to add the current date tohis notebook, instead of simply typing it in he was looking for a functioncalled ‘Date’ as Microsoft Word would have it. He said “It doesn’t seemlogical that I can’t press on some function to insert a date like in Word.”O1 also expressed similar views regarding her confusion when she tried torelate OneNote with Microsoft Word: “I was not too sure how to go backHome. It seems a little bit different from what I mostly work on. I mostlywork on Word and whenever I am not too sure I will just go back to Home,this [OneNote] doesn’t seem to work like that. It has to depend on my luck.”In contrast, children seemed to rely more on their real-life experiences asthey had less computer experience, which often had a positive effect on theirexploration. For example, explaining how he found the ‘Researcher’ option,31C2 mentioned “When the teacher gives you homework to do and you dontknow much about it, she says you do research. So I thought Researcher wassomething that could research for you.” Similar to C2, C1 also expressed howhe mapped the features to real-life examples “Like, if you look at the Ruleroption, I think about the ruler straight away. You can measure. Math meansyou can do plus, minus, division.”5.2.4 Tooltips can be helpful but are infrequently usedby the three age groupsOne way for the application to communicate what a feature does is throughthe tooltip. All three age groups made significantly more number of selectionswithout first reading the tooltip. On average, only 9.2%, 15.8%, and 21.7%(for adults, children, and older adults respectively) of the total selectionsfor each age group were tooltip selections. This is particularly surprisingin the case of older adults because they had spent significantly more timeskimming the interface, and yet they did not leverage the tooltip that often.This indicates that even when older adults were skimming they were justlooking around for features that might help them to proceed.Next, if we look at the nature of the tooltip selections that were madeby each age group, older adults made significantly more successful selectionsthan adults (main effect: F(2,27) = 5.0797, p < .01 and pairwise comparisons:p = .03) when they did read the tooltip, whereas children were in between.A median of 14.6% of the successful selections made by older adults weretooltip selections, compared to 10.3% for children and only 4% for adults.This may suggest that tooltips were more impactful for older adults to helpthem map the icon and terminology associated with a feature to its function,and they might have struggled less to determine the relevant features if,during skimming, they had made more use of the tooltip.325.3 Facing BreakdownsIn this section, we discuss the hurdles that participants faced during explo-ration and the strategies that they used to overcome those.5.3.1 Children are quick at recovering from breakdowns,older adults take timeChildren had the shortest average total duration of off-task actions, only 1.1minutes, compared to 1.7 and 4.1 minutes for adults and older adults (Figure5.2). This suggests that children were able to understand the system feedbackand address the outcome of their actions. Even when they made an off-taskselection, they were quick to detect it and move on to another option. Forexample, in the Art task, a lot of participants did not realize right away thatthey had to activate the drawing mode by clicking a ‘Hand’ icon first beforedrawing. Talking about her experience of being able to quickly recover froman off-task action, such as trying to drag the pen tool icon to draw which wasnot a supported interaction, C3 said “I was dragging it [pen] as I was reallytrying to draw. Then I looked again and saw the hand and hovered over thatto see what it did.”Similar to children, adults were also relatively good at understandingthe feedback from the system to determine whether things worked as theyexpected or not. Some adults also went through the breakdown of not clickingon the ‘Hand’ icon and only selecting a pen. However, they were also ableto quickly realize that they had to find a workaround. A2 mentioned,“...because it wasn’t working, this is like the obvious [Hand] icon, like the finger,just that icon is very readable.”Older adults, on the other hand, spent a significantly longer time carryingout various off-task actions than children and adults (main effect: F(2,27) =9.1356, p < .01 and pairwise comparisons: p = .04, p = .04) and were alsosignificantly more unsure of their actions (main effect: F(2,27) = 9.5846, p <33.01 and pairwise comparisons: p = .002, p = .003 for older adults - childrenand older adults - adults). Reflecting on his experience with the drawingsub-task, O3 mentioned how he did not know he had to select both thepen tool and the ‘Hand’ icon: “When you asked me to draw, I thought itwas challenging, I couldn’t think of the fact that I had to choose a color[pen]. If you weren’t here [to give a hint] I would either have to go to theweb or call somebody to ask.” In addition, many of our older adults haddifficulties understanding system feedback. For example, some selectionstrigger interface changes such as a pop-up menu or a mode-switch (e.g.,clicking on the ‘Hand’ icon only activates the drawing mode), rather than achange to the data. Older adults often did not realize this difference, andtended to use the ‘Undo’ button hoping to undo a selection’s impact on theinterface. For example, in Figure 5.4, O7 is seen selecting ‘Undo’ hopingto undo the effect of the ‘Search Notebook’ selection that had impacted theinterface by opening up a pop-up menu but she did not understand that shewas removing her content instead. O2, O4, O6, O9 and O10 also faced thisbreakdown where they were unable to understand the outcome of selectingthe ‘Undo’ button and accidentally removed their data.5.3.2 Children rely on reading when facing a break-down, older adults try out random optionsChildren were careful in making selections and did not want to do somethingwrong. Their fear of trying out wrong options could possibly be due to pastexperiences, such as C10 who mentioned “I have used the Word documentbefore, I accidentally highlighted the text. Then deleted it. I tried to solveit myself first and then I was at school, so I asked my friends and then theteacher.” When facing a breakdown such as not being able to locate thecorrect feature or being stuck with an off-task action, 7 out of 10 childrensaid that relied on the text labels to make an educated guess about thecorrect feature. Referring to how text supported him during breakdowns, C834said “Well, I would just kind of go into ‘Insert’, ‘Draw’, ‘Home’, ‘View’ andread the options. Then work off from there.” Figure 3.1 (a) and (b) furthershow an example of C2, who opened the ‘Insert’ menu and scanned the textlabels systematically, as indicated by her eye-gaze, before moving the cursorto making a selection. Besides reading the text label, 2 out of the 10 childrenmentioned that they found the tooltip to be useful such as C10 who stated“When I was in Home, I didn’t know what they meant so I would go to theoption and then stay on it. Then I could read what it was.”Adults and older adults, on the other hand, resorted to trial & error whenthey got stuck. However, for older adults, this approach was less effectiveas they would often click on random features just to see if they would workonce their initial sequence of actions was unsuccessful. Overall, 8 out of10 older adults mentioned trial & error. For example, talking about herapproach to deal with breakdowns O8 said “Because I often couldn’t find alogical sequence, I was clicking on everything one after the other to see ifsomething fit.” Similar to O8, O5 expressed “It was more hit and miss. Itwasn’t that straightforward. I was trying out things many times to see if itworks.” This further explains why older adults had a significantly highernumber of off-task selections and cycles as previously discussed.Although adults also adopted the trial & error approach when they faceda breakdown, they seemed to have better deduction strategies instead ofselecting options in a random manner. For example, A6 said “I think myapproach is read, try it, click around. I would firstly go to the menu thatlooks most relevant. Mostly just try it. If this is not the right one, go to thenext toolbar. It won’t be a big deal to click on a wrong button.” A3, A4, A5and A9 also expressed similar ideas. A9 described her approach as “Reading,seeing, trying. I wouldn’t just choose random ones. I went with the first onethat made sense if that did not work, I will look for another one.” Therefore,it suggests that even when adults were doing trial & error, they were doingit in a targeted manner.355.4 Overall Feeling and Help-Seeking ApproachIn this section, we describe the overall feeling of the participants after ex-ploring OneNote for the first time. We also touch on the differences in theirpreferred help-seeking approach.5.4.1 Children and adults seem fairly content with self-exploration, older adults feel disappointmentTo understand how the three age groups felt about using a new applicationand if it affected their confidence levels, we analyzed their self-efficacy ques-tionnaires before and after the tasks were performed. Although the within-group analysis showed no significant change in the confidence levels withineach group before and after they had used OneNote, our between-group anal-ysis revealed another story.Older adults started the tasks with significantly lower confidence levelsthan adults (p = .035) and children (p = .039), and even after interactingwith OneNote, their confidence levels stayed significantly lower (p = .008 forolder adults - adults and p = .042 for older adults - children). This indicatesthat for older adults, their experience of using OneNote did not improvetheir initial low confidence levels. If anything, it caused frustrations, as O7mentioned “It’s a headache. You shouldn’t have to touch this touch that. It’stoo much of clicking here and clicking there. If I want to do something quicklyI don’t want to click so much. I just wouldn’t use it.” Similarly, O1, O2, O8,O9, and O10 also reported that they felt unhappy with their experience. O3further mentioned that he had expected the interface to be more intuitive“It wasn’t as intuitive and as easy as I thought it would be. I guess with anynew application I need practice to get used to it.”In contrast, children and adults maintained a high confidence level bothbefore and after they had used OneNote. It seems that adults had expectedOneNote to be more complicated, like some of the other applications they36had used in the past. For example, A5 mentioned “It’s simple and I guessthe design, the way it looked, it’s not as complicated as I thought. I expectedsomething like, you know, Microsoft Excel.” Overall, 9 out of 10 adult par-ticipants expressed similar views of finding OneNote easy to use, with theexception of A2 who found it difficult: “I found it a bit hard to use. It’s notintuitive at all. Like there are some conventions where you would expect theoptions to be and I don’t see them there.”Similar to adults, children also expressed satisfaction with their experi-ence of using OneNote, although they mentioned that the interface could befurther simplified. Such as C1 who said “I am happy but what I dislike isthere is all this [sub menu options]. Then you have these [main menu op-tions]. You don’t know where what is.” In addition, C10 mentioned how heunderstood the interface better as he progressed in the tasks: “For this one[Science], it was a bit difficult. But when I did Art I got used to it.”5.4.2 Adults prefer Google search, children and olderadults prefer instructions from people or the built-in helpIn the study, we restricted the participants’ help-seeking methods to built-inhelp. However, 7 out of 10 adults mentioned in the semi-structured interviewthat they would rather use a search engine (e.g., Google Search) or video tu-torials (e.g., YouTube) if allowed. A6 and A7 also considered asking someoneif they thought the task was too technical.Children had mixed responses when it came to their help-seeking prefer-ence. From the interview responses, we found that most of them (6 out of10) preferred to solve the problem on their own first, and then ask a parent,a teacher or even google home for help afterward. For example, C6 said“Usually, I would try and figure it out more on my own. Sometimes I wouldsearch it up on the internet or ask my Dad.”37Older adults had a preference for using the built-in help. 5 out of 10older adult participants mentioned the built-in help but only 2 of them hadsuccessfully used it during the tasks. It is interesting to note that 3 outof the 5 older adults who mentioned built-in help did not realize that itwas available in OneNote. Talking about his preferred help-seeking methodO6 said “Help button, which I did not see here. I would usually definitelyuse that.” In addition to using the built-in help, O3 mentioned other helpresources that he would use: “So yes, if there is a built-in help typically Iwould try to use that. If there isn’t I would go to the web. Or if there arepeople around me that have used the application, if someone is in the nextroom, I will open the door to ask them.”The insights from the interview responses are further confirmed if we alsolook at the self-efficacy questionnaire responses for the help-seeking approachthat participants felt the most confident with. The median ratings of adultswere the highest for google search (9 on a scale of 10) and the lowest for thecondition where they had not used any similar application before (5.75). Forchildren and older adults, their median ratings were the highest for havingsomeone show them how to do the tasks (9.5 and 7 respectively) and thelowest was if there was no one around for help (5.75 and 3.5 respectively).38Chapter 6Discussion and Implications forDesignWe reflect on our key findings and discuss their implications to foster efficientexploration for children, adults and older adults. Where possible, we providespecific implications for design.We were surprised to see that children performed almost as well as theadults when using the feature-rich application, Microsoft OneNote, for thefirst time. Prior work on children’s use of problem-solving software hadindicated that children tend to feel lost in different parts of an applicationand tend to try out many different actions to get ahead [25, 19]. Yet, we sawthat even when children were exploring the application, they seemed carefulto avoid making mistakes. They often read the text labels and sometimesthe tooltips for guidance. Even when they had a breakdown, i.e. carryingout off-task actions, they were quick to detect it and recover. In contrast,older adults fell into the active user paradox [4] where they resorted to lesssystematic trial & error strategies without relying on the tooltip, once theirinitial sequence of actions had failed. This confirms prior work on older adultsbeing more negatively affected by errors [15, 16, 17], which then impacts theirinitial exploration strategy and causes them to struggle.39While it is not surprising that older adults had slower task completiontimes – there are natural declines in cognitive and motor abilities due toaging [49], as well as documented fears of exploring a new application [15] –our study further revealed additional factors that contributed to their longertask completion times. Older adults had multiple selection cycles and retriesduring exploration where they deviated from the shortest path and selectedthe same sequence of irrelevant features, that often led them to performunnecessary actions and consequently slowed them down (Figure 5.4). Adultsand children, on the other hand, had significantly fewer such cycles in theirinteraction data, instead moving on to trying out different sets of options.This could be an indication of short-term memory loss where older adultsforget the sequence of actions that they have carried out or where they doremember but are simply unsure of whether they have performed the actionscorrectly [42].Implication for Design: Detect selection cycles and offer support. Thesystem could detect cycles and retries in users’ selections and offer support.A simple possibility would be to suggest the use of built-in help as someof our older adults did not notice that there was a built-in help option. Inaddition, based on the user’s interaction, the system could invoke a heatmapshowing the recent cycles and retries of selected options, similar to Patina[50], and encourage the users to better understand the use of those features byusing the tooltip. Moreover, seeing a visual representation of their repeatedselections could motivate users to reflect on their previous actions and preventthem from repeating the wrong path.Another surprising finding was that the impact of knowledge transfer onexploration style was different for children, adults, and older adults. Adultswere the most experienced with recent technologies, which generally seemedto help them perform well. Although our older adults reported comparablepast experience with similar applications, they had not used these regularlyin the past six months. This might have caused a mismatch in their mental40models for how they expected the application to behave and the way thatit is designed today. Hence, although they spent a long time skimming theinterface, they were making more non-unique selections. This is consistentwith O’Brien’s work on understanding the effect of older adults’ prior knowl-edge on interactions with technology in general [37], where our observationalstudy provides additional insights on how having past knowledge of similarapplications more specifically does not necessarily make self exploration ef-ficient. Children, on the other hand, had used fewer computer applicationsthan our older adults but were still as confident as our adults. They weremost familiar with using tablets that offer relatively simpler applicationsthan desktop computers. Their lack of experience with personal computersand pointing devices (e.g., a mouse) may hinder their discovery of interac-tions such as the right-click menu, which has the metaphor of right-clickinga mouse, something more foreign to them.Implication for Design: Support feature discovery and skimming by ap-propriately revealing signifiers for hidden menus, including drop-downs andthe right-click menu. The interface could detect the user’s eye-gaze on thescreen together with skimming behaviour and then subtly provide signifiers.For example, the interface could highlight a ‘Reveal’ button similar to Ex-poseHK [51], which the user could click on to discover hidden menus. Thesesignifiers could be ignored or acted upon, without unnecessarily clutteringthe interface for all users. In addition, this might inspire users to explore themenu regions they had not previously paid attention to.Lastly, it was also surprising for us to find that older adults particularlystruggled to understand the scope of the ‘Undo’ mechanism that is universallyused today. They often tried to use ‘Undo’ hoping to undo a selection thathad impacted the interface, whereas the current ‘Undo’ mechanism only actson selections that involve operations on data. Hence, some of the older adultsended up selecting ‘Undo’ incorrectly and accidentally removing importantcontent.41Implication for Design: Provide feedback of user actions for both changesto the data/content and to the interface, and enable users to undo either typeof action. Although recent work has widely explored ‘Selective Undo’ , whereusers can undo specific operations instead of backtracking in a linear manner[52, 53], we recommend expanding the ‘Undo’ scope by distinguishing be-tween undoing an operation that affects the data/content or merely undoinga selection. For example, the system could offer a feature such as ‘My PastActions’ that could show the user a list of the features that they had selectedalong with whether a feature caused a change to their data/content or justto the interface. The user could then hover on the list item to undo the effectof certain selections.We envision these design recommendations to be particularly useful forboth children and older adults. At the same time, they would not get in theway of adults who do not need the extra assistance, as the support wouldbe triggered based on the user’s interactions. The adults in our study didnot seem to struggle very much. We do not interpret this as inconsistentwith prior work, but rather complementary – OneNote with approximately90 features is not likely as feature-rich as applications such as Photoshop thathas over 200 features 1, with which adults have been shown to have difficulty[20, 54]. In addition, the nature of our tasks might not have been as complexfor the adults as those studied before.1We provide an approximation of the number of features by counting the differentmenus and the options in each of them.42Chapter 7Threats to ValidityIn this chapter we acknowledge the limitations of our work by discussing thethreats to internal, construct, and external validity.7.1 Internal ValidityIn our study, we did not control for expertise which could have resulted insome of the differences that we saw in participants’ overall performance withOneNote. It is possible that older adults who have used various applicationsmore recently than the older adults who took part in our study, may havestruggled less. In addition, we recruited all of our children from one schoolwhere they may have been intellectually compatible with one another, moreso than had they been recruited from different schools.7.2 Construct ValidityThere is no universally accepted definition of exploration. Although we spenta significant amount of time iterating over the codebook and finalizing thelist of exploration events, other researchers may be able to look at the videoswith a different perspective and be able to identify codes that we may have43missed. The same applies for the interview transcripts where there may beadditional themes regarding participants’ experience of using OneNote forthe first time.7.3 External ValidityWe investigated participants’ performance with only one type of applica-tion, Microsoft OneNote, which may not have been equally complex for allthree age groups. We also looked at their exploration styles with a list oftasks which may have influenced the way in which they navigated the in-terface. Furthermore, we restricted the participants from taking externalhelp and made them focus on self-exploration whereas outside the study,it is likely that participants would use a combination of help resources andself-exploration to learn a new application.44Chapter 8Conclusion and Future WorkWhen learning a feature-rich application for the first time, users often exploredifferent menus and features to accomplish their desired tasks. Today, theseapplications are being used by children, adults and older adults alike. Ourstudy contributes insights into the interface exploration styles of the threeage groups, the challenges that they face and the strategies that they use todeal with breakdowns. We found, among other things, that children explorethe interface carefully but struggle to locate contextual menus (because oflack of mouse exposure), whereas older adults have difficulties determiningrelevant sequence of features and repeat failed selections.In terms of the study procedure and its threats to validity as discussedin the previous chapter, future work could expand the choice of applicationwith various degrees of complexity, beyond productivity, and investigate theeffectiveness of the design implications with a broader sample size. In ad-dition, future work could consider supporting other means of help-seekingalong with a task-free approach and inspect any additional events that couldfurther characterize participants’ exploration styles.Nevertheless, our work is an important step towards understanding thediversity in users’ approaches to learning through exploration. An extensionof this would be to create working prototypes that stem from our implica-45tions for design and evaluate them with the three age groups. It would beinteresting to see how well the automatic detection of cycles and skimmingbehaviours could help older adults proceed in the right direction and curtailtheir frustrations. In addition, there is scope for future work to explore waysin which children could be familiarized with contextual menus. Although wediscuss the possibility of revealing signifiers for hidden menus, future workcould further investigate different approaches to support prior interactionsthat children are already familiar with as they shift from using a tablet to acomputer. With a better understanding of the differences in children, adults,and older adults’ interface exploration styles, we hope to inspire work thataims to accommodate diverse users through design and improve softwarelearnability across age groups.46Bibliography[1] I. His and C. Potts, “Studying the evolution and enhancement ofsoftware features,” in Proceedings of the International Conference onSoftware Maintenance (ICSM’00), ser. ICSM ’00. Washington, DC,USA: IEEE Computer Society, 2000, pp. 143–. [Online]. Available:http://dl.acm.org/citation.cfm?id=850948.853409[2] L. Kaufman and B. Weed, “Too much of a good thing?:Identifying and resolving bloat in the user interface,” SIGCHIBull., vol. 30, no. 4, pp. 46–47, Oct. 1998. [Online]. Available:http://doi.acm.org/10.1145/310307.310370[3] J. McGrenere and G. Moore, “Are we all in the same ”bloat”?” inProceedings of the Graphics Interface 2000 Conference, May 15-17, 2000,Montr’eal, Qu’ebec, Canada, May 2000, pp. 187–196. [Online]. Available:http://graphicsinterface.org/wp-content/uploads/gi2000-25.pdf[4] J. M. Carroll and C. Carrithers, “Training wheels in a user interface,”Commun. ACM, vol. 27, no. 8, pp. 800–806, Aug. 1984. [Online].Available: http://doi.acm.org/10.1145/358198.358218[5] D. G. Novick, O. D. Andrade, and N. Bean, “The micro-structureof use of help,” in Proceedings of the 27th ACM InternationalConference on Design of Communication, ser. SIGDOC ’09. New47York, NY, USA: ACM, 2009, pp. 97–104. [Online]. Available:http://doi.acm.org/10.1145/1621995.1622014[6] T. Grossman, G. Fitzmaurice, and R. Attar, “A survey of softwarelearnability: Metrics, methodologies and guidelines,” in Proceedings ofthe SIGCHI Conference on Human Factors in Computing Systems, ser.CHI ’09. New York, NY, USA: ACM, 2009, pp. 649–658. [Online].Available: http://doi.acm.org/10.1145/1518701.1518803[7] D. A. Norman and S. W. Draper, User Centered System Design; NewPerspectives on Human-Computer Interaction. Hillsdale, NJ, USA: L.Erlbaum Associates Inc., 1986.[8] J. M. Carroll and M. B. Rosson, “The paradox of the active user,” inInterfacing Thought: Cognitive Aspects of Human-computer Interaction,J. M. Carroll, Ed. Cambridge, MA, USA: MIT Press, 1987, pp. 80–111.[Online]. Available: http://dl.acm.org/citation.cfm?id=28446.28451[9] J. Rieman, “A field study of exploratory learning strategies,” ACMTrans. Comput.-Hum. Interact., vol. 3, no. 3, pp. 189–218, Sep. 1996.[Online]. Available: http://doi.acm.org/10.1145/234526.234527[10] K. McKnight, K. O’Malley, R. Ruzic, M. K. Horsley, J. J. Franey, andK. Bassett, “Teaching in a digital age: How educators use technologyto improve student learning,” Journal of Research on Technology inEducation, vol. 48, no. 3, pp. 194–211, 2016. [Online]. Available:https://doi.org/10.1080/15391523.2016.1175856[11] L. Vermette, J. McGrenere, C. Birge, A. Kelly, and P. K. Chilana,“Freedom to personalize my digital classroom: Understanding teachers’practices and motivations,” in Proceedings of the 2019 CHI Conferenceon Human Factors in Computing Systems, ser. CHI ’19. NewYork, NY, USA: ACM, 2019, pp. 318:1–318:14. [Online]. Available:http://doi.acm.org/10.1145/3290605.330054848[12] R. Sewdas, A. de Wind, L. G. van der Zwaan, W. E. van der Borg,R. Steenbeek, A. J. van der Beek, and C. R. L. Boot, “Why older workerswork beyond the retirement age: a qualitative study,” in BMC PublicHealth, 2016.[13] Y. Barnard, M. D. Bradley, F. Hodgson, and A. D. Lloyd, “Learningto use new technologies by older adults: Perceived difficulties,experimentation behaviour and usability,” Computers in HumanBehavior, vol. 29, no. 4, pp. 1715 – 1724, 2013. [Online]. Available:http://www.sciencedirect.com/science/article/pii/S0747563213000721[14] S. L. Gatto and S. H. Tak, “Computer, internet, and e-mailuse among older adults: Benefits and barriers,” EducationalGerontology, vol. 34, no. 9, pp. 800–811, 2008. [Online]. Available:https://doi.org/10.1080/03601270802243697[15] J. Chin and W.-T. Fu, “Age differences in exploratory learningfrom a health information website,” in Proceedings of the SIGCHIConference on Human Factors in Computing Systems, ser. CHI ’12.New York, NY, USA: ACM, 2012, pp. 3031–3040. [Online]. Available:http://doi.acm.org/10.1145/2207676.2208715[16] R. Leung, C. Tang, S. Haddad, J. Mcgrenere, P. Graf, andV. Ingriany, “How older adults learn to use mobile devices:Survey and field investigations,” ACM Trans. Access. Comput.,vol. 4, no. 3, pp. 11:1–11:33, Dec. 2012. [Online]. Available:http://doi.acm.org/10.1145/2399193.2399195[17] N. Wagner, K. Hassanein, and M. Head, “Review: Computeruse by older adults: A multi-disciplinary review,” Comput. Hum.Behav., vol. 26, no. 5, pp. 870–882, Sep. 2010. [Online]. Available:http://dx.doi.org/10.1016/j.chb.2010.03.02949[18] M. Ka¨hko¨nen and S. Ovaska, “Initial observations on childrenand online instructions,” in Proceedings of the 2006 Confer-ence on Interaction Design and Children, ser. IDC ’06. NewYork, NY, USA: ACM, 2006, pp. 93–96. [Online]. Available:http://doi.acm.org/10.1145/1139073.1139098[19] B. Chimbo, J. H. Gelderblom, and M. R. de Villiers, “A comparativeanalysis of the meaning of ’learnability’ for child and adult users,” 2011.[20] K. Kiani, G. Cui, A. Bunt, J. McGrenere, and P. K. Chilana, “Beyond”one-size-fits-all”: Understanding the diversity in how softwarenewcomers discover and make use of help resources,” in Proceedings ofthe 2019 CHI Conference on Human Factors in Computing Systems,ser. CHI ’19. New York, NY, USA: ACM, 2019, pp. 340:1–340:14.[Online]. Available: http://doi.acm.org/10.1145/3290605.3300570[21] S. de Mul and H. van Oostendorp, “Learning user inter-faces by exploration,” Acta Psychologica, vol. 91, no. 3,pp. 325 – 344, 1996, usage of Modern Technol-ogy by Experts and Non-professionals. [Online]. Available:http://www.sciencedirect.com/science/article/pii/0001691895000607[22] B. Lafreniere, A. Bunt, and M. Terry, “Task-centric interfaces forfeature-rich software,” in Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future ofDesign, ser. OzCHI ’14. New York, NY, USA: ACM, 2014, pp. 49–58.[Online]. Available: http://doi.acm.org/10.1145/2686612.2686620[23] C.-I. Trudel and S. J. Payne, “Reflection and goal managementin exploratory learning,” International Journal of Human-ComputerStudies, vol. 42, no. 3, pp. 307 – 339, 1995. [Online]. Available:http://www.sciencedirect.com/science/article/pii/S107158198571015450[24] L. J. Couse and D. W. Chen, “A tablet computer for young children?exploring its viability for early childhood education,” Journal ofResearch on Technology in Education, vol. 43, no. 1, pp. 75–96, 2010.[Online]. Available: https://doi.org/10.1080/15391523.2010.10782562[25] N. Hudson, B. Lafreniere, P. K. Chilana, and T. Grossman,“Investigating how online help and learning resources support children’suse of 3d design software,” in Proceedings of the 2018 CHI Conferenceon Human Factors in Computing Systems, ser. CHI ’18. NewYork, NY, USA: ACM, 2018, pp. 257:1–257:14. [Online]. Available:http://doi.acm.org/10.1145/3173574.3173831[26] M. Y. Chan, S. Haber, L. M. Drew, and D. C. Park, “Training OlderAdults to Use Tablet Computers: Does It Enhance Cognitive Function?”The Gerontologist, vol. 56, no. 3, pp. 475–484, 06 2014.[27] A. Aula, “User study on older adults’ use of the web and searchengines,” Univers. Access Inf. Soc., vol. 4, no. 1, pp. 67–81, Sep. 2005.[Online]. Available: http://dx.doi.org/10.1007/s10209-004-0097-7[28] A. D. Fisk, W. A. Rogers, N. Charness, S. J. Czaja, and J. Sharit,Designing for older adults: Principles and creative human factors ap-proaches. CRC press, 2009.[29] W. A. Rogers, A. D. Fisk, S. E. Mead, N. Walker, and E. F. Cabrera,“Training older adults to use automatic teller machines,” HumanFactors, vol. 38, no. 3, pp. 425–433, 1996, pMID: 8865767. [Online].Available: https://doi.org/10.1518/001872096778701935[30] J. M. Hickman, W. A. Rogers, and A. D. Fisk, “Training older adultsto use new technology.” The journals of gerontology. Series B, Psycho-logical sciences and social sciences, vol. 62 Spec No 1, pp. 77–84, Jun2007.51[31] C. Norval, J. L. Arnott, and V. L. Hanson, “What’s on yourmind?: Investigating recommendations for inclusive social networkingand older adults,” in Proceedings of the SIGCHI Conferenceon Human Factors in Computing Systems, ser. CHI ’14. NewYork, NY, USA: ACM, 2014, pp. 3923–3932. [Online]. Available:http://doi.acm.org/10.1145/2556288.2556992[32] A. Hope, T. Schwaba, and A. M. Piper, “Understanding digital andmaterial social communications for older adults,” in Proceedings of theSIGCHI Conference on Human Factors in Computing Systems, ser.CHI ’14. New York, NY, USA: ACM, 2014, pp. 3903–3912. [Online].Available: http://doi.acm.org/10.1145/2556288.2557133[33] C. D. Stewart, V. L. Hanson, and T. J. Nind, “Assisting older adultsin assessing the reliability of health-related websites,” in CHI ’14Extended Abstracts on Human Factors in Computing Systems, ser. CHIEA ’14. New York, NY, USA: ACM, 2014, pp. 2611–2616. [Online].Available: http://doi.acm.org/10.1145/2559206.2581243[34] W. Choi, “What makes online health information credible for olderadults?: An exploratory study,” in CHI ’13 Extended Abstracts onHuman Factors in Computing Systems, ser. CHI EA ’13. NewYork, NY, USA: ACM, 2013, pp. 2671–2676. [Online]. Available:http://doi.acm.org/10.1145/2468356.2479491[35] F. Portet, M. Vacher, C. Golanski, C. Roux, and B. Meillon,“Design and evaluation of a smart home voice interface for theelderly: Acceptability and objection aspects,” Personal UbiquitousComput., vol. 17, no. 1, pp. 127–144, Jan. 2013. [Online]. Available:http://dx.doi.org/10.1007/s00779-011-0470-5[36] G. Demiris, D. P. Oliver, G. Dickey, M. Skubic, andM. Rantz, “Findings from a participatory evaluation of a52smart home application for older adults,” Technol. Health Care,vol. 16, no. 2, pp. 111–118, Apr. 2008. [Online]. Available:http://dl.acm.org/citation.cfm?id=1378666.1378670[37] M. A. O’brien, W. A. Rogers, and A. D. Fisk, “Understandingage and technology experience differences in use of prior knowledgefor everyday technology interactions,” ACM Trans. Access. Comput.,vol. 4, no. 2, pp. 9:1–9:27, Mar. 2012. [Online]. Available:http://doi.acm.org/10.1145/2141943.2141947[38] S. Chiasson and C. Gutwin, “Design principles for children’s software,”Tech. Rep. HCI-TR-05-02, 2005.[39] N. Soni, A. Aloba, K. S. Morga, P. J. Wisniewski, and L. Anthony,“A framework of touchscreen interaction design recommendations forchildren (tidrc): Characterizing the gap between research evidenceand design practice,” in Proceedings of the 18th ACM InternationalConference on Interaction Design and Children, ser. IDC ’19. NewYork, NY, USA: ACM, 2019, pp. 419–431. [Online]. Available:http://doi.acm.org/10.1145/3311927.3323149[40] M. Bakaev, “Fitts’ law for older adults: Considering a factor of age,” inProceedings of the VIII Brazilian Symposium on Human Factors in Com-puting Systems, ser. IHC ’08. Porto Alegre, Brazil, Brazil: SociedadeBrasileira de Computa&#231;&#227;o, 2008, pp. 260–263. [Online].Available: http://dl.acm.org/citation.cfm?id=1497470.1497502[41] S. Keates and S. Trewin, “Effect of age and parkinson’s disease on cursorpositioning using a mouse,” in Proceedings of the 7th InternationalACM SIGACCESS Conference on Computers and Accessibility, ser.Assets ’05. New York, NY, USA: ACM, 2005, pp. 68–75. [Online].Available: http://doi.acm.org/10.1145/1090785.109080053[42] M. Zajicek, “Interface design for older adults,” in Proceedings ofthe 2001 EC/NSF Workshop on Universal Accessibility of UbiquitousComputing: Providing for the Elderly, ser. WUAUC’01. NewYork, NY, USA: ACM, 2001, pp. 60–65. [Online]. Available:http://doi.acm.org/10.1145/564526.564543[43] A. Darejeh and D. Valbir Singh, “A review on user interface designprinciples to increase software usability for users with less computerliteracy,” Journal of Computer Science, vol. 9, no. 11, pp. 1443–1450,2013.[44] J. Nielsen and R. Molich, “User interface design guide-lines: 10 rules of thumb,” Apr 1994. [Online].Available: https://www.interaction-design.org/literature/article/user-interface-design-guidelines-10-rules-of-thumb[45] B. Shneiderman, Designing the User Interface: Strategies for EffectiveHuman-Computer Interaction, 3rd ed. Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc., 1997.[46] J. Johnson, Designing with the Mind in Mind: Simple Guide to Un-derstanding User Interface Design Rules. San Francisco, CA, USA:Morgan Kaufmann Publishers Inc., 2010.[47] D. R. Compeau and C. A. Higgins, “Computer self-efficacy: Develop-ment of a measure and initial test,” MIS Quarterly, vol. 19, no. 2, pp.189–211, 1995. [Online]. Available: http://www.jstor.org/stable/249688[48] O. Friard and M. Gamba, “Boris: A free, versatile open-source event-logging software for video/audio coding and live observations,” Methodsin Ecology and Evolution, vol. 7, 05 2016.[49] C. N. Harada, M. N. Love, and K. L. Triebel, “Normal cognitive aging.”Clinics in geriatric medicine, vol. 29 4, pp. 737–52, 2013.54[50] J. Matejka, T. Grossman, and G. Fitzmaurice, “Patina: Dynamicheatmaps for visualizing application usage,” in Proceedings of theSIGCHI Conference on Human Factors in Computing Systems, ser.CHI ’13. New York, NY, USA: ACM, 2013, pp. 3227–3236. [Online].Available: http://doi.acm.org/10.1145/2470654.2466442[51] S. Malacria, G. Bailly, J. Harrison, A. Cockburn, and C. Gutwin,“Promoting hotkey use through rehearsal with exposehk,” inProceedings of the SIGCHI Conference on Human Factors in ComputingSystems, ser. CHI ’13. New York, NY, USA: ACM, 2013, pp. 573–582.[Online]. Available: http://doi.acm.org/10.1145/2470654.2470735[52] Y. S. Yoon and B. A. Myers, “Supporting selective undo in acode editor,” in Proceedings of the 37th International Conferenceon Software Engineering - Volume 1, ser. ICSE ’15. Piscataway,NJ, USA: IEEE Press, 2015, pp. 223–233. [Online]. Available:http://dl.acm.org/citation.cfm?id=2818754.2818784[53] B. A. Myers, A. Lai, T. M. Le, Y. Yoon, A. Faulring, and J. Brandt,“Selective undo support for painting applications,” in Proceedings of the33rd Annual ACM Conference on Human Factors in Computing Systems,ser. CHI ’15. New York, NY, USA: ACM, 2015, pp. 4227–4236.[Online]. Available: http://doi.acm.org/10.1145/2702123.2702543[54] C. A. Fraser, M. Dontcheva, H. Winnemo¨ller, S. Ehrlich, andS. Klemmer, “Discoveryspace: Suggesting actions in complex software,”in Proceedings of the 2016 ACM Conference on Designing InteractiveSystems, ser. DIS ’16. New York, NY, USA: ACM, 2016, pp. 1221–1232.[Online]. Available: http://doi.acm.org/10.1145/2901790.290184955AppendixThis appendix contains all the resources that were used to conduct the userstudy discussed in Chapter 3.Call for Participation5657Consent Forms58596061626364Conceptual Model656667Task List6869Expertise Questionnaire707172737475Self-Efficacy Questionnaire76777879


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items