"Science, Faculty of"@en . "Computer Science, Department of"@en . "DSpace"@en . "UBCV"@en . "Ponsard, Antoine"@en . "2015-12-01T03:18:09"@en . "2015"@en . "Master of Science - MSc"@en . "University of British Columbia"@en . "The HCI community has identified the need to let users adapt their software to their own tasks and preferences. Yet, many users do not customize, or only do so rarely. The de facto standard customization mechanism is the settings panel, which has undergone minimal design improvements since it was introduced along with the graphical user interface in the 1980s. Entirely disconnected from the application UI, these panels afford only very indirect manipulation, as users must rely on often cryptic text labels to identify the settings they want to change. From a developer\u00E2\u0080\u0099s point of view these panels make sense: they are simple graphical representations of traditional UNIX config files. In this thesis, we propose a novel customization approach, designed from the user\u00E2\u0080\u0099s point of view. In Anchored Customization, settings are anchored to conceptually related elements of the application UI. Our Customization Layer instantiates this approach: users can see which UI elements are customizable, and access their associated settings. We designed three variants of customization layer based on multi-layered interfaces, and implemented these variants on top of a popular web application for task management, Wunderlist. A remote experiment conducted on Mechanical Turk, complemented with a face-to-face lab experiment (for a total of 60 participants) showed that the two minimalist variants were 35% faster than Wunderlist\u00E2\u0080\u0099s settings panel. This new approach provides significant benefits for users while requiring little extra work from designers and developers of applications."@en . "https://circle.library.ubc.ca/rest/handle/2429/55601?expand=metadata"@en . "Anchored CustomizationAnchoring Settings to the Application Interface to A\u001DordCustomizationbyAntoine PonsardB.Sc., \u00C3\u0089cole Polytechnique, 2012Dipl\u00C3\u00B4me d\u00E2\u0080\u0099Ing\u00C3\u00A9nieur, \u00C3\u0089cole Polytechnique, 2013A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFMASTER OF SCIENCEinThe Faculty of Graduate and Postdoctoral Studies(Computer Science)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)November 2015\u00C2\u00A9 Antoine Ponsard 2015AbstractThe HCI community has identi\u001Bed the need to let users adapt their software totheir own tasks and preferences. Yet, many users do not customize, or only doso rarely. The de facto standard customization mechanism is the settings panel,which has undergone minimal design improvements since it was introduced alongwith the graphical user interface in the 1980s. Entirely disconnected from theapplication UI, these panels a\u001Dord only very indirect manipulation, as users mustrely on often cryptic text labels to identify the settings they want to change. Froma developer\u00E2\u0080\u0099s point of view these panels make sense: they are simple graphicalrepresentations of traditional UNIX con\u001Bg \u001Bles.In this thesis, we propose a novel customization approach, designed from theuser\u00E2\u0080\u0099s point of view. In Anchored Customization, settings are anchored to concep-tually related elements of the application UI. Our Customization Layer instantiatesthis approach: users can see which UI elements are customizable, and access theirassociated settings. We designed three variants of customization layer based onmulti-layered interfaces, and implemented these variants on top of a popular webapplication for task management, Wunderlist. A remote experiment conductedon Mechanical Turk, complemented with a face-to-face lab experiment (for a to-tal of 60 participants) showed that the two minimalist variants were 35% fasterthan Wunderlist\u00E2\u0080\u0099s settings panel. This new approach provides signi\u001Bcant bene-\u001Bts for users while requiring little extra work from designers and developers ofapplications.iiPrefaceThe experiments described in this thesis were conducted with the approval of theUBC Behavioural Research Ethics Board (certi\u001Bcate number H11-01976).Parts of this thesis appear in a conference paper manuscript 1 where I amthe \u001Brst author. Joanna McGrenere helped frame and write the manuscript, andprovided supervisory assistance for this research.1A. Ponsard, J. McGrenere. Anchored Customization: Anchoring Settings to the ApplicationInterface to A\u001Dord CustomizationiiiTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiAcknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Anchored Customization . . . . . . . . . . . . . . . . . . . . . . . 11.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.1 Adaptable Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . 42.2 Adaptive and Mixed-Initiative Interfaces . . . . . . . . . . . . . . 42.3 Customization in Context . . . . . . . . . . . . . . . . . . . . . . 52.4 Interface Context . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Systematic Assessment of Feasibility . . . . . . . . . . . . . . . . . 63.1 Existing Customization Mechanisms . . . . . . . . . . . . . . . . 63.2 Feasibility of Anchored Customization . . . . . . . . . . . . . . . 74 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.1 Anchored Customization . . . . . . . . . . . . . . . . . . . . . . . 94.1.1 Display of Settings . . . . . . . . . . . . . . . . . . . . . . 134.1.2 Display of Anchors . . . . . . . . . . . . . . . . . . . . . 13iv4.2 Customization Layer . . . . . . . . . . . . . . . . . . . . . . . . . 144.2.1 Anchors Visible in a Layer . . . . . . . . . . . . . . . . . 144.2.2 Three Variants for Displaying Settings . . . . . . . . . . . 154.2.3 Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.2.4 Software Architecture . . . . . . . . . . . . . . . . . . . . 204.2.5 Implementation . . . . . . . . . . . . . . . . . . . . . . . 205 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215.1 Experiment 1: Remote Mechanical Turk . . . . . . . . . . . . . . 225.1.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225.1.2 MTurk Results . . . . . . . . . . . . . . . . . . . . . . . . 255.2 Experiment 2: Face-to-Face in the Lab . . . . . . . . . . . . . . . 265.2.1 Method Di\u001Derences from Study 1 . . . . . . . . . . . . . . 275.2.2 Face-to-Face Lab Results . . . . . . . . . . . . . . . . . . 275.3 Secondary Exploratory Analyses . . . . . . . . . . . . . . . . . . 316 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346.1 Reconciling the Performance Results . . . . . . . . . . . . . . . . 346.2 Performance/Awareness Trade-O\u001D . . . . . . . . . . . . . . . . . 356.3 Applicability to Other Desktop Software . . . . . . . . . . . . . . 356.4 Applicability to Handheld Devices . . . . . . . . . . . . . . . . . 366.5 Interpretation Gulf . . . . . . . . . . . . . . . . . . . . . . . . . . 366.6 The Developers\u00E2\u0080\u0099 Point of View . . . . . . . . . . . . . . . . . . . 366.7 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . 397.1 Long-Term E\u001Dects . . . . . . . . . . . . . . . . . . . . . . . . . . 397.2 Generating the Mapping . . . . . . . . . . . . . . . . . . . . . . . 407.3 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . 40Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42AppendicesA Mapping of settings to UI elements in Wunderlist . . . . . . . . . 47B Experiment 1 Resources . . . . . . . . . . . . . . . . . . . . . . . . . 50B.1 Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50B.2 Participant Consent Form . . . . . . . . . . . . . . . . . . . . . . 53B.3 Questionnaires . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56vB.3.1 Demographics . . . . . . . . . . . . . . . . . . . . . . . . 56B.3.2 Satisfaction . . . . . . . . . . . . . . . . . . . . . . . . . . 56B.3.3 Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . 56C Experiment 2 Resources . . . . . . . . . . . . . . . . . . . . . . . . . 59C.1 Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59C.2 Call For Participation . . . . . . . . . . . . . . . . . . . . . . . . . 59C.3 Participant Consent Form . . . . . . . . . . . . . . . . . . . . . . 61C.4 Questionnaires . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64C.4.1 Demographics . . . . . . . . . . . . . . . . . . . . . . . . 64C.4.2 Ratings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66C.4.3 Rankings . . . . . . . . . . . . . . . . . . . . . . . . . . . 66viList of Tables3.1 Summary of a systematic review of 8 PTM apps to determine thefeasibility of Anchored Customization . . . . . . . . . . . . . . . . 7viiList of Figures4.1 Three approaches to organize settings: con\u001Bg \u001Bles, settings pan-els, Anchored Customization . . . . . . . . . . . . . . . . . . . . . 104.2 Two approaches to navigate settings: tabs and Anchored Cus-tomization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.3 The settings panel o\u001Dered in Wunderlist . . . . . . . . . . . . . . 124.4 The Wunderlist web application . . . . . . . . . . . . . . . . . . . 164.5 Customization Layer displayed on top of Wunderlist . . . . . . . 164.6 Ghost anchors and collapsing mechanism . . . . . . . . . . . . . . 174.7 Minimal, Minimal+Context, and Full+Highlight variants . . . . . 174.8 Reverse highlighting in the full settings panel . . . . . . . . . . . 195.1 Modal dialog displayed at the beginning of each trial . . . . . . . 235.2 Possible states of the progress bar, displayed at the top of the win-dow during the experiment . . . . . . . . . . . . . . . . . . . . . . 245.3 Experiment 1: 95% con\u001Bdence intervals of the median durationper Customization Mechanism and Block . . . . . . . . . . . . . . 265.4 Experiment 2: 95% con\u001Bdence intervals of the median durationper Customization Mechanism . . . . . . . . . . . . . . . . . . . . 285.5 Density plots of the distribution of the percentage of trials in whichparticipants selected a \u00E2\u0080\u009Ccorrect\u00E2\u0080\u009D anchor . . . . . . . . . . . . . . . 325.6 Density plots of the distribution of the percentage of trials in whichparticipants selected a \u00E2\u0080\u009Ccorrect\u00E2\u0080\u009D anchor, for each CustomizationLayer variant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32viiiAcknowledgmentsI thank my supervisor Joanna McGrenere, for her endless patience, guidance andsupport throughout this Masters.LUNCH, MUX, and HCI@UBC were valuable occasions to meet fellow HCIresearchers at UBC. I thank all the people who organized these meetings, andall those who provided feedback on my work. In particular, I thank FranciscoEscalona, Kamyar Ardekani and Kailun Zhang, with whom I worked on threegreat projects. I also thank Peter Beshai, Derek Cormier, Mona Haraty, MatthewBrehmer and Oliver Schneider for their advice and the many insightful discussionswe had.I thank my Information Visualization professor, Tamara Munzner, who coura-geously accepted to be my second reader.Finally, I thank my family and friends for their encouragement and support.This research was funded by the Graphics Animation and New Media (GRAND)NCE as well as the NSERC Discovery Grant program.ixDedicationThere is one thing that is more dear to a Frenchman\u00E2\u0080\u0099s heart than the sight ofthe Ei\u001Del tower over the rooftops of Paris: a properly baked loaf of bread. Alas,this simple pleasure is surprisingly di\u001Ccult to \u001Bnd outside of the grassy plains ofFrance.This is why I would like to dedicate this thesis to the bakery Beyond Bread,for their remarkable e\u001Dort to propagate the baking traditions of France to thedistant Canada. As soon as I set foot in this bakery, the smell of freshly bakedbread washing over me, I knew that I had found a home away from home. Icannot be thankful enough for their country bread, with its \u001Brm crust and delicatecrumb; for their properly named pains au chocolat; for their rosemary foccacia,soft as a dream and dripping olive oil; and even for their overpriced baguette andcroissants.This work would not have been possible without them.xChapter 1IntroductionThe HCI community has identi\u001Bed the potential bene\u001Bts of supporting users toadapt their software to their own tasks and preferences [28]. Yet, many users donot customize, or only do so rarely. In today\u00E2\u0080\u0099s software, the de facto standard cus-tomization mechanism is the settings panel, also known as the \u00E2\u0080\u009Coptions menu\u00E2\u0080\u009D or\u00E2\u0080\u009Cpreferences dialog.\u00E2\u0080\u009D These panels have signi\u001Bcant usability limitations in that thesettings they o\u001Der are entirely disconnected from the application UI. There are novisual a\u001Dordances in the UI that can help answer the question: \u00E2\u0080\u009CIs it possible tocustomize X?\u00E2\u0080\u009D Users must open the settings panel and then rely entirely on textlabels to identify the settings they want to change, with little to no other con-textual information. Thus, the classic vocabulary problem applies: the languageused by designers to describe an aspect of the interface is often di\u001Derent from thelanguage of users [19]. This creates a deep gulf of execution between the user\u00E2\u0080\u0099sintentions, formulated in the context of the application UI, and the actions o\u001Deredin the settings panel. The fact that customization occurs fairly infrequently overlong periods of time [27] only exacerbates the problem: users are likely to forgetwhere things are in the settings panel between customization episodes.Despite obvious usability problems, settings panels have undergone only mi-nor improvements since they were \u001Brst introduced along with the graphical userinterface in the 1980s [24]. The interaction paradigm fundamentally hasn\u00E2\u0080\u0099t changed:the user can browse tabs of settings, tick checkboxes, and choose values fromdrop-downs. These panels essentially represent a developer\u00E2\u0080\u0099s point of view of theapplication\u00E2\u0080\u0099s customization opportunities: they are simple graphical representa-tions of traditional UNIX con\u001Bg \u001Bles.21.1 Anchored CustomizationWe propose a novel approach to customization, called Anchored Customization,designed from the user\u00E2\u0080\u0099s point of view: anchoring settings to visual elementsof the UI that are conceptually related to these settings. This reduces the gulf of2Con\u001Bg \u001Bles list in plain text the settings that can be modi\u001Bed. Users can change settings byopening and editing the \u001Ble in a text editor.1execution, and leverages users\u00E2\u0080\u0099 existing knowledge of the application UI. Whileothers have proposed the idea of moving the access point of a customization closerto the feature that it a\u001Dects [38], we take the approach further by allowing usersto access all conceptually related functions from an anchor point. For instance,a noti\u001Bcation icon in the UI can be used as the anchor to change not only howpopup noti\u001Bcations (emanating from the icon) are displayed, but also the fre-quency of various noti\u001Bcations (e.g., email), and other related settings. AnchoredCustomization leverages mental associations that users can form intuitively be-tween visual elements of the interface and changes they want to make to theirsoftware.Once settings are anchored to UI elements, there are di\u001Derent possible ways ofvisualizing and accessing them. We designed, implemented, and evaluated Cus-tomization Layer, as a concrete instantiation of the Anchored Customization ap-proach. Customization opportunities are represented in a meta-layer on top of theinterface. Users can see all UI anchors at once in this layer, and can access theirassociated settings by clicking on them. This type of layered design has been suc-cessfully used in the context of online help [8]. Although clicking on an anchoronly shows a small subset of settings initially, the user can access the full settingspanel from any anchor via a simple expansion mechanism.Our approach is pragmatic: it recognizes that settings panels are the standardin the software industry. Developers rely on this simple mechanism to provideend-user customization, since these panels are (presumably) an inexpensive partof the UI to develop. And users are accustomed to them. Introducing a new cus-tomization mechanism therefore requires careful consideration of the cost/bene-\u001Bt trade-o\u001Ds at play. We show that anchoring settings to the UI through a cus-tomization layer has the potential to provide signi\u001Bcant bene\u001Bts for users, whilerequiring only minimal extra work from the designers/developers. Our goal is topropose a well-considered interface improvement that can be adopted widely.A customization mechanism must be implemented to customize a speci\u001Bc app.Although our design approach is applicable to any software with a GUI that is cus-tomizable via a settings panel, we had to choose a particular application domainfor a \u001Brst evaluation. Since task management tools are widely used, and signif-icant individual di\u001Derences have been observed in the way people manage theirtasks [22], we considered that Personal Tasks Management (PTM) would be anappropriate \u001Brst application domain for our research.1.2 ContributionsOur work contributes the following:21. The concept of Anchored Customization: to anchor customization oppor-tunities to conceptually related elements in the application UI.2. A systematic review of PTM apps to evaluate a priori the feasibility andpotential usefulness of adopting this concept.3. A realization of this concept in a customization mechanism, the Customiza-tion Layer, with three variants that explore di\u001Derent trade-o\u001Ds betweenmulti-layered interfaces.4. An implementation architecture of these three variants for web applica-tions: our prototype augments a real-world PTM app, Wunderlist [20], butcan be adapted to other modern web apps with minimal additional work fordevelopers and designers.5. Two experiments (Mechanical Turk and face-to-face) with 60 participantsshowing performance improvements of the Customization Layer over theregular settings panel provided by Wunderlist. The face-to-face experimentalso provided a preliminary understanding of how users apprehend An-chored Customization.1.3 OverviewPrevious work relevant to this research is summarized in Chapter 2. Chapter 3presents two systematic reviews of a set of Personal Task Management apps. The\u001Brst review identi\u001Bes the customization mechanisms provided in these apps; thesecond evaluates the feasibility of anchoring settings to visual elements of theinterface. Chapter 4 outlines the design space of Anchored Customization, andexplains the design of our Customization Layer. Chapter 5 presents a two-partevaluation of our Customization Layer, in comparison to traditional settings pan-els. A \u001Brst experiment was conducted remotely on Amazon Mechanical Turk, tomeasure the performance of our Customization Layer for changing settings. Asecond experiment took place in a face-to-face settings at our university, to ver-ify and triangulate the results of the \u001Brst one. Chapter 6 discusses the insightsgathered on the three variants of our Customization Layer, and re\u001Eects on thepossibilities o\u001Dered by this new customization mechanism. Chapter 7 outlinesdirections for future work and concludes this thesis.3Chapter 2Related WorkPrevious research has studied the bene\u001Bts of letting users adapt software to theirown preferences and tasks [27, 28, 30]. What exactly constitutes customizationis not well-established in the HCI community, despite there being a rich litera-ture in this space. For example, Mackay [27] de\u001Bnes end-user customization as\u00E2\u0080\u009Cthe mechanisms by which users may specify individual preferences, and preservetheir preferred patterns of use, without writing code.\u00E2\u0080\u009D In that sense, it is closely re-lated to the concept of tailorability, in which \u00E2\u0080\u009Cend users can progressively modifya working system without ever having to leave the application domain to work ina separate underlying programming domain.\u00E2\u0080\u009D [28]. Yet the term \u00E2\u0080\u009Ctailorable\u00E2\u0080\u009D seemssomewhat outdated in the HCI community, and has been replaced by \u00E2\u0080\u009Cadaptable\u00E2\u0080\u009D.2.1 Adaptable InterfacesThe challenges of designing and building adaptable interfaces has produced a richand varied literature. Multi-layered interfaces [30, 33] group subset of featuresinto di\u001Derent layers, and allow users to choose which layer to work with. Sophisti-cated techniques have been proposed to let users modify GUIs at run time [12, 13],even without access to the source code [11, 31, 36]. The end-user developmentparadigm goes even further, by empowering users to build their own applica-tions [17, 26]. Yet, powerful adaptable approaches generally require users to spendsigni\u001Bcant time and e\u001Dort personalizing their interfaces, often by writing scriptsor editing code\u00E2\u0080\u0094which limits the potential audience to power users. As a result,these approaches have not yet been widely adopted in industry.2.2 Adaptive and Mixed-Initiative InterfacesAdaptive approaches attempt to reduce the need for user involvment by automat-ically generating user models that predict potentially useful customizations [16].However, giving control of the user interface to an automated system has signi\u001B-cant drawbacks, as it tends to make the UI spatially unstable and unpredictable [14].Mixed-initiative approaches can alleviate those problems by suggesting possible4customizations to the user, who remains in control of their interface [7, 10]. Ul-timately, adaptive and mixed-initiative approaches put the onus on developers tobuild and integrate complex user models into the interface, which is di\u001Ccult todo well [23].2.3 Customization in ContextOur work focuses on leveraging the customization opportunities already availablein existing software settings panels, with the goal of minimizing the extra costsof customization for both users and developers. The potential of \u00E2\u0080\u009Crepresentingtailoring functions in the overall interface\u00E2\u0080\u009D was identi\u001Bed 15 years ago [25], buthas received little attention since then. An early work proposed the concept ofDirect Activation [38] which states that \u00E2\u0080\u009Cthe access point of the tailoring functionshould be designed related to the one of the tailorable function\u00E2\u0080\u009D. In other words,the settings that a\u001Dect a given feature should be accessible from (or around) theaccess point of the feature they a\u001Dect. We broaden this approach by anchoringany setting to any conceptually related elements of the UI \u00E2\u0080\u0094not only elements ob-jectively a\u001Dected by a setting. For instance, the setting for changing the keyboardshortcut used to \u00E2\u0080\u009Copen a new tab\u00E2\u0080\u009D could be anchored to the button that opens anew tab. This setting does not tailor the \u00E2\u0080\u009Copen a new tab\u00E2\u0080\u009D function itself, but con-ceptually it makes sense to associate a keyboard shortcut and a button that callthe same function.2.4 Interface ContextThe bene\u001Bts of preserving interface context have been exploited in other areasof HCI. Perhaps the most common example is the context menu, which reducesthe time needed to select options related to the current location of the pointer.For example, contextual help can augment an application interface with links toa wiki [35], or user-generated Q&As [8]. While help queries tend to naturallycorrespond to elements of the interface that users are trying to learn, the corre-spondence between settings and interface elements may be less clear, as settingscan a\u001Dect software in arbitrary ways. Thus, it was not obvious at the outset ofour research that such direct correspondences could be found from settings toUI elements. Yet the literature o\u001Ders many promising examples of representingmeta-information about an application as an overlay on the app interface: recentchanges [2, 5], computational wear [29], predicted use [18], even low-level usabil-ity problems [37]. This motivated our visual design approach, which considerscustomization opportunities as a meta-layer on top of the application interface.5Chapter 3Systematic Assessment ofFeasibilityThe motivation of Anchored Customization is to address the shortcomings of set-tings panels, which appear to be the standard in the software industry. To verifythis assumption, we performed a systematic review of a set of PTM apps, investi-gating which customization mechanisms they provide. We then conducted a sec-ond review to evaluate the feasibility of the Anchored Customization approach.3.1 Existing Customization MechanismsWe chose a particular yet fairly generic application domain: Personal Task Man-agement. We selected 12 of the most popular PTM apps3, for a total of 20 uniqueapps (counting separately both desktop and Android versions). For each applica-tion we recorded:\u00E2\u0080\u00A2 which customization mechanisms were o\u001Dered;\u00E2\u0080\u00A2 how many user actions were needed to access them;\u00E2\u0080\u00A2 the location of the access points to these mechanisms in the applicationinterface;\u00E2\u0080\u00A2 the type and number of settings each mechanism could change.Our review showed that an overwhelming majority of settings were accessedvia settings panels. In fact, only 1 of the 20 applications didn\u00E2\u0080\u0099t o\u001Der a settingspanel at review time\u00E2\u0080\u0094but it now does. The structure of these panels was remark-ably similar: tabs for desktops apps, multi-level menus on Android. In both cases,additional visual delimiters such as lines or boxes were sometimes used insideeach tab or level, to create subsections of related settings. The same widgets wereused to change settings: checkboxes or toggles for binary settings, drop-downs3Evernote, OneNote, Any.do, Cal, Wunderlist, Todoist, Remember The Milk, Toodledoo, Astrid,Clear, GTasks, and Tasks.6Application Platform # settings directly indirectly notAstrid Android 41 25 6 10Evernote Desktop 31 11 12 8Gmail Web 29 16 7 6RTM Android 26 10 11 5Toodledo Web 53 37 7 9Toodledo Android 43 26 16 1Wunderlist Web 41 12 28 1Word Desktop 132 90 24 18total 396 227 111 58normalized % 100% 52% 32% 16%Table 3.1: Summary of a systematic review of 8 PTM apps to determine the fea-sibility of Anchored Customization. For each app, the settings provided in thesettings panel were classi\u001Bed as directly, indirectly, or not anchorable. To avoidbiasing the results towards the applications o\u001Dering the most settings, we nor-malized the di\u001Derent proportions of settings for each app before averaging acrossall apps.or radio buttons for multiple choices, and in some cases sliders for numbers. Theaccess points to these settings panels were also very similar: an \u00E2\u0080\u009COptions\u00E2\u0080\u009D, \u00E2\u0080\u009CPref-erences\u00E2\u0080\u009D or \u00E2\u0080\u009CSettings\u00E2\u0080\u009D menu entry at the top of the screen, or a gear or toolboxicon.We found that settings panels also contain a variety of non-settings items,such as an \u00E2\u0080\u009CAbout\u00E2\u0080\u009D page, contact information, or promotional materials for up-grading to a paid version. In that sense, they serve somewhat as a mixture ofmiscellaneous elements that were probably not deemed important enough to beintegrated in the main application interface.We did encounter a few other customization mechanisms, such as dragging aborder to resize a column, or reordering buttons via drag-and-drop. These alter-native mechanisms e\u001Dectively use direct manipulation to let users customize, butwere typically restricted to changing a very speci\u001Bc part of the interface.3.2 Feasibility of Anchored CustomizationThe widespread use of settings panels highlights the potential usefulness of An-chored Customization. We next evaluated the feasibility of this approach on asubset of apps from our \u001Brst review. We selected apps with more than 15 settings,and included Microsoft Word and Gmail which, while being general-purpose soft-7ware, can both also be used for task management. As shown in Table 3.1, wereviewed a total 8 apps (5 on desktop, 3 on Android), classifying a total of 396settings.According to our estimation, 52% of settings could be anchored unambigu-ously in the interface. A large number of these correspond to settings that directlyshow or hide interface elements, or change their position or visual appearance.Another 32% of settings could be indirectly anchored based on an intuitive mentalassociation. For instance, the frequency of automatic syncing could be anchoredto the button that performs a manual synchronization. Finally, we estimated thatonly 16% of settings could not be anchored anywhere in a meaningful way. Thesesettings correspond either to advanced options (e.g., number of weeks after whichcompleted tasks are archived), or global options (e.g., display language) which arenot related to any particular UI location.The percentages reported here are based on the raw number of settings o\u001Deredby the apps we reviewed. However, not all settings are created equal: some willbe changed more often than others. A more accurate statistic would be to weigheach setting by how often users can be expected to change it. We believe that thisadjusted metric would reduce the relative proportion of non-anchorable settingseven further. Indeed, only a small fraction of users (probably those with moretechnical backgrounds) is likely to dig into the most advanced settings.Overall, the results of our second review provide solid evidence for the vi-ability of our Anchored Customization approach, at least in the PTM domain:approximately 84% of settings can be anchored. Yet a signi\u001Bcant number of set-tings cannot be anchored (16%), which must be taken into account when designinganchored customization mechanisms.8Chapter 4DesignIn most applications, settings are a list of parameters that can take some prede-\u001Bned values. They are generally given a human readable label, and sometimes ashort description. For example, a \u00E2\u0080\u009Ccon\u001Brm before deleting\u00E2\u0080\u009D setting can be eithertrue or false, and could have the description \u00E2\u0080\u009Cshow ok/cancel popup when clickingon the delete button\u00E2\u0080\u009D. UNIX-type con\u001Bg \u001Bles are the most barebone representa-tion of settings, and the closest to the programming domain (Figure 4.1 a). Userscan manually change settings by editing the con\u001Bg \u001Ble in a text editor.Settings panels o\u001Der two key improvements over con\u001Bg \u001Bles. First, they pre-vent errors by restricting the values that a given setting can take. For instance,checkboxes only toggle booleans, and drop-downs o\u001Der a limited set of optionsto choose from. Second, settings panels often group related settings together intotabs or other forms of subsections, with the aim to reduce the time needed toaccess a particular setting. In this way, settings panels are a static abstract par-tition of settings: each setting appears in only one section; sections are based onabstract categories, such as \u00E2\u0080\u009Cshortcuts\u00E2\u0080\u009D or \u00E2\u0080\u009Cdisplay;\u00E2\u0080\u009D and these categories are de-\u001Bned by designers at the outset (Figure 4.1 b). Tabs are used to navigate from onehigh-level category to another (Figure 4.2 a). An example is shown in Figure 4.3.4.1 Anchored CustomizationThe Anchored Customization approach, by contrast, promotes a contextual many-to-many mapping of settings to UI elements (Figure 4.1 c). Any UI element canbe used as an anchor\u00E2\u0080\u0094for instance, icons, buttons, menus, even an empty area.The goal of the mapping is to provide context for each setting. One setting canbe mapped to multiple anchors, and multiple settings can be mapped to the sameanchor. The visibility of anchors and thus their associated settings changes dy-namically, depending on the current state of the interface. The Anchored Cus-tomization approach leverages users\u00E2\u0080\u0099 pre-existing knowledge of the applicationUI, instead of requiring them to learn the abstract structure of a settings panel.Hence, the key idea of Anchored Customization is that the application interfaceitself is used to organize and navigate the settings space (Figure 4.2 b).9Figure 4.1: (a) Con\u001Bg \u001Bles are the most direct representation of settings, withoutany particular structure. (b) Settings panels are a static partition of settings, basedon abstract categories. (c) Anchored Customization creates a contextual many-to-many mapping of settings to UI elements. UI elements are represented here bycolored circles.10Figure 4.2: (a) In a settings panel, tabs indicate the current location and allowusers to navigate to a di\u001Derent group of settings. (b) In Anchored Customization,elements of the application UI itself are used to navigate the settings space.11Figure 4.3: The settings panel o\u001Dered in Wunderlist, which was used as the Con-trol condition in Experiments 1 and 2. Tabs appear at the top of the panel, and aredecorated with an icon.12There are two main dimensions in the design space for mechanisms that in-stantiate Anchored Customization: the display of the settings and the display ofthe anchors.4.1.1 Display of SettingsIt is not desirable to permanently show the settings associated with the variousanchors. Most applications o\u001Der many settings, which would clutter the inter-face if they were always visible. Further, customization is typically very much asecondary infrequent activity. Thus, in general, the settings associated with ananchor should be displayed on demand, when the user expresses interest in ananchor\u00E2\u0080\u0094by clicking or hovering on it, for instance. There are many ways to dis-play settings once demanded. We explored three possibilities, inspired by multi-layered interfaces (described below). But the design space is much larger. Someinteresting dimensions include:\u00E2\u0080\u00A2 Which settings should be displayed when clicking on an anchor. Only thesettings mapped to this anchor, or a local neighborhood?\u00E2\u0080\u00A2 How to represent settings. Now that settings are placed in context withinthe interface, the text labels might become super\u001Euous. For instance, short-cuts might not need a textual description if they are mapped clearly to a but-ton that performs the same action. Settings that show or hide an elementmight be better represented by a simple visible/invisible icon. In general,di\u001Derent types of settings (or their associated anchors) could be representeddi\u001Derently.\u00E2\u0080\u00A2 How to provide access to the settings that are not associated with any an-chor.\u00E2\u0080\u00A2 How much of the structure of traditional settings panels should be retained,to help users transition to this new approach to customization.4.1.2 Display of AnchorsThere are a number of di\u001Derent possibilities when considering how the anchorscan be displayed. One familiar approach is through context menus: anchors arenot explicitly marked visually, but can be used by the user to demand a corre-sponding setting. For instance, in Microsoft Word, right-clicking the ribbon showsa contextual menu with an option to \u00E2\u0080\u009CCustomize the ribbon\u00E2\u0080\u009D. A tooltip that ap-pears when hovering over an anchor for a short duration could be used in a simi-lar way. This local, targeted approach answers the question: Is this customizable?13The downside of this approach, however, is that there is no way for the user toget a holistic overview of all the settings available within the UI. Serially invokingthe context menu from many di\u001Derent anchors is too tedious, and the user mustresort to bringing up the settings panel.An alternative approach is to provide a global overview of all the customizationopportunities available, by making all the anchors visible at once, likely througha mode. This approach helps answer the question: What is customizable? Thedownside of this approach is that entering a distinct mode could interrupt theuser\u00E2\u0080\u0099s work\u001Eow \u00E2\u0080\u0094which is why modes should be used sparingly in interactiondesign. The contextual menu approach described above might be more appropri-ate when users can easily guess (or already know) which UI element to click on toperform the desired change. In contrast, the global overview approach helps usersto become aware of the settings available, and where to \u001Bnd them. Fortunately,the two approaches are not mutually exclusive; providing both would allow usersto choose the one best suited to their current need.4.2 Customization LayerWe designed and implemented Customization Layer which instantiates the an-chored customization approach. In the design dimensions described above, thismechanism uses a global overview approach to show all anchors in an extra layeron top of the interface. We explore three ways to display settings on demand,inspired by multi-layered interfaces.4.2.1 Anchors Visible in a LayerIn a layered design, there are multiple ways to indicate which UI elements areanchors. In LemonAid\u00E2\u0080\u0099s contextual help mode [9], the interface was overlaid withyellow question marks that explicitly represent help queries. To avoid clutter, wethought it best not to add additional graphical elements to mark anchors. Instead,the anchors themselves are visually highlighted when users activate a layer ontop of the regular application (shown in Figure 4.4). In our implementation, theapp interface becomes darker and less saturated, but is still clearly visible throughthe Customization Layer. Anchors are shown with a white background and darkgray text, to optimize legibility and ensure visibility above the dimmed applicationinterface (Figure 4.5).When the user hovers over an anchor A, that anchor and all other anchorsmapped to the same settings as A are highlighted in orange (Figure 4.5). Thislinked highlighting helps users create a mental model of the mapping of settingsto UI elements. For instance, in a PTM app, hovering over a button highlights all14the buttons that share the same setting (e.g., a keyboard shortcut for marking a\u00E2\u0080\u009Ctodo\u00E2\u0080\u009D as important).There is one important special case that required attention when anchoringsettings to UI elements: what if a setting governs the visibility of the UI anchoritself? Consider for instance settings that show or hide buttons in a toolbar. Ofcourse it makes sense to anchor such a setting to the button it a\u001Dects; but afterhiding the button, how can users access its associated setting to make the buttonvisible again? To avoid this problem, we introduce the notion of ghost anchors:anchors that correspond to hidden elements in the application UI. These anchorso\u001Der the same functionality as regular anchors, but are displayed in a darker grayto indicate their transient status.Displaying all ghost anchors at the same time is not necessarily desirable. Ifan application has many optional, hideable components, showing all of them asghost anchors could break the application layout, clutter the customization layer,and overwhelm the user. Furthermore, the customization layer should look assimilar as possible to the current interface of the application, to help users alter-nate seamlessly between the two. For these reasons, we implemented a collapsingmechanism: ghost anchors are by default represented by a simple \u00E2\u0080\u009Cchevron\u00E2\u0080\u009D iconin the customization layer (Figure 4.6). To further reduce visual clutter, ghostanchors that are close to each other are collapsed under the same chevron icon,based on a simple proximity clustering algorithm. Clicking on a chevron iconreveals the ghosts that it was previously hiding.4.2.2 Three Variants for Displaying SettingsWhen the Customization Layer is shown, clicking on an anchor displays the sub-set of settings anchored to that UI element. We explored three visual representa-tions of this subset (see Figure 4.7). The Minimal panel (M) only shows the settingsin the subset, with an orange background to indicate that they are related to theanchor that was clicked. Minimal+Context (M+C) is very similar, but a tab namenext to each setting indicates from which tab of the full settings panel it comesfrom. In contrast, Full+Highlight (F+H) keeps the structure of the complete set-tings panel, and the settings from the anchor\u00E2\u0080\u0099s subset are highlighted in orange,as are the tabs in which they appear. Some tabs contain more settings than can beshown in the height of the panel, so we automatically scroll down to bring intoview the \u001Brst highlighted setting, if any.Because some settings might not be anchorable to the interface (cf. Feasibilitysection), all designs must provide a fallback access to the complete set of settings.In Full+Highlight, users can browse all the tabs freely, as they would in a regularsettings panel. In Minimal+Context, the tab names next to each setting are ac-15Figure 4.4: The Wunderlist web application. Todos appear in a vertical list, andcan be \u00E2\u0080\u009Cstarred\u00E2\u0080\u009D to convey importance. Here the user has starred the second todoand given it a due date. In the left sidebar are a collection of \u00E2\u0080\u009CSmart Lists\u00E2\u0080\u009D, intelli-gent \u001Blters that display a list of todos \u001Bltered by a particular criteria. For instance,a Smart List can show only the todos due today, or due this week, or all the to-dos already completed. There are also user-de\u001Bned todo lists, such as Groceries,Travel, and Work.Figure 4.5: Customization Layer displayed on top of Wunderlist. The user is cur-rently hovering over the bottom \u00E2\u0080\u009Cstar\u00E2\u0080\u009D button on the right-hand side of the screen;as a result, all star buttons are highlighted in orange because they share the samesettings. Clicking on an anchor displays the settings associated with it\u00E2\u0080\u0094here,shown in the Minimal panel.16Figure 4.6: (1) The user selects the \u00E2\u0080\u009CWeek\u00E2\u0080\u009D \u001Blter, then disables it (not shown). (2)The corresponding anchor is collapsed under a chevron icon. (3) Clicking on thechevron reveals the \u00E2\u0080\u009CWeek\u00E2\u0080\u009D \u001Blter as a ghost anchor.Figure 4.7: Minimal, Minimal+Context, and Full+Highlight variants showing thesettings anchored to the \u00E2\u0080\u009CWeek\u00E2\u0080\u009D Smart List. Two settings are mapped to thisSmart List: the \u001Brst changes the shortcut for opening this Smart List; the seconddetermines whether this Smart List is visible or hidden in the main interface.17tual buttons; clicking on them opens the full panel at the corresponding tab. InMinimal, a \u00E2\u0080\u009Cshow all\u00E2\u0080\u009D hyperlink is provided at the bottom of the mini panel. Bydefault, it opens the full panel at the tab that contains the most settings from theanchor\u00E2\u0080\u0099s subset. Users can return to the minimal panel by clicking a backwardarrow at the top left of the full panel.These three variants re\u001Eect di\u001Derent points in the multi-layered interface de-sign space [33]. The two minimal variants only show the anchor\u00E2\u0080\u0099s subset of thesettings initially, while Full+Highlight uses visual highlighting to distinguish thesubset from all other settings. All three designs are intended (to varying degrees)to reduce complexity, speed up access to the most relevant settings, and help theuser become aware of the full set of settings. Hence, they re\u001Eect di\u001Derent trade-o\u001Ds between a minimal subset of settings and the full settings panel.As with any multi-layered interface, transitioning from a lower layer to anupper layer can be challenging for users [33]. In our implementation, we pro-vide animated transitions between the minimal variants and the full panel to helpusers understand the relationship between the two. The minimal panels expandsmoothly to the full size, while the highlighted settings glide into their new posi-tion. The remaining (non-highlighted) settings fade in afterwards. Informal pilotsfound these relatively simple animations to be helpful for conveying the intendedmental model.As an additional way to help users understand the mapping between settingsand anchors, we augmented the full settings panel with reverse highlighting: hov-ering on any setting displays a blue halo around the corresponding anchors in theinterface (Figure 4.8). The idea is to allow users to explore the settings\u00E2\u0086\u0092 anchorsrelationship, whereas clicking on anchors and seeing anchors being highlightedin orange always follows the anchors\u00E2\u0086\u0092 settings direction.4.2.3 SearchText search is another e\u001Dective approach for accessing settings. It essentially pro-vides a way to \u00E2\u0080\u009Cjump\u00E2\u0080\u009D to a particular setting without having to locate an anchor orbrowse the settings space. Search is readily a\u001Dorded by traditional con\u001Bg \u001Bles, butrarely o\u001Dered in settings panels\u00E2\u0080\u0094except in complex software such as Eclipse. Thedownside with search is that the vocabulary problem [19] applies, as users canonly guess search terms [4]. Furthermore, relying on search may hinder users\u00E2\u0080\u0099ability to learn other settings. Although not yet implemented, CustomizationLayer is compatible with text search: UI anchors could be \u001Bltered out if theirassociated settings don\u00E2\u0080\u0099t match the search query, or visually highlighted if theydo.18Figure 4.8: The user has clicked on the \u00E2\u0080\u009CStarred\u00E2\u0080\u009D Smart List anchor. As a result,both the anchor and its associated settings are highlighted in orange. The useris now hovering on a di\u001Derent setting in the full panel\u00E2\u0080\u0094the visibility of the \u00E2\u0080\u009CAll\u00E2\u0080\u009DSmart List. This setting and all the anchors associated with it are \u00E2\u0080\u009Creverse high-lighted\u00E2\u0080\u009D with a blue halo to create a visual relationship between them.194.2.4 Software ArchitectureOur software architecture was designed to be app-independent and extensible toother web apps. Chapter 5 explains why we chose web apps, and Wunderlist inparticular, as a concrete starting point for design and evaluation. Adding our Cus-tomization Layer to a web app requires: (1) an API to read and write the settings,and (2) a mapping between settings and visual elements of the interface, providedby the designers of the app. UI elements are represented in the mapping by CSSselectors, which allows among other things selecting many similar anchors via asingle CSS class (Appendix A). Since designers are already familiar with the set-tings of their app, creating the mapping should take at most a few hours, exceptfor very large applications.4.2.5 ImplementationSince Wunderlist doesn\u00E2\u0080\u0099t provide an API to its settings, we reverse-engineeredits front-end to access the underlying settings directly. Fortunately the settingsthemselves had human-readable names, which allowed us to localize and modifythem programmatically. We then manually mapped these settings to appropriateelements of the interface, which took about two hours. From this mapping, ourcode automatically generates anchors by creating copies of the DOM elementsmatched by the CSS selectors provided. Anchors are then positioned precisely ontop of the original element, creating the illusion that the elements themselves arehighlighted.When users click on an anchor, we used Angular.js [21], a front-end JavaScriptframework, to dynamically \u001Blter the list of settings shown in Minimal and Min-imal+Context, and to highlight the appropriate settings in Full+Highlight. Thesmooth transitions between the minimal and full panels were not straightforwardto implement, since the start and end positions must be computed in advance foreach setting, before the animation starts.We also prototyped a simple point-and-click web tool that could help design-ers without web programming skills to retrieve CSS selectors for the UI elementsthey are pointing at.20Chapter 5EvaluationWe focus our evaluation on assessing the usability of our Customization Layer.Given the novelty of this new mechanism, the two experiments presented herewere intended to be exploratory. We certainly hoped that Customization Layerwould help participants \u001Bnd settings faster than browsing a traditional settingspanel; but which of the three variants would be the fastest or most preferred wasunknown. Further, we anticipated that the three variants would impact users\u00E2\u0080\u0099awareness of the full set of settings di\u001Derently: the more contextual informationa design provides, the greater the user\u00E2\u0080\u0099s awareness should be [15].Choice of the ApplicationAlthough our design approach is applicable to any software with a GUI and cus-tomizable via a settings panel, we had to choose a particular application on whichto evaluate it. We considered building a generic PTM app from the common fea-tures and visual styles we observed in our systematic application review (Sec-tion 3). The bene\u001Bt of building a generic app is increased generalizability, sincethe particularities of each app are averaged out. However, we would have had tohand-pick the customization settings available in this generic app, a process thatcould introduce some bias in the experiment. Ultimately, we thought it best toevaluate Customization Layer within an existing application with its actual set-tings. This choice maximizes the ecological validity of our experiment.We chose Wunderlist [20], one of the most popular PTM apps today (over 10million downloads on Android and iPhone) which o\u001Ders a well-designed web in-terface. Of the 20 apps from our systematic review, Wunderlist had a particularlyclean settings panel, with numerous and varied settings. It therefore appearedto be a good baseline for a fair comparison. Furthermore, we were able to accessWunderlist\u00E2\u0080\u0099s settings directly on the web client, e\u001Dectively bypassing the settingspanel to customize the app in real time.215.1 Experiment 1: Remote Mechanical TurkThe primary goal of this experiment was to evaluate the performance of our Cus-tomization Layer for changing settings, compared to a traditional settings panel.We used a between-subject design to avoid negative carry-over e\u001Dects, becauseswitching back-and-forth between two very di\u001Derent mental models could havebeen confusing for participants. We deployed this experiment on Amazon Me-chanical Turk (MTurk), an online crowdsourcing platform that facilitates the re-cruitment of a large number of participants.5.1.1 MethodParticipantsAll 48 participants (aged 19-60, median 28.5, 16 females) were regular computerusers, and none had tried Wunderlist before. We had replaced 3 participants, whowere either 2.5 Inter-Quartile Range slower than others in their condition, or did2 IQR more errors than everyone else.TaskThe experiment consisted of a sequence of settings changes. At the beginningof each trial, a popup instructed participants to change one setting to a givenvalue (Figure 5.1). Pressing a \u00E2\u0080\u009CGo!\u00E2\u0080\u009D button would close the popup and start thetimer. The instructions were written in layman\u00E2\u0080\u0099s terms, and did not necessarilyuse the same words as the label of the target setting (Appendix B.1). For example,the instruction \u00E2\u0080\u009CChange shortcut for checking o\u001D todos\u00E2\u0080\u009D applied to the setting\u00E2\u0080\u009CMark selected do-dos as completed\u00E2\u0080\u009D in the Shortcuts tab. To help ensure thatparticipants read the instructions before starting the trial, the \u00E2\u0080\u009CGo!\u00E2\u0080\u009D button waskept inactive for the amount of time required to read the instructions at an averagereading speed. During that time, the settings panel (in Control) or the UI anchors(in Customization Layer) were hidden, to prevent participants from planning theiractions before the timer was started.During the trial itself the instructions were visible at the top of the window,in case participants had forgotten them (Figure 5.2). Participants had to changeat least one setting before the \u00E2\u0080\u009CNext\u00E2\u0080\u009D button became available in the bar at thetop of the screen. Clicking this button would stop the timer, indicate whether thetask had been successfully completed, and move on to the next trial. We enforceda 2-minute time limit for each trial, after which the trial was marked as a timeoutand participants were taken to the next trial. If participants made mistakes, the22Figure 5.1: Modal dialog displayed at the beginning of each trial. The instruc-tions are repeated in a persistent progress bar at the top of the window, whichalso shows the bonus reward earned so far. Neither the settings panel nor the UIanchors are visible until participants press the \u00E2\u0080\u009CGo!\u00E2\u0080\u009D button to start the trial.settings that had been changed incorrectly were reset at the end of the trial, andthe target setting changed to the correct value.Our Customization Layer and the settings panel used in the Control Condi-tion are both accessed via the menu at the top-left of the screen, by clicking on\u00E2\u0080\u009CCustomize\u00E2\u0080\u009D or \u00E2\u0080\u009CSettings\u00E2\u0080\u009D respectively. Since activating the customization mech-anism takes exactly the same amount of time in both cases, we didn\u00E2\u0080\u0099t include thistedious step in the experiment, and instead opened the appropriate customizationmechanism automatically in all conditions.MeasuresThe duration of a trial was measured between the time when participants pressedthe \u00E2\u0080\u009CGo!\u00E2\u0080\u009D button and when they changed a setting. At the end of the experiment,participants completed two recognition questionnaires (Appendix B.3.3) to assessincidental learning. The \u001Brst was a list of 10 tab names: 5 that appeared in theWunderlist settings panel, and 5 that we made up to closely resemble actual tabnames. Participants were asked to indicate for each tab whether or not it appearedin Wunderlist. Each correct answer was worth 1 point, each incorrect answer -1,so that answering at random would result in a score of zero. The second ques-tionnaire was similarly a list of 20 settings: 5 that participants had changed, 5 thatwere present in Wunderlist but that participants hadn\u00E2\u0080\u0099t changed, and 10 made up23Figure 5.2: Possible states of the progress bar, displayed at the top of the window.The \u00E2\u0080\u009CNext\u00E2\u0080\u009D button is disabled until participants change at least one setting. Af-ter changing a setting, participants must click this button to end the trial. If thecorrect setting was changed, a positive feedback is given and the bonus rewardis incremented. Otherwise, participants are told that they made an error and aretaken to the next trial.but plausible. The scoring was identical to the \u001Brst questionnaire. Finally, partic-ipants were asked to rate their satisfaction on a 7-point Likert scale: the extent towhich they liked or disliked the customization mechanism they were using, andhow easy it was to \u001Bnd the settings they were looking for (Appendix B.3.2).ConditionsWe compared four customization mechanisms: the three variants of Customiza-tion Layer (M, M+C, F+H) and the actual settings panel o\u001Dered by Wunderlist(Control), shown in Figure 4.3. While we accurately reproduced the structure ofWunderlist\u00E2\u0080\u0099s panel in our F+H prototype, the visual style was slightly di\u001Derent:Wunderlist generally looks more polished, with custom drop-downs and an iconon each tab.Wunderlist currently o\u001Ders 41 settings in four tabs4. Out of these settings, wediscarded the application language one, as changing it would make the interfaceindecipherable for our participants. The experiment tasks were not equally di\u001C-cult: some settings were indeed harder to \u001Bnd than others, especially if their textlabel was unclear. To reduce the variability between subjects, we partitioned thesettings into two groups of 20, carefully chosen to balance the type and locationof the settings they o\u001Dered. Each participant was assigned one set. To assess anyearly learning, participants were asked to change the same group of 20 settingstwice, but in a di\u001Derent randomized order for each of the blocks of 20.The experiment required changing settings from a default value to a speci\u001Bed4Three other tabs show non-settings information, such as \u00E2\u0080\u009Cupgrade your account\u00E2\u0080\u009D and\u00E2\u0080\u009Cabout this product.\u00E2\u0080\u009D We did include these tabs in our prototype.24one. It was not clear a priori whether the default value of a setting would impactthe ability of users to \u001Bnd and change that setting. For instance, un-ticking acheckbox corresponds to negating its text label, which might be less intuitive thanticking it, following the original meaning of the label. Therefore, we controlledthe default values of the settings: participants either started the experiment withthe defaults o\u001Dered by Wunderlist, or the opposite of these defaults.DesignA 2-factor mixed design was used: 4 customization mechanisms (M, M+C, F+H,Control; between-subjects) x 2 blocks (within-subject). There were two controlvariables, fully counterbalanced between participants: group of settings (group1, group 2) and default settings (normal, opposite). Each participant completed 2blocks of 20 trials, for a total of 1920 trials across all 48 participants.ProcedureAfter accepting the HIT (work assignment in MTurk), participants were asked tocreate a temporary account on Wunderlist, using a randomly-generated email ad-dress. Then participants had to drag a custom bookmarklet (a \u00E2\u0080\u009Cbookmark applet\u00E2\u0080\u009D)to their bookmarks bar, and to click on it to load the experiment code in the Wun-derlist webapp. A series of popups walked participants through a quick tutorial,demonstrating the most useful features of Wunderlist. Participants were thengiven one practice trial, before starting any trial. In the 3 Customization Layerconditions, the practice trial contained a few additional instructions on how touse this new customization mechanism. Finally, participants were asked to com-plete the two recognition tests, to rate the customization mechanism they wereusing, and to complete a simple demographics questionnaire (Appendix B.3.1).The whole procedure lasted 32 minutes on average, and participants received anaverage compensation of $4.34, depending on their performance in the trials andrecognition questionnaires. (Incentivizing participants with a variable bonus re-ward is a standard practice for MTurk studies.)5.1.2 MTurk ResultsWe ran a mixed-design ANOVA on the duration of trials. The data was log-transformed to satisfy the assumption of normality, and we used medians to re-duce the in\u001Euence of outliers. The e\u001Dect sizes reported are generalized eta-square(\u00CE\u00B72G), interpreted as follows: .02, .13, .26 for a small, medium, and large e\u001Dect sizerespectively [1]. As shown on Figure 5.3, there was a main e\u001Dect of customiza-tion mechanism (F3,44 = 3.58, p < .05, \u00CE\u00B72G = 0.17), as well as block (F1,44 = 124.63,25Figure 5.3: Experiment 1: 95% con\u001Bdence intervals of the median duration perCustomization Mechanism and Block. M is signi\u001Bcantly faster than F+H and Con-trol, but not signi\u001Bcantly di\u001Derent from M+C. (N=48)p < .001, \u00CE\u00B72G = 0.32), but no interaction between the two. M was signi\u001Bcantly fasterthan Control (p < .05, 35% faster) and F+H (p < .05, 30% faster). No other pair wassigni\u001Bcantly di\u001Derent. All pairwise comparisons use a Bonferroni correction un-less otherwise mentioned. As anticipated, there was no e\u001Dect of the two controlvariables (the group of settings and default settings used). There were few time-outs and errors across all conditions (20 and 54, respectively, for 1920 trials), andthey had no signi\u001Bcant e\u001Dect on our results.A single-factor ANOVA on the tabs recognition scores found a signi\u001Bcant ef-fect of customization mechanism (F3,44 = 5.82, p < .01, \u00CE\u00B72G = 0.28). M scored lowerthan both F+H (p = .05) and Control (p < .05), by 2 out of 10 points lower on av-erage. No other pair was signi\u001Bcantly di\u001Derent. There was no di\u001Derence either inthe recognition scores for settings, nor in the subjective ratings provided at theend of the experiment. The median rating for ease of use and satisfaction was thesame for each of the four customization mechanisms: 2 on a scale ranging from-3 to +3.5.2 Experiment 2: Face-to-Face in the LabAlthough MTurk provided quick access to a diverse set of participants at a mod-erate cost, conducting remote experiments has limitations in terms of the insightsand qualitative feedback that can be gathered. Thus, we ran a second experimentin a face-to-face lab setting. This time the customization mechanism was treatedas a within-subject factor, to allow participants to make informed comparativejudgments.265.2.1 Method Di\u001Derences from Study 1We recruited 12 participants (age 21-42, median 24.5, 5 females), all regular com-puter users. Two had tried Wunderlist before, but were not using it regularly. Theexperiment task was identical, except that participants were encouraged to talkaloud. The experiment \u001Bt in a single one-hour session, for which participants re-ceived a $10 compensation. Contrary to MTurk, there was not variable monetaryincentive based on performance.A single-factor within-subject design was used, with the same four customiza-tion mechanisms. Participants had to change 10 di\u001Derent settings with each mech-anism, using all the 40 settings available in Wunderlist (Appendix C.1), for a totalof 12 x 40 = 480 trials. All participants started with the same defaults settings,since the previous experiment did not \u001Bnd any e\u001Dect of this control factor. Thethree Customization Layer variants were blocked together, to limit the numberof times participants had to switch between the two mental models. Half of theparticipants began with the Customization Layer block, the other half with theControl condition. Within the Customization Layer block, the presentation or-der of the three variants was fully counterbalanced. The settings were randomlyordered across all blocks.The experiment procedure was similar to the MTurk study, except for a fewchanges re\u001Eecting the within-subject design. After each mechanism condition,participants were asked to rate on a 7-point Likert scale the mechanism on threemetrics: ease of use, perceived speed, and satisfaction (Appendix C.4.2). At the endof the experiment, participants were asked to rank the four mechanisms on thesame metrics (Appendix C.4.3). No recognition questionnaires were administered,since it would have been di\u001Ccult to tease apart the learning that happened in eachcondition. The experiment was concluded by a brief semi-structured interview.Participants were asked about their perception of the four di\u001Derent customizationmechanisms, and we gathered insights on how they developed a mental model ofAnchored Customization.5.2.2 Face-to-Face Lab ResultsWe \u001Brst present the quantitative results of this second experiment, followed bythe qualitative insights we gathered.Quantitative ResultsWe ran a repeated measures ANOVA on the duration of trials. As in Experiment1, the data was log-transformed and we used medians. As shown on Figure 5.4,there was a main e\u001Dect of customization mechanism (F3,33 = 5.61, p < .01, \u00CE\u00B72G =27Figure 5.4: Experiment 2: 95% con\u001Bdence intervals of the median duration perCustomization Mechanism. M+C is signi\u001Bcantly faster than F+H and Control,but not signi\u001Bcantly di\u001Derent from M. (N=12)0.16). M+C was faster than Control (p < .05, 36% faster) and F+H (p = .06, 40%faster). No other pair was signi\u001Bcantly di\u001Derent. The order in which participantssaw the conditions (Control \u001Brst, or Customization Layer \u001Brst) had no signi\u001Bcante\u001Dect either.The subjective ratings collected after each block were analyzed with a Fried-man test. No di\u001Derences were found for satisfaction and ease of use. Perceivedspeed was borderline signi\u001Bcant (\u00CF\u008723 = 7.1, p = .07), with the majority of the dif-ference coming from M+C reported to be faster than F+H (p = .1). In terms of theranking data, participants were almost evenly divided between CustomizationLayer (6 top ranks) and Control (7) \u00E2\u0080\u0094one participant ranked two mechanisms asbest. There was also no clear consensus among which of the three CL variantswas best: with 1, 3, and 2 votes going to M, M+C, and F+H respectively.Qualitative ResultsThe comments made by participants during the experiment and their answersduring the interview were recorded by the experimenter. We report the \u001Bndingsof a thematic analysis [6].Anchored Customization Nine out of 12 participants acquired the intendedmental model of Anchored Customization: 6 acquired it immediately with the \u001Brstvariant of Customization Layer they encountered, the other 3 with the secondvariant. Once participants understood the concept of Anchored Customization,they were able to use it successfully: \u00E2\u0080\u009CI\u00E2\u0080\u0099m used to all the settings hidden away ina menu, but I think this (Minimal) makes a lot of sense\u00E2\u0080\u009D (P5), \u00E2\u0080\u009CI think the pointof this (F+H) is that I don\u00E2\u0080\u0099t need to think under which category each setting is\u00E2\u0080\u009D(P3). In some cases, participants even imagined appropriate anchors that were28not actually present in Wunderlist, which shows a clear grasp of the Anchor Cus-tomization Concept. For instance, P1 said: \u00E2\u0080\u009CI was looking for a printing icon\u00E2\u0080\u009D, whilesearching for the setting \u00E2\u0080\u009Cdon\u00E2\u0080\u0099t print completed todos when printing a list\u00E2\u0080\u009D.Three out of 12 participants did not seem to have acquired the intended men-tal model: in more than 50% of trials, they clicked on any anchor, from whichthey immediately opened the full panel and browsed the tabs to \u001Bnd the desiredsetting. P4, who was in the Minimal condition, even complained: \u00E2\u0080\u009CI don\u00E2\u0080\u0099t like\u00E2\u0080\u0098showing all\u00E2\u0080\u0099 all the time, it\u00E2\u0080\u0099s too many clicks\u00E2\u0080\u009D. These 3 participants were amongthe 6 who started the experiment in the Control condition, which could have neg-atively a\u001Dected their behavior in the subsequent Customization Layer conditions.The only time these participants searched for an appropriate anchor (rather thanjust any anchor) was when the target setting was related to one of the Smart Lists,in the left sidebar (see Figure 4.5). In these cases, participants may have reasonedby analogy with their practice trial, which instructed them to change a (di\u001Derent)Smart Lists setting.Customization Layer Variants Interestingly, at least 3 participants did notnotice any di\u001Derence between M and M+C during the trials, and only realizedthat they were di\u001Derent at the end when they were asked to rank the four cus-tomization mechanisms. Some participants liked the extra structure provided bythe tab names in M+C: \u00E2\u0080\u009CIn Minimal I don\u00E2\u0080\u0099t know where I am in the greater struc-ture. [M+C] is more organized\u00E2\u0080\u009D (P1); \u00E2\u0080\u009CThe links in [M+C] are clearer, you knowwhereyou\u00E2\u0080\u0099re going\u00E2\u0080\u009D (P8).In F+H, at least 4 participants did not see the orange highlighting, althoughit was clearly visible (Figure 4.7). This could be a case of inattentional blind-ness [34]: participants who hadn\u00E2\u0080\u0099t yet acquired the intended mental model mayhave discarded the highlighting because it had no meaning to them. Other par-ticipants thought of the highlighting as a soft suggestion from the system: \u00E2\u0080\u009CAtone point I assumed the highlighting was like a hint\u00E2\u0080\u009D (P8), \u00E2\u0080\u009C[the system] is saying:I think you are looking for this, but I\u00E2\u0080\u0099m not sure\u00E2\u0080\u009D (P12). However, no participantseemed to notice the reverse highlighting, from the settings panel to the anchors.This is a relatively subtle design element, which may not be noticeable in a shortexperiment with constraint tasks.A few participants suggested ideas for yet another variant of the Customiza-tion Layer. The settings would be displayed in an even more minimal panel, show-ing only the anchored settings. Instead of providing a \u00E2\u0080\u009Cshow all\u00E2\u0080\u009D link at the bottomof the panel itself, the full settings panel could be accessed from its usual location,typically via a menu at the top of the screen or a cog icon. Providing a \u001Bxed ac-cess points to all the settings may match users\u00E2\u0080\u0099 mental models better: \u00E2\u0080\u009CIf a setting29doesn\u00E2\u0080\u0099t have an anchor, I would look for a general menu with all settings\u00E2\u0080\u009D (P12). Itwould also address the concern of P10: \u00E2\u0080\u009CWhen clicking on show all, you are notsure that it will show all the settings\u00E2\u0080\u009D \u00E2\u0080\u0094because the \u00E2\u0080\u009Cshow all\u00E2\u0080\u009D link appears in anotherwise very contextualized panel.Wunderlist Although it was not an explicit goal, this experiment revealed someusability problems with Wunderlist\u00E2\u0080\u0099s settings panel, which was used as the Con-trol condition. The most problematic aspect is that 10 settings are hidden undera \u00E2\u0080\u009Cshow more\u00E2\u0080\u009D button in the Shortcuts tab. This design decision was probably in-tended to avoid showing all 20 shortcuts settings at once, by hiding the ones lesslikely to be changed. However, this \u00E2\u0080\u009Cshow more\u00E2\u0080\u009D button was not salient enough,and 10/12 participants missed it the \u001Brst time they looked for one of the settings itwas hiding. Interestingly, this problem was naturally alleviated in all Customiza-tion Layer variants: in both M and M+C, only the relevant settings are shownwhen clicking on an anchor, so there is no need for a hiding mechanism. F+Hautomatically unhides all of the 10 more advanced shortcuts if one of them ishighlighted, and scrolls down to the \u001Brst highlighted setting.The categories chosen by Wunderlist\u00E2\u0080\u0099s designers seemed to generally makesense for our participants. Yet a few settings appeared to be misclassi\u001Bed, whichresulted in longer search times. For instance, the setting \u00E2\u0080\u009Cenable sound for newnoti\u001Bcations\u00E2\u0080\u009D was in the General tab, next to \u00E2\u0080\u009Cenable sound for checking-o\u001D atodo\u00E2\u0080\u009D. However, most participants looked for it \u001Brst under the Noti\u001Bcations tab,which is a more precise category. This kind of ambiguity doesn\u00E2\u0080\u0099t arise in An-chored Customization, since the same setting can be mapped to any meaningfulanchor.Unsurprisingly, we found that the quality of the text labels plays a key rolein helping users \u001Bnd settings. Indeed, participants appear to \u00E2\u0080\u009Cscan\u00E2\u0080\u009D each tab veryquickly, looking for keywords that would match the instructions they were given.Because of the vocabulary problem [19], \u001Bnding the appropriate keywords to de-scribe a setting can be challenging for designers. But once a word is chosen, itshould be used consistently throughout the settings panel. For instance, our par-ticipants were very confused by the fact that noti\u001Bcations are sometimes referredto as \u00E2\u0080\u009Cactivities\u00E2\u0080\u009D in the current version of Wunderlist.Visual Design Our primary concern while designing the Customization Layerwas interaction design, not aesthetics. However, as is often the case in HCI experi-ments, some participants tended to focus on the visual appearance of the di\u001Derentinterfaces more than their behavior: 4/12 participants indicated that they liked thelook of the Wunderlist settings panel better. In particular, the presence of an icon30on each tab was appreciated (P9). This might explain why 7 participants rankedControl as well or above Customization Layer: as P10 puts it, \u00E2\u0080\u009Cfor customers prettyis more important\u00E2\u0080\u009D.The visual design of our Customization Layer was optimized to remain us-able across apps with di\u001Derent visual styles. We chose a dark overlay with whiteanchors to maximize visibility, independently of the color palette of the underly-ing application. P5 found it too dark, and not colorful enough. Furthermore, P5also disliked the fact that the anchors were in some places not well aligned witheach others, or that the elements inside them were not properly centered. This isa consequence of generating anchors from regular UI elements, which were notoriginally designed for that purpose. Of course, for commercial-grade softwarethe appearance of an application\u00E2\u0080\u0099s customization layer could be tweaked by thedesigners to be more aesthetically pleasing.5.3 Secondary Exploratory AnalysesThe fact that we obtained di\u001Derent performance results for M and M+C in ourtwo experiments was surprising and prompted us to probe the data further. Mwas found to be faster than F+H and Control in the \u001Brst study, whereas it wasM+C that was faster than F+H and Control in the second study. In both cases, thep-value and e\u001Dect sizes are similar. A closer look shows that these results are notcontradictory: M and M+C were never found to be signi\u001Bcantly di\u001Derent fromeach other, nor signi\u001Bcantly slower than F+H or Control.In order to tease this apart, we looked at the logged data with a focus to under-stand participants\u00E2\u0080\u0099 approach to \u001Bnding the target setting. Of particular interest iswhether participants found a correct anchor to click on to access the target set-ting (what we call \u00E2\u0080\u009Canchor search\u00E2\u0080\u009D), or if they defaulted to clicking on any anchorfrom which they then browsed the full settings panel (\u00E2\u0080\u009Cfull panel search\u00E2\u0080\u009D). Foreach participant, we computed the percentage of trials in which they found a cor-rect anchor, out of the total number of trials in the experiment. The distribution ofthis metric across all participants is clearly bimodal, both on MTurk and in the lab(Figure 5.5). The position of the \u00E2\u0080\u009Cvalley\u00E2\u0080\u009D (local minimum) of the two distributionsis also similar: 61% on MTurk, 63% in the lab.This analysis points to a likely hidden variable in our data: the search strategyused by participants. Indeed the three Customization Layer conditions vary alongthis extra dimension, as shown in Figure 5.6. While these variations might explainthe di\u001Derence between the two minimal variants and F+H, the distributions forM and M+C are too similar to explain the di\u001Derence in results that we observedbetween the two experiments.31Figure 5.5: Density plots of the distribution of the percentage of trials in whichparticipants selected a \u00E2\u0080\u009Ccorrect\u00E2\u0080\u009D anchor\u00E2\u0080\u0094one associated with the target setting.Participants in the right mode followed an \u00E2\u0080\u009Canchor search\u00E2\u0080\u009D strategy, while partic-ipants in the left mode resorted to a \u00E2\u0080\u009Cfull panel search\u00E2\u0080\u009D.Figure 5.6: Density plots of the percentage of trials in which participants selecteda \u00E2\u0080\u009Ccorrect\u00E2\u0080\u009D anchor, for each Customization Layer variant, on MTurk (top) and inthe lab (bottom). The minimal variants all have a higher right mode (\u00E2\u0080\u009Canchorsearch\u00E2\u0080\u009D), while F+H has higher left mode (\u00E2\u0080\u009Cfull panel search\u00E2\u0080\u009D).32For the next step in our exploratory analysis, we looked only at the data fromparticipants who adopted an \u00E2\u0080\u009Canchor search\u00E2\u0080\u009D strategy to see if we could see anydi\u001Derences between M and M+C. More speci\u001Bcally, we re-ran the ANOVA on trialduration only with participants whose average percentage of correct anchor se-lected was above the local minimum of the bimodal distribution. For the MTurkexperiment, M and M+C were both faster than Control (p < .001 each), and notsigni\u001Bcantly di\u001Derent from each other. For the lab experiment, M and M+C werealso both faster than both F+H and Control (all p < .05). This secondary analy-sis suggests that the results of MTurk and the face-to-face experiments may beconsistent, as long as you take search strategy into account: M and M+C are bothfaster than Control, and not signi\u001Bcantly di\u001Derent from each other. As with anynon-planned analysis, these results need to be interpreted with caution.33Chapter 6DiscussionThe results of our experiments are promising: whether on MTurk or in the lab, oneof the minimal variants was signi\u001Bcantly faster than Control, with a medium e\u001Dectsize; and no variant was slower than Control. These performance improvementswere obtained even though participants were exposed to Anchored Customizationfor the \u001Brst time, and received little information to help them build an appropriatemental model. We now discuss the insights gathered on the three CustomizationLayer variants, and re\u001Eect on the possibilities o\u001Dered by this new customizationmechanism.6.1 Reconciling the Performance ResultsM and M+C performed di\u001Derently in the two experiments. Our secondary anal-yses revealed a likely hidden variable, namely search strategy. For participantsusing the anchor search strategy, the results of the two experiments are consis-tent: M and M+C are both faster than Control, and not signi\u001Bcantly di\u001Derent fromeach other. Hence there must be a signi\u001Bcant di\u001Derence in how the \u00E2\u0080\u009Cfull panelsearch\u00E2\u0080\u009D participants performed in the two experiments which translated into dif-ferent relative performances for M and M+C on MTurk and in the lab. It mightjust be a statistical \u001Euke. For instance, looking at the data shows that on MTurkall 5 of of the M+C participants that did a \u00E2\u0080\u009Cfull panel search\u00E2\u0080\u009D were dispropor-tionately slower than the \u00E2\u0080\u009Cfull panel search\u00E2\u0080\u009D participants in all other conditions.This inconsistency suggests that the randomization of participant assignment toconditions may not have equalized individual di\u001Derences. More research will beneeded to further assess this issue.Few participants in the F+H condition used the \u00E2\u0080\u009Canchor search\u00E2\u0080\u009D strategy: 2/12on MTurk, 5/12 in the lab. It could be that highlighting settings in a panel is nota strong enough cue to help users acquire the Anchored Customization mentalmodel. Since the structure of the settings panels is retained in F+H, there may bea strong transfer e\u001Dect that encourages users to default to the \u00E2\u0080\u009Cfull panel search\u00E2\u0080\u009D,instead of exploring anchors. Since few participants used F+H the way it wasintended, we cannot conclude with certainty on its potential performance: is F+Hnecessarily slower than M and M+C, or can it be as fast when properly used? In34any case, F+H is not slower than Control, so it would not be detrimental. Sincesome participants perceived the highlighting as a hint, F+H could be used as a\u00E2\u0080\u009Csofter\u00E2\u0080\u009D version of Customization Layer for users who might not be comfortablewith the degree of minimalism of the two minimal variants.6.2 Performance/Awareness Trade-O\u001DOn MTurk, M was signi\u001Bcantly faster than F+H and Control, but it also scoredsigni\u001Bcantly lower on the recognition of tab names. This could be interpreted asa performance/awareness trade-o\u001D found in other multi-layered interfaces [15].Yet, users in M did just as well as others in terms of recognizing the settingsthemselves. Thus, the awareness trade-o\u001D here seems to a\u001Dect only awarenessof the structure of the upper layer (the full settings panel), not its content, asthere were no di\u001Derences in recognition scores for the settings themselves. Thisis not entirely surprising: in Customization Layer, all the settings are accessiblevia the minimal panel, albeit from di\u001Derent anchors. By contrast, the personalizedinterfaces studied by Findlater et al. [15] only display a static subset of features inthe \u001Brst layer, while the others were only visible in a di\u001Derent layer.6.3 Applicability to Other Desktop SoftwareWe focused our work on task management applications, and the question remainswhether Anchored Customization could provide similar bene\u001Bts for other typesof software. At one extreme is text editors, which are often highly customiz-able. These editors typically have very few always-visible UI elements, and relymostly on menus and keyboard shortcuts. Thus, they are not well-suited for An-chored Customization. Furthermore, text editor users are generally comfortablecustomizing their software by editing the con\u001Bg \u001Bles directly.The opposite extreme is complex software applications designed for non-technicalusers, such as Adobe Creative Suite. These applications have many widgets thatprovide access to lots of features. Anchoring settings to these various UI ele-ments is possible, but the resulting Customization Layer may be overwhelming.However, the main interface of these applications can itself be overwhelming,especially for new users. Anchored Customization would simply re\u001Eect the com-plexity of the underlying software.356.4 Applicability to Handheld DevicesOur systematic review showed that mobile apps organize their settings in verysimilar ways to desktop applications. Mobile apps generally rely on icons andbuttons for user input, since keyboard shortcuts and extensive menus are notavailable. As such, they are well suited for Anchored Customization. The lim-ited screen real estate would warrant using a minimal variant. The anchoredcustomization mechanism could be activated via a standard application menu,but touchscreens o\u001Der other possibilities: for instance, users could long-press ormulti-tap an anchor to see its associated settings, or use a special standardizedgesture to activate the Customization Layer. We observed that mobile applica-tions generally o\u001Der fewer settings than desktop apps. The introduction of awell-designed customization mechanism could lead to more customizable mobileapps.6.5 Interpretation GulfAnchored Customization was designed to reduce the gulf of execution inherentwith settings panels, but it may also reduce the gulf of interpretation [32]. Intraditional settings panels, users do not always know if modifying a settings hadthe intended e\u001Dect. Anchored Customization mitigates this problem by reducingthe spatial and temporal indirection [3] between settings and the user interface,especially for settings that cause an immediate visual change to their anchors. Inany case, designers should provide an easy \u00E2\u0080\u009Cundo\u00E2\u0080\u009D feature, especially for settingsthat don\u00E2\u0080\u0099t cause an immediate visual change. A simple solution would be to add abackward arrow in the \u00E2\u0080\u009CPreferences\u00E2\u0080\u009D or \u00E2\u0080\u009CCustomize\u00E2\u0080\u009D entry in the usual applicationmenu.6.6 The Developers\u00E2\u0080\u0099 Point of ViewBeyond its bene\u001Bts to users, our Anchored Customization approach may alsochange the way designers and developers think about customization. Althoughthey are not required to change any setting to adopt Anchored Customization,the process of mapping settings to UI elements could have a positive e\u001Dect onthe settings o\u001Dered. For instance, designers might realize that some parts of theinterface have no setting anchored to them, which would highlight a potentialopportunity for providing settings that cater to this area. Creating and labelingsettings may also become faster, since the problem of \u001Bnding appropriate wordsto refer to interface elements is mitigated by the context provided by the anchors.36Finding categories to organize settings into tabs might also become super\u001Euous.6.7 LimitationsWhile our results are promising, our experiments had limitations, which comemostly from the challenges of evaluating customization mechanisms in an arti\u001B-cial setting.Personalization systems are indeed notoriously di\u001Ccult to evaluate. End-usercustomization occurs infrequently in the real world, and at a varying degree de-pending on the individuals, the software they use, and a variety of external fac-tors [27]. Long-term \u001Beld experiments, such as the one conducted by McGrenereet al. [30], best capture these behaviors and measure the e\u001Dects of customizationmechanisms. These \u001Beld experiments are typically long and costly to run, and asa result seldom undertaken. The work presented here did not aim to study com-plex personalization behaviors over time. Instead, we focused on the usability (orlack thereof) of the customization mechanisms present in today\u00E2\u0080\u0099s consumer soft-ware. Evaluating the long-term e\u001Dects of the Anchored Customization approachremains future work.Furthermore, we compared customization mechanisms mainly on how quicklyparticipants could \u001Bnd a designated setting. However, in a real world situation,the awareness (or lack thereof) of which settings are available likely plays an im-portant role. In traditional settings panels, awareness is gained by serendipitousdiscovery (also referred to as \u00E2\u0080\u009Cincidental learning\u00E2\u0080\u009D [15]) : users happen to noticea setting of interest while searching the panel for another one. Serendipity canalso happen in Anchored Customization, but the notion of proximity is relative tothe anchors, instead of the settings panel\u00E2\u0080\u0099s structure. Because our experiment taskwas time-constrained, these di\u001Derent forms of serendipity were not well captured.To maximize ecological validity, we used Wunderlist\u00E2\u0080\u0099s actual settings panel asa baseline for the Control condition. But some participants focused their feedbackon its high quality visual design relative to our prototyped Customization Layervariants. Recreating this panel in the same visual style as our prototypes mighthave increased the internal validity of our experiments.In retrospect, the practice trial that participants were given could have beenmore e\u001Dective at conveying the intended model. Participants performed only onetrial, thus only had to click on one anchor to complete it. However, to reallyunderstand the concept of Anchored Customization, one must click on at leasttwo anchors to see that the settings o\u001Dered are di\u001Derent. Some participants tooktime during the practice trial to explore the Customization Layer on their own,clicking on multiple anchors to see the outcome. This free exploration seemed37more e\u001Dective than our practice trial, and would also be more similar to real-world conditions.Finally, our experiments only included one application with a medium numberof settings. It is possible that the number of settings o\u001Dered by an app a\u001Dects therelative performance of Customization Layer and settings panels.38Chapter 7Conclusions and Future WorkAnchored Customization is an approach that places settings in context within theapplication interface, so that users are not required to learn the abstract structureof a settings panel. A systematic review of a set of Personal Task Managementapps found that approximately 84% of the settings could be anchored in the UI.Our Customization Layer prototype reveals all the anchors as a\u001Dordances for cus-tomization. We designed three variants of Customization Layer based on multi-layered interfaces, and implemented these variants on top of a popular web ap-plication for task management, Wunderlist. Two experiments (Mechanical Turkand face-to-face) showed that the two minimalist variants were 35-36% faster thanWunderlist\u00E2\u0080\u0099s settings panel. A majority of users acquired the intended mentalmodel of Anchored Customization, and were able to use it successfully for \u001Bnd-ing a variety of settings in Wunderlist. Some users did not seem to acquire theintended mental model, yet they were able to complete the experiment tasks in thefamiliar structure of a settings panel, thanks to the fallback access we provided ineach Customization Layer variant. Better instructions and more guidance mightbe needed to help these users understand the Anchored Customization approach.7.1 Long-Term E\u001DectsEvaluating the long term e\u001Dects of this customization approach remains futurework. A longitudinal \u001Beld study would determine if a more usable customizationmechanism actually increases users\u00E2\u0080\u0099 likelihood to customize. It could also addresssubtle, but important questions, such as: are di\u001Derent variants of CustomizationLayer better suited to users with di\u001Derent level of expertise? Given the opportu-nity to use multiple variants, as well as a direct right-click access to any anchor\u00E2\u0080\u0099ssettings, which one would users choose? Would it depend on the users\u00E2\u0080\u0099 famil-iarity with the application and its settings, or is it simply a matter of personalpreference?Our prototype could be distributed to real users of Wunderlist as a browserextension, or adapted to other applications to compare the e\u001Dect of di\u001Derent typesof settings and di\u001Derent application domains. Augmenting multiple applicationswith our Customization Layer would also help to verify our assumption that An-39chored Customization requires only one-time learning: once this approach is un-derstood in the context of one particular app, the mental model should be trans-ferable to other apps.7.2 Generating the MappingCurrently, the designers of an app need to provide the mapping between settingsand UI elements. With the growing popularity of advanced front-end frameworksin web development, it might be possible to use code analysis techniques to gen-erate part of the mapping automatically. The interface could be analyzed to deter-mine which elements are a\u001Dected by a setting (e.g. changing their visibility, posi-tion, or appearance), and which elements trigger a function that is itself modi\u001Bedby a setting (e.g. changing the e\u001Dect of clicking a button). The UI elements identi-\u001Bed would automatically become anchors for the appropriate settings. Designersthen only need to verify and extend this mapping, anchoring extra settings byexploiting mental associations\u00E2\u0080\u0094to complement the functional associations thatwere detected programmatically.Another possibility would be to involve users in the mapping process. Con-trary to crowdsourced contextual help [8], users cannot be expected to generatethe entire mapping themselves, but they could tweak a designer\u00E2\u0080\u0099s mapping to bet-ter match their expectations. The idea of re\u001Bning the mapping by aggregating datafrom individual users could be expanded to other customization opportunities aswell. For instance, the most popular extensions and plugins could be anchored tothe UI.7.3 Concluding RemarksEnd-user customization is fundamentally a di\u001Ccult trade-o\u001D between upfrontcosts and uncertain bene\u001Bts in the future. Users weigh the time and e\u001Dort neededto customize against the satisfaction of using an interface matching their owntaste and preferences, and the performance gains of using a software adapted totheir work patterns. For developers and designers, providing multiple versions ofan application can require signi\u001Bcant additional development costs, which can bedi\u001Ccult to justify in an agile world where the time-to-market is critical. And thesee\u001Dorts might eventually be in vain if users never change the default settings.Settings panels represent an equilibrium in this trade-o\u001D space. Their genericUIs are inexpensive to develop, but their lack of appeal and numerous usabilityissues can repel users. As a result, few users customize their applications, or doso only rarely. This in turn does not encourage developers to provide better cus-40tomization mechanisms, and so on. Introducing a new customization mechanismtherefore requires careful consideration of its costs and bene\u001Bts.Much of the prior work reported in the literature has focused on increasingthe potential bene\u001Bts of customization, by proposing more powerful or more ex-pressive customization mechanisms. But these expected bene\u001Bts do not neces-sarily justify the additional costs incurred by users or developers. By contrast,Anchored Customization minimizes the extra costs for developers by leveragingthe customization opportunities already available in existing settings panels. Italso reduces the cost of customization for end-users, who can \u001Bnd desired set-tings faster without having to learn the abstract structure of a settings panel. Assuch, this novel approach to customization has the potential to be adopted widely.Our ultimate goal is to increase users\u00E2\u0080\u0099 satisfaction by interacting with applicationsbetter suited to their own needs and preferences.41Bibliography[1] Roger Bakeman. Recommended e\u001Dect size statistics for repeated measuresdesigns. Behavior Research Methods, 37(3):379\u00E2\u0080\u0093384, 2005.[2] Patrick Baudisch, Desney Tan, Maxime Collomb, Dan Robbins, Ken Hinck-ley, Maneesh Agrawala, Shengdong Zhao, and Gonzalo Ramos. Phosphor:Explaining transitions in the user interface using afterglow e\u001Dects. In Pro-ceedings of the 19th Annual ACM Symposium on User Interface Software andTechnology, UIST \u00E2\u0080\u009906, pages 169\u00E2\u0080\u0093178, New York, NY, USA, 2006. ACM.[3] Michel Beaudouin-Lafon. Instrumental interaction: An interaction modelfor designing post-WIMP user interfaces. In Proceedings of the SIGCHI Con-ference on Human Factors in Computing Systems, CHI \u00E2\u0080\u009900, pages 446\u00E2\u0080\u0093453,New York, NY, USA, 2000. ACM.[4] Ofer Bergman, Ruth Beyth-Marom, Ra\u001B Nachmias, Noa Gradovitch, andSteve Whittaker. Improved search engines and navigation preference in per-sonal information management. ACM Transactions on Information Systems,26(4):20:1\u00E2\u0080\u009320:24, October 2008.[5] Anastasia Bezerianos, Pierre Dragicevic, and Ravin Balakrishnan.Mnemonic rendering: An image-based approach for exposing hiddenchanges in dynamic displays. In Proceedings of the 19th Annual ACMSymposium on User Interface Software and Technology, UIST \u00E2\u0080\u009906, pages159\u00E2\u0080\u0093168, New York, NY, USA, 2006. ACM.[6] Virginia Braun and Victoria Clarke. Using thematic analysis in psychology.Qualitative Research in Psychology, 3(2):77\u00E2\u0080\u0093101, 2006.[7] Andrea Bunt, Cristina Conati, and Joanna McGrenere. Supporting interfacecustomization using a mixed-initiative approach. In Proceedings of the 12thInternational Conference on Intelligent User Interfaces, IUI \u00E2\u0080\u009907, pages 92\u00E2\u0080\u0093101,New York, NY, USA, 2007. ACM.[8] Parmit K. Chilana, Andrew J. Ko, and Jacob O. Wobbrock. LemonAid:Selection-based crowdsourced contextual help for web applications. In Pro-42ceedings of the SIGCHI Conference on Human Factors in Computing Systems,CHI \u00E2\u0080\u009912, pages 1549\u00E2\u0080\u00931558, New York, NY, USA, 2012. ACM.[9] Parmit K. Chilana, Andrew J. Ko, Jacob O. Wobbrock, and Tovi Grossman. Amulti-site \u001Beld study of crowdsourced contextual help: Usage and perspec-tives of end users and software teams. In Proceedings of the SIGCHI Confer-ence on Human Factors in Computing Systems, CHI \u00E2\u0080\u009913, pages 217\u00E2\u0080\u0093226, NewYork, NY, USA, 2013. ACM.[10] Matjaz Debevc, Beth Meyer, Dali Donlagic, and Rajko Svecko. Design andevaluation of an adaptive icon toolbar. User Modeling and User-Adapted In-teraction, 6(1):1\u00E2\u0080\u009321, 1996.[11] Morgan Dixon and James Fogarty. Prefab: Implementing advanced behav-iors using pixel-based reverse engineering of interface structure. In Proceed-ings of the SIGCHI Conference on Human Factors in Computing Systems, CHI\u00E2\u0080\u009910, pages 1525\u00E2\u0080\u00931534, New York, NY, USA, 2010. ACM.[12] James R. Eagan, Michel Beaudouin-Lafon, and Wendy E. Mackay. Crackingthe cocoa nut: User interface programming at runtime. In Proceedings ofthe 24th Annual ACM Symposium on User Interface Software and Technology,UIST \u00E2\u0080\u009911, pages 225\u00E2\u0080\u0093234, New York, NY, USA, 2011. ACM.[13] W. Keith Edwards, Scott E. Hudson, Joshua Marinacci, Roy Rodenstein,Thomas Rodriguez, and Ian Smith. Systematic output modi\u001Bcation in a 2Duser interface toolkit. In Proceedings of the 10th Annual ACM Symposium onUser Interface Software and Technology, UIST \u00E2\u0080\u009997, pages 151\u00E2\u0080\u0093158, New York,NY, USA, 1997. ACM.[14] Leah Findlater and Joanna McGrenere. Impact of screen size on performance,awareness, and user satisfaction with adaptive graphical user interfaces. InProceedings of the SIGCHI Conference on Human Factors in Computing Sys-tems, CHI \u00E2\u0080\u009908, pages 1247\u00E2\u0080\u00931256, New York, NY, USA, 2008. ACM.[15] Leah Findlater and Joanna McGrenere. Beyond performance: Feature aware-ness in personalized interfaces. International Journal of Human-ComputerStudies, 68(3):121 \u00E2\u0080\u0093 137, 2010.[16] Gerhard Fischer. Adaptive User Interfaces, chapter Shared Knowledge in Co-operative Problem-Solving Systems \u00E2\u0080\u0093 Integrating Adaptive and AdaptableComponents, pages 49\u00E2\u0080\u009368. Elsevier Science Publishers B.V., 1993.43[17] Gerhard Fischer and Elisa Giaccardi. Meta-design: A framework for thefuture of end-user development. In Henry Lieberman, Fabio Patern\u00C3\u00B2, andVolker Wulf, editors, End User Development, volume 9 of Human-ComputerInteraction Series, pages 427\u00E2\u0080\u0093457. Springer Netherlands, 2006.[18] Stephen Fitchett, Andy Cockburn, and Carl Gutwin. Improving navigation-based \u001Ble retrieval. In Proceedings of the SIGCHI Conference on Human Factorsin Computing Systems, CHI \u00E2\u0080\u009913, pages 2329\u00E2\u0080\u00932338, New York, NY, USA, 2013.ACM.[19] G. W. Furnas, T. K. Landauer, L. M. Gomez, and S. T. Dumais. The vocabularyproblem in human-system communication. Communications of the ACM,30(11):964\u00E2\u0080\u0093971, November 1987.[20] 6 Wunderkinder GmbH. Wunderlist. Version 3.12.3, 2015. http://www.wunderlist.com (Retrieved on 2015-07-01).[21] Google. Angular.js. Version 1.3.11, 2015. http://angularjs.org/ (Re-trieved on 2015-01-26).[22] Mona Haraty, Diane Tam, Shathel Haddad, Joanna McGrenere, and Char-lotte Tang. Individual di\u001Derences in personal task management: A \u001Beld studyin an academic setting. In Proceedings of Graphics Interface 2012, GI \u00E2\u0080\u009912, pages35\u00E2\u0080\u009344, Toronto, Ont., Canada, Canada, 2012. Canadian Information Process-ing Society.[23] Eric Horvitz, Jack Breese, David Heckerman, David Hovel, and Koos Rom-melse. The Lumi\u00C3\u00A8re project: Bayesian user modeling for inferring the goalsand needs of software users. In Proceedings of the Fourteenth Conference onUncertainty in Arti\u001Bcial Intelligence, UAI\u00E2\u0080\u009998, pages 256\u00E2\u0080\u0093265, San Francisco,CA, USA, 1998. Morgan Kaufmann Publishers Inc.[24] J. Johnson, T.L. Roberts, W. Verplank, D.C. Smith, C.H. Irby, M. Beard, andK. Mackey. The Xerox Star: a retrospective. Computer, 22(9):11\u00E2\u0080\u009326, Sept1989.[25] Helge Kahler, Anders M\u00C3\u00B8rch, Oliver Stiemerling, and Volker Wulf. Tai-lorable systems and cooperative work. Computer Supported CooperativeWork (CSCW), 9(1):1\u00E2\u0080\u00934, 2000.[26] Henry Lieberman, Fabio Patern\u00C3\u00B2, Markus Klann, and Volker Wulf. End-user development: An emerging paradigm. In Henry Lieberman, Fabio Pa-tern\u00C3\u00B2, and Volker Wulf, editors, End User Development, volume 9 of Human-Computer Interaction Series, pages 1\u00E2\u0080\u00938. Springer Netherlands, 2006.44[27] Wendy E. Mackay. Triggers and barriers to customizing software. In Pro-ceedings of the SIGCHI Conference on Human Factors in Computing Systems,CHI \u00E2\u0080\u009991, pages 153\u00E2\u0080\u0093160, New York, NY, USA, 1991. ACM.[28] Thomas W. Malone, Kum-Yew Lai, and Christopher Fry. Experiments withOval: A radically tailorable tool for cooperative work. ACM Transactions onInformation Systems, 13(2):177\u00E2\u0080\u0093205, April 1995.[29] Justin Matejka, Tovi Grossman, and George Fitzmaurice. Patina: Dynamicheatmaps for visualizing application usage. In Proceedings of the SIGCHIConference on Human Factors in Computing Systems, CHI \u00E2\u0080\u009913, pages 3227\u00E2\u0080\u00933236, New York, NY, USA, 2013. ACM.[30] Joanna McGrenere, Ronald M. Baecker, and Kellogg S. Booth. An evaluationof a multiple interface design solution for bloated software. In Proceedingsof the SIGCHI Conference on Human Factors in Computing Systems, CHI \u00E2\u0080\u009902,pages 164\u00E2\u0080\u0093170, New York, NY, USA, 2002. ACM.[31] Xiaojun Meng, Shengdong Zhao, Yongfeng Huang, Zhongyuan Zhang,James Eagan, and Ramanathan Subramanian. WADE: Simpli\u001Bed GUI add-on development for third-party software. In Proceedings of the 32nd AnnualACM Conference on Human Factors in Computing Systems, CHI \u00E2\u0080\u009914, pages2221\u00E2\u0080\u00932230, New York, NY, USA, 2014. ACM.[32] Donald A. Norman. Interfacing thought: Cognitive aspects of human-computer interaction. chapter Cognitive Engineering\u00E2\u0080\u0094Cognitive Science,pages 325\u00E2\u0080\u0093336. MIT Press, Cambridge, MA, USA, 1987.[33] Ben Shneiderman. Promoting universal usability with multi-layer interfacedesign. In Proceedings of the 2003 Conference on Universal Usability, CUU \u00E2\u0080\u009903,pages 1\u00E2\u0080\u00938, New York, NY, USA, 2003. ACM.[34] Daniel J Simons and Christopher F Chabris. Gorillas in our midst: Sustainedinattentional blindness for dynamic events. Perception-London, 28(9):1059\u00E2\u0080\u00931074, 1999.[35] Gunnar Stevens and Torben Wiedenh\u00C3\u00B6fer. Chic - a pluggable solution forcommunity help in context. In Proceedings of the 4th Nordic Conference onHuman-computer Interaction: Changing Roles, NordiCHI \u00E2\u0080\u009906, pages 212\u00E2\u0080\u0093221,New York, NY, USA, 2006. ACM.[36] Wolfgang Stuerzlinger, Olivier Chapuis, Dusty Phillips, and Nicolas Rous-sel. User interface fa\u00C3\u00A7ades: Towards fully adaptable user interfaces. In Pro-45ceedings of the 19th Annual ACM Symposium on User Interface Software andTechnology, UIST \u00E2\u0080\u009906, pages 309\u00E2\u0080\u0093318, New York, NY, USA, 2006. ACM.[37] Gaurav Paruthi Tao Dong, Mark S. Ackerman, Mark W. Newman. SocialOverlays: Collectively Making Websites More Usable. volume 8120 of Lec-ture Notes in Computer Science. Springer Berlin Heidelberg, Berlin, Heidel-berg, 2013.[38] Volker Wulf and Bj\u00C3\u00B6rn Golombek. Direct activation: A concept to encouragetailoring activities. Behaviour & Information Technology, 20(4):249\u00E2\u0080\u0093263, 2001.46Appendix AMapping of settings to UIelements in WunderlistThis appendix shows the JSON \u001Ble that was used to encode the mapping of set-tings to UI elements in Wunderlist\u00E2\u0080\u0099s web application. UI elements are representedby a CSS selector, which usually consists of CSS classes (starting with a \u00E2\u0080\u009C.\u00E2\u0080\u009D) and ids(starting with a \u00E2\u0080\u009C#\u00E2\u0080\u009D). A comma can be used to concatenate multiple CSS selectorsinto one. The settings associated with each anchor are speci\u001Bed via an array ofsettings names, which were chosen by the developers of Wunderlist.1 [{2 \"selector\": \".addTask -input\",3 \"settings\": [\"new_task_location\", \"shortcut_add_new_task\"]4 },5 {6 \"selector\": \".starred -wrapper, .star -wrapper\",7 \"settings\": [\"behavior_star_tasks_to_top\", \"shortcut_mark_task_starred\"]8 },9 {10 \"selector\": \".taskItem -duedate, .detail -date .token_0\",11 \"settings\": [\"date_format\", \"time_format\", \"start_of_week\"]12 },13 {14 \"selector\": \".detail -checkbox.checkBox, .taskItem -checkboxWrapper.checkBox\",15 \"settings\": [\"sound_checkoff_enabled\", \"shortcut_mark_task_done\"]16 },17 {18 \"selector\": \".taskItem -titleWrapper\",19 \"settings\": [\"show_subtask_progress\", \"shortcut_select_all_tasks\", \"shortcut_copy_tasks\", \"shortcut_paste_tasks\"]20 },21 {22 \"selector\": \".filters -collection .sidebarItem[rel=\u00E2\u0080\u0099inbox \u00E2\u0080\u0099]\",23 \"settings\": [\"shortcut_goto_inbox\"]24 },4725 {26 \"selector\": \".filters -collection .sidebarItem[rel=\u00E2\u0080\u0099all \u00E2\u0080\u0099]\",27 \"settings\": [\"smartlist_visibility_all\", \"shortcut_goto_filter_all\"]28 },29 {30 \"selector\": \".filters -collection .sidebarItem[rel=\u00E2\u0080\u0099assigned \u00E2\u0080\u0099]\",31 \"settings\": [\"smartlist_visibility_assigned_to_me\", \"shortcut_goto_filter_assigned\"]32 },33 {34 \"selector\": \".filters -collection .sidebarItem[rel=\u00E2\u0080\u0099completed \u00E2\u0080\u0099]\",35 \"settings\": [\"smartlist_visibility_done\", \"shortcut_goto_filter_completed\"]36 },37 {38 \"selector\": \".filters -collection .sidebarItem[rel=\u00E2\u0080\u0099starred \u00E2\u0080\u0099]\",39 \"settings\": [\"smartlist_visibility_starred\", \"shortcut_goto_filter_starred\"]40 },41 {42 \"selector\": \".filters -collection .sidebarItem[rel=\u00E2\u0080\u0099today \u00E2\u0080\u0099]\",43 \"settings\": [\"smartlist_visibility_today\", \"shortcut_goto_filter_today\", \"today_smart_list_visible_tasks\"]44 },45 {46 \"selector\": \".filters -collection .sidebarItem[rel=\u00E2\u0080\u0099week \u00E2\u0080\u0099]\",47 \"settings\": [\"smartlist_visibility_week\", \"shortcut_goto_filter_week\", \"today_smart_list_visible_tasks\"]48 },49 {50 \"selector\": \"#user -toolbar .activities -count, .detail -reminder.wundercon.reminder\",51 \"settings\": [\"notifications_desktop_enabled\", \"notifications_email_enabled\",52 \"notifications_push_enabled\", \"sound_notification_enabled\",\"shortcut_show_notifications\"53 ]54 },55 {56 \"selector\": \".user -avatar, .user -name, .user -arrow\",57 \"settings\": [\"shortcut_sync\", \"shortcut_goto_preferences\"]58 },59 {60 \"selector\": \"#search -toolbar .search -icon\",61 \"settings\": [\"shortcut_goto_search\"]4862 },63 {64 \"selector\": \". sidebarActions -addList\",65 \"settings\": [\"shortcut_add_new_list\"]66 },67 {68 \"selector\": \".actionBar -bottom a.tab.share\",69 \"settings\": [\"shortcut_send_via_email\"]70 },71 {72 \"selector\": \".completed -items -heading\",73 \"settings\": [\"print_completed_items\"]74 },75 {76 \"selector\": \".actionBar -bottom a.tab.last -tab\",77 \"settings\": [\"print_completed_items\", \"shortcut_copy_tasks\", \"shortcut_paste_tasks\"]78 },79 {80 \"selector\": \".detail -trash\",81 \"settings\": [\"confirm_delete_entity\", \"shortcut_delete\"]82 }]49Appendix BExperiment 1 ResourcesThis appendix contains resources used in the remote Mechanical Turk experiment,discussed in Chapter 5.1.B.1 TasksWe provide here the instructions given to participants for each of the 40 tasks.Each task has two formulations, depending on the default value of the setting.The \u001Brst formulation corresponds to changing the setting from the default valueprovided by Wunderlist; the second corresponds to changing the setting back toits default value, starting from the opposite of this default value.\u00E2\u0080\u00A2 Change the date format to DD.MM.YYYYChange the date format to MM/DD/YYYY\u00E2\u0080\u00A2 Change the time format to 12 hourChange the time format to 24 hour\u00E2\u0080\u00A2 Change the \u001Brst day of the week to be SundayChange the \u001Brst day of the week to be Monday\u00E2\u0080\u00A2 Enable sound when checking-o\u001D a todoDisable sound when checking-o\u001D a todo\u00E2\u0080\u00A2 Enable sound for new noti\u001BcationsDisable sound for new noti\u001Bcations\u00E2\u0080\u00A2 When creating a new todo, add it at the top of the current listWhen creating a new todo, add it at the bottom of the current list\u00E2\u0080\u00A2 When deleting a to-do, ask me for con\u001BrmationWhen deleting a to-do, don\u00E2\u0080\u0099t ask me for con\u001Brmation\u00E2\u0080\u00A2 When starring a to-do, move it to the top of the listDo not automatically move starred to-dos to the top of the list50\u00E2\u0080\u00A2 When printing a todo list, do not print the completed to-dosWhen printing a todo list, print also the completed to-dos\u00E2\u0080\u00A2 Show subtask progress on to-dosHide subtask progress on to-dos\u00E2\u0080\u00A2 Change shortcut for adding a new todo to CTRL + 0Change shortcut for adding a new todo to SHIFT + N\u00E2\u0080\u00A2 Change shortcut for creating a new list to CTRL + LChange shortcut for creating a new list to CTRL + K\u00E2\u0080\u00A2 Change shortcut for checking o\u001D todos to CTRL + DChange shortcut for checking o\u001D todos to CTRL + H\u00E2\u0080\u00A2 Change shortcut for starring todos to CTRL + SChange shortcut for starring todos to CTRL + M\u00E2\u0080\u00A2 Change shortcut for selecting all todos to CTRL + AChange shortcut for selecting all todos to CTRL + Q\u00E2\u0080\u00A2 Change shortcut for deleting a list or a todo to CTRL + BACKSPACEChange shortcut for deleting a list or a todo to SHIFT + BACKSPACE\u00E2\u0080\u00A2 Change shortcut for copying todos to CTRL + CChange shortcut for copying todos to SHIFT + M\u00E2\u0080\u00A2 Change shortcut for pasting todos to CTRL + VChange shortcut for pasting todos to SHIFT + P\u00E2\u0080\u00A2 Change shortcut for selecting the search box to CTRL + FChange shortcut for selecting the search box to CTRL + G\u00E2\u0080\u00A2 Change shortcut for opening preferences to CTRL + PChange shortcut for opening preferences to CTRL + .\u00E2\u0080\u00A2 Change shortcut for sharing a list via email to CTRL + EChange shortcut for sharing a list via email to CTRL + U\u00E2\u0080\u00A2 Change shortcut for showing the activities panel to CTRL + SHIFT + AChange shortcut for showing the activities panel to CTRL + SHIFT + D\u00E2\u0080\u00A2 Change shortcut for going to inbox to CTRL + IChange shortcut for going to inbox to SHIFT + B51\u00E2\u0080\u00A2 Change shortcut for showing the list of the todos assigned to you to CTRL+ 1Change shortcut for showing the list of the todos assigned to you to SHIFT+ A\u00E2\u0080\u00A2 Change shortcut for showing the list of the todos that are starred to CTRL+ 2Change shortcut for showing the list of the todos that are starred to SHIFT+ S\u00E2\u0080\u00A2 Change shortcut for showing the list of the todos due today to CTRL + 3Change shortcut for showing the list of the todos due today to SHIFT + T\u00E2\u0080\u00A2 Change shortcut for showing the list of the todos due this week to CTRL +4Change shortcut for showing the list of the todos due this week to SHIFT +W\u00E2\u0080\u00A2 Change shortcut for showing the list of all the todos to CTRL + 5Change shortcut for showing the list of all the todos to SHIFT + L\u00E2\u0080\u00A2 Change shortcut for showing the list of the completed todos to CTRL + 6Change shortcut for showing the list of the completed todos to SHIFT + C\u00E2\u0080\u00A2 Change shortcut for syncing with the server to RChange shortcut for syncing with the server to S\u00E2\u0080\u00A2 In the left sidebar, show the list of todos assigned to youIn the left sidebar, hide the list of todos assigned to you\u00E2\u0080\u00A2 In the left sidebar, show the list of todos that are due todayIn the left sidebar, hide the list of todos that are due today\u00E2\u0080\u00A2 In the left sidebar, hide the list of todos that are due this weekIn the left sidebar, show the list of todos that are due this week\u00E2\u0080\u00A2 In the left sidebar, hide the list of all the todosIn the left sidebar, show the list of all the todos\u00E2\u0080\u00A2 In the left sidebar, show the list of todos that are already completedIn the left sidebar, hide the list of todos that are already completed\u00E2\u0080\u00A2 In the lists of tasks due today and this week, show all to-dosIn the lists of tasks due today and this week, show only the todos assignedto me52\u00E2\u0080\u00A2 Enable email noti\u001BcationsDisable email noti\u001Bcations\u00E2\u0080\u00A2 Enable push noti\u001BcationsDisable push noti\u001Bcations\u00E2\u0080\u00A2 Enable desktop noti\u001Bcations [duplicated]Disable desktop noti\u001Bcations [duplicated]B.2 Participant Consent Form53Page 1 of 2 May 11, 2015 THE UNIVERSITY OF BRITISH COLUMBIA Department of Computer Science 2366 Main Mall Vancouver, B.C., V6T 1Z4 Consent Form Principal Investigator Dr. Joanna McGenere, Associate Professor, Department of Computer Science, University of British Columbia (###) ###-####\u00E2\u0080\u0094######@#####. Co-Investigator Antoine Ponsard, MSc student, Department of Computer Science, University of British Columbia (###) ###-####\u00E2\u0080\u0094######@#####. Summaries of the data and anonymous samples of comments obtained in this study may be included in a doctoral dissertation and in publications that arise from the research. Study Purpose The goal of this study is to evaluate the usability of a new interface for customizing software applications, which allows users to change pre-defined settings. In particular, we are interested in comparing this new interface with the traditional settings panel found in many software applications. Study Procedures This study is expected to require less than 45 minutes of your time. First, you will be asked to create a temporary account on Wunderlist.com, a popular todo list application, and to setup the experiment software by adding a bookmark to your web browser. Then you will be given a brief tutorial of how to use Wunderlist, and a set of tasks to complete using either Wunderlist's default settings panel or our new interface. You will be asked to fill out a questionnaire about the application, as well as some demographics information. Finally you will be asked to interact informally with a version of our prototype designed to operate with Gmail, and we will you your feedback on the prototype. The interview will be audio recorded. Compensation We are very grateful for your participation in this study. You will receive a small honorarium ($10) for your participation Potential Risks There are no known risks to your participation in this study. 54Page 2 of 2 May 11, 2015 Confidentiality Only the principal investigator and co-investigators will have access to the data and the audio recordings of the interviews (all data will be stored in a password-protected location). Confidentiality will be maintained by anonymizing the identity of participants. Contact for information about the study Please contact Antoine Ponsard (email: #####@#####) if you need more information about the study. Contact for information about the rights of research subjects If you have any concerns or complaints about your rights as a research participant and/or your experiences while participating in this study, contact the Research Participant Complaint Line in the UBC Office of Research Services at ###-###-#### or if long distance e-mail #####@##### or call toll free #-###-###-#### (Toll Free: #-###-###-####). Consent: My participation is entirely voluntary and I may refuse to participate or withdraw from the study at any time. My signature below indicates that I have received a copy of this consent form for my own records. My signature indicates that I consent to participate in this study. ____________________________________________________ Participant Signature Date ____________________________________________________ Printed Name of the Participant signing above 55B.3 QuestionnairesB.3.1 DemographicsMechanical Turk participants were administered an electronic version of the de-mographics questionnaire used Experiment 2 (Appendix C.4.1).B.3.2 SatisfactionAt the end of the experiment, participants were asked to rate the mechanism theywere using on three metrics: ease of use, perceived speed, and satisfaction.B.3.3 RecognitionAt the end of the experiment, participants completed two recognition question-naires: one on tab names, the other on individual settings. In both cases, half ofthe answers were made up by the authors, but plausible.565758Appendix CExperiment 2 ResourcesThis appendix contains resources used in the face-to-face lab experiment, discussedin Chapter 5.2.C.1 TasksThe same tasks were used than in Experiment 1 (Appendix B.1), but only the \u001Brstformulation\u00E2\u0080\u0094corresponding to Wunderlist\u00E2\u0080\u0099s default values.C.2 Call For ParticipationThis recruitment poster was posted in multiple locations on the UBC campus.59 $10 for testing new customization interfaces when:where:duration:60C.3 Participant Consent Form61Page 1 of 2 May 11, 2015 THE UNIVERSITY OF BRITISH COLUMBIA Department of Computer Science 2366 Main Mall Vancouver, B.C., V6T 1Z4 Consent Form Principal Investigator Dr. Joanna McGenere, Associate Professor, Department of Computer Science, University of British Columbia (###) ###-####\u00E2\u0080\u0094#####@#####. Co-Investigator Antoine Ponsard, MSc student, Department of Computer Science, University of British Columbia (###) ###-####\u00E2\u0080\u0094#####@#####. Summaries of the data and anonymous samples of comments obtained in this study may be included in a doctoral dissertation and in publications that arise from the research. Study Purpose The goal of this study is to evaluate the usability of a new interface for customizing software applications, which allows users to change pre-defined settings. In particular, we are interested in comparing this new interface with the traditional settings panel found in many software applications. Study Procedures This study is expected to require about 30 minutes of your time. First, you will be asked to create a temporary account on Wunderlist.com, a popular todo list application, and to setup the experiment software by adding a bookmark to your web browser. Then you will be given a brief tutorial on how to use Wunderlist, and a set of tasks to complete using either Wunderlist's default settings panel or our new interface. Finally you will be asked to fill out a questionnaire about the application, as well as some demographic information. Compensation We are very grateful for your participation in this study. You will receive a small honorarium ($2.00) for your participation, plus a bonus of up to $2.90 depending on your performance. Potential Risks There are no known risks to your participation in this study. 62Page 2 of 2 May 11, 2015 Confidentiality Only the principal investigator and co-investigators will have access to the data. Confidentiality will be maintained by anonymizing the identity of participants. Contact for information about the study Please contact Antoine Ponsard (email: #####@#####) if you need more information about the study. Contact for information about the rights of research subjects If you have any concerns or complaints about your rights as a research participant and/or your experiences while participating in this study, contact the Research Participant Complaint Line in the UBC Office of Research Services at ###-###-#### or if long distance e-mail #####@##### or call toll free #-###-###-#### (Toll Free: #-###-###-####). Consent: My participation is entirely voluntary and I may refuse to participate or withdraw from the study at any time. My signature below indicates that I have received a copy of this consent form for my own records. My signature indicates that I consent to participate in this study. ____________________________________________________ Participant Signature Date ____________________________________________________ Printed Name of the Participant signing above 63C.4 QuestionnairesC.4.1 DemographicsAfter signing the consent form, participants were asked to complete a short de-mographics questionnaire.6465C.4.2 RatingsAt the end of each condition, participants were asked to rate the mechanism theywere using on three metrics: ease of use, perceived speed, and satisfaction.C.4.3 RankingsThis questionnaire was administered at the end of the experiment, after all fourblocks of 10 trials were completed.661116768"@en . "Thesis/Dissertation"@en . "2016-02"@en . "10.14288/1.0220520"@en . "eng"@en . "Computer Science"@en . "Vancouver : University of British Columbia Library"@en . "University of British Columbia"@en . "Attribution-NonCommercial-NoDerivs 2.5 Canada"@* . "http://creativecommons.org/licenses/by-nc-nd/2.5/ca/"@* . "Graduate"@en . "Anchored customization : anchoring settings to the application interface to afford customization"@en . "Text"@en . "http://hdl.handle.net/2429/55601"@en .