UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Anchored customization : anchoring settings to the application interface to afford customization Ponsard, Antoine 2015

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2016_february_ponsard_antoine.pdf [ 2.48MB ]
Metadata
JSON: 24-1.0220520.json
JSON-LD: 24-1.0220520-ld.json
RDF/XML (Pretty): 24-1.0220520-rdf.xml
RDF/JSON: 24-1.0220520-rdf.json
Turtle: 24-1.0220520-turtle.txt
N-Triples: 24-1.0220520-rdf-ntriples.txt
Original Record: 24-1.0220520-source.json
Full Text
24-1.0220520-fulltext.txt
Citation
24-1.0220520.ris

Full Text

Anchored CustomizationAnchoring Settings to the Application Interface to AordCustomizationbyAntoine PonsardB.Sc., École Polytechnique, 2012Diplôme d’Ingénieur, École Polytechnique, 2013A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFMASTER OF SCIENCEinThe Faculty of Graduate and Postdoctoral Studies(Computer Science)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)November 2015© Antoine Ponsard 2015AbstractThe HCI community has identied the need to let users adapt their software totheir own tasks and preferences. Yet, many users do not customize, or only doso rarely. The de facto standard customization mechanism is the settings panel,which has undergone minimal design improvements since it was introduced alongwith the graphical user interface in the 1980s. Entirely disconnected from theapplication UI, these panels aord only very indirect manipulation, as users mustrely on often cryptic text labels to identify the settings they want to change. Froma developer’s point of view these panels make sense: they are simple graphicalrepresentations of traditional UNIX cong les.In this thesis, we propose a novel customization approach, designed from theuser’s point of view. In Anchored Customization, settings are anchored to concep-tually related elements of the application UI. Our Customization Layer instantiatesthis approach: users can see which UI elements are customizable, and access theirassociated settings. We designed three variants of customization layer based onmulti-layered interfaces, and implemented these variants on top of a popular webapplication for task management, Wunderlist. A remote experiment conductedon Mechanical Turk, complemented with a face-to-face lab experiment (for a to-tal of 60 participants) showed that the two minimalist variants were 35% fasterthan Wunderlist’s settings panel. This new approach provides signicant bene-ts for users while requiring little extra work from designers and developers ofapplications.iiPrefaceThe experiments described in this thesis were conducted with the approval of theUBC Behavioural Research Ethics Board (certicate number H11-01976).Parts of this thesis appear in a conference paper manuscript 1 where I amthe rst author. Joanna McGrenere helped frame and write the manuscript, andprovided supervisory assistance for this research.1A. Ponsard, J. McGrenere. Anchored Customization: Anchoring Settings to the ApplicationInterface to Aord CustomizationiiiTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiAcknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Anchored Customization . . . . . . . . . . . . . . . . . . . . . . . 11.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.1 Adaptable Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . 42.2 Adaptive and Mixed-Initiative Interfaces . . . . . . . . . . . . . . 42.3 Customization in Context . . . . . . . . . . . . . . . . . . . . . . 52.4 Interface Context . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Systematic Assessment of Feasibility . . . . . . . . . . . . . . . . . 63.1 Existing Customization Mechanisms . . . . . . . . . . . . . . . . 63.2 Feasibility of Anchored Customization . . . . . . . . . . . . . . . 74 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.1 Anchored Customization . . . . . . . . . . . . . . . . . . . . . . . 94.1.1 Display of Settings . . . . . . . . . . . . . . . . . . . . . . 134.1.2 Display of Anchors . . . . . . . . . . . . . . . . . . . . . 13iv4.2 Customization Layer . . . . . . . . . . . . . . . . . . . . . . . . . 144.2.1 Anchors Visible in a Layer . . . . . . . . . . . . . . . . . 144.2.2 Three Variants for Displaying Settings . . . . . . . . . . . 154.2.3 Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.2.4 Software Architecture . . . . . . . . . . . . . . . . . . . . 204.2.5 Implementation . . . . . . . . . . . . . . . . . . . . . . . 205 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215.1 Experiment 1: Remote Mechanical Turk . . . . . . . . . . . . . . 225.1.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225.1.2 MTurk Results . . . . . . . . . . . . . . . . . . . . . . . . 255.2 Experiment 2: Face-to-Face in the Lab . . . . . . . . . . . . . . . 265.2.1 Method Dierences from Study 1 . . . . . . . . . . . . . . 275.2.2 Face-to-Face Lab Results . . . . . . . . . . . . . . . . . . 275.3 Secondary Exploratory Analyses . . . . . . . . . . . . . . . . . . 316 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346.1 Reconciling the Performance Results . . . . . . . . . . . . . . . . 346.2 Performance/Awareness Trade-O . . . . . . . . . . . . . . . . . 356.3 Applicability to Other Desktop Software . . . . . . . . . . . . . . 356.4 Applicability to Handheld Devices . . . . . . . . . . . . . . . . . 366.5 Interpretation Gulf . . . . . . . . . . . . . . . . . . . . . . . . . . 366.6 The Developers’ Point of View . . . . . . . . . . . . . . . . . . . 366.7 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . 397.1 Long-Term Eects . . . . . . . . . . . . . . . . . . . . . . . . . . 397.2 Generating the Mapping . . . . . . . . . . . . . . . . . . . . . . . 407.3 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . 40Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42AppendicesA Mapping of settings to UI elements in Wunderlist . . . . . . . . . 47B Experiment 1 Resources . . . . . . . . . . . . . . . . . . . . . . . . . 50B.1 Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50B.2 Participant Consent Form . . . . . . . . . . . . . . . . . . . . . . 53B.3 Questionnaires . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56vB.3.1 Demographics . . . . . . . . . . . . . . . . . . . . . . . . 56B.3.2 Satisfaction . . . . . . . . . . . . . . . . . . . . . . . . . . 56B.3.3 Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . 56C Experiment 2 Resources . . . . . . . . . . . . . . . . . . . . . . . . . 59C.1 Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59C.2 Call For Participation . . . . . . . . . . . . . . . . . . . . . . . . . 59C.3 Participant Consent Form . . . . . . . . . . . . . . . . . . . . . . 61C.4 Questionnaires . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64C.4.1 Demographics . . . . . . . . . . . . . . . . . . . . . . . . 64C.4.2 Ratings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66C.4.3 Rankings . . . . . . . . . . . . . . . . . . . . . . . . . . . 66viList of Tables3.1 Summary of a systematic review of 8 PTM apps to determine thefeasibility of Anchored Customization . . . . . . . . . . . . . . . . 7viiList of Figures4.1 Three approaches to organize settings: cong les, settings pan-els, Anchored Customization . . . . . . . . . . . . . . . . . . . . . 104.2 Two approaches to navigate settings: tabs and Anchored Cus-tomization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.3 The settings panel oered in Wunderlist . . . . . . . . . . . . . . 124.4 The Wunderlist web application . . . . . . . . . . . . . . . . . . . 164.5 Customization Layer displayed on top of Wunderlist . . . . . . . 164.6 Ghost anchors and collapsing mechanism . . . . . . . . . . . . . . 174.7 Minimal, Minimal+Context, and Full+Highlight variants . . . . . 174.8 Reverse highlighting in the full settings panel . . . . . . . . . . . 195.1 Modal dialog displayed at the beginning of each trial . . . . . . . 235.2 Possible states of the progress bar, displayed at the top of the win-dow during the experiment . . . . . . . . . . . . . . . . . . . . . . 245.3 Experiment 1: 95% condence intervals of the median durationper Customization Mechanism and Block . . . . . . . . . . . . . . 265.4 Experiment 2: 95% condence intervals of the median durationper Customization Mechanism . . . . . . . . . . . . . . . . . . . . 285.5 Density plots of the distribution of the percentage of trials in whichparticipants selected a “correct” anchor . . . . . . . . . . . . . . . 325.6 Density plots of the distribution of the percentage of trials in whichparticipants selected a “correct” anchor, for each CustomizationLayer variant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32viiiAcknowledgmentsI thank my supervisor Joanna McGrenere, for her endless patience, guidance andsupport throughout this Masters.LUNCH, MUX, and HCI@UBC were valuable occasions to meet fellow HCIresearchers at UBC. I thank all the people who organized these meetings, andall those who provided feedback on my work. In particular, I thank FranciscoEscalona, Kamyar Ardekani and Kailun Zhang, with whom I worked on threegreat projects. I also thank Peter Beshai, Derek Cormier, Mona Haraty, MatthewBrehmer and Oliver Schneider for their advice and the many insightful discussionswe had.I thank my Information Visualization professor, Tamara Munzner, who coura-geously accepted to be my second reader.Finally, I thank my family and friends for their encouragement and support.This research was funded by the Graphics Animation and New Media (GRAND)NCE as well as the NSERC Discovery Grant program.ixDedicationThere is one thing that is more dear to a Frenchman’s heart than the sight ofthe Eiel tower over the rooftops of Paris: a properly baked loaf of bread. Alas,this simple pleasure is surprisingly dicult to nd outside of the grassy plains ofFrance.This is why I would like to dedicate this thesis to the bakery Beyond Bread,for their remarkable eort to propagate the baking traditions of France to thedistant Canada. As soon as I set foot in this bakery, the smell of freshly bakedbread washing over me, I knew that I had found a home away from home. Icannot be thankful enough for their country bread, with its rm crust and delicatecrumb; for their properly named pains au chocolat; for their rosemary foccacia,soft as a dream and dripping olive oil; and even for their overpriced baguette andcroissants.This work would not have been possible without them.xChapter 1IntroductionThe HCI community has identied the potential benets of supporting users toadapt their software to their own tasks and preferences [28]. Yet, many users donot customize, or only do so rarely. In today’s software, the de facto standard cus-tomization mechanism is the settings panel, also known as the “options menu” or“preferences dialog.” These panels have signicant usability limitations in that thesettings they oer are entirely disconnected from the application UI. There are novisual aordances in the UI that can help answer the question: “Is it possible tocustomize X?” Users must open the settings panel and then rely entirely on textlabels to identify the settings they want to change, with little to no other con-textual information. Thus, the classic vocabulary problem applies: the languageused by designers to describe an aspect of the interface is often dierent from thelanguage of users [19]. This creates a deep gulf of execution between the user’sintentions, formulated in the context of the application UI, and the actions oeredin the settings panel. The fact that customization occurs fairly infrequently overlong periods of time [27] only exacerbates the problem: users are likely to forgetwhere things are in the settings panel between customization episodes.Despite obvious usability problems, settings panels have undergone only mi-nor improvements since they were rst introduced along with the graphical userinterface in the 1980s [24]. The interaction paradigm fundamentally hasn’t changed:the user can browse tabs of settings, tick checkboxes, and choose values fromdrop-downs. These panels essentially represent a developer’s point of view of theapplication’s customization opportunities: they are simple graphical representa-tions of traditional UNIX cong les.21.1 Anchored CustomizationWe propose a novel approach to customization, called Anchored Customization,designed from the user’s point of view: anchoring settings to visual elementsof the UI that are conceptually related to these settings. This reduces the gulf of2Cong les list in plain text the settings that can be modied. Users can change settings byopening and editing the le in a text editor.1execution, and leverages users’ existing knowledge of the application UI. Whileothers have proposed the idea of moving the access point of a customization closerto the feature that it aects [38], we take the approach further by allowing usersto access all conceptually related functions from an anchor point. For instance,a notication icon in the UI can be used as the anchor to change not only howpopup notications (emanating from the icon) are displayed, but also the fre-quency of various notications (e.g., email), and other related settings. AnchoredCustomization leverages mental associations that users can form intuitively be-tween visual elements of the interface and changes they want to make to theirsoftware.Once settings are anchored to UI elements, there are dierent possible ways ofvisualizing and accessing them. We designed, implemented, and evaluated Cus-tomization Layer, as a concrete instantiation of the Anchored Customization ap-proach. Customization opportunities are represented in a meta-layer on top of theinterface. Users can see all UI anchors at once in this layer, and can access theirassociated settings by clicking on them. This type of layered design has been suc-cessfully used in the context of online help [8]. Although clicking on an anchoronly shows a small subset of settings initially, the user can access the full settingspanel from any anchor via a simple expansion mechanism.Our approach is pragmatic: it recognizes that settings panels are the standardin the software industry. Developers rely on this simple mechanism to provideend-user customization, since these panels are (presumably) an inexpensive partof the UI to develop. And users are accustomed to them. Introducing a new cus-tomization mechanism therefore requires careful consideration of the cost/bene-t trade-os at play. We show that anchoring settings to the UI through a cus-tomization layer has the potential to provide signicant benets for users, whilerequiring only minimal extra work from the designers/developers. Our goal is topropose a well-considered interface improvement that can be adopted widely.A customization mechanism must be implemented to customize a specic app.Although our design approach is applicable to any software with a GUI that is cus-tomizable via a settings panel, we had to choose a particular application domainfor a rst evaluation. Since task management tools are widely used, and signif-icant individual dierences have been observed in the way people manage theirtasks [22], we considered that Personal Tasks Management (PTM) would be anappropriate rst application domain for our research.1.2 ContributionsOur work contributes the following:21. The concept of Anchored Customization: to anchor customization oppor-tunities to conceptually related elements in the application UI.2. A systematic review of PTM apps to evaluate a priori the feasibility andpotential usefulness of adopting this concept.3. A realization of this concept in a customization mechanism, the Customiza-tion Layer, with three variants that explore dierent trade-os betweenmulti-layered interfaces.4. An implementation architecture of these three variants for web applica-tions: our prototype augments a real-world PTM app, Wunderlist [20], butcan be adapted to other modern web apps with minimal additional work fordevelopers and designers.5. Two experiments (Mechanical Turk and face-to-face) with 60 participantsshowing performance improvements of the Customization Layer over theregular settings panel provided by Wunderlist. The face-to-face experimentalso provided a preliminary understanding of how users apprehend An-chored Customization.1.3 OverviewPrevious work relevant to this research is summarized in Chapter 2. Chapter 3presents two systematic reviews of a set of Personal Task Management apps. Therst review identies the customization mechanisms provided in these apps; thesecond evaluates the feasibility of anchoring settings to visual elements of theinterface. Chapter 4 outlines the design space of Anchored Customization, andexplains the design of our Customization Layer. Chapter 5 presents a two-partevaluation of our Customization Layer, in comparison to traditional settings pan-els. A rst experiment was conducted remotely on Amazon Mechanical Turk, tomeasure the performance of our Customization Layer for changing settings. Asecond experiment took place in a face-to-face settings at our university, to ver-ify and triangulate the results of the rst one. Chapter 6 discusses the insightsgathered on the three variants of our Customization Layer, and reects on thepossibilities oered by this new customization mechanism. Chapter 7 outlinesdirections for future work and concludes this thesis.3Chapter 2Related WorkPrevious research has studied the benets of letting users adapt software to theirown preferences and tasks [27, 28, 30]. What exactly constitutes customizationis not well-established in the HCI community, despite there being a rich litera-ture in this space. For example, Mackay [27] denes end-user customization as“the mechanisms by which users may specify individual preferences, and preservetheir preferred patterns of use, without writing code.” In that sense, it is closely re-lated to the concept of tailorability, in which “end users can progressively modifya working system without ever having to leave the application domain to work ina separate underlying programming domain.” [28]. Yet the term “tailorable” seemssomewhat outdated in the HCI community, and has been replaced by “adaptable”.2.1 Adaptable InterfacesThe challenges of designing and building adaptable interfaces has produced a richand varied literature. Multi-layered interfaces [30, 33] group subset of featuresinto dierent layers, and allow users to choose which layer to work with. Sophisti-cated techniques have been proposed to let users modify GUIs at run time [12, 13],even without access to the source code [11, 31, 36]. The end-user developmentparadigm goes even further, by empowering users to build their own applica-tions [17, 26]. Yet, powerful adaptable approaches generally require users to spendsignicant time and eort personalizing their interfaces, often by writing scriptsor editing code—which limits the potential audience to power users. As a result,these approaches have not yet been widely adopted in industry.2.2 Adaptive and Mixed-Initiative InterfacesAdaptive approaches attempt to reduce the need for user involvment by automat-ically generating user models that predict potentially useful customizations [16].However, giving control of the user interface to an automated system has signi-cant drawbacks, as it tends to make the UI spatially unstable and unpredictable [14].Mixed-initiative approaches can alleviate those problems by suggesting possible4customizations to the user, who remains in control of their interface [7, 10]. Ul-timately, adaptive and mixed-initiative approaches put the onus on developers tobuild and integrate complex user models into the interface, which is dicult todo well [23].2.3 Customization in ContextOur work focuses on leveraging the customization opportunities already availablein existing software settings panels, with the goal of minimizing the extra costsof customization for both users and developers. The potential of “representingtailoring functions in the overall interface” was identied 15 years ago [25], buthas received little attention since then. An early work proposed the concept ofDirect Activation [38] which states that “the access point of the tailoring functionshould be designed related to the one of the tailorable function”. In other words,the settings that aect a given feature should be accessible from (or around) theaccess point of the feature they aect. We broaden this approach by anchoringany setting to any conceptually related elements of the UI —not only elements ob-jectively aected by a setting. For instance, the setting for changing the keyboardshortcut used to “open a new tab” could be anchored to the button that opens anew tab. This setting does not tailor the “open a new tab” function itself, but con-ceptually it makes sense to associate a keyboard shortcut and a button that callthe same function.2.4 Interface ContextThe benets of preserving interface context have been exploited in other areasof HCI. Perhaps the most common example is the context menu, which reducesthe time needed to select options related to the current location of the pointer.For example, contextual help can augment an application interface with links toa wiki [35], or user-generated Q&As [8]. While help queries tend to naturallycorrespond to elements of the interface that users are trying to learn, the corre-spondence between settings and interface elements may be less clear, as settingscan aect software in arbitrary ways. Thus, it was not obvious at the outset ofour research that such direct correspondences could be found from settings toUI elements. Yet the literature oers many promising examples of representingmeta-information about an application as an overlay on the app interface: recentchanges [2, 5], computational wear [29], predicted use [18], even low-level usabil-ity problems [37]. This motivated our visual design approach, which considerscustomization opportunities as a meta-layer on top of the application interface.5Chapter 3Systematic Assessment ofFeasibilityThe motivation of Anchored Customization is to address the shortcomings of set-tings panels, which appear to be the standard in the software industry. To verifythis assumption, we performed a systematic review of a set of PTM apps, investi-gating which customization mechanisms they provide. We then conducted a sec-ond review to evaluate the feasibility of the Anchored Customization approach.3.1 Existing Customization MechanismsWe chose a particular yet fairly generic application domain: Personal Task Man-agement. We selected 12 of the most popular PTM apps3, for a total of 20 uniqueapps (counting separately both desktop and Android versions). For each applica-tion we recorded:• which customization mechanisms were oered;• how many user actions were needed to access them;• the location of the access points to these mechanisms in the applicationinterface;• the type and number of settings each mechanism could change.Our review showed that an overwhelming majority of settings were accessedvia settings panels. In fact, only 1 of the 20 applications didn’t oer a settingspanel at review time—but it now does. The structure of these panels was remark-ably similar: tabs for desktops apps, multi-level menus on Android. In both cases,additional visual delimiters such as lines or boxes were sometimes used insideeach tab or level, to create subsections of related settings. The same widgets wereused to change settings: checkboxes or toggles for binary settings, drop-downs3Evernote, OneNote, Any.do, Cal, Wunderlist, Todoist, Remember The Milk, Toodledoo, Astrid,Clear, GTasks, and Tasks.6Application Platform # settings directly indirectly notAstrid Android 41 25 6 10Evernote Desktop 31 11 12 8Gmail Web 29 16 7 6RTM Android 26 10 11 5Toodledo Web 53 37 7 9Toodledo Android 43 26 16 1Wunderlist Web 41 12 28 1Word Desktop 132 90 24 18total 396 227 111 58normalized % 100% 52% 32% 16%Table 3.1: Summary of a systematic review of 8 PTM apps to determine the fea-sibility of Anchored Customization. For each app, the settings provided in thesettings panel were classied as directly, indirectly, or not anchorable. To avoidbiasing the results towards the applications oering the most settings, we nor-malized the dierent proportions of settings for each app before averaging acrossall apps.or radio buttons for multiple choices, and in some cases sliders for numbers. Theaccess points to these settings panels were also very similar: an “Options”, “Pref-erences” or “Settings” menu entry at the top of the screen, or a gear or toolboxicon.We found that settings panels also contain a variety of non-settings items,such as an “About” page, contact information, or promotional materials for up-grading to a paid version. In that sense, they serve somewhat as a mixture ofmiscellaneous elements that were probably not deemed important enough to beintegrated in the main application interface.We did encounter a few other customization mechanisms, such as dragging aborder to resize a column, or reordering buttons via drag-and-drop. These alter-native mechanisms eectively use direct manipulation to let users customize, butwere typically restricted to changing a very specic part of the interface.3.2 Feasibility of Anchored CustomizationThe widespread use of settings panels highlights the potential usefulness of An-chored Customization. We next evaluated the feasibility of this approach on asubset of apps from our rst review. We selected apps with more than 15 settings,and included Microsoft Word and Gmail which, while being general-purpose soft-7ware, can both also be used for task management. As shown in Table 3.1, wereviewed a total 8 apps (5 on desktop, 3 on Android), classifying a total of 396settings.According to our estimation, 52% of settings could be anchored unambigu-ously in the interface. A large number of these correspond to settings that directlyshow or hide interface elements, or change their position or visual appearance.Another 32% of settings could be indirectly anchored based on an intuitive mentalassociation. For instance, the frequency of automatic syncing could be anchoredto the button that performs a manual synchronization. Finally, we estimated thatonly 16% of settings could not be anchored anywhere in a meaningful way. Thesesettings correspond either to advanced options (e.g., number of weeks after whichcompleted tasks are archived), or global options (e.g., display language) which arenot related to any particular UI location.The percentages reported here are based on the raw number of settings oeredby the apps we reviewed. However, not all settings are created equal: some willbe changed more often than others. A more accurate statistic would be to weigheach setting by how often users can be expected to change it. We believe that thisadjusted metric would reduce the relative proportion of non-anchorable settingseven further. Indeed, only a small fraction of users (probably those with moretechnical backgrounds) is likely to dig into the most advanced settings.Overall, the results of our second review provide solid evidence for the vi-ability of our Anchored Customization approach, at least in the PTM domain:approximately 84% of settings can be anchored. Yet a signicant number of set-tings cannot be anchored (16%), which must be taken into account when designinganchored customization mechanisms.8Chapter 4DesignIn most applications, settings are a list of parameters that can take some prede-ned values. They are generally given a human readable label, and sometimes ashort description. For example, a “conrm before deleting” setting can be eithertrue or false, and could have the description “show ok/cancel popup when clickingon the delete button”. UNIX-type cong les are the most barebone representa-tion of settings, and the closest to the programming domain (Figure 4.1 a). Userscan manually change settings by editing the cong le in a text editor.Settings panels oer two key improvements over cong les. First, they pre-vent errors by restricting the values that a given setting can take. For instance,checkboxes only toggle booleans, and drop-downs oer a limited set of optionsto choose from. Second, settings panels often group related settings together intotabs or other forms of subsections, with the aim to reduce the time needed toaccess a particular setting. In this way, settings panels are a static abstract par-tition of settings: each setting appears in only one section; sections are based onabstract categories, such as “shortcuts” or “display;” and these categories are de-ned by designers at the outset (Figure 4.1 b). Tabs are used to navigate from onehigh-level category to another (Figure 4.2 a). An example is shown in Figure 4.3.4.1 Anchored CustomizationThe Anchored Customization approach, by contrast, promotes a contextual many-to-many mapping of settings to UI elements (Figure 4.1 c). Any UI element canbe used as an anchor—for instance, icons, buttons, menus, even an empty area.The goal of the mapping is to provide context for each setting. One setting canbe mapped to multiple anchors, and multiple settings can be mapped to the sameanchor. The visibility of anchors and thus their associated settings changes dy-namically, depending on the current state of the interface. The Anchored Cus-tomization approach leverages users’ pre-existing knowledge of the applicationUI, instead of requiring them to learn the abstract structure of a settings panel.Hence, the key idea of Anchored Customization is that the application interfaceitself is used to organize and navigate the settings space (Figure 4.2 b).9Figure 4.1: (a) Cong les are the most direct representation of settings, withoutany particular structure. (b) Settings panels are a static partition of settings, basedon abstract categories. (c) Anchored Customization creates a contextual many-to-many mapping of settings to UI elements. UI elements are represented here bycolored circles.10Figure 4.2: (a) In a settings panel, tabs indicate the current location and allowusers to navigate to a dierent group of settings. (b) In Anchored Customization,elements of the application UI itself are used to navigate the settings space.11Figure 4.3: The settings panel oered in Wunderlist, which was used as the Con-trol condition in Experiments 1 and 2. Tabs appear at the top of the panel, and aredecorated with an icon.12There are two main dimensions in the design space for mechanisms that in-stantiate Anchored Customization: the display of the settings and the display ofthe anchors.4.1.1 Display of SettingsIt is not desirable to permanently show the settings associated with the variousanchors. Most applications oer many settings, which would clutter the inter-face if they were always visible. Further, customization is typically very much asecondary infrequent activity. Thus, in general, the settings associated with ananchor should be displayed on demand, when the user expresses interest in ananchor—by clicking or hovering on it, for instance. There are many ways to dis-play settings once demanded. We explored three possibilities, inspired by multi-layered interfaces (described below). But the design space is much larger. Someinteresting dimensions include:• Which settings should be displayed when clicking on an anchor. Only thesettings mapped to this anchor, or a local neighborhood?• How to represent settings. Now that settings are placed in context withinthe interface, the text labels might become superuous. For instance, short-cuts might not need a textual description if they are mapped clearly to a but-ton that performs the same action. Settings that show or hide an elementmight be better represented by a simple visible/invisible icon. In general,dierent types of settings (or their associated anchors) could be representeddierently.• How to provide access to the settings that are not associated with any an-chor.• How much of the structure of traditional settings panels should be retained,to help users transition to this new approach to customization.4.1.2 Display of AnchorsThere are a number of dierent possibilities when considering how the anchorscan be displayed. One familiar approach is through context menus: anchors arenot explicitly marked visually, but can be used by the user to demand a corre-sponding setting. For instance, in Microsoft Word, right-clicking the ribbon showsa contextual menu with an option to “Customize the ribbon”. A tooltip that ap-pears when hovering over an anchor for a short duration could be used in a simi-lar way. This local, targeted approach answers the question: Is this customizable?13The downside of this approach, however, is that there is no way for the user toget a holistic overview of all the settings available within the UI. Serially invokingthe context menu from many dierent anchors is too tedious, and the user mustresort to bringing up the settings panel.An alternative approach is to provide a global overview of all the customizationopportunities available, by making all the anchors visible at once, likely througha mode. This approach helps answer the question: What is customizable? Thedownside of this approach is that entering a distinct mode could interrupt theuser’s workow —which is why modes should be used sparingly in interactiondesign. The contextual menu approach described above might be more appropri-ate when users can easily guess (or already know) which UI element to click on toperform the desired change. In contrast, the global overview approach helps usersto become aware of the settings available, and where to nd them. Fortunately,the two approaches are not mutually exclusive; providing both would allow usersto choose the one best suited to their current need.4.2 Customization LayerWe designed and implemented Customization Layer which instantiates the an-chored customization approach. In the design dimensions described above, thismechanism uses a global overview approach to show all anchors in an extra layeron top of the interface. We explore three ways to display settings on demand,inspired by multi-layered interfaces.4.2.1 Anchors Visible in a LayerIn a layered design, there are multiple ways to indicate which UI elements areanchors. In LemonAid’s contextual help mode [9], the interface was overlaid withyellow question marks that explicitly represent help queries. To avoid clutter, wethought it best not to add additional graphical elements to mark anchors. Instead,the anchors themselves are visually highlighted when users activate a layer ontop of the regular application (shown in Figure 4.4). In our implementation, theapp interface becomes darker and less saturated, but is still clearly visible throughthe Customization Layer. Anchors are shown with a white background and darkgray text, to optimize legibility and ensure visibility above the dimmed applicationinterface (Figure 4.5).When the user hovers over an anchor A, that anchor and all other anchorsmapped to the same settings as A are highlighted in orange (Figure 4.5). Thislinked highlighting helps users create a mental model of the mapping of settingsto UI elements. For instance, in a PTM app, hovering over a button highlights all14the buttons that share the same setting (e.g., a keyboard shortcut for marking a“todo” as important).There is one important special case that required attention when anchoringsettings to UI elements: what if a setting governs the visibility of the UI anchoritself? Consider for instance settings that show or hide buttons in a toolbar. Ofcourse it makes sense to anchor such a setting to the button it aects; but afterhiding the button, how can users access its associated setting to make the buttonvisible again? To avoid this problem, we introduce the notion of ghost anchors:anchors that correspond to hidden elements in the application UI. These anchorsoer the same functionality as regular anchors, but are displayed in a darker grayto indicate their transient status.Displaying all ghost anchors at the same time is not necessarily desirable. Ifan application has many optional, hideable components, showing all of them asghost anchors could break the application layout, clutter the customization layer,and overwhelm the user. Furthermore, the customization layer should look assimilar as possible to the current interface of the application, to help users alter-nate seamlessly between the two. For these reasons, we implemented a collapsingmechanism: ghost anchors are by default represented by a simple “chevron” iconin the customization layer (Figure 4.6). To further reduce visual clutter, ghostanchors that are close to each other are collapsed under the same chevron icon,based on a simple proximity clustering algorithm. Clicking on a chevron iconreveals the ghosts that it was previously hiding.4.2.2 Three Variants for Displaying SettingsWhen the Customization Layer is shown, clicking on an anchor displays the sub-set of settings anchored to that UI element. We explored three visual representa-tions of this subset (see Figure 4.7). The Minimal panel (M) only shows the settingsin the subset, with an orange background to indicate that they are related to theanchor that was clicked. Minimal+Context (M+C) is very similar, but a tab namenext to each setting indicates from which tab of the full settings panel it comesfrom. In contrast, Full+Highlight (F+H) keeps the structure of the complete set-tings panel, and the settings from the anchor’s subset are highlighted in orange,as are the tabs in which they appear. Some tabs contain more settings than can beshown in the height of the panel, so we automatically scroll down to bring intoview the rst highlighted setting, if any.Because some settings might not be anchorable to the interface (cf. Feasibilitysection), all designs must provide a fallback access to the complete set of settings.In Full+Highlight, users can browse all the tabs freely, as they would in a regularsettings panel. In Minimal+Context, the tab names next to each setting are ac-15Figure 4.4: The Wunderlist web application. Todos appear in a vertical list, andcan be “starred” to convey importance. Here the user has starred the second todoand given it a due date. In the left sidebar are a collection of “Smart Lists”, intelli-gent lters that display a list of todos ltered by a particular criteria. For instance,a Smart List can show only the todos due today, or due this week, or all the to-dos already completed. There are also user-dened todo lists, such as Groceries,Travel, and Work.Figure 4.5: Customization Layer displayed on top of Wunderlist. The user is cur-rently hovering over the bottom “star” button on the right-hand side of the screen;as a result, all star buttons are highlighted in orange because they share the samesettings. Clicking on an anchor displays the settings associated with it—here,shown in the Minimal panel.16Figure 4.6: (1) The user selects the “Week” lter, then disables it (not shown). (2)The corresponding anchor is collapsed under a chevron icon. (3) Clicking on thechevron reveals the “Week” lter as a ghost anchor.Figure 4.7: Minimal, Minimal+Context, and Full+Highlight variants showing thesettings anchored to the “Week” Smart List. Two settings are mapped to thisSmart List: the rst changes the shortcut for opening this Smart List; the seconddetermines whether this Smart List is visible or hidden in the main interface.17tual buttons; clicking on them opens the full panel at the corresponding tab. InMinimal, a “show all” hyperlink is provided at the bottom of the mini panel. Bydefault, it opens the full panel at the tab that contains the most settings from theanchor’s subset. Users can return to the minimal panel by clicking a backwardarrow at the top left of the full panel.These three variants reect dierent points in the multi-layered interface de-sign space [33]. The two minimal variants only show the anchor’s subset of thesettings initially, while Full+Highlight uses visual highlighting to distinguish thesubset from all other settings. All three designs are intended (to varying degrees)to reduce complexity, speed up access to the most relevant settings, and help theuser become aware of the full set of settings. Hence, they reect dierent trade-os between a minimal subset of settings and the full settings panel.As with any multi-layered interface, transitioning from a lower layer to anupper layer can be challenging for users [33]. In our implementation, we pro-vide animated transitions between the minimal variants and the full panel to helpusers understand the relationship between the two. The minimal panels expandsmoothly to the full size, while the highlighted settings glide into their new posi-tion. The remaining (non-highlighted) settings fade in afterwards. Informal pilotsfound these relatively simple animations to be helpful for conveying the intendedmental model.As an additional way to help users understand the mapping between settingsand anchors, we augmented the full settings panel with reverse highlighting: hov-ering on any setting displays a blue halo around the corresponding anchors in theinterface (Figure 4.8). The idea is to allow users to explore the settings→ anchorsrelationship, whereas clicking on anchors and seeing anchors being highlightedin orange always follows the anchors→ settings direction.4.2.3 SearchText search is another eective approach for accessing settings. It essentially pro-vides a way to “jump” to a particular setting without having to locate an anchor orbrowse the settings space. Search is readily aorded by traditional cong les, butrarely oered in settings panels—except in complex software such as Eclipse. Thedownside with search is that the vocabulary problem [19] applies, as users canonly guess search terms [4]. Furthermore, relying on search may hinder users’ability to learn other settings. Although not yet implemented, CustomizationLayer is compatible with text search: UI anchors could be ltered out if theirassociated settings don’t match the search query, or visually highlighted if theydo.18Figure 4.8: The user has clicked on the “Starred” Smart List anchor. As a result,both the anchor and its associated settings are highlighted in orange. The useris now hovering on a dierent setting in the full panel—the visibility of the “All”Smart List. This setting and all the anchors associated with it are “reverse high-lighted” with a blue halo to create a visual relationship between them.194.2.4 Software ArchitectureOur software architecture was designed to be app-independent and extensible toother web apps. Chapter 5 explains why we chose web apps, and Wunderlist inparticular, as a concrete starting point for design and evaluation. Adding our Cus-tomization Layer to a web app requires: (1) an API to read and write the settings,and (2) a mapping between settings and visual elements of the interface, providedby the designers of the app. UI elements are represented in the mapping by CSSselectors, which allows among other things selecting many similar anchors via asingle CSS class (Appendix A). Since designers are already familiar with the set-tings of their app, creating the mapping should take at most a few hours, exceptfor very large applications.4.2.5 ImplementationSince Wunderlist doesn’t provide an API to its settings, we reverse-engineeredits front-end to access the underlying settings directly. Fortunately the settingsthemselves had human-readable names, which allowed us to localize and modifythem programmatically. We then manually mapped these settings to appropriateelements of the interface, which took about two hours. From this mapping, ourcode automatically generates anchors by creating copies of the DOM elementsmatched by the CSS selectors provided. Anchors are then positioned precisely ontop of the original element, creating the illusion that the elements themselves arehighlighted.When users click on an anchor, we used Angular.js [21], a front-end JavaScriptframework, to dynamically lter the list of settings shown in Minimal and Min-imal+Context, and to highlight the appropriate settings in Full+Highlight. Thesmooth transitions between the minimal and full panels were not straightforwardto implement, since the start and end positions must be computed in advance foreach setting, before the animation starts.We also prototyped a simple point-and-click web tool that could help design-ers without web programming skills to retrieve CSS selectors for the UI elementsthey are pointing at.20Chapter 5EvaluationWe focus our evaluation on assessing the usability of our Customization Layer.Given the novelty of this new mechanism, the two experiments presented herewere intended to be exploratory. We certainly hoped that Customization Layerwould help participants nd settings faster than browsing a traditional settingspanel; but which of the three variants would be the fastest or most preferred wasunknown. Further, we anticipated that the three variants would impact users’awareness of the full set of settings dierently: the more contextual informationa design provides, the greater the user’s awareness should be [15].Choice of the ApplicationAlthough our design approach is applicable to any software with a GUI and cus-tomizable via a settings panel, we had to choose a particular application on whichto evaluate it. We considered building a generic PTM app from the common fea-tures and visual styles we observed in our systematic application review (Sec-tion 3). The benet of building a generic app is increased generalizability, sincethe particularities of each app are averaged out. However, we would have had tohand-pick the customization settings available in this generic app, a process thatcould introduce some bias in the experiment. Ultimately, we thought it best toevaluate Customization Layer within an existing application with its actual set-tings. This choice maximizes the ecological validity of our experiment.We chose Wunderlist [20], one of the most popular PTM apps today (over 10million downloads on Android and iPhone) which oers a well-designed web in-terface. Of the 20 apps from our systematic review, Wunderlist had a particularlyclean settings panel, with numerous and varied settings. It therefore appearedto be a good baseline for a fair comparison. Furthermore, we were able to accessWunderlist’s settings directly on the web client, eectively bypassing the settingspanel to customize the app in real time.215.1 Experiment 1: Remote Mechanical TurkThe primary goal of this experiment was to evaluate the performance of our Cus-tomization Layer for changing settings, compared to a traditional settings panel.We used a between-subject design to avoid negative carry-over eects, becauseswitching back-and-forth between two very dierent mental models could havebeen confusing for participants. We deployed this experiment on Amazon Me-chanical Turk (MTurk), an online crowdsourcing platform that facilitates the re-cruitment of a large number of participants.5.1.1 MethodParticipantsAll 48 participants (aged 19-60, median 28.5, 16 females) were regular computerusers, and none had tried Wunderlist before. We had replaced 3 participants, whowere either 2.5 Inter-Quartile Range slower than others in their condition, or did2 IQR more errors than everyone else.TaskThe experiment consisted of a sequence of settings changes. At the beginningof each trial, a popup instructed participants to change one setting to a givenvalue (Figure 5.1). Pressing a “Go!” button would close the popup and start thetimer. The instructions were written in layman’s terms, and did not necessarilyuse the same words as the label of the target setting (Appendix B.1). For example,the instruction “Change shortcut for checking o todos” applied to the setting“Mark selected do-dos as completed” in the Shortcuts tab. To help ensure thatparticipants read the instructions before starting the trial, the “Go!” button waskept inactive for the amount of time required to read the instructions at an averagereading speed. During that time, the settings panel (in Control) or the UI anchors(in Customization Layer) were hidden, to prevent participants from planning theiractions before the timer was started.During the trial itself the instructions were visible at the top of the window,in case participants had forgotten them (Figure 5.2). Participants had to changeat least one setting before the “Next” button became available in the bar at thetop of the screen. Clicking this button would stop the timer, indicate whether thetask had been successfully completed, and move on to the next trial. We enforceda 2-minute time limit for each trial, after which the trial was marked as a timeoutand participants were taken to the next trial. If participants made mistakes, the22Figure 5.1: Modal dialog displayed at the beginning of each trial. The instruc-tions are repeated in a persistent progress bar at the top of the window, whichalso shows the bonus reward earned so far. Neither the settings panel nor the UIanchors are visible until participants press the “Go!” button to start the trial.settings that had been changed incorrectly were reset at the end of the trial, andthe target setting changed to the correct value.Our Customization Layer and the settings panel used in the Control Condi-tion are both accessed via the menu at the top-left of the screen, by clicking on“Customize” or “Settings” respectively. Since activating the customization mech-anism takes exactly the same amount of time in both cases, we didn’t include thistedious step in the experiment, and instead opened the appropriate customizationmechanism automatically in all conditions.MeasuresThe duration of a trial was measured between the time when participants pressedthe “Go!” button and when they changed a setting. At the end of the experiment,participants completed two recognition questionnaires (Appendix B.3.3) to assessincidental learning. The rst was a list of 10 tab names: 5 that appeared in theWunderlist settings panel, and 5 that we made up to closely resemble actual tabnames. Participants were asked to indicate for each tab whether or not it appearedin Wunderlist. Each correct answer was worth 1 point, each incorrect answer -1,so that answering at random would result in a score of zero. The second ques-tionnaire was similarly a list of 20 settings: 5 that participants had changed, 5 thatwere present in Wunderlist but that participants hadn’t changed, and 10 made up23Figure 5.2: Possible states of the progress bar, displayed at the top of the window.The “Next” button is disabled until participants change at least one setting. Af-ter changing a setting, participants must click this button to end the trial. If thecorrect setting was changed, a positive feedback is given and the bonus rewardis incremented. Otherwise, participants are told that they made an error and aretaken to the next trial.but plausible. The scoring was identical to the rst questionnaire. Finally, partic-ipants were asked to rate their satisfaction on a 7-point Likert scale: the extent towhich they liked or disliked the customization mechanism they were using, andhow easy it was to nd the settings they were looking for (Appendix B.3.2).ConditionsWe compared four customization mechanisms: the three variants of Customiza-tion Layer (M, M+C, F+H) and the actual settings panel oered by Wunderlist(Control), shown in Figure 4.3. While we accurately reproduced the structure ofWunderlist’s panel in our F+H prototype, the visual style was slightly dierent:Wunderlist generally looks more polished, with custom drop-downs and an iconon each tab.Wunderlist currently oers 41 settings in four tabs4. Out of these settings, wediscarded the application language one, as changing it would make the interfaceindecipherable for our participants. The experiment tasks were not equally di-cult: some settings were indeed harder to nd than others, especially if their textlabel was unclear. To reduce the variability between subjects, we partitioned thesettings into two groups of 20, carefully chosen to balance the type and locationof the settings they oered. Each participant was assigned one set. To assess anyearly learning, participants were asked to change the same group of 20 settingstwice, but in a dierent randomized order for each of the blocks of 20.The experiment required changing settings from a default value to a specied4Three other tabs show non-settings information, such as “upgrade your account” and“about this product.” We did include these tabs in our prototype.24one. It was not clear a priori whether the default value of a setting would impactthe ability of users to nd and change that setting. For instance, un-ticking acheckbox corresponds to negating its text label, which might be less intuitive thanticking it, following the original meaning of the label. Therefore, we controlledthe default values of the settings: participants either started the experiment withthe defaults oered by Wunderlist, or the opposite of these defaults.DesignA 2-factor mixed design was used: 4 customization mechanisms (M, M+C, F+H,Control; between-subjects) x 2 blocks (within-subject). There were two controlvariables, fully counterbalanced between participants: group of settings (group1, group 2) and default settings (normal, opposite). Each participant completed 2blocks of 20 trials, for a total of 1920 trials across all 48 participants.ProcedureAfter accepting the HIT (work assignment in MTurk), participants were asked tocreate a temporary account on Wunderlist, using a randomly-generated email ad-dress. Then participants had to drag a custom bookmarklet (a “bookmark applet”)to their bookmarks bar, and to click on it to load the experiment code in the Wun-derlist webapp. A series of popups walked participants through a quick tutorial,demonstrating the most useful features of Wunderlist. Participants were thengiven one practice trial, before starting any trial. In the 3 Customization Layerconditions, the practice trial contained a few additional instructions on how touse this new customization mechanism. Finally, participants were asked to com-plete the two recognition tests, to rate the customization mechanism they wereusing, and to complete a simple demographics questionnaire (Appendix B.3.1).The whole procedure lasted 32 minutes on average, and participants received anaverage compensation of $4.34, depending on their performance in the trials andrecognition questionnaires. (Incentivizing participants with a variable bonus re-ward is a standard practice for MTurk studies.)5.1.2 MTurk ResultsWe ran a mixed-design ANOVA on the duration of trials. The data was log-transformed to satisfy the assumption of normality, and we used medians to re-duce the inuence of outliers. The eect sizes reported are generalized eta-square(η2G), interpreted as follows: .02, .13, .26 for a small, medium, and large eect sizerespectively [1]. As shown on Figure 5.3, there was a main eect of customiza-tion mechanism (F3,44 = 3.58, p < .05, η2G = 0.17), as well as block (F1,44 = 124.63,25Figure 5.3: Experiment 1: 95% condence intervals of the median duration perCustomization Mechanism and Block. M is signicantly faster than F+H and Con-trol, but not signicantly dierent from M+C. (N=48)p < .001, η2G = 0.32), but no interaction between the two. M was signicantly fasterthan Control (p < .05, 35% faster) and F+H (p < .05, 30% faster). No other pair wassignicantly dierent. All pairwise comparisons use a Bonferroni correction un-less otherwise mentioned. As anticipated, there was no eect of the two controlvariables (the group of settings and default settings used). There were few time-outs and errors across all conditions (20 and 54, respectively, for 1920 trials), andthey had no signicant eect on our results.A single-factor ANOVA on the tabs recognition scores found a signicant ef-fect of customization mechanism (F3,44 = 5.82, p < .01, η2G = 0.28). M scored lowerthan both F+H (p = .05) and Control (p < .05), by 2 out of 10 points lower on av-erage. No other pair was signicantly dierent. There was no dierence either inthe recognition scores for settings, nor in the subjective ratings provided at theend of the experiment. The median rating for ease of use and satisfaction was thesame for each of the four customization mechanisms: 2 on a scale ranging from-3 to +3.5.2 Experiment 2: Face-to-Face in the LabAlthough MTurk provided quick access to a diverse set of participants at a mod-erate cost, conducting remote experiments has limitations in terms of the insightsand qualitative feedback that can be gathered. Thus, we ran a second experimentin a face-to-face lab setting. This time the customization mechanism was treatedas a within-subject factor, to allow participants to make informed comparativejudgments.265.2.1 Method Dierences from Study 1We recruited 12 participants (age 21-42, median 24.5, 5 females), all regular com-puter users. Two had tried Wunderlist before, but were not using it regularly. Theexperiment task was identical, except that participants were encouraged to talkaloud. The experiment t in a single one-hour session, for which participants re-ceived a $10 compensation. Contrary to MTurk, there was not variable monetaryincentive based on performance.A single-factor within-subject design was used, with the same four customiza-tion mechanisms. Participants had to change 10 dierent settings with each mech-anism, using all the 40 settings available in Wunderlist (Appendix C.1), for a totalof 12 x 40 = 480 trials. All participants started with the same defaults settings,since the previous experiment did not nd any eect of this control factor. Thethree Customization Layer variants were blocked together, to limit the numberof times participants had to switch between the two mental models. Half of theparticipants began with the Customization Layer block, the other half with theControl condition. Within the Customization Layer block, the presentation or-der of the three variants was fully counterbalanced. The settings were randomlyordered across all blocks.The experiment procedure was similar to the MTurk study, except for a fewchanges reecting the within-subject design. After each mechanism condition,participants were asked to rate on a 7-point Likert scale the mechanism on threemetrics: ease of use, perceived speed, and satisfaction (Appendix C.4.2). At the endof the experiment, participants were asked to rank the four mechanisms on thesame metrics (Appendix C.4.3). No recognition questionnaires were administered,since it would have been dicult to tease apart the learning that happened in eachcondition. The experiment was concluded by a brief semi-structured interview.Participants were asked about their perception of the four dierent customizationmechanisms, and we gathered insights on how they developed a mental model ofAnchored Customization.5.2.2 Face-to-Face Lab ResultsWe rst present the quantitative results of this second experiment, followed bythe qualitative insights we gathered.Quantitative ResultsWe ran a repeated measures ANOVA on the duration of trials. As in Experiment1, the data was log-transformed and we used medians. As shown on Figure 5.4,there was a main eect of customization mechanism (F3,33 = 5.61, p < .01, η2G =27Figure 5.4: Experiment 2: 95% condence intervals of the median duration perCustomization Mechanism. M+C is signicantly faster than F+H and Control,but not signicantly dierent from M. (N=12)0.16). M+C was faster than Control (p < .05, 36% faster) and F+H (p = .06, 40%faster). No other pair was signicantly dierent. The order in which participantssaw the conditions (Control rst, or Customization Layer rst) had no signicanteect either.The subjective ratings collected after each block were analyzed with a Fried-man test. No dierences were found for satisfaction and ease of use. Perceivedspeed was borderline signicant (χ23 = 7.1, p = .07), with the majority of the dif-ference coming from M+C reported to be faster than F+H (p = .1). In terms of theranking data, participants were almost evenly divided between CustomizationLayer (6 top ranks) and Control (7) —one participant ranked two mechanisms asbest. There was also no clear consensus among which of the three CL variantswas best: with 1, 3, and 2 votes going to M, M+C, and F+H respectively.Qualitative ResultsThe comments made by participants during the experiment and their answersduring the interview were recorded by the experimenter. We report the ndingsof a thematic analysis [6].Anchored Customization Nine out of 12 participants acquired the intendedmental model of Anchored Customization: 6 acquired it immediately with the rstvariant of Customization Layer they encountered, the other 3 with the secondvariant. Once participants understood the concept of Anchored Customization,they were able to use it successfully: “I’m used to all the settings hidden away ina menu, but I think this (Minimal) makes a lot of sense” (P5), “I think the pointof this (F+H) is that I don’t need to think under which category each setting is”(P3). In some cases, participants even imagined appropriate anchors that were28not actually present in Wunderlist, which shows a clear grasp of the Anchor Cus-tomization Concept. For instance, P1 said: “I was looking for a printing icon”, whilesearching for the setting “don’t print completed todos when printing a list”.Three out of 12 participants did not seem to have acquired the intended men-tal model: in more than 50% of trials, they clicked on any anchor, from whichthey immediately opened the full panel and browsed the tabs to nd the desiredsetting. P4, who was in the Minimal condition, even complained: “I don’t like‘showing all’ all the time, it’s too many clicks”. These 3 participants were amongthe 6 who started the experiment in the Control condition, which could have neg-atively aected their behavior in the subsequent Customization Layer conditions.The only time these participants searched for an appropriate anchor (rather thanjust any anchor) was when the target setting was related to one of the Smart Lists,in the left sidebar (see Figure 4.5). In these cases, participants may have reasonedby analogy with their practice trial, which instructed them to change a (dierent)Smart Lists setting.Customization Layer Variants Interestingly, at least 3 participants did notnotice any dierence between M and M+C during the trials, and only realizedthat they were dierent at the end when they were asked to rank the four cus-tomization mechanisms. Some participants liked the extra structure provided bythe tab names in M+C: “In Minimal I don’t know where I am in the greater struc-ture. [M+C] is more organized” (P1); “The links in [M+C] are clearer, you knowwhereyou’re going” (P8).In F+H, at least 4 participants did not see the orange highlighting, althoughit was clearly visible (Figure 4.7). This could be a case of inattentional blind-ness [34]: participants who hadn’t yet acquired the intended mental model mayhave discarded the highlighting because it had no meaning to them. Other par-ticipants thought of the highlighting as a soft suggestion from the system: “Atone point I assumed the highlighting was like a hint” (P8), “[the system] is saying:I think you are looking for this, but I’m not sure” (P12). However, no participantseemed to notice the reverse highlighting, from the settings panel to the anchors.This is a relatively subtle design element, which may not be noticeable in a shortexperiment with constraint tasks.A few participants suggested ideas for yet another variant of the Customiza-tion Layer. The settings would be displayed in an even more minimal panel, show-ing only the anchored settings. Instead of providing a “show all” link at the bottomof the panel itself, the full settings panel could be accessed from its usual location,typically via a menu at the top of the screen or a cog icon. Providing a xed ac-cess points to all the settings may match users’ mental models better: “If a setting29doesn’t have an anchor, I would look for a general menu with all settings” (P12). Itwould also address the concern of P10: “When clicking on show all, you are notsure that it will show all the settings” —because the “show all” link appears in anotherwise very contextualized panel.Wunderlist Although it was not an explicit goal, this experiment revealed someusability problems with Wunderlist’s settings panel, which was used as the Con-trol condition. The most problematic aspect is that 10 settings are hidden undera “show more” button in the Shortcuts tab. This design decision was probably in-tended to avoid showing all 20 shortcuts settings at once, by hiding the ones lesslikely to be changed. However, this “show more” button was not salient enough,and 10/12 participants missed it the rst time they looked for one of the settings itwas hiding. Interestingly, this problem was naturally alleviated in all Customiza-tion Layer variants: in both M and M+C, only the relevant settings are shownwhen clicking on an anchor, so there is no need for a hiding mechanism. F+Hautomatically unhides all of the 10 more advanced shortcuts if one of them ishighlighted, and scrolls down to the rst highlighted setting.The categories chosen by Wunderlist’s designers seemed to generally makesense for our participants. Yet a few settings appeared to be misclassied, whichresulted in longer search times. For instance, the setting “enable sound for newnotications” was in the General tab, next to “enable sound for checking-o atodo”. However, most participants looked for it rst under the Notications tab,which is a more precise category. This kind of ambiguity doesn’t arise in An-chored Customization, since the same setting can be mapped to any meaningfulanchor.Unsurprisingly, we found that the quality of the text labels plays a key rolein helping users nd settings. Indeed, participants appear to “scan” each tab veryquickly, looking for keywords that would match the instructions they were given.Because of the vocabulary problem [19], nding the appropriate keywords to de-scribe a setting can be challenging for designers. But once a word is chosen, itshould be used consistently throughout the settings panel. For instance, our par-ticipants were very confused by the fact that notications are sometimes referredto as “activities” in the current version of Wunderlist.Visual Design Our primary concern while designing the Customization Layerwas interaction design, not aesthetics. However, as is often the case in HCI experi-ments, some participants tended to focus on the visual appearance of the dierentinterfaces more than their behavior: 4/12 participants indicated that they liked thelook of the Wunderlist settings panel better. In particular, the presence of an icon30on each tab was appreciated (P9). This might explain why 7 participants rankedControl as well or above Customization Layer: as P10 puts it, “for customers prettyis more important”.The visual design of our Customization Layer was optimized to remain us-able across apps with dierent visual styles. We chose a dark overlay with whiteanchors to maximize visibility, independently of the color palette of the underly-ing application. P5 found it too dark, and not colorful enough. Furthermore, P5also disliked the fact that the anchors were in some places not well aligned witheach others, or that the elements inside them were not properly centered. This isa consequence of generating anchors from regular UI elements, which were notoriginally designed for that purpose. Of course, for commercial-grade softwarethe appearance of an application’s customization layer could be tweaked by thedesigners to be more aesthetically pleasing.5.3 Secondary Exploratory AnalysesThe fact that we obtained dierent performance results for M and M+C in ourtwo experiments was surprising and prompted us to probe the data further. Mwas found to be faster than F+H and Control in the rst study, whereas it wasM+C that was faster than F+H and Control in the second study. In both cases, thep-value and eect sizes are similar. A closer look shows that these results are notcontradictory: M and M+C were never found to be signicantly dierent fromeach other, nor signicantly slower than F+H or Control.In order to tease this apart, we looked at the logged data with a focus to under-stand participants’ approach to nding the target setting. Of particular interest iswhether participants found a correct anchor to click on to access the target set-ting (what we call “anchor search”), or if they defaulted to clicking on any anchorfrom which they then browsed the full settings panel (“full panel search”). Foreach participant, we computed the percentage of trials in which they found a cor-rect anchor, out of the total number of trials in the experiment. The distribution ofthis metric across all participants is clearly bimodal, both on MTurk and in the lab(Figure 5.5). The position of the “valley” (local minimum) of the two distributionsis also similar: 61% on MTurk, 63% in the lab.This analysis points to a likely hidden variable in our data: the search strategyused by participants. Indeed the three Customization Layer conditions vary alongthis extra dimension, as shown in Figure 5.6. While these variations might explainthe dierence between the two minimal variants and F+H, the distributions forM and M+C are too similar to explain the dierence in results that we observedbetween the two experiments.31Figure 5.5: Density plots of the distribution of the percentage of trials in whichparticipants selected a “correct” anchor—one associated with the target setting.Participants in the right mode followed an “anchor search” strategy, while partic-ipants in the left mode resorted to a “full panel search”.Figure 5.6: Density plots of the percentage of trials in which participants selecteda “correct” anchor, for each Customization Layer variant, on MTurk (top) and inthe lab (bottom). The minimal variants all have a higher right mode (“anchorsearch”), while F+H has higher left mode (“full panel search”).32For the next step in our exploratory analysis, we looked only at the data fromparticipants who adopted an “anchor search” strategy to see if we could see anydierences between M and M+C. More specically, we re-ran the ANOVA on trialduration only with participants whose average percentage of correct anchor se-lected was above the local minimum of the bimodal distribution. For the MTurkexperiment, M and M+C were both faster than Control (p < .001 each), and notsignicantly dierent from each other. For the lab experiment, M and M+C werealso both faster than both F+H and Control (all p < .05). This secondary analy-sis suggests that the results of MTurk and the face-to-face experiments may beconsistent, as long as you take search strategy into account: M and M+C are bothfaster than Control, and not signicantly dierent from each other. As with anynon-planned analysis, these results need to be interpreted with caution.33Chapter 6DiscussionThe results of our experiments are promising: whether on MTurk or in the lab, oneof the minimal variants was signicantly faster than Control, with a medium eectsize; and no variant was slower than Control. These performance improvementswere obtained even though participants were exposed to Anchored Customizationfor the rst time, and received little information to help them build an appropriatemental model. We now discuss the insights gathered on the three CustomizationLayer variants, and reect on the possibilities oered by this new customizationmechanism.6.1 Reconciling the Performance ResultsM and M+C performed dierently in the two experiments. Our secondary anal-yses revealed a likely hidden variable, namely search strategy. For participantsusing the anchor search strategy, the results of the two experiments are consis-tent: M and M+C are both faster than Control, and not signicantly dierent fromeach other. Hence there must be a signicant dierence in how the “full panelsearch” participants performed in the two experiments which translated into dif-ferent relative performances for M and M+C on MTurk and in the lab. It mightjust be a statistical uke. For instance, looking at the data shows that on MTurkall 5 of of the M+C participants that did a “full panel search” were dispropor-tionately slower than the “full panel search” participants in all other conditions.This inconsistency suggests that the randomization of participant assignment toconditions may not have equalized individual dierences. More research will beneeded to further assess this issue.Few participants in the F+H condition used the “anchor search” strategy: 2/12on MTurk, 5/12 in the lab. It could be that highlighting settings in a panel is nota strong enough cue to help users acquire the Anchored Customization mentalmodel. Since the structure of the settings panels is retained in F+H, there may bea strong transfer eect that encourages users to default to the “full panel search”,instead of exploring anchors. Since few participants used F+H the way it wasintended, we cannot conclude with certainty on its potential performance: is F+Hnecessarily slower than M and M+C, or can it be as fast when properly used? In34any case, F+H is not slower than Control, so it would not be detrimental. Sincesome participants perceived the highlighting as a hint, F+H could be used as a“softer” version of Customization Layer for users who might not be comfortablewith the degree of minimalism of the two minimal variants.6.2 Performance/Awareness Trade-OOn MTurk, M was signicantly faster than F+H and Control, but it also scoredsignicantly lower on the recognition of tab names. This could be interpreted asa performance/awareness trade-o found in other multi-layered interfaces [15].Yet, users in M did just as well as others in terms of recognizing the settingsthemselves. Thus, the awareness trade-o here seems to aect only awarenessof the structure of the upper layer (the full settings panel), not its content, asthere were no dierences in recognition scores for the settings themselves. Thisis not entirely surprising: in Customization Layer, all the settings are accessiblevia the minimal panel, albeit from dierent anchors. By contrast, the personalizedinterfaces studied by Findlater et al. [15] only display a static subset of features inthe rst layer, while the others were only visible in a dierent layer.6.3 Applicability to Other Desktop SoftwareWe focused our work on task management applications, and the question remainswhether Anchored Customization could provide similar benets for other typesof software. At one extreme is text editors, which are often highly customiz-able. These editors typically have very few always-visible UI elements, and relymostly on menus and keyboard shortcuts. Thus, they are not well-suited for An-chored Customization. Furthermore, text editor users are generally comfortablecustomizing their software by editing the cong les directly.The opposite extreme is complex software applications designed for non-technicalusers, such as Adobe Creative Suite. These applications have many widgets thatprovide access to lots of features. Anchoring settings to these various UI ele-ments is possible, but the resulting Customization Layer may be overwhelming.However, the main interface of these applications can itself be overwhelming,especially for new users. Anchored Customization would simply reect the com-plexity of the underlying software.356.4 Applicability to Handheld DevicesOur systematic review showed that mobile apps organize their settings in verysimilar ways to desktop applications. Mobile apps generally rely on icons andbuttons for user input, since keyboard shortcuts and extensive menus are notavailable. As such, they are well suited for Anchored Customization. The lim-ited screen real estate would warrant using a minimal variant. The anchoredcustomization mechanism could be activated via a standard application menu,but touchscreens oer other possibilities: for instance, users could long-press ormulti-tap an anchor to see its associated settings, or use a special standardizedgesture to activate the Customization Layer. We observed that mobile applica-tions generally oer fewer settings than desktop apps. The introduction of awell-designed customization mechanism could lead to more customizable mobileapps.6.5 Interpretation GulfAnchored Customization was designed to reduce the gulf of execution inherentwith settings panels, but it may also reduce the gulf of interpretation [32]. Intraditional settings panels, users do not always know if modifying a settings hadthe intended eect. Anchored Customization mitigates this problem by reducingthe spatial and temporal indirection [3] between settings and the user interface,especially for settings that cause an immediate visual change to their anchors. Inany case, designers should provide an easy “undo” feature, especially for settingsthat don’t cause an immediate visual change. A simple solution would be to add abackward arrow in the “Preferences” or “Customize” entry in the usual applicationmenu.6.6 The Developers’ Point of ViewBeyond its benets to users, our Anchored Customization approach may alsochange the way designers and developers think about customization. Althoughthey are not required to change any setting to adopt Anchored Customization,the process of mapping settings to UI elements could have a positive eect onthe settings oered. For instance, designers might realize that some parts of theinterface have no setting anchored to them, which would highlight a potentialopportunity for providing settings that cater to this area. Creating and labelingsettings may also become faster, since the problem of nding appropriate wordsto refer to interface elements is mitigated by the context provided by the anchors.36Finding categories to organize settings into tabs might also become superuous.6.7 LimitationsWhile our results are promising, our experiments had limitations, which comemostly from the challenges of evaluating customization mechanisms in an arti-cial setting.Personalization systems are indeed notoriously dicult to evaluate. End-usercustomization occurs infrequently in the real world, and at a varying degree de-pending on the individuals, the software they use, and a variety of external fac-tors [27]. Long-term eld experiments, such as the one conducted by McGrenereet al. [30], best capture these behaviors and measure the eects of customizationmechanisms. These eld experiments are typically long and costly to run, and asa result seldom undertaken. The work presented here did not aim to study com-plex personalization behaviors over time. Instead, we focused on the usability (orlack thereof) of the customization mechanisms present in today’s consumer soft-ware. Evaluating the long-term eects of the Anchored Customization approachremains future work.Furthermore, we compared customization mechanisms mainly on how quicklyparticipants could nd a designated setting. However, in a real world situation,the awareness (or lack thereof) of which settings are available likely plays an im-portant role. In traditional settings panels, awareness is gained by serendipitousdiscovery (also referred to as “incidental learning” [15]) : users happen to noticea setting of interest while searching the panel for another one. Serendipity canalso happen in Anchored Customization, but the notion of proximity is relative tothe anchors, instead of the settings panel’s structure. Because our experiment taskwas time-constrained, these dierent forms of serendipity were not well captured.To maximize ecological validity, we used Wunderlist’s actual settings panel asa baseline for the Control condition. But some participants focused their feedbackon its high quality visual design relative to our prototyped Customization Layervariants. Recreating this panel in the same visual style as our prototypes mighthave increased the internal validity of our experiments.In retrospect, the practice trial that participants were given could have beenmore eective at conveying the intended model. Participants performed only onetrial, thus only had to click on one anchor to complete it. However, to reallyunderstand the concept of Anchored Customization, one must click on at leasttwo anchors to see that the settings oered are dierent. Some participants tooktime during the practice trial to explore the Customization Layer on their own,clicking on multiple anchors to see the outcome. This free exploration seemed37more eective than our practice trial, and would also be more similar to real-world conditions.Finally, our experiments only included one application with a medium numberof settings. It is possible that the number of settings oered by an app aects therelative performance of Customization Layer and settings panels.38Chapter 7Conclusions and Future WorkAnchored Customization is an approach that places settings in context within theapplication interface, so that users are not required to learn the abstract structureof a settings panel. A systematic review of a set of Personal Task Managementapps found that approximately 84% of the settings could be anchored in the UI.Our Customization Layer prototype reveals all the anchors as aordances for cus-tomization. We designed three variants of Customization Layer based on multi-layered interfaces, and implemented these variants on top of a popular web ap-plication for task management, Wunderlist. Two experiments (Mechanical Turkand face-to-face) showed that the two minimalist variants were 35-36% faster thanWunderlist’s settings panel. A majority of users acquired the intended mentalmodel of Anchored Customization, and were able to use it successfully for nd-ing a variety of settings in Wunderlist. Some users did not seem to acquire theintended mental model, yet they were able to complete the experiment tasks in thefamiliar structure of a settings panel, thanks to the fallback access we provided ineach Customization Layer variant. Better instructions and more guidance mightbe needed to help these users understand the Anchored Customization approach.7.1 Long-Term EectsEvaluating the long term eects of this customization approach remains futurework. A longitudinal eld study would determine if a more usable customizationmechanism actually increases users’ likelihood to customize. It could also addresssubtle, but important questions, such as: are dierent variants of CustomizationLayer better suited to users with dierent level of expertise? Given the opportu-nity to use multiple variants, as well as a direct right-click access to any anchor’ssettings, which one would users choose? Would it depend on the users’ famil-iarity with the application and its settings, or is it simply a matter of personalpreference?Our prototype could be distributed to real users of Wunderlist as a browserextension, or adapted to other applications to compare the eect of dierent typesof settings and dierent application domains. Augmenting multiple applicationswith our Customization Layer would also help to verify our assumption that An-39chored Customization requires only one-time learning: once this approach is un-derstood in the context of one particular app, the mental model should be trans-ferable to other apps.7.2 Generating the MappingCurrently, the designers of an app need to provide the mapping between settingsand UI elements. With the growing popularity of advanced front-end frameworksin web development, it might be possible to use code analysis techniques to gen-erate part of the mapping automatically. The interface could be analyzed to deter-mine which elements are aected by a setting (e.g. changing their visibility, posi-tion, or appearance), and which elements trigger a function that is itself modiedby a setting (e.g. changing the eect of clicking a button). The UI elements identi-ed would automatically become anchors for the appropriate settings. Designersthen only need to verify and extend this mapping, anchoring extra settings byexploiting mental associations—to complement the functional associations thatwere detected programmatically.Another possibility would be to involve users in the mapping process. Con-trary to crowdsourced contextual help [8], users cannot be expected to generatethe entire mapping themselves, but they could tweak a designer’s mapping to bet-ter match their expectations. The idea of rening the mapping by aggregating datafrom individual users could be expanded to other customization opportunities aswell. For instance, the most popular extensions and plugins could be anchored tothe UI.7.3 Concluding RemarksEnd-user customization is fundamentally a dicult trade-o between upfrontcosts and uncertain benets in the future. Users weigh the time and eort neededto customize against the satisfaction of using an interface matching their owntaste and preferences, and the performance gains of using a software adapted totheir work patterns. For developers and designers, providing multiple versions ofan application can require signicant additional development costs, which can bedicult to justify in an agile world where the time-to-market is critical. And theseeorts might eventually be in vain if users never change the default settings.Settings panels represent an equilibrium in this trade-o space. Their genericUIs are inexpensive to develop, but their lack of appeal and numerous usabilityissues can repel users. As a result, few users customize their applications, or doso only rarely. This in turn does not encourage developers to provide better cus-40tomization mechanisms, and so on. Introducing a new customization mechanismtherefore requires careful consideration of its costs and benets.Much of the prior work reported in the literature has focused on increasingthe potential benets of customization, by proposing more powerful or more ex-pressive customization mechanisms. But these expected benets do not neces-sarily justify the additional costs incurred by users or developers. By contrast,Anchored Customization minimizes the extra costs for developers by leveragingthe customization opportunities already available in existing settings panels. Italso reduces the cost of customization for end-users, who can nd desired set-tings faster without having to learn the abstract structure of a settings panel. Assuch, this novel approach to customization has the potential to be adopted widely.Our ultimate goal is to increase users’ satisfaction by interacting with applicationsbetter suited to their own needs and preferences.41Bibliography[1] Roger Bakeman. Recommended eect size statistics for repeated measuresdesigns. Behavior Research Methods, 37(3):379–384, 2005.[2] Patrick Baudisch, Desney Tan, Maxime Collomb, Dan Robbins, Ken Hinck-ley, Maneesh Agrawala, Shengdong Zhao, and Gonzalo Ramos. Phosphor:Explaining transitions in the user interface using afterglow eects. In Pro-ceedings of the 19th Annual ACM Symposium on User Interface Software andTechnology, UIST ’06, pages 169–178, New York, NY, USA, 2006. ACM.[3] Michel Beaudouin-Lafon. Instrumental interaction: An interaction modelfor designing post-WIMP user interfaces. In Proceedings of the SIGCHI Con-ference on Human Factors in Computing Systems, CHI ’00, pages 446–453,New York, NY, USA, 2000. ACM.[4] Ofer Bergman, Ruth Beyth-Marom, Ra Nachmias, Noa Gradovitch, andSteve Whittaker. Improved search engines and navigation preference in per-sonal information management. ACM Transactions on Information Systems,26(4):20:1–20:24, October 2008.[5] Anastasia Bezerianos, Pierre Dragicevic, and Ravin Balakrishnan.Mnemonic rendering: An image-based approach for exposing hiddenchanges in dynamic displays. In Proceedings of the 19th Annual ACMSymposium on User Interface Software and Technology, UIST ’06, pages159–168, New York, NY, USA, 2006. ACM.[6] Virginia Braun and Victoria Clarke. Using thematic analysis in psychology.Qualitative Research in Psychology, 3(2):77–101, 2006.[7] Andrea Bunt, Cristina Conati, and Joanna McGrenere. Supporting interfacecustomization using a mixed-initiative approach. In Proceedings of the 12thInternational Conference on Intelligent User Interfaces, IUI ’07, pages 92–101,New York, NY, USA, 2007. ACM.[8] Parmit K. Chilana, Andrew J. Ko, and Jacob O. Wobbrock. LemonAid:Selection-based crowdsourced contextual help for web applications. In Pro-42ceedings of the SIGCHI Conference on Human Factors in Computing Systems,CHI ’12, pages 1549–1558, New York, NY, USA, 2012. ACM.[9] Parmit K. Chilana, Andrew J. Ko, Jacob O. Wobbrock, and Tovi Grossman. Amulti-site eld study of crowdsourced contextual help: Usage and perspec-tives of end users and software teams. In Proceedings of the SIGCHI Confer-ence on Human Factors in Computing Systems, CHI ’13, pages 217–226, NewYork, NY, USA, 2013. ACM.[10] Matjaz Debevc, Beth Meyer, Dali Donlagic, and Rajko Svecko. Design andevaluation of an adaptive icon toolbar. User Modeling and User-Adapted In-teraction, 6(1):1–21, 1996.[11] Morgan Dixon and James Fogarty. Prefab: Implementing advanced behav-iors using pixel-based reverse engineering of interface structure. In Proceed-ings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’10, pages 1525–1534, New York, NY, USA, 2010. ACM.[12] James R. Eagan, Michel Beaudouin-Lafon, and Wendy E. Mackay. Crackingthe cocoa nut: User interface programming at runtime. In Proceedings ofthe 24th Annual ACM Symposium on User Interface Software and Technology,UIST ’11, pages 225–234, New York, NY, USA, 2011. ACM.[13] W. Keith Edwards, Scott E. Hudson, Joshua Marinacci, Roy Rodenstein,Thomas Rodriguez, and Ian Smith. Systematic output modication in a 2Duser interface toolkit. In Proceedings of the 10th Annual ACM Symposium onUser Interface Software and Technology, UIST ’97, pages 151–158, New York,NY, USA, 1997. ACM.[14] Leah Findlater and Joanna McGrenere. Impact of screen size on performance,awareness, and user satisfaction with adaptive graphical user interfaces. InProceedings of the SIGCHI Conference on Human Factors in Computing Sys-tems, CHI ’08, pages 1247–1256, New York, NY, USA, 2008. ACM.[15] Leah Findlater and Joanna McGrenere. Beyond performance: Feature aware-ness in personalized interfaces. International Journal of Human-ComputerStudies, 68(3):121 – 137, 2010.[16] Gerhard Fischer. Adaptive User Interfaces, chapter Shared Knowledge in Co-operative Problem-Solving Systems – Integrating Adaptive and AdaptableComponents, pages 49–68. Elsevier Science Publishers B.V., 1993.43[17] Gerhard Fischer and Elisa Giaccardi. Meta-design: A framework for thefuture of end-user development. In Henry Lieberman, Fabio Paternò, andVolker Wulf, editors, End User Development, volume 9 of Human-ComputerInteraction Series, pages 427–457. Springer Netherlands, 2006.[18] Stephen Fitchett, Andy Cockburn, and Carl Gutwin. Improving navigation-based le retrieval. In Proceedings of the SIGCHI Conference on Human Factorsin Computing Systems, CHI ’13, pages 2329–2338, New York, NY, USA, 2013.ACM.[19] G. W. Furnas, T. K. Landauer, L. M. Gomez, and S. T. Dumais. The vocabularyproblem in human-system communication. Communications of the ACM,30(11):964–971, November 1987.[20] 6 Wunderkinder GmbH. Wunderlist. Version 3.12.3, 2015. http://www.wunderlist.com (Retrieved on 2015-07-01).[21] Google. Angular.js. Version 1.3.11, 2015. http://angularjs.org/ (Re-trieved on 2015-01-26).[22] Mona Haraty, Diane Tam, Shathel Haddad, Joanna McGrenere, and Char-lotte Tang. Individual dierences in personal task management: A eld studyin an academic setting. In Proceedings of Graphics Interface 2012, GI ’12, pages35–44, Toronto, Ont., Canada, Canada, 2012. Canadian Information Process-ing Society.[23] Eric Horvitz, Jack Breese, David Heckerman, David Hovel, and Koos Rom-melse. The Lumière project: Bayesian user modeling for inferring the goalsand needs of software users. In Proceedings of the Fourteenth Conference onUncertainty in Articial Intelligence, UAI’98, pages 256–265, San Francisco,CA, USA, 1998. Morgan Kaufmann Publishers Inc.[24] J. Johnson, T.L. Roberts, W. Verplank, D.C. Smith, C.H. Irby, M. Beard, andK. Mackey. The Xerox Star: a retrospective. Computer, 22(9):11–26, Sept1989.[25] Helge Kahler, Anders Mørch, Oliver Stiemerling, and Volker Wulf. Tai-lorable systems and cooperative work. Computer Supported CooperativeWork (CSCW), 9(1):1–4, 2000.[26] Henry Lieberman, Fabio Paternò, Markus Klann, and Volker Wulf. End-user development: An emerging paradigm. In Henry Lieberman, Fabio Pa-ternò, and Volker Wulf, editors, End User Development, volume 9 of Human-Computer Interaction Series, pages 1–8. Springer Netherlands, 2006.44[27] Wendy E. Mackay. Triggers and barriers to customizing software. In Pro-ceedings of the SIGCHI Conference on Human Factors in Computing Systems,CHI ’91, pages 153–160, New York, NY, USA, 1991. ACM.[28] Thomas W. Malone, Kum-Yew Lai, and Christopher Fry. Experiments withOval: A radically tailorable tool for cooperative work. ACM Transactions onInformation Systems, 13(2):177–205, April 1995.[29] Justin Matejka, Tovi Grossman, and George Fitzmaurice. Patina: Dynamicheatmaps for visualizing application usage. In Proceedings of the SIGCHIConference on Human Factors in Computing Systems, CHI ’13, pages 3227–3236, New York, NY, USA, 2013. ACM.[30] Joanna McGrenere, Ronald M. Baecker, and Kellogg S. Booth. An evaluationof a multiple interface design solution for bloated software. In Proceedingsof the SIGCHI Conference on Human Factors in Computing Systems, CHI ’02,pages 164–170, New York, NY, USA, 2002. ACM.[31] Xiaojun Meng, Shengdong Zhao, Yongfeng Huang, Zhongyuan Zhang,James Eagan, and Ramanathan Subramanian. WADE: Simplied GUI add-on development for third-party software. In Proceedings of the 32nd AnnualACM Conference on Human Factors in Computing Systems, CHI ’14, pages2221–2230, New York, NY, USA, 2014. ACM.[32] Donald A. Norman. Interfacing thought: Cognitive aspects of human-computer interaction. chapter Cognitive Engineering—Cognitive Science,pages 325–336. MIT Press, Cambridge, MA, USA, 1987.[33] Ben Shneiderman. Promoting universal usability with multi-layer interfacedesign. In Proceedings of the 2003 Conference on Universal Usability, CUU ’03,pages 1–8, New York, NY, USA, 2003. ACM.[34] Daniel J Simons and Christopher F Chabris. Gorillas in our midst: Sustainedinattentional blindness for dynamic events. Perception-London, 28(9):1059–1074, 1999.[35] Gunnar Stevens and Torben Wiedenhöfer. Chic - a pluggable solution forcommunity help in context. In Proceedings of the 4th Nordic Conference onHuman-computer Interaction: Changing Roles, NordiCHI ’06, pages 212–221,New York, NY, USA, 2006. ACM.[36] Wolfgang Stuerzlinger, Olivier Chapuis, Dusty Phillips, and Nicolas Rous-sel. User interface façades: Towards fully adaptable user interfaces. In Pro-45ceedings of the 19th Annual ACM Symposium on User Interface Software andTechnology, UIST ’06, pages 309–318, New York, NY, USA, 2006. ACM.[37] Gaurav Paruthi Tao Dong, Mark S. Ackerman, Mark W. Newman. SocialOverlays: Collectively Making Websites More Usable. volume 8120 of Lec-ture Notes in Computer Science. Springer Berlin Heidelberg, Berlin, Heidel-berg, 2013.[38] Volker Wulf and Björn Golombek. Direct activation: A concept to encouragetailoring activities. Behaviour & Information Technology, 20(4):249–263, 2001.46Appendix AMapping of settings to UIelements in WunderlistThis appendix shows the JSON le that was used to encode the mapping of set-tings to UI elements in Wunderlist’s web application. UI elements are representedby a CSS selector, which usually consists of CSS classes (starting with a “.”) and ids(starting with a “#”). A comma can be used to concatenate multiple CSS selectorsinto one. The settings associated with each anchor are specied via an array ofsettings names, which were chosen by the developers of Wunderlist.1 [{2 "selector": ".addTask -input",3 "settings": ["new_task_location", "shortcut_add_new_task"]4 },5 {6 "selector": ".starred -wrapper, .star -wrapper",7 "settings": ["behavior_star_tasks_to_top", "shortcut_mark_task_starred"]8 },9 {10 "selector": ".taskItem -duedate, .detail -date .token_0",11 "settings": ["date_format", "time_format", "start_of_week"]12 },13 {14 "selector": ".detail -checkbox.checkBox, .taskItem -checkboxWrapper.checkBox",15 "settings": ["sound_checkoff_enabled", "shortcut_mark_task_done"]16 },17 {18 "selector": ".taskItem -titleWrapper",19 "settings": ["show_subtask_progress", "shortcut_select_all_tasks", "shortcut_copy_tasks", "shortcut_paste_tasks"]20 },21 {22 "selector": ".filters -collection .sidebarItem[rel=’inbox ’]",23 "settings": ["shortcut_goto_inbox"]24 },4725 {26 "selector": ".filters -collection .sidebarItem[rel=’all ’]",27 "settings": ["smartlist_visibility_all", "shortcut_goto_filter_all"]28 },29 {30 "selector": ".filters -collection .sidebarItem[rel=’assigned ’]",31 "settings": ["smartlist_visibility_assigned_to_me", "shortcut_goto_filter_assigned"]32 },33 {34 "selector": ".filters -collection .sidebarItem[rel=’completed ’]",35 "settings": ["smartlist_visibility_done", "shortcut_goto_filter_completed"]36 },37 {38 "selector": ".filters -collection .sidebarItem[rel=’starred ’]",39 "settings": ["smartlist_visibility_starred", "shortcut_goto_filter_starred"]40 },41 {42 "selector": ".filters -collection .sidebarItem[rel=’today ’]",43 "settings": ["smartlist_visibility_today", "shortcut_goto_filter_today", "today_smart_list_visible_tasks"]44 },45 {46 "selector": ".filters -collection .sidebarItem[rel=’week ’]",47 "settings": ["smartlist_visibility_week", "shortcut_goto_filter_week", "today_smart_list_visible_tasks"]48 },49 {50 "selector": "#user -toolbar .activities -count, .detail -reminder.wundercon.reminder",51 "settings": ["notifications_desktop_enabled", "notifications_email_enabled",52 "notifications_push_enabled", "sound_notification_enabled","shortcut_show_notifications"53 ]54 },55 {56 "selector": ".user -avatar, .user -name, .user -arrow",57 "settings": ["shortcut_sync", "shortcut_goto_preferences"]58 },59 {60 "selector": "#search -toolbar .search -icon",61 "settings": ["shortcut_goto_search"]4862 },63 {64 "selector": ". sidebarActions -addList",65 "settings": ["shortcut_add_new_list"]66 },67 {68 "selector": ".actionBar -bottom a.tab.share",69 "settings": ["shortcut_send_via_email"]70 },71 {72 "selector": ".completed -items -heading",73 "settings": ["print_completed_items"]74 },75 {76 "selector": ".actionBar -bottom a.tab.last -tab",77 "settings": ["print_completed_items", "shortcut_copy_tasks", "shortcut_paste_tasks"]78 },79 {80 "selector": ".detail -trash",81 "settings": ["confirm_delete_entity", "shortcut_delete"]82 }]49Appendix BExperiment 1 ResourcesThis appendix contains resources used in the remote Mechanical Turk experiment,discussed in Chapter 5.1.B.1 TasksWe provide here the instructions given to participants for each of the 40 tasks.Each task has two formulations, depending on the default value of the setting.The rst formulation corresponds to changing the setting from the default valueprovided by Wunderlist; the second corresponds to changing the setting back toits default value, starting from the opposite of this default value.• Change the date format to DD.MM.YYYYChange the date format to MM/DD/YYYY• Change the time format to 12 hourChange the time format to 24 hour• Change the rst day of the week to be SundayChange the rst day of the week to be Monday• Enable sound when checking-o a todoDisable sound when checking-o a todo• Enable sound for new noticationsDisable sound for new notications• When creating a new todo, add it at the top of the current listWhen creating a new todo, add it at the bottom of the current list• When deleting a to-do, ask me for conrmationWhen deleting a to-do, don’t ask me for conrmation• When starring a to-do, move it to the top of the listDo not automatically move starred to-dos to the top of the list50• When printing a todo list, do not print the completed to-dosWhen printing a todo list, print also the completed to-dos• Show subtask progress on to-dosHide subtask progress on to-dos• Change shortcut for adding a new todo to CTRL + 0Change shortcut for adding a new todo to SHIFT + N• Change shortcut for creating a new list to CTRL + LChange shortcut for creating a new list to CTRL + K• Change shortcut for checking o todos to CTRL + DChange shortcut for checking o todos to CTRL + H• Change shortcut for starring todos to CTRL + SChange shortcut for starring todos to CTRL + M• Change shortcut for selecting all todos to CTRL + AChange shortcut for selecting all todos to CTRL + Q• Change shortcut for deleting a list or a todo to CTRL + BACKSPACEChange shortcut for deleting a list or a todo to SHIFT + BACKSPACE• Change shortcut for copying todos to CTRL + CChange shortcut for copying todos to SHIFT + M• Change shortcut for pasting todos to CTRL + VChange shortcut for pasting todos to SHIFT + P• Change shortcut for selecting the search box to CTRL + FChange shortcut for selecting the search box to CTRL + G• Change shortcut for opening preferences to CTRL + PChange shortcut for opening preferences to CTRL + .• Change shortcut for sharing a list via email to CTRL + EChange shortcut for sharing a list via email to CTRL + U• Change shortcut for showing the activities panel to CTRL + SHIFT + AChange shortcut for showing the activities panel to CTRL + SHIFT + D• Change shortcut for going to inbox to CTRL + IChange shortcut for going to inbox to SHIFT + B51• Change shortcut for showing the list of the todos assigned to you to CTRL+ 1Change shortcut for showing the list of the todos assigned to you to SHIFT+ A• Change shortcut for showing the list of the todos that are starred to CTRL+ 2Change shortcut for showing the list of the todos that are starred to SHIFT+ S• Change shortcut for showing the list of the todos due today to CTRL + 3Change shortcut for showing the list of the todos due today to SHIFT + T• Change shortcut for showing the list of the todos due this week to CTRL +4Change shortcut for showing the list of the todos due this week to SHIFT +W• Change shortcut for showing the list of all the todos to CTRL + 5Change shortcut for showing the list of all the todos to SHIFT + L• Change shortcut for showing the list of the completed todos to CTRL + 6Change shortcut for showing the list of the completed todos to SHIFT + C• Change shortcut for syncing with the server to RChange shortcut for syncing with the server to S• In the left sidebar, show the list of todos assigned to youIn the left sidebar, hide the list of todos assigned to you• In the left sidebar, show the list of todos that are due todayIn the left sidebar, hide the list of todos that are due today• In the left sidebar, hide the list of todos that are due this weekIn the left sidebar, show the list of todos that are due this week• In the left sidebar, hide the list of all the todosIn the left sidebar, show the list of all the todos• In the left sidebar, show the list of todos that are already completedIn the left sidebar, hide the list of todos that are already completed• In the lists of tasks due today and this week, show all to-dosIn the lists of tasks due today and this week, show only the todos assignedto me52• Enable email noticationsDisable email notications• Enable push noticationsDisable push notications• Enable desktop notications [duplicated]Disable desktop notications [duplicated]B.2 Participant Consent Form53Page 1 of 2        May 11, 2015       THE UNIVERSITY OF BRITISH COLUMBIA           Department of Computer Science         2366 Main Mall         Vancouver, B.C., V6T 1Z4   Consent Form Principal Investigator Dr. Joanna McGenere, Associate Professor, Department of Computer Science, University of British Columbia (###) ###-####—######@#####. Co-Investigator Antoine Ponsard, MSc student, Department of Computer Science, University of British Columbia (###) ###-####—######@#####. Summaries of the data and anonymous samples of comments obtained in this study may be included in a doctoral dissertation and in publications that arise from the research.  Study Purpose The goal of this study is to evaluate the usability of a new interface for customizing software applications, which allows users to change pre-defined settings. In particular, we are interested in comparing this new interface with the traditional settings panel found in many software applications.  Study Procedures This study is expected to require less than 45 minutes of your time. First, you will be asked to create a temporary account on Wunderlist.com, a popular todo list application, and to setup the experiment software by adding a bookmark to your web browser. Then you will be given a brief tutorial of how to use Wunderlist, and a set of tasks to complete using either Wunderlist's default settings panel or our new interface. You will be asked to fill out a questionnaire about the application, as well as some demographics information. Finally you will be asked to interact informally with a version of our prototype designed to operate with Gmail, and we will you your feedback on the prototype. The interview will be audio recorded.  Compensation We are very grateful for your participation in this study. You will receive a small honorarium ($10) for your participation  Potential Risks There are no known risks to your participation in this study.   54Page 2 of 2        May 11, 2015   Confidentiality Only the principal investigator and co-investigators will have access to the data and the audio recordings of the interviews (all data will be stored in a password-protected location). Confidentiality will be maintained by anonymizing the identity of participants.  Contact for information about the study Please contact Antoine Ponsard (email: #####@#####) if you need more information about the study. Contact for information about the rights of research subjects  If you have any concerns or complaints about your rights as a research participant and/or your experiences while participating in this study, contact the Research Participant Complaint Line in the UBC Office of Research Services at ###-###-#### or if long distance e-mail #####@##### or call toll free #-###-###-#### (Toll Free: #-###-###-####).  Consent: My participation is entirely voluntary and I may refuse to participate or withdraw from the study at any time.  My signature below indicates that I have received a copy of this consent form for my own records.   My signature indicates that I consent to participate in this study.    ____________________________________________________ Participant Signature     Date  ____________________________________________________ Printed Name of the Participant signing above  55B.3 QuestionnairesB.3.1 DemographicsMechanical Turk participants were administered an electronic version of the de-mographics questionnaire used Experiment 2 (Appendix C.4.1).B.3.2 SatisfactionAt the end of the experiment, participants were asked to rate the mechanism theywere using on three metrics: ease of use, perceived speed, and satisfaction.B.3.3 RecognitionAt the end of the experiment, participants completed two recognition question-naires: one on tab names, the other on individual settings. In both cases, half ofthe answers were made up by the authors, but plausible.565758Appendix CExperiment 2 ResourcesThis appendix contains resources used in the face-to-face lab experiment, discussedin Chapter 5.2.C.1 TasksThe same tasks were used than in Experiment 1 (Appendix B.1), but only the rstformulation—corresponding to Wunderlist’s default values.C.2 Call For ParticipationThis recruitment poster was posted in multiple locations on the UBC campus.59     $10 for testing new customization interfaces when:where:duration:60C.3 Participant Consent Form61Page 1 of 2        May 11, 2015       THE UNIVERSITY OF BRITISH COLUMBIA           Department of Computer Science         2366 Main Mall         Vancouver, B.C., V6T 1Z4   Consent Form Principal Investigator Dr. Joanna McGenere, Associate Professor, Department of Computer Science, University of British Columbia (###) ###-####—#####@#####. Co-Investigator Antoine Ponsard, MSc student, Department of Computer Science, University of British Columbia (###) ###-####—#####@#####. Summaries of the data and anonymous samples of comments obtained in this study may be included in a doctoral dissertation and in publications that arise from the research.  Study Purpose The goal of this study is to evaluate the usability of a new interface for customizing software applications, which allows users to change pre-defined settings. In particular, we are interested in comparing this new interface with the traditional settings panel found in many software applications.  Study Procedures This study is expected to require about 30 minutes of your time. First, you will be asked to create a temporary account on Wunderlist.com, a popular todo list application, and to setup the experiment software by adding a bookmark to your web browser. Then you will be given a brief tutorial on how to use Wunderlist, and a set of tasks to complete using either Wunderlist's default settings panel or our new interface. Finally you will be asked to fill out a questionnaire about the application, as well as some demographic information.  Compensation We are very grateful for your participation in this study. You will receive a small honorarium ($2.00) for your participation, plus a bonus of up to $2.90 depending on your performance.   Potential Risks There are no known risks to your participation in this study.     62Page 2 of 2        May 11, 2015 Confidentiality Only the principal investigator and co-investigators will have access to the data. Confidentiality will be maintained by anonymizing the identity of participants.  Contact for information about the study Please contact Antoine Ponsard (email: #####@#####) if you need more information about the study. Contact for information about the rights of research subjects  If you have any concerns or complaints about your rights as a research participant and/or your experiences while participating in this study, contact the Research Participant Complaint Line in the UBC Office of Research Services at ###-###-#### or if long distance e-mail #####@##### or call toll free #-###-###-#### (Toll Free: #-###-###-####).  Consent: My participation is entirely voluntary and I may refuse to participate or withdraw from the study at any time.  My signature below indicates that I have received a copy of this consent form for my own records.   My signature indicates that I consent to participate in this study.    ____________________________________________________ Participant Signature     Date  ____________________________________________________ Printed Name of the Participant signing above  63C.4 QuestionnairesC.4.1 DemographicsAfter signing the consent form, participants were asked to complete a short de-mographics questionnaire.6465C.4.2 RatingsAt the end of each condition, participants were asked to rate the mechanism theywere using on three metrics: ease of use, perceived speed, and satisfaction.C.4.3 RankingsThis questionnaire was administered at the end of the experiment, after all fourblocks of 10 trials were completed.661116768

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0220520/manifest

Comment

Related Items