UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Feature Recommender : a large-scale in-situ study of proactive software feature recommendations Ardekani, Kamyar 2016

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


24-ubc_2017_february_ardekani_kamyar.pdf [ 9.85MB ]
JSON: 24-1.0339868.json
JSON-LD: 24-1.0339868-ld.json
RDF/XML (Pretty): 24-1.0339868-rdf.xml
RDF/JSON: 24-1.0339868-rdf.json
Turtle: 24-1.0339868-turtle.txt
N-Triples: 24-1.0339868-rdf-ntriples.txt
Original Record: 24-1.0339868-source.json
Full Text

Full Text

Feature RecommenderA Large-scale In-situ Study of Proactive Software FeatureRecommendationsbyKamyar ArdekaniB.Sc. Electrical Engineering, University of Tehran, 2013A THESIS SUBMITTED IN PARTIAL FULFILLMENTOF THE REQUIREMENTS FOR THE DEGREE OFMaster of ScienceinTHE FACULTY OF GRADUATE AND POSTDOCTORALSTUDIES(Computer Science)The University of British Columbia(Vancouver)November 2016© Kamyar Ardekani, 2016AbstractIn this thesis, we describe our design of Feature Recommender, a Mozilla Fire-fox browser extension, which proactively recommends features that it predicts willbenefit users based on their individual usage behaviors. The goal of these pop-upnotifications is to help users discover new features. How to maximize the effective-ness of such notifications while minimizing user interruptions remains a difficultopen problem. One approach is to carefully time when the notifications are deliv-ered.In our deployment of Feature Recommender, we study the effect of two de-livery timing parameters: delivery rate and the user’s context at the moment ofdelivery. We also investigate the effect of prediction algorithm sensitivity. We con-ducted three field studies, each about 4 weeks: (1) A preliminary study (N=10)to determine reasonable interruptible-moments; (2) A qualitative study (N=20) toassess the design and effectiveness of our extension; and (3) A near-identical study(N= ~3K) to assess quantitatively the effect of the timing parameters. Across allconditions Feature Recommender helped users adopt on average 18% of the fea-tures they were recommended, and as many as 24% when they were delivered atrandom times with a 1-per-day delivery rate limit. We show that lack of trust inrecommendations is a key factor in hindering their effectiveness.iiPrefaceThe experiments described in this thesis were conducted with the approval of theUBC Behavioural Research Ethics Board (certificate number H15-00659).Parts of this thesis appear in a conference paper manuscript, where I am the firstauthor 1. Joanna McGrenere helped in framing and writing the manuscript, andprovided supervisory assistance for this research. Gregg Lind helped in deployingFeature Recommender through Mozilla’s Shield Studies platform for user testing.1K. Ardekani, J. McGrenere, G. Lind. Large-scale In-situ Study of Proactive Feature Recom-mendations in the Post-Clippy Era.iiiTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiAcknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Feature Recommender . . . . . . . . . . . . . . . . . . . . . . . 11.2 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.1 Software Learnability . . . . . . . . . . . . . . . . . . . . . . . . 42.2 Interruption Timing . . . . . . . . . . . . . . . . . . . . . . . . . 52.2.1 Focus on Lab Experiments . . . . . . . . . . . . . . . . . 63 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.1 Feature Recommendations . . . . . . . . . . . . . . . . . . . . . 83.2 Notification Design . . . . . . . . . . . . . . . . . . . . . . . . . 113.3 Design Revisions Based on Study 2 . . . . . . . . . . . . . . . . 13iv4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.1 Study 1: Interruptible Moments Experiment . . . . . . . . . . . . 154.1.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 164.1.2 Results from Study 1 . . . . . . . . . . . . . . . . . . . . 184.2 Study 2: Qualitative Field Experiment . . . . . . . . . . . . . . . 184.2.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.2.2 Results from Study 2 . . . . . . . . . . . . . . . . . . . . 224.3 Study 3: Quantitative Field Experiment . . . . . . . . . . . . . . 274.3.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 274.3.2 Results from Study 3 . . . . . . . . . . . . . . . . . . . . 295 Discussion and Future Work . . . . . . . . . . . . . . . . . . . . . . 396 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43A Additional Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48A.1 Study 3 Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . 48B Supporting Materials . . . . . . . . . . . . . . . . . . . . . . . . . . 52vList of TablesTable 3.1 Different types of features that are recommended to users. . . . 8Table 3.2 A subset of features included in Feature Recommender withsome of their respective components. . . . . . . . . . . . . . . 9Table 4.1 Mean rating and number of delivered notifications for each mo-ment type . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Table 4.2 Data for all 12 conditions. . . . . . . . . . . . . . . . . . . . . 30viList of FiguresFigure 3.1 A notification emanating out of the light bulb icon. . . . . . . 12Figure 3.2 Notification for the Reddit Enhancement Suite extension rec-ommendation . . . . . . . . . . . . . . . . . . . . . . . . . . 12Figure 3.3 Visual changes to the notification panel . . . . . . . . . . . . 14Figure 4.1 The pop-up notification used to collect moment ratings in Study1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Figure 4.2 The onboarding notification. . . . . . . . . . . . . . . . . . . 21Figure 4.3 Mean (a) adoption rate and (b) engagement rate for all condi-tions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Figure 4.4 (a) distribution of adoption rate and engagement rate acrossparticipants (b) adoption and delivery rate for various featurerecommendations. . . . . . . . . . . . . . . . . . . . . . . . 32Figure 4.5 Effect of MOMENT and RATE on (a) adoption rate and (b)engagement rate. . . . . . . . . . . . . . . . . . . . . . . . . 34Figure 4.6 Likert-scale question results from the final questionnaire ofStudy 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35Figure 4.7 Appropriateness of the delivery timing for the two SENSITIV-ITY levels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36Figure 4.8 Perceived frequency of the delivery timing for the two RATElevels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36Figure A.1 Results of Likert-scale questions from Study 3 for differentMOMENT types. . . . . . . . . . . . . . . . . . . . . . . . . 48viiFigure A.2 Results of Likert-scale questions from Study 3 for differentSENSITIVITY types. . . . . . . . . . . . . . . . . . . . . . . 49Figure A.3 Results of Likert-scale questions from Study 3 for differentRATE types. . . . . . . . . . . . . . . . . . . . . . . . . . . . 49Figure A.4 Results of the appropriateness of timing question from Study3 for different MOMENT types. . . . . . . . . . . . . . . . . . 50Figure A.5 Results of the appropriateness of timing question from Study3 for different RATE limits. . . . . . . . . . . . . . . . . . . . 50Figure A.6 Results of the frequency of delivery question from Study 3 fordifferent MOMENT types. . . . . . . . . . . . . . . . . . . . 50Figure A.7 Results of the frequency of delivery question from Study 3 fordifferent SENSITIVITY levels. . . . . . . . . . . . . . . . . . 51Figure B.1 Study 1 consent form (page 1 of 2) . . . . . . . . . . . . . . . 53Figure B.2 Study 1 consent form (page 2 of 2) . . . . . . . . . . . . . . . 54Figure B.3 Study 1 call for participation email . . . . . . . . . . . . . . . 55Figure B.4 Study 1 emailed instructions . . . . . . . . . . . . . . . . . . 56Figure B.5 Study 2 consent form (page 1 of 2) . . . . . . . . . . . . . . . 57Figure B.6 Study 2 consent form (page 2 of 2) . . . . . . . . . . . . . . . 58Figure B.7 Study 2 call for participation email . . . . . . . . . . . . . . . 59Figure B.8 Study 2 call for participatin poster . . . . . . . . . . . . . . . 60Figure B.9 Study 2 interim questionnaire . . . . . . . . . . . . . . . . . 61Figure B.10 Study 2 final questionnaire (page 1 of 2) . . . . . . . . . . . . 62Figure B.11 Study 2 final questionnaire (page 2 of 2) . . . . . . . . . . . . 63viiiAcknowledgmentsFirst and foremost I have to thank my supervisor Joanna McGrenere, who patientlysupported me through the ups and downs of my Master’s work.I thank my collaborators at Mozilla, especially Gregg Lind, Ilana Segall, andMatthew Grimes. I thank the Natural Sciences and Engineering Research Councilof Canada (NSERC) and the Graphics, Animation, and New Media Network ofCentres of Excellence (GRAND NCE) for funding my research.I was fortunate to work with fellow graduate students who I learned a lotfrom. In particular, I want to thank Antoine Ponsard, Francisco Escalona, Sha-ran Vaswani, and Kailun Zhang. I also thank Mona Haraty, Syavash Nobarany, andOliver Schneider for their comments and feedback on my work.I thank Thomas Fritz for agreeing to be the second reader of this thesis and thehelpful feedback he provided.My special thanks go to my brother (whose wedding I missed to finish thisthesis) and my mother for being understanding and supportive during these years.Finally, I thank my aunt and her family for making my transition to Vancouverso smooth.ixChapter 1Introduction1.1 Feature RecommenderSoftware applications in widespread use today, such as web browsers, have increas-ingly become feature-rich. While features (commands) can bring power, a knownchallenge is the user’s ability to discover "new" software features, whether they bepreviously known features that the user can no longer locate or fully remember, orunknown features that could benefit the user based on his individual usage patternsand workflows [19, 35]. Discoverability has never been more challenging than to-day – not only are there more features with every new software release, but thetrend of application extensibility through plug-ins (also known as add-ons) meansthat the number of features available is virtually limitless.A number of potential solutions to feature discoverability have appeared incommercial applications and in the research literature, but all have significant lim-itations. "Tip of the day" style pop-ups offer users information about features, butthey are not personalized to user needs, hence reducing their effectiveness. Anec-dotally, we hear that they are often just outright dismissed, disabled, or ignored.Extensions and plug-ins are located separately from the application, which makesthem more difficult to find. Command recommenders, an approach in the researchliterature, are embedded within the application itself and provide personalized rec-ommendations to the user. While command recommenders have had some successin research studies, they can also suffer from poor discoverability due to the lower1level of attentional draw of their ambient interfaces [5, 16]. Further, the underlyingcollaborative filtering algorithms don’t take full advantage of a user’s behavioralpatterns that can be mined from the rich interactions available in the interface.This information can be used to infer which features a user may benefit from.One way to address those limitations is through proactive behavior-based fea-ture recommendations, which is the approach we take in this paper. Feature rec-ommendations extend the concept of command recommendations to other types offeatures available in a software application, including plug-ins. They tackle dis-coverability by notifying the user of features that are expected to be useful. Theyneed to be very carefully designed, however, otherwise they are likely to causeannoyance, which will impede any new feature adoption. Clippy offers a classicexample of a proactive feature recommender gone wrong [2, 21, 37]. Althoughpersonalized, its primary downfall was its naive inferences and that it continuallyrecommended the same features, which caused significant annoyance. In our work,we address these limitations by providing notifications that take advantage of thepatterns of usage behavior that are semantically relevant to the usefulness of thepredicted feature, and we are also conservative in re-issuing the same recommen-dation. Recommending the undo keyboard shortcut to a user that frequently usesthe undo menu item is an example of a behavior-based recommendation.In general, notification systems of all sorts need to balance a classic tradeoffbetween the attentional draw of the notification and its intrusiveness [16]. Highlyvisible notifications (e.g., through a modal dialog box that occludes the center ofthe screen) interrupt the user and cause annoyance, while notifications with toolittle visibility (e.g. a tiny flag icon at the edge of the screen) are hard to discover.Another tradeoff relates to timing the delivery of notifications. How to best time in-terruptions so as to minimize intrusiveness remains an open problem. Researchershave learned that coarse breaks in between subtasks offer the best opportunity [24].However, to the best of our knowledge, all the user studies investigating deliverytiming have been conducted in either lab settings or in relatively short field studieswith a small number of users [3, 4, 16, 17, 23, 33], which raises the question ofhow users might experience interruptions over longer periods of time when theyare less aware of being studied.Two key timing parameters that have been little studied are the user’s context2and delivery rate. The first relates to what the user is doing at the moment of notifi-cation delivery—Is the user doing something that is directly related to the notifica-tion content? or Is the user generally at an interruptible moment in her workflow?Receiving a notification in such moments could improve the effectiveness of thenotification. The second parameter is rate of notifications. We wonder to whatextent annoyance would be reduced and the adoption of features increased whennotifications are delivered further apart versus more frequently.This thesis addresses both notification delivery timing (taking into accountthe moment of notification delivery and the rate) as well as the effectiveness ofproactive behavior-based feature recommendation. We designed and built a cus-tom Feature Recommender system as a Mozilla Firefox browser extension and weconducted three field studies: (1) A two-week preliminary study with 10 partici-pants to determine reasonable interruptible-moments; (2) A four-week study with20 participants to assess qualitatively the design and effectiveness of our FeatureRecommender; and (3) A near identical five-week study with 2941 participants toassess quantitatively the effect of context and rate of notification delivery on featureadoption and annoyance using two different algorithm prediction sensitivities.1.2 OverviewIn Chapter 2 we summarize the previous work related to this thesis. Chapter 3describes how the Feature Recommender Firefox extension works and presents abrief overview of the design of its notifications. Chapter 4 describes the threefield studies we have conducted and provides their results. In Chapter 5 we discussthe results of the three studies and what we learned from them as well as possibledirections for future work. Finally, we conclude this work in Chapter 6.3Chapter 2Related WorkThis work mainly spans two areas of human-computer interaction research: soft-ware learnability and interruption timing. The former seeks to understand howusers can learn to use a software application more effectively, and the latter in-vestigates the timing of interruptions in a software environment with the goal ofminimizing disruptive effects on the user’s primary task and maximizing the effec-tiveness of the interruption.2.1 Software LearnabilityPrior work has shown that only a small percentage of features in feature-rich soft-ware environments are used by users [22, 28]. This is partly due to users’ lack ofawareness of existing features that would be personally beneficial [19, 35]. Varioussolutions have been proposed and tried in both industry and the research commu-nity to address this challenge.Recommending software features to users has shown to be an effective method [30,31]. A number of non-personalized solutions try to proactively promote featuresthat are generally useful to users. "Tip of the day", "Did you know?", or "What’snew"-type of notifications are such examples [32]. These notifications are not tai-lored to individual user needs and are often shown to the user in a decontextualizedfashion [12], thus limiting their effectiveness.Personalized approaches, by contrast, try to take advantage of the knowledge4that the system has about the user’s task, context, and usage, to avoid overwhelmingthe user. They limit recommendations to only those features that are predicted tobe specifically beneficial to a given user.When designing personalized user interfaces that adapt to users’ need, it hasbeen shown that the accuracy of the prediction algorithm has a significant effect onuser performance and utilization of the interface [11, 15]. It has also been shownthat the effectiveness of different techniques may vary based on the accuracy of theprediction algorithm [36].Command recommenders have taken a personalized approach to provide userswith command recommendations based on their usage history and task [27, 30,31]. CommunityCommands uses an item-based collaborative filtering algorithm torecommend various commands based on command usage frequency to AutoCADusers. In a field study, it was shown to significantly increase the number of uniquecommands issued by the user. There has been some work focused on improvingthe algorithms used in such command recommendation systems [30].Our work builds on the idea of command recommenders. We extend rec-ommendation of commands to all types of interface features, including keyboardshortcuts and third-party plug-ins. Command recommenders have used ambientUIs [27, 30, 31] to avoid the problems associated with interrupting users [4, 10].This, however, leads to less attentional draw and cannot fully address the problemof feature discoverability. In our work we use proactive notifications, with carefuldesign and timing to balance intrusiveness.2.2 Interruption TimingThere is a large body of research on interruptions and their impact on the user. It isnot specifically related to feature recommendations, or even recommendations atall. We cover the most salient related work here.Researchers have shown, in general, that notifications in software applicationscan negatively affect task performance and affective state [8, 9]. This has led toresearch focused on improving the timing of interruptions. An important finding isthat the structure of a task can be leveraged for identifying moments with lower costof interruption [1, 23]. In particular, interrupting the user at a subtask boundary is5an effective approach, assuming the basic task structure is known [25]. Based onthis, Iqbal et al. developed OASIS, an intelligent notification scheduling systemthat detects three different types of breakpoints based on the users’ interactionswith their computer [24].Another factor that has been shown to be effective in interruption managementis the relevance of interruptions to the user’s task and goals [3, 17]. Arroyo et al.designed an interruption mediator system that used the relevance of the user’s goaland the task at hand to the interruption for better timing [3]. However, Gould et al.showed that content similarity is not a sufficient criterion for judging the potentialdisruptiveness of interruptions and suggest that interruption management systemscannot assume that an interruption is relevant only because its content is similar tothe user’s current activity [17].The effect of frequency (rate) of interruptions on human performance has beenstudied [6, 29]. Lee et al. showed that the higher rate of interruptions negativelyaffects task performance. They found that it had greater effect on cognitive primarytask performances than on skill primary task performances [29].Research on developing intelligent systems to predict user interruptibility ex-tends beyond capturing users’ interaction with their computers. For instance, context-aware and psycho-physiological sensors have been used to capture more informa-tive features that are indicative of a user’s interruptibility [14, 40].2.2.1 Focus on Lab ExperimentsThe vast majority of interruption studies relating to notifications, including allthose that study the effect of relevance of context and interruption rate, have beenconducted as lab experiments. The work is thus incomplete in terms of document-ing the long-term effects of interruptions. We use 2-week and 4-week field experi-ments to study the effect of interruption to help address that gap. Furthermore, ourmain measure of interest is the effectiveness of the notifications—feature adoptionbased on the recommendations—rather than the effect on task performance. It hasbeen shown that the effect of prediction techniques can vary depending on predic-tion accuracy. Therefore, in Study 3 we include prediction sensitivity, which isrelated to prediction accuracy, as one of our experimental factors.6Chapter 3DesignFeature Recommender is a Mozilla Firefox browser1 extension that observes auser’s interaction behavior and proactively recommends various Firefox featuresthat it deems likely to benefit the user. These features span a variety of cat-egories, including keyboard shortcuts, third-party plug-ins, and Firefox built-intools2. They cover relatively simple features such as a keyboard shortcut to find aword in the page, which is likely to benefit more novice users, to tools such as TileTabs that are targeted more for power users who tend to use multiple tabs simulta-neously. Table 3.1 shows these categories with examples of recommended featuresfor each category. Feature Recommender was designed with the expectation ofeventual deployment in the Firefox web browser. As such, we paid special atten-tion to the following design concepts: simplicity, extensibility, unobtrusiveness,and privacy.We describe how feature recommendations are generated and the design of thenotification pop-ups.1We chose Mozilla Firefox as our target software application because of the diversity of its userbase as well as its rich plug-in ecosystem.2Features were selected from the most successful Firefox features through consulting our Mozillacollaborators as well as reading online reviews.7Feature Type Examples Number ofFeaturesFirefox exten-sionsReddit Enhancement Suite,Zotero, Amazon Assistant17KeyboardShortcutsOpen new tab, Refresh page,Full screen11Built-in Fire-fox ToolsPin Tabs, Smart Keywords,Firefox Developer Edition3Table 3.1: Different types of features that are recommended to users.3.1 Feature RecommendationsThere are two stages for the delivery of a feature recommendation: 1. Prediction:if the user exhibits a certain behavior, the feature that is likely to better supportthat behavior is put in a recommendation queue 2. Delivery: the recommendationnotification is removed from the queue and shown to the user at the right moment,which is explained below.Feature Recommender contains a list of features that can be recommended tothe user. Each feature has three associated components (trigger, context, featureuse) which are used to determine the right moment to deliver the recommendation.Each component is described as a behavior rule.A behavior rule describes a user’s behavior such as frequent file downloading.It is an expression that matches a user’s specific action, as well as optionally thenumber of times it has been exhibited [c], and its frequency [f], and other relevantparameters. For instance, downloading a file could match "download [c>5 ANDf>1]" if it has been exhibited more than 5 times with an average frequency ofgreater than once per minute of activity. Some examples are provided in Table 3.2and explained further below.8Feature Trigger rule Context rule Feature useruleDescription Title RationaleNew Tabshortcuttabs newtab-button[c>50 AND f>0.2]tabs newtab-buttonhotkey t ctrl[c>1]You can use the Ctrl+T keyboardshortcut to open a new tab fasterNew TabshortcutYou open tabs alot.SessionManagerextensionsession restored[c>4 AND f>0.004]sessionrestoredaddon hassession-managerAutomatically saves and restores thestate of some or all of your windowsand tabsSessionManagerYou switch tabsfrequentlyPin Tab visit category webApp[c>10 AND f>0.02]visit categorywebApptabs pin [n>1] Allow you to always keep your favoritewebsites like Facebook, Gmail andTwitter open and just a click away.PinnedTabsYou use somewellknown webapps frequentlyDownThemAllExtensiondownload added[c>20 AND f>1]downloadaddedaddon hasdownThemAllDownThemAll extension lets you viewand manage your downloads easier andfaster!DownloadThem All!You downloadfiles frequently.Table 3.2: A subset of features included in Feature Recommender with some of their respective components.9Trigger rule: is a behavior rule that determines when a particular feature will beput in the recommendation queue. For example, "tabs newtab-button [c>50 ANDf>0.2]" is the trigger rule for the open new tab keyboard shortcut. It is matchedwhen the user has opened a new tab using the button more than 50 times with anaverage frequency of once per 5 minutes or higher. The frequency condition is anindicator of the usage of the button by the user and the count condition preventsfrequency inflation early on when only a small amount of time has passed.Context rule: is a behavior rule that—when in the in-context delivery condition—determines when to deliver a notification that is in the queue. The context ruleconceptually represents what the relevant context for a feature is. For example,"session restored" is the context rule for recommending the Session Manager ex-tension. This rule is matched when the user restores a Firefox session. Whenrandom or interruptible moment delivery timing conditions are used, the contextrule is ignored. This is explained in the experiment design section (Section 4.2).Feature use rule: is a behavior rule that determines when Feature Recom-mender can assume that the user already knows how to use a certain feature. Forexample, "hotkey t ctrl [c>1]" is the feature use rule for the close tab keyboardshortcut and is matched when the user uses the command+w keyboard shortcut ona OS X machine more than once. If a feature use rule is matched before a triggerrule, the system knows that the user already knows that feature, and the featurerecommendation will not be queued for delivery.Other components associated with a feature include the feature name as wellas the contents that will appear on the notification panel, which we discuss be-low. Table 3.2 shows some of the most important components for four examplefeatures.The parameter values for each rule for all the features were determined throughextensive piloting and with an overarching goal of delivering recommendationsconservatively.The list of features and their components is stored in a JSON structure and canbe extended by technical users or addon developers in the future. For instance, auser can create a set of recommendations tailored for web developers. Appendix Bcontains the full JSON structure that was used in Study 3.103.2 Notification DesignThe design of proactive notifications is multi-faceted. In this work, we did notexplore the full design space but rather made reasonable choices based on existingdesigns and considerable iterative piloting. We generally looked to balance theattentional draw of a notification and its intrusiveness [16].The design we describe here was used for our third and final study (Study 3).This design underwent an extra small round of design iteration after our secondstudy (Study 2) based on what we learned in that study.The recommendations are shown in a transient pop-up notification that em-anates out of a light bulb icon in the upper right corner of the browser (Figure 3.1)and fades out after a period of time that is dependent on the number of words in themessage3. After the notification fades, it can be re-opened by clicking the light bulbbutton. The automatic fading is disabled if either the notification panel is openedby clicking the button or if the user’s mouse pointer is inside the panel. Beingtransient reduces the intrusiveness, allowing the user to more easily ignore a notifi-cation, which is quite standard in OS X and Windows. To improve re-findability, ifthe user does not interact with the panel when it pops up, the light bulb icon stays"lit" for 5 minutes. Figure 3.2 shows elements of a notification.3It fades after 8+ 1.4×Nwords seconds. It does not fade while the mouse cursor is inside thepanel.11Figure 3.1: A notification emanating out of the light bulb icon.Figure 3.2: Notification for the Reddit Enhancement Suite extension recom-mendation: 1. close button, 2. negative feedback submission, 3. in-formation page, 4. description of the feature, 5. expandable rationale(shown expanded), 6. install/try button, 7. learn more about the feature.12The user can express dislike by clicking on "I don’t like this" (Figure 3.2.2)or access additional information about Feature Recommender with "what’s this"(Figure 3.2.3) (including how to remove the extension). To reduce clutter in thepanel, these two clickable links only appear when the user hovers on the top bar.The panel always includes a description of the feature (Figure 3.2.4) and a rationalefor why it is being recommended (Figure 3.2.5), but it must be expanded to be read.Most features can be installed/tried by clicking a button on the notification panel(Figure 3.2.6); shortcuts are an example of an exception. In the case of Firefoxplug-in, this installs the plug-in. The panel always contains a "Learn more" buttonthat directs the user to a page with more information on the recommended feature(Figure 3.2.7). For instance, in the case of keyboard shortcuts or interface features,this is the relevant Firefox support page.3.3 Design Revisions Based on Study 2Above we described the final notification design used in Study 3. Here we brieflydescribe the changes that we made based on Study 2 and a 1-week pilot study with150 participants within Mozilla employees. We do this to help contextualize forthe reader what our participants in Study 2 and our pilots experienced. Figure 3.3shows the two designs side-by-side.We made two changes in addition to those shown in Figure 3.3 to the notifica-tions:1. The transient panel stays open for a longer duration to reduce the likelihoodof missing the notification and give more time to the user to read the notifi-cation.2. The light bulb blinks for a short duration and stays lit for 5 minutes after thenotification fades, unless the user interacts with the panel. This is to helpusers discover the connection between the light bulb icon and the panel andto help them understand that they can reopen the panel even after it fades.In the design used in Study 2 the light bulb icon was switched off after thepanel automatically faded.13Figure 3.3: Visual changes to the panels include: the buttons are bigger.The primary button for extensions is now “Install it” rather than“Learnmore” to make installing extensions easier. And “why” has beenchanged to “why this” for better visibility.14Chapter 4EvaluationOur evaluation consisted of three field studies in order to maximize ecologicalvalidity. Instruments used for this study can be found in Appendix B.4.1 Study 1: Interruptible Moments ExperimentWe designed Study 1 to determine the best types of moments to deliver notificationsto the user during their use of Firefox, so as to be minimally intrusive. Theseinterruptible moments are independent of the feature that is recommended in theinterrupting notification. They are used in Studies 2 and 3.Prior work has shown that users are more receptive to an interruption betweentasks [25]. We therefore assessed 5 moment types that could be correlated withswitching tasks. After several rounds of piloting, trying many different inter-ruptible moment types, we selected the 5 most promising to compare in Study 1,namely when the user: (1) starts up Firefox; (2) opens a new tab after 10 secondsof inactivity (defined as no mouse or keyboard input); (3) becomes active after 20minutes of inactivity; (4) opens a new window; and (5) navigates to a new hostname in a page (which generally means moving to a new website).154.1.1 MethodApparatusTo compare these moment types, we designed the Woodpecker Firefox extensionthat delivers pop-up notifications at the candidate moments. The pop-up notifica-tions included a 5-point Likert-scale question asking the user to rate the disrup-tiveness of the notification (Figure 4.1). Such questions have been previously usedto capture participants’ subjective rating of interruptibility [40]. The occurrenceof each moment type varies naturally based on the type itself (e.g., startup occursinfrequently) and on each user’s usage patterns. To control the effect of rate so asnot to overwhelm a participant, we limited the maximum number of notificationsof each type to a max of 2 per day and to a max 1 per 45 minutes of activity, basedon piloting. To prevent frequent moment types from reaching their daily limit tooearly, Woodpecker randomly sampled the occurrence of each moment type in realtime to determine a reasonable estimation of moment frequencies. To do so, everytime a certain moment type occurs, the notification is delivered with a coin flipwith the probability pdelivery.pdelivery =fdesiredfoccurrenceif fdesired < foccurrence1 otherwisewhere fdesired denotes a desired frequency (we chose 1 per each 45 minutes ofactivity) and foccurrence is the average frequency of a certain moment type occurring.foccurrence and pdelivery are calculated for each moment type separately. foccurrencefor a certain moment type is updated with every new occurrence of that momenttype. If a moment type naturally occurs too often on average, this sampling methodreduces its effective occurrence frequency to fdesired1.1For example, if fdesired is set to 1min−1 and a user opens new windows with an average fre-quency of foccurrence = 2min−1, we have pdelivery = 0.5 for the open new window moment type.Thus, every time the user opens a new window, it is ignored with a 50% chance.16Figure 4.1: The pop-up notification used to collect moment ratings in Study1.ParticipantsWe recruited 10 users (7 male and 3 female, ages 19-30) through advertisementsposted around the local university; all use Firefox as their primary web browserand are regular computer users. They received $20 for participating.Procedure and TaskParticipants received Woodpecker and the study instructions via email. We toldthem that when they see the Woodpecker notification, they were to rate how dis-ruptive it would be to their workflow if a pop-up notification were to appear at thatexact time.Woodpecker used the first 2 days of the study for observing and adjusting itsestimation of moment frequencies; no notifications were delivered. This was fol-lowed by 12 days during which the participant would receive notifications. Thenumber received depended on the user’s use of Firefox. The participants keptWoodpecker on their computer for 3 additional weeks to provide more usage data;no notifications were delivered during those weeks. We used the extra usage datafor refining the design of Feature Recommender for Studies 2 and Results from Study 1A total of 350 notifications were delivered. Only 2 participants opened new Firefoxwindows during the study. Therefore, we did not include this moment type inour analysis. For each participant and moment type, we calculated the medianrating. Table 4.1 shows the mean rating and number of delivered notifications foreach moment type. We used a Friedman Test to compare the ratings of the momenttypes (χ2 = 4.24, p = 0.24). We did not find a significant difference.Startup New tab Active after 20min New hostnamemean rating 1.95 2.45 1.95 2.4mean N 7.7 9.2 8.3 9.6Table 4.1: Mean rating across participants for each moment type, as well asthe mean number of notifications delivered for each type (1 = minimallydisruptive, 5 = very disruptive).Key takeaway from Study 1Despite there being no significant differences in users’ ratings of disruptivenessacross the moment types, we chose startup Firefox and becoming active after 20minutes as the two interruptible moments to use in Studies 2 and 3, because theyhad the most favorable ratings.4.2 Study 2: Qualitative Field ExperimentThe purpose of this study was twofold: (1) to collect rich qualitative feedback onboth the design of Feature Recommender and on the effectiveness of proactivelyrecommending features based on user behavior, and (2) to prototype the experi-mental setup (with factors and conditions) and the mechanics of the apparatus inpreparation for our much larger deployment in Study 3. Given our relatively smallsample size (N=20), we did not anticipate finding significant differences amongconditions in Study 2. We call this a "Qualitative Field Experiment" to emphasizeour focus on the qualitative data collected.184.2.1 MethodApparatusThe Feature Recommender Firefox extension, as described in the design section(Chapter 3).ConditionsThis study involved two types of MOMENT: In-Context and Interruptible. In-Context moments occur when the user’s activity is contextually relevant to a featurethat is queued for recommendation as described in the design section (Section 3.1).Interruptible is when either of the two moments selected from Study 1 occur. Wemust note that these moment types are not mutually exclusive. For instance, if theuser starts up a new Firefox session and opens a new tab right away, this is consid-ered an Interruptible moment (starting up Firefox) and an In-Context moment thatis relevant to the new tab keyboard shortcut feature recommendation.Study 2 also included two levels of delivery RATE: at most either 1-per-Houror 1-per-Day. 1-per-Hour forces 1 hour of no notifications (a silent period) afterthe delivery of each notification, and 1-per-Day forces a 1-day silent period.RATE and MOMENT do not affect what and how many feature recommenda-tions are queued for delivery. They only change the time at which these recommen-dations are delivered. Thus, for example, participants in the 1-per-Hour conditionare not going to receive more notifications than they would have received had theybeen in the 1-per-Day condition. They are just likely to receive them spaced moreclosely together.ParticipantsWe recruited 20 Firefox users (11 male, 9 female; ages 19-50) through adver-tisements posted around the local university; all use Firefox as their primary webbrowser and are regular computer users. They received $50 for participating.19Procedure and TaskWe met participants in our lab to install the Feature Recommender extension ontheir Firefox and explain the procedure of the experiment. To maximize ecologicalvalidity, we told them minimal about the notifications. We told them—with onesentence in the middle of the conversation—that they might or might not receivenotifications after a few days. We did not reveal the purpose of the notifications.We did not describe or show the notifications, nor said anything about whether ornot they should interact with them.The experiment consisted of three stages. (1) Pre-observation (5 days): theuser did not receive any notifications, but their usage data was collected, and thefeature recommendations began to queue for delivery. (2) Intervention (18 days):the user could receive recommendations queued for delivery. There was also onesimple onboarding notification that all users received at the beginning of this stage.This notification told them that they would receive personalized Firefox tips (Fig-ure 4.2). It did not mention its relation to the experiment. (3) Post-observation (5days): the user stopped receiving notifications. Their usage data continued to becollected to provide an indication of their steady-state behavior post recommenda-tions.20Figure 4.2: The onboarding notification.We emailed participants approximately every 5 days to request their log file,which was locally stored on their computer. Similarly, we emailed them at 6 and12 days into the intervention stage to fill out 2 very short interim questionnaires.At the end of the experiment, we met each participant in person for a second time.Participants first filled out a short final questionnaire and we then conducted asemi-structured interviewe about their experience with Feature Recommender.MeasuresFor each participant we captured the following measures through their logs: Adop-tion is using a feature at least once after receiving its recommendation. Note thatif a user uses a feature before it is queued for delivery, Feature Recommender as-sumes that the user already knows about that feature and it will therefore not berecommended to the user, nor considered an adoption. Engagement is performingat least one of: manually opening the panel, using the try or learn more buttons,expanding the rationale section, or adopting a feature. For each participant, we alsocalculated adoption rate and engagement rate as the number of feature recommen-21dations that the participant adopted or engaged with relative to the total that theyreceived.Two short interim questionnaires asked questions about delivery timing, suchas its appropriateness and the frequency of receiving notifications, as well as an-noyance, all using Likert-scale responses. The final questionnaire was similar butincluded a question on whether the participant wanted to keep the extension, andasked for feedback on their overall experience.DesignThis study consisted of a 2-factor between-subject design: 2 MOMENT (In-Context,Interruptible) x 2 RATE (1-per-Hour, 1-per-Day)4.2.2 Results from Study 2We first briefly review the quantitative results. We checked for any impact of MO-MENT and RATE on our dependent measures using 2-way ANOVA, and as expectedfound no differences; so we describe the data collapsed across conditions. 18 par-ticipants received at least one recommendation and 11/18 adopted at least one fea-ture. One participant used private browsing exclusively and another removed theextension. A total of 100 feature recommendations were delivered2, and 24 of themwere adopted. Mean adoption rate was 24% (median 17%), and mean engagementrate was 41% (median 42%). Altogether, this gave us confidence that the loggingsystem in Feature Recommender was working well.Qualitatively the findings are much more interesting. The interviews lastedfrom 6 to 27 minutes (mean 12.25); they were audio recorded and later transcribed.We used inductive thematic analysis [7], with a focus on the timing of notificationsas well as effectiveness of the recommendations. Two researchers independentlycoded the first 8 interviews, and then discussed and iterated on the themes. Theauthor then coded the remaining interviews. Themes were refined collaborativelyby all researchers, with frequent revisits to the raw data. We report the main themesthat emerged.2We note that none of the data reported here include the one onboarding notification that all usersreceived.22Participants were minimally annoyedWe were pleasantly surprised at the low levels of annoyance expressed. Not a sin-gle participant indicated any general annoyance from the pop-ups. When probed,they verbally associated this lack of annoyance with the infrequency of notifica-tions (P3, P16), the small/proper size of the notification panel (P6), the position ofthe notification panel (P8), and the ability to manually close it easily (P1, P5, P6,P7, P12, P16). Three, however, did express annoyance from certain specific noti-fications. P12 and P21 found the first notifications "distracting" and "annoying,"but after figuring out how to close them and about how they appear and fade, theywere no longer annoyed. P11 recalled one notification that she found "disruptive":"there was one where I was like trying to get something... a million things done atonce, and I was like I really don’t need this right now. But that was just one." P11received 7 notifications in total, and only complained about that one. The ques-tionnaire data triangulated with what we heard; none of the participants had agreedthat the notifications were annoying.When asked about the frequency of notifications, no one had found them toofrequent, even in the 1-per-Hour condition. In fact, four participants expressed adesire for more frequent notifications, which indicates that, if anything, we wereperhaps even too conservative: "I think they were too separated. Like I got themvery rarely so. I would have noticed it a lot more if I were getting it once a dayor something like that" [P21]. Three participants explicitly mentioned they wouldwant to receive the notifications about once per day.Closing notifications outright is a common behaviorFive participants mentioned that they would close the notification without payingattention to its source or purpose. This was especially true early on, before theyreceived the first questionnaire, which made the participants aware that the noti-fications were related to the study. We learned that a key reason for this closingbehavior is that they mistook the notifications for ads: "To be honest, I didn’t paymuch attention to them. I thought they were ads" [P1].An interesting case was P4 who stated that she ignored pop-up notifications,and expected critical notifications to be delivered again until they are attended to:23"if it’s important enough, my computer will remind me to do it again, like the Win-dows 10 [update notifications]." Although she had received 7 notifications fromFeature Recommender, and had even closed 2 of them (the others faded after time-out), she did not recall receiving any recommendations. When asked about thishabit of dismissing notifications outright, she mentioned receiving many notifi-cations from her anti-virus and operating system about updating and so she justclosed them: "I get my computer to notify me... before it updates... so you know...I’m used to kind of just closing it, cause I’m usually too invested in what I’m doingto have time to look at it."Re-findability of notifications is difficultFive participants stated that they were interested in some of the recommended fea-tures but forgot how to use them: "I really liked that one [pinning tabs]. I for-got how to use it but I liked it" [P1]. This was especially prevalent for keyboardshortcuts. Four of them showed interest in the possibility of receiving the samenotifications again to remind them of the features: "it will be just better if you canremind me about it like – hey you can use control blah [ctrl-l]" [P6].Users have little trust in proactive notificationsTrust in the context of recommendations can be seen as the user’s pre-judgmentof the utility of a notification, before they engage with it in any way (e.g. read ortry it). A complete lack of trust renders even the most useful notifications inef-fective, as lack of trust can lead to lack of attention or negative bias towards thenotifications. Below we identify specific sources of distrust that we heard from ourusers.Poor prior experience with proactive notifications: Four participants reportedthat their prior negative experiences with proactive notifications impacted their per-ception of the feature recommendations. They reported being annoyed by notifi-cations from ads (P1, P2), other extensions (P1, P15), and other applications (P4).P1 said she paid more attention to the notifications after realizing that they werefrom the study and secure: "because I knew that they weren’t some foreign adsthat [...] like before I thought it was some pop-up that was happening and I didn’t24understand why this was coming... so when I understood that it was some secure[...] that was doing it for example... I felt more comfortable." We note that 11 ofour participants had Adblocker installed on their Firefox, which is indicative of ageneral desire to protect themselves against intrusive behaviors.Firefox: Three participants showed lack of trust in Firefox itself: "I didn’t knowit was you. I thought it was just the Firefox that did it so I got rid of it so that itdoesn’t pop up anymore" [P9]. This comment also shows that our study method-ology, the instructions in particular, can impact participants’ reaction to the notifi-cations. Had P9 known that the notifications were part of the study, he may havetreated them differently. We tried to minimize this impact with our methodology.Poor early recommendations: Three participants described how finding theearly recommendations not useful led them to not pay attention to the ones thatcame later. Users’ first exposure to recommendations impact their trust in thosethat follow; it is especially important that they be useful or users will give up onthe recommendations. P16 said: "I remember seeing the last tab stuff before [inthe study]. And I kind of tried it out. And it was like ehhh this isn’t really usefulor helpful. And I just kind of predetermined that [a feature recommended to himlater] was also not helpful or something like that".Missing rationales: P8, a savvy user, expressed lack of interest in the featuresbecause he thought the recommendations were random (not personalized), despitethe onboarding notification explicitly stating otherwise. When asked about the 6recommendations he had received, he did not recall 5 of them. The one feature hedid remember was from his first feature recommendation and he stated that he knewit already. When probed further, he told us that he would have wanted the pop-upnotification to "mention like this is relevant to you because you were using that alot." This is exactly what the rationale section of the notification included, but hehad missed it since that content was collapsed by default–it had to be expandedwith a click, which he did not do. P8’s experience provides further evidence of theimpact of a "strong" first recommendation (one that has a high probability of beingadopted).To better understand the effect of rationale, we selected the half of the partici-pants with the highest overall interaction rates, (9 participants with interaction ratesof 67% or higher), and compared those who had expanded the rationale for at least25one notification (4) to those who had not (5). (Our base was the 18 participantswho had received at least one notification.) The group with no rationale expan-sions had an average adoption rate of 19.0%, which is lower than the 27.5% of thegroup with rationale expansions. Although a very preliminary analysis, it suggeststhat rationale can have an important impact on trust and adoption, something thathas also been reported in the intelligent UI literature [18, 26, 34, 38].Interaction with notification does not indicate interestDuring the final interview, five participants expressed a clear lack of interest inlearning new features. Unsurprisingly, their log data showed that none of them hadadopted any features. We were interested to see if we could distinguish these totallyuninterested users from all our other participants just by inspection of their Firefoxusage data and interactions with the notifications. The differences we saw thatare possible indicators of interest include: the average rates of manually openingthe notifications (14% vs 21%, five uninterested vs all others); interacting with thepanel buttons (7% vs 12%); expanding the rationale (7% vs 12%); and engagementrate (31% vs 41%). In all cases, they are lower for the uninterested participants, butnotably not zero. Interestingly, however, the overall interaction with notificationrates are virtually the same (66% vs 64%). Altogether, this shows that interactiondoes not necessarily imply interest in the notification. Closing a notification is aninteraction, which we saw some users do outright. We also heard from other usersthat they did attend to the notification before closing it. Altogether, interpretingclosing behavior is not straightforward. Future work should look into the timebetween when a notification is opened and closed to find if it is an indicator of lackof interest.Key takeaways from Study 21. Our notification design and the delivery timing we tested led to notificationsthat are perceived to be low-intrusion and do not annoy users.2. Users want a way to re-find recommendations from notifications they couldnot adequately remember.263. Interpreting interaction with the notifications is not straightforward.4. Users have a strong distrust in proactive notifications, and it comes from anumber of sources.5. Our methodology was minimally invasive–some users did not realize that thenotifications were part of the study until the first questionnaire, four neverbecame aware–but thereafter it did impact the perceptions and behaviors ofsome users.We refined the design of Feature Recommender used in Study 3 based on thefeedback we received from the participants in Study 2. The design revisions arediscussed in Chapter 3. The primary motivation behind the changes was to makethe notifications harder to miss, and make installing the suggested extensions eas-ier.4.3 Study 3: Quantitative Field ExperimentThe primary goal of Study 3 was to assess the impact of different delivery mo-ments, delivery rates, and prediction sensitivity of feature recommendations onusers’ adoption of features.4.3.1 MethodApparatusWe used the same Feature Recommender extension as used in Study 2, with therelatively minor design changes discussed in the design section (Section 3.3). (Theclose launch of Study 3 after Study 2 precluded making more substantive changes.)In Study 3, Feature Recommender was referred to as "Firefox Guide".ConditionsGiven the much larger sample size, we were able to test all our planned conditions.In addition to the conditions of Study 2 we introduced a Random delivery momentto use as a baseline control condition. Random moments occur with a uniform27distribution with the same median value as the interruptible moments. We used thedata from Study 1 and Study 2 to adjust this median.We also introduced the factor of SENSITIVITY with two levels: Aggressiveand Conservative. Study 2 used Conservative only. Aggressive changed the rulethreshold to be twice as sensitive. For instance, if a participant in Conservativereceived a recommendation after clicking the new tab button more than 4 timesper minute, a participant in Aggressive would receive the same recommendationafter clicking the button only 2 or more times per minute. Hence, on average,the participants in Aggressive would receive more recommendations than those inConservative. SENSITIVITY works as a mechanism for controlling the predictionaccuracy. Aggressive on average gives more recommendations to the user and thusdecreases the accuracy.ParticipantsThrough Mozilla’s test platform, a randomly sampled population of Firefox usersreceived an invitation to participate in an experiment to try a new Firefox feature,but there was no description of the feature, specifically no mention of notificationsor feature recommendations. Those who agreed would get the Feature Recom-mender extension installed on their computer. We received data from 2941 partici-pants. There was no compensation for participating.Procedure and TaskParticipants went through the same stages of the experiment as in Study 2. Theirusage data was collected anonymously on a server. Participants could opt out ofthe study at any time by removing the installed extension.MeasuresThe measures were the same as in Study 2, except that there was no interim ques-tionnaire data, and the final questionnaire was optional. More specifically, partici-pants would be directed to the final questionnaire at the end of the experiment. Wealso added an open-ended question in Study 3, where the respondents could leaveany comments they had.28DesignThis study consisted of a 3-factor, all between-subject design: 3 MOMENT (Ran-dom, Interruptible, In-Context) x 2 RATE (1-per-Hour, 1-per-Day) x 2 SENSITIV-ITY (Conservative, Aggressive).4.3.2 Results from Study 3Quantitative Performance Measures2941 participants used the extension, and were well distributed across the 12 con-ditions (mean 135, min 122, max 155). 116 removed the extension. 51 of those didso in the Pre-observation stage. Fitting a binomial generalized linear model (GLM)on the remaining 65 participants did not show any significant effects of the factorson the ratio of participants who removed the extension.1629 participants (55%) received at least one recommendation, the other 1312(45%) did not receive any. There were various reasons for this including extensiveprivate browsing, not using Firefox actively, or just not meeting the trigger thresh-olds. Of the 1629, 830 (51%) did not adopt any recommendations—we call thisgroup the zero-adopters 3. Table 4.2 and Figure 4.3 provide different views of thedata.From here on, we only include data from those participants who received atleast one recommendation in our analysis. A total of 6583 recommendations weredelivered (mean 4.04 per person), 1143 features were adopted (mean 0.70), and3176 (mean 1.95) were engaged with.3This group received a mean of 3.25 (median 3) recommendations; for those who adopted at leastone feature it was 4.86 (median 5).29Prediction Rate In Context Interruptible RandomAggressive 1 perhour4.71 / 4,15% / 39%5.54 / 5,17% / 39%5.28 / 5,17% / 37%1 perday4.51 / 4,17% / 37%5.39 / 5,16% / 36%4.76 / 4,18% / 41%Conservative 1 perhour3.14 / 3,23% / 46%2.84 / 2,19% / 42%3.04 / 2,15% / 37%1 perday2.89 / 3,21% / 43%2.98 / 3,18% / 41%2.83 / 2,24% / 48%Table 4.2: Data for all 12 conditions for the full duration of the study. Formatis: mean / median number of recommendations, mean adoption rate /mean engagement rate.Figure 4.3: Mean (a) adoption rate and (b) engagement rate for all condi-tions. Error bars are standard errors.30ADOPTION– Aggressive prediction leads to more recommendations and higheradoption. Due to the many zero-adopters, the distribution of adopted featuresacross participants has a large number of values close to 0. A negative bino-mial GLM fit the data best and revealed a significant main effect of SENSITIV-ITY (F1,1617 = 34.42, p < 0.001), where those who received notifications more ag-gressively adopted more features (M = 0.82) than those who received them con-servatively (M = 0.57). One obvious explanation for the increased adoption inAggressive is that those participants simply received more recommendations andthus had more opportunities to adopt. This is born out in a significant correla-tion between recommendations and adoptions (Spearman’s rank correlation test,ρ = 0.36, p < 0.001).ADOPTION RATE–For those who receive a lot of recommendations, Randomat-most-once-per-day has the highest adoption rate (and is significantly better thanRandom or Interruptible at-most-once-per-hour). Adoption rate normalizes theimpact of number of recommendations received on adoption, but it turns out to betricky to analyze statistically. It has a non-normal zero-inflated distribution (Fig-ure 4.4) with a non-negligible number of 1 values (79% only received one recom-mendation). This makes the application of common statistical models challenging.As a large proportion of the zero-adopters are those who have received very fewrecommendations, we try to investigate adoption rate in the population that re-ceived more recommendations. Given that, we explore adoption rate by focusingon those participants who received a lot of recommendations, which we set to be 6or more recommendations4.46 recommendations is the 3rd quartile across all participants. There are more participants fromthe Aggressive condition in this group. To balance the numbers, we equalize the number of datapoints for each number of recommendations received (6, 7, 8...) through random sampling of theAggressive group.31Figure 4.4: (a) distribution of adoption rate and engagement rate across par-ticipants (b) adoption and delivery rate for various feature recommen-dations. Delivery rate is the proportion of participants who received arecommendation for a particular feature.The sample consists of 138 participants (half from Conservative and half fromAggressive). 3-way ANOVA on this sample reveals a significant interaction effectof RATE and MOMENT (F2,126 = 6.84, p < 0.01). Pairwise comparisons (Bonfer-32roni corrected), show that those in Random at 1-per-Day had a significantly higheradoption rate (M = 28%) than Interruptible at 1-per-Hour (M = 13%, p < 0.05)and Random at 1-per-Hour (M = 11%, p < 0.05). No other pair was significantlydifferent (Figure 4.5(a)).We can see all the features and their respective adoption rate and delivery ratein Figure 4.4(b). As it can be seen some features seem to be more popular (or possi-bly easier to adopt) than others. For instance, Session Manager ("sessionManager"on the figure, fourth from the left) is a Firefox plug-in that has been recommended(delivered) to 2.5% of participants (very short green bar), and has been adopted by28% of those participants (taller red bar). By comparison, the shortcut for clos-ing a tab ("closeTabShortcut", highest green bar) has been recommended to mostparticipants (69%), but only 10% of those have adopted it.33Figure 4.5: Effect of MOMENT and RATE on (a) adoption rate and (b) en-gagement rate. Error bars are standard errors.ENGAGEMENT–Aggressive prediction leads to higher engagement. Engage-ment is our broader measure of participant interest in the recommendations. 85%of participants engaged with at least one recommendation, but when consideringall recommendations, overall mean engagement goes down to 40%. Fitting a Pois-son GLM model on engagement reveals a significant main effect of SENSITIVITY(F1,1617 = 5.06, p < 0.05), with Aggressive (M = 2.24) having a higher number ofabsolute engagements compared to Conservative (M = 1.63).ENGAGEMENT-RATE–Aggressive prediction has lowest engagement rate, andRandom at-most-once-per-day has the highest engagement rate (significantly higher34than Random at-most-once-per-hour). Unlike adoption rate, our engagement ratemeasure is not peaked at 0 (Figure 4.4). 3-way ANOVA reveals a significant maineffect of SENSITIVITY (F1,1617 = 10.61, p < 0.01) and a significant interaction be-tween RATE and MOMENT (F2,1617 = 5.12, p < 0.01). Participants in Aggressivehave a lower engagement rate than those in Conservative (38% vs 43%). A pair-wise comparison, similar to adoption rate, shows that Random at 1-per-Day hasa higher engagement rate (M = 44%) than Random at 1-per-Hour (M = 37%).Figure 4.5(b).Quantitative Questionnaire ResultsOf the 574 completed questionnaires, 233 were from participants who had receivedat least one recommendation. We used the Aligned Rank Transform [39] to ana-lyze our Likert-scale questions and report the statistically significant findings. Fig-ure 4.6 shows the Likert-scale questions and their results collapsed across all fac-tors. Appendix A shows the results of these questions for different factor levels.Figure 4.6: Likert-scale question results from the final questionnaire of Study3.APPROPRIATENESS OF THE DELIVERY TIMING: Respondents in Conserva-tive rated the timing as more appropriate than those in Aggressive (F1,218 = 5.10, p<0.05). See Figure 4.7.35Figure 4.7: Appropriateness of the delivery timing for the two SENSITIVITYlevels.FREQUENCY OF THE DELIVERY TIMING: In terms of frequency, respondentsin the 1-per-Day felt the notification rates were too low, compared to those in the1-per-Hour (F1,218 = 9.25, p < 0.01). See Figure 4.8.Figure 4.8: Perceived frequency of the delivery timing for the two RATE lev-els.DESIRE TO KEEP FEATURE RECOMMENDER: To our surprise, respondentsin 1-per-Hour were more likely to want to keep the extension (79%) compared tothose in 1-per-Day (68%) (F1,218 = 4.32, p < 0.05). Across all respondents, 73%said they wanted to keep it. To see if there is a relation between this finding and36how frequent the notifications are perceived (the finding above), we ran a Pearson’scorrelation test, but did not find any significant correlation (r = 0.11, p > 0.087).ANNOYANCE: 21.5% of the respondents agreed (16%) or strongly agreed(5.5%) that Feature Recommender annoyed them, while 51% disagreed (33.8%)or strongly disagreed (17.3%). There was no significant impact of condition onannoyance (Figure 4.6).Qualitative Questionnaire ResultsOf the 233 questionnaire respondents who had received at least one recommenda-tion, 64 responded to the question that asked for additional comments or feedback.11 responses were irrelevant to Feature Recommender. 12 responses were genericpositive feedback such as "Keep up the good work" (R32) and "Overall good"(R61). We exclude the irrelevant and generic positive responses in our analysis.Here we report the result of an inductive thematic analysis we ran on the remaining41 comments. We used the themes found in Study 2 to code these comments andalso looked for new insightful themes. We saw themes similar to those from Study2 recurring in some of the comments from Study 3 as well.INTRUSIVENESS – Two respondents reported perceiving intrusiveness: "Af-ter the first couple [recommendations] I just clicked them off because they werecovering up tabs I was needing" [R4]. However, there were 2 other respondents(R1, R18) who reported a positive overall experience, but they warned that theywould only remain positive until the moment it becomes "invasive" (R1) or "toomuch" (R18). R1 even compared it to Clippy: "All this is [okay], of course, ’sofar’. Clippy was invasive from the first, and it just got worse, so I was expectingthe worst — If [Feature Recommender] turns bad, or my expectations change, I’lldump it in a heartbeat." R18 compares it to other annoying experiences he/she hashad with notifications: "I like it, as long as not too much. I also am now usingFirefox for everything as all the other browsers don’t allow you to stop autoplayvideos. This is the thing that is really annoying [...] The [Feature Recommender]notifications are subtle and easily ignored if busy."UNINTERESTED PARTICIPANTS – Three respondents directly expressed thatthey do not have interest in new features: "I now realize that Firefox out of the box37satisfies all my needs" [R21].REFINDABILIY PROBLEM – Three respondents commented on not being ableto find the recommendations that they might miss or forget: "Is there a way toreview past notifications that were received, in case I forget something that wassuggested?" [R27].LACK OF USEFULNESS – Four respondents said that they did not find the sug-gested features "useful" or "interesting" or they knew them already: "The idea ofrecommending extensions or settings depending on the use is fine. However mostof them I already knew and the others were not interesting to me" [R48].We found two new themes that were not apparent in Study 2. While these twothemes only emerged from the data of three respondents each, they nonethelessrepresent interesting issues worthy of consideration for future deployments.Self-identified power users do not need suggestionsThree respondents who identified themselves as experienced or "power" users re-ported that they do not need Feature Recommender: "I have been using MozillaFirefox since it was branded firebird back in the day. So, I was already yearsfamiliar with everything it popped up while trying to guide me." [R16].Privacy is a concernThree participants expressed concern about their privacy. One of them reportedhis/her high trust in Mozilla compared to other software companies, and one ex-pressed that the benefit he/she received outweighed his/her privacy concern: "Ingeneral I prefer to protect my privacy, but for Firefox Guide I feel that I receivedmore benefit. Still I would not want to be always monitored" [R57].38Chapter 5Discussion and Future WorkRandom works well enough and is simpler than in-context and interruptible, solong as the delivery rate is conservative. We put considerable effort into deter-mining how to classify in-context moments and interruptible-moments. In-contextmoments were encoded in behavioral rules that we iterated on based on pilot feed-back, and determining interruptible moments was the main purpose of Study 1. It iscurious that neither of these moments performed significantly better than random,and in fact, at the slower delivery rate (max 1 per day), random had the best per-formance (e.g., highest mean engagement rate of 44%). What to conclude? Morework could be directed towards defining more accurate in-context and interruptiblemoments. But a more pragmatic approach is just to deliver recommendations atrandom times, always ensuring a conservative rate of delivery. We note that proac-tive notifications systems today (e.g., push notifications through Chrome from appslike Facebook) seem to use random delivery times with no rate limit, which our re-sults show to be the worst (Figure 4.5).Extending the feature set and generalizing to other applications is possible.Beyond the ease of delivery timing, questions arise of how easy it would be toadd additional features in Firefox or how well such proactive notifications wouldwork in other applications. Each feature added to the feature recommendation listrequires a feature use rule and trigger rule. For features within Firefox itself, the de-velopment staff could be charged with this task. For external add-ons, their creatorswould need to supply suggested rules to Firefox, which would need to be checked39carefully. Alternatively, trusted community curators could be given this role [20].The rules should initially be tuned for conservative delivery, then monitored fortheir effectiveness (engagement and adoption rates) adjusting accordingly. Whilethis presents additional effort for someone, making progress towards resolving thelongstanding feature discoverability problem [19, 35] makes it worth the effort. Interms of other applications, for example Facebook, we speculate that users mightbe open to receiving even more proactive notifications (more frequently or moreaggressively) given that, as a social media platform, notifications are inherentlypart of what it delivers. Future work should tease this apart.Being conservative is paramount for proactive recommendation notifications.Throughout this research we have been extremely cautious to minimize interrup-tion and user annoyance. Based on extensive piloting we dialed down how manynotifications would be received. The ethos has been that if we deliver too few wemay underwhelm, but little harm will be done; by contrast, if we deliver somecombination of too many, at poor times, and with insufficient usefulness we willoverwhelm and annoy, causing users to disable Feature Recommender. Worse,users might never give such proactive feature recommendations another chance.Clippy exemplifies the outcome when caution is abandoned. The backlash was ex-treme [2, 13]. As noted earlier, one of our Study 3 users commented: "All this is[okay], of course, ’so far’. Clippy was invasive from the first, and it just got worse,so I was expecting the worst — If [Feature Recommender] turns bad, or my expec-tations change, I’ll dump it in a heartbeat." We seem to have struck a reasonablebalance, and were perhaps even slightly too conservative. (e.g., 22.3% from Study3 respondents indicated that the number of recommendations delivered was toolow). While we tested 2 levels for each of prediction sensitivity and delivery rate,more work could be done to explore the space in between each of these 2 levels.Privacy hampers feature recommendation, but must be maintained. FeatureRecommender relies on client-side user data in order to predict useful features.This is common in most intelligent user interfaces. The more data collected, themore accurate the predictions. But free collection of user data in real-world settingsis unrealistic and leads to various privacy issues. We took a pragmatic approach,based on considerable consultation with our Mozilla collaborators, in terms of theuser data accessed. For example, we do not use any data when the user is inside40a private browsing window. This necessarily diminishes the quality of the pre-dictions. If proactive feature recommenders become mainstream, users could begiven greater control, for example, to allow the use of data from private browsing.The key is that the system must be transparent about what data is used. On theupside for privacy, as opposed to methods such as collaborative filtering and item-based recommendations, our behavior-based prediction method does not requireprior data collection for training a model.Using a mix of qualitative and quantitative methods is valuable. Our two mainstudies had different foci: Study 2 on qualitative feedback, Study 3 on quantitativeimpacts. Each revealed unique results, and triangulating across the two showedsimilar findings. Methodologically, we aimed to be minimally intrusive, but Study2 showed that even a short mid-study questionnaire reduced ecologically validity.On the one hand, running a 4-week field study with 20 users is considerable effortand so we needed to ensure a reasonable volume of data. (While it would havebeen informative if only few had even noticed the notifications, that would havebeen a very expensive study for learning predominantly that one outcome). On theother hand, the questionnaire impacted some users’ engagement with the notifica-tions which diminished their data quality. For these reasons, we did not introducea questionnaire midway in Study 3. Given our large sample size, we could affordto have minimal data from zero-adopters; and even determining the percentage ofsuch users is informative (relative to those that did adopt). Study 2 suggests thatmany of the zero-adopters in Study 3–especially those who received a fair numberof recommendations–were likely just un-interested users. Detecting these usersusing their reaction to the early recommendations is an interesting topic for fu-ture work. Triangulating across the studies revealed some similar themes (e.g., thedesire for better re-findability) but some differences (e.g., higher reported annoy-ance in Study 3) which could mean that we are drawing from somewhat differentpopulations or that there was an impact of method. This needs more research.41Chapter 6ConclusionsIn this thesis we introduce Feature Recommender—a Firefox extension that proac-tively recommends software features to users based on their behavior to address thechallenge of feature discoverability. We deployed Feature Recommender in two 4-week experiments to evaluate its effectiveness and compare the effect of variousdelivery moments and rates using a mixture of quantitative and qualitative meth-ods. Across all conditions Feature Recommender helped users adopt on average18% of the features they were recommended, and as many as 24% when deliveredat random times with a 1-per-day rate limit.Through the qualitative feedback that we received, we learned that lack of trustin the usefulness of proactive notifications, as well as missing or forgetting the rec-ommendations are key factors that hinder the effectiveness of these recommenda-tions. We also found out that a portion of users simply have no interest in learningnew features.This thesis leads to various paths that can be pursued in future work. Two ofthese are (1) automatic detection of uninterested users using their reaction to thefirst few notifications and (2) analyzing the effect of our experimental factors ondifferent types of feature (e.g. keyboards shortcuts).42Bibliography[1] P. D. Adamczyk and B. P. Bailey. If not now, when?: the effects ofinterruption at different moments within task execution. In Proceedings ofthe SIGCHI conference on Human factors in computing systems, pages271–278. ACM, 2004.[2] M. G. Armentano and A. Amandi. Personalized detection of user intentions.Knowledge-Based Systems, 24(8):1169–1180, 2011.[3] E. Arroyo and T. Selker. Attention and intention goals can mediatedisruption in human-computer interaction. In IFIP Conference onHuman-Computer Interaction, pages 454–470. Springer, 2011.[4] B. P. Bailey and J. A. Konstan. On the need for attention-aware systems:Measuring effects of interruption on task performance, error rate, andaffective state. Computers in human behavior, 22(4):685–708, 2006.[5] B. P. Bailey, J. A. Konstan, and J. V. Carlis. Adjusting windows: Balancinginformation awareness with intrusion. In Proceedings of the 6th Conferenceon Human Factors and the Web, pages 39–83. Citeseer, 2000.[6] B. P. Bailey, J. A. Konstan, and J. V. Carlis. Measuring the effects ofinterruptions on task performance in the user interface. In Systems, Man,and Cybernetics, 2000 IEEE International Conference on, volume 2, pages757–762. IEEE, 2000.[7] V. Braun and V. Clarke. Using thematic analysis in psychology. Qualitativeresearch in psychology, 3(2):77–101, 2006.[8] E. B. Cutrell, M. Czerwinski, and E. Horvitz. Effects of instant messaginginterruptions on computing tasks. In CHI’00 extended abstracts on Humanfactors in computing systems, pages 99–100. ACM, 2000.43[9] M. Czerwinski, E. Cutrell, and E. Horvitz. Instant messaging: Effects ofrelevance and timing. In People and computers XIV: Proceedings of HCI,volume 2, pages 71–76, 2000.[10] E. H. Ed Cutrell, Mary Czerwinski. Notification, disruption, and memory:Effects of messaging interruptions on memory and performance. InHuman-Computer Interaction–Interact ’01, pages 263–269, January 2001.[11] L. Findlater and J. McGrenere. Impact of screen size on performance,awareness, and user satisfaction with adaptive graphical user interfaces. InProceedings of the SIGCHI Conference on Human Factors in ComputingSystems, CHI ’08, pages 1247–1256, New York, NY, USA, 2008. ACM.[12] G. Fischer. User modeling in human–computer interaction. User modelingand user-adapted interaction, 11(1-2):65–86, 2001.[13] D. Fletcher. The 50 worst inventions. Time Magazine, 27, 2010.[14] J. Fogarty, S. E. Hudson, C. G. Atkeson, D. Avrahami, J. Forlizzi, S. Kiesler,J. C. Lee, and J. Yang. Predicting human interruptibility with sensors. ACMTrans. Comput.-Hum. Interact., 12(1):119–146, Mar. 2005.[15] K. Z. Gajos, K. Everitt, D. S. Tan, M. Czerwinski, and D. S. Weld.Predictability and accuracy in adaptive user interfaces. In Proceedings of theSIGCHI Conference on Human Factors in Computing Systems, CHI ’08,pages 1271–1274, New York, NY, USA, 2008. ACM.[16] J. Gluck, A. Bunt, and J. McGrenere. Matching attentional draw with utilityin interruption. In Proceedings of the SIGCHI Conference on HumanFactors in Computing Systems, CHI ’07, pages 41–50, New York, NY, USA,2007. ACM.[17] S. J. Gould, D. P. Brumby, and A. L. Cox. What does it mean for aninterruption to be relevant? an investigation of relevance as a memory effect.In Proceedings of the Human Factors and Ergonomics Society AnnualMeeting, volume 57, pages 149–153. SAGE Publications, 2013.[18] S. Gregor and I. Benbasat. Explanations from intelligent systems:Theoretical foundations and implications for practice. MIS quarterly, pages497–530, 1999.[19] T. Grossman, G. Fitzmaurice, and R. Attar. A survey of softwarelearnability: Metrics, methodologies and guidelines. In Proceedings of the44SIGCHI Conference on Human Factors in Computing Systems, CHI ’09,pages 649–658, New York, NY, USA, 2009. ACM.[20] M. Haraty, A. Bunt, and J. McGrenere. Online customization sharingecosystems: Components, roles, and motivations. In Proceedings of the 20thACM Conference on Computer-Supported Cooperative Work & SocialComputing, page To appear. ACM, 2017.[21] E. Horvitz, J. Breese, D. Heckerman, D. Hovel, and K. Rommelse. Thelumiere project: Bayesian user modeling for inferring the goals and needs ofsoftware users. In Proceedings of the Fourteenth conference on Uncertaintyin artificial intelligence, pages 256–265. Morgan Kaufmann Publishers Inc.,1998.[22] I. Hsi and C. Potts. Studying the evolution and enhancement of softwarefeatures. In Software Maintenance, 2000. Proceedings. InternationalConference on, pages 143–151. IEEE, 2000.[23] S. T. Iqbal and B. P. Bailey. Leveraging characteristics of task structure topredict the cost of interruption. In Proceedings of the SIGCHI conference onHuman Factors in computing systems, pages 741–750. ACM, 2006.[24] S. T. Iqbal and B. P. Bailey. Effects of intelligent notification managementon users and their tasks. In Proceedings of the SIGCHI Conference onHuman Factors in Computing Systems, pages 93–102. ACM, 2008.[25] S. T. Iqbal, P. D. Adamczyk, X. S. Zheng, and B. P. Bailey. Towards anindex of opportunity: understanding changes in mental workload during taskexecution. In Proceedings of the SIGCHI conference on Human factors incomputing systems, pages 311–320. ACM, 2005.[26] A. D. Jameson. Understanding and dealing with usability side effects ofintelligent processing. AI Magazine, 30(4):23, 2009.[27] M. A. A. Khan, V. Dziubak, and A. Bunt. Exploring personalized commandrecommendations based on information found in web documentation. InProceedings of the 20th International Conference on Intelligent UserInterfaces, IUI ’15, pages 225–235, New York, NY, USA, 2015. ACM.[28] B. Lafreniere, A. Bunt, J. S. Whissell, C. L. A. Clarke, and M. Terry.Characterizing large-scale use of a direct manipulation application in thewild. In Proceedings of Graphics Interface 2010, GI ’10, pages 11–18,Toronto, Ont., Canada, Canada, 2010. Canadian Information ProcessingSociety.45[29] B. C. Lee and V. G. Duffy. The effects of task interruption on humanperformance: A study of the systematic classification of human behavior andinterruption frequency. Human Factors and Ergonomics in Manufacturing& Service Industries, 25(2):137–152, 2015.[30] J. Matejka, W. Li, T. Grossman, and G. Fitzmaurice. Communitycommands:command recommendations for software applications. In Proceedings of the22nd annual ACM symposium on User interface software and technology,pages 193–202. ACM, 2009.[31] E. Murphy-Hill, R. Jiresal, and G. C. Murphy. Improving softwaredevelopers’ fluency by recommending development environment commands.In Proceedings of the ACM SIGSOFT 20th International Symposium on theFoundations of Software Engineering, page 42. ACM, 2012.[32] D. A. Norman and S. W. Draper. User centered system design. NewPerspectives on Human-Computer Interaction, L. Erlbaum Associates Inc.,Hillsdale, NJ, 1986.[33] V. Pejovic and M. Musolesi. Interruptme: Designing intelligent promptingmechanisms for pervasive applications. In Proceedings of the 2014 ACMInternational Joint Conference on Pervasive and Ubiquitous Computing,UbiComp ’14, pages 897–908, New York, NY, USA, 2014. ACM.[34] P. Pu and L. Chen. Trust building with explanation interfaces. InProceedings of the 11th international conference on Intelligent userinterfaces, pages 93–100. ACM, 2006.[35] B. Shneiderman. Direct manipulation: a step beyond programminglanguages. Computer, 16(2):57–69, 1983.[36] T. Tsandilas and m. c. schraefel. An empirical assessment of adaptationtechniques. In CHI ’05 Extended Abstracts on Human Factors in ComputingSystems, CHI EA ’05, pages 2009–2012, New York, NY, USA, 2005. ACM.[37] B. Whitworth. Polite computing. Behaviour & Information Technology, 24(5):353–363, 2005.[38] M. Wiebe, D. Y. Geiskkovitch, and A. Bunt. Exploring user attitudestowards different approaches to command recommendation in feature-richsoftware. In Proceedings of the 21st International Conference on IntelligentUser Interfaces, pages 43–47. ACM, 2016.46[39] J. O. Wobbrock, L. Findlater, D. Gergle, and J. J. Higgins. The aligned ranktransform for nonparametric factorial analyses using only anova procedures.In Proceedings of the SIGCHI Conference on Human Factors in ComputingSystems, CHI ’11, pages 143–146, New York, NY, USA, 2011. ACM.[40] M. Züger and T. Fritz. Interruptibility of software developers and itsprediction using psycho-physiological sensors. In Proceedings of the 33rdAnnual ACM Conference on Human Factors in Computing Systems, CHI’15, pages 2981–2990, New York, NY, USA, 2015. ACM.47Appendix AAdditional FiguresThis appendix includes additional figures that are not included in the body of thethesis.A.1 Study 3 QuestionnaireFigure A.1: Results of Likert-scale questions from Study 3 for different MO-MENT types.48Figure A.2: Results of Likert-scale questions from Study 3 for different SEN-SITIVITY types.Figure A.3: Results of Likert-scale questions from Study 3 for differentRATE types.49Figure A.4: Results of the appropriateness of timing question from Study 3for different MOMENT types.Figure A.5: Results of the appropriateness of timing question from Study 3for different RATE limits.Figure A.6: Results of the frequency of delivery question from Study 3 fordifferent MOMENT types.50Figure A.7: Results of the frequency of delivery question from Study 3 fordifferent SENSITIVITY levels.51Appendix BSupporting MaterialsThis appendix primarily contains the study materials from our different studies.52Figure B.1: Study 1 consent form (page 1 of 2)53Figure B.2: Study 1 consent form (page 2 of 2)54Figure B.3: Study 1 call for participation email55Figure B.4: Study 1 emailed instructions56Figure B.5: Study 2 consent form (page 1 of 2)57Figure B.6: Study 2 consent form (page 2 of 2)58Figure B.7: Study 2 call for participation email59Figure B.8: Study 2 call for participatin poster60Figure B.9: Study 2 interim questionnaire61Figure B.10: Study 2 final questionnaire (page 1 of 2)62Figure B.11: Study 2 final questionnaire (page 2 of 2)63Features JSON File1 [2 {3 "id": "welcome",4 "priority": 0,5 "trigBehavior": "",6 "delivContext": "*",7 "featUseBehavior": "",8 "presentationData": {9 "*": {10 "message": "Here you will see awesome Firefox tips that arejust for you. Click the lightbulb icon to see your most recentrecommendation.",↪→↪→11 "rationale": "To help you get more out of Firefox.",12 "title": "Welcome to Firefox Guide!"13 },14 "doorhanger": {15 "secondaryButtonLabel": "Learn more",16 "primaryButtonLabel": "Got it!"17 }18 },19 "respCommandMap": {20 "doorhanger": {21 "secondaryButton": "info",22 "primaryButton": "pass"23 }24 },25 "tags": ""26 },27 {28 "id": "newTabShortcut",29 "priority": 1,30 "trigBehavior": "tabs newtab-button -c >50 -if <5",31 "delivContext": "tabs newtab-button",32 "featUseBehavior": "hotkey t -cmd -c >1",33 "presentationData": {34 "*": {35 "message": "You can use the <spanclass=’key’>Ctrl</span>+<span class=’key’>T</span> keyboardshortcut to open a new tab faster!",↪→↪→36 "rationale": "You open tabs a lot.",37 "title": "New Tab shortcut"38 },39 "doorhanger": {40 "secondaryButtonLabel": null,41 "primaryButtonLabel": "Learn more"42 }6443 },44 "respCommandMap": {45 "doorhanger": {46 "primaryButton": "open url -lhttps://support.mozilla.org/en-US/kb/keyboard-shortcuts-perform-firefox-tasks-quickly#w_windows-tabs"↪→↪→↪→47 }48 },49 "tags": "keyboard-shortcut"50 },51 {52 "id": "closeTabShortcut",53 "priority": 1,54 "trigBehavior": "chrome click tabbrowser-tabs anonidclose-button -c >50 -if <5",↪→55 "delivContext": "chrome click tabbrowser-tabs anonidclose-button",↪→56 "featUseBehavior": "hotkey w -cmd -c >1",57 "presentationData": {58 "*": {59 "message": "You can use the <spanclass=’key’>Ctrl</span>+<span class=’key’>W</span> keyboardshortcut to close a tab faster!",↪→↪→60 "rationale": "You close tabs a lot.",61 "title": "Close Tab shortcut"62 },63 "doorhanger": {64 "secondaryButtonLabel": null,65 "primaryButtonLabel": "Learn more"66 }67 },68 "respCommandMap": {69 "doorhanger": {70 "primaryButton": "open url -lhttps://support.mozilla.org/en-US/kb/keyboard-shortcuts-perform-firefox-tasks-quickly#w_windows-tabs"↪→↪→↪→71 }72 },73 "tags": "keyboard-shortcut"74 },75 {76 "id": "reloadShortcut",77 "priority": 2,78 "trigBehavior": "chrome click urlbar-reload-button -c >10 -if<30",↪→79 "delivContext": "chrome click urlbar-reload-button",80 "featUseBehavior": "hotkey r -cmd -c >1",6581 "presentationData": {82 "*": {83 "message": "You can use the <spanclass=’key’>Ctrl</span>+<span class=’key’>R</span> keyboardshortcut to reload the page faster!",↪→↪→84 "rationale": "You reload the page frequently.",85 "title": "Page Reload shortcut"86 },87 "doorhanger": {88 "secondaryButtonLabel": null,89 "primaryButtonLabel": "Learn more"90 }91 },92 "respCommandMap": {93 "doorhanger": {94 "primaryButton": "open url -lhttps://support.mozilla.org/en-US/kb/keyboard-shortcuts-perform-firefox-tasks-quickly#w_navigation"↪→↪→↪→95 }96 },97 "tags": "keyboard-shortcut"98 },99 {100 "id": "fullScreenShortcut",101 "priority": 2,102 "trigBehavior": "chrome fullscreen window -c >10 -if <60",103 "delivContext": "chrome fullscreen window",104 "featUseBehavior": "hotkey f11 -c >1",105 "presentationData": {106 "*": {107 "message": "You can use the <span class=’key’>F11</span>keyboard shortcut to toggle full screen view quickly!",↪→108 "rationale": "You toggle full screen view a lot.",109 "title": "Full Screen shortcut"110 },111 "doorhanger": {112 "secondaryButtonLabel": null,113 "primaryButtonLabel": "Learn more"114 }115 },116 "respCommandMap": {117 "doorhanger": {118 "primaryButton": "open url -lhttps://support.mozilla.org/en-US/kb/keyboard-shortcuts-perform-firefox-tasks-quickly#w_miscellaneous"↪→↪→↪→119 }120 },66121 "tags": "keyboard-shortcut",122 "platform": "winnt; linux"123 },124 {125 "id": "fullScreenShortcut-darwin",126 "priority": 2,127 "trigBehavior": "chrome fullscreen window -c >10 -if <60",128 "delivContext": "chrome fullscreen window",129 "featUseBehavior": "hotkey f -cmd -shift -c >1",130 "presentationData": {131 "*": {132 "message": "You can use the <spanclass=’key’>Ctrl</span>+<span class=’key’>shift</span>+<spanclass=’key’>F</span> keyboard shortcut to toggle full screenview quickly!",↪→↪→↪→133 "rationale": "You toggle full screen view a lot.",134 "title": "Full Screen shortcut"135 },136 "doorhanger": {137 "secondaryButtonLabel": null,138 "primaryButtonLabel": "Learn more"139 }140 },141 "respCommandMap": {142 "doorhanger": {143 "primaryButton": "open url -lhttps://support.mozilla.org/en-US/kb/keyboard-shortcuts-perform-firefox-tasks-quickly#w_miscellaneous"↪→↪→↪→144 }145 },146 "tags": "keyboard-shortcut",147 "platform": "darwin"148 },149 {150 "id": "findShortcut",151 "priority": 2,152 "trigBehavior": "finder opened -c >10 -if <120",153 "delivContext": "finder opened",154 "featUseBehavior": "hotkey f -cmd -c >1",155 "presentationData": {156 "*": {157 "message": "Use the <span class=’key’>Ctrl</span>+<spanclass=’key’>F</span> keyboard shortcut to quickly find a wordin the page.",↪→↪→158 "rationale": "You use the find tool a lot.",159 "title": "Find shortcut"160 },161 "doorhanger": {67162 "secondaryButtonLabel": null,163 "primaryButtonLabel": "Learn more"164 }165 },166 "respCommandMap": {167 "doorhanger": {168 "primaryButton": "open url -lhttps://support.mozilla.org/en-US/kb/keyboard-shortcuts-perform-firefox-tasks-quickly #w_search"↪→↪→169 }170 },171 "tags": "keyboard-shortcut"172 },173 {174 "id": "findNextShortcut",175 "priority": 3,176 "trigBehavior": "chrome command cmd_findAgain -c >10 -if <120",177 "delivContext": "chrome command cmd_findAgain",178 "featUseBehavior": "hotkey g -cmd -c >1",179 "presentationData": {180 "*": {181 "message": "Use the <span class=’key’>Ctrl</span>+<spanclass=’key’>G</span> keyboard shortcut to quickly find the nextoccurence of a word in the page.",↪→↪→182 "rationale": "You use the find again tool a lot.",183 "title": "Find Again shortcut"184 },185 "doorhanger": {186 "secondaryButtonLabel": null,187 "primaryButtonLabel": "Learn more"188 }189 },190 "respCommandMap": {191 "doorhanger": {192 "primaryButton": "open url -lhttps://support.mozilla.org/en-US/kb/keyboard-shortcuts-perform-firefox-tasks-quickly #w_search"↪→↪→193 }194 },195 "tags": "keyboard-shortcut"196 },197 {198 "id": "urlBarShortcut",199 "priority": 3,200 "trigBehavior": "chrome click urlbar -c >50 -if <10",201 "delivContext": "chrome click urlbar",202 "featUseBehavior": "hotkey l -cmd -c >1",203 "presentationData": {204 "*": {68205 "message": "Use the <span class=’key’>Ctrl</span>+<spanclass=’key’>L</span> keyboard shortcut to easily select andchange the current page address.",↪→↪→206 "rationale": "You frequently select the address bar.",207 "title": "Address Bar Select shortcut"208 },209 "doorhanger": {210 "secondaryButtonLabel": null,211 "primaryButtonLabel": "Learn more"212 }213 },214 "respCommandMap": {215 "doorhanger": {216 "primaryButton": "open url -lhttps://support.mozilla.org/en-US/kb/keyboard-shortcuts-perform-firefox-tasks-quickly#w_windows-tabs"↪→↪→↪→217 }218 },219 "tags": "keyboard-shortcut"220 },221 {222 "id": "addBookmarkShortcut",223 "priority": 3,224 "trigBehavior": "chrome click bookmarks-menu-button anonidbutton -c >5 -if <150",↪→225 "delivContext": "chrome click bookmarks-menu-button anonidbutton",↪→226 "featUseBehavior": "hotkey d -cmd -c >1",227 "presentationData": {228 "*": {229 "message": "You can use the <spanclass=’key’>Ctrl</span>+<span class=’key’>D</span> keyboardshortcut to quickly bookmark the current page.",↪→↪→230 "rationale": "You frequently use bookmarking.",231 "title": "Bookmark Shortcut"232 },233 "doorhanger": {234 "secondaryButtonLabel": null,235 "primaryButtonLabel": "Learn more"236 }237 },238 "respCommandMap": {239 "doorhanger": {240 "primaryButton": "open url -lhttps://support.mozilla.org/en-US/kb/keyboard-shortcuts-perform-firefox-tasks-quickly#w_bookmarks"↪→↪→↪→241 }69242 },243 "tags": "keyboard-shortcut"244 },245 {246 "id": "copyShortcut",247 "priority": 2,248 "trigBehavior": "chrome command cmd_copy -c >10 -if <30",249 "delivContext": "chrome command cmd_copy",250 "featUseBehavior": "hotkey c -cmd -c >1",251 "presentationData": {252 "*": {253 "message": "Use the <span class=’key’>Ctrl</span>+<spanclass=’key’>C</span> keyboard shortcut to easily copy aselection.",↪→↪→254 "rationale": "You copy text frequently.",255 "title": "Copy shortcut"256 },257 "doorhanger": {258 "secondaryButtonLabel": null,259 "primaryButtonLabel": "Learn more"260 }261 },262 "respCommandMap": {263 "doorhanger": {264 "primaryButton": "open url -lhttps://support.mozilla.org/en-US/kb/keyboard-shortcuts-perform-firefox-tasks-quickly #w_editing"↪→↪→265 }266 },267 "tags": "keyboard-shortcut"268 },269 {270 "id": "pasteShortcut",271 "priority": 2,272 "trigBehavior": "chrome command cmd_paste -c >10 -if <30",273 "delivContext": "chrome command cmd_paste",274 "featUseBehavior": "hotkey v -cmd -c >1",275 "presentationData": {276 "*": {277 "message": "Use the <span class=’key’>Ctrl</span>+<spanclass=’key’>V</span> keyboard shortcut to paste a copiedelement.",↪→↪→278 "rationale": "You use copy-paste frequently.",279 "title": "Paste shortcut"280 },281 "doorhanger": {282 "secondaryButtonLabel": null,283 "primaryButtonLabel": "Learn more"284 }70285 },286 "respCommandMap": {287 "doorhanger": {288 "primaryButton": "open url -lhttps://support.mozilla.org/en-US/kb/keyboard-shortcuts-perform-firefox-tasks-quickly #w_editing"↪→↪→289 }290 },291 "tags": "keyboard-shortcut"292 },293 {294 "id": "lastTabShortcut",295 "priority": 5,296 "trigBehavior": "tabs switched position last -c >20 -if <10",297 "delivContext": "tabs switched position last",298 "featUseBehavior": "hotkey 9 -cmd -c >1",299 "presentationData": {300 "*": {301 "message": "Use the <span class=’key’>Ctrl</span>+<spanclass=’key’>9</span> keyboard shortcut to switch to the lastopen tab faster.",↪→↪→302 "rationale": "You frequently switch to the rightmost tabback and forth.",↪→303 "title": "Select Last Tab shortcut"304 },305 "doorhanger": {306 "secondaryButtonLabel": null,307 "primaryButtonLabel": "Learn more"308 }309 },310 "respCommandMap": {311 "doorhanger": {312 "primaryButton": "open url -lhttps://support.mozilla.org/en-US/kb/keyboard-shortcuts-perform-firefox-tasks-quickly#w_windows-tabs"↪→↪→↪→313 }314 },315 "tags": "keyboard-shortcut",316 "platform": "winnt; darwin"317 },318 {319 "id": "lastTabShortcut-linux",320 "priority": 5,321 "trigBehavior": "tabs switched position last -c >20 -if <10",322 "delivContext": "tabs switched position last",323 "featUseBehavior": "hotkey 9 -alt -c >1",324 "presentationData": {325 "*": {71326 "message": "Use the <span class=’key’>Alt</span>+<spanclass=’key’>9</span> keyboard shortcut to switch to the lastopen tab faster.",↪→↪→327 "rationale": "You frequently switch to the rightmost tabback and forth.",↪→328 "title": "Select Last Tab shortcut"329 },330 "doorhanger": {331 "secondaryButtonLabel": null,332 "primaryButtonLabel": "Learn more"333 }334 },335 "respCommandMap": {336 "doorhanger": {337 "primaryButton": "open url -lhttps://support.mozilla.org/en-US/kb/keyboard-shortcuts-perform-firefox-tasks-quickly#w_windows-tabs"↪→↪→↪→338 }339 },340 "tags": "keyboard-shortcut",341 "platform": "linux"342 },343 {344 "id": "downThemAll",345 "priority": 2,346 "trigBehavior": "download added -c >20 -if <60",347 "delivContext": "download added",348 "featUseBehavior": "addon has{DDC359D1-844A-42a7-9AA1-88A850A938A8}",↪→349 "presentationData": {350 "*": {351 "message": "The DownThemAll extension lets you view andmanage your downloads easier and faster!",↪→352 "rationale": "You download files frequently.",353 "title": "Download Them All!"354 },355 "doorhanger": {356 "secondaryButtonLabel": "Learn more",357 "primaryButtonLabel": "Install it"358 }359 },360 "respCommandMap": {361 "doorhanger": {362 "secondaryButton": "open url -l https://addons.mozilla.org/en-US/firefox/addon/ downthemall/?src=fr-dh",↪→363 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-US/firefox/addon/downthemall/?src=fr-dh-btn"↪→↪→72364 }365 },366 "tags": "extension"367 },368 {369 "id": "turnOffTheLights",370 "priority": 3,371 "trigBehavior": "visit hostname www.youtube.com -c >20 -if<30",↪→372 "delivContext": "visit hostname www.youtube.com",373 "featUseBehavior": "addon has stefanvandamme@stefanvd.net",374 "presentationData": {375 "*": {376 "message": "The Turn Off the Lights extension lets youwatch videos as if you were in the cinema.",↪→377 "rationale": "You like watching videos in Firefox.",378 "title": "Turn Off the Lights and enjoy!"379 },380 "doorhanger": {381 "secondaryButtonLabel": "Learn more",382 "primaryButtonLabel": "Install it"383 }384 },385 "respCommandMap": {386 "doorhanger": {387 "secondaryButton": "open url -l https://addons.mozilla.org/en-US/firefox/addon/ turn-off-the-lights/?src=fr-dh",↪→388 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-US/firefox/addon/turn-off-the-lights/?src=fr-dh-btn"↪→↪→389 }390 },391 "tags": "extension"392 },393 {394 "id": "redditEnhancementSuite",395 "priority": 1,396 "trigBehavior": "visit hostname www.reddit.com -c >5 -if <180",397 "delivContext": "visit hostname www.reddit.com",398 "featUseBehavior": "addon has jid1-xUfzOsOFlzSOXg@jetpack",399 "presentationData": {400 "*": {401 "message": "The Reddit Enhancement Suite extension enhancesyour reddit browsing experience.",↪→402 "rationale": "You frequently visit reddit.",403 "title": "Enhance your reddit experience!"404 },405 "doorhanger": {406 "secondaryButtonLabel": "Learn more",73407 "primaryButtonLabel": "Install it"408 }409 },410 "respCommandMap": {411 "doorhanger": {412 "secondaryButton": "open url -l https://addons.mozilla.org/en-us/firefox/addon/ reddit-enhancement-suite/?src=fr-dh",↪→413 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-us/firefox/addon/reddit-enhancement-suite/?src=fr-dh-btn"↪→↪→414 }415 },416 "tags": "extension"417 },418 {419 "id": "amazonAssistant",420 "priority": 3,421 "trigBehavior": "visit hostname www.amazon.com -c >10 -if <45",422 "delivContext": "visit hostname www.amazon.com",423 "featUseBehavior": "addon has abb@amazon.com",424 "presentationData": {425 "*": {426 "message": "Amazon Assistant helps you make better shoppingdecisions by showing product comparisons at thousands of retailsites.",↪→↪→427 "rationale": "You visit Amazon frequently.",428 "title": "Amazon Assistant"429 },430 "doorhanger": {431 "secondaryButtonLabel": "Learn more",432 "primaryButtonLabel": "Install it"433 }434 },435 "respCommandMap": {436 "doorhanger": {437 "secondaryButton": "open url -l https://addons.mozilla.org/en-US/firefox/addon/ amazon-browser-bar/?src=fr-dh",↪→438 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-US/firefox/addon/amazon-browser-bar/?src=fr-dh-btn"↪→↪→439 }440 },441 "tags": "extension"442 },443 {444 "id": "pinterestPinBtn",445 "priority": 3,446 "trigBehavior": "visit hostname www.pinterest.com -c >4 -if<120",↪→74447 "delivContext": "visit hostname www.pinterest.com",448 "featUseBehavior": "addon has{677a8f98-fd64-40b0-a883-b8c95d0cbf17}",↪→449 "presentationData": {450 "*": {451 "message": "Lets you pin images and upload your Pins toPinterest quickly and easily with a special toolbar button.",↪→452 "rationale": "You visit Pinterest frequently.",453 "title": "Pinterest Pin Button"454 },455 "doorhanger": {456 "secondaryButtonLabel": "Learn more",457 "primaryButtonLabel": "Install it"458 }459 },460 "respCommandMap": {461 "doorhanger": {462 "secondaryButton": "open url -l https://addons.mozilla.org/en-us/firefox/addon/ pinterest-pin-button/?src=fr-dh",↪→463 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-us/firefox/addon/pinterest-pin-button/?src=fr-dh-btn"↪→↪→464 }465 },466 "tags": "extension"467 },468 {469 "id": "imdbSearch",470 "priority": 3,471 "trigBehavior": "visit hostname www.imdb.com -c >4 -if <120",472 "delivContext": "visit hostname www.imdb.com",473 "featUseBehavior": "addon has{c4080853-c699-4120-b8e0-618bff8a4474",↪→474 "presentationData": {475 "*": {476 "message": "Lets you search IMDB by selecting some text andusing a right-click.",↪→477 "rationale": "You visit IMDB frequently.",478 "title": "IMDB Search"479 },480 "doorhanger": {481 "secondaryButtonLabel": "Learn more",482 "primaryButtonLabel": "Install it"483 }484 },485 "respCommandMap": {486 "doorhanger": {487 "secondaryButton": "open url -l https://addons.mozilla.org/en-US/firefox/addon/ imdb-search-1/?src=fr-dh",↪→75488 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-US/firefox/addon/imdb-search-1/?src=fr-dh-btn"↪→↪→489 }490 },491 "tags": "extension"492 },493 {494 "id": "sessionManager",495 "priority": 2,496 "trigBehavior": "session restored -c >4 -if <240",497 "delivContext": "session restored",498 "featUseBehavior": "addon has{1280606b-2510-4fe0-97ef-9b5a22eafe30}",↪→499 "presentationData": {500 "*": {501 "message": "Automatically saves and restores the state ofsome or all of your windows and tabs.",↪→502 "rationale": "You restore your Firefox sessionfrequently.",↪→503 "title": "Session Manager"504 },505 "doorhanger": {506 "secondaryButtonLabel": "Learn more",507 "primaryButtonLabel": "Install it"508 }509 },510 "respCommandMap": {511 "doorhanger": {512 "secondaryButton": "open url -l https://addons.mozilla.org/en-US/firefox/addon/ session-manager/?src=fr-dh",↪→513 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-US/firefox/addon/session-manager/?src=fr-dh-btn"↪→↪→514 }515 },516 "tags": "extension"517 },518 {519 "id": "integratedGoogleCal",520 "priority": 3,521 "trigBehavior": "visit webapp googleCal -c >5 -if <60",522 "delivContext": "visit webapp googleCal",523 "featUseBehavior": "addon has intgcal@egarracingteam.com.ar",524 "presentationData": {525 "*": {526 "message": "Integrate Google Calendar with Firefox! Noextra login needed.",↪→527 "rationale": "You visit Google Calendar a lot.",76528 "title": "Integerated Google Calendar"529 },530 "doorhanger": {531 "secondaryButtonLabel": "Learn more",532 "primaryButtonLabel": "Install it"533 }534 },535 "respCommandMap": {536 "doorhanger": {537 "secondaryButton": "open url -l https://addons.mozilla.org/en-US/firefox/addon/ integrated-google-calendar/?src=fr-dh",↪→538 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-US/firefox/addon/integrated-google-calendar/?src=fr-dh-btn"↪→↪→539 }540 },541 "tags": "extension"542 },543 {544 "id": "treeStyleTab",545 "priority": 3,546 "trigBehavior": "tabs open -n >30",547 "delivContext": "tabs opened",548 "featUseBehavior": "addon has treestyletab@piro.sakura.ne.jp",549 "presentationData": {550 "*": {551 "message": "Helps you manage and view your tabs more easilyby showing tabs like a tree.",↪→552 "rationale": "You keep many tabs open.",553 "title": "Tree Style Tab"554 },555 "doorhanger": {556 "secondaryButtonLabel": "Learn more",557 "primaryButtonLabel": "Install it"558 }559 },560 "respCommandMap": {561 "doorhanger": {562 "secondaryButton": "open url -l https://addons.mozilla.org/en-US/firefox/addon/ tree-style-tab/?src=fr-dh",↪→563 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-US/firefox/addon/tree-style-tab/?src=fr-dh-btn"↪→↪→564 }565 },566 "tags": "extension"567 },568 {569 "id": "forecastfox",77570 "priority": 4,571 "trigBehavior": "visit category weather -c >5 -if <90",572 "delivContext": "visit category weather",573 "featUseBehavior": "addon has forecastfox@s3_fix_version",574 "presentationData": {575 "*": {576 "message": "Get weather forecasts from AccuWeather.com inyour Firefox in an unobtrusive way!",↪→577 "rationale": "You visit known weather forecast websitesfrequently.",↪→578 "title": "Forecastfox"579 },580 "doorhanger": {581 "secondaryButtonLabel": "Learn more",582 "primaryButtonLabel": "Install it"583 }584 },585 "respCommandMap": {586 "doorhanger": {587 "secondaryButton": "open url -l https://addons.mozilla.org/en-US/firefox/addon/ forecastfox-fix-version/?src=fr-dh",↪→588 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-US/firefox/addon/forecastfox-fix-version/?src=fr-dh-btn"↪→↪→589 }590 },591 "tags": "extension"592 },593 {594 "id": "cleanestAddonManager",595 "priority": 4,596 "trigBehavior": "addon count extension -n >15",597 "delivContext": "addon pageshow",598 "featUseBehavior": "addon has cam@sdrocking.com",599 "presentationData": {600 "*": {601 "message": "Clean up your Addon Manager with thisextension.",↪→602 "rationale": "You have many Firefox add-ons.",603 "title": "Cleanest Addon Manager"604 },605 "doorhanger": {606 "secondaryButtonLabel": "Learn more",607 "primaryButtonLabel": "Install it"608 }609 },610 "respCommandMap": {611 "doorhanger": {78612 "secondaryButton": "open url -l https://addons.mozilla.org/en-US/firefox/addon/ cam/?src=fr-dh",↪→613 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-US/firefox/addon/cam/?src=fr-dh-btn"↪→↪→614 }615 },616 "tags": "extension"617 },618 {619 "id": "privateTab",620 "priority": 3,621 "trigBehavior": "private browse open -c >10 -if <30",622 "delivContext": "private browse open",623 "featUseBehavior": "addon has privateTab@infocatcher",624 "presentationData": {625 "*": {626 "message": "You find private windows useful? This extensionlets you open tabs that are private.",↪→627 "rationale": "You use private windows.",628 "title": "Private Tab"629 },630 "doorhanger": {631 "secondaryButtonLabel": "Learn more",632 "primaryButtonLabel": "Install it"633 }634 },635 "respCommandMap": {636 "doorhanger": {637 "secondaryButton": "open url -l https://addons.mozilla.org/en-US/firefox/addon/ private-tab/?src=fr-dh",↪→638 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-US/firefox/addon/private-tab/?src=fr-dh-btn"↪→↪→639 }640 },641 "tags": "privacy-invasive; extension"642 },643 {644 "id": "tileTabs",645 "priority": 5,646 "trigBehavior": "tabs switched -c >75 -f >4",647 "delivContext": "tabs switched",648 "featUseBehavior": "addon has tiletabs@DW-dev",649 "presentationData": {650 "*": {651 "message": "Lets you display tabs in a tiled layout inFirefox.",↪→652 "rationale": "You switch tabs frequently.",79653 "title": "Tile Tabs"654 },655 "doorhanger": {656 "secondaryButtonLabel": "Learn more",657 "primaryButtonLabel": "Install it"658 }659 },660 "respCommandMap": {661 "doorhanger": {662 "secondaryButton": "open url -l https://addons.mozilla.org/en-US/firefox/addon/ tile-tabs/?src=fr-dh",↪→663 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-US/firefox/addon/tile-tabs/?src=fr-dh-btn"↪→↪→664 }665 },666 "tags": "extension"667 },668 {669 "id": "stylish",670 "priority": 2,671 "trigBehavior": "addon count theme -n >4",672 "delivContext": "addon pageshow",673 "featUseBehavior": "addon has{46551EC9-40F0-4e47-8E18-8E5CF550CFB8}",↪→674 "presentationData": {675 "*": {676 "message": "Stylish lets you restyle Firefox and manypopular websites like Google, Facebook, etc. with themes andskins.",↪→↪→677 "rationale": "You use Firefox themes.",678 "title": "Get Stylish!"679 },680 "doorhanger": {681 "secondaryButtonLabel": "Learn more",682 "primaryButtonLabel": "Install it"683 }684 },685 "respCommandMap": {686 "doorhanger": {687 "secondaryButton": "open url -l https://addons.mozilla.org/en-US/firefox/addon/ stylish/?src=fr-dh",↪→688 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-US/firefox/addon/stylish/?src=fr-dh-btn"↪→↪→689 }690 },691 "tags": "extension"692 },80693 {694 "id": "zotero",695 "priority": 3,696 "trigBehavior": "visit category academic -c >10 -if <30",697 "delivContext": "visit category academic",698 "featUseBehavior": "addon has zotero@chnm.gmu.edu",699 "presentationData": {700 "*": {701 "message": "Capture and organize your academic research allin your Firefox.",↪→702 "rationale": "You visit well-known academic websitesfrequently.",↪→703 "title": "Zotero"704 },705 "doorhanger": {706 "secondaryButtonLabel": "Learn more",707 "primaryButtonLabel": "Install it"708 }709 },710 "respCommandMap": {711 "doorhanger": {712 "secondaryButton": "open url -l https://www.zotero.org/",713 "primaryButton": "open url -lhttps://www.zotero.org/download/"↪→714 }715 },716 "tags": "extension"717 },718 {719 "id": "profilist",720 "priority": 2,721 "trigBehavior": "firefox profiles -n >4",722 "delivContext": "firefox startup",723 "featUseBehavior": "addon has Profilist@jetpack",724 "presentationData": {725 "*": {726 "message": "Switch profiles easily and smoothly right fromyour main menu.",↪→727 "rationale": "You have multiple Firefox profiles.",728 "title": "Profilist"729 },730 "doorhanger": {731 "secondaryButtonLabel": "Learn more",732 "primaryButtonLabel": "Install it"733 }734 },735 "respCommandMap": {736 "doorhanger": {81737 "secondaryButton": "open url -l https://addons.mozilla.org/en-US/firefox/addon/ profilist/?src=fr-dh",↪→738 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-US/firefox/addon/profilist/?src=fr-dh-btn"↪→↪→739 }740 },741 "tags": "extension"742 },743 {744 "id": "tabGroups",745 "priority": 4,746 "trigBehavior": "tabs open -n >60",747 "delivContext": "tabs open -n >30",748 "featUseBehavior": "addon has tabgroups@quicksaver",749 "presentationData": {750 "*": {751 "message": "Are an easy way to organize a lot of tabs. Youcan visually group related tabs, switch between groups, andquickly search through all of your tabs to switch to a specificone.",↪→↪→↪→752 "rationale": "You tend to have many tabs open.",753 "title": "Tab Groups"754 },755 "doorhanger": {756 "secondaryButtonLabel": "Learn more",757 "primaryButtonLabel": "Install it"758 }759 },760 "respCommandMap": {761 "doorhanger": {762 "secondaryButton": "open url -l https://addons.mozilla.org/en-US/firefox/addon/ tab-groups-panorama/?src=fr-dh",↪→763 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-US/firefox/addon/tab-groups-panorama/?src=fr-dh-btn"↪→↪→764 }765 },766 "tags": "extension"767 },768 {769 "id": "s3GoogleTranslator",770 "priority": 3,771 "trigBehavior": "visit category translation -c >10 -if <60",772 "delivContext": "visit category translation",773 "featUseBehavior": "addon has s3google@translator",774 "presentationData": {775 "*": {82776 "message": "Add the power of Google Translate’s automatictranslation to your browser!",↪→777 "rationale": "You use popular translation web apps.",778 "title": "S3.Google Translator"779 },780 "doorhanger": {781 "secondaryButtonLabel": "Learn more",782 "primaryButtonLabel": "Install it"783 }784 },785 "respCommandMap": {786 "doorhanger": {787 "secondaryButton": "open url -l https://addons.mozilla.org/en-US/firefox/addon/ s3google-translator/?src=fr-dh",↪→788 "primaryButton": "amo install -lhttps://addons.mozilla.org/ en-US/firefox/addon/s3google-translator/?src=fr-dh-btn"↪→↪→789 }790 },791 "tags": "extension"792 },793 {794 "id": "firefoxDeveloper",795 "priority": 1,796 "trigBehavior": "dev select -c >30 -if <60",797 "delivContext": "dev toolbox-ready",798 "featUseBehavior": "firefox channel aurora",799 "presentationData": {800 "*": {801 "message": "Brings your core dev tools together with somepowerful new ones that will extend your ability to work acrossmultiple platforms from one place.",↪→↪→802 "rationale": "You use Firefox developer tools frequently.",803 "title": "Firefox Developer Edition"804 },805 "doorhanger": {806 "secondaryButtonLabel": "Download",807 "primaryButtonLabel": "Learn more"808 }809 },810 "respCommandMap": {811 "doorhanger": {812 "secondaryButton": "open url -l https://www.mozilla.org/en-US/firefox/developer/?src=fr-dh-btn",↪→813 "primaryButton": "open url -l https://www.mozilla.org/en-US/firefox/developer/?src=fr-dh"↪→814 }815 },816 "tags": "download"83817 },818 {819 "id": "privateBrowsing",820 "priority": 1,821 "trigBehavior": "history delete -c >10 -if <90",822 "delivContext": "history delete",823 "featUseBehavior": "private browse open -c >1",824 "presentationData": {825 "*": {826 "message": "Private Browsing allows you to browse theInternet without saving any information about which sites andpages youve visited.",↪→↪→827 "rationale": "You edit your browsing history frequently.",828 "title": "Private Browsing"829 },830 "doorhanger": {831 "secondaryButtonLabel": "Learn more",832 "primaryButtonLabel": "Try It"833 }834 },835 "respCommandMap": {836 "doorhanger": {837 "secondaryButton": "open url -lhttps://support.mozilla.org/en-US/kb/private-browsing-use-firefox-without-history",↪→↪→838 "primaryButton": "open url -l about:privatebrowsing -window-private"↪→839 }840 },841 "tags": "privacy-invasive; tool"842 },843 {844 "id": "pinTab",845 "priority": 3,846 "trigBehavior": "visit category webApp -c >10 -if <45",847 "delivContext": "visit category webApp",848 "featUseBehavior": "tabs pin -n >1",849 "presentationData": {850 "*": {851 "message": "Allow you to always keep your favorite websiteslike Facebook, Gmail and Twitter open and just a click away.",↪→852 "rationale": "You use some well-known web appsfrequently.",↪→853 "title": "Pinned Tabs"854 },855 "doorhanger": {856 "secondaryButtonLabel": "Learn more",857 "primaryButtonLabel": "Try It"858 }84859 },860 "respCommandMap": {861 "doorhanger": {862 "secondaryButton": "open url -lhttps://support.mozilla.org/en-US/kb/pinned-tabs-keep-favorite-websites-open",↪→↪→863 "primaryButton": "pin tab"864 }865 },866 "tags": "tool"867 },868 {869 "id": "smartKeywords",870 "priority": 4,871 "trigBehavior": "visit category specializedSearch -c >20 -if<45",↪→872 "delivContext": "visit category specializedSearch",873 "featUseBehavior": "search-engine alias -n >0",874 "presentationData": {875 "*": {876 "message": "Smart Keywords are an easy way to searchspecific websites directly from the Firefox Location bar.",↪→877 "rationale": "You visit known websites such as Wikipediathat have specialized search engines.",↪→878 "title": "Search Smarter!"879 },880 "doorhanger": {881 "secondaryButtonLabel": null,882 "primaryButtonLabel": "Learn more"883 }884 },885 "respCommandMap": {886 "doorhanger": {887 "primaryButton": "open url -l https://support.mozilla.org/en-US/kb/how-search-from-address-bar"↪→888 }889 },890 "tags": "tool"891 }892 ]85


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items