Science, Faculty of
Computer Science, Department of
DSpace
UBCV
Janzen, Izabelle Foster
Doctor of Philosophy - PhD
University of British Columbia
Mobile devices deliver ‘notifications’ that interrupt the user for events like receiving text messages and emails. However, frequent interruptions reduce productivity and can increase stress. Notification management tools can reduce this distraction but substantial barriers, such as time and effort, prevent most users from managing their notifications. In this dissertation, I present three projects that aim to understand how to better support proactive notification management and I systematically design a novel notification management tool to better address user needs. We used a qualitative, user-centered approach consisting of interviews (N=71) and surveys (N=140). First, we explored how the notification management needs of older adults compared to the general population because other forms of distraction disproportionately impact older adults but their notification management needs had not been studied. Through semi-structured interviews (N=20) with both younger and older adults, we identified that many users desire a sense of control and agency over their attention that was difficult to achieve with current notification management technology. Second, we characterized how users could proactively control notifications through four design axes. We created five notification management design concepts that included design elements from under-explored areas of our design axes. Through semi-structured interviews (N=30), we probed user reactions to our design concepts to identify what design elements help address user concerns, such as a lack of control in notification management. The results of this study suggested there was strong interest to include reflective design elements within notification management. Lastly, we leveraged these results and a supplementary survey of users’ notification preferences (N=140) to design a novel notification management tool. The Reflective Spring Cleaning design supported infrequent notification management with visualizations of notification usage data and suggestions for personalization. We evaluated this design through a longitudinal study organized around semi-structured interviews (N=21). Our results demonstrated how the concept encouraged and supported users to proactively manage notifications through critical reflection on the impact of notifications. Our work outlines key design directions for how notification management tools can support proactive notification management and provide users a sense of control over their attention.
Design Opportunities for Personalized ProactiveNotification ManagementbyIzabelle Foster JanzenMSc., University of British Columbia, 2016B.A.Sc., McMaster University, 2014A THESIS SUBMITTED IN PARTIAL FULFILLMENTOF THE REQUIREMENTS FOR THE DEGREE OFDoctor of PhilosophyinTHE FACULTY OF GRADUATE AND POSTDOCTORALSTUDIES(Computer Science)The University of British Columbia(Vancouver)December 2022© Izabelle Foster Janzen, 2022The following individuals certify that they have read, and recommend to the Fac-ulty of Graduate and Postdoctoral Studies for acceptance, the thesis entitled:Design Opportunities for Personalized Proactive Notification Manage-mentsubmitted by Izabelle Foster Janzen in partial fulfillment of the requirements forthe degree of Doctor of Philosophy in Computer Science.Examining Committee:Joanna McGrenere, Professor, Computer Science, UBCSupervisorTamara Munzner, Professor, Computer Science, UBCSupervisory Committee MemberThomas Fritz, Associate Professor, Informatics, University of ZurichSupervisory Committee MemberRachel Pottinger, Professor, Computer Science, UBCUniversity ExaminerElizabeth W Dunn, Professor, Psychology, UBCUniversity ExaminerRita Orji, Associate Professor, Computer Science, Dalhousie UniversityExternal ExamineriiAbstractMobile devices deliver ‘notifications’ that interrupt the user for events like receiv-ing text messages and emails. However, frequent interruptions reduce productivityand can increase stress. Notification management tools can reduce this distractionbut substantial barriers, such as time and effort, prevent most users from managingtheir notifications.In this dissertation, I present three projects that aim to understand how to bet-ter support proactive notification management and I systematically design a novelnotification management tool to better address user needs. We used a qualitative,user-centered approach consisting of interviews (N=71) and surveys (N=140).First, we explored how the notification management needs of older adults com-pared to the general population because other forms of distraction disproportion-ately impact older adults but their notification management needs had not beenstudied. Through semi-structured interviews (N=20) with both younger and olderadults, we identified that many users desire a sense of control and agency overtheir attention that was difficult to achieve with current notification managementtechnology.Second, we characterized how users could proactively control notificationsthrough four design axes. We created five notification management design con-cepts that included design elements from under-explored areas of our design axes.Through semi-structured interviews (N=30), we probed user reactions to our de-sign concepts to identify what design elements help address user concerns, such asa lack of control in notification management. The results of this study suggestedthere was strong interest to include reflective design elements within notificationmanagement.iiiLastly, we leveraged these results and a supplementary survey of users’ no-tification preferences (N=140) to design a novel notification management tool.The Reflective Spring Cleaning design supported infrequent notification manage-ment with visualizations of notification usage data and suggestions for personal-ization. We evaluated this design through a longitudinal study organized aroundsemi-structured interviews (N=21). Our results demonstrated how the concept en-couraged and supported users to proactively manage notifications through criticalreflection on the impact of notifications.Our work outlines key design directions for how notification management toolscan support proactive notification management and provide users a sense of controlover their attention.ivLay SummaryMobile devices cause users stress by interrupting them frequently, such as whenreceiving emails, but managing these interruptions is difficult and time consuming.To understand how to better support proactive notification management, we ran 71interviews and surveyed 140 users across three studies. We identified that manyusers desired a sense of control over their attention which was difficult to achievewith current notification management tools. We created five design concepts and aprototype notification management tool to explore how to better provide this senseof control. We evaluated the concepts and tool with participants and found thatincluding visualizations of a user’s notification usage within a system that pro-vides targeted suggestions for notification management helped participants choosewhat notifications they really needed. This concept effectively supported notifica-tion management by encouraging users to control their notifications and our toolimproved the experience of the majority of participants.vPrefaceAll of the work presented henceforth was conducted in the eDapt research lab underProfessor Joanna McGrenere. All projects and methods were approved under theUniversity of British Columbia’s research ethics board certificate H16-02949.All research chapters within this dissertation have been published or will besubmitted to high quality conferences or journals in Human Computer Interac-tion (HCI). All studies I present in this dissertation received feedback from oneor more collaborators. For example, my supervisor, Dr. Joanna McGrenere, fol-lowed the entire course of my PhD research and provided feedback on my ideas forstudy design, data analysis, prototype design as well as manuscript composition.Further, she helped edit manuscripts for all papers. Below I describe my role andcontributions to each paper in depth.In Chapter 2, I report on an interview study with 20 participants conductedas part of my research proficiency exam between May and September of 2017.The resulting paper received peer review and was published at the 2018 GraphicsInterface Conference on Graphics and Human Computer Interaction. Thus, thepaper is reproduced in Chapter 2 with a few minor changes. I was the primaryauthor responsible for the majority of study design and manuscript composition.I conducted, transcribed, and analyzed all 20 interviews. However, all stages ofthis study involved feedback on my ideas from two collaborators, my supervisor,Joanna McGrenere, and a fellow PhD student, Francesco Vitale. Further, bothcollaborators provided support during data analysis by coding several transcriptsand participating in group discussions on the emerging themes. Lastly, as partof my research proficiency exam, I presented an initial draft of our results to Dr.Tamara Munzner and Dr. Thomas Fritz whose feedback helped guide our finalvisubmission to the conference.Izabelle F Janzen, Francesco Vitale, and Joanna McGrenere. (2018) Controland Personalization: Younger versus Older Users’ Experience of Notifications.Proceedings of the 44th Graphics Interface Conference (GI ’18).In Chapter 3, I report on an interview study with 30 participants conductedbetween 2019 and 2020. The research was submitted for peer review and we haveincorporated the feedback received into this dissertation. We expect to submit arevised manuscript for additional peer review based on the revisions to the workthat appear in this dissertation chapter. We made substantial revisions to addresskey concerns, such as the scope of the design space. I was the primary authorresponsible for the majority of study design, prototype creation, and manuscriptcomposition. I conducted, transcribed, and analyzed all 30 interviews. Similar tothe study in Chapter 2, my collaborators Dr. Joanna McGrenere, and PhD studentSang-Wha Sien provided feedback on my ideas throughout the process and aideddata analysis through coding several transcripts. Dr. Tamara Munzner and Dr.Thomas Fritz provided feedback during my thesis proposal on study design andresearch questions.In Chapter 4, I report on two studies that were combined into one paper. Thefirst, reported on briefly, was a supplemental survey study of 140 participants onnotification preferences conducted between 2018 and 2019. The second study,which is the main focus of the paper, was a longitudinal evaluation of a notifica-tion management prototype with 21 participants conducted in 2021. These workstogether were combined into one paper and peer reviewed before publication at the2022 ACM Conference on Human Factors in Computing Systems. The paper isreproduced here with minor changes. Dr. Joanna McGrenere provided feedbackthroughout the process for both studies and two other PhD students, Kevin Chowand Sang-Wha Sien, supported data analysis in the longitudinal evaluation by cod-ing several transcripts. For the longitudinal evaluation, Dr. Tamara Munzner andDr. Thomas Fritz from my supervisory committee provided feedback on studydesign, prototype design, and manuscript editing as part of annual PhD progressreports. I was responsible for the majority of study design, prototype creation, andmanuscript composition. I conducted, transcribed, and analyzed all 21 interviewsin the longitudinal evaluation and the 140 responses from the foundational survey.viiIzabelle F Janzen and Joanna McGrenere. (2022) Reflective Spring Clean-ing: Using Personal Informatics to Support Infrequent Notification Personaliza-tion. Proceedings of the SIGCHI Conference on Human Factors in ComputingSystems (CHI ’22).viiiTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xivList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvList of Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiAcknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1.1 Why managing distraction from notifications is so important 21.1.2 Why delivering only the notifications worth the distractionis so difficult . . . . . . . . . . . . . . . . . . . . . . . . 21.1.3 Why distraction from notifications is likely to worsen with-out further research . . . . . . . . . . . . . . . . . . . . . 31.1.4 Why specifying delivery preferences to a notification sys-tem is so difficult . . . . . . . . . . . . . . . . . . . . . . 4ix1.1.5 Why users often avoid personalizing notifications with cur-rent systems . . . . . . . . . . . . . . . . . . . . . . . . . 51.2 Research Goals, Research Questions and Approach . . . . . . . . 61.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.4 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 Control and Personalization: Younger versus Older Users Experi-ence of Notifications (Study 1) . . . . . . . . . . . . . . . . . . . . . 122.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.2.1 Notifications and Interruption . . . . . . . . . . . . . . . 152.2.2 Personalization Schemes . . . . . . . . . . . . . . . . . . 162.2.3 Older Adults . . . . . . . . . . . . . . . . . . . . . . . . 172.3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.3.1 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . 182.3.2 Participants and Recruiting . . . . . . . . . . . . . . . . . 192.3.3 Data Collection . . . . . . . . . . . . . . . . . . . . . . . 212.3.4 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 222.4 Findings and Discussion . . . . . . . . . . . . . . . . . . . . . . 232.4.1 Theme 1: Resigned to Notifications . . . . . . . . . . . . 232.4.2 Theme 2: Control and Agency . . . . . . . . . . . . . . . 272.4.3 Theme 3: Complex Categories of Notifications . . . . . . 312.5 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . 352.5.1 Standardize Personalization above the App Level . . . . . 352.5.2 Design for Ongoing Machine-Human Partnership . . . . . 362.6 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 Designing Notification Management Tools to Support a Sense of Con-trol (Study 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433.2.1 Notifications and Interruptions . . . . . . . . . . . . . . . 43x3.2.2 Notification Personalization and Management . . . . . . . 443.2.3 Reflection on Notifications . . . . . . . . . . . . . . . . . 463.3 Design Axes for Involving Users in Notification Management . . . 473.3.1 Process to create the design axes . . . . . . . . . . . . . . 473.3.2 Final Design Axes . . . . . . . . . . . . . . . . . . . . . 503.4 Design Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . 523.4.1 Process to Create Design Concepts . . . . . . . . . . . . . 523.4.2 Final Design Concepts . . . . . . . . . . . . . . . . . . . 553.5 Concept Exploration Study Methodology . . . . . . . . . . . . . 593.6 Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 653.6.1 Theme 1: Design Concept Preferences . . . . . . . . . . . 653.6.2 Theme 2: Need for Control and Risk Management . . . . 683.6.3 Theme 3: Age-associated Differences in Notification Man-agement Needs . . . . . . . . . . . . . . . . . . . . . . . 693.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 713.7.1 Future notification management systems should combinemanual notification rules with customizable suggestions toleverage their complementary roles . . . . . . . . . . . . 713.7.2 Decision support and degree of automation design axesshould be carefully considered in future notification man-agement tools . . . . . . . . . . . . . . . . . . . . . . . . 723.7.3 Identified differences in life and work structure could im-ply a need for design elements that support periodic man-ual revision of notification management preferences . . . 743.7.4 Concerns about the risks of personalizing could be man-aged through trial periods for personalizations . . . . . . . 753.8 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 763.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764 Reflective Spring Cleaning: Using Personal Informatics to SupportInfrequent Notification Personalization (Study 3) . . . . . . . . . . . 784.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 794.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82xi4.2.1 Factors Impacting Receptivity to Notifications . . . . . . 834.2.2 Approaches to Software Personalization and NotificationManagement . . . . . . . . . . . . . . . . . . . . . . . . 844.2.3 Challenges in Reflective Design and Personal Informatics 854.3 Design Process . . . . . . . . . . . . . . . . . . . . . . . . . . . 864.3.1 Supplementary Survey to Generate a Corpus of Useful Sug-gestions . . . . . . . . . . . . . . . . . . . . . . . . . . . 864.3.2 Iterative Refinement and Final Design of the ReflectiveSpring Cleaning Design . . . . . . . . . . . . . . . . . . 874.4 Methods for the Longitudinal Study of Reflective Spring Cleaning 934.4.1 Participants & Recruitment . . . . . . . . . . . . . . . . . 934.4.2 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . 944.4.3 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . 974.5 Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 984.5.1 Overall Usage of Spring Cleaning Tool . . . . . . . . . . 984.5.2 Critical reflection about notification usage was prompted . 1014.5.3 Individual differences in motivations to reflect . . . . . . . 1044.5.4 Degree to which needs were met impacted continued en-gagement . . . . . . . . . . . . . . . . . . . . . . . . . . 1064.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1074.6.1 Targeted infrequent reflection could better engage less in-trinsically motivated users . . . . . . . . . . . . . . . . . 1084.6.2 Associating PI with software personalizations enhances de-sign goals for both PI and personalization . . . . . . . . . 1094.6.3 Notification Spring Cleaning could be incorporated as apart of holistic wellness systems . . . . . . . . . . . . . . 1094.6.4 Opportunities remain to refine Reflective Spring Cleaning 1104.7 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1114.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1125 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1145.1 Summary of Results and Contributions . . . . . . . . . . . . . . . 114xii5.1.1 Refined prior characterizations of notification managementpractices and needs . . . . . . . . . . . . . . . . . . . . . 1145.1.2 Design elements, concepts, and axes to support controlover notifications . . . . . . . . . . . . . . . . . . . . . . 1155.1.3 Novel design approach to notification management . . . . 1165.1.4 Design directions based on empirical work . . . . . . . . 1175.2 Limitations and Notes for Interpretation . . . . . . . . . . . . . . 1185.2.1 Generalizability of our central claims . . . . . . . . . . . 1185.2.2 Design of Reflective Spring Cleaning . . . . . . . . . . . 1195.2.3 Heterogeneity of Older Users . . . . . . . . . . . . . . . 1215.3 Implications and Additional Future Work . . . . . . . . . . . . . 1225.3.1 Implications for UI and product design . . . . . . . . . . 1225.3.2 Implications for future HCI research . . . . . . . . . . . . 1235.3.3 Implications for algorithm design . . . . . . . . . . . . . 1245.3.4 Implications for Policy . . . . . . . . . . . . . . . . . . . 1255.4 Closing Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . 127Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128A Supporting Materials . . . . . . . . . . . . . . . . . . . . . . . . . . 169A.1 Supplementary Survey Results from Chapter 4 . . . . . . . . . . . 169A.2 Categories of App from What Abstraction from Chapter 4 . . . . 179B Supplementary survey for Study 3 Questions . . . . . . . . . . . . . 180C Additional Documents . . . . . . . . . . . . . . . . . . . . . . . . . . 222C.1 Study 1 Documents . . . . . . . . . . . . . . . . . . . . . . . . . 222C.2 Study 2 Documents . . . . . . . . . . . . . . . . . . . . . . . . . 227C.3 Study 3 Documents . . . . . . . . . . . . . . . . . . . . . . . . . 230xiiiList of TablesTable 2.1 Study 1 demographics and device usage. . . . . . . . . . . . . 20Table 2.2 Categories of notifications in Study 1. . . . . . . . . . . . . . . 32Table 3.1 Our four design axes for proactive notification management. . . 49Table 3.2 Where industry and research tools are placed in our design axes. 52Table 3.3 Where our five design concepts are placed within our design axes. 54Table 3.4 The 10 concerns users addressed in Study 2 . . . . . . . . . . 61Table 3.5 Study 2 demographics and device usage. . . . . . . . . . . . . 63Table 4.1 Suggestions shown in Study 3. . . . . . . . . . . . . . . . . . 88Table 4.2 Study 3 demographics and device usage. . . . . . . . . . . . . 95Table A.1 The 20 suggestions shown in the supplementary survey. . . . . 172xivList of FiguresFigure 2.1 Aranda’s Framework for the experience of notifications. . . . 16Figure 3.1 Early Study 2 paper prototypes. . . . . . . . . . . . . . . . . 55Figure 3.2 Screenshots of the first four design concepts in Study 2. . . . . 56Figure 3.3 The three variations of the Suggestions design. . . . . . . . . 57Figure 4.1 Early paper prototype sketches for Study 3. . . . . . . . . . . 90Figure 4.2 The three facets of our data abstraction. . . . . . . . . . . . . 92Figure 4.3 The final Reflective Spring Cleaning interface. . . . . . . . . 92Figure 4.4 Overview of the procedure for the longitudinal reflection study. 94Figure 4.5 Average number of notifications received in Study 3. . . . . . 100Figure A.1 Histogram of ages in supplementary survey. . . . . . . . . . . 171Figure A.2 Average suggestions accepted in supplementary survey. . . . . 173Figure A.3 Suggestions accepted by group in supplementary survey. . . . 174Figure A.4 Desired trial period length in supplementary survey. . . . . . . 175Figure A.5 Enough information provided in supplementary survey. . . . . 176Figure A.6 Perceived risk in supplementary survey. . . . . . . . . . . . . 177Figure A.7 Impact of trial period in supplementary survey. . . . . . . . . 178Figure C.1 Advertisement for Study 1. . . . . . . . . . . . . . . . . . . . 223Figure C.2 Advertisement for Study 1, specified to older adults. . . . . . 224Figure C.3 Consent form for Study 1, page 1. . . . . . . . . . . . . . . . 225Figure C.4 Consent form for Study 1, page 2. . . . . . . . . . . . . . . . 226Figure C.5 Advertisement for Study 2. . . . . . . . . . . . . . . . . . . . 227xvFigure C.6 Consent form for Study 2, page 1. . . . . . . . . . . . . . . . 228Figure C.7 Consent form for Study 2, page 2. . . . . . . . . . . . . . . . 229Figure C.8 Advertisement for Study 3. . . . . . . . . . . . . . . . . . . . 230Figure C.9 Advertisement for the supplementary survey in Study 3. . . . 231Figure C.10 Consent form for Study 3, page 1. . . . . . . . . . . . . . . . 232Figure C.11 Consent form for Study 3, page 2. . . . . . . . . . . . . . . . 233Figure C.12 Consent form for the supplementary survey in Study 3. . . . . 234xviList of AbbreviationsHCI Human Computer InteractionML Machine LearningxviiAcknowledgmentsFirst and foremost, I want to thank my supervisor, Dr. Joanna McGrenere, for herthoughtful advice and support. She was always patient, thoughtful, and pushed meto do better, which made her a considerate and highly valued supervisor.I am also grateful for my supervisory committee members, Dr. Tamara Mun-zner and Dr. Thomas Fritz, for their extremely helpful feedback and advice through-out my research. In particular, I thank Dr. Tamara Munzner for her key guidance onhow to frame the visualization and design space research that became a key focusof the later half of my research. I thank Dr. Thomas Fritz for providing a neededand thoughtful voice, especially to discuss the role of notification management ingroup collaboration and communication.I am also thankful to X and Y, my external evaluators... [Add in when this isdecided]I also want to briefly thank all the various MUX and eDapt lab members thatprovided feedback on my work at various points. In particular, Francesco Vitale,Sang-Wha Sien, Kevin Chow, Kamyar Ardekani, Oliver Schneider, Soheil Kian-zad, Laura Cang, Paul Bucci, Taslim Arefin Khan, Yelim Kim, Shareen Mahmud,Jessalyn Alvina, Emmanouil Giannisakis, Hanieh Shakeri, Hayley Guillou, AnnaOffenwanger, Dongwook Yoon, Karon MacLean, Kellogg Booth and Hasti Seifiall provided helpful comments.I thank the various UBC staff and faculty that have supported me in differentcapacities, such as Laura Selander and Joyce Poon who provided administrativesupport.I want to thank all of the anonymous reviewers for my various submissionsto GI, CHI, IJHCS, and CSCW who generously took the time to review my work.xviiiTheir feedback helped guide my research at various points, especially during CHI’srevise and resubmit process.A special thanks goes out to Dominik Weber, Alexandra Voit, and Niels Henze,who created the open source notification log application and framework that col-lected notification data for our prototype in Chapter 4. In addition, Dominik Weberanswered several technical questions on using the app effectively and outlined whatits technical limitations were.I thank all of the many participants in my studies who made this research pos-sible by taking the time to discuss their notification management needs and expe-riences with me.Finally, I am eternally grateful for all my various friends, family, and my wife,Liorea Arin, for their continuous support, advice and caring.Funding for this work was provided by the University of British Columbia,Aging Gracefully across Environments to Ensure Well-being, Engagement andLong Life (AGEWELL) - Networks of Centres of Excellence (NCE) (grant No.AWCAT-2017-113) and Natural Sciences and Engineering Research Council ofCanada (NSERC) (grant No. RGPIN-2017-04549, RGPIN-2016-0432).xixChapter 1IntroductionIn this chapter, we outline the main motivations for our work and the context forwhy notification management continues to be a significant problem across differingmobile technology platforms. We then describe the research questions, approachand methods we took to address this problem. Finally, we summarize the maincontributions of this research.1.1 MotivationComputers have long provided software developers with tools to interrupt the userin order to provide them with time sensitive information, such as sounding a tonewhen an email is received [8, 102, 183, 235]. More recent mobile devices havepopularized the concept of a notification centre, where any application may askthe system to deliver a standardized interruption called a ‘notification’ to a unifiedinterface [183]. Decades of prior work has explored how to balance the distractionnotifications cause against their usefulness. So why is our interest in notificationmanagement so timely [4, 25, 29, 101, 102, 112, 113, 120, 163, 187, 244]? We ar-gue that the proliferation of constantly connected mobile technology supported byfrequent notifications has exacerbated underlying issues around stress, distractionand mobile addiction in the modern workplace [43, 87, 106, 134, 134, 135, 152,184, 223]. These concerns around stress and distraction have led to a resurgence inHuman Computer Interaction (HCI) research and industry creating novel notifica-1tion management systems [53, 134, 159, 195, 235, 250, 251] in addition to our ownresearch. Here, we explain the main research challenges in the problem space anddiscuss why addressing the problem has remained such a challenge for so long.1.1.1 Why managing distraction from notifications is so importantRecent research on productivity in the workplace has become increasingly con-cerned with the phenomena of burnout from stress and identified frequent distrac-tion as one source of that stress [92, 116, 134, 135, 148, 150, 208]. Most priornotification management work aimed to reduce the productivity loss associatedwith distraction from interruptions, such as by filtering or deferring unimportantnotifications [2, 8, 9, 11, 27, 52, 70, 103, 207]. However, recent research hasidentified that aggressive approaches to reducing distraction, such as stay focusedapps that turn off internet connection for a set time, increase stress for some users[13, 112, 134, 153, 160, 183, 191, 214]. Achieving this balance leads us to specifythe challenging, highest-level research problem that is the focus of our work: Howcan a device deliver only the notifications that are worth the stress and distractionthey cause?1.1.2 Why delivering only the notifications worth the distraction is sodifficultPrevious work has explored the factors that affect which notifications users are re-ceptive to receiving despite the distraction they cause and found them complex,contextual and difficult to precisely predict [13, 70, 160, 186, 195, 244]. To sum-marize this broad thrust of research: the user’s context when receiving the notifica-tion, the sender of the notification, and the content of the notification itself stronglyimpact which notifications users value according to complex, individual prefer-ences. For example, the perceived importance of the user’s primary task contextwhen they receive notifications influences which notifications they are willing torespond to, such as preferring to prioritize eating a family dinner over notificationsfrom work [13, 56, 70, 89, 91, 160, 244]. Meanwhile, many aspects of the contentof a notification, such as sender, app, urgency to respond, relevance to work or in-terest in the subject can influence whether it is perceived as relatively more or less2valuable to receive. For example, messages from close social contacts are usuallymore important than messages from acquaintances, strangers or automated mailinglists [13, 67, 70, 70, 160, 172, 244]. At the start of my research in 2016, Aranda etal. at Google Research synthesized these insights with ethnographic data to createa model of notifications that helped to motivate this research and revisions to theAndroid notification system [13, 98]. They describe two axes (enjoyment and timeurgency) that create four categories of notifications: VIP (enjoyable, urgent andimportant messages from close social contacts), Mission Critical (urgent, painfulwork), Nagging (non-urgent, painful work), and fun (non-urgent, enjoyable con-tent) [13].The core challenge informing our work is that, despite the existence of suchhigh-level trends in notification preferences, each individual user has a complex,non-obvious set of personal preferences for notification delivery. For example, auser may feel messages from close family are very important, but may still havecontexts, such as around major deadlines, where they would prefer to avoid beingdistracted for those notifications [13, 160]. Predicting notification preferences isfurther complicated by how individual differences amongst users may influencenotification preferences, such as prior research suggesting that older adults aremore negatively impacted by distraction than younger users [46]. The complex-ity of individual preferences towards notification delivery makes it difficult for anotification system to precisely identify which notifications are worth the distrac-tion and leads us to further specify the research problem of delivering only usefulnotifications: How can a notification system better support users to specify whichnotifications they are willing to receive despite the distraction they cause?1.1.3 Why distraction from notifications is likely to worsen withoutfurther researchBoth our research and prior research on which notifications users value suggestthat many notifications that fit Aranda’s “Fun” category are disliked by users butfrequently sent by apps to generate revenue [13, 159, 160]. Examples of notifi-cations generating revenue include sending notifications to remind users to spendmoney on time limited sales or events, engage with social media, or view prod-uct recommendations. In some cases, such as video games that notify users to3purchase timed “loot-box” items, these reminders to engage may cause users di-rect harm. Several prior studies have suggested these games are correlated withproblem gambling [48, 57, 227]. However, prior to very recent updates to the no-tification management tools in mainstream mobile operating systems, users wererequired to manually opt out of notifications for each app, often using a separatesettings panel [96, 98]. While users may eventually silence applications that sendtoo many notifications, they also avoid spending the effort to personalize unlessthere is a specific triggering event, such as a sudden increase in advertisements,that convinces them action is needed [13, 18, 160]. The potential to generate rev-enue from notifications provides app developers strong monetary incentives to fig-ure out just how many unwanted and distracting notifications they can send beforetriggering users to uninstall or unsubscribe to notifications.1.1.4 Why specifying delivery preferences to a notification system isso difficultRather than taking a very broad approach to notification management, such as mut-ing all notifications while working, most research and industry projects in this areaallow users to personalize how and which types of notifications are delivered basedon the user’s individual preferences. For example, less salient alert modalities maybe applied to notifications the user perceives as unimportant, such as slowly bright-ening an ambient light when advertising email is received [8, 9, 11, 97, 164, 207].Three main approaches have been explored prior to this research to personalizenotification delivery:• User defined preferences through a manual settings panel [16, 53, 78, 195,222].• Automated deferment of notifications to opportune moments or filtering unim-portant notifications based on Machine Learning (ML) models of notificationusage and device sensor data [40, 40, 70, 74, 91, 101, 105, 144, 145, 157,188, 207, 236, 238].• “Mixed-initiative” suggestions for how to personalize notifications based onmachine-learning models of notification usage data [96, 159]. For example,4suggesting the user should filter notifications that contain certain keywordsassociated with advertising. The user is then in control to accept or rejectthese suggestions.All proposed approaches to notification management come with implicit and ex-plicit design trade-offs and each approach is still being being explored in relativelyrecent work [16, 40, 53, 91, 96, 144, 159, 195]. For example, research has iden-tified that the effort involved in using manual settings panels provides a sufficientenough barrier to deter most users [18, 159, 214, 252]. Meanwhile, users expressstrong concerns over automated deferment systems because they fear missing crit-ical notifications due to ML inaccuracy [11, 70, 102, 159, 207]. Lastly, suggestionsfor notification management cause a small amount of distraction to the user whendelivered. Thus, designers have to carefully balance how much the system helpsthe user reduce distraction against the distraction it causes [13, 96, 159, 160, 229].What is clear, however, is that, despite the research on these varying approaches,which has led to revisions in popular industry notification management systems,users continue to express substantial frustration and also avoid notification man-agement [13, 96, 98, 110, 161, 191, 214, 252].1.1.5 Why users often avoid personalizing notifications with currentsystemsOne reason why users may avoid spending the effort to manage their notifica-tions despite their continued frustration is that they may have normalized beingfrequently interrupted. Research has identified how many potentially problematichabits of technology use, which include stressful distractions from notifications,can quickly become unconscious and normalized [73, 127, 218]. For example, re-cent research has suggested users have normalized feeling uncomfortable about thedata apps gather about their users [217]. To challenge the normalization of prob-lematic habits, personal informatics and reflective design research has encouragedusers to deeply reflect on their personal behaviors through visualized behavior data[73, 218]. For an effective example of such a system, after concentrated reflec-tion on a personal informatics tool visualizing peer technology use, users felt theyhad overestimated the usefulness and necessity of being constantly connected to5mobile technology [28, 61, 127, 168, 201]. While research had just begun layingthe technical groundwork for users to reflect on notification use prior to our work,we were concerned that users who underestimated the disruptiveness of notifica-tions may avoid managing them [249]. Prior research had suggested that manyusers avoid personalizing notifications due to substantial barriers, such as time, ef-fort, and knowledge of existing personalizations [16, 18, 214]. However, as ourresearch progressed, we became increasingly concerned that users would continueto avoid pushing through these barriers unless they were convinced notificationswere disruptive enough to justify the effort.Notification research in industry and academia has focused primarily on reduc-ing the barriers to personalizing notifications, possibly in response to numerouscritiques in the popular press about the cost of distraction from mobile notifica-tions [95, 96, 98, 110, 161]. For example, the most recent revision of IOS, IOS15,was released in late Sept 2021 and allows users to personalize many different appsat once through a new “focus” mode [96]. Users who opt in to the new “focusmode” will manually create a set of preferences for notification delivery, but re-ceive machine assistance in the form of suggestions of which contacts to allowthrough filters and identification of key contexts to personalize around, such aswhile working or relaxing. Focus mode will likely improve the experience of someusers and reduce distraction, especially while working, though ML accuracy re-mains a concern. Our later research in Chapter 4 is the first to explore a combinedframing to solve this problem. We ask: How can a system both encourage users toreflect on whether they need to personalize their notifications as well as reduce thebarriers to conducting that personalization?1.2 Research Goals, Research Questions and ApproachUser-centered design is a common methodological approach in both HCI researchand product design that focuses on making tools and technology that will be use-ful to users by centering their needs throughout several design phases [1]. In thepre-design phase, user-centered design attempts to define which problems usersactually have through direct contact with users, such as through interviews or ob-servations. In the early design phase, user-centered design rapidly explores design6alternatives that might address these problems to identify promising design direc-tions. Lastly, in the later design phases, the most promising design directions areexplored in more depth and evaluated for their ability to address users’ real prob-lems ideally in naturalistic settings. Our work followed the user-centered designprocess by addressing three main goals that roughly align to these major phases.Our goals were as follows:1. During our pre-design research (Chapter 2), we aimed to understand howunder-explored user groups, such as older adults, experienced notifications andclarify users’ underlying mental model of which notifications should be personal-izable through a rich description of the types of notifications they perceived. Olderadults’ notification experiences were studied in particular because prior researchhas shown other forms of distraction are more costly for them [46]. The researchquestions we used to accomplish this goal were:• What types of notifications do users remember receiving, how do they per-ceive the impact of these notifications, and what strategies do they use tomanage them? (Chapter 2)• What individual differences occur in the prior research question for userswithin and between age groups? (Chapter 2)2. During our early design phase (Chapter 3), we aimed to characterize thepotential notification management designs and their ability to support proactivelyspecifying notification preferences to the notification system. This was done tounderstand how users’ need for control over notifications, a key result from Study1, could be addressed by the design of notification management tools. The researchquestion we used to accomplish this goal was:• Which design concepts and elements within notification management sys-tems do users feel best allow them to express and refine their personalizationpreferences? (Chapter 3)3. During our last design phase (Chapter 4), we aimed to deeply explore how anovel and promising design element from the prior phase, reflective visualizationsof notification use, would be experienced by users within a notification manage-ment tool. We note that our recruitment focus in this study moved away from7retired older adults as we suspected there would be substantial individual differ-ences in their notification use and our prototype assumed users had at least somework related notifications. The research questions we used to accomplish this goalwere:• How would simple visualizations of notification use in a notification man-agement tool influence users’ motivation to personalize their notifications?(Chapter 4)• How would a notification management system providing explicit opportuni-ties for reflection on notification use affect users’ notification managementpractices and experience? (Chapter 4)To answer these research questions, we conducted three studies:• Study 1 (Chapter 2): A foundational interview study of notification manage-ment preferences with 20 participants that uses inductive analysis to under-stand their notification management needs.• Study 2 (Chapter 3): A research through design inspired exploration of thedesign concepts and elements used in notification management tools, sup-ported by interviews with 30 participants.• Study 3 (Chapter 4): A summative, longitudinal evaluation of a novel no-tification management technique with 21 participants. The design of thistool was informed by a supplementary survey including 140 participants alsobriefly reported in this chapter.1.3 ContributionsThe contributions of our work are focused on the following areas:Theoretical Contributions:• Characterization of individual differences in how older and younger adultsexperience notifications and clarifying their mental model of which notifica-tions they perceive can and should be personalized. (Chapter 2)8• Refinement of Aranda’s model of notifications and identifying that the im-portance of the “fun” axis is exaggerated. Users cared primarily about whethera notification required an urgent response unless all notifications were equallyurgent. (Chapter 2)Design Contributions• A set of five design concepts exploring how users might proactively man-age notifications in the future. Together with four design axes, these designconcepts serve as exemplars for how users could more directly control noti-fication management. (Chapter 3)• The proof of concept Reflective Spring Cleaning system which was designedto encourage users to manage their notifications by offering explicit oppor-tunities for reflection on notification use. (Chapter 4)• A set of potential design directions for future notification management toolsacross all three studies. (Chapters 2, 3 and 4)Empirical Contributions• An interview study with 20 younger and older users exploring age-associatedindividual differences in notification preferences. The results of this studyidentified key, unaddressed challenges in the research space, such as a lackof a sense of agency over notifications in current notification managementsystems. (Chapter 2)• An interview study with 30 participants exploring the perceived uses of de-sign concepts and elements that could express notification preferences to anotification management system. The results of this study identified severalpromising design concepts that might provide a better sense of control forusers. (Chapter 3)• A set of two studies to support the design and evaluation of the ReflectiveSpring Cleaning concept. The first was a longitudinal study of 21 partici-pants, involving interviews. The second was a supplementary survey with9140 users. The results of these studies suggest that Reflective Spring Clean-ing effectively supports proactive notification management for many users.(Chapter 4)1.4 OutlineThe rest of the dissertation uses the following structure. In Chapter 2, we presentStudy 1, where we conducted exploratory interviews with 20 younger and olderusers of notifications. We identify a need for a sense of control and agency in noti-fication management, slightly more pronounced in older adults, and clarify users’underlying mental model of which notifications can and should be personalized.Study 1 represents our first steps into the user-centered design process by explor-ing individual differences in notification management needs amongst key groupsof users.In Chapter 3, we present Study 2, which probes users’ reactions to notificationmanagement design concepts that might support more proactive control over noti-fications. Through interviews with 30 users of varying ages and occupations, weidentify the complementary roles of different design elements within notificationmanagement systems. Further, we identify strong user interest in more reflectivedesign elements that might support thoughtful control over notifications.In Chapter 4, we present Study 3, which deeply explores and evaluates howthese reflective design elements could support proactive notification management.Through a supplementary survey of 140 participants probing reactions to notifica-tion personalizations and low fidelity prototyping, we created the Reflective SpringCleaning design concept. We evaluated this concept in a longitudinal study with21 participants and showed its ability to encourage notification management andimprove the majority of our participants’ experiences with notifications. Further,we identified that, while spring cleaning, participants were prompted to reflect crit-ically about their own ability and need to manage notifications.Finally, in Chapter 5, we summarize our results and discuss the implications ofour work. We note that we do not include an explicit summary of related work inits own chapter of this dissertation. Instead, we leave each research chapter withthe related work section originally in the corresponding peer reviewed paper or10manuscript submission to keep more targeted sections, such as the focus on olderadults in Study 1 or the focus on design for reflection in Study 3, beside the researchwhere it is most relevant.11Chapter 2Control and Personalization:Younger versus Older UsersExperience of Notifications(Study 1)In this chapter we present Study 11, which investigates how older and youngerusers think about, interact with, and personalize their notifications. Older adults’notification management habits had not been studied before, but our focus waswarranted because older adults are typically more negatively impacted by distrac-tion, such as taking longer to return to task [46]. By specifically exploring andcontrasting the needs of people in these two age cohorts we uncovered a key desireamongst many, particularly older users for a sense of control and agency over noti-fications. The importance of explicitly designing notification management tools tosupport this sense of control is the core underlying theme throughout the rest therest of my work.1Originally published in Izabelle F Janzen, Francesco Vitale, and Joanna McGrenere. (2018)Control and Personalization: Younger versus Older Users’ Experience of Notifications. Proceedingsof the 44th Graphics Interface Conference (GI ’18).122.1 IntroductionMobile devices are more ubiquitous now than ever before and deliver a large num-ber of notifications. While notifications can be helpful, such as for emergencyalerts, they interrupt workflows and hamper productivity [13]. Thus, notificationoverload is a problem of great research interest. Academic work has tracked the in-creasing scale of the problem and investigated ways to improve delivery [159, 191].Our main focus in this work is understanding the experience of receiving noti-fications. A key motivator was work by Aranda et al. which suggested a model ofthis experience based on two main dimensions: enjoyment and time urgency [13].Aspects of that work were used as the basis for the redesign of notifications in thelatest Android OS, Oreo [98]. We attempt to further refine and extend their modelto support future design.We consider notifications to be an implementation of interface interruptionthrough a unified notification centre that provides different alert modalities, such assound, vibration or silent. These are most common on mobile phones and tablets,but notification centres have also been added to recent desktop operating systemsas well, such as Windows 10. Notifications are varied but we include anythingdisplayed in the notification centre of one of these devices, such as update installa-tions, text messages, emails or social media awareness.A key area of interest within the experience of notifications is understandinghow that experience varies for a wide range of users. By supporting diverse userswe may open up new markets and improve the experience for the general popu-lation (similar to curb cuts for wheelchairs allowing bikes to travel more easily).We were specifically curious to see if age would impact the notification user expe-rience. Older adults are more susceptible to interruption and have less familiaritywith mobile technology [46, 137]. While older adults’ experiences with mobiletechnology has been studied [133], to our understanding, no research has focusedspecifically on their use of notifications. Thus, one goal of this study is to investi-gate how the notification needs of younger and older users differ.We also wanted to understand how these differences could be leveraged toimprove notification user experience. One approach is providing personalizationsettings. There are several competing techniques for providing such settings. These13include user controlled, such as through a settings panel, adaptive interfaces wheresettings are adjusted automatically, or machine-human partnerships where settingsare suggested to the user [32, 79]. Choosing between these approaches is a complextrade-off between control and convenience.Regardless of technique chosen, a notification personalization system will re-quire personalizations to be applied to some “type” of notification understood bythe user, such as Facebook notifications. Erroneous personalizations can be partic-ularly problematic for notifications so it is critical that the user clearly understandwhich notifications are impacted [159]. Previous work has suggested notificationfilter rules that we conjecture might be confusing to less technical users [159].Thus, the second goal of our study is to investigate what broader categories ofnotifications users perceive and how that might impact their personalization needs.To address our two high-level goals, we conducted semi-structured interviewswith 20 participants around their notification usage and personalization prefer-ences. We split the participants into two groups, 10 were younger (19-30 yearsold) and 10 were older (48-74). We sought to answer the following research ques-tions:• How do users perceive the impact of the notifications they receive?• What strategies do users employ to manage their notifications and why?• What types of notifications do users (consciously or not) perceive?• What individual differences occur in the above for users within and betweenage groups?The contributions of this work are as follows: We refine previous work to showthat enjoyment plays a minor role in the experience of notifications, usually dom-inated by urgency, directness and social closeness. We reveal that older adults inparticular desire a sense of control over their notifications that is difficult to achievewith current systems. Lastly, we suggest categorization schemes of notificationsinformed by what types of notifications users perceive.142.2 Related WorkDesigning for interruptions is a longstanding area of research in HCI so we focuson work relevant to notifications and older adults.2.2.1 Notifications and InterruptionDesigning interruptions to minimize cost, such as quick time to resume task or lowcognitive load, has been well studied. Studies have investigated what moments intask completion are least costly to interrupt [2, 70] and modelled how task recoveryis accomplished [8]. There is also work investigating the impact of different alertdesign decisions [11] noting trade-offs between alert salience and interruption cost.Significant work has been done to quantify how often people are interruptedby notifications [112, 191, 214]. Estimates vary, but typically the numbers areconsidered high and indicative of a significant problem. Complementary studieshave investigated qualitative perceptions of what users value in notifications [13,160]. Users indicate frustration with notification overload and care more aboutdirect messages from socially close contacts than other notifications. There is alsowork on the etiquette systems users create around notifications [108, 172] notingusers associated apps with social groups.Aranda et al. synthesized this work into the model of notifications displayed inFigure 2.1 which we attempt to refine [13]. They present two axes, enjoyment andurgency, that determine the experience of receiving a notification. They proposeseveral categories of notifications based on these axes, VIP (for messages fromclose family and friends) Mission Critical (for highly urgent work) Nagging (forless urgent work) and Fun (non-urgent but interesting).Much work has tried to improve the user experience of mobile notifications.For example, deferring interruptions to opportune times to reduce distraction is acommon technique [70, 187]. Another piece of research described PrefMiner, anapp that suggests notification filter rules to users, and was a major motivator forour study [159]. Despite their suggestions being conservative, users only accepted56 percent of them. We conjectured that this might be because it was not obviouswhat notifications would be removed by the filter rules. Thus, understanding whatgroups of notifications users perceive that could potentially be personalized around15Figure 2.1: Framework for the experience of Notifications presented byAranda et al. Several categories are generated based on varying en-joyment and urgency [13].became a major focus of our work.2.2.2 Personalization SchemesFew users personalize software because of many barriers, such as technical knowl-edge or time investment, but users experience triggers, such as particularly annoy-ing notifications, that convince them to push through those barriers [18]. To reducethose costs, researchers have developed adaptive interfaces that update settings forusers automatically [79]. However, such adaptation comes with complex tradeoffs.If UI elements change position for example, the performance benefit from learningis reduced. Users may also experience frustration if the adaptions are incorrect[79].16Interfaces can incorporate adaptability to different degrees. “Mixed-initiative”personalization, or equivalently “machine-human partnerships” provide greater con-trol to the user. In these systems the computer suggests personalizations to the userand refines those suggestions over time [32, 33]. These systems have been appliedto notification management but still have complex design issues [159]. The sug-gestions themselves may be distracting. Also, the communication must be clearor the user’s response will not be accurate. Our study attempts to understand howusers think about notifications so that this communication can be refined.2.2.3 Older AdultsOlder adults are a growing segment of the population. HCI research involvingolder adults has used varying age cutoffs to define “older adult”, such as 50, 55,60, 65, or 70+ [31, 71, 233]. For the purposes of our work, we simply wanted twogroups that were relatively younger and older than the other. We aimed for 50+for our older group, but had two participants just under, as we will explain in theParticipants section. Aging is accompanied by declines in perceptual, motor andcognitive abilities. These differences change users’ technology requirements andpresent tough design challenges [71].Various research projects have focused on how older adults use and learn newtechnology [133, 137, 246]. They can face many challenges from simple familiar-ity with tools and metaphors, to font size readability problems [23]. Many studieshave used the new opportunities offered by connected technology to solve life chal-lenges or improve interfaces for older adults [156, 211].Older adults are more negatively impacted by interruption [46] which poseschallenges to the design of certain interfaces and applications [31]. Thus, we con-jectured that older adults may be more negatively impacted by notification over-load. We also conjectured that older adults may be less able to personalize giventhe barriers they face learning new technology [137, 246]. However, to our knowl-edge, no previous study has specifically focused on the notification needs and ex-periences of older users.172.3 MethodologyParticipants completed a semi-structured interview around their notification prac-tices, conducted in a lab. A series of five pilot interviews with members of our labwere done to refine the interview questions and priming activity. After collectingbasic demographics, we progressed into the interview which took about an hour.The interview had three stages, with the first two being more structured.2.3.1 ProcedureIn phase 1, we sought to understand how participants feel about notifications andhow they manage them. We aimed to let participants describe these things withoutpushing them to a specific cognitive frame. We asked open ended, neutrally wordedquestions about these topics, such as “How do you feel notifications impact yourdaily life”. We allowed participants to talk about whatever the term notificationsmeant to them, providing our definition if there was confusion. Some participantstalked about notifications they got on their tablet, laptop, or phone. We also askedusers what strategies they use to manage their notifications.In phase 2, participants completed a priming activity to get them to reflecton what types of notifications they get. Participants were shown a series of 75cards. Each card had an example notification written on it that covered (1) theapp that delivered it, (2) who it’s from and (3) context about the content. Examplesinclude “Email confirming that a package you ordered has been shipped and will bedelivered next week” or “Text from your boss telling you to update your timesheetby the end of the week”. After removing notifications each participant thoughtthey would never receive, participants were asked to categorize the notifications inwhatever way makes sense to them. Participants then added labels to the categories.We asked follow-up questions around why they created each group, how they mightdescribe it, how they felt about that group, and what notifications within it wereparticularly important or unimportant.We used a log of the primary researchers’ notifications over a week to createour initial notification example list. We refined this by asking our pilot participantsto share some of their notifications. We found 75 notifications was about the upperlimit to keep the activity manageable. We reviewed the examples and balanced18them on a couple of coarse subjective factors. These included social closeness,such as from a friend, family member or someone you don’t know, and time ur-gency, such as in an hour, tomorrow or next week, which were important to theexperience of notifications in other studies [13]. We made sure several top socialmedia apps had representation, such as Facebook, Twitter, Youtube or Instagram,as well as email, texting and system messages. We tried to make sure that each apphad representation for all levels of closeness and urgency, but that was not alwayspossible, such as system messages which cannot be from family. We continued tocollect notification examples from participants in the main study, but changes wereminor past the first few interviews.Phase 3 was more open ended. We focused on asking participants how theymight like to personalize their devices, now that they had had some time to reflect.Prior to the study we had compiled a rough list of the notification settings availablein the default Android operating system, a number of the top-rated apps under asearch for “notification managers”, as well as a number of common social mediaapps. Discussing all settings would have been impossible, so we grouped them intothree approaches and asked users how they might feel about them and which theywould use. Specifically we asked about: 1. Removing unwanted notifications, suchas filtering, unsubscribing, 2. highlighting important notifications, such as specialsounds, visuals or repeated alarms, or 3. downplaying less important notifications,such as silent or batched delivery. However, we let participants steer the conversa-tion based on what they wanted, and we brought up nuances and edge cases thathad occurred in previous interviews to foster discussion. We also mentioned a hy-pothetical personalization system that suggested such settings to them, and if theywere interested, how much control would they want over it.2.3.2 Participants and RecruitingWe recruited two cohorts of ten participants, whose demographics information islisted in Table 2.1. The first was recruited exclusively from a general call on ouruniversity’s paid studies board (four men, six women). Participants simply had tobe 18 years old or older, speak English and have used a device that delivers notifi-cations for more than six months. This recruitment criteria intentionally skews this19Table 2.1: Demographics and device usage data of all 20 participants in Study1. Amount of phone usage and typical strategies for when they checktheir phone were self described by participants. The younger cohort ofparticipants are listed in the top half of the table and older participants inthe bottom half.P# Age Gender Phone Use(hrs/day)Check Phone Occupation1 y 25 male 8+ when it rings student - comp sci2 y 22 female 8+ 100 times a day student - psych3 y 25 female 8+ twice per hour student - comp sci5 y 20 male 1-2 quite often student - econ6 y 30 female 2-3 every so often admin assistant7 y 20 female 2 when it rings student - arts9 y 21 female 8 every few min student - nutrition10 y 24 female 4 15 times a day student - psych11 y 23 male 8+ every half hour student - criminol-ogy12 y 19 male 8+ every 5 minutes student - comp sci4 o 67 female 2 when it rings retired - program-mer8 o 48 male 8 once per day IT Consultant13 o 51 male 5 when it rings admin Staff14 o 62 female 1-3 4-5 times/day retired - translator15 o 64 male 1-3 every half hour writer16 o 48 male 4 when it rings musician17 o 50 female 8+ when not busy small business18 o 74 female 8+ when it rings retired - program-mer19 o 57 female 2 when it rings admin staff20 o 61 male 2 when it rings personal trainer20cohort to younger adults and was mostly students, a group likely to receive manynotifications. The age ranged from 19 to 30 and averaged to 22.9 with participantscoming from a variety of ethnicities. All participants reported significant experi-ence with notifications. They used phones for at least several hours a day and atleast periodically checked notifications every hour or two. We refer to this cohortas the “younger” group.Our second cohort was recruited through the general call on the universityboard (as per above) and additionally through ads in online classifieds and printads in the newspaper. In those ads, we specified 50 years of age or older. We diddecide to use two participants (48 years old) from the general call even though theywere not 50 because (1) they were very near the cutoff, (2) they were much olderthan the oldest in the younger cohort (30), and (3) their data seemed to align morewith the older participants than the younger group. Our goal was having two agecohorts to uncover age effects, but not specifically to map them to particular ageranges. We recognize that this group is a bit younger than what is reported in theolder adults literature. Age in this cohort ranged from 48 to 74 and averaged to58.2 with an equal balance of men and women. Three participants were retired,but still did some contract work. Others had a variety of occupations from com-puter programmer to fitness instructor. Most regularly used computers as part oftheir work. Again, participants came from varied ethnicities. They reported similarlevels of phone usage to the younger group, but varied more in notification usage.A few reported only checking them once or twice a day, such as in the morning andevening. We refer to this cohort as the “older” group.In this report we refer to younger and older participants by the order they wererecruited with an added character to disambiguate the cohort, such as Py3 is partic-ipant 3, who is younger, and Po8 is participant 8 who is older. Participants in bothcohorts were remunerated with fifteen dollars for participating.2.3.3 Data CollectionWe recorded three data sets. Rough notes were taken by the interviewer, while thegroups that participants created in the priming activity were recorded in spread-sheets by the interviewer. The interview was audio recorded and transcribed ver-21batim.2.3.4 AnalysisTo analyze our data, we conducted a thematic analysis per the framework presentedby Braun and Clark [30]. We chose to allow our themes to be reflective of rela-tively small groups of participants (even three or four). We expected there to belarge variation in participants’ experiences and wanted to find subtle issues thatmight only impact small groups. We also chose to focus on a descriptive inter-pretive analysis as we were looking to identify where participants perceived theproblems to be. We also attempted to let themes bubble up in an inductive mannerbecause we felt a deductive approach might distract from our goal of understand-ing user behavior. We note that this research was conducted within a Constructivistphilosophical framework. As such we do not claim our results are the objective ex-perience of our participant. They are a rigorously created interpretation necessarilyinfluenced by the analyst’s theoretical experiences and biases.Analysis progressed in an iterative fashion through open coding. In total, thelead researcher coded every transcript with two collaborators coding three tran-scripts each as candidate codes and categories were generated. Coding was collab-orative with each researcher being able to see and react to each other’s codes. Thesewere brought to a series of research team meetings to be debated and refined intopotential themes. Based on these initial results we iteratively identified additionalfoci for analysis. Performing additional coding passes and thematic refinement asnecessary. For instance, after identifying possible concerns around a lack of controlin older participants, we conducted an additional coding pass on the younger groupto identify any related concerns. During this process the research team continuedto meet to ensure the themes continued to reflect the data and remained coherent.We focused our analysis using crystallization [58, 209, 210]. Crystallization isa process to produce knowledge about a particular phenomenon through generat-ing a deepened, complex interpretation [58]. Unlike data triangulation, which usesmultiple analysts and data sources to collectively identify the “correct” interpreta-tion of data, crystallization presupposes no fixed “truth” exists. Instead it aims todeepen our knowledge about the phenomena through different forms of represent-22ing, organizing, and analyzing details in data. For example, multiple collaborativecoders were used in our research to add complexity to the understanding of thetopic through multiple perspectives [232]. We also considered employing memberchecks, but felt that since the tasks were not related to participants life histories,they might have difficulty remembering the details of their comments months later.As is common practice in constructivist research, we reflect on how our ex-periences could influence the analysis. Throughout the study we discussed ourexperiences with notifications, and problems personalizing them. One researcherstrove to continually delete and unsubscribe from unhelpful notifications. Otherresearchers felt frustrated with how many notifications they got, but couldn’t takethe time to experiment with personalizing them more. We acknowledge these ex-periences are part of what motivates our research goals, and may have informedthe issues we focused on in our analysis.2.4 Findings and DiscussionWe describe our qualitative data with 3 main themes. Theme 1 (Resigned to No-tifications) and Theme 2 (Control and Agency) refer to participants’ responsesto questions about how they feel about and manage their notifications. Theme 3(Types of Notifications) more specifically refers to how participants completed thepriming activity and interview.2.4.1 Theme 1: Resigned to NotificationsWe wanted to get to know the participants and refine previous work around theexperience of receiving notifications. In particular, previous work has highlightedhow overwhelming notifications can be to users [13]. However, our participantsdescribed more nuanced and conflicted feelings. Most had developed strategies tomanage notifications that were good enough, if occasionally frustrating. We alsorefine the framework for the experience of notifications presented by Aranda et al.[13]; analyzing the nuanced factors involved and suggest that the importance ofinteresting or fun content in notifications may have been overstated.23Most Participants Have StrategiesThe vast majority of our participants had developed strategies they found accept-able to deal with notifications. Most younger participants (eight of ten) had mod-ified notification settings. Most used filtering controls, such as filtering a groupchat, app or uninstalling overly talkative apps. A few highlighted important appsor senders by assigning them different alert sounds. Most older participants alsoused these settings but the proportion was slightly smaller. They also focused moreon in-app controls, such as unsubscribe buttons. A few put specific focus on emailnotifications, mentioning categorization rules to organize which previously readnotifications they still needed to respond to. No participants in either group re-ported installing or using a third-party notification manager app.Despite most participants using notification settings, the majority also usedcomplementary strategies to manage notifications. The use of complementarystrategies was more common in the older cohort who tended to mention these astheir primary strategy to manage notifications (8 in 10 versus 6 in 10 for younger).Some participants used preventative strategies to prevent “bad” notifications beingsent, such as selectively installing apps or contact lists. Py10 “I don’t give myphone number to randoms, so, I know that it’s from a friend, family member, myboss, my boyfriend.” Occasionally participants would negotiate social expectationsof what was sent on each app with senders, such as text messages if it’s important,to speed response time. Py3 “If they needed a response...they would text me rightaway or they would just call me.” About half used self-discipline or hiding thephone. Py6 “I try to be more disciplined about it now. Especially when I’m atwork.” Several participants also mentioned mental systems to prioritize what theyresponded to, such as responding immediately to emails from the boss, but dealingwith other notifications later.Notifications Are a Necessary AnnoyanceParticipants rarely described notifications as purely helpful, but participants feltthey needed to exist. Py3: “They’re disruptive...and I wish that they didn’t exist.That being said, they have to exist in the field of study that I’m in.” We arguemany considered notifications more of a necessary annoyance than a major prob-24lem. However, these feelings were more negative with some older adults. Oneolder participant (Po8) had had such a negative reaction that he blocked them com-pletely. We unpack this difference more in Theme 2 but continue to examine thebroader trend below.Users Resigned to NotificationsDespite the obvious work to manage notifications, we noticed a common sentimentthat the problem was manageable and not worth fixing. This sentiment was moreprominent in the younger group, but examples came from both. Despite a num-ber of annoyances, many thought that the status quo worked ‘well enough’ not tobother investing more time into improving it.Py1: “It’s not, I wouldn’t say it’s the nuisance? It’s not too overwhelming atthis point, so it’s not a problem, but sometimes it can get annoying, obviously.”For those who committed to personalizing however, the results could be unsat-isfactory or at best bearable. As a case example, take Po17, who put a remarkableamount of effort into personalizing her phone. Po17: “You have to change any-thing that you don’t want. Which can take hours... I’ve gone from each setting to,you know, in depth to make sure I have everything set up properly.”Despite this, Po17 was not happy with the end result. Some personalizationsdidn’t do what she expected them to: “I had set it up so it would alert me, but Ididn’t set it up properly so it was alerting me about everything......it started tellingme about everybody being on their phone [laughs] I don’t need to know that.”Users: Tell Me What I Need to Respond toWe attempt to refine the framework for the experience of notifications put forwardby Aranda et al. [13] We were able to support some of their findings. Specificallythat directness, social closeness and urgency are highly valued but we expand bynoting that underlying these is whether these notifications have consequences, suchas missed appointments or ignored friends, for not responding.In line with Aranda et al., numerous participants expressed that the things theycared most about are direct messages [13]. Py6: “I don’t pay much attention tothat [group chats among friends]...unless they specifically message me.” People25directly messaging the participant generally expect or desire a response, and mightfeel rejected if they miss their message.Social closeness and context also played a large and nuanced role in how im-portant messages from real people generally were. As one might expect, typicallycloser contacts were more important to respond to because participants cared moreand didn’t want them to think they were being ignored. Py9: “These are people,I’m not close friends to, and it’s just very irrelevant.” Po13: “I’d say I have a veryclose relationship with my immediate family, that’s probably most important, andclosely behind that work, then personal.”However in other cases some younger participants felt friends and family werethe people who would understand if you didn’t respond right away. Younger par-ticipants might have felt this way due to them largely being students at a universitywith a large international student population where a lag in communication is ex-pected to family in their home country. Py5 (an international student whose parentslive in Mexico) showcases this attitude well: Py5: “I would put my parents mes-sages as the least important...I haven’t received shocking news from my parentsthat I had to address immediately.”Interested Only if Not BusyWhile utilitarian urgency and usefulness of notifications were dominant concerns,participants reacted more negatively to notifications that had uninteresting content,such as events they’re not going to. However, how much they valued more interest-ing but less urgent notifications was heavily contextual. We found that participantsgenerally had little patience for these notifications unless they weren’t busy. Po16:“Maybe for example it’s after 7 or 8 [PM]. And I am interested in somebody’scomments... At that point, very minimally those types of things might drip backinto practical.”Theme 1 DiscussionA key interesting finding in this theme is that current personalizations are beingused, but occasional frustration with notification remains. We argue for more re-search effort to refine these systems and we present approaches to accomplish this26in our recommendations.We also go back to our point that underlying valued notifications is usuallywhether there are consequences for not responding or attending to the notificationpromptly. These could be social consequences or utilitarian consequences, suchas missing an important bill or appointment. This also implied that respondingto work messages from one’s boss often took high priority. Po20: “I’d be moreimpressed or moved if it came from a boss. Only because the need to have a certainimpression made.”Lastly, we find it hard to advocate for interest as a primary determinant of theexperience of notifications. We argue that it is essentially the tie breaker: partici-pants value interesting notifications, but not at the cost of interrupting critical tasks.Referring back to Aranda et al.’s [13] framework, we found that the enjoyabilityaxis and “Fun” category seem over emphasized. The VIP, Mission Critical andNagging categories are consistent with our findings, but are mostly determined byurgency and social closeness.2.4.2 Theme 2: Control and AgencyUsers have nuanced individual differences that can change their notification needsand preferences. However, human beings share a variety of near universal basicpsychological needs that underlie these preferences [125]. In this theme we mostlyrefer to the relevant need for a feeling of mastery and competence or self-efficacy[54]. A subset of participants had particularly negative reactions to how notifica-tions interrupted and distracted them. We propose an underlying theme to explaintheir reactions: that they disrupt users’ sense of control and agency over their at-tention, infringing upon their need for agency and efficacy. We note that thesefeelings were present in both groups, but more vehemently expressed by our olderparticipants.Harm from Lack of Control for Some Older ParticipantsWe found a subset of three older participants (Po8, Po13, and Po18) who expressedharshly negative views of how notifications felt disempowering and out of theircontrol. While Po13 had a less extreme reaction than the others, all had made27significant life sacrifices in order to regain a sense of control over their attention.Po8 and Po18 had the most extreme reactions towards notifications. Both par-ticipants were older but technically skilled. Po8 was a 48 year old computer pro-grammer. Po18 was a retired 74 year old, who had done mainframe programming.Po8 had tried notifications on his laptop for a while before turning them off entirelybecause he felt like a “slave” to them. Po18 refused to upgrade to a smart-phoneto avoid mobile notifications, but still got some notifications on her iPad. Beforeagreeing to the interview she warned us she only left those on because “I justhaven’t been bothered to figure out how to remove it - so I put up with it and com-plain about it.” Importantly, these reactions occurred in spite of significant effortto personalize their notifications. Po8 used batched messages for websites that al-lowed it, and highlighted different senders by ringtones. Po18 had attended several“Facebook for seniors” classes and started unsubscribing from friends sending toomany notifications.Po18 seemed to make a point to avoid notifications because she didn’t wantto feel dependent upon them, and didn’t like how they infringed upon face to facetime. Po18: “I would say they’re an irritant...I refuse to allow myself to becomedependent upon them the way a younger person is.” She also expressed that shefelt like her attention and emotions were being manipulated by notifications, whichwas harmful to her self worth. Po18: “Sometimes I have total days where I’mnot committed, I don’t feel like doing anything...and nobody, and the box doesn’tding, then you think nobody likes me [laughs]. It’s so irrational and that I see asmanipulation.”Po8 however, focused most of his concern on how notifications made him feellike he was no longer in control of his time. Po8: “I look at notifications differentthan other people. I use social media at your convenience. I don’t react to otherpeople’s agenda.” Po8 describes how the expectations of communication havechanged to be more and more immediate. This change may have felt disempow-ering because it impacted his ability to focus. Reduced focus may have made himfeel unable to complete work effectively and impacted his self-efficacy. This lackof control is referenced in his rejection and removal of notifications “In principleI don’t want to react to notifications. We became slaves to our phones.” However,this is not to say he doesn’t value notifications, just that he doesn’t agree to how28they are currently delivered. “I’d agree to notifications once a day or twice a day,but once I say yes then I’m at their mercy.”Po13, a 51 year older historian working for the government, mirrored the ex-periences we noted in Theme 1 and seemed very resigned to the situation. To him,notifications were a fact of life due to his job. However, he felt that he could notstand to be so constantly connected, and forced himself to leave the computer offon Sundays. “They are certainly essential to my daily life... but... I really dislikethe thought of being so connected and...I always try to take a break from thingson Sunday.” This, however, causes some anxiety about missing important worknotifications.Concerns of Control Are Shared in Both CohortsWe found that these concerns extended to a small subgroup of younger participantsand two more older adults. However, these participants did not make the life sacri-fices to address them. The younger group expressed less negativity with this issue,but the underlying sentiment was the same. Py11 made comments that mirrored themore extreme reactions of Po8 and Po18, but laughed about it as a funny annoy-ance. Py11: “I feel like a slave to notifications at times [laughs]...but, at the sametime, I’m grateful for notifications because...notifications help communication feelcurrent.”Py9 also described how she didn’t like how the new version of the outlookemail client took control away from her and decided to highlight certain emailsas important for her. Py9: “They’re choosing emails that they think are moreimportant...but I consider all my emails kind of equally important.” Similarly someolder adults felt annoyed because the alert sound made them pay attention to thingsthey didn’t want to. Po4: “Weather apps wouldn’t be [so] bad if [they were]silent...What I don’t like is the beepy sound.”More commonly, however, a few participants in both groups felt that they didn’twant to be on their phones as much as they were, but the demands of work andtheir social circles forced them to. Some even said that they viewed the instantcommunication demanded of them as unhealthy. Py10: “I think my usage hasspiked because of my work...and it’s not healthy.”29Desire for Partial ControlThis desire for a sense of control has significant impacts on how participants wantedto personalize their devices. All but one participant in both groups expressed thatthey would like some control in validating or implementing personalizations evenif a system made it more automatic. Py6: “I would say that I would still like tohave fifty percent control of overseeing what’s been filtered through.”Some participants were strongly against total system control Po15: “Probablynot. I wouldn’t intuitively respond to someone else organizing me” but reactedmore positively when we clarified that they could have control to approve sugges-tions Po15: “Well, suggestions is okay, not so bad.”Interestingly, while we were assessing desired personalizations, we noticedolder participants expressed less interest for many basic changes, such as filter-ing. We posit that our suggestions were possibly threatening their sense of controlof their attention and knowledge of what they receive. Many older participantswanted to know everything they get so they got to make the decision to respond bythemselves. Po13: “To my mind it’s either spam or it isn’t. If it isn’t spam... then Iwant to decide.”A tension with this is that participants were not always conscious of how manynotifications they get, so they would not always have the information they neededto make informed decisions. We noticed participants occasionally being surprisedafter reading through our example notifications and realizing they receive morethan they thought. Po19: “boy I get a lot of notifications [laughs] I can’t ignorethose.” Participants also occasionally re-organized their categorizations during theinterview. With even just a bit of probing participants would find edge case notifi-cations they hadn’t thought about that might require a different response.Theme 2 DiscussionWe might suggest a few reasons for why younger participants’ perceptions of con-trol were less negative. One component may be physiological. Younger partici-pants have been shown to be faster at recovering from interruption [46]. Experiencemay also play a part: they will have had more time to develop coping strategies.Younger participants may also not have experienced a time when communication30was less instant, which would make it difficult to imagine things being different.We also might conjecture that some participants who were not conscious of howmany notifications they receive might feel more annoyed if they reflected on itmore deeply, though this would need to be investigated in future work.A key observation to explain the negative feelings in the older group is thatnotification settings are also often very limited. Prior notification tools generallyonly allowed users to express a rough idea of “what” they want to be notified about,such as by disabling apps or groups, “how” they can be notified about it, such asthrough sound or vibration, and occasionally some contact specific requests, suchas different ring tones. However, users desiring to express more subtle preferencesare dis-empowered. Very rarely can users express “when” they want to be no-tified about something, such as a batch every few hours or only in the evening,“why” something might be important enough to overrule these settings or “where”they would want to receive notifications, such as only while at home. Lacking the“when” control in particular is problematic as checking email less frequently mayreduce stress [134]. To some extent users can adapt their usage of the device to im-plement these desires, such as leaving the phone muted and checking infrequentlybut explicit controls should be available.We argue that any project or technology attempting to improve notification sys-tems must pay special attention to providing users a sense of agency and controlover their attention. We note two important design areas that will be impacted:which personalizations are provided and how they are applied. Our recommen-dations section will provide some preliminary ideas on how these systems can beimproved to provide a better sense of agency.2.4.3 Theme 3: Complex Categories of NotificationsPrevious work has used simple filtering rules, such as filtering notification thatcontain “candy” and “crush” in the content [159] to apply personalizations to no-tifications, however, they had a low adoption rate [159]. We speculate this wasbecause they might be confusing to non-technical users. It is not immediately clearhow many and which notifications the rules will filter. Thus, in this theme wepresent an overview of the common categories of notifications perceived by users,31Table 2.2: We assigned participants’ groupings to the categories on the leftthrough thematic analysis. Counts and example labels are provided foryounger (N=10) and older (N=9) participants. One older participant wasexcluded (Po8) because he had already turned off notifications.Our CodedCategoryYounger Labels # ofYoungerOlder Labels # ofOlderDay to DayReminders“Util” “Re-minders”8 “Practical” “Calen-dar”5Familyand (close)Friends Mes-saging“Friends/FamilyConnections”5 “Family” “Text” 4Work “Business” 7 “Employment” 6General Mes-saging“Facebook Mes-senger” “Texts”6 “Personal” “Text” 5Social MediaActivity“Instagram”“Facebook Posts”6 “Annoying” “Face-book”4System “Phone System” 6 “System” 1Email “Email” 7 “Email” 3and how the nuanced contexts surrounding these categories impact personalizationbehavior.In the priming activity, participants categorized and discussed a set of 75 ex-ample notifications. Through thematic analysis of that discussion we produced ourown categorization of notifications seen in Table 2.2. These are broad categoriesthat tended to underlie the nuanced and contextual groups participants created. Forexample, a group labelled “text” could be in family messaging and/or general mes-saging depending on who the participant texted with.We noticed that a greater proportion of younger participants focused on nam-ing lots of individual apps, such as Instagram, Youtube, Reddit, while older partic-ipants focused more on describing what aspect of their life the notification relatedto, such as work, family, personal. In particular, the more app based categories,such as system and email, were much less common in the older group. Older par-ticipants also had more variation in their grouping schemes, and there were a fewcompletely unique categorizations, such as participant 16 who simply categorizedthem by usefulness as 1. Practical 2. Annoying or 3. Don’t care. However, even32in the younger group there was large variation in their groupings, and the majoritywere not just a listing by app name.The construction of these categories has several implications for notificationpersonalization. For the several system-generated groups like reminders and socialmedia activity, participants’ reactions to them were mostly based on urgency. If anotification from these groups did not provide immediate utilitarian value, such asreminding them of an important forgotten meeting, debugging a wifi issue, a disas-ter alert, participants generally did not like or want them. A number of participantsalso expressed that batching or silent delivery for the less urgent of these would benice. Py10: “Stuff that I’ll see on my news feed I don’t need a notification for.”Another important observation was the distinction between several types ofmessaging including family, personal and work messaging. Similar to the VIPchannel presented in Aranda et al.’s model [13], messages from family and closefriends are often considered as separate and more important than other messages.However, this did not necessarily mean that responding to them was urgent, justthat they needed to see them to decide whether or not to respond. Some friendswere considered closer than others and edged into more general messaging. Workmessages from bosses also tended to be highly important.Finally, we noticed that many, especially younger, participants also includeda catch-all group for the email they received. However, email is often an enablerfor other groups in our categorization. Many apps send both a mobile notificationand email for the same event, such as a comment response. Despite this, we cre-ate a catch-all email group because email often seemed to be prioritized in a verydifferent way. Participants expressed that they used email for more long form com-munication where immediate responses were not needed. Despite the lack of timeurgency, they wanted them quickly because they were being used to discuss moreformal topics, such as work or school. Py12: “If I’m waiting for an email backfrom a job...Those all tend to be emails.”Theme 3 DiscussionWe stress that these results are some broad suggestions for common groups anddesires in personalized notification delivery. While there are general trends for33how participants want to respond to each group, there is almost always significantvariation. A social media activity notification could be about some emergency im-portant enough to override users’ original preferences not to be interrupted. Workmight have an emergency worthy of a notification over the weekend. Detecting andidentifying such nuanced circumstances remains an open and difficult problem.An interesting age related difference is the decreased presence of email groupsfrom older participants. Older participants may not have included email groupsbecause they viewed email as a more universal form of communication, whereasyounger participants segmented it from day to day messaging; however this wouldneed to be confirmed in future work.These results prompt us to review how users negotiate what they are notifiedabout. There should be transparency in what notifications are affected by personal-izations. However, a user often agrees to be notified implicitly as part of installingan app. During setup, apps will request permissions from the user, such as maps,or contacts, and notifications may be included, or simply on by default. Usersmay feel pressured to accept these permissions, as the app may not work other-wise. However, these systems rarely provide context for why the app needs thosepermissions or what they will do with them. With respect to notifications, we aremissing details such as what will they notify you about, how frequently, and inwhat way? In addition, each app having its own settings does not line up withhow, especially older participants, view notifications. The separate settings pan-els for each app necessitates duplicate setup for say, similar chat or social mediaapplications.Lastly, we note that users’ perceptions about what notifications they get andwhich they want evolves over time. Asking yes/no at installation is not an in-formed decision. Thus, the system asks for the users’ forgiveness of unwantednotifications rather than seeking permission. Not to mention that some bad actorsexplicitly attempt to make it hard to remove unwanted notifications, such as tinyunsubscribe links in email ads. As we noted in Theme 2, many participants alsoshy away from filtering improper notifications. Missing important messages wasseen as a significant risk. While the cost for fixing improper notifications falls dis-proportionately on the shoulders of users, there is still a cost to developers. Theannoyance caused by these notifications is often cited as a reason to uninstall apps34[13].2.5 RecommendationsIn contrast to previous work, most of our participants expressed they were notoverwhelmed by notifications but had resigned themselves to a “good enough” so-lution. However, a classic goal of HCI research is taking frustrating but somewhatusable systems and refining them to be more effective. Thus we present two designdirections to improve notifications.2.5.1 Standardize Personalization above the App LevelThere are numerous apps that send notifications installed on every phone, oftenwith overlapping goals and similar notifications sets, such as chat messages. Eachapp requires effort to learn how to personalize, figuring out where the settingspanel is and how to use it. We argue that designers need to rethink how notificationpersonalizations are applied and what role the app plays in personalization. Specif-ically, that personalization needs to be standardized across apps and applied at ahigher level.Many users engage in a lot of personalization, but still experience frustrationwith notifications. Even if most apps provide proper settings, one or two missedtalkative apps is enough to be frustrating. Herein lies the conflict: most apps needto be properly tailored to the user, but that requires a lot of duplicate setup.We also note that the problem is unlikely to be solved by notification guide-lines. Each app developer has differing levels of skill, expertise and goals. It iscommon for a few bad actors (or naive developers) to try and drive traffic to theirsite with extra notifications [13]. A first step would be to pull together the noti-fication settings of similar apps, such as chat or social media. These more globalsettings should still allow in-app personalization, but also quick and general set-tings for common needs. Ideally these categories would be similar to those wedescribe in Theme 3. However, we acknowledge our categorization may requirenuanced and difficult to measure contextual preferences. For example, identify-ing socially close contacts, time urgent notifications, or identifying open periods todeliver less urgent content may involve difficult machine learning.352.5.2 Design for Ongoing Machine-Human PartnershipWe argue these settings should be maintained with an ongoing machine-humanpartnership. Participants may not have enough information to make informed deci-sions about their notifications during setup. Especially if those personalizations donot do what users expect. Similarly, participants were occasionally not consciousof how many notifications they were receiving. Lacking awareness of what per-sonalizations did and what notifications users received would make it even moredifficult to make an informed decision about which personalizations they need.We also note that apps are constantly being updated and changed. For exam-ple, very recent major changes to the notification system in Android 8 redesignedthe notification experience [98]. Grouping notifications from the same app andhighlighting direct messages are good first steps to speeding response time. Appdevelopers may now offer new settings, such as to customize notification shade.While these updates are positive, not all users will know about them, or want tore-personalize.So how do we maintain these settings? A fully adaptive interface is unlikelyto be received well. Many, especially older participants, will reject such a systembecause it takes control of their notifications away from them. There is a need foran ongoing machine-human partnership. The user must be in the driver seat andhave the final say in any personalization, but they should not have to do all thework. The system should highlight key settings and bring them to the attentionof the user in unobtrusive ways. Such a system would be able to respond to newsettings, as well as refine them as edge cases are uncovered and users get a betteridea of what they want.Some systems have already made attempts to implement such a paradigm [159].However, more care needs to be taken in how the suggestions are presented tousers. Which categories of notifications will be affected by a suggested personal-ization should be more clearly explained and perhaps be related to our categoriza-tion scheme. Further, we argue that attempts need to be made to reduce perceivedrisk of accepting suggestions, perhaps through offering trial periods for differentsuggestions. For example, “we notice you don’t click on this type of notificationvery much, do you want to try leaving it on silent for a week and we’ll check back36in later?”Design iteration is also needed to evaluate what new settings are feasible be-yond repackaging currently implemented settings. Users need to be able to specifymore subtle wants and needs for their notifications. For example, they might wantto specify when and where they are notified and identify higher closeness contacts,but implementation is tricky. Social media analysis may help identify close con-tacts but privacy remains an issue [84]. Other possibilities may require excessiveinput or intrusive tracking of location or audio.2.6 LimitationsWe focused on providing deep reflections on a smaller group of participants to aiddesign ideation, rather than a more generalized sample. It would be valuable toevaluate a larger sample of the population to confirm the extent of concerns withagency. In particular, we suspect the older group may have sub-group differencesbetween retirees and not, but we did not have enough data to confirm this. A seriesof online surveys or more targeted recruiting might help us tackle this issue. Thesesurveys should also include investigating what factors besides age could influencethese feelings such as personality traits or cultural background. Studies could alsoemploy more holistic observation to get a broader understanding of notificationcategorization.2.7 ConclusionThis chapter investigated how younger and older users perceive and think aboutnotifications. Through thematic analysis of semi-structured interviews we refinedour ideas of users’ experiences with notifications. Contrary to previous research,many users seem resigned to the current situation rather than overwhelmed. Partic-ipants’ experiences with notifications also seemed to be far more strongly linked tourgency, directness, and social closeness than enjoyment, though context remainsa nuanced and difficult issue. We also identified that some older users especiallydesire a feeling of agency over their attention that is hard to achieve with currentnotification systems, which some younger users share. Lastly, we described com-mon groupings of notifications that users perceive, and we noted ways this could37contribute to personalization systems. These findings led us to advocate that thecurrent system of personalizations being provided by individual applications needsto be redesigned into a more unified solution. We also argued that an ongoingmachine-human partnership will be critical to refining notification personalization.Designers should also look for ways to empower users to specify more nuanceddesires for notifications such as when and where they want them to be delivered,as well as identify close social contacts.In the next chapter, we will complement this study by exploring how users’need for control and agency over notifications might be addressed through the de-sign of notification management tools. Then, in Chapter 4 we will develop a novelnotification management tool that supports and encourages users to control andmanage notifications.38Chapter 3Designing NotificationManagement Tools to Support aSense of Control (Study 2)In this chapter, we present Study 2, a concept exploration study [77] based on in-terviews with 30 participants that we conducted as a followup to Study 1 (Chapter2). Here, we focus on exploring how the need for control over attention uncoveredin Chapter 2 impacts the design of notification management systems, and we ex-plore how notification management tools could better enable proactive notificationmanagement. As part of this effort, we characterize four design axes for how no-tification management systems could involve users in personalization and presentfive design concepts exploring different ways users could proactively manage noti-fications. The insights gained from our concept exploration study evaluating thesedesign concepts strongly directed the design of our notification management toolin Chapter 4 by pushing us towards exploring reflective visualizations in greaterdepth.3.1 IntroductionUbiquitous mobile devices are more and more frequently interrupting users to pro-vide potentially time-sensitive information, such as email, medication reminders,39emergency alerts, or device status [7, 45, 47, 68, 109, 122, 153, 170, 191, 214,255, 263]. Mobile operating systems provide a unified API for apps to deliverthese interruptions, often called notifications, in a consistent style and estimatessuggest users may receive an average of 40 or more notifications a day [191, 214].The impact of receiving frequent notifications has been studied and shown to causeusers stress, hurt productivity, and increase inattention [72, 134, 135]. Notificationreceptivity research has described which notifications users are willing to receive,despite these costs, in different contexts and found that users have complex, in-dividual preferences for notification delivery [13, 16, 53, 56, 160]. For example,users might generally prefer to receive notifications from socially close, personalcontacts but have various exceptions, such as when they are in a meeting or beforea key deadline [13]. A common approach to ensuring that notifications match auser’s preferences is through notification personalization, which is the focus of thisresearch. Personalization systems allow users to define preferences for notification,such as how or when to be notified, but research suggests many users avoid person-alizing due to barriers like time, effort, and expertise [13, 16, 34, 191, 214, 250].To reduce the barriers to notification personalization, adaptive context-awarenotification management systems have been extensively researched [10, 132, 157,188, 200, 236]. Context-aware notification management tools analyze various sen-sor data sources, such as mobile phone GPS location, to identify a user’s cur-rent context, such as their task, location, and ability to respond to a notification[132, 157, 167]. Combined with other data sources, such as the subject of the no-tification, sender of the notification, or preferences of the user, these systems canautomatically decide whether to deliver or delay the notification to a more appro-priate time [69, 158, 167, 250]. However, context-aware notification systems havea number of substantial drawbacks. For example, accuracy of prediction needs tobe high to avoid missing important notifications [132, 159] but gathering sufficientsensor data to make accurate predictions can hurt battery life or involve intrusivetracking [10, 132, 157, 188, 200, 236].While many of these concerns have been addressed in prior research, a studyon older versus younger users of notifications raised a key concern of how usersalso desire a greater sense of control over their notifications (Chapter 2). Thisneed for control was slightly more pronounced amongst older users, who are more40negatively impacted by distraction [46, 137], and more automatic context-awarenotification management systems might worsen this experience. There are sometechniques to request feedback from users in context-aware notification manage-ment systems, such as reinforcement learning [188, 236] or personalization sug-gestions [159]. However, these techniques have their own design trade-offs, aretypically evaluated in isolation or in comparison to current phone settings, andmay still reduce user control [159, 188]. We performed a more holistic explorationof how users might manage notifications to reveal additional, promising directionsfor design.Our goal was therefore to explore the impact of users’ desire for control on thedesign of notification management tools to identify how users could more proac-tively manage their notifications. The core research questions of our study were:1. Which notification management design elements and high-level design conceptshelp users feel like they have expressed their needs effectively? 2. Which noti-fication management design elements and high-level design concepts help usersrefine their knowledge and understanding of which notification personalizationsthey want? Our second research question was added because we were concernedless technical users might face higher barriers communicating their preferences inthese systems and need additional support [18]. To answer these questions, weconducted three distinct pieces of work that form the contributions of this research.Our first piece of work was to characterize a set of four design axes for notifica-tion management systems that describe how users could be more directly involvedin notification management. The intent was for these axes to help us reason aboutwhat unexplored designs might better provide a sense of control. This characteri-zation was informed by extensive literature review, analysis of current notificationmanagement tools, iterative discussion amongst our research team, and a conceptexploration study that we elaborate on further below. The design axes are: 1. Tar-getable Notifications (the types of notifications users can personalize), 2. PossibleActions (the actions that can be taken on the notifications the user targets for per-sonalization), 3. Decision Support (the design elements the tool uses to help theuser decide which personalizations they need), and 4. Degree of Automation (thedegree to which the system provides design elements to automatically manage no-tifications).41Our second piece of work was to design and partially implement five designconcepts exploring different ways users could proactively manage notifications.These design concepts use design elements touching different areas of our designaxes to explore the perceived uses of design elements in notification managementtools. Our design process was inspired by Research Through Design [262] andprimarily focused on low-fidelity prototyping to allow us to probe user reactionstowards high-level notification management concepts and the design elements usedto communicate preferences to the system. Together with the design axes, we sug-gest these design concepts can act as exemplars to provide notification manage-ment tool designers with inspiration for how they might enable users to proactivelymanage notifications in future tools. The combination of design axes and designconcepts form the first major contribution of this work.Our third piece of work and last contribution was an empirical evaluation ofthese design concepts through a concept exploration study with 30 participantsacross a wide range of ages and occupations. During one-hour long semi-structuredinterviews we elicited user reactions to the five design concepts. Based on these in-terviews we identified the following key results: 1. Different high-level notificationmanagement design concepts serve separate but complementary roles in notifica-tion management when presented with appropriate design elements. For example,manual logical rules were effective for initial setup, while targeted suggestionswere effective for refinement. 2. Users expressed strong curiosity towards receiv-ing more decision support in future notification management systems. 3. Manyusers desire more control over their notifications, reinforcing our earlier work inChapter 2 that many users strongly value a sense of control over notifications. Fur-ther, this sense of control was not well served by heavily automated deferment andfiltering approaches to notification management tools that are similar to very recentresearch [40, 144].The two primary contributions of the study are as follows:• A set of five notification management design concepts exploring differentways users can proactively manage notifications. These concepts can serveas exemplars for how users could more proactively manage their notificationsin the future.42• The results of our concept exploration study exploring the uses of variousnotification management design elements in our design concepts throughsemi-structured interviews. These results identified several promising ar-eas of design, such as reflective visualizations of notification use, that mightprovide a better sense of control for users.A secondary contribution of the study is:• A characterization of four design axes for how users can be more directlyinvolved in notification management, which should help designers reasonabout what design elements might be explored in future notification man-agement systems.3.2 Related WorkWe focus our discussion of related work on designing for interruptions, previ-ous approaches to notification management, and emerging reflective approachesto managing notifications.3.2.1 Notifications and InterruptionsSoftware interruptions are a common approach to provide users with potentiallytime-sensitive information through various devices, but come at the cost of dis-tracting users [24, 26, 36, 38, 111, 153, 155, 174, 200, 220]. Literature describeshow users have a “primary task” that interruptions distract them from which theywill return to after responding [70, 153, 183, 200]. The term “notification” is usedto describe a type of interruption broadly used in both the literature and popularpress. Recent mobile devices have created a singular, unified system that all appli-cations deliver their software interruptions (called notifications) through called a“notification centre” [65, 157]. The impact of various alert design decisions, suchas modality, size, duration, or loudness, has been well studied. For example, ithas been suggested that less important notifications should reduce cognitive loadby appearing passively in the periphery, while notifications with important contentshould maximize readability and noticeability through sounds or vibration [2, 11,68, 70, 85, 94, 114, 118, 149, 179, 190, 245, 261, 263]. Studies have also identified43various moments, such as when switching tasks, that are less disruptive to interruptfor both traditional desktop and mobile interaction [37, 42, 183, 235, 241, 244]and explored which notifications users value enough to merit an immediate inter-ruption [2, 13, 69, 70, 157, 158, 160, 188, 238]. Numerous systems then try to deferdelivery of most interruptions to these optimal moments to enhance primary taskrecovery speed [8, 10, 16, 25, 70, 105, 105, 111, 154, 160, 196]. However, somechallenges have been noticed with these systems such as deferred notifications ac-cumulating on mobile devices, and a delicate trade-off between reducing interrup-tion and maximizing awareness of important information [105, 159, 188, 236, 239].Notifications have been noted as a significant source of distraction, so recentresearch has attempted to quantify the scale of the problem [112, 152, 191, 214].Estimates vary, but the consensus appears to indicate that numbers are large andreflect a substantial problem. Qualitative perceptions of notifications have beenstudied to identify what users value in their notifications [13, 139, 160]. For ex-ample, the sender of the notification [39, 139], the device delivering it [24, 45, 83,149, 220, 248, 261], the app sending it [172], the task it’s related to [112, 157],and the content [69, 158, 216] all play important roles. Aranda et al. synthesizedthis work into a model of how users experience notifications [13]. They presenttwo axes, enjoyment and urgency, that determine the experience of receiving a no-tification. They propose several categories of notifications based on these axes:VIP (urgent, enjoyable messages from close family and friends), Mission Critical(painful, urgent work), Nagging (painful, non-urgent work), and Fun (non-urgent,enjoyable content). Our work in Chapter 2 extended this result and suggested thatthe importance of the enjoyment axis may be overstated; that it acts more as a tiebreaker between equally urgent notifications. However, both studies found thatusers tend to value urgent notifications, particularly from socially close contacts.We used this framework to help ground our analysis in prior work.3.2.2 Notification Personalization and ManagementWithin the personalization literature, there are known strategies that users em-ploy to manage notifications such that they are more useful and less distracting[13, 131, 191, 237]. Some users focus on less technical, temporary approaches,44such as turning the device off while working, while others are more preventative,such as carefully pruning which applications are installed and allowed to send no-tifications [13, 26, 39, 129, 253]. Personalizing notification preferences with thesettings in the default notification centre is more rare, but can be useful for some[244]. Numerous “notification manager” apps have also been developed for com-mercial use to make manual personalization easier, such as by setting up custom“do not disturb” hours where notifications are not delivered [12, 78, 219, 222]. Themanual personalization tools presented in literature tend to focus on making thesesettings more expressive or reactive to context, providing flexible personalizationtoolboxes to explore [16, 53, 91, 195, 252]. The granularity of the personalizationsusers were allowed to create in these tools varied along several axes. For exam-ple, some work has tried deferring all notifications to more opportune times, whileother work focused on letting users create nuanced contextual or content based per-sonalizations [12, 16, 105, 158]. This large variety of possible controls informedour “Targetable Notifications,” and “Possible Actions” design axes. In particularwe tease apart the way tools varied between broader, sweeping personalization andvery specific, targeted personalization in the “Targetable Notifications” axis.Users may avoid using these settings due to various barriers, such as effort topersonalize or lack of experience with current personalization systems, but can ex-perience triggers that prompt them to push through those barriers [18, 244]. Adap-tive interfaces update settings for users automatically to reduce those costs, and arevery common in modern interruptibility research [79, 111, 128, 144, 157, 188, 236,238]. Adaptive notification management systems typically defer or filter a user’snotifications by using ML to predict which notifications users are receptive towardsreceiving despite the distraction they cause [132, 157, 238]. For example, sensordata from ubiquitous devices can reveal a user’s context [75, 130, 141, 185, 225],task [76], or personal preferences [144, 145, 159], which can be used to predictwhich notifications are appropriate [157, 188, 236, 238].Research has also developed interfaces with different levels of adaptability.“Mixed-initiative” personalization, also called “machine-human partnerships” bal-ance user control and effort. They involve a dialog where the computer suggestspersonalizations to the user, who chooses which to apply [32, 33, 35, 111, 229].Designing these systems can be challenging due to inherent design trade-offs, es-45pecially within the domain of notifications. For example, the benefit of more fre-quent suggestions must be weighed against the distraction they cause comparedto automatic adaptation. Recent work on notifications has suggested these de-signs should be explored further [144] to extend earlier systems that identify andsuggest filtering rules for notifications, similar to classic email filtering systems[159]. Meanwhile, industry applications have explored various kinds of notifica-tion personalization. The Google News App shows a more adapative approach byautomatically identifying notification-worthy stories with a slider for notificationfrequency. iOS15 provides a more mixed-initiative “focus mode” for notificationsthat, when turned on, suggests a customizable “focus” to the user that changes hownotifications are delivered based on machine learning identified context [96]. Vari-ation between manual and adaptive personalization was key to informing one ofour core design axes, “Degree of Automation”.3.2.3 Reflection on NotificationsWe experience notifications throughout our day-to-day lives, which often distractus from key tasks that we need to return to [28]. This distraction makes it chal-lenging to gauge the full impact of notifications on our lives and make personaliza-tion decisions. Taking a cue from personal informatics research, several researchprojects have implemented apps that track, visualize and encourage them to reflecton their notification usage [28, 160, 198, 212, 249]. These more reflective notifica-tion management tools were core to the creation of our “Decision Support” designaxis. However, a concern with similar reflective tools in personal informatics re-search is whether they do actually accomplish their inherent goal of helping usersadopt more optimal behavior, such as turning the phone to silent while working,by reflecting on their personal behavior [240]. Seeking to build off this body ofresearch, our designs sought to incorporate this reflection directly into the notifi-cation personalization process. We hoped this might make the insights generatedeasier for users to utilize. We will later show in this chapter that users are very curi-ous about these reflective design elements during notification management. Chap-ter 4 then demonstrates that a properly designed notification reflection system canindeed encourage users to manage their notifications.463.3 Design Axes for Involving Users in NotificationManagementDesign spaces are theoretical tools that allow researchers to describe the potentialdifferences between designs through sets of design axes [147]. Design space analy-sis allows us to systematically analyze trade-offs between designs and reason aboutwhich unexplored design elements and approaches could compete with or improveexisting tools. To support our design exploration into how users could gain a bettersense of control over their notifications, we characterized four design axes for howusers could proactively manage notifications as shown in Table 3.1.3.3.1 Process to create the design axesWhile we do not present these design axes as a full design space for notificationmanagement, our process to create our design axes was inspired by several differ-ent pieces of design space research and involved an extensive academic literaturereview [82, 99, 153, 226, 243, 258]. We generated a corpus of potentially relevantpapers through approximately a dozen iterative searches involving permutationsof terms like notifications, interruptions, or personalization on the ACM DL andGoogle Scholar. We filtered these searches to target conference papers and jour-nal articles in SIGCHI and related venues such as MobileHCI, ToCHI, UbiComp,UIST and CSCW. For each search, we manually skimmed the titles and abstracts ofthe first 200 results and selected papers that met the following criteria: 1. Made adesign contribution to help users manage notifications. 2. Involved user interactionsomewhere within a system that manages notifications. This search resulted in theselection of 32 papers. We read each paper in depth and snowballed our sample byfollowing their references. In total over 40 notification management papers wereread and analyzed [3, 11, 16, 24, 28, 37, 42, 44, 53, 66, 68, 79, 80, 83, 90, 91, 100,101, 104, 105, 108, 111, 114, 126, 128, 129, 144, 149, 158–160, 164, 171, 179,182, 187, 189, 190, 192, 193, 195, 213, 215, 220, 241, 245, 247–252, 261].We also characterized numerous current notification management tools fromindustry (the top 20 results from a search for notification managers, many havingover one hundred thousand installs), and the default notification settings availableon android, iOS and many common social media or messaging applications, such47as Facebook, Twitter, Messenger or Slack [78, 96, 98, 222]. Features, settings, in-teraction metaphors, and design concepts from both industry and academic systemswere categorized and analyzed in a lightweight process similar to thematic analy-sis [30]. This analysis included reviewing design spaces in other somewhat similardomains like ambient awareness displays or availability-sharing systems for inspi-ration about possible design axes [82, 99, 197, 258]. At a series of our researchgroup meetings over the course of the research, initial categories of notificationmanagement systems were iteratively refined into a series of four relevant axes ofdesign. These axes are described below and we provide examples of where vari-ous notification management designs from research and industry are placed withinthese design axes in Table 3.2.48Table 3.1: Our four main axes for the design of proactive notification man-agement tools. The Targetable Notifications, and Possible Actions axestogether describe the base unit of personalization available to the user.Degree of Automation and Decision Support describe the different de-sign elements used to support user personalization.493.3.2 Final Design Axes• Targetable Notifications: Some notification management designs, such asdo-not-disturb mode, may choose to broadly aggregate notifications frommany senders about different subjects into one easy-to-apply personalization[12]. Other tools support more targeted personalization on individual cate-gories of notification. This axis provides examples of different ways specifictypes of notifications can be targeted for personalization, such as by sender[139], context [53], content [69, 158], and which events, such as receivingfriend requests, should generate notifications.• Possible Actions: The possible actions that can be taken on the targeted noti-fications such as changing the delivery timing through various deferment orfiltering strategies [40, 41, 105, 144, 159]. In addition, notification displaycan be customized around a variety of factors such as the sound used to alertthe user of a new notification [53], changing display order [145] or the visualproperties of the notification when read by the user, such as colour or size[164].• Decision Support: Gauging the impact of notification personalizations canbe challenging because we typically interact with notifications while focusedon other tasks. This axis characterizes a variety of common ways notificationmanagement tools may support users to reflect on their notifications and de-cide what personalizations they need. These include visualizations of usagedata [249], overview statistics [28], example notifications, and suggestionrationales [33].• Degree of Automation: Notification management systems will vary on howand when to engage users in setting notification preferences. More manualsystems might take no action until a user manually changes settings [16].However, more automated systems might seek to engage the user with sug-gestions or reminders to personalize or even manage notifications automat-ically, such as an adaptive system deferring non-urgent notifications [159].Table 3.1 gives examples of various design elements, such as logical rulesfor manual personalization, that are particularly common in these different50approaches to automating notification management.We present these axes not as an authoritative design space, but as a startingpoint for designers who may need to create notification management tools that in-volve users somewhere in personalization. In practice, a common activity for manymobile app developers is deciding which notifications their app should generate,how users can opt out of them, and how much they should rely on the operatingsystem to control notifications for them. To aid this process, in Table 3.1, we haveroughly sorted all examples within the targetable notifications axis from left to rightbased on relative importance of sender, such as family or friends [13, 139, 160],content, such as in messages sent directly to the user [13], and interruptible con-texts such as in meetings or checking messages [37, 40, 41]. We suggest that anearly step in the design process for these developers should be going through thesecategories and analyzing how many notifications their app is sending users thatwould be considered particularly important. As we will describe in our discussion,if there are relatively few important notifications, they should make personalizationbe low effort, likely through more adaptive management. Conversely, if users willreceive many important notifications, developers likely should include one or moreelements of decision support.We note that while these design axes are primarily independent of each other,there are a few potential intersections where the choices in one design axis in-fluence the possible options within the others. Most notably, extremely adaptivedesigns may not allow users to target notifications, such as by sender, given thatthe system takes over full control. In theory, the only user-controlled personal-ization that makes sense in fully adaptive systems is deciding whether to turn onthe automation, which could be informed by decision support. However, in prac-tice, users of adaptive systems usually have at least some limited input, such as theoption to select “see fewer notifications like this” when they swipe away a newsstory.51Table 3.2: Examples of where notification management designs from re-search and industry are placed within our design axes.Design Axis TargetableNotificationsPossibleActionsDecisionSupportDegree ofAutomationScheduled do-not-disturbmode (fromAndroid OS)Broadly bysender and con-textChangedisplayNone ManualFacebook Set-tingsPrecisely bysender, content,and eventChangedisplayNone ManualGoogle NewsSettingsBroadly bycontentChangedisplayNone AdaptiveBlock Apps [12] Broadly bycontent andcontextChangedisplayOverviewStatisticsManualNotificationCleaner [78]Broadly bycontent andcontextChangedisplayNone ManualPrefminer[159] Broadly bysender, andcontentChangedisplayNone MixedSnooze[250] Precisely bysender, content,and contextChangedeliverytimingNone ManualDashboard[249] None None Usage Vi-sualizationManual3.4 Design ConceptsWe present five design concepts that explore how users could proactively managenotifications in the future in Figures 3.2 and 3.3.3.4.1 Process to Create Design ConceptsOur design process drew heavily on our design axes and was highly iterative. Earlylow fidelity prototypes, such as those shown in Figure 3.1, explored a variety ofideas we brainstormed to refine prior notification management design tools with52new or under-explored design elements from our design axes. These refinementswere intended to address concerns we had seen in prior literature or noticed in ourown use of notifications. For example, the top left image in Figure 3.1 shows adesign sketch exploring how a notification management tool might be tailored to-wards less technical or older users who are more negatively impacted by distraction[46]. In this sketch, the system suggesting personalizations to the user describes thegoal of the suggestion with a trial period for accepting it to help less technical usersunderstand what it does without committing to it. We also brainstormed conceptsexploring how design elements that align to under-explored areas of our designaxes could be included in future notification management tools. For example, thebottom image of Figure 3.1 shows a visualization-based concept for personalizingnotifications that eventually became our reflective visualization design concept.We limited ourselves to five design concepts because we learned while design-ing the final study that we could only practically discuss five to six notificationmanagement design concepts in a reasonable-length study session. Five conceptswas enough to explore a wide variety of ways to target notifications and provide de-cision support while varying degree of automation. Prior work has often exploreddifferent ways of incorporating machine support that align to different degrees ofautomation [144, 159, 250], so we used three design concepts, Notification Rules,Adaptive Examples, and Suggestions to explore how these approaches could berefined with new design elements, such as more precise targeting of notificationsin automated systems. We then used the final two design concepts, Crowd Ideasand Reflective Visualizations, to explore new design concepts aligned to under-explored areas of the design space. For example, we explored different waysof adding decision support within notification management. Table 3.3 describeswhere our design concepts are placed within our design axes.To more deeply explore how the choice of design elements would impact theperceived uses of a design concept, we created several variants of our suggestionsconcept, which leverages mixed-initiative suggestions. Each variant explored theconcept of personalization suggestions framed with different design elements, suchas personalization by logical rule or example. We chose the mixed-initiative sug-gestion concept to explore these variations because it was a relatively simple singlescreen interface which made it easy for users to understand the differences between53Table 3.3: Where our five design concepts are placed within our design axes.Design Axis TargetableNotificationsPossibleActionsDecisionSupportDegree ofAutomation1. NotificationRulesPrecisely bysender, con-tent, and con-textChange de-livery timing,and displayNone Manual2. CrowdIdeasPrecisely bysender, con-tent, and con-textChange de-livery timing,and displaySuggestionrationalesMixed3. ReflectiveVisualizationBroadly bycontent andsenderChange de-livery timing,and displayUsage vi-sualization,overviewstatisticsMixed4. AdaptiveExamplesPrecisely bycontentChange de-livery timing,and display.Example noti-ficationsAdaptive5. Suggestions:Rules VariantBroadly bysender, con-tent, and con-textChange de-livery timing,and displaySuggestionrationalesMixed5. Suggestions:Concerns Vari-antPrecisely bysender andcontextChange de-livery timing,and displayNone Mixed5. Suggestions:ExamplesVariantPrecisely bycontentChange de-livery timing,and displaySuggestionrationales,example noti-ficationsMixedvariants without spending much more time than would be required to understandthe more complex prototypes [159]. The most promising concepts and variantswere the basis for our final medium-fidelity prototypes after several rounds of in-ternal feedback and iteration. For example, later versions of the adaptive examplesconcept added additional context for what the user was doing when they receivedthe notifications they are personalizing.54Figure 3.1: Several early paper prototypes. Image 1. shows an early sug-gestion interface, 2. shows an early version of the adaptive examplesinterface and 3. shows early version of the reflective visualizations.3.4.2 Final Design ConceptsWe summarize where each design concept stands within our design axes in Table3.3 and the final design concepts are as follows:1. Notification Rules55Figure 3.2: Screenshots of the first four design concepts.• Description: Refining similar prior interfaces [250], users create heuris-tics for how they would like their notifications to be delivered. Usersapply an “action,” such as Promote with special sound or Ignore withno sound, to a set of notifications defined by broad selectors on type,such as direct messaging, work related or email, sender, such as fam-ily/friends or coworkers, and context received, such as working or re-laxing on the weekends. These combine into a logical rule, such as“Ignore direct messages from acquaintances while working.”• Refined or novel elements: Refined prior systems where user createsrules for deferring or filtering notifications [16, 53] to support a broaderrange of actions, such as promoting messages from important contacts.2. Crowd Ideas56Figure 3.3: The three variations of the Suggestions design. Left to right: RuleSuggestions, Example Suggestions and Concern Suggestions.• Description: Uses crowd-sourcing to help users manage notifications,through an automated library of popular notification delivery rules. Itcurates these suggestions using the goal that they help the user accom-plish, such as focusing at work, connecting with family or avoidingjunkmail. To encourage users to explore, the Crowd Ideas design pro-vides users with a “trial period” design element for these personaliza-tions. When a user applies a personalization to their notification sys-tem, the system will ask if they are still enjoying that personalization aweek later and wish to keep using it.• Refined or novel elements: To our knowledge, no current research hasexplored a similar crowd-sourced way to access the notification man-agement suggestions defined in prior work [159], or the idea of trialperiods to notification management. Some novice or older users mayfeel less willing to explore new personalizations on their own for fearof breaking their notification systems [19]. This fear of personalizingled us to explore a design concept that would provide scaffolding forusers as they personalize their device. This user driven personalizationwould also not require them to wait days or weeks for personalization57suggestions based on their own interactions with notifications (the coldstart problem) [159, 221].3. Adaptive Examples• Description: A machine learning system automatically manages all no-tifications, deciding which ones need to be ignored, promoted and soon. In order to give users some amount of agency in notification man-agement, the system shows users a list of notifications it was unsure ithad delivered correctly. The users then review the list and correct thesystem, impacting future delivery of similar notifications.• Refined or novel elements: Similar highly automated notification man-agement tools have been released in industry [224, 260]. For example,the Android Google News app algorithmically chooses to send notifica-tions to users about particular stories, but if a user repeatedly dismissesa type of story, it allows them to choose to receive fewer of that type.Some prior work has explored using in-the-moment electronic sam-pling to collect similar data, but this may cause distraction comparedto the user choosing to manually engage in the process [242].4. Reflective Visualization• Description: The core of this concept is a stacked bar chart visual-ization of the amount of notifications users received in the last month.Sender is encoded by categorical color of the stacked bars, while notifi-cation type or context is encoded along the x axis. To manage notifica-tions, users click on the various bars to designate types of notificationsto manage. They are then taken to a second page where they give anautomatic machine learning system a goal to be applied to that type ofnotification, such as promote more of them or ignore more of them.• Refined or novel elements: Other research has created similar visual-izations of notification history but, to our knowledge, little research hasyet connected this directly to notification personalization [160, 249].We felt this might help the visualizations have a more direct impact onnotification user experience.585. Suggestions• Description: We provide three variants of these suggestions to explorethe role of different design elements within a single notification man-agement design concept. The first, more targeted variant, Rule Sugges-tions, suggests Notification Rules to users. Each was tagged accord-ing to a goal they accomplish and let users ask for more details aboutwhich notifications they impact. The second variant, Example Sugges-tions, provides more decision support by showing a prior notificationand suggesting a popular way of handling similar notifications. Thelast variant, Concern Suggestions, is more targeted, prompting the userto share concerns they might have about types of notifications, such aswork messages in the evenings interrupting too often, to influence thegoals of an adaptive notification management system.• Refined or novel elements: Other research has explored the generalidea of suggestions for notification personalization [159] but our con-cepts more deeply explore how these personalizations can be describedto users and what decision support should be provided.For completeness, we also prototyped pages that connect to these design con-cepts, such as a landing page, a log of all notifications, and a list of personaliza-tions applied. Our design concepts were implemented as largely horizontal proto-types using standard web development toolkits, such as HTML, JQuery and TwitterBootstrap. Certain buttons were not functional, such as the show more button in theAdaptive Examples concept, and we explained to users that none of the data theymight see in the designs, such as the size of bars in the Reflective Visualization,was from their own devices.3.5 Concept Exploration Study MethodologyThis study explored how to better allow users to proactively manage their notifi-cations by eliciting reactions to our design concepts [77]. Participants completeda one-hour, semi-structured interview discussing their reactions to five notificationmanagement design concepts, conducted in-lab. We conducted five pilot interviews59to finalize the interview questions. The interview alternated between small taskswhere participants interacted with the various design concepts and questions aboutthose design concepts. The interview finished with a wrap-up discussion of thedesign concepts.ProcedureAt the start of the study, participants filled out a brief demographics and deviceexpertise survey. Afterwards, we asked participants to describe what sorts of noti-fications they received, how they use them and to describe any approaches they’veused in the past to manage notifications. We then proceeded into the main part ofthe study, where participants interacted with each of the five design concepts andanswered questions about their experiences with them. The design concepts wereshown to users in a random order and described verbally before interaction.To help participants explore each design concept fully, we provided them witha simple task. Similar to role playing, participants were given a small paper cardthat listed a potential concern about notifications that one could use the designconcept to address. For example, one of these cards was: “If there is a work emer-gency your boss will send you a text, rather than emailing you. These messagesshould be prioritized so you don’t miss them, even if you’re currently in a meet-ing.” Participants were then asked to do their best to use the design concept toaddress that concern, though there was no explicitly correct answer. For variation,we created a total of 10 concerns. We chose them to represent complaints withnotifications raised in prior research [13, 160], through personal experience, andpilot testing. We intentionally chose them to span several important areas of notifi-cation management, such as focusing at work, or managing social media, and varyin complexity, such as the number of contextual clauses. For each design concept,participants were asked to address two of the ten concerns, also chosen randomlywithout replacement. The full list of concerns are listed in Table 3.4.We asked participants a series of questions about the experience of interactingwith the design concept before moving on to the next. These questions includedany first impressions they had of it, any issues they may have run into, and if theywould consider trying out a similar concept if it was released commercially. In60Table 3.4: The 10 concerns users addressed in Study 2After work, you receive many emails about things happening in the office. However,they usually aren’t urgent. You’d like to not be distracted by them in the evenings.You like to ignore your phone while you’re working. However, if your family textsyou, it’s usually an emergency. You’d like to see these messages as soon as possible.You use social media (e.g. facebook, instagram) to keep up to date on what’s go-ing on with your friends. However, you’d like not to be interrupted by social medianotifications if you’re at work.You find that emails typically don’t need an urgent response from you. Therefore,you’d like not to be interrupted if you receive an email while working.Some of your less close friends occasionally send you messages to catch up. How-ever, you don’t talk to them very often so you’d like to not be interrupted for thesemessages while working.If there is a work emergency your boss will send you a text, rather than emailingyou. These messages should be prioritized so you don’t miss them, even if you’recurrently in a meeting.Your family and close friends are very important to you. You’d like to see their mes-sages as soon as possible. However,if you’re in an important meeting you’d like notto be interrupted.You get a lot of notifications from different people you follow on social media siteslike Youtube. However, only one or two of these people are important enough thatyou’d like a notification for them.Emails typically don’t need an urgent response, so you don’t want to be interruptedfor them. However, since you’re older dad only contacts you by email, you’d stilllike to be notified for his messages.You have a lot of acquaintances and friends you only chat with online. These mes-sages are fun and you’d like to get them quickly. However, you don’t want them tointerrupt a meeting.addition, there were a few design concept specific questions, such as whether par-ticipants would consider contributing data to the Crowd Ideas concept. To ensurethat no design was overly onerous to use, we also asked participants to report ontask load and complexity. As expected, differences for this were very minor be-tween designs so we do not report on the results.Once participants had interacted with and discussed all five design concepts,we moved on to a wrap-up discussion. We asked participants to describe whichdesign concepts they thought were particularly useful and explain their rationalefor why they might use them over the other designs. They were allowed to selectmore than one design. We also asked participants follow-up questions on several61topics, such as if they would have liked to combine design elements from any of thedesign concepts, and how they felt about the design elements within the concepts,such as the logical rules presented in the Notification Rules concept.ParticipantsWe made a conscious effort to recruit a diverse sample of participants through adson a university paid studies board, online classifieds, print ads in the newspaperand posters placed around our North American city, such as in coffee shops andcommunity centers. In the ads, we said we were specifically interested in talkingto users of a wide range of ages to ensure a wider sample than just universitystudents. In addition, participants were required to speak English and have “ownedand used a device, such as phone, laptop or tablet, that receives notifications formore than six months”. Participants received $15 for participating. We aimedto recruit a total of thirty participants. While we did not have explicit quotas forrecruitment, we made a conscious effort to balance out how many participants ofvarious age ranges we recruited. This recruitment was done iteratively until we feltwe had enough different perspectives that interviewing additional participants wasnot adding substantial nuance to our data.All participant demographics are displayed in Table 3.5. Participant ages rangedfrom 19 to 73 (13 men, 17 women), coming from a variety of ethnicities. Partic-ipant experience and usage of notifications varied. 15 younger participants (19-34 years old) were all university students except one participant saving money toreturn to school. They tended to report high usage of notifications and expertisewith mobile devices. Another ten participants were middle-aged professionals (40-53), reporting similar levels of notification usage. Six of those ten did some formof office work, such as computer programmer, account executive, administrativeassistant or contract management. The remaining four participants were a smallbusiness owner, a nanny, a tennis coach and a film producer. Lastly, five partici-pants were retired older adults (61-73), who reported relatively low usage of mo-bile devices. Many participants in the younger and middle-aged groups reportedoccasionally using the default notification settings panel, though none had installedthird party notification management apps.62Table 3.5: Demographics and device usage data from all 30 participants inStudy 2. Amount of phone usage and expertise with mobile devices wereself described by participants. Participants are grouped by cohort as stu-dents ( s), older professionals ( p), and retired older adults ( o)1 s 24 male student - ugrad 5 intermediate2 s 22 female student - grad 8 expert3 s 33 male student - grad 5 intermediate4 s 22 female student - grad 5 expert5 s 25 male bike courier 5 expert6 s 26 female student - grad 5 expert7 s 22 female student - ugrad 3 expert8 s 21 female student - ugrad 8 expert10 s 19 female student - ugrad 3 expert11 s 21 male student - ugrad 3 expert14 s 20 female student - ugrad 5 expert15 s 22 female student - ugrad 8 intermediate18 s 34 male student - ugrad 8 intermediate19 s 21 male student - ugrad 5 expert26 s 18 female student - ugrad 5 expert13 p 40 male lecturer 3 expert16 p 49 male programmer 1 beginner22 p 50 female management advisor 8 intermediate23 p 50 male film producer 5 intermediate24 p 53 male tennis coach 3 intermediate25 p 40 female admin assistant 1 intermediate27 p 45 female nanny 5 intermediate28 p 49 male account executive 8 expert30 p 52 female contract management 5 intermediate31 p 50 female runs retail store 5 intermediate63P# Age Gender Occupation Phone use(hrs/day)Expertise9 r 62 female retired - student 1 beginner12 r 61 male retired - surgeon 0.5 intermediate17 r 73 male retired - teacher 1 beginner20 r 65 female retired - dentist 0.5 beginner21 r 64 female retired - social worker 1 intermediateData Collection and AnalysisThe interviews were audio recorded and transcribed verbatim. In addition, someobservational notes were taken by the interviewer. To analyze this data, we con-ducted a thematic analysis using the framework presented by Braun and Clarke[30]. We allowed our themes to be reflective of smaller groups of participants tocatch issues specific to an age cohort. We also focused on a descriptive interpre-tive analysis to see which design concepts participants gravitated towards. Themeswere allowed to bubble-up inductively because we didn’t want to limit the scope ofour analysis by a prior framework. Our research was conducted in a Constructivistphilosophical framework. As such, we do not claim our results are the objectiveexperience of our participants. They are a rigorously created interpretation neces-sarily influenced by the analyst’s theoretical experiences and biases.To start analysis, five transcripts (two from each of the younger and middle-aged group, one from the older group) were initially open-coded by two researcherson the team (including the lead researcher) and another two transcripts were readby a third. Coding was collaborative and iterative with each researcher being ableto see and react to the others’ codes. Our process supported crystallization throughmultiple collaborative analysts to enhance the depth and nuance of our interpre-tation [209]. The lead researcher then went on to code all transcripts and the re-sults were brought to several research team meetings discussing possible candidatethemes. After these meetings, we iteratively identified additional foci for analy-sis, performing coding refinement as necessary. For instance, after identifying thatmeeting social expectations was a key worry, we conducted additional coding toidentify worries about meeting such expectations.643.6 FindingsOur analysis has two main foci. First we describe participants high-level reac-tions to the different design concepts as well as the design elements within them(Themes 1 and 2). Second, we describe an analysis of how individual differencesof participants, such as types of notifications received or the structure of work intheir occupation, may have influenced how they anticipated they would use the de-sign concepts in their everyday life (Theme 3). For context, we label participantsaccording to three clusters that reflect both age and occupation status, youngerStudent, middle-aged Professional, and Retired older adult: P#{s,p,r} For exam-ple, P6 s refers to participant 6, who is a younger student.3.6.1 Theme 1: Design Concept PreferencesParticipants described how the different design concepts had unique core qualitiesthat made them prefer one over the others for certain tasks while personalizingnotifications. For example, the Notifications Rules design was familiar and couldcreate more detailed and expressive settings during early personalization, while theSuggestions concept would help them refine pre-existing preferences. Participantsidentified through their lists of particularly useful design concepts that the Noti-fication Rules concept (16 participants), the Reflective Visualization concept (16participants) and the Suggestion concept (14 participants) were more useful thanthe others. In comparison, only one participant identified the Crowd Ideas con-cept as particularly useful, and four chose Adaptive Examples. Most participantsidentified two designs as particularly useful on average.Complementary Roles of Notification Rules and SuggestionsParticipants often described how they felt the Notification Rules and Suggestiondesign concepts had different strengths that would both be appreciated in futurenotification management tools. For example, the Notification Rules design of-fered participants a great deal of control and freedom through the manually createdlogical rules design element. P11 s:“[The notification rules concept] just offersmore flexibility and more choices [than the other designs].” Meanwhile, severalparticipants described how the Suggestions tool offered a simple way to refine65preferences they might not have thought of. P10 s: “[Notification Rules] is forpeople that are just starting out but [Suggestions] is for after you use it for a whileand want to improve it.” Further, many participants expressed concerns over theperceived effort required for purely manual personalization relative to simply ac-cepting suggestions. P28 p: “I’ve got to go to my email after this and add acouple rules. That’s going to be a very arcane experience... [Either Suggestions orCrowd Ideas] is a lot simpler.” Despite many participants desiring reduced effortin notification management, which would align to relatively broad targets withinautomated personalization in our design axes, they also clearly expressed a needfor targeted notification personalization. For example, several participants felt theyneeded the Notification Rules design as a backup to the lower effort more auto-mated designs. P11 s: “I would start with [Suggestions] for general use, and thenfor more specific settings, then I would turn it to [Notification Rules].”Negative Reactions Towards Online Learning and Adaptive NotificationManagementThe Adaptive Examples design concept explored adaptively managing notificationswith online ML models informed by explicit user sampling, which is very similar todesigns proposed in recent notification management research [40, 96, 144]. How-ever, many of our participants expressed concerns over this concept and few ratedit as particularly useful. Perhaps the most common concern or fear raised by par-ticipants was the possibility of missing important or meaningful notifications ormessages. This was often cited as one of the primary reasons to avoid more auto-matic systems like the Adaptive Examples design concept. P1 s: “I’d be worriedthat I would accidentally mute certain things...I feel like you don’t have fine controlover it.” Importantly, this lack of trust appeared to be tied into a concern about theambiguity of relatively black box management systems, and what it was learningfrom user input. P6 s: “It showed some emails...I really can’t trust the system, likeI don’t know if it focusing on the context and the sender or is it only the sender.”66Strong curiosity for more Decision Support Through Visualization andSuggestions:Many participants described how they would appreciate having more decision sup-port in future notification management systems. For example, reflective visualiza-tions of notification usage are an under-explored area of research, but our resultssuggest that many users were intrigued and interested in viewing such data whilepersonalizing [212, 249]. Around half of our participants included it in their listof particularly useful designs, either because the data would be interesting to lookat, or would help inform their personalization decisions. P1 s: “I think it’s agood idea to inform your notification decisions from the data that you see.” How-ever, many participants also expressed some concern over whether the insights theycould gain from viewing real data in a higher fidelity prototype would be action-able. P6 s: “Yes, interesting to see, but it’s not something to [...] act on.”Many participants also described how a key reason they preferred the Sug-gestion design concept was that design elements from the different Suggestionsvariants helped them decide what personalizations they needed. P15 s: “Havingsuggestions on what to do with these settings would be great. Because I wouldopen [the notification rules concept] and I wouldn’t really know what to do.” Fur-ther, from our variants of the Suggestion designs, we observed how participantsgenerally preferred Suggestions that provided decision support through contextualdata, such as example notifications or contexts impacted. For example, all par-ticipants who described the Suggestions concept as particularly useful preferredthe Concerns (which prompted for targeted feedback on a type of notification in aspecific context) or the Examples variants (which gave an example of a recent no-tification and suggested how similar notifications could be delivered in the future)to the variant which suggested more general logical rules. They described how themore targeted variants (Concerns and Examples), gave them greater insight intothe context of the personalization decisions which was useful for fine tuning theirpreferences through suggestions. P8 s: “I like the choices it gave you...you couldreally focus on that specific scenario.”673.6.2 Theme 2: Need for Control and Risk ManagementConsistent with our work in Chapter 2, many participants desired a sense of controlover their attention that was difficult to achieve with current phone settings. P22 p:“If this is based on me, I like to be in control.” As we analyzed the data, we beganto notice that differences in individuals’ willingness to take certain risks seemed tounderlie whether they were willing to accept reduced user control. Consistent withprior work, concerns or fears of missing important or meaningful notifications ormessages was often cited as one of the primary reasons to avoid more automateddesigns such as the Adaptive Examples concept [144]. However, we extend thisresult by suggesting that this need for control varied depending on notificationsubject and that certain design elements, such as trial periods for personalization,may help reduce user concerns.Concerns over Control Varied Depending on Design Concept andNotification SubjectMany participants desired a sense of control over their notifications that made themhesitant to accept machine support. P25 p: “No, I want to control it. I don’t wantsome AI doing it for me.” This need for control was also somewhat orthogonal tothe efficiency of managing notifications, as some participants reported subjectivevalue to spending the effort to engage more deeply. P23 p: “There’s agency asthe user to select what is going to most exactly fix my problem...I found it moresort of mentally taxing to accomplish it, yet I was more satisfied with the result.”However, several participants did express that there would be substantial value toadaptive notification management, if it could be limited to relatively low-priority oreasy to predict types of notifications. P11 s: “[For] my family or work stuff I wouldprobably pick my own [settings]... but social media, friends, or other unimportantmessages, I would leave to the smart system.”Trial Periods May Help Reduce Perceived RiskOur key novel finding, however, is that the “trial period” design element intro-duced in both the Crowd Ideas and Suggestions designs was appreciated by mostparticipants as a way to reduce the perceived risk of personalization. They often68expressed that, because they didn’t have to permanently commit to a new person-alization, they might be willing to try out more. P3 s: “I don’t generally try newthings too much. But if it’s trial for seven days...if I like it I can keep it, and if Idon’t like it I can just throw it away.” Others suggested that it would help themavoid making mistakes while manually personalizing their phone. P18 s: “I thinkit would help with people who are afraid to make choices on their devices becausethey don’t want to mess anything up.” There was, however, some disagreement onthe appropriate duration for these trial periods. Some middle-aged professionalsexpressed that a week would not be enough time to capture the full variation oftheir schedule. P16 p: “Let’s say for two weeks, or three weeks and only afterthat I would pass judgment.” In contrast, several of the younger students expressedthat a week would be too long because they would have gotten frustrated earlier.P10 s: “I don’t know if I would get that notification before I already deleted theapp.” This may be a reflection of different volumes of notifications.3.6.3 Theme 3: Age-associated Differences in NotificationManagement NeedsOur participants structured their day-to-day life and work schedules in substantiallydifferent ways, likely, at least in part, as a consequence of their differing occupa-tions or ages. These differing structures appeared to influence both the specificnotification personalizations our participants desired, and created unique designchallenges for notification management.Need to Focus at Work and Appear Professional Key Goal for Middle-AgedUsersParticipants’ expectations for how their contacts should communicate with them atwork were a key issue in how some participants, especially middle-aged profes-sionals, managed notifications. These participants seemed to mostly want to tailortheir devices to only receive “mission critical” notifications while working [13].P22 p: “If somebody is just saying, ‘Hey look what’s going on?’...you might betempted. But usually I don’t really have any [free] moments at work because youhave to focus.” These participants also occasionally described how the relativelyrigid structure of their full time jobs required them to focus, but made it easier to69plan notification preferences. P30 p: “If you have something that you’re doingconsistently day in and day out...I think those [Notification Rules] are great. Youcan say hey I’m going to be at a staff meeting every Friday from this time to thistime.”Some participants also felt pressure to engage in different forms of notifica-tion management than they would normally prefer in order to focus at work andmeet social expectations for professionalism. P27 p: “I would always say, ‘Oh, ofcourse, I want to be notified by my family...’ But the reality is, to be successful ina business and a workplace, those kinds of notifications would be interfering withyour day.” Users often expressed frustration with other contacts breaking thesesocial expectations. For example, some of these participants disliked expectationsto respond urgently to notifications about friendly messages while busy. P22 p:“Certain people expect something right away and whatever. But that’s not my defi-nition of real friends.” They also expressed that they believed close contacts wouldbe more understanding of their personal preferences for communication. P16 p:“People know when I’m at work don’t email me, don’t text me, or don’t spam dur-ing work hours.”Skepticism to Popularity Based Suggestions Amongst Younger Users:Some younger students’ reactions towards receiving suggestions or browsing ideasof how others manage notifications provides an interesting contrast to the concernof meeting social expectations. Often, these younger participants expressed dis-interest in design elements that show others’ notification personalizations, such asthe popular categories in the Crowd Examples concept, because they would not berelevant or meet their own needs. P10 s: “Everyone is so different, I think I wouldjust do what’s best for me. And I’m not one to get peer pressured by other peopleand whatever they are doing.” Sometimes, this was reinforced by prior experience,such as hearing about an idea for notification management from a friend, then notenjoying it once it was actually applied. P8 s: “I had a friend who managed hernotifications by setting certain sounds for certain people...I tried it out once, but itdidn’t quite work for me, because I still want to be in the loop of other things.”70Impermanence of Settings Amongst Younger Users:Several of our younger student participants occasionally described their notifica-tion management practices with a surprising sense of impermanence. They de-scribed frequently modifying them to meet contextual needs rather than maintain-ing stable preferences. For example, they restricted social media notifications neardeadlines or exams. Using Aranda’s framework for the experience of notificationsbased on urgency and enjoyment, they cut down on “nagging” or “fun” notifica-tions during busy periods [13]. P3 s: “You are not in the same mental state overthe year. If I have a deadline coming up, I don’t want to be disturbed even by any-one.” Some participants even expressed the desire to make such a process easier asnuanced contextual needs are complicated and shifting. P10 s: “Every situationis different and trying to cater your settings to each circumstance...it takes effortevery single time.” This may reflect the more transient nature of student life, whichprior research has suggested is a time of great change and growth [81, 123].3.7 DiscussionOur results describe three themes on how participants reacted to the different de-sign concepts and how the individual differences of participants appeared to in-fluence their notification management preferences. In our discussion, we explorethe strengths, weaknesses, and applications of design elements within the differentconcepts. Based on these implications, we provide suggestions for using the designaxes and design directions that should be explored further in future research. Wealso explore how some of these individual differences, such as life structure, mightbe used to support notification management in the future.3.7.1 Future notification management systems should combinemanual notification rules with customizable suggestions toleverage their complementary rolesFuture work should look to combine the most useful design elements of the “No-tification Rules” and “Suggestions” design concepts. While similar systems havebeen explored individually in prior research, our research suggests these conceptsmay serve separate but complementary notification management sub-tasks within71a single tool [16, 53, 159]. The manually created logical rules used to target noti-fications for personalization in the “Notification Rules” concept could effectivelyaccomplish both broad and targeted personalization. This range of supported gran-ularities in targetable notifications appeared to make these rules effective during theinitial setup of a device, and could target more specific issues that automated de-signs may have difficulty identifying. More automated personalized suggestionscould be added to support refining or maintaining a notification system throughmore targeted and simplified design elements, such as personalization by contextor example.The manual and suggestion based design concepts could also be tailored withdesign elements to better support their perceived uses in future notification man-agement tools. Prior manual personalization systems using logical rules for notifi-cation management were somewhat more limited, focusing mainly on filtering ordeferring inappropriate notifications [16]. We argue that including these limited ac-tions unnecessarily limits expressivity. Meanwhile, our design allowed participantsa broader set of actions, such as promoting with a special sound, and participantsappreciated having these more specific controls related to context, such as work-ing, commuting or relaxing, while still being broad enough to specify preferencesacross multiple similar apps at once. Similarly, prior suggestion based designscommunicated which notifications to filter using relatively broad targets in theirlogical rule design elements [159]. We found more targeted design elements, suchas personalization by context or example, helped structure users’ decisions on sug-gestions through context. We suggest that care be taken to allow users to tailorthe suggestions further, similar to how they could select an action in the ConcernSuggestions variant to allow users a greater sense of control in personalization fora relatively small effort.3.7.2 Decision support and degree of automation design axes shouldbe carefully considered in future notification management toolsOur results suggest app developers deciding what notifications to send users andwhat notification controls to provide on top of OS level settings, if any, shouldcarefully consider adding more decision support and automation. Several partici-pants described wanting more automatic notification management design elements72for less important notifications, and many participants reported substantial desirefor more design elements providing decision support. Given observed concernsthat running many sensors hurts device performance [10, 132, 157], we suggestthese app developers should take care to keep these design elements as simple aspossible if they are run in addition to OS controls. For example, an app like a gamemight put a simple hard cap on number of notifications sent per day or hour thatusers could configure.To provide the decision support requested, design elements like visualizationsof notification usage or examples of impacted notifications could be provided. Twopast studies have explored tools that allow users to view visualizations of their noti-fication usage through personal informatics [160, 249]. Evaluation of these specifictools’ ability to change users’ notification usage was either preliminary, such asusing very small number of participants, or not the focus of the study itself. How-ever, much prior work has gone into understanding the process of behavior changeand whether personal informatics tools help accomplish this [28, 143, 199, 240].There appears to be theoretical backing within the commonly used transtheorti-cal model of behavior change to support similar tools’ ability to affect behaviorthrough processes such as raising consciousness, or encourage self-monitoring andcontingency management [143, 240]. A recent meta-review cautiously substanti-ated the finding that personal informatics may enhance behavior change throughthese mechanisms [240]. Our key finding is that users desire access to this datawhile personalizing notifications.One possible design element which merits further research is the idea of aperiodic “notification check-in.” This might appear like a typical suggestion fornotification management, but prompts the user to engage in a more in-depth re-flection on their notification usage. Rather than simply receiving a prompt withthe ability to apply a suggested personalization, users might click the suggestion toopen up the notification management app. Here they would be asked to considera graph of their notification use over the past few weeks and be provided with afew suggested personalizations. The graph could then be animated or highlightedto show how many notifications the suggestions would have impacted if they hadbeen accepted at the last check in. Closely coupling these suggestions, their im-pact on notification use and the reflective visualizations, may help users identify73new, more effective notification management behaviors, which has been noted as aweakness of current personal informatics tools [240].3.7.3 Identified differences in life and work structure could imply aneed for design elements that support periodic manual revisionof notification management preferencesNotification personalizations were considered relatively impermanent for youngerusers, who described revising the notification personalizations they applied arounddeadlines or for different school terms. Determining the precise cause of this im-permanence was beyond the scope of this work. However, we theorize that therelatively higher technological skill of younger users and the somewhat transientnature of university student life may play major roles. Past research has identifiedbeginning university as a time of substantial personal and professional change andhas investigated how best to support students to succeed in the transition to highereducation [81, 123]. This transition involves interacting with a substantial new setof social contacts and changes in day-to-day schedule. These social networks arenot static, and past research has tried to understand how substantial life shifts liketransferring between degrees or colleges impact students [22, 107, 231]. Impor-tantly, we note that, while we only saw this impermanence amongst our youngerusers, shifts in notification preferences due to changes in schedule or around dead-lines are likely to happen to a wide variety of users in many occupations.The relative impermanence of notification settings amongst younger users canhave both positive and negative implications for more automated notification man-agement tools, and might imply a need for periodic manual revision of notifica-tion preferences. In theory, if shifts in notification preferences are reasonably pre-dictable, such as during exams or before deadlines, then future notification man-agement systems could automatically detect and adjust for them. For example, amore automated system might silence social media notifications in the last few daysbefore a deadline for a priority task on a to-do list. However, if users are makingthese changes by hand, they may be more accurate than ML predictions, even if thismanual revision may be cumbersome. Thus, further work should investigate designelements to support periodic manual revision of notification preferences. For ex-ample, if a future notification management tool detects substantial changes in daily74schedule, it might prompt the user to reflect on how well their current notificationpreferences match their needs and define new contextual rules for notification de-livery. Visualizations of notification usage data or examples of notifications thatshould be personalized could be given to help inspire participants to personalize innew ways.3.7.4 Concerns about the risks of personalizing could be managedthrough trial periods for personalizationsSimilar to prior work, many of our participants were hesitant to personalize, partic-ularly with the Notification Rules concept, because they risked missing importantnotifications, such as those from family and friends, due to the risk of unforeseenedge cases [13]. There are many factors both within and between groups of usersthat could affect which risks participants would accept while personalizing. Forexample, previous work has suggested that aging may reduce risk taking behaviorin certain scenarios [51, 124, 151]. Further, differences in core personality traitshave also been modelled to show certain traits are associated with increased risktaking behavior, and older users are more negatively impacted by excessive dis-traction [31, 46, 49, 55, 71, 169, 228]. The relative role and impact of these andother individual differences on creating these concerns was unclear in this work,and could be explored further in future work.A key finding is that the trial period design element within our design conceptsmay help mitigate these risk concerns of personalizing. Prior work has suggestedincluding rationales with suggestions to increase trust in the system [33]. Withinour designs, we created the “trial period” design element to more specifically targetthe risk of applying personalizations permanently. This trial period concept wasgenerally well received and participants expressed it would lead them to try morepersonalizations in the future. Future work would need to see if these commentstranslate to changes in behavior when applied in the field. However, given the lackof particularly strong negative reactions to trial periods, the concept seems to meritfurther exploration and development. Even if there is no measurable change in userbehavior, there may be an emotional benefit to reducing users’ anxieties over tryingnew personalizations.753.8 LimitationsAs an early phase in the design process, interviews enabled us to explore the po-tential uses of many design elements within notification management systems andidentify what opportunities remain to enhance these systems in future work, similarto prior work by Vitale et al. in personal data management [243]. The limitationis that our method did not allow participants to try the designs with their own no-tifications over time and assess how they impacted their notification experience. Alongitudinal field study is needed to reveal true adoption of potential design con-cepts, and is an important next step in this research. Our sample size in someclusters is too small to make firm conclusions on users’ specific needs, such asolder adults. Our total sample size was reasonable for an interview-based compar-ison of design concepts. For comparison, Vitale used 16 participants to comparefive design concepts while we used 30 [243]. However, including a larger andmore diverse participant sample could reveal latent themes, especially within ageclusters.3.9 ConclusionIn this chapter, we explored how users could more proactively manage notifica-tions to address user concerns around a lack of control. We characterized fourdesign axes targeted for enabling this communication, and presented five designconcepts exploring different ways users could proactively manage notifications.We conducted a concept exploration study through semi-structured interviews witha diverse sample of 30 participants to gauge the perceived uses of various designelements in our design axes used to communicate how notifications should be de-livered to the user. We identified how design elements aligning to particular areasof the design axes enabled different design concepts to play separate but comple-mentary roles in notification personalization. For example, manual personalizationtools supporting both precisely targeted and broadly aggregated notification per-sonalization, such as our Notification Rules concept, were effective for broad, earlysetup of the device and later troubleshooting. Meanwhile, more automated person-alization suggestions could help users refine their personalizations. Our resultsalso suggest that notification management tool designers should carefully consider76adding more decision support and automation to the designs, and we providedguidance on when this is appropriate. Lastly, we uncovered themes describing theway that the varying life structures of participants of different ages and occupa-tions appeared to influence their notification management strategies. For example,some younger students treated their notification settings as surprisingly imperma-nent, which may imply a need for periodic revision of notification preferences.This research identified a number of powerful opportunities to refine notificationmanagement practices in future research.Based on these results and contributions, we decided that the role of decisionsupport and reflection in notification management should be explored more deeply.Thus, the results of this study directly guided and inspired the design work con-ducted in Study 3 (Chapter 4).77Chapter 4Reflective Spring Cleaning: UsingPersonal Informatics to SupportInfrequent NotificationPersonalization (Study 3)This final research chapter1 builds upon the results from all previous chapters todesign and evaluate a novel notification management tool. Our prior insights thatmany users strongly desire a sense of control over notifications (Chapter 2), thatthe adaptive notification systems explored in recent research do not provide thissense of control (Chapter 3), and that there is substantial interest in more decisionsupport (Chapter 3) led us to build the ‘Reflective Spring Cleaning’ design concept.Reflective Spring Cleaning combines design ideas from across the dissertation bysupporting users during notification management with visualizations of their notifi-cation use. We then used a prototype of this design concept to evaluate the extent towhich this concept supports users’ notification management practices. Our resultsare promising and suggest the concept could effectively support proactive manage-ment for many users, especially by increasing participants awareness of their own1Originally Published in Izabelle F Janzen and Joanna McGrenere. (2022) Reflective SpringCleaning: Using Personal Informatics to Support Infrequent Notification Personalization. Proceed-ings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’22).78ability and need to remove unnecessary distraction. We note that our recruitmentfocus in this study moved away from retired older adults as we suspected therewould be substantial individual differences in their notification use and our designassumed users had at least some work related notifications. This work opens upa new and interesting design approach to notification management that we argueshould be explored further in future work.4.1 IntroductionIncreased distractions through notifications, often in the form of pop-up messageson a mobile phone, is a problem that has been of great research interest and a focusof public discourse [102, 110, 161, 183]. Such distraction has been shown to causestress to users and can be particularly frustrating if the notifications are not urgent,relevant or useful [13, 56, 70, 134, 160, 191]. However, users have many contex-tual and individual differences in what notifications they are receptive to receiv-ing despite this stressful distraction. For example, the importance of seeing workmessages on the weekends or social messages while working vary for each user[13, 108, 160, 172, 192]. In this work we explore the problem of designing a no-tification management system where users can actively define which notificationsthey are receptive to receiving despite the distraction they cause [13, 16, 53, 250].Prior notification management tools in both industry and academia have usedsoftware personalization to support users actively defining their notification prefer-ences [12, 78, 96, 98, 159, 183]. The techniques used are very diverse but usuallyinvolve the user describing which types of notifications they do not want to receive,such as by creating rules to filter unnecessary notifications [16, 53, 91, 105, 187,235, 250]. However, many users avoid personalizing due to barriers such as thetime required to create settings until they are convinced the benefits of personaliz-ing are worth the effort [18, 159, 160, 214]. For example, they might be triggeredto personalize by receiving a sudden increase in particularly disruptive notifica-tions [18]. These triggering events are often necessary to motivate personalizationbecause problematic habits of technology use commonly become unconscious andnormalized [73, 127, 218]. Related, wellness systems using journals or visualiza-tions of usage data have been shown to encourage reflection on user habits and79critically challenge user assumptions about the way they use their mobile technol-ogy [5, 73, 127, 194, 218].Deep reflection about notification use could motivate users to personalize theirnotifications, but reflection is a complex, non-linear process that takes focusedtime in specific activities, such as through writing a journal or viewing personalinformatics [5, 21, 73, 218]. Personal informatics (PI) systems are tools that en-able users to track, visualize, and analyze data about personal behaviors like webbrowsing habits to encourage deeper reflection on how to improve those behaviors[64, 143, 199, 218, 240]. Prior notification personalization tools, to the best ofour knowledge, have not explicitly provided opportunities for focused reflectionwhich may limit user motivation to personalize [13, 159, 214, 252]. For example,users may assume notifications are not worth spending the effort to personalizeif they underestimate the number or disruptiveness of notifications they receive[18, 159, 160]. While reflection may increase user motivation to personalize no-tifications, research on PI systems in other domains, such as mental health, hasalready suggested that conducting thorough PI data collection and deep reflectionpresents substantial barriers to users that are not intrinsically motivated by self-tracking [203, 204]. The goal of our work was therefore to explore how a notifi-cation management system could provide PI visualizations of notification use thatprovide opportunities for explicit, useful reflection, while minimizing the complex-ity and overhead of using those visualization components so that users would notbe further deterred.PI of notifications was very briefly explored in one prior study [249]; they laidthe technical groundwork to create visualizations of notification usage data but onlydid a very small (n=3) evaluation of impressions of graphs of their recorded data[249]. To accomplish our goal of minimizing the complexity and overhead of us-ing PI visualizations of notification data, we explored the idea of a simplified dataabstraction. Data abstractions are theoretical tools widely used in the visualiza-tion literature that map raw data, such as individual notifications, into semanticallyrelevant categories for visualization [166]. Identifying a data abstraction and as-sociated encoding of that data into a visualization that captures the most relevanttrends in notification data, while remaining as simple and easy to understand aspossible, was the focus of the design process for our visualization components.80To allow participants to actually personalize their notifications, we integratedthese reflective visualizations with suggestions for notification management. Ourprototype suggestion system used simple contextual rules, shown in Figure 4.1,to simulate how a machine learning system could provide personalized sugges-tions to the user. This personalization approach, where the user and the system“work together” to achieve a personalization, often referred to as mixed-initiativepersonalization, is the current state of the art notification management technique[96, 159]. For example, Prefminer is a system that learns how users respond tonotifications and suggests different filtering rules to remove irrelevant notificationssuch as “filter Facebook notifications with ‘candy’ and ‘crush’ in the title” [159].These suggestion based approaches do reduce the barriers to personalization byleveraging machine intelligence, but users still need to be engaged enough in noti-fication management to fully interact with the suggestions.Our core research questions are: RQ1 How would simple visualizations ofnotification use in a notification management tool influence users’ motivation topersonalize their notifications? RQ2 How would such a system affect users’ noti-fication management practices and experience?To answer these questions, we created a novel notification management de-sign called Reflective Spring Cleaning. This design approach involves visualiza-tions of the user’s notification usage data, personalized suggestions for notifica-tion management, linked highlighting to show how notifications will be impactedby the suggestions, and a prompt to Spring Clean very infrequently, such as ev-ery six months. Our design approach thus differs substantially from prior mixed-initiative suggestion systems that frequently, such as one suggestion every few daysor weeks, give users individual personalization suggestions with no visualization.For the visualization itself, we created a novel “who-what-when” data abstractionthat maps prior domain knowledge of what types of notifications users value tothree visualizable facets of notifications. Our data abstraction [166] was createdthrough extensive synthesis of prior qualitative work and was refined using low-fidesign iteration [13, 160, 191]. We conducted a four-week longitudinal study toevaluate the effectiveness of our Reflective Spring Cleaning approach and data ab-straction where participants passively logged notifications for two weeks, then per-sonalized with Spring Cleaning, followed by two more weeks of passive logging,81and finally a debriefing interview where they reflected on their new experience withnotifications.Our findings revealed that the Reflective Spring Cleaning design prompted par-ticipants to reflect critically on their habits of notification use, which raised theirawareness of their need and ability to personalize those notifiations. Further, thefacets of our data abstraction (Who, What and When) all provided useful insightsabout notifications. Reflecting on these abstractions supported utilitarian notifica-tion management, satisfied participants’ brief intrinsic curiosity, and helped themto minimize perceived sources of stress from distraction. Finally, individual differ-ences in past notification management behavior influenced participants’ willing-ness to continue engaging in Reflective Spring Cleaning. The primary contribu-tions of this work are:• Design of the Reflective Spring Cleaning approach that combines researchfrom mixed-initiative personalization systems [159, 195] with visualizationsof PI data [249] to encourage critical reflection on notification use and moti-vate users to overcome barriers to personalizing notifications [18, 214].• Evaluation of the Reflective Spring Cleaning design in a structured study in-volving participants’ own notification data, showing that it supports severalmotivations for reflection and can improve participants’ experiences with no-tifications. This improvement in experience was observed despite the poten-tial barrier reflection adds compared to prior notification management sys-tems [159, 195].• Exploration of the novel data abstraction created in the Reflective SpringCleaning design to demonstrate that the data abstraction facets provide usefulinsights that are actionable in combination with the included personalizationfeatures. This combination of PI and personalization may have broader im-plications for making reflective systems more actionable in general [64, 240].4.2 Related WorkWe provide an overview of research into what notifications users are receptive to82receiving, personalization techniques to apply those preferences to a notificationsystem, and ways to support reflection through personal informatics.4.2.1 Factors Impacting Receptivity to NotificationsThe factors that affect which notifications users are receptive to receiving despitethe distraction they cause are complex, contextual and difficult to precisely predict[13, 70, 160, 195, 244]. Work has identified both contextual and content based fac-tors impacting which notifications users are receptive to or otherwise value. Thecontextual factors usually revolve around what primary tasks the notification isdistracting the user from, such as avoiding distraction from work during a familydinner [13, 56, 70, 91, 160, 244]. Content factors are many and varied but caninclude different senders. For example, close social contacts are usually more im-portant than acquaintances, strangers or automated systems. Other content factorsinclude urgency to respond, relevance to work or even just interest in the content[13, 70, 160, 244]. Importantly, these contextual and content factors have beenshown to be dependent on each other and subject to strong individual differences,necessitating user-level personalization. For example family messages might typi-cally be more important, but in certain contexts, like work, they still might need tobe ignored for some users [13, 160].An early approach to improve the experience of these systems was using amachine to automatically deliver notifications when users are least disrupted bythem [11, 70, 102, 207]. Commonly, this involves delivering notifications whenthe user is already switching between tasks [70, 207]. However, this approach hasfallen somewhat out of favor, possibly due to users fearing that they may misskey notifications or that they lack control when they are not in the loop [13].Research to understand which notifications should be deferred in these systemsoften derives meaning about which notifications need to be delivered promptlybased on in-the-moment experience sampling methodologies or behavior analysis[112, 159, 160, 160, 191, 234]. However, users may decide to further personalizetheir notifications manually and might have difficulty remembering which notifi-cations were problematic days or weeks after receiving them. Our work exploresto what extent deeper data-driven reflection during notification management might83encourage users to personalize, such as by helping them remember which notifica-tions were problematic.4.2.2 Approaches to Software Personalization and NotificationManagementSoftware personalization involves a system adapting to the user needs and differ-ences, such as which notifications they are receptive to receiving, and has been ex-plored in many domains [18, 32–34, 79, 80, 138]. Manual personalization has beenthe most common prior approach to software personalization. However, users ex-perience barriers to personalization that triggering events, such as particularly frus-trating errors, push users to overcome despite the effort required [18]. The barriersare sufficiently high that few users tend to personalize software [18, 159, 214, 252].However, users still report substantial frustration with notifications, thus pushingthe research community to try to reduce these barriers [13, 191, 214, 252]. Typ-ically, machine intelligence is used to learn useful adaptations to the existing set-tings based on what the system knows about each user [18, 32–34, 79, 80, 138].Research has identified a spectrum of techniques for applying this learning fromfully adaptive systems (where the machine automatically optimizes the settings) tomixed-initiative systems (where the machine suggests improvements to the user)that have all been applied to notifications [12, 18, 32–34, 78–80, 98, 138, 183, 251].These come with tough design trade-offs such as a lack of control in adaptive de-ferment systems, or increased distraction in mixed-initiative [16, 53, 70, 159, 251].While there are a variety of personalization approaches to which we could haveexplored adding focused reflection, mixed-initiative suggestions are the state ofthe art personalization technique for notifications [13, 96, 160]. Further, mixed-initiative suggestions reduce the barriers to personalize notifications, provide greateruser control than automated deferment systems, and the suggestions themselvescould serve as a small prompt to reflect more deeply on notification use [16, 18, 34,159]. For example, in industry, iOS15 is currently in developer beta and exploresmixed-initiative personalization through a new “focus” mode where on-board arti-ficial intelligence will automatically determine the current phone context such asworking or commuting [96]. Users can choose and then customize a suggested “fo-cus” such as ignoring non-work messages while working, similar to prior do-not-84disturb features on both Android and iOS. Focus mode will likely improve a user’snotification experience, though users will still need to be engaged in notificationmanagement enough to properly customize which foci should be applied in whichcircumstances [70, 79, 159, 235]. Our work explores to what extent users’ deci-sions about how to deliver notifications in a similar system would be influenced byreflection on their notification use, and more specifically, whether reflection wouldencourage users to engage with the system.4.2.3 Challenges in Reflective Design and Personal InformaticsReflection involves serious thought and consideration for purposes such as learn-ing or critical review and needs to be done in appropriate environments to achievediffering levels of depth and nuance [73, 218]. Particularly deep reflection that in-volves identifying and questioning deeply held assumptions is described as criticalreflection [162]. For example, mental health and wellness researchers have arguedthat users should be encouraged to deeply reflect on how healthy and necessarytheir use of mobile technology is [5, 20, 62, 127, 218]. A wide variety of toolshave been developed to support deep reflection on health and wellness by moni-toring concrete data, such as through guided journaling [21, 194], using personalinformatics to track personal behaviors and mood [59, 143, 208, 240, 249], or visu-alizing data about peer technology use [61, 127, 168, 201]. Research suggests thatdeep reflection is either a primary outcome of research [17, 21, 62, 73, 146, 205]or a way to encourage the user to refine and improve their personal behavior[64, 143, 240]. For example, research has shown that reflection on personal in-formatics can help raise consciousness of potential issues and maintain new behav-ior through self-monitoring, which are key steps in established models of behaviorchange [64, 240].Prior research has identified a number of key research challenges in prior re-flective interfaces. Without an intrinsic motivation to self-track, such as strongcuriosity about one’s quantified self or a need to manage a chronic health condi-tion, general users may quickly stop tracking for many reasons [60, 93, 203, 204].For example, onerous tracking and analysis requirements, a lack of actionable take-aways, or difficulty identifying what data should be tracked may cause frustration85and dropout [60, 64, 180, 181, 240]. While some users may feel satisfied in whatthey learned and be happy to disengage from self-tracking, others do so out offrustration [63, 115, 117]. To minimize the chances of this occurring, researchhas explored ways to simplify many elements of the self-tracking process, suchas by displaying subsets of recorded data and using simplified visual representa-tions [60, 86, 206]. We sought to extend this line of research to the domain ofnotifications, specifically by creating a simplified data abstraction that would mapnotification data into visualizable categories that summarize the most importanttrends within notification use.While we were not specifically aiming to improve prior personal informaticssystems, we did worry that many users of notification management software maynot have extensive experience with other self-tracking tools. Supporting users whoare not already experienced with and intrinsically motivated by self-tracking hasbeen a key and on-going research challenge within personal informatics [62, 115,203]. We wondered to what extent our approach of simple self-tracking elementsincluded in a notification management system would generate deep reflection forusers motivated more by notification management than self-tracking in general.4.3 Design ProcessThe design process to create our prototype followed two main stages. In the firststage, we used a pre-study survey to generate notification management suggestionsthat our mixed-initiative design would present to users. In the second stage, weused iterative prototyping techniques to explore what potential data abstractionscould encourage users to reflect on notification use and how those reflective com-ponents could be included alongside our suggestions.4.3.1 Supplementary Survey to Generate a Corpus of UsefulSuggestionsIn order to evaluate whether PI visualizations encourage participants to engage innotification management, we needed a corpus of reasonable suggestions that par-ticipants could react to while reflecting. We had tentatively created a list of 20potentially useful notification management suggestions based on our analysis of86prior work on qualitative and quantitative trends in notification receptivity, as wellas our own personal experience [13, 160]. For example, prior research has shownsome users choose to hide their devices to avoid being distracted while working,so this list included silencing personal notifications in work hours [13, 160]. Other,more contextual suggestions were based on research about what notifications usersvalue, such as important work messages or those from close family members thatwe suggested to highlight with a special sound [13, 160]. Variations of these andother common notification preferences were explored and varied, such as by tryingmore specific versions that applied to different contexts. The full list of pre-studysurvey suggestions is shown in Appendix A.1 of supplementary materials. To en-sure that our longitudinal study participants would not be distracted from the vi-sualizations by obviously poor suggestions, we conducted a short supplementarysurvey to identify which of the 20 suggestions participants would find most useful.In total, this supplementary survey was distributed to 140 participants (meanage=24.9, SD=5.7; 73 students along with various forms of knowledge workerssuch as clerks, web developers, analysts or consultants) through convenience sam-pling on online recruiting forums and the university paid studies board asking “Forthe following list, think about whether or not you would accept such a suggestion.”Users could select “yes,” “no,” or “not sure” for each suggestion shown in Ap-pendix A.1 of supplementary materials. Ethical approval for the study was givenby our university research ethics board. Seven suggestions were candidates forinclusion in the longitudinal study because they met the following criteria: Theycould be mimicked with current phone settings; a minimum of 50% of the pre-study survey participants would accept them; and they changed notification deliv-ery modality rather than fully removing notifications. The final criterion essentiallyavoided fully filtering out important messages, and was deliberately a conservativeapproach. Refined versions of these seven suggestions are shown in Table 4.1 andwere displayed to users in the longitudinal study presented in Section 4.4.4.3.2 Iterative Refinement and Final Design of the Reflective SpringCleaning DesignOur design process to create the Reflective Spring Cleaning concept was highlyiterative through brainstorming, low-fi sketching, and wire framing. For example,87Table 4.1: The 7 notification management suggestions shown to participantsin the longitudinal studySuggestion Text Show to participants in longi-tudinal study ifHighlight messages from priority contacts >4 notifications per day frompriority contactsSilence personal notifications during workinghours>10% of notifications in workhours are personal contentHighlight work notifications <40% of notifications are fromworkSilence system notifications >3% of total notifications aresystem notificationsSilence social media notifications >4 social media notificationsper day with no senderSilence work notifications during off-work hours >10% of notifications in off-work hours are from workHighlight work notifications during off-workhours<10% of notifications in off-work hours are from workwe created a variety of sketches of possible categorizations of notifications basedon prior work about which notifications users are receptive to, or willing to receivedespite the distraction they cause. One such sketch is shown at the top of Figure4.1. We used simple visual encodings in each sketch, such as bar charts or time-lines, to reduce the barriers to conducting reflection and personalization [18, 166].For example, we explored breaking down notifications in a bar chart grouped byhow socially close the sender was to the recipient, or which context the notifica-tion was received in, because both factors strongly impact users’ willingness toreceive those notifications [13, 160]. Further, we explored a large number of pos-sible labels to communicate the often fuzzy boundaries between different groupsof people sending notifications. We conducted informal cognitive walkthroughswhere we role-played a user personalizing their notifications for the first time andviewed each sketch describing their notifications. For each sketch, we discussedwhat major takeaways a user would receive from viewing the graphs and whetherthose insights could impact the decision to accept the seven suggestions selectedfor use in the longitudinal study from our pre-study corpus [13, 160]. The keyhigh-level design insight from these walkthroughs was that it was difficult to under-88stand trends in a participant’s notifications if they were categorized on more thanone factor affecting notification receptivity all at once. For example, prior worksuggested differentiating urgent “Mission Critical” notifications from non-urgent“Nagging” notifications, but urgency reflects both variance in who sent the notifi-cation and in what context it was received [13]. We explored various approacheswhere these factors were split into several complementary but independent data ab-straction facets which became the basis for our “who-what-when” data abstraction.Examples are shown in Figure 4.1. The potential cognitive cost of viewing severalgraphs of notification data was a key reason our design approach settled on veryinfrequent personalization in the Spring Cleaning concept.To explore the value of the Reflective Spring Cleaning concept and our dataabstraction, we built a semi-functional HTML prototype leveraging pre-existingtools [251]. This allowed us to focus predominantly on the design of the prototype,rather than technical implementation of notification interception and logging whichhas been well explored in prior research [159, 160, 249, 251]. We built on priortools by processing the raw notification data from participants, logged using opensourced tools to roughly classify it into the categories visualized in our data ab-straction facets [251]. For example, the data did not include any predefined sendercategory and many messaging apps differed in how this data was reported, suchas in the ‘title’ attribute or just as part of the text body. Our prototype includes arough estimate of time spent reading notifications based on prior work and a shortnotification categorization form (around five minutes to complete) for participantsto identify work notifications, priority contacts and primary working hours.We piloted our longitudinal study (Section 4.4) with seven participants in orderto assess the seven suggestions from the pre-study survey. The pilot participantsviewed all seven particularly useful suggestions in the pre-study survey and weasked them to describe which were most relevant to their needs and why. We turnedthese insights into relevance guidelines for our program to know when a suggestionmight be useful. For example, we might suggest silencing personal notifications ifa substantial portion of their notifications in work hours are personal according to arough threshold, such as >10%. The final prototype would then show three to fourrelevant suggestions to each user in a random order based on the guidelines in Table4.1; at least three suggestions were presented to each user. The final prototype is89shown in Figure 4.3. The final data abstraction facets we showed to participantsare as given in Figure 4.2 and below.Figure 4.1: Examples of early design sketches for Reflective Spring Clean-ing. 1. Shows an early view of when notifications are received. 2.A later more refined sketch that splits apart the emerging abstractions,currently showing the From “Who” facet.WhatThis facet describes what types of apps sent the participant notifications and isvisualized with a bar chart of the categories. Categorizations of apps have beenmade in both industry and academia [160, 214]. For example, the Google PlayStore lists 26 categories of apps to download [88]. Through our informal cognitivewalkthroughs we explored what insights participants might gain from reflecting onthese commercial breakdowns of apps, but many of the categories were assignedfor apps that rarely send notifications. Our final design was inspired by the typesof apps that Pielot reported sent the most notifications in their large-scale obser-vational studies of notifications from over 40,000 users, as it remains the mostcomprehensive and inclusive study of notification usage [191, 214]. The “What”facet includes 11 categories (System, Social media, Games, Email, Text, Voice,Calendar, News, Tools, Market and Other), as shown in Appendix A.1.90WhoThis facet describes the types of people who sent the participant notifications basedon prior work exploring notification receptivity and social relationships [13, 160,172]. Social relationships are often complex and individual, so we prompted par-ticipants to describe which contacts they valued most [13, 160, 172]. The finalcategories are: A group of important personal contacts provided by participantscalled “Priority Contacts.” “Work Contacts” are identified by notifications partici-pants receive from a work messaging app, email address, or work contacts such asa boss. Remaining notifications that identify a sender called “Other Contacts”. Allother notifications are called “No Sender Given.”WhenThis facet describes the notifications participants received while working or not,which strongly impacts notification receptivity [13, 70, 160, 191]. Our “When”categorization split notifications into two contexts divided by two types of notifi-cations. The contexts are: “Primary Work Hours”, as defined by the participantfor each day of the week through a form completed just before Spring Cleaning,and it’s complement context, “Off-work hours.” These contexts are further brokendown by two types of notification content: “Work Content” and “Personal Con-tent.”Lastly, we enhanced the visualizations of these facets by allowing participantsto compare the different weeks of data as shown in Figure 4.3. Each visualizationallows participants to click on the categories to refine the data shown. For example,if users clicked on the “Other Contacts” category in the “Who” facet they sawa bar chart of all the senders that were not work or priority. In addition, whenusers clicked on the suggestions within the interface, we used linked-highlightingthrough a black bar overlaid each category within each facet to display how manynotifications would have been impacted by the suggestion.91Figure 4.2: The three facets of our data abstraction. Red bars are all 0 becausethis user is not currently comparing different weeks of data.Figure 4.3: The final Reflective Spring Cleaning Interface showing real an-nonymized data. Here the user is viewing the “What” facet, compar-ing Week 1 to Week 3 and has clicked the “silence personal notifica-tions during working hours” suggestion indicated by the red arrow. Theblack bars indicate the notifications that would have been impacted bythe clicked suggestion had the personalization already been in place; thered bars show what notifications were received in Week 3 after applyingthe suggestion.924.4 Methods for the Longitudinal Study of ReflectiveSpring CleaningTo understand the usefulness of and motivating our approach of including PI visual-ization in notification personalization, we evaluated our Reflective Spring Cleaningprototype in a four-week longitudinal study.4.4.1 Participants & RecruitmentParticipants were recruited through online calls for participation on classified fo-rums and university paid studies boards using convenience sampling. Ethics ap-proval for the study was given by our university research ethics board. Techno-logical restrictions meant participants needed to use Android devices and be ableto identify an email account, messaging app or set of contacts primarily devotedto work so we could properly process their data. Several Android phone brandsincluding Xiaomi and Huawei were excluded because of known issues with thelogging app that made data gathering inconsistent. We also required participantsto be 18+ years of age, speak English and “feel they receive a lot of notificationsand might be interested in better managing them.” This requirement targeted smart-phone users who could be open to the idea of notification personalization and arethe key demographic for notification management.Our final participant sample, shown in Table 4.2, included data from 21 (meanage= 26.5, SD=8.5) participants, nine of whom completed the optional email fol-low up. One participant dropped out due to technical issues with data loggingand another additional participant dropped out of the study due to their concernsthat the suggestion they wanted to apply, silencing non-work notifications, couldinterrupt their small business if implemented with current settings. They are notincluded in our dataset. As has been observed in prior work, recruitment from uni-versity paid studies boards run by the psychology department can be gender biasedin favour of women, which was the case in this study with four men and 18 womenparticipating [178].Our recruitment criteria seeking participants who were somewhat dissatisfiedwith their notifications led to our participants being relatively engaged in notifi-cation management. 13 participants reported at least occasionally personalizing93Figure 4.4: Overview of the procedure for the longitudinal reflection study.notifications through the phone’s notification settings, do-not-disturb mode, or astay focused app. Of the remaining nine participants, seven described occasionallyuninstalling apps that send particularly annoying notifications such as games theyno longer play, while the remaining two did not manage notifications. While mostparticipants described having mildly negative experiences with their notifications,several participants felt overwhelmed and kept their devices permanently on silent.While we did not have explicit recruitment quotas, we did make a conscious effortto recruit a roughly even split of participants who are students and professionals(11 students, 10 professionals) to evaluate our design with a broad set of users.We note that these groups did not specifically map to any age ranges so they arenot the same as in Chapter 3. Of the professionals, their occupations varied andincluded: mechanic, research coordinator, copywriter, admissions officer, adminassistant, teacher, and territory manager.4.4.2 ProcedureAn overview of the procedure is shown in Figure 4.4.Step 1. Two Weeks of Passive Logging: After confirming eligibility for par-ticipation via email, participants were instructed to install the notification loggingapp [249]. Participants were then instructed to use their phone normally while thesystem passively collected data on their notification use. Passive data collectionlasted two weeks and we checked data was being recorded properly the day afterinstallation and half-way through the period.Step 2. Spring Cleaning Session: Participants were asked to manually exportthe log data from their devices. The next day participants participated in the SpringCleaning session via Zoom, together with a research team member. After usingour notification categorization form to provide a work email, priority contacts, and94Table 4.2: Demographics and device usage data from all 21 participants inStudy 3. Participants 8 and 17 dropped out and are not included in thisdata. Amount of phone usage and expertise with mobile devices wereself described by participants.P# Age Gender Occupation Phone use(hrs/day)expertise1 29 female student - grad 8+ hours intermediate2 19 female student - ugrad 5-7 hours expert3 18 female student - ugrad 3-4 hours intermediate4 26 female student - grad 8+ hours expert5 25 female student - grad 3-4 hours expert6 32 female student - grad 8+ hours expert7 22 female student - grad 3-4 hours expert9 24 female student-grad 3-4 hours expert10 24 female research coordinator 5-7 hours intermediate11 21 male student - ugrad 8+ hours expert12 22 male student - grad 3-4 hours intermediate13 24 female school liason coordinator 5-7 hours intermediate14 26 female copyrighter and editor 8+ hours expert15 18 male student - ugrad 5-7 hours expert16 24 female student - grad 5-7 hours expert18 25 female counsellor 5-7 hours expert19 35 female teacher 3-4 hours intermediate20 22 female admin assistant 8+ hours intermediate21 19 female student - ugrad 5-7 hours expert22 49 female territory manager 3-4 hours expert23 32 female tutor 1-2 hours intermediate95working hours, participants were shown the Reflective Spring Cleaning prototypevia the researcher’s shared screen, which they could interact with remotely. Toencourage initial reflection, the suggestions for notification management were hid-den to start and participants were instructed to explore and talk-aloud anything thatwas interesting or surprising in their data. Initial reflection took between threeand seven minutes (avg 4 min 51 sec). Participants then completed a brief semi-structured interview on the things they learned about their notification use and theirimpressions of the data abstraction facets. Afterwards, participants were shownthe notification management suggestions and asked if they wanted to apply any ofthem. If so, the researcher guided them through the instructions to set up that sug-gestion on their device. Participants were also asked if they had any other changesthey wanted to make to their notification settings and to explain why they wantedto change those settings. Participants who did think of such changes were asked tomake them during or after the session. If asked by the participant, the interviewerdescribed where to find the settings they wanted.Step 3. Two More Weeks of Passive Logging: Participants were instructedto use their phone normally for the two weeks of passive logging after the firstreflection session. Participants were instructed to feel free to make any furtherchanges they wanted to their notification settings, but to keep track of any changesmade.Step 4. Debriefing Interview Involving Reflection: Participants’ notificationdata was exported again. Participants completed a final semi-structured reflectionsession via Zoom. Participants described their impressions of how their notificationexperience differed since personalizing. Participants then explored all the samevisualizations as before with updated data. Participants were informed they coulduse the drop down bars at the top of the visualizations to compare different weeksof data, but were not required to do so. Figure 4.3 shows a participant comparingtheir data from before Spring Cleaning (blue) to after Spring Cleaning (red) on thewhat abstraction. The black highlights show the impact the indicated suggestionwas predicted to have before personalizing. After reflecting, participants discussedthe design as a whole, the effectiveness of the personalizations they applied, andwhether they might keep using them.Step 5. Brief Optional Email Follow-up: A month after the final reflection96session we sent participants a two question optional follow-up by email. Partic-ipants described whether they continued to use the personalizations they appliedand whether they had made any further changes to notification settings.Video recordings of the Spring Cleaning session (Step 2) and the final debrief-ing interview (Step 4) were transcribed verbatim. We also recorded demographicsdata and asked participants to self-rate expertise and usage of various devices in-cluding phones and tablets in Step 2.4.4.3 Data AnalysisThematic analysis of transcript data progressed iteratively in two main phases, un-der the framework by Braun and Clarke [30]. Our first phase of analysis wasmore targeted and focused on understanding what sorts of insights participantswere gaining about their notification use while reflecting on the data abstraction.Understanding what insights were generated was key, as the usefulness of Reflec-tive Spring Cleaning was dependent on whether generated insights were useful forpersonalization. In this phase, the lead researcher would manually scrub throughtranscripts looking for times participants talked aloud during the reflection ses-sion or described learning something about their notifications in comments. Thesepotential insights were then categorized by whatever design element may have gen-erated them, and qualitative codes were applied to describe what type of commentor insight they were. For example, several insights were described as “surpriseover who sends the most messages.” Special focus was given to categorizing anycomment where participants either explicitly or implicitly implied their notifica-tion personalization decisions had changed. We then analyzed these codes intocandidate themes of insights gained.A second phase of thematic analysis was then conducted using inductive opencoding. We focused on a descriptive analysis of the overall impact of the design,concerns participants had over notifications, and how their experience with notifi-cations shifted over the course of the study. The primary consideration for relia-bility was increasing the nuance of the analysis through multiple perspectives byhaving input at various stages from as many researchers as possible, all of whomcould use their own unique experiences and perspectives on how they use and man-97age notifications. The goal of this iterative analysis was to incorporate these mul-tiple perspectives to approach crystallization [30, 58]. In total, the lead researchercoded every transcript, two other researchers independently coded three transcriptseach, and a final researcher coded a further two transcripts. Candidate themeswere then developed and refined through discussion in a series of meetings of theresearch group, along with several other HCI faculty in associated institutions.4.5 FindingsWe first provide an overview of how participants personalized with the SpringCleaning tool and it’s impact on their notification experience. We then describethrough three qualitative themes the impact of reflection on notification manage-ment and participants’ motivation to manage notifications. Theme 1 describes thereflection engaged in by participants. Theme 2 describes their motivations to con-duct this reflection. Theme 3 describes their willingness to engage in further noti-fication management after Spring Cleaning.4.5.1 Overall Usage of Spring Cleaning ToolWe give an overview of how the Spring Cleaning tool was used by participants.Further, we describe how these changes influenced participants’ experiences withnotifications and improved that experience for the majority.All participants conducted personalization, 9 accepting our suggestions and12 creating their own personalizationsIn the personalization session 9/21 participants applied one of our suggestions totheir phone (four highlighted work notifications with a special sound, five silencedwork or personal notifications at suggested times). The rate of acceptance of oursuggestions was lower in this study than in the pre-study survey where, for exam-ple, 75% of participants wanted to silence non-work notifications during workinghours. Some of this difference is likely because the pre-study survey did not requireparticipants to actually try the personalization and risk possible errors.All participants who did not apply one of our suggestions (12/21) applied otherpersonalizations, usually based on insights they gained while reflecting. Of these9812 participants, six reported that the “What” facet showed too many notificationscoming from some sources. They described uninstalling apps or notification ser-vices such as games or social media from those sources in the personalization ses-sion. Two participants reported the “Who” facet showed too many notificationsfrom certain types of group chats, and silenced them or set them to only notify theparticipant when tagged. P3 - “In the past I didn’t silence [group chats], becausethey’re not that bothersome... But then seeing them visualized... they take up alot of space and I should definitely change the settings.” The remaining four par-ticipants reported their reflection prompted them to do a broader cleanup of theirnotifications over the next few days. P15 - “I blocked all the notifications thatweren’t useful for me. Social media, some of the emails and all the other stuff thatI could disable I disabled.” No participant who accepted one of our suggestionsapplied other personalizations.The majority felt the personalization conducted improved their experienceThe notification management suggestions we provided participants changed themodality of how notifications were delivered but did not change the total numberof notifications delivered, so we did not anticipate significant quantitative differ-ences and none were observed as shown in Figure 4.5. Descriptively, participantsreceived fewer notifications in the two weeks after Spring Cleaning (avg 3669)compared to the two weeks before Spring Cleaning (avg 3989). However, naturalvariation in the data set was high (SD 2451), such as work notifications shiftingduring holidays or around deadlines. P9 - “I’m not working right now, just finish-ing up exams... So I haven’t been receiving [as many] messages related to schoolor work anyways.”It was encouraging to see that more than half the participants (13/21) reported asubjective improvement in their notification experience after Spring Cleaning andthat this group was evenly split between those accepting our suggestions (6) andthose applying their own personalizations (7). Of these 13 participants who re-ported improved notification experience, four described being less distracted andmore productive, three said they were less overwhelmed and stressed by notifica-tions, two described having better boundaries between work and personal time, and99Figure 4.5: Average number of notifications received by participants brokendown by whether they accepted our suggestions or created their ownpersonalizations, with bars for standard deviation. Estimated averagenumber of notifications impacted by those personalizations is visualizedin grey.one described receiving fewer notifications not relevant to their current tasks. Theremaining three participants simply described that their notifications just felt bet-ter than normal. We estimate that the suggestions participants accepted impactedsimilar proportions of their notifications (avg 1644 or 31% of their notifications)to the personalizations participants created for themselves (avg 781 or 25% oftheir notifications). Overall, participants personalized an average of 28% of theirnotifications, which is comparable to prior mixed-initiative systems. For exam-ple, Prefminer reported filtering an average of 16% (12/71) of participants’ dailynotifications [159].While typically participants considered the overall improvements from SpringCleaning as small positive refinements, in some cases, this improvement was con-sidered very positive if the personalizations impacted a large and noticeable num-ber of notifications. In the most dramatic example, P22 - “cancelled every possible100notification I could” and observed a drop from 4335 notifications in the two weeksbefore using Spring Cleaning to 1545 in the two weeks after. We roughly estimate2250 notifications of the total 2790 reduction can be attributed to personalization.For example, several apps were uninstalled which removed all their notifications.While debriefing, P22 expressed “I’m blown away by how much just a couple sim-ple changes in those notifications can improve productivity and lighten the loada bit.” Of those participants who did not report improvement, they usually de-scribed the changes as being easy to overlook. For example, changing the ringtonefor work notifications wouldn’t matter if they always checked their phone when itbuzzed. Some participants also described that the changes were not noticeable dueto natural variation in their number of notifications. For example, P16 describedturning off system updates and weather reports they didn’t need, which removed1060 notifications, but their overall number of total notifications increased (6183before, 6782 after) due to greater messaging. P16 - “I was getting less from certainthings but in terms of overall effect, it didn’t seem very different.”Summary.All participants personalized their notifications with Reflective Spring Cleaningeither by accepting our suggestions or applying other personalizations inspired bytheir reflection. The majority felt their personalizations improved their notificationexperience.4.5.2 Critical reflection about notification usage was promptedParticipants frequently described observing surprising events or trends of notifica-tion use while interacting with the visualizations in the Spring Cleaning tool. Theseinsights prompted participants to critically reflect on how disruptive notificationswere in their everyday lives, especially by identifying that they were receivingmore unnecessary notifications than they expected. Further, participants linkedtheir enhanced understanding of the cost of notifications to their motivation to ap-ply personalizations in the study. Reflection also influenced which personalizationsthey thought would be effective to reduce the cost (RQ1).101High volume of unnecessary notifications to remove, often derived from the“What” facet, surprised participants.Most (18/21) participants felt they had underestimated the number of notificationsthey received, especially related to apps participants felt they didn’t need. For ex-ample, P22 noticed they received over 200 notifications from games in two weeksand remarked: “Now the sad part is it makes sense. I receive them multiple timesa day. I would love for them to be off my phone during the day.” Participantsfrequently made these observations about receiving a surprising amount of unnec-essary notifications while viewing the “What” facet (15/18). P7 - “The first thingI noticed is that I didn’t think I received this many text notifications... I’ll mutemost of them.” In addition to underestimating the overall number of unnecessarynotifications, participants also frequently reported realizing through the “What”facet that the largest sources of distracting notifications were different than theyexpected. P5 - “I thought with the amount of time I’ve been on social media Iwould have way more [social media notifications]...it’s kind of sad that I have somany system notifications.” Removing these unnecessary notifications was a keygoal of many personalizations participants created.Habits of notification use that contextualized these insights were identified,often through the “Who” and “When” facets.Participants described getting a better understanding of the context of their ownday-to-day habits of phone use through reflection on the “Who” and “When” facets.For example, P3 remarked while viewing the refined timeline of their notificationsin the “When” facet: “I guess Saturday is a day off. It would be kind of expected[to have more notifications] but I didn’t realize it was that big of a spike!” Fur-ther, the “Who” and “When” facets provided insights that tended to contextualizeproblems identified in “What” within a participant’s real-life habits. For example,potentially problematic trends, like seemingly excessive messaging in work hoursmight actually make sense if it was done in an appropriate context. P10 - “Who’sthat [sending me many texts]? Oh, that’s the person we give the blood to. It’sa work thing. Okay, so this isn’t horrible. I am pleasantly surprised because Ithought it could be much worse.”102Increased awareness of ability and need to remove unnecessary distractionParticipants often described how prior to the study they had a vague sense thatnotifications were distracting but were more of a necessary nuisance. P5 - “In thepast, I was not managing my notifications because I thought I didn’t let it get tome.” However, after participating in the study participants described being moreaware of the costs of receiving many notifications and that they had agency toremove unnecessary notifications rather than let them continue. P3 - “I’m moreaware of what customization is available to me, and that it only takes a little bit ofeffort to make quite a large difference.” Participants being more aware of how topersonalize is perhaps not surprising, given our prototype’s limitations and that weinstructed participants on how to mimic their desired personalizations using currentphone settings. However, we note it here because motivating users to exercise suchagency was a key goal of this work and similar personalization training tutorialscould be included in future Spring Cleaning systems. Further, some participantsreported that part of this agency was due to increased awareness of the problemfrom reflection rather than just being more trained on how to personalize. P23 - “Iguess I am aware of the things that I was seeing that I wasn’t aware I was seeing...how actually I had control of that so I didn’t have to look at it if I thought it wouldbe not very useful.”Reflection helped participants understand which personalizations wouldlikely reduce distractionWe saw several cases where participants changed their mind about whether to ac-cept suggestions while reflecting on their data. For example, P9, a student, was ini-tially hesitant about accepting our suggestion to silence work notifications duringoff-work hours but did so after remarking: “Looking at how many [work] notifica-tions I was getting... it’s not very many and they’re not very urgent... it’s usuallyjust [group projects] deciding when to zoom.” A few participants also reported howthe linked-highlighting of which notifications would have been impacted by thosesuggestions changed their decisions on whether to accept them. P2 - “When youget personalization suggestions [like that] you don’t take them seriously. Whenyou have it presented to you in a display, it convinces you that you do get a lot.103[You’re] more incentivised to change your settings.”Summary.Reflection guided by our data abstractions prompted participants to critically ex-amine their notification usage habits. Participants identified that they were receiv-ing more unnecessary notifications than they expected and described having anincreased awareness of their ability and need to personalize notifications.4.5.3 Individual differences in motivations to reflectMost participants described having a core utilitarian motivation to reflect in orderto better manage notifications. However, individual participants further describedhaving specific secondary motivations such as brief intrinisic curiosity and a desireto minimize stress from distraction.Identifying accumulated issues provided utilitarian value.The vast majority (20/21) of participants reflected in the final debriefing interviewthat they saw utilitarian value in Spring Cleaning and would likely use somethingsimilar if it was released commercially or as part of the next OS update. For exam-ple, P7 said “It would be very useful, especially to know how many notifications[you received] and how distracting those notifications can be.” At first glancethe magnitude of this finding seemed possibly inconsistent with our overview thatsuggested only the slight majority of participants reported improved notificationexperience. What we heard from our participants helped to clarify that the reflec-tion itself was valuable for utilitarian reasons. Many participants described howbetter understanding their often hard to track notification habits helped them makeinformed choices about how they managed notifications. P3 - “It’s hard to graspwhat notifications you’re getting unless you see it visualized like this. Otherwise,when you get a new phone you have to go in and customize everything, it’s a lot ofwork.” A few participants also described how after personalizing, later reflectionmight be useful to monitor ongoing distraction. P20 - “I would use it to make sureI’m on track with my notifications and that I’m not getting unnecessary ones.”104Curiosity was a brief intrinsic motivation to reflectWhile we did not explicitly ask participants if they would use the design withoutthe suggestions, many participants described how they wanted to infrequently usethe visualizations independent of notification management because they were sim-ply curious about how they used notifications and were interested to know more.P10 - “I’m just a data nerd, I like seeing it laid out like this. When you get 20 noti-fications a day it doesn’t seem that overwhelming but then looking at 516 in a weekit seems a bit crazy.” Meanwhile P1 described how they would view such visual-izations on their own because it was interesting to know where they were spendingtheir time: “I definitely find it interesting...It’s really hard to realize [where you’respending your time], if you don’t track your notifications.” Further, some partici-pants described wanting to Spring Clean again sometime months into the future tounderstand how their needs and habits shift over time. P23 - “People’s schedulesand their preferences change... I would say that I would [Spring Clean] maybeevery two months.”Notifications were a source of stress to minimizeMany participants described how they perceived notifications as a frequent sourceof stress or anxiety in their lives. P3 - “Email notifications are usually somethingimportant. [They] give me a shock of anxiety because I’m like ’oh, my God, Ihave to check what is going on’.” Some participants described how a key outcomeof reflection was quantifying the extent of this stress. P13 - “It’s a bit stressfulnow that I see it. That’s a lot more [notifications] than I thought [I received].Beforehand, it didn’t stress me out.” Further, some participants described wantingto minimize their number of unnecessary notifications to reduce this feeling ofstress. P21 - “It starts to feel like anxiety when you open your phone and see a tonof notifications that you don’t need.”Summary.Participants displayed several different motivations for reflecting on their notifica-tion use with the Spring Cleaning design. Identifying accumulated issues providedutilitarian value while Spring Cleaning, curiosity lead to brief intrinsic interest to105understand participants’ notification habits independent from personalization, andreflection helped participants quantify and want to minimize the stress of receivingnotifications.4.5.4 Degree to which needs were met impacted continuedengagementParticipants described how after becoming more motivated to manage notificationsby Reflective Spring Cleaning, they usually felt content with how well their refinedpreferences met their needs. Other participants who actively managed notificationsprior to the study reported making further changes to maintain their new experi-ence, but had little desire to reflect further in the short term.Participants described contentment with refined preferences.After experiencing their refined notification preferences for two weeks, relativelyfew (5/21) participants felt there were any further changes they wanted to maketo their notifications. P2 - “I can’t think of any modifications that I would makeas of now.” Most participants described being generally content with how welltheir notification preferences were met until something substantial changed in theirnotification use. P4 - “Maybe once I’m done with college I might change my no-tification settings again. For now, I don’t have any plans.” Further, this sense ofcontentment with their current notification management practices appeared to per-sist amongst the nine participants who completed our follow-up survey one monthafter the final interview. P22-“YES [I have continued using my new settings] -I love having less notifications - it’s awesome!” Of the nine participants who re-sponded to the follow-up, five chose not to make further changes. P21 - “I have notdone anything to change my notifications because I feel quite content with them.”Participants actively managing notifications described maintaining refinedexperience but limited interest in more frequent reflection.Some participants who actively managed notifications prior to Spring Cleaning de-scribed more complex patterns of notification management afterward. While theyoften described being content with how well their major concerns were addressed106by their personalizations, they still considered conducting further notification man-agement. For example, in the study P14 personalized her phone to ignore workIMs in the evening, but was still considering whether she should do the same forwork emails. P14 - “I’m still debating if I should hold on to the emails. I haven’tmade a decision about that.” Of the nine participants who responded to the fol-lowup survey, five had actively managed notifications before Spring Cleaning andfour of those five continued making more changes afterwards. Two described up-dating the preferences of new apps they installed and two made more substantialchanges to address further needs. P11 - “I muted most social media apps becauseI found they became too distracting in day to day life. Exception for some closecontacts.” However, even participants who continued to actively manage notifi-cations described how they felt that reflection on notification use had diminishingreturns. P10 - “I don’t know if there’s enough detail in notification management tofine tune things to a really meaningful degree every six months, at least for me...I’mconsistent in my social media use.” Diminishing returns on the utility of reflectionled even the most engaged users who actively managed notifications before andafter Spring Cleaning to suggest they didn’t need to reflect further for at least afew more months. P15 - “I would rather do it on a three month basis...Six months,there would be too much data.”Summary.Individual differences in participants’ prior notification management practices in-fluenced the degree to which participants notification management needs were metby Spring Cleaning and their notification management practices after Spring Clean-ing. While most participants described feeling content with the new state of theirnotification preferences, some participants who actively managed notifications inthe past continued managing notifications to maintain and refine their experience.All participants expressed limited interest in more frequent reflection.4.6 DiscussionWe focus our discussion on analyzing the effects of combining simplified personalinformatics with notification management and the potential implications this ap-107proach might have for future reflective design or personalization research. We fur-ther present design recommendations for future versions of the Reflective SpringCleaning concept that can be leveraged in other PI systems where software person-alization may help refine user behavior.4.6.1 Targeted infrequent reflection could better engage lessintrinsically motivated usersPrior research has shown that participants who are not intrinsically motivated toself-track, such as those that need to manage medical conditions like diabetes,often quickly become discouraged and stop using PI tools [115, 204, 259]. Therelatively recent lived-informatics model of PI has already started to frame lapsesin tracking as a normal and positive stage of self-tracking workflow if it aligns withparticipant motivations [62, 64, 73, 115, 204, 240]. For example, participants mayhappily disengage with a self-tracking tool after a few weeks of tracking that satis-fies their curiosity and helped them explore the problem [63]. However, by contrastit could be a negative experience if the user does not find the reflection helped orwas too cumbersome to complete and led to a feeling of failure [63, 204]. Wesaw that despite being relatively engaged to manage notifications, our participantsonly felt intrinsic motivation to reflect very infrequently. For example, many par-ticipants described that a very infrequent reflection session during Spring Cleaningprovided utilitarian value to understand accumulated changes in their notificationuse. Our findings suggest that future PI research, beyond notification management,could even frame lapses in tracking between targeted reflection sessions as a de-sign goal that may align the intensity of self-tracking with the motivations of moregeneral users who do not have strong intrinsic motivation to frequently self-track[63, 115, 204]. For example, future web browsers might explore adding a personalinformatics element that prompts users to engage in a yearly five to ten minutereflection session targeted to help users quickly understand what personal data isbeing gathered about them, or who has access. Addressing the privacy concernsaround extensive self-tracking is an on-going challenge in personal informatics lit-erature, which might help motivate such a system [64].1084.6.2 Associating PI with software personalizations enhances designgoals for both PI and personalizationPrior research has identified two major issues in PI and personalization literature:1. Users of personalization systems often lack reasons or motivation to overcomebarriers to refining their settings [18, 214, 252]. 2. Users who approach reflectionfrom the perspective of behavior change often want clear actionable takeawaysfrom their reflection in order to refine their behavior, which meta studies havesuggested are often hard to identify [64, 240]. Our findings show that the novelcombination of PI and personalization effectively makes complementary progresson these issues. Reflection on notifications using our data abstractions revealed thesurprising and unconscious costs of distraction to our users, which provided a trig-ger to personalize [18]. Meanwhile, the ability to personalize notification settingsto address the issues identified through reflection provided clear takeaways to re-flection. Future software personalization systems should explore ways to visualizewhatever behavioral data they already collect in order to help guide personalizationdecisions. For example, recent online learning platforms have begun to leveragestudent data to allow educators to analyze and personalize learning activities to suittheir students’ preferred learning styles [6, 15]. We see opportunities to open thisdata up to students themselves so they can track and analyze how they best learnnew material and perhaps see recommendations for additional activities tailored tothose preferences. Further, we showed that 5-10 minutes of reflection on simplifiedPI did not appear to be a substantial barrier to conducting a personalization session.Though we did not assess precisely how frequently reflection should be done, ourdata suggests that reflection would be done relatively infrequently, such as at thestart of term to identify what courses students might find the most challenging.4.6.3 Notification Spring Cleaning could be incorporated as a part ofholistic wellness systemsMany participants described engaging with Spring Cleaning to reduce unneces-sary distraction and improve productivity. Recent research has identified that toolsshould not focus on a narrow conception of productivity but rather embrace themore holistic realities of how modern work must interweave work and personal109tasks for wellness [13, 92, 160, 254]. The stress caused by distraction [134] wasa key motivating factor behind our work and several of our participants wantedto manage notifications to reduce stress. They described appreciating how usingSpring Cleaning pushed them to establish clearer boundaries against work mes-sages received in personal time. The potential for targeted notification personaliza-tion to encourage such reflection suggests that personal informatics systems thatsupport users to track and refine their mental health might benefit from a notifi-cation Spring Cleaning feature [142, 165, 168]. Future work could explore howusers of these tools, who might be more concerned about maintaining wellnessthan general notification management, might differ in how they personalize withSpring Cleaning. For example, a few of our participants described how receivinga lot of personal chat messages at work was helpful and positive as long as it oc-curred during appropriate breaks, such as during lunch. Expanding the participantpool to target users who already track and reflect on their holistic well-being couldclarify the extent wellness considerations influence data interpretation. Further, ifa short Spring Cleaning session provides interesting and useful insights for userswho already self-track, those tools might suggest integrating notification data intotheir regular wellness tracking habits. Prior work in health and wellness systemshave used various forms of guided self-experimentation to help users explore whatdata to integrate into their self-tracking systems, though this can be a time intensiveand difficult process [119, 121, 140]. Further work is needed to explore the extentto which a Spring Cleaning session as a prelude to more intensive tracking mightcompare to these self-experimentation systems [64].4.6.4 Opportunities remain to refine Reflective Spring CleaningPrior work has explored a variety of approaches to mixed-initiative notificationmanagement that all come with many design trade-offs when compared to SpringCleaning [16, 33, 96, 159]. For example, notification management suggestionslike Prefminer cause distraction from the relatively frequent suggestions them-selves and do not contextualize the suggestions with notification data [34, 159].Suggestion rationales have been explored in other domains to help provide thiscontext, but aren’t read by many users [33]. Comparison was supported by the110study design, but our data suggests the Reflective Spring Cleaning design may bean effective alternative to traditional mixed-initiative suggestions for notification;participants personalized an average of 28% of their notifications and the majorityof participants reported improved experience. However, there is room to refine theReflective Spring Cleaning design and future comparative work should continue toexplore how reflection best fits into notification management workflows.About half of our participants independently created their own personaliza-tions, often directly inspired by insights gained while reflecting. However, studyparticipation involved some training on how to personalize notifications, and ev-eryday users may be less able to make such personalizations themselves. Designrevisions to Spring Cleaning should enable relatively easy independent personal-ization, such as directly linking the visualizations to manual notification person-alization. The easiest implementation could be a simple clickable link anywhereapps are explicitly referenced in the visualizations to the appropriate settings panelfor those apps. For example, if users click the bar for Twitter in the “What” facet,the system will open up the Twitter notification settings panel. Linking the visual-izations to current phone settings would reduce the knowledge barriers to findingthe appropriate settings panels, even if the more complex personalizations such asthe rules-based do-not-disturb feature used to implement our suggestions wouldneed additional support. Even more refined versions of Reflective Spring Cleaningwould enable users to directly personalize notifications from the graphs. For exam-ple, Reflective Spring Cleaning could offer the ability to select bars in the graphsto define a set of notifications to personalize, then assign a rule for how they shouldbe delivered such as highlight, silence or batched delivery (without having to dothis in a settings panel).4.7 LimitationsThe main limitation of our work is that the prototype was relatively low fidelity.While our prototyping approach allowed us to focus on the design of the reflectivecomponents, the suggestions were not overly personalized to each user. The secondlimitation in our method is that participants may have generated additional insightsby self-reflecting in order to answer our interview questions. We were careful to111observe participants’ initial reactions to their data without further prompting. How-ever, discussing what they learned during reflection in the interview may have im-pacted participants’ personalization decisions. We acknowledge that participantsin the wild would not be explicitly prompted to engage in this higher-level reflec-tion of their takeaways from the visualizations. We’ve noted that our conveniencesampling of participants was intentionally limited to those who are relatively en-gaged in notification management. Future work remains to extend our sample toa wider population to explore how different backgrounds and motivations mightimpact how users react to Reflective Spring Cleaning. Adding member checkingto the study protocol would also enhance validity.4.8 ConclusionIn this chapter we explored a novel approach to notification management called Re-flective Spring Cleaning. Rather than prior mixed-initiative systems that interruptusers with suggestions for notification management, Reflective Spring Cleaninginfrequently engages users in a longer reflection and personalization session con-textualized by visualizations of notification usage data to encourage reflection onthe impact of the notifications they receive. We explored Reflective Spring Clean-ing in a four week qualitative study where participants passively tracked their no-tifications, personalized with the Spring Cleaning prototype, and later reflected onhow their notification experience changed. Participants often critically reflectedon their notification usage. This critical reflection helped many participants real-ize their own need and ability to control notifications, which encouraged many topersonalize. Spring Cleaning improved the notification experience of the major-ity of our participants, often by reducing stress or distraction. Our design conceptincludes a novel “who-what-when” data abstraction that simplifies and contextu-alizes key variation within a user’s notifications. We found this data abstractionhelped to guide participants’ reflection and supported different user motivationsto reflect. We discuss various design implications for future personal informaticsand personalization tools. For example, the combination of personalization withpersonal informatics helped give users clear actionable takeaways from reflection,which has been a challenge in other personal informatics tools. We suggest that112similar approaches could be explored in other personal informatics systems, suchas a wellness tool prompting users to very infrequently reflect on how they usetheir mobile devices. Reflective Spring Cleaning takes a novel approach to notifi-cation management which continues to be a pressing issue in both academic andindustry discourse. Our work brings us an important step forward in empoweringthe millions of people impacted by disruptive notifications to manage where theirattention is directed.113Chapter 5ConclusionIn this dissertation, we presented results from three studies involving 71 interviewsand 140 survey respondents that collectively described users’ notification man-agement practices and needs along with key design opportunities to better addressthem. We started by investigating individual differences in the notification manage-ment needs and practices of an understudied population (Chapter 2). We identifiedthat many users desired a sense of control over their attention that was difficultto achieve with current notification management tools. We then explored five de-sign concepts to better support proactive control of notifications (Chapter 3). Thenwe used insights from user reactions to these design concepts to guide the cre-ation of a new notification management approach, Reflective Spring Cleaning, thatencourages control over notifications through infrequent personalization sessionssupported by targeted reflection (Chapter 4).5.1 Summary of Results and ContributionsIn this section, we summarize the key results and contributions of the dissertation.5.1.1 Refined prior characterizations of notification managementpractices and needsSeveral studies have characterized users’ preferences for notifications and practicesto manage them, which we refined in Chapter 2 [13, 160, 214, 252]. Our first sub-114stantive contribution was extending and refining these characterizations. For ex-ample, we found that a user’s interest or enjoyment of the content of a notificationwas only relevant as a tie-breaker between equally urgent notifications and we clar-ified a user’s model of which notifications should be able to be personalized [13].Further, by specifically exploring the needs of older adults, a key but under-studiedgroup of notification users, we raised the importance of providing users a senseof control over their attention. The importance of a sense of control, which usersfelt was already lacking in current manual notification personalization tools, wasa subtle but key result generated by this work with strong implications for othernotification researchers. For example, many prior works reduced the barriers tomanaging notifications, such as user effort, through machine learning that predictsand adapts to trends in what notifications users value at the cost of reduced usercontrol [70, 91, 101, 105, 207]. By contrast, the need for user control and agencyled our design work in Chapter 3 to focus less on reducing the barriers to notifi-cation management and more on making the notification management process feelexpressive, useful and worth the effort. We felt these design goals aligned moreeffectively with user needs in the space. In addition, while our later design workshifted our focus away from the specific needs of older adults towards a generalnotification system, we did see indications in Chapter 3 that age-associated indi-vidual differences are worth further investigation in future work. For example, wesaw early indications that shifting work structures from students to professionalsinfluenced users’ notification needs, which may offer new opportunities for notifi-cation management design.5.1.2 Design elements, concepts, and axes to support control overnotificationsIn Chapter 3, we leveraged our insights into notification management needs andpractices to conduct a thorough exploration of potential notification managementdesign elements and approaches that could provide a sense of control over notifica-tions. To generate these concepts we characterized four axes of design for how no-tification management tools could include users in notification management. Theseaxes included: Targetable Notifications (the types of notifications users can person-alize), Possible Actions (the actions that can be taken on the notifications the user115targets for personalization), Decision Support (the design elements used to helpthe user decide which personalizations to apply to their devices), and Degree ofAutomation (the degree to which the system automatically manages notifications).We created five design concepts exploring how users might better proactively man-age notifications, such as through the addition of decision support or more carefulselection of design elements to communicate how notifications should be person-alized. We used user reactions to these concepts to refine the potential approachesto notification management in order to understand trade-offs in the extent to whichthey allow users to express and refine their preferences. We used these insightsto create a novel notification management prototype in Chapter 4. In addition, wepresent the design concepts and axes as exemplars for how users might proactivelymanage notifications in future tools. These exemplars should be useful for appdevelopers deciding which notification personalizations to support on top of thedefault operating system settings.5.1.3 Novel design approach to notification managementThe primary contribution of this dissertation involved synthesizing all of these in-sights to identify a novel and effective approach to notification management thatwe present in Chapter 4: Reflective Spring Cleaning. Rather than just trying toreduce the barriers to notification management through machine support, this ap-proach also encourages users to manage notifications through targeted reflectionon their impact. We built a cohesive, proof-of-concept system to explore how re-flection could be added to a notification management tool. Our system included thefollowing design elements: 1.Visualizations of a user’s notification usage data. 2.Personalized suggestions for notification management. 3. Linked highlighting toshow how notifications will be impacted by the suggestions. 4. A prompt to SpringClean very infrequently, such as every six months. Through a deep, longitudinalevaluation of this concept with 21 participants, we demonstrated how ReflectiveSpring Cleaning can improve many users’ notification experiences and raise theirawareness of their ability and need to personalize notifications. There remains sub-stantial design and algorithmic work to refine this concept before it could conceiv-ably be deployed in industry. However, we demonstrated that this concept makes116progress towards two major design goals in this space. First, by keeping reflectionand personalization targeted and very infrequent, such as around five minutes ev-ery six months, this approach keeps the barriers to notification management small.Second, our evaluation showed the concept both encouraged and supported partic-ipants to control their notifications through critical reflection on their notificationuse.5.1.4 Design directions based on empirical workThroughout the dissertation, we have explored and proposed various design ap-proaches that could help users manage their notifications. We see these designapproaches, such as the Reflective Spring Cleaning concept, as part of a broaderset of design implications that collectively reframe the goals and requirements forfuture notification management research, rather than simply providing a single de-sign approach to explore further.We would argue that, when notification management research evaluates newtechniques, it has a tendency to focus heavily on objective measures of notifica-tion management utility [27]. Specifically, measures such as prediction accuracy,time to respond to notifications, number of notifications, time to return to task, orcognitive load are often considered the primary indicators of successful notifica-tion management systems [112, 132, 144, 157, 159, 200, 236]. For example, thisyear’s CHI conference published two papers, aside from our work, presenting newnotification management systems [41, 144]. Both used machine learning to auto-matically suppress or defer notifications to opportune moments and argued theywere effective due to high accuracy for predicting which notifications to deliverimmediately.We argue, however, that this focus on objective measures of utility can obscurehow useful users actually find these systems. We are not raising a new concern;Booker et al. argued in 2004 that overall qualitative impressions of notificationinterfaces were more conclusive for their usefulness than quantitative performance[27]. However, our research has renewed this concern throughout Chapter 2, Chap-ter 3, and Chapter 4 where we identified a need for control over user attention that islargely orthogonal from objective notification management performance. Further,117in Chapter 3, we showed that adaptive notification management designs, whichare common in recent notification management work, did not provide this sense ofcontrol despite high quantitative performance in other papers [41, 144].We argue that future notification management research needs to refocus its at-tention towards the holistic, qualitative experience of using these notification man-agement tools, especially how best to support user control and agency. We seeopportunities for a broad set of other design approaches to explore new ways ofproactively managing notifications to achieve this goal. For example, having usersproactively defining logical rules for notification deferral was explored in Chapter3 and in prior work several years ago [159, 250] but is worth exploring further inmore depth. In addition, we also see value in research that explores adding designelements in automated deferment systems that could provide the missing sense ofcontrol. For example, users might be provided with periodic reports of what hasbeen deferred and asked to provide input on which other notifications are not worthreceiving immediately.5.2 Limitations and Notes for InterpretationBelow we describe the key limitations of this work and how it should be inter-preted, such as the strength of the claims within our constructivist framing, impactof spring cleaning on notification experience, and the extent to which our samplewas representative of older adults.5.2.1 Generalizability of our central claimsA key implication of using a constructivist epistemology in this work is that we donot claim to show generalized statistical or subjective effects on a broader popu-lation through a large representative sample. Instead, we frame the contributionsof this work as generating theory for understanding [202]. By collecting, analyz-ing and categorizing common user experiences into useful chunks we create theorythat can help designers understand where notification management systems can failtheir users. For example, our results allowed us to describe the complex categoriesof notifications users perceive in Chapter 2, and the design axes for how to includeusers in notification management in Chapter 3. When interpreted in combination118with prior notification management research, this knowledge helps designers rea-son about how notification management tools could better serve their users, likewe did while designing the Reflective Spring Cleaning concept in Chapter 4.Here we state the claims we are able to make within our epistemological fram-ing more precisely: We now know that at least some older users experience particu-larly negative feelings towards notifications connected to a lacking sense of controlwhich at least some younger users share (Chapter 2). We now know at least someusers experience strong curiosity for more decision support in notification manage-ment (Chapter 3) and that the decision support in the Reflective Spring Cleaningdesign prompted at least some users to critically reflect on their own ability andneed to manage notifications (Chapter 4). Our key final claim from Chapter 4, thatthis process of critical reflection and personalization improved the notification ex-perience of the majority of participants, shows that our final design concept has thepotential to substantially improve notification management for at least some users.While our participants were fairly representative of general users of notificationsin that they were not particularly technical or expert at data analysis, we do notclaim to know how pervasive this improvement might be, or the precise size ofthe effect. An example source of potential bias that prevents us from knowing thepervasiveness of these effects was that the lead researcher conducted the studies,and thus had vested interest in the designs performing well. When asked directly,she answered participants that she had created the prototypes, which may haveled to more positive responses than otherwise. Future quantitative research couldhelp crystallize our results by exploring the frequency and size of these phenomenain a larger sample. However, given that the majority of participants expressed atleast some improvement from Spring cleaning, we do have at least promising earlyindications that these experiences may not be rare.5.2.2 Design of Reflective Spring CleaningOur prototyping approach to Reflective Spring Cleaning led to several key limi-tations to the results of Study 3. First, we required participants to self-categorizetheir notifications, such as by identifying a work email and important contacts,which many participants would likely avoid in the wild. It’s plausible natural lan-119guage machine learning models could automatically infer information about userrelationships [175] but these sorts of human relationships are notoriously complex.For example, friendly co-workers might send each other non-work messages ontheir work email, which even our self-categorization approach could not differen-tiate. Future research should continue to explore how accurately we can categorizeusers’ notifications for visualization, and how accurate users require these catego-rizations to be. A second major limitation of the final Reflective Spring Cleaningdesign is that we did not actually prototype the prompt that would ask users to en-gage in the reflection session every six months. Instead, our evaluation assumedall participants had already clicked a prompt to open the Reflective Spring Clean-ing app and we asked participants to explore as much as they wanted after thatpoint. This had several impacts on our methodological choices and was one ofour primary reasons not to focus on mixed-methods or quantitative statistics inthis evaluation. To illustrate, the overall quantitative impact of Reflective SpringCleaning on users’ notification management experience is highly dependent on twocomplex, interlinked factors. 1. The usefulness of the reflection session itself. 2.How many participants actually engage in (i.e. click to open) this reflection whenprompted.Systems that prompt users to engage with technology have been heavily stud-ied in both the personalization and reflection literature [14, 159, 187, 225]. Thereremain a number of complex, problem specific design choices that are very diffi-cult to infer from prior literature. For example, prompt frequency, prompt visualdesign and the context within which the prompt is delivered heavily influence howmany participants would engage when prompted [14, 187]. Further, prior notifi-cation management systems that prompt users do so much more frequently thanReflective Spring Cleaning’s every six months, such as several times a week [159].It was unclear whether prior design advice would continue to apply to ReflectiveSpring Cleaning without another study on this problem. If our prompt prototypedid not successfully engage users we would not collect much data on whether thereflection itself was useful. Rather than risk the study providing no reliable resultsdue to investigating these two factors at once, we limited our focus to understand-ing whether the reflection itself was useful. This would help us assess whetherbuilding a more complete system would be worthwhile. However, this means that120we have limited insight into the overall impact of Reflective Spring Cleaning onusers in the wild with our methods because we don’t know how many users wouldactually engage with the prompt. We leave it for future research to understand howto motivate users to engage with Reflective Spring Cleaning, such as by promptingthem when they are frustrated enough with their devices to seek help by installinga stress management app.5.2.3 Heterogeneity of Older UsersThe last major limitation in our research is the relative coarseness of the concep-tualization of the term “older adults” both within our research and the broaderHCI literature. As we argued in Chapter 2, researchers have constructed numer-ous definitions for the term (e.g. users that are 50, 55, 60, 65 or 75+ years old)and there is little consensus on a single authoritative definition [31, 71, 233]. Ourresults, particularly in Chapter 2, provide limited insight on the more restrictiveversions of these definitions, such as above 65 or 70 years old. For example, manyof the “older” participants in the Chapter 2 and 3 studies were still working or hadrecently retired from relatively technical work, such as programming. While wehave little data to confirm this, it seems plausible they would have substantiallymore expertise with mobile technology than users who had been fully retired formany years.We highlight here a need for more holistic and detailed characterization of thisuser group in future notification management research. We suspect future notifica-tion management research will find substantial, design relevant individual differ-ences between older users before and after retirement that are separate from normalaging related cognitive decline. For example, retirement associated factors such asincreased social isolation could drastically reduce the amount of notifications re-ceived and thus the need for notification management [211]. Further, there are avariety of other age associated differences that could influence the design of notifi-cation management tools, and Reflective Spring Cleaning in particular. From per-sonal experience conducting this research, many older adults face challenges com-municating about notifications in the abstract without specific examples to respondto. To support these users, future research could explore, for example, whether the121notification management suggestions given to participants in the Reflective SpringCleaning design should be shown in a more evocative way. Instead of an abstracttext based logical rule like “promote personal notifications during off-work hours”we might provide interactive multi-media clarification for terminology when usershover over the suggestion. For example, we might preview the suggested ring-tonewhen they hover over the term “promote” or show screenshots of example “per-sonal” notifications when users hover over that term. Future work remains to delveinto the role these individual differences play on notification management in muchmore depth.5.3 Implications and Additional Future WorkNext we outline key implications for future work in product design, HCI research,algorithm design and policy.5.3.1 Implications for UI and product designA key implication of our work is that software personalization systems should ex-pand our ideas around low-effort, infrequent, manual personalization. Within thepersonalization literature, the ability to widely observe, analyze and predict humanbehavior through ML has reduced the burden on users, such as by automatically fil-tering or deferring notifications and suggesting features to learn [14, 18, 32, 34, 91].However, Chapters 2 and Chapter 3 highlighted how, despite the need to reducebarriers to personalization, many users react negatively to excessive machine sup-port and are often more concerned with maintaining a sense of control over theirdevices. Through the Reflective Spring Cleaning concept, we demonstrated howinfrequent personalization sessions, supported by data-driven reflection, offer anew and potentially powerful way to address these user’s concerns.A key implication of this work is it highlights the value of exploring newtechniques to enable efficient manual personalization to support future ReflectiveSpring Cleaning style designs. The Reflective Spring Cleaning concept we evalu-ated used mixed-initiative suggestions for personalization rather than offering di-rect manual personalization. However, we noted in Chapter 4 that more users endedup personalizing manually through their existing smartphone settings panels than122by accepting our suggestions. We suggest future Reflective Spring Cleaning sys-tems should try to better support users by allowing manual personalization withinthe visualizations themselves. For example, users might zoom or pinch categoriesof notifications to set limits on the number of notifications particular apps are al-lowed to send in an hour or day. Including a robust way to manually personalizewhile Spring Cleaning will be critical to eventual deployment in the field.5.3.2 Implications for future HCI researchWe suggest future work might explore fundamental questions of how Spring Clean-ing should be folded into holistic software personalization systems. One of theprimary benefits of the Spring Cleaning concept is the low overall effort requiredby users to conduct targeted reflection and personalization for a few minutes everyfew months. However, we have noted in Chapter 4 that many problem domainscould benefit from Spring Cleaning style personalization, such as mental health,productivity, and personalized education apps. If Spring Cleaning were to be morewidely adopted, it is questionable whether users would continue to engage deeplyif they were asked to cycle through Spring Cleaning in many apps.While we continue to recommend that future research should explore SpringCleaning in these new problem domains, we also argue that longer term researchshould explore the effect of Spring Cleaning multiple apps. For example, futurefield deployments of Spring Cleaning systems should carefully track time spentreflecting, and which factors might influence a user to avoid Spring Cleaning foreach problem. Further, an interesting direction to explore is the use of a largerreflection session geared towards multiple problem domains, or even the device asa whole. For example, productivity tracking apps currently might track which appsoccupy most of a user’s time. In addition to visualizing this data for reflection onpersonal productivity, it could suggest useful software personalizations from moretypes of apps, such as mental health, productivity or notification management.Lastly, while it was not a key focus of this research, maintaining privacy withinSpring Cleaning applications is a key and on-going concern for future research.Recent research has argued that our culture has normalized affective discomfortaround data tracking practices while users continue to use ‘creepy’ apps because123they do not see other realistic options to complete their tasks [217]. These concernshave become more timely and important in the wake of the recent overturning ofRoe V. Wade, because concerns have been raised that self-tracking apps includ-ing menstrual cycle data may present opportunities for targeted arrests of women[230]. We argue that personalization researchers and OS developers have a duty tomake a reasonable effort to reduce the potential for self-tracking to become intru-sive and damaging in Spring Cleaning style personalization systems. For example,within our Spring Cleaning system, we briefly had access to the full notificationcontent of users to extract the senders of notifications because Android does notcurrently report a specified ‘sender’ field. Future work should explore whether thisdata needs to be included in the system, whether the data can be collected in lessintrusive ways, and how to ensure data is not stored indefinitely.5.3.3 Implications for algorithm designThe design ideas we explored to support proactive notification management high-light that ML research needs to continue studying how to best predict notificationmanagement preferences. For example, in the Reflective Spring Cleaning concept,we provided some basic personalized suggestions but they would need to be moreaccurately tailored in a production system. Over the course of this research, thebest techniques to accomplish this prediction of notification preferences have beenshifting. At the start of this research, a common technique was to base predictionsfor notification importance on prior trends in user behavior, such as whether usersclicked on or dismissed the notification [159, 160]. However, recent research hasraised concerns that user notification response behavior is strongly influenced byconfounding notification content factors [69, 144, 158, 242]. For example, usersmight only want to read the headline of a news notification, or prefer to respondto emails through the app rather than clicking the notification. These confoundingfactors make it difficult to infer notification importance based on relatively easilycollected user behavior metrics.It has been argued that the next important improvement to predicting notifica-tion preferences is better analysis and prediction of notification content [69, 144,158, 242]. Our qualitative results would consistently agree that notification con-124tent has a very strong impact on which notifications users consider important. Forexample, in Chapter 4, we noted that the Spring Cleaning system prompted usersto critically reflect on their usage of notifications and realize they received manyunimportant notifications that were not worth the disruption they caused. Further,we noted that this realization usually came while viewing the visualization of theapps that sent the user notifications, rather than when they were delivered. Forthese users, it seems likely that the content of the notification, as implied by the appsending them, was a stronger indicator of notification importance than contextualfactors, as implied by what they were doing when they received the notification.If content factors are truly so key to notification importance, it stands to reasonthat future ML research should try to more deeply understand notification content.For example, natural language processing tools could read a user’s notifications toidentify which ones require responses. As we have already noted, however, privacyis an important and understudied issue in notification management. Even if usershave normalized affective discomfort around data tracking [217], researchers havean ethical obligation to reduce the potential for harm from such extensive usertracking. We suggest that any ML research attempting to explore deeper analysis ofnotification content needs to make a parallel effort to reduce the extent and durationof tracking. For example, ML systems have already explored identifying sociallyclose, important contacts through social media interactions [256], and self trackingcould be stopped once these important contacts are identified. Other research hasalso suggested creating tools to help users reflect on and personalize the data theywould like tracked, which could be expanded into notification management tools[50, 136].5.3.4 Implications for PolicyRecent years have seen regulators from various governments focus increasing scrutinyon protecting consumers from issues exacerbated by ubiquitous mobile technol-ogy. For example, in 2021, several Canadian provinces passed more stringent re-porting requirements for privacy breaches online [176, 177], and China recentlyrequired mobile games to disclose the odds underlying randomized purchases thatmay mimic gambling [48, 257]. Given the stress and distraction that mobile noti-125fications may cause users [134, 159, 214, 235], we suggest that excessive notifica-tions, particularly from advertisers, may justify increased scrutiny from regulators.A key focus of this dissertation and many other notification research projectshas been refining notification management technology to reduce the barriers oftime, effort and expertise that prevent many users from managing notifications[13, 159, 160, 214]. A common personal experience that illustrates why frequentnotification management is necessary is how most apps default to sending as manynotifications as possible once installed. While there may be some prompting toconsent to notifications and data tracking, concerns have been raised that the designof these prompts encourages users to blindly click through them [173]. By requir-ing users to explicitly opt out of receiving notifications, app developers are effec-tively empowered to send unnecessary notifications that generate revenue, such asadvertisements. Operating system developers have taken steps to make it easier forusers to opt out of certain notifications. However, we argue OS developers likelyexperience pressure from partner businesses who do not want them to limit theirability to drive engagement to their apps through notifications. With notificationmanagement systems becoming more automatic and adaptive [96, 98], it is plau-sible OS may offer partner businesses and apps paid services to skirt notificationpreferences or prioritize their notifications.Rather than hoping for industry to self-regulate, we suggest regulators can andshould debate whether individual consumers or app developers should be the onesrequired to spend the effort to reduce distraction from notifications. There are sev-eral possible policy directions that might be worth public debate and exploration.For example, similar to China’s bill on disclosing odds in gaming applications[257], regulators might require apps to provide users with an estimate of how manynotifications the average user receives from them before installing. Further, regula-tors might require developers to show users a list of the types of notifications theysend during installation and require users to more explicitly opt in to each typebefore receiving notifications.1265.4 Closing RemarksNotification management has been studied extensively in the HCI community, butcreating a design that substantially improves users’ notification experience remainschallenging due to tough design trade-offs [13, 16, 91, 159, 214]. We now havestrong reasons to believe that substantial progress can be made soon. Advancesin machine learning have continued to make it easier to predict and adapt to userpreferences [91, 159, 214]. In addition, our work has broken new ground andexplored how to provide users a sense of control over their attention that priorsystems have lacked. With users continuing to receive more and more notifications,pushing notification management concepts from academia forward into industrialsystems remains an important and now more achievable goal.127Bibliography[1] C. Abras, D. Maloney-Krichmar, J. Preece, et al. User-centered design.Bainbridge, W. Encyclopedia of Human-Computer Interaction. ThousandOaks: Sage Publications, 37(4):445–456, 2004. → page 6[2] P. D. Adamczyk and B. P. Bailey. If not now, when? the effects ofinterruption at different moments within task execution. In Proceedings ofthe SIGCHI Conference on Human Factors in Computing Systems, CHI’04, page 271–278, New York, NY, USA, 2004. Association for ComputingMachinery. ISBN 1581137028. doi:10.1145/985692.985727. URLhttps://doi.org/10.1145/985692.985727. → pages 2, 15, 43, 44[3] P. D. Adamczyk, S. T. Iqbal, and B. P. Bailey. A method, system, and toolsfor intelligent interruption management. In Proceedings of the 4thInternational Workshop on Task Models and Diagrams, TAMODIA ’05,page 123–126, New York, NY, USA, 2005. Association for ComputingMachinery. ISBN 1595932208. doi:10.1145/1122935.1122959. URLhttps://doi.org/10.1145/1122935.1122959. → page 47[4] R. F. Adler and R. Benbunan-Fich. Juggling on a high wire: Multitaskingeffects on performance. International Journal of Human-ComputerStudies, 70(2):156–168, 2012. ISSN 1071-5819.doi:https://doi.org/10.1016/j.ijhcs.2011.10.003. URLhttps://www.sciencedirect.com/science/article/pii/S1071581911001431.→ page 1[5] N. Ahmadpour and K. A. Cochrane. From information to reflection-designstrategies for personal informatics. In 2017 IEEE Life Sciences Conference(LSC), pages 103–106, New York, NY, USA, 2017. IEEE.doi:10.1109/LSC.2017.8268154. → pages 80, 85[6] H. A. Alamri, S. Watson, and W. Watson. Learning technology models thatsupport personalization within blended learning environments in higher128education. TechTrends, 65(1):62–78, 2021.doi:10.1007/s11528-020-00530-3. URLhttps://doi.org/10.1007/s11528-020-00530-3. → page 109[7] A. S. J. Ali, L. Abdullah, M. Musa, M. A. Yunos, N. W. W. Ki, Z. Tukiran,R. Hamdan, and M. H. A. Zainuddin. Towards iot-based notificationsystem for agriculture electric fence. In 2020 IEEE Student Conference onResearch and Development (SCOReD), pages 269–273, 2020.doi:10.1109/SCOReD50371.2020.9250958. → page 40[8] E. M. Altmann and J. G. Trafton. Memory for goals: An activation-basedmodel. Cognitive science, 26(1):39–83, 2002. → pages 1, 2, 4, 15, 44[9] E. M. Altmann and J. G. Trafton. Task interruption: Resumption lag andthe role of cues. In Proceedings of the Cognitive Science Society,volume 26, 2004. → pages 2, 4[10] C. Anderson, I. Hu¨bener, A.-K. Seipp, S. Ohly, K. David, and V. Pejovic. Asurvey of attention management systems in ubiquitous computingenvironments. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 2(2), jul 2018. doi:10.1145/3214261. URLhttps://doi.org/10.1145/3214261. → pages 40, 44, 73[11] A. E. Andrews, R. M. Ratwani, and J. G. Trafton. The effect of alert type toan interruption on primary task resumption. In Proceedings of the HFESAnnual Meeting 2009, pages 1–5, Washington, DC0, USA, 2009. HFES.→ pages 2, 4, 5, 15, 43, 47, 83[12] apps.wverlaek@gmail.com. Block apps: Digital wellbeing.https://play.google.com/store/apps/details?id=com.wverlaek.block, 2022.Accessed: 2022-08-08. → pages 45, 50, 52, 79, 84[13] J. Aranda, N. Ali-Hasan, and S. Baig. I’m just trying to survive: Anethnographic look at mobile notifications and attention management. InProceedings of the 18th International Conference on Human-ComputerInteraction with Mobile Devices and Services Adjunct, MobileHCI ’16,page 564–574, New York, NY, USA, 2016. Association for ComputingMachinery. ISBN 9781450344135. doi:10.1145/2957265.2957274. URLhttps://doi.org/10.1145/2957265.2957274. → pages2, 3, 4, 5, 13, 15, 16, 19, 23, 25, 27, 33, 35, 40, 44, 45, 51, 60, 69, 71, 75, 79, 80, 81, 83, 84, 87, 88, 89, 91, 110, 114, 115, 126, 127[14] K. Ardekani. Feature recommender: a large-scale in-situ study of proactivesoftware feature recommendations. Master’s thesis, University of British129Columbia, 2016. URLhttps://open.library.ubc.ca/collections/ubctheses/24/items/1.0339868. →pages 120, 122[15] S. Assami, N. Daoudi, and R. Ajhoun. Personalization criteria forenhancing learner engagement in mooc platforms. In 2018 IEEE GlobalEngineering Education Conference (EDUCON), pages 1265–1272, NewYork, NY, USA, 2018. IEEE. doi:10.1109/EDUCON.2018.8363375. →page 109[16] J. Auda, D. Weber, A. Voit, and S. Schneegass. Understanding userpreferences towards rule-based notification deferral. In Extended Abstractsof the 2018 CHI Conference on Human Factors in Computing Systems,CHI EA ’18, New York, NY, USA, 2018. Association for ComputingMachinery. ISBN 9781450356213. doi:10.1145/3170427.3188688. URLhttps://doi.org/10.1145/3170427.3188688. → pages4, 5, 6, 40, 44, 45, 47, 50, 56, 72, 79, 84, 110, 127[17] A. Ayobi, P. Marshall, and A. L. Cox. Reflections on 5 years of personalinformatics: Rising concerns and emerging directions. In Proceedings ofthe 2016 CHI Conference Extended Abstracts on Human Factors inComputing Systems, CHI EA ’16, page 2774–2781, New York, NY, USA,2016. Association for Computing Machinery. ISBN 9781450340823.doi:10.1145/2851581.2892406. URLhttps://doi.org/10.1145/2851581.2892406. → page 85[18] N. Banovic, F. Chevalier, T. Grossman, and G. Fitzmaurice. Triggeringtriggers and burying barriers to customizing software. In Proceedings of theSIGCHI Conference on Human Factors in Computing Systems, CHI ’12,page 2717–2726, New York, NY, USA, 2012. Association for ComputingMachinery. ISBN 9781450310154. doi:10.1145/2207676.2208666. URLhttps://doi.org/10.1145/2207676.2208666. → pages4, 5, 6, 16, 41, 45, 79, 80, 82, 84, 88, 109, 122[19] Y. Barnard, M. D. Bradley, F. Hodgson, and A. D. Lloyd. Learning to usenew technologies by older adults: Perceived difficulties, experimentationbehaviour and usability. Computers in Human Behavior, 29(4):1715–1724,2013. ISSN 0747-5632. doi:https://doi.org/10.1016/j.chb.2013.02.006.URLhttps://www.sciencedirect.com/science/article/pii/S0747563213000721.→ page 57130[20] E. P. Baumer. Reflective informatics: Conceptual dimensions for designingtechnologies of reflection. In Proceedings of the 33rd Annual ACMConference on Human Factors in Computing Systems, CHI ’15, page585–594, New York, NY, USA, 2015. Association for ComputingMachinery. ISBN 9781450331456. doi:10.1145/2702123.2702234. URLhttps://doi.org/10.1145/2702123.2702234. → page 85[21] E. P. Baumer, V. Khovanskaya, M. Matthews, L. Reynolds,V. Schwanda Sosik, and G. Gay. Reviewing reflection: On the use ofreflection in interactive system design. In Proceedings of the 2014Conference on Designing Interactive Systems, DIS ’14, page 93–102, NewYork, NY, USA, 2014. Association for Computing Machinery. ISBN9781450329026. doi:10.1145/2598510.2598598. URLhttps://doi.org/10.1145/2598510.2598598. → pages 80, 85[22] J. B. Berger and G. D. Malaney. Assessing the transition of transferstudents from community colleges to a university. NASPA Journal, 40(4):1–23, 2003. doi:10.2202/1949-6605.1277. URLhttps://doi.org/10.2202/1949-6605.1277. → page 74[23] M. Bernard, C. Liao, and M. Mills. Determining the best online font forolder adults. Usability News, 3(1):1–4, 2001. → page 17[24] A. Bhatia, D. Kundu, S. Agarwal, V. Kairon, and A. Parnami. Soma-noti:Delivering notifications through under-clothing wearables. In Proceedingsof the 2021 CHI Conference on Human Factors in Computing Systems,CHI ’21, New York, NY, USA, 2021. Association for ComputingMachinery. ISBN 9781450380966. doi:10.1145/3411764.3445123. URLhttps://doi.org/10.1145/3411764.3445123. → pages 43, 44, 47[25] G. Biegel and V. Cahill. A framework for developing mobile,context-aware applications. In Second IEEE Annual Conference onPervasive Computing and Communications, 2004. Proceedings of the,pages 361–365, 2004. doi:10.1109/PERCOM.2004.1276875. → pages1, 44[26] J. Birnholtz, J. Davison, and A. Li. Attending to attention: How do peopleattract, manage, and negotiate attention using mobile devices? MobileMedia & Communication, 5(3):256–274, 2017.doi:10.1177/2050157917714504. URLhttps://doi.org/10.1177/2050157917714504. → pages 43, 45131[27] J. E. Booker, C. M. Chewar, and D. S. McCrickard. Usability testing ofnotification interfaces: Are we focused on the best metrics? In Proceedingsof the 42nd Annual Southeast Regional Conference, ACM-SE 42, page128–133, New York, NY, USA, 2004. Association for ComputingMachinery. ISBN 1581138709. doi:10.1145/986537.986568. URLhttps://doi.org/10.1145/986537.986568. → pages 2, 117[28] J. Borghouts, D. P. Brumby, and A. L. Cox. Timetofocus: Feedback oninterruption durations discourages distractions and shortens interruptions.ACM Trans. Comput.-Hum. Interact., 27(5), aug 2020. ISSN 1073-0516.doi:10.1145/3396044. URL https://doi.org/10.1145/3396044. → pages6, 46, 47, 50, 73[29] J. P. Borst, N. A. Taatgen, and H. Van Rijn. The problem state: a cognitivebottleneck in multitasking. Journal of Experimental Psychology: Learning,memory, and cognition, 36(2):363, 2010. ISSN 1939-1285.doi:https://doi.org/10.1037/a0018106. → page 1[30] V. Braun and V. Clarke. Using thematic analysis in psychology. QualitativeResearch in Psychology, 3(2):77–101, 2006.doi:10.1191/1478088706qp063oa. URLhttps://www.tandfonline.com/doi/abs/10.1191/1478088706qp063oa. →pages 22, 48, 64, 97, 98[31] M. Brehmer, J. McGrenere, C. Tang, and C. Jacova. Investigatinginterruptions in the context of computerised cognitive testing for olderadults. In Proceedings of the SIGCHI Conference on Human Factors inComputing Systems, CHI ’12, page 2649–2658, New York, NY, USA,2012. Association for Computing Machinery. ISBN 9781450310154.doi:10.1145/2207676.2208656. URLhttps://doi.org/10.1145/2207676.2208656. → pages 17, 75, 121[32] A. Bunt, C. Conati, and J. McGrenere. Supporting interface customizationusing a mixed-initiative approach. In Proceedings of the 12th InternationalConference on Intelligent User Interfaces, IUI ’07, page 92–101, NewYork, NY, USA, 2007. Association for Computing Machinery. ISBN1595934812. doi:10.1145/1216295.1216317. URLhttps://doi.org/10.1145/1216295.1216317. → pages 14, 17, 45, 84, 122[33] A. Bunt, J. McGrenere, and C. Conati. Understanding the utility ofrationale in a mixed-initiative system for gui customization. In User132Modeling 2007, pages 147–156, Berlin, Heidelberg, 2007. Springer BerlinHeidelberg. ISBN 978-3-540-73078-1. → pages 17, 45, 50, 75, 110[34] A. Bunt, C. Conati, and J. Mcgrenere. Mixed-initiative interfacepersonalization as a case study in usable ai. AI Magazine, 30:58–64, 122009. doi:10.1609/aimag.v30i4.2264. → pages 40, 84, 110, 122[35] M. Busch, E. Mattheiss, R. Orji, A. Marczewski, W. Hochleitner,M. Lankes, L. E. Nacke, and M. Tscheligi. Personalization in serious andpersuasive games and gamified interactions. In Proceedings of the 2015Annual Symposium on Computer-Human Interaction in Play, CHI PLAY’15, page 811–816, New York, NY, USA, 2015. Association for ComputingMachinery. ISBN 9781450334662. doi:10.1145/2793107.2810260. URLhttps://doi.org/10.1145/2793107.2810260. → page 45[36] P. Cedillo, A. Garcı´a, J. D. Ca´rdenas, and A. Bermeo. A systematicliterature review of electronic invoicing, platforms and notification systems.In 2018 International Conference on eDemocracy & eGovernment(ICEDEG), pages 150–157, 2018. doi:10.1109/ICEDEG.2018.8372338.→ page 43[37] C. C. Chang, M.-H. Wu, Y.-J. Lee, X. Chang, and Y.-J. Chang. OpportuneMoments for the Multi-Stage Notification Responding Process: APreliminary Investigation, page 9–10. Association for ComputingMachinery, New York, NY, USA, 2021. ISBN 9781450384612. URLhttps://doi.org/10.1145/3460418.3479278. → pages 44, 47, 51[38] W.-J. Chang, L.-B. Chen, and K.-Y. Su. Deepcrash: A deep learning-basedinternet of vehicles system for head-on and single-vehicle accidentdetection with emergency notification. IEEE Access, 7:148163–148175,2019. doi:10.1109/ACCESS.2019.2946468. → page 43[39] Y.-J. Chang, Y.-J. Chung, and Y.-H. Shih. I think it’s her: Investigatingsmartphone users’ speculation about phone notifications and its influenceon attendance. In Proceedings of the 21st International Conference onHuman-Computer Interaction with Mobile Devices and Services,MobileHCI ’19, New York, NY, USA, 2019. Association for ComputingMachinery. ISBN 9781450368254. doi:10.1145/3338286.3340125. URLhttps://doi.org/10.1145/3338286.3340125. → pages 44, 45[40] K.-W. Chen, Y.-J. Chang, and L. Chan. Predicting opportune moments todeliver notifications in virtual reality. In CHI Conference on Human133Factors in Computing Systems, CHI ’22, New York, NY, USA, 2022.Association for Computing Machinery. ISBN 9781450391573.doi:10.1145/3491102.3517529. URLhttps://doi.org/10.1145/3491102.3517529. → pages 4, 5, 42, 50, 51, 66[41] K.-W. Chen, Y.-J. Chang, and L. Chan. Predicting opportune moments todeliver notifications in virtual reality. In CHI Conference on HumanFactors in Computing Systems, CHI ’22, New York, NY, USA, 2022.Association for Computing Machinery. ISBN 9781450391573.doi:10.1145/3491102.3517529. URLhttps://doi.org/10.1145/3491102.3517529. → pages 50, 51, 117, 118[42] Y.-C. Chen, K.-C. Kao, Y.-J. Lee, F. Shih, W.-C. Chiu, and Y.-J. Chang.Killing-Time Detection from Smartphone Screenshots, page 15–16.Association for Computing Machinery, New York, NY, USA, 2021. ISBN9781450384612. URL https://doi.org/10.1145/3460418.3479295. →pages 44, 47[43] J. Cheng, M. Burke, and E. G. Davis. Understanding perceptions ofproblematic facebook use: When people experience negative life impactand a lack of control. In Proceedings of the 2019 CHI Conference onHuman Factors in Computing Systems, CHI ’19, page 1–13, New York,NY, USA, 2019. Association for Computing Machinery. ISBN9781450359702. doi:10.1145/3290605.3300429. URLhttps://doi.org/10.1145/3290605.3300429. → page 1[44] C. M. Chewar and D. S. McCrickard. Educating novice developers ofnotification systems: Targeting user-goals with a conceptual framework. InD. Lassner and C. McNaught, editors, Proceedings of EdMedia + InnovateLearning 2003, pages 2759–2766, Honolulu, Hawaii, USA, 2003.Association for the Advancement of Computing in Education (AACE).URL https://www.learntechlib.org/p/14305. → page 47[45] F. L. Cibrian, K. D. Lakes, A. Tavakoulnia, K. Guzman, S. Schuck, andG. R. Hayes. Supporting Self-Regulation of Children with ADHD UsingWearables: Tensions and Design Challenges, page 1–13. Association forComputing Machinery, New York, NY, USA, 2020. ISBN 9781450367080.URL https://doi.org/10.1145/3313831.3376837. → pages 40, 44[46] W. C. Clapp, M. T. Rubens, J. Sabharwal, and A. Gazzaley. Deficit inswitching between functional brain networks underlies the impact ofmultitasking on working memory in older adults. Proceedings of the134National Academy of Sciences, 108(17):7212–7217, 2011.doi:10.1073/pnas.1015297108. URLhttps://www.pnas.org/doi/abs/10.1073/pnas.1015297108. → pages3, 7, 12, 13, 17, 30, 41, 53, 75[47] J. Clemente, F. Li, M. Valero, and W. Song. Smart seismic sensing forindoor fall detection, location, and notification. IEEE Journal ofBiomedical and Health Informatics, 24(2):524–532, 2020.doi:10.1109/JBHI.2019.2907498. → page 40[48] J. Close, S. G. Spicer, L. L. Nicklin, M. Uther, J. Lloyd, and H. Lloyd.Secondary analysis of loot box data: Are high-spending “whales” wealthygamers or problem gamblers? Addictive Behaviors, 117:106851, 2021.ISSN 0306-4603. doi:https://doi.org/10.1016/j.addbeh.2021.106851. URLhttps://www.sciencedirect.com/science/article/pii/S0306460321000368.→ pages 4, 125[49] D. A. Cobb-Clark and S. Schurer. The stability of big-five personalitytraits. Economics Letters, 115(1):11–15, 2012. ISSN 0165-1765.doi:https://doi.org/10.1016/j.econlet.2011.11.015. URLhttps://www.sciencedirect.com/science/article/pii/S0165176511004666.→ page 75[50] J. Colnago, Y. Feng, T. Palanivel, S. Pearman, M. Ung, A. Acquisti, L. F.Cranor, and N. Sadeh. Informing the Design of a Personalized PrivacyAssistant for the Internet of Things, page 1–13. Association for ComputingMachinery, New York, NY, USA, 2020. ISBN 9781450367080. URLhttps://doi.org/10.1145/3313831.3376389. → page 125[51] G. Coudin and T. Alexopoulos. ‘help me! i’m old!’ how negative agingstereotypes create dependency among older adults. Aging & MentalHealth, 14(5):516–523, 2010. doi:10.1080/13607861003713182. URLhttps://doi.org/10.1080/13607861003713182. PMID: 20480414. → page75[52] M. Czerwinski, E. Cutrell, and E. Horvitz. Instant messaging: Effects ofrelevance and timing. In People and computers XIV: Proceedings of HCI,volume 2, pages 71–76, 2000. URLhttps://cir.nii.ac.jp/crid/1572261550271255552. → page 2[53] L. De Russis and A. Monge Roffarello. On the benefit of adding userpreferences to notification delivery. In Proceedings of the 2017 CHI135Conference Extended Abstracts on Human Factors in Computing Systems,CHI EA ’17, page 1561–1568, New York, NY, USA, 2017. Association forComputing Machinery. ISBN 9781450346566.doi:10.1145/3027063.3053160. URLhttps://doi.org/10.1145/3027063.3053160. → pages2, 4, 5, 40, 45, 47, 50, 56, 72, 79, 84[54] E. L. Deci, R. M. Ryan, M. Gagne´, D. R. Leone, J. Usunov, and B. P.Kornazheva. Need satisfaction, motivation, and well-being in the workorganizations of a former eastern bloc country: A cross-cultural study ofself-determination. Personality and Social Psychology Bulletin, 27(8):930–942, 2001. doi:10.1177/0146167201278002. URLhttps://doi.org/10.1177/0146167201278002. → page 27[55] J. M. Digman. Higher-order factors of the big five. Journal of personalityand social psychology, 73(6):1246, 1997.doi:https://doi.org/10.1037/0022-3514.73.6.1246. → page 75[56] F. Draxler, C. Schneegass, J. Safranek, and H. Hussmann. Why did youstop? - investigating origins and effects of interruptions during mobilelanguage learning. In Mensch Und Computer 2021, MuC ’21, page 21–33.Association for Computing Machinery, New York, NY, USA, 2021. ISBN9781450386456. doi:10.1145/3473856.3473881. URLhttps://doi.org/10.1145/3473856.3473881. → pages 2, 40, 79, 83[57] A. Drummond and J. D. Sauer. Video game loot boxes are psychologicallyakin to gambling. Nature human behaviour, 2(8):530–532, 2018.doi:10.1038/s41562-018-0360-1. URLhttps://doi.org/10.1038/s41562-018-0360-1. → page 4[58] L. L. Ellingson. Engaging crystallization in qualitative research: Anintroduction. Sage, Newbury Park, CA, USA, 2009. → pages 22, 98[59] C. Elsden and D. S. Kirk. A quantified past: Remembering with personalinformatics. In Proceedings of the 2014 Companion Publication onDesigning Interactive Systems, DIS Companion ’14, page 45–48.Association for Computing Machinery, New York, NY, USA, 2014. ISBN9781450329033. doi:10.1145/2598784.2602778. URLhttps://doi.org/10.1145/2598784.2602778. → page 85[60] D. Epstein, F. Cordeiro, E. Bales, J. Fogarty, and S. Munson. Taming datacomplexity in lifelogs: Exploring visual cuts of personal informatics data.136In Proceedings of the 2014 Conference on Designing Interactive Systems,DIS ’14, page 667–676, New York, NY, USA, 2014. Association forComputing Machinery. ISBN 9781450329026.doi:10.1145/2598510.2598558. URLhttps://doi.org/10.1145/2598510.2598558. → pages 85, 86[61] D. A. Epstein, B. H. Jacobson, E. Bales, D. W. McDonald, and S. A.Munson. From ”nobody cares” to ”way to go!”: A design framework forsocial sharing in personal informatics. In Proceedings of the 18th ACMConference on Computer Supported Cooperative Work and SocialComputing, CSCW ’15, page 1622–1636, New York, NY, USA, 2015.Association for Computing Machinery. ISBN 9781450329224.doi:10.1145/2675133.2675135. URLhttps://doi.org/10.1145/2675133.2675135. → pages 6, 85[62] D. A. Epstein, A. Ping, J. Fogarty, and S. A. Munson. A lived informaticsmodel of personal informatics. In Proceedings of the 2015 ACMInternational Joint Conference on Pervasive and Ubiquitous Computing,UbiComp ’15, page 731–742, New York, NY, USA, 2015. Association forComputing Machinery. ISBN 9781450335744.doi:10.1145/2750858.2804250. URLhttps://doi.org/10.1145/2750858.2804250. → pages 85, 86, 108[63] D. A. Epstein, M. Caraway, C. Johnston, A. Ping, J. Fogarty, and S. A.Munson. Beyond abandonment to next steps: Understanding and designingfor life after personal informatics tool use. In Proceedings of the 2016 CHIConference on Human Factors in Computing Systems, CHI ’16, page1109–1113, New York, NY, USA, 2016. Association for ComputingMachinery. ISBN 9781450333627. doi:10.1145/2858036.2858045. URLhttps://doi.org/10.1145/2858036.2858045. → pages 86, 108[64] D. A. Epstein, C. Caldeira, M. C. Figueiredo, X. Lu, L. M. Silva,L. Williams, J. H. Lee, Q. Li, S. Ahuja, Q. Chen, P. Dowlatyari, C. Hilby,S. Sultana, E. V. Eikey, and Y. Chen. Mapping and taking stock of thepersonal informatics literature. Proc. ACM Interact. Mob. WearableUbiquitous Technol., 4(4), dec 2020. doi:10.1145/3432231. URLhttps://doi.org/10.1145/3432231. → pages 80, 82, 85, 86, 108, 109, 110[65] C. Esposito, F. Palmieri, and K.-K. R. Choo. Cloud message queueing andnotification: Challenges and opportunities. IEEE Cloud Computing, 5(2):11–16, 2018. doi:10.1109/MCC.2018.022171662. → page 43137[66] S. E. Fahlman. Selling interrupt rights: A way to control unwanted e-mailand telephone calls [technical forum]. IBM Systems Journal, 41(4):759–766, 2002. doi:10.1147/sj.414.0759. → page 47[67] A. P. Felt, S. Egelman, and D. Wagner. I’ve got 99 problems, but vibrationain’t one: A survey of smartphone users’ concerns. In Proceedings of theSecond ACM Workshop on Security and Privacy in Smartphones andMobile Devices, SPSM ’12, page 33–44, New York, NY, USA, 2012.Association for Computing Machinery. ISBN 9781450316668.doi:10.1145/2381934.2381943. URLhttps://doi.org/10.1145/2381934.2381943. → page 3[68] L. Findlater, B. Chinh, D. Jain, J. Froehlich, R. Kushalnagar, and A. C. Lin.Deaf and hard-of-hearing individuals’ preferences for wearable and mobilesound awareness technologies. In Proceedings of the 2019 CHI Conferenceon Human Factors in Computing Systems, CHI ’19, page 1–13, New York,NY, USA, 2019. Association for Computing Machinery. ISBN9781450359702. doi:10.1145/3290605.3300276. URLhttps://doi.org/10.1145/3290605.3300276. → pages 40, 43, 47[69] J. E. Fischer, N. Yee, V. Bellotti, N. Good, S. Benford, and C. Greenhalgh.Effects of content and time of delivery on receptivity to mobileinterruptions. In Proceedings of the 12th International Conference onHuman Computer Interaction with Mobile Devices and Services,MobileHCI ’10, page 103–112, New York, NY, USA, 2010. Associationfor Computing Machinery. ISBN 9781605588353.doi:10.1145/1851600.1851620. URLhttps://doi.org/10.1145/1851600.1851620. → pages 40, 44, 50, 124[70] J. E. Fischer, C. Greenhalgh, and S. Benford. Investigating episodes ofmobile phone activity as indicators of opportune moments to delivernotifications. In Proceedings of the 13th International Conference onHuman Computer Interaction with Mobile Devices and Services,MobileHCI ’11, page 181–190, New York, NY, USA, 2011. Associationfor Computing Machinery. ISBN 9781450305419.doi:10.1145/2037373.2037402. URLhttps://doi.org/10.1145/2037373.2037402. → pages2, 3, 4, 5, 15, 43, 44, 79, 83, 84, 85, 91, 115[71] A. D. Fisk, S. J. Czaja, W. A. Rogers, N. Charness, and J. Sharit.Designing for older adults: Principles and creative human factorsapproaches. CRC press, 2009. → pages 17, 75, 121138[72] N. Fitz, K. Kushlev, R. Jagannathan, T. Lewis, D. Paliwal, and D. Ariely.Batching smartphone notifications can improve well-being. Computers inHuman Behavior, 101:84–94, 2019. ISSN 0747-5632.doi:https://doi.org/10.1016/j.chb.2019.07.016. URLhttps://www.sciencedirect.com/science/article/pii/S0747563219302596.→ page 40[73] R. Fleck and G. Fitzpatrick. Reflecting on reflection: Framing a designlandscape. In Proceedings of the 22nd Conference of the Computer-HumanInteraction Special Interest Group of Australia on Computer-HumanInteraction, OZCHI ’10, page 216–223, New York, NY, USA, 2010.Association for Computing Machinery. ISBN 9781450305020.doi:10.1145/1952222.1952269. URLhttps://doi.org/10.1145/1952222.1952269. → pages 5, 79, 80, 85, 108[74] P. E. Fortin and J. Cooperstock. Understanding smartphone notifications’activity disruption via in situ wrist motion monitoring. In CHI Conferenceon Human Factors in Computing Systems Extended Abstracts, CHI EA ’22,New York, NY, USA, 2022. Association for Computing Machinery. ISBN9781450391566. doi:10.1145/3491101.3519695. URLhttps://doi.org/10.1145/3491101.3519695. → page 4[75] P. E. Fortin, E. Sulmont, and J. Cooperstock. Detecting perception ofsmartphone notifications using skin conductance responses. In Proceedingsof the 2019 CHI Conference on Human Factors in Computing Systems,CHI ’19, page 1–9, New York, NY, USA, 2019. Association for ComputingMachinery. ISBN 9781450359702. doi:10.1145/3290605.3300420. URLhttps://doi.org/10.1145/3290605.3300420. → page 45[76] B. Friedrichs, L. D. Turner, and S. M. Allen. Discovering types ofsmartphone usage sessions from user-app interactions. In 2021 IEEEInternational Conference on Pervasive Computing and CommunicationsWorkshops and other Affiliated Events (PerCom Workshops), pages459–464, 2021. doi:10.1109/PerComWorkshops51409.2021.9431034. →page 45[77] N. Fronemann and M. Peissner. User experience concept exploration: Userneeds as a source for innovation. In Proceedings of the 8th NordicConference on Human-Computer Interaction: Fun, Fast, Foundational,NordiCHI ’14, page 727–736, New York, NY, USA, 2014. Association forComputing Machinery. ISBN 9781450325424.139doi:10.1145/2639189.2641203. URLhttps://doi.org/10.1145/2639189.2641203. → pages 39, 59[78] fruitranger. Notification manager - cleaner. https://play.google.com/store/apps/details?id=com.cortana.notificationmanager.notificationclean&hl=en,2017. Accessed: 2020-3-11. → pages 4, 45, 48, 52, 79, 84[79] K. Z. Gajos, M. Czerwinski, D. S. Tan, and D. S. Weld. Exploring thedesign space for adaptive graphical user interfaces. In Proceedings of theWorking Conference on Advanced Visual Interfaces, AVI ’06, page201–208, New York, NY, USA, 2006. Association for ComputingMachinery. ISBN 1595933530. doi:10.1145/1133265.1133306. URLhttps://doi.org/10.1145/1133265.1133306. → pages 14, 16, 45, 47, 84, 85[80] K. Z. Gajos, K. Everitt, D. S. Tan, M. Czerwinski, and D. S. Weld.Predictability and accuracy in adaptive user interfaces. In Proceedings ofthe SIGCHI Conference on Human Factors in Computing Systems, CHI’08, page 1271–1274, New York, NY, USA, 2008. Association forComputing Machinery. ISBN 9781605580111.doi:10.1145/1357054.1357252. URLhttps://doi.org/10.1145/1357054.1357252. → pages 47, 84[81] T. Gale and S. Parker. Navigating change: a typology of student transitionin higher education. Studies in Higher Education, 39(5):734–753, 2014.doi:10.1080/03075079.2012.721351. URLhttps://doi.org/10.1080/03075079.2012.721351. → pages 71, 74[82] J. P. Garcia Vazquez, M. D. Rodriguez, and A. G. Andrade. Designdimensions of ambient information systems to assist elderly with theiractivities of daily living. In Proceedings of the 12th ACM InternationalConference Adjunct Papers on Ubiquitous Computing - Adjunct, UbiComp’10 Adjunct, page 461–464, New York, NY, USA, 2010. Association forComputing Machinery. ISBN 9781450302838.doi:10.1145/1864431.1864487. URLhttps://doi.org/10.1145/1864431.1864487. → pages 47, 48[83] S. Ghosh, L. Winston, N. Panchal, P. Kimura-Thollander, J. Hotnog,D. Cheong, G. Reyes, and G. D. Abowd. Notifivr: Exploring interruptionsand notifications in virtual reality. IEEE Transactions on Visualization andComputer Graphics, 24(4):1447–1456, 2018.doi:10.1109/TVCG.2018.2793698. → pages 44, 47140[84] E. Gilbert and K. Karahalios. Predicting tie strength with social media. InProceedings of the SIGCHI Conference on Human Factors in ComputingSystems, CHI ’09, page 211–220, New York, NY, USA, 2009. Associationfor Computing Machinery. ISBN 9781605582467.doi:10.1145/1518701.1518736. URLhttps://doi.org/10.1145/1518701.1518736. → page 37[85] C. Glatz, S. S. Krupenia, H. H. Bu¨lthoff, and L. L. Chuang. Use the rightsound for the right job: Verbal commands and auditory icons for atask-management system favor different information processes in the brain.In Proceedings of the 2018 CHI Conference on Human Factors inComputing Systems, CHI ’18, page 1–13, New York, NY, USA, 2018.Association for Computing Machinery. ISBN 9781450356206.doi:10.1145/3173574.3174046. URLhttps://doi.org/10.1145/3173574.3174046. → page 43[86] L. Goldberg, B. Lide, S. Lowry, H. A. Massett, T. O’Connell, J. Preece,W. Quesenbery, and B. Shneiderman. Usability and accessibility inconsumer health informatics: Current trends and future challenges.American Journal of Preventive Medicine, 40(5, Supplement 2):S187–S197, 2011. ISSN 0749-3797.doi:https://doi.org/10.1016/j.amepre.2011.01.009. URLhttps://www.sciencedirect.com/science/article/pii/S0749379711000869.→ page 86[87] V. M. Gonza´lez and G. Mark. ”constant, constant, multi-tasking craziness”:Managing multiple working spheres. In Proceedings of the SIGCHIConference on Human Factors in Computing Systems, CHI ’04, page113–120, New York, NY, USA, 2004. Association for ComputingMachinery. ISBN 1581137028. doi:10.1145/985692.985707. URLhttps://doi.org/10.1145/985692.985707. → page 1[88] Google. Google play store. https://play.google.com/store/apps, 2021.Accessed: 2021-09-1. → page 90[89] S. Grandhi and Q. Jones. Conceptualizing interpersonal interruptionmanagement: A theoretical framework and research program. In 200942nd Hawaii International Conference on System Sciences, pages 1–10,2009. doi:10.1109/HICSS.2009.124. → page 2[90] S. A. Grandhi and Q. Jones. Knock, knock! who’s there? putting the userin control of managing interruptions. International Journal of141Human-Computer Studies, 79:35–50, 2015. ISSN 1071-5819.doi:https://doi.org/10.1016/j.ijhcs.2015.02.008. URLhttps://www.sciencedirect.com/science/article/pii/S1071581915000415.→ page 47[91] T. Gross and A.-L. Mueller. Notificationmanager: Personal boundarymanagement on mobile devices. In C. Ardito, R. Lanzilotti, A. Malizia,H. Petrie, A. Piccinno, G. Desolda, and K. Inkpen, editors,Human-Computer Interaction – INTERACT 2021, pages 243–261, Cham,2021. Springer International Publishing. ISBN 978-3-030-85610-6. →pages 2, 4, 5, 45, 47, 79, 83, 115, 122, 127[92] H. Guillou, K. Chow, T. Fritz, and J. McGrenere. Is your time well spent?reflecting on knowledge work more holistically. In Proceedings of the 2020CHI Conference on Human Factors in Computing Systems, page 1–9, NewYork, NY, USA, 2020. Association for Computing Machinery. ISBN9781450367080. URL https://doi.org/10.1145/3313831.3376586. →pages 2, 110[93] R. Gulotta, J. Forlizzi, R. Yang, and M. W. Newman. Fosteringengagement with personal informatics systems. In Proceedings of the 2016ACM Conference on Designing Interactive Systems, DIS ’16, page286–300, New York, NY, USA, 2016. Association for ComputingMachinery. ISBN 9781450340311. doi:10.1145/2901790.2901803. URLhttps://doi.org/10.1145/2901790.2901803. → page 85[94] S. Gu¨nther, F. Mu¨ller, M. Funk, and M. Mu¨hlha¨user. Slappyfications:Towards ubiquitous physical and embodied notifications. In ExtendedAbstracts of the 2019 CHI Conference on Human Factors in ComputingSystems, CHI EA ’19, page 1–2, New York, NY, USA, 2019. Associationfor Computing Machinery. ISBN 9781450359719.doi:10.1145/3290607.3311780. URLhttps://doi.org/10.1145/3290607.3311780. → page 43[95] T. Harris. How better tech could protect us from distraction.https://www.ted.com/talks/tristan harris how better tech could protect us from distraction?language=en, 2014. Accessed: 2021-09-01. → page 6[96] B. Heater and S. Perez. Apple refines ios 15 notifications with focus,summary features. https://techcrunch.com/2021/06/07/142apple-refines-ios-15-notifications-with-focus-summary-features/, 2021.Accessed: 2021-09-01. → pages 4, 5, 6, 46, 48, 66, 79, 81, 84, 110, 126[97] A. Hennig, F. Neusser, A. A. Pawelek, D. Herrmann, and P. Mayer.Standing out among the daily spam: How to catch website owners’attention by means of vulnerability notifications. In CHI Conference onHuman Factors in Computing Systems Extended Abstracts, CHI EA ’22,New York, NY, USA, 2022. Association for Computing Machinery. ISBN9781450391566. doi:10.1145/3491101.3519847. URLhttps://doi.org/10.1145/3491101.3519847. → page 4[98] J. Hildenbrad. Android oreo will make you love notifications again.https://www.androidcentral.com/android-o-and-notifications-everything-you-need-know, 2017. Accessed:2017-11-10. → pages 3, 4, 5, 6, 13, 36, 48, 79, 84, 126[99] J. D. Hincapie´-Ramos, S. Voida, and G. Mark. A design space analysis ofavailability-sharing systems. In Proceedings of the 24th Annual ACMSymposium on User Interface Software and Technology, UIST ’11, page85–96, New York, NY, USA, 2011. Association for Computing Machinery.ISBN 9781450307161. doi:10.1145/2047196.2047207. URLhttps://doi.org/10.1145/2047196.2047207. → pages 47, 48[100] C.-Y. Ho, M. I. Nikolic, M. J. Waters, and N. B. Sarter. Not now!supporting interruption management by indicating the modality andurgency of pending tasks. Human Factors, 46(3):399–409, 2004.doi:10.1518/hfes.46.3.399.50397. URLhttps://doi.org/10.1518/hfes.46.3.399.50397. PMID: 15573541. → page47[101] J. Ho and S. S. Intille. Using context-aware computing to reduce theperceived burden of interruptions from mobile devices. In Proceedings ofthe SIGCHI Conference on Human Factors in Computing Systems, CHI’05, page 909–918, New York, NY, USA, 2005. Association for ComputingMachinery. ISBN 1581139985. doi:10.1145/1054972.1055100. URLhttps://doi.org/10.1145/1054972.1055100. → pages 1, 4, 47, 115[102] H. M. Hodgetts and D. M. Jones. Reminders, alerts and pop-ups: The costof computer-initiated interruptions. In J. A. Jacko, editor,Human-Computer Interaction. Interaction Design and Usability, pages818–826, Berlin, Heidelberg, 2007. Springer Berlin Heidelberg. ISBN978-3-540-73105-4. → pages 1, 5, 79, 83143[103] E. Horvitz and J. Apacible. Learning and reasoning about interruption. InProceedings of the 5th International Conference on Multimodal Interfaces,ICMI ’03, page 20–27, New York, NY, USA, 2003. Association forComputing Machinery. ISBN 1581136218. doi:10.1145/958432.958440.URL https://doi.org/10.1145/958432.958440. → page 2[104] E. Horvitz, P. Koch, and J. Apacible. Busybody: Creating and fieldingpersonalized models of the cost of interruption. In Proceedings of the 2004ACM Conference on Computer Supported Cooperative Work, CSCW ’04,page 507–510, New York, NY, USA, 2004. Association for ComputingMachinery. ISBN 1581138105. doi:10.1145/1031607.1031690. URLhttps://doi.org/10.1145/1031607.1031690. → page 47[105] E. Horvitz, J. Apacible, and M. Subramani. Balancing awareness andinterruption: Investigation of notification deferral policies. In L. Ardissono,P. Brna, and A. Mitrovic, editors, User Modeling 2005, pages 433–437,Berlin, Heidelberg, 2005. Springer Berlin Heidelberg. ISBN978-3-540-31878-1. → pages 4, 44, 45, 47, 50, 79, 115[106] E. C. M. C. E. Horvitz. Notification, disruption, and memory: Effects ofmessaging interruptions on memory and performance. InHuman-Computer Interaction: INTERACT, volume 1, page 263, 2001.URL https://cir.nii.ac.jp/crid/1570854176402202240. → page 1[107] E. Hovdhaugen. Transfer and dropout: different forms of student departurein norway. Studies in Higher Education, 34(1):1–17, 2009.doi:10.1080/03075070802457009. URLhttps://doi.org/10.1080/03075070802457009. → page 74[108] R. Hoyle, S. Das, A. Kapadia, A. J. Lee, and K. Vaniea. Was my messageread? privacy and signaling on facebook messenger. In Proceedings of the2017 CHI Conference on Human Factors in Computing Systems, CHI ’17,page 3838–3842, New York, NY, USA, 2017. Association for ComputingMachinery. ISBN 9781450346559. doi:10.1145/3025453.3025925. URLhttps://doi.org/10.1145/3025453.3025925. → pages 15, 47, 79[109] Y. Huang, L.-Q. Tao, J. Yu, X. Li, Z. Wang, K. Zheng, and X. Chen. Healthmonitoring and automatic notification device based on laser-inducedgraphene. IEEE Transactions on Electron Devices, 67(10):4488–4492,2020. doi:10.1109/TED.2020.3015458. → page 40[110] E. Hunt. One ping after another: why everyone needs a notification detox.https://www.theatlantic.com/technology/archive/2015/07/144push-notifications-versus-phone-calls/398081/, 2020. Accessed:2021-09-01. → pages 5, 6, 79[111] S. T. Iqbal and B. P. Bailey. Effects of intelligent notification managementon users and their tasks. In Proceedings of the SIGCHI Conference onHuman Factors in Computing Systems, CHI ’08, page 93–102, New York,NY, USA, 2008. Association for Computing Machinery. ISBN9781605580111. doi:10.1145/1357054.1357070. URLhttps://doi.org/10.1145/1357054.1357070. → pages 43, 44, 45, 47[112] S. T. Iqbal and E. Horvitz. Notifications and awareness: A field study ofalert usage and preferences. In Proceedings of the 2010 ACM Conferenceon Computer Supported Cooperative Work, CSCW ’10, page 27–30, NewYork, NY, USA, 2010. Association for Computing Machinery. ISBN9781605587950. doi:10.1145/1718918.1718926. URLhttps://doi.org/10.1145/1718918.1718926. → pages 1, 2, 15, 44, 83, 117[113] S. T. Iqbal, X. S. Zheng, and B. P. Bailey. Task-evoked pupillary responseto mental workload in human-computer interaction. In CHI ’04 ExtendedAbstracts on Human Factors in Computing Systems, CHI EA ’04, page1477–1480, New York, NY, USA, 2004. Association for ComputingMachinery. ISBN 1581137036. doi:10.1145/985921.986094. URLhttps://doi.org/10.1145/985921.986094. → page 1[114] T. Ishige and J.-i. Imai. Peripheral notification that does not interfere withuser’s concentration by interactive projection. In 2020 Joint 11thInternational Conference on Soft Computing and Intelligent Systems and21st International Symposium on Advanced Intelligent Systems(SCIS-ISIS), pages 1–5, 2020. doi:10.1109/SCISISIS50064.2020.9322748.→ pages 43, 47[115] M. H. Jarrahi, N. Gafinowitz, and G. Shin. Activity trackers, priormotivation, and perceived informational and motivational affordances.Personal and Ubiquitous Computing, 22(2):433–448, 2018.doi:10.1007/s00779-017-1099-9. URLhttps://doi.org/10.1007/s00779-017-1099-9. → pages 86, 108[116] L. Jia, v. Chudy´, P. Neumeister, and Z. Bo. Study on the job burnout amonguniversity teachers and its countermeasures. In Proceedings of the 2017International Conference on E-Education, E-Business and E-Technology,ICEBT 2017, page 9–12, New York, NY, USA, 2017. Association forComputing Machinery. ISBN 9781450353311.145doi:10.1145/3141151.3141157. URLhttps://doi.org/10.1145/3141151.3141157. → page 2[117] S. L. Jones and R. Kelly. Dealing with information overload inmultifaceted personal informatics systems. Human–Computer Interaction,33(1):1–48, 2018. doi:10.1080/07370024.2017.1302334. URLhttps://doi.org/10.1080/07370024.2017.1302334. → page 86[118] C. H.-L. C. Kao, M.-W. Hung, X. Zhang, P.-C. Huang, and C.-W. You.Probing user perceptions of on-skin notification displays. Proc. ACMHum.-Comput. Interact., 4(CSCW3), jan 2021. doi:10.1145/3432943.URL https://doi.org/10.1145/3432943. → page 43[119] R. Karkar, J. Schroeder, D. A. Epstein, L. R. Pina, J. Scofield, J. Fogarty,J. A. Kientz, S. A. Munson, R. Vilardaga, and J. Zia. Tummytrials: Afeasibility study of using self-experimentation to detect individualized foodtriggers. In Proceedings of the 2017 CHI Conference on Human Factors inComputing Systems, CHI ’17, page 6850–6863, New York, NY, USA,2017. Association for Computing Machinery. ISBN 9781450346559.doi:10.1145/3025453.3025480. URLhttps://doi.org/10.1145/3025453.3025480. → page 110[120] I. Katidioti, J. P. Borst, M. K. van Vugt, and N. A. Taatgen. Interrupt me:External interruptions are less disruptive than self-interruptions. Computersin Human Behavior, 63:906–915, 2016. ISSN 0747-5632.doi:https://doi.org/10.1016/j.chb.2016.06.037. URLhttps://www.sciencedirect.com/science/article/pii/S0747563216304654.→ page 1[121] M. Kay, E. K. Choe, J. Shepherd, B. Greenstein, N. Watson, S. Consolvo,and J. A. Kientz. Lullaby: A capture & access system for understandingthe sleep environment. In Proceedings of the 2012 ACM Conference onUbiquitous Computing, UbiComp ’12, page 226–234, New York, NY,USA, 2012. Association for Computing Machinery. ISBN 9781450312240.doi:10.1145/2370216.2370253. URLhttps://doi.org/10.1145/2370216.2370253. → page 110[122] S. Kayukawa, K. Higuchi, J. a. Guerreiro, S. Morishima, Y. Sato, K. Kitani,and C. Asakawa. Bbeep: A sonic collision avoidance system for blindtravellers and nearby pedestrians. In Proceedings of the 2019 CHIConference on Human Factors in Computing Systems, CHI ’19, page 1–12,New York, NY, USA, 2019. Association for Computing Machinery. ISBN1469781450359702. doi:10.1145/3290605.3300282. URLhttps://doi.org/10.1145/3290605.3300282. → page 40[123] J. T. Kelly, M. M. Kendrick, R. A. Newgent, and C. J. Lucas. Strategies forstudent transition to college: A proactive approach. College StudentJournal, 41(4), 2007. → pages 71, 74[124] S. M. Kennison and E. Ponce-Garcia. The role of childhood relationshipswith older adults in reducing risk-taking by young adults. Journal ofIntergenerational Relationships, 10(1):22–33, 2012.doi:10.1080/15350770.2012.645739. URLhttps://doi.org/10.1080/15350770.2012.645739. → page 75[125] D. T. Kenrick, V. Griskevicius, S. L. Neuberg, and M. Schaller. Renovatingthe pyramid of needs: Contemporary extensions built upon ancientfoundations. Perspectives on Psychological Science, 5(3):292–314, 2010.doi:10.1177/1745691610369469. URLhttps://doi.org/10.1177/1745691610369469. PMID: 21874133. → page 27[126] A. Khalil. Context-aware telephony and its users: Methods to improve theaccuracy of mobile device interruptions. PhD thesis, Indiana University,2006. → page 47[127] V. Khovanskaya, E. P. Baumer, D. Cosley, S. Voida, and G. Gay.”everybody knows what you’re doing”: A critical design approach topersonal informatics. In Proceedings of the SIGCHI Conference on HumanFactors in Computing Systems, CHI ’13, page 3403–3412, New York, NY,USA, 2013. Association for Computing Machinery. ISBN 9781450318990.doi:10.1145/2470654.2466467. URLhttps://doi.org/10.1145/2470654.2466467. → pages 5, 6, 79, 80, 85[128] I. Kim, H. Goh, Y. Noh, and U. Lee. Poster: Exploring User Contexts andNeeds for Context-Aware Smartphone Distraction Management, page41–43. Association for Computing Machinery, New York, NY, USA, 2021.ISBN 9781450384612. URL https://doi.org/10.1145/3460418.3479307.→ pages 45, 47[129] M. Kim, I. Kim, and U. Lee. Beneficial neglect: Instant messagenotification handling behaviors and academic performance. Proc. ACMInteract. Mob. Wearable Ubiquitous Technol., 5(1), mar 2021.doi:10.1145/3448089. URL https://doi.org/10.1145/3448089. → pages45, 47147[130] R. Kojima, R. Legaspi, Y. Mishima, and S. Wada. Classical MachineLearning and Deep Neural Network Ensemble Model for GPS-BasedActivity Recognition, page 369–373. Association for ComputingMachinery, New York, NY, USA, 2021. ISBN 9781450384612. URLhttps://doi.org/10.1145/3460418.3479386. → page 45[131] A. Komninos, A.-E. Frengkou, and J. Garofalakis. Hush now! contextfactors behind smartphone ringer mode changes. Pervasive and MobileComputing, 71:101332, 2021. ISSN 1574-1192.doi:https://doi.org/10.1016/j.pmcj.2021.101332. URLhttps://www.sciencedirect.com/science/article/pii/S1574119221000080.→ page 44[132] F. Kunzler, J.-N. Kramer, and T. Kowatsch. Efficacy of mobilecontext-aware notification management systems: A systematic literaturereview and meta-analysis. In 2017 IEEE 13th International Conference onWireless and Mobile Computing, Networking and Communications(WiMob), pages 131–138, 2017. doi:10.1109/WiMOB.2017.8115839. →pages 40, 45, 73, 117[133] S. Kurniawan. An exploratory study of how older women use mobilephones. In UbiComp 2006: Ubiquitous Computing, pages 105–122, Berlin,Heidelberg, 2006. Springer Berlin Heidelberg. ISBN 978-3-540-39635-2.→ pages 13, 17[134] K. Kushlev and E. W. Dunn. Checking email less frequently reduces stress.Computers in Human Behavior, 43:220–228, 2015. ISSN 0747-5632.doi:https://doi.org/10.1016/j.chb.2014.11.005. URLhttps://www.sciencedirect.com/science/article/pii/S0747563214005810.→ pages 1, 2, 31, 40, 79, 110, 126[135] K. Kushlev, J. Proulx, and E. W. Dunn. ”silence your phones”: Smartphonenotifications increase inattention and hyperactivity symptoms. InProceedings of the 2016 CHI Conference on Human Factors in ComputingSystems, CHI ’16, page 1011–1020, New York, NY, USA, 2016.Association for Computing Machinery. ISBN 9781450333627.doi:10.1145/2858036.2858359. URLhttps://doi.org/10.1145/2858036.2858359. → pages 1, 2, 40[136] M. Kuznetsov, E. Novikova, and I. Kotenko. An approach to formaldesription of the user notification scenarios in privacy policies. In 202230th Euromicro International Conference on Parallel, Distributed and148Network-based Processing (PDP), pages 275–282, 2022.doi:10.1109/PDP55904.2022.00049. → page 125[137] J. C. LAM and M. K. LEE. Digital inclusiveness–longitudinal study ofinternet adoption by older adults. Journal of Management InformationSystems, 22(4):177–206, 2006. doi:10.2753/MIS0742-1222220407. URLhttps://doi.org/10.2753/MIS0742-1222220407. → pages 13, 17, 41[138] P. Langley. Machine learning for adaptive user interfaces. In G. Brewka,C. Habel, and B. Nebel, editors, KI-97: Advances in Artificial Intelligence,pages 53–62, Berlin, Heidelberg, 1997. Springer Berlin Heidelberg. ISBN978-3-540-69582-0. → page 84[139] H.-P. Lee, K.-Y. Chen, C.-H. Lin, C.-Y. Chen, Y.-L. Chung, Y.-J. Chang,and C.-R. Sun. Does who matter? studying the impact of relationshipcharacteristics on receptivity to mobile im messages. In Proceedings of the2019 CHI Conference on Human Factors in Computing Systems, CHI ’19,page 1–12, New York, NY, USA, 2019. Association for ComputingMachinery. ISBN 9781450359702. doi:10.1145/3290605.3300756. URLhttps://doi.org/10.1145/3290605.3300756. → pages 44, 50, 51[140] J. Lee, E. Walker, W. Burleson, M. Kay, M. Buman, and E. B. Hekler.Self-experimentation for behavior change: Design and formative evaluationof two approaches. In Proceedings of the 2017 CHI Conference on HumanFactors in Computing Systems, CHI ’17, page 6837–6849, New York, NY,USA, 2017. Association for Computing Machinery. ISBN 9781450346559.doi:10.1145/3025453.3026038. URLhttps://doi.org/10.1145/3025453.3026038. → page 110[141] J. Lee, S. Park, T. Kim, and H. Kim. Time-invariant features-based onlinelearning for long-term notification management: A longitudinal study.Applied Sciences, 12(11), 2022. ISSN 2076-3417.doi:10.3390/app12115432. URLhttps://www.mdpi.com/2076-3417/12/11/5432. → page 45[142] K. Lee and H. Hong. Mindnavigator: Exploring the stress andself-interventions for mental wellness. In Proceedings of the 2018 CHIConference on Human Factors in Computing Systems, page 1–14, NewYork, NY, USA, 2018. Association for Computing Machinery. ISBN9781450356206. URL https://doi.org/10.1145/3173574.3174146. → page110149[143] I. Li, A. K. Dey, and J. Forlizzi. Understanding my data, myself:Supporting self-reflection with ubicomp technologies. In Proceedings ofthe 13th International Conference on Ubiquitous Computing, UbiComp’11, page 405–414, New York, NY, USA, 2011. Association for ComputingMachinery. ISBN 9781450306300. doi:10.1145/2030112.2030166. URLhttps://doi.org/10.1145/2030112.2030166. → pages 73, 80, 85[144] T. Li, J. K. Haines, M. F. R. de Eguino, J. I. Hong, and J. Nichols. Alertnow or never: Understanding and predicting notification preferences ofsmartphone users. ACM Trans. Comput.-Hum. Interact., jul 2021. ISSN1073-0516. doi:10.1145/3478868. URL https://doi.org/10.1145/3478868.→ pages 4, 5, 42, 45, 46, 47, 50, 53, 66, 68, 117, 118, 124[145] T.-C. Lin, Y.-S. Su, E. H. Yang, Y. H. Chen, H.-P. Lee, and Y.-J. Chang.“put it on the top, i’ll read it later”: Investigating users’ desired displayorder for smartphone notifications. In Proceedings of the 2021 CHIConference on Human Factors in Computing Systems, CHI ’21, New York,NY, USA, 2021. Association for Computing Machinery. ISBN9781450380966. doi:10.1145/3411764.3445384. URLhttps://doi.org/10.1145/3411764.3445384. → pages 4, 45, 50[146] D. Lupton. Self-tracking cultures: Towards a sociology of personalinformatics. In Proceedings of the 26th Australian Computer-HumanInteraction Conference on Designing Futures: The Future of Design,OzCHI ’14, page 77–86, New York, NY, USA, 2014. Association forComputing Machinery. ISBN 9781450306539.doi:10.1145/2686612.2686623. URLhttps://doi.org/10.1145/2686612.2686623. → page 85[147] A. MacLean, R. Young, V. Bellotti, and T. Moran. Design space analysis:Bridging from theory to practice via design rationale. Proceedings ofEsprit, 91:720–730, 1991. → page 47[148] M. Mahapatra and S. P. Pati. Technostress creators and burnout: A jobdemands-resources perspective. In Proceedings of the 2018 ACM SIGMISConference on Computers and People Research, SIGMIS-CPR’18, page70–77, New York, NY, USA, 2018. Association for Computing Machinery.ISBN 9781450357685. doi:10.1145/3209626.3209711. URLhttps://doi.org/10.1145/3209626.3209711. → page 2[149] A. Mairena, C. Gutwin, and A. Cockburn. Peripheral notifications in largedisplays: Effects of feature combination and task interference. In150Proceedings of the 2019 CHI Conference on Human Factors in ComputingSystems, CHI ’19, page 1–12, New York, NY, USA, 2019. Association forComputing Machinery. ISBN 9781450359702.doi:10.1145/3290605.3300870. URLhttps://doi.org/10.1145/3290605.3300870. → pages 43, 44, 47[150] G. Mark, Y. Wang, and M. Niiya. Stress and multitasking in everydaycollege life: An empirical study of online activity. In Proceedings of theSIGCHI Conference on Human Factors in Computing Systems, CHI ’14,page 41–50, New York, NY, USA, 2014. Association for ComputingMachinery. ISBN 9781450324731. doi:10.1145/2556288.2557361. URLhttps://doi.org/10.1145/2556288.2557361. → page 2[151] M. Mather, N. Mazar, M. A. Gorlick, N. R. Lighthall, J. Burgeno,A. Schoeke, and D. Ariely. Risk preferences and aging: The “certaintyeffect” in older adults’ decision making. Psychology and aging, 27(4):801,2012. ISSN 1939-149. doi:https://doi.org/10.1037/a0030174. → page 75[152] S. Mayer, L. Lischke, P. W. Woz´niak, and N. Henze. Evaluating thedisruptiveness of mobile interactions: A mixed-method approach. InProceedings of the 2018 CHI Conference on Human Factors in ComputingSystems, CHI ’18, page 1–14, New York, NY, USA, 2018. Association forComputing Machinery. ISBN 9781450356206.doi:10.1145/3173574.3173980. URLhttps://doi.org/10.1145/3173574.3173980. → pages 1, 44[153] D. S. McCrickard, C. M. Chewar, J. P. Somervell, and A. Ndiwalana. Amodel for notification systems evaluation—assessing user goals formultitasking activity. ACM Trans. Comput.-Hum. Interact., 10(4):312–338,dec 2003. ISSN 1073-0516. doi:10.1145/966930.966933. URLhttps://doi.org/10.1145/966930.966933. → pages 2, 40, 43, 47[154] D. C. McFarlane. Comparison of four primary methods for coordinatingthe interruption of people in human-computer interaction.Human–Computer Interaction, 17(1):63–139, 2002.doi:10.1207/S15327051HCI1701\ 2. URLhttps://doi.org/10.1207/S15327051HCI1701 2. → page 44[155] D. C. McFarlane and K. A. Latorella. The scope and importance of humaninterruption in human-computer interaction design. Human–ComputerInteraction, 17(1):1–61, 2002. doi:10.1207/S15327051HCI1701\ 1. URLhttps://doi.org/10.1207/S15327051HCI1701 1. → page 43151[156] A. R. McNeill, L. Coventry, J. Pywell, and P. Briggs. Privacyconsiderations when designing social network systems to supportsuccessful ageing. In Proceedings of the 2017 CHI Conference on HumanFactors in Computing Systems, CHI ’17, page 6425–6437, New York, NY,USA, 2017. Association for Computing Machinery. ISBN 9781450346559.doi:10.1145/3025453.3025861. URLhttps://doi.org/10.1145/3025453.3025861. → page 17[157] A. Mehrotra and M. Musolesi. Intelligent notification systems: A survey ofthe state of the art and research challenges. CoRR, abs/1711.10171, 2017.URL http://arxiv.org/abs/1711.10171. → pages 4, 40, 43, 44, 45, 73, 117[158] A. Mehrotra, M. Musolesi, R. Hendley, and V. Pejovic. Designingcontent-driven intelligent notification mechanisms for mobile applications.In Proceedings of the 2015 ACM International Joint Conference onPervasive and Ubiquitous Computing, UbiComp ’15, page 813–824, NewYork, NY, USA, 2015. Association for Computing Machinery. ISBN9781450335744. doi:10.1145/2750858.2807544. URLhttps://doi.org/10.1145/2750858.2807544. → pages 40, 44, 45, 47, 50, 124[159] A. Mehrotra, R. Hendley, and M. Musolesi. Prefminer: Mining user’spreferences for intelligent mobile notification management. In Proceedingsof the 2016 ACM International Joint Conference on Pervasive andUbiquitous Computing, UbiComp ’16, page 1223–1234, New York, NY,USA, 2016. Association for Computing Machinery. ISBN 9781450344616.doi:10.1145/2971648.2971747. URLhttps://doi.org/10.1145/2971648.2971747. → pages2, 3, 4, 5, 13, 14, 15, 17, 31, 36, 40, 41, 44, 45, 46, 50, 52, 53, 54, 57, 58, 59, 72, 79, 80, 81, 82, 83, 84, 85, 89, 100, 110, 117, 118, 120, 124, 126, 127[160] A. Mehrotra, V. Pejovic, J. Vermeulen, R. Hendley, and M. Musolesi. Myphone and me: Understanding people’s receptivity to mobile notifications.In Proceedings of the 2016 CHI Conference on Human Factors inComputing Systems, page 1021–1032, New York, NY, USA, 2016.Association for Computing Machinery. ISBN 9781450333627. URLhttps://doi.org/10.1145/2858036.2858566. → pages2, 3, 4, 5, 15, 40, 44, 46, 47, 51, 58, 60, 73, 79, 80, 81, 83, 84, 87, 88, 89, 90, 91, 110, 114, 124, 126[161] R. Meyer. Push notifications are as distracting as phone calls.https://www.theatlantic.com/technology/archive/2015/07/push-notifications-versus-phone-calls/398081/, 2015. Accessed:2021-09-01. → pages 5, 6, 79152[162] J. Mezirow. On critical reflection. Adult Education Quarterly, 48(3):185–198, 1998. doi:10.1177/074171369804800305. URLhttps://doi.org/10.1177/074171369804800305. → page 85[163] C. A. Monk, D. A. Boehm-Davis, and J. G. Trafton. The attentional costsof interrupting task performance at various stages. Proceedings of theHuman Factors and Ergonomics Society Annual Meeting, 46(22):1824–1828, 2002. doi:10.1177/154193120204602210. URLhttps://doi.org/10.1177/154193120204602210. → page 1[164] P. Mu¨ller, S. Staal, M. Baˆce, and A. Bulling. Designing for noticeability:Understanding the impact of visual importance on desktop notifications. InCHI Conference on Human Factors in Computing Systems, CHI ’22, NewYork, NY, USA, 2022. Association for Computing Machinery. ISBN9781450391573. doi:10.1145/3491102.3501954. URLhttps://doi.org/10.1145/3491102.3501954. → pages 4, 47, 50[165] S. A. Munson. Rethinking assumptions in the design of health and wellnesstracking tools. Interactions, 25(1):62–65, dec 2017. ISSN 1072-5520.doi:10.1145/3168738. URL https://doi.org/10.1145/3168738. → page 110[166] T. Munzner. A nested model for visualization design and validation. IEEETransactions on Visualization and Computer Graphics, 15(6):921–928,2009. doi:10.1109/TVCG.2009.111. → pages 80, 81, 88[167] Y. Muranaka, M. Al-Sada, and T. Nakajima. Unobtrusive notification basedon auditory changes in ambient music. In 2021 IEEE 3rd GlobalConference on Life Sciences and Technologies (LifeTech), pages 231–234,2021. doi:10.1109/LifeTech52111.2021.9391913. → page 40[168] E. L. Murnane, T. G. Walker, B. Tench, S. Voida, and J. Snyder. Personalinformatics in interpersonal contexts: Towards the design of technologythat supports the social ecologies of long-term mental health management.Proc. ACM Hum.-Comput. Interact., 2(CSCW), nov 2018.doi:10.1145/3274396. URL https://doi.org/10.1145/3274396. → pages6, 85, 110[169] N. Nicholson, E. Soane, M. Fenton-O’Creevy, and P. Willman. Personalityand domain-specific risk taking. Journal of Risk Research, 8(2):157–176,2005. doi:10.1080/1366987032000123856. URLhttps://doi.org/10.1080/1366987032000123856. → page 75153[170] H. Nigam, A. K. Saini, S. Banerjee, and A. Kumar. Indoor environment airquality monitoring and its notification to building occupants. In TENCON2019 - 2019 IEEE Region 10 Conference (TENCON), pages 2444–2448,2019. doi:10.1109/TENCON.2019.8929232. → page 40[171] A.-M. Nivala and L. T. Sarjakoski. Preventing interruptions in mobile mapreading process by personalisation. In Proceedings of The 3rd Workshopon’HCI in Mobile Guides, pages 13–16, 2004. → page 47[172] M. Nouwens, C. F. Griggio, and W. E. Mackay. ”whatsapp is for family;messenger is for friends”: Communication places in app ecosystems. InProceedings of the 2017 CHI Conference on Human Factors in ComputingSystems, CHI ’17, page 727–735, New York, NY, USA, 2017. Associationfor Computing Machinery. ISBN 9781450346559.doi:10.1145/3025453.3025484. URLhttps://doi.org/10.1145/3025453.3025484. → pages 3, 15, 44, 79, 91[173] M. Nouwens, I. Liccardi, M. Veale, D. Karger, and L. Kagal. Dark Patternsafter the GDPR: Scraping Consent Pop-Ups and Demonstrating TheirInfluence, page 1–13. Association for Computing Machinery, New York,NY, USA, 2020. ISBN 9781450367080. URLhttps://doi.org/10.1145/3313831.3376321. → page 126[174] R. Nunes Oliveira, V. Roth, A. Felippeto Henzen, J. M. Simao, P. Nohama,and E. C. Gomes Wille. Notification oriented paradigm applied to ambientassisted living tool. IEEE Latin America Transactions, 16(2):647–653,2018. doi:10.1109/TLA.2018.8327425. → page 43[175] M. Nurek and R. Michalski. Combining machine learning and socialnetwork analysis to reveal the organizational structures. Applied Sciences,10(5), 2020. ISSN 2076-3417. doi:10.3390/app10051699. URLhttps://www.mdpi.com/2076-3417/10/5/1699. → page 120[176] L. A. of British Columbia. Bill 22 – 2021: Freedom of information andprotection of privacy amendment act. https://www.leg.bc.ca/parliamentary-business/legislation-debates-proceedings/42nd-parliament/2nd-session/bills/first-reading/gov22-1, 2021. Accessed:2022-3-11. → page 125[177] N. A. of Quebec. Projet de loi n 64. http://m.assnat.qc.ca/en/travaux-parlementaires/projets-loi/projet-loi-64-42-1.html, 2021.Accessed: 2022-3-11. → page 125154[178] A. Offenwanger, A. J. Milligan, M. Chang, J. Bullard, and D. Yoon.Diagnosing bias in the gender representation of hci research participants:How it happens and where we are. In Proceedings of the 2021 CHIConference on Human Factors in Computing Systems, CHI ’21, New York,NY, USA, 2021. Association for Computing Machinery. ISBN9781450380966. doi:10.1145/3411764.3445383. URLhttps://doi.org/10.1145/3411764.3445383. → page 93[179] H. Ogawa, K. Matsumura, and A. Fujii. Appropriate timing and length ofvoice news notifications. In ACM International Conference on InteractiveMedia Experiences, IMX ’21, page 194–198, New York, NY, USA, 2021.Association for Computing Machinery. ISBN 9781450383899.doi:10.1145/3452918.3465488. URLhttps://doi.org/10.1145/3452918.3465488. → pages 43, 47[180] F. Ohlin and C. M. Olsson. Intelligent computing in personal informatics:Key design considerations. In Proceedings of the 20th InternationalConference on Intelligent User Interfaces, IUI ’15, page 263–274, NewYork, NY, USA, 2015. Association for Computing Machinery. ISBN9781450333061. doi:10.1145/2678025.2701378. URLhttps://doi.org/10.1145/2678025.2701378. → page 86[181] F. Ohlin, C. M. Olsson, and P. Davidsson. Analyzing the design space ofpersonal informatics: A state-of-practice based classification of existingtools. In M. Antona and C. Stephanidis, editors, Universal Access inHuman-Computer Interaction. Access to Today’s Technologies, pages85–97, Cham, 2015. Springer International Publishing. ISBN978-3-319-20678-3. → page 86[182] T. Okoshi, J. Ramos, H. Nozaki, J. Nakazawa, A. K. Dey, and H. Tokuda.Reducing users’ perceived mental effort due to interruptive notifications inmulti-device mobile environments. In Proceedings of the 2015 ACMInternational Joint Conference on Pervasive and Ubiquitous Computing,UbiComp ’15, page 475–486, New York, NY, USA, 2015. Association forComputing Machinery. ISBN 9781450335744.doi:10.1145/2750858.2807517. URLhttps://doi.org/10.1145/2750858.2807517. → page 47[183] T. Okoshi, J. Nakazawa, and H. Tokuda. Interruptibility research:Opportunities for future flourishment. In Proceedings of the 2016 ACMInternational Joint Conference on Pervasive and Ubiquitous Computing:Adjunct, UbiComp ’16, page 1524–1529, New York, NY, USA, 2016.155Association for Computing Machinery. ISBN 9781450344623.doi:10.1145/2968219.2968543. URLhttps://doi.org/10.1145/2968219.2968543. → pages 1, 2, 43, 44, 79, 84[184] A. Oulasvirta, T. Rattenbury, L. Ma, and E. Raita. Habits make smartphoneuse more pervasive. Personal and Ubiquitous computing, 16(1):105–114,2012. doi:10.1007/s00779-011-0412-2. URLhttps://doi.org/10.1007/s00779-011-0412-2. → page 1[185] C. Z. Patrikakis, D. G. Kogias, C. Chatzigeorgiou, D. Kalyvas,E. Katsadouros, and C. Giannousis. A method for measuring urban spacedensity of people and deliver notification, with respect to privacy. In 2021IEEE International Conference on Consumer Electronics (ICCE), pages1–6, 2021. doi:10.1109/ICCE50685.2021.9427758. → page 45[186] C. L. Paul, A. Komlodi, and W. Lutters. Again?!! the emotional experienceof social notification interruptions. In P. Campos, N. Graham, J. Jorge,N. Nunes, P. Palanque, and M. Winckler, editors, Human-ComputerInteraction – INTERACT 2011, pages 471–478, Berlin, Heidelberg, 2011.Springer Berlin Heidelberg. ISBN 978-3-642-23771-3. → page 2[187] V. Pejovic and M. Musolesi. Interruptme: Designing intelligent promptingmechanisms for pervasive applications. In Proceedings of the 2014 ACMInternational Joint Conference on Pervasive and Ubiquitous Computing,UbiComp ’14, page 897–908, New York, NY, USA, 2014. Association forComputing Machinery. ISBN 9781450329682.doi:10.1145/2632048.2632062. URLhttps://doi.org/10.1145/2632048.2632062. → pages 1, 15, 47, 79, 120[188] V. Pejovic and M. Musolesi. Anticipatory mobile computing: A survey ofthe state of the art and research challenges. ACM Comput. Surv., 47(3), apr2015. ISSN 0360-0300. doi:10.1145/2693843. URLhttps://doi.org/10.1145/2693843. → pages 4, 40, 41, 44, 45[189] S. A. Petersen, J. Cassens, A. Kofod-Petersen, and M. Divitini. To be ornot to be aware: Reducing interruptions in pervasive awareness systems. In2008 The Second International Conference on Mobile UbiquitousComputing, Systems, Services and Technologies, pages 327–332, 2008.doi:10.1109/UBICOMM.2008.35. → page 47[190] J. Petford, I. Carson, M. A. Nacenta, and C. Gutwin. A comparison ofnotification techniques for out-of-view objects in full-coverage displays. In156Proceedings of the 2019 CHI Conference on Human Factors in ComputingSystems, CHI ’19, page 1–13, New York, NY, USA, 2019. Association forComputing Machinery. ISBN 9781450359702.doi:10.1145/3290605.3300288. URLhttps://doi.org/10.1145/3290605.3300288. → pages 43, 47[191] M. Pielot, K. Church, and R. de Oliveira. An in-situ study of mobile phonenotifications. In Proceedings of the 16th International Conference onHuman-Computer Interaction with Mobile Devices & Services, MobileHCI’14, page 233–242, New York, NY, USA, 2014. Association for ComputingMachinery. ISBN 9781450330046. doi:10.1145/2628363.2628364. URLhttps://doi.org/10.1145/2628363.2628364. → pages2, 5, 13, 15, 40, 44, 79, 81, 83, 84, 90, 91[192] M. Pielot, R. de Oliveira, H. Kwak, and N. Oliver. Didn’t you see mymessage? predicting attentiveness to mobile instant messages. InProceedings of the SIGCHI Conference on Human Factors in ComputingSystems, CHI ’14, page 3319–3328, New York, NY, USA, 2014.Association for Computing Machinery. ISBN 9781450324731.doi:10.1145/2556288.2556973. URLhttps://doi.org/10.1145/2556288.2556973. → pages 47, 79[193] M. Pielot, T. Dingler, J. S. Pedro, and N. Oliver. When attention is notscarce - detecting boredom from mobile phone usage. In Proceedings ofthe 2015 ACM International Joint Conference on Pervasive and UbiquitousComputing, UbiComp ’15, page 825–836, New York, NY, USA, 2015.Association for Computing Machinery. ISBN 9781450335744.doi:10.1145/2750858.2804252. URLhttps://doi.org/10.1145/2750858.2804252. → page 47[194] A. Pirzadeh, L. He, and E. Stolterman. Personal informatics and reflection:A critical examination of the nature of reflection. In CHI ’13 ExtendedAbstracts on Human Factors in Computing Systems, CHI EA ’13, page1979–1988. Association for Computing Machinery, New York, NY, USA,2013. ISBN 9781450319522. doi:10.1145/2468356.2468715. URLhttps://doi.org/10.1145/2468356.2468715. → pages 80, 85[195] R. Poguntke, C. Schneegass, L. Van der Vekens, R. Rzayev, J. Auda,S. Schneegass, and A. Schmidt. Notimodes: An investigation ofnotification delay modes and their effects on smartphone users. InProceedings of the Conference on Mensch Und Computer, MuC ’20, page415–419, New York, NY, USA, 2020. Association for Computing157Machinery. ISBN 9781450375405. doi:10.1145/3404983.3410006. URLhttps://doi.org/10.1145/3404983.3410006. → pages 2, 4, 5, 45, 47, 82, 83[196] S. Porcu, A. Floris, M. Anedda, V. Popescu, M. Fadda, and L. Atzori.Quality of experience eye gaze analysis on hbbtv smart home notificationsystem. In 2020 IEEE International Symposium on Broadband MultimediaSystems and Broadcasting (BMSB), pages 1–6, 2020.doi:10.1109/BMSB49480.2020.9379794. → page 44[197] Z. Pousman and J. Stasko. A taxonomy of ambient information systems:Four patterns of design. In Proceedings of the Workshop on AdvancedVisual Interfaces, volume 2006, pages 67–74, 01 2006.doi:10.1145/1133265.1133277. → page 48[198] A. Prasad, L. S. LaFreniere, V. Taneja, and Z. Beals. AddressingProblematic Smartphone Use with a Personalized, Goal-Based Approach,page 131–134. Association for Computing Machinery, New York, NY,USA, 2021. ISBN 9781450384612. URLhttps://doi.org/10.1145/3460418.3479319. → page 46[199] J. O. Prochaska and W. F. Velicer. The transtheoretical model of healthbehavior change. American journal of health promotion, 12(1):38–48,1997. URL https://cir.nii.ac.jp/crid/1570572700023323648. → pages73, 80[200] H. Puranik, J. Koopman, and H. C. Vough. Pardon the interruption: Anintegrative review and future research agenda for research on workinterruptions. Journal of Management, 46(6):806–842, 2020.doi:10.1177/0149206319887428. URLhttps://doi.org/10.1177/0149206319887428. → pages 40, 43, 117[201] A. Puussaar, A. K. Clear, and P. Wright. Enhancing personal informaticsthrough social sensemaking. In Proceedings of the 2017 CHI Conferenceon Human Factors in Computing Systems, CHI ’17, page 6936–6942, NewYork, NY, USA, 2017. Association for Computing Machinery. ISBN9781450346559. doi:10.1145/3025453.3025804. URLhttps://doi.org/10.1145/3025453.3025804. → pages 6, 85[202] P. Ralph. Toward methodological guidelines for process theories andtaxonomies in software engineering. IEEE Transactions on SoftwareEngineering, 45(7):712–735, 2019. doi:10.1109/TSE.2018.2796554. →page 118158[203] A. Rapp and F. Cena. Self-monitoring and technology: Challenges andopen issues in personal informatics. In C. Stephanidis and M. Antona,editors, Universal Access in Human-Computer Interaction. Design for Alland Accessibility Practice, pages 613–622, Cham, 2014. SpringerInternational Publishing. ISBN 978-3-319-07509-9. → pages 80, 85, 86[204] A. Rapp and F. Cena. Personal informatics for everyday life: How userswithout prior self-tracking experience engage with personal data.International Journal of Human-Computer Studies, 94:1–17, 2016. ISSN1071-5819. doi:https://doi.org/10.1016/j.ijhcs.2016.05.006. URLhttps://www.sciencedirect.com/science/article/pii/S107158191630060X.→ pages 80, 85, 108[205] A. Rapp and M. Tirassa. Know thyself: A theory of the self for personalinformatics. Human–Computer Interaction, 32(5-6):335–380, 2017.doi:10.1080/07370024.2017.1285704. URLhttps://doi.org/10.1080/07370024.2017.1285704. → page 85[206] A. Rapp, A. Marcengo, L. Buriano, G. Ruffo, M. Lai, and F. Cena.Designing a personal informatics system for users without experience inself-tracking: a case study. Behaviour & Information Technology, 37(4):335–366, 2018. doi:10.1080/0144929X.2018.1436592. URLhttps://doi.org/10.1080/0144929X.2018.1436592. → page 86[207] R. M. Ratwani, A. E. Andrews, J. D. Sousk, and J. G. Trafton. The effect ofinterruption modality on primary task resumption. In Proceedings of theHuman Factors and Ergonomics Society Annual Meeting, volume 52,pages 393–397, 2008. doi:10.1177/154193120805200441. URLhttps://doi.org/10.1177/154193120805200441. → pages 2, 4, 5, 83, 115[208] RescueTime. Rescuetime. https://www.rescuetime.com/, 2021. Accessed:2021-09-1. → pages 2, 85[209] L. Richardson. New writing practices in qualitative research. Sociology ofsport journal, 17(1):5–20, 2000. → pages 22, 64[210] L. Richardson and E. St Pierre. A method of inquiry. Collecting andinterpreting qualitative materials, 3(4):473, 2008. → page 22[211] M. D. Rodrı´guez, V. M. Gonzalez, J. Favela, and P. C. Santana.Home-based communication system for older adults and their remotefamily. Computers in Human Behavior, 25(3):609–618, 2009. ISSN0747-5632. doi:https://doi.org/10.1016/j.chb.2008.08.017. URL159https://www.sciencedirect.com/science/article/pii/S0747563208001684.→ pages 17, 121[212] A. M. Roffarello and L. De Russis. Understanding, discovering, andmitigating habitual smartphone use in young adults. ACM Trans. Interact.Intell. Syst., 11(2), jul 2021. ISSN 2160-6455. doi:10.1145/3447991. URLhttps://doi.org/10.1145/3447991. → pages 46, 67[213] S. Rosenthal, A. K. Dey, and M. Veloso. Using decision-theoreticexperience sampling to build personalized mobile phone interruptionmodels. In K. Lyons, J. Hightower, and E. M. Huang, editors, PervasiveComputing, pages 170–187, Berlin, Heidelberg, 2011. Springer BerlinHeidelberg. ISBN 978-3-642-21726-5. → page 47[214] A. Sahami Shirazi, N. Henze, T. Dingler, M. Pielot, D. Weber, andA. Schmidt. Large-scale assessment of mobile notifications. InProceedings of the SIGCHI Conference on Human Factors in ComputingSystems, CHI ’14, page 3055–3064, New York, NY, USA, 2014.Association for Computing Machinery. ISBN 9781450324731.doi:10.1145/2556288.2557189. URLhttps://doi.org/10.1145/2556288.2557189. → pages2, 5, 6, 15, 40, 44, 79, 80, 82, 84, 90, 109, 114, 126, 127[215] C. Schmandt and S. Marti. Active messenger: E-mail filtering and deliveryin a heterogeneous network. Human–Computer Interaction, 20(1-2):163–194, 2005. doi:10.1080/07370024.2005.9667364. URLhttps://www.tandfonline.com/doi/abs/10.1080/07370024.2005.9667364.→ page 47[216] F. Schulze and G. Groh. Conversational context helps improve mobilenotification management. In Proceedings of the 18th InternationalConference on Human-Computer Interaction with Mobile Devices andServices, MobileHCI ’16, page 518–528, New York, NY, USA, 2016.Association for Computing Machinery. ISBN 9781450344081.doi:10.1145/2935334.2935347. URLhttps://doi.org/10.1145/2935334.2935347. → page 44[217] J. S. Seberger, I. Shklovski, E. Swiatek, and S. Patil. Still creepy after allthese years:the normalization of affective discomfort in app use. In CHIConference on Human Factors in Computing Systems, CHI ’22, New York,NY, USA, 2022. Association for Computing Machinery. ISBN1609781450391573. doi:10.1145/3491102.3502112. URLhttps://doi.org/10.1145/3491102.3502112. → pages 5, 124, 125[218] P. Sengers, K. Boehner, S. David, and J. J. Kaye. Reflective design. InProceedings of the 4th Decennial Conference on Critical Computing:Between Sense and Sensibility, CC ’05, page 49–58, New York, NY, USA,2005. Association for Computing Machinery. ISBN 1595932038.doi:10.1145/1094562.1094569. URLhttps://doi.org/10.1145/1094562.1094569. → pages 5, 79, 80, 85[219] I.-g. Shin, J.-m. Seok, and Y.-k. Lim. Ten-minute silence: A newnotification ux of mobile instant messenger. In Proceedings of the 2019CHI Conference on Human Factors in Computing Systems, CHI ’19, page1–13, New York, NY, USA, 2019. Association for Computing Machinery.ISBN 9781450359702. doi:10.1145/3290605.3300672. URLhttps://doi.org/10.1145/3290605.3300672. → page 45[220] A. S. Shirazi and N. Henze. Assessment of notifications on smartwatches.In Proceedings of the 17th International Conference on Human-ComputerInteraction with Mobile Devices and Services Adjunct, MobileHCI ’15,page 1111–1116, New York, NY, USA, 2015. Association for ComputingMachinery. ISBN 9781450336536. doi:10.1145/2786567.2794338. URLhttps://doi.org/10.1145/2786567.2794338. → pages 43, 44, 47[221] N. Silva, D. Carvalho, A. C. Pereira, F. Moura˜o, and L. Rocha. The purecold-start problem: A deep study about how to conquer first-time users inrecommendations domains. Information Systems, 80:1–12, 2019. ISSN0306-4379. doi:https://doi.org/10.1016/j.is.2018.09.001. URLhttps://www.sciencedirect.com/science/article/pii/S0306437918303260.→ page 58[222] SoloVPN and N. N. CleanerProductivity. Notification blocker and cleanerand heads-up off.https://play.google.com/store/apps/details?id=co.easy4u.ncleaner, 2020.Accessed: 2020-3-11. → pages 4, 45, 48[223] C. Stothart, A. Mitchum, and C. Yehnert. The attentional cost of receivinga cell phone notification. Journal of experimental psychology: humanperception and performance, 41(4):893, 2015. ISSN 1939-1277.doi:https://doi.org/10.1037/xhp0000100. → page 1[224] R. Sutton, K. Fraser, and O. Conlan. A reinforcement learning andsynthetic data approach to mobile notification management. In161Proceedings of the 17th International Conference on Advances in MobileComputing and Multimedia, MoMM2019, page 155–164, New York, NY,USA, 2019. Association for Computing Machinery. ISBN 9781450371780.doi:10.1145/3365921.3365932. URLhttps://doi.org/10.1145/3365921.3365932. → page 58[225] Y. Takayama, S. Ishii, A. Yokokubo, and G. Lopez. Detecting ForwardLeaning Posture Using ESense and Developing a Posture ImprovementPromoting System, page 178–179. Association for Computing Machinery,New York, NY, USA, 2021. ISBN 9781450384612. URLhttps://doi.org/10.1145/3460418.3479321. → pages 45, 120[226] N. Terzimehic´, R. Ha¨uslschmid, H. Hussmann, and m. schraefel. A review& analysis of mindfulness research in hci: Framing current lines ofresearch and future opportunities. In Proceedings of the 2019 CHIConference on Human Factors in Computing Systems, CHI ’19, page 1–13,New York, NY, USA, 2019. Association for Computing Machinery. ISBN9781450359702. doi:10.1145/3290605.3300687. URLhttps://doi.org/10.1145/3290605.3300687. → page 47[227] S. M. Tham and G. P. Perreault. A whale of a tale: Gaming disorder andspending and their associations with ad watching in role-playing andloot-box gaming. Journal of Gambling Issues, 46, 2021.doi:https://doi.org/10.4309/jgi.2021.46.5. → page 4[228] S. Tok. The big five personality traits and risky sport participation. SocialBehavior and Personality: an international journal, 39(8):1105–1111,2011. ISSN 0301-2212. doi:doi:10.2224/sbp.2011.39.8.1105. URLhttps://www.ingentaconnect.com/content/sbp/sbp/2011/00000039/00000008/art00010. → page 75[229] G. F. Tondello, R. Orji, and L. E. Nacke. Recommender systems forpersonalized gamification. In Adjunct Publication of the 25th Conferenceon User Modeling, Adaptation and Personalization, UMAP ’17, page425–430, New York, NY, USA, 2017. Association for ComputingMachinery. ISBN 9781450350679. doi:10.1145/3099023.3099114. URLhttps://doi.org/10.1145/3099023.3099114. → pages 5, 45[230] R. Totchinsky. How period tracking apps and data privacy fit into apost-roe v. wade climate. https://www.npr.org/2022/05/10/1097482967/roe-v-wade-supreme-court-abortion-period-apps, 2022. Accessed:2022-05-10. → page 124162[231] B. K. Townsend and K. Wilson. “a hand hold for a little bit”: Factorsfacilitating the success of community college transfer students to a largeresearch university. Journal of College Student Development, 47(4):439–456, 2006. doi:doi:10.1353/csd.2006.0052. → page 74[232] S. J. Tracy. Qualitative quality: Eight “big-tent” criteria for excellentqualitative research. Qualitative Inquiry, 16(10):837–851, 2010.doi:10.1177/1077800410383121. URLhttps://doi.org/10.1177/1077800410383121. → page 23[233] S. Trewin, J. T. Richards, V. L. Hanson, D. Sloan, B. E. John, C. Swart, andJ. C. Thomas. Understanding the role of age and fluid intelligence ininformation search. In Proceedings of the 14th International ACMSIGACCESS Conference on Computers and Accessibility, ASSETS ’12,page 119–126, New York, NY, USA, 2012. Association for ComputingMachinery. ISBN 9781450313216. doi:10.1145/2384916.2384938. URLhttps://doi.org/10.1145/2384916.2384938. → pages 17, 121[234] T. J. Trull and U. W. Ebner-Priemer. Using experience samplingmethods/ecological momentary assessment (esm/ema) in clinicalassessment and clinical research: introduction to the special section.Psychological assessment, 21(4), 2009. ISSN 1939-134X.doi:https://doi.org/10.1037/a0017653. → page 83[235] L. D. Turner, S. M. Allen, and R. M. Whitaker. Interruptibility predictionfor ubiquitous systems: Conventions and new directions from a growingfield. In Proceedings of the 2015 ACM International Joint Conference onPervasive and Ubiquitous Computing, UbiComp ’15, page 801–812, NewYork, NY, USA, 2015. Association for Computing Machinery. ISBN9781450335744. doi:10.1145/2750858.2807514. URLhttps://doi.org/10.1145/2750858.2807514. → pages 1, 2, 44, 79, 85, 126[236] L. D. Turner, S. M. Allen, and R. M. Whitaker. Interruptibility predictionfor ubiquitous systems: Conventions and new directions from a growingfield. In Proceedings of the 2015 ACM International Joint Conference onPervasive and Ubiquitous Computing, UbiComp ’15, page 801–812, NewYork, NY, USA, 2015. Association for Computing Machinery. ISBN9781450335744. doi:10.1145/2750858.2807514. URLhttps://doi.org/10.1145/2750858.2807514. → pages 4, 40, 41, 44, 45, 117[237] L. D. Turner, S. M. Allen, and R. M. Whitaker. Push or delay?decomposing smartphone notification response behaviour. In A. A. Salah,163B. J. Kro¨se, and D. J. Cook, editors, Human Behavior Understanding,pages 69–83, Cham, 2015. Springer International Publishing. ISBN978-3-319-24195-1. → page 44[238] L. D. Turner, S. M. Allen, and R. M. Whitaker. Reachable but notreceptive: Enhancing smartphone interruptibility prediction by modellingthe extent of user engagement with notifications. Pervasive and MobileComputing, 40:480–494, 2017. ISSN 1574-1192.doi:https://doi.org/10.1016/j.pmcj.2017.01.011. URLhttps://www.sciencedirect.com/science/article/pii/S1574119217300640.→ pages 4, 44, 45[239] L. D. Turner, S. M. Allen, and R. M. Whitaker. The influence of concurrentmobile notifications on individual responses. International Journal ofHuman-Computer Studies, 132:70–80, 2019. → page 44[240] E. T. K. van Dijk, J. H. Westerink, F. Beute, and W. A. IJsselsteijn.Personal informatics, self-insight, and behavior change: A critical reviewof current literature. Human–Computer Interaction, 32(5-6):268–296,2017. doi:10.1080/07370024.2016.1276456. URLhttps://doi.org/10.1080/07370024.2016.1276456. → pages46, 73, 74, 80, 82, 85, 86, 108, 109[241] P. Verma and S. Patil. Exploring Privacy Aspects of SmartphoneNotifications. Association for Computing Machinery, New York, NY, USA,2021. ISBN 9781450383288. URLhttps://doi.org/10.1145/3447526.3472065. → pages 44, 47[242] A. Visuri, N. van Berkel, T. Okoshi, J. Goncalves, and V. Kostakos.Understanding smartphone notifications’ user interactions and contentimportance. International Journal of Human-Computer Studies, 128:72–85, 2019. ISSN 1071-5819.doi:https://doi.org/10.1016/j.ijhcs.2019.03.001. URLhttps://www.sciencedirect.com/science/article/pii/S1071581919300205.→ pages 58, 124[243] F. Vitale, W. Odom, and J. McGrenere. Keeping and discarding personaldata: Exploring a design space. In Proceedings of the 2019 on DesigningInteractive Systems Conference, DIS ’19, page 1463–1477, New York, NY,USA, 2019. Association for Computing Machinery. ISBN 9781450358507.doi:10.1145/3322276.3322300. URLhttps://doi.org/10.1145/3322276.3322300. → pages 47, 76164[244] A. Voit, D. Weber, and N. Henze. Qualitative investigation of multi-devicenotifications. In Proceedings of the 2018 ACM International JointConference and 2018 International Symposium on Pervasive andUbiquitous Computing and Wearable Computers, UbiComp ’18, page1263–1270, New York, NY, USA, 2018. Association for ComputingMachinery. ISBN 9781450359665. doi:10.1145/3267305.3274117. URLhttps://doi.org/10.1145/3267305.3274117. → pages 1, 2, 3, 44, 45, 83[245] A. Voit, T. Kosch, H. Weingartner, and P. W. Woz´niak. The attentionkitchen: Comparing modalities for smart home notifications in a cookingscenario. In 20th International Conference on Mobile and UbiquitousMultimedia, MUM 2021, page 90–97, New York, NY, USA, 2021.Association for Computing Machinery. ISBN 9781450386432.doi:10.1145/3490632.3490660. URLhttps://doi.org/10.1145/3490632.3490660. → pages 43, 47[246] N. Wagner, K. Hassanein, and M. Head. Computer use by older adults: Amulti-disciplinary review. Computers in Human Behavior, 26(5):870–882,2010. doi:https://doi.org/10.1016/j.chb.2010.03.029. URLhttps://www.sciencedirect.com/science/article/pii/S0747563210000695.→ page 17[247] D. Warnock, M. McGee-Lennon, and S. Brewster. The role of modality innotification performance. In P. Campos, N. Graham, J. Jorge, N. Nunes,P. Palanque, and M. Winckler, editors, Human-Computer Interaction –INTERACT 2011, pages 572–588, Berlin, Heidelberg, 2011. SpringerBerlin Heidelberg. ISBN 978-3-642-23771-3. → page 47[248] D. Weber, A. Voit, P. Kratzer, and N. Henze. In-situ investigation ofnotifications in multi-device environments. In Proceedings of the 2016ACM International Joint Conference on Pervasive and UbiquitousComputing, UbiComp ’16, page 1259–1264, New York, NY, USA, 2016.Association for Computing Machinery. ISBN 9781450344616.doi:10.1145/2971648.2971732. URLhttps://doi.org/10.1145/2971648.2971732. → page 44[249] D. Weber, A. Voit, H. V. Le, and N. Henze. Notification dashboard:Enabling reflection on mobile notifications. In Proceedings of the 18thInternational Conference on Human-Computer Interaction with MobileDevices and Services Adjunct, MobileHCI ’16, pages 936–941, New York,NY, USA, 2016. ACM. ISBN 978-1-4503-4413-5.doi:10.1145/2957265.2962660. URL165http://doi.acm.org.ezproxy.library.ubc.ca/10.1145/2957265.2962660. →pages 6, 46, 50, 52, 58, 67, 73, 80, 82, 85, 89, 94[250] D. Weber, A. Voit, J. Auda, S. Schneegass, and N. Henze. Snooze!investigating the user-defined deferral of mobile notifications. InProceedings of the 20th International Conference on Human-ComputerInteraction with Mobile Devices and Services, MobileHCI ’18, New York,NY, USA, 2018. Association for Computing Machinery. ISBN9781450358989. doi:10.1145/3229434.3229436. URLhttps://doi.org/10.1145/3229434.3229436. → pages2, 40, 52, 53, 56, 79, 118[251] D. Weber, A. Voit, and N. Henze. Notification log: An open-sourceframework for notification research on mobile devices. In Proceedings ofthe 2018 ACM International Joint Conference and 2018 InternationalSymposium on Pervasive and Ubiquitous Computing and WearableComputers, UbiComp ’18, page 1271–1278, New York, NY, USA, 2018.Association for Computing Machinery. ISBN 9781450359665.doi:10.1145/3267305.3274118. URLhttps://doi.org/10.1145/3267305.3274118. → pages 2, 84, 89[252] D. Weber, A. Voit, and N. Henze. Clear all: A large-scale observationalstudy on mobile notification drawers. In Proceedings of Mensch UndComputer 2019, MuC’19, page 361–372, New York, NY, USA, 2019.Association for Computing Machinery. ISBN 9781450371988.doi:10.1145/3340764.3340765. URLhttps://doi.org/10.1145/3340764.3340765. → pages5, 45, 47, 80, 84, 109, 114[253] M. Wiberg and S. Whittaker. Managing availability: Supportinglightweight negotiations to handle interruptions. ACM Trans.Comput.-Hum. Interact., 12(4):356–387, dec 2005. ISSN 1073-0516.doi:10.1145/1121112.1121114. URLhttps://doi.org/10.1145/1121112.1121114. → page 45[254] J. C. Williams, J. L. Berdahl, and J. A. Vandello. Beyond work-life“integration”. Annual Review of Psychology, 67(1):515–539, 2016.doi:10.1146/annurev-psych-122414-033710. URLhttps://doi.org/10.1146/annurev-psych-122414-033710. PMID: 26442669.→ page 110166[255] J. Woodward, Y.-P. Chen, K. Jurczyk, K. M. Ross, L. Anthony, and J. Ruiz.A survey of notification designs in commercial mhealth apps. In ExtendedAbstracts of the 2021 CHI Conference on Human Factors in ComputingSystems, CHI EA ’21, New York, NY, USA, 2021. Association forComputing Machinery. ISBN 9781450380959.doi:10.1145/3411763.3451639. URLhttps://doi.org/10.1145/3411763.3451639. → page 40[256] A. Wu, J. M. DiMicco, and D. R. Millen. Detecting professional versuspersonal closeness using an enterprise social network site. In Proceedingsof the SIGCHI Conference on Human Factors in Computing Systems, CHI’10, page 1955–1964, New York, NY, USA, 2010. Association forComputing Machinery. ISBN 9781605589299.doi:10.1145/1753326.1753622. URLhttps://doi.org/10.1145/1753326.1753622. → page 125[257] L. Y. Xiao, L. L. Henderson, Y. Yang, and P. W. S. Newall. Gaming thesystem: suboptimal compliance with loot box probability disclosureregulations in china. Behavioural Public Policy, page 1–27, 2021.doi:10.1017/bpp.2021.23. → pages 125, 126[258] X. Yan, K. Madier, S. Y. Park, and M. Newman. Towards low-burdenin-situ self-reporting: A design space exploration. In CompanionPublication of the 2019 on Designing Interactive Systems Conference 2019Companion, DIS ’19 Companion, page 337–346, New York, NY, USA,2019. Association for Computing Machinery. ISBN 9781450362702.doi:10.1145/3301019.3323905. URLhttps://doi.org/10.1145/3301019.3323905. → pages 47, 48[259] N. Yang, G. van Hout, L. Feijs, W. Chen, and J. Hu. Enhancing intrinsicmotivation in physical activity through quantified-self data sharing. EAIEndorsed Transactions on Pervasive Health and Technology, 6(21):e4,2020. URL https://eudl.eu/doi/10.4108/eai.11-5-2020.16441. → page 108[260] Y. Yuan, A. Muralidharan, P. Nandy, M. Cheng, and P. Prabhakar. Offlinereinforcement learning for mobile notifications, 2022. URLhttps://arxiv.org/abs/2202.03867. → page 58[261] A. Zenner, M. Speicher, S. Klingner, D. Degraen, F. Daiber, and A. Kru¨ger.Immersive notification framework: Adaptive & plausible notifications invirtual reality. In Extended Abstracts of the 2018 CHI Conference onHuman Factors in Computing Systems, CHI EA ’18, page 1–6, New York,167NY, USA, 2018. Association for Computing Machinery. ISBN9781450356213. doi:10.1145/3170427.3188505. URLhttps://doi.org/10.1145/3170427.3188505. → pages 43, 44, 47[262] J. Zimmerman, J. Forlizzi, and S. Evenson. Research through design as amethod for interaction design research in hci. In Proceedings of theSIGCHI Conference on Human Factors in Computing Systems, CHI ’07,page 493–502, New York, NY, USA, 2007. Association for ComputingMachinery. ISBN 9781595935939. doi:10.1145/1240624.1240704. URLhttps://doi.org/10.1145/1240624.1240704. → page 42[263] Y. Zou, S. Danino, K. Sun, and F. Schaub. You ‘might’ be affected: Anempirical analysis of readability and usability issues in data breachnotifications. In Proceedings of the 2019 CHI Conference on HumanFactors in Computing Systems, CHI ’19, page 1–14, New York, NY, USA,2019. Association for Computing Machinery. ISBN 9781450359702.doi:10.1145/3290605.3300424. URLhttps://doi.org/10.1145/3290605.3300424. → pages 40, 43168Appendix ASupporting MaterialsA.1 Supplementary Survey Results from Chapter 4Chapter 4 briefly reports a summary of a supplementary survey study that was usedto help guide the design of our Reflective Spring Cleaning concept. While not afull research chapter and contribution, we more fully report on the details of thissupplementary survey here because it was was used at various points within thisdissertation to support study and prototype design. For example, some of the deci-sion support design elements explored in Chapter 3, such as examples of impactednotifications, were explored briefly within this survey. As we report in Chapter 4,the survey was distributed to 140 participants (mean age=24.9, SD=5.7; 73 stu-dents along with various forms of knowledge workers such as clerks, web devel-opers, analysts or consultants) through convenience sampling on online recruitingforums and the university paid studies board. Each participant was shown the 20suggestions for notification management listed in Table A.1 and asked “For the fol-lowing list, think about whether or not you would accept such a suggestion.” Userscould select “yes,” “no,” or “not sure” for each suggestion shown in Table A.1 ofsupplementary materials.To expand upon these previously reported details, we also broke the 140 partic-ipants reacting to these suggestions into four evenly sized between subjects groups.To explore some initial ideas for how we might improve suggestion based notifica-tion management tools before Study 2, each of these groups framed the suggestions169in a slightly different way. The first group, Control Condition, displayed the sug-gestions exactly as shown in Table A.1. The second group, 14 day trial, asked userswhether they would accept the suggestion for a 14 day trial period rather than per-manently applying them to their phones. Trial periods were described exactly asthey were in Chapter 3. The third group, 2 day trial, asked users whether theywould accept the suggestion for a shorter trial period. The last group, expandeddescription, added an example of a notification that would be impacted by thesuggestion after it was described. A key research question was the extent to whichframing the suggestion in these different ways would influence users to accept moresuggestions or feel more confident with their decision. To support data interpreta-tion, we also collected various demographic data, as well as some supplementaryquestions to help understand how framing these suggestions influenced users de-cisions for which suggestions to accept. We observed no statistically significantdifferences between conditions in terms overall number of suggestions accepted.However we did see some very tentative visual differences in the supporting ques-tions that convinced us they were worth exploring further in a qualitative study,which became the basis of Study 2. For example, participants in the extended de-scription group more often reported that they had enough information to make aninformed choice, and that the suggestions were less risky to accept in Figures A.5and A.6. The value of similar design elements providing more decision support innotification management then became a key result in both Chapters 3 and 4. De-scriptive graphs of the data collected are presented in Figures A.1, A.2, A.3, A.4,A.5, A.6, and A.7.170Figure A.1: Histogram showing the ages of participants in the supplementarysurvey.171Table A.1: The 20 Notification Management Suggestions given in the tar-geted survey with rates of acceptance.AcceptanceRateSuggestion Text83 Automatically turn notification delivery to silent during scheduledmeetings on your calendar.82 Highlight work emails that need you to respond to them within a daywith a special ringtone or vibration.79 Set working hours (e.g. 9am to 5pm), where less important non-worknotifications (e.g. social media activity or messages from non-closefriends) are silent.79 Ask once if you want to unsubscribe from any email advertisements.75 Highlight messages from important contacts (e.g. your parents part-ner, spouse, children or very close friends) with a special ringtone orvibration.71 Deliver system messages silently68 Deliver social media activity notifications (e.g. status updates, not chatmessages) that are not directly related to, or reference you silently.64 Highlight all work or school related email with a special ringtone orvibration.64 Silently deliver email advertisements.57 Set off-work hours (e.g. after 6pm, Monday to Friday) where worknotifications are silent.57 Deliver social media activity notifications (e.g. status updates but notchat messages) silently at all times.54 Provide a digest notification of important meetings from your calendarthat day in the morning.39 Deliver broadcast style work messages silently.39 Highlight direct messages from chat applications (not including groupchats) with a special sound or vibration.36 Deliver social media activity notifications (e.g. status updates, but notchat messages) in a batch every hour.32 Deliver notifications for group chat messages in a batch every hour.29 Deliver social media notifications (including chat messages) in a batchevery hour.29 Do not deliver social media notifications (including chat messages)during the day. Instead deliver a batch of everything you received dur-ing the day in the evening (e.g. at 6pm).29 Do not deliver notifications from video games during the day. Insteaddeliver a batch of the ones you received during the day in the evening.29 Deliver all notifications from video games silently at all times.172Figure A.2: Bar chart showing the average number of suggestions acceptedby group in the supplementary survey. The error bars show standarddeviations.173Figure A.3: Histograms showing the number of suggestions accepted by par-ticipants for each group in the supplementary survey.174Figure A.4: Histogram showing users’ desired trial period length in the sup-plementary survey.175Figure A.5: Histogram showing whether users felt they had enough informa-tion to know what the suggestion would do in the supplementary surveyby group.176Figure A.6: Histogram showing users’ perceived risk for accepting sugges-tions in the supplementary survey by group.177Figure A.7: Histogram showing how many users felt that a trial period wouldchange their mind in the non-trial period groups.178A.2 Categories of App from What Abstraction fromChapter 4Below we list for detailed descriptions for the categories of app present within the“What” data abstration in Study 3 Chapter 4.• System - Software updates and OS messages• Media - various forms of social media (e.g. twitter, linkedin)• Games - video games and other forms of entertainment• Email - Any email notifications• Text - Any direct messaging (e.g. discord, slack)• Voice - Direct phone calls and video conferencing• Calendar - Reminders and calendar events• News - News reading apps• Tools - Productivity tools (e.g. google drive or slides)• Market - Purchasing, sales or apps for specific stores• Other - Catchall other179Appendix BSupplementary survey for Study3 Questions180 Page 1 of 41 Notification Personalization Start of Block: General Feelings about Notifications Q84 This survey asks a number of questions related to notifications that you receive on a phone, tablet, or computer. These are typically delivered with a sound, vibration, or visual alert (depending on your settings). For example, notifications appear like this on an android phone: Here is an example notification on a windows 10 device: Now lets get started! Q1.1 How old are you? ________________________________________________________________ Q1.2 What languages do you speak? o Language 1 (1) ________________________________________________ o Language 2 (2) ________________________________________________ o Language 3 (3) ________________________________________________ o Language 4 (4) ________________________________________________ Q1.3 What is your primary occupation? (e.g., cashier, student, stay-at-home parent) ________________________________________________________________ ________________________________________________________________ 181 Page 2 of 41 ________________________________________________________________ ________________________________________________________________ ________________________________________________________________ Q1.4 Do you keep consistent work hours? (e.g. 9am to 5pm or night shift?) o yes (1) o no (2) o Most of the time but not always (3) Q1.5 On which of these devices do you receive notifications regularly? ▢ Android Phone (1) ▢ iPhone (2) ▢ Other Phone (3) ▢ Tablet (4) ▢ Laptop (5) ▢ Desktop (6) ▢ Other (7) ________________________________________________ 182 Page 3 of 41 Q1.6 What of the following apps do you tend to use frequently? ▢ Social Media (e.g. Facebook, Instagram) (1) ▢ Email (2) ▢ Texting (SMS, iMessage) (3) ▢ Instant Messaging (e.g. Facebook messenger, Whatsapp) (4) ▢ Video games (5) ▢ Video Chat (e.g. Skype, Google Hangouts) (6) ▢ Camera (7) ▢ Calendar (9) ▢ Other, please specify (8) ________________________________________________ 183 Page 4 of 41 Q82 Are there specific times during the week where you tend to get more notifications? Check all that apply ▢ During work hours (e.g. 9am-5pm) (1) ▢ In the morning (2) ▢ In the evening (3) ▢ On the weekends (4) ▢ No, It's pretty consistent (6) ▢ I don't notice a difference (7) ▢ Other: Please Specify (5) ________________________________________________ Q83 What would be a rough estimate for how many notifications you get in an hour where you are getting a lot of notifications? ________________________________________________________________ Q1.8 How useful or helpful do you find notifications? o They are almost always useful (1) o Most are quite useful, but a noticeable proportion of them aren't (2) o Only a few are useful, most aren't (3) o They aren't generally useful (4) 184 Page 5 of 41 Q85 How distracting do you find notifications? o They are a constant source of distraction (1) o They are frequently distracting, but usually manageable (2) o Occasionally they are distracting or frustrating, but not often (3) o I very rarely find them distracting (4) Q86 What volume level do you typically use for notifications? o Sound/ringtone (1) o Vibration only (2) o Silent (3) 185 Page 6 of 41 Q1.9 Which of the following ways have you chosen to personalize your phone's notification settings in the past? ▢ Turned off notifications for certain apps (1) ▢ Given specific senders or apps different sounds vibrations or alerts (2) ▢ Adjusted in-app notification settings (e.g. turn off notifications for certain facebook contacts) (3) ▢ Set up batching systems for notifications (e.g. deliver all notifications received in a batch every hour) (4) ▢ Installed a notification manager app (5) ▢ Set up email filtering rules (6) ▢ Set up priority display for certain notifications (7) ▢ Set quiet hours where notifications are silent (e.g. in the evening or at work) (9) ▢ Other (8) ________________________________________________ Q1.11 What do you value most about notifications? ________________________________________________________________ 186 Page 7 of 41 Q1.12 Which of the following issues do you frequently experience with notifications? ▢ They distract me from what I'm doing (1) ▢ Their content just isn't relevant to my interests or needs (2) ▢ They don't come at the right time to actually help me (3) ▢ They interrupt Face-to-Face time (4) ▢ I have a hard time ignoring them when I need to focus (5) Q1.14 Do you find multi-tasking difficult? o Strongly agree (1) o Agree (2) o Neutral (3) o Disagree (4) o Strongly Disagree (5) Q1.15 Do you frequently worry about not responding to other's messages quickly enough? o Strongly agree (1) o Agree (2) o Neutral (3) o Disagree (4) o Strongly Disagree (5) 187 Page 8 of 41 End of Block: General Feelings about Notifications Start of Block: Block 3 Description Q18 For the next set of questions, imagine you are personalizing a phone’s notification settings. Currently you would use a settings panel such as the one from Facebook shown below to change how notifications are delivered: Consider a possible future system where the phone might automatically suggest certain settings from such a menu for you to use. For the following questions think about whether or not you would accept such a suggestion if your phone were to provide it to you. These settings may not all be possible with current technology, but the key is whether or not you think the setting would be useful if it could be implemented and was suggested to you. End of Block: Block 3 Description Start of Block: Acceptance - Group A Q2.1 Select which of the following notification personalizations you think you would accept if they were suggested to you? 188 Page 9 of 41 Note that if you accepted this suggestion, if you did not like the result you would have to turn it off in the notifications settings panel yourself. 189 Page 10 of 41 Yes (1) No (2) Not Sure (5) I do not get this type of notification (4) Set off-work hours (e.g. after 6pm, Monday to Friday) where work notifications are silent. (1) o o o o Set working hours (e.g. 9am to 5pm), where less important non-work notifications (e.g. social media activity or messages from non-close friends) are silent. (2) o o o o Deliver social media activity notifications (e.g. status updates, but not chat messages) in a batch every hour. (3) o o o o Deliver social media notifications (including chat messages) in a batch every hour. (4) o o o o Deliver social media activity notifications (e.g. status updates but not chat messages) silently at all times. (5) o o o o 190 Page 11 of 41 Do not deliver social media notifications (including chat messages) during the day. Instead deliver a batch of everything you received during the day in the evening (e.g. at 6pm). (6) o o o o Highlight messages from important contacts (e.g. your parents partner, spouse, children or very close friends) with a special ringtone or vibration. (7) o o o o Highlight work emails that need you to respond to them within a day with a special ringtone or vibration. (8) o o o o Highlight all work or school related email with a special ringtone or vibration. (9) o o o o Deliver broadcast style work messages silently. (10) o o o o Automatically turn notification delivery to silent during scheduled meetings on your calendar. (11) o o o o 191 Page 12 of 41 Silently deliver email advertisements. (12) o o o o Ask once if you want to unsubscribe from any email advertisements. (21) o o o o Do not deliver notifications from video games during the day. Instead deliver a batch of the ones you received during the day in the evening. (13) o o o o Deliver all notifications from video games silently at all times. (14) o o o o Provide a digest notification of important meetings from your calendar that day in the morning. (15) o o o o Deliver system messages silently (16) o o o o Highlight direct messages from chat applications (not including group chats) with a special sound or vibration. (17) o o o o 192 Page 13 of 41 Deliver notifications for group chat messages in a batch every hour. (18) o o o o Deliver social media activity notifications (e.g. status updates, not chat messages) that are not directly related to, or reference you silently. (19) o o o o Q78 For the following suggestion: Do not deliver notifications for social media activity (not including chat messages) 193 Page 14 of 41 Rate to what degree you agree with the following statements Strongly Disagree (1) Disagree (2) Neutral (6) Agree (3) Strongly Agree (4) I would accept this suggestion (1) o o o o o I think I would like the result of this setting if I accepted it (2) o o o o o I think the suggestion would be risky to accept (3) o o o o o I am confident enough information has been provided to me to understand what the suggestion would do (4) o o o o o I do not get this type of notification (5) o o o o o Q87 Do you think you would have accepted different or more suggestions if you were given a trial period? Note that if you accepted a suggestion for a trial period, the setting would be turned on, but after 194 Page 15 of 41 a few days the system will ask again if you like the result of the setting and want to leave it on. If you say no the system will turn it back to the way it was. o Most of my responses would change (1) o Many of my responses would change (2) o A few of my responses would change (3) o None of my responses would change (4) Q88 If you were given a trial period for these suggestions, how long would you want it to be? o 2 days (1) o 4 days (2) o 7 days (3) o 10 days (4) o 14 days (5) o Other (6) ________________________________________________ End of Block: Acceptance - Group A Start of Block: Acceptance - Group B Q121 Select which of the following notification personalizations you think you would accept if they were suggested to you? Note that if you accept the suggestion it would be applied with a two day trial. The setting would 195 Page 16 of 41 be turned on, but after two days the system will ask again if you like the result of the setting and want to leave it on, or want to go back to the way it was. 196 Page 17 of 41 Yes (1) No (2) Not Sure (5) I do not get this type of notification (4) Set off-work hours (e.g. after 6pm, Monday to Friday) where work notifications are silent. (Two day trial) (1) o o o o Set working hours (e.g. 9am to 5pm), where less important non-work notifications (e.g. social media activity or messages from non-close friends) are silent. (Two day trial) (2) o o o o Deliver social media activity notifications (e.g. status updates, but not chat messages) in a batch every hour. (Two day trial) (3) o o o o Deliver social media notifications (including chat messages) in a batch every hour. (Two day trial) (4) o o o o 197 Page 18 of 41 Deliver social media activity notifications (e.g. status updates but not chat messages) silently at all times. (Two day trial) (5) o o o o Do not deliver social media notifications (including chat messages) during the day. Instead deliver a batch of everything you received during the day in the evening (e.g. at 6pm). (Two day trial) (6) o o o o Highlight messages from important contacts (e.g. your parents partner, spouse, children or very close friends) with a special ringtone or vibration. (Two day trial) (7) o o o o Highlight work emails that need you to respond to them within a day with a special ringtone or vibration. (Two day trial) (8) o o o o 198 Page 19 of 41 Highlight all work or school related email with a special ringtone or vibration. (Two day trial) (9) o o o o Deliver broadcast style work messages silently. (Two day trial) (10) o o o o Automatically turn notification delivery to silent during scheduled meetings on your calendar. (Two day trial) (11) o o o o Silently deliver email advertisements. (Two day trial) (12) o o o o Ask once if you want to unsubscribe from any email advertisements. (Two day trial) (21) o o o o Do not deliver notifications from video games during the day. Instead deliver a batch of the ones you received during the day in the evening. (Two day trial) (13) o o o o 199 Page 20 of 41 Deliver all notifications from video games silently at all times. (Two day trial) (14) o o o o Provide a digest notification of important meetings from your calendar that day in the morning. (Two day trial) (15) o o o o Deliver system messages silently. (Two day trial) (16) o o o o Highlight direct messages from chat applications (not including group chats) with a special sound or vibration. (Two day trial) (17) o o o o Deliver notifications for group chat messages in a batch every hour. (Two day trial) (18) o o o o Deliver social media activity notifications (e.g. status updates, not chat messages) that are not directly related to, or reference you silently. (Two day trial) (19) o o o o 200 Page 21 of 41 Q79 For the following suggestion: Do not deliver notifications from video games (two day trial) Rate to what degree you agree with the following statements Strongly Disagree (1) Disagree (2) Neutral (5) Agree (3) Strongly Agree (4) I would accept this suggestion (1) o o o o o I think I would like the result of this setting if I accepted it (2) o o o o o I think the suggestion would be risky to accept (3) o o o o o I am confident enough information has been provided to me to understand what the suggestion would do (4) o o o o o I do not get this type of notification (5) o o o o o 201 Page 22 of 41 Q92 If you were given a trial period for these suggestions, how long would you want it to be? o 2 days (1) o 4 days (2) o 7 days (3) o 10 days (4) o 14 days (5) o Other (6) ________________________________________________ End of Block: Acceptance - Group B Start of Block: Acceptance - Group C Q76 Select which of the following notification personalizations you think you would accept if they were suggested to you? Note that if you accept the suggestion it would be applied with a 14 day trial. The setting would 202 Page 23 of 41 be turned on, but after 14 days the system will ask again if you like the result of the setting and want to leave it on, or want to go back to the way it was. 203 Page 24 of 41 Yes (1) No (2) Not Sure (5) I do not get this type of notification (4) Set off-work hours (e.g. after 6pm, Monday to Friday) where work notifications are silent. (14 day trial) (1) o o o o Set working hours (e.g. 9am to 5pm), where less important non-work notifications (e.g. social media activity or messages from non-close friends) are silent. (14 day trial) (2) o o o o Deliver social media activity notifications (e.g. status updates, but not chat messages) in a batch every hour. (14 day trial) (3) o o o o Deliver social media notifications (including chat messages) in a batch every hour. (14 day trial) (4) o o o o 204 Page 25 of 41 Deliver social media activity notifications (e.g. status updates but not chat messages) silently at all times. (14 day trial) (5) o o o o Do not deliver social media notifications (including chat messages) during the day. Instead deliver a batch of everything you received during the day in the evening (e.g. at 6pm). (14 day trial) (6) o o o o Highlight messages from important contacts (e.g. your parents partner, spouse, children or very close friends) with a special ringtone or vibration. (14 day trial) (7) o o o o Highlight work emails that need you to respond to them within a day with a special ringtone or vibration. (14 day trial) (8) o o o o 205 Page 26 of 41 Highlight all work or school related email with a special ringtone or vibration. (14 day trial) (9) o o o o Deliver broadcast style work messages silently. (14 day trial) (10) o o o o Automatically turn notification delivery to silent during scheduled meetings on your calendar. (14 day trial) (11) o o o o Silently deliver email advertisements. (14 day trial) (12) o o o o Ask once if you want to unsubscribe from any email advertisements. (14 day trial) (21) o o o o Do not deliver notifications from video games during the day. Instead deliver a batch of the ones you received during the day in the evening. (14 day trial) (13) o o o o Deliver all notifications from video games silently at all times. (14 day trial) (14) o o o o 206 Page 27 of 41 Provide a digest notification of important meetings from your calendar that day in the morning. (14 day trial) (15) o o o o Deliver system messages silently. (14 day trial) (16) o o o o Highlight direct messages from chat applications (not including group chats) with a special sound or vibration. (14 day trial) (17) o o o o Deliver notifications for group chat messages in a batch every hour. (14 day trial) (18) o o o o Deliver social media activity notifications (e.g. status updates, not chat messages) that are not directly related to, or reference you silently. (14 day trial) (19) o o o o Q80 For the following suggestion: 207 Page 28 of 41 Do not deliver social media notifications if they arrive during work hours (e.g. 9am to 5pm) (14 day trial) Rate to what degree you agree with the following statements Strongly Disagree (1) Disagree (2) Neutral (5) Agree (3) Strongly Agree (4) I would accept this suggestion (1) o o o o o I think I would like the result of this setting if I accepted it (2) o o o o o I think the suggestion would be risky to accept (3) o o o o o I am confident enough information has been provided to me to understand what the suggestion would do (4) o o o o o I do not get this type of notification (5) o o o o o 208 Page 29 of 41 Q90 If you were given a trial period for these suggestions, how long would you want it to be? o 2 days (1) o 4 days (2) o 7 days (3) o 10 days (4) o 14 days (5) o Other (6) ________________________________________________ End of Block: Acceptance - Group C Start of Block: Acceptance - Group D Q77 Select which of the following notification personalizations you think you would accept if they were suggested to you? 209 Page 30 of 41 Note that if you accepted this suggestion, if you did not like the result you would have to turn it off in the notifications settings panel yourself. 210 Page 31 of 41 Yes (1) No (2) Not Sure (5) I do not get this type of notification (4) Set off-work hours (e.g. after 6pm, Monday to Friday) where work notifications are silent. (e.g. a text from your boss telling you the schedule for next week has been changed) (1) o o o o Set working hours (e.g. 9am to 5pm), where less important non-work notifications (e.g. social media activity or messages from non-close friends) are silent. (e.g. A Facebook message from a friend you haven't talked to in a few months asking to hang out sometime) (2) o o o o 211 Page 32 of 41 Deliver social media activity notifications (e.g. status updates, but not chat messages) in a batch every hour. (e.g. A Facebook post from a close friend about a major life event, or a Facebook post from a popular band you like that is selling tickets for a show in your area) (3) o o o o Deliver social media notifications (including chat messages) in a batch every hour. (e.g. A Facebook post from a close friend about a major life event or a Facebook message from a close friend asking to hang out next week) (4) o o o o 212 Page 33 of 41 Deliver social media activity notifications (e.g. status updates but not chat messages) silently at all times. (e.g. A Facebook post from a close friend about a major life event or a Facebook post from a popular band you like that is selling tickets for a show in your area) (5) o o o o Do not deliver social media notifications (including chat messages) during the day. Instead deliver a batch of everything you received during the day in the evening (e.g. at 6pm). (e.g. A Facebook post from a close friend about a major life event or a Facebook message from a close friend asking to hang out next week) (6) o o o o 213 Page 34 of 41 Highlight messages from important contacts (e.g. your parents partner, spouse, children or very close friends) with a special ringtone or vibration. (e.g. A text from your parents, partner or spouse asking about your day, or asking you to pick up groceries on the way home) (7) o o o o Highlight work emails that need you to respond to them within a day with a special ringtone or vibration. (e.g. A text from your boss asking you to come early today, or a text from your boss telling you to update your time-sheet by the end of the day) (8) o o o o 214 Page 35 of 41 Highlight all work or school related email with a special ringtone or vibration. (e.g. an email from a business asking if you were interested in having an interview for a new job, or an email from your boss asking for a status update on a project you're working on) (9) o o o o Deliver broadcast style work messages silently. (e.g. An email to the entire department at work about ongoing network maintenance, or an email to the entire company about changes to the businesses overtime policy) (10) o o o o Automatically turn notification delivery to silent during scheduled meetings on your calendar. (11) o o o o 215 Page 36 of 41 Silently deliver email advertisements. (e.g. A Facebook post about a new sale going on at a store you frequent, or a Facebook post about an online sale starting next week) (12) o o o o Ask once if you want to unsubscribe from any email advertisements. (e.g. A Facebook post about a new sale going on at a store you frequent, or a Facebook post about an online sale starting next week) (21) o o o o 216 Page 37 of 41 Do not deliver notifications from video games during the day. Instead deliver a batch of the ones you received during the day in the evening. (e.g. A notification from a game you play that your energy/lives have been replenished and you can play more, or a notification that a game you play has been updated with new content) (13) o o o o Deliver all notifications from video games silently at all times. (e.g. A notification from a game you play that your energy/lives have been replenished and you can play more, or a notification that a game you play has been updated with new content) (14) o o o o 217 Page 38 of 41 Provide a digest notification of important meetings from your calendar that day in the morning. (e.g. A reminder that you have a party/event being held tonight and a doctors appointment in the afternoon) (15) o o o o Deliver system messages silently. (e.g. A system message that a new wi-fi hotspot is available, or a system message telling you that you are approaching your data cap for the month) (16) o o o o Highlight direct messages from chat applications (not including group chats) with a special sound or vibration. (e.g. A text from your boss telling you the schedule from next week has been changed, or a text from an acquaintance you met at a party recently saying it was good to meet you) (17) o o o o 218 Page 39 of 41 Deliver notifications for group chat messages in a batch every hour. (e.g. A Facebook group chat message about issues people are having with a school project due in a few days, or a facebook group chat message from your friends asking you to organize a camping trip) (18) o o o o Deliver social media activity notifications (e.g. status updates, not chat messages) that are not directly related to, or reference you silently. (e.g. A Facebook post from a club you are in saying that they are holding a movie night on friday) (19) o o o o Q81 For the following suggestion: Do not deliver notifications for email advertisements (e.g. an email about a sale going on at a local grocery store, or a Facebook post about a new product) 219 Page 40 of 41 Rate to what degree you agree with the following statements Strongly Disagree (1) Disagree (2) Neutral (5) Agree (3) Strongly Agree (4) I would accept this suggestion (1) o o o o o I think I would like the result of this setting if I accepted it (2) o o o o o I think the suggestion would be risky to accept (3) o o o o o I am confident enough information has been provided to me to understand what the suggestion would do (4) o o o o o I do not get this type of notification (5) o o o o o Q89 Do you think you would have accepted different or more suggestions if you were given a trial period? Note that if you accepted a suggestion for a trial period, the setting would be turned on, but after 220 Page 41 of 41 a few days the system will ask again if you like the result of the setting and want to leave it on. If you say no the system will turn it back to the way it was. o Most of my responses would change (1) o Many of my responses would change (2) o A few of my responses would change (3) o None of my responses would change (4) Q91 If you were given a trial period for these suggestions, how long would you want it to be? o 2 days (1) o 4 days (2) o 7 days (3) o 10 days (4) o 14 days (5) o Other (6) ________________________________________________ End of Block: Acceptance - Group D Start of Block: Block 7 Q95 Thank you for your participation in our survey! If you would like to be considered for our raffle reimbursement (a 1 in 10 chance for a $25 gift card) please enter your email below. ________________________________________________________________ End of Block: Block 7 221Appendix CAdditional DocumentsC.1 Study 1 Documents222Page | 1 Advertisement Version 1.1 – February 2 2017 Subject: Highly Personalized User Interfaces Study We are conducting a study as part a larger investigation of personalization behavior and individual differences in how users interact with notifications. As part of this effort we are investigating how different groups of people think about, organize and classify the notifications and messages they receive. This includes but is not limited to mobile devices. Activities involved will be mainly limited to talking about your experiences with notifications as well as sorting and grouping cards showing potential notifications. You will be asked to share a few (5-10) of your own notifications in as much detail as you desire if you are comfortable with it. The study will take approximately one hour to one hour and fifteen minutes of your time. You will be compensated $15 for participating. The study is available to anyone who meets the following: • Is 18 years of age or older• Owns and has used a device (e.g. phone, laptop, tablet) that receives notifications formore than six months• Speaks English fluentlyWhen Participation times are negotiable, and will take place at the UBC Computer Science building. If you are interested, please contact Izabelle Janzen. The principal investigator for this study is Dr. Joanna McGrenere, Professor, Computer Science, UBC. Figure C.1: Advertisement for Study 1.223Page | 1 Advertisement_olderadults Version 1.0 – July 10 2017 Subject: Highly Personalized User Interfaces and Older Adults Study We are conducting a study as part a larger investigation of personalization behavior and individual differences in how older adults interact with notifications. As part of this effort we are investigating how older adults think about, organize and classify the notifications and messages they receive. This includes but is not limited to mobile devices. Activities involved will be mainly limited to talking about your experiences with notifications as well as sorting and grouping cards showing potential notifications. You will be asked to share a few (5-10) of your own notifications in as much detail as you desire if you are comfortable with it. The study will take approximately one hour to one hour and fifteen minutes of your time. You will be compensated $15 for participating. The study is available to anyone who meets the following: • Is 50 years of age or older• Owns and has used a device (e.g. phone, laptop, tablet) that receives notifications formore than six months• Speaks English fluentlyWhen Participation times are negotiable, and will take place at the UBC Computer Science building. If you are interested, please contact Izabelle Janzen. The principal investigator for this study is Dr. Joanna McGrenere, Professor, Computer Science, UBC. Figure C.2: Advertisement for Study 1, specified to older adults.224Consent Form – Version 1.1 – 2017-January 30 page 1/2 UBC Department of Computer Science ICICS/CS Building 201-2366 Main MallVancouver, B.C., V6T 1Z4Highly Personalized User Interfaces Consent Form Principal Investigator Joanna McGrenere, Professor, Department of Computer ScienceCo-Investigator Izabelle F. Janzen, M.Sc. Student, Department of Computer ScienceProject Purpose and Procedures The purpose of this study is to investigate how users perceive, communicate about and organize the notifications they receive. This includes but is not limited to mobile devices. During the study, we will ask you questions about your experiences with notifications and perform some simple activities with example notifications (e.g. sorting them into groups). You will also be asked to share some of your own notifications in as much detail as you wish if you are comfortable with it. Confidentiality Your identity will be kept confidential. We will record the groupings you make as well as some demographic data, but no identifying information (such as your name) will be stored with this data, nor will it be associated with the data after it has been analyzed. We will also record audio of your participation, and may take notes on comments you make as you are sorting the cards, but this data will also be stored anonymously. The results will be made public through scholarly publications, presentations and academic theses; however, no identifying information will be included in any of these. Risks/Remuneration/Compensation We do not anticipate any significant risks for taking part in this study. You are free to withdraw at any point in your participation, even after it has been completed. You will receive an honorarium of $15 for your participation. You will be eligible for the honorarium even if you withdraw from the study. Figure C.3: Consent form for Study 1, page 1.225Consent Form – Version 1.1 – 2017-January 30 page 2/2 Contact Information about the Project If you have any questions or require further information about the project you may contact Izabelle Janzen or Dr. Joanna McGrenere. Contact for Concerns About the Rights of Research Subjects If you have any concerns or complaints about your rights as a research participant and/or your experiences while participating in this study, contact the Research Participant Complaint Line in the UBC Office of Research Ethics. Consent We intend for your participation in this project to be pleasant and stress-free. Taking part in this study is entirely up to you. You have the right to refuse to participate in this study. If you decide to take part, you may choose to pull out of the study at any time without giving a reason and without any negative consequence. Your signature below indicates that you have received a copy of this consent form for your own records. Your signature indicates that you consent to participate in this study. I, (print name) ___________________________ agree to participate in the study as outlined above. My participation in this study is voluntary and I understand that I may withdraw at any time. ____________________________________________________________________ Participant Signature Date _____________________________________ Printed Name of the Participant signing above Figure C.4: Consent form for Study 1, page 2.226C.2 Study 2 DocumentsPage | 1 Advertisement Version 1.2 – August 14 2019 Subject: Highly Personalized User Interfaces Study We are conducting a study as part a larger investigation of personalization behavior and individual differences in how users interact with notifications. As part of this effort we are investigating how users can most effectively communicate their preferences to a notification management system.. Activities involved will be mainly limited to talking about your experiences with notifications as well creating new notification settings with a series of prototype notification management tools. The study will take approximately one hour to one hour and thirty minutes of your time. You will be compensated $15 for participating. The study is available to anyone who meets the following: • Owns and has used a device (e.g. phone, laptop, tablet) that receives notifications formore than six months• Speaks English fluentlyAnd meets one of the following: • Is between 18 and 35 years of age• Is between 40 and 55 years of age• Is 60 years of age or olderWhen Participation times are negotiable, and will take place at the UBC Computer Science building. If you are interested, please contact Izabelle Janzen. The principal investigator for this study is Dr. Joanna McGrenere, Professor, Computer Science, UBC. Figure C.5: Advertisement for Study 2.227Consent Form – Version 1.2 – 2019-August 14 Ethics ID: H16-02949 page 1/2 UBC Department of Computer Science ICICS/CS Building 201-2366 Main MallVancouver, B.C., V6T 1Z4Highly Personalized User Interfaces Consent Form Principal Investigator Joanna McGrenere, Professor, Department of Computer Science Co-Investigator Izabelle F. Janzen, M.Sc. Student, Department of Computer Science Project Purpose and Procedures The purpose of this study is to investigate how users perceive, communicate about and organize the notifications they receive as well as how preferences about those notification can be effectively expressed to a notification management system. This includes but is not limited to mobile devices. During the study, we will ask you questions about your experiences with notifications and create some notification settings with prototype notification management tools. Confidentiality Your identity will be kept confidential. We will record the groupings you make as well as some demographic data, but no identifying information (such as your name) will be stored with this data, nor will it be associated with the data after it has been analyzed. We will also record audio of your participation, and may take notes on comments you make as you are sorting the cards, but this data will also be stored anonymously. The results will be made public through scholarly publications, presentations and academic theses; however, no identifying information will be included in any of these. Risks/Remuneration/Compensation We do not anticipate any significant risks for taking part in this study. You are free to withdraw at any point in your participation, even after it has been completed. You will receive an honorarium of $15 for your participation. You will be eligible for the honorarium even if you withdraw from the study. Figure C.6: Consent form for Study 2, page 1.228Consent Form – Version 1.2 – 2019-August 14 Ethics ID: H16-02949 page 2/2 Contact Information about the Project If you have any questions or require further information about the project you may contact Izabelle Janzen, or Dr. Joanna McGrenere. Contact for Concerns About the Rights of Research Subjects If you have any concerns or complaints about your rights as a research participant and/or your experiences while participating in this study, contact the Research Participant Complaint Line in the UBC Office of Research Ethics. Consent We intend for your participation in this project to be pleasant and stress-free. Taking part in this study is entirely up to you. You have the right to refuse to participate in this study. If you decide to take part, you may choose to pull out of the study at any time without giving a reason and without any negative consequence. Your signature below indicates that you have received a copy of this consent form for your own records. Your signature indicates that you consent to participate in this study. I, (print name) ___________________________ agree to participate in the study as outlined above. My participation in this study is voluntary and I understand that I may withdraw at any time. ____________________________________________________________________ Participant Signature Date _____________________________________ Printed Name of the Participant signing above Figure C.7: Consent form for Study 2, page 2.229C.3 Study 3 DocumentsPage | 1 Advertisement_phase4 Version 1.1 – March 23 2021 Subject: Highly Personalized User Interfaces Study We are conducting a study evaluating a novel notification management technique as part a larger investigation of notification personalization behavior. The study will involve 1. Passive, partially anonymous logging of your notification usage data. 2. An interactive zoom session where you view visualizations of this usage data and receive suggestions for personalization and 3. A debriefing zoom interview after you experience using these notification personalizations for several weeks. The study will take approximately 4 weeks. However, you will only be required to actively participate in two sessions. A 1-hour interactive session approximately two weeks into the study, and a shorter 45-minute debriefing interview at the end of the study. You will be compensated $50 for participating. You can choose to withdraw from the study at any point for any reason and will receive a pro-rated amount. The study is available to anyone who meets the following: • Owns and has used an Android smartphone from a brand other than Huawei or Xiaomifor more than six months.• Feels they receive a lot of notifications and might be interested in better managingthem.• Has an email account or messaging app primarily devoted to work, or can identify a fewkey work contacts (e.g. boss, direct coworkers)• Speaks English• 18+ years oldWhen Participation times are negotiable, and will take place at the UBC Computer Science building. If you are interested, please contact Izabelle Janzen. The principal investigator for this study is Dr. Joanna McGrenere, Professor, Computer Science, UBC. Figure C.8: Advertisement for Study 3.230Page | 1 Survey Advertisement Version 1.0 – June 13 2018 Subject: Highly Personalized User Interfaces Study We are conducting a study as part a larger investigation of personalization behavior and individual differences in how users interact with notifications. As part of this effort we are conducting a brief survey on notification usage and personalization. The survey will take approximately 10-15 minutes of your time. As a reward for participation you will be entered into a raffle with a 1 in 10 chance to win a $25 gift card. The study is available to anyone who meets the following: • Is 18 years of age or older• Owns and has used a device (e.g. phone, laptop, tablet) that receives notifications formore than six months• Speaks English fluentlyIf you have any questions about the study, please contact Izabelle Janzen. The principal investigator for this study is Dr. Joanna McGrenere, Professor, Computer Science, UBC. Figure C.9: Advertisement for the supplementary survey in Study 3.231Consent Form_phase4 – Version 1.2 – 2021-March 31 Ethics ID: H16-02949 page 1/3 UBC Department of Computer Science ICICS/CS Building 201-2366 Main MallVancouver, B.C., V6T 1Z4Study Title: Highly Personalized User Interfaces Consent Form Principal Investigator Joanna McGrenere, Professor, Department of Computer Science Co-Investigator Izabelle F. Janzen, M.Sc. Student, Department of Computer Science Project Purpose and Procedures The purpose of this study is to explore novel approaches to how users can manage and personalization their notification systems (e.g on mobile devices). This includes reflecting on visualizations of notification usage and receiving suggestions for personalization. The study will have 4 parts. 1. A period of two weeks of passive logging of your notification usage. 2. A personalization session and zoom interview where you explore visualizations of your notification usage data and receive suggestions for personalization. 3. Two further weeks of passive logging where you experience a notification personalization. 4. A debriefing zoom interview on the experience of using the applied personalization. You should login to the zoom interview using a nickname. There will be a brief, optional 2 question email survey to follow-up two weeks after the study. Confidentiality Your identity will be kept confidential. We will record: 1. Partially anonymized notification usage data. 2. Demographic Information and 3. Video recordings of the interviews for transcription and analysis. No identifying information (such as your name) will be associated with the data during or after data analysis and any video recordings will be kept strictly confidential. The partially anonymized notification usage data will contain only information directly relevant to the visualizations (e.g. notification senders, apps, time received) and you can choose to delete any entries you don’t wish to share manually for any reason. The results will be made public through scholarly publications, presentations and academic theses; however, no identifying information will be included in any of these. Risks/Remuneration/Compensation We do not anticipate any significant risks for taking part in this study. You are free to withdraw at any point in your participation, even after it has been completed. Figure C.10: Consent form for Study 3, page 1.232Consent Form_phase4 – Version 1.2 – 2021-March 31 Ethics ID: H16-02949 page 2/3 You will receive an honorarium of $50 in either cash or as a gift card for your participation. You will be eligible for a prorated honorarium if you withdraw from participation before the study is completed. Figure C.11: Consent form for Study 3, page 2.233Welcome to the research study! We are interested in understanding notification usage a personalization behavior. You will be presented with information relevant to usage and personalization of notifications and asked to answer some questions about it. Please be assured that your responses will be kept completely confidential. The study should take you around 10-15 minutes to complete, and you will receive a one in ten chance at a $25 gift cardfor your participation. Your participation in this research is voluntary. You have the right to withdraw at any point during the study, for any reason, and without any prejudice. If you would like to contact the Principal Investigator in the study to discuss this research, please e-mail Izabelle Janzen. By clicking the button below, you acknowledge that your participation in the study is voluntary, you are 18 years of age, and that you are aware that you may choose to terminate your participation in the study at any time and for any reason. Please note that this survey will be best displayed on a laptop or desktop computer. Some features may be less compatible for use on a mobile device. Figure C.12: Consent form for the supplementary survey in Study 3.234
Thesis/Dissertation
eng
Computer Science
Vancouver : University of British Columbia Library
Attribution-NonCommercial-ShareAlike 4.0 International
Graduate
McGrenere, Joanna
Design opportunities for personalized proactive notification management
Text