Science, Irving K. Barber Faculty of (Okanagan)
Computer Science, Mathematics, Physics and Statistics, Department of (Okanagan)
DSpace
UBCO
Khatra, Karanmeet Singh
Master of Science - MSc
University of British Columbia
There is a growing concern that smartphone usage in front of family or friends can be bothersome and even deteriorate relationships. In my thesis, I design solutions to improve co-located communication between partners. I start my exploration with a survey examining co-located smartphone usage among partners. Results show that people often feel frustrated when their partner uses a smartphone in front of them and are not fully aware of their partner’s smartphone activities when co-located. This motivated me to design a smartphone application, CoAware, for sharing smartphone activity-related information between partners. Results from a user study with couples show that CoAware has the potential to improve smartphone activity awareness among co-located partners. However, CoAware doesn’t fully grasp the communication facilitation aspect desired. Thus, I further explore ways to strengthen conversation between co-located couples by introducing a smartphone agent which is designed to interact with humans - like a human. With a user study, I investigate the effects of agent mediation on communication dynamics around co-located smartphone usage in couples. Results reveal that the agent helps motivate users to reduce smartphone usage and is beneficial in promoting co-located interactions.
Designing Solutions to MediateCo-located Smartphone UsagebyKaranmeet Singh KhatraA THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFMASTER OF SCIENCEinTHE COLLEGE OF GRADUATE STUDIES(Computer Science)THE UNIVERSITY OF BRITISH COLUMBIA(Okanagan)December 2022© Karanmeet Singh Khatra, 2022The following individuals certify that they have read, and recommendto the College of Graduate Studies for acceptance, a thesis/dissertation en-titled:Designing Solutions to Mediate Co-located Smartphone Us-agesubmitted by Karanmeet Singh Khatra in partial fulfilment of the require-ments of the degree of Master of Science.Dr. Mohammad Khalad Hasan, CMPS, Irving K. Barber Faculty of ScienceSupervisorDr. Shan Du, CMPS, Irving K. Barber Faculty of ScienceSupervisory Committee MemberDr. Apurva Narayan, CMPS, Irving K. Barber Faculty of ScienceSupervisory Committee MemberDr. Jahangir Hossain, School of Engineering, Faculty of Applied ScienceUniversity ExamineriiAbstractThere is a growing concern that smartphone usage in front of family orfriends can be bothersome and even deteriorate relationships. In my thesis,I design solutions to improve co-located communication between partners.I start my exploration with a survey examining co-located smartphone us-age among partners. Results show that people often feel frustrated whentheir partner uses a smartphone in front of them and are not fully awareof their partner’s smartphone activities when co-located. This motivatedme to design a smartphone application, CoAware, for sharing smartphoneactivity-related information between partners. Results from a user studywith couples show that CoAware has the potential to improve smartphoneactivity awareness among co-located partners. However, CoAware doesn’tfully grasp the communication facilitation aspect desired. Thus, I furtherexplore ways to strengthen conversation between co-located couples by in-troducing a smartphone agent which is designed to interact with humans- like a human. With a user study, I investigate the effects of agent me-diation on communication dynamics around co-located smartphone usagein couples. Results reveal that the agent helps motivate users to reducesmartphone usage and is beneficial in promoting co-located interactions.iiiLay SummaryThere is a growing concern that people overuse smartphones even whenthey are co-located with family members (e.g., partners), which sometimesleads to anger and frustration. In my thesis, I design smartphone-based solu-tions to improve co-located communication between partners. More specif-ically, I design two smartphone applications enabling couples to be awareof their partners co-located smartphone usage and helping them to improvetheir communication dynamics with a smartphone agent. Results from userstudies show that my solutions assist couples in being aware of their partnersco-located smartphone usage and improve their communication dynamics.ivPrefaceThis thesis is the original, unpublished work by the author, Karanmeet.This study was granted approval by the University of British ColumbiaBehavioural Research Ethics Board- Certificate Number: H19-01032-A004.vTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . viList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . xiiDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiChapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . 1Chapter 2: Related Work . . . . . . . . . . . . . . . . . . . . . . 52.1 Sharing Information between Devices . . . . . . . . . . . . . . 52.1.1 Information Sharing through Digital Devices . . . . . 52.1.2 Usage-Aware Co-Located Smartphones . . . . . . . . . 62.2 Smartphone Usage and Effects on Individuals and Groups . . 62.2.1 Smartphone Over Usage and Effects on Individuals . . 62.2.2 Co-Located Smartphone Usage and Effects on Groups 72.3 Current Approaches to Reduce Smartphone Interaction . . . 82.3.1 Co-located Sharing Strategies . . . . . . . . . . . . . . 82.4 Smartphone Agents and Effects . . . . . . . . . . . . . . . . . 102.4.1 Interacting with a Smartphone Agent . . . . . . . . . 102.4.2 Mediation and Intervention Techniques via SmartphoneAgents . . . . . . . . . . . . . . . . . . . . . . . . . . . 10viTABLE OF CONTENTS2.4.3 Effects of Social Entities on Individuals . . . . . . . . 11Chapter 3: Study 1: Exploration of Co-Located SmartphoneUsage and Activity Awareness . . . . . . . . . . . . 133.1 Online Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.2 Participants and Study Procedure . . . . . . . . . . . . . . . 143.3 Data Analysis and Results . . . . . . . . . . . . . . . . . . . . 143.3.1 Demographics and smartphone usage . . . . . . . . . . 143.3.2 Co-located smartphone usage . . . . . . . . . . . . . . 153.3.3 Rules or mutual understanding on smartphone use . . 183.3.4 Strategies to reduce co-located smartphone usage . . . 203.3.5 Sharing information, privacy, and trust . . . . . . . . 203.3.6 Smartphone usage awareness . . . . . . . . . . . . . . 213.3.7 Co-Located Content Sharing . . . . . . . . . . . . . . 223.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23Chapter 4: The Design of CoAware . . . . . . . . . . . . . . . . 254.1 CoAware Features . . . . . . . . . . . . . . . . . . . . . . . . 254.2 Implementation Details . . . . . . . . . . . . . . . . . . . . . 30Chapter 5: Study 2: Explorations of CoAware . . . . . . . . . 315.1 Participants and Procedure: . . . . . . . . . . . . . . . . . . . 315.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32Chapter 6: Design of Smartphone-based apps . . . . . . . . . . 356.1 Interpretation of Results . . . . . . . . . . . . . . . . . . . . . 356.2 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366.2.1 Agent-based mediation . . . . . . . . . . . . . . . . . . 366.2.2 Text-based approach . . . . . . . . . . . . . . . . . . . 366.2.3 Application Development . . . . . . . . . . . . . . . . 36Chapter 7: Study 3: Explorations of Agent-based and Text-based Apps . . . . . . . . . . . . . . . . . . . . . . . 427.1 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427.2 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437.3.1 Effect on Interpersonal Relationship . . . . . . . . . . 447.3.2 Effects on Phone Usage Patterns . . . . . . . . . . . . 477.3.3 Effects on Verification Behavior Patterns . . . . . . . 497.3.4 Privacy Considerations . . . . . . . . . . . . . . . . . . 51viiTABLE OF CONTENTS7.3.5 Others . . . . . . . . . . . . . . . . . . . . . . . . . . . 52Chapter 8: Discussion and Design Recommendations . . . . . 53Chapter 9: Limitations and Future Work . . . . . . . . . . . . 58Chapter 10:Conclusion . . . . . . . . . . . . . . . . . . . . . . . . 61Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63viiiList of TablesTable 3.1 Categories of apps that participants used the most,©K. Khatra (2022) . . . . . . . . . . . . . . . . . . . 15Table 3.2 Reasons for using smartphones around partners, ©K.Khatra (2022) . . . . . . . . . . . . . . . . . . . . . . . 16Table 3.3 Locations that participants had set rules for restrictedsmartphone use, ©K. Khatra (2022) . . . . . . . . . . 18Table 3.4 The level of smartphone usage information partici-pants are comfortable with sharing with their partner,©K. Khatra (2022) . . . . . . . . . . . . . . . . . . . 23Table 3.5 The level of smartphone usage information partici-pants are comfortable with receiving from their part-ner, ©K. Khatra (2022) . . . . . . . . . . . . . . . . . 23ixList of FiguresFigure 3.1 Participants self-reported smartphone application us-age frequency when co-located with their partner,©K. Khatra (2022) . . . . . . . . . . . . . . . . . . . 17Figure 4.1 Once the access is gained to the partner’s device, thepartner can see the app the app category, ©K. Kha-tra (2022) . . . . . . . . . . . . . . . . . . . . . . . . . 26Figure 4.2 Once the access is gained to the partner’s device, thepartner can see the app name, ©K. Khatra (2022) . 26Figure 4.3 Once the access is gained to the partner’s device, thepartner can see a screen showing the screen contentof the app that the partner is viewing, ©K. Khatra(2022) . . . . . . . . . . . . . . . . . . . . . . . . . . . 27Figure 4.4 A connection initiates with a request to gain accessto the partners device, ©K. Khatra (2022) . . . . . . 27Figure 4.5 CoAware sending usage statistics, ©K. Khatra (2022) 28Figure 4.6 CoAware initiating a close request, ©K. Khatra (2022) 29Figure 4.7 CoAware showing a summary, ©K. Khatra (2022) . . 29Figure 4.8 CoAware sharing detailed information about apps usedsince the connection was established, ©K. Khatra(2022) . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Figure 5.1 Mean rating for (a) the usefulness of the access lev-els, (b) creating activity awareness with notificationstrategies, and (c) the usefulness of the notificationstrategies, ©K. Khatra (2022) . . . . . . . . . . . . . 33Figure 6.1 Upon clicking the play button, the background ser-vice starts, ©K. Khatra (2022) . . . . . . . . . . . . 37Figure 6.2 Connection request text in the text-based app, ©K.Khatra (2022) . . . . . . . . . . . . . . . . . . . . . . 38xLIST OF FIGURESFigure 6.3 Connection request text in the agent-based app,©K.Khatra (2022) . . . . . . . . . . . . . . . . . . . . . . 38Figure 6.4 Currently running app name and duration,©K. Kha-tra (2022) . . . . . . . . . . . . . . . . . . . . . . . . . 38Figure 6.5 Shows permission given and if two phones are withinrange, ©K. Khatra (2022) . . . . . . . . . . . . . . . 39Figure 6.6 Agent-based app: an agent asks if users are interestedin knowing what their partner is doing, ©K. Khatra(2022) . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Figure 6.7 Text-based home screen after partner has stoppedsharing, ©K. Khatra (2022) . . . . . . . . . . . . . . 40Figure 6.8 Agent-based home screen after partner has stoppedsharing, ©K. Khatra (2022) . . . . . . . . . . . . . . 41xiAcknowledgementsI would like to thank my supervisor Dr. Mohammad Khalad Hasan.Without him, my time at the university and doing research wouldn’t be asenjoyable. He was always available for meetings and would always providegreat feedback for me to build on. He also connected me with other pro-fessors and provided me with new opportunities to endure. His work ethicis unmatched, even if I had questions to ask him at 1:00 AM. My thesiswouldn’t be half the product it is without his support along the way.I would like to acknowledge helpful suggestions from my committee mem-bers: Dr. Shan Du, and Dr. Apurva Narayan.I owe a huge gratitude to my loving family. Their love and supportthroughout the years has provided me with the motivation to complete mythesis.xiiDedicationI dedicate my thesis to my loving parents, Jasvinder and Baljinder Kha-tra. I want to thank them for supporting me my whole life in whateverendeavour I take on.xiiiChapter 1IntroductionSmartphones continue playing a pivotal role in our daily communica-tions with family and friends [8, 65]. Smartphones not only enable seamlesscommunication over long distances, but also allow access to informationanywhere, anytime. However, there is growing evidence that people mayoveruse smartphones, both when alone and in the presence of others, i.e., ina co-located situation [4, 45]. Moreover, smartphones are designed as privateand personal devices: the activities that take place on the screen can, whendesired, easily remain completely unknown to co-located persons. Not beingaware of a co-located person’s on-screen activities can cause frustration andeven anxiety [68, 75].A significant amount of work has explored smartphone overuse and itsconsequences [4, 45, 91]. Much less attention has been devoted to designingsolutions for co-located activity awareness which could mitigate the frus-tration associated with smartphone overuse and improve interpersonal com-munication. A few recent studies have attempted to increase smartphoneactivity awareness by helping people to be more aware of co-located people’ssmartphone activities, providing a rich shared experience, and even motivat-ing people to initiate interaction with nearby persons [43, 68]. These studiessuggested different strategies to raise awareness, such as using ‘talk-aloud’to pass on what one is doing on the device [68] or to attach a second displayto the back of the phone to show on-screen activities to co-located individ-uals [43]. Though these solutions have the potential to increase smartphoneactivity awareness, I want to take this a step further by not only reducingthis smartphone usage but also further influence the communication dynam-ics by making partners more aware of what each other are doing on theirsmartphone.Social relationships can greatly shape the degree and nature of people’sinformation sharing with other co-located individuals [14, 25]. For example,the information sharing patterns of people with their partner, parents, andchildren may be very different, and may even vary largely depend on the ageof the individuals or the length of their relationship [7, 34]. To narrow downthe focus, I concentrate on an in-depth investigation of various aspects of1smartphone use by co-located partners who are married or in a common-law relationship, or in any other (romantic) relationship. I focus on suchrelationships because partners are often co-located for a substantial por-tion of each day and their mutual understanding is important for a healthyhome environment [17, 31, 66, 94]. Couple relationships are already verynuanced and complex and include many aspects, e.g., closeness, connect-edness, interpersonal trust, and perceptions of empathy. This makes themeven challenging to study on their own as a focal relationship [14, 31, 64].In my thesis, I aim to design solutions to improve co-located communi-cation between partners. Thus, this thesis is split into 2 different phases,with 3 studies taking place between these 2 phases. The first phase focuseson designing and evaluating an application (i.e., app) focusing on increas-ing smartphone activity awareness among co-located partners. The secondphase revolves around designing another agent-based app that plays a rolein communication facilitation between partners.Phase 1 begins by conducting a crowd-sourced study (Study 1) aimingto explore the following research objectives: 1) understand people’s smart-phone habits when being co-located with their partner, 2) gain insight intothe concerns that people have about their partner’s smartphone usage, 3)understand the rules and privacy issues partners have regarding their co-located smartphone usage, and 4) explore the strategies that people taketo become aware of their partner’s smartphone on-screen activities. Studyresults show that people often use smartphones while co-located with thepartner, which sometimes leads to anger and frustration. I also found thatpeople often need to respond to their partner’s queries about their smart-phone activities. Most people share information truthfully, although thedetails of the shared information vary widely between apps. Furthermore,many people feel that they are not fully aware of their partner’s smartphoneactivities when co-located. This lack of awareness can lead to unpleasantsituations. Some strategies such as ‘talk-aloud’ are being used to be awareof others’ smartphone activities, yet there remains a lack of expressive toolsto support smartphone activity awareness.Study 2 of Phase 1 is guided by the findings from Study 1. I exploredways to increase smartphone activity awareness among co-located partners.My goal was to investigate smartphone-based solutions to help partnersbecome aware of each other’s smartphone activities and to help them im-prove their interpersonal communication. I developed a smartphone app,CoAware, that enables users to create co-located smartphone usage aware-ness by sharing the names, categories, or screens of apps being used by one’spartner. Additionally, CoAware enables partners with ways to send notifi-2cations to each other such that they might motivate the partner to reduceco-located smartphone usage. I continued with an in-lab study with coupleswho explored and provided feedback on the features offered by CoAware.The results revealed that high-level information, such as sharing app namesand sending notifications, is useful to provide co-located smartphone usageawareness; however, low-level information about phone usage (e.g., screensharing) and allowing co-located partners to control a different person’sphone were seen to be less necessary and sometimes overbearing. I alsofound that awareness of a partner’s smartphone activities was not desiredby all participants. Some participants were satisfied with the level of in-formation they knew about, and some were fine relying on social protocolswith their partner to remedy challenging situations.Phase 2 began with interpreting the results of Phase 1. I found thatthe features that were to intrusive (ex: screen share, and controlling an-other person’s phone) needed to be removed. In addition, I found that theapp didn’t fully grasp the communication facilitation that I wanted to seebetween couples, thus I needed to design a new solution to tackle this prob-lem. The solution to these problems I decided on, was just allowing appname to be shared between couples and also adding in another app that Irefer to as the agent-based app which is based around a smartphone agent.Smartphone agents, which are designed to interact with humans - like a hu-man, have been shown to be useful in many smartphone usage contexts. Forinstance, prior research explored the use of agent-based smartphone appli-cations (i.e., app) for health education and counselling [49], for promotingactivities among users with traumatic brain injury [80], to deliver adviceregarding medical conditions [9] or for diabetes self-management education[49]. They showed that users became more efficient using agent-based sys-tems [56], had positive perceptions [76] and overall high satisfaction of theagent-based systems [35, 40, 83]. However, I am unaware of prior work lever-aging smartphone-based agent systems to facilitate communication betweenco-located individuals.For Phase 2, I explore the effect of using a smartphone agent systemin influencing communication among co-located people. More specifically,I investigated how a smartphone agent would affect co-located couples be-havioural changes, relationships and communication dynamics. Hence, Idesigned two smartphone apps: (a) an agent-based app that exchangessmartphone usage activities (e.g., app name, app usage duration) amongcouples while mimicking a virtual agent presenting the information to them.The agent was represented with a name, Anna, an avatar, allowing her toleverage voice modality to send messages and notifications to partners via3audio, and (b) a text-based app where a person can access and exchange thesame features manually.I conducted a between-subject user study (Study 3 of Phase 2) with 12couples to investigate the effect of the smartphone agent and text-based appswhile asking them to use the apps for seven days. I collected their feedbackwith an exit semi-structured interview where results showed that both thetext- and agent-based apps helped couples reduce co-located smartphoneusage, access information about what each other was doing, and initiateconversations. However, with the agent-version of the app, couples got thesense of a social entity intervening with them. In addition, the app helpedthem to stay on track and focus on communication with each other, thusproviding valuable mediating support in couples’ communication and rela-tionship dynamics.The chapters of this thesis are structured as follows. First, chapter 2begins with sharing work related to sharing information between electronicdevices, smartphone overuse and effects on individuals and co-located peo-ples, and current approaches to reduce smartphone usage. Chapter 2 alsoconsists of visiting areas of work related to current approaches to interactingwith smartphone agents, mediating and intervention techniques via smart-phone agents and effects of social entities on individuals. Chapter 3 beginswith Study 1 of Phase 1 providing the goals, procedure and results. Chapter4 then uses the results of the first study to discuss the app design, which inChapter 5 brings up Study 2 of Phase 1. Moreover, Chapter 6 begins withinterpreting the results of Study 2 of Phase 1 and presents an agent-basedapp. Chapter 7 discusses Study 3 of Phase 2, including the goals, procedureand results. Chapter 8 discusses the recommendations to design similarstrategies that aim at improving interpersonal communication . Chapter 9includes the limitations of the work and future work. Finally, chapter 10provides a conclusion.My thesis makes the following contributions:− design and implementation of a smartphone app that tracks couplesindividual and co-located smartphone usage− design and implementation of a smartphone agent that simulates ahuman role in facilitating conversation between individuals− exploration of notifications and a smartphone agent as a way to reduceco-located smartphone usage− design guidelines for future research at developing co-located interper-sonal interaction4Chapter 2Related WorkI divided my reviewed research work for designing my first text-based app(CoAware) and then my second agent-based app into four major sectionsnamely, Sharing Information between Devices, Smartphone Overuse and Ef-fects on Individuals and Groups, Current Approaches to Reduce SmartphoneInteraction, and Smartphone Agents and Effects.2.1 Sharing Information between Devices2.1.1 Information Sharing through Digital DevicesSharing digital devices and accounts are common practices amongsthousehold members, yet this topic has received less attention from an aware-ness perspective. Instead, the topic has mostly been motivated by the prosand cons associated with device sharing. Sharing devices can be viewedas an all-or-nothing approach for sharing information [44, 61], which givesrise to privacy and security issues [7, 42]. Studies also showed that devicesharing concerns depend on the user’s relation to the other user as wellas the types of data being shared, which suggests the need for better pri-vacy and security models for device sharing [27, 29, 44]. The intricate waysthat couples communicate [14] and their need to have a nearly continuousconnection have gained deep attention in the literature [31, 66, 94]. Thismotivated context sharing among people in close relations. Prior research[11, 24, 25, 50] showed how contextual information sharing (heart rate, dis-tance from home, etc.) can be leveraged to make partners aware of eachother’s context and activities. Both Buschek et al. [11] and Griggio et al.[25] observed that context-awareness improves the sense of connectednessand pointed out interpretability and privacy concerns that may arise dueto inferred additional information from the given context. While contextsharing helps people in close relationships to be aware of each other’s activ-ities, smartphone addiction and overuse of other digital tools may create anawareness barrier even when people are present in the same context.52.2. SMARTPHONE USAGE AND EFFECTS ON INDIVIDUALS AND GROUPS2.1.2 Usage-Aware Co-Located SmartphonesThough there has been substantial work on different sensing solutionsthat allow users to track the state of the device [32, 88], very little is knownabout how to detect two co-located smartphones and how to share infor-mation between them. Prior research showed that on-device sensors (e.g.,tilt) could be used to make a smartphone context-aware of its state (e.g.,orientation) [32], and usage context (e.g., resting on a table) [88]. Beyondsensing a device state or context, researchers explored external sensing so-lutions to enable smartphones to track surrounding activities and environ-ments [12, 26, 28, 53] and their social acceptance [2], but in limited contexts.To date due to the lack of advances in sensing solutions, such an approachhas received little attention in the context of using the space for co-locatedcollaborative interactions for promoting interpersonal engagement.2.2 Smartphone Usage and Effects on Individualsand Groups2.2.1 Smartphone Over Usage and Effects on IndividualsSmartphone overuse and smartphone addiction are active areas of re-search that examine people’s smartphone usage behavior. Over the pastcouple of years there has been an increase in smartphone usage [87, 90]. Re-searchers have investigated ways of detecting smartphone addiction based onusers’ smartphone usage behavior. For example, a recent study [65] identifieslifestyle and social media related apps to be associated with smartphone ad-diction. Researchers have explored and designed solutions to reduce smart-phone usage. These include tracking users’ smartphone usage and providingthem with summarized information to motivate them to limit their smart-phone usage [82], designing solutions to allow users to use smartphones fora time period throughout a day [33], and locking the smartphone into adaily use time limit [48]. All three studies showed signs of limiting user’ssmartphone usage. Researchers showed that reducing smartphone usage iscritical as over usage can lead to health effects such as depression, anxietyand a lack of sleep quality [16, 36, 37]. Lee et al. [55] conducted a studyusing questionnaires on 370 middle school students to analyze the effectsof smartphone use patterns on smartphone addiction. Results suggestedthat smartphone overuse led to a lack of control, withdrawal, mood mod-ification, and loss of interest. Not only can smartphone overuse affect anindividual’s health but also their work/academic productivity. Duke et al.62.2. SMARTPHONE USAGE AND EFFECTS ON INDIVIDUALS AND GROUPS[18] used a questionnaire to investigate whether there was any correlationbetween smartphone addiction, interruptions due to smartphones, and pro-ductivity during working. Results showed that smartphone overuse led toa decrease in productivity at the workplace and at home. Samaha et al.[86] conducted a study involving an online questionnaire and 300 universitystudents, aiming to explore what effects smartphone addiction had on stresslevels and academic performance. Results showed that smartphone addic-tion was positively related to stress as well as negatively effected academicperformance.2.2.2 Co-Located Smartphone Usage and Effects on GroupsIn addition to overuse in individual smartphone usage, research has alsoshown smartphone overuse to be an increasing aspect in co-located environ-ments [4, 45]. Kawsat et al. [45] investigated smartphone overuse by logginginternet usage from 86 Belgium households and revealed that the overusewas typically prominent in rooms such as kitchens and living rooms. Therising smartphone usage has led to individuals having difficulties separatingtheir smartphone life from their personal life [13, 85]. Additionally, smart-phone over-usage has shown to lead to a decrease in work productivity ingroups, cause strains in relationships, and even escalate to violence [18, 46].Further, prior studies have looked into how smartphone overuse by parents,can effect their children. Adair et al. [91] held interviews where they re-vealed that children often noticed when their parent using their smartphonesand resulted in negatives to the children’s cognitive and social development.Radesky et al. [81] conducted a study which reported on how parents woulduse their phones around children at lunchtime. They showed that the us-age resulted in the children misbehaving as they would notice their parentson their phones and have minimal interaction with the children. On therelationship aspect, Proulx et al. [79] reported that excessive smartphoneoveruse can lead to marital disputes and thus lead to stress and depres-sion. Oduor et al. [69] investigated the positives and negatives of co-locatedmobile device usage in a one-to-one space and wanted to understand howsmartphone usage in homes shape the household dynamics. They showedthat often times users using phones around family members for non-urgentactivities can trigger frustrations among them. Thus, solutions need to bedeveloped to attempt to reduce smartphone overuse in co-located scenariosand encourage more face-to-face interaction.72.3. CURRENT APPROACHES TO REDUCE SMARTPHONE INTERACTION2.3 Current Approaches to Reduce SmartphoneInteractionPrior research explored ways to reduce smartphone usage and encour-age in person communication with others. The two main areas researchedinclude: co-located sharing strategies, where smartphone activities and up-dates are shared amongst co-located individuals; and notification-based strate-gies, where smartphone notifications are more intelligently put through be-tween smartphones.2.3.1 Co-located Sharing StrategiesResearchers explored strategies to share what co-located individuals aredoing on their smartphone and find ways to reduce their smartphone usage.Seeburger et al [89] designed an app, Capital Music, that allows users toshare real-time song choices with other co-located individuals. The app pro-vides insight into how users feel sharing non-privacy sensitive informationsuch as music selections with one another. Capital Music also attempts tobreak users out of the “cocoon” effect, which takes place when users are con-fined to only themselves and have no realization of activities around them.Other researchers have investigated ways to mitigate or limit smartphonedistractions in a group environment. Prior research explored using differentfeatures such as locking users out of their smartphone device [52]. Minsamet al. [52] designed an application (Lock n’ LoL) to help users focus on theirgroup activities by allowing group members to limit their smartphone usagetogether. Lock n’ LoL provides synchronous social awareness of each other’slimiting behavior. Using a research study with 976 students, results showedthat Lock n’ LoL helped users mitigate smartphone distractions. Previousresearch has also gone into sharing what a person is doing on a display[43, 51]. Pradthana et al. [43] built a prototype that reveals what a user iscurrently doing on their smartphone on a display on the back of their mobiledevice. Following a ten-day trial results showed the prototype was beneficialin increasing awareness of nearby user’s mobile activities while also trigger-ing interactions with one another. Kim et al. [47] developed Let’s FOCUS,an app built to help college students stay focused to their class. The appwas designed to remind the students to explicitly restrict their mobile phoneuse voluntarily while they are in class. Results revealed the 379 studentsused the app to limit their smartphone usage by 9335 (24.6 on average)hours while they were in the classroom over a six week period. Cuotto etal. [15] and Paasovaara et al. [72] developed two apps, Idliketoknow and82.3. CURRENT APPROACHES TO REDUCE SMARTPHONE INTERACTIONNext2You, that are designed to encourage social interaction anomymouslyand via gamification between co-located individuals. Both apps exhibitedsigns of acting as an icebreaker to further facilitate conversation betweenindividuals. Constantly, family meal times are distracted by personal de-vices [60], and thus Ferdous et al. [21] developed TableTalk, an app thattakes personal devices and combines them into a shared display on the ta-ble to enrich meal time interactions. Results from a user study showed thatTableTalk encouraged more conversations and socialization in the co-locatedfamily. While all these strategies are designed to reduce smartphone usageby sharing what individuals are doing on their devices in a co-located en-vironment, none of the prior work explored how to design an agent-basedsolution to motivate reducing co-located smartphone usage.Notification-Based StrategiesOftentimes notifications can distract users from social interactions whendelivered at inopportune times [63]. Prior work explored ways (i.e., whenand how) to present smartphone notifications to users. For instance, Park etal. [74] investigated break-point-based notifications, where notifications areonly sent to users when an opportune moment has arrived. They exploredthree different breakpoint styles to deliver a notification: a long stretch of si-lence from the group, co-located individuals leaving the table/environment,and when co-located individuals are using their smartphone. Results showedthat the app was able to accurately detect the break-points, reduce notifica-tion interruptions, and encourage social interaction. Similar to the previousidea, Okoshi et al. [71] explored user interaction-based (e.g., closing anapp) and physical activity-based (e.g., when a user stops running) break-points in users’ daily life and appropriately sending notifications at thesebreakpoints. Results from user studies revealed that this break-point-basedsystem resulted in a significant reduction in users workload perception. Linet al. [73] conducted another study focusing on intelligently exchanging no-tifications to enhance social interactions. The authors built a system thattakes social context into account and appropriately sent notifications basedon this factors. Two more research studies also proposed developing an in-telligent system that aims to automatically find opportune times to sendnotifications or prompt users to engage with content [62, 78]. They foundusers to engage with more notifications and content in the systems built inthese research studies, in comparison to baselines.92.4. SMARTPHONE AGENTS AND EFFECTS2.4 Smartphone Agents and Effects2.4.1 Interacting with a Smartphone AgentThere has been research done with smartphone agents in persuasivetechnologies to enhance users experience such as in speech and voice [101],healthcare [6, 9, 49, 54, 77, 80], and smartphone interaction [67, 95, 96].For instance, Yamamoto et al. [101] designed a 3D-Computer Generated(CG) virtual agent that allowed users to interact with their smartphonesthrough voice command without any delays. Results showed the agentenabled users to interact with it more naturally then existing systems aswell as have a short delay in their interaction. Moreover, researchers ex-plored leveraging virtual agents in the medical field, where the agents havebeen designed to provide advise regarding medical conditions to patients[6, 9, 49, 54, 80]. Furthermore, Phillip et al. [77] developed a smartphoneapp called KANOPEE that enables users to interact with a virtual agentdesigned to help with their sleep. The virtual agent is an AI program us-ing decision tree architecture[93], to help develop a personalized sleep rec-ommendations for 10 days for participants to follow. Results showed thatparticipants welcomed and accepted the virtual agent concept to assist withtheir sleep and consequently their sleep quality reported to be improved im-mensely. Virtual agents have been explored in other contexts such as virtualtours and navigation. For instance, Uchiya et al. [95, 96] developed a voiceinteraction agent, using a 3D model GPS and Google Maps, that guidedindividuals through a campus tour. Study results showed that the virtualagent contributed to cost reduction at the university, reduction of time andeffort required by actual tour navigators, and removed the problem of timerestrictions for on campus activities. Obremski et al. [67] implemented aprototype that focused on humans interacting with an autonomous Intelli-gent Virtual Agent (IVA) that pretended to act as a smartphone. To mybest knowledge, all the prior work on smartphone agent focused on assistingindividuals to achieve certain goals. I aspire to develop a smartphone-basedagent encourage co-located users to reduce smartphone activities.2.4.2 Mediation and Intervention Techniques viaSmartphone AgentsResearchers explored leveraging mediation and intervention techniqueson smartphones. For instance, Webb et al. [100] constructed a mediationapp used to reduce psychological distress. Similarly, Mahlo et al. [59] cre-ated a mindfulness-mediation program for older adults. Both showed signifi-102.4. SMARTPHONE AGENTS AND EFFECTScant improvements to the persons well being after using the mediation apps.Stieger et al. [92] designed PEACH, an intervention app that is designed tocoach individuals and motivate them to change aspects of their personality.They found the intervention app to be effective in changing aspects of indi-vidual’s personality. In similar work, Dupuy et al. [19] developed a virtualagent app, KANOPEE2, that autonomously screened and alleviated insom-nia symptoms through an intervention program and offered personalizedadvice and relaxation techniques. KANOPEE2 was shown to reduce insom-nia symptoms using these intervention techniques. In more recent studies,Ismail et al. [41] presented a conceptual framework for agent-mediation tomonitor users self quarantining to lessen the amount of work on health of-ficers. The proposed agent would have two tasks assigned to it: checkingthe quarantined individual’s body temperature at certain time periods andreporting if it was abnormal; and, determining through longitudinal andlatitudinal data if the quarantining individual had moved to far from wherethey should have been quarantining. The authors revealed the agent to bea starting framework for further implementation as only using sensors wasdeemed to be insufficient to perform the tasks of monitoring self-quarantinedindividuals. Researchers also explored ways to design smartphone-based so-lutions to mediate a touristic experience [20, 84, 102]. For instance, Wanget al. [98] explored the mediation mechanisms of smartphones, by dwellinginto their apps used as well as the stories the travellers had while trav-elling. Results showed that smartphones can actually change the touristsbehavior by taking on a mediating approach. However, very little is knownon smartphone agents using mediation and intervention techniques betweenindividuals for co-located smartphone usage. In my research, I aim to fur-ther explore the direct use of an smartphone agent and the effects of agentmediation in co-located smartphone usage.2.4.3 Effects of Social Entities on IndividualsResearchers have examined effects on social entities into both online (i.e.,over the computer) and in-person interaction techniques between people(i.e., with a robot) and an agent. For instance, Watanbe et al. [99] ran afield study with an android robot that made conversation with visitors whomwere interacting with it via multiple displays. The authors were interestedin investigating which kind of functions the android robot needed in order tobe recognized as human like by the visitors. Results showed that the visitorsviewed the android to be human like while also suggesting the android hadan effect on their social influence acting as an advertisement effect. Liszio112.4. SMARTPHONE AGENTS AND EFFECTSet al.[57] conducted a study involving integrating social entities into VirtualReality (VR) games to enhance sociability. In a study conducted with 75participants the authors compared the effects of another human player anda virtual agent on the player’s experience. Results showed that by addingsocial entities, players had a decreased sense of loneliness whilst playing thegame, which enhanced their experience. My designed smartphone agentcan potentially be viewed as a social entity that facilitates conversationsbetween co-located individuals. Consequently, I aim to determine whetheradding in a social entity will cause any effect on co-located people. To myknowledge no previous work has explored smartphone agents, particularlyanthropomorphized agents, in persuasive spaces.12Chapter 3Study 1: Exploration ofCo-Located SmartphoneUsage and ActivityAwarenessI started my exploration by conducting a crowdsourced study investigat-ing people’s smartphone usage, rules, trust and privacy concerns, and activ-ity awareness when they use smartphones in the presence of their partner.Prior research has shown that crowdsourcing platforms, such as AmazonMechanical Turk (AMT), are popular and convenient tools for conductinguser studies and collecting reliable data [3]. I used AMT to run my study.3.1 Online SurveyI created an online survey with 58 questions to collect data from smart-phone users. The survey contained five sections: i) 18 questions to collectdemographic information about participants and their partners (e.g., age,nationality, gender, education, and household conditions); ii) 15 questionsabout smartphone usage (e.g., how often, where and what types of apps),usage rules in the household, and privacy-related issues; iii) 9 questionsabout trust-related issues that arise when people share their smartphoneusage activities with their partner and other family members; iv) 5 ques-tions targeted at smartphone usage behavior and habits when co-locatedwith the partner; and v) 11 questions regarding the awareness of the part-ner’s smartphone usage and possible strategies used to share usage relatedinformation with a partner. In total, I used 15 open-ended questions, 26single/multiple-choice questions, and 17 5-point Likert scale questions. Mostof the open-ended questions were used to collect descriptive responses aboutco-located smartphone usage where the Likert scale questions were designedto quantify results and to obtain shades of perceptions regarding issues on133.2. PARTICIPANTS AND STUDY PROCEDUREsmartphone usage.The single/multiple-choice questions were primarily usedto collect demographic data.3.2 Participants and Study ProcedureI posted my survey as a Human Intelligence Task (HIT) to AMT (witha $1.00 compensation). I specified two qualifications for participants: aminimum of 70% approval rate and a minimum of 50 previously completedHITs. I also set the following requirements for the workers: they (a) mustown a smartphone, (b) be either married, or in a common-law, or in apartner relationship, where (c) the partner must also own a smartphoneand (d) currently live in the same household. In total, I collected 109 re-sponses in seven days. I subsequently removed 31 responses which containedone or more unanswered questions and/or invalid answers. Consequently,I analyzed data from 78 participants (34 female, 44 male). On average,participants took 25 minutes to complete all the questions.3.3 Data Analysis and ResultsI applied a thematic analysis on the qualitative data where two re-searchers separately went through all the comments to perform open coding.Later they consolidated and reconciled codes into a common code set. Self-reported quantitative data were analyzed using standard statistical methodssuch as mean and standard deviation.3.3.1 Demographics and smartphone usageThe majority of my participants were from two age ranges: between 24and 34 years (28 participants) and between 35 and 44 years (29 participants).Three participants were aged between 18 and 24 years, nine between 45 and54 years, and nine between 55 and 64 years. Only two participants were 65years or older. Fifty-five participants were from the USA, 23 from India.On average, my participants had been in their relationship for 13.3 years.Participants and their partners had been using smartphones for 8.5 and 7.9years, respectively. Participants reported using smartphones an average of3.3 (SD=2.0) hours per day.I asked participants what categories of apps they used the most. Table3.1 shows the results (Note: participants could choose more than one appcategory). I observe that communication (e.g., email, text message, skype,143.3. DATA ANALYSIS AND RESULTSApp Category UsageAmount(%)Communication 90%Social Media 83%Internet 85%Location-sharing 19%Health-related 29%Table 3.1: Categories of apps that participants used the most, ©K. Khatra(2022)phone calls), social media (e.g., Facebook, Instagram, Snapchat) and theInternet apps (e.g., reading news, hobby-related browsing, banking) wereused the most at least once a day. Meanwhile, location-sharing apps (e.g.,Glympse, Life360, Find My Friends) and health-related apps (e.g., fitnesstracking, sports, or medicines apps) were used less at least once a day.3.3.2 Co-located smartphone usageParticipants reported that they are co-located with their partner for aconsiderable amount of time (mean 5.9, SD=3.6 hours/day excluding sleep-ing time) and that they often engage in collaborative activities with theirpartner, such as cooking or watching movies (mean 2.2, SD=1.4 hours/day).In response to an open-ended question, participants reported various rea-sons for using their smartphones when co-located with the partner. Ta-ble 3.2 shows the results. In total, I analyzed 167 coded responses whichcan be categorized into the following five broad categories: 1) communica-tion/ socialization with friends or family members (38% of the responses), 2)work-related activities (20% of the responses), 3) checking information andupdates for own interest (20% of the responses), 4) finding information fora purpose shared with the co-located partner (13% of the responses), and 5)personal entertainment (9% of the responses). These results are similar toearlier qualitative results [68] which showed that people use smartphones inthe presence of their family members to check notifications, find information,and fill time when they are bored. my results also revealed that co-locatedsmartphone use frequently happens at home (44% of the responses), mostlyin the living room and bedroom. Many participants talked about using theirphones at home when co-located with their partners. “ watching a movieon tv at home and he was upset that I checked my phone.” [P20, female,153.3. DATA ANALYSIS AND RESULTSReason Number ofPeople (%)Communication/ socialization with friends orfamily members38%Work-related activities 20%Checking information and updates for owninterest20%Finding information for a purpose shared withco-located partner13%Personal entertainment 9%Table 3.2: Reasons for using smartphones around partners, ©K. Khatra(2022)relationship 27 years]Other common places for co-located smartphone use include restaurants(18% of the responses), public spaces such as in shopping malls or parks(24%), during social gatherings (7%), and inside cars (7%). All participantsreported frequent occurrences (at least once a month) of their partner ex-pressing concerns regarding their co-located smartphone usage. “I pulledout my phone just to go on it before the food came and he complained.”[P1, female, relationship 4 years] || “We were in bed together and not reallypaying attention to what she was saying.” [P16, male, relationship 21 years]I coded a total of 114 responses regarding the situations (places andactivities) when this had happened: at mealtime, either at home or in arestaurant (29% of the responses), while watching tv/movies together athome (16%), in public places, such as in a shopping mall (15%), in thebedroom at bedtime (13%), during on-going conversations (10%), and insome other situations such as at a social gathering or while in the car. Twoparticipants from India responded that it had happened while being in atemple.Participants’ concerns were mostly related to the lack of attention towhat they had expected their partner to concentrate on during a conversa-tion or other activity. Sometimes they were concerned about disregardingfamily time and social engagement (especially when surrounded by familyor friends in social gatherings). Several participants mentioned that theirfailure at paying attention sometimes led to frustrations and tensions be-tween them. “When we sit together and talk to each other in our living areaat home, I go through messages in WhatsApp. That time my partner gets163.3. DATA ANALYSIS AND RESULTSFigure 3.1: Participants self-reported smartphone application usage fre-quency when co-located with their partner, ©K. Khatra (2022)irritated, thinking that I am not listening to him.” [P10, female, relation-ship 21 years] || “During a dinner at his friend homeI was using my phonecontinuously to text my friends. He signaled me not to use the phone ata get together because it seems odd when I am not involving in the event.I keep on texting my friends he raised and fought with me.” [P21, female,relationship 10 years]Participants reported that their partner expressed concerns about theirco-located smartphone usage primarily due to disruption in their qualitytime, and sometimes expressed anger and annoyance. On average, partici-pants reported that they spend 2.3 hours a day on their smartphones whileco-located with their partners. For each app category, at least 30% of theparticipants who use those apps more than once a day reported to use themless frequently while co-located with their partner.I also asked participants about the apps that they often use when theyare co-located with the partner. Figure 3.1 shows the results. I observe thatthey frequently surf the Internet (e.g., reading news, browsing, banking),use communication apps (e.g., email, text message, Skype, phone calls) andsocial media apps (e.g., Facebook, Instagram, Snapchat). However, theyrarely use health-related apps (e.g., fitness tracking or medicines apps) andlocation sharing apps (e.g., Glympse, Life360). The results suggest thatpeople prefer to use communication and other related apps to connect tofamilies and friends when co-located with their partner.173.3. DATA ANALYSIS AND RESULTSLocation Number ofPeople (%)Mealtime 33%Family time with kids 21%Collaborative activities 16%Bedtime 12%Social gatherings 7%Driving 5%Table 3.3: Locations that participants had set rules for restricted smart-phone use, ©K. Khatra (2022)3.3.3 Rules or mutual understanding on smartphone useI asked participants questions about rules or agreements set in the house-hold to reduce co-located smartphone usage. More than one-third of theparticipants (35%) mentioned having some house rules. The rest (65%) saidthey did not have any formal agreement, yet they shared a mutual under-standing with their partner. “We’re responsible and adult enough to knowwhen it’s time to use the phone or not.” [P52, male, relationship 11 years]The participants who reported to have rules or agreements (43 codedresponses) for smartphone use had rules based on either locations or situ-ations. Table 3.3 shows the results. I observe that common locations ofrestricted smartphone use was during mealtime, family time with kids andcollaborative activities. Whereas, bedtime, social gatherings, and drivingwere less common areas of restricted smartphone usage. “We agreed to notuse our smartphones during dinner unless it’s an emergency.” [P46, male,relationship 26 years]Often the rules or agreements were set to ensure quality time within thefamily and in social gathering: “I am in agreement with him that we do notuse our phones when it’s quality time for us to be together or when we’rewith others in a social situation, unless everyone is using them, too, forsome reason (like looking up some info or playing a game together).” [P58,male, relationship 8 years]The rules also came from self-realization of being disconnected: “Onceme and my spouse was continuously using the phone when we were at homewe realized that we didn’t speak to each other. That moment we decidednot to use phones unless an emergency when we both are together.” [P21,female, relationship 10 years]I asked the 51 participants, who did not have any rules, how they would183.3. DATA ANALYSIS AND RESULTSfeel about creating them. Fifty percent of these participants welcomed theidea of having some rules for ensuring proper engagement with the partner.Twenty-five percent expressed being somewhat neutral about agreeing onrules. The remaining 25% opposed the idea of agreeing on rules. They didso as they felt that rules would intrude on their smartphone activities orthat it may be an “overkill” between adults who should be able to act ontheir own accord. Some stated that they shared mutual respect not to usesmartphones in certain situations and do not need any rules. “We shouldcome up with guidelines for smartphone usage that would make me feel betterabout our communication with one another.” [P37, female, relationship 14years]The average length of relationships for the participants who have no ruleswas larger (avg. 14 years) compared to the participants who reported to haverules (avg. 10 years). Binomial Logistic Regression showed no significantdifference in gender and relationship length between these two groups.I asked participants whether they have any rules regarding smartphoneuse for other family members, excluding themselves, such as their parents(e.g., an older adult living at home with their adult children), teenage chil-dren, or younger children. Most participants mentioned that they do nothave any rules for other adults in the home as they are responsible adultswho do not use smartphones frequently. “There is no rule as my mom is anaged woman and did not use phones every day.” [P39, male, relationship 12years]However, 40 participants with children have strategies and rules to con-trol the child’s smartphone usage. For instance, out of 52 responses, 37% ofthe responses were about time-based restrictions (e.g., no more than 30 minper day), 23% were about content-based restrictions (e.g., only for games andwatching YouTube videos), 21% were location-based restrictions (e.g., notat the dining table, in the bedroom or washroom), and 6% were about age-based restrictions (e.g., no phone before 8 years). Such “no phone” policieswere primarily set to ensure that the children were engaged in more purpose-ful activities and to ensure they spent enough time with family members.Typical responses were: “We do not allow our sons to use their smartphonesin private such as their bedroom or bathroom. We also have their settingsconfigured so their phones may not be used between 10pm and 6am.” [P46,father of 2 children]193.3. DATA ANALYSIS AND RESULTS3.3.4 Strategies to reduce co-located smartphone usageI also asked my participants whether they know of or used any apps orother tools to reduce smartphone use in co-located situations. The majority(67 out of 78) mentioned that they do not know of any such solutions thatcould either help them be more aware of each other’s smartphone activitiesor help to reduce co-located smartphone usage. The remaining participants(11) mentioned that they are aware of apps to restrict usage time. Theymentioned using iPhone’s Screen Time [39], Night Mode [38], Offtime [70]to track their daily smartphone usage activities and to limit smartphoneapp access after a certain amount of time.I used a 5-point Likert scale to collect participants’ opinions on theimportance of using apps or other strategies to reduce co-located smartphoneusage. I observed that younger participants felt that it was more importantto have apps or strategies in comparison to the older participants. Out of 28participants in the age range 25 to 34 years, 17 participants expressed highimportance (rating 3 or more) of having such apps or strategies, whereasonly 25 of 50 participants in the age range 35 to 65+ years expressed highimportance.3.3.5 Sharing information, privacy, and trustAbout 74% of participants said that they told their partner what theywere doing on their smartphone when co-located at least a few times aweek. I also asked them how truthful they are when sharing information.Twenty five out of 34 females and 19 out of 44 males said that they shareaccurate/truthful information about their smartphone activities with theirpartner. Participants who said they told the truth commented that they donot have anything to hide from their partner and do not want to lose thetrust. “We value honesty in our relationship, not that we do anything shadyon our phones, but if we did, I would immediately inform her of anything Idid, and vice versa.” [P52, male, relationship 11 years]On the other hand, participants who said they did not always share ac-curate information with their partners did so because they were trying tosafeguard their privacy or ensure personal boundaries. For instance, someparticipants mentioned that they are not comfortable sharing financial in-formation, business matters, photos, videos and things that they search ontheir phone. This is due to the sensitivity of this information, and sometimesto maintain personal space [94]. “I might be slightly embarrassed about therandom things I look up.” [P33, female, relationship 20 years]203.3. DATA ANALYSIS AND RESULTSIn a follow-up 5-point Likert question (5=very confident to 1=not confi-dent at all), participants were asked to indicate their confidence level aboutwhether their partners tell true information about their smartphone activ-ities. Both male and female participants had strong confidence that theirpartners share accurate information with them, which reflects their averagescore of 4.45 (SD=0.84) and 4.68 (SD=0.79), respectively. Only six par-ticipants gave a rating of 3 or below and expressed their past experience offinding their partners not being truthful. This experience could consequentlycreate an impact on their level of trust in the future. “About 9 months agomy partner expressed that due to past cheating by previous partners, she feltparanoid when I was using my phone to chat with other people.” [P8, male,relationship 1 year]3.3.6 Smartphone usage awarenessI examined participants’ awareness about exactly what their partner isdoing on their smartphone and how interested they are in knowing whattheir partner does on their smartphone. Seventy eight percent of the partic-ipants responded that they are not fully aware of their partner’s smartphoneactivities. In some cases, participants reported that this lack of awarenesstriggered misunderstanding among the co-located partners as their partnersmake assumptions based on their smartphone activities. A potential reasonfor such an assumption could be the limited information that can be seenfrom a distance about a person’s usage activities. Similar results were foundby Oduor et al. [68] who reported a lack of smartphone activity awarenessamong co-located family members.I included questions on the common strategies for sharing activity aware-ness with co-located partners. Participants reported that such awareness wasoften achieved by asking questions of their partners where they respondedverbally or showed their screen to their partner. This action sometimes ledto frustration and anxiety among partners. “I normally just ask what he’sdoing (especially if he laughs!) and he’ll always tell me.” [P1, female, rela-tionship 4 years] || “My partner usually gets aggravated when I ask what heis doing, because usually, he is trying to figure something out on his phone.”[P11, female, relationship 15 years]In exploring how interested participants were in knowing the partner’ssmartphone activities, I found that male participants were more interested(66% of the males were interested) in knowing what their partner is doingon the smartphone than female participants (58%). On the contrary, ina question asking about their partner’s interest in knowing what they are213.3. DATA ANALYSIS AND RESULTSdoing on their smartphone, I found that 86% of male participants reportedtheir partners to be interested in knowing their activities, whereas 68% of thefemales reported the same. I observed a trend that this interest decreasedgradually with the increase of age range. In the age range 25 to 34 years,82% expressed interest, whereas in the age range 35 to 44 years only 70%showed interest.3.3.7 Co-Located Content SharingI collected information on level (details vs. abstract) of smartphoneactivity information that the participants are comfortable to share with theirpartner and the level of information that they would like to receive from theirpartner. I provided them with three different levels that they could choosefrom for sharing or receiving: (i) detailed information (e.g., chatting with“Alex” in Facebook), (ii) an app’s name (e.g., using Facebook), and (iii)activity information (e.g., playing games). Additionally, they could writeany other abstractions that they might be comfortable with. I collected 119coded responses for sharing and 120 coded responses for receiving level ofinformation as they were allowed to select multiple levels.Table 3.4 shows the level of smartphone usage information participantsare comfortable sharing with their partner. Many participants are com-fortable with providing very detailed information to their partners (37%responses), whereas others reported preferring to share only the app name(34% responses) or general activity information (29% responses). The otherparticipants reported only feeling comfortable with providing less or no in-formation at all. I also found that the preferred sharing level varies acrossapps. 37 participants mentioned that they share details when using com-munication apps whereas only 19 participants share details while they arebrowsing the Internet. I observed similar results for receiving informationfrom their partner. Table 3.5 shows the results. Many participants indicatedthat they would like to receive detailed information from their partners (34%responses), whereas others expressed to receive only the app name (36% re-sponses) or activity information (29% responses). The other participants(only 1%) reported feeling comfortable with receiving any level or no infor-mation at all.I further asked participants to provide examples where they share smart-phone usage information with different people (e.g., partner, family mem-bers). I observed that it is common to share different levels of informationwith different people: “I would give less details based on how well I know theperson. My partner and family get more information that colleagues.” [P57,223.4. DISCUSSIONLevel of Information Number ofPeople (%)Detailed information 37%App names 34%Activity information 29%Table 3.4: The level of smartphone usage information participants are com-fortable with sharing with their partner, ©K. Khatra (2022)Level of Information Number ofPeople (%)Detailed information 34%App names 36%Activity information 29%No information 1%Table 3.5: The level of smartphone usage information participants are com-fortable with receiving from their partner, ©K. Khatra (2022)male, relationship 2 years] || “To my partner, I share all the information; tomy family members, I share only app name or activity name” [P53, male,relationship 7 years]3.4 DiscussionResults from the survey revealed that people use smartphones in thepresence of their partner even though their partner expressed concerns aboutthe usage. In general, people can see when partners use smartphones infront of them, but exactly what a partner is doing on the phone cannot beeasily inferred from an observer’s viewpoint (also found by [43]). my workbuilds on prior work by illustrating the locations and activities in whichthis occurs, the rules people have setup to help mitigate issues, and howthey feel about sharing usage information. Participants reported that co-located usage and asking about their partner’s activities sometimes triggersaggravating situations. However, there are no known technological solutionsavailable that would help them share their smartphone usage informationwith their partner while offering some degree of privacy to increase activityawareness and create shared experiences in co-located contexts.These findings motivated me to think of a means to improve people’sawareness about their partner’s phone activities while co-located by sup-233.4. DISCUSSIONporting different levels of information (details vs. abstract), such that theycan make informed decisions about how to handle the situation. Addition-ally, prior research showed that improved awareness could help couples tostay in sync [5] and be motivated to limit phone usage and, thus, improvethe quality of domestic life [58]. I recognize that awareness of a partner’ssmartphone activities was not desired by all participants. Some participantswere satisfied with the level of information they knew about, and some werefine relying on social protocols with their partner to remedy challengingsituations. Thus, a natural progression warrants an examination of designsolutions that might work for people who were more interested in additionalknowledge of what their partners were doing on their phone in a hope toimprove social interactions. Prior work showed that smartphone activityawareness can be achieved with additional hardware instrumentation suchas attaching an additional display on the backside of a phone to providecues to smartphone activities [43]. Results revealed that users did not feelcomfortable using it and were unwilling to reveal app related informationdue to privacy concerns. Instead, I developed an app-based solution on anunmodified smartphone to provide new opportunities to increase activityawareness among partners. With my design, CoAware, users can intuitivelyshare activity information with various granularity levels to help avoid pri-vacy concerns.24Chapter 4The Design of CoAwareInformed by my findings, I designed a smartphone app, Co-located Aware-ness (CoAware), intended for sharing smartphone usage information be-tween co-located partners.4.1 CoAware FeaturesCoAware was designed to share users’ smartphone usage informationsuch as the number of times an app has been launched, the duration it hasbeen used for, and the time it was initially opened. I used a foreground appchecker external library [97] that allows access to smartphone app usage in-formation. In addition, I developed a solution to directly share screens fromone smartphone to another via WiFi direct. Based on these capabilities,I developed three techniques to share app usage information between twoco-located smartphones with CoAware. As my survey results showed thatpeople prefer to share information by varying degree of details about appusage, I designed the app to have three different levels of access; from verylimited information which users may be more comfortable sharing (e.g., anapp category) to very specific information (e.g., viewing the screen) thatcould possibly be more privacy intrusive. Thus, users can choose what levelof sharing they and their partner are comfortable with. The specific levelsare:App Category: This access level provides users with a high-level viewof app usage information, where only the types of apps being used are shownand not the app names (Figure 4.1). For instance, apps that are used forcontacting other people (e.g., email, text message, skype, phone calls) aremapped to and labeled as “Communication.” Commonly used apps arecategorized into different labels.App Name: In this access level, CoAware tracks the name of a runningapp on the co-located phone (Figure 4.2). The app name is displayed on theother phone.Screen Share: In this access level, CoAware captures images of aphone’s screen and transfers it to the co-located phone every 50ms. In254.1. COAWARE FEATURESFigure 4.1: Once the access is gained to the partner’s device, the partnercan see the app the app category, ©K. Khatra (2022)Figure 4.2: Once the access is gained to the partner’s device, the partnercan see the app name, ©K. Khatra (2022)this way, co-located users are aware of the exact on-screen activities of eachother (Figure 4.3).With CoAware, when two devices are co-located, one device sends a con-nection request and asks permission to access App Name, App Category orScreen Share information. The receiving device shows the request in a pop-up (Figure 4.4) where the user of the device can accept or reject the request.If accepted, the sender device gains access to the app name, app category orthe device screens of the other device. It also starts logging the app usageinformation (e.g., running app) on the receiving device. CoAware providesthree notification strategies to allow co-located partners to send informationthrough the app. I wanted to provide various levels of information exchangeand control.Message: Users can send a preset or custom message to their partner.Examples of preset messages are “It feels like you’ve been using Gmail fora while now, can we talk instead?” and “Hey, it’s me. How are thingsgoing?” Custom messages allow users to type anything in a text box andsend the text to their partner. I included this possibility to offer flexibility interms of how users like to communicate with their partners. This messaging264.1. COAWARE FEATURESFigure 4.3: Once the access is gained to the partner’s device, the partnercan see a screen showing the screen content of the app that the partner isviewing, ©K. Khatra (2022)Figure 4.4: A connection initiates with a request to gain access to the part-ners device, ©K. Khatra (2022)feature is similar to sending a text message; however, I hoped that presetsuggestions for messages might help to create courteous exchanges between274.1. COAWARE FEATURESpartners and not heighten tensions. This reflects findings from my surveywhere some participants said they would gently ask their partner about theirsmartphone usage if they felt it was inappropriate.Usage Statistics: Users can send app usage statistics such as “You havebeen using Gmail for 4 min and 54 seconds.” to their partner (Figure 4.5).Such messages are created using the duration of the longest-running appamong the currently running ones, since the longest-running app is oftenlikely to keep the user engaged for a longer period of time. This reflectsfindings that some participants did not realize how long they were on theirphone in the presence of others; thus, some additional awareness informationcould be useful to regulate behavior on one’s own.Figure 4.5: CoAware sending usage statistics, ©K. Khatra (2022)Close Request: Users can send a request to close the currently activeapp that their partner is interacting with. The partner sees a prompt suchas “May I request you to close Facebook?” or “I was hoping you couldclose Gmail. OK?” (Figure 4.6). The partner can cancel the request andcontinue using the app. If the partner agrees (tapping the Ok button), theapp shows a 30-second countdown timer. Rather than closing the currentlyrunning app instantly, the time is given to let the person finish the currentactivity [10]. When the 30 seconds are over, CoAware closes the currentlyrunning app. my goal here was to make the app closing somewhat gracefuland delayed and less of an immediate interruption. This reflects findingsfrom my survey where, again, some participants said that they might asktheir partner to change their behavior on their smartphone. I recognize thatcausing actions to occur on someone else’s phone may come across as beingstrong or overly assertive to some people. I wanted to explore this idea tosee how people would react to it in further studies.284.1. COAWARE FEATURESFigure 4.6: CoAware initiating a close request, ©K. Khatra (2022)Using the Summary tab, one can see the usage statistics for the apps.The summary includes the app/category name, the total time the app orcategory has been used and the number of launches since the current con-nection was established (Figure 4.7). If the access level App Name or ScreenShare is given by the partner, then one can see the app names. The accesslevel App Category only allows one to see the app categories. Using theDetails tab, one can see more detailed information about individual apps orapp category launches (depending on the access level), such as the name,launch time, and duration of use (Figure 4.8). Overall, I recognize that noteveryone will find the features I propose in CoAware to be useful. Somemay find them to be overbearing, some may find them to be not needed atall, and some may find them to suit their needs well. This is as expectedand purposeful such that I could explore my design ideas more and see whatreactions participants would give with a fully working system that providessuch options.Figure 4.7: CoAware showing a summary, ©K. Khatra (2022)294.2. IMPLEMENTATION DETAILSFigure 4.8: CoAware sharing detailed information about apps used since theconnection was established, ©K. Khatra (2022)4.2 Implementation DetailsCoAware was built on Android SDK 4.4 and leverages smartphones’WiFi Direct to establish peer-to-peer connections between two smartphones.I used WiFi direct as this technology allows two devices to connect di-rectly without requiring them to connect via Wi-Fi routers or wireless accesspoints, thus enabling sharing information between co-located users in anylocation (e.g., at home, park). Prior research has shown that smartphoneactivity can be shared with others by instrumenting the device (e.g., at-taching an additional display to the back of the device) [43] which can raiseprivacy concerns due to the visibility of private content in some commoncontexts (e.g., public places). Hence, I designed an application solution thatdoes not require any additional hardware instrumentation.30Chapter 5Study 2: Explorations ofCoAwareI conducted a study in a lab setting as an initial attempt to get feedbackfrom participants about CoAware. Note that the lab study was designed togain insights about the benefits, privacy risks and opportunities of CoAwareand generate directions to guide future researchers while designing apps toincrease activity awareness among co-located people. I investigated users’feedback on the three access levels for sharing app information amongst co-located partners and explored their opinions on the three notification strate-gies provided by CoAware. Additionally, I collected participant feedback onprivacy issues and on how CoAware creates awareness and I asked for gen-eral feedback on CoAware’s features. Naturally, I could have explored myideas using a field study where participants from various socio-cultural back-grounds could have tried out CoAware over a prolonged period of time. I didnot use this approach given that CoAware is still at an early design stage.Field studies bring the risk of participants not trying out all of the featureswithin a design. I felt it was more reasonable to gather initial participantfeedback such that the general ideas presented in CoAware could be assessedto understand which may hold the most merit. Then, either CoAware orother applications like it could be created and explored through longer-termusage. The caveat is that my study does not provide generalized resultsacross a range of real-world situations. I do acknowledge that a further fieldstudy is required to evaluate the app in the wild.5.1 Participants and Procedure:I recruited 22 participants (11 couples) from the local community (alarge city within North America) to participate in the study. Two partic-ipants were 18-24 years old, 11 were 25-34 years old, 4 were 35-44 yearsold, 2 participants were 45-54 years old, and 3 participants were 55-64 yearsold. All participants were smartphone users and have been in their partner315.2. RESULTSrelationship for an average of 9.5 (SD=5.7) years. None of them had expe-rience using tools to reduce smartphone usage or to support the awarenessof someone else’s smartphone use.I used two smartphones, Google Pixel 3 and Google Nexus 5, for thestudy. I first showed participants how to use CoAware. Next, participantswere given the following two tasks to complete:Establish a connection: One person (sender) sends a connection re-quest and the other person (receiver) accepts it.Access information: The sender accesses the app name, app category,and screen on the receiver’s device while the receiver 1) browses informa-tion on an e-commerce website (Amazon) to find a suitable camera costingless than $500, 2) finds a rumor/gossip about their favourite actor/actress,and 3) plays a game of their choosing. Once the tasks are completed, theparticipants switch roles as sender and receiver and repeat the tasks.I then used a questionnaire to collect their opinion on the access levelsand notification strategies to create co-located smartphone usage activityawareness, privacy concerns related to CoAware, and design suggestionsto improve the app. I asked participants close-ended questions using 5-point Likert scales regarding (Q1) the usefulness of the three access levelsin creating awareness about their partner’s smartphone activities, (Q2) theusefulness of the three notification strategies to motivate them in reducingco-located smartphone usage, (Q3) their comfort level when receiving a no-tification from their co-located partner (for each of the three notificationsstrategies). Additionally, they were asked open-ended questions about pri-vacy and awareness issues. In the end, I also asked to provide feedback andsuggestions regarding CoAware’s features. A session lasted approx. 40 min.in total. I used thematic analysis to analyze qualitative responses where Iiteratively reviewed the responses to look for main themes.5.2 Results(Q1) Figure 5.1a shows the mean rating on how useful the three accesslevels were to create awareness of partners’ activities. I found a mean ratingof 3.91 (SD=0.53) for App Name, 3.0 (SD=0.93) for App Category, and4.14 (SD=0.89) for Screen Share. I observed that Screen Share and AppCategory were rated more useful in creating an activity awareness than AppCategory. (Q2) Figure 5.1b shows the mean rating for the usefulness of threenotification strategies to motivate reducing co-located smartphone usage was4.18 (SD=1.05) for Message, 3.27 (SD=0.83) for Usage Statistics, and 2.55325.2. RESULTSFigure 5.1: Mean rating for (a) the usefulness of the access levels, (b) cre-ating activity awareness with notification strategies, and (c) the usefulnessof the notification strategies, ©K. Khatra (2022)(SD=1.18) for Close Request. (Q3) Figure 5.1c shows that participants weremore comfortable receiving a notification with Message (4.18, SD=1.05) andUsage Statistics (3.5, SD=0.9) than with Close Request (2.5, SD=1.3).In the open-ended questions about privacy and awareness issues, par-ticipants expressed that CoAware would be helpful to maintain their timecommitment to each other, create more smartphone usage awareness so thatthey do not interrupt their partner during an important ongoing activity,and that Screen Share would help them during co-located collaborative ac-tivities such as sharing information with others. “[CoAware] can be veryuseful for creating awareness as it allows us to check what other has beendoing, especially when he is on phone for a long time.” [P7, female, rela-tionship 5 years] || “The app will help when we want to show something toeach other but sitting different places in a room.” [P3, male, relationship 5years]Additionally, participants mentioned that there are other potential usecases for CoAware (e.g., sharing information with their partner, monitoringtheir children’s smartphone usage). “Sharing feature is useful as I can showphotos and videos to my wife; I can share the game that I am playing to myson.” [P18, male, relationship 13 years]Participants also expressed privacy concerns regarding the Screen Share.For instance, two females mentioned that they used some apps to track theirhealth-related issues which they might not feel comfortable sharing. Otherswanted to have a personal digital space away from their partner which theydid not want to be intruded in, as this might create stress and tension infamily life. “It will hamper my privacy, I may not feel comfortable at allfor sharing screens of my messages and emails” [P4, female, relationship 5years]335.2. RESULTSParticipants provided suggestions to improve CoAware. For example,instead of showing pop-ups, they suggested using standard notifications thatcommonly appear at the top of the screen. Six participants also suggestedthat partners should be allowed to only send a fixed number of notificationswithin a certain time (e.g., 10 notifications per day). Some participantswanted more notification styles and strategies or more statistics to bettermotivate the partner to engage with them. Two participants also felt thatinstead of sharing the entire screen with their partner, a blurred image orcustom screen area could be shared as this would protect privacy.34Chapter 6Design of Smartphone-basedappsI will now move to the second phase of my thesis which is focused aroundresults of the first phase and then comparing an updated version of thetext-based app with a smartphone agent-based app that plays a role incommunication facilitation between partners.6.1 Interpretation of ResultsBased on the results on Phase 1, I decided on two main factors thatneeded to be explored. The first being that participants found some fea-tures of the app to be too intrusive, most commonly screen share and screencontrol, and thus wanting them to be removed. I figured the easiest way tohandle this complaint by allowing app app name to be shared between cou-ples and disable any other type of information sharing on two smartphones.I hypothesize that this would not only solve this problem but also be a goodsolution to reduce any further issues that could arise. CoAware also didn’tfacilitate communication between couples as much as I desired it to. There-fore, I explored smartphone-based agents which are designed to interact witha human like a human and be effective for interventions and mediations suchas promoting physical activities, or to reduce psychological distress. Little isknown about leveraging smartphone-based agents to play a role in commu-nication and facilitate conversation between co-located individuals. Thus,I introduce a smartphone-based agent that acts as a conversation facilita-tor between co-located individuals. Following, I compare the effects of agentmediation on communication dynamics around co-located smartphone usagein couples in contrast to a text-based notification approach. At the time Iwasn’t fully aware of which direction the conversational agent may influencecommunication.356.2. DESIGN6.2 DesignThe goal of this phase is to explore the effects of agent-based mediationon communication dynamics among co-located couples. In addition, I wantto compare the results of agent-based mediation with a text-based approach,which I discuss below.6.2.1 Agent-based mediationAgent-based mediation involves a smartphone agent (i.e., an app) de-signed to facilitate conversation between individuals. More specifically, Idesigned the app simulating an agent playing a human role in exchanginginformation between couples. The agent is given a name (i.e., Ana) withan avatar. In addition, the agent can share information about couples’smartphone usage, such as how long they have been using smartphones andsuggest them to be engaged with their partners. When sharing information,the agent-based app reads messages out loud on users devices, in addition toshowing the same information with text notifications. Further, the messagesexchanged between the co-located individuals differ depending on the app.For instance, the agent-based app is more structured towards the mediatingagent providing the message. An example message includes “Your partnerwants to let you know that you have been using <appname>for X minutesand X seconds” which is sent with both voice and text. Thus, the agent-based system is designed with the ability to anthropomorphize and act as acommunication facilitator between partners.6.2.2 Text-based approachA text-based approach involves no facilitator or social entity interven-ing in conversation between individuals. This approach is tailored towardspartner-to-partner communication via standard text while supporting sim-ilar features that were developed for the agent-based system. For instance,users can send information about their partner’s smartphone usage whichshows on the partner’s device as a text notifications (“You used <app-name>for X minutes and X seconds”). Thus, there is no communicationfacilitation involved, other than what the individuals do themselves.6.2.3 Application DevelopmentI developed two smartphone applications (i.e., apps) to support a set offeatures for both agent and text-based approaches. I used the Android OS366.2. DESIGNplatform to develop the apps. The apps’ leverage Google’s Firebase RealtimeDatabase to connect and exchange information between two devices. I alsoused the database to store device-related information such as their locations(i.e., latitude/longitude) and the currently active app on the device withduration. I use an Android foreground app checker library [97] to access appusage-related information such as the currently running app with duration.Both apps store additional information, such as the time when an app islaunched, when a connection request is sent to establish communicationbetween devices, when users exchange information with each other (e.g.,ping their partner or share usage stats), and when they end the connectionbetween two devices. Note that there are other existing approaches, suchas using Bluetooth or WiFi direct for exchanging information between twodevices. However, I decided to use Firebase Realtime Database as it offersan efficient solution while causing minimal connection-related issues [1].Establish ConnectionTo start exchanging information, one device first sends a connectionrequest to another device (Figure 6.1). As I intend only to share informationFigure 6.1: Upon clicking the play button, the background service starts,©K. Khatra (2022)while the two devices are co-located, I used distance-based co-location (i.e.,two devices within 100m) which is also used in previous studies [30]. Oncea connection request is initiated from a device, the app checks the FirebaseDatabase to calculate whether the other device is within 100 meters andsends a connection request to the other device accordingly. The devicereceiving the connection request then shows the request as a notification(Figure 6.2).376.2. DESIGNFigure 6.2: Connection request text in the text-based app, ©K. Khatra(2022)Figure 6.3: Connection request text in the agent-based app, ©K. Khatra(2022)A user can then either accept or reject the request to share information.Once a partner accepts a connection request, the app displays a connectionstatus Permission given: Yes at the bottom of the screen, indicating thattheir partner has given them permission to exchange information via theapp (Figure 6.4).Figure 6.4: Currently running app name and duration, ©K. Khatra (2022)In addition, both apps show whether two devices are within the 100-meter Within Range: Yes at the bottom of the screen. Once the connection386.2. DESIGNrequest is accepted, the sending device can now see the receiver’s devicecurrently used app name as well as the duration it has been running for(Figure 6.4). At the bottom of the screen, the app will show whether or notthe other device has given them permission to access the information as wellas notify them if they are within range of one another (Figure 6.5).Figure 6.5: Shows permission given and if two phones are within range,©K.Khatra (2022)In addition, with the agent-based app, once a user sends a connectionrequest, the receiver prompts with a voice command asking if they are in-terested in knowing what their partner is doing now (Figure 6.6).Figure 6.6: Agent-based app: an agent asks if users are interested in knowingwhat their partner is doing, ©K. Khatra (2022)This is different from the text-based app, where the user has to manuallyclick on a button to know what their partner is doing. A user can termi-nate an ongoing connection between two devices by pressing the “Stop LiveUpdates” button on the user interface. Once the connection is stopped, theapp also displays a connection status “Permission given: No” at the bottomof the screen, indicating that their partner has stopped sharing information(Figure 6.7 - text-based app , Figure 6.8 - agent-based app).396.2. DESIGNFigure 6.7: Text-based home screen after partner has stopped sharing, ©K.Khatra (2022)In addition to the functionalities discussed above, I designed the follow-ing two features:Usage StatisticsBoth apps allow users to send information about their partners’ smart-phone usage (i.e., the time that their partners have spent on an app) viasmartphone notifications, though, this feature differs slightly for the textand agent-based apps. As mentioned before, the text-based app is centeredaround a direct conversation between couples via text, whereas the agent-based app is centered around the agent mediating the conversation.Ping PartnerAnother feature of the apps is the option to ping their partner with apreset message. I designed a set of preset messages to enable fast com-munication between couples without requiring them to type. Text-basedmessages are more styled for day-to-day communication between couplesvia text. Example message includes: “It feels like you’ve been using ¡app-name¿ for a while now. Can we talk instead?”. For the agent-based app,the messages were intended to give couples the impression that the medi-ating agent is taking charge of the conversation. An example message forthe agent-based app includes: “You have been using ¡appname¿ for a while.Want to talk to your partner instead”. As mentioned previously, both mes-sages are sent as notifications; however, agent-based messages use audio toread the text aloud.406.2. DESIGNFigure 6.8: Agent-based home screen after partner has stopped sharing,©K. Khatra (2022)41Chapter 7Study 3: Explorations ofAgent-based and Text-basedAppsI conducted a between-subject design study exploring the effects of agentmediation on communication dynamics around co-located smartphone usageand compared the results to the text-based approach.7.1 ParticipantsTwenty-four participants (i.e., 12 couples) were recruited from the localcommunity (a large city in North America) to participate in the study. Ionly recruited participants that used Android phones, and commonly wereco-located with their partner after work or weekends. Each couple wascompensated with $100 for their participation.Two participants were 19 years old, 13 were 20-30 years old, eight were31-40 years old, and one was 41-50 years old, with an average age of allparticipants being 28.8 (SD=6.0) years old . All couples had been in a rela-tionship for at least two years, with the average length being 4.8 (SD=2.8)years. They reported spending an average of 7.5 hours (SD=1.5 hours) co-located with their partner per day, with most of them mentioning that theyspent more time together on the weekends. In addition, they reported own-ing a smartphone for an average of 9.8 (SD=3.3) years while spending anaverage of 4.4 (SD=1.9) hours per day on their smartphone.7.2 ProcedurePeople who had expressed interest in participating in the study receiveda consent form which included study details. Once they signed the consentform and returned it to one of the authors, the first meetings were sched-uled. Due to the Covid-19, I scheduled the meeting online via Zoom, where I427.3. RESULTSmet each couple individually (i.e., two people in a group at the same time).Six couples were randomly selected to use the agent-based app while theremaining six couples used the text-based app. During the meeting, I ex-plained the study goals, installed the app, and showed how it worked. Theparticipants were asked to use the app for seven days. During this time, theapp automatically logged user interactions and activities with the app onthe Google’s Firebase Realtime Database. After seven days, I set up anotherZoom meeting to conduct an exit interview with each couple individually. Iused a set of interview questions focused on three main areas: i) questionsrelated to general couple relationships (e.g., time spent together), ii) expe-rience of using the app, and iii) the effects of the app on their co-locatedsmartphone usage during the study period. All of these questions were askedas open-ended questions, with follow up questions being asked in case moreinformation was needed. Each interview lasted approximately 30 minutes.7.3 ResultsI used thematic analysis to analyze qualitative responses, where I itera-tively reviewed the responses to look for main themes. The iterative processwas done as follows: analyzed the text-based interviews and agent-basedinterviews separately, looking for any major themes that existed in the an-swers; viewing the themes as a whole between the two and then generatea major theme list that would be used to compare the two sets of results;finally, looking at the results once again for the text-based and agent-basedapps with the major themes in mind and identifying sub-themes that existedwithin them, which are to be used as the comparison tool in this study. Tworesearchers performed the coding, followed by redefining them in a consul-tation with the entire group (three researchers) involved in the study. Inaddition, I report on quantitative data collected during the study period.I first looked at couples’ responses to a few preliminary questions relatedto smartphone usage in co-located situations. I found that they expressedconcerns about their partner’s co-located smartphone usage, which was alsoreported in prior studies [30, 69]. Six out of twelve couples stated that theirpartner had previously complained about using smartphones in co-locatedsituations. Four couples mentioned that these cases primarily occurred rightafter work or before bedtime. One participant stated: “Many times, rightafter work, my partner will be on their smartphone for a long time, whichcreated frustrations among us.” [C1, Partner 1, relationship 6 years] Theyalso mentioned that when using phones for a long time, they occasionally437.3. RESULTSrequested others to stop using their phone and engage in collaborative ac-tivities. I next explore the effects on agent and text-based systems on theircommunication dynamics.The following sections report on couples’ responses that are groupedinto three main themes: Effect on Interpersonal Relationship, Effects onPhone Usage Patterns, and Effects on Verification Behavior Patterns whichI discuss below. In addition, I report on the privacy considerations andother concerns regarding the apps that participants shared with us.7.3.1 Effect on Interpersonal RelationshipI asked participants questions on how the app promoted co-located inter-actions, and how they used the app to be aware of their partners activities.Thus, aiming to explore how the app effected the partner’s relationship dy-namic. In the next section, I will first discuss the results that I observed forthe text-based app and then I introduce the effect that was observed withthe agent-based app.Text-Based AppThe text-based app had 4 major sub-themes that I observed as hav-ing an effect on couples relationship: Co-located Usage Awareness, Timeand Focus Management, Building Connections, and Starting and MediatingConversations.Co-located Usage Awareness: Participants mentioned that being awareof what each other were doing on smartphones had an effect on how theyinteracted among one another. The effect was seen in: (i) partners’ appusage awareness, and (ii) their own smartphone usage. First, the app allowedthem to know what their partner was doing on their smartphone and followup with them, when necessary. One participant commented “I was worriedabout my partner being on the phone for long, and it was nice to check andsee what they are doing.”[C8, Partner 1, relationship 2 years] Secondly, beingaware of what each other were doing on their smartphone also had an effecton participants themselves. They expressed that they were more aware thattheir partner would be checking up on them and were more cautious abouttheir phone usage. “I was worried throughout the day because I told her Iwould be busy... I start using WhatsApp at certain points and she sees thatand sends a notification and I get worried.” [C1, Partner 1, relationship 6years]Time Management : Participant’s expressed that the app helped them447.3. RESULTSto manage their time better in different ways such as engaging with theirpartner more often, and be more on-time. For instance, the app allowedtheir partner to trigger notifications when they were using phones for a longtime. This eventually helped them to stop using their phones and engagewith their partner. One participant mentioned: “ I was co-located with herand doing something on my phone, and I will get a notification... I needto give her some of my time as well.” [C1, Partner 2, relationship 6 years]In addition, the app also helped them to manage their time better in dailyactivities and be on time. Two couples mentioned the app helped them stayon-time with activities they were supposed to do together. “I was takingtoo long to get ready as I was busy with my phone, and my partner pingedme through the app. I felt that I had to hurry up as we had somewhere togo.”[C9, Partner 2, relationship 5 years]Building Connections: The app had an effect on strengthening couplesrelationships by allowing them to connect easily via smartphones - whichare not not present on other apps. In addition, the app provides them withusage stats notifications - which help them to be aware of others activitiesand connect with them accordingly . For instance, participants mentionedthat the app was a nice compliment to building connection as it allowedthem to connect more easily which is commonly not offered in other apps.“It was nice to keep track of her and seeing what she was doing on her phoneand then I could send a message to link up later which helped to improvethe connection that we already had.” [C1, Partner 2, relationship 6 years]In addition, the usage statistic’s notification was also found to be a valuableasset in building this relationship by providing one another with time spentthey spend on their smartphone. This helps them to be more connectedwith each other by triggering them to do more collaborative activities. “Mypartner spends a lot of time on the phone... I sent him a usage statisticsnotification to stop what he is doing and go shopping or watch a movie.”[C1, Partner 1, relationship 6 years]Starting and Mediating Conversations: Participants voiced that the appacted as an entry point to starting a conversation and mediating it by usingthe features of the app. They mentioned that the app allowed them toformulate queries about their partners smartphone usage which they couldfurther use to mediate conversations with their partner. One participantcommented, “My partner was listening to music and reaching out and beinglike, oh, you’re listening to music. What song are you listening to? Andhaving a conversation about that.” [C12, Partner 2, relationship 2.5 years]457.3. RESULTSAgent-Based AppSimilar to the text-based app, the agent-based app had an effect oncouples’ relationship. However, I observed two new sub-themes that werenot seen in the text-based app: (i) participants perception of the agentas a third party and (ii) the agent’s role for mediation in communicationbetween the couples. First, I will look at two sub-themes that were similarthen discuss the two that were different.Co-located Usage Awareness: Similar to the text-based app, the agent-based app had an effect on promoting app usage awareness. The app helpedparticipants to be more aware of what their partners were doing on theirphones. One participant mentioned “It does really promote co-located in-teraction because instead of me asking my partner each time, the app doesinstead and brings more attention to the app usage.” [C6, Partner 2, rela-tionship 4.5 years] Furthermore, by providing more co-located smartphoneusage awareness between one another the agent-based app helped in individ-ual’s relationship dynamics. Participants mentioned that the app helped toreduce any aggravation between couples. “There has been a lot less pent-upaggravation due to the app being used. We can monitor each others smart-phone usage through that and not have to constantly ask each other.” [C3,Partner 2, relationship 2 years]Starting and Mediating Conversation: The agent-based app also servedas an entry point into conversations among couples. Three couples men-tioned that the app helped them find ice breakers for their conversation aswell as serving a reminder to engage in conversation with their partner. Oneparticipant mentioned: “The app tells you to stop whatever you are doingand focus on what is important. I am giving my partner less attention - thusI need to pay her more attention.” [C6, Partner 2, relationship 4.5 years]Agent as a Third Party : Unlike the text-based app, couples viewed theagent as a third party mediating their relationship. Interestingly, I observedthat participants received the agent as a third party both positively andnegatively. A few couples viewed the agent positively where they consideredthe agent as another social entity mediating conversations among them.They expressed that the agent-version was not just a basic app sending themnotifications, it is a third party, conveying messages about their partnersintent to them. A participant explained ”I really liked that it was a thirdparty, it wasn’t just the app saying, ’oh hi I want to spend time with you’,it was your partner.”[C3, Partner 2, relationship 2 years] On the contrast,two participants viewed the agent negatively where they thought that itwas unorthodox for a third party to invade their relationship. Instead, they467.3. RESULTSwanted to connect with their partner directly (e.g., via texting) which theyconsidered being more effective. One participant mentioned “It was a littleweird for a third person to kind of make that happen. direct texting wouldbe more efficient.”[C11, Partner 2, relationship 2 years].Mediation Support of Communication Request : The second effect thatdiffered in comparison to the text-based app was that the agent was viewedas providing mediating support in communication between couples. Thesupport was seen in two different ways: partners talking more freely withone another and agent reading notifications for them out loud. Firstly, thiscommunication support enabled participants to talk more with each otheras it wasn’t them directly sending the other person a notification it wasthe “third party” (i.e., agent) who was sending notifications. Two couple’scommented that they often get annoyed when they send notifications totheir partner and enjoyed how the third party agent kind of took this outof their hands. One participant explained “I get really annoyed when mypartner asks me to hangout at certain times, and when I get the popupthat your partner wants to hangout, it was nice that it was coming fromsomeone else.”[C3, Partner 2, relationship 2 years] From the logged data, Iobserved that the agent-based app pinged couples 50 times and sent usagestats 45 times. Meanwhile, for the text-based app pinged couples 176 timesand usage stats 71 times. Thus, contributing further to the fact the agent-based app was used less to send notifications as it wasn’t needed as much.Furthermore, the agent-based app also provided a entry into conversation incouples as the notification style was welcomed. Participants explained theyliked how the app reads notifications out loud for them. “...to have an appthat like makes this statement like I’ve been seeing you on your phone for awhile, do you want to hang out with me? - it removed a sense of judgmentfrom my partner.”[C3, Partner 1, relationship 2 years]7.3.2 Effects on Phone Usage PatternsI asked participants questions on how the app changed their own be-havioural patterns throughout the course of the study, aiming to explorehow the app effected their behavior before taking part in the study andduring it. Throughout the course of the study, participants observed thatthe app had an effect on their phone usage behaviours. These includedimmediate behavior changes in their daily life routine.477.3. RESULTSText Based AppIn the text-based app, two of the behavior changes induced included asense of guilt, and concentration.Guilt : Participants, in general, expressed that the app triggered feelingsof guilt during the study period for their smartphone usage behavior. Theapp had features, such as ping partners and usage stats, which made thembe aware of their smartphone over usage, especially in co-located situations.Additionally, they mentioned that they were aware that their partner couldmonitor their smartphone usage through the app - which eventually affectedtheir behavior. Four participants said they had felt a sense of guilt whenthey weren’t engaged with their partner. “The notification made me feelguilty when I wasnt hanging out with my partner.” [C9, Partner 2, rela-tionship 5 years] Next, one participant mentioned they noticed a sense ofguiltiness as they felt as though they were being spied on. Thus, they tookit upon themselves to find more times to change what they were doing tospend time with their partner.Concentration: The app served as a vessel for participants to be focusedon tasks they needed to do without being distracted by their phone. Thus,the app allowed them to be more productive and helped them to be moreorganized. For instance, one participant mentioned “Maybe you’ve spent alot of time on the phone, then you get a notification...I’ve spent a long timedoing online searching and it’s time for me to get back to work.” [C1, Part-ner 2, 6 years] Another participant noted that the usage stats notificationplayed a role in them remembering their plans for the day. “At points I real-ized that I spent an hour my phone and she is just sitting next to me. And,when I receive the notification, I remember we have plans to go shopping...and stop what I am doing and refocus on whatever I am supposed to do withmy partner.” [C1, Partner 2, 6 years]Agent Based AppIn accordance with the text-based app, the agent-based app also servedto help participants concentrate on tasks at hand. However, I observed adifferent theme that was not seen in the text-based app - participants feltmore inclined to reduce their smartphone usage while using the agent-basedapp.Concentration: Similar to the text-based app, participants viewed the487.3. RESULTSapp as a vessel to be more focused on tasks needed to be completed. Thisis primarily due to the notifications that they received through the appwhich eventually encouraged them to be focus on activities that they shoulddo. For example, one participant stated “The app kept reminding us to getour focus back on doing things that are important because we found our weare spending too much time on our phones and less between ourselves.”[C6,Partner 1, relationship 4.5 years]Reducing Smartphone Usage Participants using the agent-based appmentioned that the app served as a factor in reducing their smartphoneusage over the course of the study. The app allowed couples to see each oth-ers smartphone activities and allowed them to send notifications if needed -which eventually served as a reminder to reduce smartphone usage in manysituations (e.g., meal time). One participant said “Previously I had anxietywhen my phone wasnt near me. Throughout the study I kept my phone atleast a hands length apart from me.”[C3, Partner 1, relationship 2 years].Additionally, during meal times participants were encouraging themselveseven more to put down their smartphone. “If we are having breakfast to-gether, I have been trying to put the phone face down away from me.”[C3,Partner 2, relationship 2 years] This is also supported from the logged datawhere I found that the agent app was opened 312 times while the text-basedapp was used 595 times, thus invoking less situations with the agent-basedapp to check on their partners’ smartphone usage.7.3.3 Effects on Verification Behavior PatternsI was interested to explore how the verifying features (i.e., being ableto see how long and what their partner had been using an app) of theapp effected their smartphone usage activities. Thus, I asked participantsquestions on how they used the app to check their partners smartphoneusage.Text-Based AppParticipants using the text-based app expressed that it helped them beaware of what their partner was doing, contribute to limiting their own self-usage, and validate the smartphone usage of their partner.Attentional Awareness Participants stated that they were more atten-tionally aware of what their partner was doing during the study. In contrastto before using the app, the app brought a whole new level of smartphone497.3. RESULTSactivity awareness amongst one another in the relationship. A participantmentioned “The app brought more attention to me to see what my partnerwas doing on his phone throughout the day.” [C9, Partner 1, relationship 5years]. Couples also specified they would use the app to try and pry awaytheir partner from being glued to their phones by sending a notification tothem instead of saying it out loud.Limiting Self-Usage I also observed that the usage stats notification ofthe app contributed to participants to limit their own smartphone usage.After receiving a usage stats notification, participants would often stop us-ing their phone right away whilst also reaching out to their partner.“I likehow the app is limiting your time. Maybe you have spent a lot of time onWhatsApp or a different app it will alert you that you have spent a lot oftime on this.” [C1, Partner 1, relationship 6 years]Validating Usage of Each Other Participants also used the app to val-idate what their partner said they had been doing in two different ways:(i) to check if their partner was being truthful about their smartphone us-age and (ii) to form a closer bond between them. Oftentimes, participantswould check the app immediately after their partner had told them whatthey had been doing. One participant explained“Is she lying to me becauseshe told me she was doing a different thing then I check the app and she isdoing something else?” [C1, Partner 2, relationship 6 years] Also, anotherparticipant mentioned that the app usage feature can form a closer bondbetween the two because of the ability to determine the truth.Agent-Based AppI observed the verifying behaviors that the app served was mainly thesame across the text-based app and agent-based app. Both contributed tobeing more aware of what each other were doing and also limiting their ownusage. The minor difference lied in the fact of verifying each others usage.Instead of having the app ask out loud what their partner was doing, twoparticipants stated they would rather just ask them in person themselves,and thus preferred direct conversations.Attentional Awareness As was the case with the text-based app, partic-ipants communicated that the agent-based app brought more attention towhat each partner was doing on their phones. For example, a participantexplained “Instead of me asking my partner each time they are using theirphone the app helps to do it instead and brings more attention to the app507.3. RESULTSusage.” [C6, Partner 2, relationship 4.5 years]Limiting Self-Usage Participants also viewed the agent-based usage statsnotification to help limit their smartphone usage. Participants mentionedthe usage tracking feature of the app contributed to this limitation. “Meand my partner can keep track of how much time we spend on the phone andstop using it accordingly.” [C6, Partner 1, relationship 4.5 years]Direct Conversations Two participants using the agent-based app ex-plained they would rather ask the person out loud what they had been do-ing instead of the agent saying it for them. Both participants clarified thatthis was due to them having a very close relationship, where they aren’tconcerned of what each other are doing on their smartphone. They men-tioned they found that the validating factor behind the agent was weird.An example of this was a participant mentioned “Id rather just use the sen-tence via my tongue instead of having a third person say it.”[C5, Partner1, relationship 7 years], In terms of trying to check what their partner wasdoing another participant mentioned “If I want to know what he is doing onhis phone I would rather just ask him.”[C5, Partner 2, relationship 7 years]However, there were participants that mentioned the validation of usage viachecking the app was beneficial to them as was the case in the text-basedversion.7.3.4 Privacy ConsiderationsI was interested in exploring what features of the app could cause privacy-related concerns and how this could be changed in the future. Thus, I askedparticipants questions about how comfortable they felt sending and receivingnotifications using the app, their concerns over how the app handled data,and any privacy concerns. I found that participants were generally comfort-able with how the notifications were sent while showing no concerns aboutthe app using location data to determine their co-location. In addition,participants were generally not concerned about the privacy aspect, as theapp only tracked standard data such as the name of the app they are usingwith duration and didn’t go into too much detail (e.g., app content). Oneparticipant mentioned “It doesn’t take sensitive information like passwordsor cards and therefore I felt safe using it.”[C12, Partner 2, relationship 2.5years] For the trust aspect, one participant mentioned they weren’t hidinganything from their partner so they were fine with sharing their smartphoneusage the information.517.3. RESULTS7.3.5 OthersI asked participants if they would like a fully automated AI agent inthe future, covering more features such as real-time notifications to coupleson their app usage in co-located and non-co-located contexts. Participantssaid they are open to an AI agent that intrudes in their activities and re-lationships to a certain extent, specifically, not crossing a boundary. Forinstance, one participant expressed that AI agents could potentially abusenotifications by sending them frequent messages when they use smartphonesfor longer. Another interesting result I found was that the hearing the voiceout loud startled one of the participants in contrast to a simple notification.One couple even mentioned that they were so zoned in on their phone, theycouldn’t even hear their partner talk but once the agent voice notificationcame thru they regained focused and talked with their partner.52Chapter 8Discussion and DesignRecommendationsI will first discuss discussion and design recommendations for Phase 1 ofthis study. Through my studies, I was able to uncover a range of ways thatapplications like CoAware should be designed, based on its strengths andweaknesses.Ensuring personal digital space: The participants’ lower comfortratings when using Screen Share illustrate that there is often a personaldigital space among partners which they typically want to preserve. Thus,I suggest that applications like CoAware should focus on sharing higher-level or more abstract information (e.g., the app name) instead of sharingvery detailed information such as the screen content. Low-level informationakin to what I provided as one feature within CoAware is likely too muchfor many people and could inadvertently create greater tensions betweenpartners.Level of access: Participants expressed concerns about using the CloseRequest feature within CoAware as it takes some control over a partner’sphone and may disrupt their on-going activities. This, again, could createfurther tension between partners. Instead, participants felt that solutionsthat alert others of what they may want to change in their own behavior,rather than take control, would be more acceptable. Thus, when design-ing apps for communication between devices, it is important to carefullyconsider how much control one should have over another person’s device.Notification Strategies: Participants found pop-up notifications tobe distracting and somewhat overbearing. Thus, I suggest that applicationslike CoAware use standard notification mechanisms that are already foundon smartphones. Pop-ups notifications can be distracting and the forcedchange in activity (e.g., switching from a game to notification UI) couldcreate new frustrations.Determining whether the phone is in use: Sometimes a phone mayremain active although the user may not be engaged with it. This meansthat usage information provided to one’s partner may not be accurate. One53possibility could be to rely on information about the user’s on-screen taps incombination with information about the running apps to determine usagehistory.Study 1 of Phase 1 provided insights and direction for my design workwith CoAware and helped lead to the aforementioned design suggestions. Itsresults also moved beyond prior literature (e.g., [69]) to allow me to moredeeply understand when, where, and during what activities a system likeCoAware would potentially be used in real-world situations. This can helpguide future studies that want to test applications similar to CoAware toknow when, where, and how such testing should be done. One could alsoimagine using Study 1’s data on locations and context to think about waysto further refine applications such as CoAware. For example, users could begiven options to customize applications so that they are able to choose whattypes of information they are comfortable revealing to their partner basedon location, time, and activity. Such information could also be inferred byapplications and then adjusted as needed by users.I also recognize that there is a darker side to applications like CoAwareand designers should be cautious in this regard. The challenge with appsthat track or share mobile device usage between partners is that they canpotentially alter relationship dynamics, given an increased access to infor-mation [22]. This could create issues around trust or control between part-ners. While my results did not reveal such concerns, they are most certainlypossible. Further research is required to understand partners informationsharing behaviour with others and its impact on their relationship. I alsoacknowledge that apps with features like CoAware could be seen as beinghighly problematic for relationships that contain domestic abuse or familyviolence [23]. As apps like CoAware enable access to information on part-ners’ devices, this could lead to the system being misused (e.g., coercing toshare information constantly, surveilling one’s partner) and create anxietyand tensions within a relationship. There are no easy solutions for suchtypes of situations. CoAware could, for example, ask users for details abouttheir relationship satisfaction before making features available to them. Yetpartners in an abusive relationship could easily answer untruthfully. Appscould track couples’ information sharing behaviours and provide warnings ifacts that appear to resemble surveillance occur, or features could be turnedoff based on certain negative behaviours. However, this may, again, not bea complete solution and may be hard to detect. As such, designers need tobe cautious to think about the possible negative consequences of apps withfeatures similar to CoAware.I will now move into my discussion on Phase 2 as well as my design54recommendations after the development of my agent-based app.My results suggest that the agent-based mediation was successful in re-ducing smartphone usage. Over the course of the study participants men-tioned they changed their smartphone usage behavior in comparison to be-fore. Their smartphone usage behavior differed in reducing their usage atmeal times, keeping their smartphones a hand’s length away, and findingtimes to put their smartphone away and instead interact with their part-ner. Additionally, the smartphone agent made participants more inclined tostart focusing on tasks, such as work, and connecting with each other. Theaudio feedback helped couples to change what they were doing immediatelyas they would get startled from the notification. Not only could the agentcontribute in this way, but as the agent voiced messages out loud, peoplecould feel embarrassed when the notification would be spoken thus leadingthem to not use their phone as much.The smartphone agent also acted as a mediating support of communi-cation between couples and improved co-located interactions. By acting asa intermediary channel between co-located individuals, they felt as thoughthey could talk more freely with one another. Thus, invoking more con-versations because the agent would hypothetically be the one initiating theconversation and not the couples themselves, as the agent would voice outthe message out loud. Hence, helping to reduce any complaints that wouldcome from their partner for smartphone overuse. Furthermore, as the noti-fications are preset messages, it removes any complications that may occurin normal messaging, such as an unwanted tone of a message. In addition,the agent was perceived as a third person or another social entity in theinteraction between two co-located individuals. To encore, all these aspectswere observed from a mediating agent that simulated playing a third partyin exchanging information between couples. Thus, it would be interestingto see how future research can leverage more features for the smartphoneagent to further develop this mediating aspect.I do also acknowledge that there are potential negatives to having asmartphone agent involved in conversations between couples. Two partic-ipants mentioned they thought the third party agent was invading theirrelationship at times and would rather just use normal text messaging. Theparticipants did clarify however, this was due to them having a close knit re-lationship with one another and don’t believe having the smartphone agentis necessary for them.From my study results, I offer the following design recommendations forfuture researchers:Smartphone Agent Behavior Participants expressed that the smart-55phone agent acted as a social entity in their conversation and mimickedhuman capabilities. Designers should consider leveraging agents for otherdomains (e.g., group chat messaging), knowing that people view it as a an-other social entity in the communication between one another. Also, theagent-based app enticed users to focus more on tasks and each other, incomparison to the text-based. Designers can use the agent to convey moreimportant messages (e.g., tasks to be done at work), which is a result of theagent-based app enticing users to feel more inclined to read/listen to themin contrast to a simple notification seen in many existing apps.Enable Awareness Participants mentioned that being aware of whateach other were doing contributed to how they interacted while co-located.Most of this awareness stemmed from checking the usage statistics of theapp. Developers should consider this feature to be integrated in future ren-ditions of this app or other smartphone information sharing apps, to enhanceconversation amongst co-located individuals. Additionally, this awarenessfactor also played a part in one individual’s mindset as they were more at-tentionally aware of what they were doing on their smartphone at all times.Thus, designers could use this aspect in apps that aim to reduce smartphoneusage for individuals.Assisting in Time Management Participants specified the apps as-sisted them in their own time management. Future OS or applications couldbe designed to help users manage their time better. For example, connectingthis app with a calendar app could allow for further integration’s of provid-ing more valuable notifications. Per say, if an event is schedule with familymembers at a certain time and I am still using my phone, I could receive anotification letting me know that I have been on my phone for too long andreminding me to go to the event. Also, the time management aspect couldbe useful for study tracking apps, where users are confined to only use theirphone for a certain period of time until the app triggers a notification toremind them to stop.Help Mediating Conversations/Communication Participants men-tioned that both sets of apps (text and agent-based) contributed to startingand mediating conversation. Designers could leverage these aspects in vari-ous apps (e.g., dating or messaging apps). The app could use an AI basedsystem to see how much communication I have with my family membersand then remind me to connect with people if I haven’t talked to them ina long time. The app could even offer helpful insights to further mediatethe conversation by coming up with preset messages to send knowing thatfrom my study results these helped mediate the conversation between co-located individuals. Furthermore, initiating conversations can be oftentimes56difficult and designers could leverage this app aspect to give helpful startingconversation starters.57Chapter 9Limitations and Future WorkI will first go over limitations and future work of the first phase of mystudy.My crowdsourced survey has some inherent limitations. Since I usedAMT, my participant’s demographics were determined by the demograph-ics (USA and India) of the ATM. With a larger sample and with participantsfrom more cultures, it is possible to investigate how people’s perceptions ofco-located phone usage differ between cultures. It would also be interestingto conduct an in-person study with interviews to determine whether theresults differ from those that have been obtained from crowdsourced study.In an in-person study, I would be able to see how people use the design forreal-life situations which may not completely match the tasks in my studyand essential to ground and guide the developments of CoAware. This wouldfurther help me cover ethical aspects of smartphone activity-awareness be-tween couples which might be missed in my online survey. The challengesand potential problems that may arise with an increased awareness, espe-cially in abusive and problematic relationships, needs further investigation.I believe such future research would provide me with important insights intothe scope and impact (both positive and negative) of using CoAware andsimilar technological solutions in sensitive family situations.I concentrated on partners’ co-located smartphone activity awarenessand investigated their opinions on CoAware. However, I envision extendingmy approach with CoAware to other relationship types, such as betweenparents and children, where the parent could use CoAware to monitor andcontrol the child’s smartphone activities. This would require an in-depthstudy of parent-children relationships and the consideration of many otheraspects, such as the diversity of house rules, family traditions of raisingchildren, child age, and the educational backgrounds of parents.We do acknowledge that WiFi direct poses issues such as eavesdroppingas a user with an Android smartphone can connect to another Androidsmartphone through WiFi direct. For instance, a user located within 100musing the same app (i.e., CoAware) can connect their phones and listento the data being sent. Thus, using other solutions such as, for example,58Bluetooth could be a potential solution to be used instead of using WiFidirect.With my initial encouraging results and reactions, I plan to further de-velop CoAware and to perform a longitudinal study with a full-fledged ver-sion to examine its effect on sustained behavior change among people. Ad-ditionally, CoAware inspires the design of context-aware smartphones wherethe solution can trigger notifications based on preset rules based on loca-tions and any surrounding people to reduce the smartphone use of co-locatedpersons.Next, I move onto limitations and future work of the second phase of mystudy.My study has some inherent limitations, which I discuss below. I con-ducted the study with 12 couples from Canada which may influence thesmartphone usage patterns I saw. Depending on what societal backgroundpeople come from, they can either be more inclined to use their smartphoneswhen around each other or have strict rules in place to reduce this behav-ior. Consequently, it would be interesting to run a further study to see theresults with a larger and more diverse set of couples with different ages,backgrounds, and relationship lengths. As with any research study takingplace over the past couple of years, Covid-19 could play a role in participantsresponses. When everything returns back to normal, it would be interestingto see whether the results would remain the same.I used distance-based co-location for this study, where I considered cou-ples to be co-located when they are within 100 metres of one another. How-ever, other forms of co-location could be explored. For example, two userscan be considered as co-located when they are within their view or sight. Ineed advanced vision-based solutions on smartphones (e.g., capturing 360-degree views) to implement such co-location - which can be a potential futurework. Virtual Private Networks (VPN) can also be exploited in this systemas individuals may be able to disguise their location and be someone theyare not. In other words, the app sees them as being co-located when theyare not.Very little was known about smartphone agents exchanging messages andnotifications between people. Thus, I envision further developing my agentto include more features. Developing an AI-based agent that allows cus-tomizable parameters to be set to the AI to accurately detect when to sendmessages to the users could also be added. Interacting with the smartphoneagent through voice would be a potential extension for the future. Anotherstudy could be conducted to determine the best way to send notificationsthrough an agent, such as breakpoint-based notification strategies where no-59tifications are only sent by identifying breakpoints at more opportune times[71]. It would be interesting to allow users to send custom messages and seehow it influences communications between them. Additionally, includinggamification techniques on the apps can be a potential venue for future re-search. For instance, we can investigate reward-based solutions where userscan get some redeemable points once they do not use the smartphones fora certain amount of time.For Study 3, I mainly focused on qualitative data. However, furtherstudies can be conducted focusing on collecting quantitative data. It wouldbe interesting to see how the app affected participants’ smartphone usagein pre/post-study conditions. We can possibly run studies for a longer timewhere we spend one week collecting users’ smartphone usage data withoutany agent or text-based app (pre-condition). Then one of the apps (i.e.,agent or text) can be used for one week to see participants smartphoneusage patterns (post-study). Having such data would allow us to directlycompare users’ smartphone usage.60Chapter 10ConclusionIn this thesis, I aimed to design a solution to improve co-located com-munication between partners. This was done by splitting my work into twophases.The first phase consisted of conducting an initial crowd-sourced study(via AMT) to investigate people’s smartphone usage, rules, trust and privacyconcerns, and activity awareness when they use smartphones in the presenceof their partner. Results showed that people often use smartphones whileco-located with the partner, which sometimes leads to anger and frustration,and in addition partner’s tend to be untruthful of what app they are usingat sometimes, thus being unaware of what each other are doing.This motivated me to design a smartphone app, Co-located Awareness(CoAware), intended for sharing smartphone usage information between co-located partners. CoAware was designed to share users’ smartphone usageinformation such as the number of times an app has been launched, the du-ration it has been used for, and the time it was initially opened. I continuedwith an in-lab study with couples who explored and provided feedback onthe features offered by CoAware. Results showed that low-level informationwas seen to be unnecessary and too much in comparison to the high-levelinformation that was seen to be useful. Additionally, Co-Aware didn’t fullygrasp the communication facilitation aspect that was desired to be seenbetween couples.This first phase led us with valuable information to develop on for mysecond phase. The main two changes focused around only sharing app namebetween partners’ and developing a brand new smartphone agent-based appthat could be compared to a text-based application that technically wasan updated version of Co-Aware. The lone study of Phase 2 starting bydesigning these two versions of the apps (text-based and agent-based) andthen conducting a user study with these. The design of the two smartphoneapps were as follows: (a) an agent-based app that exchanges smartphoneusage activities (e.g., app name, app usage duration) among couples whilemimicking a virtual agent presenting the information to them. The agentwas represented with a name, Anna, an avatar, allowing her to leverage61voice modality to send messages and notifications to partners via audio,and (b) a text-based app where a person can access and exchange the samefeatures manually. The text-based approach involves no facilitator or socialentity intervening in conversation between individuals. This approach istailored towards partner-to-partner communication via standard text. Theagent-based approach involves a smartphone agent (i.e., an app) designedto facilitate conversation between individuals. More specifically, I designedthe app simulating an agent playing a human role in exchanging informationbetween couples. Afterwards,I conducted a between-subject design study,exploring the effects of agent mediation on communication dynamics aroundco-located smartphone usage and compared the results to the text-basedapproach. Results revealed that the agent is beneficial in promoting co-located interactions by being a valuable support in communication requestsbetween individualsFinally, I discussed design guidelines, limitations and future work ideasin this co-located communication space. The main design guidelines I fo-cused on for my first phase were ensuring personal digital space betweenindividuals, the level of access granted for sharing, notification strategies,and determining whether the user’s phone was in use. For the second phasethe main design guidelines were insights into the smartphone agent behavior,the app enabling awareness of what their partner was doing, assisting themin time management throughout the day, and helping mediating conversa-tions between partners. As for the limitations it would be interesting to seeif results would change pre/post Covid. In addition, it would be interestingto see whether my findings were influenced by the participants age, back-ground, and relationship length. Building towards the future, interactingwith the smartphone agent through voice would be a stepping stone to thisproduct to enhance the ”agent” based aspect of the app. My thesis work onmy smartphone agent can be the first step for researchers to build upon tofurther influence the communication dynamics in co-located individuals.62Bibliography[1] Firebase realtime database — firebase documentation. https://firebase.google.com/docs/database. → pages 37[2] Ahlstro¨m, D., Hasan, K., and Irani, P. Are you comfortabledoing that? Acceptance studies of around-device gestures in and forpublic settings. In Proceedings of the 16th International Conference onHuman-Computer Interaction with Mobile Devices & Services (NewYork, NY, USA, 2014), MobileHCI ’14, Association for ComputingMachinery, pp. 193–202. → pages 6[3] Alallah, F., Neshati, A., Sheibani, N., Sakamoto, Y., Bunt,A., Irani, P., and Hasan, K. Crowdsourcing vs laboratory-stylesocial acceptability studies? examining the social acceptability of spa-tial user interactions for head-worn displays. In Proceedings of the2018 CHI Conference on Human Factors in Computing Systems (NewYork, NY, USA, 2018), CHI ’18, Association for Computing Machin-ery, pp. 1–7. → pages 13[4] Ames, M. G. Managing mobile multitasking: The culture of iphoneson stanford campus. In Proceedings of the 2013 conference on Com-puter supported cooperative work (2013), pp. 1487–1498. → pages 1,7[5] Bales, E., Li, K. A., and Griwsold, W. Couplevibe: Mobile im-plicit communication to improve awareness for (long-distance) couples.In Proceedings of the ACM 2011 Conference on Computer SupportedCooperative Work (New York, NY, USA, 2011), CSCW ’11, Associa-tion for Computing Machinery, pp. 65–74. → pages 24[6] Baptista, S., Wadley, G., Bird, D., Oldenburg, B., Speight,J., Group, M. D. C. R., et al. Acceptability of an embodiedconversational agent for type 2 diabetes self-management educationand support via a smartphone app: mixed methods study. JMIRmHealth and uHealth 8, 7 (2020), e17038. → pages 1063Bibliography[7] Barkhuus, L. The mismeasurement of privacy: Using contextualintegrity to reconsider privacy in hci. In Proceedings of the SIGCHIConference on Human Factors in Computing Systems (New York, NY,USA, 2012), CHI ’12, Association for Computing Machinery, pp. 367–376. → pages 1, 5[8] Beech, S., Geelhoed, E., Murphy, R., Parker, J., Sellen,A., and Shaw, K. Lifestyles of working parents: Implications andopportunities for new technologies. Tech. rep., HP Tech report HPL-2003-88 (R. 1), 2004. → pages 1[9] Bickmore, T. W., Kimani, E., Trinh, H., Pusateri, A.,Paasche-Orlow, M. K., and Magnani, J. W. Managing chronicconditions with a smartphone-based conversational virtual agent. InProceedings of the 18th International Conference on Intelligent Vir-tual Agents (New York, NY, USA, 2018), IVA ’18, Association forComputing Machinery, p. 119124. → pages 3, 10[10] Bo¨hmer, M., Hecht, B., Scho¨ning, J., Kru¨ger, A., andBauer, G. Falling asleep with angry birds, facebook and kindle:A large scale study on mobile application usage. In Proceedings of the13th International Conference on Human Computer Interaction withMobile Devices and Services (New York, NY, USA, 2011), MobileHCI’11, Association for Computing Machinery, p. 4756. → pages 28[11] Buschek, D., Hassib, M., and Alt, F. Personal mobile messag-ing in context: Chat augmentations for expressiveness and awareness.ACM Trans. Comput.-Hum. Interact. 25, 4 (Aug. 2018). → pages 5[12] Butler, A., Izadi, S., and Hodges, S. Sidesight: Multi-”touch”interaction around small devices. In Proceedings of the 21st AnnualACM Symposium on User Interface Software and Technology (NewYork, NY, USA, 2008), UIST ’08, Association for Computing Machin-ery, pp. 201–204. → pages 6[13] Chesley, N. Blurring boundaries? linking technology use, spillover,individual distress, and family satisfaction. Journal of Marriage andFamily 67, 5 (2005), 1237–1248. → pages 7[14] Cramer, H., and Jacobs, M. L. Couples’ communication channels:What, when & why? In Proceedings of the 33rd Annual ACM Confer-ence on Human Factors in Computing Systems (New York, NY, USA,64Bibliography2015), CHI ’15, Association for Computing Machinery, pp. 709–712.→ pages 1, 2, 5[15] Cuotto, N., Krebs, T., and Bekkali, A. Idliketoknow: a mobiletool for interacting with strangers in a collocated area. → pages 8[16] Demirci, K., Akgo¨nu¨l, M., and Akpinar, A. Relationship ofsmartphone use severity with sleep quality, depression, and anxietyin university students. Journal of behavioral addictions 4, 2 (2015),85–92. → pages 6[17] Derix, E. C., and Leong, T. W. Days of our lives: Family experi-ences of digital technology use. In Proceedings of the 30th AustralianConference on Computer-Human Interaction (New York, NY, USA,2018), OzCHI ’18, Association for Computing Machinery, p. 332337.→ pages 2[18] Duke, E´., and Montag, C. Smartphone addiction, daily inter-ruptions and self-reported productivity. Addictive behaviors reports 6(2017), 90–95. → pages 7[19] Dupuy, L., Morin, C. M., de Sevin, E., Taillard, J., Salles,N., Bioulac, S., Auriacombe, M., Micoulaud-Franchi, J.-A.,and Philip, P. Smartphone-based virtual agents and insomnia man-agement: A proof-of-concept study for new methods of autonomousscreening and management of insomnia symptoms in the general pop-ulation. Journal of Sleep Research (2021), e13489. → pages 11[20] DHaro, L. F., Kim, S., Yeo, K. H., Jiang, R., Niculescu, A. I.,Banchs, R. E., and Li, H. Clara: a multifunctional virtual agentfor conference support and touristic information. In Natural languagedialog systems and intelligent assistants. Springer, 2015, pp. 233–239.→ pages 11[21] Ferdous, H. S., Ploderer, B., Davis, H., Vetere, F., O’Hara,K., Farr-Wharton, G., and Comber, R. Tabletalk: integratingpersonal devices and content for commensal experiences at the familydinner table. In Proceedings of the 2016 ACM international joint con-ference on pervasive and ubiquitous computing (2016), pp. 132–143.→ pages 9[22] Freed, D., Palmer, J., Minchala, D., Levy, K., Ristenpart,T., and Dell, N. “a stalker’s paradise”: How intimate partner65Bibliographyabusers exploit technology. In Proceedings of the 2018 CHI Confer-ence on Human Factors in Computing Systems (New York, NY, USA,2018), CHI ’18, Association for Computing Machinery, pp. 1–13. →pages 54[23] Freed, D., Palmer, J., Minchala, D. E., Levy, K., Risten-part, T., and Dell, N. Digital technologies and intimate part-ner violence: A qualitative analysis with multiple stakeholders. Proc.ACM Hum.-Comput. Interact. 1, 46 (2017). → pages 54[24] Gasimov, A., Magagna, F., and Sutanto, J. Camb: Context-aware mobile browser. In Proceedings of the 9th International Con-ference on Mobile and Ubiquitous Multimedia (New York, NY, USA,2010), MUM ’10, Association for Computing Machinery. → pages 5[25] Griggio, C. F., Nouwens, M., McGrenere, J., and Mackay,W. E. Augmenting couples’ communication with ¡i¿lifelines¡/i¿:Shared timelines of mixed contextual information. In Proceedings ofthe 2019 CHI Conference on Human Factors in Computing Systems(New York, NY, USA, 2019), CHI ’19, Association for ComputingMachinery, pp. 1–13. → pages 1, 5[26] Gustafson, S., Bierwirth, D., and Baudisch, P. Imaginaryinterfaces: Spatial interaction with empty hands and without visualfeedback. In Proceedings of the 23nd Annual ACM Symposium onUser Interface Software and Technology (New York, NY, USA, 2010),UIST ’10, Association for Computing Machinery, pp. 3–12. → pages6[27] Hang, A., von Zezschwitz, E., De Luca, A., and Hussmann,H. Too much information! user attitudes towards smartphone sharing.In Proceedings of the 7th Nordic Conference on Human-Computer In-teraction: Making Sense Through Design (New York, NY, USA, 2012),NordiCHI ’12, Association for Computing Machinery, pp. 284–287. →pages 5[28] Hasan, K., Ahlstro¨m, D., and Irani, P. Ad-binning: Leverag-ing around device space for storing, browsing and retrieving mobiledevice content. In Proceedings of the SIGCHI Conference on HumanFactors in Computing Systems (New York, NY, USA, 2013), CHI ’13,Association for Computing Machinery, pp. 899–908. → pages 666Bibliography[29] Hasan, K., Mondal, D., Ahlstro¨m, D., and Neustaedter, C.An exploration of rules and tools for family members to limit co-located smartphone usage. In Proceedings of the 11th Augmented Hu-man International Conference (New York, NY, USA, 2020), AH ’20,Association for Computing Machinery. → pages 5[30] Hasan, K., Mondal, D., Khatra, K., Ahlstro¨m, D., andNeustaedter, C. Coaware: Designing solutions for being aware of aco-located partners smartphone usage activities. In Graphics Interface2021 (2020). → pages 37, 43[31] Hassenzahl, M., Heidecker, S., Eckoldt, K., Diefenbach, S.,and Hillmann, U. All you need is love: Current strategies of mediat-ing intimate relationships through technology. ACM Trans. Comput.-Hum. Interact. 19, 4 (2012). → pages 2, 5[32] Hinckley, K., Pierce, J., Sinclair, M., and Horvitz, E. Sens-ing techniques for mobile interaction. In Proceedings of the 13th An-nual ACM Symposium on User Interface Software and Technology(New York, NY, USA, 2000), UIST ’00, Association for ComputingMachinery, pp. 91–100. → pages 6[33] Hiniker, A., Hong, S., Kohno, T., and Kientz, J. A. Mytime:designing and evaluating an intervention for smartphone non-use. InProceedings of the 2016 CHI conference on human factors in comput-ing systems (2016), pp. 4746–4757. → pages 6[34] Hiniker, A., Schoenebeck, S. Y., and Kientz, J. A. Notat the dinner table: Parents’ and children’s perspectives on fam-ily technology rules. In Proceedings of the 19th ACM Conferenceon Computer-Supported Cooperative Work & Social Computing (NewYork, NY, USA, 2016), CSCW ’16, Association for Computing Ma-chinery, pp. 1376–1389. → pages 1[35] Hudlicka, E. Virtual training and coaching of health behavior: ex-ample from mindfulness meditation training. Patient education andcounseling 92, 2 (2013), 160–166. → pages 3[36] Hwang, K.-H., Yoo, Y.-S., and Cho, O.-H. Smartphone overuseand upper extremity pain, anxiety, depression, and interpersonal rela-tionships among college students. The Journal of the Korea ContentsAssociation 12, 10 (2012), 365–375. → pages 667Bibliography[37] I˙Nal, E. E., Demirci, K., C¸etI˙ntu¨rk, A., Akgo¨nu¨l, M., andSavas¸, S. Effects of smartphone overuse on hand function, pinchstrength, and the median nerve. Muscle & nerve 52, 2 (2015), 183–188. → pages 6[38] Inc., A. Use night shift on your iphone, ipad, and ipod touch,2019. Retrieved October 9, 2020 from https://support.apple.com/en-ca/HT207570. → pages 20[39] Inc., A. Use screen time on your iphone, ipad, or ipod touch,2019. Retrieved October 9, 2020 from https://support.apple.com/en-ca/HT208982. → pages 20[40] Ireland, D., Atay, C., Liddle, J. J., Bradford, D., Lee, H.,Rushin, O., Mullins, T., Angus, D., Wiles, J., McBride, S.,et al. Hello harlie: enabling speech monitoring through chat-bot con-versations. Studies in health technology and informatics 227 (2016),55–60. → pages 3[41] Ismail, S., Mazuki, H., and Husin, H. S. Agent-mediation in radtframework for self-quarantine monitoring. In 2021 4th InternationalSymposium on Agents, Multi-Agent Systems and Robotics (ISAMSR)(2021), IEEE, pp. 13–19. → pages 11[42] Jacobs, M., Cramer, H., and Barkhuus, L. Caring about shar-ing: Couples’ practices in single user device access. In Proceedings ofthe 19th International Conference on Supporting Group Work (NewYork, NY, USA, 2016), GROUP ’16, Association for Computing Ma-chinery, pp. 235–243. → pages 5[43] Jarusriboonchai, P., Malapaschas, A., Olsson, T., andVa¨a¨na¨nen, K. Increasing collocated people’s awareness of the mo-bile user’s activities: A field trial of social displays. In Proceedings ofthe 19th ACM Conference on Computer-Supported Cooperative Workamp; Social Computing (New York, NY, USA, 2016), CSCW ’16, As-sociation for Computing Machinery, p. 16911702. → pages 1, 8, 23,24, 30[44] Karlson, A. K., Brush, A. B., and Schechter, S. Can i borrowyour phone? understanding concerns when sharing mobile phones. InProceedings of the SIGCHI Conference on Human Factors in Com-puting Systems (New York, NY, USA, 2009), CHI ’09, Association forComputing Machinery, pp. 1647–1650. → pages 568Bibliography[45] Kawsar, F., and Brush, A. B. Home computing unplugged: why,where and when people use different connected devices at home. InProceedings of the 2013 ACM international joint conference on Per-vasive and ubiquitous computing (2013), pp. 627–636. → pages 1, 7[46] Kim, H.-J., Min, J.-Y., Min, K.-B., Lee, T.-J., and Yoo, S. Re-lationship among family environment, self-control, friendship quality,and adolescents smartphone addiction in south korea: Findings fromnationwide data. PloS one 13, 2 (2018), e0190896. → pages 7[47] Kim, I., Jung, G., Jung, H., Ko, M., and Lee, U. Let’s focus:Location-based intervention tool to mitigate phone use in college class-rooms. In Proceedings of the 2017 ACM International Joint Confer-ence on Pervasive and Ubiquitous Computing and Proceedings of the2017 ACM International Symposium on Wearable Computers (NewYork, NY, USA, 2017), UbiComp ’17, Association for Computing Ma-chinery, p. 101104. → pages 8[48] Kim, J., Jung, H., Ko, M., and Lee, U. Goalkeeper: Exploringinteraction lockout mechanisms for regulating smartphone use. Pro-ceedings of the ACM on Interactive, Mobile, Wearable and UbiquitousTechnologies 3, 1 (2019), 1–29. → pages 6[49] Kimani, E., Bickmore, T., Trinh, H., Ring, L., Paasche-Orlow, M. K., and Magnani, J. W. A smartphone-based virtualagent for atrial fibrillation education and counseling. In Internationalconference on intelligent virtual agents (2016), Springer, pp. 120–127.→ pages 3, 10[50] Kjeldskov, J., and Paay, J. Just-for-us: A context-aware mo-bile information system facilitating sociality. In Proceedings of the 7thInternational Conference on Human Computer Interaction with Mo-bile Devices & Services (New York, NY, USA, 2005), MobileHCI ’05,Association for Computing Machinery, pp. 23–30. → pages 5[51] Kleinman, L., Hirsch, T., and Yurdana, M. Exploring mobiledevices as personal public displays. In Proceedings of the 17th In-ternational Conference on Human-Computer Interaction with MobileDevices and Services (2015), pp. 233–243. → pages 8[52] Ko, M., Choi, S., Yatani, K., and Lee, U. Lock n’ lol: Group-based limiting assistance app to mitigate smartphone distractions in69Bibliographygroup activities. In Proceedings of the 2016 CHI Conference on HumanFactors in Computing Systems (New York, NY, USA, 2016), CHI ’16,Association for Computing Machinery, p. 9981010. → pages 8[53] Kratz, S., and Rohs, M. Hoverflow: Exploring around-device inter-action with ir distance sensors. In Proceedings of the 11th InternationalConference on Human-Computer Interaction with Mobile Devices andServices (New York, NY, USA, 2009), MobileHCI ’09, Association forComputing Machinery, pp. 1–8. → pages 6[54] Laranjo, L., Dunn, A. G., Tong, H. L., Kocaballi, A. B.,Chen, J., Bashir, R., Surian, D., Gallego, B., Magrabi, F.,Lau, A. Y., et al. Conversational agents in healthcare: a systematicreview. Journal of the American Medical Informatics Association 25,9 (2018), 1248–1258. → pages 10[55] Lee, H., Kim, J. W., and Choi, T. Y. Risk factors for smartphoneaddiction in korean adolescents: smartphone use patterns. Journal ofKorean medical science 32, 10 (2017), 1674–1679. → pages 6[56] Levin, E., Levin, A., et al. Evaluation of spoken dialogue technol-ogy for real-time health data collection. Journal of Medical InternetResearch 8, 4 (2006), e590. → pages 3[57] Liszio, S., Emmerich, K., and Masuch, M. The influence of socialentities in virtual reality games on player experience and immersion.In Proceedings of the 12th international conference on the foundationsof digital games (2017), pp. 1–10. → pages 12[58] Magazine, G. G. What is your phone doing to yourrelationships?, 2021. Retrieved February 10, 2021 fromhttps://greatergood.berkeley.edu/article/item/what_is_your_phone_doing_to_your_relationships. → pages 24[59] Mahlo, L., and Windsor, T. D. Feasibility, acceptability, andpreliminary efficacy of an app-based mindfulness-meditation programamong older adults. The Gerontologist 61, 5 (2021), 775–786. → pages10[60] MailOnline, D. K. f. Tech is taking over the dinner table: Thirdof kids distracted by phones at meal times and social media sites arethe biggest draw, Sep 2014. → pages 970Bibliography[61] Matthews, T., Liao, K., Turner, A., Berkovich, M., Reeder,R., and Consolvo, S. ”she’ll just grab any device that’s closer”: Astudy of everyday device & account sharing in households. In Proceed-ings of the 2016 CHI Conference on Human Factors in ComputingSystems (New York, NY, USA, 2016), CHI ’16, Association for Com-puting Machinery, pp. 5921–5932. → pages 5[62] Mehrotra, A., Hendley, R., and Musolesi, M. Prefminer: Min-ing user’s preferences for intelligent mobile notification management.In Proceedings of the 2016 ACM International Joint Conference onPervasive and Ubiquitous Computing (2016), pp. 1223–1234. → pages9[63] Mehrotra, A., Pejovic, V., Vermeulen, J., Hendley, R., andMusolesi, M. My phone and me: understanding people’s receptivityto mobile notifications. In Proceedings of the 2016 CHI conference onhuman factors in computing systems (2016), pp. 1021–1032. → pages9[64] Neustaedter, C., Harrison, S., and Sellen, A. Connectingfamilies. The impact of new (2013). → pages 2[65] Noe¨, B., Turner, L. D., Linden, D. E., Allen, S. M., Winkens,B., and Whitaker, R. M. Identifying indicators of smartphoneaddiction through user-app interaction. Computers in human behavior99 (2019), 56–65. → pages 1, 6[66] Nouwens, M., Griggio, C. F., and Mackay, W. E. ”whatsappis for family; messenger is for friends”: Communication places in appecosystems. In Proceedings of the 2017 CHI Conference on HumanFactors in Computing Systems (New York, NY, USA, 2017), CHI ’17,Association for Computing Machinery, pp. 727–735. → pages 2, 5[67] Obremski, D., Scha¨fer, A. L., Lange, B. P., Lugrin, B.,Ganal, E., Witt, L., Nun˜ez, T. R., Schwarz, S., and Schwab,F. Put that away and talk to me - the effects of smartphone in-duced ostracism while interacting with an intelligent virtual agent.In Proceedings of the 9th International Conference on Human-AgentInteraction (New York, NY, USA, 2021), HAI ’21, Association forComputing Machinery, p. 428432. → pages 1071Bibliography[68] Oduor, E., Neustaedter, C., Odom, W., Tang, A., Moallem,N., Tory, M., and Irani, P. The frustrations and benefits of mo-bile device usage in the home when co-present with family members.In Proceedings of the 2016 ACM conference on designing interactivesystems (2016), pp. 1315–1327. → pages 1, 15, 21[69] Oduor, E., Neustaedter, C., Odom, W., Tang, A., Moallem,N., Tory, M., and Irani, P. The frustrations and benefits of mo-bile device usage in the home when co-present with family members.In Proceedings of the 2016 ACM conference on designing interactivesystems (2016), pp. 1315–1327. → pages 7, 43, 54[70] Offtime. Unplug, it’s enough, 2019. Retrieved October 9, 2020 fromhttps://offtime.app/index.php. → pages 20[71] Okoshi, T., Ramos, J., Nozaki, H., Nakazawa, J., Dey, A. K.,and Tokuda, H. Reducing users’ perceived mental effort due to in-terruptive notifications in multi-device mobile environments. In Pro-ceedings of the 2015 ACM International Joint Conference on Pervasiveand Ubiquitous Computing (2015), pp. 475–486. → pages 9, 60[72] Paasovaara, S., Olshannikova, E., Jarusriboonchai, P.,Malapaschas, A., and Olsson, T. Next2you: a proximity-basedsocial application aiming to encourage interaction between nearby peo-ple. In Proceedings of the 15th International Conference on Mobile andUbiquitous Multimedia (2016), pp. 81–90. → pages 8[73] Park, C., Lim, J., Kim, J., Lee, S.-J., and Lee, D. Mitigatingsmartphone interruptions during social interactions. → pages 9[74] Park, C., Lim, J., Kim, J., Lee, S.-J., and Lee, D. Don’t botherme. i’m socializing! a breakpoint-based smartphone notification sys-tem. In Proceedings of the 2017 ACM Conference on Computer Sup-ported Cooperative Work and Social Computing (New York, NY, USA,2017), CSCW ’17, Association for Computing Machinery, p. 541554.→ pages 9[75] Park, N., and Lee, H. Social implications of smartphone use: Ko-rean college students’ smartphone use and psychological well-being.Cyberpsychology, Behavior, and Social Networking 15, 9 (2012), 491–497. → pages 172Bibliography[76] Philip, P., Bioulac, S., Sauteraud, A., Chaufton, C., andOlive, J. Could a virtual human be used to explore excessive day-time sleepiness in patients? Presence: teleoperators and virtual envi-ronments 23, 4 (2014), 369–376. → pages 3[77] Philip, P., Dupuy, L., Morin, C. M., de Sevin, E., Bioulac,S., Taillard, J., Serre, F., Auriacombe, M., Micoulaud-Franchi, J.-A., et al. Smartphone-based virtual agents to helpindividuals with sleep concerns during covid-19 confinement: feasibil-ity study. Journal of medical Internet research 22, 12 (2020), e24268.→ pages 10[78] Pielot, M., Cardoso, B., Katevas, K., Serra`, J., Matic, A.,and Oliver, N. Beyond interruptibility: Predicting opportune mo-ments to engage mobile phone users. Proceedings of the ACM on In-teractive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017),1–25. → pages 9[79] Proulx, C. M., Helms, H. M., and Buehler, C. Marital qualityand personal well-being: A meta-analysis. Journal of Marriage andfamily 69, 3 (2007), 576–593. → pages 7[80] Rabinowitz, A., and Collier, G. Development of a conversa-tional agent for promoting increased activity in users with traumaticbrain injury. Archives of Physical Medicine and Rehabilitation 102, 10(2021), e83. → pages 3, 10[81] Radesky, J. S., Kistin, C. J., Zuckerman, B., Nitzberg, K.,Gross, J., Kaplan-Sanoff, M., Augustyn, M., and Silver-stein, M. Patterns of mobile device use by caregivers and childrenduring meals in fast food restaurants. Pediatrics 133, 4 (2014), e843–e849. → pages 7[82] RescueTime. Rescuetime: time management software for stay-ing productive and happy in the modern workplace, 2019. Re-trieved October 9, 2020 from https://github.com/ricvalerio/foregroundappchecker/. → pages 6[83] Rhee, H., Allen, J., Mammen, J., and Swift, M. Mobile phone-based asthma self-management aid for adolescents (masmaa): a feasi-bility study. Patient preference and adherence 8 (2014), 63. → pages373Bibliography[84] Roberta, K. V., Wisudawati, L. M., Razi, M., et al. Webbased virtual agent for tourism guide in indonesia. In InternationalConference on Advances in Computing and Communications (2011),Springer, pp. 146–153. → pages 11[85] Roberts, J. A., and David, M. E. My life has become a majordistraction from my cell phone: Partner phubbing and relationshipsatisfaction among romantic partners. Computers in human behavior54 (2016), 134–141. → pages 7[86] Samaha, M., and Hawi, N. S. Relationships among smartphoneaddiction, stress, academic performance, and satisfaction with life.Computers in human behavior 57 (2016), 321–325. → pages 7[87] San˜udo, B., Fennell, C., and Sa´nchez-Oliver, A. J.Objectively-assessed physical activity, sedentary behavior, smart-phone use, and sleep patterns pre-and during-covid-19 quarantine inyoung adults from spain. Sustainability 12, 15 (2020), 5890. → pages6[88] Schilit, B., Adams, N., and Want, R. Context-aware computingapplications. In 1994 First Workshop on Mobile Computing Systemsand Applications (1994), IEEE, pp. 85–90. → pages 6[89] Seeburger, J., Foth, M., and Tjondronegoro, D. Capital mu-sic: Personal expression with a public display of song choice. In Pro-ceedings of the 6th Nordic Conference on Human-Computer Interac-tion: Extending Boundaries (New York, NY, USA, 2010), NordiCHI’10, Association for Computing Machinery, p. 777780. → pages 8[90] Serra, G., Scalzo, L. L., Giuffre`, M., Ferrara, P., andCorsello, G. Smartphone use and addiction during the coronavirusdisease 2019 (covid-19) pandemic: cohort study on 184 italian childrenand adolescents. Italian Journal of Pediatrics 47, 1 (2021), 1–10. →pages 6[91] Steiner-Adair, C., and Barker, T. H. The big disconnect: Pro-tecting childhood and family relationships in the digital age. Harper-Collins Publishers, 2014. → pages 1, 7[92] Stieger, M., Nißen, M., Ru¨egger, D., Kowatsch, T.,Flu¨ckiger, C., and Allemand, M. Peach, a smartphone-and con-versational agent-based coaching intervention for intentional personal-74Bibliographyity change: study protocol of a randomized, wait-list controlled trial.BMC psychology 6, 1 (2018), 1–15. → pages 11[93] Systems, S. Decision tree diagram. https://www.sparxsystems.com/enterprise_architect_user_guide/14.0/guidebooks/tools_ba_decision_tree_diagram.html. → pages 10[94] Thayer, A., Bietz, M. J., Derthick, K., and Lee, C. P. I loveyou, let’s share calendars: Calendar sharing as relationship work. InProceedings of the ACM 2012 Conference on Computer Supported Co-operative Work (New York, NY, USA, 2012), CSCW ’12, Associationfor Computing Machinery, p. 749758. → pages 2, 5, 20[95] Uchiya, T., Yoshida, M., Yamamoto, D., Nishimura, R., andTakumi, I. Design and implementation of open-campus event systemwith voice interaction agent. Journal of Mobile Multimedia (2015),237–250. → pages 10[96] Uchiya, T., Yoshida, M., Yamamoto, D., and Takumi, I. De-velopment of open-campus event system using voice interaction agent.In 2015 18th International Conference on Network-Based InformationSystems (2015), IEEE, pp. 591–595. → pages 10[97] Valrio, R. Foreground app checker for android, 2019. Re-trieved October 9, 2020 from https://github.com/ricvalerio/foregroundappchecker/. → pages 25, 37[98] Wang, D., Park, S., and Fesenmaier, D. R. The role of smart-phones in mediating the touristic experience. Journal of Travel Re-search 51, 4 (2012), 371–387. → pages 11[99] Watanabe, M., Ogawa, K., and Ishiguro, H. Field study: canandroids be a social entity in the real world? In Proceedings of the2014 ACM/IEEE international conference on Human-robot interac-tion (2014), pp. 316–317. → pages 11[100] Webb, C., Hirshberg, M. J., Davidson, R. J., and Goldberg,S. Personalized prediction of response to smartphone-delivered medi-tation training. → pages 10[101] Yamamoto, D., Oura, K., Nishimura, R., Uchiya, T., Lee, A.,Takumi, I., and Tokuda, K. Voice interaction system with 3d-cg virtual agent for stand-alone smartphones. In Proceedings of the75BibliographySecond International Conference on Human-Agent Interaction (NewYork, NY, USA, 2014), HAI ’14, Association for Computing Machin-ery, p. 323330. → pages 10[102] Yueh, Y. T., Chiu, D. K., Leung, H.-f., and Hung, P. C. A vir-tual travel agent system for m-tourism with semantic web service baseddesign and implementation. In 21st International Conference on Ad-vanced Information Networking and Applications (AINA’07) (2007),IEEE, pp. 142–149. → pages 1176
Thesis/Dissertation
eng
Computer Science
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Hasan, Mohammad Khalad
Designing solutions to mediate co-located smartphone usage
Text