Science, Irving K. Barber Faculty of (Okanagan)
Computer Science, Mathematics, Physics and Statistics, Department of (Okanagan)
DSpace
UBCO
Rey, Bradley
Doctor of Philosophy - PhD
University of British Columbia
Smartwatches have become increasingly popular for health and fitness tracking. Worn on the body, and equipped with powerful sensors, smartwatches enable comprehensive and ubiquitous health data collection. Exploring this collected data on the smartwatch is, however, less comprehensive and is largely unrealized. The small screen size and many daily scenarios of use are often considered challenges which limit exploratory capability. Yet, the immediate availability of the smartwatch remains to offer potential for actionable and in-situ insights derived from the collected data. This thesis introduces databiting, a novel concept for lightweight and transient data exploration that supports in-situ exploration on the smartwatch. Databiting bridges the gap between currently offered glanceable data visualizations, and more extensive data analysis available on other devices. Before realizing databiting, we conducted an empirical study with 18 individuals to better understand their smartwatch data exploration needs. We captured and analyzed 205 personal health data queries to understand where, when, and why these queries were desired. This analysis combined qualitative insights with quantitative metrics, uncovering patterns that can inform the design and implementation of databiting on the smartwatch. We further identified key dimensions of these queries, such as interrogatives, data sources, aggregations, and filtering mechanisms, which provide a foundation for enhancing natural language processing capabilities. We then examined preferences of voice assistant responses to personal health data queries, as an additional output modality to visualization. Insights were gathered on perceived quality, behavior, comprehensibility, and efficiency to a variety of answer structures, providing guidelines to support databiting through an effective output modality during cognitively loaded tasks, such as during a workout. Using these findings, we then developed DataWatch, a novel smartwatch application which facilitates in-situ exploration of past collected fitness data through lightweight and transient touch and speech interactions. DataWatch supports Single Value, Browse, and Compare exploration types. Furthermore, users can explore their data during different workout phases---before, during, and after a workout---enhancing one's ability to gain insights and make informed decisions while in-situ. Overall, this research demonstrates the feasibility and benefits of databiting on smartwatches, providing a foundation for future advancements in personalized data exploration on wearable devices. DataWatch exemplifies how smartwatch health applications can begin to evolve within their means to meet user expectations, enhance the user experience, provide increased actionable insights, and motivate deeper engagement with health data.
Databiting: Enabling Lightweight,Transient, and In-Situ Exploration ofPersonal Fitness Data on SmartwatchesbyBradley ReyB.Sc., The University of Manitoba 2020A DISSERTATION SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHYinThe College of Graduate Studies(Computer Science)THE UNIVERSITY OF BRITISH COLUMBIA(Okanagan)December 2024© Bradley Rey, 2024The following individuals certify that they have read, and recommend to theCollege of Graduate Studies for acceptance, the dissertation entitled:DATABITING: ENABLING LIGHTWEIGHT, TRANSIENT, AND IN-SITU EX-PLORATION OF PERSONAL FITNESS DATA ON SMARTWATCHESsubmitted by BRADLEY REY in partial fulfilment of the requirements of the degreeof Doctor of PhilosophyPourang Irani, Irving K. Barber Faculty of Science, UBC OkanaganSupervisorKaron MacLean, Computer Science, UBC VancouverSupervisory Committee MemberRamon Lawrence, Irving K. Barber Faculty of Science, UBC OkanaganSupervisory Committee MemberMegan Smith, Creative and Critical Studies, UBC OkanaganUniversity ExaminerPetra Isenberg, INRIAExternal ExamineriiAbstractSmartwatches have become increasingly popular for health and fitness track-ing. Worn on the body, and equipped with powerful sensors, smartwatches enablecomprehensive and ubiquitous health data collection. Exploring this collected dataon the smartwatch is, however, less comprehensive and is largely unrealized. Thesmall screen size and many daily scenarios of use are often considered challengeswhich limit exploratory capability. Yet, the immediate availability of the smart-watch remains to offer potential for actionable and in-situ insights derived fromthe collected data. As such, this dissertation introduces databiting, a novel conceptfor lightweight and transient data exploration that supports in-situ exploration.Before realizing databiting, we conducted an empirical study with 18 individu-als to better understand their smartwatch data exploration needs. We captured andanalyzed 205 personal health data queries to understand where, when, and whythese queries were desired. This analysis combined qualitative insights with quan-titative metrics, uncovering patterns that inform the design and implementation ofdatabiting on the smartwatch. We further identified key dimensions such as inter-rogatives, data sources, aggregations, and filtering mechanisms, which provide afoundation for enhancing natural language processing capabilities.We then examined preferences of voice assistant responses to personal healthdata queries, as an additional output modality to visualization. Insights were gath-ered on perceived quality, behavior, comprehensibility, and efficiency to a varietyof answer structures, providing guidelines to support databiting through an effec-tive and complementary output modality.Using these findings, we then developed DataWatch, a novel smartwatch appli-cation which facilitates in-situ exploration of fitness data through lightweight andtransient touch and speech interactions. DataWatch supports Single Value, Browse,iiiand Compare exploration types. Furthermore, users can explore their data duringdifferent workout phases—before, during, and after a workout—enhancing one’sability to gain insights and make informed decisions while in-situ.Overall, this research demonstrates the feasibility and benefits of databiting onsmartwatches, providing a foundation for future advancements in personalized dataexploration on wearable devices. DataWatch exemplifies how smartwatch healthapplications can begin to evolve within their means to meet user expectations, pro-vide increased actionable insights, and motivate deeper engagement with healthdata.ivLay SummaryThis thesis focuses on enhancing how people interact with their personal healthdata on smartwatches. We introduce the concept of databiting, with the goalof allowing exploration of data in both a quick and easy manner. Through alearned understanding of peoples’ smartwatch data exploration needs, we devel-oped DataWatch. DataWatch is an Apple Watch application allowing people toquestion and explore their past collected fitness data before, during, and after aworkout. By enabling further exploration on the smartwatch, from that of thecurrent visualizations, DataWatch provides the potential for increased actionableinsights and motivation in exploring one’s data. Our findings not only demonstratethat people desire more interactive and insightful health data exploration on theirsmartwatches, but also validate that such exploration directly on the smartwatchis both feasible and beneficial. Finally, results guide future advancements for in-creasingly personalized and capable health data exploration on the smartwatch.vPrefacePublications Included in ThesisSections of this thesis have been published in conference proceedings or jour-nals. We note, both the ACM and IEEE (publishers for which the following workhas been published to) provide confirmation that published material can be used ina dissertation12. The following is a list of publications in which portions of thisthesis work appeared, organized by chapter. For each of the publications listed be-low, I was a either a lead or co-lead for the identification and design of the research,building of study methodologies and research instruments, conducting of study par-ticipation with participants, analysis of data, and preparation of manuscripts sub-mitted. Moreover, each contributing publication was the result of collaborations,both local and abroad. For each, I further highlight the nature of any collaborativework that took place below.Portions of Chapter 3Bradley Rey, Bongshin Lee, Eun Kyoung Choe and Pourang Irani. 2024.Databiting: Lightweight, Transient, and Insight Rich Exploration of Personal Data,in IEEE Computer Graphics and Applications, vol. 44, no. 2, pp. 65-72, March-April 2024. DOI: https://doi.org/10.1109/MCG.2024.3353888The article was a collaborative effort among myself, Bongshin Lee, Eun Ky-oung Choe, and Pourang Irani. The collaboration took shape through joint meet-ings focused on concept creation and development, as well as later editorial work.1https://authors.acm.org/author-resources/author-rights2https://www.ieee.org/content/dam/ieee-org/ieee/web/org/pubs/permissions_faq.pdfviPortions of Chapter 4Bradley Rey, Bongshin Lee, Eun Kyoung Choe, and Pourang Irani. 2023.Investigating In-Situ Personal Health Data Queries on Smartwatches. Proceedingsof the ACM Interactive Mobile Wearable Ubiquitous Technologies, 6, 4, Article179, December 2022, 19 pages. DOI: https://doi.org/10.1145/3569481The article was a collaborative effort among myself, Bongshin Lee, Eun Ky-oung Choe, and Pourang Irani. The collaboration took shape through combined ef-forts surrounding ideation, study design and editorial work. Furthermore, Charles-Olivier Dufresne-Camaro supported efforts in the data analysis/coding processhighlighted within the chapter.Portions of Chapter 5Bradley Rey, Charles-Olivier Dufresne-Camaro, and Pourang Irani. 2023. To-wards Efficient Interaction for Personal Health Data Queries on Smartwatches. InProceedings of the 25th International Conference on Mobile Human-Computer In-teraction, Article 18, 1–7. https://doi.org/10.1145/3565066.3608700The article was a collaborative effort among myself, Charles-Olivier Dufresne-Camaro, and Pourang Irani. Charles-Olivier Dufresne-Camaro supported effortsin the data analysis/coding process highlighted within the chapter and providededitorial support.Portions of Chapter 6Bradley Rey, Yumiko Sakamoto, Jaisie Sin and Pourang Irani. 2024. Un-derstanding User Preferences of Voice Assistant Answer Structures for PersonalHealth Data Queries. In Proceedings of the 25th International Conference onConversational User Interfaces, Article 18, 1–7. https://doi.org/10.1145/3565066.3608700The article was a collaborative effort among myself, Yumiko Sakamoto, JaisieSin, and Pourang Irani. Yumiko Sakamoto collaborated on study design, supporteddata analysis procedures, and editorial work. Jaisie Sin collaborated on study de-sign and editorial work.viiThesis Publications in ProgressSections of this thesis contain work of a publication in progress, and is to besubmitted for peer-review at a later date.Portions of Chapter 7Bradley Rey et al. DataWatch: Fostering Exploration of Physical ActivityData on the Smartwatch Leveraging Speech and Touch Interaction for In-Situ In-sightAll work in this chapter is solely my own. However, discussions surround-ing the work put forward in Chapter 3 and Chapter 4 influenced aspects of theDataWatch application and study methodology. As such, Bongshin Lee, Eun Ky-oung Choe, and Pourang Irani have provided indirect collaboration.Other PublicationsBelow is a list of additional works that were published during my time as aPh.D. student, for which I contributed varying roles:1. Shariff AM Faleel, Yishuo Liu, Roya A Cody, Bradley Rey, Linghao Du,Jiangyue Yu, Da-Yuan Huang, Pourang Irani, and Wei Li. 2023. T-Force:Exploring the Use of Typing Force for Three State Virtual Keyboards. InProceedings of the 2023 CHI Conference on Human Factors in ComputingSystems (CHI ’23). Association for Computing Machinery, New York, NY,USA, Article 723, 1–15. https://doi.org/10.1145/3544548.35809152. Bradley Rey, Kening Zhu, Simon Tangi Perrault, Sandra Bardot, Ali Ne-shati, and Pourang Irani. 2022. Understanding and Adapting Bezel-to-Bezel Interactions for Circular Smartwatches in Mobile and EncumberedScenarios. Proc. ACM Human Computer Interaction 6, MHCI, Article 201(September 2022), 28 pages. https://doi.org/10.1145/35467363. Sandra Bardot, Bradley Rey, Lucas Audette, Kevin Fan, Da-Yuan Huang,Jun Li, Wei Li, and Pourang Irani. 2022. One Ring to Rule Them All: AnviiiEmpirical Understanding of Day-to-Day Smartring Usage Through In-SituDiary Study. Proc. ACM Interactive Mobile Wearable Ubiquitous Technolo-gies 6, 3, Article 100 (September 2022), 20 pages. https://doi.org/10.1145/35503154. Anuradha Herath, Bradley Rey, Sandra Bardot, Sawyer Rempel, LucasAudette, Huizhe Zheng, Jun Li, Kevin Fan, Da-Yuan Huang, Wei Li, andPourang Irani. 2022. Expanding Touch Interaction Capabilities for Smart-rings: An Exploration of Continual Slide and Microroll Gestures. In Ex-tended Abstracts of the 2022 CHI Conference on Human Factors in Com-puting Systems (CHI EA ’22). Association for Computing Machinery, NewYork, NY, USA, Article 292, 1–7. https://doi.org/10.1145/3491101.35197145. Fouad Alallah, Shariff Faleel, Yumiko Sakamoto, Bradley Rey, and PourangIrani. 2022. SSCA: Situated Space-time Cube Analytics. EuroVis 2022 -Short Papers. https://doi.org/10.2312/evs.202210886. Ali Neshati, Fouad Alallah, Bradley Rey, Yumiko Sakamoto, Marcos Ser-rano, and Pourang Irani. 2021. SF-LG: Space-Filling Line Graphs for Vi-sualizing Interrelated Time-series Data on Smartwatches. In Proceedings ofthe 23rd International Conference on Mobile Human-Computer Interaction(MobileHCI ’21). Association for Computing Machinery, New York, NY,USA, Article 5, 1–13. https://doi.org/10.1145/3447526.34720407. Sandra Bardot, Surya Rawat, Duy Thai Nguyen, Sawyer Rempel, HuizheZheng, Bradley Rey, Jun Li, Kevin Fan, Da-Yuan Huang, Wei Li, andPourang Irani. 2021. ARO: Exploring the Design of Smart-Ring Interac-tions for Encumbered Hands. In Proceedings of the 23rd International Con-ference on Mobile Human-Computer Interaction (MobileHCI ’21). Asso-ciation for Computing Machinery, New York, NY, USA, Article 12, 1–11.https://doi.org/10.1145/3447526.34720378. Shariff AM Faleel*, Bibhushan Raj Joshi* and Bradley Rey* (*equal authorcontribution). Writely: Force Feedback for Non-Dominant Hand WritingixTraining. 2021. IEEE World Haptics Conference (WHC), Montreal, QC,Canada, 2021, pp. 340-340. https://doi.org/10.1109/WHC49131.2021.95172099. Ali Neshati, Bradley Rey, Ahmed Shariff Mohommed Faleel, Sandra Bar-dot, Celine Latulipe, and Pourang Irani. 2021. BezelGlide: Interacting withGraphs on Smartwatches with Minimal Screen Occlusion. In Proceedings ofthe 2021 CHI Conference on Human Factors in Computing Systems (CHI’21). Association for Computing Machinery, New York, NY, USA, Article501, 1–13. https://doi.org/10.1145/3411764.344520110. Sandra Bardot, Sawyer Rempel, Bradley Rey, Ali Neshati, Yumiko Sakamoto,Carlo Menon, and Pourang Irani. 2020. Eyes-free graph legibility: usingskin-dragging to provide a tactile graph visualization on the arm. In Pro-ceedings of the 11th Augmented Human International Conference (AH ’20).Association for Computing Machinery, New York, NY, USA, Article 1, 1–8.https://doi.org/10.1145/3396339.3396344Use of Generative AIThroughout, the use of generative AI (e.g., ChatGPT, Gemini, etc.) has beenlimited to only that of editorial support. Specifically, ChatGPT has been used invarious places throughout to critique my writing in a holistic manner, rather thanproviding individual grammatical edits. Prompts such as “How well do you feelthese paragraphs work together in providing [the desired argument]?” and “Whatare some areas of improvement that could be incorporated into the shared text?”were used. No other generative AI models/tools were used. All figures are originalcreations, not created through the use of generative AI.Ethics ApprovalResearch undertaken within this thesis spanned both the University of Man-itoba, where I began my degree, and the University of British Columbia Okana-xgan, where I completed my degree. Within both institutions, ethics approval wasobtained before conducting studies with participants. Specifically the followingcertificate numbers were obtained:− Chapter 4 - University of Manitoba, Research Ethics Board 1, CertificateNumber HE2021-0070− Chapter 6 - University of British Columbia Okanagan, Behavioural Re-search Ethics Board, Certificate Number H23-00807− Chapter 7 - University of British Columbia Okanagan, Behavioural Re-search Ethics Board, Certificate Number H23-00805xiTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviiiList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Research Challenges . . . . . . . . . . . . . . . . . . . . . . . . 21.1.1 Conceptualization . . . . . . . . . . . . . . . . . . . . . 51.1.2 What . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.1.3 When, Where, Why . . . . . . . . . . . . . . . . . . . . 61.1.4 How . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.1.5 Databiting Validation . . . . . . . . . . . . . . . . . . . 81.2 Research Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.3 Thesis Contributions . . . . . . . . . . . . . . . . . . . . . . . . 102 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.1 Personal Informatics . . . . . . . . . . . . . . . . . . . . . . . . 132.2 Wearable Computing . . . . . . . . . . . . . . . . . . . . . . . . 15xii2.3 General Smartwatch Usage . . . . . . . . . . . . . . . . . . . . . 192.4 The Smartwatch and Personal Data Exploration . . . . . . . . . . 223 Databiting: Lightweight, Transient, and Insight Rich Exploration ofPersonal Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.1 The Databiting Concept . . . . . . . . . . . . . . . . . . . . . . 303.1.1 Lightweight, Transient, and Insight Rich Exploration . . . 313.1.2 Device Agnostic Exploration . . . . . . . . . . . . . . . 333.1.3 Data Agnostic Exploration . . . . . . . . . . . . . . . . . 333.2 Expected Benefits of Databiting . . . . . . . . . . . . . . . . . . 343.2.1 Introductory and Intermediary Access . . . . . . . . . . . 343.2.2 Increased In-Situ Insight . . . . . . . . . . . . . . . . . . 343.2.3 Perceived Usefulness . . . . . . . . . . . . . . . . . . . 353.3 Research Considerations . . . . . . . . . . . . . . . . . . . . . . 353.3.1 Contextual and Attentional Factors . . . . . . . . . . . . 353.3.2 Interaction Modalities . . . . . . . . . . . . . . . . . . . 363.3.3 Databiting and Broader Exploration . . . . . . . . . . . . 373.3.4 Personalization and Customization . . . . . . . . . . . . 383.3.5 Evaluation Challenges . . . . . . . . . . . . . . . . . . . 413.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424 Eliciting In-Situ Personal Health Data Queries on the Smartwatch . 434.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.2 Relevant Related Work . . . . . . . . . . . . . . . . . . . . . . . 454.2.1 In-Situ Data Collection Methods . . . . . . . . . . . . . 454.3 User Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464.3.1 Participants . . . . . . . . . . . . . . . . . . . . . . . . . 464.3.2 Data Collection Method . . . . . . . . . . . . . . . . . . 474.3.3 Data Collection Application . . . . . . . . . . . . . . . . 474.3.4 Study Procedure . . . . . . . . . . . . . . . . . . . . . . 514.4 Analysis and Results . . . . . . . . . . . . . . . . . . . . . . . . 534.4.1 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 53xiii4.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 544.4.3 Natural Language for Personal Health Data Exploration byLay Individuals . . . . . . . . . . . . . . . . . . . . . . 624.5 Discussion and Future Work . . . . . . . . . . . . . . . . . . . . 644.5.1 In-Situ and Non-In-Situ Preparation-for-Action . . . . . . 644.5.2 Query Insight Category Dependent on In-Situ Activity . . 654.5.3 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . 654.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 675 Towards Natural Language Interaction for Personal Health Data Querieson Smartwatches . . . . . . . . . . . . . . . . . . . . . . . . . . . . 685.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 695.2 Relevant Related Work . . . . . . . . . . . . . . . . . . . . . . . 705.2.1 Natural Language Query Analysis . . . . . . . . . . . . . 705.3 Dataset and Analysis . . . . . . . . . . . . . . . . . . . . . . . . 715.3.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . 715.3.2 Coding Procedure . . . . . . . . . . . . . . . . . . . . . 715.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 725.4.1 Attributes of Requested Data . . . . . . . . . . . . . . . 735.4.2 Aggregations . . . . . . . . . . . . . . . . . . . . . . . . 755.4.3 Interrogatives . . . . . . . . . . . . . . . . . . . . . . . 775.4.4 Filtering Mechanisms . . . . . . . . . . . . . . . . . . . 775.4.5 Components of a Personal Health Data Query . . . . . . 815.4.6 Response Expectations . . . . . . . . . . . . . . . . . . . 815.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 825.5.1 Lay-Person Exploration of Personal Health Data . . . . . 825.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 826 User Preferences of Voice Assistant Answers to Personal Health DataQueries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 846.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 856.2 Relevant Related Work . . . . . . . . . . . . . . . . . . . . . . . 87xiv6.2.1 Interaction with Voice Assistants . . . . . . . . . . . . . 876.2.2 Voice Assistants in Health Contexts . . . . . . . . . . . . 876.2.3 Voice Assistant Answer Structures . . . . . . . . . . . . 886.3 User Studies Methodology . . . . . . . . . . . . . . . . . . . . . 896.3.1 Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . 896.3.2 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . 906.3.3 Question Answer Structures . . . . . . . . . . . . . . . . 936.3.4 Participant Recruitment . . . . . . . . . . . . . . . . . . 946.3.5 Data and Analysis . . . . . . . . . . . . . . . . . . . . . 956.4 Study 1 - Response Types . . . . . . . . . . . . . . . . . . . . . 956.4.1 Questions . . . . . . . . . . . . . . . . . . . . . . . . . 956.4.2 Participants . . . . . . . . . . . . . . . . . . . . . . . . . 986.4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 986.5 Study 2 - Insight Categories . . . . . . . . . . . . . . . . . . . . 1046.5.1 Questions . . . . . . . . . . . . . . . . . . . . . . . . . 1046.5.2 Participants . . . . . . . . . . . . . . . . . . . . . . . . . 1076.5.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 1076.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1116.6.1 Implications for the Design of VA Interactions for PersonalHealth Data Queries . . . . . . . . . . . . . . . . . . . . 1116.6.2 Human Emulation and Unwavering Perceptions . . . . . 1136.6.3 Comparisons With Commercial Personal Health Data Ques-tion and Answering . . . . . . . . . . . . . . . . . . . . 1146.6.4 UEQ+ Semantic Differentials . . . . . . . . . . . . . . . 1166.6.5 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . 1176.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1187 : Enabling In-Situ Databiting on the Smartwatch Lever-aging Touch and Speech . . . . . . . . . . . . . . . . . . . . . . . . 1197.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1197.2 DataWatch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1227.2.1 Design Rationales . . . . . . . . . . . . . . . . . . . . . 122xv7.2.2 User Interface and Interaction Design . . . . . . . . . . . 1267.2.3 Interacting with DataWatch . . . . . . . . . . . . . . . . 1297.2.4 Implementation . . . . . . . . . . . . . . . . . . . . . . 1317.3 Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1327.3.1 Recruitment . . . . . . . . . . . . . . . . . . . . . . . . 1327.3.2 Study Procedure . . . . . . . . . . . . . . . . . . . . . . 1337.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1347.4.1 Interaction Usages . . . . . . . . . . . . . . . . . . . . . 1377.4.2 Motivation for Exploration . . . . . . . . . . . . . . . . 1417.4.3 Speech Interaction for Personal Health Data Queries . . . 1427.4.4 Benefits of Databiting . . . . . . . . . . . . . . . . . . . 1437.4.5 General Reactions to DataWatch . . . . . . . . . . . . . 1467.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1477.5.1 Feasibility of Databiting on the Smartwatch . . . . . . . 1477.5.2 Reflecting on Design Guidelines . . . . . . . . . . . . . 1487.5.3 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . 1507.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1518 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1528.1 Revisiting Lightweight and Transient Interaction for Insight RichExploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1528.2 Towards Speech Interaction for Personal Health Data Exploration 1558.3 Semi-Longitudinal Study Methodologies . . . . . . . . . . . . . 1568.4 Thesis Limitations and Future Work . . . . . . . . . . . . . . . . 1608.4.1 Who is Databiting? . . . . . . . . . . . . . . . . . . . . 1608.4.2 Study With Apple Device Users . . . . . . . . . . . . . . 1618.4.3 Dataset of Desired Queries . . . . . . . . . . . . . . . . 1628.4.4 Part-of-Speech Tagging Versus Large Language Models . 1638.4.5 Enabling Preemptive and Proactive Insight . . . . . . . . 1648.4.6 User Adoption and Behavior Change . . . . . . . . . . . 165xvi9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1679.1 Summary Contributions of the Thesis . . . . . . . . . . . . . . . 168Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204Appendix A: Study Tutorials . . . . . . . . . . . . . . . . . . . . . . . 204Chapter 4 Tutorial Slides . . . . . . . . . . . . . . . . . . . . . . 204Chapter 7 Tutorial Slides . . . . . . . . . . . . . . . . . . . . . . 222Appendix B: Dataset of Collected Queries . . . . . . . . . . . . . . . . 239Chapter 4 Collected Data . . . . . . . . . . . . . . . . . . . . . . 239xviiList of Tables1.1 Recorded personal health data question and answers using Siri onan iPhone 14 Pro running iOS 17.3. We demonstrate this capabilityon an iPhone, rather than an Apple Watch, as functionality is fur-ther limited on the Apple Watch. The question asked, the voice as-sistant’s response, supplementary information displayed on screen,and the success of the interaction are provided. . . . . . . . . . . 44.1 Summary of demographic information, health data collection, smart-watch usage experience, and number of responses reported fromour study participants. . . . . . . . . . . . . . . . . . . . . . . . . 554.2 Query insight categories. Please note, categories are not mutuallyexclusive. ** denotes a new insight category found in our work;* denotes an expanded insight category as compared to [3, 29, 30,108]. Q is a question; C is a command. . . . . . . . . . . . . . . . 574.3 Summary of the daily activities participants were undertaking atthe time of a response, and the relation of the reported queries tothese activities: not related, before, during, after. . . . . . . . . 605.1 Counts of the codes for the attributes, aggregations, and filters areshown, delineated by the in-situ relation for which the query waselicited. We note, that 5 queries contained both time and activitydependent filtering, as such the total count for the filtering columnsresults in 210. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80xviii6.1 Questions and answers used in Study 1. The forward slash denotesthe separation between the choice of heart rate or step count topics,one of which was chosen by the participant for use throughout thestudy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 976.2 Questions and answers used in Study 1. The forward slash denotesthe separation between the choice of heart rate or step count topics,one of which was chosen by the participant for use throughout thestudy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1066.3 Recorded personal health data question and answers using Siri onan iPhone 14 Pro running iOS 17.3. . . . . . . . . . . . . . . . . 1157.1 Summary of demographic information, health data collection, smart-watch usage experience, and number of tracked workouts and re-sponses reported from our study participants. . . . . . . . . . . . 136xixList of Figures1.1 To the left of the grey dashed line is one example of the currentstate of exploration for mobile data visualizations. To the right,databiting is enabled for physical activity related information. Thishas the potential to provide greater influence and insight of thecurrent activity taking place. Notably, more in-depth long-termdata exploration is left for a later time. . . . . . . . . . . . . . . . 31.2 Outline of the components of this thesis, and how each interacts. . 122.1 The lived personal informatics model by Epstein et al. [59] as il-lustrated by Moore et al. [128]. . . . . . . . . . . . . . . . . . . . 153.1 Representations of current mobile data exploration applications arehighlighted, grouped by general information richness: applicationswhich afford glanceable and micro-visualizations (at times com-bined into dashboards) (top) and applications which provide po-tential for heavyweight data analysis (bottom). . . . . . . . . . . . 293.2 In blue (bottom left), we highlight glanceable and micro visualiza-tions for data exploration. In green (top right), we highlight heavy-weight data exploration and analysis. In pink (middle), databiting,as a concept, promotes the need for increasing information richnesswhile exploration remains lightweight and transient. We encouragethe reader to envision how visual data exploration can consume thisarea of the graph. We present these rectangles as sketched illustra-tions to signify the fuzzy boundaries of these forms of explorationand potential variance within them. . . . . . . . . . . . . . . . . . 32xx4.1 A flow diagram of the questions asked within our data collectionapplication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505.1 Components within a personal health data query explored withinthis chapter. We highlight various interrogatives (e.g., ”What was”),aggregations (e.g., ”peak”), data attributes (e.g., ”heart rate”), andfilters (e.g., ”last hike”) to be considered when processing a query. 695.2 Left: Counts of the data types the queries were focused towards.Right: Sub-codes within the Physical Activity code. . . . . . . . . 745.3 Counts of the aggregations (mathematical transforms of the re-quested data) found within the queries. The aggregations are fur-ther noted by explicit and implicit references. . . . . . . . . . . . 765.4 Counts of the interrogatives (elements used to express questions/commandsand intent) found within the queries. . . . . . . . . . . . . . . . . 775.5 Counts of the filtering mechanisms found within the elicited queries(elements use to filter a subset of all data collected). . . . . . . . . 786.1 Study 1 mean UEQ+ ratings with 95% confidence intervals: Qual-ity (a), Behaviour (b), Comprehensibility (c), and Efficiency (d).Ratings are compared by the response types (Value, Open, Binary,Range) and answer structures explored in the study. . . . . . . . . 1006.2 Study 1 mean UEQ+ ratings with 95% confidence intervals for a)Comprehensibility - separated by simplicity and ambiguity; and b)Efficiency - separated by speed and efficiency. Ratings are com-pared across the response types (Value, Open, Binary, Range) andanswer structures explored in the study. . . . . . . . . . . . . . . 1016.3 Participant’s preference of answer structure for each response type(Value, Open, Binary, Range) (a). Participant’s perceptions of voiceassistants (b). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103xxi6.4 Study 2 mean UEQ+ ratings with 95% confidence intervals: Qual-ity (a), Behaviour (b), Comprehensibility (c), and Efficiency (d).Ratings are compared by the insight categories (Current Status andValue (CSV), Historical and Trend (HT), Combination and Com-parison (CC), Goal and Performance (GP), Preemptive and Proac-tive (PP), and Contextual (CT)) and answer structures explored inthe study. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1086.5 Study 2 mean UEQ+ ratings with 95% confidence intervals forComprehensibility - separated by the simplicity and ambiguity.Ratings are compared across the insight categories (Current Sta-tus and Value (CSV), Historical and Trend (HT), Combination andComparison (CC), Goal and Performance (GP), Preemptive andProactive (PP), and Contextual (CT)) and answer structures ex-plored in the study. . . . . . . . . . . . . . . . . . . . . . . . . . 1096.6 Participant’s preference of answer structure for each insight cate-gory (Current Status and Value (CSV), Historical and Trend (HT),Combination and Comparison (CC), Goal and Performance (GP),Preemptive and Proactive (PP), and Contextual (CT)) (a). Partici-pant’s perceptions of voice assistants (b). . . . . . . . . . . . . . . 110xxii7.1 DataWatch supports in-situ exploration of past tracked workoutdata through multi-modal interaction. People can track and con-trol their workouts while viewing their currently collected met-rics , as is typical with current smartwatch health applications.Different to these applications, people can then at any time long-press elements on screen (elements which can be long-pressed,and thus queried, are denoted with a blue dot) to initiate a spokenquery regarding the interacted element . Upon providing a query,DataWatch processes the query to show the resulting output to theperson . In this Figure, a databiting instance involving compara-tive exploration and temporal filtering is queried during a person’sworkout. Please note, screens which appear beside and below thesmartwatch are screens which can be swiped/scrolled to, from thecurrent displayed screen. . . . . . . . . . . . . . . . . . . . . . . 1217.2 DataWatch supports in-situ exploration before starting a workout.On the homepage, scrolling through the many workouts availablefor tracking, people can long-press on a workout type of interest toinitiate a query . In this Figure, a databiting instance using an ac-tivity filter and browse exploration capabilities is demonstrated .Please note, the list of workouts available extends beyond what isshown below the smartwatch on the left, and is reduced here forspace preservation. . . . . . . . . . . . . . . . . . . . . . . . . . 1247.3 DataWatch supports in-situ exploration after completing a work-out. On the summary screen, shown post workout, people canlong-press on a metric of interest to initiate a query . In this Fig-ure, a databiting instance using a temporal filter and single valueexploration capabilities is demonstrated . Please note, contentdisplayed around the smartwatch on the left, can be scrolled to andfrom. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126xxiii7.4 DataWatch’s input screens are shown demonstrating live query in-put. By default the keyboard is brought forward (left), which canbe switched to dictation capture (right) by selecting the micro-phone icon at the bottom right of the screen . Input can eitherbe cancelled or sent for processing through the controls at the topof either input screen . . . . . . . . . . . . . . . . . . . . . . . 1287.5 When a provided query is either not valid or resolving the data wasnot successful, a contextual message is displayed. Three potentialmessages are shown here, while others exist depending on the issuefaced. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1297.6 Instances of databiting (represented by colored blocks) are high-lighted within workout timelines and organized by participants (P1-P5) and days of use (each row). Note that the spacing betweenicons and the physical length of the workouts (beginning to end)do not correspond to any specific results or times. However, thesize of the colored databiting blocks reflects the total explorationtime for each instance. . . . . . . . . . . . . . . . . . . . . . . . 1387.7 Instances of databiting (represented by colored blocks) are high-lighted within workout timelines and organized by participants (P6-P10) and days of use (each row). Note that the spacing betweenicons and the physical length of the workouts (beginning to end)do not correspond to any specific results or times. However, thesize of the colored databiting blocks reflects the total explorationtime for each instance. . . . . . . . . . . . . . . . . . . . . . . . 1397.8 Instances of databiting (represented by colored blocks) are high-lighted within workout timelines and organized by participants (P11-P12) and days of use (each row). Note that the spacing betweenicons and the physical length of the workouts (beginning to end)do not correspond to any specific results or times. However, thesize of the colored databiting blocks reflects the total explorationtime for each instance. . . . . . . . . . . . . . . . . . . . . . . . 140xxiv8.1 For discussion purposes we introduce the above framing of lightweightand transient data exploration. We provide this here, rather than inChapter 3, due to the fuzzy and hard-to-quantify nature of theseboundaries. As such, with this figure we aim to provoke thought-ful discussion rather than establish definitive measures early on.While we recognize that the smartphone, tablet, and laptop canprovide glanceable and initial interactions we place them on theright of the figure as this is where many interactions are meant totake place. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153xxvAcknowledgementsFirst and foremost, I would like to express my gratitude and appreciation to mysupervisor, Dr. Pourang Irani. I will be forever grateful for your belief in me, theopportunities given, guidance, conversations (both academic and non-academic),encouragement, and support provided throughout this journey. Your initial decisionto hire me while an undergraduate student, and everything that has come since, hasforever shaped my and my family’s future. This is something I will never be ableto repay, and can only hope to carry forward. Thank you.I would like to thank my internal committee members, Dr. Karon MacLeanand Dr. Ramon Lawrence, for their time and effort in meeting with us throughoutand providing insights and feedback. Transferring from the University of Manitobato the University of British Columbia affected the normal program deadlines, andboth Dr. MacLean and Dr. Lawrence accepted the internal committee role withouthesitation. I also want to thank my external committee members, Dr. Petra Isenbergand Dr. Megan Smith for their time and effort in evaluating this thesis.A special thank you to all my external collaborators and lab-members (of boththe HCI Lab at the University of Manitoba and OVI Lab at the University of BritishColumbia). In particular, I would like to thank Dr. Yumiko Sakamoto, Dr. Bong-shin Lee, Dr. Eun Kyoung Choe, Dr. Sandra Bardot, Charles-Olivier Dufresne-Camaro, Anuradha Herath, Shariff Faleel, Dr. Maxime Daniel, Dr. Fouad Alallah,Samar Sallam, Preeti Vyas, Dr. Ali Neshati, Dr. Jaisie Sin, Dr. Simon Perrault, andDr. Kening Zhu. To my collaborators, working with you has taught me everythingI know about conducting research and has shown me the level of excellence re-quired to succeed. To my lab-members, I am grateful for the countless hours spentdiscussing research problems, brainstorming solutions, providing research support,and the fun times along they way. Meeting all of you has shaped this journey forxxvithe better, and I hope to continue our relationships beyond this degree.To my family, thank you for instilling in me the values needed to completesuch a degree. Without your nurturing and encouragement this would not havebeen possible. To my partner, Danica, we did it. Thank you for believing in me,supporting and encouraging this degree, moving across the country with me, pa-tiently listening while I ramble about ideas (even when you barely knew what I wastalking about), reminding me to celebrate the highs, and for comforting me duringthe lows. This has been a shared journey to say the least, and I am now more thanever excited for our future.Finally, I would like to extend my gratitude to all those who have directly orindirectly contributed to my research and are not named above. Particularly, I wantto thank the participants who generously gave their time and effort, and the fundingagencies that made this work possible. I hope my work, as well as future motivatedwork, can carry forward a positive impact within many lives.xxviiChapter 1Introduction“If the smartwatch can’t eventually get smarter and more useful, itrisks becoming a footnote.” - Walt Mossberg, The VergeThe potential for smartwatches to become truly influential personal health toolslies in their ever-growing capabilities. Built with a wide range of sensors, process-ing power, network connectivity, and various interaction modalities (i.e., visual, au-ditory, and haptic output as well as touch, speech, and gesture input), smartwatchesare well-equipped to allow for collection and monitoring of personal health data inreal-time. So much so, the smartwatch’s current ability to ubiquitously collectfitness and other health metrics is a major contributing factor to its widespreadadoption [49, 52].Data collection is just the first step. It is only once this data is explored and an-alyzed that insights can be drawn and meaningful actions taken. The smartwatch,worn on the wrist, is seamlessly integrated into daily activities such as sleeping,jogging, working, or simply while on-the-go. This level of integration makes thesmartwatch an ideal tool for continuous data collection. Moreover, and impor-tantly, this integration offers capability for convenient and accessible exploration[148, 192]. In fact, individuals are becoming increasingly dependent on the smart-watch, even more so than their other devices, for analyzing their collected datathroughout their daily activities [34]. This in-situ exploration on the smartwatch iscrucial, as it empowers individuals to quickly and easily gain actionable insightsfrom their health data, enabling them to adjust their behavior in real-time, to opti-mize their activities, and achieve their goals [40, 62].The smartwatch’s full potential as a personal health tool remains largely unreal-ized, however, due to inadequate capabilities for data exploration [37, 70, 131]. The1largest barrier for exploration on the smartwatch is often seen as its small screensize, limiting typical touch interactions and visual output [136, 155]. As such,guidelines for exploring data specifically on the smartwatch often focus on effi-ciently designing micro and glanceable visualizations [3, 13, 15, 26, 68, 85, 135,136] (i.e., small screen visualizations that can be viewed and understood withinfive seconds [13, 68]). While beneficial in their own right, these visualizations stillonly allow for limited and discrete exploratory capabilities. This in turn, remainsto hinder utility and access to a broader range of desired insights [37, 70, 131].By allowing users to further explore their data directly on the smartwatch, in-dividuals can quickly and easily gain actionable insights beyond what is currentlypossible. To illustrate this necessity we provide a storied example, illustrated inFigure 1.1: Consider a hiker, Sam, who checks their smartwatch during a hike.Current systems only display a handful of metrics, such as the current pace. Sam,while viewing their pace, is uncertain if they are slower than usual. This lack of in-formation richness and overall usefulness of the interface ultimately hampers fullengagement and a larger range of benefits that could be available to smartwatchusers [70, 145]. Through enhanced exploration, Sam could tap on the pace num-ber, and ask “How does today’s pace compare to my last six hikes?”. A graphappears, revealing that their pace is indeed slower than their average. This quickin-situ exploration on the smartwatch provides a deeper and relational understand-ing, enabling Sam to immediately adjust their pace towards achieving their fitnessgoals.1.1 Research ChallengesWhile the exploration in the above storied example may seem simple, there area handful of underlying research challenges that need to be solved before enablingsuch exploration directly on the smartwatch.To further highlight the complexity of the problem, and to better situate thiswork, we take a moment to demonstrate capability of such querying of data oncurrently marketed devices. To our knowledge, certain Apple devices through in-teraction with their voice assistant Siri, are the only commercial devices which can2Figure 1.1: To the left of the grey dashed line is one example of the current stateof exploration for mobile data visualizations. To the right, databiting is enabledfor physical activity related information. This has the potential to provide greaterinfluence and insight of the current activity taking place. Notably, more in-depthlong-term data exploration is left for a later time.answer some personal health data questions; other commercial devices, and theiroperating systems, often only recognize key words and simply provide a promptto open a respective health application. As such, we asked Siri a range of personalhealth data questions3. These questions and responses can be seen in Table 1.1.Notably, aside from questions pertaining to current and recently captured metrics,responses are unsuccessful or unsupported. Furthermore, when responses are un-successful, they often provide outright incorrect information. Thus, it is clear thatthe ability to accurately and efficiently process and respond to a diverse range ofhealth data queries on the smartwatch necessitates further efforts.3This is a new feature (as of December 2023) that only works with iPhones and iPads runningiOS and iPadOS greater than 17.2 and Apple Watch Series 9 and Ultra 2 running watchOS greaterthan 10.2. Queries were asked on an iPhone 14 Pro running iOS 17.4.3Table 1.1: Recorded personal health data question and answers using Siri on an iPhone 14 Pro running iOS 17.3. Wedemonstrate this capability on an iPhone, rather than an Apple Watch, as functionality is further limited on the Apple Watch.The question asked, the voice assistant’s response, supplementary information displayed on screen, and the success of theinteraction are provided.Question Vocal Response Supplementary Information SuccessfulWhat is my current step count? 8739 steps8739StepsTodayYesHow far have I walked this week? 7.2 km7.2 km2.4 km Daily AverageWalking + Running DistanceMay 26 - June 1, 2024YesWhat is my move ring at? You’ve burned 210 out of your 500 calorie goalMove RingToday 10:02 amYesWhat’s my average distance walked in my last seven walks? You’ve walked or run 2 km today2 kmWalking + Running DistanceTodayNo (IncorrectInformation)How much time did I spend walking in September 2023? 124.8 km124.8 km4.1 km Daily AverageWalking + Running DistanceSeptember 2023No (IncorrectInformation)What was my average running pace in March? I can’t help you with thatNo (Unable toRespond)How far have I walked since January 1st? 3.7 km3.7 kmWalking + Running DistanceJanuary 1, 2024No (IncorrectInformation)4The provided examples in Table 6.3 highlight a limitation in current solutions,and are underscored by a variety of research challenges that remain to be explored.First, we must understand what health data queries people feel are important. Fur-thermore, we must recognize when, where, and why these queries arise to affordcontext and understanding to the handling of the query. Then, specific understand-ing is needed of the natural language used which dictates how we can processsuch queries. Moreover, exploring how the smartwatch responds to these queriesis equally as important. Only through addressing these challenges can we begin torealize the potential for broader, lightweight, and transient health data interactionon the smartwatch. Below, we discuss these research challenges in further detailand, throughout, the structure of the remainder of the thesis to follow. This outlinecan also be viewed graphically in Figure 1.2.1.1.1 ConceptualizationCan we conceptualize data exploration that represents a lightweight and tran-sient bridge between short- and long-form data exploration?The first focus of this thesis is on the conceptualization of our targeted explo-ration capability of the smartwatch. Traditional research in data exploration oftenconcentrates on either short, glanceable, or longer, more detailed, exploration. InChapter 3, we position our work through a classification of what we term databit-ing—an intermediate level of data exploration, complementing short- and long-form exploration, that remains to provide richer insight while being transient andlightweight. We conceptualize databiting through metaphor, highlight the envi-sioned benefits of such exploration, and discuss associated research challenges.Chapter 3 approaches this conceptualization through a broader perspective of dataexploration, hinting at personal health data exploration on wearable devices whereappropriate.1.1.2 WhatWhat personal health data queries are of interest to people for exploration ontheir smartwatches throughout their daily lives?5In order to move forward in enabling databiting on the smartwatch, Chapter 4looks to first understand what specific personal health data queries people have forexploration on their smartwatch, throughout their daily lives. Given the wide rangeof data available and the diverse contexts in which smartwatches are used, peoplelikely have varied and specific desired exploration needs. Currently, our knowl-edge in this area is limited. Research is comprised of lab-based or survey studies[3, 154], which do not account for in-the-wild experiences, suffering from potentialrecall bias [75]. Furthermore, most prior work does not share concretely reportedqueries, rather they highlight broader insights into exploration practices. To ad-dress the what challenge, Chapter 4 details a week-long in-the-wild study designedto elicit queries from participants. We utilized a custom-built smartwatch appli-cation which enabled participants to quickly and easily record a desired personalhealth data query and other contextual information. Throughout the study, we col-lected 205 personal health data queries which have been made publicly availableto support and facilitate future research in this area.1.1.3 When, Where, WhyHow does being in-situ influence the desired personal health data queries ofsmartwatch users?Our next focus is on understanding the context of the personal health dataqueries by examining when, where, and why these queries are desired. Under-standing when and where a query arises can help to give it context. Traditional,desktop, data exploration often happens in one static setting. In contrast, smart-watch data exploration, which can happen anytime and anywhere, can occur undervaried contexts and while in-situ. This in turn, may influence both the questionsposed and even the answers provided. Furthermore, the why challenge looks atincreasing our knowledge of the broader exploratory insights that are desired. Fur-ther to Chapter 4, we analyze additional contextual information captured duringour study to explore these dimensions. We find six insight categories to which ourcollected queries exist within. Compared with previous work, we highlight one en-tirely new Preemptive and Proactive insight category, as well as expanded Current6Status/Value and Contextual insight categories.1.1.4 HowHow can we expand the expressivity of data exploration on the smartwatch,through both multi-modal input and output modalities, within the confines of thesmartwatch’s capabilities and data exploration needs?With a concrete understanding of what personal health data queries are desired,we then focus on how to allow for such exploration. Importantly exploration mustremain lightweight and transient to support databiting on the smartwatch. Giventhe limited screen size and input capabilities of smartwatches, designing an intu-itive and efficient user interface is key. As such, this research challenge focuses onhow to support both both input and output modalities.Through analysis of the personal health data queries collected, how can we pro-cess the components within a query to efficiently handle naturally spoken querieson the smartwatch?Without the screen real estate for menus, toggles, sliders, etc., touch interac-tions can be difficult and limited on a smartwatch [61]. Speech, on the other hand,allows for the conveyance of more complex queries, a low barrier in expressingintent, flexibility in phrasing and querying [8, 42, 182, 184], and does not requireas much screen real estate. Through the use of speech and touch, benefits of bothcan be utilized to enhance smartwatch data exploration. As touch is an alreadyestablished input modality, we focus our work in Chapter 5 on examining how thepreviously collected queries are phrased. We analyze the components of speechwithin the queries, including interrogatives, data sources, aggregations, and fil-tering mechanisms. By understanding these elements, we aim to later develop apart-of-speech tagging framework for processing and responding to such queriesdirectly on the smartwatch.How should voice assistant answers be structured to support auditory responsesto personal health data queries?7Output modalities are equally important towards enabling databiting on thesmartwatch. While research exists on micro and glanceable smartwatch visualiza-tions [3, 13, 15, 26, 68, 85, 86, 135, 136], Chapter 6 focuses on exploring auditoryresponses to personal health data questions. Previous work with voice assistants forquestion and answering focuses only on general knowledge queries (e.g., ”What isthe weather today?”) [2, 71, 127]. By exploring voiced responses for personalhealth data queries, we aim to offer an output modality that leverages the strengthsof auditory feedback, which can be beneficial in times where the visual system isoverloaded [20, 147], as often seen on-the-go.Through the use of a custom-built pseudo voice assistant, implemented usingGoogle’s Web Speech API, Chapter 6 provides an online survey study where par-ticipants directly interacted with the developed voice assistant. Participants wouldask a guided personal health data question and receive a voiced response in oneof three defined answer structures (minimal, keyword, and full sentence). Re-sponses were ranked by participants for quality, behaviour, comprehensibility, andefficiency. Notably, full sentence answers were better perceived across all rankedmetrics. The insights gained from this chapter can be used to further enhance thedatabiting experience on smartwatches, ensuring that voiced responses are effec-tive and appropriate.1.1.5 Databiting ValidationFinally in Chapter 7, we integrate the knowledge gained in the preceding chap-ters to build a smartwatch workout application, DataWatch, that supports not onlyworkout tracking, but also enables databiting. Participants installed DataWatch ontheir own smartwatch, using it throughout the week-long study within their day-to-day lives. We then validate and observe the application’s use, and any subsequentdatabiting that takes place. Through the findings of our study, both quantitativeand qualitative, we showcase the concept of databiting as a valid means for dataexploration on the smartwatch, pushing the boundaries of how the smartwatch isperceived for data exploration. This final step brings together all aspects of theresearch conducted throughout this thesis to demonstrate and validate the practical8feasibility and benefits of databiting on smartwatches for lightweight and transientpersonal health data exploration anytime and anywhere.1.2 Research ScopeThe scope of this thesis follows a funnelled approach. In Chapter 3, we in-troduce the concept of databiting from a broad perspective. We conceptualizelightweight and transient exploration of any data on any device, while also high-lighting benefits and challenges present. As the thesis progresses, the focus funnelstowards a more specific context: personal health data on smartwatches. This fo-cus is motivated by the immediate impact that can be realized through advancingcapabilities of an already mature and widely adopted tool like the smartwatch.Furthermore, the smartwatch is a device that may have the most to benefit from alightweight and transient form of data exploration, due to its immediate access to arange of personal data being collected, limited screen real estate inhibiting lengthyexploration, and typical and accessible on-the-go usage.While Chapter 4, Chapter 5, and Chapter 6 focus on exploration of a widerange of personal health data, our final chapter, Chapter 7, specifically focuses onexploration of personal fitness/workout data on the smartwatch. This narrowed fo-cus is justified by the immediate availability and practicality of fitness data on thesmartwatch compared to any and all types of personal health data (e.g., women’shealth, mood, or nutritional information). Moreover, our elicitation study in Chap-ter 4 revealed that personal fitness and workout data were the most queried byparticipants, highlighting the high level interest in exploring such data.By starting broad and funnelling inwards, we ensure that knowledge gained cantranscend the immediate context of this thesis and as such apply to various types ofdata and devices. Importantly, however, by concluding with a focus on explorationof personal fitness data on the smartwatch, we create opportunity for our work toprovide immediate impact.91.3 Thesis ContributionsThis thesis contributes the following high-level contributions, while lower-level, chapter-specific, contributions are further discussed within each chapter andwhere applicable:C1: An introduction of the term databiting, conceptualized as lightweight andtransient data exploration, that bridges the gap between quick, glanceabledata visualizations and more extensive, detailed data analysis. Further toproviding a conceptualization of databiting, we also discuss potential bene-fits and highlight future, and necessary, research directions. This foundationsets the stage for developing more intuitive and effective ways for peopleto engage with their personal data, both on smartwatches and other mobileplatforms.C2: An elicited, and now publicly available, dataset of 205 personal health dataqueries desired for exploration on a smartwatch throughout daily life. Thisdataset, coupled with a thorough qualitative and quantitative analysis, offersvaluable insights into what personal health data query needs people have,including when, where and why these queries arise.C3: Interaction requirements for lightweight and transient exploration of per-sonal health data on smartwatches. We identify key dimensions of our cap-tured queries, such as interrogatives, data sources, aggregations, and filter-ing mechanisms, which are crucial for enhancing input methods, particularlynatural language processing capabilities. Additionally, we gather insights onuser preferences for different output structures from voice assistants, focus-ing on perceived quality, behavior, comprehensibility, and efficiency. Thesefindings provide essential guidelines for optimizing both input and output in-teractions, ensuring that data exploration on smartwatches can be supportedin being lightweight and transient.C4: Development and validation of DataWatch, an Apple Watch application thatdemonstrates the practical implementation of databiting. By integrating10multimodal interactions, specifically touch and speech, DataWatch enhancesin-situ exploration of past workout data, demonstrating the feasibility anduser benefits of providing lightweight and transient data exploration capabil-ities directly on the smartwatch.11Figure 1.2: Outline of the components of this thesis, and how each interacts.12Chapter 2Related WorkThroughout this chapter, we highlight research pertinent to the broader topicswithin this thesis. These include personal informatics, wearable computing, gen-eral smartwatch usage, and smartwatch data exploration. In each chapter to follow,chapter-specific related work is provided to better situate and justify topics andmethods explored throughout.2.1 Personal InformaticsPersonal informatics (PI), an interdisciplinary field encompassing human-computerinteraction and visualization research, emphasizes the collection, comprehension,and utilization of personal data [59, 107]. The core principle of PI is that of uti-lizing data to gain self-knowledge and enhance overall well-being. Lupton [118]describes this as “practices in which people knowingly and purposely collect in-formation about themselves, which they then review and consider applying in theirlives.”The Quantified Self Exploring personal health data to gather insight into one’spersonal health journey has long been a goal of “Quantified Selfers” (QS) [118].Thanks to the increasing capability and prevalence of mobile and wearable devices,a broader audience is now experiencing PI. These devices, equipped with advancedsensors for data collection, empower individuals to track a wide range of healthmetrics, from physical activity and sleep patterns to heart rate and stress levels. Asa result of the wider spread adoption of wearables, PI is transitioning from a niche,QS, interest to now mainstream practice.While collecting personal data is essential, true value emerges when individu-13als can explore and interact with their data presented in meaningful ways. Quan-tified Selfers conduct exploration for various reasons, such as maximizing perfor-mance [28, 197], reflecting on overall health [10, 28, 36, 40, 107], reinforcing priorknowledge and understanding [36, 58], viewing trends [28, 58], satisfying simplecuriosity [28, 197], and comparing data [3, 28–30].These reasons for exploration afford explicit queries which can be further cat-egorized [3, 28, 29, 57, 98, 107]. These categories include questions about currentstatus (e.g., “What is my current step count?”), historical data (e.g., “What was myaverage daily step count last month?”), goals (e.g., “How many more calories doI need to burn to reach my daily goal?”), discrepancies (e.g., “Did I exercise asmuch this week as I expected to?”), context (e.g., “How has the time of my walkaffected my pace?”), and factors (e.g., “Has losing weight and getting more sleepbeen helping my mood?”).Modelling Personal Informatics Research has modeled the lived personal infor-matics journeys that people take when exploring their collected data for the varietyof reasons above. Li et al. [107] first proposed a model to help understand howpeople use personal informatics tools. This model comprises of five stages, whichinclude preparation, collection, integration, reflection, and action. The main out-come of this model is actionable insight that allows a person to create positivebehaviour change from their reflection of the collected data. This model has beeniterated upon many times. In fact, one year later, Li et al. [108] further divided thereflection stage into maintenance and discovery stages to better highlight the typesof reflection that can be done.Other researchers have further iterated upon this model. As the above modelonly highlights reflection as a separate and subsequent action to collection, Choeet al. [28] discuss how reflection is not always a separate stage, and often occurs intandem with the data collection itself; this is especially true when using modern fit-ness trackers and smartwatches. Finally, Epstein et al. [59] further added decidingto track and selection of a tracking tool to the beginning of Li et al.’s model, whilealso adding lapsing and resuming to the end. This highlights the increasingly livedexperiences of people, by suggesting there exists a repeated and looping nature14Figure 2.1: The lived personal informatics model by Epstein et al. [59] as illus-trated by Moore et al. [128].across many of the stages. See Figure 2.1 for a visual representation of this model.Interestingly, these stages have not been equally studied [60]; of importance tothis thesis is the integration stage, which is one of the more understudied. In-tegration involves preparing for reflection by combining and transforming data[107, 128] and is one of the more complex stages to facilitate. This complexityarises from the uniquely varied and large number of data exploration needs re-quired, and the difficulty in providing tools that afford such potential exploration.Research has attempted to curb this complexity through narrowed design spacesand algorithms that detect a greater number of insights [91, 92]. However, thesesolutions are often fixed and do not involve user interaction and/or control whichcan have further negative effects [10, 92]. It is crucial that we develop more dy-namic and user-centered integration tools. These tools should enable individualsto generate unique insights through data transformation across tracking and ac-tion stages, providing greater reflection and actionable insights beyond predefinedmetrics.2.2 Wearable ComputingThad Starner [185] defined wearable computers as “any body-worn computerthat is designed to provide useful services while the user is performing other tasks.”15Today, examples of wearable computers include smartwatches, smartrings, cheststrap sensors, smart soles for shoes, body-worn inertial measurement units, smartglasses, and brain-computer interfaces [166]. As of March 2024, wearable saleswere up 10.5% from that of the previous year, largely driven by the sale of newsmart glasses, and sales are projected to continue to grow at around 4% per yearup to around 650 million units sold in 2028 [112]. As it stands, one of the mainservices offered by wearables is to ubiquitously capture and track personal healthmetrics from the wearer [34, 69], which can then be later explored to support per-sonal informatics and healthier living. These fitness metrics, and algorithms as-sociated with their use, often track physical activity information (i.e., step count,distance, pace, time, cadence, etc.), sleep quality (i.e., hours slept, sleep rhythm,etc.), caloric burn, heart rate, energy expenditure, and oxygen saturation [43, 170].Attributes of Wearables It is important to note that wearable computers offerdifferent and adapted services from that of typical desktop computers and other mo-bile technologies (e.g., smartphone or tablet). With this in mind, the goal shouldnot be to replace these traditional devices; rather, we can complement them byleveraging attributes that are unique to a wearable device. From a wearer’s per-spective, Steve Mann described six key attributes of wearable computing that makewearables, and their services, unique to that of other devices [123]:1. Nonrestrictive to the wearer: Wearable devices should not impede the wearer’sability to physically perform other tasks. The physical miniaturization ofsensors and hardware in today’s wearable and on-body devices, has greatlyallowed for for unrestricted use to become a reality [170]. Particularly, thecompact size of smartwatches, and wrist-worn location, ensures they do notphysically hinder daily activities, making them great devices to have on usat all times. Importantly, this is achieved while still offering a wide rangeof sensors, robust processing power, network connectivity, and various inter-action modalities (i.e., visual, auditory, and haptic output as well as touch,speech, and gesture input). Together, this makes the smartwatch more ver-satile than many other wearables, such as a chest-worn heart rate monitor,16which may require to be even smaller than the smartwatch or worn in a lessaccessible location.2. Non-monopolizing the wearer’s attention: Wearable devices, and their ser-vices, should be designed under the assumption that computing will be asecondary activity, not the primary focus of attention. For example, taskssuch as hiking, jogging, working, or walking to and from locations shouldremain the primary activity and focus, while interaction with a wearable suchas the smartwatch is second [14]. In fact, not monopolizing interaction is im-perative. If interaction with a wearable, or any mobile device for that matter,requires too much attention and focus during a primary task, then peoples’interaction performance will decrease [164], they will miss elements withintheir surroundings [82], and will change aspects of their primary activity,such as how they walk [44]. Consequently, any interaction with the wear-able device during these times must be quick and lightweight such as to notdistract from the primary task at hand.3. Observable by the wearer: The output medium of a wearable device shouldbe constantly perceptible to the wearer. Many wearables offer haptic feed-back to convey pertinent information [9, 81, 147]. Smartwatches, affordedby the inclusion of a screen, also offer visual feedback to allow observation.However, to truly support non-monopolized interaction, it is crucial to pro-vide alternative feedback options for when the visual system is overloaded[19]. In such cases, auditory or natural language output can be an effectivemeans to allow the wearer to receive observable feedback without needingto visually engage with the wearable [151].4. Controllable by the user: Wearers should be able to take control of the de-vice whenever they wish. The smartwatch is wrist-worn making it easilyaccessible. This body-worn location makes it a perfect device for user inputand engagement in many daily scenarios of use [68, 148]. In fact in certaindaily scenarios, such as at the gym, use of the smartwatch can eclipse thatof other devices such as the smartphone [34]. This access and convenience17allows users to interact with their device on the go, whether they are workingout, commuting, or undertaking other activities. Importantly, however, inter-action with the smartwatch often comes at the cost of monopolization (asseen in the second attribute above). This need to support control even dur-ing differing usage scenarios is the reason that some research has exploredusing the smartwatch during eyes-free, walking, and encumbered conditions[137, 155, 176, 199, 200].5. Attentive to the environment: Wearable devices should provide situationalawareness to the wearer. As the use of wearables goes beyond a traditionaldesktop setting as is used throughout a person’s daily life [125, 148], thesmartwatch and its sensors have the potential to recognize these varied usagescenarios and even the wearer’s bodily information, incorporating gatheredinformation when and where necessary. For instance, a smartwatch can de-tect when a user is exercising and adjust its display to show relevant fitnessmetrics [86].6. Communicative to others: These devices should function as communicationmediums when desired by the wearer. More so than other wearables on themarket, one of the main uses of the smartwatch is as a communication device[148, 192], providing quick access to notifications and networking capabilityto allow messaging and other communication. Traditionally, interaction withthe smartwatch, such as for communication, has been a challenge due to thesmall screen size [134]. For this reason, many research works have exploredadding varied on- and around-device input capabilities [25, 66, 79, 199].However, many of these works often do not take into account commoditysmartwatch capabilities and common interaction modalities, such as gen-eral touch interactions and the now powerful natural language processingwhich underlies speech input. Furthermore, many gestural techniques ex-plored can be tiring and slow [4], while some touch techniques often invoke alarge number of pre-touch efforts (i.e., lifting and moving your finger to con-duct repeated interactions) which should aim to be kept to a minimum [176].Leveraging simple commodity interactions can provide efficient and effec-18tive communication capabilities, allowing intuitive interactions for users tocommunicate with one another, or perhaps even with a system when explor-ing data.2.3 General Smartwatch UsageSmartwatches are one of the most wide-spread of any wearable currently on themarket [168]. It is estimated that in the next four years, until 2028, the smartwatchmarket will continue to grow about 5.4% from current sales, to reach roughly 230million units sold worldwide [49]. As smartwatches already offer maturity, con-venience, and widespread adoption, this thesis focuses on pushing the boundariesand capabilities of the smartwatch over that of other wearables which offer lesspotential for interaction with collected health data. In the following paragraphs,we discuss reasons for smartwatch use, general usage details, and long-term usagetrends.Reasons for Adoption and Use People adopt smartwatches for a variety of rea-sons, including performance expectancy, which is the belief that the smartwatchwill enhance their performance [32]. Perceived usefulness also plays a crucial role,as people expect the smartwatch to help them achieve their goals [126]. Addition-ally, the compatibility of smartwatches with other devices and services encouragesadoption [32]. Lastly, aesthetics are an important factor, as the design and appear-ance of smartwatches contribute to their attractiveness, especially as they are wornon the body [32].Subsequently, once adopted, research in recent years has abundantly exploredthe reasons for use of both smartwatches and fitness trackers [34, 69, 88, 111,125, 148, 165, 192, 193, 203]. These works, spanning six years from early adop-tion to current trends, found that smartwatches are generally used for time check-ing, notifications, activity tracking, communication, information management (i.e.,weather, stocks, etc.) payment transactions, and navigation.Increased smartwatch adoption has largely been driven by the smartwatch’sperceived capacity for health data collection and usefulness in promoting health19and well-being [52, 174]. Not only can the smartwatch track general activitymetrics, something for which proximity to the body is a key factor [130], butthey also have the potential to aid in detecting COVID-19 symptoms and provideconstant health monitoring, making them valuable for individuals of all ages, in-cluding those with dementia, depression, and high-stress [113, 206]. Now, smart-watches are expected to soon offer further sophisticated health monitoring featuresfrom those mentioned above [74, 89, 93, 99, 186], including broader well-being,stress management, breathing detection, and medication/diet tracking. This holis-tic health tracking primes the smartwatch for a lead role with respect to not onlypersonal health data collection but also immediate access to the data, for greaterusefulness and performance expectancy, by allowing for increased and in-situ in-sights.Usage Metrics Interactions with smartwatches are typically brief, with mostlasting less than 5 seconds [148, 192], with average of only 1.9 seconds [148].Activity-related interactions tend to be slightly longer, ranging from 5 to 10 sec-onds and averaging up to 18 seconds [103, 148, 192]. However, applications suchas maps and messaging can greatly extend interaction times, reaching up to 45seconds [148], suggesting a natural maximum length for individual smartwatch in-teractions. Visuri et al. [192] categorize these interactions into two types: peekingor glancing at the screen, which accounts for about 65% of interactions, and moreengaged interactions, which make up the remaining 35%. They also note that themajority of interactions, 82%, are initiated intentionally by the user, with notifica-tions prompting the remaining 18%. This active and intentional interaction with thesmartwatch bodes well for its further integration into daily life, as it demonstratespeoples’ willingness to engage with the smartwatch for various purposes beyondjust passive notifications. Such engagement underscores the versatility and util-ity of smartwatches, highlighting their role not only as convenient tools for quickglances but also as intentional parts of a larger device ecosystem.Smartwatch interactions, whether quick glances or more engaged actions, oc-cur in various locations and during many daily activities, such as walking to andfrom destinations, commuting, working out, or performing tasks at work [148,20192]. The broad range of usage scenarios and the convenience of a wrist-worndevice provide significant advantages over even the smartphone, as smartwatchesoffer quick and easy access to information. This is particularly beneficial duringactivities where using a smartphone might be cumbersome or impractical (e.g., dur-ing a hike or walking to the transit station). Notably, during activity tracking, Chunet al. [34] have found that interactions with the smartwatch in fact surpass thosewith a smartphone, emphasizing the smartwatch’s ease of use and accessibility dur-ing in-situ activities. With this greater integration into daily life, it is important thatwe provide people as much access to the information that they desire, anytime andanywhere.Long-Term Usage Trends Interestingly, usage typically declines over time forboth activity trackers [38, 41, 70] and smartwatches [131, 142, 173]. This declinein use is often attributed to a mismatch between user expectations/goals and theactual offerings of the smartwatch [38, 41, 70, 142]. Users may start with spe-cific goals in mind, such as improving overall health and fitness, but then findthat the smartwatch does not adequately support these goals through capturedmetrics, accuracy of metrics, potential for exploration, and expected functional-ity [70, 142, 144]. This limited functionality can lead to overall dissatisfaction,causing users to gradually abandon the smartwatch [131]. Externally, people mayeven stop using their smartwatch due to a lack of motivation and/or life factors thatthat can interrupt/de-prioritize data tracking [7].Research has identified several factors, however, that can motivate continueduse of smartwatches. One key factor is perceived usefulness through utilitarianaspects (i.e., tracking quality), where people find value in the tracked data that iscollected [33, 78, 142, 173]. However, hedonic aspects (i.e., fun and pleasurableaspects) are also important as they positively contribute to enjoyment and self-expression which in turn increases satisfaction and then continuation of use [31, 78,173]. Afforded customization and personalization, through utilitarian and hedonicaspects further improve perceived value which in turn factors into continuation ofuse [117]. Finally, the convenience characteristics of smartwatches, such as theirwrist-worn accessibility, ease of use, and quick access to needed information all21play a significant role in maintaining continued use [142].2.4 The Smartwatch and Personal Data ExplorationUser Needs and Exploration Goals Research has shown that people have di-verse informatics needs and motivations specific to health data exploration onthe smartwatch. Common reasons for using the smartwatch as a personal healthtool include high-level understanding of activities [58], comparative exploration[3, 161], accountability and participatory interaction [133, 194], motivation [3],as well as goal- and performance-based use [3, 6, 22]. Importantly, the reasonsfor tracking health data can evolve and change over time [59], further requiringdiversity in the tools provided.Research has also noted that data exploration is desired not only during anactivity, but also immediately before and after [3, 148]; this reflection, using asmartwatch, temporally near an activity can be increasingly intertwined with theactivity itself, and thus beneficial to influence real-time and actionable decisions[3, 69, 107]. In turn, this can further help to set and achieve meaningful personalhealth goals. For instance, individuals might want to adjust their activity levelsbased on their daily step count or improve on their sleep quality and length byanalyzing sleep data immediately after waking up.Data Exploration on Smartwatches: Challenges and Opportunities One ofthe current challenges in achieving the above mentioned desired exploration, isthat current smartwatches standardize visualizations and displayed metrics, andthus a person’s underlying needs and goals [180]. Simply put, people with similarsmartwatches and operating systems are presented with identical visualizations andmetrics (albeit with their own numeric values). Given the diverse and highly uniquenature of personal health data exploration, it is essential to develop tools that allowfor unique and personalized exploration and understanding of the tracked data.Of the metrics that are displayed, Spiel et al. [180] highlight that while smart-watches and other fitness trackers are designed to capture quantitative metrics, theyoften overlook the qualitative experiences, understanding, and reflection associated22with these metrics. This disconnect forces users to translate qualitative goals, suchas losing weight, into simple quantitative metrics, like increasing daily step counts[138], without reflecting deeper.From the user’s perspective, it remains difficult to find mobile health applica-tions that suit their needs and health goals [161], with people citing a lack of infor-mation richness and overall usefulness regarding their smartwatch health applica-tions [145]. Neshati et al. [134] note that smartwatch users are seeking answers totheir personal health data queries that are simply not available to them; these typesof desired queries are exemplified as ”how am I doing so far?” or ”how am I doingcompared to my friend, Jane?” [108], and are not currently supported within a timeframe that supports common smartwatch use. These missing features, underscoredby our limited knowledge of concretely reported queries, hamper full engagementand a broader range of benefits [70].While a growing body of research revolves around smartwatches and wearableactivity trackers, only about 10% of such research is centered around the collecteddata and the ability to convey appropriate meaning and function [171]; other re-search focuses on topics such as privacy, acceptance, adoption, and abandonment,behaviour change, and hardware. Of the works that do focus on data exploration,they often highlight the technical complexities of visually rich personal health datarepresented on a small smartwatch screen which creates an exploration environ-ment with limited usability, customization, and interaction [34].Much of the limited exploratory capability on the smartwatch stems from thesmall screen of a smartwatch which presents several challenges, including screenocclusion and fat finger issues [172] during interaction. The limited screen sizeconstraints not only inhibit interaction but also visualization options, making itdifficult to display comprehensive data trends and comparisons over time [3, 134].While displaying data alone is a challenge, further providing filtering options, tog-gles, etc. can be cumbersome to navigate through potentially involving extensivescreens, or lists of options. These challenges importantly present opportunitiesfor innovative interfaces and visualizations tailored specifically for smartwatches.As data collection becomes more advanced, and the smartwatch further matures,there is a growing need for creative solutions that not only condense complex in-23formation into digestible, easy-to-understand formats, but also enhance explorationpotential, enabling users to gain unique and personal insights with minimal inter-action.Interaction Techniques for Smartwatch Data Exploration While many re-search works have broadly explored how to expand interaction capabilities on thesmartwatch that mitigate the small screen challenges the [1, 48, 95, 137, 141, 155,162, 181, 204] (only a select few of the many research works), very few focus theseinteractions for use in data exploration. To note, some work has been exploredwhich utilizes the smartwatch as an input tool within a multi-device explorationenvironment (i.e., in conjunction with a large wall display) [80, 201]. However,these works do not involve exploration directly on the smartwatch screen.To our knowledge, Neshati et al. [137] is the only work that directly focuses oncreating an input technique for interacting with charts on the smartwatch screen.Their approach successfully utilized a portion of the bezel around the screen toreduce fat-finger and screen occlusion issues, allowing users to accurately targetindividual data points within a displayed chart. However, this input technique hasits limitations, as it primarily enables users to view individual values for specificdata points of a chart already on screen, lacking the uniqueness and functionalityneeded for expanded interaction and even deeper exploration.Visualization Techniques for Smartwatch Data Exploration Practitioners, de-signers, and researchers all acknowledge the need to adequately represent self-tracking data on the smartwatch, especially given the typical short interaction timesand their small screen sizes. Currently, various visualization techniques are usedon commodity smartwatches to reflect complex health data, including bar, donut,radial bar, and line charts [13, 136], as well as text, icons, and pictograms [84].Each has its pros and cons. Bar charts are widely used and easy to understandbut can be space-inefficient. Donut charts are common but not always effective atpresenting proportions. Radial bar charts are not efficient for comparing more thana few data points. Line charts provide clear representations of time-series data butcan become cluttered if too many data points are displayed.24Regardless of the visualization method chosen, however, ensuring glanceabil-ity is paramount for effective data representation on smartwatches. To be trulyeffective on a smartwatch, visualizations must be glanceable—they should conveyessential information within five seconds or less [14, 68]. This quick comprehen-sion is key given the brief nature of smartwatch interactions.To this effect, case studies of smartwatch visualizations have been conducted,to first better understand people’s perceptions and ability to perform common dataanalytic tasks through visualization on the smartwatch. These include to under-stand glanceability [13] and visual parameters that make up the visualizations[85, 120]. Blascheck et al. [13] pinpoint this problem by investigating percep-tions and efficiencies of bar, donut, and radial bar charts for a common two-pointdata comparison task. They find that 24 data points represented as a bar or donutchart allowed for quick and efficient comparison. Furthermore, Islam et al. [85]studied a variety of sleep data visualization designs to elicit the most preferred andsuggest design guidelines for differing form factors.Other visualizations techniques have been explored to push the boundaries ofthe current techniques used. Amini et al. [3] proposed, through both user studyand designer sessions, a variety of visualization ideas each which target a dataexploration need that people had on the smartwatch. Visualizations proposed in-cluded simple text and numbers, standard charts, and icons and symbols. For moreinvolved data analysis on the smartwatch, and recognizing that many data pointscould be involved, Neshati et al. [135, 136] and Tufte [189] proposed compressionof line graphs, often called sparklines. These charts were shown to not only retainsalient features necessary, but also remain to allow for the performing of commonanalytic tasks (i.e., trend detection, max/min detection, and value comparison).Summary While these works are crucial for supporting glanceable explorationof collected data on smartwatches, and positing interaction capabilities for specificdata analysis, our work in this thesis recognizes an opportunity for a complemen-tary approach. Although the average interaction time with a smartwatch is 1.9 sec-onds and often under 5 seconds, usage time increases significantly during physicalactivity, averaging 18 seconds, and reaches up to 45 seconds for other interactions25[148]. We believe that, given the diverse exploration goals of smartwatch users,personal health data needs, and perceived advantages of exploration directly on thesmartwatch while in-situ, we can and should enhance the exploratory capabilitiesof smartwatches. By extending, and complementing current solutions to allow formore personalized exploration directly on the smartwatch, we can remain withinthe typical bounds of smartwatch interaction. This approach can provide users withdeeper insights anytime and anywhere, enhancing the utility and impact of healthdata collection and exploration directly on the smartwatch.26Chapter 3Databiting: Lightweight,Transient, and Insight RichExploration of Personal DataWe first conceptualize a form of data exploration that serves as alightweight and transient bridge between short- and long-form explo-ration; something we recognize and strive for throughout the follow-ing chapters. Conceptualizing, rather than providing a strict defini-tion, acknowledges the fluid boundaries of various data explorationuses and user needs. This chapter resulted in a publication in Com-puter Graphics and Applications as a Visualization Viewpoint [158].The article was a collaborative effort among myself, Bongshin Lee,Eun Kyoung Choe, and Pourang Irani. The collaboration took shapethrough countless meetings focused on concept creation and devel-opment, as well as editorial work. Any mention of ‘we/our’ in thischapter refers to my co-authors.As device hardware and software advance, enabling broader access to personaldata, new opportunities for data exploration arise: data exploration has the poten-tial to intertwine with our lived experiences and day-to-day activities. However,conducting data exploration in many scenarios of use poses unique challenges. Itis crucial that exploration does not hinder, but rather assists, a wide range of scenar-ios and contexts in which we find ourselves seeking insight [106]. In this chapter,27we advocate for the visualization and personal informatics research communitiesto focus on the development of lightweight and transient exploration techniquesthat remain to enable insight rich access to personal data.Current systems offer one of two approaches to personal data exploration.Glanceable and micro visualizations have been widely adopted in mobile appli-cations [3, 13, 15, 26, 68, 85, 135, 136], at times combined into dashboards (Fig-ure 3.1 left). They provide concise and focused representations of information in alimited space and context for users to easily grasp information at a glance. How-ever, despite their popularity and necessity, they offer only specific insights andallow limited interaction, leaving users without the ability to cater to their personaland situational needs. In contrast, heavyweight applications (e.g., Tableau, Excel,custom script writing) have been designed to enable longer-form and more com-prehensive data exploration (Figure 3.1 right). These applications often requireconsiderable time and knowledge to use. These barriers make them at times in-accessible or inconvenient (e.g., during physical activity, while walking a pet, orwhile cooking). Between these approaches, a significant gap in the field of per-sonal informatics and visualization arises: limited information richness hindersusers’ ability to better comprehend and leverage personal data through explorationthat can be efficiently undertaken during broader contexts.Within this chapter, we conceptualize and discuss the notion of databiting,a term we coined to indicate lightweight, transient, and insight rich exploration.We further delineate five research considerations—contextual factors, interactionmodalities, the complementary relationship between databiting and other forms ofexploration, personalization, and evaluation challenges—focused towards enablingand understanding databiting. Importantly, these research areas can work in con-cert to provide lightweight and transient access to richer personal insight anytimeand anywhere. By embracing ideas and approaches outlined in this article, we canempower individuals to effortlessly gain insights from their data as needed, trans-forming the way they explore and interact with their personal data. Together, thisrecognizes the importance of our interactions with personal data and emphasizesthe significance of seamlessly integrating rich personal insights into our daily lives.As such, the key contributions of this chapter are two-fold:28Figure 3.1: Representations of current mobile data exploration applications arehighlighted, grouped by general information richness: applications which affordglanceable and micro-visualizations (at times combined into dashboards) (top) andapplications which provide potential for heavyweight data analysis (bottom).29C1: An introduction to the term databiting, conceptualized as lightweightand transient data exploration, that bridges the gap between glanceableand longer-form data exploration. We provide a conceptualization ofdatabiting, highlight its unique characteristics, and discuss potentialbenefits.C2: A delineation of open research challenges and goals for the re-search community to pursue in enabling databiting. Specifically, wediscuss contextual factors, interaction modalities, the relationship be-tween complementary forms of data exploration, personalization andcustomization, as well as evaluation challenges.3.1 The Databiting ConceptWe conceptualize databiting as the act of interacting with data to gain increas-ingly rich insight through lightweight and transient exploration. The result is adatabite, concise personal insight that extends upon what can be derived fromglanceable or micro visualizations. Databiting as both a new concept and a topicfor research is fluid in nature: Boundaries defining insight and data explorationmethods allowing for such insight are not rigidly defined or fixed.To illustrate this concept, we draw upon analogy. Databiting can be seen asequivalent to eating a small and easily consumable snack. The size of a snack andthe number of bites required may vary from person to person and from contextto context. Yet, what remains constant is the lightweight and transient nature ofsnacking compared to consuming a meal (often a reasonably large amount of food).In the context of data exploration, databiting equates to the consumption of bite-sized information that provides rich insights or sustenance in the moment. Thisbuilds upon simply viewing a mobile data visualization and does not require morein-depth and long-form data exploration, which can be done later when necessaryor more appropriate.Importantly, databiting is not meant to replace either long-form exploration ofdata or shorter-form viewing of glanceable visualizations; rather, it is complemen-30tary to them. By bridging the gap between these two forms of exploration, databit-ing offers a new, complementary, form of exploration that pushes the boundaries ofwhat is currently attainable. This integration of exploration methods can foster amore comprehensive, valuable, and unique (i.e., richer) understanding of personaldata, yet remain accessible in a lightweight and transient manner. By offering arange of exploration options, across devices and throughout a range of usage sce-narios, we expect individuals can derive greater benefits from their data-driveninsights anytime and anywhere.3.1.1 Lightweight, Transient, and Insight Rich ExplorationTime and effort, which should remain small, are key factors in the context ofdatabiting. Glanceable and micro visualizations excel at providing quick and easydata engagement, however, remain limited in their level of insight conveyed.We envision that there remains a large opportunity to enable access to moreinformation rich insights while maintaining a lightweight and transient approach,see Figure 3.2. Consider a scenario in which a runner is stopped at a traffic light,waiting to cross the street. The primary task is their run and the focus on thesurrounding environment. Secondary to this, they look at their heart rate zone data.Seeking a databite they simply tap on a stacked bar chart which highlights theircurrent heart rate zone. This action reveals increasingly detailed insight into thetime spent in each heart rate zone, enabling the runner to concentrate on enteringor maintaining a specific zone as they proceed with their run. Notably, this is notfunctionality that is currently available.As can be seen, databiting can offer richer insight without requiring substantialeffort, engagement, or time. This allows data exploration to occur as a secondarytask, alongside a primary ongoing activity (e.g., while out for a walk) or duringa recurring daily activity (e.g., riding a bus home). Through the prioritization ofsimplicity rather than detail and complex insight, the small size of databites ensuresthat they can be easily and appropriately consumed.31Figure 3.2: In blue (bottom left), we highlight glanceable and micro visualizationsfor data exploration. In green (top right), we highlight heavyweight data explo-ration and analysis. In pink (middle), databiting, as a concept, promotes the needfor increasing information richness while exploration remains lightweight and tran-sient. We encourage the reader to envision how visual data exploration can con-sume this area of the graph. We present these rectangles as sketched illustrations tosignify the fuzzy boundaries of these forms of exploration and potential variancewithin them.323.1.2 Device Agnostic ExplorationDue to their highly portable nature, the immediate availability of data beingcollected, and the smaller form factors limiting heavier exploration, databiting isparticularly well-suited for smartwatches and other wearable devices. However,the emphasis of databiting is not on the specific device used or insights gainedbut is in the manner in which data is accessed. As such, databiting can be seenas device agnostic and can be done on any device that grants access to relevantdata, ranging from wearables, smartphones, and tablets, to laptops. For instance,before switching to a different task on a laptop, a researcher first quickly checks thecurrent screen-time information. With simple mouse interactions, they learn abouta negative trend that results from increasing the duration of one continuous workblock and thus decide to take regular breaks, starting with an immediate break.Similarly, a tablet user exploring their financial data while in transit could use thestylus to circle a transaction and draw a line to another to view how they are related.Taking the interaction a step further, the user could employ pre-defined gestures toaccomplish further exploration (e.g., displaying a transaction timeline or findingsimilar transactions).3.1.3 Data Agnostic ExplorationThe versatility of databiting extends beyond that of personal data, making itapplicable to a wide range of domains and contexts involving data exploration.While the initial conceptualization revolves around personal data for individualinsights, the fundamental principles of lightweight, transient, and insight rich ex-ploration can be seamlessly applied to various data categories. For instance, inthe business context, professionals can benefit from quick insights (e.g., during ameeting). Databiting can enable easy access to necessary insights to foster discus-sions and inform decision-making without the need for extensive data preparationor analysis. In the context of scientific research, researchers running a study canuse databiting to garner quick insights about the amount and quality of collecteddata. This can help them decide to continue or stop and refine, if needed.333.2 Expected Benefits of DatabitingWe discuss the envisioned potential benefits of databiting that are not fullyrealized with the current capabilities of mobile data exploration. Further study isneeded to identify and demonstrate any tangible benefits that may exist.3.2.1 Introductory and Intermediary AccessAs highlighted earlier, databiting has the potential to bridge the gap betweenbrief information access (short-form exploration) and comprehensive knowledgegain (long-form exploration). Offering lightweight, transient, and insight rich ac-cess to personal data allows for engagement with smaller bite-sized snippets oftimely information. This approach provides an accessible entry point for individu-als new to data exploration to begin to explore and understand their data.Furthermore, a bite-sized approach to exploring data can serve as an interme-diary step, providing individuals with a gateway for more in-depth data explorationat a later time. By offering exploration in easily consumed pieces, databiting canspark further interest, familiarity, and excitement. When presented with quick andaccessible personal insights, individuals may become more curious and motivatedto further explore their data. Over time, this increased engagement may foster agreater sense of familiarity and confidence with data; ultimately facilitating furtherexploration and a deeper understanding.3.2.2 Increased In-Situ InsightDue to the lightweight, transient, and insight rich nature, we anticipate databit-ing will be beneficial for obtaining data-driven insights during in-situ exploration.In-situ instances of exploration refer to moments in which data analysis and re-flection occur closely related to an ongoing activity, enabling immediate and directimpact [108]. Databiting can extend in-situ data access, currently offered throughglanceable and micro visualizations, by providing further information richness.This leads to potentially increased actionable insights that are directly relevant tothe ongoing activity. Whether it is using a smartphone to explore caloric intake for34the day while cooking, asking a smart speaker about your home energy consump-tion for the day as you arrive from work, or checking productivity on your laptopwhile studying, databiting can empower individuals to gain valuable, personalizedinsights that inform immediate decision-making.3.2.3 Perceived UsefulnessA current challenge concerning devices and applications that allow for per-sonal data access and exploration is a lack of perceived usefulness [131]. Currentofferings often fail to meet expectations, resulting in the abandonment of devices,applications, and even the collection of data altogether [38]. While addressing per-ceived usefulness is a multifaceted challenge, databiting can serve as a promisingstart. Increased information richness and personalized insights enabled by databit-ing have the potential to enhance the perceived usefulness of the devices and appli-cations used for databiting, and the value derived from collected data. In turn, thiscould lead to greater overall outcomes as the collection and exploration of personaldata are not outright abandoned.3.3 Research ConsiderationsIn this section, we discuss how we can begin to enable databiting through re-search, to achieve increased access to personal data anytime and anywhere. Specif-ically, we discuss research considerations that aim to overcome existing challengesand benefit from promising opportunities.3.3.1 Contextual and Attentional FactorsContextual factors can become increasingly integrated with databiting, shapingthe recognition of potential exploration, the integration of relevant data, and evenoutput provided to answer a question asked during a databiting instance. Unlikeconventional data exploration, in which context might be analyzed as an additionalfactor later on, databiting can be seen to easily integrate context. Imagine a trav-eler, using a mobile app to explore a new city. Basic on-screen insights might35reveal location in relation to nearby attractions. However, databiting can leveragecontextual data such as time of day, personal preferences, and current location.This integration of contextual data can enhance the overall experience and providethe potential for increasingly relevant in-situ insights. Focusing research on under-standing and incorporating contextual factors within data exploration can signifi-cantly impact exploration that is possible. For example, we can study techniquesto incorporate contextual attributes into data visualizations, develop context-awarerecommendation algorithms, and look to gain a better understanding of how con-text can be incorporated into queries and answers desired for databiting.Beyond integrating context into the data itself, the situational demands andunderlying contexts of in-situ activities can also influence data exploration ca-pabilities. Factors such as physical exertion, cognitive load, attentional capacity,and movement can affect a person’s ability to engage with databiting effectively.Research into these dimensions can provide valuable and actionable insights foroptimizing databiting to better fit the context of use. For example, leveragingmulti-source data streams [110]—such as sensor data (e.g., heart rate, skin conduc-tance, accelerometer) combined with self-reported measures of perceived workloadand task difficulty—can help build a comprehensive understanding of an person’sexploratory capability in real time. Using this contextual information, we couldmodel expected performance in terms of efficiency and effectiveness, akin to ap-proaches that have been used to measure arm fatigue during mid-air gestures [77].3.3.2 Interaction ModalitiesEnabling databiting requires considering appropriate interaction modalities thatare not only efficient but also cater to the unique constraints of in-situ scenarios.This is not straightforward, especially because we aim to increase the level of dataaccessibility across devices with potentially limited interaction spaces (e.g., smart-watches, smartphones, and other wearables).To facilitate lightweight, transient, and insight rich exploration of personaldata, a multimodal approach through natural language and the device’s primaryinteraction method can be used (see Figure 1.1). The primary input modality,often36touch, allows for direct manipulation, discrete selections, and when natural lan-guage keywords cannot be remembered [98]. Importantly, natural language (e.g.,speech) benefits from enabling fast and flexible expression of complex queries [8].Recent research in personal health applications on smartphones has demonstratedthe benefits of multi-modal touch and speech interactions for gaining insights intopersonal health data [98]. Notably, individual and combined interactions were of-ten used for differing, yet equally important, components of the data explorationprocess. Touch and speech combined showed promise for refinement of an initialquery or displayed graph, much like databiting may extend upon glanceable or mi-cro visualizations. However, a better understanding of individuals’ personal dataquery requirements needs to be further addressed to fully recognize the interactiveneeds of databiting.Furthermore, exploring alternative output modalities can complement currentlyavailable data visualizations. Devices such as headphones, earbuds, and home as-sistants offer opportunities to incorporate natural language responses when databit-ing. This can appropriately provide access to data when people’s visual systems areoverloaded [19]. There is limited work on the requirements for natural languageresponses to personal data queries. Further research can focus on formulating effi-cient responses, determining the level of conveyed insights, and ensuring the mem-orability of responses. Integrating these findings with existing data visualizationapproaches can optimize access to personal data.3.3.3 Databiting and Broader ExplorationThe nature of databiting invites opportunities to consider a relationship withfurther long-form data exploration. Understanding how people can transition fromone to the other and how previous in-depth exploration can inform future databitingwill contribute to a cohesive and personalized data exploration experience. How-ever, it is crucial to recognize that the seamless transition between databiting andbroader exploration experiences will likely require extensive device and app inter-operability. Future research is needed to address these technical hurdles.Databiting can afford a stepping stone for people to delve into in-depth data37exploration when more appropriate. For example, as seen in Figure 1.1, Sam mayrecognize that their pace was improving for a while, until the most recent hike. Tobetter understand why today’s hike had such a decrease in pace, they can save thedatabite such that they are reminded to further explore for an external cause (e.g.,decreased sleep) at a later time (e.g., while at a desk) and on an appropriate device(e.g., on a tablet). It becomes important to understand how individuals can saveand manage databites, and any new questions that arise, for future exploration.Conversely, the results of prior in-depth data exploration have the potentialto influence and enrich future databiting. For example, an individual may havetaken the time to pore through their credit card statements, itemizing transactionsinto categories and noting a budget limit for each. When in the store, this pastexploration may influence databiting which is directed towards understanding if apurchase can and should be made within a predefined category.These examples further highlight an underlying challenge concerning interop-erability. Currently, fragmentation and non-standardized access to collected datahinders cohesive personal data exploration. For instance, data saved on one devicemay not seamlessly integrate with a user’s preferred in-depth exploration tool onanother device (e.g., to analyze collected data from an Apple Watch in Tableau, ex-porting and importing of data is required). Furthermore, data collected on multipledevices may be stored using different protocols and formats. Without a concertedfocus on interoperability, the potential for databiting to seamlessly complementbroader exploration methods, and vice versa, may be hampered, limiting overalleffectiveness.3.3.4 Personalization and CustomizationPersonalization: Studying and supporting personalization is a key aspect whenlooking to optimize databiting and enhancing the exploration process. By tailoringinsights, recommendations, and visualizations to individuals, and their contexts ofuse, personalization ensures that relevant information is efficiently presented. Thisreduces the required time and effort, enabling people to quickly access valuableinsights and make informed decisions.38Through personalized databiting, technology mediation (i.e., artificial intelli-gence, machine learning, etc.) can alleviate individuals from manual data analysis.For example, a personalized databite may provide instant workout recommenda-tions or in-situ alerts to metrics trending in a specific direction. A potential reduc-tion in temporal workload allows people to focus on the actionable outcomes ofdatabiting rather than the exploratory process. Furthermore, serendipitous discov-ery is a benefit of visual data exploration. Personalization can be used to mediatetailored serendipitous insight, even with lightweight and transient exploration.When developing personalized systems that provide increased insights withminimal interaction, both technical and experiential research questions arise. Howcan user behavior and preferences be accurately captured and utilized for person-alization in databiting? It is crucial to ensure that notifications and informationprovided align with internal models and desires, as undesired interactions can nega-tively impact the user experience. This further leads to the question of how person-alization impacts user engagement and satisfaction with databiting. Understandingand building around a person’s individual preferences is fundamental to deliveringeffective personalized insights.Furthermore, identifying the most relevant data and patterns for providing mean-ingful and timely personalized insights is essential. This further aligns with under-standing the contexts of use. For example, a system must determine which datapoints are most valuable to the user at any given time and adapt to changing con-texts and preferences. Algorithms that support personalization must consider a vastamount of information, including user behavior and past interactions, preferences,historical data, contextual factors, and more. Specifically, we need to understandwhich types of data and patterns are most relevant for providing meaningful andtimely insights. By addressing these questions, our future goal will be to developpersonalization strategies and algorithms that enhance user experience and makedatabiting a more intuitive and efficient process through personalization.Customization: For those who prefer a higher degree of control, offering cus-tomization options is a critical aspect of effective databiting. Customization allowsusers to take the time to preemptively tailor their data exploration applications to39their preferences, which can include the creation of custom charts, setting spe-cific data filters, and adjusting the look and feel of their data exploration interface.By enabling users to customize these aspects, explicit databiting (i.e., querying ofdata) is not necessarily required. Yet, a challenge that remains in this space is howto effectively provide people the ability to create their own exploration experiences.To address this, future research should explore several key questions: Howcan customization tools be designed to be intuitive and accessible for all users,regardless of their technical proficiency? What methods can be employed to ensurethat customized settings are easily adjustable as user needs and preferences evolve?Additionally, how can we support users in discovering and utilizing customizationoptions to their fullest potential?One promising solution lies in developing multi-modal systems. By integratingmultiple modes of interaction, users can more effectively tailor their databitingexperiences to fit their unique needs and contexts. This approach not only enhancesthe usability of customization features but also ensures that the system remainsadaptable and responsive to the diverse ways in which users interact with theirdata.Spectrum of Personalization and Customization: Finally, to fully realize thepotential of personalization and customization, balancing automated personaliza-tion with user control is also essential to maintain system intuitiveness and flexibil-ity. It is unlikely that personalization and customization is a binary choice. Rather,individuals will likely fall along a spectrum between the two. This balance mayvary between people, and even for an individual person given varying goals andcontexts of use. Research in this space should work to model user preferences thatcan align a person along this spectrum. By then adapting a user interface to allowfor the desired level of personalization and customization, we can afford a greateruser experience and efficiency for databiting.403.3.5 Evaluation ChallengesTo realize the full potential of databiting, two areas of evaluation must beconsidered. First, it is essential, yet challenging, to understand an individual’sneeds and goals when interacting with personal data while in-situ. Modest meth-ods such as sketching or surveys lack real-world data, while heavyweight ap-proaches through the creation of working mobile applications are costly and time-consuming. Balancing methods through data engagement interviews and Wizard-of-Oz studies can help capture needs in various daily contexts, although in-labstudy methods may suffer from recall bias [75].With the above in mind, accommodating in-situ studies is crucial for a com-prehensive understanding of databiting. Current methods such as Experience Sam-pling [39] and Diary Studies [17], while valuable, have limitations in not alwayscapturing the full range of potential study data. Further developing appropriate re-search methods within situated environments is required. By capturing real-worldusage and contextual factors, we can gain a more nuanced understanding of how in-dividuals engage with databites in their everyday lives, further advancing the fieldand maximizing potential.Second, to assess the value of databiting once it has taken place, measuringeffectiveness of the exploration conducted and insight enabled by databiting is im-portant. Capturing data as closely tied to the databiting experience can be increas-ingly valid and important to analyze. This further would allow us to understand theimpact of databiting compared to glanceable and micro visualizations, and longer-form data exploration. Longitudinal studies can capture behavior change over time,but studying and capturing the immediate influence of databiting is not straightfor-ward. Given the lightweight and transient nature of databiting, as well as the in-situinstances of interaction, it is essential to develop methods, tools, and scales that as-sess the immediate effects of databiting without further distracting users from theircurrent activities or being compounded as a part of the databiting interaction. En-suring that evaluation techniques are minimally intrusive will help gather accuratedata on the effectiveness of databiting in real-world scenarios.Several research questions arise from these challenges: How can immediate41impacts of databiting be accurately measured in real-time? What minimally intru-sive evaluation techniques can be developed for in-situ databiting interactions? Onegoal with respect to these research questions would be to understand these impactswithout the need to analyze any captured personal health data. Given a variety ofprivacy concerns, and the increased difficulty in studying raw personal health data,creating methods that do not require such data would be important. Finally, along-side understanding the immediate impact of a databiting interaction, how can weassess the cognitive and emotional impact of databiting on users? Addressing thesequestions will help in developing robust evaluation strategies that capture the im-mediate and even perhaps the long-term impacts of databiting, ensuring that thesesystems are efficient and appropriately designed.3.4 SummaryIn this chapter, we conceptualize databiting as the process of extracting in-creasingly rich insight through lightweight and transient data exploration. Givenincreasingly powerful devices, coupled with the vast collection of personal data,and in-situ use, this form of exploration has the potential to complement the waypeople interact with and understand their personal data throughout their daily lives.By bridging the gap between glanceable and micro visualizations (short-form ex-ploration) and heavyweight (long-form) exploration, databiting can provide com-plementary insight, enabling people to better access their personal data.We have introduced the concept of databiting, provided examples and expectedbenefits, and delineated research considerations that remain to be undertaken. Weenvision that this key, yet under-explored, concept of personal informatics maysoon become a reality. This thesis and its later chapters aim to push the bound-aries of this beginning, making databiting on the smartwatch a complementary partof personal data exploration. We hope this chapter alone inspires research com-munities towards the creation of applications and tools that enable databiting. Weanticipate that both exciting challenges and opportunities will arise, which in turnwill shape the future of databiting and access to personal data anytime and any-where, for the better.42Chapter 4Eliciting In-Situ Personal HealthData Queries on the SmartwatchWith databiting in mind, we must first gain knowledge of what per-sonal health data queries are even of interest to people throughouttheir daily lives. Surprisingly, there is little concrete knowledge aboutthe specific queries people desire. Only once queries are known, canwe begin to understand and build for them. In this chapter, we nowbegin to focus our efforts on the smartwatch as a mature tool that hasthe potential to provide and benefit from our envisioned databitingcapability. This chapter resulted in a publication in the journal onInteractive, Mobile, Wearable, and Ubiquitous Technologies [157].The article was a collaborative effort among myself, Bongshin Lee,Eun Kyoung Choe, and Pourang Irani. The collaboration took shapethrough combined efforts in ideation, study design and editorial work.Furthermore, Charles-Olivier Dufresne-Camaro supported efforts inthe data analysis/coding process highlighted throughout this chapter.Any mention of ‘we/our’ in this chapter refers to my co-authors andcolleague.4.1 IntroductionUnderstanding user intentions for personal health data exploration and interac-tion in-situ on smartwatches remains largely unexplored. A notable study captured43a dataset of desired queries from smartwatch users [154], providing valuable in-sights, but relied on a survey, which is prone to recall bias [75]. Current researchon smartwatch personal health data spans two main domains: first, identifyingand addressing the needs of smartwatch users regarding their personal health data[3, 134]; and second, developing data visualization [3, 13, 26, 68, 135, 136] andinteraction [137] techniques to overcome the limitations posed by the small screensize of smartwatches. Although these studies offer significant insights, they oftenlack a broader empirical understanding of the specific data exploration desires ofsmartwatch users.To understand, and begin to support databiting on the smartwatch, this chapterfirst elicits what exploratory queries smartwatch users desire throughout their dailylives. Additionally, we reflect on why the queries provided were desired. Finally,we look to better understand any implications of being in-situ (i.e., when/wheredata exploration is desired). More specifically, we focus on the following tworesearch questions:RQ1: What personal health data queries are of interest to people forexploration on their smartwatches throughout their daily lives?RQ2: How does being in-situ influence the desired personal healthdata queries of smartwatch users?To answer these research questions, we conducted a week-long study withApple Watch users (N = 18), and concluded with a final interview. Throughoutthe week, we elicited a total of 205 desired queries from participants that wouldallow them to better explore and have access to their collected personal healthdata on their smartwatch. Using a custom built data collection application, par-ticipants reported responses throughout their daily lives, which included a naturallanguage query and current activity information. We learned that participants notonly looked to utilize their smartwatch for data exploration temporally surround-ing a tracked physical activity [3], or in-situ, but also found it beneficial to usethe smartwatch throughout daily activities for immediate and often discrete explo-ration. Furthermore, with respect to why participants looked to explore data, we44highlight a new Preemptive and Proactive insight category, and find expanded Cur-rent Status as well as Contextual insight categories when compared to previousworks [3, 29, 30, 108].The key contributions of this chapter are twofold:C1: An empirical study to capture health data queries desired for ex-ploration on a smartwatch throughout daily life. From this study weprovide a dataset of the 205 natural language queries captured throughour study4.C2: A thorough analysis of the queries in our captured dataset, toprovide a better understanding of where, when, and why queries aredesired. Combining qualitative insights with quantitative metrics, weuncover patterns and trends that can inform the design and implemen-tation of effective databiting on smartwatches.4.2 Relevant Related Work4.2.1 In-Situ Data Collection MethodsTypical in-lab data collection methods, such as interviews, surveys, and focusgroups are often subject to recall bias [75]. Thus, ecologically valid methods ofdata collection have been utilized in research across domains. These methods in-clude, Diary study [17, 23, 37, 51, 73, 90, 177] and Experience Sampling Method(ESM) (or the equivalent Ecological Momentary Assessment) [39, 45, 105, 190].Both methods often utilize a pre-built survey or questionnaire for participants to re-spond with, but a notable difference between Diary and ESM studies is the naturein which data is meant to be captured. ESM studies utilize notifications sent to par-ticipants, where data capture is intended to be done in immediate succession to thealert. On the other hand, Diary studies allow for the collection of in-situ self-reportdata whenever participants have a response to report. Notifications are then used asreminder rather than a trigger to submit responses. To further reduce data capture4https://smartwatch-personal-health-data-queries.github.io/45burdens, researchers have begun to utilize smartphones and smartwatches for suc-cessful data collection [76, 99, 202]. The always-available and body-worn natureof a smartwatch allows for easy access to the data collection tool. Furthermore,notifications on smartwatches have a high level of awareness from the participant[24, 148], have shown to allow for higher response rates, and are perceived as beingless distracting during daily life [83, 150].4.3 User StudyOur in-situ data collection spanned the course of a week, in which we employeda diary study method. As part of the study, we installed a custom data collectionapplication on participant’s smartwatches, through which they recorded a desiredpersonal health data query using natural language (either spoken or written), as wellas additional activity information. Participants provided responses throughout theirdaily lives, specifically when they felt it would be beneficial to access such infor-mation concerning their personal health data directly on their smartwatch. Ethicsfor this study was obtained from and approved by our institutional review board.4.3.1 ParticipantsWe recruited 18 participants (P1–P18; ten females and eight males) from Red-dit. We advertised our study across subreddits relating to personal health as wellas a number of general city subreddits across Canada. Our inclusion criteria werethose who (1) were aged 18 years or older; (2) own an Apple Watch Series 3 ornewer with watchOS 7 or higher installed, and have a paired iPhone; (3) have ac-cess to Zoom and a stable internet connection; (4) are native English speakers; (5)have no motor, visual, or speech impairments; (6) currently collect at least one ofthe following personal health data: sleep, nutrition, physical activity, steps, calo-ries, women’s health, and/or mindfulness data on their Apple Watch; and (7) havebeen regularly collecting personal health data on their Apple Watch for at leastthree months.46In appreciation for their time and effort, participants were offered up to $40CAD. The amount a participant received was not tied to the number of responses,but rather the number of participation days. We provided the equivalent of $9.50CAD for attending the introductory session, and another equivalent of $9.50 CADfor attending the final interview. During the week long data collection, we addedan additional equivalent of $3 CAD for each day for which a participant providedat least one response. We provided compensation after the final interview, or uponwithdrawal, in the form of an electronic Amazon.ca gift card.4.3.2 Data Collection MethodTo understand what personal health data queries lay users have, researchershave previously used focus groups, lab based experiments, and questionnaires[3, 154]. While these methods provide a positive general understanding, we aimto mitigate the potential for recall bias through concretely reported, in-the-wild,responses. Within our work, desired responses could arise at any time within aparticipant’s daily life. Therefore, we utilized a Diary study method combinedwith ESM’s random-interval sampling. We note, both fixed interval-based andevent-triggered collection methods would have restricted responses to specific andpredictable times or to certain activities respectively. A Diary study method com-bined with ESM’s random-interval sampling allows for a wide range of boundarypushing and ecologically valid queries to be captured, during a range of times andactivities, without restriction, as they are deemed beneficial to the participant.4.3.3 Data Collection ApplicationWe created a data collection application5 for the Apple Watch using Swift,and deployed it via Apple’s TestFlight beta program to each participant’s individ-ual Apple Watch. We chose only the Apple Watch due to the immensely simplelogistics in installing our application remotely. This also ensured consistency indata collection procedures. Our application utilized Google’s Firebase RealtimeDatabase to collect and store the responses submitted. As our goal is to capture5https://github.com/reyb/Personal-Health-Query-Recorder47queries for personal health data exploration on smartwatches, the application wasdesigned purely for data collection; it did not answer the desired queries from ourparticipants.Data Collection QuestionsOur application consists of up to four questions for participants to respondto. All questions were required to be answered when submitting a response. Thequestions and input methods were designed to support fast and easy reporting ofresponses while mitigating interaction difficulties on a small-screen device. Thisincluded the use of natural language reporting for open-ended questions (Q1 andQ2), leveraging the flexibility and ease in reporting ability [99], and single screenoptions for Q3 and Q3-1. Our application captures the following information, il-lustrated in a flow diagram shown in Figure 4.1 and described below:Q1 (open-ended): What question or command do you have of your healthdata? This allowed us to capture the personal health data query from the partic-ipant. Q1 elicited a query regarding what participants were interested in quicklyexploring or accessing on their smartwatch. Participants could either speak or typea query using the Apple Watch’s built in text-entry methods. Upon recording thequery, it could be reviewed on-screen in real-time and repeated to correct errors, ifneeded.Q2 (open-ended): What are you currently doing? This helped us gain generalcontextual and daily activity information of the participant’s daily life at the time ofrecording a desired query. For simplicity on the part of the participant, we allowedfor natural language input in the same manner as the first question.Q3 (dichotomous): Is your response related to your current activity? As apersonal health query may or may not relate to the current activity being performed,this question allowed us to better understand the association between a participant’scurrent activity and their personal health data query given in Q1. Either a ”Yes” or”No” answer could be selected.Q3-1 (multiple choice): Where are you in your activity? This would appearto the participant only if they selected ”Yes” in the previous question. From this48question we aimed to gain further knowledge, understanding the in-situ momentsurrounding an activity, that a need for exploring personal health data arises. ”Be-fore,” ”During,” or ”After” could be selected. In order to better understand whena desired query may be temporally related to an activity, we ensured participantsunderstood that our definition of relation could also include just before starting andafter completion of an activity. For example, a participant could be going to thegym looking to soon start their workout (Before), actively engaged in their workout(During), or heading back to the change room having just finished (After). By fur-ther exploring this time distinction, we can introduce a greater level of granularityand aim to understand when to provide exploratory capability or further insight tosmartwatch users.49Figure 4.1: A flow diagram of the questions asked within our data collection application.50RemindersTo elicit many responses, our application employed two forms of reminders.First, we used push notifications. We customized these notifications to each partic-ipant, based on their self-declared wake and sleep times. Furthermore, the notifi-cations were systematically random in that a notification would be sent at a partici-pant’s declared wake time, and then concurrently sent throughout the day betweenone to two hour intervals (chosen at random) after the prior notification. Notifica-tions in this manner were repeated until the participant’s declared sleep time. Thismethod was chosen to ping participants at different times throughout each day,ideally attempting to remind them of the study at different in-situ moments withintheir daily lives.Second, our application also leveraged a watch-face widget, which could beused on a range of available watch faces. All participants were asked to updatetheir watch face to incorporate this widget. We offered a small circular widgetwhich was continuously displayed on a participant’s home watch-face and addi-tionally provided a counter of the number of responses a participant had submittedthroughout the study. Moreover, this widget allowed quick and easy access to ourapplication, and thus recording a response, by simply tapping on the widget.4.3.4 Study ProcedureOur study included three stages: an introduction and tutorial session, a seven-day in-the-wild data collection period, and a final interview. The procedure andstudy materials were iterated upon during two pilots with people who were re-cruited in the same means as our participants, thus meeting our study’s inclusioncriteria. Participants provided consent at the start of the study.Introduction and Tutorial Session.To start the study, the participant joined a Zoom meeting, ˜45 minutes long,where we introduced and acknowledged their interest and participation in our study.Participants were encouraged and asked to interject with any comments and/orquestions during the meeting. The researcher shared presentation slides (please51refer to the supplemental material) via the screen sharing functionality. The goalof the project was expressed to the participant along with other important remarksbefore they completed a demographic survey. Then, the researcher guided theparticipant through the installation and setup process of the application on the par-ticipant’s own Apple Watch to be used throughout the data collection stage of thestudy. This included setting up notifications and the watch-face widget for re-minders. The researcher then gave participants a walk-through of the applicationthrough an on-screen emulator running the application on the researcher’s com-puter and shared via screen sharing. Upon completion of the walk-through, theresearcher gave participants unrelated-to-the-study practice trials to ensure the ap-plication worked smoothly and any technical issues related to data collection wereappropriately handled.Finally, the researcher gave an explanation on the potential queries and purposeof the study. The researcher asked participants to provide queries that they deemas beneficial, without worrying about current technological limitations. Querieswould ideally allow participants to better explore or access their personal healthdata directly on their smartwatch. The researcher also asked participants to onlyprovide a query when it both arose within their daily lives and was deemed to bea data exploration task they would like to perform on their smartwatch. The re-searcher explained to participants that feedback or answers to their queries wouldnot currently be given, however they should envision receiving this directly on thesmartwatch. No specific examples were provided to the participants, so as not tobias their potential queries. However, high-level categories of health data explo-ration (e.g., history of data, goals/performance data) were discussed to invigorateideation.Data Collection.Participants used the application to submit responses over the course of thenext seven days, and were instructed to wear their watch as they normally would.A response could be submitted at any time throughout the day. We did not require aminimum number of responses throughout the study, to not elicit forced responses52from participants. Due to the potential for participants to be active when a queryarose, we instructed them to only provide a response through the application whenit was safe for them to do so. Notifications sent each day acted solely as reminderof the study.Final Interview.After the final day of data collection, a Zoom meeting was held where a re-searcher conducted a semi-structured interview with each participant. The meetingwas audio recorded and later transcribed. The goals of the interview were: (1)to gain a better understanding of the smartwatch’s role within each participant’shealth journey; (2) to explore additional information regarding in-situ smartwatchuse for personal health data exploration; and (3) to discuss preferences in terms ofinteraction and visualization when exploring personal health data on a smartwatch.To aid in recollection, a report of each participant’s queries were shown to them viaZoom’s screen sharing functionality. Finally, the researcher answered any remain-ing questions from a participant, thanked them, and provided the compensation.Each interview took between 30 and 45 minutes.4.4 Analysis and Results4.4.1 AnalysisA total of 229 responses were logged through our application. First, we man-ually inspected the data, curating a dataset that only included valid responses.Through this process, we discarded 24 responses which fell under three categories:(1) the query had no specific element of collected personal health data (e.g., ”Am Iover the food poisoning from yesterday?”); (2) the query was related to smartwatchfunctionality rather than personal health data exploration (e.g., ”Is there a betterway to track active minutes?”, ”How much battery does tracking a walk use?”, ”Isthere a way to account for temperature while working out?”); (3) the query wasaction based and did not allow for exploration (e.g., ”Set my bedtime for 12 AMand wake me up by 8 AM”, ”Start outdoor run”, ”Record 96 ounces of water for53the day”). After this process, 205 valid responses remained for analysis.We qualitatively analyzed the valid responses, containing a query, activity ofthe participant, and relation to this activity through an open coding process. To dothis, we followed the same approach as used by Srinivasan et al. [184]. To specify,two researchers first explored the reported queries and activities for broad themes,subsequently creating a coding schema. Then, the same researchers individuallycoded random subsets of the data after which they came together to compare resultsfor agreement. They refined the schema and codes, and individually coded a newrandom subset of data, until an 85% agreement was reached. Once the researcherscame to agreement, the data was independently coded in full using the mutually-agreed upon codes, again working together to reach full consensus as needed.4.4.2 ResultsTable 7.1 summarizes the demographic, smartwatch usage, and health data col-lection information, as well as response counts of our study participants. Partic-ipants were aged from 18 to 56 (M = 29.8) and held a range of occupations. Atthe time of conducting the study, participants had collected personal health datafor an average of 39.3 months (SD = 32.7 months) and had used a smartwatch foran average of 31.3 months (SD = 26.0 months). Collected personal health data byparticipants mainly focused in two categories, physical activity (18/18 participants)and sleep (7/18 participants) data. Of our 205 valid responses there was an averageof 11.4 responses per participant (SD = 5.5; Min = 3; Max = 22).54Table 4.1: Summary of demographic information, health data collection, smartwatch usage experience, and number ofresponses reported from our study participants.Alias Age Gender OccupationHealth DataCollectionSmartwatchUsageCollected Data Study ResponsesP1 19 F Customer Service 1y 2m 1y 2m Sleep, Physical Activity, Women’s Health 8P2 31 F Care Coordinator/Dementia Counselor 10y 0m 2y 6m Sleep, Physical Activity, Women’s Health 10P3 38 M Communications Advisor 5y 6m 5y 6m Sleep, Physical Activity 16P4 35 F Teacher 3y 0m 3y 0m Physical Activity, Women’s Health 9P5 33 M Information Security Specialist 5y 1m 1y 0m Sleep, Physical Activity 9P6 24 F Student 1y 0m 0y 8m Nutrition, Physical Activity 3P7 42 M Claim Evaluator 3y 0m 3y 0m Physical Activity 15P8 25 F Scientific Evaluator 0y 3m 0y 3m Physical Activity 6P9 36 F Post Doctoral Fellow 0y 8m 0y 8m Physical Activity, Women’s Health, Mindfulness 20P10 30 M Student 7y 0m 7y 0m Physical Activity 12P11 23 F Student 2y 3m 2y 3m Physical Activity, Mindfulness 16P12 56 F Retired Lawyer 1y 9m 1y 9m Physical Activity 7P13 22 F Educational Assistant 6y 4m 6y 4m Physical Activity, Mindfulness 18P14 21 M Student 2y 1m 2y 1m Physical Activity 22P15 18 F Customer Service 0y 5m 0y 5m Nutrition, Physical Activity, Mindfulness 11P16 25 M Software Engineer 6y 7m 6y 7m Sleep, Nutrition, Physical Activity, Mindfulness 5P17 40 M City Planner 1y 11m 1y 11m Sleep, Physical Activity, Mindfulness 15P18 18 M Student 0y 11m 0y 11m Sleep, Physical Activity 355We coded our valid responses, leading to the following five dimensions: (1)personal health data insight category, (2) current daily activity, (3) whether thequery was related to the activity, (4) the time in activity if related, and (5) the querytype. Taken together, these can enable a better understanding of the characteristicssurrounding desired smartwatch data queries, combined with the types of insightthat are of interest during certain daily activities. Below we detail each dimensionand their codes.Expanded Personal Health Data Insight CategoriesThrough the queries desired by participants, we coded the overarching in-sight category for which each query aligned. Table 4.2 shows the codes, descrip-tions, and selected queries. We expanded upon categories from previous works[3, 29, 30, 108] to assign our collected queries into the following codes: CurrentStatus or Value (89, 42.9%), Historical or Trend (67, 32.7%), Combination or Com-parison (59, 28.8%), Goals or Performance (57, 27.8%), Preemptive and Proactive(47, 22.9%), and Contextual (24, 11.7%). These codes are not mutually exclu-sive, and thus a query can have multiple codes (and as such the percentages aboveare individually calculated from the 205 total queries). For example, ”How muchdancing do I need to do to burn 800 calories?” (P18) can be seen as fitting intoboth the Preemptive and Proactive and the Goal and Performance categories. Ofthe six codes, participants reported queries in a minimum of 3 codes and maximumof 6 (M = 5.0, SD = 1.0).56Table 4.2: Query insight categories. Please note, categories are not mutually exclusive. ** denotes a new insight categoryfound in our work; * denotes an expanded insight category as compared to [3, 29, 30, 108]. Q is a question; C is a command.Response Category (# of Responses, % ofTotal, # of Participants)Description Example Responses [all directly quoted]Current Status or Value *(89, 42.9%, 17)Current, single value, metric that is collectedand/or aggregated on the smartwatch to begiven to the user.(Q) Am I over or under my calorie goal at the moment? (P15)*(Q) How many calories did I burn that workout? (P4)(Q) What was my peak heart rate during my workout? (P15)*(C) Give me a report for my readiness for activity. (P16)Historical or Trend(67, 32.7%, 18)Previously collected metrics, prior to thecurrent day’s or activity’s. Can often be used toexplore changes over time.(Q) How long on average does it take me to fall asleep? (P3)(Q) How many steps have I taken this week? (P9)(C) I would like to check a trend in my sleep in the past seven days. (P1)(C) Show me my body weight trends for this month. (P6)Combination or Comparison(59, 28.8%, 17)Combine and/or compare two or more differentmeasured values. These can be done over time,between metrics, between activities, orbetween oneself and others.(C) Compare my running stats with the same time last year. (P17)(C) Show me a graph of my runs both time and distance in 2021. (P11)(Q) Does my walking pace change when I walk with someone else? (P12)(Q) How many calories were burned in today’s work out compared to yesterday? (P6)Goals or Performance(57, 27.8%, 14)Goals, such as for steps, calorie intake, caloriesburned, distance travelled, etc. Performancestems from completing a goal, as well asquality metrics such as fast/slow or best/worst.(Q) How fast did I finish my 1st kilometer of my hike today? (P6)(Q) What was my best kilometer during my run? (P4)(Q) Which activity had the highest calories burned per minute? (P18)(C) Tell me when I reach a nine minute walking pace. (P12)Preemptive and Proactive **(47, 22.9%, 15)Advice or information that will allow one to bepreemptive or proactive in making decisionsand/or to prepare for an event in the future.(Q) How long do I need to run three times per week to achieve the November challenge? (P17)(C) Give me a suggested work out based on my readiness score. (P16)(Q) Is there a day of the week I am more likely to beat (friend) in our fitness challenge? (P12)(Q) How much dancing do I need to do to burn 800 calories? (P18)Contextual *(24, 11.7%, 9)Impact and/or affect of an external or collectedmetric on another.(Q) How is the air quality affecting my walk? (P16)(Q) Is my cycle affecting my sleep? (P2)(Q) Is my running pace slower in the days following a strength training workout? (P13)*(Q) What is the impact of my sleep in my running? (P17)*57Current Status or Value insight was often about more than just the simple met-rics captured. Amini et al. [3] found that participants explored step count, distance,calories, pace, speed, and heart rate during an activity. While this remains to beseen in our study, we found a broader definition of current metrics desired; theseincluded heart rate zones, total values from activities throughout the day, peak val-ues or fluctuations of metrics throughout the activity, and aggregated values suchas perceived exertion (see Table 4.2 for specific queries). These examples, whichcan be seen as increasingly unique-to-user, should be considered within smart-watch health applications to expand the usefulness and benefit to a broader rangeof smartwatch users.Contextual exploration was utilized by participants to find cause and effect be-tween a range of data. Choe et al. [30] discussed participants’ interest to includeand explore the effect that external data such as time of day, location, or weathercan have on their own collected personal health data. While our captured queriesfrom participants garnered similar contextual information, we also note that partic-ipants were inclined to look for cause and effect using their own collected data ascontext. For example, ”Is my cycle affecting my sleep?” (P2), ”Does my walkingpace change when I walk with someone else?” (P12), ”What is the impact of mysleep in my running?” (P17), and ”Does weightlifting focusing on different musclegroups affect my heart rate?” (P14). As participants used the smartwatch to cap-ture a range of health and activity data either automatically or through a discreteinput, they desired to explore context surrounding these captured instances on thesmartwatch.Preemptive and Proactive queries, a new form of insight brought forward fromour study compared to previous works [3, 29, 30, 108], make up ˜20% of our col-lected data from 15 of 18 participants (see Table 4.2 for specific queries). Smart-watches influence decision-making through on-screen metrics related to activities[3, 108]; for example, a runner glancing at their smartwatch can adjust their pace ifit’s below their desired value. Our participants, however, were looking for a widerrange of influential exploration from their smartwatch, such as to help choose a58workout for the day, plan an activity based on goals, or to pick up on elements thatthey alone may not be able to predict. This form of insight was seen as a means forpreparing oneself for a future event rather than simply reflecting on current or pastmetrics.Participants were looking to utilize these Preemptive and Proactive insightsfrom their smartwatch to influence immediate and in-situ decision making, as wellas for some daily and longer term planning. This longer term planning was oftenseen as tied to the Goals or Performance category, as 23/47 Preemptive and Proac-tive queries were also to gain insight on how to achieve immediate or future goals.Finally, while about a half of the queries in this category were temporally related toan activity (i.e., immediate influence), we further note that Preemptive and Proac-tive queries were desired on the smartwatch throughout the day (i.e., longer-termplanning).Smartwatch Personal Health Data Queries Desired throughout DailyActivitiesAnalyzing the daily activities of participants led to seven codes. These codescategorize the current activity being performed in a participant’s daily life, duringwhich their desire for exploration on the smartwatch arose. These codes include:Physical Activity, Self Care, Work, Leisure, Sleep, Transportation, and Other; thecodes, their descriptions, and counts can be seen in Table 4.3. Codes were mutuallyexclusive, and thus each response was given a single code. Of the seven codes,participants reported responses in a minimum of 2 codes and maximum of 6 (M =4.1, SD = 1.4).59Table 4.3: Summary of the daily activities participants were undertaking at the time of a response, and the relation of thereported queries to these activities: not related, before, during, after.Daily Activity Description & Examples# of Responses (% ofTotal), # of ParticipantsQuery Related to DailyActivityPhysical ActivityBody movement that requires more energythan resting. (e.g., dancing, walking,running, weight training, yoga, sports, etc.)70 (34.1%), 16Self CareActivities that pertain to normal day-to-dayhuman function. (e.g., cooking, eating,chores, morning ready routine, etc.)44 (21.5%), 12WorkFulfilling duties either for job or school.(e.g., job based tasks, studying, attendingmeetings, etc.)29 (14.1%), 12LeisureActivities performed for relaxation and fun.(e.g., reading, watching TV, lounging, etc.)25 (12.2%), 14SleepThe absence of wakefulness. (e.g., nappingor nightly rest)17 (8.3%), 9TransportationMoving from point A to point B. (e.g.,driving, taking the bust/metro, taxi, etc.)10 (4.9%), 5OtherActivities that do not fit within the priorcategories. (e.g., within the study interview,due to mHealth app notifications)10 (4.9%), 560We asked participants whether their personal health data query was related totheir current activity as reported above (Q3), and thus in-situ. This was either a Yes(107, 52.2%) or No (98, 47.8%) answer; examples from our captured data include”Leaving the gym” - ”Show me my heart rate chart from today’s gym session” asbeing related while ”At work” - ”What was my fastest kilometer in my run?” asnot. We see an almost equal distribution overall, however, when combined withthe activity we see a distinction. The most drastic example of this distinction isin relation to Physical Activity. Here, 97% of queries reported were related tothe Physical Activity being done. In contrast, all other daily activities, aside fromSleep, provided time within daily life for increased insight and reflection that wasunrelated to the participant’s current activity. Within our captured responses, 16 of18 participants found it beneficial to report a query both related and unrelated tothe current activity they were doing.When queries were related to the daily activity being done by the participant,we additionally asked whether they were just Before (24, 22.4%), During (38,35.6%), or After (45, 42.0%) the activity. The results collected follow closely withour demographic survey which asked when participants aim to explore their dataon their smartwatch, Before (4/18 participants), During (10/18 participants), andAfter (15/18 participants).Participants noted during the interviews that exploration Before activities couldonly do so much to affect the activity collecting the health data once started. In re-lation to our insight categories, broader Preemptive and Proactive as well as generalGoal insight was most often queried Before an activity. This was seen as a meansto help prepare for the activity at hand, rather than exploring current or past data.During an activity, specifically for Physical Activity, expanded insight beyondsimple metrics pulled participants away from being in the moment and focusedon the activity at hand, thus was not as desired. P5 mentioned ”It wasn’t much[exploration] like during the workout ’cause in the workout I found that is more likejust concentrating on whatever I was doing and I didn’t really have any questionsto ask.” This is reflected in the Current Status or Value insight category being themost queried During an activity. In fact, of the queries that took place surroundingPhysical Activity, 26.5% were During the activity, or in-situ, of which 68% of these61were simply to understand a Current Status or Value.Finally, exploration after an activity allowed for immediate reflection to takeplace which could help influence future activities. Interestingly, participants mostoften looked to perform data exploration surrounding Physical Activity after thecompletion of and regarding the activity itself (32/66 queries). This reflection Afteran activity was mainly of the Current Status or Value, Historical or Trend, andCombination or Comparison insight categories.From the interviews held with participants, we note that these captured resultsmay not provide the entire picture. First, while our results provide a general under-standing of what queries and when these queries are desired, we note that acrosscombined insight categories, daily activity, and relation to the activity, our resultsshow queries reported in 128 unique combinations of these. This highlights thedeeply unique and personal aspect to personal health data exploration needs. Sec-ond, two participants suggested that while they may utilize different in-situ mo-ments within an activity to explore their data, the activity itself was not always thedetermining factor when they aim to explore their data on the smartwatch. Notably,current overarching goals set by a participant and how long they had used a smart-watch for tracking data could affect the type and time of their exploration. As P7discussed, ”Because I’ve been using it [the smartwatch] for like a couple years,uhm, I think I’m pretty good at like knowing what kind of workout will make me hitmy calorie burn goal or get the steps I need or those kinds of things.”4.4.3 Natural Language for Personal Health Data Exploration byLay IndividualsParticipants had first-hand experience using natural language throughout ourin-situ study. This included being in different environments and surrounded byothers. We note that all participants utilized speech to record their queries, with twoparticipants typing a subset. This aspect allowed us to discuss during the interviewparticipants’ thoughts towards the use of natural language.62Positive Reaction Towards the Use of Speech Based Smartwatch InterfacesParticipants were overall excited for the potential to use speech to explore theirpersonal health data. Sixteen (16) participants suggested they would use a speech-based interface for data exploration on smartwatches if it was available. P13 stated”I’d prefer it because it’s quicker for me. You know, like if I’m in the middle ofdoing something, it’s easy just for me to say it and then move on.” with P16 adding”the responses get more tailored, like if you ask it and it like answers perfectly, itjust makes it even better to use.” P12 discussed how natural language could evenbenefit the in-situ nature of smartwatch use during walking, ”It works really reallywell while walking, ... if you have your earbuds in or like if you lift the watch toyour mouth you can get a decent signal without much interruption and you don’thave to talk super loudly.” These discussions suggested that speech could be a greattool for smartwatch interaction, which has an already limited interaction space andbecomes even more limited while mobile [176].Reduced Privacy Concerns for Speech-Based Personal Data ExplorationPrior research with a focus on natural language and speech as an input modalitydiscusses limitations regarding privacy and social acceptability [55, 98, 116]. Thisincludes disrupting others and the awkwardness of talking to a device. Throughour discussion with participants, we found that half (9) had limited to no con-cerns regarding the use of natural language for personal health data exploration.Natural language was mainly seen by participants as beneficial and acceptable fortwo reasons. First, natural language is becoming more commonplace, through im-proved interfaces and the broader use of Bluetooth headphones/earbuds. Second,the data exploration that was often desired was not seen as overly private, personal,or specific to a user; thus, participants were open to speaking the questions andcommands around others if needed. P15 summarized this by saying, ”This kind ofstuff [the responses provided] is more objective data, which I guess to me, is notas private as like thoughts and opinions, that’s more subjective. That I’d be con-cerned about someone thinking negatively versus like if they see that oh she didn’tget out of bed today.” We, however, note that for half of our participants who were63concerned with using natural language for personal health data exploration, theirqueries were seen as being increasingly personal to them, and thus were uncom-fortable in public settings. Yet, while this still hinders the potential use of naturallanguage interfaces for some, we remain to see an overall difference in attitudecompared to the prior mentioned works.Additionally, as personal health data exploration has the potential to becomeincreasingly personal, applications will likely become more aware of the uniqueexploratory needs of users. Previous research has suggested that privacy concernsdo not determine whether people use mHealth applications or not [16]. In relationto this, all our participants (18) suggested that, due to the increase in perceivedbenefit, as long as the application came from a trusted source and had the nec-essary measures in place [205], they had no problems providing these types ofexploratory questions or commands. This follows a privacy pragmatist approachwhich suggests a person may have strong feelings about privacy, yet they are will-ing to allow access to their information for their own benefit [87], and is often seenwithin younger populations [102].4.5 Discussion and Future Work4.5.1 In-Situ and Non-In-Situ Preparation-for-ActionReflecting on personal data using a smartwatch can occur increasingly close tothe action, for which the reflection is related, to benefit on-the-fly decisions [3, 69,107]. In fact, when reflection and action are related, Ploderer et al. [149] suggestthere exists reflection-in-action (i.e., real time and Current Status or Value insight)and reflection-on-action (i.e., aggregation of data such as Historical or Trend, orContextual insight) enabling both maintenance and discovery, respectively [108].Rather than reflection-on-action only taking place in-situ and immediately afteran activity, our results also showcase a need for discovery and reflection whileaway from an action for which the query would be related. We further postulatethat smartwatch users are looking for additional discovery through preparation-for-action. This form of exploration was deemed as beneficial by our participants most64often for Preemptive and Proactive as well as goal and performance insight. Thispreparation-for-action not only happens in-situ, immediately prior to an action, butalso away from the action for which the reflection was related. Many researchworks focus on the in-situ exploratory capabilities of the smartwatch [3, 86, 99,163], however expanding on the smartwatch’s capabilities during non-in-situ andprior to in-situ usage scenarios could be critical for further adoption, continued use,and a range of unrealized benefits.4.5.2 Query Insight Category Dependent on In-Situ ActivityWhile coding our data, we recognized that identical queries could lead to dif-ferent insight categories. For these, the insight category of the query was highlydependent on the daily activity being performed and its relation to the query (i.e.,whether the query was being reported in-situ or not). As an example from ourdataset, ”What is my average walking pace?” (P3) can imply and elicit differentmeaning depending on when it is asked. For instance, if the query was asked in-situwhile during the middle of a walk, the answer could likely be seen as the averagewalking pace of only the current walk (Current Status). However, if the querywas asked while sitting down at work the answer may require the calculation ofthe average walking pace across all walks recorded (Historical or Trend). Thus,utilizing contextual, when and where, information available from the smartwatch’ssensors as well as user-initiated activities can at times become a key componentin understanding a lay person’s personal health data query and information needson a smartwatch. This can then be crucial in regards to formulating appropriateresponses to allow for lightweight and transient data exploration that is beneficialto smartwatch users.4.5.3 LimitationsWe recognize that our strict inclusion criteria resulted in the exclusion of indi-viduals with impairments. Thus, our dataset is not fully representative of all whoutilize a smartwatch for personal health data collection, exploration, and healthmonitoring. While our work largely provides a general understanding of personal65health data queries on smartwatches, we suggest these aforementioned user groupsshould be studied in their own regard.Due to the early nature of our work, we chose to not provide feedback to ourparticipants (i.e., the answers and data representations in response to queries). Thisallowed us to capture a broad range of desired smartwatch data queries withoutinfluencing further potential responses. However, this design does limit us in un-derstanding any forms of continual and serendipitous exploration on the smart-watch, such as asking a single followup question based on feedback given. Duringtypical data exploration, one explicit query is often not enough to fully representwhat is interesting to a person. Feedback given in response to a query can oftenlead to subsequent and unanticipated queries [129]. Moreover, this does not al-low for discovery of new and interesting information beyond what is asked. Whilethe smartwatch may not allow for lengthy data exploration, the limit to which thesmartwatch can enable follow-ups, the relationship between the smartwatch andother data exploratory tools (i.e., smartphone, tablet, desktop), and the impact thatgiven feedback has on smartwatch personal health data exploration should be fur-ther studied.Finally, as with any elicitation study there are additional limitations to note.First, we are likely not able to capture all usage scenarios participants may experi-ence, and for which a desire for smartwatch data exploration could arise. Examplesof such times include a person preparing for a marathon or someone on vacation.Second, participants often do not realize a device’s full potential limiting the rangeof responses submitted. For example, blood glucose monitoring has a potentialfuture within smartwatches, yet was not a component in any queries reported byour participants. We aimed to mitigate both of these limitations through recruitingeighteen participants, each running the study for one full week (including week-days and a weekend), and expressed for participants to not worry about currenttechnological limitations instead focusing on queries that they desired to be possi-ble. As such, we believe that our results remain to provide a broad understandingand new insight into queries desired, which in future can be translated to a rangeof usage scenarios and newly captured data.664.6 SummaryThis chapter offers an empirical understanding of what personal health dataqueries smartwatch users wish to explore throughout their daily lives. Through anin-situ diary study with 18 participants over a week, we captured queries that havethe potential to facilitate exploration on a smartwatch in various daily contexts.Through the results of this data collection and a final interview with participants,we offer the elicited queries through a public dataset (i.e., what), report on queryinsight categories (i.e., why), and query relation to daily activities (i.e., when andwhere). Participants reported a desire to utilize the smartwatch for momentary andimmediate personal health data exploration, not only during in-situ moments butalso across a range of daily activities. We suggest several key implications for thedesign of smartwatch mHealth applications; including supporting preemptive andproactive exploration, expanding upon current status and contextual exploration,allowing for exploration away from an in-situ tracked activity, and the offering ofnatural language interaction.67Chapter 5Towards Natural LanguageInteraction for Personal HealthData Queries on SmartwatchesWith the knowledge of what queries are desired, we can now beginto focus on how to enable interaction for such queries. This chapter,and the next, centers on the following broad research question: Howcan we expand the expressivity of data exploration on the smartwatch,through both multi-modal input and output modalities, within the con-fines of the smartwatch’s capabilities and data exploration needs?This chapter specifically highlights necessary components to considerwhen allowing for the handling of the given queries. Work presentedin this chapter resulted in a publication in the conference on MobileHuman-Computer Interaction [156]. The article was a collaborativeeffort among myself, Charles-Olivier Dufresne-Camaro, and PourangIrani. Charles-Olivier Dufresne-Camaro conducted the coding pro-cess alongside myself and provided editorial support. Any mention of‘we/our’ in this chapter refers to my co-authors.685.1 IntroductionSmartwatches, through advancing input modalities such as touch, speech, ges-turing, and buttons/dials, have the potential to enable broader interaction with thecollected data. Fundamentally, however, our lack of knowledge surrounding theinteractive requirements for personal health data queries hinders progress. Morespecifically, in this chapter, we focus on the following research question:RQ1: Through analysis of the personal health data queries collected,what components comprise a personal health data query desired forexploration on the smartwatch and provided by lay users?Figure 5.1: Components within a personal health data query explored within thischapter. We highlight various interrogatives (e.g., ”What was”), aggregations (e.g.,”peak”), data attributes (e.g., ”heart rate”), and filters (e.g., ”last hike”) to be con-sidered when processing a query.In this chapter, we focus on understanding and characterizing components ofpersonal health data queries desired for exploration on the smartwatch. Through-out, our motivation lies in increasing the capability for data exploration directly on69the smartwatch. Yet, results conveyed in this chapter can be seen as being benefi-cial for a range of devices. To achieve our research goal, we extend and compareto work previously done [154], through the analysis of our previously captureddataset. We explore various dimensions, including attributes of data requested,aggregation methods, mechanisms for filtering, and interrogatives used within thequeries; see Figure 5.1 for an example of these dimensions combined within asingle query. By analyzing across these dimensions, we provide a better under-standing of how people want to explore and access their personal health data onsmartwatches. In turn, the results shared can influence interaction in applicationscatering to smartwatch data exploration.Our contribution is as follows:C1: Identification of key dimensions in personal health data queries,such as interrogatives, data sources, aggregations, and filtering mech-anisms. These insights provide a foundation for enhancing the naturallanguage processing capabilities on a range of smartwatches, key tothe development of part-of-speech tagging capabilities.5.2 Relevant Related Work5.2.1 Natural Language Query AnalysisIn this section, we highlight work on natural language query analysis as theseoften must characterize and dismantle queries for understanding and processing.While we analyze a natural language query dataset in this work, we highlight thatwe do not propose natural language as the only means of interaction with personalhealth data on the smartwatch. Instead, we utilize works in this area to simplygain a better understanding of the components involved when querying data and tosituate our analysis among related work.With the increasing capability for devices to perform natural language pro-cessing (NLP), even on a smartwatch, many toolkits have become available foruse [114, 124]. These toolkits help perform common NLP tasks such as languageidentification, tokenization, sentiment analysis, named entity recognition, and part-70of-speech tagging. However, these toolkits do not explicitly focus on parsing healthrelated information, nor do they offer solution if we do not know what to look forwhen performing tokenization and part-of-speech tagging. As such, we must firstgain a general understanding the components used within potential personal healthrelated queries before using these tools.Natural language interfaces (NLIs) have become increasingly popular for gen-eral interaction (e.g., Siri, Alexa, and Google Assistant have become more per-vasive in daily life) [21, 122] and for visual data exploration [98, 183]. Whilemainly focused towards data experts, research has collected and explored naturallanguage queries across multiple explicit and implicit dimensions. These includedata attributes, chart types, data encodings, aggregations, design references, ques-tion words, and verb tenses [65, 154, 169, 184, 187]. While not all dimensionsare necessary for personal health data querying on the smartwatch (i.e., some ofthe prior works focus on specific applications such as for visualization creation),these provide insight into required components which we can then code for andsubsequently analyze to handle and process a data-driven query.5.3 Dataset and Analysis5.3.1 DatasetWe use the previously collected dataset from Chapter 4 in our analysis. Withinthe dataset, our analysis focuses on the queries themselves while also incorporatingother elements of the dataset for granularity (i.e., the relation of the query to acurrent activity).5.3.2 Coding ProcedureAnalyzing the dataset, we coded components of the queries. To ensure codingaccuracy, we followed a procedure used within related work [184] and similar toChapter 4. Specifically, our procedure was as follows: Two researchers first ex-plored the queries independently, creating a coding schema that would outline thepotential dimensions that could be assessed. After discussion, dimensions were71chosen for which to analyze and code. As a team, a code book was created. Then,the same researchers individually coded a random subset (10% of all queries avail-able) of the data. The assigned codes were then compared for agreement. Until85% agreement was reached, the code book was refined and a new subset of datawas individually coded. Once agreement was obtained, the entire dataset was in-dependently coded using the finalized and agreed upon code book. Finally, anyremaining code disagreements were discussed and resolved until a full consensuswas reached and a single code was assigned for each dimension explored.The dimensions explored within the coding schema, and codes used within,followed closely with prior work exploring natural language query interfaces [154,184]. Hence, we analyze the data type requested, data attributes, filtering mecha-nisms, and the interrogatives used.Throughout, explicit and implicit/semantic codes are utilized to describe as-pects of the data. As the dataset was captured through lay-users, we allow syn-onyms when using the explicit code rather than deferring these to another code.For example, peak can be seen as an explicit aggregation for max. In contrast,how many is an implicit/semantic aggregation of count. These will be highlightedfurther in their respective subsections below. Lastly, when reporting codes, weprovide round brackets containing the count of queries the code captures and thepercentage of the dataset this represents.5.4 ResultsExploration through Commands; Information Immediacy through QuestionsGuided by the definitions and codes created by Srinivasan et al. [183] fornatural language data exploration, we found that participants’ queries were framedeither as a Question (173, 84.4%) or Command (32,6 15.6%).The majority of Commands, provided by over a half of participants (11/18),were of the Historical or Trend as well as Combination or Comparison insight cat-6We categorized two instances of queries from P18, “Summary of my sleep cycles,” as an (im-plicit) Command.72egories (21/30). Examples include “Show me a graph comparing my caloric intakeover the last week” (P7), “Show me a graph of my runs both time and distance in2021” (P11), “Give me a report for my readiness for activity” (P16), and “Com-pare cycle data from today to the same day in my last cycle” (P2). Through theseCommands, we can see that the desired outcome of the participant is not explic-itly clear (i.e., while we can try to provide an optimal visualization, there is nota discrete answer that can be given). Additionally, these Commands often sug-gested that participants had the intention to further explore or view a larger rangeof collected health data, often through on-screen visual representation directly onthe smartwatch.Conversely, a Question was often much more direct and closed-ended, with theintended result of the insight seemingly known to the participant. All participantsprovided queries in the form of a Question. Examples include “Have I stood upthis hour?” (P17), “How long on average does it take me to fall asleep?” (P3),“How many hours did I sit yesterday?” (P9), “How many steps did I get duringthat 2 kilometer walk?” (P4), and “What was my calories burned in the last 30minutes?” (P3).These Questions benefit the type and length of interaction that is typically un-dertaken by smartwatch users [148, 192], as they can allow for direct feedback.P14 discussed during the interview, “I think the question sort of implies immediacy[...] and I think it’s the immediacy that the watch would be nice if it covered.” Thisfinding provides us with valuable information surrounding the intent and perceiveduse of the smartwatch for personal health data exploration, especially while in-situ.Often times, throughout one’s day, discrete and immediate insight is valuable. Thisinsight, while potentially leading to further exploration, does not require it, andthus the use of a smartphone or desktop application is not immediately needed.5.4.1 Attributes of Requested DataThe data requested can be organized into categories, differentiated through ac-tivity and the data that is collected. Figure 5.2 (left) highlights the eight codes usedto quantify data types of interest. Not surprisingly, as the smartwatch is generally73Figure 5.2: Left: Counts of the data types the queries were focused towards. Right:Sub-codes within the Physical Activity code.used as a fitness and physical activity tracker, physical activity data constituted themajority of queries desired (105, 51%). This includes data such as steps, generaltracked activities (e.g., “Show me a history of all my dance workouts.” or “Howmany times have I worked out this week?”), and general metrics (e.g., “Did I closeall my rings today?” or “How many active minutes am I at?”); see Figure 5.2 (right)for a complete breakdown.Furthermore, heart data (e.g., heart rate, ECG) (29, 14%) and sleep data (e.g.,sleep time, wake time, duration, sleep stages) (18, 9%) were of interest. For granu-larity, we chose to code heart data separate from physical activity and sleep as thisdata is captured across and independent of both of the former mentioned codes.Not surprisingly, data that is currently tracked automatically on the smartwatch(i.e., physical activity, heart, and sleep data) were the most queried. In contrast,other data types such as weight, women’s health, and nutrition (e.g., number ofcalories or meals eaten, water intake) which require discrete input or are not cur-rently supported, were less queried. As more data becomes automatically sensedand calculated for tracking, these can be expected to also have relatively higherinterest for data exploration.Attribute references can be seen as words within a query that correspond toa data attribute or specific data point within the collected available data. These74references were either (i) explicit, where the reference in the query was specificto a data point being captured (e.g., “What is my current heart rate?” and “Howmany steps did I get during that 2 km walk?”) or (ii) implicit, where the referenceto data within the query was too broad, could hold different meaning for differentpeople, or required collection of multiple data points (e.g., “Compare my runningstats from the same time last year” and “Is my work out better at my home gym orcommercial gym?”).From the 205 total queries in the dataset, we find a large majority, 80% (164) ofqueries, utilize explicit references to data attributes. Only 20% (41) of the queriesshowcase implicit referencing. Implicit queries were less immediately data-drivenand often contained broad interest into a topic and the need to aggregate data frommultiple sources (e.g. “What’s the best exercise for me today?”). Queries withimplicit referencing of attributes may also present challenges with regard to pro-viding appropriate response. No clear difference in the use of explicit or implicitattribute referencing was seen for queries registered at different times within activ-ity or when away from activity.5.4.2 AggregationsAggregation references include words that would enable the conducting of amathematical transform on the data. This is common when performing data analy-sis (e.g., obtaining the sum, count, average, etc.). Our exploration of aggregationswas first coded into the type of aggregation requested; see Figure 5.3. We foundfive forms of aggregation and a sixth non-aggregation form: (i) Count (59, 29%)(e.g., “How many runs have I completed thus far in 2021?”), (ii) Average (29,14%) (e.g., “What is my average step count per day”), (iii) Min/Max (10, 5%)(e.g., “What was my fastest kilometer in my run?”). Other synonyms include:slowest, highest, lowest, peak, best, and worst. (iv) Total (4, 2%) (e.g., “How manymiles have I accumulated through walking, running, and biking over the course ofthis year”?), (v) Variance (3, 2%) (e.g., “How much has my pace fluctuated duringmy walk”), and (vi) N/A and Current value (97, 48%), where no aggregation isnecessary and a value is simply being requested (e.g., “What is my resting heart75Figure 5.3: Counts of the aggregations (mathematical transforms of the requesteddata) found within the queries. The aggregations are further noted by explicit andimplicit references.rate?”).We further explored aggregation references through either (i) explicit aggrega-tion, where direct reference to an aggregation transform was used, (37, 18%) (e.g.,“What’s my average walking pace per kilometer”→ Average) and (ii) implicit ag-gregation, when phrasing was used, (168, 82%) (e.g, “How long does it take aftera walk to get back to resting heart rate?”→ Average and “How many calories didI burn in the last 4 hours?” → Count). Notably, when we explore the aggrega-tions through this lens, the vast majority of aggregations are performed utilizingimplicit requests. When looking at majority, only for average and min/max didpeople utilize explicit referencing more than implicit referencing.765.4.3 InterrogativesFigure 5.4: Counts of the interrogatives (elements used to express ques-tions/commands and intent) found within the queries.Interrogatives, or question words, can provide insight into the aggregation de-sired, indicate questions versus commands, underscore query intent, and hint atappropriate forms of output (e.g., show me compared to tell me). Figure 5.4 showsthe interrogatives coded. Of interest, over 60% of queries contain either “how” (77,38%) or “what” (52, 25%) question interrogatives. These can be further brokendown into the interrogatives “how many” (53/77), often implying a count aggrega-tion, and “what is/was” (38/52), often implying a single value calculation.5.4.4 Filtering MechanismsFiltering of data is a common exploratory task (e.g., “Compare walking paceSeptember and October” filters the data to the months of September and Octoberwhile excluding other data). Four codes were used to differentiate filtering mecha-nisms used within the queries: (i) N/A (42, 20%), where no filtering was needed asthe entire data related to the query would be used, (ii) Current (34, 17%) where the77Figure 5.5: Counts of the filtering mechanisms found within the elicited queries(elements use to filter a subset of all data collected).current or most recent value would be filtered removing data collected in the past,(iii) Time dependent (85, 41%), where a notion of time was used to filter data (e.g.,“What was my highest heart rate in the last hour?”), and (iv) Activity dependent(49, 24%), where an instance of an activity is used to filter data rather than an ex-plicit notion of time (e.g., “What was my best kilometer during my run?”, “Showme my heart rate chart from today’s gym session.”, or “Was the 1st km of my hikefaster than the last kilometer today?”).Notably, activity dependent filtering is a subset of time dependent filtering,however is referenced with respect to an activity or activities rather than the spe-cific time period. The smartwatch is inherently a device that captures instances ofactivity (e.g., tracking nightly sleep, individual runs or walks, when food/water isconsumed, when weighing yourself using a connected scale). Queries containingactivity-dependent filtering allow filtering to be accomplished naturally, by refer-encing when the smartwatch was used to track data, without the need for a user toremember or further explore specific times an event or activity occurred. Interest-ingly, the number of queries containing activity dependent filtering shows increase78after an activity. This even slightly eclipses time dependent filtering immediatelyafter an activity.Table Table 5.1 provides a complete breakdown of the filtering, aggregations,and data attributes; this table provides these relation to each other along with thein-situ association as highlighted in Chapter 4.79Table 5.1: Counts of the codes for the attributes, aggregations, and filters are shown, delineated by the in-situ relation forwhich the query was elicited. We note, that 5 queries contained both time and activity dependent filtering, as such the totalcount for the filtering columns results in 210.Attributes Aggregation FilteringExplicit Implicit Explicit Implicit N/A Current Time ActivityBefore(24)20 4 2 22 13 1 6 4During(38)32 6 7 31 9 8 14 7Yes(107)After(45)36 9 13 32 2 6 16 20No(98)76 22 15 83 18 19 44 13Totals 164 41 37 168 42 34 85 49805.4.5 Components of a Personal Health Data QueryLooking at the results together, four components can be seen to make up apersonal health data query on the smartwatch. These include the interrogative orquestion word(s), the data subject or attribute(s), the aggregation term, and thefiltering mechanism. Importantly, however, not all are needed when querying data.At a minimum, all queries in the dataset contain an interrogative and data subject.This coincides with the results found by Rawassizadeh et al. [154]. Thus, somequeries do not contain aggregation or filtering terms, often implying exploration ofa current value or of all data captured (e.g., “What is my resting heart rate?”). Thisis important for interaction of data on the smartwatch, as no matter the interactionmodality considered, we must obtain this information at a minimum. Furthermore,we restate that the interrogative can be used as the aggregation term. As such, thisis not always explicitly required (e.g., “How many (→ count) steps did I take in thepast seven hours?” versus “What was my peak heart rate during my workout?”).5.4.6 Response ExpectationsThe query provided can in fact also hint at the expected response type. As such,to better understand the potential forms of output required, we further coded theappropriate and expected response type for each of the queries in the dataset.Throughout, we found four possible response types: 1) Value (115, 56.1%),a single value response (e.g., What was my average heart rate in the last hour?- would elicit a single average heart rate to be reported filtered by the conditionprovided), 2) Open (41, 20.0%), a response requiring a list of data/information ofany size (e.g., In what workouts does my heart rate reach zone 5? - would elicit alist of workouts for which the provided condition is true), 3) Binary (32, 15.6%),a yes/no response (e.g., Is my heart rate higher than normal? - would elicit atits lowest level a simple yes/no response), and 4) Range (17, 8.3%), comparativetwo-value responses (e.g., What is my average heart rate on weekdays comparedto weekends? - would elicit the output of two unique average heart rates filtered bythe provided conditions).815.5 Discussion5.5.1 Lay-Person Exploration of Personal Health DataThroughout, we target a broader audience who are interested in exploring andgaining insight from their personal health data. Our analysis is one of the fewto focus on lay-person queries. The majority of work in this area has focusedon individuals who are familiar with data exploration practices and tools (e.g.,data analysts) [94, 183, 184], with to our knowledge only one work analyzinglay-person queries for smartwatch data exploration [154].Within our findings, the influence of lay-person exploration was most evi-dent through implicit aggregation and activity dependent filtering. The majorityof queries within the dataset analyzed contained implicit aggregation, a vast dif-ference from the only quarter of queries captured with data experts [184]. Whilethe work done by Rawassizadeh et al. [154] highlights both explicit and implicitnotions of time and location, this dimension is not quantified for comparison nordiscussed for aggregation. Furthermore, prior work has put focus on time depen-dent filtering as this is a primary dimension of personal health data [98]; Rawas-sizadeh et al. also suggest time as the only filtering mechanism [154]. However,activity dependent filtering found in our analysis can be seen as a means for lay-users to more easily recall events performed rather than specific times they wereperformed. Both these implicit means of exploring data are understandably easierfor people, especially on the smartwatch where visual exploration is limited due tothe small-screen size and focus may be on a primary in-situ task at hand [14].5.6 SummaryThis chapter provides an understanding of the different components that com-prise the queries desired for personal health data exploration on the smartwatch.We identified several dimensions related to these queries, including the requesteddata types, attributes, aggregation methods, filtering mechanisms, and interroga-tives used. Our findings emphasize the need for more comprehensive and lay-useraccess to a range of differing data aggregation and filtering options. By considering82these insights, practitioners and application designers can develop better applica-tions that meet users’ specific needs and preferences for interaction with personalhealth data. The implications of this chapter provide valuable guidelines for fu-ture works aimed at enhancing the utilization of smartwatches as effective personalhealth tracking and exploration devices, including our own work within Chapter 7.83Chapter 6User Preferences of VoiceAssistant Answers to PersonalHealth Data QueriesContinuing our focus on enabling interaction for such queries, thischapter shifts attention to the complement of input: output. Specif-ically, as there is plenty of smartwatch visualization research con-ducted and guidelines proposed, we explore how to respond to per-sonal health data queries through spoken, voice assistant, output. Spo-ken output can provide an efficient and complementary means to re-spond to personal health data queries, especially while on-the-go whenthe visual system can be overloaded [19]. Work presented in this chap-ter resulted in a publication in the conference on Conversational UserInterfaces [159]. The article was a collaborative effort among myself,Yumiko Sakamoto, Jaisie Sin, and Pourang Irani. Yumiko Sakamotocollaborated on study design, data analysis, and editorial work. JaisieSin collaborated on study design and editorial work. Any mention of‘we/our’ in this chapter refers to my co-authors.846.1 IntroductionThe rapid integration of voice assistants (VAs) within an array of devices istransforming the way we interact with technology. VAs now offer seamless inter-action across standalone smart-speakers, smartphones, and wearables such as thesmartwatch. The current capabilities of VAs allow for the handling of general com-mands (e.g., controlling entertainment devices) and answering of basic questions(e.g., the current weather).Now, pushing the boundaries of what a VA is capable of, we refer to the sto-ried example provided in the introduction of this thesis. As Sam asks, ”How doestoday’s pace compare to my last six hikes?” - the VA could respond with ”Yourpace today is slower than your average pace by 2 minutes and 28 seconds per kilo-meter.” Currently, however, this level of question and answering through VA inter-action is not possible, yet is understandably very plausible. Furthermore, researchhas shown that it can even be beneficial to use natural language (i.e., auditory orspoken) interactions when the visual system is overloaded [20]; this occurs oftenwhile a person is in-situ and actively engaged in a workout or other activity.Looking at past work in this area, research has focused on the use of VAs for ad-dressing general health knowledge questions [2, 127] and for their use in healthcare[50]. Furthermore, research has explored VA answer structures, specifically min-imal, keyword, and full sentence, however only for common tasks and questions[71]. Ultimately, however, we do not have an understanding of how VA answersshould be structured, specifically for personal health data questions. Uncertaintyabout how to provide answers is one such element that limits the potential of VAsto adequately respond to users’ growing personal information needs, and thus thegreater potential for databiting while on-the-go. This missing knowledge can evenfactor into user satisfaction and adoption of these systems all together [175, 188].In this chapter, we focus on the following research questions:RQ1: How should voice assistant answers be structured to supportspoken responses to personal health data queries?RQ2: How are voice assistants perceived as a tool to query personal85health data compared to general voice assistant interactions?To address this knowledge gap we implemented a browser-based pseudo-VAwhich allowed for study participants to ask and receive answers to a variety of per-sonal health data questions. Across two online studies, 82 participants interactedwith our VA, asking questions and receiving answers to a combined total of 30unique question-answer pairs. Within our studies, we explored three answer struc-tures (i.e., Minimal, Keyword, and Full Sentence), each paired with questions fromfour personal health question response types (i.e., Open, Range, Binary, Value)and six known personal health insight categories (i.e., Contextual, Preemptive andProactive, Goal and Performance, Combination and Comparison, Historical andTrend, and Current Status).We found that in contrast to general VA question and answers, where shortanswers were preferred, participants preferred the Full Sentence answer structuregiven the response types and insight categories studied. Moreover, Full Sentenceanswers allowed for clarity in the answer when data was ambiguous, yet remainedefficient despite the longer mean response time compared with other answer struc-tures.Our contributions are two-fold:C1: Two studies, utilizing a custom-built browser-based pseudo-VA,allowing participants to explore personal health questions and answersacross a total of 30 unique personal health question-answer pairs. Weoffer the implementation of our browser-based pseudo voice assistantthat is used within both studies.C2: Comprehensive user preferences of different answer structures forvoice assistants in response to personal health data queries. We gatherempirical insights into perceived answer quality, behaviour, compre-hensibility, efficiency, and preference for personal health data ques-tions. The findings provide elementary guidelines that complementknown guidelines for current voice assistant question and answeringcapabilities.866.2 Relevant Related Work6.2.1 Interaction with Voice AssistantsVoice assistants (VAs) are quickly being adopted for use, now with over halfof Americans using a VA in their homes [143]. To provide reason for such adop-tion, current research on natural language interaction (e.g., speech) has highlightedits utility in facilitating micro and hands-free interaction [5], especially when thevisual system is overloaded (e.g., while tracking a walk) [19] and within varioussimultaneous activities [151]. These reasons for using a VA fit perfectly within thevision of databiting and the necessity for lightweight and transient exploration thatdoes not hinder an ongoing activity, but rather assists and provides complement.As we interact with a VA using natural language, research understandably ex-plores the idea of personification and human-likeness in VA answers and dialogue[109], as well as social interactions [152]. However, in stark contrast to this, othershave highlighted a need for technical systems [53], rather than human. Notably,increased human-likeness tends to increase trust and privacy concerns [27, 119],concerns that are at the forefront for personal health data [191, 207]. The contrastin human-like versus technical can greatly influence the experience of the VA in-teraction. As such, it is not only important to understand answer structures, butalso necessary to continue to uncover preferences and perceptions as new VA in-teractions, such as for personal health data queries, become possible.6.2.2 Voice Assistants in Health ContextsConversational user interfaces (CUIs) encompass a variety of interactive sys-tems designed to facilitate natural language and conversational interactions. CUIsare an emerging means for people to gain general health-related information [2, 18,127, 198], to self-report health and fitness data [121, 153], and to fill out health-related forms when health literacy is low [101]. CUIs in health contexts can takevarious forms, such as text-based chatbots, virtual assistants, and voice-activatedplatforms [50, 104]. Each of these forms offer unique capabilities for supportinghealth-related interactions and at times offer solutions to potential hurdles such as87mispronunciation and recognition of medical terms and credibility [12, 146].VAs, as a subset of CUIs, are becoming increasingly adopted as they are an em-bedded technology in many of our smart devices. VAs in health contexts currentlyallow users to query general health topics and symptoms [18], often providing linksto online sources. As personal health data monitoring and exploration becomesmore commonplace [35], along with a rise in Artificial Intelligence and Large Lan-guage Models, the intersection of VA interaction and personal health data queryingwill quickly become a reality. Despite this potential, current VA systems do notfully leverage their capabilities for question-answering tasks. With little activelyworking technology in this area, we largely do not know how a VA should answerpersonal health data queries, or how a VA is viewed for such tasks.6.2.3 Voice Assistant Answer StructuresInvestigations into VA communication styles have often focused from user todevice. However, the feedback and response styles of the VAs not only play intoa major design principle [139], but also a critical role in users’ perceptions andadoption of these devices [175]. Yet, the effect of machine-to-human expressionhas been under investigated. For example, while factors like interruption [56] andconversational repair [46] have been explored, to our knowledge, only one workhas explored answered structures for general VA use [71].By exploring marketed VA answer practices as well as conducting a user studyto compare various answer structures, Haas et al. [71] provide a comprehensiveanalysis of the experiences and user preferences regarding different VA answers.They first found that commercial VAs opt to convey more humanistic and full sen-tence answers for many common questions and commands. Only for home au-tomation, where the outcome of the VA interaction is also noticeable within onesenvironment, were shorter keyword answers used. Then, in their own user study,Haas et al. found that minimal answers (those that provide the answer and noother supplemental or contextual information) were preferred for most command-based interactions while keyword answers (those that provide the answer and abrief confirmation of the keyword in the request) were preferred for most query-88based interactions. This highlights a more utilitarian use of a VA than is currentlyrecognized, where brief and basic answers suffice. Yet, many of the general usequestions and commands explored are not personal. More specifically, there is aneed for research to explore and understand people’s expectations regarding answerstructures when querying personal health data through VAs.6.3 User Studies MethodologyWithin two online user studies we used the same apparatus, procedure and datacollection described below for both, and derived from previous work [71]. Wefollow the same procedure as previous work so that we can make direct compar-isons and reflections between our findings, focused on personal health data ques-tion and answers, and previously published findings, which focus on currently af-forded question and answers as well as task-based commands.6.3.1 ApparatusWe built a browser-based pseudo-VA using Javascript and the WebSpeech API7.The WebSpeech API enables two important elements: (1) Speech recognition(i.e., recognizing a personal health data question) and (2) Speech synthesis, wherespeech (i.e., the answer to the question) can be vocalized, for which we used the“Google US English Female” voice. We chose the female voice for its similarityto the default voices current VAs use. The pseudo-VA used within the user studiescan be demoed in Google Chrome8. Furthermore, the underlying code providesimplementation details which can be used in future.We utilize the term pseudo to describe our VA, as the functionality of our VAwas limited to only handle questions desired within our user studies. As such,the VA was not fully functional as we would expect a commercial VA to be (e.g.,Google Assistant, Siri, Amazon Alexa, etc.). While this limits capability, it pro-vides experimental control. During interaction with our pseudo-VA, the partici-pant’s question would be recognized and then processed by checking for keywords7https://wicg.github.io/speech-api/8https://vaphdqa.github.io/vaphdqa/89(and varying synonyms) specific to each question. Only when our pseudo-VA rec-ognized all required keywords, was the appropriate answer vocalized. If recogni-tion could not be made (e.g., required keywords were missing) a response wouldencourage the participant to follow the prompt given and to try again.When interacting with our pseudo-VA, a button on-screen was used to triggerthe start of recognition, rather than utilizing keyword detection as many commer-cial VAs do. This design choice was done for privacy reasons. As such, partici-pants had control of when recognition would begin, with recognition ending oncethe WebSpeech API recognized a natural stop in the question spoken. Further-more, the text in the button would change to inform the participant when the VAwas listening.6.3.2 ProcedureEach study comprised three distinct stages: an introduction, the main trials, anddemographic surveys. The procedure, VA recognition, and data collection were re-fined across two separate pilot phases for each study. In the first pilot phase, asingle participant took part in the study while sharing their screen on Zoom andproviding think-aloud feedback throughout. The second pilot phase involved threeparticipants, none of whom took part in the first phase, who mirrored the proce-dures that our main study participants would follow. This process allowed us torefine the methodology and address any potential issues.Within each of the studies, we include two attention check questions. Onequestion was built into the pseudo-VA, mimicking a typical study trial. Specif-ically, all participants had to ask about their calorie intake for the day, to whichthe pseudo-VA responded with instructions that needed to be followed on the nextscreen. The second attention check question was simply slotted within a demo-graphic survey, asking participants to choose a specific response.Throughout the studies, the pseudo-VA was embedded into a larger Qualtricssurvey and opened in Google Chrome. Once open, participants proceeded throughthe survey, as described below. The study procedure was approved by our insti-tution’s ethics review board. All participants provided informed consent prior to90starting a study.Study IntroductionBefore starting the main trials, participants were guided through a study intro-duction process, designed to optimize interaction and comfort with the browser-based pseudo-VA and the study in general. The study introduction comprised ofthe following elements: (1) Voice input and output verification. Participants couldtest their input and output by continually interacting with the pseudo-VA, usingpreset and generic questions. (2) Study task. We provided a brief overview of thestudy’s context and tasks. (3) Practice trials. Participants experienced three trials,during which they were prompted to check their calendar for the following day.Note, as with the main trials, we did not have access to a participant’s personaldata; throughout, generic data was used when providing responses. (4) Personalhealth data preference. To further enhance interest and focus within our study, weasked participants to choose between heart rate and step count data as their pre-ferred question and answer topic for the remainder of the study. These topics forpersonal health data exploration were chosen for their popularity [3, 156], and canbe seen as topics that are of interest throughout many daily contexts [157].Main TrialsThe main trials in our study were structured into blocks, with each block focus-ing on questions requiring a specific response type (i.e., Open, Range, Binary, andValue) or pertaining to a specific category of personal health data (i.e., Contextual,Preemptive and Proactive, Goal and Performance, Combination and Comparison,Historical and Trend, and Current Status) in each of Study 1 (see Table 6.1) andStudy 2 (see Table 6.2) respectively.To ensure balance and minimize potential order effect, both the question blocksand answer structures within a single question block were ordered using a LatinSquare design. Participants were randomly assigned to a question block order-ing and for each question block randomly assigned an answer structure ordering.Throughout, we ensured as best we could that an equal number of each order for91both the questions and answers was shown across all participants in the study. Thisstudy design allowed for a systematic exploration of how different answer struc-tures performed across various dimensions of personal health data queries.Trials were grouped in threes, corresponding to each question block. Partic-ipants asked the same question for each trial in a question block while receivingeach of the three differently structured answers. We used a repeated measuresdesign, in which each participant had to ask each of the four or six different ques-tions for each of the three answer structures. This results in a total of 12 or 18question/answer interactions (trials) for each participant, for Study 1 and Study 2respectively.During each trial, participants were prompted to ask a personal health dataquestion using their own words. After providing the question, participants heardan answer given by the VA. Following a successful interaction with the VA foreach trial, participants were automatically directed to complete the User Experi-ence Questionnaire Plus (UEQ+) [100, 167]. The UEQ+ survey provides insightsinto participants’ subjective experiences and satisfaction levels with the voice as-sistant answer structures.Following the work which created the UEQ+ survey for voice assistants [100],and previous work [71], we paired two semantic scales to compose a single expe-rience quality factor. We chose two semantic differentials with the highest foundloading from three scales: 1) Behaviour, consisting of the scales artificial - naturaland unlikable - likeable, 2) Comprehensibility consisting of the scales complicated- simple and unambiguous - ambiguous, and 3) Efficiency consisting slow - fastand inefficient - efficient. Finally, we incorporated a fourth scale: 4) Quality, whichconsisted of the scales with the third and fourth highest loading, yet still recom-mended by the creators [100]. The Quality scale consisted of the scales unintel-ligent - intelligent and inappropriate - suitable. We opted for these scales as thescales with the highest loading for Quality were not helpful - helpful and useless- useful. Through preliminary discussion, we felt these scales required the use ofactual personal health data for a participant to fully evaluate these semantics.After completing a question block, participants were asked to rank-order theirpreference for the three answer structures heard during that block. This ranking92task aimed to elicit participants’ subjective preferences with a forced response akinto selecting a single option if given the choice on their own device.It is important to again note that all answers provided during the trials weregeneric, and no personal health data from participants was used in generating VAanswers. This approach ensured consistency and privacy in the study design whilestill allowing for a thorough exploration of participants’ preferences with respectto VA answer structures for personal health data queries.Surveys and Open FeedbackOur study incorporated surveys and open feedback to gather insights into par-ticipants’ demographics, personality traits, and preferences regarding voice assis-tant usage. We gathered information about age, gender, background, VA usage,and personal health data collection practices. Additionally, participants completedthe Ten-Item Personality Inventory (TIPI) [67] to assess personality traits and theAttitudes For Technology Interaction (ATI) [64] scale to assess their attitudes andcomfort with technology. We also explored attitudes towards voice assistants forboth general and personal health data use, separating these surveys to mitigatecarry-over responses. Finally, participants were given the opportunity to provideopen feedback to express their thoughts, suggestions, and/or concerns.6.3.3 Question Answer StructuresThe questions prompted for participants to ask within each study were derivedfrom a public dataset of personal health data queries captured in-the-wild from ex-perienced smartwatch users [157]. The questions within our studies were carefullyselected to represent the elicitation of different response types in Study 1 and var-ious desired categories of personal health insights in Study 2. Within each studysection below, we further highlight the questions used and how they were derived.For both studies, the questions were answered using three answer structures:Minimal, Keyword, and Full Sentence. These answer structures have been utilizedin previous work [71], and mimic the Full and Brief options offered by Google’sAssistant. Minimal answers solely contain the information required to answer the93question. Keyword answers provide the answer and confirmation of the questionasked. Full sentence answers provide full sentence responses emulating human-like sentence structure. Notably, Minimal and Full Sentence answer structuresfollow humanistic response behaviours, while Keyword does not. As we allowedfor both step count and heart rate as question topics to promote interest and en-gagement within the study, we aimed to ensure as much consistency as possiblebetween the questions and answers for either data source. This included ensuringas much commonality between answers as possible while also ensuring answerswere of similar lengths; see Table 6.1 and Table 6.2 for all questions and answersused (for both step count and heart rate topics). No matter the question topic cho-sen by participants, each participant saw the same number of question and answertrials during their respective study (i.e., 12 in Study 1 and 18 in Study 2).6.3.4 Participant RecruitmentRecruitment of participants was done through Prolific9. Prior to participation,potential participants completed an eligibility survey to ensure they met our inclu-sion criteria: participants were required to (1) Have their first language be English.Therefore, answer structures could be properly evaluated by native speakers; (2)Have used a VA before. As such, participants held experiences either good or badwith the use of VAs and their answers; (3) Currently collect and/or explore per-sonal health data. By having experience with personal health data, participantscould have defined expectations and preferences.When partaking in the main study, we asked participants to place themselvesin a room with as little distraction as possible. We further required the use ofa desktop/laptop computer with Google Chrome installed and for participants tohave working microphone and speaker/headphones. No matter their responses, allparticipants who took part were paid 0.25 GBP for completing the eligibility and6.5 GBP for completing the study.9https://www.prolific.com/946.3.5 Data and AnalysisThe data from each study is analyzed across four key areas: (1) UEQ+ scoresfor each question-answer pair, (2) rank order preferences, (3) attitudes towardsVAs, and (4) open feedback.Important differences are highlighted for each study below with means andthe confidence interval boundaries listed in text and graphically presented withinfigures. Throughout, we opt to utilize confidence intervals rather than relying onp-values. Graphically presenting confidence intervals allows us to systematicallyassess any effects at play while also gauging practical significance [47, 54]. Re-porting on confidence intervals rather than p-values has become increasingly pop-ular in HCI literature [11]. Confidence intervals offer greater understanding for abroader audience and do not suffer from the illusion of truth sometimes providedby a p-value [54]. As no statistical testing is being performed, we explicitly do notutilize significance terminology. Rather we find and report on differences throughconfidence intervals which do not overlap [47, 54]. Before calculating the meansand confidence intervals all outliers for each pairwise analysis were removed asany data point outside three standard deviations.As the creators of the UEQ+ scale do not test for inter-item reliability, wecalculated the Cronbach’s α for each pairwise comparison using .70 as a cutoff[140]. When the α did not reach the cutoff for any UEQ+ scale, we report on thecombined UEQ+ scale, while further exploring its semantic differentials separately.6.4 Study 1 - Response Types6.4.1 QuestionsFor Study 1, the prompted questions were categorized by the type of responsethe question would elicit. The possible response types, as seen in Chapter 5,are: 1) Value, a single value response, 2) Open, a response requiring a list ofdata/information of any size, 3) Binary, a yes/no response, and 4) Range, compar-ative two-value responses.Table 6.1 shows all questions and answers used within the study. Answers took95an average of 2.8, 4.5, and 6.3 seconds to convey to the participant for each ofMinimal, Keyword, and Full Sentence answer structures respectively.96Table 6.1: Questions and answers used in Study 1. The forward slash denotes the separation between the choice of heart rateor step count topics, one of which was chosen by the participant for use throughout the study.ResponseTypeExpected Question Minimal Keyword Full SentenceValueWhat was my average heart rate in the lasthour? / What was my step count in the lasthour?71 beats per minute / 1,375stepsAverage heart rate last hour, 71 beats perminute / Step count last hour, 1,375 stepsIn the last hour, your average heart rate was 71 beatsper minute / In the last hour, your step count was1,375 stepsOpenIn what workouts does my heart rate reachzone five? / In what workouts does mystep count reach 2,000 steps?Indoor running, outdoorcycling, and rowing / Indoorrunning, outdoor running, andhikingWorkouts reaching zone five, indoorrunning, outdoor cycling, and rowing /Workouts reaching 2,000 steps, indoorrunning, outdoor running, and hikingIn the past, indoor running, outdoor cycling, androwing workouts have brought your heart rate intozone five / In the past, you have reached 2,000 stepsduring indoor running, outdoor running, and hikingworkoutsBinaryIs my heart rate higher than normal? / Ismy step count higher than normal?Yes / YesYes, heart rate higher than normal / Yes,step count higher than normalYour current heart rate is higher than your normalheart rate / Your current step count is higher than yournormal step countRangeWhat is my average heart rate onweekdays compared to weekends? / Whatis my average step count on weekdayscompared to weekends?85 beats per minute comparedto 76 beats per minute / 9,820steps compared to 10,680stepsWeekdays, 85 beats per minute.Weekends, 76 beats per minute /Weekdays, 9,820 steps. Weekends, 10,680stepsYour average heart rate during the week is 85 beatsper minute. While on the weekends, your averageheart rate is 76 beats per minute / Your average stepcount during the week is 9,820 steps. While on theweekends, your average step count is 10,680 stepsMean and Standard Deviation ofResponse Times (seconds):M=2.6, SD=1.3 / M=3.0,SD=1.6M=4.2, SD=1.1 / M=4.8, SD=1.4 M=5.7, SD=2.0 / M=6.7, SD=2.4976.4.2 ParticipantsThirty-four participants took part, with one participant failing the attentionchecks. Of the 33 participants whose data we used for analysis their ages rangedfrom 18 to 63 years old (M = 35.0, SD = 11.7; 24 Females, 9 Males). Furthermore,19 participants self-identified as White, nine as Black/African, three as Hispanic,and two as Asian. Twenty-two (22) participants indicated they use Google’s Assis-tant, 17 use Apple’s Siri, 11 use Amazon’s Alexa , six use Samsung’s Bixby, andtwo use Microsoft’s Cortana. As marketed VAs offer similar responses [71], wewere not concerned with bias from using a specific VA. Fifteen (15) participantsstated using a voice assistant more than once per day, two once per day, eight afew times a week, two once a week, and six less than once per week. On average,participants had been collecting personal health data for 56.5 months (SD = 32.4months). On average, Study 1 took 24.9 minutes to complete (SD = 11.2 minutes).Eleven (11) participants chose heart rate while 22 participants chose step count astheir data type to explore.6.4.3 ResultsQuality, Behaviour, Comprehensibility, EfficiencyMean participant ratings with 95% confidence intervals for answer quality, be-havior, comprehensibility, and efficiency are shown in Figure 6.1.Quality For answer Quality (a composite of helpfulness and usefulness), differ-ences were observed within Range, Binary, and Value response types, but not forOpen. Within the Range response type, participants rated the Full Sentence an-swer structure as having the highest Quality (M = 6.36, CI [6.11, 6.61]) comparedto both Keyword (M = 5.77, CI [5.48, 6.07]) and Minimal (M = 4.94, CI [4.56,5.32]). Furthermore, Keyword answers were rated higher than Minimal. For theBinary response type, the Full Sentence answer structure received higher ratings(M = 6.11, CI [5.84, 6.38]) compared to both Keyword (M = 4.95, CI [4.56, 5.35])and Minimal (M = 4.38, CI [3.79, 4.97]). For the Value response type, participants98rated the Keyword answer structure lower (M = 5.74, CI [5.43, 6.06]) when com-pared to the Full Sentence answer structure (M = 6.33, CI [6.09, 6.57]). Only whenthe response type was Open, the answer structure did not influence the perceivedQuality.We compared the quality ratings of Keyword answer structures across the fourresponse types. In the Binary response type, Keyword answers were rated lower(M = 4.95, CI [4.56, 5.35]) when compared to other response types. Similarly,for the Minimal answer structure, Binary responses received the lowest rating (M= 4.38, CI [3.79, 4.97]) when compared to Open and Value response types. Fur-thermore, Full Sentence answers consistently received higher ratings across all re-sponse types, suggesting a potentially higher perceived quality.Behaviour In examining the answer Behaviour (a mean composite of natural-ness and likability), a similar pattern for Full Sentence was evident. Regardlessthe response type, Full Sentence always yielded the highest ranking; differencesbetween Full Sentence and the other answer structures within each response typewere found for all except Value. Moreover, the means for Full Sentence did notvary across the four response types. This was consistent with both other two an-swer structure types (i.e., the Behaviour ratings were not influenced by ResponseType). Thus, to further explore, we created mean scores for each answer structuretype. As anticipated, the Full Sentence structure yielded the highest mean (M =6.19, CI [5.99, 6.41]) while Minimum Sentence (M = 5.28, CI [4.97, 5.59]) andKeyword (M = 5.57, CI [5.32, 5.80]) did not vary.Comprehensibility In terms of answer Comprehensibility (a mean compositeof simplicity and ambiguity), only one answer structure effect was found acrossresponse types. For the Range response type, Full Sentence and Keyword answerstructures were seen as equally comprehensible while Minimal was seen as theleast comprehensible (M = 5.06, CI [4.56, 5.56]). For other response types, theanswer structure did not affect the level of comprehensibility. Noticeably, using aMinimal answer structure for both Range (M = 5.06, CI [4.56, 5.56]) and Binary(M = 5.23, CI [4.67, 5.79]) response types was seen as less comprehensible than if99(a) (b)(c) (d)Figure 6.1: Study 1 mean UEQ+ ratings with 95% confidence intervals: Quality(a), Behaviour (b), Comprehensibility (c), and Efficiency (d). Ratings are com-pared by the response types (Value, Open, Binary, Range) and answer structuresexplored in the study.100(a)(b)Figure 6.2: Study 1 mean UEQ+ ratings with 95% confidence intervals for a) Com-prehensibility - separated by simplicity and ambiguity; and b) Efficiency - separatedby speed and efficiency. Ratings are compared across the response types (Value,Open, Binary, Range) and answer structures explored in the study.101used for Open and Value.The level of Cronbach’s α for Comprehensibility was 0.66. As such we sep-arated the semantic differentials used and further explored complicated - simpleand ambiguous - unambiguous separately; see Figure 6.2a. Differences across thesemantic differentials are present for the Minimal answer structure in the Binaryresponse type. More specifically, the Minimal answer was rated as being highlysimple but is different when compared to ambiguity, suggesting the response issimple yet ambiguous.Efficiency For answer Efficiency (a mean composite of efficient and fast), nodifferences were found.However, the level of Cronbach’s α for Efficiency was 0.65. As such we sepa-rated the semantic differentials used and further explored slow - fast and inefficient- efficient separately; see Figure 6.2b. Full Sentence answers were rated as moreefficient than they were fast for the Range, Binary, and Value response types. Thissuggests that while an answer does not have the fastest mean response times, as isthe case with Full Sentence, they are still viewed as efficient by participants.Interestingly, we realized that the sentence structure did not affect the perceivedspeed. VA’s answers using Full Sentence, Keyword, and Minimal structures wereperceived equally fast regardless of the actual response time (see Table 6.1 formean response times).Preference and Attitudes Towards Voice AssistantsParticipants generally favored the Full Sentence answer structure; see Fig-ure 6.3a. If Full Sentence answers were not preferred, then Minimal was oftenthe preferred answer structure. This preference pattern suggests that participantsprioritize responses that exhibit human structuring of answers. Notably, while par-ticipants demonstrated clear preferences for the Full Sentence answer structurethroughout, only Range and Binary saw the majority of participants chose FullSentence as their preferred answer structure. This is likely due to a need for addi-tional context and clarity within these response types. In contrast, Value and Open102(a) (b)Figure 6.3: Participant’s preference of answer structure for each response type(Value, Open, Binary, Range) (a). Participant’s perceptions of voice assistants (b).response types allow for implicit interpretation and internal verification given ananswer (i.e., if asking what workouts a person took 2000 steps within, cycling is anobvious wrong answer). This internal verification leads participants to perceive aslightly lesser need for Full Sentence responses; instead, favoring greater flexibilityand brevity in the answer structure.Participants perceived voice assistants for personal health data exploration andgeneral use similarly, indicating a consistent perception across these question do-mains; see Figure 6.3b. Interestingly, participants rated highly that a VA both wasviewed as a technical system and should prioritize efficiency. Attributes related tohuman likeness and social interaction were not as strongly desired, with a VA so-cial companion being even less preferred than human likeness. This suggests thatparticipants value technical capabilities and efficiency in VAs, and may not neces-sarily expect or desire human-like or social qualities (e.g., as a trainer or coach).Open FeedbackAcross 18 unique comments, the open feedback provided by participants re-vealed three main topics. Firstly, two participants expressed a newfound interestin utilizing voice assistants for personal health data exploration. Second, regarding103the study procedures, comments were generally positive, with seven participantsexpressing enjoyment and satisfaction. However, one participant suggested a needfor a slower speed during interactions with the voice assistant. Additionally, twoparticipants suggested the use of a different voice, specifically male, during inter-actions; no other interactive comments, such as recognition issues were mentioned.Finally, in terms of preferences, participants interestingly expressed diverse and op-posing opinions. Two participants suggested favoring concise and clear responses,while another further mentioned they found human-like responses to be unsettling.Conversely, three participants preferred longer answers, particularly to confirm thatthe voice assistant understood their questions and attributing human-like qualitiesto these responses.6.5 Study 2 - Insight CategoriesIn our second study, we focus on insight categories of personal health dataquestions rather than broader response types. Notably, a question asked within apersonal health data insight category can result in most response types, dependinghow the question is asked. Given the very few differences in UEQ+ scores be-tween answer structures in both Value and Open responses in Study 1, we extendStudy 1 by choosing to study Value responses in Study 2. This decision was drivenby the versatility of Value responses, which are applicable across all insight cat-egories, whereas Open responses are not. This study then offers a more nuancedunderstanding of user preferences in voice assistant answers for personal healthdata questions.6.5.1 QuestionsThe prompted questions were chosen to represent known personal health in-sight categories, both from previous work [3, 29? , 30] and that of our own find-ings in Chapter 4. These categories include: 1) Current Status (CSV), derivedfrom a current measured value, 2) Historical and Trend (HT), provides insightinto past data, 3) Combination and Comparison (CC), derived from combin-104ing/comparing data sources, time periods, and/or activities, 4) Goal and Perfor-mance (GP), derived from user goals and performance metrics, 5) Preemptive andProactive (PP), provides insight into a future action, and 6) Contextual (CT), pro-vides context to gain insight. When choosing questions from within our previouslycaptured dataset, we first opted for questions that were categorized into a singleinsight category. However, we note that some insight categories overlap with theHistorical and Trend category.Answers took an average of 1.8, 3.0, and 4.0 seconds for Minimal, Keyword,and Full Sentence answer structures respectively. As per our study goal, all answersinvoked a Value response type. All questions and answers used within this studycan be seen in Table 6.2.105Table 6.2: Questions and answers used in Study 1. The forward slash denotes the separation between the choice of heart rateor step count topics, one of which was chosen by the participant for use throughout the study.InsightCategoryExpected Question Minimal Keyword Full SentenceCurrent Status(CSV)What is my current heart rate? / What ismy current step count?68 beats per minute / 7,350stepsCurrent heart rate, 68 beats per minute /Current step count, 7,350 stepsYour heart rate is currently at 68 beats per minute /Your step count is currently at 7,350 stepsHistorical andTrend (HT)What was my average daily heart rate lastweek? / What was my average daily stepcount last week?78 beats per minute / 10,270stepsAverage daily heart rate, 78 beats perminute / Average daily step count, 10,270stepsYour daily average heart rate last week was 78 beatsper minute / Your daily average step count last weekwas 10,270 stepsCombination andComparison(CC)Is my heart rate different from myaverage? / Is my step count different frommy average?Higher than average / Higherthan averageCurrent heart rate, higher than average /Current step count, higher than averageYour current heart rate is higher than your averageheart rate / Your current step count is higher than youraverage step countGoal andPerformance(GP)Which day of the week is my heart ratethe highest? / Which day of the week ismy step count the highest?Saturdays / SaturdaysHighest heart rate, Saturdays / Most stepstaken, SaturdaysYour heart rate is the highest on Saturdays / Your stepcount is the highest on SaturdaysPreemptive andProactive (PP)How long should I control my breathing toget to my resting heart rate? / How farshould I walk to get to 10,000 steps?Two minutes / 2.5 kilometresTo reach your resting heart rate, twominutes / To reach 10,000 steps, 2.5kilometresTo reach your resting heart rate, you should controlyour breathing for two minutes / To reach 10,000steps, you should walk a distance of 2.5 kilometresContextual (CT)Is my heart rate lower in the morning,afternoon, or evening? / Is my step countlower in the morning, afternoon, orevening?Evening / AfternoonLowest heart rate, evening / Lowest stepcount, afternoonYour heart rate is the lowest in the evening / Your stepcount is the lowest in the afternoonMean and Standard Deviation ofResponse Times (seconds):M=1.6, SD=0.4 / M=1.9,SD=0.7M=2.7, SD=0.4 / M=3.2, SD=0.7 M=3.6, SD=0.8 / M=4.3, SD=1.01066.5.2 ParticipantsOf the 52 participants who took part, three participants were removed for notpassing our attention checks. Of the 49 participants whose data we used for analy-sis their ages ranged from 18 to 67 years old (M = 34.5, SD = 11.2; 30 Females, 19Males). Furthermore, 31 participants self-identified as White, 13 as Black/African,one as Hispanic, one as Asian, two as Multiracial, and one as Middle Eastern.Twenty-eight (28) participants indicated they use Google’s Assistant, 23 use Ap-ple’s Siri, 26 use Amazon’s Alexa, five use Samsung’s Bixby, and four use Mi-crosoft’s Cortana. Twenty (20) participants stated using a voice assistant morethan once per day, five once per day, 16 a few times a week, three once a week,and five less than once per week. Participants had been collecting personal healthdata for an average of 51.5 months (SD = 34.2 months). On average, Study 2 took24.9 minutes to complete (SD = 9.4 minutes). Ten (10) participants chose heartrate while 39 chose step count as their data type to explore.6.5.3 ResultsQuality, Behaviour, Comprehensibility, EfficiencyMean participant ratings with 95% confidence intervals for answer quality, be-haviour, comprehensibility, and efficiency are shown in Figure 6.4.Quality Within insight categories, no differences in perceived quality are seen forMinimal and Keyword answer structures. However, Full Sentence shows a highermean Quality for Contextual (M = 6.15, CI [5.91, 6.38]) and Goal Performance(M = 6.31, CI [6.11, 6.5]) insight categories compared to other answer structures.As well, Full Sentence (M = 5.94, CI [5.7, 6.18]) shows a higher quality thanMinimal (M = 5.15, CI [4.81, 5.5]) in the Combination and Comparison insightcategory. Notably, across insight categories, there are no differences for each of theindividual answer structures, suggesting that the insight category does not changeperceived Quality of an answer structure.107(a) (b)(c) (d)Figure 6.4: Study 2 mean UEQ+ ratings with 95% confidence intervals: Quality(a), Behaviour (b), Comprehensibility (c), and Efficiency (d). Ratings are com-pared by the insight categories (Current Status and Value (CSV), Historical andTrend (HT), Combination and Comparison (CC), Goal and Performance (GP), Pre-emptive and Proactive (PP), and Contextual (CT)) and answer structures exploredin the study.108Figure 6.5: Study 2 mean UEQ+ ratings with 95% confidence intervals for Com-prehensibility - separated by the simplicity and ambiguity. Ratings are comparedacross the insight categories (Current Status and Value (CSV), Historical and Trend(HT), Combination and Comparison (CC), Goal and Performance (GP), Preemp-tive and Proactive (PP), and Contextual (CT)) and answer structures explored inthe study.Behaviour In parallel to Study 1, we once again noticed a general trend whereinFull Sentence resulted in the highest scores for Behaviour (naturalness and likabil-ity). However, Minimal Sentence and Keyword, which generally scored lower thanFull Sentence, did not vary from one another.Comprehensibility The level of Cronbach’s α for Comprehensibility was 0.49.Thus, we investigated the semantics (complicated - simple and ambiguous - unam-biguous) independently; see Figure 6.5. Noticeably, as in Study 1, it is the Minimalanswer structure which provides differences comparing across the two semanticsin the Contextual, Goal and Performance, Combination and Comparison, as wellas Historical and Trend insight categories. Each time the answer is rated as moresimple while being ambiguous.109(a) (b)Figure 6.6: Participant’s preference of answer structure for each insight category(Current Status and Value (CSV), Historical and Trend (HT), Combination andComparison (CC), Goal and Performance (GP), Preemptive and Proactive (PP),and Contextual (CT)) (a). Participant’s perceptions of voice assistants (b).Efficiency For Efficiency (a mean composite of “efficient” and “fast”) no differ-ences were found. The answer structure did not influence the levels of perceivedefficiency. No other effects were found.Preference and Attitudes Towards Voice AssistantsAs with Study 1, we again see the Full Sentence answer structure as being thepreferred majority for the remaining insight categories. Only for the Current Statusinsight category is this less pronounced. As complexity in the question increases,from that of a Current Status question, Full Sentence seems to be preferred mainlyfor its Quality and Behaviour. Finally, perceptions of VAs across both studies werecomparable.Open FeedbackAcross 19 unique comments the same three topics arose from Study 1. First,three participants expressed an interest in using a VA for personal health data ex-ploration. However, two participants shared that they prefer to perform visual data110analysis. These comments are important; our aim is not to replace visual dataanalysis. Instead, our work aims to diversify approaches to explore personal healthdata. Second, nine study procedure comments highlighted that the study went well.One participant mentioned the VA could slow down its answers while only threeparticipants expressed the VA was sensitive which caused some interpretation is-sues throughout (sensitivity was expressed as being due to background noise, alearned accent, and an illness influencing speech). Finally, only one participantcommented that they preferred concise and clear answers so as minimize the timetaken for the interaction.6.6 Discussion6.6.1 Implications for the Design of VA Interactions for PersonalHealth Data QueriesComparisons with General VA InteractionsOur results indicated that users preferred Full Sentence answers for their per-sonal health data queries. This runs counter to prior work, which has suggested thatMinimal and Keyword responses are ranked positively, and sometimes preferred,for common VA tasks (i.e., knowledge queries, home automation, reminders, cal-endar queries) [71]. We believe this discrepancy arises due to the brevity of Min-imal and Keyword responses which fall short in conveying the level of compre-hension required for many personal health data queries. For example, if a personasks ”What is on my calendar tomorrow?”, the VA could respond using a Minimalanswer structure, stating ”Lunch with Tyler, 1PM. Games with Danica, 7PM.” Thecontent in the answer itself provides a connotation of calendar events and does notproduce an answer that is ambiguous.In contrast, if a person asks ”What is my average daily step count in the lastweek?”, providing a Minimal answer, such as ”10,320 steps”, does little to conveythat the question was properly understood. Many other possibilities exist for asimilar answer (e.g., average daily step count in the last month or current stepcount.) Such ambiguity has been previously noted as a barrier in using personal111health data for clinical purposes [196], and now appears to be a common barrierfor end-users exploring their own personal health data.These findings highlight the importance of tailoring answer structures to spe-cific questions, both general and health-related. Furthermore, contextual informa-tion plays a key role in VA personal health data exploration, where users compre-hend the information provided holistically, rather than focusing solely on singlenumerical or categorical values. As such, design considerations for VA personalhealth data interactions should prioritize confirmation and inclusion of key aspectsof the data.Efficient Full Sentence AnswersDespite being longer with respect to response time, Full Sentences were per-ceived as equally efficient as Minimal and Keyword answers. We contemplateseveral reasons for this observation. First, contextual information provided withina Full Sentence may contribute to a more comprehensive understanding of the an-swer, thereby reducing the need for follow-up questions or clarification. This canultimately enhance efficiency, even if individual responses take longer. Second,the context in which the answer is given may influence its perceived efficiency.In distracting environments, concise responses may be preferred, while in quietersettings, more detailed answers may be deemed appropriate. Thus, the thresholdof an efficient answer may vary depending on the situational context, allowing forflexibility in response length without compromising perceived efficiency.Not only do our findings indicate that Full Sentence is perceived to be efficient,but they suggest that there may be room to augment Full Sentence answer contentwithout sacrificing perceived efficiency. This is due to the fact that we observedthat participants feel Full Sentence answers were equally as fast as Keyword andMinimal answers. Looking ahead, leveraging the capabilities of Full Sentenceanswers could allow for serendipitous information, akin to visual data exploration.For example, if a person asks “What is my average daily step count in the lastweek?”, the VA could provide a Full Sentence answer stating, “Your average dailystep count in the last week is 10,320 steps. This is higher than the previous week,112keep it up!”Moreover, future investigations could aim to enhance the depth of informationconveyed, the number of data points included, and/or the influence of certain an-swers to enrich user interactions with VAs. For example, if a person asks ”Is myheart rate lower in the morning, afternoon, or evening”, the VA could respond with”Your heart rate is the lowest in the evening, and is roughly 15 beats per minutelower than other times of day.” Each of these areas of exploration should explorethe context in which VA interactions take place (i.e., at home or while on a walk).Future work in these areas can build from the results found in our work whilethen aiming to provide more comprehensive guidelines for personal health dataquestions and answers, allowing for VA interactions that suit people’s needs andpreferences and could provide greater influence.6.6.2 Human Emulation and Unwavering PerceptionsOur results highlighted unwavering and confident perceptions within both ofour studies for the use of VAs for personal health data question and answering.This can be seen in the highly similar responses captured in our VA perceptionsurvey questions (see Figure 6.3b and Figure 6.6b, and follow the results of previ-ous work exploring general VA use [71]. As seen from this reported data, strikinga balance between technicality and efficiency, while providing answers that em-ulate full sentences (and therefore human likeness) is key. Importantly, however,we must be cognizant that VA interactions should remain to invoke as little socialinteraction as possible (e.g., the VA should not emulate a fitness coach). Notably,VA responses which emulate human behaviour, in part conveyed through Minimaland preferred Full Sentence answer structures, have been shown to raise people’sexpectations of the VA [115].While expectation and capability may be a concern in the earlier life cycles ofVAs, rapid improvements to VA performance will likely mitigate these concernsover time. Of more interest, is that human-like responses can lead to incorrect andinappropriate use of a VA [160]. Furthermore, trust and privacy become concernswhen the VA is seen as increasingly human-like [27, 119], a concern that is ampli-113fied with respect to personal data [191, 207] over that of general knowledge (e.g.,the weather or population of the USA). Therefore, we encourage designers of VAtechnologies to pay close attention to the balance required to accommodate thesepreferences.6.6.3 Comparisons With Commercial Personal Health Data Questionand AnsweringTo better situate this chapter’s findings we once more explore current capabil-ities with respect to VA personal health data question and answering. As such, weagain asked Siri a range of personal health data questions. As functionality is stilllimited (i.e., Siri can not answer many of the questions within our dataset and thisour study), we explored questions, and slight variations of questions promoted byApple10. Notably, the use of Siri for the exploration of personal health data is cou-pled with a display, rather than through a standalone device (e.g., in a standalonesmart speaker and as seen in our study). As such, we provide this information fordiscussion purposes only.10https://tinyurl.com/siriphdqa114Table 6.3: Recorded personal health data question and answers using Siri on an iPhone 14 Pro running iOS 17.3.Question Vocal ResponseSupplementaryInformation DisplayedWhat is my current step count? [Step Count] stepsStepsTodayWhat’s my heart rate? As of February 2, 2024 11:48 AM, it was 77 BPMHeart Rate[Month] [Day] [Year] [Time]How far did I walk yesterday? [Distance] kmWalking + Running DistanceYesterdayHow far have I walked thisweek?[Distance] km[Distance] km Daily AverageWalking + Running Distance[Month] [Day] - [Month][Day] [Year]What is my move ring at?You’ve burned [Current Calorie Burn] out of your [Calorie BurnGoal] calorie goalMove RingToday [Time]115Our study findings reveal preferred differences compared to how Siri answersquestions. While Siri predominantly uses Minimal vocal responses and Keywordinformation displayed on screen, our studies highlight a preference for Full Sen-tence answers. Notably, Siri rarely employs Full Sentences, except for activity ringdata, where multiple data points are conveyed, and when there are contextual de-viations from the question (see Table 6.3, ”What’s my heart rate?” - heart rate datawas not current and Siri responded with the last sample recorded). This divergencefrom our study findings in VA practice highlights varied strategies in voice assistantdesign, with Siri prioritizing brevity and visual support. However, the presence oruse of a screen may not always be optimal (as in the example shared in the Intro-duction where the focus should remain on the hiking environment). Such insightand discussion sheds light on the diverse approaches that can be adopted by voiceassistants in managing personal health data and more importantly underscores theimportance of understanding user preferences, expectations, and contexts.6.6.4 UEQ+ Semantic DifferentialsIn our study, despite using an adapted version of the the UEQ+ survey to mea-sure VA user experience [100], we observed conflicting semantic differentials forEfficiency and Comprehensibility. The lack of correlation found suggests that thesemantic differentials may not be effectively capturing the same intended user ex-perience factor. These findings highlight the need for future work to refine theUEQ+ scale for VA interactions, aligning semantic differentials more closely withuser experiences. Specifically, better assessing combinations of the semantic dif-ferentials while also exploring the potential for new overall user experience factorscould enhance the scale’s utility and effectiveness.While it can be argued that two items do not need to directly relate to provide aworthwhile assessment, as we do in this work, the divergent performance of thesefactors raises some concerns. For example, a response may be both simple andambiguous, resulting in a lower overall comprehensibility. However, it is better toisolate factors that measure the same element of user experience for a more com-prehensive and nuanced understanding. By doing so, practitioners can pinpoint116specific areas for improvement and create more targeted solutions. This approachnot only enhances the accuracy of evaluations but also leads to more effective andpreferred interactions, ultimately improving overall user satisfaction and interac-tion quality.6.6.5 LimitationsOur study has three main limitations. First, its online nature restricts the gen-eralizability of findings to real-world VA interactions, which can be potentiallyinfluenced by contextual factors and day-to-day use. Such examples include ask-ing a personal health data question during a walk or conversely while sitting athome relaxing. Thus, future research should extend our findings through in laband real-world settings for enhanced validity and reliability. Second, our partici-pant pool consisted of individuals familiar with VAs and personal health data (i.e.,potential sampling bias) While we are confident that focusing initially on this de-mographic provides insights into general results, we acknowledge it may not fullyrepresent the broader population for whom VAs could be used. To address this,future research could include participants with varying levels of VA familiarity andincreasingly diverse demographic backgrounds (e.g., older adults). Third, whileStudy 2 focused on insight categories, only Value-based responses were utilized.While this aligned with the study’s goal, it does leave other combinations of in-sight category and response type to still be evaluated which in turn could provideincreasingly fine-tuned guidelines for VA answers. Furthermore, while our studyoffered heart rate and step count as data types of choice throughout each study,to engage participants within the study, we recognize that questions pertaining tospecific data types could result in different desired answers structures. Future workcould perform a comparative study across the many data types captured withinone’s personal health data. Our studies offer initial insights into answer structuresfor VA interactions involving personal health data questions. By exploring diverseresponse types and insight categories, notably applicable to any data type, we laythe groundwork for designing and developing VA interactions involving personalhealth data.1176.7 SummaryThrough the use of a custom-built browser-based pseudo-voice assistant (VA),this chapter investigates differing answer structures in response to personal healthdata queries. Combined with knowledge gained from the previous chapters, andknown guidelines for smartwatch visual output design, we now have the tools nec-essary to begin building a smartwatch application which can appropriately handlea range of personal health data queries desired by people.Specifically in this chapter, two user studies involving a total of 82 participantswere conducted, during which participants interacted with our VA, posing ques-tions and ranking their experiences and preferences of three distinct answer struc-tures: Minimal, Keyword, and Full Sentence. We provide empirical findings thatreveal a notable preference for full sentence answers, which consistently demon-strated higher quality, behavior, comprehensibility, and efficiency across variousresponse types (Open, Range, Binary, and Value) and personal health insight cate-gories (Contextual, Preemptive and Proactive, Goal and Performance, Combinationand Comparison, Historical and Trend, and Current Status). These results come at acontrast to previous work which explore answer structures for general VA use. Ourresults suggest that full sentence answers offer less ambiguity, and despite theirlonger response time, full sentence answers were perceived as equally efficient.Along with other findings, such as a desire for VAs to be efficient and technicalrather than social entities (e.g., as a fitness coach), we provide design implicationsin line with these results that offer insight into future VA systems handling personalhealth data queries.118Chapter 7: Enabling In-SituDatabiting on the SmartwatchLeveraging Touch and SpeechIn this chapter, we present our most focused approach to databitingon the smartwatch. Building on the knowledge gained from previ-ous chapters, we now explore the use of databiting for in-situ explo-ration of tracked workout data through a custom-built smartwatch ap-plication, DataWatch. DataWatch specifically targets tracked work-out data, which holds the greatest potential for in-situ exploration, asseen in Chapter 4, and was the most frequently queried type of data,as noted in Chapter 5. Work presented in this chapter has not beensubmitted to a venue, however, we do anticipate submitting this workin future. Building of the DataWatch application, study design, andanalysis are entirely my own work. However, discussions surroundingthe work put forward in Chapter 3 and Chapter 4 influenced aspectsof the DataWatch application and study methodology.7.1 IntroductionWe present DataWatch, a novel and custom-built smartwatch workout track-ing application designed to facilitate databiting—lightweight and transient data119exploration—specifically for tracked fitness data while in-situ. We designed anddeveloped DataWatch (Figure 7.1) to enable users to interact with their fitness datadirectly on their smartwatch, supporting three types of exploration: 1) Single Value– displaying aggregated textual values, 2) Browse – showing charted values frompast workouts, and 3) Compare – comparing past workout data with current met-rics. Users can explore data during three workout phases: 1) Before, 2) During,and 3) After a workout. Additionally, DataWatch supports two filtering methods:1) Temporal and 2) Activity filtering.Smartwatches, due to their limited screen space, cannot or do not accommo-date touch-based controls and options alongside their glanceable visualizations.For instance, the current Apple Watch applications restrict users to viewing databy predefined time segments, such as the current day or week. A lack of screenspace makes it impractical to navigate to separate pages to adjust widgets, options,and query types. For example, time-based interactions, such as entering specificdates, times, and ranges, are tedious on smartwatches, leading most current healthapplications to limit exploration to static glanceable visualizations.Yet, in certain everyday situations, such as when people are in-situ, having theflexibility for lightweight and transient data exploration directly on the smartwatchcan greatly benefit individuals. As such, and to address the challenges of touchnoted above, DataWatch draws inspiration from previous research advocating thebenefits of multimodal interaction [98]. Specifically, we incorporate speech as aprimary input modality to overcome the limitations of touch-based controls pre-viously mentioned. Speech-based interaction takes little screen space, is flexible,and allows users to express their intent easily [8, 42], making it an ideal solutionfor smartwatches.To better understand how such databiting capability is perceived and utilized,we conducted an exploratory, observational, study with 12 individuals who trackworkouts regularly and are interested in their personal fitness data. Participants inour study downloaded DataWatch to their own Apple Watch, and used DataWatchto track their workouts over the period of one week, while also databiting whendesired.Our contributions are three-fold:120Figure 7.1: DataWatch supports in-situ exploration of past tracked workout datathrough multi-modal interaction. People can track and control their workouts whileviewing their currently collected metrics , as is typical with current smartwatchhealth applications. Different to these applications, people can then at any timelong-press elements on screen (elements which can be long-pressed, and thusqueried, are denoted with a blue dot) to initiate a spoken query regarding the in-teracted element . Upon providing a query, DataWatch processes the query toshow the resulting output to the person . In this Figure, a databiting instanceinvolving comparative exploration and temporal filtering is queried during a per-son’s workout. Please note, screens which appear beside and below the smartwatchare screens which can be swiped/scrolled to, from the current displayed screen.121C1: The design and implementation of DataWatch which represents anovel approach to smartwatch data exploration. To our knowledge,DataWatch is the first app to leverage touch and speech to enableexploration of past tracked workout data directly on the smartwatch.It facilitates databiting—lightweight and transient data exploration—allowing users to interact with their personal fitness data while in-situ.C2: An empirical exploratory study was conducted with 12 AppleWatch users using DataWatch over the course of one week. Throughquantitative and qualitative analysis, we provide insights into how peo-ple explore their past collected workout data, their motivations forsuch exploration, the perceived actionable insights and usefulness, andtheir general reactions to DataWatch.C3: Reflecting on our observations, and participants’ feedback, wediscuss implications of DataWatch’s design and provide opportunitiesfor further development of databiting. Through this discussion, ouraim is to continue to better support personal data exploration on thesmartwatch in the future.7.2 DataWatchTo enable flexible and personalized in-situ fitness data exploration on the smart-watch, we designed and developed DataWatch. DataWatch is an Apple Watchworkout application that leverages speech as well as combined touch and speechinteractions for lightweight and transient exploration of previously collected fitnessdata. In this section, we describe the design rationales for DataWatch, highlight theuser interface and interactions, and discuss technical aspects of DataWatch’s im-plementation.7.2.1 Design RationalesDR1: Utilize Intuitive Layouts, Interactions, and Visualizations to Support LayPersons DataWatch targets everyday smartwatch users who track workouts and122want to further explore their fitness data. As such, we recognize that users may nothave expertise in data visualization or analytics. Therefore, DataWatch prioritizesintuitive design, interactions, and visualizations.DataWatch is designed to visually and interactively mirror Apple’s defaultWorkouts application. This familiarity can greatly reduce any effort and timeneeded when learning to use a new application. Thus, people can focus their at-tention on the novel exploration capabilities, and not the interface itself. As such,DataWatch employs only one additional interaction above and beyond that of Ap-ple’s Workouts application; a long-press (i.e., touch and hold), which can be doneon a variety of on-screen elements denoted by a blue dot (see Figure 7.1 ), toinitiate a query. This then brings forward a keyboard/microphone for which userscan use to provide a query (see Figure 7.1 ).DataWatch utilizes line charts, which are common and intuitive for lay users[136], and text-based representations of data. Furthermore, text is used to highlightkey insights, when queried, within a line chart. This can further simplify compre-hension of data and provide immediate insight from the requested data. For exam-ple, Figure 7.1 highlights the use of both a line chart and textual representationsof data as seen in DataWatch.DR2: Leverage the Complementary Nature of Speech and Touch Interactionson the Smartwatch The limited screen real estate of a smartwatch limits both thevisual components that can be displayed as well as the input mechanisms afforded.To address the limited screen real estate and input limitations, we leverage naturaland familiar modalities: speech as well as touch and speech. Touch interactionsprovide familiar and direct manipulation of on-screen elements [98], such as themetrics displayed during a workout (see Figure 7.2 ), while speech offers a moreexpressive way to convey intent and a higher freedom of expression [8, 42, 182].Our goal was to utilize these complementary modalities for lightweight andtransient exploration, rather than provide complete parity between touch and speechinteractions. This complementing of input modalities is best exemplified in Fig-ure 7.2, where the touched metric is used as input for the processing of the querysubsequently provided. Utilizing a combined touch and speech approach recog-123nizes that touch alone would not suffice for databiting on a smartwatch, while alsokeeping databiting as close to the data of interest and without needing additionalscreen real estate for widgets, options, and exploratory controls. We further detailhow we utilize both touch and speech in Sections 7.3.2 and 7.3.3 below.Figure 7.2: DataWatch supports in-situ exploration before starting a workout. Onthe homepage, scrolling through the many workouts available for tracking, peoplecan long-press on a workout type of interest to initiate a query . In this Figure,a databiting instance using an activity filter and browse exploration capabilities isdemonstrated . Please note, the list of workouts available extends beyond what isshown below the smartwatch on the left, and is reduced here for space preservation.DR3: Support Comprehensive In-Situ Exploration Reflecting on personal datausing a smartwatch can occur increasingly close to the action, for which the re-flection is related, to benefit on-the-fly decisions [3, 69, 107]. The term in-situexploration is often thought to only encompass exploration of data that is currentlybeing tracked while performing the activity tracking that data. However, we take a124broader stance, considering in-situ exploration to include any interaction with datathat occurs in the context of and environment where the activity takes place. Assuch, DataWatch supports exploration before, during, and after a tracked workout,which allows for a comprehensive in-situ exploratory experience.Before starting a workout, people can benefit from insights that help themprepare and set goals (i.e., preparation-for-action). This preparation phase is keyfor setting realistic and achievable targets, ensuring users are well-informed. Fig-ure 7.2 highlights this before activity querying of data.During a workout, real-time data exploration can provide actionable insights,insights which can potentially be influential to the activity being performed (i.e.,reflection-in-action [149]). Displayed metrics on screen update continually through-out the workout to show currently captured and calculated values associated withthe workout. These can provide actionable information if people know how thecurrent value displayed relates to a previously defined goal or historical data. How-ever, if they do not, databiting can provide actionable insights previously unavail-able to the person. Figure 7.1 highlights an example of during workout queryingof data.After completing a workout, reflection on the collected data (i.e., reflection-on-action [149]) allows users to understand their performance, identify areas forimprovement, and track progress over time. This reflective phase is essential forlonger-term motivation and continuous improvement. As such, a summary screenshows the aggregate metrics captured within the workout just completed and Fig-ure 7.3 highlights after workout querying of data.DR4: Enable Flexible Temporal Filtering to Support Visually-Constrained Ex-ploration In addition to traditional temporal filtering (e.g., “The last six months”,“January 1 to February 1”, “Last month”, “Since June 1st”, etc.), supporting activity-based filtering allows people to explore and temporally filter their data implicitly.Temporal filtering, while useful, can be limiting when users cannot remember theexact dates of their past activities and when visuals on-screen do not highlight thesedates for further filtering. By incorporating activity-based filtering, users can ex-plore their data based on the last number of activities performed. This enhances125Figure 7.3: DataWatch supports in-situ exploration after completing a workout.On the summary screen, shown post workout, people can long-press on a metric ofinterest to initiate a query . In this Figure, a databiting instance using a temporalfilter and single value exploration capabilities is demonstrated . Please note,content displayed around the smartwatch on the left, can be scrolled to and from.accessibility and ease of use, ensuring that valuable data is not overlooked sim-ply because the user cannot pinpoint when it was collected. Figures 7.2 and 7.3highlight activity-based filtering, while Figure 7.1 highlights a temporal filter.7.2.2 User Interface and Interaction DesignBase Functionality At its core, DataWatch is a workout tracking application.As such, DataWatch allows participants to choose an open-goal workout type totrack (e.g., indoor walk, outdoor cycle, hike, swim, elliptical, yoga, etc.), see Fig-ure 7.2 (left). Upon starting a workout, live metrics are captured and shown onscreen across a variety of workout-specific metrics pages which can be viewed by126scrolling up/down, see Figure 7.1 . Controls, including pause/resume, end, anda microphone for querying are available by swiping left from any of the metricspages, while music playback controls are available by swiping to the right. Uponcompletion of a workout, the workout is saved, and a summary screen providesaggregated collected metrics of the workout, see Figure 7.3 (left).Databiting DataWatch currently supports databiting of five captured metrics: 1)Duration, 2) Distance, 3) Pace, 4) Calories Burned, and 5) Heart Rate. Thesemetrics are not only common across most all workout types and in many mHealthapplications [97], but are also some of the most desired for exploration by smart-watch users according to our findings in Chapter 5.Of this data, people can conduct databiting using three types of exploration: 1)Single Value - provides an aggregated value textually represented on-screen; seeFigure 7.3 . People can access single value exploration through keywords suchas “what is”, “what was”, “how many”, etc., 2) Browse - provides a line chart of allqueried data as well as any supplemental aggregate information through text and acorresponding dashed line within the chart; see Figure 7.2 . People can accessBrowse exploration through keywords such as “show”, “explore”, “highlight”, etc.,and 3) Comparison - provides similar exploration to that of Browse, however, alsoincludes the currently captured metric; see Figure 7.1 . As a comparison chartis displayed, the currently captured metric is highlighted as a larger line mark andupdates live within the chart as the captured metric changes. People can accessComparison exploration through keywords such as “compare”, “combine”, etc.Query Input The use of each of touch and speech input modalities can be ac-cessed throughout varying parts of DataWatch. Speech alone can be initiated fromthe microphone button located in the controls screen during a workout, see Fig-ure 7.1 . To utilize touch and speech, a person can long-press any on any inter-active element denoted by a blue dot (i.e., an activity before a workout or a metricduring and/or after a workout, see Figures 7.1 , 7.2 , and 7.3 ). Importantly,DataWatch recognizes the element touched and incorporates this into the spokenquery; this can be seen in the example in Figure 7.1 through , where heart127Figure 7.4: DataWatch’s input screens are shown demonstrating live query input.By default the keyboard is brought forward (left), which can be switched to dic-tation capture (right) by selecting the microphone icon at the bottom right of thescreen . Input can either be cancelled or sent for processing through the controlsat the top of either input screen .rate is not mentioned within the query yet is understood as the data desired forexploration due to the initial touch interaction.Furthermore, DataWatch recognizes the specific workout type being interactedwith, currently tracked, or having just been tracked, when databiting occurs. Thisapplies before starting a workout when a query is initiated by long-pressing ona workout type, and during or after a specifically chosen workout. This workoutrecognition within a query is best exemplified in Figure 7.1 , where DataWatchshows data from only walking workouts within the time period provided. This levelof recognition allows people to focus on data pertinent to a particular workout typewhile ensuring that the databiting process is in-situally relevant.Upon initiating a query, a keyboard is overlaid on the currently visible screen.By default the keyboard is shown, allowing for a natural language query to be typedout, while the microphone button can also be tapped to initiate speech input; see128Figure 7.4 . When using either the keyboard or microphone, the query enteredby the participant is seen live as they provide input. When finished, a user canclick “Done” which sends the query for processing and resolution, or, if desired,the person can cancel the input to go back to the screen from which the queryinteraction was initiated; see Figure 7.4 .Feedback When DataWatch can process the query and is able to resolve the dataneeded, the exploration result is overlaid on top of the currently displayed activity,metric, control, or summary screen. To end the exploration, the person taps on“Close” (see Figures 7.1 , 7.2 , 7.3 ). Conversely, when DataWatch is unableto process or resolve the query, a contextual message is overlaid. After reading themessage the person can tap “OK” (see Figure 7.5) to go back to the screen fromwhich they initiated the input.Figure 7.5: When a provided query is either not valid or resolving the data was notsuccessful, a contextual message is displayed. Three potential messages are shownhere, while others exist depending on the issue faced.7.2.3 Interacting with DataWatchWe illustrate DataWatch’s interface and interactions through a storied usagescenario: Sam is starting their day with an indoor walk on their gym’s track whileusing DataWatch to track their activity. We note, this scenario does not encompassall queries possible.129Before Starting the Walk Before starting his walk, Sam uses DataWatch to getmotivated and prepare-for-action (as illustrated in Figure 7.2). He opens DataWatchand rather than tapping on Indoor Walk to get started, he first taps and holds on thedesired workout. He then states “Show me my longest walk in the last 10 walks.”DataWatch quickly handles his command and displays that his longest walk fromhis last 10 was 24 minutes. Feeling good today, Sam is motivated by this data tosurpass his previous record and wants to aim for 30 minutes.During the Walk A few minutes into the walk, Sam is curious about his heartrate’s performance during a specific period (as illustrated in Figure 7.1). First, henotices his current heart rate is around 99 BPM. Curious about how this comparesto the latter part of last year when he was more regularly at the gym, Sam tapsand holds the heart rate metric and simply says, “Compare to my average fromJuly 1st to Dec 31st.” The resulting chart shows his current heart rate is roughlyalongside his average but on a slight downward trend. Realizing he isn’t pushinghimself as much as he used to, and could, Sam decides to increase his effort toreach the higher values he saw earlier in the explored period. Throughout his walk,he keeps an eye on his BPM, ensuring he pushes himself to his target value. Thisreflection-in-action helps Sam adjust his effort to meet his new-found goal.After Ending the Walk After completing his walk, Sam becomes curious abouthis recent performance while scrolling through his summarized metrics. Sam seeshis distance metric and decides to tap and hold to ask a question (as illustrated inFigure 7.3). Unsure of exactly when he has been to the gym in the last week or so,Sam asks DataWatch, “How far have I walked in the last five walks?” The responseindicates he has walked 7.2 km in total. This was lower than Sam expected; he hadhoped it would be higher. Taking this reflection-on-action as motivation, he sets anew goal for his next walk: to walk 2.8 km, which would bring his total to 10 kmover the last six walks. This quick post-workout reflection helps Sam set futuretargets and stay motivated to further improve his fitness.1307.2.4 ImplementationDataWatch leverages the Apple Watch and watchOS platform, specifically de-signed for Series 4 watches onwards running watchOS 9.0 or later. Thus, DataWatchcan run on 10 generations of Apple Watches dating back to 2018; this is in fact onlylimited by Apple’s own watchOS restrictions which inhibit older watches from in-stalling watchOS 9.0, required for some of the framework access utilized withinDataWatch. DataWatch incorporates two parts: a primary watch application, whichhandles workout tracking, user queries, and interaction, and a companion iPhoneapplication, which handles the query processing and resolving in the background.To ensure compatibility with accompanying iPhones, iOS 17.0 or above is required.Both applications were built entirely in the Swift programming language.Apple’s HealthKit11 framework is used to read and write workout data withinDataWatch while maintaining user privacy and control. Prioritizing user privacy,DataWatch first seeks permission to read and write workout data. Furthermore,people have the ability to later rescind or allow permission of this data if desired.Since all personal tracked data is stored on the user’s iPhone, the current ver-sion of DataWatch requires the iPhone to be unlocked to resolve a query. This isan Apple-imposed security measure to protect personal health data. Only the mostrecent week’s data is stored on the Apple Watch, which is insufficient for mostqueries. Therefore, unlocking the iPhone grants access to the necessary data. Notethat the companion iPhone app operates in the background and does not need to beopened for DataWatch to function.To enable natural language interaction, a custom Natural Language Processor(NLP) and Resolver (NLR) were built. When a user interacts with DataWatch, theirquery, and additional contextual information about the workout and interaction, aresecurely transferred to the iPhone via Watch Connectivity12. Once transferred tothe iPhone, the NLP parses the query using part-of-speech tagging, identifying pa-rameters such as the desired data source, query intent (through interrogatives used),data filters (temporal and activity), and aggregations. Chrono, a natural language11https://developer.apple.com/documentation/healthkit12https://developer.apple.com/documentation/watchconnectivity131time-period parsing library originally built in JavaScript13 and modified for Swift14was further custom-modified for use within DataWatch to handle temporal filteringof data. Following the NLP’s completion, resulting in a set of found tags, an NLRanalyzes the extracted tags alongside the current watch screen information and userinteraction details to then fully resolve the interaction, lookup the necessary data,and perform any needed calculations. Finally, the results are shared back to thewatch, again through Watch Connectivity, for appropriate display.7.3 Study7.3.1 RecruitmentWe recruited participants from Reddit and through advertised posters on ourUniversity’s two campuses. We advertised our study across subreddits relating topersonal health, apple watch use, as well as a number of general city subredditsacross Canada. Our inclusion criteria were those who (1) were aged 18 years orolder; (2) own an Apple Watch Series 4 or newer with watchOS 9.0 or higherinstalled, and have a paired iPhone; (3) have access to Zoom and a stable internetconnection; (4) are native English speakers; (5) currently live in Canada; (6) usethe Apple Workout application for at least one of the following workout tracking(indoor walk, outdoor walk, indoor run, outdoor run, elliptical, hike, indoor cycle,stair stepper, rower, yoga, functional strength training, core training, Pilates, dance,tai chi, cool down); and (7) have been regularly collecting personal health data ontheir Apple Watch for at least three months.In appreciation for their time and effort, participants were offered up to $20CAD. The amount a participant received was not tied to the number of responses,but rather their overall participation. We provided the equivalent of $5 CAD forattending the introductory session, and another equivalent of $5 CAD for attendingthe final interview. During the week long data collection, we added an additionalequivalent of $10 CAD if the participant used DataWatch to track a workout at13https://github.com/wanasit/chrono14https://github.com/quire-io/SwiftyChrono132least once. Notably, this was not tied to databiting instances, rather only generalapp use. We provided compensation after the final interview, or upon withdrawal,in the form of an electronic Amazon.ca gift card.7.3.2 Study ProcedureOur study included three stages, similar to that of the study conducted in Chap-ter 4: an introduction and tutorial session, a seven-day in-the-wild data collectionperiod, and a final interview. The procedure and study materials were iterated uponduring two pilot participants with people who were recruited in the same means asour participants, thus meeting our study’s inclusion criteria. All participants pro-vided consent at the start of the study, in accordance to our approved ethics protocolreviewed by our institution’s ethics review board.Introduction and Tutorial Session.To start the study, the participant joined a Zoom meeting, ˜30 minutes long,where we introduced and acknowledged their interest and participation in our study.Participants were encouraged and asked to interject with any comments and/orquestions during the meeting. The researcher shared presentation slides (pleaserefer to the supplemental material) via the screen sharing functionality. The goalof the project was expressed to the participant along with other important remarksbefore they completed a demographic survey. Then, the researcher guided theparticipant through the installation and setup process of the application on the par-ticipant’s own Apple Watch to be used throughout the data collection stage of thestudy. The researcher then gave participants a walk-through of the applicationthrough a live demo from the researcher’s Apple Watch which was mirrored to theresearcher’s iPhone and the iPhone’s screen shared on Zoom. Upon completion ofthe walk-through, the researcher reiterated the general functionality of DataWatchthrough slides, also highlighting queries that were not possible. Finally, partici-pants tried a query on their Apple Watch to ensure everything was working andany technical issues related to interaction, data access, etc. were handled.133Data Collection.Participants used DataWatch over the course of the next seven days to tracktheir workouts as usual, while being encouraged to use the databiting capabilitiesas desired. We did not require a minimum number of databiting instances through-out the study, so as to not elicit forced uses and queries from participants. Dueto the potential for participants to be active (e.g., during a run) when a desireddatabiting instance arose, we instructed them to only provide a response throughthe application when it was safe for them to do so.Final Interview.After the final day of data collection, a Zoom meeting was held where a re-searcher conducted a semi-structured interview with each participant. The meetingwas audio recorded and later transcribed. The goals of the interview were: (1) togarner general reactions and experiences of using DataWatch, (2) to explore per-ceptions towards the use of speech as an input modality for personal health dataqueries, (3) to discuss insights gained as a result of databiting, and (4) better un-derstand motivations behind databiting in-situ. To aid in recollection, a report ofeach participant’s queries were shown to them via Zoom’s screen sharing function-ality. Finally, the researcher answered any remaining questions from a participant,thanked them, and provided the compensation. Each interview took ˜30 minutes.7.4 ResultsTable 7.1 summarizes the demographic, smartwatch usage, and health data col-lection, workout tracking details, as well as databiting counts of our study partic-ipants. Of our total participants, one was removed from analysis as they did notuse DataWatch to track a single workout throughout the week of data collection,nor use it for databiting even prior to a workout. Of the remaining 12 participants,participants were aged from 20 to 44 (M = 27.9) and held a range of occupations.At the time of conducting the study, participants had collected personal health datafor an average of 46.2 months (SD = 41.6 months) and had used a smartwatch134for an average of 36.0 months (SD = 33.0 months) while wearing the smartwatchfor an average of 6.6 days per week (SD = 0.8 days). Participants largely trackedwalks, runs, hikes, cycles, and functional strength training workouts, while includ-ing other workouts such as swimming, soccer, elliptical, Pilates, dance, yoga, core,and high intensity interval training.In the following subsections, we report on quantitative results from the use ofDataWatch, as well as emergent themes found from our qualitative analysis of theinterview scripts.135Table 7.1: Summary of demographic information, health data collection, smartwatch usage experience, and number oftracked workouts and responses reported from our study participants.Alias Age Gender OccupationHealth DataCollectionSmartwatchUsage (Total/ Days perWeek)Number ofWorkoutsTrackedDatabitingInteractionsP1 22 M Student 0y 8m 0y 8m / 5 2 0P2 21 M Student 7y 4m 3y 9m / 7 3 13P3 38 M Consultant 6y 4m 6y 4m / 7 2 13P4 44 M Professor 13y 0m 10y 0m / 7 7 4P5 23 M IT Administrator 0y 9m 0y 8m / 5 9 11P6 38 F Student 2y 0m 1y 0m / 7 9 28P7 39 M Senior Solution Architect 4y 1m 4y 1m / 7 3 4P8 22 M Student 0y 4m 0y 4m / 7 11 8P9 24 FCommunications & ProgramCoordinator4y 0m 4y 0m / 7 7 8P10 20 M Student 1y 9m 1y 9m / 7 2 17P11 23 F Research Assistant 2y 6m 1y 0m / 6 2 7P12 21 F Sales Associate 3y 5m 2y 5m / 7 1 51367.4.1 Interaction UsagesWithin our study, 58 total workouts were tracked using DataWatch (Avg = 4.8,Min = 1, Max = 11, SD = 3.4), including walks, runs, cycles, gym sessions, coretraining, and rowing. In and around these workouts, a total of 120 databiting inter-action attempts were made (Avg = 9.8, Min = 0, Max = 28, SD = 7.1). Figures 7.6,7.7, and 7.8 highlight all databiting interactions for each participant. Notably, thisdistribution of databiting interactions follows very closely with the results found inChapter 4, see Table 4.3 the first row corresponding to Physical Activity.Among the 120 databiting interactions, no critical errors occurred (i.e., a mes-sage or result was always provided). However, 18 (15.0%) databiting interactionsfailed due to uncertainty of the term length/long and much (i.e., “How long was mylast workout” - DataWatch requested that a specific data type be included as long,length, and much have connotations of both distance and time metrics), 12 (10.0%)failed due to an unresolved error related to queries that included an edge case. Thisedge case filtered out the most recent workout through activity-based filtering, suchas (P6 - “What’s my highest heart rate during the last exercise”), 14 (10.8%) weresimply unsupported queries (i.e., comparisons before a workout where no currentmetrics were available to compare, or using multiple aggregations in a query). Asa result, 76 databiting interaction attempts were successfully executed.Input, Processing, and Exploration Time Input took an average of 12.7 sec-onds (Min = 5.2, Max = 44.4). This includes the time from when the keyboardwas brought forward on the watch, to when the participant clicked done to send thequery for processing. Then, our processor and resolver worked quickly, processingand resolving the queries in an average of 0.3 seconds (Min = 0.01, Max = 8.5). Fi-nally, participants viewed the resulting visualization for an average of 7.4 seconds(Min = 1.4, Max = 30.4). Combining times for input, processing and resolving, aswell as viewing results, we found a total average exploration time of 21.7 seconds(Min = 7.4, Max = 58.8).137Figure 7.6: Instances of databiting (represented by colored blocks) are highlightedwithin workout timelines and organized by participants (P1-P5) and days of use(each row). Note that the spacing between icons and the physical length of theworkouts (beginning to end) do not correspond to any specific results or times.However, the size of the colored databiting blocks reflects the total explorationtime for each instance.138Figure 7.7: Instances of databiting (represented by colored blocks) are highlightedwithin workout timelines and organized by participants (P6-P10) and days of use(each row). Note that the spacing between icons and the physical length of theworkouts (beginning to end) do not correspond to any specific results or times.However, the size of the colored databiting blocks reflects the total explorationtime for each instance.139Figure 7.8: Instances of databiting (represented by colored blocks) are highlightedwithin workout timelines and organized by participants (P11-P12) and days of use(each row). Note that the spacing between icons and the physical length of theworkouts (beginning to end) do not correspond to any specific results or times.However, the size of the colored databiting blocks reflects the total explorationtime for each instance.Exploration Instances To trigger input on the watch, participants almost exclu-sively used the activities and metrics, directly interacting with the data they wereinterested in exploring. However, one databiting interaction was initiated usingthe microphone button on the controls screen during a workout (P5 - “Tell me myaverage pace from the last five runs”). This occurred early on in the participant’sworkout, likely when a pace was not yet displayed on the metrics screen giventhe limited captured metrics during the started workout (see Figure 7.6, P5’s firstworkout on the first day).Participants largely preferred the Value exploration type, with 89 (74.2%) databit-ing interactions requesting this type of visualization. Value was followed by Com-pare which was used for 16 (13.3%) databiting interactions, while Browse wasused for 15 (12.5%). While a clear preference of exploration type was seen, partic-ipants conducted databiting instances almost equally Before (44, 36.7%), During(34, 28.3%), and After (42, 35.0%). While the majority of databiting interactionswere done in-situ, some participants (P2, P5, P10, P12) also used DataWatch toexplore data away from a workout (25 instances of the 120 — highlighted above inthe Before counts as the list of workouts screen was interacted with, 20.8%). Noclear distinction between the exploration type and time of exploration was found,however Browse exploration was slightly more often conducted before and after a140workout.Finally, we saw almost half of databiting interactions use activity filtering (53,44.2%). Temporal filtering was used in 46 (38.3%) of databiting interactions, whilethe remaining required all data collected to be used as no filter was provided. Thereagain seemed to be no pattern evident for the use of a specific filter for varyingexploration types or during varying exploration times.7.4.2 Motivation for ExplorationTwo major themes arose which suggested reasons for databiting in-situ on thesmartwatch: verification of performance (P3-7, P9, P10) and curiosity (P3-4, P6-8). Participants used databiting to quickly check their workout metrics before andafter an exercise to first understand their performance and to then later ascertaintheir progress towards a goal. For example, P8 mentioned “I mean I’d see that[time walked] and I’d be like, oh, that’s interesting, 9 minutes. Usually I feellike it’s 10. Let me just check how does this compare with my other times?...itis faster.” While during a workout, databiting interactions pertaining to heart ratewere often used as a gauge for effort and performance. Heart rate was seen as ametric that participants could control and “feel” (P2, P6-8) in-situ, more so, forexample, than calories burned or distance. To this effect P6 stated “I felt that forthis exercise...that my heart rate is very high in this exercise and I wanted to verifythat.” P8 then went as far to say “I only monitor [on the smartwatch] only when Ifeel that my heart rate is really going up.”Curiosity, piqued by the emergence of the databiting functionality, also moti-vated exploration on DataWatch for a handful of participants (P3-4, P6-8), even fordata types previously unexplored. P8 highlighted “Like my workout at the gym, Idon’t really focus too much on my heart rate, especially when I’m not doing car-dio. But I figured you know, since I have it there...I was surprised to see that it didget pretty high.” P7 added their experience with the distance metric, “On average,walk for two kilometres. That was a surprise. I never measure distance, I just walkuntil I feel that I don’t want to anymore.”1417.4.3 Speech Interaction for Personal Health Data QueriesAll participants, except two (P9, P11) agreed that speech interaction on thesmartwatch was an appropriate method for databiting interaction. It was seen as“comfortable” (P3), “easy” (P4), and a ‘‘very feasible option and especially giventhe small screen of the the watch” (P7).The most salient discomfort of speech interaction, noted by a range of par-ticipants (P1-6, P8, P10-12), was the physical act of raising the smartwatch toone’s mouth, which can feel awkward in the absence of Bluetooth headphones. P2summarized this by saying “So I did kind of feel a little bit self self-conscious oflike kind of talking my watch because I think it kind of looks kind of funny causepeople are like, why are you talking to watch? I would say I was cognizant...I def-initely was like in the back of my mind like, hey, I’m like talking to my watch rightnow. Just kind of looks funny and people might see me talking, but it didn’t reallymatter that much.” P6 further added that “I feel that I need to explain what I’mdoing.” However, we postulate, the increasing prevalence of voice assistant usagethrough headphones mitigates this issue in future. As voice interaction continuesto integrate seamlessly into daily routines, it not only enhances the practicality ofsmartwatches for health data exploration but also aligns with evolving user behav-ior patterns.A secondary theme that arose from two participants (P5 and P6) consideredworkouts as a group behaviour (e.g., walking with a partner) and how speech in-teractions with the watch can hinder the social context in which databiting maybe desired. For instance, P5 said “Obviously, like if I’m right next to someoneand they’re walking with me to go somewhere I might not want to do that just to,you know, be more engaged in that conversation.” This emphasizes the importanceof developing touch-only and more versatile multi-modal solutions for effectivedatabiting on the smartwatch.Otherwise, our discussions with participants highlighted minimal concerns aboutasking data-oriented personal health questions in public settings (P2-4, P6-10).Three reasons for this arose throughout our interviews. First, questions available toask in DataWatch were not deemed to be overly intimate. To this effect P2 shared “I142wasn’t really worried and I feel like personally the data wasn’t specific or personalenough to really warrant me caring.” While this was the case for the data availableto be explored in DataWatch, we recognize that other queries (e.g., queries regard-ing blood sugar could be viewed as more private than those about heart rate) mayprovide differing perceptions. Second, participants (P2, P7, P10) felt indifferent tothe presence of others, stating it is not other’s business nor would they likely carebeyond hearing them. P7 mentioned “It’s nobody’s business and if they’re curiousto listen you’re welcome to. But I’m sure they forget about it.” Finally, it was notedthat speech interaction was acceptable as long as the responses were not audiblybroadcasted. P8 added “I’m not out loud saying my whole health data, so I’m theonly one that sees it in the end.” While the question was perhaps not intimate, wemust recognize that the data in response to the databiting interaction remains as anintimate element.7.4.4 Benefits of DatabitingIn Chapter 3, we discuss potential benefits of databiting, including actionableinsight and motivation for further exploration. Here, we discuss each of these tobetter understand the benefits of databiting through DataWatch.Actionable InsightGiven the increased insight available through databiting, one envisioned ben-efit is that of actionable insight. While metrics on screen can provide this, it islimited, especially if a specific goal or value is not predetermined. By databiting,current metrics can be associated with past data, providing a more complete viewof trends and comparisons. Throughout, over half of our participants (P3, P5-9,P12) discussed instances where databiting in DataWatch provided actionable in-sight beyond what was currently achievable in the commodity workout trackingapplications that do not support databiting.Some participants found that databiting encouraged them to push or reservethemselves. For example, P3 highlighted how databiting caused them to push theirheart rate “What’s the average heart rate like?...it was lower than I expected, so143I thought, oh, I can push myself more. I’m like I should be able to get that up to130.” This was also the case for P6 and P8, as well as P9 who conversely realizedtheir peak heart rate was higher than desired and aimed to keep it within a lowerrange; P9 mentioned “I could show like the highest and I think it was like 182 orsomething, which I was never aware of that when I was working out. I was beingmore mindful and then I was focused more on my heart rate.” P8 also highlightedhow databiting influenced their pace and pushed them to try and go faster.Rather than pushing or reserving a value of a certain metric, participants alsofound actionable insight by using resulting answer to then encourage them to main-tain values. P5 mentioned “Am I exerting myself to the degree I wanted to? Iunderstand that I have a training plan set out, but in the moment you kind of justforget like how how much am I actually doing. While I was thinking through thatin my head, I could just ask that...I felt like I got the answer very quickly and I feltthat it was very helpful [to keep me on track].”For some, however, actionable insight was workout, and even current trackinggoal, dependent (P1, P2, and P10). P1, a participant that in fact did not conducta single databiting interaction, discussed specific queries that would be beneficialat different instances in their training, but were not currently relevant. SpecificallyP1 discussed “I think seeing my maximum would be pretty helpful, yeah, becausethat way I can kind of get a sense of like what I’ve done previously...what I’m kindof capable of, and like, am I able to do more today in this particular day. I thinkfor me it can definitely like motivate me...I’m always like oh let’s maybe push itanother couple like points.”P2 discussed how databiting interactions during gym sessions were not as ac-tionable for them, however they imagined databiting with respect to pace during arun to be influential (P2 was dealing with an injury preventing them from runningduring the time of the study), “It’s whenever I have something like this [databit-ing], it’s not for the sake of interest, it’s more sort of the sake of can it help meright? It’s that the data that I was provided when I was doing, like a weightliftingworkout, wasn’t really relevant to me. I didn’t really care what my heart was. Ididn’t really care how many calories I was burning cause that’s not why I’m work-ing out right. Something like a run, for example, would be nice to compare my144pace. If I’m trying to keep pace, that makes sense because I can actually adjustmy workout as I’m doing it. It’s very workout dependent, any workout that you’regoing to be consistently checking your stats that makes a lot more sense to be toexplore your data further.Further ExplorationAnother benefit discussed previously is that of using databiting as a steppingstone to further explore one’s data in a more traditional manner. While the majorityof participants did not feel the need to further explore their data, interestingly, wefound through discussion that four participants (P5, P8, P9, and P12) in fact didconduct additional data exploration due to a databiting interaction.Each participant had varying reasons: P5 found that databiting reinvigoratedtheir motivation to explore their data (they had previously exported data to Notionand done their own data analysis). P8 used databiting and initial curiosity to ex-plore their pace for the first time, which in turn sparked a new-found interest inexploring and monitoring pace. They mentioned “I actually got interested in look-ing at my pace, like I’m going to use that as a metric now. Because it was so easilyaccessible and easily seen on the app I was like, oh, wait like, why don’t I check mypaces for the last few months on my phone in more detail.” P12 was similar, in thatdatabiting allowed for initial exploration, which resulted in curiosity that initiatedfurther exploration on the smartphone. Finally, P9 further explored their data as ameans to verify a performance value they uncovered through databiting while ona walk. They had found that during a databiting interaction, their heart rate wassignificantly lower than expected, and used their phone to further look into theircollected workouts to better understand why: “Yeah, I did have to do that [pull outtheir phone] because I was like this is so weird. Why is it so low? Because likewhenever I do my workout my average is usually at least above 100 something, soI was like really confused. But then I pulled it up [on the phone] and I was okaythat makes sense because we did like a really really slow paced walk [with a friendthat likes to casually walk].”1457.4.5 General Reactions to DataWatchIn general, all participants expressed excitement and positivity towards thedatabiting feature included in DataWatch, expressing their belief that it should befunctionality included by default. For instance, P8 highlighted “I think this is some-thing that smartwatch should absolutely have. I mean, a big reason I wanted to geta watch was to not have to pull up my phone out of my pocket, especially when I’mbeing active or when I’m like at the gym or whatever.” While P9 further contrastedtheir experience with DataWatch to that of Apple’s default Workout application;they mentioned “I did appreciate the additional data that it was able to give me sothat was really cool because I never really thought about it. But you actually don’tget any of that with Apple’s [Workout app]. So I thought it was really, really cool.”DataWatch notably enabled quick and easy access to information, our desiredgoal; as P5 highlights “After a workout, you’re exhausted. You don’t really havethat kind of mental frame of mind to go through the past, but if you just havea quick thought and you’re like, OK, I want to ask that I thought it was superhelpful in that. It’s more convenient, it’s faster and it’s less effort...I don’t reallywant to be always going through my fitness data on my phone.” P8 also mentioned,echoed further by P4, that the interactions were well designed and suited the watch,saying “Especially when I’m being active or when I’m like at the gym or whatever,I’d rather be able to do it [exploration] my watch, just as interactions are a lotshorter, I don’t get distracted or I don’t have to, like, push through my pocket ormy backpack to get it [the phone].”However, a few drawbacks still remained and were noted by participants. First,an overarching topic that came up in most interviews was the hindrance of havingto unlock the iPhone in order to conduct databiting on the watch. P3 mentioned “Itended up a bit more frustrating just because, as you said, you know the the hard-ware and software limitations...oh, like your phone is not unlocked, your phone islocked, unlock it and then ask.” While this was a unfortunate aspect that could notbe avoided, we do believe this is a factor that can be simply mitigated in futureoperating system updates. Second, it was discussed a handful of times that certainworkouts simply offered too much friction for databiting during the workout itself.146P5 mentioned “I have the thought of the question I have in mind during the run, itis just the execution of that process...there’s some friction there. And so during themiddle of that run, it just felt like okay maybe I’ll come back to this later at somepoint when I have the time.” Supported by our recorded uses of DataWatch, nodatabiting was conducted during a run that was not immediately after starting a runor immediately before ending the run. This highlights that databiting, as currentlyimplemented, may need to be even more lightweight for increasingly physical andcognitively demanding workouts.Lastly, while no formal question was asked with respect to whether participantswould want to continue using DataWatch, over that of the Apple Workout appli-cation, three participants (P1, P4, P10) asked us, unsolicited, if they could keepDataWatch and continue to use the app on their smartwatch outside of the study.This unsolicited interest suggests a preference for DataWatch, highlighting its per-ceived benefits and user satisfaction. It indicates that the databiting features andusability of DataWatch may offer a more appealing or effective experience thanexisting alternatives, promoting the idea of databiting and its inclusion in applica-tions on the smartwatch.7.5 Discussion7.5.1 Feasibility of Databiting on the SmartwatchPrevious research has established natural interaction times for various smart-watch application categories [148, 192]. Specifically, fitness applications havebeen shown to have interaction times on average of 18 seconds. Furthermore,smartwatch interaction has been seen to have a maximum acceptable boundary ex-tending to 45 seconds for tasks involving navigation and communication. In ourstudy databiting interactions using DataWatch were on average 21.7 seconds (Min= 7.4, Max = 58.8). This finding suggests that databiting interactions not only re-main well within the acceptable limit of 45 seconds, they in fact align rather closelywith the typical interaction times observed for fitness applications. This adher-ence to established interaction boundaries, alongside the strong general reactions147to DataWatch, underscores our applications effectiveness in providing a seamlessand efficient experience for in-situ exploration of fitness data on smartwatches.In addition to the quantitative and temporal aspects, all participants agreed thatdatabiting functionality on the smartwatch was both feasible and highly desired.This consensus underscores several key points: First, databiting offers a level ofexploration that accommodates a wide range of users. In our study, participantsvaried in their exploration experience—from those who used the smartwatch pri-marily for general activity tracking to others who exported their data into other pro-grams for in-depth analysis. Second, databiting was not perceived as a hindranceto the activities being tracked. This collective endorsement for databiting on thesmartwatch highlights the potential for databiting to enhance the smartwatch expe-rience, with respect to fitness tracking and data exploration, making it a versatiletool for all users.7.5.2 Reflecting on Design GuidelinesDR1: Utilize Intuitive Layouts, Interactions, and Visualizations to SupportLay Persons All participants agreed that DataWatch effectively emulated theApple Workout application, making it intuitive and easy to adopt without requiringsignificant upfront effort. However, several areas for improvement were identified.Participants noted the absence of features such as automatic recognition of work-out start and end times, as well as more complex metrics like power and cadence,which are now offered by Apple. Incorporating these features in future updateswould enhance the intuitive and expected behavior of the application, further sup-porting its usability for laypersons.With respect to visualization, we opted for line charts to display resulting infor-mation. Participants commented on the benefit of viewing data through the chartspresented. However, our choice could be enhanced by incorporating sparklines[135] and bar charts, which may provide clearer and more accessible representa-tions. While we aimed to facilitate data exploration through visual representation,and wanted to incorporate all data points involved with the query, the limited num-ber of queries resulting in this type of exploration underscores the complexity of148the task. Most queries requested a small subset of filtered data, however, as wesaw 19 queries which still requested all captured data. Simplifying the representedinformation could promote more efficient and intuitive exploration directly on thesmartwatch and in-situ. We may in turn also allow for a greater level of multi-modal input, where data could be algorithmically aggregated (e.g., data capturedin the past year could be grouped by months) and a single bar could be tapped toprovide details-on-demand. By refining our visualization approach, we can bettersupport users in making sense of larger amounts of data quickly and effectively.DR2: Leverage the Complementary Nature of Speech and Touch Interactionson the Smartwatch While speech interaction was widely adopted within ourstudy, the occasional use of the phone for text input and remaining concerns ofsocial awkwardness encourage us to think about potential areas for improving themulti-modality of DataWatch. Additionally, participants noted that speech interac-tion could be cumbersome at times, particularly during workouts such as running.While speech interaction is flexible, touch interaction capabilities could be im-proved to offer set exploration features throughout. Enhancing touch interactionscould involve more detailed metrics/visualizations accessible through tapping onrespective metrics. Furthermore, list pickers could allow for preset adjustmentsto the already on-screen visualizations. In future, implementing an increasinglymulti-modal approach offers a best of both worlds, allowing all to feel comfortableexploring their data in-situ on the smartwatch.DR3: Support Comprehensive In-Situ Exploration Our study further revealedthat participants had a relatively equal interest in exploring fitness data not onlyduring workouts but also before and after them. This was compounded by the un-intentional capturing of databiting interactions that occurred away from a workoutitself. This comprehensive, in-situ, exploration desire aligns with our findings inChapter 4, where users expressed a need for continuous access to their health datathroughout their daily routines. This result further highlights the need to broadenthe scope of smartwatch applications beyond in-workout tracking and monitoringof metrics to encompass more holistic health monitoring and data-driven decision-149making throughout the day and surrounding one’s workouts.Moreover, participants’ interest in exploring data before workouts again sug-gests a preemptive and proactive approach to fitness data exploration and under-standing, where users seek quick and immediate information from the smartwatchto plan and prepare. This finding also highlights the potential for smartwatches tosupport preemptive health behaviors and goal setting, enhancing user engagementbeyond during and after-workout analysis.DR4: Enable Flexible Temporal Filtering to Support Visually-ConstrainedExploration Activity-based filtering was predominantly utilized throughout ourstudy, allowing us to largely achieve our design goal of offering implicit temporalfiltering through activity-based filters (i.e., ...last 5 runs). During the study, how-ever, we observed instances where participants expressed interest in more granularfiltering options beyond generalized activity instances (e.g., different running dis-tances or cycling routes). For example, some users indicated a desire to filter databy specific running distances, such as by the last seven 5 km or 10 km routes, orby geographic cycling routes/locations. Furthering the activity-based filtering ca-pability could cater to increasingly diverse user needs and provide more nuancedinsights into fitness data trends.7.5.3 LimitationsOne limitation of the current version of DataWatch is its reliance on the user’siPhone being unlocked to access personal health data. This requirement adds anextra step to the interaction (not captured in our times reported above) and disruptsthe in-situ smartwatch experience. While we believe that this limitation was notenough to impact our study findings, it was a frustration noted by many partici-pants. In future, we simply expect to not be limited by this. Companies in controlcan simply offer increased storage directly on their smartwatches, provide alter-native wearable authentication methods, and/or improve synchronization betweendevices when data is requested.Another notable limitation is the absence of the Speech framework supported150directly on the Apple Watch15. Because of this, DataWatch could not in fact pro-vide auditory feedback in response to databiting queries, as studied in the previouschapter. This limitation reduces DataWatch’s versatility in-situ, as auditory feed-back could benefit user interaction while performing other tasks [20]. Given thatthe majority of databiting interactions involved single Value explorations, auditoryfeedback could have been structured as per our previous Chapter’s findings, andbeen easily incorporated. Future iterations, once Speech is supported, could in-clude this element. Including auditory responses could broaden the application’sappeal, offering intuitive and increasingly multimodal interaction while in-situ.7.6 SummaryDataWatch is a novel, custom-built, smartwatch fitness application that enablesdatabiting—lightweight and transient data exploration—while in-situ of a trackedworkout. In this chapter, we highlighted design guidelines as well as the userinterface and interaction design of DataWatch, which was built from the results ofprevious chapters.To better understand and support the argument for databiting as a viable ex-ploration paradigm on the smartwatch, we then conducted an exploratory, obser-vational study with 10 participants. Participants used DataWatch to track theirworkouts over the period of one-week, databiting when desired. Our findings, in-cluding many instances of databiting, underscore that databiting interactions canoccur within a reasonable amount of time directly on the smartwatch and whilein-situ. Moreover, unanimous participant agreement emphasizes the utility and de-sirability of databiting as a feature that should be integrated into smartwatch fitnessapplications.15https://developer.apple.com/documentation/speech151Chapter 8Discussion8.1 Revisiting Lightweight and Transient Interaction forInsight Rich ExplorationThroughout this thesis, we intentionally refrain from quantifying lightweightand transient. As discussed in Chapter 3, the concept of databiting can vary signif-icantly between individuals and contexts. This inherent variability creates a fuzzyboundary, making precise quantification challenging. However, one constant re-mains: within any given person or context, databiting tends to be more lightweightand transient compared to comprehensive data exploration. Conversely, it requiresmore effort and time than visually assessing a glanceable visualization. Yet, databit-ing must remain feasible in-situ and while on-the-go. Analogously, in terms ofeating, snacking is undoubtedly more lightweight and transient than consuming afull meal yet requires slightly more effort than simply viewing and assessing foodin front of you. Our goal with databiting was to offer data exploration equivalentto snackingWithout quantifying lightweight and transient, a question remains: Did weachieve our goal in enabling databiting—lightweight and transient data explorationfor increasingly rich insight? We believe there are results from within our finalstudy which support the success of databiting. Figure 8.1 highlights aspects oftransient and lightweight exploration, which we provide for discussion purposesonly and not as a concrete means for measure.152Figure 8.1: For discussion purposes we introduce the above framing of lightweight and transient data exploration. Weprovide this here, rather than in Chapter 3, due to the fuzzy and hard-to-quantify nature of these boundaries. As such, withthis figure we aim to provoke thoughtful discussion rather than establish definitive measures early on. While we recognizethat the smartphone, tablet, and laptop can provide glanceable and initial interactions we place them on the right of the figureas this is where many interactions are meant to take place.153First, transience is evident in the short duration of databiting interactions ob-served in our final study. Participants were able to quickly integrate their data,through input of a query, and even interpret the result without prolonged engage-ment. In fact, the entire databiting interaction took less than 30 seconds on average.While less transient than a glanceable visualization, importantly, this remains wellwithin the typical interaction time on a smartwatch [148, 192] while also remain-ing largely under the typical interaction time seen on the smartphone, tablet, anddesktop which often involves interaction that are minutes at a time [132, 132, 179].Lightweight interactions are more difficult to quantify. As such, we recognizea lightweight interaction as one that simply does not require multiple and iterativeinputs from the person. Furthermore, this interaction should not hinder the pri-mary task at hand. From our interviews with participants, and the endorsementof databiting by all, we argue that the lightweight capability was largely achieved.However, while databiting was in fact conducted during workouts, further sup-porting its lightweight nature, we also recognize that the current implementationof databiting was not seen as lightweight enough to be conducted during certainworkouts that require more attention and are faster paced (i.e., running). This find-ing suggests that further work needs to be done to reduce the effort required toachieve a databite under such circumstances. Such possibilities were discussed byour participants in our final study and include simplifying the databiting interac-tions through pre-meditated design of visualizations or selection of saved databit-ing queries that are readily available.Our goal, given that the interaction needed to be transient and lightweight, wasto ensure that people could still achieve insight rich generation of knowledge be-yond what was currently possible. While some insight was actionable, participantsstill expressed finding new insights, verifying unknown values, and aligning theirperformance with their goals which was not a possibility prior given the displayedmetrics on the smartwatch. This allowed users to make informed decisions on-the-go without the need for more time-consuming analysis. As a result, DataWatchsuccessfully bridged the gap between glanceable information and detailed explo-ration, enhancing the overall utility and user experience of the data collected by thesmartwatch.154The next logical question is how far can databiting go? In other words, howmuch insight can be provided within a transient time frame and without mucheffort. Regarding this question, we believe we are nearing the leading edge ofdatabiting interaction. Beyond the transient and lightweight boundaries discussedin previous paragraphs, our studies in Chapters 5 and 7 reveal that value-baseddatabiting was most commonly desired and queried. Value-based responses areintriguing as they do not offer the ability to further explore information or gatherserendipitous insight afforded by a visualization. This may have been influencedby an understanding of the smartwatch’s limitations by our participants, where theyrecognize other devices as more efficient for such exploration. Together, consid-ering the temporal databiting findings, the lightweight maximum that was seen astoo much during certain workouts, and the value responses, this indicates that wehave likely found a reasonable maximum for databiting, providing quick, easy, ac-tionable insights while maintaining ease of use and efficiency on the smartwatch.8.2 Towards Speech Interaction for Personal HealthData ExplorationThrough DataWatch’s speech interaction capability in Chapter 7, and that ofour elicitation app in Chapter 4, spoken queries were integrated into everyday sce-narios of our participants. This included being at the gym, walking alongside oth-ers, and even in public settings such as a sports match. Notably, and for the largemajority, we found a significant shift towards a new, more positive, view of us-ing speech as in input modality for providing personal health data queries. Thisis largely impactful, as our studies were conducted in-the-wild using working pro-totypes. Previous work that has found opposite, more negative, viewpoints haslargely been conducted in isolated, in-lab, or survey environments [55, 98, 116].This shift was largely driven by a few common ideologies: First, bluetoothheadphones and interactions with voice assistants are becoming common place,making the act of interacting less socially awkward than previously noted. Second,the immediate benefit and ease of interacting through speech, especially while in-155situ, was recognized by participants and outweighed drawbacks that may otherwisearise. Third, data exploration of common metrics, as long as the answer is notmade aware to the public, was not seen as personal enough to cause concern. Forexample, simply asking if one’s heart rate has been above their average is not aquery that gives away valuable and personal information.However, it is important to recognize that this positive attitude towards speechinteraction is not universal. Half of our participants in our elicitation study in Chap-ter 4 still had reserved privacy concerns, while one participant in our DataWatchstudy in Chapter 7 did not use speech at all. Future research should continue toexplore ways to address these concerns, perhaps through increased multi-modalinteractive capabilities.8.3 Semi-Longitudinal Study MethodologiesThe choice of a one-week in-the-wild study methodology, used twice withinthis thesis, was primarily motivated by the need to balance participant engagementwith the feasibility of accurate recall. Pilot testing revealed that in one week par-ticipants could comfortably integrate the study application into their daily lives,while periods longer than one week led to noticeable memory recall issues. There-fore, a one-week study provided an optimal balance, allowing for natural use andreliable data collection. Below we discuss benefits and drawbacks that were eitherexplicitly considered or a resulting lesson of the methodology used.Benefits A variety of benefits, discussed below, arose from use of a one-weekin-the-wild study methodology, which were underscored by results presented inprevious chapters.First, conducting a one-week study offers significant advantages in terms offeasibility and practicality compared to increasingly longitudinal studies. Theshorter, one-week, duration is easier to manage and demands fewer resources,which benefits both us as researchers and the participants who are giving their time.With a one-week duration, the need for intermediate check-in sessions and pro-longed support is minimized, simplifying the study process. Logistical challenges,156such as scheduling and maintaining participant compliance, are also reduced, mak-ing it easier to coordinate and conduct the studies. Additionally, the shorter studyperiod lowers overall costs, making it a more economical choice. Yet, consider-ing these benefits, a one-week study remains to ensure that the necessary data iscollected without overhead unnecessary to the shorter-term goals of the projects.Second, the practicality of the one-week study duration allows for quick andfrequent iteration of the study design. As demonstrated in the pilots conductedin Chapters 4 and 7, the full and complete study methodology was able to beemployed, providing us with comprehensive insights regarding not only the in-troductory tutorials, but also the technical limitations of the applications, and thestructure and flow of our interview. Together, having a complete understanding ofthe study methodology, and even true data (i.e., data captured in the exact meansour study participants would report data) from which to analyze, we were able toiterate components of our study design in a timely, yet complete, manner.Third, a one-week study duration remained to allow for participant recall ofinteractions and experiences. Over a longer duration, recall issues can set in, lead-ing to less reliable quantitative reporting. In fact, during the interviews at the endof the week, we observed that some participants occasionally needed a brief mo-ment to recall contextual information related to an interaction, query, or use of theapplications that happened earlier in the week. While no complete recall issuestook place, we recognize the one week duration as bound to consider for recallfrom participants using an application throughout their daily lives. By limiting thestudy to one week, and importantly providing details of responses and interactionswithin the interview (an element that was iterated upon in the study conducted inChapter 4 in part to the practical time frame used), participants were better able toaccurately recall details about their usage patterns, behaviors, and experiences withthe study applications. This overall recall accuracy results in higher quality data, asparticipants’ reports are more likely to be precise and reflective of their true inter-actions. Consequently, the qualitative insights gained from our one-week studiescan be considered reliable, allowing us to be confident in the results reported.Finally, we believe that a one-week period strikes an optimal balance for par-ticipant engagement, as it is long enough for users to become familiar with the157novel application they are using, beyond that of the tutorials provided, while alsoproviding a chance to fully incorporate it into their daily lives. In fact P9 fromour final study supported this aspect by discussing “Yeah, I think my first couple ofones were just like, okay, yeah, let’s just play around and see. But then, afterwardsI guess, the numbers I was looking for were a little more meaningful.” We believethat the relatively short time frame also helps to maintain participant interest andmotivation, as it avoids the fatigue and disengagement that can occur in longerstudies. By keeping the study period concise, participants remain focused, inter-ested, and committed; of our 30 total participants across the studies in Chapters 4and 7 we only had one participant drop out of the the study in Chapter 4, due toexternal reasons which they felt impeded their ability to participate.Drawbacks While there are a variety of benefits, a one-week in-the-wild studymethodology can have drawbacks which we discuss below.First, while a one-week period provides valuable insights into initial and short-term databiting interactions, it may overlook long-term usage patterns and/or theeffects of sustained use over extended periods. For instance, changes in usagefrequency (i.e., number of databiting interactions during a workout), or shifts inuser preferences as interactions find a norm, may not come to be within just oneweek. This is an acknowledged limitation (which we further discuss below in ourthesis limitations discussion). Increasingly longitudinal approaches are needed tocomprehensively understand how databiting evolves over time and the benefits thatare thus derived. Balancing the immediate understanding of short-term insightswith the need for long-term understanding is a key direction for future work whichcan aim to provide a more holistic and comprehensive view of user behavior as itpertains to databiting.Second, the novelty effect present when using a new application, or simplybeing involved in a study itself, presents a drawback that must considered. Ini-tial enthusiasm and curiosity can influence participants’ interactions and behaviorsduring the early stages of using a new app or device. Examples of this within ourwork may include elicited queries which were provided simply due to participationin the study but were not queries that regularly arise, or databiting interactions in158DataWatch which would not be initiated beyond the novelty period of using theapplication. Some participants, especially those that may have conducted groupedexploration away from, before, or after a workout, are likely not to show this be-haviour over an extended period of use (in Chapter 7—P3, P6, P10). Consequently,data collected during the one-week study periods may not accurately reflect long-term, habitual, usage patterns as previously discussed above.To mitigate this drawback within our study design, we had participants inter-act with DataWatch during the tutorial session (an aspect that was iterated uponto support participants in their novelty periods); often, many started a workout inDataWatch to view the metric screens, while we also encouraged participants toask questions both of us and to the DataWatch application (of which the data wasexcluded from our analysis). We further provided remuneration that was not tiedto a number of responses, encouraged use only when it was deemed beneficial, andreinforced the notion that interactions were not being evaluated against others inthe study. Combined, we believe this allowed for immediate impacts of noveltyto be mitigated, however, we must remain mindful of this potential bias and con-sider common strategies to mitigate its impact. Further strategies for mitigationinclude conducting follow-up studies after the novelty has worn off or adjustingdata analysis to account for initial enthusiasm outside a certain bounds.Finally, capturing data in-the-wild poses unique challenges, especially whenwe consider our desired lightweight and transient nature of databiting. As such,a handful of elements were discussed throughout the creation of our final studydesigns. Natural use was deemed as paramount; we felt that interactions during thestudies should act as close to the desired interactions as possible. Consequently,intrusive methods such as frequent surveys (even if short) after every databitinginteraction was to be avoided to maintain the authenticity of user interactions andperceptions of use from participants. Furthermore, the number of questions askedwithin our elicitation study were in fact reduced to better coincide with in-situ use.Achieving comprehensive data collection is of course crucial to understand ourresearch questions. As we are losing information that can be gained immediatelyand in-situ, a blend of quantitative and quantitative insights, including through theuse of semi-structured interviews, ensures a comprehensive understanding of user159experiences. To further supplement the data collection, we could in future includenotifications for supplemental data capture throughout the day, when may be moreappropriate, reflecting back on use since the last reporting. These mixed methodsapproaches can further enrich data capture in-the-wild while not disrupting theintended interactions.8.4 Thesis Limitations and Future WorkChapter-specific limitations and future work were highlighted within each pre-vious chapter where applicable. Furthermore, Chapter 3, Section 3.3, highlightsin detail future research directions in pursuit of enabling databiting. In this thesis,we largely focused on the interaction modalities research challenge, necessary tofirst enable and validate such databiting exploration on the smartwatch. As previ-ously discussed, other future research directions include 1) incorporating contex-tual factors, 2) the relationship between databiting, glanceable visualizations, andlonger-form exploration, 3) personalization and customization, and 4) evaluationchallenges. Here, we provide more general thesis-level limitations and future workas a result of these limitations.8.4.1 Who is Databiting?Throughout this thesis, an absent challenge is that of who is interested indatabiting. More specifically, we do not focus on how varying populations mayrequire different databiting interactions to explore data, and even the queries thatare of importance. As this is early work aimed at first understanding and thenbuilding this form of lightweight and transient data exploration in-situ on the smart-watch, our focus is on the larger general population that has been collecting datafor a minimum of three months. This criterion ensures that participants have someknowledge of their needs and data, providing a more informed basis for our re-search.However, this approach notably excludes individuals. As example, those whohave only recently purchased a smartwatch may have very different health queries160given their initial exploration of the smartwatch’s capabilities. Moreover, thosebeing introduced to the collection of their personal health data may have varyingqueries and databiting needs. Additionally, our work does not account for spe-cific groups such as athletes, who may have unique performance-related queries,or older adults and other individuals with chronic health conditions who might pri-oritize different types of data. By not addressing these varied user groups, our workmay not fully capture the diverse range of queries and needs for databiting acrossdifferent demographic groups.Future research should aim to explore these differences to create more inclu-sive and tailored databiting solutions that cater to the specific needs of a broaderpopulation. This includes conducting studies with diverse demographic groups,considering factors such as age, fitness level, health status, health goals, experi-ence collecting and exploring data, socioeconomic background, and cultural con-text. By doing so, we can ensure that databiting interactions evolve to meet thevaried and specific needs of everyone, making data exploration on smartwatchesmore accessible and impactful.8.4.2 Study With Apple Device UsersThroughout this thesis, we focused our study on Apple Watch users. This deci-sion was driven by two main factors: First, the time-consuming nature of develop-ing for individual watch platforms led us to choose a single platform. Unlike smart-phones, where frameworks like React Native can be used across multiple platforms(e.g., Android and iOS), smartwatches do not offer similar cross-platform devel-opment tools. Second, we chose the Apple Watch because it has the largest userbase in North America, making participant recruitment easier. Although this deci-sion excludes users of other smartwatch brands (e.g., Google, Samsung, Garmin),we believe that the common sensors, interaction modalities, and typical usage pat-terns across all smartwatches minimize the potential diversity of queries that mighthave been elicited and any differences in the functionality of the final application.While perhaps a query regarding a derived metric (i.e., a combined score valueencompassing a range of raw captured metrics such as a Readiness Score or En-161ergy Level) would arise, it is likely to remain to fall within an overarching insightcategory. Therefore, while our focus on the Apple Watch is a limitation, it is un-likely to significantly impact the generalizability of our findings, where as who (asdiscussed above) is likely to provide greater influence.8.4.3 Dataset of Desired QueriesThroughout, our objective was to understand and enable databiting on thesmartwatch. One key aspect of this understanding was to first gather what querieswere of desire. Much of the work put forward in this thesis then relies on the datasetcaptured in Chapter 4. Specifically, the dataset captured contains 205 queries from18 participants (on average 11.4 queries per participant). The closest work to thisby Rawassizadeh et al. [154], provides a dataset of 716 queries from 131 par-ticipants (on average 5.5 queries per participant). However, study methodologyshould be carefully considered. While it is feasible to collect queries quickly andefficiently through online survey, producing a large dataset in the process, collect-ing data with real-world validity is much more cumbersome, yet valuable.A natural question that arises from any elicitation study is whether we havecaptured all possibilities, either specific queries, broader insight categories, and/ormechanisms used within a query. We cannot say with certainty that we capturedall possibilities, and it is entirely the case that given additional participants anddevice capabilities that a specific query would arise that has not been previouslymentioned. An example of such a non-reported query could relate to outlier de-tection (e.g., “Were there any instances of irregular heartbeats or atrial fibrillationin the last day?”). However, we do believe that our dataset provides a compre-hensive and importantly real-world report of desired queries. Specifically, whencoding insight categories of the queries, not only did we find examples of insightcategories previously noted in research, we in fact expanded upon categories whilealso introducing a new category. Furthermore, when comparing our results to thedataset captured by Rawassizadeh et al. [154], we found identical components thatmake up a query. Again, however, we were able to expand upon filtering mech-anisms and showcase divergence of interrogatives given our real-world context of162the provided queries.As our goal was not to train a model for understanding and resolving of queries,a large dataset is not needed to accomplish our research goals. By coding queriesinto overarching structures and insight categories, we were able to derive meaning-ful patterns and insights from the collected data. Furthermore, the real-world con-text of our data collection adds to the validity and applicability of our results, pro-viding a strong foundation for not only our successful development of DataWatch,but also for future research to build upon which has not been previously seen. How-ever, more data would undoubtedly enhance the robustness and generalizability ofour findings. Future studies should aim to expand the dataset further, capturinga wider variety of queries and participant demographics to validate and extendthe insights derived from our current dataset. This would help to ensure that thedatabiting interactions we have identified are representative of a broader user baseand can be more effectively utilized in diverse real-world contexts while also be-ginning to allow for training of models which can support such personal health dataexploration.8.4.4 Part-of-Speech Tagging Versus Large Language ModelsDeploying large language models (LLMs) on smartwatches for personal healthdata exploration raises significant challenges that are not easily mitigated. Theseinclude ethical concerns surrounding data ownership and control, model trainingrequiring substantial data, transparency of model features, and compliance withregulatory frameworks [72]. Moreover, LLMs pose risks such as biased responses,potential over-reliance or unsafe use, and exploitation of user trust to gain privateinformation [195], all of which could negatively impact personal health outcomes.Additionally, on-board computation, though improving year over year, remains alimitation for older smartwatch models still in circulation. For these reasons, andto prioritize creating an immediate and accessible impact, we focused on imple-menting a part-of-speech tagging approach.Despite these challenges, LLMs are rapidly emerging as a highly capable andpromising tool. Thus, they should not be ignored. Unlike part-of-speech tag-163ging, LLMs offer advanced query processing and contextual understanding. Forinstance, a complex query such as, “Compare my heart rate and pace at each kilo-meter of my run yesterday and my last 5K”, exceeds the capabilities of our currentapproach. Furthermore, queries which require understanding, such as “How hasmy sleep affected my running pace?” can not be currently handled. Recent re-search has demonstrated the potential of LLMs in data informatics tasks [96, 178].By leveraging their ability to synthesize and interpret complex natural languagequeries, LLMs could unlock more dynamic and flexible interactions with personalhealth data. Looking ahead, future work in personal data exploration and inter-face customization will undoubtedly benefit from integrating LLMs while buildingfrom and incorporating our findings throughout.8.4.5 Enabling Preemptive and Proactive InsightPreemptive and Proactive exploration was a new insight category found withinour collected dataset of desired queries. We believe that this was observed due toour study setup where we emphasized to our participants that they should not worryabout the capabilities of existing technologies; whereas previous data explorationwork has been conducted with a working prototype, such as DataWatch was lateron, despite its limited functionality [30, 63, 98].Currently, mHealth applications and smartwatch operating systems provide aprimitive form of preemptive and proactive insight. Notifications, suggestions, ormotivational reminders are often utilized to proactively encourage people to standup, breathe, or move after sedentary periods. However, these are limited in theirexpressiveness and ability, and do not allow for data exploration to be included.Our elicited queries were much more involved. The queries within this categorywere aimed towards people utilizing their personal health data to prepare for futureevents through system recommendations, see Chapter 4 Section 4. This included,querying when to workout, how long to workout for, and specific workouts to do.We actively chose not to support such queries within DataWatch at this time.The looming limitation is that of ethical concerns in providing misguided infor-mation which could in turn have adverse impacts. For example, if a person asks164for a distance they should run, we must confidently be able to justify the response.A value of 5 kms versus 10 kms is a difference that could significantly impact aperson’s physical well-being, potentially leading to injury or overexertion if therecommendation is not appropriately tailored to the individual.In the pursuit of enabling this form of insight in the future, many questionsarise; to showcase these, we highlight two queries from our dataset: ”How muchdancing do I need to do to burn 800 calories?” and ”Give me a suggested work-out based on my readiness score?”. First, how should the answer be calculated?Calculating the appropriate answer can be a challenge, especially when little priordata is available or standards are unknown. Second, when providing an answer,how can we convey uncertainty and variance to the user? These queries often donot have a discrete answer available, with more factors and external data needingto be considered. Finally, is this form of insight ethically possible? With the abilityto recommend, and ultimately have a person act upon an answer, this carries with itthe importance of not misleading a person which could have ramifications. Theseareas of future study are important in enabling this form of beneficial and desiredinsight on a smartwatch, and even for a broader set of devices.8.4.6 User Adoption and Behavior ChangeOur work is limited when considering the scalability and long-term use ofDataWatch and the underlying databiting interactions that it is built upon. Thisthesis primarily focuses on initial user intentions and preferences, which may notaccurately reflect long-term user behavior. While this initial study allows us to ex-plore and argue the feasibility of such interactions, it does not support the notion oflong-term use. Given an extended period of time, people may exhibit different en-gagement patterns, as we already saw within our week long studies. This differinguse can be influenced by evolving personal goals, changing contexts of use, and theperceived benefit that is gained from such interactions. However, as we see fromthe lived personal informatics model proposed by Epstein et al. [59], explorationpractices revolve and evolve as time goes on. This suggests that initial databitinginteractions may be targeted again over time even after a period of non-use.165Furthermore, sustained interactions with a novel system such as DataWatch areinherently challenging to achieve. While initial adoption may be driven by noveltyand curiosity, maintaining consistent usage requires the technology to integrateseamlessly into users’ daily routines and provide continuous value. While we ar-gue that integration in-situ is wholly feasible, sustained value is something we didnot explore. Factors such as usability, perceived usefulness, and ongoing user en-gagement strategies play critical roles in fostering long-term interest of databitingpractices to the benefit of the people conducting such exploration.Overall, while databiting and the following DataWatch application shows promiseand feasibility for lightweight and transient access to personal data, further re-search is needed to address the limitations related to long-term user adoption andbehavior change as a result of interactions and insights gained. Investigating howusers interact with the technology over extended periods, understanding the factorsthat influence sustained engagement, the influence of databiting on further explo-ration and vice versa, and quantifying long-term behaviour change as a result ofdatabiting practices will all provide further valuable insights for improving the thescalability and long-term usability of databiting.166Chapter 9ConclusionThe research, and its outcomes, in this thesis provide an understanding andimplementation of databiting, including the conceptualization of this lightweightand transient approach to data exploration (Chapter 3), elicitation of queries fromsmartwatch users in their daily lives (i.e., what, when, where, and why queries aredesired—Chapter 4), and an analysis of input and output requirements (i.e., howwe can support databiting on the smartwatch—Chapters 5 and 6). This founda-tion of knowledge then supported the development of DataWatch (Chapter 7), asmartwatch workout tracking application that facilitates in-situ databiting explo-ration of past workout fitness data through lightweight and transient interactionsusing touch and speech, enhancing peoples’ ability to gain insights during variousphases of their workouts.The findings from this research provide overarching insight that carries impor-tance for the future of smartwatch and broader on-the-go mobile health applica-tions. Specifically, the potential for smartwatches to offer deeper, more interactiveengagement with personal health data is not only desired but wholly achievable.People seek more from their smartwatches than passive data collection; they wantinsightful and in-situ engagement that informs and motivates their fitness activ-ities. DataWatch exemplifies how smartwatch health applications can evolve tobegin to meet these expectations by facilitating in-situ exploration of past work-out data through lightweight and transient touch and speech interactions—knownas databiting. This approach to data exploration was demonstrated to be feasibleon the smartwatch while in-situ and offers benefits including introducing increasedactionable insights and motivation to further engage with collected data.1679.1 Summary Contributions of the ThesisIn summary, the overarching contributions of this thesis are as follows:C1: An introduction of the term databiting, conceptualized as lightweight andtransient data exploration, that bridges the gap between quick, glanceabledata visualizations and more extensive, detailed data analysis. Further toproviding a conceptualization of databiting, we also discuss potential bene-fits and highlight future, and necessary, research directions. This foundationsets the stage for developing more intuitive and effective ways for peopleto engage with their personal data, both on smartwatches and other mobileplatforms.C2: An elicited, and now publicly available, dataset of 205 personal health dataqueries desired for exploration on a smartwatch throughout daily life. Thisdataset, coupled with a thorough qualitative and quantitative analysis, offersvaluable insights into what personal health data query needs people have,including when, where and why these queries arise.C3: Interaction requirements for lightweight and transient exploration of per-sonal health data on smartwatches. We identify key dimensions of our cap-tured queries, such as interrogatives, data sources, aggregations, and filter-ing mechanisms, which are crucial for enhancing input methods, particularlynatural language processing capabilities. Additionally, we gather insights onuser preferences for different output structures from voice assistants, focus-ing on perceived quality, behavior, comprehensibility, and efficiency. Thesefindings provide essential guidelines for optimizing both input and output in-teractions, ensuring that data exploration on smartwatches can be supportedin being lightweight and transient.C4: Development and validation of DataWatch, an Apple Watch application thatdemonstrates the practical implementation of databiting. By integratingmultimodal interactions, specifically touch and speech, DataWatch enhancesin-situ exploration of past workout data, demonstrating the feasibility and168user benefits of providing lightweight and transient data exploration capabil-ities directly on the smartwatch.169Bibliography[1] Sunggeun Ahn, Jaeyeon Lee, Keunwoo Park, and Geehyuk Lee. Evaluationof edge-based interaction on a square smartwatch. International Journal ofHuman-Computer Studies, 109:68–78, 2018. → pages 24[2] Emily Couvillon Alagha and Rachel Renee Helbing. Evaluating the qualityof voice assistants’ responses to consumer health questions about vaccines:an exploratory comparison of alexa, google assistant and siri. BMJ health &care informatics, 26(1):e100075, 2019. → pages 8, 85, 87[3] Fereshteh Amini, Khalad Hasan, Andrea Bunt, and Pourang Irani. Data rep-resentations for in-situ exploration of health and fitness data. In Proceedingsof the 11th EAI international conference on pervasive computing technolo-gies for healthcare, pages 163–172, 2017. doi: 10.1145/3154862.3154879.→ pages xviii, 2, 6, 8, 14, 22, 23, 25, 28, 44, 45, 47, 56, 57, 58, 64, 65, 91,104, 124[4] Ahmed Sabbir Arif and Ali Mazalek. A survey of text entry techniquesfor smartwatches. In Human-Computer Interaction. Interaction Platformsand Techniques: 18th International Conference, HCI International 2016,Toronto, ON, Canada, July 17-22, 2016. Proceedings, Part II 18, pages 255–267. Springer, 2016. → pages 18[5] Daniel L Ashbrook. Enabling mobile microinteractions. Georgia Instituteof Technology, 2010. → pages 87[6] Stavros Asimakopoulos, Grigorios Asimakopoulos, and Frank Spillers. Mo-tivation and user engagement in fitness tracking: Heuristics for mobile170Bibliographyhealthcare wearables. In Informatics, volume 4, page 5. MDPI, 2017. →pages 22[7] Christiane Attig and Thomas Franke. Abandonment of personal quantifica-tion: A review and empirical study investigating reasons for wearable activ-ity tracking attrition. Computers in Human Behavior, 102:223–237, 1 2020.ISSN 07475632. doi: 10.1016/j.chb.2019.08.025. → pages 21[8] Jillian Aurisano, Abhinav Kumar, Alberto Gonzales, Khairi Reda, JasonLeigh, Barbara Di Eugenio, and Andrew Johnson. Show me data”: Obser-vational study of a conversational interface in visual data exploration. InIEEE VIS, volume 15, page 1, 2015. → pages 7, 37, 120, 123[9] Karlin Bark, Jason Wheeler, Gayle Lee, Joan Savall, and Mark Cutkosky.A wearable skin stretch device for haptic feedback. In World Haptics 2009-Third Joint EuroHaptics conference and Symposium on Haptic Interfacesfor Virtual Environment and Teleoperator Systems, pages 464–469. IEEE,2009. → pages 17[10] Frank Bentley, Konrad Tollmar, Peter Stephenson, Laura Levy, Brian Jones,Scott Robertson, Ed Price, Richard Catrambone, and Jeff Wilson. Healthmashups: Presenting statistical patterns between wellbeing data and con-text in natural language to promote behavior change. ACM Transactions onComputer-Human Interaction (TOCHI), 20(5):1–27, 2013. → pages 14, 15[11] Lonni Besanc¸on and Pierre Dragicevic. The continued prevalence of di-chotomous inferences at chi. In Extended Abstracts of the 2019 CHIConference on Human Factors in Computing Systems, CHI EA ’19, page1–11, New York, NY, USA, 2019. Association for Computing Machin-ery. ISBN 9781450359719. doi: 10.1145/3290607.3310432. URL https://doi.org/10.1145/3290607.3310432. → pages 95[12] Timothy W Bickmore, Ha Trinh, Stefan Olafsson, Teresa K O’Leary, RezaAsadi, Nathaniel M Rickles, and Ricardo Cruz. Patient and consumer safety171Bibliographyrisks when using conversational assistants for medical information: an ob-servational study of siri, alexa, and google assistant. Journal of medicalInternet research, 20(9):e11510, 2018. → pages 88[13] Tanja Blascheck, Lonni Besanc¸on, Anastasia Bezerianos, Bongshin Lee,and Petra Isenberg. Glanceable visualization: Studies of data comparisonperformance on smartwatches. IEEE transactions on visualization and com-puter graphics, 25(1):630–640, 2018. doi: 10.1109/TVCG.2018.2865142.→ pages 2, 8, 24, 25, 28, 44[14] Tanja Blascheck, Frank Bentley, Eun Kyoung Choe, Tom Horak, and PetraIsenberg. Characterizing glanceable visualizations: From perception to be-havior change. In Mobile Data Visualization, pages 151–176. Chapman andHall/CRC, 2021. → pages 17, 25, 82[15] Tanja Blascheck, Lonni Besanc¸on, Anastasia Bezerianos, Bongshin Lee,Alaul Islam, Tingying He, and Petra Isenberg. Studies of part-to-wholeglanceable visualizations on smartwatch faces. In 2023 IEEE 16th Pa-cific Visualization Symposium (PacificVis), pages 187–196, 2023. doi:10.1109/PacificVis56936.2023.00028. → pages 2, 8, 28[16] Nadine Bol, Natali Helberger, and Julia CM Weert. Differences in mobilehealth app use: a source of new digital inequalities? The Information Soci-ety, 34(3):183–193, 2018. doi: 10.1080/01972243.2018.1438550. → pages64[17] Niall Bolger, Angelina Davis, and Eshkol Rafaeli. Diary methods: Cap-turing life as it is lived. Annual review of psychology, 54(1):579–616,2003. URL https://doi.org/10.1146/annurev.psych.54.101601.145030. → pages 41, 45[18] Robin Brewer, Casey Pierce, Pooja Upadhyay, and Leeseul Park. An empir-ical study of older adult’s voice assistant use for health information seeking.ACM Trans. Interact. Intell. Syst., 12(2), jul 2022. ISSN 2160-6455. doi:172Bibliography10.1145/3484507. URL https://doi.org/10.1145/3484507. → pages87, 88[19] Stephen Brewster, Joanna Lumsden, Marek Bell, Malcolm Hall, and StuartTasker. Multimodal ’eyes-free’ interaction techniques for wearable devices.In Proceedings of the SIGCHI Conference on Human Factors in Comput-ing Systems, CHI ’03, page 473–480, New York, NY, USA, 2003. Associa-tion for Computing Machinery. ISBN 1581136307. doi: 10.1145/642611.642694. URL https://doi.org/10.1145/642611.642694. → pages17, 37, 84, 87[20] Stephen Brewster, Joanna Lumsden, Marek Bell, Malcolm Hall, and StuartTasker. Multimodal’eyes-free’interaction techniques for wearable devices.In Proceedings of the SIGCHI conference on Human factors in computingsystems, pages 473–480, 2003. doi: 10.1145/642611.642694. → pages 8,85, 151[21] Laura Burbach, Patrick Halbach, Nils Plettenberg, Johannes Nakayama,Martina Ziefle, and Andre´ Calero Valdez. ”hey, siri”, ”ok, google”, ”alexa”.acceptance-relevant factors of virtual voice-assistants. In 2019 IEEE Inter-national Professional Communication Conference (ProComm), pages 101–111, 2019. doi: 10.1109/ProComm.2019.00025. → pages 71[22] Ana Isabel Canhoto and Sabrina Arp. Exploring the factors that supportadoption and sustained use of health and fitness wearables. Journal of Mar-keting Management, 33(1-2):32–60, 2017. → pages 22[23] Scott Carter and Jennifer Mankoff. When participants do the capturing: therole of media in diary studies. In Proceedings of the SIGCHI conference onHuman factors in computing systems, pages 899–908, 2005. doi: 10.1145/1054972.1055098. → pages 45[24] Marta E Cecchinato, Anna L Cox, and Jon Bird. Always on (line)? userexperience of smartwatches and their role within multi-device ecologies. InProceedings of the 2017 CHI Conference on Human Factors in Computing173BibliographySystems, pages 3557–3568, 2017. doi: 10.1145/3025453.3025538. → pages46[25] Xiang’Anthony’ Chen, Tovi Grossman, and George Fitzmaurice.Swipeboard: a text entry technique for ultra-small interfaces that sup-ports novice to expert transitions. In Proceedings of the 27th annual ACMsymposium on User interface software and technology, pages 615–620,2014. → pages 18[26] Yang Chen. Visualizing large time-series data on very small screens. InProceedings of the Eurographics/IEEE VGTC Conference on Visualization:Short Papers, pages 37–41, 2017. doi: 10.2312/eurovisshort.20171130. →pages 2, 8, 28, 44[27] Eugene Cho, S. Shyam Sundar, Saeed Abdullah, and Nasim Motalebi. Willdeleting history make alexa more trustworthy? effects of privacy and con-tent customization on user experience of smart speakers. In Proceedings ofthe 2020 CHI Conference on Human Factors in Computing Systems, CHI’20, page 1–13, New York, NY, USA, 2020. Association for ComputingMachinery. ISBN 9781450367080. doi: 10.1145/3313831.3376551. URLhttps://doi.org/10.1145/3313831.3376551. → pages 87, 113[28] Eun Kyoung Choe, Nicole B. Lee, Bongshin Lee, Wanda Pratt, and Julie A.Kientz. Understanding quantified-selfers’ practices in collecting and ex-ploring personal data. In Proceedings of the SIGCHI Conference onHuman Factors in Computing Systems, CHI ’14, page 1143–1152, NewYork, NY, USA, 2014. Association for Computing Machinery. ISBN9781450324731. doi: 10.1145/2556288.2557372. URL https://doi.org/10.1145/2556288.2557372. → pages 14[29] Eun Kyoung Choe, Bongshin Lee, et al. Characterizing visualization in-sights from quantified selfers’ personal data presentations. IEEE computergraphics and applications, 35(4):28–37, 2015. doi: 10.1109/MCG.2015.51.→ pages xviii, 14, 45, 56, 57, 58, 104174Bibliography[30] Eun Kyoung Choe, Bongshin Lee, Haining Zhu, Nathalie Henry Riche, andDominikus Baur. Understanding self-reflection: how people reflect on per-sonal data through visual data exploration. In Proceedings of the 11th EAIInternational Conference on Pervasive Computing Technologies for Health-care, pages 173–182, 2017. doi: 10.1145/3154862.3154881. → pages xviii,14, 45, 56, 57, 58, 104, 164[31] Jaewon Choi and Seongcheol Kim. Is the smartwatch an it product or a fash-ion product? a study on factors affecting the intention to use smartwatches.Computers in Human Behavior, 63:777–786, 2016. → pages 21[32] Veerisa Chotiyaputta and Donghee Shin. Explicating consumer adoption ofwearable technologies: a case of smartwatches from the asean perspective.International Journal of Technology and Human Interaction (IJTHI), 18(1):1–21, 2022. → pages 19[33] Stephanie Hui-Wen Chuah. You inspire me and make my life better:Investigating a multiple sequential mediation model of smartwatch con-tinuance intention. Telematics and Informatics, 43:101245, 2019. doi:10.1016/j.tele.2019.101245. → pages 21[34] Jaemin Chun, Anind Dey, Kyungtaek Lee, and Seung Jun Kim. A qualitativestudy of smartwatch usage and its usability. Human Factors and ErgonomicsIn Manufacturing, 28:186–199, 7 2018. ISSN 15206564. doi: 10.1002/hfm.20733. → pages 1, 16, 17, 19, 21, 23[35] Arlene E Chung, Ashley C Griffin, Dasha Selezneva, and David Gotz.Health and fitness apps for hands-free voice-activated assistants: contentanalysis. JMIR mHealth and uHealth, 6(9):e9705, 2018. → pages 88[36] Chia-Fang Chung, Kristin Dew, Allison Cole, Jasmine Zia, James Foga-rty, Julie A Kientz, and Sean A Munson. Boundary negotiating artifacts inpersonal informatics: patient-provider collaboration with patient-generateddata. In Proceedings of the 19th ACM conference on computer-supportedcooperative work & social computing, pages 770–786, 2016. → pages 14175Bibliography[37] Karen Church and Barry Smyth. Understanding the intent behind mobileinformation needs. In Proceedings of the 14th international conference onIntelligent user interfaces, pages 247–256, 2009. doi: 10.1145/1502650.1502686. → pages 1, 2, 45[38] James Clawson, Jessica A. Pater, Andrew D. Miller, Elizabeth D. Mynatt,and Lena Mamykina. No longer wearing: Investigating the abandonmentof personal health-tracking technologies on craigslist. In Proceedings ofthe 2015 ACM International Joint Conference on Pervasive and Ubiqui-tous Computing, UbiComp ’15, page 647–658, New York, NY, USA, 2015.Association for Computing Machinery. ISBN 9781450335744. doi: 10.1145/2750858.2807554. URL https://doi.org/10.1145/2750858.2807554. → pages 21, 35[39] Sunny Consolvo and Miriam Walker. Using the experience sampling methodto evaluate ubicomp applications. IEEE pervasive computing, 2(2):24–31,2003. doi: 10.1109/MPRV.2003.1203750. URL https://doi.org/10.1109/MPRV.2003.1203750. → pages 41, 45[40] Sunny Consolvo, Beverly Harrison, Ian Smith, Mike Y Chen, KatherineEveritt, Jon Froehlich, and James A Landay. Conducting in situ evaluationsfor and with ubiquitous computing technologies. International Journal ofHuman-Computer Interaction, 22(1-2):103–118, 2007. → pages 1, 14[41] Coorevits, Lynn and Coenen, Tanguy. The rise and fall of wearable fitnesstrackers. In Academy of Management, page 24, 2016. → pages 21[42] Kenneth Cox, Rebecca E Grinter, Stacie L Hibino, Lalita Jategaonkar Ja-gadeesan, and David Mantilla. A multi-modal natural language interface toan information visualization environment. International Journal of SpeechTechnology, 4(3):297–314, 2001. doi: 10.1023/A:1011368926479.→ pages7, 120, 123[43] Kate Crawford, Jessa Lingel, and Tero Karppi. Our metrics, ourselves: Ahundred years of self-tracking from the weight scale to the wrist wearable176Bibliographydevice. European Journal of Cultural Studies, 18(4-5):479–496, 2015. →pages 16[44] Patrick Crowley, Pascal Madeleine, and Nicolas Vuillerme. Effects of mo-bile phone use during walking: a review. Critical Reviews™ in Physical andRehabilitation Medicine, 28(1-2), 2016. → pages 17[45] Mihaly Csikszentmihalyi and Reed Larson. Validity and reliability of theexperience-sampling method. In Flow and the foundations of positive psy-chology, pages 35–54. Springer, 2014. doi: 10.1007/978-94-017-9088-8 3.→ pages 45[46] Andrea Cuadra, Shuran Li, Hansol Lee, Jason Cho, and Wendy Ju. My bad!repairing intelligent voice assistant errors improves interaction. Proceedingsof the ACM on Human-Computer Interaction, 5(CSCW1):1–24, 2021. →pages 88[47] Geoff Cumming and Sue Finch. Inference by eye: confidence intervals andhow to read pictures of data. American psychologist, 60(2):170, 2005. →pages 95[48] Rajkumar Darbar, Prasanta Kr Sen, and Debasis Samanta. Presstact: Sidepressure-based input for smartwatch interaction. In Proceedings of the 2016CHI Conference Extended Abstracts on Human Factors in Computing Sys-tems, pages 2431–2438, 2016. → pages 24[49] Statista-Empowering People With Data. Number of users of smartwatchesworldwide from 2019 to 2028, 2023. URL https://www.statista.com/forecasts/1314339/worldwide-users-of-smartwatches. On-line; accessed 2024-05-30. → pages 1, 19[50] Caroline de Cock, Madison Milne-Ives, Michelle Helena van Velthoven,Abrar Alturkistani, Ching Lam, and Edward Meinert. Effectiveness of con-versational agents (virtual assistants) in health care: Protocol for a sys-tematic review. JMIR Res Protoc, 9(3):e16934, Mar 2020. ISSN 1929-177Bibliography0748. doi: 10.2196/16934. URL https://www.researchprotocols.org/2020/3/e16934. → pages 85, 87[51] David Dearman, Melanie Kellar, and Khai N Truong. An examinationof daily information needs and sharing opportunities. In Proceedings ofthe 2008 ACM conference on Computer supported cooperative work, pages679–688, 2008. doi: 10.1145/1460563.1460668. → pages 45[52] Milad Dehghani. Exploring the motivational factors on continuous usageintention of smartwatches among actual users. Behaviour & InformationTechnology, 37(2):145–158, 2018. doi: 10.1080/0144929X.2018.1424246.→ pages 1, 20[53] Philip R. Doyle, Justin Edwards, Odile Dumbleton, Leigh Clark, and Ben-jamin R. Cowan. Mapping perceptions of humanness in intelligent personalassistant interaction. In Proceedings of the 21st International Conferenceon Human-Computer Interaction with Mobile Devices and Services, Mo-bileHCI ’19, New York, NY, USA, 2019. Association for Computing Ma-chinery. ISBN 9781450368254. doi: 10.1145/3338286.3340116. URLhttps://doi.org/10.1145/3338286.3340116. → pages 87[54] Pierre Dragicevic. Fair Statistical Communication in HCI, pages 291–330.Springer International Publishing, Cham, 2016. ISBN 978-3-319-26633-6.doi: 10.1007/978-3-319-26633-6 13. URL https://doi.org/10.1007/978-3-319-26633-6_13. → pages 95[55] Aarthi Easwara Moorthy and Kim-Phuong L Vu. Privacy concerns foruse of voice activated personal assistant in the public space. Interna-tional Journal of Human-Computer Interaction, 31(4):307–335, 2015. doi:10.1080/10447318.2014.986642. → pages 63, 155[56] Justin Edwards, Christian Janssen, Sandy Gould, and Benjamin R. Cowan.Eliciting spoken interruptions to inform proactive speech agent design. InProceedings of the 3rd Conference on Conversational User Interfaces, CUI’21, New York, NY, USA, 2021. Association for Computing Machinery.178BibliographyISBN 9781450389983. doi: 10.1145/3469595.3469618. URL https://doi.org/10.1145/3469595.3469618. → pages 88[57] Chris Elsden, David S Kirk, and Abigail C Durrant. A quantified past: To-ward design for remembering with personal informatics. Human–ComputerInteraction, 31(6):518–557, 2016. → pages 14[58] Daniel Epstein, Felicia Cordeiro, Elizabeth Bales, James Fogarty, and SeanMunson. Taming data complexity in lifelogs: exploring visual cuts of per-sonal informatics data. In Proceedings of the 2014 conference on Designinginteractive systems, pages 667–676, 2014. doi: 10.1145/2598510.2598558.→ pages 14, 22[59] Daniel A Epstein, An Ping, James Fogarty, and Sean A Munson. A livedinformatics model of personal informatics. In Proceedings of the 2015ACM international joint conference on pervasive and ubiquitous comput-ing, pages 731–742, 2015. → pages xx, 13, 14, 15, 22, 165[60] Daniel A Epstein, Clara Caldeira, Mayara Costa Figueiredo, Xi Lu, Lucas MSilva, Lucretia Williams, Jong Ho Lee, Qingyang Li, Simran Ahuja, QiuerChen, et al. Mapping and taking stock of the personal informatics literature.Proceedings of the ACM on Interactive, Mobile, Wearable and UbiquitousTechnologies, 4(4):1–38, 2020. → pages 15[61] Daniel A Epstein, Tanja Blascheck, Sheelagh Carpendale, RaimundDachselt, and Jo Vermeulen. Challenges in everyday use of mobile visu-alizations. In Mobile Data Visualization, pages 209–240. Chapman andHall/CRC, 2021. → pages 7[62] Andrey Esakia, D. Scott McCrickard, Samantha Harden, and MichaelHorning. Fitaware: Promoting group fitness awareness throughglance-able smartwatches. In Proceedings of the 2018 ACM International Con-ference on Supporting Group Work, GROUP ’18, page 178–183, NewYork, NY, USA, 2018. Association for Computing Machinery. ISBN179Bibliography9781450355629. doi: 10.1145/3148330.3148343. URL https://doi.org/10.1145/3148330.3148343. → pages 1[63] Clayton Feustel, Shyamak Aggarwal, Bongshin Lee, and Lauren Wilcox.People like me: Designing for reflection on aggregate cohort data in personalinformatics systems. Proceedings of the ACM on Interactive, Mobile, Wear-able and Ubiquitous Technologies, 2(3):1–21, 2018. doi: 10.1145/3264917.→ pages 164[64] Thomas Franke, Christiane Attig, and Daniel Wessel. A personal re-source for technology interaction: development and validation of the affin-ity for technology interaction (ati) scale. International Journal of Human–Computer Interaction, 35(6):456–467, 2019. → pages 93[65] Siwei Fu, Kai Xiong, Xiaodong Ge, Siliang Tang, Wei Chen, and YingcaiWu. Quda: natural language queries for visual data analytics. arXiv preprintarXiv:2005.03257, 2020. → pages 71[66] Jun Gong, Zheer Xu, Qifan Guo, Teddy Seyed, Xiang’Anthony’ Chen, Xi-aojun Bi, and Xing-Dong Yang. Wristext: One-handed text entry on smart-watch using wrist gestures. In Proceedings of the 2018 CHI Conference onHuman Factors in Computing Systems, pages 1–14, 2018. → pages 18[67] Samuel D Gosling, Peter J Rentfrow, and William B Swann Jr. A verybrief measure of the big-five personality domains. Journal of Research inpersonality, 37(6):504–528, 2003. → pages 93[68] Ru´ben Gouveia, Fa´bio Pereira, Evangelos Karapanos, Sean A Munson, andMarc Hassenzahl. Exploring the design space of glanceable feedback forphysical activity trackers. In Proceedings of the 2016 ACM internationaljoint conference on pervasive and ubiquitous computing, pages 144–155,2016. doi: 10.1145/2971648.2971754. → pages 2, 8, 17, 25, 28, 44[69] Ru´ben Gouveia, Evangelos Karapanos, and Marc Hassenzahl. Activitytracking in vivo. volume 2018-April. Association for Computing Machin-180Bibliographyery, 4 2018. ISBN 9781450356206. doi: 10.1145/3173574.3173936. →pages 16, 19, 22, 64, 124[70] Rebecca Gulotta, Jodi Forlizzi, Rayoung Yang, and Mark Wah Newman.Fostering engagement with personal informatics systems. In Proceedingsof the 2016 ACM Conference on Designing Interactive Systems, DIS ’16,page 286–300, New York, NY, USA, 2016. Association for Computing Ma-chinery. ISBN 9781450340311. doi: 10.1145/2901790.2901803. URLhttps://doi.org/10.1145/2901790.2901803. → pages 1, 2, 21, 23[71] Gabriel Haas, Michael Rietzler, Matt Jones, and Enrico Rukzio. Keep itshort: A comparison of voice assistants’ response behavior. In Proceedingsof the 2022 CHI Conference on Human Factors in Computing Systems, CHI’22, New York, NY, USA, 2022. Association for Computing Machinery.ISBN 9781450391573. doi: 10.1145/3491102.3517684. URL https://doi.org/10.1145/3491102.3517684. → pages 8, 85, 88, 89, 92, 93, 98,111, 113[72] Philipp Hacker, Andreas Engel, and Marco Mauer. Regulating chatgpt andother large generative ai models. In Proceedings of the 2023 ACM Con-ference on Fairness, Accountability, and Transparency, FAccT ’23, page1112–1123, New York, NY, USA, 2023. Association for Computing Ma-chinery. ISBN 9798400701924. doi: 10.1145/3593013.3594067. URLhttps://doi.org/10.1145/3593013.3594067. → pages 163[73] Jonna Ha¨kkila¨, Farnaz Vahabpour, Ashley Colley, Jani Va¨yrynen, and TimoKoskela. Design probes study on user perceptions of a smart glasses con-cept. In Proceedings of the 14th International Conference on Mobile andUbiquitous Multimedia, pages 223–233, 2015. doi: 10.1145/2836041.2836064. → pages 45[74] Tian Hao, Chongguang Bi, Guoliang Xing, Roxane Chan, and Linlin Tu.Mindfulwatch: A smartwatch-based system for real-time respiration moni-toring during meditation. Proceedings of the ACM on Interactive, Mobile,Wearable and Ubiquitous Technologies, 1(3):1–19, 2017. → pages 20181Bibliography[75] Gabriella M Harari, Nicholas D Lane, Rui Wang, Benjamin S Crosier, An-drew T Campbell, and Samuel D Gosling. Using smartphones to collectbehavioral data in psychological science: Opportunities, practical consider-ations, and challenges. Perspectives on Psychological Science, 11(6):838–854, 2016. doi: 10.1177/1745691616650285. → pages 6, 41, 44, 45[76] Javier Hernandez, Daniel McDuff, Christian Infante, Pattie Maes, KarenQuigley, and Rosalind Picard. Wearable esm: differences in the experiencesampling method across wearable devices. In Proceedings of the 18th in-ternational conference on human-computer interaction with mobile devicesand services, pages 195–205, 2016. doi: 10.1145/2935334.2935340. →pages 46[77] Juan David Hincapie´-Ramos, Xiang Guo, Paymahn Moghadasian, andPourang Irani. Consumed endurance: a metric to quantify arm fatigueof mid-air interactions. In Proceedings of the SIGCHI Conference onHuman Factors in Computing Systems, CHI ’14, page 1063–1072, NewYork, NY, USA, 2014. Association for Computing Machinery. ISBN9781450324731. doi: 10.1145/2556288.2557130. URL https://doi.org/10.1145/2556288.2557130. → pages 36[78] Jon-Chao Hong, Pei-Hsin Lin, and Pei-Chi Hsieh. The effect of consumerinnovativeness on perceived value and continuance intention to use smart-watch. Computers in Human Behavior, 67:264–272, 2017. → pages 21[79] Jonggi Hong, Seongkook Heo, Poika Isokoski, and Geehyuk Lee. Split-board: A simple split soft keyboard for wristwatch-sized touch screens.In Proceedings of the 33rd Annual ACM Conference on Human Factors inComputing Systems, pages 1233–1236, 2015. → pages 18[80] Tom Horak, Sriram Karthik Badam, Niklas Elmqvist, and RaimundDachselt. When david meets goliath: Combining smartwatches with a largevertical display for visual data exploration. In Proceedings of the 2018 CHIConference on Human Factors in Computing Systems, pages 1–13, 2018. →pages 24182Bibliography[81] Ya Huang, Kuanming Yao, Jiyu Li, Dengfeng Li, Huiling Jia, Yiming Liu,Chun Ki Yiu, Wooyoung Park, and Xinge Yu. Recent advances in multi-mode haptic feedback technologies towards wearable interfaces. MaterialsToday Physics, 22:100602, 2022. → pages 17[82] Ira E Hyman Jr, S Matthew Boss, Breanne M Wise, Kira E McKenzie, andJenna M Caggiano. Did you see the unicycling clown? inattentional blind-ness while walking and talking on a cell phone. Applied Cognitive Psychol-ogy, 24(5):597–607, 2010. → pages 17[83] Stephen Intille, Caitlin Haynes, Dharam Maniar, Aditya Ponnada, and JustinManjourides. µema: Microinteraction-based ecological momentary assess-ment (ema) using a smartwatch. In Proceedings of the 2016 ACM Inter-national Joint Conference on Pervasive and Ubiquitous Computing, pages1124–1128, 2016. doi: 10.1145/2971648.2971717. → pages 46[84] Alaul Islam, Anastasia Bezerianos, Bongshin Lee, Tanja Blascheck, and Pe-tra Isenberg. Visualizing information on watch faces: A survey with smart-watch users. In 2020 IEEE Visualization Conference (VIS), pages 156–160,2020. doi: 10.1109/VIS47514.2020.00038. → pages 24[85] Alaul Islam, Ranjini Aravind, Tanja Blascheck, Anastasia Bezerianos, andPetra Isenberg. Preferences and effectiveness of sleep data visualizations forsmartwatches and fitness bands. In CHI Conference on Human Factors inComputing Systems, pages 1–17, 2022. → pages 2, 8, 25, 28[86] Alaul Islam, Tanja Blascheck, and Petra Isenberg. Context Specific Visu-alizations on Smartwatches. Posters of the European Conference on Vi-sualization (EuroVis), June 2022. URL https://inria.hal.science/hal-03694026. Poster. → pages 8, 18, 65[87] Carlos Jensen, Colin Potts, and Christian Jensen. Privacy practices of in-ternet users: Self-reports versus observed behavior. International Journalof Human-Computer Studies, 63(1-2):203–227, 2005. doi: 10.1016/j.ijhcs.2005.04.019. → pages 64183Bibliography[88] Hayeon Jeong, Heepyung Kim, Rihun Kim, Uichin Lee, and Yong Jeong.Smartwatch wearing behavior analysis. Proceedings of the ACM on Interac-tive, Mobile, Wearable and Ubiquitous Technologies, 1:1–31, 9 2017. doi:10.1145/3131892. → pages 19[89] Ravinder Jerath, Mohammad Syam, and Shajia Ahmed. The future of stressmanagement: integration of smartwatches and hrv technology. Sensors, 23(17):7314, 2023. → pages 20[90] Tero Jokela, Jarno Ojala, and Thomas Olsson. A diary study on combiningmultiple information devices in everyday activities and tasks. In Proceed-ings of the 33rd annual ACM conference on human factors in computingsystems, pages 3903–3912, 2015. doi: 10.1145/2702123.2702211. → pages45[91] Simon L Jones. Exploring correlational information in aggregated quantifiedself data dashboards. In Adjunct Proceedings of the 2015 ACM InternationalJoint Conference on Pervasive and Ubiquitous Computing and Proceedingsof the 2015 ACM International Symposium on Wearable Computers, pages1075–1080, 2015. → pages 15[92] Simon L Jones and Ryan Kelly. Dealing with information overload in mul-tifaceted personal informatics systems. Human–Computer Interaction, 33(1):1–48, 2018. → pages 15[93] Haruka Kamachi, Tahera Hossain, Fuyuka Tokuyama, Anna Yokokubo, andGuillaume Lopez. Prediction of eating activity using smartwatch. In AdjunctProceedings of the 2021 ACM International Joint Conference on Pervasiveand Ubiquitous Computing and Proceedings of the 2021 ACM InternationalSymposium on Wearable Computers, pages 304–309, 2021. → pages 20[94] Jan-Frederik Kassel and Michael Rohs. Valletto: A multimodal interfacefor ubiquitous visual analytics. In Extended Abstracts of the 2018 CHI Con-ference on Human Factors in Computing Systems, pages 1–6, 2018. doi:10.1145/3170427.3188445. → pages 82184Bibliography[95] Frederic Kerber, Tobias Kiefer, Markus Lo¨chtefeld, and Antonio Kru¨ger.Investigating current techniques for opposite-hand smartwatch interaction.In Proceedings of the 19th International Conference on Human-ComputerInteraction with Mobile Devices and Services, pages 1–12, 2017. → pages24[96] Taewan Kim, Donghoon Shin, Young-Ho Kim, and Hwajung Hong. Di-arymate: Understanding user perceptions and experience in human-ai col-laboration for personal journaling. In Proceedings of the 2024 CHIConference on Human Factors in Computing Systems, CHI ’24, NewYork, NY, USA, 2024. Association for Computing Machinery. ISBN9798400703300. doi: 10.1145/3613904.3642693. URL https://doi.org/10.1145/3613904.3642693. → pages 164[97] Yoojung Kim, Bongshin Lee, and Eun Kyoung Choe. Investigating dataaccessibility of personal health apps. Journal of the American Medical In-formatics Association, 26(5):412–419, 03 2019. ISSN 1527-974X. doi: 10.1093/jamia/ocz003. URL https://doi.org/10.1093/jamia/ocz003.→ pages 127[98] Young-Ho Kim, Bongshin Lee, Arjun Srinivasan, and Eun Kyoung Choe.Data@hand: Fostering visual exploration of personal data on smartphones;leveraging speech and touch interaction. In Proceedings of the 2021CHI Conference on Human Factors in Computing Systems, CHI ’21, NewYork, NY, USA, 2021. Association for Computing Machinery. ISBN9781450380966. doi: 10.1145/3411764.3445421. URL https://doi.org/10.1145/3411764.3445421. → pages 14, 37, 63, 71, 82, 120, 123,155, 164[99] Young-Ho Kim, Diana Chou, Bongshin Lee, Margaret Danilovich, AmandaLazar, David E Conroy, Hernisa Kacorri, and Eun Kyoung Choe. Mymove:Facilitating older adults to collect in-situ activity labels on a smartwatch withspeech. In Proceedings of the 2022 CHI Conference on Human Factors inComputing Systems, pages 1–21, 2022. → pages 20, 46, 48, 65185Bibliography[100] Andreas M. Klein, Andreas Hinderks, Martin Schrepp, and Jo¨rgThomaschewski. Construction of ueq+ scales for voice quality: measur-ing user experience quality of voice interaction. In Proceedings of Men-sch Und Computer 2020, MuC ’20, page 1–5, New York, NY, USA, 2020.Association for Computing Machinery. ISBN 9781450375405. doi: 10.1145/3404983.3410003. URL https://doi.org/10.1145/3404983.3410003. → pages 92, 116[101] Rafal Kocielnik, Raina Langevin, James S. George, Shota Akenaga, AmeliaWang, Darwin P. Jones, Alexander Argyle, Callan Fockele, Layla Ander-son, Dennis T. Hsieh, Kabir Yadav, Herbert Duber, Gary Hsieh, and An-drea L. Hartzler. Can i talk to you about your social needs? under-standing preference for conversational user interface in health. In Pro-ceedings of the 3rd Conference on Conversational User Interfaces, CUI’21, New York, NY, USA, 2021. Association for Computing Machinery.ISBN 9781450389983. doi: 10.1145/3469595.3469599. URL https://doi.org/10.1145/3469595.3469599. → pages 87[102] Michelle Kwasny, Kelly Caine, Wendy A Rogers, and Arthur D Fisk. Pri-vacy and technology: folk definitions and perspectives. In CHI’08 Ex-tended Abstracts on Human Factors in Computing Systems, pages 3291–3296. 2008. doi: 10.1145/1358628.1358846. → pages 64[103] Ricardo Langner, Lonni Besanc¸on, Christopher Collins, Tim Dwyer, Pe-tra Isenberg, Tobias Isenberg, Bongshin Lee, Charles Perin, and ChristianTominski. An introduction to mobile data visualization. In Mobile DataVisualization, pages 1–32. Chapman and Hall/CRC, 2021. → pages 20[104] Liliana Laranjo, Adam G Dunn, Huong Ly Tong, Ahmet Baki Kocaballi,Jessica Chen, Rabia Bashir, Didi Surian, Blanca Gallego, Farah Magrabi,Annie Y S Lau, and Enrico Coiera. Conversational agents in healthcare:a systematic review. Journal of the American Medical Informatics Associ-ation, 25(9):1248–1258, 07 2018. ISSN 1527-974X. doi: 10.1093/jamia/ocy072. URL https://doi.org/10.1093/jamia/ocy072. → pages 87186Bibliography[105] Reed Larson and Mihaly Csikszentmihalyi. The experience samplingmethod. In Flow and the foundations of positive psychology, pages 21–34.Springer, 2014. doi: 10.1007/978-94-017-9088-8 2. → pages 45[106] Bongshin Lee, Eun Kyoung Choe, Petra Isenberg, Kim Marriott, and JohnStasko. Reaching broader audiences with data visualization. IEEE Com-puter Graphics and Applications, 40(2):82–90, 2020. doi: 10.1109/MCG.2020.2968244. URL https://doi.org/10.1109/MCG.2020.2968244.→ pages 27[107] Ian Li, Anind Dey, and Jodi Forlizzi. A stage-based model of per-sonal informatics systems. In Proceedings of the SIGCHI Conferenceon Human Factors in Computing Systems, CHI ’10, page 557–566, NewYork, NY, USA, 2010. Association for Computing Machinery. ISBN9781605589299. doi: 10.1145/1753326.1753409. URL https://doi.org/10.1145/1753326.1753409. → pages 13, 14, 15, 22, 64, 124[108] Ian Li, Anind K. Dey, and Jodi Forlizzi. Understanding my data, myself:Supporting self-reflection with ubicomp technologies. In Proceedings ofthe 13th International Conference on Ubiquitous Computing, UbiComp ’11,page 405–414, New York, NY, USA, 2011. Association for Computing Ma-chinery. ISBN 9781450306300. doi: 10.1145/2030112.2030166. URLhttps://doi.org/10.1145/2030112.2030166. → pages xviii, 14, 23,34, 45, 56, 57, 58, 64[109] Gesa Alena Linnemann and Regina Jucks. ‘can i trust the spoken dialoguesystem because it uses the same words as i do?’—influence of lexicallyaligned spoken dialogue systems on trustworthiness and user satisfaction.Interacting with Computers, 30(3):173–186, 2018. → pages 87[110] Li Liu, Yuxin Peng, Shu Wang, Ming Liu, and Zigang Huang.Complex activity recognition using time series pattern dictionarylearned from ubiquitous sensors. Information Sciences, 340-341:41–57, 2016. ISSN 0020-0255. doi: https://doi.org/10.1016/j.ins.2016.187Bibliography01.020. URL https://www.sciencedirect.com/science/article/pii/S0020025516000311. → pages 36[111] Xing Liu, Tianyu Chen, Feng Qian, Zhixiu Guo, Felix Xiaozhu Lin, Xi-aofeng Wang, and Kai Chen. Characterizing smartwatch usage in the wild.pages 385–398. Association for Computing Machinery, Inc, 6 2017. ISBN9781450349284. doi: 10.1145/3081333.3081351. → pages 19[112] Ramon Llamas. Global shipments of wearable devices saw modestgrowth in 2023 with improved demand expected in 2024 and beyond, ac-cording to idc, Mar 2024. URL https://www.idc.com/getdoc.jsp?containerId=prUS51975524. → pages 16[113] Kok Yin Long, Kamalanathan Shanmugam, and Muhammad Ehsan Rana.An evaluation of smartwatch contribution in improving human health. In2023 17th International Conference on Ubiquitous Information Manage-ment and Communication (IMCOM), pages 1–4. IEEE, 2023. → pages 20[114] Edward Loper and Steven Bird. Nltk: The natural language toolkit. arXivpreprint cs/0205028, 2002. → pages 70[115] Ewa Luger and Abigail Sellen. ”like having a really bad pa”: The gulfbetween user expectation and experience of conversational agents. In Pro-ceedings of the 2016 CHI Conference on Human Factors in Computing Sys-tems, CHI ’16, page 5286–5297, New York, NY, USA, 2016. Associationfor Computing Machinery. ISBN 9781450333627. doi: 10.1145/2858036.2858288. URL https://doi.org/10.1145/2858036.2858288. →pages 113[116] Yuhan Luo, Young-Ho Kim, Bongshin Lee, Naeemul Hassan, and Eun Ky-oung Choe. Foodscrap: Promoting rich data capture and reflective foodjournaling through speech input. In Designing Interactive Systems Confer-ence 2021, pages 606–618, 2021. doi: 10.1145/3461778.3462074. → pages63, 155188Bibliography[117] Yumei Luo, Lei Yang, Qiongwei Ye, and Qichen Liao. Effects of customiza-tion and personalization affordances on perceived value and continuance in-tention of smartwatch use. Technological Forecasting and Social Change,194:122752, 2023. → pages 21[118] Deborah Lupton. The quantified self. John Wiley & Sons, 2016. → pages13[119] Michal Luria, Rebecca Zheng, Bennett Huffman, Shuangni Huang, JohnZimmerman, and Jodi Forlizzi. Social boundaries for personal agents in theinterpersonal space of the home. In Proceedings of the 2020 CHI Con-ference on Human Factors in Computing Systems, CHI ’20, page 1–12,New York, NY, USA, 2020. Association for Computing Machinery. ISBN9781450367080. doi: 10.1145/3313831.3376311. URL https://doi.org/10.1145/3313831.3376311. → pages 87, 113[120] Kent Lyons. Visual parameters impacting reaction times on smartwatches.In Proceedings of the 18th International Conference on Human-ComputerInteraction with Mobile Devices and Services, pages 190–194, 2016. →pages 25[121] Raju Maharjan, Darius Adam Rohani, Per Bækgaard, Jakob Bardram, andKevin Doherty. Can we talk? design implications for the questionnaire-driven self-report of health and wellbeing via conversational agent. In Pro-ceedings of the 3rd Conference on Conversational User Interfaces, CUI’21, New York, NY, USA, 2021. Association for Computing Machinery.ISBN 9781450389983. doi: 10.1145/3469595.3469600. URL https://doi.org/10.1145/3469595.3469600. → pages 87[122] Suresh Malodia, Nazrul Islam, Puneet Kaur, and Amandeep Dhir. Why dopeople use artificial intelligence (ai)-enabled voice assistants. IEEE Trans-actions on Engineering Management, pages 1–15, 2021. doi: 10.1109/TEM.2021.3117884. → pages 71[123] Steve Mann. Wearable computing as means for personal empowerment. In189BibliographyProc. 3rd Int. Conf. on Wearable Computing (ICWC), pages 51–59, 1998.→ pages 16[124] Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel,Steven Bethard, and David McClosky. The stanford corenlp natural lan-guage processing toolkit. In Proceedings of 52nd annual meeting of the as-sociation for computational linguistics: system demonstrations, pages 55–60, 2014. → pages 70[125] Donald McMillan, Barry Brown, Airi Lampinen, Moira McGregor, EveHoggan, and Stefania Pizza. Situating wearables: Smartwatch use in con-text. volume 2017-May, pages 3582–3594. Association for Computing Ma-chinery, 5 2017. ISBN 9781450346559. doi: 10.1145/3025453.3025993.→ pages 18, 19[126] Saumya Misra, Rachana Adtani, Yuvraj Singh, Simran Singh, and Devan-shu Thakkar. Exploring the factors affecting behavioral intention to adoptwearable devices. Clinical Epidemiology and Global Health, 24:101428,2023. → pages 19[127] Joao Luis Zeni Montenegro, Cristiano Andre´ da Costa, and Rodrigo da RosaRighi. Survey of conversational agents in health. Expert Systems with Appli-cations, 129:56–67, 2019. ISSN 0957-4174. doi: https://doi.org/10.1016/j.eswa.2019.03.054. URL https://www.sciencedirect.com/science/article/pii/S0957417419302283. → pages 8, 85, 87[128] Jimmy Moore, Pascal Goffin, Jason Wiese, and Miriah Meyer. Exploringthe personal informatics analysis gap:“there’s a lot of bacon”. IEEE Trans-actions on Visualization and Computer Graphics, 28(1):96–106, 2021. →pages xx, 15[129] Jimmy Moore, Pascal Goffin, Jason Wiese, and Miriah Meyer. An interviewmethod for engaging personal data. Proceedings of the ACM on Interactive,Mobile, Wearable and Ubiquitous Technologies, 5(4):1–28, 2021. doi: 10.1145/3494964. → pages 66190Bibliography[130] Bobak Mortazavi, Ebrahim Nemati, Kristina VanderWall, Hector G Flores-Rodriguez, Jun Yu Jacinta Cai, Jessica Lucier, Arash Naeim, and MajidSarrafzadeh. Can smartwatches replace smartphones for posture tracking?Sensors, 15(10):26783–26800, 2015. → pages 20[131] Suhaib Mujahid, Giancarlo Sierra, Rabe Abdalkareem, Emad Shi-hab, and Weiyi Shang. An empirical study of android wear usercomplaints. Empirical Software Engineering, 23:3476–3502, 2018.doi: 10.1007/s10664-018-9615-8. URL https://doi.org/10.1007/s10664-018-9615-8. → pages 1, 2, 21, 35[132] Hendrik Muller, Jennifer Gove, and John Webb. Understanding tablet use:A multi-method exploration. In Proceedings of the 14th international con-ference on Human-computer interaction with mobile devices and services,pages 1–10, 2012. → pages 154[133] Sean A Munson, Hasan Cavusoglu, Larry Frisch, and Sidney Fels. So-ciotechnical challenges and progress in using social media for health. Jour-nal of medical Internet research, 15(10):e226, 2013. → pages 22[134] Ali Neshati, Yumiko Sakamoto, and Pourang Irani. Challenges in displayinghealth data on small smartwatch screens. In ITCH, pages 325–332, 2019.doi: 10.3233/978-1-61499-951-5-325. → pages 18, 23, 44[135] Ali Neshati, Yumiko Sakamoto, Launa Leboe-Mcgowan, Jason Leboe-McGowan, Marcos Serrano, and Pourang Irani. G-sparks: Glanceablesparklines on smartwatches. In 45th Conference on Graphics Interface (GI2019), pages 1–9, 2019. doi: 10.20380/GI2019.23. → pages 2, 8, 25, 28,44, 148[136] Ali Neshati, Fouad Alallah, Bradley Rey, Yumiko Sakamoto, Marcos Ser-rano, and Pourang Irani. Sf-lg: Space-filling line graphs for visualizinginterrelated time-series data on smartwatches. In Proceedings of the 23rdInternational Conference on Mobile Human-Computer Interaction, Mobile-HCI ’21, New York, NY, USA, 2021. Association for Computing Ma-191Bibliographychinery. ISBN 9781450383288. doi: 10.1145/3447526.3472040. URLhttps://doi.org/10.1145/3447526.3472040. → pages 2, 8, 24, 25,28, 44, 123[137] Ali Neshati, Bradley Rey, Ahmed Shariff Mohommed Faleel, Sandra Bar-dot, Celine Latulipe, and Pourang Irani. Bezelglide: Interacting withgraphs on smartwatches with minimal screen occlusion. In Proceedingsof the 2021 CHI Conference on Human Factors in Computing Systems,CHI ’21, New York, NY, USA, 2021. Association for Computing Ma-chinery. ISBN 9781450380966. doi: 10.1145/3411764.3445201. URLhttps://doi.org/10.1145/3411764.3445201. → pages 18, 24, 44[138] Jasmin Niess and Paweł W. Woz´niak. Supporting meaningful personalfitness: The tracker goal evolution model. volume 2018-April. Associ-ation for Computing Machinery, 4 2018. ISBN 9781450356206. doi:10.1145/3173574.3173745. → pages 23[139] Don Norman. The design of everyday things: Revised and expanded edition.Basic books, 2013. → pages 88[140] Jum Nunnally. Psychometric theory. (No Title), 1994. → pages 95[141] Masa Ogata and Michita Imai. Skinwatch: skin gesture interaction for smartwatch. In Proceedings of the 6th Augmented Human International Confer-ence, pages 21–24, 2015. → pages 24[142] Obi Ogbanufe and Natalie Gerhart. Watch it! factors driving continued fea-ture use of the smartwatch. International Journal of Human–Computer In-teraction, 34(11):999–1014, 2018. doi: 10.1080/10447318.2017.1404779.→ pages 21, 22[143] Kenneth Olmstead. Nearly half of americans use digital voice assistants,mostly on their smartphones. Pew Research Center, 12, 2017. → pages 87[144] Debajyoti Pal, Suree Funilkul, and Vajirasak Vanijja. The future of smart-watches: assessing the end-users’ continuous usage using an extended192Bibliographyexpectation-confirmation model. Universal Access in the Information So-ciety, 19:261–281, 2020. → pages 21[145] Debajyoti Pal, Anuchart Tassanaviboon, Chonlameth Arpnikanondt, andBorworn Papasratorn. Quality of experience of smart-wearables: Fromfitness-bands to smartwatches. IEEE Consumer Electronics Magazine, 9:49–53, 1 2020. ISSN 21622256. doi: 10.1109/MCE.2019.2941462. →pages 2, 23[146] Adam Palanica, Anirudh Thommandram, Andrew Lee, Michael Li, and YanFossat. Do you understand the words that are comin outta my mouth? voiceassistant comprehension of medication names. NPJ digital medicine, 2(1):55, 2019. → pages 88[147] Sabrina Paneels and Jonathan C Roberts. Review of designs for haptic datavisualization. IEEE Transactions on Haptics, 3(2):119–137, 2009. doi:10.1109/TOH.2009.44. → pages 8, 17[148] Stefania Pizza, Barry Brown, Donald McMillan, and Airi Lampinen. Smart-watch in vivo. pages 5456–5469. Association for Computing Machinery, 52016. ISBN 9781450333627. doi: 10.1145/2858036.2858522. → pages 1,17, 18, 19, 20, 22, 26, 46, 73, 147, 154[149] Bernd Ploderer, Wolfgang Reitberger, Harri Oinas-Kukkonen, and Julia vanGemert-Pijnen. Social interaction and reflection for behaviour change, 2014.→ pages 64, 125[150] Aditya Ponnada, Caitlin Haynes, Dharam Maniar, Justin Manjourides, andStephen Intille. Microinteraction ecological momentary assessment re-sponse rates: Effect of microinteractions or the smartwatch? Proceedingsof the ACM on interactive, mobile, wearable and ubiquitous technologies, 1(3):1–16, 2017. doi: 10.1145/3130957. → pages 46[151] Martin Porcheron, Joel E. Fischer, Stuart Reeves, and Sarah Sharples.Voice interfaces in everyday life. In Proceedings of the 2018 CHI Con-ference on Human Factors in Computing Systems, CHI ’18, page 1–12,193BibliographyNew York, NY, USA, 2018. Association for Computing Machinery. ISBN9781450356206. doi: 10.1145/3173574.3174214. URL https://doi.org/10.1145/3173574.3174214. → pages 17, 87[152] Amanda Purington, Jessie G. Taft, Shruti Sannon, Natalya N. Bazarova, andSamuel Hardman Taylor. ”alexa is my new bff”: Social roles, user sat-isfaction, and personification of the amazon echo. In Proceedings of the2017 CHI Conference Extended Abstracts on Human Factors in Comput-ing Systems, CHI EA ’17, page 2853–2859, New York, NY, USA, 2017.Association for Computing Machinery. ISBN 9781450346566. doi: 10.1145/3027063.3053246. URL https://doi.org/10.1145/3027063.3053246. → pages 87[153] Pallavi Rao Gadahad and Anirudha Joshi. Wearable activity trackers in man-aging routine health and fitness of indian older adults: Exploring barriersto usage. In Nordic Human-Computer Interaction Conference, NordiCHI’22, New York, NY, USA, 2022. Association for Computing Machinery.ISBN 9781450396998. doi: 10.1145/3546155.3546645. URL https://doi.org/10.1145/3546155.3546645. → pages 87[154] Reza Rawassizadeh, Chelsea Dobbins, Manouchehr Nourizadeh, ZahraGhamchili, and Michael Pazzani. A natural language query interface forsearching personal information on smartwatches. In 2017 IEEE Interna-tional Conference on Pervasive Computing and Communications Workshops(PerCom Workshops), pages 679–684, 2017. doi: 10.1109/PERCOMW.2017.7917645. → pages 6, 44, 47, 70, 71, 72, 81, 82, 162[155] Bradley Rey, Kening Zhu, Simon Tangi Perrault, Sandra Bardot, Ali Ne-shati, and Pourang Irani. Understanding and adapting bezel-to-bezel interac-tions for circular smartwatches in mobile and encumbered scenarios. Proc.ACM Hum.-Comput. Interact., 6(MHCI), sep 2022. doi: 10.1145/3546736.URL https://doi.org/10.1145/3546736. → pages 2, 18, 24[156] Bradley Rey, Charles-Olivier Dufresne-Camaro, and Pourang Irani. To-wards efficient interaction for personal health data queries on smartwatches.194BibliographyIn Proceedings of the 25th International Conference on Mobile Human-Computer Interaction, MobileHCI ’23 Companion, New York, NY, USA,2023. Association for Computing Machinery. ISBN 9781450399241.doi: 10.1145/3565066.3608700. URL https://doi.org/10.1145/3565066.3608700. → pages 68, 91[157] Bradley Rey, Bongshin Lee, Eun Kyoung Choe, and Pourang Irani. In-vestigating in-situ personal health data queries on smartwatches. Proc.ACM Interact. Mob. Wearable Ubiquitous Technol., 6(4), jan 2023. doi:10.1145/3569481. URL https://doi.org/10.1145/3569481. → pages43, 91, 93[158] Bradley Rey, Bongshin Lee, Eun Kyoung Choe, and Pourang Irani. Databit-ing: Lightweight, transient, and insight rich exploration of personal data.IEEE Computer Graphics and Applications, 44(2):65–72, 2024. doi:10.1109/MCG.2024.3353888. → pages 27[159] Bradley Rey, Yumiko Sakamoto, Jaisie Sin, and Pourang Irani. Under-standing user preferences of voice assistant answer structures for personalhealth data queries. In ACM Conversational User Interfaces 2024, CUI’24, New York, NY, USA, 2024. Association for Computing Machinery.ISBN 9798400705113. doi: 10.1145/3640794.3665552. URL https://doi.org/10.1145/3640794.3665552. → pages 84[160] Lionel Robert. The growing problem of humanizing robots. Robert,LP (2017). The Growing Problem of Humanizing Robots, InternationalRobotics & Automation Journal, 3(1), 2017. doi: 10.15406/iratj.2017.03.00043. → pages 113[161] John Rooksby, Mattias Rost, Alistair Morrison, and Matthew Chalmers. Per-sonal tracking as lived informatics. pages 1163–1172. Association for Com-puting Machinery, 2014. ISBN 9781450324731. doi: 10.1145/2556288.2557039. → pages 22, 23[162] Le´a Saviot, Frederik Brudy, and Steven Houben. Wristband. io: expanding195Bibliographyinput and output spaces of a smartwatch. In Proceedings of the 2017 CHIConference Extended Abstracts on Human Factors in Computing Systems,pages 2025–2033, 2017. → pages 24[163] Alexander Schiewe, Andrey Krekhov, Frederic Kerber, Florian Daiber, andJens Kru¨ger. A study on real-time visualizations during sports activitieson smartwatches. In 19th International Conference on Mobile and Ubiq-uitous Multimedia, MUM ’20, page 18–31, New York, NY, USA, 2020.Association for Computing Machinery. ISBN 9781450388702. doi: 10.1145/3428361.3428409. URL https://doi.org/10.1145/3428361.3428409. → pages 65[164] Bastian Schildbach and Enrico Rukzio. Investigating selection and read-ing performance on a mobile phone while walking. In Proceedings of the12th international conference on Human computer interaction with mobiledevices and services, pages 93–102, 2010. → pages 17[165] Steven Schirra and Frank R Bentley. ” it’s kind of like an extra screen formy phone” understanding everyday uses of consumer smart watches. InProceedings of the 33rd Annual ACM Conference Extended Abstracts onHuman Factors in Computing Systems, pages 2151–2156, 2015. → pages19[166] Anna Schneider and Rene Arnold. Wearables from head to toe: Are theyfriend or foe? an empirical landscaping of health and fitness wearables andapps in six countries to identify emerging policy challenges. An EmpiricalLandscaping of Health and Fitness Wearables and Apps in Six Countries toIdentify Emerging Policy Challenges (July 31, 2022), 2022. → pages 16[167] Martin Schrepp and Jo¨rg Thomaschewski. Design and validation of aframework for the creation of user experience questionnaires. Inter-national Journal of Interactive Multimedia and Artificial Intelligence, 5(7):88–95, 12/2019 2019. ISSN 1989-1660. doi: 10.9781/ijimai.2019.06.006. URL https://www.ijimai.org/journal/sites/default/196Bibliographyfiles/files/2019/06/ijimai20195_7_9_pdf_19082.pdf. → pages92[168] Suranga Seneviratne, Yining Hu, Tham Nguyen, Guohao Lan, Sara Khalifa,Kanchana Thilakarathna, Mahbub Hassan, and Aruna Seneviratne. A sur-vey of wearable devices and challenges. IEEE Communications Surveys &Tutorials, 19(4):2573–2620, 2017. → pages 19[169] Vidya Setlur, Sarah E. Battersby, Melanie Tory, Rich Gossweiler, and An-gel X. Chang. Eviza: A natural language interface for visual analysis. InProceedings of the 29th Annual Symposium on User Interface Softwareand Technology, UIST ’16, page 365–377, New York, NY, USA, 2016.Association for Computing Machinery. ISBN 9781450341899. doi: 10.1145/2984511.2984588. URL https://doi.org/10.1145/2984511.2984588. → pages 71[170] Ren-Jay Shei, Ian G Holder, Alicia S Oumsang, Brittni A Paris, andHunter L Paris. Wearable activity trackers–advanced technology or ad-vanced marketing? European Journal of Applied Physiology, 122(9):1975–1990, 2022. → pages 16[171] Grace Shin, Mohammad Hossein Jarrahi, Yu Fei, Amir Karami, Nicci Gafi-nowitz, Ahjung Byun, and Xiaopeng Lu. Wearable activity trackers, accu-racy, adoption, acceptance and health impact: A systematic literature review,5 2019. ISSN 15320464. → pages 23[172] Katie A Siek, Yvonne Rogers, and Kay H Connelly. Fat finger worries: howolder and younger users physically interact with pdas. In Human-ComputerInteraction-INTERACT 2005: IFIP TC13 International Conference, Rome,Italy, September 12-16, 2005. Proceedings 10, pages 267–280. Springer,2005. → pages 23[173] Carolin Siepmann and Pascal Kowalczuk. Understanding continued smart-watch usage: the role of emotional as well as health and fitness factors. Elec-197Bibliographytronic Markets, 31(4):795–809, 2021. doi: 10.1007/s12525-021-00458-3.→ pages 21[174] Adriana Silva and Bra´ulio Alturas. The benefits of the use of smartwatchesin promoting regular physical activity. In 2021 16th Iberian Conferenceon Information Systems and Technologies (CISTI), pages 1–6, 2021. doi:10.23919/CISTI52073.2021.9476473. → pages 20[175] Jaisie Sin, Dongqing Chen, Jalena G Threatt, Anna Gorham, and CosminMunteanu. Does alexa live up to the hype? contrasting expectations frommass media narratives and older adults’ hands-on experiences of voice in-terfaces. In Proceedings of the 4th Conference on Conversational User In-terfaces, pages 1–9, 2022. → pages 85, 88[176] Gaganpreet Singh, William Delamare, and Pourang Irani. D-swime: A de-sign space for smartwatch interaction techniques supporting mobility andencumbrance. In Proceedings of the 2018 CHI Conference on Human Fac-tors in Computing Systems, CHI ’18, page 1–13, New York, NY, USA, 2018.Association for Computing Machinery. ISBN 9781450356206. doi: 10.1145/3173574.3174208. URL https://doi.org/10.1145/3173574.3174208. → pages 18, 63[177] Timothy Sohn, Kevin A Li, William G Griswold, and James D Hollan. Adiary study of mobile information needs. In Proceedings of the sigchi con-ference on human factors in computing systems, pages 433–442, 2008. doi:10.1145/1357054.1357125. → pages 45[178] Inhwa Song, SoHyun Park, Sachin R. Pendse, Jessica Lee Schleider, Mun-mun De Choudhury, and Young-Ho Kim. Exploreself: Fostering user-drivenexploration and reflection on personal challenges with adaptive guidanceby large language models, 2024. URL https://arxiv.org/abs/2409.09662. → pages 164[179] Yang Song, Hao Ma, Hongning Wang, and Kuansan Wang. Exploring andexploiting user search behavior on mobile and tablet devices to improve198Bibliographysearch relevance. In Proceedings of the 22nd international conference onWorld Wide Web, pages 1201–1212, 2013. → pages 154[180] Katta Spiel, Fares Kayali, Louise Horvath, Michael Penkler, Sabine Harrer,Miguel Sicart, and Jessica Hammer. Fitter, happier, more productive? thenormative ontology of fitness trackers. In Extended abstracts of the 2018CHI conference on human factors in computing systems, pages 1–10, 2018.→ pages 22[181] Srinath Sridhar, Anders Markussen, Antti Oulasvirta, Christian Theobalt,and Sebastian Boring. Watchsense: On-and above-skin input sensingthrough a wearable depth sensor. In Proceedings of the 2017 CHI Con-ference on Human Factors in Computing Systems, pages 3891–3902, 2017.→ pages 24[182] Arjun Srinivasan and John Stasko. Natural language interfaces for dataanalysis with visualization: Considering what has and could be asked. InProceedings of the Eurographics/IEEE VGTC conference on visualization:Short papers, pages 55–59, 2017. doi: 10.2312/eurovisshort.20171133. →pages 7, 123[183] Arjun Srinivasan, Bongshin Lee, Nathalie Henry Riche, Steven M Drucker,and Ken Hinckley. Inchorus: Designing consistent multimodal interactionsfor data visualization on tablet devices. In Proceedings of the 2020 CHIconference on human factors in computing systems, pages 1–13, 2020. doi:10.1145/3313831.3376782. → pages 71, 72, 82[184] Arjun Srinivasan, Nikhila Nyapathy, Bongshin Lee, Steven M Drucker, andJohn Stasko. Collecting and characterizing natural language utterances forspecifying data visualizations. In Proceedings of the 2021 CHI Conferenceon Human Factors in Computing Systems, pages 1–10, 2021. doi: 10.1145/3411764.3445400. → pages 7, 54, 71, 72, 82[185] Thad Starner. How wearables worked their way into the mainstream, 2014.→ pages 15199Bibliography[186] Marc Strik, Sylvain Ploux, Daniel Weigel, Joske van der Zande, Anouk Vel-raeds, Hugo-Pierre Racine, F Daniel Ramirez, Michel Haı¨ssaguerre, andPierre Bordachar. The use of smartwatch electrocardiogram beyond arrhyth-mia detection. Trends in Cardiovascular Medicine, 2023. → pages 20[187] Melanie Tory and Vidya Setlur. Do what i mean, not what i say! design con-siderations for supporting intent and context in analytical conversation. In2019 IEEE Conference on Visual Analytics Science and Technology (VAST),pages 93–103, 2019. doi: 10.1109/VAST47406.2019.8986918. → pages 71[188] Milka Trajkova and Aqueasha Martin-Hammond. ”alexa is a toy”: exploringolder adults’ reasons for using, limiting, and abandoning echo. In Proceed-ings of the 2020 CHI conference on human factors in computing systems,pages 1–13, 2020. → pages 85[189] Edward R Tufte and Edward Rolf Tufte. Beautiful evidence, volume 1.Graphics Press Cheshire, CT, 2006. → pages 25[190] Niels Van Berkel, Denzil Ferreira, and Vassilis Kostakos. The experiencesampling method on mobile devices. ACM Computing Surveys (CSUR), 50(6):1–40, 2017. doi: 10.1145/3123988. → pages 45[191] Lev Velykoivanenko, Kavous Salehzadeh Niksirat, Noe´ Zufferey, MathiasHumbert, Ke´vin Huguenin, and Mauro Cherubini. Are those steps worthyour privacy? fitness-tracker users’ perceptions of privacy and utility. Proc.ACM Interact. Mob. Wearable Ubiquitous Technol., 5(4), dec 2022. doi:10.1145/3494960. URL https://doi.org/10.1145/3494960. → pages87, 114[192] Aku Visuri, Zhanna Sarsenbayeva, Niels Van Berkel, Jorge Goncalves,Reza Rawassizadeh, Vassilis Kostakos, and Denzil Ferreira. Quantify-ing sources and types of smartwatch usage sessions. volume 2017-May,pages 3569–3581. Association for Computing Machinery, 5 2017. ISBN9781450346559. doi: 10.1145/3025453.3025817. → pages 1, 18, 19, 20,21, 73, 147, 154200Bibliography[193] Aku Visuri, Niels van Berkel, Jorge Goncalves, Reza Rawassizadeh, Den-zil Ferreira, and Vassilis Kostakos. Understanding usage style transforma-tion during long-term smartwatch use. Personal and Ubiquitous Computing,2021. ISSN 16174917. doi: 10.1007/s00779-020-01511-2. → pages 19[194] Ryan Vooris, Matthew Blaszka, and Susan Purrington. Understanding thewearable fitness tracker revolution. International Journal of the Sociologyof Leisure, 2:421–437, 2019. → pages 22[195] Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, JonathanUesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, AtoosaKasirzadeh, Zac Kenton, Sasha Brown, Will Hawkins, Tom Stepleton,Courtney Biles, Abeba Birhane, Julia Haas, Laura Rimell, Lisa Anne Hen-dricks, William Isaac, Sean Legassick, Geoffrey Irving, and Iason Gabriel.Ethical and social risks of harm from language models, 2021. → pages 163[196] Peter West, Max Van Kleek, Richard Giordano, Mark J. Weal, and NigelShadbolt. Common barriers to the use of patient-generated data across clini-cal settings. In Proceedings of the 2018 CHI Conference on Human Factorsin Computing Systems, CHI ’18, page 1–13, New York, NY, USA, 2018.Association for Computing Machinery. ISBN 9781450356206. doi: 10.1145/3173574.3174058. URL https://doi.org/10.1145/3173574.3174058. → pages 112[197] Mark Whooley, Bernd Ploderer, and Kathleen Gray. On the integration ofself-tracking data amongst quantified self members. In Proceedings of the28th International BCS Human Computer Interaction Conference on HCI2014, pages 151–160. British Computer Society BCS, 2014. → pages 14[198] Jayme L Wilder, Devin Nadar, Nitin Gujral, Benjamin Ortiz, RobertStevens, Faye Holder-Niles, John Lee, and Jonathan M Gaffin. Pediatricianattitudes toward digital voice assistant technology use in clinical practice.Applied clinical informatics, 10(02):286–294, 2019. → pages 87[199] Pui Chung Wong, Kening Zhu, and Hongbo Fu. Fingert9: Leveraging201Bibliographythumb-to-finger interaction for same-side-hand text entry on smartwatches.In Proceedings of the 2018 CHI Conference on Human Factors in Comput-ing Systems, pages 1–10, 2018. → pages 18[200] Pui Chung Wong, Kening Zhu, Xing-Dong Yang, and Hongbo Fu. Ex-ploring eyes-free bezel-initiated swipe on round smartwatches. In Proceed-ings of the 2020 CHI Conference on Human Factors in Computing Systems,pages 1–11, 2020. → pages 18[201] Yiqi Xiao and Lu Liu. User-defined interactions for visual data explorationwith the combination of smartwatch and large display. IEEE Access, 2024.→ pages 24[202] Xinghui Yan, Shriti Raj, Bingjian Huang, Sun Young Park, and Mark WNewman. Toward lightweight in-situ self-reporting: An exploratory studyof alternative smartwatch interface designs in context. Proceedings of theACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 4(4):1–22, 2020. doi: 10.1145/3432212. → pages 46[203] Zhice Yang, Zeyu Wang, Jiansong Zhang, Chenyu Huang, and Qian Zhang.Wearables can afford: Light-weight indoor positioning with visible light. InProceedings of the 13th annual international conference on mobile systems,applications, and services, pages 317–330, 2015. → pages 19[204] Junhan Zhou, Yang Zhang, Gierad Laput, and Chris Harrison. Aurasense:enabling expressive around-smartwatch interactions with electric field sens-ing. In Proceedings of the 29th Annual Symposium on User Interface Soft-ware and Technology, pages 81–86, 2016. → pages 24[205] Leming Zhou, Jie Bao, Valerie Watzlaf, and Bambang Parmanto. Barriers toand facilitators of the use of mobile health apps from a security perspective:mixed-methods study. JMIR mHealth and uHealth, 7(4):e11223, 2019. doi:10.2196/11223. → pages 64[206] Tingting Zhu, Peter Watkinson, and David A Clifton. Smartwatch data help202Bibliographydetect covid-19. Nature biomedical engineering, 4(12):1125–1127, 2020.→ pages 20[207] Michael Zimmer, Priya Kumar, Jessica Vitak, Yuting Liao, and Katie Cham-berlain Kritikos. ‘there’s nothing really they can do with this informa-tion’: unpacking how users manage privacy boundaries for personal fitnessinformation. Information, Communication & Society, 23(7):1020–1037,2020. doi: 10.1080/1369118X.2018.1543442. URL https://doi.org/10.1080/1369118X.2018.1543442. → pages 87, 114203AppendicesAppendix A: Study TutorialsChapter 4 Tutorial SlidesThe following slides were used during the tutorial portion of the study con-ducted in Chapter 4. A live demo of the application was included during the tuto-rial.204Study Information, Setup, and TutorialStudy Overview● This study will have you: ○ Meet via Zoom for introduction, application setup, and a short demographic survey (~45 minutes) ○ Use a smartwatch application to simply report responses throughout your day (1 week / 7 days) ○ Meet via Zoom to do a one-on-one interview (~45 minutes) ● We will do everything in our power to provide a good experience for you during the study and hope for your involved participation throughout which will compensate up to $30 USD for your time by way of an Amazon gift card205OverviewOnline Consent - In Progress Study Overview Demographic Survey Download Smartwatch Application and Tutorial Practice, Final Thoughts, and Participate!Online ConsentPlease go to the link below to read and, if comfortable, sign the online consent form. If any questions arise, do not hesitate to ask.QR Code removed for anonymity purposes206OverviewOnline Consent - Done Study Overview - In Progress Demographic Survey Download Application and Tutorial Practice, Final Thoughts, and Participate!Study GoalsOur high level goal is to allow you to better explore and understand your personal health data on your smartwatch.207Study Goals• To explore what questions and/or commands people have of their personal health data on their smartwatch • To explore when and how people wish to interact with their personal health data on their smartwatchImportant Remarks● We are NOT collecting ANY of your personal health data or other metrics/data behind the scenes, such as your location; the application can NOT access this information ● We are NOT testing for specific responses or evaluating your performance in the application and throughout in the questionnaire and interview208Important Remarks● We are only COLLECTING what you provide within the application itself: 1. The question or command you have of your personal health data 2. Your current activity 3. If the question relates to your activity a. If so, the time within that activity ● We are INTERESTED in any questions or commands you have of your personal health data ● We are looking for HONEST responses throughout ● Your SAFETY when using the application is paramount, so please only respond if and when it is safe to do soOverviewOnline Consent - Done Study Overview - Done Demographic Survey - In Progress Download Application and Tutorial - Done Practice, Final Thoughts, and Participate!209Demographic SurveyPlease go to the link below to fill out the demographic survey. If any questions arise, do not hesitate to ask. I will let you know your participant ID to include in the survey.QR Code removed for anonymity purposesOverviewOnline Consent - Done Study Overview - Done Demographic Survey Download Application and Tutorial - In Progress Practice, Final Thoughts, and Participate!210Download the Application1. Install TestFlight on your iPhone ● This is Apple’s Beta software testing application and allows us to share our application with just you and not everyone in the worldDownload the Application2. Read through and accept Apple’s terms and conditions if comfortable ● We will not be accepting any crash report information, as such we will not collect any background information from your device211Download the Application3. Simply open the invitation email sent to your email addressActual screenshot removed for anonymity purposesDownload the Application4. This will open the TestFlight application. Now, simply click Install. You are good to go!Actual screenshot removed for anonymity purposes212Application First Steps● Let’s set up the following so that the application is ready to go for the study: 1. Watch face component ○ This will provide a home-screen reminder of the study for the next week 2. Notification times ○ Notifications will occur randomly throughout the provided start and end times to help remind you of the study in progressStudy ReminderApplication Usage● Please watch as I demonstrate the application. Please let me know if you have any questions!213Application Usage● When you open the application, you will see the home screen ○ This is the screen where you can record your questions about your data ○ Tap the “Tap to Record” button to speak or write your questionWhat question would you like to ask of your data?Press this button to record or write your questionApplication Usage● Here, you can select speech or writing to record your questionSelect either speech or writingTap ‘done’ when finished recording. This brings you to the next pageYour question appears hereYou can tap cancel at any time214Application Usage● Activity ○ This will ask you to record or write your current activityPress this button to record or write your answerWhat are you currently doing?Application Usage● Here, you can select speech or writing to record your questionSelect either speech or writingTap ‘done’ when finished recording. This brings you to the next pageYour question appears hereYou can tap cancel at any time215Application Usage● Question Related to your Activity ○ This question will ask if your question is related to your current activity (Yes) or not (No)Select whether your question is related to your current activityYou can go back at any time here Is your question related to your current activity?YesNoApplication Usage● Time in the Activity ○ If your question is related to your activity, we will ask where in the activity you currently areSelect a time within the activity here. This will bring you to the next screenYou can go back at any time here216Application Usage● Final Screen ○ This screen allows you to review your response before submittingSubmit your response here. Tapping this brings you back to the first home screenYou can go back at any time hereScroll (slide your finger on the screen or use the crown) to see more if the screen can not show everythingHere’s what you entered: Question: Test Activity: Test Related to Activity: Yes Time: DuringOverviewOnline Consent - Done Study Overview - Done Demographic Survey - Done Download Application and Tutorial - Done Practice, Final Thoughts, and Participate! - In Progress217Practice Responses1. First practice response ● Open the study application on your watch ● Record ‘This is my first response for the study’, select ‘done’ when done ● Record ‘Application Training’, select ‘done’ when done ● Select ‘Yes’ ● Select ‘During’ ● If happy, select ‘Submit’ ● Congratulations you just quickly and easily submitted your first response!Practice Responses2. Second practice response ● Open the study application on your watch ● Record ‘This is my second practice trial’, select ‘done’ when done ● Record ‘An activity not related to training’, select ‘done’ when done ● Select ‘No’ ● If happy, select ‘Submit’ ● Congratulations you just quickly and easily submitted your second response!218What We Are Interested In• We are interested in any questions or commands related to: • Personal health data that you capture/track on your smartwatch and wish to explore further directly on your smartwatch • We want your responses to ideally allow you to better explore and access your personal health dataWhat We Are Interested In• Possible categories of questions or commands you may be interested in include: • Current status of data • History of data • Goals, performance, and/or record based • Missed data or discrepancies • Contextual • Combinations and/or comparisons of data219What We Are Interested In• Don’t worry about current technology or applications • Express whatever questions or commands you have, however feels natural to you, that you want answers to! • Duplicates are okay, if they arise multiple times within the week • We just ask that you only provide responses when they are relevant to you • We do want the response to have an element of personal health data to it • The responses should be something you want to be able to access on your smartwatch • The questions/responses you provide will help create better technologies and applications for you in the future!What We Are Not Interested In• We aren’t interested in questions related to day-to-day smartwatch function • These would be typical questions you might ask of Siri: • What is the weather today? • What is the time? • Has my friend messaged me back? • Do I have any emails?220Final Thoughts• You can use the application any time you have a question or command of your data • You do not need to wait for a notification or specific time • Enjoy this opportunity to be able to ask all the questions and say commands of your data on your smartwatch you’ve ever wanted, or think would be helpful, for a better future experience for you! • Please wear your watch as you normally would throughout the study • Only submit responses when it is safe to do so Thanks!Thanks for your time in participating in our study, again if there are any questions or concerns throughout, please do not hesitate to contact us! 221Chapter 7 Tutorial SlidesThe following slides were used during the tutorial portion of the study con-ducted in Chapter 7. A live demo of the application was included during the tuto-rial.222Study Information, App Setup, and TutorialVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_SlidesThank you!Version 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides223Consent FormPlease use the provided qr code shared onscreen to complete the consent formVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_SlidesToday’s Intro Session Overview1. General Study Overview (~5 minutes)2. Demographic Survey (~10 minutes)3. Smartwatch App Installation (~5 minutes)4. Smartwatch App Tutorial (~10 minutes)Version 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides2241. General Study Overview• This study consists of three major components1. Introductory study session (currently taking place)• Learn about our study goals and how you can use speech and touch to explore your personal health data on the smartwatch 2. One-week use of our smartwatch application• Explore your personal health data on your smartwatch throughout daily life and recorded workouts 3. Final interview session• Interview about your experience using the smartwatch applicationVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides1. General Study Overview• Our high level goal is to allow you to better explore your personal health data directly on your smartwatch• We are interested in how using speech and touch can allow for explorationVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides2251. Important Remarks• We are NOT collecting and recording ANY of your personal health data, location, or audio recordings• We are NOT testing for specific responses or evaluating your performance throughout the study (i.e., you are NOT being tested and there are no right or wrong answers)Version 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides1. Important Remarks• We are INTERESTED in how speech and touch can be used to access your own personal health data on the smartwatch• We are INTERESTED in the questions you ask and when you ask these questionsVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides226Any questions so far?Version 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_SlidesToday’s Intro Session Overview1. General Study Overview (~5 minutes) ✅2. Demographic Survey (~10 minutes)3. Smartwatch App Installation (~5 minutes)4. Smartwatch App Tutorial (~10 minutes)Version 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides2272. Demographic SurveyPlease use the provided qr code shared onscreen to complete the surveyVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_SlidesToday’s Intro Session Overview1. General Study Overview (~5 minutes) ✅2. Demographic Survey (~10 minutes) ✅3. Smartwatch App Installation (~5 minutes)4. Smartwatch App Tutorial (~10 minutes)Version 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides2283. Smartwatch Application InstallationInstall TestFlight• Install TestFlight on your iPhone through the App Store • This is Apple’s Beta software testing application and allows us to share our application with just you and not everyone in the world • After installing TestFlight, open TestFlight and accept the terms and conditions• We will not be recording crash report information, as such are NOT collecting background data from your deviceVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides3. Smartwatch Application InstallationInstall Our Smartwatch Application• Scan the QR code below with your iPhone • This will open TestFlight and allow you to install our app on your Apple WatchVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides2293. Smartwatch Application InstallationInstall Our Smartwatch ApplicationVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_SlidesToday’s Intro Session Overview1. General Study Overview (~5 minutes) ✅2. Demographic Survey (~10 minutes) ✅3. Smartwatch App Installation (~5 minutes) ✅4. Smartwatch App Tutorial (~10 minutes)Version 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides2304. Smartwatch Application Tutorial• Why do a tutorial?• To familiarize you with our smartwatch application • To help you smoothly interact with and explore your data during the study • To answer any questions that you may have regarding the applicationVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides4. Smartwatch Application TutorialTutorial Shown On-Screen With Live ApplicationVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides231Exploration Times• You can explore data at three unique times within a workout• Before starting a workout – long press on the workout type that you are interested in exploring data for • During a workout – long press on a metric that has a blue dot beside it and that you’d like to explore, or swipe to the controls screen and use the microphone • After a workout – long press on a metric that has a blue dot beside it and that you’d like to exploreVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_SlidesExploration Modes• You can explore data in three unique ways• Value – Use “What Is/Was, Highlight, Find” in your question • Browse – Use “Show, Explore, Browse, View” in your question • Compare – Use “Compare, Combine, Contrast” in your questionVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides232Example Time Expressions“This month” June 1 – June 23, 2024“From March 1 to 20” March 1 – March 20, 2024“Summer 2020” June 1 – August 31, 2020 “Since March 1” March 1 – June 23, 2024Around last Christmas December 22 – December 28, 2024Last/past week/month/year… …September 2023 September 1 – September 30, 2023…Version 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_SlidesExample Activity Expressions“Last time” Your last tracked workout“Last workout” Your last tracked workout“Last 5 times” Your last 5 tracked workouts“Last 9 times” Your last 9 tracked workouts“Past 7 walks” Your last 7 walks“Past 4 bike rides” Your last 4 bike rides…Version 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides233Calculated Values• You can ask for the average, min, max, total of data• Average is calculated by default if nothing is mentioned. To get an average you can also use average, mean, overall • To get a min/max, you can use min/max, shortest/longest, slowest/fastest, etc. • To get a total of data added up, you can use total, how many/much/far/long, countVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_SlidesData• You can ask questions about the following data• Duration • Distance • Pace • Calories Burned • Heart RateVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides234A Couple Things To Note• DataWatch can not answer questions that make inferences of your data• Examples of questions not supported:• Is my pace faster than last week?• Have I walked farther than last month?• Am I on track to meet my goal?• Should I run 5km today?Version 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_SlidesA Couple Things To Note• DataWatch also does not support exploring multiple time periods• Examples of questions not supported:• Compare my pace from last month to April 2023• What was my total distance walked this January and last January?• Show my average heart rate this month to May last yearVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides235A Couple Things To Note• DataWatch does support the ability to view your collected data and/or get a calculated value from your collected data• Examples of questions supported:• Compare to my pace last year• Show me my distance walked in the last month• What is my average heart rate in the last five walks?Version 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_SlidesSome Other Questions to Get You Started• Here are some additional examples of questions that DataWatch supports:• How does this compare to my average in the last five workouts?• What was the farthest I walked in the last month?• How far have I walked in 2024?• Show me my fastest pace AprilVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides236One Caveat• To ask a question on the watch, your iPhone needs to be unlocked• This is a security feature from Apple which does not allow access to your collected health data unless your device is unlocked• We recommend simply holding your iPhone in your watch wearing hand, unlock your phone, and then interact on the watchVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_SlidesAny questions?Version 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides237Final Thoughts• We hope you enjoy this opportunity to further explore your personal health data on your smartwatch• Please use our application to record any workouts you do throughout the one-week study period• Please wear your smartwatch as you normally would throughout the study• Please only ask questions and/or commands when it is safe to do soVersion 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_SlidesThank you! Please reach out with any questions or concerns if needed.Version 1.0, June 29, 2023, H23-00805, Appendix_G_Intro_Slides238Appendix B: Dataset of Collected QueriesChapter 4 Collected DataThe following is the publicly available dataset captured from the smartwatchapplication during our study. The dataset has been made available, as linked inour publication at https://smartwatch-personal-health-data-queries.github.io.239activity query relatedToActivity timeEating lunch How much dancing do I need to do to burn 800 calories Yes DuringOn break Summary of my sleep cycles NoDancing During which song did I burn the most calories Yes AfterOn break Summary of my sleep cycles NoLying down Which activity had the highest calories burned per minute NoTaking a break from workHow many times was I awake last night and for how long NoFinished taking a practice final examWhen during the last two hours with my highest heart rate Yes AfterDancing Compare calories burned this dance session to past sessions Yes After240About to go on a walk How many more minutes of exercise do I need to reach my exercise goal for the day Yes BeforeWorkingBased on my activity during the month of October how should I adjust my three goals for the month of NovemberNoAbout to go on a walk How many steps have I taken so far today Yes BeforeSitting down Did I close all three of my rings for the day NoI'm sitting Did I close all of my rings today NoSitting How many steps did I take today NoJust finished a workout How many times have I worked out this week Yes AfterOn a walk How many steps have I taken so far today Yes DuringWorkingHow many times did I work out in the past week and how does this number compare to that the number of times I worked out the week before thatNo241Working How many times did I close all three rings during the month of October NoAt work how many days left to my period NoI am doing a little bit of work out compare the heart rate of my friends and mine Yes DuringDoing homework I would like to check a trend is my sleep in the past seven days NoPreparing dinner What is my current calorie intake Yes BeforeGoing to work Should I go for a run today? NoGetting ready for work Am I over or under my calorie goal at the moment NoGoing out How many calories that I login for breakfast NoAt work How much uninterrupted sleep did l get last night? No242Working out What is my current bloodpressure Yes DuringBus Did I complete my water goal yesterday? Nostudying in library How many calories did I burn in the last 4 hours? NoBrushing teeth What's my average sleeping time this past week NoGetting out of bed Do I need more sleep? Yes AfterArriving home How many steps left do I need to complete my goal NoGoing to the gym When was the last time I ran and what was the time Yes BeforeCleaning up after workout What was my peak heart rate during my workout Yes AfterBrushing teeth Did I reach my water intake goal? No243Walking home How many steps did I take in the past seven hours Yes DuringIn bed Was my sleep longer than last time? Yes BeforeGetting ready to sit down for the eveningWhat type of workout do I burn the most calories per minute historically NoOut on a walk trudging through the snowWhat is my average walking pace in the winter versus the summer Yes DuringLaying on the couch watching a documentary about how efficient coyotes gate isHas my heart rate during walking decreased since I started tracking walks NoJust woke up from a nap at 10:57 and didn't have my standing minute yetWhen am I most and least likely to get and miss my standing minutes NoSitting in the car at 6:30 PM with several hundred calories left clues my move ringWhat kind of workout do I usually do if I have 200 or more calories to close my moving ring NoJust returned home from a short walkWhat is my weekly average walking kilometres with the walking workout? Yes AfterHockey game intermissionDo I burn more calories when I work out in the morning afternoon or evening? No244Sitting in the meeting with BradleyIs my running pace slower in the days following a strength training workout? NoFinal study meeting with BradleyIs my first 2 km of walking pace faster or slower than my last two NoGetting ready to go to the gym Give me a report for my readiness for activity Yes BeforeGym What's my blood sugar level Yes AfterGetting ready to go to the gym Why is my Sleep report from last night so bad NoGym Give me a suggested work out based on my readiness score Yes BeforeDinner preparations If I have tacos tonight how will that affect my sleep and readiness score for tomorrow NoGrocery shopping How will not going to the gym during the holiday weekend affect me NoGrocery shopping for ThanksgivingHow has the stress from the last two hours of grocery shopping for Thanksgiving affected my readiness score for todayYes During245Outdoor walk How is the air quality affecting my walk Yes DuringGym How did today's gym session compared to the last one given that I was sick this time Yes AfterWalking to the car Is my heart rate slower or faster than normal Yes DuringDrinking lots of beer Does drinking beer impact my sleep Yes BeforeLaying on the sofa sickDoes my heart rate look any different compared to the average Yes DuringJust finished exercising How many steps did I take my outdoor walk Yes AfterFinished walking How many steps did I do on that short walk Yes AfterSitting When is my period going to start NoStanding What is my resting heart rate No246Sitting How many hours did I sit yesterday Yes DuringSitting at my desk How much of my day have I spent sitting Yes DuringGoing to bed What was my average resting heart rate today NoWorking What has my resting heart rate been since I put my watch on Yes DuringFinished mopping How many calories did I burn mopping Yes AfterStanding up How many hours was I sitting for today NoSitting How many steps have I taken this week NoWalking How many steps did I just take Yes AfterStanding How many calories did I burn today so far No247Getting ready for bed How many hours did I spend sitting today NoGetting ready for bed What is my average step count per day NoRunning How many steps did I take during my run Yes AfterGetting ready for bed What was my peak heart rate of the day? NoGetting ready to exercise How many runs have I completed thus far in 2021 Yes BeforeEating lunch after run Show me a graph of my runs both time and distance in 2021 Yes AfterJust finished my work outHow many times and for how long did my heart rate rise above 130 today? Yes AfterHomework How many miles have I accumulated through walking, running, biking over the course of this year NoMaking breakfast Over the last three months how many days have I beat Marianne on step count No248Walking Tell me when I reach a nine minute walking pace Yes DuringStanding What is the average time I closed my stand ring during last year Yes DuringI just got up Tell me the specific days of the week in the last three months where my weight tended to trend up or down NoMaking breakfast Over the last year when I breathe when directed by my watch does it lower my heart rate NoGetting ready to go for a walk Tell me when I reach a nine minute kilometre pace Yes BeforeGetting dressedWhat has Marianne's most active day of the week been over the last three months and what has my most active day of the week been on averageNoGetting dressed How many times have I tested for signs of atrial fib relation since I got my watch NoHaving lunch What is the average time I start tracking my steps in the day over the last year NoWatching the news How many times have I not closed my rings in the last year No249I just got up Is my heart rate consistently the same when I first put my watch on in the morning Yes DuringI just finished taking an ECG reading What have my ECG readings been sense August 1 Yes AfterWalking Tell me when I reach a nine minute per kilometre pace Yes DuringReading a notification about Marianne on our fitness challengeIs there a day of the week I am more likely to beat Marianne in our fitness challenge Yes DuringI am leaving on a solo walkDoes my walking pace change when I walk with someone else Yes BeforeWalking How much has my pace fluctuated during my walk Yes DuringCooking breakfast What are the average steps I take on a Sunday NoYardwork What was my highest heart rate in the last hour Yes AfterWeighing myself What was my average weight in October Yes After250Walking What was my average resting heart rate in October NoWatching TV How many steps do I have left to make my goal today NoStarting my day Show me a chart of my weight loss over the last two weeks, my steps, and my caloric intake NoAbout to eat Can I eat squash soup and two pieces of toast and still be below my calorie budget Yes BeforeWaking up starting my day How much sleep did I get last night Yes AfterAbout to lay down How long have I been lying down today since first waking up Yes BeforeGetting my day startedShow me a graph comparing my caloric intake over the last week NoWorking How many calories do I have left on my budget for the day NoWorking Show me the history of my weight over the last two years No251Working How long does it normally take me to walk 5000 steps NoAbout to go walking How many steps do I have left to do today Yes BeforeWorking How many calories do I have left in my bank for today? NoDriving How many steps do I have left to make my goal NoPlanning my next run while working When is a good time for me to run after the rain stops Nobreakfast what is the impact of rain in my running performance Noworking when do I need to exercise to improve my vo2 max Nowaking up what is the rowing equivalent for my running NoRowing What is my stroke rate while rowing Yes During252Having breakfast How can I improve my resting heart rate NoEating What are the effects of BMI Yes Duringlooking at HRV indicator what does heart rate variability mean Yes DuringWorking What is my performance compared to same time last year NoOutdoor walk How long do I need to run three times per week to achieve the November challenge Yes Duringpreparing to sleep what helps me get better sleep? Yes Beforewaking up why am I sleeping bad lately? Yes AfterHaving breakfast How can I increase my VO2 Max Nohaving lunch what is the impact of my sleep in my running? No253Watching a video Have I stood up this hour Nohaving breakfast compare my running stats with the same time last year NoWeight lifting Am I in the right heart rate range for this type of work out Yes DuringOutdoor walk How does my walking pace change depending on the length of a walk Yes DuringWalking outside How long does it take after a walk to get back to resting heart rate Yes AfterWeightlifting Is my work out better at my home gym or commercial gym Yes AfterWeightlifting Does weightlifting focussing on different muscle groups affect my heart rate Yes AfterStationary bike How long does it take for my heart to reach optimum heart rate during a cycling work out Yes DuringSitting How much lower is my activity today because it's Friday Yes During254Working out Compare the heart rate of this work out to last work out Yes AfterWaking up and going to the gym What workout do I have planned for today Yes BeforeLaundry How many kilometers do I need to walk to get 10,000 steps NoMaking lunch What was my best kilometer during my run NoDriving What was my fastest kilometer in my run NoDriving When was the last time I did a full 5K NoDriving passenger Show me a graph of my 5 km outdoor runs over time NoJust out of the shower How many steps did I get during my dance workout today NoJust out of the shower Show me a history of all my dance workouts No255Going for a walk How long should I walk to get 10,000 steps today Yes BeforeWatching TV When should I be expecting my period NoDance workout How many more steps do I need to get to 10,000 Yes BeforeGetting ready to go for a walk Give me a 2 km walk route starting from home Yes BeforeBack from walk How many steps did I get during that 2 km walk Yes AfterEnd of work out How many calories did I burn that work out Yes AfterJust getting off the treadmill How many steps am I at today Yes AfterOn a zoom meeting How many calories have I eaten today NoDance workout How many active minutes am I at Yes During256Eating lunch How many steps have I taken today NoWalking What's my average walking pace per kilometer Yes DuringEating supper How many minutes of aerobic activity would it take me to burn 200 cal? NoWorking out What's my daily calories burned this week? Yes AfterGoing for a walk What's the fastest that I walked 2 km outdoors? Yes BeforeWalking Show me a graph of my average walking speed during outdoor walk Yes AfterWaking up How many hours of deep sleep did I get last night? Yes AfterWaking up Show me a graph of my sleep quality from last night Yes AfterEating breakfast What's my blood oxygen level when I'm sleeping? No257At the park What's my average steps per day? Yes DuringEating supper What are my calories burned on weekends versus weekdays? NoWorking out Compare max heart rate from this work out to Monday's work out. Yes AfterWork out What are average calories burned for HIIT work out? Yes AfterWorking out What are my average non-active calories burned per day? Yes AfterWork out Alert me when my heart rate gets back to my average resting heartbeat Yes AfterI am at work How long on average does it take me to fall asleep? NoWorking How much time do I spend sitting on average Monday to Friday? Yes DuringWorking What's my average calories burned in the morning versus afternoon versus evening? No258Playing with kids. What was my calories burned in the last 30 minutes? Yes DuringWorking at my desk How many calories have a burned since 8 AM? NoWalking What is my average walking pace? Yes AfterWalking Compare walking pace September and October Yes AfterSitting at my desk What is my blood sugar at NoGoing to the gym Do I need to eat something before I go to the gym Yes BeforeWalking home from the gymShould I stop trying for linear growth based on my last workout Yes AfterSitting home What are the factors most likely impeding my squat growth NoSitting on my desk Based on my current biometrics when will be the best time for me to work out today No259Dead lifting How many reps was my last set Yes DuringWalking to the gym What's the best exercise for me today Yes BeforeSquat What was my RpE perceived exertion Yes DuringWalking home What is my projected one rep maximum NoSitting at home About how long will it take me to reach my squat deadlifts and benchpress goals NoWatching TV How many calories did I just eat NoSitting around being tired Is my cycle affecting my sleep Yes DuringTrying to sleep Compare data points from nutrition and heartrate and sleep NoNothing Compare cycle data from today to the same day in my last cycle No260Reviewing the days activities Compare data points from different types of activity Yes AfterReviewing the day of activityCompare data points across different activities within a cycle of a specific number of days Yes AfterThinking about what to make for dinnerWhat is the recommended calorie intake for dinner today Yes BeforeJust got home relaxingWas the 1st km of my hike faster than the last kilometre today NoWalking home from the gym How many calories did I burn playing badminton today Yes AfterWalking home from the gym Show me my heart rate chart from today's gym session Yes AfterWalking How many steps did I do yesterday versus today Yes DuringWeighing myself Show me my weight loss trends in the past month Yes DuringThinking about what to make for breakfast What is my calorie budget for today Yes Before261Reflecting on my fitness journey and today's goalsHow many calories were burned in today's work out compared to yesterday NoWalking home from the gymOn average was my heart rate lower during today's run compared to yesterday Yes AfterEating a banana How many calories is in a medium size banana Yes DuringDoing homework How many steps should I take today and is it less than yesterday NoWeighing myself Show me my bodyweight trends for this month Yes DuringJournalling What was my average heart rate during weight training today NoJournalling How many calories over budget am I today NoJust got home relaxing now How fast did I finish my 1st km on my hike today No262
Thesis/Dissertation
eng
Computer Science
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Irani, Pourang
Databiting : enabling lightweight, transient, and in-situ exploration of personal fitness data on smartwatches
Text