Advancing the Understanding of Android Unlockingand UsagebyLina QiuBEng in Computer Science and Engineering, University of Electronic Science andTechnology of China, 2015A THESIS SUBMITTED IN PARTIAL FULFILLMENTOF THE REQUIREMENTS FOR THE DEGREE OFMaster of Applied ScienceinTHE FACULTY OF GRADUATE AND POSTDOCTORALSTUDIES(Electrical and Computer Engineering)The University of British Columbia(Vancouver)May 2018© Lina Qiu, 2018The following individuals certify that they have read, and recommend to the Facultyof Graduate and Postdoctoral Studies for acceptance, a thesis/dissertation entitled:ADVANCING THE UNDERSTANDING OF ANDROID UNLOCKING AND USAGEsubmitted by LINA QIU in partial fulfillment of the requirements forthe degree of MASTER OF APPLIED SCIENCEin ELECTRICAL AND COMPUTER ENGINEERINGExamining Committee:KONSTANTIN BEZNOSOV, ELECTRICAL AND COMPUTER ENGINEERINGCo-supervisorALEXANDER DE LUCA, RESEARCH SCIENTIST OF GOOGLE, ZURICHCo-supervisorSupervisory Committee MemberJULIA RUBIN, ELECTRICAL AND COMPUTER ENGINEERINGAdditional ExaminerAdditional Supervisory Committee Members:Supervisory Committee MemberSupervisory Committee MemberiiAbstractResearch efforts have been made towards creating mobile authentication systemsto better serve users’ concerns regarding usability and security. While previousworks have revealed real world smartphone authentication usage patterns, severalaspects still need to be explored. In this research, we fill some of these knowledgegaps, including how age influences smartphone use. To this end, we performeda two-month long field study on a diverse North American study pool (N = 137).We examined how smartphone usage correlates with users’ ages, their choice ofunlocking mechanisms (e.g., PIN vs. Pattern) and the types of activities theyundertook while unlocking their phones. Study results reveal that there are indeedsignificant differences across age and unlocking mechanisms. For instance, olderparticipants interacted significantly less-frequently with their devices, and for asignificantly shorter amount of time each day. Fingerprint users had significantlymore device sessions than other mechanism groups. In addition, we also observedthat most participants regularly shared their devices with others, while they alsolikely underestimated the sensitivity of the data stored on them. Overall, these ob-servations provide important messages for designers and developers of smartphoneauthentication systems.iiiLay SummaryThis thesis presents the results of a research project to create a better understandingof real world smartphone authentication usage patterns, including how age corre-lates with smartphone use. To conduct this research, we performed a two-monthfield study on a diverse North American study pool (N = 137). We examined howsmartphone usage correlates with users’ age, their choices of unlocking mecha-nisms (e.g., PIN vs. Fingerprint) and the types of activities they undertook duringthe unlocking process. Study results show that there are indeed significant dif-ferences across age and unlocking mechanisms. For instance, older age groupsinteracted with their devices significantly less frequently, and for a significantlyshorter amount of time each day. Fingerprint users had significantly more de-vice sessions than PIN and Swipe/None users. In addition, we also observed thatmost participants regularly shared their devices with others, while they also likelyunderestimated the sensitivity of the data stored on their devices.ivPrefaceThis research was the product of a fruitful collaboration between the author ofthe thesis and the following people: Ildar Muslukhov and Konstantin Beznosov(supervisor) from the University of British Columbia (UBC), and Alexander DeLuca (co-supervisor) from Google, Zurich. The idea of the project was built uponthe following previous work:A. Mahfouz, I. Muslukhov, and K. Beznosov. Android users in the wild: Theirauthentication and usage behavior. Special Issue on Mobile Security, Privacy andForensics, Volume 32, Pages 50-61, Elsevier, October 2016.The work presented herein is prepared to submit to CHI Conference on HumanFactors in Computing Systems, 2019 and is in preparation:L. Qiu, A. De Luca, I. Muslukhov, and K. Beznosov. Lock of Ages: TowardsUnderstanding Age Influence on Smartphone Authentication. In preparation.I was responsible for designing and implementing the study application (app),recruiting participants, collecting and analyzing the data, and writing the manuscript,while Ildar Muslukhov helped set up the backend data collection server. All co-authors actively participated in discussions on the app design, participant recruit-ment, data collection and analysis, and paper writing process. For this study, Iobtained ethics approval from the Behavioural Research Ethics Board (BREB) atUBC. Approval H16-00343, titled “Android Usage”.vTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xAcknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.1 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.1.1 Locks/Unlocks/Failed Unlocks . . . . . . . . . . . . . . . 10vi4.1.2 User Sessions . . . . . . . . . . . . . . . . . . . . . . . . 114.1.3 Activities . . . . . . . . . . . . . . . . . . . . . . . . . . 114.1.4 Device Sharing Survey . . . . . . . . . . . . . . . . . . . 114.1.5 Contextual Survey . . . . . . . . . . . . . . . . . . . . . 124.2 Data Transmission . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186.1 (Un)locking Behaviour and Age . . . . . . . . . . . . . . . . . . 186.1.1 Session Lengths . . . . . . . . . . . . . . . . . . . . . . . 206.1.2 Number of Sessions Per Day . . . . . . . . . . . . . . . . 216.1.3 Daily Usage Lengths . . . . . . . . . . . . . . . . . . . . 236.1.4 Error Rates . . . . . . . . . . . . . . . . . . . . . . . . . 246.1.5 Auto/Manual Locks . . . . . . . . . . . . . . . . . . . . . 256.2 Activity While Unlocking . . . . . . . . . . . . . . . . . . . . . . 276.2.1 Error Rates While At Still vs On the Move . . . . . . . . . 306.2.2 How Age Predicts Whether Authentication is Used WhileAt Still vs On the Move . . . . . . . . . . . . . . . . . . . 316.2.3 How Age and Still/Move Predict Lock Types Used ForAuthentication . . . . . . . . . . . . . . . . . . . . . . . 316.3 Device Sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . 336.4 App Usages and Sensitivities . . . . . . . . . . . . . . . . . . . . 357 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387.1 Age Makes a Difference . . . . . . . . . . . . . . . . . . . . . . . 387.2 No One-Fits-All Solution . . . . . . . . . . . . . . . . . . . . . . 397.3 Design and Evaluate for Movement . . . . . . . . . . . . . . . . . 407.4 Device Sharing is Common . . . . . . . . . . . . . . . . . . . . . 407.5 Most Sessions Have Low Sensitivity . . . . . . . . . . . . . . . . 418 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43vii9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47A Survey Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50A.1 Device Sharing Survey . . . . . . . . . . . . . . . . . . . . . . . 50A.2 Contextual Survey . . . . . . . . . . . . . . . . . . . . . . . . . . 51B Data Collection Visualization . . . . . . . . . . . . . . . . . . . . . . 52viiiList of TablesTable 5.1 Participant demographics, N = 137 . . . . . . . . . . . . . . . 17Table 6.1 Overview of the usage patterns that were and were not signifi-cantly influenced by age. . . . . . . . . . . . . . . . . . . . . . 19Table 6.2 The number of sessions per age group. . . . . . . . . . . . . . 22Table 6.3 Binary logistic regression model: whether authentication is usedwhile at still vs on the move by age. An * denotes significance(p < .05). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31Table 6.4 Multinomial logistic regression model: how age and still/movestates predict lock type used for authentication. An * denotessignificance (p < .05). . . . . . . . . . . . . . . . . . . . . . . 32Table 6.5 Top 5 most-used apps and the corresponding averaged sensitiv-ity ratings. Sensitivity level has been collected using a 5-pointLikert scale (1: Not at all, 2: Slightly, 3: Moderately, 4: Very, 5:Extremely.) with question: “Please rate for each app on the list:If this person were watching your screen, and you were usingthis app right now, how much would it affect your privacy?” . . 35ixList of FiguresFigure 6.1 Unlocking mechanism distribution among age groups. Num-bers below each age group represent the total number of partic-ipants from that group. . . . . . . . . . . . . . . . . . . . . . 19Figure 6.2 Average session length per age group (minutes), N = 136. Thered squares represent the means. . . . . . . . . . . . . . . . . 20Figure 6.3 Average session length per unlocking mechanism (minutes),N = 133 (the three password participants were removed fromthe figure, as the sample size was too small). The red squaresrepresent the means. . . . . . . . . . . . . . . . . . . . . . . 21Figure 6.4 Average number of sessions per age group, N = 137. The redsquares represent the means. . . . . . . . . . . . . . . . . . . 22Figure 6.5 Average number of sessions per unlocking mechanism, N =134 (the three password participants were removed from thefigure, as the sample size was too small). The red squaresrepresent the means. . . . . . . . . . . . . . . . . . . . . . . 23Figure 6.6 Average daily usage length per age group (minutes), N = 136.The red squares represent the means. . . . . . . . . . . . . . . 24Figure 6.7 Average daily usage length per unlocking mechanism (min-utes), N = 133 (the three password participants were removedfrom the figure, as the sample size was too small). The redsquares represent the means. . . . . . . . . . . . . . . . . . . 25xFigure 6.8 Error rates distribution among age groups, N = 87 (the threepassword participants were removed from the figure, as thesample size was too small). Numbers below each age grouprepresent the total number of participants from that group. . . 26Figure 6.9 Distribution of locking types among age groups, N = 46. Num-bers below each age group represent the total number of partic-ipants from that group. . . . . . . . . . . . . . . . . . . . . . 27Figure 6.10 Distribution of locking types among unlocking mechanisms,N = 45 (the only one password participant in this analysis wasremoved from the figure, as the sample size was too small).Numbers below each unlocking mechanism represent the totalnumber of participants from that group. . . . . . . . . . . . . 28Figure 6.11 Distribution of activities during unlock, N = 119. . . . . . . . 29Figure 6.12 Distribution of unlocks that happened while at still/move sortedby age groups, N = 119. The red squares represent the means. 30Figure 6.13 Distribution of answers to: “Since last week, how many timeshave you shared this device with others?” . . . . . . . . . . . 32Figure 6.14 Distribution of answers to: “Since last week, how many timeshave you shared this device with others?” among age groups,with “0” excluded from the figure, N = 130. The percentagesshown in y-axis are the percents of received responses of devicesharing times for all participants among each age group. . . . 33Figure 6.15 Distribution of device sharing with various groups of people,N = 81 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Figure 6.16 Participants’ perceptions of app sensitivity (5-point Likert-typescale (1: Not at all, 2: Slightly, 3: Moderately, 4: Very, 5:Extremely.)) across age groups, N = 129. The red squaresrepresent the means. . . . . . . . . . . . . . . . . . . . . . . 36xiFigure 6.17 Distribution of session sensitivity ratings (5-point Likert-typescale (1: Not at all, 2: Slightly, 3: Moderately, 4: Very, 5:Extremely.)), N = 129. . . . . . . . . . . . . . . . . . . . . . 37Figure B.1 A Visualization of Data Collection Process from December 8,2016 to August 10, 2017, N = 276. Total # of Participants WhoCompleted the Study: 142; Total # of Participants from WhomWe Failed to Recover Data: 43; Total # of Participants fromWhom We Received Partial Data: 7; Total # of ParticipantsWho Withdrew from the Study: 84 . . . . . . . . . . . . . . . 53xiiAcknowledgmentsFirst and foremost, I would like to thank my supervisor, Konstantin Beznosov,and co-supervisor, Alexander De Luca, for giving me this opportunity to work onthis very interesting project; I am grateful for their patient guidance and supportthroughout this journey.Next, I would like to thank my collaborator, colleague, and supportive friend,Ildar Muslukhov, for providing me with a great deal of help and many suggestions,both on the technical and intellectual level, during the various stages of thisresearch.Further, I would like to thank an additional colleague and supportive friend,Primal Wijesekera, for his feedback and constructive discussions on how to writemy thesis, and his detailed review of it. I would like to thank Ross Sheppard forhis invaluable help with proofreading the conference paper upon which this thesisis based.I would also like to thank Ahmed Mahfouz for performing his initial project onunderstanding Android users in the wild, which inspired the idea for the currentproject. Last but not least, I would like to thank my beloved family, especially myparents, sisters and brother, and all those who patiently supported me during thisjourney.xiiiDedicationTo my beloved parents, siblings,and all my family members whonever ceased from supporting me.xivChapter 1IntroductionSmartphones have become one of today’s most commonly used computing plat-forms, with the Android being one of the world’s most popular mobile operatingsystems [16]. Advances in these devices’ capabilities enable the storage of (andaccess to) large amounts of data, some of which can be retained as sensitive orprivate in nature [15]. Also, due to the small size and high mobility of such devices,unauthorized access to sensitive data has become a significant threat. For instance,it has been shown that one out of every five users in the United States has accessedanother user’s smartphone without permission [14].To protect smartphones from unauthorized access, all mobile operating sys-tems provide authentication-based device locking. However, more than 40% ofall smartphone users do not use a secure locking mechanism [5]. Furthermore,most smartphone users tend to choose easy-to-guess unlocking secrets [3]. Theinconvenience of currently deployed unlocking methods, lack of motivation, andlack of awareness about the sensitivity of the data stored on their devices are oftenused to justify why a secure lock is not being used [5, 9, 15].Recent studies [7, 8, 12, 13], some of which were conducted in the wild,have shed light on how and under which circumstances smartphone users employdifferent unlocking mechanisms. For instance, Mahfouz et al. [13] used a studentsample to investigate authentication process parameters, such as the time required1to unlock devices, authentication error rates, and the types of apps used withineach session. Harbach et al. [7] focused on authentication speed, error counts, andthe types of errors observed in different unlocking mechanisms, based on a studysample of 134 participants. While these efforts have provided the first valuableinsight on the performance of smartphone unlocking mechanisms in the wild, thereare still plenty of unknowns.Such examples include the unknown correlations between users’ (un)lockingbehaviours and factors like age and the undertaken activity. Specifically, a recentonline survey on secure smartphone locking across eight countries suggested thatage might be linked to the locking behaviour of users, e.g., older users weresignificantly less likely to use a secure lock screen [8]. It is, however, still unclearhow age affects smartphone unlocking due to the lack of real-world studies focusingon age. In addition, it is also unknown if unlocking statistics correlate with thetype of activity a user undertakes while unlocking their device.In order to fill this knowledge gap and provide initial insights into how agecorrelates with smartphone authentication, we conducted a longitudinal field studywith 137 participants. The main goal was to show what behavioural patterns inauthentication were influenced by age and which were similar across age groups.In contrast to previously published research, our participants had diverse back-grounds and covered many age groups. Each participant was required to installour smartphone usage monitoring application and run it for at least two months.The monitoring app collected detailed data on (un)locking, session usages, activi-ties, etc. We focused our analysis on studying how user age, the used unlockingmechanism (and both together), and current activity correlated with (un)lockingbehaviour.The results of the analysis revealed that age has a statistically significantimpact on the average number of sessions and the average daily usage length.Participants of older ages tended to interact with their phone less frequently, andfor a significantly shorter period of time per day. These variables are importantsince this means that, for instance, authentication takes up a greater portion of2users’ overall device usage. For other patterns such as error rates, the results wererather consistent across all age groups. While one might hypothesize that youngerusers perform better at (un)locking, our results do not support this. This meansthat, for existing authentication systems, there is still room for improvement. Wealso found that the unlocking mechanism chosen significantly correlates with theaverage number of sessions, which corroborates with findings reported by Harbachet al. [9]. We extended the previous research to incorporate Fingerprint usersby showing that they tended to have significantly more sessions than PIN andSwipe/None users. Furthermore, we found that about 50% of the unlocks happenedwhile people were moving. In addition, while over 60% of the participants sharedtheir device with others, most of them shared with their family members andfriends, which corroborates with the iPhone sharing patterns reported by Cherapauet al. [3].3Chapter 2Related WorkDespite the availability of a variety of smartphone unlocking methods (e.g., PIN,alphanumeric passwords, unlock patterns, and biometrics), the adoption of smart-phone locking still falls below the expected rate [9, 18]. A recent study [5] showedthat most smartphone users are likely to underestimate the sensitivity of their dataand how they access it with their devices. Thus, many users do not protect their de-vices or data properly [8]. One potential reason for this is the inconvenience causedby unlocking and the time this takes [9]. As a consequence, many alternativesystems were proposed to make the authentication process easier [1, 10, 11, 17].SnapApp [1], which provides a time-constrained quick-access option, is an ex-ample that reduces the authentication workload by keeping users logged-in in amore secure way than having their device unlocked all the time. However, thisapproach allows for only limited improvements on usability and the security of theauthentication systems, as such short-term access is limited to ten subsequent useswithin ten minutes of the last secure unlock (e.g., PIN, Android Pattern, etc.).To design new mechanisms that are in line with users’ real needs, researchersand designers need data on how users utilize their devices in the real world. Whilethere is little information on whether current authentication systems match users’expectations with respect to usability and security, a few studies have providedinsights [7, 8, 12, 13].4Mahfouz et al. [13] studied how different smartphone use patterns correlatewith the time it takes users to unlock their device, how often users make a mistakeduring authentication, and which authentication methods users choose for devicelocking. The findings suggest that users who lock their devices interact with themmore frequently and for longer sessions than those who do not. In addition, thecost of unlocking is low when compared with overall smartphone usage and usersdo not mind adopting unlocking methods with a higher error rate (e.g., Androidpattern), as long as they allow faster input of the unlocking secret.Hintze et al. [12] investigated the number of interactions performed on smart-phones per day, the average interaction duration, and the total daily device usagetime through use of a state machine based on screen on/off events. Here the authorsanalyzed mobile device data logs from 1,960 Android smartphones. These logswere collected by the authors of the Device Analyzer project. The authors reportthat, on average, participants interacted with their devices 57 times per day, amongwhich 43% of time the device was actually unlocked, and the total daily deviceusage was 117 minutes.Harbach et al. [8] conducted a global-scale survey on Google Consumer Surveys(GCS) with 8,286 participants from eight countries to investigate whether users’attitudes towards smartphone unlocking differed between various nationalities.The findings indicate that demographic differences, including both nationalityand age, should be considered when designing new authentication systems forsmartphones. The authors also conclude that, despite the apparent differencesbetween nationalities, the inconvenience of unlocking is still one of the majorreasons for the low adoption rate of current authentication systems, especially forolder users. The researchers used an online survey to investigate how age affectsusers’ adoption rates of and attitudes towards secure lock screens. Alternatively,we employed real world data to look at how age correlates with usage, e.g., thenumber of sessions per day and other parameters.In another month long field study, Harbach et al. [7] collected data from asubset of PhoneLab users, all of which were affiliated with a university. The5authors instrumented LG Nexus 5 smartphones (with Android 4.4) to study theperformance of Android unlocking mechanisms in situ. They found that PIN userstake longer to unlock while committing fewer errors than pattern users, who tend tounlock their phones more frequently and are more prone to making errors. However,on average, PIN and pattern users spend a similar amount of total unlocking time.In addition, the authors offer a benchmark against which any newly designedunlocking mechanisms can be evaluated.While the previous work provides many insights on smartphone authenticationin situ, all the aforementioned studies suffer from their samples being skewedtowards predominantly tech-savvy young participants. In addition, we are unawareof any prior real world studies that have investigated how various age groups differin their smartphone unlocking practices. To fill this gap, we have conducted afield study with a large and diverse participant pool, focusing our analysis on howsmartphone authentication behaviour correlates with participants’ ages.6Chapter 3Research QuestionsThe main research questions we wanted to address and the rationale behind eachsuch question are as follows:• RQ1 - What influence does age have on smartphone (un)locking?Answering RQ1 provides a better understanding of whether smartphoneusers’ (un)locking behaviours, e.g., how frequent they unlock their devices,how long they interact with their devices after each unlock, etc., differswith age. This aids the research community in assessing the necessity ofdesigning efficient authentication systems for different age groups.• RQ2 - How does the choice of unlocking mechanisms correlate with the(un)locking behaviour of smartphone users?A previous study [9] revealed that smartphone users’ (un)locking behaviourdiffers across authentication systems. Answering RQ2 would help improveour understanding of whether these differences in usage hold true for a morediverse study sample, including participants of older age groups, and for newauthentication systems, e.g., Fingerprint. We will discuss the diversity of ourstudy sample in detail in Section 5• RQ3 - What is the correlation between the type of user activity and userunlocking behaviour?7Answering RQ3 provides a better understanding of how smartphone usersreact to usage scenarios during different activities, in terms of their unlockingbehaviour, including unlocking frequency and error rates, etc. Understandingthis question can provide insight into how platforms can support incrementalauthentication, or different approaches matching the nature of activities usersare involved in.• RQ4 - How frequent is device sharing?• RQ5 - Whom do users share their devices with?• RQ6 - How does users’ preferences for device sharing correlate with age?Answering RQ4-6 provides insight into smartphone users’ device sharinghabits, including sharing frequency, targets, and age correlations. Thiscan further assist us in evaluating the to what extent should authenticationsystems support sharing use cases.• RQ7 - What are the apps that users access the most, and how sensitive arethese apps from the point of view of the users?• RQ8 - How do users’ general perceptions of app sensitivity correlate withage?Finally, answering RQ7 and RQ8 provides a better understanding of smart-phone users’ most used applications and their perceptions of the sensitivityof specific apps or all apps in general. In particular, RQ7 provides a deeperinsight into how users perceive the sensitivity of the apps they accessedmost, while RQ8 focuses more on whether users’ general perceptions ofapp sensitivity differ between age groups. Having a better understanding ofthese two important research questions can help us and the wider researchcommunity to further estimate if authentication can be omitted for certainuse cases, and whether this can hold true for all age groups.8Chapter 4Methodology4.1 Data CollectionOur participants installed a custom-built study application (study app) on theirsmartphones and ran it for at least 60 days (the first 60 days were used for theanalysis). The study app ran in the background and collected relevant usage data.To reach a broad audience, we made the app available through the Google PlayStore.At the first launch, the application presented a consent form to the participant,where they were required to provide consent in order to participate in the study.Once consent was obtained, the application offered participants the opportunityto opt out of providing certain data records. In particular, we made the collectionof activity data optional. Afterwards, the app directed the participants to an entrysurvey that focused on collecting demographic data. In addition, participants wereasked to report which unlocking mechanisms they were currently using, if any.The app required certain types of access permission in order that it be able tomonitor important events in Android, such as device unlocking. For instance, userswere required to enable the device administrator and usage statistics privileges forthe app. Each participant received instructions on how to activate such permissionfor the study application.9After obtaining consent and receiving the required permission, the app beganto collect the data. In particular, the app recorded all lock and unlock events andlogged the start and end time stamps of each user session. In addition, the apprecorded the names of all apps that the participant opened during each session. Theapp also collected user activity data, but only if the participant consented to itscollection. Finally, the app collected user responses to two surveys. We describeall collected data types in details below.4.1.1 Locks/Unlocks/Failed UnlocksTo detect smartphone locking/unlocking events, the app logged both when thescreen turned off and on and the keyguard lock/remove event. To account for failedunlocking attempts, it also monitored password failure events. In addition, the appread smartphone settings to differentiate between two different modes of devicelocking: that is, if the device was locked due to an autolock, i.e., a certain timeoutafter inactivity, or if the device was locked manually by a user who clicked thepower button. The API to read this configuration, however, was only available inAndroid 6 and older models. Thus, we could not programmatically differentiate asto how the device was locked when participants used Android 7. Thus, participantsusing Android 7 or newer models were asked to report their configurations inthe entry survey. To help them do so, we provided instructions on how to readthis value in the settings of the device. Due to similar limitations, our app wasonly programmatically capable of detecting the used unlocking mechanism fordevices running Android 5 or older versions. For participants using Android 6 orlater versions, our study app asked them (during the entry and exit surveys) thetypes of unlocking mechanisms they used. Finally, to detect when a user changedtheir password for device unlocking, the app monitored the password changeevents. Every time such events were detected, the app asked the participants thechanges they had made, i.e., whether they had changed the authentication method,unlocking secret code, or both.104.1.2 User SessionsFor the purpose of this study, we define a user session as the period of time betweenthe user unlocking the screen and then locking again by an explicit locking actionor through a timeout. Within each user session the study app recorded all eventsbeginning when the user first interacted with a new application, i.e., the applicationthat the user saw on their screen. When such a change was detected, the app loggedthe current date and time together with the name of the new foreground application.Note that each application was identified by both (1) the application package name,which is unique throughout Google Play Store, and (2) the application name, whichwas readable, but not necessarily unique. For example, the application identifierfor WhatsApp is {com.whatsapp;WhatsApp}.4.1.3 ActivitiesThe app detected and collected the participant’s activities (e.g., running, tilting,1walking [6]) as soon as any change in activity type was reported by Android. Thisactivity data was collected only if the participant provided consent. The aim was tounderstand their session usage patterns and locking behaviour to figure out how tooptimize the locking approaches available in the platform.4.1.4 Device Sharing SurveyThe Device Sharing Survey sampled the participants’ experience with sharing theirsmartphone. Presented once a week, the survey asked each participant to recallall instances of them sharing their device during the past week. By choosing aone-week recall, we aimed to reduce the memory burden on the participants and,thus, to make it easier for them to provide the correct information. The survey alsoasked the participants to categorize the people they shared their devices with; i.e.,friends, family, roommates, and others. In addition, participants were also askedwhether they had had any concerns that someone from their social circle would1Tilting the device around the horizontal or vertical axes11access their smartphone without their permission. Participants were allowed topostpone answering this survey for either an hour or a day.We argue that device sharing habits are relevant for authentication. The mostimportant reason for this is that sharing with close people potentially requiresauthentication systems to be compatible with this behaviour.4.1.5 Contextual SurveyWe incorporated into the study app a contextual survey to measure the participants’perceptions of the sensitivity of their apps in specific usage contexts. Experiencesampling has been used in previous research (e.g., see [4, 9]) and has allowedresearchers to better understand how a participant’s context impacts on their securityand privacy decisions.In particular, participants were asked to quickly assess their surroundings andreport if someone were able to view the contents of their smartphone’s screen.Afterwards, the participants were asked to rate in that context the sensitivity ofthe apps they had spent the most time using. Throughout the day, the study apprandomly selected unlocking events and presented the sensitivity survey to theparticipants. In order to keep this task unobtrusive, while allowing coverage ofa wide range of contexts, the study app dynamically adjusted the likelihood ofpresenting the survey so as to present it six times a day at most. Participants wereallowed to skip the survey by clicking the “Not now” button.We argue that participants’ perceptions of app sensitivity are also relevantto authentication, as this can help evaluate whether authentication systems arenecessary under all use cases.Both surveys were implemented as mini-questionnaires. The questions can befound in Appendix A.124.2 Data TransmissionTo protect the confidentiality of the collected data, the study app used an app-specific location on the internal storage of the Android OS. Such storage is pro-tected by the Android’s access control subsystem, and is only readable for thecorresponding app. In addition, we encrypted all data logged throughout the daywith a symmetric encryption key, generated at run time. We encrypted this key witha hard-coded public key, and then appended it to the encrypted data logs beforesubmitting them to the back-end server.Encrypted logs were uploaded to our back-end server once a day, near mid-night. Throughout the study, we downloaded the new data on a daily basis. Afterdecrypting the data, we checked for data corruption; if this had not taken place, weadded the data to our dataset.13Chapter 5ParticipantsWe recruited our participants from North America (US and Canada) throughAmazon Mechanical Turk, Twitter, Facebook, our university mailing lists, and TheSample Network.1 Android smartphone users who were 19 years of age or olderwere eligible to participate in our study. Everyone was allowed to participate usingonly one device.Each participant who ran the study app for 60 days received a compensation of$40 USD and was entered into a raffle for one iPad Pro 2. The chances for eachparticipant to win the raffle were additionally increased each time they answereddevice sharing and contextual surveys. As part of the compensation, we provideda report to each participant with a statistical description of how they used theirsmartphone during the study.In order to make sure that the data collection process was robust, we conducteda pilot study for 15 days with six participants. We obtained approval from theresearch ethics board of our university before conducting the pilot study.Overall, we recruited 276 participants. Considering that all participants beganthe study at different times, all data2 used in the analysis were collected betweenDecember 8, 2016 and August 10, 2017. Out of the 276 participants, 185 completed1The Sample Network (http://thesamplenetwork.com/) was used for recruitment in the US only.2A figure that visualizes the data collection process is presented in Appendix B14the study by providing us with their data for 60 days or more. We note that due to atechnical issue, our back-end server had been failing silently for 10 days. While thestudy app did resubmit data for that period for most participants, we were unableto recover the data from that 10-day period for 43 of our participants, resulting inusable data from a total of only 142 participants.For each participant, we selected only the first 60 days of their usage. Tomake sure that we did not mix data from different authentication methods, wealso analyzed how often our participants changed their unlocking methods. Ifa participant changed their unlocking method during our 60-day study period,we verified whether one of these methods accounted for 95% or more of the logrecords. If so, we retained the participant’s data. Otherwise, we removed the datafrom the analysis; we did this for three such participants.Finally, we excluded two participants who specified that their unlocking mech-anism was “knock (pattern)” and “hold for a set amount of time”. This reduced ourparticipant pool down to 137 total participants.Of the 137 participants whose data we included in the analysis, 123 (89.8%)were from the US, 81 (59.1%) were female and 56 (40.9%) were male. Agesranged from 19 to 63 years, with a mean age of 40 and median age of 38 (SD= 12.5). Participants had diverse education levels, with 56 (41%) having a highschool diploma and 35 (26%) having earned a Bachelor’s degree. The occupationsand salaries of our participants also varied, as shown in Table 5.1.To evaluate our study sample, we compared it against the smartphone ownershippopulation in the US reported by the Pew Research Center [2]. Statistical resultsdid not reveal any significant differences between our participants’ demographicsand the one presented in the Pew Research Center report in terms of age (χ2 = 20,p = .22), gender (χ2 = 2, p = .16), education level (χ2 = 15, p = .24), or salary(χ2 = 20, p = .22). We divided all participants into five age groups, based on agegroup definitions from the Pew Research Center report. The distribution of theparticipants across all five age groups is shown in Table 5.1.15Parameter Property # of participantsResidence US 123Canada 14Gender Female 81Male 56Age 19-24 1725-34 4235-44 2645-54 3155-63 21Education Less than High School 1High School 56Professional School 23University (Bachelor’s) 35Master or PhD 17Other 5Occupation Management 9Professional 27Clerical Support Worker 16Service and Sales Worker 18Craft and Trades Worker 6Machine Operator 1Elementary Occupation 3Student 14Self-employed 3Unemployed/Retired/Disabled 28Salary Less than $30,000 37(US, N=123) $30,000-$49,999 17$50,000-$74,999 3416$75,000-$99,999 17$100,000+ 17Prefer not to specify 1Salary Less than $30,000 5(Canada, N=14) $30,000-$49,999 4$50,000-$74,999 1$75,000-$99,999 0$100,000+ 0Prefer not to specify 4Table 5.1: Participant demographics, N = 13717Chapter 6Results6.1 (Un)locking Behaviour and AgeFigure 6.1 outlines the distribution of unlocking mechanisms for participants amongthe different age groups. We conducted statistical tests on usage data, includingsession lengths, the number of sessions, daily usage lengths, error rates, and thenumber of auto/manual locks among the predefined age groups and in terms ofdifferent unlocking mechanisms. We consider all of these factors relevant forauthentication behaviour as they are potentially influenced by or influence the(choice of) authentication method. For example, if a group has a great number ofsessions per day, they are exposed to authentication more often.As most significance tests require individual data items to be independent,we therefore averaged the usage data per participant before conducting the tests.As shown in Table 6.1, we found that usage data including number of sessions,daily usage lengths, auto/manual locks were significantly affected by age, whereasothers were not. This emphasizes that designers and developers should take ageinto consideration while designing new authentication systems in smartphones.To investigate unlocking differences, we conducted the significance tests amongall mechanism groups, including “Password”, and we did not find any significantdifferences in all analyzed usages between “Password” and other mechanism18Figure 6.1: Unlocking mechanism distribution among age groups. Numbers beloweach age group represent the total number of participants from that group.Usage statistic Age Matters?Session Lengths NoNumber of Sessions Per Day YesDaily Usage Lengths YesError Rates NoAuto/Manual Locks YesWhether authentication happened at still / on the move YesUnlocking Mechanism (at still / on the move) YesTable 6.1: Overview of the usage patterns that were and were not significantly influ-enced by age.19groups. Thus, we dropped the three password participants while presenting theresults for unlocking differences, considering that the size of “Password” group (3)was very small and it lacks sufficient measurement power to perform the tests. Inthe following subsections, we explain the age and unlocking differences for eachdimension of usage separately in detail.6.1.1 Session LengthsIn total, our data set contained 260,735 user sessions. On average, each sessionlasted 11.51 minutes (SD = 15.46 minutes). Figures 6.2 and 6.3 show the distribu-tion of average session length across various age groups and unlocking mechanisms,respectively. There was no statistically significant difference in the length of thesessions, either across all age groups (Kruskal-Wallis χ2 = 9.41, p = .052) or acrossall unlocking mechanism groups (Kruskal-Wallis χ2 = 3.84, p = .43).5.3911.397.451516.725.1 5.76 4.3611.939.1501020304019−24 ( 17 )25−34 ( 42 )35−44 ( 26 )45−54 ( 30 )55−63 ( 21 )Age GroupAverage session length (minutes)Figure 6.2: Average session length per age group (minutes), N = 136. The redsquares represent the means.209.51 12.49 7.4411.587.47 8.824.35 5.79020406080Android Pattern( 20 )PIN( 44 )Fingerprint( 22 )Swipe/None( 47 )Unlocking MechanismAverage session length (minutes)Figure 6.3: Average session length per unlocking mechanism (minutes), N = 133(the three password participants were removed from the figure, as the samplesize was too small). The red squares represent the means.6.1.2 Number of Sessions Per DayOn average, participants had 32 sessions daily with their smartphones (SD = 26).Figure 6.4 shows that younger participants interacted more frequently with theirdevices than older participants (Kruskal-Wallis χ2 = 27.98, p < .001). Furtheranalysis (Bonferroni-corrected Conover-Iman post-hoc) revealed that participantsin the “55-63” group had significantly fewer sessions than participants in theyounger groups (except for the “45-54” age group). We also found that participantsfrom the age group “45-54” interacted with their devices significantly less thanthose from the “19-24” and “25-34” groups. The means and standard deviationsfor each group are presented in Table 6.2.In addition to the age group correlation, we also tested the impact of the typeof the unlocking mechanism on the number of sessions, which turned out to be2148.1141.8328.2224.1715.6738.8231.2923.0718.3610.78025507510019−24 ( 17 )25−34 ( 42 )35−44 ( 26 )45−54 ( 31 )55−63 ( 21 )Age GroupAverage number of sessionsFigure 6.4: Average number of sessions per age group, N = 137. The red squaresrepresent the means.Age group Mean Std19-24 48.11 23.7825-34 41.83 33.0335-44 28.22 17.2245-54 24.17 19.2455-63 15.67 12.04Table 6.2: The number of sessions per age group.significant (χ2 = 16.81, p = .002). Furthermore, the Conover-Iman post-hoc testwith Bonferroni correction showed that “Fingerprint” users (M = 53.64, SD =35.48) had significantly more sessions than those using PIN (M = 23.90, SD =21.15) and Swipe/None (M = 30.88, SD = 24.01). The distribution of the numberof sessions across different unlocking mechanisms is shown in Figure 6.5.2231.2323.953.6430.8829.3419.6648.4826.17050100150Android Pattern( 20 )PIN( 45 )Fingerprint( 22 )Swipe/None( 47 )Unlocking MechanismAverage number of sessions per dayFigure 6.5: Average number of sessions per unlocking mechanism, N = 134 (thethree password participants were removed from the figure, as the sample sizewas too small). The red squares represent the means.6.1.3 Daily Usage LengthsIn general, we found that younger participants tended to use their smartphonesmore frequently than older ones, i.e., in session counts. At the same time, ananalysis of session length did not reveal any statistically significant differencebetween the age groups.Figure 6.6 shows the daily averages for the total amount of time the participantsused their smartphones per age group, and statistical analysis revealed that therewas a significant difference (Kruskal-Wallis χ2 = 15.58, p = .004). A post-hocConover-Iman test (Bonferroni-corrected) revealed that participants from the agegroup “55-63” (M = 106.89, SD = 89.48) used their smartphones significantly lessfrequency of usage per day than participants from the age groups “19-24” (M =215.93, SD = 115.24), “25-34” (M = 251.24, SD = 161.70), and “45-54” (M =23254.10, SD = 200.65).Figure 6.7 shows the average amount of daily device usage time for participantsusing different unlocking mechanisms. Statistical analysis did not reveal anysignificant differences in total device usage time per day across various unlockingmechanism groups (Kruskal-Wallis χ2 = 6.10, p = .19).215.93251.24159.05254.1106.89230.32 217.21126.21209.9162.99020040060019−24 ( 17 )25−34 ( 42 )35−44 ( 26 )45−54 ( 30 )55−63 ( 21 )Age GroupAverage daily usage length (minutes)Figure 6.6: Average daily usage length per age group (minutes), N = 136. The redsquares represent the means.6.1.4 Error RatesTo assess how often participants made mistakes while unlocking, we calculatedtheir error rates, i.e. how often their unlocking attempts failed. Since participantsusing the “Swipe/None” method cannot make errors while unlocking, we removedthese participants from our analysis on error rates, which reduced the analysissample to 90 participants. We tested both the impact of the type of unlocking24211.65179.28239.75 224.08220.38136.45204.93 202.650200400600Android Pattern( 20 )PIN( 44 )Fingerprint( 22 )Swipe/None( 47 )Unlocking MechanismAverage daily usage length (minutes)Figure 6.7: Average daily usage length per unlocking mechanism (minutes), N = 133(the three password participants were removed from the figure, as the samplesize was too small). The red squares represent the means.mechanism and age against the error rate. Statistical analysis did not reveal anysignificant differences among either age groups (Kruskal-Wallis χ2 = 4.02, p =.40) or unlocking mechanisms (Kruskal-Wallis χ2 = 0.90, p = .82). Figure 6.8shows the distribution of error rates for participants based on their age groups andthe unlocking methods used.6.1.5 Auto/Manual LocksWe removed “Swipe/None” participants from our analysis of auto/manual locks,because in Android OS the autolock setting is not enabled for them. Furthermore,we removed 26 participants who disabled the autolock functionality manually,either at the beginning of the study or during the study. Overall, we analyzedvarieties of locking behaviour based on the data collected from 64 participants.2501020304019−24( 13 )25−34( 33 )35−44( 14 )45−54( 16 )55−63( 14 )Age GroupError rates [%]UnlockingMechanism Android PatternFingerprintPINFigure 6.8: Error rates distribution among age groups, N = 87 (the three passwordparticipants were removed from the figure, as the sample size was too small).Numbers below each age group represent the total number of participants fromthat group.By default, the value of the autolock timeout is set to 5 seconds. Our analysisrevealed that out of the 64 participants, 27 retained the default autolock timeout,while 18 reduced the autolock time to 0, which locks the device immediately afterit enters sleep mode. Since we were not able to differentiate autolocks from manuallocks when the timeout was set to 0, we further excluded those 18 participantsfrom the analysis. This reduced our participant pool for this analysis to 46.To evaluate whether the participants relied more on autolocks or manual locks,we calculated the percentages of autolocks over the total number of locks per dayper user. Statistical analysis revealed that people’s locking behaviour on manuallock vs autolock differed among the age groups (Kruskal-Wallis χ2 = 11.74, p= .02), but not across the unlocking mechanisms (Kruskal-Wallis χ2 = 2.26, p26Figure 6.9: Distribution of locking types among age groups, N = 46. Numbers beloweach age group represent the total number of participants from that group.= .52). Further analysis showed that participants in the age group “55-63” (M =70.22, SD = 26.52) had significantly more autolocks than those in the younger agegroups, “19-24” (M = 26.86, SD = 29.71), “25-34” (M = 32.54, SD = 27.22) and“35-44” (M = 21.40, SD = 14.44). The overall percentage of autolocks in eachage group and unlocking mechanism group are presented in Figures 6.9 and 6.10,respectively.6.2 Activity While UnlockingOf the 137 participants, 119 chose to provide us with their activity data. To identifyuser activities during unlocks, we mapped their timestamps. We used one minuteas the threshold, meaning that if no activity records occurred one minute before or27Figure 6.10: Distribution of locking types among unlocking mechanisms, N = 45(the only one password participant in this analysis was removed from thefigure, as the sample size was too small). Numbers below each unlockingmechanism represent the total number of participants from that group.after the time when the unlock took place, then we considered that we had failed1to detect the type of activity associated with this unlock. When multiple activityrecords occurred within a one-minute timeframe, we selected the closest activity(time-wise) as the one that user had undertaken during the unlock.Among all unlocking events collected for the 119 participants, we removedthose that were tagged with “unknown”2 activity and those for which we had failedto detect activities, which amounted to around 50.2% of the unlocks. Figure 6.11shows the distribution of activity types over the remaining 49.8% of unlocks. We1The potential reason for this could be that the sensor used by the API (Google’s ActivityRecognition API) for detecting the activity type was not started successfully while the unlock tookplace.2Because there is not enough data for Google’s Activity Recognition API to determine withsignificant confidence the activity the user is currently performing.2856.4421.817.630.2513.87Still In a vehicle On foot On a bicycle Tilting Figure 6.11: Distribution of activities during unlock, N = 119.found that more than half of these unlocks took place when the participants’ deviceswere still, whereas the other 44.55% of unlocks occurred while the device wasmoving. Only 0.25% of the unlocks happened while the participants were on abicycle. The distributions of activities among the unlocking mechanisms did notdiffer notably from one another.For the following, we removed the activity type “tilting”, which we consideredas not significant enough of a move to influence unlocking performance. After-wards, we categorized all activities into two states, still and move (all activitiesother than still). As shown in Table 6.1, we found that age was a good predictor forwhether authentication was used while the device was still or on the move, and theused authentication mechanism. Such findings are important as they help designersand developers to estimate the adoption rates of newly-designed authenticationsystems for users from different age groups and under different usage scenarios(still vs. move).29025507510019−24 ( 12 )25−34 ( 35 )35−44 ( 25 )45−54 ( 28 )55−63 ( 19 )Age GroupPercentage of unlocks happened while at still/move [%]StillOrMove Still MoveFigure 6.12: Distribution of unlocks that happened while at still/move sorted by agegroups, N = 119. The red squares represent the means.6.2.1 Error Rates While At Still vs On the MoveTo assess how error rates correlate with activity types, we additionally excludedparticipants who used “Swipe/None” as errors are not possible with this mechanism.We then counted the number of failed and total unlocking attempts that eachparticipant performed while they were at “still” and “move” states. We thencalculated the likelihood of making an error for each participant for each activitystate. However, a paired-samples t-test did not reveal any significant differences inthe likelihoods that participants would make an error while their devices were still(M = 4.34, SD = 7.04) or moving (M = 4.82, SD = 7.26); t(73) = 1.27, p = .21.30Variable Estimate Std. Err. z value Odds RatioIntercept .428 .019 22.67* 1.53Age=25-34 .084 .022 3.83* 1.09Age=35-44 -.003 .025 -0.11 1.00Age=45-54 .665 .025 26.96* 1.94Age=55-63 .525 .031 16.74* 1.69Table 6.3: Binary logistic regression model: whether authentication is used while atstill vs on the move by age. An * denotes significance (p < .05).6.2.2 How Age Predicts Whether Authentication is UsedWhile At Still vs On the MoveTo evaluate how often participants unlocked their smartphones while they were atstill and moving states, we calculated the percentage of unlocks for each mobilitytype (still and move) per user over the entire study period. Figure 6.12 presents thedetailed distribution of unlocks among the age groups and activity types. To furtherunderstand how age interacts with activity types (still/move), we fitted a binarylogistic regression model to predict whether authentication will be used when usersare at still and on the move based on age. We used age group “19-24” as thereference category for the model. The analysis revealed that age had a significanteffect on predicting whether authentication happened while at still or on the move.Table 6.3 gives an overview of the fitted model.6.2.3 How Age and Still/Move Predict Lock Types Used ForAuthenticationAnother aspect worth looking at is the correlation between the lock type usedfor unlocks and other factors including age and activity states (still, move). Wefirst excluded the lock type “Password” from the analysis, since very few (4)participants had used it. Afterwards, we fitted a multinomial logistic regressionmodel to predict the lock type based on the participants’ age and underwent31Variable Estimate Std. Err. z value Odds Ratio Estimate Std. Err. z Odds Ratio Estimate Std. Err. z Odds RatioSwipe/None vs. Android Pattern Swipe/None vs. Fingerprint Swipe/None vs. PINIntercept -.140 .042 -3.30* .87 .180 .039 4.60* 1.20 -.197 .043 -4.56* .82Age=25-34 -.438 .053 -8.21* .65 .365 .046 7.89* 1.44 .698 .050 14.06* 2.01Age=35-44 -1.341 .061 -22.09* .26 -.859 .051 -16.93* .42 -.274 .053 -5.22* .76Age=45-54 -2.195 .072 -30.69* .11 -2.432 .068 -35.82* .09 -.907 .055 -16.47* .40Age=55-63 -.684 .087 -7.87* .50 -1.356 .095 -14.31* .26 .827 .067 12.26* 2.29Activity=Still .214 .057 3.72* 1.24 .536 .052 10.32* 1.71 .547 .056 9.68* 1.73Age=25-34:Activity=Still -.020 .071 -0.29 .98 -.389 .061 -6.41* .68 -.443 .065 -6.84* .64Age=35-44:Activity=Still .493 .077 6.41* 1.64 -1.077 .069 -15.65* .34 -.590 .069 -8.61* .55Age=45-54:Activity=Still -.358 .088 -4.07* .70 -1.262 .087 -14.48* .28 -1.005 .070 -14.40* .37Age=55-63:Activity=Still .451 .102 4.42* 1.57 -.283 .111 -2.55* .75 -1.324 .085 -15.60* .27Table 6.4: Multinomial logistic regression model: how age and still/move statespredict lock type used for authentication. An * denotes significance (p < .05).Figure 6.13: Distribution of answers to: “Since last week, how many times have youshared this device with others?”activities, with interaction between age and activity types. We used lock type“Swipe/None”, the age group “19-24” and the activity state “move” as the referencecategories for the model. In general, our results show that all the main effectswere significant, in addition to the interaction effect between age and activity states(still/move). Table 6.4 provides an overview of the fitted model.32Figure 6.14: Distribution of answers to: “Since last week, how many times haveyou shared this device with others?” among age groups, with “0” excludedfrom the figure, N = 130. The percentages shown in y-axis are the percents ofreceived responses of device sharing times for all participants among each agegroup.6.3 Device SharingOur study app presented participants with the device sharing survey once a week.Out of our 137 participants, 130 answered at least one of the device sharing surveys.In total, we received 897 responses. On average, each participant answered 7 suchquestionnaires (SD = 2.4, Min = 1, Max = 17). Overall, out of the 130 participants,81 reported that they regularly shared their device with others. In all the devicesharing survey responses that we received, participants reported over 60% of thetime that they had not shared their devices with others during the preceding week.The detailed distribution of device sharing times is presented in Figure 6.13.Figure 6.14 shows how responses on device sharing times (greater than 0)are distributed among the age groups. To evaluate how preferences for sharing330102030405060708090Coworkers FamilyFriendsRoommatesUnknown PeopleType of people% ParticipantsFigure 6.15: Distribution of device sharing with various groups of people, N = 81differ with age, we further divided all responses into two categories: not shared(responses on sharing for 0 time) and shared (all responses except not shared). Wethen calculated the corresponding percentages of all shared responses for eachparticipant. Interestingly, while a Kruskal-Wallis test revealed that participantsdiffered significantly in their device sharing preference among the age groups (χ2= 11.73, p = .02), further analysis (Bonferroni-corrected Conover-Iman post hoc)did not reveal any significant difference in the percentages between any two agegroups.As shown in Figure 6.15, family and friends are the top two types of peoplewith whom participants regularly shared their devices, followed by roommates andco-workers. Interestingly, about 3.7% of the 81 participants also reported that theyshared their devices with unknown people.34Overall 19-24 25-34 35-44 45-54 55-63App name rating App name rating App name rating App name rating App name rating App name ratingChrome 1.74 Chrome 2.19 Chrome 1.82 Chrome 1.82 Facebook 1.38 Chrome 1.86Facebook 1.87 Gmail 2.73 Gmail 2.29 Contacts 1.73 Chrome 1.18 Contacts 2.77Google Play 1.51 Snapchat 3.38 Facebook 1.90 Facebook 1.99 Messenger 1.50 Settings 1.95Settings 1.53 YouTube 2.00 Google Play 1.71 Settings 1.39 Google Play 1.16 Facebook 2.46Gmail 2.24 Settings 1.97 Contacts 2.07 Google Play 1.53 Contacts 1.32 Google Play 1.58Table 6.5: Top 5 most-used apps and the corresponding averaged sensitivity ratings.Sensitivity level has been collected using a 5-point Likert scale (1: Not at all,2: Slightly, 3: Moderately, 4: Very, 5: Extremely.) with question: “Please ratefor each app on the list: If this person were watching your screen, and you wereusing this app right now, how much would it affect your privacy?”6.4 App Usages and SensitivitiesIn this section, we report on the apps that participants accessed the most duringthe study, with sensitivity ratings for these apps. We also present results on howparticipants from different age groups perceived the sensitivities of their apps.We excluded the data of the 8 participants who had not granted permission forour study app to collect their application details. In addition, we excluded specialsystem apps, such as launcher and systemui, from the analysis, since these appsform a part of Android OS, which the user did not actively open or access.In total, we received 18,496 app sensitivity survey responses. On average, eachparticipant answered 136 such questionnaires (SD = 102.9, Min = 1, Max = 732).Our analysis showed that our participants used 2,976 unique apps overall. Onaverage, each participant used 72 unique apps during our study, ranging from 10 to200 (SD = 36, Median = 67).We then identified the most-used apps. To do this, we first aggregated thenumber of days on which each application was launched by each participant.Afterwards, we summed up the days used for each app by all the participants;we then considered the sum as the total number of days that this app was usedduring the study, and sorted all the apps by this number. Table 6.5 shows the top 5most-used apps for all the participants overall and across each age group.To evaluate how sensitive the apps are from the participants’ points of view,352.131.951.781.551.9421.591.32 1.241.651234519−24 ( 16 )25−34 ( 39 )35−44 ( 25 )45−54 ( 30 )55−63 ( 19 )Age GroupsAverage sensitivity ratings per participantFigure 6.16: Participants’ perceptions of app sensitivity (5-point Likert-type scale(1: Not at all, 2: Slightly, 3: Moderately, 4: Very, 5: Extremely.)) across agegroups, N = 129. The red squares represent the means.we calculated the average sensitivity ratings for each application across all theparticipants, who rated the apps in their specific groups. The sensitivity ratings forthe top 5 most-used apps in each group are also shown in Table 6.5.Overall, Chrome was the most used app among all the age groups, whileparticipants from the “19-24” group rated it with the highest sensitivity rating,compared with the other groups. The top 5 most-used apps for the participantsfrom the “35-44” and “55-63” groups were identical with the exception of order.To assess how participants’ perceptions of app sensitivity differ with age, wecalculated the average sensitivity ratings among all the rated applications for eachparticipant. On average, the participants rated 1.83 for all apps (SD = 0.94, Min =1, Median = 1.43, Max = 4.96). Figure 6.16 shows the distribution of the averagesensitivity ratings per participant across the age groups. However, a Kruskal-Wallistest did not reveal any significant differences among the age groups (χ2 = 7.04, p36010203040501 2 3 4 5Session Sensitivity Ratings% SessionsFigure 6.17: Distribution of session sensitivity ratings (5-point Likert-type scale (1:Not at all, 2: Slightly, 3: Moderately, 4: Very, 5: Extremely.)), N = 129.= .13).We also checked how many sessions included apps that the respective user ratedhighly-sensitive (4 or 5 on a 5-point Likert-type scale; 1: Not at all, 2: Slightly,3: Moderately, 4: Very, 5: Extremely). To do this, we defined and calculated thesensitivity ratings for each session as the maximum of all app ratings collectedfrom it. As shown in Figure 6.17, only 27.2% of the 16, 893 rated sessions wereconsidered highly-sensitive.37Chapter 7DiscussionThe results presented in this paper provide initial insights into how individuals’smartphone authentication patterns differ by age group, thus, extending on previousstudies [7, 8, 12, 13].7.1 Age Makes a DifferenceFirst of all, our data shows that smartphone usage patterns indeed differ based notonly on unlocking mechanisms, but also on age. Our results quantitatively providesupport for previous claims that demographic differences such as age should betaken into account while designing new authentication systems for smartphones [8].For instance, we found that participants in older age groups interacted with theirdevices significantly less frequently than younger groups. They also interactedwith their devices for a significantly shorter amount of time each day. This meanstwo things: 1) they are exposed to the authentication mechanism less frequentlythan younger groups and 2) authentication takes up a greater portion of the overallinteraction with their devices.In addition, age also significantly correlated with the locking behaviour of ourparticipants. We found that older groups relied more on autolocks than the othergroups. Overall, the “55-63” participants were more than three times as likely to38use the autolock feature than participants from the “19-24” and “35-44” groups,and about two times more likely than those from the “25-34” and “45-54” groups.As the autolock feature presents a trade-off between security and usability, futureresearch into the popularity of the autolock function among older users and theirawareness of the respective security implications is necessary.Another interesting result to highlight here is that age was a good predictorfor whether authentication was used with a moving device vs a still device. Incombination with those two states, age was also a good predictor for which au-thentication system is used. This result further helps predict and evaluate whethercertain authentication systems would likely be adopted among various age groups,which is important for designing future systems.7.2 No One-Fits-All SolutionWe found that participants using different unlocking mechanisms use their devicesdifferently. Specifically, Fingerprint users tend to have more device sessions thanother groups, including the PIN and Swipe/None groups. In addition, PIN andAndroid unlock pattern users were found to be two times more likely to rely onautolock features than Fingerprint users.Different usage across the authentication systems has been shown before [9].Our results extend these previous findings by showing that these usage differenceshold true for more diverse samples as well, including for older age groups. We arealso the first to extend these findings to Fingerprint users.Our numbers do not allow us to infer the reasons for why users pick theirrespective authentication system (or the direction of causality). It is possible thatpeople select their authentication system with respect to their needs (e.g., shorterauthentication time due to more sessions, which was more likely in younger agegroups). Another possible interpretation could be that participants adapt theirbehaviour to the authentication system they have chosen.That said, while we cannot make claims about the reasons for various userchoices, the results of our study indicate that offering a selection of different39authentication systems for smartphones seems to be the right thing to do. Usershave different preferences and this approach allows them to choose the system thatbest fits their needs and the way they are using their devices.7.3 Design and Evaluate for MovementOur results show that, overall, about 50% of unlocks happened when the user wasmoving (we detected the phone movement; hence, it was safe to assume that theparticipant was moving). This was a common observation across all age groups.One might hypothesize that participants were more likely to make unlockingerrors while moving. Our dataset, however, shows that this was unlikely. Thenumbers of failed unlocking attempts when participants’ devices were still andwhen they were moving were very similar across the age groups. This mightindicate that current unlocking mechanisms are robust against a moving state.A consequence of this and the fact that people do regularly authenticate them-selves while moving, means that newly-designed unlocking mechanisms shouldbe as robust in the presence of movement as the existing mechanisms, in orderto be acceptable for smartphone unlocking. This requirement not only influenceshow unlocking systems should be designed (e.g., constant eye contact might not bepossible) but also how they are evaluated in studies (in-lab vs. in the wild).A potential caveat to our findings is that our data collection instrument failed todetect the activity types for around 50% of authentication events, due to technicallimitations (the activity recognition API reported “unknown” or failed to detect theundertaking activity). As such, readers should take this into consideration whileinterpreting these results.7.4 Device Sharing is CommonA majority of the participants in our study (around 60%) had shared their deviceswith others. While most of them reported that they had shared their devices withfamily members and friends, there is a small group of people (3) who had shared40their devices with unknown people. Our Android results corroborate with theiPhone sharing patterns reported by Cherapau et al. [3], who found that 60% ofiPhone users have also shared their passcodes with others (with partners, family,and friends being their top three sharing recipients).Furthermore, while we found that age significantly correlates with the numberof sharings taking place per week, it was still a very common task across all theage groups.This means that authentication systems will benefit from being flexible inallowing sharing (to a certain extent) while continuing to remain protected. Forexample, a behavioural biometrics system that checks for anomalies to lock adevice should be designed in a way that it does not make sharing impossible. Anauthentication mechanism that can be shared among a trusted set of people withoutrevealing their own secrets could also be useful for people who tend to share theirphones with others.7.5 Most Sessions Have Low SensitivityOur results show that, in the majority of cases, participants accessed apps that theyconsidered to have low sensitivity (roughly 72% of sessions). This is similar towhat was found by Harbach et al. [9]. In their study, in most sessions, participantsaccessed data that they considered to have little to no sensitivity. Additionally, wefound that, while participants’ individual perception of app sensitivity varies (from1 to 4.96), it does not correlate with age.An important caveat here is that this data relies on self-reporting, and as shownin previous work, users often underestimate the actual privacy risks associated withsmartphone access [5].Nonetheless, this data provides further support for the claim that a significantportion of smartphone functionality could be made available without the need forauthentication. This is especially important due to the large number of sessions(and time) that users spend authenticating themselves on their devices. Smartphonemanufacturers have been moving along that direction recently (e.g., activating41alarms without logging in to the phone). We argue that identifying more of theseuse cases could help to further reduce the amount of time that smartphone usersspend in unlocking their devices, without compromising their privacy and security.42Chapter 8LimitationsWhile we have worked to mitigate any potential problems with the setup of thisstudy, a few limitations must be kept in mind when processing the present results.First, our server was silently down for 10 days at the end of May, when most ofour participants were supposed to complete the study. This unexpected issue forcedus to drop unrecoverable data for certain participants. This could have potentiallybiased our study sample. However, we do not consider this a major threat to thevalidity of our results, since we did not find statistically significant differencesbetween the distributions of our sample demographics and the demographics ofthe US smartphone ownership reported by the Pew Research Center.We measured session length as the time between screen on/off (with keyguardremoved) events. Since, with this measurement, we could not remove the screenoff timeout from the session length calculations, the reported session lengths couldpotentially have been longer than the actual ones. Our comparisons of sessionlengths among the groups might also be biased as a result, as different participantswould have set the screen off timeout to different values, varying from 0 secondsto 24 hours.Due to technical limitations, we were unable to programmatically detect theunlocking mechanisms and states of autolock settings employed by participantsusing Android 6 (64 users) and 7 devices (31 users). Therefore, we asked those43participants to report their unlocking mechanisms (Android 6 and above users)and the autolock settings (Android 7 and above) in the entry and exit surveys. Asa consequence, the correctness of our collected data partially depends on userrecollection. However, the results give us no reason to believe that they wouldhave incorrectly reported this data. In addition, the two other mini-questionnaires(the device sharing survey and the contextual survey) that we presented during thestudy also relied on self-reported data and may have impacted on the correctnessof our results, despite being as close to the actual events as possible. Furthermore,our device sharing questions did not define the concrete “sharing” scenarios, i.e.,sharing content on the smartphone screen with others, handing over device undersupervision, or handing over device unsupervised, and the interpretation of sharingdepends on the participants.On average, the confidence level of the detected activities is 73.37 (SD = 25.20,Min = 21, Median = 75, Max = 100). Therefore, our activity analysis results arehighly dependent on the accuracy of the activity recognition API.44Chapter 9ConclusionIn this research, we provide the first detailed investigation of how age of smartphoneusers correlates with their unlocking behaviour. We conducted a longitudinal fieldstudy with 137 participants from North America, who installed our study appon their Android phones and ran it for at least 60 days. The results of our studysuggest that age does correlate with certain patterns of smartphone use related tounlocking. In particular, we observed that older users interacted with their devicesfor significantly shorter amounts of time than younger ones, while, at the sametime, they relied more on autolock features. Overall, all participants spent similaramounts of time interacting with their devices per user session, whereas youngergroups were likely to use their devices for significantly more time in total each day.We highlight that, when designing new authentication systems, varying age-relatedusage patterns should be taken into consideration.We also show that about 50% of unlocks happened when smartphone userswere in a state of motion. This indicates that it is important for newly designedunlocking mechanisms to be robust against activities involving movement.Additionally, we found that user interaction differs depending on the unlockingmechanisms they use. Fingerprint users were found to interact with their devicesignificantly more frequently than PIN and Swipe/None users. Our results didnot reveal significant differences in other usage data across all types of unlocking45mechanism.We found that about 62% of participants shared their devices with others, withfamily members and friends being the top two device sharing targets. Finally, weidentified the top five most-accessed apps for all participants and for participantsfrom each age group. Our participants provided low sensitivity ratings for the most-used apps on their devices, which indicates that users are likely to underestimatethe value of data stored on their devices [5]. Last but not least, our study did notshow that participants’ general perception of app sensitivity varies by age.We see two promising areas for future work in this field. First, now that we havea better understanding of how users’ (un)locking behaviours vary under differentcircumstances such as age, it would be interesting to find out why these differencesexist. Furthermore, exploring the understanding of security and usability trade-offsbetween age groups and unlocking mechanisms can potentially shine further lighton users’ decision making processes.46Bibliography[1] D. Buschek, F. Hartmann, E. von Zezschwitz, A. De Luca, and F. Alt.Snapapp: Reducing authentication overhead with a time-constrained fastunlock option. In Proceedings of the 2016 CHI Conference on HumanFactors in Computing Systems, CHI ’16, pages 3736–3747, New York, NY,USA, 2016. ACM.[2] P. R. Center. Smartphone ownership 2013, 2013.[3] I. Cherapau, I. Muslukhov, N. Asanka, and K. Beznosov. On the impact oftouch ID on iphone passcodes. In Eleventh Symposium On Usable Privacyand Security (SOUPS 2015), pages 257–276, Ottawa, 2015. USENIXAssociation.[4] M. Cherubini and N. Oliver. A refined experience sampling method tocapture mobile user experience. In In Presented at the InternationalWorkshop of Mobile User Experience Research part of CHI, pages 1–12,Boston, MA, USA, 2009. ACM.[5] S. Egelman, S. Jain, R. S. Portnoff, K. Liao, S. Consolvo, and D. Wagner.Are you ready to lock? In Proceedings of the 2014 ACM SIGSACConference on Computer and Communications Security, CCS ’14, pages750–761, New York, NY, USA, 2014. ACM.[6] Google. Detected activity types, 2017.[7] M. Harbach, A. De Luca, and S. Egelman. The anatomy of smartphoneunlocking: A field study of android lock screens. In Proceedings of the 2016CHI Conference on Human Factors in Computing Systems, CHI ’16, pages4806–4817, New York, NY, USA, 2016. ACM.47[8] M. Harbach, A. De Luca, N. Malkin, and S. Egelman. Keep on lockin’ in thefree world: A multi-national comparison of smartphone locking. InProceedings of the 2016 CHI Conference on Human Factors in ComputingSystems, CHI ’16, pages 4823–4827, New York, NY, USA, 2016. ACM.[9] M. Harbach, E. von Zezschwitz, A. Fichtner, A. De Luca, and M. Smith. It’sa hard lock life: A field study of smartphone (un)locking behavior and riskperception. In Symposium On Usable Privacy and Security (SOUPS 2014),pages 213–230, Menlo Park, CA, 2014. USENIX Association.[10] E. Hayashi, S. Das, S. Amini, J. Hong, and I. Oakley. Casa: Context-awarescalable authentication. In Proceedings of the Ninth Symposium on UsablePrivacy and Security, SOUPS ’13, pages 3:1–3:10, New York, NY, USA,2013. ACM.[11] E. Hayashi, O. Riva, K. Strauss, A. J. B. Brush, and S. Schechter. Goldilocksand the two mobile devices: Going beyond all-or-nothing access to a device’sapplications. In Proceedings of the Eighth Symposium on Usable Privacy andSecurity, SOUPS2012, pages 2:1–2:11, New York, NY, USA, 2012. ACM.[12] D. Hintze, R. D. Findling, M. Muaaz, S. Scholz, and R. Mayrhofer. Diversityin locked and unlocked mobile device usage. In Proceedings of the 2014ACM International Joint Conference on Pervasive and UbiquitousComputing: Adjunct Publication, UbiComp ’14 Adjunct, pages 379–384,New York, NY, USA, 2014. ACM.[13] A. Mahfouz, I. Muslukhov, and K. Beznosov. Android users in the wild:Their authentication and usage behavior. Pervasive and Mobile Computing,32:50–61, 2016.[14] D. Marques, I. Muslukhov, T. Guerreiro, L. Carriço, and K. Beznosov.Snooping on mobile phones: Prevalence and trends. In Twelfth Symposiumon Usable Privacy and Security (SOUPS 2016), pages 159–174, Denver, CO,2016. USENIX Association.[15] I. Muslukhov, Y. Boshmaf, C. Kuo, J. Lester, and K. Beznosov. Know yourenemy: The risk of unauthorized access in smartphones by insiders. InProceedings of the 15th International Conference on Human-computerInteraction with Mobile Devices and Services, MobileHCI ’13, pages271–280, New York, NY, USA, 2013. ACM.48[16] P. Northcraft. Android: The Most Popular OS in the World, 2014. LastAccessed: June 2017.[17] O. Riva, C. Qin, K. Strauss, and D. Lymberopoulos. Progressiveauthentication: Deciding when to authenticate on mobile phones. InPresented as part of the 21st USENIX Security Symposium (USENIX Security12), pages 301–316, Bellevue, WA, 2012. USENIX.[18] D. Van Bruggen, S. Liu, M. Kajzer, A. Striegel, C. R. Crowell, and J. D’Arcy.Modifying smartphone user locking behavior. In Proceedings of the NinthSymposium on Usable Privacy and Security, SOUPS ’13, pages 10:1–10:14,New York, NY, USA, 2013. ACM.49Appendix ASurvey QuestionsA.1 Device Sharing Survey1. Would you be concerned if someone in your social circle is able to accessthis device without your permission?(a) YES(b) NO2. Since last week, HOW MANY TIMES have you shared this device withothers? (provided with a breakdown number list that user can specify them-selves)3. What kind of people did you share this device with? (Choose all that apply.)(a) Friends(b) Roommates(c) Family(d) Co-workers(e) Unknown people(f) Other (please specify)50A.2 Contextual Survey1. Who can see the content of your screen right now?(a) Unknown person(b) Known person(c) Both(d) Nobody2. Please rate how likely it is that someone is watching your screen right now.(a) Very unlikely(b) Unlikely(c) Neutral(d) Likely(e) Very likely3. Please rate for each app on the list: If this person were watching your screen,and you were using this app right now, how much would it affect yourprivacy?(a) Extremely(b) Very(c) Moderately(d) Slightly(e) Not at all51Appendix BData Collection VisualizationFigure B.1 visualizes the data collection process of this research project fromDecember 8, 2016 to August 10, 2017. The x-axis represents the date that the datawas collected, and the y-axis shows the numeric ID of each participant. Each dotin the figure represents the data collected for a participant on a specific day. Eachhorizontal line represents the entire data collection process for each participant.As shown in Figure B.1, all participants are categorized to four groups, namely“Completed”, “Failed to recover”, “Partial data”, “Withdrew”. While “Completed”means the participants successfully provided us with data for 60 (or more) days,“Withdrew” means the participants stopped participating before the end of the study.“Failed to recover” participants were those whose data were lost due to the back-endserver issue (our server was silently down for 10 days) and were not recovered bythe end of the study (August 10, 2017). Participants who were categorized to the“Partial data” group were those whose data were not sent successfully to the serverdue to unknown issues related to their devices.520100200Jan Apr JulDateParticipant IDParticipant TypeCompletedFailed to recoverPartial dataWithdrewVisualization of Data Collection ProcessFigure B.1: A Visualization of Data Collection Process from December 8, 2016 toAugust 10, 2017, N = 276. Total # of Participants Who Completed the Study:142; Total # of Participants from Whom We Failed to Recover Data: 43; Total #of Participants from Whom We Received Partial Data: 7; Total # of ParticipantsWho Withdrew from the Study: 8453