UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Email flow awareness and deferral strategies Siu, Nelson 2006

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata


831-ubc_2006-0109.pdf [ 6.9MB ]
JSON: 831-1.0064895.json
JSON-LD: 831-1.0064895-ld.json
RDF/XML (Pretty): 831-1.0064895-rdf.xml
RDF/JSON: 831-1.0064895-rdf.json
Turtle: 831-1.0064895-turtle.txt
N-Triples: 831-1.0064895-rdf-ntriples.txt
Original Record: 831-1.0064895-source.json
Full Text

Full Text

Email Flow Awareness and Deferral Strategies by Nelson Siu B.A.Sc, University of British Columbia, 2003 THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED SCIENCE in The Faculty of Graduate Studies (Electrical and Computer Engineering) The University of British Columbia February 2006 © Nelson Siu, 2006 A B S T R A C T The investigative methods traditionally employed by email researchers do not adequately capture the intricacies of email use over an entire day. This thesis improves upon these methods by using a novel combination of daylong in-situ shadowing and passive interface monitoring to provide new insights into how new messages are handled. The findings revealed previously undocumented actions for dealing with email on an ongoing basis, and showed that deferred message handling is a critical usage strategy that has been understudied. Interface logs were used to explore the defer action further and identified two types of email deferral: intra-session and inter-session deferral. This investigation highlights the frequent use of email as a real-time task and attention management tool, and demonstrates why this usage should be supported within email clients. This thesis finally makes the recommendation that future interface improvements should provide more flexible and lightweight mechanisms for integrating email into the user's workflow. ii T A B L E OF C O N T E N T S ABSTRACT ii LIST OF TABLES vi LIST OF FIGURES vii ACKNOWLEDGEMENTS viii DEDICATION x CHAPTER 1 INTRODUCTION 1 1.1 SIGNIFICANCE 2 1.2 RESEARCH GOALS 3 1.3 ORGANIZATION OF THIS THESIS : 3 1.4 CONTRIBUTIONS OF THIS THESIS • 3 CHAPTER 2 BACKGROUND AND PREVIOUS WORK 5 2.1 EVOLUTION OF EMAIL USAGE 5 2.1.1 Diversity in Email Use -.-5 2.1.2 Email Overload 6 2.1.3 User Motivations 7 2.2 IMPLEMENTATION ORIENTED RESEARCH 8 2.2.1 Novel Visualizations •• • 8 2.2.2 Automated Classification 9 2.3 EMAIL SOCIAL NETWORK RESEARCH 9 2.4 SUMMARY H 2.5 THESIS Focus • 12 CHAPTER 3 OBSERVATIONAL STUDY OF EMAIL 13 3.1 MOTIVATION • 13 3.2 METHODOLOGY 16 3.2.1 Early pilot -J<> 3.2.2 Full Day Shadowing 19 3.2.3 Data Analysis : 22 3.3 FULL D A Y OBSERVATIONAL SESSION RESULTS 24 iii 3.3.1 Glance , 24 3.3.2 Scan 25 3.3.3 Defer • 26 3.4 DISCUSSION 29 3.5 CHAPTER SUMMARY , 32 CHAPTER 4 PASSIVE INTERFACE MONITORING STUDY 34 4.1 MOTIVATION .....34 4.2 METHODOLOGY 35 4.2.1 Monitoring Tool • 35 4.2.2 Participant Selection '• 36 4.2.3 Study Protocol • 38 4.2.4 Data Analysis 40 4.3 RESULTS 45 4.3.1 Context Independent Actions on New Messages 45 4.3.2 Motivational Factors for Deferred Message Handling..: 50 4.4 DISCUSSION 55 4.5 CHAPTER SUMMARY 57 CHAPTER 5 DESIGN RECOMMENDATIONS. : 58 5.1 NEW MESSAGE STATES '. » 58 5.2 LIGHTWEIGHT vs. HEAVYWEIGHT SUPPORT 59 5.3 SUPPORTING EMAIL FLOW ; ; 60 5.4 CHAPTER SUMMARY • 65 CHAPTER 6 CONCLUSIONS AND FUTURE WORK :.. 66 6.1 RESEARCH GOALS SUMMARY • 66 6.2 CONTRIBUTIONS OF THIS WORK 67 6.3 FUTURE WORK • 69 6.3.1 Methodological Refinements 69 6.3:2 Follow-up Research •• 70 iv 6.3.3 Long Term Focus 77 6.4 CONCLUSION 71 REFERENCES 72 APPENDIX A - DATA ANALYSIS 75 V LIST OF T A B L E S TABLE 1 - VENOLIA'S EMAIL CONCEPTUAL MODEL : 13 TABLE 2 - PILOT STUDY SUBJECT SUMMARIES 17 TABLE 3 - FULL DAY STUDY SUBJECT DESCRIPTIONS ...„ 20 TABLE 4 - SUMMARY OF DATA CAPTURED BY THUNDERBIRD MONITORING EXTENSION 36 TABLE 5 - STUDY SUBJECT SUMMARY 37 TABLE 6 - USER INSTALLATION CUSTOMIZATIONS 39 TABLE 7 - EVENT SEPARATION THRESHOLDS USED TO ESTABLISH THE ACTIVITY SESSIONS 42 TABLE 8 - TIME SEPARATIONS FOR SESSION LEADING SEND EVENTS 43 TABLE 9 - TIME SEPARATIONS FOR SEND EVENTS WITHIN SESSIONS 43 TABLE 10 - LOGGING DATA SUMMARY 44 TABLE 11 - SUMMARY OF NEW MESSAGE HANDLING ACTIONS 46 TABLE 12 - COMPARISON OF REPLY PERCENTAGES AMONGST SUBJECTS 48 TABLE 13 - COMPARISON OF DEFERRALS AMONGST SUBJECTS 48 TABLE 14 - USAGE COMPARISON FOR THE TWO DEFERRAL TYPES 49 TABLE 15 - INTRA-SESSION DEFER TIMES 49 TABLE 16- INTER-SESSION DEFER TIMES 49 TABLE 17 - TYPES OF ACTIONS OBSERVED DURING AN INTRA-SESSION DEFER 52 TABLE 18 - COMPARING IMMEDIATE REPLY AND INTRA-SESSION DEFER COMPOSE TIMES 53 TABLE 19 - INITIAL EMAIL ACTION CLASSIFICATION 75 vi LIST OF FIGURES FIGURE 1 - INTEGRATED EMAIL WORKFLOW MODEL FOR NEW MESSAGES ; 29 FIGURE 2 - N E W MAIL NOTIFICATIONS ON THUNDERBIRD FOR O S X AND Y A H O O WIDGETS 30 FIGURE 3 - COLOUR CODING FOR RAW LOG EVENTS 41 FIGURE 4 - EVENT SEPARATION THRESHOLD CALCULATION 41 FIGURE 5 - ACTIVITY SESSIONS HIGHLIGHTED IN "EVENT TIMESTAMP" COLUMN 42 FIGURE 6 - AUTOMATED SCRIPT IDENTIFIES IMMEDIATELY HANDLED MESSAGES 44 FIGURE 7 - COMPARISON OF NEW MESSAGE DEFERRAL METHODS 47 FIGURE 8 - SINGLE MESSAGE HANDLING MODEL 55 FIGURE 9 - EMAIL SCRATCH AREA 62 FIGURE 10 - SCREEN MOCK-UP OF THE "SEND WITH ALERT" FUNCTION 64 FIGURE 11 - MOZILLA DATA LOGGING EXTENSION ARCHITECTURE 77 vii A C K N O W L E D G E M E N T S This thesis has been a long journey and a wonderful time of growth for me. I certainly would not be where I am now without the help of so many, and I owe these people a huge debt of gratitude. I am glad that I have a chance here to thank some of these people. To my supervisor Lee, I thank you for the wisdom that you have imparted upon me. I thank you for exposing me to research and the incredible possibilities of discovery. I thank you for your patience, encouragement, and grace as I struggled through those hard long months, and I thank you for believing in me during times when even I might not have believed in myself. To Sid, for welcoming me into your lab and always being there for me when I needed, a new perspective on things. Your passion for research is infectious, and you are such an inspiration. To Bill, for trusting me and giving me the opportunity to pursue my love of teaching. I will never be able to thank you enough for your unwavering confidence in me. To Tony, thanks for always being there for me. You have been such an amazing friend and mentor to me from day one. I could not have finished this thesis without your insight and encouragement. You are my hero. To all my friends in the HCT lab, Meghan, Phillip, Florian, Ian, Reynald, Steve, Changsong, Chris, Amir, Troy, Robert, Kento, and Gabriel, thanks for all the great times. viii To Patrick, Linus, and Mark for your work on the early stages of the interface logging tool. Without your help I could not have waded through the maze that is Mozilla. To Dave and Hugues, for the great times we had on the court, and to Ed, who I could always rely on for a good laugh to lighten my day. To the gang, Max, Larry, Wai Tao, Michael, Sassan, Erin, Zion, Alex, and Larix for being there for me. And a special thanks must go out to Max, who spent countless hours going over every word in this thesis. ix Dedicated with Love CHAPTER 1 INTRODUCTION Email is an important communication medium and integral part of everyday life. Therefore, studying email to improving its efficiency and effectiveness has the potential to benefit a great number of people and organizations [37]. Previous research has documented the tight integration of email into people's workflows to the point where it has become more than just a communication system [23]. Email serves as a "habitat" [14] where people also engage in task management [5], file transfer, and information storage [36]. Unfortunately, the increased reliance on email also brings with it the problem of information overload [12,36], and client interfaces have failed to keep pace with this changing environment. Researchers have developed a good understanding of broad email usage patterns. We know for example that users leave messages in the inbox as reminders and avoid filing them when possible [36]. Other studies have explored when users decide to respond to email [32], how users handle a large influx of messages [25], and how email affects organizational dynamics [28]. This body of knowledge however has very few specifics on how to design an interface that will help with the overload problem and support the various ways people use email. Modern email networks already rely heavily on filters to exclude spam messages, and it is conceivable that in the near future, users will be bombarded with so much information that they will require some form of automated assistance to filter even their relevant emails. Questions such as how a user determines the importance of a message, or whether the order that emails are handled is 1 linked to its perceived importance will have to be answered before these improved clients can be designed. Existing techniques for studying email however are not completely suited to answering these questions. 1.1 Significance Email studies to date usually employ a mixture of short interviews and questionnaires to understand what users are doing. These techniques suffer from several drawbacks. For example, short interviews are not well suited for studying interactions throughout the day, and as Chapter 3 will show, the presence of the investigator may very well affect the email actions under study. Questionnaires are often used to gather data on aspects of email that are hard to observe directly, such as how the decision to reply is arrived at [10], or what the user is thinking when they take care of email in the morning [25]. This assumes that users are fully aware of their own actions when answering, which cannot be guaranteed when dealing with some subtle actions that the user takes for granted. The work contained in this thesis represents a novel approach to email research that will explore techniques more appropriate for studying email over a medium to long period. These techniques are improvements in methodology that will address the important problem of investigator intrusion, and attempt to capture email usage in its most natural state. These improvements will in turn reveal new usage actions that have a direct impact on the design of new client interfaces, and add to our understanding of email handling and prioritization strategies. 2 1.2 Research Goals The goal of this thesis is to improve the understanding of email handling strategies throughout the entire day. This study will gather detailed email interaction data through a combination of daylong in-situ observations and passively recorded interface logs. This thesis is framed by the following three questions: • How can existing email research techniques be improved upon? • Are there undocumented interactions that current methods of studying email miss? • How can clients better support these usage patterns? 1.3 Organization of this Thesis This thesis is organized as follows. The background and related work on email is first presented in Chapter 2. Chapter 3 discusses a series of daylong observation studies that describe email usage throughout the day from a qualitative perspective. Chapter 4 presents the continuation of this work using an interface logging tool that records the subject's email interactions over a month long period. The design recommendations resulting from these two studies are discussed in Chapter 5, followed by conclusions and recommendations for future work in Chapter 6. 1.4 Contributions of this Thesis My contributions in this thesis: • I will show that the novel application of daylong in-situ shadowing in conjunction with interface logging is well suited for conducting email research. • I will identify three previously undocumented strategies for dealing with incoming email. These include glance, scan, and defer. 3 • I will identify delayed handling as an integral aspect of email use with clear implications for interface design, along with two previously undifferentiated modes of email deferral: intra-session deferral, and inter-session deferral. • I will demonstrate that task management is deeply embedded into people's ongoing email practices, and illustrate how the current understanding of this usage pattern is incomplete. • I make the recommendation that future interface improvements will need to provide more flexible, and lightweight ways with which to integrate task management into people's email environment. 4 CHAPTER 2 BACKGROUND AND PREVIOUS WORK Email is a ubiquitous communication medium in modern society and one of the most successful computer applications ever devised. As a result, researchers have examined email as a topic of study for many years. In this chapter, three major areas of email research are presented. We will first review the work on understanding user intentions and documenting email usage in the workplace. This is followed by a treatment of the work on novel interfaces, along with the results of recent research on the relationships contained within email social networks. 2.1 Evolution of Email Usage By 1982, the volume of incoming email was beginning to overwhelm many users. Denning wrote in [12] about "The Receiver's Plight", in which he described his frustrations with the constant barrage of new messages, and posed the question of how users can be expected to deal with the ever increasing amount of email. We are still grappling with this problem over twenty years later in a world where email has become even more important and pervasive. 2.1.1 Diversity in Ema i l Use Researchers wanted to understand how users coped with the deluge of messages as email traffic grew during the mid 80s and 90s. Mackay showed in [23] how email was more than just a communication system in that it also supported both time management and task management activities. It was found that email use was incredibly diverse, with users varying in both their 5 message prioritization strategies and their willingness to expend effort organizing information. Mackay suggested that any tools designed to support email should try to provide a great deal of flexibility in its affordance, rather than addressing only a specific problem or mode of use. She also found that people fall into two categories for organizing incoming messages: prioritizers and archivers. Prioritizers are concerned with minimizing their time processing email and are comfortable using filters to classify new mail, even at the risk of missing important messages. Archivers prefer to see every message in order to ensure an important one does not slip by, and feel uneasy about permanently deleting messages in case it was needed again. 2.1.2 Emai l Overload Over time users began to receive more messages than they could handle, a problem that is compounded by the lack the tools designed to manage this influx of information. Whittaker showed in [36] that email clients were being creatively employed to support task management and the archiving of information. The term email overload was coined by Whittaker to describe this phenomenon of co-opting email for activities that it was never designed to support. The study also looked at how people filed their messages, classifying them as no filers, spring cleaners, and frequent filers. No filers left most of their messages in the inbox and used the search function to retrieve them. Spring cleaners dealt with their inbox clutter with intermittent clean-ups every few months and had largely ineffective filing schemes. Finally, frequent filers made daily passes through their inbox to file or delete messages and were the group that had the most success when it came to retrieving messages from their folders. People used clients in new unforeseen ways as email became more embedded in their work practices and communication. Ducheneaut showed in [14] that email has evolved so much that it 6 effectively served as a habitat in which users spent a good deal of their day. It was a hub from which users communicated, collaborated, and delegated their work. The message box metaphor of email was not equipped to handle these additional duties and did not do a good job of supporting these activities. 2.1.3 User Motivations Building on the work of Mackay and Whittaker, researchers have tried to understand the motivations behind various email handling strategies. Tyler and Tang used a series of interviews to identify what affected a user's email responsiveness and their propensity to reply in a timely manner [32]. Key factors included their familiarity with the author, the urgency of the message, and the interaction history with the contact. Users were keen to manage their responsiveness image with their correspondents, choosing to reply promptly if the other party was expected to do the same, or delaying a response purposely to avoid the perception of availability on a particular matter. Dabbish in [10] attempted to characterize the motivations for taking action on a message by surveying a large number of users with questionnaires. It was found that the importance of a message rises if it was work related or if it required action by the user. Interestingly, the importance of a message only had a modest impact on the likelihood of a reply. Dabbish posits that additional factors must be at play when people determine how they will handle email, and that a number of contextual circumstances are likely involved in the user's reply decision process. 7 2.2 Implementation Oriented Research In response to the issue of information overload and the task management role that email has taken on, a number of projects have tried to overhaul the email interface [1,3,5,7, 15,18,20,22,25,26,27,34,39]. The majority of these systems have focused on creating novel visualization techniques for email, while others looked into the feasibility of automated message importance analysis. 2.2.1 Novel Visualizations A strong theme in email visualization research has been the drive to support task-based operations within clients. Belotti et al. followed up their work in [14] with the Taskmaster system [5], in which the inbox was augmented to support a task-centric view of the user's workspace. Users were able to group emails by its relationship to active tasks, track and update the progress of various tasks, or group contacts by their involvement in particular projects. Gwizkda also examined the effectiveness of viewing email as tasks like the Taskmaster system, but supplemented it with a timeline denoting pending action within each email [18]. The Taskview interface displayed tasks in a vertical column with future date references marked on a horizontal timeline. Users were able to identify task items quickly and cross-reference their deadlines by scanning this timeline. The work was based closely on the Timestore system by Yiu et al. [39], which also presented emails with a horizontal timeline but organized the interface based on the sender instead of pending tasks. Other researchers meanwhile integrated novel visualizations into prototype clients while maintaining the message metaphor of current systems. A number of these projects examined the 8 issue of thread visualization. Rohall et al. in their ReMail prototype incorporated an interactive visualization that depicted a message's relationship with other messages in a view called a thread map [26]. A node in the thread map represented each individual message in the thread, while the edges in the graph showed the connections between these messages. Venolia's work on email workflow also lead to a thread based browser prototype called Grand Central, which allowed users to view threads at varying levels of detail using two adjacent panes [34]. The motivation for these thread-centric features came from the realization that users were frustrated by the scattered nature of the normal inbox, where multiple messages from the same thread can be interspersed amongst unrelated messages. 2.2.2 Automated Classification A number of projects have attempted to apply machine-learning techniques to the problem of excessive emails. Agent-based systems have been created to summarize email content [1], classify email as work related [7], and determine email importance in order to prioritize its presentation to the user [20]. The goal of these systems was to help users deal with as many emails in as short a time as possible. All of these systems have been one off prototypes that never matured beyond a research stage, and none have been widely deployed outside of a lab environment. The highly fluid and contextual nature of email makes it very difficult for an agent to learn a user's preference, which in turn has limited the success of automated email assistance thus far. 2.3 Email Social Network Research Email is a highly social medium, and as.such, email archives represent a rich source of information about people's relationships and their interactions with those around them. A 9 growing number of researchers have looked to studying email social networks to try to understand the patterns contained within. For example, Tyler [32] found it was possible to infer the individuals in positions of leadership in a company, as well as discover the informal collaborative communities within an organization by capturing the email exchanges between groups of users. Interviews conducted with members of the monitored teams found that the clustering was indeed accurate, and representative of the actual people they dealt with regularly. Farnham's Personal Map took the approach of depicting a user's social circle from their own personal email archive, providing an egocentric view of their network [15]. An interface presented the user's contacts in a radial arrangement with "important" contacts close to the center. This measure of importance was derived from the contact's tendency to appear in the email archive. The interface also offered suggestions for contacts to include in the recipient list based on the network when composing new messages. Whittaker et al. investigated the use of a social desktop representation for organizing communication and information functions in Contact Map [38]. The interface displayed user contacts in representative groupings according to email interactions. Each contact was represented by a node that provided direct access to the emails, documents, and web pages associated with that person. Users were able to manipulate the grouping of these nodes according to their preference, and reported that they found the Contact Map useful for finding frequent contacts in comparison to traditional email address books. 10 2.4 Summary Email is unique because its asynchrony affords the user a level of control not found in other electronic communication mediums. Users are able to attend to emails on their own terms, rather than being "pushed" to initiate communication as with instant messaging or the telephone. This property makes email a natural environment within which to embed activities such as task management and scheduling. Many researchers have examined the possibility of supporting these activities more directly, although the low success rate of these systems point to a need to re-examine what we know about email use before embarking on future designs. The failure of recent feature additions to email clients highlights a disparity between the designer's system model and the user's mental model. Consider for example the disuse of prioritization flags and read acknowledgement receipts [36]. These features were designed to improve message prioritization and awareness. Instead, sender-based prioritization turns out to be of little value to the receiver because it rarely reflects the message's actual importance to them. Likewise, most users never acknowledge reading receipts because it reveals the time when a message is read. Users value their ability to handle messages at their leisure, and are keenly aware of the image they project to others [32]. Revealing to the sender when a message is opened places an expectation on the recipient to handle the issue, setting up a scenario that may be unrealistic for the user given their situation. Future interface improvements need to be driven by a deeper understanding of email use. As Mackay pointed out in [23], any tools for supporting email must account for individual variations in usage, but not be so specific that it loses general applicability to email users. For 11 example, the fact that many users leave their task related messages in their inbox instead of filing them into a "pending" folder suggests that they view task tracking as a lightweight operation. The proposed solutions to the task management problem have usually required the user to explicitly identify tasks or attempt to enter them into a calendar [5,18,26]. These actions require a much higher level of involvement than commonly used strategies like re-flagging a message as "unread." The proposed systems also do not provide enough flexibility to allow for general sorting, grouping, or annotating of the tasks items so that users can customize the solution to suit their own needs. Knowing how much effort a user is willing to expend for a particular task directly affects whether a new feature should be implemented using a heavyweight or a lightweight approach. The data needed for this level of understanding will require novel investigative methodologies beyond the interviews or questionnaires that past studies have employed. This thesis will attempt to explore what these methodologies may be. 2.5 Thesis Focus The goal of this thesis is to improve upon existing email research techniques and in turn use the new methodology to help understand how users prioritize email handling. Building upon the existing work described in Section 2.1, Chapter 3 describes a set of qualitative interviews looking at the email actions employed by users throughout the day. Chapter 4 presents the results of a passive email interface logging tool that investigates email usage on a message-by-message basis. Finally, Chapter 5 and 6 proposes a set of design implications based on the work done in this thesis and concludes with recommendations for future work. 12 CHAPTER 3 OBSERVATIONAL STUDY OF EMAIL1 In this chapter, a series of full day observational studies of email users are discussed. Based on these observations, a set of common strategies for managing continuous incoming email is identified. When stationed for long periods at their computer, we find that users glance, scan, and defer their messages on an ongoing basis to manage their email attention budget. The following sections will discuss why this type of usage should be examined, how it improves upon previous studies that observed users over only a short duration. 3.1 Motivation In [35], Venolia presented a conceptual model of email workflow consisting of five different activities: flow, triage, archive, retrieve, and task management. Activity Description Flow Keeping up with incoming messages as they arrive Triage Dealing with email that accumulated while the user was away Archive Storing email for subsequent retrieval Retrieve Accessing previously archived email Task Management Using email to as a task list and as a reminder mechanism Table 1 - Venolia's email conceptual model ' I would like to thank Anthony Tang for his help in conducting the daylong shadowing study and for his contributions to the ideas in this chapter. 13 Of the five areas, there have been many past studies dealing with email triage, archiving, and retrieval [2,8,10,14,25,32,34,36]. The least studied of these five activities is email flow, which is surprising since users often spend the entire day with their email clients opened [14]. This knowledge gap exists perhaps because of how difficult it is to obtain email usage data over periods of days or even weeks. There is relatively little empirical data in the literature detailing exactly how users handle new mail arrivals. Gathering this type of data is crucial because it can determine how interface improvements can better support users who utilize email continuously. Tracking a user's email actions throughout the day is essential to understanding email flow. This requires an investigative method that can collect data continuously over an extended period. Email studies to date have usually used the following techniques for gathering data: targeted interviews lasting one to two hours, self-report questionnaires, or offline inbox analysis. These methods have been useful for answering questions the.researchers had at the time, such as "How do people go about triaging mail [25]", or "Which email does the user find important to reply to [10]?" However, they fail to capture the contextual dynamics of email use throughout the workday. For example, several studies have used short in-situ interviews at the beginning of a user's day to observe triage behaviour [14,23,25]. Unfortunately, this method removes email activities from the broader work context and does not capture the balance between monitoring email and focusing on a core task (e.g. composing a report). In an attempt to solve this problem, some have utilized self-report questionnaires to study email usage during all parts of the day [10]. Questionnaire studies can be very effective for probing a user's perception of email and what 14 they consider as factors in determining message importance. However, they assume the user is retrospectively cognizant of most or all of their email activities. This assumption is likely to breakdown when studying transient or subtle email monitoring strategies since these actions could be taken for granted or be viewed as irrelevant by the user. Finally, researchers have analyzed inbox files offline to examine a range of issues from email social networks [15,32,38] to email filling [36]. As with all the methods listed above though, this technique cannot provide the data that this thesis is interested in: a user's continuous email monitoring strategies. Therefore, there is good reason to believe that a full day in-situ observation of users may be able to extract novel transient email behaviours not previously found using short interviews, questionnaires, or inbox analysis. 15 3.2 Methodology To inform the design of the full day study, a series of 30-minute exploratory interviews were performed to determine how comfortable users were working with their email in the presence of an investigator. An addition goal of the pilot study was to investigate how users assess the importance of messages and whether this affected the order the messages are handled. Even though the results did not provide any significant new insight into the prioritization process, it did highlight the need to move forward with a full day study, as the investigator's presence at times appeared to have a significant effect on the subject's reading behaviour. This section discusses the results from the pilot study and the observational protocol for the subsequent full day study. 3.2.1 Early pilot Five volunteer participated in the short 30-minute pilot interviews (Table 2). The participants were recruited from a pool of university staff and graduate students that were easily accessible to the investigator. Email volumes varied from user to user with some receiving only 5-10 emails a day to some receiving over 60 a day. Users were asked to avoid checking their email before the interview. The interview began with a set of general usage questions such as when did they check their mail and whether they monitored for new messages throughout the day. The investigator then watched the user perform triage on their new messages and asked clarifying questions about what was observed only after the triage was finished. Notes were taken throughout the interview with a focus on how email importance was arrived at. An audio recording of the entire interview was made for reference. Gender-appropriate pseudonyms will be used from here on to preserve anonymity. 16 Name Description of Pilot Participant Emeril • Graduate student • Receives 5-7 emails a day • Email client: ISP webmail portal Melanie • Co-op program coordinator •. Interacts with coop students and employers primarily via email • Day consists of a mixture of "desk time" coupled with site visits and meetings • Receives 30-40 emails a day • Email client: MS Outlook Anthony • Graduate student • Receives 10-15 emails a day • Email client: A mixture of webmail and MS Outlook Ian • Department Technician • Coordinates purchasing, shipping, and receiving mostly via email • Receives 30-40 emails a day • Email client: MS Outlook Lisa • Graduate Student • Receives 15-20 emails a day • Email client: Department webmail portal Table 2 - Pilot study subject summaries The findings of the pilot study supported the earlier results of [14,36] which stated that users seldom filed emails, often referred to their archives as an information store, and utilized emails as an ad-hoc list for task management. The participants employed both single-pass and multi-pass reading styles during email triage [25]. With single-pass reading, users first scan the new arrivals in chronological order (or reversed chronological order as some users preferred). Messages requiring a response are handled immediately before moving on to the next email. On the other hand, multi-pass users first remove less important or junk messages from the unread list on their initial pass. Messages that can be quickly replied to are also handled at this stage. Then 17 on their second pass, the user rescans the list to catch the important messages that may require more time to reply to (now more easily seen). Pilot participants were queried about the importance of their messages after the triage. They were asked how they determined which messages were important, and whether it affected the order they read the messages. All the participants cited the author and subject line as the way they determined whether to read a message, though the context of their personal situation was closely linked as well to the decision process. For example in the case of Melanie, she placed the highest importance on emails from potential co-op employers during job placement periods. These emails may contain employment offers or requests to post jobs with the office. These would be deemed of utmost importance, followed by emails from her co-workers, and then other miscellaneous messages. The other participants also had similar responses that were specific to their own individual circumstances. Users do not always read messages in the order from most important to least important as intuition may suggest. For example, Ian receives approximately 40-50 emails a day from faculty, graduate students, and undergraduates who need to make purchases or inquire about the order status for a piece of equipment. Often these requests are highly time sensitive. Given this, Ian was observed during the interview reading all his new emails in sequential order from oldest to newest. Upon questioning, he revealed that it was simpler for him to go through everything in a single pass so that he did not miss anything important, and reading his mail this way ensured that no messages slip through. He believed that if he read his messages out of order based on their importance, then his inbox would become a more confusing mix of read and unread emails. 18 The most significant result gathered from the pilot was that the investigator's presence significantly affected the observation results. In one instance I asked Melanie whether she always reads her messages sequentially in chronological order. To my surprise, she said that in fact she did not usually do so. Normally on that day she would have gone straight to certain high priority messages and then left for a staff meeting within 5 minutes. She would then complete the triage process upon returning to her desk. The formal scheduling of the interview and dedicating time to interact with the investigator had affected the participant's normal email usage. At this point I took a step back and tried to devise new methods of studying email that minimized investigator intrusion, leading to the full day shadowing technique discussed in the section below, and the passive interface monitoring technique discussed in Chapter 4. 3.2.2 Fu l l Day Shadowing A full day shadowing has advantages over short scheduled interviews because the participant can be observed using email over a wide range of situations. This is desirable because there may be modes of email usage that have gone undocumented due to the limitations of previous investigative. methods. A full day study may still suffer from the problem of investigator intrusion, but it is hoped that by being a passive observer, the investigator's presence will diminish as the participant goes about their normal work routine. The use of daylong in-situ shadowing for studying work practices is very common in fields such as anthropology or organizational behaviour [6,24], and it has been well demonstrated that this type of inquiry is well suited for collecting naturalistic observations with the aim of improving interface design [21]. The following sections will detail the subject selection process, the observation protocol used, and the data analysis procedures used for the study. 19 Participant Selection Participant solicitations for the daylong shadowing study were sent via a targeted email broadcast. One female and three male volunteer with backgrounds from both industry and academia agreed to participate in the study. The participants held the positions of a research lab manager, an IT administrator, a software development manager, and a university administrative office worker. All used email as a core communications medium for their work, with each receiving over 50 emails per day. Each subject received $10 in remuneration for participating in the study. Gender-appropriate pseudonyms have been used to provide anonymity while referring to the participants. Name Description Flora • Univers i ty administrative office worker • Access ib le to students and faculty • Inbox is open all day (no folders) Larry • L e a d program manager at a large software firm • Entire day "in meetings or doing email, sometimes both—doing email in meetings" • Inbox open all day ( P C , laptop), and checks email on a SmartPhone as well Owen • Head IT administrator at software f irm • Manages team o f four, delegating tasks by email being a core task • T w o inboxes open all day (Windows P C , L i n u x P C ) , and checks emai on Blackberry Will • Univers i ty research lab manager • IT troubleshooting, inventory control • Inbox open all day (laptop) Table 3 - Full day study subject descriptions Observation Protocol The goal of the study is to observe email handling behaviours in as natural a form as possible. Throughout the daylong session, the investigator strives to maintain a low profile and not become a distraction to the participant. Each observation session consists of three components: 20 • Interview - participants are asked to describe their job function and the way email integrates into their communication and workflow. They are also asked to describe anything they do with email that they felt may be of interest to the investigator. • Shadowing - In this part of the session, the investigator falls into the background and observes the participant as they go about their day. Occasional questions from the investigator are used to clarify interesting actions, but this is done very sparingly to minimize the disruption to the participant. • Questionnaire - The questionnaire collects basic demographic information about the user and their email usage. It is given to the user 30 minutes before the end of their day. After the initial interview, the researcher assumes a passive posture behind the user. An audio recorder is running at all times unless the participant objected to being recorded (Larry and Owen). The investigator is positioned far enough away to ensure the text on the participant's desktop is unreadable, but close enough to observe on-screen actions and gestures. The investigator shadows the participant at all times except during lunch or restroom breaks. If the participant attends meetings where they could access email, the investigator will shadow there as well if none of the meeting participants object. During the study, some participants verbalized their actions as they performed them even though none were explicitly asked to think aloud. In these instances, the investigator attempted to minimize any ensuing conversation in order to allow the user to return their focus to the task and away from the investigator. Detailed notes covering the user's email actions are taken on a minute-by-minute basis during the shadow phase. Tasks that appear to stem from an email or result in email generation are 21 noted as well. The participant is never queried about the specifics of their messages or tasks, and the notes are generated only from what the investigator can observe or infer. During periods when the participant is occupied with a task irrelevant to the study, the observer will take the opportunity to note the participant's workspace configuration, their interaction with other work colleagues, and any piece of information that may place the participant's email usage in better context (e.g. the participant's company has a branch office in another time zone). Although the participant is not questioned about any personal, project specific or proprietary information, they do often perform interesting actions with email that is worth following up on. In these cases, the researcher queues up the questions as they arise, and verbally asks the user only when they are clearly resting. These queries may occur no more than once an hour to maintain the investigator's low profile. After each session, the notes on a participant are reviewed before the next one is interviewed. A list of interesting phenomena to look out for at subsequent sessions is generated after each session. Oyer time, well-established activities are given less attention and more time is spent observing new emerging themes. 3.2.3 Data Analysis The notes and audio recordings from all the sessions were reviewed to gauge the types of actions the participants used to deal with email during the day. Since this research is interested in email flow (see Section 3.1), specific focus was given to identifying actions that provided information about new message arrivals, were related to handling new messages, and showed how attention for email was balanced with attention for the primary work task. No transcripts were made of 22 the audio recordings as these were used solely for confirming specific quotes by the participants. The data analysis consisted of writing the noted actions on a whiteboard and grouping similar actions into clusters until a coherent theme emerged. This method is very similar to the affinity diagramming analysis method [19], except that a whiteboard was used due to convenience instead of small pieces of paper during the grouping phase (See Appendix A). The groups were iteratively identified and the analysis refined until clear themes emerged. 23 3.3 Full Day Observational Session Results The long duration shadowing revealed three distinct actions that users employ to deal with incoming email in between triage sessions. Participants use the glance, scan, and defer actions to maintain awareness of their email flow and to balance their attention between email and their main work task. Users transition smoothly between these strategies during extended periods at their workstation. This section presents the distinguishing characteristics of each action, coupled with anecdotal examples taken from the raw observation data. 3.3.1 Glance Participants often glanced at their inbox pane to maintain an awareness of incoming messages. This interaction is very brief, often lasting less than a second, and is employed when one is deeply involved in a non-email work task. In line with the existing observations [20], our participants did not make use of any audio notifications when new mail arrived because it was too distracting. At most, the email client would display an icon in the system task tray or a small transparent popup in the lower right screen corner with a digest of the message for two seconds. Since these methods do not reveal how many new messages are in the inbox, participants glance at the entire inbox pane to gauge how many new messages have arrived. The glance action is a lightweight form of email use because it is brief, opportunistic, and convenient. Many users keep their email client open at all times [14], and as they switch between applications on their desktop, they may also take a fleeting glimpse of the inbox. Due to the extremely short duration of the glance action, it is likely unable to provide an exact count 24 of the number of new emails in the inbox. Instead, the participants are likely trying to get only a rough sense of how new messages there are. 11:21 am: Flora is working on a paper task. As she reaches for the "Sign Here" sticky notes, she glances at her email client, which has been left open and visible. The email client has 7 unread messages. Flora mutters "Holy smokes, " and stops working on her paper task to scan through her inbox. This course level of awareness may be enough because the user is probably only interested in the rate of incoming email, with sudden fluctuations indicating some "emergencies." 11:30 am: Larry's inbox suddenly "hiccups," scrolling down with 10 new unread emails. Within 5 seconds, Larry minimizes his current window, and opens up the newest email, which is an issue that needs to be resolved within the hour. Participants will sometimes initiate email triage directly because of the glance, but the most common transition from a glance is to a scan of the inbox. This occurs when the information from a glance gives the users enough motivation to escalate their email involvement. 3.3.2 Scan A scan of the inbox provides the user with an awareness of emails requiring immediate attention. Scans require a higher level of cognitive involvement than glances, and they often occur at the task boundaries when the user completes one task and is regrouping for the next one. While scanning the inbox, the user primarily attends to two aspects of the email message: the author, and the subject line. Messages that are expected replies of earlier messages will likely be identified at this stage, as well as any unexpected messages from important people (e.g. direct supervisor), or messages about urgent tasks (e.g. network failure emergency). 25 Unlike the opportunistic glance action, scans have been observed to be a deliberate action lasting approximately 5 to 30 seconds. Scans can occur frequently during the day, with some participants performing it as much as twice an hour. Efficiency seems to be an important factor as scans tend to take place during short breaks in the workflow. This is the likely reason only the author and subject lines are examined, with very few examples of participants using the preview pane as part of the scanning action. Participants only scanned the area of the message list that contained unread or unhandled messages. Unless an important email was spotted, participants returned to their non-email work upon completion of the scan. In the event a scan turns up many important messages, the user may take the opportunity to make a complete pass of the inbox and take care of many messages in quick succession. This "triage-like" usage is different from a full triage session (e.g. at the beginning of the day, returning from a meeting). The mindset of the user when performing such a quick pass is focused on returning to their pending non-email work as soon as possible, whereas a full triage session's time is totally focused on email being the primary work task at that moment in time. 3.3.3 Defer A defer action is used to identify emails for later revisit. These emails can be considered the user's "overflow" messages: messages that cannot or should not be dealt with now. While the participants were generally good at keeping up with their emails in real-time, there were signs that pointed to creative strategies being employed to flag messages for later action. For example, participants were seen "re-flagging" read messages as unread to signify additional work was needed to handle the message [36]. On some desktops, there were messages spawned into standalone windows separate from the inbox, while other users had half-written replies opened 26 that were not in their focus of attention. All these actions point to a strong desire or need to queue up emails for subsequent processing. Deferral is a strategy used by many people in other aspects of their lives outside of email. The prominent use of Post-it™ notes in offices, as well as the common use of "paper piles" [24] demonstrates an inherent need or desire to organize tasks on the fly. These ad-hoc queues introduce very little overhead when the time comes to revisit a task and remove it from the queue. In the case of email, the opened windows and half-written replies serve the purpose of being both a reminder as well as a low cost task suspension mechanism. The defer action is often performed as part of a scan. Users selectively pick only important emails to read during this short time period, and actively defer messages for later re-visitation in order to manage their time and attention budget. Deferrals are often used during a full triage as well, and this type of use is discussed later on in Chapter 4. Deferrals can be either active or passive in nature. Active deferrals are those that have been read by the reader and then "flagged" for later revisit. This flagging can take the form of remarking the message as unread [36], making a mental note of the message, or a persistent window spawned to display the message outside the inbox. Passive deferrals involve messages that are not read at the scan stage, but are mentally noted as requiring attention later when time was available. 27 Although some messages are deferred as part of the scan, it does not imply that the user assigned a low "importance level" to these messages. There are many reasons to defer a message at any moment in time. A message may be deferred because it is in fact very important, and therefore the reply to it must be crafted with care and extensive proofreading. A message may be deferred because it is long and informational, and is being queued up for reading when a long period of downtime presents itself [36]. Finally, a message may be deferred if it has been deemed unimportant and does not deserve any attention now, if ever. Closely coupled with deferral is the revisit of a message once time is available. Revisits have often been observed at two different times: at a task transition boundary, and towards the end of the day. Once an important task is completed, participants often check their email (i.e. scan), and then take the opportunity to catch up on previously deferred emails before gearing up for the next task in their workflow. This type of revisit is often motivated by the fact that the missing information that caused the initial deferral has now arrived, and the original email can now be handled. The second type of revisit was often observed at the end of the day. Participants were seen taking inventory of all the messages that came in over the last day (sometimes over the previous 2-3 days) and handling the deferred messages before leaving work. Revisit deals with deferred emails that still represent tasks, and is distinct from the retrieve activity identified by [35]. Retrieve is an action performed on archived emails that have been already handled. The defer and revisit actions are closely tied with handling email flow, whereas retrieve is purely an information retrieval activity. 28 3.4 Discussion The pi lot study and day long shadowing h ighl ight a prev ious ly undocumented interconnectedness between f l o w and task management. Study part icipants re l ied on their inbox as an ad-hoc tool for direct ing their attention throughout the day, and used glance and scan to support their si tuational awareness. G l a n c i n g prov ides the in i t ia l awareness that potent ial new tasks have arr ived. Scann ing then pr ior i t izes the new messages for hand l ing , tak ing into account the user 's current wo rk l oad and other contextual factors. F i na l l y , the defer ac t ion prov ides a means o f de-escalat ing the user 's ema i l invo lvement so that h igher pr ior i ty tasks can be attended to. In short, the three f l o w related act ions ident i f ied here demonstrate a c lose integrat ion between ongo ing emai l awareness and the user 's ephemeral task management act iv i t ies (F igure 1). Th i s leads to a very dif ferent type o f ema i l w o r k f l o w mode l than that proposed by V e n o l i a et. al [35], w h i c h featured each ema i l act iv i ty i n very compartmenta l ized s i los that exp l i c i t l y isolated f l o w act ivi t ies f rom task management act iv i t ies. l i l i f j a t i o n , Priorit ized R e a d Queue Glance Autnor * Subjects New M e s s a g e s (no awareness) New M e s s a g e s (Mental arrival) Ivlcbodyfcr Prioritization "Scan" • Defer Queue Ignore Flow Related Activities Ad-hoc Task Management Activities Figure 1 - Integrated email workflow model for new messages 29 The type of task management that we observed was very different from the type that has been referred to in the email literature. There has been a strong drive to integrate email clients and calendaring applications as seen in recent versions of Outlook and the Mozilla Sunbird project. From our observations, even though users do utilize calendars to manage long-term events and meetings requiring coordination with other parties, the majority of task management still occurs within the inbox. Many of these items are too trivial to track formally in a calendar. The user treats these messages more as quick task reminders, viewing them like physical Post-It note. Later in Chapter 5, the importance of this observation will be discussed, and the argument made that email clients should provide more integrated task management functions right in the inbox rather than separating these functions into a separate application. On a different front, the observation that users are glancing at the inbox to gauge the number of new arrivals suggests that existing notification icons are inadequate. Instead of showing a binary change to the client's icon on the desktop to signify that new mail exists, a simple change to displaying the number of new arrivals on the changed icon would much more useful. Coincidentally, this functionality is already available on the Mac OS X version of Mozilla Thunderbird or from various mail checking widgets for the Yahoo Widget Engine (Figure 2). Figure 2 - New mail notifications on Thunderbird for OS X and Yahoo Widgets 30 The shadowing revealed that deferral is a common email handling technique. Circumstances often cause emails to be half read, replies to be partially composed, and the tasks in a message to be left in a suspended state. Existing email literature has largely neglected the study of how-users postpone required actions, choosing instead to focus on other aspects of email such as triage or task management. Recent innovations such as Google's Gmail have begun to address email deferral with the ability to tag messages as to-do items and viewing them outside of the inbox. However, this mechanism breaks down for high volume users who defer a large number of messages over a long period. In such situations, the flagged list itself becomes overwhelmed and users are again at risk of losing important messages. In short, there is relatively little real world data on the factors motivating deferral. Questions such as how long messages are delayed or how many deferred messages users keep track of at a time remains unanswered. The shadowing study encountered a number of challenges that could affect the generalizability of the stated results. The most obvious shortcoming of course is the small number of participants that were recruited and the limitation of observing them for only one day. Many people were quite reluctant to having a researcher study their activities due to the confidentiality of their work, while others were simply uncomfortable working with someone watching over their shoulders. For similar reasons, those that eventually agreed to being observed only permitted one day of observation. The limited access to the participants meant that what was observed represents only a partial glimpse of their email habits, and thus the results presented here may not yet capture the full range of usage possibilities. 31 The observation protocol for this study stressed the importance of maintaining a low profile in order to avoid altering the very behaviours we are examining. In this respect, the study has been successful. Participants stated in the feedback section of, their questionnaires that the investigator had little to no effect on the activities that they engaged in during the day. For the majority of the shadowing, participants simply ignored the investigator while focusing on their work tasks and interacting with their colleagues. Most importantly, any disturbance from the investigator's presence likely did not affect the email behaviours under study, since the participants were not told exactly what aspect of their email use we were researching. Nevertheless, the ideal situation would be to study the user's email interactions without intruding into their work environment. What is needed to address the issue of investigator intrusion is a technique that can monitor a user's email usage remotely. This technique should collect data on the user over several days and possibly even weeks at a time. One possible solution is to equip participants with a modified email client that can log their email interactions on a click-by-click basis, and relay this data back to the investigator. The next chapter will discuss such an approach. 3.5 Chapter Summary This chapter presented the results of a set of full day in-situ observations of email users. It was found that there were gaps in the range of observable user actions from previous email studies, which largely neglected email actions occurring in the middle of the day when a great deal of usage occurs. The new observations show that the existing models for email workflow are incomplete, and fail to show the close interconnectedness between email awareness and its role in ongoing task management. Using the data collected from the full day shadowing study, three 32 previously undocumented actions were identified that characterized how users manage their email flow. • Users glance at their inbox to stay informed as to the rate of incoming messages • Users scan the their inbox to identify important new messages requiring attention • Users defer messages for later revisit to manage the amount of attention they gave to email. This may be done passively by not opening a message until later, or it may be done actively after reading its contents and deciding that it should be attended to later. 33 CHAPTER 4 PASSIVE INTERFACE MONITORING STUDY The daylong observation sessions highlighted how users optimized their attention budget (i.e. glance, scan, defer). Building upon that work, this chapter will present the results of a minimally invasive study of email interaction using passive interface logging. Users had their email actions recorded and analyzed using a specially modified email client. The results will expand upon the defer action described in the shadowing study, and reveals two sub-categories of email deferral: intra-session deferral, and inter-session deferral. This chapter highlights why this type of study is needed, how it was conducted, and the results on the two types of deferral. 4.1 Motivation The daylong shadowing provided insights as to how users manage their daily email attention budget, but the results could be improved if more time with the participants was available. One possible solution is to utilize a special data-logging interface to record the user's actions over a long period. This would remove the need to have a physical presence in the user's workspace, while providing data on their email interactions for as long as needed. It is also an elegant solution because the data from the logging tool can be simply emailed back to the investigator. Several alternative approaches to the logging tool were considered including a video capture of the participant's workspace, user journals, and even eye trackers. The video capture solution was rejected because few users would allow themselves to be videotaped continuously for weeks at a time. Having the user keep a journal of their email actions would limit the dataset to events 34 the user is aware of, as well as biasing the events recorded to those when the user is free. Finally, eye trackers were considered to specifically investigate the glance and scan actions, but this was deemed too cumbersome from a setup and privacy point of view for participants to have a camera recording them for long periods. 4.2 Methodology A month long study involving four participants was conducted using the special data logging interface. The tool generates an event record each time the participant downloads, reads, reviews, or sends a message. To the author's knowledge, this type of data gathering has never been used to examine email reading behaviour at this level of detail. The only previous email research effort that used keystroke logging was concerned solely with email filing behaviour after the messages were handled [2]. This section discusses the details about the logging interface, the participant recruiting process and the data analysis procedures. 4.2.1 Monitoring Tool The open source email client Mozilla Thunderbird was used as the platform to build the data collecting application. The data capture component is an addition to the normal Thunderbird client and uses its facilities for plug-in extensions to augment the interface with data collecting functionality. This results in an email client with no visible changes to the interface, while an extension passively collects interaction data on the participant in the background (Table 4). Any users already using Thunderbird can participate in the study simply by downloading and installing the extension. 35 Data Category Parameters captured I Interaction Data • Time of read / send event • Message read duration • Method of defocusing from a message (e.g. Moved onto next message, switched to another application) Message Information • Author • Subject Line • Message ID • Thread ID • Number of lines in message body • Time of actual download • Time of message timestamp Archival Information • Send and receive frequencies with all contacts • Author, subject lines, and timestamp of all archived messages Inbox Status • Number of unread messages • Index of selected message in thread pane listing • Total number of messages Table 4 - Summary of data captured by Thunderbird Monitoring Extension The captured data is stored locally on the participant's computer using the Resource Description Framework (RDF) format. An automatic dialog prompts the participant to email the data, visible to them as clear text, back to the investigator once a week. 4.2.2 Part icipant Selection A targeted email broadcast to candidates likely to trust the investigator was used to solicit users for participation. This was done because of the extreme difficulty of soliciting general users due to the study's risk of exposing sensitive data. Four users volunteered for the study. All were linked to the university in their job functions and used email extensively for their work. 36 Name Description of Participants Steven Ian Anita • Associate Professor Receives on average 20 emails a day Existing Client: Netscape Communicator (a close relative to Mozilla Thunderbird) Department Technician Receives on average 40 emails a day Also participated in pilot study Existing Client: Microsoft Outlook Administrative assistant Deals with a large variety of department issues from student and faculty Receives on average 20 emails a day Existing Client: Cyrusoft Mulberry (OS X) W i l l University research lab manager . IT troubleshooting, inventory control, misc. lab facility issues Receives on average 30 emails a day Also participated in full day observational study Existing Client: Netscape Communicator (a close relative to Mozilla Thunderbird) Table 5 - Study subject summary These four participants are suitable for the study because each o f them receives a consistently large volume of email per day. A l l the participants are experienced email users and use email as their main medium for communication. This ensured a significant amount of their work related communication would be captured during the study. O f the participants listed in Table 5, Ian spends a good deal o f time at his desk due to the nature o f his position, while Anita, W i l l , and Steven are often in and out of their offices. A drawback o f this subject pool is that two of the four participants did not have previous exposure to M o z i l l a Thunderbird before the study. The remaining two already used M o z i l l a M a i l , which is a closely related client to Thunderbird in terms of interface layout and menu organization. Using an unfamiliar client may result in a participant using less than their full repertoire o f email handling actions (e.g. advanced filtering, short cut keys, customized 37 templates, etc). Every effort is made to ensure the equivalent features are available when the modified client is installed at the start of the study (See Section 4.2.3). The second drawback of this pool of users is that they are not part of a functional team that communicated with each other frequently. The participants rarely had to email each other because their jobs are all unrelated. Had the participants been all part of one cohesive group, it may have afforded the opportunity to observe the communication dynamics of a team and the impact that has on email handling at the individual level. 4.2.3 Study Protocol The month-long study starts with a 30-minute orientation session where the investigator explains the goals of the study, reiterate the potential privacy concerns, and configures the subject's email environment for data collection. Participants are informed verbally and via the signed consent form that the study w i l l be recording their action within their email client, logging which messages they read, when they read each message, and the duration a message is examined. They are also made fully aware that the email message headers in their email archive wi l l be collected excluding the content in the message body. After the participant is fully informed about the study and given their consent, the investigator proceeds with the logging software installation. The latest version of M o z i l l a Thunderbird (version 1.0.6 was used) is downloaded onto the user's machine from the main distribution website (http://www.mozilla.org/products/thunderbird/). The participant's existing email archives, server settings, and filters are all imported into Thunderbird from their existing client. 38 Since none of the participants used Thunderbird as their email client before the study, some user specific configuration had to be performed in each case. Subject Customization Steven • Due to the large size of Steven's archive (12000+), it was not imported to Thunderbird due to memory limitations of participant's computer • Configured email signature function to match old client Ian • Added extension for mail redirect functionality • Configured email signature function to match old client • Demonstrated toolbar customization functions at user's request Anita • No customizations from the normal installation and import Will • Removed the preview pane from view as per the user's preference • Demonstrated filter functions at user's request Table 6 - User installation custom izations Attention is paid to ensure that participants are acclimatized to the interface changes and that as many of their customizations are migrated over to Thunderbird as possible. Settings such as the folder structure and address books are automatically imported by the installation program. The investigator has to assist the participant in some cases with configuring their email signature, recreating various filters, and changing the look of the interface to their satisfaction (e.g. removing the preview pane, adding or removing tool bar buttons). After the importing process, the data collection extension is installed and the participants go about using their email as normal. The extension prompts the participant to email back the logs each week. At the end of the study period, the investigator uninstalls the extension and helps to reconfigure the participant's email environment back to its original setup. 39 4.2.4 Data Analysis We want to expand our understanding of how new messages are handled by the user since this research interested in email flow. To achieve this, a semi-automated open coding technique is used to classify the low-level events in the interaction logs. Open coding is a method of data analysis where representational "codes" are applied to data events, which then facilitates a discussion of the relationship of the events to each other based on observable patterns in that encoding. It is one of several coding methods used in grounded theory, a methodology often used in sociology to generate theories inductively from a set of low-level observation data [16,31]. The outcome of the open coding process in this case is a set of actions describing how new messages are read by, replied to, or deferred by the user. The first step in the analysis is to prepare the data for visual examination by coloring each event by type. The open coding method requires numerous passes over the data in search of emergent themes and the color-coding aids the visualization of event types and user activity periods during this process. There are three types of events in the logs that have been pre-labeled by the monitoring tool at this stage: 1st Reads, Reviews, and Sends. A 1st Read event indicates the user has opened an unread message. A review event occurs when the user reexamines a previously read message, and a send event marks when an outgoing message is sent. Each of these events is given a different colour within the log display (Figure 3). 40 A j B C J 1 Event Msg Num RD(Minutes) Event Timestamp 2 1 00:00:03 Tue 8/16/200515:00:16 3 Review 1 00:00:03 Tue 8/16/200515:00:16 9 2 00:00:33 Tue 8/16/200516:59:14 10 3 00:00:30 Tue 8/16/2005 17:00:33 11 4 00:01:18 Tue 8/16/2005 17:01:03 ~12 Tue 8/16/200517:02:20 22 5 00:00:24 Wed 8/17/200510:00:41 23 24 6 00:00:18 Wed 8/17/2005 10:01:05 7 00:00:40 Wed 8/17/200510:01:23 26 Wed 8/17/200510:13:14 27 Review 4 00:00:03 Wed 8/17/200510:18:06 28 Send Wed 8/17/200510:19:33 29 1 st Read 8 00:01:09 Wed 8/17/2005 10:19:38 30 Send Wed 8/17/200510:28:03 31 1 st Read 9 00:01:04 Wed 8/17/2005 10:28:22 32 Review 9_f 00:14:29 Wed 8/17/2005 10:29:34 38 1st Read 10 00:00:11 Wed 8/17/200511:30:48 39 ""Til 00:00:01 Wed 8/17/200511:31:14 ~40 12 00:00:58 Wed 8/17/2005 li:31j38 41 Review Review 5 00:00:06 Wed 8/17/200511:33:11 42 5 00:00:04 Wed 8/17/2005 11:33:21 48 •1st Read 1 3 00:01:16 | Wed 8/17/2005 13:32:49 AA nnnnnc Figure 3 - Colour coding for raw log events The next step is to highlight the periods of email activity. This is accomplished by setting a time limit between consecutive events. If the time separation between two events is above a predetermined threshold, then a new activity session is created. This artificial time threshold is needed because the monitoring tool cannot access the interrupt events required to track the user's non-email activities using mouse movements or keyboard strokes. This event separation threshold (EST) is different for each participant and is determined as follows. Event Separation Threshold = Mean a( + Std. Dev Event Tiinestamps 11:31:14 AM 11:31:38 AM 11:33:11 AM Event Separations 24s ;.93s Remove iJ's above 95'" percentile Figure 4 - Event Separation Threshold Calculation 41 First, the time separations At between consecutive events are tallied up for all events. This dataset is filtered at the 95 th percentile mark to exclude large At outliers caused by long durations away from email (e.g. Weekend, Weeklong vacation). From the remaining events, the mean and standard deviation are calculated and summed to form the EST. Using this threshold, each participant's log is highlighted to reveal the periods of user activity (Figure 5). 12 Send :•: 1st Read 21 1st Read 24 1st Read 26 Send 27 : Rev iew 28 Hend 29 30 31 p H i 32 jReview 38 39 40 Msg Num RD(Minutes) Event Timestamp 1 00:00:03 Tue BMOOU 15:00:16 1 00:00:03 Tue o H H H j 15:00:16 2 00:00:33 18:59:14 3 00:00:30 T u e S n f l H 17:00:33 4 00:01:18 T i j ^ 01:03 Tue br^^^R ^7:02:20 5 00:00:24 Wed 6/17/200510:00:41 8 00:00:18 Wed 8/17/2005 10:01:05 7 00:00:40 Wed 8/17/2005 10:01:23 4 00:00:03 00:01:09 00:01:04 00:14.29 00:00:11 1 00:00:01 00:00:58 00:00:06 00:00:04 nnni"ifii Wed 8/17/2005 10:13:14 Wed 8/17/200510:18:06 Wed 8/17/200510:19:33 Wed 8/17/2005 10:19:38 Wed 8/17/2005 10:28:03 Wed 8/17/2005 1 0:28:22 Wed 8/17/200510:29:34 W e i We> Wed 8 Wed 8, Wed 8 , r Email Activity Session Figure 5 - Activity sessions highlighted in "Event Timestamp" column Steven Ian Will Anita | Event Separation Threshold (mins.) 15 9 7 8 Table 7 - Event Separation Thresholds used to establish the activity sessions Due to the lack of general mouse and keyboard activity data, this method of session identification may lead to false conclusions if the participant takes an unusually long time to compose a message in mid-session. The send event could be mistakenly marked as the start of a session even though the user continues to handle more messages after sending the reply, 42 new indicating a continuation of the session. Closer examination of the cases where a send event leads off the session shows that the separation times are well above those of send events within activity sessions (Table 8 and Table 9). This suggests that these session leading send events are most likely the start of another activity session, rather than a continuation of an existing session coupled with an unusually long message composition. Therefore, the activity session separations are accurate and can be relied on. Steven Ian Will Anita Time Separation Mean (mins) 74 86 76 33 Time Separation Median (mins) 34 64 34 23 Table 8 - Time separations for session leading send events Steven Ian Will Anita Time Separation Mean (mins) 5 4 3 3 Time Separation Median (mins) 3 2 2 2 Table 9 - Time separations for send events within sessions The open coding process is a combination of manual interpretation assisted by pattern matching scripts that allow the search to scale to the large number of events collected during the study. Once a usage pattern is identified during the manual pass (e.g. many messages are replied to immediately), a spreadsheet script is used to identify all the instances of that pattern and highlight it in the log. In the case of the immediate reply actions for example, the script looks for all instances of 1st reads followed by a send event. It checks to see the author and recipients match between the two events and that the subject lines are the same. Once confirmed the classification is assigned and highlighted (Figure 6). 43 A B C I H K 1 Event M s g Mum RD(Minutes) M s g Treatment Event T imestamp _ „ . Z i 00:00:03 Done t u e 8/16/200515:00:16 3 Review 1 00:00:03 Tue 8/16/200515:00:16 ~ T 1 "z 00:00:33 Done Tue 8/16/200516:59:14 i d 3 00:00:30 Done Tue 8/16/200517:00:33 11 1st Read 4 00:0113 Immediate Handle Tue 8/16/200517:01:03 12 -•"HSend | H I Tue 8/16/2005 17:02:20 22 5 00:00:24 Done V e d 8/17/200510:00:41 23 6 00:00:18 Done Wed 8/17/200510:01:05 2+ 7 00:00:40 Immediate Handle Wed 8/17/200510:01:23 26 ->IH Send Wed 8/17/200510:13:14 27 Review 4 00:00:031 Wed 8/17/200510:18:06 23 Wed 8/17/200510:13:33 _ 1st Read 3 00:01:03 Immediate Handle Wed 8/17/200510:19:38 30 Send Wed 8/17/200510:28:03 31 9 00:01:04 Done Wed 8/17/200510:28:22 32 Review 9 00:14:23 Wed 3/17/2005 10:23:34 38 10 00:00:11 Deferred 2hrs (Inter) Wed 8/17/200511:30:43 39 11 00:00:01 Done Wed 8/17/200511:31:14 40 12 00:00:58 Done Wed 8/17/200511:31:38 41 Review 5 00:00:06 Wed 8/17/200511:33:11 42 Review 5 00:00:04 Wed 3/17/200511:33:21 nn.n1.1c n^.~~ Figure 6 - Automated script identifies immediately handled messages Approximately 4800 email handling events in total were recorded between the four participants over a one month period (Table 10). Technical difficulties interfered with Wi l l ' s logging program and it had to be uninstalled ahead of schedule. Ani ta had difficulties sending messages for a period of a week and a half due to problems with the department server. This explains the unusually low outgoing message count from her logs. Her data was not used during the preparation of the results presented in this chapter because of this. Steven Ian W i l l Anita Days Monitored 37 35 20 34 Number of Handling Events Recorded 1299 1725 1021 768 Messages Received 644 1005 500 366 Messages Sent 336 389 298 56 Table 10 - Logging data summary 44 4.3 Results The data analysis focused on the participant's actions on new unread messages because this thesis is concerned with the flow aspect of email. The open coding analysis reveals four distinct action classes for handling new messages: No reply, Immediate Handle, Intra-Session Defer, and Inter-Session Defer. This section presents the results in two parts. The action classes are first described in a quantitative and context independent manner to provide a picture of how and when they are used. This is then followed by a discussion on the possible motivations and circumstances that may trigger these actions. 4.3.1 Context Independent Actions on New Messages A set of four actions for dealing with new messages emerged from logging data. The first and simplest of these is the No Reply action. Any messages that are only read once and do not trigger a reply are considered to belong to this class. Most of the new message events fall into this category. Immediate reply is the most common action for messages that require a response. An event is classified as an immediate reply if the response is sent immediately after the 1st read of the message, and before any other 1st reads occur. To scan for instances of this usage, a script looks for all the cases where a send event immediately follows a 1st read, and checks to see that the two messages contain the same subject line and has the same contact in the author/recipient field. It is an effective search technique because users rarely changed the subject line when replying. 45 The final remaining actions deal with messages that require a response but which have been delayed. Previous work has shown that users may delay their replies [10], but the log data has revealed that in fact there are two subtle types of deferral at work. Participants sometimes delay replies until the end of an activity session, choosing to read the remaining new messages before returning to write the reply. This type of delay falls under the class of an intra-session defer. In other instances, a participant delays a reply until a subsequent activity session. For these cases, the initial reading of the message represents an inter-session deferral (Figure 7). To automate the search for these two defer actions, a script looks for the first send event occurrence that has the same contact and subject lines as the incoming message, and checks that these are separated by at least one additional read event. It then checks if the send event occurs within the same activity session as the message read. If it is, then the read event is labeled as an intra-session deferral, else it is an inter-session deferral. New Message Handling Actions Definition No Reply The current message is never replied to nor reviewed alter the initial reading. Immediate Reply The next action event by the user is a reply to the current message. Intra-Session Defer The current message is responded to before the current activity session concludes, however the reply is composed only after the user has read at least one other new message. Inter-Session Defer The current message is responded to but the reply does not occur in the current activity session. . . Table 11 - Summary of new message handling actions 46 Read message 1 Ser^ reply to message:!' :R^:rr»ssage::2 ;Read>rrfiessage:3 Immediate Reply Read'message^' Read; message 2;; :Read message 3 ... Send reply to message 1 Intra-session Deferral .Within session boundary Activity Session A Read message 1 Read messaged Read message 3 Read message 4 Activity Session B Send reply to message 1 ...Read other messages .Read other messages' Across session boundary : Inter-session Deferral Figure 7 - Comparison of new message deferral methods The open coding breakdown of the message handling actions is consistent with the results from Dabbish's questionnaire based study [10], which asked users to estimate their handling actions on new messages. For example, users report that the majority of messages do not require a reply based on a rough estimate their inbox in the Dabbish study. The real world data from this study confirms that result even though there are individual variations as would be expected (Table 12). The number of messages requiring immediate reply vs. those with deferred replies is also comparable between the two studies (Table 13). This study however goes beyond the Dabbish study in that it is in a position to provide additional quantitative measures of the deferral process. 47 Reply Percentages No Reply Need Reply Dabbish estimates [10] 64% 36% Steven 72% 28% Ian 84% 16% Will 65% 35% Table 12 - Comparison of reply percentages amongst subjects Reply Type Immediate Reply Deferred Reply Dabbish estimates [10] 64% 36% Steven 70% 30% Ian 80% 20% Will 72% 28% Table 13 - Comparison of deferrals amongst subjects The data reveals that there are variations between the participants when deciding how long replies are delayed (Table 15). There are also differences in how often intra-session deferral and inter-session deferral are used (Table 14). For example, Ian shows a much longer average inter-session delay than the other users since part of his job entails responding to emails about purchase requests (Table 16). These messages often cannot be answered until a delivery is made and so the message is deferred until that time. We also see that Ian delays his emails across activity sessions much more often than Will, who seems to prefer handling more of his replies in the immediate session. Overall, the average delay for an inter-session deferral is less than a day, which suggests that the users are keeping up with their daily emails. 48 Users Intra-Session Defer Inter-Session Defer Steven 33% (18) 67% (36) Ian 28% (9) 72% (23) Will 50% (24) 50% (24) Table 14 - Usage comparison for the two deferral types Intra-Session Defer Delay (Minutes) Steven Ian Will Mean 10.1 17.6 4.6 Median 10.5 11.0 4.0 Std. Dev 5.5 25.2 2.8 Table 15 - Intra-session defer times Inter-Session Defer Delay (Minutes) Steven Ian Will Mean 600 5253 459 Median 123 2281 120 Std. Dev 967 5950 698 Table 16- Inter-session defer times 49 4.3.2 Motivational Factors for Deferred Message Handling The remainder of this section focuses on describing the two types of email deferral. Defer stands out from the other actions because clients have never been intentionally designed to handle this type of usage. Nevertheless, email deferral is a commonly used course of action as evident in the observation study and in the email interface logs. Delayed email handling has been previously observed by other researchers as well [23, 32], though there have been no attempts to study it directly. As email volumes continue to rise,' heavy email users may begin to experience difficulties tracking their deferred messages. An understanding of the underlying motivations for deferring a reply is needed in .order to see if it can be supported directly in email clients. Studying the motivations behind email actions is difficult due to the context sensitive nature of email. Regardless of what type of data gathering technique is used (i.e. questionnaires, interviews, or interface logs), there will be some important contextual information missing that could contain the actual reason for an observed action. The analysis of the two defer subtypes to follow recognizes these shortcomings and what is presented here represents the author's best possible explanations based on the data from this study. Intra-Session Defer There are four scenarios that describe why users delay replying until the end of a session. The following actions can be observed from the log data in between a message being opened and its eventual reply. • The user looks up information before replying • The user reads other topically related new messages before replying 50 • The user reads another unrelated new messages before replying • The user responds to other messages or composes a new message to a 3 r d party before replying The motivations for the first two actions are straightforward and are easily explained. A user will review archived messages before replying when they require reference information. The messages consulted are sometime even self-addressed emails that serve as reminders for important issues. At other times, a new message will be part of a group discussion thread with multiple responses arriving in the user's inbox at the same time. The user first reads the other messages in the thread to come up to speed on everything said by the other group members before replying to the thread. The reasons driving the latter two actions are more difficult to explain, but an understanding of them also offers the most potential for future research on deferral. When a user decides to respond to or open another unrelated message instead of replying to the current one, they may be also mentally prioritizing their tasks at the same time. For example, there are several instances where Steven receives a message from his son at his work email address. He often puts off writing the reply until he has finished responding to his work related emails. This means either he feels the work emails are more important and should be taken care of immediately, or he may the reply to his son as more important and hence deserving his full attention at the end of the see session. 51 It is not possible to gauge the relative importance of a deferred email compared to the messages handled immediately based the log data alone. From the observational study results (Chapter 3) and existing research [10,20,36], factors such as workload, time availability, or issue urgency are likely part of the importance calculation. The next step in this research would be to use the passive interface logging in combination with a real-time polling of message importance to determine how importance is correlated with handling order, and how it affects the decision to delay a message reply. Reason for intra- Steven Ian Will session defer Looked up archive 3 0 7 information Read unrelated 8 3 7 messages Read related 3 4 5 messages Responded to other 4 2 5 messages Total 18 9 24 Table 17 - Types of actions observed during an intra-session defer As a side note, it was hypothesized during the data analysis that an intra-session defer may be used because the user estimates the reply composition will take longer than normal. The composition times between immediate reply and intra-session deferred messages were compared and the results showed that in fact they are comparable (Table 18). This is clearly not a conclusive result due to the small sample size and limitations in the data logger, but it highlights an interesting question that is worthy of additional investigation. 52 Study Participant Mean Immediate Reply Compose Time (minutes) Mean Intra-Session Defer Compose time" (minutes) P(T <=t) t Critical Steven 4.1 4.6 0.34 1.76 Ian 2.1 1.4 0.24 1.83 Table 18 - Comparing Immediate Reply and Intra-Session Defer Compose Times ca messas Inter-Session Defer Attempts to infer user context is much harder for inter-session deferrals. In this case, the events ptured between the initial defer action and the eventual reply usually does not reveal why the ge was deferred. Nevertheless, there are clues within the logs and research literature that help to explain some of the reasons for performing inter-session deferrals, which include: • The user needs additional information or to perform a task before responding • The user is managing their responsiveness image [32] • The user has a higher priority tasks that trumps the need to respond at this time The data for several participants show that an incoming message sometimes spawns additional emails to a third party for information or clarification, while the response to the incoming ige is delayed until this clarification has been received. In the case o f W i l l , there were messac several cases when an email from a vendor triggered a new message to a faculty member in the 2 Compose time is calculated from the moment the user defocuses on the event just prior to the reply, to the moment the message is actually sent. 3 Will's data was not used since the logger was unable to capture when he defocused from the email client and thus could not calculate a compose time. Anita was not included as well due to the lack of send data from her record (refer to 4.2.4). 53 lab, likely for some combination of input or information. The reply to the vendor was written immediately after the response from the faculty member was received, likely containing new instructions or information regarding the purchase. This pattern was observed across three of the four participants. One of the more interesting explanations for an inter-session deferral is its use as a technique for responsiveness image management [32]. A user may chose not to write a response even though they have the time, energy, and required information to do so. This strategy is used when the user would like to maintain a level of distance and inapproachability. People who do this are often in positions that place a constant demand for their time from numerous sources (e.g. managers, secretaries, professors, and more). Evidence for this type of usage requires more contextual data than what the interface logs can capture unfortunately. Studying this motivation for deferral would likely require a combination of interface logging data with concurrent user journals and follow up qualitative interviews. Finally, users may be simply too busy to write a response after reading a message. As mentioned earlier during the pilot study in Chapter 3, one user (Melanie) revealed to the investigator that she often checked her email in the morning quickly before running off to a scheduled staff meeting. The meeting takes a higher priority to responding to emails at that very moment. Upon returning, she reviews any new messages and proceeds to respond to all the deferred messages. In short, users will often leave their email in various states of completion, either due to a blockage at the task level or preemption by a higher priority task. 54 4.4 Discussion The logging data represents a first step towards constructing a mental model of the user's low-level email handling actions for the first time. This extends upon the flow and task management model (Figure 1) which covers the user's email workflow on multiple messages. Once a single message has been identified for processing, the following model depicts the various motivations for the observed handling actions in the interface log data (Figure 8). This model assumes that intra-session deferred messages are delayed because they are either less important or will take longer to reply to than immediately handled messages. Further study is still required though to establish the motivations for performing intra-session deferrals. Important or easy to do No action required Single New Message Read Action required Desire to continue reading Search Archive Impression Management Requned Immediate Handle (i e reply, fwd) Done Intra-Session Defer Inter-Session Defer User's Mental Focus User action on single message Figure 8 - Single Message Handling Model 55 The quantitative data clearly reveals an ongoing practice of delayed email handing as a central part of people's email handling strategy. Users have gravitated towards email as an environment for handling their tasks because of the asynchronous nature of email, which provides them flexibility and control over their personal scheduling in a way few other communication mediums can match. Even though this observation could have been made from the qualitative data alone, the clarity with which the deferral action was seen in the interaction logs provided a much more efficient way of identifying this behaviour. The use of interface monitoring as a method of investigating email usage holds a great deal of promise, of which this study has only begun to explore. Future studies can for example look for a possible relationship between perceived message importance and the reading order employed by the user. One may also try to probe the motivations behind other aspects of email use by linking the interface monitoring to additional contextual data such as eye tracker information. It would have been ideal if it were possible to apply both the daylong shadowing and the interface monitoring on the same user. The investigator would be able to note the timestamp of interesting actions during the shadowing, and be able to rely on the interface logs to provide the interaction details in post analysis. This frees up additional time for observing higher-level actions such as the user glancing at their inbox or scanning for expected messages. Unfortunately, this type of arrangement was not possible for this thesis due to logistical and technical problems. This chapter has concentrated on deferral because it is an understudied area of email. To recap, this thesis has so far described three types of deferring action in email: deferred opening, intra-56 session deferral, and inter-session deferral. The first type occurs when a user scans the inbox and decides to delay the reading of a new message. An intra-session deferral refers to the user delaying a reply until other messages have been read or handled during a single email session. If a reply for a message is somehow postponed beyond the current session, then it becomes an inter-session deferral. The distinction between intra and inter-session deferral is important because they occur over different time scales and have different implications for future research. Intra-session deferral holds promise as a mechanism for observing message prioritization directly, which in turn can be used to understand how email importance is assigned. Inter-session deferral on the other hand may provide a quantitative picture of how email is being used for task management [5], answering questions such as how many pending tasks are users tracking and how long do they have to remember them. 4.5 Chapter Summary This chapter discussed the results from a quantitative study of low-level email interaction using a passive interface logging tool. The analysis centered on how new messages were handled on initial reading and revealed that users employed one of four actions: No reply, Immediate reply, Intra-session defer, or Inter-session defer. This quantitative analysis was significant because it clearly showed deferral was a critical aspect of email usage that has been given little attention to date. The frequent use of delayed handling in email points to a close interconnectedness between its use as a communication medium, and its role in managing the task flow of the user. The identification of defer as composed of two different categories addresses a gap in the understanding of delayed email handling in the literature. Finally, a treatment of the motivations behind these two types of deferral has been presented. 57 CHAPTER 5 DESIGN RECOMMENDATIONS This chapter makes three design recommendations based on the results. First, a message should take on new states beyond read and unread to allow it to mirror the user's mental representation of that message. An analysis of why this is important and how the existing model limits the user's ability to organize rapidly changing tasks will be presented. Secondly, it is proposed that email client research should explore improvements that are not targeted at niche problems within email, and do so in a manner that adds minimal overhead to the user's workload. Finally, a set of interface changes addressing the glance, scan, and defer actions is presented taking into account the proposed changes in design philosophy. 5.1 New Message States The results from Chapter 3 and 4 highlight how messages cannot always be categorized using a simple binary state (i.e. read or unread) as most email clients do. There are strong signs from the data showing how messages can often be blocked or partially read. If messages are to serve as digital artifacts of the user's task list and to-do's [36], then they must be allowed to take on more flexible states in order to better represent their sometimes "indeterminate status." For example, users should be able to declare a message as important for revisit and then have quick views that will show only these messages in isolation [17]. 58 This notion of additional message states is already being explored in new systems like Chandler [9], which allows the user to add "ticklers" to messages to automatically bring them back into view later. Chandler allows messages to be tagged as "later" or "done" and provides isolated views to look at only those items. Google's Gmail is another recent interface development that allows users to tag messages into a separate view. Users can click on the "star" present on each message to tag it and view the collection of starred messages in a dedicated list. The shortcoming with both these approaches is that the messages in these special views are still rigidly locked in a traditional list form. There are no provisions for arbitrary grouping, re-ordering, or annotating of messages, all of which are crucial affordances when organizing tasks in the physical space [24]. Later in this chapter, the importance of these capabilities will be discussed and a potential solution for solving them presented. 5.2 Lightweight vs. Heavyweight Support The discovery that a large component of email is task related [4,14] has influenced much of the recent developments in new interface design. These systems try to integrate email with the tasks they represent [5,18], or allow for semi-automatic entry of messages into a calendar application [30]. Most of them require significant user input in order to function, such as explicitly tagging messages as tasks or approving extracted dates in the email body as new calendar entries. This represents a significant departure in philosophy from the lightweight actions users traditionally employ with email, which can entail simply re-flagging a message as unread for reminding purposes. There is not enough evaluation of these emerging systems to show that a heavyweight solution to managing email will serve users better. Based on the observations from the shadowing and 59 logging study, I argue that in fact a lightweight approach to novel interfaces may be the way forward for improving the email experience. Task based interfaces may be good for tracking large long-term issues, but many of the tasks within emails are too ephemeral and trivial to be tracked. Moreover, tasks often do not have confirmed dates associated with them to enter into a calendar. Mackay recommends against creating email tools that are too specific and solve only niche problems [23]. She observed that email users are diverse, and thus the tools for assisting them should also be as flexible as possible to help as many users as possible. The narrow focus on the task dimension of email in interface design is not sufficient to serve the needs of many users [11]. The next section in this chapter proposes that users require something more akin to a tradition tabletop with folders and memos spread out on its surface. Items can be added, grouped, ordered, or removed with ease and little overhead. 5.3 Supporting Email Flow Defer Much of this thesis has been dedicated to discussing email flow, and the prominent role of deferral in email usage. Users can have multiple deferred messages simultaneously for various reasons, and they currently have no means of consolidating them into a single view. In conventional clients such as Outlook, Eudora or Thunderbird, the onus is on the user to remember which messages need to be revisited, which can become increasingly difficult as the number of tracked items go up. Gmail's "star" feature allows for a simple extraction of tagged messages, but once in its consolidated view, the messages cannot be grouped, annotated, or re-60 ordered arbitrarily. These features are important because they are the same actions people use to manage rapidly changing documents in the physical world, and some of these strategies should translate well into digital environment. Malone conducted studies into how people organized their desks while studying office information systems [24], and found that the seemingly chaotic distribution of items on a table surface is in fact purposeful, and can serve as reminders for the tasks they will be used for. He also notes that people have a difficult time categorizing the paper stacks they create and generate many untitled "piles" of information as a result. This is very similar to the email environment, where users leave messages in their inbox as reminders and have an equally difficult time categorizing their messages when filing [36]. Unlike the physical desk though, users do not have access to tools that allow them to arrange messages arbitrarily according to tasks, order them by importance, or displayed for reminding purposes. What users need is a general scratch area where messages can be grouped, ordered, and annotated with minimal overheard. Given the rapidly dropping prices of large screen LCDs and the increasing prevalence of dual monitor configurations, a separate sub-area of the desktop can be easily set aside for the purpose of email organization. When an email is deferred for later reading, or if the task it pertains to is blocked, the user can simply drag the message from the inbox into one of the "task slots" on the email scratch area (Figure 9). Each slot is sub-divided so that multiple messages can be placed in adjacent lines within the slot, with lines of text entered directly in the same area for annotating the grouping. Other items that may be useful to allow in the slots include pointers to relevant files, URLs, contact notes, and more. The 61 groupings are persistent so that users do not have to devise a suitable name for it or explicitly save them, much like how paper piles on a desk do not need labeling or saving. Users can rearrange the position o f the slots from the default grid as they wish. The slots dock with each other once they are in close proximity to assist in a neat arrangement. A single button is used to flush a slot once the user is done with it, or they may wish to save the grouping but put it out of immediate view for the time being. Drag messages from inbox to scratch area grouping directly in task slot Author 1 | Subject X Author 101 Subject Y Author 2) Subject Y Atttxr 23 j Subject Z Proj Data Proj. URL Note: these are project H , meeting mintues 1 Hush f Save ] : Flush | Save Task Slot 3 Task Slot 4 Reorder list ' as needed Group with relevant material Save or Flush item grouping Figure 9 - Email scratch area The email scratch area immediately solves the problem of having reminder emails scroll out of view in the inbox, and it provides a highly flexible area for dealing with the many types of deferred messages discussed in this thesis. It is a lightweight solution because messages are only a mouse drag away, with alternative methods such as shortcut keys or mouse gestures directly on the inbox possible. Artif icial and cumbersome constraints such as explicitly turning messages into tasks are avoided as in the TaskMaster system. Unlike in Gmail , users are able to group messages together as they see fit, reordering them to create an ordered to-do list i f necessary, and annotating the groupings to customize them. The slots' visibility on the desktop means that 62 selected messages can serve as reminding artifacts, with the user's memory being refreshed each time they come into view. This property means there is no need to set an arbitrary time to bring a message back into the inbox like in Chandler with its tickler function. Users often do not know when they will revisit a message themselves, and the scratch area allows them to revisit deferred issues when they realize they want to. Another consequence of the scratch area is that messages can exist in a fuzzy state for as long as the user requires. It is very flexible because it makes no assumptions as to what the user may call the state of each message, and does not require an explicit assignment (e.g. to-do, meeting, etc). A slot may contain an unread message that was dragged over immediately upon arrival for later reading. It may have messages pertaining to blocked items, or others that are suspended because the user is too tired to continue working. Whatever the case, the messages remain within easy reach and do not become lost in the inbox as they can be with conventional clients. The idea to combine email with other digital documents on the desktop is not a new idea, but the proposed solution is novel because it is targeting task related issues in email and deferred message management. Presto was one of the earliest systems to have both emails and other documents co-existing on the desktop [13]. The ReMail system by Rohall et. al. showed that annotation on emails can be a useful addition to the interface [26], and Whittaker's ContactMap showed how allowing users to flexibly organize emails on a desktop space is appealing [38]. The scratch area idea embodies all of these concepts into one coherent email function. 63 Scan The use of scanning in email is another area where potential for design improvements can be found. The scan action's origin may be traced back to the coarse granularity of new mail notification alerts. Many users turn off the alerts because it activates even when a new message is irrelevant to them. In turn, users scan their inbox to check if something interesting has arrived, or they may look specifically for an expected reply. The anticipation of an important message may motivate the user to perform more scans than they would normally, and causing unnecessary disruption to their work in the process. Clients can improve upon this situation by alerting the user only when replies to important outgoing messages arrive. This is accomplished by having the user flag important outgoing messages at send time. Such a "send with alert" feature is a lightweight approach since it is a one-touch operation, yet it is powerful because it gives explicit instructions to the notification mechanism that it will be appropriate to interrupt the user when the reply is received. The user can then work without interruption knowing the email client will notify them if an important reply arrives. 1 | S e a r c h t u d o r a jHJ 1 -l.nl.xj 1 & Send | Send w/Alert; st pic! as fror as!! ;t pic! ;t picl m m i i *l *| I «§ • 11 I Figure 10 - Screen mock-up of the "Send with Aler t " function 64 Glance We end by discussing the glance action and its design implications. Chapter 3 presented the idea that inbox glancing is useful for establishing the number of new messages that has arrived. Rather than having the full inbox window opened underneath other applications to facilitate the glance, a simple icon indicating the number of message arrivals would be sufficient. Mozilla Thunderbird for OS X and the Yahoo Widget Engine already provides this functionality (Figure 2), and projects such as SNARF [25] have looked at extending it even further by separating the new mail count according to groups the user interacts with such as "family", "co-workers", or "friends", etc. 5.4 Chapter Summary This chapter has proposed that email interfaces should support a more flexible representation of message states, and that novel interfaces should have a lightweight interaction philosophy driving their design to minimize the work required from already overloaded users. A set of novel interface ideas to support the glance, scan, and defer actions has been presented. For the defer action, it was suggested that a large email scratch area where messages can be fluidly arranged would support the needs of the delayed email handling. For the scan action, a send-alert button that tells the email client which replies to outgoing messages deserves special notification when it reply arrives has been proposed. These designs embody the philosophies presented in section 5.1 and 5.2 while integrating the lessons learned from previous research systems. 65 CHAPTER 6 CONCLUSIONS AND FUTURE WORK This thesis has examined the strategies that users employ during continuous email interaction. The combination of daylong qualitative observations with multi-week quantitative interaction log data provided unique insights on real-time email handling actions. The work has concentrated on the flow aspect of email because of the heavy emphasis on email triage in previous research. The results provide evidence that a crucial aspect of email use revolves around deferred handling, and that the established separation between email flow and task management in the literature does not accurately capture actual usage. This chapter summarizes the research contributions made by this thesis and the future research directions that stem from this work. 6.1 Research Goals Summary The initial research hypothesis of this thesis was that by studying how users determined message importance, it might be possible to link the importance of a message to quantitative metrics such as communication frequency or reading order. This in turn would allow the interface to present messages in a more useful way than the default chronological ordering. Understanding this process proved difficult as the study participants all had very individual reasons for their choices. However, in attempting to study this process it was found that the physical presence of the investigator had a discernable effect on the participant's natural email usage behaviours. This 66 raised questions about the completeness of existing email research in documenting the full range of email interactions and guided the direction of the work in this thesis. My research goals were as follows: Goal 1: Devise methods of studying email that minimized intrusion and improved upon existing investigative techniques in order to obtain the most natural usage observations possible. Goal 2: Extract any previously unnoticed interactions from this new dataset and build up a picture of the motivations behind these actions. Goal 3: Determine how existing email clients fail to support these actions and develop a set of design recommendations. 6.2 Contributions of This Work In this thesis I have demonstrated: • The usefulness of daylong in-situ shadowing for email research when used in close conjunction with passive long duration interface monitoring. • The separation between email flow and task management activities as documented by Venolia [74] does not fully capture their actual roles. Instead, flow and task management in email has been observed to be a closely related set of actions helping the user decide what they should work on next. • The task management observed in email appeared to be mostly lightweight in nature, suggesting the drive to integrate email clients with calendaring applications may be solving only a part of the task management problem dealing with meeting coordination. This type of solution cannot support the user's local short-term task management that 67 was often observed during this study. Supporting this type of action would require novel mechanisms for quickly grouping, ordering, and annotating messages within the inbox itself. There is a need to move away from existing interface design philosophies and embrace a more flexible representation of message states, and to encourage the use of lightweight interactions when creating new email support tools. Three previously undocumented actions that users employ to maintain email awareness based on the results of the shadowing study,: o Glance - An opportunistic and brief look at the inbox with the goal of detecting the presence and amount of new messages, o Scan - A short duration examination of the inbox with the goal of identifying important messages to handle immediately during the day. o Defer - The delayed handling of a message in deference to higher priority tasks. The goal of this action is to balance the user's limited attention budget with other non-email related work. The potential of passively monitoring email clients as a means of gathering usage data. As a result of this new quantitative data on email usage, two forms of deferral that take place after reading a new message has been identified: intra-session deferral, and inter-session deferral. o Intra-session deferral has potential implications for future research because it may provide observable evidence of the user's email prioritization process at work. o Inter-session deferral is of interest to designers of novel email interfaces that want to support delayed email handling and help users keep track their deferred items. 68 • A set of proposed designs that support email deferral and scanning. To support the defer action, a general purpose email scratch area which allows the user to arbitrarily group, order, and annotate deferred messages has been suggested. To support the scan action, a send with alert button may be used to instruct the interface that it is appropriate to notify the user when the reply to an important outgoing message arrives. 6.3 Future Work There are a number of possible areas, for improvement given the lessons from this work. They can be broken down into three categories: i) Changes to the shadowing protocol and logging tool capabilities, ii) potential follow up studies, and iii) long term research outlook. 6.3.1 Methodological Refinements • The difficulty in obtaining subjects was a large obstacle for both the observational study as well as the interface monitoring. Future extension to this work will have to explore new means of overcoming the significant privacy concerns in order to attract a greater number of participants to the study. • The intrusion of the investigator during shadowing may be further mitigated if the study ran over the span of several days, which should give the participant more time to become adjusted to working with someone observing them. In addition, the shadowing sessions could be video taped to give a larger team of investigators access to the raw data. The independent analysis of the larger team would improve the impartiality and reliability of the findings. The video data would also allow the user to comment on the validity of the findings when reviewed side by side with the investigator after the analysis. 69 • The interface logging tool currently does not collect data on window focus, and keyboard or mouse activity, and can only infer what is occurring in the time between message selections and send events. The focus and activity data would allow us to detect whether email use or other activities are taking place during the long separation between events, or if the user is simply away from the computer. 6.3.2 Fol low-up Research One of the early goals of this research was to try to link a user's perception of message importance to observable behaviours, as well as meta-data properties such as reading order and communication frequency. This proved to be elusive as there was no data on how important each message was to the user at the time of handling. Future work may attempt to combine the passive interface logging tool with an active polling component that queries the user about the importance of certain messages. Interesting messages to ask about include the intra-session deferred message that were identified in this thesis, as well as messages from contacts of varying communication frequency with the user. The polling could provide a simple five-point ranking for the user to rate how important a specific message was to them. This can then feed back into the interaction log analysis to see if there is any correlation between message importance and any dimensions of email meta-data, as well as explain the differences in motivation behind the use of an intra-session defer vs. an immediate reply. Another avenue of investigation with this type of low-level importance data would be to attempt to model the user's prioritization scheme. For example, are users handling messages using an "earliest deadline" approach, where tasks with the closest deadline receives attention first, or may be they are using a semantic based prioritization, where the perceived importance of the 70 task alone dictates the order of email handling. This type of inquiry is not currently feasible given the lack of message level importance data from the user, but future studies may choose to examine this problem in more detail. 6.3.3 Long Term Focus Future email research should attempt to understand better the mental model that users have of their work. Currently our understanding of email rests at the action level, where we can describe what users do and the patterns they exhibit fairly well. The tools that we have developed for email reflects this in that they also only support the actions observed thus far, with little understanding of the true reasons behind them. The crucial piece of the puzzle will be to explain why users are acting the way they are, and then design tools to support those motivations directly. 6.4 Conclusion This thesis has shown the effectiveness of combining in-situ shadowing techniques with passive interface logs for studying email interaction. This research has expanded our understanding of email flow by highlighting three actions that users employ (glance, scan, defer) during prolonged periods with email. The study also provided an in-depth look at delayed email handling and has identified two previously undifferentiated categories of email deferral (i.e. intra and inter session defer). Finally, this thesis has presented a case for taking future email research in a direction that emphasizes lightweight interactions, coupled with flexible components that allow the user to organize a wide range of tasks. 71 REFERENCES 1. Abu-Hakima, S., McFarland, C , & Meech, J.F. An agent based system for email highlighting, in Proc. AGENTS 2001, ACM Press (2001), 224-225. 2. Baiter, Olle (2000): Keystroke Level Analysis of Email Message Organization. In Proceedings of the ACM CHI 2000 Human Factors in Computing Systems Conference. April 1-6, 2000, The Hague, The Netherlands, p. 105-112. 3. Baiter, O. and C. L. Sidner (2002). Bifrost inbox organizer: giving users control over the inbox. NordiCHI '02: Proceedings of the second Nordic conference on Human-computer interaction, ACM Press, 111-118. 4. Bellotti, V.; Ducheneaut, N.; Howard, M. A.; Smith, I. E. Taskmaster: recasting email as task management. CSCW 2002 Workshop on Re-designing E-mail for the 21st Century; 2002 November 16; New Orleans, LA. 5. Bellotti, V., N. Ducheneaut, et al. (2003). Taking email to task: the design and evaluation of a task management centered email tool. CHI '03: Proceedings of the SIGCHI conference on Human factors in computing systems, ACM Press, 345-352. 6. Blomberg, J. et al (1993) "Ethnographic field methods and their relation to design" in D. Schuler and A. Namioka (eds.) Participatory Design: Perspectives on systems design. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. (pp. 123-154). 7. Boone, G. Concept features in Re:Agent, an intelligent email agent. In Proc. AGENTS 1998, ACM Press (1998), 141-148. 8. Cadiz, J. J., Dabbish, L., Gupta, A., & Venolia, G. D. (2001). Supporting Email Workflow. MSR-TR-2001-88: Microsoft Research. 9. Chandler in a Nutshell -http://wiki.osafoundation.org/bin/view/Journal/ChandlerInFifteenMinutes. 10. Dabbish, Laura A., Kraut, Robert E., Fussell, Susan R., Kiesler, Sara B.: Understanding email use: predicting action on a message. CHI 2005: 691-700 11. Danis, C , Kellogg, W. A., Lau, T., Dredze, M., Stylos, J., and Kushmerick, N. 2005. Managers' email: beyond tasks and to-dos. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (Portland, OR, USA, April 02 - 07, 2005). CHI '05. ACM Press, New York, NY, 1324-1327. 12. Denning, P. J. 1982. ACM president's letter: electronic junk. Commun. ACM25, 3 (Mar. 1982), 163-165. 13. Dourish, P., Edwards, W. K., LaMarca, A., and Salisbury, M. 1999. Presto: an experimental architecture for fluid interactive document spaces. ACM Trans. Comput.-Hum. Interact. 6, 2 (Jun. 1999), 133-161. 72 14. Ducheneaut, N. and Bellotti, V., 2001. Email as habitat: an exploration of embedded PIM. Interactions 8 5, pp. 30-38 ACM Press, New York, NY . 15. Farnham, Shelly. 2003. Personal Map: Automatically Modeling the User's Online Social Network. Position paper for CSCW 16. Glaser, Barney G., Strauss, A. The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago, IL: Aldine Publishing Co, 1967. 17. Google Gmail - http://gmail.google.com 18. Gwizdka, J. (2002). Reinventing the Inbox: Supporting Task Management of Pending Tasks in Email. In Proceedings of CHI 2002 Human Factors in Computing Systems, ACM, NY, 550-551. 19. K. Holtzblatt and S. Jones, "Contextual Inquiry: A Participatory Technique for System Design," Participatory Design: Principles and Practice. Aki Namioka and Doug Schuler (Eds.), Hillsdale, NJ: Lawrence Earlbaum Pub. 1993. 20. Horvitz, E., Jacobs, A., & Hovel, D. Attention-sensitive alerting. In Proc. UAI 1999, Morgan Kaufmann (1999), 305- 313. 21. Hughes, J., King, V., Rodden, T., and Anderson, H. The role of ethnography in interactive systems design. Interactions, 2(2), 56-65. 22. Lockerd, A. and T. Selker (2003). DriftCatcher: The Implicit Social Context of Email. Human-Computer Interaction INTERACT 2003, IOS Press, 813-816. 23. Mackay, W.E., 1988. More than just a communication system: diversity in the use of electronic mail: Proceedings of Conference on Computer-Supported Cooperative Work CSCWT998, ACM Press, New York, NY pp. 26-28 . 24. Malone, Thomas W.. How do people organize their desks? Implications for the design of office information systems. ACM Transactions on Office Systems, 1(1):99—112, January 1983. 25. Neustaedter, C , Brush, A., Smith, M., Beyond "From" and "Received": Exploring the Dynamics of Email Triage. In Proceedings of CHI 2005, pp. 1977- 1980. 26. Rohall, S. L., Gruen, D., Moody, P., & Kellerman, S.(2001). Email Visualizations to Aid Communications. In Proceedings of InfoVis 2001 The IEEE Symposium on Information Visualization, IEEE. 12-15. 27. Segal, R. B., & Kephart, J. O. (1999). MailCat: An Intelligent Assistant for Organizing E-mail. In Proceedings of The Third Annual Conference on Autonomous Agents, ACM NY, 276-282. 28. Sproull, Lee and Sara Kiesler. 1991. Connections:New Ways of Working in the Networked Organization. Cambridge: MIT Press. 29. Stern, M., Identifying and Understanding Dates and Times in Email, IBM Technical Report RC-22875, 2003. 73 30. Stern, M. K. 2004. Dates and times in email messages. In Proceedings of the 9th international Conference on intelligent User interface (Funchal, Madeira, Portugal, January 13 - 16, 2004). IUI '04. ACM Press, New York, NY, 328-330. 31. Strauss, A, Corbin, J. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Newbury Park, CA: Sage Publications, 1990 32. Tyler, Joshua R., Tang, John C. When Can I Expect an Email Response? A Study of Rhythms in Email Usage. Proc. ECSCW 2003 33. Tyler, J. R., Wilkinson, D. M., and Huberman, B. A. 2003. Email as spectroscopy: automated discovery of community structure within organizations. In Communities and Technologies, M. Huysman, E. Wenger, and V. Wulf, Eds. Kluwer B.V., Deventer, The Netherlands, 81-96. 34. Venolia, G. and C. Neustaedter. Understanding sequence and reply relationships within email conversations: A mixed-model visualization. In Proc. CHI 2003 35. Venolia, G., Dabbish, L., Cadiz, J.J., and Gupta, A. (2001). Supporting Email Workflow. Microsoft Research Tech Report MSR-TR-2001-88 36. Whittaker, S. and Sidner, C , 1996. Email overload: exploring personal information management of emailin: Proceedings of the Conference on Human Factors in Computing Systems CHI' 1996, ACM Press, New York pp. 276-283 . 37. Whittaker, Steve, Terveen, Loren, Nardi, Bonnie A. (2000): Let's Stop Pushing the Envelope and Start Addressing It: A Reference Task Agenda for HCI. In Human-Computer Interaction, 15 (2) p. 75-106 38. Whittaker, S., Jones, Q., Nardi, B., Creech, M., Terveen, L., Isaacs, E., and Hainsworth, J. 2004. ContactMap: Organizing communication in a social desktop. ACM Trans. Comput.-Hum. Interact. 11,4 (Dec. 2004), 445-471. 39. Yiu, K., R. Baecker, N. Silver and B. Long (1997). A time-based interface for electronic mail and task management. In Proc. HCI International 1997 v. 2, Elsevier, 19-22. 74 APPENDIX A - DATA ANALYSIS Qualitative shadowing data The raw descriptive data collected during the shadowing study was processed using an affinity diagram process. This process groups related ideas and observed actions together into categories in an inductive manner. After each observation session, the notes were reviewed by the investigators for interesting quotes and actions. Some of these quotes or actions were written onto Post-it notes or a whiteboard to facilitate a discussion on the relationship between the items. Related items were progressively moved together and clustered. Due to the required discussion component of this technique, at least two investigators needed to be present during the process (i.e. Mr. Tony Tang and the author). This review process has two purposes. It allowed us to update our model with new data, while generating a wish list of observations for us to concentrate on during subsequent shadowing sessions. After observing two of the four study participants, we came up with the following interim categorization of actions to classify what we saw. Action Type Used When Information Gathered Time Spent Glance -In between thoughts -During a task, semi focused -Number of emails -Roughly looking for bold lines <= is Scan -After social break -When bore -Author and subject -Important message extraction lOs-lmin Scan reminder -In the middle of a task -Resuming from interruption -What was I doing? 5s-30s Review/ "Re-find" -Between task boundaries -Consult when tasks from ext. memory completed -What's the next task to do? -Mostly P2 items from earlier Triage 20s-lmin Triage -After a long break -Future task items 5min-30 -Scan found too many messages Table 19 - Initial email action classification 75 We separated scan and scan reminder at this stage because our first participant Flora (see Table 3) was an administrative office worker, and she had a distinct mode of use where she scanned the inbox after being interrupted by visitors to find where she left off. We were not sure about the role of the review yet in the overall scheme of the user's workflow at this stage. After this initial analysis, we made it a priority to look for the following aspects of use in our next two participant shadowing sessions: • What is the role of email during the day? • What are the user's goals when performing each action category (i.e. glance, scan, etc)? • When is attention given to email? • What work activities are being performed when users performs a glance or a scan? The affinity diagramming process was repeated again after performing two more shadowing sessions. Using the data from all four users, the action categories was revised to the simpler glance, scan, and defer as described in Chapter 3. The scan-reminder action was deemed part of the broader action of scan that was concerned with looking for what to do next. This essentially is what the user is doing when they scan the inbox to remind themselves of where they left off. At the same time, we no longer saw the triage action as a distinct process that was much more than a larger scale version of scan. We saw that users used similar reading patterns whether they were just coming back to their desks, or during prolonged periods at their desk. Finally, we recognized the revisit action as the end of a chain of actions that began when a message was deferred. Hence, we updated our model to feature the initial deferral as the more prominent action. For more information, please refer to Chapter 3 for a detailed treatment of the glance, scan, and defer actions. 76 Quantitative Study The interface data was collected using a Mozilla Thunderbird extension that periodically emailed back the user's interaction logs. The extension is written in Javascript, with access to the user's actions made possible using the various XPCOM components in the Mozilla framework. XPCOM components provide the API for accessing all parts of the user's email data, including the full header and subject lines of all messages. "DOM Interface Events i Moziih X P C O M Mail Thunderbird Functions fnslMsgMpss<igKSfr\.ic&) I^Millll^sll X P C O M File and Data Sources Functions (ns.l =ile ns l fOFServ ico ) emailDataColledor xul Event Timing Controller errtallDalaColleclorjs Client Start/Stop j ] Monitor clientStartStopTiming.js Timing State Machine M§gp r r i i ngH?nd l i r . g js Single Message Attributes Functions sha'ed j=. RDF Data Source Management DSFileHandllng.Js. Figure 11 - Mozilla data logging extension architecture The extension notes each message opening or sending action in real time and stores the event in a local RDF repository. On its first encounter with a message, the extension creates an entry detailing the message's header meta-data from the attribute fields of its XPCOM object. At the same time the message added to a master list of all encountered messages by its message ID. This master list makes for an easy entry point into the data during parsing and analysis. Below is an example of such an initial entry, with each line being a self-describing property of the RDF entry. 77 Individual message record: <RDF:Description RDF:about="msgID:ElE56Li-0000oj-V6@ukrs58866.pur3.net" authorEmail="example@example.com" fullFromString="Name of author <example@example.com>" subject="This Weeks Featured Products" threadlD=" 162534115"> <lineCount NC:parseType="Integer">247</lineCount> <emailTimeStamp NC:parseType="Integer">l 124217586</emailTimeStamp> <readEventListResource RDF:resource="readEventList:ElE56Li-0000oj-V6@ukrs58866.pur3.net"/> <messageSize NC:parseType="Integer">24189</messageSize> <downloadTime NC:parseType="Integer">l 124217632</downloadTime> </RDF:Description> Master list of recorded messages by message ID: <RDF:Seq RDF:about="urn:msgIDList"> <RDF:li RDF:resource="msgID:ElE56Li-0000oj-V6@ukrs58866.pur3.net "/> <RDF:li RDF:resource="msglD:0B1670DFlE4DB3FC520909D7CE87@exchange4.ubc.ca7> <RDF:li RDF:resource="msgID:017201 c5a 1 af$65e28560$293a5289@stores2'7> <RDF:liRDF:resource="msgID:20050830235415.291263B003E@mail.ece.ubc.ca7> </RDF:Seq> In addition to this one-time only entry for the message, the data logger also records each time the message is opened and the reading duration. To access this data, each message contains a read event list resource as part of its individual record entry (see above example under "readEventListResource"). Every time a message is revisited by the user, an additional entry is appended to this record. The following is an example of what such a list looks like. In this case, it is an example of a message that has been opened multiple times. Sample read event entry showing multiple message openings: <RDF:Seq RDF:about=" readEventList:ElE56Li-0000oj-V6@ukrs58866.pur3.net 1"> <RDF:li>47849| 1124739020390|me/Inbox| 104,105,106,107| 108| 102|TBDefocusStop</RDF:li> <RDF:li>4226| 1124739245674|me/Inbox| 107| 108| 102|TBDefocusStop</RDF:li> <RDF:li>12698|l 125091108313|me/lnbox|219,220,221|222|102|NormalSelect</RDF:li> </RDF:Seq> The parameters in each entry are delimited by a "|". Using the last entry from this example and reading from left to right these parameters are: 78 1. Read duration in milliseconds (i.e. 12698) 2. Event timestamp in seconds since Jan 1st, 1970 (i.e. 1125091108313) 3. Message folder location (i.e. me/Inbox) 4. Table index in the thread pane of other unread messages (i.e. 219,220,221) 5. Total number of messages in the inbox (i.e. 222) 6. Index of this message in the thread pane table (i.e. 102) 7. Type of event ending the message reading (i.e. NormalSelect) The data logger stops the timing for a message opening in three ways: NormalSelect The use has selected another message to read from the thread pane. TBDefocusStop The user has defocused from the Thunderbird window. DblClickRefocusStop The user was reading a message in a double click spawned window but now has refocused on the main inbox window. In addition to collecting data on current messages, the data logger also searches through the user's email archives to examine their past communication patterns. The information recorded on these messages is less detailed than for those observed in real-time. The data is organized by the sender, with a list created for every sender found in the user's repository. Another list is also maintained containing the messages exchanged between any two sender-recipient pair. Sample archive entry showing send/receive pairings: <RDF:Seq RDF:about="from:sampleAuthor@ece.ubc.ca"> . <RDF:li RDF:resource="to:nsiu@telus.net"/> <RDF: 1 i RDF:resource="to:avahei@iust.ac.ir"/> <RDF:li RDF:resource="to:zy7100@hotmail.com"/> <RDF:li RDF:resource="cc:studySubject@ece.ubc.ca"/> <RDF:li RDF:resource="cc:hillary@apsc.ubc.ca"/> </RDF:Seq> We see that the sender "sampleAuthor@ece.ubc.ca" has sent messages to a number of recipients, including one instance when our study recipient was cc'ed on one of his messages. Turning to the second list that looks at only the messages exchanged between the pair, we can expect to see 79 the following. In this case, we see the sample author and the study subject have exchanged three messages before. Sample archive entry showing specific pairing details: <RDF:Seq RDF:about="from: sampleAuthor@ece.ubc.ca cc:studySubject@ece.ubc.ca"> <RDF:li>l 124236940|0|43027E8C.3070603@ece.ubc.ca|One more thing|abc|Sent</RDF:li> <RDF:li>l 124299173|0|430371A5.8090501@ece.ubc.ca|One more thing|abc|Sent</RDF:li> <RDF:li>l 124311408|0|4303A170.5010209@ece.ubc.ca|One more thing|abc|Sent</RDF.ii> </RDF:Seq> Each entry is again delimited with a "|" as before and can be interpreted as follows reading from left to right: 1. Message timestamp in seconds since Jan 1st, 1970 (i.e. 1124311408) 2. Mozilla Thunderbird's internal thread ID (i.e. 0) 3. Message ID from message header (i.e. 4303A170.5010209@ece.ubc.ca) 4. Subject line (i.e. One more thing) 5. User's account ID (i.e. abc) 6. Folder in which the message was found (i.e. Sent) Finally, the data logger also sends back a record of when the email client was started and stopped. This is in the form of a UNIX timestamp, with each 13 digit number representing the number of seconds since Jan 1st, 1970. Sample of a partial email client start stop log: start: 1124229612211 stop: 1124229613814 start: 1124229748237 stop: 1124229749258 start: 1124229914446 stop: 1124229915347 The source code and installation package for the data collection extension can be found at http://www.ece.ubc.ca/~nsiu/EmailDataCollector/ 80 


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items