UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

NextMove : task coordination in a distributed agile development environment Mak, David K. M. 2007

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_2007-0498.pdf [ 12.32MB ]
Metadata
JSON: 831-1.0101113.json
JSON-LD: 831-1.0101113-ld.json
RDF/XML (Pretty): 831-1.0101113-rdf.xml
RDF/JSON: 831-1.0101113-rdf.json
Turtle: 831-1.0101113-turtle.txt
N-Triples: 831-1.0101113-rdf-ntriples.txt
Original Record: 831-1.0101113-source.json
Full Text
831-1.0101113-fulltext.txt
Citation
831-1.0101113.ris

Full Text

NextMove: Task Coordination in a Distributed Agile Development Environment by D a v i d K. M . M a k B . A . S c , Un ivers i t y of Br i t ish C o l u m b i a , 2 0 0 4 T H E S I S S U B M I T T E D IN P A R T I A L F U L F I L L M E N T O F T H E R E Q U I R E M E N T S F O R T H E D E G R E E O F M A S T E R O F A P P L I E D S C I E N C E in T h e Facu l t y of G r a d u a t e S t u d i e s (Electrical and Computer Engineering) T h e Un ive rs i t y of Br i t ish C o l u m b i a S e p t e m b e r 2 0 0 7 © D a v i d K. M . M a k , 2 0 0 7 ABSTRACT While task coordination has always been a challenge for software project managers, the emerging trend of distributed development adds a new dimension to the conundrum. This thesis describes NextMove, a tool which relieves the project manager from the mundane work of day-to-day task assignment to team members, as well as the heuristic processes behind NextMove. NextMove leverages common practices in software engineering and multi-criteria decision-making processes in order to continuously evaluate the pertinence of tasks that are currently available in the project and advise the team on what to do next as the project progresses. Two methods were used to validate the evaluation approaches behind NextMove: project simulations and experiment with human subjects. Simulation results show that teams that make use of NextMove's heuristic process consistently post shorter project completion times than at least 80% of randomized executions of the same project. The experiment yielded less optimistic results, with conflicting results between different experiment sessions. However, important lessons were learned during the experiment and they open avenues for further research and improvement on NextMove. ii TABLE OF CONTENTS ABSTRACT ii TABLE OF CONTENTS iii LIST OF TABLES vii LIST OF FIGURES viii GLOSSARY xi ACKNOWLEDGEMENTS xii DEDICATION xiii CHAPTER 1 INTRODUCTION 1 1.1 SIGNIFICANCE 2 1.2 RESEARCH GOALS 3 1.3 ORGANIZATION OF THIS THESIS 3 1.4 CONTRIBUTIONS OF THIS THESIS 4 CHAPTER 2 CONTEXT AND PROBLEM STATEMENT 5 2.1 CHARACTERISTICS OF DISTRIBUTED AGILE DEVELOPMENT 5 2.1.1 Distributed Development 6 2.1.2 Agile Methods 6 2.1.3 Process Models 7 2.1.4 Team Structure 8 2.1.5 Difficulties in Agile Distributed Development 8 2.2 CONTEXTUAL MODEL AND PROBLEM DEFINITION 11 2.2.1 Scenario 11 2.2.2 Feature Model 12 2.2.3 Process Model 12 2.2.4 Project Model 13 2.2.5 Problem Definition 17 CHAPTER 3 REVIEW OF CURRENT STATE OF THE ART 18 3.1 CURRENT STATE OF THE ART IN COORDINATION METHODOLOGIES 18 3.1.1 Policy-based Task-Assignment Strategy 18 iii 3.1.2 Multi-Criteria Task Allocation 19 3.1.3 Optimizing Task Allocation using Genetic Algorithms 20 3.1.4 Summary 20 3.2 DESIGN CONSIDERATION 21 3.2.1 Activity Theory • 21 3.2.2 Continuous Collaboration 22 3.2.3 Group Awareness in Distributed Environment 23 3.2.4 Summary 24 3.3 PROJECT MANAGEMENT TOOLS 24 3.3.1 Microsoft Office Project 25 3.3.2 Open Workbench 25 3.3.3 Rally Agile Life Cycle Management 27 3.3.4 Xplanner 27 3.3.5 Summary 28 C H A P T E R 4 T A S K C O O R D I N A T I O N S T R A T E G Y 29 4.1 TASK PRIORITIZATION 29 4.2 TASK ALLOCATION 31 4.2.1 Recommended Task List 36 4.3 BENEFITS OF METHODOLOGY '• 36 C H A P T E R 5 I M P L E M E N T A T I O N 3 7 5.1 WHY ECLIPSE? : 37 5.2 N E X T M O V E ARCHITECTURE 38 5.3 STATISTICS AND EXTERNAL LIBRARIES 40 5.4 SERVER .• 40 5.5 USER INTERFACE 41 5.5.1 Team Backlog View 41 5.5.2 Personal Backlog View 44 5.5.3 Team Activities View 45 iv 5.5.4 Parking Lot View : 46 5.5.5 Artifacts View 47 5.5.6 Log View 48 5.6 DESIGN DISCUSSION 49 C H A P T E R 6 S I M U L A T I O N 5 0 6.1 SIMULATION D A T A 50 6.1.1 Project Data 50 6.1.2 Team Member Data 54 6.2 SIMULATION M O D E L ; 55 6.3 DATA ANALYSIS 58 6.4 RESULTS 59 6.4.1 Scenario Ax 59 6.4.2 Scenario Ay 61 6.4.3 Scenario Bx 63 6.4.4 Scenario By 65 6.4.5 Scenario Cx 67 6.4.6 Scenario Cy 69 6.5 DISCUSSION 71 6.5.1 Observation from the Results -• 71 6.5.2 Simulation Model Limitations 72 C H A P T E R 7 E X P E R I M E N T 74 7.1 EXPERIMENTAL PLANNING 74 7.1.1 Goals 74 7.1.2 Experimental Units 74 7.1.3 Experimental Material 75 7.1.4 Tasks 76 7.1.5 Hypothesis 77 7.1.6 Procedure 78 V 7.1.7 Analysis Procedure 78 7.2 RESULTS 79 7.2.1 Project A 79 7.2.2 Project B 81 7.2.3 Project C 83 7.3 DISCUSSION 85 7.3.1 Summary of Results 85 7.3.2 Ineffectiveness of NextMove 86 7.3.3 People Factor 88 7.4 VALIDITY 89 CHAPTER 8 CONCLUSIONS AND FUTURE WORK 92 8.1 RESEARCH GOALS SUMMARY 92 8.2 CONTRIBUTIONS OF T H IS WORK 93 8.3 FUTURE WORK 94 8.3.1 Simulation Model Improvements 94 8.3.2 Experimental Methods Refinements 96 8.3.3 Long Term Development of NextMove 97 8.4 CONCLUSION 98 REFERENCES 100 v i LIST OF TABLES T A B L E 1. R E L A T I V E IMPORTANCE OF TASKS 34 T A B L E 2. N O R M A L I Z E D V A L U E S OF T A B L E 1 34 T A B L E 3. T E A M M E M B E R S ' RELATIVE J A V A PROGRAMMING ABILITIES 34 T A B L E 4. N O R M A L I Z E D R E L A T I V E COMPETENCIES FOR E A C H T E A M M E M B E R 35 T A B L E 5. R A N K I N G T H E S K I L L S - B A S E D SUITABILITY RATINGS OF T H E T E A M M E M B E R S 35 T A B L E 6. F E A T U R E LIST FOR PROJECT A 51 T A B L E 7. F E A T U R E LIST FOR PROJECT B 53 T A B L E 8. F E A T U R E LIST FOR PROJECT B 54 T A B L E 9. O R D E R OF A S S I G N M E N T OF TREATMENTS 76 T A B L E 10. S U M M A R Y OF E X P E R I M E N T A L RESULTS 85 LIST OF FIGURES FIGURE 1 - MATRIX DEPICTING RELATIONSHIP BETWEEN ACTIVITIES, FEATURES AND TASKS 13 FIGURE 2 - MATRIX DEPICTING RELATIONSHIP BETWEEN WORK PRODUCTS, FEATURES AND TASKS 14 FIGURE 3 - U M L CLASS DIAGRAM OF THE PROJECT MODEL 16 FIGURE 4 - OPEN WORKBENCH GENERAL USER INTERFACE 26 FIGURE 5 - OPEN WORKBENCH TASK ASSIGNMENT DIALOG 26 FIGURE 6 - R A L L Y DASHBOARDS : TEAM STATUS 27 FIGURE 7 - XPLANNER STORY VIEW 28 FIGURE 8 - LAYER DIAGRAM OF N E X T M O V E 3 9 FIGURE 9 - N E X T M O V E T E A M BACKLOG VIEW WITH TASKS COLOURED BY THEIR ACTIVITIES 41 FIGURE 10 - N E X T M O V E T E A M BACKLOG VIEW WITH TASKS COLOURED BY THEIR FEATURES 42 FIGURE 11 - N E X T M O V E TASK INFORMATION DIALOG 43 FIGURE 12 - N E X T M O V E MA N U A L TASK ALLOCATION DIALOG 44 FIGURE 13 - N E X T M O V E PERSONAL BACKLOG VIEW WITH THE CONTEXT M E N U 45 FIGURE 14 - N E X T M O V E T E A M ACTIVITIES VIEW 46 FIGURE 15 - N E X T M O V E PARKING L O T VIEW 47 FIGURE 16 - N E X T M O V E ARTIFACTS VIEW 48 FIGURE 17 - N E X T M O V E L O G VIEW 48 FIGURE 18 - PROCESS FLOW DIAGRAM OF PROJECT A 52 FIGURE 19 - PROCESS FLOW DIAGRAM OF PROJECT B 53 FIGURE 2 0 - PROCESS FLOW DIAGRAM OF PROJECT C 54 FIGURE 21 - GENERAL EXECUTION FLOW OF EACH SIMULATED TEAM MEMBER 56 FIGURE 2 2 - DETAILED EXECUTION FLOW WITHIN THE "FIND AND WORK ON TASK" ACTIVITY 56 FIGURE 23 - COMPARISONS OF PROJECT COMPLETION TIMES IN SCENARIO A X : 59 FIGURE 2 4 - COMPARISONS OF M E A N FEATURE COMPLETION TIMES IN SCENARIO A X 60 FIGURE 25 - COMPARISONS OF WEIGHTED-MEAN FEATURE COMPLETION TIMES IN SCENARIO A X 61 FIGURE 2 6 - COMPARISONS OF PROJECT COMPLETION TIMES IN SCENARIO A Y 62 viii FIGURE 27 - COMPARISONS OF M E A N F E A T U R E COMPLETION T IMES IN SCENARIO A Y 62 FIGURE 28 - COMPARISONS OF W E I G H T E D - M E A N FEATURE COMPLETION T IMES IN SCENARIO A Y 63 FIGURE 29 - COMPARISONS OF PROJECT COMPLETION T IMES IN SCENARIO B X 64 FIGURE 30 - COMPARISONS OF M E A N F E A T U R E COMPLETION T IMES IN SCENARIO B X 64 FIGURE 31 - COMPARISONS OF W E I G H T E D - M E A N FEATURE COMPLETION T IMES IN SCENARIO B X 65 FIGURE 32 - COMPARISONS OF PROJECT COMPLETION T IMES IN SCENARIO B Y 66 FIGURE 33 - COMPARISONS OF M E A N F E A T U R E COMPLETION T IMES IN SCENARIO B Y 66 FIGURE 34 - COMPARISONS OF W E I G H T E D - M E A N FEATURE COMPLETION T IMES IN SCENARIO B Y 67 FIGURE 35 - COMPARISONS OF PROJECT COMPLETION T IMES IN SCENARIO C X 68 FIGURE 36 - COMPARISONS OF M E A N F E A T U R E COMPLETION T IMES IN SCENARIO C X 68 FIGURE 37 - COMPARISONS OF W E I G H T E D - M E A N FEATURE COMPLETION T IMES IN SCENARIO C X 69 FIGURE 38 - COMPARISONS OF PROJECT COMPLETION T IMES IN SCENARIO C Y 70 FIGURE 39 - COMPARISONS OF M E A N F E A T U R E COMPLETION T IMES IN SCENARIO C Y 70 FIGURE 40 - COMPARISONS OF W E I G H T E D - M E A N FEATURE COMPLETION T IMES IN SCENARIO C Y 71 FIGURE 41 - E X E C U T I O N T I M E L I N E FOR PROJECT A U N D E R T H E RANDOMIZED CONDITION 72 FIGURE 42 - E X E C U T I O N T I M E L I N E FOR PROJECT A U N D E R T H E HEURISTICS-BASED CONDITION 72 FIGURE 43 - T H E W O R K S IMULATOR INTERFACE 77 FIGURE 44 - COMPARISON OF T O T A L PROJECT COMPLETION T I M E IN PROJECT A 80 FIGURE 45 - COMPARISON OF M E A N F E A T U R E COMPLETION T I M E IN PROJECT A 80 FIGURE 46 - COMPARISON OF W E I G H T E D - M E A N F E A T U R E COMPLETION T I M E IN PROJECT A 81 FIGURE 47 - COMPARISON OF T O T A L PROJECT COMPLETION T I M E IN PROJECT B '. 82 FIGURE 48 - COMPARISON OF M E A N F E A T U R E COMPLETION T I M E IN PROJECT B 82 FIGURE 49 - COMPARISON OF W E I G H T E D - M E A N F E A T U R E COMPLETION T I M E IN PROJECT B 83 FIGURE 50 - COMPARISON OF T O T A L PROJECT COMPLETION T I M E IN PROJECT C 84 FIGURE 51 - COMPARISON OF M E A N F E A T U R E COMPLETION T I M E IN PROJECT C 84 FIGURE 52 - COMPARISON OF W E I G H T E D - M E A N F E A T U R E COMPLETION T I M E IN PROJECT C 85 FIGURE 53 - G A N T T C H A R T FOR PROJECT C (BASELINE TREATMENT ) 87 FIGURE 54 - G A N T T C H A R T FOR PROJECT A (PARTIAL TREATMENT ) 89 ix FIGURE 55 - E X A M P L E OF T I M E Z O N E ASSIGNMENTS GLOSSARY Activity: a guideline for a certain type of task. A n activity modifies certain s work products and may requires roles with certain expertise to perform it. Artifact: a piece of project related document, source code or file which are produced or modified by a task. The scope and format of an artifact are defined by its parent work product. Feature: a client-defined unit of functionality which adds value to the software system. Features are also known as user stories. Process: an abstract model which defines the relationships between different activities and relationships between different work products. Role: a set of responsibilities within the project. A team member may take on different roles as the project progresses. Task: a unit of work which can be allocated to individual team members. The scope and actions of a task are defined by its parent activity. Work Product: a template which defines the scope and format of a physical artifact. ACKNOWLEDGEMENTS A s an alleged novel writer, perhaps I am expected to write something insightful or even witty in this acknowledgement section. However, I truly believe that nothing is more sincere than a few simple word of thanks to those who helped me along throughout my graduate school years. To my supervisor Philippe, I thank you for giving me the opportunity to explore the world of research. Thank you for your generous support, without which my graduate school experience would not have been nearly as fulfilling and memorable. And I thank you for your patience and guidance throughout this long journey. To all my friends in S E A L , Yvonne, Eve, Mandana, Larix, Steve, Agung, Davide, Jaana, thank you all for the great times. I wi l l always remember our lab as a place full of happy memories and fun. B i g thanks to my friends Wilson, Sunny, Joyce and Pamela for your company and invaluable support since my undergraduate years. D e d i c a t e d with L o v e My parenty, vny tl&ter cmd/ CHAPTER 1 INTRODUCTION Coordinating and allocating tasks are important elements of project management. Traditionally, the responsibility of managing resources falls to the project manager. However, as the project becomes more sophisticated, the project manager faces the problem of having to collect, analyze and disseminate a large amount of information during the course of the resource allocation process. Solutions to this problem often result in one of the two extremes: coordination according to heavy up-front planning and hierarchical team structure, or ad-hoc self-coordination. Neither type of solutions is optimal for a volatile environment of software projects. Up-front planning is too rigid and inflexible to respond to the constant and inevitable changes in a software project. Self-coordination, which leaves the individual team members to their own devices, may descend into chaos i f it is not properly monitored, since the software project team needs to work as a tight unit at all times to ensure that various components are continuously and smoothly integrated. Advances in electronic communications and the development of distributed collaborative tools make distributed software development possible. The term "working together" no longer applies solely to co-located teams. A modern project team may be distributed over a number of rooms in the same building, or over different buildings in the same city, or even over different cities around the globe. 1 In recent years, the emergence of agile development has attracted much attention and interest from the software development community. The Agile Movement helps reduce the reliance on heavy up-front planning and written artifacts. However, because many Agile practices depend on direct, face-to-face, informal communications, managing the dynamic and tactical activities of discovering, scheduling and allocation of tasks is difficult in a distributed environment. Agile teams need to find a way to dynamically manage a project backlog, which is a list of tasks to be done in a project, and attain a high level of collaboration while working in a distributed fashion. 1.1 Significance Over the years, numerous tools and management systems have been created to help teams manage tasks in distributed projects. Some of these tools and systems leave the task coordination problem to the project manager, who has to manually assign tasks. Others try to automatically determine the "most optimal" task allocation sequence for each team member and force the team to follow those pre-determined schedules. Chapter 3 reviews the aforementioned types of tools and systems. They tend to focus on the procedural aspects of a project and therefore impose numerous restrictions on the team. Inevitably, users would try to circumvent the system as they see little benefit in locking themselves in a rigid and seemingly draconian system which offers little assistance when changes in the project occur. Chapter 3 also reviews a number of proposed methodologies in modeling and decision-making to help facilitate task coordination. These methods fail to consider all of the different aspects in software development, such as the need for quick reactions to changes in circumstances, which is the key concept of agile development. 2 The work described in this thesis integrates common practices in software project management in order to devise an approach that wou ld help the project manager handle the chores o f day-to-day task coordinat ion. The proposed approach aims to combine the quantitative and qualitative aspects o f a software project in order to discover tasks that are most pertinent to the project 's success, as we l l as to rate the suitabil i ty o f each team member to perform those tasks. To demonstrate the effectiveness o f the proposed strategy, N e x t M o v e was developed as a proof-of-concept. The too l 's design makes use o f concepts from computer-supported col laborative work ( C S C W ) in order to max imize benefit for the end-user wh i le m in im iz ing the amount o f user-intervention. 1.2 Research Goals The objective o f this thesis is to assist the project manager in coordinat ing a distributed software development project wh i ch wou ld result in improvements in the ef f ic iency and job satisfaction o f the development team. In summary, the goals o f this thesis are: • identify the advantages and disadvantages o f current practices • create a tool to assist the project manager in performing task coordinat ion activit ies • validate proposed approaches and processes wi th three metrics: total project complet ion time ( T P C T ) , mean feature complet ion t ime ( M F C T ) , weighted-mean feature complet ion time ( W F C T ) 1.3 Organization of this Thesis This thesis consists o f f ive major sections. Chapter 2 describes the background behind the task coordinat ion problem. Relevant literature and the current state o f project management tools are reviewed in Chapter 3. Chapter 4 presents the approach o f evaluating task coordination 3 problems. Chapter 5 describes the architecture and user interface of NextMove, an implementation of the proposed methodology. Chapter 6 summarizes the results of simulations, which validates the effectiveness of the heuristic process as described in Chapter 4. Chapter 7 presents the results of the experiments in which the usefulness of NextMove was evaluated using human subjects. Lastly, conclusions and suggestions for future work are outlined in Chapter 8. 1.4 Contributions of this Thesis M y contributions in this thesis are: • proposed a heuristics-based process to determine the most important tasks in a project at various points in time, as well as determine the most suitable person to perform those tasks • implemented NextMove, which helps teams coordinate their tasks and increase project transparency • validated the approach through simulations of real-life project situations • evaluated the effectiveness of the prototype tool NextMove through testing with human subjects • parts of the work of this thesis was published in three conferences ([1], [2], [3]) 4 CHAPTER 2 C O N T E X T AND P R O B L E M S T A T E M E N T A software project poses as much of a managerial challenge as a technical challenge. While careful planning is essential to risk management and budgeting, it is equally important for a team to effectively monitor and respond to emergent changes quickly and effectively. Agile development methods reflect the recognition of the need to break away from the reliance on heavy, documentation-intensive, rigid up-front planning. Following the trend of globalization, software developers start to experiment with distributed development in the hopes of increasing competitiveness by reducing cost and leveraging the talents of software practitioners around the world. Unsurprisingly, the new world of distributed agile development adds a new dimension of complexity to software project coordination and poses a number of new problems. This chapter first discusses the various facets of distributed agile development, and how its task coordination problem is different. Then, the problem statement and model wi l l be defined within the context discussed. 2.1 Characteristics of Distributed Agile Development This thesis focuses on software development teams that are individually and geographically distributed (instead of teams that are composed by distributed sub-teams, each with its own team structure) and make use of agile development methodologies, more specifically, Scrum [4], Feature-Driven Development [5], eXtreme Programming [6], Lean [7] and Crystal [8]. Direct communications, which play an integral part in agile methodologies, are often made difficult by 5 the inadequacies of remote communication technologies, such as video conferencing. The realities of distributed development are preventing teams from taking full advantage of the strengths of agile methodologies, as discussed in the following sections. 2.1.1 Distributed Development In recent years, distribution of software development teams has become an increasingly common practice in the software development industry. The motives behind such moves include [9]: taking full advantage of the global resource pool, localizing products so that they are better suited to different markets, reducing development time by "around-the-clock" development and because of the multinational nature of the organization. However, distributed software development teams often run into difficulties in coordinating their work, especially when they are using agile methodologies. 2.1.2 Agile Methods Proponents of lightweight agile methodologies contend that agile methods are better suited to the fast pace of contemporary software projects than traditional plan-based development methods [10]. agile methods put a much stronger emphasis on informal communication over formal written artifacts [11]. While direct, informal communications provide flexibility to the project, they are not suitable for information retention, which is sometimes necessary in order to exchange information within a distributed group. One example of an agile methodology is the Scrum process, as coined by Ken Schwaber [4]. In the Scrum process, a team would keep a list of identified tasks, commonly known as a backlog. Each task is a short-term activity which should take no longer than several days to complete. The 6 backlog is continuously updated with new tasks added or irrelevant tasks removed as changes in the project occur. Tasks are prioritized and each team member would commit to a number of tasks. Short daily meetings, known as the daily Scrum or daily stand-up meeting, would be held throughout the course of the project in order for the project manager to collect feedback from team members. Scrum meetings serve two additional functions: team members get to know one another's progress and gather information on one another's work. Hence, they wil l know the needs for each team member to get his/her job done. After the Scrum, the project manager would try to resolve issues that were raised during the Scrum by allocating resources to perform the tasks on hand. In the realm of feature-driven development, a feature is defined as a client-valued functionality that can be implemented within a specific (and usually short) timeframe [12]. Each feature is treated as an incremental unit of the system. Feature-driven development methodology focuses on features instead of tasks in planning because features are what customers understand [13]. After each milestone, the priorities of the planned features can be revisited by the customer and adjusted accordingly. The advantage of feature-driven development is that the project is divided into incremental and executable parts, hence enabling the organization to have early, partial releases. 2.1.3 Process Models While an overly-detailed and rigid process model may prove to be detrimental to the agility (i.e. the ability to respond to changes) of the team, an abstract software development process model is useful as a communication tool to convey the general direction of the project. A process model describes the various activities in a project and their inter-dependencies, as well as any 7 documents or artifacts which are related to the activities. There are a number of languages and meta-models, ranging from data flow diagrams to Petri nets [14]. The work in this thesis focuses on development processes which are defined by the Software Engineering Process Meta-model (SPEM)[15]. 2.1.4 Team Structure Compared to project teams in traditional software development, agile software development teams are less hierarchical in nature. A n agile team usually consists of a project manager (or project leader) and a number of team members of equal standing within the organization's structure. Agile software projects often employ the concept of roles. A role describes a set of activities performed and a set of artifacts owned [16] [17]. When a project team member fulfills a role, he or she is responsible for performing the activities and maintaining the artifacts that are included in the role. Each team member may take on different roles at various times or multiple roles simultaneously over the course of the project. For example, at the beginning of the project, a team member may take on the role of a requirements analyst; later on, that member would fulfill the role of implementer and developer of the user interface; towards the end of the project, the team member would become a tester and ensure certain functions are working properly. Hence, Agile project teams prefer generalists who are versatile over specialists who can perform only one type of task extremely well. 2.1.5 Difficulties in Agile Distributed Development While agile methodologies and distributed development have their advantages, a number of issues are encountered where such problems do not exist in a traditional, co-located 8 environment. These issues are the root causes of the task coordination problem on which this thesis is focused. Studies such as one by Herbsleb and Mockus [18] have shown that distributed development may indirectly introduce delay, as compared to same-site development. These delays may be attributed to the lower frequency of communications, difficulty in exchanging information with other team members, and lack of understanding of the work that distant colleagues are performing. These problems are not confined to teams that are globally distributed. Teams that are separated merely by walls within the same building may face similar communication problems. Agile approaches such as Scrum processes often experience difficulties in distributed development environments [19]. For example, it is difficult to replicate the daily Scrum when a team is scattered across the globe. Logistics concerns such as finding a suitable time for meeting and limitations in technologies are the main drivers of such a conundrum. Hollan and Stornetta contend that video/audio conferencing technologies have not been able to emulate the effects of social presence in a face-to-face communication environment [20]. Current technologies in video conferencing and instant messaging do not provide the necessary level of interaction to facilitate face-to-face, informal communications, which in turn create and maintain working relationships. This is because these technologies do not fully conform to established social mechanisms, such as eye contact and tone of voice. 9 A flat hierarchical team structure that is often found in software development teams poses a number of issues in communication and information seeking. With the absence of informal face-to-face communication between team members, the project team is left with two options: to have the project manager collect all the information and allocate tasks to team members, or have the team members coordinate amongst themselves. Each approach has its own disadvantages. Effective, timely assignment of tasks requires an experienced and skilled project manager. Before deciding on the allocation of tasks, the project manager would need to objectively assess each team member's skills and experiences, make sure that tasks wi l l be completed on time, and prioritize the tasks. While an experienced manager can often make correct decisions using intuition and experience, such a task may prove to be overwhelming for new, inexperienced project managers. In addition, in the event that the project manager has to take a leave from the project, contextual information on allocation decisions would be lost. When team members self-organize, they tend to choose tasks that they like or tasks that they know how to perform because they understand the context. However, the tasks that are chosen purely on personal preferences may not be the tasks that are most critical to the project's success. In addition, since the chain of command in a software project is not always clear, team members may have difficulties in deciding whom they should talk to in order to communicate any problems that they encounter (e.g., one team member cannot continue his work because a modification is needed in another module for which another team member is responsible). Similarly, team members may not know whom to ask for information on a particular part of the system because each team member's responsibilities are not always visible to all members. This 10 perceived lack of transparency in the project would lead to communication overhead and replication of work. 2.2 Contextual Model and Problem Definition This section describes a scenario and the contextual model of the task coordination problem, to both of which the work of this thesis can be applied. The scenario and contextual model are based on the issues described in the previous sub-section. 2.2.1 Scenario Consider a group of three students, Joe, Bob and David (no relation with the author of this thesis) who are developing a web application for travel agencies as a summer project. Prior to the start of the project, they decided to adopt some agile practices, such as feature-driven development, in order to beat their competitors to the market. Joe is known as a software development guru who is good at most areas (e.g., documentation, Java programming, testing). Bob has some experience in testing, but not much experience in Java. David has worked on several Java school projects before but has not taken the course on testing yet. Throughout the summer, the team cannot meet in person because of a number of reasons: Joe is working as a co-op in Japan; Bob went home to Taipei for the summer; and David is staying in Vancouver. Originally, they designated Joe as the project manager. However, one month into the project, Joe soon found himself overworked because he needed to keep track of the progress of himself, Bob and David. Since Bob and David did not have much opportunity to communicate directly because of time differences (both Bob and David sleep and work during normal hours, 11 unlike graduate students), they rely on Joe to determine what to do next and allocate tasks to them. When Joe is busy with his job, sometimes he would post the pending tasks on the team's private message board and let Bob and David work out who is doing what. It turns out that David and Bob tend to volunteer for tasks that they like better. Some tasks were left untouched for weeks, and others were done unnecessarily quickly - both Bob and David, albeit unknowingly, were in a race to perform same tasks that they liked in order to force the other person to perform the undesirable tasks. Hence, the team needs to improve its coordination: to find out which tasks are pertinent at the current time, and to find the most suitable person to perform those tasks. 2.2.2 Feature Model At the planning phase of the project, a number of features would be collaboratively defined by the customer and the project team. A feature consists of two major attributes: • priority: the importance of the feature, which is also related to the order in which the features should be completed • size: the size of features in terms of function points [21] 2.2.3 Process Model In the context of this thesis, the process model serves as the template for the tasks and artifacts in a project. In a process model, there are a number of inter-related activities. Each activity represents a step towards attaining the overall goal of the process. A n activity may be associated with one or more work products as inputs or outputs. A n activity has a number of attributes: 12 • workflow relationships: how the activity is related to other activities in the workflow of the process • time required: the time required to perform this activity, expressed as a percentage of the total execution time of the process • required skills: the skills required to perform the activity and their relative importance In addition, a number of work products are defined within a process model to serve as guidelines to the types of artifacts to be created in a project. In each work product model, the work product's relationships with one or more activities and dependencies on other work products are defined. 2.2.4 Project Model The feature model and the process model combine to create the project model. For each feature, an instance of a process is created. A process instance consists of a set of tasks (instances of activities) and a set of artifacts (instances of activities). The relationship between features, activities and tasks can be more clearly illustrated in the following matrix: Figure 1 - Matrix depicting Relationship between Activities, Features and Tasks 13 The relationship between work products, features and artifacts are similar: Figure 2 - Matrix depicting Relationship between Work Products, Features and Tasks 2.2.4.1 Task Mode l The work in this thesis supports two types of tasks in a software project: process-based tasks, which are created from the combination of features and the process model; and ad-hoc tasks, which are tasks that are not regularly performed. Although ad-hoc tasks may not directly affect the final deliverables or intermediate artifacts, they are nonetheless critical to the completion of the project. A n example of an ad-hoc task is the setup of a piece of test equipment. Both process-based and ad-hoc tasks have the following attributes: • workflow relationship: the preceding tasks that need to be done before the task can be executed, as well as subsequent tasks that depend on the task. The workflow relationship of a process-based task is inherited from the activities relationships • effort/time estimation: the amount of time required to perform the task; for process-based tasks, the effort estimation for process-based tasks are calculated as a function of feature size • task readiness: whether the task is ready to be executed (e.g., the tasks on which it depends has been completed) 14 • required skills: skills required to perform the task, and their relative importance; for process-based tasks, this attribute wi l l be inherited from the activity model. These required skills wi l l be compared with team members' skills to determine the most suitable task performer A process-based task would have a number of additional attributes: • activity: the activity (goal) that the task would (partially) satisfy • feature: the feature that the task is related to • artifact relationship: input/outputs of the task 2.2.4.2 Artifact Mode l In the NextMove project model, artifacts refer to both textual and graphical documentation, such as use case, or software code. A n artifact is modeled with the following attributes: • work product: the class of work product that the artifact belongs to • state: the current status of the artifact; an artifact's status is modified by different tasks • artifact dependencies: other artifacts that may impact the artifact when they are changed 2.2.4.3 Team Member Mode l A project team consists of one or more team members. Each team member has different sets of skills and attributes. These skills and attributes include: • skills: skills and experience in performing the aforementioned types of tasks, such as programming skills, and documentation skills • familiarity with artifact: as the project progresses, certain team members would be more familiar with certain sets of artifacts 15 • workload: the amount of work the team member has taken on at that time This approach to modeling enables a distinct separation between the abstract process model and the physical project. According to van der Hoek, Redmiles et al. [22], formal processes work best when they are used to provide high-level guidance and support rather than to impose a strict sequence of activities on the team. The separation of process and project allows for more flexibility in the execution of the project. The U M L object diagram in Figure 3 illustrates the relationship between processes, activities, tasks, and artifacts. r espons ib l e for P r o c e s s k} TT1 Rc >le -/ \ Team Member per fo rm d e p e n d s on Activity 7V re la tes to -5H re la tes to Work Product ft— per fo rm Task modi f ies Artifact d e p e n d s on Feature re la tes to Figure 3 - U M L Class Diagram of the Project Model 16 2.2.5 Problem Definition The establishment of the NextMove project model allows the task coordination problem to be described in a more formal manner. In essence, the project manager needs to arrange a set of tasks T, which consists of n tasks, into an n-tuple that ranks the tasks from the most to least important at an instance of the project. The arrangement of the n-tuple is constrained by the attributes of the feature, task, and artifact models. Then, for each task r„ each with a set of requirements R, from the task model, the project manager needs to arrange the team members M into another n-tuple. The ranking would be produced according to their attributes Am from the team member model. A t the end, the two n-tuples would be combined to determine the most pertinent task for each team member. Furthermore, any method that successfully resolves this problem must also satisfy the following criteria: • communicable: the task-allocation process should be clear, so that team members understand the rationale behind each decision. This is especially important for a distributed team, because the project manager may not be available at all times to respond to queries from team members • non-intrusive: the method should not impose a strict course of action on each team member. Team members should be allowed to use their own judgment in organizing their tasks according to the current situation • flexible: the method should be able to accommodate both process-based tasks and ad-hoc tasks • allow feedback: allow team members to be involved in the task-coordination process 17 CHAPTER 3 REVIEW OF CURRENT STATE OF THE ART This chapter summarizes and reviews the current state of the art of task coordination and is divided into three parts. First, research and methodologies on the task coordination problem wil l be reviewed. This is followed by a summary of literature that is relevant to the design of NextMove. Finally, since NextMove is billed as a project management tool, it is useful to review some of the currently available project management tools. 3.1 Current State of the Art in Coordination Methodologies Although there are a number of studies on task delegation on multiple-member teams, there have been surprisingly few that focus specifically on software development or distributed projects. This section summarizes three types of methodologies that focus primarily on the subject of task allocation. 3.1.1 Policy-based Task-Assignment Strategy Shim, Lee et al. [23] propose a model-based methodology that allows modeling of task assignment policies. The proposed methodology makes use of UML-based meta-models to describe the four elements that are crucial to task assignment: organization, process, work products, and project. These models hold information on team structure policies, task assignment, artifacts and project state, respectively. In order to test and demonstrate the usefulness of the proposed methodology, a prototype has been implemented. A study on the 18 effectiveness of the prototype showed a significant reduction in request queuing time (i.e. time between request initiation and request resolution). Shim, Lee et al. contend that their policy-based methodology is adaptable to different development environments, as the policy models are extensible. However, there are some omissions in the model, such as the definition of workflow and the relationships between tasks and artifacts. In addition, the model did not allow for changes in policies when emergent situations arise during the course of the project's execution. 3.1.2 Multi-Criteria Task Allocation Shen, Tzeng et al. [24] proposed a multi-criteria task assignment model within workflow management systems. The model describes qualitative measures, such as individual capabilities, using linguistic scales and those measures are quantified by fuzzy numbers. The evaluation procedure assesses the suitability of a worker for a task using four criteria: capability, work load, similarities of tasks (between the task being evaluated and tasks previously performed by worker), and relationship between team members (for tasks which require more than one performer). The ratings of all the evaluated criteria are summed to produce an overall score for each candidate task performer. The multi-criteria task allocation methodology offers a partial solution to the second part of the task coordination - how to find the suitable candidates for each task. However, the methodology assumes that the priority of the tasks has been pre-determined, or the workflow is sequential in a way that it is obvious what the next task that needs to be performed is. Zhang, Cao and Zhang [25] also proposed a similar approach to improve task distribution in workflow management systems. 19 3.1.3 Optimizing Task Allocation using Genetic Algorithms Duggan, Byrne et al. [26] propose an optimization method for task allocation during the construction phase of software development. This method makes use of multi-objective evolutionary algorithms ( M O E A ) and genetic algorithms (GA) . Duggan, Byrne, and Lyons contend that a software project has a number of goals and they sometimes conflict with one another (e.g., minimization of defects vs. minimization of time to release). M O E A helps the project manager by balancing various objectives and generating sets of optimized schedules for each team member. While this approach is effective for estimating costs and schedules, problems may arise when it is used to perform task allocation during the execution of the project. As task-coordination and allocation problem are treated as static optimization problems, emergent changes are not taken into account. Moreover, since the operations in task allocation using M O E A are complex and hard to understand, it is up to the project manager to convince team members that following a rigid, computer-generated schedule is in the best interest of the project. Chang and Christensen [27] also proposed a similar, genetic algorithms based model for managing task schedules. 3.1.4 Summary Each of the aforementioned methodologies represents a type of task coordination strategy, each with its advantages and disadvantages. Shim, Lee et al. suggested that task allocation decisions should be based upon a set of predefined policies. While their solution is relatively easy to implement (compared to the other two methodologies), it is also inflexible in terms of dealing with changes during the course of the project, as the policies may not account for all possible scenarios that may occur in the project. On the other hand, Shen, Tzeng et al. proposed the use of 20 multi-criteria decision mechanisms to resolve the task allocation problem, but did not address the question of task prioritization. The task allocation optimizer as introduced by Duggan, Byrne et al. treats the task coordination problem as a static analysis problem, which may not be suitable in the highly dynamic world of software development. The work of this thesis attempts to draw from the strengths of the reviewed work and address the weaknesses of the prior arts. 3.2 Design Consideration This sub-section reviews the concepts and framework that were taken into consideration as NextMove and its underlying heuristics process were designed. 3.2.1 Activity Theory Barthelmess and Anderson [28] analyze the current state of the art in software development environments in three aspects of collaborative work: subjects (e.g., roles), objects (e.g., tasks), and community (e.g., teams). They classified collaborative environments into three levels: coordinated, cooperative and co-construction. Collaboration on a coordinated level usually has team members follow a rigid and pre-defined sequence of work, and work is organized to lower the need for interpersonal collaboration. On the cooperative level, team members have to communicate with each other in order to accomplish tasks which are interdependent. Co-construction implies that work practices are repeated and ways of accomplishing a task (i.e., the process) are being constantly refined. Barthelmess and Anderson contend that the three levels constitute the instantaneous status of an activity, and the status can change dynamically during the course of the project. 21 Barthelmess and Anderson argue that a large part of the problems in contemporary software engineering is concerned with communications. The pace of change in the software development world does not allow the development of common grounds which allow practitioners to effectively communicate with each other. Hence, it is important to have other means of facilitating communication among team members. Barthelmess and Anderson contend that most process-centred management tools are focused solely on activities on the coordinated level and overlook the aspect of collaboration. In this way, they are merely tools that rigidly assign roles to team members. Barthelmess and Anderson propose that tool developers need to incorporate communication support into management tools. Team members should be able to select a sequence of actions instead of being "told" what to do by the environment. In addition, the environment should allow team members to convey explanations or context about a task or an artifact. 3.2.2 Continuous Collaboration Van der Hoek, Redmiles et al. [22] contend that existing software engineering tools, such as configuration management systems, put limitations on human activity and creativity, because of the tools' reliance on the formal process. A process-based tool that coordinates tasks according to predefined processes fails to support the informal or ad-hoc practices which are essential in software development. On the other hand, although C S C W tools help team members gain awareness of one another's activities and facilitate self-coordination, increase in the scale of the project often leads to information overload. 22 Therefore, van der Hoek, Redmiles, and Dourish attempt to combine the strengths of both process-based and C S C W approaches and propose a new approach called continuous collaboration. For example, mechanisms are present to inform team members of the others' activity that might be relevant to them in order to create a sense of awareness. Formal checkpoints and measures would be used to provide a high-level guideline on the direction of the project. Van der Hoek, Redmiles, and Dourish contend that current software engineering tools are not capable of effectively supporting continuous collaboration. Hence, developers have to construct tools such that they do not impose a certain mode of collaboration onto the users. 3.2.3 Group Awareness in Distributed Environment Gutwin, Schneider and Penner [29] conducted a study on the dynamics of a distributed software development team, with focus on how team members maintain awareness of one another's work over remote distances. According to Dourish and Bellotti [30], group awareness is defined as the understanding of the others' activities and is essential for any co-located or distributed group to collaborate successfully. Through interviews with developers involved in three distributed open source projects (NetBSD, Apache server, Subversion), they observe that the project teams relied on both formal (e-mails on the project mailing list) and informal written communications (IRC chat) to maintain awareness of what others are doing. The effectiveness of coordination and collaboration of the three projects are shown by the success of the resulting product. However, they note that the nature of open source projects may make the results of their study not directly applicable to all distributed projects. The projects had flexible schedules and did not have to face the pressure of releasing a product at a tight timeframe; because of the voluntary nature of open source projects, 23 members who are not able to maintain awareness using the teams' methodology would drop out of the project. Nonetheless, Gutwin, Penner, and Schneider conclude that some principles found in the study can be applied on a broader context. A distributed team should provide different venues for verbal communication, which is essential for maintaining group awareness. Also, the study found overhearing (i.e. observing discussions without actively participating in them) a useful means of gathering awareness information. Gutwin, Penner, and Schneider suggested that distributed collaboration tools, such as version control systems and messaging systems, should be designed to better capture group awareness information, reduce information seeking time for users, and increase the transparency of the project. 3.2.4 Summary The aforementioned works on collaboration provided guidelines for NextMove's general design. NextMove is designed to provide flexibility such that team members wi l l be able to select a course of action instead of having a tool dictating their actions. The user interface wil l explain the reasons for certain tasks to be recommended. NextMove takes advantage of formally-defined process models to provide guidance on the project. Finally, measures are implemented to convey group awareness information (e.g., who is working on which task) to the team members in order to increase project transparency. 24 3.3 Project Management Tools There are numerous project management tools currently on the market. This section reviews general-purpose project management tools (e.g., Microsoft Office Project) as well as software development project specific tools and focused on the aspect of task coordination and tracking. 3.3.1 Microsoft Office Project Microsoft Office Project [31] is one of the better known general purpose project planning and management tools. Its latest version, Project 2007, emphasizes project scheduling, presentation of project information (through charts and diagrams), and provides guidance on setting up projects. In Project 2007, tasks are assigned by the project manager. Team members and the project manager exchange task information (e.g., current progress) through the page created for the task. 3.3.2 Open Workbench Open Workbench [32] is seen as an open-source alternative to Microsoft Office Project. Its general view shows the list of tasks and their schedules as a Gantt chart, as well as a list of resources (e.g. labour, equipment, material) and their schedules in the corresponding times. Figure 4 below is an overview of Open Workbench's interface. 25 [1 pjopen WorkbeiKtt {Projects - Gaatt Chart] .1.1 x | • He Edit _cw Jooti _hdow adp _i_J»J l ^ t f Jd>tf A j*=!*rl3fc*> -1 X M ft' l lJI iStf^ _B|(AII Resources) j>j|T»«k _Ja si fea <S i" % ; J* «, 3 Ganu Chart PhaM Laval Gantt zl Implement Ship R starvation 04/09/2007 07/09/2007: $ Tact Ship Retervation 08/09/2007 13/09/2007 • Prototype Flight Retervation 05/09/2007 10/09/2007 Test Train Reservation 06/09/2007 08/09/2007 j :: [ 7 est Hotel Retervation 04/09/2007 04/09/2007 David 0 00 0 00 000 0 00 0 00 ooo 0 00 0 00 0.00 000 000 000 0 00 ooo 000 ooo ooo H (lob mm 0 00 000 0 00 000 0 00 0 00 0.00 ooo 000 000 ooo 0.00 0.00 000 ooo 0.00 U [JoV~ 0.00 0 00 0.00 0.00 000 000 0 00 0.00 0.00 000 dob 0.00 0.00 0 00 ooo 0.00 0 00 n I _ - ™ Figure 4 - Open Workbench General User Interface In Open Workbench, tasks are created, prioritized and linked (i.e. defining the relationships between tasks) manually. The tasks are then assigned to resources and the estimated time for the team member to complete the task is entered, as shown in Figure 5. -2JXJI Task Properties - Implement Ship Reservation General Resources | Dependencies | Advanced | Notes | £jssigned Resources: ID Name Estimate Actual David 4.00 J Project Resources: Release I I ID Name Category f Joe Developer David Developer H Bob Developer Assign OK Cancel Help Figure 5 - Open Workbench Task Assignment Dialog 26 Users are able to look at the performer of each task by viewing the individual task information dialog; however, there is no quick overview of each team members' current work in progress or past work. 3.3.3 Rally Agile Life Cycle Management Rally Agile Life Cycle Management suite [33] ("Rally") is tailored towards agile software development teams. It incorporates a number of agile practices, such as backlog and user stories. In Rally, tasks and user stories are created, prioritized, and assigned manually. In the team status view, users would be able to see the list of tasks with no owner, as well as the task lists of each individual team members (Figure 6). For each task listed as owned by a team member, there is a state indicator which displays the current state of the task (e.g., defined, in progress, completed). Al Rark ID Name WoiV.Ptocbct Release Slats Capacity Estimate To Do Actuate Ovwner j Q _*><>>' mer (7 Tmdn) .-• - .' .., ' y . 7.0 . 7.0 ' M • i 1.0 TA2 O-dion the tort to see on overview of the Flaiyapo SI: Introducton to ItefoOon Zero Tutorial. Cbck tn the plus iron to the left B[> hi 1.0 1.0 &0 1.0 TA! CM on this tsvt. tn finwQatf tn ttiis <M»I pace trw <hi Task. 51: Introduction to rotation 2ao Tutnri.il, Clirfc nn thr •ptus' k-nn tn the Irft 13 1 1.0 1.0 &0 3.0 TAS Add tlw.stw S3: Add Next Iteration S,RH>-ftsr • 1.0 1.0 £ 0 3.0 TA4 AuW Rdaou* S3: Add Next Ite»*tlwi SRdeas* EI' 1.0 1.0 &® 5.0 TA7 Wan I teuton CS: Plan New Iteration L0 1.D &® 6.0 TA9 Update Lteer itajnec iti: Track Iteration BHD L0 1.0 6.0 TAB Update! asb Sb; Track Iteration DI£1HJ 1.0 1.0 £ 0 2.0 TA3 AddTeair.ManbeiB S3; 5e«> TWJC Te.m EIL?J[5lJ '.0 '.° t»b3boH»bbob.com £i>£J| Figure 6 - Rally Dashboards: Team Status 3.3.4 Xplanner Xplanner [34] is an open source project planning tool for large eXtreme programming teams. The tool focuses on task tracking and increasing project transparency. Tasks which belong to the same user story are listed and their progress are indicated with coloured progress bars (Figure 7). The progress of user stories within an iteration are tracked and displayed in a similar way. Unfortunately, due to technical difficulties, the author of the thesis was unable to perform an in-27 depth review in the task coordination and allocation aspect of the tool. In Xplanner's documentation, there is no mention of guidance on task coordination. r Stilty: I sr hunitly III rturViItt E*a L<* V*H ii= eookmatki xpck Unlaw tjiip iff | '< Www Story; L« ErondtyBrdorttii XPlanner Story: Ltr Erondly BrdorVdt 1 http//enamplo com/design nolcs txl Integrations I People 2 3 2 2 3 Ihis is thi In*! leelure Another one Tliis is another feature Ewlmited Hours: 14 0 Actual Hours: 6 6 Taak Mama Typa Lodnly sirn Ultura Luduty .lutti Feature Oudefy ulinii Milori Eat Acc. Diapoaition Action* 30 ND Dlmnd Qt* E3 80 ND Planned :/ 13 3.0 ND Blunnid (jjf i l-dil Story I Create Task Gr* Add Nole Figure 7 - Xplanner Story View 3.3.5 Summary The aforementioned tools provide varying levels of progress and task tracking, with examples including the Open Workbench's graphical project overview and the iteration and story view of Xplanner. However, they are lacking in support or guidance for task prioritization or allocation. NextMove does not intend to replace these tools, but to supplement them by filling in the missing area of support. 28 CHAPTER 4 TASK COORDINATION STRATEGY This chapter is dedicated to describing the strategies behind the task recommendation process performed by NextMove. The process involves two steps. The first step is to prioritize currently available tasks. Then, a suitability ranking of team members is produced for each available task. This process is continuously executed to produce a personalized recommended task list for each team member as circumstances change over the course of the project. 4.1 Task Prioritization Throughout the course of the project, the team backlog wi l l contain a number of tasks to be performed. The task-prioritization process ranks them according to four criteria: schedule, artifact relationship, feature priority and workflow dependencies. Given a task, t, that would modify an artifact a into state s at the time of r's completion, r's priority score in terms of artifact relationship depends on the number of other tasks that require a to be at state s in order to start execution. In addition, when a team member is waiting to execute a task that requires a, he or she can request a to be changed into state s. Hence, the priority score (PS) of a task, in terms of artifact relationships, is derived by: artifact ~ Ptasks ^ tasks Pteam^-team where R t a s ks is the number of tasks that require the artifacts produced by t, and R t eam is the number of requests to work on the artifact from the team. Ptasks and p t e a m are weight factors that 29 can be adjusted according to the needs of different environments. The artifact relationship factor gives team members a way to provide feedback on what needs to be done, from their point of view. For a set of sequentially related tasks, critical-path analysis (CPA) [35] can be performed to estimate the minimum amount of time needed to complete the set of tasks, as well as to identify the tasks that are critical to the timely completion of all tasks. Before calculating the priority scores in terms of schedule for each task, C P A is performed on groups of tasks belonging to the same feature. The analysis determines the earliest and latest start time of the task, relative to the start time of the feature. In turn, the schedule priority score of each task is determined by: 1 TPD - LST PS\~U„J„I„ — ; r + schedule l + ' L S T _ E S T ) • where L S T and EST denote the latest and earliest start time of the task, respectively, and TFD is the total time required to complete the feature. The first term of the equation determines the degree of criticality of the task. The second part of the equation represents the degree of urgency of the task. Hence, a task that is on the critical path and has to be performed soon wil l be rated the highest. Feature-driven development and frequent releases are two important concepts in an agile development environment. In an agile project, a number of features (also known as user stories) are defined. A feature often represents a functionality that is useful for the user. A team schedules the completion of a certain number of features in each release. Hence, a customer can frequently evaluate the progress of the project and make requests for changes early. Feature priority is an important consideration in the task-prioritization process. It is modeled as follows: 30 ^feature task where Ftask denotes the relative priority of the feature for which the task is assigned. The value of Ftask is directly proportional to the importance of the feature. The workflow dependencies category is added to make sure that tasks that are ready to be executed are recommended over tasks that are not yet ready. The formal expression for this category is ^task ~* (-^readiness = -^max ) where E t a s k is the Boolean expression for whether or not the task is ready, and P S m a x is the maximum rating possible in the task-prioritization process. Hence, when a task is ready to be executed, it is always rated higher than tasks that are not ready. The priority scores of a task are aggregated using a weighted sum. D C _ yl D O • J D O • 1 D C , D O ° final artifact artifact feature feature schedule schedule readiness where the X values are the weights of each prioritization category, and can be adjusted by the team as different situations call for different emphasis. For example, the feature priority category would be weighted higher than the others i f a team wants to make sure that the important features are completed on time for the next release. 4.2 Task Allocation A s described by Duggan, Byrne and Lyons [26], task allocation is a multiple objective problem (please see Chapter 3). The goals of the task-allocation process include the following: 31 • locate the most skilled person for the task: in theory, a person whose strengths match those that are required by the task wi l l perform the task most efficiently • balance workload: in a large project, the team wi l l work more efficiently i f tasks are distributed among the team members instead of overburdening a minority of members who excel in many fields • minimize overhead: for tasks that require a high degree of familiarity with the related artifacts, it is better to locate a team member who has previous experience with the artifact than to force another member to spend time familiarizing himself or herself with the artifact before performing the task When allocating tasks, project managers need to strike a balance among these goals. Reaching a desirable solution to this problem requires years of experience, intuition and rational thinking. In addition, there may not be a team member whose skill set is perfectly matched to the requirements of the task. In this case, the team member who satisfies the most important requirements of the task would become the most desirable person to execute the task. The analytic hierarchy process (AHP) [36] is used as part of a conflict-resolution strategy in the task-allocation process. A H P assists decision makers in modeling a multi-criteria evaluation problem in a structured way. The advantage of A H P is that it can evaluate qualitative factors (e.g., team member skills) alongside quantitative factors (e.g., team member workload) in the same process. 32 A H P can be used to determine which team member is most suitable for a task in terms of skills, workload and familiarity with certain parts of the system. In the A H P , the relative importance between a pair of attributes is expressed in a 9-point scale. The following example with fictional figures illustrates how A H P is used in the proposed approach. Coming back to the scenario in section 0, Joe, Bob and David would like to know the most suitable person (and the next best candidates i f the most suitable person is not free) for the task of architectural design for the file transfer functionality. Table i illustrates the skills required to perform the task and each skill 's relative importance to the task's successful completion. The number in each cell should be interpreted as "the importance of <attribute name of the row> as compared to the importance of <attribute name of the column>." For example, the cell in row 1, column 2 of the table should be read as "the importance of Java programming skills as compared to the importance of documentation skills." The values in Table 1 are normalized in Table 2 in order to obtain the weight of each requirement, a . Table 3 shows the team member's relative competencies in Java programming. Those values are also normalized in a fashion similar to that in Table 2 in order to produce each team member's overall competency in Java (r in the following equation). In turn, the documentation and design skills of Joe, Bob and David are compared and normalized likewise. The result of the normalizations is shown in Table 4. The skill-based suitability ratings of each of the team members are calculated as follows: SR — CC Y ~\~ OL Y -\- CX Y n Java n,Java Documentation n,Documentation Design n,Design 33 where r ^ v / is the normalized relative value of team member n's ability in a particular skill. The skill-based ratings of the three team members are ranked in Table 5. Java Documentation Design Java 1.00 3.00 5.00 Documentation 0.33 1.00 3.00 Design 0.20 0.33 1.00 Table 1. Relative importance or tasks Java Documentation Design a Java 0.65 0.69 0.56 0.63 Documentation 0.22 0.23 0.33 0.26 Design 0.13 0.08 0.11 0.11 Table 2. Normalized values of Table 1 David Bob Joe David 1.00 0.33 0.20 Bob 3.00 1.00 0.33 Joe 5.00 3.00 1.00 Table 3. Team members' relative Java programming abilities 34 David Bob Joe Java 0.11 0.26 0.63 Design 0.11 0.11 0.78 Documentation 0.75 0.18 0.07 Table 4. Normalized relative competencies for each team member Rating Joe 0.611 David 0.423 Bob 0.262 Table 5. Ranking the Skills-based Suitability Ratings of the Team Members In order to take into account workload, an additional factor must be included in the analysis. The workload factor from above can be interpreted as the inverse of how critical the task is. If a task is critical to the project and has a strict skill requirement (i.e., it cannot be performed without certain skills), the workload factor would have a lower rating relative to other skills. Likewise, each team member's familiarity with a particular part of the project is evaluated as one of the attributes in the analysis. Overhead due to context switching is incurred when the team member is unfamiliar with the sub-system that he/she is working on and needs time to acquaint him/herself with the artifacts. Hence, a team member who has previously worked on the same part of the system is rated more highly than a team member who has no previous experience with it. 35 4.2.1 Recommended Task List The end results of the task-prioritization process and the task-allocation process are combined to produce a recommended task list for each team member. The recommended task list would be sorted by the overall recommendation rating, RR, for each task, calculated by: RRoverall = CCTPRRTP + aTARRTA where RR T p and RRTA are the recommendation ratings produced from the task prioritization, and 0(.TP and CCTA are constants that determine the importance of task-specific priority ratings and team-member-specific suitability ratings. 4.3 Benefits of Strategy The strategy proposed in this chapter assists project managers in coordinating tasks by aggregating and analyzing different types of input in an organized, rational fashion. The following features make the NextMove framework an attractive option for project managers: • understandable: compared to optimization algorithms such as multi-objective evolutionary algorithms, A H P is much more transparent and understandable • choice of action: team members are more inclined to make use of a system that allows them to choose from a list of recommended tasks based on their intuition and preference, rather than a system that imposes a strict course of action • flexibility: weights and constants in the calculation can be adjusted to accommodate changes of team strategies • allowance of feedback: criteria such as artifact dependencies allow team members to provide feedback and be involved in the task-coordination process 36 CHAPTER 5 IMPLEMENTATION NextMove is an integral part of the work of this thesis. It is a collaborative work assistant that is designed to help distributed teams coordinate their tasks and provide team members with an extensive overview of the team's activities. It makes use of the heuristics-based evaluation process described in Chapter 4 to provide guidance on which task is currently most pertinent to the project's success. In addition, NextMove's user interface is designed to augment the information finding abilities of the team; hence team members can more easily gain awareness of the rest of the team's activities. This chapter documents the design decisions, features and functionalities of NextMove, as well as how its features wi l l help distributed teams in coordinating their work more efficiently. 5.1 Why Eclipse? According to Grudin [37], collaborative group applications often fail because of the perceived lack of benefit of using the tool in the light of the extra work required. Hence, NextMove aims to minimize the extra effort required in order to take advantage of the tool's features. NextMove's integration with a popular integrated development environment would help reduce overhead, since users wi l l not need to leave their work environment in order to make use of the tool (i.e., having switched from their development environment and start an external tool). The Eclipse Project is an open-source project which aims to provide a vendor-neutral open development platform and application frameworks for building software. A number of 37 programming languages, including Java, C++, and PHP are supported by Eclipse natively or by projects which extend the Eclipse Platform. The Eclipse Platform is written in the Java language, can be deployed in different types of development workstations such as Linux, Solaris, Mac OS X and Windows and is extensible through its extension architecture. Over the years, a community of developers created a large variety of extensions with applications ranging from code generation to project management. Eclipse was chosen as the first deployment platform for NextMove for the following reasons: • widespread adoption: Eclipse is one of the most popular Java integrated development environments (IDE) • integration with users' workspace: as a plug-in, NextMove can integrate seamlessly into the Eclipse IDE, avoiding the overhead of requiring team members to open a separate application • collaboration with other plug-ins: NextMove may work with other Eclipse plug-ins, such as version control systems and bug tracking systems to further automate the process of information gathering • platform independent: the Eclipse Platform provides an abstraction layer between the extensions and different operating systems. Hence, NextMove may be used in different development environments 5.2 NextMove Architecture NextMove is implemented as a distributed client-server system, with the client implemented as an Eclipse extension and the server as a Java application. The system consists of a database backend, a server, and one or more clients. 38 NextMove is written primarily in Java, and a M y S Q L database is used for persistent storage. The clients and the server communicate through the Java remote method invocation (RMI) protocol. The client, implemented as an Eclipse extension, serves as the user interface. As the user performs actions on the user interface, the server retrieves the relevant information from persistent storage and performs the necessary calculations (e.g., generation of the recommended tasks list). The server wi l l then notify all clients of updates through R M I callbacks. The layer diagram below illustrates the relationship between the various high-level modules of the system: Cl ien t S i d e N e x t M o v e E c l i p s e P l u g - i n r J a v a R e m o t e M e t h o d P r o t o c o l k t > P r o c e s s i n g L o g i c k r M y S Q L D a t a b a s e S e r v e r S i d e Figure 8 - Layer Diagram of NextMove 39 5.3 Statistics and External Libraries The current version of NextMove consists of about 13,000 lines of code (excluding the external module for executing the experiments as described in Chapter 7). There are approximately 140 classes placed inside 31 packages. Approximately 950 methods were implemented. Apart from the Standard Widget Toolkit (SWT) [38] libraries provided by the Eclipse Platform, NextMove also makes use of two external libraries: • MySQL® Connector/J [39]: for connecting to the M y S Q L database backend • J A M A : a Java Matrix Package [40]: for calculations in the analytic hierarchy process (please see section 4.2) 5.4 Server The server is responsible for handling client requests, generating recommended task lists, and storing persistent project information. When the team initializes a project on the server, the server wi l l load a SPEM-compliant X M L file, which contains process information, and other project data files in order to generate the initial set of tasks and team member information. The logic component generates recommended task lists as the state of the project changes (e.g., a team member indicated that a task is completed, new tasks are added to the project), and the server notifies all active clients. The server only generates and sends information that the user is currently viewing in order to reduce processing load on the server and network traffic. In addition, the server wi l l respond to team members' requests for information (e.g., other team members' backlogs, information on tasks or artifacts). 40 5.5 User Interface In the Eclipse Platform, the workbench contains a collection of workbench windows. Each workbench window contains one or more pages, and each page contains a collection of editors and views. A view is typically used to navigate a hierarchy of information, open an editor, or display properties for the active editor. [41] A combination of five views forms the framework of NextMove's user interface. The names of team members, tasks and artifacts mentioned in the following sub-sections are related to the scenario described in section 0. 5.5.1 Team Backlog View The team backlog view displays the list of tasks that are currently available (i.e. neither completed nor adopted by any team member) in the project. The tasks on the list are ranked according to the heuristics-based approach as described in Chapter 4. Figure 9 below is a screenshot of the team backlog view. H F ® : s in | n3 f & Task Name Sta . . . | ™% -Prototyping for Train Reservation Gather Requirements for Ship Reservation ! tf Test and Fix Bugs for Hotel Reservation l . H Implement Components for Flight Reservation Test and Fix Bugs for Flight Reservation 1 Implement Components for Train Reservation Implement Components for Bus Reservation mm Test and Fix Bugs for Train Reservation Test and Fix Bugs for Bus Reservation PI Implement Components for Ship Reservation Test and Fix Bugs for Ship Reservation r E Figure 9 - NextMove Team Backlog View with Tasks Coloured by their Activities 41 The left-most column contains the names of the tasks on the list, and can be coloured according to the task's activity (Figure 9 above) or the features that the tasks belong to (Figure 10 below). The colours of each activity or feature can be viewed by bringing up a pop-up legend. The middle column indicates the status of the tasks. When a task's status column is green, it indicates that all dependencies of that task are satisfied. Red in the status column indicates that another task has to be completed or started before the task is ready to be executed. The rating column explains the rationale behind the rankings. Each component in the horizontal stacked bar-graph indicates the task's rating in each of the categories. p F is : £ L Task Name Sta . . . Ratir mm f • • r Figure 10 - NextMove Team Backlog View with Tasks Coloured by their Features Right-clicking on one of the tasks listed wi l l bring up a context menu with the following options: adopt the task, or to view/edit information related to the task (Figure 11 below). For example, i f David right-clicks on the task "Gather Requirements for Ship Reservation" and selects "Adopt Task," the task wi l l be removed from the Team Backlog and added to David's Personal Backlog View. This indicates to Joe and Bob that David is committed to gather requirements for ship reservation. 42 | lask Information General Information [ Workflow infonrotion || Arifact Information | Requires: Task Name | Relationship Required by: Task Name Relationship Implement Components for Ship Reservat... finish-start relationship Prototyping for Ship Reservation start-start relationship OK Cancel Figure 11 - NextMove Task Information Dialog There are three additional functions available in the team backlog view: Adding a task, adding a feature, and manually assigning tasks. At times, process steps of a feature may need to be revisited. For example, Joe found inconsistencies in the requirements for hotel reservation when he was implementing the feature. He can create a new instance of "Gather Requirements for Hotel reservation" task and note that the requirements need to be clarified. When a process-based task is added, NextMove wil l automatically generate "downstream" tasks that are dependent on the added task. Hence, Joe wil l not need to manually add all subsequent tasks that need to be done after the new task is completed. The add feature function wil l add a new feature to the project, as well as automatically generate a set of tasks according to the process used in the project. The manual task allocation feature (see Figure 12 below) is only available for the project manager. It allows the project manager to allocate tasks to team members and to remove tasks from the team member's backlog when necessary. For example, i f Joe (the project manager) 43 thinks Bob has taken too many tasks that he wi l l not finish on time, he can de-allocate some or all of the tasks and put them back into the team backlog. = A s s i g n / U n a s s i g n T a s k s t o M e m b e r s Team, Backlog Prototyping for Tram Reservation Gather Requirements for Ship Reservation Test and Fix Bugs for Hotel Reservation Implement Components for Flight Reservation Test and Fix Bugs for Flight Reservation Prototyping for Ship Reservation Implement Components for Train Reservation Implement Components for Bus Reservation Test and Fix Bugs for Tram Reservation Test and Fix Bugs for Bus Reservation Implement Components for Ship Reservation Test and Fix Bugs for Ship Reservation Team Member;Backlog •Allocate -> <-.Unallocate]| Prototyping for Hotel Reservation Gather Requirements for Train Reservation Select a team member: • David Joe Jane Mary "I'V WW* [,' ' OK*. |prcTrTti~T^ Figure 12 - NextMove Manual Task Allocation Dialog 5.5.2 Personal Backlog View The Personal Backlog View (Figure 13) displays the lists of tasks on the team member's work list. The list in this view uses the same colouring scheme for task names, task status and recommendation ratings, as the team backlog view and the list are also sorted according to the heuristics from NextMove. For each tasks on his/her personal backlog, the user can choose to view information related to the task, mark the task as completed, or push the task back into the team backlog. 44 R F (S .£ ^ Task Name Sta. . . Rating Gather Requirements for Bus Reservation Prototyping for Bus Reservation n • implement Components Push Back Task Mark Task as Completed Edit Task Information R Show Colours by Activities F Show Coburs by Features ® Clear Colours „S Show Legend Figure 13 - NextMove Personal Backlog View with the Context Menu For example, i f David decides to take a short holiday (it is summer after all), and there are tasks left in his backlog, he can choose to push them back to the team backlog so that others can work on them instead of holding up the progress of the team (or forcing Joe to take them away). On the other hand, when Bob has finished the prototyping stage for the bus reservation feature, he can right-click on the corresponding task in his Personal Backlog View and choose "Mark Task as Completed." The task wi l l be removed from Bob's backlog and the system wil l update the status on the tasks that are dependent on the task "Protoyping for Bus Reservation." 5.5.3 Team Activities View The Team Activities View provides a by-member listing of all the current activities of the team. Expanding the tree branches for each team member wi l l reveal the tasks that the team member has adopted. Just as in the Team Backlog View and Personal Backlog View, the task names under each team member's name can be coloured by their activities or by their features. The green colour in the status bar indicates that the task is ready to be executed and red shows that the dependencies of the tasks are not yet satisfied. 45 n F Task Name Status B David Gather Requirements for Hotel Reservation Prototyping for Flight Reservation Implement Components for Hotel Reservation •BHH !-! Joe Gather Requirements for Flight Reservation Implement Components for Flight Reservation Test and Fix Bugs for Hotel Reservation a Bob Prototyping for Bus Reservation Gather Requirements for Bus Reservation Test and Fix Bugs for Bus Reservation Implement Components for Bus Reservation Figure 14 - NextMove Team Activities View 5.5.4 Parking Lot View Agile development teams often use Parking Lot Charts [42] as a progress and task tracking tool. On a Parking Lot Chart, tasks are written on pieces of post-it notes of different colours, denoting tasks for different features. Small stickers of different colours wi l l be attached onto the post-it note to indicate the change in status of the task. Hence, the parking lot board provides an easy, at-a-glance overview of the project progress. The Parking Lot View (Figure 15) in NextMove is based on the concepts of the Parking Lot Chart. In its initial state, the Parking Lot View lists the features currently in the project in the order of priority. The progress bar on the right indicates the amount of work that has been done or currently in progress for the corresponding feature. The blue component shows the amount of work that is completed, and the yellow component indicates the amount of work in progress. The length of the progress bars are calculated by accumulating the estimated duration of each completed or in-progress task. 46 When each of the features is expanded, the user wi l l be able to see a detailed task-by-task view of the status of each task. The status column in the middle is similar to the ones in the team backlog and personal backlog views, with the additional colours of yellow to indicate that the task is current under execution and blue to indicate that the task has been completed. The right-side column of each task shows the names of team members who are working on the task or have worked on the task. H H | | H H I H H i ^ ^ l H H | Sta... B Hotel Reservation Gather Requirements for Hotel Reservation David Prototyping for Hotel Reservation Bob Test and Fix Bugs for Hotel Reservation Implement Components for Hotel Reservation David B Flight Reservation Gather Requirements for Flight Reservation Joe Prototyping for Flight Reservation Joe Test and Fix Bugs for Flight Reservation Implement Components for Flight Reservation El Bus Reservation IB Train Reservation E Shi p Reservation Figure 15 - NextMove Parking Lot View 5.5.5 Artifacts View During the course of the project, team members may find themselves unable to proceed with their tasks because another task has not been completed. Hence, a mechanism is needed to enable team members to provide feedback as to what needs to be done. Through the Artifacts View (Figure 16), the user wi l l be able to initiate requests for modification for a particular artifact. For example, Joe needs the requirements for hotel reservation before he can start implementing that feature, but Bob, who adopted the task of "Gathering Requirements for Flight Reservation" is working on other tasks and has left the requirements half-done. In this situation, 47 Joe can use the Artifacts View and request the requirements document for hotel reservation to be completed. When recommended task lists are generated, the server would take these requests into consideration (please see section 4.1) and raise the priority score of the tasks that relate to the artifacts that need to be modified. : B Prototype Prototype of Hotel Reservation Prototype of Flight Reservation Prototype of Bus Reservation Prototype of Train Reservation Prototype of Ship Reservation B Requirements Specification Requirements Specificatio Edit Artifact Information Requirements Specificatio Request Artifact Modification Requirements Specification of Train Reservation Requirements Specification of Ship Reservation ffl Test Case ffl Component Figure 16 - NextMove Artifacts View 5.5.6 Log View The Log View provides a detailed listing of all actions performed on the NextMove system. Through this filterable view, the team wi l l be able to track different types of activities by different team members. Date Team Me... Action Comment [2007.07.11 - 04:10:55 PDT j David Logon 2007.07.11 - 04:11:04 PDT David Adopt Task task name: Gather Requirements for Hotel Reservation 2007.07.11 - 04:11:08 PDT David Adopt Task task name: Gather Requirements for Bus Reservation 2007.07.11 - 04:11:23 PDT Joe Logon 2007.07.11 - 04:11:31 PDT Joe Adopt Task task name: Gather Requirements for Flight Reservation 2007.07.11 - 04:11:32 PDT Joe Adopt Task task name: Prototyping for Flight Reservation 2007.07.11 - 04:11:47 PDT Bob Logon 2007.07.11 - 04:11:52 PDT Bob Adopt Task task name: Prototyping for Hotel Reservation Figure 17 - NextMove Log View 48 5.6 Design Discussion NextMove is designed to help foster communications, increase the speed and ease of information retrieval and allocate tasks. B y using heuristics to determine the most pertinent tasks to the team, the NextMove tool helps relieve the project manager of the mundane task of matching tasks with performers. In addition, team members need not to wait for instructions from the project manager in order to know which task to work on next. The concept of task adoption serves as a mental model for taking responsibility of performing a task, and allows team members to use their intuition and choose their courses of actions. The Team Activities View and the Parking Lot View help team members gain awareness of others' activities. Team members wil l find it easier to locate the right person of whom to ask questions by looking up the performers of other tasks. In addition, NextMove has a number of features that increase the likelihood of user adoption. In the Team Backlog View, team members are given the freedom to choose tasks to adopt instead of having the system impose a strict sequence of action. Hence, team members are able to use their intuition and not feel restrained by a seemingly draconian system. NextMove makes use of process models to automatically generate tasks that relate to a feature, which decreases the amount of data entry (and cost to adapt to the tool) for the team. Integration with the Eclipse Platform helps reduce the context switching time required when the user needs to make use of NextMove while he or she is working. 49 CHAPTER 6 SIMULATION A s a prel iminary step to validate the heuristics used in the task recommendation process described in chapter 4, a series o f simulations was conducted. Th is chapter describes the simulat ion model and process, presents a summary o f the results and discusses the results' s ignif icance and the l imitat ions o f the simulat ion model . 6.1 Simulation Data Three sets o f project data and two sets o f team member data were used for simulat ion. Project data includes process informat ion (activit ies, the ski l ls that each act iv i ty requires, work products, and their relationships) as we l l as feature information (priority, size). Team member information includes member-by-member compar ison o f their competencies in var ious categories. Th is subsection describes the details o f each set o f data. 6.1.1 Project Data The different project data used for the simulat ion are intended to reflect upon different types o f software projects. E a c h has a different number o f process steps (activit ies) and the three projects w i l l hereinafter be referred to as project A , project B and project C . Project A consists o f ten activit ies and five features, wi th a m i x o f paral lel and sequential activit ies. The names, priori t ies and size o f each feature are as fo l lows: 50 N a m e Prior i ty S ize (in function points) Game Cl ien t 5 5 Onl ine Chat 5 3 Scor ing System 3 3 Opt ions M e n u 3 3 Mu l t ip layer Server 3 1 Table 6. Feature List for Project A This data set serves as an example o f a relatively large-scale project wi th longer development cycles. Teams that work in a relatively we l l -known domain or on projects that are heavi ly contract-based, may opt for this development model . The process diagram in Figure 18 illustrates the process f low o f the project. The percentage below each activi ty name is the estimated amount o f t ime required to perform the activi ty in relation to the overal l estimated t ime to execute the process. The letters " F S " on the arrow denotes a finish-start relationship (the activity wh ich the arrow is point ing to cannot start executing before the or ig inat ing activi ty is completed). The letters " S S " on the arrow denotes a start-start relationship (the act ivi ty to wh ich the arrow is point ing to can start when the originat ing task has started). 51 Encrt Stakeholder Request 10% FS Architectural Analysis 8% FS Class Design 13% FS Implement Components 13% SS Perform Unit Test 8% Find Actors and Use Cases 8% FS 1 Use Case Analysis 10% FS FS 1 Design Tests 7% FS -FS- Execute Test 10% -FS- Fix a Defect 13% Figure 18 - Process Flow Diagram of Project A The process for project B consists of four sequential activities. There are ten features. The names, priorities and sizes of each feature are as follows: 52 N a m e Prior i ty Size (in function points) Col laborat ive Ed i t i ng 7 3 Jo in Session 5 1 Create Document 5 2 Save Document 5 2 Instant Chat 3 3 Ca l lback Funct ions 3 2 L o g i n 3 1 A d d M e m b e r 3 2 No t i f y Change 3 2 Server G U I 1 3 Table 7. Feature List for Project B This data set represents projects that have short development cyc les and a large number o f features. Examples o f such a project include performing service pack updates after feedback is gathered f rom the f ie ld , or responding to a series o f change requests in a web site. The process f low diagram be low illustrates the process f low o f project B . Elicit Implement Requirements FS > Components FS > 20% 30% Execute Test 25% - F S — H Fix a Defect 25% Figure 19 - Process Flow Diagram of Project B Project C ' s process is also consisted o f four activit ies, but they are arranged in a way that two activit ies can be executed concurrently. The names, priorit ies and sizes o f each feature are as fo l lows: 53 Name Priority Size (in function points) Hotel Reservation 5 3 Flight Reservation 5 3 Bus Reservation 3 3 Train Reservation 3 3 Ship Reservation 3 3 Table 8. Feature List for Project B Project C is an example of projects where the team has to constantly revise different artifacts at the same time because it is unfamiliar with the domain. The process flow diagram below illustrates the process flow of project C. Gather Requirements FS Implement Components -SS-Test and Fix Bugs Figure 20 - Process Flow Diagram of Project C 6.1.2 Team Member Data NextMove uses the team member data to determine the task recommendation ratings on the suitability category (see section 4.2). The team member simulation data rates each simulated team member in three skill categories: Java programming skills, documentation and testing. The 54 ratings are expressed in a relative form so that they can be used as the inputs of an analytic hierarchy process. Team member data set x is made up of three simulated team members: Joe, Bob and David. Their skills are as described by the scenario in section 0: Joe is modeled as a comprehensive expert who is better than the other two team members in all skill areas. David and Bob have strengths in Java programming and testing, respectively. Team member data set y is consisted of the members from data set x and two additional simulated team members: Mary and Jane. Both Mary and Jane have balanced competencies but are less skilled than Joe. 6.2 Simulation Model The simulator intends to imitate a real-world project environment by simulating the behaviour of team members on a project team. At the initialization of the simulation, a number of tasks, as well as team member information, are generated. Task information is generated from a process model and preloaded features information. After the simulation starts, it runs in a round-robin, iterative fashion, and terminates when all tasks in the project are labelled as "done." Figure 21 illustrates the general execution flow of each simulated team member, and Figure 22 shows the details of the "find and work on task" activity. 55 1 Initialize Data Increment Project Time not all tasks are done all tasks are done Find and work on task Figure 21 - General Execution Flow of Each Simulated Team Member Task queue is empty Task queue not empty Determine which task to work on Work on task Task is completed- -Task is not completed Notify task completion Task queue is not full Find a task to work on Task queue is full Figure 22 - Detailed Execution Flow within the "Find and Work on Task" Activity Each simulated team member had up to three tasks in their personal backlog. When a simulated team member needs to take on a task, it would pick a task and adopt it from the team backlog. The time it takes for a team member to complete a particular task is calculated by combining the estimated time of the task and the competencies (as derived from the team member information) of the team member. During each time period, when a team member works on a task, the time remaining for the task is decremented. When a task's time remaining is zero and the task's workflow conditions (e.g., task B cannot finish before task A is finished in a finish-finish relationship) are satisfied, the team member indicates that the task is completed. Three sets of project data and two sets of team member data were created for simulation. In combination, the project data and the team member data create six scenarios. For each scenario, simulations are executed under two conditions: randomized condition and heuristics condition. The two conditions share the same basic simulation model and only differ in the way the tasks lists are generated when a simulated team member picks a task. Under the randomized condition, the system would rank readily executable tasks at the top and the ones that are not immediately executable at the bottom. The ranking of the tasks within the two groups are randomized. The reason for the partial randomization of the task list (as opposed to total randomization in which both the readily executable and non-executable tasks are mixed together) is to avoid deadlocking situations in which all simulated team members' personal backlogs are full but none of the tasks in those backlogs can be executed. One thousand simulations were executed for each scenario under the randomized condition in order to create baseline results, against which the results of the heuristic condition are compared. 57 Under the heuristics condition, the tasks list is generated according to the heuristics-based approach as outlined in Chapter 4. A simulated team member would pick the most recommended task (at a point in time) by the system. 6.3 Data Analysis At the end of each iteration of simulations, three measures wi l l be taken as metrics on the efficiency of the team: • project completion time (TPCT): the total time it takes for the project to complete. This is used as a measure of the team's overall efficiency. • mean feature completion time (MFCT) : the average time it takes for a feature to be completed in one scenario. This serves as an indicator of whether most features are completed early in the project or completed late in the project. • weighted-mean feature completion time (WFCT): before taking the mean of the feature completion times in one scenario, feature completion times are weighted according to the priority of the feature, with the higher priority features weighted more heavily. This measure indicates whether the team completes the more important features first. Results from simulation under the heuristics condition were compared with a corresponding set of results under the randomized condition to see how they rank amongst the randomized results. 58 6.4 Results The scenarios are named according to the project data and team member data which the scenarios are derived from. For example, Scenario Ax refers to a scenario with project data set A and team member data set x. 6.4.1 Scenario Ax The bar chart below summarizes the results of the simulation for scenario Ax. The bar with a dotted pattern shows the result from the heuristics-based condition in comparison to the randomized results. Each solid bar represents one data point amongst the one thousand iterations of randomized simulation. For example, the 80th percentile project completion time is shorter than 80% of all project completion times from randomized simulations. The shortest case refers to the shortest time among the set of randomized simulations. Scenario Ax - Project Completion Time Median 80th percentile 85th percentile Heuristics Based 90th percentile 95th percentile Shortest Case 40 42 44 46 48 50 52 Simulated Time (hours) 54 56 Figure 23 - Comparisons of Project Completion Times in Scenario Ax 59 The PCT under the heuristics-based condition, highlighted in yellow, lies somewhere in between the 85th and the 90th percentile. The longest PCT among the randomized results (not shown on the chart above), is 69 simulated hours. Scenario Ax - Mean Feature Completion Time Median SOth percentile 85th percentile 90th percentile 95th percentile Shortest Case Heuristics Based 45\2 30 32 34 36 38 40 42 Simulated Time (hours) 44 46 Figure 24 - Comparisons of Mean Feature Completion Times in Scenario Ax The heuristics-based condition produced the shortest M F C T among the samples, indicating that the simulated team took less time to produce deliverables (i.e., completed features) under the heuristics-based condition. 60 Scenar io Ax - W e i g h t e d - M e a n Feature Comp le t i on Time Median 80th percentile 85th percentile 90th percentile 95th percentile Shortest Case Heuristics Based 30 32 34 36 38 40 42 Simulated Time (hours) 44 46 48 Figure 25 - Comparisons of Weighted-Mean Feature Completion Times in Scenario Ax The W F C T under the heuristics-based condition is lower than the short W F C T under the randomized condition by more than 10%. Since the feature priorities are weighted in the calculation, a lower W F C T indicates that the higher-prioritized features are completed before features of lower priorities. 6.4.2 Scenario Ay The setting for scenario A y is similar to scenario A x , with the exception of two additional simulated team members. The PCT under the heuristics condition is slightly shorter than the 95th percentile time. This may indicate that the heuristics become more effective as the number of team members increases. The M F C T and W F C T results show a similar trend as A x . 61 Scenario Ay - Project Completion Time Median 80th percentile 85th percentile 90th percentile 95th percentile Heuristics Based Shortest Case 30 32 34 36 38 40 Simulated Time (hours) 42 44 Figure 26 - Comparisons of Project Completion Times in Scenario Ay Scena r i o A y - M e a n Fea tu re C o m p l e t i o n T i m e Median 80th percentile 85th percentile 90th percentile 95th percentile Shortest Case Heuristics Based 25 27 29 31 Simulated Time (hours) 33 35 Figure 27 - Comparisons of Mean Feature Completion Times in Scenario Ay 62 Scenar io Ay - W e i g h t e d - M e a n Feature Comple t ion Time Median 80th percentile 85th percentile 90th percentile 95th percentile Shortest Case Heuristics Based 20 22 24 26 28 30 32 34 36 Simulated Time (hours) Figure 28 - Comparisons of Weighted-Mean Feature Completion Times in Scenario Ay 6.4.3 Scenario B x In scenario Bx , the simulated team took slightly longer than the 80th percentile time under the heuristics condition. However, the heuristics condition produced the shortest M F C T and WFCT. The significantly lower (by 20%) W F C T under the heuristics condition shows that although the PCT may be longer under the heuristics conditions, the simulated team was able to complete the more important features and deliver them at an earlier time. 63 Scenario Bx - Project Completion Time Median Heuristics Based 80th Percentile 85th Percentile 90th Percentile 95th Percentile Shortest Case 20 30 40 50 Simulated Time (hours) 60 70 Figure 29 - Comparisons of Project Completion Times in Scenario Bx Scena r i o Bx - M e a n Fea tu re C o m p l e t i o n T i m e Median 80th Percentile 85th Percentile 90th Percentile 95th Percentile Shortest Case Heuristics Based 30 32 34 36 38 40 Simulated Time (hours) 42 43.1 44 Figure 30 - Comparisons of Mean Feature Completion Times in Scenario Bx Scenar io Bx - W e i g h t e d - M e a n Feature Comple t ion Time Median 80th Percentile 85th Percentile 90th Percentile 95th Percentile Shortest Case Heuristics Based 25 30 35 Simulated Time (hours) 40 45 Figure 31 - Comparisons of Weighted-Mean Feature Completion Times in Scenario Bx 6.4.4 Scenario By The PCT under the heuristics condition is shorter than the 95th percentile time but is 3.5 hours longer than the shortest case, and continues the trend that heuristics-based results are ranked higher with more simulated team members. The increased number of team members may have helped narrow the gap between the W F C T from the randomized results versus the W F C T under the heuristics-based condition. 65 Scenario By - Project Completion Time Median 80th Percentile 85th Percentile 90th Percentile 95th Percentile Heuristics Based Shortest Case 20 25 30 35 Simulated Time (hours) 40 43.5 45 Figure 32 - Comparisons of Project Completion Times in Scenario By Scena r i o By - M e a n Fea tu re C o m p l e t i o n T i m e Median 80th Percentile 85th Percentile 90th Percentile 95th Percentile Heuristics Based Shortest Case I » -\ 4 -24 26 28 Simulated Time (hours) 32 Figure 33 - Comparisons of Mean Feature Completion Times in Scenario By 66 Scenar io By - W e i g h t e d - M e a n Feature Comple t ion T ime 20 22 24 26 28 30 32 Simulated Time (hours) Figure 34 - Comparisons of Weighted-Mean Feature Completion Times in Scenario By 6.4.5 Scenario Cx Simulation results from scenario Cx produced some interesting results, as the distribution of PCT is heavily slanted towards the shortest case time, 38.5 hours. This is probably due to the high number of tasks that can be executed in parallel. The PCT under the heuristics condition is the same as the shortest case. The M F C T and W F C T results show that the heuristics are still useful, helping the team to complete important features first. 67 Scenario Cx - Project Completion Time T 35 36 37 38 39 40 Simulated Time (hours) Figure 35 - Comparisons of Project Completion Times in Scenario Cx Figure 36 - Comparisons of Mean Feature Completion Times in Scenario Cx 6 8 Scenar io Cx - W e i g h t e d - M e a n Feature Comple t ion T ime Median 80th Percentile 85th Percentile 90th Percentile 95th Percentile Shortest Case Heuristics Based 20 22 24 26 28 30 Simulated Time (hours) 32 34 Figure 37 - Comparisons of Weighted-Mean Feature Completion Times in Scenario Cx 6.4.6 Scenario C y The results from scenario Cy are somewhat the opposite of the trends that were shown in scenarios with project A and B. The PCT from the heuristics-based condition in Cy does not rank as high as the P C T in C x under the heuristics-based condition. In addition, the M C F T under the heuristics-based conditions was longer than the 90th percentile, unlike results from all other scenarios. 69 Scenario Cy - Project Completion Time Median 80th Percentile Heuristics Based 95th Percentile 90th Percentile 85th Percentile Shortest Case 20 21 22 23 24 25 Simulated Time (hours) 26 27 Figure 38 - Comparisons of Project Completion Times in Scenario Cy Figure 39 - Comparisons of Mean Feature Completion Times in Scenario Cy 7 0 Scenar io Cy - W e i g h t e d - M e a n Feature Comple t ion Time Median 80th Percentile 85th Percentile 90th Percentile 95 th Percentile Heuristics Based Shortest Case 15 16 17 18 19 20 Simulated Time (hours) 21 22 Figure 40 - Comparisons of Weighted-Mean Feature Completion Times in Scenario Cy 6.5 Discussion The overall result of the simulations is positive, which indicates that the heuristics process has the potential to be a useful tool for making coordination decision. However, there are some limitations to the simulation model which may affect the results of the simulations. This section discusses some observation taken from the results, as well as limitations to the current simulation model. 6.5.1 Observation from the Results The heuristics-based conditions consistently post times equal or shorter than the 85th percentile of all randomized results in all categories (with the exception of the P C T in scenario Bx). The simulation results also show the flexibility of NextMove's heuristics, as the heuristics can be 71 fine-tuned to adapt to different project situations. The heuristics conditions also required less switching between features for the simulated team members. Figure 41 and Figure 42 show the timeline of each team member during the course of the simulated project. Colours under the timeline represent different features, and each coloured block indicates that the simulated team member worked on the corresponding feature during that period.. Figure 41 is the timeline of project A executed under the randomized condition, and the timeline in Figure 42 is the result under the heuristics-based condition of the same project. The two figures show that the simulated team members under the randomized condition often had to switch between features, whilst there were less switching between features under the heuristics-based condition. Team Member 1 Time Une 1 fflTiTfllflTj; I fllflBBBM! nffTJuTl BITJI1! IBHlBBfflSfflfflBHIBfflnlufl Joe P T T iTT' TfT" I IBI1BIBI i Figure 41 - Execution Timeline for Project A Under the Randomized Condition Team Member 1 Time Line 1 David . l u l l i i n i l P H I IBB' mmm BB mmm Joe i |5 II t i 11 l l l . j i . l t il...< L 1 I Ml T l f T J f T i r UM 11 II11II I i i i i i i i . . . . . . „. B i l l ! Mm Figure 42 - Execution Timeline for Project A Under the Heuristics-based Condition 6.5.2 Simulation Model Limitations The simulation model represents a simplistic project model, where there is no exceptional case arising during the project's execution and there are no inherent logistical problems. A number of improvements can be added to the simulator to better reflect the realities of distributed software development. 72 Logistical issues such as time-zone differences can be built into the simulation model to better reflect the realities of distributed development, by placing restrictions on the work hours of each virtual team members. During the course of a software development project, emergent events, such as changes in the feature list or changes in personnel, are fairly common. These events should be modeled in the simulation in order to test the NextMove framework's exception-handling capabilities. The simulation assumes that the team members have perfect communications (e.g. Bob is informed instantaneously when Joe has completed a task). This is an unrealistic assumption because communication overhead wi l l always incur on a team with two members or more. Time elapsed due to communication overhead should be added to the refined model. It is probably also unrealistic to assume that team members would always take the most highly recommended task from the team backlog. Hence, virtual team members should select tasks based on their preferences. The simulation's accuracy can be further improved by including the effect of preferences on productivity (e.g., a team member may be less efficient when he or she is working on an undesirable task). 73 CHAPTER 7 EXPERIMENT In addition to simulations, experiments with human subjects were conducted to further study the effectiveness of NextMove as a project coordination tool. While simulations have the advantage of being able to run repeatedly with relatively low cost in time and effort, they cannot evaluate the effects of user interaction on NextMove. As subjects have different perceptions of the tool as they start using it, their mode of usage may not be anticipated by the simulator nor the author of the thesis. 7.1 Experimental Planning 7.1.1 Goals The following measures were recorded during the experiment and analyzed afterwards: total project completion time (TPCT), mean feature completion time (MFCT) , weighted-mean feature completion time (WFCT) . The three measures listed above correspond to the metrics which were used in the simulation. Please see the Analysis Procedure section for details. 7.1.2 Experimental Units The population of subjects was drawn from the graduate student population, and the subjects had some experience in working in software projects. In each experiment session, three subjects formed a group that represented a distributed project team. The grouping of the subjects in each session was subjected to the schedules and availability of the subjects. A l l three treatments 74 (please see below) were assigned to each group. Subjects who had completed a session were asked not to inform other subjects on the results of their session. There were three sessions in total. 7.1.3 E x p e r i m e n t a l M a t e r i a l The objects of the experiments were three projects that make use of different development processes. They were the same three projects which were used in the simulation. However, in order for project A and B to be finished within a reasonable time for the subjects, the number of features in project A was reduced to two and the number of features of project B was reduced to five. Three types of treatments were assigned to the subjects. The treatments were labeled as: baseline, partial and full. Under the baseline treatment, the group had access only to instant messaging software as the communication tool while coordinating their work. Under the partial treatment, the group had access to NextMove as an aid, but the heuristics process on task recommendation was disabled. Instead, the recommended task lists were randomized lists of tasks. The group also had access to the same communication tool as in the baseline treatment. Under the full treatment, the group had access to the full-featured NextMove tool, as well as the communication tool in the baseline treatment. The order of assignment of treatments and objects for each session was as follows: 75 Session 1 Session 2 Session 3 Order Treatment Project Treatment Project Treatment Project 1 baseline A partial A full A 2 partial B full B baseline B 3 full C baseline C partial C Table 9. Order of Assignment of Treatments The rotation of the treatments in different sessions was to mitigate the learning effects which may occur over the course of a session (e.g. the team members were more efficient in the third project than in the first project not because of difference in treatments, but because they got used to working with the experiment interface). 7.1.4 Tasks In each session, the subjects worked in groups of three as a project team. Each subject took on the role of a team member, namely Bob, Joe and David. The subjects are informed of the pre-assigned attributes (in Java programming, documentation and testing) for each role. The subjects sat in different parts of a computer laboratory in order to simulate a distributed environment. The participants worked on the three projects under different treatments, as specified by the above table. During the experiment, a participant emulated their "work" on a task by selecting it in the Work View (see Figure 43 below) and clicking on the "work" button. If a task was dependent on another task, the participant was required to enter the keyword that was supplied from the other 76 task. The keywords needed for executing certain tasks were randomly generated before the start of each scenario. Tl r u ' i ' y ' ' II i fy Team Member, Name Joe| ' * , Task 1 aequ:red Keyword: Progress: i J l i i l N i status: I'HiJJi'IjT' •JffT •fpwwj-task completed1 -. 1 II k ' ' 1' "IT!" " H i f, tl' »'l.kt,*«!kirw,.^  .. A thl m fi,„ 'II In, , i > i u , L U U , 1 I f f ! ' ! W '.'lliiit'", M««S| iMZm. III Jiff' ' ,1 U — T V T TTfT Figure 43 - The Work Simulator Interface After the work button was clicked by the subject, a timer would start and count down until time ran out or the participant interrupted. The timer length was dependent on the size of the feature, as well as the attributes assigned to the team member (i.e. since David was more competent than Bob in programming, the programming task timer length for David would be shorter than Bob's). The participant could interrupt the timer by clicking on either the "stop" or "cancel" button. When the "stop" button was pressed, the timer would stop and the progress on the task would be saved on the system. When the "cancel" button was pressed, the work on the task would not be recorded on the system. The "cancel" button was implemented in case the subject had selected a task which he or she did not mean to choose. 7.1.5 Hypothesis Prior to the start of the experiment, the following hypotheses were made: 77 1. teams are most effective when they make use of the fully-featured NextMove tool 2. teams are more effective with the help of NextMove tool with the heuristics process turned off than when they have no assistance 3. teams benefit the most from NextMove in more complex (i.e. higher degree of interdependency) projects 7.1.6 Procedure At the beginning of each session, the subjects were given a synopsis of the experimental procedures and a tutorial on the experiment tool. Questions regarding the tool and the experimental procedures were answered. In addition, 10-minute tutorials on the functionalities of NextMove were given to the subjects before they started using the tool. As aids, all subjects were given the following guides for each project: the process flowchart, the feature list, and the relative competencies of each team member. At the beginning of the experiment, all project data was reset to its initial state. Then the subjects started working on the first project with the first type of treatment and performing the tasks as outlined in the previous section. 7.1.7 Analysis Procedure The analysis procedure after the experiments was similar to the corresponding procedure for the simulation. From each project in the experiment, three measures were taken as metrics on the efficiency of the team: • total project completion time (TPCT): the time between the first task being started and the completion of the last task 78 • mean feature completion time (MFCT) : the average time it takes for a feature to be completed • weighted-mean feature completion time (WFCT): similar to the M F C T , but with the average weighted according to the priorities of the features 7.2 Results The results of the experiment are presented on a project-by-project basis. For each project, the TPCT, M F C T and W F C T of the three treatments are compared through horizontal bar charts. Each column of a chart is labeled as "treatment name, sy," where y indicates the session (1, 2, or 3) that the data point is taken from. 7.2.1 Project A The results from project A show that the project that ran under the full treatment (with the fully-featured NextMove tool) took the shortest time to complete. It also posted the shortest M F C T and W F C T . However, when the heuristics processes were disabled (in the partial treatment), the project took longer to complete than when the subject did not make use of NextMove at all (in the baseline treatment). The M F C T and W F C T for the partial treatment also significantly lagged behind (by about 10 minutes) the other two treatments, indicating that most features were not completed until the later stage of the project. 79 Project A - Total Project Completion Time 0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00 Actual Time (minutes) Figure 44 - Comparison of Total Project Completion Time in Project A Project A - Mean Feature Completion Time 0.00 5.00 10.00 15.00 20.00 25.00 30.00 Actual Time (minutes) Figure 45 - Comparison of Mean Feature Completion Time in Project A 80 Project A - Weigh ted-Mean Feature Comple t ion T ime 0.00 5.00 10.00 15.00 20.00 25.00 30.00 Actual Time (minutes) Figure 46 - Comparison of Weighted-Mean Feature Completion Time in Project A 7.2.2 Project B The results from project B are a complete reversal from the corresponding results from project A , as the TPCT, M F C T and W F C T from the partial treatment are the lowest among the three treatments. The project instance under the full treatment returned the worst results (all of TPCT, M F C T and W F C T from the full treatment are about twice as long as the corresponding times from the partial treatment). It is also worth noting that the group from session 2 posted the longest TPCT, M F C T and W F C T in both project A and B . 81 Project B - Total Project Completion Time full, s2 baseline, s3 partial, s1 14.97 0.00 2.00 4.00 6.00 8.00 10.00 12.00 14.00 16.00 Actual Time (minutes) Figure 47 - Comparison of Total Project Completion Time in Project B Project B - Mean Feature Completion Time full, s2 baseline, s3 partial, s1 0.00 2.00 4.00 6.00 Actual Time (minutes) 8.00 10.00 12.00 Figure 48 - Comparison of Mean Feature Completion Time in Project B Pro jec t B - W e i g h t e d - M e a n Fea tu re C o m p l e t i o n T i m e 0.00 2.00 4.00 6.00 8.00 10.00 12.00 Actual Time (minutes) Figure 49 - Comparison of Weighted-Mean Feature Completion Time in Project B 7.2.3 Project C The group that did not have access to NextMove (the baseline treatment) managed to return the best results in project C. The group under the partial treatment posted slightly slower M F C T and W F C T , while the full treatment has the worst time in all three categories. 83 Project C - Total Project Completion Time full. s1 partial. s3 baseline, s2 0.00 2.00 4.00 6.00 8.00 Actual Time (minutes) 10.00 12.00 Figure 50 - Comparison of Total Project Completion Time in Project C Project C - Mean Feature Completion Time full, s1 partial, s3 baseline, s2 0.00 5.34 2.00 4.00 6.00 Actual Time (minutes) 9.23 8.00 10.00 Figure 51 - Comparison of Mean Feature Completion Time in Project C Pro jec t C - W e i g h t e d - M e a n Fea tu re C o m p l e t i o n T i m e 0.00 2.00 4.00 6.00 8.00 10.00 Actual Time (minutes) Figure 52 - Comparison of Weighted-Mean Feature Completion Time in Project C 7.3 Discussion The results from the experiment are inconclusive at best, with all three treatments posting the best results in three different projects. The results fail to show that NextMove, with or without heuristics enabled, is capable of improving the efficiency of a distributed project team. This section summarizes the experimental results and offers some possible explanations. 7.3.1 Summary of Results Since the orders of results in all categories for each of the three projects are the same, the results of the experiments can be summarized using the following table: Project A Project B Project C Shortest Time full partial baseline Medium Time baseline baseline partial Longest Time partial full full Table 10. Summary of Experimental Results 85 Only the results in project A fu l f i l led hypothesis (1) (with the fu l l treatment post ing the best time) and only the results in project B fu l f i l led hypothesis (2) (as the partial treatment posted better t ime than the baseline treatment, but the fu l l treatment posted the longest t imes). The ful l treatment posted the shortest t ime in project A , wh ich has relat ively complex interdependencies between tasks compared to project B and project C . Therefore, hypothesis (3) holds true in the context o f the three projects invo lved. 7.3.2 Ineffectiveness of NextMove Whi le s imulat ion results show that N e x t M o v e ' s heuristics does improve a distributed team's ef f ic iency, differences between the simulat ion and experimental condit ions may have caused the discrepancies in results. A s the task recommendat ion list is set up to rank readi ly executed tasks on the top o f the list, it is possible that subjects who took on the role o f the most competent team member wou ld adopt tasks f rom a lower-pr ior i ty feature, i f the tasks f rom the higher-prior i ty features were al l adopted. W h e n tasks f rom the higher-prior i ty features become avai lable, they may have been adopted by roles that were less competent (hence the tasks wou ld take longer to complete). In instances where there are more features than team members (such as project B and C ) , it may be possible that the subjects choose to div ide up the features and work on them independently, hence e l iminat ing the communicat ion overhead. A n example o f this situation can be seen in the fo l low ing Figure 53, w h i c h is the Gantt chart o f the execut ion o f project C under the baseline treatment. Each hor izontal bar depicts a task, and the feature to wh ich the task belongs is labeled on the y-ax is . Each pattern represents a team member, w i th sol id pattern associated wi th B o b , 86 vertical stripes associated with Joe, and brick pattern associated with David. The chart shows that each team member focused on one or two feature. Bob concentrated on Bus Reservation; Joe focused on Flight Reservation and Ship Reservation and David concentrated on Hotel Reservation and Train Reservation. Project C (Baseline Treatment) 0.00 1.00 2.00 3.00 4.00 5.00 Actual Time (Minutes) 6.00 7.00 S.OO Figure 53 - Gantt Chart for Project C (Baseline Treatment) Such situations may not be realistic in a real-world software project because of the inter-dependencies between features (which the experiment did not investigate). The example from above may also help explain the reason for the full treatment getting the best results in project A where there were only two features but the tasks are very much interrelated. One of the advantages of NextMove's heuristics is that it is versatile in the way that it can be adjusted to accommodate different goals (e.g. minimize project completion time, or optimize the 87 project for a quicker first release) and situations. However, when the experiments were executed, the same heuristics were used for all three projects, in order to decrease the number of variables in the experiment. Project B and C under the full treatment may return better results i f project-specific calibrated heuristics are used. 7.3.3 P e o p l e F a c t o r As with any real-life software projects, group dynamics and individual behaviours play a big part in determining the outcome of the experiment. Understanding the effects of the people factor wil l help the design and execution of possible further studies. Although the subjects were given a tutorial of NextMove, which introduces various components of the user interface and what they can do with them, the subjects were not explicitly told what to do with the tool. Hence, some subject may have used the tool in manners which were unexpected. For example, in the project depicted in the Gantt chart of Figure 54, subjects who played the role of David and Bob (brick pattern and solid pattern) took on a large number of tasks early on. A s a result, Joe (who was the most competent of the three team members) was deprived of tasks and did not do much work. Therefore, time was wasted during the early stages of the project. 88 Project A (Partial Treatment) Mu tiplayer Server Mu tiplayer Server Mu tiplayer Server Mu tiplayer Server Mu tiplayer Server Mu tiplayer Server Mu tiplayer Server Mu tiplayer Server Mu tiplayer Server Mu tiplayer Server Mu tiplayer Server t 3 Mu tiplayer Server n £ Game Client Game Client Game Client Game Client Game Client Game Client Game Client Game Client Game Client Game Client Game Client Game Client 1 15.00 Actual Time (Minutes) Figure 54 - Gantt Chart for Project A (Partial Treatment) During the experiment, the manual override options (please see chapter 5) were not made available to any of the subject. Such mechanisms could have helped the team resolve the aforementioned priority inversion and task-hogging situations. One of the most important lessons learned from the experiments is that while a tool can be fully featured and designed to be as user-friendly as possible, it is still up to the team members to collaborate as a team in order for a project to be successful. 7.4 Validity When evaluating the validity of the aforementioned experiments, there are some issues to be considered: 89 sample size: due to time and resource constrains, only nine subjects have participated in the study. The small sample size means that any outliers in the data wi l l have a significant impact on the overall results. For example, i f the resulting times from the partial treatment and full treatment in session 2 are halved, the overall results and the conclusion drawn from the results wi l l be completely different learning effect: the efficiency of the subjects may be affected by the increasing familiarity with the experimental tool. The design of the studies attempts to mitigate this effect by rotating the order of treatments. However, the order of the projects were not rotated; hence it is not known whether the results wi l l be different i f the subjects execute the projects in different orders skill differences: apart from the efficiency of coordination, metrics such as total project completion times are affected by the technical skill levels (e.g., proficiency in Java programming) of the team members involved. Hence, the experimental tool was designed such that the tasks are not dependent on the subject's technical skills. However, peripheral skills such as familiarity with instant messaging application (which was used in the experiment) still affect an individual's and the team's efficiency. It is difficult to account for such subtlety with only empirical data generalizability: whenever an experiment is designed to emulate a real-life environment, there is always the question of whether the results wi l l be readily applicable to the real world. Although the design of the aforementioned experiment attempts to model a distributed environment as closely as possible, there are some shortcomings. Users develop usage patterns over long periods of time and it can be argued that the effectiveness (or the lack thereof) of NextMove cannot be seen until the users have 9 0 become fully accustomed to using it. In addition, as the tasks in the experiment are simply button pressing and waiting for the timer to expire, the effects of task preferences (as mentioned in section 2.1.5) were not present 91 CHAPTER 8 CONCLUSIONS AND FUTURE WORK The work of this thesis focused on the problem of task coordination in a distributed agile project. Its main body of work consists of a heuristics-based process to help answer the questions of "what to do next" and "who should do it" in a distributed software development project. Based on the heuristics-based process, NextMove was built as a project coordination tool. The validation process includes both automated simulations and experimentation with human subjects. The results provide some evidence that the heuristics-based process is effective in a bounded environment (i.e., the simulation), but NextMove's effectiveness as a project management tool in a real-world, less controlled environment remains to be further investigated. This chapter recaps the contributions made by this thesis to the field of software project management and outlines possible future work and studies. 8.1 Research Goals Summary The goal of this thesis is to improve the efficiency of a distributed agile team. In a distributed environment, the project manager often becomes solely responsible for collecting all relevant information (e.g. current progress of each team member, issues that team members encounter during their work) and allocating tasks to team members. This is a particularly prevalent problem in a distributed environment because there is a significant amount of informal communication between the team members. The work of this thesis aims to relieve the project manager's burden by providing recommendations to individual team members on which task to work on next. The 92 recommendations are based on a heuristics-based process. The success of NextMove was measured by the following metrics: • project completion time (PCT or TPCT) : the time between the first task being started and the completion of the last task • mean feature completion time (MFCT) : the average time it takes for a feature to be completed • weighted-mean feature completion time (WFCT): similar to the M F C T , but the average is weighted according to the priorities of the features 8.2 Contributions of This Work The work of this thesis made the following contributions: • reviewed the current state of project management tools and identified task allocation as a weak area of support • developed a heuristics-based process for finding important tasks and allocating them to the best candidate. The process is based on a number of concepts that are commonly used by project managers, such as critical path analysis and feature-driven development. Hence, the reasoning behind the analysis can be easily understood without going through a steep learning curve • implemented NextMove, a project management tool which is based on the aforementioned heuristics-based process. In addition, NextMove's user interface is designed to help team members gain awareness of their activities and facilitate communications between them • validated the effectiveness of the heuristic process in a bounded and controlled simulation environment. The results indicate that the heuristics process outperforms the 93 80th percentile out of a set of partially randomized execution sequences in all but one of the eighteen metrics collected • conducted a series of experiments to test the effectiveness of NextMove as a tool with human subjects. However, the results are not consistent with the simulation results. No treatments were able to show a clear advantage in all of the project scenarios. A number of possible reasons for these results are discussed in chapter 7 • parts of the work of this thesis have been published academically, [ l j is a synopsis of the heuristics process; while [2] focuses on the user interface of NextMove, and [3] presents overviews of both the heuristic process and NextMove, as well as preliminary simulation results 8.3 Future Work During the course of the work on this thesis, three areas of future work and studies have been identified. They are: i) improvements of the simulation model to reflect more complicated situations, ii) refinements of the experimental methods in order to produce more valid results and iii) long term improvements to NextMove in order to improve its effectiveness. 8.3.1 Simulation Model Improvements Section 6.5.2 outlined the limitations of the current simulation model. In light of these limitations, this section proposes a number of improvements to the simulation model. A limit on the periods of time that the simulated team member can work can be defined, and periods of work time for each team members can be shifted to reflect real-world time-zone differences. For example, i f the simulated project timeline is divided into 24-hour cycles, and 94 each simulated team member works in an eight-hour per iod in each cyc le , the work period o f each team member can be assigned to different parts o f the cyc le , as il lustrated by Figure 55: Figure 5 5 - Example of Time Zone Assignments The simulated team members w i l l only be taking on new tasks and work on tasks in their own back log in the shaded part o f the t imel ine. Hence, each team member is work ing in three different t ime zones. The work periods can also be arranged in different ways to reflect the different locations o f team members and thus the effects o f different degrees o f t ime differences can be investigated. The simulator can be modi f ied to a l low events to be inserted at a speci f ic t ime. For example, a new, high-pr ior i ty feature may be added after the project has been executed for fifteen hours. The result ing response f rom the simulated team wi th and without the heuristics-based process can be compared in order to evaluate how wel l the heurist ics-based process adapts to emergent situations. In addit ion to the possible communicat ion delays introduced by t ime-zone differences, the number o f team members on a team also affects the communicat ion ef f ic iency o f a team, especial ly w i th late addit ions o f team members (i.e., the "tar-pit effect" [43]). The number o f required communicat ion channels in a team increases by n-1 (where n is the team size) wi th each 95 additional member. Therefore, the communication delay in the simulation model can be modeled as a polynomial function in relation with the team size. A n "efficiency factor" can be built into the simulation model in order to reflect the effects of team member preferences and inaccuracies in the team member skill evaluation process. The results of different efficiency factors can be compared to determine how accurate the input data has to be in order for the data to be considered useful. 8.3.2 Experimental Methods Refinements • In order to increase the confidence level of the data, a larger pool of subjects should be selected. If the size of the subject pool permits, some subjects can be placed into groups with other subjects with whom they are acquainted, and others placed into groups with subjects with whom they do not have social relationships. This wi l l help investigate how social relationships play a rple in affecting the usage pattern of NextMove (e.g., whether they wi l l depend more or less on the tool i f the subjects know each other). • One of the subjects can be assigned to be the project manager of the team and make use of the manual task allocation features (please see Chapter 5) in NextMove. Such mechanism could have helped the team in resolving the aforementioned priority inversion and task-hogging situations. • The subjects should be given more time to develop familiarity with NextMove, so that less mistakes due to unfamiliarity with the tool (e.g., clicking the unintended button) wi l l be made which may skew the results. 96 • Real projects that span over longer periods should be used instead of a "work simulator." In this way, investigators wi l l be able to observe how usage patterns develop over time and whether opt-out behaviours wi l l occur, in addition to the quantitative data gathered. 8.3.3 L o n g T e r m D e v e l o p m e n t o f N e x t M o v e In addition to improvements to the simulation model and experimental methods, there are a number of other possible extensions to NextMove. These extensions would further augment the framework's adaptability to different situations and increase the quality of its coordination recommendations. One of the major emphases of the design of NextMove is to minimize the amount of work that the users need to do in order to gain value from the tool. One way of achieving this goal is to interface NextMove with other project management tools and platforms. Tools such as version control systems (e.g. C V S , Subversion), bug and issue tracking systems (e.g., Bugzilla) and collaboration platforms such as Jazz contain valuable information which NextMove can make use of in order to generate project and team member information. The task-allocation process can be further augmented to include personalities as factors in order to find the best fit between task requirements and team member capabilities. A paper by Acufia, Juristo and Moreno [44] describes how team member personality information is used to match team members to roles. The current task model assumes that only one team member works on a particular task at one time. However, some agile practices may require some tasks to be performed by more than one 97 team member at the same t ime (e.g., pair programming in eXtreme Programming) . Therefore, N e x t M o v e should be extended to support col laborative tasks. Without accurate informat ion on the characteristics o f tasks and team members, N e x t M o v e ' s heuristics-based process wou ld not be able to produce meaningful or useful results. Therefore, a project data col lect ion methodology should be developed and used in conjunction wi th N e x t M o v e . One o f the possible avenues in this area o f development is the usage o f machine learning techniques to adjust the ratings o f each team member over t ime. A s described in Chapter 5, the current implementation o f N e x t M o v e is based on a client-server model . Wh i l e this approach is straight forward, it may lead to a system that suffers f rom rel iabi l i ty and performance issues as the number o f users increases. Future work may explore alternative architectures, such as peer-to-peer based design, as proposed by Jun and Y u n [45]. N e x t M o v e is not meant to be a catch-al l solution to the task coordinat ion and communicat ion problems in a distributed software development environment. Rather, it a ims to augment a team's abi l i ty to dynamica l ly coordinate tasks in a distributed environment. Therefore, it is best to f ind ways to integrate N e x t M o v e wi th other project management/tracking tools and gather useful information f rom them, rather than reinvent the wheel by attempting to expand to other areas such as messaging and artifact repositories. 8.4 Conclusion This thesis identif ies task coordinat ion in a distributed agile team as an important problem, and describes a heurist ics-based process that helps teams in mak ing decisions on task al location. The 98 heuristics-based process is implemented as NextMove, a tool which aims to increase efficiency in task coordination, as well as project transparency. The heuristics process was tested under simulation and the results show that the process is potentially useful for improving the efficiency of the team. In addition, this thesis describes the experiments that were conducted to study the interaction between human subjects and NextMove. However, the results are not conclusive and further studies wi l l be required to verify NextMove's usefulness in a real-world project environment. The work of this thesis serves as a first step towards a solution for increasing the quality of task coordination decisions in a distributed agile environment. The heuristics-based process balances the need for optimizing coordination from a global perspective and the need for team members to know and have control over their next moves in the project. Interactions between individuals and effective responses to changes are two of the main emphasis the Agile Manifesto [11], as well as two challenges faced by a distributed team. The work of this thesis helps teams answer these challenges. 99 REFERENCES 1] D. K . M . Mak and P. B . Kruchten, "Task Coordination in an Agile Distributed Software Development Environment," in 2006 IEEE Canadian Conference on Electrical and Computer Engineering - CCECE2006. Ottawa, Canada: IEEE, 2006. 2] D. K . M . Mak and P. B . Kruchten, "NextMove: A Distributed Project Management Tool," in The IASTED International Conference on Software Engineering. Innsbruck, Austria: A C T A Press, 2007. 3] D. K . M . Mak and P. B . Kruchten, "NextMove: A Framework for Distributed Task Coordination," in 18th Australian Software Engineering Conference - ASWEC 2007. Melbourne, Australia, 2007. 4] K . Schwaber and M . Beedle, Agile Software Development with SCRUM. Upper Saddle River, N J : Prentice-Hall, 2002. 5] S. Palmer and J. Felsing, A Practical Guide to Feature-Driven Development, 1st ed. Upper Saddle River, N J : Prentice Hall , 2002. 6] K . Beck, Extreme Programming Explained: Embrace Change. Boston: Addison-Wesley, 2000. 7] M . Poppendieck and T. Poppendieck, Lean Software Development - An Agile Toolkit. Boston, M A : Addison-Wesley, 2003. 8] A . Cockburn, Crystal Clear. Upper Saddle River: Pearson Education, Inc, 2004. • 9] J. D . Herbsleb and D . Moitra, "Global software development," Software, IEEE, vol. 18, pp. 16,2001. TO] B . Boehm, "Get ready for agile methods, with care," Computer, vol . 35, pp. 64, 2002. T1] M . Fowler and J. Highsmith, "The Agile Manifesto," in Software Development, vol. 9, 2001, pp. 28-35. 12] J. Pang and L . Blair, "Refining Feature Driven Development - A Methodology for Early Aspects," in Workshop on Early Aspects: Aspect-Oriented Requirements Engineering and Architecture Design, AOSD Conference, 2004. 13] J. Highsmith and A . Cockburn, "Agile software development: the business of innovation," Computer, vol . 34, pp. 120, 2001. 14] J. L . Peterson, Petri Net Theory and the Modeling of Systems. Upper Saddle River, NJ , U S A : Prentice Hal l PTR, 1981. T5] P. Kruchten and S. Cook, "The Software Process Engineering Metamodel (SPEM), O M G Specification." Boston, M A : O M G , 2002. 16] P. Kruchten, The Rational Unified Process: An Introduction, 3 ed. Boston: Addison-Wesley, 2003. 17] P. Krol l and P. Kruchten, Rational Unified Process Made Easy: A Practitioner's Guide. Boston: Addison-Wesley, 2003. 18] J. D . Herbsleb and A . Mockus, "An empirical study of speed and communication in globally distributed software development," IEEE Transactions on Software Engineering, vol.29, pp. 481,2003. 100 [19] D . Turk, R. France, and B . Rumpe, "Limitations of Agile Software Processes," presented at Proceedings of the Third International Conference on extreme Programming and Agile Processes in Software Engineering, 2002. [20] J. Hollan and S. Stornetta, "Beyond being there," in SIGCHI Conference on Human Factors in Computing Systems. Monterey, California, United States: A C M Press, 1992. [21] M . Cohn, Agile Estimating and Planning. Upper Saddle River: Prentice Hall , 2005. [22] A . van der Hoek, D. Redmiles, P. Dourish, A . Sarma, R. Silva Filho, and C. de Souza, "Continuous Coordination: A New Paradigm for Collaborative Software Engineering Tools," presented at Workshop on Direction in Software Engineering Environments at International Conference on Software Engineering, Edinburgh, Scotland, 2004. [23] J. Shim, S. Lee, and C. Wu, " A Unified Approach for Software Policy Modeling: Incorporating Implementation into a Modeling Methodology," Lecture Notes in Computer Science, vol . 2813, pp. 118-130, 2003. [24] M . Shen, G . - H . Tzeng, and D.-R. L iu , "Multi-criteria task assignment in workflow management systems," in 36th Annual Hawaii International Conference on System Science. Hawaii, 2003, pp. 9. [25] X . - G . Zhang, J. Cao, and S.-S. Zhang, "Fuzzy Synthesis Evaluation Improved Task Distribution in WfMS," Lecture Notes in Computer Science, pp. 927-934, 2004. [26] J. Duggan, J. Byrne, and G. J. Lyons, " A Task Allocation Optimizer for Software Construction," in IEEE Software, vol. 21, 2004, pp. 76-82. [27] C. K . Chang and M . Christensen, " A Net Practice for Software Project Management," vol. 16, 1999, pp. 80. [28] P. Barthelmess and K . M . Anderson, " A View of Software Development Environments Based on Activity Theory," Comput. Supported Coop. Work, vol . 11, pp. 13-37, 2002. [29] C. Gutwin, K . Schneider, and R. Penner, "Group Awareness in Distributed Software Development," presented at A C M Conference on Computer-Supported Cooperative Work, 2004. [30] P. Dourish and V . Bellotti, "Awareness and coordination in shared workspaces," presented at the 1992 A C M conference on Computer-supported cooperative work, Toronto, Ontario, Canada, 1992. [31] Microsoft Corporation, "Project Home Page - Microsoft Office Online," 2007, Retrieved Sep 4, 2007 from http://office.rn i crosoft.com/en-ca/ pro j ect/default.aspx [32] O. Workbench, "Open Workbench - Open Source Project Management and Project Scheduling for Windows," 2006, Retrieved Sep 4, 2007 from http://www.openworkbench.org/ [33] Rally Software Development Corp, "Rally Software Development," 2007, Retrieved Sep 4, 2007 from http://www.rallydev.com/ [34] XPlanner, "XPlanner," 2006, Retrieved Sep 4, 2007 from http://www.xplanner.org/ [35] K . G . Lockyer, Critical path analysis and other project network techniques, 4th ed. London: Pitman, 1984. [36] T. L . Saaty, The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation. New York: Mcgraw-Hil l , 1980. [37] J. Grudin, "Why C S C W applications fail: problems in the design and evaluation of organizational interfaces," in Proceedings of the 1988 ACM conference on Computer-supported cooperative work. Portland, Oregon, United States: A C M Press, 1988. 101 [38] The Eclipse Foundation, "SWT: The Standard Widget Toolkit," 2007, Retrieved Sep 8, 2007 from http://www.eclipse.org/swt/ [39] M y S Q L A B , " M y S Q L A B : MySQL® Connector/J," 2007, Retrieved Sep 8, 2007 from http://www.mvsql.eom/products/connector/j/ [40] National Institute of Standards and Technology, " J A M A : A Java Matrix Package," 2005, Retrieved Sep 8, 2007 from http://math.nist.gov/iavanumerics/jama/ [41] D . Springgay, "Creating an Eclipse View," 2001, Retrieved July 11, 2007 from http://www.eclipse.org/articles/viewArticle/ViewArticle2.html [42] M . Griffiths, "Summarizing Progress with Parking Lot Diagrams," 2007, Retrieved Sep 8, 2007 from http://leadinganswers.tvpepad.com/leading_answers/2007/02/summarizing_pro.html [43] F. P. Brooks, The Mythical Man-Month: Essays on Software Engineering, 20th Anniversary Edition, 1st ed: Addison-Wesley Professional, 1995. [44] S. T. Acuna, N . Juristo, and A . M . Moreno, "Emphasizing human capabilities in software development," Software, IEEE, vol. 23, pp. 94, 2006. [45] Y . Jun, Y . Yun, and G. K . Raikundalia, "SwinDeW-a p2p-based decentralized workflow management system," IEEE Transactions on Systems, Man and Cybernetics, Part A, vol. 36, pp. 922, 2006. 102 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0101113/manifest

Comment

Related Items