KNOWLEDGE REQUIREMENTS ANALYSIS: A CASE STUDY by PALASH BERA B.Sc. Utkal University, India 1995; PGDBM, EPM, Meerut, India 1997 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE (Business Administration) in THE FACULTY OF GRADUATE STUDIES (Management Information Systems Division, Faculty of Commerce and Business Administration) We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA November 2002 © Palash Bera, 2002 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of Coywy^ t v c e. o\y\ck lru<^ i^ ~*A> c^ci t~\VW I'C, Wj/w The University of British Columbia Vancouver, Canada Date 7^0 ^ , "2- cro DE-6 (2788) Abstract Knowledge is a prime enabler to successfully carry out business processes within an organization. This thesis identifies a methodology to extract knowledge requirements from business processes and applies this methodology to a case study. The methodology used here consists of two methods, event driven process diagramming technique (a process modeling technique) and CommonKADS (a knowledge engineering method). The knowledge requirements analysis is conducted by developing the CommonKADS models- Organizational, Task, Agent, Knowledge and Communication. Furthermore the knowledge model was applied to a specific domain to demonstrate how to construct knowledge models. The main contribution of this thesis is the effective demonstration of combining the two methods event driven process diagramming technique and CommonKADS to extract knowledge requirements from business processes and testing the combined method to a case study. There are two methodological contributions of this thesis. First, operationalization of CommonKADS models with minor modifications to the original CommonKADS literature and second, generation of new methods, procedures and structures to the CommonKADS models as found necessary. A structure of knowledge system was developed as a consequence of conducting the knowledge requirements analysis. i i Table of Contents Abstract ii Table of Contents i i i List of Tables v List of Figures vi Acknowledgements vii 1 Introduction 1 1.1 Research Objectives 1 2 Literature Review 3 2.1 Knowledge 3 Knowledge Management 5 Knowledge Management Processes 8 2.2 Business Processes 12 Definitions 13 Characteristics 14 Modeling Business Processes 15 Knowledge and Business Processes 16 2.3 Knowledge Engineering 17 Knowledge Engineering and Knowledge Management 17 Knowledge Engineering techniques 18 Analysis of Knowledge Engineering methodologies 19 3 The Methods 22 3.1 Case Study method 22 3.2 CommonKADS 23 Overview 23 CommonKADS Model Suite 25 Organization model 28 Task model 30 The knowledge model 30 Constructing the Knowledge model 32 3.3 Process and activity analysis 33 Activities 33 Events 35 Logical symbols 36 Event driven process-diagramming technique & CommonKADS 39 3.4 Knowledge and process analysis 39 Structured Interview method 40 i i i Activity analysis method 40 3.5 Structure of analysis 41 4 Case Study- Office of Research Services 43 Stakeholders 43 Objectives of the ER process 43 Ethical Review process 44 Problems 44 Objectives of the project 45 5 The Analysis 46 5.1 Organization Model 48 Organization Model-1 (OM-1) 48 Organization Model-2 (OM-2) 50 Process Analysis of the Organization 51 Knowledge-intensive activities 58 Organizational Model-3 (OM-3) 59 Organization Model 4 (OM-4) 61 Organization Model -5 (OM-5) 62 5.2 Task Model (TM) 63 5.3 Agent Model (AM) 68 Organization Task Agent Model (OTA-1) 71 5.4 The Knowledge Model 74 Knowledge Model Construction 74 Knowledge Identification phase 75 Knowledge Specification phase 77 5.5 Communication model 82 5.6 Revised models of commonKADS 84 6 Application of the Knowledge model 85 6.1 Methodology 85 6.2 Results 86 7 Conclusion 90 7.1 Contributions 90 7.2 Application of Event driven Process diagramming technique 92 7.3 CommonKADS difficulties and opportunities 93 7.4 Future Research 94 8 References: 96 9 Appendices 98 9.1 Appendix A 99 9.2 Appendix B 107 9.3 Appendix C 112 9.4 Appendix D 121 iv List of Tables Table 1: Structure of Organization Model-1 49 Table 2: Organization Model-1 49 Table 3:Organization Model-2 50 Table 4: Activities by importance ranking for processing of new applications 53 Table 5: Activities to be improved without the aid of IT for processing of new applications 54 Table 6 : Activities to be improved with the aid of IT for processing of new applications 55 Table 7: Alternative activities suggested for processing new applications 57 Table 8: Knowledge intensive activities 59 Table 9: Knowledge assets for Organization Model-3 60 Table 10: Worksheet Organization Model-4 61 Table 11: Organization Model-5 63 Table 12: Analysis of activity 1.12 (review applications)- part A 64 Table 13: Analysis of activity 1.12 (review applications)- part B 64 Table 14: Task Model-2 Knowledge item analysis of 'experience' 66 Table 15: Knowledge items- deficiencies 67 Table 16:Agent Model of ORS 69 Table 17: Knowledge analysis for activity 1.11 (study applications) by structured interview 70 Table 18: Knowledge analysis for activity 1.12 (review applications) by structured interview 70 Table 19: Organization Task Agent Model-1 for activity 1.11 (study applications) 72 Table 20: Organization Task Agent Model-1 for activity 1.12 (review applications) 73 Table 21: Abstracted class of research applications related to school 86 Table 22: Inferences for the domain research applications related to school 89 Table 23 Activities by importance ranking for processing of approved applications 112 Table 24: Activities to be improved with the aid of IT for processing of approved applications : 112 Table 25: Activities to be improved with the aid of IT for handling queries 112 Table 26 Alternative activity suggested for processing approved applications 113 Table 27: Alternative activities suggested for handling queries 113 Table 28: Worksheet Organization Model-3 114 Table 29: Knowledge analysis of activity 1.11 (Study applications) 115 Table 30: Knowledge analysis of activity 1.12 (Review applications) 116 Table 31 Task Model-2 -Knowledge item analysis of - policy guidance and minutes of last ER meeting 117 Table 32:Task Model-2 - Knowledge item analysis-Manual on tri council policy and ethics related literature 118 Table 33:Task Model-2 - Knowledge item analysis of - experience on handling applications 119 V List of Figures Figure 1: Processing of Knowledge Figure 2: Knowledge Management steps (Marquardt & Kearsley 1999) Figure 3: Knowledge interaction (Wiig et al. 1997) 1 Figure 4: Knowledge levels and status of applicability (Hicks et al. 2002) 1 Figure 5: CommonKADS model suite 2: Figure 6: Structure of the Organization Model 2! Figure 7: Layout of CommonKADS models 3 Figure 8: A n Activity 3< Figure 9: Example of an activity 3: Figure 10: Events 3: Figure 11: Example-Event 3: Figure 12: Events and Activity 3i Figure 13: Beginning and End of processes 3i Figure 14: " A N D ' construct triggering an activity 3' Figure 15: Example: " A N D ' construct triggering an activity 3' Figure 16: " A N D ' construct being triggered by an activity 3' Figure 17: 'Exclusive OR' being triggered by an activity 31 Figure 18: Example: 'Exclusive OR' being triggered by an activity 3^ Figure 19: 'Exclusive OR' triggering an activity 3 Figure 20: Activity Analysis format 4 Figure 21: Structure of Analysis 4 Figure 22: Framework of knowledge analysis using CommonKADS 4' Figure 23: Structure of Analysis (Organization Model) 4: Figure 24:Organizational structure of ORS 5 Figure 25: Structure of Analysis (Task Model) 6 Figure 26: Structure of Analysis (Agent Model) 6: Figure 27: Structure of Analysis (Knowledge Model) !• Figure 28: Structure of Knowledge model 7: Figure 29: Hierarchy of Knowledge intensive tasks 7' Figure 30: Inference structure of the modified assessment task 7' Figure 31: Structure of Analysis (Communication Model) 8: Figure 32: Communication model for the activity 1.12 (review applications) 8' Figure 33: Categories of norms for the domain 'applications related to schools' 8' Figure 34: Inference structure of the assessment task 12' Acknowledgements I thank Prof. Yair Wand for his directions, support and the opportunity he gave me to work with him. I thank Dorit Nevo for her extensive support, encouragement and help whenever needed to develop and write this thesis. I thank Prof. Carson Woo and Dr. Jacob Steif for their support and suggestive comments in developing this thesis. M y sincere appreciation and thanks to Shirley Thompson- Manager, Office of Research Services for her help, support and encouragement. My thanks to Jim Frankish-Chair of the ORS committee, Brent Sauder- Director of ORS, members of the Ethics Review committee and ORS staff for helping and supporting me to conduct this study. vii 1 Introduction Knowledge is seen as an invaluable resource in organizations. It is a prime enabler to successfully carry out the business processes within the organization, by creating values for the recipients of organizations products & services. In spite of acknowledging the importance of knowledge in the context of organizational processes, there have not been many research initiatives taken on analyzing knowledge in business processes. This thesis attempts to insight into knowledge in business processes. 1.1 Research Objectives The purpose of this paper is to explore the research question: "How can we extract knowledge from business processes?" By extraction we mean identification of knowledge needs, users, sources and ways to capture & share knowledge. The following objectives are identified for this thesis: 1. Identify a methodology for extracting knowledge requirements from business processes. 2. Applying this methodology to a case study: 2.1. Analyze business processes and identify knowledge intensive processes in an Organization 2.2. Identify knowledge sources, needs and users in knowledge intensive processes 2.3. Develop models to capture and share knowledge (knowledge generated, by whom) in a business process using a knowledge engineering method (CommonKADS) 2.4. Apply one of the models developed to real situations. 3. Draw conclusions regarding the effectiveness of methodology to extract knowledge requirements from business processes. 1 The first objective deals with identification of a proper methodology to extract knowledge from business processes. This methodology is developed using two methods, first event driven process modeling (Steif 2001) for business process analysis and second using a knowledge engineering method- CommonKADS to conduct knowledge requirements analysis of business processes. The second objective is focused on using this knowledge engineering method for business processes. The approach is based on analyzing knowledge in business processes in the context of a case study. 2 2 Literature Review 2.1 Knowledge Often data and information have been used to gain competitive advantages in organizations. It would seem obvious that the collective data and information would lead to develop knowledge in organizations and this knowledge would be well capitalized. But often this is not the case, organizations struggle to define and identify knowledge that is often used in various contexts such as business processes. Organizational assets that are embedded in human brains like experience, expertise are often not captured and used by organizations. The challenge lies in identifying the knowledge sources, users and flows in an organization. Knowledge is about "how to" attain a certain goal to perform a certain task. Before we explore the concepts of knowledge in details we discuss two building blocks of knowledge- data and information. The other important ingredients of knowledge are 'experience' and 'expertise'. Data is usually considered to be textual, either numeric or alphabetical. It is often classified into structured data and unstructured data, but data lacks inherent meaning and provides no sustainable basis for action. Information can be considered as an element 'describing a fact' (Hicks et al. 2002). Information is data that comes with value-added interpretations-it's organized for some purpose, and is meant to have an impact on the recipient's behavior. Some researchers (Mcmohan et al., 1995; Wall, 1986) suggest a classification of information- formal and informal. The primary difference between formal and informal information is the structured nature of formal information, although both may share common mechanism for their exchange. Formal information is an element of information that provides a specific context and measure. It 3 provides a structure or a focus so that individuals exposed to it may infer the same knowledge from it. For example structured textual information may be numeric or alphabetic. Informal information is considered to cover unstructured information, the majority of which is developed through interactions between two or more individuals. The information may change dynamically as content is added or altered. For example verbal conversation is an example of unstructured information as it is a dynamic process and information sets are added, removed or altered as the discussion progresses. Knowledge is often placed in a hierarchy with data and information below it. Peter Drucker (Drucker, 1998) referred knowledge; "Information is data endowed with relevance and purpose. Converting into information thus requires knowledge. And knowledge by definition, is specialized." Knowledge and information are interrelated. Information is what is used to change a course of action. Sometimes knowledge informs what information you need, where to look for and how to use. We have selected the definition of knowledge as given by Davenport & Prusak for our study. "Knowledge is a fluid mix of framed experience, values, contextual information, and expert insight that provides a framework for evaluating and incorporating new experiences and Information. It originates and is applied in the minds of knowers. In organizations, it often becomes embedded not only in documents or repositories but also in organizational routines, processes, practices and norms." (Davenport & Prusak, 1998, p 5) This definition makes it clear that knowledge is a mixture of various elements; it is fluid and formally structured; it is intuitive and therefore hard to capture in words or understand completely in logical terms. Knowledge exists within people, part and parcel of human complexity and is unpredictable. There are two aspects of knowledge in related to its generation. These are knowledge element and the knowledge assets. (Hicks et al. 2002). Knowledge assets 4 are the knowledge regarding markets, products, technologies and organizations, that a business owns or needs to own and which enable its business processes to generate profits, add value, etc. (Macintosh et al. 1998). Knowledge elements are inferred from one another from one or more elements of information. This information can be formal or informal. Knowledge elements suitable for decision-making are considered to be perspectives of formal information. Figure 1 depicts the process of obtaining knowledge. Data is processed into information. Knowledge is applied to the information and knowledge of possible outcomes is formed. Together "Data" and "Information" form important ingredients of knowledge. ^ ^ P r ^ e s s e d into Applied knowledge Data Information Outcome or prediction Figure 1: Processing of Knowledge Knowledge Management Knowledge management (KM) is not only about managing knowledge assets but also managing the processes that act upon the knowledge assets. These processes include develop, preserve, share and use knowledge. (Macintosh et al. 1998) Therefore, Knowledge management involves the identification and analysis of available and required knowledge assets and knowledge asset related processes. It also involves the subsequent planning and control of actions to develop both the assets and the processes so as to fulfill organizational objectives. Knowledge management can also be defined as the ways and means that organizations capture, store and access (reuse) knowledge to accomplish enterprise goals. Successful K M has three principal tenets: • Organizational processes and rules (e.g. taxonomy) • Knowledgeable and engaged individuals • Appropriate technology to support knowledge sharing Brint.com (a popular knowledge management portal) defines Knowledge Management as "caters to the critical issues of organizational adaptation, survival and competence in face of increasingly discontinuous environmental change.... Essentially, it embodies organizational processes that seek synergistic combination of data and information processing capacity of information technologies, and the creative and innovative capacity of human beings." (Brint.com 2002) A l l the above definition of knowledge management enforces the importance of organizational (or business) processes. Marquardt & Kearsley (1999) argue that a cohesive knowledge management system involves five stages as knowledge transitions from source to use: (1) knowledge acquisition (2) knowledge storage and mining (3) knowledge analysis (4) knowledge sharing and dissemination and (5) knowledge application and validation. Figure 2: Knowledge Management steps (Marquardt & Kearsley 1999) 6 Knowledge acquisition can be achieved through knowledge sourcing and knowledge creation. Knowledge sourcing happens within and outside the organization. The major issue in sourcing the knowledge within the organization is on how to capture the tacit knowledge that includes: employee's expertise, experience, memories, and assumption, all of which can be of high value to the organization. This tacit sources are usually difficult to communicate or explain, but can result in substantial benefit to organizations. External sourcing of knowledge can be obtained by doing: benchmarking, attending conferences, hiring consultants, etc. Knowledge can be created through innovation, problem solving, experimentation, and demonstration projects. A knowledge store system enables an organization to contain and retain knowledge, so that it becomes organization's property and will retain within it when the employee leave the company. However, knowledge is usually scattered and difficult to find, and is also liable to disappear without a trace because it is not stored. Knowledge analysis is analyzing the knowledge that is required to be stored. Organizations have to determine what data and information will be used for the organizational operations, and then decide what methods will be used to store the knowledge. The considerations that should be taken into accounts are: (1) knowledge stored should be easily and quickly accessible; (2) it should not only in the form of "topics" but also based on learning needs of staff, organizational goals for continuous improvement, and user expertise; (3) knowledge stored should be updated so it remains accurate and valid. Knowledge needs to be shared and disseminate across the organization to support organizational success. Knowledge sharing involves both the organizational and technological movement of information and knowledge. Knowledge can be distributed within an organization both intentionally and unintentionally. Organizational ways to transfer knowledge are individual written communication, training, internal conference, briefing, internal publications, job rotation/transfer, and mentoring. The technological modes to transfer knowledge can be in the 7 form of electronic mail, bulletin boards, and conferencing, which allow interaction among organization's members. Technology is needed to gain optimum application value from corporate knowledge (Marquardt & Kearsley 1999). The implementation of technology in utilizing the knowledge is for examples through: (1) diagnosing and troubleshooting in customer service; (2) shared databases to ensure identical information storing and updating; (3) establishing an installed networked IT infrastructure for all employees; (4) creating enterprise-wide data, object, and knowledge repositories; (5) automating operation, management, and support activities; and (6) developing integrated performance support system and knowledge discovery and data mining applications. Putting knowledge to use is the final K M step. This is achieved by the continued development and use of knowledge as part of individuals' day-to-day work, and as part of decision-making. Knowledge Management Processes Wiig et. al (1997) developed a knowledge management cycle as a conceptual framework to manage knowledge management techniques. This framework is important to mention as it discusses the K M process in context to business processes. They describe K M as a cyclic process consisting of four tasks: review, conceptualize, reflect and act. These phases interact with knowledge in several ways, including developing new knowledge, distributing existing knowledge, combining knowledge and consolidating knowledge. Review The first phase is the review phase that consists of two parts, monitor performance and evaluate performance. Monitoring performance is looking for opportunities for continuing improvement 8 in the existing practices. Evaluation of performance is done based on the goals of the organization or knowledge management strategies. This is to determine if results have been achieved based on some predefined measures. The possible criteria to evaluate the knowledge infrastructure are: • Content related: Does the infrastructure contain the right knowledge? Is the knowledge consistent? • Time related: Is the knowledge infrastructure stable? Susceptible to organizational change? • Usability: Is the knowledge in a form that permits ease use of learning? • Availability: Do people who require the knowledge have ready access to it? Conceptualize This is a very important phase. It is the phase where one identifies and analyzes the knowledge assets in a project. It consists of sub-tasks to identify, represent and classify knowledge in relation to the organizational processes and to the roles of the employees. The outcome of this task is a model of the current knowledge infrastructure. This task requires some sort of model of the organization in terms of the business processes and the role of people in these processes. Ideally libraries of existing, generic models should be used as a starting point here. A second requirement is a means to identify knowledge and its role in the business processes. Documenting the knowledge in some (semi-) formal representation is required, since this will drive the other knowledge management activities. The next tasks would be to identify the roles that use knowledge and link the knowledge assets to business processes. Wiig et al. use a simple diagram to depict the interaction of knowledge and ask questions: what' business processes use 'which' particular knowledge assets in their operation? 'where' and 'when' time and location are 9 attributes of the knowledge asset? Although it is often difficult, it is important to identify these knowledge assets. Figure 3: Knowledge interaction (Wiig et al. 1997) Wiig et al. advocate different techniques to associate knowledge with roles, such as knowledge mapping, knowledge scripting and profiling. To identify the processes that need the knowledge, they suggest task environment analysis, critical knowledge function analysis, knowledge use and requirement analysis and knowledge flow analysis. Finally, a method to describe the knowledge is needed. These descriptors capture the attributes of the knowledge. After the knowledge has been organized, it is to be analyzed for strong and weak points. Two methods are suggested, bottleneck analysis and SWOT (Strengths, Weaknesses, Opportunities and Threats) analysis. Bottleneck analysis examines for constraints that limit the ability of a process to execute more efficiently. The SWOT analysis is used to analyze knowledge from the perspective of the organizational goals. Reflect The 'reflect' phase is used to produce improvement plans that are to be carried out in the 'act' phase. The quality of the decisions made in this phase depends heavily upon the previous phase. 10 It is not uncommon for the wrong conclusions to be made before a proper analysis. Wiig et al. advise either elaborating on the SWOT analysis, bottleneck analysis or thinking of the improvements in terms of programs: • Effectiveness improvement programs • Knowledge building programs • Strategic action programs Improvements must be prioritized using decision analysis for implementation. To do this, some value must be placed for the improvements using a methodology. Once the improvements are identified, they are incorporated into improvement plans for execution. Act This phase cannot be attributed as a knowledge management activity. Several enablers of plans already exist in areas such as human resource management, information technology management and organizational development. The business process model should provide the identification of main process functions and activities, main decisions and decision makers and allow the reuse of a model as guideline for future process planning, management and control. Hicks et al. (2002) have identified four levels of knowledge elements, shown in figure 4 with associated states of applicability. At the highest level, generic principles or general knowledge is applied to unfamiliar situations. Here the applicability of individual elements to the new situation is not an issue because they are generic. However decisions are rarely taken based on the single aspect of knowledge and it is likely that the necessary knowledge can only be generated through the application of a combination of knowledge elements. Consequently, knowledge regarding the assimilation of knowledge elements is required, knowledge about knowledge (meta knowledge). 11 In the case of specific and case knowledge elements, which are to be applied self-directed then knowledge regarding applicability and assimilation can be captured. For such case formalizations of accepted knowledge processes, such as best practices, or the explicit capture of knowledge as derived by an individual is required. Finally, for generic or specific knowledge applied to familiar situations it is required that knowledge describing the limitations or scope of applicability be made available, so that misapplication does not occur. Extrapolation of knowledge Knowledge level T Level of applicability General Principles < i Applicable to unfamiliar situations and domains Generic knowledge • <-t Specific knowledge <-Applicable to unfamiliar situations for a particular domain Applicable to familiar situations within a particular domain Cases t < Applicable to specific situation within a particular domain Figure 4: Knowledge levels and status of applicability (Hicks et al. 2002) 2.2 Business Processes Processes, by definition, represent ways of doing something-accomplishing some task or other. Every task, however, can always be integrated into a more encompassing and comprehensive 12 task or divided into multiple subtasks. As a consequence, any process can be aggregated into a higher-level process or subdivided into lower-level processes. Definitions In order to get work done, every organization creates and aligns specific sequences of tasks to achieve particular purposes. For example, a substantial number of related tasks must be executed in a specific sequence in order to receive and fulfill customers' orders or to purchase and acquire components from suppliers. When a number of tasks cumulate to constitute the execution of some substantial organizational (or business) requirement, they are commonly referred to as a business or organizational process. Business executives have different perspectives on what business processes mean to their organization. The meaning of business process depends on the context and person using the term. Darnton (1997) provides the following definition of business processes based on survey on various academic works: " [A business process is] simply a structured, measured set of activities designed to produce a specified output for a particular customer or market. It implies a strong emphasis on how work is done within an organization, in contrast to a product focus's emphasis on what. A process is thus a specific ordering of work activities across time and place, with a beginning, and end, and clearly identified inputs and outputs: a structure for action." (Darnton 1997, p.10). Davenport (1993) asserts a simplified version of this definition, as "processes are the structures by which an organization does what is necessary to produce value for its customers." Nickols (1998) defines business processes as streams of activity that flows across functional boundaries. For this reason business processes are said to be fragmented, that is scattered across so-called 13 'functional silos'. In this study, we view business process as a collection of related activities that has a common objective to accomplish. For example, a business process of processing a business transaction may consist of a set of activities like identifying, validating for authenticity and approving the transaction. A l l these activities are related but distinct from each other and have specific inputs, outputs and objective. We discuss the components of the activities more in the methodology section where we use a technique to analyze business processes. Characteristics Based on the definitions and analysis of business processes in the earlier section we can summarize the characteristics of business processes: • Business processes are portions of streams of activity that contribute to business results • Business processes involve a specific ordering of work tasks or activities across time • The collection of tasks and activities together transforms inputs into outputs • Inputs may take many forms including data and information, technology, and people • Business processes typically manifest an identifiable beginning and end • The tasks and activities serve as a focal point in bringing individuals together in order to get work done • How the tasks and activities get sequenced, interrelated, and executed can, and typically does, change significantly over time • Typically a single process always connects to multiple other processes Pritchard & Armistead (1999) mention the common features of managing business processes. • A process architecture is developed as a means of understanding the organization; this may involve the mapping of business processes. • Process owners are appointed with responsibility for the overall process. 14 • Process metrics and effectiveness criteria are established and cascaded down to frontline teams. • Performance monitoring is tailored to address the process dimension. • Improvement opportunities are identified and action being taken. • The organization plans, communicates and trains around the process model. Nickols (1998) mentions two types of business processes-transformational and transactional. Transformational business processes are concerned with converting organizational inputs into organizational outputs. Example- obtaining from suppliers the inputs necessary to sustain the functioning of the organization. Transactional business processes are concerned with exchanging outputs for new inputs to continue the cycle of events of which any given process is a part. Example - Converting products and services coming in, to products and services going out. Degrees of importance were attached to business processes, often through the appointment of process owners. The ability to superimpose information flows and lines of communication that support knowledge creation and information sharing throughout the process architecture appears to be a prerequisite for Business Process Management. Modeling Business Processes The following approach can be taken to model the business processes: • How the organization carries out its business - modeling the business processes • What the processes manipulate - modeling the resources • Who carries out the processes - modeling capabilities, roles and authority • Where a process is carried out - modeling of the communication between agents 15 Knowledge and Business Processes Success in an increasingly competitive marketplace depends critically on the quality of knowledge that organizations apply to their key business processes. For example supply chain processes depend on knowledge of diverse areas including raw materials, planning, manufacturing and distribution. In order for employees to be able to execute business or decisional processes they must possess some 'working knowledge' (e.g. about process functionality, required process inputs and delivering outputs). Working knowledge is constantly developed and updated through receiving information from the internal environment (i.e. the execution of business and decisional processes) and from the external environment. Therefore capturing of working knowledge of employees represents the main objective of K M . According to the suitability for formalization and structuring, such working knowledge can be divided into two groups: formalizable and non formalizable knowledge. (Kalpic & Bernus, 2002). The formalization and structural description of innovative and creative processes, such as some management decisions is a difficult task, due to the fact that the set of constituent processes is not predefined, nor is the exact nature of their combination well understood by those who have the knowledge. In contrast the group of i l l -structured and structured (repetitive or algorithmic) processes can be formalized and structured at least to a degree; consequently the knowledge about these processes is considered formalizable. (for example- coordination of manufacturing activities). The formalizable part of knowledge is extremely important and valuable for K M , because this may be distributed and thus, shared with relative ease. In general it can be said that K M S is primarily focused on solutions for the capture, organize and distribution of tacit formalizable knowledge. 16 2.3 Knowledge Engineering Traditionally in the context of Expert Systems Knowledge Engineering (KE) was viewed as a process of extracting or mining from the expert's head and transporting it in computational form to a machine. K E was considered to be a transfer process. "This transfer and transformation of problem-solving expertise from a knowledge source to a program is the heart of the expert-system development process." (Hayes-Roth, Waterman & Lenat, 1983) K E practice has taught that even though knowledge-based systems can be built and usefully employed, most knowledge still resides in the heads of humans and in documents that cannot be easily formalized. Today K E is approached and considered as a modeling activity. Knowledge engineering has several different applications; the construction of knowledge system is one of them. Knowledge Engineering (KE) is considered as a modeling activity to build knowledge based system with the aim of realizing problem-solving capabilities comparable to a domain expert. The final aim of K E is to build knowledge based systems which means building computer models with the aim to realize problem-solving capabilities comparable to a domain expert. Knowledge Engineering and Knowledge Management Knowledge engineering and knowledge systems can be viewed in this perspective: knowledge engineering as a methodology to be used as one of the instruments, and knowledge systems as one of the important products to be used in knowledge management. Knowledge engineering offers concepts and methods useful for knowledge management. Some of these concepts are highlighted by Schreiber et al. (1999) • Knowledge-oriented analysis helps to quickly map out fruitful areas for knowledge-management actions. 17 • Task and agent analysis has shown to be very useful for clarifying knowledge bottlenecks in specific areas. Techniques like these are relevant to business process redesign and improvement where knowledge work is involved. • Knowledge engineering places strong emphasis on the conceptual modeling of knowledge-intensive activities. The K E techniques have proved to be very useful in clarifying the major tacit aspects of knowledge, in a way enabling and stimulating fruitful communications with a variety of people (managers, specialists, end users, customers) who often do not have a background in information technology. • The accumulated experience of knowledge engineering shows that there are many recurring structures and mechanism in knowledge work. This has, for example, led to libraries of task models that are applicable across different domains. This approach offers many useful insights into constructing the reusable information architectures and software components that are increasingly needed in modern IT-based organizations. In this study we would use a Knowledge Engineering technique to analyze knowledge requirements of an organization. The usefulness of this technique is discussed in the conclusion section. Knowledge Engineering techniques Knowledge Engineering methods and tools address the use of a company's knowledge assets. They provide approaches to designing and building knowledge-based applications. A number of knowledge modeling methodologies have been developed based on the knowledge-level notion, out of which popular ones are CommonKADS knowledge model (Schreiber et al.. 1999), M I K E (Angele et al. 1996), PROTEGE-U (Puerta et al. 1992) and Knowledge Audit- developed by Anderson Consulting (Dignum & Heimannsfeld, 1999). 18 Analysis of Knowledge Engineering methodologies In this section we will describe four modeling frameworks briefly that address various aspects of model-based Knowledge Engineering approaches. After the descriptions of these methods, one of these methods would be selected based on its suitability. CommonKADS (Schreiber et al. 1999) A prominent knowledge-engineering approach is KADS, which was further developed to CommonKADS. A basic characteristic of CommonKADS is the construction of a collection of models, where, each model captures specific aspects of the Knowledge Systems to be developed as well as of its environment. In CommonKADS the Organization Model, the Task Model, the Agent Model, the Communication Model, the Knowledge Model and the Design Model are distinguished. The first four models analyze the organizational environment and the corresponding critical success factors for a knowledge system. Within the Organization Model the organizational structure is described together with a specification of the functions that are performed by each organizational unit. Furthermore, the deficiencies of the current business processes, as well as opportunities to improve these processes are identified. The Task Model provides a hierarchical description of the tasks that are performed in the organizational unit in which the Knowledge system will be installed. This includes a specification of which agents are assigned to what tasks. The Agent Model specifies the capabilities of each agent involved in the execution of the tasks at hand. Within the Communication Model the various interactions between the different agents are specified. The knowledge and communication models yield the conceptual description of problem-solving functions and data that are to be handled and delivered by a knowledge system. The design model converts this into a technical specification that is the basis for software system implementation. 19 PROTEGE II (Puerta et al. 1992) The PROTEGE-II approach aims at supporting the development of Knowledge based systems (KBS) by the reuse of ontologies. An ontology is a formal, explicit specification of a shared conceptualization (Gruber, 1995). A 'conceptualization' refers to an abstract model of some phenomenon in the world. In addition PROTEGE-II puts emphasis on the generation of custom-tailored knowledge-acquisition tools from ontologies. PROTEGE-II relies on decomposing the tasks into subtasks. In PROTEGE-II the input and output of a method is specified by method ontology that, defines the concepts and relationships that are used by the problem solving methodology for providing its functionality. A second type of ontology used within PROTEGE-II is domain ontology. It defines a shared conceptualization of a domain that is reusable components for building up a KBS. PROTEGE-11 proposes the notion of an application ontology to extend domain ontologies with specific concepts and relationships. MIKE (Angele et al. 1996) The MIKE approach (Model-bused and Incremental Knowledge Engineering) provides a development method for Knowledge systems covering all steps from the initial elicitation through specification to design and implementation. M I K E proposes the integration of semiformal and formal specification techniques and prototyping into an engineering framework. Integrating prototyping and support for an incremental and reversible system development process into a model-based framework is actually the main distinction between M I K E and CommonKADS. In MIKE, the entire development process is divided into a number of sub activities: Elicitation, Interpretation, Formalization/ Operationalization, Design and Implementation. Each of these activities deals with different aspects of the system development. 20 Knowledge Audit (Dignum & Heimannsfeld, 1999) Knowledge Audit is a more practical oriented K E technique developed by Andersen Consulting. The most usual knowledge problems in an organization are unbalanced distribution, fragmentation, unavailability and inaccessibility of knowledge. Knowledge audit is based on the following steps: • Identification of organization goals (or department goals, process goals or activity goals). • Identification of problems that hinder the achievement of these goals. • Identification of the organizational processes to achieve organization goals. • Analysis of these processes from a knowledge perspective. • Analysis of the problems from a knowledge perspective. • Identification and definition of knowledge problems and generic solutions. • Implementation of concrete solutions. CommonKADS is prominent for having defined the structure of the knowledge model, M I K E puts emphasis on a formal and executable specification of the design model as the result of the knowledge-acquisition phase, PROTEGE -II exploits the notion of ontologies and Knowledge audit emphasizes on knowledge processes. There is a similarity of CommonKADS and M I K E in the stage of model designing but CommonKADS also considers building the Organization model and understanding the knowledge problems of the entire organization. Moreover CommonKADS is structured and well-formalized methodology. Based on the above observations we chose CommonKADS as the Knowledge Engineering methodology for our study. 21 3 The Methods 3.1 Case Study method Benbasat et al. (1987) defines case study as an examination of a phenomenon in its natural setting, employing multiple methods of data collection to gather information from one or few entities (people, groups, organizations). They advocate the use of case study on certain types of problems, "...those in which research and theory are at their early, formative stages, and 'sticky practice-based problems' where the experience of the actors are important and the context of action is critical." (Benbasat et al. 1987, p.369) Benbasat et al. (1987) mention some key advantages of using the case study method. These are: • Information systems can be studied in a natural setting and theories can be generated from practice. • Allows understanding the nature and complexity of the processes taking place. • An appropriate way to research an area where few previous studies have been carried out. Case study method is appropriate in capturing knowledge of practitioners. Case study researchers Stake (1995) and Yin (1984) have suggested techniques for organizing and conducting the research successfully. They propose six steps that should be used in a case study. These are: • Determine and define the research questions • Select the cases and determine data gathering and analysis techniques • Prepare to collect the data • Collect data in the field • Evaluate and analyze the data • Prepare the report 22 The case study method is aptly chosen as the methodology for this thesis as it takes the advantages of the method mentioned above. Knowledge Engineering is a relatively new area and case study method gives an opportunity to test the methodology developed for the knowledge requirements analysis. The steps for the case study method mentioned above were followed. The research question and objectives for this thesis were defined at the outset (section 1.1). An U B C organization - Office of Research Services (ORS) was facing problems in one of their operations -Ethics Review process involving human subjects. These problems were perceived as knowledge related (described in section 4). After initial study of the problems it was analyzed that the problems may be related to improper knowledge capture & sharing between the stakeholders of the organization and thus knowledge requirements analysis of their business processes may be helpful to solve the problems. The organization agreed to participate in this research, thus giving the opportunity to implement the research objectives. Data collection methods were developed (structured interviews with the stakeholders, observations of the Organizational processes) and data were collected over a period of two months. After analyzing the data the potential solutions were identified (section 5). The methodology developed for knowledge requirements analysis was tested through this case study. 3.2 CommonKADS Overview CommonKADS is a leading methodology to support structured knowledge engineering. CommonKADS has been developed over 15 years and is now being used in some application projects. The developers of CommonKADS (Schreiber et al.) claim that CommonKADS is the European de facto standard for knowledge analysis and knowledge-intensive system 23 development, and has been adopted as a whole or has been partly incorporated in existing methods by many major organizations in Europe, as well as in the US and Japan. CommonKADS enables spotting the opportunities and bottlenecks in how organizations develop, distribute and apply their knowledge resources, and so gives tools for corporate knowledge management. CommonKADS supports the development of knowledge systems that support selected parts of the business process. CommonKADS provides the methods to perform a detailed analysis of knowledge-intensive tasks and processes and provides a clear link to modern object-oriented development and uses notations compatible with Unified modeling language (UML). Below we summarize the CommonKADS methodology from 'The Knowledge Engineering and Management: The CommonKADS Methodology, - G. Schreiber, H. Akkermans, A. Anjewierden, R. Hoog, N. Shadbolt, W. Velde and B. Wielinga' (Schreiber et al. 1999). We will use this methodology to analyze knowledge requirements in business processes of the organization (ORS). The commonKADS methodology offers a structured approach. It is based on the following fundamental principles underlying modern knowledge engineering. • Knowledge engineering is not some kind of 'mining from the expert's head', but consists of constructing different models of the human knowledge. • The knowledge-level principle in knowledge modeling concentrates on the conceptual structure of knowledge. • Knowledge has a stable internal structure that is analyzable by distinguishing knowledge types and roles. • A knowledge project must be managed by learning from experiences in a controlled way. 24 CommonKADS Model Suite The CommonKADS model suite is the practical expression of the principles underlying knowledge analysis. It constitutes the core of commonKADS knowledge-engineering methodology. The top-down model suite is drawn in figure 5. The three groups of models are interconnected by the solid lines. These lines indicate that the second and third layers of models are derived from the upper layers of models. For example the knowledge model in the second model is generated from the organization, task and agent models. Organizational model Task Model Agent model Knowledge model Communication model Design model Figure 5: CommonKADS model suite Figure 5 shows three groups of models, because there are essentially three types of questions that must be answered before we understand each model in the model suite. 1. Why? Why is a knowledge system a potential help or solution? For which problem? Which benefits, costs, and Organizational impacts does it have? Understanding the Organizational context and environment is the most important issue here. 25 2. What? What is the nature and structure of the knowledge involve? What is the nature and structure of the corresponding communication? The conceptual description of knowledge applied in a task is the main issue here. 3. How? How must the knowledge be implemented in a computer system? How do the software architecture and the computational mechanism look? The technical aspects of the computer realization are the main focus here. A l l these questions are answered by developing (piece of) aspect models. CommonKADS has a predefined set of models, each of them focusing on limited aspects, but together providing a comprehensive view. These models are explained briefly below: Organizational model - the organization model supports the analysis of the major features of an organization, in order to discover problems and opportunities for knowledge systems, establish their feasibility and assess the impacts on the organization of intended knowledge actions. Task model - Tasks are the relevant subparts of a business process. The task model analyzes the global task layout, its inputs and outputs, preconditions and performance criteria, as well as needed resources and competencies. Agent model - Agents are executors of a task. An agent can be human, an Information System or any other entity capable of carrying out a task. The purpose of the agent model is to understand the roles and competences that the various actors bring in them to perform a shared task. It also yields input information for other CommonKADS models, specially the communication model. The agent model describes the characteristics of agents, in particular their competencies, authority to act, and constraint in this respect. Knowledge model - the purpose of it is to explicate in detail the types and structures of the knowledge used in performing a task. It provides an implementation-independent description of the role that different knowledge components play in problem-solving, in a way that is 26 understandable by humans. This makes the knowledge model an important vehicle for communication with experts and users about the problem-solving aspects of a knowledge-system, during both development and system execution. Communication model - Since several agents may be involved in a task, it is important to model the communication transactions between the agents involved. The communication model does this in a conceptual and implementation-independent way, just as with knowledge model. The purpose of the communication model is to specify the information exchange procedures to realize the knowledge transfer between agents. The process of communication model goes in terms of three subsequent layers, from global to detailed specifications as described below: • The overall communication plan, which governs the full dialogue between the agents • The individual transactions that link two tasks carried by two different agents • The information exchange specification that details the internal message structure of a transaction. Design model - The above commonKADS model together can be seen as constituting the requirements specification for the knowledge system, broken down in different aspects. Based on these requirements, the design model gives the technical system specification in terms of architecture, implementation platform, software modules, representational constructs and computational mechanisms needed to implement the functions laid down in the knowledge and communication models. Together the organization, task and agent models analyze the organizational environment and the corresponding critical success factors for a knowledge system. The knowledge and communication models yield the conceptual description of problem-solving functions and data 27 that are handled and delivered by a knowledge system. The design model converts this into a technical specification that is the basis for software implementation. The Organization, task and knowledge models are described in details in the following sections: Organization model It is almost impossible to model the entire organization. Therefore a structured approach is necessary to model the organization. Different aspects like - organization structure, processes, personnel and resources are the important components of an organization. These components with their interaction with other components give the model of the organization. The components are modeled with current and future situations. By comparing these descriptions, one gets a good feel of the value, feasibility and acceptance of new-knowledge oriented solutions. The organizational model with its components is shown in figure 6. 28 Organization-Model 1 Organization-Model 2 Organization-Model 3 Organization-Model 3 (OM-1) (OM-2) (OM-3) (OM-4) Problems & Opportunities General context Potential solutions Organization's focus area description: Structure Process • Process breakdown People Resources Knowledge -•• Knowledge Assets Figure 6: Structure of the Organization Model The model is divided into five parts -OM-1, OM-2, OM-3, OM-4 and OM-5 (OM-refers organization model). The first four parts (OM-1 to OM-4) are shown in figure 6. The horizontal lines (solid) show the components of each part and the vertical lines (dotted) show that the components are derived from other parts. Organization Model-1 (OM-1) focuses on problems and opportunities as seen in the wider organizational context. It contains the broader categories such as organization's mission, goals, strategy, etc. Organization Model-2 (OM-2) concentrates on the specific aspects of the organization like business processes, people, resources and knowledge. The process component in OM-2 plays a central role within the CommonKADS 29 Organization-analysis process. The business process is broken down into smaller tasks in Organization Model-3 (OM-3). A rough indication is given on how knowledge-intensive these tasks are and what knowledge is used. Organization Model-4 (OM-4) describes the knowledge assets used in each task. Organization Model-5 (OM-5) describes the business and functional feasibility of implementation of suggested solutions. Task model CommonKADS defines task as a subpart of a business process that: • Represents a goal-oriented activity adding value to the Organization • Handles inputs and delivers outputs in a structured and controlled way • Consumes resources • Requires (and provides) knowledge and other competencies • Is performed by responsible and accountable agents. The task model-1 (TM-1) is drawn with the refinement of Organization Model-3 (OM-3). It consists of goals and values of tasks, dependency and flows (input and output), timing and control (frequency and duration, precondition and post condition), agents, knowledge and competence and resources. The items of knowledge and competence are key items in the task model and therefore modeled separately as task model-2 (TM-2). TM-2 concentrates on bottlenecks and improvements relating to specific areas of knowledge. The knowledge item is analyzed in TM-2 by nature of knowledge (formal, experience, tacit, etc), form of knowledge (mind, paper, electronic, etc) and availability of knowledge (time, space, access, quality, form). The knowledge model Detailed requirement engineering is split in CommonKADS into two parts, the knowledge model and communication model. The knowledge model specifies the knowledge and reasoning 30 requirements of the prospective system and the communication model specifies the needs and desires with respect to the interface with other agents, (i.e. user interface or the interface of another system) Organizational model Task model Agent model Communication model Knowledge intensive task Knowledge model Design Model Figure 7: Layout of CommonKADS models When constructing a knowledge model it is assumed that a knowledge-intensive task has been selected from the Organizational tasks and that the main knowledge items involved in this task have been identified. Communication model and knowledge models are developed from the knowledge intensive task and the design model is an outcome of these two models. A knowledge model has three parts; each part is a knowledge category and captures a related group of knowledge structures. The first category is domain knowledge, (domain specific knowledge) second is the inference knowledge (basic inference steps to make use the domain knowledge) and third is the task knowledge (what goals an application pursues). Domain Knowledge Describes the main static information and knowledge objects in an application domain. A domain-knowledge description typically consists of two types of ingredients: domain schemes and knowledge base. The schema describes the static information/knowledge structure of the 31 application domain. It describes domain-knowledge types, such as concepts, relations and rule types. A knowledge base contains instances of those knowledge types. Inference knowledge Describes the lowest level of functional decomposition in the knowledge model. These basic information-processing units are called inferences. At the lowest level of decomposition of tasks, they are linked with inferences. An inference uses knowledge contained in some knowledge base to derive new information from its dynamic input. Task knowledge Describes the goals and the strategies that will be employed with tasks. The tasks are decomposed into smaller tasks and the tasks that are not decomposable are called primitive tasks and the rest tasks are called composite tasks. CommonKADS like other knowledge modeling approaches uses the concept of 'reusing' combination of model elements. CommonKADS has catalogued predefined sets of model elements- like ready-made building blocks. An example of the predefined sets of model element is task template. A task template is a partial knowledge model in which inference and task knowledge are specified. A task template is chosen for the analysis and discussed in the analysis section. Constructing the Knowledge model The process of knowledge-model construction can be decomposed into three stages. They are knowledge identification, knowledge specification and knowledge refinement. • Knowledge identification Information sources that are useful for knowledge modeling are identified. This phase can be considered as the preparation phase for the next phase. A lexicon or glossary of domain terms is 32 constructed. Existing model components are surveyed and the components that could be reused are identified. • Knowledge specification In this stage the specification of the knowledge model is developed. First a task template is chosen and an initial domain schema is constructed using the list of reusable model components identified in the earlier model. In terms of the domain knowledge the emphasis is on the domain schema. • Knowledge refinement In this stage attempts are made to validate the knowledge models as much as possible and to complete the knowledge bases by inserting a more or less complete set of knowledge instances (e.g. instances of rule types). Often simulation techniques (paper or computer) are used to validate the initial specification that comes out of the previous stage. The result of the simulation is often a good indicator of the quality of the knowledge model developed. 3.3 Process and activity analysis The processes and activities of the Organization are identified and drawn using 'event driven process-diagramming technique' developed by Jacob Steif (2001). This technique shows the sequence of activities and events using boxes, arrows and text. The notation is currently being taught at the University of British Columbia in business system analysis courses. The constructs of 'event driven processing-diagramming technique' are defined in the following sections. Activities Activities describe what is to be done (figure 8). They are seen as actions that are performed to change the state of something. Activities can be decomposed into operations. For an example an activity- reviewing an application may be decomposed into two operations, read the application 33 and write comments about the application. Agents describe who performs the activity. Resources describe what is used in the activity. Resources could be an information system, database or certain equipments necessary to conduct the activity. Rules describe controls (if any) on the activity. Activity also contains the property of 'roles' of agents, which can be quite specific depending on the activity or the operation. The role of an agent could be different in different operations even in the same activity. Not all activities will necessarily have all the properties mentioned above. The inputs to the activities are resources and/or information objects (in the form of resources), rules and knowledge. Outputs are physical objects, knowledge, and/or information objects (in the form of resources). The inputs and outputs are generally not shown with the activity diagram, they are shown separately. An example of an activity is shown in figure 9. f Activity name ^ Operation 1 Operation 2 {agent 1, agent 2..} [resource 1,...] (rulel, rule 2..) V J Figure 8: An Activity 34 r Review Application Read application Write comments {Chair} [Database] (review application after it is filled completely) Figure 9: Example of an activity Events Events describe when something is done or is to be done. Events consist of an arrow and an event name (figure 10). The direction of the arrow also denotes the sequencing and flow of consecutive activities and events. Event name Event name Figure 10: Events Application reviewed Figure 11: Example-Event 35 An event can be thought of as changes to the state of one or more objects that happen at a particular point in time. Events are seen triggering an activity or being triggered by an activity (figure 12). Event name Activity name Operation 1 Operation 2 {agent 1, agent 2..} [resource 1,...] ( rule l , rule 2..) Event name Figure 12: Events and Activity Events occur and trigger processes or activities. When events are triggered by, or trigger, a process they are denoted with a circle symbol to show the beginning or end of the process (in figure 13). Events that terminate an activity can be used as the input of another activity. Start of process • End of process i Figure 13: Beginning and End of processes Logical symbols Events can be drawn to show various combinations. When more than one event occurs in a process or activity, there are different logical conditions that may be expressed. 36 A N D : ' A N D construct means that all the events associated occur. This is denoted by a diamond symbol shown in figure 14. Activity name Figure 14: " A N D ' construct triggering an activity An example of the A N D construct is shown below: Application received Guideline to review application received Review application Figure 15: Example: " A N D ' construct triggering an activity In figure 16, both events must occur when triggering the activity. In the next figure, both events are triggered by the activity and they must occur together. ( ^ Activity name Figure 16: " A N D ' construct being triggered by an activity 37 O R : 'OR' construct means that one or more of the events may or may not occur. Showing one or more events triggering or being triggered by an activity denotes this. E X C L U S I V E O R : The 'Exclusive OR' (XOR) construct means that exactly one and only one of the choices will occur. This can be seen triggering (figure 17) and being triggered by (figure 19) an activity. Figure 17: 'Exclusive OR' being triggered by an activity Application approved Application not approved Figure 18: Example: 'Exclusive OR' being triggered by an activity r Activity name Figure 19: 'Exclusive OR' triggering an activity 38 Event driven process-diagramming technique & CommonKADS In this section we establish the link between event driven process-diagramming technique and CommonKADS. In the description of CommonKADS, the term tasks were used frequently as independent distinguishable units of decomposed activities consisting of inputs and outputs. The task model of commonKADS, specifically Task Model-1 (TM-1) analyzes each task by its goals, values, dependency and flows (input and output), timing and control (frequency and duration, precondition and post condition), agents, knowledge & competence and resources. Event driven process-diagramming technique analyzes activities by its constructs - input, output, agents, resources, rules and operations (figure 8). Thus we observe similarity between the task as defined in commonKADS and activity defined in Event driven process-diagramming technique. With the addition of some more constructs (Conditions for triggering, Roles, Rules and Description) in analyzing activity (activity sheets in Appendix B), we replace TM -1 of CommonKADS with activity analysis of event driven process-diagramming technique. Subsequently in this study the term activity is used instead of task. 3.4 Knowledge and process analysis In this section we discuss the methods used to extract data and information for analysis. The data and information gathered are used to derive various models of CommonKADS. The methods used for this study are Structured Interviews and Activity analysis. These methods are described next: 39 Structured Interview method The interview is the most common elicitation technique and takes many forms from the completely unstructured interview to planned structured interview. The structured interview is a formal version of the interview where the researcher plans and directs the session. The structured interview has the advantage of providing structured transcripts that are easier to analyze than unstructured transcripts ones. The structured interview method was used as a knowledge elicitation technique for analyzing activities. A l l the subjects were contacted by email to participate in interviews. The subjects signed the consent forms (Appendix B) and agreed to participate in the interviews. For each interview, a set of interview questions was prepared (Appendix B). To identify the knowledge requirements a set of questions regarding knowledge was asked. These questions are related to what knowledge is available to do the activity effectively and efficiently, what knowledge is required to do the activity effectively and efficiently, and how would the knowledge involved in the activity be shared. Activity analysis method This method was developed to analyze the activities described in the 'events driven process -diagramming technique'. Each activity is drawn in a separate sheet and the following characteristics of the activities are identified: purpose, actors, input, output and conditions for triggering. Each activity is divided into operations (if there are any) and the following information is identified: operation(s), roles, resources (data & knowledge), rules & descriptions. To generate the activity sheets the corresponding actors involved in the activities were contacted. They were asked to fill the formats (figure 20) for the activities where they are 40 involved. Two filled activity sheets on the prescribed format are given in Appendix B (for activity 1.11- study applications and 1.12- review applications). Name: Number: Inputs Name {Actors} (Resources) V Outputs * Purpose Actors Input Output Conditions for triggering Operation(s) Roles Resources (data & knowledge) Rules Description Figure 20: Activity Analysis format 3.5 Structure of analysis The layout of techniques/method used and commonKADS models generated for knowledge-requirements analysis is mentioned in figure 21. Event driven process modeling technique ! Activity analysis method Structured interviews Agent Model (AM) Communication Model (CM) Organization, Task and Agent Model (OTA-1) I Analysis of selected | research aDDlications ! Knowledge Model (KM) Knowledge Model (KM) - for a sub domain W Figure 21: Structure of Analysis 41 The diagram demonstrates the path taken and steps for analysis. The boxes with solid lines represent the models to be generated and boxes with broken lines represent methods or techniques to be used. The arrows indicate the directions of the boxes that are generated or derived from. Figure 21 shows four techniques/method (event driven process modeling technique, activity analysis method, structured interviews and analysis of research applications) used together to generate five commonKADS models (organization, task, agent, communication and knowledge models). The activity sheets of activity analysis methods are generated from event driven process modeling technique. The activity analysis method is used to generate organization, task and agent models. The structured interviews are used to generate organization model and combination of organization, Task and Agent model (OTA-1). The knowledge model for the sub domain is developed from the knowledge model and the analysis of selected research applications. While analyzing each of the commonKADS models, this structure is presented highlighting the part which is being analyzed. 42 4 Case Study- Office of Research Services Stakeholders The Office of Research Services (ORS) is an internal organization of University of British Columbia (UBC) located at the university campus. The following stakeholders of the organization are identified - UBC, ORS, Researchers at UBC and Ethical Review (ER) committee. The key responsibility of ORS is to ensure compliance with government regulations and granting agency requirements for the use of human subjects, animal subjects and biohazardous materials in research conducted at U B C . Any research project carried out by a person affiliated with the U B C that involves one of the above mentioned areas must conform to the University policy on research and get approval from ORS prior conducting the research. U B C as a stakeholder has to ensure maintaining high quality and standard of research abiding the research policies as set up by U B C and the Government. The Ethical Review committee consists of distinguished U B C faculty from different departments and is responsible for making correct and consistent decisions of Ethical Review applications. U B C researchers have to comply with the research policies as set up by UBC, apply for Ethical Review for their research applications and conduct research. The scope of the case study is limited to Ethical Review processing of research related to human subjects only. Objectives of the ER process The objectives are to have consistent decisions of ER applications, process the applications in a relatively short period of time and ensure that research applications meet ethics compliance. 43 Ethical Review process The researcher applies for ER by filling a paper based ER application and submits to ORS. The applications are stored and forwarded to the ER committee for review. The ER committee discusses the ethical issues of the applications. If an application addresses the necessary ethical issues then the committee approves the application and a certificate of approval is issued to the researcher for a period of one year. If an application does not address all the required ethical issues then a memo is sent to the researcher mentioning the ethical deficiencies found in the application. The researcher addresses these issues and the application is reviewed again. This cycle continues till all the ethical concerns are addressed and certificate of approval is given to the researcher. The main processes and the activities of ORS are drawn using event driven process diagramming technique in Appendix D. The descriptions of the events of the activities are mentioned in Appendix A . Problems The following problems are identified in ORS related to processing of ethical applications: • The turnaround time for processing applications is relatively long • The decisions for processing applications might be inconsistent • The process of submitting applications is cumbersome • A large number of applications is processed by few committee members and ORS personnel In most cases applications are processed within six weeks. This time frame fluctuates depending on the number of applications received by ORS in a month. Because of large number of applications received, generally not all applications are reviewed in a scheduled meeting and therefore some applications are deferred to the next meeting to review. The backlog of applications increases and further delays the review process. The decisions for processing 44 applications might be inconsistent sometimes. The committee members are not aware of the past decisions taken on similar or same applications. Thus similar applications that were processed in the past might have ended with different decisions. The knowledge of the previous decisions are not captured or utilized to make new decisions. This may lead to inconsistency of decisions. The process of application to get ethics approval is cumbersome for the researchers. The researcher has to download the ethics application from the ORS website, fill the application (often some fields in the application are not related to his/her research) and make 20 copies of the application and other supporting documents before submitting to ORS. Objectives of the project The undertaken ORS project has the following objectives. First, to reduce the processing time of the ER applications. Second, to help the ER committee to make more consistent decisions. Third, to reduce the workload of ER committee, ORS and researchers. Approach The approach taken to address the problems of ORS is to conduct knowledge requirements analysis of the ORS processes. While analyzing the processes there could be some processes or activities where the knowledge associated is not captured, used or shared properly. For example, the problem of inconsistency of decision-making seems directly related to improper knowledge capture and sharing between the actors. A probable solution that may generate from the requirements analysis is creation of a knowledge system where the decisions of the past ER applications are available to committee members. It is evident that not all the problems could be solved by this approach. Introducing Information Technology in some activities can probably alleviate the problem of processing time of applications, like- submission of ER applications 45 online to ORS, online access to ER applications by ER committee to review, etc. While conducting knowledge requirements analysis, each activity of the processes is analyzed closely. 5 The Analysis The CommonKADS methodology is used to develop models mentioned in the CommonKADS suite. Some elements of knowledge modeling of CommonKADS were revised and used to suit the requirements of ORS. The modifications are mentioned in (section 5.5). In this section we describe the generation of the CommonKADS models. A road map to carry out knowledge-oriented Organization and activity analysis as suggested in commonKADS is shown in the figure 22. The worksheets in the commonKADS model have predefined structures. Developing these worksheets generates the commonKADS models. 46 Start Organization Model 1: Worksheet Problems, Solutions, context I Organization Model 2: Worksheet Description of organizational focus refine Organization Model 3: Worksheet Process breakdown Integratq • refine Organization Model 4: Worksheet Knowledge assets Integrate - J Organization Model 5: Worksheet Judge feasibility (if infeasible) Stop •# Task Model 1 Worksheet Task analysis Task Model2 Worksheet Knowledge item analysis Agent Model 1: Worksheet Agent model Integrate comparing both the old & new situations " Organization Task Agentl: Worksheet Assess impact & changes (decision document) -•9 Context analysis Figure 22: Framework of knowledge analysis using CommonKADS ready The communication and knowledge model is built on the basis of organization, task and agent models (figure 7). As seen from the figure 22 these models are integrated and developed step by step. If no feasible solution of the business problem at the end of the organizational model is 47 found out then the task model is not developed. Otherwise the task model and the rest of the analysis is continued to develop. Most of the commonKADS worksheets used for analysis where modified slightly, with few being developed completely from scratch. The decision document OrganizationTaskAgent-1 (OTA-1) is the basis for generating knowledge model, which is discussed later. We now analyze each of the worksheet, starting with Organization model (OM-1, OM-2, OM-4 and OM-5) 5.1 Organization Model Structure: Event driven process modeling technique ! Activity analysis "K method Structured interviews 4^ T , Organization.Model (OM) Task Model (TM) Agent Model (AM) Communication Model (CM) Organization, Task and Agent Model (OTA-1) _ I Analysis of selected research applications 1 Knowledge Model (KM) Knowledge Model (KM) - for a sub domain Figure 23: Structure of Analysis (Organization Model) Method: The Organization Model (OM) is derived from activity analysis method and the structured interviews conducted with the actors of the related activities. Organization Model-1 (OM-1) Purpose: to identify knowledge-oriented problems and opportunities in the Organization Organization and task analysis constitute the key elements of the CommonKADS methodology. Organizational aspects often constitute the critical success factor for the introduction of 48 knowledge systems. The structure of OM-1 as described in CommonKADS is mentioned in table 1 below: Organization Model Problems and opportunities Worksheet OM-1 Problems and opportunities Make a shortlist of perceived problems and opportunities, based on interviews, brainstorm and visioning meetings, discussions with managers, etc. Organizational context Indicate in a concise manner key features of the wider organizational context, so as to put the listed opportunities and problems into proper perspective. Important features to consider are- mission, vision, goals of the organization, Important external factors the organization has to deal with, Strategy of the organization, its value chain and the major value drivers. Solutions List possible solutions for the perceived problems and opportunities, as suggested by the interviews and discussions held, and about the features of the organizational context Table 1: Structure of Organization Model-1 The worksheet gives an overview of the organization from a high level. The perceived problems and solutions are anticipated and discussed. Next the key features of the organizational context are identified like mission, strategy, goals etc. Finally the possible solutions of the perceived problems are mentioned. Table 2 describes the organizational model -1 worksheet developed for ORS. Organization Model Problems and opportunities Worksheet OM-1 Problems and opportunities Perceived problems: high turnaround time for processing applications, decision making for applications sometimes inconsistent, process for filling applications cumbersome, large number of applications to be processed by few committee members and office personnel Organizational context The objective of the organization is to ensure compliance with government regulations and granting agency requirements for the use of human subjects in research conducted at U B C To ensure compliance the organization has to maintain consistency, transparency and fairness in enacting all the activities of its organizational processes. Solutions After interviewing the actors of the organizational processes, observing and participating passively in the organizational processes (like attending ER meetings); the possible solutions for the perceived problems are identified. These are streamlining the organizational processes with and without the help of Information Technology, identifying knowledge gaps in each activity by conducting knowledge requirements analysis and suggesting knowledge models to be implemented as knowledge system. Table 2: Organization Model-1 49 Four perceived problems in this case study have been identified. These are: • The turnaround time for processing applications is relatively long • The decisions for processing applications might be inconsistent • The process of submitting applications is cumbersome • A large number of applications is processed by few committee members and ORS personnel The above problems were identified after discussions with the stakeholders of the processes. It is noteworthy that not all these problems may be addressed by developing knowledge models-some of the problems could be solved using other techniques. For example, streamlining the organizational processes with and without the help of Information Technology may help to address the first (partly), third and fourth problems. Knowledge requirements analysis and developing knowledge model may solve the second problem. Organization Model-2 (OM-2) Purpose: to identify structure, process, people, resources and knowledge of the organization The OM-2 worksheet explains what important components of the organization should be considered. Table 3 summarizes the worksheet OM-2 developed on the context of ORS. Organizational Model Variant aspect worksheet OM-2 STRUCTURE Organizational structure- refer organizational chart- figure-24 PROCESS Described in details in Organization Model -3 (OM-3) PEOPLE Personnel who are involved in the activities of the Organizational processes. They are the Director, Manager, Chair, ER member, ER Assistant, Researchers RESOURCES Information Systems consisting database of applications on oracle, support documents-policies and guidelines, website, physical applications filed and stored KNOWLEDGE Described in details in OM-4 on knowledge assets CULTURE & POWER The organization has a congenial working environment. The employees are friendly, affable and responsive. Customer calls and visits are treated promptly & with respect Table 3 Organization Model-2 50 Office of R e s e a r c h Serv ices Director M a n a g e r A n i m a l Care I M a n a g e r Ethics Behav ior C o m m i t t e e m e m b e r s A n i m a l Ca^e Commit tee m e m b e r s Ethics Behav ior j ER Ass is tant 1 A n i m a l Ca re & Ethics Behav ior ER Ass is tant 2 A n i m a l Care & Ethics Behav io r Figure 24:Organizational structure of ORS The organization structure of ORS is drawn in figure 24. A single box represents the committee members, but there are fourteen committee members associated with the processing of applications for Ethics Review. They are shown as associate of the Manager though they do not belong to the organization directly. The dotted line represents that the ER committee works as co-workers of the Manager and do not report to her. There is a link that has been established between the ER Assistants as they both work interchangeably for Animal Care review and Ethics Review. The actors of the processes are identified. They are the Director, Manager (Ethics Behavior), Chair, ER committee, ER Assistant and Researcher. The Chair is also part of the committee. The resources are also identified in the processes and discussed in Organization Model-3 (OM-3). Process Analysis of the Organizat ion The rigor of analyzing the business processes makes the task of constructing the knowledge model easy. CommonKADS emphasizes analysis of business processes for improvement of 51 efficiency. Therefore before we proceed to generate worksheet OM-3 for activity analysis, we analyze the business processes in details using the interviews conducted with the stakeholders (structured questionnaire appendix- A). The analysis of the business process will not only help to identify processes that could be improved with and without the help of Information technology, but also help to identify the activities which have significant knowledge components in them. To manage the amount of information attained in the interviews, in the analysis below we examine three indicators of activities: first, we identify the activities that were perceived as 'important' or 'highly important' by participants. Second, we summarize the scope of potential improvements in each activity (with or without the aid of Information Technology) and third, we identify an alternative ways to perform activities. Within each of these sections we examine three groups of processes: • Processing of new applications. • Processing of approved applications. • Handling queries. Another set of questions related to knowledge was asked to each subject. The topics related to knowledge are: • Identification of available knowledge for each activity • Knowledge required to do an activity effectively and efficiently • Method to share the knowledge involved to conduct each activity The information collected regarding knowledge related to each activity is used to identify activities which are knowledge intensive and also for analysis at later stages of developing knowledge model. 52 Process analysis by importance Generally, the subjects perceived all activities as important, which means that there are no redundant activities in the ER process. The most important activities noted for the 'processing of new applications' are mentioned in table 4. The process diagrams in Appendix D show all the processes and the activities. Activity # Activity Actor(s) 1.1 Stamp Date ER Assistant 1.11 Study Applications Committee member, Chair 1.12 Review applications Committee member, Chair 1.13 Summarize and enter information ER Assistant 1.15a Verify complete application Manager 1.16 Sent certificate of approval/memo ER Assistant 1.17 Receive and enter comments Manager 1.2 Check Applications ER Assistant 1.21 Receive amendments ER Assistant 1.22 Mail notice to PI ER Assistant 1.3 Enter data ER Assistant 1.9 Send documents to committee ER Assistant 1.20 Enter amendments ER Assistant 1.10 Receive applications Committee member, Chair 1.19 Review amendments ER Assistant, Chair Table 4: Activities by importance ranking for processing of new applications Most important activities in the context of 'processing of approved applications for amendments/renewals' are given in table 23 in Appendix C. Finally, none of the activities for 'processing of queries' were classified as highly important. In other words, these activities are not perceived as critical. We now turn to examine the proposed improvements in each activity within the three groups of processes. It is interesting to see that the proposed changes in the critical activities - described in the tables - are all IT supported, as our analysis shows. 53 Process analysis by 'possible improvements' A significant number of activities can be potentially improved. The improvements are classified into two types - improvements with the aid of Information Technology (IT) and without the aid of Information technology. A significant number of activities could be improved with the aid of IT. We begin by examining the non-IT improvements within each group of activities and follow with a similar analysis for the IT based improvements. Activities that could be improved without the aid of IT For 'processing of new app ications:' No. Activity Current situation Possible improvements 1.7 Prepare agenda, summary of applications for meeting At present these documents are prepared by the Manager and the ER assistant assists to collate these documents to send to the members and chair The documents could be prepared by ER Assistant instead of the Manager reducing the workload of the manager 1.12 Review applications There is no limit on the number of applications that are send to the committee members for review. If some of the members are absent in the meeting the applications are reallocated to the rest of the members. The classification between expedited and non-expedited applications not enforced at present Limit the number of proposals to be reviewed per primary reviewers. The classification of applications (Expedited and non-expedited applications) as created by the Ethics Manager should be implemented as early as possible to reduce the workload of the committee members 1.2 Check Applications When the applications are received by ORS. There is no check on the completeness of the applications. They are sent directly to the committee members In this activity the basic mistakes (like consent form not on letter head, page no of consent form missing, etc.) should be checked and errors should be captured when the applications are received by ORS. A check list to be prepared for ER Assistant to go through the applications to check errors before submitted to reviewers Table 5: Activities to be improved without the aid of IT for processing of new applications Table 5 refers the activities that could be improved without the aid of IT for processing new applications. Several reviewers on interviews suggested improving activity 1.2 - 'Check applications', as this activity directly affects the activities where the reviewers are involved (1.9, 1.10, 1.11 and 1.12). The reviewers felt that applications should be checked for basic errors 54 before they receive them. At present all applications that are received are forwarded to the committee without undergoing a basic check on the completeness of the application. A checklist can be provided to the ER Assistant to examine each application for basic errors. The ER Assistant can communicate these errors immediately to the researcher saving considerable amount of time for reviewing and processing the application. A l l the processes suggested for improvements for 'processing of approved applications for renewals/amendments' and 'processes for query handling' are related to the aid of IT and therefore discussed next. Activities that could be improved with the aid of IT -For 'processing of new applications' Table 6 refers the activities that could be improved with the aid of IT for processing new applications. No. Activity Possible improvements Send electronically Receive electronically Other method 1.20 Enter amendments • 1.2 Receive amendments • 1.2 Sent certificate of approval/memo • 1.2 Check pending grants/Teaching Hospital application Send email 1.5 Prepare 20 copies • 1.3 Enter data • 1.1 Stamp Date • 1.10 Receive applications • 1.15a Verify complete application Automatic checking of documents for errors (like missing page no., etc) 1.1 Collect and enter data • 1.2 Mail notice to Principal Investigator Send email 1.2 Receive and enter comments Send email Table 6: Activities to be improved with the aid of IT for processing of new applications 5 5 The improvements as mentioned in the table are broadly of two types - first automating the process (inputs and outputs should be done electronically), second communicating by email. Automating the activity can be divided into two types, sending data electronically and receiving data electronically. For example, activity 1.15a 'Verify complete application' can be improved by automatically checking the application for completeness. The checking can be partially automated and the rest of the checking has to be done by Ethics Manager manually. Examples for electronic checking are: finding signature, date and page numbers not filled or mentioned by the researchers on the applications. The final suggestion for process improvement is signing the certificate electronically by Chair. The Chair would grant permission to send certificates to the PI by email in consultation with the Manager. This activity would thus reduce the turnaround time for processing of applications, as the ORS is dependent on campus mail to send the certificate of approvals to PI. It is interesting to note that most of the activities that can be automated are of very high importance. The details of the activities that could be improved with the aid of TT for processing approved applications and queries are mentioned in tables 24 and 25 in Appendix C. Most of the activities in the process- processing of queries can be done electronically if the database is accessible to i the researchers. Improvements of the present website of ORS like adding a web page on Frequently Asked Questions (FAQ's), creating an index page, more information hyper linked, would reduce the number of queries received by the ORS office. Suggestions for alternate activities The alternate activities that were suggested in interviews are closely related to the possible improvements of the activities. Alternate activities for 'processing of new applications:' The details of the alternative activities are mentioned in table 7. 56 No Activity Present practice Alternate activity 1.10 Receive applications The hard copies of all applications are sent to each committee member Assigned applications are received as hard copies and the rest are available on web. Enough copies of all the applications are available during the ER meeting 1.1 Stamp Date The date stamp is put on applications when received Electronically receipt of application and supporting documents 1.21 Receive amendments Amendments are received in hard copies Sent the amendments to ORS office electronically 1.5 Prepare 20 copies Deferred applications are received in hard copies Submit the deferred applications electronically 1.3 Enter data Only first eight fields of the applications are entered manually in the database The entire application is stored electronically in the database. Meet/speak to Principal Investigator (new activity) When the Principal investigator failed to address the ethical issues second time, the Chair sets a meeting with him/her and address the issues face to face or over telephone Table 7: Alternative activities suggested for processing new applications This set of alternative activities was generated as the actors were not satisfied with the present practice. The most important alternate activity suggested is of activity 1.10 'Reviewers receiving applications'. This activity helps the reviewers to receive less paper applications and more importantly the reviewers need not carry a box of review applications to the review meeting. In the alternate activity, the reviewers carry the applications to the meeting where they are the principal investigators. By submitting applications online, some of the activities become redundant. For example, activity 1.1-'stamp date', activity 1.21-''receive amendments', activity 1.5- 'prepare 20 copies' and activity 1.3- 'enter data', become unnecessary and therefore the alternative activities are doing them electronically. The final suggestion is an introduction to a new activity (after the activity 1.19). This activity was suggested as the principal investigator failed to understand the changes suggested to him/her by the committee and therefore would not. address those issues in the subsequent amendments or deferred applications. The alternative activities for 'processing of approved applications for amendments/renewals and 'handling queries are mentioned in table 26 and 27 in Appendix C. 57 The above section completes the first part of the analysis. We now analyze the activities related to knowledge. Knowledge-intensive activities CommonKADS define knowledge-intensive activities as those that require significant amount of knowledge in order to accomplish them. CommonKADS does not suggest any method to determine knowledge intensive activities rather suggest ranking knowledge intensive activities (mentioned in next section). The knowledge in these activities is generally resources like experience, rules or documents. An example of a knowledge intensive activity could be 'taking decisions on applications'. The act of decision making itself could be complex and require knowledge in various forms (electronic, paper based, experience, etc) in order to process it. An example of non-knowledge intensive activity could be 'filing of applications'. This activity does not require many resources to accomplish. We suggest a criterion to distinguish between knowledge-intensive and non knowledge-intensive activity. The outcome of a knowledge intensive activity is not always predictable or definite but in most cases the outcome of the non knowledge-intensive activity is already known or can be predicted easily. In most cases the non knowledge-intensive activities are routine activities done quite frequently. Referring to the process diagrams and activities (Appendix 4) we find that activities like 'stamp date' (1.1), 'prepare 20 copies' (1.5) or 'store applications till deadline' (1.4) can be considered to be non knowledge intensive activities. The outcomes of all these activities are predictable and these activities do not require any significant knowledge to accomplish them. We identify the knowledge intensive activities by analyzing the activity sheets that are created for each activity using activity analysis method (section 3.4). Sample of two activity sheets are mentioned in Appendix A. The activities where the subjects failed to identify or mention 58 knowledge resources involved in the activity and knowledge required to do the activity effectively and efficiently were considered to be non knowledge intensive activities. Most of the activities of ORS in all the processes fall into this category. In other activities the subjects were able to identify at least some knowledge resources involved in the activity and hence classified as knowledge intensive. The following knowledge intensive activities were identified in all the three processes and generated from the activity sheets. Activity # Description Actor (s) 1.2 Check correctness of applications ER Assistant 1.11 Study Applications (before the meeting) Chair, Committee members 1.12 Review applications Chair, Committee members 1.13 Summarize and enter information Manager 1.15a Verify complete application Manager 2.2 Review Amendments Chair, Manager 3.4 Refer guideline/policy (for query process) ER Assistant, Manager Table 8: Knowledge intensive activities It is interesting to note that all knowledge intensive activities are the most important activities in the processes. These activities become the cynosure of our study and we will ignore studying the non knowledge intensive activities for our analysis henceforth. In the next section we continue to build the Organization model with OM-3 Organizational Model-3 (OM-3) Purpose: to identify the knowledge assets of each knowledge intensive activity along with the actors of the activity Activity sheets of the knowledge intensive activities are used to generate organizational model-3. CommonKADS suggests ranking these activities in order of significance to understand the 59 knowledge intensiveness. There is no definite rule defined for ranking the activities, but the criteria for ranking are based on costs involved, frequency, resources and criticality of the activities. We developed a composite score for the activities based on the above criteria (mechanism mentioned in Annexure C). Table 28 in Appendix C mentions all the knowledge intensive activities with the scores for OM-3 for ORS. The summarized table 9 for OM-3 identifies the knowledge asset and actors for each knowledge intensive activity below: No. Activity Knowledge Asset Actors 1.2 Check correctness of applications Guidelines to fill the applications ER Assistant 1.11 Study Applications (before the meeting) Manual on tri council policy, other documents and books on ethics, note book on ethics supplied by ORS (guidance notes), previous minutes of the meeting, guidelines on policies, research experience (teaching research courses & policies), research experience (conducting related to human subjects) Committee members, Chair 1.12 Review applications Tri-council policies and other guidelines, Committee members (including lawyer), Manager, Experience of handling applications Committee members, Chair 1.13 Summarize and enter information Guidelines and experience Manager 1.15a Verify complete application Familiarity with the guidelines, institutional memory (18 years experience in handling ethics applications) Manager 2.2 Review Amendments Ethics related research experience Manager, Chair 3.4 Refer guideline/policy Existing knowledge about guidelines and policy Manager, ER Assistant Table 9:Knowledge assets for Organization Model-3 We identified the two most knowledge intensive activities in the processes based on the overall scores they received (4 & 5 out of 5). These are activity 1.11 (study applications) and 1.12 (review applications). The high scores of these activities can be justified, as without these activities the processing of new applications will not be possible. The knowledge assets for both these activities are varied (experience, paper based) and more than other activities. 6 0 Organization Model 4 (OM-4) Purpose: to classify knowledge assets and analyze them The perspective taken here is the knowledge pieces are significant as assets and actively used by people in the organization. An important issue in this part of the study is to single out dimensions in which knowledge assets may be improved, informed and accessed in time, space or in quality. The organization model-4 in context to ORS is mentioned in table 10: Knowledge Asset Possessed by Used in activity Right Form? Right Place? Right Quality? Guidelines to fill the applications ER Assistant 1.2 No, more intensive guideline required No No, requires a new set of document from the existing resource 1. Documents (policies, minutes of meeting, books related to research) 2. research experience (teaching, conducting) Chair, Committee members 1.11 1. No, the knowledge is too general and should be more focused, 2. No, requires conversion from tacit knowledge to explicit knowledge No (in both cases) 1. No, Quality to be improved by creating documents extracting from the present documents 2. No 1. Documents (policies, manuals) 2.. Research experience (conducting and teaching) Chair, Committee members 1.12 1. No, the knowledge is not specific, 2- No, requires conversion from tacit knowledge to explicit knowledge No (in both cases) 1. No, specific knowledge to be developed which can be documented 1. Documents (guidelines) 2. Experience (handling ethics applications) Manager 1.13 1. No, the knowledge is not specific, 2- No, requires conversion from tacit to explicit knowledge No (in both cases) l . N o , 2. No Familiarity with the guidelines, experience in handling ethics applications Manager 1.15a No, more intensive guideline required No No Ethics related research experience Chair, Manager 2.2 Yes Yes Yes Existing knowledge about guidelines and policy ER Assistant, Manager 3.4 No, needs to be more intensive No No Table 10: Worksheet Organization Model-4 There are two categories of knowledge assets generally used in the activities mentioned above. First, paper based knowledge. Generally this knowledge is available in the form of documents, books or in some other paper based formats. Second category of knowledge is ethics experience. 61 The experience is classified into two types, research based experience (experience obtained by conducting ethics based research) and teaching based experience (by teaching ethics related courses). Incidentally this classification matches with the classification of knowledge into explicit and tacit type (Nonaka, and Takeuchi, 1995). The first category (paper based knowledge) is explicit knowledge and second category is tacit knowledge (experience). The forms of the knowledge in most of the knowledge assets have the scope of improvements. The paper-based knowledge is considered to be not in right form as the knowledge available in this form is too general and not focused. The experience-based knowledge can be captured into a knowledge system and thus be converted to explicit from tacit to some extent. Organization Model -5 (OM-5) Purpose: to conduct a brief feasibility analysis of the possible solution suggested This worksheet is a decision document and checks whether the possible solution of the problem mentioned in organization model-1 (identifying knowledge problems in the organization) is possible to implement or not. The decision for implementing the solution will depend on three criteria: • Opportunity area for applications and direction for best solution. • Cost benefit analysis (business feasibility) • Availability of technology for solutions (technical feasibility) At this stage of the analysis, we tentatively suggest creating a knowledge system to alleviate the problem of inconsistency of decision-making for applications. We believe that developing a knowledge system could be a potential solution to this problem. We anticipate that the knowledge-system will classify the previous decisions in some categories and these categories will be available for the decision makers to refine, modify and use for the future decisions they 62 take. The exact mechanism and components of the knowledge system will be discussed once the knowledge model is developed of ORS. Table 11 below describes the Organization Model-5 Organizational Model Checklist for feasibility decision document: Worksheet OM-5 Business feasibility Problem: decision making for applications inconsistent Possible solution: Use of IT to create a knowledge-based system The expected benefits for the solution are committee members would be able to make more consistent decisions for each application. The expected costs have to be calculated based on the suggested knowledge based system to be developed. There is no organizational changes required There are no economical or business risks involved Technical feasibility In order to perform the activities, the knowledge system has to be developed. The system would provide support to the reviewers to make more consistent decision. The system has to be integrated with the existing database. The technology is available to create this knowledge system. Table 11: Organization Model-5 5.2 Task Model (TM) Structure: Event driven process modeling technique Activity analysis method Structured interviews Agent Model (AM) Communication Model (CM) Organization, Task and Agent Model (OTA-1) I Analysis of selected research applications Knowledge Model (KM) 1 Knowledge Model (KM) - for a sub domain Figure 25: Structure of Analysis (Task Model) Method: The Task Model (TM) is derived from activity analysis method and some components of task models are derived from the Organization model. The first task model (TM-1) is directly 63 generated from the activity sheets and used instead of TM -1 worksheet as prescribed by commonKADS Task Model -1 (TM-1) Purpose: to analyze each knowledge intensive activity The knowledge intensive activities are analyzed on the following criteria: Purpose, Input, Output, Actors, Resources, Conditions for triggering, Operations), Roles, Rules and Description. The information for each of these activities is collected from the activity sheets mentioned in Appendix B (activity sheet 1.11 and activity sheet 1.12). Below tables together analyzes activity 1.12 (review applications). Purpose Actors Input Output Conditions for triggering To review the applications Members, Chair, Manager Applications Reviewed applications When other documents for meeting are prepared Table 12: Analysis of activity 1.12 (review applications)- part A Operation(s) Roles Resources (data & knowledge) Rules Description Primary reviewers review applications by making comments and other reviewers make additional comments Reviewers Guideline notes, research experience, Tri council policy Summarize conclusion Chair Take notes for each application reviewed Manager Case law from 18 years experience, guidelines Table 13: Analysis of activity 1.12 (review applications)- part B It is interesting to note that one activity has three operations and roles for each of these operations are different and the actors of the operations use different resources. Task Model-2 (TM-2) Purpose: to analyze each knowledge item identified in the knowledge intensive activities 64 The items of knowledge and competency are key items in the task model and for this reason it is again modeled in a separate worksheet Task Model-2. Task Model-2 has a highly important function, since it concentrates in details on bottlenecks and improvements relating to specific areas of knowledge. Each knowledge item identified in activity 1.11 (study applications-table-29 on appendix C) and 1.12 (review applications-table 30 on appendix C) are analyzed using the worksheet Task Model-2. We analyze knowledge items in three dimensions: nature, form and availability. CommonKADS defines several criteria of nature, form and availability. Each of these criteria is mentioned in table 14. The knowledge items for the activity 1.11 (study applications) are -• Manual on tri council policy (an Ethics guideline prepared by Government of Canada) • Other documents and books on ethics • Note book on ethics supplied by ORS (guidance notes) • Minutes of the latest ER meeting • Guidelines on policies • Research experience (teaching research courses & policies related to Ethics at UBC) • Research experience (conducting Ethics related research at UBC) The knowledge items for activity 1.12 (review applications) are -• Tri-council policies and other guidelines • Experience of handling applications We further combined these knowledge items for Task Model-2 based on common items: • Manual for tri council policy • Literature related to ethics (books and documents) • Guidance and policy notes 65 • Minutes of the latest ER meeting • Experience (teaching and research) • Experience of handling applications. Table 14 analyzes the knowledge item-experience. Task Model Knowledge Item Worksheet Task Model-2 Item Experience (teaching and research) Nature of the knowledge Bottleneck/ to be improved Formal, rigorous Empirical Quantitative Heuristics, rule of thumb Highly-specialized, domain-specific Experience-based V need to share the experience to other members Action-based Incomplete Uncertain, may be incorrect Quickly changing V research skills are changing, need to capture those skills Hard to verify Tacit, hard to transfer difficult to transfer to explicit knowledge Form of the knowledge Mind V need to share the knowledge in other form like paper or electronic Paper Electronic Action Skill V research skills learnt by individual experience Other Availability of knowledge Limitation in time Limitation in space Limitation in access V cannot be easily shared Limitation in quality Limitation in form V need to transfer to other forms Table 14: Task Model-2 Knowledge item analysis of 'experience' The bottlenecks and possible improvements on categories of knowledge are mentioned in the table. The analysis of rest of the knowledge items of the two activities is done on the same 66 format (Appendix C- table 31, 32 and 33). Some of the tables have two knowledge items analyzed together. Based on the knowledge items analysis, we summarize the deficiencies in the table below: Knowledge item Deficiency Manual of tri-council policy Too formal & detailed, not available in electronic form. Summaries and highlighted issues should also be available to the committee members Policy guidance Needs regular update and be available in electronic form Minutes of last ER meeting Available only in paper, not linked to earlier minutes. Decisions of earlier meetings not known Experience (research & teaching) Tacit knowledge type, knowledge is not shared with other committee members Experience (handling applications) Knowledge is tacit type and not shared with other ORS personnel Ethics literature Sources of the Knowledge are generally not known or reliable. Too general in form Table 15: Knowledge items- deficiencies Thus we find that there are deficiencies in each of the knowledge items. This gives an opportunity to improve the knowledge items. An important knowledge item- 'minutes of last ER meeting' has brief descriptions and decisions of the applications of the last concluded ER meeting. This information is available to the committee members in a discrete form (paper) and all the earlier decisions of the applications are not linked with the minutes (only the decisions of the last concluded meeting are available to the committee members). This discrete piece of information is not very useful to the committee members as it is paper based and secondly the decisions of applications are not categorized. Thus we foresee the need of a knowledge system where we can capture the decisions of all the previous applications. The categories of the decisions have to be created in this system and have to be in electronic form so that the committee members can access and share the decisions. Each new decision also becomes the part of the knowledge system. 67 5.3 Agent Model (AM) Structure: Event driven process i ! Activity analysis i ! Structured interviews modeling technique • ^ method i ! Analysis of selected research aDDlications Knowledge Model (KM) Knowledge Model (KM) - for a sub domain Figure 26: Structure of Analysis (Agent Model) Method: The Agent Model (AM) is derived from activity analysis method Purpose: to understand the roles and competencies that various actors (stakeholders) of the organization bring with them to perform a shared activity. In the context of knowledge, CommonKADS suggests three kinds of stakeholders in the organization, they are: knowledge providers, knowledge users and knowledge-decision makers. The agent model reorganizes the knowledge so that we can look at it from the perspective of the agents (actors) involved in the organization. CommonKADS suggests mentioning knowledge, competencies, and responsibilities for each agent in the agent model worksheet. We modify the Agent Model worksheet by integrating it with the classification of actors and making it a more compact table. Table 16 summarizes the modified Agent Model in the context of ORS. 68 Knowledge providers Knowledge users Knowledge-decision Maker Definition The specialists or experts in whom the knowledge of a certain area resides The people that need to use this knowledge to carry out their work successfully The persons that make decisions that affect the work of either the knowledge provider or the users Stakeholders Chair, ER committee, Manager Manager, ER Assistants Chair, ER committee Rationale Each of the stakeholder has expertise in his/her area Use the knowledge of Chair and ER committee to their work to process applications where decisions are made Take decisions on applications Knowledge Chair-expertise of conducting Ethics related research and summarize/conclude decisions, ER committee-expertise in Ethics research, Manager- expertise in processing ethics applications expertise in processing ethics applications Chair-expertise of conducting Ethics related research and summarize/conclude decisions, ER committee-expertise in Ethics research Table 16: Agent Model of ORS The next step is to integrate all the CommonKADS models developed so far into a decision document (Organization Task Agent Model) and set proposed actions. The proposed actions are the result of two sets of documents - first, observing the shortcomings of the knowledge items in Task Model-2 (table 15) and second a set of information generated while interviewing the actors of the Organization (Questionnaire in Appendix B). The information consists of knowledge availability, knowledge required to do the activity efficiently and effectively and methods to share knowledge for the activities 1.11 (study applications) and 1.12 (review applications) (table 29, 30 appendix C). A summary of this information is given on the below tables: 69 K-Available K-Required K-Sharing methods • Manual on tri council • Summary of highlighted • Training policy & important points of • On the job learning • Other documents and tri-council policies • Attending ethics related books on ethics • 1-day orientation for seminars (share your • Note book on ethics new ethics members ideas with ethics supplied by ORS (Review process committee of other (guidance notes) familiarization, universities • Previous minutes of the documents related to • Learning by observing meeting ethics on research) other ethics members • Guidelines on policies • Regular updates on • Research experience changes in review (teaching research processes courses & policies) • Recommended reading • Research experience for ethics reviewing (conducting related to (underlying principles, human subjects) challenges, current policies, privacies, etc) Table 17: Knowledge analysis for activity 1.11 (study applications) by structured interview K-Available K-Required K-Sharing methods • Tri-council policies and other guidelines • Committee members (including lawyer) • Manager • Experience of handling applications • Condensed Ethics resources • List of the main ethical issues (like longitudinal data, process of conducting research with school boards, etc) and an example of each of those issues and the record on what decisions where made. • Precedence setup by the committee like what is accepted ethics behavior and what is not accepted. • Manual of decision making, past record of decision making, principles on underlying decisions or exceptions on decisions. • Case law— develops a set of principles for each important ethical issue; attach a list of protocols relevant to that issue and the decisions on those protocols. • Documents on earlier decisions • Learning by observation • Orientation Table 18: Knowledge analysis for activity 1.12 (review applications) by structured interview Integrating Organization, Task and Agent models Organization Task Agent Model (OTA-1) Structure: , r 1 r ! Structured interviews Event driven process modeling technique ! Activity analysis method I Knowledge Model (KM) Organization Model (OM) Task Model (TM) Agent Model (AM) Communication Model (CM) Oigani/aiiiin, Task and Agent Model (OTA-1) Analysis of selected research applications 1 Knowledge Model (KM) - for a sub domain Method: This model is generated from Organization, Task and Agent models. Structured interviews are also used to generate the worksheet for this model Purpose: to bring to the surface improvements through the newly identified knowledge items that lead to better use of knowledge in the organization than before. This worksheet is a document for managerial decision-making about changes and improvements necessary to the organization. The main items of this worksheet are the proposed actions, impacts and changes in the organization due to the proposed actions. The proposed knowledge based actions are for the knowledge problems identified earlier (in Organization Model-5). The proposed solution is a set of knowledge items. These knowledge items are new or improved versions from the previous knowledge items identified in this activity. They match with the analysis of the required knowledge to do this activity efficiently and effectively (mentioned in the previous section). The future direction of the knowledge project can be drawn from the sheet Organization Task Agent-1. There are two proposed solutions that are identified, first for the 71 activity 1.11 (read applications) and second for the activity 1.12 (review applications) and are described in two separate tables (19, 20). Organization al task, Agent models Worksheet- OTA-1: Checklist for Impact and Improvement Decision Document Proposed actions The proposed actions are: 1. A new document to be prepared containing the following contents: a. Understanding review process b. Underlying fundamental principles of reviewing c. Standards and challenges d. Main ethical concerns (with examples) e. Accepted and non-accepted ethical behavior 2. 1-day orientation for new Ethics member a. Process familiarization b. Referring to the documents mentioned above c. Sharing of experiences with existing reviewers 3. A supporting document has to be prepared with the following contents: a. Summary and highlighted issues of tri-council policies and other guidelines b. Trends and current practices of ethics review (examples- changes in research methodologies related to ethics) Impacts and changes in organization For this activity, the proposed knowledge system would require changes in resources and knowledge. There would be no changes in people, processes and structure of the organization. The new resources are identified and would be created with the help of existing resources. A knowledge systems would be created which would capture the experience of actors of the activities Activity specific changes The proposed changes would not change the way the activity is done. The change lie in resources and knowledge Attitudes and commitments The proposed changes would help the actors involved in the activities to conduct the activities better and efficiently. The first proposed change in the resources available to the committee members- process familiarization documents, would help the members to understand more about the task (What is the task? and how the task is done?) It is also proposed to incorporate orientation to each new member, thus the members get familiarized to the activities and the tasks involved in the job. The changes fill the gaps that the members had at present of not understanding the task and methodology to accomplish the task effectively. Therefore it is foreseen that the members would accept the recommendations and find them useful Table 19: Organization Task Agent Model-1 for activity 1.11 (study applications) 72 Creating these documents becomes the part of the solution for inconsistency in decision-making. After creating these knowledge items the reviewers have access to rich knowledge to perform this activity efficiently and effectively. Organizational task, Agent models Worksheet- OTA-1: Checklist for Impact and Improvement Decision Document Proposed actions The proposed actions are: 1. To prepare a decision document containing: 1.1. List of main ethical issues (with examples) 1.2. Decisions of these issues 2. To prepare a knowledge system to capture decision making with the following characteristics: 2.1. To be built on underlying principles and exceptions on decisions 2.2. Past records of decision making will be captured in the system 2.3. Would be able to link each decisions of research application with the underlying principles, the underlying principles become standard principles. Impacts and changes in organization The knowledge based system will be available in electronic form and therefore shared by the users (Ethics members) The knowledge system has to be integrated with the existing database of ethics applications of the organization. Activity specific changes The proposed changes will not change the way the activity is done, but there would be another operation added in the activity that is "check decision database for decision on similar application" The proposed change would bring consistency on the decision-making process. Attitudes and commitments The decision document would be very useful in a review meeting as the members can refer to it to discuss any ethical concern. The electronic decision system would add a very small amount of time to review the application as the member would have to access the system to see the past decisions. But after usage of this decision system, the members will find it very useful and supportive to refer while reviewing application. Table 20: Organization Task Agent Model-1 for activity 1.12 (review applications) The knowledge based solution for this activity is to capture decisions of applications. The applications will be classified into categories and decisions regarding each category would be found out. The system would define rules for classifying applications and decisions of the applications. The main contribution of this worksheet is identifications of new and improved knowledge items for the two knowledge intensive activities, presented in this worksheet as proposed actions. 73 5.4 The Knowledge Model Structure: Event driven process modeling technique Activity analysis method Structured interviews Organization Model (OM) | | Task Model (TM) | | Agent Model (AM) uion, lask ana ^ I i Analysis lodel (OTA-1) ! research 1=^ .-^ Knowledge Model (KM) Knowledge Model (KM) v/,- : • . I - for a sub domain Figure 27: Structure of Analysis (Knowledge Model) Method: The knowledge model is built from the improved or new knowledge items mentioned in Organization, Task and Agent model-1. The knowledge item that can potentially be developed to a knowledge system is selected. No worksheets are used here and a specific method is followed to generate knowledge model as suggested in CommonKADS (figure 28). Purpose: to construct an actual knowledge model Knowledge Model Construction We continue to study the possibility of developing a knowledge model for the knowledge system for activity 1.12 (review applications). We name this system as-knowledge system to capture decisions. 74 CommonKADS gives a structured approach to develop the knowledge model (figure 28) The arrows represent the stages to be done in order. The knowledge refinement stage is omitted from this model, as it requires validating and refining the model after implementing it. Stages Knowledge identification Typical activities -domain familiarization -list potential model components for reuse Knowledge specification -choose task template -construct initial domain conceptualization -complete knowledge-model specification Knowledge refinement -validate knowledge model -knowledge based refinement Figure 28: Structure of Knowledge model We develop the first two stages to build the knowledge model. Knowledge Identification phase Domain familiarization The starting point for this activity is the list of knowledge items described in the worksheet Task Model-2 (table 14, 31, 32 and 33). Actors related to this activity should study and become familiar with these items. In this case the domain is diverse in nature (ethical issues of different research areas). The knowledge items developed would help to understand and familiarize 75 oneself within the domain. These knowledge items can serve the purpose of a guideline and act as a central source of information. Knowledge Identification phase: List potential Model/components for reuse In this stage the potential knowledge components are identified. A task template has to be chosen for this model that fits in a domain. Task Templates Task Templates form a type of a reusable combination of model elements. A task template is a partial knowledge model in which inference and task knowledge are specified. A task template supplies the user with inference and tasks that are typical for solving a particular problem. Task Types CommonKADS has adopted and refined task types from cognitive psychology literature for use in knowledge engineering. The task hierarchy as used in commonKADS is shown in figure 29. Knowledge-intensive task analytic task classification synthetic task prediction design assessment monitoring assignment modeling scheduling Figure 29: Hierarchy of Knowledge intensive tasks Each box represents a task; the arrows pointing out to the tasks are the main tasks. The tasks that appear on the lower part of the diagram are subtypes of the tasks mentioned just above them. Two groups of task types- analytic task and synthetic tasks are differentiated. The distinguishing 76 feature between the two groups is the "system" the task operates on. A l l analytic tasks take as input some data about the system, and produce some characterization of the system as output. In contrast, for synthetic tasks the system does not yet exist, the purpose of the task is to construct a system description. Analytic and synthetic tasks are further subdivided into a number of task types. Analytic tasks are classification, assessment, diagnosis, monitoring, and prediction. For example, classification is of analytic task type. The object features are inputs and object classes are outputs. Synthetic tasks are design, modeling, planning, scheduling, and assignment. An example of the synthetic task type is planning. The inputs are goals and requirements and output is action plan. We can use the assessment class form the task template as a potential model. Knowledge Specification phase Choose task template This is an important phase of constructing the knowledge model, as a task template has to be chosen. Several features of the application task can be important in choosing an appropriate task template: • The nature of the output (the "solution"): e.g., a fault category, a decision category. • The nature of the input: what kinds of data are available for solving the Problem? Out of several task templates the assessment task is chosen for ORS. This template is chosen from many task templates defined by CommonKADS. The idea of using pre defined task template is to reduce the amount of time to define certain tasks that have been already well defined for some other contexts. Assessment is a sub-class of analysis task. The assessment task as an input uses certain decisions and for output gives a decision class. This task rightly fits into the knowledge model that is being developed for the case study. The general characteristics of this task are briefly described below with an example. 77 General characterization of assessment task Goal Find a decision category for a case, based on a set of domain-specific norms. Typical example Decide whether a person gets a loan she applied for. Terminology Case: the case to be assessed, e.g., data about the lender and the requested loan. Decision category: e.g., eligible-for-loan yes or no Norms: domain knowledge that is used in making the decision, e.g., rules relating income to the amount requested. Input Data about the case Output A decision category (eligible or not). The task types are associated with inference structures. Inference structures are graphical representations of set of inferences. The graphical conventions used to draw inference structure are mentioned in the Appendix C The inference structure of the assessment task is mentioned in figure 34 in Appendix C. It begins with an analysis of individual cases (e.g., specific research applications) that are abstracted to create a representative abstracted case. The abstracted case possesses the similar aspects of individual cases. From the abstracted case norms are developed. A norm is a model of what should exist or be followed for such cases. For the ER process, a norm could be a set of rules on what how to review a specific category of research applications. At the next step, for evaluating a specific case, a norm is selected from group of norms based on the characteristics of the case whose decision is made. An abstracted case with the help of the norm is used to evaluate a case. The result of this evaluation is a norm value. For example a value can be zero -meaning 'reject' or one - meaning 'accept'. In the case of this study, the norm values can be set as research application is accepted (1), proviso is issued (2) and the application is deferred (3). (There is no rejection of research applications). The norm value obtained is matched with the predefined norm values to obtain a decision as final outcome. We 78 modified the knowledge model described above to match the specific characteristics of the ER process. The modified inference structure is drawn in figure 30. Cases decision Figure 30: Inference structure of the modified assessment task Specification activity: Construct initial domain schema The goal of this activity is to construct an initial data model (representation of data) of the task method that is chosen. The initial domain schema consists of two parts: Domain-specific conceptualizations These are the domain structures that are recognized directly in a domain, and likely to be present in the in any application independent of the way in which it is being use. Method-specific conceptualizations A second set of domain structure is introduced because these are needed to solved a certain problem in a certain way. The domain structure (domain related) in this case consists of - research applications, researchers and decisions. Together they represent the domain. The second set of domain structure (method related) consists of criteria for decisions. There is no clear criterion defined 79 for decisions of research applications. In section 6, for a specific set of applications a set of criteria is created. Specification activity: Complete knowledge model specification This activity summarizes the knowledge model that is developed. CommonKADS defines the three elements of knowledge- task, domain and inference. These three elements of knowledge are modified for ORS and presented below: Task knowledge The knowledge associated with the task (called task knowledge) is developed. The task assessment is identified for the model development and steps for the task knowledge are: • Assess individual research application • Abstract the description from applications • Match the application for decisions Domain knowledge From the assessment task template the following domains are identified: case-datum (individual case-research application) and norms. Inference knowledge The Inferences provide the link between the tasks & their methods and the domain schema. In order to realize the assessment tasks the following inferences are identified. These inferences are adapted to our case study. • Abstract case -Most of the case data (research applications) need to be abstracted. Abstraction is modeled here as an inference that is repeated until no more abstraction can be made. The abstracted features are added to the abstracted case. 80 • Specify norm - After abstraction, the first step that needs to be taken is to find the norms or criteria that can be used for this case. In most assessment tasks the norms used are at least partially dependent on the case, thus acts as an input role for this inference. • Select norm - From the set of norms generated by the previous inference, one norm needs to be selected for evaluation. Often the domain knowledge is available that indicates an ordering of norm evaluation. This knowledge can be used to guide selection. • Evaluate norm- Evaluate the selected norm with respect to the case data. Generally this function produces a truth-value for the norm, but for ORS truth-value is not possible as there is no false value (application never rejected). Therefore numerical values are given against the norms (1-accept, 2-proviso issued and 3- application deferred). • Match to see whether a solution can be found This inference checks whether the results of the evaluation lead to a decision. This completes the knowledge model of CommonKADS. The final stage of the knowledge model (knowledge refinement) is not used here, as the knowledge model has to be implemented before refining. It is noted that some of the components of the knowledge model specially the domain & inference knowledge where built incomplete for ORS. The reason for this is the domain of the 'research applications' is vast and therefore it is not possible to identify the norms and rules for the entire domain. An alternate way to create domain knowledge is to decompose the domains into sub domains and create case abstractions and rules for each sub domain. In this case the overall assessment task remains the same but for each sub domain different case abstractions and norms are created. This is done in section 6. A sub domain is selected and using the framework of assessment task the elements of the knowledge model (domain and inference) 81 are created. If the domain and inference knowledge for a sub domain are created properly then the knowledge model can be used successfully in practice. 5.5 Communication model Structure: Event driven process i modeling technique f ! Activity analysis i method Structured interviews Organization Model (OM) | | Task Model (TM) | Agent Model (AM) Analysis of selected research applications 1 Knowledge Model (KM) ^ Knowledge Model (KM) - for a sub domain Figure 31: Structure of Analysis (Communication Model) Method: Communication model is developed from the Organization, Task and Agent model. This model is built together with knowledge model. The knowledge model built is directly used in the communication model. The communication model is developed using - 'event driven process diagramming technique' instead of the graphical conventions used by commonKADS. No worksheet is used to generate this model. Purpose: is to specify the information exchange procedures to realize the knowledge transfer between actors. CommonKADS define a new concept called transaction in this model, which describes the communicative acts between activities and knowledge model. We develop a communication model for the activity 1.12 (review applications). A transaction tells what information objects are 82 exchanged between what actors and what activities. Transactions are the building blocks for the full dialogue between two actors. The knowledge model is also built simultaneously and acts as an actor in this case. The first step of this model is to identify the transactions for this activity. Transactions can be viewed as decomposed activities -operations that interact with a knowledge system. The transactions of the activity 1.12 (review application) have been identified in table 13. These are: • Make comments • Summarize decisions • Make notes of decisions • Provide knowledge The actors of the activity play different roles in the transactions and use different knowledge sources. Once the application is received for review, the committee members make comments about the applications and the Chair summarizes and concludes the decisions. The Manager takes notes of the decisions. The communication model is drawn using the event driven processing diagram (Figure 32). In this diagram the main activity 'review application' is decomposed into operations and each operation is shown as a separate activity. The communication model demonstrates how the actor- 'knowledge system' communicates with three other actors (ER committee, Chair and Manager). It is to be noted that this model does not generate specification for the knowledge system rather shows how the knowledge system communicates with the actors of the activity. 83 From activity 1.11 -Make comments | {ER committee} Comments made Summarize decisions {Chair} Knowledge referred Refer knowledge Knowledge referred Decisions made Make notes of decisions {Manager} Applications •reviewed • Refer knowledge Provide knowledge {K-system} (K-assets) Refer knowledge Knowledge referred Figure 32: Communication model for the activity 1.12 (review applications) 5.6 Revised models of commonKADS The following modifications were done on the commonKADS models to apply in the context of the case study: 1. In Organization model (OM-1), organizational context on external factors of the organization or business strategy of the Organization were not discussed as they do not apply for ORS 2. Activity analysis method used to generate Task Model-1 (TM-1) instead of commonKADS TM -1 . This was done because the activity analysis method analyzes each activity as required for later use in other commonKADS models (TM-2) 3. Agent model was integrated with the definition of the type of knowledge actors 4. Knowledge refinement model a part of the knowledge model is not developed for ORS as it was out of scope of this study 5. Design model was not developed for ORS as it was out of scope of this study 84 6 Application of the Knowledge model While creating the knowledge model, most important activity was to identify appropriate task template for ORS- the assessment task template in this case. After inculcating assessment task in the knowledge model it is important to define the knowledge components in the knowledge models clearly (task, domain, inference). This section would focus on defining the domain and inference knowledge of a sub domain. 6.1 Methodology Structure: Event driven process modeling technique ! Activity analysis method Organization Model (OM) Structured interviews J Task Model (TM) Agent Model (AM) Communication Model (CM) Organization, Task and Agent Model (OTA-1) I Analysis of selected research applications 1 Knowledge Model (KM) Knowledge Model (KM) - for a sub domain w Purpose: to define the knowledge elements (domain and inference) of the knowledge model for a sub domain Method: The domain of 'research applications' is vast. It has been found that some of these research applications deal with 'schools'. A sub domain is defined- 'research applications that are 85 related to schools'. In this category all the applications that are any way related to schools are considered. The research topics related to schools include: school and community, policies, learning process for students, evaluation of teaching method, etc. By searching the ORS database of research applications with the keyword 'school', 53 applications were identified. There could be more applications related to schools but those applications were not identified as the search was done on the title of the research applications. The applications were not selected whose title do not contain the word 'school'. From these 53 applications, 14 applications were selected randomly. These 14 applications were studied closely to develop the elements of the knowledge model (domain and inference). Studying all the common characteristics of the applications developed the abstracted case and developing the norms. 6.2 Results The domain knowledge of the sub domain contains: the case datum (individual research application) and norms. The structures of the norms and abstract case are not defined in commonKADS. Analyzing the domain generates the structures mentioned here. We start with mentioning the abstracted case. Category Details Topics sensitive and non-sensitive Methods Interviews, observations, questionnaire Subjects Teachers, students, teacher candidates of U B C Recruitment process direct contact (in class through the teacher), voluntary participation Supporting documents Approval letter from agency, Informed consent form for students, Informed consent form for parents, Instruments scripts for questionnaire and/or interviews. Process of conducting research After getting permission from school board contact the administration of school, then contact the teacher of the class. Speak to students and explain the research, handover the parental consent form (if applicable) and student consent forms to students. Collect the consent forms and then at a later date(s) conduct the research. Table 21 Abstracted class of research applications related to school 86 The categories and the descriptions of the abstracted case represent a general case (research application). The norms are represented into two parts, first - the categories (figure 33) and second the rules. Topic •Non-sensitive • Sensitive Method •Interview • Questionnaire • Observation •Combination Subject •Student • Teacher • Both Recruitment Process •Direct contact • Voluntary participation Category A Researcher's Qualification &ability to handle situations Privacy of data collected Category B Supporting documents • Student consent form • Teacher consent form (if teacher=subject) • Approval letter of agency (school, school board) • Parent consent form (if student age <19 yrs) •Instrument scripts (interview, questionnaire) Category C Figure 33: Categories of norms for the domain 'applications related to schools' Developing Norms. A norm - (In our case corresponding to school related applications) - can be described as a set of rules and procedures that should be followed when assessing applications. To develop specific norms for this example we incorporated specific reviewers' practices into the information provided from the abstract case. We began by classifying the categories of the abstract case into wider categories represented as A , B, and C in Figure 33. The classification was driven by the review process, which distinguishes between information included in the application form (Category A) and supporting documents (Category C). In addition, category B was added to represent additional information that is required in the case of applications with sensitive topics. The categories of norms are created by classifying the research applications into categories- topic, method, subject, type of supporting documents and recruitment process 87 (category A & C). More categories were identified that apply only to certain type of applications to get category B. The norms can be classified into two types, norm 1- research application with non-sensitive titles (for example, research related to schools policies) and norm-2- research applications with sensitive titles (for example, research related to suicidal behavior, bullying in school, etc.). The research applications are handled separately by the two norms. The rules for the two norms are: NORM - A (RESEARCH WITH SENSITIVE TOPICS) 1. Check whether all the boxes and at least one item from each box is used in the research application from category A. 2. Check whether all the items from category C are used in the research application. 3. Check all the items of category B 4. Check the explicit mentioning of the research objectives in the consent forms. 5. Check coercion (influential relationship between the subject and the researcher that may affect the research quality) 6. Check the handling of data when audio/video used in the research method 7. Issue certificate of approval if condition 1 to 6 are satisfied else issue proviso. NORM - B (RESEARCH WITH NON SENSITIVE TOPICS) 1. Check whether all the boxes and at least one item from each box is used in the research application from category A. 2. Check whether all the items from category C are used in the research application. 3. Check coercion (influential relationship between the subject and the researcher that may affect the research quality) 4. Check the handling of data when audio/video used in the research method 5. Issue certificate of approval if condition 1 to 4 are satisfied else issue proviso. The norms can help the committee members to give decisions for new research applications. 88 Table 22 summarizes the inference knowledge for this domain. General inferences Specific inferences Abstract case Refer table 21 Specify norm Refer figure 33 and 7 rules mentioned above Select norm Select norm 1 or norm 2 based on the title of the research application Evaluate norm Use abstract case, norm and case specific norms Match to see whether a solution can be found Match with the decisions 1, 2 or 3 Table 22: Inferences for the domain research applications related to school The general inferences were obtained from the knowledge model (section 5.5); the specific inferences are derived from the abstracted case and norms as mentioned above. This section demonstrates the method to create a knowledge model for a particular domain. While developing a knowledge model, this method will be useful to develop abstract case and norms for any domain. 89 7 Conclusion 7.1 Contributions Overall contributions The overall contributions of the thesis are summarized below: • Combining 'event driven process diagramming technique' (a process modeling technique) with 'CommonKADS' (a knowledge engineering technique) to extract knowledge requirements from business processes. • Testing the above mentioned methodology (combination of the two techniques) by applying it to a case study. • Developing a practical procedure to map processes and identify knowledge related to them. • Demonstrating how to derive knowledge-requirements from business processes by generating commonKADS models • Further applying the knowledge model of commonKADS to a specific domain and demonstrating how the model can be used in other domains. The methodological contributions of this thesis are of two types. First, operationalization of commonKADS models with minor modifications to the original commonKADS literature. These contributions are listed in section 5.6. Second, generation of new methods, procedures and structures to the commonKADS models as found necessary. These contributions are listed in the next section as specific contributions. 90 Specific contributions • Activity analysis method (Section 3.4 ) • Procedure to rank knowledge intensive activities (Appendix C) • Structure of the 'abstracted case' (Table 21) and 'norms' (Figure 33) developed to apply the knowledge model to a specific domain. • Use of 'event driven process diagramming technique' to draw the communication model (Figure 32). Additional contributions We suggest using the commonKADS method in conjunction with 'Event driven process diagramming technique' to conduct knowledge requirements analysis for business processes. The combined methodology is suitable for analyzing knowledge requirements for any organization as long as some of the organizational processes or activities are knowledge intensive. The methodology is context independent and thus is more general than context dependent methods applied in specific organizations for similar analysis. The methodology identifies knowledge sources, knowledge flows and knowledge users clearly. Furthermore it suggests how to use a knowledge system to convert tacit knowledge (for example, personal experience) to explicit knowledge in the knowledge system to some extent. Therefore tacit organizational knowledge (not recorded in documents) can be captured using this methodology. Another important aspect of this methodology is that it can help restructure the organizational processes/activities for better capturing and sharing knowledge and ultimately for better functioning of the organization. This can be demonstrated in the case study by comparing an activity before and after using commonKADS. The activity 'review applications' (1.12) mentioned in Appendix B (activity sheet 1.12) is drawn before the commonKADS methodology was applied to the case. After applying this methodology the modified activity (1.12) is shown in 91 the communication model (figure 32). The modified activity is more knowledge intensive now as it uses additional knowledge source (a knowledge system). The actors of the activity have access to the knowledge system and therefore the activity should be operationalized in a better way than before. In the context of the case study, the problems identified in the organization model -1 (OM-1, problems and issues of ORS case) can be addressed with solutions after conducting the knowledge requirements analysis; specifically the problem of 'inconsistency of decision-making' can be addressed by implementing a knowledge system. The structure of the knowledge system that addresses the inconsistency problem is identified in the knowledge model (Figure 30). We analyzed activities and found possible improvements for some activities related to LT and non-IT implementations (Tables 5, 6, 24 and 25). Problems of 'turnaround time for processing applications is long' and 'process of submitting applications is cumbersome' can be addressed to some extent by implementing the improvements. 7.2 Application of Event driven Process diagramming technique The process diagrams generated using event driven process diagramming technique gave the complete picture of the organizational processes with related activities. Though tasks of commonKADS are drawn using U M L (Unified Modeling Language) activity diagrams, but commonKADS does not suggest drawing processes of the organization with any method. Thus event driven process-diagramming technique complemented well with commonKADS to generate both process and activity diagrams. Event drive process diagramming technique helped to identify knowledge-intensive activities easily as the knowledge and resources used in each activity were captured in the diagrams. The technique also helped to generate Organization Model -3 (activity analysis) and Task Model -1 (refinement of OM-3). Without this technique it 92 would have been difficult to capture knowledge and other information from each activity in a systematic and structured way. The constructs of 'event driven processing technique' were useful in identifying knowledge requirements specially while developing Organization and Task models of CommonKADS. 7.3 CommonKADS difficulties and opportunities CommonKADS is a structured method but the method allows flexibility so it can be customized according to the problem. Overall, the model was very useful in conducting the knowledge requirements analysis. This is evident from the fact that the knowledge assets, new knowledge items, knowledge intensive activities were defined and identified clearly in the study. The stakeholders of the organizations (Manager & Chair) appreciated these findings and were interested to implement the knowledge system in the organization. The flexibility of the method allowed using a new process analysis method - event driven processing diagramming technique. CommonKADS does not suggest using any process modeling method but the event driven methodology meshed easily with the CommonKADS models specially Organization and task models. Though CommonKADS is a comprehensive method, there were some drawbacks found while using it. In particular: • Some models in the model suite were not linked properly. For example, the communication model was not properly integrated with the knolwedge model. The elements of communication model (inferences) were not directly related to the knowledge model. Though the agent model helped to identify the agents of the Organization with their knowledge and expertise but the model did not provide any help to generate the knowledge model. 93 • As the primary aim of the CommonKADS is to develop a complete knolwedge system, the knowledge model was found to be quite detailed. When the organization's knowledge intensive actiities are not complex; some steps to generate knowledge model (such as developing domain schema, method specific conceptualizations) become unnecessary. • For a simple knowledge based systems CommonKADS may end up with large documentation because of using worksheet for most of the models. One Knowledge system was developed for one knolwedge item in this sudy and 21 tables were generated for it. • CommonKADS aims to model the overall organization and model knolwedge intensive activities. This integrated approach of CommonKADS increases the documentation. It would be better if there were two distinct methods used in CommonKADS, one- to model the Organization and the other -to model knowledge intensive activities. In that case the users would get more freedom to choose the model they would like to develop. CommonKADS can be more useful in practical applications if some of the documentation are removed unless they are not absolutely related to the knowledge model. The agent model and some parts of organizational model such as Organization Model-2- (dealing with organizational focus & objectives) are not directly related to the Knowledge model. They could be removed from the model suite or mentioned separately to model the organization. This would reduce the documentation and also increase the focus of CommonKADS to develop knowledge model. Users who do not want to model the organization will still have to develop all the models mentioned in the commonKADS suite (Figure 5). 7.4 Future Research Knowledge requirements analysis of business processes is relatively a new area of research. It will be interesting to find the use of other Knowledge Engineering methods to do knowledge requirements analysis of business processes. A comparison of other knowledge engineering 94 methods (MIKE, PROTEGE II, etc) with CommonKADS can reflect more on the limitations, improvements and usability of CommonKADS. Another avenue of research in this area is to identify the suitability of knowledge engineering methods on organizational size or complexity of operations. It would also be interesting to actually develop the design model of CommonKADS to a knowledge system and study the usability and usefulness of the system. The above case study focuses on activities that are not very knowledge intensive or complex. The limitations and usefulness of CommonKADS wil l be more prominent if CommonKADS is applied to organizations where the processes are more complex and knowledge intensive (for an example processes in consulting organizations). 95 8 References: 1. Angele J., Fensel D., Studer R. (1998) Developing knowledge-based systems with MIKE. Journal of Automated Software Engineering. 2. Benbasat I., Goldstein D. & Mead M . (1987) The Case Research Strategy in Studies of Information Systems. MIS Quarterly, 11(3). 369-386 3. Brint.com http://www.brint.comAm/whatis.htm (accessed August 25, 2002) 4. Davenport T. H . (1993). Process Innovation: Reengineering Work through Information Technology, Harvard Business School Press, Boston, M A . 5. Davenport T. H . and Prusak L. (1998). Working Knowledge: How Organizations manage what they know. Harvard Business School Press, Boston. 6. Dignum, V . and Heimannsfeld, K. (1999). Knowledge Management for Requirements Engineering. Proceedings to the Knowledge Acquisition Workshop KAW99. University of Calgary. 7. Darnton, G. and Darnton M . (1997). Business Process Analysis. Boston M A : International Thomson Business Press. 8. Drucker, P. F. (1988). "The coming of the new Organization". 1998. Boston, M A : Harvard Business School Press. 9. Eriksson H. , Shahar Y . , Tu S. W., Puerta A . R. and Musen M . A. (1995). Task modeling with reusable problem-solving methods; Artificial Intelligence (79) 293-326. 10. Gruber T. R., (1995) Towards principles for the design of ontologies used for knowledge sharing, International Journal of Human Computer Studies (43) 907-928. 11. Hayes-Roth F., Waterman D. A . and Lenat. D.B. (1983). Building Expert Systems; Addison-Wesley. New York. 12. Hicks B.J., Culley S.J., Allen R.D., Mullineux G. (2002). A framework for the requirements of capturing, storing and reusing information and knowledge in engineering design. International Journal of Information Management (22) 263 -280 13. Kontonya, G., and Sommerville, I. (1998). Requirements Engineering: Processes and Techniques. John Wiley and Sons, New York. 96 14. Kalpic, B. and Bernus, P. (2002). Business process modeling in industry- the powerful tool in the enterprise management. Computers in Industry. 47 -299-318 15. Macintosh A. , Filby I., Kingston J., and Tate A. (1998). AIAI-TR-228 Knowledge Asset Road Maps; in Proceedings of the Second International Conference on Practical Aspects of Knowledge Management (PAKM98), 29-30 October, Basel, Switzerland. 16. Marquardt, M . and Kearsley, G. (1999). Technology-Based Learning: Maximizing Human Performance and Corporate Success, St. Lucie Press, Florida. 17. McMohan, C. A , Pitt, D. J., Yang, Y. , Williams J. H . (1995). An information management system for informal design data. Engineering with Computers, 11, 123-135. 18. Nonaka, I. and Takeuchi, H . (1995). The knowledge creating company: How Japanese Companies Create the Dynamic of Innovation. Oxford University Press, New York. 19. Nickols, F. (1998). The difficult process of identifying processes. Knowledge and Process Management 5(1) 14-19. 20. Puerta A.R., Egar J.W., Tu S.W. and Musen M . A . A multiple-method knowledge acquisition shell for the automatic generation of knowledge acquisition tools, Knowledge Acquisition 4 (1992) 171- 196. 21. Pritchard J. P., Armistead C. (1999). Business Process Management - lessons from European business, Business Process Management Journal; Bradford; 5(1). 22. Rolland, C , Nurcan, S., Grosz, G. (1999). Enterprise knowledge development: the process view, Information and Management 36, 165-184. 23. Schreiber G., Akkermans H. , Anjewierden A. , Hoog R., Shadbolt N . , Velde W. and Wielinga B. (1999). Knowledge Engineering and Management-The CommonKADS, The MIT Press. 24. Stake, R. E. (1995). The art of case study research. Thousand Oaks, C A : Sage. 25. Steif J., (2001) "Process Modeling" Lecture notes, Department of Commerce and Business Administration, University of British Columbia, 1-60 26. Wall, R. A. (1986). Finding and using product information. U K : Gower. 27. Wiig, K. M . , R. de Hoog. and Van der Spek, R. (1997). Supporting Knowledge Management: A selection of methods and techniques. Expert systems with applications. 13 (1). 28. Yin, R. K. (1984). Case study research: Design and methods. Newbury Park, C A : Sage. 97 9 Appendices Appendix A - Description of processes and activities of ORS Appendix B - Consent form and Interview Questionnaire script Appendix C - Tables and figures Appendix D- Process and activity diagrams of ORS 9.1 Appendix A The main processes involved in getting approval of ethics in research involving human subjects are as following: • Processing new applications to get ethics approval • Processing amendments/renewal to already approved research projects • Handling queries related to approval of ethics The three processes and the activities in each process are described here. The events are numbered in order to identify them clearly. The resources and actors in each activity are mentioned in the activity diagram. Description of activities for getting ethics approval for new applications: Event- Received new applications with documents New applications are received for ethics approval before the deadline for the scheduled applications. The applications are accepted 14 days prior to the meeting. The applications are processed as they are received. A l l the applications that are received before the deadline are processed for the next committee meeting. 1.1 Event- Stamped application & documents When an application is received it is stamp dated first. 1.2 Events- Documents complete/documents incomplete Next the application is checked for any missing documents. If there are some missing documents then it is communicated back to the researcher for missing documents. Event- Received amendments for deferred applications 99 The amendments for the deferred application are received and the date when amendment is received, comments (example- consent form version dates have changed) are entered in the database. 1.5 Event- copies prepared ER Assistant prepares 20 copies of the amendments for deferred applications. 1.3 Event- Primary reviewers assigned The basic information about the application is entered in the database. These information are received date, meeting date, name & address of principal and co-investigator, source of funds, project period, institution where the research is conducted, mailing address of for correspondence and the title of the project. The information is entered irrespective of the completeness of the application form. If any information is missing then a note is sent to the principal investigator about the missing documents. The database automatically assigns two principal reviewers for the application. The ER assistant also ensures that the principal investigator of a research application is not the same person who is assigned to review the application. 1.4 Events- Applications stored/Applications forwarded The applications are stored after the data is stored in the database till the deadline for the next meeting. Event- entered data on database After data of each of the new applications/amendments are entered in the database supporting documents are prepared by ER assistant & the Ethics Manager after the deadline 1.7 Event- Documents prepared for the meeting The supporting documents are mailer (titles & principal investigators of all applications), agenda of the meeting, protocol review forms, behavioral posting form (internal document-not sent to 100 the review committee), and the previous minutes of meetings. A l l the applications are collated along with the supporting documents. 1.8 Event- previous minutes attached The minutes of the previous meetings are attached along with the supporting documents. Then these documents that are prepared are attached with the applications that are received. 1.9 Event-Applications and documents send Documents are sent to the reviewers by campus mail before the meeting. The reviewer gets around a week to ten days to review the applications. 1.10 Event- Applications and documents received The committee members and the Chair receive documents and applications. 1.11 Events- comments prepared for meeting/sent comments if not attending The principal reviewers and the chair of the committee read the applications in great details and write down their comments on protocol review forms. The principal reviewers also study other applications that they were not assigned to review but not in details. Extra protocol forms are also given to the reviewers to review applications that were not assigned to them in case they find some applications that are related to their research interests. If the reviewer is not been able to come to the meeting then he/she sends his/her comments by mail/fax to the Ethics Manager in advance. 1.12 Event- Applications reviewed In the meeting all the applications are reviewed sequentially. If all the applications are not reviewed in a committee meeting then they are reviewed in the next meeting. In the committee meeting the applications that were not reviewed in the last meeting are reviewed first and then the new applications are reviewed. The committee members, Chair and the Ethics Manager are present in the meeting that lasts about 2.5 hours to 3 hours. The committee takes the decisions 101 about the applications and the Ethics Manager notes down the decisions in Behavioral posting form. 1.13 Events- proviso issued/deferral notice issued/application ok/application ok with comments The Ethics Manager collects the protocol review sheets from the board members and prepares the minutes of the meeting. There are four possible outcomes for each application, first-approval, second approval with comments, proviso (amendments required but reviewed by only the Chair) and deferred (amendments required and reviewed by the committee again). 1.22 Event- Notice sent The notice for proviso or deferral is sent to the principal investigator 1.21 Events- Received amendments for deferred applications/amendments received for proviso The amendments for deferred applications are received and entered in the database. The processing of amendments of deferred applications are treated same as that of new applications like assigning primary reviewers, etc. (continued in the process diagram of the first page) Other amendments that are received are entered in the database. 1.20 Event- amendments entered for proviso The amendments received from principal investigator are entered in the database. 1.19 Events- Amendments not OK/Amendments OK If the amendments that are reviewed by Chair are not satisfactory then a notice is prepared and sent to principal investigator. If the amendments are satisfactory then they are checked for pending grants or Teaching Hospital applications. 1.14 Event- certificate approved The Chair issues the certificate of approval. 102 1.15 Events- Teaching Hospital application pending, grant Teaching Hospital application not pending, grant pending If the application is complete after it is finally approved by Chair; the Ethics Manager checks whether funding for the research is still pending or not. If the grant is pending then the Manager updates the database of pending funding and advises toward officer that the grant is no longer pending. If the research application is not sent to the teaching hospital then a memo is sent to the principal investigator to send the application to the teaching hospital. If there is no pending grant or Teaching Hospital application then the certificate of approval is sent to ER assistant for mailing to the principal investigator. 1.15 a Event- Final check on application The Ethics manager checks the complete application before it is sent to the PI. 1.18 Event- PI notified The PI is notified that the grant has been updated. 1.16 Events- certificate of approval/memo sent If there is no pending grant or application submitted to teaching hospital (whatever the case may be) the certificate of approval is sent to the principal investigator. If there is a pending grant then the principal investigator is advised about the pending grant and the certificate is sent to the principal investigator. If the hospital certificate is pending then the certificate of approval is withheld and a note is sent to the principal investigator about the pending hospital certificate. 1.17 Event- comments entered If a comment about the research is attached along with the certificate of approval then depending on the comment the principal investigator has to send the reply of the comment or he/she may continue with the research. If the comment is received then it is entered in to the database. 103 Description of activities for getting ethics approval for approved applications (renewal/amendments) Event- Received amendment form/documents The ER assistant receives the amended form or/and documents 2.1 Event -Data entered The comments, date of receive are entered into the database and the amended documents along with the original documents are send to chair for review. 2.2 Events- Documents satisfactory/documents not satisfactory The chair reviews the amendments and if he/she is satisfied with the amendments then he/she forwards to the manager. If the documents are not satisfactory then it is sent to Ethics manager to prepare notice for further amendments. 2.7 Event- Notice sent The request for further amendments is sent to the principal investigator. 2.3 Event- certificate issued The chair issues the certificate of approval 2.4 Events- Teaching Hospital application pending, grant Teaching Hospital application not pending, grant pending If the application is complete after it is finally approved by Chair; the Ethics Manager checks whether funding for the research is still pending or not. If the grant is pending then the Manager updates the database of pending funding and advises toward officer that the grant is no longer pending. If the research application is not sent to the teaching hospital then a memo is sent to the principal investigator to send the application to the teaching hospital. If there is no pending grant 104 or Teaching Hospital application then the certificate of approval is sent to ER assistant for mailing to the principal investigator. 2.6 Event- PI notified The PI is notified that the grant has been updated. 2.5 Event- certificate sent If there is no pending grant or application is submitted to teaching hospital (whatever the case may be) the certificate of approval is sent to the principal investigator. Event- send reminder for expiring research When a research proposal is close to expire, a reminder is send to the principal investigator to renew the research. 2.8 Events- renew research requested/research expired The research is either expired or a request is sent to the ethics review office to renew the research. Description of activities for handling queries Event- received status query Queries are received to know the status of the application. ER assistant handles the queries. 3.1 Event- Data searched Based on the application number, Principal Investigator's name the status of the application is found. 3.2 Event- requested documents send If the principal investigator has not received documents that are related to the research application, ER assistant sends a copy of the document to the principal investigator. In other cases only the status of the application is informed. Event- received approval process query 105 Often the research office receives queries related to interpretation of policies, provisos or regulatory requirements. The Ethics Manager handles this query. 3.4 Event-details found Requested details of the guideline or policy are found from the policy handbook or internet. 3.5 Event-process informed The processes of filling the application form or research policy are informed. Definitions of Terms Proviso: A proviso is a memo that is send to a researcher to modify the application in order to get it approved. A proviso is issued when there are no major amendments to be done on the applications. The committee does not review provisos again only the Chair reviews it after receiving the amendments. Deferred Applications: When the applications do not address most of the ethical issues, then they are deferred to another meeting after getting the amendments from the researcher. The committee seeks a full review of the deferred applications as there are some major ethical concerns not addressed in the applications. Certificate of approval: The certificate of approval is issued when there is no ethical concern regarding an application. This certificate is signed by the Chair of the committee and is valid for one year. 106 STRUCTURED QUESTIONNAIRE FOR INTERVIEW (ETHICS REVIEWERS) INTRODUCTION I introduce myself as a master's student in Management Information Systems, commerce department. I have done my M B A in marketing and worked 4 years as a training consultant in India. I'm presently doing a project in Office of Research Services (ORS), U B C for analyzing the ethical review processes. This study is about mapping the ethical review processes and identifying for possible improvements. In this regard I would like to take help from you for possible improvements of ethics review processes. I also seek your permission to record the conversation during the interview process. The interview would take in total approximately 45 minutes and would consist of the following steps: 1. Going through the ethics review processes. 2. Analyzing the processes for possible improvements (only relevant processes where you are involved) 3. Analyzing the processes of knowledge requirements (only relevant processes where you are involved) & your opinion of technology use in the processes. 109 STRUCTURED QUESTIONNAIRE FOR INTERVIEW (ETHICS REVIEWERS) STEP 1 (approximate 15-20 minutes)—Process familiarization I will go through the ethical review processes with you. In the attached 4 pages the ethical review processes are drawn. The legends of the processes are also defined here. I also define the following terms in order to understand the processes: Processes are sequences of events and activities. Activity- a mechanism by which some objects change their state Events- a notable occurrence at a specific point in time in which changes to the state of one or more objects occurs. STEP 2 (approximate 15 minutes)- Process Analysis For each of the activity of the processes (where you are involved) please mention the following: • How important this activity is? (highly important/important/somewhat important/not important at all.) • Would you like to improve the process (es) that is associated with this activity? If yes, then how? • Would you suggest an alternative process for the activity? • Would you recommend automating this process electronically? STEP 3 (approximate 20 minutes)- Knowledge Analysis We would now analyze the knowledge that is associated with the processes. Knowledge here is defined as "provides guidance to humans so that they can make judgments and formulate decisions" Based on the above definition of the knowledge, please identify (only on the processes where you are involved with) the following: • Knowledge that is available to you for each of these activities • Knowledge that is required by you to efficiently and effectively perform each of these activities Please contact palash@interchange.ubc.ca for any clarifications or further suggestions. 110 ACTIVITY SHEET Name: Study applications before meeting Number: 1.11 Applications and documents received Study applications before meeting! Read applications Write comments {Members, Chair) Comments prepared for meeting Send comments by mail if not reviewing Purpose Actors Input Output Conditions for triggering To study applications before meeting Members, Chair Applications & documents Comments on applications When applications and other documents are received for review Operation(s) Roles Resources (data & knowledge) Rules Description Read applications Members, chair Research experience on Ethics, tri-council policy and guidelines Write comments Members, chair ACTIVITY SHEET Name: Review applications Comments prepared for meeting Number: 1.12 Sent comments by mail if not attending Review applications {Members, Chair, Manager} (Application package) Applications reviewed • Purpose Actors Input Output Conditions for triggering To review the applications Members, Chair, Manager Applications Reviewed applications When other documents for meeting are prepared Operation(s) Roles Resources (data & knowledge) Rules Description Primary reviewers review applications and other reviewers comment on the application Reviewers Guideline notes, research experience, Tri council policy Summarize conclusion Chair Take notes for each application reviewed Manager Case law from 18 years experience, guidelines 111 9.3 Appendix C Activity # Activity Actor (s) 2.1 Enter request Manager, ER Assistant 2.5 Mail certificate ER Assistant 2.7 Prepare memo for Principal Investigator Manager Table 23 Activities by importance ranking for processing of approved applications No. Activity Possible improvements 2.1 Enter request Could be checked and entered in the database electronically 2.4 Check pending grants/Teaching Hospital application Send the email to PI about updating grants automatically 2.5 Mail certificate Could be sent electronically 2.7 Prepare memo for PI Send all the communications to the PI as soon as possible by email 2.8 Receive & enter comments Could be sent electronically Table 24: Activities to be improved with the aid of IT for processing of approved applications No. Activity Possible improvements 3.1 Identify application The researcher can log in to the website and check the status of the application 3.2 Print & send documents requested Information could be sent over email other hard copies could be sent by mail 3.3 Inform status Could be done by the researchers themselves by looking the status of the application on the web site 3.4 Refer guideline/policy Information could be available in organized way in the web, like creating FAQ's , index page, Application filled on web should have online help about how to fill the application with appropriate links 3.5 Answer requests This activity could be avoided if more information is available on the web Table 25: Activities to be improved with the aid of IT for handling queries 112 No. Activity Alternate activity 2.1 Enter request Sent the amendments to ORS office electronically Table 26 Alternative activity suggested for processing approved applications No. Activity Alternate activity 3.1 Identify application The researcher can log in to the website and check the status 3.3 Inform status Could be done by the researchers themselves by looking the status of the application on the web site Table 27: Alternative activities suggested for handling queries Scoring method for ranking knowledge intensive activities in Organization Model 3 CommonKADS suggests to measure the intensiveness of the activities by four criteria- cost (c), Resources (R), frequency and criticality of the activity (CR). The exact mechanism to determine the intensiveness is left to the user. For our case, we omitted the criterion- frequency as each activity is conducted in the process exactly same number of time. Each activity is always conducted once for each process. Scores from 1 to 5 are given to the 3 criteria. 1 refers minimum score and 5- refers maximum. By studying the activities, scores are given to each activity 1 for -least cost, least resources used and least critical and 5 for maximum cost, maximum resources used and most critical. The final score is calculated based on the formula: Final score = [(points on cost) +(points on resource) + (points on criticality)]/3 The simple average is used here, as it is perceived that each criterion has the same weightage 113 Organizational Model Process breakdown Worksheet OM-3 Score No. Activity Performed by Knowledge Asset Intensive? C R CR Final 1.2 Check correctness of applications ER Assistant Guidelines to fill the applications Yes 2 4 4 3 1.11 Study Applications Chair, Committee members Manual on tri council policy, other documents and books on ethics, note book on ethics supplied by ORS (guidance notes), previous minutes of the meeting, guidelines on policies, research experience (teaching research courses & policies), research experience (conducting related to human subjects) Yes 4 4 5 4 1.12 Review applications Chair, Committee members Tri-council policies and other guidelines, Committee members (including lawyer), Manager, Experience of handling applications Yes 4 5 5 5 1.13 Summarize and enter information Manager guidelines and experience Yes 2 4 4 3 1.15a Check application Manager Familiarity with the guidelines, institutional memory (18 years experience in handling ethics applications) Yes 2 4 4 3 2.2 Review Amendments Chair, Manager Ethics related research experience Yes 3 2 4 3 3.4 Refer guideline/policy ER Assistant, Manager Existing knowledge about guidelines and policy Yes 3 2 2 2 Table 28: Worksheet Organization Model-3 114 Subject Knowledge available Knowledge required Knowledge Sharing method SI Previous training, manuals on tri council policy, other documents on ethics The documents are too long, a summary of highlighted points is necessary. 1 day orientation necessary on review processes and regular updates on changes in review processes Difficult to share directly. Best is to learn on the job. Training could be a way of sharing knowledge S2 Note book on ethics supplied by ORS (guidance notes) Guidance notes are necessary for every body. Orientation session for every 3 months of the processes. Orientation and discussions S3 previous minutes of the meeting. I ordered books which I felt would be useful on my own Documents related to ethics on research. Sometimes I search ethical issues based on research applications on internet. University should provide a recommended reading (underlying principles, challenges, current policies, privacies, etc) Orientations S4 guidelines on policies read guidelines and other ethics related documents. Go to the Ethics related seminars and share your ideas with other ethics committee of other universities. S5 My research experience (teaching research courses), policies Guideline policies and experience, sometimes resource related to the research methodology Discussions on ethics subject, reading ethics resources S6 Experience on conducting research on qualitative studies related to human subjects thorough reading of tri council policies. The available documents are more theoretical and not so much practical. workshops - oral presentations, as the activity is more practical therefore we have to spend time on actual process to learn by watching S7 Studied and taught ethics, board of several ethical committee and thus have a background on ethics Formal orientation Informal knowledge learnt while going through the process. Some guidance documents could be given when they join Table 29: Knowledge analysis of activity 1.11 (Study applications) 115 Subject Knowledge available Knowledge required Knowledge Sharing method SI Tri-council policy, presentations/conferences on ethics Summary and highlighted documents in addition to the documents already available new reviewers need to ask questions at the meeting and observe & note the comments made by other reviewers; discuss the policy issues at the start of the meeting S2 Guidelines and policies We need some standard comments to review applications on issues which we discuss but do not really know how to handle (like longitudinal data, process of conducting research with school boards, etc) Orientation every 3 months. Sit down with the new member and discuss the processes S3 Ethics courses which I took when I was a graduate, no material apart from tri council policy. Precedence setup by the committee like what is accepted ethics behavior, what is not. Manual of decision making, past record of decision making, principles underlying decisions or exceptions on decisions. To create more resources on these documents documents on decision making S4 The most important resource is the other committee members and Shirley, then access to the folder given to us with ethics policies and guidelines Listen to the meetings to understand how to review application when you join new. Go through the guidelines and websites S5 Policies and other personnel (Lawyers, Manager) who has knowledge on Ethics in the committee Varied personnel of the committee and their rich experience some documents on earlier decisions S6 Practical knowledge of reviewing applications over the years and also the guideline policies specially tri council policies Practical knowledge Get experience to review the applications by listening to the comments of the other reviewers S7 materials for new people, guidance notes, responsibilities of review members, different set of skills List of the main ethical issues and an example of each of those issues and the record on what decisions where made. For example- active consent- develop a set of principles for active consent, attach a list of protocols relevant to active consent and the decisions on those protocols. Overtime this would be useful- similar to case law. decision making documents (case laws) and more guidelines Table 30: Knowledge analysis of activity 1.12 (Review applications) 116 T a s k M o d e l K n o w l e d g e I t e m W o r k s h e e t T M - 2 K n o w l e d g e I t e m W o r k s h e e t T M - 2 Item policy guidance previous minutes of meeting N a t u r e o f t h e k n o w l e d g e B o t t l e n e c k / t o b e i m p r o v e d B o t t l e n e c k / t o b e i m p r o v e d Formal, rigorous V decisions on last meeting known, not of any other meetings Empirical Quantitative Heuristics, rule of thumb Highly-specialized, domain-specific Experience-based Action-based Incomplete V need to be updated with the changes in methodologies Uncertain, may be incorrect Quickly changing Should be well monitored document Hard to verify Tacit, hard to transfer F o r m o f t h e k n o w l e d g e Mind Paper V V only available in paper to the members, members cannot access the past decisions Electronic Action Skill Other A v a i l a b i l i t y o f k n o w l e d g e Limitation in time Limitation in space Limitation in access Limitation in quality frequent update required Limitation in form V Available as a discrete source of information on paper Table 31:Task Model-2 -Knowledge item analysis of - policy guidance and minutes of last ER meeting 111 T a s k M o d e l K n o w l e d g e I t e m W o r k s h e e t T M - 2 K n o w l e d g e I t e m W o r k s h e e t T M - 2 Item Manual on tri council policy Ethics related literature (books & documents) N a t u r e o f t h e k n o w l e d g e B o t t l e n e c k / t o b e i m p r o v e d B o t t l e n e c k / t o b e i m p r o v e d Formal, rigorous V too detailed, need a summary of the document Empirical Quantitative Heuristics, rule of thumb Highly-specialized, domain-specific V too detailed, need highlighted issues V sources unknown Experience-based Action-based Incomplete Uncertain, may be incorrect Quickly changing Hard to verify V sources not verified, necessary to identify a set of prescribed readings Tacit, hard to transfer F o r m o f t h e k n o w l e d g e Mind Paper V may be useful if available electronically V books Electronic information on internet Action Skill Other A v a i l a b i l i t y o f k n o w l e d g e Limitation in time Limitation in space Limitation in access V not accessible to all the members Limitation in quality Limitation in form Only in book format Table 32:Task Model-2 - Knowledge item analysis-Manual on tri council policy and ethics related literature 118 T a s k M o d e l K n o w l e d g e I t e m W o r k s h e e t TM -2 Item Experience of handling applications N a t u r e o f t h e k n o w l e d g e B o t t l e n e c k / t o b e i m p r o v e d Formal, rigorous Empirical Quantitative Heuristics, rule of thumb Highly-specialized, domain-specific Experience-based V tacit knowledge to be converted to explicit Action-based Incomplete Uncertain, may be incorrect Quickly changing Hard to verify Tacit, hard to transfer F o r m o f t h e k n o w l e d g e Mind V Paper Electronic Action Skill Other A v a i l a b i l i t y o f k n o w l e d g e Limitation in time Limitation in space Limitation in access V need to share to other members Limitation in quality Limitation in form V Table 33:Task Model-2 - Knowledge item analysis of - experience on handling applications • Rectangles represent dynamic knowledge roles. The name of the knowledge role is written in the rectangle. • Ovals represent inferences. Arrows are used to indicate input-output dependencies between roles and inferences. • A rounded box notation is used to indicate a transfer function • A static role name is written between two horizontal lines. This representation is purposely similar to data stores in DFD's (Data Flow Diagrams) as static roles incorporate the same 'storage' notion. • If a data-dependency line starts with a small solid circle, it indicates that the input or output should be interpreted as a set of objects playing this role. List 1 :Conventions used to draw Inference structure Case Figure 34: Inference structure of the assessment task 120 9.4 Appendix D The processes and activities are drawn in the next four pages. The abbreviations used i n the diagram are: • ER Assistant- Ethical Review Assistant • PI- Principal Investigator • TH- Teaching Hospital 121 ZZl © o n 1/5 3* OTQ B TEL o ' P a o S3 © OTQ O P |-« O «<< P >i o o 5' CTQ 3 re so rt" So «-•• o a OTQ re re o T3 O o 3 PZl © o fB 3' CTQ © fB a & H O* 65 © ' S9 1 re fB 5= CD a 3 fB = SZl c CD o 2 <' T3 o < rt o 2 <" re a. (S3 rt c rt 3" (a 3 § = O O 3 P OQ 70 S L" » 1 a & 3' PS re 1 o rt ffl g on 05 rt "I c rt R > I & f t OS o SO O r< CL rt S» a — o rt » n o 3* n Cu rt rt Cu a 3 5' OTQ C rt rt'