UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

On-line education: on the job training with computer conferencing (the virtual education oracle) Guenther, John J. 1997

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_1997-0173.pdf [ 5.85MB ]
Metadata
JSON: 831-1.0054901.json
JSON-LD: 831-1.0054901-ld.json
RDF/XML (Pretty): 831-1.0054901-rdf.xml
RDF/JSON: 831-1.0054901-rdf.json
Turtle: 831-1.0054901-turtle.txt
N-Triples: 831-1.0054901-rdf-ntriples.txt
Original Record: 831-1.0054901-source.json
Full Text
831-1.0054901-fulltext.txt
Citation
831-1.0054901.ris

Full Text

ON-LINE E D U C A T I O N On trie job training with computer conferencing (The Virtual education oracle) hy John J.Guenther B .A. - Athabasca University A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS F O R T T H E DEGREE; OF M A S T E R OF A R T S IN THE FACULTY OF GRADUATE STUDIES (UBC) (Department of Education; Curriculum and Instruction) We accept this thesis as conforming to the required standard: THE UNIVERSITY OF BRITISH COLUMBIA April 1997 @JohnJ. Guenther, 1997 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of The University of British Columbia Vancouver, Canada DE-6 (2/88) 11 A B S T R A C T Through the delivery of a building code education module over the internet, the effectiveness of on-line education in a work environment was examined. Building officials served as the expert facilitators and instructors and the students were architects and designers. Guests from an arm of the National Research Council involved in building code issues were also invited to take part. Thirty-nine selected students were placed in two groups (Group I and Group H) controlling for age, work experience, computer skill and knowledge of the building code. Two groups, each with about 20 students were selected to participate in an on-line computer conference. Group I received an on-line curriculum with posttests after each section and Group n received only posttests. Academic achievement comparisons were then made between the two groups. T-tests were used to compare achievement for the two groups of learners, one with an on-line curriculum and one without an on-line curriculum, but both given the opportunity to conference and answer specific section review or posttest questions. Dependent variables were identified as instructor access, motivation, participation levels, comparisons to the traditional classroom, level and convenience of on-line involvement, virtual classroom overall rating, course rating, instructor rating, interest, ability to synthesize ideas, academic achievement, and group communication. The independent variables were computer attitudes, expectations about the conferencing system, interpersonal sphere of control, terminal access, and curriculum design. Ill Frequencies were compiled and displayed in graph form to portray variables. Comparisons were made using Pearson Correlations. Results indicate that: • the internet can be a valuable tool for student access and knowledge-building exercises, supporting the hypothesis that students who experience group or collaborative learning in the virtual classroom are more likely to judge the outcomes of an on-line course to be superior to the outcomes of traditional classrooms; • students who depend on their own effort rather than "luck" are more likely to regularly and actively participate on-line; • those with high viewed the on-line experience with some trepidation indicating that they would not prefer to take another on-line course; • the curriculum did not affect gains in building code knowledge between curriculum and non-curriculum groups when exposed to a computer conferencing delivery system; • the interface continues to confound students and leads to frustration that can debilitate the learning experience; and • on-the-job learning is constrained by time and work load demands. iv Table of Contents Abstract ii Table of Contents iv List of Tables vii List of Figures viii Acknowledgments ix CHAPTER I Introduction 1 Overview and background 1 Purpose of this study 3 Project overview 4 Methodology overview 4 Definitions and glossary 7 CHAPTER II Literature Review 10 Traditional, Distance and Virtual Classrooms 10 Advantages and disadvantages of computer 10 conferencing Face to face learning compared to computer 12 conferencing Development of virtual communities 14 Summary 16 Content development and course design 17 Knowledge building 18 Learner and facilitator roles 20 Faciltator role 21 The learner 22 Summary 24 Adult professionals and work 25 General Characteristics 25 Summary 27 Costs 27 Chapter Summary 29 CHAPTER in Methodology 31 Research basis and project description 31 Subject selection 35 Instruments 36 Questions 40 Summary 41 Validity, Reliability and Design Limitations 42 CHAPTER IV Results 43 Conferencing Traffic 43 Collaborative Learning 44 Computer Attitudes 46 Interest and Synthesis 48 Collaboration Index 50 Instructor Rating Index 52 Course Rating Index 54 Virtual Classroom Overall 56 Access and Quality Assessments 57 Group Comparisons 60 Pearson Correlations 61 Access and Activity Conditions by Outcome 62 Process and Assessments of the Virtual 63 Classroom Student Characteristics and Selected Outcome 64 Interviews 65 CHAPTER V Conclusions 67 Chapter overview 67 Summary of Findings 67 Summary of interview responses 74 Summary 74 Further Research 75 Bibliography 77 Appendices 84 Appendix A - Subject response to Interview Question 1.5 84 Appendix B - Subject response to Interview Question 1.8 86 Appendix C - Subject response to Interview Question 2.6 87 Appendix D - Subject response to various interview Questions 89 Appendix E - Subject response to various interview Questions 90 Appendix F - Instruments 92 Appendix F-l-Questionnaire 1 92 Appendix F-2-Questionnaire 2 94 Appendix F-3-Questionnaire 3 98 Appendix F-4-Pretest 104 Appendix F-5-Posttests 107 Appendix F-6-Pre-interview 109 vi Appendix F-7-Post interview 111 Appendix G-Conferencing Itinerary 113 Appendix H - Interviews 114 Appendix I - Project Development 125 Appendix J - Pre and Postest results 127 vii List of Tables Table A- l Instrument Matrix 39 Table 2 Instructor Rating Index 53 Table 3 Group Comparison across Q 2 and 3 61 Table 4 Access and Activities by Outcome 62 Table 5 Process and Assessment of Virtual Classroom 63 Table 6 Student Characteristics and Selected Outcomes 64 viii List of Figures Figure 1 Project Flow 32 Figure 1 a) Conferencing Traffic Bar Chart 43 Figures 2-7 Collaborative Learning Bar Charts 45-46 Figures 8-16 Computer Attitudes Bar Charts 47-48 Figures 17-22 Interest and Synthesis Bar Charts 49-50 Figures 23-28 Collaborative Index Bar Charts 51 -52 Figures 29-35 Course Rating and Time Bar Charts 54-56 Figures 36-39 Virtual Classroom Overall Index Bar Chart 57-58 Figures 40-46 Access and Quality Assessment Bar Charts 58-60 Acknowledgements It is with great pleasure that I thank the following people for their kind assistance and invaluable support in bringing this project to fruition. The work was a combined effort that could not have been accomplished without them. Mary Lynn Ann Farquhar, Research Assistant Chris Farquahar, Research Assistant Dr. Marv Westrom, Advisor, UBC Dr. Ian Wright, Committee, UBC Dr. Lucia Cheung, Committee, UBC Brian Palmquist, Committee, UBC Dr. Gary Hepburn, Committee, UBC Cyberstore, Service Provider Jason Wilkes, Assistant City of North Vancouver Doug Watts, Industrial Research Assistance Program Russ Thomas, National Research Council Don Hazleden, Canada Mortgage and Housing Peter Sweeney, City of Vancouver George Humphrey, City of Burnaby Ric McWilliam, City of Burnaby Rick Bortolussi, City of Richmond Bob Light, City of Richmond Jack Robertson, Building Standards Branch Marguerite Lovelace, Building Standards Branch David Tanner, Building Standards Branch Electric Playground, Assistant Architectural Institute of B.C. and various members Dr. Erv Schiemann, Advisor, University of Calgary 1 Chapter One - Introduction Overview This chapter introduces an on-line computer conferencing project named "Codeworks" developed by municipal building officials and delivered to design architects and engineers. The described need is to make building code course material more available to design architects while they work, creating a virtual community with facilitators (municipal officials) for the dissemination and interpretation of building code knowledge. A summary of the methodology describes the need to compare: • the effectiveness of on-line and traditional courses; and • the effect of the curriculum on on-line learners. Background Communication occurs between architects and municipal building officials on a regular basis as they process building permits from design, through working drawings, permitting and occupancy phases. The building permit process is the main contact and point of "expertise exchange". It is, in many ways, a meeting of minds. This confluence occurs at the counter, over the phone, through the fax machine, in meetings and through various contacts on the job site. A needs analysis was conducted as part of this project, utilizing questionnaires and interviews to / determine the key objectives in this exchange and to identify the educational mechanisms that could be brought to bear. Specifically focusing on the design profession and building officials, the gaps in knowledge and perceptions were studied. 2 To rid projects of major surprises such as running over-budget and failing to meet completion dates, design firms employ many mechanisms. These include contracting with fire protection design consultants, developing building code experts in each design office or encouraging a designer to study the building code. However, primary gaps occur in understanding code concepts, structure and application. Most students interviewed for the project stressed that code contradictions and divergent interpretations often lead to confusion and misapplication. Also, noted was the inability to exchange information directly with municipal building officials. Architects and engineers who have recently graduated from university are not fully familiar with the application of the major building safety codes, yet they face an array of regulatory issues when they take on construction work. To remedy this situation in British Columbia and to assist municipalities in administering the building code, courses have been initiated for practising construction design professionals through a certified professional program1 currently delivered at the University of British Columbia. This voluntary program uses a traditional classroom delivery format and involves three months of intensive building code training. Successful completion of the course leads to formal certification, currently recognised by a few Vancouver lower mainland permits and licenses jurisdictions. Integration of code principles into the university architecture and engineering programs has been attempted by several post secondary institutions in Canada. However, criticisms of this integration arise due to the ill-preparedness of the learner to synthesize complicated code ' The course has run for sixteen years and is supported by the Architectural Institute of B.C., the Building Officials Association of B.C., the Association of Professional Engineers and the Cities of Vancouver and Surrey. The course is delivered for three months of the year to part time students who, for the most part, take time out from practicing architecture or engineering. The course material covers commercial code design requirements and is taught by code experts, professional engineers and practicing architects. principles with building designs2 during their formal training and to retain that knowledge over time (Walkington, Pemberton, Eastwell, 1994). Walkington et al (1994) note that it is important to translate formal training into a "cognitive apprenticeship.. .where students are enculturated into the culture of the engineer, for example, by participating in authentic practises, activities and social interaction" (p. 162). Contact with building officials, the practical application of theory, extensive work experience and curriculum design tailored to student's needs is integral to this apprenticeship. Purpose of this study The problem is that a new delivery system is required that is convenient for people who are working that can take advantage of this "cognitive apprenticeship", is tailored to learning needs and can examine the importance of curriculum design and delivery. Information overload has been noted by Harasim (1987) as a primary failing of computer conferencing systems. By eliminating the curriculum, but retaining the interactive nature of on-line conferencing, would it be possible to achieve equal gains in knowledge, thus reducing design time and costs? Do students have positive views of the on-line experience and what do they see as critical components of the experience? The purpose of this study is to investigate some aspects of the effectiveness of on-line conferencing in teaching and learning about building codes on-the-job. Computer conferencing seems to fit the demands of a "cognitive apprenticeship" by permitting information transfer from the municipal office (where authentic practices, activities and social interaction occur) to the learner (architectural design offices) on demand, allowing self-paced learning. Nipper (1987) states that "the strength of the corporate learner (involved in electronic 2 Listed by Walkington et al are four broad categories of practical work, including knowledge about materials, devices and techniques, safety codes and practices, specific equipment and techniques (p. 164). 4 classrooms), educationally speaking, is the disciplined and focused way in which he/she uses the medium, by making relevant contributions to the subject-related discussions in the electronic classrooms" (p. 169). Project overview The name "Codeworks", was chosen for the project because it portrays the delivery of building code curriculum modules based on four subsections of the 1992 B.C. Building Code (3.1.2, 3.1.3., 3.2.1, and 3.2.2.3) in a conferencing format over the internet. With the data links provided by Westel and Cyberstore, the project fashioned a web site located at http://www.conexus2000.com. Readily available computer hardware within design and municipal offices allowed participants to connect to the internet and then conference within the web site. The web site consisted of four sections: design news (for information only), the curriculum, a conferencing protocol section, and a password restricted conferencing section. The on-line course with computer conferencing was delivered directly into the workplace. The curriculum content was primarily text based, without hypertext capability, supplemented with graphics that elaborated key definitions. Section reviews or posttests followed each curriculum module. Methodology overview Key benchmark times and dates were established. On February 20, 1996, a seminar provided a project overview from Codeworks researchers, with web site presentations from the City of Vancouver and the Provincial Building Standards Branch. Ninety personnel from architectural, engineering and building official offices attended. Questionnaires 1, 2 and 3 (Appendixes F-l , 2 3 These code sections deal with the entry level requirements for building code classification, multiple occupancy, and construction parameters. 5 and 3 respectively) were circulated. Based on age, code and work experience, students were selected and placed in two groups. Building officials within municipal offices acted as facilitators for both groups. Each group could then conference with each other or with the facilitators, but only one group could access the Codeworks curriculum. However, both groups were evaluated using all the instruments: pre and posttests, pre and post-interviews and questionnaires. Pre-interviews were conducted in March of 1996. In March and April, on-line curriculum development was developed with the assistance of the Building Officials Association. At the same time, 40 design offices and four municipal offices installed on-line linkages to the internet. As accredited building code courses were not available comparisons were made to perceptions of traditional classrooms experienced by the sample. Mason (1989) discusses the development of computer conferencing and the changing roles of the technology. He notes that computer conferencing effectiveness depends upon the size of the audience and the virtual aspects of the project. According to Mason (1989) computer conferencing is a large leap from the traditional means of communication that uses only the lecture format. In the lecture format a good deal of the discussion is lost with lecture time consumed by the repetition of information (Harasim 1990). In the virtual realm each exchange can serve as a springboard for the development of easily accessible knowledge. Responses to on-line lecturing from a disbursed audience establishes recorded exchanges that everyone can work from and manipulate. Other benefits of conferencing include the potential of a support group within the office to assist with the technology and curriculum, and the ability, while learning on-line, to adjust the appearance of the text and search for cross references. 6 Interaction between the students and the instructor in this study was based on municipal client interaction scenarios as suggested by Harasim (1987) who said that: "...contact between instructors and students should be frequent and intense; debate and dialogue should play a greater role (than in the undergraduate courses)" (p. 119). The replacement of the classroom environment by the virtual realm has been researched by Harasim (1990); Hiltz (1994); Mason and Kaye (1989). Various types of learning have been described by Hiltz (1994): rote learning, integrative/knowledge-building, attitudinal change, and application. Two areas of primary concern in this study are attitudinal change and integrative/critical knowledge-building skills4 which are measured across the variables of attitudes toward computers, expectations about the system, personal and interpersonal sphere of control and the effect of the curriculum. The four central research questions are: Will there be significant gains in code knowledge between Groups "I" and "U" when Group "I" is exposed to a computer conferencing delivery system with a defined curriculum and "II" is exposed to a computer conferencing system without a defined curriculum? Will students with a greater sphere of control on either the personal or the interpersonal levels be more or less likely to regularly and actively participate on-line, take another on-line course, rate the virtual classroom as easier and more effective than the traditional one, and have a positive view of the instructor or facilitator? 4These types of learning are described by Hiltz as: "integrative/critical knowledge-building, wherein the student is able to pull together or synthesize diverse facts, ideas, or procedures by analyzing and organising them into larger conceptual frameworks" ; and "... attitudinal change, whereby the student acquires, for instance, an 'appreciation' of literature or art. standards for ethical behaviour in their occupation, less prejudiced feelings about other racial or cultural groups or increased interest in pursuing further knowledge in a particular field." (p.76,77). 5 Hiltz (1994) describes the various spheres of control that have also been researched by Rotter (1966) and Paulhus (1983). Personal sphere of control is described as: "...a subscale of measures being a result of one's effort rather than "luck". Interpersonal control measures control over people in groups" (p.68). Measurements of these effects are contained in the images of yourself section of questionnaire 2. 7-Will students who experience group or collaborative learning in the virtual classroom have positive6 views of on-line course work? Will conferencing students with good computer terminal access at either home or in the office, who spend more time on-line and view on-line courses as more convenient, report positive views of on-line courses across a number of variables7 and will students report positive views of the on-line course across the same variables? Definitions and Glossary The following definitions are provided to assist the reader in understanding the technical jargon used in the computer conferencing environment. Architects - Each province in Canada regulates the educational and practice requirements of the architectural profession. This profession is chiefly responsible for all elements of building design (eg. building envelope, life safety, and esthetics) and plays a coordinating role in structural, mechanical or electrical issues. In B.C. the profession is self-regulated under the Architect's Act. Building Code - The National Building Code is published by the National Research Council in Canada. It is a model document that is empowered and modified by provincial legislation, with the title changed to reflect the province of adoption (eg. British Columbia Building Code). The public review process for the document occurs over a 5-year cycle with the particular year of publishing assigned to the name (eg. 1995 National Building Code). Each province will then review the code through another public process, making amendments and then assigning a year to the provincial proclamation (eg. 1992 B.C. Building Code). The Building Code covers life safety, health, social, and environmental issues. 6 Positive measurements are based on correlations with on-line convenience, computer terminal access, time spent on-line, increased communication with the students, improved access to the facilitator, experienced increased motivation by reading assignments of other students and found comments and assignments by other students useful. 7 The variables measured are computer attitudes, instructor rating, interest in the course, the ability to synthesize ideas, views on the virtual classroom, overall increased collaboration, course rating and course access and quality. Computer Conferencing (CC) - The aspect of grouping a series of computers so that communication can occur on a real-time or asynchronous basis. Discussion is usually text based but can also include graphical support. Users enter comments through a keyboard with a send command and data is then posted for a response. Posted data is available for each group to view, or specific e-mails can be sent for one-to-one discussion. Data is threaded based on the time of the entry and then may be recorded for future retrieval and manipulation. Computer Mediated Communication (CMC) - Any discussion or input/output of data that uses a computer for information transfer. Multi-media presentations using graphics, real-time video, audio or text-based communications are examples. Distance Education (DE) - Transfer of knowledge outside of the conventional classroom including print-based correspondence courses that could be supplemented with audio, video, computer conferencing or teleconferencing Engineers - As with architects, engineers practice under the provisions of a provincial statute requiring them to be self regulating. Engineering is subdivided into a number of disciplines including geotechnical (soil), hydrotechnical (effects of water), structural, and fire protection. Expert Systems - A method of access to a body of knowledge that is structured so that any access will allow a systematic search and retrieval of pertinent data. For example, the University of Manitoba developed an expert system for interpreting the building code. Face-to-Face (FtF) - Communication where the participants can see and talk to each other in person, and is contrasted to on-line communication. The exchange is characterized by body language and immediacy of exchange without a record. Groupware - Groupware is software with computer conferencing elements that allows group communication. 9 On-the-job learning - A curriculum is delivered directly into a conventional work environment. Changes to the job situation are not anticipated and the elements of time constraints and peer effect are studied. Traditional Classroom (TC) - This is the conventional learning environment involving a room, teacher and students, usually conducted face-to-face. Class size and orientation (rows, round table, small groups) will vary based on logistics, subject matter, and pedagogy. Virtual Classroom (VC) - The use of computers or electronic media to deliver a curriculum. Computer conferencing is the main element of this delivery system and has all the appearances of attempting to simulate the best elements of the traditional classroom and face-to-face communication. 10 Chapter 2 - Literature Review Traditional, Distance and Virtual Classrooms In this chapter I will: • discuss the advantages and disadvantages of computer conferencing in educational settings; • compare, face-to-face learning with computer conferencing; and • discuss the formation of virtual classroom communities of learning. Many studies cited below compare computer conferencing with traditional and distance education systems (Smith and Kelsey, 1987; Kaye and Mason, 1989; Levinson, 1990; Davie, 1988; Harasim, 1987, 1990). All of these studies found that computer conferencing displays positive results when compared to other modes of delivery. However, Hiltz (1994) notes no significant difference in mastery between virtual and traditional classrooms but cautions that other educational outcomes need to be measured to fully examine the effectiveness of virtual classrooms. She recommends that the following variables be examined: access to educational experiences, access to professor, course participation, ability to express ideas, level of interest, ability to synthesize ideas and see connectivity, and computer comfort. Only by researching the particular variables can we identify the advantages and disadvantages of virtual classrooms. Advantages and disadvantages of computer conferencing Computer conferencing is the grouping of a number of computers to promote real-time or asynchronous communication. Applied in an educational setting, this can be described as a "virtual classroom." This section will investigate the advantages and disadvantages of computer conferencing in educational settings. 11 As noted by Davie (1988), when compared to a traditional "correspondence course", the virtual classroom allows for faster information exchange. The interactive nature of computer conferencing makes students feel they have control in their studies. As a result, student confidence is boosted. More assignments are submitted and more courses are completed. Harasim (1987) speculates on some other advantages of computer conferencing, some of which are very similar to those mentioned by Davie. These are: • increased quantity and intensity of interaction; • access to the collective, written knowledge and support of other students; • a more democratic and sharing environment; • convenience of access: the 24 hour class. A further advantage of the virtual classroom is the element of asynchronicity. Burge (1994) notes that the flexibility of the virtual classroom allows adult students to study within certain time blocks from their own desks at work. Harasim (1987) sees asynchronicity as deleterious, owing to the lack of immediate responses from the instructor and other classmates. Further student problems are noted by Harasim (1987) and Burge (1994): • having too much information to process, which can cause difficulty in synthesizing ideas; • the repetition of ideas may be viewed as boring; • the difficulty in following on-line discussion threads; • the loss of visual cues and the immediacy of face-to-face communication; 1 2 • wrist and eye strain; • the pressure to log on frequently, which demands time; and • the self imposed exclusion from discussion (lurking). Information management appears to be a key facet of computer conferencing. Harasim (1987) notes that" the first days and weeks of using a new communication medium can be stressful" (p. 129). Hiltz and Turoff (1987) indicate that the convenience of electronic access to study in the home and at work is usually thwarted by the inconvenience of interruption and also by the need to establish new study habits and time management strategies. So, although the virtual classroom provides the benefits of asynchronicity, a democratic environment, convenience of access, increased motivation and favourable attitudes towards learning, it is accompanied by the disadvantages of information overload, loss of visual cues, health concerns, the inability to follow on-line discussion threads, and interface encumbrances. Face-to-face learning compared to computer conferencing Hillman et al (1994) propose that fear of the computer is part of every learner's repertoire in a virtual classroom. They contend that an interface is more confounding than face-to-face communication because of the less dynamic facet of the exchange. The learner must rely on fewer contacts and less corrective feedback. Immediate tutor and peer access is lost. For this reason, to be effective, the interface must offer ease in creating and responding to messages. Hacker (1994) concludes: "The problem of computers not increasing and maybe even decreasing productivity is attributable to the fact that computers are difficult to use for most people" (p. 3). He argues that the computer literates are not aware of this. Computer programmers often type in 13 excess of 60 wpm and tend to assume that keyboarding skill is not a major impediment to interface dexterity. Thus the encumbering and confounding aspects of computer conferencing require the development of "learn to learn" (Eastmond, 1994) strategies. The learner interface must be incorporated into the curriculum to be valued, appreciated and used. Although computer conferencing may introduce elements of interactivity into the curriculum, concerns with the interface must be addressed for educational outcomes to be as effective as face-to-face learning. Harasim (1990) points out that like face-to-face learning, on-line learning promotes social interaction. She states: "Historically the social affective and cognitive benefits of peer interaction and collaboration have been available only in face-to-face learning. The introduction of on-line education opens unprecedented opportunities for educational interactivity" (p.42). However, Hiltz et al indicate that educational interactivity may suffer from the inequality of participation: "... in the virtual mode, there usually emerges a dominant person who tends to receive a disproportionate number of messages" (p.230). Commenting on this dominance, Bales and Borgatta (1955) write: 'This tendency toward inequality of participation... has summative side effects on the social organization of this group. The man (sic) who gets his speech in first begins to build a reputation" (p.34). In relation to participation Hiltz (1994) concludes that: "CC tends to produce relatively more of the types of communication that support high-quality decisions, and relatively less of the types that lead to group agreement" (p.243). She also claims that: "...asking for opinions appear to help the CC groups and harm the quality of decisions in 14 FtF." Hiltz notes that: ".. .this analysis may be pushing the current data beyond their reliability limits, and should be taken only as suggestive of an intriguing line of research" (p. 244). Some of the differences between traditional and virtual classrooms depend upon the nature of the communication. Insofar as face-to-face communication provides the participants with more information and cues, Rice (1987) hypothesizes that the virtual classroom could lead to changes in the socialization patterns and political structure within organizations. "Unlike face-to-face communication, where relationships among individuals are influenced by socioeconomic status differences, norms, physical appearance and speech behaviour, individuals using CMC are not required to use indirect paths of interpersonal connections to communicate with others, perhaps socially distant users. They can simply send a message to any person or set of persons on the system" (p. 91). The development of virtual communities Mason and Kaye (1989) see the development of large virtual classrooms as critical to the extension of "organizational boundaries". They indicate that this form of teaching will allow large teams of faculty, tutors, and alumni to produce a "critical mass" of users. This should build the database, expand the opportunities for curriculum development, and increase the amount and quality of human resources. The larger group lends credence to systematic problem solving ventures and information gathering and dissemination. The paradigms developed by Kaye and Mason (1990) indicate the development of a large community of users tied into an information structure to assist in understanding the large amounts of information. Rice and Cae (1983) examined the interactive components of computer 15 conferencing systems with the following results. Students exchanged information (100%), followed closely by asking questions (95%), exchanging opinions (81%) and staying in touch (84%). Surprisingly, the least mentioned highly interactive components were solving disagreements (15%), getting to know someone (14.5%), and bargaining and negotiation (18%). As noted by Eastmond (1992), computer conferencing promotes open ended discussion based on process rather than product learning. Feenberg (1989) supports this view by stating that: "...computer conferencing favors open-ended comments which invite a response, as opposed to closed and complete pronouncements" (p. 26). He goes on to say that computer conferencing has the following strengths: "computer conferencing supports both large and small group interaction; allows interaction with other individuals or the instructor, encourages rapid feedback; and provides information exchange based upon the student's own schedule" (p. 26). Thus a virtual community is developed. Morrison and Lauzon (1992) describe a conferencing medium that counters the views of the new paradigm spoken of by Mason and Kaye (1990) and Turoff (1990). Morrison et al (1992) contend that the bulk of research has focused on "..learning and design issues, neglecting the all important area of how we can facilitate students' actual linkage with host computers so that they may, in fact, participate in on-line education" (p.6). Hiltz (1989), when schooling students in the acceptance of the virtual community, considers three variables: subjective satisfaction, use and perceived benefits, and successful implementation or adoption. She blames the conflicting results of a number of studies on the "different indicators of acceptance, different user populations, or differences among the systems" 16 (p.387). She concludes that: "Evidently it is 'personal' networking that provides the contacts that may aid professionals in their careers, and those who do not feel that the medium is personal in nature will not try to use it for such activities"(p. 394). She concludes that acceptance of computer conferencing is multidimensional with moderately positive correlations between subjective satisfaction and benefits. However, computer conferencing usage, subjective satisfaction and perceived benefits may vary independently. Summary Computer conferencing places the student at the center of knowledge (Kaye and Mason, 1989). In the resource-based approach, which the internet may provide, the teaching can be more intimate and more cooperative. Although large communities of learning may connect people and ideas in a critical mass (Kaye and Mason, 1989) computer accessibility, user familiarity and technical difficulties remain problematic (Morrison et al, 1992). Computer conferencing permits group exchanges, whereas traditional correspondence courses do not. Advantages over traditional distance education courses (Harasim, 1987 and Davie , 1988) were noted as the democratic environment, ease of access and asynchonicity. Disadvantages were seen as information overload and the loss of visual cues. However, the ability to download information is seen as one way of capitalizing on asynchronicity and overcoming information overload. Two other advantages are peer interaction and the ability to communicate one-to-one, one to many, or many to many (Harasim, 1990; Burge, 1994). 17 Content development and course design This section discusses: • the elements required to design an effective computer conferencing system; • the importance of modularization; • computer conferencing screen activities; • difficulties with the computer interface and methods to overcome them; • the importance of knowledge building and expert systems; • creative and critical thinking with respect to computer conferencing; and • the importance of the learner and facilitator. According to Scriven it is important to modularize courses, evaluate and adapt materials and to develop student and staff support. Scriven (1991) points out that this modularization is needed as professionals "...would be more appropriately served by short modules which assist in the solution of immediate problems" (p.300). In addition the set up of the conference depends on the structure of the groupware. "Foster (1985) suggests a main conference, private conferences, public sub-conferences, document workspace, a bulletin board and an area to collect surveys" (Eastmond, p. 30). Mason (1988) divides conferences into interactive components that simulate the college functions: virtual cafes and faculty lounges, a technical conference for questions, and the main conference. Besides the interface, several scholars propose that upper division baccalaureate and graduate seminars are a natural fit for computer conferencing because of the manageable class size, the discussion expectation and the closer professor-student relationship (Harasim, 1987; Roberts, 18 1988; Hiemstra, 1989). Although the professor-student relationship is important, Davie (1988) uses learning partners or peers to assist in course development and progress. In her research study, participating students reported satisfaction with the learning partner exercise. Tessmer (1988) indicates that the use of the subject specialist encourages another type of professor-student relationship, which can add expert knowledge to the computer conferencing environment. However, he supports the use of surveys, questionnaires and more testing points to overcome the proclivity of the subject specialist to dominate discussion. The use of questionnaires provides a framework for student self analysis, thus assisting in addressing learning outcomes, critical to the success of the project and important for consensus in the computer conferencing environment. These tools draw out the subject specialists' special interests and gives the students a basis for discussion. Knowledge building The expert system model (Frye, Olynick, and Pinkney, 1992) refers to the development of a body of knowledge that can later be accessed by user groups. Through this access, a larger and more integrated database evolves. This type of knowledge building, which involves the learner as an active participant is viewed as a critical component of effective education and integral to computer conferencing (Hiltz, 1994; Harasim, 1990). The plethora of information on the internet is conducive to knowledge building exercises. Harasim (1990) notes that: "Active sharing and seeking of information and playing with ideas is central to on-line collaboration. The shared text-based space seems particularly conducive to stimulating brainstorming activities and group synergy, sparking ideas or identifying new associations" (p.54). She notes that: "...learning is much more an evolutionary, sense making, 19 experiential process of development than of simple acquisition" (p. 55). She expands upon this by stating: "From my observation, in order to facilitate sense-making and knowledge building within on-line group discussion activities, the system needs to support three educational processes: idea generating (and gathering), idea linking and idea structuring " (p. 55). Harasim (1990) connects divergent thinking with idea generating, and convergent thinking with idea linking. Idea generating is the compilation and formulation of ideas and is expressed in computer conferencing. She contends that idea generating is not yet that advanced within the computer conferencing environment. Feenberg (1990) and Hiltz (1986) note that information vastness floods the minds of the users, leading to despair and eventual withdrawal from on-line courses, so idea linking and generating are key to program success. Social interaction with data has been shown to accentuate interest among computer conferencing groups (Harasim, 1990), but it appears that data manipulation amongst users poses the biggest challenge to any computer conferencing learning (Levinson, 1990). Expert system development depends on the manipulation and posting of data to achieve idea structuring (Harasim, 1990). Harasim (1987) notes that: "Students 'have the floor' and control (to a considerable degree) how much they write and participate" (p. 133). For learning to take place it is imperative that a collaborative learning environment be developed with focused discussions: "...particularly within the seminar activities, to avoid 'on line brainstorming' a situation in which comments do not relate to and build upon one another" (p. 133). The learner and the facilitator roles Knowledge building is inextricably tied to the development of on-line relationships, both with peers and the professor. An active moderator is critical to program success, but the democratic nature of the on-line environment places more stress on task development and peer interaction (Mason, 1988; Hillman, Willis and Gunawardena, 1994; McCreary and Van Duren, 1987; Harasim, 1990). McCreary and Van Duren (1987) outline three roles considered paramount in any conferencing architecture: "...individual participant; the "Conference moderator" and the "diffusion manager" (p.l 17). "The diffusion manager must entice members of the organization to engage each other via on line communication" (p. 118). The professor, instructor or diffusion manager, to be effective, must play a non-dominant facilitator role. Wilkes (1991) points out the weakness in all three roles: "From observations and interviews it was concluded that the computer conferencing system exaggerates an instructor's weaknesses. If instructors are boring in a face-to-face setting, they can reach indescribable depths of insipidity..." in the conferencing environment (p.49). Davie (1988) "...reported that on-line courses rated better with students who were motivated and well-prepared and who took advantage of the increased chance to interact with their professors and peers" (p. 58). Harasim (1990) argues that motivation and anxiety are reduced when working with peers instead of with the instructor. She elaborates: "It may be this building of new relationships that facilitates a better grasp of the material" (p. 44). 21 Facilitator role Zemke (1981) indicates that research on adult learners tends to focus on asking questions of preference: "A trainee may prefer listening to lectures but learn best by practice and application exercises" (p. 10). So, task development and peer interaction may be stronger motivating factors for the adult learner than facilitator involvement. This is borne out by Rice (1987b) who examines task and socioemotional content of a number of on-line statements that occur through a large computer conferencing environment. He measured "...socioemotional content which is defined as interactions that show solidarity, relief, agreement, and antagonism, tension and disagreement. He also measured task or dimensional content which is defined as interactions that ask for or give information or opinion" (p. 93). He uses the term "professional" to describe the task oriented nature of some computer conferencing communication and concludes that: "Even a professionally oriented CMC system, involving users who do not otherwise know each other, can support a reasonable amount of socioemotional content" (p. 101). Within his study he notes that nearly 30% of the sentences of the students in a computer conferencing course were of this socioemotional nature, which is a significant amount of affective communication for a task oriented environment. Burge (1994) indicates that the facilitator needs to synthesize and summarize ideas to promote learning. She sees any restrictions on group focus as an impediment to group dynamics, especially if it occurs in the early stages of computer conferencing. Assisting adults in learning is seen by Galbraith (1989) as a transaction process: "...in which the facilitator interacts with learners, content, other people and material to plan and implement an educational program" (p. 10). He goes on to say that most facilitators are guides through the educational process and are expert in content but not well schooled in program delivery. Some of the ideal roles assigned to 22 the facilitator are counselor, content resource person, learning guide, program developer, and institutional representative. As stated by Daloz, (1989, p.l 1), besides accenting interpersonal skills and being adept at transmitting content, the facilitator must "...have the ability to assist adults in the process of learning how to change their perspectives...." Further, the "element of good teaching becomes the provision of care rather than use of teaching skills and transmission of knowledge" (p. 11). The learner Focused discussion has been identified (Mason 1988; Harasim 1990) as key to the advancement of learning and the constructive scaffolding of concepts. This would preclude brainstorming because it detracts from the information flow and fails to engender relevant comfortable ideas and feelings (Burge, 1994). This detraction from the flow is identified by Burge (1994) as an attempt to create the "volatility of conversation" (Grint, 1992) without producing prattle or what one Open University student has called "chewing gum for the eyes" (Grint 1992, p. 160). Burge (1994) sees sub-conferences as a reflective tool to promote meta-cognitive learning strategies. Burge (1994) concludes that sub-conferences should "encourage students to contribute cogent and focused messages to the appropriate sub-conference" (p. 38) and thus eliminate the 'chewing gum'. Although impromptu creativity can result in unproductive contributions, there are strengths seen in the anonymity and ability to contemplate responses from other students (Mason, 1988). However, some learners "lurk" (refuse to interact with others) (Mason, 1988; Harasim, 1990) impeded by unfamiliarity with the technology. Similar to the concern raised by Mason et al (1989) is that of Lewis (1993) who discusses the learner's potential reluctance to face the 23 "interactive mirror," although group social commitment should ideally encourage the desire to learn. Taking on the responsibilities of course direction and assessment of the other student's conferencing contributions is integral to the vitality and evolution of the on-line curriculum. According to Burge (1994) learner contributions may be encouraged by the anonymity of the system, but anonymity may be lessened by the fact that contributions are recorded. Thus, making a point known, especially within contexts that involve sensitive material, can detract from progressive and positive discussion and discourage learners from contributing. She suggests that research should be expanded to examine: "the transferability of the results and to develop our understanding of how learners behave strategically in a CC context and perceive their tasks" (p.39) and thus reduce the student's sensitivity to conferencing contributions. Another important learning variable is student feelings about computer conferencing. Hillman et al (1994) examine and quantify learning approaches by studying students' feelings about the delivery system. They propose that the adult learner is unaware of planning learning strategies, but that certain standards arise:"... establishing study patterns, scheduling effective study time, working with others,... seeking specific tasks and structure, and demonstrating competence to the instructor" (p. 138). Comparing the typical distance learning and computer conferencing environments, they find that students require inordinate time and resources which detracts from learning. Thus student's affective responses must be taken into account and should be assessed in course evaluations. Wilkes (1991) also explores the motivational orientation of various learners. His project studied 156 students enrolled in an electronic distance education course at Utah State University. 24 Independent variables were noted as: motivational orientation of the participants, demographic data, and course data. The dependent variables were the participant's perceptions of the learning environment in the areas of satisfaction, material environment and involvement. Some of the conclusions reached are that: • electronic data material is inferior to other forms of educational media but the frustrations of full time students are similar to those of part time students studying in traditional classroom settings; • teachers have a significant bearing on student involvement; • "motivational orientations do not appear to be a factor in the decision of individuals who dropped out of their computer conferencing classes. Time, home and work demands seem to be the major reasons why they discontinue their classes" (p. 49). Wilkes (1991) concludes that: "There appears to be little practical relationship between motivational orientations and participant's satisfaction (p.49). Summary Course modularization, with increased test points is critical to on-line program success (Tessmer, 1988; Scriven, 1991). However, the large amounts of discussion data must be structured to the content and fashioned into a framework so students can compare and analyze idea development that will later support an effective body of knowledge (Harasim, 1987; Mason and Kaye, 1989; Eastmond, 1992). Expert (subject specialist) or facilitator involvement must be guided to reduce substantive conflict and the domination of discussion (Tessmer, 1988). Learners may "lurk" due to the fear of the technology, preponderance of contributors, or the inability to offer feedback on the medium or course content (sub-conferences) (Burge, 1994; Mason, 1988). Further, it was found that student motivation and satisfaction were not related (Wilkes, 1991). 25 Adult professionals and work This section will discuss: • the peculiar nature of adult learners at work; and • the general characteristics of adult learners; General Characteristics Adult learners appear to be particularly well suited to the computer conferencing environment due to their proclivity to be autonomous, goal and activity oriented, and the desire to bond with others (McCreary and Houle, 1990; Johansen et al 1979). The work environment may build bridges to other forms of communication and computer resources (Johansen et al 1979) and thus support activity oriented education. Johansen et al (1979) note that on-the-job learning outcomes are affected by how, with whom, when and where people work. They noted that a good many people operated their computers from home, which is a significant variable in computer conferencing acceptance as people are more likely to favour computer conferencing learning if it fits in with their schedule. Another general characteristic of the adult learner is his resistance to the use of computers (Lewis 1988). Other research indicates that "..those who feel that the introduction of computers preempts their judgment, offers less room for individual decision making, and limits their control over their work space, tend to be more anxious and resistant to the technology because it threatens their sense of self (Licata and Zuboff, 1988, p. 5). They found that technology acceptance was affected by skill acquisition and not by age. 26 Age determinates may be further explained by Simard (1988) who describes phases of adult learning developed from a three year study conducted in 1988 and notes a number of life phases that could affect program objectives and development. Data were gathered through semi-structured interviews and two exams; one retrospective over the previous 5 years of experience and the other prospective over the following 5 years. Phases involved perceptions of learning and the degree to which each would be recognized and utilized within the training environment. The perspective of these phases provides a guideline for the interpretation of the age variable. The study notes that the age groups 28-32 require the development of occupational goals. The 38-42 year old is eager to develop avenues of learning outside of the normal institutional setting: "...for the adult of this age, the reality of continuing education is generally confined to on-the-job training; he seems to totally reject organized or institutionalized activities of adult education..." (p. 25). This age group, more than any other, prefers to concentrate energies within the work environment, but continues to learn. Consideration of these phases may be integral to program development and content delivery. Enhancing content to make sure that it is relevant is seen as one element that can be more fulfilling for the student. However, the need for a personal sphere of control "..is considerably more complex. This complexity is why the concept of personal control is a particularly salient one in distance education theory and practice" (Garland, 1994, p. 51). Using an ethnographic study Garland (1994) found that: "...personal control means students being in control of their personal learning situation, that is, being in a position to be self-efficacious. It does not mean their being in control of the entire educational transaction" (p.47). Paramount to any conferencing program is the feeling that the student is having an impact over the material and is able to contribute to the discussion. Garland notes: "These mature distance education students 27 have acquired the status of adults; their difficulty is psychologically maintaining this status and power while undertaking the role of student. Their needs for respect, personal control, and fulfillment are often frustrated" (p. 48). But interdependence is also important to the adult learner. Adult learners want "...a solid body of knowledge with links to prior understanding, practical experience and access to resources to fill in prerequisite knowledge, opportunities for interactivity, good feedback and if problems arise, empathetic counsel" (p. 55). The need to communicate and work towards autonomy is critical to the adult learner and is supported by Boud (1994) and Schlossberg (1994) who say : "...autonomy is a recognition and acceptance of interdependence" (p. 55). Summary Adult learners need a sense of identity and integrity and clear goals (Garland, 1994; Shaw, 1992). Greater sphere of control and cohesiveness of the adult learner is more important than the age of the learner (Simard, 1988; Wilson, 1994; Garland, 1994). However, age phases may dictate the acceptance of on-the-job learning initiatives (Simard, 1988) and older students may be more resistant to the technology than their younger counterparts (Licatta et al, 1988). Costs Costs will dictate system implementation ideas and project decisions. In terms of the year 1984 Rice (1987b) states a: "...survey found that half of the sampled organizations expected CMCS would be a significant portion of their 1987 telecommunications budget up from 5% in 1984. A 1985 survey reported that 71 percent of Fortune 1000 companies planned to have CMCS (computer mediated communication systems) by 1986. By the end of 1985, there were over 1 28 million commercial 'electronic mailboxes' sending 13.5 million messages per month, prompted by the installation of nearly 6 million personal computers with communications capabilities" (p.200, 201). The expansion of the internet and the extended resource capabilities is also considered by Santoro (1994): "...as of the end of 1993 it is estimated that thousands of individual TCP/TP networks worldwide are connected to the internet, with at least 2 million individual computers attached (Elmer-DeWitt 1993, p. 73)". Santoro (1994) states that: "..in the summer of 1993 almost 13 million U.S. workers used computer networks to "telecommute" to the office on at least a part-time basis" (p. 77). This explosion in the potential of computer conference courses leads to questions about their economic viability. Shifts are apparent from "..major cost elements in distance education, as opposed to face-to-face education, from student-related, recurrent, teaching costs to course-related materials development and infrastructure costs" (Kaye 1987, p. 154). Phelps, Wells, Ashworth and Hahn (1991) state that the literature up to 1991 concludes that distance education can be less expensive than resident instruction, depending upon student enrollment and the fixed costs of course development and delivery. The costs associated with the development of the internet has greatly reduced the charges for production. E-mail capability has allowed for the ease of two way communication. Web site set up and rental costs run in the order of $1000 and the conferencing software development at a further $1000. As a result, curriculum development charges remain the number one cost factor in development of any course (Scriven 1991). However, the resource and time constraints of facilitators do enter the equation. According to Davie (1988) the use of conferencing and the role of the facilitator are major cost factors. 2 9 Chapter Summary This review of research illuminates some of the findings concerning virtual classrooms. These findings include: • The virtual classroom is often compared to the traditional classroom to determine the effectiveness of the computer conferencing system. Advantages and disadvantages are elicited but it appears that key elements of comparison are asynchronicity and the convenience of access; • Computer interface encumbrances and information overload must be considered in system design; • The nature of the computer conferencing encourages a democratic environment among learners and facilitators leading to high quality task decisions overcoming social barriers that may be present in traditional classrooms; • Information and resource exchange may encourage the development of virtual communities but this is tempered by computer accessibility and familiarity difficulties; • Course design will not only depend on modularization and the ease of navigation through the system, but also the roles of the learners and facilitators; • Knowledge building exercises have the potential of developing an evolving database; • Adult learners are well suited to computer conferencing as they are goal oriented, cohesive, and autonomous; • The age of the learner did not have as much effect on technology acceptance as skill acquisition; • Personal sphere of control is a characteristic that identifies adults who should work well in on-line systems; • Studies indicate that an effective system can be developed at reasonable costs and that curriculum costs far outweigh software costs. The review also raised questions for future research. These include: • Does personal sphere of control affect views of the virtual classroom? • Will the virtual classroom increase educational efficiency? • Are there other ways of overcoming the encumbrance of the interface? 30 Does the convenience of computer access have a bearing on views of the virtual classroom? What other learner/facilitator concerns affect virtual classroom acceptability? What type of messages will be generated in on-line research and how will they compare with other face-to-face communication (Rice and Cae, 1983)? Is there better facilitator access in the on-line environment than in the traditional one? How will the development of a virtual community be affected by the computer conferencing groupware design? Chapter 3 - Methodology 31 Research basis and project description The idea was to develop an on-line computer building code course and testing knowledge gains between curriculum and non-curriculum groups, and attitudes towards the medium. Objectives were vetted through a small group of team members (Appendix I). A project timetable was established (Appendix G) with an orientation meeting as the first event. Refer to Figure 1 for the project flow chart. Codeworks utilized various activities to ensure that participants were familiar with the technology before beginning. These included: • project team formation was encouraged by introducing students to each other through e-mail with topical and general discussions; • gathering of feedback and project development occurred when students expressed concerns about internet access, netscape versions, and text appearance; • orientation meetings and demonstrations were conducted to introduce the internet capabilities, assemble questionnaire data and demonstrate conferencing objectives; • on-line commencement with issues discussion and e-mail trial occurred when the project network was developed and could be tested with students; • the curriculum was then delivered in sections or 'chunks' with discussion issues at the end of each section. Students were alerted (by phone and e-mail) to change modules when all section tests were received; and • data collection and closure occurred when over ten tests from each group were received and discussion had ended in all the conferencing sections. Events Data Gathered Conferencing development Web site Development Project Beginning -Orientation Meeting Questionnaire #1 General data Age, code, computer and work exDerience N Pre-test Building Code 4-Definitions Short and long form Phone Pr 3-lnterviews .... ... V Questionnaire #2 Curriculum Delivery Posttests Building Code Views on Conferencing Computer experience Work Experience Code Experience Independent variables Spheres of Control Expectations about the conferencing System Computer Attitudes Conferencing E-mail True and False Short and Long Form Post-Questionnaire #3 - ± Virtual Classroom Rating Facilitator Access Instructor Rating (Dependent Variables) \. Post-Interviews • Views on Computer Conferencing S Figure 1 - Project Flow 33 Through on-line instructions and the pre-research seminar, each subject was informed of the delivery method, expectations and general course content (four subsections of the 1992 B.C. Building Code, 3.1.2, 3.1.3., 3.2.1, and 3.2.2.). Participants were encouraged to develop on-line internet linkages over the next several weeks. General code discussion topics were introduced initially, through which users became familiar with conferencing and the internet technology. As an incentive to participation by both groups, guests (subject specialists) from an arm of the National Research Council involved in building code issues were invited to take part. Warm up discussion on 'hot topics' led the on-line exchange. The hot topics were presented through a discussion paper, posted on-line supplemented with inserted questions. For example, examining the controversial issue of building stucco cladding prompted participants to develop their interface skills by using e-mail, web site searching and conferencing. Two groups were then created controlling for age, work and building code experience (derived from questionnaire 1). Both groups had access to the conferencing section of the web site and the section reviews, but only Group I had access to the on-line curriculum. After the initial discussion phase on stucco, Group I participants began with modular sections of the building code. Each section was followed by a review which was primarily submitted on-line with some discussion. Pre-tests, phone interviews, and two pre-questionnaires (one and two) were conducted before the students went on-line. Curriculum modules were then posted on the web site in four stages followed by section reviews (posttests). Students could then conference on the section review topics or discuss social issues within the coffee shop section. Communication between groups was recorded on-line through the conferencing module and in the e-mail 34 database. Questionnaire 3 was then circulated to and completed by all the students, followed by phone post-interviews leading to project closure. A moderator (co-investigator) was responsible for posting relevant data to the main web site, coaching discussion and answering queries. Facilitators were code professionals working within a government environment, who offered curriculum and code guidance. A conferencing protocol and facilitator guideline document assisted each group in navigating the conferencing and facilitator areas. It was decided to restrict access to the conferencing section to participants to protect the project against outside interference. Thus passwords were assigned for each conferencing level. General public conferencing traffic would disrupt students and confound data collection. To protect the confidentiality of discussions and to structure group discussion five conferencing areas were defined: a) General code conferencing Participants from both groups entered this conference and discussed focused code issues on-line. Free access to this common conference was provided for all participants. b) Facilitator conference Facilitators were provided with the opportunity to discuss sensitive code issues within this conferencing section, thus protecting the confidential nature of some code interpretations. This level required confidentiality and therefore was restricted to facilitators. Discussion levels included policy and code interpretation. 35 c) Groups I and II - Student's conferences Group I received on-line curriculum material and section reviews. Group JJ could access section reviews (posttests) extracted from the formal curriculum of Group I. d) Coffee shop This conferencing level allowed for social interaction between participants without cluttering the other main conferences. Elements of the discussions were examined for content and concept specific data and compared to the group interaction. General comments on program delivery were also accepted here and moderated as discussion proceeded. Summaries were based on the feedback and suggested comments for improvement. Subject selection The on-line course was developed with participants selected through a faxed request for participation sent to 120 architectural offices within Greater Vancouver. On February 20, 1996, a seminar was held to introduce the project to those able to attend. During this session, the research objectives were presented. Internet technology and its development by service providers, provincial and municipal agencies were also discussed. Thirty-nine participants completed project consent forms and questionnaire 1 (see Table A-l). They were placed in two groups using stratified sampling (Group I and Group H) controlling for variables that could confound the study including age, work experience, computer and code knowledge. Twenty subjects were placed into Group II and nineteen in Group I. Group size was restricted to keep events manageable for the computer conferencing module and to allow evaluation of subject achievement. Each design and municipal office was informed of the confidential nature of the 36 data and were instructed concerning the research proposal, the methods of reporting, communication, and testing. Most of the facilitators were not able to take part in the conference as they could not develop on-line linkages and pressing workload demands restrained their participation. Three facilitators from the cities of Richmond, Burnaby and Vancouver contributed to the facilitator's conference eleven times. On-line discussion exchanges explored code concepts, discussed technical difficulties and debated the curriculum presentation. Conferencing on-line required active phone, e-mail contact and faxed data to encourage participation. Instruments Of the 41 subjects who started in the Codeworks project, two left after completing pre-questionnaires due to electronic access difficulties, and four because of workload demands. These six are not included in the study. Two entered the project after the pretest and their data is included in the study. The matrix in Table A-1 describes the instrument completion for the subjects and includes those that remained in the project. The pretests were completed by 17 subjects (6 for Group JJ and 11 for Group I). The posttests were completed by 27 subjects (15 from Group I and 12 from Group II). Questionnaire 1 was completed by 39 subjects; Questionnaire 2 by 37; and Questionnaire 3 by 24. Twenty-two subjects were pre-interviewed and 32 were post interviewed. Subject attrition was heaviest in the pretests due to the extensive work (two to three hours) required to complete the examinations. Pre and posttests (Appendix F-4 and F-5) were formulated to test code knowledge and were based on standardized tests prepared by the Building 37 Officials Association of B.C. The performance on these tests formed the basic criteria to answer research question 1 and compare code knowledge gains between Groups I and LT (curriculum and non-curriculum). Pre and posttest questions were short and long form requesting clarification of code definitions and code concepts. Questionnaire 1 was completed by 39 of the subjects and gathered demographic and work experience data (Appendix F-l) to assist in placing the students in each group. Questionnaire 2 and Questionnaire 3 measured variables as follows: • computer terminal access - using a Likert scale this variable consisted of three questions, two asked in questionnaire 2, whether the student had a computer at home, or at work and one asked in questionnaire 3, is taking on-line courses more convenient and is expressed as a Pearson Correlation; • computer attitudes, sphere of control based on the images of yourself section of questionnaire 2 and is expressed as a Pearson correlation based on the sum of the answers to the following: • When I get what I want, it's usually because I worked hard for it • I prefer games involving some luck over games requiring pure skill • Even when Pm feeling self confident about most things, I still seem to lack the ability to control social situations • I can learn almost anything if I set my mind to it • I have no trouble making and keeping friends • It's pointless to keep working on something that is too difficult for me • I'm not good at guiding the course of a conversation with several others • On any sort of exam or competition I like to know how well I do relative to everyone else • I can usually establish a close personal relationship with someone I find attractive • My major accomplishments are entirely due to my hard work and ability • When I make plans I am almost certain to make them work • When being interviewed, I can usually steer the interviewer toward the topics I want to talk about and away from those I wish to avoid • I usually don't set goals because I have a hard time following through on them • If I need help in carrying off a plan of mine, it's usually difficult to get others help • Competition discourages excellence • If there's someone I want to meet, I can usually arrange it. • Other people get ahead just by being lucky • I often find it hard to get my point of view across to others • In attempting to smooth over a disagreement, I usually make it worse 38 • interpersonal sphere of control or collaboration index. Dependent variables are: • achievement (pre and posttests see Appendixes F-4 and F-5) • virtual classroom overall rating, course rating, course outcome index, interest and synthesis index and instructor rating. Both pre and post interviews (Appendices F-6 and F-7) were conducted to support the data gathered in the questionnaires and tests. The interview questions were formulated using procedures identified by Hiltz (1994) and probed work, building code experience, computer attitudes, and conferencing expectations. Table A - l Instrument Matrix SUBJECT Q. 1 Pr.l Pr.T. P.T. P.l. Q.2 0 3 1 X X X X X X 2 x x x x 3 x x x x x x 4 X X X X X X 5 x x x x 6 x x x x 7 x x x x x x 8 x x x x x 9 x x x x x x x 10 x x 11 X X X X X 12 X X X X X X . 1 3 x x x x x x 14 X X x x x x 15 x x x x x x x 16 x x x x x x 17 x x x 18 x x x x x 19 x x x x x x 20 x x x 21 x x x 22 x x x x x 23 x x x x 24 x x x x x 25 x x x x x x x 26 x x x x x x 27 x x 28 x x x x x x 29 x x x x x x x 30 x x x x 31 x 32 x x x x 33 x x x x x x x 34 x x x x 35 x x x x x x 36 x x x x x 37 x x x x x x 38 x x 39 x x x x x x Totals 39 22 17 27 32 37 24 Q -1 = questionnaire 1 Q - 2 = questionnaire 2 Q - 3 = questionnaire 3 Pr. T = Pretest Group 1 = Subjects 21-39 PT = Posttest Group 2 = Subjects 1-20 Pr. I = Pre-interview PI = Post-interview X denotes instrument completion, blank is not completed 40 Questions 1. Will there be significant gains in code knowledge between Groups "I" and "JJ" when Group "I" is exposed to a computer conferencing delivery system with a defined curriculum and "U" is exposed to a computer conferencing system without a defined curriculum? Answering this question involves comparing achievement on pre and posttests. The research methodolgy for the first question is developmental and quasi-experimental, studying two groups of approximately twenty architects (each), assessing achievements on standardised building code tests (pre and posttests) after exposure to a computer conferencing environment, with on-line curriculum modules delivered over the internet for a six week period with: • a group of students (Group I) learning on-the-job responding to an on-line building code course with a defined curriculum. These students can access computer conferencing; and • group of students (Group H) learning on-the-job, responding to an on-line building code course with summative tests, but without a defined curriculum. These students can access computer conferencing. 2. Will students with a greater sphere of control on either the personal or the interpersonal levels be more or less likely to regularly and actively participate on-line, take another on-line course, rate the virtual classroom as easier and more effective than the traditional one, and have a positive view of the instructor or facilitator? To answer this question measurements of the independent variables personal and interpersonal sphere of control were conducted using the images of yourself section of questionnaire 2. Views of the on-line educational experience were measured using the variables virtual classroom rating, course outcome, instructor rating, and views on taking another on-line course and then compared with the independent variables using a Pearson correlation. 8 Hiltz (1994) describes the various spheres of control that have also been researched by Rotter (1966) and Paulhus (1983). Personal sphere of control is described as: "...a subscale of measures being a result of one's effort rather than "luck". Interpersonal control measures control over people in groups" (p.68). Measurements of these effects are contained in the images of yourself section of questionnaire 2. 41 3. Will students who experience group or collaborative learning in the virtual classroom have positive9 views of on-line course work? To answer this question an index measuring collaboration was compiled using questionnaire 3 measured across the variables (virtual classroom rating, collaboration index, terminal access, time on-line, and the convenience of on-line courses) that measured the effectiveness of the computer conferencing and made comparisons to the traditional classroom. Measurements of the type of conferencing exchange provide an indication of the type and complexity of interaction between participants. 4. Will conferencing students with good computer terminal access at either home or in the office, who spend more time on-line and view on-line courses as more convenient report positive views of on-line courses across a number of variables10 and will students report positive views of the on-line course across the same variables? To answer this question computer access was measured across the variables noted and frequencies were measured for each variable. The case study method is used in the last three questions to examine the virtual classroom mode of delivery and compare the changes in attitudes and academic achievement. It is believed that the virtual classroom will report higher subjective satisfaction than previous subject experience with the traditional classroom. Summary • Codeworks began with a small group of students who assisted in the development of project objectives. • Four research questions were asked to test knowledge gains and attitudes towards the on-line system of education. 9 Positive measurements are based on correlations with on-line convenience, computer terminal access, time spent on-line, increased communication with the students, improved access to the facilitator, experienced increased motivation by reading assignments of other students and found comments and assignments by other students useful. . 1 0 The variables measured are computer attitudes, instructor rating, interest in the course, the ability to synthesize ideas, views on the virtual classroom overall increased collaboration, course rating and course access and quality. 4 2 • Students were selected from a group of lower mainland architectural offices and events were then organized to deliver the instruments to all the students. The main groups were divided into two groups to test the need and effect of the curriculum. • The conferencing format was designed for the web site and passwords were assigned for each group and facilitators. Validity. Reliability and Design Limitations Computer conferencing on building code issues is a potential bridge between municipal and design offices. This research examined the effectiveness of municipal/client interaction on building code issues only and is not generalizable to other conferencing settings. The study is aimed at a profession that is generally familiar with computer technology. Generalizability to other groups (i.e. contractors and draftsmen) is limited due to computer accessibility and training backgrounds. 43 Chapter 4 - Results Conferencing Traffic Conferencing and e-mail use was monitored to answer question 2 by gauging the level and type of system involvement. The number and types of interactions were recorded with the following representations (Harasim, 1987) (type of interaction is bracketed): • exchanging information (7)- 30% (80/265) • staying in touch (6)- 20% (53/265) • getting to know someone (5) - 5% (13/265) • solving problems (4) - 5% (13/265) • asking questions (3)- 70% (185/265) • task oriented (2)-17% (45/265) • socioemotional content (1) - 83% (220/265) Conferencing traffic 0 50 100 Percentage expression Figure 1 a). Conferencing Traffic 4 4 The conferencing exchanges were databased for future reference. The reported number of e-mails was 95 and the number of conference submissions was 170 for a total number of exchanges of 265. Computer conferencing indexes were compiled to answer Questions 2, 3 and 4. Frequencies are reported in the following figures and tables. Pearson correlations were then developed to make comparisons Collaborative learning Questions 3 and 4 queries views on collaborative learning. The majority of students agreed on the following (see Table 1): • that the assignments were useful; • that they did not have to work as hard for on-line classes; • that they communicated less with other students as a result of the computerized conference; • that they communicated less using other media as a result of computer conferencing; • that they got more out of on-line conferencing when compared to traditional classrooms; and • that they experienced increased educational efficiency. 45 I found reading assignments of other students to be useful to me. 6 6 0 0 . f | B i P i • i- i l 1 2 3 4 5 6 7 Strongly agree to strongly disagree Figure 2. Found reading of assignments useful N=20, Mean = 2.9, SD = 1.3 I felt inhibited in taking part in discussion. Z 1 2 3 4 5 6 7 Strongly agree to strongly disagree Figure 3. Felt inhibited in discussion N=21, Mean = 4.0, SD=1.5 I didn't have to work as hard for on-line classes a e TJ 6 3 4 ° 2 i o 3 I'm 1 • 2 • m _ 12 3 4 5 6 7 Strongly agree to strongly disagree Figure 4. Did not have to work as hard for on-line classes. N=20, Mean = 3.3, SD= 1.4 I communicated more with other students as a result of the computerized conference. 1 2 3 4 5 6 7 Strongly agree to strongly disagree Figure 5. Communicated more with other students. N=21, Mean = 5.1, SD=1.4 46 I would have gotten more out of a traditional course. Did use of the system increase the efficiency of your education (the quality of work that you can complete in a given time)? 5 8 1 2 3 4 5 6 7 Strongly agree to strongly disagree 1 2 3 4 5 6 7 Strongly agree to strongly disagree Figure 6. Got more out of traditional course N=20, Mean=3.2, SD= 1.4 Figure 7. Increase in educational efficiency N=17,Mean=3.6,SD=1.4 These results indicate a positive view of the collaborative aspects of computer conferencing in accessing assignments, taking part in discussion, and in educational efficiency. These results answer Question 4 positively. However, data gathered indicated support for the traditional classroom (figure 6). Interview responses expressed concerns over system time delays and the short duration of the program. Computer attitudes Question 4 queries the views on computer attitudes after taking the on-line course. The majority of subjects found that the use of computers (Figures 8-16 - derived from questionnaire 3) was stimulating (100%), fun (100%), easy (92%), personal (72%), helpful (58%), threatening (53%), efficient (81%), demanding (54%), and desirable (65%). Thus the majority of attitudes were positive, lending credence to support to the virtual classroom environment as expressed in question 4. 47 Dull or Stimulating 20 1 2 3 4 5 6 7 Stimulating to Dull Figure 8. Dull or stimulating N = 24, Mean = 2.03, SD=0.79 Easy to Difficult 1 2 3 4 5 67 Easy to Difficult Figure 10. Easy to difficult N=24, Mean = 2.89, SD=1.22 Hindering to Helpful 12 1 2 3 4 5 6 7 Hindering to Helpful Fun to Dreary Fun to Dreary Figure 9. Fun to dreary N=24, Mean = 2.14, SD=.0.79 Personal to Impersonal 1 2 3 4 5 6 7 Personal to Impersonal Figure 11. Personal to impersonal N=24, Mean = 4.38, SD=1.29 ber of Students Threatening to Unthreatening ber of Students :,\\\ E u 3 Z 1 2 3 4 5 6 7 Threatening to Unthreatening Figure 12. Hindering to helpful N=24, Mean = 5.42, SD=1.29 Figure 13. Threatening to unthreatening N=24, Mean = 5.84, SD=1.55 48 Efficient to Inefficient 15 15 2 f K M 2 j 3 tn i 9 l l 0 0 0 1 2 3 4 5 6 7 Efficient to Inefficient Figure 14. Efficient to inefficient N=24, Mean = 2.68, SD=1.34 at £ 10 « 8 3 co 6 0 4 * 2 1 o Demanding to Obliging 1 0 9 9 1 - 1 4 1 1 2 or 1 l l 1 : » 3 ' * 5 6 7 Demanding to Obliging Figure 15. Demanding to obliging N=24, Mean = 3.54, SD=1.37 Desirable to Undesirable 14 Desirable to Undesirable Figure 16. Desirable to undesirable N=24, Mean = 2.09, SD=1.32 Interest and Synthesis Question 4 asks about interest in the on-line course and the ability to synthesize ideas as noted in Figures 17-22. Students reported more interest in the subject and the ability to see relationships between important topics and ideas supporting a positive view of the on-line course. 49 I became more Interested in the subject 10 8 tn 6 L 3 i i EE s z 4 2 + 0 7 ski 1 3 4 5 Strongly agree to strongly disagree Figure 17. Became more interested in subject. N=24, Mean = 2.53, SD=.98 I was stimulated to do additional reading Strongly agree to strongly disagree Figure 18. Stimulated to do additional reading. N=24, Mean=3.14, SD=1.06 I was stimulated to discuss related topics outside class Strongly agree to strongly disagree Figure 19. Stimulated to discuss related topics. N=24, Mean = 3.19, SD=1.08 learned to identify central issues in this field Strongly agree to strongly disagree Figure 20. Learned to identify central issues. N=24, Mean=3.05, SD=1.05 50 My ability to integrate facts and develop generalizations improved 8 c o TJ 3 CO "5 6 -4 • 2 2 E 1 2 3 4 ! Strongly agree to strongly disagree Figure 21. Ability to integrate facts and develop generalizations improved. N=24, Mean = 3.17, SD=.79 I learned to see realationships between important topics and ideas (A 12 1 2 3 4 5 Strongly agree to strongly disagree Figure 22. Learned to see relationships between important topics and ideas. N=24, Mean=2.71, SD=.78 Collaboration Index Question 4 asks about views on the collaborative learning experience. The ability to work together to develop ideas and seek solutions is seen as a significant contribution to an effective educational experience. The results of the collaboration index are expressed in Figures 23-28 (derived from questionnaire 3) and summaries follow: • felt they had an individual experience (99%); and • level of communication outside of class (62% never) 51 I developed new friendships in this class c o TJ 3 CO o i i E 3 Z 8 6 4 + 2 0 5 1 2 3 4 5 Strongly agree to strongly disagree Figure 23. Develop new friendships. N=21, Mean = 3.1, SD=1.04 I learned to value other points of view (A 10 c » TJ 3 (A "S 4) A E 3 Z Strongly agree to strongly disagree Figure 24. Value other points of view. N=21,Mean = 2.9,SD=1.10 Individual vs. Group Learning Individual vs. Group Experience 1 2 3 4 5 6 Individual to Group The help 1 got from students was w S Q TJ 3 6 7 M i l l Numl c 1 2 3 4 5 6 Crucially important to useless or misleading Figure 25. Individual vs. group experience. Figure 26. The help I got from students. N=24, Mean = 1.54, SD=.25 N=20, Mean 4.1, SD= 1.29 52 Students in my class tended to be 2 3 4 5 6 Not at all cooperative to extremely cooperative How often did you communicate with other students outside of class, by computer, fact-to-face or on the telephone? 1 2 3 4 5 6 Never to Constantly Figure 27. Students in my class. N=18, Mean = 3.61, SD=.98 Figure 28. How often did you communicate using other media? N= 21, Mean = 2.19, SD=1.29 The feeling of being alone seems to have pervaded the collaboration index. The active communication levels indicated that interaction was not sufficient to support a collaborative viewpoint. In part, this could be due to the limited time frame of the research and the lack of opportunity to develop liaisons. Post interview responses to the question 1.5 : "Did you feel that you were part of a group or class working together, or helpful, and to what extent did you feel this was a waste of time?" may assist in understanding this view and are recorded in Appendix A. Instructor rating Index Question 4 examines views on instructor access. The facilitators were to assist the subjects in understanding specific code concepts and generating discussion on various code issues. The inability to secure on-line connections and the time limitations of municipalities meant the co-53 investigator fulfilled the facilitator roles. Results of the questionnaires are stated in Table 2 (derived from questionnaire 3) and some summaries follow: Table 2 • Instructor Rating Instructor rating used Likert scale measurements from 1-5, Strongly agree to Strongly Disagree Issue N Mean SD Instructor organized course well 24 3.04 .88 Grading was fair and impartial 24 2.94 .24 Instructor seems to enjoy teaching 24 2.67 .91 Instructor lacks sufficient knowledge about this subject 24 3.95 .83 Students were encouraged to express ideas 24 3.80 .66 Instructor presented material clearly and summarized main points 24 3.70 2.46 Instructor discussed points of view other than his/her own 24 2.74 .81 The student was able to get personal help in the course 24 2.64 1.05 Instructor presented material in a boring manner 24 3.33 .91 Instructor critiqued my work in a constructive and helpful way 24 3.05 1.07 Overall, I would rate this teacher as:(Excellent to Poor - 5 points) 24 3.19 .93 54 Responses from the interviews may help clarify subject responses. Appendix B contains responses to question 1.8: "How would you describe your relationship to the facilitator on line? Do you feel MORE or LESS able to communicate and relate to your teacher?" The majority of subjects felt detached and not supported by the instructor. This will have a bearing on the support for computer conferencing asked by question 4. Course Rating Index Question 4 queries the views of on-line courses. The course rating measures attitudes toward the course. The overall course rating (Figures 29-35-derived from questionnaire 3) provides an indication of feelings of on-line communication concepts and central ideas. Positive views about the course are tempered by neutral feelings in a number of categories. Again the depth of conferencing activity, job constraints, and interface difficulties as displayed in the interviews, affect these results. A summary of majority opinions follow: • course was a waste of time (disagree - 3.91 mean); and • gained understanding of basic concepts (2.74 mean). The course overall was a waste of time 1 15 « •o I 1 0 2 i o _L3_ 1 2 3 4 5 Strongly agree to strongly disagree How would you rate this course overall? to 10 1 2 3 4 5 Excellent to Poor Figure 29. Course overall was a waste of time. Figure 30. How would you rate this course overall? N=23, Mean = 3.91, SD=1.09 N=21, Mean=3.19, SD=.93 55 I became more interested in the subject Strongly agree to strongly disagree I learned a great deal of factual material 1 8 o> 1 6 •8 4 + 2 i o 5 1 2 3 4 5 Strongly agree to strongly disagree Figure 31.1 became more interested in the subject. Figure 32.1 learned a great deal of factual material. N=21,Mean=3.17,SD=.79 N=22, Mean=3.09, SD=1.11 I gained a good understanding of basic concepts •' • ' • ' £ r ' 3 Strongly agree to strongly disagree learned to Identify central issues in this field Strongly agree to strongly disagree Figure 33. Gained a good understanding of concepts. Figure 34. Learned to identify central issues. N=19, Mean=2.74, SD=.99 N=20, Mean=3.05, SD=1.05 56 1 developed the ability to communicate clearly about this subject 15 13 o » 111 ^ ^ ^ ^ ^ 1 2 3 4 5 Strongly agree to strongly disagree Figure 35. Developed the ability to communicate clearly about the subject. N=23, Mean=3.33, SD=.97 Virtual Classroom overall Question 4 queries views of the on-line experience. The virtual classroom (Figures 36-39-derived from questionnaire 3) on-line experience is summarized in this section. Measurements of educational quality are offered. Following are majority responses: • they would choose to take another on-line course; and • increased the quality of my education. Interview questions also express the views concerning the on-line course. The pre-interview question 2.6 asked for a response from 1-7, strongly disagree to strongly agree, to the statement that "Computer conferencing as I know it should provide some interesting ways of understanding the code." In the post-interview, the same question was asked again, and the subject's earlier response was provided for comment and comparison. The responses are contained in Appendix C. Again time constraints, internet access, and the presentation of textual material affected subjects views of the virtual classroom, although the potential of information exchange and instructor access within the environment was recognized. 57 I would NOT choose to take another on-line course Strongly agree to strongly disagree I found the course to be a better learning experience than normal face-to-face courses Strongly agree to strongly disagree Figure 36. Not choose to take another on-line course. Figure 37. Better learning experience than face-to-face courses. N=21,Mean=4.9,SD=1.5 N=20, Mean=4.5, SD=1.7 I learned a great deal more because of the use of Codeworks Strongly agree to strongly disagree Figure 38. Learned a great deal more because of Codeworks. N=20, Mean=4.7, SD=0.3 Did the use of the system increase the quality of your education Strongly agree to strongly disagree Figure 39. Did use of system increase quality of education. N=17, Mean=3.6, SD=1.4 Access and Quality Assessments Question 4 queries the views of computer access and the quality of the on-line course. The following opinions represent the assessment of the quality of the course and are described in Figures 40-46 which is derived from questionnaire 3: 58 • better access to professors; • taking on-line courses is more convenient; • assignments read by other students increased my motivation; • when I became very busy I was more likely to stop participating (70% agree); • I felt more involved and active in the course; and • comments by others were useful. Having the computerized system available provided better access to the professors Strongly agree to strongly disagree Taking on-line courses is more convenient 10 ID O •o 3 55 £ 10 8 6 Hi: •nn 2 2. 2 1 •0 1 2 3 4 5 6 7 Strongly agree to strongly disagree Figure 40. Computerized system provided better access to professor. N=20, Mean=3.8, SD=1.6 Figure 41. Taking on-line courses is more convenient. N=21,Mean=3.0, SD=1.4 59 When I became very busy with other things, I was more likely to stop participating in the on-line class than I would have been to "cut" a weekly face-to-face lecture 1 2 3 4 5 6 7 Strongly agree to strongly disagree The fact that my assignments would be read by the others students increased my motivation to do a thorough job 7 1 2 3 4 5 6 7 Strongly agree to strongly disagree Figure 42. Students reading my assignments increased my motivation. N=20, Mean=3.3, SD=1.2 Figure 43. Became very busy I stopped participating in on-line class. N=20, Mean=2.0, SD=1.1 The on-line or virtual classroom mode is more boring than traditional classes 6 1 2 3 4 5 6 7 Strongly agree to strongly disagree Figure 44. On-line mode more boring than traditional. N=21,Mean=3.3, SD=1.6 I felt more "involved" in taking an active part in the course 1 2 3 4 5 6 7 Strongly agree to strongly disagree Figure 45. Felt more involved in taking part in course. N=21,Mean=3.5, SD=1.3 60 I found the comments made by other students to be useful to me 1 2 3 4 5 6 7 Strongly agree to strongly disagree Figure 46. Found comments made by other students useful. N=20,Mean=3.0,SD=l.l The strength of this assessment lies in reduced participation, likely due to the constraints imposed on the subjects by their jobs (as reported in interviews). However, overall, subjects found the on-line course more convenient, they felt more involved (than in traditional classrooms), there was increased student motivation when students realized that others reading their assignments, and students reported better access to the facilitator. Group comparisons Independent t-tests were used to compare demographics and experience for both Groups I and U. There were no significant differences between the means across the variables of age, computer experience, work attitudes, work experience and computer experience (questionnaire 1). Group I and II comparisons to pre and posttest performance were conducted. No significant difference between the means was found across pre and posttests. 61 Group I and II comparisons were conducted across all the questions asked in questionnaire 2 and 3. A significant difference was found between the means across the variable of communication and is displayed in Table 3. Table 3- Group Comparison across Questionnaire 2 and 3 How often did you communicate with other students outside of class, by computer, face-to-face or on the telephone? Variable Number of Cases Mean SD SE of Mean Group I 10 2.8000 1.317 .416 Group II 11 1.6364 1.027 .310 t-test for Equality ot Means Variances t-value df 2-Tail Siq SE of Diff 95% CI for Diff Equal 2.27 19 .035 .513 (.019,2.236) Unequal 2^ 24 17.02 .039 .519 (.069, 2.258) tadalO/doc The ability of the students to use the curriculum and have access to conferencing material could have played a role in these group differences. Group I with a curriculum basis may not have needed the added media tools for task clarification. Pearson Correlations Questions 2, 3 and 4 compare indexes of computer conferencing activity, views and attitudes. Three matrixes were constructed (Tables 4, 5 and 6) to compare the variables. All correlations were conducted using questionnaires 2 and 3 as reported in Chapter 3, based on Likert scale measurements. 62 Access and Activity Conditions by Outcome (Table 4) Question 4 compares computer terminal access to a number of on-line experiences. The following correlations were found: • terminal access difficulties are negatively correlated with computer attitudes, the collaboration index, synthesis index, and interest index, supporting the view that computer availability is critical to the support of these variables and positive views of computer conferencing; • having a terminal at home is negatively correlated with the collaboration index, the virtual classroom overall, and the synthesis index indicating that home access does not affect these variables; and • on-line course convenience is positively correlated with course rating and interest supporting the view that those who viewed the course as convenient would also rate the course high and exhibit increased interest in on-line activity. However, the convenience of on-line courses is negatively correlated with the synthesis index indicating that even though the course was viewed as convenient, positive views about identifying central issues, integrating facts, learning to see relationships, and the view that comments made by other students were useful were not supported. Table 4 Access and Activity Conditions by Outcome Pearson Correlation Coefficients Have a terminal at Total Time on-line Terminal Access a Taking O/L more Home during Course Problem Convenient Computer -.0055 .2332 .5204* -.1789 Attitudes Collaboration -.5270** .4044 .4497* -.3613 Index Instructor Rating .0094 .0108 .1651 .2402 VC Overall -.4793* -.2801 -.2342 -.4211 Course Rating -.2452 .0602 .0946 .5204* Synthesis Index -.4976* .2854 .4824* -.7738** Interest Index -.3223 .2828 .5746** .5058* Posttest -.1058 .1038 .3659 -.1613 "Significant at the 0.01 level 'Significant at the 0.05 level N = 20 63 Process and Assessments of the Virtual Classroom (Table 5) Table 5 presents correlations of variables related to research questions 3 and 4, comparing the collaboration index with important classroom experiences such as communicating with other students, access to the professor, increased motivation and usefulness by reading and responding to other students assignments and comments. Also, increased communication with other students was compared to computer attitudes, collaboration, instructor, the virtual classroom overall, course rating, synthesis and interest indexes, and the posttest results. Correlations and conclusions are explained in Chapter 5. Variables are extracted from questionnaires 2 and 3 and are explained in Chapter 3. Table 5 Process and Assessments of the Virtual Classroom Communication with students Access to Professor Increased Motivation Comments Assignments Useful Computer Attitudes Collaboration Index Instructor Rating .1337 -.5377* .3579 -.4596* .5392* .4459* -.3002 -.4977* .2795 -.1328 -.6084** -.0730 -.3026 -.2001 -.3772 VC Overall .6411" .5629** .5490* -.3909 .3080 Course Rating -.6176" .6251" .4755* .1382 -.0605 Synthesis Index -.4429* -.4743 -.6312" -.5738** -.3455* Interest Index -.3799 .5438* .5038* -.3548 -.5506* Posttest .2877 .1807 .2845 .0187 -.2375 * Significant at "Significant N=20 the 0.05 jevel at the 0.01 level Note: Communication with Communicated more with other students students: Access to Professor: Provided better access to professor Assignments Useful: Found reading Assignments of other students useful Comments: Found comments made by other students useful Increased motive: The fact that assignments would be read by other students increased motivation 64 Students Characteristics and Selected Outcomes (Table 6) Table 6 presents the correlation affecting question 2, by examining the effects of interpersonal and personal sphere of control on the course outcome index, instructor rating index, virtual classroom overall index, that the on-line learning is easy, that students would not take another on-line course, and the total hours on-line. The personal sphere of control is significantly correlated with the course outcome index11. However, the interpersonal sphere of control is significantly correlated to the view that the respective subjects would not like to take another course on-line supporting an affirmative response to question 2. From interviews it was observed that students were frustrated by the slow pace of instruction and the sophistication of the conferencing environment. Slow system reaction time, and large amounts of textual material detracted from the group experience. Table 6 Pearson Correlation Coefficients Between Students Characteristics and Selected Outcomes Computer Attitudes Virtual Class. Expect. Personal Sphere of Control Inter personal Course Outcome Index .2134 -.0847 .6679" -.1213 Instructor Rating Index .2239 -.0237 .3968 -.0015 VC Overall Index -.0440 -.1152 .1496 -.3552 VC easy to learn .1574 -.2783 .1194 -.0137 Not take another online .2319 .2815 .0770 .6500" Total Hours On-line .0041 -.1494 .2332 .4044 ••Significant at the 0.01 level 11 The course outcome index is a Likert scale measurement of the questions: the course overall was a waste of time and how would you rate the course overall and is composed of figures 29-35. 65 Interviews In the following questions posed in the post-interview, subjects were asked to compare the traditional and virtual classroom: "What do you like best about the virtual classroom approach...that is, what is good about it compared to a course given in the traditional classroom?" 1) Flexibility - choose own time to work (11/32) 2) Delivered right to the office, no travel involved (3/32) 3) Support group within the office (2/32) 4) Opens the breadth and depth of communication (1/32) "What do you currently like least?" 1) Complicated interaction with other students (9/32) 2) Too much information to process in text form (5/32) 3) Increased motivation necessary to do the work (4/32) 4) Inadequate pictures, animation, and sound (4/32) 5) Should have everyone on line simultaneously (1/32) 6) On-line help and instructions (1/32) 7) Difficult to do work when you have to set your own schedule (1/32) 8) More formalization of the instruction and people know each other (1/32) 9) Nervous about seeing their own words posted (1/32) Interviews of two students are contained in Appendix H. Their depict specific views regarding systems improvement, including answers to the following: What worked well? What could be improved? 66 Comments on the technology. How was your motivation affected? What do you think about conferencing? Initial responses gathered from the pre-interviews assist in assessing project results and contained in Appendix D and E when subjects commented on the use of computers and of teaching they found best. Conferencing passwords and reading data were perceived; awkward. 6 7 Chapter 5 - Conclusions Chapter overview This chapter draws conclusions regarding the Codeworks research project and discusses: • the framework for the study; • conclusions for each research question; • implications of research findings; and • further research. Summary of findings Subjects entered the course with full assurance of confidentiality knowing that reported results would not harm their present or future academic standing. Questionnaire 1 gauged experience and expertise in the areas of work, code and computer experience. Questionnaires 2 and 3 reflected the study by Hiltz (1994) and made many of the same comparisons. Along with the interviews, these were the main data sources. Pre and posttests were administered and t-test comparisons noted no significant difference in achievement between groups affected by the presence of the curriculum. Some interesting results were found through comparison of achievement across the variables using Pearson Correlations. The study began with the intent of comparing the lecture-driven certified professional program offered at UBC with an on-line equivalent course. The certified professional class size was approximately 15, but only three of these students volunteered to participate. In addition a discrepancy with time allotments between the virtual classroom and the traditional classroom 68 would not permit a balanced comparison. The on-line course content mirrored by the certified professional program was only one to two hours in length. To allow conference participants time to conduct course work while in a busy work environment, the on-line course was offered asynchronously over a six week period. Indexes for correlations were constructed using criteria defined by Hiltz (1994). Key measurements were sphere of control, computer attitudes, course rating, facilitator rating, motivation, comments by other students, views of reading assignments, and virtual classroom expectations. Conclusions follow: 1. Will there be significant gains in code knowledge between Groups "I" and "JJ" when Group "I" is exposed to a computer conferencing delivery system with a defined curriculum and "JJ" is exposed to a computer conferencing system without a defined curriculum? Answering this question involved achievement on pre and posttests. The scores indicated there was no significant difference in course results, and that they were unaffected by the curriculum (see Appendix J). This has significant impact on on-line course design and implementation suggesting that it may be appropriate to rely less on curriculum design, but concentrate more on conference management. Although the samples were small and the course delivery time frames short, each group was controlled for age, work, code and computer experience. These variables ranged from novice to experienced. The promotion of on-line participation discussion commenced with 'hot topic' issues (stucco) in a problem solving format, that could easily be adapted to an educational environment with summative and formative tests. As displayed in the interviews curriculum design elements such 69 as text based content detracted from discussion and project completion and could likely be deleted from a curriculum based on issues discussion. The use of issues discussion and conferencing topics rather than a structured curricula may alleviate information overload concerns expressed by Harasim (1987) and Burge (1994). The facilitator's time could then focus on other disadvantages of the on-line experience (Hiltz and Turoff, 1987; Harasim, 1987; Burge, 1994; and Hilman et al, 1994): • drawing out lurkers by using an active moderator; • developing real-time discussion mechanisms encouraging the immediacy of exchange; • setting time management priorities for students; and • overcoming interface encumbrances by encouraging early conferencing exchanges. 2. Will students with a greater sphere of control on either the personal or the interpersonal levels be more or less likely to regularly and actively participate on-line, take another on-line course, rate the virtual classroom as easier and more effective than the traditional one, and have a positive view of the instructor or facilitator? Answering this question involves the variables of virtual classroom rating, course outcome, instructor rating (after taking the on-line course), and views on taking another on-line course (after taking the on-line course) using questionnaires 2 and 3 applying a Pearson correlation. The personal sphere of control is positively correlated with the course outcome index supporting the view that a high degree of effort results in a positive view of the conferencing environment and therefore the view that students were more likely to actively participate on-line. The personal sphere of control prescribes a student that works hard, is goal oriented, enjoys challenges, feels ii control of circumstances and seeks agreement in a debate. The course outcome index defines 70 two general reactions displayed in figures 29 to 35 that are worthy of comparison to this proclivity within the on-line environment. Three questions forming part of this variable are: • Was the course was a waste of time overall? • How would you rate this course overall? • Did you gain a good understanding of concepts? Most students felt strongly that the course was not a waste of time (3.91 mean on a 5 point Likert scale) and rated the course as good overall (3.19 on a 5 point Likert scale). The correlation supports the view that personal sphere of control elements are integral to the positive views of on-line courses. Control over course direction, the democratic aspects of the medium, and the convenience of access (Davie, 1988) promote this relationship. However, the only correlation for the interpersonal sphere of control was a positive one with the view that the respective subjects would not like to take another on-line course. The interpersonal sphere of control is represented by the collaboration index which is expressed in figures 23 to 28 and is further described in research question 3. As reported in the results students felt they developed new friendships in the class, but had an individual experience, hardly ever communicated with other forms of media outside the class and did not view other students as cooperative. Interviews with students indicate that they preferred the immediacy of exchange present in the traditional classroom (Davie, 1988), but also log ons by students indicate that 60% of students entered the conference less than four times over the life of the project. Even with these interaction rates conferencing required constant vigilance by the facilitator to encourage lurkers to sign-on and enter comments (Harasim, 1987; Burge, 1994). Most students expressed frustration with the pace of instruction and the difficulty in taking time out from normal job 71 duties to conference. Figure 43 echoes this view with the strong majority of students stating that when they became very busy with other things, they were more likely to stop participating in the on-line course (mean 2.0 - strongly agree on a 7 point Likert scale). As most students felt that the course was an individual experience it is not surprising that positive correlations were not found with the virtual classroom experience. The types of discussions that were generated by conferencing gives us an indication of the degree of collaboration and the focus of the discussion. Harasim (1990) stresses that a collaborative environment will be developed with focused discussions. The nature of this project displayed more social messages than task oriented or problem solving (figure 1 a)) ones and is consistent with Harasim's (1987) findings. 3. Will students who experience group or collaborative learning in the virtual classroom have positive12 views of on-line course work? Answering this question involves the variables virtual classroom rating, collaboration index, terminal access, time on-line, and the convenience of on-line courses using questionnaires 2 and Here comparisons were made between the collaboration index and important classroom experiences such as communicating with other students, access to the professor, increased motivation and usefulness by reading other students assignments, and whether they found comments made by other students useful. 1 2 Positive measurements are based on correlations with on-line convenience, computer terminal access, time spent on-line, increased communication with the students, improved access to the facilitator, experienced increased motivation by reading assignments of other students and found comments and assignments by other students useful. 72 This question was posed to further explore the aspects of on-line conferencing and collaboration examining correlations with other variables. The collaborative learning index was made up of six questions (figures 23 to 28) as discussed in question 2. Correlations were found with the collaboration index as follows: • easy access to a computer terminal had a negative impact on the collaborative nature of the students. A relatively low collaboration index is dictated by low scores in the views that students had an individual experience, did not communicate by other means outside the class, and did not view other students as cooperative. The short duration of the course and the interference of job duties likely detracted from the need to sign-on and conference. • communicated less with other students, assignments read by other students did not increase motivation and comments made by other students were not found useful, in spite of the collaborative nature of the students. The feeling that traditional courses could offer more immediacy and interaction seemed to support the collaborative nature more than on-line conferencing (figure 6). Kaye and Mason (1989) and Hiltz (1989 and 1994) indicate that other communication media such as traditional classrooms and print mediated distance learning should supplement but not compete with the on-line experience. Personal networking is seen as critical to prompt communication and the open ended nature of computer conferencing (Feenberg, 1989). So even though students displayed a collaborative nature they demanded more ways of expressing themselves than strictly through the on-line experience. • better access to the professor. Access to the facilitator within a democratic environment is seen as an advantage of computer conferencing and key to the development of group dynamics (Burge, 1994). However, the instructor rating (table 2) revealed scores which were mainly neutral in many categories. Significant numbers of students felt that the instructor did discuss other points of view other than his own and enjoyed teaching. Interviews of students indicated the ability to e-mail and develop new ways of communicating with instructors (in this case government officials) were seen as the positive attributes of conferencing. 4. Will conferencing students with good computer terminal access at either home or in the office, who spend more time on-line and view on-line courses as more convenient report positive views of on-line courses across a number of variables13 and will students report positive views of the on-line course across the same variables? Answering this question involves the variables virtual classroom rating, collaboration index, computer attitudes, interest and synthesis indexes, instructor rating index, course rating index, access and quality assessments using interviews, questionnaires and measurements of 1 3 The variables measured are computer attitudes, instructor rating, interest in the course, the ability to synthesize ideas, views on the classroom overall increased collaboration, course rating and course access and quality. 73 conferencing activity. The collaboration index has been compared to the terminal access variables in research questions 2 and 3. Frequencies displayed in the figures 2 to 46 support a number of positive views of on-line conferencing. Significant findings are that: • the course was not a waste of time. From interviews students had a positive view of the conferencing potential identifying the medium as convenient. The course offered access to building code interpretations not readily available within their normal work environment. • that students gained an understanding of basic concepts. As one group did not outperform the other it is suggested that the code ability (as displayed in the posttests) were as much a result of conferencing as the curriculum. • the use of computers was stimulating (100%), fun (100%), easy (92%), personal (72%), helpful (58%), threatening (53%), efficient (81%), demanding (54%), and desirable (65%). The majority enjoyed the conferencing experience but had reservations concerning the demands of the technology and its effect over job duties. • that students reported more interest in the subject, the ability to see relationships between important topics and ideas. They would choose to take another on-line course and on-line conferencing increased the quality of their education all of which supports a positive view of the on-line course. The excitement of on-line conferences lies in their ability to tap into vast resources and open up a virtual world to the learner (Mason and Kaye, 1989). Even though building code discussion was confining, students enjoyed the interactive nature of the community. Interesting correlations with computer terminal access and on-line convenience were found as follows: • terminal access difficulties are positively correlated with computer attitudes, the collaboration index (discussed previously), the interest and the synthesis indexes and the virtual classroom overall rating. Computer attitudes measured a range of views with mainly positive results. The interest and synthesis indexes measured student views of their ability to integrate facts, discuss central issues and related topics. The virtual classroom overall gauged the effectiveness of the system. Access to a computer terminal would normally be thought of as integral to these functions but they are portrayed as negative correlations. I suggest that the positive views of the conferencing environment were because many students recognized the potential of the system to contact subject specialists and work within a virtual community. on-line course convenience is correlated with the course rating, synthesis index and interest index. Course rating and the interest index are positive correlations lending credence to support positive views of the computer conferencing. Interviews indicated that students enjoyed the asynchronicity advantage reported by Harasim (1987). The inability for strong interactive components and problem solving mechanisms to develop is again blamed on the short course duration, involved course tasks and interface difficulties (reference interviews Appendix H). Summary of interview responses Harasim (1987) indicates that the loss of visual cues was unimportant, but the need for personal 'chit chat' was critical to stimulate socialization experiences. However, the need for graphical display was supported by the students (primarily architectural designers). Some designers had difficulty with the appearance of the text and the screen; some felt that they should have been walked through the material. The medium is seen as the impeding force in communicating (Hillman et al, 1994; Eastmond, 1994). Some of the subjects were novices with the internet and e-mail and thus needed to learn how to use the software. Although virtual access presented certain anonymous advantages, fear of the permanent record deterred some, supporting the views of Davie (1988), Harasim (1990), and Mason and Kaye (1989). Face-to-face immediacy was seen as being as important as real-time conferencing and therefore not a replacement concurring with Harasim (1990). Motivation of students was restricted by job time constraints. Summary The major findings of this study are that: • on-line instructional design does not appear to have an effect on test scores in the computer conferencing environment which is likely due to the six week duration of the course, the peer effect within the offices, and the complicated textual nature of the course. 75 • personal sphere of control is positively correlated with the course outcome index supporting the view that a high degree of effort supports a positive view of the conferencing environment. However, the interpersonal sphere of control is positively correlated to the view that the respective subjects would not like to take another course on-line. From interviews it can be seen that students were frustrated by the pace of instruction and the sophistication of the conferencing environment. Slow system reaction time, and large amounts of textual material likely detracted from the group experience. • collaborative attitudes supported acceptance of the conferencing environment. However, the immediacy and ease of communication within traditional classrooms were major factors supporting a negative correlation. • communicating more with other students was positively correlated with the virtual classroom overall rating supporting the view that conferencing is integral to positive on-line course work. However, this variable was negatively correlated with the course rating and the synthesis index indicating that those with low levels of communication still supported a positive view of the course and the synthesis of ideas. Interview material supports positive perceptions and the potential of on-line communication in spite of the level of conferencing. Further research The virtual building code web site, through the use of the core conferencing module, could evolve into a resource base for the use of all construction participants. A knowledge base akin to an expert system would be accessible by users ranging from novice to expert. The interface utility was enhanced with building code issues discussion, orientation meetings, and one-to-one e-mail discussions. Increased interface familiarity may then develop on-line skills spurring learner motivation to participate more actively in an on-line curriculum. Introduction to the interface would assist each participant in developing relationships, forming problem solving mechanisms, and contributing to an evolving body of knowledge. With the vast resources and accessibility of the internet, participation would eventually be expanded to include contractors, engineers, suppliers, manufacturers and other associations. Geographically and functionally diverse locations (i.e. involvement of the National Research Council in Ottawa and 76 requests from equipment suppliers) would develop. Expansion of subjects would demand an active moderator to summarize arguments and discussion, and distill material for future curriculum and information dissemination. For example product suppliers could be included in conferencing discussions as an educational resource to enhance product usage within building code terms. Entering the conference at this knowledge level would increase the awareness of designers, and building officials in practical applications of code concepts and assist in the interaction of key sectors of the construction community perhaps encouraging on-line education. More research is required into: • How can offices be connected on-line and provide adequate time to interact actively with the instructor. Achievements and attitudes would then be tested? • How would enhanced graphics and video improve attitudes and conferencing use? • How are on-the-job work environments affected by peers, subject specialists and time constraints, and how can this effect be measured? • Would the use of a facilitator team, which, in this case would connect a larger number of municipalities and assist in fashioning a virtual community, encourage on-line usage? • What innovative ways of designing discussion are available to enhance curricula design and what hypertext searching mechanisms can be employed effectively? • How can computer conferencing programs be developed to make effective use of face-to-face instruction? • How can the constraint of on-line communication with legal opinion be reduced? 77 Bibliography Ahern, T. (1994). The effect of the interface on the structure of interaction in computer mediated small-group discussion. Journal of Educational Computing Research. 11(3). 235-250. Anderson, T . , & Mason, R. (1993). International computer conferencing for professional development: The Bangkok Project. The American Journal of Distance Education. 7(2), 5-17. Bates, A.W. (1991). Third generation distance education: The challenge of new technology. Research in Distance Education. April. 1991. Bates, T. (1986). Computer assisted learning or communications: Which way for information technology in distance education? Journal of Distance Education. 1(1). 41-57. Bohlin, R., & Milheim, W. (1994). Analyses of the instructional motivation needs of adults. Canadian Journal of Educational Communication. 23(1), 47-55. Boston, R. (1992). Remote delivery of instruction via the pc and modem: What have we learned? The American Journal of Distance Education. 6(3). 45-57. Brookfield S. (1994) Self-directed learning from theory to practise. In Garland, M. (1994) The adult need for "Personal Control" provides a cogent guiding concept for distance education. Journal of Distance Education. 9(1). 45-59. Brown, J.S. (1985) Processs vs. product: A perspective on tools for communal and informal and electronic learning. In Harasim, L. (1990) Online Education. New York, Westport, Connecticut, London. Praeger Publishing. Boud, D. (1988) Moving toward student autonomy. In Garland, M. (1994) The adult need for "Personal Control" provides a cogent guiding concept for distance education. Journal of Distance Education. 9(1). 45-59. Burge, E. (1994). Learning in computer conferenced contexts. The Learner's Perspective. Journal of Distance Education. 9(1). 19-43. Carr, D. (1986) The meanings of the adult independent library learning project. In Wilson, V. (1994). Developing the adult independent learner: Information literacy and the remote external student. Distance Education. 15(2). 254-277. Chacon, F. (1992). A taxonomy of computer media in distance education. Open Learning. Feb. 12-27. 78 Clark, R. (1985). Evidence for confounding in computer-based instruction studies: Analyzing the Meta-AnalvsisJducational Communication and Technology Journal. 33(4). 249-262. Hillman, DC.A, Willis, DJ, Gunawardena, C.N, (1994). Clark, T., & Verduin, J. (1989). Distance Education: its effectiveness and potential use in lifelong learning. Lifelong Learning. 12(4). 24-26. Collis, B. (1991). Telecommunications-based training in Europe: A state-of-the-art report. The American Journal of Distance Education. 5 (2), 31-41. Daloz L. (1986) Effective teaching and mentoring. In Galbraith, M.(1989). Essential skills for the facilitator of adult learning. Lifelong Learning. 12(6). 10-13. Davie, L. (1988). Facilitating adult learning through computer-mediated distance education. Journal of Distance Education. 3 (2). 55-69. de Vries, L., Naidu, S., Ougbemiro, J., & Collis, B. (1995). On-line professional staff development: An evaluation study. Distance Education. 16 (1). 157-173. D'Souza, P. (1988). A CAI approach to teaching an office technology course. Journal of Educational Technology Systems. 17(2). 135-140. D'Souza, P. (1991). Use of electronic mail as an instructional aid: An exploratory study. Journal of Computer Based Instruction. 18(3). 106-110. Eastmond, D., (1994). Adult distance study through computer conferencing. Distance Education. 15(1). 129-152. Eastmond, D. (1992). Effective facilitation of computer conferencing. Continuing Higher Education Review. 56 (1&2). 23-34. Ercegovac, Z. (1989) Augmented assistance in online catalogue subject searching. In Wilson, V. (1994). Developing the adult independent learner: Information literacy and the remote external student. Distance Education. 15(2). 254-277 Everett, D . , & Ahern, T. (1994). Computer-mediated communication as a teaching tool: A case study. Journal of Research on Computing in Education. 26(3), 336-357. Feenberg, A-» and Bellman, B. (1990) Social factor research in computer-mediated communications. In Harasim, L. (1990) Online Education. New York, Westport, Connecticut, London. Praeger Publishing. Feldman, M. (1986) Constraints on communication and electronic messaging. In Harasim, L. (1987) Teaching and learning on-line: issues in computer-mediated graduate courses. Canadian Journal of Educational Communication. 16(2). 117-135. Fisher, J. (1993). A framework for describing developmental change among older adults. Adult Education Quarterly. 43(2). 76-89. 79 Frick. E. (1982) Teaching information structure. In Wilson, V. (1994). Developing the adult independent learner: Information literacy and the remote external student. Distance Education, 15(2), 254-277. Frye, Olynick, and Pinkney (1992) Development of an Expert System. National Research Council, Montreal, Canada. Galbraith, M. (1989). Essential skills for the facilitator of adult learning. Lifelong Learning. 12(6). 10-13. Garland, M. (1994). The adult need for "Personal Control" provides a cogent guiding concept for distance education. Journal of Distance Education. 9(1). 45-59. Gundry, J. (1991) Understanding collaborative learning in networked organizations. In Kaye, A. (Ed.)(1992). Collaborative learning through computer conferencing. New York: Springer-Verlag. Hacker, K., (1994). Is Computer-Mediated Communication Contributing to Organisational Productivity? (Opinions/position papers, essays) ED 367 043. Harasim, L. (1990). Online Education. New York, Westport, Connecticut, London. Praeger Publishing. Harasim, L. (1987). Teaching and learning on-line: issues in computer-mediated graduate courses. Canadian Journal of Educational Communication. 16(2), 117-135. Harris, J. (1995). Mining the Internet. Learning and Leading with Technology. Dec/Jan. 1995. 36-39. Hilman, D.C.A., Willis, D.J., Gunawardena, C.N., (1994). Learner-interface interaction in distance education: an extension of contemporary models and strategies for practitioners. The American Journal of Distance Education. 8(2). 30-41. Hiltz, S. and Turoff, M. (1985) Structuring computer-mediated communication systems to avoid information overload. In Harasim, L. (1987) Teaching and learning on-line: issues in computer-mediated graduate courses. Canadian Journal of Educational Communication. 16(2). 117-135. Hiltz, S. (1994). The Virtual Classroom. Norwood, New Jersey: Ablex Publishing Corporation. Hiltz, S., & Johnson, K. (1989). Measuring acceptance of computer-mediated communication systems. Journal of the American Society for Information Science. 40(6), 386-397. Johansen, R., & DeGrasse, R. (1979). Computer-based teleconferencing: effects on working patterns. Journal of Communication. Summer 1979. 80 Jones, A. Petre, M. (1994). Computer-based practical work at a distance a case study. Computers Education. .22(1). 27-37. Kaye, A. (Ed.). (1992). Collaborative learning through computer conferencing. New York: Springer-Verlag. Kaye, T. and Mason, R. (1988) Toward a new paradigm for distance education. In Harasim, L.(1990) Online Education. New York, Westport, Connecticut, London. Praeger Publishing Kaye, T. (1987). Introducing computer-mediated communication into a distance education system. Canadian Journal of Educational Communication. 16(2). 153-166. Kulik, C.C., Kulik, J., & Shwalb, B.J., (1986). The effectiveness of computer-based adult education: a meta-analysis. Journal of Educational Computing Research. 2(2). 235-252. Lauzon, A., & Moore, G. (1989). A fourth generation distance education system: integrating computer-assisted learning and computer conferencing. The American Journal of Distance Education. (1). 38-49. Levinson, P. (1990) Computer conferencing in the context of the evolution of the media. In Harasim, L.(1990) Online Education. New York. Westport, Connecticut, London. Praeger Publishing. Lewis, R., (1993). Learning technologies from a human actor's view. Computers Education. 21(1/2). 173-180. Lewis, L. (1988). Adults and computer anxiety: fact or fiction? Lifelong Learning. 11(8). 5-8,12. Licata, B. (1984) New technology implementation and use. In Lewis, L. (1988). Adults and computer anxiety: fact or fiction? Lifelong Learning. 11 (8). 5-8, 12. MacConnell, D. (1990). Case study: The educational use of computer conferencing. ETTI. 27, 2. 190-208 Maryska, R. (1995). Using the internet to expand resources. Business Education Forum. Dec. 1995, 19-22. Mason, R. (1988). Computer conferencing: A contribution to self-directed learning. British Journal of Educational Technology. 19(1). 28-41. Mason, R., & Kaye, A. (Eds.). (1989). Mindweave. (1st, ed). Toronto: Pergamon Press. McCreary , E. (1986) Three behaviour models for computer-mediated communication. In Harasim, L. (1990). Online Education. New York, Westport, Conneticut, London. Praeger Publishing. McCreary, E., & Van Duren, J. (1987). Educational applications of computer conferencing. Canadian Journal of Educational Communication. 16(2). 107-115. 81 McMahen, C , & Dawson, A. (1995). The design and implementation of environmental computer-mediated communication (CMC). Projects. Journal of Research on Computing in Education. 21(3). 318-335. Menmuir, J. (1995). Quality assurance in the off-campus delivery of professional development opportunities. Open Learning. Feb. 1995,43-46 Moore, G. (1988). Asynchronous electronic communication: A North American-Thai collaboration. The American Journal of Distance Education. 2(1). 52-63. Moore, M.G, (1989). Three types of interaction. The American Journal of Distance Education. 3(2). 1-6. Morrison, D., & Lauzon, A. (1992), A.C. reflection on some of the technical issues of "Connecting" learners in online education. Research in Distance Education. July 1992. Nipper, S. (1990) Developing acceptance: how can integrated systems be used for industry oriented distance education and training? In Kaye, T. (1990). Introducing computer-mediated communication into a distance education system. Canadian Journal of Educational Communication. 16(2). 153-166. Phillips, A., & Pease, P. (1987). Computer conferencing and education: complementary or contradictory concepts? The American Journal of Distance Education. 1(2). 44-52. Phillips, C. (1990). Making friends in the electronic student lounge. Distance Education. 11(2). 320-333 Phillips, G. (1994). Introduction to "The Internet". Communication Education. 43. April, 73-86. Phillips, G., Santoro, G., & Kuehn, S. (1988). The use of computer-mediated communication in training students in group problem-solving and decision-making techniques. The American Journal of Distance Education, 2(1), 28-51. Phelps, R., Wells, R.., Ashworth, R., & Hahn, H. (199^.Effectiveness and costs of distance education using computer-mediated communication. The American Journal of Distance Education. 5(3). 6-18. Rada R., Acquah, S., Baker, B., & Ramsey, P. (1993). Collaborative learning and the much system. Computers Education. 20(3). 225-233. Rachal, J. (1988). Taxonomies and typologies of adult education. Lifelong Learning. 12(2). 20-21. Rawson, J. (1990). Simulation at a distance using computer conferencing. ETTI27. 3. Rice, R., & Love, G. (1987a). Electronic emotion. Communication Research. 14(1). 85-108. 82 Rice, R. (1987b). Computer-mediated communication and organizational innovation. Journal of Communication. 37(4), 65-94. Riel, M., & Levin, J. (1990). Building electronic communities: success and failure in computer networking. Instructional Science. 19. 145-169. Roberts, L. (1988) Computer conferencing a classroom for distance learning. In Eastmond, D. (1992) Effective facilitation of computer conferencing. Continuing Higher Education Review. 56(1 &2), 23-34. Scriven, Bruce. (1991). Distance education and open learning-implications for professional development and retraining. Distance Education. 12(2). 297-305. Simard, D. (1988). Phases of working life and adult education. Lifelong Learning. 12(2), 24-26. Shavelson, R.J., Stasz, C , Schlossman, S. Webb, N., Hotta, J.Y., and Goldstein, S. (1986) Evaluating student outcomes from telecourse instruction. In Hiltz, S. (1994). The Virtual Classroom. Norwood, New Jersey: Ablex Publishing Corporation. Shaw, D, (1993). Computer-aided instruction for adult professionals: A Research Report. Journal of Computer-Based Instruction. 19(2). 54-57. Sheehy, Gould and Levinson (1981) In Zemke, R. and S.(1981) 30 things we know for sure about adult learning. Training Delivery Systems for Adult Learners. 10-13. Schlossberg, N., Lynch, A., and Chickering, A. (1989) Improving higher education for adults. In Garland, M.(1994) The adult need for "Personal Control" provides a cogent guiding concept for distance education. Journal of Distance Education. 9(1), 45-59. Smith, Stephen. (1994). Communication and the constitution in cyberspace. Communication Education. 43. 87-96. Smith, P., and Kelley, M. (1987) Distance education and the mainstream. In Mason, R., Kaye, A. (Eds.) (1989). Mindweave. (1st. ed) Toronto: Pergamon Press. Tessmer, M. (1988). Subject specialist consultation in instructional design: Higher Education. Journal of Instructional Development. 11(2). 29-36. Updegrove, D. (1991). Electronic mail in education. Educational Technology. April 1991 Wagner, E. (1990). Looking at distance education through an educational technologist's eyes. The American Journal of Distance Education, 4(1). 53-68. Walkington, J., Pemberton, P, & Eastwell, J. (1994). Practical work in engineering: A challenge for distance education. Distance education. 15 (1). 160-170. 83 White, G., & Rose, C. (1988). Empowering the older adult learner: Community education as a delivery system. Lifelong Learning, 11(7), 20-22, 30. Wigand, R. (1988). Integrated services digital networks: concepts, policies and emerging issues. Journal of Communication, 38 (1). 43-51. Wilkes, C , & Burnham, B. (1991). Adult learner motivations and electronic distance education. The American Journal of Distance Education. 5(1), 43-51. Wilson, V. (1994). Developing the adult independent learner: Information literacy and the remote external student. Distance Education, 15 (2), 254-277. Zemke, R. & S. (1981). 30 Things we know for sure about adult learning. Training Delivery Systems for Adult Learners, 10-13. 84 Appendix A~ - Selected Subject responses to Post-Interview Question 1.5 Did you feel that you were part of a group or class working together and to what extent did you feel this was a waste of time? Subject 2 - "I felt part of group"; Subject 3 - "I felt more alone than part of a class. Guidelines for making postings should have been available"; Subject 4 - "I felt alone - not as if I were part of a group. It was more like reading mail than conferencing. Like a correspondence course"; Subject 5 - "I felt as if I were on my own. The only feedback I got was from reading the comments of other participants"; Subject 7 - "I felt I was part of a discussion group"; Subject 9- "There were only two or three people who posted messages that I saw. I thought it was fine. In the coffee shop, for example, it was interesting to read how others interpreted the code"; Subject 15 - "I felt part of a class"; Subject 17 - "I felt alone. I never was aware of being in a group"; Subject 19 - "I felt somewhat part of a group. Actually, the socializing aspect came from people responding to my comments. Not from the instructor"; Subject 21 - "Yes, I did feel as though it was a class"; Subject 22 - "I think that there were enough topics on the go at one time so that I was working with my team to find answers. But the interaction wasn't happening. Maybe there weren't enough people interested in any one particular topic"; 85 Subject 23 - "I didn't feel absolutely alone. I felt as if I were working with one or two others. But not with a class"; Subject 24 - "There was some interesting feedback from Subject 27 ~ some great communication there. So that was my main contact"; and Subject 27 - "I felt pretty much alone". 86 Appendix B - Selected Student responses to Post Interview Question 1.8 Students responded to Question 1.8: "How would you describe your relationship to the facilitator on-line? Do you feel M O R E or LESS able to communicate and relate to your teacher?" Subject 1 - "I felt comfortable"; Subject 4 - T never did communicate with my teacher. He didn't have a face or a name"; Subject 5 - "I'd prefer face to face interaction"; Subject 8 - "Yes, I felt able - not more, but adequately"; Subject 9 - "I felt comfortable with our communication. Probably more able to communicate, as I had access to email"; Subject 11 - "I didn't know who he was. I needed a course outline, I think. And an introduction to the instructor"; Subject 12 - "I had no communication with anyone like that. I felt that we needed someone to run the show. I felt that my work was going into outer space with no response"; Subject 20 - "I would feel better if I could just raise my hand and speak." 87 Appendix C- Selected student responses to post-interview question 2.6. Question 2.6: "The pre-interview question 2.6 asked for a response from 1-7, strongly disagree to to strongly agree, to the statement that the computer conferencing as I know it should provide some interesting ways of understanding the code. Your reaction was ( ). What is your reaction now?" Subject 1 (gave it a 4) - "My feelings haven't changed much, but I have great time constraints.'" Subject 2 (gave it a 7) - "I have no reaction to this, because I didn't experience it as conferencing. I expected that conferencing would be direct. If someone were on line and we had a session together, that would be conferencing. But the time lag makes it into just email, nothing more."; Subject 3 (gave it a 5) - "I would lower my agreement to a 4, because it (computer conferencing) didn't live up to my expectations of a Virtual Classroom. A lot of the course work was repetitive to me."; Subject 8 (gave it a 5) - "I'd rate it the same. People have to get used to the idea of computer conferencing before it works well."; Subject 9 (gave it a 5) - "I would continue to rate this statement at a five. The technology is there and I see a potential for its growth."; Subject 13 (gave it a 6) - "A 7.1 like the way it (the course) has been presented. The one page of questions was good. I could easily print it out, and it wasn't too much. Now (as a result of taking the course) I'm stumping code consultants on the job."; Subject 15 (gave it a 3) - "Slightly higher now, having gone through the process."; Subject 17 (gave it a 5) - "I'd give the same rating. Computer conferencing has a lot of potential, but in this instance, the site needs more structure."; 88 Subject 20 (gave it a 6) - "I'm the same. People work well on computers. They have more time to think about what they are going to say."; Subject 24 (gave it a 4) - "It stays the same. There are some positive things about it (computer conferencing), but there are some negative things. Reading on the screen is dissatisfying. It's hard on my eyes."; Subject 27 (gave it a 4) - "I'd go up to a 6 now. I see the potential there—but I think there's a lot of work that has to be done to make it reach its full potential-such as the audio component and the graphics."; and Subject 31 (gave it a 5) - "I've been working with the code for a couple of years now, and I think you have to work with it to truly learn it. So I wouldn't change my rating." 89 Appendix D - Selected Student responses to pre-interview comments on the use of computers Subject 1: "The quest for speed can be intoxicating and one forgets to concentrate on accuracy." Subject 2: "With the internet people communicate easily - people can stay current. With a code on line program people could keep in touch with changes in the code. But this doesn't allow for idiosyncratic interpretations of the code in various municipalities" Subject 3:"There will be problems in understanding how to use the internet" Subject 4: " I am it (the computer educational support). And I'm not an expert. I cant program. I'm self taught. I rely on friends for help." Subject 1 T'Extreme. I've been using computers since '73." Subject 24: "On-line discussions would make precedents available-they would have to somehow be monitored for accuracy, to avoid further misinterpretations." 90 Appendix E * Selected Student responses to pre-interview questions What form of teaching is the best? Subject 3:"Direct teaching lectures — because of immediacy. You get interaction" Subject 4: "By computer would be best, except for the problem that not everyone has access to a computer. Lectures are good, because they allow for interaction. Home study is also good, as it allows you to proceed at your own pace." Subject 1 l:"Anything graphic is good-there are too few diagrams in the code." Subject 14: "Visuals. Computer conferencing access." Subject 16:"Seminars are best, as they bring you face-to-face with the experts - code upgrading seminars are the most useful. However, I can see how computer assisted learning would be useful in conjunction." Subject 17:"The CP course was weak, because of the varying quality of presentation of code material, the inexperienced instructors, and the inadequate testing." Subject 24:"Computer conferencing sounds like a great idea. Magazines and newsletters are also good. CMHC used to promote innovative solutions to housing questions - that was a good program - it delivered lots of good info." Code experience. Describe your code review process. Subject l:"Osmosis. The senior people in the office instruct younger workers on the code." Subject 14:"Ask senior code expert in-house. Phone for municipal guidelines." Who should be involved in code education programs? 91 Subject l:"Plan checkers, architects, and code specialists" Subject 12:Groups involved in the industry, and established educational channels. Every group impacted should be involved." Appendix F - l Questionnaire 1 92 .1 NAME EMPLOYEES. 2 PHONE 1.3 FAX 1.4 INTERNET 1.5BIRTHDATE_ .6 YEAR OF GRAD. 1.7 FROM WHICH UNIVERSITY 1.8 YEAR OF ARTICLING COMPLETION 1.8.1. P.ENG. ARCH. .0 PERSONAL DATA 1.1.1 EMPLOYER . 1.1.1.1. NO. OF .9 ARE YOU A CERTIFIED PROFESSIONAL(CP) IN B.C.? Y N .10 DO YOU HAVE SOME OTHER CODE CERTIFICATION? Y _ _ N Please specify. .11 LENGTH OF COURSE (Hours) 1.12 GENDER 1.13AGE 1 14 OTHER FORMAL CERTIFICATION 1.14.1. FROM WHICH INSTITUTION 1.14.2 YEAR OF GRADUATION 1.14.3. OTHER COURSE COMPLETION OF NOTE (please state year and length) _ 2.0 COMPUTER EXPERIENCE 2.1 Do you use a computer at home? Y N 2.1.1 In the office Y N 2.2 How would you describe your computer ability? Very poor Good Excellent 1 2 3 4 5 6 7 2.3 Do you personally design buildings on the computer? Y N 2.4 What types of computer programs do you use? 2.4.1 Word processing 2.4.2 CAD 2.4.3 Spreadsheets 2.4.4 Graphics 2.4.5 Internet 2.5 How many hours do you spend on the computer per work day? (hours) 2.6 Computer conferencing as I know it should provide some interesting ways of understanding the code. Strongly Disagree Agree Strongly Agree 2 3 4 5 6 7 2.7 Have you learned through the computer previously? Y N 2.8 Have you taken computer training programs? (Y/N) 2.8.1. In the following areas? 2.8.1.1. Word Processing 2.8.1.2. CAD 2.8.1.3. Spreadsheets 2.8.1.4. Graphics 2.8.1.5. Internet 3.0 CODE KNOWLEDGE 3.1 How would you describe your code knowledge? Very poor Good Excellent 1 2 3 4 5 6 7 3.2 Do you have access to others with extensive code experience in the office? Y N 3.3 Are you THE code expert in the office? Y N 3.4 Rank from 1 (low) to 7(highest) your most common sources of code knowledge. 94 Appendix F-2 Questionnaire 2 BASELINE QUESTIONNAIRE FOR ARCHITECTS CODEWORKS RESEARCH PROJECT NAME: CODE SECTIONS: FACILITATORS: DATE: Mode - Mode In which class was presented (1) Completely Online (2) Partially Online (3) All Offline SOME BACKGROUND INFORMATION If you feel that any of these items invade your privacy, you are of course free to decline to answer them. How important are each of the following reasons for your taking this course and this particular section or mode of delivery course? Very Important, Somewhat Important, or Not Important? Very Somewhat Not Important Important Important PROFESSIONAL INTEREST GENERAL INTEREST VIRTUAL EXPLORATION CAREER DEVELOPMENT FACILITATORS REPUTATION CURIOUS I was curious about how the technology works CONVENIENCE More convenient than traditional classes EXPECTED GRADE What grade do you expect to receive in this course? A B C D EXPECTED DIFFICULTY How easy or difficult to you expect this course to be? E A S Y . 1 : 2 : 3 : 4 : 5 DIFFICULT SEX Male Female AGE 17-18 19-21 21 -25 26-34 35 + MAJOR NATIONALITY 95 (1) Canadian (2) Other IMAGES OF YOURSELF Please read each of the following and indicate how much you agree or disagree (1 = Completely DISAGREE: 7 means Completely AGREE). When I get what I want if s usually because I worked hard for it. DISAGREE 1 2 3 AGREE 6 7 I find it easy to play an important part in most group situations. DISAGREE 1 2 3 AGREE 6 7 I Drefer qames involving some luck over games requiring pure skill. DISAGREE 1 2 3 AGREE 6 7 Even when I'm feeling self-confident about most things, I still seem to lack the ability to control social situations DISAGREE 1 2 AGREE 6 7 I can learn almost anything if I set my mind to it. DISAGREE 1 2 AGREE 6 7 I have no trouble making and keeping friends. DISAGREE 1 2 AGREE 6 7 If s pointless to keep working on something that is too difficult for me. DISAGREE 1 2 3 4 AGREE 6 7 I'm not good at guiding the course of a conversation with several others DISAGREE 1 2 3 4 5 On any sort of exam or competition I like to know how well I do relative to everyone else. DISAGREE 1 2 3 4 5 AGREE 6 7 AGREE 6 7 I can usually establish a close personal relationship with someone I find attractive. DISAGREE 1 2 3 4 5 AGREE 6 7 Mv major accomplishments are entirely due to my hard work and ability. DISAGREE 1 2 3 4 AGREE 6 7 When I make plans I am almost certain to make them work. DISAGREE 1 2 3 AGREE 6 7 When being interviewed, I can usually steer the interviewer toward the topics I want to talk about and away from those I wish to avoid DISAGREE 1 2 AGREE 6 7 I usually don't set goals because I have a hard time following through on them. DISAGREE 1 2 3 4 5 AGREE 6 7 If I need help in carrying off a plan of mine, if s usually difficult to get others to help. DISAGREE 1 2 3 4 5 AGREE 6 7 Competition discourages excellence. DISAGREE 1 2 AGREE 6 7 96 If there's someone I want to meet, I can usually arrange it. DISAGREE 1 2 3 4 AGREE 6 7 Other people get ahead just by being lucky. DISAGREE 1 2 3 4 AGREE 6 7 I often find it hard to get my point of view across to others. DISAGREE 1 2 3 4 AGREE 6 7 In attempting to smooth over a disagreement, I usually make it worse. DISAGREE AGREE 3 7 YOUR PREVIOUS EXPERIENCE WITH COMPUTERS COMPUTER EXPERIENCE Which of the following best describes your previous experience with computer systems? (1) I am a NOVICE; seldom or never use computers (2) I have OCCASIONALLY used computer terminals and systems before (3) I have FREQUENTLY used computer systems (4) Use of computers is central to my PROFESSIONAL work For each of the following pairs of words, please circle the response that is closest to your CURRENT FEELINGS ABOUT USING COMPUTERS. For instance, for the first pair of words, if you feel computer systems in general are completely "stimulating" to use and not at all "dull," circle "1"; "4" means that you are undecided or neutral or think they are equally likely to be stimulating or dull; "3" means you feel that they are slightly more stimulating than dull, etc. Stimulating 1 2 3 4 5 6 7 Dull Fun 1 2 3 4 5 6 7 Dreary Easy 1 2 3 4 5 6 7 Difficult Personal 1 2 3 4 5 6 7 Impersonal Hindering 1 2 3 4 5 6 7 Helpful Threatening 1 2 3 4 5 6 7 Unthreatening Efficient 1 2 3 4 5 6 7 Inefficient Demanding 1 2 3 4 5 6 7 Obliging Reliable 1 2 3 4 5 6 7 Unreliable Desirable 1 2 3 4 5 6 7 Undesirable EXPECTATIONS ABOUT THE CONFERENCING SYSTEM Indicate your expectations about how it will be to use this system by circling the number which best indicates where your feelings lie on the scales below. 1 Hard to leam 1 Impersonal 1 Frustrating 1 3 3 3 5 5 6 6 6 6 Easy to leam 7 Friendly 7 Not frustrating 7 98 Appendix F-3 - Questionnaire 3 POST-COURSE QUESTIONNAIRE FOR ARCHITECTS CODEWORKS PROJECT NAME: CODE SECTIONS:. FACILITATORS: COURSE EFFECTIVENESS There are three sets of items in this section; we would like you to try to separate them out in your thinking. The first relates to the teaching or presentation style and effectiveness of your instructor; the second, to the course content; and the third, to the outcomes of the course for you. Later in the questionnaire, those who participated in an experimental mode of delivery will make direct comparisons between this course and traditional courses. For each of the following, please indicate the response that corresponds to the following scale: SA = Strongly Agree A = Agree N = Neither agree nor disagree (neutral) SD = Strongly Disagree The course content was interesting to me: SA Course content is important or valuable: Course goals were clear to me: SA SA COURSE CONTENT A N A N A N D D D SD SD SD Work requirements and grading system were unclear from the beginning: SA SD The reading assignments are poor: The lecture material is poor: The students had to work hard: This course was a waste of time: SA SA SA SA Is this course taught at an appropriate level? : 1 : 2 Too easy : 3 Just right A A A A : 5 Too difficult How would you rate this course over-all? (l)Excellent - (2) Very good (3) Good (4) Fair D D D D SD SD SD SD (5) Poor CHARACTERISTICS OF THE TEACHING SA A N Facilitator organized the course well SA N D D SD SD Grading was fair and impartial SA SD Facilitators seems to enjoy teaching 99 SA A N D SD Facilitators lacks sufficient knowledge about the subject area SA A N D SD Students were encouraged to express ideas SA A N D SD Instructor presented material clearly and summarized points SA A N D SD Facilitator discussed points of view other than her/his own SA A N D SD The student was able to get personal help in this course SA A N D SD Facilitator presented material in a boring manner SA A N D SD Facilitator critiqued my work in a constructive and helpful way SA A N D SD Overall, I would rate this facilitator as: (Facilitator names _ (1) Excellent (2) Very good (3) Good 4) Fair (5) Poor Comments about the facilitator or the teaching? OUTCOMES OF THE COURSE I became more interested in the subject SA A N D SD I learned a great deal of factual material SA A N D SD I gained a good understanding of basic concepts SA A N D SD I learned to identify central issues in this field SA A N D SD I developed the ability to communicate clearly about the subject SA A N D SD My skill in critical thinking was increased SA A N D SD I developed an understanding of ethical issues SA A My ability to integrate facts and develop generalizations improved SA A N D N D I regularly completed the required readings SA A N D I was stimulated to do additional reading SA A N D I participated actively in class discussion SA A N D I was stimulated to discuss related topics outside of class SA A N D The written assignments aided my learning SA A N D I regularly completed the written assignments SA A N D 1 was forced to think for myself SA A N D 1 became more confident in expressing my ideas SA A N D 1 developed new friendships in this class SA A N D 1 learned to value other points of view SA A N D 1 was motivated to do my best work 100 SA A N D I gained a better understanding of myself SA A N D I increased my competence with computers SA A N D I learned to see relationships between important topics and ideas SA A N D My ability to critically analyze written material was improved SA A N D GENERAL INFORMATION About how much TOTAL time have your spent each week on this course (including "in class" and out, reading and writing, on and offline) 1. Less than one hour 2. 1-2 hours 3. 3-4 hours 4. 5-9 hours 5. Ten hours or more DIFFICULT How easy or difficult was this course for you? EASY 1 2 3 4 What grade do you expect to receive in this course? A B C D F Individual vs. Group Learning Some courses are essentially a very INDIVIDUAL experience; contact with other students does not play an important part in your learning. In other courses, communication with other students play a dominant role. For THIS COURSE, please circle the number below that seems to be what you experienced. Individual experience The help I got from other students was -1 : Crucially important to me Student in my class tended to be 1 : : Not at all cooperative 1 : i Group experience 4 5 Useless or misleading 4 5 Extremely cooperative 4 5 Extremely competitive Not at all competitive How often did you communicate with other students outside of class, by computer, "face-to-face" or on the telephone? • 2 : 3 : 4 : 5 : Constantly 1 Never ATTITUDES TOWARD COMPUTERS For each of the following pairs of words, please circle the response that represents where you fall on the scale in terms of your CURRENT FEELINGS ABOUT USING COMPUTERS. Stimulating Fun Easy Personal 6 Dull 6 Dreary 6 Difficult 6 Impersonal 101 Hindering Threatening Efficient Demanding Reliable Desirable 2 2 2 2 2 2 3 3 3 Undesirable ATTITUDES TOWARD MEDIA To what extent do you agree with the following statements? I enjoy listening to lectures 1 : 2 : 3 : 4 : ! Strongly Agree I like to read. 1 : 2 : 3 : 4 : : Strongly Agree I have difficulty expressing my ideas in writing 1 : 2 : 3 : 4 : Strongly Agree I like to take part in class discussion 1 : 2 : 3 : 4 : Strongly Agree 6 Helpful Unthreatening 6 Inefficient 6 Obliging Unreliable 6 Strongly Disagree 6 7 Strongly Disagree 6 7 Strongly Disagree 6 7 Strongly Disagree PARTICIPATION IN THE ONLINE COURSE If your participated in a traditional course or a course which did not include any online work, skip the rest of the questionnaire. Is access to a terminal or micro for the online class a problem for you? 1 : 2 : 3 : 4 Serious Problem Not a Problem To what extent has the slow response of the EIES system been a problem or barrier for you? 1 : 2 : 3 : 4 : 5 : Serious Problem Not a Problem How much problem have you had with "busy" lines or no available ports to EIES? : 1 2 3 4 Serious Problem Not a Problem To what extent has the slow response of the EIES system been a problem or barrier for you? : 1 : 2 : 3 : 4 5 Serious Problem Not a Problem EXPERIENCES WITH EIES Indicate your experiences with using this system by circling the number which best indicates where your feelings lie on the scales below. Hard to learn Impersonal Frustrating Unproductive 2 2 2 2 3 3 3 3 4 4 4 4 5 5 5 5 Easy to learn 6 Friendly 6 Not frustrating 6 Productive 102 Did use of the System increase the efficiency of your education (the quantity of work that you can complete in a given time)? 1 • 2 : 3 4 5 6 Definitely yes U n s u r e Definitely not Did use of the System increase the quality of your education? 1 Definitely yes 3 4 5 Unsure Definitely not COMPARISON TO TRADITIONAL CLASSROOMS Please compare online "classes" to your previous experiences with "face to face" college-level courses. To what extent to you agree with the following statements about the comparative process and value of the EIES online course or portion of a course in which you participated? (Circle a number on the scales.) Taking online courses is more convenient. 1 : 2 : 3 Strongly Agree Strongly Disagree I felt more "inhibited" in taking part in the discussion. 1 : 2 : 3 : Strongly Agree Strongly Disagree I didn't have to work as hard for online classes. 1 : 2 : 3 Strongly Agree Strongly Disagree I communicated more with other students in the class as a result of the computerized conference. 1 : 2 : 3 : 4 : 5 -Strongly Agree Having the computerized conferencing system available provided better access to the professor(s). 1 : 2 : 3 : 4 : 5 Strongly Agree The fact that my assignments would be ready by the other students increased my motivation to do a thorough job. 1 : 2 : 3 : 4 : 5 Strongly Agree When I became very busy with other things, I was more likely to stop participating in the online class than I would have been to "cuf a weekly face-to-face lecture. 1 : 2 : 3 : 4 : 5 Strongly Agree The online or virtual classroom mode is more boring than traditional classes. 1 2 : 3 4 5 Strongly Agree I felt more "involved" in taking an active part in the course. 1 : 2 : 3 : 4 : 5 Strongly Agree I found the comments made by other students to be useful to me. 1 2 3 4 5 Strongly Agree I found reading the reviews or assignments of other students to be useful to me. 1 : 2 : 3 : 4 : 5 Strongly Agree I would NOT choose to take another online course. 1 : 2 : 3 : Strongly Agree 6 Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree Strongly Disagree 6 Strongly Disagree Strongly Disagree Strongly Disagree 6 Strongly Disagree 104 Appendix F-4 - Pretest PRETEST ON 3.1.2. • 3.1.3.. 3.2.1. and 3.2.2. OF THE B.C. BUILDING CODE 1992 INSTRUCTIONS: This is a self-administered and independently completed open book exam. Please do not take any longer than 2 hours to complete the following questions: 1.0 In your own words define: (4 marks each) 1.1.GRADE 1.2 FIRST STOREY 1.3 BUILDING AREA 1.4 MAJOR OCCUPANCY 1.5 OCCUPANCY 1.6 BUILDING HEIGHT 1.7 FLOOR AREA 1.8 STOREY 1.9 HEIGHT 1.10 FIRE RESISTANCE RATING 1.11 FTRE SEPARATION 1.12 HORIZONTAL FIRE SEPARATION 1.13 VERTICAL FTRE SEPARATION 1.14 MEZZANINE 2.0 Classify the following uses according to occupancy: (2 marks each) 2.1 PET STORE 2.2. BAKERY 2.3. FTRE H A L L _ _ 2.4 GROUP HOME 2.5. DAY C A R E _ _ 2.6 WELDING SHOP 2.7. CONCRETE BLOCK MANUFACTURING PLANT 2.8 FAST FOOD RESTAURANT WITHOUT SEATING 2.9 FAST FOOD RESTAURANT WITH SEATING 2.10 POLICE STATION 900 SQ. M. 1 STOREY IN BUILDING HEIGHT 2.11 50 UNIT 105 CONDOMINIUM WITH A COMMUNITY CARE FACILITY IN ONE SUITE (classification of suite) 3.0 What are the two instances where major occupancy does not need to be the most restrictive occupancy applying to the whole building (2 marks) 4.0 What are the fire separations required between the following occupancies? (1 mark each) 4.1 D and D 4.2 D and E 4.3 E and 2 dwelling units in a 2 storey building 4.4. E and C 4.5F-landB-2 4.6 Non-major occupancies of C andF-1 4.7 Non-major occupancies of E and C 4.8 Between major occupancies surrounding an atrium classified as D and C 4.9 Non-major occupancies of E and E 5.0 Sketch the following building and label the most restrictive major occupancy. Then classify the uses according to 3.2.2, and indicate the amount of fire resistance rating: (8 marks each) 5.1 1000 sq.m. in building area. 1st. floor - storage garage, 2nd. floor - 50% office and 50% grocery store, 3rd. floor - residential, 4th. floor - one suite non-ambulatory use with an area of 110 sq. m, and 5th. floor - mechanical room penthouse with an 80 sq. m. mezzanine, facing 2 streets, sprinklered. 5.2 2 storeys in building height, 800 sq.m. in building area. 1st. floor - library, 2nd. floor -offices - 700 sq.m. and council chambers - 100 sq.m., facing one street, unsprinklered. 6.0 Sketch and classify the following buildings according to 3.2.2., and indicate the amount of fire resistance rating: (8 marks each) 6.13 storeys, storage garage in basement, store on 1st. floor, residential on 2 and 3rd. storeys, mezzanine on 3rd. storey less than 100 sq.m., mezzanine on 2nd. storey less than 100 sq. m. building area 1000 sq.m., facing 2 streets, sprinklered, with a tennis court on the roof. 6.2 4 storeys, building area 1000 sq. m., basement - tenant storage, 1st floor - children's custodial home 80 sq.m., store of 920 sq.m., 2nd. floor to 4th. floor residential, crawl space 2.0 m in height, sprinklered, and facing 1 street, with exterior balconies. 7.0 Determine the number of streets this building faces ( 5 marks) Lane 6 m wide with 30 m sides SITE PLAN - N.T.S. 107 Appendix F-5 - Posttests Section Review 1. a)Classify the following uses according to occupancy: PET STORE BAKERY FIRE HALL GROUP HOME DAY CARE WELDING SHOP CONCRETE BLOCK MANUFACTURING PLANT FAST FOOD RESTAURANT WITHOUT SEATING FAST FOOD RESTAURANT WITH SEATING POLICE STATION 900 SQ. M. 1 STOREY IN BUILDING HEIGHT 50 UNIT CONDOMINIUM WITH A COMMUNITY CARE FACILITY IN ONE SUITE (classification of suite) b) What determines the occupancy classification of a fast food restaurant? 2. If an arena was used more often than "occasionally" for trade shows how would you classify and determine construction requirements? How would you define occasional use? 3. In which instances is major occupancy the most restrictive occupancy applying to the whole building? 4. Table 3.1.3.A determines fire separations between all occupancies T F 5. What are the fire separations required between the following occupancies:D and D D and E E and 2 dwelling units in a 2 storey building E and C F-1 and B-2 Non-major occupancies of C and F-1 Non-major occupancies of E and C 4.8 Between major occupancies surrounding an atrium classified as D and C_ Non-major occupancies of E and E 6. Contact at least one facilitator and state the reasons why an F-1 major occupancy building cannot house an A,B or C occupancy. 7. From the following sketch determine: a) the fire rating between each occupancy b) the major occupancy of the building cjthe occupancy classifications 108 hardware store 300 sq.m. restaurant no seating 200 sq.m. woodworking shop 300 sq.m. cafeteria space 50 sq.m. day care 150sq.m. department store 600 sq.m. BUILDING FLOOR PLAN building area=1500 sq.m. Section 2 Review and Comments This section review discusses code jurisdiction and application. 1. a) Which building code is in effect in the City of Victoria? b)Which jurisdiction applies the code? c)What two documents empower the building code? 2. The National Building Code may be applied in many jurisdictions in the Province of B.C. in the absence of any other legislation. T F 3. The public review process has no legal effect nationally. T F 4. The municipality may alter the building code in certain key areas such as: a) Fire walls and fire separations T F b) Fire protection by requiring increased sprinklerization T F c) Enforcement techniques such as adopting other codes that may have more effect T F d) Fees and legal remedies T F 5. Conferencing with at least two other people on-line describe the code adoption process for municipal jurisdictions in the Province (excluding Vancouver). The discussion should include descriptions and relationships of the following functions: public review, national model code, time cycles, legislative authority, provincial and municipal amendments, and jurisdiction. 109 Appendix F-6 - Pre-Interview INTERVIEW: I would now like to conduct a short (15 minute) regarding your views. TELEPHONE CONTACT F O R M - CANDIDATES NAME • PHONE DATE INTERVIEWER INTERVIEW QUESTIONS. > , \ .7.". \ - 5 1.0 COMPUTER EXPERIENCE 1;„ J Computers are used in many offices to enhance work functions such as drafting, planning presentations, word processing, and internet communications. Many codes are being delivered in data based formats with integrated search mechanisms. Software has developed to the extent that user friendly navigation is enhanced and expanded. Computer conferencing software is attempting to fill the interactional gaps missing in most distance delivery programs. However, to what extent does the delivery of material in this format expand the ability of the learner to learn while working (just-in-time) while synthesizing the classroom interactions and consequently improve on classroom delivery formats? 1.1 Have you programmed a computer before? y n 1.2 What CAD program do you use? 1.3 What problems can you foresee with a computer delivered code program at work? 1.4 How would you describe your familiarity with the computer? 1.5 What are the advantages and disadvantages? 1.5.1. Advantages 1.5.2. Disadvantages 1.6 What is the most effective method of code official contact? 1.7 How often do you use the computer at work? Hours per day Comments 1.8 Do you have computer educational support at work? y n 1.8.1 How would you describe its sufficiency? 2.o.comwm^^i^^^. . - J Code understanding is mainly achieved by osmosis at work and important but rare contacts with building officials. However, it is seen as a synthesis of design and safety principles. There are significant number of confrontations over code precepts leading to costly delays and frustrations between major participants. Expanding the role of the building official and bridging the communication gaps existent between the design and safety professions is seen as another major advantage o f the conferencing delivery format. 2.1 Describe your code review process? 2.2 What reference material do you use? 2.3 Which building officials office do you contact most often? and how often? [ 110 2.4 What forms of contact do you use? 2.4.1. Which is the most beneficial and why? 2.5 What ways can government interaction be improved? 2.6 What are some of the main problems with today's code? 2.7 What part of the code review process do you find most interesting? most confusing? 2.8 Can you name the major construction codes? 2.9 How would you describe your code understanding? 2.91 What has contributed the most? 2.911 The least? 2.10 What assessments do you have of the current code application process? The current adoption process? 2.11 Who should be involved in code education programs? (Please give priorities) 2.12 There are many types of code delivery, video, slides, basic lecture, computer assisted, computer conferencing. What is the best delivery format? and why? 3.0 WORK EXPERIENCE ..' ''"-"1*, 3.1 What are the differences between municipal jurisdictions in code application? 3.11 What ways can they be improved? 3.2 Confrontation occurs on the job site, what is the biggest reason for that confrontation? 3.3 Your work experience includes previous construction experience. If and how has this prepared you for the design field? 3.4 What could be added to design experience to enhance effectiveness? Thank you very much. Please contact me or the other investigators if you have any questions. Post Interview INSTRUCTIONS Please familiarize yourself with the research objectives . Be sure to read the introduction to the candidate before proceeding to the following questions. INTRODUCTION Good day. My name is Lynn Farquhar and I am assisting in the investigation and development of the CONEXUS/CODEWORKS PROJECT that you have volunteered to participate in. We would like to follow up on the project by asking that you respond to a couple of questions. These will be gathered and reported confidentially and will assist in building the CONEXUS WEB SITE. The interview should not take any longer than 15 minutes. Ill Appendix F-7 - Post Interview TELEPHONE INTERVIEW - AFTER PROGRAM DELIVERY STUDENT NAME INTERVIEWER DATE TIME: START FINISH LOCATION INTERVffiW QUESTIONS , A Gaining knowledge about the building code has been one of the main objectives of this course. But also it is important to consider the degree to which the technology has permitted you to access resources and knowledge. The building code tends to be viewed as very complicated and interpretive. Some pre-course interviews were conducted in which candidates described their views on a number of issues including the building code and how it should be applied and taught, your work experience and your computer knowledge. We would now like to explore your views concerning this computer conferencing code course by having you answer the following questions: 1.1 How did you first hear about this code course? What were your initial feelings or reactions...what attracted you, what didn't sound good about this approach? 1.2 How about the initial training session...after it was over, did you feel that you would be able to sign on line and find and access the material or was there something that was not clear about what the procedure would be? 1.3 Was the computer easily accessible in the office and where did you go to use it? Were there any problems with availability? Did you have any sort of regular schedule each week when you would sign on line to participate, or how was it that you decided when to log on? 1.4 What were your initial feelings or impressions about the online class during the lead up in the first week? Can you remember what you particularly liked, or what you didn't like or found confusing? (probe....anything else?) 1.5 What were your initial reactions to reading the comments or contributions by the other participants...to what extent did you find this interesting or helpful, and to what extent did you feel this was a waste of time? Why? Did you feel that you were part of a group or class working together, or helpful, and to what extent did you feel this was a waste of time? Why? Did you feel that you were part of a group or class working together, or did you feel that your were pretty much alone in learning the material? (If felt part of group). Did you or the instructor do anything in particular that helped you to be able to work and socialize with other participants in the on line class? 112 1.6 How about the lecture-type material presented by the instructor... did you find it easier to understand that material in writing, or do you think you would have learned it better if you had listened to it in spoken form? Why? 1.7 Did you ever look at or join any of the public conferences on the system, besides the conference within your group? If yes, which ones, and what did you think of them? If no...why not? Did you ever exchange messages with anybody online that was not connected with the project? If yes....how did this happen? How did you feel about this experience of communicating with "strangers"? 1.8 How would you describe your relationship to the facilitator online? Do you feel MORE or LESS able to communicate and relate to your teacher? Why? 1.9 Q9 examines initial and current reactions to on-the-job learning with the computer based on the initial questionnaire. Question 2.6 asked for a response from 1-7 strongly disagree to strongly agree, to the statement that Computer conferencing as I know it should provide some interesting ways of understanding the code. Your reaction was What if your reaction now? Have you developed any particular routines or tricks of the trade that are making computer conferencing more valuable to you than it was at first? At this point in your online course, what do you like best about the Virtual Classroom approach... that is, what is good about it compared to a course given in the traditional classroom? (probe .... anything else?) What do you currently like least, or feel are the greatest problems or shortcomings about this mode of course delivery? 1.10 What advice would you give a student who is thinking of signing up for an online course? How about your instructor...what advice would you give about how they could be more effective if they try teaching this course online again? 1.11 Is there anything else you would like to tell us about your experiences... anything that was especially funny or memorable, or valuable, or unpleasant about your experience? 113 Appendix G- Conferencing itinerary: 1. February 20: Session commences with seminar at PMTI in North Vancouver. Project introduction and development of conferencing networkers. Completion of consent form and surveys. 2. March 25-29: Gathering and collating of data to identify two groups of users. Facilitator orientation instruction and testing on line. Development of user references and on line linkages for conferencing participants. Facilitator pre-course interviews and questionnaires. 3. Anril 16-17: Conferencing warm up discussion with facilitators to test and run system. Discussion subject development will include current topics such as stucco application, objectives based codes and hot topic generation. 4. Anril 18-19: Conferencing warm up for all participants. Discussion topics will extend from facilitator warm up. Issues identified as stucco application, objectives based codes and hot topics. 5. April 22-26:(Week 1). April 29-Mav 3:(Week 2). May 6-10: (break). May 13-17: (Week 3) Conferencing on line with code course material, tests, and discussion items. Time lines will be established for contribution deadlines and project development. Code sections are 3.1.2, 3.1.3, 3.2.1, and 3.2.2. 6. Mav 20-24: (Week 4) Cool down session. Debriefing, project summary, overall comments and posttest. 7. Mav 27-.Tune 30 Data gathering and reporting. Project team reconfiguration and on line course redevelopment. 8.September 1 - November 30 Finalization of on line course development. Exploration and development of Web Site of linkages. Municipal coordination and facilitation. itinl/jg/mar96 Appendix H - Selected Interviews Subject 19 - Pre-Interview 1.0 COMPUTER EXPERIENCE Computers are used in many offices to enhance work functions such as drafting, planning presentations, word processing, and internet communications. Many codes are being delivered in data based formats with integrated search mechanisms. Software has developed to the extent that user friendly navigation is enhanced and expanded. Computer conferencing software is attempting to fill the interactional gaps missing in most distance delivery programs. However, to what extent does the delivery of material in this format expand the ability of the learner to learn while working (just-in-time) while synthesizing the classroom interactions and consequently improve on classroom delivery formats? 1.1 Have you programmed a computer before? No 1.2 What CAD program do you use? Generic Cad 1.3 What problems can you foresee with a computer delivered code program at work? 1.3.1. None, except that people will need to be convinced that it's user friendly. 1.4 How would you describe your familiarity with the computer? Ok, but I'm not comfortable with Windows. 1.5 What are the advantages and disadvantages? 1.5.1. Advantages 1.5.1.1. There's a higher standard of presentation. More accurate documentation. 1.5.1.2. The quality [of the product] is higher. 1.5.1.3. Accuracy — in my case, for conceptual and design planning. I can't imagine hand drafting anymore. 1.5.2. Disadvantages 1.5.2.1. It's slower. 1.6 What is the most effective method of code official contact? Phoning John Guenther [much laughter]. The phone. 1.7 How often do you use the computer at work? Hours per day 4 1.8 Do you have computer educational support at work? No 1.8.1 How would you describe its sufficiency?All I use are the manuals that come with the software. 2.0 CODE EXPERIENCE Code understanding is mainly achieved by osmosis at work and important but rare contacts with building officials. However, it is seen as a synthesis of design and safety principles. There are significant number of confrontations over code precepts leading to costly delays and frustrations between major participants. Expanding the role of the building official and bridging the communication gaps existent between the design and safety professions is seen as another major advantage o f the conferencing delivery format. 2.1 Describe your code review process? On a given project, I'll do a classification on fire ratings and stair exiting patterns, then I'll do a formal summary for the building department I'm dealing with. 2.2 What reference material do you use? ULC; CSA. 2.3 Which building officials office do you contact most often? North Van City; Port Coquitlam; Coquitlam. and how often? Seldom — three or four times per year. 2.4 What forms of contact do you use?I set up a meeting by written memo or by phone. 2.4.1. Which is the most beneficial and why?Phone. Then face to face is good, too. It's nice to meet on site to discuss contentious issues. 2.5 What ways can government interaction be improved? 2.5.1. By making more time available to get to know the people. 2.5.2. Open dialogue and listen to the opinions of the public. 2.6 What are some of the main problems with today's code? 2.6.1. Problems? Maybe equivalences. I like it [the code] -- it's interpretive. There are things that just can't be expressed in black and white. 2.7 What part of the code review process do you find most interesting? 2.7.1 .Fire separations^  2.7.2. Exiting patterns. 2.7.3. Mezzanines and inter connective floors, most confusing? 2.7.4. Limiting distances. 2.8 Can you name the major construction codes? BC Building code; Fire Safety code; Electrical code. 2.9 How would you describe your code understanding?Pretty good — but I'm an open-booker. 2.91 What has contributed the most? Twenty-six years of experience. 2.911 The least? My confusion over limiting distance. 2.10 What assessments do you have of the current code application process? 2.10.1. Government offices need more manpower, especially in plan checking departments. 2.10.2. There should be more code review in government offices, by plan checkers. An incorrect height of railing should be picked up on the plans, not in the field. The current adoption process? 2.10.4. It takes too long. 2.11 Who should be involved in code education programs? (Please give priorities) The teachers should be guys such as myself. Professionals. Also, inspectors. The students should be drafts men. 2.12 There are many types of code delivery, video, slides, basic lecture, computer assisted, computer conferencing. What is the best delivery format? and why? Reading the code out of a binder is fine by me. But on computer, too, if possible. 3.0 WORK EXPERIENCE 3.1 What are the differences between municipal jurisdictions in code application? Dramatic, because of personalities, policies and procedures. 3.11 What ways can they be improved? By legislating a uniform intent. 3.2 Confrontation occurs on the job site, what is the biggest reason for that confrontation? Someone screws up -- the inspector, the builder, or the professional. Poor documentation is a reason ~ bad plans. Ass-covering is a bit reason for confrontation. When a mistake is found, an immediate defence mechanism clicks in. 3.3 Your work experience includes previous construction experience. If and how has this prepared you for the design field? It was absolutely invaluable. I know the methodology, the steps, stages, details and the interface. 3.4 What could be added to design experience to enhance effectiveness? More full service work. More old-style architectural training. They should see the project right through from tendering to completion. Perhaps designers could provide supervision on site. That would be invaluable. Thank you very much. Please contact me or the other investigators if you have any questions. Subject 27 - Post Interview INTERVIEW QUESTIONS Gaining knowledge about the building code has been one of the main objectives of this course. But also it is important to consider the degree to which the technology has permitted you to access resources and knowledge. The building code tends to be viewed as very complicated and interpretive. Some pre-course interviews were conducted in which candidates described their views on a number of issues including the building code and how it should be applied and taught, your work experience and your computer knowledge. We would now like to explore your views concerning this computer conferencing code course by having you answer the following questions: 1.1 How did you first hear about this code course? What were your initial feelings or reactions...what attracted you, what didn't sound good about this approach? I guess that I heard about it through the ATBC. I thought it was a great idea. It seemed like a good chance to create some dialogue. 1.2 How about the initial training session...after it was over, did you feel that you would be able to sign on line and find and access the material or was there something that was not clear about what the procedure would be? ~ I thought that the procedure was unclear. Overall, I had no comprehension of what the objectives were. That still remains unclear. Also, I had trouble with the passwords. I was sent a memo where the password was in capitals. This didn't work when I went to apply it. It should be been in lower case letters. Maybe a memo outlining all the possible scenarios where problems might occur would have helped alleviate a lot of my confusion. 1.3 Was the computer easily accessible in the office and where did you go to use it? Were there any problems with availability? Did you have any sort of regular schedule each week when you would sign on line to participate, or how was it that you decided when to log on? I did it both at home and at the office. I have my own terminal at my desk. I sign on-line every morning. 1.4 What were your initial feelings or impressions about the online class during the lead up in the first week? Can you remember what you particularly liked, or what you didn't like or found confusing? (probe.... any thing else?) There were notices that came out saying that something would be posted and yet nothing was posted afterward. I understand that this had nothing to do with John — it was the server that was responsible for the delays. But I found this confusing. But, when everything was up and running, the actual web page was clear enough. 1.5 What were your initial reactions to reading the comments or contributions by the other participants...to what extent did you find this interesting or helpful, and to what extent did you feel this was a waste of time? Why? I made some interesting contacts. I don't think it was a waste of time, but I sense that a lot of us were nervous about putting our thoughts in writing and then posting them for everyone to see. Did you feel that you were part of a group or class working together, or helpful, and to what extent did you feel this was a waste of time? Why? Did you feel that you were part of a group or class working together, or did you feel that your were pretty much alone in learning the material? I felt pretty much alone. (If felt part of group). Did you or the instructor do anything in particular that helped you to be able to work and socialize with other participants in the on line class? 1.6 How about the lecture-type material presented by the instructor... did you find it easier to understand that material in writing, or do you think you would have learned it better if you had listened to it in spoken form? Why? I would prefer a lecture. There's more immediacy when someone is speaking. I think that the web site should have had audio and lots of graphics to be as effective [as a lecture]. 1.7 Did you ever look at or join any of the public conferences on the system, besides the conference within your group? Yes. If yes, which ones, and what did you think of them? If no...why not? Did you ever exchange messages with anybody online that was not connected with the project? I went into the coffee shop and the al discussion. I think I posted one message. I didn't exchange messages with anyone not connected with the project. If yes....how did this happen? How did you feel about this experience of communicating with strangers? O.K. I do that every day anyhow. 1.8 How would you describe your relationship to the facilitator online? Do you feel MORE or LESS able to communicate and relate to your teacher? Why? I feel no different than I would if I were not online. 1.9 Q9 examines initial and current reactions to on-the-job learning with the computer based on the initial questionnaire. Question 2.6 asked for a response from 1-7 strongly disagree to strongly agree, to the statement that Computer conferencing as I know it should provide some interesting ways of understanding the code. Your reaction was 4 What is your reaction now? I'd go up to a six now. I see the potential there — but I think there's a lot of work that has ta.be done to make it reach its full potential ~ such as the audio component and the graphics. Have you developed any particular routines or tricks of the trade that are making computer conferencing more valuable to you than it was at first? Yes, two things. Now I first prepare the written material before posting it. Also, I capture the material on the site and print it. At this point in your online course, what do you like best about the Virtual Classroom approach... that is, what is good about it compared to a course given in the traditional classroom? (probe .... anything else?) I like the flexibility I'm given in terms of accessing the information. I can do it at my own time. I like having the opportunity to review information, too. I can take my time. What do you currently like least, or feel are the greatest problems or shortcomings about this mode of course delivery? The on-screen design of the course material. This must be changed. There should be audio; graphics with scroll bars; compartmentalization of video and text; interactive response tools; better word processing. The form that had to be filled out is an example. It was hard to use. It took a long time. When the steps you have to go through to make something work are time-consuming and awkward, it's discouraging . 1.10 What advice would you give a student who is thinking of signing up for an online course? I'd advise them to find a way to have a face-to-face or voice connection as well. How about your instructor...what advice would you give about how they could be more effective if they try teaching this course online again? An instructor — I'd tell him to thoroughly explore the design and media aspects of the material and to make the presentation interesting as possible. 1.11 Is there anything else you would like to tell us about your experiences... anything that was especially funny or memorable, or valuable, or unpleasant about your experience? (no answer) 1.12 What other comments do you have? I think that there shouM be more linkages to outside resources in the web site. Affliliated interest groups should be linked. And the technical stuff has to be cleaned up. The technical problems when the site was starting up should be fixed for good. That was really starting off on the wrong foot. Subject 21 - Post Interview INTERVIEW QUESTIONS Gaining knowledge about the building code has been one of the main objectives of this course. But also it is important to consider the degree to which the technology has permitted you to access resources and knowledge. The building code tends to be viewed as very complicated and interpretive. Some pre-course interviews were conducted in which candidates described their views on a number of issues including the building code and how it should be applied and taught, your work experience and your computer knowledge. We would now like to explore your views concerning this computer conferencing code course by having you answer the following questions: 1.1 How did you first hear about this code course? What were your initial feelings or reactions...what attracted you, what didn't sound good about this approach? I know John Guenther through the Board of Variance in North Vancouver. He told me about it. My first reaction was that it was an excellent idea. I thought it would give me a chance to work more on the computer. 1.2 How about the initial training session...after it was over, did you feel that you would be able to sign on line and find and access the material or was there something that was not clear about what the procedure would be? Yes, I felt ok. 1.3 Was the computer easily accessible in the office and where did you go to use it? Were there any problems with availability? Did you have any sort of regular schedule each week when you would sign on line to participate, or how was it that you decided when to log on? I signed on at the office. Yes, I had a schedule. I worked on-line first thing in the morning and the last thing before I went home in the evening. 1.4 What were your initial feelings or impressions about the online class during the lead up in the first week? Can you remember what you particularly liked, or what you didnet like or found confusing? (probe....anything else?) It was clear to me what was supposed to happen, but I couldn't do certain things I was supposed to be able to do. There were times when I could get the first cl conference information, yet if I would go back into it later that day, only 50% would show up, although it said the document was full. 1.5 What were your initial reactions to reading the comments or contributions by the other participants...to what extent did you find this interesting or helpful, and to what extent did you feel this was a waste of time? Why? It was helpful to see what other people were thinking and doing. Did you feel that you were part of a group or class working together, or helpful, and to what extent did you feel this was a waste of time? Why? Did you feel that you were part of a group or class working together, or did you feel that your were pretty much alone in learning the material? Yes, I did feel as though it was a class. (If felt part of group). Did you or the instructor do anything in particular that helped you to be able to work and socialize with other participants in the on line class? Apart from the odd prompts sent by John Guenther, I felt as if I had no instructor. 1.6 How about the lecture-type material presented by the instructor... did you find it easier to understand that material in writing, or do you think you would have learned it better if you had listened to it in spoken form? Why? I've taken a code course already. I found that one a lot more gruelling. It was more intensive. 1.7 Did you ever look at or join any of the public conferences on the system, besides the conference within your group? Yes. If yes, which ones, and what did you think of them? If no...why not? Did you ever exchange messages with anybody online that was not connected with the project? al and dl. No, I don't think I exchanged messages with anyone not connected with the project. If yes....how did this happen? How did you feel about this experience of communicating with strangers? It felt like quite a novelty to be able to go in there and introduce myself. 1.8 How would you describe your relationship to the facilitator online? Do you feel MORE or LESS able to communicate and relate to your teacher? Why? Yes, I felt more able to communicate with my teacher, as I could use both E mail and the phone. 1.9 Q9 examines initial and current reactions to on-the-job learning with the computer based on the initial questionnaire. Question 2.6 asked for a response from 1-7 strongly disagree to strongly agree, to the statement that Computer conferencing as I know it should provide some interesting ways of understanding the code. Your reaction was 7 What is your reaction now? Seven. I haven't changed my mind. Have you developed any particular routines or tricks of the trade that are making computer conferencing more valuable to you than it was at first? When I was trying to E mail at the outset, I would be typing the same message at the terminal three or four times. So I started printing out everything I wrote so that I wouldn't lose it in the transfer. This way, I was able to eliminate the Codeworks background and also save some cybertime. But I still feel like a novice. At this point in your online course, what do you like best about the Virtual Classroom approach... that is, what is good about it compared to a course given in the traditional classroom? (probe .... anything else?) If need be, you have a support group within the office. I like that aspect. And time is a valuable commodity. It's great to be able to educate yourself at work. And there's no commuting involved. What do you currently like least, or feel are the greatest problems or shortcomings about this mode of course delivery? The interaction with peers is lost. 1.10 What advice would you give a student who is thinking of signing up for an online course? I would advise a student thinking of signing up for THIS online course that it's for those who have an interest in how to deal with the code, and not to treat code understanding as a barrier to learning design. How about your instructor...what advice would you give about how they could be more effective if they try teaching this course online again? At the initial seminar at the Pacific Marine Institute, we should have had an initial group session to meet other conference delegates. The instructor should have then divided us into small groups so that we could follow through the instructions all together, at the same time. In the coffee shop and in the al conference, I found that people were expressing their frustration with the course. 1.11 Is there anything else you would like to tell us about your experiences... anything that was especially funny or memorable, or valuable, or unpleasant about your experience? Not being able to get everything I needed on the screen was a problem. And I didn't like the background — it was distracting. And I found the quality of the graphics poor. 1.12 What other comments do you have? inter3/jg/96 125 Appendix I Project Development - Concerns Introduction Thank you for your attendance and contribution at the Codeworks session held on December 11, 1995 at the Sun Microsystems office. The key objective of the session was to garner feedback in preparation for an expanded seminar to be held in February, 1996. Your contributions were very valuable. Summary of discussion Discussion areas follow: 1) How will the system data that is generated be managed and manipulated into a usable form? The idea is to deliver programs that are current and relevant. The source of the program should be a team of individuals that, in the case of code information, include the key stakeholders (contractors, architects, consultants, and building officials). The opportunity to interact positively and coalesce valuable information is a key objective of the project. Maintenance and re-generation of the database, it is hoped will be part of the ongoing efforts of associations and government agencies. 2) The system should not be described as an expert one, unless it is thought of as a repository of opinions that can serve as a touchstone for further reference. 3) How is the technology different than can be currently be generated by internet email? Web technology and authenticated newsgroups allow restricted interaction to define objectives and build information that is definitive and searchable. Presently email is used for mainly one on one communication and is usually initiated by specific questions isolated from the main bodies of knowledge. Not that the information is not valuable, but rather that it is sporadic and not managed in a concerted fashion. Many of us use email within the office to discuss a number of subjects including: a) problem solving b) filing information and reference c) notification and publicity This project examines problem solving mechanisms and the retention of knowledge, through idea generation and systematic management. 4) Where would the participants find the time to utilize and access the information? How functional is information unless it can be easily accessed and processed, when it is needed most? Creating the opportunities to interact is seen as one of the keys to building relationships between construction participants. Distilling and manipulating the information so that it is beneficial is important to project success. The project will concentrate on one or two specific sections of the code and test information gains, in order to gauge the effectiveness of the opportunity to interact through the technology. Although the technology is a concern in building familiarity, it is not paramount to project success. Success, it is suggested is mainly dependent upon the willingness and desire to interact to create innovative solutions to both fundamental and functional problems. To this end it is suggested we examine pervasive problems and develop mechanisms that will achieve ongoing processes that can address solutions. 5) Some building developments tend to be site and time dependent. The final authority must continue to reside with the authority. Although many site specific problems arise, most are 126 loaded with performance decisions that transcend the peculiarities of each site. One example could be 4 and 5 storey wood frame buildings on sloping sites. It is suggested that information and criteria be pooled and catalogued so that decisions are reached within a broader framework and thus are of a higher and more consistent standard. There will continue to be anomalies but many will contain attributes that require attention. Reference and compilation will also assist in the code development process. 6)Many of the issues that arise are due to process type concerns, such as the types of information to be contained on drawings. What can be done to get at this? We discussed the two major areas of access to information. One is the publishing side which gives updated parameters and conditions for project submission. The interactive capability allows the publishing side to be augmented with relevant questions and current data. (eg. what is the latest zoning requirement for certain types of buildings). Conclusions The next event in the project is a seminar, offering a pilot session of the interactive technology. Presenters will, again be from the service provider, integrator and software sides. This will be supplemented with presentations from an architect, Building Standards and a municipality regarding the use of the technology. The session will be held in mid-February for building officials, architects and some municipal computer personnel. The conferencing participants will be identified at this session and with the conferencing beginning in early March and concluding in early April. The project will be seeking funding from the Industrial Research Assistance Program, in the early part of January, and hopes to deliver: 1) On-line code course on part 3 of the building code (completion November 1995) 2) Code database for use by architects. The first project will establish a mechanism for gathering and collating code issues. Building permit processing, building envelopes and performance codes will be catalogued and structured. 3) Object oriented access (icon) established through Netscape and Cyberstore to disseminate and generate pertinent information to the architectural profession in the areas of products, design, code data, and news. Your support and continued involvement is appreciated. Please call if you have any questions or comments regarding the project. orient3/jg/dec95 Appendix J - Pre and P o s t t e s t Comparisons pretest posttest j grcup change • 1 • .86 . .77 2.00 -.09 2 : .55 , .72 j 2.00 .17 3 • .76 ! .70 j 2.00 -.06 4 .76 : .64 2.00 -.08 5 .76 .70 2.00 -.06 6 : .75 1 .67 2.00 -.08 7 .77 . .63 1.00 -.14 8 ; .40 : .58 1.00 .18 9 j .53 ; .45 1.00 -.08 10 • .76 .65 1.00 -.11 11 •65 .57 1.00 -.08 12 . .68 : .57 1.00 -.09 13 .70; .66 1.00 . -.04 14 .90 1.00 .75 ' .57 1.00 -.18 16 .68 .67 1.00 -.01 17 ; .78 ' .84 j 1.00 .06 t - t e s t s for Independent Samples of GROUP Number Variable of Cases Mean SD SE of Mean CHANGE GROUP 1 10 -.0490 .105 .033 GROUP 2 6 -.0333 .100 .041 Mean Difference = -.0157 Levene's Test for Equality of Variances: F= .075 P= .788 t- t e s t f o r Equality of Means 95% Variances t-value df 2-Tail Sig SE of D i f f CI for D i f f Equal -.29 14 .773 .053 (-.130, .099) Unequal -.30 11.05 .772 .053 (-.132, .100) 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0054901/manifest

Comment

Related Items