UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Teacher acceptance of the instructional environment scales (Ties) MacLeod, W. B. 1994

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_1994-0192.pdf [ 3.42MB ]
Metadata
JSON: 831-1.0087358.json
JSON-LD: 831-1.0087358-ld.json
RDF/XML (Pretty): 831-1.0087358-rdf.xml
RDF/JSON: 831-1.0087358-rdf.json
Turtle: 831-1.0087358-turtle.txt
N-Triples: 831-1.0087358-rdf-ntriples.txt
Original Record: 831-1.0087358-source.json
Full Text
831-1.0087358-fulltext.txt
Citation
831-1.0087358.ris

Full Text

TEACHER ACCEPTANCE OF THE INSTRUCTIONAL ENVIRONMENT SCALE (TIES) by W.B. MACLEOD, B.A., York University, 1969 M.Ed., Brock University, 1991 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS in THE FACULTY OF GRADUATE STUDIES Department of Educational Psychology and Special Education We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA March, 1994 c ) Copyright: W.B. MacLeod, 1994 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. (Signature) Department of i ^ ^ f e > w t / / \ T - / C £ * 6yy r ^^e.^ej=^c /^o£^c The University of British Columbia Vancouver, Canada Date ^f^^/Y,/?^ DE-6 (2/88) 11 ABSTRACT The Instructional Environment Scale (TIES), a measure designed to assist professionals in systematic analysis of a student's classroom environment, has not been readily accepted by classroom teachers. A review was conducted of the theory and the research upon which TIES was based and the research concerning factors influencing the acceptance of interventions by classroom teachers. From the latter research, it was hypothesized here that improved knowledge about TIES would enhance acceptance on the part of classroom teachers. Two experiments were conducted. In both experiments, subjects were presented with one of two descriptions of TIES. One of these was described as the low information condition, while the other was designated the high information condition. In the first experiment, the information was presented on videotape while in the second a lecture format was used. Information gained from the presentation was measured by a 20 item test of knowledge of TIES devised for the study. Acceptance of TIES was evaluated using a measure adapted from the Behaviour Intervention Rating Scale (BIRS: Von Brock and Elliott, 1987). Coefficients of reliability were found to be .64 and .65 for the knowledge measure and .91 for the acceptability measure. Subjects were student teachers and experienced teachers enrolled in summer session courses in educational psychology and special education. Demographic variables such as teaching experience, special education background, and level of students taught had no significant influence on the results. The null hypothesis that there would be no difference in acceptability ratings of TIES when teachers were given more information about the instrument could not be rejected in either experiment. Subject characteristics and methodological difficulties were discussed as limitations to the generalization of the results. iii Table of Contents Abstract ii List of Tables v Acknowledgement vi I. Introduction 1 A: Statement of the Problem 2 B: Purposes of the Study 2 C: Importance of the Study 3 II. Literature Review 4 A: Theoretical and Research Base of TIES 4 1. The 12 Components of TIES 7 i: Instructional Presentation 7 ii : Classroom Environment 9 iii : Teacher Expectations 11 iv: Cognitive Emphasis 12 v: Motivational Strategies 13 vi : Relevant Practice 14 vii : Academic Engaged Time 15 viii: Informed Feedback 16 ix: Adaptive Instruction 17 x: Progress Evaluation 18 xi : Instructional Planning 18 xii : Student Understanding 19 2 . Technical Adequacy of TIES 20 B: Factors Influencing Acceptability 21 1. Methodological Developments 24 2. Selected Research on Acceptability 25 iv 3. Eight Influences on Acceptability 29 C: Summary 32 III. Methodology 33 A: Nature of the Study 33 B: Instrumentation 34 C: Design of the Study 35 D: Subjects 3 6 E: Procedure 39 F: Data Processing and Analysis 39 G: Assumptions 39 IV. Results 41 A: Information Scores and Acceptability Ratings 41 B: Test Reliability 47 i . Information Measure 47 ii. Acceptability Measure 52 C: Written Comments by Participants 56 D: Discussion 58 V: Summary and Conclusions 64 References 67 Appendix A 73 Appendix B 81 Appendix C 83 V List of Tables Table 1: Subject Characteristics as a Percentage of the Total Number in Each Experiment 38 Table 2: Means and Standard Deviations for Information and Acceptability Results: Experiment 1 41 Table 3: Means and Standard Deviations for Information and Acceptability Results: Experiment 2 42 Table 4: Results for Information and Acceptability Measures by Subject Variables: Experiment 1 44 Table 5: Results for Information and Acceptability Measures by Subject Variables: Experiment 2 45 Table 6: Order Effects for Information and Acceptability Measures: Experiment 1 46 Table 7: Item Analysis Results on Information Measure: Experiment 1 48 Table 8: Results of Item Analysis Results for Acceptability Measure and Item Correlations with Total Test and Information Measure: Experiment 1 53 V I Ac knowledgement I would like to express my deep appreciation to a number of people who through their support, encouragement, advice, and assistance helped to make completion of this work possible. To Dr. W.T. McKee, who served as supervisor to this thesis, I wish to express profound thanks for his patience, support, encouragement, and assistance with all aspects of the study. My research committee members, Dr. Robert Conry and Dr. Perry Leslie, offered constuctive criticism and supportive comments on numerous occasions. I am truly grateful for their guidance. I also wish to express appreciation to those graduate students, sessional instructors, and faculty members who were cooperative in permitting access to their classes for recruitment of subjects and who allocated class time for this research to be conducted. Very special appreciation is accorded to Marilyn Ransby who expended much time and effort in assisting with the production of the videotape. Most of all, I wish to thank my family: Stephanie, Khione, Jesse, and Janine. Without their being willing to give of time, effort, and support, this work would not have been possible. 1 I: Introduction The Instructional Environment Scale (TIES) was developed by Ysseldyke and Christenson (1987a) to assist professionals in determining the degree to which a student's academic and behavior problems are a function of the instructional environment. In an effort to tie assessment to intervention in the classroom, TIES uses classroom observation combined with student and teacher interviews, recorded on the Data Record Form, to develop a systematic description of the student's interaction with the classroom environment in order to design instructional interventions for individual students. TIES enables users to make qualitative appraisals about what is happening instructionally for an individual student. Having gathered data by the three methods, the user combines the information on a qualitative rating scale (the Instructional Rating Form). A profile sheet is then completed which provides an inventory of the student's instructional environment. It is used to identify the student's needs and to suggest actions which might be taken in order to address those needs. Despite its intent and its potential as indicated in the literature (Kratochwill and Sheridan, 1990; Lloyd and Blandford, 1991), there has been resistance on the part of classroom teachers to the use of TIES as an assessment instrument (personal communication with J. Ysseldyke, January, 1992) . The reaction of teachers was anticipated by the authors when they included in the manual a section entitled "What TIES is not designed to do" (Ysseldyke & Christenson, 1987a, p. 4). They indicated that "TIES is not a teacher evaluation scale," stressing instead that the instrument was to be used to assess the instructional environment as it pertains to individual students. 2 The issue of treatment acceptability has been discussed at length in the literature (Elliott, Witt and Kratochwill, 1991; Algozzine, Ysseldyke, Christenson and Thurlow, 1983; Clark and Elliott, 1988; McKee, 1984). It appears that some of the same factors which affect the acceptability of proposed treatments on the part of teachers may influence the degree to which teachers will be amenable to the use of assessment instruments such as TIES in their classrooms; namely, knowledge of the nature and purpose of the instrument and the time commitment expected on the part of the teacher. A: Statement of the Problem TIES is an instrument which appears to have potential in helping to design classroom interventions for academic and behavior problems. It has a sound theoretical and research base, but there is a problem of acceptability on the part of classroom teachers. This may be due to limited understanding of TIES by teachers, the procedures for its use, and the rationale upon which it is based. It is possible that with increased understanding there will be a corresponding increase in acceptability. B: Purposes of the Study 1. To evaluate the acceptability of TIES on the part of classroom teachers. 2. To test the hypothesis that increased knowledge of TIES and its purpose will enhance teacher acceptability. 3. To determine whether there are other factors which might be affecting the acceptance of TIES by classroom teachers. Those to be examined in this study are amount of classroom experience, level of student being taught (eg. 3 elementary/secondary), and amount of special education coursework or experience. C: Importance of the Study Sattler (1990) identified "four pillars of assessment: norm-referenced tests, interviews, observations, and informal assessment" (p. 3). TIES, in incorporating two of these pillars, interviews and observation, seems ideally suited to examining the instructional environment of students. Because of the trend to mainstreaming of special needs students, it appears that classroom teachers and those responsible for assisting them in designing and implementing interventions in the classroom need tools to identify those aspects of the classroom environment which can be adapted to meet the needs of exceptional pupils. Because the classroom teacher is more than ever the key player in this process, it is important that any proposed assessment which involves classroom observation and involvement of the teacher in an interview format be acceptable to the teacher. As part of the consultation process, school psychologists, resource teachers, counsellors and other support personnel need to be aware that the success of assessment and intervention are dependent upon the full acceptance of the proposed assessment on the part of the classroom teacher. Exploration of those factors which impact upon teacher acceptability of instruments such as TIES should be of assistance in designing more effective assessments and interventions to the benefit of all engaged in the process, especially the students whose success in school hinges upon effective classroom strategies. 4 II. Literature Review Four aspects of this study will be the focus of this chapter. Attention will be given to the theoretical and the research base of TIES, beginning with the rationale for assessment of the classroom environment in the first place, continuing with an examination of the research base supporting each of the 12 components of the scale, and concluding with reliability and validity issues related to the development of the scale. Reference will also be made to research in which TIES has been used. Consideration will be given to the issue of social validity as it relates to the problem identified in chapter one of this paper. The second section of the literature review will be concerned with a significant body of information dealing with the issue of teacher acceptability. These works have tended to focus upon the acceptability of classroom interventions, but those aspects which pertain to this study will be discussed. Thirdly, some attention will be given to the notion of teacher expertise and knowledge of interventions and methodologies as they impact upon the acceptability and the success of interventions. Finally, consideration will be given to some methodological issues which impact upon this study. A: Theoretical and Research Base of TIES In establishing a theoretical base for TIES, Ysseldyke and Christenson (1987a) stressed that student performance in school is a function of interaction between the student and the environment. They cited authors such as Bandura (1978) and Anastasi (1976) in support of their position. Bandura stated that behavior is determined by interaction among behavioral, cognitive, and environmental influences. Anastasi believed that individual behaviour 5 resulted from a combination of individual characteristics and situational variables. In another article, Ysseldyke and Christenson (1987b) contended that there was general agreement that instructional interventions based on norm-referenced measures of student traits were of limited effectiveness. They cited evidence demonstrating that special educators had been unable to identify interactions between student aptitude and prescriptive teaching with the result that the emphasis on learner characteristics had resulted in little progress for special needs students. This view was expanded later by Ysseldyke and Marston (1990) who took an interactive view of exceptionality and advocated that three perspectives; process dysfunction, experiential defect, and experiential deficit be considered in assessment and instructional planning. This emphasis on the relationship between personal and environmental variables was reinforced by McKee & Witt (1990) who argued that handicapped children need to be provided a supportive environment which would enable them to make the transition to normal or mainstream settings. Assessing the needs and skills of the child represented only part of the equation as environmental demands had to be consistent with the ability of an individual to function within that environment. McKee & Witt also pointed out that there are social and political issues involved when school psychologists attempt to advise teachers regarding instructional variables in the classroom. Viewing the problem as residing solely within the child was seen as the major barrier to effective intervention in the instructional environment. The assumptions underlying assessment of the classroom environment were discussed by Lentz and Shapiro (1986). They reasoned that effective remediation involved managing the interaction between the referred student and the classroom environment. They stated that a major purpose of psychological assessment should be that of helping to alter classroom variables with 6 classification of students taking place only after the demonstrated failure of classroom interventions. Their second assumption was that students might be referred for a variety of reasons such as inadequate instruction in basic skills or failure to perform skills which the student has, in fact, mastered -both of which may have to do with the student's interaction with the classroom environment. Their final assumption was that, regardless of setting, successful remedial attempts involved direct instruction of deficit skills or manipulation of the classroom environment. They concluded that a structured environmental assessment using teacher interviews, direct observation, and inspection of students' work was crucial in order to create an effective connection between assessment and intervention. There has been extensive research dealing with factors influencing the behavioural, cognitive, and affective outcomes experienced by students in classrooms. Walberg (1984) synthesized the outcomes of over 3000 studies in arriving at the conclusion that there were three major influences on student learning: aptitude, classroom instruction, and school environment. Other authors (eg. McKee & Witt (1990), Bickel & Bickel (1986), Samuels (1986), and Lentz & Shapiro (1986)) have compiled more detailed lists of factors which could be divided among these three categories. In developing TIES, Ysseldyke & Christenson (1987a) refined a list of about 200 descriptive statements to arrive at 12 components which they felt best identified the essential aspects of an effective classroom environment: instructional presentation, classroom environment, teacher expectations, cognitive emphasis, motivational strategies, relevant practice, academic engaged time, informed feedback, adaptive instruction, progress evaluation, instructional planning, and student understanding. They used three criteria for the selection of these elements; (a) they were repeatedly mentioned in the 7 literature as being important for improving academic success, (b) they were easily observable, and (c) research evidence supported their inclusion as principles of effective instruction. These criteria are cited in the manual as confirming the content validity of TIES. Attention will now be given to some of the research supporting these components. 1. The 12 Components of TIES i: Instructional Presentation Instructional presentation includes factors related to lesson development, clarity of directions and checking for student understanding. Among the authors cited by Ysseldyke and Christenson in support of the instructional presentation component were Carroll, 1963; Good, 1983; Rosenshine and Stevens, 1985; Anderson, Evertson and Brophy, 1979; Good and Grouws, 1979; Hunter and Russell, 1981; and Englert, 1984a. Carroll (1963) proposed a model of school learning comprised of five elements; aptitude, ability, perseverance, opportunty to learn, and quality of instruction. It is this latter element which Ysseldyke and Christenson attempt to address in the TIES component called instructional presentation. Carroll stated that quality of instruction was the "most elusive quality" (p. 729} in his model. He hypothesized that variations in the pacing of instruction might influence student understanding and perseverance. Good & Grouws (1979) constructed a study in which they investigated the effectiveness of an experimental math programme for grade four students. They developed their programme by repeatedly observing relatively effective and ineffective math teachers as measured by gain scores on math achievement tests in a naturalistic study and by incorporating variables found to be important in other studies of mathematics instruction. 8 Teachers in the experimental group received training in the programme and they were given a manual to assist in implementation. The programme stressed such factors as daily review, lesson development, controlled practice, and checking student progress during seatwork, all of which are components of instructional presentation as defined in the TIES manual. Students in the experimental group obtained significantly higher gain scores on two measures when compared with peers whose teachers continued to follow their usual approach. Anderson, Evertson and Brophy (1979) utilized linear regression to examine the relationship of 55 teaching variables with the gain scores in reading demonstrated by grade one students. Although they found that the implementation of a number of their treatment components by teachers in the experimental classes was less than they desired, they did find that TIES related factors such as increased opportunity to respond and guided practice were significantly related to student achievement. Englert (1984a) examined instructional pacing and found that student achievement improved when teachers maintained a brisk pace in presenting new material, had higher student accuracy in responding, and prompted students following incorrect answers. Her sample consisted of student teachers in a special education programme who were rated as either high or low in effectiveness and who served as tutors with special education students during practice teaching. In a chapter examining the components of effective teaching, Rosenshine and Stevens (1985) expressed pleasure with their finding that independent researchers have arrived at similar conclusions with respect to components of effective instruction. Their summary of findings closely matches the characteristics of effective instructional presentation as described in the TIES manual. More recently, Englert, Tarrant, and Mariage (1992) reviewed the literature on effective teaching. They reported on numerous studies which have continued to be 9 supportive of the elements of instructional presentation as described by Ysseldyke and Christenson. Effective teachers were described as follows; they introduce and orient their students to content, they model and demonstrate concepts, they check for understanding, they maintain a brisk pace and a high success rate, and they monitor student progress. ii: Classroom Environment The second TIES component, classroom environment, addresses management issues such as behavioural expectations, instructional routines, and classroom atmosphere. Among those cited in support of this component in the TIES manual are Good (1983), Doyle (1985), Anderson et. al. (1979), Brophy, (1983), Englert (1984a), and Emmer, Evertson & Anderson, (1980). Good (1983) and Brophy (1983) stressed that effective classroom managers used techniques which prevent misbehaviour by fostering student cooperation and involvement. Doyle (1985) discussed the nature of the classroom environment. He said that classroom settings have distinctive features which affect students regardless of the teacher's approach or organization. He concluded that the key to a teacher's success depended upon the ability to anticipate events in the classroom and upon having the skills to monitor and guide student activities. In their study of grade one reading classes, Anderson et. al. (1979) concluded that, "Good classroom management underlies all other principles and makes it possible to implement them in the classroom" (p.222). Their results indicated that effective teachers were particularly adept at managing transition times and in correcting behaviour. Englert (1984a) stressed that students must be taught rules and routines, and that these must be demonstrated, rehearsed, and monitored in a manner similar to the reaching of any skill. These notions found support in research conducted by Emmer, Evertson, & Anderson (1980) . 10 They examined correlates of effective classroom management used by grade three teachers at the beginning of the year. They found that effective classroom managers established themselves as leaders who early in the year worked on rules and procedures until the children learned them. They focussed on helping children adjust to the classroom system and adjusted to the needs and concerns of individual students. Thus, there appears to be general agreement that classroom management is an integral part of effective instruction (Jones and Jones, 1986; Emmer, 1987; Safran, Safran and Barcikowski, 1988), and there is evidence that individual differences influence student response to various management strategies. Burstein (1986) examined the effects of classroom organization on mainstreamed preschool children. She found that regardless of setting, handicapped children spent less time on task, interacted more frequently with adults, and associated with peers less frequently than did nonhandicapped children. In an extensive discussion of instructional issues related to adolescents with learning problems, Schumaker, Deshler, and McKnight (1991) pointed to three major factors which appear to be related to the problems of these students: individual disabilities or deficiencies, curricular and setting factors in the secondary school, and inappropriate teaching practices. They speculated that the high dropout rate among at risk students was due to an environment in secondary schools which was hostile to these students. Thus, it appears that students with learning and behaviour problems experience the school environment and respond to it differently throughout their school careers. Those aspects of the TIES classroom environment component which deal with the teaching of instructional routines and establishing a positive atmosphere in the classroom seem particularly suited to these students. 11 iii: Teacher Expectations The influence of teacher expectations on student achievement appears to be well documented but controversial with respect to the nature of its effects (Rosenthal & Jacobson, 1968; Brophy & Good, 1986; Good & Brophy, 1987; Myles & Ratzlaff, 1988). In discussing this, the third component of TIES, Yssseldyke and Christenson stressed that expectations must be clearly communicated to the student. They cited work by Wong, Wong, & LeMare (1982), Weinstein (1983), Good and Brophy (1984), and Anderson (1985). Wong et. al. conducted two experiments with fifth, sixth and seventh grade learning disabled and normally achieving students. Using a factorial design, they found that all children given explicit instruction about a criterion task in reading comprehension performed significantly better in comprehension and recall than controls. Their results suggested that teachers who clearly communicate specific objectives of assignments are more likely to foster task-appropriate learning activities and enhanced performance on the part of students. Weinstein (1983) contended that students vary in their ability to process classroom information and that teachers must engage in appropriate levels of communication. Good and Brophy (1984) indicated that in order to maintain appropriate attitudes and expectations toward students, teachers needed to monitor present progress and needs of students as well as their own biases and expectations. Anderson (1985) reported on a study in which grade one reading students in eight classes were observed and interviewed about their seatwork. It was found that students were more concerned with task completion than with the purpose of the task. She indicated that low achieving students in particular were more concerned with appearing to work hard and with completing assignments rather than with developing an understanding of material. More recently, Deno & Espin (1991) cited extensive evidence to show that teachers who set clear academic 12 performance goals and who collected periodic data on student progress were able to effect greater achievement. iv: Cognitive Emphasis The fourth TIES component, cognitive emphasis, was developed out of research which demonstrated that students who know how to think to solve problems achieve at a higher level in school. Winne (1985) referred to this process as cognitive achievement. In support of this component, Ysseldyke and Christenson cited authors such as Winne & Marx (1982), Weinstein & Mayer (1985), and Winne (1985). Winne & Marx (1982) videotaped 10 classroom lessons from each of five classrooms, grades four, five, and seven. They also interviewed the students and the teachers. Data were analysed through three stages out of which three major categories emerged: orienting or controlling the goals to which students worked, cognitive processing or the ways in which students think during instruction in order to achieve intended results, and consolidating or promoting storage and retrievability of content. There was marked lack of correspondence between the instructional goals of the teachers and the resulting cognitive processes used by students, and both students and teachers distinguished between teaching acts which were preparatory and supportive of learning as opposed to those which were intended to direct learning. When students had mastered a cognitive response, they were more likely to recognize the teacher's intention. This suggests that the teaching of cognitive processes might be beneficial. In a later paper, Winne (1985) proposed 15 steps which could be used to train students in cognitive processing during instruction. An alternative approach which looked at the types of cognitive approaches needed for various kinds of learning tasks was presented by Weinstein and Mayer (1985) . They explored techniques to enhance 13 learning in both basic and more complex school tasks. When TIES was being developed, this was seen by the authors as a relatively new area of research, but they were attracted to the notion that a student's thinking processes mediate between teacher input and student achievement. v: Motivational Strategies The fifth component of TIES refers to the enthusiasm with which lessons are presented, the levels of variety and interest, orientation of the teacher toward intrinsic or extrinsic rewards, and the student sense of competence. Among those cited in the TIES manual, Wittrock (1985) defined motivation as "the process of initiating, sustaining, and directing activity" (p.304) . Brophy (1983) pointed out that motivation can be both a general trait where students see learning as valuable and take pride in acquiring knowledge and skills, and it can be viewed as situation specific in that individuals vary their efforts depending upon their perception of the value of the learning task relative to the amount of effort they will have to exert and their chances of success. He stressed that attention should be devoted more to developing intrinsic motivation in students so that they will focus more on learning than performance. In discussing research on locus of control and learned helplessness, Wittrock concluded that both intrinsic and extrinsic motivation need to be present in order for successful learning to take place. He stated that in order for children to develop and maintain motivation, they have to learn that their efforts will be rewarded with success. A note of caution was indicated in a study by Dweck (1975) who, in studying learned helplessness, found that children in her "success only" condition were unable to constructively handle subsequent failure, while those who were 14 allowed occasional failure in an "attributional retraining" condition responded to failure by putting forth more effort. Thus, in their operational definition of motivational strategies, Ysseldyke and Christenson stressed the importance of carefully monitoring student progress and rates of success so that tasks might be presented at instructional levels appropriate to the student. vi: Relevant Practice The notion of the appropriateness of tasks is an important element of the sixth component of TIES, relevant practice. There is also an important relationship with the notion of appropriate success rates as mentioned above. Ysseldyke and Christenson state that two types of practice, controlled, or guided, and independent seatwork, need to be present to enhance student achievement. They cite authors such as Durkin (1979), Samuels (1981), Good (1983), Marliave and Filby (1985) and Rosenshine and Stevens (1985). Durkin (1979), after structured observation of reading comprehension in both reading and social studies classes, found that students spent most of their time in these classes at "busy work", completing work sheets that in many cases drove the presentation of brief lessons that focussed on giving the students the information they needed to complete the sheets. Several days often elapsed from the time children read a passage and the session in which the teacher questioned them about it so that it was impossible to assess the true purpose of the questionning. Good (1983), in his study of mathematics instruction, found that those teachers whose students made greater gains provided appropriate practice activities and seemed to be looking for evidence that their instructions had been understood by students. Students of less effective 15 teachers spent more time at seatwork but did not have a good understanding of what they were doing or why they were doing it. Samuels (1981) observed that in a junior high remedial reading class, many of the students did not understand the language of instruction; terms such as, "paragraph," "sentence" and, even, "word," were unfamiliar to them. He stressed that teachers need to make sure that students understand the language of instructions and the conventions of print such as the uses of upper and lower case letters and punctuation marks. In their model of effective instruction, Rosenshine and Stevens (1985) provided a detailed guide to presentation, guided practice, and independent practice. They cited evidence to show that experienced teachers could be taught these skills quite easily and that the students of these teachers had higher achievement scores and engagement rates. Marliave and Filby (1985) demonstrated that there is a relationship between relevant practice, high success, and time on task. They indicated that it was particularly important that children experience success in the early grades and that low achievers continue to require a high degree of success during instruction. They found that repeated practice on instructional objectives in reading led to gains in fluency and accuracy in oral reading. vii: Academic Engaged Time In defining the seventh component, academic engaged time, Ysseldyke and Christenson stressed three aspects of the instructional environment. They stated that students must, firstly, have the opportunity to engage in academic work. Then, the student must become involved in responding to questions and participating in academic activities. Lastly, student attention to a learning task must be maintained. They cited Good (1983) who stressed that research 16 results demonstrate that achievement is related to both the quality of instruction and the time spent on academic tasks. After a comprehensive review of the literature, Anderson (1984) concluded that academic engaged time seemed to be influenced by other components of effective instruction; selecting tasks of appropriate difficulty, communicating to students what it is that they have to learn and how they are to demonstrate their learning, arranging activities so that there is a high degree of continuity and monitoring of student progress, reinforcing appropriate task oriented activities on the part of students, providing feedback to students about their academic progress on specific tasks and goals, and correcting errors and misunderstandings as they present themselves. Similar approaches were recommended by Englert (1984a) who stressed the monitoring of seatwork to ensure a high degree of accuracy in student responding. She also stressed that the primary purpose of seatwork should be that of consolidation of previously learned skills in order to produce automaticity. Practice of new tasks should be supervised so that errors can be readily corrected. By attending to academic engaged time, the individual conducting classroom observation may clearly obtain cues related to other components of TIES. viii: Informed Feedback As noted above, informed feedback appears to be an essential component of effective instruction. Bloom (1985) noted that the need for what he called preseverance on the part of students could be reduced if students were provided with appropriate resources accompanied by frequent feedback as to their progress and specific instruction and explanation where they encountered difficulty. In his discussion of feedback and correctives, Rosenshine (1983) 17 described four types of student responses and cited research to suggest the appropriate teacher response. His basic conclusion was that errors should always be corrected. He emphasized the detection and reteaching of student errors early in the instructional sequence so that errors would not become habitual. ix: Adaptive Instruction The ninth TIES component deals with the kinds of adjustments that can be made to accommodate handicapped students within the mainstream classroom. The major work cited in the TIES manual in support of this component is that of Waxman, Wang, Anderson, and Walberg (1985a, 1985b) along with Wang (1984,1985). Waxman et. al. conducted a meta-analysis of 38 studies involving adaptive instruction. They found an average effect size of .45 with effect sizes of .39 for cognitive outcomes, .69 for behavioural outcomes, and .60 for affective results. Wang (1984,1985) described research involving the "adaptive learning environments model" (ALEM). ALEM matched student needs and learning experiences based upon awareness of each student's learning characteristics, past performannce, present level of competence, and the nature of the tasks to be performed. A primary emphasis was the fostering of students' responsibility in making choices in the planning and evaluation of their own learning. The programme was implemented in 138 kindergarten to grade three classrooms with generally positive results in attitude and achievement when compared with controls in mainstream-withdrawal settings. 18 x: Progress Evaluation A key component of ALEM was reported to be continuous monitoring of student progress. The tenth TIES component, progress evaluation, was developed out of the work of Bloom (1985), Deno and Mirkin (1980), and Gettinger (1984a, 1984b). Bloom based his views on the work of Carroll (1963) who believed that aptitude for learning is a function of the amount of time taken to master a learning task. Bloom held that given enough time, effort, and help, 95% of students could attain mastery in a subject. Gettinger conducted two studies which were designed to test Carroll's notion that degree of learning was a function of time spent on learning. Mastery of a task was compared by giving one group of children a fixed number of trials to learn a task, while another group was allowed to self regulate in that they could repeat trials until they felt that they had mastered the task. The children who were allowed an unlimited number of trials scored significantly higher on all measures. Gettinger stressed that continuing analysis of individual differences in learning rate was necessary to determine those points in the learning process where children might need additional practice or remediation. In order to facilitate progress evaluation, Deno and Mirkin developed a technique for graphing student progress in mastering an objective so that both the student and the teacher could be continually aware of progress. xi: Instructional Planning The eleventh TIES component is closely tied to the components of adaptive instruction and progress evaluation. As cognitive, affective, and motivational profiles of students become evident, and their needs for such considerations as added time or remedial assistance are revealed, the role of the teacher 19 becomes that of planning to accomodate these needs. Bloom (1985) and Good and Brophy (1984) advocated a mastery learning model in which instruction is individualized within classrooms so that students progress at varying rates through the curriculum. The difficulty of achieving this was highlighted in an observational study conducted by Bennett, Desforges, Cockburn, and Wilkinson (1984) . They found, for example, that teachers tended to make assessments of student progress by focussing on the product of a child's performance rather than the processes or strategies employed in developing the work, and they found numerous problems with the ways in which tasks were designed with there being particular confusion between practice and new work. They advocated that there be more sharing of information and more support for teachers attempting to implement a mastery learning model. TIES might help to fill this role. Ysseldyke and Christenson include two functions in instructional planning; instructional diagnosis and instructional prescription. Eleven factors are taken into consideration in the diagnostic phase, and and 20 factors are included in the prescriptive function. xii: Student Understanding The final component of TIES, student understanding, refers to the accuracy with which the student understands instructional goals and comprehends the directions and processes needed to complete an assignment. Good (1983) stated that students think about instruction in different ways and have varying interpretations of teacher behaviour. In their study of student cognitions about instruction in a naturalistic setting, Winne and Marx (1982), encountered numerous incidents of lack of correspondence between teachers' intentions and students' cognitive responses. The findings of Anderson (1985) 20 that grade one students did not understand the importance of learning a skill but believed that their task was to get work finished tended to support the notion that student understanding of the purposes of learning tasks is an important factor in effective instruction. Peterson and Swing (1982) videotaped grade five/six math classes and replayed the videos in segments for students, asking them a series of questions about their attention, understanding, and use of cognitive strategies. They found that students' self reports in these areas were better predictors of achievement than were observer ratings of time on task behaviour. The above findings serve to support the notion that a student interview is an essential feature of TIES, for without knowing the cognitive and affective responses of the student to instructional components, it is difficult to determine the effectiveness of the implementation of the teacher's intentions. This review of the 12 components of TIES sampled only a fraction of the works which could be cited from a broad range of literature. The underlying research and theoretical foundations appear to be consistent with the writings of authors such as Bloom and Good who were referenced in support of several components. 2. Technical Adequacy of TIES Ysseldyke and Christenson rested their case for the validity of TIES by concentration on content validity as exemplified by the extensive theoretical and research evidence which they cited in support of each of the 12 components. They argued that because TIES is not a norm referenced test but a qualitative scale, interrater reliability should be of primary concern. 21 Twenty-eight observers were asked to complete TIES observation records while watching tapes of each of two teachers instructing small groups of elementary school students. They were also asked to score interviews of teachers and students. Interrater reliabilities on each of the components ranged from .83 to .96 with coefficients for 10 of the 12 components being greater than .90. Initial content validity of TIES has been supported through the review of the literature to this point, but a thorough analysis of this issue has yet to be conducted. Interrater reliability appears to be adequate. Of concern in the present study, though, are the factors which prevent its acceptance by classroom teachers, factors which may be more consistent with the notion of social validity. B: Factors Influencing Acceptability The intent of this section is to examine the factors contributing to the acceptability of treatment strategies on the part of teachers. The information presented to this point gives an indication of the complexity of treatment implementation. Prior to any intervention being proposed, the classroom teacher may have many reasons for being either favourably or unfavourably predisposed to the acceptance and implementation of an intervention. In the body of this review, consideration is given to the theoretical framework, the measurement devices that have been developed, and factors that have been shown in the research literature as having impact upon the notion of treatment acceptability. A recent report by Schumm & Vaughn (1992) highlighted some of the issues presented by general classroom teachers with respect to the mainstreaming of special education students. Seven hundred and seventy-five elementary, middle, 22 and high school teachers were surveyed to determine their perceptions and feelings about planning for mainstreamed students as well as their planning practices. It was found that the teachers were willing to have mainstreamed students in their classes as long as the students did not exhibit emotional or behavioural problems. They were willing to make some adaptations to tests or assignments, but were less likely to adapt curriculum or create new or revised objectives. Elementary teachers were more likely to make these kinds of adaptations than were middle or high school teachers. They saw planning for mainstreamed students as being hampered by budgetary and accountability factors, access to equipment and materials, the physical environment in the classroom and the school, class size, lack of teacher preparation, problems with emotionally handicapped students, and limited instructional time. All of these factors can be seen as precursors of teacher acceptability of interventions in the classroom. Gersten, Walker, & Darch (1988) found that teachers who were more competent to deal with special needs students as measured by supervisors using the Teacher Effectiveness Evaluation Form (TEEF) were also less likely to accept the placement of such students in their classes, less likely to tolerate maladaptive behaviour in their classes, and more likely to expect higher standards of achievement and behaviour in their classes. However, these same teachers indicated "a greater willingness to receive technical assistance in dealing with the behaviour and learning deficits they identified as problematic" (p. 437). For the consultant, this presents an interesting challenge. Teachers who are most likely to be successful in working with mainstreamed students are least likely to accept the students in the first place, but, upon accepting the students, these teachers are more likely to request assistance from consultants. Hawryluk & Smallwood (1986) indicated 23 that consultants must consider the teacher variables of knowledge, skills, cognition, and affect, before beginning the consultation process with a teacher. Authors such as Elliott (1988) and McKee (1984) have pointed to the concept of social validity introduced by Wolf (1978) as providing a framework for the concept of treatment acceptability. Wolf proposed that teachers, parents, and children evaluate treatments in terms of "the social significance of goals . . . the social appropriateness of procedures ... (and) the social importance of effects" (p. 207). Kazdin (1981) defined treatment acceptability as follows: "judgments by laypersons, clients, and others of whether treatment procedures are appropriate, fair, and reasonable for the problem or client" (p.493) . Elliott, Witt, & Kratochwill (1991) indicated that many treatments which appear to be effective are never used in real life settings or that they may be abandoned after very brief use. They suggest that these things happen because "many installed interventions simply don't fit into the routines and environments of help givers (i.e., teachers)" (p.102). Models of treatment acceptability have been presented by Witt & Elliott (1985), and Reimers, Wacker, & Koeppl (1987). Witt & Elliott suggested that acceptability derives from the interrelationships among four factors: acceptability, use, integrity, and effectiveness. Reimers et al. added a fifth factor; knowledge of the treatment. Elliott, Witt, & Kratochwill (1991) suggested that these models do not encompass all of the factors impacting upon treatment acceptability, but that they have provided a useful framework for the development of research questions. It seems clear that the acceptability of treatments has become an area of concern over the past number of years. It has become increasingly apparent that wishes, attitudes, background knowledge, and experience of the consumer, 24 especially the teacher who is usually given the responsibility of implementing a treatment within the classroom, have to be considered in the development of any intervention plan. To accomplish these goals, a number of methodologies and measurement instruments have been developed. 1. Methodological Developments Because it was difficult to evaluate the acceptability of treatments that had been applied in classrooms due to the variations in treatment and the limited number of respondents available who had experienced each of these, Kazdin (1980a, 1980b) developed what came to be known as the analogue study. Subjects were presented with a hypothetical case description and were asked to rate the acceptability of possible treatments using the Treatment Evaluation Inventory (TEI: Kazdin, 1980a). This instrument was more suited for use with clinical populations. It was revised for use in school settings by Witt & Martens (1983) and renamed the Intervention Rating Profile (IRP) for which two forms were developed, the IRP-15 and the IRP-20, having 15 and 20 items respectively. The original IRP-20 contained five factors; general acceptability, risk of the intervention to the target child, amount of teacher time required, effect of the intervention on other children, and amount of teacher skill required. The IRP-15 was comprised of the 15 items that correlated most clearly with intervention acceptability (Elliott, 1988) . A more recent form of this instrument, the Behaviour Intervention Rating Scale (BIRS) was developed by Von Brock & Elliott (1987). It was printed in its entirety in Elliott, Witt, & Kratochwill (1991) and in Elliott & Von Brock Treuting (1991) . To the 15 items of the IRP-15 which dealt with treatment 25 acceptability were added another nine items which were designed to measure treatment effectiveness. Elliott & Von Brock Treuting (1991) reported on a validation study of the BIRS using the analogue technique. Their subjects were 216 teachers attending graduate courses. They were given one of two problem descriptions combined with one of three possible interventions and one of three possible paragraphs containing information about treatment effectiveness. Two primary factors were found; acceptability and effectiveness, as was a secondary factor; time of effectiveness. The authors suggested that the BIRS might be used by consultants in a variety of situations to provide information from consumers about possible treatment plans so that the possibility of selecting the most acceptable treatment might be enhanced or that adjustments might be made to proposed plans. To this point, it can be seen that a methodology has been developed for evaluating treatment acceptability, the analogue technique, and that rating scales have been developed for this purpose. Attention will now focus upon a number of studies which have been reported in recent years with the intent of identifying factors which influence acceptability of treatment. 2. Selected Research on Acceptability In a study in which the IRP-20 was used to evaluate experienced teacher acceptability ratings of positive and negative interventions, Elliott, Witt, Galvin, & Peterson (1984), found that teacher acceptability ratings were influenced by the following factors: severity of the problem behaviour, complexity of the proposed intervention (this included the time which would be required of the teacher in order to implement the treatment), and whether the 26 treatment involved contained positive or negative consequences for the student (positive consequences were more acceptable). Beginning with a screening sample of 107 elementary school teachers who completed a test measuring knowledge of behavioural principles, McKee (1984) selected two groups of 32 teachers, a high knowledge and a low knowledge group. All were given descriptions of the same case with background information about the child being omitted in 50% of the descriptions. All of the subjects evaluated four possible interventions: medication, time out from reinforcement, reinforcement of incompatible behaviour, and positive practice. Results indicated that teachers with greater knowledge of behavioural treatments rated all four of the proposed interventions more positively. Reinforcement of incompatible behaviour was rated as the most acceptable intervention, while medication was viewed as least acceptable. Added background information had no effect. Hall & Didier (1987) asked 73 student teachers to read two problem descriptions; one passive (refusal to complete assignments) and one active (talking out). Three interventions were proposed; behavioural, pragmatic, and humanistic, with each being rated using the IRP-15. Subjects were also asked to rate the severity of the behaviour in each case, to rate the three interventions in terms of frequency of utilization, and to indicate whether they thought the behaviours could be handled in the regular classroom or whether outside assistance might be necessary. The humanistic approach was rated as the most acceptable in this study, while the pragmatic approach was seen as least acceptable. The subjects also indicated that they tended to use interventions which they rated most highly, there being a high correlation between acceptability and frequency of use. The authors rightly pointed out that results may have been influenced by the theoretical orientation of the 27 training received by their subjects and that the results might not generalize to more experienced teachers. In a related study, Hall & Wahrman (1988) asked 72 experienced, regular education teachers to indicate how often they used pragmatic, humanistic, behavioural, or other approaches to classroom management. They then were asked to rate the same interventions and cases used in the Hall & Didier (1987) study. Grade level taught was found to have no effect. Frequency of use was found to be a factor only in the case of the behavioural intervention which was also the type most frequently used by the teachers in the study. The humanistic and behavioural approaches were found to have the highest acceptability rating as in the previous study, while the pragmatic approach was least acceptable. In a study examining the responses of children, teachers, and school psychologists to three forms of group contingencies, Elliott, Turco, & Gresham (1987) found that all three groups expressed a preference for group reinforcement plans in which reinforcement was based upon individual behaviour or the behaviour of the entire group. A plan in which a small group within the class determined the consequence for the entire group was seen as least acceptable. Of note was the result which showed that severity of the problem had no effect. Of interest in this study might have been an item analysis of the results on the IRP-15 and on the Children's Intervention Rating Profile (CIRP) to see if perceived fairness of the treatment acted as a significant variable. Three variables; effectiveness information, type of intervention, and problem severity were examined by Von Brock & Elliott (1987) . Subjects were 216 teachers attending graduate classes. The BIRS was used to measure effectiveness and acceptability of three proposed treatments, three types of 28 effectiveness information and two levels of severity of behaviour. Demographic variables were also considered, but no effects could be ascribed to sex, race, age, type of class taught, years of teaching experience, or academic degree. Two interventions, response cost and token economy, were rated as more acceptable than the third, time-out. In the case of the mild problem scenario, effectiveness information was a significant factor with research based information providing the strongest effect. When the problem presented was more severe in nature, effectiveness information had no effect. When subjects rated an intervention as less acceptable, they also rated it as less effective. The effect of education on acceptability was investigated by Tingstrom (1989) . Thirty-four experimental subjects (85% of whom were enrolled in a teacher education program) received lectures on general learning principles and their application as interventions in the classroom. Thirty-nine control subjects who represented the general undergraduate population did not receive the lectures. A pre and posttest design was used in that students were presented with a case scenario and asked to rate four possible interventions using the TEI. Pretesting showed no difference between the groups. Posttesting yielded no change in ratings by the control group while the experimental group's ratings of all four interventions increased. Although there were a number of weaknesses in the design of this study, it demonstrated that education or increased knowledge can enhance the acceptability of interventions. Although the studies summarized above represent only a portion of the acceptability research of recent years, it is possible to examine the degree to which these results support previous thinking. Reviews by Elliott (1988) and Elliott et. al. (1991) proposed a number of factors which should be 29 considered by consultants when proposing interventions to teachers. Attention will now be given to the manner in which the studies summarized here provide added information about some of these variables. 3. Eight Influences on Acceptability Jargon: Elliott et. al. (1991) cited research (Witt, Moe, Gutkin, & Andrews, 1984) which indicated that the label used to identify a treatment influenced the acceptability of that treatment. They found that an intervention labelled pragmatic was more acceptable than the same intervention labelled behavioural or humanistic. In the Hall & Wahrman (1988) study cited above, experienced teachers reported using behavioural approaches most often. This was confirmed in their rating the unlabelled behavioural intervention as most acceptable, a result similar to that found in Hall & Didier (1987). These studies suggest that teacher response to consultant language may be affected by teacher knowledge of interventions, the frequency with which they have used them, and the content and orientation of their training. Years of Experience: Witt et al. (1984) found that teachers tended to rate all interventions with decreasing acceptability as their experience increased. Von Brock & Elliott (1987), however, found no effect with respect to years of teaching experience, and although this information was collected in other studies (e.g., Elliott & Von Brock Treuting, 1991; McKee, 1984), results were not reported. This influence of this factor in determining the degree of teacher acceptability of interventions is unclear. 30 Knowledge of Behaviour Principles: Knowledge appears to be an important factor influencing teacher acceptability ratings. In addition to the results presented by McKee (1984) and Tingstrom (1989), Schumm & Vaughn (1992) found that teachers who were surveyed regarding their concerns about mainstreaming listed lack of teacher preparation as an important concern. Hall & Didier (1987) found that teachers rated behavioural intervention favourably and that it was the one that they reported using most frequently. It could be argued that knowledge was a component in this result as well (Reimers, Wacker, & Koeppl, 1987). For the consultant, the importance of this factor is positive; as teachers become more familiar with interventions in regular classes, their acceptability ratings should increase. In addition, the educational role of the consultant, practised effectively, could have a positive impact on acceptability. Type of Training: Hall & Didier (1987) stated that the theoretical orientation of the training received by their student teacher subjects might have influenced their rating a humanistic approach as the most acceptable. Consultants could do some questioning of teachers during initial interviews to gather information about background and training of individual teachers as this might provide some cues as to the kinds of intervention which the teacher might find acceptable. Class Management Techniques Used: The notion that interventions which teachers tended to use most frequently would also be viewed as most acceptable was confirmed (Hall & Didier, 1987; Hall & Wahrman, 1988) . Information about strategies that teachers have used in the past can be helpful to the consultant in designing interventions. 31 Time Required was viewed by teachers in the Schumm & Vaughn (1992) study as an important issue in their acceptance of mainstreamed students in the classroom. In their work with the BIRS, Von Brock & Elliott (1987) and Elliott & Von Brock Treuting (1991) found time of effectiveness to be a factor. Time required for implementation and time required for the treatment to become effective can impact upon acceptability. Type of Treatment was found to be a variable in most of the studies reported here. Behavioural interventions tended to be seen more positively than humanistic approaches. It appears that both ecological factors such as trends toward mainstreaming and teacher variables such as background knowledge and experience are important in determining the acceptability of various strategies. Reported Effectiveness was found by Von Brock & Elliott (1987) to be a significant factor when the problem was perceived as less severe with research based information having the strongest impact on acceptability. However, when the problem was perceived to be more severe, effectiveness ceased to be an issue. The paradoxical nature of this result indicates a need for more research. One hypothesis might be that as the severity of the problem increases and the teacher feels less competent in dealing with the problem, the desire for the consultant to serve in the role of expert increases in order that a quick solution might be provided and equilibrium restored. 32 C: Summary As an innovative way of assessing the classroom environment as it impacts upon students with behaviour and learning problems, TIES appears to hold potential for enabling classroom teachers to adapt instruction within mainstream classes so that these students might better realize their potential. Review of the 12 factors that comprise TIES supports the authors' contentions regarding content validity and interrater reliability. Because it uses three sources of information, combining the perspectives of an observer, the teacher and the student, TIES provides ratings derived from more sources than most assessment measures. It may be because it is a different approach to assessment that it is meeting resistance from teachers. Examination of the acceptability literature suggests that knowledge is a significant factor in the acceptance of treatment procedures on the part of teachers. This leads to the hypothesis that if teachers were better informed about a measure such as TIES and could see the relevance of its use in their classrooms, their level of acceptance might be enhanced. The analogue approach has been used in assessing the acceptability of various treatment methods for classroom use. In this study, it was proposed that it be utilized to test the hypothesis that an assessment measure, TIES, would become more acceptable to teachers if their knowledge about it was increased. The purposes of the present study, then, were to evaluate the acceptability of TIES by classroom teachers, to test the hypothesis that being given increased information about TIES would increase its acceptance by teachers, and to investigate the influence of three additional factors; amount of classroom experience, level of student being taught, and amount of special education background. 33 III. Methodology In order to address the research questions concerning teacher acceptability of TIES, the effect of knowledge on acceptability ratings and other factors which affect acceptability, two experiments were conducted using an analogue approach. In the first experiment, videotaped presentations were used to inform teachers of the essential elements of TIES. In the second experiment, the same information was presented in a lecture format aided by overheads and handouts. The chapter begins with an overview of the procedures followed in the study and continues with more detailed description of instrumentation and experimental design, the subjects of the study, data collection procedures, and an outline of the analysis of the data. A. Nature of the Study In order to address the research questions posed in this study it was necessary to determine a method by which teachers could be exposed to information about TIES in a controlled manner. Scripted, videotaped descriptions of the TIES were used in order to present teachers in two groups with either more or less information about the instrument. These were designated as high and low information groups. The degree of information accumulated from the video presentations was measured using a multiple choice test devised by the author. The instrument used to assess teacher acceptability was adapted from a measure developed by Von Brock and Elliott (1987). In the second experiment, subjects were presented the same information in a lecture format as indicated above and the same measures were used. 34 B. Instrumentation Two instruments were designed for use in this study. The first is a 20 item, multiple choice test of information about TIES (Appendix I). Each item had five possible resonses with the fifth response in each case being, "I don't know." Maximum possible score on the instrument was 20. The test was developed during a graduate course in test design and and its initial reliability of .83 was determined in a pilot study with 18 graduate students in the Faculty of Education serving as subjects. The second instrument (Appendix II) has been adapted from the Behavior Intervention Rating Scale (BIRS). Von Brock and Elliott (1987) developed the BIRS to evaluate the reaction of various consumer groups (teachers, parents, hospital staff etc.) to interventions or treatments which were presented as written vignettes. Elliott (1988) reported that the total BIRS yielded a reliability (coefficient alpha) of .97 while the three factors of acceptability, effectiveness, and time yeilded alphas of .97, .92, and .87 respectively. For the purposes of the present study, the name of the instrument has been changed to the Rating Instrument for the Instructional Environment Scale (RITIES). It consists of 27 items, as opposed to 24 items on the BIRS. Additional items (#'s 9, 16, & 23) and wording changes to other items deal with the specifying of academic and behavioral outcomes as projected in the goals of TIES and with the perceptions of teachers with respect to both time required for administration and the perceived threat of TIES being used as an instrument to evaluate teachers. Item scores on this measure could range from 1 (strongly agree) to 6 (strongly disagree). Total scores on the measure could range from 27 to 162. Eleven of the 27 items were negatively weighted to control for response bias. An item analysis of this instrument and the 35 multiple choice knowledge measure were conducted using the Laboratory of Educational Research Test Analysis Package (LERTAP) developed by Nelson (1974) . C. Design of the Study 1. Dependent variables The dependent measure in this study consisted of total acceptability scores obtained on the RITIES. 2. Independent Variables The use of two video presentations and, in the second experiment, two scripted lectures, one giving detailed information about TIES and the other giving a brief description of the instrument served as treatments. The second experiment was conducted as a check on the first experiment when it became apparent that a number of subjects were reacting negatively to the video presentation. In the second experiment, the lectures were accompanied by overheads and print handouts in order to maximize the opportunity for subjects to retain information. The level of information about TIES obtained by the two groups was assessed through the use of the multiple choice knowledge measure. In addition, the demographic variables, grade and subject taught, level of training, years of teaching experience and amount of special education background were examined as to their effect on the acceptability ratings. The administration of the questionnaires was counterbalanced in order to control for order effects. In null form, the hypothesis predicted that there would be no difference between the high knowledge and the low knowledge groups on acceptability of 36 TIES as measured by RITIES. In addition, the interactions of experience and knowledge and experience and acceptability were examined. D.Subjects Subjects for both experiments were recruited from summer session classes in the Department of Eucational Psychology and Special Education, Faculty of Education, University of British Columbia. All subjects had classroom teaching experience and were either completing their teacher education programme after having completed a practicum, or they were educators who were upgrading their qualifications. The majority of subjects, however, were student teachers with less than one year of experience. With permission of class instructors, a brief presentation regarding the nature of the study was given either at the beginning or the end of a class session. Those who gave voluntary consent were scheduled for the study. In all cases, the study procedures were conducted either at the beginning or the end of class time. A number of instructors allowed the study to be conducted during classes because they perceived that the information presented during sessions was relevant to course content. In both studies, non-participating students were not in the room while the study was being conducted. Groups were assigned alternately to either the high or low information condition. One hundred and sixty-five subjects participated in the first study; 83 received the low information presentation while 82 received the high information presentation. Fifty-two subjects participated in the second study with 20 receiving the low knowledge presentation and 32 receiving the high knowledge presentation. A summary of subject characteristics is presented in Table 1. Subjects were placed in categories as shown with the intention of exploring the effects of experience, level, and 37 special education background as categorical variables. Special education background was categorized in the following manner. The courses in which they were enrolled at the time of the study were not included in the rating. They were categorized as having none, minimal, medium, or extensive background. Those in the none category had no previous special education courses or special education experience. Those placed in the minimal category had completed one or two courses in special education or had one year or some classroom experience but no coursework. Those in the medium category had completed at least three courses in special education with no classroom experience or they had completed at least one course combined with classroom experience. Finally, the extensive category consisted of those who had completed at least three courses and had classroom experience in special education. The data presented in Table 1 indicate that the subjects who volunteered for the two experiments differed in some respects. While subjects with less than one year of experience formed the majority in both experiments, most of those participating in the first experiment were secondary teachers while most of those in the second experiment were primary level teachers. In terms of their special education background, the majority of subjects in both experiments had either minimal or no special education background. 38 Table 1: Subject Characteristics as a Percentage of the Total Number in Each Experiment Experiment 1 Experiment 2 N % N % Experience: <1 yr. 1-5 yr. >5 yr. 128 19 18 78 12 11 43 4 5 83 8 10 Level: primary intermediate senior 35 27 103 21 16 62 33 15 4 63 29 8 Special Education Background: none minimal medium extensive 69 48 30 18 42 29 18 11 16 22 10 4 32 42 19 8 Total N 165 52 39 Er Procedure In the experimental session, groups were shown the high and low information video presentations or, in the second experiment, were given the long or short lecture on an alternating basis. They were then asked to complete a brief demographic form and the information and acceptability measures. The latter were prepared and completed in alternating order such that the information instrument was completed first by 50% of the respondents while the other half completed the acceptability measure first. F. Data Processing and Analysis Analysis of the measurement instruments was conducted using LERTAP which yielded scores for each subject along with estimates of item and test reliability. Analysis of variance was conducted to test the hypothesized relationship between acceptance of TIES and the amount of information that subjects had about the instrument. The difference between the high and the low information groups with respect to their acceptance of TIES was tested to assess the difference between the two groups. In addition ANOVA's were conducted to determine the influence of demographic factors and order effects. F. Assumptions No effort was made to balance groups with respect to demographic characteristics. Although an attempt was made to examine the effects of these kinds of variables, it was assumed that they would account for a small portion of the variance. 40 It was assumed that the counter balancing of the order of presentation of the information measure and the accepability measure would control for order effects. Initial presentation of the acceptability measure was likely to interfere with recall when the information measure was administered subsequently (Winzer & Grigg, 1992, p. 373) It was assumed that the reliability estimates provided for the BIRS would transfer to a great extent to RITIES. 41 IV: Results In this section, findings bearing upon the central hypothesis were presented first. This was followed by examination of the influence of demographic variables and order effects. The reliability of the measurement instruments was then examined with attention being devoted to particular items on the information measure which appeared to be either easy or difficult and to those which received either high or low levels of endorsement on the acceptability measure. Finally, as an aide in interpreting the results, written comments by participants were reported. A: Information Scores and Acceptability Ratings Table 2 provides results of information scores and acceptability ratings for the first experiment. The score on information is out of a possible 20 points. Item scores on acceptability range from 1 (strongly disagree) to 6 (strongly agree). Acceptability scores could range from 27 to 162. Table 2 Means and Standard Deviations for Information and Acceptability Measures: Experiment 1 Information Acceptability N Mean S.D. F/p Mean S.D. F/p Low 83 7.21 2.97 111.78 13.13 High 82 7.97 4.58 1.62/.20 102.94 14.72 16.56/.0001 42 Although the high information group obtained higher mean scores than the low information group, the difference was not significant (E(l, 163) = 1.62, p_ =.20). The results on the acceptability measure were significant, E(l, 163) = 16.56, p. = .0001, but examination of the means revealed the difference to be in the opposite direction to that predicted with the result that the null hypothesis could not be rejected. The mean item scores on acceptability were 4.14 for the low information group and 3.81 for the high information group with four being the slightly agree point on the scale. Results for the second experiment, presented in Table 3, were similar. Table 3 Means and Standard Deviations for Information and Acceptability Measures: Experiment 2 Information Acceptability N Mean S.D. F/p Mean S.D. F/p Low 20 8.38 3.35 114.93 14.34 High 32 9.92 4.34 1.85/.18 113.38 14.64 .14/.71 The high information group again obtained higher mean scores than the low information group, but the difference, E(l, 50) = 1.85, £ = .18, was not significant. The mean acceptability rating was again in the direction opposite to that predicted with the difference being non-significant, E(l, 50) = .14, 43 £ = .71. The mean item ratings were 4.26 for the low information group and 4.20 for the high information group. Results were also examined by groupings according to years of experience, level taught, and amount of special education training and experience. As presented in Table 4 and Table 5 these results indicated no significant findings. One trend which appeared in both experiments had those with the most special education training and experience tending to provide the most positive ratings on the acceptability measure. Table 4 Results for Information and Acceptability Measures by Subject Variables: Experiment l Information Measure Acceptability Measure Number Mean SD F/p Group Mean SD F/p Item Mean Teaching Experience <1 yr. 1-5 yr. >5 yr. Level primary interm. senior 128 19 18 35 27 103 7.57 6.29 9.06 8.71 7.94 7.11 3 3 3 3 3. 4. .65 .85 .67 .51 .41 .02 2.42/.09 2.41/.09 106.76 106.03 113.33 110.19 105.7 106.88 13.7 19.3 14.7 16.8 15.5 13.6 1.72/.18 .88/.42 3.95 3.93 4.2 4.08 3.91 3.96 Special Ed. Training none 69 7.23 4.38 minimal 48 8.17 3.94 medium 3 0 7.11 2.88 extensive 18 8.22 2.79 .87/.46 104.5 108.03 109.55 113.14 14.8 12.2 2.12/.10 17 14.1 3.87 4 4.06 4.19 Note: Special Education Training was defined as follows: none: no special education courses or experience minimal: one or two courses in special education or some classroom experience medium: completion of at least three courses or completion of at least one course in special education combined with classroom experience extensive: completion of at least three special education courses with classroom experience Table 5 Results for Information and Acceptability Measures by Subject Variables: Experiment 2. Information Measure Acceptability Measure Number Mean SD F/p Group Mean SD F/p Item Mean Teaching Experience <1 yr. 43 9.62 4.12 113.2 15 4.19 1-5 yr. 4 7.5 2.7 .70/.50 120.75 9.25 .51/.60 4.47 >5 yr. 5 8.25 4.11 115.2 12.3 4.27 Level primary 33 9.39 4.38 110.59 12.9 4.1 interm. 15 9.42 3.56 .10/.90 119.4 16 2.69/.08 4.42 senior 4 8.44 3.29 121.5 15 4.5 Special Ed. Training none 16 9.53 4.85 115.84 17.8 4.29 minimal 22 9.38 3.42 112.95 12.9 4.18 .56/.65 .54/.66 medium 10 9.88 4.35 110.7 13.4 4.1 extensive 4 6.88 3.15 120.25 11.1 4.45 46 Results were also examined by groupings according to years of experience, level taught, and amount of special education training and experience. As presented in Table 4 and Table 5 these results indicated no significant findings. One trend which appeared in both experiments had those with the most special education training and experience tending to provide the most positive ratings on the acceptability measure. Order effects in experiment one were studied by examining the difference between mean scores when subjects completed the information and acceptability measures either first or second. These results are presented in Table 6. Table 6 Order Effects for Information and Acceptability Measures: Experiment 1 Information Acceptability N Mean S.D. F/p N Mean S.D. Item Mean F/p First 84 8.24 3.86 81 104.91 15.76 3.89 Second 81 6.90 3.79 5.10/.03 84 109.60 13.16 4.06 4.29/.04 The results presented in Table 6 indicate that respondents tended to obtain higher scores on the information measure when it was presented first. This difference, F (1, 163) =5.10 was significant at the .03 level. For the acceptability measure, the results (F (1, 163) = 4.29) were significant at the .04 level in the opposite direction with acceptability ratings tending higher when the measure was presented after the information instrument. 47 B: Test Reliability: i. Information Measure The Hoyt Estimate of Reliability for the information instrument constructed by the author was .64 (SEM =1.90) in the first experiment and .65 (SEM = 1.86) in the second experiment. The range of scores was 0 - 19 in the first experiment and 4 - 18 in the second. In an effort to obtain some insight into the results, item analysis was conducted on the data from the first experiment. These are presented in Tables 7 and 8. An examination of those items which yielded the highest and lowest percentages correct on the information and acceptability measures was conducted in order to assess the quality of those particular items. The following information items were answered correctly by less than 25% of the respondents. Percentage of respondents selecting each option is indicated in brackets. Keyed correct responses are highlighted. 8. The purpose of the teacher interview is to (a) provide the teacher with a list of recommended changes which should be made in the instructional environment. (19%) (b) provide an opportunity for the teacher to give more detail about the student's background and classroom performance. (36%) (c) provide feedback to the teacher as a result of classroom observation. (19%) A Table 7 Item Analysis Results on Information Measure: Experiment 1 Item Responses (N) percent point-bis biserial acceptab Item A B C D E other correct correl correl correl 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 9 6 80+ 11 15 67 + 18 31 34 + 29 2 107 + 9 44 41 2 126+ 39 + 13 15 104 + 44 10 8 62 + 22 35 59 7 3 32 21 26 + 31 29 3 13 42 1 1 0 4 1 96 + 2 19 97 + 31 50 115 + 39 5 89 65 + 2 132 + 7 13 110 + 0 36 97 + 55 16 10 37 5 19 + 38 9 82 + 18 10 6 51 + 3 3 21 14 139 + 16 14 19 33 76 18 10 25 36 8 7 14 31 17 42 24 15 49 26 10 1 1 1 2 1 3 1 1 1 2 4 1 1 3 1 2 2 2 2 1 62.7 58.4 48.2 57.8 37.3 40.4 58.4 11.4 20.5 69.3 49.4 64.5 15.7 39.2 30.7 79.5 75.9 23.5 66.3 83.7 0.27 0.28 0.39 0.49 0.25 0.38 0.54 0.22 0.36 0.33 0.29 0.4 0.36 0.39 0.23 0.38 0.33 0.45 0.44 0.52 0.34 0.36 0.49 0.62 0.32 0.49 0.69 0.36 0.52 0.44 0.36 0.52 0.54 0.5 0.3 0.54 0.45 0.62 0.57 0.77 0.26 0.26 0.09 0.23 -0.17 0.02 0.26 -0.01 0.1 0.12 0.2 0.19 0 0.21 -0.04 0.15 0.28 -0.02 0.2 0.27 + : the keyed correct response point bis correl : point biserial correlation acceptab correl: acceptability correlation, the point biserial correlation of a correct answer with the score on the acceptability measure 49 (d) provide information in those areas where observation is inappropriate or information is incomplete. (11% - correct response) (e) I don't know. (15%) 9. Instructional presentation refers to (a) the manner in which lessons develop in the classroom. (21% - correct response) (b) the enthusiasm and tone of the teacher's instruction of the student. (4%) (c) the use of various modalities in presenting the lesson. (30%) (d) the atmosphere and tone created by the classroom teacher. (23%) (e) I don't know. (22%) 13. Adaptive instruction emphasizes (a) student responsibility for task completion. (5%) (b) the individualization of instruction for handicapped students. (16% - correct response) (c) the commitment of the classroom teacher to diversity within the classroom. (54%) (d) the preparation of regular classroom students for the integration of special needs students. (6%) (e) I don't know. (19%) 18. It is recommended that classroom observations using TIES occur (a) during basic skills instruction. (24% - correct 50 response) (b) when a variety of activities are taking place. (25%) (c) early in the school year. (8%) (d) when the student is most likely to be experiencing difficulty. (13%) (e) I don't know. (30%) Examination of the data related to these items in Table 7 suggested that they were too difficult for most subjects. A check was made to ensure that the correct answer was coded for these items. This proved to be the case. Because the test was designed to be used with teachers and other professionals in education and would not likely be used with another group, it was assumed that the point biserial correlation should be used to select items of medium difficulty for those subjects (Crocker and Algina, (1986), p. 320). In the case of item eight, the point biserial correlation of .22 is well below the moderate range of .40 - .60, indicating that those who answered this item correctly did not necessarily score highly on the whole test. All of the distractors were effective in attracting responses. Items 9 and 13, however, with point biserial correlations of .36, come closer to the moderate range. Item 13 contained a particularly strong distractor item which was selected by 54% of the respondents. Item 18, while being answered correctly by only 24% of the respondents, correlated with the total test .45, indicating that those who answered it correctly stood a moderate chance of scoring well on the whole test. Those items which were correctly answered by more than 75% of the respondents were as follows. 16. A cognitive emphasis in teaching involves (a) a focus upon writing skills. (1%) 51 (b) clear identification of labels and definitions. (2%) (c) asking students to explain their answers and processes used to solve problems. (80% - correct response) (d) frequent drill and repetition of facts. (2%) (e) I don't know. (15%) 17. For feedback to be effective (a) it must be task specific and explicit. (76% - correct response) (b) there must be frequent praise of effort. (8%) (c) there has to be a high degree of student success. (4%) (d) it must take the form of tangible rewards. (2%) (e) I don't know. (9%) 20. Of the three sources of information used in TIES, the most important is (a) classroom observation. (9%) (b) the student interview. (1%) (c) the teacher interview. (0%) (d) a balanced view of all three. (84% - correct response) (e) I don't know. (6%) All of these items appeared to have weak distractors which made choice of the correct response easy for education students. Because these items did not correlate strongly with the total score, a correct response would tend to be spurious in that it would bear weak relationship to the total information score of the individual involved. Items 16 and 17 with point biserial correlations of .38 and .33 did not correlate strongly with the total test while item 20 did show a strongly moderate correlation of .52. 52 Item responses on the information measure did not correlate highly with the total scores obtained by individuals on the acceptability measure. Items 17 and 20 (see above) correlated most highly at .28 and .27 respectively, indicating that no one item or group of items on the information measure correlated strongly with total scores on the acceptability measure. ii. Acceptability Measure The Hoyt Estimate of Reliability for the acceptability measure (RITIES) was .91 (SEM = 4.27) in the first experiment and .91 (SEM = 4.14) in the second experiment. The range of scores was 42 - 144 in the first experiment and 74.5 - 149 in the second experiment. Item analysis of results on the acceptability measure are provided in Table 8. The mean scores for items fell in the range of 3.3 to 4.42 on a six point scale where 3 = slightly agree and 4 = slightly agree. Those which fell closest to the slightly disagree point on the scale were as follows. 19. Application of TIES would improve a child's academic performance to the point that it would not noticeably deviate from that of classmates. (3.3) 21. Soon after implementing the results of TIES, the teacher would notice a positive change in problem behaviors or in academic performance. (3.39) 22. A child's performance will remain at an improved level long after the results of TIES are implemented. (3.34) 23. Using TIES requires excessive time on the part of the teacher. (3.54 - weighting reversed) Table Result Total Item 1 2 3* 4 5* 6 7 8* 9* 10 11* 12 13 14* 15 16 17* 18 19 20* 8 :s of Test 1 5 4 5 2 29 3 2 16 17 1 7 1 2 15 2 2 6 4 6 2 Item and 2 5 6 47 15 60 9 7 57 56 7 54 7 8 60 4 4 64 10 27 25 Analysis for Acceotabilitv Measure Information Item 3 9 7 61 24 38 20 17 60 60 17 60 17 16 54 11 9 55 40 49 70 Jfeasure: Responses 4 65 74 33 80 14 69 53 22 17 63 24 66 62 22 64 66 22 76 64 43 5 75 63 15 33 12 55 71 6 10 62 12 66 68 9 68 68 13 25 8 12 (N) 6 5 9 2 4 3 8 14 3 1 10 1 6 6 3 12 13 1 1 1 2 _Experiment 1 other mean/SD 1 2 2 7 9 1 1 1 4 5 7 2 3 2 4 3 4 9 10 11 4.31/.97 4.3/.96 3.9/1.04 3.86/.96 4.4/1.21 4.14/1.01 4.37/1.01 4.28/1.05 4.29/1.04 4.28/.94 4.08/1.0 4.26/.9 4.25/.95 4.24/1.09 4.39/.91 4.42/.91 4.14/1.02 3.7/.9 3.3/.94 3.7/.89 and Item < Item-Total r 0.4 0.51 0.51 0.6 0.33 0.61 0.78 0.58 0.49 0.71 0.67 0.79 0.73 0.72 0.71 0.7 0.63 0.55 0.35 0.58 Continued . Correlations with Information r 0.1 0.04 0.21 0.04 0.1 0.01 0.19 0.16 0.13 0.28 0.25 0.16 0.17 0.15 0.28 0.13 0.13 0.02 -0.17 0.1 Table 8 (continued) Results of Item analysis of Acceptability Measure and Item Correlations with Total Test and Information Measure: Experiment 1 Item Responses (N) Item-Total Information Item 1 2 3 4 5 6 other mean/SD 21 22 23* 24 25 26* 27* 5 8 6 4 5 3 1 19 23 24 11 16 15 26 57 50 51 32 44 79 68 65 61 50 67 74 37 35 11 13 22 39 16 17 20 1 1 5 3 1 4 4 7 9 7 9 9 10 11 3.39/.9 3.34/.99 3.54/1.1 3.85/.99 3.53/.92 3.59/.94 3.61/.98 0.41 0.45 0.15 0.65 0.34 0.43 0.6 -0.04 -0.08 0.16 0.08 0.07 0.11 0.08 * : weighting reversed weighting: in all cases weighting was from "strongly disagree (1) to "strongly agree" (6) 55 25. When comparing a child who has academic or behavior problems with a child who performs well, the performance of the two children would be more alike after TIES had been utilized. (3.53) 26. The use of TIES would not produce enough improvement in a child's performance so that the performance is no longer a problem. (3.59 - weighting reversed) Five of these items required prediction of future success as a result of the application of TIES. In order to be endorsed more strongly, respondents would need to be very convinced of the efficacy of the measure. The mean score of 3.54 on item 23 indicated that respondents were not convinced that the time required to implement TIES would be beneficial. Those items most strongly endorsed were the following. 1. This would be an acceptable way of assessing a child's academic problem. (4.31) 2. This would be an acceptable way of assessing a child's problem behavior. (4.3) 5. There is not one child in my class whose problem is severe enough to warrent the use of this instrument. (4.4 -weighting reversed) 7. I would be willing to have TIES used in my classroom. (4.37) 15. Overall, the use of TIES should be beneficial for the child. (4.39) 16. Overall, the use of TIES should be beneficial for the teacher. (4.42) 56 Generally these items contain moderate language and allow a degree of agreement without the necessity for expressing stong endorsement. Items with relatively low correlations with the total measure were 5 (.33), 19 (.35), 23 (.15), and 25 (.34). Item 5 was among those items which received a relatively high endorsement from respondents, while the others were among those which were generally rated lower (see above). Since many of the respondents were student teachers with limited experience, item five may have proved confusing due to their limited exposure to classroom teaching. Items 19 and 25 required very stong endorsements so that even those who felt favourably disposed to TIES might have provided a negative rating on these items. The low correlation of item 23 to the total test suggested that even those respondents who would generally be positively disposed were not convinced that use of TIES was worth their time as teachers. There appeared to be no strong correlation between responses to any single item on the acceptability measure with individual total scores on the information measure as the intercorrelation was found to be .19. C: Written Comments by Participants A number of respondents produced unsolicited written comments. Some of these were as follows. "Sorry, but I don't feel that I am familiar enough with TIES to respond to these questions." (i.e., the acceptability measure) "The video caught me while my mind was on something else - toward the beginning of it, anyway." "I do not feel qualified to answer most of these questions without more in depth knowledge of TIES." 57 "... the technical jargon used in this assessment strategy turns me off from its onset." "I didn't get too much from the video." "I don't think the video presentation provided a very positive introduction to this rating instrument. The monotone, uninteresting voice and background noise was distracting, the presentation format (reading a list of characteristics and examples) was not stimulating at all; the information given was repetitive; the visuals were limited and poor ..." "I would strongly recommend administering TIES questionnaires at a time when students have not just been sitting for 3 1/2 hours on a friday afternoon. The tape was too long, dry, and difficult to follow ... It is ridiculous to expect an informed reaction to TIES upon watching that very poor, low grade video ..." "I would highly recommend a better video presentation if any information is to be retained - I didn't take anything in. (I'm not talking about quality). I was really a willing subject going into this, but it would be unfair of me to answer the questions based on what I got out of the video!" "I cannot rate this instrument since I did not take in the information on the video. All the ideas you seem to present are not being used in the case of the video ..." "I have absolutely no idea!! I found the video extremely boring and hard to pay attention to." "I really don't even know what TIES is about. The video was very difficult to pay attention to because it used the talking head approach. It is friday, last class and I tuned out - sorry." 58 "I don't feel I had enough information about how it would be used to be able to answer these questions. I think I have a clear understanding of what it is, but not how its used ..." In summary, the null hypothesis could not be rejected in either experiment. The demographic variables of years of experience, level taught, and amount of special education background did not influence scores on either the information or the acceptability measures. The Hoyt Estimate of Reliability for the information measure was .64. Item analysis revealed that a number of questions may have been too difficult (i.e. those answered correctly by less than 25% of the subjects) while others may have been too easy (i.e. those answered correctly by more than 25% of the subjects. Item responses on the information measure did not correlate highly with total scores obtained by individuals on the acceptability measure. The reliability estimate for the acceptability measure was .91. Correlations between individual item responses on the acceptability measure and total scores on the information measure were generally low. Complaints about the video dominated unsolicited written responses provided by subjects in the study. D: Discussion In this portion of the paper, attention will first be given to factors which emerged during the study which may have had a bearing on the results obtained followed by discussion of additional issues which would need to be considered in continued research in the area of teacher acceptability of assessment approaches. As originally proposed, the present study was comprised of one experiment, but it soon became evident that the video presentation used in the original 59 model was eliciting a negative response from subjects, particularly in the high knowledge condition. This information came from subject comments following experimental sessions and from unsolicited comments written on answer sheets, especially those used for recording responses to the acceptability measure. Complaints involved poor production values and the sense that too much information was included in the presentation. Some complained that there was too much background noise, others that it would be impossible to retain that much information on such brief exposure. As a result, it was decided that a second experiment would be conducted in which the videotape would be replaced with a scripted lecture accompanied by overheads and handouts. Results showed that although both knowledge scores and acceptability ratings tended to be higher in the second experiment, the null hypothesis again could not be rejected. In an attempt to explain the direction of the results, there are other possible factors relevant to the design of the study. A recent paper by Aldrich and Martens (1993) utilized an analogue approach to explore the responses of teachers to two approaches to consultation, one based upon behavioural approaches, and the other based on TIES. Several aspects of their approach may be contrasted with that taken in the present study. The first of these involved the analogue design. In the present study, subjects were asked to picture in their minds a student who had displayed learning or behaviour problems in their classrooms. This was done with the knowledge that participants in the study would have had experience with a variety of grade levels and subject areas. Aldrich and Martens began their presentation to all subjects with a videotaped presentation of a grade two girl exhibiting a variety of problem behaviours both in a small reading group 60 and during independent seatwork. They followed this with a rating scale instrument which permitted subjects to rate the extent to which the portrayal was realistic. Subjects were then asked to read one of two analyses of the situation presented in the video. One of these was generated by using the TIES protocol to assess the classroom situation shown in the video. As a result, only those TIES components which applied to the situation were utilized and only to the extent that they pertained to the video. These aspects of the Aldrich and Martens design suggest that the present study might have been improved through the presentation of a common stimulus problem with some measure of its apparent relevance to the subjects in the study. Also, by using only those components of TIES which pertained to the problem situation, Aldrich and Martens effectively limited the amount of information that subjects had to absorb. As noted above, some subjects in the present study complained that they were asked to absorb too much information in too short a time. Although Aldrich and Martens do not report the total time that subjects took to participate in their study, it would have been considerably longer than the approximately 3 0 minutes taken by subjects in the present study as the classroom problem videotape was 23 minutes in length followed by reading of text and completion of a series of measurement instruments. Thus, the present study may not have achieved the hypothesized result due to failure to follow the analogue design model more closely, by presenting stimulus information in too condensed a form in too little time, and by failing to monitor more closely the degree to which the stimulus to which subjects were being asked to respond was realistic or relevant to their experience. Previous use of videotaped vignettes in acceptability research has also used classroom vignettes to test the relative perceptions of the labels behavior modification or humanistic education on the part of classroom 61 teachers {Woolfolk & Woolfolk, 1979; Woolfolk, Woolfolk, & Wilson, 1977) . It appears that videotape may be better used to present classroom situations or enactments rather than more didactic, knowledge based material. Stanovich and Cunningham (1993), in a study of the relationships among print exposure, television exposure, ability, and knowledge acquisition, found that print exposure accounted for 37.1% of the variance in knowledge composite scores while the relationship between television exposure and knowlege measures tended to be negative. This would suggest that the use of a videotape presentation to convey information, as was done in the present study, was not likely to be successful. Comparison of the results on the information measure for experiments one and two indicated that more information was retained in both the high and low information conditions in experiment two. Caution must be exercised in interpreting this trend as the subject populations in the two groups were somewhat different with the majority of teachers in the first experiment having a secondary school background while those in the first experiment tended to deal with primary aged children. The tendency for teachers with more special education training and experience to be more accepting might indicate that the concepts incorporated in TIES have more relevance to those teachers than they do to those with relatively little background and experience in special education. The findings of Gersten, Walker, & Darch (1988) that teachers who were most competent to deal with special needs children were most willing to receive assistance in dealing with learning and behavior problems would support this finding. Future research might highlight this as a variable to be controlled and investigated more thoroughly. It was clear that the counter balancing of administration of the information and acceptability measures was necessary as there were significant 62 order effects. These calculations were conducted for experiment one only. The scores on the information instrument declined when it was administered in the second position, suggesting that completing the acceptability measure first may have interfered with the retention of information from the video tape. That acceptability scores were higher when that instrument was administered in the second position might have been partly due to knowlege being reinforced by having the information measure administered first. There was a large difference between the reliability estimates of the two measures utilized in the present study. The acceptability measure retained the relability demonstrated in a pilot study with graduate students, while the reliability of the knowledge instrument declined substantially. This may have been due to the relative homogeneity of the sample in the pilot group who as graduate students in educational psychology and special education would be expected to bring to the task prior knowledge which would make their responses to the instrument differ from those of classroom teachers. The pilot group also received their information in a different format. True variability may have been reflected in this sample in that they were compliant with all study procedures and experienced a relatively distraction free information presentation. They were encouraged to follow along as the experimenter read from a draft version of the video script which was collected before they completed the knowledge and information measures. As indicated above, this may have been a distinct advantage as information communicated through print media may be more readily retained. In the present study, the scores on the information test were higher in the second experiment, adding to the evidence that the video used in experiment one may have been ineffective in conveying information. The number of items on the information measure that proved 63 difficult for respondents in experiment one provided additional evidence that they had difficulty in retaining information from the video. The written comments by participants provided additional evidence in the analysis of the null results of the present study. In almost all cases they complained about the quality of the video and their inability to retain the information presented. They also attributed their lack of response to fatigue, especially members of one group who participated late on a Friday afternoon. One participant complained of the amount of jargon in the presentation, echoing the findings of Witt et. al. (1984), who found that interventions described in pragmatic terms were rated as more acceptable than those same interventions described in either humanistic or behavioral terms. It was likely that in an effort to compress information about TIES into a brief script, the author of the present study paid too little attention to this factor. Although the null hypothesis that there would be no difference between high and low information groups in their acceptability ratings of TIES, it should be noted that knowledge of the instrument may yet be shown to be a factor in acceptability ratings. Limitations of the present study prevent one from commenting upon the the findings of authors in studies of treatment acceptability (McKee, 1984; Tingstrom, 1988; and Schumm and Vaughn, 1992) . These may yet be shown to apply to the acceptability of TIES as an assessment measure. 64 V: Summary and Conclusions The main hypothesis of the present study was that increased knowledge of TIES on the part of classroom teachers would enhance their acceptability ratings of the scale. This arose out of literature which indicated that TIES had potential as a measure which would enable school psychologists and others to better design classroom interventions for learning and behaviour problems. Personal communication with the author indicated that the scale was not being used extensively due to resistance on the part of classroom teachers. A review of research on treatment acceptability and resistance on the part of teachers indicated that teacher knowledge had been found to be a factor which influenced the degree of acceptability of an intervention. A common method of evaluating the acceptability of interventions has been through use of what is known as an analogue study. The present study was designed with the intention of applying the analogue method to investigate teacher acceptance of an assessment measure. Scripts were developed which explained TIES, one which provided basic information and a second which provided more detail. For the first experiment, these were videotaped. Two measurement instruments were also developed, one to assess the information derived from the video and the other to assess the acceptability of TIES. Subjects were students taking summer session courses in the Department of Educational Psychology and Special Education at the University of British Columbia. In the first experiment, they were shown either the long or the short version of the video in groups after which they completed a brief demographic information form and the knowledge and acceptability measures. The procedure was similar in the second experiment 65 except that the short and long presentations were given in lecture form accompanied by overheads and handouts. In both experiments, no significant difference was found between the high and low knowlege groups. In examining the methodology and the results, it became clear that a primary reason for the failure to obtain significant results resided in the methodology adopted for the study. Previous analogue studies made use of either print materials or videotaped vignettes to establish a scenario in which a proposed treatment would be applied. In the present study, videotape was used to present detailed information about TIES. It seemed evident that this was not effective as a means of informing subjects in experiment one. In addition, the relatively poor quality of the video production proved to be an irritant to a number of respondents. It seems that video should not be used in a study unless the financial and technical resources are available to enable high quality production. As shown in the study by Aldrich and Martens (1993), it is not necessary to give complete details of the treatment in order for subjects to feel that they have adequate information to make a judgement. Among the demographic variables considered in the present study, degree of teacher training and expertise in special education appeared to hold the most promise as a variable which might be related to the acceptability of TIES. Future research which would define this variable more specifically and utilize a more heterogeneous sample might find this to be a significant variable. It would likely be more productive to design a study in which groups of teachers would be more closely matched on variables such as expertise, experience, and level taught. The low reliability found for the information measure may have been related to the difficulty of a number of the items, the difference between the 66 subjects in the piloting of the instrument and those in the two experiments, and the inefficiency of the videotape as a method of conveying knowledge. It is evident that when devising an instrument, the subject population which is used to establish reliability should resemble the proposed experimental pool as closely as possible. In addition, the knowledge conveyed to subjects in both cases should be equivalent in both content and form of presentation. It became clear from unsolicited written reponses provided by subjects that not enough attention was devoted to the timing of the study with respect to subject fatigue. One group of subjects in particular complained of their inability to respond appropriately after having sat through a long class on a friday afternoon. Another subject complained of the extensive use of jargon in the video presentation, another factor which had been noted in previous reseach but had not been given sufficient attention in the present study. In conclusion, there appear to have been a number of possible reasons for the failure to find significant results in the present study. Generally, closer attention to methods used in previous research, more care in the selection of subjects for the analysis of proposed measurement devices, and greater sensitivity to the language and medium of presentation of information might yield some interpretable results. 67 REFERENCES Aldrich, S.F., & Martens, B.K. (1993). The effects of behavioral problem analysis versus instructional environment information on teachers' perceptions. School Psychology Quarterly,8,110-124. Algozzine, B., Ysseldyke, J.E., Christenson, S., & Thurlow, M. (1983). Teachers' intervention choices for children exhibiting different behaviors in school. Minneapolis: University of Minnesota, Institute for Research on Learning Disabilities. Anastasi, A. (1976). Psychological Testing (4th ed.). New York: MacMillan. Anderson, L.M. (1985) . What are students doing when they do all that seatwork? In C.W. Fisher & D.C. Berliner (Eds.), Perspectives on instructional time (pp. 189-202) . New York: Longman. Anderson, L.M., Evertson, CM., & Brophy, J.E. (1979). An experimental study of effective teaching in first-grade reading groups. The Elementary School Journal, 79, 193-222. Anderson, L.W. (1984). Instruction and time on task: A review. In L.W. Anderson (Ed.), Time and school learning, (pp. 143-163). London: Croom Helm. Bennett, N., Desforges, C , Cockburn, A., & Wilkinson, B. (1984). The quality of pupil learning experiences. London: Erlbaum. Bandura, A. (1978). The self system in reciprocal determinism. American Psychologist, 33, 344-351. Bickel, W.E. & Bickel, D.P. (1986). Effective schools, classrooms, and instruction: Implications for special education. Exceptional Children, 52, 489-500. Bloom, B.S. (1985) . Learning for mastery. In C.W. Fisher & D.C. Berliner (Eds.), Perspectives on instructional time. (pp. 73-93). New York: Longman. Brophy, J. (1983). Conceptualizing student motivation. Educational Psychologist, 18, 200-215. Brophy, J.E. & Good, T.L. (1986) . Teacher behavior and student achievement. In N.C. Wittrock (Ed.), Handbook of research in teaching (pp. 328-375). New York: Macmillan. Burstein, N.D. (1986) . The effects of classroom organization on mainstreamed preschool children. Exceptional Children, 52, 425-434. Carroll, J.B. (1963). A model of school learning. Teachers College Record, 64, 723-733. Clark, L., & Elliott, S.N. (1988). The influence of treatment strength information of knowledgeable teachers' pretreatment evaluations of social skills training methods. Professional School Psychology, 3, 241-251. Crocker, L. & Algina, J. (1986) . Introduction to classical and modern test theory. Orlando, FL: Holt, Rinehart & Winston. 68 Deno, S.L. & Espin, C.A. (1991). Evaluation strategies for preventing and remediating basic skills deficits. In G. Stoner, M.R. Shinn, & H.M. Walker (Eds.), Interventions for achievement and behavior problems. Silver Spring, MD: The National Association of School Psychologists. Deno, S.L., & Mirkin, P.K. (1980). Data based IEP development: An approach to substantive compliance. Teaching Exceptional Children, 12, 92-99. Doyle, W. (1985). Classroom organization and management. In M.C.Wittrock (Ed.), Improving teaching: 1988 ASCD yearbook. Alexandria, VA: Association for Supervision and Curriculum Development. Durkin, D. (1979) . What classroom observation reveals about reading comprehension instruction. Reading Research Quarterly, 14, 481-533. Dweck, C.S. (1975) . The role of expectations and attributions in the alleviation of learned helplessness. Journal of Personality and Social Psychology, 31, 674-685. Elliott, S.N. (1988) . Acceptability of behavioral treatments in behavioral settings. In J.C. Witt, S.N. Elliott, & F.M. Gresham (Eds.), Handbook of behavior therapy in education (pp. 121-150). New York: Plenum. Elliott, S.N., Turco, T.L., & Gresham, F.M. (1987). Consumers' and clients' pretreatment acceptability ratings of classroom-based group contingencies. Journal of School Psychology, 25, 145-154. Elliott, S.N. & Von Brock Treuting, M. (1991). The behavior intervention rating scale: Development and validation of a pretreatment acceptability and effectveness measure. Journal of School Psychology, 29, 43-51. Elliott, S.N., Witt, J.C, Galvin, G.A., & Peterson, R. (1984). Acceptability of positive and reductive behavioral interventions: Factors that influence teacher decisions. Journal of School Psychology, 22, 353-360. Elliott, S.N., Witt, J.C, & Kratochwill, T.R. (1991). Selecting, implementing, and evaluating classroom interventions. In G. stoner, M.R. Shinn, & H.M. Walker (Eds.) Interventions for achievement and behavior problems (pp. 99-135) . Silver Spring, MD: National Association of school Psychologists. Emmer, E.T. (1987). Classroom management and discipline. In V. Richardson-Koehler (Ed.), Educator's handbook: A research perspective. New York: Longman. Emmer, E.T., Evertson, CM., & Anderson, L.M. (1980). Effective classroom management. Elementary School Journal, 80, 219-231. Englert, C.S. (1984a). Effective direct instruction practices in special education settings. Remedial and Special Education, 5, 38-47. Englert, C.S. (1984b). Measuring teacher effectiveness from the teacher's point of view. Focus on Exceptional Children, 17, 1-14. Englert, C.S., Tarrant, K.L., & Mariage, T.V. (1992). Defining and redefining instructional practice in special education: Perspectives on good teaching. Teacher Education and Special Education, 15, 62-86. 69 Gersten, R., Walker, H., & Darch, C. (1988). Relationship between teachers' effectiveness and their tolerance for handicapped students. Exceptional Children, 54, 433-438. Gettinger, M. (1984a). Achievement as a function of time spent in learning and time needed for learning. American Educational Research Journal, 21, 617-628. Gettinger, M. (1984b). Measuring time needed for learning to predict learning outcomes. Exceptional Children, 51, 244-248. Good, T.L. (1983). Classroom research: A decade of progress. Educational Psychologist, 18, 127-144. Good, T.L. & Brophy, J.E. (1984). Looking in classrooms (3rd. ed.). New York: Harper and Row. Good, T.L. & Brophy, J.E. (1987). Looking in classrooms (4th. ed.). New York: Harper and Row. Good, T.L., & Grouws, D.A. (1979). The Missouri mathematics effectiveness project. Journal of Educational Psychology, 71, 355-362. Hall, C.W., & Didier, E. (1987). Acceptability and utilization of frequently used intervention strategies. Psychology in the Schools, 24, 153-161. Hall, C.W., & Wahrman, E. (1988). Theoretical orientations and perceived acceptability of intervention strategies applied to acting-out behavior. Journal of School Psychology, 26, 195-198. Hawryluk, M.K., & Smallwood, D.L. (1986). Assessing and addressing consultee variables in school-based behavioral consultation. School Psychology Review, 15, 519-528. Hunter, M. & Russell, D. (1981) . Planning for effective instruction: Lesson design. In Increasing your teaching effectiveness (pp. 23-41) . Palo Alto: Learning Institiute. Jones, C. & Jones, L. (1986). Comprehensive classroom management. Newton, MA: Allyn and Bacon. Kazdin, A.E. (1980a). Acceptability of alternate treatments for deviant child behavior. Journal of Applied Behavior Analysis, 13, 259-273. Kazdin, A.E. (1980b). Acceptability of time-out from reinforcement procedures for deviant child behavior. Behavior Therapy, 11, 329-344. Kazdin, A.E. (1981). Acceptability of child treatment techniques: The influence of treatment efficacy and adverse side effects. Behavior Therapy, 12, 493-506. Kratochwill, T.R. & Sheridan, S.M. (1990). Advances in behavioral assessment. In T.B. Gutkin & C.R. Reynolds (Eds.), The handbook of school psychology: Second edition (pp. 328-364). New York: Wiley. Lentz, F.E. & Shapiro, E.S. (1986). Functional assessment of the academic environment. School Psychology Review, 15, 346-357. Lloyd, J.W. & Blandford, B.J. (1991). Assessment for instruction. In H.L. Swanson (Ed.) Handbook on the assessment of learning disabilities: Theory, research, and practice. Austin, Pro-ed. 70 Marliave, R. & Filby, N.N. (1985). Success rate: A measure of task appropriateness. In C.W. Fisher & D.C. Berliner (Eds.), Perspectives on instructional time. (pp. 217-235) . New York: Longman. McKee, W.T. (1984). Acceptability of alternative classroom treatment strategies and factors affecting teachers' ratings. Unpublished master's thesis, University of British Columbia, Vancouver. McKee, W.T. & Witt, J.C. (1990) . Effective teaching: A review of instructional and environmental variables. In T.B. Gutkin & C.R. Reynolds (Eds.), The handbook of school psychology: second edition. New York: Wiley. Myles, D.W., & Ratzlaff, H.C. (1988). Teachers' bias towards visible ethnic minority groups in special education referrals. B.C. Journal of Special Education, 12. Nelson, L.R. (1974). Laboratory of educational research test analysis package (Computer program). Dunedin, New Zealand: Education Department, University of Otago. Peterson, P.L., & Swing, S.R. (1982). Beyond time on task: Students' reports of their thought processes during classroom instruction. Elementary School Journal, 82, 481-491. Reimers, T.M., Wacker, D.P., & Koeppl, G. (1987). Acceptability of behavioral treatments: a review of the literature. School Psychology Review, 16, 212-227. Rosenshine, B. (1983). Teaching functions in instructional programs. The Elementary School Journal, 83, 335-351. Rosenshine, B., & Stevens, R. (1985). Teaching functions. In M.C. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 376-391). New York: Macmillan. Rosenthal, R., & Jacobson, L. (1968). Pygmalion in the classroom. New York: Holt, Rinehart and Winston. Safran, P., Safran, J.S., & Barcikowski, R.S. (1988). Assessing teacher manageability: A factor analytic approach. Behavioral Disorders, 13, 245-252. Samuels, S.J. (1981). Some essentials of decoding. Exceptional Education Quarterly, 2, 11-25. Samuels, S.J. (1986). Why children fail and what to do about it. Exceptional Children, 53, 7-16. Sattler, J.M. (1990). Assessment of children: Third edition. San Diego: Sattler. Schumaker, J.B., Deshler, D.D., & McKnight, P.C. (1991). Teaching routines for content areas at the secondary level. In G. stoner, M.R. Shinn, & H.M. Walker (Eds.), Interventions for achievement and behavior problems. Silver Spring, MD: National Association of School Psychologists. Schumm, J.S. & Vaughn, S. (1992). Planning for mainstreamed special education students: Perceptions of general classroom teachers. Exceptionality, 3, 81-98. 71 Stanovich, K.E. & Cunningham, A.E. (1993). Where does knowledge come from? Specific associations between print exposure and information acquisition. Journal of Educational Psychology, 85, 211-229. Tingstrom, D.H. (1989) . Increasing acceptability of alternative behavioral interventions through education. Psychology in the Schools, 26, 188-194. Von Brock, M.B., & Elliott, S.N. (1987). Influence of treatment effectiveness information on the acceptability of classroom interventions. Journal of School Psychology, 25, 131-144. Walberg, H.J. (1984). Improving the productivity of America's schools. Educational Leadership, 41, 19-30. Wang, M.C. (1984). Time use and the provision of adaptive instruction. In L.W. Anderson (Ed.) Time and school learning (pp. 167-203). London: Croom Helm. Wang, M.C. (1985) . An analysis of program design implications for teaching and student use of school time. In C.W. Fisher & D.C. Berliner (Eds.), Perspectives on instructional time. (pp. 237-262). New York: Longman. Waxman, H.C., Wang, M.C, Anderson, K.A., & Walberg, H.J. (1985a). Adaptive education and student outcomes: A quantitative analysis. Journal of Educational Research, 78, 228-236. Waxman, H.C., Wang, M.C, Anderson, K.A., & Walberg, H.J. (1985b). Synthesis of research on the effects of adaptive education. Educational Leadership, 43, 26-29. Weinstein, R.S. (1983) . Student perception of schooling. The Elementary School Journal, 83, 287-312. Weinstein, R.S. & Mayer, R.E. (1985). The teaching of learning strategies. In M.C. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 315-327). New York: Macmillan. Winne, P.H. (1985). Steps toward promoting cognitive achievements. The Elementary School Journal, 85, 673-693. Winne, P.H. & Marx, R.W. (1982). Students' and teachers' views of thinking processes for classroom learning. .Elementary School Journal, 82, 493-518. Winzer, M. & Grigg, N. (1992) . Educational psychology in the Canadian classroom. Scarborough, Ontario: Prentice-Hall. Witt, J.C., & Elliott, S.N. (1985). Acceptability of classroom management strategies. In T.R. Kratochwill (Ed.), Advances in school psychology (Vol.4, pp.251-288). Hillside, NJ: Erlbaum. Witt, J.C, & Martens, B.K. (1983). Assessing the acceptability of behavioral interventions used in classrooms. Psychology in the Schools, 20, 510-517. Witt, J.C, Moe, G., Gutkin, T.B., & Andrews, L. (1984). The effect of saying the same thing in different ways: The problem of language and jargon in school-based consultation. Journal of School Psychology, 22, 361-367. 72 Wittrock, M.C. (1985). Student thought processes. In M.C. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 297-314). New York: Macmillan. Wolf, M.M. (1978) . Social validity: the case of subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203-214. Wong, B.Y.L., Wong, R., & LeMare, L. (1982). the effects of knowledge of criterion task on comprehension and recall in normally achieving and learning disabled children. Journal of Educational Research, 76, 119-126. Woolfolk, R.C. & Woolfolk, A.E. (1979). Modifying the effect of the behavior modification label. Behavior Therapy, 10, 575-578. Woolfolk, A.E., Woolfolk, R.C, & Wilson, G.T. (1977). A rose by any other name ...: Labeling bias and attitudes toward behavior modification. Journal of Consulting and Clinical Psychology, 45, 184-191. Ysseldyke, J.E. & Christenson, S.L. (1987a). TIES: The instructional environment scale: A comprehensive methodology for assessing an individual student's instruction. Austin: Pro-ed. Ysseldyke, J.E. & Christenson, S.L. (1987b). Evaluating students' instructional environments. Remedial and Special Education, 8, 17-24. Ysseldyke, J.E. & Marston, D. (1990) . The use of assessment information to plan instructional interventions: A review of the research. In T.R. Gutkin & C.R. Reynolds (Eds.), The handbook of school psychology: Second edition. New York: Wiley. 73 Appendix A The Instructional Environment Scale (TIES) Video Script Welcome to our video on the Instructional Environment Scale or TIES. Just for a minute, put yourself in the familiar position of having some difficulty with one of the students in your class. You asked that someone come to assess the situation so that you could work more effectively with that student and make life easier for the rest of the class. You may wish to consider using a new approach to this kind of problem. Its called the Instructional Environment Scale or TIES for short. It is designed to assist in the systematic analysis of a student's instructional environment. Using observation and interviewing, it ties assessment to intervention. We know that learning and behaviour are not independent of their surroundings. Behaviour and achievement are determined by interactions among behavioural, cognitive, and environmental influences. Student characteristics are important, but they interact with what's happening in the classroom to create academic and behavioural outcomes. The instructional environment involves more than curriculum and materials. Factors such as the nature of instructional planning, classroom management, the presentation of instruction, motivational 74 techniques, checking for understanding, opportunities for practice, the nature of feedback, and the monitoring of student progress are all important aspects of the instructional environment. If you agree to our using this procedure, we would do three things. First, you and I would engage in a detailed, structured interview. Second, I would come into your classroom to observe. Finally, I would conduct a structured interview with the student. The results would be combined on an instructional rating form which would indicate ways in which the classroom environment could be changed to create a better learning environment for the student. I hope you'll agree that TIES is a good way to address the problem you described in your referral. 75 Break (Titles with voice over) Her are some of the things that TIES is intended to do. TIES provides an objective view of how each of several characteristics of the instructional environment impact upon the student. PAUSE TIES provides information about which areas of the instructional environment are working well for this student and which areas may be modified to improve the student's learning and behaviour. PAUSE TIES emphasizes the child's performance. It examines the child's response to instruction and looks at the progress the student is making. PAUSE TIES provides information which is designed to help the teacher and other staff plan future instruction for the student. PAUSE Now I would like to give you some more detail as to the purposes of TIES and the 12 components that it measures. Its major purposes are, first, to systematically describe the extent to which a student's academic or behavior problems are a function of factors in the instructional environment and, secondly, to identify starting points in designing appropriate instructional interventions for individual students. It can also be used for prereferral intervention to assist teachers in developing plans for individual students at the time a problem is first identified; it can be used by resource teachers, school psychologists, counsellors, and others in consultation with teachers; it can be used to contrast environments which may serve as possible 76 alternative placements for students; and, finally, it can be used in teacher training to assist beginning teachers in applying the principles of effective instruction. There are several things that TIES is not designed to do. It is not a teacher evaluation scale. It contains only a fraction of the factors that would need to be considered in evaluating teachers, and it considers many factors which are clearly not under the control of the teacher. It is not a norm referenced test. There is no reason to compare the instructional environments of students. It is not designed to compare schools or school districts. It is used only to obtain information about individual students' instructional environments. As mentioned earlier, information is gathered in three ways: 1. by interviewing the student's teacher, 2. by observing the student in the classroom, and 3. by interviewing the student. Data from these three sources are used to complete an Instructional Rating Form. It is recommended that classroom observation occur when the teacher is presenting, explaining, or demonstrating a lesson followed by independent seatwork. Preferably, observation should occur during basic skills instruction where presentation and practice can be observed in a selected content area. 77 TIES should be used only when the observation has been prescheduled with the classroom teacher. The purposes of the teacher interview are: (a) to provide data on those areas for which observation is inappropriate or results in incomplete information and (b) to understand the rationale for the teacher's instructional decisions. The purposes of the student interview are: first, to provide information on the student's perception and understanding of academic tasks and, second, to check the student's success rate on assigned work. I would now like to give you some detail about the 12 components which make up TIES. The components were selected using three criteria: (a) they were repeatedly mentioned in the literature as being important for improving academic success. (b) they were easily observable. (c) research evidence supported their inclusion as principles of effective instruction. The first component is instructional presentation. Lesson development is believed to be the most important aspect of instruction. Lessons usually involve a specific sequence of events often referred to as the demonstrate - prompt - practice sequence. There should be a high degree of teacher-student interaction, brisk instructional pace, and immediate corrective feedback. 78 The second component is classroom environment. This includes classroom management which is important for achieving an academic focus in the classroom. Behavioral rules and organizational routines have been identified as key to increasing student engagement rates. The purpose of effective classroom management is to maintain order, thereby increasing the productive use of classroom time. The third component examines teacher expectations. High academic expectations and student accountability for meeting those expectations must be communucated clearly to the student. The fourth component, cognitive emphasis, is based on the idea that students who understand how to solve problems and how to approach a task achieve more in school. Teachers should ask students to explain their answers and the processes used to solve problems. Motivational strategies are involved in the fifth component. The relationship between achievement and motivation is consistently demonstated in research. Effective instruction is characterized by the type of lesson or extra strategies which optimize student motivation. Relevant practice is the sixth component. There are two kinds of practice - controlled practice and independent practice - which need to be present in order to optimize achievement. Tasks must be appropriate and relevant to the teacher's goal and lesson presentation. 79 Academic engaged time is the seventh component. This is based on research which demonstrated that the percentage of class time a student spends involved in academic tasks is a moderate predictor of student success. It has been found that informed feedback, the eighth component, must be specific and explicit, and that it is most effective when it provides students with increased opportunities to respond. Immediate knowledge of results is an important condition of student learning. Adaptive instruction, the ninth component, is viewed as a major way for handicapped students to stay in the mainstream and experience a basic level of success. It involves clearly communicated instructional goals at an appropriate level of difficulty. It includes careful monitoring and feedback, and it emphasizes student responsibility for task completion. Progress evaluation is the tenth component. Monitoring student progress during seatwork has been shown to be an essential correlate of student achievement. Good instructional planning involves an appropriate match between student characteristics and instruction delivered. Student, task, and classroom characteristics must be considered to achieve an appropriate match. 80 The final component is student understanding. We know that student perception of tasks and directions is critical for increasing engaged time and achievement. Students perceptions of teacher intentions and goals are not always consistent with those intended by the teacher. I hope this description has helped you to get a clearer picture of what TIES is about. I want to make it clear that you have the right to decide whether it should be used in your classroom. Please let me know if you have any questions or concerns. 81 Appendix B Rating Instrument for the Instructional Environment Scale (TIES) Please rate each of the following statements by circling the appropriate number on the answer sheet as follows: Strongly Slightly Slightly strongly Disagree Disagree Disagree Agree Agree Agree 1 2 3 4 5 6 1. This would be an acceptable way of assessing a child's academic problem. 2. This would be an acceptable way of assessing a child's problem behavior. 3. The use of this scale would not prove effective in changing a child's academic or behavior problems. 4. I would suggest the use of this scale to other teachers. 5. There is not one child in my class whose problem is severe enough to warrent the use of this instrument. 6. Most teachers would find TIES suitable for use with students who have academic and behavior problems. 7. I would be willing to have TIES used in my classroom. 8. The use of TIES would result in negative side-effects for the child. 9. The use of TIES would result in negative consequences for the teacher. 10. TIES could be used appropriately with a variety of children. 11. The components of TIES are not consistent with the approach I try to use in my own classroom. 12. TIES is a fair way to approach academic and behavior problems. 13. Ties is a reasonable way of approaching academic and behavior problems. 14. I do not like the procedures used in TIES. 15. Overall, the use of TIES should be beneficial for the child. 16. Overall, the use of TIES should be beneficial for the teacher. 17. Applying the findings of TIES would be unlikely to improve academic and behavior problems in the classroom. 18. The use of TIES would produce lasting improvement in academic performance and in behavior. 19. Application of TIES would improve a childs academic performance to the point that it would not noticeably deviate from that of classmates. 20. Application of TIES would not improve a child's behavior to the point that it would resemble that of classmates. 21. Soon after implementing the results of TIES, the teacher would notice a positive change in problem behaviors or in academic performance. 82 22. A child's performance will remain at an improved level long after results of TIES are implemented. 23. Using TIES requires excessive time on the part of the teacher. 24. Using TIES should not only improve the child's performance in the classroom, but also in other settings (eg. other classrooms, home). 25. When comparing a child who has had academic or behavior problems with a child who performs well, the performance of the two children would be more alike after TIES had been utilized. 26. The use of TIES would not produce enough improvement in a child's performance so the performance is no longer a problem. 27. Other behaviors or academic difficulties related to the identified problems are unlikely to be improved through the use of TIES. 83 Appendix C Assessment of TIES Knowledge The following questions are designed to measure your knowledge of the information contained in the foregoing presentation. Please respond to all items by circling either a,b,c,d or e on the answer sheet. 1. Ties is based on the belief that (a) characteristics of the student are primarily responsible for academic achievement and classroom behavior. (b) that student progress and behavior are the product of interaction among behavioral, cognitive, and environmental influences. (c) the home environment of the student is the most important determinant of academic progress and classroom behavior. (d) the classroom environment is the primary influence on student behavior and learning. (e) I don't know. 2. According to the TIES model, the purpose of any assessment should be (a) to find out what is wrong with the student. (b) to identify the special needs of the student. (c) to recommend appropriate placement for the student. (d) to design a plan of intervention for the student. (e) I don't know. 3. A major purpose of using TIES is to (a) systematically describe the extent to which a student's academic or behavior problems are a function of factors in the instructional environment. (b) evaluate the degree to which the student's classroom performance is appropriate for that setting. (c) compare the performance of a student with that student's peers. (d) develop an intervention plan which will be implemented by the classroom teacher. (e) I don't know. 4. Another major purpose of TIES is to (a) evaluate the effectiveness of teachers in their instruction of special needs students. (b) determine the suitability of possible alternative placements for the student. (c) identify starting points in designing appropriate instructional interventions for individual students. (d) examine the appropriateness of the material being taught with respect to the needs of the individual student. (e) I don't know. 5. TIES is designed to gather information on the following number of components of effective instruction: (a) 7 (b) 12 (c) 15 (d) 10 (e) I don' t know. 6. TIES is not designed to (a) compare the instructional environments of various students. (b) provide a qualitative assessment of the instructional environment. (c) evaluate the nature and quality of a student's instructional environment in alternative settings. (d) assist beginning teachers in improving their instructional skills with individual students. (e) I don't know. 84 7. TIES involves (a) interviews with parents, teachers, and students followed by classroom observation. (b) classroom observation combined with consultation between observer and teacher. (c) classroom observation and interviews with the student and the teacher. (d) ratings of the student's performance by peers, the teacher, and independent observers. (e) I don't know. 8. The purpose of the teacher interview is to (a) provide the teacher with a list of recommended changes which should be made in the in the instructional environment. (b) provide an opportunity for the teacher to give more detail about the student's background and classroom performance. (c) provide feedback to the teacher as a result of the classroom observation. (d) provide information in those areas where observation is inappropriate or information is incomplete. (e) I don't know. 9. Instructional presentation refers to (a) the manner in which lessons develop in the classroom. (b) the enthusiasm and tone of the teacher's instruction of the student. (c) the use of various modalities in presenting the lesson. (d) the atmosphere and tone created by the classroom teacher. (e) I don't know. 10. The purpose of effective classroom management is to (a) enable the teacher to work with individuals. (b) reinforce the expectations of the community. (c) maintain order so as to increase the productive use of classroom time. (d) prevent students from disrupting the lesson being presented by the teacher. (e) I don't know. 11. Teacher expectations must be (a) adjusted to meet the standards of the community. (b) adapted to the level of the students in the class. (c) flexible and realistic. (d) clearly communicated to the students. (e) I don't know. 12. Motivation of students is (a) clearly related to achievement. (b) is not a good predictor of student performance. (c) usually beyond the control of the classroom teacher. (d) most strongly related to the home environment. (e) I don't know. 13. Adaptive instruction emphasizes (a) student responsibility for task completion. (b) the individualization of instruction for handicapped students. (c) the commitment of the classroom teacher to diversity within the classroom. (d) the preparation of regular classroom students for the integration of special needs students. (e) I don't know. 14. Student understanding of tasks and directions is (a) dependant upon the ability of the student to comprehend instructions. (b) often subject to distractions within the classroom. (c) critical for increasing engaged time and achievement. (d) dependant upon the reading level of the student (e) I don't know 85 15. TIES components are based upon (a) a comprehensive theory of effective instruction. (b) interviews conducted with teachers and students. (c) parent ratings of teacher effectiveness. (d) findings of educational research (e) I don't know. 16. A cognitive emphasis in teaching involves (a) a focus upon writing skills. (b) clear identification of labels and definitions. (c) asking students to explain their answers and processes used to solve problems. (d) frequent drill and repetition of facts. (e) I don't know. 17.For feedback to be effective (a) it must be task specific and explicit. (b) there must be frequent praise of effort. (c) there has to be a high degree of student success. (d) it must take the form of tangible rewards. (e) I don't know. 18. It is recommended that classroom observations using TIES occur (a) during basic skills instruction. (b) when a variety of activities are taking place. (c) early in the school year. (d) when the student is most likely to be experiencing difficulty. (e) I don't know. 19. Observations should occur (a) without warning to the teacher. (b) at a time convenient to the school principal. (c) at a time which has been prescheduled with the teacher. (d) when the student is likely to be performing at the optimal level. (e) I don't know. 20. Of the three sources of information used in TIES, the most important is (a) classroom observation. (b) the student interview. (c) the teacher interview. (d) a balanced view of all three. (e) I don't know. 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0087358/manifest

Comment

Related Items