UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Socioeconomic gradients in mathematics achievement : findings for Canada from the Third International… Frempong, George 2000

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_2000-565459.pdf [ 6.83MB ]
Metadata
JSON: 831-1.0055011.json
JSON-LD: 831-1.0055011-ld.json
RDF/XML (Pretty): 831-1.0055011-rdf.xml
RDF/JSON: 831-1.0055011-rdf.json
Turtle: 831-1.0055011-turtle.txt
N-Triples: 831-1.0055011-rdf-ntriples.txt
Original Record: 831-1.0055011-source.json
Full Text
831-1.0055011-fulltext.txt
Citation
831-1.0055011.ris

Full Text

Socioeconomic Gradients in Mathematics Achievement: Findings for Canada from the Third International Mathematics and Science Study by GEORGE FREMPONG B.Sc, University of Science and Technology, Kumasi, 1980 M A . , University of British Columbia, 1994 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE F A C U L T Y OF GRADUATE STUDIES Department of Curriculum Studies We accept this thesis as conforming to the required standards THE UNIVERSITY OF BRITISH C O L U M B I A September 2000 ©George Frempong, 2000 U B C Special Collections - Thesis Authorisation Form http://www.library.ubc.ca/spcoll/thesauth.html In presenting t h i s t h e s i s i n p a r t i a l f u l f i l m e n t of the requirements f o r an advanced degree at the U n i v e r s i t y of B r i t i s h Columbia, I agree that the L i b r a r y s h a l l make i t f r e e l y a v a i l a b l e f o r reference and study. I f u r t h e r agree that permission f o r extensive copying of t h i s t h e s i s f o r s c h o l a r l y purposes may be granted by the head of my department or by h i s or her r e p r e s e n t a t i v e s . I t i s understood that copying or p u b l i c a t i o n of t h i s t h e s i s f o r f i n a n c i a l gain s h a l l not be allowed without my w r i t t e n permission. Department of C ^ t C A j L X A W\ S T M frlgg The U n i v e r s i t y of B r i t i s h Columbia Vancouver, Canada Date 0^ < r r - i 2_. — 2-O-D o 1 of 1 10/13/2000 12:24 PM A B S T R A C T Understanding the processes that allow all students to successfully learn mathematics has been an important objective for most education systems including those in Canada. Educational systems however, have not achieved this goal as many students with low socioeconomic status, females, and minority students fail to achieve an adequate knowledge of mathematics. Much of the discussion regarding this lack of achievement concerns classroom resources and practices, school policies within educational systems, and the specific domain of mathematics achievement considered. This study conceptualizes a successful mathematics classroom in terms of its level of mathematics achievement and how equitably achievement is distributed. The study employs multilevel models and the Canadian data from the Third International Mathematics and Science Study to address three main research issues: 1) the extent to which differences in mathematics achievement is attributable to gender, family background, classrooms, and the province where a student attends school; 2) whether the variation in achievement is specific to a mathematics domain; and 3) whether the variation among six provinces (Newfoundland, New Brunswick, Ontario, Alberta, British Columbia, and Quebec) in the levels of their mathematics achievement is associated with various aspects of school policy and practices. The analyses indicate a slight male advantage in mathematics achievement, and a large, significant gap in achievement associated with the socioeconomic status (SES) of the students' families. Students from low SES backgrounds are disadvantaged as they tend to have relatively low achievement in mathematics within classrooms, especially in Proportionality, Measurement, and Fractions. The most successful classrooms are those in which students from disadvantaged backgrounds excel in mathematics. Disadvantaged students excel in mathematics classrooms in which there are fewer groupings, the mathematics teachers are specialized, and in schools with lower pupil-teacher ratio. Mathematics achievement is equitably distributed in provinces with high mathematics achievement levels. Provincial achievement levels are stable across mathematics domains; that is, provinces with high achievement levels in one domain also tend to have high achievement levels in other domains. On average, Quebec's mathematics achievement is higher than the other provinces in all mathematics domains, and at all levels of SES. This high achievement level in Quebec is partially attributed to higher teacher specialization, lower pupil-teacher ratio, and lower within-school remedial tracking. The study recommends a comprehensive longitudinal study employing multilevel models with a focus on what other provinces can learn from Quebec's advantage in mathematics. Such a study should conceptualize successful mathematics classrooms as those in which an average student excels in mathematics and where mathematics achievement is equitably distributed. - i n -T A B L E OF C O N T E N T S Abstract ii Table of Contents iv List of Tables ix List of Figures xi Acknowledgments xii Dedication xiii CHAPTER 1 Introduction 1 Why Mathematics Outcomes 4 Equity, Excellence, and Gradients 5 Mathematics Domains 8 Research Method 11 Research Obj ective 12 CHAPTER 2 Literature Review 14 Theoretical Background 14 Family Background 18 Between-Classroom Segregation 19 Curriculum Exposure and Tracking 20 Mathematics Instructional Practices 22 Teachers and Mathematics Learning 23 -iv-Gender 25 The Canadian Context 28 Mathematics Teaching and Curriculum Content 32 Between-School Segregation 33 Within-School Tracking 35 Mathematics Teachers 35 Children from Immigrant families 36 Summary 37 CHAPTER 3 Data and Research Method 38 Population and Sample 38 Sampling Design and Sample Weights 40 Research Method 42 Models and Analyses 44 Null and Baseline Model 44 Classroom Effects 46 Multivariate, Multilevel Models 47 CHAPTER 4 Mathematics Achievement and Students' Backgrounds 49 Scale of Mathematics Achievement 49 Variables Describing Student's Backgrounds 50 Provincial Differences in Students' Backgrounds 51 Mean of Students' SES in a Class (Mean SES) 53 -v-Relationship Between Mathematics and Students' Backgrounds 53 Double Jeopardy Effect 59 Analyses of Explained Variance 63 Adjusted Mean Scores 66 Contextual Effects 68 Summary of Findings 69 CHAPTER 5 The Stability of Achievement across Domains of Mathematics 71 Mathematics Domains 71 Scaling of the Domains 73 Statistical Models and Analyses 73 Immigrant Status Gradient 77 Gender Differences 79 SES Gradients 82 Double Jeopardy Effect 82 Analysis of Explained Variance 84 Adjusted Mean Scores 88 Domain-Specific Contextual Effect 88 Summary of Findings 93 CHAPTER 6 Provincial Differences in Mathematics Achievement: What Matters? 96 Description of Variables Describing Schooling Processes 97 Curriculum Coverage 97 -vi-Instructional Practices and Other School Processes 98 Statistical Models and Analyses 103 Grade Effect 105 Effects Associated with Students' Background 105 Mean SES Effects 106 Curriculum Coverage Effect 106 Effect Associated with Instructional Practices 107 Effects Associated with Variables Describing Other School Processes 107 Analyses of Variance Components 108 Residuals of Classroom Achievement Scores 114 Why the Six Provinces Differ in their Achievement Levels 114 Summary of Findings 117 CHAPTER 7 Summary and Discussion of Findings and Implications 119 Major Findings 120 Implications for Further Research 129 REFERENCES 131 APPENDIX A National and International Studies 143 National Longitudinal Study of Children and Youth (NLSCY) 143 International Adult Literacy (IALS) 144 School Achievement Indicator (SAIP) 144 -vii-APPENDIX B Figure 2-1. Inter-Provincial Differences in Mathematics Scores - v i i i -List of Tables Table 4.1 Descriptive Statistics, Pearson Correlations, and Factor Loadings 51 for Measures of the SES Composite Table 4.2 Students' Background Variables: Descriptive Statistics by Province 52 Table 4.3 Estimates of Gradients by Province 55 Table 4.4 Variance Components and Provincial Mathematics Achievement 64 Levels Table 4.5 The Effect of Students' Background Characteristics on Mathematics 67 Achievement by Province Table 4.6 The Effect of Students Background Characteristics, and Classroom 68 mean SES on Mathematics Achievement by Province Table 5.1 Distribution of the TIMSS Test Items by Mathematics Domain 72 Table 5.2 Parameter Estimates of Within-and Between-Classroom Correlations 75 Table 5.3 Pearson Product Moment Inter-correlation Matrix: Provincial 76 Achievement levels Table 5.4 Estimates of Gradients by Mathematics Achievement Domain 80 by Province Table 5.5 Domain Achievement Levels and their Variance Components 80 by Province Table 5.6 Reliability Estimates of the Between-Classroom Variance 86 Table 5.7 Relationship Between Domain-specific-Outcomes and SES, Gender, 87 Immigrant status, and Grade Table 5.8 Relationship Between Domain-specific-Outcomes and SES, Gender, 89 Immigrant status, Grade, and Classroom mean SES Table 5.9 Variance Components and Variance Explained by Students' 90 Background Characteristics Table 5.10 Variance Components and Variance Explained by Students' 92 Background Characteristics and Classroom mean SES -ix-Table 6.1 Percentage of Curriculum Coverage by Province 98 Table 6.2 Description of Classroom and School Process Variables 99 Table 6.3 Descriptive of Classroom and School Process Variables 101 Table 6.4 Parameter Estimates of the Models 104 Table 6.5 Analysis of Variance Components and Parameter Correlation 109 Estimates Table 6.6 Classroom Profile Within Provinces: Achievement Levels 112 by SES Gradients -x-List of Figures Figure 4.1 Gender Differences in Mathematics Achievement by Province 54 Figure 4.2 Differences in Mathematics Achievement Between Immigrants 56 and Non-immigrants by Province Figure 4.3 Relationship Between Mathematics Achievement and SES 58 by Province Figure 4.4 Provincial Mathematics Scores and their SES Gradients 58 Figure 4.5 The Relationship Between Mathematics Score and Socioeconomic 60 status at Different levels of SES Figure 4.6 Students' Mathematics Achievement Scores at Different 62 levels of SES and meanSES of Classroom Figure 5.1 Relationship Between Immigrant status and Achievement in 78 Mathematics Domains Figure 5.2 Gender Differences in Mathematics Achievement by Domains 79 Figure 5.3 Relationship Between Socioeconomic status and Achievement in 81 Mathematics Domains Figure 5.4 Relationship Between mean SES and Achievement Level in a 83 Mathematics Domain Figure 6.1 Relationship Between Adjusted Classroom Achievement 110 Scores in Fraction and their SES Gradients by Province Figure 6.2 Between Classroom Variation at Different Levels of 113 Socioeconomic status Figure 6.3 Mean Classroom Deviation Scores by Province for Different Models 115 - x i -A C K N O W L E D G M E N T S This is to acknowledge and thank a number of people who contributed in various ways to the completion of this dissertation. My sincere gratitude goes to my research supervisor, Dr. Douglas Willms. His thoughtful advice, encouragement, and support made the completion of this dissertation possible. I would also like to express my profound appreciation and gratitude to my academic-program advisor, Dr. Jim Sherrill, for his guidance and help throughout the course of my study at U B C . I am extremely grateful to Dr. Anthony Clarke for his willingness to step into Jim's shoes to ensure the completion of this dissertation. My thanks also go to the staff at the Canadian Institute for Policy Research (CRISP), particularly Dr. Elizabeth Sloat who read the initial draft of this dissertation and offered very useful advice. Statistics Canada, and the Spencer Foundation through the major grant program entitled, School and Community Effects on Children's Educational and Health Outcomes provided financial support. I am grateful to both organizations for their support. Finally, many thanks to my families: my wife, Ancilla, and the children; Brandy, Belinda, and Billy, and my mother, uncles, brothers, and sisters. They all supported and prayed for a successful completion of this dissertation and I am so grateful. -xii-In Memory of my Grandpa (Nana Kwaku Sarfo) and my Dad (Joseph Akwasi Frimpong) for their Love and Trust -xiii-Chapter 1 Introduction The fundamental objective of any public education system is to provide mass education that will ensure quality school outcomes for all students. The belief is that the stock of skills and experiences essential to an individual's well-being requires basic school knowledge such as mathematics. Also, the uniform distribution of this basic knowledge among social groups is necessary for a society's overall productivity and social cohesion. Schools and school systems can play a major role not only in providing the means for students to acquire this knowledge, but also in the equitable distribution of this knowledge. This research study examines factors and processes that play a role in shaping patterns of mathematics achievement among Canadian students across six provinces. Over the past three decades, the "production function" theory has dominated the understanding of researchers on how schools can be effective in their role as sources of knowledge for students (see Lee & Bryk, 1989; Raudenbush & Willms, 1995). This theory posits that the productivity of a school or school system, measured in schooling outcomes, such as mathematics achievement, is a function of students' home background characteristics, school inputs, and school processes (see Lau, 1979; Levin, 1980). The theory is consistent with findings that children tend to have poor schooling outcomes if they are from poor and less-educated families (see White, 1982), or attend schools with a high concentration of disadvantaged children (Willms, 1992). Also consistent with this theory are the findings that children have superior schooling outcomes in classrooms with small class size where students are taught by qualified and specialized teachers (see Darling-Harmond, 2000). A positive school climate, and effective instructional practices are also associated with high achievement levels (Willms, 1992). Analyses employing production function theory usually entail estimates of the proportion of total variation in students' schooling outcomes associated with factors pertaining to schooling inputs and schooling processes, controlling for factors concerned with students' home backgrounds. The underlying assumption is that, in general, schooling can enhance the learning of students over and above the effect of family background. The approach is concerned mainly with identifying factors associated with school excellence gauged by average test scores, and is insensitive to equity in the distribution of school outcomes, or the processes pertaining to the academic success of disadvantaged students. In recent years, there has been growing evidence indicating that variation among schooling systems in achievement levels is determined mainly by their success with disadvantaged students (Willms, 1997). Much of the discussion on how to improve the schooling outcomes of these disadvantaged students has emphasized the need to understand how students of differing status perform in schools, and whether their performance is related to particular schooling processes. Research indicates that a number of factors such as classroom instructional practices, the nature of interaction between students and teachers, as well as the attitudes, values, and expectations of students are directly related to students' academic success (see Willms, 1992). In "successful schools" there is greater "academic pressure" through processes where the principal and teachers project the notion that all students can be successful academically. Such schools have high academic expectations for all students. The expectations are often manifested in teaching practices, the type and amount of homework, the use of time in the classroom, the allocation of resources, and the assignment of students into courses. The evidence suggests that understanding the process through which schooling systems affect the achievement levels of students of different status is fundamental to understanding the variation in school effects. The production function approach is, therefore, inadequate since it fails to account for how schools affect the achievement levels of students from diverse backgrounds. Willms (1999) has proposed an approach that combines the production function concept and what he calls "socioeconomic gradient". A gradient refers to a gap in schooling outcomes between minority and majority groups, or between males and females. The term "socioeconomic gradient" refers to the relationship between individuals' school achievement and their socioeconomic status (SES). SES describes a person's access to and control over wealth, prestige, and power. It is typically measured through factors such as income, the prestige of a person's occupation, and his or her level of education (see White, 1982). The socioeconomic gradient is a reliable indicator of social equity since it highlights the gap in school achievement between advantaged and disadvantaged groups. Shallow gradients indicate that schooling outcomes are distributed equitably among children with varying SES, while steep gradients demonstrate less equitable distribution. The approach entails an understanding of the processes associated with the variation in achievement levels and the variation in socioeconomic gradients. The socioeconomic gradient approach can enhance an understanding of how schools affect students from diverse backgrounds by addressing two fundamental questions: (1) To what extent do schooling systems vary in their outcomes for students of differing status? and (2) What school policies and practices improve levels of schooling outcomes and reduce inequalities between high and low-status groups? (Willms, 1992). This research study attempts to address these questions by employing the production function and gradient approaches to identify the sources of variation in students' mathematics outcomes, and discern how schooling practices and educational policies affect the mathematics outcomes of students from diverse backgrounds. The study also seeks to identify factors that characterize effective school systems; that is, those with high achievement levels and shallow socioeconomic gradients. Why Mathematics Outcomes This research study focuses on mathematics outcomes for two reasons. Mathematics is an important school subject required for graduation from high school in many education systems; and, perhaps more importantly, students' future career opportunities are increasingly becoming reliant on their acquired mathematical knowledge (National Council of Teachers of Mathematics (NCTM), 1991). Over the past two decades, the demand for low skilled workers has decreased, while the continued proliferation of high technology industries has increased the demand for skilled workers (Organization for Economic Co-operation and Development (OECD), 1994, 1995). Employment in the technology sectors requires an understanding of data analyses, mathematical models, and accounting procedures that require mathematical literacy (NCTM, 1989,1991). Yet mathematics, probably more than any other school subject, has been used by educators to classify students according to ability, which in turn determines access to higher education and all its potential privileges (Harris, 1991). In higher education, students with inadequate mathematics skills are likely to be denied the opportunity to enroll in about 70% of courses that lead to future professional careers (Sells, 1973). Mathematics has become a filtering device that places some individuals and groups into permanent disadvantaged positions in society (Willis, 1989). Females, minority students, and students from low socioeconomic backgrounds (SES), because of their poor performance in mathematics, are often the victims of such a filtering process (Fennema & Leder, 1990; Harris, 1991). This is particularly worrisome and unjust when schools, because of practices which are insensitive to students' background characteristics, are known to contribute to this problem ( N C T M , 1989: Oakes, 1990). Oakes (1990) argues that a common practice among schools in the United States is to offer a number of modified mathematics courses and to place disadvantaged students in less challenging courses with less qualified teachers. Such a practice tends to reduce the opportunities for disadvantaged students to do well in mathematics. A number of researchers have called for reforms that would redress this practice. The N C T M (1989, p.4) supports this call: The social injustice of past schooling practices can no longer be tolerated .... Mathematics has become a critical filter for employment and full participation in our society. We cannot afford to have the majority of our population mathematically illiterate: Equity has become an economic necessity. Equity, Excellence, and Gradients The N C T M (2000) calls for a commitment to equity that "supports another central goal of mathematics education-namely, excellence"(p. 2). The contention is that understanding the processes for achieving excellence and equity in mathematics education is likely to ensure successful mathematics outcomes for all students. The processes for achieving the two goals are "complex and interrelated" (NCTM, 2000), and require an understanding of mathematics learning. One major objective of this research study is to discern the relation between excellence and equity in students' mathematics achievement among educational systems in Canada. In this research study, mathematics achievement levels are used as a measure of excellence, while socioeconomic gradient is used as an indicator of equity. A socioeconomic gradient refers to the relationship between mathematics achievement and SES. Gradients also refer to the achievement gap between immigrants and non-immigrants, and between males and females. There is a need for comprehensive studies to examine the characteristics of schools and classrooms or educational systems that promote high mathematics achievement levels and shallow gradients. Researchers in mathematics education have usually emphasized issues associated with either mathematics achievement levels or gradients, with less emphasis on the relationship between them. An important issue is whether classrooms, schools, or school systems with high mathematics achievement levels tend to have shallow or steep gradients; that is, can schools achieve excellence without compromising equity? There are quite a number of research studies on gradients. Most of these studies focus on gender and race differences in mathematics achievement, rather than with differences associated with students' SES (Secada, 1992). Secada (1992) maintains that researchers and educators do not "bristle with the same sense of outrage that the poor do not do well in mathematics as their middle-class peers as they do with similar findings along other groupings" (p.640) such as that of males versus females for instance. In the mid-1960s, the Coleman report demonstrated that American children from low SES families performed poorly in mathematics (Coleman, 1990; Coleman et al., 1966). In 1972, the First International Mathematics Study (FIMS) supported Coleman's claim with data from other countries. A meta-analysis by White (1982) provides evidence for the consistent positive relationship between SES and students' mathematics achievement. Recent studies confirm the enduring effect of students' SES background on school achievement (Lee, Smith & Croninger, 1997; Rumberger & Willms, 1992; Willms, 1997). Despite these findings, relatively few studies have explored the cause of the poor performance of students from low SES families (Reyes and Stanic, 1988). Rather, researchers prefer to use SES as a control variable to assess differences in achievement among groups, instead of looking for the underlying causes of differences in mathematics achievement associated with the SES (White, 1982; Secada, 1992). One explanation for the lack of interest in addressing the SES-based differences in mathematics achievement is that, unlike gender, where males and females can easily be identified, students cannot easily be labelled in terms of the factors comprising social class or SES. Furthermore, other researchers argue that the poor performance of students from low SES families in general is attributable to their home environments, and hence, schools can do little to help the situation (see Coleman et al, 1966). Other researchers claim there are processes in schools and classrooms that tend to contribute to the poor performance of low SES students (Lee & Smith, 1995; Oakes, 1990; Willms, 1986). For example, using data from the National Assessment of Educational Progress (NAEP), Lee and Smith (1995) demonstrate that the mathematics achievement levels of low SES students are higher in schools that used nontraditional instructional practices, such as cooperative learning, compared with schools that used traditional teaching methods. Oakes (1990) emphasizes that processes in schools such as "tracking" deny some students the exposure to advanced mathematical concepts, a practice that affects the mathematics achievement levels of low SES students more than high SES students. Studies of middle and secondary schools have shown that schools vary considerably in both levels of performance and steepness of socioeconomic gradients, and that some of this variation is associated with particular schooling processes (e.g., Ho & Willms, 1996; Lee & Smith, 1993). Other studies have demonstrated the growing significance of individual classrooms in students' academic outcomes (see Goldstein, 1997). These studies demonstrate that classrooms and schools can make a difference in students' mathematics achievement, especially the achievement of low SES students. In general, a typical mathematics classroom consists of students with diverse backgrounds and one expects students from disadvantaged backgrounds to achieve low in mathematics. This low achievement reflects not only the history of their prior participation in mathematics but also on the immediate social environment of their classroom and their participation in communal practices within the mathematics classroom. Cobb (1998) points out that it is possible to create a classroom learning environment that emphasizes the diversity of students' reasoning in mathematics learning. One expects excellence in students' mathematics achievement within such a mathematics classroom to be independent of background characteristics. Understanding how students of diverse backgrounds come to excel in mathematics is fundamental to understanding classroom success in mathematics learning. Mathematics Domains A growing concern involving school and classroom researchers is the description of school outcome measures employed by researchers. Researchers have used a narrow range of outcome measures, usually mathematics and reading, and student performance in these subjects has typically been described using the "total aggregated score". A number of researchers however, argue that scores aggregated across different domains within a subject represent general ability in that subject, rather than domain-specific achievement that can be linked to curriculum and instruction (Kupermintze et al; 1995; Muthen et al., 1995). Furthermore, researchers advocate tests that are well-matched to within subject domains, because student performance often varies across domains (Stedman, 1994; Westbury, 1992). Others note that because total aggregated scores are dependent on the relative weighting of subject domains (Cronbach, 1971), ranks based on total aggregated scores are unstable, and they can result in unfair comparisons (Guskey & Kifer, 1990; Mislevy, 1995). Therefore, studies that disaggregate achievement into its various domains provide a better indication of the effects of teaching and the curriculum. A conceptually simple approach to this problem is to carry out separate analyses for each achievement domain. Such an approach however does not fully exploit the data, because information linking individual students' scores across domains is lost. For example, one might like to ask whether a schooling system that is effective in algebra is also effective in geometry, and whether a schooling system that has shallow gradients in algebra also has shallow gradients in geometry. Also, Robitaille et al., (1993) argue that the underlying processes for the deployment of the curriculum in a classroom determine the patterns of content coverage and create the context for teaching practices, which determine the overall achievement profile of an educational system. One will therefore expect the curriculum-driven patterns of teaching Geometry, and the distribution of the opportunity to learn Geometry in an educational system to gauge their students' performance in Geometry. A multivariate, multilevel model (Goldstein, 1995; Thum, 1997) is appropriate for analyzing domain-specific data that are nested hierarchically. Furthermore, research in mathematics education that seeks to understand how schools and classrooms affect students from diverse backgrounds has relied on qualitative and traditional statistical models. The qualitative approach includes interviews and classroom observations usually involving small groups of students or teachers. A cursory look at research reports, conference proceedings, and the handbook of research in mathematics education indicates the predominance of qualitative research methods in mathematics education (see Grouws, 1992). This approach raises questions about the ability to generalize the research findings, because samples are usually not random, and, in most instances, are too small to draw any inferences to a known larger population. In general, it is not plausible to have intensive interviews and classroom observations with large numbers of students. The few quantitative studies in mathematics education have relied on traditional statistical methods such as multiple regression analysis and analysis of variance (ANOVA). These statistical procedures are limited, especially for complex data sets with an hierarchical structure. An understanding of how classrooms and educational policies affect students' mathematics achievement requires data describing large samples of students and classrooms, and complex statistical procedures that exploit the multilevel structure of the data. This research study employs the Canadian data from the Third International Mathematics and Science Study (TIMSS). Canada's participation in TIMSS marks the first time that the entire country has taken part in an international study in education. The Canadian study includes a large representative sample of public and private schools, with data from teachers and students from all provinces of the country in both official languages (English and French) (Robitaille, Taylor & Orpwood, 1997). Five provinces - Alberta, British Columbia, New Brunswick, Newfoundland, and Ontario over-sampled their population such that comparisons among these provinces is possible. Comparison among the provinces in their mathematics outcomes and SES gradients is particularly important because each province has complete jurisdiction over its educational systems (Willms, 1997). In recent years, the provinces have participated in national and international studies for the purpose of learning from one another other the factors contributing to the enhancement of -10-school outcomes of their students. The analysis of data from these studies has yielded two important findings: first, the provinces vary in their mathematics outcomes and in the quantitative skills of their youths (see Council of Ministers of Education, Canada (CMEC), 1993, 1997; Frempong & Willms, in press; Willms, 1996, 1997); second, there is a negative relationship between the level of quantitative skills of youth in a province and SES gradients; third, the variation among provinces in their levels of quantitative skills at the high SES levels is small. These findings indicate that the gap between advantaged and disadvantaged youths is small in provinces with high quantitative skills (Willms, 1996). The second finding suggests that provinces attaining high quantitative skills tend to do so by raising the skills of their disadvantaged youth. Thus, an understanding of how disadvantaged children fare in schools within provinces can provide useful information about the sources of variation among provinces in their schooling outcomes. The School Achievement Indicators Program (SAIP) and the International Adult Literacy Study (AILS) however are limited in that they do not include data on students' school and classroom processes. This research study employs data from TIMSS to examine the variation among provinces in six domains of mathematics (Algebra, Fractions, Geometry, Measurement, Proportionality, and Statistics), and attempts to explain this variation with variables describing school and classroom processes. Research Method The analyses of the data for this research study employ multilevel statistical procedures. These are complex statistical procedures that allow researchers to estimate models with nested data sets. In studies where students are nested within classrooms, the statistical procedures allow for regression analyses within classrooms and provide estimates of intercepts and regression coefficients (gradients), and the variation of these estimates within and among classrooms. The analyses also employ multivariate, multilevel statistical procedures so that a multilevel analysis of the six domains of mathematics can be carried out simultaneously. (See Chapter 3 for the discussion of these procedures.) The application of multilevel models in research is becoming increasingly popular, especially in medicine, economics, and education (Goldstein, 1995; 1997). Multilevel statistical models, and the research and practical issues they address, are highly relevant in many areas of mathematics education (see, Frempong & Willms, in press, Willms & Jacobson, 1990). Such an important and emerging research tool has yet to make its debut in the main-stream of mathematics education research. The application of this important statistical tool would serve as a referent to researchers in mathematics education who might wish to employ multilevel statistical models to address relevant research issues. These models could also be very useful for other TIMSS countries to assess similar issues in their education systems. Research Objective The main objective of this research is to compare grade 7 and grade 8 mathematics classrooms across Canada, and within six Canadian educational systems based on their achievement levels and socioeconomic gradients across six domains of mathematics. Factors associated with levels and socioeconomic gradients are also examined. The following analytical approaches were employed: 1) tracing the sources of variation in students' mathematics achievement levels; 2) identifying classroom and school factors associated with this variation; and 3) describing the sources of differences among six Canadian provinces in their mathematics achievement levels. -12-The motivation to pursue these objectives is rooted in the belief that education systems have the obligation to provide education that ensures opportunity for excellent mathematics outcomes for all students irrespective of their background characteristics. The thesis of this research study is that, to understand the schooling processes leading to successful mathematics learning for all students, one needs to understand the schooling processes associated with both excellence and equity. Chapter 2 discusses the theory and research relevant to issues addressed in this research study. Chapter 3 describes the data, the statistical methods, and the statistical models used. The analyses are carried out in three phases and are presented in Chapters 4, 5, and 6. Chapter 7 provides a summary and a discussion of the findings and their implications. -13-Chapter 2 Literature Review This chapter reviews the theoretical perspectives and previous studies informing the processes and characteristics associated with mathematics learning and variation in mathematics achievement levels. The review is also situated in the Canadian context in an attempt to understand the variation among provinces in their mathematics achievement. Theoretical Background One can view mathematics learning from two complementary perspectives: the "individualist" perspective, and the socio-cultural perspective. The individualist perspective explains learning as an acquisition of knowledge through processes that occur within the individual. Individuals "actively construct their mathematical way of knowing" (Cobb, 1994, p. 13). The socio-cultural perspective envisions knowledge as being distributed, and that learning occurs through interaction with and participation in socio-cultural practices. From the individualist perspective, the process within the individual is more important, while from the socio-cultural perspective, the activities within learning environments are more important. The theories underlying the individualist perspective have been advanced by a number of researchers (see Cobb, 1994). The main argument is that during the formative years children acquire a network of ideas, and the connections among these ideas allow them to make sense of new information (see Schoenfed, Smith and Arcavi 1993; Siegler and Klahr 1982). The conceptual organization of this new information occurs within the individual. When children encounter new information, they try to associate it with their existing knowledge and personal experiences (Mayer, 1992) in an attempt to construct a meaningful link between familiar and -14-unfamiliar information. Sometimes, the new information is elaborated and integrated into an existing schema within an individual. Other times, the existing schema is adapted to fit the new information. Learning, therefore, involves processes of individual knowledge construction. However, the knowledge construction and the desire of a child to continue to process the new information are always relative to a given goal (von Glasserfeld, 1992). In this model, an individual's conceptual process relative to a purpose or goal is posited as the driving force of learning. From this perspective, one will expect students who value, like, and see some usefulness in mathematics, and who can connect mathematics to their real life experiences to persist in mathematics learning. This argument is consistent with Bandura's (1977) social cognitive theory which holds that individuals evaluate and make decisions about their ability to carry out certain tasks which they believe have desirable outcomes. The expectancy value theory (see Atkinson, 1964) supports this argument; it posits that students persist in a task, such as solving mathematics problems, if they expect to be successful and if they value the task. The theory is supported by research which shows that students' attitudes towards learning, particularly their attitudes towards mathematics learning, are important determinants or results of successful mathematics learning (Macleod, 1992). Three main issues are emphasized in this theory: whether students like mathematics, whether mathematics is useful to students, and whether students are confident in mathematics (McLeod, 1992). The individualist perspective focuses on the processes by which individuals attempt to make sense of mathematical knowledge. Although past experiences play an important role in this process, the individualist perspective usually gives less prominence to the context of the past and immediate social and cultural experiences and the role this context plays in the construction of -15-mathematical knowledge. The socio-cultural perspective emphasizes mathematics learning situated in a social and cultural context. The theoretical position of this perspective is motivated largely through the work of Vygotsky (Nunes, 1992), who argues that, in general, learning occurs when an individual internalizes a social experience through interacting with a peer or adult (Vygotsky, 1988). The process of learning occurs through cognitive processes that originate and form through social interaction. Vygotsky (1978) stresses the importance of social interaction with more experienced others through the concept of the "zone of proximal development" (zpd) and the role of culturally developed instruments as psychological tools for thinking. The zpd is defined as the distance between a child's independent problem-solving ability and his or her potential for success through collaboration with others. Leant'ev (1981) supports Vygotsky's view but stresses the importance of engagement in activity. He maintains that learning occurs through interaction and participation in activity. Other researchers emphasize the importance of locating learning in the co-participation in cultural practices (Lave & Wenger, 1991; Rogoff, 1990). In this model, the students' social engagements through interaction with more experienced others, and through participation in cultural activities are the driving forces for learning. From this perspective, the variation in the processes that allow students to interact with peers and teachers and fully participate in mathematics communal practices is the source of the variation in students' mathematics learning. In this sense, the learning environments are more important than an individual's cognitive processes. Some mathematics educators now view individuals' cognitive process and their learning environments as equally important for understanding students' mathematics learning (see N C T M , 2000). The belief is that the two perspectives are reflexively related such that one does not exist without the other (Cobb, 1998). That is, cognitive processes within individuals and the -16-context of students' active participation in classroom mathematical practices are both important for understanding students' success in mathematics learning. This is consistent with a view of a mathematics classroom as a community with norms and practices, and also, as "a collection of individuals who mutually adapt to each other" (Cobb, 1998, p. 1-44). This view suggests that the norms and practices of a mathematics classroom as well as the characteristics of individuals within the classroom are equally important in determining how students come to understand mathematics. Cobb (1998) describes the mathematics classroom practices as involving "taken-as-shared ways of reasoning, arguing, and using tools that are established by a classroom community" (p. 1-44). Although a typical mathematics classroom consists of a number of diverse students such that the practices and mathematical interpretations of some students may be less sophisticated than others simply because of their background experiences, classroom mathematical practices can be designed to support the mathematical development of all students. This requires that the classroom community allow all students the opportunity to participate in the classroom mathematical practices. Cobb (1998) contends that "students who cannot participate in these practices are no longer members of the classroom community from a mathematical point of view" (p. 1-44). He argues that individuals learn mathematics by participating in and contributing to an emerging mathematical understanding in a classroom community. In other words, the most important aspects of classroom mathematics learning are: the emerging mathematical practices, the immediate social situation of the students' mathematics learning, and the nature of students' participation in mathematical practices. The immediate implication of this view is that successful mathematics learning for every student is possible through a mathematics classroom that "brings the diversity of students' reasoning to the fore while simultaneously seeing that diversity is socially situated" (Cobb, 1998, p. 1-45). -17-Family Background A major influence on students mathematics learning is the family environment. Over the past few decades, a number of studies have demonstrated the importance of students' home background characteristics on students' mathematics achievement. Research indicates that students' schooling outcomes are related to the socioeconomic status of their family (Coleman et al., 1966; Dossey et al., 1988; White, 1982). In the United States, Coleman et al, (1966) indicates that the SES background of students accounts for most of the variation in mathematics achievement levels in schools. The study further demonstrates that children from low SES families performed poorly in mathematics. Recent studies indicate that there is still a significant achievement gap between low and high SES students (Caldas & Bankston, 1997; Rumberger & Willms, 1992; Secada, 1992; Willms, 1997). Willms and Kerr (1987) showed that the differences between low and high SES groups could be as high as 1.5 standard deviations. The high achievement of students from high SES families can be attributed to the parents' support and to academic activities within the family (see White, 1982). Within high SES families, children have access to mathematics-related materials such as calculators, computers, and supplemental books, but, more important, their parents assist them with their mathematics work. Also, unlike many low SES students, middle and high SES students tend to be raised in cultural and familial environments that naturally prepare them for school such that they easily adapt to school and classroom environments (Oakes, 1990; Secada, 1992). Therefore, students from low SES families tend to be less prepared for classroom mathematical activities and are particularly disadvantaged. -18-Between-Classroom Segregation Between-classroom segregation occurs when classrooms vary significantly in the background characteristics of their students, such that certain classrooms have a high concentration of high SES students, while other classrooms have a high concentration of low SES students. A number of studies have shown that the characteristics of a school and the composition of its student intake have effects on student outcomes over and above the effects associated with an individual student's ability and socioeconomic status (Brookover et al, 1978; Rumberger and Willms, 1992; Willms, 1986). In most schooling systems, students tend to have better schooling outcomes when they are enrolled in a school with a high concentration of students who have above-average ability and family socioeconomic status. During the 1980s, sociologists attributed this effect, called the "contextual effect", to peer effects associated with bright and motivated students working together (Clifford & Heath, 1984). However, contextual effects may be attributable to other schooling processes, such as parental support and the disciplinary climate of the classroom and school. Contextual effects may be particularly strong on mathematics achievement. Studies of school effects indicate that students are more likely to have higher achievement in mathematics classrooms with high SES students than in classrooms with low SES students (see Ho and Willms, 1996). Research suggests that contextual effects may be greater for disadvantaged students than for students with more social and economic resources (Willms, 1985, 1997). However, only a few studies have explicitly tested hypotheses pertaining to interactions between classroom context and student background. The hypothesis is that the chances of success for disadvantaged students in mathematics is doubly jeopardized when they are enrolled in a classroom with a high number of other disadvantaged students. If this is the case, it means that when students are segregated by mathematics ability between classrooms within a school community, one would -19-expect a marginal increase in the achievement of students from advantaged backgrounds, but a substantial decrease in achievement of students from disadvantaged backgrounds. Understanding the nature of contextual effects, and the processes through which contextual effects become entrenched in a schooling system, is fundamental to understanding school effects. This research study examines the extent of contextual effects or "double jeopardy" effect for Canadian students in grade 7 and grade 8. Curriculum Exposure and Tracking Segregation may also occur when students are tracked into different courses within schools whereby academically able students are placed in advanced courses, and academically weak ones in less advanced courses. A key finding of the Second International Mathematics Study (SIMS) was that schools in the United States offer numerous curricular options for students even at the early grades. Differences in the content of these courses accounted for a significant proportion of the variation in students' mathematics performance (McKnight, et al., 1987). Further studies using data from the National Assessment of Educational Progress (NAEP) indicate that high mathematics achievement is associated with the level of mathematics courses to which students have been exposed (Mitchell et al, 1999; Reese, et al., 1997; Hoffer, 1997). Such studies suggest that practices in schools that enhance or limit students' exposure to the mathematics curriculum are likely to affect their mathematics achievement. Students in advanced courses are expected to be high achievers, while those in less advanced courses are expected to be low achievers. The extent of curriculum coverage often varies from course to course with students in more advanced courses more likely to cover more material than their counterparts in less advanced courses. -20-There is growing evidence in research from United States that tracking puts low achievers at a serious disadvantage rather than substantially benefitting high achievers (Oaks, 1985). Low achievers enrolled in lower tracks are denied the opportunity to interact with high-achieving peers, and worse still, they receive a restricted curriculum. In general, the interaction between teachers and students in these lower tracked classes is less motivating, less supportive, and less demanding of higher-order reasoning and responses (Good & Brophy, 1986). These interactions tend to focus on students' behavior rather than on academics, and teaching presentations are less clear and less focused on higher-order cognitive goals (Oaks, 1985) thereby exacerbating differential access to knowledge and learning. The research indicates that the achievement of these students is lower than students of similar aptitude enrolled in academic (college-bound) programs or in untracked classes (Oaks, 1985). Oaks (1985, 1990) claims curricular differences between tracked and non-tracked classes explain much of the difference between the achievement of advantaged and disadvantaged students. When students of similar backgrounds and initial achievement levels were exposed differentially to either more or less challenging curriculum materials, those given the richer curriculum opportunities outperformed those placed in less challenging classes (Oaks, 1985). Lee & Smith (1995) have also shown that secondary schools with highly differentiated (or tracked) course offerings have the lowest achievement among economically disadvantaged students. This finding suggests that the distribution of mathematics achievement levels is less equitable in schools where students are tracked into mathematics courses. -21-Mathematics Instructional Practices Another important factor contributing to the variation in students' mathematics achievement levels is the variation in instructional practices within mathematics classrooms. These instructional practices determine what and how much mathematics students are exposed to, and consequently, how much these students build on their mathematical knowledge and confidence in mathematics. Making the right pedagogical decisions that are consistent with mathematics learning processes and that would enhance students' mathematics learning is neither easy nor precise. However, researchers believe that instructional practices that promote autonomous, active participation in mathematical practices and personal construction of mathematical knowledge (Ball, 1993; Cobb et al., 1993; Lampert, 1990) engender in students some autonomy, and foster in students the feeling of control over mathematics tasks. Other practices where teachers play the more traditional directive roles and where teachers emphasize quizzes and rely solely on the mathematics textbook are less effective because they tend to emphasize skill acquisition through drill and less student involvement in classroom activities (Lee, Smith and Croninger, 1997). A strategy that teachers usually employ to engage students in their mathematics learning is to create small groups. Students are put into small groups for mathematics learning activities either within the classroom or outside the classroom. Groupings are intended to give students the opportunity to discuss, debate, and present their own views, and to listen to views of others. These interactions within the small groups allow students to resolve cognitive conflicts and enrich their understanding of mathematical concepts. This practice is sometimes referred to as cooperative learning. A number of empirical studies have demonstrated that a cooperative learning environment enhances students' learning (see Cohen, 1994; Johnson & Johnson, 1994), -22-while competitive learning environments are less effective for disadvantaged students (Seymour, 1992, 1995; Seymour & Hewitt, 1997). An important factor in mathematics learning is the ability of students to make connections between mathematics concepts and their life experiences. Teachers try to accomplish this goal through problem-solving activities involving contemporary life experiences. Empirical support for these instructional practices comes from studies that have demonstrated that individuals' understanding of concepts in mathematics is profoundly enhanced through their participation in cultural practices such as shopping, selling candies, and packing crates (Lave and Wenger,1991; Saxe, 1991). Teachers also rely on technologies to improve their instructional practices. The electronic technology employed by teachers includes calculators and computers. The belief is that these tools can assist students to learn mathematics with deeper understanding (Dunham and Dick, 1994; Rojano 1996). Studies have demonstrated the positive impact of calculator use on students' problem-solving capabilities and performance in mathematics (Hembree and Dessart, 1986). Research also indicates that computers do enhance students' understanding of two-dimensional and three-dimensional geometry and concepts related to algorithms (Healey, 1993). However, like any tool, their impact on mathematics achievement depends on whether teachers and students make appropriate choices about the use of these technologies in the right context. Teachers and Mathematics Learning A number of studies have demonstrated the importance of teacher attributes on mathematics learning (Sanders & Rivers, 1996; Wright, Horn, & Sanders, 1997). These studies indicate that students assigned repeatedly to classrooms with ineffective teachers tend to have lower achievement than those who are assigned regularly to classrooms with effective teachers -23-(Sanders & Rivers, 1996). The studies also revealed a strong bias in the assignment of students to teachers of different effectiveness levels; disadvantaged students, such as African-American students, were more likely to be assigned to ineffective teachers (Sanders & Rivers, 1996). A number of factors, including academic ability, years of education, years of teaching experience, knowledge of subject matter, and teaching behaviors in the classroom, are related to teacher effectiveness (see Darling-Harmond, 2000). In a study of middle school mathematics teachers, where teachers were matched by years of experience and school setting, students of fully certified mathematics teachers experienced significantly larger gains in achievement than those taught by teachers not certified in mathematics. The differences in student gains were greater for algebra classes than general mathematics (Hawk, Coble, & Swanson, 1985). However, it makes sense that knowledge of the material to be taught is essential to good teaching, but also that the need for subject matter expertise would grow smaller beyond some minimal essential level which exceeds the demands of the curriculum being taught (Darling-Harmond, 2000). This interpretation is consistent with findings that teachers' content preparation, as measured by course-work in the subject field, is positively related to student achievement in mathematics and science, but that the relationship is curvilinear, with diminishing returns to student achievement when teachers' subject matter courses are above a threshold level (e.g., five courses in mathematics) (Monk, 1994). Hawkins (1998) point out that eighth grade mathematics students taught by teachers with an undergraduate or graduate major in mathematics scored higher than students taught by teachers with majors in some other subject areas. The study also indicated that mathematics achievement levels were higher for those whose teachers have certificates in mathematics and rated themselves as knowledgeable in the current teaching and learning perspectives in mathematics compared with those students whose teachers have certificates in other subject areas -24-and who were less conversant with the current teaching and learning approaches in mathematics. These findings demonstrate the importance of quality teachers for students' mathematics learning. Gender Another factor in the variation in students' mathematics achievement levels is their gender. A number of studies demonstrate that males and females differ in their mathematics achievement (see Leder, 1990 for a review). The differences tend to favor males, particularly on mathematics problems involving higher level cognitive skills (Bielinski & Davison , 1998). Other studies also indicate that gender differences in mathematics have decreased over the years (Friedman, 1989). The success in reducing these differences has been attributed to a consistent and sustained effort by researchers, educators, parents, and others to trace the sources and treat the symptoms of gender differences (NCTM, 2000). Leder (1990) grouped the sources of gender differences into two categories: factors within individuals and factors associated with the interaction of individuals with their environments. The factors within individuals include the attitude and motivation of females to learn mathematics. Females are posited to have low confidence in mathematics, consider mathematics less useful in their future endeavors, and are less motivated to learn mathematics. On the other hand, parents, peers, teachers, and society in general encourage and expect males to do well in mathematics. Males therefore tend to have a positive attitude towards mathematics, consider mathematics essential for their future careers, and are highly motivated to learn mathematics. Ma and Willms (1999) found that it is mainly attitude, not ability, affecting the decision of females to either pursue or drop courses in advanced mathematics. -25-Recent studies attempt to discern the influence of schooling processes on gender differences and whether schools can identify "interventions that will eliminate the differences" (Fennema and Carpenter, 1998). Much of these studies on gender differences has focused on the general female and male populations in a typical school or classroom (Leder and Fennema, 1990). The underlying assumption in these studies is that gender differences in mathematics are uniform across schools and classrooms. This assumption may be wrong as schools and classrooms are different in many respects, including quality of teachers, and instructional practices. Thus, one should expect "differential classroom effectiveness" by gender within classrooms. The term "classroom effectiveness" connotes classroom processes that should improve students' mathematics learning. These processes involve students' interactions and participation in mathematics classroom activities. Fennema and Peterson (1985) conducted a study of classroom effectiveness in 36 grade 4 classrooms. They examined how the mathematics achievement of females and males was related to their classroom activities, especially teachers' and students' interaction patterns. Mathematics achievement was classified into two categories: high-cognitive mathematics, and low-cognitive mathematics. The study indicates that females in classrooms with competitive classroom environments had lower scores in the low-cognitive mathematics test items than females in less competitive environments. In contrast, Stanic and Reyes (1986) find that cooperative learning environments, where students have the opportunity to receive assistance from either the teacher or other students, are likely to enhance the mathematics achievement levels of both males and females. Attempts by researchers to understand how gender differences become entrenched in schools and classrooms have also focused on differential enrollment patterns of males and -26-females in mathematics courses. The results suggest that a large proportion of the gender gap in mathematics achievement may be attributable to differential course taking (Fennema & Sherman, 1977; Pallas & Alexander, 1983). These studies show that significant gender differences in favor of males usually appear in Grades 9 or 10 when some mathematics courses are optional. Ma and Willms (1999) find that the participation rates in advanced mathematics courses are similar for males and females until the end of their junior year, but, in the transition to the final year of high school, females were more likely to cease taking advanced mathematics. There has been debate about whether differential course-work or other psycho-social variables account for a large proportion of the gender gap in mathematics achievement (Benbow & Stanley, 1980; Pallas & Alexander, 1983). Controlling for number of courses taken, Pallas and Alexander (1983) found significant gender differences in mathematics achievement indicating that factors other than course-work account for gender differences in mathematics. Another important concern of gender differences in mathematics achievement is that the magnitude and direction of the differences depends on the area of mathematics. For example, Fennema and Carpenter (1981) found that among 17-year-old students, male dominance in mathematics tends to increase with an increase in the difficulty level of mathematics problems. Other studies have found differences favoring males in mathematical concepts but favoring females in computational skills. The second and third international mathematics studies by the International Association for the Evaluation of Educational Achievement (IEA) also indicate that gender differences among educational systems depend on the domain of mathematics (Travers & Westbury, 1989; Schmidt et al, 1996). The question is whether these differences also vary among schools and whether there are school and classroom processes that can account for these variations. -27-This research study addresses three main issues concerning gender differences in mathematics: 1) whether certain mathematics domains are especially difficult for females or males; 2) whether gender difference varies among classrooms; and 3) whether classroom characteristics are associated with gender differences. The Canadian Context The administration of education in Canada is quite complex, as there is no unified national system. The provinces, through their ministries of education, control almost all aspects of education, especially those pertaining to curriculum, teacher certification, accreditation of schools, and the reporting of students' progress. These ministries work with the local school boards to implement provincial and local policies about the operation of schools, determine the extent and what curriculum to emphasize, and hire teachers and support staff. Therefore, one would expect students' learning activities within provinces to vary. However, over the years, there has been much interest among educators in the provinces to exchange information about policies and practices within their jurisdictions, with the intention of using this information to initiate policies that would improve the achievement levels of their students. The Council of Ministers of Education, Canada (CMEC), which is a body comprised of the Ministers of Education from the provinces and territories, has been instrumental in this exchange of information. The council along with Statistics Canada and Human Resources and Development Canada (HRDC) has assisted in coordinating international and national studies to assess and compare students' mathematics achievement levels. These studies include the National Longitudinal Study of Children and Youth (NLSCY), the International Adult Literacy Study (IALS), and the School Achievement Indicators Program (SAIP), (see Appendix A for a detailed description of these studies). -28-Frempong and Willms (in press) have assembled data from these studies in an attempt to determine the differences among provinces in their mathematics achievement levels. The analyses involved scaling the mathematics scores in these studies such that the achievement difference between two average students in adjacent grades is equal to a year of schooling. Their analyses indicates that at the early grade levels (grades 2 and 4) there is little variation among provinces on their achievement levels. The variation tends to widen at the higher grade levels (grades 6, 7, 8, and 11) suggesting a possible variation in the effectiveness of schools within the provinces. An important observation from their analyses is the consistently high mathematics achievement level of Quebec (QU) at the higher grade levels and across all studies. Some provinces such as Alberta (AB) and British Columbia (BC) scored relatively high in most of the studies. On the other hand the achievement levels for Newfoundland (NF), New Brunswick (NB), and Ontario (ON) were consistently low across studies (see appendix B for a summary of the findings). In a previous analysis of the N L S C Y data, Willms (1996) demonstrates that there are large and statistically significant differences among the provinces in their mathematics achievement, even after taking into account the family background of students. His analysis indicates that differences among the provinces in their adjusted scores appear as early as grade 2, where students in Ontario lagged behind the national average by about one month of schooling, and those in New Brunswick, Manitoba, Nova Scotia, British Columbia, and Quebec scored about one to four months of schooling above the national average. By the end of grade 4, the background adjusted scores for Quebec were about six months of schooling higher than the national average, and by the end of grade 6, it was nearly one full year of schooling higher. These results show that differences among provinces widen as children progress through the elementary grades, indicating that factors associated with schooling -29-processes might be responsible for the mathematics achievement differences among the provinces. Students' achievement levels are often determined by the opportunities they have to learn. The variation among provinces in their mathematics achievement levels might, therefore, be a reflection of the variation in the opportunities available for students to learn mathematics within the provinces. Although there are curriculum reform initiatives within the provinces, apparently heading in the same direction, the provinces have long traditions of and beliefs about what they perceive to be appropriate mathematical practices within their schooling systems. These traditions and beliefs are likely to influence aspects of the teaching and coverage of mathematics within the provinces. Willms's (1996) analysis of N L S C Y data further indicates a significant influence of the students' background characteristics on their mathematics scores. At the grade 2 level, the scores for girls were, on average, two months of schooling lower than boys. But at grades 4 and 6, girls outperformed boys by about one month of schooling in grade 4, and about two-and-a-half months of schooling in grade 6. These findings are consistent with an earlier study involving about 31 elementary schools in British Columbia, where differences between males and females in their mathematics computations scores at grade 3 were not statistically significant but females outpaced males in mathematics computation in later grades. The Willms's (1996) analysis further indicated the importance of the students' family background in their mathematics scores. The socioeconomic status of a student's family, especially the level of education of a student's mother, was "the strongest predictor of mathematics achievement across the three grade levels" (Willms, 1996, p.72). These findings indicate that students from poorer, less well educated families tend to perform poorly in mathematics. -30-In another analysis using IALS data, Willms (1999) demonstrates that the quantitative literacy skills among youth vary by SES levels; youth from high SES families tend to have higher quantitative skills than those from low SES families. The quantitative literacy skill test required respondents to understand mathematical operations and concepts embedded in texts. The gender differences on the quantitative literacy skills were not statistically significant. His analysis further indicates that there is a relationship between quantitative literacy skills and the education levels of the parents of youths aged 16 to 25 in Canada; youths with educated parents are likely to have high quantitative skills. This relationship varied substantively among the provinces. Willms refers to this relationship as a gradient. A steep gradient indicates a wide gap in quantitative literacy skills between youths with high educated parents and those with low educated parents. A shallow gradient indicates a small gap in quantitative skills between youth with more educated parents and those with less educated parents. Quebec and Manitoba have relatively shallow gradients, while the other provinces had relatively steep gradients indicating that quantitative skills among the youth in Quebec and Manitoba are equitably distributed compared with other provinces. The analysis further reveals that the variation among the provinces is greater at the lower levels of the parents' level of education — there is relatively little variation at the higher levels. These findings demonstrate that the provinces with high levels of quantitative skills have reached this high level by raising the quantitative skills of youth from disadvantage backgrounds. The measures of quantitative skills were also measures of mathematics skills, so, the question is whether these findings are true for school mathematics. This is an important question as educators and researchers attempt to provide quality mathematics education for all. In a review of mathematics education in Canada, Taylor (1997) observes that the major goal of mathematics educators is to understand the processes for enhancing the learning of mathematics. At the provincial level, the major objective is to provide -31-an adequate opportunity for all with the hope that Canada as a nation will have a well educated populace and competent workforce at all times. In British Columbia, for example, the goals, as listed in the provincial curriculum guide, are: the provision of mathematics to develop skills in logical analysis and to present problem solutions in a clear and precise manner, and to provide the mathematics necessary to function in society, engage in lifelong learning, and pursue further formal study in mathematically-related areas (Taylor, 1997, p.74). This desire to provide quality mathematics education to all students has precipitated initiatives in the way mathematics is taught and in the way a mathematics curriculum is organized. Mathematics Teaching and Curriculum Content A number of aspects of teaching and learning in Canadian mathematics classrooms have changed over the years. Taylor's (1997) review indicates that there is now a movement toward a more interactive and participatory learning, where students are actively engaged in the learning process as opposed to the traditional direct instruction, where students are less active in the classroom instructional environments. He suggests that the trend in mathematics teaching and learning is to develop the students' conceptual understanding of mathematical ideas with less emphasis on rote learning of procedures and rules. A direct consequence of this interactive learning is instructional practices that encourage students to interact with each other through small group and cooperative learning. In addition, projects and assignments, where students apply mathematical knowledge and procedures to issues around them, are encouraged. The belief is that by integrating mathematics into real world applications, students will view mathematics as relevant and, consequently, develop their interest and confidence in mathematics. -32-Taylor (1997) further suggests that the content of mathematics curricula is changing, with concepts previously taught at higher levels being introduced at earlier grades. In Fractions, for example, decimals are now introduced at earlier grade levels with less emphasis on common fractions. The increasing importance of data has increased the emphasis on statistics and data analyses, which have become a major part of the mathematics curricula at all levels, beginning in elementary school. Other areas, such as concept formation, estimation, and geometry, are gradually being introduced at the lower grades. These trends in mathematics education and the extent to which teachers and students adapt to such teaching environments and curriculum changes are likely to result in variations among students in their mathematics achievements and their interest and confidence in mathematics. An important goal of this research study is to determine the extent to which the variation in instructional practices and curriculum coverage affect students' mathematics achievement levels. The intention is to provide data and findings that inform discussions about characteristics of classroom environments for effective learning. Between-Schools Segregation School districts assign students to schools based on either "closed or open" enrollment policies. In closed enrollment policies, school districts allocate students to schools based on their residential locations. Students who reside in a particular neighborhood attend a particular school. In most cities, socioeconomic status determines residential location and, therefore, this policy tends to segregate students into schools based on socioeconomic backgrounds of students. In open enrollment policies, parents are allowed to enroll their child in any school. Parents with more social and economic resources in terms of power and prestige are more likely to take advantage of this policy by enrolling their child in the best schools. If this happens, then students -33-from high SES families are likely to be concentrated in the best schools while those with less resourceful parents would be enrolled in the worst schools. Other policies and programs, such as private or denominational schooling, charter or magnet schools, French immersion programs and programs for "gifted" students, also have the potential to contribute to segregation of students in schools based on socioeconomic lines. The contention is that parents of middle-class backgrounds are more likely to take advantage of these policies and programs than parents from low social-class backgrounds. Consequently, these schools would have a larger concentration of students from middle class backgrounds. Segregation of students into schools based on social-class lines needs special attention because peer interactions, and students' backgrounds and experiences play a crucial role in any learning environment, particularly when learning is conceived of as participation and sharing of ideas. While learning is enhanced when bright and motivated students are grouped together, learning is less likely to occur in environments where students are mainly from disadvantaged backgrounds and are less motivated. Unlike students from disadvantaged backgrounds where the school or classroom might be the only major source of learning, students from advantaged backgrounds can always fall back on their educational resources at home to compensate for any loss at school. Consequently, any form of segregation based on students' background characteristics or ability is likely to worsen the learning situations for disadvantaged students. Researchers have yet to document the extent to which particular policies and practices in Canada have contributed to the segregation of students along socioeconomic lines, and how this segregation has affected students' achievement. This study does not attempt to fill this research gap; however, it does describe the extent of between-classroom segregation in Canada and how this affects classroom mathematics achievement levels and the achievement levels of students from lower socioeconomic backgrounds. -34-Within-School Tracking Another controversial practice within schools in Canada is tracking. Tracking refers to practices where students in the same grade in a school take different mathematics courses based on their perceived ability. A l l students in Canada are required to take mathematics courses until grade 9 in all provinces and up to grade 11 in some provinces. The mathematics courses are the same for all students at the elementary school level. Elective mathematics courses usually begin in middle school where students may be allowed to choose courses, often with assistance from a school guidance counselor. In middle school, two mathematics courses are usually offered: academic and general (Taylor, 1997). Tracking also occurs when students in remedial classes are removed from regular classrooms. This research study examined this form of tracking on students' mathematics achievement levels. Tracking in any form is controversial. Advocates for tracking contend that the practice allows more able students to proceed faster through the school system, and thus successfully meet their expectations in concert with their ability. The opponents of tracking claim that the practice is likely to lower expectations and achievement of slower students, and limit their academic and career opportunities (Taylor, 1997). Mathematics Teachers Teachers play a central role in students' mathematics learning through their decisions about what mathematics to teach and how to teach it. These decisions are based on teachers' translations and interpretations of the intended mathematics curriculum for students in a particular province. Also, teachers' interpretation and delivery of the mathematics curriculum probably vary, depending on the characteristics of teachers, such as their experiences and specialization in teaching mathematics. These characteristics are likely to influence the -35-mathematical understanding perceived and experienced by their students. Very little research on teacher attributes and how they affect students' mathematics achievement levels has been done in Canada. TIMSS provides data for this kind of study. This research study included teacher variables in the analyses to determine how these variables effect students' mathematics achievement levels and especially the mathematics achievement of disadvantaged students. Children from Immigrant families Canada is a country of immigrants-about a quarter of a million immigrants enter Canada every year (see HRDC, 1999). According to the Human Resources and Development Canada report (HRDC, 1999) about 20 percent of these immigrants are children below the age of 12. These children come from different cultures and sometimes speak languages other than English and French, and over a third of the families of these children are poor. Children from immigrant families are, therefore, disadvantaged in school as they do not easily adapt to their new school culture. The interactions of these children with peers and teachers are also limited because of their backgrounds. The school outcomes of these children are, therefore, likely to be lower than their counterparts from non-immigrant families. Furthermore, the school systems in the provinces are quite different from each other. The provinces also differ in terms of the programs and policies aimed at easing the transition of immigrants into the Canadian cultures and schooling systems. These differences are likely to reflect the extent of the school achievement gap between immigrants and non-immigrants across the provinces. This research study examined the achievement gap between immigrants and non-immigrants in six provinces across six domains of mathematics. -36-Summary Chapter 2 has provided a summary of two perspectives of students' mathematics development. The perspectives indicate that conceiving mathematics learning as a cognitive process within the individual, as well as involving interaction and participation of individuals within learning environments, provides a useful framework for tracing and explaining the, sources of students' mathematics learning and their mathematics achievement levels. This research study examined a number of sources of variation in students' mathematics achievement levels. These sources include: students' home, gender, immigrant status, classrooms, teachers, and the province in Canada where they attend school. The examination requires complex statistical analyses. These statistical procedures are discussed in the next chapter. - 3 7 -Chapter 3 Data and Research Method This chapter describes the data, research methods, and the statistical models employed in the analyses. It first describes the source and nature of the TIMSS data, and then presents the statistical methods and models. Population and Sample This research study employed data from the Third International Mathematics and Science Study (TIMSS) - a study of classrooms across Canada and around the world. TIMSS included about 41 countries, which makes it the largest and most comprehensive comparative project to assess students' school outcomes in mathematics. The International Association for the Evaluation of Educational Achievement (IEA) coordinated TIMSS from Canada and the United States. The main objective of TIMSS was to provide data on the teaching and learning of mathematics and science in elementary, lower and upper secondary schools around the world with the hope that analyses of these data would inform teachers, educators, and policy makers about the classroom processes associated with students' mathematics and science outcomes. The framework of the study presumes that certain processes linked to curriculum and instruction have a direct relationship with the students' achievement and their attitude toward these subjects. There were three target populations: population 1 - students in adjacent grades containing a majority of 9-year-olds (grades 3 and 4 in most countries), population 2 - students in adjacent -38-grades containing a majority of 13-year-olds (grades 7 and 8 in most countries), population 3 -students in their final year of secondary schooling (grade 12 in most countries). This research study utilized the Canadian population 2 data describing the mathematics achievement levels of 13-year-old students in Canada. In Canada, these students are in grades 7 and 8 (Secondaire I and II in Quebec). Both grades are part of the secondary school system in all provinces except British Columbia, where grade 7 is part of the elementary program (Taylor, 1997). The TIMSS Canada population 2 data were collected from a random sample of Canadian schools and classrooms. The random sampling and selection were carried out by Statistics Canada and data were collected in the spring of 1995. Over 16 000 students and their teachers and principals participated in the population 2 component of the study in Canada. Students wrote achievement tests that included both multiple-choice and constructed-response items which covered a broad range of concepts in mathematics. The students also responded to questionnaires about their backgrounds, their attitudes towards mathematics, and instructional practices within their classrooms. Principals completed a school questionnaire describing school inputs and processes, and teachers responded to questionnaires about classroom processes and curriculum coverage. An important feature of TIMSS Canada is that five provinces - British Columbia (BC), Alberta (AB), Ontario (ON), New Brunswick (NB), and Newfoundland (NF) over-sampled their population such that sample sizes are sufficiently large to allow for inter-provincial comparisons. A sixth "province" - a collective group representing "Other French"—was created by isolating the students who wrote the TIMSS test in French. One will expect the majority of students in the "Other French" to come from Quebec, because there are comparatively few Francophone -39-students who wrote the TIMSS tests in French from provinces such as Saskatchewan, Manitoba, and Nova Scotia whose students' population comprised of Anglophones and Francophones. Sampling Design and Sample Weights TIMSS employed a complex sampling design. The design involved a two-staged cluster sampling whereby a random sample of schools was selected at the first stage, and classrooms within schools at the second stage. In each school, one classroom was sampled from each target grade (grades 7 and 8 for population 2). In Canada, all students in the sampled classrooms were included in the study, such that the classrooms were selected with equal probability. The school selection process entailed a stratified probability sampling proportional to the size of a school, determined by the number of students in the school. Students within schools therefore have probabilities of selection that differ from students within another'school, and therefore the probability of being selected differs among students. This variation in sampling probabilities makes it impossible to generalize to the population (grade 7 and 8 populations) without incorporating design weights. The sampling design weight is the inverse of the probability of being selected. The weights were assigned such that the sample represents the population from which it is drawn so that estimates of population characteristics will reflect the population of interest. The use of appropriate sample weights also ensures that the different subgroups constituting the sample are proportionally represented in the computation of population estimates. There are four types of sampling weights available for use with TIMSS data: student weights, school weights, student-teacher weights, and teacher weights. Each of the four types of weights are appropriate for particular assessments and questionnaire items. The student -40-weights (TOTWGT) are appropriate for any aggregation of student assessment results or questionnaire items. School weights are appropriate for any aggregation of school questionnaire items. Teacher weights, which are available only for the U . S . sample, are appropriate for any aggregation of teacher questionnaire items, or a combination of teacher and school questionnaire items (see TIMSS International Study Center, 1997). Student-teacher weights are appropriate for aggregations of combinations of teacher questionnaires and student questionnaire items. Gonzalez et al, (1997) provides details about sampling design and sample weights in TIMSS. In this study, the student and the classroom were both units of analysis. The classrooms were all selected with equal probabilities and therefore did not require any weighting. The study utilized teacher variables as well as variables describing school processes to discern how these variables influence classroom variation in mathematics achievement. Teachers, however, were not a unit of analysis. The intent of the study was to generalized to students and their classrooms, but not to grade 7 and 8 teachers. Teacher weights would therefore not be appropriate in this context (even if these weights were available in the TIMSS Canada data). The normalized students' weights were used in all analyses to ensure that the findings can be generalized to the entire Canadian grades 7 and 8 populations, as well as to the provinces that over-sampled their populations. The sum of the sampling weights for a sample is an estimate of the size of population of students, so that using the raw sample weights inflate sample size and therefore standard errors of estimates. The weights were normalized by dividing students' weights by the mean sampling weight. The sum of the normalized weights is equal to the sample size and therefore using the normalized weights ensures appropriate standard errors. The weights were normalized separately for each province so that findings can be generalized to the -41-population of the six provinces (Newfoundland, New Brunswick, Ontario, Alberta, British Columbia, Quebec). The use of weights as described above is designed for traditional statistical models. Weighting in multilevel models where two or more sampling weights are often required has not been widely investigated. Multilevel modeling is consistent with analyzing data from a complex multistage sampling design, but using weights to address differences in sampling probabilities can be problematic (Pfefferann et al. 1998). Unweighted estimates may be biased, but the weighting options in most multilevel statistical soft-wares such as H L M has not been implemented to adequately provide the desired results. Furthermore, researchers are yet to come out with reliable weights for multilevel models. Pfeffermann et al. (1998) discuss and provide means for calculating weights in multilevel models. They refer to their weighted estimates as "pseudo-maximum likelihood estimates" (PMLE) which seems to have desirable properties for models with random intercepts but can give biased estimates when "level 1 sample sizes are related to level 2 weights"(Pfeffermann et al, 1998, p.38). The authors concluded in this paper that weights in multilevel models should be used with caution. P M L E is yet to be implemented in any of the major multilevel statistical soft-wares. This study used weights following the traditional approach for multilevel models, which conceptually provides estimates that can be generalized to the populations of interest Research Method A number of issues confront researchers in mathematics education interested in classroom effects on students' mathematics achievement levels. The first issue is that the data structure required for such analyses is hierarchical: smaller units are nested within larger units. -42-In this case, students are nested within classrooms. Within classrooms, one will expect students' mathematics achievement levels to be related because they are subject to the same set of teachers and learning conditions for at least part of the day. This relationship among students is referred to as an intra-class correlation. The traditional statistical models, such as multiple regression, cannot adequately incorporate the intra-class correlation coefficient in the computation of the estimates in statistical models. Researchers using traditional statistical models would either aggregate the student-level data to the classroom, or disaggregate the classroom-level data to the student-level. In the first case, the model assumes that the intra-correlation is 0, while in the second case the assumption is that intra-class correlation is 1. In both cases the traditional statistical approach does not adequately present the complex realities of classrooms where intra-correlation may differ from 0 or 1. Furthermore, Bryk and Raudenbush (1992) demonstrate that the reliability of parameter estimates in such models is compromised when data are either aggregated or disaggregated as is done in the traditional statistical procedures. The ability of multilevel statistical models to closely represent the realities of classrooms and also provide reliable parameter estimates was a major motivation for employing this procedure. The second issue is how to model variation in gradients. In this research study, the interest was in whether gender gap and SES gradients vary from classroom to classroom. These types of models were first conceptualized by Burstein and others (see Burstein, 1980; Burstein et al., 1978). Their approach, referred to as a 'slope-as-outcome' model, characterizes regression coefficients as slopes, which are estimated at the lower level of analyses and then treated as outcomes at the next level. This approach was particularly appealing as it allowed researchers to explore the cross-level interactions or relationships -43-among variables at different levels. The approach made it possible to estimate parameters within units (classrooms) and model them as a function of between-unit variables or characteristics. But this strategy relied on traditional statistical procedures and was plagued with a number of problems including the unreliability of the estimated regression slopes, the inability of the model to distinguish between parameter and sampling variance, and the complexity of the estimation procedures for multiple slopes-as-outcomes (Bryk & Raudenbush, 1992; Raudenbush & Bryk, 1986). However, advances in statistical theories and computations, such as empirical Bayes estimation procedures, provided the needed headway for a credible and reliable approach to estimate the complex variance and covariance components in a model. In recent years, researchers have developed statistical models and computer programs for analysing hierarchically structured data under various names: variance covariance components models (Aitkin & Longford, 1986), multilevel linear models (Goldstein, 1987), and hierarchical linear models (Raudensbush & Bryk, 1986). Although there are differences in the algebraic operations of these models, they all share common properties, which ensure more accurate parameter estimates compared to the traditional approaches. Furthermore, multilevel statistical procedures allow researchers to model processes occurring within nested levels which can then be specified to indicate how the explanatory variables at a higher level of aggregation influence outcomes at a lower level and pose hypotheses about relations occurring at each level and across levels. The models also allow researchers to estimate the variance of random effects. For example, it is possible with these models to test whether SES gradients or levels of achievement are homogeneous across units of classrooms or individuals. Also, more advanced models, such as multivariate, multilevel - 4 4 -models with multiple dependent variables, are possible. A number of computer programs (e.g., GENMOD, W H L M , M L w i N , and V A R C L ) are available (see comparison of these programs by Kreft, de Leeuw, & van der Leeden, 1994). The Windows version of the hierarchical linear model (HLM) was used for all analyses in this research study. Models and Analyses Classroom effects on students' mathematics achievement levels were examined with a two-level H L M where the first level (the within-student level) estimated the parameters associated with individual characteristics, and the second level (between-classroom level) modelled the relationships between students characteristics and classroom characteristics. The approach entails the estimate of a separate regression equation for each classroom, which yields a set of intercepts (i.e., levels of outcome adjusted for students' background) and slopes (i.e., gradients). The set of intercepts and slopes become the outcome variables at the second level of the model, which are regressed on variables describing the characteristics of classrooms. Null and Baseline Model Typically, multilevel models start with a "null model", that is, a model with no independent variables. The model can be expressed in algebraic form as (Bryk and Raudenbusch, 1992): Poj = Yoo + U 0 J , -44-so that = Yoo + U 0 J + r,j5 where Y^, the mathematics score for student i in classroom j , is partitioned into p0j, the estimated classroom mean score, and r^  the deviation score for student i in classroom j . P0j is further partitioned into y00, the mean of the estimated classroom means, and U 0 J , the deviation of an estimated classroom mean from the mean of the estimated classroom means. The variance of ry is referred to as the 'within-classroom variation'-the mean square difference between students' scores and their classroom mean. The variance on U 0 j is referred to as the 'between-classroom variation', which is the mean square difference between classroom mean scores and the mean of classroom means. Thus, it is a measure of classroom effect. The above equations also indicate that the total variation on students' scores is the sum of the between-classroom variation and the within-classroom variation. This demonstrates that the null model simply partitions the total variation into between-and within-classroom variations. The proportion of the total variation among classrooms is also a measure of the intra-class correlation coefficient. Classrooms are nested within schools so the model should include school as a level. However, in TIMSS Canada, there is only one classroom sampled for each grade, which severely compromises the accuracy of school-level estimates. Therefore, the total variation is partitioned into student and classroom components. This approach is also problematic as grades 7 and 8 scores are likely to be correlated However, using data for grade 6 students in New Brunswick, Willms (1998) partitioned the total variance into students, classroom, schools, and districts components. He found that 86.5 percent of this variance was among students within classrooms, and 7 percent -45-was between classrooms. Only 4.7 percent was among schools, and 1.5 percent was among school districts. This suggests that the correlation between the scores of students in the same school but in different classes is relatively small. This research study ignored the possible correlation between the scores of grade 7 and grade 8 in the same school, but one would expect this correlation to be very small. Classroom Effects Once a null model indicates a significant variation among classrooms (U0j) on their mathematics achievement levels, one can add independent variables to the null model. At the students' level, one can conceptualize the analysis as comprising separate regression equations for each classroom which can be expressed as: Yy = Poj + P . j ( S E S ) u + P^Gender)^ + r y ; where is the adjusted mathematics score for student i in classroom j . P0j (the adjusted classroom mathematics achievement score), Py ( S E S gradient), and P 2 j (Gender gradient) are the set of regression parameters for each of the j classrooms. P,j and P 2 j are measures of the extent of inequality between students with differing S E S and Gender respectively. The S E S and Gender effects can, therefore, be conceptualized as the average within classroom effects. The regression parameters from the first set of analyses (i.e, adjusted classroom achievement levels and the gradients) can become the outcome variables. These can be regressed on classroom level variables such as whether a teacher uses small groups (GROUPING). The regression equations can be expressed as: P0 J = Yoo + Yo.CGROUPING^ + U 0 j ; -46-where Yoo is m e rnean of the adjusted classroom means adjusted for grouping, y 0 i is the effect of grouping on classroom mathematics achievement levels net the effect of students background characteristics, U 0 j is the residual on Y o o - a measure of the extent to which a classroom mathematics achievement level deviates from y0o- U 0 j can be interpreted as the 'effect' associated with each classroom, P ^ Y . o + Yi.CGROUPINGJj + Uy; where P y is the mean of the adjusted classroom SES gradients, adjusted for grouping. Y u is the effect of grouping on SES gradients, TJ,j is the residual on yi0-a measure of the extent to which a classroom SES gradient deviates from y 1 0, P2J = Y20 + Y 2.(GROUPING)j + U 2 j ; where y20 is the mean of the adjusted classroom gender gradients, Y21 is the effect of grouping on gender gradients, U2j is the residual on B2o-a measure of the extent to which a classroom gender gradient deviates from B20. The effect of GROUPING on classroom achievement levels is referred to as 'between-classroom effect'. Grade, a dummy variable (coded; grade 8=0, grade7=l) were included in the equations for all analyses to control for differences in achievement between 7 t h and 8 th grades. Multivariate, Multilevel Models Another important feature of H L M is that it allows researchers to model multiple outcomes at the lowest level, which is referred to as the multivariate multilevel model. The first level of the model is intra-individual. It employs a set of dummy variables to denote the response variable considered. In this research study there were six dummy variables representing Fractions, Geometry, Algebra, Statistics, Measurement, and Proportionality. The -47-level 1 equation has no intercept and no variance term; it simply defines the multivariate structure. The second level of the model describes relationships between outcome measures and covariates for individuals. It includes a measure of SES and two dummy variables denoting gender and immigrant status. The third level describes relationships between SES-adjusted school means and covariates, and between within-classroom gradients and covariates. An important feature of multivariate, multilevel models is that they allow for the estimate of the correlation between pairs of the outcomes variables at the different levels. In this research study, these models made it possible to estimate the correlation between pairs of the six domains of mathematics not only at the student level but also at the classroom level. In other words, the stability of the students' mathematics achievement levels and classroom mathematics achievement levels across the domains of mathematics could be estimated. The next three chapters describe three separate stages of analyses and findings employing simple multilevel models as well as multivariate multilevel models. -48-Chapter 4 Mathematics Achievement and Students' Backgrounds This chapter examines the extent to which students' background characteristics affect their achievement in mathematics classrooms. The analyses attempt to address two main research questions: 1) To what extent do provinces vary in their levels of mathematics achievement and in their gradients? 2) Are provincial differences in mathematics achievement associated with the success or failure of any particular group, such as children with low SES? Scale of Mathematics Achievement Students' overall mathematics achievement scores were re-scaled using data for the grade 7 and grade 8 sample such that the mean score for the seventh grade was equal to 7 and mean score for the eighth grade was equal to 8. This is often referred to as a grade equivalent scale. In this study, the scale is referred to as "math years" representing "years-of-schooling"; seven years for the average grade 7 student and 8 years for the average grade 8 students. In Canada, since students spend about 10 months per year in school, the scores can easily be converted into months of schooling by simply multiplying a score by 10. The scale assumes a linear growth in mathematics skills across a fairly wide range of grades, say from about grade 3 through grade 11. This is a rather liberal assumption that could not be confirmed with data to show, for example, that the mean score of Canadian grade 3 students is close to a score of 3.0. However, the scale provides a useful re-expression of the magnitude of the difference in the mathematics scores to a "years-of-schooling" metric based on TIMSS mathematics test for grades 7 and 8 students. The re-expression does not affect the -49-statistical significance of any observed differences among provinces or other groupings of students. Variables Describing Students' Backgrounds Three students' background variables were used in the analyses. The first variable is "immigrant status", which is a dichotomous variable denoting whether a student was born in Canada (coded 0) or outside Canada (coded 1). Most of the students born outside Canada are likely to come from immigrant families. Students' gender was coded one for females and zero for males. Socioeconomic status (SES) was derived from a factor analysis of students' responses to questionnaires describing level of education (coded in years of education) of their parents (mother and father), and a number of educationally-related materials they have access to at home. These materials included calculator, computer, modem, study desk, CD or video player, television, bookshelves, and books, such as dictionary and encyclopedia SES is a standardized composite score of students' responses to these questionnaires. Table 4.1 provides the descriptive statistics for the composite variables of SES. The table indicates that a student from an average SES family would have parents with about 13 years of education and about 6 out of the 8 educationally related materials at home (see first row in table). There is a positive correlation between the variables indicating that students whose fathers have more education tend to have mothers with more education. The more highly educated families also tend to have more educationally-related materials at home. The last row of Table 4.1 provides the factor loadings for the first principal component of a principal component factor analysis of the three variables. The factor loadings ranged from 0.86 for variables describing -50-father's and mother's education to 0.65 for the variable describing home possession emphasizing the relative significance of father's and mother's education as measures of SES. Table 4.1 Descriptive Statistics, Pearson Correlations, and Factor Loadings for Measures of the SES Composite Father's Education Mother's Education Home Possessions Mean 13.09 13.06 5.57 Standard Deviation 2.65 2.53 1.54 Pearson Correlation Father's Education Mother's Education Home possessions 0.64 0.35 0.33 Factor Loadings for SES Composite 0.86 0.86 0.65 Provincial Differences in Students' Backgrounds Table 4.2 presents the weighted descriptive statistics of students' background variables by province. The table indicates that about 49 percent of grade 7 and grade 8 students in Canada are females. The percentage is almost the same in the six provinces, except in BC where about 44 percent of the students are females. The table shows that about 8 percent of Canadian grade 7 and grade 8 students are from immigrant families. The percentage varies considerable among provinces: BC has the highest concentration of immigrants (about 14 percent) followed by Ontario (10 percent), Alberta (8 percent), and Quebec (6 percent). Newfoundland and New Brunswick have the lowest concentration of immigrants, 2 percent and 3 percent respectively. -51-The provinces also vary in the SES of their students. The SES mean for BC (0.27) and Alberta (0.10) were quite high compared to -0.30 for Quebec, -0.29 for Newfoundland, and -0.13 for New Brunswick. Ontario's SES mean (0.03) was close to the national average. The relative SES scores for the provinces were quite consistent with Willms (1996) (see Table 4.1, p.78) where SES scores for the provinces were measured employing data from the National Longitudinal Survey of Children and Youth (NLSCY). The table in Willms (1996) indicates Alberta, BC, and Ontario have scores above the national average, while scores for Newfoundland, New Brunswick, and Quebec are below the national average. Table 4.2 Students' Background Variables: Descriptive Statistics by Province Canada NF NB ON A B BC Q U Female (%) 49 49 49 50 49 44 51 Immigrants (%) 8 2 3 10 8 14 6 SES Student level Mean -0.03 -0.29 -0.13 0.03 0.10 0.27 -0.30 Standard Deviation 1.02 0.99 0.95 1.04 0.98 0.99 0.94 Classroom level Mean (Mean SES) -0.06 -0.30 -0.13 0.04 0.08 0.24 -0.40 Reliability (p) 0.71 0.70 0.67 0.63 0.76 0.68 0.83 Var among classrooms as % of total SES var 19.6 13.8 9.2 19.6 13.7 12.0 18.8 -52-Mean of Students' SES in a Class (Mean SES) Students' SES scores were aggregated to the classroom level by calculating the mean of students' SES in a class. This is referred to as the "Mean SES" which is a measure of the composition of the students in a classroom based on their SES backgrounds. Table 4.2 indicates that the averages of the Mean SES of classrooms for the provinces vary. As expected, provinces with a high SES also tend to have a high Mean SES. Table 4.2 also provides reliability estimates of Mean SES which were estimated using H L M . The H L M reliability estimate is a measure of how reliably one can distinguish among classrooms on their mean scores. The reliability scores ranged from 0.63 in Ontario to 0.83 in Quebec, indicating that one can distinguish among classrooms within provinces based on their mean SES. The last row of the table shows the percentage of the variance in SES among classrooms. The percentage ranged from 9.2 in New Brunswick to 19.6 in Ontario. This measure is an indicator of the extent to which classrooms are segregated in terms of students' SES backgrounds (Willms, 1986). SES segregation is more prominent in Quebec and in Ontario than in the other provinces. The extent of SES segregation in Canadian grade 7 and grade 8 classrooms is less than that found among grade 8 students in the United States (see Ho and Willms, 1996). Relationship Between Mathematics and Students' Backgrounds The next stage of the analyses employed multilevel linear models to examine the relationship between students' mathematics test scores and three predictor background variables: Socioeconomic status, gender, and immigrant status. The analyses involved a series of separate analyses for each province whereby the effects of these background variables on mathematics achievement were estimated. The contextual effect for each province was also estimated by -53-adding mean SES into the equation for estimating SES gradients. The multilevel linear model had two levels, students and classrooms, so that the estimates can be conceptualized as the average within-classroom estimates for each province. These estimates are displayed in Figures 4.1 to 4.4 and in Tables 4.3 to 4.6. These estimates are intended to provide a description of the extent to which students' background characteristics affect their mathematics achievement levels and how this effect varies among the six provinces. Figure 4.1 Gender •fferences in IVfetherrHtics Aiiieverre^ Note: * indicates statistical significance at p<0.05 -54-Figure 4.1 displays the differences between males and females in their mathematics achievement levels. The Figure indicates a male advantage in mathematics. Males scored slightly higher than females in all provinces. The differences between females and males in their mathematics achievement scores were less than three months of schooling in New Brunswick, Ontario and BC, about 2 months of schooling in Alberta, just over 1 month of schooling in Newfoundland, and less than 1 month of schooling in Quebec. The differences were statistically significant (p<0.05) only in Ontario and Alberta (see Table 4.3). Table 4.3 Estimates of Gradients by Province Gradients SES Gender Immigrant Status (Female=l, Male=0) (Immigrants=l, non-immigrants=0) Newfoundland (NF) New Brunswick (NB) Ontario (ON) Alberta (AB) British Columbia (BC) Quebec (QU) 0.75(0.09) 0.79(0.07) 0.60(0.03) 0.55(0.05) 0.55(0.10) 0.21(0.06) -0.12(0.21) -0.24(0.16) -0.30(0.06) -0.21(0.11) -0.26(0.19) -0.06(0.10) -0.92(0.39) -0.19(0.42) -0.35(0.11) -0.44(0.21) 0.56(0.28) -0.92(0.26) Note Standard errors in brackets, bold indicates statistical significance atp<0.05. -55-Figure 42 Differences in Mathematics A^evement Between Innigrants and Ncfvirrmgrants by Province 0.7 r Note: * indicates statistical significance at pO.05 Figure 4.2 displays the differences among provinces in their immigrant gradients. In all the provinces, except BC, the mathematics achievement scores of immigrants were lower than the scores for non-immigrants. In BC, the mean mathematics achievement of immigrants was -56-about 5 months of schooling higher than the mathematics achievement of non-immigrants. The high mathematics achievement of immigrant students in BC is interesting and warrants further study that should include an investigation about differences in mathematics achievement levels between sub-groups in the immigrant population. The performance of students from oriental countries, including Singapore, Japan, Korea, and Hong Kong, on the TIMSS tests was relatively high (see Robitaille, Taylor and Orpwood, 1997),. Figure 4.2 also shows that the mathematics achievement gap between immigrants and non-immigrants was particularly high in Newfoundland (over a year of schooling) and in Quebec (about 7 months of schooling), and relatively low in New Brunswick, Ontario, and Alberta. Table 4.3 indicates that the differences associated with immigrant status were statistically significant (p<0.05) in all provinces except in New Brunswick. The table also shows a statistical significant (p<0.05) effect of socioeconomic status on mathematics achievement in the six provinces. Figure 4.3 displays the provincial mathematics achievement levels for students whose SES were within the 10th and 90 t h percentiles. The figure shows Ontario with the lowest mathematics achievement score and Quebec with the highest mathematics score at all levels of SES. The mathematics score for Newfoundland was relatively low at the low levels of SES, but very high at high levels of SES. The figure also shows that there is little variation among provinces in their mathematics scores at the high levels of SES. The variation is quite large at the low levels of SES. For instance, the mathematics achievement gap between Quebec and the lowest achieving province, Ontario, is about 1.5 years of schooling at the high levels of SES. This gap increases to about 2.5 years of schooling at the low levels of SES. The most interesting finding about this variation is that provinces with high mathematics achievement levels tend to have shallow SES gradients (see Figure 4.4). -57-Figure 4.3 Relationship Between Mathematics Achievement and SES by Province - 2 - 1 - 0 1 2 Socioeconomic status Figure 4.4 Provincial Mathematics Scores 9 r and SES Gradients _ Q U O) c 1 u (/> * 7 S2 n AB , BC ON NF • NB 0.2 0.4 0.6 0.8 SES gradient Figure 4.4 shows Quebec with the highest mathematics achievement level for average SES students and the shallowest SES gradient. Newfoundland and New Brunswick have lower mathematics scores and higher SES gradients compared with those for Alberta and BC. These -58-findings suggest that provinces with high mathematics achievement levels tend to do so by raising the mathematics achievement levels of their low SES students. The immediate implication for this finding is that policies intended to boost the mathematics achievement levels for low achieving provinces should emphasize policy initiatives that would raise the mathematics achievement of their low SES students. Double Jeopardy Effect Students from low SES families are often disadvantaged in terms of learning opportunities both at home and at school. These students are further disadvantaged when the policies of a schooling system are such that students from low SES families are concentrated in certain classrooms, while those from high SES families are concentrated in other classrooms. In this study, the extent of this disadvantage is referred to as the "double jeopardy effect" and is measured by including both SES and classroom mean SES in a model. This model is intended to indicate that while there is a positive effect associated with the SES of students, there is also a positive effect associated with the mean SES of a classroom over and above the effect associated with a student's family background. Figure 4.5 illustrates the extent of the double jeopardy effect in Canadian classrooms. Only students with SES scores between the 10 th and 90 t h percentiles were included in this graph so that the mathematics achievement scores of students from the extremely poor and least educated families and those from extremely rich highly educated families are not represented in the graph. The lines in the graph represent student mathematics achievement levels based on their SES and the mean SES of their classrooms. Classrooms were classified into three categories based on their mean SES scores; scores at the 25 t h percentile and below were considered low, -59-scores between the 25' and 75' percentiles were considered average, while scores at the 75 percentile and above were considered high. 9.0 as ? • •= 8.0 15 7.5 | 7.0 6.5 F i g u r e 45 T h e R e l a t i o n s h i p B e t w e e n IVbunatHUcs S c o r e a n d S o c i o e c o n o m c s t a t u s at Different l e v e l s o f C l a s s r o o m nBan S E S L o w S E S C l a s s r o o m U g h S E S C l a s s r o o m A v e r a g e S E S C l a s s r o o m -1 0 1 Scdoecxxxrric status -60-Figure 4.5 shows students with SES between -0.5 and 0.5 could be found in any of the classrooms categorized as low, average, and high mean SES. The mathematics achievement gap between a student with -0.5 SES in a low mean SES classroom and a student with 0.5 SES in a high mean SES classroom is about a year of schooling. The gap between the student with the lowest SES score (-2.0) in a low mean SES classroom and a student with the highest SES score (about 1.5) in a high mean SES classroom is about two and half years of schooling. The extent of the double jeopardy effect varied from province to province. Figure 4.6 presents graphs indicating the extent of the double jeopardy effect across the provinces. The figure indicates that there is a large mathematics achievement gap between low SES students in low mean SES classrooms and high SES students in high mean SES classrooms in all the provinces. The mathematics achievement between a students with the lowest SES score and a student with the highest SES score in a high SES classroom is about 3 years of schooling in Newfoundland, New Brunswick, Ontario, and Alberta, and about 2 years of schooling in British Columbia, and Quebec. In Quebec and Alberta, a larger proportion of this gap is due to the mathematics achievement difference between high and low mean SES classrooms. However, in comparison with Quebec, the average SES gradient within classrooms in Alberta is relatively high. Newfoundland has a very steep SES gradient but a low mean SES effect. The SES and mean SES effects are similar in the other provinces (New Brunswick, Ontario, and B.C.). -61-F i g u r e 4.6 S t u d e n t s ' M a t h e m a t i c s A c h i e v e m e n t S c o r e s at different l e v e l s o f S E S a n d m e a n S E S o f Q a s s r o o m s N e w f o u n d l a n d Average N e w B r u n s w i c k A l b e r t a - 2 - 1 0 1 2 Sodoeconorric status - 2 - 1 0 1 2 Sodoeconorric status - 2 - 1 0 1 2 Sodoeconorric status - 2 - 1 - 0 1 2 - 2 - 1 0 1 2 - 2 - 1 - 0 1 2 Sodoeconorric status Sodoeconorric status Sodoeconorric status - 6 2 -Analyses of Explained Variance The initial analysis has revealed that, in general, immigrants, female students, and students from low SES families have low mathematics achievement in classrooms within the provinces, indicating that students background characteristics do make a difference on their mathematics achievement levels. The question is, how much of the variation on students' mathematics achievement levels within-and between-classrooms can be explained by students' background characteristics? The next stage of analyses examined the effect of students' background characteristics on the within-and between-classroom variation in students' mathematics achievement levels. The analyses involved three models. The first model partitioned the total variation on the students' mathematics achievement scores into within-and between-classrooms. The model includes grade level in the equation. One major limitation of the TIMSS Canada survey was that only one class per grade per school was sampled which confounds the within-classroom and within-school effects. Grade, a dummy variable (l=grade 7, and 0=grade 8), was included in the equation so that the total variance was partitioned into between and within classrooms rather than within schools. The intercept in the model indicates the mean mathematics achievement level for grade 8 classrooms. Background variables of the students were added into the equation in the second model. These variables were centred on their grand (Canadian) means in each of the analyses so that it is possible to compare provincial mathematics achievement levels based on variables in the equation of the model. Centring, in general, involves re-scaling a variable to define zero on the variable so that the intercept of the equation in a model can be described. For instance, in the model where 8 t h grade rather than 7 t h grade was selected as the centre (grade 8=0), and where SES = 0 denotes a student with average SES background, the intercept defines the achievement - 6 3 -level of an average SES student in a grade 8 classroom. The third model included mean SES in the equation. Tables 4.4, 4.5, and 4.6 present the results from these models. Table 4.4 presents estimates from the first model. The intercept represents the average score for grade 8 classrooms within the provinces. The expected average from the whole Canadian sample was 8 years of schooling. The table indicates that on average grade 8 classroom mathematics achievement levels in Alberta, BC, and Quebec were higher than the expected national score. The average was particularly high in Quebec, about one year of schooling higher than the national average. The averages for the other provinces (Newfoundland, New Brunswick, and Ontario) were lower than the expected national average. New Brunswick had the lowest average. Table 4.4 Variance Components and Provincial Mathematics Achievement Levels NF NB ON A B BC QU Intercept 7.18(0.18) 1.39(0.19) 1.41(0.08) 8.20(0.15) 8.13(0.19) 9.04(0.22) Grade (8=0, 7=1) -1.28(0.22) -0.11(0.26) -0.89(0.11) -1.04(0.21) -2.15(0.27) -0.9(0.30) Variance Components Between Classrooms 0.50 0.42 0.75 0.79 0.30 1.72 Within Classrooms 5.77 5.48 5.08 5.38 4.98 3.90 Percent Total Variation Between Classrooms (ICC) 8.0 7.1 12.9 12.8 5.7 30.6 Note: Standard errors in brackets, bold indicates statistical significance atp<0.05. -64-The lower portion of the table shows the variance components where the total variation of the students' mathematics achievement is partitioned into the within-and between-classroom variations. The table indicates that in all the provinces, the within classroom variation is bigger than the variation between classrooms indicating that differences among students in their mathematics achievement levels within classrooms are bigger than differences between classroom mathematics achievement levels. This means that the variation between classrooms accounts for only a small proportion of the total variation among students in their mathematics achievement levels. The between-classroom variation accounted for about 6 percent of the total variation in B C , about 8 percent in Newfoundland, about 7 percent in New Brunswick. The percentage was particularly high in Quebec (about 31 percent), slightly high in Ontario, and in Alberta, only about 13 percent. However, in all provinces the between-classroom variations were statistically significant (p<0.05) indicating that classrooms within the provinces do make a difference in students' mathematics achievement levels. The table also presents estimates of the mathematics achievement differences between grade 7 and grade 8 students (growth rate) by province. The national average growth rate was set at 1 year. The table indicates that the growth rates for Ontario, Alberta and Quebec were close to the national average. The growth rate for New Brunswick (0.82) was about 18 percent lower than the expected growth rate of 1 year. The growth rate for BC was particularly high (more than twice the expected growth rate ). This is interesting and warrants further investigations. Since in BC, grade 7 is the terminal point of their elementary school system and grade 8 is the beginning of secondary school, one plausible area for investigation would be the differences in the elementary and secondary schools in terms of content coverage. -65-Table 4.5 presents the effect of students' background variables net of the effect of grade. In this model the three background variables were added into the equation to determine how much of the variation in students' mathematics could be explained by these background variables. As expected, in all provinces, both the within-and between-classroom variations were reduced. The provinces differ, however, in the extent to which students' background characteristics explained the variation in students' mathematics achievement levels. Students' background characteristics explained over 40 percent of the variation between-classrooms in Newfoundland and in New Brunswick, about 30 percent in Ontario, Alberta, and BC, and only about 15 percent in Quebec. The background characteristics did not explain much of the within-classroom variation; about 8 percent in Newfoundland and New Brunswick, about 6 percent in Ontario, 5 percent in BC, about 4 percent in Alberta, and less than 2 percent in Quebec. These findings indicate that some of the observed differences between classrooms within provinces are due to the background characteristics of the students. Despite the adjustment, the between-classroom variation for all the provinces is statistically significant indicating that the observed differences in classroom mathematics achievement levels are not fully attributable to students' background characteristics. Adjusted Mean Scores The first row of Table 4.5 shows the estimates of the adjusted mean scores for each province. These are the expected scores for grade 8 students in a classroom with a representative mix of males and females, immigrants and non-immigrants, and of average SES backgrounds. The scores are close to the unadjusted mean (compare first rows in Tables 4.4 and 4.5). The adjusted scores for Newfoundland and New Brunswick are about 2 months of schooling higher. -66-The adjusted score for Quebec is a bit higher, while Ontario and Alberta's adjusted scores remained relatively the same. The adjusted score for B C is more than 2 months of schooling less. These findings indicate that some of the variation among the provinces in their mathematics scores can be attributed to the background characteristics of their students. Newfoundland and New Brunswick, which have relatively low SES students, fare quite well in their adjusted scores. Table 4.5 The Effect of Students' Background Characteristics on Mathematics Achievement by Province N F N B O N A B B C Q U Intercept 1.91(0.15) 1.53(0.16) 1.40(0.07) 8.15(0.13) 8.47(0.74) 9.12(0.20) Grade (8=0, 7=1) -1.30(0.19) -0.86(0.22) -0.91(0.10) -1.02(0.18) -1.98(0.23) -0.92(0.29) Immigration -0.86(0.36') -0.29(0.45) -0.23(0.11) -0.34(0.20) 0.52(0.26) -0.94(0.20) Female -0.30(0.79) -0.21(0.16) -0.36(0.06) -0.21(0.10) -0.26(0.18) -0.05(0.70) SES 0.19(0.08) 0.81(0.09) 0.62(0.03) 0.56(0.05) 0.55(0.09) 0.22(0.05) Classroom Variance Components Between 0.32 0.21 0.51 0.53 0.20 1.45 Within 5.31 5.04 4.78 5.15 4.70 3.84 Percent Variance Explained by Students' Backgrounds Between 36.0 50.0 32.0 32.9 33.3 15.7 Within 8.2 8.0 5.9 4.3 5.6 1.5 Note: Standard errors in brackets, bold indicates statistical significance atp<0.05. -67-Table 4.6 The Effect of Students' Background Characteristics, and Classroom Mean SES on Mathematics Achievement by Province NF NB ON A B BC QU Intercept 8.11(0.75) 7.67(0.75) 7.38(0.07) 8.04(0.72) 8.38(0.25) 9.66(0.27) Grade (8=0, 7=1) -1.30(0.79) -0.93(0.20) -0.93(0.09) -0.96(0.76) -1.91(0.27) -0.98(0.25) Immigration -0.89(0.40 -0.31(0.44) -0.2(0.08) -0.35(0.79) 0.52(0.26) -0.95(0.20) Female -0.30(0.72) 0.20(0.76) -0.36(0.05) -0.27(0.70) -0.26(0.7*) -0.06(0.70) SES 0.75(0.07) 0.75(0.09) 0.57(0.05) 0.5(0.06) 0.53(0.09) 0.18(0.05) MeanSES 0.49(0.25) 0.91(0.50) 0.59(0.70) 1.04(0.55) 0.26(0.55) 1.31(0.27) Classroom Variance Components Between 0.29 0.13 0.44 0.40 0.21 1.07 Within 5.30 5.03 4.78 5.15 4.69 3.84 Percent Variance Explained by Students' Backgrounds and their Mean SES Between 42.0 69.0 41.3 49.4 30.0 37.8 Within 8.2 8.2 5.9 4.3 5.8 1.5 Note: Standard error in brackets, bold indicates statistical significance atp<0.05. Contextual Effects Mean SES was added into the equation in the last model. Table 4.6 displays the effect of mean SES on students' mathematics achievement levels. The table indicates a positive significant (p<.05) effect of mean SES on mathematics achievement suggesting that students in classrooms with high concentration of high SES students tend to have a higher achievement than students in a classroom of low concentration of low SES students. The contextual effect, the mathematics achievement difference between classrooms with a unit difference in mean SES is over a year of schooling in Quebec and just about a year in Alberta and New Brunswick. The effect was relatively low in Newfoundland, Ontario, and British Columbia. The effect was not -68-statistically significant in BC. Contextual effect also explained a substantial portion of the variation among classrooms in all the provinces except BC, where there was a slight increase in variation. In New Brunswick, students' background characteristics and mean SES decreased the between-classroom variance by about 69 percent. The decrease is 42 percent in Newfoundland, 41 percent in Ontario, and about 38 percent in Quebec. The adjusted scores for Newfoundland, New Brunswick, and Quebec increased slightly, while those for Ontario, Alberta, and BC decreased slightly. Summary of Findings The first set of analysis has revealed that females, students from low SES families, and students from immigrant families have a lower achievement in mathematics than their counterparts indicating that possibly classrooms do contribute to differences in students' mathematics achievement. There is group-based inequality or gradient. These gradients vary from province to province. The provinces with high achievement levels tend to have low SES gradients. In other words, mathematics achievement is equitably distributed in provinces with high achievement levels. The analyses further demonstrated that students from low SES families are particularly disadvantaged in mathematics because of classroom segregation based on students' SES backgrounds. Students with similar SES backgrounds were more likely to acquire high mathematics achievement levels if they were in high SES mathematics classrooms so that the mathematics achievement gap between low SES students in low SES classrooms and high SES students in high SES classrooms tend to widen when students are segregated in schools because of their SES backgrounds. This gap is quite wide (over 2 years of schooling) in the six provinces. -69-The analyses also showed that classrooms vary in their mathematics achievement levels indicating that classrooms do make a difference in students' mathematics achievement. The students' background characteristics accounted for a substantial percentage of the variation among classrooms and a small percentage of the within-classroom variation. The percentage of variation among classrooms explained ranged from 50 percent in Newfoundland to about 16 percent in Quebec. This finding indicates that students' background characteristics do make a big difference in classroom mathematics achievement levels in Newfoundland, but make little difference in classroom mathematics achievement levels in Quebec. The within-classroom variation explained ranged from about 8 percent in Newfoundland and New Brunswick to less than 2 percent in Quebec. These low percentages suggest that there are a number of other students' characteristics not included in this model that are associated with within-classroom variation among students in their mathematics achievement. Further analyses with classroom mean SES in the model increased the percentage of between-classroom variation explained, but not the within classroom variation, demonstrating the importance of the composition of students in a classroom on mathematics achievement. The mathematics score was a composite of six domains of mathematics (Algebra, Fraction, Geometry, Measurement, Proportionality, and Statistics). Provincial mathematics achievement levels may not be stable across these domains, especially if the mathematic test tapped certain domains of mathematics which were not emphasized in classrooms within the provinces. If this is true, then the observed gradients and provincial differences may be domain-specific. The next set of analysis involved complex multivariate multilevel models where the scores for the six domains of mathematics were used as outcomes. -70-Chapter 5 The Stability of Achievement Across Domains of Mathematics The principal objective of the second set of analyses was to determine whether gradients and levels of mathematics achievement varied across the domains of mathematics, and whether contextual effects depended upon the domain of mathematics. The analyses employed a multivariate, multilevel statistical model whereby the six domains of mathematics were used simultaneously as mathematics outcomes. Mathematics Domains Students in this study were examined in six domains of mathematics: Algebra, Fractions, Geometry, Measurement, Proportionality, and Statistics. The Algebra questions tested for a basic understanding of algebraic expressions, linear equations, and patterns and relationships. Students were expected to use algebraic notations and their understanding of concepts of Algebra to solve mathematical and practical real-world problems. In Fractions, the items covered basic skills and concepts in arithmetic as well as number sense that required students to use their intuition about numbers and their ability to use various strategies involving mental computations to solve problems. Questions in this domain demanded that students demonstrate an understanding of the properties and operations of numbers. The questions also assessed students' understanding of numerical relationships, their ability to reason mathematically, and their ability to communicate the reasoning they used to solve problems involving number sense, properties, and operations. The questions classified as Geometry tested students' understanding of geometric figures and their properties. Students were tested on their understanding of concepts related to properties of -71-angles and polygons, such as symmetry, congruence, and similarity. The Measurement questions assessed students' conceptual and procedural understanding of measurement units, the ability to use measurement tools, and the ability to solve problems related to perimeter, area, and volume. In Proportionality, students' understanding of the relationships among quantities involving multiplicative factors was required to correctly answer the questions. The Statistics questions involved an understanding of data representations, analysis of data, and probability. The TIMSS mathematics tests included 150 items distributed among the domains. Table 5.1 displays the distribution of test items by domain. Seventy-five percent of the items were multiple-choice questions, and 25 percent were open-ended questions. Table 5.1 Distribution of the TIMSS Test Items by Mathematics Achievement Domain Domain Number of items % of total items Algebra 27 18 Fractions 51 34 Geometry 23 15 Measurement 18 12 Proportionality 11 7 Statistics (Data Representation, Analysis 20 14 and Probability) The table indicates that the TIMSS test items for grade 7 and grade 8 students emphasized Fractions. There were 51 questions related to Fractions and Number Sense which -72-formed about 34 percent of the total number of items. A considerable percentage of the items also covered Algebra, Geometry, Measurement, and Statistics. Proportionality was the least emphasized in the test. There were only 11 items covering Proportionality which constituted 7 percent of the total items. Geometry covered about 15 percent of the total test items, Measurement covered 12 percent, while about 14 percent of the items were categorized as Statistics. Scaling of Mathematics Domains The percentage correct in each domain was computed for each student. The scores were scaled such that for each domain, the mean score for grade 7 was 7 and the mean score for grade 8 was 8. This scale represents "years-of-schooling", seven years for the average grade 7 student and 8 years for the average grade 8 students. As discussed in Chapter 4 (refer to scale of mathematics achievement scores), the scale presumes a linear growth in mathematics skills across a fairly wide range of grades. This assumption, however, could not be supported with data beyond that for grades 7 and 8. Therefore, the scale is just another re-expression of the magnitude of the difference in the mathematics scores to a "years-of-schooling" metric based on TIMSS mathematics test for grade 7 and 8 students. Statistical Models and Analysis The analysis employed multivariate, multilevel statistical models. The models had two main characteristics: multiple students' outcomes (the 6 domains of mathematics), and the nested data structure (students within classrooms). The data had a 3-level structure: the six outcome variables nested within students, and students nested within classrooms. The equation for the -73-first level (the null model) consisted of a student's mathematics score with a set of dummy variables identifying the six domains of mathematics. This equation simply partitioned a student's mathematics score into the six domains of mathematics. The students' level variables were at level 2, while the third level contained classroom level variables. An important feature of multivariate, multilevel models is the ability of such models to estimate covariance or correlation among random intercepts and gradients at higher levels. For instance, in this analysis, the models allowed for the estimate of the correlation among the six mathematics domains at the student level as well as at the classroom level. The correlations at the students' level are the parameter estimates of the within-classroom correlations. These estimates measure the "stability" of students' mathematics achievement across mathematics domains and address the question of whether students with high achievement in one domain tend to have high achievement in the other domains. The correlations at the classroom level are the parameter estimates of the between-classroom correlations. These estimates indicate the stability of classroom mathematics achievement levels across the mathematics domains; that is, it ascertains whether classrooms that are effective in one domain are effective in another. Table 5.2 presents the findings of the parameter estimates of the within-and between-classroom correlations for both unadjusted and adjusted scores. The within-classroom estimate is the mean of the correlation between pairs of domain scores for students within a classroom, while the between-classroom estimate is the relationship between classroom achievement levels across mathematics domains. The within-classroom estimates are shown in the upper correlation matrix. -74-Table 5.2 Parameter Estimates of Within- and Between-Classroom Correlations (upper correlation matrix represents the within-classroom correlation, lower correlation matrix represents the between-classroom correlation) Algebra Fractions Geometry Measures Proportion Statistics Unadjusted Classroom Means Algebra 0.41 0.30 0.35 0.31 0.30 Fractions 0.93 0.33 0.44 0.31 0.45 Geometry 0.86 0.89 0.28 0.29 0.27 Measures 0.93 0.98 0.91 0.34 0.30 Proportion 0.91 0.99 0.84 0.95 0.22 Statistics 0.94 0.96 0.90 0.96 0.95 Means Adjusted for Grade and Students' Background Characteristics Algebra 0.40 0.29 0.34 0.30 0.28 Fractions 0.92 0.31 0.42 0.29 0.43 Geometry 0.84 0.88 0.26 0.28 0.25 Measures 0.90 0.97 0.90 0.33 0.28 Proportion 0.87 0.99 0.81 0.93 0.20 Statistics 0.93 0.95 0.89 0.95 0.93 Means Adjusted for Grade, Students' Background Characteristics, and Mean SES Algebra 0.40 0.29 0.34 0.30 0.28 Fractions 0.91 0.31 0.42 0.29 0.43 Geometry 0.83 0.89 0.26 0.28 0.25 Measures 0.90 0.97 0.91 0.33 0.28 Proportion 0.87 0.98 0.81 0.93 0.20 Statistics 0.92 0.95 0.89 0.95 0.93 -75-Table 5.2 also shows a low positive correlations between pairs of scores indicating that while in general students' with high achievement scores in one domain tend to have high achievement scores in the other domains, this trend is not consistently true for most students. For instance, the correlation between proportionality and Statistics is 0.22. This indicates that the positive relationship between Proportionality and Statistics is true for only four percent of the grades 7 and 8 student populations. The correlation matrix remained relatively unchanged when scores were adjusted for students' background characteristics (see the upper correlation matrix in Table 5.2). These correlations were consistently positive between pairs of domain scores. The between-classroom estimates are shown in the lower correlation matrix in the table. The table demonstrates a strong positive correlation (ranging from 0.81 to 0.99) between domains indicating that classrooms with high achievement levels in one domain tend to have high achievement in the other domains of mathematics. The correlations remained relatively stable with the adjustment of scores for students' background characteristics. Table 5.3 Pearson Product Moment Inter-correlation Matrix; Provincial Achievement Levels (the lower correlation matrix is adjustedfor SES) Algebra Fractions Geometry Measures Proportion Statistics Algebra 0.96 0.96 0.96 0.99 0.99 Fractions 0.99 0.91 0.96 0.99 0.98 Geometry 0.95 0.94 0.93 0.96 0.98 Measures 0.99 0.98 0.93 0.96 0.99 Proportion 0.97 0.97 0.90 0.98 0.99 Statistics 0.99 0.98 0.95 0.99 0.98 The next analyses employed a series of multivariate, multilevel models for each province. The first model included only a dummy variable grade 7 (7=1, 8=0) so that the intercept -76-represents the provincial average for grade 8 students. The Pearson Moment Correlations between pairs of domain scores are represented in the upper correlation matrix in Table 5.3. The subsequent analyses involved models where the background variables (immigrant status, gender, and SES) were added separately into the equations. The lower correlation matrix in Table 5.3 is the pairs of correlations for provincial scores adjusted for SES. The table demonstrates that provinces with high achievement in one domain tend to have high achievement in another domain indicating the stability of provincial achievement levels across domains. The correlation coefficients remained relatively the same with the adjustment of domain scores for students' background characteristics. The analyses also included a model for the entire Canadian sample where both SES and mean SES were in the equation such that contextual and double jeopardy effects could be estimated. The estimates of gradients (SES, gender, and immigrant status) in each mathematics domain for each of the six provinces as well as estimates of contextual effects for each domain are displayed in Figures 5.1 to 5.4. Immigrant status gradient Figure 5.1 shows the relationship between immigrant status and the six domains of mathematics. The figure indicates that, with the exception of New Brunswick and B.C., students from immigrant families tend to have low achievement levels in all domains. In New Brunswick and B.C, non-immigrants' achievement scores in Algebra, Geometry, and Measurement were higher than those of immigrants. Students from immigrant families in Newfoundland had very low scores (about a year of schooling below non-immigrants) in all six domains except in Measurement. The achievement gap between immigrants and non-immigrants in Quebec was quite high in Fractions, Geometry, Measurement, and Proportionality. Table 5.4, however, shows that the effect of immigrant status on achievement although large in some provinces such Newfoundland, was not statistically significant (p<0.05). This is probably due the small number of immigrant students within classrooms in the provinces (see Table 4.2). -77-Figure 5.1 Relationship Between Irrrnigrant status and Achievement in Mathematics Domains (Note:lrrrnigrant status coded; irnrnigrants=1, norvimmgrants=0) -78-Figure 5.2 Gender Differences in Mathematics Achievement by Domains Algebra Female Advantage NF ON BCOJ Measurement Female Advantage |NB BC Male Advantage 0.1 Fraction Female Advantage Male Advantage Proportionality Female Advantage -1.4 BC Male Advantage Geometry 0.1 | - 0 . 4 JC *5 -0.9 I -1.4 0.1 NB FemaleAdvantage NF Male Advantage O) c O -0.9 QU |£ I -1.4 Statistics Female Advantage BC_ NF ON AB QU NB Male Advantage Gender Differences Figure 5.2 displays the relationship between gender and the six domains of mathematics. In general, male achievement scores tend to be higher than those of females. The male advantage is conspicuous in Fractions, Measurement, and Proportionality, particularly in Proportionality where the difference is as high as about 13 months of schooling in -79-Newfoundland. The difference is about 10 months of schooling in Ontario, 9 months of schooling in BC and about 7 months of schooling in Alberta. In Measurement, male performance was about 7 months of schooling above that of females in Newfoundland, but about 4 months in Quebec. The difference is about 6 months of schooling in BC, 4 months in Ontario, and 3 months in Alberta, and about 5 months in New Brunswick. Most of these differences were not statistically significant at p<0.05 (see Table 5.4). Table 5.4 Estimates of Gradients by Mathematics Achievement Domain by Province NF N B ON A B BC Q U SES Gradient Algebra 0.63(0.12) 0.60(0.09) 0.48(0.03) 0.41(0.06) 0.44(0.09) 0.18(0.08) Fraction 1.34(0.29) 1.48(0.12) 0.97(0.05) 0.83(0.10) 0.46(0.16) 0.35(0.09) Measurement 1.17(0.21) 0.95(0.13) 0.81(0.05) 0.59(0.10) 0.84(0.19) 0.31(0.10) Proportionality 1.42(0.50) 1.50(0.22) 0.92(0.08) 1.02(0.13) 0.93(0.27) 0.57(0.17) Statistics 1.13(0.31) 1.21(0.16) 0.93(0.06) 0.95(0.11) 0.67(0.20) 0.29(0.09) Geometry 0.77(0.18) 0.86(0.12) 0.57(0.04) 0.54(0.08) 0.45(0.16) 0.20(0.08) Gender Gradient Algebra 0.01(0.40) -0.03(0.17) 0.02(0.06) -0.08(0.11) 0.04(0.26) 0.05(0.12) Fraction -0.20(0.23) -0.40(0.26) -0.21(0.10) -0.27(0.17) -0.25(0.32) -0.45(0.15) Measurement -0.67(0.24) -0.49(0.25) -0.43(0.10) -0.33(0.18) -0.65(0.30) -0.38(0.20) Proportionality -1.28(0.35) -0.95(0.41) -0.99(0.18) -0.78(0.27) -1.04(0.52) -0.89(0.29) Statistics -0.01(0.22) -0.54(0.29) -0.23(0.11) -0.12(0.20) 0.07(0.38) -0.17(0.17) Geometry -0.26(0.20) 0.09(0.19) -0.00(0.08) -0.07(0.16) 0.03(0.30) -0.06(0.14) Immigrant Status Gradient Algebra -1.06(0.41) 0.14(0.51) -0.21(0.10) -0.21(0.21) 0.70(0.24) -0.12(0.29) Fraction -1.09(0.70) -0.65(0.98) -0.76(0.18) -0.70(0.17) 0.65(0.46) -1.35(0.43) Measurement 0.05(0.68) 0.24(0.95) -0.69(0.17) -1.11(0.32) 0.36(0.48) -0.80(0.51) Proportionality -1.20(1.55) 0.72(1.34) -0.65(0.31) 0.04(0.55) -0.17(0.80) -1.03(1.46) Statistics -1.01(0.69) 1.02(0.88) -1.14(0.20) -1.16(0.38) -0.26(0.51) -0.79(0.42) Geometry -1.52(0.74) -0.43(0.87) -0.61(0.17) -0.14(0.30) 0.61(0.40) -1.01(0.38) *Note Standard Errors in brackets, bold indicates statistical significance atp<0.05. -80-Figure 5.3 Relationship Between Socioeconomic status and a Achievement in Mathematics Domain - 2 - 1 - 0 1 2 - 2 - 1 0 1 2 - 2 - 1 - 0 1 2 Socioeconomic status Socioeconomic status Socioeconomic status -81-SES Gradients Figure 5.3 displays the relationship between socioeconomic status and the six domains of mathematics by province. The figure indicates that students from Quebec seem to perform better at all levels of socioeconomic status and in each of the six domains. Quebec also appears to have the shallowest gradients among the provinces in all six domains. The gradient was steeper in Proportionality than in any other domain indicating the achievement gap between students from low SES background and high SES background is more pronounced in Proportionality than any other domain. The steepness of the gradients ranged from as high as about one-and-a-half years of schooling in New Brunswick and Newfoundland, about a year of schooling in Ontario, Alberta, and B.C and about six months of schooling in Quebec. There is a consistent pattern where New Brunswick and Newfoundland seem to have the steepest gradients followed closely by Ontario, Alberta, and BC, and then Quebec in all six domains. At the lowest levels of SES, students from New Brunswick, Newfoundland, and Ontario consistently perform worse in all domains. In all provinces the shallowest gradient was in Algebra. The range was about four months of schooling in Quebec to seven months of schooling in New Brunswick and Newfoundland. The figure also indicates that the biggest differences among the provinces on their achievement scores in the six domains seem to occur at the lower levels of SES. At the higher levels of SES the variation among the provinces in their achievement levels is narrower. This pattern is consistent with earlier findings where the total mathematics score was used as outcome. The SES gradients in the six domains and in all provinces, unlike most of the gender and immigrant status gradients, were all statistically significant, indicating the importance of the relationship between home background and students' mathematics achievement. Double Jeopardy Effect The analyses further indicated statistically significant contextual effects; that is, a positive relation between mean SES and the domains of mathematics. Figure 5.4 displays the estimates of contextual effects for each of the six domains. The figure indicates that students from average SES families (SES=0) can be found in high SES classrooms, in average SES classrooms, as well as in low SES classrooms. In each of the six domains, the predicted scores for average SES -82-students in high SES classrooms were higher than those in low SES classrooms. Figure 5.4 Relationship Between Mean SES and Achievement Level in a Mathematics Domain 10 _c o o £ O (0 o !2 (0 >-Algebra -1.5 -0.0 1.5 Socioeconomic status 10 o> I 9 o S 8 I' re >- 6 Fraction High Average -1.5 -0.0 1.5 Socioeconomic status 10 £ g O o « 8 (O O 7 2> re 2 6 Geometry ,verage -1.5 -0.0 1.5 Socioeconomic status 10 O) : ! 9 o o S 8 (O o 7 !2 re ^ 6 Measurement High Average -1.5 -0.0 1.5 Socioeconomic status 0)10 c 1 9 W 8 o !2 7 m a> >" 6 Proportionality High, Average -1.5 -0.0 1.5 Socioeconomic status 10 O) .£ 9 o o 5 8 O 7 !2 re ^ 6 Statistics High, Average -1.5 -0.0 1.5 Socioeconomic status The achievement gap between an average SES student in a high SES classroom and an average SES student in a low SES classroom is about a year of schooling in Fractions, about 6 months of schooling in Measurement, Proportionality, and Statistics, and about 4 months of schooling in Algebra and Geometry. The SES gradients in Fractions, Measurement, Proportionality, and Statistics were quite steep such that the double jeopardy effects were very high in these domains. The achievement gap between a low SES student (SES—1.5) in a low SES classroom and a high SES student (SES=1.5) in a high SES classroom is over 3 years of -83-schooling in Fractions, Proportionality, and Statistics, and about two and half years of schooling in Measurement. The gap is just over a year of schooling in Algebra and Geometry. These findings demonstrate the magnitude of the relevance of the students' SES backgrounds, and the composition of students in classrooms on students' mathematics achievement levels. Analysis of Explained Variance The next analysis examined the extent to which students' background characteristics affect the variation in mathematics achievement levels between-and within-classrooms, and differences in achievement levels among the six provinces. The analysis involved three multivariate, multilevel models for each province. The first model included a dummy variable grade (grade 8=0, grade7=l) so that the intercepts are the average grade 8 mathematics achievement level in a domain for each province. The students' background variable (immigrant status, gender, and SES) were added into the equation in model 2. Classroom mean SES was added to the equation in model 3. Table 5.5 presents estimates for the first model. The scores for the domains were scaled such that the expected average grade 8 score for the whole Canadian sample is 8 years of schooling. The table indicates the scores for Quebec in the six domains were consistently higher than the scores for the other provinces (see the first row of table). The score for Quebec in Geometry was over 2 years of schooling higher than the expected national average. In the other domains, grade 8 students in Quebec scored about a year of schooling higher than the national average. Also the scores for grade 8 students in B.C were above the national average. B.C's scores in Fractions, Proportionality, and Statistics were about a year of schooling higher than the national average. The scores for Newfoundland, New Brunswick, and Ontario were below the expected national average. -84-Table 5.5 Domain Achievement levels and their Variance Components by Province NF NB ON A B B C Q U Intercept Algebra 1.91(0.14) 1.44(0.19) 1.41(0.07) 8.12(0.14) 8.43(0.7*) 9.00(0.77) Fractions 1.52(0.24) 1.39(0.32) 1.09(0.12) 8.32(0.19) 8.99(0.2*) 9.41(0.26) Geometry 131(0.22) 6.63(0.26) 1.28(0.10) 1.82(0.17) 8.42(0.55) 10.10(0.27; Measurement 1.41(0.24) 1.42(0.23) 1.38(0.10) 8.41(0.20) 8.60(0.27) 9.11(0.24; Proportionality 1.83(0.32) 1.26(0.43) 1.39(0.14) 8.42(0.28) 9.02(0.52) 9.38(0.55; Statistics 1.49(0.23) 1.01(0.35) 1.20(0.12) 8.26(0.23) 8.11(0.30) 9.66(0.27) Grade (8=0, 7=1) Algebra -1.23(0.20) -0.61(0.27) -0.18(0.10) -0.89(0.19) -1.56(0.26) -1.26(0.25; Fractions -1.11(0.33) -0.68(0.45) -0.92(0.17) -1.15(0.28) -2.21(0.40) -0.42(0.55; Geometry -0.81(0.31) -0.38(0.56) -0.96(0.14) -1.31(0.25) -1.19(0.50) -0.59(0.57; Measurement -1.00(0.33) -1.08(0.55; -1.00(0.15) -1.21(0.28) -1.82(0.39) -0.63(0.55) Proportionality -1.91(0.45) -0.84(0.67; -0.98(0.20) -1.31(0.39) -1.95(0.75) -1.18(0.50; Statistics -1.21(0.32) -0.96(0.50) -0.96(0.16) -0.94(0.33) -2.10(0.43) -1.10(0.39) Variance Components: Within Classroom Algebra 6.39 5.86 6.40 6.30 6.51 5.77 Fractions 16.88 17.35 16.24 15.64 14.69 9.92 Geometry 14.47 13.69 13.53 13.67 12.07 10.04 Measurement 18.38 18.19 18.01 17.94 17.63 15.49 Proportionality 44.17 45.87 44.03 42.44 44.23 41.37 Statistics 21.94 22.66 21.37 20.61 22.75 13.11 Between Classroom Algebra 0.43 0.43 0.68 0.64 0.21 1.00 Fractions 1.17 1.19 1.97 1.24 0.53 2.39 Geometry 1.03 0.65 1.31 0.86 1.37 1.42 Measurement 1.13 0.23 1.21 1.10 0.34 1.76 Proportionality 1.61 1.53 1.74 1.77 2.27 3.26 Statistics 0.80 1.32 1.52 1.71 0.35 2.42 Percentage of total variance between classrooms (Intra-class correlation coefficient, ICC) Algebra 6.3 6.8 9.6 9.2 3.1 14.8 Fractions 6.5 6.4 10.8 7.3 3.5 19.4 Geometry 6.6 4.5 8.8 5.9 10.2 12.4 Measurement 5.8 1.2 6.3 5.8 1.2 10.2 Proportionality 3.5 3.2 3.8 4.0 4.9 7.3 Statistics 3.5 5.5 6.6 7.7 1.5 15.6 Note: Standard error in brackets, bold indicates statistical significance atp<0.05 The expected differences in achievement gap between grade 7 and grade 8 (growth rate) was 1.0 for all the domains. In general, the growth rates in the six domains for all the provinces except B.C were close to the expected score. The BC growth rate was consistently high in the six -85-domains. The growth rate was about 15 months of schooling for Algebra, and about 2 years of schooling for the other domains. The growth rate was particularly low in Fractions for Quebec (about 4 months of schooling) and for New Brunswick (about 7 months of schooling). The table also shows that the within-classroom variance for the six domains in the six provinces was lower than the between-classroom variance, indicating that, for the six domains, a large proportion of the total variation was within classrooms. The last section of Table 5.5 displays the percentage of total variation between classrooms. The table indicates that the percentage ranges from as low as 1.2 in Measurement for New Brunswick and B.C to as high as 19.4 in Fractions for Quebec. The between-classroom variation in Measurement for New Brunswick and for BC and the between-classroom variation in Statistics for BC were not statistically significant (p < 0.05) indicating that classrooms within these provinces cannot be easily distinguished on their achievement level in the two domains. In general, the proportion of the total variation among students in their mathematics achievement attributable to differences in between-classroom mathematics achievement levels was quite low in all domains. Table 5.6 Reliability of the Between Classroom Variance Estimates NF NB ON A B BC Q U Algebra 0.56 Fractions 0.56 Geometry 0.57 Measurement 0.54 Proportionality 0.41 Statistics 0.41 0.60 0.64 0.68 0.40 0.79 0.59 0.67 0.63 0.42 0.83 0.50 0.62 0.57 0.69 0.75 0.22 0.54 0.57 0.29 0.71 0.41 0.42 0.47 0.51 0.64 0.55 0.55 0.64 0.24 0.80 The reliabilities of the between-classroom variances are displayed in Table 5.6. These reliability estimates are measures of the extent to which one can reliably distinguish between classrooms in their mathematics achievement levels. The estimates are quite low in all provinces, except in Algebra (0.79), Fractions (0.83), and Statistics (0.80) for Quebec. As expected, the reliability estimates for Measurement in New Brunswick and BC, and for Statistics -86-in B C were quite low (less than .3). Table 5.7 Relationship between Domain Specific Outcomes and SES, Gender, Immigrant status, and Grade by Province NF NB ON A B B C QU Intercept Algebra 8.02(0.13) 1.55(0.17) 1.41(0.06) 8.10(0.13) 8.24(0.17) 9.08(0.17) Fractions 1.84(0.19) 1.51(0.27) 1.10(0.10) 8.24(0.18) 8.19(0.26) 9.51(0.24) Geometry 1.49(0.21) 6.13(0.24) 1.29(0.09) 1.18(0.16) 8.24(0.33) 10.15(0.20) Measurement 1.15(0.21) 1.59(0.22) 1.39(0.09) 8.41(0.18) 8.31(0.24) 9.21(0.23) Proportionality 8.12(0.30) 1.54(0.40) 1.41(0.13) 8.35(0.26) 8.15(0.49) 9.51(0.34) Statistics 1.16(0.20) 1.28(0.31) 1.21(0.10) 8.11(0.20) 8.58(0.27) 9.16(0.26) Grade (8=0, 7=1) Algebra -1.21(0.18) -0.13(0.24) -0.80(0.09) -0.88(0.18) -1.50(0.23) -1.21(0.24) Fractions -1.13(0.26) -0.82(0.37) -0.96(0.14) -1.13(0.25) -2.22(0.36) -0.44(0.34) Geometry -0.85(0.28) -0.45(0.30) -0.98(0.13) -1.29(0.23) -1.13(0.47) -0.61(0.28) Measurement -0.96(0.29) -1.18(0.30) -1.03(0.13) -1.21(0.26) -1.18(0.34) -0.64(0.33) Proportionality -1.85(0.41) -1.01(0.55) -1.02(0.19) -1.35(0.36) -1.96(0.68) -1.20(0.48) Statistics -1.18(0.26) -1.10(0.43) -0.99(0.14) -0.92(0.29) -2.08(0.38) -1.12(0.36) Immigrant status Algebra -1.09(0.41) 0.11(0.46) -O.W(O.IO) -0.11(0.20) 0.60(0.28) -0.20(0.24) Fractions -1.11(0.66) -0.84(0.77) -0.56(0.16) -0.60(0.31) 0.52(0.42) -1.32(0.32) Geometry -1.51(0.63) -0.49(0.70) -0.50(0.15) -0.10(0.30) 0.53(0.39) -1.00(0.31) Measurement -0.05(0.70) 0.10(0.80) -0.52(0.16) -1.06(0.34) 0.14(0.45) -0.80(0.39) Proportionality -1.28(1.09) 0.45(1.28) -0.46(0.25) 0.15(0.52) 0.38(0.73) -1.01(0.62) Statistics -1.06(0.26) 0.86(0.90) -0.96(0.18) -1.03(0.36) 0.41(0.52) 0.83(0.37) Gender (male=0, female=l) Algebra -0.05(0.13) 0.04(0.16) -0.01(0.06) -0.10(0.11) 0.04(0.20) 0.01(0.11) Fractions -0.34(0.20) -0.28(0.27) -0.26(0.09) -031(0.17) -0.21(0.30) -0.44(0.15) Geometry -0.34(0.19) 0.16(0.25) -0.04(0.08) -0.10(0.16) 0.03(0.28) -0.05(0.15) Measurement -0.19(0.21) -0.40(0.29) -0.48(0.09) -0.36(0.18) -0.65(0.32) -0.36(0.18) Proportionality -1.42(0.33) -0.81(0.46) -1.04(0.15) -0.83(0.28) -1.04(0.53) -0.85(0.29) Statistics -0.11(0.23) -0.40(0.32) -0.28(0.10) -0.11(0.19) 0.03(0.37) -0.15(0.17) S E S Algebra 0.64(0.07) 0.62(0.09) 0.52(0.03) 0.42(0.06) 0.46(0.10) 0.21(0.06) Fractions 1.31(0.11) 1.49(0.15) 1.03(0.05) 0.83(0.09) 0.49(0.16) 0.34(0.08) Geometry 0.80(0.10) 0.88(0.14) 0.61(0.04) 0.55(0.09) 0.46(0.15) 0.20(0.08) Measurement 1.20(0.11) 0.98(0.15) 0.86(0.05) 0.59(0.10) 0.88(0.17) 0.29(0.10) Proportionality 1.49(0.18) 1.50(0.25) 0.96(0.07) 1.04(0.15) 0.95(0.28) 0.51(0.16) Statistics 1.16(0.12) 1.22(0.18) 0.91(0.05) 0.94(0.10) 0.16(0.20) 0.30(0.09) Note: Standard errors in brackets, bold indicates statistical significance atp<0.05. -87-Table 5.7 presents estimates for model 2 where students' background variables were included in the equation. The table indicates a significant SES effect on mathematics achievement across the domains in all provinces. A large number of the gender and immigrant effects were not statistically significant. However, some effects were quite large (see, for example, immigrant effects for Newfoundland) indicating that the lack of evidence of the existence of this effect in the population is probably due to the small number of immigrants in classrooms within the provinces. The relative achievement levels of the provinces remained the same, that is, high achievement levels for Quebec, and low achievement levels for New Brunswick and Ontario. Adjusted Mean Scores As expected the adjusted scores for Newfoundland, New Brunswick, and Quebec in all domains increased while the scores for Alberta, and BC decreased. The adjusted scores for Ontario remained relatively the same in the six domains (compare the first six columns in Tables 5.6 and 5.7). However, the changes were not big enough to alter the relative positions in provincial achievement levels. This explains why the students' background characteristics had little influence on the stability of provincial achievement levels. Domain-Specific Contextual Effects Table 5.8 shows the effect of mean SES on achievement. The effect is quite high in the six domains for Quebec; about a year of schooling in Algebra, and close to 2 years of schooling in Measurement. The effects were statistically significant in all domains in Ontario and in Alberta. The effect was statistically significant for Fractions in all provinces. In all other -88-domains, one or more provinces indicated a statistically non significant mean SES effect. Table 5.8 Relationship between Domain Specific Outcomes and SES, Gender, Immigrant status. Grade and Classroom Mean SES NF NB ON AB BC QU Intercept Algebra Fractions Geometry Measurement Proportionality Statistics 8.12(0.75; 8.12(0.20> 7.65(0.25; 7.94(0.25; 8.38(0.55; 7.96(0.27; 1.69(0.18) 1X1(0.26) 6.81(0.25) 1.12(0.23) 1.81(0.41) 7.55(0.57; 1.41(0.06) 7.09(0.70; 1.29(0.09) 1.38(0.09) 7.40(0.75; 7.21(0.70; 7.99(0.72; 8.11(0.77; 1.68(0.16) 8.29(0.77; 8.23(0.26) 8.01(0.79; 8.11(0.18) 8.50(0.26) 1.90(0.35) 8.06(0.26) 8.11(0.50) 8.42(0.29) 9.54(0.17) 10.18(0.25) 10.10(0.20) 9.89(0.23) 10.29(0.36) 10.39(0.27) Grade (8=0, 7=1) Algebra Fractions Geometry Measurement Proportionality Statistics -1.21(0.18) -1.11(0.24) -0.84(0.28) -.95(0.28) -1.84(0.40) -1.11(0.25) -0.81(0.23) -0.96(0.35) -0.53(0.33) -1.25(0.30) -1.16(0.53) -1.26(0.41) -0.81(0.09) -0.98(0.75; -0.99(0.75; -1.04(0.75; -1.03(0.79) -1.00(0.74; -0.83(0.767 -1.07(0.25; -1.24(0.22) -1.21(0.24) -1.30(0.55) -0.85(0.26; -1.47(0.25) -2.14(0.54; -1.63(0.44; -1.73(0.53; -1.81(0.65; -2.04(0.37; -1.32(0.27; -0.52(0.30) -0.68(0.25) -0.12(0.28) -1.29(0.45) -1.19(0.33) Immigrant status Algebra Fractions Geometry Measurement Proportionality Statistics -1.10(0.41) -1.22(0.66) -1.58(0.65; -0.01(0.69) -1.31(1.09) -1.09(0.77) 0.10(0.46) -0.86(0.77; -0.51(0.70; 0.09(0.50) 0.43(1.28) 0.83(0.90) -0.10(0.70) -0.52(0.76; -0.48(0.75; -0.51(0.76; -0.45(0.26) -0.94(0.18) -0.11(0.20) -0.61(0.32) -0.10(0.34) -1.06(0.34) 0.14(0.52) -1.04(0.56) 0.60(0.25) 0.50(0.42) 0.51(0.39) 0.10(0.45) 0.42(0.73) 0.44(0.57; -0.20(0.24) -1.33(0.32) -1.00(0.57; -0.80(0.55) -1.08(0.57) 0.83(0.37) Gender (male=0, female=l) Algebra Fractions Geometry Measurement Proportionality Statistics -0.05(0.73; -0.32(0.20) -0.34(0.79) -0.19(0.21) -1.41(0.35; -0.10(0.25; 0.03(0.76) -0.30(0.27) 0.15(0.25; -0.41(0.29; -0.83(0.46) -0.42(0.52) -0.01(0.06) -0.26(0.09; -0.04(0.05; -0.48(0.09; -1.05(0.75) -0.28(0.70; -0.10(0.77; -0.32(0.77) -0.10(0.76) -0.36(0.75; -0.83(0.25; -0.18(0.79) 0.05(0.20; -0.24(0.50) 0.06(0.25) -0.61(0.52) -0.91(0.52) 0.05(0.37) 0.01(0.11) -0.44(0.75) -0.06(0.75; -0.37(0.75) -0.87(0.29; -0.16(0.77) SES Algebra Fractions Geometry Measurement Proportionality Statistics 0.62(0.07) 1.21(0.72; 0.75(0.77; 1.14(0.72) 1.41(0.79; 0.58(0.09) 1.38(0.76; 0.80(0.74; 0.91(0.16) 1.39(0.26) 0.48(0.03) 0.92(0.05; 0.56(0.05; 0.81(0.05; 0.93(0.05; 0.36(0.06; 0.76(0.70; 0.50(0.09) 0.53(0.70; 0.99(0.16) 0.44(0.77; 0.38(0.77) 0.38(0.76) 0.14(0.18) 0.14(0.30) 0.11(0.07) 0.29(0.09) 0.14(0.09) 0.19(0.77) 0.56(0.29; 1.04(0.74) 1.12(0.75) 0.92(0.06) 0.86(0.11) 0.69(0.21) 0.26(0.10) Classroom mean SES Algebra Fractions Geometry Measurement Proportionality Statistics 0.33(0.22; 0.80(0.54; 0.52(0.09; 0.94(0.21) 0.25(0.50) 1.18(0.24; 1.11(0.50) 0.51(0.54; 0.64(0.54) 0.11(0.50) 0.80(0.52) 1.49(0.52) 0.88(0.50) 0.83(0.46) 1.61(0.57; 1.60(0.67; 1.15(0.74; 0.11(0.14) 0.66(0.74) 0.75(0.27; 0.75(0.75) 1.19(0.50) 0.93(0.25) 1.06(0.32) 1.11(0.46) 1.47(0.54; 1.07(0.44) 1.18(0.54; 0.95(0.44) 2.10(0.57; 0.61(0.50) 1.71(0.54; 1.43(0.29) 1.83(0.55; 1.78(0.55; 1.61(0.55) ./Vote; Standard errors in brackets, bold indicates statistical significance atp<0.05. -89-Table 5.9 Variance Components and Percentage of Variance Explained by Students' Background Characteristics NF NB ON AB BC QU Variance Components for intercepts with only Grade, SES, Gender, Immigrant status in model Within Classroom Algebra 6.07 5.59 6.21 6.19 6.30 5.74 Fractions 15.68 15.76 15.50 15.11 14.52 9.76 Geometry 13.96 13.13 13.25 13.47 11.92 9.99 Measurement 17.24 17.40 17.40 17.62 17.01 15.41 Proportionality 42.11 44.09 43.02 41.48 43.57 40.84 Statistics 21.13 21.61 20.57 19.95 22.43 13.05 Between Classroom Algebra 0.31 0.30 0.50 0.49 0.13 0.86 Fractions 0.47 0.60 1.21 0.84 0.32 1.96 Geometry 0.81 0.48 1.02 0.65 1.14 1.17 Measurement 0.69 0.09 0.79 0.81 0.09 1.48 Proportionality 1.02 0.93 1.30 1.33 1.57 2.81 Statistics 0.51 0.82 0.97 1.11 0.09 2.07 Variance Explained (percent) Within Classroom Algebra 5.0 4.6 3.0 6.5 3.2 0.5 Fractions 7.1 9.2 4.6 3.4 1.2 1.6 Geometry 3.5 4.1 2.1 1.5 1.2 0.5 Measurement 6.2 4.3 3.4 1.8 3.5 0.5 Proportionality 4.7 3.9 2.3 2.3 1.5 1.3 Statistics 3.7 4.6 3.7 3.2 1.4 0.5 Between Classroom Algebra 23.3 30.2 26.5 23.4 38.0 14.0 Fractions 59.8 49.6 38.6 32.3 39.0 18.0 Geometry 21.4 26.2 22.1 24.4 16.8 17.6 Measurement 38.9 60.9 34.7 20.9 73.6 15.9 Proportionality 36.6 39.2 25.3 24.9 30.8 13.8 Statistics 36.3 37.9 36.2 35.1 74.2 14.5 Bold indicates statistical significance atp<0.05. Table 5.9 presents the variance components and the percentage of variance explained by the students' background variables. The upper portion of the table shows that across the six domains and in most provinces there is a significant variation between classrooms, even after controlling for students' background characteristics. The exceptions are in New Brunswick and -90-in B.C. In New Brunswick, the between classroom variance was not significant in Measurement. While in B.C., there was significant variation only in Geometry and Proportionality. The bottom portion of the table demonstrates that the students' background characteristics explained a higher percentage of the between classroom variance than the within classroom variance. The percentage of the within classroom variance explained ranged from less that a percentage in Quebec to about 9 percent in Fractions for New Brunswick. The students' background characteristics explained a larger proportion of the between-classroom variance in Fractions than in the other domains. Table 5.10 displays the variance components and the percentage of variance explained by students' background variables and mean SES. The between-classroom variance was statistically significant across all domains indicating that factors other than students' backgrounds and mean SES can explain the differences among classrooms within province across the domains. In BC the classroom mathematics achievement levels were quite uniform in all domains except in Geometry and Proportionality. The within-classroom variance for all domains across provinces remained relatively the same indicating that mean SES had little or no effect on within-classroom variance. On the other hand the between-classroom variance was reduced in all domains across provinces. The reduction was quite substantial in Quebec. In Algebra for instance, background characteristics and mean SES explained about 41 percent of the between-classroom variance compared to 14 percent explained by only background variables. -91-Table 5.10 Variance Components and Variation Explained by Students' background Characteristics and their Classroom mean SES NF NB ON A B B C Q U Variance Components for intercepts with only Grade, SES, Gender, Immigrant status, and mean SES in model Within Classroom Algebra 6.07 5.58 6.21 6.19 6.30 5.75 Fractions 15.60 15.71 15.11 15.11 14.48 9.92 Geometry 13.97 13.10 13.25 13.47 11.93 10.04 Measurement 17.23 17.37 17.40 17.62 16.93 15.49 Proportionality 42.09 44.06 43.00 41.48 43.49 41.37 Statistics 21.11 21.62 20.57 19.94 22.44 13.11 Between Classroom Algebra 0.25 0.25 0.44 0.36 0.12 0.59 Fractions 0.36 0.42 0.98 0.63 0.19 1.38 Geometry 0.77 0.45 0.92 0.52 0.92 0.81 Measurement 0.66 0.05 0.71 0.61 0.05 0.89 Proportionality 0.97 0.69 1.21 1.14 1.02 2.23 Statistics 0.19 0.56 0.86 0.79 0.03 1.89 Percent variance explained students' background characteristics and their classroom mean SES Within Classroom Algebra 5.0 4.8 3.0 1.7 3.2 0.5 Fractions 7.6 9.5 4.7 3.4 1.4 1.5 Geometry 3.5 4.3 2.1 1.5 1.1 0.6 Measurement 6.3 4.5 3.4 1.8 4.0 0.6 Proportionality 4.7 3.9 2.3 2.3 1.9 1.3 Statistics 3.8 4.6 3.7 3.3 1.4 0.5 Between Classroom Algebra 30.2 41.9 35.3 43.8 42.9 41.0 Fractions 69.2 64.7 50.3 49.2 64.2 42.5 Geometry 25.2 30.8 29.8 39.5 32.8 43.0 Measurement 41.6 78.3 41.3 44.5 85.3 49.4 Proportionality 39.8 54.9 30.5 35.6 55.1 31.6 Statistics 76.3 57.6 43.4 53.8 91.4 21.9 Bold indicates statistical significance atp<0.05. -92-Summary of Findings The analyses indicated that students' mathematics achievement levels were not stable across domains; students with high achievement scores in a domain did not necessarily have high achievement scores in another domain. This means that some students may find for example, Geometry easy but Algebra difficult. This is likely to happen if students do not easily transfer their knowledge from one domain to another perhaps because of differences in their conceptual understanding of these domains, or because of differences in the way these domains are taught. There were no variables in the TIMSS to test these hypotheses. On the contrary, classroom and provincial mathematics achievement levels remained relatively stable across the six domains of mathematics; a classroom or province with a high achievement level in one domain tended to have a high achievement level in another domain. Quebec had the highest achievement levels across the six domains, followed by Alberta and B.C. The achievement levels for Newfoundland, New Brunswick, and Ontario were relatively low across the six domains. Quebec's advantage was very prominent in Geometry. The relative provincial achievement levels remained stable even after adjusting for students' background characteristics. The analyses further indicated that the provinces differed on their gradients and the gradients varied across the six domains. The relatively low achievement of students from immigrant families was particularly visible in Newfoundland and in Quebec across the six domains. In B.C., students from immigrant families had slightly higher achievement levels than their counterparts in Algebra, Fractions, and Geometry. In Statistics and Proportionality, immigrants have an advantage in New Brunswick. Gender differences in mathematics favored males particularly in Measurement and Proportionality. SES gradients were steeper in Fractions -93-and Proportionality. The provinces varied in their SES gradients across the domains. Quebec had the shallowest SES gradients across the six domains. Provinces with shallow gradients tended to have high achievement levels. Another interesting finding was the importance of the classroom SES composition in mathematics achievement levels. Classrooms with high concentration of low SES students tend to have low achievement levels across mathematics domains so that students from low SES families in low SES classrooms are particularly disadvantaged in mathematics. There is a large achievement gap between low SES students in low SES classrooms and high SES students in high SES classrooms. The gap is greater than two years of schooling across the six domains. The analyses further indicated significant variation between classrooms in the provinces across the six domains, especially in Fractions. Students' background characteristics explained a substantial percentage of the variation between classrooms within provinces on their achievement levels in the six domains. The percentages ranged from as low as 13.8% in Proportionality for Quebec to as high as 73.6% in statistics for B.C. There is significant variation in domain outcomes between classrooms within provinces, even after controlling for students' background characteristics, suggesting that other factors associated with classroom and school processes may be responsible for the variation between classrooms. The students' background characteristics explained only a small proportion of the variation within classrooms in all mathematics domains, meaning that factors other than students' background characteristics are responsible for the within-classroom variation on students' mathematics achievement levels. These findings raised a number of questions pertaining to the sources of variation among classrooms and among provinces. The third set of analysis involved models to understand how -94-variables, such as curriculum coverage, classroom instructional practices, and other school process variables, affect mathematics achievement levels. The analyses employed students' mathematics scores in Fractions as outcomes. -95-Chapter 6 Provincial Differences in Mathematics Achievement: What Matters? The first and second analyses indicated that students from low SES families are particularly disadvantaged in mathematics, especially when they are in classrooms with a high concentration of low SES students. The analyses have also shown that the six provinces differ in their mathematics achievement levels, and that the relative achievement levels are stable across the domains of mathematics. Furthermore, mathematics achievement is equitably distributed in provinces with high achievement levels. These findings suggest that discerning the variation among provinces in their achievement levels requires not only an understanding of the processes associated with achievement levels, but also an understanding of the processes associated with the variation in socioeconomic gradients. The third set of analyses had two principal objectives: to determine school and classroom processes associated with excellence and equity in mathematics, and to examine and explain the differences among provinces in their mathematics achievement levels. Research indicates that children's homes and their schools are two important learning environments, so that test scores should generally reflect the effect of students' home backgrounds as well as the effect of schooling processes. Therefore, i f one major objective of this analysis is to understand the effect of schooling processes on both achievement levels and gradients, then one needs test items that are sensitive to schooling processes and that also reflect the effect of students' background characteristics. A relatively large proportion of the TIMSS mathematics test items examined students' understanding of Fractions. Furthermore, the SES effect on Fractions and the between-classroom variation in Fractions is relatively large (see -96-Tables 5.4 and 5.6), so that school and classroom processes are more likely to be sensitive to achievement levels in Fractions than in the other domains. For these reasons, Fractions was used as an outcome variable in the third set of analysis. Description of the Variables Associated with Schooling Processes Curriculum Coverage Teachers responded to questionnaires about the coverage of 37 curriculum content topics in mathematics pertaining to the domains of mathematics in which students were examined. The aim was to determine whether students within a particular classroom have been exposed to a certain mathematics curriculum concept during the school year and during the previous year. Two variables (coverage, and pre-coverage) were derived from teachers' responses to these questionnaires. Coverage was defined as the percentage of the 37 curriculum topics students were exposed to in the school year, and pre-coverage as the percentage of the 37 curriculum topics students were exposed to the previous year. Table 6.1 presents a description of the extent of curriculum coverage by domain across the six provinces. Table 6.1 indicates that on average, teachers covered about 66 percent of the concepts. 19 percent of these concepts were covered in the previous year. There is substantial variation in coverage across the domains. Teachers covered a large percentage of the concepts in Fractions (89 percent) and Measurement (78 percent). Less than 50 percent of the concepts in Algebra, Proportionality, and Statistics were covered. About 30 percent of the concepts in Fractions, and 26 percent of the concepts in Measurement were covered in the previous year, compared to less than 10 percent of the concepts in Algebra, Proportionality, and Statistics. Within the provinces, curriculum coverage followed a similar pattern whereby a large percentage of the concepts in -97-Fractions, and Measurement were covered but a smaller percentage of the concepts in Algebra, Proportionality, and Statistics were covered. The coverage of Fractions was over 80 percent in all of the provinces. The coverage of Geometry was close to 50 percent in most provinces except in Quebec where the coverage was over 70 percent. The analysis indicates that students in all the provinces were given more exposure to the concepts in Fractions than the concepts in the other domains. Table 6.1 Percentage of Curriculum Coverage by Province Canada NF NB ON A B BC QU Coverage (%) A l l domains 66 64 67 67 66 67 61 Algebra 35 23 43 37 37 52 24 Fractions 89 90 89 89 92- 88 81 Geometry 57 55 53 58 45' 48 71 Measurement 78 78 82 79 79 61 76 Proportionality 42 41 42 40 48 51 37 Statistics 39 44 40 41 24 48 36 Pre-coverage (%) A l l domains 19 15 26 18 23 10 23 Algebra 04 02 09 04 08 04 02 Fractions 30 26 42 28 35 18 36 Geometry 13 11 13 15 14 01 15 Measurement 26 17 37 25 31 11 30 Proportionality 00 00 00 00 00 00 00 Statistics 08 05 10 08 05 06 15 Instructional Practices and other School Processes Students responded to a wide range of questions in the questionnaire about instructional activities within their mathematics classroom. Table 6.2 presents a description of these instructional practices and other school process variables. -98-Table 6.2 Description of Classroom and School Process Variables Variables/Items Description Questionnaire items Classroom Instructional Practices (Coding: 0=never, l=once in a while, 2=often, 3-always) Grouping Work in pairs or in small groups in class. Work together in small groups on problems or project Problem-solving Show how to do mathematics problems. Mathematics problems with everyday life things. Explain rules and definitions. Discuss practical mathematical problems related to real life. Try to solve examples related to a new topic. Traditional Copy notes from the board Worksheets or test books Look at text while teacher talks Technology Computer Use computers Calculator Use calculators Assessment Quiz Have a quiz or test Homework Teacher gives homework School Processes Math Period Proportion of a teacher's total scheduled classroom teaching allocated for mathematics. Pupil-Teacher-Ratio (PTR) Total number of students per teacher in a school Remedial Tracking Coding (l=Yes,0 = No) Remedial teaching, students withdrawn from regular class School Disciplinary Problems Coded (0 = never, 1.5 = once or twice. 3.5 = 3-4times, 6 = 5or more). My thing got stolen Another student got hurt Things of a friend got stolen My friend got hurt by a student -99-The classroom instructional practices were categorized as grouping, problem solving, traditional, technology, and assessment. "Grouping" is the extent to which students work in pairs or small groups during mathematics lessons or on projects. "Problem solving" is the extent and nature of problem-solving activities students are exposed to in a mathematics classroom. The problem-solving activities included giving students problems involving practical and everyday life experiences. "Traditional" practice is the extent to which teachers use the traditional teaching practices such as copying notes from the board, working from textbooks or worksheets, and relying extensively on textbooks. "Technology" is a description of the extent to which calculators and computers are used in mathematics classrooms. "Assessment" includes quizzes and homework. The school process variables are math period, pupil-teacher-ratio (PTR), remedial tracking, and school disciplinary problems. "Math period" was constructed by dividing the total number of periods a teacher is scheduled to teach mathematics by the total number of periods allocated to that same teacher. This variable served as a proxy for a teacher's specialization in mathematics teaching. Given the challenge mathematics teaching poses to a number of teachers, one will expect that teachers who spend relatively more time teaching mathematics are likely to specialize in this field. There may however, be cases where teachers are assigned to teach mathematics because there are no qualified mathematics teachers. "PTR" is the total number of students per teacher in a school. The PTR variable was constructed by dividing the total school enrollment by the full-time teacher equivalent (FTE) of a school. "Remedial Tracking" is a dummy variable denoting whether in a particular school, students in remedial classes are removed from regular classes. "School disciplinary problems" measured the extent of -100-disciplinary problems, such as stealing, in a school. Table 6.3 provides the descriptive statistics of these variables. Table 6.3 Descriptives of Classroom and School Process Variables by Province Canada NF NB ON A B B C Q U Classroom Instructional Practices (Coding: 0= =never, 1: =once in a while, 2=often, 3= =always) Traditional mean 2.10 2.19 1.99 2.10 2.09 2.06 2.11 S.D. 0.26 0.19 0.21 0.27 0.27 0.31 0.24 Problem-solving mean 2.05 2.08 1.96 1.96 1.97 1.99 1.99 S.D. 0.22 0.19 0.23 0.22 0.22 0.28 0.20 Grouping mean 1.13 1.24 0.96 1.19 0.95 1.06 1.06 S.D. 0.52 0.50 0.61 0.48 0.56 0.54 0.59 Assessment Quiz mean 1.95 2.02 1.96 1.95 1.87 2.01 1.95 S.D. 0.31 0.26 0.22 0.34 0.27 0.31 0.24 Homework mean 2.43 2.46 2.57 2.43 2.57 2.42 2.21 S.D. 0.51 0.38 0.37 0.48 0.44 0.68 0.73 Technology Computer mean 0.38 0.50 0.24 0.44 0.26 0.21 0.30 S.D 0.43 0.48 0.34 0.44 0.39 0.28 0.42 Calculator mean 1.72 1.94 1.40 1.62 1.78 2.01 2.01 S.D 0.70 0.62 0.69 0.66 0.74 0.69 0.62 School Processes Maths Periods (Ratio) 0.58 0.56 0.64 0.48 0.68 0.62 0.91 Pupil-teacher-ratio (PTR) 21.10 21.27 21.70 22.43 20.22 21.10 15.04 Remedial Tracking (percent) 79 76 83 85 68 90 46 School Disciplinary Problems mean 1.00 0.86 0.97 1.06 0.96 1.18 0.82 S.D. 0.43 0.41 0.32 0.47 0.31 0.37 0.36 -101-Variables describing instructional practices were based on students' perceptions of the extent of the described instructional activities in their classrooms. The original coding in which l=ALMOST ALWAYS, 2=PRETTY OFTEN, 3=ONCE IN A WHILE, 4=NEVER was re-coded as (4=0) (3=1) (2=2) (1=3), so that higher numbers represent frequent occurrence of a particular instructional practice in a classroom. The mean of these constructs at the classroom level is a measure of the magnitude of these instructional activities in a particular classroom. Table 6.3 shows that teachers employ traditional and problem-solving instructional practices quite often, but only once in a while do they put students into groups for instructional purposes or for project assignments. Students are assessed quite often through quizes and homework. Calculators are used quite often in mathematics classrooms, but computers are almost never used for mathematics instructional purposes. The pattern is similar in the six provinces; more traditional and problem-solving practices, less grouping, more assessment and calculator usage, but less use of computers. The descriptive statistics of the school process variables are in the bottom row of Table 6.3. The table indicates that, on average, mathematics teachers are scheduled to teach more mathematics than other subjects—about 60 percent of teachers' scheduled teaching periods are allocated for mathematics teaching, there are about 21 students per teacher in schools, students in remedial classes are removed from regular mathematics classrooms in 79 percent of schools, and school disciplinary problems are rare (the coding for school disciplinary problems employed the same coding as for instructional practices). The school process variables vary substantively among the provinces. On average, 90 percent of the scheduled teachers' teaching periods in Quebec are allocated to mathematics compared to a national average of 58 percent and only 48 -102-percent in Ontario. The average is 68 percent in Alberta, 64 percent in new Brunswick, 62 percent in B.C, and 56 percent in Newfoundland. This suggests that there are more specialized mathematics teachers in Quebec than in the other provinces. The pupil-teacher-ratio in Quebec is quite low, 15 students per teacher, compared to a national average of 21. Less than 50 percent of the schools in Quebec track their students in remedial classes compared to a national average of 79 percent and 90 percent in BC. Over 80 percent of schools in New Brunswick and Ontario track their students. There is remedial tracking in 76 percent of the schools in Newfoundland and 68 percent of the schools in Alberta. School disciplinary problems are quite low in all provinces. The descriptive analyses demonstrate that the provinces are similar in their curriculum coverage and their instructional practices, but quite different in other schooling processes such as remedial tracking. The next analysis involved multilevel models to assess the effect of classroom and school-level variables on mathematics achievement, and how this affects the differences among provinces in their mathematics achievement levels. Statistical Models and Analyses The analysis involved 4 models. The first model included only grade (7=1, 8=0) in the equation. This was used as the baseline model to estimate the extent to which classrooms make a difference in students' mathematics achievement. Background variables (immigrant status, gender, and SES) were added into the equation in the second model. In the third model mean SES was added into the equation. The last model included grade, the student background variables, and variables describing curriculum coverage, instructional practices, and school processes. The model also included a dummy variable (miscoverage) denoting whether a classroom had missing data on the curriculum coverage variable. -103-Table 6.4 Parameter Estimates of the Models Model 1 Model 2 Model 3 Model 4 Intercept Gradient SES Gender Intercept 7.71(0.70) Immigrant status Gender SES Mean SES Grade -1.01(0.74) Curriculum miscoverage Coverage Previous coverage Instructional practices Traditional Grouping Problem-solving Technology Calculator Computer Assessment Homework Quiz School processes PTR PTRsquared Remedial Tracking Maths Period (ratio) Discipline Problems 7.77(0.09) 7.94(0.09) -0.59(0.75) -0.59(0.75) -0.37(0.0*) -0.37(0.0*) 0.86(0.04) 0.81(0.04) 0.71(0.74) -1.04(0.73) -1.05(0.75) 7.47(0.09) -0.57(0.70) -0.32(0.08) 0.80(0.04) 0.80(0.74) 0.63(0.72) -0.55(0.75) -0.06(0.74) 0.14(0.54) 0.06(0.77) 0.34(0.06) -0.25(0.06) 0.24(0.06) -0.03(0.06) 0.01(0.04) -0.03(0.04) 0.06(0.04) 0.07(0.04) -0.32(0.08) -0.26(0.72) -0.09(0.09) 0.02(0.76) -0.10(0.06) 0.11(0.04) 0.09(0.09) 0.00(0.09) -0.02(0.07) 0.10(0.75) -0.01(0.05) 0.05(0.09) 0.09(0.05) 0.04(0.05) 0.03(0.076) -0.04(0.07) 0.02(0.07) 0.002(0.00) -0.31(0.75) -0.03(0.70) 0.05(0.79) 0.75(0.79) -0.28(0.75) -0.21(0.24) -0.50(0.06) — — Note: Standard errors in brackets, bold indicates a statistical significance atp<0.05. -104-There were a large number of missing data on the variable describing curriculum coverage. The inclusion of miscoverage in the model allowed for the replacement of missing data on coverage with its mean, without biasing the curriculum coverage effect (Cohen & Cohen, 1983). An initial analysis indicated significant variation among classroom, in their gender and SES gradients, so both gradients were allowed to vary from classroom to classroom in the analysis. Table 6.4 presents estimates of the models. Grade Effect Table 6.4 indicates that the average grade 8 achievement score is about a year higher than the average grade 7 achievement score. Grade is a dummy variable (grade 8=0, grade 7=1) so that the intercept in the equation is the weighted national mean for grade 8. At the student level the weighted mean score is 8, because the data were standardized such that the average grade 7 student had a mean of 7 and the average grade 8 student had a mean of 8. The estimated national average from model 1 is 7.71 years of schooling which is somewhat lower than the overall student mean indicating that the average score for most grade 8 classrooms was lower than 8 years of schooling. Effects Associated with Students' Backgrounds The estimates from the second model indicate that immigrant students, females, and low SES students have relatively low achievement scores in Fractions. These estimates are the average of the within-classroom estimates. The negative coefficient for immigrant status (-0.59) indicates that, on average, immigrants scored about 6 months of schooling lower than non-immigrants. Gender was coded with female equal to one and male equal to zero so that the negative coefficient (-0.37) indicates that, on average, females scored about 4 months of -105-schooling lower than males. The average coefficient for SES is 0.86. This indicates that for each unit difference in SES between two typical students, there is a difference of about 9 months of schooling in favor of the student from the higher SES family. The average within-classroom gradient for SES remained the same (0.80) in models 3 and 4, but changed slightly for the gender gradient (from -0.37 to -0.32, a decrease of 14 percent). Mean SES Effects Estimates from model 3 show a positive effect (0.71) of mean SES on classroom achievement levels, indicating that students' achievement levels in Fractions are higher in classrooms with a higher concentration of high SES students. The effects indicate that if two typical students of similar background characteristics take mathematics lessons from classrooms with a unit difference in mean SES, the student in the higher mean SES classroom is likely to have about 7 months of schooling higher achievement in Fractions than the other students in the lower mean SES classrooms. The mean SES effect reduced from 0.71 in model 3 to 0.63 (about 11 percent) in model 4. This indicates that some of the variables that were introduced into the last model are correlated with mean SES. The last two columns of Table 6.4 display estimates of the last model. Curriculum Coverage Effects The results in Table 6.4 indicate an effect (0.14) of curriculum coverage. This effect was not statistically significant (P>0.05) indicating that the small difference in achievement (about 1 month of schooling) between classrooms with full coverage of the concepts in Fractions and -106-those with no coverage in Fractions is probably due to chance. The model included other variables describing classroom and school processes. Effects Associated with Instructional Practices Significant positive effects in Fractions are associated with the frequency of homework assignments and increased use of calculators. This means that Achievement scores tend to increase with frequent assignment of homework and increase use of calculators. The coefficients for traditional instructional practices and computer use are negative, indicating that achievement scores decline with increase use of textbooks and computers. Computer use in mathematics teaching is a recent innovation in mathematics teaching practices which poses challenges to both teachers and students. Teachers need to understand the complexity of their use for teaching purposes and students need to view computers as a learning tool. Grouping and problem-solving instructional practices had no effect on classroom achievement levels. However, grouping had a positive effect on SES gradients, indicating that the SES gradient is steeper in classrooms where students are put into small groups for instructional practices. This finding suggests that students from low SES families do not benefit from this type of instructional practice. Effects Associated with Variables Describing Other School Process Significant negative effects on achievement are associated with pupil-teacher-ratio, with remedial tracking, and with school disciplinary problems. This means that classroom achievement levels are higher in schools with low pupil-teacher-ratio, in schools where remedial students are not removed from regular classrooms, and in schools where there are fewer disciplinary problems. -107-Table 6.4 also shows a significant positive effect associated with teacher specialization in mathematics, indicating that classroom mathematics achievement levels are high in classrooms where teachers specialized in the teaching of mathematics. The SES gradients in these classrooms are less steep (about 3 months of schooling less) indicating that students from low SES families tend to do well in classrooms where teachers are specialized in mathematics teaching. Pupil-teacher-ratio had a positive effect on SES gradients and a positive effect on the gender gradient. This means that low SES students and females who attend schools with relatively low pupil-teacher-ratio tend to have higher achievement levels than their counterparts in schools with high student-teacher-ratio. A decrease in student-teacher-ratio by 10 is likely to increase the average classroom achievement level by 4 months of schooling, decrease the average within-classroom SES gradient by 2 months of schooling, and reduce the achievement gap between males and females by 3 months of schooling. Analyses of Variance Components Table 6.5 presents the variance components for the intercept, SES gradient, and gender gradient for the four models. The table demonstrates that students' background characteristics accounted for about 15 percent of the between-classroom variation and about 6 percent of the within-classroom variation. Mean SES and student background characteristics explained about 16 percent of between classroom variation, while curriculum coverage, and the classroom and school process variables, along with students' background characteristics and mean SES accounted for over 39 percent of the between-classroom variation. Curriculum coverage and the other schooling process variables reduced the between-classroom variance from 1.86 to 1.31 (about 30 percent). The between-classroom variation in the gradients did not change much across -108-the models: the variation in the SES gradient reduced slightly from 0.30 to 0.28, while, the variation of the gender gradient increased slightly from 0.96 to 0.98. Table 6.5 Analysis of Variance Components and Parameter Correlation Estimates Model 1 Model 2 Model 3 Model 4 Variance Components Between Classrooms Intercept Gradient SES Gender Within-classrooms Intercept 2.37 15.3 1.93 0.30 0.96 14.26 1.86 0.29 0.97 14.25 1.31 0.28 0.98 14.22 Percent Variance Explained Intercept Between Classrooms Within-classrooms Gradient SES Gender 14.5 6.3 16.4 6.3 0.0 39.1 6.3 25.0 Between Intercept and SES gradient Intercept and gender gradient Parameter Correlation Estimates -0.25 0.04 -0.28 0.05 -0.26 0.01 Note: Bold indicates statistical significance at p<0.05. Table 6.5 reveals that the between classroom variation on the intercept and gradients were all statistically significant (p<0.05) in the four models indicating that the variables in these -109-models could not explain all the variation among classrooms in neither their achievement levels nor in their SES and gender gradients The last two rows of the table show the parameter correlation estimates between the intercepts and gradients for the last three models. They indicate a moderate negative correlation (-0.25 to -0.28) between the intercept and SES gradient, and a weak positive correlation (0.01 to 0.05) between intercept and gender gradient. These findings suggest that classrooms with high achievement levels in Fractions tend to have shallow gradients, that is, achievement is equitably distributed in classrooms with high achievement levels. Figure 6.1 (Relationship Between Adjusted Classroom Achievement Scores in Fraction and their SES gradients by Province i i i i i i i i i i i i i -0.1 0.4 0.9 1.4 1.9 2.4 2.9 SES gradient -no-Figure 6.1 presents a scatter plots of estimates of within-classroom SES gradients and their achievement levels. The classrooms are differentiated by province. The regression line for each province indicates a negative relationship between classroom achievement levels and SES gradients suggesting that, in all provinces, achievement is equitably distributed in classrooms with high achievement levels. This finding demonstrates that policies designed to boost the achievement levels of low SES students are likely to raise the overall achievement levels within the provinces. The regression equation describing the relationship between classroom achievement levels and SES gradients for the entire Canadian sample indicates that a unit decrease in the SES gradient is likely to increase the average classroom achievement level by about 16 months of schooling. In further analyses, the classrooms were classified into four groups based on their achievement levels and their SES gradients. The first group represented classrooms with scores above the national average and SES gradients below the national average. The high score and low gradient indicates that most students including low SES students do well in classrooms in this group. The second group comprised of classrooms with scores above the national average and gradients above the national average. The high gradient indicates that low SES students' achievement is low in classrooms within this group. The third group consisted of classrooms with scores below the national average and SES gradients above the national average so that the achievement levels of low SES students is worse in this quadrant. The classrooms in the fourth group have achievement levels below the national average and SES gradients below the national average. Most students, irrespective of SES backgrounds, have relatively low mathematics achievement in classrooms in this quadrant. - I l l -If school and classroom effectiveness could be defined in terms of a high achievement level for all irrespective of students' SES background, then the classrooms in the first quadrant should be considered as the most effective classrooms, while those in the third quadrant could be regarded as the least effective. The distribution of classrooms within provinces based on the above classification is displayed in Table 6.6a. Tablel 6.6 Classroom Profiles Within Provinces: Achievement Levels by SES Gradients Canada NF N B ON A B BC QU Percent out of Total Classrooms Within a Province High score and low gradient (QI) 33.8 24.7 20.5 22.0 45.5 50.0 88.8 High score and high gradient (Q2) 14.2 29.9 23.1 9.5 24.2 15.6 5.0 Low score and high gradient (Q3) 39.2 39.0 48.7 52.7 19.2 15.6 2.5 Low score and low gradient (Q4) 12.8 6.5 7.7 15.9 11.1 18.8 3.8 Table 6.6 reveals that in about 34 percent of the classrooms, mathematics achievement is equitably distributed and the mathematics achievement level is relatively high. This percentage is quite high compared to less than 15 percent of the classrooms where mathematics achievement levels are relatively high, but a there is a large achievement gap between advantaged and disadvantaged students. The distribution varies from province to province. The percentage of classrooms with excellence and equitable distribution of mathematics achievement ranges from -112-as low as about 21 percent in New Brunswick to as high as about 89 percent in Quebec. The percentages for Newfoundland (24) and Ontario (22) were quite low compared to the percentages for Alberta (45.5) and B.C (50.0). Quadrant three represents classrooms with low mathematics achievement levels and where there is a large gap between advantaged and disadvantaged students. Disadvantaged students are least likely to be successful in learning mathematics in these classrooms. Less than 3 percent of classrooms in Quebec compared to over 50 percent in Ontario belong to this category. The percentage is 39 percent in Newfoundland, 49 percent in New Brunswick, about 19 percent in Alberta, about 16 percent in B.C. These findings demonstrate that a relatively large proportion of classrooms in Quebec have achieved excellence and equity in mathematics achievement. -113-The SES backgrounds of students also had a relationship with the total between-classroom variation. Figure 6.2 shows the relationship between SES and total between-classroom variation for the three models. The graph shows the between-classroom variation at different levels of SES (between the 10th and 90 l h percentiles). For all these models, the between-classroom variation is larger at the lower levels of SES than at the higher levels of SES, indicating that classroom mathematics achievement levels tend to converge at the high levels of SES. This means that what differentiates classrooms on their achievement levels is the performance of their low SES students. A comparison of the lines for models 2 and 3 with the line for model 4 shows that the reduction in the between-classroom variation resulting from including school process variables is relatively the same at all levels of SES. Residuals of Classroom Achievement Scores An important feature of the program (WHLM) used for the multilevel analysis is that it provides estimates of residuals for both random intercepts and random slopes (gradients). In this analysis the residuals for the random intercept (Uoj) are estimates of the deviation of classroom achievement levels from the estimated grand mean (national average score). Therefore, these residuals are estimates of the extent to which classroom achievement scores differ from the national average. Why the Six Provinces Differ in their Achievement Levels The next analysis employed the residual of the intercept from the models to examine differences among the provinces in their mathematics achievement levels. The residuals from the models were used to describe the extent to which the average classroom within each of the six -114-provinces differed (deviated) from the national average. Figure 6.3 displays the differences among the provinces in the extent to which on average the achievement levels of classrooms within the provinces differ from the national average. 2.0 1.5 Figure 6.3 Mean Classroom Deviation Scores by Province for the Different Models NF NB ON AB BC QU O 1.0 O I 0.5 CD >-0.0 -0.5 A -• -O -i » ® -•o I •• A - ...o Modell Model2 Model3 Model4 Models The first model (model 1) indicates that on average the achievement levels of classrooms in Quebec are over one and half years above the national average; classrooms in Alberta and B.C are about 5 months above the national average; and the achievement levels of classrooms in the other provinces (Newfoundland, New Brunswick, and Ontario) are below the national average. The second model (model 2) has been adjusted for students' background characteristic. The -115-figure indicates that the adjustment did not significantly change the differences among the provinces in their achievement levels. The third model (model 3) includes students' background characteristics and mean SES so that the provincial scores have been adjusted. Figure 6.3 shows a slight increase in the average scores for Newfoundland and New Brunswick, and a slight decrease in the average for B.C., such that the average scores for the four provinces (Alberta, B.C., Newfoundland, and New Brunswick) appear to be indistinguishable. Quebec's average score increases a bit, while the average for Ontario decreases just slightly and therefore increasing the achievement gap between Quebec and Ontario. The last model (model 4) included all variables in the equation. Figure 6.3 indicates a sharp decrease in the average score for Quebec (from 1.82 to 1.06), an increase for the average score for Ontario (-0.42 to -0.20), and relatively stable scores for the other provinces. The differences among the provinces in their average scores reduced drastically. Two main conclusions can be drawn from these analysis: the differences in achievement levels between the Atlantic provinces (Newfoundland, and New Brunswick) and the western provinces (Alberta, and B.C) are largely due to students' background characteristics. Students of similar background characteristics and in classrooms with similar mean SES in these provinces performed equally well in Fractions. The second conclusion is that the achievement level of Quebec is partially attributable to certain schooling processes such as their highly specialized mathematics teachers, their low pupil-teacher-ratio, and their policy to keep their remedial students within regular classrooms. -116-Summary of Findings A number of important findings emerged from the third set of analyses. The analyses reveal a statistically significant variation among classrooms in their SES gradients indicating differences among classrooms in their distribution of students' achievement in Fractions along socioeconomic lines. The SES gradient has a negative relationship with classroom achievement level implying that students' achievement levels in Fractions are equitably distributed in classrooms with high achievement levels. Further analyses indicated that: classrooms that have achieved equity tend to have low pupil-teacher-ratio, specialized mathematics teachers, and rarely group students to work on mathematics problems; and classrooms with high achievement levels are characterized by less traditional instructional practices, more use of calculator but no computer, more homework, lower pupil-teacher-ratio, no remedial tracking, specialized mathematics teachers, and less school disciplinary problems. The analyses also reveals that Quebec, Alberta, and B.C., which have relatively high achievement levels, tend to have a high percentage of their classrooms where low SES students have high achievement levels, whereas the provinces with relatively low achievement levels tend to have a high proportion of classrooms where low SES students have low achievement levels. Two major findings unfolded from the analyses on the variation among provinces in their mathematics achievement. First, the analyses indicated that students' background characteristics seem to account for a large part of the variation among four of the six provinces (Newfoundland, New Brunswick, Alberta, and BC) in their achievement levels. The average SES student in an average mean SES classroom in these provinces performed almost at the same levels on the Fractions test. The second finding is that the differences in achievement levels between Quebec and the other provinces is relatively smaller when provincial achievement levels are assessed -117-based on their students with average background characteristics in schools and classrooms with similar classroom and school processes. The Quebec advantage in mathematics can partially be explained in terms of the differences in classroom and school processes between Quebec and the other provinces. The six provinces are quite similar in their instructional practices. Quebec has a slight advantage in the coverage of Geometry, has the lowest pupil-teacher-ratio, has the highest proportion of specialized mathematics teachers, and has the smallest proportion of classrooms where remedial students are removed from regular classrooms. -118-Chapter 7 Summary and Discussion of Major Findings and their Implications Knowledge of mathematics is becoming increasingly important as most careers now, and a greater proportion of jobs in the future will require skills in mathematics. The increased need for mathematics skills has placed significant demands on schooling systems to move toward school policies and practices that provide opportunities to enhance the abilities of all students, especially those from disadvantaged backgrounds, to acquire knowledge in mathematics. The equitable distribution of this knowledge among the general population is essential for a society's overall productivity and social cohesion. This calls for research that seeks to identify successful classrooms in terms of their excellence in mathematics outcomes and the equitable distribution of these outcomes, as well as the schooling practices associated with successful classrooms. The main focus of this study was to understand the schooling processes that allow excellence in mathematics for all in Canadian grades 7 and 8 classrooms. The study employed multilevel statistical models. The initial analyses explored the extent to which the differences in students' mathematics achievement were attributable to gender, family background, classroom, and the province where students attended school. The subsequent analyses examined the differences specific to particular mathematics domains, and determined whether there were school and classroom processes associated with excellence, and with the equitable distribution of mathematics achievement. The study also attempted to discern why six provinces in Canada differed in their mathematics achievement levels. A number of important findings emerged from the analyses which have been discussed in detailed in the previous three chapters. This chapter summarizes and discusses the main findings and their implications. -119-Major Findings There are seven major findings. (1) Students within mathematics classrooms vary in their achievement levels according to their gender and family backgrounds. As discussed in the first and third chapters, a number of previous studies have demonstrated the relatively low mathematics achievement of females and students from low SES families. This study uncovered that these findings also hold in the context of Canadian grades 7 and 8 classrooms, and that the achievement levels of females and low SES students are lower in Proportionality, Measurement, and Fractions than other domains of mathematics. The relatively low achievement of females and low SES students in these domains could be attributed to a number of factors, including the possibility that teachers present concepts in these domains in ways that do not allow these students to utilize their knowledge from one domain to understand another. Further analyses revealed that, in general, within a mathematics classroom, students' achievement levels were not stable across the six domains of mathematics; that is, students with high scores in one domain did not necessarily have high scores in other domains. These findings indicated that a student's success or failure in mathematics learning is domain-specific and is also related to the backgrounds of students. More research is needed to determine whether the inconsistency in students' achievement across domains of mathematics is related to instructional practices. The process of learning mathematics involves building on prior knowledge and experiences so that one expects teachers to present mathematical concepts in ways that allow their students to see the interconnection of ideas in mathematics. -120-(2) Socioeconomic gradients vary significantly among classrooms, and there is some evidence that gradients decrease with increasing classroom mathematics achievement levels. A socioeconomic gradient is an indicator of how well a classroom has achieved an equitable distribution of mathematics achievement along socioeconomic lines. Steep gradients indicate large disparities between advantaged and disadvantaged students within a classroom, whereas shallow gradients indicate a more equitable distribution of mathematics achievement. Excellence in mathematics achievement for all requires that one thinks about both equity and excellence together; that is, gradients need to be considered alongside levels of achievement. From this view, one would expect any schooling system, including those in Canada, to strive to achieve equity through educational policy and reform initiatives that are likely to bolster the achievement levels of less advantaged students up to those of advantaged students. School policies and practices that achieve equity but result in lower achievement levels for certain groups are undesirable. Also undesirable are schooling practices that result in disadvantaged students being disadvantaged in schools and classrooms. There are examples of schools that are successful in achieving both excellence and equity (e.g., see Lee & Bryk, 1989). The findings in this study indicate that gradients do vary significantly among classrooms, and that there is a modest negative relationship between excellence and equity; that is, mathematics achievement is equitably distributed in classrooms with high achievement levels. Thus, there are successful classrooms in Canada, but the most successful classrooms tend to be those where students from disadvantaged socioeconomic backgrounds excel in mathematics. -121-(3) A more equitable distribution of achievement within mathematics classrooms was related to teachers avoiding practices which involve small grouping, where mathematics teachers are specialized, and in schools where pupil-teacher-ratio is low. The finding pertaining to small grouping is not consistent with the expected theory at least with respect to the notion that such a practice would provide weak students lacking certain mathematics skills with the opportunity to learn them from their more advanced peers. The theory holds that interaction among students within small groups through discussion, debating, and expressing ideas creates the opportunity for multiple acceptable solutions to mathematics problems. The belief is that, through these interactions, students would experience cognitive conflicts, evaluate their reasoning, and enrich their understanding about mathematical concepts. However, as Springer, Stanne, and Donovan (1999) have noted, without the appropriate structures to make each member of a small group accountable for learning, the expected benefits of small groupings may not be realized, since the interaction would be in most instances merely sharing answers instead of ideas. A number of studies indicate that effective interactions characterized by high-level deliberations about issues that enhance conceptual understanding occur when teachers clearly define issues, give specific guidelines, and define roles for members in a group (see Johnson and Johnson, 1994). TIMSS data did not include variables describing structure and dynamics within the small groups, and therefore, the motivation of low SES students to interact with other students in their small grouping could not be evaluated. Further detailed study on the effect of small groupings on students' mathematics learning is needed. The association of low pupil-teacher-ratio and teacher specialization with equitable distribution of mathematics achievement makes sense as one would expect knowledgeable mathematics teachers to be deeply committed to the teaching of mathematics and could more -122-easily keep up to date with latest curriculum developments and innovations in mathematics teaching. And in schools with low pupil-teacher-ratio, one will expect small class sizes that will allow these teachers to utilize all resources and strategies at their disposal to ensure that all students excel in learning mathematics. (4) Excellence in mathematics is possible in classrooms where a teacher's instructional practice is less traditional, where calculators are used regularly but computers are not used, where teachers regularly assign homework, where there are fewer disciplinary problems, where teachers specialize in mathematics instruction, where pupil-teacher-ratio is low, and where remedial students are not removed from regular mathematics classrooms. The finding regarding traditional instructional practice was expected as it is consistent with the contemporary views of mathematics educators that such an instructional practice makes students less active in classroom mathematics learning. The recommended instructional practice is students' active interactions and participation in mathematics, and problem-solving involving real-life experiences. The finding indicating lower mathematics achievement from regular computer use in mathematics classrooms runs contrary to what some researchers might expect. However, i f in mathematics classrooms where teachers regularly use computers, too much instructional time is spent teaching the rudiments of using a computer rather than the concepts of mathematics, one would expect such a result. This could change in a decade or so from now, when a larger proportion of students will have basic computing skills, and mathematics classrooms will be equipped with more computers. The introduction of calculators more than a decade ago went through a similar situation. The high achievement levels associated with regular use of calculators demonstrates that calculators do enhance students' mathematics learning. -123-One would expect that regular homework assignments should bolster mathematics achievement. This study suggested this is the case, but it also suggested that regular quizzes do not necessarily lead to high mathematics achievement. This finding demonstrates the importance of regular homework as opposed to regular quizzes. The finding regarding disciplinary climate is consistent with the literature. A school environment where there are few disciplinary problems is required for learning. Such an environment allows teachers and students to devote more of their time in academic activities. The TIMSS data did not include a strong measure of classroom discipline, which one would expect to have yielded even stronger effects. Successful mathematics learning is likely in classrooms where the teachers specialize in mathematics and in schools with lower pupil-teacher-ratio. The issue of teacher specialization and pupil-teacher-ratio is controversial. Opponents of teacher specialization have argued that knowledge in other subject areas is essential for instructional organization. In schools with low pupil-teacher-ratio, teachers are likely to have either fewer number of class periods or smaller class size that should enhance their teaching effectiveness. However, while a number of studies have indicated that reducing class size leads to higher achievement levels, the class size effects have been relatively small (e.g., Finn & Achilles, 1999; Willms & Kerckhoff, 1995; Willms & Somers, 1999) prompting opponents to caution that the likely benefits from wide-scale reduction in class size should be assessed relative to similar effects of other reforms that are potentially less costly. On the positive side, however, lowering pupil-teacher-ratio and increasing the number of teachers who specialized in mathematics seem to also reduce inequities indicating that disadvantaged students in Canadian grades 7 and 8 classrooms are likely to benefit substantively -124-from a low pupil-teacher-ratio and an increase in the number of specialized mathematics teachers. The relatively high achievement levels of classrooms in schools where remedial students are not removed from regular classrooms is also important. In a large number of schools within the provinces weak students are offered remedial classes. In some of these schools, however, students in remedial classes are removed from the regular classes. This is a form of tracking. The major motivation for this type of grouping in Canadian grade 7 and grade 8 classrooms is not known. In the United States, however, research indicates that this form of tracking is designed to ensure homogeneity of students in terms of their academic ability (see Mevarech and Kramarski, 1997). The belief is that teachers would be more efficient in teaching students with similar ability levels, and consequently produce high achievement levels. This study contradicts this belief. The study indicated that it is possible to achieve excellence in mathematics even with weak students in regular mathematics classrooms. Removing the academically weak students from regular classrooms is inconsistent with socio-cultural learning theory as the practice denies these students the opportunity to learn from their more able counterparts. (5) The average socioeconomic status of a classroom has an effect on student achievement over and above the effects associated with a child's own family background. This finding demonstrates that students, irrespective of their SES backgrounds, are likely to score higher in mathematics if they are in classrooms with high mean SES. This is consistent with the findings of a number of studies pertaining to contextual effects associated with one's peer group, and it demonstrates the importance of peer interaction of talented and motivated students in classroom mathematics learning. The finding also calls for caution in the way students are distributed in schools as this could severely hamper the successful mathematics -125-learning of students from disadvantaged home backgrounds. The term "double jeopardy" is used in this study to indicate that a child from a family with poor socioeconomic status has an even worse chance of success in a school where the majority of students are also from families with low socioeconomic status. A number of researchers have noted that when students are segregated through residential segregation, private schooling, or choice arrangements within the public sector, advantaged students benefit slightly, but disadvantaged students do considerably worse. And if the desire of a schooling system is to ensure quality mathematics outcomes for all students, then policies that tend to segregate students according to ability or socioeconomic status should be viewed with caution. This caution speaks to policies and programs for private and denominational schooling, charter or magnet schools, French immersion programs, and programs for "gifted" students. These programs have the potential to contribute to segregation of students in schools based on socioeconomic status. The reason is that parents with middle-class backgrounds are more likely to take advantage of these policies and programs than parents with low social-class backgrounds so that these schools will have a larger concentration of students with middle class backgrounds. Also, i f these specialized programs within a school result in most disadvantaged children being segregated in one classroom, then the teacher would have to meet the needs of more disadvantaged children, instead of just a few. While these programs and policies may have been initiated with the best of intentions, their potentially undesirable consequences for disadvantaged students need to be taken into consideration for justifying their continued existence. There should be a continuous monitoring of these programs to assess their efficacy as well as their effect on disadvantaged students' achievement in mathematics. -126-(6) There are large and statistically significant differences among the Canadian provinces, both in their levels of academic achievement in mathematics and in their SES gradients. The analyses revealed that the six provinces can be clustered into three groups: Newfoundland, New Brunswick, and Ontario, with achievement levels which were below the national average; Alberta, and B.C., with achievement levels above the national average; and Quebec, with achievement levels well above the national average. The provinces also varied in their SES gradients. The SES gradients were relatively shallow in Quebec but steep in Newfoundland and New Brunswick. The SES gradients for Ontario, Alberta, and B.C. were close to the national average. The findings indicated that the provinces of Quebec, Alberta, and B.C with high mathematics achievement levels tended to have shallow gradients, whereas the other provinces, Newfoundland and New Brunswick, with lower mathematics achievement levels tended to have steep gradients. This finding indicates that mathematics achievement is equitably distributed in provinces with high achievement levels. In other words, some provinces excel in mathematics, but the most successful provinces are those where disadvantaged students from low SES backgrounds excel in mathematics. This is quite evident in the way provincial achievement levels are distributed along socioeconomic lines. The variation is wider at the lower SES levels than at the higher SES levels suggesting that provinces with high average mathematics achievement levels tend to do so by raising the achievement levels of their low SES students. The distribution also indicated that students from low SES families are likely to have high achievement levels if they attended schools in Quebec and low achievement levels i f they attended schools in Newfoundland, New Brunswick, or Ontario. For students from high SES families, attending a school in Quebec may give them a slight advantage but would not matter in -127-any of the other five provinces. The study also provided evidence that classroom mathematics achievement differences and gradients are linked to differences in the provinces, indicating that a student in a classroom might successfully learn mathematics, while another student of similar family background may not be as successful, simply because of the province in which the student received mathematics instruction. Quebec, Alberta, and B.C. have relatively high proportions of their classrooms that are successful in achieving excellence and equity. The provincial achievement levels and their SES gradients were stable across the six domains of mathematics. This means that a province with a high achievement level and shallow SES gradients in one domain also tends to have high achievement levels and shallow gradients in the other domains of mathematics. (7) Some of the differences among the six provinces in their levels of academic achievement in mathematics are attributable to provincial differences in schooling processes. The analyses demonstrated that students' background characteristics could not account for all the differences among the six provinces in their achievement levels. However, the inclusion of variables describing mathematics instructional practices and other school processes explained some of the differences in achievement levels between Quebec and the other provinces, and all the variation in achievement levels among the other five provinces. There are no major differences between Quebec and the other provinces in their classroom instructional practices. Quebec differs from the other provinces in its low pupil-teacher-ratio, its specialized mathematics teachers, and in the small proportion of schools where students are removed from regular mathematics classrooms. Incidentally, these variables are also associated with excellence -128-and equity within mathematics classrooms so that one can be confident in attributing some of the differences between Quebec and the other provinces to differences in these school processes. Implications for Further Research The initial expectation in this study was that the differences in content and pace of the mathematics curriculum among the provinces would account for the differences in mathematics achievement levels between Ontario and Quebec, and between the Atlantic and Western provinces. Willms (1996) noted that these differences were evident in two international studies conducted in the 1980s, and hypothesized that the Quebec advantage in mathematics is attributable to the faster-paced and more centralized curriculum in Quebec, and to the high emphasis Quebec places on mathematics learning. This study included data to test the curriculum hypothesis. The study found a slight inter-provincial difference associated with curriculum coverage. However, this was certainly not the most important factor, and it could not explain the large disparities among provincial mathematics achievement levels. Rather, the differences seem to be associated with a number of factors pertaining to school culture, the way students are organized for instruction, and to some extent school resources. Nevertheless, this study has demonstrated that even with fairly blunt measures used in an international study, the analyses were quite successful in explaining some of the Quebec advantage and nearly all of the variation among the other five provinces. However, there is still a need for more comprehensive research studies for provinces to monitor their schooling systems and to learn from each other strategies to improve mathematics learning for all. The only national monitoring system in Canada, the SAIP, does not include data to measure children's socioeconomic status, which is fundamental to understanding the variation of -129-school effects on mathematics learning. Another major problem is that neither the SAIP nor the large international studies in which Canada has been involved followed a cohort of students through their schooling. This is essential, since school effectiveness by definition is a measure of growth in learning, not students' achievement at a particular stage in school. School effectiveness research studies in Canada would benefit from this study in three significant ways. First, the conceptualization of school effectiveness in terms of excellence and SES gradient allow researchers to identify the strength and weaknesses in educational policies in terms of how these policies are successful in ensuring that every student has the opportunity to excel in mathematics. This conceptualization can serve as a useful framework for discerning school effectiveness. Second, the results from this study constitute a solid base on which future research on school effectiveness in Canada can be built. Third, the complex multilevel statistical procedures that were employed in this study are more consistent with the complex realities researchers of school effects seek to portray. Educational systems in general are complex in structure, and the processes within these structures are also complex such that they require tools with a comparable degree of complexity. The complex statistical techniques employed in this study can serve as models for the analyses of future studies. -130-References Aitkin, M . A . & Longford., N . T. (1986). Statistical modeling of data on teaching styles. Journal of the Roval Statistical Society. A( 144), 419-461. Atkinson, J. W. (1964). An introduction to motivation. Princeton, NJ: Van Nostrand. Baker, D. P. (1993). Compared to Japan, the U.S. is a low achiever ... really. Educational Researcher 22(3). 18-20. Ball, D. L. (1993). With an eye on the mathematical horizon: Dilemmas of teaching elementary school mathematics. Elementary School Journal 93: 373-397. Bandura, A. (1977). Self-efficacy: Towards a unifying theory of behavioral change. Psychological Review 84, 191-215. Benbow, C. P. & Stanley, J. C. (1980). Sex differences in mathematical ability: Fact or artifact? Science 210: 1262-1264. Berliner, D. C. (1993). International comparison of student achievement: A false guide for reform. National Forum: 25-29. Bielinski, J. D. & Davison, M . L. (1998). "Gender Differences by item difficulty interactions in multiple-choice." American Educational Research Journal 35(3): 455-476. Brookover, W.B., Beady, C.H., Flood, P., Schweitzer, J., and Wisenbaker, J. (1978). School social systems and student achievement. New York: Praeger. Bryk, A . S. & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, California: SAGE Publications, Inc. Burstein, L. (1980). The analysis of multilevel data in educational research and evaluation.In D. C. Berliner (Ed.), Review of Research in Education, (pp. 158-231), Washington, DC American Educational Research Association. Burstein, L., Linn, R. L. , & Capell, F. J. (1978). Analyzing multilevel data in the presence of heterogeneous within-class regressions. Journal of Educational Statistics. 3, 347-383. Caldas, S. J. & Bankston, C. (1997). Effect of school population socioeconomic status on individual academic achievement. The Journal of Educational Research 90, 269-277. Carey, D. A. , Fennema, E., Carpenter, T. P., & Franke, M . L. (1995). Equity and mathematics education. In W. G. Secada, E. Fennema and L. B. Adajian (Eds.), New directions for equity in mathematics education (pp. 93-125). Cambridge, Cambridge University Press. -131-Clifford, P., & Heath, A. (1984). Selection does make a difference. Oxford Review of Education. 10, 85-97. Cobb, P. (1989). Experiential, cognitive, and anthropological perspectives in mathematics education. For the Learning of Mathematics 9(2), 32-42. Cobb, P. (1994). Where is the mind? Constructivist and sociocultural perspectives on mathematical development. Educational researcher 23(7), 4-4. Cobb, P. (1998). Analyzing the mathematical learning of the classroom community: The case of statistical data analysis. In O. Alwyn & K. Newstead (Eds.), Proceedings of the 22 n d conference of the International Group for the Psychology of Mathematics Education Vol.1 (pp.33-48). Bellville, South Africa: Kwik Kopy Printing. Cobb, P., Wood, T., & Yackel, E. (1993). Discourse, mathematical thinking, and classroom practice. In N . Minick, E. Forman & A. Stone (Eds.), Education and mind: Institutional, social, and developmental processes, (pp. 91-119.) New York, Oxford University Press. Cohen, E. G. (1994). Restructuring the classroom: Conditions for productive small groups. Review of Educational Research 64(1): 1-19. Cohen, J. & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences (2 n d Edition). Hillsdale, NJ: Lawrence Erlbaum Associates. Coleman, J. S. (1990). Equality and achievement in education. Denver, Westview Press. Coleman, J. S., Campell, E. Q., Hobson, C. J., McPartland, J., Mood, A . M . , Weinfeld, F. D., & York, R. L . (1966). Equality of educational opportunity. Washington, D. C: U.S., Government Printing Office. Council of Ministers of Education, Canada (1993). School achievement indicators program: Mathematics assessment. Toronto, Ontario, Council of Ministers of Education. Council of Ministers of Education, Canada (1997). School achievement indicators program: Mathematics assessment. Toronto, Ontario, Council of Ministers of Education. Cronbach, L. J. (1971). Test validation. In R. L. Thorndike (Ed.), Educational Measurement.. Washington, DC, American Council of Education. Darling-Harmond, L . (2000). Teacher quality and student achievement: A review of state policy evidence. Educational Policy Analysis Archives 8(1), 1-48. DesLauriers, R. (1990). The impact of employee illiteracy on Canadian business. Ottawa, Ontario, Conference Board of Canada, HRDC. -132-Dossey, J. A. , Mullis, I. V . S., Linquist, M . M . , & Chambers, D. L. (1988). The mathematics report card: Are we measuring up? Princeton, NJ: National Assessment of Educational Progress, Educational Testing Services. Dreeben, R., & Gamoran, A. (1986). Race, instruction, and learning. American Sociological Review. 51, 660-69. Dunham, P. H. & Dick, T. P. (1994). Research on graphing calculators. Mathematics Teacher 86: 440-45. Dupre, J. S. (1996). Taming the monster: Dept, budget, and federal-provincial fiscal relations at the fin de siecle. In C. Dunn (Ed.), Provinces: Canadian provincial politics (379-397). Scarborough, ON, Broadview Press. Fennema, E. and Carpenter, T. P.(1981). The second international assessment and sex-related differences in mathematics. Mathematics Teacher (74): 554-559. Fennema, E. and Carpenter, T.P. (1998). New perspectives on gender differences in mathematics: An introduction. Educational Researcher (27)5: 4-5. Fennema, E. & Leder, G. C (Eds.). (1990). Mathematics and gender. New York, Teachers College Press. Fennema, E. & Meyer, M . R. (1989). Gender, equity and mathematics. In W. G. Secada (Ed.) Equity in education.(pp.l46-157). Lewes, England, Falmer Press. Fennema, E. & Peterson, P. L (1985). Autonomous learning behavior: A possible explanation of gender-related differences in mathematics. In L. C. Wilkinson & C. B. Marret (Eds.), Gender influences in classroom interaction, (pp. 17-35). New York: Academic Press. Fennema, E. & Sherman, J. (1977). Sex-related differences in mathematics achievement, spatial visualization and affective factors. American Educational Research Journal 14(1), 51-71. Fennema, E. & Sherman, J. (1978). Sex-related differences in mathematics achievement and related factors: A further study. Journal for Research in Mathematics Education 9, 189-203. Finn, J. D. & Achilles, C. M . (1999). Tennesse's Class size study: Findings, implications, misconceptions. Educational Evaluation and Policy Analysis 21(2): 97-109. Frankenstein, M . (1995). Equity in mathematics education: Class in the world outside the class. In W. G. Secada, E. Fennema & L. B. Adajian (Eds.), New direction for equity in mathematics education (pp. 165-190). Cambridge, Cambridge University Press. Frempong, G., & Willms, J. D. (in press). The importance of quality schools. In J. D. Willms (Ed.), Vulnerable children in Canada. -133-Friedman, L. (1989). Mathematics and the gender gap: A recent analysis of recent studies on sex differences in mathematics tasks. Review of Educational Research 59(2): 185-214. Furino, A . (1988). Cooperation and Competion in the Global Economy. Cambridge, M A , Balinger. Goldstein, H. (1987). Multilevel Models in Educational and Social Research. New York: Oxford University Press. Goldstein, H. (1995). Multilevel statistical models. London: Arnold. Goldstein, H. (1997). Methods in school effectiveness research. School effectiveness and school improvement. 8: 369-95. Gonzalez, E. J. et al, (Ed.) (1997). User guide for the TIMSS international database: Primary and middle school years 1995 assessment. Chestnut Hi l l , M A , USA, TIMSS International Study Center, Boston College. Good, T. L . & Brophy, J. E. (1986). School effects. In M . C. Wittrock (Ed.), Handbook of research in teaching (pp. 43-67). New York, Macmillan. Grouws, D. A. (Ed.) (1992). Handbook of research on mathematics teaching and learning: a project of the National Council of Teachers of Mathematics. New York, Macmillan Publushing Company. Guskey, T. R. & Kifer, E. W. (1990). Ranking school districts on the basis of statewide test results: Is it meaningful or misleading? Educational Measurement: Issues and Practices 9(1), 11-16. Harris, M . , (Ed.) (1991). Schools, mathematics and work. London: The Falmer Press. Harvey, G. & Klein, S. S. (1989). Understanding and measuring equity in education: A conceptual framework.In W. G. Secada (Ed.), Equity in education (pp. 43-67). New York, The Falmer Press. Hawk, P., Coble, C. R., & Swanson, M . (1985). Certification: It does matter. Journal of Teacher Education 36(3): 13-15. Hawkins, E. F., Stancavage, F. B., & Dossey, J. A.(1998). School policies and practices affecting instruction in mathematics: Findings from the national assessment of educational progress. Washington, D.C: U.S. Department of Education, Office of Educational Research and Improvement. Hawkins, J. D. & Lishner, D. M . (1987). Schooling and delinquency. In E. H. Johnson (Ed.), Handbook on crime and delinquency prevention, (pp. 179-221). New York, Greenwood. -134-Healey, C. C. (1993). Discovery courses are great in theory, but.... In L. S. Judah, M . Yerushalmy and B. Wilson (Eds.), The geometric supposer: What is a case of?. Hilldale, N . J., Lawrence Erlbaum Associates. Heath, A . (1990). Class inequalities in education in the twentieth century. Journal of the Royal Statistical Society. Series A 153, 1-16. Hembree, R. D. & Dessart, D. J.(1986). Effect of hand-held calculators in pre-college mathematics education: A meta-analysis. Journal for Research in Mathematics Education 17: 83-99. Hershkowitz, R. (1990). Psychological aspects of learning geometry. In P. Nesher & J. Kilpatrick (Eds.), Mathematics and cognition: a research synthesis by the international group for the psychology of mathematics education (pp. 70-94). Cambridge, Cambridge University Press. Ho, S., & Willms, J. D. (1996). The effects of parental involvement on eighth grade achievement. Sociology of Education, 69, 126-141. Hoffer, T. (1997). High school graduation requirements: Effects of dropping out and achievement. Teachers College Record 98: 608-628. Human Resources Development Canada (1999). Applied Research Bulletin: A special edition on child development. Ottawa: Human Resources Development Canada. Johnson, D. W., & Johnson, R. T. (1994). Learning together and alone: Cooperative, competitive, and individualistic learning. Boston: Allyn & Bacon. Kieran, C. (1990). Cognitive processes involved in learning algebra. In P. Nesher & J. Kilpatrick (Eds.), Mathematics and cognition: a research synthesis by the international Group for the psychology of mathematics education (pp.96-112). Cambridge, Cambridge University Press. Knapp, M . S. & Woolverton, S. (1995). Social class and schooling. In J. A . Banks (Ed.), Handbook of reseaerch on multicultural education. New York, Macmillan: 548 - 569. Kohr, R. L. , Masters, J. R. et al. (1989). The relationship of race, class, and gender with mathematics achievement for fifth-, eighth-, and eleventh-grade students in Pennsylvavia schools. Peabody Journal of Education 66(2), 147-171. Kozol, J. (1991). Savage inequalities. New York, Crown. Kreft, I. G. G., de Leeuw, J. & van der Leeden, R. (1994). Review of five multilevel analysis programs: BMDP-5V, GENMOD, H L M , M L 3 , V A R C L . American Statistician 48(4): 324-335. -135-Kupermintz, H. , Ennis, M . M . , Hamilton, L. S., Talbert, J. E., & Snow, R. E. (1995). Enhancing the validity and usefulness of large-scale educational assessments: I. NELS: 88 mathematics achievement. America Educational Research Journal 32(3), 523-554. Lampert, M . (1990). When the problem is not the question and the solution is not the answer: mathematical knowing and teaching. American Educational Research Journal 27(1): 29-64. Lau, C. J. (1979). Educational production function. In D. M . Windham. (Ed.), Economic dimensions of education. Washington, D.C: National Academy of Education. Lave, J. & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge, Cambridge University Press. Leder, G. C. (1990). Gender differences in mathematics: An overview. In E. Fennema and G. C. Leder (Eds.), Mathematics and gender. New York: Teachers College Press: 10-26. Leder, G. C. & Fennema, E. (1990). Gender differences in mathematics: A synthesis. In E. Fennema and G. C. Leder (Eds.), Mathematics and gender.. New York: Teachers College Press: 188-200. Leder, G. C. (1992). Mathematics and gender: changing perspectives.In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning: A project of the National Council of Teachers of Mathematics (pp. 597-622). New York, Macmillan. Lee, V . E. & Bryk, A.(1989). A multilevel model of the social distribution of high school achievement. Sociology of Education 62(3), 172-192. Lee, V . E. & Smith, J. B. (1993). Effects of school restructuring on the achievement engagement of middle-grade students. Sociology of Education 62: 172-192. Lee, V . E. & Smith, J. B. (1995). Effects of high school restructuring and size on gains in achievement and engagement for early secondary school students. Sociology of Education 66: 241-70. Lee, V . E., J. B. Smith, & Croninger, R. G. (1997). How high school organization influences the equitable distribution of learning in mathematics and science. Sociology of Education 70(April), 128-150. Leot'ev, A . N . (1981). The problem of activity in psychology. In J. V . Wertsch (Ed.), The concept of activity in society psychology. Armonk, N Y : Sharp. Levin, H. (1980). Education production theory and teacher inputs. In C. E. Bidwell & D. M . W. (Eds.), The analysis of educational productivity (pp. 203-231). Cambridge, M A : Ballinger. -136-Levin, H. M . & Kelley, C. (1994). Can education do it alone? Economics of Education Review 13(2), 97-108. Linn, R. L. (1988). Accountability: The comparison of educational systems and the quality of test results. Educational Policy 1. 181-198. Ma, X. and J. D. Willms (1999). Dropping out of advanced mathematics: How much do students and schools contribute to the problem? Educational Evaluation and Policy Analysis 21(4): 365-383. Mayer, R. E. (1992). Cognition and instruction: Their historic meeting within educational psychology. Journal of Educational Psychology 84, 405-412. McKnight, C. C , Crosswhite, J. A. , Dossey, J. A. , Kifer, E., Swafford, S. O., Travers, K . J.& Cooney, T. J. (1987). The underachieving curriculum: Assessing U.S. school mathematics from international perspective. Champaign, IL, Stipes Publishing. McLeod, D. B. (1992). Research on affect in mathematics education: A reconceptualization. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning: a project of the National Council of Teachers of Mathematics (pp. 575-596) New York: Macmillan Publishing Company. McPherson, A. F. & Willms, J. D.(1987). Equilisation and improvement: Some effects of comprehensive reorganization in Scotland. Sociology 21, 509-539. Meece, J. L.,Eccles, J. et al. (1982). Sex differences in math achievement: Towards a model of academic choice. Psychological Bulletin 91, 324-348. Mevarech, Z. R. & Kramarski, B. (1997). Improve: A multidimensional method for teaching mathematics in heterogeneous classrooms. American Educational Research journal 34(2): 365-394. Meyer, M . (1989). Equity: The missing element in recent agendas for mathematics education. Peabody Journal of Education 66(2), 6-21. Meyer, M . (1991). Equity: The missing element in recent agendas for mathematics education. Peabody Journal of Education 66(2), 6-21. Mitchell, J. H. , Hawkins, E. F., Jakwerth, P. M . , & Stancavage, F. B (1999). Student work and teacher practices in mathematics. Washington, DC: National Center for Education Statistics. Mislevy, R. J. (1995). What can we learn from international assessments? Educational Evaluation and Policy Analysis 17(4), 419-437. -137-Monk, D. H. (1994). Subject matter preparation of secondary mathematics and science teachers and student achievement. Economics of Education Review 13(2): 125-145. Muthen, B., Huang, L. , Jo, B., Khoo, S., Goff, G., Novak, J. & Shih, J. (1995). Opportunity-to-learn effects on achievement: Analytical aspects. Educational Evaluation and Policy Analysis 17(3), 371-403. National Council of Teachers of Mathematics (1989). Curriculum and evaluation standards for school mathematics. Reston, V A , National Council of Teachers of Mathematics, Inc. National Council of Teachers of Mathematics (1991). Professional standards for teaching mathematics. Reston, V A , National Council of Teachers of Mathematics, Inc. National Council of Teachers of Mathematics (2000). Principles and standards for school mathematics. Reston, V A , National Council of Teachers of Mathematics, Inc. Nunes, T. (1992). Ethnomathematics and everyday cognition. In D. A . Grouws. (Ed.), Handbook of research on mathematics teaching and learning (pp. 557-574). New York, Macmillan. Pallas, S. M . & Alexander, K. L . (1983). Sex differences in quantitative SAT performance: New evidence on the differential coursework hypothesis. American Educational Research Journal 20: 165-182. Oakes, J. (1985). Keeping track: How schools structure inequality. New Haven, CT: Yale University Press. Oakes, J. (1988). Tracking in Mathematics and Science Education: A structural contribution to unequal schooling. In L.Weis (Ed.), Class, race, and gender in American education.(pp.l06-125). Albany, N Y : State University of New York Press. Oakes, J. (1990). Multiplying inequalities: The effect of races, social class, and tracking on opportunities to learn mathematics and science. Santa Monica, Ca, R A N D . Organization for Economic Co-operation and Development (1994). The jobs study. Paris, Author Organization for Economic Co-operation and Development (1995). Employment outlook. Paris, Author. Pfeffermann D., Skinner, C.J., Holmes, D.J., Goldstein, H . and Rasbash, J . (1998), Weighting for unequal selection probabilities in multilevel models, Journal of the Roval Statistical Society. Series B, 60, Part 1, 23-40. Raudenbush, S. W. & Bryk, A . S. (1986). A hierarchical models for studying school effects. Sociology of Education 59: 1-17. -138-Raudenbush, S. W. & Willms, J. D. (1995). The estimation of school effect. Journal of Educational and Behavioural Statistics 20(4), 307-335. Reese, C. M . , Miller, K. E., Mazzeo, J., & Dossey, J. A . (1997). NAEP 1996 mathematics report card for the nation and the states. Washington, DC: National Center for Education Statistics. Reyes, H. & Stanic, G. M . A. (1988). Race, sex, socioeconomic status, and achievement. Journal for Research in Mathematics Education 19(1), 27-43. Robitaille, D. F. & Garden, R. A. (1996). Research questions and study design. Vancouver, Pacific Educational Press. Robitaille, D. F., Taylor, A . R. & Orphood, G. (1997). The TIMSS-Canada Report. Vancouver, University of British Columbia. Robitaille, D. F., Schmidt, W. H., Sebta, R., McKnight, C , Britton, J. & Nicol, C. (1993). Curriculum frameworks for mathematics and science. Vancouver, Pacific Education Press. Rogoff, B. (1990). Apprenticeship in thinking: Cognitive development in social context. Oxford, Oxford Press. Rojano, T. (1996). Developing algebraic aspects of problem solving within a spreadsheet environment. In N . Bednarz, C. Kieran & L. Lee (Eds.) Approaches to algebra: perspectives for research and teaching.. Dordrecht, Netherlands, Kluwer. Romer, P. (1993). Ideas and things. The EconomisKSeptember 11th-17th): 70-71. Rumberger, R. (1995). Dropping out of middle school: A multilevel analysis of students and schools. American Educational Research Journal 32(2), 583-625. Rumberger, R., & Willms, J. D. (1992). The impact of racial and ethnic segregation on the achievement gap in California high schools. Educational Evaluation and Policy Analysis 14(4): 377-396. Sanders, W. L . &. Rivers, J. C. (1996). Cumulative and residual effects of teachers on future student academic achievement. Knoxville, University of Tennessee Value-Added Research and Assessment Center. Saxe, G. B. (1991). Culture and cognitive development: Studies in mathematical understanding. Hillsdale, N.J., Lawrence Erlbaum Associates. Schmidt, W. H., McKnight, C. C , & Raizen, S. A. (1996). A splintered vision: An investigation of U.S. science and mathematics education. Boston, Mass: Kluwer. -139-Schmidt, W. H. et.al (1997). Many visions, many aims: A cross-national investigation of curricular intentions in school mathematics. Dordrecht, The Natherlands, Klwer Academic Publishers. Schoenfeld, A. H. , Smith, J. P. & Arcavi, A . A . (1993). Learning. In R. Glaser (Ed.), Advances in instructional psychology Vol . 4, (pp. 55-175). Hillsdale, N.J., Lawrence Erlbaum Associates. Secada, W. G. (1992). Race, ethnicity, social class, language, and achievement in mathematics. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learnings project of the National Council of Teachers of Mathematics (pp. 623-660). New York, Macmillan Publishing Company. Secada, W. G. & Meyer, M . R. (1989). Needed: An agenda for equity in mathematics education. Special issue of Peabodv Journal of Education 66(2), 22-56. Sells, L. W. (1973). High school mathematics as the critical filter in the job market. Conference on Minority Graduate Education, Berkleley: University of California. Seymour, E. (1992). The problem iceberg in science, mathematics, and engineering education: Student explanations of high attrition rates. Journal of College Science Teaching 21(4): 230-238. Seymour, E. (1995). The loss of women from science, mathematics, and engineering undergraduate majors: An explanatory account. Science Education 79(4): 437-473. Seymour, E. and Hewitt, N . (1997). Talking about leaving: Why undergraduates leave the sciences. Boulder, CO: Westview. Shapiro, H. S. & Purple, D. Eds. (1993). Critical social issues in American education. New York, Longman Publishing Group. Siegler, R. & Klahr, D.(1982). When do children learn? The relationship between existing knowledge and the acquisition of new knowledge. In R. Glaser (Ed.), Advances in instructional psychology Vol . 2 (pp. 121-211). Hilldale, N . J., Lawrences Erlbaum Associates. Snow, C , Barnes,W., Chandler, J., Goodman, I., & Hemphill, L . (1991). Unfulfilled expectations: Home and school influences on literacy. Cambridge, Massachusetts, Harvard University Press. Springer, L. , Stanne, M . E., & Donovan, S. S (1999). Effects of small group learning on undergraduates in science, mathematics, engineering, and technology: A meta-analysis. Review of Educational Research 69(1): 21-25. -140-Stanic, G. M . A . & Reyes, L . H. (1986). Gender and race differences in mathematics: A case study of a seventh-grade classroom. American Educational Research Association, San Francisco. Statistics Canada (1991). Adult Literacy in Canada: Results of a national study. Ottawa, Ontario, Minister of Industry, Science, and Technology. Stedman, L . C. (1994). Incomplete explanations: The case of U.S. performance in the international assessments of education. Educational Researcher 23(7), 24-32. Taylor, A . R. (1997). Canada.. In D. F. Robitaille (Ed.), National Contexts for Mathematics and Science: An encyclopedia of the education systems participating in TIMSS. (pp.70-81) Vancouver, Pacific Educational Press:. The Creative Research Group (1987). Literacy in Canada: A research report. Toronto, Southam News, Ottawa. Thum, Y . M . (1997). Hierarchical linear models for multivariate outcomes. Journal of education and behavioral statistics 22(1), 77-108. TIMSS International Study Center (1997). User's guide for the Third International Mathematics and Science Study (TIMSS) and U.S. augmented data files. Chestnut Hi l l , M A , USA, TIMSS International Study Center Travers, K. J. and Westbury, I (1989). The IEA Study of Mathematics 1: Analysis of Mathematics Curricula. Oxford, Pergamon Press. von Glassersfeld, E. (1992). Constructivism reconstructed: A reply to Suchting. Science and Education 80, 379-384. Vygotsky, L. S. (1978). Mind and society: The development of higher psychological processes. Cambridge, Harvard Education Press. Walberg, H. (1983). Scientific literacy and economic productivity in international perspective. Daedalus 112(2), 1-28. Westbury, I. (1992, June-July). Comparing American and Japanese achievement: Is the United States really a low achiever? Educational Researcher 21(5), 18-24. Westbury, I. (1993, April). American and Japanese achievement... again. Educational Researcher 22(3), 21-25. Wheeler, D. (1989). Contexts for research on the teaching and learning of algebra. In S. Wagner and C. Kieran (Eds.), Research agenda for mathematics education: Research issues in the learning and teaching of algebra. Hillsdale, NJ, Erlbaum. 4, 278-287. -141-White, K. R. (1982). The relation between socio-economic status and academic achievement. Psychological Bulletin 91. 461-481. Willis, S. (1989). Real girls don't do maths: Gender and the construction of privilege. Geelong, Australia, Deakin University. Willms, J. D. (1985). The balance thesis: contextual effects of ability on pupils O-grade examination results. Oxford Review of Education. 11(1), 33-41. Willms, J. D. (1986). Social class segregation and its relationship to pupils' examination results in Scotland. American Sociological Review. 51(2), 224-241. Willms, J. D. (1992). Monitoring school performance: A guide for educators. Lewis, The Falmer Press. Willms, D. J. (1996). Indicators of mathematics achievement in Canadian elementary schools. Growing up in Canada: National Longitudinal Survey of Children and Youth. Ottawa, Ontario, Statistics Canada: 69-82. Willms, J. D. (1997). Literacy skills and social class. Policy Options 18(6), 22-26. Willms, J. D. (1998). Basic concepts in hierarchical linear modeling with applications for policy analysis. Paper presented at the Conference of the International Association of Survey Statisticians and the International Association for Official Statistics, Aguascalientes, Mexico. Willms, J. D. (1999). Inequalities in literacy skills among youth in Canada and the United States. Ottawa, Statistics Canada. Willms, J. D., & Jacobson, S. (1990). Growth in mathematics skills during the intermediate years: Sex differences and school effects. International Journal of Educational Research 14,157-174. Willms, J. D. & Kerckhoff, A . C. (1995). The challenge of developing new social indicators. Educational Evaluation and Policy Analysis 17(1), 113-131. Willms, J. D. & Kerr, P. D.(1987). Changes in sex differences in Scottish examination results since 1976. Journal of Early Adolescent 7(1), 85-105. Willms, J. D. & Somers, M . (1999). Schooling outcomes in Latin America. Fredericton: Canadian Institute for Social Policy, University of New Brunswick. Wright, S. P., Horn, S. P., & Sanders, W. L . (1997). Teacher and classroom context effects on student achievement: Implications for teacher evaluation. Journal of Personnel Evaluation in Education: 57-67. -142-Appendix A National and International Studies National Longitudinal Study of Children and Youth (NLSCY) N L S C Y is survey which was conducted by Statistics Canada on behalf of Human Resources Development Council (HRDC). The primary aim of this survey is to understand factors associated with child development and well being. This survey is a longitudinal study intended to collect data for a number of years on the characteristics and life experiences of Canadian children as they mature from infancy to adulthood. The first cycle of data were collected in 1994-95. The data involved over 13 000 households and about 23 000 children aged 0 to 11 years. The data on household and children were collected from the "person most knowledgeable" (PMK) about the household. The P M K responded to a number of questions pertaining to their household and child through interviews. The response includes basic demographic and socioeconomic information about the household. The N L S C Y data also include information about the mathematics skills of a child in a household who was in grade 2 or above. This information was collected through a mathematics test which was administered by either the child's teacher or the principal. This test was a short and abbreviated version of the Mathematics computation Test of the standardized Canadian Achievement Tests, Second Edition (CAT/2). The test was designed to measure a child's skill and understanding in some basic mathematics concepts such as addition, subtraction, multiplication, and division. Three versions of the test (level 2, level 4, and level 6) indicating the grade level of a child were administered. The level 2 version was administered to children in -143-grades 2 and 3, level 4 version to children in grades 4 and 5, and level 6 version to children in grades 6 and 7. Children were assigned scores based on the number of correct responses on the test, the level of the test, and based on the performance of children in the norm sample (see Growing up in Canada, 1996). International Adult Literacy Study (IALS) IALS was conducted by the Organization of Economic Cooperation and Development (OECD) and Statistics Canada in 1994 to determine the level and distribution of literacy among the adult population of some OECD countries including Canada. This survey entailed interviews and testing of a representative sample of adults and youth in seven OECD countries. In Canda about 5 660 adults aged 16 to 90 were sampled. These adults were tested on their literacy skills in three domains: prose, document, and quantitative skills. The test on prose assessed participants' ability to read, understand and use information from written text. The document test assessed participants' skills to locate and use information from a number of sources such as transportation schedules, and maps. The quantitative part of the test assessed the ability of respondents to locate, comprehend and utilize mathematical ideas embedded in a text. The test scores were scaled into levels with lower levels (level 1) described as "simple" and upper levels (level 5) as "complex". School Achievement Indicators Program (SAIP) SAIP was initiated in 1989 by the Council of Ministers of Canada (CMEC) to address the question of 'how well students within the provinces are doing in mathematics, language, and -144-science'. The composition of C M E C includes ministers responsible for education in the provinces. SAIP test has 3 components: mathematics, science, and language. The first SAIP assessment in mathematics was in 1993 and the second was in 1997. The students who wrote the test were mostly 13 years-olds or 16 year-olds. Most 13-year-olds were in grade 8, while most of the 16-year-olds were in grade 11. Both groups of students wrote the same tests. About 26 000 13-year-olds and 22 000 16-year-olds were tested in 1997. Students were randomly sampled controlling for the different sizes of student populations in the provinces so that the data for each province is statistically reliable in estimating parameters for each province. The mathematics test assessed students' general knowledge in mathematics and their skills in mathematics problem solving. Students were examined in their use of strategies to solve mathematics problems in four main mathematics areas: numbers and operations, algebra and functions, measurement and geometry, and data management and statistics. These areas of mathematics are supposed to be quite popular in the curricula for all provinces and are identified as useful areas of mathematics in the curriculum standards document of the National Council of Teachers of Mathematics (NCTM). Educators from each province assessed the test materials to ensure that the tests materials matched their curriculum objectives. Students' performances in mathematics were described using a scale with five levels. These levels represented a continuum of students' mathematics knowledge and skills acquired over their entire elementary and secondary school experiences in mathematics. Level 1 described the early stage of mathematics competency which includes: adding, subtracting, dividing, and multiplying. Level 5 describes the mathematics competency acquired by students who have completed advanced mathematics courses expected of students near the end of their -145-secondary schooling. These courses include topics such as the use of algebraic expressions and properties of circles. - 1 4 6 -Appendix B Figure 1. Inter-Provinicial Differences in Mathematics Scores NF • NS • PE • NB * QU » ON • MN + SK X AB o BC * w » k o x • • 4 X _ • • •G X • A O X X o X + o • O X J i I i L NLSCY-Grade 2 NLSCY-Grade 4 NLSCY-Grade 6 TIMSS • Grade 7 TIMSS - Grade 8 SAIP93-Age13 SAIP97 - age 13 SAIP93-Age16 SAIP97-Age16 IALS - Youth Aged + 16 to 25 J , I i I i L -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 Years of Schooling 2.0 -147-

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0055011/manifest

Comment

Related Items