Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Investigating factors underlying undergraduate physics students’ attitudes and beliefs about physics… MacDonald, Alexandra Leigh 2014

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2014_september_macdonald_alexandraleigh.pdf [ 1.93MB ]
Metadata
JSON: 24-1.0167588.json
JSON-LD: 24-1.0167588-ld.json
RDF/XML (Pretty): 24-1.0167588-rdf.xml
RDF/JSON: 24-1.0167588-rdf.json
Turtle: 24-1.0167588-turtle.txt
N-Triples: 24-1.0167588-rdf-ntriples.txt
Original Record: 24-1.0167588-source.json
Full Text
24-1.0167588-fulltext.txt
Citation
24-1.0167588.ris

Full Text

 Investigating Factors Underlying Undergraduate Physics Students’ Attitudes and Beliefs about Physics through a Revalidation of CLASS  by  ALEXANDRA LEIGH MACDONALD  B.Sc., St. Francis Xavier University, 2011   A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  MASTER OF ARTS  in  THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES  (Science Education)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)   July, 2014 © Alexandra Leigh MacDonald, 2014 ii  Abstract The Colorado Learning Attitudes about Science Survey (CLASS) has been widely used to measure students’ attitudes and beliefs about learning physics. This usage is paired with the assumption that the eight factors determined by the developers underlie students’ attitudes and beliefs about physics in all populations. Confirmatory factor analysis did not support the existence of these eight factors amongst students enrolled in introductory physics courses at a large research university in Western Canada. Thus, to understand the factors that underlie students’ attitudes and beliefs about physics as conveyed by the CLASS data collected at the university, a revalidation procedure was undertaken. The investigation of underlying factors included performing and interpreting the results of exploratory factor analysis, confirmatory factor analysis, and reliability tests.  Exploratory analysis of the survey data suggested five factors that underlie students’ responses to the survey items in an introductory physics course for engineering students. Analysis of data from the introductory physics courses for engineering students and a calculus-based physics courses for science students confirmed the existence of three of the five emergent factors. These three factors have been labelled ‘Awareness of Real World Connections’, ‘Self-Efficacy’, and ‘Constructive Connectivity’. This emergent model indicates strong patterns in students’ attitudes and beliefs about physics from the beginning of their undergraduate careers. Future research is needed to support the existence of these revalidated factors and the robustness of this model in multiple populations. The potential for a tool that can provide insight into students’ attitudes about physics, and how these attitudes are shaped, to shape undergraduate physics learning are significant.   iii  Preface This thesis is an original intellectual product of the author, Alexandra MacDonald. The data collection was covered by UBC Ethics Certificate number H13-02464.    iv  Table of Contents Abstract ..................................................................................................................................... ii Preface...................................................................................................................................... iii Table of Contents ..................................................................................................................... iv List of Tables .......................................................................................................................... vii List of Figures ........................................................................................................................... x Acknowledgements .................................................................................................................. xi Dedication ............................................................................................................................... xii 1 Introduction ....................................................................................................................... 1 1.1 Background to the Study ............................................................................................ 3 1.2 Significance of this Research ..................................................................................... 5 1.3 Researcher’s Perspective ............................................................................................ 5 1.4 Research Questions and an Overview of the Methodology ....................................... 6 2 Literature Review.............................................................................................................. 8 2.1 Attitudes ..................................................................................................................... 9 2.1.1 What are Attitudes?............................................................................................. 9 2.1.2 Attitudes and Learning ...................................................................................... 10 2.1.3 Instrumentation and Assessment of Attitudes................................................... 13 2.2 Theoretical Framework ............................................................................................ 38 2.2.1 Validity ............................................................................................................. 39 2.2.2 Self-Efficacy ..................................................................................................... 45 2.2.3 Constructivist Learning Theories ...................................................................... 46 2.3 Literature Summary.................................................................................................. 47 3 Methodology ................................................................................................................... 48 3.1 Research Questions .................................................................................................. 48 3.2 Study Context and Participants ................................................................................ 49 3.3 Study Procedures ...................................................................................................... 51 3.3.1 Data Collection ................................................................................................. 51 3.3.2 Data Analysis .................................................................................................... 52 3.1 Ethical Considerations.............................................................................................. 65 3.2 Limitations ............................................................................................................... 65 v  3.3 Methodology Summary ............................................................................................ 66 4 Results and Discussion ................................................................................................... 67 4.1 Examining the Original CLASS Model ................................................................... 67 4.2 Generating a Revalidated Model .............................................................................. 68 4.2.1 Exploratory Factor Analysis ............................................................................. 68 4.2.2 Testing Reliabilities .......................................................................................... 72 4.2.3 Emergent Model................................................................................................ 77 4.3 Testing the Emergent Model .................................................................................... 79 4.4 Revalidated Model ................................................................................................... 82 4.5 Results Summary...................................................................................................... 84 4.6 Factors ...................................................................................................................... 85 4.7 A revalidated CLASS ............................................................................................... 87 4.8 Potential Implications of the Revalidated Model ..................................................... 88 4.8.1 Theory ............................................................................................................... 89 4.8.2 Research ............................................................................................................ 90 4.8.3 Curriculum Development.................................................................................. 91 4.9 Discussion Summary ................................................................................................ 94 5 Conclusions ..................................................................................................................... 96 5.1 Summary .................................................................................................................. 96 5.2 Conclusions: Answering the Research Questions .................................................... 97 5.2.1 Question 1: What are the factors underlying undergraduate students’ attitudes and beliefs about physics prior to participating in an introductory physics course? ...... 97 5.2.2 Question 2: What might the implications of these factors be on course design and consequent student physics learning? ...................................................................... 97 5.3 Future Research ........................................................................................................ 98 References ............................................................................................................................. 101 Appendices ............................................................................................................................ 116 Appendix A: The CLASS ................................................................................................. 116 Appendix B:  Pattern and structure matrices when seven factors are extracted from all 41 items of the survey ............................................................................................................ 118 Appendix C: Pattern and structure matrices for seven extracted factors after removing item 33 from the data. ............................................................................................................... 120 Appendix D: Pattern and structure matrices for six extracted factors after removing item 33 from the data ................................................................................................................ 122 vi  Appendix E: Pattern and structure matrices for five extracted factors after removing item 33 from the data ................................................................................................................ 124 Appendix F: Pattern and structure matrices for five extracted factors. Items 1, 2, 6, 9, 10, 13, 15, 16, 19, 20, 22, 23, 33, 36, and 38 have been removed. ......................................... 126    vii  List of Tables Table 2.1 Dimensions of the EBAPS tool. Structure of scientific knowledge, nature of knowing and learning, and real-life applicability overlap with MPEX dimensions, (italics) described below. An example Likert item from each dimension is shown. (Elby, 2001) ...................... 16 Table 2.2 The six dimensions of the MPEX survey. Favourable and unfavourable ways of thinking along each dimension are indicated (Redish et al., 1998). ...................................................... 18 Table 2.3: Example items from each of the six dimensions of the MPEX (University of Maryland Physics Education Research Group, n.d.) ............................................................................... 19 Table 2.4: The scientific and cognitive dimensions of VASS (Halloun, 1997) ................................. 23 Table 2.5 General Profile Characteristics (Halloun & Hestenes, 1998). ............................................ 25 Table 2.6 The CLASS categories and items, as proposed by the developers. Items with a * are not useful in their current forms, and need to undergo revisions. Italicized items are part of multiple categories (Adams et al., 2006). ............................................................................... 27 Table 2.7: The five items experts did not reach a consensus about in the UK ................................... 30 Table 2.8 A comparison of the CLASS, MPEX, MPEX2, EBAPS, and VASS tools (Adams et al., 2006; Elby, 2001; Halloun, 1997; McCaskey, 2009; Redish et al., 1998). ............................ 33 Table 2.9: An example of an interpretive argument for a trait interpretation ..................................... 42 Table 3.1: Outline of the topics covered in each course ..................................................................... 50 Table 3.2: Number of students in each data set .................................................................................. 50 Table 3.3: Five CLASS items were reverse-coded ............................................................................. 52 Table 3.4: Total variance explained by the extracted factors. ............................................................ 55 Table 3.5: The factor matrix for exploratory factor analysis with Promax rotation, using the items from the CLASS suggested to probe facets of problem solving. ............................................ 56 viii  Table 3.6: The pattern matrix for exploratory factor analysis with Promax rotation, using the items from the CLASS suggested to probe facets of problem solving. Shaded cells identify items that load to each factor. ........................................................................................................... 56 Table 3.7: The pattern matrix for exploratory factor analysis with Promax rotation, using the items from the CLASS suggested to probe facets of problem solving. Shaded cells identify items that load to each factor ............................................................................................................ 57 Table 3.8: Factor correlation matrix. There is a high correlation between the two extracted factors, which suggests that the two factors main in fact be a single factor. ....................................... 58 Table 4.1: Pattern matrix for the emergent model. Five factors are extracted. Items 1, 2, 4, 6, 8, 9, 10, 11, 13, 15, 16, 19, 20, 22, 23, 33, 36, 38, and 39 have been removed. Shaded items load on each factor. .............................................................................................................................. 70 Table 4.2: The structure matrix when five factors are extracted after removing items that did not load in the previous analysis. Shaded items belong to each factor. ................................................ 71 Table 4.3: Five extracted factors. In this model, all of the items load to a factor. .............................. 71 Table 4.4: Reliability analysis for Factor 1. ........................................................................................ 72 Table 4.5: Reliability analysis for Factor 2. ........................................................................................ 73 Table 4.6: Reliability statistics for Factor 2 after removing items 5 and 40. ...................................... 73 Table 4.7: Reliability analysis for Factor 3. ........................................................................................ 74 Table 4.8: Reliability analysis for Factor 2; item 25 has been added to the factor. ............................ 74 Table 4.9: Inter-item correlation matrix for Factor 4 .......................................................................... 75 Table 4.10: Inter-item correlation matrix for Factor 5 ........................................................................ 75 Table 4.11: Reliability statistics for the 22-item scale which emerged from exploratory factor analysis. ................................................................................................................................... 76 ix  Table 4.12: Reliability statistics for the 18-item scale resulting after removing items 35, 5, 40, and 32................................................................................................................................................. 77 Table 4.13: Emergent factor model and items. Model 1. .................................................................... 78 Table 4.14: Goodness-of-fit indices for Model 1................................................................................ 80 Table 4.15: Goodness-of-fit indices for Model 2................................................................................ 80 Table 4.16: Goodness-of-fit indices for Model 3................................................................................ 81 Table 4.17: Goodness-of-fit indices for Model 4................................................................................ 81 Table 4.18: Goodness-of-fit indices for Model 5................................................................................ 81 Table 4.19: Confirmatory factor analysis results for 153 – 2010. N = 601. ....................................... 81 Table 4.20: Confirmatory factor analysis results for five models, 101-2012 (N = 760) .................... 82 Table 4.21: Confirmatory factor analysis results for 101 – 2013 (N = 647) ...................................... 82 Table 4.22: Revalidated factor model underlying the CLASS instrument, as confirmed by this research. .................................................................................................................................. 83 Table 4.23: Factors that were found and subsequently removed from the emergent Model 1 during this research. ........................................................................................................................... 83 Table 4.24: The 16 items that were removed from the scale. Bolded items were initially part of a factor, but were subsequently removed. ................................................................................. 84 Table 4.25: The first factor in the revalidated model. ......................................................................... 86 Table 4.26: Second factor in the revalidated model ........................................................................... 86 Table 4.27: Third factor in the revalidated model .............................................................................. 87   x  List of Figures Figure 2.1: An example of a multiple-choice item (Elby, n.d.b) ............................................ 16 Figure 2.2: An example of a debate-style item from the EBAPS (Elby, n.d.b). ..................... 17 Figure 2.3: An example of a pair of statements (Halloun, 1997)  .......................................... 23 Figure 2.4 Contrasting Alternatives Design scoring method. (1) only a; (2) mostly (a), rarely (b); (3) more (a) than (b); (4) Equally (a) and (b); (5) more (b) than (a); (6) mostly (b), rarely (a); (7) Only (b) never (a); (8) neither (a) nor (b) (Halloun & Hestenes, 1998). .......................................................................................................................... 24 Figure 3.1: Factor structure and loadings for suggested factor structure underlying problem solving items for the CLASS instrument. ................................................................... 59   xi  Acknowledgements First and foremost, I would like to thank my supervisor Dr. Marina Milner-Bolotin. Her support, feedback, and advice were invaluable during the development, execution, and reporting of my research. I would also like to thank my committee members, Dr. Samson Nashon and Dr. James Carolan for their insight and guidance throughout this process.  I would like to thank the Department of Physics and Astronomy for allowing me access to the data used in this study. In particular, Dr. James Carolan has been instrumental in locating, organizing, and producing this data for my use.  To my friend Heather Fisher, for helping me navigate these past two years. I would truly have been lost without your friendship and support. Andrea Webb, Ashley Welsh, and Guopeng Fu, thank you for having the questions, the answers, the stories, and the joy.   And finally, to my family – your love and support are endless.   xii  Dedication For Margaret1  1 Introduction  Today’s society is heavily dependent on knowledge and innovation in science, technology, engineering, and mathematics (STEM) (“Spotlight on Science Learning,” 2013). Moreover, there is an increased demand for a scientifically literate society  (Huang & Anderson, 2010) and qualified people to pursue careers in STEM (Hue, Sales, Comeau, & Lynn, 2010). Most careers in these areas require relevant undergraduate degrees. Thus, it makes good sense to pay attention to undergraduate STEM education to ensure that students graduating with these degrees have received a quality education (Heilbronner, 2011). It is important to examine not only the quality but the appropriateness of the experiences students are receiving in the classroom and how these experiences impact undergraduate learning (Heilbronner, 2011). Due to the fundamental differences that exist between disciplines each area of STEM should be considered as its own entity (Blickenstaff, 2005; Hanson, 1996). These differences tend to shape students’ attitudes and beliefs about the individual STEM discipline. Numerous studies have been done in this regard, including the use of quantitative instruments to assess these personality constructs (Adams et al., 2006; Baldwin, Ebert-May, & Burns, 1999; Bauer, 2005, 2008). Previous studies have demonstrated that a broad range of factors affect a student’s decision to pursue and persist in a physics degree (Adeyemi, 2010; Ceci, Williams, & Barnett, 2009; Heilbronner, 2011; Kost, Pollock, & Finkelstein, 2009; Seymour & Hewitt, 1997; Smith, 2011; Wang, Eccles, & Kenny, 2013; Why aren’t more women in science?, 2007). Two of these factors are students’ attitudes and beliefs about physics and learning physics (Adeyemi, 2010). There also exists a body of literature on how attitudes towards science, and physics can impact students’ learning and perceptions of what learning physics should be like (Dickinson & Flick, 2  1998; Milner-Bolotin, Antimirova, Noack, & Petrov, 2011; Van Etten, Freebern, & Pressley, 1997).  To further explore these impacts, it is crucial to have accurate and reliable tools.  A range of quantitative tools that measure students’ attitudes towards physics and learning physics are discussed in the literature. These include tools that measure students’ attitudes and epistemological views about physics and learning physics (Adams et al., 2006; Elby, 2001; Redish, Saul, & Steinberg, 1998). Although these tools measure slightly different traits, the overarching goal is often to understand the factors which influence student learning and what students’ decisions say about their attitudes towards physics and physics related professions. The current study focuses on understanding the factors that underlie students’ attitudes and beliefs about physics, as implied by the Colorado Learning Attitudes about Science Survey (CLASS) (Adams et al., 2006) data for first year physics students at a Western Canadian University. The CLASS is intended for use at all levels of undergraduate physics learning, across all fields of physics, and was found to embody an eight factor model (Adams et al., 2006). However, an inspection of the CLASS data from different introductory physics courses at a large research university in Western Canada did not support the existence of the eight factor model. Moreover, the validity of the instrument has been revisited only twice since its development, and neither study examined the factor structure (Donnelly, MacPhee, Hardy, & Bates, 2013; Sawtelle, Brewe, & Kramer, 2009). The current study aims to investigate the factors that underlie students’ CLASS responses at a large research university in Western Canada through an iterative revalidation and confirmation process.  3  1.1 Background to the Study  This study evolved out of a desire to understand the factors influencing students’ persistence in undergraduate physics degrees to graduation. In particular, I aimed to describe the factors underlying students’ attitudes and beliefs about physics. The Physics Department at the study institution collects large quantities of data about students’ attitudes and beliefs towards physics using the CLASS (Adams et al., 2006). As a precursor to analysis, the structure and scoring method of the survey was examined. The CLASS is divided into eight categories, suggested by the developers’ personal experience as well as previous attitude surveys (Elby, 2001; Halloun, 1997; Redish et al., 1998), which were confirmed by statistical analysis (Adams et al., 2006). However, confirmatory factor analysis did not support the existence of this factor structure in the population of interest, and it was suggested that six additional constraints be placed on the model for the data to fit the model. This negative result, coupled with similar insights from an examination of the validity of the CLASS-Chem (Heredia & Lewis, 2012), furthered my desire to investigate the factors and factor structure that underlie undergraduate physics students’ CLASS data through a revalidation process.  The CLASS has been used in many studies to examine students’ attitudes and beliefs about physics and learning physics. However, except for a study examining whether or not students’ interpretations of the survey items are consistent at a primarily Hispanic institution (Sawtelle et al., 2009), no studies have been conducted to add to the validity evidence surrounding the instrument. In particular, no revalidation studies have been conducted to investigate the factors and factor structure underlying students’ attitudes and beliefs about physics implied by their responses to the CLASS. Similar gaps in the literature exist surrounding 4  other attitudes surveys (Elby, 2001; Halloun, 1997; McCaskey, 2009; Redish et al., 1998). It is often challenging to select appropriate research instruments, and the availability of validity evidence can streamline the decision (Heredia & Lewis, 2012). Moreover, given that validity of a test interpretation may only be context specific, there is always the possibility of multiple interpretations (Kane, 2009). This research study aims to add to the body of literature on the factors influencing undergraduate physic students’ attitudes and beliefs about physics and implications for curriculum, theory, and research.  There is a possibility that any differences between the factors at the institution where the data were collected for the initial CLASS validation and the factors discovered in the current study can be attributed to a considerable difference in the student populations at each institution. For example, 16% of all undergraduate students at the study institution are designated by the university as international students (Farrar, 2013), compared to only 3.7% at the University of Colorado, Boulder (Planning, Budget, and Analysis, 2013). Among first year students at the study institution last year, 29% of students identified as white, while 36% of students identified as Chinese and 7% identified as South Asian (Farrar, 2013). Among transfer students, 42% identified as white, 24% as Chinese, and 5% as South Asian (Farrar, 2013). The sample of students involved in the initial development of the CLASS was 80% Caucasian (Adams et al., 2006). These differences have effects on the factors and factor structure influencing or underlying the participants’ attitudes and beliefs about physics and supports the revalidation of the CLASS in this population..  5  1.2 Significance of this Research In order to conduct meaningful research, researchers need to be confident that the tools they use are reliable and measure the constructs of interest. Without evidence supporting the validity and reliability of research tools, researchers cannot generate dependable and valuable conclusions. This study demonstrates how factors influencing students’ attitudes and beliefs about physics might be a function of the context in which the subjects reside. This study also provides insight into how the validity and reliability of survey tools might be interpreted, a view not espoused in the present literature surrounding the CLASS (Adams et al., 2006). It is critical for an instrument’s validity and reliability to be interpreted in tandem with a consideration of the context (Heredia & Lewis, 2012). Thus this study examines the factors and factor structure that governed CLASS data on students’ attitudes and beliefs about physics through a revalidation of CLASS.  1.3 Researcher’s Perspective  I graduated with a Bachelor’s of Science in Physics and Mathematics from St. Francis Xavier University in Antigonish, Nova Scotia, in 2011. I have always been passionate about ways to attract more people to STEM degree programs, specifically Physics. During my undergraduate degree, I was employed as a lab instructor for first-year physics students. I helped undergraduate students to conduct and complete a variety of lab activities. As a result, I observed first-hand how students’ attitudes can change in a year, a week, or even over the course of a three-hour lab, and how these attitudes are sometimes slow to shift in any direction.  This background inspired my decision to investigate the factors that underlie students’ attitudes and beliefs about physics through an analysis of CLASS data. 6  1.4 Research Questions and an Overview of the Methodology  One way to determine the factors underlying students’ attitudes and beliefs about physics is to develop and validate an appropriate instrument using exploratory and confirmatory factor analyses. Often, there are already validated and published instruments already exist, which researchers can adopt and use to collect data. When such an instrument is used, it is vital to inspect the new data set for the existence of the previously determined and published factors, as this data collection usually occurs in a different population than the original validation. If the factors are confirmed, it supports the robustness of the factor. It is an indication of the universal nature of the factors and hence supports the universal validity of the instrument. Nonetheless, there are times when the initially determined factors cannot be confirmed in another data set obtained through the same instrument. In this regard, it is necessary to pursue a revalidation process to determine the factors and factor structure that underlie the respondents’ scores in the population of interest.  Although CLASS has been previously validated (Adams et al., 2006) and widely used to assess undergraduate students’ attitudes towards and beliefs about science (Adams et al., 2006; Alhadlaq et al., 2009; Gray, Adams, Wieman, & Perkins, 2008; Milner-Bolotin et al., 2011; Otero & Gray, 2008) there are reasons to examine the factors that underlie students’ attitudes and beliefs about physics at the study institution. The eight factors determined to underlie students’ attitudes and beliefs could not be confirmed in the CLASS data collected from students at the study institution. This finding compelled me to investigate the factors and factor structure underlying these students’ attitudes and beliefs about physics through a revalidation of CLASS using four distinct sets of CLASS data.  Therefore this study aims to investigate the following research questions:  7  1. What are the factors underlying undergraduate students’ attitudes and beliefs about physics prior to participating in an introductory physics course?  2. What might the implications of these factors be on course design and consequent student physics learning?  This study follows a quantitative design aimed at investigating the factors underlying students’ attitudes and beliefs about physics, and the implications these factors have for student learning. Data were collected from students enrolled in two introductory physics courses, one for engineers (n = 1193), and one for science students (n = 1407) using the CLASS. Statistical analysis of the collected data assisted in pinpointing the factors underlying students’ attitudes and beliefs about physics, as measured by the CLASS (Question 1). In order to provide suggestions for course design and the implications to student learning, an analysis of the items was performed, drawing on metacognitive theories (Question 2). The statistical analysis in this study differs from the recursive process used during the development of the CLASS (Adams et al., 2006). 8  2 Literature Review  This section will provide an overview of concepts and research relevant to this study. I begin with a discussion of attitudes and beliefs, including a definition of each term and their value to learning. I then describe a variety of quantitative tools that have been developed to measure and describe students’ attitudes about learning physics, including the Colorado Learning Attitudes About Science Survey (CLASS) (Adams et al., 2006). The CLASS has been used in the current study to investigate the factors underlying the students’ attitudes and beliefs about physics. An overview of research studies and their findings is then provided. Finally, a discussion of the theories which informed the statistical analysis in this study and the interpretation of the results is given.    Today’s society relies heavily on knowledge and innovation in science, technology, engineering, and mathematics (STEM). Moreover, there is a high level of importance placed on scientific literacy and an increased demand for qualified people to pursue careers in STEM fields (“Spotlight on Science Learning,” 2013). As a result, there is a continuing need for attention to understanding how students learn in order to improve conceptual understanding, attitudes towards science, and enrollment in science-related degrees (“Spotlight on Science Learning,” 2013). There has been particular focus on examining students’ incoming knowledge and attitudes and comparing these with their knowledge and attitudes following related course experiences (Adams, Perkins, Dubson, Finkelstein, & Wieman, 2005; Bates, Galloway, Loptson, & Slaughter, 2011; Hestenes, Wells, & Swackhamer, 1992; Kost et al., 2009; Kost-Smith, 2011; Milner-Bolotin et al., 2011; Otero & Gray, 2008; Perkins & Gratny, 2010; Pollock, 2005). These studies support the constructivist view that students are not blank slates and do not come to 9  education without prior knowledge, experiences, and ways of learning, and these prior notions inform and have a significant impact on how they learn (Driver, 1983; Kelly, 1955; McCaskey, 2009; Shulman, 1986). The skills and knowledge learned and required for learning in each field of STEM vary significantly. It is, therefore, important to conduct educational research in the individual fields of STEM (Blickenstaff, 2005). There exist a number of survey tools to examine attitudes towards science in general. However, these do not provide sufficient insight into students’ attitudes towards learning in each discipline and should be examined in tandem with other research evidence, especially when investigating the factors and factor structures that underlie respondents’ attitudes and beliefs about physics (Redish et al., 1998; Thomas, Anderson, & Nashon, 2008).  2.1 Attitudes 2.1.1 What are Attitudes? My review of the literature revealed many definitions for beliefs and attitudes (Aiken, 2002; Ajzen & Madden, 1986; Eagly & Chaiken, 1993; Osborne, Simon, & Collins, 2003). Hence, defining the terms beliefs and attitudes, like many other abstract concepts, is a difficult task for physics education researchers. Many education researchers have struggled for decades to agree on a singular definition of attitudes and beliefs, or a theory that underpins research in this area (Osborne et al., 2003). It is common for a variety of constructs to be included in a definition of “attitudes,” despite the distinct boundaries recognized in psychology, sociology, and education literature (Bauer, 2005). These constructs include attitude, beliefs, views, interests, values, self-efficacy, self-esteem, and self-concept (Bauer, 2005). To adequately assess and measure 10  students’ attitudes about learning physics, it is important to clearly operationalize definitions of attitudes and beliefs, and what an assessment tool intends to measure (Kane, 2006).   In this study, a belief is defined as the “personal knowledge or understandings that are antecedents of attitudes and subjective norms; establish behavioural intentions” (Ajzen & Madden, 1986; Fishbein & Ajzen, 1975, both as cited in Bauer, 2005, p. 1864). An attitude is “a psychological tendency that is expressed by evaluating a particular entity with some degree of favour or disfavour" (Eagly & Chaiken, 1993, p. 1 [emphasis in the orignial]). This implies that before an attitude can be formed, beliefs must be made. Mager proposes that an attitude is “a learned predisposition to respond favourably or unfavourably toward an attitude object” (as cited in Bauer, 2005, p. 1864 ). In this case, the attitude object is physics and physics learning. In the CLASS, these attitudes are framed as falling on a scale between “expert-like” and “novice-like” as defined by responses to the instrument provided by panel of academics in various fields of physics (Adams et al., 2006).  This study focuses on students’ attitudes and beliefs about science, specifically physics, as opposed to scientific attitudes. Attitudes and beliefs about science include an individual’s opinions about the importance of science, both for the individual and society, the difficulty associated with science, as well as their innate desire to learn science (Osborne et al., 2003). Conversely, Osborne (2003) defines scientific attitudes as characteristics such as critical thinking skills, curiosity, the requirement of validity, and respect for knowledge.  2.1.2 Attitudes and Learning A high level of importance has been placed on scientific literacy (Huang & Anderson, 2010; “Spotlight on Science Learning,” 2013) and on increasing the number of people choosing 11  careers in these fields (Hue et al., 2010). For these reasons, one of the main goals of science education in Canada is to increase students’ scientific literacy (Council of Ministers of Education, Canada, 1997). Scientific literacy includes critical thinking skills and the ability to apply scientific concepts and ways of thinking to authentic situations outside of the classroom (CMEC, 1997; Gormally, Brickman, & Lutz, 2012).  In light of this need for a scientifically literate population, many curricular and pedagogical changes have been and continue to be implemented worldwide, including Peer Instruction (Mazur, 1997) and Modeling Instruction (Hestenes, 1987). By reforming the way today’s students are taught, it is possible to affect the quality of their learning and increase their interest in STEM fields. It is important to examine the success of these interventions, and the impact they have on student learning, interest in science, scientific literacy, and desire to pursue careers in the sciences (Heilbronner, 2011).  Student success in science learning can be measured in many different ways. Frequently, success is measured in terms of grades earned on a course. By comparing grades, we can place students on a scale which indicates academic proficiency. It is important to be cautious when examining academic standing based solely on grades. For example, some research indicates  that female students obtain higher marks on assignments that are completed outside of class and over a longer period of time, while male students excel in in-class, high-stakes assessments such as exams and tests, with no significant statistical difference between final grades by gender (Kost et al., 2009). Perhaps a more efficient way to examine student success is to interpret learning gains, which are based upon a student’s knowledge and ability before and after instruction, and the understanding that a student’s scores can only increase or decrease as much as their initial scores 12  allow. It is also possible to measure student success through standardized testing, formative assessments, and conceptual understanding inventories.  It is generally believed that as a result of instruction, students’ attitudes towards science should shift towards being more expert-like (Adams & Wieman, 2011; Slaughter, Bates, & Galloway, 2011). Students’ attitude, views, expectations, and epistemological beliefs are vital to their learning (Brewe, Kramer, & O’Brien, 2009) and impact how they conceive of science knowledge, the value they associate with science, and whether or not they perceive science to apply to the real world (Bates et al., 2011). These attitudes also affect how students perceive university science courses and careers in science (Milner-Bolotin et al., 2011; Redish et al., 1998). Additionally, attitudes can serve to differentiate between experts and novices in the field (Brewe et al., 2009).  Negative attitudes towards physics and learning physics can contribute to the current low levels of enrollment in physics departments across Canada (Milner-Bolotin et al., 2011). Whereas positive attitudes, conceptions of relevance, and personal interest are correlated with a desire to pursue physics beyond high school (Briggs, 1976; Perkins, Adams, Pollock, Finkelstein, & Wieman, 2005). By identifying students’ attitudes at the outset of a course, an instructor can attempt to develop a plan to address their students’ strengths and weaknesses, with the intent of influencing their attitudes towards being more expert-like (Adams & Wieman, 2011; Bransford, Brown, & Cocking, 2000; Milner-Bolotin et al., 2011). Consequently, the intent is that more students will choose to pursue studies in physics if their attitudes align more with those of experts (Adams & Wieman, 2011).  Researchers found that students who intend to pursue physics as a major exhibit more expert-like attitudes at the beginning of introductory physics courses than those who do not (Perkins & Gratny, 2010; Slaughter et al., 2011). 13  2.1.3 Instrumentation and Assessment of Attitudes  A variety of survey tools have been developed to help instructors identify and understand students’ attitudes about physics and learning physics (Adams et al., 2006; Elby, 2001; Halloun & Hestenes, 1998; Halloun, 1997; Redish et al., 1998). These survey tools have been designed for implementation before and after instruction, and provide the instructor (and researchers) with insight into how students’ attitudes shift over the course of an instructional period. By design, the tools follow a constructivist theory by drawing upon students’ initial knowledge, which supports the idea that students do not come to instruction as blank slates (Smith III, diSessa, & Roschelle, 1994). These tools also aim to illuminate how students approach physics knowledge and learning, highlighting how they construct their own understandings. Through awareness and acceptance of students’ ways of knowing and learning at the beginning of a course, it is possible that instructors can make conscious decisions to support their students in thinking more like experts in the field.    A plethora of instruments have been developed to examine attitudes towards a variety of topics, both within and outside of educational settings. While this research focuses particularly on measuring attitudes towards learning physics, it is valuable to note the existence of a range of available instruments with similar intentions.   In physics education, the Epistemological Beliefs Assessment for Physical Science (EBAPS), Maryland Physics Expectations survey (MPEX and MPEX2), and the Colorado Learning Attitudes About Science Survey (CLASS) have been a significant part of the literature for the past two decades (Adams et al., 2006; Elby, 2001; McCaskey, 2009; Redish et al., 1998). The three surveys are constructed with different aims in mind, and are designed specifically for physics education. The CLASS development team used MPEX as a model and guide for the 14  development of their tool. These three, as well as the Views About Science Survey (VASS) (Halloun, 1997) are described in further detail below.   Three alternate versions of the CLASS have been developed to focus on chemistry (CLASS-Chem), biology (CLASS-Bio), and experimental physics (E-CLASS) (Barbera, Adams, Wieman, & Perkins, 2008; Semsar, Knight, Birol, & Smith, 2011; Zwickl, Finkelstein, & Lewandowski, 2012).  Other physics attitudes surveys include the Self-Concept and Competence Scale in Physics (SCACSP) and the Exploring Physics Confidence Survey (EPCS). In chemistry, the Chemistry Self-Concept Inventory (CSCI), Chemistry Attitude and Experience Questionnaire (CAEQ) and Attitude Toward the Subject of Chemistry Inventory (ASCI) were developed for use in college and university chemistry courses (Bauer, 2005, 2008; Dalgety, Coll, & Jones, 2003). The College Biology Self-Efficacy Instrument (CBSEI) is also used in biology (Baldwin et al., 1999). Many tools have been developed that focus on attitudes towards science in general including Views of the Nature of Science (VNOS), Attitude Toward Science in School Assessment (ASTSSA), Scientific Attitude Inventory (SAI II), Modified Attitudes Towards Science Inventory (mATSI), and Changes in Attitudes About the Relevance of Science (CARS) (Germann, 1988; Halloun, 1997; Hong & Lin-Siegler, 2012; Lederman, Abd-El-Khalick, Bell, & Schwartz, 2002; Moore & Foy, 1997; Siegel & Ranney, 2003).  2.1.1.1 EBAPS  The Epistemological Beliefs Assessment for Physical Sciences (EBAPS) was developed to service high school students, as opposed to CLASS, MPEX, and VASS which were designed primarily for college and university level physics students (Elby, 2001). EBAPS focuses on examining students’ epistemological beliefs about physics learning, separate from expectations 15  about a physics course. This prompted the developers to move away from items in which the ‘answer the teacher wants’ is relatively obvious, which could compel students to choose this kind of teacher-centered answer over what they really believe (Redish, 2003a). To separate these views, the EBAPS asks students to choose what they would do in a given situation, as opposed to what they think about it. To further accomplish this, EBAPS is not limited to agree-disagree items and includes multiple-choice questions and mock-debates about important topics (Elby, 2001; Redish, 2003a) The EBAPS consists of 30 items: 17 agree-disagree, six multiple-choice (e.g. Figure 2.1), and seven debates (e.g. Figure 2.2). These items are then divided into five subscales, the first three of which overlap with MPEX subscales (Elby, 2001). These scales are structure of knowledge, nature of learning, real-life applicability, evolving knowledge, and source of ability to learn (Table 2.1).   16  Table 2.1 Dimensions of the EBAPS tool. Structure of scientific knowledge, nature of knowing and learning, and real-life applicability overlap with MPEX dimensions, (italics) described below. An example Likert item from each dimension is shown. (Elby, 2001) Dimension Description Likert Item Structure of Scientific Knowledge (Concepts, Coherence) Is science a collection of weakly or strongly connected pieces of information?  When it comes to understanding physics or chemistry, remembering facts isn’t very important.    Nature of Knowing and Learning (Independence) Is information absorbed or constructed? When learning science, people can understand the material better if they relate it to their own ideas.     Real-Life Applicability (Reality Link)  Do scientific ways of thinking have a purpose outside of science-specific contexts? General view of the relevance of science, as opposed to an individual’s desire to apply science.  Understanding science is really important for people who design rockets, but not important for politicians.    Evolving Knowledge Where do students fall on the continuum between absolutism and extreme relativism?  When it comes to controversial topics such as which foods cause cancer, there’s no way for scientists to evaluate which scientific studies are the best.  Everything’s up in the air!    Source of ability to learn Who is able to learn science? Distinct from beliefs about personal ability, focusing on the value of effort and strong learning strategies.  Someone who doesn’t have high natural ability can still learn the material well even in a hard chemistry or physics class   Figure 2.1: An example of a multiple-choice item (Elby, n.d.b) 22. To be successful at science... a. Hard work is much more important than inborn natural ability. b. Hard work is a little more important than natural ability. c. Natural ability and hard work are equally important. d. Natural ability is a little more important than hard work. e. Natural ability is much more important than hard work 17   Figure 2.2: An example of a debate-style item from the EBAPS (Elby, n.d.b). Similar to the other surveys discussed here, EBAPS is scored on a single scale based on expert views (Elby, 2001). Scores range from 0 to 100, with a 0 indicating very unfavourable, 50 neutral, and 100 very favourable. Due to the differing question types, these values do not represent the percentage of responses a student gives that correspond to an expert view. Each item is given a score from 0 (not sophisticated) to 4 (very sophisticated), which can include half-point scores (Elby, n.d.a). For example, an item can have the scoring key of A (4) B (2.5) C (2) D (0) E (1.5). The scheme was designed to account for variances in the levels of sophistication presented by options on each item (Elby, n.d.a).  2.1.1.2 MPEX  The Maryland Physics Expectations survey (MPEX) (Redish et al., 1998) aims to measure  students’ expectations about physics and learning physics as they begin a physics course. Students’ expectations at the outset of a course will have an impact on what and how they learn (Redish, 2003b). Through developing a deeper understanding of students at the outset 26. Justin:  When I’m learning science concepts for a test, I like to put things in my own words, so that they make sense to me. Dave: But putting things in your own words doesn't help you learn. The textbook was written by people who know science really well. You should learn things the way the textbook presents them.  a. I agree almost entirely with Justin. b. Although I agree more with Justin, I think Dave makes some good points. c. I agree (or disagree) equally with Justin and Dave. d. Although I agree more with Dave, I think Justin makes some good points. e. I agree almost entirely with Dave.  18  of a course, it is possible to develop a course that will focus on developing positive understandings of physics concepts.   The MPEX survey was designed to enable researchers and educators to examine large populations of students simultaneously at one time. The survey consists of 34 multiple-choice items where responses are indicated on a five-point Likert scale ranging from strongly disagree to strongly agree. The items are separated into six facets which describe various dimensions of students’ attitudes about how to do physics and their approach to learning physics. Favourable and unfavourable responses for each dimension are generalized in Table 2.2. These dimensions are independence, coherence, concepts, reality link, math link, and effort. These dimensions are not unique, and some items contribute to more than one dimension. This is to be expected, as the dimensions are not independent of each other.  Table 2.2 The six dimensions of the MPEX survey. Favourable and unfavourable ways of thinking along each dimension are indicated (Redish et al., 1998). Dimension Favourable Unfavourable Independence Takes responsibility for constructing own understanding Takes what is given by authorities (teacher, text) without evaluation    Coherence Believes physics needs to be considered as a connected, consistent framework  Believes physics can be treated as unrelated facets or “pieces”    Concepts Stresses understanding of the underlying ideas and concepts Focuses on memorizing and using formulas    Reality Link Believes ideas learned in physics are relevant and useful in a wide variety of real contexts Believes ideas learned in physics have little relation to experiences outside the classroom    Math Link Considers mathematics as a convenient way of representing physical phenomena  Views the physics and the math as independent with little relationship between them.     Effort Makes the effort to use information available and tries to make sense of it Does not attempt to use available information effectively  19   The survey items were chosen based on an extensive literature review and experience with undergraduate physics students. Example items from each dimension are given in Table 2.3.  Table 2.3: Example items from each of the six dimensions of the MPEX (University of Maryland Physics Education Research Group, n.d.) Dimension Item Independence Only very few specially qualified people are capable of really understanding physics.   Coherence Knowledge in physics consists of many pieces of information each of which applies primarily to a specific situation.   Concepts Learning physics is a matter of acquiring knowledge that is specifically located in the laws, principles, and equations given in class and/or in the textbook.   Reality Link Physical laws have little relation to what I experience in the real world.   Math Link In doing a physics problem, if my calculation gives a result that differs significantly from what I expect, I'd have to trust the calculation.   Effort I spend a lot of time figuring out and understanding at least some of the derivations or proofs given either in class or in the text.  Student interpretation of the items was validated through interviews, in which students provided explicit examples from their classes. During these interviews, students gave opposing answers to similar items. This result indicated that the students’ lacked a clear understanding of the nature of physics. These understandings were also tempered by other factors, such as the context in which the item is presented. As a result, these items do not contribute to a student’s score. This is another example of why it is important to be cautious when interpreting the results of a survey, as it is possible for different situations to elicit opposing attitudes.   Similar to the CLASS tool (Adams et al., 2006), the MPEX survey is scored in relation to expert responses (Redish et al., 1998). Seasoned physics instructors (considered to be experts) were consulted to develop this scoring baseline. To validate this scoring key, five groups of physics learners and teachers were consulted. These groups included first-year undergraduate 20  students, high school teachers, and university faculty with the intent to include a range of abilities. It was expected that an increasing level of expert-like expectations would be observed. The scores upheld this hypothesis, and the researchers determined that the tool appropriately measures how expert-like an individual’s expectations are about physics.  2.1.3.1.1 MPEX II  Nearly a decade following the initial development of the MPEX, McCaskey (2009) chose to update the survey in hopes of improving the validity and usefulness of the tool. This included identifying problematic items and clusters (factors) which were either altered or removed, and the addition of several new items. A qualitative and quantitative investigation of the validity of the tool was then performed.   The 34 items of the MPEX are divided into six dimensions (Tables 2.2, 2.3). However, Saul and others have consistently observed a deterioration amongst scores in the effort cluster across various implementations of the tool (as cited in McCaskey, 2009). The large decreases could have various explanations, including a “New Year’s Resolution” effect, and so the five items were removed from the MPEX II. An additional 13 items were removed, three of which were eliminated due to the extent to which they depended on the context of the course. One goal of the redesign efforts was to ensure that all items were independent from course context and the student’s level of conceptual understanding. This is important, as it is difficult for a student to recognize and be aware of the context with which they respond to an item; whether it is a personal or course-related context. In order to overcome context-related interpretation issues, debate items similar to those seen on EBAPS (Elby, 2001) were included. These items were phrased to probe students’ behaviour in authentic situations. In addition to these debate-style 21  questions, five Likert scale items were chosen from EBAPS and reworded to be specific to a physics context.   All items on the MPEX II were written in future tense to provide consistency between those students who have not yet taken a physics course and those who have. Each item was rewritten in the third person. This shift away from first person separates each response from whether or not a student learned something as a result of instruction. In addition to this distinction, items were redesigned to provide clear separations between classroom, lab, and real world experiences.   In order to validate the MPEX2, students participated in interviews following completion of a pencil-and-paper version of the tool. This enabled identification of possible ambiguities amongst the items, particularly between course expectations and general epistemological views. These ambiguities are detrimental to the usefulness of the MPEX2 as the tool is designed to examine students’ expectations of a course.  The clusters of the MPEX were largely maintained in the MPEX2, and these were not subjected to statistical analysis. Researchers discuss the appropriateness of using factor analysis to determine and confirm reliable clusters amongst psychological data, describing the advantages and disadvantages of relying too heavily on the statistical analysis (Yerdelen-Damar, Elby, & Eryilmaz, 2012). McCaskey (2009) chose to forgo factor analysis as he was more interested in the context-rich validity of the tool. McCaskey then suggested that the clusters should be interpreted as instructional goals, and instructors should strive to see improvements within each goal.  22  2.1.1.3 VASS  The Views About Science Survey (VASS) was developed at Arizona State University and Lebanese University to assess the nature of the relationship between how students view knowing and learning science, and how they actually understand science (Halloun, 1997). This survey is different from the other physics attitudes surveys mentioned in this chapter but provides interesting insight into another response system aimed at a similar goal of separating expert and novice views about science. The survey was created following research findings that the views educators anticipate that students, at all levels, will develop during a science course do not align with the views students hold on to (as cited in Halloun, 1997).    The survey is designed to examine students’ views along two dimensions, scientific and cognitive. The scientific dimension focuses on foundational views of science while the cognitive dimension is aimed at how an individual thinks about learning science (Table 2.4). Both are further divided into three categories.  23  Table 2.4: The scientific and cognitive dimensions of VASS (Halloun, 1997) Category Description Scientific Dimension Structure of Science Science as a coherent vs. loosely related body of information   Methodology The scientific method applies in all situations. Math is a tool for interpreting and understanding instead of a source of information.    Validity of Science Scientific knowledge is not permanent, and can be called into question.  Cognitive Dimension Learnability It is possible for anyone to learn science. Achievement is determined by the amount of effort put forward.    Reflective Thinking In order to develop a valuable understanding of science, a learner must: 1. Focus more on understanding concepts 2. Approach situations from a variety of angles 3. Be aware of discrepancies in their knowledge 4. Construct and reconstruct knowledge to be useful to the learner   Personal Relevance Science is pertinent in the real world, and is directly relevant to everyone, not just scientists.    The tool consists of 30 items: 13 to measure the scientific dimensions; and 17 to measure the cognitive dimensions. Each item consists of a pair of contrasting statements which represent primary and opposing views that have been consistently observed in the scientific and lay communities respectively (Figure 2.3) (Halloun & Hestenes, 1998). Participants who completed the survey would then identify the extent to which they agree with either statement on an 8-point scale in a Constrasting Alternative Design (Figure 2.4) (Halloun, 1997).   Figure 2.3: An example of a pair of statements (Halloun, 1997)   The laws of physics are: 1. Inherent in nature of things and independent of how humans think 2. Invented by physicists to organize their knowledge about the natural world 24   Figure 2.4 Contrasting Alternatives Design scoring method. (1) only a; (2) mostly (a), rarely (b); (3) more (a) than (b); (4) Equally (a) and (b); (5) more (b) than (a); (6) mostly (b), rarely (a); (7) Only (b) never (a); (8) neither (a) nor (b) (Halloun & Hestenes, 1998).  Similar to other attitude surveys in physics, VASS is scored based on expert views (Halloun & Hestenes, 1998). Physics teachers and professors completed the survey, allowing the developers to generate standard profiles against which students’ views could be compared and classified. Most often, experts chose polarized views at one end of the scale, choosing one statement completely over the alternative. This response describes the expert view for each item. A small group of teachers chose middle options characterized by a mixed view between being an expert and being a novice. These are the transitional profiles. Finally, a folk view is defined as the opposite of the expert view, a choice of the opposing view entirely over the primary view. Student profiles are determined by the number of items they answer with expert or folk views (Table 2.5). These profiles were created following an in-depth analysis of the responses teachers provided to the survey. High transitional and low transitional are demonstrative of the gradual change in views from folk, to mixed, to expert. Students who are classified as experts along the cognitive dimension are said to be critical learners, while the others are either passive learners or lost. Along the scientific dimensions, students range from scientific realists to naïve realists (Halloun, 1997).    25  Table 2.5 General Profile Characteristics (Halloun & Hestenes, 1998). Profile Number of Items out of 30 Expert 19 items or more with expert views High Transitional 15 to 18 items with expert views Low Transitional 11 to 14 items with expert views and an equal or smaller number of items with folk views Folk 11 to 14 items with expert views but a larger number of items with folk views OR 10 items or less with expert views  The VASS has been updated since this iteration, although it is not commented on here. The dimensions and scoring structure have remained relatively static, and the updated version is available  2.1.1.4 CLASS The Colorado Learning Attitudes about Science Survey (CLASS) (Adams et al., 2006) was developed at the University of Colorado to adequately measure students’ beliefs towards physics over the course of a semester. The instrument was developed in tandem with a standard procedure for analyzing the collected data (Adams et al., 2006).  The CLASS was developed based on previous surveys and made use of various items from each of these tools, and new items were added. Items were chosen and designed to focus on physics as an entire field, as opposed to a particular physics course (Adams et al., 2006). The items were also written to be clear to a student who had little or no prior experience in physics (Adams et al., 2006).  Students are required to provide their course, student ID, and gender when completing the CLASS survey, which is frequently administered online. The survey (Appendix A: The CLASS) consists of 42 items answered on a five-point Likert scale that ranges from strongly 26  disagree to strongly agree. These ratings are then reduced to a three-point scale (disagree, neutral, and agree). Six items (4, 7, 9, 31, 33, 41) are unscored and do not contribute to an individual’s overall score because experts did not reach a consensus when answering these items, or the item is not informative as currently written. These six unscored items are still included in order to provide instructors and researchers with further information, representing a confusion between the pedagogical value of a measurement and the value of that measurement towards measuring a construct or trait. Of the remaining 36 items, 27 are divided amongst eight categories (Table 2.6). These categories are: ‘Personal Interest’; ‘Real World Connections’; ‘Conceptual Understanding’; ‘Applied Conceptual Understanding’; ‘Sense Making/Effort’; ‘Problem Solving General’; ‘Problem Solving Sophistication’; and ‘Problem Solving Confidence’. The categories are not unique, and many items are part of multiple categories. These items are indicated by italics. Nine statements are uncategorized (Adams et al., 2006). Due to the number of uncategorized statements, this research aims to uncover possible factors that underlie the observable items and explain students’ attitudes about learning physics.    27  Table 2.6 The CLASS categories and items, as proposed by the developers. Items with a * are not useful in their current forms, and need to undergo revisions. Italicized items are part of multiple categories (Adams et al., 2006). Category Items Example Item Real World Connections 28, 30, 35, 37 Learning physics changes my ideas about how the world works. (28)    Personal Interest 3, 11, 14, 25, 28, 30 I enjoy solving physics problems. (25) Sense Making/Effort 11, 23, 24, 32, 36, 39, 42 I am not satisfied until I understand why something works the way it does. (11)    Conceptual Understanding 1, 5, 6, 13, 21, 32 Knowledge in physics consists of many disconnected topics. (6)    Applied Conceptual Understanding 1, 5, 6, 8, 21, 22, 40 When I solve a physics problem, I locate an equation that uses the variables given in the problem and plug in the values. (8)    Problem Solving: General 13, 15, 16, 25, 26, 34, 40, 42 In physics, mathematical formulas express meaningful relationships among measurable quantities.(26)    Problem Solving Confidence 15, 16, 34, 40 I can usually figure out a way to solve physics problems. (34)    Problem Solving Sophistication 5, 21, 22, 25, 34, 40 If I want to apply a method used for solving one physics problem to another problem, the problems must involve very similar situations. (22)    Not Scored 4, 7*, 9, 31, 33, 41* It is useful for me to do lots and lots of problems when learning physics. (4)    Uncategorized 2, 10, 12, 17, 18, 19, 20, 27, 29, 38 There is usually only one correct approach to solving a physics problem. (10)   The survey categories were generated using a recursive process, which involved examining both predetermined categories, based on literature and the researchers’ experience, and statistically generated categories generated through exploratory factor analysis. Each potential category was examined separately from the others using Principle Component Analysis (J. D. Brown, n.d.; Muijs, 2011). The existence of an underlying factor structure was not 28  investigated (Adams et al., 2006). This is another reason this study aims to suggest a factor structure underlying the variables, as this process of determining factors produces a structure of little psychometric value, consisting of subscales that cannot necessarily be accurately measured concurrently (T. A. Brown, 2006b; Heredia & Lewis, 2012). Each step of the process employed reduced basis factor analysis, involving only the items assumed to belong to the category, as well as several items that might fit. After the initial analysis, ill-fitting items were removed, or the category was analyzed as two separate categories. Additional items, observed to be highly correlated with the current category in question, were then added to the analysis. This sequence of events was repeated until each category consisted of at least three items and was considered to be robust. A good category consisted of items with high, and similar, factor loadings (above 0.7). Category robustness was determined using a formula that related the average absolute value of the correlational coefficients between statements (cc), the average absolute value of the factor loadings for the category (fl), the shape of the scree plot generated through factor analysis (ΔE), the number of statements in the category (N), and the Pearson product moment correlation (R2) (Equation 1). The closer a value is to ten, the more robust the category.   (1)  Three controls are built-in to the survey to limit data from students who have randomly chosen answers. First, item #31 reads: “We use this statement to discard the survey of people who are not reading the statements. Please select agree (not strongly agree) for this statement.” If students fail to answer correctly, their results are scrutinized and often rejected. Second, a timer is included with the online survey. Students who complete the survey in three minutes or less are discarded. This control is not used at the study institution. Third, if students answer most of the 2352 RNEflccRobustness  29  questions with the same response (25 of 42) their results will be discarded. Additionally, a student’s scores are rejected if they fail to respond to six or more of the scored items.  Students’ CLASS responses are scored in relation to the answers expected from experts in the field, allowing for immediate comparison between students and experts. In the scoring of the survey, “strongly agree” and “agree” are treated as equal responses, as are “strongly disagree” and “disagree”. This is to account for discrepancies between students’ interpretations of agree and strongly agree. Each statement is then scored based on whether it aligns with the expert response: 1 – agree; 0 – neutral; -1 – disagree. Each student then receives a “percent favourable” and a “percent unfavourable” score, which describes the extent to which the student’s beliefs and attitudes line up with those of experts. The possibility exists that a student will neither agree nor disagree with the expert response, providing a neutral response; so it is not required for the % favourable and % unfavourable scores to add to 100%. Within each category, percent favourable and unfavourable scores are also calculated. Due to the nature of the scoring, with overall scores and category scores, it is important that the factor structure of the survey holds in the population in question.  An individual’s CLASS score is determined based on how many responses match with the expert response key(defining how expert-like a student’s attitudes are) (Adams et al., 2006). This key was determined based on how 16 experts answered the survey tool. A recent study in the UK examined the validity of this scoring scheme by asking (421) experts (including 162 academics, 56 post-doctoral fellows, 115 post-graduate students, and 53 industry members) to complete the survey (Donnelly et al., 2013). To confirm expert agreement with the pre-determined CLASS expert scoring, a threshold of 66% was chosen. This threshold was met and 30  surpassed for 35 of the 41 items (the 42nd item being control-item 31). However, five items failed to obtain a 66% agreement amongst academics (136 males, 26 females) (Table 2.7).  Table 2.7: The five items experts did not reach a consensus about in the UK Item 5 After I study a topic in physics and feel that I understand it, I have difficulty solving problems on the same topic. 12  I cannot learn physics if the teacher does not explain things well in class. 14 I study physics to learn knowledge that will be useful in my life outside of school. 16 Nearly everyone is capable of understanding physics if they work at it. 37 To understand physics, I sometimes think about my personal experiences and relate them to the topic being analyzed.  Additionally, there existed differences in the number of male and female academics whose responses agreed with the CLASS scoring scheme (Donnelly et al., 2013). These discrepancies suggest two things. First, expert responses might depend on context. The original scoring scheme was developed in the United States, while this study was conducted in the United Kingdom. It is possible that experts in different populations hold different attitudes. Second, it might be prudent to re-examine how the CLASS is scored, or to generate a separate scoring scheme to be used in different contexts, where influences on students’ responses may vary. However, this data only represent one new context, and additional data are needed to come to a stronger conclusion about context-dependency of the scoring scheme. The current study represents an effort to examine the structure of the factors underlying students’ attitudes and beliefs in a population different from that in which the original validation study took place.  Survey items are carefully chosen to provide researchers with the most accurate representation of a construct in an individual. For the CLASS, items were chosen to meet with several important criteria (Adams et al., 2006). First, items were designed to cater to a wide range of issues pertinent to learning physics. Second, the wording was carefully chosen to be clear and concise, with only a single possible interpretation. Third, items were designed to hold 31  meaning for a student who has never taken a physics course. Fourth, “expert” and “novice” answers were required to be unambiguous. Fifth, statements were to be valuable, valid, and reliable, in order to keep the length of the survey to a minimum. Sixth, although not related to item choice, the scoring had to be straightforward and automated. Finally, items were to fit into statistically robust categories. Additionally, the items were designed to cater to physics as a complete academic discipline, as opposed to individual topic areas or a single course.  In validating the original statements, 42 students were interviewed (Adams et al., 2006). These students were meant to represent a diverse population, consisting of equal numbers of male and female students, as well as 20% non-Caucasian students. However, this sample is not descriptive of the population at all post-secondary institutions, and it is possible that this sample could create a bias that would not be present at an institution that has a higher ethnic population (Sawtelle, Brewe, & Kramer, 2012). It is for this reason that Sawtelle, Brewe, and Kramer (2012) chose to investigate how students interpreted the items at an institution where the population is 60% Hispanic. Of the 42 items on the CLASS, 20 were chosen to discuss with students during interviews. These items were based on one of two factors: an expectation of misinterpretation based on previous experience with MPEX, or a desire to further investigate students’ cognitive attitudes surrounding an item (Sawtelle et al., 2012). Student responses were then coded as being “consistent”, “inconsistent”, or “undetermined”, in relation to the single intended interpretation outlined by the CLASS developers.  Overall, 94% of students interpreted the items in the intended frame. However, two items caused particular issue: item 6 and item 21. Only 62% of students correctly interpreted item 21 (Appendix) (Sawtelle et al., 2012). It is interesting to note that students who misinterpreted the item still chose the “expert” response (disagree). However, students believed the item asked 32  about finding the answer to the question, as opposed to a single equation used to solve the question (Sawtelle et al., 2012). Item 6 (Knowledge in physics consists of many disconnected topics) was correctly interpreted by 74% of students. Again, students chose the “expert” response (strongly disagree) based on a false understanding of the item. The item is intended to probe whether or not students see connections between the various areas of physics. The interviewed students interpreted the item as asking about the extent to which physics is connected to topics outside of physics, such as biology, chemistry, and mathematics (Sawtelle et al., 2012). The remaining items were interpreted by at least 86% of students.  Although in general, students correctly interpreted the intended meaning of the items, the fact that there was some confusion is noteworthy. The CLASS Items were designed to that each would only be open to a single interpretation by all possible users – including novices and experts (Otero & Gray, 2008). Despite this positive result, researchers should not take for granted that the students in their population of interest will correctly interpret the items of the tool.  The preceding section has outlined the various available instruments that quantitatively measure students’ attitudes, expectations, and epistemologies about physics and physics learning. Table 2.8 provides a summary of the similarities and differences across the above surveys. This comparison includes descriptions of: number of items, types of items, number of facets described by the items, how the facets were chosen, how scores are calculated, and how the items were designed.       33  Table 2.8 A comparison of the CLASS, MPEX, MPEX2, EBAPS, and VASS tools (Adams et al., 2006; Elby, 2001; Halloun, 1997; McCaskey, 2009; Redish et al., 1998). Survey Items Item Type Number of Categories Categories Determined by Scoring Item Design CLASS 42 5-point Likert 8 “Statistically robust” categories based on factor analysis, inter-item correlations, and reliability, hypothesized from literature and experience. Based on expert responses. Percent favourable/unfavourable As clear as possible, single interpretation, applicable across physics disciplines. Designed with undergraduate students in mind.  MPEX 34 5-point Likert 6 Categories chosen based on literature review and experience with physics education.   Expert responses.  Aimed at probing students’ expectations about a specific course. Written to be used in many courses. MPEX II 37 5-point Likert (25) Multiple-choice (8) Debates (4) 3 clusters and 5 sub-clusters Chosen to convey rough instructional goals Expert responses Aimed at probing students’ expectations about a specific course. Written to be used in many courses. EBAPS 30 5-point Likert (17) Multiple-choice (6) Debates (7) 5 Categories overlap with MPEX Based on agreement with expert responses, ranging from 0 to 100. 0 – total disagreement; 100 – total agreement.  Designed for use in high school and undergraduate education.  Aims to probe what students do instead of what they think.  VASS 30 Contrasting Alternatives Design (CAD) 2 dimensions, 3 categories in each  Unclear in the literature Four categories based on number of favourable (expert-like) responses (expert, high transitional, low transitional, folk) Each item consists of a pair of statements. Students must then identify the extent to which they agree with one over the other on an 8-point scale (figure X). 34  2.1.3.1.2 Shifts in Attitudes Numerous studies have been conducted using these instruments in introductory and upper year courses, courses for majors, courses for non-majors, and engineering courses across institutions in the United States, Canada, Saudi Arabia, and the United Kingdom (Adams et al., 2006; Alhadlaq et al., 2009; Barrantes et al., 2009; Brewe et al., 2009; Brewe, Traxler, de la Garza, & Kramer, 2013; Elby, 2001; Gire, Jones, & Price, 2009; McCaskey, 2009; Milner-Bolotin et al., 2011; Otero & Gray, 2008; Redish & Hammer, 2009; Slaughter et al., 2011; Slaughter, Bates, & Galloway, 2012). The findings from these studies have, for the most part, been consistent in showing that students’ enrolled in traditional introductory physics courses demonstrate a shift away from expert-like attitudes towards physics (Adams et al., 2006; Kost-Smith, 2011; Perkins, Gratny, Adams, Finkelstein, & Wieman, 2006; Slaughter et al., 2011). A few courses, designed to explicitly target students’ attitudes and epistemological beliefs, resulted in positive shifts (Brewe et al., 2009, 2013; Elby, 2001; Perkins et al., 2005; Redish & Hammer, 2009). Another course which included several educational reforms showed positive shifts (Pollock, 2005), while a single course that included many active engagement pedagogies led to positive shifts (Milner-Bolotin et al., 2011). Few studies show mixed results, with positive shifts in some categories but not all (Elby, 2001), and two professors with similar epistemological course goals show opposing shifts across the semester (McCaskey, 2009). The studies also reveal differences between male and female students, majors and non-majors, and first- and third-year students. Before students enter university, their attitudes towards physics are often firmly established, be they positive or negative (Milner-Bolotin et al., 2011). Unconnected to their high 35  school experiences in physics, many students in introductory physics courses have a strong idea of what learning and doing physics means for professionals, and they are able to articulate how these attitudes differ from their own (Gray et al., 2008; Milner-Bolotin et al., 2011). At the outset of two introductory courses, distinctions have been observed between majors’ and non-majors’ attitudes towards physics, and non-majors scored lower or equal to majors in each of the eight categories outlined by CLASS (Perkins et al., 2005; Slaughter et al., 2011). In Slaughter et al.’s (2011) study, it is interesting to note that all students enrolled in the course met the same entrance requirements as there is only one introductory physics course, designed for both majors and non-majors. Both groups demonstrated significant shifts away from expert-like attitudes overall, with students not intending to major in physics shifting further. Post-instruction, significant differences were observed in ‘Sense-making/Effort’ and ‘Conceptual Understanding’ (Slaughter et al., 2011). A similar observation has been made elsewhere, with physics majors consistently demonstrating more expert-like attitudes towards physics than students intending to major in other sciences (Gire et al., 2009; Perkins & Gratny, 2010).  It is also interesting to compare students’ attitudes at the beginning and end (or near-end) of a degree in physics. Particularly, researchers have wondered whether students reach a plateau of expert-like thinking. One study found no statistical significance between students’ first- and third-year overall favourable or unfavourable scores (Slaughter et al., 2012). This study followed the same 35 students as they moved through their physics degree. These findings are similar to those found using a cross-sectional methodology, in which students were chosen from each year of study during a single academic year (Bates et al., 2011). Between years, significant shifts were found in favourable and unfavourable scores in several categories (Slaughter et al., 2012). Decreases were seen in favourable scores in the ‘Problem Solving: General’, ‘Personal Interest’, 36  and ‘Problem Solving Sophistication’ categories. Significant increases were seen in ‘Problem Solving: General’, ‘Personal Interest’, and ‘Real-World Connections’. Bates et al., (2011) reported score differences between each year of study, and found the statistically significant decreases in favourable scores from second to third year, and again from third to fourth year. These values are interesting, as they suggest that students’ attitudes fluctuate amongst the categories of the CLASS, while remaining relatively static overall. Barrantes et al. (2009) found that while there were no significant differences between freshman and senior students’ attitudes, seniors’ scores indicated they placed greater value on the general skills obtained during physics instruction than the fact-based content (Pawl, Barrantes, Pritchard, & Mitchell, 2012). It is these attitudes, hidden within the overall score, which provide great insight to instructors hoping to create change in their students’ attitudes towards physics.   2.1.3.1.3 Attitudes and Course Design Many studies have come to the conclusion that unless a course is aimed specifically at altering and improving students’ attitudes and beliefs about learning physics, students shift away from expert-like thinking (Brewe et al., 2009; Perkins et al., 2005; Pollock, 2005; Redish & Hammer, 2009). In a study comparing students’ conceptual gains and attitude shifts, Milner et. al. (2011) observed a relationship between students’ prior knowledge and experience in physics and their attitude shifts. In all students, a significant positive correlation was observed between all students’ scores on the Force Concept Inventory (Hestenes et al., 1992) and their favourable scores overall, and in ‘Problem Solving: General’, ‘Problem Solving: Confidence’, ‘Problem Solving Sophistication’, ‘Conceptual Understanding’, and ‘Applied Conceptual Understanding’. These correlations were present at both the outset and conclusion (weeks two and 12, 37  respectively) of the course (Milner-Bolotin et al., 2011). Although not a described feature of the study, the course in question employed a variety of technology-enhanced active engagement pedagogies, as well as assessments that were chosen to align with the instructors’ learning goals. It is possible that these decisions were unintentionally focused on the development of expert-like thinking in physics.   At Florida International University, researchers designed a course to provide students with “authentic science experiences” with the intent that these experiences would increase students’ expert-like thinking (Brewe et al., 2009, 2013). Research supporting this belief is inconclusive with studies both in support (Lindsey, Hsu, Sadaghiani, Taylor, & Cummings, 2012; Otero & Gray, 2008) and against (as cited in Brewe et al., 2009). Brewe et al.’s (2009) findings indicate that Modelling Instruction methods have a positive effect on students’ attitudes, with significant shifts in students’ overall, ‘Problem Solving General’, ‘Conceptual Understanding’, ‘Problem Solving Sophistication’, ‘Applied Conceptual Understanding’ over the fall semester, and ‘Personal Interest’ over the spring semester. The findings of this study are consistent with those of Otero and Gray (2008) and Lindsey et al. (2012), who implemented courses that focused on developing students’ scientific thinking through an inquiry-based curriculum. In the four studies highlighted here, course enrollment was kept under 50 students, although Otero and Gray (2008) included a single course with 100 students. Brewe et al. (2013) suggest that this could have a positive impact on students’ expert-like attitudes, as the small number allows students to work in an environment similar to that seen in science research and careers.    Due to the multitude of factors involved in conducting a course, including pedagogies, enrollment, instructor, and differences between students, it is difficult to pinpoint precise causes 38  for the variations observed in these studies. Future research will be needed to further elaborate upon these causes.  The explicit relationship between students’ learning gains and attitudinal shifts has not been extensively explored. However two studies revealed interesting preliminary results. Pollock (2005) found a correlation between students’ learning gains, as measured by the Force and Motion Conceptual Inventory (Thornton & Sokoloff, 1998), and their attitudes about physics, as measured by the CLASS (Adams et al., 2006). Students with a normalized learning gain above 0.2 demonstrated an increase in favourable attitudes. While students with very low learning gains showed a decrease. A variety of pedagogical reforms were implemented in this course.   A second study examined the correlation between learning gains and attitudinal shifts in students enrolled in traditional introductory physics courses at a variety of institutions (Perkins et al., 2005). Decreases in students’ attitudes were observed in courses that were not aimed at impacting students’ attitudes, including a traditional course and one that was taught using interactive engagement pedagogies. In the courses designed to improve attitudes, increases were observed in students’ overall scores.  2.2 Theoretical Framework The analysis and interpretation of the study data drew largely from principles of exploratory and confirmatory factor analyses including notions of validity and reliability complemented by interpretive frameworks which are informed by constructivist perspectives (Driver, 1983; Hodson, 1998; McCaskey, 2009).  The discussion draws from the current version of the CLASS as well as other relevant attitudes surveys. Aspects of the SEMLI_S also informed this interpretation (Thomas et al., 2008), including self-efficacy (Bandura, 1977, 1986; Thomas 39  et al., 2008) awareness (Anderson & Nashon, 2007; Thomas et al., 2008), and constructive connections (Thomas et al., 2008), as well as the importance of real world connections often included in other attitude surveys (Redish, 2003a).  2.2.1 Validity  A discussion of validity is a necessary portion of the development of a valid and reliable instrument. Concerns about validity are central to the accurate use of test scores in interpretation, generalization, and extrapolation. As Kane (2009) states:  Testing programs are designed to support certain kinds of interpretations and uses. The resulting test scores have a wide variety of important uses in education, health care, scientific research, and public policy. It is therefore of some concern to society as a whole and to many stakeholders that the uses made of test scores and the interpretations assigned to test scores are appropriate and achieve their goals. Although it may be difficult to fully articulate the reasoning involved in the interpretation and use of test scores, it is important to do so (p. 50).   In this section, the scope and value of validity, the necessary evidence for validity, as well as construct validity will be discussed.  2.2.1.1 What is Validity This discussion begins with an overarching description of validity, including what validity means for test developers and users. Although various types of validity are often cited in the literature (content, criterion, predictive, concurrent), these are better interpreted as types of evidence in support of overall validity of an instrument (American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing (U.S.), 1999). Validity, now defined as “a unitary concept”, “refers to the degree to which evidence and 40  theory support the interpretations of test scores entailed by the proposed uses of tests” (American Educational Research Association et al., 1999, p. 9). Particularly in the fields of education and psychology, “validity and validation have been consistently defined in terms of interpretations and uses of test scores” (Kane, 2009, p. 42) and the “evaluative judgments about the appropriateness of proposed interpretations and uses” (American Educational Research Association et al., 1999, p. 9). These judgments are based upon collected evidence, which can include examinations of the test format and content, how the test is scored, as well as the procedures involved in writing and scoring the test. The types of evidence required to create a strong argument for validity depend entirely on the intended interpretations and uses of the test scores (American Educational Research Association et al., 1999; Kane, 2009). In the context of this work and the CLASS instrument, this refers to the extent to which a researcher can make conclusions based on the overall and categorical favourable and unfavourable scores, as well as the implications of these interpretations. For example, to what extent does a high score in personal interest indicate that the student truly has a high level of interest in the area of physics? Defining the proposed interpretation is equivalent to defining the construct or concept the tool aims to measure (American Educational Research Association et al., 1999). It should also be noted that for each possible interpretation and use of test scores, evidence must be collected in support of their validity (American Educational Research Association et al., 1999). 2.2.1.2 Interpretive Arguments When considering the validity of an instrument, it is important to begin with an interpretive argument (Kane, 2006). This provides the developer (and user) with a single statement of how the test scores are intended to be used and interpreted (Kane, 2006). An 41  interpretive argument consists of a “network of inferences and assumptions leading from observed performances to the conclusions and decisions based on performances” (Kane, 2006, p. 23). In other words, the interpretive argument outlines the reasoning and rationale that supports the procedure followed to get from an observation to a conclusion generated from a group of observations (American Educational Research Association et al., 1999). In the frame of the CLASS, this could include an argument in support of relating students’ scores to those of experts to define the students as “expert-like” or “novice-like”. Cronbach, Linn, and Ryan indicate that without clearly and adequately outlining the interpretive argument, and thus the proposed interpretations and uses of the test scores, the validity of an instrument cannot be sufficiently evaluated (as cited in Kane, 2006). The above review of the research that employs the CLASS highlights the impact of a weakly-defined interpretive argument. A variety of studies with differing goals, interpretations, and conclusions exist, all relying on the same instrument. However, the intended generalizations and extrapolations to be made with CLASS test scores are ill-defined, and the conclusions of the afore-mentioned studies should be received with caution. This miscommunication between developers and users of a test is not uncommon, and the onus falls on both parties to provide sufficient evidence in support of a proposed interpretation or use of test scores (Kane, 2009).  Cronbach stated that arguments in support of or against validity evaluate individual aspects of the interpretive argument (as cited in Kane, 2006). An interpretive argument can be broken down into four main parts, scoring, generalization, extrapolation, and implications. This structure provides guidance for both the proposed interpretations and uses of test scores, as well as the validation of these propositions (Kane, 2006). An example is given in Table 2.9 Table 2.9(Kane, 2006). These inferences are supported by the assumptions and evidence that create a 42  validity argument (Cronbach, 1988, as cited in Kane, 2006). For example, if the interpretation of test scores depends on a model of underlying factors present in the construct, collected data should fit the proposed model, and the strength of the model fit would be evidence for or against this proposed interpretation (Kane, 2006). It should also be noted that while an interpretive argument may hold across many populations, it is always possible for an inference to come into question in a particular case (Kane, 2006). This study focuses on examining the scoring inference of the CLASS instrument in this population, particularly the validity of the factors and subsequent subscale scoring as suggested by the developers.  Table 2.9: An example of an interpretive argument for a trait interpretation (Kane, 2006) I1: Scoring: from observed performance to the observed score  A1.1 The scoring rule is appropriate. A1.2 The scoring rule is applied as specified. A1.3 The scoring is free of bias. A1.4 The data fit any scaling model employed in scoring. I2: Generalization: from observed score to universe score  A2.1 The sample of observations is representative of the universe of generalization. A2.2 The sample of observations is large enough to control random error. I3: Extrapolation: from universe score to target score  A3.1 The universe score is related to the target score. A3.2 There are no systematic errors that are likely to undermine the extrapolation. I4: Implication: from target score to verbal description  A4.1 The implications associated with the trait are appropriate.  A4.2 The properties of the observed scores support the implications associated with the trait label  Kane (2006) outlined two major phases in the validation of a proposed interpretive argument, development and appraisal. The development stage can be separated into three steps: (1) create an interpretive argument and develop an appropriate test plan; (2) develop a test that is consistent with the proposed interpretation; and (3) evaluate the relevant inferences and assumptions as thoroughly as possible. The development stage can be seen as seeking to evaluate and confirm the validity of the proposed interpretations and uses (Kane, 2009).  43  The appraisal phase involves a critical evaluation of all aspects of the interpretive argument (Kane, 2009). As Cronbach stated, “a proposition deserves some degree of trust only when it has survived serious attempts to falsify it” (as cited in Kane, 2006).  A thorough examination of the interpretive argument may uncover a variety of hidden assumptions, true or not, which could have strong implications for the validity argument as a whole (Kane, 2009). For this reason, it is important to fully examine an interpretive argument for possible holes in the logic and rationale supporting its validity.  The appraisal stage includes a thorough examination of evidence relating to the validity and quality of a claim proposed by the interpretations and uses of a test, and involves generating a set of propositions in support of or refuting the proposed interpretations and uses (American Educational Research Association et al., 1999; Kane, 2006, 2009). The development of a strong validity argument involves collecting various forms of evidence, and weaving them together to support the clarity, coherence, and plausibility of a proposed interpretive argument (American Educational Research Association et al., 1999; Kane, 2009). Previous work surrounding the validity of the CLASS has examined the validity of the expert scoring system used in scoring the CLASS (Robyn’s poster/presentation) and whether or not students interpret the test items as expected (Sawtelle et al., 2012). This research aims to examine underlying factor structure and subsequent scoring model proposed by the CLASS to examine the validity of the proposed scoring structure.  2.2.1.3 Construct Validity  In 1957, Loevinger proposed that “construct validity is the whole of validity from a scientific point of view” (as cited in Messick, 1981). While construct validity does not provide a 44  full foundation for validity, it is indeed a central and guiding principle to discussions of validity in educational and psychological measurement (Messick, 1981). Construct validity has been defined as the “degree to which the relationships among items and components conform to the construct on which the proposed test score interpretations are based” (American Educational Research Association et al., 1999). This definition relies heavily on the presence of a clearly defined theory supporting the proposed interpretations and uses of the test (Kane, 2009), and  Cronbach and Meehl suggest that construct validity is of high value “whenever a test is to be interpreted as a measure of some attribute or quality which is not operationally defined” and “for which there is no adequate criterion” (as cited in Kane, 2009). The development of the CLASS omits relation to any guiding theory, choosing to follow the lead of previous instruments and rely upon experience in the classroom.  Construct validity illuminates the internal structure of test scores, including the homogeneity of the items both within factors and across the test (American Educational Research Association et al., 1999). This includes an examination of the underlying factor structure of the test scores as well as the internal consistency of the items (American Educational Research Association et al., 1999). Internal consistency provides a measure of the unidimensionality of the test items (Cunningham, 1986). A high internal consistency indicates that the items provide measurements about the same construct, whereas a lower internal consistency suggests that more than one construct is being measured in the test items (Cunningham, 1986). This study will use factor analysis and Cronbach’s alpha to examine the construct validity of the test, although the strength of the argument is weakened due to the lack of an operationalized definition of “attitudes towards physics” on the part of the developers. 45  2.2.2 Self-Efficacy In science education, self-efficacy is defined as a measure of an individual’s confidence in their ability to reach a desired outcome (Anderson & Nashon, 2007; “Self-Efficacy,” 2010).  Essentially, a high self-efficacy indicates an individual is confident in their ability to achieve a desired outcome. Central to the concept of self-efficacy is the notion that individuals are capable of controlling the decisions they make and the behaviours they pursue (Bandura, 1977; Liehr & Smith, 2008, p. 183). Particularly, self-efficacy impacts persistence and the amount of effort a learner is willing to put in to be successful when approaching obstacles and can ultimately impact whether or not an individual decides to pursue related behaviours (Bandura, 1977). Self-efficacy plays a key role in students’ learning as it impacts their willingness and ability to cope with unknown situations and construct their own knowledge and understandings (Anderson, Nashon, & Thomas, 2009; Bandura, 1977; Hacker, 1998). Self-efficacy is not an evaluation of the skills an individual has, but rather what the individual believes they are capable of doing with those skills (Bandura, 1986, p. 391).  Through the improvement of self-efficacy, students can become more confident in their ability to approach learning obstacles and be successful. Instructors should be cognizant of students’ self-efficacy and help students develop high levels of self-efficacy (Mercer, Nellis, Martínez, & Kirk, 2011). O’Connor and Korr suggest that through well-thought out pedagogical choices, instructors have the ability to positively influence students’ self-efficacy (as cited in Mercer et al., 2011). For example, instructor behaviour, expectations, feedback, and communication can impact students’ self-efficacy (Mercer et al., 2011). By helping students develop a strong sense of self-efficacy, it is possible that students will make a more concerted 46  effort in their learning (Bandura, 1977) and be more successful in becoming successful (Pajares, 1996).  2.2.3 Constructivist Learning Theories A major tenet of physics education research is the notion that students do not come to instruction as blank slates without any prior instruction (Driver, 1983; Hodson, 1998; McCaskey, 2009). They already hold ideas about how the world works, and they build new knowledge on the foundation of these understandings. Additionally, learning does not occur in isolation (Vygotsky, 1987) and depends on a plethora of factors including prior knowledge as well as social interactions (Wink & Putney, 2002) and attitudes (Adams et al., 2006). These factors impact how students interpret new information they encounter (Hodson, 1998, p. 10). For this reason, constructivist teaching and learning theories suggest instructors attempt to gain insight into not only what their students know, but how they came to know it (Driver, 1983; Hodson, 1998). Research has shown that students who are active participants in the construction of their own knowledge are generally more successful (Thomas et al., 2008).  Particularly in science learning, the ability to internalize information to develop a personal standpoint is a defining attribute of a scientifically literate individual (Hodson, 1998, p. 3). Millar suggested that “the process of eliciting, clarification, and construction of new ideas takes place internally, within the learner’s own head” and “occurs whenever any successful learning takes place” (as cited in Hodson, 1998, p. 34). Millar also suggests that this process should occur regardless of the form instruction takes (as cited in Hodson, 1998, p. 34). Constructivist learning includes consciously creating and recognizing relationships between concepts and pieces of information (Thomas et al., 2008). Although there is no single method for 47  implementing constructivist science teaching, Hodson (1998) provides four general steps to serve as a guideline:   identify students’ ideas and views;  create opportunities for students to explore their ideas and test their robustness in explaining phenomena, accounting for events and making predictions;  provide stimuli for students to develop, modify and, where necessary, change their ideas and views;  support their attempts to rethink and reconstruct their ideas and views. (p. 34) To adequately identify students’ prior knowledge and views, it is important to ensure that the instruments used to acquire this knowledge are both valid and reliable. In the following chapters, I will discuss validity as part of the development of strong tools to measure students’ attitudes and beliefs towards physics at the outset of a course.  2.3 Literature Summary Based on a review of the literature as described above, it appears that no further research has been done surrounding the validity of the underlying factors and resultant scoring scheme of the CLASS. This research will focus on examining these factors.  48  3 Methodology In this section, I outline the study methodology. The research questions are reiterated, followed by the study context. Subsequently, the statistical tests used to analyze the collected data are described. It is important to note that the statistical analysis differs from the validation process followed during the development of the CLASS (Adams et al., 2006). Finally, ethical considerations and limitations of the study are described.  3.1 Research Questions As discussed in the literature review, students’ attitudes and beliefs about physics and learning physics is a complex area of research, and a variety of tools have been developed to measure these constructs. The authors of the Colorado Learning Attitudes About Science Survey (CLASS) determined eight factors as underlying student responses to the items about their attitudes and beliefs about science (Adams et al., 2006). However, confirmatory factor analysis using data from the population of students at the study institution did not support the existence of these eight factors. Thus, it became necessary to investigate the factors that underlay these students’ CLASS responses through a revalidation of CLASS. This study analyzed data collected using the CLASS (Adams et al., 2006) to determine the factors that underlay the students’ attitudes and beliefs about physics and physics learning as well as revalidated the instrument guided by  the following research questions: 1. What are the factors underlying first year undergraduate students’ attitudes and beliefs about science prior to participating in First Year Physics courses? 49  2. What might be the implications of these factors on course design and consequent student physics learning?  Statistical analysis of the survey data was used to explore the factors that underlie students’ responses to the survey items about their attitudes and beliefs about science prior to taking an introductory physics course.  3.2 Study Context and Participants This study analyzes undergraduate students’ attitudes and beliefs about physics and physics learning prior to participating in first year courses at a large research university in Western Canadian. The data were collected from various cohorts of undergraduate students enrolled in their first undergraduate physics course (either engineering or calculus-based), from different years (2010-2013) and stored by the Department of Physics and Astronomy in the Faculty of Science at the university. In general, all students who wish to pursue and obtain a Physics degree will take an introductory physics course in their first year. The large number of students enrolled in these courses lends itself well to performing statistical analyses, such as factor analysis and reliability tests, that rely on large sample sizes to provide generalizable claims (Muijs, 2011). Two introductory physics courses were chosen for this research. The first is an introductory course for engineering students. The second is a calculus-based physics course and designed for a broader audience of science students which includes majors and non-majors. These two courses were chosen based on their high enrollments and the breadth of the material they cover. The engineering course is six-credits and runs from September to April. The calculus-based course is divided into two sequential three-credit courses, one in the fall semester 50  and one in the winter semester. This research focuses on the first semester of this curriculum. The topics discussed in each of the two courses are described in Table 3.1. Two years of data for each course were then chosen, based on availability (Table 3.2).  Table 3.1: Outline of the topics covered in each course Engineering Calculus-based  Thermometry  Thermal properties of matter  Heat  Oscillations  Waves  Sound  Buoyancy, Pressure, Bernoulli's Equation  Thermodynamics and Heat Transfer  Simple and Damped Harmonic Motion  Waves and the Doppler Effect   Table 3.2: Number of students in each data set Course Total Female Male No Gender  Engineering – 2010 601 126 451 24 Engineering – 2011 592 133 436 23 Calculus-based – 2012 760 465 290 5 Calculus-based – 2013  647 335 282 30  Within the Faculty of Science at this university, particularly in the Department of Physics and Astronomy, there has been a strong initiative to include evidence-based pedagogies in science courses at all undergraduate levels (“Carl Wieman Science Education Initiative at the University of British Columbia”, n.d.). Introductory courses have been at the forefront of this initiative. In conjunction with their efforts to improve teaching at the institution, an interested group of science educators engages in ongoing data collection with the intent to assess student learning and attitudes about science. This study analyzes one portion of previously collected data.  The two courses described here are large, introductory courses. The courses are divided into a number of sections and each is taught by a single professor or instructor who is supported 51  by a number of Graduate Teaching Assistants. In the engineering course, students participate in a weekly tutorial session during the fall semester, and there is no laboratory requirement. In the calculus-based course, students participate in a lab or tutorial each week, on a two week cycle. A variety of pedagogical tools are implemented in these courses, including but not limited to Peer Instruction, PeerWise, pre-reading assignments, and in-class activities (“Carl Wieman Science Education Initiative at the University of British Columbia”, n.d.). 3.3 Study Procedures This study is analytical in nature. The goal of this research is to analyze quantitative data to investigate the factors that underlie students’ attitudes and beliefs about physics and physics learning through a revalidation of the CLASS, which is the instrument used to obtain the data from the students. This study aims to understand the factors that underlie students’ responses to the CLASS instrument.  3.3.1 Data Collection  The data about students’ attitudes and beliefs used in this study came from the Department of Physics and Astronomy at a large research university in Western Canada. The Department of Physics and Astronomy collects data using the CLASS instrument, created at the University of Colorado (Adams et al., 2006), at the beginning and end of undergraduate physics courses, to assess students’ beliefs and attitudes about physics, as well as the success of certain educational reforms. The survey is administered online, and little or no course credit is provided to students. Students are asked to provide their student ID, course number and section, as well as 52  their gender. The data were anonymized before being given to the researcher to ensure confidentiality and privacy of participants.   Collected data were organized and sorted using Microsoft Excel (Microsoft, 2010). Students’ responses to the survey were scored according to the CLASS template provided by the developers in Microsoft Excel. Students who failed to meet the control requirements when completing the survey were separated from the group. Data that met the control criteria were scored according to the scheme supplied by the developers.  3.3.2 Data Analysis  Before beginning analysis, the scores for negatively stated items were reverse coded consistent with statistical requirement for analyzing data on personality constructs (Salkind, 2007). Responses to items that began with “I do not”, “I cannot” or another negative statement (Table 3.3) were reversed across the neutral option (3) so that a 1 became a 5, and a 5 became a 1.  Table 3.3: Five CLASS items were reverse-coded Item 11 I am not satisfied until I understand why something works the way it does.  12 I cannot learn physics if the teacher does not explain things well in class. 13 I do not expect physics equations to help my understanding of the ideas; they are just for doing calculations. 20 I do not spend more than five minutes stuck on a physics problem before giving up or seeking help from someone else.  21 If I don’t remember a particular equation needed to solve a problem on an exam, there’s nothing much I can do (legally!) to come up with it.   This study employed an iterative application of exploratory factor analysis and reliability tests to arrive at a suggested model underlying the CLASS data in the 2011 course for 53  engineering students. The 42 items of the survey were considered as a whole. Subsequently, confirmatory factor analysis was used to adapt and confirm this model in the 2010 engineering course, as well as the 2012 and 2013 calculus-based courses.   3.3.1.1 Factor Analysis Factor analysis is a statistical technique to reduce a number of variables, such as items on a test, to a smaller number of latent variables (Floyd & Widaman, 1995). It is referred to as a data reduction technique. Factor analysis enables researchers to determine whether observable variables are related to a group of unobservable factors (Jowett, 1958; Muijs, 2011). Factor analysis is commonly used to develop and validate psychological scales due to the unobservable nature of many constructs researchers are interested in investigating. A factor is created by a group of observable variables which relate to an underlying unobservable, latent variable (Muijs, 2011). Variables in a factor are strongly correlated with each other, and weakly correlated with variables outside the factor (Muijs, 2011). These latent variables are descriptive of the overarching construct of interest. For example, the eight categories of the CLASS each measure one unobservable aspect of students’ attitudes about physics. Factor analysis aims to explain as much of the total variance and covariance observed in the variables as possible, with each factor explaining a piece of the total.  There are three main varieties of factor analysis: exploratory factor analysis; principal components analysis: and confirmatory factor analysis. This research makes use of exploratory factor analysis and confirmatory factor analysis to examine the validity of the CLASS instrument.  54  3.3.1.1.1 Exploratory Factor Analysis  Exploratory factor analysis is used to discern the underlying structure of a group of variables which measure observable traits when the researcher is aware of or believes in the existence of an underlying structure (J. D. Brown, n.d.). Exploratory factor analysis is the first step to determining how a scale divides into latent variables, and which items measure each of these latent variables. The goal is to extract the minimum number of factors that explains the maximum amount of variance among the items (Thompson, 2004). In the case of this study, we are also interested in including the maximum number of items from the scale.  Exploratory factor analysis differs from principal components analysis when considering the variance included in the correlation matrix. When performing principal components analysis, all variance is included: error variance, shared variance, and unique variance (J. D. Brown, n.d.). When performing exploratory factor analysis, the only variance included for each variable in the correlation matrix is the variance that variable shares with another variable. Principal components analysis is used more often when researchers hold no prior expectations about the existence of a factor structure, while exploratory factor analysis is more common when a structure is hypothesized.   In this study, exploratory factor analysis was performed using the SPSS software package (“IBM - SPSS Amos - Canada,” 2013). The maximum likelihood estimates was used to examine possible factor structures present in the CLASS, as this is the exploratory factor analysis method in the SPSS software.   Consider the following example, examining the items of the CLASS identified to belong to the problem solving categories (Problem Solving General, Problem Solving Confidence, and 55  Problem Solving Sophistication). The items are referred to as “PS#”. Three factors were extracted. However, one factor contained fewer than three variables. It is recommended that a valuable factor contain at least three items (Muijs, 2011). Subsequently, two factors were extracted to arrive at two valuable factors.  Each factor should explain as much of the variance in the data as possible, with each factor explaining less variance than the last (Muijs, 2011). As seen in Table 3.4, the first factor (eigenvalue = 2.232) explains 20.295% of the variance, while the second factor (eigenvalue = .463) explains 4.211%. Together, the two factors account for only 24.505% of the total variance in the system.  Table 3.4: Total variance explained by the extracted factors.   Initial Eigenvalues Extraction Sums of Squared Loadings Rotation Sums of Squared Loadingsa Factor Total % of Variance Cumulative % Total % of Variance Cumulative % Total 1 2.934 26.671 26.671 2.232 20.295 20.295 2.030 2 1.233 11.211 37.882 .463 4.211 24.505 1.795 3 .980 8.907 46.789     4 .932 8.468 55.258     5 .854 7.762 63.019     6 .825 7.503 70.522     7 .740 6.731 77.253     8 .695 6.321 83.573     9 .656 5.961 89.535     10 .604 5.491 95.026     11 .547 4.974 100.000     The unrotated factor loadings are seen in Table 3.5. This information is difficult to interpret, as most items load on the first factor to explain the majority of the variance in the data (Muijs, 2011). A promax rotation was used, allowing the factors to correlate, as the factors were not believed to be orthogonal. When using a promax rotation, the pattern matrix (Table 3.6) identifies the rotated factor loadings, while the structure matrix (Table 3.7) describes the 56  correlation between the item and each factor. All of the factor loadings are above 0.3 (any loadings below this cut-off are hidden from the table here), and two factors are evident from the data. Factor 1: {PS 10, PS1, PS5, PS6, PS9}. Factor 2: {PS7, PS8, PS11, PS3, PS4}.  Table 3.5: The factor matrix for exploratory factor analysis with Promax rotation, using the items from the CLASS suggested to probe facets of problem solving.   Factor  1 2 PS10 .624  PS9 .616  PS1 .534  PS7 .520  PS5 .515  PS8 .373 .318 PS6 .366  PS3 .364  PS11 .339  PS2   PS4   Extraction Method: Maximum Likelihood. a. 2 factors extracted. 3 iterations required. Table 3.6: The pattern matrix for exploratory factor analysis with Promax rotation, using the items from the CLASS suggested to probe facets of problem solving. Shaded cells identify items that load to each factor.   Factor  1 2 PS10 .598  PS1 .583  PS5 .570  PS6 .552  PS9 .396  PS2   PS7  .553 PS8  .552 PS11  .386 PS3  .381 PS4  .337 Extraction Method: Maximum Likelihood.   Rotation Method: Promax with Kaiser Normalization. a. Rotation converged in 3 iterations.  57  Table 3.7: The pattern matrix for exploratory factor analysis with Promax rotation, using the items from the CLASS suggested to probe facets of problem solving. Shaded cells identify items that load to each factor  Factor  1 2 PS10 .641 .454 PS9 .578 .537 PS1 .569 .355 PS5 .551 .340 PS6 .433  PS2   PS7 .407 .585 PS8  .484 PS3  .407 PS11  .392 PS4   Extraction Method: Maximum Likelihood.   Rotation Method: Promax with Kaiser Normalization. Item PS2 failed to load higher than 0.3, in both the pattern and structure matrices, which suggests the item should be removed until it can be further examined and improved. The structure matrix also suggests that PS4 is weakly correlated to factor 2. The item and scale should be examined to determine whether the item belongs in the factor or should be removed. The correlation matrix (Table 3.8) identifies how strongly each factor is correlated with the other factors extracted from the data. High correlations suggest that two (or more) factors actually represent a single factor, and provide evidence of a possible hierarchical structure in the data (Muijs, 2011). In this example analysis, all of the items relating to problem solving were selected from the CLASS instrument. It is therefore logical that the two factors are strongly correlated, as they aim to measure a similar aspect of students’ attitudes towards physics.      58  Table 3.8: Factor correlation matrix. There is a high correlation between the two extracted factors, which suggests that the two factors main in fact be a single factor. Factor 1 2 1 1.000 .647 2 .647 1.000 Extraction Method: Maximum Likelihood.    Rotation Method: Promax with Kaiser Normalization. 3.3.1.1.2 Confirmatory Factor Analysis Confirmatory factor analysis uses structural equation modeling to enable a researcher to determine if measures of observable variables are consistent with a hypothesized model of an unobservable construct (T. A. Brown, 2006c). Confirmatory factor analysis is used to validate the factor structure (factors and loadings) as well as the number of subscales present in a survey tool (T. A. Brown, 2006c). Confirmatory factor analysis provides both convergent and discriminant validity by confirming or refuting whether variables measure the same thing. In this study, the goal of confirmatory factor analysis is to determine if the data collected fits the model proposed by Adams et al. (2006), and to provide insight into the validity of a proposed factor structure.  In this study, confirmatory factor analysis was performed using the AMOS add-in to SPSS (“IBM - SPSS Amos - Canada,” 2013). Confirmatory factor analysis was used in two capacities: to examine how well the collected data fit the model suggested by Adams et al. (2006), and to examine how data from a second course fit a model suggested by the EFA conducted in this research. Figure 1 shows an example of how AMOS represents a factor structure, using the structure suggested by the above analysis of the problem solving items (Table 3.5). The latent variables are represented by the ovals on the right-hand side, Factor1 and Factor2. The observed variables (students’ responses to the items) are indicated by the 59  rectangles. Arrows connect each observed variable to the appropriate latent variable. The error variances are indicated on the left hand side. These are unobserved, but indicate variance in the observed variables that is not explained by the latent variable (T. A. Brown, 2006c).   Figure 3.1: Factor structure and loadings for suggested factor structure underlying problem solving items for the CLASS instrument. There is a major difference between confirmatory factor analysis and exploratory factor analysis. When performing exploratory factor analysis the researcher is looking for structure, a model that describes the organization of a construct. Confirmatory factor analysis takes this model and attempts to confirm its validity, as well as evaluating how well the data fits the model (“Confirmatory Factor Analysis,” 2012). Model fit is evaluated based on several factors: goodness-of-fit, localized areas of strain, and the parameter estimates.  60  Goodness-of-fit can be indicated by a variety of indices (T. A. Brown, 2006d). A non-significant χ2 indicates a good absolute fit, although large sample sizes make this difficult to obtain. Due to the susceptibility of χ2 to be influenced by sample size, the standardized root mean square residual (SRMR) is commonly used as a second absolute fit index, with 0.0 indicating a perfect fit. Smaller SRMR values indicate a better model fit, with .08 as an upper bound for a good fit. The comparative fit index (CFI) is one of several comparative fit indices which compares the user-suggested model to a baseline (or null) model that is generally more restricted. Unlike the SRMR, values closer to 1.0 indicate a good fit, with values above .95 indicating a good fit and values between .9 and .95 indicating an acceptable fit. The root mean square error of approximation (RMSEA) is an “error of approximation index” which examines whether a model fits “reasonably well” in a population. Values of RMSEA range between 0.0 and 1.0, with smaller values indicating a more reasonable fit. Values close to or below .06 are cited as providing a good fit. Confidence intervals are calculated for RMSEA, based on the noncentral χ2 distribution, and should fall below the upper limit of .08. RMSEA is from the parsimony class of model fit indices, which compare models that provide equal model fits at the absolute level, and favours models with fewer freely estimated parameters. These indices are considered together to determine the quality of a model in each of the courses. The second aspect of a model fit that should be inspected is the existence of localized areas of strain. While goodness-of-fit indices provide information about how well the model fits the data globally, it is important to consider if there are any areas of strain in the model fit at a local level (T. A. Brown, 2006e). In order to determine whether or not these areas exist, it is necessary to inspect the modification indices and residuals of the analysis (T. A. Brown, 2006e).  61  Modification indices are calculated for all fixed and constrained parameters in the model (T. A. Brown, 2006e). This includes the error terms and latent variables. Modifications indices provide an estimate of how much the model χ2 would decrease if the parameter were freely estimated (T. A. Brown, 2006e). In general, a lower χ2 is more likely to be non-significant, as the difference between the value and the number of degrees of freedom will decrease. Small modification indices suggest a good fit, whereas larger modification indices suggest a parameter should be re-examined to determine if it should be freely estimated or constrained (T. A. Brown, 2006e). Residuals provide another means of examining the existence of localized areas of strain. The residual matrix describes the difference between the sample variance-covariance matrix and the one suggested by the model (T. A. Brown, 2006e). A small residual suggests that the variance and covariance were adequately reproduced by the suggested model and parameter estimates (T. A. Brown, 2006e). In order to assess whether a residual is “small”, the standardized residual is calculated by dividing the residual by the estimated standard errors. Then, the standardized residuals can be treated in a manner analogous to z-scores. An appropriate standardized residual falls between -2.00 and 2.00, indicating an absence of areas of strain and supporting a good global fit (T. A. Brown, 2006e).  Finally, if the output suggests a good model fit it is important to examine the parameter estimates to ensure that they make sense (T. A. Brown, 2006e). In particular, the size and direction of factor loadings require examination. Standardized factor loadings above 1.0 are problematic, and items should load in the expected direction (positive or negative). A factor loading indicates the amount by which an observation should increase or decrease if the overall factor score were to increase by one unit (T. A. Brown, 2006e). Low factor loadings suggest that 62  the item does not load strongly to the factor, and it could be prudent to remove the item from the factor (T. A. Brown, 2006a).   Information from the modification indices and standardized residuals can inform alterations to the model that would improve the fit. For example, a high modification index might suggest that two error variances are correlated (T. A. Brown, 2006a). When error variances are not correlated, the researcher is making a statement that all correlation between the items in a factor is due solely to the latent variable, and all measurement error is random (T. A. Brown, 2006a). Conversely, correlating two error variances indicates that some of the shared variance is attributable to the factor, and some is due to an outside influence (T. A. Brown, 2006a). When choosing to correlate two error variance terms, it is important that this decision is supported by theory or method effects. Two items that ask the same question, but are phrased differently, for example would support correlated errors. If two error variances are correlated for one reason, the error variances of all other pairs of items that meet this requirement must also be correlated for consistency (T. A. Brown, 2006a). High standardized residuals (greater in magnitude than 2.0) suggest that more parameters are needed in the model (T. A. Brown, 2006e). This could include removing an item with consistently high residual values, or if the item has a high residual for items in a particular factor only, the item could be cross-loaded onto that factor. Finally, it is important to recall that confirmatory factor analysis relies heavily not only on empirical analysis, but theoretical foundation (T. A. Brown, 2006b). Suggested factor structures should be supported by theory, as should all alterations to the structure (T. A. Brown, 2006b).  63  3.3.1.2 Reliability  When scores from a scale of items are added together to create a single score, we need to be sure that the items are measuring the same thing; that they are internally consistent (Bland & Altman, 1997). Cronbach’s alpha is commonly used when reporting internal consistency of items of a scale. It provides a measure of how much the variables in a scale measure the same construct (Muijs, 2011). By examining the internal structure of a scale, it is possible to measure its reliability without the need for a second test (Revelle & Zinbarg, 2009). It is important to measure the reliability of the scale as a whole, as well as any subscales within it from which a score is calculated (Shavelson, Webb, & Rowley, 1989). Each subscale should meet reliability criteria to ensure that the items are measuring the same construct. In this work, Cronbach’s alpha was used to determine whether items should be added or removed from individual factors.  Cronbach’s alpha is a measurement that meets in the middle of the reliability measurements that came before, including split-half reliability, without the need to arbitrarily divide scales into seemingly equal halves (Cronbach, 1951; Revelle & Zinbarg, 2009). Cronbach’s alpha is relatively simple to understand, easy to implement and interpret, and is nicely included in many statistical software packages. It is therefore a popular reliability measure in the psychology literature, despite the existence of a variety of other measures that may be more appropriate (Revelle & Zinbarg, 2009).  Cronbach’s alpha is a measure of internal consistency of the items in a scale (Cronbach, 1951). It is calculated by examining the correlations between all of the items in the scale (Muijs, 2011). High correlations, and thus a high alpha value, suggest that the items are measuring the same thing, and the items are internally consistent (Muijs, 2011). Like all correlations, the value 64  of Cronbach’s alpha ranges from 0 to 1, with 0 indicating no correlation or consistency, and 1 indicating perfect correlation and consistency (Muijs, 2011). A cutoff of 0.7 is commonly cited in the literature (Heredia & Lewis, 2012; Muijs, 2011). However, this value should be interpreted with care, as several other factors have an impact on alpha (Cortina, 1993). To calculate Cronbach’s alpha, the total number of items (K), the average variance for the sample (v) and the average of the covariance between items ( ) are combined, as seen in equation (x) (Muijs, 2011).   (3.1) Any weighting of item scores must occur before calculating the variance of each item, and subsequently Cronbach’s alpha (Bland & Altman, 1997). This includes reverse coding the data. In this study, the five items were reverse-coded prior to all analysis (Table 3.3). It is important to be cautious when interpreting Cronbach’s alpha for three main reasons. First, Cronbach’s alpha is sensitive to the number of items in a scale, and is susceptible to inflation as this number increases (Muijs, 2011). The ratio of K to (K – 1) demonstrates how α relies on the number of items, as a small K will have a greater impact on the value than a larger one. Second, Cronbach’s alpha is considered to be a lower bound for reliability (Revelle & Zinbarg, 2009). It is possible for a scale to be more reliable than an α calculation can reveal. Third, high values of Cronbach’s alpha could indicate that the items are overly repetitive, and some items should be removed (Adams & Wieman, 2011). Calculation of Cronbach’s alpha should be conducted in tandem with a close examination of the items included in the analysis.  c   cKvcK165  Overall reliability is important, although less so than that of the subscales. Cronbach’s alpha is more useful for scales that measure a single construct – such as the subscales of a larger scale (Adams & Wieman, 2011). Although the instrument as a whole aims to measure the same overarching construct the individual scales measure facets of this construct, which can skew the internal consistency of the test as a whole (Cortina, 1993).  3.1 Ethical Considerations   The researcher did not collect any data, but instead used secondary data collected by the Department of Physics and Astronomy in 2010, 2011, 2012, and 2013. The data were anonymized using a key that could allow the researcher to trace students across years of study, connecting the data to an individual student without identifying the student. The study was approved by the Behavioural Research Ethics Board at the study university on September 23, 2013 (Error! Reference source not found.).  3.2 Limitations  When choosing a research tool, it is important to consider the population for which the tool is reliable and valid. This study examines the validity and reliability of the CLASS instrument in a single population. Evidence supporting consistent reliability and validity across populations is lacking from the literature, and this study should not be taken to be generalizable across populations. It is important for researchers to consider available evidence and gather more evidence before committing to the use of a research tool. Limitations of this study stem from the origin of the data. The statistical data were collected in 2010-2013, and it is possible that the data may not provide an accurate image of the 66  current student population in the Department of Physics and Astronomy at the study institution. The collected data are also only representative of students at this university and may not be generalizable to other institutions. The Department of Physics and Astronomy has undergone various teaching reforms in the last six years, due to the Carl Wieman Science Educational Initiative presence in the Faculty of Science (“Carl Wieman Science Education Initiative at the University of British Columbia”, n.d.). These initiatives may have impacted the nature of the responses we receive from students, and may not be applicable to a university that does not currently employ similar teaching methods.   This study is entirely quantitative in nature. In order to address each area of validity concerns it is important to conduct interviews with students (or members of the group of interest) to gain understanding of how items might be related, and how items could be interpreted. This study considers qualitative and quantitative data collected at other institutions along with the interpretations of a single researcher at the study institution. To gain a fully-formed understanding of problematic items, more data is required from this institution.   Likert scale data is nominal in nature, as the distance between rankings is not consistent from one participant to the next. However, SPSS and AMOS (“IBM - SPSS Amos - Canada,” 2013, “IBM SPSS software,” 2013) consider all data to be continuous when performing factor analysis. This could skew the results of this study.  3.3 Methodology Summary The validity and reliability of the CLASS survey tool was examined using Cronbach’s alpha, exploratory factor analysis, and confirmatory factor analysis. The results of these analyses are presented and discussed in Chapter 4.  67  4 Results and Discussion To answer the research questions, a variety of statistical analyses were performed. Firstly, the existence of the factors (categories) previously identified in the CLASS (Adams et al., 2006) was investigated through confirmatory factor analysis (CFA). This indicated that the published factor structure is non-existent in the population of students analyzed in this study. This made it necessary to investigate the factors that underlie the students’ responses to the original CLASS questionnaire. This study followed a validation process that differed from the one used in the development of the original CLASS tool, which is described in the Literature Review (Adams et al., 2006). A new model was generated through a revalidation process using exploratory factor analysis (EFA). The existence of the new extracted factors was then investigated in three other student groups using CFA.  Following a description of the results, I discuss the interpretation of the findings. Labels are given to the confirmed factors, and implications are provided for theory, research, and curriculum design.  4.1 Examining the Original CLASS Model  Confirmatory factor analysis was performed in AMOS using each of the data sets involved in this analytical study (“IBM - SPSS Amos - Canada,” 2013). For each course (data set), the AMOS test output read: “The model is probably unidentified. In order to achieve identifiability, it will probably be necessary to impose 6 additional constraints.”  This result supported the decision to re-examine the factor structure underlying students’ responses to the CLASS items, as the current factors could not be confirmed or measured 68  separately and simultaneously (T. A. Brown, 2006b; Heredia & Lewis, 2012) in this particular population.  4.2 Generating a Revalidated Model Because the original factor structure does not exist in the population of students analyzed in this study, it was necessary to investigate the factors that underlie students’ responses to the CLASS in this population. Exploratory factor analysis (EFA) was used to analyze the data from an introductory engineering physics. Subsequently, the robustness of the factors was investigated by performing confirmatory factor analysis (CFA) using other data sets. Item 31 was not included in any of the analysis, as it is the control item and all students’ answers are the same (agree, 4). 4.2.1 Exploratory Factor Analysis Initially, three factors were extracted. Subsequently, four, five, six, seven, eight, and nine factors were extracted. These iterations were performed to determine a factor structure that included the maximum number of items as seen in the pattern matrix, and supported by the structure matrix.  When seven factors were extracted, only nine items did not load onto a factor (Appendix B:  Pattern and structure matrices when seven factors are extracted from all 41 items of the survey). However, the number of robust factors (including three or more items) should be equal to the number of factors extracted. When seven factors were extracted, the seventh factor contained a single item (33). This item was removed, and again seven factors were extracted (Appendix C: Pattern and structure matrices for seven extracted factors after removing item 33 69  from the data.). Two factors contained only two items, so the analysis was run again to extract six factors (Appendix D: Pattern and structure matrices for six extracted factors after removing item 33 from the data). Then five factors were extracted (Appendix E: Pattern and structure matrices for five extracted factors after removing item 33 from the data), to come up with a factor structure in which all of the factors contained at least three items. In this model, 14 items did not load onto factors (Items 1, 2, 6, 9, 10, 13, 15, 16, 19, 20, 22, 23, 26, and 38). These items were removed, and five factors were again extracted (Appendix F: Pattern and structure matrices for five extracted factors. Items 1, 2, 6, 9, 10, 13, 15, 16, 19, 20, 22, 23, 33, 36, and 38 have been removed.). This process was repeated, removing another four items (4, 8, 11, and 39), to gain a factor structure in which all of the items loaded to a factor. Table 4.1 shows the pattern matrix for this model and Table 4.2 the structure matrix. These five factors explain 29.615% of the variance observed in the data.  For convenience, factor loadings below .280 are not included. Shaded items load on a single factor. Item 25 loaded strongly on two factors, and had similar correlations to each of the two factors (lighter shading). The two factors were examined in terms of reliability to determine whether the item increased the internal consistency of one factor more than the other. Although item 42 has a loading below the recommended cut off of .300 (.292), the item was included in the tool based on the high correlation between the item and Factor 3 (.440). The items are identified by “v#” in all of the following tables.    70  Table 4.1: Pattern matrix for the emergent model. Five factors are extracted. Items 1, 2, 4, 6, 8, 9, 10, 11, 13, 15, 16, 19, 20, 22, 23, 33, 36, 38, and 39 have been removed. Shaded items load on each factor.  Item Factor 1 2 3 4 5 v3 .630     v37 .621     v14 .596     v35 -.522     v30 .508     v28 .473     v40  -.665    v5  -.652    v34  .587    v12  .458    v21  .376    v25  .353 .332   v24   .602   v26   .351   v32   -.302   v42   .292   v17    .569  v29   -.290 .456  v27    .310  v41     .592 v18     .499 v7     .317 Extraction Method: Maximum Likelihood.   Rotation Method: Promax with Kaiser Normalization.a a. Rotation converged in 6 iterations.    71  Table 4.2: The structure matrix when five factors are extracted after removing items that did not load in the previous analysis. Shaded items belong to each factor.  Item Factor 1 2 3 4 5 v3 .650 .374 .373   v14 .556  .310   v35 -.525 -.415  .323  v37 .523     v30 .523  .344   v28 .502  .333   v40 -.313 -.638    v34 .355 .600 .305   v5  -.553    v21  .446  -.289  v12  .382    v25 .448 .527 .528   v24   .510   v26 .408 .355 .486   v42 .409 .286 .440   v32 -.329 -.361 -.420   v17    .564  v29   -.296 .464  v27    .360  v41     .591 v18     .507 v7     .303 Extraction Method: Maximum Likelihood.   Rotation Method: Promax with Kaiser Normalization.   The items in each extracted factor (as seen in Table 4.1 and Table 4.2) are listed in Table 4.3. Table 4.3: Five extracted factors. In this model, all of the items load to a factor.  Factor Items Factor 1 3, 14, 28, 30, 35, 37 Factor 2 5, 12, 25, 34, 40 Factor 3 24, 25, 26, 32, 42 Factor 4 17, 27, 29 Factor 5 7, 18, 41   72  4.2.2 Testing Reliabilities Reliability analyses were performed on questionnaire data to assess the ability of each factor to measure a single aspect of students’ attitudes towards physics.  4.2.2.1 Factor 1 Analysis of the items in Factor 1 revealed a reliability well below the ideal .7 for the six extracted items (α = .428). Removing item 35, as suggested by the analysis (Table 4.4), increased the reliability of the five-item factor to =.683.  Table 4.4: Reliability analysis for Factor 1.  Item Scale Mean if Item Deleted Scale Variance if Item Deleted Corrected Item-Total Correlation Cronbach's Alpha if Item Deleted v3 17.5355 5.620 .417 .236 v14 17.3547 6.212 .376 .282 v28 17.1571 6.491 .334 .312 v30 17.1199 6.722 .316 .327 v35 19.2905 10.470 -.409 .683 v37 17.7838 5.561 .405 .240  4.2.2.2 Factor 2 The internal consistency of Factor 2 returned a concerning negative value (6 items, α = -.333). To improve this value, items 5 and 40 were removed (Table 4.5). This increased the reliability of the four item factor to .510. After removing items 5 and 40, it was clear that item 25 should remain in the factor to maintain a valuable level of internal consistency (Table 4.6).    73  Table 4.5: Reliability analysis for Factor 2. Item Scale Mean if Item Deleted Scale Variance if Item Deleted Corrected Item-Total Correlation Cronbach's Alpha if Item Deleted v5 15.1959 4.753 -.289 .036 v12 15.8041 3.542 -.009 -.437a v21 14.3666 3.349 -.019 -.446a v25 13.8986 3.370 .134 -.634a v34 14.1588 3.921 .020 -.426a v40 15.7399 5.320 -.369 .058 a. The value is negative due to a negative average covariance among items. This violates reliability model assumptions. You may want to check item codings.  Table 4.6: Reliability statistics for Factor 2 after removing items 5 and 40.  Item Scale Mean if Item Deleted Scale Variance if Item Deleted Corrected Item-Total Correlation Cronbach's Alpha if Item Deleted v12 11.0743 4.110 .232 .505 v21 9.6368 3.612 .285 .465 v25 9.1689 4.070 .370 .384 v34 9.4291 4.394 .359 .406 4.2.2.3 Factor 3 Having determined that item 25 was a valuable addition to Factor 2 (Table 4.6), I considered the internal consistency of Factor 3. Item 25 loaded almost equally on Factor 2 and Factor 3 (Tables 4.1 and 4.2). Without item 25, the internal consistency of Factor 3 is almost zero (4 items, α = -.006), and the analysis suggests removing item 32 to increase the reliability to .439 (with three items) (Table 4.7). Adding item 25 to the four item factor increased the reliability from -.006 to .210, however removing item 32 would increase this value to .567 (Table 4.8). Based on this analysis, Factor 3 includes four items: 24, 25, 26, and 42.    74  Table 4.7: Reliability analysis for Factor 3.  Item Scale Mean if Item Deleted Scale Variance if Item Deleted Corrected Item-Total Correlation Cronbach's Alpha if Item Deleted v24 12.1149 3.510 .270 -.195a v26 12.4358 2.974 .414 -.171a v32 12.2061 3.284 .364 .439 v42 12.3530 3.190 .353 -.384a a. The value is negative due to a negative average covariance among items. This violates reliability model assumptions. You may want to check item codings.  Table 4.8: Reliability analysis for Factor 2; item 25 has been added to the factor. Item Scale Mean if Item Deleted Scale Variance if Item Deleted Corrected Item-Total Correlation Cronbach's Alpha if Item Deleted v24 14.0101 3.293 .172 .141 v25 14.3311 2.828 .293 -.006a v26 14.1014 3.090 .256 .055 v32 16.3699 5.032 -.331 .567 v42 14.2483 2.867 .298 -.006a a. The value is negative due to a negative average covariance among items. This violates reliability model assumptions. You may want to check item codings.  4.2.2.4 Factors 4 & 5 The internal consistency measures of Factor 4 (3 items, α = .422) and Factor 5 (3 items, α = .454) are lower than the ideal .7. However, it is important to remember that Cronbach’s alpha is influenced by the length of the scale (Cortina, 1993). Each of these scales contains only three items, and the alpha value would likely increase if more, consistent, items were added to the factors. The factors are still included in the emergent factor structure at this point as the alpha values are close to .5, and the scale consists of only three items. The inter-item correlation matrices for each factor are shown in Table 4.9 and Table 4.10, respectively. The low correlations between the items suggest that, while the items do load together, they do not create strong factors. The factors have been left in this model, although a re-examination of the items is suggested. Low alpha values also suggest that a scale is not unidimensional, as items must be 75  internally consistent if they are homogeneous (Cortina, 1993). It is, therefore, possible that there exists other layers to these factors, but there are not enough items in the factor to adequately piece apart these differences.  Table 4.9: Inter-item correlation matrix for Factor 4  v17 v27 v29 v17 1.000 .205 .263 v27 .205 1.000 .162 v29 .263 .162 1.000  Table 4.10: Inter-item correlation matrix for Factor 5  v7 v18 v41 v7 1.000 .175 .180 v18 .175 1.000 .293 v41 .180 .293 1.000 4.2.2.5 Whole Scale The reliability of the 22-item scale, before removing items 5, 35, and 40 as suggested by reliability analyses of the individual factors, is lower than the accepted .7 cutoff value (α = .417) (Table 4.11). Removing the suggested items drastically improves the reliability of the 18 item scale to .617 (Table 4.12). Removal of these items is supported by analysis of the internal consistency of the complete scale, which indicates removing these items would result in an increased Cronbach’s alpha value (Table 4.11).    76  Table 4.11: Reliability statistics for the 22-item scale which emerged from exploratory factor analysis. Item Scale Mean if Item Deleted Scale Variance if Item Deleted Corrected Item-Total Correlation Cronbach's Alpha if Item Deleted v3 60.6706 26.959 .293 .319 v14 60.4899 27.536 .296 .325 v28 60.2922 28.065 .254 .336 v35 62.4257 32.289 -.173 .434 v37 60.9189 26.985 .276 .322 v5 61.7466 30.213 -.012 .402 v12 62.3547 30.256 -.002 .398 v21 60.9172 30.739 -.058 .416 v25 60.4493 28.326 .233 .342 v34 60.7095 29.760 .111 .370 v40 62.2905 32.548 -.199 .434 v24 60.1284 29.026 .181 .355 v26 60.2196 28.706 .222 .347 v32 62.4882 32.535 -.199 .432 v42 60.3666 28.016 .278 .332 v17 62.1081 29.217 .104 .371 v27 61.5236 28.463 .127 .364 v29 62.9510 30.707 .017 .386 v7 61.5693 28.977 .110 .369 v18 61.9916 28.076 .170 .352 v41 61.0574 27.594 .233 .336   The internal consistency of the emergent 18-item tool suggests that the removal of the items in Factor 2 would increase the overall reliability of the scale (Table 4.12). However, the items were left in the model for two reasons. First, the internal consistency of the factor was acceptable. Second, it is expected for the reliability of a scale to be decreased when multiple factors underlie the data (Cortina, 1993). This is because a scale with multiple factors (or dimensions) no longer measures a single construct, which means that the scale is not unidimensional. Exaggerated low or high alpha values also suggest a multi-dimensional scale.   77  Table 4.12: Reliability statistics for the 18-item scale resulting after removing items 35, 5, 40, and 32  Scale Mean if Item Deleted Scale Variance if Item Deleted Corrected Item-Total Correlation Cronbach's Alpha if Item Deleted v3 56.2162 33.882 .448 .567 v14 56.0355 35.219 .401 .579 v28 55.8378 35.872 .356 .586 v30 55.8007 36.322 .340 .589 v37 56.4645 34.811 .352 .583 v12 57.9003 37.863 .124 .618 v21 56.4628 37.511 .121 .621 v25 55.9949 35.508 .402 .580 v34 56.2551 37.101 .297 .596 v24 55.6740 37.668 .216 .605 v26 55.7652 36.552 .337 .590 v42 55.9122 36.013 .364 .586 v17 57.6537 39.553 .001 .635 v27 57.0693 38.515 .051 .632 v29 58.4966 40.775 -.080 .634 v7 57.1149 38.115 .103 .622 v18 57.5372 37.816 .109 .622 v41 56.6030 36.619 .218 .604  4.2.3 Emergent Model Using exploratory factor analysis, five viable factors were extracted from the data (Tables 4.1 and 4.2). The items in each factor are listed in Table 4.3 These factors were refined based on Cronbach’s alpha values for each factor. This involved deletion of items that contributed to the violation of reliability rules or had a detrimental effect on the internal consistency of the factor.  Table 4.13 outlines the model that emerged from the above exploratory factor analysis and reliability analysis. We will call this Model 1.    78  Table 4.13: Emergent factor model and items. Model 1.  Factor Items Factor 1 3. I think about the physics I experience in everyday life. 14. I study physics to learn knowledge that will be useful in my life outside of school. 28. Learning physics changes my ideas about how the world works. 30. Reasoning skills used to understand physics can be helpful to me in my everyday life. 37. To understand physics, I sometimes think about my personal experiences and relate them to the topic being analyzed. Factor 2 12. I cannot learn physics if the teacher does not explain things well in class. 21. If I don't remember a particular equation needed to solve a problem on an exam, there's nothing much I can do (legally!) to come up with it. 25. I enjoy solving physics problems. 34. I can usually figure out a way to solve physics problems. Factor 3 24. In physics, it is important for me to make sense out of formulas before I can use them correctly. 25. I enjoy solving physics problems. 26. In physics, mathematical formulas express meaningful relationships among measurable quantities. 42. When studying physics, I relate the important information to what I already know rather than just memorizing it the way it is presented. Factor 4 17. Understanding physics basically means being able to recall something you've read or been shown. 27. It is important for the government to approve new scientific ideas before they can be widely accepted. 29. To learn physics, I only need to memorize solutions to sample problems. Factor 5 7. As physicists learn more, most physics ideas we use today are likely to be proven wrong. 18. There could be two different correct values to a physics problem if I use two different approaches. 41. It is possible for physicists to carefully perform the same experiment and get two very different results that are both correct. Control Item 31. We use this question to discard the survey of people who are not reading the statements. Please select agree - option 4 (not strongly agree) to preserve your answers.  In the next section, the fit of Model 1 is tested in three other courses. Four alternative models, Models 2, 3, 4, and 5, evolve based on the confirmatory factor analysis output.  79  4.3 Testing the Emergent Model In order to test the generalizability of the emergent factor model (Table 4.13) in courses outside the 2011 engineering course, confirmatory factor analysis was used to evaluate model fit in three other courses. These courses included the previous year’s introductory engineering physics course, 2010, as well as two years of the introductory physics course for science students, 2013 and 101 (Table 3.2).   As a reminder, the second engineering course (from 2010) was used for the first test of the emergent model, due to the assumed similarities between the 2010 and 2011 cohorts. The 2013 calculus-based course was tested next, followed by the 2012 calculus-based course. When changes were made to the model based on the model fit for one course, the new model was checked for the other two courses.   Model 1 is the five-factor model outlined in Table 4.13, which emerged from exploratory factor analysis and reliability tests. Analysis of Model 1 in each course necessitated the removal of item 7, which loaded very weakly (below .300) to Factor 5. A good factor has at least three items, and as a result Factor 5 was removed, resulting in the four-factor Model 2. Examination of the modification indices and standardized residual matrix for the fit of Model 2 in each of the courses suggested that item 21 might cross load on Factors 2 and 4. Model 3 takes into account this modification, and Factor 5 remains absent. Model 4 maintains the structure of Model 3, with two additions: item 27 now cross loads on Factor 3 and Factor 4, and item 30 now cross loads on Factor 1 and Factor 4. In a quest to find a simple factor structure underlying the data, I returned to the fit of Model 2, which suggested also removing Factor 4 because item 27 loaded weakly (below .300). In addition, modification indices of the previous analyses in each course suggested correlating the error covariances for items 3 and 37. This decision is supported by the items, 80  which are similar statements (item 3: I think about the physics I experience in everyday life, and item 37: To understand physics I sometimes think about my personal experiences and relate them to the topic being analyzed). This three-factor model is Model 5.  The model fit indices for each model are seen in Tables 4.14, 4.15, 4.16, 4.17, and 4.18. Here the Engineering course is referred to as “Eng” and the Calculus-Based course as “CB”. The progression of the goodness-of-fit indices for each iteration in each course can be seen in Tables 4.19, 4.20, and 4.21. Moving through the five models, it is clear that the models improve at each stage. For each iteration of the emergent model, the fit is best for the engineering course. This is to be expected, as fewer differences exist between students in two years of the same course than between the students in two different courses.   Table 4.14: Goodness-of-fit indices for Model 1 Index Eng –2010 CB –2012 CB – 2013 χ2 217.821** 385.884** 338.002** df 124 124 124 SRMR .0460 .0502 .0505 CFI .926 .881 .897 RMSEA (90% CI) .036 (.028, .043) .053 (.047, .059) .052 (.045, .058) **p<0.005  Table 4.15: Goodness-of-fit indices for Model 2 Index Eng –2010 CB –2012 CB – 2013 χ2 167.129** 306.239** 253.650** df 83 83 83 SRMR .0478 .0490 .0499 CFI .927 .892 .913 RMSEA (90% CI) .041 (.032, .050) .060 (.052, .067) .056 (.049, .064) **p<0.005    81  Table 4.16: Goodness-of-fit indices for Model 3 Index Eng –2010 CB –2012 CB – 2013 χ2 166.873** 254.807** 220.813** df 82 82 82 RMR .0477 .0429 .0456 CFI .926 .916 .929 RMSEA (90% CI) .042 (.032, .051) .053 (.045, .060) .051 (.043, .059) **p<0.005  Table 4.17: Goodness-of-fit indices for Model 4 Index Eng –2010 CB –2012 CB – 2013 χ2 141.715** 204.401** 167.630** df 79 79 79 RMR .0409 .0390 .0368 CFI .946 .939 .955 RMSEA (90% CI) .036 (.027, .046) .046 (.038, .053) .042 (.033, .050) **p<0.005  Table 4.18: Goodness-of-fit indices for Model 5 Index Eng –2010 CB –2012 CB – 2013 χ2 90.478** 144.752** 138.667** df 49 49 49 RMR .0379 .0379 .0388 CFI .959 .947 .948 RMSEA (90% CI) .038 (.025, .050) .051 (.041, .060) .053 (.043, .064) **p<0.005  Table 4.19: Confirmatory factor analysis results for Engineering – 2010. N = 601.  Index Model 1 Model 2 Model 3 Model 4 Model 5 χ2 217.821** 167.129** 166.873** 141.715** 90.478** df 124 83 82 79 49 RMR .0460 .0478 .0477 .036 .0379 CFI .926 .927 .926 .946 .959 RMSEA (90% CI) .036 (.028, .043) .041 (.032, .050) .042 (.032, .051) .036 (.027, .046) .038 (.025, .050) **p<0.005    82  Table 4.20: Confirmatory factor analysis results for Calculus-Based – 2012 (N = 760)  Table 4.21: Confirmatory factor analysis results for Calculus-Based – 2013 (N = 647) Index Model 1 Model 2 Model 3 Model 4 Model 5 χ2 338.002** 253.650** 220.813** 167.630** 138.667** df 124 83 82 79 49 RMR .0505 .0499 .0456 .0368 .0388 CFI .897 .913 .929 .955 .948 RMSEA (90% CI) .052 (.045, .058) .056 (.049, .064) .051 (.043, .059) .042 (.033, .050) .053 (.043, .064) **p<0.005  Although the fit is acceptable in all courses for Model 4, Model 5 provides a better fit in each course. For this reason, Model 5, with three factors, has been confirmed as the revalidated model underlying the CLASS instrument from this analysis.  4.4 Revalidated Model Table 4.22 outlines the revalidated three-factor model confirmed by this work (Model 5). Table 4.23 includes the factors, which were extracted from the data, but were subsequently removed based on the outcome of the confirmatory factor analysis. Table 4.24 contains the items, which were removed from the survey tool following exploratory factor analysis.    Index Model 1 Model 2 Model 3 Model 4 Model 5 χ2 385.884** 306.239** 254.807** 204.401** 144.752** df 124 83 82 79 49 RMR .0502 .0490 .0429 .0390 .03779 CFI .881 .892 .916 .939 .947 RMSEA (90% CI) .053 (.047, .059) .060 (.052, .067) .053 (.045, .060) .046 (.038, .053) .051 (.041, .060) **p<0.005 83  Table 4.22: Revalidated factor model underlying the CLASS instrument, as confirmed by this research.  Factor Items Factor 1 Awareness of Real World Connections 3 I think about the physics I experience in everyday life. 14 I study physics to learn knowledge that will be useful in my life outside of school. 28 Learning physics changes my ideas about how the world works. 30 Reasoning skills used to understand physics can be helpful to me in my everyday life. 37 To understand physics, I sometimes think about my personal experiences and relate them to the topic being analyzed. Factor 2 Self-Efficacy 12 I cannot learn physics if the teacher does not explain things well in class. 21 If I don't remember a particular equation needed to solve a problem on an exam, there's nothing much I can do (legally!) to come up with it. 25 I enjoy solving physics problems. 34 I can usually figure out a way to solve physics problems. Factor 3 Constructive Connectivity 24 In physics, it is important for me to make sense out of formulas before I can use them correctly. 25 I enjoy solving physics problems. 26 In physics, mathematical formulas express meaningful relationships among measurable quantities. 42 When studying physics, I relate the important information to what I already know rather than just memorizing it the way it is presented. Control Item 31 We use this question to discard the survey of people who are not reading the statements. Please select agree - option 4 (not strongly agree) to preserve your answers.  Table 4.23: Factors that were found and subsequently removed from the emergent Model 1 during this research.  Factor Items Factor 4 17 Understanding physics basically means being able to recall something you've read or been shown. 27 It is important for the government to approve new scientific ideas before they can be widely accepted. 29 To learn physics, I only need to memorize solutions to sample problems. Factor 5 7 As physicists learn more, most physics ideas we use today are likely to be proven wrong. 18 There could be two different correct values to a physics problem if I use two different approaches. 41 It is possible for physicists to carefully perform the same experiment and get two very different results that are both correct.    84  Table 4.24: The 16 items that were removed from the scale. Bolded items were initially part of a factor, but were subsequently removed.  Removed Items  1 A significant problem in learning physics is being able to memorize all the information I need to know. 2 When I am solving a physics problem, I try to decide what would be a reasonable value for the answer. 4 It is useful for me to do lots and lots of problems when learning physics. 5 After I study a topic in physics and feel that I understand it, I have difficulty solving problems on the same topic. 6 Knowledge in physics consists of many disconnected topics. 8 When I solve a physics problem, I locate an equation that uses the variables given in the problem and plug in the values. 9 I find that reading the text in detail is a good way for me to learn physics. 10 There is usually only one correct approach to solving a physics problem. 11 I am not satisfied until I understand why something works the way it does. 13 I do not expect physics equations to help my understanding of the ideas; they are just for doing calculations. 15 If I get stuck on a physics problem on my first try, I usually try to figure out a different way that works. 16 Nearly everyone is capable of understanding physics if they work at it. 19 To understand physics I discuss it with friends and other students. 20 I do not spend more than five minutes stuck on a physics problem before giving up or seeking help from someone else. 22 If I want to apply a method used for solving one physics problem to another problem, the problems must involve very similar situations. 23 In doing a physics problem, if my calculation gives a result very different from what I'd expect, I'd trust the calculation rather than going back through the problem. 32 Spending a lot of time understanding where formulas come from is a waste of time. 33 I find carefully analyzing only a few problems in detail is a good way for me to learn physics. 35 The subject of physics has little relation to what I experience in the real world. 36 There are times I solve a physics problem more than one way to help my understanding. 38 It is possible to explain physics ideas without mathematical formulas. 39 When I solve a physics problem, I explicitly think about which physics ideas apply to the problem. 40 If I get stuck on a physics problem, there is no chance I'll figure it out on my own. 4.5 Results Summary The analysis of quantitative data in the preceding sections explored the relationships between the items of the CLASS survey. A five-factor model emerged from this analysis, and three of these factors are included in the revalidated model. The three revalidated factors are 85  labelled: ‘Awareness of Real World Connections’, ‘Self-Efficacy’, and ‘Constructive Connectivity’. In the following sections I will further discuss the value of these factors, and the possible implications for the development of theory, research, and curriculum. First, I will discuss the revalidated factors, confirmed through confirmatory factor analysis, that underlie students’ attitudes and beliefs about physics before entering an introductory physics course. Second, possible implications for instruction and curricular design will be suggested. Suggestions for improving the robustness of this revalidated model will also be discussed. 4.6 Factors The first factor (Table 4.25) contains items which focus on the relationship between classroom learning and “real-world” experiences. It is interesting to note that the items are not unidirectional, and focus on making connections both from the classroom to the outside world and from the outside world to the classroom. In determining a label for this first factor, the goal was to describe how students think about the dual directionality of these connections. ‘Awareness of Real World Connections’ is the label applied to the construct being measured by this factor. In order to respond favourably to the items (based on the current CLASS scoring scheme (Adams et al., 2006)) a student must have some awareness of how classroom learning and the real world are connected. In other words, their attitudes and beliefs are in part shaped by how aware the individual is about connections between what they learn and the real world.    86  Table 4.25: The first factor in the revalidated model. Factor Title Items Awareness of Real World Connections 3 I think about the physics I experience in everyday life. 14 I study physics to learn knowledge that will be useful in my life outside of school. 28 Learning physics changes my ideas about how the world works. 30 Reasoning skills used to understand physics can be helpful to me in my everyday life. 37 To understand physics, I sometimes think about my personal experiences and relate them to the topic being analyzed.  The second factor (Table 4.26) is more difficult to describe. The items suggests the factor probes self-efficacy, self-confidence, problem solving confidence, persistence, or how the student views themselves in relation to physics learning. Based on how self-efficacy has been operationalized in this work, the label is given to this factor. Self-efficacy is a measure of how confident a student is in their ability to reach a desired outcome (Bandura, 1977; “Self-Efficacy,” 2010), which is reflected in the four items of Factor 2. Self-efficacy also informs the amount of effort a student is willing to invest in a particular task in order to be successful. A high level of self-efficacy indicates a student is more likely to persist through challenging tasks.  Table 4.26: Second factor in the revalidated model Factor Title Items Self-Efficacy 12 I cannot learn physics if the teacher does not explain things well in class. 21 If I don't remember a particular equation needed to solve a problem on an exam, there's nothing much I can do (legally!) to come up with it. 25 I enjoy solving physics problems. 34 I can usually figure out a way to solve physics problems.  The third factor (Table 4.27) focuses more evidently on the skills needed to make sense of new physics knowledge. The original CLASS category ‘Problem Solving General’ included items 25, 26, and 42, while items 24 and 42 were part of the ‘Sense Making/Effort’ category. 87  These items were interpreted to probe the processes students use to make sense of their learning, and connect concepts together. The ability to construct an understanding of the connections that exist between various concepts is an indicator of a scientifically literate individual (Hodson, 1998, p. 3). Following the language used by Thomas, Anderson, and Nashon, (2008), the third factor is labeled ‘Constructive Connectivity’.   Table 4.27: Third factor in the revalidated model Factor Title Items Constructive Connectivity 24 In physics, it is important for me to make sense out of formulas before I can use them correctly. 25 I enjoy solving physics problems. 26 In physics, mathematical formulas express meaningful relationships among measurable quantities. 42 When studying physics, I relate the important information to what I already know rather than just memorizing it the way it is presented. These three factors measure distinct yet connected aspects of students’ attitudes about physics. In the following section I describe potential implications of these factors for future use of the CLASS instrument, as well as the development of theory, pursuit of research, and curricular design.  4.7 A revalidated CLASS This research identifies the fluid nature of the factors and factor structures underlying different groups of students’ attitudes and beliefs about physics as measured by the CLASS (Table 4.22). This investigation was prompted by the inability to reproduce the existence of the eight published factors of the CLASS (Adams et al., 2006) through confirmatory factor analysis of data collected in this population. Five factors emerged from this process, and the three factors underlying students’ attitudes and beliefs towards physics in this population are included in the revalidated model. In the future, researchers with an interest in using the CLASS should be 88  conscious of the validity of the instrument in their population. In addition, the extent to which students’ scores are extrapolated should be approached with caution. This study does not investigate the full implications of scores collected using the CLASS. The interpretive argument of the CLASS is currently incomplete, and CLASS scores have been interpreted in a number of different lenses (e.g. Otero & Gray, 2008; Perkins et al., 2005; Slaughter et al., 2011) . In order to continue to improve the validity of the CLASS instrument, it is important to develop every aspect of the interpretive argument surrounding the interpretation and use of the test scores. Suggestions for further improvement of the validity of the instrument will be made in the final chapter of this thesis.  4.8 Potential Implications of the Revalidated Model The three factors that emerged from and were confirmed in this research (‘Awareness of Real World Connections’, ‘Self-Efficacy’, and ‘Constructive Connectivity’) propose a means of examining three facets of students’ attitudes and beliefs about physics. These have the potential to impact theory, course design, and research in relation to students’ attitudes and beliefs about physics. These implications do not exist in isolation. Theory informs course design, pedagogical decisions, and research projects. Research projects can impact aspects of course design, both as a part of or as a result of research. As a result of research, theory can be augmented, altered, or abandoned. The following is a discussion of possible implications within each domain, supported by suggestions for overlap with the other two. It should be noted that these are merely suggestions, and rely heavily on the need to improve the validity of the CLASS instrument.  89  4.8.1 Theory Previous development of instruments to measure students’ attitudes towards physics has chosen to identify items and dimensions of attitudes largely based on the researchers’ choice and experience (Adams et al., 2006; Elby, 2001; Redish et al., 1998; Yerdelen-Damar et al., 2012). Developing a stronger theoretical foundation for the study of students’ attitudes about physics will assist the pursuit of research in a variety of issues central to physics education research. In particular, the development of valid, quantitative survey tools relies heavily on clearly defined theories. To date, the majority of the available quantitative instruments designed to measures students’ attitudes and beliefs have a weak foundation in theory. This study suggests a possible theoretical structure underlying students’ attitudes towards physics before beginning an introductory, undergraduate physics course. In order to further develop this theory, more research is needed to investigate the prevalence of this theory across populations. This structure could inform course design, and the impact of these pedagogical decisions can be determined through a well thought-out research project.  The three factors in the revalidated model research measure different, yet related, aspects of students’ attitudes about learning physics. An individual with a high self-efficacy will put more effort into constructing their own understanding of physics knowledge. Having a deep conceptual understanding of physics concepts suggests a strong awareness of how physics is present in real world situations. This awareness could then influence how an individual creates connections between physics concepts. There is need for a more thorough understanding of how these three factors are related.  This understanding then has the potential to influence how introductory physics courses are designed and inform research projects aimed at examining both how these factors manifest and how they are impacted by pedagogical decisions.  90  Finally, it is necessary to further understand how attitudes are developed and shaped in particular contexts. Research projects aimed at developing a deeper awareness of the factors that influence students’ attitudes towards physics, both positively and negatively, will add to the physics education literature.  4.8.2 Research As described above, valuable research requires a strong theoretical foundation. However, to fully understand the connection between students’ attitudes about physics, achievement, persistence, and learning gains, more research is needed in this area. As previously discussed, the relationship between theory, course design, and research is not unidirectional. Further research will help develop and inform a strong theory underlying the dimensions of students’ attitudes about physics and learning physics.  To better understand how attitudes towards physics are shaped, more research is needed on several fronts. It is crucial to have valid and reliable instruments that measure students’ attitudes, which requires more research evidence examining the validity of these instruments, including the CLASS. This could involve a re-examination of the items not included in the revalidated three-factor structure that emerged from and was confirmed by this research. The stability of the factors over time deserves scrutiny as well. This research focuses solely on students’ attitudes towards physics at the outset of an introductory course, at the very beginning of their undergraduate career. To extend the applicability and implications of the revalidated factors in this study, it is necessary to investigate the factors that underlie students’ attitudes as they move through an undergraduate program. This includes pre- and post- instruction for all years of study.  91  Developing a deeper understanding of how attitudes about physics impact students, as well as how these attitudes are shaped, has the ability to inform curriculum development at all levels of undergraduate education. In order to have adequate reason to make these adjustments, understanding the factors that influence attitudes over time is vital. These factors could include instructional methods and learning experiences, as well as the type and amount of feedback provided by the instructor. Additionally, the relationship between attitudes and student success or persistence is not clear. This is due in part to the complex nature of the persistence puzzle (Adeyemi, 2010; Chen & Weko, 2009; Goulden, Mason, & Frasch, 2011; Sawtelle et al., 2012; Seymour & Hewitt, 1997). Further investigation of the connection between students’ attitudes and these other factors would add significantly to the understanding of why students succeed and persist.  4.8.3 Curriculum Development As a group, these factors have the potential to suggest interesting curricular decisions in introductory physics courses. Understanding how attitudes towards physics are shaped will inform the need for specific focus on the development of these attitudes, along with physics knowledge at all levels of undergraduate education It should be noted, however, that the following are merely suggestions based upon the statistical analysis conducted in this study and research studies conducted with the original CLASS survey. Given that the current study calls into question the validity and reliability of the original survey structure and subsequent scoring system, these results are taken with trepidation, and serve as a guide, rather than a rule, for implementation and investigation. The development of a second version of the Maryland Physics Expectations survey (MPEX2) suggested focusing on the item clusters as broad instructional 92  goals (McCaskey, 2009). This discussion adopts a similar mentality, although it is believed that with further validation work the psychometric value of measurements obtained using the CLASS instrument could be improved.  Previous research has suggested increases in students’ overall attitudes can occur when courses are designed to provide students with authentic physics-related experiences (Redish & Hammer, 2009). This literature, combined with the existence here of a factor probing how students connect classroom learning to the real-world suggests that this experience is an active participant in developing expert-like attitudes towards physics. Current examples of these experiences include Modeling Instruction (Brewe et al., 2009, 2013; Hart, 2008; Hestenes, 1987) and inquiry-based pedagogies (Lindsey et al., 2012; Otero & Gray, 2008). These studies also suggest maintaining smaller class sizes for a variety of reasons. Science labs are often composed of smaller communities, in which collaboration is encouraged. The large classes seen in many introductory physics courses tend to turn into information pipelines, with the instructor transmitting information with the assumption that students will incorporate it into their understanding. Smaller class sizes allow instructors to spend more time with students, helping the individual develop their own understanding and skills.  Another example would be providing more students with the opportunity to conduct their own research as part of a capstone project. Involving students directly in the process of developing and conducting a research project enables students to experience firsthand how science exists outside of the classroom. Providing students with authentic experiences can help them understand how physics learning can connect to the world outside of the classroom. In turn, this understanding can improve how students interpret the connections between physics concepts.  93  Turning the discussion to the dual-directionality seen in the items of the ‘Real World Connection’ factor, instructors should consider how students connect classroom learning to real-world experiences, and how real-world experiences impact student learning. Providing authentic experiences to students can facilitate these connections. Possible avenues could include having students examine real-world phenomena (rainbows, planetary orbits, and echolocation) and describe the physics involved. Going in the opposite direction, students could be asked to consider the number of ways physical concepts are manifested in their daily lives. Both activities require students to think at a deeper level about how physics can be valuable outside of the classroom. These skills are also invaluable in making sense of problem solving and experimental results, which in turn help students build relevant skills  In this structure, the ‘Self-Efficacy’ factor could be interpreted as an indication of how willing a student is to struggle with physics content. As previously discussed, self-efficacy is a measure of how confident a student is in their ability to successfully use the skills at their disposal (Bandura, 1986, p. 391). This factor supports pedagogical decisions designed to help students develop problem solving skills and the ability to construct knowledge with limited assistance from the instructor. Research has shown that students’ problem solving skills evolve over the course of instruction (Ogilvie, 2009). Providing specific opportunities for students to experience and practice struggling with authentic problems could help students develop a deep conceptual understanding of how physics concepts are related, while gaining confidence in their ability to confront unknown situations.  The presence of a factor probing how students tend to construct their knowledge suggests an investigation into how students develop understandings of physics concepts and developing methods to support students in this construction. A basic constructivist tenet of physics education 94  research is the focus on the knowledge students bring to instruction (McCaskey, 2009). Interpreting how students construct understandings of new information can provide instructors with valuable insight into how to best offer educational support. High scores (maintaining the current CLASS expert scoring system) indicate students approach knowledge integration in an expert-like manner and require less support. Lower expert-like scores suggest students need a deeper level of support in building connections between concepts, formulas, and contexts. In order to have a lasting impact on students’ attitudes towards physics, these efforts should be imbedded in the curriculum at all years of study, from first year through graduate studies. In many institutions, introductory courses boast a variety of reformed instructional methods, with upper-year courses tending more towards a more traditional approach. Slaughter, Bates, and Galloway, (2012) found that students’ attitudes remained relatively stagnant as they moved from first to third year when instruction turned increasingly to more traditional methods. Further research is required to examine the impacts of alternative teaching methods at all levels of undergraduate education, as well as how attitudes are shaped as students move through a degree program.  4.9 Discussion Summary In this study, five factors underlying students’ attitudes students’ attitudes at the outset of an introductory physics course for engineers emerged from statistical analysis (Table 4.3). Three of these factors were confirmed in a second set of data from the same course, as well as two years of an introductory, calculus-based physics course for science students (Table 4.22   95  Table 4.22). These factors are ‘Awareness of Real World Connections’ (Table 4.25), ‘Self-Efficacy’ (Table 4.26), and ‘Constructive Connectivity’ (Table 4.27). Implications of these factors include furthering theoretical understandings of how students’ attitudes change and are shaped, researching the factors that influence attitude change, and altering curriculum to focus on increasing students’ attitudes towards physics.  In the next and final chapter of this thesis, I present a summary of the research project and findings. I also answer the research questions and suggest areas of future research.   96  5 Conclusions 5.1 Summary Physics education researchers have turned to attitude surveys to provide insight into what attitudes and beliefs students enter instruction with, and how these shift over the course of instruction (Adams et al., 2006; Elby, 2001; Halloun, 1997; Redish et al., 1998). This study aimed to add to the validity evidence surrounding the CLASS, a popular physics attitudes survey, by answering the following research questions:  1. What are the factors underlying undergraduate students’ attitudes and beliefs about physics prior to participating in an introductory physics course?  2. What might the implications of these factors be on course design and consequent student physics learning?  The study followed a quantitative design and relied on statistical analysis of data collected from first year engineering (n = 1193) and science students (n = 1407) at a large research university using the CLASS. Statistical analysis of the collected data did not support the existence of the eight categories of the published CLASS (Adams et al., 2006). Instead, the analysis indicated the existence of five factors, and confirmed three, underlying students’ attitudes and beliefs about physics as measured by the CLASS. These three factors are labelled ‘Awareness of Real World Connections’, ‘Self-Efficacy’, and ‘Constructive Connectivity’. Interpretation of the three revalidated factors, supported by the metacognitive theories that guided this research, informed suggestions for implications to theory, research, and curriculum.   97  5.2 Conclusions: Answering the Research Questions 5.2.1 Question 1: What are the factors underlying undergraduate students’ attitudes and beliefs about physics prior to participating in an introductory physics course?  The analysis of quantitative data in the preceding sections explored the relationships between the items of the CLASS survey in order to suggest an alternate framework that would be valid in the study population. Through exploratory factor analysis, five factors emerged as underlying students’ attitudes towards physics before undergraduate instruction. Three of these factors were confirmed across multiple data sets using confirmatory factor analysis. The three revalidated factors are: ‘Awareness of Real World Connections’; ‘Self-Efficacy’; and ‘Constructive Connectivity’. The first factor probes how students connect classroom learning to the real world and how students connect real world learning to the classroom. The second factor identifies how confident a student is in their ability to learn physics and tackle problems in their learning. The third factor examines the extent to which students construct their own understanding of the information available to them.  5.2.2 Question 2: What might the implications of these factors be on course design and consequent student physics learning?  The existence of these three factors across two courses over a four year period has several implications for the development of theory, research projects, and curriculum. This was previously explored in Section 4.8 of the Results and Discussion chapter. Space exists to further develop the theory surrounding students’ attitudes and beliefs about physics and 98  learning physics using the factor structure found in this study as a guide. This can be accomplished through research projects designed to investigate how attitudes are shaped and the factors that influence these changes. In turn, well-developed theory and a desire to conduct further research can inform a well-rounded curriculum designed to have a positive impact on students’ attitudes about physics at all levels of instruction. In order to continue using the CLASS as a research tool, it is suggested that more evidence be collected in support of the validity of the tool.  5.3 Future Research The findings in this study support the need for further examination of the factors that underlie students’ attitudes and beliefs about learning physics.  It is always recommended to add whatever evidence possible to the validity argument for or against an instrument (Heredia & Lewis, 2012). For example, the factor model of the current published version of the CLASS identifies eight factors (Adams et al., 2006), while the revalidated model in this research contains three. Five factors emerged from exploratory factor analysis, and it is possible that a different subset of these factors could exist in different populations. The CLASS was developed to be applicable across a broad range of populations, contexts, and levels of undergraduate study. To increase the validity of the revalidated factors in this study, it is recommended that additional data be collected from this population and others to examine the stability of these factors across contexts.  Also of interest is the stability of these factors in the same group of students over the course of instruction. The CLASS was designed for use pre- and post- instruction (Adams et al., 2006). In this work, I discuss students’ attitudes at the outset of a course, before 99  instruction. To continue the use of the CLASS as a measure of shifts in students’ attitudes over the course of instruction, the existence of the three revalidated factors should be investigated in students’ data post-instruction. Additionally, to extrapolate from changes in attitude scores to changes in other variables, such as achievement, persistence, or the effectiveness of an intervention, further research is required to examine the nature of the relationships among these elements.   The CLASS was designed to be valid and reliable across all areas of physics and at all years of study (Adams et al., 2006). This research reports on the factor structure in two introductory physics courses: a course for engineering students; and a calculus-based course for science majors. Further evidence is required to support the use of this factor model in courses beyond the first year of instruction. To increase the validity of this model outside of this population, further analysis should be conducted at other institutions.  The current, published version of the CLASS contains 42 items. Ten of these items do not belong to any of the eight categories, while six are unscored and do not impact the overall score. In this study, only 13 of the original 42 items were retained in the revalidated factor model. The remaining items should be examined and re-evaluated for their value to the current instrument. It is possible that with some alterations, the items could be valuable additions to the three-factor model provided here.  Finally, the implications of students’ scores on the CLASS should be investigated. The interpretive argument of the CLASS is currently unclear as to how students’ scores can be generalized and extrapolated. Furthering this understanding will increase the validity of 100  the instrument and the breadth and strength of research into students’ attitudes and beliefs about physics.   101  References Adams, W. K., Perkins, K. K., Dubson, M., Finkelstein, N. D., & Wieman, C. E. (2005). The Design and Validation of the Colorado Learning Attitudes about Science Survey. AIP Conference Proceedings, 790(1), 45–48. doi:10.1063/1.2084697 Adams, W. K., Perkins, K. K., Podolefsky, N. S., Dubson, M., Finkelstein, N. D., & Wieman, C. E. (2006). New instrument for measuring student beliefs about physics and learning physics: The Colorado Learning Attitudes about Science Survey. Physical Review Special Topics-Physics Education Research, 2(1), 010101. Adams, W. K., & Wieman, C. E. (2011). Development and Validation of Instruments to Measure Learning of Expert‐Like Thinking. International Journal of Science Education, 33(9), 1289–1312. doi:10.1080/09500693.2010.512369 Adeyemi, A. Y. (2010). Factors impacting females’ decision to pursue mathematics-related careers: A case study approach (M.Ed.). University of Windsor (Canada), Canada. Retrieved from http://search.proquest.com/docview/755040997/abstract?accountid=14656 Aiken, L. R. (2002). Attitudes and related psychosocial constructs: Theories, assessment, and research. Thousand Oaks, Calif: Sage Publications. Ajzen, I., & Madden, T. J. (1986). Prediction of goal-directed behavior: Attitudes, intentions, and perceived behavioral control. Journal of Experimental Social Psychology, 22(5), 453–474. doi:10.1016/0022-1031(86)90045-4 Alhadlaq, H., Alshaya, F., Alabdulkareem, S., Perkins, K. K., Adams, W. K., & Wieman, C. E. (2009). Measuring students’ beliefs about Physics in Saudi Arabia. In 2009 Physics Education Research Conference Proceedings (Vol. 1179, pp. 69–72). 102  Retrieved from http://www.researchgate.net/publication/234953999_Measuring_Students_Beliefs_about_Physics_in_Saudi_Arabia/file/32bfe51215b92c19f0.pdf American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing (U.S.). (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Anderson, D., & Nashon, S. (2007). Predators of knowledge construction: Interpreting students’ metacognition in an amusement park physics program. Science Education, 91(2), 298–320. doi:10.1002/sce.20176 Anderson, D., Nashon, S. M., & Thomas, G. P. (2009). Evolution of Research Methods for Probing and Understanding Metacognition. Research in Science Education, 39(2), 181–195. doi:10.1007/s11165-007-9078-1 Baldwin, J. A., Ebert-May, D., & Burns, D. J. (1999). The development of a college biology self-efficacy instrument for nonmajors. Science Education, 83(4), 397–408. doi:10.1002/(SICI)1098-237X(199907)83:4<397::AID-SCE1>3.0.CO;2-# Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215. doi:10.1037/0033-295X.84.2.191 Bandura, A. (1986). Social foundations of thought and action. Prentice-Hall. Barbera, J., Adams, W. K., Wieman, C. E., & Perkins, K. K. (2008). Modifying and Validating the Colorado Learning Attitudes about Science Survey for Use in Chemistry. Journal of Chemical Education, 85(10), 1435. doi:10.1021/ed085p1435 103  Barrantes, A., Pawl, A., Pritchard, D. E., Sabella, M., Henderson, C., & Singh, C. (2009). What do Seniors Remember from Freshman Physics? Retrieved from http://dspace.mit.edu/handle/1721.1/76363 Bates, S. P., Galloway, R. K., Loptson, C., & Slaughter, K. A. (2011). How attitudes and beliefs about physics change from high school to faculty. Physical Review Special Topics - Physics Education Research, 7(2), 020114. doi:10.1103/PhysRevSTPER.7.020114 Bauer, C. F. (2005). Beyond “Student Attitudes”: Chemistry Self-Concept Inventory for Assessment of the Affective Component of Student Learning. Journal of Chemical Education, 82(12), 1864. doi:10.1021/ed082p1864 Bauer, C. F. (2008). Attitude toward Chemistry: A Semantic Differential Instrument for Assessing Curriculum Impacts. Journal of Chemical Education, 85(10), 1440. doi:10.1021/ed085p1440 Bland, J. M., & Altman, D. G. (1997). Cronbach’s alpha. BMJ : British Medical Journal, 314(7080), 572. Blickenstaff, J. C. (2005). Women and science careers: leaky pipeline or gender filter? Gender and Education, 17(4), 369–386. doi:10.1080/09540250500145072 Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn. National Academy Press Washington, DC. Retrieved from http://www.csun.edu/~SB4310/How%20People%20Learn.pdf Brewe, E., Kramer, L., & O’Brien, G. (2009). Modeling instruction: Positive attitudinal shifts in introductory physics measured with CLASS. Physical Review Special Topics - Physics Education Research, 5(1), 013102. doi:10.1103/PhysRevSTPER.5.013102 104  Brewe, E., Traxler, A., de la Garza, J., & Kramer, L. H. (2013). Extending positive CLASS results across multiple instructors and multiple classes of Modeling Instruction. Physical Review Special Topics-Physics Education Research, 9(2). doi:10.1103/PhysRevSTPER.9.020116 Briggs, B. H. (1976). Student attitudes to physics. Physics Education, 11(7), 483. doi:10.1088/0031-9120/11/7/003 Brown, J. D. (n.d.). Principal components analysis and exploratory factor analysis—Definitions, differences, and choices Definitions, differences, and choices. Brown, T. A. (2006a). CFA model revision and comparison. In Confirmatory Factor Analysis for Applied Research (pp. 157–211). New York: Guilford Press. Brown, T. A. (2006b). Confirmatory factor analysis for applied research. New York: Guilford Press. Brown, T. A. (2006c). Introduction. In Confirmatory Factor Analysis for Applied Research (pp. 1–11). New York: Guilford Press. Brown, T. A. (2006d). Introduction to CFA. In Confirmatory Factor Analysis for Applied Research (pp. 40–102). New York: Guilford Press. Brown, T. A. (2006e). Specification and Interpretation of CFA Models. In Confirmatory Factor Analysis for Applied Research (pp. 103–156). New York: Guilford Press. Ceci, S. J., Williams, W. M., & Barnett, S. M. (2009). Women’s underrepresentation in science: Sociocultural and biological considerations. Psychological Bulletin, 135(2), 218–261. doi:10.1037/a0014412 Chen, X., & Weko, T. (2009). Students who study STEM in postsecondary education US Department of Education NCES 2009-161 July 2009. Aurora, 202, 502–7334. 105  CMEC (Council of Ministers of Education, Canada). (1997). Common framework of science  learning outcomes: Pan-Canadian protocol for collaboration on school curriculum. CMEC (Council of Ministers of Education, Canada). Retrieved from http://publications.cmec.ca/science/framework/Pages/english/table.html Confirmatory Factor Analysis. (2012). In A. C. Rencher & W. F. Christensen, Methods of Multivariate Analysis (pp. 479–500). Hoboken, NJ, USA: John Wiley & Sons, Inc. Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98–104. doi:10.1037/0021-9010.78.1.98 Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. doi:10.1007/BF02310555 Cunningham, G. K. (1986). Validity. In Educational and psychological measurement (pp. 120–133). New York : London: Macmillan ; Collier Macmillan. Dalgety, J., Coll, R. K., & Jones, A. (2003). Development of chemistry attitudes and experiences questionnaire (CAEQ). Journal of Research in Science Teaching, 40(7), 649–668. doi:10.1002/tea.10103 Dickinson, V. L., & Flick, L. B. (1998). Beating the System: Course Structure and Student Strategies in a Traditional Introductory Undergraduate Physics Course for Nonmajors. School Science and Mathematics, 98(5), 238–246. doi:10.1111/j.1949-8594.1998.tb17297.x Donnelly, R., MacPhee, C., Hardy, J., & Bates, S. P. (2013). Academics’ Attitudes and Beliefs about Physics from a UK Perspective. Presented at the American Association of Physics Teachers, Portland, Oregon. Driver, R. (1983). The pupil as scientist? Milton Keynes: Open University Press. 106  Eagly, A., H., & Chaiken, S. (1993). The psychology of attitudes. Fort Worth: Harcourt Brace Jovanovich College Publishers. Elby, A. (2001). Helping physics students learn how to learn. American Journal of Physics, 69(S1), S54. doi:10.1119/1.1377283 Farrar, D. H. (2013). 2013 Annual Report on Enrolment: Vancouver Campus (p. 27). Vancouver: University of British Columbia. Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention, and behavior: an introduction to theory and research. Reading, Mass: Addison-Wesley Pub. Co. Floyd, F. J., & Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7(3), 286. Germann, P. J. (1988). Development of the Attitude toward Science in School Assessment and its Use to Investigate the Relationship between Science Achievement and Attitude toward Science in School. Journal of Research in Science Teaching, 25(8), 689. Gire, E., Jones, B., & Price, E. (2009). Characterizing the epistemological development of physics majors. Physical Review Special Topics - Physics Education Research, 5(1), 010103. doi:10.1103/PhysRevSTPER.5.010103 Gormally, C., Brickman, P., & Lutz, M. (2012). Developing a Test of Scientific Literacy Skills (TOSLS): Measuring Undergraduates’ Evaluation of Scientific Information and Arguments. Cell Biology Education, 11(4), 364–377. doi:10.1187/cbe.12-03-0026 Goulden, M., Mason, M. A., & Frasch, K. (2011). Keeping Women in the Science Pipeline. The ANNALS of the American Academy of Political and Social Science, 638(1), 141–162. doi:10.1177/0002716211416925 107  Gray, K. E., Adams, W. K., Wieman, C. E., & Perkins, K. K. (2008). Students know what physicists believe, but they don’t agree: A study using the CLASS survey. Physical Review Special Topics - Physics Education Research, 4(2), 020106. doi:10.1103/PhysRevSTPER.4.020106 Hacker, D. J. (1998). Definitions and empirical foundations. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 1–23). Mahwah, N.J: Lawrence Erlbaum Associates Publishers. Halloun, I. (1997). Views About Science and physics achievement: The VASS story. In AIP Conference Proceedings (Vol. 399, pp. 605–614). American Institute of Physics. doi:10.1063/1.53156 Halloun, I., & Hestenes, D. (1998). Interpreting VASS Dimensions and Profiles for Physics Students. Science & Education, 7(6), 553–577. doi:10.1023/A:1008645410992 Hanson, S. L. (1996). Who Survives in Science? In Lost Talent (pp. 147–171). Philadelphia: Temple University Press. Hart, C. (2008). Models in Physics, Models for Physics Learning, and Why the Distinction may Matter in the Case of Electric Circuits. Research in Science Education, 38(5), 529–544. doi:10.1007/s11165-007-9060-y Heilbronner, N. N. (2011). Stepping onto the STEM Pathway: Factors Affecting Talented Students’ Declaration of STEM Majors in College. Journal for the Education of the Gifted, 34(6), 876–899. Heredia, K., & Lewis, J. E. (2012). A Psychometric Evaluation of the Colorado Learning Attitudes about Science Survey for Use in Chemistry. Journal of Chemical Education, 89(4), 436–441. doi:10.1021/ed100590t 108  Hestenes, D. (1987). Toward a modeling theory of physics instruction. American Journal of Physics, 55(5), 440–454. Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force concept inventory. The Physics Teacher, 30(3), 141. doi:10.1119/1.2343497 Hodson, D. (1998). Teaching And Learning Science: Towards a Personalized Approach. McGraw-Hill International. Hong, H.-Y., & Lin-Siegler, X. (2012). How learning about scientists’ struggles influences students’ interest and learning in physics. Journal of Educational Psychology, 104(2), 469–484. doi:10.1037/a0026224 Huang, C.-I., & Anderson, J. O. (2010, January 1). A multilevel analysis of scientific literacy. Hue, G., Sales, J., Comeau, D., & Lynn, D. G. (2010). The American Science Pipeline: Sustaining Innovation in a Time of Economic Crisis. CBE-LIFE SCIENCES EDUCATION, 9(4), 431–434. doi:10.1187/cbe.09-12-0091 IBM - SPSS Amos - Canada. (2013, November 14). CT253. Retrieved November 14, 2013, from http://www-03.ibm.com/software/products/en/spss-amos/ IBM SPSS software. (2013, October 29). CT500. Retrieved November 12, 2013, from http://www-01.ibm.com/software/analytics/spss/ Jowett, G. H. (1958). Factor Analysis. Journal of the Royal Statistical Society. Series C (Applied Statistics), 7(2), 114–125. doi:10.2307/2985313 Kane, M. (2006). Validation. In Educational Measurement (4th ed., pp. 17–64). Westport, Ct: Praeger Publishers. Kane, M. (2009). Validating the Interpretations and Uses of Test Scores. In Concepts of Validity (pp. 39–64). Charlotte, NC: Information. 109  Kelly, G. (1955). The psychology of personal constructs (1st ed.). New York: Norton. Kost, L., Pollock, S., & Finkelstein, N. (2009). Characterizing the gender gap in introductory physics. Physical Review Special Topics - Physics Education Research, 5(1). doi:10.1103/PhysRevSTPER.5.010101 Kost-Smith, L. E. (2011). Characterizing, modeling, and addressing gender disparities in introductory college physics (Ph.D.). University of Colorado at Boulder, United States -- Colorado. Retrieved from http://search.proquest.com.ezproxy.library.ubc.ca/docview/867841330/abstract?accountid=14656 Lederman, N. G., Abd-El-Khalick, F., Bell, R. L., & Schwartz, R. S. (2002). Views of nature of science questionnaire: Toward valid and meaningful assessment of learners’ conceptions of nature of science. Journal of Research in Science Teaching, 39(6), 497–521. doi:10.1002/tea.10034 Liehr, P. R., & Smith, M. J. (2008). Middle range theory for nursing. New York: Springer Pub. Retrieved from http://site.ebrary.com/id/10265237 Lindsey, B. A., Hsu, L., Sadaghiani, H., Taylor, J. W., & Cummings, K. (2012). Positive attitudinal shifts with the Physics by Inquiry curriculum across multiple implementations. Physical Review Special Topics - Physics Education Research, 8(1). doi:10.1103/PhysRevSTPER.8.010102 Mazur, E. (1997). Peer Instruction: A User’s Manual. Prentice Hall. McCaskey, T. L. (2009). Comparing and contrasting different methods for probing student epistemology and epistemological development in introductory physics. Retrieved from http://drum.lib.umd.edu/handle/1903/9824 110  Mercer, S. H., Nellis, L. M., Martínez, R. S., & Kirk, M. (2011). Supporting the students most in need: Academic self-efficacy and perceived teacher support in relation to within-year academic growth. Journal of School Psychology, 49(3), 323–338. doi:10.1016/j.jsp.2011.03.006 Messick, S. (1981). Evidence and Ethics in the Evaluation of Tests. Educational Researcher, 10(9), 9–20. doi:10.3102/0013189X010009009 Milner-Bolotin, M., Antimirova, T., Noack, A., & Petrov, A. (2011). Attitudes about science and conceptual physics learning in university introductory physics courses. Physical Review Special Topics - Physics Education Research, 7(2). doi:10.1103/PhysRevSTPER.7.020107 Moore, R. W., & Foy, R. L. H. (1997). The Scientific Attitude Inventory: A Revision (SAI II). Journal of Research in Science Teaching, 34(4), 327. Muijs, D. (2011). Developing scales and measures: item and factor analysis. In Doing Quantitative Research in Education with SPSS (2nd edition., pp. 198–224). London ; Thousand Oaks: Sage Publications. Ogilvie, C. A. (2009). Changes in students’ problem-solving strategies in a course that includes context-rich, multifaceted problems. Physical Review Special Topics - Physics Education Research, 5(2), 020102. doi:10.1103/PhysRevSTPER.5.020102 Osborne, J., Simon, S., & Collins, S. (2003). Attitudes towards science: A review of the literature and its implications. International Journal of Science Education, 25(9), 1049–1079. doi:10.1080/0950069032000032199 111  Otero, V. K., & Gray, K. E. (2008). Attitudinal gains across multiple universities using the Physics and Everyday Thinking curriculum. Physical Review Special Topics - Physics Education Research, 4(2), 020104. doi:10.1103/PhysRevSTPER.4.020104 Pajares, F. (1996). Self-Efficacy Beliefs in Academic Settings. Review of Educational Research, 66(4), 543–578. doi:10.2307/1170653 Pawl, A., Barrantes, A., Pritchard, D. E., & Mitchell, R. (2012). What do seniors remember from freshman physics? Physical Review Special Topics - Physics Education Research, 8(2), 020118. doi:10.1103/PhysRevSTPER.8.020118 Perkins, K. K., Adams, W. K., Pollock, S. J., Finkelstein, N. D., & Wieman, C. E. (2005). Correlating Student Beliefs With Student Learning Using The Colorado Learning Attitudes about Science Survey. AIP Conference Proceedings, 790(1), 61–64. doi:10.1063/1.2084701 Perkins, K. K., & Gratny, M. (2010). Who Becomes a Physics Major? A Long-term Longitudinal Study Examining the Roles of Pre-college Beliefs about Physics and Learning Physics, Interest, and Academic Achievement. In AIP Conference Proceedings (Vol. 1289, p. 253). Retrieved from http://link.aip.org/link/apcpcs/v1289/i1/p253/s1 Perkins, K. K., Gratny, M. M., Adams, W. K., Finkelstein, N. D., & Wieman, C. E. (2006). Towards characterizing the relationship between students’ interest in and their beliefs about physics. AIP Conference Proceedings, 818(1), 137–140. doi:10.1063/1.2177042 112  Pollock, S. J. (2005). No Single Cause: Learning Gains, Student Attitudes, and the Impacts of Multiple Effective Reforms. In AIP Conference Proceedings (Vol. 790, pp. 137–140). doi:10.1063/1.2084720 Redish, E. F. (2003a). Evaluating Our Instruction: Surveys. In Teaching Physics: with the Physics Suite (pp. 91–114). Hoboken, NJ: John Wiley & Sons. Redish, E. F. (2003b). Teaching physics: with the physics suite. Hoboken, NJ: John Wiley & Sons. Redish, E. F., & Hammer, D. (2009). Reinventing college physics for biologists: Explicating an epistemological curriculum. American Journal of Physics, 77(7), 629–642. doi:10.1119/1.3119150 Redish, E. F., Saul, J. M., & Steinberg, R. N. (1998). Student expectations in introductory physics. American Journal of Physics, 66(3), 212. doi:10.1119/1.18847 Revelle, W., & Zinbarg, R. E. (2009). Coefficients Alpha, Beta, Omega, and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145–154. doi:http://dx.doi.org.ezproxy.library.ubc.ca/10.1007/s11336-008-9102-z Salkind, N. J. (Ed.). (2007). Reverse Scaling. In Encyclopedia of Measurement and Statistics (Vol. 3, pp. 843–846). Thousand Oaks, CA: SAGE Reference. Retrieved from http://go.galegroup.com/ps/i.do?id=GALE%7CCX3470700399&v=2.1&u=ubcolumbia&it=r&p=GVRL&sw=w&asid=9a5f94a244436ae616ea1e6976fc4199 Sawtelle, V., Brewe, E., & Kramer, L. (2009). Validation study of the Colorado Learning Attitudes about Science Survey at a Hispanic-serving institution. Physical Review Special Topics - Physics Education Research, 5(2). doi:10.1103/PhysRevSTPER.5.023101 113  Sawtelle, V., Brewe, E., & Kramer, L. H. (2012). Exploring the relationship between self-efficacy and retention in introductory physics. Journal of Research in Science Teaching, 49(9), 1096–1121. doi:10.1002/tea.21050 Self-Efficacy. (2010). In Encyclopedia of Identity Encyclopedia of identity. 2455 Teller Road,  Thousand Oaks  California  91320  United States: SAGE Publications, Inc. Retrieved from http://knowledge.sagepub.com/view/identity/n235.xml Semsar, K., Knight, J. K., Birol, G., & Smith, M. K. (2011). The Colorado Learning Attitudes about Science Survey (CLASS) for Use in Biology. CBE-Life Sciences Education, 10(3), 268–278. doi:10.1187/cbe.10-10-0133 Seymour, E., & Hewitt, N. M. (1997). Talking about leaving : why undergraduates leave the sciences. Boulder (Colo.): Westview Press. Shavelson, R. J., Webb, N. M., & Rowley, G. L. (1989). Generalizability theory. American Psychologist, 44(6), 922–932. doi:10.1037/0003-066X.44.6.922 Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 4–14. Siegel, M. A., & Ranney, M. A. (2003). Developing the changes in attitude about the relevance of science (CARS) questionnaire and assessing two high school science classes. Journal of Research in Science Teaching, 40(8), 757–775. doi:10.1002/tea.10110 Slaughter, K. A., Bates, S. P., & Galloway, R. K. (2011). The Changes in Attitudes and Beliefs of First Year Physics Undergraduates: A Study Using the CLASS Survey. International Journal of Innovation in Science and Mathematics Education (formerly 114  CAL-Laborate International), 19(1). Retrieved from http://ojs-prod.library.usyd.edu.au/index.php/CAL/article/view/4780 Slaughter, K. A., Bates, S. P., & Galloway, R. K. (2012). A longitudinal study of the development of attitudes and beliefs towards physics. AIP Conference Proceedings, 1413(1), 359–362. doi:10.1063/1.3680069 Smith, E. (2011). Women into science and engineering? Gendered participation in higher education STEM subjects. British Educational Research Journal, 37(6), 993–1014. doi:10.1080/01411926.2010.515019 Smith III, J. P., diSessa, A. A., & Roschelle, J. (1994). Misconceptions reconceived: A constructivist analysis of knowledge in transition. The Journal of the Learning Sciences, 3(2), 115–163. Spotlight on Science Learning. (2013). Retrieved December 5, 2013, from http://letstalkscience.ca/our-research/Spotlight2013 Thomas, G., Anderson, D., & Nashon, S. (2008). Development of an Instrument Designed to Investigate Elements of Science Students’ Metacognition, Self‐Efficacy and Learning Processes: The SEMLI‐S. International Journal of Science Education, 30(13), 1701–1724. doi:10.1080/09500690701482493 Thompson, B. (2004). Exploratory Factor Analysis Decision Sequence. In Exploratory and confirmatory factor analysis: Understanding concepts and applications. (pp. 27–48). American Psychological Association. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=pzh&AN=2004-13115-003&site=ehost-live&scope=site 115  Thornton, R. K., & Sokoloff, D. R. (1998). Assessing student learning of Newton’s laws: The Force and Motion Conceptual Evaluation and the Evaluation of Active Learning Laboratory and Lecture Curricula. American Journal of Physics, 66(4), 338–352. doi:10.1119/1.18863 Van Etten, S., Freebern, G., & Pressley, M. (1997). College Students’ Beliefs about Exam Preparation. Contemporary Educational Psychology, 22(2), 192–212. doi:10.1006/ceps.1997.0933 Vygotskiĭ, L. S. (1987). The collected works of L.S. Vygotsky. New York: Plenum Press. Wang, M.-T., Eccles, J. S., & Kenny, S. (2013). Not Lack of Ability but More Choice: Individual and Gender Differences in Choice of Careers in Science, Technology, Engineering, and Mathematics. Psychological Science, 24(5), 770–775. doi:10.1177/0956797612458937 Why aren’t more women in science?: top researchers debate the evidence. (2007) (1st ed.). Washington, DC: American Psychological Association. Wink, J., & Putney, L. G. (2002). A vision of Vygotsky. Allyn and Bacon. Yerdelen-Damar, S., Elby, A., & Eryilmaz, A. (2012). Applying beliefs and resources frameworks to the psychometric analyses of an epistemology survey. Physical Review Special Topics - Physics Education Research, 8(1). doi:10.1103/PhysRevSTPER.8.010104 Zwickl, B. M., Finkelstein, N., & Lewandowski, H. J. (2012). Development and validation of the Colorado learning attitudes about science survey for experimental physics. In arXiv preprint arXiv:1207.2418. Retrieved from http://arxiv.org/abs/1207.2418   116  Appendices Appendix A: The CLASS 1. A significant problem in learning physics is being able to memorize all the information I need to know.  2. When I am solving a physics problem, I try to decide what would be a reasonable value for the answer.  3. I think about the physics I experience in everyday life.  4. It is useful for me to do lots and lots of problems when learning physics.  5. After I study a topic in physics and feel that I understand it, I have difficulty solving problems on the same topic.  6. Knowledge in physics consists of many disconnected topics.  *7. As physicists learn more, most physics ideas we use today are likely to be proven wrong.  8. When I solve a physics problem, I locate an equation that uses the variables given in the problem and plug in the values.  9. I find that reading the text in detail is a good way for me to learn physics.  10. There is usually only one correct approach to solving a physics problem.  11. I am not satisfied until I understand why something works the way it does.  12. I cannot learn physics if the teacher does not explain things well in class.  13. I do not expect physics equations to help my understanding of the ideas; they are just for doing calculations.  14. I study physics to learn knowledge that will be useful in my life outside of school.  15. If I get stuck on a physics problem on my first try, I usually try to figure out a different way that works.  16. Nearly everyone is capable of understanding physics if they work at it.  17. Understanding physics basically means being able to recall something you've read or been shown.  18. There could be two different correct values to a physics problem if I use two different approaches.  19. To understand physics I discuss it with friends and other students.  20. I do not spend more than five minutes stuck on a physics problem before giving up or seeking help from someone else.  21. If I don't remember a particular equation needed to solve a problem on an exam, there's nothing much I can do (legally!) to come up with it.  22. If I want to apply a method used for solving one physics problem to another problem, the problems must involve very similar situations.  23. In doing a physics problem, if my calculation gives a result very different from what I'd expect, I'd trust the calculation rather than going back through the problem.  24. In physics, it is important for me to make sense out of formulas before I can use them correctly.  25. I enjoy solving physics problems.   117  26. In physics, mathematical formulas express meaningful relationships among measurable quantities.  27. It is important for the government to approve new scientific ideas before they can be widely accepted.  28. Learning physics changes my ideas about how the world works.  29. To learn physics, I only need to memorize solutions to sample problems.  30. Reasoning skills used to understand physics can be helpful to me in my everyday life.  31. We use this question to discard the survey of people who are not reading the statements. Please select agree - option 4 (not strongly agree) to preserve your answers.  32. Spending a lot of time understanding where formulas come from is a waste of time.  33. I find carefully analyzing only a few problems in detail is a good way for me to learn physics.  34. I can usually figure out a way to solve physics problems.  35. The subject of physics has little relation to what I experience in the real world.  36. There are times I solve a physics problem more than one way to help my understanding.  37. To understand physics, I sometimes think about my personal experiences and relate them to the topic being analyzed.  38. It is possible to explain physics ideas without mathematical formulas.  39. When I solve a physics problem, I explicitly think about which physics ideas apply to the problem.  40. If I get stuck on a physics problem, there is no chance I'll figure it out on my own.  *41. It is possible for physicists to carefully perform the same experiment a     118  Appendix B:  Pattern and structure matrices when seven factors are extracted from all 41 items of the survey Table B-1: Pattern Matrix  Factor Item 1 2 3 4 5 6 7 v3 .618       v14 .586       v35 -.553       v37 .532       v30 .507       v28 .469       v11 -.324       v16        v26        v12  -.656      v40  .588      v5  .587      v1  .460      v20  -.440      v34  -.368   .293  .362 v25  -.310      v6  .309      v8  .278      v27   .440     v4   .437     v9   .363     v17   .331     v22   .302     v24    .693    v29    -.357    v32    -.317    v10        v2     .432   v21  -.294   .373   v19     .349   v38     .331   v23     -.325   v15        v36        v41      .619  v18      .519  v7      .302  v13        v33       .315 v42        v39        Extraction Method: Maximum Likelihood.   Rotation Method: Promax with Kaiser Normalization.a a. Rotation converged in 10 iterations. 119  Table B-2: Structure Matrix Item Factor 1 2 3 4 5 6 7 v3 .620 -.309   .352   v14 .554    .300   v35 -.543 .419      v30 .529   .306 .285   v28 .503    .288   v37 .491       v11 -.410   -.326    v42 .397   .328 .341   v26 .389 -.317  .325 .326   v16 .298       v40 -.288 .616   -.488   v5  .560 .292     v34 .302 -.549   .498  .400 v25 .446 -.490  .388 .446   v21  -.489  .361 .476   v12  -.450      v6  .393      v20  -.388      v1  .385      v13  -.322   .316   v27   .471     v4   .422     v17   .416    .291 v22  .316 .333     v9   .307     v8        v24    .574    v32 -.354 .393  -.456 -.294   v29    -.401    v10    -.294    v15 .284    .376   v2     .329   v36 .291    .326   v23     -.307   v19     .278   v38        v41      .568  v18      .509  v7      .282  v33       .299 v39        Extraction Method: Maximum Likelihood.   Rotation Method: Promax with Kaiser Normalization.  120  Appendix C: Pattern and structure matrices for seven extracted factors after removing item 33 from the data. Table C-1: Pattern Matrix Items Factor 1 2 3 4 5 6 7 v3 .605       v14 .559       v37 .518       v35 -.517 .312      v30 .492       v28 .444       v11 -.316       v16        v39        v12  -.527      v5  .503      v1  .498      v20  -.495      v40  .488      v6  .372      v32  .334     -.286 v8  .291      v25        v10        v27   .439     v4   .425     v9   .357     v17   .324     v22   .285     v36        v34    .700    v42    .281    v26        v41     .629   v18     .517   v7     .302   v13        v38      .391  v21  -.381    .381  v19      .344  v2      .339  v23      -.309  v15        v24       .635 v29       -.336 Extraction Method: Maximum Likelihood.   Rotation Method: Promax with Kaiser Normalization. a. Rotation converged in 14 iterations.  121  Table C-2: Structure Matrix Items Factor 1 2 3 4 5 6 7 v3 .631   .352    v14 .556       v30 .529      .299 v35 -.520 .437      v37 .502       v28 .500       v42 .411   .363   .388 v11 -.406      -.333 v26 .394   .387   .380 v16 .287       v39        v40  .566  -.465  -.396  v5  .539  -.318    v21  -.529    .523 .287 v6  .431      v20  -.429      v1  .422      v12  -.409  .284    v22  .343      v13  -.294      v10  .280      v8        v27   .458     v4   .409     v17  .333 .390     v9   .308     v34 .306 -.398  .698    v25 .428 -.434  .452  .333 .408 v41     .575   v18     .503   v7        v38      .346  v23      -.342  v15      .328  v36 .281     .306  v19      .305  v2      .288  v24       .584 v32 -.322 .416    -.294 -.427 v29  .287     -.364 Extraction Method: Maximum Likelihood.   Rotation Method: Promax with Kaiser Normalization.   122  Appendix D: Pattern and structure matrices for six extracted factors after removing item 33 from the data Table D-1: Pattern Matrix  Items Factor 1 2 3 4 5 6 v3 .605      v14 .569      v35 -.538 .285     v37 .529      v30 .502      v28 .443      v11 -.305      v38       v40  .586     v5  .546     v21  -.524     v20  -.470     v12  -.417     v1  .414     v6  .397     v22  .303     v32  .296   -.283  v10       v13       v34   .702    v42   .366    v26   .284    v25       v39       v2       v4    .462   v27    .399   v9    .310   v17  .285  .298   v16       v8       v19       v36       v15       v24     .570  v29     -.351  v23       v41      .620 v18      .513 v7      .297 Extraction Method: Maximum Likelihood.   Rotation Method: Promax with Kaiser Normalization. a. Rotation converged in 13 iterations. 123  Table D-2: Structure Matrix Items Factor 1 2 3 4 5 6 v3 .625  .396    v14 .550      v35 -.542 .439     v30 .532    .310  v37 .497      v28 .497      v11 -.409    -.346  v16 .284      v36 .280      v38       v40  .596 -.462    v5  .556     v21  -.539   .352  v6  .436     v20  -.413     v1  .395     v12  -.380     v22  .359     v13  -.302 .284    v8       v34 .303 -.442 .683    v25 .430 -.434 .480  .375  v26 .384  .409  .322  v42 .402  .409  .358  v15   .320    v39       v2       v4    .430   v27    .426   v17  .325  .353   v9    .287   v19       v24     .533  v32 -.337 .399   -.425  v29     -.381  v10  .284   -.289  v23     -.288  v41      .589 v18      .510 v7      .290 Extraction Method: Maximum Likelihood.   Rotation Method: Promax with Kaiser Normalization.  124  Appendix E: Pattern and structure matrices for five extracted factors after removing item 33 from the data Table E-1: Pattern Matrix  Items Factor 1 2 3 4 5 v3 .701     v14 .573     v37 .564     v30 .526     v28 .483     v35 -.447     v11 -.391     v42 .383     v26 .342     v39 .289     v16      v38      v2      v40  -.664    v5  -.603    v34  .551    v12  .503    v25  .311    v20      v13      v1      v15      v17   .416 -.321  v27   .393   v4   .354   v8   .291   v6      v9      v22      v19      v36      v24    .441  v29    -.414  v21  .373  .390  v32    -.336  v10      v23      v41     .556 v18     .522 v7     .299 Extraction Method: Maximum Likelihood.   Rotation Method: Promax with Kaiser Normalization.a a. Rotation converged in 8 iterations.  125  Table E-2: Structure Matrix  Items  Factor  1 2 3 4 5 v3 .655 .326    v14 .544     v30 .522   .297  v28 .501     v37 .496     v35 -.483 -.385  -.281  v42 .448   .289  v26 .440 .318  .302  v11 -.416   -.317  v36 .319   .280  v15 .308 .290    v16 .293     v39      v38      v2      v40 -.311 -.648  -.331  v34 .386 .570    v5  -.559 .310   v25 .484 .496  .418  v12  .413    v6  -.365    v20  .344  .316  v13  .338  .285  v1  -.329    v22  -.305    v27   .430   v17   .427 -.301  v4   .374   v8   .294   v19      v9      v21  .494  .501  v32 -.349 -.362  -.462  v24 .293   .412  v29    -.395  v10    -.327  v23    -.310  v41     .557 v18     .527 v7     .303 Extraction Method: Maximum Likelihood.   Rotation Method: Promax with Kaiser Normalization.  126  Appendix F: Pattern and structure matrices for five extracted factors. Items 1, 2, 6, 9, 10, 13, 15, 16, 19, 20, 22, 23, 33, 36, and 38 have been removed. Table F-1: Pattern Matrix Items Factor 1 2 3 4 5 v3 .656     v37 .630     v14 .582     v35 -.517     v30 .504     v28 .471     v11      v39      v40  -.653    v5  -.622    v34  .589    v12  .475    v21  .366    v25  .362  .334  v17   .573   v29   .416 -.323  v27   .340   v8      v4      v24    .597  v26    .327  v32    -.308  v42    .297  v41     .605 v18     .499 v7     .306 Extraction Method: Maximum Likelihood.   Rotation Method: Promax with Kaiser Normalization.a a. Rotation converged in 7 iterations.         127  Table F-2: Structure Matrix Items Factor 1 2 3 4 5 v3 .664 .373  .371  v14 .548   .307  v37 .522     v30 .519   .344  v35 -.512 -.405 .297   v28 .506   .350  v11 -.393   -.372  v39      v40 -.309 -.631    v34 .352 .604  .300  v5  -.555 .291   v21  .436    v12  .390    v17   .564   v29   .425 -.325  v27   .390   v8      v4      v25 .450 .529  .531  v24    .507  v26 .407 .354  .473  v42 .416 .289  .446  v32 -.336 -.358  -.426  v41     .594 v18     .502 v7     .297 Extraction Method: Maximum Likelihood.   Rotation Method: Promax with Kaiser Normalization.   

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0167588/manifest

Comment

Related Items