A S S E S S M E N T I N A T R I B A L C O L L E G E C O N T E X T : A C A S E S T U D Y O F N O R T H W E S T I N D I A N C O L L E G E by A N N E M A R I E K A R L B E R G B.Sc. (Honours), The University of Toronto, 1986 M . P . H . , Tulane University, 1989 B .Ed . , The University of Toronto, 1992 A THESIS S U B M I T T E D I N P A R T I A L F U L F I L M E N T O F T H E R E Q U I R E M E N T S F O R T H E D E G R E E O F D O C T O R OF P H I L O S O P H Y in T H E F A C U L T Y O F G R A D U A T E S T U D I E S (Educational Studies) T H E U N I V E R S I T Y OF B R I T I S H C O L U M B I A July 2007 © Anne Marie Karlberg, 2007 A B S T R A C T Approximately 32 tribal colleges are located on reservations in the United States. Their aim is to provide Native American students with a culturally relevant and meaningful post-secondary education. Assessment uses methods of applied research to improve student learning. The aim of this study is to advance theoretical and applied knowledge in the field of assessment within tribal colleges. This undertaking is noteworthy given that tribal colleges are vital to the development and future of Native American communities. I use a case study methodology to examine a specific assessment program that is being developed at Northwest Indian College (NWIC) , a tribal college in Washington state. In this study, I provide responses to three research questions: (1) What criteria are best used to evaluate an assessment program in a tribal college context? (2) Which elements of the N W I C assessment program are most and least successful according to the evaluative criteria established in Research Question 1? and (3) What preconditions and other contextual factors contribute to the relative success or failure of different elements of the N W I C assessment program? I review the history of Native Americans in higher education, provide an overview and critique of the emerging assessment movement, and discuss the state of assessment within the tribal college system. This study demonstrates that assessment can be done in a tribal college context in a culturally respectful and meaningful way and provides insights into how this can be approached. M y findings suggest that effective tribal college assessment programs use three types of information to assess student learning — direct indicators, indirect indicators, and institutional and community data — and assess each of these at the tribal community, college, program, and course levels. It is equally important to pay attention to the inputs into the assessment program (i.e., plans and resources) and how the assessment program is carried out. Furthermore, the intention of the assessment program is to impact the college's context. Articulating and revisiting the tribal college's mission — with its focus on the self-determination of Native peoples — is a critical initial step in the development of assessment programs that should be emphasized in this framework. T A B L E O F C O N T E N T S A B S T R A C T i i T A B L E O F C O N T E N T S i v L I S T O F T A B L E S v i i i L I S T O F F I G U R E S i x A C K N O W L E D G E M E N T S x C H A P T E R 1 I N T R O D U C T I O N 1 Research Questions 3 N W I C Assessment Program Case Study 3 Positioning Myse l f 6 Significance of Organizational Insider Research 9 Overview of the Doctoral Thesis 11 C H A P T E R 2 N A T I V E A M E R I C A N S I N H I G H E R E D U C A T I O N — H I S T O R Y A N D O V E R V I E W O F T R I B A L C O L L E G E S 1 3 The History of Native Americans in Higher Education in the United States 14 The Founding of Tribal Colleges in the United States (1960s) 31 Tribal Colleges Today 38 Conclusion 58 C H A P T E R 3 A S S E S S M E N T I N T H E U N I T E D S T A T E S — O V E R V I E W A N D C R I T I Q U E 6 0 What is Assessment? 61 Why do Assessment? 63 Criticisms and Concerns about Assessment 66 Benefits of Effective Assessment Processes 81 Mainstream Approaches to Assessment 82 Types of Information Used for Assessment Purposes 84 Assessment Issues Specific to Two-Year Colleges and Tribal Colleges 97 Emerging Innovative Approaches to Assessment 100 Planning and Reporting 105 Conclusion 109 C H A P T E R 4 A S S E S S M E N T I N T R I B A L C O L L E G E S 1 1 0 Guiding Principles 110 State of Assessment in Tribal Colleges 116 Conclusion 137 iv C H A P T E R 5 M E T H O D O L O G Y 1 3 9 Case Studies ; 139 The CIPP Model 147 Research Design 150 Respondents 151 Data Analysis 152 C H A P T E R 6 E V A L U A T I O N F R A M E W O R K A N D R E S U L T S 1 5 3 Context (Research Question 3) 153 Inputs (Research Questions 1 and 2) 156 Process (Research Questions 1 and 2) 171 Products (Research Questions 1 and 2) 180 Conclusion 210 C H A P T E R 7 C O N C L U S I O N S 2 1 1 Response to Research Question 1 (Criteria) 212 Response to Research Question 2 (Most and Least Successful Elements) 219 Response to Research Question 3 (Preconditions and Contextual Factors) 224 Implications of Research 228 Limitations of the Study 240 Suggestions for Future Research and Work 242 Concluding Remarks 243 R E F E R E N C E S 2 4 5 A P P E N D I X A S E C T I O N O F T H E N W I C A S S E S S M E N T P L A N ( D R A F T ) ( 2 0 0 7 - 2 0 1 2 ) 2 6 3 Introduction 263 Inputs 264 Processes 267 Products 271 A P P E N D I X B S U R V E Y O F T E A C H I N G A N D A S S E S S M E N T M E T H O D S ( 2 0 0 3 ) 2 8 8 A P P E N D I X C N W I C A S S E S S M E N T S U R V E Y 2 8 9 A P P E N D I X D N W I C I N S T I T U T I O N A L R E V I E W B O A R D L E T T E R O F A P P R O V A L 3 0 4 A P P E N D I X E U B C B E H A V I O U R A L R E S E A R C H E T H I C S B O A R D ' S C E R T I F I C A T E O F A P P R O V A L 3 0 5 A P P E N D I X F S U M M A R Y O F C L O S E D - E N D E D A S S E S S M E N T S U R V E Y R E S P O N S E S :. 3 0 6 A P P E N D I X G M I S S I O N S T A T E M E N T 3 1 2 Results, Challenges, and Strengths 312 Suggestions and Actions 313 v A P P E N D I X H A D M I N I S T R A T O R S ' R O L E S 3 1 6 Results, Challenges, and Strengths 316 Suggestions and Actions 317 A P P E N D I X I F A C U L T Y M E M B E R S ' R O L E S 3 1 8 Results, Challenges, and Strengths 318 Suggestions and Actions 318 A P P E N D I X J A S S E S S M E N T P R O C E S S 3 1 9 Results, Challenges, and Strengths 319 Suggestions and Actions 320 A P P E N D I X K E M B E D D I N G A S S E S S M E N T I N C O L L E G E P R O C E S S E S 3 2 1 Results, Challenges, and Strengths 321 Suggestions and Actions 322 A P P E N D I X L L E A R N I N G , T E A C H I N G , A N D A S S E S S M E N T A P P R O A C H E S 3 2 4 Results, Challenges, and Strengths 324 Suggestions and Actions 325 A P P E N D I X M C O L L E G E O U T C O M E S I N I T I A T I V E 3 2 8 Results, Challenges, and Strengths 328 Suggestions and Actions 329 A P P E N D I X N C U L T U R A L O U T C O M E S I N I T I A T I V E 3 3 0 Results, Challenges, and Strengths 330 Suggestions and Actions 331 A P P E N D I X O C O U R S E O U T C O M E S I N I T I A T I V E 3 3 2 Results, Challenges, and Strengths 332 Suggestions and Actions 333 A P P E N D I X P I N D I R E C T I N D I C A T O R S 3 3 4 Results, Challenges, and Strengths 334 Suggestions and Actions 335 A P P E N D I X Q I N S T I T U T I O N A L D A T A 3 3 6 Results, Challenges, and Strengths 336 Suggestions and Actions 337 A P P E N D I X R N W I C R U B R I C S 3 3 9 Written Communication Rubric (Draft) 339 Oral Communication Rubric (Draft) 341 Computer Skills Rubric (Draft) 343 Quantitative Skills Rubric (Draft) 344 Reading Skills Rubric (Draft) 345 v i A P P E N D I X S E X A M P L E O F A C O M P L E T E D N W I C C O U R S E O U T C O M E S F O R M 3 4 6 A P P E N D I X T N E E D S A S S E S S M E N T ( 2 0 0 3 ) 3 4 8 A P P E N D I X U S T U D E N T O P I N I O N S U R V E Y — E X T R A Q U E S T I O N S ( 2 0 0 4 ) 3 5 1 A P P E N D I X V A L U M N I S U R V E Y — E X T R A Q U E S T I O N S ( 2 0 0 3 ) 3 5 4 A P P E N D F X W A N E X A M P L E O F A N N W I C C O U R S E E V A L U A T I O N F O R M 3 5 8 A P P E N D I X X N W I C R E T E N T I O N S T R A T E G I E S 3 6 1 A P P E N D I X Y C H E C K L I S T F O R T R I B A L C O L L E G E A S S E S S M E N T P R O G R A M S . 3 6 3 Inputs 3 6 3 Processes 3 6 5 Products 3 6 7 A P P E N D I X Z C O V E R L E T T E R F O R T H E N W I C A S S E S S M E N T S U R V E Y 3 7 0 vii L I S T O F T A B L E S Table 3.1 Examples of Direct Methods for Assessing Student Learning 86 Table 3.2 A Section of the N W I C Rubric for Evaluating Writing Skills (Draft) 89 Table 3.3 Mean Aggregate Scores from a Native American History Class Assignment Assessing the N W I C Outcome "Writ ing Standard English" 90 Table 3.4 Relationship between Individual Student Grading and Assessment 91 Table 3.5 Example of a Section of a Curriculum Map 93 Table 3.6 Examples of Indirect Methods for Assessing Student Learning 95 Table 3.7 Examples of Institutional Data 96 Table 3.8 Example of a Section of an Environmental Studies Program Assessment Plan 108 Table 5.1 List of Research Questions, Data Collection Methods, and CIPP Areas 149 Table 5.2 Demographic Breakdown of Staff who Responded to the Survey 151 Table 6.1 Draft List o f Faculty Cultural Outcomes 189 Table 7.1 Examples of Activities at Each Level of a Tribal College Assessment Program 239 v i i i L I S T O F F I G U R E S Figure 2.1 Map of Tribal Colleges and Universities in the United States 41 Figure 6.1 Time-line of Assessment Activities at N W I C (Products of the Assessment Program) 182 Figure 7.1 Conceptual Model of the Components of a Tribal College Assessment Program... 218 Figure 7.2 Simplified Conceptual Mode l of the Components of a Tribal College Assessment Program 231 Figure 7.3 Examples of Products of a Tribal College Assessment Program 236 ix A C K N O W L E D G E M E N T S This thesis is the culmination of years of work, learning, and growth, and it would never have been possible without the support of many people. I wish to acknowledge with sincere thanks my committee members, Lesley Andres, Jo-ann Archibald, and Tom Sork, who were nothing but encouraging from the start. The dedicated and outstanding staff at Northwest Indian College are a daily inspiration to me. I am especially appreciative for the ongoing interest and support of President Cheryl Crazy B u l l , Rissa Wabaunsee, Ted Will iams, Barbara Roberts, and Sharon Kinley, all o f whom found time in their busy schedules to review the thesis and provide valuable insights. Also , thank you to my colleagues at Northwest Indian College - Brian Compton, B i l l Freeman, and Elva Eisel - and to Greg Gagnon from the University of North Dakota for reviewing the survey and thesis and providing helpful feedback. Thank you to all the faculty, administrators, and staff for their ongoing support of, and perseverance with, the assessment, teaching, and learning work. I appreciate the friendships from my Educational Studies cohort and especially the constant moral support from Michelle Pidgeon. For the editing of this thesis, I am most grateful to Kathy Roesch, whose input has enriched significantly its quality. To my mother, Marcia Kuhlberg, the most honourable person I know, I am proud to be your daughter. Michael , thank you for believing in this project. Your insight and capacity continually amaze me. To our beautiful and talented daughters, Jessa and Leah, thank you for your patience. I love you more than you can imagine. A n d finally, to all tribal college students: This work is ultimately for you. C H A P T E R 1 I N T R O D U C T I O N This study examines assessment in a Native American tribal college context. Tribal colleges are located on reservations in the United States. Their aim is to provide primarily Native American students with a culturally relevant and meaningful post-secondary education. Assessment uses methods of applied research to improve a college's mission-related performance — especially as it relates to student learning; however, Native American theory and research protocols suggest that, to date, the type of data and methods of data collection that have characterized most mainstream assessment are not well-suited to a Native American context (Battiste & Henderson, 2000; Boyer, 2003; Carnegie Foundation for the Advancement of Teaching, 1997; Haig-Brown & Archibald, 1996; Hampton, 1995b; Kirkness & Barnhardt, 1991; Menzies, 2001; National Sciences and Engineering Research Council of Canada, 1999; Royal Commission, 1996; L . Smith, 1999). Fortunately, approaches to assessment are changing and more progressive assessment programs are now becoming valuable tools in higher education throughout the United States — even though they are not yet well developed in Native American contexts. In fact, a study of American tribal colleges by the Carnegie Foundation for the Advancement of Teaching (Carnegie, 1997) determined that, due to the lack of. financial resources, expertise, and technology, even the most basic assessment data are generally not available in tribal colleges. Nonetheless, Indigenous educators are beginning to recognize assessment as an emerging priority — for internal improvement as well as for accreditation purposes (Blanchard et al., 2000; Ortiz & HeavyRunner, 2003; L . Smith, 1999; Swisher & Tippeconnic, 1999). The World Indigenous Nations Higher Education Consortium (2003), which "provides an international forum and support for Indigenous Peoples to pursue common goals through higher education" (p. 1 1), has the goal of creating "an accreditation body for Indigenous education initiatives and systems that identify common criteria, practices, and principles by which Indigenous Peoples l ive" (p. 1). The general concern among Native American administrators is that i f tribal colleges do not articulate for themselves a meaningful approach to assessment, then external accreditation bodies w i l l do it for them. Elaborating on this theme, Swisher (Hunkpapa Lakota), president of Haskell Indian Nations University in Lawrence, Kansas, and Tippeconnic (Comanche), professor of education at The Pennsylvania State University and Director of the American Indian Leadership Program, explain that assessment research in tribal colleges should revolve around teaching, curricula, pedagogy, and higher education issues such as "enrollment, retention, and graduation" (1999, p. 298). They state that Indigenous institutions should develop "alternative assessment or unbiased standardized tests to assess student achievement and abilities" (p. 298) as they can play a valuable role in decision making to improve practice. Demmert (2005), an Alaska Tlingit Native and professor at Western Washington University's College of Education, also comments on the need to focus on developing "measures for assessing e.g., cognitive development, language development, academic achievement, for identifying those who need interventions, and for the monitoring of student progress" (p. 21). He states that "new measures that are culturally and, where applicable, linguistically appropriate (i.e., in Native languages)" and that "could be researcher-developed or could be produced by test publishers.. .are specifically needed that would address culture-based education" (p. 21). Finally, he says that educators in Indigenous educational institutions can "examine existing approaches to program evaluation and incorporate evaluation in the planning and implementation of any new (or existing but unevaluated) programs. The fruits of any such program evaluations should be 2 shared broadly, via a central clearinghouse, publications, or via any networks or list-serves that exist" (p. 22). In this context, the primary aim of this study is to advance theoretical and applied knowledge in the field of assessment, within tribal colleges. In 2002, administrators and faculty at Northwest Indian College (NWIC) , a tribal college on the Lummi Reservation in northwestern Washington state, asked me to be their assessment coordinator and to assist them in devising and implementing an assessment program that is appropriate and meaningful to their tribal college mission. For my doctoral research, I present the context of N W I C s assessment program and then document and critically evaluate the inputs, processes, and products of this innovative assessment effort, so that other tribal colleges can learn from our experience. R e s e a r c h Q u e s t i o n s Through this research, I seek to answer the following questions: (1) What criteria are best used to evaluate an assessment program in a tribal college context? (2) Which elements of the N W I C assessment program are most and least successful according to the evaluative criteria established in Research Question 1 ? (3) What preconditions and other contextual factors contribute to the relative success or failure of different elements of the N W I C assessment program? N W I C A s s e s s m e n t P r o g r a m C a s e S t u d y I carried out this research at N W I C on the Lummi Reservation in Washington state. N W I C is 1 of approximately 32 tribal colleges throughout the United States that prepare Native 3 American students for positions of leadership and self-determination within their communities, through culturally appropriate pedagogy and curricula. The Lummi Reservation The Lummi Reservation occupies approximately 12,500 acres of land on a peninsula that extends into the Pacific Ocean in the northwestern corner of Washington state (Lummi Indian Business Council , 1993). It is located just a 2-hours' drive south of Vancouver, British Columbia, and a 2-hours' drive north of Seattle. There are approximately 3,800 enrolled Lummis, with about 2,100 Lummis currently l iving on the reservation (U.S. Census Bureau, 2000). The Lummi are part of the Coast Salish cultural and linguistic group, and their main sources of employment are fishing, gathering shellfish, the gaming industry, agriculture, and forestry. The History of Higher Education on the Lummi Reservation Higher education on the Lummi Reservation is rooted in the community's desire to provide its older youth and adults with an educational program that reflects and respects Lummi 's cultural heritage, knowledge, customs, and needs. A s a result, in 1973, the Lummi Reservation took the first step in this direction by establishing the Lummi Indian School of Aquaculture. This school was designed to train Lummis and other Native Americans in the technical aspects of operating fish and shellfish hatcheries. Trained technicians went on to work in Native American owned and operated hatcheries throughout the United States and Canada. B y the early 1980s, however, demand for these technicians declined drastically. A t about the same time, the high rate of general unemployment and the low rate of high school completion on the reservation created an awareness of the need for a post-secondary tribal 4 college in northwest Washington; therefore, in 1983, the Lummi Indian Business Council closed the Lummi Indian School of Aquaculture and established a public, non-profit community college. The purpose of the college was to provide adult basic education (i.e., basic reading, writing, math, and life skills) and post-secondary education to northwest Washington Native American communities (Hayes, 1990; N W I C , 2002). Thus the Lummi Nation established Lummi Community College in 1983 as a Native American administered community college. Initially, Lummi Community College operated under a contract with Whatcom Community College, which is an accredited college nearby. Within the scope of this contract, courses offered at Lummi Community College had to meet standards set by Whatcom Community College for maintaining its own accreditation (NWIC, 1991). In 1988, the Lummi Community College charter was expanded to include higher educational opportunities for tribes in Washington, Oregon, and Idaho. Because members of various Northwest tribes attended the college, a more representative name for the college was desired. In 1989, it began operating independently of Whatcom Community College and, in order to more accurately reflect its new mission and the wider communities it now served, its ~ name was officially changed to Northwest Indian College (NWIC, 2002). N W I C Today N W I C is a tribally administered institution chartered by the Lummi Indian Business Council . Although the college predominantly relies on grants for its funding, it is also federally funded through the Bureau of Indian Affairs. In 1993, the Northwest Association of Schools and Colleges accredited N W I C (NWIC, 2002). N W I C is the only accredited tribal college in Washington, Oregon, and Idaho. Although the college primarily serves four northwestern states (Washington, Oregon, Idaho, and Alaska), some of its 650 students come from all corners of the 5 United States, as well as from Canada. N W I C has six extended campuses — on the Colvi l le , Muckleshoot, Nez Perce (Idaho), Port Gamble S 'Klal lam, Swinomish, and Tulalip reservations — which offer face-to-face classes and classes transmitted on interactive television (ITV) from the main Lummi campus. Interactive television involves students attending classes at extended campuses while having a live connection to the main campus class. In addition, a significant portion of the college enrollment is generated through extension services, such as independent learning. N W I C is located on the old site of the Lummi Day School, built in 1910. The original Lummi Day School building now houses the college's library. The college campus consists of mainly portable structures. Although the current president, the vice presidents, and most of the staff are Native American, most of the faculty members are not. Crazy B u l l (2006a), N W I C ' s president, summarizes her expectations for the college: We are a place where the vision of our ancestors comes together with the present — our students today — to build a future for coming generations. We are the place where our understanding of traditions and cultural practice are translated into contemporary education. We are the place where Native knowledge is honored, new leadership blossoms, and where students find their voice, (p. 1) Thus, N W I C strives to provide a holistic approach to education in a culturally relevant and supportive environment that provides students with as many opportunities for success as possible. Its mission statement emphasizes the college's hope: "Through education, Northwest Indian College promotes Indigenous self-determination and knowledge." Positioning M y s e l f In February 2002, N W I C hired me to coordinate its assessment program. Since that time, I have been facilitating and coordinating a participatory assessment process related to teaching and learning at the college. In this regard, I am a cultural outsider doing organizational insider 6 research. A s a White woman, from a relatively privileged background, my understanding of the world has been shaped by my own culture, education, and life experiences. A s a result, I am challenged to continually reflect on how my own worldviews impact my research and assessment work within a tribal college context (Haig-Brown, 1992). If I am going to be involved in creating knowledge that relates to Indigenous people, I must situate myself as a learner within the tradition of Indigenous knowledge (Mclsaac, 2000). From this position, I can learn much both personally and professionally (Haig-Brown, 1992) while, at the same time, I also hope that through a participatory research process, I can assist in generating new knowledge that directly serves N W I C and from which other tribal colleges can benefit. Many Indigenous scholars have articulated their thoughts regarding non-Natives working cooperatively within Indigenous communities and educational institutions. Linda Tuhiwai Smith (2000), professor of Maori Education and Director o f the International Research Institute for Maor i and Indigenous Studies at the University of Auckland, New Zealand, states that non-Natives need not be excluded from contributing to Native American educational research. Mihesuah (2004), editor of the American Indian Quarterly, echoes this thought and believes that "a balanced, inclusive methodology is ideal" (p. xi). Graham Smith (1992), Visi t ing Distinguished Professor in the Department of Educational Studies graduate program at the University of British Columbia and a Maori education scholar, articulates situations in which it may be appropriate for non-Indigenous researchers to conduct research in Indigenous communities. One of these is i f the researcher empowers the community to address issues and concerns in a manner that leads to beneficial outcomes. Furthermore, Marker (2000), Director of T s " k e l First Nations Graduate Studies in Education at the University of British Columbia, states clearly that "the quality of research in First Nations education is not improved simply by having 7 Aboriginal people doing the writing. It is improved by a more detailed analysis that includes the perspectives and location of both Natives and non-Natives" (p. 31). This is the perspective that the N W I C administration adopted when they hired me as their assessment coordinator in 2002. In their 1998 accreditation review, administrators at N W I C received several recommendations regarding assessment that were essential to address before their next review. The following recommendations related to assessment: (1) The team recommends that the college move deliberately to a process of broad-based continuous planning and evaluation (Standard I .B). (2) The team recommends that the college adopt and consistently implement an educational assessment plan, which provides for a series of outcomes measures that are internally consistent and in accord with its mission (Standard 2; Policy 2.2). (3) The team recommends that the college adopt a consistent systematic process of evaluation of faculty, staff, and administration (Standards 3.A.2, 6.C.3; policy 4.1). (Northwest Commission of Colleges and Universities, 1998) The administrators decided the assessment work was worthwhile and in their best interest, but that it should be done in a culturally meaningful way. A s a result, the administrators' primary interest was to find someone who could respect their cultural protocols, beliefs, and values and integrate them with conventional research procedures and best practices (Marker, 2004; L . Smith, 1999; Steinhauer, 2002). Unable to find or attract a Native person with the required skills and expertise in this regard, they accepted me as a partner in their assessment work. Just as some non-Natives have contributed positively to Native American struggles regarding control over education (Haig-Brown, 1992), I have now become an ally linking N W I C to relevant resources and expertise in the non-Native world. Through my work at N W I C , I also have opportunities to educate non-Native communities of interest, such as accreditation bodies and funding agencies, and to translate Native values and interests into language that is familiar to 8 these external agencies. Nonetheless, I can conduct this research only because I have been invited into and accepted within this community and have been asked to perform this work. Moreover, the literature on research within Native American communities emphasizes the importance of the commitment of researchers to a participatory process, based on a respectful and reciprocal relationship between the researcher and the community (Haig-Brown & Archibald, 1996; Hampton, 1995b; Kirkness & Barnhardt, 1991; Menzies, 2001; National Sciences and Engineering Research Council o f Canada, 1999; Royal Commission on Aboriginal Peoples, 1996; L . Smith, 1999). In addition, as alluded to above, the research itself needs to be relevant and meaningful to the community and conducted in a responsible and ethical manner (Darou, Kurtness, & Hum, 2000; Greenwood & Levin , 1998; Royal Commission, 1996; L . Smith, 1999; Stringer, 1999). Ideally, these principles should guide research performed in any community, but past experience within Native American communities has left them acutely sensitive to abuses in this regard. In my role as a bridge between N W I C and the field of assessment, it is especially important that I introduce and facilitate approaches to assessment that are consistent with these research principles. N W I C administrators have been extremely encouraging and supportive of this doctoral research. In the following section, I briefly discuss the relevance of insider research and clarify my own position in this regard. S i g n i f i c a n c e o f O r g a n i z a t i o n a l I n s i d e r R e s e a r c h Various approaches to organizational insider research — such as participative inquiry, cooperative inquiry, participatory action research, and action inquiry — have been developed in the past decade as this practice has been increasingly legitimized (Bensimon, Polkinghorne, Bauman, & Vallejo, 2004). A l l o f these approaches involve the community identifying a problem or concern and the researcher collaborating with the community in order to address the 9 problem or concern through a participatory research process. A s a form of action research, assessment connects insider research to pedagogical practice. B y deriving assessment research from community-identified needs, by engaging the community as participants in the research process, and by reporting the results back to the community in meaningful ways, Bensimon et al. (2004) contend practical knowledge is produced. In this regard, they refer to a "practitioner-as-researcher" (p. 109) model, a method of research for bringing about college-wide change, in which "stakeholders produce the knowledge" (p. 109) (instead of external researchers) to provide information that illuminates local problems in a specific context in order to work toward a solution; thus the insiders conduct the research and produce the information. The benefits of organizational insider research are now well documented (L. Smith, 1999). For instance, by interviewing key people and being a participant-observer, insiders can effectively draw on their experiences and observations within the community as well as use "reflective thinking" to enhance their research (Ffaig-Brown, 1992; Hampton, 1995b; Pelto & Pelto, 1978). Within a college context, this information can be used to improve institutional performance and generate practical information that is effective for initiating change (Bensimon et al., 2004); however, as Linda Smith (1999) emphasizes, "the critical issue with insider research is the constant need for reflexivity" (p. 137). Reflexivity refers to the ability to reflect fairly upon one's actions and responsibilities and to respond continually to changing circumstances and needs. In a tribal college context, involving faculty, staff, students, and community members in this reflective process can improve learning and teaching and ensure that assessment processes are serving the college effectively (Stufflebeam, 2003a). To do this, however, inside researchers need, among other things, to train participants in assessment techniques, help them acquire or develop assessment resources, seek feedback on draft 10 assessment reports, facilitate the use of relevant information in databases, and assist in creating realistic assessment plans (Stufflebeam, 2003a). O v e r v i e w o f t h e D o c t o r a l T h e s i s The doctoral thesis is divided into seven chapters. In chapter 1,1 have established the research questions and positioned myself as a non-Native person working at a tribal college and participating in organizational insider research that affects this community. In addition, this chapter introduced N W I C as a case study. It provided an overview of the Lummi Reservation; reviewed the history of higher education on the Lummi Reservation; and examined where N W I C is today. Chapter 2 provides a summary of Native Americans in higher education, including a history and overview of tribal colleges in the United States. Chapter 3 provides an overview and critique of contemporary approaches to assessment. Chapter 4 is divided into two sections. The first section summarizes guiding principles of culturally appropriate research in Native American communities that are most relevant to the future of assessment at tribal colleges. The second section reviews emerging assessment practices and discussions in tribal colleges, along with elements from progressive mainstream assessment thinking that can inform assessment in a tribal college setting. Chapter 5 overviews relevant approaches to methodology — including the case study method and the Context-Input-Process-Product (CIPP) Model — and data analysis. Chapter 6 articulates an evaluation framework for the context, inputs, processes, and products of N W I C s assessment program by outlining in each section a set of elements or criteria that may be taken into consideration when establishing an assessment program in a tribal college and summarizing the results of the research. In chapter 7,1 overview the responses to the three research questions and present a list of elements that might assist tribal colleges to develop their assessment 11 programs. The thesis concludes with a discussion of the theoretical and applied implications of this research, the limitations of this study, and suggestions for future research and work. One last note before I conclude this chapter: Because there is no one term by which Native Americans prefer to refer to themselves, in this doctoral thesis, the use of terms such as tribal, Indian, Indigenous, Aboriginal, First Nations, and Native American reflect the vocabulary most appropriate to each context. Certain terms are used because they were used at the time by Native Americans, because they are preferred by one of the primary audiences for this doctoral thesis (the Native American members of the tribal college community), or because they are a part of common names, such as tribal college, Bureau of Indian Affairs, etc. 12 C H A P T E R 2 N A T I V E A M E R I C A N S I N H I G H E R E D U C A T I O N — H I S T O R Y A N D O V E R V I E W O F T R I B A L C O L L E G E S The tribal college system in the United States is poorly understood and has not been recognized for what it has accomplished. The primary purpose of this chapter is to outline the history, early development, and contemporary state of tribal colleges. Tribal colleges are a relatively new phenomenon in the torturous and painful history of education for Native Americans in the United States. Neither the significance of this achievement nor the conceptual bases underlying the creation and functioning of these colleges can be fully appreciated divorced from an understanding of the historical experience from which they evolved. This chapter, therefore, reviews both that historical experience and the contemporary picture of tribal colleges. The first section of this chapter provides an overview of the history of Native Americans in U .S . higher education through three distinct periods: (1) the colonial period, when the objective was converting Native Americans to Christianity through Indian missions in early colleges; (2) the federal period, during which the federal government largely ignored Native American higher education, yet enacted a series of policies that were devastating to the Indian community; and (3) the self-determination period, characterized by significant shifts in federal policy toward self-determination, as wel l as the founding of tribally controlled colleges (Carney, 1999). Each of these periods significantly influenced the evolution and direction of higher education for Native Americans in the United States. The second section of this chapter examines the evolution and founding of tribal colleges in the United States, beginning with Navajo Community College, the first tribal college in the U.S . It provides an overview of the most significant policies and events that led to the establishment of tribal colleges, and the current state of these colleges, including an examination of the unique features of tribal colleges that increase Native American student access, retention, 13 and success. Finally, it summarizes other fundamental characteristics tribal colleges have in common, including shared challenges. T h e H i s t o r y o f N a t i v e A m e r i c a n s i n H i g h e r E d u c a t i o n i n t h e U n i t e d S t a t e s The Colonial Period (1500s - 1775) Traditionally, the education of Indian youth consisted of the family, extended family, and elders preparing them for the challenges they would face throughout life. Through this unstructured process, life skills, history, and religious and cultural norms and expectations were passed on (Oppelt, 1990). In contrast, the objective of early European North Americans, from the arrival of Columbus, was the "c iv i l iz ing" of Native Americans through conversion to Christianity and the elimination of their traditional ways (Oppelt, 1990). Conversion, they believed, would necessitate making Native Americans literate. Thus, during the early 1600s, higher education for Indians focused on literacy and vocational training combined with the primary goal of converting them to Christianity (Carney, 1999). Throughout both the colonial and, later, the federal period, the practice of removing Indians from their homes to be educated was a predominating feature of White education for Indians. This practice likely originated in 1568, when the Jesuits established a school in Havana, Cuba, to serve Indians from Florida (Carney, 1999). This school was the first attempt of many in the last 450 years to remove Indians from their homes, a practice that severed them from the cultural influences of their families and tribes, with the underlying intention of assimilating them into mainstream society. The first effort to establish a college for Indians, which occurred in the Jamestown settlement of Virginia in 1616 to 1622 (Carney, 1999), was unsuccessful; but had it been 14 successful, it would have been the first college of any kind in the United States. In 1616, Pocahontas, the first convert in Jamestown, traveled to England to raise funds for educating Indians. K i n g James was so impressed with her that he ordered the church to send money to fund a university for Native Americans in Jamestown, which he wanted to call Henrico College in honor of his son. Unfortunately, the large sums of money contributed for the college were diverted to the Virginia Company. In 1622, after an Indian uprising against the Virginia settlement that kil led many colonists (Carney, 1999), the idea of founding a college for Indians in Jamestown ended; however, the practice of sending Indians to England to raise funds for Indian education, deceptive as it continued to be, lasted throughout the colonial period. Surprisingly, of the first nine colleges established in the United States, three (Harvard, Wi l l i am and Mary, and Dartmouth) included Native Americans in their vision. In 1636, the New College (now Harvard University) in New England was the first college founded in the United States. In its 1650 charter, Harvard professed as one of its goals "the education of the English and Indian youth of this country in knowledge; and godliness" (quoted in Carney, 1999, p. 1). Although the first Indian student attended Harvard in 1653, it was not until 1665 that the first one graduated. Few Indians ever attended Harvard until the 20th century, largely because of its irrelevant curriculum and the rampant sickness and disease ravaging Indian communities (Carnegie, 1997). In 1970, Harvard experienced an increase in Native American enrollment and renewed its commitment to Native American education by establishing the American Indian Program at the Graduate School of Education. Between 1970 and 1995, Native American students in the Graduate School of Education at Harvard earned 156 master's degrees and 19 doctorates (Encyclopedia of North American Indians, 2005c); however, in 2003-2004, only 0.7% of the student body at Harvard was Native American (Mass Mentor, 2004), thus demonstrating 15 that Harvard still struggles to fulfill its original mission to educate Indians, among others (Karabel, 2005). In 1693, the College of Wi l l i am and Mary in Virginia became the second college chartered by the crown in the United States. Its mission statement expressed the desire "that the youth may be piously educated in good Letters and Manners, and that the Christian faith may be propagated amongst the Western Indians, to the Glory of Almighty G o d " (quoted in Carney, 1999, p. 1). A handful of Indian students attended the college between 1705 and 1721 and then, again, from 1743 to 1776. Most of these students received support from the Boyle endowment, the last student receiving funds in 1776. In 1783, the Indian-serving section of the school closed, and the endowment was reallocated to provide higher education opportunities for Black students (Oppelt, 1990). In 2003-2004, less than 0.5% of the College of Wi l l i am and Mary ' s total enrollment were Native American and 5.5% were Black (Electronic Campus, 2004b). The next six colleges established in the United States, founded between 1701 and 1766, had no Indian mission. These six colleges are now Yale University in Connecticut, Princeton University of New Jersey, Columbia University in New York, the University o f Pennsylvania, Brown University in Rhode Island, and Rutgers The State University of New Jersey (Wikipedia, 2005). Although none of these colleges encouraged Native Americans to enroll, the College of New Jersey did admit three Indian students, none of whom graduated during that time (Carney, 1999). Finally, in 1769, the ninth college, Dartmouth College in New Hampshire, also professed a mission for educating Indian students and included in its charter "the education and instruction of youths of the Indian tribes in this Land in reading, wrighting, and all parts of Learning which shall appear necessary and expedient for civi l izing and Christianizing Children of Pagans as well 16 as in all liberal Arts and Sciences; and also of English Youth and any others" (quoted in Carney, 1999, p. 1). A s Pocahontas had done earlier, another Native American, Samson Occum, traveled to England in search of funds for Indian higher education. Born in 1723, Samson Occum, a Mohegan, was one of the finest scholars of his time and eventually became a teacher and a Presbyterian minister. He was revered as the perfect example of a "c ivi l ized" and educated Christian Indian (Oppelt, 1990). His teacher, Eleazar Wheelock, was an evangelical minister. Thanks to Wheelock's success in educating Occum, Wheelock became excited about the idea of educating Indians. Wheelock convinced Occum to travel to England and Scotland in 1766 and 1768 to raise money for Indian higher education. The fund-raising trip was an immediate success, and, as a result, Dartmouth College was established in 1769. Although he was successful in raising funds that led to the establishment of Dartmouth, Occum later felt betrayed when few Indian students ever attended the college and the funds he raised were not used for their intended purpose. Although Dartmouth demonstrated minimal success in educating Native Americans, its recent history has been more promising. With a new president in 1969, Dartmouth was "refounded" (Encyclopedia of North American Indians, 2005b). This president reaffirmed Dartmouth's commitment to Native American higher education and made a pledge to maintain 3% Native American undergraduate enrollment. He has also devoted significant financial assistance and resources to their education. The undergraduate Native American graduation rate has increased from 50% in 1970 to 72% in 2004 (Dartmouth College, 2005) and, in 2003-2004, 3% of Dartmouth's total undergraduate enrollment (i.e., from 4,098 students total) was Native American (Xap, 2004). In the first 125 years of its existence, only 58 Indian students attended Dartmouth College, and 11 graduated (Carney, 1999). 17 During the period when these first U .S . colleges were founded, the purpose of recruiting Indian students was essentially either to access funds devoted to an Indian mission or to convert Indian students to Christianity. The first three colleges to profess a mission for Indians all experienced similar financial challenges: fraudulent behaviour, misappropriation of funds from Indian education to the general education of the colony, and a small Indian population base from which to increase Indian student enrollment. The only source of large donations for the colleges was England, where fundraisers were sent on the false pretext of raising money to educate Indians (Carney, 1999). Whatever the intentions of these colleges, 50 Native Americans actually attended these early institutions before 1776, and one graduated (Carney, 1999). A number of factors may have contributed to these failed, half-hearted experiments in higher education for Indians. For example, many Indians resisted a White education, believing it would be irrelevant to their vocation and culture and fearing it would lead to the acculturation of their people (Oppelt, 1990). The racism and lack of support that Indian students experienced from the local colonists also hurt the missionaries' cause (Oppelt, 1990; Szasz, 1988). Furthermore, few people in society could attend college at that time, as a college education was reserved for only the elite male population. The colleges' presidents eventually realized that unless Native Americans found a practical and meaningful use for their education, higher education for Indians would remain a futile exercise (Carney, 1999). The Federal Period (1775 - 1928) The so-called federal period in the history of Native American education cannot be easily characterized, except that it was devastating to Native American communities and education. It was during this period that the new federal government was delegated responsibility for the education of the Indian peoples under its jurisdiction, but this did not signal the beginning of 18 either consistent or beneficial policies and approaches to Indian education. Two competing, and equally dreadful, points of view alternately dominated the direction that Indian education took for much of this time: (a) the assimilationist view, which favored vocational training for Indian people to prepare them for manual labor in White society and (b) the "removal" view, which favored removing Indians from White society by relocating them to separate, segregated tracts of land. The assimilationist view ultimately dominated (Carnegie, 1997). During both the colonial and the federal periods, the culture and values of Indians attending colleges were neither appreciated nor reinforced. In addition, Indians had no control over the purpose of their own education and thus feared they would lose their culture i f they attended these institutions (Carnegie, 1997). Treaty Period (1778 - 1871) With the founding of the United States in 1776, the responsibility for Indian-related endeavours was delegated to the federal government (Carney, 1999), and Indian tribes now had to deal with this new authority. During the treaty period (1778 to 1871), the government negotiated 645 treaties with tribes (97 of which included minimal educational provisions), thus recognizing the sovereignty of those communities. Tribes therefore became "dependent nations" of the federal government (Carney, 1999). This began an era, lasting until the 1960s, when the higher education of Indians was largely neglected. The Indian Civil ization Ac t (1819) The Indian Civil ization Ac t of 1819, the most important legislation during the federal period, granted $10,000 a year for educating Indians in "civi l ized" agricultural practices and Christianity (Carney, 1999; Oppelt, 1990). In effect, only a small amount of this money was 19 ever allocated to Indian education (University of Illinois, 2000). Because the government had been unsuccessful in its attempts to educate Indian children up to this time, from 1770 to the 1870s, these minimal funds for educating Indians were channeled through missionaries who, once again, were in charge of the Indians' education. The purpose of this religious instruction was yet again mass conversion (Carnegie, 1997) — it also called for the obliteration of Indian religious practices, dances, and cultural practices (University of Illinois, 2000). Because it provided money to churches to educate and promote Christianity, the Indian Civil ization Ac t program ended in 1870 when controversy arose over the issue of the separation of church and state (Carney, 1999). During the early 1820s, when the federal government's support for Indian education consisted of minimal financial assistance for Indian students attending eastern colleges, there were no higher education institutions for Native Americans in the U.S . (Carney, 1999). It was also during this time that the historical trend of the government's not allocating appropriated funds for Indian education originated, as the minimal provisions for Indian students were infrequently honored (Carney, 1999). Removal Policies (1825) In 1825, the forced relocation of Indians westward, with its concomitant loss of their lands, became the primary federal policy for Indian populations (Carney, 1999). The little involvement the federal government had with education during this period was limited to policy and funding. Educational policy shifted its focus from higher education (to convert and civilize Indians) to vocational training (combined with religious education) to prepare Indians for largely manual jobs. 20 About this time, several tribes recognized that getting a White education might help their cause (Oppelt, 1990). Most notably, the Cherokee and Choctaw tried to negotiate provisions for education in many of their treaties (Carney, 1999). The Choctaw even planned a higher education institution, but their plans fell through when they were forcibly removed from their lands. Nonetheless, the Cherokee and Choctaw were still able to obtain funding for some of their top students to attend colleges in the east (Barbara Wright & Tierney, 1991).1 Later, after the C i v i l War (1861 - 1865), the Choctaw were able to provide scholarships from tribal funds for students to attend college. Both tribes established excellent school systems and remained committed to higher education (Carney, 1999; Oppelt, 1990). Assimilation Policies (1870s) Since their first encounter with Indians, Whites have discussed the pros and cons of assimilation versus removal and have been rather haphazard in their attempts at both. U p until the 1860s, removal advocates predominated as Indian tribes were forced farther and farther west (Carney, 1999). In the 1870s, however, assimilation replaced removal as the dominant policy. A s White populations continued to increase, there was less available land on which to relocate Indian communities. In 1870, the federal government provided $100,000 for Indian industrial schools that were to be under federal control for the first time (Barbara Wright & Tierney, 1991). The underlying intentions were again to inculcate White values and to assimilate Indians into low-level positions in society. Furthermore, because individual Indians would soon be allocated 1 Mainly from the Civilization Fund and Scottish (Occum's) Fund. 21 tracts of land, it was thought that Indians should be educated about farming practices (Carney, 1999). Federal assimilation policy found its expression in two significant pieces of new legislation: the Dawes General Allotment Ac t (1887) and the Curtis Ac t (1898). Together, these two acts affected two essential aspects of the Indian way of life and values: education and land (Szasz, 1999). The Dawes General Allotment Ac t officially removed land from tribal control and allotted individuals parcels of land. The logic was that it would be easier to assimilate Indians into the larger society i f there were no tribal structures or tribal land (DeJong, 1993). The Curtis Ac t abolished tribal courts and governments (DeJong, 1993) and resulted in the loss of tribal sovereignty (Carney, 1999). Throughout the 1870s, the government's approach to Indian education also included moving students from their families and tribes into residential schools, whereby, it was believed, they could be assimilated into American society (Carnegie, 1997). In 1879, the first government boarding school was established in Carlisle, Pennsylvania. The Carlisle Indian School was founded by Richard Pratt and, by 1899, 25 "Carlisle" schools had been established, with 20,000 Indian students attending 25 boarding schools in 15 states (Encyclopedia of North American Indians, 2005a; Szasz, 1999). In these schools, Native American children were provided with vocational education and some religious training. It was hoped the children would later return to their families as missionaries and convert them to Christianity (Carney, 1999). Children were forced to speak English and were not allowed to speak their Native languages or practice their culture in any way. Pratt honestly believed Native American students could excel in that environment and that they had a capacity to learn, a belief held by few people at that time — he also believed that it was best to " k i l l the Indian and save the man" (Encyclopedia of North 22 American Indians, 2005a, p. 1; Szasz, 1999). Many of the children were physically, emotionally, and sexually abused — and hundreds of them died under these harsh circumstances (Encyclopedia of North American Indians, 2005a; Szasz, 1999). The approach was deeply flawed: Dropout rates were high, attendance was not mandatory, inappropriate training was provided, and no employment was offered to graduates in the community. Thus, in 1917, the last Carlisle school closed and Indian students returned to their communities and their traditional ways of life (Carnegie, 1997). A s the schools failed, so too did the assimilation policy. Despite the failure and closing of the Carlisle schools, Pratt was able to convince the public that Indians could be educated. A s a result, these schools were the progenitors of what finally evolved into more progressive educational initiatives. Some of these schools developed into vocational boarding schools, a number of which have survived and operate today (Szasz, 1999). Haskell Indian Nations University in Lawrence, Kansas, was established in 1884, and the Institute of American Indian Arts in Santa Fe, New Mexico, was founded in 1890 (Szasz, 1999). Both of these institutions are now federally chartered and provide exclusively Native American higher education opportunities. In particular, Haskell has been influential in that it has provided, and it continues to provide, Native Americans from diverse backgrounds with culturally relevant educational programs. Thus, Pratt's vision and his belief in educating Indians — however misguided it was — considerably shaped the progress and course of post-secondary education for Native Americans in the United States. The Evolution of Private Indian Colleges (1880s) Only two privately funded Native American colleges originated in the 19th century: "Indian University" (now Bacone College) in Muskogee, Oklahoma, and the Croatan Normal 23 School (now the University of North Carolina at Pembroke) (Carney, 1999). The first private Native American college, Bacone College (established in 1880), was based on Christian principles. Although it was also open to non-Natives, it was the first tribally controlled college and the only one in the U .S . for many years (Carney, 1999). A l m o n Bacone was a missionary and teacher, a proponent of providing a curriculum built upon the local culture and taught in the communities themselves. He was concerned about the decreasing Indian population. The Creek Council granted the college 160 acres to establish a campus in 1910. Bacone College was the first land grant college founded by Native Americans (Barbara Wright & Tierney, 1991). In 2003 to 2004, 39% of its 914 enrolled undergraduate students were Native American (Electronic Campus, 2004a). The second private Native American college, the University of North Carolina at Pembroke, was founded in 1887 and is the only state-supported four-year college founded specifically for Native Americans (Carney, 1999). Initially, the General Assembly of North Carolina established it to serve the Lumbees (i.e., the Croatan), with the aim of educating Indian teachers. Over the years, the college struggled financially and finally evolved into a mainly White institution. Although from 2003 to 2004, 21% of all enrolled undergraduate students at the University of North Carolina at Pembroke were Native American (of a total of 3,813), the college has abandoned its focus on Native American culture (Carney, 1999; Electronic Campus, 2004c). Between 1887 and 1954, Bacone College and the University of North Carolina at Pembroke were the only private institutions of higher education in the United States to serve mostly Native American students. It was difficult to establish Native American colleges for a number of reasons, including pervasive racism, the small Native American population base, the 24 dispersion of Native Americans, and federal control of all levels of Indian education (Carney, 1999). Furthermore, the only government assistance available to Native Americans in higher education at that time was limited scholarships and loans (Carney, 1999). In addition to Bacone College, there are two other small private Native American colleges still functioning today: American Indian College in Phoenix, Arizona, which was founded in 1972, and Nazarene Indian Bible College in Albuquerque, New Mexico, which was established in 1975 (Albuquerque Colleges and Universities, 2005; American Indian College, 2005). American Indian College is sponsored by the General Council o f the Assemblies of God and offers bachelor's degrees in Christian ministry and elementary education. Nazarene Indian Bible College offers limited bachelor's and associate's degrees in Bibl ica l studies and human resources management. The 20 t h Century During the late 1800s and early 1900s, changes in the perceptions and attitudes of a number of government officials, as well as the general public, toward Native Americans began to appear. More people were beginning to understand not only that Native Americans were worthy of a higher education but also that their education would benefit all of society. Furthermore, they were starting to believe that acculturation did not benefit society, that Native American culture should be valued and preserved (Carney, 1999). Despite these changes, Native American higher education, in practice, remained substandard and continued to offer only a basic curriculum that focused on low-level vocational skills (Carney, 1999). B y the turn of the century, the Native American population in the United States was less than 250,000, the lowest it had ever been, compared to an estimated 5 mil l ion at the time of 25 Columbus's arrival (Carnegie, 1997).2 Tribal governments had been powerless to stop the federal government's attempts to eradicate their cultural identity and sense of self (Carney, 1999). B y this time, the federal government's policies had had a devastating impact on Native American communities across the country, which had lost much of their language, land, culture, and hope. Disenfranchised in every way, Native American communities had become dependent on the government for their survival (Carnegie, 1997). In the early 1900s, only some tribes, one state, and some religious organizations paid any attention to Native American higher education; the federal government essentially ignored their plight. Since the early 1900s, some individuals believed that Native Americans should control their own higher education institutions (Stein, 2003). In 1911, August Breuninger made the first serious proposal to start a university that would enable Native Americans better access to higher education (Crum, 1989). He envisioned a curriculum that would focus on Native American culture and proposed that the college be associated with a museum. Breuninger's ideas, as well as others', were dismissed by assimilationists, and, over the next 50 years, other proposals to start Native American vocational colleges also failed (Carnegie, 1997; Stein, 2003). In the late 19 t h and early 20 t h century, a trend began whereby schools evolved into normal schools, which trained teachers, and then, occasionally, into colleges. Instead of transforming into Indian colleges, however, a number of Indian schools instead became White colleges, because of the financial need for a wider student base (Carney, 1999). These included Ottawa University in Kansas, Sheldon Jackson College in Alaska, the University of Tulsa in Oklahoma, Northeastern State University in Oklahoma (where Native Americans still have a significant 2 This number has been steadily increasing. In 2000, the U.S. Census reported that 4.1 million people (or 1.5% of the total population) claim some Native American ancestry (U.S. Census Bureau, 2004). 26 presence today), and Fort Lewis College in Colorado (Carney, 1999). The Indian students from these closed schools were sent to the public schools, where their culture was unappreciated (Carney, 1999). B y 1920, interest in Native American culture increased significantly in intellectual circles. Reformers became interested in the Pueblo Indian society and the preservation of Native American culture (Carney, 1999). This movement was to set the stage for the next era. The Self-Determination Period (1928 - present) U p until this time, "the principle of local citizen control of schools, that is a keystone of public education in the United States, was never extended to the education of Indians because they were deemed to be unqualified to manage their own schools" (Oppelt, 1990, p. 14). Things were about to change as the 20 t h century brought with it the most significant federal policy shifts toward self-determination in history (Carney, 1999). Two major reforms forced the government to begin to relate differently to Native Americans. The first, the Snyder Ac t of 1921, gave the Bureau of Indian Affairs direct responsibility for Indian higher education, even though, as in previous years, this was still limited to the support of students rather than institutions (Carney, 1999). The second, passed in 1924, was the Indian Citizenship Act , which granted U.S . citizenship to all Native Americans (Carney, 1999). These two pieces of legislation were significant in that they forced the government to regard Native Americans more justly, as citizens who deserved the right to control the future of higher education. It also set the stage, a number of years later, for the dreadful findings of the Meriam Report. 27 The Meriam Report (1928) In 1928, the Meriam Report, which was also known as the "Problem of Indian Administration," condemned the federal government for the abuse Native Americans had long endured (Carnegie, 1997; National Advisory Council on Indian Education, 1993). It particularly criticized federally run boarding schools, charging that because of the abuse experienced in these schools, the situation for Native American communities was bleak. It criticized the emphasis on vocational training, finding it completely unrelated to Indians' traditional ways and providing little hope for employment. It found that Native Americans were prevented from managing their own programs and being forced to use substandard services (i.e., health and education programs). The report encouraged more Native Americans to pursue higher education and suggested the federal government provide additional funding and financial aid for this purpose (Oppelt, 1990). Although the report recognized the importance of traditional culture and values, as well as community-based education, it stopped short of recommending the establishment of tribally controlled colleges (Carney, 1999). Nonetheless, the government's gradual shift from its paternalism to acceptance of self-determination stemmed from this report. In its wake, many boarding schools were closed, more Indian high schools with a greater percentage of Native American teachers were established, Native Americans were trained to serve in Bureau of Indian Affairs positions, and Native American culture was formally and actively preserved through financial support. These were the first concrete steps the federal government took away from the policy of acculturation. In 1932, as a result of the findings of the Meriam Report, President Franklin Roosevelt and Commissioner of Indian Affairs, John Collier, initiated significant changes affecting Native American communities (Carnegie, 1997). Collier, who was knowledgeable about Native American cultures and innovative community-based education models, began making decisions 28 based on Native American needs, advocating self-government and sovereignty for Native American communities (Oppelt, 1990). The goal now became empowering Native communities to restore their culture and to work with the government. For example, because most teachers were White, there was a movement to train Native Americans teachers. The situation was improving, but White men still continued to make most policies, and Native Americans themselves had little voice in the design of their own education. In addition, the potential for improved opportunities was set back when funding for these programs was reduced during and after World War II (Carnegie, 1997). Despite these challenges, the government's attitude toward Native Americans had changed significantly, paving the way for Indian self-government. The Indian Reorganization Ac t (1934) With the advent of the Indian Reorganization Act (also known as the Wheeler-Howard Act or the Indian B i l l of Rights) in 1934 and the concomitant termination of the Dawes General Allotment Act , tribal communities were finally given the right of self-government (Oppelt, 1990). Thus, Native American history moved from a dark period of assimilation and elimination to a period of tolerance and preservation (Carney, 1999). The Indian Reorganization Ac t resulted in the creation of a minimal loan fund for Native American college students and increased funding for higher education (Oppelt, 1990). A t about the same time, the Johnson-O'Malley Act also provided substantial vocational student loan funds and allowed states to provide education. Nonetheless, these two acts had little effect on increasing the numbers of Native Americans taking advantage of the opportunities for higher education because the education offered was still not relevant to Native American students, nor did it recognize or support their culture and values (Carney, 1999). 29 During the late 1940s and early 1950s, there was little apparent progress. The Bureau of Indian Affairs, 3 and some tribes, did begin scholarship programs, but the Bureau of Indian Affairs primarily concentrated its efforts on establishing elementary schools, a number of high schools, and a couple of technical institutes. A s a result, no tribally controlled colleges were formed during this time. The success of the scholarship programs was negligible as well; students still faced financial limitations, cultural barriers, inadequate academic preparation, minimal student support services, and long distances to the colleges (Carney, 1999). B y 1954, Congress again terminated its responsibility for Native Americans and thus funding for more than 100 Native American tribes (Carney, 1999). "Indian Education" This discussion would not be complete without pointing out that in Native American communities, there has always been disagreement over the appropriateness of Native Americans pursuing higher education. The juxtaposition of the two words 'Indian' and 'education' has almost always been problematic in spite of the agreement by Indian parents and Anglo policymakers on the importance of education for Indians (Bradley, 1980). Part of the problem is that Indian education is inherently a bicultural enterprise that has been directed at two sometimes competing and sometimes complementary goals: assimilation and self-determination (Havighurst, 1981). (Hampton, 1995b, pp. 7-8) The purpose of "Indian education" is to know oneself; to determine from where one has come, as well as one's passions (vocation); and to dedicate one's life to those pursuits. The ultimate goal is to become a complete individual and to provide the next generation with a sense of what is important and what is valued in society (Cajete, 2000). "Indian education must enhance 3 The Bureau of Indian Affairs scholarship program started in 1948. 30 Aboriginal consciousness of what it means to be an Indian, thus empowering and enriching individual and collective lives" (Battiste, 1995, p. xv). In order to do this, knowledge of one's community is essential (Marker, 2004). Finally, "the effectiveness of education is measured by how well it prepares people to handle the problems and opportunities of life in their own time and place" (Watt-Cloutier, 2000, p. 114). For over 350 years, the few Native Americans who could pursue higher education had to attend mainstream colleges that aimed at assimilating them into White society or recruiting them for missionary work (Carnegie, 1997; Karabel, 2005). These attempts to "educate" the Indians largely failed. Finally, by the late 1950s, the c iv i l rights era that was on the horizon, combined with the new community college movement, would bring new momentum and attitudes to support the founding of the first tribally controlled colleges (Stein, 1992). These efforts would be the first attempts of pursuing the vision of a true Indian education. T h e F o u n d i n g o f T r i b a l C o l l e g e s i n t h e U n i t e d S t a t e s ( 1 9 6 0 s ) In the early 1960s, Forbes, a Native American professor in California, proposed creating a Native American university. His vision for the university included providing kindergarten to grade 12 teacher training; recording the written form of Native American languages; teaching language, culture, and arts; and offering adult basic education courses to bring adults up to college-level work. Most importantly, he believed that Native Americans should control their own institutions (Stein, 2003). Forbes laid the groundwork for the tribally controlled colleges that were established in the following decade. Deganawidah-Quetzalcoatl University, founded in 1971 in Davis, California, was based on Forbes's proposal (In Search of Aztlan, 2004). Also during the 1960s, greater numbers o f Native Americans started attending college (Oppelt, 1990). This trend coincided with a number of factors and events. First, Native 31 American leaders renewed their commitment to self-determination, and tribal members increasingly regarded higher education as a way to serve the tribe, advance the process of self-determination, and restore traditional culture and knowledge (Carnegie, 1997). Second, in 1968, under President Johnson, the government committed to the policy of "self-determination" (Carney, 1999). Every president since that time has supported this policy (Carnegie, 1997). Third, in 1966, for the second time ever, a Native American, Robert Bennett, was named commissioner of the Bureau of Indian Affairs. 4 He was committed to self-determination and, thus, education became a priority (Carney, 1999). Finally, following Wor ld War II, access to higher education improved when the GI B i l l provided funding for returning veterans to pursue college degrees (Carnegie, 1997). Many Native American veterans took advantage of this opportunity but, discouraged by racism and discrimination, also dropped out at unusually high rates (Carnegie, 1997). Navajo Community College (1960s) In 1957, the Navajo Nation, with a population near 200,000, realized that with few members having the ability to lead the tribe, they needed to provide higher education i f they were to attain self-determination (Carnegie, 1997). To achieve this objective, the Navajo Nation established a scholarship fund to enable tribal members to attend colleges and universities. Over 50% of the students awarded scholarships dropped out by the end of their first year because they could not navigate the foreign cultures of those institutions (Carnegie, 1997; R. Nichols & Monette, 2003). Then, in 1966, the federal government funded educational services to the Navajo tribe through the Rough Rock Demonstration School. This successful project eventually 4 The first Native American was named commissioner of the Bureau of Indian Affairs in 1869. 32 led to the establishment of Navajo Community College (now called Dine College) in 1968, the first tribally controlled college in the United States (National Advisory Council on Indian Education, 1993). The college was federally funded and secured further funding from the Navajo tribe, the Office of Economic Opportunity, and the Donner Foundation. The clearly articulated principles implemented at Navajo Community College became the model for all future tribally controlled colleges: The college would be controlled and managed by the local people " in order to enhance the understanding of their heritage, language, history, and culture" (Stein, 2003, p. 31); would foster a sense of cultural pride and self-esteem, by integrating tribal culture, language, art, and history; would encourage the ability of students to work effectively in the larger society; would provide individualized attention to address social as well as academic challenges; and would seek new innovative teaching methodologies to effectively transmit knowledge (Carnegie, 1997; Stein, 2003). Coinciding with the c iv i l rights movement, the tribal college movement arose from an increased sense of pride and a resolve to legitimize and preserve cultural traditions, values, and knowledge through the process of self-determination (Carney, 1999). These objectives were to be accomplished through the creation of higher education institutions that would provide some of the many needed academic and vocational skills to tribal members. Some tribes experimented by bringing local community colleges into their communities, but, because the curriculum was culturally irrelevant, the effort died out (Carnegie, 1997). The closest higher education model available at the time was that of the rapidly expanding community college system (Stein, 1992). Therefore, founders of the tribal college movement chose to base the tribal college system on that model, which embraced community control (Stein, Shanley, & Sanchez, 2003). Tribal colleges and community colleges both valued 33 the importance of local control, local needs, open admission, job training, and local community development. One difference was that tribal colleges had as their goal the self-determination of Native people through education. Other differences between community colleges and tribal colleges lay primarily in the sources of funding, the governance structure, and the focus on culture (Carnegie, 1997; Stein, 1992). Following the example of the Navajo, in the 1970s, several other tribes founded their own tribally controlled colleges. In 1970, Deganawidah-Quetzalcoatl University in California was established, followed, in 1971, by Sinte Gleska University and Oglala Lakota College, both in South Dakota (Stein, 1992). During their accreditation processes, many tribal colleges maintained relationships with mainstream universities and colleges (e.g., the mainstream institution would provide courses through the tribal college). Indian leaders reasoned that agreements made with non-Native American colleges and universities would provide support while the tribal colleges were gaining strength in their pursuit of self-determination (R. Nichols & Monette, 2003). Federally Supported Native American Colleges (1962 - 1971) The Bureau of Indian Affairs currently operates and funds two Native American higher education universities: Haskell Indian Nations University in Lawrence, Kansas, which evolved from an Indian school in 1884 and became a junior college in 1970, and Southwest Indian Polytechnic Institute, in Albuquerque, New Mexico, which also evolved from an Indian school and then was founded in 1971 as a higher education institution providing technological training (Carney, 1999; Crazy B u l l , 2007). The Institute of American Indian Art in Santa Fe, New Mexico, is a federally chartered institution with a specific mission to support Native American arts and culture. It was founded in 1962. 34 The Kennedy Report (1969) and Indian Education Ac t (1972) In 1969, about mid-way through the period when the Bureau of Indian Affairs established its first tribal colleges, the Kennedy Report, also known as the report of the Senate Special Subcommittee on Indian Education (chaired by Robert Kennedy and then later by Edward Kennedy), examined Indian education and the dismal rate of success of students at all levels (DeJong, 1993). Reminiscent of the Meriam Report, there was still no mention of the importance of establishing tribally controlled colleges. The years following the release of this report saw a flurry of bills supporting Indian education. One of these, the Indian Education Ac t of 1972, required Indian education programs across the country to address low achievement and high dropout rates among their students. Another, the Indian Self-Determination and Educational Assistance Ac t of 1975, committed Congress to improving the social and economic conditions of Native Americans significantly by providing funding for technical training, Bureau of Indian Affairs support, and financial assistance for taking control of their own programs (Carnegie, 1997). It also gave tribes the right to contract with the federal government to operate educational programs serving their tribal members, such as Head Start, while maintaining local control of elementary and high schools. This added control over educational programs provided Native Americans with the impetus for creating additional educational opportunities in their communities and the confidence that self-determination was more than a hope. The American Indian Higher Education Consortium (1973) In 1973, the presidents of the first six tribal colleges founded the American Indian Higher Education Consortium (AIHEC) , an institution that represents the interests of the tribal colleges and is instrumental in supporting their work in assisting Native American communities in their national effort toward self-determination (Benham, 2003). B y pooling their resources, A I H E C 35 assisted the early tribal colleges with accreditation procedures, curricula development, and planning (R. Nichols & Monette, 2003). Today, A I H E C represents 32 colleges in the United States and is governed by all the member institutions. Its mission is to maintain commonly held standards of quality in American Indian education; support the development of new tribally controlled colleges; promote and assist in the development of legislation to support American Indian higher education; encourage greater participation by American Indians in the development of higher education policy. ( A I H E C , 2004, p. 1) The second central organization of tribal colleges, the American Indian College Fund, was established in 1989 through A I H E C to raise funds and provide financial assistance to tribal colleges and their students (Stein et al., 2003). In 1986, tribal college students founded the A I H E C Student Congress "to promote leadership, self-governance, cultural preservation, educational achievement, community development, economic sustainability and success for all Native people" ( A I H E C Student Congress, 2004, p. 1). The Tribally Controlled Community College Assistance Ac t (1978) Three pieces of legislation addressing Native education followed upon each other in 1978: the Education Amendments Ac t of 1978, which focused on elementary and high school education; the Higher Education Act , which provided increased funding both for Native American students attending college and for Native American college programs (Barbara Wright & Tierney, 1991); and, finally and most significantly, the Tribally Controlled Community College Assistance Act , which provided minimal federal funding for the main operations of 36 tribally controlled colleges (Benham, 2003; Carnegie, 1997; R. Nichols & Monette, 2003). "After a lengthy struggle" (Shanley, 2003, p. 63) with A I H E C , the government provided this minimal funding annually to each tribally controlled college for each Native American student it served. Native Americans with new skills were required to implement this new legislation effectively; therefore, the need for educated leaders and higher education, specifically tribal colleges, became obvious (Carnegie, 1997). Land Grant Status (1994) In 1994, through the Mor r i l l Act , land grant status was granted to 29 tribal colleges (Stein et al., 2003). This important designation provided these colleges with support from mainstream institutions and provided minimal funding for agricultural and natural resource programs. Furthermore, in 1996, the White House Initiative on Tribal Colleges and Universities created a partnership between the federal government and tribal colleges and mandated that tribal colleges receive increased resources and services (Stein et al., 2003). From the beginning of the self-determination period through 1994, measures were created with the intention of improving Native American access to higher education, strengthening educational programs, increasing graduation rates, and educating people "who are grounded in their own culture, yet are prepared with the social, civic, and work skills they need to live and contribute to a multicultural, global society" (Kellogg Foundation, 1998, p. 2). Tribal colleges were attempting to f i l l a void by providing a link between higher education in mainstream society and tribal communities, so that their graduates would be capable of functioning in both worlds (Carnegie, 1997). 37 Tribal Colleges Today Purpose and Philosophy In the United States, tribal colleges evolved because existing systems of higher education still required assimilation and were not benefiting Native American students, who were often geographically isolated, had fewer opportunities to access post-secondary education, or did not have the academic preparation to succeed in mainstream post-secondary institutions. For example, in 1984, only 3% of Native Americans graduated from college, compared to 16% of non-Native Americans (Tierney, 1991). A s Boyer (1997) notes, especially since [enactment of] the GI B i l l and the Higher Education Act , the federal government, as well as individual colleges and universities had encouraged Indian students to enroll. But as more did, it became clear that access did not guarantee academic success. The dropout rate for American Indians remained at 90 percent or higher at many institutions, (p. 25) In response to these concerns, tribal colleges were established for a two-fold purpose: (1) to preserve and restore traditional tribal culture and knowledge and (2) to provide certificates or associate's (two-year) degrees that would provide Native American students with better employment opportunities and facilitate transfer to four-year colleges or universities (Farm, 2002; Stein, 2003). Tribal colleges, in the broadest sense, are committed to the self-determination and survival of Native peoples through education (Boyer, 2003). A s N W I C President Crazy B u l l (2004) elaborates, "self-determination means choice, access, and control over human and natural resources" (p. 1). The colleges provide Native American students from isolated areas, who might not otherwise pursue post-secondary options, with access to higher education (Boyer, 2003). In addition, as Boyer (1990) points out, more Native American students are attending tribal colleges to avoid racist and non-relevant curricula, to counter prior 38 negative experiences in mainstream community colleges and universities, and to be educated in an environment that addresses the needs of tribal communities. In addition to serving individual students, tribal colleges also have a wider mandate: to address issues of social justice as well as the social and economic needs of their tribal communities, through research, instruction, service, and other innovative means ( A I H E C , 1998; Benham, 2003; Stein, 2003). Furthermore, tribal colleges provide a supportive learning environment for students who generally have not been successful in mainstream colleges; promote tribal culture, art, history, traditions, and language (the preservation of language is seen as being of primary value in protecting Native American culture); provide services that enhance the communities they are in; and are centers of research and scholarship that also benefit their tribal communities. Furthermore, tribal colleges provide Native American people with a means of realizing equality and sharing in the opportunities of the larger society; for collective social and economic mobility; o f overcoming dependency and "neo-colonialism"; of providing the expertise and leadership needed by First Nations communities; to demystify mainstream culture and learn the politics and history of racial discrimination.. .to address communal need for "capacity-building" to advance themselves as a distinct and self-determining society, not just as individuals. (Kirkness & Barnhardt, 1991, p. 4) Finally, in the context of Native American perspectives and culture, tribal college programs also include some of the more mainstream liberal arts purposes to which Axelrod (2002) refers, such as developing the critical thinking, analytical, and communication skills of their students. Thus, tribal colleges provide access to post-secondary education, a first step to acquiring social capital (Bourdieu, 1986), and an essential step to self-determination. Enrollment and Locations Since the establishment of the first tribal college in 1968, Native American participation in higher education has increased at a rapid rate, as more than one third of all Native American 39 students in two-year colleges are attending tribal colleges ( A I H E C , 1998; Ortiz & HeavyRunner, 2003). Currently, there are 32 tribal colleges serving approximately 30,000 full- and part-time students, about 85% of whom live below the poverty line ( A I H E C , 2005; Benham, 2003). A s indicated by Figure 2.1, tribal colleges are located throughout the United States, mainly in the northern plains states (i.e., Montana, North Dakota, and South Dakota). 40 Figure 2.1 Map of Tribal Colleges and Universities in the United States Note. Map from the American Indian College Fund (2007). Founding dates are in parentheses. Arizona 1. Dine College Tsaile, Arizona (1968) 2. Tohono O'odham Comm. College Sells, Arizona (1998) Colorado Amer. Indian College Fund (headquarters) Denver, Colorado (1989) Kansas 3 Haskell Indian Nations University* Lawrence, Kansas (1970) Michigan 4. Bay Mil ls Community College Brimley, Michigan (1984) 5. Keweenaw Bay Ojibwa Comm. Col l . Baraga, Michigan (1975) 6. Saginaw Chippewa Tribal College Mount Pleasant, Michigan (1998) Minnesota 7. Fond du Lac Tribal and Comm. College Cloquet, Minnesota (1987) 8. Leech Lake Tribal College Cass Lake, Minnesota (1992) 9. White Earth Tribal and Comm. College Mahnomen, Minnesota * These colleges are federally chartered. Montana North Dakota 10. Blackfeet Community College Browning, Montana (1979) 22. Cankdeska Cikana Comm. College Fort Totten, North Dakota (1974) 11. Chief Dull Knife College Lame Deer, Montana (1975) 23. Fort Berthold Community College New Town, North Dakota (1988) 12. Fort Belknap College Harlem, Montana (1984) 24. Sitting Bul l College Fort Yates, North Dakota (1986) 13. Fort Peck Community College Poplar, Montana (1978) 25. Turtle Mountain Community College Belcourt, North Dakota (1972) 14. Little Big Horn College Crow Agency, Montana (1980) 26. United Tribes Technical College Bismarck, North Dakota (1987) 15. Salish Kootenai College Pablo, Montana (1977) 16. Stone Child College Box Elder, Montana (1984) Nebraska 17. Little Priest Tribal College Winnebago, Nebraska (1996) 18. Nebraska Indian Comm. College South Dakota 27. Oglala Lakota College Kyle, South Dakota (1971) 28. Sinte Gleska University Mission, South Dakota (1970) 29. Sisseton Wahpeton College Sisseton, South Dakota (1984) Macy, Nebraska (1978) New Mexico 19. Navajo Technical College Crownpoint, New Mexico (1993) 20. Institute of American Indian Arts* Santa Fe, New Mexico (1962) 21. Southwestern Indian Polytechnic Institute* Albuquerque, New Mexico (1971) Washington 30. Northwest Indian College Bellingham, Washington (1983) Wisconsin 31. College of Menominee Nation Keshena, Wisconsin (1993) 32. Lac Courte Oreilles Ojibwa Comm. Col l . Hayward, Wisconsin (1982) 41 Although the largest tribal college (Dine College) has approximately 1,500 students, most tribal colleges are small, with enrollments of less than 400. Even with such small enrollments, the colleges have the potential to significantly influence their communities where the populations they serve are usually also small (i.e., a couple of thousand people) and graduating a handful of tribal members can make a difference. Furthermore, because some tribal colleges are gaining reputations as "legitimate institutions of higher learning" (Carnegie, 1997, p. 36), they now recruit Native Americans from outside their immediate areas. Programs Initially, all 32 tribal colleges were established with the intention of providing vocational training to upgrade tribal members' skills to become employed, and the majority still focus much of their curricular efforts in this direction through certificates and two-year degrees (e.g., construction trades, medical coding, office professional training) (Carnegie, 1997). Furthermore, though most began as two-year colleges, currently, seven tribal colleges offer four-year bachelor's degrees,5 and two offer master's degrees6 (American Indian Higher Education Consortium, 2005). In addition, some two-year tribal colleges offer transfer degrees by forming partnerships and articulation agreements with local universities that provide opportunities for tribal college students to transfer and complete their bachelor's degrees (Carnegie, 1997). The most common programs of study are the ones that directly support the social and economic development of the tribes they serve: education (especially early childhood education and K-8 teacher training), health, business, and environmental studies. 5 Sinte Gleska University, Oglala Lakota College, Salish Kootenai College, Haskell Indian Nations University, Institute of American Indian Arts, Sitting Bull College, and Turtle Mountain Community College. 6 Sinte Gleska University and Oglala Lakota College. 42 Students Overall, the students at most tribal colleges are different in significant ways from their non-Native American counterparts at other colleges (Boyer, 1990). Most obviously, approximately 80% to 85% of students attending tribal colleges are Native American, and the average student is a single mother (typically in her late 20s or early 30s) (Carnegie, 1997; Farm, 2002; Ortiz & HeavyRunner, 2003). Although the majority of tribal college students work, most still have substantial financial limitations that affect their ability to provide themselves and their families with stable housing and food. College is a significant financial burden for most tribal college students. In addition, students often lack basic reading, writing, and math skills. A l l these factors combine to result in higher stop-out1 and dropout rates compared to students in mainstream community colleges (Ness, 2002; Oppelt, 1990). On the bright side, most tribal college students are the first in their families to attend college, approximately 40% of the graduates pursue further education, and many graduates find meaningful work ( A I H E C , 2004; Farm, 2002; Bobby Wright & Weasel Head, 1990). Factors Influencing Access, Retention, and Success in Mainstream Higher Education Native Americans have been proportionately under-represented in colleges and universities since the advent of higher education in the United States (Kirkness & Barnhardt, 1991). Native American students face two major hurdles related to college: getting accepted into and graduating from a mainstream college (Carnegie, 1997). Bourdieu's (1986) work on class reproduction clearly demonstrates that many people do not have a real "choice" to pursue their education further because of social-structural barriers that prevent them from advancing. Rather 7 A student stops-out when he or she skips a term and returns at a later time. 43 than assist individuals to advance, education systems perpetuate social inequalities. Life experiences and employment outcomes can still be predicted based on social class and gender, demonstrating that all people do not have equal opportunities to learn (Andres, 1994; Dougherty, 1987; Furlong & Cartmel, 1997; Guiton & Oakes, 1995; M c C a l l , 1992). Bourdieu (1986) regards capital as "accumulated labor (in its materialized form or its "incorporated," embodied form) which, when appropriated on a private, i.e., exclusive, basis by agents or groups of agents, enables them to appropriate social energy in the form of reified or living labor" (p. 241). Capital is "the set of actually usable resources and powers" (Bourdieu, 1984, p. 114) and it takes time to amass. Social and cultural capital relate most to this thesis. He defines social capital, or power derived by social means, as the connections an individual has to social networks and resources, including to people and groups (1986). Social capital plays an important role in the acquisition of other forms of valued capital. Cultural capital is the accumulation of advantages a person has as a result of cultural resources, which are passed on primarily through the family, such as behaviour, attitudes, and habits. The various forms of capital an individual accrues as a result of life experiences w i l l assist h im or her to realize his or her goals. The course of an individual's life is a product of interacting factors related to social structure, individual agency, organizational processes, and historical precedence (Gerhardt, 1996). In addition to economic barriers, a number of social and cultural barriers self-exclude Native Americans and make it difficult for them to pursue higher education (Anderson, 1993; Ortiz & HeavyRunner, 2003). The three factors most commonly identified in the literature that influence the access, retention, and success of Native American students in higher education and 44 are related to cultural and social capital are (a) family support and background, (b) family obligations and domestic challenges, and (c) academic preparation. The first factor found to be critical to access, retention, and success is family support and encouragement (i.e., social capital) (Falk & Aitken, 1984; L i n , 1990; Wenzlaff & Biewer, 1996). Family support and encouragement are measured in a number of ways, and Native American students are disadvantaged on nearly all measures. For example, students who have close family members who have succeeded in college are more likely to be successful themselves (Wenzlaff & Biewer, 1996). Few Native American college students have this advantage (i.e., cultural capital); most are the first in their families to attend college. Parents' education and household income are also strong determinants of post-secondary access and success, and mean earnings increase with educational level (Andres, 1994; Finnie, 2001; Leek, Onge, & Lalancette, 1995). Higher parental educational attainment and higher earnings have a positive effect on children's educational attainment (Haveman & Wolfe, 1995). Such parents have, for example, more social, economic, and cultural resources to assist their children to succeed. Furthermore, parents can play an important role in motivating their children to pursue post-secondary education, and those who place a high value on higher education transmit that value to their children (Andres, 1994; Knighton, 2002). Thus, Native American students who have graduated from college tend to encourage other family members, including their children, to pursue higher education (American Indian College Fund, 2002). Family obligations and domestic challenges constitute the second factor influencing access, retention, and success for Native American students and include issues in students' lives such as childcare or transportation needs, housing problems, relationship difficulties, health problems, and employment concerns. These types of challenges, which are widespread in Native 45 American communities, often interfere with students' attempts to pursue higher education (Boyer, 1997; Ness, 2002). Extremely high unemployment rates combined with high rates of suicide, alcohol and drug addiction, poverty, and domestic abuse indicate conditions that often prove to be insurmountable (Boyer, 1997). When a student is faced with challenges such as these, survival, rather than educational advancement, becomes the goal (Raffo & Reeves, 2000). Academic preparation is the third major factor outlined in the literature as critical to improving access, retention, and success in college because mainstream elementary and high schools are typically not responsive to the needs of Native American students. Native American students often lack fundamental academic skills, such as basic reading, writing, math, and study skills, as well as basic career information. Remediation eventually becomes a considerable barrier because students have to spend significant time and money taking developmental coursework to upgrade their skills before they can move on to college-level work (Boyer, 1997). In addition to the three main factors listed above, the literature identifies other barriers that prevent Native American students from attending or succeeding in higher education. The most prevalent of these include distance from colleges, lack of financial resources, lack of motivation, low self-esteem, loneliness or alienation at school, inability to integrate in the college culture, lack of support groups, lack of long-range commitment or career goals, conflict o f values and customs, and poor fit (i.e., academically, socially, or culturally) between the institution and the student (Machamer, 2000; Ness, 2002; Ortiz & HeavyRunner, 2003; Shishkoff, Thomas, & Al-Bayat i , 1999). Furthermore, once at college or university, Native American students often feel cultural discontinuity between what they learn in the classroom and what they know from their culture and tribal communities, as mainstream colleges do little to make the curriculum and teaching styles culturally relevant to students (Reyhner, Lee, & Gabbard, 1993). Again, lack of 46 cultural capital disadvantages Native American students and decreases their likelihood of success. The racism many Native American students experience on college and university campuses also affects their chances to succeed (Archibald & Selkirk, 1995). Instructors may have lower expectations for Native American students (Archibald & Selkirk, 1995; Shields, 1999), and the lack of mentors or role models reduces success rates. Finally, in addition to the challenges outlined above, Native American students face many of the same barriers that mainstream college students face, such as the complexities of course scheduling, inadequate academic advising and counseling services, and problems accessing student financial aid (Andres & Carpenter, 1997; Archibald & Selkirk, 1995; Benjamin, 1994; Ness, 2002). Many of these factors combine to make pursing a college education a low priority. Unique Features of Tribal Colleges that Increase Student Access, Retention, and Success Tribal colleges offer several advantages that ease the challenges facing Native American students outlined above. These advantages include convenient locations and easier access, lower cost, an appreciation of Indigenous knowledge and culture, appropriate teaching methodologies, relevant curricula, extensive student support services, and dedicated faculty. Each of these is elaborated upon below. Location, Access, and Cost A primary advantage tribal colleges offer is their location. Because many Native American students can remain at home while attending a local tribal college, the financial burden of a college education is significantly reduced, and they can be close to the support of their 47 extended families and maintain their cultural connections. This is especially important because most graduates prefer to seek employment in or near their own communities (Hayes, 1990; Ortiz & HeavyRunner, 2003). In addition, tuition fees at tribal colleges are generally lower than at community colleges, and open admission policies mean that many students who cannot meet admission requirements elsewhere, or who lack clear career ambitions, still have the opportunity to pursue further education (Hayes, 1990; Ortiz & HeavyRunner, 2003; Shishkoff et al., 1999). Indigenous Knowledge and Culture Indigenous knowledge and culture shape tribal colleges and their philosophy of education (Carnegie, 1997; N W I C , 2002). Although there is no universal agreement about the nature of Indigenous knowledge, there is some consensus in Indigenous knowledge and research literature that it comprises an integrated and holistic approach to knowledge (Archibald, 1999, 2001; Battiste & Henderson, 2000; Cajete, 1994; Castellano, 2000; Dei , 2000; Dei , Ha l l , & Rosenberg, 2000; Hanohano, 1999; Marker, 2000; L . Smith, 1999; White & Archibald, 1992). Battiste and Henderson (2000) quote the Royal Commission's understanding that Indigenous knowledge is "a cumulative body of knowledge and beliefs, handed down through generations by cultural transmission, about the relationship of l iving beings (including humans) with one another and their environment" (p. 42). Indigenous knowledge is "oral, experiential, holistic, and conveyed in narrative or metaphorical language"; it "is rooted in personal experience and lays no claim to universality" (Castellano, 2000, p. 25). Castellano highlights three sources of Indigenous knowledge: traditional teachings, which are preserved and passed down from elders; empirical observation, which is acquired through careful observation; and, revelation, which is obtained through visions 48 and dreams (p. 23). These forms of knowledge are "qualitative and subjective rather than quantitative and objective" (p. 27) in nature and are "rooted in local cultural traditions, values, and belief systems. It is a worldview that shapes the community's relationships with its environments. It is a knowledge base that is crucial for group and community survival" (Dei, 2000, p. 79). Marker (2000) writes that "an Aboriginal approach to teaching and learning would emphasize how knowledge and sense of selfhood come from a concrete place" (p. 41). Accordingly, values based on kinship, relationships, respect of elders, and connections to the land are integral to tribal colleges' mission. "Students learn firmly that who they are and what they believe has great value. Rather than being a disorienting experience for students, college represents a reinforcement of values inherent in the tribal community" (Carnegie, 1989, p. 56). One of the major challenges tribal colleges face is how to found the curriculum and college experience with this holistic and integrated approach to Indigenous knowledge and education. Pedagogy One of the most significant and, at the same time, challenging factors that contributes to tribal colleges' success is that they reflect the culture, values, and history of their surrounding reservations, which results in greater sensitivity to the unique cultural needs of their students (Ortiz & HeavyRunner, 2003). B y framing the curriculum in a holistic and culturally relevant way, the instructors reaffirm the identity and values of their students, as well as of the surrounding Native American communities (P. Hughes, 1998; P. Wilson, 1994). The tribal college philosophy infuses familiar cultural elements and a practical approach into the post-secondary educational experience (Pavel & Colby, 1992). Battiste and Henderson (2000) note that elements of Indigenous knowledge can best be passed on to students when 49 traditional teaching methods such as apprenticeship, ceremonies, oral stories, and practice are employed. Tribal colleges provide Native Americans with an opportunity for higher education, with smaller class sizes, in a less competitive environment (Lin , 1990; Shishkoff et al., 1999). Furthermore, whereas the focus of the curriculum at mainstream universities tends to be on "decontextualized literal knowledge" (Kirkness & Barnhardt, 1991, p. 4), traditional, oral, or Indigenous knowledge is "interconnected" (Marker, 2004, p. 180), "is integrated in everyday life," and is "acquired through direct experience and participation in real-world activities" (Kirkness & Barnhardt, 1991, p. 4). Ideally, Indigenous knowledge forms the foundation of the classroom and tribal college experience. Related to this natural inclination toward real-life and service-oriented experiences is a relatively new movement in mainstream colleges, and now in some tribal colleges, toward incorporating service learning into the curriculum. Service learning lends itself well to the tribal college context: Opportunities for community service work are incorporated into courses, allowing students to apply the concepts they are learning in a relevant and meaningful way (Crazy B u l l , 2004). Because tribal colleges work endlessly to provide culturally relevant experiences for their students, service learning provides the structured link between the community and the students' learning experiences. Crazy B u l l (2004) emphasizes that "these experiences can serve as the foundation for social action, which is critically needed to overcome poverty and loss in our communities" (p. 1). Likewise, in order to respond to the educational needs of isolated students in remote communities, learning technologies, such as interactive television, have become a more accepted mode for learning (Carnegie, 1997). Independent learning and online classes are also becoming more popular, although there continue to be problems with access to computers and the Internet 50 in remote areas (Barden, 2003). Neither of these modes for learning is ideal and, as a result, tribal colleges struggle to find ways for isolated students to access higher education using more appropriate instructional methodologies. Faculty members at tribal colleges continually experiment with culturally responsive teaching methodologies to increase their students' success. Curriculum Like mainstream colleges, general education requirements at tribal colleges dictate that students take a variety of courses to complement their core area of study (Carnegie, 1997). On the other hand, the incorporation of traditional culture and knowledge throughout the curriculum, including the general education requirements, is an effort to make the curriculum more relevant, interrelated, and accessible to students (Boyer, 1990; Carnegie, 1997). The curriculum at tribal colleges, therefore, emphasizes not only academic requirements but also the cultural knowledge and traditional pedagogies of Native American communities (Pavel & Colby, 1992). Therefore, all tribal colleges offer Native American studies courses in areas such as history, art, philosophy, botany, and local language. Restoring the use of traditional languages is of particular importance to Indigenous knowledge, for as Mann (2003) emphasizes, "when a language is at risk, so is a culture" (p. xxi i i ) . According to Amiotte and Al l en (1989), "one of the key reasons for the tribal college's success has been the belief and practice that students can remain Indian, can practice tribal traditions and retain tribal values and also be successful students" (p. 1). Student Support Services Extensive student service programs are offered at many tribal colleges. These services include additional academic support, peer support, and individual and financial assistance (Stein, 2003). Students registering at a tribal college are assessed and then enrolled in courses based on 51 their ability (Stein, 2003). Many students require remedial coursework, which is often self-paced and offers individual attention in a supportive learning environment (Machamer, 2000). Students are treated with respect by staff and faculty, as the well-being of each student is of primary importance to the tribal college and community (Kirkness & Barnhardt, 1991). Finally, tribal college staff and faculty attempt to foster in students a strong sense of self-esteem and resiliency, a goal-completion mentality, and an ability to "walk in two worlds" without losing personal identity (Ness, 2002). They try to prepare students academically, assist with family matters (where appropriate), and actively support students in obtaining financial aid. Because "educational attainment is an important determinant of [future] job opportunities and relative well-being" (De Broucker & Underwood, 1998, p. 30), this support system is crucial not only to students' academic success, but to their (and their community's) well-being. Thus, tribal colleges provide a critical transition to higher education for many Native American students, whereby cultural and social capital is gained and cultural identity is affirmed (Tierney, 1999). Faculty Faculty members play a key role in students' success at tribal colleges. Currently, most instructors at tribal colleges are non-Native because the extremely low wages, minimal benefits, and the limited number of qualified applicants at tribal colleges make it difficult to attract Native American faculty. Tribal colleges attempt to increase the proportion of Native faculty because these faculty provide role models for students and help them to realize the importance of education (Carnegie, 1997). The Carnegie Foundation (1997) identified a number of challenges instructors at tribal colleges face that limit professional development opportunities and result in high turnover rates. These include heavy teaching loads, isolation from colleagues at mainstream colleges, limited 52 funds, and under-prepared students. Generally, however, those who overcome the challenges and stay at the colleges tend to be sensitive to the culture in which they are serving and dedicated to their students. Faculty are found to be enthusiastic, supportive, caring, respectful, knowledgeable, and accessible (Carnegie, 1997). Students frequently report satisfaction with having a significant personal connection to faculty who are dedicated to their well-being (Ness, 2002). This satisfaction might be attributable to instructor-student learning relationships being more reciprocal and respectful in nature, when compared to mainstream colleges (Kirkness & Barnhardt, 1991). In addition to professional educators, tribal colleges may employ part-time instructors who are tribal members with limited formal education but who have expertise in particular areas such as language, art, history, and culture. Elders also play an important role in the tribal colleges, either as instructors or advisors to the colleges on issues of cultural importance (Carnegie, 1997). Other Common Features and Challenges of Tribal Colleges Although each tribal college has a distinctive culture and serves a unique community and economy, they all share fundamental similarities, including the challenges that they face (Stein et al., 2003). The common features of tribal colleges include their governance structure, their focus on "strengthening economic empowerment" (Benham, 2003, p. 11), Indigenous research, and funding sources, all of which are expanded upon below (Carnegie, 1997). Governance A l l tribal colleges are chartered by at least one tribal government and have a board of directors who are mostly Native American (Carnegie, 1997; Stein et al., 2003). In order to 53 become accredited, they must demonstrate administrative distance from their tribal governments (i.e., autonomy), so they can develop their own policies and avoid conflict-of-interest situations (Stein et al., 2003). Tribal colleges are endorsed as post-secondary educational institutions serving the cultural, social, and economic needs of their communities. Their presidents and most of the administrators are well-qualified and credentialed Native American educators who are committed to the long-term success of their institutions (Carnegie, 1997). Economic Empowerment Tribal colleges are "seen as integral and contributing partners (in some situations, leading ones) in the cultural and economic growth of a community" (Benham, 2003, p. 5). Higher education is important for economic development and for improving quality of life, the key to the social and economic growth of Native people, as well as to their continued survival. Tribal colleges empower local communities in a number of ways (Fann, 2002). They provide higher education opportunities to tribal members that are specific to the needs of their communities, resulting in many graduates remaining in their communities to work. They also spend money, create employment, and provide various services to their local communities — a significant achievement, given that many reservations have unemployment rates anywhere from 45% to 90% (Boyer, 1997). Even though their primary obligation is to the specific needs of the local community, they also recognize and attempt to address the more general needs of other Native American tribes, as well as the national and international communities (Barden, 2003). Tribal colleges are committed to providing their local communities with social and economic development initiatives through education (Barden, 2003). In the past, the primary development strategy was to attract businesses to the reservations, but most of these initiatives were unsuccessful. Tribes learned that they needed to assess their local strengths and needs and 54 to develop culturally appropriate initiatives that could be sustained over the long term (Barden, 2003). A couple of tribal colleges have done just that and own and operate their own businesses, which provide employment for tribal members and which they hope w i l l eventually be sold to individuals in the community (Barden, 2003). 8 Tribal colleges provide vocational training and education so that students can return to the community as contributing and skilled members. From almost the beginning, tribal colleges, like their mainstream counterparts, have provided services to the markets, including curricula that train students to f i l l specific market needs (Trow, 1993). For example, for the gaming industry, which has become a huge business for some tribes, tribal colleges may provide training for employees or management skills to staff. A reciprocal benefit of this relationship is that the colleges may also receive dividends from casinos to support the college (Barden, 2003). Research Research, or the production of Indigenous knowledge, is an important goal of tribal colleges. In order to conduct research, tribal colleges need financial and human resources, funding, faculty time, and advanced students — none of which they currently have. Whereas well-endowed research universities are generally involved in the creation of new knowledge, up until this point, tribal colleges have had to focus primarily on surviving and on transmitting and preserving Indigenous knowledge (Trow, 1993) and "rebuilding respect for traditional ways of knowing" (Carnegie, 1997, p. x). For example, Salish-Kootenai College has an environmental testing project. 55 These conditions are now starting to change. In 1989, with the conception of the Tribal College Journal, 9 a forum for sharing research and ideas among tribal colleges was created (Tribal College Journal, 2005). In order to engage in research to advance Indigenous knowledge, however, in addition to adequate funding, tribal colleges can create their own research agendas and define Indigenous scholarship for themselves (Kirkness & Barnhardt, 1991). A s they progress in this direction, tribal colleges hope to be leading centers of Indigenous research and scholarship that benefit their tribal communities socially, economically, and culturally (Stein, 2003). Funding Undoubtedly, the biggest challenge facing tribal colleges today is the lack of operational funding. Tribal colleges are possibly the "most underfunded institutions of higher education in the United States" (Gipp, 2003, p. xiv). Some of their financial challenges stem from the fact that tribal colleges are generally located in poor communities, have low tuition fees, and do not have wealthy alumni on whom to rely (Carnegie, 1997). Furthermore, it is costly for tribal colleges to deliver education to students who are under prepared for college. Under these conditions, sources of tribal college funding are limited to tribal funds, grants, state or local funds, private and corporate donations, philanthropic organizations (e.g., the Kellogg Foundation and Ford Foundation), and federal appropriation (Carnegie, 1997; Stein et al., 2003). Federal money provides subsistence existence to most tribal colleges through the Tribally Controlled Community College Assistance Ac t of 1978, which is administered by the Bureau of Indian Affairs (Carnegie, 1997). Although tribal colleges rely heavily on federal 9 Initially, it was called the Tribal College Journal of American Indian Higher Education. 56 funds for core operational funding, the federal government has not, for years, provided them with the amount authorized. For example, during the 1980s, Congress appropriated $6,000 per Native American student but only provided tribal colleges with $1,900 of that amount per student (Barbara Wright & Tierney, 1991). During the 2004-2005 academic year, Congress authorized $6,000 per Native American student and provided tribal colleges with $4,447 per Native American student (Goetz, 2005). Most commonly, no amount is provided for students who are not Native American or who do not meet the eligibility requirements of the act; overall, 15 to 20% of students attending tribal colleges are not Native American (Carnegie, 1997; Fann, 2002; Ortiz & HeavyRunner, 2003). A s a result of the inadequacy of federal support, some tribal colleges have come to rely on grant funding in order to provide for their fundamental needs. L ike many universities that compromise their autonomy through corporate sponsorship and donations, some tribal colleges struggle to pursue their goals in the face of requirements that are frequently determined by the grants they secure (Axelrod, 2002). Thus, the more sources of funding a college has, the greater its institutional autonomy (Trow, 1993). Tribal colleges are in a unique position with regard to funding. Although state funding and property taxation are the main sources of support for community colleges, these sources are not available to tribal colleges because state governments are not mandated to provide them with funding and property taxes are not collected on trust property on reservations (Fann, 2002; Stein et al., 2003). This remains a significant problem for tribal colleges as they continue to struggle with a lack of resources (Shanley, 2003). Tribal colleges have an additional financial challenge. They are often located in remote areas with small population bases. Class sizes are, therefore, small, and it is more costly to 57 deliver programs to relatively few students. Moreover, because many of their students live in even more isolated areas, the colleges must use expensive innovative solutions, such as interactive television or online classes, to reach them. Tribal colleges must also use some of their limited resources to provide support services to these distance students (Barden, 2003). Because of the shortage of funds, tribal colleges typically have only "bare minimum" facilities — facilities that are frequently dilapidated, old, modular structures. Classroom space is usually limited and faculty share offices. Many tribal colleges also lack dining facilities, a gymnasium, student residences, childcare services, computers, or lab equipment (Carnegie, 1997). In addition to the inadequate facilities, low per-student expenditures, lack of comprehensive student services, low staff and faculty salaries, under-funded libraries, and generally inadequate budgets continue to plague tribal colleges (Carnegie, 1997; Bobby Wright & Weasel Head, 1990). Upgrades in the facilities and services offered to tribal college students would certainly contribute significantly to their college experience and to learning. There is a great need for tribal colleges to obtain permanent, stable funding — and the federal government has the responsibility to provide this funding for the tribes with whom it has treaties (Stein et al., 2003). A s a result, the Carnegie Foundation, in its 1997 report, recommended that the government begin to honor and provide the full amount it appropriates to each tribal college student annually. This would be a significant step forward for tribal colleges in their search for stable funding. Conclusion Despite the lack of basic funding, of agreement about how to fulfill their "Indigenous" mandates, and of student preparation, tribal colleges have had a significant impact on Native American communities. In fact, more Native American students than ever are transferring to 58 four-year institutions from tribal colleges (Ortiz & HeavyRunner, 2003). This is significant because, as Ast in (1985) points out, there is sufficient evidence that completing a bachelor's degree benefits the individual significantly more than completing an associate's degree. B y assisting Native American students "who have not enjoyed full access to equal educational opportunity" (Boyer, 2003, p. 127) to transfer successfully to four-year institutions, tribal colleges are developing cultural, social, and academic capital among their students (Andres, 1994; Bourdieu, 1986; Ortiz & HeavyRunner, 2003; Tierney, 1999). In order to overcome the challenges Native Americans have faced in the last 450 years, tribal colleges continuously experiment with better ways to preserve culture and to base their curricula on Indigenous knowledge. This commitment is consistent with Castellano's (2000) hope that one day "Aboriginal knowledge w i l l resume its place as the basis of decision making and social order" (p. 34) in Native American communities. Tribal colleges are trying to support this process and to advance the processes of decolonization and self-determination ( A I H E C , 2004; Benham, 2003; Carney, 1999; Crazy B u l l , 2004; Marker, 2000; R. Nichols & Monette, 2003; Patterson, 1999; Shanley, 2003; Tippeconnic & McKinney , 2003). The Carnegie Foundation (1997) asserts: "Without question, the most significant development in American Indian communities since Wor ld War II was the creation of tribally controlled colleges" (p. 1). "Tribal colleges," they state, "are crucial to the future of Native American societies, and of our nation" (p. 4). 59 C H A P T E R 3 A S S E S S M E N T I N T H E U N I T E D S T A T E S — O V E R V I E W A N D C R I T I Q U E Assessment in higher education has a dual purpose: first, to improve student learning and performance (formative purpose), and second, to demonstrate to external accreditation bodies that the relationships between the institution's mission and learning outcomes (summative purpose) are evident (Trimble et al., 2000). Conventional assessment programs have tended to collect basic data such as student retention, graduation, transfer, and employment rates, success after transfer rates, and employer and graduate satisfaction, in order to guide and promote program development, long-range planning, external evaluations, and fund raising — all largely data used for summative purposes (Frye, 1999; Trimble et al., 2000); however, progressive assessment programs are emerging that are faculty- and student-driven, action-oriented, focused on mission-related performance and innovations, and attentive to student learning outcomes (Mentkowski, Rogers, Doherty, & Loacker, 2000). The first section of this chapter presents an overview of the nature and purpose of assessment. It also reviews numerous criticisms of assessment and suggests strategies for addressing them. The second section examines mainstream approaches and responses to assessment, reviews the types of data that can be used for assessment purposes, and discusses issues specific to two-year colleges. Some of the more innovative approaches to assessment, which foster institutional learning, are then presented. Finally, basic planning and reporting aspects of the assessment initiative, which are essential for closing the loop and improving learning, are outlined. 60 W h a t i s A s s e s s m e n t ? In this doctoral thesis, I use the term assessment to refer to "the systematic collection o f information about student learning, using the time, knowledge, expertise, and resources available in order to inform decisions about how to improve learning" (Walvoord, 2004, p. 2). A s an evolving concept in education, what I am calling assessment is sometimes referred to by different terms in the assessment literature. Terms such as student outcomes assessment, educational outcomes assessment, institutional assessment, and institutional effectiveness are all used in relatively interchangeable ways by different authors (Banta, 2002). I have chosen to simply use the term assessment in part for readability and in part because each of the other terms has self-limiting connotations that do not embrace the fullest sense of the concept as it is evolving in the literature and as I w i l l be using it. For instance, the term student outcomes assessment tends to connote individual student evaluation and obscure the importance of assessing the educational systems that facilitate student learning, while the terms institutional assessment and institutional effectiveness tend to take the focus off student learning and are easily confused with simplistic accountability measures such as job placement rates, balanced budgets, and resource allocation. Assessment can be thought of as action research, a scholarly endeavour used "to inform local action" (Walvoord, 2004, p. 2). The focus of assessment today, even for accreditation bodies, is on a formative outcomes-oriented process that can be used to better understand students and for "institutional self-study, financial retrenchment, [and] program evaluation" (American Association of Higher Education, 2004, p. 1). Angelo (1995) further elaborates on this definition: Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically.gathering, analyzing, and interpreting 61 - evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance. When it is embedded effectively within larger institutional systems, assessment can help us focus our collective attention, examine our assumptions, and create a shared academic culture dedicated to assuring and improving the quality of higher education, (p.7) Assessment, as it is being used here, is not about evaluating individual student performance (Seybert, 2003); rather, it is about evaluating the overall achievement of a group of students in order to provide feedback to students, faculty, parents, the college, policy makers, and the public about the current effectiveness and future refinement of educational programs (Pellegrino, Chudowsky, & Glaser, 2001). The American Association of Higher Education ( A A H E ) (2004) points out that Angelo's definition of assessment (above) is based on the following feedback that he received from experts in the assessment field: that the primary purpose of assessment is to improve student learning; that effective assessment efforts are not restricted to the classroom, but include all the institutional processes that affect learning; that assessment is rooted in a process embedded within a larger organic structure; that effective assessment programs result in a coherent, linked, and focused curriculum; and that the tension between the two purposes of assessment — that is, improvement and accountability — must, and can, be resolved. Assessment programs identify college, program, and course outcomes that derive from the institution's mission, and they compare these intended outcomes with the actual results achieved (Seybert, 2003). Outcomes and assessment can thus be divided into three levels: college outcomes and assessment (i.e., across the entire campus environment); program outcomes and assessment (i.e., within specific programs and departments); and, course outcomes and assessment (i.e., in specific courses and by particular instructors) (Maki , 2004). 62 Why do Assessment? The overarching purpose of assessment in higher education, as outlined above, is twofold: improvement and accountability (Trimble et al., 2000). A s Farmer points out, "excellence in education does not occur accidentally" (quoted in Green & Castelli, 2002, p. 7). Assessment, at its best, is a process through which excellence can be systematically cultivated in all aspects of higher education. Accountability The initial impetus for assessment, however, was narrower in its focus and largely derived from the second purpose identified above, accountability. Accountability refers to summative evaluations reported to the community, funding sources, and accreditation bodies about the use of educational resources (e.g., human, financial) in order to make decisions ( A A H E , 2004). Though assessment programs can address accountability concerns, they are broader in their scope and purpose. A s public stakeholders, legislators, accreditation bodies, funding agencies, parents, and students began scrutinizing higher education in recent decades, they began demanding more financial efficiency and accountability, along with a more competency-based approach to education, including a change in focus from instructor-centered teaching to student-centered learning. Pressure has also been exerted from some sectors to shift from a liberal arts focus to workforce training, resulting in less distinction between education and training. Due to concerns about the quality of higher education, colleges have been increasingly pressed to demonstrate value-added benefits for individual students as well as for surrounding communities and potential employers (Banta, 1999; D i l l , 2003; Lieberman, 2005). Other pressures forcing the unprecedented rise in assessment and accountability include increasing competition among colleges, credential inflation, and the trend toward corporate-like 63 organizational models (Atkinson, 2004; Gaither, Nedwek, & Neal, 1995). The advent of new technologies has impacted assessment significantly; thanks to effortless modes of communication, it is easier now for stakeholders to voice their complaints about higher education. Expectations for information are also higher as it is simpler now to collect, analyze, and report data (Knight, 2003). N o w that higher education is accessible to the masses, and is no longer only a luxury of the elite, there is greater demand for accountability and for identifying inefficiencies in the management of institutions in order to reduce public spending, increase the meaningful use and management of resources, improve planning and policy functions, and ensure academic quality (Di l l , 2003). A s a result, in the United States, formal processes of accreditation have become the norm and serve the following needs: Regional accreditation of postsecondary institutions is a voluntary, non-governmental, self-regulatory process of quality assurance and institutional improvement. It recognizes higher education institutions for performance, integrity, and quality to merit the confidence of the educational community and the public. Accreditation or preaccreditation by a postsecondary regional accrediting agency qualifies institutions and enrolled students for access to federal funds to support teaching, research, and student financial aid. (Northwest Commission on Colleges and Universities, 2005) For two-year tribal colleges, the importance of accreditation cannot be overstated because the acceptance of their students' transfer credits by four-year colleges depends upon it (Knight, 2003). In the United States, six accreditation bodies are responsible for determining whether a college has a suitable mission and goals, whether it has the appropriate resources to reach those goals, and whether those resources are being used in the most effective ways. The review process involves a peer review by experts from other colleges, a self-study, and a review of the assessment program that evaluates articulated performance outcomes (Di l l , 2003). The entire 64 process is meant to encourage continued and ongoing self-study and improvement, to impart evaluation skills to participants, and to institutionalize assessment (King, 2004). The Evolution of Assessment Assessment is not a new phenomenon; faculty have been doing assessment for centuries (Walvoord & Anderson, 1998). Traditionally, assessment was confined to the classroom, and, over the last several decades, the assessment movement has evolved substantially. In the 1960s and 1970s, assessment programs began by trying to measure institutional effectiveness through readily available quantitative data such as retention and graduation rates, time to degree completion, enrollment capacities, transfer and employment rates, and performance on standardized tests. B y the 1980s, however, educators recognized that most of these measures did not indicate whether student learning was actually occurring nor did they probe the nature and quality of that learning. Hence, educators sought more sophisticated assessment indicators, often qualitative in nature, to assess actual student learning. This required a more integrated approach to education and assessment, with greater potential (or, some would say, risk) for educational reform (Stark & Lattuca, 1997). Moreover, it became clearer that accreditation bodies were not interested in the actual findings of assessment, but rather were interested in determining whether or not the college had the processes in place to collect assessment data and to use them to improve programs, services, and curricula (Walvoord, 2004). Colleges were now being called upon to collect information in a more meaningful format and to use the results to make more informed decisions (King, 2004). 65 Improvement of Student Learning Assessment advocates assert that when assessment is done well , it emphasizes evaluation and reflection (involving students, faculty, staff, administrators, and community members) at the college, program, and course levels and results in a continuous and incremental improvement of learning and teaching. Therefore, though external accountability pressures created the initial need for evidence that institutions of higher education were accomplishing their goals, progressive colleges have since been working to channel these pressures into meaningful and constructive assessment programs by defining the assessment process in their own terms (Mentkowski et al., 2000). Thus, assessment programs can be used to stimulate improvements in learning and teaching, while simultaneously demonstrating accountability commitments to the wider community (Banta, 1999). Furthermore, in the absence of the external accountability pressures that public institutions are increasingly facing, some private colleges have pioneered strong and innovative assessment programs precisely because they provide an effective means for pursuing educational excellence (Farmer, 1988; Mentkowski et al., 2000). A s these colleges demonstrate, when assessment is done well , it can improve student learning, clarify and strengthen the mission of a college, improve program quality and performance, inform planning and decision making, support requests for funding, and assist in meeting and exceeding accreditation requirements (Bresciani, 2002; Diaz-Lefebvre, 2003; Green & Castelli, 2002; National Forum on Assessment, 1995; Pellegrino et a l , 2001; Seybert, 2003; The State University of New York Geneseo, 2004). C r i t i c i s m s a n d C o n c e r n s a b o u t A s s e s s m e n t Institutional change does not come without resistance. This resistance is rooted in a range of concerns, including concerns about the relationship between assessment, power, and 66 administrative control; concerns that assessment programs threaten faculty independence; and concerns about the validity and purpose of measuring higher learning outcomes. In the discussion that follows, frequently cited criticisms of assessment are discussed and strategies for addressing them are suggested. Power Concerns Critical planning theorists express concerns that colleges are institutions of power that selectively shape information, communication, participation, impressions, and attention to issues (Forester, 1989). They emphasize that information is a source of power in several ways: It responds to the social needs of the institution; it provides a way for groups with fewer means to participate in the planning process; and it assists those who already have power in the system to retain it (Forester, 1989). Assessment programs require technical expertise and a central location for data and information at colleges (King, 2004). Thus, assessment programs can extend the power of administrators by extending their possession and control of information, including their control over who has access to what information and when (Forester, 1989). Assessment coordinators may determine what information is relevant and timely for decision-making purposes, may shape participation in relevant decision-making processes, and may influence these processes by interpreting assessment data (Forester, 1989). In all o f these ways, assessment programs can advance the interests, agendas, and perspectives of administrators. A t the same time, such programs can exclude the interests, agendas, and perspectives of those who are in less privileged positions — including faculty, staff, and even students. 67 A s a result of power dynamics, some faculty are concerned that assessment protocols w i l l become rigid and w i l l be prescribed in a top-down manner and that assessment data w i l l be used inappropriately by policy makers or administrators to rationalize administrative decisions that faculty may not support (Banta, 2002; Coffman, 2004). These concerns are well founded. How administrators draw attention to assessment issues and information can have significant political consequences that may include impacts on professional advancement, on workload pressures and expectations, and on resource allocations. Furthermore, administrators are themselves accountable to, or under pressure from, a range of external entities, from accreditation bodies to legislators to powerful lobbies and special interest groups. In this context, assessment programs can become instruments of socio-political control that powerful external agencies and groups use to reshape institutional missions and priorities and to redirect scarce resources according to their own interests and agendas. Alternative Perspective Though many assessment programs were initiated through the demands of external entities, such as legislators and accreditation bodies, assessment practice appears to be shifting toward an internal locus of control with a focus on internal audiences and maximum relevancy to faculty, staff, and students (Coffman, 2004). Although assessment can serve as an instrument of external control, the best way to ensure an internal locus o f control is for faculty and administrators to embrace assessment and take the lead in defining assessment measures and processes according to their internal values and priorities — rather than waiting for external agencies to define them according to the political pressures of the day. Moreover, within institutions, the way administrators present information and interact with instructors can either result in indifference, suspicion, and rejection, or it can build 68 understanding, trust, and consent (King, 2004). The manner in which power is exercised is the key. Forester (1993) emphasizes that in order for planning practices to be positive and socially transformative, they must be "sensitive to the continual reproduction of relations of knowledge, power, trust, and attention" (p. 103). Individuals in positions of power can strive to present information in an open, unbiased, and user-friendly manner; commit to using the results of the assessment program; create opportunities for interdependence to enhance collaboration and to build trust; anticipate conflicts, obstacles, and concerns; develop keen listening and speaking abilities; cultivate a network of meaningful relationships; and respond in sincere, respectful, cooperative, and nurturing ways (Forester, 1989; K i n g , 2004). A collaborative response enhances a sense of collective identity and purpose, an approach that results in better solutions and has the potential to transform working relationships (Forester, 1989). Meaningful participation in the process, whereby faculty's perspectives are validated, increases commitment to the process and, thus, chances for the college's success. In order to do this, it is essential that adniinistrators relinquish some of their power and collaborate more with faculty (Coffman, 2004). Issues of power w i l l always remain, but, i f administrators sincerely listen to faculty and staff, this power can be used to transform educational institutions into models of decision making built on shared values and interests (Forester, 1989; K i n g , 2004). Thus, the individuals involved have ethical responsibilities to not silence voices and to understand how their presentation of analyses influences power relations within the organization, so that they can empower the faculty, students, administrators, and staff in positive ways (Forester, 1989). 69 Academic Freedom and Evaluation Concerns Some faculty have raised concerns that assessment findings may be used to evaluate and reprimand them or their students. A related concern is that assessment processes w i l l increase external control over curriculum and testing. Within the new assessment regime, critics assert, some faculty w i l l try to anticipate what external bodies are looking for and adapt their teaching to what they believe w i l l be measured. This kind of behaviour may discourage instructional creativity and risk-taking as well as increase competition among programs (Ewell , 2002; Fisher, Rubenson, & Rockwell , 2000; Lieberman, 2005; Rodrigues, 2002). Alternative Perspective Assessment proponents point out that assessment processes do not give administrators the authority to advise faculty on how to teach (Lieberman, 2005). Instead, the goal of assessing student learning is to assist faculty, as a group, to improve the learning process; the data gathered under these conditions can be used to improve curricula, programs, and instruction from the bottom up rather than the top down (Walvoord, 2004). In an effective assessment program, the emphasis is on the entire group of instructors and when evidence of poor student learning in a particular class emerges, it can and should be addressed with a supportive and encouraging response from the department or college. Moreover, in a two-year college context where teaching is the primary responsibility of instructors, proponents believe that assessment makes instructors more accountable and that some level of accountability is healthy, because it can result in more meaningful learning experiences for students (Walvoord, 2004). Because assessment programs use aggregate rather than individual data about learning, and do not 70 associate students' names with their scores, fears about student privacy and evaluation are unfounded (Lieberman, 2005). Finally, because accreditation bodies' focus is now on individual college-generated outcomes (ideally determined by the faculty), the risks of discouraging instructional creativity and "teaching to the test" are diminished. Resources Concerns Many faculty also assert that the time, money, and effort required to learn about and implement complicated assessment processes could more profitably be directed toward supporting conventional teaching, learning, and service activities (Fisher, Rubenson, & Rockwell , 2000; Rodrigues, 2002; Walvoord, 2004; Zorzi , McGuire , & Perrin, 2002). In addition, they argue that faculty already grade students and this should be an adequate form of evaluation 1 0 (Rodrigues, 2002). Finally, because faculty often receive no compensation or reward for learning about and implementing assessment processes, many are not motivated to devote the extra time necessary to carry out these added responsibilities. Alternative Perspective These concerns w i l l be addressed from two perspectives: those of administrators and faculty, both of whom are essential to the implementation of a successful assessment program. In both cases, a transformational change is needed: administrators "toward a culture that more Refer to the section titled Grades in chapter 3 for a discussion about grading and its relationship to assessment. 71 freely embodies the principles of a learning organization" (Banta, 2002, p. 284) and faculty toward a culture of active responsibility for documenting and improving student learning. Assessment scholars agree that administrators must provide visible advocacy as well as real material support for assessment i f it is to succeed. The value they place on it can be shown by their taking an active role and interest in assessment; collaborating with faculty, staff, students, and others involved in the process; allocating financial, technical, and human resources to assessment; and referring regularly to the assessment process and its results in reports and presentations to both internal and external audiences (Center for the Study of Higher Education, 2002). In an effective assessment climate, administrators only hold individuals and programs accountable for what they have control over (i.e., monitoring improvement rather than comparing performance), and they use the results to improve the collective benefit o f the programs, rather than to encourage competition between programs (Ewell , 2002). They recognize and value feedback, ideas, and input from staff and faculty, and they incorporate these suggestions into their plans and decisions. Furthermore, time-consuming assessment processes and meetings are streamlined and simplified, and administrative obstacles are reduced so that instructors' time may be used wisely (Rodrigues, 2002). For example, assessment processes should only be designed to provide information the college can use; not to collect data for which there is no identified use (Banta, 1999; Gray & Goodman, 2003; J. Nichols, 2002; Seybert, 2003; Walvoord, 2004). To facilitate the assessment process, administrators may also need an assessment coordinator who has the responsibility to support and coordinate institution-wide assessment activities; provide expertise and consulting for all educational programs and administrative 72 departments; collect data in an efficient manner; and provide a centralized location for the storage, analysis, and dissemination of assessment results. In addition, administrators also need a process for evaluating the effectiveness, including the cost-effectiveness, of the assessment program itself (J. Nichols, 2002). A s an increasing number of colleges attempt to implement assessment programs (El -Khawas, 1995), faculty often emerge as the main source of opposition for the reasons discussed above as well as others (Cohen, 1994). For instance, traditionally, faculty have focused on the process, not the product, o f teaching. Assessment is a cooperative process that requires agreement about mission, program goals, and learning outcomes; methods and standards of assessment; interpretation of results; and use of information to improve the learning experiences for students. Many faculty, however, are used to teaching in relative isolation and are not wel l -prepared for this type of collaborative approach (Banta, 2002). In addition, developing assessment programs takes time and energy, and many faculty already feel overburdened. A s a result, faculty often prefer to let administrators deal with assessment (Walvoord, 2004). Yet, proponents of assessment agree that a successful assessment program needs to be driven by strong faculty leadership, implying primarily that faculty participate fully in the design of learning outcomes (Seybert, 2003). To be effective, faculty can take ownership of assessment and embrace it as an intrinsically valuable developmental process whereby teaching and learning can be continually improved through evaluation, reflection, and identification of needs for change. A sense of ownership, and hence cooperation, is enhanced when faculty and their colleagues themselves create meaningful assessment processes, rather than having them imposed by the administration (Rodrigues, 2002). This requires faculty being involved daily in 73 assessment processes that reflect what they value, such as their teaching, disciplines, and research (Rodrigues, 2002). Faculty ownership and participation result in a continuous process of improved learning and teaching. Toward this end, strategies to make the assessment process more streamlined include integrating assessment into the curriculum; embedding assessment in classroom assignments to serve both grading requirements and collective performance purposes through the use of rubrics (Ewell , 2002); determining in advance clear learning outcomes, high standards, and clear criteria for levels of achievement; agreeing upon the tools that w i l l be used consistently to measure success; pre-determining a process of using this information in meaningful ways to improve teaching and learning; communicating assessment results frequently; connecting "assessment, as a form of systematic inquiry, with the scholarship of teaching" (Banta, 2002, p. 288-289); and using the results to improve programs and student learning experiences. Finally, assessment scholars agree that it is wise to start the process with interested and supportive faculty; to provide them with the necessary opportunities, incentives, and material resources to learn about assessment; and to celebrate, credit, thank, and reward their efforts and achievements publicly through institutional processes and practices (e.g., with stipends, promotion, tenure, release time, etc.) (Banta, 1999, 2002; K i n g , 2004; J. Nichols, 2002). Measuring Higher Learning Concerns Faculty raise many legitimate concerns about the validity of measuring educational outcomes, as well as the socio-political consequences of the movement toward greater accountability. Most notably, many instructors think it is impossible to measure the important 74 goals of higher education and lament the skewed picture that results from too great a reliance on quantitative data (Rodrigues, 2002). They have a point. Too often, during the planning processes in which assessment approaches have been developed, meaningful outcomes and activities have been excluded in favour of more easily quantifiable measures. Not only is reliance on quantitative data easier, it is politically popular in a society that is preoccupied with measurement and testing (Banta, 2002; Fisher, Rubenson, & Rockwell , 2000). A s a result, many faculty argue, quantitative indicators have a disproportional influence relative to qualitative indicators that may be more difficult to measure, but more meaningful or important (A. Wilson & Cervero, 1997). This "technical rational" approach, based on scientific problem-solving, limits understanding of what is considered to be appropriate theory or practice and excludes certain approaches to education and problem-solving that are not quantitative (A. Wilson & Cervero, 1997). A s Axelrod (2002) states, "traditional teaching methods based mainly on memorization, frequent testing, and other quantitative assessments may satisfy the current craving for educational 'accountability,' but on their own they insufficiently cultivate the life of the mind" (p. 124). Alternative Perspective Though faculty voice concerns about measuring higher types of learning in a meaningful way, proponents of assessment point out that instructors are frequently called upon to use their professional judgment to evaluate the work of their colleagues, students, and themselves by criteria accepted in their fields. Therefore, it is not unreasonable to expect faculty to use the same kind of expert judgment to assess how their students' work compares to the higher expectations of the college (Ewell , 2002; Walvoord, 2004). Furthermore, faculty can transform the discourse about accountability by devising meaningful ways to demonstrate quality or 75 student performance and creating "processes that develop a sense of common purpose and shared accountability" (Gaither et al., 1995, p. 4). Outcomes discussions among faculty often initiate insightful and meaningful conversations and are " in service of understanding the discipline and how it might and should affect students. Ending with a potentially measurable outcome may be the least important result" (University of Washington, 2005, p. 1). In addition, because education is a fundamental contributor to collective social progress, the responsibility of faculty, and everyone else engaged in the educational enterprise, is to devote their talents and insights to the admittedly difficult, but nonetheless necessary, task of continually improving its quality. A s Ewel l (2002) points out, "large-scale assessment of higher-order collegiate abilities can be an important tool of social policy, i f it is directed toward determining gaps in the nation's store of 'educational capital' and filling them" and not used for "rating and governing institutional performance" (p. 14). Therefore, the discourse about the value of educational assessment "must recognize the role of liberal education in furthering the public good" (Fisher, Rubenson, & Rockwell , 2000, p. 24) and the importance of education in instilling social values and responsibility. Through their contributions, faculty can contribute to more creative and flexible assessment programs, incorporating qualitative as well as quantitative measures, which may result in what Wilson and Cervero (1997) refer to as "emancipatory learning" (p. 100) and in the emergence of alternative planning models based on the ideals of social justice and social transformation. 76 Other Measurement Issues Concerns Other related concerns faculty often raise about measurement are the following: First, assessment concentrates on tools of the trade, such as the current focus on rubrics, rather than on important questions (Ewell , 2002). Second, colleges frequently focus on target outcomes rather than the processes (Harris, 1998). Third, it is difficult to develop indicators and methods that capture meaningful data or to agree about definitions of outcomes and connect measurable outcomes to a single educational process (Cohen, 1994; J. Nichols, 2002). Fourth, quantitative measures can reduce the complexity of situations, are often inaccurate and inconsistent when compared among colleges, are burdensome, and are unable to provide meaningful information that allows colleges to draw useful conclusions (Fisher, Rubenson, & Rockwell , 2000). Put another way, Coffman (2004) characterizes quantitative measures as those that focus on the "logical versus the creative, the planned versus the emergent, the revolutionary versus the transformational, the strategic fit versus reaching with a stretch, and the pursuit of a strategy versus going after effectiveness" (p. 21). Alternative Perspective Though concerns about over-reliance on quantitative data are valid, approaches to assessment are evolving in a direction that makes these concerns less relevant. The socio-political discourse mentioned above has had a profound impact on the evolution of assessment. Banta (2000) summarizes how the current thinking developed. These concerns, she says, forced practitioners to sharpen the philosophical grounding of the movement — rooting it in the tenets of scholarship and the process of teaching and learning. It also reemphasized that the evidence used by assessment must always rest upon a peer-based 77 conrmunity of judgment (American Association for Higher Education, 1992; Mentkowski, Astin, Ewel l , & Moran, 1991). Finally, the debate forced explicit recognition of the fact that evidence is consistently constrained by the context in which it is generated (Mentkowski & Rogers, 1988) and by the uses to which it is put (Messick, 1988). Epistemological issues of this kind thus remain at the heart of the movement and remain healthily and vigorously contested (Ewell , 1989; Harris & Sansom, 2001). But protests based solely on principle or politics have steadily diminished, (p. 18) Thus, just as the focus of the debate has shifted, so too has the focus of assessment. Today, assessment tends to focus on the institution's goals and a determination of whether students are attaining those goals — rather than on how outcomes compare to other colleges. This position has resulted in assessment becoming a more meaningful endeavour. Fortunately, accreditation bodies are supporting this approach, replacing a focus on rigid data-driven reports with a focus on encouraging and expecting colleges to use their data, especially qualitative information, to improve curricula and programs (Banta, 2002). Local ly developed, authentic assessment approaches are now preferred (Fisher, Rubenson, & Rockwell , 2000; Stufflebeam, 2001). Alverno College, discussed later in this chapter, provides a leading example of the effort to apply authentic qualitative indicators to the learning process. Student Learning Concerns Another concern some faculty have cited is the lack of evidence that assessment improves learning (Ewell , 2002). A related concern is that the policies resulting from assessment w i l l not improve institutional climate. 78 Alternative Perspective It is true that assessment processes, as relatively recent developments in higher education, have not yet yielded much data to support the claim that they improve learning; however, assessment advocates would argue that the only way to determine whether such evidence w i l l be forthcoming is to experiment with assessment approaches and monitor the results. When properly conducted, assessment may well be a valuable tool for improving program quality and performance and, thus, enhancing students' learning. Furthermore, assessment scholars emphasize that, i f students are provided with numerous, varied, and meaningful opportunities to practice skills and receive feedback, the assessment process itself can improve learning (Mentkowski et al., 2000; Stufflebeam, 2001). A t Alverno College, for example, faculty assert that when assessment is done well , it results in learning that lasts (Mentkowski et al., 2000). For a more thorough discussion of this topic, refer to the Assessment as Learning section in chapter 4. Dual Purposes of Assessment Concern Another source of tension over assessment stems from its dual purposes: improvement, which is internally designed in order to improve an institution's own teaching and learning, and accountability, which is externally imposed (through state mandates, government, and accreditation bodies 1 1) to judge how well an institution is meeting the external goals of these agencies. These two purposes can be contradictory and incompatible (Ewell , 2002). 1 1 Faculty sometimes see university administrators, such as presidents and deans, as external. 79 Alternative Perspective Assessment scholars assert that this issue is resolvable. To begin with, it is necessary and useful to recognize the nature of the different purposes of formative (improvement) and summative (accountability) evaluation. Both types of evaluation are essential, but they must be dealt with as separate and distinct processes. Second, formative information can be extracted from the existing processes designed to acquire summative information. For example, a grade provides a summative evaluation of a student's work; on the other hand, an assessment rubric provides formative information that can lead to an immediate focused response and remediation to improve learning (Ewell , 2002). B y recognizing the tension between assessment for improvement and assessment for accountability, by dealing with them in separate processes, and by extracting formative information from summative processes, assessment processes may be streamlined and tensions between them resolved. Accreditation bodies are beginning to assist in the process of resolving this tension. In the last couple of years, they have shifted their focus from requiring prescribed data-driven reports (i.e., addressing external concerns) to focusing on the fulfillment of each college's mission and goals and how the college uses its assessment results to improve student learning (i.e., addressing internal concerns). Assessment is a Fad Concern Some faculty are still convinced that assessment is just a fad and w i l l disappear (Ewell , 2002). 80 Alternative Perspective Proponents of assessment insist that this is not the case and that the assessment movement w i l l continue to gain momentum (Ewell , 2002). This momentum can be detected in the assessment literature where the debate is no longer about the pros and cons o f assessment but about better means of accomplishing its purposes (Banta, 2002). Banta affirms, "just as assessment revitalized accreditation, accreditation's insistence on assessment has kept the assessment movement alive and thriving" (p. 252). Politicians, external stakeholders, the media, and the market are all obsessed with data and information (Walvoord & Anderson, 1998). Furthermore, Banta (2002) points out, "assessment has been a point of confluence for a raft of issues in education" (p. 252) (e.g., the use of technology as an instructional mode, outcomes-based curricula, writing-across-the-curriculum, learning communities, service learning, and problem-based learning), all of which must demonstrate their effectiveness. A l l o f these developments have served to shift educational focus from teaching to learning and to propel the assessment movement forward (Ewell , 2002). B e n e f i t s o f E f f e c t i v e A s s e s s m e n t P r o c e s s e s Despite the concerns and criticisms discussed above advocates believe that good assessment practice can be used to improve colleges and to meet the needs of accreditation bodies and other stakeholders. Assessment is an ever-evolving, dynamic process, that can be adapted to the needs of the college culture (Banta, 2002; M a k i , 2004; J. Nichols, 2002; University of Wisconsin, 2004). When done well , advocates insist, the benefits of assessment are numerous: It initiates meaningful conversations at all levels within a college; provides an opportunity for creating a shared vision for the future of the college, based on common values; redirects resources towards priorities outlined in the mission and goals; increases the college's 81 responsiveness to the needs of the coirirnunity; builds cohesion, collaboration, relationships, and trust among faculty, staff, and administrators; re-values teaching, service, and students; improves the instructional capacity of the college as well as its public image; increases students' confidence in the college and in themselves; provides the basis for college planning and budgeting decisions; provides financial or reputational rewards at the individual, department, or college-wide level; and demonstrates accountability, the responsible use of limited resources, to the public (Fisher, Rubenson, & Rockwell , 2000; Fisher, Rubenson, Rockwell , Grosjean, & Atkinson-Grosjean, 2000; Lieberman, 2005; Walvoord, 2004; Zorzi et al., 2002). Assessment data can be used to improve everything from curriculum and course content, pedagogy, assessment tools, internships, and faculty-student interactions, to facilities, course staffing and scheduling, class size, inclusion of students in faculty research, and student advising (Walvoord, 2004, p. 62). Indications of the assessment movement's advancement include the creation of teaching and learning centers and greater faculty interest in teaching and learning scholarship (e.g., the promotion of assessment in various disciplines' associations, publications of journal articles, and presentations at conferences) (Ewell, 2002). Based on an explicit understanding and analysis of the critiques, Banta's (2002) view reaffirms the significance of the assessment movement, which, she says, has "begun to build a whole new infrastructure for teaching and learning, for improvement and accountability.. .Building that infrastructure takes time. It progresses at a glacial pace. But it reshapes the whole landscape" (p. 257). M a i n s t r e a m A p p r o a c h e s t o A s s e s s m e n t Many assessment advocates refer to the following set of nine principles that they believe form the foundation of good practice for assessing student learning: 82 1. The assessment of student learning begins with educational values. 2. Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time. 3. Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes. 4. Assessment requires attention to outcomes and also to the experiences that lead to those outcomes. 5. Assessment works best when it is ongoing, not episodic. 6. Assessment fosters wider improvement when representatives from across the educational community are involved. 7. Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about. 8. Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change. 9. Through assessment, educators meet responsibilities to students and to the public. ( A A H E , 1992, p. 1) In varying degrees and in varying ways, colleges are attempting to integrate these principles into their assessment of student learning. In addition to agreement about integrating these principles, a consensus about the steps involved in the development o f an effective program of assessment appears to be emerging in the assessment literature (Banta, 1999; J. Nichols, 2002; Seybert, 2003; The State University of New Y o r k Geneseo, 2004; Walvoord, 2004). The first step is to either establish or update the college's mission statement and articulate college and program outcomes. Outcome statements describe "what students should be able to demonstrate, represent, or produce based on their learning histories" (Maki , 2004, p. 60). Next, indicators and standards are established to operationalize these outcomes, methodologies are selected for the data collection, evidence is gathered of how well students are attaining the outcomes, and the information is analyzed. 83 Finally, administrative and communication systems are designed to ensure the findings are actually incorporated into decision-making processes and used to refine policy and practice at the college, program, and course levels. T y p e s o f I n f o r m a t i o n U s e d f o r A s s e s s m e n t P u r p o s e s Effective assessment programs use three types of data —- often referred to as direct indicators, indirect indicators, and institutional data — for assessing success at the college, program, and course levels (Palomba & Banta, 1999). Whereas direct indicators require that students demonstrate their learning through, for example, essays, capstone projects, tests, and presentations, indirect indicators ask students to reflect on their learning through, for example, graduate or student satisfaction surveys, interviews, and focus groups. Institutional data, on the other hand, are institution-level measures that do not necessarily indicate student learning but do reflect the overall condition and effectiveness of the college and may include, for example, retention and graduation rates, student-faculty ratios, and enrollment trends (Ewell , 1997; Walvoord, 2004). It is worth noting that in the assessment literature, there is some disagreement and inconsistency regarding which specific indicators fall within each of these three broad categories (and even some disagreement regarding the utility of the categorization scheme itself). Direct Indicators It is widely accepted that certain teaching strategies — such as self-reflecting, practicing and repeating, applying concepts to a relevant context, teaching material to peers, writing about a subject, and asking essential questions — foster deep learning (Maki , 2005). It is also known that "what and how students learn depends to a major extent on how they think they w i l l be assessed" (Biggs, 1999, p. 141). 84 Traditional and Authentic Assessment Tools Direct methods of assessing student learning tend to fall into two general categories: traditional tests (e.g., exams) and authentic assessment tools. Both of these approaches attempt to assess the observable performance of students. Table 3.1 provides examples of both traditional tests and authentic assessment tools that can be used directly to assess student learning, at the college, program, or course levels (Cai State San Bernadino, 2004; Gipps & Stobart, 2003; Green & Castelli, 2002; Knight, 2003; M a k i , 2002; J. Nichols, 2002; Palomba & Banta, 1999; Seybert, 2003; The State University of New York Geneseo, 2004). 85 Table 3.1 Examples of Direct Methods for Assessing Student Learning Traditional tests Certification exams Cognitive assessment tests College competency tests Comprehensive tests / exams Critical thinking tests Entrance tests General knowledge tests Graduate entrance exams (e.g., G R E ) Licensure exams National exams Oral communication tests Placement tests for entering students (e.g., math, writing or reading) Pre- and post-tests for attitudes and mastery of knowledge Professional exams Quantitative problem solving tests Standardized tests Writing proficiency and reading competency tests Authentic assessment tools Apprenticeships Capstone projects or experiences Case studies Competency-based curricula Essays (written projects, pre and post) Internships (externally reviewed) Interviews Job performance Journals Juried reviews Observations of student behavior or skills Oral presentations and exams Peer evaluation Performance-based mastery tests Portfolio system Presentations Projects (e.g., abstracts, advertisements, brochures, budget with rationale, research) Reflection logs Self-assessment Simulation exercises Thesis / major project / doctoral thesis Video and audio tape evaluations (pre/post) Workplace competency Traditional tests may be standardized or locally designed and may be of different types (e.g., multiple choice, fill-in-the-blank, short answer, true or false, etc.). Although they may or may not reflect a student's ability to truly understand or apply the concepts and material, some 86 scholars believe that traditional tests provide information about cognitive outcomes (Palomba & Banta, 1999). Authentic assessment tools, on the other hand, may provide students with the opportunity to demonstrate behaviours and performance in more "real-life" and applied situations. These authentic assessment tools ideally measure the knowledge and abilities expected by professionals in the students' respective fields (Wiggins, 1998). The main disadvantages with using authentic assessment tools is that they require a large investment in resources and in time setting up the systems; they take more time and effort to score; they may not be reliable; they often lack comparable norms; and they usually cover a narrow range of skills (Stufflebeam, 2001). Grades The relationship between grades and assessment is not as straightforward as it might first appear. Typically, completion of a course, or receipt of a grade, does not in itself provide evidence of learning or meeting a learning outcome (Strain, 2003). The fact that a student obtains an A in a course does not mean that he or she has gained understanding or knowledge as a result of the course. For instance, the student may have entered the course with the understanding or knowledge required to earn an ,4. Grades do not necessarily measure how much students know, value, or are able to do; how much they have learned due to a course; or, i f learning has occurred, what elements impacted their learning most (Center for the Study of Higher Education, 2002). Furthermore, because the way grades are determined can vary greatly among instructors and colleges, they cannot be used for comparison purposes. Nonetheless, in a tribal college setting, there are two circumstances when it is generally acceptable to use the actual course grades for assessment purposes. The first is when students 87 move from developmental to college-level classes. The second is when students transfer from a two-year tribal college to a four-year university. In both these instances, grades may be used to assess successful transfer from one level to the next (J. Nichols, 2002; Walvoord, 2004). B y comparing a student's performance at the next level with others at that level, the tribal college can get a sense of how well prepared their students are and how effective their grading system is. A n effective grading process can serve several purposes. It can enhance evaluation of the student, can extend communication between the instructor and student, can motivate students to learn, can organize course concepts, and can be used to improve teaching (Walvoord & Anderson, 1998). Grading, however, is a complex process, is never totally objective, and can greatly affect student learning, both positively and negatively. It is a "socially constructed, context-dependent process" which, at its best, "can be a powerful tool for learning" (Walvoord & Anderson, 1998, p. 10). If the grading process is approached using rubrics (e.g., see Table 3.2), where criteria are made explicit, are evaluated using a scale, and are used to foster improvement, then "the grading process is an excellent basis for direct assessment of learning" (Walvoord, 2004, p. 15). Rubrics articulate criteria that instructors use to evaluate a specific assignment, and these results can be used to communicate and compare aggregate results (Walvoord, 2004). A s a result, instructors can determine specifically where students are having the most difficulty and whether, after adjusting teaching strategies, students improve on a specific criterion over quarters. Moreover, in terms of assessment, rubrics make grading public and analyzable (Walvoord & Anderson, 1998). 88 Table 3.2 A Section of the N W I C Rubric for Evaluating Writing Skills (Draft) Level of Proficiency Criterion Beginning (1) Developing (2) | Accomplished (3) Exemplary (4) Outcome a: The student writes standard English 1. Idea and content Writes with unclear purpose or central theme. Does not clearly define or support position on topic. Uses limited or disconnected details that disrupt the unity of the paper. Partially focuses on topic with minimal or no support of position. Writing is basic, too general for the reader to develop a clear understanding. Maintains clear focus throughout the paper with sufficient . appropriate details indicating awareness, knowledge, and insight. Writes clearly and with focus; relevant details support the central theme. 2. Organization / structure Writes with organization that is unclear or inappropriate to the thesis; lacks transitions between ideas. Writes with some signs of logical organization; may include abrupt or illogical shifts and ineffective flow of ideas. Makes few transitions between ideas. Supports thesis and purpose through organization and paragraphing; most transitions are appropriate, but sequence of ideas may need improvement. Reiterates introductory elements in conclusion. Provides clear introduction and reinforcing conclusion. Orders writing logically with effective transitions, providing sufficient information in the appropriate places. 3. Word choice Chooses nonspecific or distracting words that limit meaning. May include slang and colloquialisms. Chooses ordinary words using adequate verbs, nouns, adjectives, and phrases. Chooses correct words that result in clarity. Chooses interesting, specific and accurate words that contribute to communicating the writer's purpose. 4. Writing conventions: Grammar / spelling / punctuation Writes with a minimal grasp of standard writing conventions; numerous errors impair readability. Writes with a basic grasp of the standard writing conventions; occasional errors may impair readability. Writes with a good grasp of standard writing conventions: capitalization is proper; punctuation is smooth and enhances meaning; spelling and grammar are essentially correct. Writes with a strong grasp of the standard writing conventions; all conventions are properly applied. 5. Presentation / formatting Produces writing that looks untidy and does not follow basic formatting rules (e.g. margins, headers and subheaders). Produces writing that looks fairly neat but violates some formatting rules. Produces writing that looks neat but violates one or two formatting rules. Produces clean, neat, and easily read document in which the form and presentation of the text enhance the written message. 89 In order to use the grading process for the direct assessment of learning, Walvoord (2004) advises instructors to ensure that the assessment tool actually measures the learning outcomes. It must state explicitly in writing the criteria for evaluating student work in sufficient detail to identify students' strengths and weaknesses; develop systematic ways of feeding information about student strengths and weaknesses back to decision makers.. .and using that information for programmatic improvement, (p. 15) Using the grading process for the direct assessment of learning can be accomplished in a number of ways. For example, in a supportive faculty meeting environment, instructors can present their students' assignments, the written criteria (e.g., rubric) (see Table 3.2), and the class's aggregate scores compared to previous quarters (see Table 3.3) for feedback. Faculty can then make recommendations for improving student learning, and someone can take minutes of the meeting to record the exchange. The instructional changes that result from these sessions can later be documented. Table 3.3 Mean Aggregate Scores from a Native American History Class Assignment Assessing the N W I C Outcome "Writing Standard English" Quarter Criterion Fal l 2004 Winter 2005 Spring 2005 Fall 2005 Winter 2006 1. Ideas and content 4.0 3.5 3.7 3.5 4.0 2. Organization / structure 2.0 2.5 3.0 3.2 3.5 3. Word choice 3.0 3.5 3.5 3.7 3.9 4. Writing conventions: Grammar, spelling, punctuation 3.7 4.0 4.0 4.2 4.4 5. Presentation / formatting 3.0 3.0 3.2 3.4 3.5 Note. Adapted from Walvoord (2004). 90 Table 3.3 illustrates how the grading process may be used for assessment purposes. In this example, instructors assessed the N W I C outcome writing standard English in this course over a number of quarters. The second criterion, organization and structure, had the lowest score in the fall quarter of 2004. The instructors focused more of their attention on this criterion during the next few quarters, and results indicate modest improvement each quarter. O f course, grades also serve a purpose in evaluating individual student performance in a course because they summarize the level of the student's achievement of a set of outcomes. Grades inform students about how well they perform in a class compared to other students, yet without clear and detailed rubrics they do not indicate in which areas the student is strong or weak (Palomba & Banta, 1999). For instance, in Table 3.4, although all four students varied significantly on their final grades, all were successful in demonstrating criterion 3 (word choice). Table 3.4 Relationship between Individual Student Grading and Assessment Student Student Student Student Average of Criterion A B C D criterion 1. Ideas and content 5 3 4 3 3.75 2. Organization / structure 5 2 2 3 3 3. Word choice 5 5 5 5 5 4. Writing conventions: Grammar, spelling, punctuation 5 3 4 4 4 5. Presentation / formatting 4 2 3 3 3 Total 24 15 18 18 Grade A C B B Note. Adapted from J. Nichols and K . Nichols (2000, p. 43). 91 Therefore, i f we want to understand student achievement of a certain learning outcome, we can gather information specific to that learning outcome, repeatedly, over time (J. Nichols & Nichols, 2000). Some scholars think grades may be inappropriate in Native American institutions, as grades tend to single out individuals from the group (Fixico, 2000). Wi th the strong emphasis in Native American culture on cooperation, rather than competition, and the unity of the community and group, it may be important for tribal colleges to develop "learning-oriented" rather than "grade-oriented" students (Walvoord & Anderson, 1998). Nonetheless, because tribal colleges must respond to the demands of the universities to which their students transfer, it is important for tribal colleges to determine ways to use the grading system to their advantage. Classroom Assessment Techniques Some direct methods for assessing student learning are actually not graded. For instance, short, frequent, ungraded attempts to assess student learning, referred to as classroom assessment techniques provide immediate in-class feedback from students about which concepts are clear and which are not (Angelo & Cross, 1993). One example of a classroom assessment technique is called the "muddiest point," where the instructor asks the students at the end of a class to write down, in one minute, the concept that was least clear to them during the class. The instructor gathers the sheets, reviews the responses, and then responds to them the following class (Angelo & Cross, 1993). These formative assessment techniques provide instructors with the opportunity to make adjustments to courses mid-stream (or even mid-class) and to build trust and relationships with their students (Lieberman, 2005). The use of other formative assessment techniques, such as self-assessment and peer reviews, are also useful and tend to be intrinsically rewarding for 92 students. Although it is difficult to tie these efforts formally to program assessment, assessment scholars are convinced that employing these strategies improves student retention and engagement and provides instructors with feedback as to what students are or are not learning (Banta, 2004; Walvoord & Anderson, 1998). Curriculum Map A n effective tool for determining where each of the college outcomes is being assessed is the curriculum map. B y listing all program or college course requirements, this simple matrix summarizes the level to which students are expected to master each of the outcomes in each of the required courses (see Table 3.5). Table 3.5 Example of a Section of a Curriculum Map College outcome 1. 2. Written 3. Oral 5. Quant. 6. Reading Cultural comm. comm. 4. Technology skills skills skills Required course a b a b a b a b c d e f a b a b E N G L 101 B B B B B B M A T H 102 B B B B B B I O L 101 B B B B P O L S 101 B B B HIST 201 I I I I Note. Adapted from Walvoord (2004). The level to which each goal is assessed in each class is identified: B = beginner, I = intermediate, A = advanced. 93 The curriculum map provides an efficient and useful way to identify gaps in the program where outcomes may be neglected. For example, the abbreviated curriculum map in Table 3.5 indicates that Reading skill 6b — extending students' vocabulary through reading — is not being assessed in this curriculum. A draft list o f N W I C outcomes is posted in chapter 6 (College Outcomes). Indirect Indicators The second category of data used for assessing student success is indirect indicators. Indirect indicators provide information about students' perceptions of their learning experiences. Examples of indirect indicators of student learning are listed in Table 3.6, according to the enrollment status of the student (Alfred, Ewel l , Hudgins, & McClenney, 2000; Banta, 1999; Ca l State San Bernadino, 2004; C S H E , 2002; Green & Castelli, 2002; Lancaster et a l , 2003; M a k i , 2002; Seybert, 2003). Although indirect indicators can provide valuable information about the strengths and weaknesses of a college, program, or course, Palomba and Banta (1999) argue that these data must be combined with activities that directly assess learning in order to produce reliable assessment results. Thus, indirect measures can support results of direct measures or supplement them when they are difficult to measure. 94 Table 3.6 Examples of Indirect Methods for Assessing Student Learning Current students: • Campus climate surveys • Course evaluations by students • Engagement surveys • Evaluations of programs and services by students • Exit interviews and focus groups • Self-evaluation . Student goal attainment and values inventories • Student perceptions of non-cognitive outcomes • Student satisfaction, experience or opinion surveys (of program completers and non-completers) Institutional Data In contrast to the first two indicators, which either directly or indirectly evaluate student learning, institutional data, also referred to as non-measures of student learning, can be used for monitoring internal improvement of the college (Green & Castelli, 2002; Walvoord, 2004). This type of information is typically quantitative data and it may be compared longitudinally or may be used to monitor the progress of an entire cohort of students. Typically, colleges use institutional data to address important concerns or answer pertinent questions. Examples of institutional data are listed below in Table 3.7 according to their data source (Alfred et al., 2000; Banta, 1999; C S H E , 2002; Green & Castelli, 2002; M a k i , 2002; J. Nichols, 2002; Seybert, 2003; Walvoord, 2004). Graduates: • Graduation follow-up studies and surveys: reporting satisfaction with college program and career success / advancement and performance at subsequent colleges; completing goals; obtaining appropriate employment; tacking of honors, awards, and achievements . Employer satisfaction Other students: • Non-returning student survey (goal completion information) Faculty: • Faculty survey of student engagement 95 Table 3.7 Examples of Institutional Data Current students: • Grades . G P A s • Levels of participation in the following: -> co-curricular activities -» learning communities -> community service / volunteerism Graduates: . G P A s . Grades (i.e., performance at receiving universities) . Graduation rates and numbers • Honours or other achievements by graduates • Job placement rates Faculty / staff/ administrators: • Faculty: -> faculty/student ratios -> faculty flow, retention and salaries -> faculty evaluations: peer review and chair/supervisor evaluation -> publications and recognitions . Administrator reviews/evaluations • Teaching and assessment methodologies survey • Staff reviews/evaluations Enrollment services: • Attrition/retention rates (for all levels of courses): -> course level (completion rates) -» program level -> college level • Course-taking patterns and profiles . Demographic data / diversity of student body (number of students) • Enrollment trends / admissions practices • Grade distribution analysis • Length of time to degree • Success rate of developmental students in subsequent college-level courses Program: • Content analysis and review of program, courses, syllabi and curricula . Cost • Enrollment patterns • Course-taking patterns • Degree completion rates . Number of graduates . Student G P A s Institutional: • Comparison of outcomes with peer institutions • Cost-benefit analysis (financial costs) • Curriculum / syllabus analysis (e.g., analysis of transfer student preparation) • Strategic planning reviews Community: • Community perception of program effectiveness • Responsiveness to community needs 96 Ideally, assessment at a college involves a combination of approaches integrating multiple sources of information from all three types of data (i.e., direct indicators, indirect indicators, and institutional data). A s s e s s m e n t I s s u e s S p e c i f i c t o T w o - Y e a r C o l l e g e s a n d T r i b a l C o l l e g e s Assessment can be particularly challenging in two-year colleges because of their wide-ranging and numerous missions (Banta, 2004). Tribal colleges face many of these same challenges. These colleges typically provide a wide range of options: vocational programs, developmental courses, adult basic education and G E D courses, continuing community education courses, and, of course, college-level courses and degrees for transfer to four-year institutions. A s a result, the nature of the student body and faculty at community colleges is quite different from that of those at four-year colleges — and both the students and instructors affect assessment. Students attend community college for a number of reasons. Some students enroll at a college to complete adult basic education courses; other students enter college with the goal of completing an associate's of arts degree in order to transfer to a four-year institution; and others enter with the goal of completing a two-year terminal or vocational degree in order to obtain employment. Still others enroll in one or two courses just for interest, with no intention of ever completing a degree. A s a result, these students are more likely to appear to stop-out or dropout than students attending four-year colleges. Due to open admission policies, community college students also have a wider range of academic abilities and preparation, which makes it difficult to conduct any cohort-based research that demonstrates value added. Also , because community college students tend to be more transient (sometimes as a result of financial limitations), tracking them once they leave the college is a significant problem (Banta, 2004; Mundhenk, 97 2004). Because many community college students attend their classes and then leave campus and because it is more difficult to engage two-year college students in meaningful ways outside of the classroom, assessments are generally embedded in their courses (Banta, 2004). Instructors at community colleges are more likely to be part-time, adjunct, and overwhelmed with teaching responsibilities. Furthermore, because of their temporary nature, part-time and adjunct faculty are unlikely to attend faculty meetings or be involved in discussions regarding assessment of student learning, and, therefore, less likely to "buy-in" to the assessment program. Because of their higher teaching load, faculty at community colleges are less likely to be involved in assessment research, but are more likely to be involved in developing student learning outcomes and meaningful classroom assessment techniques. A s a result, most of the assessment research at community colleges tends to be conducted by one person in an assessment department and focuses mainly on information gathering for external reports and institutional surveys (Mundhenk, 2004). Due to limited resources, community colleges integrate a number of approaches in then-assessment efforts including combining standardized measures of student learning with locally developed measures (Banta, 2004); examining student goals at entry and exit; reviewing student functional skills at entry and exit; exploring active student involvement in learning through administering student engagement surveys; following up on the grades and retention of graduates who transfer to four-year institutions; asking local employers about their needs; and exploring course completion, retention, and graduation information (Banta, 2004). Many colleges start by simply setting realistic goals and limiting selection of their learning outcomes to the three or four most important ones (Walvoord, 2004). Regardless of the findings, the results of each of these 98 measures provide a point at which to initiate meaningful conversations among the members of the college community. On a positive note, the mission of community colleges, that of serving the local community's needs, lends itself well to the "public nature of assessment" (Mundhenk, 2004, p. 38). Community colleges are in the business of providing comprehensive services to external bodies to which they are accountable, such as associate's transfer degrees to students who transfer to universities and two-year terminal technical degrees to students who seek employment in local businesses. Because community colleges serve the needs of these external entities, their assessment efforts tend to reflect that focus. Thus community colleges frequently seek information about the level of graduates' preparation by such means as surveying employers, surveying graduates, and tracking transfer students through transfer reports and communications with receiving four-year colleges. A s a result of this external feedback, community colleges and their instructors adjust their programs, curricula, and teaching methodologies. I f done effectively, community colleges have the opportunity to transform themselves through this essential accountability process. It appears that assessment has been more successful, to date, in smaller colleges (Farmer, 1988; Mentkowski et al., 2000). This is probably because smaller colleges are generally more committed to excellence in teaching (as opposed to research) and are more able to cultivate a unified vision among faculty and staff, who can be involved directly in the development of the mission, goals, outcomes, and assessment program. Developing an assessment program appears to be more difficult in larger colleges where faculty frequently perceive the goals of the college as conflicting with their own interests or even the interests of their students. Despite these constraints and hurdles, more and more colleges, both large and small, are attempting to 99 implement assessment programs (Palomba & Banta, 1999). The need to satisfy accreditation requirements undoubtedly provides much of the initial and ongoing motivation in most cases, as does the lure of additional funding or the threat of reduced funding (J. Nichols, 2002; Seybert, 2003). E m e r g i n g I n n o v a t i v e A p p r o a c h e s t o A s s e s s m e n t Engagement One innovative approach to assessment in two-year colleges is the use of student engagement surveys, such as the Community College Survey of Student Engagement — a standardized national survey that attempts to assess student motivation and involvement in their learning process — or engagement. This survey solicits students' perceptions of their level of engagement in various aspects of college life, such as collaborative learning, student effort, academic challenge, student-faculty interaction, and support for learners (Community College Survey of Student Engagement, 2004). This information is valuable because research indicates that students who are more engaged in educationally purposeful activities like these and in the college environment are more likely to succeed (Kuh, 2001; J. Nichols, 2002). Not surprisingly, preliminary research conducted in this area found that the students who withdrew from college were the least engaged (R. Hughes & Pace, 2003). Colleges can use the results from this highly respected national survey to initiate meaningful conversations about increasing retention rates and improving student learning, as well as improving the quality of the overall college experience. Chickering and Gamson (1987) refer to seven good practices in undergraduate education, which result in effective teaching and learning. These practices 100 encourage student-faculty contact; encourage cooperation among students; encourage active learning; give prompt feedback; emphasize the time the student devotes to the task; communicate high expectations; and, respect diverse talents and ways of learning, (p. 15) Ast in reaffirms and expands upon these practices by outlining three essential "conditions of excellence" for student learning: a high level of meaningful and active student involvement in the learning process; high expectations by the students themselves and by the instructors; and high quality instructor assessment and feedback (Astin, 1996). The Community College Survey of Student Engagement is built upon this sound educational research, attempts to determine the level at which students are engaged in the practices articulated by Chickering and Gamson and Astin, and provides a forum for discussing learning and teaching. In addition to the Community College Survey of Student Engagement, there is a corresponding Faculty Survey of Student Engagement, which focuses on faculty perceptions of student engagement, the importance faculty place on different types of learning and development, the frequency and nature of faculty members' interactions with students, and how faculty members organize class time (Pike, 2003). The perceptions of faculty and advisors toward each of these areas can be compared with those of students, which can inform conversations about teaching and learning and can lead to refinements in both. Because many of the items included in these two surveys raise awareness of issues and attitudes that faculty and advisors can recognize in conversations with students, the use of these surveys has the potential to improve student retention (R. Hughes & Pace, 2003). B y utilizing these surveys periodically to assess student engagement, colleges can demonstrate improvement in the learning environment. 101 Assessment as Learning In the United States, a few colleges have taken the lead in engaging students in the assessment process itself— as a means of promoting learning. Alverno College, a small Catholic liberal arts college for women in Milwaukee, Wisconsin, is one of these. Alverno College has been a leader in assessment for the last 30 years and has pioneered the concept of assessment as learning (Mentkowski et al., 2000). Faculty and staff at Alverno College believe that students should learn something through every assessment process and, i f assessment is done well , the result should be learning that lasts (Mentkowski et al., 2000). Thus, assessment at Alverno College is as much a part of the learning process as it is a part of the evaluation process — and the success of this approach has been impressive. Serving a predominantly low income, minority population, Alverno College (both its faculty and students) has received numerous awards and widespread recognition for its achievements (Alverno College, 2004). Mentkowski (2000) explains that this success is a result of Alverno College's philosophy of assessment, which is based on the following principles: 1. If learning is to be integrative and experiential, assessment must judge performance. 2. If learning is to be characterized by self-awareness, assessment must include expected outcomes, explicit public criteria, and student self-assessment. 3. If learning is to be active and interactive, assessment must include feedback and external perspectives as well as performance. 4. I f learning is to be developmental, assessment must be cumulative and expansive. 5. If learning is to be transferable, assessment must be multiple in mode and context, (p. 60) These principles form the foundation of all learning and teaching at Alverno College, as it pushes the concept of authentic assessment farther than most other institutions. In the process, Alverno College established a standard of experiential learning by integrating internships, 102 content, and abilities, and by defining learning outcomes as abilities. To support this approach, Alverno College's curriculum is ability-based: Students are required to demonstrate competence at increasingly advanced levels in eight abilities throughout the course of their studies. Alverno's eight college-wide abilities are "communication, analysis, problem-solving, valuing in decision making, social interaction, global perspectives, effective citizenship, and aesthetic responsiveness" (Mentkowski et al., 2000, p. 63). Because the assessment of these eight abilities is integrated into a coherent educational system, students make connections across different disciplines (as well as within their major). Furthermore, Alverno College students are required to track the progress of their own learning outcomes. Because the importance of self-reflection is emphasized, students at Alverno become motivated, responsible, and accountable for their own learning and progress (Mentkowski et al., 2000). A t a workshop at Alverno in June 2003,1 observed one of the many intermediate-level ability-based assessments of the communication ability. The process was simple yet impressive. Alverno staff placed students in teams of eight and gave students a sophisticated problem to solve, with an ethical dimension related to their field. Several retired professionals in their field, from outside the college community, observed the students consulting for approximately one hour and provided them with detailed and immediate individual and group feedback according to specific criteria. The assessment also required students to provide a self-assessment and peer review of their performance. It was obvious that by the end of the afternoon, the students had gained significant and valuable insight into strengths and weaknesses of their ability to communicate. Many academics and institutions have written about and supported some of the principles listed above (Gipps & Stobart, 2003), but few have been as successful as Alverno College in 103 implementing them in such a comprehensive assessment system. For instance, Strain (2003) recommends that all assessment tools used in the classroom should directly benefit students and their learning. The Center for the Study of Higher Education (2002) suggests assessing learning outcomes at three points during the student's college experience: at admission (beginner level), at the midpoint of their college degree (intermediate level), and at graduation (advanced level). They recommend that each program map out their curricula. For each learning outcome, they recommend outlining in a chart the level of learning based on Bloom's Taxonomy (i.e., knowledge, comprehension, application, analysis, synthesis, and evaluation). Tinto (1993) reaffirms the importance of immediate feedback to effective individual student learning and Woosley et al. (2003) stress the importance of putting larger scale structures in place to respond immediately to students who go astray in their first year of college. Furthermore, the National Postsecondary Education Cooperative (2002) affirms Alverno's competency-based approach in a report stating "strong competency-based initiatives produce meaningful assessment results that are used in making critical decisions about ways to improve student learning" (p. 19). Other aspects of Alverno's approach that are referred to in a report include the importance of the transferability and applicability of knowledge and understanding, practice and feedback, and meta-cognition for self-correction (Pellegrino et al., 2001). This report notes that assessment cannot be kept isolated from curriculum and instruction — although it stops short of stating that every assessment process should result in learning. The Committee on the Foundations of Assessment (Pellegrino et al.), in an extensive report produced in 2001, used as the foundation of its work two basic principles: (a) students should always learn something from being assessed and (b) this information should be used to improve student learning. Supporting Alverno's approach, the committee emphasizes that in 104 order for assessment to be successful and improve learning, students must understand the learning outcomes and their relevancy. It reflects on the importance of integrally relating instruction with assessment as wel l as timely feedback. The committee also recommends what Alverno has long known, that "assessment practices need to move beyond a focus on component skills and discrete bits of knowledge to encompass the more complex aspects of student achievement" (Pellegrino et al., 2001, p. 3). Alverno faculty address this concern by assessing their students while they are applying the knowledge and understanding they have gained through their education to other situations (Mentkowski et al., 2000). Thus, while reports such as this one are just beginning to articulate and recognize these important principles, Alverno has embodied them for the past 30 years. Alverno's success in this regard is largely due to its faculty's research on the assessment of student learning. Regardless of the professors' discipline, Alverno faculty are encouraged to engage in assessment-related research. P l a n n i n g a n d R e p o r t i n g The Assessment Plan In its Accreditation Handbook, the Northwest Commission on Colleges and Universities (2003), the accreditation body that oversees the colleges and universities in the northwestern United States, emphasizes the importance of the college and program assessment plans: The Northwest Commission on Colleges and Universities expects each institution and program to adopt an assessment plan responsive to its mission and its needs. In so doing, the Commission urges the necessity of a continuing process of academic planning, the carrying out of those plans, the assessment of the outcomes, and the influencing of the planning process by the assessment activities, (p. 37) The assessment plan serves either one or both of the following purposes: first, to outline existing assessment strategies and to recommend ways to improve assessment; or, second, to 105 sirmmarize outcomes of assessment measures and recommend ways to improve student learning (Walvoord, 2004, p. 9). If the purpose of the assessment plan is to improve assessment, then the assessment report may include the following information: the person responsible for overseeing assessment; the resources and structures available for assessment; how faculty created the learning outcomes and how they are used for assessment; the measures of student achievement, the reasons chose these measures, how they relate to the outcomes, and how they are administered; how assessment results are used for improvement of learning; and recommendations to improve the assessment program (Walvoord, 2004). If the purpose of the assessment plan is to improve student learning, then the assessment report should include a list of the learning outcomes; data from assessment measures and what the results suggest about student achievement of the learning outcomes; and recommendations to improve student learning (Walvoord, 2004). N W I C ' s 2002 - 2006 assessment report primarily focuses on the first purpose, recommending ways to improve assessment. The 2007 to 2012 assessment report w i l l primarily focus on the second purpose, recommending ways to improve student learning (although it w i l l also recommend ways to improve assessment). The assessment plan coordinates campus-wide assessment efforts and college activities, presents to external bodies a well-conceived approach to assessment, and provides a systematic way to determine the extent to which outcomes have been achieved. The assessment plan focuses on educational outcomes and is dynamic in nature (J. Nichols, 2002). Furthermore, the assessment plan may be a simple matrix that overviews the time-frame for accomplishment of each of the items and may include strategic plan, the college assessment plan, the academic program assessment plans, and a faculty evaluation process (Eastern Oregon University, 2004). 106 In order to build an assessment plan or report that is useful to the college, several steps must be taken: first, embed assessment in essential processes (such as strategic planning, curriculum review, program review, grading, developing independent learning programs or learning communities, and teaching evaluations); second, articulate college outcomes; third, conduct an assessment inventory (i.e., an inventory of what the college is already doing or planning in terms of assessment and how the information is used for making decisions and improving the college); fourth, highlight strengths and weaknesses and make recommendations for improvement; fifth, improve the assessment program; sixth, improve student learning; and, finally, write the plan or report (Walvoord, 2004). Closing the Loop Assessment plans can be implemented at the course, program, and college levels. For instance, a basic assessment plan at the course level may include the instructors' learning outcomes; the tests and assignments; the instructors' criteria and standards (i.e., rubrics); student scores over time; and evidence of feedback into learning and teaching (Walvoord & Anderson, 1998). A n example of a section of N W I C ' s assessment plan for 2007 - 2012 is included in Appendix A . Using a matrix, similar to the section of the program-level plan in Table 3.8, provides a framework for encouraging faculty and staff to use the assessment plan and results to improve learning. 107 Table 3.8 Example of a Section of an Environmental Studies Program Assessment Plan Program Assessment criterion outcome and procedure Assessment result Use of result Students w i l l 80% of students Only 70% of students This concept w i l l be be able to completing the completing the program had reinforced in two propose program w i l l score 4 or a score of 4 or higher on this additional required solutions to higher on the rubric for rubric for the final project in environmental studies and solve real- this outcome for the the graduating seminar. courses. world final project in the problems by graduating seminar. applying the correct numerical data. Note. Adapted from J. Nichols (2002, p. 247). After the college implements its plan, it may report its results in various ways. For instance, Lower Columbia College presents the results of a different section of its assessment plan in a reflective meeting every second month to its board, faculty, staff, and students (Weinstein, 2003). In this way, it can focus on a new area every couple of months and present more manageable portions of information to its community. Its assessment process includes a deep reflection on the results and recommendations for changes for the future. It also reports on results of changes made in previous years. Other colleges opt for annual report cards that report on the progress of institutional data (Banta, 1999). This option can be overwhelming as it requires a more substantial annual reporting project and is more of a one-time effort. Other faculty and staff meet regularly through faculty retreats, meetings, forums, and so forth to reflect on experience gained through the assessment process and translate this learning into actionable reforms (Ewell , 2002). Regardless of the reporting method chosen, because feedback of the assessment results is a significant step in the assessment process, it should not be rushed; 108 adequate time should be allowed for reflecting upon the information (Rodrigues, 2002). Furthermore, effective assessment requires that colleges move beyond merely documenting processes to evaluating the college's overall effectiveness (Banks & Colby, 1989). Conclusion A well-planned and supported assessment program has the potential to lead to a transformed college environment and toward learning that lasts; however, transformation takes time and is influenced by the college's culture (Mentkowski et al., 2000). Thus, in order to create a culture of assessment, assessment must permeate all aspects of the college — from planning, budgeting, program mission statements, and the first-year experience program, to the catalogue, publications, website, job descriptions, and promotion. Just as self-assessment and reflection are essential for the development of the student, so, too, are they necessary to sustaining a process of institutional transformation, which w i l l expand awareness of strengths and weaknesses in the quest to become a learning-centered institution (Mentkowski et al., 2000). 109 C H A P T E R 4 A S S E S S M E N T I N T R I B A L C O L L E G E S This chapter is divided into two sections. The first section discusses the guiding principles of culturally appropriate research in Native American communities that are most relevant to the future of assessment at tribal colleges. The second section provides a discussion of the state of tribal college assessment in the United States and an initial exploration of possibilities for adapting best practices in mainstream assessment to a tribal college context in culturally appropriate ways. G u i d i n g P r i n c i p l e s The literature on research with Native American communities emphasizes the commitment of researchers to a participatory process, based on a respectful and reciprocal relationship between the researcher and the community, when they conduct research in these communities (Haig-Brown & Archibald, 1996; Hampton, 1995b; Kirkness & Barnhardt, 1991; Menzies, 2001; National Sciences and Engineering Research Council of Canada, 1999; Royal Commission on Aboriginal Peoples, 1996; L . Smith, 1999). In addition, the research itself needs to be relevant and meaningful to the community and conducted in a responsible and ethical manner (Darou et al., 2000; Greenwood & Levin , 1998; Kirkness & Barnhardt, 1991; Royal Commission, 1996; L . Smith, 1999; Stringer, 1999). Over the last century, researchers have conducted a substantial amount of research in Indigenous communities; however, repeatedly, researchers have ignored basic principles of respect and ethics (L. Smith, 1999). Furthermore, researchers have misinterpreted and excluded Native Americans from telling their stories (L. Smith, 1999). A s a result, scholars of Indigenous research and knowledge are now formally articulating principles that are essential to integrate 110 into research conducted in Indigenous contexts (Battiste & Henderson, 2000; Blanchard et al., 2000; Haig-Brown & Archibald, 1996; Hampton, 1995a; Kirkness & Barnhardt, 1991; Menzies, 2001; N S E R C , 1999; Royal Commission, 1996; G . Smith, 1992; L . Smith, 1999; Steinhauer, 2002; S. Wilson, 2003). Methods of research appropriate in Native American communities are unique in that they "privilege(s) Indigenous concerns, Indigenous practices, and Indigenous participation" (L. Smith, 1999, p. 107) by recognizing their distinctive worldviews and knowledge; honouring Aboriginal traditions and lands; "emphasizing the social, historical and political contexts, which shape" their lives; "privileging the voices, experiences and lives of Aboriginal people and Aboriginal lands; and, identifying and redressing issues of importance" to them (Martin, 2001, p. 3). This section reviews and elaborates upon the essential principles of respect, responsibility, reciprocity, and relevance that apply most directly to the assessment processes at tribal colleges and to this research (Kirkness & Barnhardt, 1991). Respect The importance of researchers or, in this context, tribal college assessment coordinators maintaining respectful relations and a sense of humility with those participating in the assessment process is emphasized throughout the literature on Native American research (Haig-Brown & Archibald, 1996; Menzies, 2001; N S E R C , 1999; Royal Commission, 1996; L . Smith, 1999). If the traditions, culture, and knowledge of the tribal college community are protected and validated, the community becomes empowered. " A n essential and defining element of ethical issues in research on Indigenous peoples is respect for Indigenous peoples' laws and institutions" (Battiste & Henderson, 2000, p. 144). Taking it a step further, Linda Smith (1999) maintains not only that research "processes are expected to be respectful" but that, when carried 111 out properly, they can also "enable people, to heal and to educate" and "lead one small step further towards self-determination" (p. 128). Recognizing that this is the ideal and that it is a difficult process, Menzies (2001) also outlines four basic steps comprising a respectful research protocol. First, the researcher or the community initiates meaningful consultation in response to a college need or concern. Second, the researcher refines the research plan in consultation with the appropriate faculty, staff, students, or community members. The tribal college contributes a lot of the input and directs the project's goals and outcomes. It ultimately determines i f the projects and methodologies being proposed are appropriate. Third, the research is conducted. Fourth, the researcher, always maintaining close contact with the tribal college community, writes, analyzes, revises, and distributes the report to the community. Involving many members of the tribal college community in this way, Smith (1999) asserts, results in an effective process and a significant increase in the capacity of the community. Needless to say, this approach often results in a prolonged research process and thus requires self-reflection, patience, flexibility, and openness on the part of the researcher (Steinhauer, 2002). I attempt to follow these fundamental guidelines, both in my work and in my doctoral research, as I have responded to and been flexible with college needs; consulted continually with faculty, professional staff, and administrators; had numerous faculty, professional staff, and administrators review and edit my work; and reported results back to the tribal college community. Responsibility Many of the principles and thoughts that are articulated in Indigenous research literature are also gaining momentum in mainstream research literature and approaches. One of these "approaches to participative inquiry" (Bensimon et al., 2004, p. 109), called Participatory Act ion 112 Research (also known as Community Act ion Research), seeks to maximize responsible research practices by engaging communities as full participants in the research process in order to empower them (Greenwood & Levin , 1998; Stringer, 1999). A collaborative and community-based approach "overtly engages the human relationships that are involved, is concerned with the style and manner of communication among people, and purposefully includes all those affected by the research as active participants in the process" (Stringer, 1999, p. xx). B y forming partnerships and by using community-level resources (A. Wilson & Cervero, 1997), these approaches give those who are being researched control over the process, enabling them to build local community skills and resources (Hudson & Taylor-Henley, 2001; Weber-Pillwax, 2001). Responsible research approaches include the use of ethical research protocols and professional practices (Royal Commission, 1996; L . Smith, 1999). They also require an understanding of tribal college protocols and a reflective and non-judgmental attitude (S. Wilson, 2003). A s L . Smith (1999) points out, "the processes of consultation, collective meetings.. .and shared decision making" (p. 129) are important aspects of this process. Thus, ultimately, the tribal college decides which research is pursued; who the research is for; the effect it w i l l have; who carries it out; how it is carried out; how it is determined the research is worthwhile; who owns the research; who w i l l benefit; and how the findings w i l l be shared (L. Smith, 2000). Reciprocity "Practices of reciprocity" involve "negotiating better research relationships" (L. Smith, 1999, p. 136 and p. 119) and are skills that are fundamental in the pursuit of respectful research processes with Native American communities. Research in a tribal college context recognizes the inter-connectedness and interrelatedness of the physical, intellectual, spiritual, and emotional dimensions and involves the individual, family, and community (Graveline, 1998; Weber-113 Pillwax, 1999; S. Wilson, 2003). Because this research is community-oriented and requires a holistic approach, the assessment coordinator is accountable to the tribal college community, rather than to his or her own pursuit of knowledge (Cardinal, 2001; Steinhauer, 2002). Historically, Native American people have not benefited from research conducted in their communities; therefore, the research should originate in the tribal college community, serve its needs, and enrich its members' lives (Marker, 2004; Weber-Pillwax, 2001; Weiss, 1998). Reciprocity implies a collaborative research process, one that enables community members to participate in the planning, implementation, and analysis o f research results (Royal Commission, 1996). It requires everyone's contributing input and control equally and results in "changing the power relationship in research partnerships" (Blanchard et al., 2000, p. 10). It encourages researchers to listen, to hear, to attempt to understand, and only then to speak (Blanchard et al., 2000). It emphasizes the importance of frequently reporting back to the community for accountability purposes so that the community can use the information learned for improvement (L. Smith, 2000). It implies that the research questions being asked are relevant, valuable, and important to the community, and it implies a commitment to building relationships, which takes time. Furthermore, the Report of the Royal Commission on Aboriginal Peoples (1996) suggests that "wherever possible, research should support the transfer of skills to individuals and increase the capacity of the community to manage its own research" (p. 327). Relevance The Royal Commission also suggests important ways for ensuring that research is relevant to the community. When the tribal college is involved in defining the problems to be examined, it insures their relevance to the improvement of the community. A s a result, the 114 research engages the community directly in formulating solutions to problems it experiences. Ways this can be accomplished include involving the tribal college community members in the design of the project and incorporating local practices with technical knowledge (Greenwood & Levin, 1998). In addition, involving the community in the review of the findings and their reporting validates the content of the research and further builds upon the principles of relevance and reciprocity (Stringer, 1999). Research conducted in a tribal college needs to be useful to the community, to recognize and legitimize Indigenous knowledge 1 2 and skills, and to be grounded in the lives of the students and community (Kirkness & Barnhardt, 1991; Nelson-Barber, LaFrance, Trumbull, & Aburto, 2005; S. Wilson, 2003). Thus, Darou, Kurtness, and Hum (2000) concur that research in Native American communities needs to display significant social value as well as adaptation of the instruments to the culture. N W I C has developed a mechanism for protecting the research interests of the college and the tribal communities it serves. Its Institutional Review Board reviews all research pertaining to the college and its tribal communities with the goal of conducting effective and culturally acceptable research that benefits its communities. In a similar way, N W I C has a five-member Board of Trustees that governs the college. The trustee members are appointed by the Lummi Indian Business Council and are representative of the tribes N W I C serves. The role of the N W I C Board of Trustees is to maintain accountability to the larger tribal communities. In an effort to do this, the president and administrative team meet monthly with the Board of Trustees to share triumphs and to consult about challenges. The Board of Trustees and the Institutional Review Board both work to assist in maintaining respectful, reciprocal, relevant, and responsible relations between N W I C ' s endeavours and the local communities it serves. 1 2 Refer to chapter 2 for a discussion about Indigenous Knowledge and Culture. 115 Research in tribal colleges has the potential to transform local communities (Blanchard et al., 2000; Nelson-Barber et al., 2005; L . Smith, 1999), but, to do so, it "requires a context that is consciously considered and purposefully incorporated into the research by the researcher" (Weber-Pillwax, 2001, p. 166). A s long as the above-mentioned protocols of respect are followed, mainstream research methodologies can be integrated into those of tribal colleges (Marker, 2004); however, ultimately an Indigenous research methodology, based on Indigenous epistemology, is fundamental to the creation of Indigenous knowledge (Weber-Pillwax, 1999, 2001; S. Wilson, 2003). Blanchard et al. (2000) elaborate further on the importance of research in Native American communities: "We are interested in research and enthusiastic about its usefulness, but we want it to be inclusive of our knowledge, values and inquiry perspective, and we want to be genuine partners in its production, distribution and application" (p. 3). S t a t e o f A s s e s s m e n t i n T r i b a l C o l l e g e s To date, tribal colleges unfortunately have found it difficult to gather even basic assessment data. For instance, quantitative data measuring the most basic conventional institutional data referred to earlier (e.g., retention and graduation rates, time to degree completion, transfer and employment rates) are often difficult to collect in tribal colleges due to inadequate technology and to a lack of experienced human resources (Gagnon, 2007). Thus, the Carnegie Foundation (1997) asserts, "increased funding for staff dedicated to this research role is a necessary next step in the growth of the tribal colleges" (p. 94). Moreover, these conventional indicators alone do not provide a useful picture of what matters most to tribal colleges (Boyer, 2003). 116 While conventional assessment frameworks clearly do not reflect the values or address the needs of Native American communities, progressive assessment frameworks may reflect these values and address these needs, due to their more inclusive, relational, and process-oriented approaches (Haig-Brown & Archibald, 1996; Kirkness & Barnhardt, 1991; National Sciences and Engineering Research Council o f Canada, 1999; Royal Commission on Aboriginal Peoples, 1996; L . Smith, 1999). The more innovative assessment programs focused on student learning outcomes are gaining attention as valuable tools in post-secondary education, yet they are not well developed in a Native American context (Carnegie, 1997). I f they are developed in Native American contexts, these progressive assessment frameworks may anticipate and address concerns that derive from critical educational theory regarding issues of power and control in assessment and the political and economic context and motives surrounding assessment practices (A. Wilson & Cervero, 1997; Forester, 1993; L . Smith, 1999). Traditionally, mainstream assessment has not been characterized by either the principles articulated above or the methodologies their implementation requires. Rather, it has been driven somewhat narrowly by Euro-American definitions of student success, definitions that have, to some extent, been imposed on tribal colleges in the past. In addition, as is the case of mainstream colleges, tribal colleges are now being asked to demonstrate institutional effectiveness to accreditation bodies in more rigorous ways. A s there is a potential tension between outcomes-based assessment and Native American worldviews, some tribal college advocates initially were understandably concerned that Euro-American definitions of student success would be imposed on tribal colleges even more than they have been in the past (Crazy B u l l , 1994); however, with the new emphasis on learning outcomes assessment, many tribal college advocates believe they now have an opportunity to re-defme their own measures of 117 success and, therefore, their own curricular and pedagogical values, in more culturally appropriate ways (Boyer, 2003). Moreover, even though mainstream colleges are focusing their assessment efforts primarily on academic learning outcomes, tribal colleges believe that cultural outcomes are at least as important as cognitive, psychomotor, and affective outcomes, i f not more important (Astin, 1999). The broader mandate of tribal colleges also provides them with an opportunity to widen the scope of their assessment efforts by articulating and documenting community-wide outcomes in the tribal communities they serve. Indeed, tribal colleges emphasize community-wide outcomes, such as the "self-determination of Native Peoples" (NWIC, 2005b, p. 4), in their mission statements. These outcomes are difficult to measure, especially in two-year colleges. To approach assessment in a meaningful way, tribal colleges would need to track not only their ability to improve the lives of students, but also their ability to improve conditions within tribal communities. In this context, the task of assessment in tribal colleges "is to measure a college's ability to both serve a community and build community" (Boyer, 2003, p. 143). To date, Paul Boyer is the only person to articulate elements of an assessment model in a tribal college context. He suggests that tribal college assessment programs address six levels of accountability, which can satisfy the needs of accreditation bodies and funding agencies while also providing a means for tribal colleges to monitor their progress, demonstrate their strengths, act on their weaknesses, and fulfill their mission statements. He suggests that tribal colleges focus their assessment efforts on the following six aspects: mission statement, cultural outcomes, larger campus community, learning outcomes, wider tribal community, and communities beyond the reservation (Boyer, 2003). In this section, I elaborate upon and apply Boyer's framework by drawing on current best 118 practices in mainstream assessment and adapting them in culturally appropriate ways to a tribal college context. Miss ion Statement Boyer's first area of accountability concerns familiarity with, and support for, the institution's mission statement. A s mentioned earlier, tribal colleges were established with a two-fold mission: (a) to preserve traditional tribal culture and knowledge and (b) to provide certificates or associate's (two-year) degrees that provide Native American students with better employment opportunities and facilitate transfer to four-year colleges or universities (Farm, 2002). This two-fold mission is not well understood, is complex, and is challenging to implement, let alone measure. Boyer (2003) suggests that faculty, staff, students, and the larger tribal community can collectively examine the mission statement, question it, refine it, and engage in on-going dialogue about it, in order to increase the level of familiarity with and support for it. "Appreciation of the mission statement," he states, "encourages members of a college community to look beyond a lack of institutional resources and focus instead on the strength of the college's vision" (p. 144). The mission statement and goals of a tribal college should provide a clear understanding of the college's institutional intentions so that program outcomes can be directly developed from and linked to them (J. Nichols, 2002). The mission statement and institutional goals are generally articulated through a strategic planning initiative. Once articulated, the mission statement and goals provide guidance for administrative decisions, direction for each of its programs, a framework within which academic units can assess and improve their programs, and a structure for overall assessment and improvement (J. Nichols, 2002). Therefore, the mission statement and goals of the college need to be specific enough to provide clear direction for the 119 college and highlight its unique niche (Mentkowski et al., 2000). When the mission and goals are clarified, and assessment indicators are developed, students are more likely to "experience the curriculum as purposeful and connected" (Mentkowski et al., 2000, p. 322). It is therefore important that the indicators chosen to demonstrate student success reflect the college's mission and goals. When this is the case, it also becomes apparent when the mission statement and goals no longer reflect the direction of the college and need to be updated. In view of the challenges tribal colleges face both in fulfilling their ambitious missions and goals and in meeting the needs of the students they serve, a process whereby they are able to measure their success and make adjustments and refinements in the direction they are taking is vital. For example, in addition to preparing students to transfer to four-year colleges, many tribal colleges provide access to career training and retraining, developmental coursework, continuing education, and, most significantly, cultural enrichment. Moreover, tribal colleges serve a challenging student population, many of whom live below the poverty line and have minimal academic preparation. Students represent a wide range of ages and have various intentions for enrollment (Boyer, 1990). Therefore, agreeing on a set of institutional goals, which addresses the needs of a diverse student population, can be quite a challenge. The process of ongoing reflection needed for refinement of mission statements, goals, and indicators assists tribal colleges' efforts to address these challenges and more efficiently focus their resources on their students' needs. Cultural Outcomes The second area pertains to how successful the tribal college is at supporting cultural outcomes, such as increasing students' sense of cultural identity and increasing connection to their tribe and community (American Indian College Fund, 2002). Tribal colleges are only just 120 beginning to articulate these outcomes, let alone assess them (Marker, 2000). For instance, Turtle Mountain Community College, a tribal college in North Dakota, bases its curriculum on the seven Ojibwa teachings of bravery, wisdom, love, respect, honesty, humility, and truth (Ortiz & Boyer, 2003b). The college developed a survey to examine its students' learning experiences and to determine how effectively faculty were integrating traditional culture and knowledge into the curriculum. The first time staff conducted the survey the results indicated that only a handful of instructors were basing the curriculum on the culture. These results stimulated a flurry of open discussions about priorities and new approaches. After many discussions and in-service meetings, increased cultural awareness resulted in greater integration of culture into the classroom. Turtle Mountain Community College is now trying to determine how students are applying this cultural knowledge outside the classroom (Yellow Bi rd , 1998-1999). In the winter quarter of 2003, N W I C took a similar approach and had similar results. It conducted a survey of instructors to determine the teaching and assessment approaches they were using (see Appendix B , Survey of Teaching and Assessment Methods, for a copy of the report). This survey also asked basic questions about writing, technology, and Native American cultural content in courses taught that quarter. A t least 35% of all courses reported having minimal to no cultural enrichment with Native American materials, and only 15% reported being primarily based on Native American materials. Twenty one percent of courses used Native American speakers and approximately half the courses reported meeting the needs of students by incorporating Native American texts, perspectives, films, writers, websites, music, and art. The discussion of these results contributed to faculty enthusiasm for establishing clear standards, including cultural standards, for all N W I C programs. 121 Tohono O'Odham Corrvmunity College has another innovative approach. It requires all staff and students to take the college's Native American language and history courses within their first year of employment or attendance. Several faculty have reported enhanced understanding of cultural issues and a more unified approach with this initiative (St. Clair, 2006). One of the most challenging areas of this kind of assessment in a tribal college context is that of first determining and then assessing cultural outcomes. This is one of the areas where the least amount of clarity exists as it is up to each individual college to determine for itself its desired cultural outcomes. In addition, assessing these outcomes can become extremely difficult when individuals from different tribes are represented and have different views about this sensitive subject (Gagnon, 2007). Alaskan Native educators have developed a set of culturally responsive standards for students, educators, schools, curriculum, and community, which may well serve as a model for Indigenous colleges. These standards are articulated in a document titled Alaska Standards for Culturally Responsive Schools (Assembly of Alaska Native Educators, 1998). Finally, Demmert (2005) highlights six criteria he believes are essential for culturally-based Native American education programs: Culturally-based Native American education programs (a) require students to learn Native languages; (b) have "pedagogies that stress traditional cultural characteristics"; (c) have teaching strategies that "are congruent with the traditional culture as well as contemporary ways of knowing and learning"; (d) have a "curriculum that is based on traditional cultures, which recognizes the importance of Native spirituality and places"; (e) "have strong Native community participation including partnering with parents, elders, and other community resources"; and (f) have "knowledge and use of the social and political mores of the community" (p. 4). When articulating cultural outcomes, tribal 122 colleges may want to consider Demmert's suggestions and the Alaska Standards for Culturally Responsive Schools (Assembly of Alaska Native Educators, 1998). Learning Outcomes Learning outcomes pertain to measuring student learning, including cognitive, affective, and psychomotor outcomes (I discussed cultural outcomes in the Cultural Outcomes section, in chapter 4) (American Indian College Fund, 2002; Archibald & Selkirk, 1995). A l l learning outcomes may be demonstrated at three levels: the college level, where students are assessed throughout their time at the college with regard to a set of general education or college outcomes; the program level, where students demonstrate their learning of program outcomes through, for example, capstone experiences; and the course level, where students are evaluated on their short-term learning of course outcomes. A n example of a common college outcome may be that students will be able to write Standard English. A program outcome might be that students will be able to conduct environmental assessments, impact monitoring, modeling, and prediction. A course outcome in a math course might be that students will be able to complete an income tax return. College Outcomes Tribal colleges are just beginning to articulate college outcomes. When students are evaluated for college outcomes both at entrance to and at exit from college, this is referred to as value-added assessment. Value-added assessment, where students are their own controls, emphasizes gains in students' abilities to identify the impact of their college education. This approach requires buy-in from faculty and staff, effective leadership, an effective computerized tracking system, a commitment to use the results, and time (Stufflebeam, 2001). Experts in the 123 assessment field advocate the use of course-embedded assessments (also known as secondary analysis of course papers or projects) rather than, for example, the use of separate "testing days" to assess college outcomes (Banta, 2003; Suskie, 2003). A s the name implies, course-embedded assessments involve evaluation of student work that is done in the classroom; in this case, however, samples of work may be assessed by faculty a second time to assess college outcomes. Course-embedded assessment becomes even more essential in tribal colleges where many students enroll with the intention of only completing one or two courses (J. Nichols, 2002). The advantages of this approach over separate testing days for a tribal college are that it is cost-effective (because the same assessment tool can be used for two purposes) and some students are motivated by grades to perform their best. In addition, in the opinion of Gagnon (2007), a professor of Indian Studies at the University of North Dakota and a tribal college evaluator, more innovative assessment strategies that take advantage of small class sizes may be employed at tribal colleges: Small size means that assessment can be more informal — less testing instrument oriented. I f you have only 5 faculty members teaching general ed. courses, then they can get together regularly to describe student learning based on their intimate knowledge of students in their courses and in the context of the college, (p. 1) Tribal colleges are beginning to think about college-wide assessment in more creative ways. Program Outcomes The assessment of program outcomes is central to a tribal college's commitment to accomplishing its mission and goals (J. Nichols, 2002; Rossi, Lipsey, & Freeman, 2004). Because implementation of the college's mission takes place at the program level through the identification of outcomes linked to the college's mission and goals, assessment results indicating that program outcomes are being accomplished imply that the college's mission is 124 being accomplished (J. Nichols, 2002). Tribal colleges are just beginning to develop program-level assessment processes (Gagnon, 2007). Once an institution has agreed upon its mission statement and goals, there is general consensus among assessment experts about the subsequent steps that should be taken in developing a program assessment plan (Maki , 2002; Mentkowski et al., 2000; Rossi et al., 2004; Stark & Lattuca, 1997). The first step is for each program to identify and define its learning outcomes. These outcomes clearly state what the program would like its students to be able to do upon graduation, as well as specify the expected level of performance. Next, the related activities and capstone experiences that w i l l be used to teach the material are identified, as well as appropriate evaluation methods (both quantitative and qualitative) for assessing student achievement for each outcome at key stages in the program. The number of outcomes to be assessed (perhaps three or four) is limited to those that are generally weak and reasonably attainable, both "measurable" and observable, in the allocated period of time (The State University of New York Geneseo, 2004; Walvoord & Anderson, 1998). After determining how the results w i l l be used for program improvement, the faculty can then implement the evaluation methods to determine the extent to which learning outcomes are being achieved. In order to do this, criteria are identified that clearly articulate successful accomplishment of learning outcomes (e.g., 80% of students completing the program w i l l score 4 or higher on the rubric for this outcome for the final project in the graduating seminar) (refer to Table 3.8). Later, results are communicated to faculty, staff, and students so that recommendations and decisions can be made to improve learning. Ideally, the tribal college implements its assessment activities on a regular cycle during which it may review and possibly modify its mission, goals, or program outcomes. Ultimately, 125 of course, the results are used to improve learning and teaching. A s expected, a time-line is developed to track the assessment of program outcomes, and the entire assessment process is reviewed to determine whether it is providing the most useful information possible (Bresciani, 2003; Hatfield, 1997; The State University of New York Geneseo, 2004; University of Wisconsin, 2004). To assist in organizing this information, the process of systematically determining which outcomes have been achieved and how the information w i l l be used can be recorded in a program assessment plan similar to the one in Table 3.8. A point that is worth emphasizing is the order in which assessment activities at the program level are implemented: The intended educational outcomes are established first and then the appropriate means of assessment'are identified (J. Nichols & Nichols, 2000). The approach to assessment planning outlined above provides a valuable tool as it requires faculty and administrators to "close the loop" and to demonstrate the use of assessment results to improve learning (J. Nichols & Nichols, 2000). James Nichols (2002) points out additional considerations that are equally important to the organization of a successful program assessment plan. First, it is important to gain the confidence of program chairs, faculty, and staff by carefully explaining the process and how the data w i l l be used. A s discussed in chapter 3, the key to the success of any assessment program is that it is faculty-driven, so the administrators' roles are limited to encouraging and providing support and resources. Second, to ease the burden on faculty, who commonly lack the time and resources to be adequately engaged with the development of an assessment plan, existing data and course-embedded assessments should be used as much as possible (J. Nichols, 2002). Third, successful program assessment plans focus their efforts on end results (outcomes), rather than evaluating the process, which has been the traditional focus of assessment efforts at this level 126 ( S U N Y G , 2004). This outcomes-based approach w i l l be unfamiliar and even uncomfortable to some faculty as the focus is on the impact of faculty actions (i.e., learning), rather than on the action itself (i.e., teaching). Finally, a successful program-level assessment process educates the students about the role of assessment in their education (University of Wisconsin, 2004). These different elements of a successful program-level assessment process can be incorporated into a wide variety of situations. Because tribal colleges provide an assortment of programs to serve students with various educational objectives or intentions (e.g., two-year college transfer, continuing education, vocational training), the methods chosen to assess different programs might vary. Nevertheless, it is wise for all programs to select a combination of direct and indirect indicators of student learning, as well as institutional data. For instance, when assessing a program that provides vocational training, a tribal college might collect information about how students performed in their apprenticeships or internship (direct indicator), how graduates evaluate their career training through focus groups or surveys (indirect indicators), how successful students have been in obtaining suitable employment (institutional data), or whether the college is meeting local tribal or regional labor market and economic needs (institutional data) (Alfred et al., 2000; Banta, 1999; Seybert, 2003). Likewise, a developmental education program at a tribal college may evaluate a writing assignment (direct indicator), conduct a course evaluation (indirect indicator), document course completion rates (institutional data), or monitor student migration from non-credit to credit courses and programs (institutional data) (Banta, 1999). Other forms of program evaluation include external exam, portfolio, classroom, capstone course, research or senior thesis assessment (direct indicators); student self-assessment and entry, midway and exit surveys, focus groups or interviews (indirect indicators). Finally, as a side note, at the program level, although only a few relevant learning outcomes are 127 generally assessed, they are likely to be assessed in different courses at different stages of program completion. Course Outcomes A s I discussed in the previous section, program assessment combines the assessment of key learning outcomes in capstone experiences (direct indicators) with indirect indicators, and institutional data and reports the information back to the department in relation to program goals to improve learning and teaching. In contrast, in course assessment, the work of students is primarily evaluated by the instructors, using mostly direct indicators that are based on course outcomes to generate grades and to improve their teaching and the students' learning (Walvoord, 2004). James Nichols (2002) notes that learning outcomes are "changes that result from instruction" and must be "focused on student learning and the improvement of teaching and learning" (p. 157). These learning outcomes may include those required to be successful in upper level courses, in obtaining employment, or in becoming a contributing member of society (J. Nichols, 2002). It is preferable to assess these learning outcomes in the classroom using authentic assessment tools. A n example of an indirect indicator of student learning at the course level is the use of student course evaluations that include questions about how well students think they have mastered each of the learning outcomes as well as the quality of the instruction (Walvoord, 2004). Authentic Assessment Techniques In a tribal college context, elements of Indigenous knowledge can best be passed on to students when traditional teaching and assessment methods, such as apprenticeships, observations, ceremonies, and practice are employed (Battiste & Henderson, 2000). Crazy B u l l 128 (2006b) emphasizes that "Native people are experts in experiential learning, learning by observation and practice, and in reflective learning. Our cultural teachings are grounded in this type of learning" (p. 4). The incorporation of traditional culture and knowledge throughout the curriculum makes the curriculum more relevant and accessible to students (Boyer, 1990; Fixico, 2000). Because students respond more favorably to experientially grounded and contextualized learning experiences, the use of authentic assessment tools over traditional tests is a natural and more appropriate approach to assessment in tribal colleges. Kirkness and Barnhardt (1991) emphasize the importance of measuring students' achievement through the students' ability to effectively perform meaningful and contextually appropriate tasks.. .that allow for the integration of various forms of knowledge and the application and display of that knowledge in a variety of ways. (p. 10) N W I C , as an example, has experimented with assessing learning outcomes in more authentic and creative ways. Its Tribal Environmental and Natural Resources Management Program "emphasized practical competency" and "an ability to synthesize and to solve complex real-world problems": Important measures of success include whether the students stay grounded in cultural values; maintain self-respect and a healthy sense of self within the context of community; are able to use critical and integrative abilities to problem solve and imagine creative solutions; and make substantial improvement in writing, reading comprehension, speaking and computational abilities. (Burns et al., 2002, pp. 57-58) B y making the curriculum, as well as the assessment of student learning, more relevant and authentic, students are able to make meaningful connections between their coursework and their life experiences. A s instructors in tribal colleges are being encouraged to increase the relevancy of the coursework to their students' lives and to use more authentic assessment methods in class, portfolios are becoming a more widely accepted and encouraged tool for documenting student 129 learning and improvement over time. They provide a more creative and real-life option than traditional tests and are an effective way of evaluating general knowledge and learning in tribal college programs. Student portfolios are a collection of the students' work over the course of their college experience and may include writing samples, self-assessments and reflections, videotapes, speeches, and so on. Colleges that have introduced portfolio systems have found that portfolios encourage better communication between faculty, students, and advisors (J. Nichols, 2002). Student portfolios also encourage the use of multiple assessment methods to evaluate competency of important learning outcomes (Maki , 2002; S U N Y G , 2004). Ideally, several faculty should evaluate student work at multiple points of their college career (Walvoord, 2004). Tribal colleges are already experimenting with promising assessment methods, such as portfolios, self-evaluation, and applied "real-life" situations, which can demonstrate student progress in the more elusive affective learning outcomes. Larger Campus Community Boyer's third area of accountability pertains to the larger campus community, including direct indicators (specifically college outcomes), indirect indicators, and institutional data, which I previously discussed in this doctoral thesis. Because tribal colleges' resources are substantially limited, it is highly recommended that they begin the assessment processes by using data that are currently available and taking an inventory of their assessment practices in a systematic way (i.e., an assessment inventory) ( C S H E , 2002; J. Nichols, 2002). Many tribal colleges are already doing some assessment work and have existing data — either direct or indirect indicators, or institutional data — that can be tapped. For instance, some have entrance or licensure exams, some test graduating students (with standard or localized tests or performance tests), and others conduct graduate or satisfaction surveys. In addition, most 130 colleges have an institutional data system (e.g., financial aid, registration, and payroll) from which they generate reports that can also be used for assessment purposes (e.g., transcript analysis). For example, all accredited tribal colleges are expected to generate Integrated Postsecondary Education Data System (IPEDs) reports, which include information about students, degree programs, faculty, and finances, and these reports may be adapted for use in the assessment program. Direct Indicators (College Outcomes) Some tribal colleges are beginning to articulate broad college outcomes, such as writing proficiency, through entrance tests and then capstone courses, videotapes of performances, portfolios, internships, or exit exams. M u c h work remains to be done in this regard as this work is grueling and time-consuming and gets at the core values of the college. Refer to the section titled College Outcomes in chapter 4 for a more in-depth discussion of college outcomes. Indirect Indicators After a tribal college inventories its current assessment procedures, James Nichols (2002) recommends conducting attitudinal surveys. There are many types of surveys that can be administered to current students, alumni, employers, faculty, staff, administrators, and the tribal community, depending on the needs of the college. Colleges can either use existing standardized surveys or, depending on the resources available, develop their own. Fortunately, many standardized surveys now allow tribal colleges to add a set of their own questions at the end of the survey, which provides for customizability. Conducting national attitudinal surveys is one way to be active in the assessment process, the advantages being the costs are minimal and the tests provide baseline information while the assessment program is being developed. Especially 131 in a tribal college context, alumni surveys can provide valuable information about further educational progress and performance, satisfaction with the college experience, and job acquisition and performance (J. Nichols, 2002). Finally, because a significant factor in student retention and success is satisfaction, these surveys can also provide useful information for assessment (Banta, 1999; J. Nichols, 2002; Ouimet, 2003). Institutional Data Most tribal colleges are now tracking patterns of student enrollment, retention, graduation, transfer, and, sometimes, employment. Although it would be extremely valuable for tribal colleges to collect, use, and interpret this information, several constraints exist (over and above lack of finances and resources). One of these is the challenge of creating standardized definitions for calculating retention, graduation, and course completion rates. Up until recently, it was virtually impossible to compare data among tribal colleges, or from year to year within tribal colleges, because of the lack of consistent definitions. Tribal colleges are starting to define these rates. In fact, in 2004, the American Indian Higher Education Consortium initiated a new tribal college project — called the American Indian Measures for Success: The American Indian Measures for Success ( A I M S ) project is a two year project with two goals: to define relevant quantitative and qualitative indicator data of American Indian student success, as determined by the tribal colleges, and to develop and implement a strategy for collecting, analyzing, and presenting annually the success indicator data using electronic information management tools. Through data collection and analysis, this project w i l l be the foundation for systemic reform that significantly increases — and for the first time, accurately measures — American Indian success in higher education. (Systemic Research, 2006, p. 1) This comprehensive and uniform set of tribal college data is now being collected annually from all tribal colleges and is being used by A I H E C to appeal for tribal college funding and support. 132 One constraint of collecting assessment information is that important data, which are needed to interpret assessment results, are frequently missing. For example, knowing a student's intentions (e.g., to obtain a two-year degree, to learn a traditional language or culture) upon enrollment is central to any assessment effort — especially in a tribal college setting — because it is difficult to monitor student success without it. For instance, in order to calculate a graduation rate, the college must only include in the calculation those students who enrolled with the intention of completing a two-year degree (J. Nichols, 2002). The college can use this information for many purposes. B y combining the students' intentions or reasons for attending college with other information such as G P A , gender, age, or number of children, important information can be generated about patterns of student flow. Moreover, tribal colleges can determine how successful students are at accomplishing their goals and explore whether students upgrade their original goals after attending college (J. Nichols, 2002). The information gleaned from multiple assessment activities at the larger campus community level should include direct and indirect indicators and institutional data and can be used extensively for many purposes, including initiating meaningful conversations about improvement among faculty, staff, students, and the community. Wider Tribal Community The next level of accountability Boyer refers to is to the wider tribal community — the "importance of connections, service, and community" (Ortiz & Boyer, 2003a, p. 46). Toward this end, tribal colleges are exploring what it means to be agents of change in the larger community, as well as how to measure the impact of the college on the community. Eventually, tribal colleges hope to strengthen local economies, improve standards of l iving, increase household income, revitalize Native language and culture, restore traditional values and skills, 133 rebuild cultural knowledge, and generally provide hope for tribal community members (Boyer, 2003). For example, Turtle Mountain Community College is making an initial attempt to measure its impact on the wider tribal community; all staff (including faculty) are asked to participate in extended community affairs, and information about their staffs involvement in these community activities is reported in their final assessment report (Yel low Bi rd , 1998-1999). Given their limited resources, it is extremely difficult for tribal colleges to assess their impact on the wider tribal community. Community impact surveys of local tribal members can enquire about the amount of contact they have with the college as well as their overall impressions and satisfaction with its programs and services (Alfred et al., 2000). In addition, tribal colleges could benefit from needs assessments to determine the priorities from the tribal members' perspectives. Furthermore, alumni surveys can provide much-needed information regarding the ultimate economic, personal, cultural, and other benefits for the students and communities served by tribal colleges. Finally, Crazy B u l l (2004) notes that the trend toward incorporating service learning at tribal colleges has provided a formal mechanism for student and community engagement. Service learning experiences and learning communities help colleges and universities address key elements of accreditation — student outcomes, assessment, and community engagement... Success of students — their retention and completion of their programs of study are closely linked to their engagement — so these opportunities also strengthen the most basic mission of higher education in the United States — to develop an educated, pluralistic, democratic population, (p. 1) President Crazy B u l l explains that despite over 30 years of control of Indian Education by Native people, we still struggle to create culturally-relevant opportunities. Tribal colleges that have been using service learning strategies find that those opportunities are very grounded in community and that they can make a difference in their communities, (p. 1) 134 A s expected, a great deal of work needs to be done in assessing the impact of tribal colleges on their wider tribal communities and in creating culturally relevant learning opportunities. Nonetheless, the trend toward formally incorporating service learning into tribal college curricula is making that job easier. Finally, in order to assess the impact of tribal colleges on local communities, tribal colleges may develop direct indicators, indirect indicators, and data at the community level to monitor their progress. Communities beyond the Reservation Boyer's final level of accountability pertains to communities beyond the reservation, including accreditation bodies, federal funding agencies, and charitable organizations who are supporting the tribal colleges. Although positive in many ways, these relationships sometimes result in tension when tribal colleges' values are questioned or challenged. For instance, Crazy B u l l (2006) states that for tribal colleges, accreditation means "participating in a process that requires us to integrate our understanding of the world as Native people and as Native educators with a different, often western/European based understanding of higher education" (p. 6). She outlines special accreditation issues for tribal colleges including the unique cultural environment, language experiences, social and familial relationships, tribal government's role, rural and often economically poor and isolated environments, older student population, increasing need to respond to younger students with different expectations, majority of students are women, unusually high number of non-college ready students, generally very limited financial and human resources compared to other educational institutions, often different delivery strategies in use, and pedagogy of teaching and learning based in tribal knowledge and worldview. (p. 7) 135 However, as long as tribal colleges are dependent upon, and accountable to, these agencies for accreditation, funding, or support, tribal colleges w i l l have to use assessment in creative ways to resolve this tension. Work in this area has barely begun. With the limited time and scarce resources at their disposal, tribal colleges struggle with balancing the needs of the accreditation bodies and funding agencies with the priorities and values of the college, as well as their capacity to generate the data. In addition to the overwhelming challenge this presents, Marker (2000) points out a potential challenge in the accreditation process: " A s First Nations continue to create programs that celebrate and promote language and identity, they must negotiate cultural outcomes with agencies and institutions that control funding and accreditation" (p. 30). To address this difficulty, tribal colleges might ultimately have to play a role in educating the accreditation bodies and funding agencies regarding culturally appropriate outcomes. If tribal colleges are successful, the accreditation bodies and funding agencies might be more receptive to the colleges' "culturally responsive structures and programs" (Marker, 2000, p. 30) and responsive to the creative, unique, and unconventional ways tribal colleges choose to assess their cultural outcomes. The challenges for colleges w i l l be to determine how to assess such outcomes in a culturally appropriate manner. On the other hand, this challenge presents an opportunity to break new ground, not only in the field of tribal college assessment, but also in the entire assessment field — which has traditionally ignored these kinds of outcomes. In the meantime, the World Indigenous Nations Higher Education Consortium ( W I N H E C ) formed in 2002 with the intention of creating an accreditation process for Indigenous higher education institutions worldwide. It is "committed to building partnerships that restore and retain Indigenous spirituality, cultures and languages, homelands, social systems, economic 136 systems and self determination" and "united in the collective synergy of self determination through control of higher education" (WTNHEC, 2003, p. 3). Although unique in its mission in that it recognizes Indigenous culture, traditions, worldviews, and history as valuable assets, like mainstream accreditation bodies, W L N H E C evaluates the quality of a college by how well it is able to accomplish its mission ( W I N H E C , 2003). W I N H E C membership includes individuals from institutions from Alaska, Aotearoa (New Zealand), Australia, Canada, Hawaii , Saamiland (Norway) and the continental United States. To date, W I N H E C has approved three undergraduate programs — from Te Wananga o Aotearoa, Te Whare Wananga o Awanuiarangi, and Te Wananga-o-Raukawa — all colleges in New Zealand. In 2006, it supported an accreditation request from the Seven Generations Education Institute of Ontario and Sami allaskuvla/Sami University (Norway). Other institutions intending to pursue W I N H E C accreditation include the First Nations Technical Institute (Tyendinaga Mohawk Territory, Canada); K e A l a Pono (an Indigenous college in Hawaii); Halau Wanana and K a Haka 'ula o Ke 'e l ikolani (at the University of Hawaii); and the University College of the North Manitoba (Canada) ( W I N H E C , 2006, p. 13). Conclusion Using a holistic approach, the first half of this chapter proposed a set of ethical considerations that need to permeate assessment processes at tribal colleges with the belief that the assessment process itself is as important as the ultimate product. Linda Smith (1999) points out that "negotiating and transforming institutional practices and research frameworks is as significant as the carrying out of actual research programs" (p. 140). This w i l l be evidenced when the work proceeds in a respectful, responsible, and reciprocal manner, where relationships are developed through a participatory and consultative process, and when the project can 137 maintain its relevance for the college and all those involved. The Northwest Commission on Colleges and Universities (2005) reaffirms this belief when it emphasizes that personnel from all segments of the institution, including faculty, staff, students, administration, and the governing board, must be included in the accreditation, self-study, and assessment processes. This participatory approach is important because processes of consultation and shared decision making are essential aspects of educational research in a tribal college context (L. Smith, 1999). The second half of this chapter discussed current assessment initiatives in tribal colleges in the United States and explored possibilities for adapting best practices in mainstream assessment to a tribal college context in culturally appropriate ways. I used Boyer's (2003) suggestion of focusing tribal college assessment efforts on six levels of accountability — the mission statement, cultural outcomes, larger campus community, learning outcomes, wider tribal community, and communities beyond the reservation — as a framework to explore these assessment initiatives. 138 C H A P T E R 5 M E T H O D O L O G Y In order to respond to the research questions — (1) What criteria are best used to evaluate an assessment program in a tribal college context? (2) Which elements of the N W I C assessment program are most and least successful according to the evaluative criteria established in Research Question 1? and (3) What preconditions and other contextual factors contribute to the relative success or failure of different elements of the N W I C assessment program? — I conducted a case study of the assessment program at N W I C . This approach honours the principles outlined by scholars of Indigenous research in chapter 4, uses several data collection methods, such as surveys, participant-observation, and document review, and is informed by the Context-Input-Process-Product (CIPP) Model developed by Daniel Stufflebeam (2003b). The case study approach and CIPP model are described below in more detail. C a s e S t u d i e s According to Robert Y i n (2003), a case study is an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident.. .The case study inquiry copes with the technically distinctive situation in which there w i l l be many more variables of interest than data points, and as one result, relies on multiple sources of evidence, with data needing to converge in a triangulating fashion, and as another result, benefits from the prior development of theoretical propositions to guide data collection and analysis, (p. 13-14) We can learn a lot from a single case that can apply to other cases and situations (Stake, 1995). A case study is appropriate under a number of circumstances: when it is a critical case used to test existing theory; when it is a unique case; when it is a representative or typical case; when it is a revelatory case; and when it is a longitudinal case (Yin , 2003). In addition, a case may be used as a pilot. 139 This case study — the assessment program at N W I C — fulfills several of these conditions. The study is representative, in that N W I C is a typical tribal college, and it is longitudinal, in that N W I C is easily accessible to me over a long period of time. It is also a revelatory case in that it can reveal crucial information about assessment in a tribal college context from which other tribal colleges can learn (by modifying generalizations about mainstream assessment to a tribal college context). Finally, it is a pilot, in that it is one of the first studies of assessment in a tribal college context. The literature outlines various limitations of case studies. In fact, some researchers call case studies weak, subjective, imprecise, and lacking rigor (Stufflebeam, 2001; Y i n , 2003). Y i n (2003), who has written a comprehensive text about case studies, counters these arguments by suggesting that these criticisms of the case study method are "misdirected" (p. 10). He believes the lack of rigor often characterizing these studies has been mainly due to careless research techniques because skills for conducting case studies have not been well defined up until now. Stake (1995), another case study researcher, sees the subjective and qualitative nature of case studies as a strength that is essential to understanding (as long as precautions are taken to minimize misunderstanding). Also , in the past, case studies commonly took a long time to complete, were expensive, had little impact on social practice, and resulted in long, poorly written reports (Stake, 1995; Y i n , 2003). A l l o f these concerns can be addressed i f the researcher takes the time to properly organize the case study. Others believe that case studies leave little grounds for generalization; however, Y i n (2003) claims that, although case studies cannot be used for "statistical generalization" (i.e., case studies typically do not collect empirical data that lend themselves to statistical analysis, so this form of generalization is not typically made from case studies), they can be used to "generalize theories" (also referred to as "analytic 140 generalization") (pp. 31-32). I w i l l discuss the generalizability of case study findings in more detail later in this chapter. Case studies are commonly used because they are valuable tools, especially in evaluation research, that offer great flexibility to examine a program in an in-depth, holistic way as it naturally occurs (Alk in & Christie, 2004; Feagin, Orum, & Sjoberg, 1991; Merriam, 1998; Schwandt, 1994; Stufflebeam, 2001; Y i n , 2003). A case study can be used in evaluation research, such as this study, where it is used to explain, describe, illustrate, and explore various elements of the N W I C assessment program and how these may apply to different situations (J. Gal l , Ga l l , & Borg, 2005; M . Gal l , Ga l l , & Borg, 2003; Y i n , 2003). Stufflebeam (2001) evaluated 22 different evaluation approaches using the program evaluation standards of utility, feasibility, propriety, and accuracy and found that the case study method was one of the most effective. These program evaluation standards (The Joint Committee on Standards for Educational Evaluation, 1994) — some of which have already been discussed in various sections of this doctoral thesis — are abided by during my research. The accuracy standard comprises reliability and validity issues, which are of particular concern and which are elaborated upon below. Validity and Reliability Issues in Case Study Design The quality of a case study design is essential, and it can be evaluated using several validity and reliability criteria. Whereas validity is "the extent to which a measure actually measures what it is intended to measure" (Rossi et al., 2004, p. 436), reliability is "the extent to which a measure produces the same results when used repeatedly to measure the same thing" (Rossi et al., 2004, p. 433) The following section overviews three different types of validity and one type of reliability criterion and suggests ways of improving the quality of a case study in 141 each of these respects. In this case study, I attempted to maximize validity and reliability by addressing all of these criteria. Construct Validi ty Construct validity is the process of "establishing correct operational measures for the concepts being studied" (Yin , 2003, p. 34). This is a challenge in case study research because of the subjective nature of data collection (Denzin & Lincoln, 1994; Patton, 1987); therefore, in order to minimize this concern, the researcher uses multiple sources of information and "establish[es] a chain of evidence" (Yin , 2003, p. 34) during the data collection phase. A "chain of evidence" is a clear association between the research questions, the information, and the conclusions (Yin , 2003). The rationale for using multiple sources of evidence.. .allows an investigator to address a broader range of historical, attitudinal, and behavioral issues. However, the most important advantage presented by using multiple sources of evidence is the development of converging lines of inquiry, a process of triangulation. (p. 98) Thus, triangulation can be used to "disconfirm major assertions" (Stake, 1995, p. 87). Also , in order to improve construct validity, during the writing phase, the researcher has drafts of the report reviewed by participants in the case study. This way, facts and interpretations can be corroborated and an opportunity is provided to make corrections (Yin , 2003). I used a combination of complementary data collection methods or sources of information — including surveys, participant-observation, and document review — during this case study to triangulate the data in an attempt to maximize construct validity (Stake, 1995). In this case study, I used at least one of these three methods to gather the information needed to answer each of the three research questions. I used the participant-observation method for each question. To illustrate: I used a review of the literature combined with my observations and 142 experiences in the assessment field at a tribal college as a basis for the response to Research Question 1. In addition to these two methods, the responses to research questions 2 and 3 are based on surveys of professional staff, faculty, and administrators — and, to a lesser extent, on the analysis of N W I C documents. Surveys, participant-observation, and document review w i l l be discussed further in the following section. Surveys Structured surveys can be used to target specific topics and can provide valuable insight. A t the same time, there is a danger that bias or inaccuracy can be introduced into the process due to a number of reasons: survey questions may be unclear or imprecise; respondents may remember inaccurately or provide dishonest responses; and the relationship between the researcher and the respondent may be poor. In this case, many staff, faculty, and administrators and the N W I C Institutional Review Board reviewed the survey questions and provided valuable input. In addition, I protected anonymity. Participant-observation One advantage of participant-observation as a means of collecting information is that the observations of real activities and behaviours take place in context. In addition, this method is unobtrusive and provides access and insight into people's struggles in challenging situations (Stake, 1995). The disadvantages of the participant-observation method include that it can be time-consuming and selective and the possibility that the presence of an observer may influence or even change the course of the events being observed. Furthermore, a bias may result due to the researcher's manipulating events (Yin , 2003, p. 86). Most of these disadvantages were minimized in this case study because I am an N W I C employee and my observations are a by-product of my work. 143 Document review Document review involves reviewing relevant documents to the research in order to make inferences and to verify basic information (Yin , 2003). Sometimes it provides information that is unanticipated (Stake, 1995). The advantages of document review are that documents are permanent, secure, unobtrusive, and provide a wide coverage over time and of events. The disadvantages include that they can be difficult to retrieve, may be incomplete and, therefore, biased, the author may be biased, or access might be denied. In this study, I reviewed and referred to N W I C ' s accreditation reports, strategic plan, and survey reports. Because each of these methods (i.e., surveys, participant-observation, and document review) has advantages and disadvantages, construct validity is maximized when several are used together. Furthermore, when both the researcher's perspective and the participants' perspectives are considered, the research tends to be better informed. Internal Validi ty Internal validity is the extent to which a condition can be determined to be caused by another condition (Stake, 1995). The researcher can address this concern during the data analysis stage by comparing results with expected outcomes for consistency (Stake, 1995), by building explanations, by addressing alternative explanations or interpretations, by being attentive to all the information, and by using logic models (such as tracing events over time in a flow chart) (Yin , 2003). In addition, triangulating data can minimize threats to internal validity. Internal validity is more of a concern for explanatory or causal case studies. 144 External Validi ty External validity is the extent to which a study's conclusions or findings can be generalized to another population or situation (J. Gal l et al., 2005; M . Ga l l et al., 2003). A s mentioned earlier, empirical data from case study research cannot be generalized to other cases; however, findings in a case study can be generalized to theory, and this theory can be used to examine other cases (Flyvbjerk, 2006; Y i n , 2003). Therefore, a theoretical proposition can orient the analysis (Yin , 2003, p. 111). Stake (1995) states that "valid modification of generalization can occur in case study" (p. 8) research; therefore, the researcher should "indicate how the findings might be extrapolated, how they could be interpreted in various circumstances, and how they accommodate theoretical discourse" (p. 93). Following this logic, I can identify criteria from mainstream assessment research and determine which of these criteria applies to assessment in a tribal college context. From these findings, theory can be generated that can be applied to other tribal colleges (Feagin et al., 1991). Marshall and Rossman (1999), Lincoln and Guba (1985), and Talbot (1995) discuss four ways to enhance the trustworthiness of a study: credibility/validity (through triangulation of sources, data, and methods); transferability/generalizability (through thick descriptions); dependability (through adjusting the steps of the research design); and confirmability (through verifying steps of the research, keeping a research log including thoughts along the way, and having participants in the study review the report). Furthermore, Lincoln and Guba underscore the importance of ensuring the researcher spends ample time in the community of interest. A s an insider to the college, I took all o f these steps in this study in order to improve its trustworthiness. 145 Reliability Reliability is the extent to which the procedures, such as data collection methods, are repeatable with the same results (J. Gall et al., 2005; M. Gall et a l , 2003; Yin, 2003). In order to increase the reliability of a case study, and minimize bias and error, during the data collection phase, the researcher should follow case study protocol and create a database or system for storing and organizing the information. This information may include notes (such as from observations), documents that have been collected, quantitative data (such as from surveys), and narratives (Yin, 2003). Throughout this research, I have kept thorough records of my observations and a database of quantitative and qualitative survey responses. In addition, all NWIC documents that are referred to are kept in my possession. Essential Components of Research Design Case study protocol recommends the proposal include an overview of the project (which it did), identification of the case (i.e., the case is NWIC), a list of research questions (with sources of information and method for obtaining information for each), identification of the audience (the audience for this research is the tribal college community, the NWIC community, and my doctoral thesis committee), an outline of field procedures, and an outline of the report. "The logic linking the data to the propositions" is then identified, as are the "criteria for interpreting the findings" (Yin, 2003, p. 21). Furthermore, it is beneficial in case studies to develop and test a theory or hypothesis from the beginning of the research (from the research and literature review). Researchers who conduct successful case studies generally start writing their reports early on in the process — I, for example, began thinking and writing about this topic four years ago and wrote much of the case study as time progressed. Researchers reflect constantly upon 146 their practices — because I have worked at N W I C for over five years, I have a good sense of what methodological approaches might be effective, and I have informally adjusted the design of the study based on these observations. For example, online surveys (accessible through personal emails) have proven to be an effective way of gathering information in a short period of time at N W I C , so that is why I decided to pursue this mode of survey delivery for my research. Also , initially, after conducting the survey, I was going to interview individuals, but staff responses to the survey were so comprehensive and in-depth and the response rate was so high, that it was unnecessary to conduct the interviews. Researchers who conduct successful case studies identify their case so that readers can integrate previous knowledge with what is being presented and references can be accurately reviewed ( N W I C is identified as this case study). They leave respondents in the report anonymous (I removed names from the data) and have drafts of the report reviewed by participants who were involved in the case study (the president, several administrators, faculty, and a member of N W I C ' s Institutional Review Board reviewed drafts of the report) (Stake, 1995; Y i n , 2003). Exemplary case studies are revelatory, in that they reveal something that has not been known in the past; are complete, in that an exhaustive amount and quality of information is produced; consider different perspectives; and engage the reader (Yin , 2003). The CIPP Model The CIPP Model is an approach in which educational initiatives are evaluated at various stages of development by examining context, inputs, processes, and products (Stufflebeam, 2001, 2003a, 2003c, 2004). L ike case studies, the CIPP model encourages a multi-method approach using both qualitative and quantitative methods, such as surveys, literature reviews, document reviews, participant-observation, interviews, and so forth. It is an established 147 educational evaluation framework and uses an inclusive, participatory research approach, which is suitable in this context (Nelson-Barber et a l , 2005). It is especially appropriate for use in a Native American context due to the value it places on process (L. Smith, 1999). The CIPP model has evolved since 1965 with the help of educators like Egon Guba. 1 3 A t that time, U . S . public schools were discontented about the evaluation approach — "controlled, variable manipulating, comparative experiments" (Stufflebeam, 2003a, p. 24) — available to evaluate federally funded projects for disadvantaged students. Stufflebeam tried to encourage schools and evaluators to design functional evaluations that would assist in decision making; thus, those initial evaluations focused on process (assessing program implementation) and product evaluations (assessing outcomes). Stufflebeam and his colleagues soon realized that the process and product evaluations lacked a focus on goals so they added a context evaluation component. Finally, in 1967, Stufflebeam added input evaluation to assist with program planning. The CIPP model has recently evolved so that the products stage is now subdivided into the evaluation of impact, effectiveness, sustainability, and generalizability. I chose an adaptation of this model because the research questions correspond with the four components of the CIPP evaluation framework. Stufflebeam (2003a) makes it clear that "the CIPP Model has to be applied flexibly" (p. 24) and that not all four areas have to be evaluated. In this case, I am evaluating the N W I C assessment program mid-stream. For the purposes of this doctoral research, the context and several of the inputs (i.e., strategic planning, assessment inventory, assessment plan, assessment coordinator, financial support, and technical support) are described, and the remaining inputs (i.e., mission statement, administrators, and faculty) and the process and product are evaluated. 1 3 Egon Guba was Daniel Stufflebeam's mentor at Ohio State University (Stufflebeam, 2003a). 148 The response to Research Question 1 addresses the inputs, processes, and products; the response to Research Question 2 addresses the inputs, processes, and the first three aspects of product (impact, effectiveness, and sustainability); and the response to Research Question 3 relates to context and product (generalizability). I present all of these responses in chapter 6 r (Results). In chapter 7, the conclusion of the doctoral thesis, I discuss the model's generalizability (i.e., the fourth aspect of product) and suggest aspects of assessment that tribal colleges may want to consider when developing their assessment programs. Table 5.1 provides a graphical summary of the data collection methods I used for each of the research questions and the CIPP areas that correspond to each. Table 5.1 List of Research Questions, Data Collection Methods, and CIPP Areas Research question Data collection method CIPP area (1) What cr
UBC Theses and Dissertations
Assessment in a tribal college context : a case study of Northwest Indian College Karlberg, Anne Marie 2007
Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.
- 831-ubc_2007-317969.pdf [ 19.04MB ]
- JSON: 831-1.0055897.json
- JSON-LD: 831-1.0055897-ld.json
- RDF/XML (Pretty): 831-1.0055897-rdf.xml
- RDF/JSON: 831-1.0055897-rdf.json
- Turtle: 831-1.0055897-turtle.txt
- N-Triples: 831-1.0055897-rdf-ntriples.txt
- Original Record: 831-1.0055897-source.json
- Full Text